2024-03-23T17:20:23.213 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-23T17:20:23.213 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/dbench}} --owner scheduled_yuriw@teuthology --name yuriw-2024-03-22_13:15:24-fs-wip-yuri11-testing-2024-03-21-0851-reef-distro-default-smithi --archive /home/teuthworker/archive/yuriw-2024-03-22_13:15:24-fs-wip-yuri11-testing-2024-03-21-0851-reef-distro-default-smithi/7617485 --verbose -- /home/teuthworker/archive/yuriw-2024-03-22_13:15:24-fs-wip-yuri11-testing-2024-03-21-0851-reef-distro-default-smithi/7617485/orig.config.yaml 2024-03-23T17:20:23.276 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-23T17:20:23.373 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2024-03-22_13:15:24-fs-wip-yuri11-testing-2024-03-21-0851-reef-distro-default-smithi/7617485 branch: wip-yuri11-testing-2024-03-21-0851-reef description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/dbench}} email: yweinste@redhat.com first_in_suite: false job_id: '7617485' kernel: client: branch: testing kdb: true sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: yuriw-2024-03-22_13:15:24-fs-wip-yuri11-testing-2024-03-21-0851-reef-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-yuri11-testing-2024-03-21-0851-reef ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - slow metadata IO - SLOW_OPS - slow request sha1: 82316b04821a6c2c1d458a881920454bbadb8837 subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: 82316b04821a6c2c1d458a881920454bbadb8837 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - nowsync syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri11-testing-2024-03-21-0851-reef sha1: 82316b04821a6c2c1d458a881920454bbadb8837 owner: scheduled_yuriw@teuthology priority: 99 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 1919 sha1: 82316b04821a6c2c1d458a881920454bbadb8837 sleep_before_teardown: 0 subset: 111/120000 suite: fs suite_branch: wip-yuri11-testing-2024-03-21-0851-reef suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_82316b04821a6c2c1d458a881920454bbadb8837/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 82316b04821a6c2c1d458a881920454bbadb8837 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi086.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJRQyru1W+ghUxJWxdqkZUO5kVJdVfWONkkLLciLrJ9z+5QoL7EDbRzS8VPdBSbWKQArG55IemPWMBMHzILdm4w= smithi092.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIEuY40VvuSMzhcnfJdlnURI+2xwRebarDR06M8+49B80T++wnnyoBDtnKIPpV4d10Njp3bOEwZA099mEhzidMU= smithi152.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHPklPFNoqQV2QJdy1E8+l4+OUuE8WnKB+jELm1CuTAauM1CiWsTsxcdIpYvOYaYJv8unQ5ChARjrAVR0O63nNs= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools:rhel8 --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-22_13:15:24 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-23T17:20:23.374 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_82316b04821a6c2c1d458a881920454bbadb8837/qa; will attempt to use it 2024-03-23T17:20:23.374 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_82316b04821a6c2c1d458a881920454bbadb8837/qa/tasks 2024-03-23T17:20:23.374 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-23T17:20:23.376 INFO:teuthology.task.internal:Checking packages... 2024-03-23T17:20:23.420 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '82316b04821a6c2c1d458a881920454bbadb8837' 2024-03-23T17:20:23.420 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-23T17:20:23.420 INFO:teuthology.packaging:ref: None 2024-03-23T17:20:23.420 INFO:teuthology.packaging:tag: None 2024-03-23T17:20:23.420 INFO:teuthology.packaging:branch: wip-yuri11-testing-2024-03-21-0851-reef 2024-03-23T17:20:23.421 INFO:teuthology.packaging:sha1: 82316b04821a6c2c1d458a881920454bbadb8837 2024-03-23T17:20:23.421 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-yuri11-testing-2024-03-21-0851-reef 2024-03-23T17:20:23.638 INFO:teuthology.task.internal:Found packages for ceph version 18.2.2-954.g82316b04 2024-03-23T17:20:23.639 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-23T17:20:23.646 INFO:teuthology.task.internal:no buildpackages task found 2024-03-23T17:20:23.646 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-23T17:20:23.652 INFO:teuthology.task.internal:Saving configuration 2024-03-23T17:20:23.675 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-23T17:20:23.683 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-23T17:20:23.711 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi086.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-03-22_13:15:24-fs-wip-yuri11-testing-2024-03-21-0851-reef-distro-default-smithi/7617485', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-23 17:15:05.012809', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJRQyru1W+ghUxJWxdqkZUO5kVJdVfWONkkLLciLrJ9z+5QoL7EDbRzS8VPdBSbWKQArG55IemPWMBMHzILdm4w='} 2024-03-23T17:20:23.736 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi092.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-03-22_13:15:24-fs-wip-yuri11-testing-2024-03-21-0851-reef-distro-default-smithi/7617485', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-23 17:15:05.011460', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIEuY40VvuSMzhcnfJdlnURI+2xwRebarDR06M8+49B80T++wnnyoBDtnKIPpV4d10Njp3bOEwZA099mEhzidMU='} 2024-03-23T17:20:23.760 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi152.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-03-22_13:15:24-fs-wip-yuri11-testing-2024-03-21-0851-reef-distro-default-smithi/7617485', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-23 17:15:05.014011', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHPklPFNoqQV2QJdy1E8+l4+OUuE8WnKB+jELm1CuTAauM1CiWsTsxcdIpYvOYaYJv8unQ5ChARjrAVR0O63nNs='} 2024-03-23T17:20:23.760 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-23T17:20:23.767 INFO:teuthology.task.internal:roles: ubuntu@smithi086.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-03-23T17:20:23.767 INFO:teuthology.task.internal:roles: ubuntu@smithi092.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-03-23T17:20:23.767 INFO:teuthology.task.internal:roles: ubuntu@smithi152.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-03-23T17:20:23.767 INFO:teuthology.run_tasks:Running task console_log... 2024-03-23T17:20:23.871 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7efe1d4b0160>, signals=[15]) 2024-03-23T17:20:23.872 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-23T17:20:23.878 INFO:teuthology.task.internal:Opening connections... 2024-03-23T17:20:23.878 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi086.front.sepia.ceph.com 2024-03-23T17:20:23.880 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T17:20:23.954 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi092.front.sepia.ceph.com 2024-03-23T17:20:23.955 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T17:20:24.028 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi152.front.sepia.ceph.com 2024-03-23T17:20:24.028 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi152.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T17:20:24.106 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-23T17:20:24.113 DEBUG:teuthology.orchestra.run.smithi086:> uname -m 2024-03-23T17:20:24.134 INFO:teuthology.orchestra.run.smithi086.stdout:x86_64 2024-03-23T17:20:24.134 DEBUG:teuthology.orchestra.run.smithi086:> cat /etc/os-release 2024-03-23T17:20:24.193 INFO:teuthology.orchestra.run.smithi086.stdout:NAME="Red Hat Enterprise Linux" 2024-03-23T17:20:24.194 INFO:teuthology.orchestra.run.smithi086.stdout:VERSION="8.6 (Ootpa)" 2024-03-23T17:20:24.194 INFO:teuthology.orchestra.run.smithi086.stdout:ID="rhel" 2024-03-23T17:20:24.195 INFO:teuthology.orchestra.run.smithi086.stdout:ID_LIKE="fedora" 2024-03-23T17:20:24.195 INFO:teuthology.orchestra.run.smithi086.stdout:VERSION_ID="8.6" 2024-03-23T17:20:24.195 INFO:teuthology.orchestra.run.smithi086.stdout:PLATFORM_ID="platform:el8" 2024-03-23T17:20:24.195 INFO:teuthology.orchestra.run.smithi086.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-03-23T17:20:24.195 INFO:teuthology.orchestra.run.smithi086.stdout:ANSI_COLOR="0;31" 2024-03-23T17:20:24.195 INFO:teuthology.orchestra.run.smithi086.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-03-23T17:20:24.195 INFO:teuthology.orchestra.run.smithi086.stdout:HOME_URL="https://www.redhat.com/" 2024-03-23T17:20:24.195 INFO:teuthology.orchestra.run.smithi086.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-03-23T17:20:24.195 INFO:teuthology.orchestra.run.smithi086.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-23T17:20:24.196 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:20:24.196 INFO:teuthology.orchestra.run.smithi086.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-03-23T17:20:24.196 INFO:teuthology.orchestra.run.smithi086.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-03-23T17:20:24.196 INFO:teuthology.orchestra.run.smithi086.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-03-23T17:20:24.196 INFO:teuthology.orchestra.run.smithi086.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-03-23T17:20:24.196 INFO:teuthology.lock.ops:Updating smithi086.front.sepia.ceph.com on lock server 2024-03-23T17:20:24.221 DEBUG:teuthology.orchestra.run.smithi092:> uname -m 2024-03-23T17:20:24.242 INFO:teuthology.orchestra.run.smithi092.stdout:x86_64 2024-03-23T17:20:24.242 DEBUG:teuthology.orchestra.run.smithi092:> cat /etc/os-release 2024-03-23T17:20:24.301 INFO:teuthology.orchestra.run.smithi092.stdout:NAME="Red Hat Enterprise Linux" 2024-03-23T17:20:24.301 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION="8.6 (Ootpa)" 2024-03-23T17:20:24.301 INFO:teuthology.orchestra.run.smithi092.stdout:ID="rhel" 2024-03-23T17:20:24.301 INFO:teuthology.orchestra.run.smithi092.stdout:ID_LIKE="fedora" 2024-03-23T17:20:24.301 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION_ID="8.6" 2024-03-23T17:20:24.302 INFO:teuthology.orchestra.run.smithi092.stdout:PLATFORM_ID="platform:el8" 2024-03-23T17:20:24.302 INFO:teuthology.orchestra.run.smithi092.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-03-23T17:20:24.302 INFO:teuthology.orchestra.run.smithi092.stdout:ANSI_COLOR="0;31" 2024-03-23T17:20:24.302 INFO:teuthology.orchestra.run.smithi092.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-03-23T17:20:24.302 INFO:teuthology.orchestra.run.smithi092.stdout:HOME_URL="https://www.redhat.com/" 2024-03-23T17:20:24.302 INFO:teuthology.orchestra.run.smithi092.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-03-23T17:20:24.302 INFO:teuthology.orchestra.run.smithi092.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-23T17:20:24.302 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-23T17:20:24.302 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-03-23T17:20:24.302 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-03-23T17:20:24.302 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-03-23T17:20:24.302 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-03-23T17:20:24.303 INFO:teuthology.lock.ops:Updating smithi092.front.sepia.ceph.com on lock server 2024-03-23T17:20:24.321 DEBUG:teuthology.orchestra.run.smithi152:> uname -m 2024-03-23T17:20:24.343 INFO:teuthology.orchestra.run.smithi152.stdout:x86_64 2024-03-23T17:20:24.343 DEBUG:teuthology.orchestra.run.smithi152:> cat /etc/os-release 2024-03-23T17:20:24.407 INFO:teuthology.orchestra.run.smithi152.stdout:NAME="Red Hat Enterprise Linux" 2024-03-23T17:20:24.407 INFO:teuthology.orchestra.run.smithi152.stdout:VERSION="8.6 (Ootpa)" 2024-03-23T17:20:24.407 INFO:teuthology.orchestra.run.smithi152.stdout:ID="rhel" 2024-03-23T17:20:24.407 INFO:teuthology.orchestra.run.smithi152.stdout:ID_LIKE="fedora" 2024-03-23T17:20:24.407 INFO:teuthology.orchestra.run.smithi152.stdout:VERSION_ID="8.6" 2024-03-23T17:20:24.407 INFO:teuthology.orchestra.run.smithi152.stdout:PLATFORM_ID="platform:el8" 2024-03-23T17:20:24.407 INFO:teuthology.orchestra.run.smithi152.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-03-23T17:20:24.407 INFO:teuthology.orchestra.run.smithi152.stdout:ANSI_COLOR="0;31" 2024-03-23T17:20:24.407 INFO:teuthology.orchestra.run.smithi152.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-03-23T17:20:24.408 INFO:teuthology.orchestra.run.smithi152.stdout:HOME_URL="https://www.redhat.com/" 2024-03-23T17:20:24.408 INFO:teuthology.orchestra.run.smithi152.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-03-23T17:20:24.408 INFO:teuthology.orchestra.run.smithi152.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-23T17:20:24.408 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-23T17:20:24.408 INFO:teuthology.orchestra.run.smithi152.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-03-23T17:20:24.408 INFO:teuthology.orchestra.run.smithi152.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-03-23T17:20:24.408 INFO:teuthology.orchestra.run.smithi152.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-03-23T17:20:24.408 INFO:teuthology.orchestra.run.smithi152.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-03-23T17:20:24.408 INFO:teuthology.lock.ops:Updating smithi152.front.sepia.ceph.com on lock server 2024-03-23T17:20:24.429 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-23T17:20:24.441 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-23T17:20:24.447 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-23T17:20:24.448 DEBUG:teuthology.orchestra.run.smithi086:> test '!' -e /home/ubuntu/cephtest 2024-03-23T17:20:24.450 DEBUG:teuthology.orchestra.run.smithi092:> test '!' -e /home/ubuntu/cephtest 2024-03-23T17:20:24.453 DEBUG:teuthology.orchestra.run.smithi152:> test '!' -e /home/ubuntu/cephtest 2024-03-23T17:20:24.471 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-23T17:20:24.478 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-23T17:20:24.478 DEBUG:teuthology.orchestra.run.smithi086:> test -z $(ls -A /var/lib/ceph) 2024-03-23T17:20:24.511 DEBUG:teuthology.orchestra.run.smithi092:> test -z $(ls -A /var/lib/ceph) 2024-03-23T17:20:24.513 DEBUG:teuthology.orchestra.run.smithi152:> test -z $(ls -A /var/lib/ceph) 2024-03-23T17:20:24.539 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-23T17:20:24.619 INFO:teuthology.run_tasks:Running task kernel... 2024-03-23T17:20:24.635 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2024-03-23T17:20:24.636 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-03-23T17:20:24.636 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-03-23T17:20:24.636 DEBUG:teuthology.orchestra.run.smithi086:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-23T17:20:24.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T17:20:24.656 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&ref=testing 2024-03-23T17:20:24.993 DEBUG:teuthology.task.kernel:sha1 for client.0 is 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 2024-03-23T17:20:24.993 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5ec3e6e88b2e2783e09498588bb1f433ae9c8986"... 2024-03-23T17:20:24.993 DEBUG:teuthology.orchestra.run.smithi086:> uname -r 2024-03-23T17:20:25.013 INFO:teuthology.orchestra.run.smithi086.stdout:4.18.0-513.9.1.el8_9.x86_64 2024-03-23T17:20:25.014 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-513.9.1.el8_9.x86_64 vs 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 2024-03-23T17:20:25.014 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-03-23T17:20:25.014 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum upgrade -y linux-firmware 2024-03-23T17:20:26.543 INFO:teuthology.orchestra.run.smithi086.stdout:Updating Subscription Management repositories. 2024-03-23T17:20:26.544 INFO:teuthology.orchestra.run.smithi086.stdout:Unable to read consumer identity 2024-03-23T17:20:27.028 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:20:27.028 INFO:teuthology.orchestra.run.smithi086.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-03-23T17:20:27.028 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:20:28.774 INFO:teuthology.orchestra.run.smithi086.stdout:Copr repo for python3-asyncssh owned by ceph 19 kB/s | 3.5 kB 00:00 2024-03-23T17:20:31.166 INFO:teuthology.orchestra.run.smithi086.stdout:Extra Packages for Enterprise Linux 6.8 MB/s | 16 MB 00:02 2024-03-23T17:20:34.808 INFO:teuthology.orchestra.run.smithi086.stdout:lab-extras 474 kB/s | 24 kB 00:00 2024-03-23T17:20:35.636 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-03-23T17:20:35.637 INFO:teuthology.orchestra.run.smithi086.stdout:Nothing to do. 2024-03-23T17:20:35.638 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-03-23T17:20:35.760 INFO:teuthology.task.kernel:Downloading kernel 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 on client.0... 2024-03-23T17:20:35.760 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=5ec3e6e88b2e2783e09498588bb1f433ae9c8986 2024-03-23T17:20:35.910 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/5ec3e6e88b2e2783e09498588bb1f433ae9c8986/centos/8/flavors/default/x86_64/ 2024-03-23T17:20:35.910 DEBUG:teuthology.orchestra.run.smithi086:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_g5ec3e6e88b2e-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/5ec3e6e88b2e2783e09498588bb1f433ae9c8986/centos/8/flavors/default/x86_64/ --input-file=- 2024-03-23T17:20:35.912 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi086.front.sepia.ceph.com to complete... 2024-03-23T17:20:50.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23 17:20:50 URL:https://4.chacra.ceph.com/r/kernel/testing/5ec3e6e88b2e2783e09498588bb1f433ae9c8986/centos/8/flavors/default/x86_64/kernel-6.8.0_g5ec3e6e88b2e-1.x86_64.rpm [401601496/401601496] -> "/tmp/kernel.x86_64.rpm" [1] 2024-03-23T17:20:50.904 INFO:teuthology.orchestra.run.smithi086.stderr:FINISHED --2024-03-23 17:20:50-- 2024-03-23T17:20:50.904 INFO:teuthology.orchestra.run.smithi086.stderr:Total wall clock time: 15s 2024-03-23T17:20:50.904 INFO:teuthology.orchestra.run.smithi086.stderr:Downloaded: 1 files, 383M in 15s (26.4 MB/s) 2024-03-23T17:20:50.907 INFO:teuthology.task.kernel:Installing kernel 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 on client.0... 2024-03-23T17:20:50.907 DEBUG:teuthology.orchestra.run.smithi086:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-03-23T17:20:52.749 INFO:teuthology.orchestra.run.smithi086.stdout:Verifying... ######################################## 2024-03-23T17:20:52.845 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing... ######################################## 2024-03-23T17:20:53.004 INFO:teuthology.orchestra.run.smithi086.stdout:Updating / installing... 2024-03-23T17:21:33.342 INFO:teuthology.orchestra.run.smithi086.stdout:kernel-6.8.0_g5ec3e6e88b2e-1 ######################################## 2024-03-23T17:21:34.900 INFO:teuthology.orchestra.run.smithi086.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-23T17:21:54.938 INFO:teuthology.orchestra.run.smithi086.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-23T17:24:07.085 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi086.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-03-23T17:24:07.086 DEBUG:teuthology.orchestra.run.smithi086:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-03-23T17:24:07.147 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e 2024-03-23T17:24:07.147 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/System.map 2024-03-23T17:24:07.147 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/config 2024-03-23T17:24:07.147 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel 2024-03-23T17:24:07.147 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch 2024-03-23T17:24:07.147 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86 2024-03-23T17:24:07.147 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto 2024-03-23T17:24:07.148 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-03-23T17:24:07.148 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-03-23T17:24:07.148 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-03-23T17:24:07.148 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-03-23T17:24:07.148 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-03-23T17:24:07.148 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-03-23T17:24:07.148 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-03-23T17:24:07.148 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/crc32c-intel.ko 2024-03-23T17:24:07.148 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-03-23T17:24:07.148 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-03-23T17:24:07.148 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-03-23T17:24:07.148 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-03-23T17:24:07.148 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/serpent-avx2.ko 2024-03-23T17:24:07.149 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-03-23T17:24:07.149 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-03-23T17:24:07.149 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-03-23T17:24:07.149 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-03-23T17:24:07.149 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-03-23T17:24:07.149 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-03-23T17:24:07.149 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-03-23T17:24:07.149 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel 2024-03-23T17:24:07.149 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel/cpu 2024-03-23T17:24:07.149 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel/cpu/mce 2024-03-23T17:24:07.149 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-03-23T17:24:07.149 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm 2024-03-23T17:24:07.150 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm/kvm-amd.ko 2024-03-23T17:24:07.150 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm/kvm-intel.ko 2024-03-23T17:24:07.150 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm/kvm.ko 2024-03-23T17:24:07.150 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto 2024-03-23T17:24:07.150 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/ansi_cprng.ko 2024-03-23T17:24:07.150 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/anubis.ko 2024-03-23T17:24:07.150 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/arc4.ko 2024-03-23T17:24:07.150 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx 2024-03-23T17:24:07.150 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_memcpy.ko 2024-03-23T17:24:07.150 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_pq.ko 2024-03-23T17:24:07.150 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_raid6_recov.ko 2024-03-23T17:24:07.150 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_tx.ko 2024-03-23T17:24:07.151 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_xor.ko 2024-03-23T17:24:07.151 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/raid6test.ko 2024-03-23T17:24:07.151 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/authenc.ko 2024-03-23T17:24:07.151 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/authencesn.ko 2024-03-23T17:24:07.151 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/blake2b_generic.ko 2024-03-23T17:24:07.151 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/blowfish_common.ko 2024-03-23T17:24:07.151 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/blowfish_generic.ko 2024-03-23T17:24:07.151 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/camellia_generic.ko 2024-03-23T17:24:07.151 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cast5_generic.ko 2024-03-23T17:24:07.151 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cast6_generic.ko 2024-03-23T17:24:07.151 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cast_common.ko 2024-03-23T17:24:07.152 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/ccm.ko 2024-03-23T17:24:07.152 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cmac.ko 2024-03-23T17:24:07.152 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/crc32_generic.ko 2024-03-23T17:24:07.152 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/crypto_engine.ko 2024-03-23T17:24:07.152 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/crypto_user.ko 2024-03-23T17:24:07.152 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cts.ko 2024-03-23T17:24:07.152 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/des_generic.ko 2024-03-23T17:24:07.152 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/dh_generic.ko 2024-03-23T17:24:07.152 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/echainiv.ko 2024-03-23T17:24:07.152 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/essiv.ko 2024-03-23T17:24:07.152 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/fcrypt.ko 2024-03-23T17:24:07.152 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/khazad.ko 2024-03-23T17:24:07.153 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/lz4.ko 2024-03-23T17:24:07.153 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/lz4hc.ko 2024-03-23T17:24:07.153 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/md4.ko 2024-03-23T17:24:07.153 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/michael_mic.ko 2024-03-23T17:24:07.153 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/pcbc.ko 2024-03-23T17:24:07.153 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/pcrypt.ko 2024-03-23T17:24:07.153 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/rmd160.ko 2024-03-23T17:24:07.153 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/seed.ko 2024-03-23T17:24:07.153 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/serpent_generic.ko 2024-03-23T17:24:07.153 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/tcrypt.ko 2024-03-23T17:24:07.153 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/tea.ko 2024-03-23T17:24:07.153 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/twofish_common.ko 2024-03-23T17:24:07.153 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/twofish_generic.ko 2024-03-23T17:24:07.154 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/vmac.ko 2024-03-23T17:24:07.154 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/wp512.ko 2024-03-23T17:24:07.154 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/xcbc.ko 2024-03-23T17:24:07.154 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/xor.ko 2024-03-23T17:24:07.154 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/xxhash_generic.ko 2024-03-23T17:24:07.154 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers 2024-03-23T17:24:07.154 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi 2024-03-23T17:24:07.154 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/acpi_ipmi.ko 2024-03-23T17:24:07.154 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/acpi_pad.ko 2024-03-23T17:24:07.154 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/custom_method.ko 2024-03-23T17:24:07.154 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/ec_sys.ko 2024-03-23T17:24:07.154 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/platform_profile.ko 2024-03-23T17:24:07.154 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/sbs.ko 2024-03-23T17:24:07.155 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/sbshc.ko 2024-03-23T17:24:07.155 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/video.ko 2024-03-23T17:24:07.155 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata 2024-03-23T17:24:07.155 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/acard-ahci.ko 2024-03-23T17:24:07.155 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/ahci_platform.ko 2024-03-23T17:24:07.155 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/ata_generic.ko 2024-03-23T17:24:07.155 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/libahci_platform.ko 2024-03-23T17:24:07.155 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_acpi.ko 2024-03-23T17:24:07.155 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ali.ko 2024-03-23T17:24:07.155 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_amd.ko 2024-03-23T17:24:07.155 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_artop.ko 2024-03-23T17:24:07.155 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_atiixp.ko 2024-03-23T17:24:07.155 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_atp867x.ko 2024-03-23T17:24:07.156 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_cmd640.ko 2024-03-23T17:24:07.156 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_cmd64x.ko 2024-03-23T17:24:07.156 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_cypress.ko 2024-03-23T17:24:07.156 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_efar.ko 2024-03-23T17:24:07.156 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt366.ko 2024-03-23T17:24:07.156 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt37x.ko 2024-03-23T17:24:07.156 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt3x2n.ko 2024-03-23T17:24:07.156 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt3x3.ko 2024-03-23T17:24:07.156 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_it8213.ko 2024-03-23T17:24:07.156 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_it821x.ko 2024-03-23T17:24:07.156 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_jmicron.ko 2024-03-23T17:24:07.156 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_marvell.ko 2024-03-23T17:24:07.156 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_mpiix.ko 2024-03-23T17:24:07.156 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_netcell.ko 2024-03-23T17:24:07.156 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ninja32.ko 2024-03-23T17:24:07.157 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ns87410.ko 2024-03-23T17:24:07.157 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ns87415.ko 2024-03-23T17:24:07.157 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_oldpiix.ko 2024-03-23T17:24:07.157 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_opti.ko 2024-03-23T17:24:07.157 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_optidma.ko 2024-03-23T17:24:07.157 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_pcmcia.ko 2024-03-23T17:24:07.157 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_pdc2027x.ko 2024-03-23T17:24:07.157 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-03-23T17:24:07.157 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_piccolo.ko 2024-03-23T17:24:07.157 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_rdc.ko 2024-03-23T17:24:07.157 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sch.ko 2024-03-23T17:24:07.157 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_serverworks.ko 2024-03-23T17:24:07.157 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sil680.ko 2024-03-23T17:24:07.157 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sis.ko 2024-03-23T17:24:07.158 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sl82c105.ko 2024-03-23T17:24:07.158 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_triflex.ko 2024-03-23T17:24:07.158 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_via.ko 2024-03-23T17:24:07.158 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pdc_adma.ko 2024-03-23T17:24:07.158 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_inic162x.ko 2024-03-23T17:24:07.158 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_mv.ko 2024-03-23T17:24:07.158 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_nv.ko 2024-03-23T17:24:07.158 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_promise.ko 2024-03-23T17:24:07.158 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_qstor.ko 2024-03-23T17:24:07.158 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sil.ko 2024-03-23T17:24:07.158 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sil24.ko 2024-03-23T17:24:07.158 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sis.ko 2024-03-23T17:24:07.158 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_svw.ko 2024-03-23T17:24:07.158 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sx4.ko 2024-03-23T17:24:07.159 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_uli.ko 2024-03-23T17:24:07.159 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_via.ko 2024-03-23T17:24:07.159 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_vsc.ko 2024-03-23T17:24:07.159 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/auxdisplay 2024-03-23T17:24:07.159 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/auxdisplay/ks0108.ko 2024-03-23T17:24:07.159 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/base 2024-03-23T17:24:07.159 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/base/regmap 2024-03-23T17:24:07.159 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/base/regmap/regmap-i2c.ko 2024-03-23T17:24:07.159 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/base/regmap/regmap-mmio.ko 2024-03-23T17:24:07.159 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/bcma 2024-03-23T17:24:07.159 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/bcma/bcma.ko 2024-03-23T17:24:07.159 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block 2024-03-23T17:24:07.159 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/aoe 2024-03-23T17:24:07.160 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/aoe/aoe.ko 2024-03-23T17:24:07.160 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/brd.ko 2024-03-23T17:24:07.160 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/drbd 2024-03-23T17:24:07.160 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/drbd/drbd.ko 2024-03-23T17:24:07.160 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/floppy.ko 2024-03-23T17:24:07.160 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/loop.ko 2024-03-23T17:24:07.160 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/mtip32xx 2024-03-23T17:24:07.160 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-03-23T17:24:07.160 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/nbd.ko 2024-03-23T17:24:07.160 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/null_blk 2024-03-23T17:24:07.160 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/null_blk/null_blk.ko 2024-03-23T17:24:07.160 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/pktcdvd.ko 2024-03-23T17:24:07.160 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/rbd.ko 2024-03-23T17:24:07.160 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/virtio_blk.ko 2024-03-23T17:24:07.161 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/xen-blkback 2024-03-23T17:24:07.161 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-03-23T17:24:07.161 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/xen-blkfront.ko 2024-03-23T17:24:07.161 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/zram 2024-03-23T17:24:07.161 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/zram/zram.ko 2024-03-23T17:24:07.161 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char 2024-03-23T17:24:07.161 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hangcheck-timer.ko 2024-03-23T17:24:07.161 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random 2024-03-23T17:24:07.161 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/amd-rng.ko 2024-03-23T17:24:07.161 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/intel-rng.ko 2024-03-23T17:24:07.161 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-03-23T17:24:07.161 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/via-rng.ko 2024-03-23T17:24:07.161 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/virtio-rng.ko 2024-03-23T17:24:07.161 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi 2024-03-23T17:24:07.162 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-03-23T17:24:07.162 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-03-23T17:24:07.162 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-03-23T17:24:07.162 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_si.ko 2024-03-23T17:24:07.162 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-03-23T17:24:07.162 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-03-23T17:24:07.162 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/lp.ko 2024-03-23T17:24:07.162 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/mwave 2024-03-23T17:24:07.162 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/mwave/mwave.ko 2024-03-23T17:24:07.162 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ppdev.ko 2024-03-23T17:24:07.162 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tlclk.ko 2024-03-23T17:24:07.162 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm 2024-03-23T17:24:07.162 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm.ko 2024-03-23T17:24:07.162 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_atmel.ko 2024-03-23T17:24:07.163 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_infineon.ko 2024-03-23T17:24:07.163 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_nsc.ko 2024-03-23T17:24:07.163 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_tis.ko 2024-03-23T17:24:07.163 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-03-23T17:24:07.163 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/uv_mmtimer.ko 2024-03-23T17:24:07.163 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/virtio_console.ko 2024-03-23T17:24:07.163 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq 2024-03-23T17:24:07.163 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-03-23T17:24:07.163 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-03-23T17:24:07.163 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/p4-clockmod.ko 2024-03-23T17:24:07.163 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-03-23T17:24:07.163 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/powernow-k8.ko 2024-03-23T17:24:07.163 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/speedstep-lib.ko 2024-03-23T17:24:07.163 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto 2024-03-23T17:24:07.164 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/ccp 2024-03-23T17:24:07.164 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-03-23T17:24:07.164 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/ccp/ccp.ko 2024-03-23T17:24:07.164 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel 2024-03-23T17:24:07.164 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat 2024-03-23T17:24:07.164 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_common 2024-03-23T17:24:07.164 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-03-23T17:24:07.164 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-03-23T17:24:07.164 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-03-23T17:24:07.164 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/padlock-aes.ko 2024-03-23T17:24:07.164 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/padlock-sha.ko 2024-03-23T17:24:07.164 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/virtio 2024-03-23T17:24:07.164 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-03-23T17:24:07.164 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dca 2024-03-23T17:24:07.164 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dca/dca.ko 2024-03-23T17:24:07.165 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/devfreq 2024-03-23T17:24:07.165 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-03-23T17:24:07.165 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma 2024-03-23T17:24:07.165 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/dw 2024-03-23T17:24:07.165 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/dw/dw_dmac.ko 2024-03-23T17:24:07.165 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/ioat 2024-03-23T17:24:07.165 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/ioat/ioatdma.ko 2024-03-23T17:24:07.165 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac 2024-03-23T17:24:07.165 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/amd64_edac.ko 2024-03-23T17:24:07.165 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/e752x_edac.ko 2024-03-23T17:24:07.165 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/edac_mce_amd.ko 2024-03-23T17:24:07.165 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i3000_edac.ko 2024-03-23T17:24:07.165 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i3200_edac.ko 2024-03-23T17:24:07.166 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i5100_edac.ko 2024-03-23T17:24:07.166 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i5400_edac.ko 2024-03-23T17:24:07.166 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i7300_edac.ko 2024-03-23T17:24:07.166 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i7core_edac.ko 2024-03-23T17:24:07.166 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i82975x_edac.ko 2024-03-23T17:24:07.166 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/ie31200_edac.ko 2024-03-23T17:24:07.166 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/sb_edac.ko 2024-03-23T17:24:07.166 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/x38_edac.ko 2024-03-23T17:24:07.166 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/firmware 2024-03-23T17:24:07.166 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/firmware/edd.ko 2024-03-23T17:24:07.166 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/firmware/iscsi_ibft.ko 2024-03-23T17:24:07.166 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio 2024-03-23T17:24:07.166 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-ich.ko 2024-03-23T17:24:07.167 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-viperboard.ko 2024-03-23T17:24:07.167 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid 2024-03-23T17:24:07.167 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-appleir.ko 2024-03-23T17:24:07.167 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-aureal.ko 2024-03-23T17:24:07.167 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-axff.ko 2024-03-23T17:24:07.167 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-dr.ko 2024-03-23T17:24:07.167 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-elecom.ko 2024-03-23T17:24:07.167 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-elo.ko 2024-03-23T17:24:07.167 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-emsff.ko 2024-03-23T17:24:07.167 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-gaff.ko 2024-03-23T17:24:07.167 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-gt683r.ko 2024-03-23T17:24:07.167 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-gyration.ko 2024-03-23T17:24:07.167 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-holtek-kbd.ko 2024-03-23T17:24:07.167 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-holtek-mouse.ko 2024-03-23T17:24:07.168 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-holtekff.ko 2024-03-23T17:24:07.168 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-hyperv.ko 2024-03-23T17:24:07.168 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-icade.ko 2024-03-23T17:24:07.168 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-keytouch.ko 2024-03-23T17:24:07.168 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-kye.ko 2024-03-23T17:24:07.168 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-lcpower.ko 2024-03-23T17:24:07.168 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-led.ko 2024-03-23T17:24:07.168 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-lenovo.ko 2024-03-23T17:24:07.168 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-logitech-dj.ko 2024-03-23T17:24:07.168 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-03-23T17:24:07.168 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-multitouch.ko 2024-03-23T17:24:07.168 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-ortek.ko 2024-03-23T17:24:07.168 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-penmount.ko 2024-03-23T17:24:07.168 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-petalynx.ko 2024-03-23T17:24:07.169 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-picolcd.ko 2024-03-23T17:24:07.169 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-pl.ko 2024-03-23T17:24:07.169 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-plantronics.ko 2024-03-23T17:24:07.169 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-primax.ko 2024-03-23T17:24:07.169 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-rmi.ko 2024-03-23T17:24:07.169 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-arvo.ko 2024-03-23T17:24:07.169 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-common.ko 2024-03-23T17:24:07.169 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-isku.ko 2024-03-23T17:24:07.169 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-kone.ko 2024-03-23T17:24:07.169 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-03-23T17:24:07.169 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-konepure.ko 2024-03-23T17:24:07.169 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-03-23T17:24:07.169 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-lua.ko 2024-03-23T17:24:07.169 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-pyra.ko 2024-03-23T17:24:07.169 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-ryos.ko 2024-03-23T17:24:07.170 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-savu.ko 2024-03-23T17:24:07.170 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat.ko 2024-03-23T17:24:07.170 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-saitek.ko 2024-03-23T17:24:07.170 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-samsung.ko 2024-03-23T17:24:07.170 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sensor-hub.ko 2024-03-23T17:24:07.170 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sjoy.ko 2024-03-23T17:24:07.170 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sony.ko 2024-03-23T17:24:07.170 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-speedlink.ko 2024-03-23T17:24:07.170 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-steelseries.ko 2024-03-23T17:24:07.170 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sunplus.ko 2024-03-23T17:24:07.170 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-thrustmaster.ko 2024-03-23T17:24:07.170 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-tivo.ko 2024-03-23T17:24:07.170 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-tmff.ko 2024-03-23T17:24:07.170 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-topseed.ko 2024-03-23T17:24:07.171 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-twinhan.ko 2024-03-23T17:24:07.171 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-uclogic.ko 2024-03-23T17:24:07.171 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-waltop.ko 2024-03-23T17:24:07.171 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-wiimote.ko 2024-03-23T17:24:07.171 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-xinmo.ko 2024-03-23T17:24:07.171 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-zpff.ko 2024-03-23T17:24:07.171 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-zydacron.ko 2024-03-23T17:24:07.171 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/uhid.ko 2024-03-23T17:24:07.171 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/wacom.ko 2024-03-23T17:24:07.171 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv 2024-03-23T17:24:07.171 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv/hv_balloon.ko 2024-03-23T17:24:07.171 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv/hv_utils.ko 2024-03-23T17:24:07.171 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv/hv_vmbus.ko 2024-03-23T17:24:07.171 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon 2024-03-23T17:24:07.172 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/abituguru.ko 2024-03-23T17:24:07.172 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/abituguru3.ko 2024-03-23T17:24:07.172 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/acpi_power_meter.ko 2024-03-23T17:24:07.172 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ad7414.ko 2024-03-23T17:24:07.172 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ad7418.ko 2024-03-23T17:24:07.172 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adc128d818.ko 2024-03-23T17:24:07.172 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1025.ko 2024-03-23T17:24:07.172 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1026.ko 2024-03-23T17:24:07.172 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1029.ko 2024-03-23T17:24:07.172 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1031.ko 2024-03-23T17:24:07.172 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm9240.ko 2024-03-23T17:24:07.172 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ads7828.ko 2024-03-23T17:24:07.172 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7410.ko 2024-03-23T17:24:07.172 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7411.ko 2024-03-23T17:24:07.172 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7462.ko 2024-03-23T17:24:07.173 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7470.ko 2024-03-23T17:24:07.173 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7475.ko 2024-03-23T17:24:07.173 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7x10.ko 2024-03-23T17:24:07.173 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/amc6821.ko 2024-03-23T17:24:07.173 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/applesmc.ko 2024-03-23T17:24:07.173 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/asb100.ko 2024-03-23T17:24:07.173 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/asc7621.ko 2024-03-23T17:24:07.173 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/asus_atk0110.ko 2024-03-23T17:24:07.173 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/atxp1.ko 2024-03-23T17:24:07.173 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/coretemp.ko 2024-03-23T17:24:07.173 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-03-23T17:24:07.173 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/dme1737.ko 2024-03-23T17:24:07.173 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ds1621.ko 2024-03-23T17:24:07.173 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ds620.ko 2024-03-23T17:24:07.174 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/emc1403.ko 2024-03-23T17:24:07.174 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/emc6w201.ko 2024-03-23T17:24:07.174 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/f71805f.ko 2024-03-23T17:24:07.174 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/f71882fg.ko 2024-03-23T17:24:07.174 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/f75375s.ko 2024-03-23T17:24:07.174 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/fam15h_power.ko 2024-03-23T17:24:07.174 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/fschmd.ko 2024-03-23T17:24:07.174 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/g760a.ko 2024-03-23T17:24:07.174 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/g762.ko 2024-03-23T17:24:07.174 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/gl518sm.ko 2024-03-23T17:24:07.174 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/gl520sm.ko 2024-03-23T17:24:07.174 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/hwmon-vid.ko 2024-03-23T17:24:07.174 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/i5500_temp.ko 2024-03-23T17:24:07.174 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/i5k_amb.ko 2024-03-23T17:24:07.175 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ibmaem.ko 2024-03-23T17:24:07.175 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ibmpex.ko 2024-03-23T17:24:07.175 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ina209.ko 2024-03-23T17:24:07.175 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ina2xx.ko 2024-03-23T17:24:07.175 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/it87.ko 2024-03-23T17:24:07.175 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/k10temp.ko 2024-03-23T17:24:07.175 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/k8temp.ko 2024-03-23T17:24:07.175 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lineage-pem.ko 2024-03-23T17:24:07.175 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm63.ko 2024-03-23T17:24:07.175 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm73.ko 2024-03-23T17:24:07.175 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm75.ko 2024-03-23T17:24:07.175 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm77.ko 2024-03-23T17:24:07.175 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm78.ko 2024-03-23T17:24:07.176 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm80.ko 2024-03-23T17:24:07.176 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm83.ko 2024-03-23T17:24:07.176 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm85.ko 2024-03-23T17:24:07.176 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm87.ko 2024-03-23T17:24:07.176 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm90.ko 2024-03-23T17:24:07.176 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm92.ko 2024-03-23T17:24:07.176 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm93.ko 2024-03-23T17:24:07.176 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm95234.ko 2024-03-23T17:24:07.176 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm95241.ko 2024-03-23T17:24:07.176 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm95245.ko 2024-03-23T17:24:07.176 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc2945.ko 2024-03-23T17:24:07.176 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4151.ko 2024-03-23T17:24:07.176 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4215.ko 2024-03-23T17:24:07.176 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4222.ko 2024-03-23T17:24:07.176 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4245.ko 2024-03-23T17:24:07.177 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4260.ko 2024-03-23T17:24:07.177 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4261.ko 2024-03-23T17:24:07.177 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max16065.ko 2024-03-23T17:24:07.177 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max1619.ko 2024-03-23T17:24:07.177 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max1668.ko 2024-03-23T17:24:07.177 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max197.ko 2024-03-23T17:24:07.177 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max6639.ko 2024-03-23T17:24:07.177 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max6650.ko 2024-03-23T17:24:07.177 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max6697.ko 2024-03-23T17:24:07.177 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/mcp3021.ko 2024-03-23T17:24:07.177 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct6683.ko 2024-03-23T17:24:07.177 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct6775-core.ko 2024-03-23T17:24:07.177 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct6775.ko 2024-03-23T17:24:07.177 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct7802.ko 2024-03-23T17:24:07.178 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pc87360.ko 2024-03-23T17:24:07.178 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pc87427.ko 2024-03-23T17:24:07.178 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pcf8591.ko 2024-03-23T17:24:07.178 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus 2024-03-23T17:24:07.178 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-03-23T17:24:07.178 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-03-23T17:24:07.178 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-03-23T17:24:07.178 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/max16064.ko 2024-03-23T17:24:07.178 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/max34440.ko 2024-03-23T17:24:07.178 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/max8688.ko 2024-03-23T17:24:07.178 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-03-23T17:24:07.178 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-03-23T17:24:07.178 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-03-23T17:24:07.179 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-03-23T17:24:07.179 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-03-23T17:24:07.179 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-03-23T17:24:07.179 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/powr1220.ko 2024-03-23T17:24:07.179 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sch5627.ko 2024-03-23T17:24:07.179 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sch5636.ko 2024-03-23T17:24:07.179 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sch56xx-common.ko 2024-03-23T17:24:07.179 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sht15.ko 2024-03-23T17:24:07.179 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sht21.ko 2024-03-23T17:24:07.179 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/shtc1.ko 2024-03-23T17:24:07.179 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sis5595.ko 2024-03-23T17:24:07.179 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/smsc47b397.ko 2024-03-23T17:24:07.179 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/smsc47m1.ko 2024-03-23T17:24:07.179 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/smsc47m192.ko 2024-03-23T17:24:07.180 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/thmc50.ko 2024-03-23T17:24:07.180 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp102.ko 2024-03-23T17:24:07.180 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp103.ko 2024-03-23T17:24:07.180 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp401.ko 2024-03-23T17:24:07.180 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp421.ko 2024-03-23T17:24:07.180 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/via-cputemp.ko 2024-03-23T17:24:07.180 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/via686a.ko 2024-03-23T17:24:07.180 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/vt1211.ko 2024-03-23T17:24:07.180 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/vt8231.ko 2024-03-23T17:24:07.180 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83627ehf.ko 2024-03-23T17:24:07.180 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83627hf.ko 2024-03-23T17:24:07.180 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83781d.ko 2024-03-23T17:24:07.180 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83791d.ko 2024-03-23T17:24:07.180 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83792d.ko 2024-03-23T17:24:07.180 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83793.ko 2024-03-23T17:24:07.181 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83795.ko 2024-03-23T17:24:07.181 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83l785ts.ko 2024-03-23T17:24:07.181 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83l786ng.ko 2024-03-23T17:24:07.181 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c 2024-03-23T17:24:07.181 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/algos 2024-03-23T17:24:07.181 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-03-23T17:24:07.181 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-03-23T17:24:07.181 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses 2024-03-23T17:24:07.181 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-03-23T17:24:07.181 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-03-23T17:24:07.181 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-03-23T17:24:07.181 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-03-23T17:24:07.181 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-03-23T17:24:07.181 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-03-23T17:24:07.181 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-03-23T17:24:07.182 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-03-23T17:24:07.182 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-i801.ko 2024-03-23T17:24:07.182 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-isch.ko 2024-03-23T17:24:07.182 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-03-23T17:24:07.182 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-03-23T17:24:07.182 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-03-23T17:24:07.182 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-parport.ko 2024-03-23T17:24:07.182 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-03-23T17:24:07.182 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-03-23T17:24:07.182 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-03-23T17:24:07.182 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-03-23T17:24:07.182 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-03-23T17:24:07.182 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-03-23T17:24:07.182 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-via.ko 2024-03-23T17:24:07.183 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-03-23T17:24:07.183 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-03-23T17:24:07.183 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-dev.ko 2024-03-23T17:24:07.183 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-mux.ko 2024-03-23T17:24:07.183 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-03-23T17:24:07.183 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-smbus.ko 2024-03-23T17:24:07.183 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-stub.ko 2024-03-23T17:24:07.183 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband 2024-03-23T17:24:07.183 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core 2024-03-23T17:24:07.183 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_cm.ko 2024-03-23T17:24:07.183 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_core.ko 2024-03-23T17:24:07.183 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_umad.ko 2024-03-23T17:24:07.183 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-03-23T17:24:07.183 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/iw_cm.ko 2024-03-23T17:24:07.183 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/rdma_cm.ko 2024-03-23T17:24:07.184 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-03-23T17:24:07.184 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw 2024-03-23T17:24:07.184 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/cxgb4 2024-03-23T17:24:07.184 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-03-23T17:24:07.184 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx4 2024-03-23T17:24:07.184 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-03-23T17:24:07.184 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx5 2024-03-23T17:24:07.184 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-03-23T17:24:07.184 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mthca 2024-03-23T17:24:07.184 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-03-23T17:24:07.184 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp 2024-03-23T17:24:07.184 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/ipoib 2024-03-23T17:24:07.184 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-03-23T17:24:07.184 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/iser 2024-03-23T17:24:07.185 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-03-23T17:24:07.185 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/isert 2024-03-23T17:24:07.185 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-03-23T17:24:07.185 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srp 2024-03-23T17:24:07.185 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-03-23T17:24:07.185 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srpt 2024-03-23T17:24:07.185 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-03-23T17:24:07.185 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input 2024-03-23T17:24:07.185 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport 2024-03-23T17:24:07.185 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-03-23T17:24:07.185 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport/fm801-gp.ko 2024-03-23T17:24:07.185 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport/gameport.ko 2024-03-23T17:24:07.185 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/joydev.ko 2024-03-23T17:24:07.185 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard 2024-03-23T17:24:07.185 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard/gpio_keys.ko 2024-03-23T17:24:07.186 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse 2024-03-23T17:24:07.186 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/appletouch.ko 2024-03-23T17:24:07.186 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/bcm5974.ko 2024-03-23T17:24:07.186 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/cyapatp.ko 2024-03-23T17:24:07.186 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/elan_i2c.ko 2024-03-23T17:24:07.186 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/sermouse.ko 2024-03-23T17:24:07.186 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-03-23T17:24:07.186 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/synaptics_usb.ko 2024-03-23T17:24:07.186 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/vsxxxaa.ko 2024-03-23T17:24:07.186 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/rmi4 2024-03-23T17:24:07.186 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/rmi4/rmi_core.ko 2024-03-23T17:24:07.186 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio 2024-03-23T17:24:07.186 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/altera_ps2.ko 2024-03-23T17:24:07.186 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/arc_ps2.ko 2024-03-23T17:24:07.187 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-03-23T17:24:07.187 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/serio_raw.ko 2024-03-23T17:24:07.187 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/sparse-keymap.ko 2024-03-23T17:24:07.187 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds 2024-03-23T17:24:07.187 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-blinkm.ko 2024-03-23T17:24:07.187 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-lm3530.ko 2024-03-23T17:24:07.187 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-lp3944.ko 2024-03-23T17:24:07.187 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-ss4200.ko 2024-03-23T17:24:07.187 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger 2024-03-23T17:24:07.187 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-03-23T17:24:07.187 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-03-23T17:24:07.187 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-03-23T17:24:07.187 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-03-23T17:24:07.187 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-03-23T17:24:07.187 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-03-23T17:24:07.188 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-03-23T17:24:07.188 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-03-23T17:24:07.188 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-03-23T17:24:07.188 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md 2024-03-23T17:24:07.188 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/bcache 2024-03-23T17:24:07.188 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/bcache/bcache.ko 2024-03-23T17:24:07.188 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-bio-prison.ko 2024-03-23T17:24:07.188 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-cache-smq.ko 2024-03-23T17:24:07.188 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-cache.ko 2024-03-23T17:24:07.188 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-crypt.ko 2024-03-23T17:24:07.188 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-delay.ko 2024-03-23T17:24:07.188 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-flakey.ko 2024-03-23T17:24:07.188 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-log-userspace.ko 2024-03-23T17:24:07.189 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-multipath.ko 2024-03-23T17:24:07.189 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-queue-length.ko 2024-03-23T17:24:07.189 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-raid.ko 2024-03-23T17:24:07.189 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-round-robin.ko 2024-03-23T17:24:07.189 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-service-time.ko 2024-03-23T17:24:07.189 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-switch.ko 2024-03-23T17:24:07.189 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-thin-pool.ko 2024-03-23T17:24:07.189 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-verity.ko 2024-03-23T17:24:07.189 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/persistent-data 2024-03-23T17:24:07.189 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-03-23T17:24:07.189 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid0.ko 2024-03-23T17:24:07.189 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid1.ko 2024-03-23T17:24:07.189 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid10.ko 2024-03-23T17:24:07.189 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid456.ko 2024-03-23T17:24:07.190 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message 2024-03-23T17:24:07.190 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion 2024-03-23T17:24:07.190 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptbase.ko 2024-03-23T17:24:07.190 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptctl.ko 2024-03-23T17:24:07.190 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptfc.ko 2024-03-23T17:24:07.190 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptlan.ko 2024-03-23T17:24:07.190 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptsas.ko 2024-03-23T17:24:07.190 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptscsih.ko 2024-03-23T17:24:07.190 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptspi.ko 2024-03-23T17:24:07.190 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd 2024-03-23T17:24:07.190 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/lpc_ich.ko 2024-03-23T17:24:07.190 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/lpc_sch.ko 2024-03-23T17:24:07.190 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/mfd-core.ko 2024-03-23T17:24:07.190 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/sm501.ko 2024-03-23T17:24:07.191 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/viperboard.ko 2024-03-23T17:24:07.191 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/vx855.ko 2024-03-23T17:24:07.191 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/wl1273-core.ko 2024-03-23T17:24:07.191 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc 2024-03-23T17:24:07.191 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/altera-stapl 2024-03-23T17:24:07.191 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-03-23T17:24:07.191 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/apds9802als.ko 2024-03-23T17:24:07.191 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/apds990x.ko 2024-03-23T17:24:07.191 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/bh1770glc.ko 2024-03-23T17:24:07.191 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/cb710 2024-03-23T17:24:07.191 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/cb710/cb710.ko 2024-03-23T17:24:07.191 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom 2024-03-23T17:24:07.191 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom/at24.ko 2024-03-23T17:24:07.192 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-03-23T17:24:07.192 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom/max6875.ko 2024-03-23T17:24:07.192 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/enclosure.ko 2024-03-23T17:24:07.192 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/hpilo.ko 2024-03-23T17:24:07.192 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/isl29003.ko 2024-03-23T17:24:07.192 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/isl29020.ko 2024-03-23T17:24:07.192 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/lis3lv02d 2024-03-23T17:24:07.192 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-03-23T17:24:07.192 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-03-23T17:24:07.192 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei 2024-03-23T17:24:07.192 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei/mei-me.ko 2024-03-23T17:24:07.192 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei/mei-txe.ko 2024-03-23T17:24:07.192 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei/mei.ko 2024-03-23T17:24:07.193 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-gru 2024-03-23T17:24:07.193 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-gru/gru.ko 2024-03-23T17:24:07.193 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-xp 2024-03-23T17:24:07.193 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-xp/xp.ko 2024-03-23T17:24:07.193 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-xp/xpc.ko 2024-03-23T17:24:07.193 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-03-23T17:24:07.193 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/tifm_7xx1.ko 2024-03-23T17:24:07.193 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/tifm_core.ko 2024-03-23T17:24:07.193 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/tsl2550.ko 2024-03-23T17:24:07.193 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/vmw_balloon.ko 2024-03-23T17:24:07.193 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/vmw_vmci 2024-03-23T17:24:07.193 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-03-23T17:24:07.193 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net 2024-03-23T17:24:07.193 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/bonding 2024-03-23T17:24:07.194 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/bonding/bonding.ko 2024-03-23T17:24:07.194 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa 2024-03-23T17:24:07.194 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/b53 2024-03-23T17:24:07.194 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/b53/b53_common.ko 2024-03-23T17:24:07.194 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/bcm-sf2.ko 2024-03-23T17:24:07.194 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/mv88e6060.ko 2024-03-23T17:24:07.194 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/mv88e6xxx 2024-03-23T17:24:07.194 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-03-23T17:24:07.194 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dummy.ko 2024-03-23T17:24:07.194 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/eql.ko 2024-03-23T17:24:07.194 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet 2024-03-23T17:24:07.194 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com 2024-03-23T17:24:07.194 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-03-23T17:24:07.194 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-03-23T17:24:07.195 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-03-23T17:24:07.195 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-03-23T17:24:07.195 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390 2024-03-23T17:24:07.195 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/8390.ko 2024-03-23T17:24:07.195 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-03-23T17:24:07.195 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-03-23T17:24:07.195 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-03-23T17:24:07.195 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/adaptec 2024-03-23T17:24:07.195 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-03-23T17:24:07.195 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/agere 2024-03-23T17:24:07.195 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/agere/et131x.ko 2024-03-23T17:24:07.195 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/alteon 2024-03-23T17:24:07.195 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-03-23T17:24:07.195 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/altera 2024-03-23T17:24:07.195 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-03-23T17:24:07.196 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd 2024-03-23T17:24:07.196 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-03-23T17:24:07.196 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-03-23T17:24:07.196 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-03-23T17:24:07.196 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros 2024-03-23T17:24:07.196 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/alx 2024-03-23T17:24:07.196 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-03-23T17:24:07.196 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1c 2024-03-23T17:24:07.196 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-03-23T17:24:07.196 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1e 2024-03-23T17:24:07.196 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-03-23T17:24:07.196 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atlx 2024-03-23T17:24:07.196 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-03-23T17:24:07.196 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-03-23T17:24:07.197 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom 2024-03-23T17:24:07.197 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-03-23T17:24:07.197 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-03-23T17:24:07.197 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-03-23T17:24:07.197 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-03-23T17:24:07.197 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-03-23T17:24:07.197 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/genet 2024-03-23T17:24:07.197 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-03-23T17:24:07.197 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-03-23T17:24:07.197 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/brocade 2024-03-23T17:24:07.197 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/brocade/bna 2024-03-23T17:24:07.197 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-03-23T17:24:07.197 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio 2024-03-23T17:24:07.197 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb 2024-03-23T17:24:07.197 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-03-23T17:24:07.198 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-03-23T17:24:07.198 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-03-23T17:24:07.198 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-03-23T17:24:07.198 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-03-23T17:24:07.198 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-03-23T17:24:07.198 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-03-23T17:24:07.198 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-03-23T17:24:07.198 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-03-23T17:24:07.198 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/cisco 2024-03-23T17:24:07.198 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/cisco/enic 2024-03-23T17:24:07.198 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-03-23T17:24:07.198 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec 2024-03-23T17:24:07.198 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip 2024-03-23T17:24:07.198 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-03-23T17:24:07.199 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-03-23T17:24:07.199 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-03-23T17:24:07.199 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-03-23T17:24:07.199 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-03-23T17:24:07.199 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-03-23T17:24:07.199 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dlink 2024-03-23T17:24:07.199 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-03-23T17:24:07.199 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-03-23T17:24:07.199 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dnet.ko 2024-03-23T17:24:07.199 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/emulex 2024-03-23T17:24:07.199 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/emulex/benet 2024-03-23T17:24:07.199 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-03-23T17:24:07.199 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/ethoc.ko 2024-03-23T17:24:07.200 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/fealnx.ko 2024-03-23T17:24:07.200 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel 2024-03-23T17:24:07.200 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e100.ko 2024-03-23T17:24:07.200 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000 2024-03-23T17:24:07.200 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-03-23T17:24:07.200 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000e 2024-03-23T17:24:07.200 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-03-23T17:24:07.200 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/fm10k 2024-03-23T17:24:07.200 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-03-23T17:24:07.200 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/i40e 2024-03-23T17:24:07.200 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-03-23T17:24:07.200 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/iavf 2024-03-23T17:24:07.200 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-03-23T17:24:07.200 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igb 2024-03-23T17:24:07.200 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-03-23T17:24:07.201 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igbvf 2024-03-23T17:24:07.201 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-03-23T17:24:07.201 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbe 2024-03-23T17:24:07.201 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-03-23T17:24:07.201 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbevf 2024-03-23T17:24:07.201 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-03-23T17:24:07.201 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/jme.ko 2024-03-23T17:24:07.201 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell 2024-03-23T17:24:07.201 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-03-23T17:24:07.201 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell/skge.ko 2024-03-23T17:24:07.201 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-03-23T17:24:07.201 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox 2024-03-23T17:24:07.201 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx4 2024-03-23T17:24:07.202 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-03-23T17:24:07.202 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-03-23T17:24:07.202 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx5 2024-03-23T17:24:07.202 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-03-23T17:24:07.202 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-03-23T17:24:07.202 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/micrel 2024-03-23T17:24:07.202 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-03-23T17:24:07.202 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/myricom 2024-03-23T17:24:07.202 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/myricom/myri10ge 2024-03-23T17:24:07.202 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-03-23T17:24:07.202 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/natsemi 2024-03-23T17:24:07.202 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-03-23T17:24:07.202 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-03-23T17:24:07.203 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/neterion 2024-03-23T17:24:07.203 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-03-23T17:24:07.203 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/nvidia 2024-03-23T17:24:07.203 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-03-23T17:24:07.203 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/packetengines 2024-03-23T17:24:07.203 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-03-23T17:24:07.203 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-03-23T17:24:07.203 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic 2024-03-23T17:24:07.203 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/netxen 2024-03-23T17:24:07.203 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-03-23T17:24:07.203 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-03-23T17:24:07.203 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-03-23T17:24:07.203 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-03-23T17:24:07.203 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rdc 2024-03-23T17:24:07.204 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-03-23T17:24:07.204 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek 2024-03-23T17:24:07.204 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-03-23T17:24:07.204 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-03-23T17:24:07.204 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/atp.ko 2024-03-23T17:24:07.204 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-03-23T17:24:07.204 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rocker 2024-03-23T17:24:07.204 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-03-23T17:24:07.204 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sfc 2024-03-23T17:24:07.204 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-03-23T17:24:07.204 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/silan 2024-03-23T17:24:07.204 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-03-23T17:24:07.204 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sis 2024-03-23T17:24:07.204 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sis/sis190.ko 2024-03-23T17:24:07.205 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sis/sis900.ko 2024-03-23T17:24:07.205 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc 2024-03-23T17:24:07.205 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-03-23T17:24:07.205 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-03-23T17:24:07.205 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-03-23T17:24:07.205 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-03-23T17:24:07.205 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro 2024-03-23T17:24:07.205 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro/stmmac 2024-03-23T17:24:07.205 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-03-23T17:24:07.205 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-03-23T17:24:07.205 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun 2024-03-23T17:24:07.205 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/cassini.ko 2024-03-23T17:24:07.205 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/niu.ko 2024-03-23T17:24:07.205 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/sungem.ko 2024-03-23T17:24:07.205 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-03-23T17:24:07.206 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/tehuti 2024-03-23T17:24:07.206 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-03-23T17:24:07.206 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/ti 2024-03-23T17:24:07.206 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/ti/tlan.ko 2024-03-23T17:24:07.206 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/via 2024-03-23T17:24:07.206 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-03-23T17:24:07.206 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-03-23T17:24:07.206 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/wiznet 2024-03-23T17:24:07.206 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-03-23T17:24:07.206 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-03-23T17:24:07.206 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/xircom 2024-03-23T17:24:07.206 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-03-23T17:24:07.206 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/geneve.ko 2024-03-23T17:24:07.206 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/hyperv 2024-03-23T17:24:07.207 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-03-23T17:24:07.207 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ieee802154 2024-03-23T17:24:07.207 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ieee802154/fakelb.ko 2024-03-23T17:24:07.207 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ifb.ko 2024-03-23T17:24:07.207 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ipvlan 2024-03-23T17:24:07.207 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ipvlan/ipvlan.ko 2024-03-23T17:24:07.207 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/macvlan.ko 2024-03-23T17:24:07.207 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/macvtap.ko 2024-03-23T17:24:07.207 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio 2024-03-23T17:24:07.207 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio.ko 2024-03-23T17:24:07.207 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-03-23T17:24:07.207 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-03-23T17:24:07.207 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio/mdio-regmap.ko 2024-03-23T17:24:07.208 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mii.ko 2024-03-23T17:24:07.208 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/net_failover.ko 2024-03-23T17:24:07.208 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/netconsole.ko 2024-03-23T17:24:07.208 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/nlmon.ko 2024-03-23T17:24:07.208 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/pcs 2024-03-23T17:24:07.208 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/pcs/pcs-lynx.ko 2024-03-23T17:24:07.208 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-03-23T17:24:07.208 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy 2024-03-23T17:24:07.208 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/amd.ko 2024-03-23T17:24:07.208 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-03-23T17:24:07.208 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-03-23T17:24:07.208 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm7xxx.ko 2024-03-23T17:24:07.209 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm87xx.ko 2024-03-23T17:24:07.209 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/broadcom.ko 2024-03-23T17:24:07.209 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/cicada.ko 2024-03-23T17:24:07.209 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/davicom.ko 2024-03-23T17:24:07.209 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/dp83640.ko 2024-03-23T17:24:07.209 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/et1011c.ko 2024-03-23T17:24:07.209 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/icplus.ko 2024-03-23T17:24:07.209 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/lxt.ko 2024-03-23T17:24:07.209 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/marvell.ko 2024-03-23T17:24:07.209 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/micrel.ko 2024-03-23T17:24:07.209 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/national.ko 2024-03-23T17:24:07.209 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/phylink.ko 2024-03-23T17:24:07.209 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/qsemi.ko 2024-03-23T17:24:07.209 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/realtek.ko 2024-03-23T17:24:07.210 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/smsc.ko 2024-03-23T17:24:07.210 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/ste10Xp.ko 2024-03-23T17:24:07.210 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/vitesse.ko 2024-03-23T17:24:07.210 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp 2024-03-23T17:24:07.210 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/bsd_comp.ko 2024-03-23T17:24:07.210 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_async.ko 2024-03-23T17:24:07.210 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_deflate.ko 2024-03-23T17:24:07.210 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_generic.ko 2024-03-23T17:24:07.210 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_mppe.ko 2024-03-23T17:24:07.210 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_synctty.ko 2024-03-23T17:24:07.210 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/pppoe.ko 2024-03-23T17:24:07.210 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/pppox.ko 2024-03-23T17:24:07.211 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/pptp.ko 2024-03-23T17:24:07.211 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/slip 2024-03-23T17:24:07.211 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/slip/slhc.ko 2024-03-23T17:24:07.211 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/slip/slip.ko 2024-03-23T17:24:07.211 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/sungem_phy.ko 2024-03-23T17:24:07.211 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/tap.ko 2024-03-23T17:24:07.211 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team 2024-03-23T17:24:07.211 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team.ko 2024-03-23T17:24:07.211 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_activebackup.ko 2024-03-23T17:24:07.211 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_broadcast.ko 2024-03-23T17:24:07.211 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-03-23T17:24:07.211 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_random.ko 2024-03-23T17:24:07.211 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-03-23T17:24:07.211 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/tun.ko 2024-03-23T17:24:07.211 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/veth.ko 2024-03-23T17:24:07.212 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/virtio_net.ko 2024-03-23T17:24:07.212 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vmxnet3 2024-03-23T17:24:07.212 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-03-23T17:24:07.212 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vxlan 2024-03-23T17:24:07.212 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vxlan/vxlan.ko 2024-03-23T17:24:07.212 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/xen-netback 2024-03-23T17:24:07.212 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/xen-netback/xen-netback.ko 2024-03-23T17:24:07.212 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/xen-netfront.ko 2024-03-23T17:24:07.212 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ntb 2024-03-23T17:24:07.212 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ntb/ntb.ko 2024-03-23T17:24:07.212 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme 2024-03-23T17:24:07.212 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme/host 2024-03-23T17:24:07.212 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme/host/nvme-core.ko 2024-03-23T17:24:07.212 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme/host/nvme.ko 2024-03-23T17:24:07.213 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport 2024-03-23T17:24:07.213 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport.ko 2024-03-23T17:24:07.213 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport_cs.ko 2024-03-23T17:24:07.213 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport_pc.ko 2024-03-23T17:24:07.213 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport_serial.ko 2024-03-23T17:24:07.213 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci 2024-03-23T17:24:07.213 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/hotplug 2024-03-23T17:24:07.213 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-03-23T17:24:07.213 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/pcie 2024-03-23T17:24:07.213 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/pcie/aer_inject.ko 2024-03-23T17:24:07.213 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/xen-pcifront.ko 2024-03-23T17:24:07.213 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia 2024-03-23T17:24:07.213 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia/i82092.ko 2024-03-23T17:24:07.213 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia/pd6729.ko 2024-03-23T17:24:07.214 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia/yenta_socket.ko 2024-03-23T17:24:07.214 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pinctrl 2024-03-23T17:24:07.214 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pinctrl/intel 2024-03-23T17:24:07.214 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-03-23T17:24:07.214 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform 2024-03-23T17:24:07.214 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86 2024-03-23T17:24:07.214 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/acer-wmi.ko 2024-03-23T17:24:07.214 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/acerhdf.ko 2024-03-23T17:24:07.214 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/apple-gmux.ko 2024-03-23T17:24:07.214 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/asus-laptop.ko 2024-03-23T17:24:07.214 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-03-23T17:24:07.214 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/asus-wmi.ko 2024-03-23T17:24:07.214 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/classmate-laptop.ko 2024-03-23T17:24:07.215 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-03-23T17:24:07.215 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-03-23T17:24:07.215 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-03-23T17:24:07.215 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-03-23T17:24:07.215 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/hdaps.ko 2024-03-23T17:24:07.215 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/intel 2024-03-23T17:24:07.215 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-03-23T17:24:07.215 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/intel_ips.ko 2024-03-23T17:24:07.215 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/msi-wmi.ko 2024-03-23T17:24:07.215 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/mxm-wmi.ko 2024-03-23T17:24:07.215 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-03-23T17:24:07.215 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/samsung-laptop.ko 2024-03-23T17:24:07.215 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/samsung-q10.ko 2024-03-23T17:24:07.215 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/topstar-laptop.ko 2024-03-23T17:24:07.216 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-03-23T17:24:07.216 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/toshiba_haps.ko 2024-03-23T17:24:07.216 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/wmi-bmof.ko 2024-03-23T17:24:07.216 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/wmi.ko 2024-03-23T17:24:07.216 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/powercap 2024-03-23T17:24:07.216 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/powercap/intel_rapl_common.ko 2024-03-23T17:24:07.216 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/powercap/intel_rapl_msr.ko 2024-03-23T17:24:07.216 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps 2024-03-23T17:24:07.216 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients 2024-03-23T17:24:07.216 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients/pps-gpio.ko 2024-03-23T17:24:07.216 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients/pps-ldisc.ko 2024-03-23T17:24:07.216 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients/pps_parport.ko 2024-03-23T17:24:07.216 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/pps_core.ko 2024-03-23T17:24:07.217 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ptp 2024-03-23T17:24:07.217 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ptp/ptp.ko 2024-03-23T17:24:07.217 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ptp/ptp_kvm.ko 2024-03-23T17:24:07.217 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc 2024-03-23T17:24:07.217 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-bq32k.ko 2024-03-23T17:24:07.217 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1286.ko 2024-03-23T17:24:07.217 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1307.ko 2024-03-23T17:24:07.217 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1374.ko 2024-03-23T17:24:07.217 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1511.ko 2024-03-23T17:24:07.217 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1553.ko 2024-03-23T17:24:07.217 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1672.ko 2024-03-23T17:24:07.217 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1742.ko 2024-03-23T17:24:07.217 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds2404.ko 2024-03-23T17:24:07.218 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds3232.ko 2024-03-23T17:24:07.218 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-em3027.ko 2024-03-23T17:24:07.218 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-fm3130.ko 2024-03-23T17:24:07.218 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-isl12022.ko 2024-03-23T17:24:07.218 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-isl1208.ko 2024-03-23T17:24:07.218 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-m41t80.ko 2024-03-23T17:24:07.218 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-m48t35.ko 2024-03-23T17:24:07.218 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-m48t59.ko 2024-03-23T17:24:07.218 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-max6900.ko 2024-03-23T17:24:07.218 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-msm6242.ko 2024-03-23T17:24:07.218 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf2127.ko 2024-03-23T17:24:07.218 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf85063.ko 2024-03-23T17:24:07.218 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf8523.ko 2024-03-23T17:24:07.218 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf8563.ko 2024-03-23T17:24:07.218 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf8583.ko 2024-03-23T17:24:07.219 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rp5c01.ko 2024-03-23T17:24:07.219 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rs5c372.ko 2024-03-23T17:24:07.219 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-03-23T17:24:07.219 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rx8025.ko 2024-03-23T17:24:07.219 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rx8581.ko 2024-03-23T17:24:07.219 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-03-23T17:24:07.219 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-x1205.ko 2024-03-23T17:24:07.219 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi 2024-03-23T17:24:07.219 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/3w-9xxx.ko 2024-03-23T17:24:07.219 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/3w-sas.ko 2024-03-23T17:24:07.219 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/3w-xxxx.ko 2024-03-23T17:24:07.219 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/BusLogic.ko 2024-03-23T17:24:07.219 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/a100u2w.ko 2024-03-23T17:24:07.219 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aacraid 2024-03-23T17:24:07.219 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aacraid/aacraid.ko 2024-03-23T17:24:07.220 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/advansys.ko 2024-03-23T17:24:07.220 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aic7xxx 2024-03-23T17:24:07.220 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-03-23T17:24:07.220 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-03-23T17:24:07.220 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/am53c974.ko 2024-03-23T17:24:07.220 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/arcmsr 2024-03-23T17:24:07.220 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-03-23T17:24:07.220 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/atp870u.ko 2024-03-23T17:24:07.220 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/be2iscsi 2024-03-23T17:24:07.220 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-03-23T17:24:07.220 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bfa 2024-03-23T17:24:07.220 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bfa/bfa.ko 2024-03-23T17:24:07.220 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2fc 2024-03-23T17:24:07.220 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-03-23T17:24:07.221 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2i 2024-03-23T17:24:07.221 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-03-23T17:24:07.221 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ch.ko 2024-03-23T17:24:07.221 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/csiostor 2024-03-23T17:24:07.221 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/csiostor/csiostor.ko 2024-03-23T17:24:07.221 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi 2024-03-23T17:24:07.221 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb3i 2024-03-23T17:24:07.221 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-03-23T17:24:07.221 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb4i 2024-03-23T17:24:07.221 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-03-23T17:24:07.221 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-03-23T17:24:07.221 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/dc395x.ko 2024-03-23T17:24:07.221 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler 2024-03-23T17:24:07.221 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-03-23T17:24:07.222 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-03-23T17:24:07.222 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-03-23T17:24:07.222 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-03-23T17:24:07.222 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/dmx3191d.ko 2024-03-23T17:24:07.222 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/esas2r 2024-03-23T17:24:07.222 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/esas2r/esas2r.ko 2024-03-23T17:24:07.222 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/esp_scsi.ko 2024-03-23T17:24:07.222 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fcoe 2024-03-23T17:24:07.222 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fcoe/fcoe.ko 2024-03-23T17:24:07.222 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-03-23T17:24:07.222 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fnic 2024-03-23T17:24:07.222 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fnic/fnic.ko 2024-03-23T17:24:07.222 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/hpsa.ko 2024-03-23T17:24:07.222 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/hptiop.ko 2024-03-23T17:24:07.223 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/hv_storvsc.ko 2024-03-23T17:24:07.223 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/initio.ko 2024-03-23T17:24:07.223 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ipr.ko 2024-03-23T17:24:07.223 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ips.ko 2024-03-23T17:24:07.223 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/isci 2024-03-23T17:24:07.223 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/isci/isci.ko 2024-03-23T17:24:07.223 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-03-23T17:24:07.223 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/iscsi_tcp.ko 2024-03-23T17:24:07.223 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libfc 2024-03-23T17:24:07.223 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libfc/libfc.ko 2024-03-23T17:24:07.223 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libiscsi.ko 2024-03-23T17:24:07.223 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libiscsi_tcp.ko 2024-03-23T17:24:07.223 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libsas 2024-03-23T17:24:07.223 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libsas/libsas.ko 2024-03-23T17:24:07.224 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/lpfc 2024-03-23T17:24:07.224 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/lpfc/lpfc.ko 2024-03-23T17:24:07.224 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid 2024-03-23T17:24:07.224 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid.ko 2024-03-23T17:24:07.224 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-03-23T17:24:07.224 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-03-23T17:24:07.224 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-03-23T17:24:07.224 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mpt3sas 2024-03-23T17:24:07.224 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-03-23T17:24:07.224 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mvsas 2024-03-23T17:24:07.224 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mvsas/mvsas.ko 2024-03-23T17:24:07.224 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mvumi.ko 2024-03-23T17:24:07.224 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/pm8001 2024-03-23T17:24:07.224 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-03-23T17:24:07.224 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/pmcraid.ko 2024-03-23T17:24:07.225 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla1280.ko 2024-03-23T17:24:07.225 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla2xxx 2024-03-23T17:24:07.225 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-03-23T17:24:07.225 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-03-23T17:24:07.225 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla4xxx 2024-03-23T17:24:07.225 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-03-23T17:24:07.225 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/raid_class.ko 2024-03-23T17:24:07.225 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_debug.ko 2024-03-23T17:24:07.225 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_fc.ko 2024-03-23T17:24:07.225 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-03-23T17:24:07.225 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_sas.ko 2024-03-23T17:24:07.225 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_spi.ko 2024-03-23T17:24:07.225 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_srp.ko 2024-03-23T17:24:07.225 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ses.ko 2024-03-23T17:24:07.226 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/st.ko 2024-03-23T17:24:07.226 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/stex.ko 2024-03-23T17:24:07.226 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/sym53c8xx_2 2024-03-23T17:24:07.226 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-03-23T17:24:07.226 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/virtio_scsi.ko 2024-03-23T17:24:07.226 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/vmw_pvscsi.ko 2024-03-23T17:24:07.226 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/wd719x.ko 2024-03-23T17:24:07.226 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ssb 2024-03-23T17:24:07.226 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ssb/ssb.ko 2024-03-23T17:24:07.226 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target 2024-03-23T17:24:07.226 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/iscsi 2024-03-23T17:24:07.226 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-03-23T17:24:07.226 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/loopback 2024-03-23T17:24:07.226 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/loopback/tcm_loop.ko 2024-03-23T17:24:07.226 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_file.ko 2024-03-23T17:24:07.227 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_iblock.ko 2024-03-23T17:24:07.227 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_mod.ko 2024-03-23T17:24:07.227 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_pscsi.ko 2024-03-23T17:24:07.227 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_user.ko 2024-03-23T17:24:07.227 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/tcm_fc 2024-03-23T17:24:07.227 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-03-23T17:24:07.227 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal 2024-03-23T17:24:07.227 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel 2024-03-23T17:24:07.227 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal 2024-03-23T17:24:07.227 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-03-23T17:24:07.227 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-03-23T17:24:07.227 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-03-23T17:24:07.227 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-03-23T17:24:07.227 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-03-23T17:24:07.227 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-03-23T17:24:07.228 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-03-23T17:24:07.228 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-03-23T17:24:07.228 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-03-23T17:24:07.228 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-03-23T17:24:07.228 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-03-23T17:24:07.228 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-03-23T17:24:07.228 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-03-23T17:24:07.228 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-03-23T17:24:07.228 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-03-23T17:24:07.228 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-03-23T17:24:07.228 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-03-23T17:24:07.228 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-03-23T17:24:07.228 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty 2024-03-23T17:24:07.228 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/ipwireless 2024-03-23T17:24:07.229 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-03-23T17:24:07.229 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/n_gsm.ko 2024-03-23T17:24:07.229 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/n_hdlc.ko 2024-03-23T17:24:07.229 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/nozomi.ko 2024-03-23T17:24:07.229 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial 2024-03-23T17:24:07.229 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/8250 2024-03-23T17:24:07.229 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-03-23T17:24:07.229 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/arc_uart.ko 2024-03-23T17:24:07.229 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/jsm 2024-03-23T17:24:07.229 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/jsm/jsm.ko 2024-03-23T17:24:07.229 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs 2024-03-23T17:24:07.229 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/core 2024-03-23T17:24:07.229 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/core/ufshcd-core.ko 2024-03-23T17:24:07.229 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/host 2024-03-23T17:24:07.229 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-03-23T17:24:07.230 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio 2024-03-23T17:24:07.230 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio.ko 2024-03-23T17:24:07.230 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_aec.ko 2024-03-23T17:24:07.230 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_cif.ko 2024-03-23T17:24:07.230 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_pci_generic.ko 2024-03-23T17:24:07.230 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_sercos3.ko 2024-03-23T17:24:07.230 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb 2024-03-23T17:24:07.230 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class 2024-03-23T17:24:07.230 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/cdc-acm.ko 2024-03-23T17:24:07.230 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/cdc-wdm.ko 2024-03-23T17:24:07.230 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/usblp.ko 2024-03-23T17:24:07.230 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/usbtmc.ko 2024-03-23T17:24:07.230 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/host 2024-03-23T17:24:07.230 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/host/sl811-hcd.ko 2024-03-23T17:24:07.230 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/image 2024-03-23T17:24:07.231 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/image/mdc800.ko 2024-03-23T17:24:07.231 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/image/microtek.ko 2024-03-23T17:24:07.231 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc 2024-03-23T17:24:07.231 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/adutux.ko 2024-03-23T17:24:07.231 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/appledisplay.ko 2024-03-23T17:24:07.231 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/emi26.ko 2024-03-23T17:24:07.231 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/emi62.ko 2024-03-23T17:24:07.231 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/ezusb.ko 2024-03-23T17:24:07.231 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/idmouse.ko 2024-03-23T17:24:07.231 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/iowarrior.ko 2024-03-23T17:24:07.231 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/isight_firmware.ko 2024-03-23T17:24:07.231 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/ldusb.ko 2024-03-23T17:24:07.231 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/legousbtower.ko 2024-03-23T17:24:07.231 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/sisusbvga 2024-03-23T17:24:07.231 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-03-23T17:24:07.232 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/trancevibrator.ko 2024-03-23T17:24:07.232 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/usb3503.ko 2024-03-23T17:24:07.232 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/usblcd.ko 2024-03-23T17:24:07.232 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/usbsevseg.ko 2024-03-23T17:24:07.232 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/uss720.ko 2024-03-23T17:24:07.232 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/yurex.ko 2024-03-23T17:24:07.232 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/phy 2024-03-23T17:24:07.232 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/phy/phy-generic.ko 2024-03-23T17:24:07.232 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial 2024-03-23T17:24:07.232 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/aircable.ko 2024-03-23T17:24:07.232 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ark3116.ko 2024-03-23T17:24:07.232 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/belkin_sa.ko 2024-03-23T17:24:07.232 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ch341.ko 2024-03-23T17:24:07.232 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/cp210x.ko 2024-03-23T17:24:07.232 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/cyberjack.ko 2024-03-23T17:24:07.233 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/cypress_m8.ko 2024-03-23T17:24:07.233 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/digi_acceleport.ko 2024-03-23T17:24:07.233 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/empeg.ko 2024-03-23T17:24:07.233 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ftdi_sio.ko 2024-03-23T17:24:07.233 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/garmin_gps.ko 2024-03-23T17:24:07.233 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/io_edgeport.ko 2024-03-23T17:24:07.233 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/io_ti.ko 2024-03-23T17:24:07.233 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ipaq.ko 2024-03-23T17:24:07.233 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ipw.ko 2024-03-23T17:24:07.233 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ir-usb.ko 2024-03-23T17:24:07.233 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-03-23T17:24:07.233 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/keyspan.ko 2024-03-23T17:24:07.234 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/keyspan_pda.ko 2024-03-23T17:24:07.234 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/kl5kusb105.ko 2024-03-23T17:24:07.234 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/kobil_sct.ko 2024-03-23T17:24:07.234 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/mct_u232.ko 2024-03-23T17:24:07.234 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/mos7720.ko 2024-03-23T17:24:07.234 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/mos7840.ko 2024-03-23T17:24:07.234 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/navman.ko 2024-03-23T17:24:07.234 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/omninet.ko 2024-03-23T17:24:07.234 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/opticon.ko 2024-03-23T17:24:07.234 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/option.ko 2024-03-23T17:24:07.234 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/oti6858.ko 2024-03-23T17:24:07.234 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/pl2303.ko 2024-03-23T17:24:07.234 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/qcaux.ko 2024-03-23T17:24:07.234 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/qcserial.ko 2024-03-23T17:24:07.235 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/quatech2.ko 2024-03-23T17:24:07.235 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/safe_serial.ko 2024-03-23T17:24:07.235 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/sierra.ko 2024-03-23T17:24:07.235 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/spcp8x5.ko 2024-03-23T17:24:07.235 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ssu100.ko 2024-03-23T17:24:07.235 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/symbolserial.ko 2024-03-23T17:24:07.235 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-03-23T17:24:07.235 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-03-23T17:24:07.235 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/usb_debug.ko 2024-03-23T17:24:07.235 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/usb_wwan.ko 2024-03-23T17:24:07.235 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/visor.ko 2024-03-23T17:24:07.235 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/whiteheat.ko 2024-03-23T17:24:07.235 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/xsens_mt.ko 2024-03-23T17:24:07.235 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage 2024-03-23T17:24:07.236 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/uas.ko 2024-03-23T17:24:07.236 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-alauda.ko 2024-03-23T17:24:07.236 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-cypress.ko 2024-03-23T17:24:07.236 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-datafab.ko 2024-03-23T17:24:07.236 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-03-23T17:24:07.236 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-freecom.ko 2024-03-23T17:24:07.236 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-isd200.ko 2024-03-23T17:24:07.236 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-03-23T17:24:07.236 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-karma.ko 2024-03-23T17:24:07.236 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-onetouch.ko 2024-03-23T17:24:07.236 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-realtek.ko 2024-03-23T17:24:07.236 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-sddr09.ko 2024-03-23T17:24:07.236 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-sddr55.ko 2024-03-23T17:24:07.236 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-usbat.ko 2024-03-23T17:24:07.236 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/usb-storage.ko 2024-03-23T17:24:07.237 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip 2024-03-23T17:24:07.237 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip/usbip-core.ko 2024-03-23T17:24:07.237 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip/usbip-host.ko 2024-03-23T17:24:07.237 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-03-23T17:24:07.237 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio 2024-03-23T17:24:07.237 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/pci 2024-03-23T17:24:07.237 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-03-23T17:24:07.237 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/pci/vfio-pci.ko 2024-03-23T17:24:07.237 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/vfio.ko 2024-03-23T17:24:07.237 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-03-23T17:24:07.237 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost 2024-03-23T17:24:07.237 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost.ko 2024-03-23T17:24:07.237 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost_iotlb.ko 2024-03-23T17:24:07.237 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost_net.ko 2024-03-23T17:24:07.238 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost_scsi.ko 2024-03-23T17:24:07.238 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video 2024-03-23T17:24:07.238 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight 2024-03-23T17:24:07.238 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/apple_bl.ko 2024-03-23T17:24:07.238 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/lcd.ko 2024-03-23T17:24:07.238 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/platform_lcd.ko 2024-03-23T17:24:07.238 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio 2024-03-23T17:24:07.238 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio.ko 2024-03-23T17:24:07.238 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_balloon.ko 2024-03-23T17:24:07.238 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_mmio.ko 2024-03-23T17:24:07.238 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_pci.ko 2024-03-23T17:24:07.238 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-03-23T17:24:07.238 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-03-23T17:24:07.238 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_ring.ko 2024-03-23T17:24:07.238 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1 2024-03-23T17:24:07.239 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/masters 2024-03-23T17:24:07.239 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/masters/ds2482.ko 2024-03-23T17:24:07.239 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/masters/ds2490.ko 2024-03-23T17:24:07.239 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves 2024-03-23T17:24:07.239 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-03-23T17:24:07.239 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-03-23T17:24:07.239 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-03-23T17:24:07.239 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-03-23T17:24:07.239 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-03-23T17:24:07.239 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-03-23T17:24:07.239 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-03-23T17:24:07.239 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-03-23T17:24:07.239 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-03-23T17:24:07.239 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_smem.ko 2024-03-23T17:24:07.239 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_therm.ko 2024-03-23T17:24:07.240 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/wire.ko 2024-03-23T17:24:07.240 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog 2024-03-23T17:24:07.240 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/alim1535_wdt.ko 2024-03-23T17:24:07.240 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/alim7101_wdt.ko 2024-03-23T17:24:07.240 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/f71808e_wdt.ko 2024-03-23T17:24:07.240 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/hpwdt.ko 2024-03-23T17:24:07.240 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/i6300esb.ko 2024-03-23T17:24:07.240 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-03-23T17:24:07.240 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/iTCO_wdt.ko 2024-03-23T17:24:07.240 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/ib700wdt.ko 2024-03-23T17:24:07.240 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/ibmasr.ko 2024-03-23T17:24:07.240 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-03-23T17:24:07.240 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/it8712f_wdt.ko 2024-03-23T17:24:07.241 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/it87_wdt.ko 2024-03-23T17:24:07.241 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/machzwd.ko 2024-03-23T17:24:07.241 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/nv_tco.ko 2024-03-23T17:24:07.241 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/pcwd_pci.ko 2024-03-23T17:24:07.241 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/pcwd_usb.ko 2024-03-23T17:24:07.241 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-03-23T17:24:07.241 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/sch311x_wdt.ko 2024-03-23T17:24:07.241 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/softdog.ko 2024-03-23T17:24:07.241 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/sp5100_tco.ko 2024-03-23T17:24:07.241 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/via_wdt.ko 2024-03-23T17:24:07.241 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-03-23T17:24:07.241 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/w83877f_wdt.ko 2024-03-23T17:24:07.241 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/w83977f_wdt.ko 2024-03-23T17:24:07.241 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/wdt_pci.ko 2024-03-23T17:24:07.241 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/xen_wdt.ko 2024-03-23T17:24:07.242 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen 2024-03-23T17:24:07.242 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-acpi-processor.ko 2024-03-23T17:24:07.242 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-evtchn.ko 2024-03-23T17:24:07.242 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-gntalloc.ko 2024-03-23T17:24:07.242 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-gntdev.ko 2024-03-23T17:24:07.242 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-pciback 2024-03-23T17:24:07.242 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-03-23T17:24:07.242 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-privcmd.ko 2024-03-23T17:24:07.242 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xenfs 2024-03-23T17:24:07.242 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xenfs/xenfs.ko 2024-03-23T17:24:07.242 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs 2024-03-23T17:24:07.242 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/9p 2024-03-23T17:24:07.242 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/9p/9p.ko 2024-03-23T17:24:07.242 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/affs 2024-03-23T17:24:07.243 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/affs/affs.ko 2024-03-23T17:24:07.243 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/befs 2024-03-23T17:24:07.243 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/befs/befs.ko 2024-03-23T17:24:07.243 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/binfmt_misc.ko 2024-03-23T17:24:07.243 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/btrfs 2024-03-23T17:24:07.243 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/btrfs/btrfs.ko 2024-03-23T17:24:07.243 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cachefiles 2024-03-23T17:24:07.243 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cachefiles/cachefiles.ko 2024-03-23T17:24:07.243 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ceph 2024-03-23T17:24:07.243 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ceph/ceph.ko 2024-03-23T17:24:07.243 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/coda 2024-03-23T17:24:07.243 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/coda/coda.ko 2024-03-23T17:24:07.243 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cramfs 2024-03-23T17:24:07.243 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cramfs/cramfs.ko 2024-03-23T17:24:07.243 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/dlm 2024-03-23T17:24:07.244 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/dlm/dlm.ko 2024-03-23T17:24:07.244 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ecryptfs 2024-03-23T17:24:07.244 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ecryptfs/ecryptfs.ko 2024-03-23T17:24:07.244 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/f2fs 2024-03-23T17:24:07.244 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/f2fs/f2fs.ko 2024-03-23T17:24:07.244 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat 2024-03-23T17:24:07.244 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat/fat.ko 2024-03-23T17:24:07.244 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat/msdos.ko 2024-03-23T17:24:07.244 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat/vfat.ko 2024-03-23T17:24:07.244 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fuse 2024-03-23T17:24:07.244 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fuse/cuse.ko 2024-03-23T17:24:07.244 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fuse/fuse.ko 2024-03-23T17:24:07.244 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/gfs2 2024-03-23T17:24:07.244 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/gfs2/gfs2.ko 2024-03-23T17:24:07.245 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfs 2024-03-23T17:24:07.245 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfs/hfs.ko 2024-03-23T17:24:07.245 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfsplus 2024-03-23T17:24:07.245 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfsplus/hfsplus.ko 2024-03-23T17:24:07.245 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/isofs 2024-03-23T17:24:07.245 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/isofs/isofs.ko 2024-03-23T17:24:07.245 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/jfs 2024-03-23T17:24:07.245 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/jfs/jfs.ko 2024-03-23T17:24:07.245 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/lockd 2024-03-23T17:24:07.245 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/lockd/lockd.ko 2024-03-23T17:24:07.245 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/minix 2024-03-23T17:24:07.245 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/minix/minix.ko 2024-03-23T17:24:07.245 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/netfs 2024-03-23T17:24:07.246 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/netfs/netfs.ko 2024-03-23T17:24:07.246 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs 2024-03-23T17:24:07.246 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/blocklayout 2024-03-23T17:24:07.246 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-03-23T17:24:07.246 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/filelayout 2024-03-23T17:24:07.246 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-03-23T17:24:07.246 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/flexfilelayout 2024-03-23T17:24:07.246 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-03-23T17:24:07.246 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/nfs.ko 2024-03-23T17:24:07.246 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/nfsv3.ko 2024-03-23T17:24:07.246 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/nfsv4.ko 2024-03-23T17:24:07.246 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs_common 2024-03-23T17:24:07.246 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs_common/grace.ko 2024-03-23T17:24:07.246 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs_common/nfs_acl.ko 2024-03-23T17:24:07.247 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfsd 2024-03-23T17:24:07.247 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfsd/nfsd.ko 2024-03-23T17:24:07.247 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nilfs2 2024-03-23T17:24:07.247 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nilfs2/nilfs2.ko 2024-03-23T17:24:07.247 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls 2024-03-23T17:24:07.247 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-celtic.ko 2024-03-23T17:24:07.247 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-centeuro.ko 2024-03-23T17:24:07.247 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-croatian.ko 2024-03-23T17:24:07.247 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-cyrillic.ko 2024-03-23T17:24:07.247 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-gaelic.ko 2024-03-23T17:24:07.247 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-greek.ko 2024-03-23T17:24:07.247 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-iceland.ko 2024-03-23T17:24:07.247 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-inuit.ko 2024-03-23T17:24:07.247 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-roman.ko 2024-03-23T17:24:07.248 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-romanian.ko 2024-03-23T17:24:07.248 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-turkish.ko 2024-03-23T17:24:07.248 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp1250.ko 2024-03-23T17:24:07.248 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp1251.ko 2024-03-23T17:24:07.248 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp1255.ko 2024-03-23T17:24:07.248 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp737.ko 2024-03-23T17:24:07.248 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp775.ko 2024-03-23T17:24:07.248 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp850.ko 2024-03-23T17:24:07.248 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp852.ko 2024-03-23T17:24:07.248 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp855.ko 2024-03-23T17:24:07.248 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp857.ko 2024-03-23T17:24:07.248 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp860.ko 2024-03-23T17:24:07.248 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp861.ko 2024-03-23T17:24:07.249 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp862.ko 2024-03-23T17:24:07.249 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp863.ko 2024-03-23T17:24:07.249 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp864.ko 2024-03-23T17:24:07.249 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp865.ko 2024-03-23T17:24:07.249 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp866.ko 2024-03-23T17:24:07.249 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp869.ko 2024-03-23T17:24:07.249 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp874.ko 2024-03-23T17:24:07.249 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp932.ko 2024-03-23T17:24:07.249 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp936.ko 2024-03-23T17:24:07.249 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp949.ko 2024-03-23T17:24:07.249 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp950.ko 2024-03-23T17:24:07.249 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_euc-jp.ko 2024-03-23T17:24:07.249 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-1.ko 2024-03-23T17:24:07.249 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-13.ko 2024-03-23T17:24:07.249 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-14.ko 2024-03-23T17:24:07.250 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-15.ko 2024-03-23T17:24:07.250 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-2.ko 2024-03-23T17:24:07.250 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-3.ko 2024-03-23T17:24:07.250 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-4.ko 2024-03-23T17:24:07.250 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-5.ko 2024-03-23T17:24:07.250 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-6.ko 2024-03-23T17:24:07.250 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-7.ko 2024-03-23T17:24:07.250 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-9.ko 2024-03-23T17:24:07.250 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_koi8-r.ko 2024-03-23T17:24:07.250 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_koi8-ru.ko 2024-03-23T17:24:07.250 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_koi8-u.ko 2024-03-23T17:24:07.250 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_ucs2_utils.ko 2024-03-23T17:24:07.250 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_utf8.ko 2024-03-23T17:24:07.251 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2 2024-03-23T17:24:07.251 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/cluster 2024-03-23T17:24:07.251 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-03-23T17:24:07.251 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlm 2024-03-23T17:24:07.251 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-03-23T17:24:07.251 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlmfs 2024-03-23T17:24:07.251 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-03-23T17:24:07.251 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2.ko 2024-03-23T17:24:07.251 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-03-23T17:24:07.251 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-03-23T17:24:07.251 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-03-23T17:24:07.251 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/overlayfs 2024-03-23T17:24:07.251 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/overlayfs/overlay.ko 2024-03-23T17:24:07.251 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/pstore 2024-03-23T17:24:07.251 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/pstore/ramoops.ko 2024-03-23T17:24:07.252 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/reiserfs 2024-03-23T17:24:07.252 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/reiserfs/reiserfs.ko 2024-03-23T17:24:07.252 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/romfs 2024-03-23T17:24:07.252 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/romfs/romfs.ko 2024-03-23T17:24:07.252 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb 2024-03-23T17:24:07.252 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/client 2024-03-23T17:24:07.252 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/client/cifs.ko 2024-03-23T17:24:07.252 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/common 2024-03-23T17:24:07.252 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/common/cifs_arc4.ko 2024-03-23T17:24:07.252 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/common/cifs_md4.ko 2024-03-23T17:24:07.252 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/squashfs 2024-03-23T17:24:07.252 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/squashfs/squashfs.ko 2024-03-23T17:24:07.252 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/sysv 2024-03-23T17:24:07.252 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/sysv/sysv.ko 2024-03-23T17:24:07.252 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/udf 2024-03-23T17:24:07.252 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/udf/udf.ko 2024-03-23T17:24:07.253 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ufs 2024-03-23T17:24:07.253 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ufs/ufs.ko 2024-03-23T17:24:07.253 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/xfs 2024-03-23T17:24:07.253 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/xfs/xfs.ko 2024-03-23T17:24:07.253 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel 2024-03-23T17:24:07.253 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/rcu 2024-03-23T17:24:07.253 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/rcu/rcutorture.ko 2024-03-23T17:24:07.253 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/torture.ko 2024-03-23T17:24:07.253 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/trace 2024-03-23T17:24:07.253 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-03-23T17:24:07.253 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib 2024-03-23T17:24:07.253 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/asn1_encoder.ko 2024-03-23T17:24:07.253 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crc-itu-t.ko 2024-03-23T17:24:07.254 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crc8.ko 2024-03-23T17:24:07.254 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crypto 2024-03-23T17:24:07.254 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crypto/libarc4.ko 2024-03-23T17:24:07.254 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crypto/libdes.ko 2024-03-23T17:24:07.254 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/libcrc32c.ko 2024-03-23T17:24:07.254 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lru_cache.ko 2024-03-23T17:24:07.254 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lz4 2024-03-23T17:24:07.254 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lz4/lz4_compress.ko 2024-03-23T17:24:07.254 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lz4/lz4hc_compress.ko 2024-03-23T17:24:07.254 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/math 2024-03-23T17:24:07.254 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/math/cordic.ko 2024-03-23T17:24:07.254 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/raid6 2024-03-23T17:24:07.254 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/raid6/raid6_pq.ko 2024-03-23T17:24:07.254 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/reed_solomon 2024-03-23T17:24:07.255 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/reed_solomon/reed_solomon.ko 2024-03-23T17:24:07.255 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/ts_bm.ko 2024-03-23T17:24:07.255 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/ts_fsm.ko 2024-03-23T17:24:07.255 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/ts_kmp.ko 2024-03-23T17:24:07.255 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/zstd 2024-03-23T17:24:07.255 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/zstd/zstd_compress.ko 2024-03-23T17:24:07.255 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/mm 2024-03-23T17:24:07.255 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/mm/hwpoison-inject.ko 2024-03-23T17:24:07.255 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net 2024-03-23T17:24:07.255 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan 2024-03-23T17:24:07.255 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/6lowpan.ko 2024-03-23T17:24:07.255 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_dest.ko 2024-03-23T17:24:07.255 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_fragment.ko 2024-03-23T17:24:07.256 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_hop.ko 2024-03-23T17:24:07.256 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_ipv6.ko 2024-03-23T17:24:07.256 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_mobility.ko 2024-03-23T17:24:07.256 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_routing.ko 2024-03-23T17:24:07.256 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_udp.ko 2024-03-23T17:24:07.256 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802 2024-03-23T17:24:07.256 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/garp.ko 2024-03-23T17:24:07.256 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/mrp.ko 2024-03-23T17:24:07.256 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/p8022.ko 2024-03-23T17:24:07.256 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/psnap.ko 2024-03-23T17:24:07.256 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/stp.ko 2024-03-23T17:24:07.256 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/8021q 2024-03-23T17:24:07.256 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/8021q/8021q.ko 2024-03-23T17:24:07.257 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p 2024-03-23T17:24:07.257 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet.ko 2024-03-23T17:24:07.257 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet_fd.ko 2024-03-23T17:24:07.257 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet_rdma.ko 2024-03-23T17:24:07.257 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet_virtio.ko 2024-03-23T17:24:07.257 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/batman-adv 2024-03-23T17:24:07.257 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/batman-adv/batman-adv.ko 2024-03-23T17:24:07.257 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge 2024-03-23T17:24:07.257 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/br_netfilter.ko 2024-03-23T17:24:07.257 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/bridge.ko 2024-03-23T17:24:07.257 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter 2024-03-23T17:24:07.257 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-03-23T17:24:07.257 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_among.ko 2024-03-23T17:24:07.257 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_arp.ko 2024-03-23T17:24:07.258 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-03-23T17:24:07.258 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-03-23T17:24:07.258 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_ip.ko 2024-03-23T17:24:07.258 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-03-23T17:24:07.258 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_limit.ko 2024-03-23T17:24:07.258 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_log.ko 2024-03-23T17:24:07.258 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_mark.ko 2024-03-23T17:24:07.258 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-03-23T17:24:07.258 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-03-23T17:24:07.258 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-03-23T17:24:07.258 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-03-23T17:24:07.258 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_snat.ko 2024-03-23T17:24:07.258 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_stp.ko 2024-03-23T17:24:07.258 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-03-23T17:24:07.259 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-03-23T17:24:07.259 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-03-23T17:24:07.259 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-03-23T17:24:07.259 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtables.ko 2024-03-23T17:24:07.259 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-03-23T17:24:07.259 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-03-23T17:24:07.259 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-03-23T17:24:07.259 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ceph 2024-03-23T17:24:07.259 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ceph/libceph.ko 2024-03-23T17:24:07.259 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/core 2024-03-23T17:24:07.259 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/core/failover.ko 2024-03-23T17:24:07.259 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/core/pktgen.ko 2024-03-23T17:24:07.259 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp 2024-03-23T17:24:07.259 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp.ko 2024-03-23T17:24:07.259 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp_diag.ko 2024-03-23T17:24:07.259 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp_ipv4.ko 2024-03-23T17:24:07.260 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp_ipv6.ko 2024-03-23T17:24:07.260 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dns_resolver 2024-03-23T17:24:07.260 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dns_resolver/dns_resolver.ko 2024-03-23T17:24:07.260 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa 2024-03-23T17:24:07.260 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/dsa_core.ko 2024-03-23T17:24:07.260 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_brcm.ko 2024-03-23T17:24:07.260 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_dsa.ko 2024-03-23T17:24:07.260 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_none.ko 2024-03-23T17:24:07.260 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_trailer.ko 2024-03-23T17:24:07.260 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154 2024-03-23T17:24:07.260 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/6lowpan 2024-03-23T17:24:07.260 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-03-23T17:24:07.260 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/ieee802154.ko 2024-03-23T17:24:07.260 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/ieee802154_socket.ko 2024-03-23T17:24:07.261 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4 2024-03-23T17:24:07.261 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ah4.ko 2024-03-23T17:24:07.261 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/esp4.ko 2024-03-23T17:24:07.261 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/fou.ko 2024-03-23T17:24:07.261 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/gre.ko 2024-03-23T17:24:07.261 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/inet_diag.ko 2024-03-23T17:24:07.261 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ip_gre.ko 2024-03-23T17:24:07.261 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ip_tunnel.ko 2024-03-23T17:24:07.261 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ip_vti.ko 2024-03-23T17:24:07.261 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ipcomp.ko 2024-03-23T17:24:07.261 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ipip.ko 2024-03-23T17:24:07.261 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter 2024-03-23T17:24:07.262 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/arp_tables.ko 2024-03-23T17:24:07.262 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-03-23T17:24:07.262 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-03-23T17:24:07.262 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-03-23T17:24:07.262 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-03-23T17:24:07.262 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-03-23T17:24:07.262 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-03-23T17:24:07.262 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-03-23T17:24:07.262 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-03-23T17:24:07.262 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-03-23T17:24:07.262 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_security.ko 2024-03-23T17:24:07.262 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-03-23T17:24:07.262 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-03-23T17:24:07.263 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-03-23T17:24:07.263 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-03-23T17:24:07.263 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-03-23T17:24:07.263 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-03-23T17:24:07.263 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-03-23T17:24:07.263 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-03-23T17:24:07.263 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-03-23T17:24:07.263 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-03-23T17:24:07.263 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_bic.ko 2024-03-23T17:24:07.263 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_dctcp.ko 2024-03-23T17:24:07.263 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_diag.ko 2024-03-23T17:24:07.263 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_highspeed.ko 2024-03-23T17:24:07.263 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_htcp.ko 2024-03-23T17:24:07.263 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_hybla.ko 2024-03-23T17:24:07.264 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_illinois.ko 2024-03-23T17:24:07.264 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_lp.ko 2024-03-23T17:24:07.264 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_scalable.ko 2024-03-23T17:24:07.264 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_vegas.ko 2024-03-23T17:24:07.264 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_veno.ko 2024-03-23T17:24:07.264 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_westwood.ko 2024-03-23T17:24:07.264 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_yeah.ko 2024-03-23T17:24:07.264 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tunnel4.ko 2024-03-23T17:24:07.264 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/udp_diag.ko 2024-03-23T17:24:07.264 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/udp_tunnel.ko 2024-03-23T17:24:07.264 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/xfrm4_tunnel.ko 2024-03-23T17:24:07.264 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6 2024-03-23T17:24:07.264 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ah6.ko 2024-03-23T17:24:07.265 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/esp6.ko 2024-03-23T17:24:07.265 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/fou6.ko 2024-03-23T17:24:07.265 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ip6_tunnel.ko 2024-03-23T17:24:07.265 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-03-23T17:24:07.265 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ip6_vti.ko 2024-03-23T17:24:07.265 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ipcomp6.ko 2024-03-23T17:24:07.265 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter 2024-03-23T17:24:07.265 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-03-23T17:24:07.265 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-03-23T17:24:07.265 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-03-23T17:24:07.265 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-03-23T17:24:07.265 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-03-23T17:24:07.265 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-03-23T17:24:07.266 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-03-23T17:24:07.266 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-03-23T17:24:07.266 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-03-23T17:24:07.266 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-03-23T17:24:07.266 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-03-23T17:24:07.266 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-03-23T17:24:07.266 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-03-23T17:24:07.266 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-03-23T17:24:07.266 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-03-23T17:24:07.266 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-03-23T17:24:07.266 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-03-23T17:24:07.266 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-03-23T17:24:07.267 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-03-23T17:24:07.267 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-03-23T17:24:07.267 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-03-23T17:24:07.267 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-03-23T17:24:07.267 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-03-23T17:24:07.267 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-03-23T17:24:07.267 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/sit.ko 2024-03-23T17:24:07.267 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/tunnel6.ko 2024-03-23T17:24:07.267 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/xfrm6_tunnel.ko 2024-03-23T17:24:07.267 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/key 2024-03-23T17:24:07.267 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/key/af_key.ko 2024-03-23T17:24:07.267 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp 2024-03-23T17:24:07.267 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_core.ko 2024-03-23T17:24:07.267 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_debugfs.ko 2024-03-23T17:24:07.268 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_eth.ko 2024-03-23T17:24:07.268 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_ip.ko 2024-03-23T17:24:07.268 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_ip6.ko 2024-03-23T17:24:07.268 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_netlink.ko 2024-03-23T17:24:07.268 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_ppp.ko 2024-03-23T17:24:07.268 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/llc 2024-03-23T17:24:07.268 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/llc/llc.ko 2024-03-23T17:24:07.268 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mac802154 2024-03-23T17:24:07.268 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mac802154/mac802154.ko 2024-03-23T17:24:07.268 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mpls 2024-03-23T17:24:07.268 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mpls/mpls_gso.ko 2024-03-23T17:24:07.268 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter 2024-03-23T17:24:07.268 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset 2024-03-23T17:24:07.269 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set.ko 2024-03-23T17:24:07.269 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-03-23T17:24:07.269 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-03-23T17:24:07.269 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-03-23T17:24:07.269 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-03-23T17:24:07.269 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-03-23T17:24:07.269 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-03-23T17:24:07.269 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-03-23T17:24:07.269 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-03-23T17:24:07.269 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-03-23T17:24:07.269 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-03-23T17:24:07.269 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-03-23T17:24:07.269 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-03-23T17:24:07.270 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-03-23T17:24:07.270 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-03-23T17:24:07.270 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-03-23T17:24:07.270 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs 2024-03-23T17:24:07.270 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs.ko 2024-03-23T17:24:07.270 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-03-23T17:24:07.270 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-03-23T17:24:07.270 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-03-23T17:24:07.270 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-03-23T17:24:07.270 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-03-23T17:24:07.270 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-03-23T17:24:07.270 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-03-23T17:24:07.270 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-03-23T17:24:07.271 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-03-23T17:24:07.271 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-03-23T17:24:07.271 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-03-23T17:24:07.271 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-03-23T17:24:07.271 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-03-23T17:24:07.271 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conncount.ko 2024-03-23T17:24:07.271 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack.ko 2024-03-23T17:24:07.271 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-03-23T17:24:07.271 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-03-23T17:24:07.271 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-03-23T17:24:07.271 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_h323.ko 2024-03-23T17:24:07.271 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_irc.ko 2024-03-23T17:24:07.271 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-03-23T17:24:07.272 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-03-23T17:24:07.272 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-03-23T17:24:07.272 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_sane.ko 2024-03-23T17:24:07.272 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_sip.ko 2024-03-23T17:24:07.272 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-03-23T17:24:07.272 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-03-23T17:24:07.272 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_dup_netdev.ko 2024-03-23T17:24:07.272 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_flow_table.ko 2024-03-23T17:24:07.272 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_flow_table_inet.ko 2024-03-23T17:24:07.272 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_log_syslog.ko 2024-03-23T17:24:07.272 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat.ko 2024-03-23T17:24:07.272 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_amanda.ko 2024-03-23T17:24:07.272 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_ftp.ko 2024-03-23T17:24:07.273 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_irc.ko 2024-03-23T17:24:07.273 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_sip.ko 2024-03-23T17:24:07.273 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_tftp.ko 2024-03-23T17:24:07.273 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_synproxy_core.ko 2024-03-23T17:24:07.273 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_tables.ko 2024-03-23T17:24:07.273 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink.ko 2024-03-23T17:24:07.273 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_acct.ko 2024-03-23T17:24:07.273 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_log.ko 2024-03-23T17:24:07.273 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_osf.ko 2024-03-23T17:24:07.273 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_queue.ko 2024-03-23T17:24:07.273 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_chain_nat.ko 2024-03-23T17:24:07.273 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_compat.ko 2024-03-23T17:24:07.274 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_ct.ko 2024-03-23T17:24:07.274 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_dup_netdev.ko 2024-03-23T17:24:07.274 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_fib.ko 2024-03-23T17:24:07.274 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_fwd_netdev.ko 2024-03-23T17:24:07.274 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_hash.ko 2024-03-23T17:24:07.274 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_limit.ko 2024-03-23T17:24:07.274 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_log.ko 2024-03-23T17:24:07.274 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_masq.ko 2024-03-23T17:24:07.274 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_nat.ko 2024-03-23T17:24:07.274 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_numgen.ko 2024-03-23T17:24:07.274 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_queue.ko 2024-03-23T17:24:07.274 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_quota.ko 2024-03-23T17:24:07.274 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_redir.ko 2024-03-23T17:24:07.274 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_reject.ko 2024-03-23T17:24:07.275 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_reject_inet.ko 2024-03-23T17:24:07.275 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_socket.ko 2024-03-23T17:24:07.275 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_tunnel.ko 2024-03-23T17:24:07.275 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_AUDIT.ko 2024-03-23T17:24:07.275 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CHECKSUM.ko 2024-03-23T17:24:07.275 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CLASSIFY.ko 2024-03-23T17:24:07.275 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-03-23T17:24:07.275 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CT.ko 2024-03-23T17:24:07.275 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_DSCP.ko 2024-03-23T17:24:07.275 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_HL.ko 2024-03-23T17:24:07.275 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_HMARK.ko 2024-03-23T17:24:07.275 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_IDLETIMER.ko 2024-03-23T17:24:07.276 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_LED.ko 2024-03-23T17:24:07.276 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_LOG.ko 2024-03-23T17:24:07.276 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_MASQUERADE.ko 2024-03-23T17:24:07.276 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_NETMAP.ko 2024-03-23T17:24:07.276 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_NFLOG.ko 2024-03-23T17:24:07.276 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_NFQUEUE.ko 2024-03-23T17:24:07.276 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_RATEEST.ko 2024-03-23T17:24:07.276 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_REDIRECT.ko 2024-03-23T17:24:07.276 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_SECMARK.ko 2024-03-23T17:24:07.276 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TCPMSS.ko 2024-03-23T17:24:07.276 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-03-23T17:24:07.276 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TEE.ko 2024-03-23T17:24:07.276 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TPROXY.ko 2024-03-23T17:24:07.276 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TRACE.ko 2024-03-23T17:24:07.276 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_addrtype.ko 2024-03-23T17:24:07.277 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_bpf.ko 2024-03-23T17:24:07.277 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_cgroup.ko 2024-03-23T17:24:07.277 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_cluster.ko 2024-03-23T17:24:07.277 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_comment.ko 2024-03-23T17:24:07.277 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connbytes.ko 2024-03-23T17:24:07.277 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connlabel.ko 2024-03-23T17:24:07.277 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connlimit.ko 2024-03-23T17:24:07.277 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connmark.ko 2024-03-23T17:24:07.277 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_conntrack.ko 2024-03-23T17:24:07.277 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_cpu.ko 2024-03-23T17:24:07.277 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_dccp.ko 2024-03-23T17:24:07.277 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_devgroup.ko 2024-03-23T17:24:07.277 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_dscp.ko 2024-03-23T17:24:07.278 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_ecn.ko 2024-03-23T17:24:07.278 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_esp.ko 2024-03-23T17:24:07.278 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_hashlimit.ko 2024-03-23T17:24:07.278 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_helper.ko 2024-03-23T17:24:07.278 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_hl.ko 2024-03-23T17:24:07.278 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_ipcomp.ko 2024-03-23T17:24:07.278 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_iprange.ko 2024-03-23T17:24:07.278 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_ipvs.ko 2024-03-23T17:24:07.278 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_l2tp.ko 2024-03-23T17:24:07.278 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_length.ko 2024-03-23T17:24:07.278 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_limit.ko 2024-03-23T17:24:07.278 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_mac.ko 2024-03-23T17:24:07.278 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_mark.ko 2024-03-23T17:24:07.278 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_multiport.ko 2024-03-23T17:24:07.279 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_nat.ko 2024-03-23T17:24:07.279 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_nfacct.ko 2024-03-23T17:24:07.279 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_osf.ko 2024-03-23T17:24:07.279 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_owner.ko 2024-03-23T17:24:07.279 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_physdev.ko 2024-03-23T17:24:07.279 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_pkttype.ko 2024-03-23T17:24:07.279 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_policy.ko 2024-03-23T17:24:07.279 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_quota.ko 2024-03-23T17:24:07.279 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_rateest.ko 2024-03-23T17:24:07.279 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_realm.ko 2024-03-23T17:24:07.279 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_recent.ko 2024-03-23T17:24:07.279 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_sctp.ko 2024-03-23T17:24:07.279 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_set.ko 2024-03-23T17:24:07.280 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_socket.ko 2024-03-23T17:24:07.280 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_state.ko 2024-03-23T17:24:07.280 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_statistic.ko 2024-03-23T17:24:07.280 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_string.ko 2024-03-23T17:24:07.280 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_tcpmss.ko 2024-03-23T17:24:07.280 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_time.ko 2024-03-23T17:24:07.280 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_u32.ko 2024-03-23T17:24:07.280 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netlink 2024-03-23T17:24:07.280 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netlink/netlink_diag.ko 2024-03-23T17:24:07.280 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/nsh 2024-03-23T17:24:07.280 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/nsh/nsh.ko 2024-03-23T17:24:07.280 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch 2024-03-23T17:24:07.280 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/openvswitch.ko 2024-03-23T17:24:07.280 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/vport-geneve.ko 2024-03-23T17:24:07.281 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/vport-gre.ko 2024-03-23T17:24:07.281 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/vport-vxlan.ko 2024-03-23T17:24:07.281 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/packet 2024-03-23T17:24:07.281 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/packet/af_packet_diag.ko 2024-03-23T17:24:07.281 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds 2024-03-23T17:24:07.281 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds/rds.ko 2024-03-23T17:24:07.281 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds/rds_rdma.ko 2024-03-23T17:24:07.281 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds/rds_tcp.ko 2024-03-23T17:24:07.281 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched 2024-03-23T17:24:07.281 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_csum.ko 2024-03-23T17:24:07.281 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_gact.ko 2024-03-23T17:24:07.281 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_mirred.ko 2024-03-23T17:24:07.281 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_nat.ko 2024-03-23T17:24:07.281 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_pedit.ko 2024-03-23T17:24:07.281 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_police.ko 2024-03-23T17:24:07.282 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_simple.ko 2024-03-23T17:24:07.282 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_skbedit.ko 2024-03-23T17:24:07.282 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_vlan.ko 2024-03-23T17:24:07.282 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_basic.ko 2024-03-23T17:24:07.282 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_bpf.ko 2024-03-23T17:24:07.282 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_flow.ko 2024-03-23T17:24:07.282 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_fw.ko 2024-03-23T17:24:07.282 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_route.ko 2024-03-23T17:24:07.282 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_u32.ko 2024-03-23T17:24:07.282 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_cmp.ko 2024-03-23T17:24:07.282 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_ipset.ko 2024-03-23T17:24:07.282 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_meta.ko 2024-03-23T17:24:07.282 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_nbyte.ko 2024-03-23T17:24:07.282 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_text.ko 2024-03-23T17:24:07.283 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_u32.ko 2024-03-23T17:24:07.283 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_choke.ko 2024-03-23T17:24:07.283 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_codel.ko 2024-03-23T17:24:07.283 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_drr.ko 2024-03-23T17:24:07.283 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_fq.ko 2024-03-23T17:24:07.283 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_gred.ko 2024-03-23T17:24:07.283 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_hfsc.ko 2024-03-23T17:24:07.283 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_hhf.ko 2024-03-23T17:24:07.283 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_htb.ko 2024-03-23T17:24:07.283 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_ingress.ko 2024-03-23T17:24:07.283 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_mqprio.ko 2024-03-23T17:24:07.283 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_mqprio_lib.ko 2024-03-23T17:24:07.283 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_multiq.ko 2024-03-23T17:24:07.283 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_netem.ko 2024-03-23T17:24:07.283 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_pie.ko 2024-03-23T17:24:07.284 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_plug.ko 2024-03-23T17:24:07.284 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_prio.ko 2024-03-23T17:24:07.284 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_qfq.ko 2024-03-23T17:24:07.284 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_red.ko 2024-03-23T17:24:07.284 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_sfb.ko 2024-03-23T17:24:07.284 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_sfq.ko 2024-03-23T17:24:07.284 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_tbf.ko 2024-03-23T17:24:07.284 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_teql.ko 2024-03-23T17:24:07.284 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sctp 2024-03-23T17:24:07.284 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sctp/sctp.ko 2024-03-23T17:24:07.284 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sctp/sctp_diag.ko 2024-03-23T17:24:07.284 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc 2024-03-23T17:24:07.284 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/auth_gss 2024-03-23T17:24:07.284 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-03-23T17:24:07.284 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-03-23T17:24:07.284 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/sunrpc.ko 2024-03-23T17:24:07.285 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/xprtrdma 2024-03-23T17:24:07.285 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-03-23T17:24:07.285 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/tipc 2024-03-23T17:24:07.285 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/tipc/diag.ko 2024-03-23T17:24:07.285 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/tipc/tipc.ko 2024-03-23T17:24:07.285 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/unix 2024-03-23T17:24:07.285 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/unix/unix_diag.ko 2024-03-23T17:24:07.285 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock 2024-03-23T17:24:07.285 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-03-23T17:24:07.285 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-03-23T17:24:07.285 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vsock.ko 2024-03-23T17:24:07.285 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vsock_diag.ko 2024-03-23T17:24:07.285 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vsock_loopback.ko 2024-03-23T17:24:07.285 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/xfrm 2024-03-23T17:24:07.286 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/xfrm/xfrm_ipcomp.ko 2024-03-23T17:24:07.286 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security 2024-03-23T17:24:07.286 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys 2024-03-23T17:24:07.286 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys/encrypted-keys 2024-03-23T17:24:07.286 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-03-23T17:24:07.286 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys/trusted-keys 2024-03-23T17:24:07.286 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys/trusted-keys/trusted.ko 2024-03-23T17:24:07.286 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/virt 2024-03-23T17:24:07.286 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/virt/lib 2024-03-23T17:24:07.286 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/virt/lib/irqbypass.ko 2024-03-23T17:24:07.286 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.alias 2024-03-23T17:24:07.286 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.alias.bin 2024-03-23T17:24:07.286 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin 2024-03-23T17:24:07.286 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin.bin 2024-03-23T17:24:07.287 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin.modinfo 2024-03-23T17:24:07.287 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.dep 2024-03-23T17:24:07.287 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.dep.bin 2024-03-23T17:24:07.287 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.devname 2024-03-23T17:24:07.287 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.order 2024-03-23T17:24:07.287 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.softdep 2024-03-23T17:24:07.287 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.symbols 2024-03-23T17:24:07.287 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.symbols.bin 2024-03-23T17:24:07.287 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/vmlinuz 2024-03-23T17:24:07.292 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-g5ec3e6e88b2e 2024-03-23T17:24:07.292 DEBUG:teuthology.orchestra.run.smithi086:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-03-23T17:24:07.316 INFO:teuthology.orchestra.run.smithi086.stdout:postinstall scriptlet (using /bin/sh): 2024-03-23T17:24:07.316 INFO:teuthology.orchestra.run.smithi086.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-03-23T17:24:07.316 INFO:teuthology.orchestra.run.smithi086.stdout: /usr/bin/kernel-install add 6.8.0-g5ec3e6e88b2e /lib/modules/6.8.0-g5ec3e6e88b2e/vmlinuz 2024-03-23T17:24:07.316 INFO:teuthology.orchestra.run.smithi086.stdout:fi 2024-03-23T17:24:07.316 INFO:teuthology.orchestra.run.smithi086.stdout:for file in vmlinuz System.map config; do 2024-03-23T17:24:07.316 INFO:teuthology.orchestra.run.smithi086.stdout: if ! cmp --silent "/lib/modules/6.8.0-g5ec3e6e88b2e/${file}" "/boot/${file}-6.8.0-g5ec3e6e88b2e"; then 2024-03-23T17:24:07.316 INFO:teuthology.orchestra.run.smithi086.stdout: cp "/lib/modules/6.8.0-g5ec3e6e88b2e/${file}" "/boot/${file}-6.8.0-g5ec3e6e88b2e" 2024-03-23T17:24:07.316 INFO:teuthology.orchestra.run.smithi086.stdout: fi 2024-03-23T17:24:07.316 INFO:teuthology.orchestra.run.smithi086.stdout:done 2024-03-23T17:24:07.316 INFO:teuthology.orchestra.run.smithi086.stdout:preuninstall scriptlet (using /bin/sh): 2024-03-23T17:24:07.316 INFO:teuthology.orchestra.run.smithi086.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-03-23T17:24:07.316 INFO:teuthology.orchestra.run.smithi086.stdout:new-kernel-pkg --remove 6.8.0-g5ec3e6e88b2e --rminitrd --initrdfile=/boot/initramfs-6.8.0-g5ec3e6e88b2e.img 2024-03-23T17:24:07.317 INFO:teuthology.orchestra.run.smithi086.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-03-23T17:24:07.317 INFO:teuthology.orchestra.run.smithi086.stdout:kernel-install remove 6.8.0-g5ec3e6e88b2e 2024-03-23T17:24:07.317 INFO:teuthology.orchestra.run.smithi086.stdout:fi 2024-03-23T17:24:07.317 INFO:teuthology.orchestra.run.smithi086.stdout:postuninstall scriptlet (using /bin/sh): 2024-03-23T17:24:07.317 INFO:teuthology.orchestra.run.smithi086.stdout:if [ -x /sbin/update-bootloader ]; then 2024-03-23T17:24:07.317 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/update-bootloader --remove 6.8.0-g5ec3e6e88b2e 2024-03-23T17:24:07.317 INFO:teuthology.orchestra.run.smithi086.stdout:fi 2024-03-23T17:24:07.318 DEBUG:teuthology.orchestra.run.smithi086:> sudo rpm -qi grub2-tools 2024-03-23T17:24:07.391 INFO:teuthology.orchestra.run.smithi086.stdout:Name : grub2-tools 2024-03-23T17:24:07.391 INFO:teuthology.orchestra.run.smithi086.stdout:Epoch : 1 2024-03-23T17:24:07.391 INFO:teuthology.orchestra.run.smithi086.stdout:Version : 2.02 2024-03-23T17:24:07.391 INFO:teuthology.orchestra.run.smithi086.stdout:Release : 123.el8 2024-03-23T17:24:07.392 INFO:teuthology.orchestra.run.smithi086.stdout:Architecture: x86_64 2024-03-23T17:24:07.392 INFO:teuthology.orchestra.run.smithi086.stdout:Install Date: Thu 14 Dec 2023 12:39:42 AM UTC 2024-03-23T17:24:07.392 INFO:teuthology.orchestra.run.smithi086.stdout:Group : System Environment/Base 2024-03-23T17:24:07.392 INFO:teuthology.orchestra.run.smithi086.stdout:Size : 9265450 2024-03-23T17:24:07.392 INFO:teuthology.orchestra.run.smithi086.stdout:License : GPLv3+ 2024-03-23T17:24:07.392 INFO:teuthology.orchestra.run.smithi086.stdout:Signature : RSA/SHA256, Thu 31 Mar 2022 07:24:35 PM UTC, Key ID 199e2f91fd431d51 2024-03-23T17:24:07.392 INFO:teuthology.orchestra.run.smithi086.stdout:Source RPM : grub2-2.02-123.el8.src.rpm 2024-03-23T17:24:07.392 INFO:teuthology.orchestra.run.smithi086.stdout:Build Date : Mon 28 Mar 2022 02:12:09 PM UTC 2024-03-23T17:24:07.392 INFO:teuthology.orchestra.run.smithi086.stdout:Build Host : x86-vm-09.build.eng.bos.redhat.com 2024-03-23T17:24:07.392 INFO:teuthology.orchestra.run.smithi086.stdout:Relocations : (not relocatable) 2024-03-23T17:24:07.392 INFO:teuthology.orchestra.run.smithi086.stdout:Packager : Red Hat, Inc. 2024-03-23T17:24:07.392 INFO:teuthology.orchestra.run.smithi086.stdout:Vendor : Red Hat, Inc. 2024-03-23T17:24:07.392 INFO:teuthology.orchestra.run.smithi086.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-23T17:24:07.392 INFO:teuthology.orchestra.run.smithi086.stdout:Summary : Support tools for GRUB. 2024-03-23T17:24:07.393 INFO:teuthology.orchestra.run.smithi086.stdout:Description : 2024-03-23T17:24:07.393 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:24:07.393 INFO:teuthology.orchestra.run.smithi086.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-23T17:24:07.393 INFO:teuthology.orchestra.run.smithi086.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-23T17:24:07.393 INFO:teuthology.orchestra.run.smithi086.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-23T17:24:07.393 INFO:teuthology.orchestra.run.smithi086.stdout:hardware devices. 2024-03-23T17:24:07.393 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:24:07.393 INFO:teuthology.orchestra.run.smithi086.stdout:This subpackage provides tools for support of all platforms. 2024-03-23T17:24:07.394 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-23T17:24:07.394 INFO:teuthology.task.kernel:Updating grub on smithi086 to boot 6.8.0-g5ec3e6e88b2e 2024-03-23T17:24:07.394 DEBUG:teuthology.orchestra.run.smithi086:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-23T17:24:08.177 INFO:teuthology.orchestra.run.smithi086.stderr:Generating grub configuration file ... 2024-03-23T17:24:10.517 INFO:teuthology.orchestra.run.smithi086.stderr:done 2024-03-23T17:24:10.519 DEBUG:teuthology.orchestra.run.smithi086:> mktemp 2024-03-23T17:24:10.576 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/tmp.JcfuMdqHxj 2024-03-23T17:24:10.576 DEBUG:teuthology.orchestra.run.smithi086:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.JcfuMdqHxj 2024-03-23T17:24:10.654 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 0666 /tmp/tmp.JcfuMdqHxj 2024-03-23T17:24:10.808 DEBUG:teuthology.orchestra.remote:smithi086:/tmp/tmp.JcfuMdqHxj is 6KB 2024-03-23T17:24:10.856 DEBUG:teuthology.orchestra.run.smithi086:> rm -fr /tmp/tmp.JcfuMdqHxj 2024-03-23T17:24:10.872 DEBUG:teuthology.orchestra.run.smithi086:> sudo /bin/ls /boot/loader/entries || true 2024-03-23T17:24:10.949 INFO:teuthology.orchestra.run.smithi086.stdout:2db68f91c8f90721a6596efe657a5324-0-rescue.conf 2024-03-23T17:24:10.950 INFO:teuthology.orchestra.run.smithi086.stdout:2db68f91c8f90721a6596efe657a5324-6.8.0-g5ec3e6e88b2e.conf 2024-03-23T17:24:10.950 INFO:teuthology.orchestra.run.smithi086.stdout:963f8b6c99184de696879b3f7da9ecce-0-rescue.conf 2024-03-23T17:24:10.950 INFO:teuthology.orchestra.run.smithi086.stdout:963f8b6c99184de696879b3f7da9ecce-4.18.0-372.9.1.el8.x86_64.conf 2024-03-23T17:24:10.950 INFO:teuthology.orchestra.run.smithi086.stdout:963f8b6c99184de696879b3f7da9ecce-4.18.0-513.9.1.el8_9.x86_64.conf 2024-03-23T17:24:10.951 DEBUG:teuthology.orchestra.run.smithi086:> sudo grub2-set-default 2db68f91c8f90721a6596efe657a5324-6.8.0-g5ec3e6e88b2e 2024-03-23T17:24:11.146 DEBUG:teuthology.orchestra.run.smithi086:> sudo shutdown -r now 2024-03-23T17:24:41.149 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-03-23T17:24:41.149 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-03-23T17:24:41.150 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T17:25:41.152 DEBUG:teuthology.orchestra.remote:timed out 2024-03-23T17:25:50.153 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-03-23T17:25:50.154 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T17:25:53.214 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.86 2024-03-23T17:26:05.215 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-03-23T17:26:05.216 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T17:26:05.596 DEBUG:teuthology.orchestra.run.smithi086:> true 2024-03-23T17:26:06.031 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-03-23T17:26:06.031 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.8.0-g5ec3e6e88b2e"... 2024-03-23T17:26:06.031 DEBUG:teuthology.orchestra.run.smithi086:> uname -r 2024-03-23T17:26:06.071 INFO:teuthology.orchestra.run.smithi086.stdout:6.8.0-g5ec3e6e88b2e 2024-03-23T17:26:06.071 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-g5ec3e6e88b2e vs 6.8.0-g5ec3e6e88b2e 2024-03-23T17:26:06.071 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-23T17:26:06.071 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2024-03-23T17:26:07.072 DEBUG:teuthology.parallel:result is None 2024-03-23T17:26:07.073 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-23T17:26:07.080 INFO:teuthology.task.internal:Creating test directory... 2024-03-23T17:26:07.080 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-23T17:26:07.084 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-23T17:26:07.087 DEBUG:teuthology.orchestra.run.smithi152:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-23T17:26:07.125 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-23T17:26:07.131 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-23T17:26:07.139 INFO:teuthology.task.internal:Creating archive directory... 2024-03-23T17:26:07.140 DEBUG:teuthology.orchestra.run.smithi086:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-23T17:26:07.169 DEBUG:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-23T17:26:07.171 DEBUG:teuthology.orchestra.run.smithi152:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-23T17:26:07.226 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-23T17:26:07.233 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-23T17:26:07.233 DEBUG:teuthology.orchestra.run.smithi092:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-23T17:26:07.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T17:26:07.257 DEBUG:teuthology.orchestra.run.smithi152:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-23T17:26:07.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T17:26:07.276 DEBUG:teuthology.orchestra.run.smithi086:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-23T17:26:07.280 DEBUG:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-23T17:26:07.299 DEBUG:teuthology.orchestra.run.smithi152:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-23T17:26:07.358 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-23T17:26:07.372 INFO:teuthology.orchestra.run.smithi152.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-23T17:26:07.373 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-23T17:26:07.390 INFO:teuthology.orchestra.run.smithi152.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-23T17:26:07.401 INFO:teuthology.orchestra.run.smithi086.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-23T17:26:07.430 INFO:teuthology.orchestra.run.smithi086.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-23T17:26:07.434 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-23T17:26:07.441 INFO:teuthology.task.internal:Configuring sudo... 2024-03-23T17:26:07.441 DEBUG:teuthology.orchestra.run.smithi086:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-23T17:26:07.445 DEBUG:teuthology.orchestra.run.smithi092:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-23T17:26:07.447 DEBUG:teuthology.orchestra.run.smithi152:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-23T17:26:07.506 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-23T17:26:07.515 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-23T17:26:07.515 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-23T17:26:07.519 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-23T17:26:07.522 DEBUG:teuthology.orchestra.run.smithi152:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-23T17:26:07.556 DEBUG:teuthology.orchestra.run.smithi086:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-23T17:26:07.668 DEBUG:teuthology.orchestra.run.smithi086:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-23T17:26:07.761 DEBUG:teuthology.orchestra.run.smithi086:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-23T17:26:07.830 DEBUG:teuthology.orchestra.run.smithi086:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-23T17:26:07.911 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-23T17:26:07.972 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-23T17:26:08.036 DEBUG:teuthology.orchestra.run.smithi092:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-23T17:26:08.083 DEBUG:teuthology.orchestra.run.smithi092:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-23T17:26:08.163 DEBUG:teuthology.orchestra.run.smithi092:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-23T17:26:08.208 DEBUG:teuthology.orchestra.run.smithi092:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-23T17:26:08.252 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-03-23T17:26:08.252 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-23T17:26:08.321 DEBUG:teuthology.orchestra.run.smithi152:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-23T17:26:08.365 DEBUG:teuthology.orchestra.run.smithi152:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-23T17:26:08.422 DEBUG:teuthology.orchestra.run.smithi152:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-23T17:26:08.509 DEBUG:teuthology.orchestra.run.smithi152:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-23T17:26:08.554 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-03-23T17:26:08.555 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-23T17:26:08.625 DEBUG:teuthology.orchestra.run.smithi086:> sudo service rsyslog restart 2024-03-23T17:26:08.629 DEBUG:teuthology.orchestra.run.smithi092:> sudo service rsyslog restart 2024-03-23T17:26:08.631 DEBUG:teuthology.orchestra.run.smithi152:> sudo service rsyslog restart 2024-03-23T17:26:08.681 INFO:teuthology.orchestra.run.smithi092.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-23T17:26:08.736 INFO:teuthology.orchestra.run.smithi086.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-23T17:26:08.791 INFO:teuthology.orchestra.run.smithi152.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-23T17:26:09.214 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-23T17:26:09.224 INFO:teuthology.task.internal:Starting timer... 2024-03-23T17:26:09.224 INFO:teuthology.run_tasks:Running task pcp... 2024-03-23T17:26:09.236 INFO:teuthology.run_tasks:Running task selinux... 2024-03-23T17:26:09.245 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-03-23T17:26:09.246 DEBUG:teuthology.orchestra.run.smithi086:> sudo service auditd rotate 2024-03-23T17:26:09.383 INFO:teuthology.orchestra.run.smithi086.stdout:Rotating logs: 2024-03-23T17:26:09.387 DEBUG:teuthology.orchestra.run.smithi092:> sudo service auditd rotate 2024-03-23T17:26:09.498 INFO:teuthology.orchestra.run.smithi092.stdout:Rotating logs: 2024-03-23T17:26:09.499 DEBUG:teuthology.orchestra.run.smithi152:> sudo service auditd rotate 2024-03-23T17:26:09.575 INFO:teuthology.orchestra.run.smithi152.stdout:Rotating logs: 2024-03-23T17:26:09.577 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-23T17:26:09.577 DEBUG:teuthology.orchestra.run.smithi086:> /usr/sbin/getenforce 2024-03-23T17:26:09.622 INFO:teuthology.orchestra.run.smithi086.stdout:Permissive 2024-03-23T17:26:09.622 DEBUG:teuthology.orchestra.run.smithi092:> /usr/sbin/getenforce 2024-03-23T17:26:09.649 INFO:teuthology.orchestra.run.smithi092.stdout:Permissive 2024-03-23T17:26:09.650 DEBUG:teuthology.orchestra.run.smithi152:> /usr/sbin/getenforce 2024-03-23T17:26:09.673 INFO:teuthology.orchestra.run.smithi152.stdout:Permissive 2024-03-23T17:26:09.673 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi086.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi092.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi152.front.sepia.ceph.com': 'permissive'} 2024-03-23T17:26:09.673 DEBUG:teuthology.orchestra.run.smithi086:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-23T17:26:09.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T17:26:09.739 DEBUG:teuthology.orchestra.run.smithi092:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-23T17:26:09.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T17:26:09.769 DEBUG:teuthology.orchestra.run.smithi152:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-23T17:26:09.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T17:26:09.799 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-23T17:26:09.799 DEBUG:teuthology.orchestra.run.smithi086:> sudo /usr/sbin/setenforce permissive 2024-03-23T17:26:09.869 DEBUG:teuthology.orchestra.run.smithi092:> sudo /usr/sbin/setenforce permissive 2024-03-23T17:26:09.897 DEBUG:teuthology.orchestra.run.smithi152:> sudo /usr/sbin/setenforce permissive 2024-03-23T17:26:09.924 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-23T17:26:09.934 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-23T17:26:09.951 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-23T17:26:09.952 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi086.front.sepia.ceph.com,smithi092.front.sepia.ceph.com,smithi152.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-23T17:37:17.699 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi086.front.sepia.ceph.com'), Remote(name='ubuntu@smithi092.front.sepia.ceph.com'), Remote(name='ubuntu@smithi152.front.sepia.ceph.com')] 2024-03-23T17:37:17.700 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-03-23T17:37:17.701 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T17:37:17.788 DEBUG:teuthology.orchestra.run.smithi086:> true 2024-03-23T17:37:17.895 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-03-23T17:37:17.895 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-03-23T17:37:17.896 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T17:37:17.971 DEBUG:teuthology.orchestra.run.smithi092:> true 2024-03-23T17:37:18.045 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-03-23T17:37:18.045 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi152.front.sepia.ceph.com' 2024-03-23T17:37:18.046 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi152.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T17:37:18.128 DEBUG:teuthology.orchestra.run.smithi152:> true 2024-03-23T17:37:18.202 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi152.front.sepia.ceph.com' 2024-03-23T17:37:18.203 INFO:teuthology.run_tasks:Running task clock... 2024-03-23T17:37:18.213 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-23T17:37:18.213 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-23T17:37:18.214 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-23T17:37:18.218 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-23T17:37:18.218 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-23T17:37:18.220 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-23T17:37:18.221 DEBUG:teuthology.orchestra.run.smithi152:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-23T17:37:18.254 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-23T17:37:18.270 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-23T17:37:18.277 INFO:teuthology.orchestra.run.smithi152.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-23T17:37:18.284 INFO:teuthology.orchestra.run.smithi086.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-23T17:37:18.293 INFO:teuthology.orchestra.run.smithi152.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-23T17:37:18.316 INFO:teuthology.orchestra.run.smithi086.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-23T17:37:18.338 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: ntpd: command not found 2024-03-23T17:37:18.351 INFO:teuthology.orchestra.run.smithi092.stdout:506 Cannot talk to daemon 2024-03-23T17:37:18.362 INFO:teuthology.orchestra.run.smithi152.stderr:sudo: ntpd: command not found 2024-03-23T17:37:18.366 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-23T17:37:18.374 INFO:teuthology.orchestra.run.smithi152.stdout:506 Cannot talk to daemon 2024-03-23T17:37:18.381 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-23T17:37:18.388 INFO:teuthology.orchestra.run.smithi152.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-23T17:37:18.403 INFO:teuthology.orchestra.run.smithi152.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-23T17:37:18.405 INFO:teuthology.orchestra.run.smithi086.stderr:sudo: ntpd: command not found 2024-03-23T17:37:18.432 INFO:teuthology.orchestra.run.smithi086.stdout:506 Cannot talk to daemon 2024-03-23T17:37:18.448 INFO:teuthology.orchestra.run.smithi092.stderr:bash: ntpq: command not found 2024-03-23T17:37:18.451 INFO:teuthology.orchestra.run.smithi092.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-23T17:37:18.451 INFO:teuthology.orchestra.run.smithi092.stdout:=============================================================================== 2024-03-23T17:37:18.451 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T17:37:18.452 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T17:37:18.452 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T17:37:18.452 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T17:37:18.464 INFO:teuthology.orchestra.run.smithi086.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-23T17:37:18.471 INFO:teuthology.orchestra.run.smithi152.stderr:bash: ntpq: command not found 2024-03-23T17:37:18.475 INFO:teuthology.orchestra.run.smithi152.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-23T17:37:18.475 INFO:teuthology.orchestra.run.smithi152.stdout:=============================================================================== 2024-03-23T17:37:18.475 INFO:teuthology.orchestra.run.smithi152.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T17:37:18.475 INFO:teuthology.orchestra.run.smithi152.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T17:37:18.475 INFO:teuthology.orchestra.run.smithi152.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T17:37:18.475 INFO:teuthology.orchestra.run.smithi152.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T17:37:18.496 INFO:teuthology.orchestra.run.smithi086.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-23T17:37:18.587 INFO:teuthology.orchestra.run.smithi086.stderr:bash: ntpq: command not found 2024-03-23T17:37:18.594 INFO:teuthology.orchestra.run.smithi086.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-23T17:37:18.594 INFO:teuthology.orchestra.run.smithi086.stdout:=============================================================================== 2024-03-23T17:37:18.594 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T17:37:18.594 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T17:37:18.594 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T17:37:18.594 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T17:37:18.595 INFO:teuthology.run_tasks:Running task pexec... 2024-03-23T17:37:18.605 INFO:teuthology.task.pexec:Executing custom commands... 2024-03-23T17:37:18.606 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi086.front.sepia.ceph.com 2024-03-23T17:37:18.606 DEBUG:teuthology.orchestra.run.smithi086:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-03-23T17:37:18.606 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi092.front.sepia.ceph.com 2024-03-23T17:37:18.607 DEBUG:teuthology.orchestra.run.smithi092:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-03-23T17:37:18.607 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi152.front.sepia.ceph.com 2024-03-23T17:37:18.607 DEBUG:teuthology.orchestra.run.smithi152:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-03-23T17:37:18.984 INFO:teuthology.orchestra.run.smithi092.stdout:Updating Subscription Management repositories. 2024-03-23T17:37:18.988 INFO:teuthology.orchestra.run.smithi152.stdout:Updating Subscription Management repositories. 2024-03-23T17:37:19.086 INFO:teuthology.orchestra.run.smithi086.stdout:Updating Subscription Management repositories. 2024-03-23T17:37:20.923 INFO:teuthology.orchestra.run.smithi092.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 27 kB/s | 4.1 kB 00:00 2024-03-23T17:37:20.953 INFO:teuthology.orchestra.run.smithi152.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 40 kB/s | 4.1 kB 00:00 2024-03-23T17:37:21.011 INFO:teuthology.orchestra.run.smithi086.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 36 kB/s | 4.5 kB 00:00 2024-03-23T17:37:21.310 INFO:teuthology.orchestra.run.smithi092.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 27 kB/s | 4.5 kB 00:00 2024-03-23T17:37:21.328 INFO:teuthology.orchestra.run.smithi152.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 41 kB/s | 4.5 kB 00:00 2024-03-23T17:37:21.674 INFO:teuthology.orchestra.run.smithi086.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 31 kB/s | 4.1 kB 00:00 2024-03-23T17:37:24.394 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-03-23T17:37:24.395 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-03-23T17:37:24.395 INFO:teuthology.orchestra.run.smithi092.stdout: Package Architecture Version Repository Size 2024-03-23T17:37:24.395 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-03-23T17:37:24.395 INFO:teuthology.orchestra.run.smithi092.stdout:Resetting modules: 2024-03-23T17:37:24.395 INFO:teuthology.orchestra.run.smithi092.stdout: container-tools 2024-03-23T17:37:24.395 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-23T17:37:24.395 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-03-23T17:37:24.395 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-03-23T17:37:24.395 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-23T17:37:24.587 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2024-03-23T17:37:24.588 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-03-23T17:37:24.588 INFO:teuthology.orchestra.run.smithi152.stdout: Package Architecture Version Repository Size 2024-03-23T17:37:24.588 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-03-23T17:37:24.589 INFO:teuthology.orchestra.run.smithi152.stdout:Resetting modules: 2024-03-23T17:37:24.589 INFO:teuthology.orchestra.run.smithi152.stdout: container-tools 2024-03-23T17:37:24.589 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-23T17:37:24.589 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2024-03-23T17:37:24.589 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-03-23T17:37:24.589 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-23T17:37:24.719 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-03-23T17:37:24.720 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-03-23T17:37:24.720 INFO:teuthology.orchestra.run.smithi086.stdout: Package Architecture Version Repository Size 2024-03-23T17:37:24.720 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-03-23T17:37:24.720 INFO:teuthology.orchestra.run.smithi086.stdout:Resetting modules: 2024-03-23T17:37:24.720 INFO:teuthology.orchestra.run.smithi086.stdout: container-tools 2024-03-23T17:37:24.720 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:37:24.720 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction Summary 2024-03-23T17:37:24.720 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-03-23T17:37:24.720 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:37:37.107 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-03-23T17:37:37.571 INFO:teuthology.orchestra.run.smithi092.stdout:Updating Subscription Management repositories. 2024-03-23T17:37:38.698 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2024-03-23T17:37:38.893 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-03-23T17:37:39.168 INFO:teuthology.orchestra.run.smithi152.stdout:Updating Subscription Management repositories. 2024-03-23T17:37:39.468 INFO:teuthology.orchestra.run.smithi092.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 30 kB/s | 4.1 kB 00:00 2024-03-23T17:37:39.485 INFO:teuthology.orchestra.run.smithi086.stdout:Updating Subscription Management repositories. 2024-03-23T17:37:39.844 INFO:teuthology.orchestra.run.smithi092.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 31 kB/s | 4.5 kB 00:00 2024-03-23T17:37:41.017 INFO:teuthology.orchestra.run.smithi152.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 33 kB/s | 4.1 kB 00:00 2024-03-23T17:37:41.302 INFO:teuthology.orchestra.run.smithi086.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 30 kB/s | 4.5 kB 00:00 2024-03-23T17:37:41.409 INFO:teuthology.orchestra.run.smithi152.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 39 kB/s | 4.5 kB 00:00 2024-03-23T17:37:41.951 INFO:teuthology.orchestra.run.smithi086.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 32 kB/s | 4.1 kB 00:00 2024-03-23T17:37:42.926 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-03-23T17:37:43.084 INFO:teuthology.orchestra.run.smithi092.stdout:========================================================================================================== 2024-03-23T17:37:43.084 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-03-23T17:37:43.085 INFO:teuthology.orchestra.run.smithi092.stdout:========================================================================================================== 2024-03-23T17:37:43.085 INFO:teuthology.orchestra.run.smithi092.stdout:Installing group/module packages: 2024-03-23T17:37:43.085 INFO:teuthology.orchestra.run.smithi092.stdout: aardvark-dns x86_64 2:1.7.0-1.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 1.0 M 2024-03-23T17:37:43.085 INFO:teuthology.orchestra.run.smithi092.stdout: buildah x86_64 1:1.31.3-3.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 8.8 M 2024-03-23T17:37:43.085 INFO:teuthology.orchestra.run.smithi092.stdout: cockpit-podman noarch 75-1.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 739 k 2024-03-23T17:37:43.085 INFO:teuthology.orchestra.run.smithi092.stdout: crun x86_64 1.8.7-1.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 239 k 2024-03-23T17:37:43.085 INFO:teuthology.orchestra.run.smithi092.stdout: netavark x86_64 2:1.7.0-2.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-03-23T17:37:43.085 INFO:teuthology.orchestra.run.smithi092.stdout: python3-podman noarch 4.6.0-2.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 154 k 2024-03-23T17:37:43.085 INFO:teuthology.orchestra.run.smithi092.stdout: skopeo x86_64 2:1.13.3-3.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 8.1 M 2024-03-23T17:37:43.085 INFO:teuthology.orchestra.run.smithi092.stdout: toolbox x86_64 0.0.99.4-5.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 2.5 M 2024-03-23T17:37:43.085 INFO:teuthology.orchestra.run.smithi092.stdout: udica noarch 0.2.6-20.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 49 k 2024-03-23T17:37:43.085 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-03-23T17:37:43.086 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2024-03-23T17:37:43.086 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2024-03-23T17:37:43.086 INFO:teuthology.orchestra.run.smithi092.stdout: yajl x86_64 2.1.0-12.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2024-03-23T17:37:43.086 INFO:teuthology.orchestra.run.smithi092.stdout:Installing weak dependencies: 2024-03-23T17:37:43.086 INFO:teuthology.orchestra.run.smithi092.stdout: criu-libs x86_64 3.18-4.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 39 k 2024-03-23T17:37:43.086 INFO:teuthology.orchestra.run.smithi092.stdout:Installing module profiles: 2024-03-23T17:37:43.086 INFO:teuthology.orchestra.run.smithi092.stdout: container-tools/common 2024-03-23T17:37:43.086 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-23T17:37:43.086 INFO:teuthology.orchestra.run.smithi092.stdout:Enabling module streams: 2024-03-23T17:37:43.086 INFO:teuthology.orchestra.run.smithi092.stdout: container-tools rhel8 2024-03-23T17:37:43.086 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-23T17:37:43.086 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-03-23T17:37:43.086 INFO:teuthology.orchestra.run.smithi092.stdout:========================================================================================================== 2024-03-23T17:37:43.086 INFO:teuthology.orchestra.run.smithi092.stdout:Install 13 Packages 2024-03-23T17:37:43.087 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-23T17:37:43.087 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 26 M 2024-03-23T17:37:43.087 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 87 M 2024-03-23T17:37:43.087 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-03-23T17:37:43.282 INFO:teuthology.orchestra.run.smithi092.stdout:(1/13): criu-libs-3.18-4.module+el8.9.0+20326+3 203 kB/s | 39 kB 00:00 2024-03-23T17:37:43.324 INFO:teuthology.orchestra.run.smithi092.stdout:(2/13): python3-pyxdg-0.25-16.el8.noarch.rpm 397 kB/s | 94 kB 00:00 2024-03-23T17:37:43.357 INFO:teuthology.orchestra.run.smithi092.stdout:(3/13): python3-pytoml-0.1.14-5.git7dea353.el8. 92 kB/s | 25 kB 00:00 2024-03-23T17:37:43.466 INFO:teuthology.orchestra.run.smithi092.stdout:(4/13): yajl-2.1.0-12.el8.x86_64.rpm 223 kB/s | 41 kB 00:00 2024-03-23T17:37:43.533 INFO:teuthology.orchestra.run.smithi092.stdout:(5/13): aardvark-dns-1.7.0-1.module+el8.9.0+212 4.8 MB/s | 1.0 MB 00:00 2024-03-23T17:37:43.741 INFO:teuthology.orchestra.run.smithi092.stdout:(6/13): buildah-1.31.3-3.module+el8.9.0+21243+a 23 MB/s | 8.8 MB 00:00 2024-03-23T17:37:43.767 INFO:teuthology.orchestra.run.smithi092.stdout:(7/13): crun-1.8.7-1.module+el8.9.0+21243+a5865 1.0 MB/s | 239 kB 00:00 2024-03-23T17:37:43.833 INFO:teuthology.orchestra.run.smithi092.stdout:(8/13): cockpit-podman-75-1.module+el8.9.0+2124 2.0 MB/s | 739 kB 00:00 2024-03-23T17:37:43.975 INFO:teuthology.orchestra.run.smithi092.stdout:(9/13): python3-podman-4.6.0-2.module+el8.9.0+2 737 kB/s | 154 kB 00:00 2024-03-23T17:37:44.051 INFO:teuthology.orchestra.run.smithi092.stdout:(10/13): netavark-1.7.0-2.module+el8.9.0+21243+ 12 MB/s | 3.7 MB 00:00 2024-03-23T17:37:44.218 INFO:teuthology.orchestra.run.smithi092.stdout:(11/13): skopeo-1.13.3-3.module+el8.9.0+21243+a 21 MB/s | 8.1 MB 00:00 2024-03-23T17:37:44.293 INFO:teuthology.orchestra.run.smithi092.stdout:(12/13): toolbox-0.0.99.4-5.module+el8.9.0+2124 8.0 MB/s | 2.5 MB 00:00 2024-03-23T17:37:44.376 INFO:teuthology.orchestra.run.smithi092.stdout:(13/13): udica-0.2.6-20.module+el8.9.0+21243+a5 152 kB/s | 49 kB 00:00 2024-03-23T17:37:44.377 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-03-23T17:37:44.377 INFO:teuthology.orchestra.run.smithi092.stdout:Total 20 MB/s | 26 MB 00:01 2024-03-23T17:37:44.449 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2024-03-23T17:37:44.615 INFO:teuthology.orchestra.run.smithi152.stdout:========================================================================================================== 2024-03-23T17:37:44.615 INFO:teuthology.orchestra.run.smithi152.stdout: Package Arch Version Repository Size 2024-03-23T17:37:44.615 INFO:teuthology.orchestra.run.smithi152.stdout:========================================================================================================== 2024-03-23T17:37:44.616 INFO:teuthology.orchestra.run.smithi152.stdout:Installing group/module packages: 2024-03-23T17:37:44.616 INFO:teuthology.orchestra.run.smithi152.stdout: aardvark-dns x86_64 2:1.7.0-1.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 1.0 M 2024-03-23T17:37:44.616 INFO:teuthology.orchestra.run.smithi152.stdout: buildah x86_64 1:1.31.3-3.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 8.8 M 2024-03-23T17:37:44.616 INFO:teuthology.orchestra.run.smithi152.stdout: cockpit-podman noarch 75-1.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 739 k 2024-03-23T17:37:44.616 INFO:teuthology.orchestra.run.smithi152.stdout: crun x86_64 1.8.7-1.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 239 k 2024-03-23T17:37:44.616 INFO:teuthology.orchestra.run.smithi152.stdout: netavark x86_64 2:1.7.0-2.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-03-23T17:37:44.616 INFO:teuthology.orchestra.run.smithi152.stdout: python3-podman noarch 4.6.0-2.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 154 k 2024-03-23T17:37:44.616 INFO:teuthology.orchestra.run.smithi152.stdout: skopeo x86_64 2:1.13.3-3.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 8.1 M 2024-03-23T17:37:44.616 INFO:teuthology.orchestra.run.smithi152.stdout: toolbox x86_64 0.0.99.4-5.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 2.5 M 2024-03-23T17:37:44.616 INFO:teuthology.orchestra.run.smithi152.stdout: udica noarch 0.2.6-20.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 49 k 2024-03-23T17:37:44.616 INFO:teuthology.orchestra.run.smithi152.stdout:Installing dependencies: 2024-03-23T17:37:44.616 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2024-03-23T17:37:44.617 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2024-03-23T17:37:44.617 INFO:teuthology.orchestra.run.smithi152.stdout: yajl x86_64 2.1.0-12.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2024-03-23T17:37:44.617 INFO:teuthology.orchestra.run.smithi152.stdout:Installing weak dependencies: 2024-03-23T17:37:44.617 INFO:teuthology.orchestra.run.smithi152.stdout: criu-libs x86_64 3.18-4.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 39 k 2024-03-23T17:37:44.617 INFO:teuthology.orchestra.run.smithi152.stdout:Installing module profiles: 2024-03-23T17:37:44.617 INFO:teuthology.orchestra.run.smithi152.stdout: container-tools/common 2024-03-23T17:37:44.617 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-23T17:37:44.617 INFO:teuthology.orchestra.run.smithi152.stdout:Enabling module streams: 2024-03-23T17:37:44.617 INFO:teuthology.orchestra.run.smithi152.stdout: container-tools rhel8 2024-03-23T17:37:44.617 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-23T17:37:44.617 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2024-03-23T17:37:44.617 INFO:teuthology.orchestra.run.smithi152.stdout:========================================================================================================== 2024-03-23T17:37:44.617 INFO:teuthology.orchestra.run.smithi152.stdout:Install 13 Packages 2024-03-23T17:37:44.617 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-23T17:37:44.618 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 26 M 2024-03-23T17:37:44.618 INFO:teuthology.orchestra.run.smithi152.stdout:Installed size: 87 M 2024-03-23T17:37:44.618 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2024-03-23T17:37:44.660 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-03-23T17:37:44.682 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-03-23T17:37:44.682 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-03-23T17:37:44.800 INFO:teuthology.orchestra.run.smithi152.stdout:(1/13): python3-pyxdg-0.25-16.el8.noarch.rpm 515 kB/s | 94 kB 00:00 2024-03-23T17:37:44.826 INFO:teuthology.orchestra.run.smithi152.stdout:(2/13): python3-pytoml-0.1.14-5.git7dea353.el8. 119 kB/s | 25 kB 00:00 2024-03-23T17:37:44.864 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-03-23T17:37:44.867 INFO:teuthology.orchestra.run.smithi152.stdout:(3/13): criu-libs-3.18-4.module+el8.9.0+20326+3 158 kB/s | 39 kB 00:00 2024-03-23T17:37:44.910 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-03-23T17:37:44.919 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-03-23T17:37:44.934 INFO:teuthology.orchestra.run.smithi152.stdout:(4/13): yajl-2.1.0-12.el8.x86_64.rpm 306 kB/s | 41 kB 00:00 2024-03-23T17:37:45.038 INFO:teuthology.orchestra.run.smithi086.stdout:========================================================================================================== 2024-03-23T17:37:45.039 INFO:teuthology.orchestra.run.smithi086.stdout: Package Arch Version Repository Size 2024-03-23T17:37:45.039 INFO:teuthology.orchestra.run.smithi086.stdout:========================================================================================================== 2024-03-23T17:37:45.039 INFO:teuthology.orchestra.run.smithi086.stdout:Installing group/module packages: 2024-03-23T17:37:45.039 INFO:teuthology.orchestra.run.smithi086.stdout: aardvark-dns x86_64 2:1.7.0-1.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 1.0 M 2024-03-23T17:37:45.039 INFO:teuthology.orchestra.run.smithi086.stdout: buildah x86_64 1:1.31.3-3.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 8.8 M 2024-03-23T17:37:45.039 INFO:teuthology.orchestra.run.smithi086.stdout: cockpit-podman noarch 75-1.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 739 k 2024-03-23T17:37:45.039 INFO:teuthology.orchestra.run.smithi086.stdout: crun x86_64 1.8.7-1.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 239 k 2024-03-23T17:37:45.039 INFO:teuthology.orchestra.run.smithi086.stdout: netavark x86_64 2:1.7.0-2.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-03-23T17:37:45.039 INFO:teuthology.orchestra.run.smithi086.stdout: python3-podman noarch 4.6.0-2.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 154 k 2024-03-23T17:37:45.039 INFO:teuthology.orchestra.run.smithi086.stdout: skopeo x86_64 2:1.13.3-3.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 8.1 M 2024-03-23T17:37:45.039 INFO:teuthology.orchestra.run.smithi086.stdout: toolbox x86_64 0.0.99.4-5.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 2.5 M 2024-03-23T17:37:45.039 INFO:teuthology.orchestra.run.smithi086.stdout: udica noarch 0.2.6-20.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 49 k 2024-03-23T17:37:45.039 INFO:teuthology.orchestra.run.smithi086.stdout:Installing dependencies: 2024-03-23T17:37:45.040 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2024-03-23T17:37:45.040 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2024-03-23T17:37:45.040 INFO:teuthology.orchestra.run.smithi086.stdout: yajl x86_64 2.1.0-12.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2024-03-23T17:37:45.040 INFO:teuthology.orchestra.run.smithi086.stdout:Installing weak dependencies: 2024-03-23T17:37:45.040 INFO:teuthology.orchestra.run.smithi086.stdout: criu-libs x86_64 3.18-4.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 39 k 2024-03-23T17:37:45.040 INFO:teuthology.orchestra.run.smithi086.stdout:Installing module profiles: 2024-03-23T17:37:45.040 INFO:teuthology.orchestra.run.smithi086.stdout: container-tools/common 2024-03-23T17:37:45.040 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:37:45.040 INFO:teuthology.orchestra.run.smithi086.stdout:Enabling module streams: 2024-03-23T17:37:45.040 INFO:teuthology.orchestra.run.smithi086.stdout: container-tools rhel8 2024-03-23T17:37:45.040 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:37:45.040 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction Summary 2024-03-23T17:37:45.040 INFO:teuthology.orchestra.run.smithi086.stdout:========================================================================================================== 2024-03-23T17:37:45.040 INFO:teuthology.orchestra.run.smithi086.stdout:Install 13 Packages 2024-03-23T17:37:45.040 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:37:45.041 INFO:teuthology.orchestra.run.smithi086.stdout:Total download size: 26 M 2024-03-23T17:37:45.041 INFO:teuthology.orchestra.run.smithi086.stdout:Installed size: 87 M 2024-03-23T17:37:45.041 INFO:teuthology.orchestra.run.smithi086.stdout:Downloading Packages: 2024-03-23T17:37:45.118 INFO:teuthology.orchestra.run.smithi152.stdout:(5/13): aardvark-dns-1.7.0-1.module+el8.9.0+212 3.5 MB/s | 1.0 MB 00:00 2024-03-23T17:37:45.151 INFO:teuthology.orchestra.run.smithi152.stdout:(6/13): cockpit-podman-75-1.module+el8.9.0+2124 3.3 MB/s | 739 kB 00:00 2024-03-23T17:37:45.175 INFO:teuthology.orchestra.run.smithi086.stdout:(1/13): python3-pytoml-0.1.14-5.git7dea353.el8. 186 kB/s | 25 kB 00:00 2024-03-23T17:37:45.194 INFO:teuthology.orchestra.run.smithi152.stdout:(7/13): crun-1.8.7-1.module+el8.9.0+21243+a5865 3.1 MB/s | 239 kB 00:00 2024-03-23T17:37:45.201 INFO:teuthology.orchestra.run.smithi086.stdout:(2/13): criu-libs-3.18-4.module+el8.9.0+20326+3 248 kB/s | 39 kB 00:00 2024-03-23T17:37:45.226 INFO:teuthology.orchestra.run.smithi086.stdout:(3/13): python3-pyxdg-0.25-16.el8.noarch.rpm 511 kB/s | 94 kB 00:00 2024-03-23T17:37:45.284 INFO:teuthology.orchestra.run.smithi086.stdout:(4/13): yajl-2.1.0-12.el8.x86_64.rpm 376 kB/s | 41 kB 00:00 2024-03-23T17:37:45.361 INFO:teuthology.orchestra.run.smithi152.stdout:(8/13): buildah-1.31.3-3.module+el8.9.0+21243+a 18 MB/s | 8.8 MB 00:00 2024-03-23T17:37:45.386 INFO:teuthology.orchestra.run.smithi152.stdout:(9/13): python3-podman-4.6.0-2.module+el8.9.0+2 800 kB/s | 154 kB 00:00 2024-03-23T17:37:45.478 INFO:teuthology.orchestra.run.smithi152.stdout:(10/13): netavark-1.7.0-2.module+el8.9.0+21243+ 11 MB/s | 3.7 MB 00:00 2024-03-23T17:37:45.493 INFO:teuthology.orchestra.run.smithi086.stdout:(5/13): aardvark-dns-1.7.0-1.module+el8.9.0+212 3.5 MB/s | 1.0 MB 00:00 2024-03-23T17:37:45.535 INFO:teuthology.orchestra.run.smithi086.stdout:(6/13): cockpit-podman-75-1.module+el8.9.0+2124 2.9 MB/s | 739 kB 00:00 2024-03-23T17:37:45.561 INFO:teuthology.orchestra.run.smithi152.stdout:(11/13): toolbox-0.0.99.4-5.module+el8.9.0+2124 14 MB/s | 2.5 MB 00:00 2024-03-23T17:37:45.621 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-03-23T17:37:45.644 INFO:teuthology.orchestra.run.smithi086.stdout:(7/13): crun-1.8.7-1.module+el8.9.0+21243+a5865 1.6 MB/s | 239 kB 00:00 2024-03-23T17:37:45.653 INFO:teuthology.orchestra.run.smithi152.stdout:(12/13): udica-0.2.6-20.module+el8.9.0+21243+a5 282 kB/s | 49 kB 00:00 2024-03-23T17:37:45.753 INFO:teuthology.orchestra.run.smithi086.stdout:(8/13): buildah-1.31.3-3.module+el8.9.0+21243+a 17 MB/s | 8.8 MB 00:00 2024-03-23T17:37:45.795 INFO:teuthology.orchestra.run.smithi152.stdout:(13/13): skopeo-1.13.3-3.module+el8.9.0+21243+a 19 MB/s | 8.1 MB 00:00 2024-03-23T17:37:45.796 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2024-03-23T17:37:45.796 INFO:teuthology.orchestra.run.smithi152.stdout:Total 22 MB/s | 26 MB 00:01 2024-03-23T17:37:45.869 INFO:teuthology.orchestra.run.smithi086.stdout:(9/13): netavark-1.7.0-2.module+el8.9.0+21243+a 11 MB/s | 3.7 MB 00:00 2024-03-23T17:37:45.895 INFO:teuthology.orchestra.run.smithi086.stdout:(10/13): python3-podman-4.6.0-2.module+el8.9.0+ 614 kB/s | 154 kB 00:00 2024-03-23T17:37:46.028 INFO:teuthology.orchestra.run.smithi086.stdout:(11/13): toolbox-0.0.99.4-5.module+el8.9.0+2124 16 MB/s | 2.5 MB 00:00 2024-03-23T17:37:46.045 INFO:teuthology.orchestra.run.smithi086.stdout:(12/13): udica-0.2.6-20.module+el8.9.0+21243+a5 329 kB/s | 49 kB 00:00 2024-03-23T17:37:46.080 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2024-03-23T17:37:46.101 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2024-03-23T17:37:46.101 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2024-03-23T17:37:46.143 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : aardvark-dns-2:1.7.0-1.module+el8.9.0+21243+a58653 1/13 2024-03-23T17:37:46.154 INFO:teuthology.orchestra.run.smithi086.stdout:(13/13): skopeo-1.13.3-3.module+el8.9.0+21243+a 20 MB/s | 8.1 MB 00:00 2024-03-23T17:37:46.154 INFO:teuthology.orchestra.run.smithi086.stdout:-------------------------------------------------------------------------------- 2024-03-23T17:37:46.154 INFO:teuthology.orchestra.run.smithi086.stdout:Total 23 MB/s | 26 MB 00:01 2024-03-23T17:37:46.331 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2024-03-23T17:37:46.340 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2024-03-23T17:37:46.371 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : netavark-2:1.7.0-2.module+el8.9.0+21243+a586538b.x 2/13 2024-03-23T17:37:46.516 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : yajl-2.1.0-12.el8.x86_64 3/13 2024-03-23T17:37:46.609 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction check 2024-03-23T17:37:46.635 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction check succeeded. 2024-03-23T17:37:46.635 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction test 2024-03-23T17:37:46.684 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 4/13 2024-03-23T17:37:46.863 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 5/13 2024-03-23T17:37:47.044 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2024-03-23T17:37:47.130 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction test succeeded. 2024-03-23T17:37:47.133 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 6/13 2024-03-23T17:37:47.139 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction 2024-03-23T17:37:47.425 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-podman-4.6.0-2.module+el8.9.0+21243+a58653 7/13 2024-03-23T17:37:47.614 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : aardvark-dns-2:1.7.0-1.module+el8.9.0+21243+a58653 1/13 2024-03-23T17:37:47.861 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : netavark-2:1.7.0-2.module+el8.9.0+21243+a586538b.x 2/13 2024-03-23T17:37:47.869 INFO:teuthology.orchestra.run.smithi086.stdout: Preparing : 1/1 2024-03-23T17:37:48.045 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : yajl-2.1.0-12.el8.x86_64 3/13 2024-03-23T17:37:48.264 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 4/13 2024-03-23T17:37:48.309 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : crun-1.8.7-1.module+el8.9.0+21243+a586538b.x86_64 8/13 2024-03-23T17:37:48.381 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : aardvark-dns-2:1.7.0-1.module+el8.9.0+21243+a58653 1/13 2024-03-23T17:37:48.492 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 5/13 2024-03-23T17:37:48.534 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : buildah-1:1.31.3-3.module+el8.9.0+21243+a586538b.x 9/13 2024-03-23T17:37:48.583 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : netavark-2:1.7.0-2.module+el8.9.0+21243+a586538b.x 2/13 2024-03-23T17:37:48.704 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 6/13 2024-03-23T17:37:48.754 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : yajl-2.1.0-12.el8.x86_64 3/13 2024-03-23T17:37:48.954 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-podman-4.6.0-2.module+el8.9.0+21243+a58653 7/13 2024-03-23T17:37:48.957 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 4/13 2024-03-23T17:37:48.966 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : udica-0.2.6-20.module+el8.9.0+21243+a586538b.noarc 10/13 2024-03-23T17:37:49.117 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 5/13 2024-03-23T17:37:49.351 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 6/13 2024-03-23T17:37:49.570 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-podman-4.6.0-2.module+el8.9.0+21243+a58653 7/13 2024-03-23T17:37:49.834 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : toolbox-0.0.99.4-5.module+el8.9.0+21243+a586538b.x 11/13 2024-03-23T17:37:49.841 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : crun-1.8.7-1.module+el8.9.0+21243+a586538b.x86_64 8/13 2024-03-23T17:37:50.036 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : skopeo-2:1.13.3-3.module+el8.9.0+21243+a586538b.x8 12/13 2024-03-23T17:37:50.075 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : buildah-1:1.31.3-3.module+el8.9.0+21243+a586538b.x 9/13 2024-03-23T17:37:50.244 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : cockpit-podman-75-1.module+el8.9.0+21243+a586538b. 13/13 2024-03-23T17:37:50.479 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : crun-1.8.7-1.module+el8.9.0+21243+a586538b.x86_64 8/13 2024-03-23T17:37:50.488 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : udica-0.2.6-20.module+el8.9.0+21243+a586538b.noarc 10/13 2024-03-23T17:37:50.662 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: cockpit-podman-75-1.module+el8.9.0+21243+a586538b. 13/13 2024-03-23T17:37:50.662 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/13 2024-03-23T17:37:50.662 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 2/13 2024-03-23T17:37:50.662 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 3/13 2024-03-23T17:37:50.663 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 4/13 2024-03-23T17:37:50.663 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : aardvark-dns-2:1.7.0-1.module+el8.9.0+21243+a58653 5/13 2024-03-23T17:37:50.663 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : buildah-1:1.31.3-3.module+el8.9.0+21243+a586538b.x 6/13 2024-03-23T17:37:50.663 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : cockpit-podman-75-1.module+el8.9.0+21243+a586538b. 7/13 2024-03-23T17:37:50.663 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : crun-1.8.7-1.module+el8.9.0+21243+a586538b.x86_64 8/13 2024-03-23T17:37:50.663 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : netavark-2:1.7.0-2.module+el8.9.0+21243+a586538b.x 9/13 2024-03-23T17:37:50.663 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-podman-4.6.0-2.module+el8.9.0+21243+a58653 10/13 2024-03-23T17:37:50.663 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : skopeo-2:1.13.3-3.module+el8.9.0+21243+a586538b.x8 11/13 2024-03-23T17:37:50.663 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : toolbox-0.0.99.4-5.module+el8.9.0+21243+a586538b.x 12/13 2024-03-23T17:37:50.693 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : buildah-1:1.31.3-3.module+el8.9.0+21243+a586538b.x 9/13 2024-03-23T17:37:51.079 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : udica-0.2.6-20.module+el8.9.0+21243+a586538b.noarc 10/13 2024-03-23T17:37:51.405 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : toolbox-0.0.99.4-5.module+el8.9.0+21243+a586538b.x 11/13 2024-03-23T17:37:51.666 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : skopeo-2:1.13.3-3.module+el8.9.0+21243+a586538b.x8 12/13 2024-03-23T17:37:51.898 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : cockpit-podman-75-1.module+el8.9.0+21243+a586538b. 13/13 2024-03-23T17:37:52.017 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : toolbox-0.0.99.4-5.module+el8.9.0+21243+a586538b.x 11/13 2024-03-23T17:37:52.239 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : skopeo-2:1.13.3-3.module+el8.9.0+21243+a586538b.x8 12/13 2024-03-23T17:37:52.332 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: cockpit-podman-75-1.module+el8.9.0+21243+a586538b. 13/13 2024-03-23T17:37:52.332 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/13 2024-03-23T17:37:52.332 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 2/13 2024-03-23T17:37:52.332 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 3/13 2024-03-23T17:37:52.332 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 4/13 2024-03-23T17:37:52.332 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : aardvark-dns-2:1.7.0-1.module+el8.9.0+21243+a58653 5/13 2024-03-23T17:37:52.332 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : buildah-1:1.31.3-3.module+el8.9.0+21243+a586538b.x 6/13 2024-03-23T17:37:52.332 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : cockpit-podman-75-1.module+el8.9.0+21243+a586538b. 7/13 2024-03-23T17:37:52.333 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : crun-1.8.7-1.module+el8.9.0+21243+a586538b.x86_64 8/13 2024-03-23T17:37:52.333 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : netavark-2:1.7.0-2.module+el8.9.0+21243+a586538b.x 9/13 2024-03-23T17:37:52.333 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-podman-4.6.0-2.module+el8.9.0+21243+a58653 10/13 2024-03-23T17:37:52.333 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : skopeo-2:1.13.3-3.module+el8.9.0+21243+a586538b.x8 11/13 2024-03-23T17:37:52.333 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : toolbox-0.0.99.4-5.module+el8.9.0+21243+a586538b.x 12/13 2024-03-23T17:37:52.389 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : cockpit-podman-75-1.module+el8.9.0+21243+a586538b. 13/13 2024-03-23T17:37:53.077 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: cockpit-podman-75-1.module+el8.9.0+21243+a586538b. 13/13 2024-03-23T17:37:53.077 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/13 2024-03-23T17:37:53.077 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 2/13 2024-03-23T17:37:53.077 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 3/13 2024-03-23T17:37:53.077 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 4/13 2024-03-23T17:37:53.077 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : aardvark-dns-2:1.7.0-1.module+el8.9.0+21243+a58653 5/13 2024-03-23T17:37:53.078 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : buildah-1:1.31.3-3.module+el8.9.0+21243+a586538b.x 6/13 2024-03-23T17:37:53.078 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : cockpit-podman-75-1.module+el8.9.0+21243+a586538b. 7/13 2024-03-23T17:37:53.078 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : crun-1.8.7-1.module+el8.9.0+21243+a586538b.x86_64 8/13 2024-03-23T17:37:53.078 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : netavark-2:1.7.0-2.module+el8.9.0+21243+a586538b.x 9/13 2024-03-23T17:37:53.078 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-podman-4.6.0-2.module+el8.9.0+21243+a58653 10/13 2024-03-23T17:37:53.078 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : skopeo-2:1.13.3-3.module+el8.9.0+21243+a586538b.x8 11/13 2024-03-23T17:37:53.078 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : toolbox-0.0.99.4-5.module+el8.9.0+21243+a586538b.x 12/13 2024-03-23T17:38:01.542 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : udica-0.2.6-20.module+el8.9.0+21243+a586538b.noarc 13/13 2024-03-23T17:38:01.542 INFO:teuthology.orchestra.run.smithi092.stdout:Installed products updated. 2024-03-23T17:38:02.130 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-23T17:38:02.130 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-03-23T17:38:02.131 INFO:teuthology.orchestra.run.smithi092.stdout: aardvark-dns-2:1.7.0-1.module+el8.9.0+21243+a586538b.x86_64 2024-03-23T17:38:02.131 INFO:teuthology.orchestra.run.smithi092.stdout: buildah-1:1.31.3-3.module+el8.9.0+21243+a586538b.x86_64 2024-03-23T17:38:02.131 INFO:teuthology.orchestra.run.smithi092.stdout: cockpit-podman-75-1.module+el8.9.0+21243+a586538b.noarch 2024-03-23T17:38:02.131 INFO:teuthology.orchestra.run.smithi092.stdout: criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86_64 2024-03-23T17:38:02.131 INFO:teuthology.orchestra.run.smithi092.stdout: crun-1.8.7-1.module+el8.9.0+21243+a586538b.x86_64 2024-03-23T17:38:02.131 INFO:teuthology.orchestra.run.smithi092.stdout: netavark-2:1.7.0-2.module+el8.9.0+21243+a586538b.x86_64 2024-03-23T17:38:02.131 INFO:teuthology.orchestra.run.smithi092.stdout: python3-podman-4.6.0-2.module+el8.9.0+21243+a586538b.noarch 2024-03-23T17:38:02.131 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2024-03-23T17:38:02.131 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyxdg-0.25-16.el8.noarch 2024-03-23T17:38:02.131 INFO:teuthology.orchestra.run.smithi092.stdout: skopeo-2:1.13.3-3.module+el8.9.0+21243+a586538b.x86_64 2024-03-23T17:38:02.131 INFO:teuthology.orchestra.run.smithi092.stdout: toolbox-0.0.99.4-5.module+el8.9.0+21243+a586538b.x86_64 2024-03-23T17:38:02.131 INFO:teuthology.orchestra.run.smithi092.stdout: udica-0.2.6-20.module+el8.9.0+21243+a586538b.noarch 2024-03-23T17:38:02.131 INFO:teuthology.orchestra.run.smithi092.stdout: yajl-2.1.0-12.el8.x86_64 2024-03-23T17:38:02.131 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-23T17:38:02.131 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-03-23T17:38:02.298 DEBUG:teuthology.parallel:result is None 2024-03-23T17:38:05.519 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : udica-0.2.6-20.module+el8.9.0+21243+a586538b.noarc 13/13 2024-03-23T17:38:05.520 INFO:teuthology.orchestra.run.smithi152.stdout:Installed products updated. 2024-03-23T17:38:06.063 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-23T17:38:06.063 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2024-03-23T17:38:06.063 INFO:teuthology.orchestra.run.smithi152.stdout: aardvark-dns-2:1.7.0-1.module+el8.9.0+21243+a586538b.x86_64 2024-03-23T17:38:06.063 INFO:teuthology.orchestra.run.smithi152.stdout: buildah-1:1.31.3-3.module+el8.9.0+21243+a586538b.x86_64 2024-03-23T17:38:06.063 INFO:teuthology.orchestra.run.smithi152.stdout: cockpit-podman-75-1.module+el8.9.0+21243+a586538b.noarch 2024-03-23T17:38:06.063 INFO:teuthology.orchestra.run.smithi152.stdout: criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86_64 2024-03-23T17:38:06.063 INFO:teuthology.orchestra.run.smithi152.stdout: crun-1.8.7-1.module+el8.9.0+21243+a586538b.x86_64 2024-03-23T17:38:06.063 INFO:teuthology.orchestra.run.smithi152.stdout: netavark-2:1.7.0-2.module+el8.9.0+21243+a586538b.x86_64 2024-03-23T17:38:06.063 INFO:teuthology.orchestra.run.smithi152.stdout: python3-podman-4.6.0-2.module+el8.9.0+21243+a586538b.noarch 2024-03-23T17:38:06.064 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2024-03-23T17:38:06.064 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyxdg-0.25-16.el8.noarch 2024-03-23T17:38:06.064 INFO:teuthology.orchestra.run.smithi152.stdout: skopeo-2:1.13.3-3.module+el8.9.0+21243+a586538b.x86_64 2024-03-23T17:38:06.064 INFO:teuthology.orchestra.run.smithi152.stdout: toolbox-0.0.99.4-5.module+el8.9.0+21243+a586538b.x86_64 2024-03-23T17:38:06.064 INFO:teuthology.orchestra.run.smithi152.stdout: udica-0.2.6-20.module+el8.9.0+21243+a586538b.noarch 2024-03-23T17:38:06.064 INFO:teuthology.orchestra.run.smithi152.stdout: yajl-2.1.0-12.el8.x86_64 2024-03-23T17:38:06.064 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-23T17:38:06.064 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2024-03-23T17:38:06.257 DEBUG:teuthology.parallel:result is None 2024-03-23T17:38:06.537 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : udica-0.2.6-20.module+el8.9.0+21243+a586538b.noarc 13/13 2024-03-23T17:38:06.537 INFO:teuthology.orchestra.run.smithi086.stdout:Installed products updated. 2024-03-23T17:38:07.151 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:38:07.151 INFO:teuthology.orchestra.run.smithi086.stdout:Installed: 2024-03-23T17:38:07.151 INFO:teuthology.orchestra.run.smithi086.stdout: aardvark-dns-2:1.7.0-1.module+el8.9.0+21243+a586538b.x86_64 2024-03-23T17:38:07.151 INFO:teuthology.orchestra.run.smithi086.stdout: buildah-1:1.31.3-3.module+el8.9.0+21243+a586538b.x86_64 2024-03-23T17:38:07.152 INFO:teuthology.orchestra.run.smithi086.stdout: cockpit-podman-75-1.module+el8.9.0+21243+a586538b.noarch 2024-03-23T17:38:07.152 INFO:teuthology.orchestra.run.smithi086.stdout: criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86_64 2024-03-23T17:38:07.152 INFO:teuthology.orchestra.run.smithi086.stdout: crun-1.8.7-1.module+el8.9.0+21243+a586538b.x86_64 2024-03-23T17:38:07.152 INFO:teuthology.orchestra.run.smithi086.stdout: netavark-2:1.7.0-2.module+el8.9.0+21243+a586538b.x86_64 2024-03-23T17:38:07.152 INFO:teuthology.orchestra.run.smithi086.stdout: python3-podman-4.6.0-2.module+el8.9.0+21243+a586538b.noarch 2024-03-23T17:38:07.152 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2024-03-23T17:38:07.152 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyxdg-0.25-16.el8.noarch 2024-03-23T17:38:07.152 INFO:teuthology.orchestra.run.smithi086.stdout: skopeo-2:1.13.3-3.module+el8.9.0+21243+a586538b.x86_64 2024-03-23T17:38:07.152 INFO:teuthology.orchestra.run.smithi086.stdout: toolbox-0.0.99.4-5.module+el8.9.0+21243+a586538b.x86_64 2024-03-23T17:38:07.152 INFO:teuthology.orchestra.run.smithi086.stdout: udica-0.2.6-20.module+el8.9.0+21243+a586538b.noarch 2024-03-23T17:38:07.152 INFO:teuthology.orchestra.run.smithi086.stdout: yajl-2.1.0-12.el8.x86_64 2024-03-23T17:38:07.152 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:38:07.152 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-03-23T17:38:07.381 DEBUG:teuthology.parallel:result is None 2024-03-23T17:38:07.381 INFO:teuthology.run_tasks:Running task install... 2024-03-23T17:38:07.390 DEBUG:teuthology.task.install:project ceph 2024-03-23T17:38:07.390 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '82316b04821a6c2c1d458a881920454bbadb8837'}} 2024-03-23T17:38:07.391 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '82316b04821a6c2c1d458a881920454bbadb8837'} 2024-03-23T17:38:07.391 INFO:teuthology.task.install:Using flavor: default 2024-03-23T17:38:07.396 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-23T17:38:07.396 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2024-03-23T17:38:07.397 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '82316b04821a6c2c1d458a881920454bbadb8837', 'tag': None, 'wait_for_package': False} 2024-03-23T17:38:07.397 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=82316b04821a6c2c1d458a881920454bbadb8837 2024-03-23T17:38:07.399 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '82316b04821a6c2c1d458a881920454bbadb8837', 'tag': None, 'wait_for_package': False} 2024-03-23T17:38:07.400 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=82316b04821a6c2c1d458a881920454bbadb8837 2024-03-23T17:38:07.401 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '82316b04821a6c2c1d458a881920454bbadb8837', 'tag': None, 'wait_for_package': False} 2024-03-23T17:38:07.401 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=82316b04821a6c2c1d458a881920454bbadb8837 2024-03-23T17:38:07.554 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-yuri11-testing-2024-03-21-0851-reef/82316b04821a6c2c1d458a881920454bbadb8837/centos/8/flavors/default/ 2024-03-23T17:38:07.554 INFO:teuthology.task.install.rpm:Package version is 18.2.2-954.g82316b04 2024-03-23T17:38:07.568 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-yuri11-testing-2024-03-21-0851-reef/82316b04821a6c2c1d458a881920454bbadb8837/centos/8/flavors/default/ 2024-03-23T17:38:07.568 INFO:teuthology.task.install.rpm:Package version is 18.2.2-954.g82316b04 2024-03-23T17:38:07.613 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-yuri11-testing-2024-03-21-0851-reef/82316b04821a6c2c1d458a881920454bbadb8837/centos/8/flavors/default/ 2024-03-23T17:38:07.613 INFO:teuthology.task.install.rpm:Package version is 18.2.2-954.g82316b04 2024-03-23T17:38:07.877 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri11-testing-2024-03-21-0851-reef/82316b04821a6c2c1d458a881920454bbadb8837/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri11-testing-2024-03-21-0851-reef/82316b04821a6c2c1d458a881920454bbadb8837/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri11-testing-2024-03-21-0851-reef/82316b04821a6c2c1d458a881920454bbadb8837/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-23T17:38:07.877 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-03-23T17:38:07.877 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-23T17:38:07.890 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri11-testing-2024-03-21-0851-reef/82316b04821a6c2c1d458a881920454bbadb8837/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri11-testing-2024-03-21-0851-reef/82316b04821a6c2c1d458a881920454bbadb8837/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri11-testing-2024-03-21-0851-reef/82316b04821a6c2c1d458a881920454bbadb8837/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-23T17:38:07.890 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-03-23T17:38:07.890 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-23T17:38:07.911 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-23T17:38:07.911 DEBUG:teuthology.orchestra.run.smithi152:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/82316b04821a6c2c1d458a881920454bbadb8837/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-23T17:38:07.924 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-23T17:38:07.924 DEBUG:teuthology.orchestra.run.smithi092:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/82316b04821a6c2c1d458a881920454bbadb8837/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-23T17:38:07.931 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri11-testing-2024-03-21-0851-reef/82316b04821a6c2c1d458a881920454bbadb8837/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri11-testing-2024-03-21-0851-reef/82316b04821a6c2c1d458a881920454bbadb8837/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri11-testing-2024-03-21-0851-reef/82316b04821a6c2c1d458a881920454bbadb8837/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-23T17:38:07.931 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-23T17:38:07.931 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-23T17:38:07.984 DEBUG:teuthology.orchestra.run.smithi152:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-23T17:38:07.997 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-23T17:38:07.997 DEBUG:teuthology.orchestra.run.smithi086:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/82316b04821a6c2c1d458a881920454bbadb8837/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-23T17:38:07.998 DEBUG:teuthology.orchestra.run.smithi092:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-23T17:38:08.064 DEBUG:teuthology.orchestra.run.smithi152:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-23T17:38:08.084 DEBUG:teuthology.orchestra.run.smithi092:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-23T17:38:08.104 DEBUG:teuthology.orchestra.run.smithi086:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-23T17:38:08.116 INFO:teuthology.orchestra.run.smithi092.stdout:check_obsoletes = 1 2024-03-23T17:38:08.118 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum clean all 2024-03-23T17:38:08.132 INFO:teuthology.orchestra.run.smithi152.stdout:check_obsoletes = 1 2024-03-23T17:38:08.134 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum clean all 2024-03-23T17:38:08.193 DEBUG:teuthology.orchestra.run.smithi086:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-23T17:38:08.257 INFO:teuthology.orchestra.run.smithi086.stdout:check_obsoletes = 1 2024-03-23T17:38:08.260 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum clean all 2024-03-23T17:38:08.514 INFO:teuthology.orchestra.run.smithi092.stdout:Updating Subscription Management repositories. 2024-03-23T17:38:08.517 INFO:teuthology.orchestra.run.smithi152.stdout:Updating Subscription Management repositories. 2024-03-23T17:38:08.702 INFO:teuthology.orchestra.run.smithi086.stdout:Updating Subscription Management repositories. 2024-03-23T17:38:09.823 INFO:teuthology.orchestra.run.smithi092.stdout:61 files removed 2024-03-23T17:38:09.875 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-23T17:38:09.965 INFO:teuthology.orchestra.run.smithi152.stdout:61 files removed 2024-03-23T17:38:10.023 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-23T17:38:10.212 INFO:teuthology.orchestra.run.smithi092.stdout:Updating Subscription Management repositories. 2024-03-23T17:38:10.361 INFO:teuthology.orchestra.run.smithi152.stdout:Updating Subscription Management repositories. 2024-03-23T17:38:10.399 INFO:teuthology.orchestra.run.smithi086.stdout:61 files removed 2024-03-23T17:38:10.456 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-23T17:38:10.899 INFO:teuthology.orchestra.run.smithi086.stdout:Updating Subscription Management repositories. 2024-03-23T17:38:12.527 INFO:teuthology.orchestra.run.smithi152.stdout:ceph packages for x86_64 120 kB/s | 80 kB 00:00 2024-03-23T17:38:12.721 INFO:teuthology.orchestra.run.smithi092.stdout:ceph packages for x86_64 72 kB/s | 80 kB 00:01 2024-03-23T17:38:13.227 INFO:teuthology.orchestra.run.smithi152.stdout:ceph noarch packages 15 kB/s | 9.9 kB 00:00 2024-03-23T17:38:13.421 INFO:teuthology.orchestra.run.smithi092.stdout:ceph noarch packages 15 kB/s | 9.9 kB 00:00 2024-03-23T17:38:13.535 INFO:teuthology.orchestra.run.smithi086.stdout:ceph packages for x86_64 103 kB/s | 80 kB 00:00 2024-03-23T17:38:14.030 INFO:teuthology.orchestra.run.smithi092.stdout:ceph source packages 2.6 kB/s | 1.5 kB 00:00 2024-03-23T17:38:14.169 INFO:teuthology.orchestra.run.smithi086.stdout:ceph noarch packages 17 kB/s | 9.9 kB 00:00 2024-03-23T17:38:14.305 INFO:teuthology.orchestra.run.smithi092.stdout:Copr repo for python3-asyncssh owned by ceph 14 kB/s | 3.5 kB 00:00 2024-03-23T17:38:14.478 INFO:teuthology.orchestra.run.smithi152.stdout:ceph source packages 1.2 kB/s | 1.5 kB 00:01 2024-03-23T17:38:14.620 INFO:teuthology.orchestra.run.smithi152.stdout:Copr repo for python3-asyncssh owned by ceph 30 kB/s | 3.5 kB 00:00 2024-03-23T17:38:14.854 INFO:teuthology.orchestra.run.smithi086.stdout:ceph source packages 2.3 kB/s | 1.5 kB 00:00 2024-03-23T17:38:15.021 INFO:teuthology.orchestra.run.smithi086.stdout:Copr repo for python3-asyncssh owned by ceph 25 kB/s | 3.5 kB 00:00 2024-03-23T17:38:15.559 INFO:teuthology.orchestra.run.smithi092.stdout:Extra Packages for Enterprise Linux 13 MB/s | 16 MB 00:01 2024-03-23T17:38:16.113 INFO:teuthology.orchestra.run.smithi152.stdout:Extra Packages for Enterprise Linux 11 MB/s | 16 MB 00:01 2024-03-23T17:38:16.130 INFO:teuthology.orchestra.run.smithi086.stdout:Extra Packages for Enterprise Linux 15 MB/s | 16 MB 00:01 2024-03-23T17:38:19.115 INFO:teuthology.orchestra.run.smithi092.stdout:lab-extras 377 kB/s | 24 kB 00:00 2024-03-23T17:38:19.680 INFO:teuthology.orchestra.run.smithi152.stdout:lab-extras 424 kB/s | 24 kB 00:00 2024-03-23T17:38:20.339 INFO:teuthology.orchestra.run.smithi086.stdout:lab-extras 277 kB/s | 24 kB 00:00 2024-03-23T17:38:20.616 INFO:teuthology.orchestra.run.smithi092.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 46 MB/s | 66 MB 00:01 2024-03-23T17:38:21.090 INFO:teuthology.orchestra.run.smithi086.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 13 MB/s | 8.8 MB 00:00 2024-03-23T17:38:21.190 INFO:teuthology.orchestra.run.smithi152.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 46 MB/s | 66 MB 00:01 2024-03-23T17:38:24.309 INFO:teuthology.orchestra.run.smithi086.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 38 MB/s | 59 MB 00:01 2024-03-23T17:38:31.291 INFO:teuthology.orchestra.run.smithi152.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 13 MB/s | 8.8 MB 00:00 2024-03-23T17:38:31.458 INFO:teuthology.orchestra.run.smithi092.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 45 MB/s | 59 MB 00:01 2024-03-23T17:38:34.325 INFO:teuthology.orchestra.run.smithi152.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 44 MB/s | 59 MB 00:01 2024-03-23T17:38:36.410 INFO:teuthology.orchestra.run.smithi086.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 45 MB/s | 66 MB 00:01 2024-03-23T17:38:42.138 INFO:teuthology.orchestra.run.smithi092.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 14 MB/s | 8.8 MB 00:00 2024-03-23T17:38:49.306 INFO:teuthology.orchestra.run.smithi092.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2024-03-23T17:38:49.308 INFO:teuthology.orchestra.run.smithi092.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2024-03-23T17:38:49.327 INFO:teuthology.orchestra.run.smithi092.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2024-03-23T17:38:49.328 INFO:teuthology.orchestra.run.smithi092.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2024-03-23T17:38:49.330 INFO:teuthology.orchestra.run.smithi092.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2024-03-23T17:38:49.331 INFO:teuthology.orchestra.run.smithi092.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2024-03-23T17:38:49.334 INFO:teuthology.orchestra.run.smithi092.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2024-03-23T17:38:49.338 INFO:teuthology.orchestra.run.smithi092.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2024-03-23T17:38:49.340 INFO:teuthology.orchestra.run.smithi092.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2024-03-23T17:38:49.560 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-03-23T17:38:49.569 INFO:teuthology.orchestra.run.smithi152.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2024-03-23T17:38:49.571 INFO:teuthology.orchestra.run.smithi152.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2024-03-23T17:38:49.574 INFO:teuthology.orchestra.run.smithi092.stdout:=================================================================================================================================== 2024-03-23T17:38:49.574 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-03-23T17:38:49.574 INFO:teuthology.orchestra.run.smithi092.stdout:=================================================================================================================================== 2024-03-23T17:38:49.574 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-03-23T17:38:49.574 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-13.el8_9 rhel-8-for-x86_64-baseos-rpms 144 k 2024-03-23T17:38:49.574 INFO:teuthology.orchestra.run.smithi092.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2024-03-23T17:38:49.574 INFO:teuthology.orchestra.run.smithi092.stdout: ceph x86_64 2:18.2.2-954.g82316b04.el8 ceph 6.4 k 2024-03-23T17:38:49.575 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-base x86_64 2:18.2.2-954.g82316b04.el8 ceph 5.5 M 2024-03-23T17:38:49.575 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-fuse x86_64 2:18.2.2-954.g82316b04.el8 ceph 853 k 2024-03-23T17:38:49.575 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-954.g82316b04.el8 ceph 165 k 2024-03-23T17:38:49.575 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr x86_64 2:18.2.2-954.g82316b04.el8 ceph 1.6 M 2024-03-23T17:38:49.575 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-cephadm noarch 2:18.2.2-954.g82316b04.el8 ceph-noarch 138 k 2024-03-23T17:38:49.575 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-dashboard noarch 2:18.2.2-954.g82316b04.el8 ceph-noarch 1.7 M 2024-03-23T17:38:49.575 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-954.g82316b04.el8 ceph-noarch 7.4 M 2024-03-23T17:38:49.575 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-rook noarch 2:18.2.2-954.g82316b04.el8 ceph-noarch 49 k 2024-03-23T17:38:49.575 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-radosgw x86_64 2:18.2.2-954.g82316b04.el8 ceph 13 M 2024-03-23T17:38:49.575 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-test x86_64 2:18.2.2-954.g82316b04.el8 ceph 47 M 2024-03-23T17:38:49.575 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-volume noarch 2:18.2.2-954.g82316b04.el8 ceph-noarch 263 k 2024-03-23T17:38:49.576 INFO:teuthology.orchestra.run.smithi092.stdout: cephadm noarch 2:18.2.2-954.g82316b04.el8 ceph-noarch 172 k 2024-03-23T17:38:49.576 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-mirror x86_64 2:18.2.2-954.g82316b04.el8 ceph 261 k 2024-03-23T17:38:49.576 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-top noarch 2:18.2.2-954.g82316b04.el8 ceph-noarch 18 k 2024-03-23T17:38:49.576 INFO:teuthology.orchestra.run.smithi092.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2024-03-23T17:38:49.576 INFO:teuthology.orchestra.run.smithi092.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2024-03-23T17:38:49.576 INFO:teuthology.orchestra.run.smithi092.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2024-03-23T17:38:49.576 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs-devel x86_64 2:18.2.2-954.g82316b04.el8 ceph 31 k 2024-03-23T17:38:49.576 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs2 x86_64 2:18.2.2-954.g82316b04.el8 ceph 775 k 2024-03-23T17:38:49.576 INFO:teuthology.orchestra.run.smithi092.stdout: librados-devel x86_64 2:18.2.2-954.g82316b04.el8 ceph 139 k 2024-03-23T17:38:49.576 INFO:teuthology.orchestra.run.smithi092.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2024-03-23T17:38:49.576 INFO:teuthology.orchestra.run.smithi092.stdout: libuuid-devel x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 100 k 2024-03-23T17:38:49.576 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-devel x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 2.3 M 2024-03-23T17:38:49.577 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 1.5 M 2024-03-23T17:38:49.577 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-contrib x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 812 k 2024-03-23T17:38:49.577 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-server x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 5.1 M 2024-03-23T17:38:49.577 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cephfs x86_64 2:18.2.2-954.g82316b04.el8 ceph 197 k 2024-03-23T17:38:49.577 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rados x86_64 2:18.2.2-954.g82316b04.el8 ceph 371 k 2024-03-23T17:38:49.577 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rbd x86_64 2:18.2.2-954.g82316b04.el8 ceph 365 k 2024-03-23T17:38:49.577 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rgw x86_64 2:18.2.2-954.g82316b04.el8 ceph 107 k 2024-03-23T17:38:49.577 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-fuse x86_64 2:18.2.2-954.g82316b04.el8 ceph 95 k 2024-03-23T17:38:49.577 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-mirror x86_64 2:18.2.2-954.g82316b04.el8 ceph 3.5 M 2024-03-23T17:38:49.577 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-nbd x86_64 2:18.2.2-954.g82316b04.el8 ceph 181 k 2024-03-23T17:38:49.577 INFO:teuthology.orchestra.run.smithi092.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2024-03-23T17:38:49.577 INFO:teuthology.orchestra.run.smithi092.stdout:Upgrading: 2024-03-23T17:38:49.577 INFO:teuthology.orchestra.run.smithi092.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2024-03-23T17:38:49.577 INFO:teuthology.orchestra.run.smithi092.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2024-03-23T17:38:49.578 INFO:teuthology.orchestra.run.smithi092.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2024-03-23T17:38:49.578 INFO:teuthology.orchestra.run.smithi092.stdout: libblkid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-03-23T17:38:49.578 INFO:teuthology.orchestra.run.smithi092.stdout: libfdisk x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 254 k 2024-03-23T17:38:49.578 INFO:teuthology.orchestra.run.smithi092.stdout: libmount x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 237 k 2024-03-23T17:38:49.578 INFO:teuthology.orchestra.run.smithi092.stdout: librados2 x86_64 2:18.2.2-954.g82316b04.el8 ceph 3.7 M 2024-03-23T17:38:49.578 INFO:teuthology.orchestra.run.smithi092.stdout: librbd1 x86_64 2:18.2.2-954.g82316b04.el8 ceph 3.6 M 2024-03-23T17:38:49.578 INFO:teuthology.orchestra.run.smithi092.stdout: libsmartcols x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 179 k 2024-03-23T17:38:49.578 INFO:teuthology.orchestra.run.smithi092.stdout: libuuid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 99 k 2024-03-23T17:38:49.578 INFO:teuthology.orchestra.run.smithi092.stdout: openssl x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 711 k 2024-03-23T17:38:49.578 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-libs x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 1.5 M 2024-03-23T17:38:49.578 INFO:teuthology.orchestra.run.smithi092.stdout: util-linux x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 2.5 M 2024-03-23T17:38:49.578 INFO:teuthology.orchestra.run.smithi092.stdout: util-linux-user x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 103 k 2024-03-23T17:38:49.579 INFO:teuthology.orchestra.run.smithi092.stdout: xfsdump x86_64 3.1.8-7.el8_9 rhel-8-for-x86_64-baseos-rpms 333 k 2024-03-23T17:38:49.579 INFO:teuthology.orchestra.run.smithi092.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2024-03-23T17:38:49.579 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-03-23T17:38:49.579 INFO:teuthology.orchestra.run.smithi092.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2024-03-23T17:38:49.579 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-common x86_64 2:18.2.2-954.g82316b04.el8 ceph 22 M 2024-03-23T17:38:49.579 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-grafana-dashboards noarch 2:18.2.2-954.g82316b04.el8 ceph-noarch 23 k 2024-03-23T17:38:49.579 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mds x86_64 2:18.2.2-954.g82316b04.el8 ceph 2.3 M 2024-03-23T17:38:49.579 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-modules-core noarch 2:18.2.2-954.g82316b04.el8 ceph-noarch 245 k 2024-03-23T17:38:49.579 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mon x86_64 2:18.2.2-954.g82316b04.el8 ceph 4.8 M 2024-03-23T17:38:49.579 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-osd x86_64 2:18.2.2-954.g82316b04.el8 ceph 18 M 2024-03-23T17:38:49.579 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-prometheus-alerts noarch 2:18.2.2-954.g82316b04.el8 ceph-noarch 15 k 2024-03-23T17:38:49.579 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-selinux x86_64 2:18.2.2-954.g82316b04.el8 ceph 24 k 2024-03-23T17:38:49.579 INFO:teuthology.orchestra.run.smithi092.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-03-23T17:38:49.579 INFO:teuthology.orchestra.run.smithi092.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-03-23T17:38:49.580 INFO:teuthology.orchestra.run.smithi092.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-03-23T17:38:49.580 INFO:teuthology.orchestra.run.smithi092.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2024-03-23T17:38:49.580 INFO:teuthology.orchestra.run.smithi092.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2024-03-23T17:38:49.580 INFO:teuthology.orchestra.run.smithi092.stdout: libcephsqlite x86_64 2:18.2.2-954.g82316b04.el8 ceph 142 k 2024-03-23T17:38:49.580 INFO:teuthology.orchestra.run.smithi092.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2024-03-23T17:38:49.580 INFO:teuthology.orchestra.run.smithi092.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-03-23T17:38:49.580 INFO:teuthology.orchestra.run.smithi092.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-03-23T17:38:49.580 INFO:teuthology.orchestra.run.smithi092.stdout: libpq x86_64 13.11-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2024-03-23T17:38:49.580 INFO:teuthology.orchestra.run.smithi092.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-03-23T17:38:49.580 INFO:teuthology.orchestra.run.smithi092.stdout: librabbitmq x86_64 0.9.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 47 k 2024-03-23T17:38:49.580 INFO:teuthology.orchestra.run.smithi092.stdout: libradosstriper1 x86_64 2:18.2.2-954.g82316b04.el8 ceph 474 k 2024-03-23T17:38:49.580 INFO:teuthology.orchestra.run.smithi092.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-03-23T17:38:49.580 INFO:teuthology.orchestra.run.smithi092.stdout: librgw2 x86_64 2:18.2.2-954.g82316b04.el8 ceph 6.5 M 2024-03-23T17:38:49.580 INFO:teuthology.orchestra.run.smithi092.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2024-03-23T17:38:49.581 INFO:teuthology.orchestra.run.smithi092.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2024-03-23T17:38:49.581 INFO:teuthology.orchestra.run.smithi092.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-03-23T17:38:49.581 INFO:teuthology.orchestra.run.smithi092.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2024-03-23T17:38:49.581 INFO:teuthology.orchestra.run.smithi092.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2024-03-23T17:38:49.581 INFO:teuthology.orchestra.run.smithi092.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-03-23T17:38:49.581 INFO:teuthology.orchestra.run.smithi092.stdout: oniguruma x86_64 6.8.2-2.1.el8_9 rhel-8-for-x86_64-appstream-rpms 188 k 2024-03-23T17:38:49.581 INFO:teuthology.orchestra.run.smithi092.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-03-23T17:38:49.581 INFO:teuthology.orchestra.run.smithi092.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-03-23T17:38:49.581 INFO:teuthology.orchestra.run.smithi092.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2024-03-23T17:38:49.581 INFO:teuthology.orchestra.run.smithi092.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2024-03-23T17:38:49.581 INFO:teuthology.orchestra.run.smithi092.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-03-23T17:38:49.581 INFO:teuthology.orchestra.run.smithi092.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-03-23T17:38:49.581 INFO:teuthology.orchestra.run.smithi092.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-03-23T17:38:49.582 INFO:teuthology.orchestra.run.smithi092.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-03-23T17:38:49.582 INFO:teuthology.orchestra.run.smithi092.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-03-23T17:38:49.582 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-03-23T17:38:49.582 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-argparse x86_64 2:18.2.2-954.g82316b04.el8 ceph 45 k 2024-03-23T17:38:49.582 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-common x86_64 2:18.2.2-954.g82316b04.el8 ceph 126 k 2024-03-23T17:38:49.582 INFO:teuthology.orchestra.run.smithi092.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-03-23T17:38:49.582 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-03-23T17:38:49.582 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-03-23T17:38:49.582 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-03-23T17:38:49.582 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cryptography x86_64 3.2.1-7.el8_9 rhel-8-for-x86_64-baseos-rpms 559 k 2024-03-23T17:38:49.582 INFO:teuthology.orchestra.run.smithi092.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-03-23T17:38:49.583 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-03-23T17:38:49.583 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-03-23T17:38:49.583 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-03-23T17:38:49.583 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-03-23T17:38:49.583 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-03-23T17:38:49.583 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-03-23T17:38:49.583 INFO:teuthology.orchestra.run.smithi092.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-03-23T17:38:49.583 INFO:teuthology.orchestra.run.smithi092.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-03-23T17:38:49.583 INFO:teuthology.orchestra.run.smithi092.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-03-23T17:38:49.583 INFO:teuthology.orchestra.run.smithi092.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-03-23T17:38:49.583 INFO:teuthology.orchestra.run.smithi092.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-03-23T17:38:49.583 INFO:teuthology.orchestra.run.smithi092.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-03-23T17:38:49.584 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-03-23T17:38:49.584 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-03-23T17:38:49.584 INFO:teuthology.orchestra.run.smithi092.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-03-23T17:38:49.584 INFO:teuthology.orchestra.run.smithi092.stdout: python3-packaging noarch 16.8-9.el8 codeready-builder-for-rhel-8-x86_64-rpms 48 k 2024-03-23T17:38:49.584 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-03-23T17:38:49.584 INFO:teuthology.orchestra.run.smithi092.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-03-23T17:38:49.584 INFO:teuthology.orchestra.run.smithi092.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-03-23T17:38:49.584 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-03-23T17:38:49.584 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-03-23T17:38:49.584 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-03-23T17:38:49.584 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-03-23T17:38:49.584 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-03-23T17:38:49.584 INFO:teuthology.orchestra.run.smithi092.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-03-23T17:38:49.585 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-03-23T17:38:49.585 INFO:teuthology.orchestra.run.smithi092.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-03-23T17:38:49.585 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-03-23T17:38:49.585 INFO:teuthology.orchestra.run.smithi092.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-03-23T17:38:49.585 INFO:teuthology.orchestra.run.smithi092.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-03-23T17:38:49.585 INFO:teuthology.orchestra.run.smithi092.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-03-23T17:38:49.585 INFO:teuthology.orchestra.run.smithi092.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-03-23T17:38:49.585 INFO:teuthology.orchestra.run.smithi092.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-03-23T17:38:49.585 INFO:teuthology.orchestra.run.smithi092.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-03-23T17:38:49.585 INFO:teuthology.orchestra.run.smithi092.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-03-23T17:38:49.585 INFO:teuthology.orchestra.run.smithi092.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-03-23T17:38:49.585 INFO:teuthology.orchestra.run.smithi092.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-03-23T17:38:49.585 INFO:teuthology.orchestra.run.smithi092.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-03-23T17:38:49.585 INFO:teuthology.orchestra.run.smithi092.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-03-23T17:38:49.586 INFO:teuthology.orchestra.run.smithi092.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-03-23T17:38:49.586 INFO:teuthology.orchestra.run.smithi092.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-03-23T17:38:49.586 INFO:teuthology.orchestra.run.smithi092.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2024-03-23T17:38:49.586 INFO:teuthology.orchestra.run.smithi092.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-03-23T17:38:49.586 INFO:teuthology.orchestra.run.smithi092.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2024-03-23T17:38:49.586 INFO:teuthology.orchestra.run.smithi092.stdout:Enabling module streams: 2024-03-23T17:38:49.586 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql 10 2024-03-23T17:38:49.586 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-23T17:38:49.586 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-03-23T17:38:49.586 INFO:teuthology.orchestra.run.smithi092.stdout:=================================================================================================================================== 2024-03-23T17:38:49.586 INFO:teuthology.orchestra.run.smithi092.stdout:Install 130 Packages 2024-03-23T17:38:49.586 INFO:teuthology.orchestra.run.smithi092.stdout:Upgrade 16 Packages 2024-03-23T17:38:49.586 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-23T17:38:49.587 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 213 M 2024-03-23T17:38:49.587 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-03-23T17:38:49.589 INFO:teuthology.orchestra.run.smithi152.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2024-03-23T17:38:49.590 INFO:teuthology.orchestra.run.smithi152.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2024-03-23T17:38:49.592 INFO:teuthology.orchestra.run.smithi152.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2024-03-23T17:38:49.593 INFO:teuthology.orchestra.run.smithi152.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2024-03-23T17:38:49.595 INFO:teuthology.orchestra.run.smithi152.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2024-03-23T17:38:49.599 INFO:teuthology.orchestra.run.smithi152.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2024-03-23T17:38:49.601 INFO:teuthology.orchestra.run.smithi152.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2024-03-23T17:38:49.816 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2024-03-23T17:38:49.831 INFO:teuthology.orchestra.run.smithi152.stdout:=================================================================================================================================== 2024-03-23T17:38:49.831 INFO:teuthology.orchestra.run.smithi152.stdout: Package Arch Version Repository Size 2024-03-23T17:38:49.831 INFO:teuthology.orchestra.run.smithi152.stdout:=================================================================================================================================== 2024-03-23T17:38:49.831 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2024-03-23T17:38:49.831 INFO:teuthology.orchestra.run.smithi152.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-13.el8_9 rhel-8-for-x86_64-baseos-rpms 144 k 2024-03-23T17:38:49.831 INFO:teuthology.orchestra.run.smithi152.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2024-03-23T17:38:49.831 INFO:teuthology.orchestra.run.smithi152.stdout: ceph x86_64 2:18.2.2-954.g82316b04.el8 ceph 6.4 k 2024-03-23T17:38:49.831 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-base x86_64 2:18.2.2-954.g82316b04.el8 ceph 5.5 M 2024-03-23T17:38:49.831 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-fuse x86_64 2:18.2.2-954.g82316b04.el8 ceph 853 k 2024-03-23T17:38:49.831 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-954.g82316b04.el8 ceph 165 k 2024-03-23T17:38:49.831 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr x86_64 2:18.2.2-954.g82316b04.el8 ceph 1.6 M 2024-03-23T17:38:49.831 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-cephadm noarch 2:18.2.2-954.g82316b04.el8 ceph-noarch 138 k 2024-03-23T17:38:49.831 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-dashboard noarch 2:18.2.2-954.g82316b04.el8 ceph-noarch 1.7 M 2024-03-23T17:38:49.831 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-954.g82316b04.el8 ceph-noarch 7.4 M 2024-03-23T17:38:49.831 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-rook noarch 2:18.2.2-954.g82316b04.el8 ceph-noarch 49 k 2024-03-23T17:38:49.832 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-radosgw x86_64 2:18.2.2-954.g82316b04.el8 ceph 13 M 2024-03-23T17:38:49.832 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-test x86_64 2:18.2.2-954.g82316b04.el8 ceph 47 M 2024-03-23T17:38:49.832 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-volume noarch 2:18.2.2-954.g82316b04.el8 ceph-noarch 263 k 2024-03-23T17:38:49.832 INFO:teuthology.orchestra.run.smithi152.stdout: cephadm noarch 2:18.2.2-954.g82316b04.el8 ceph-noarch 172 k 2024-03-23T17:38:49.832 INFO:teuthology.orchestra.run.smithi152.stdout: cephfs-mirror x86_64 2:18.2.2-954.g82316b04.el8 ceph 261 k 2024-03-23T17:38:49.832 INFO:teuthology.orchestra.run.smithi152.stdout: cephfs-top noarch 2:18.2.2-954.g82316b04.el8 ceph-noarch 18 k 2024-03-23T17:38:49.832 INFO:teuthology.orchestra.run.smithi152.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2024-03-23T17:38:49.832 INFO:teuthology.orchestra.run.smithi152.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2024-03-23T17:38:49.832 INFO:teuthology.orchestra.run.smithi152.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2024-03-23T17:38:49.832 INFO:teuthology.orchestra.run.smithi152.stdout: libcephfs-devel x86_64 2:18.2.2-954.g82316b04.el8 ceph 31 k 2024-03-23T17:38:49.832 INFO:teuthology.orchestra.run.smithi152.stdout: libcephfs2 x86_64 2:18.2.2-954.g82316b04.el8 ceph 775 k 2024-03-23T17:38:49.832 INFO:teuthology.orchestra.run.smithi152.stdout: librados-devel x86_64 2:18.2.2-954.g82316b04.el8 ceph 139 k 2024-03-23T17:38:49.832 INFO:teuthology.orchestra.run.smithi152.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2024-03-23T17:38:49.832 INFO:teuthology.orchestra.run.smithi152.stdout: libuuid-devel x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 100 k 2024-03-23T17:38:49.832 INFO:teuthology.orchestra.run.smithi152.stdout: openssl-devel x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 2.3 M 2024-03-23T17:38:49.833 INFO:teuthology.orchestra.run.smithi152.stdout: postgresql x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 1.5 M 2024-03-23T17:38:49.833 INFO:teuthology.orchestra.run.smithi152.stdout: postgresql-contrib x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 812 k 2024-03-23T17:38:49.833 INFO:teuthology.orchestra.run.smithi152.stdout: postgresql-server x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 5.1 M 2024-03-23T17:38:49.833 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cephfs x86_64 2:18.2.2-954.g82316b04.el8 ceph 197 k 2024-03-23T17:38:49.833 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rados x86_64 2:18.2.2-954.g82316b04.el8 ceph 371 k 2024-03-23T17:38:49.833 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rbd x86_64 2:18.2.2-954.g82316b04.el8 ceph 365 k 2024-03-23T17:38:49.833 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rgw x86_64 2:18.2.2-954.g82316b04.el8 ceph 107 k 2024-03-23T17:38:49.833 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-fuse x86_64 2:18.2.2-954.g82316b04.el8 ceph 95 k 2024-03-23T17:38:49.833 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-mirror x86_64 2:18.2.2-954.g82316b04.el8 ceph 3.5 M 2024-03-23T17:38:49.833 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-nbd x86_64 2:18.2.2-954.g82316b04.el8 ceph 181 k 2024-03-23T17:38:49.833 INFO:teuthology.orchestra.run.smithi152.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2024-03-23T17:38:49.833 INFO:teuthology.orchestra.run.smithi152.stdout:Upgrading: 2024-03-23T17:38:49.833 INFO:teuthology.orchestra.run.smithi152.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2024-03-23T17:38:49.833 INFO:teuthology.orchestra.run.smithi152.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2024-03-23T17:38:49.833 INFO:teuthology.orchestra.run.smithi152.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2024-03-23T17:38:49.833 INFO:teuthology.orchestra.run.smithi152.stdout: libblkid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-03-23T17:38:49.834 INFO:teuthology.orchestra.run.smithi152.stdout: libfdisk x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 254 k 2024-03-23T17:38:49.834 INFO:teuthology.orchestra.run.smithi152.stdout: libmount x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 237 k 2024-03-23T17:38:49.834 INFO:teuthology.orchestra.run.smithi152.stdout: librados2 x86_64 2:18.2.2-954.g82316b04.el8 ceph 3.7 M 2024-03-23T17:38:49.834 INFO:teuthology.orchestra.run.smithi152.stdout: librbd1 x86_64 2:18.2.2-954.g82316b04.el8 ceph 3.6 M 2024-03-23T17:38:49.834 INFO:teuthology.orchestra.run.smithi152.stdout: libsmartcols x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 179 k 2024-03-23T17:38:49.834 INFO:teuthology.orchestra.run.smithi152.stdout: libuuid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 99 k 2024-03-23T17:38:49.834 INFO:teuthology.orchestra.run.smithi152.stdout: openssl x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 711 k 2024-03-23T17:38:49.834 INFO:teuthology.orchestra.run.smithi152.stdout: openssl-libs x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 1.5 M 2024-03-23T17:38:49.834 INFO:teuthology.orchestra.run.smithi152.stdout: util-linux x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 2.5 M 2024-03-23T17:38:49.834 INFO:teuthology.orchestra.run.smithi152.stdout: util-linux-user x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 103 k 2024-03-23T17:38:49.834 INFO:teuthology.orchestra.run.smithi152.stdout: xfsdump x86_64 3.1.8-7.el8_9 rhel-8-for-x86_64-baseos-rpms 333 k 2024-03-23T17:38:49.834 INFO:teuthology.orchestra.run.smithi152.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2024-03-23T17:38:49.834 INFO:teuthology.orchestra.run.smithi152.stdout:Installing dependencies: 2024-03-23T17:38:49.834 INFO:teuthology.orchestra.run.smithi152.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2024-03-23T17:38:49.834 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-common x86_64 2:18.2.2-954.g82316b04.el8 ceph 22 M 2024-03-23T17:38:49.834 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-grafana-dashboards noarch 2:18.2.2-954.g82316b04.el8 ceph-noarch 23 k 2024-03-23T17:38:49.835 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mds x86_64 2:18.2.2-954.g82316b04.el8 ceph 2.3 M 2024-03-23T17:38:49.835 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-modules-core noarch 2:18.2.2-954.g82316b04.el8 ceph-noarch 245 k 2024-03-23T17:38:49.835 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mon x86_64 2:18.2.2-954.g82316b04.el8 ceph 4.8 M 2024-03-23T17:38:49.835 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-osd x86_64 2:18.2.2-954.g82316b04.el8 ceph 18 M 2024-03-23T17:38:49.835 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-prometheus-alerts noarch 2:18.2.2-954.g82316b04.el8 ceph-noarch 15 k 2024-03-23T17:38:49.835 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-selinux x86_64 2:18.2.2-954.g82316b04.el8 ceph 24 k 2024-03-23T17:38:49.835 INFO:teuthology.orchestra.run.smithi152.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-03-23T17:38:49.835 INFO:teuthology.orchestra.run.smithi152.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-03-23T17:38:49.835 INFO:teuthology.orchestra.run.smithi152.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-03-23T17:38:49.835 INFO:teuthology.orchestra.run.smithi152.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2024-03-23T17:38:49.835 INFO:teuthology.orchestra.run.smithi152.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2024-03-23T17:38:49.835 INFO:teuthology.orchestra.run.smithi152.stdout: libcephsqlite x86_64 2:18.2.2-954.g82316b04.el8 ceph 142 k 2024-03-23T17:38:49.835 INFO:teuthology.orchestra.run.smithi152.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2024-03-23T17:38:49.835 INFO:teuthology.orchestra.run.smithi152.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-03-23T17:38:49.835 INFO:teuthology.orchestra.run.smithi152.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-03-23T17:38:49.836 INFO:teuthology.orchestra.run.smithi152.stdout: libpq x86_64 13.11-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2024-03-23T17:38:49.836 INFO:teuthology.orchestra.run.smithi152.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-03-23T17:38:49.836 INFO:teuthology.orchestra.run.smithi152.stdout: librabbitmq x86_64 0.9.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 47 k 2024-03-23T17:38:49.836 INFO:teuthology.orchestra.run.smithi152.stdout: libradosstriper1 x86_64 2:18.2.2-954.g82316b04.el8 ceph 474 k 2024-03-23T17:38:49.836 INFO:teuthology.orchestra.run.smithi152.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-03-23T17:38:49.836 INFO:teuthology.orchestra.run.smithi152.stdout: librgw2 x86_64 2:18.2.2-954.g82316b04.el8 ceph 6.5 M 2024-03-23T17:38:49.836 INFO:teuthology.orchestra.run.smithi152.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2024-03-23T17:38:49.836 INFO:teuthology.orchestra.run.smithi152.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2024-03-23T17:38:49.836 INFO:teuthology.orchestra.run.smithi152.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-03-23T17:38:49.836 INFO:teuthology.orchestra.run.smithi152.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2024-03-23T17:38:49.836 INFO:teuthology.orchestra.run.smithi152.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2024-03-23T17:38:49.836 INFO:teuthology.orchestra.run.smithi152.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-03-23T17:38:49.836 INFO:teuthology.orchestra.run.smithi152.stdout: oniguruma x86_64 6.8.2-2.1.el8_9 rhel-8-for-x86_64-appstream-rpms 188 k 2024-03-23T17:38:49.836 INFO:teuthology.orchestra.run.smithi152.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-03-23T17:38:49.836 INFO:teuthology.orchestra.run.smithi152.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-03-23T17:38:49.837 INFO:teuthology.orchestra.run.smithi152.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2024-03-23T17:38:49.837 INFO:teuthology.orchestra.run.smithi152.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2024-03-23T17:38:49.837 INFO:teuthology.orchestra.run.smithi152.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-03-23T17:38:49.837 INFO:teuthology.orchestra.run.smithi152.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-03-23T17:38:49.837 INFO:teuthology.orchestra.run.smithi152.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-03-23T17:38:49.837 INFO:teuthology.orchestra.run.smithi152.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-03-23T17:38:49.837 INFO:teuthology.orchestra.run.smithi152.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-03-23T17:38:49.837 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-03-23T17:38:49.837 INFO:teuthology.orchestra.run.smithi152.stdout: python3-ceph-argparse x86_64 2:18.2.2-954.g82316b04.el8 ceph 45 k 2024-03-23T17:38:49.837 INFO:teuthology.orchestra.run.smithi152.stdout: python3-ceph-common x86_64 2:18.2.2-954.g82316b04.el8 ceph 126 k 2024-03-23T17:38:49.837 INFO:teuthology.orchestra.run.smithi152.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-03-23T17:38:49.837 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-03-23T17:38:49.837 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-03-23T17:38:49.837 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-03-23T17:38:49.837 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cryptography x86_64 3.2.1-7.el8_9 rhel-8-for-x86_64-baseos-rpms 559 k 2024-03-23T17:38:49.838 INFO:teuthology.orchestra.run.smithi152.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-03-23T17:38:49.838 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-03-23T17:38:49.838 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-03-23T17:38:49.838 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-03-23T17:38:49.838 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-03-23T17:38:49.838 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-03-23T17:38:49.838 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-03-23T17:38:49.838 INFO:teuthology.orchestra.run.smithi152.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-03-23T17:38:49.838 INFO:teuthology.orchestra.run.smithi152.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-03-23T17:38:49.838 INFO:teuthology.orchestra.run.smithi152.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-03-23T17:38:49.838 INFO:teuthology.orchestra.run.smithi152.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-03-23T17:38:49.838 INFO:teuthology.orchestra.run.smithi152.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-03-23T17:38:49.838 INFO:teuthology.orchestra.run.smithi152.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-03-23T17:38:49.838 INFO:teuthology.orchestra.run.smithi152.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-03-23T17:38:49.838 INFO:teuthology.orchestra.run.smithi152.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-03-23T17:38:49.838 INFO:teuthology.orchestra.run.smithi152.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-03-23T17:38:49.839 INFO:teuthology.orchestra.run.smithi152.stdout: python3-packaging noarch 16.8-9.el8 codeready-builder-for-rhel-8-x86_64-rpms 48 k 2024-03-23T17:38:49.839 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-03-23T17:38:49.839 INFO:teuthology.orchestra.run.smithi152.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-03-23T17:38:49.839 INFO:teuthology.orchestra.run.smithi152.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-03-23T17:38:49.839 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-03-23T17:38:49.839 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-03-23T17:38:49.839 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-03-23T17:38:49.839 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-03-23T17:38:49.839 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-03-23T17:38:49.839 INFO:teuthology.orchestra.run.smithi152.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-03-23T17:38:49.839 INFO:teuthology.orchestra.run.smithi152.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-03-23T17:38:49.839 INFO:teuthology.orchestra.run.smithi152.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-03-23T17:38:49.839 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-03-23T17:38:49.839 INFO:teuthology.orchestra.run.smithi152.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-03-23T17:38:49.839 INFO:teuthology.orchestra.run.smithi152.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-03-23T17:38:49.839 INFO:teuthology.orchestra.run.smithi152.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-03-23T17:38:49.840 INFO:teuthology.orchestra.run.smithi152.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-03-23T17:38:49.840 INFO:teuthology.orchestra.run.smithi152.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-03-23T17:38:49.840 INFO:teuthology.orchestra.run.smithi152.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-03-23T17:38:49.840 INFO:teuthology.orchestra.run.smithi152.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-03-23T17:38:49.840 INFO:teuthology.orchestra.run.smithi152.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-03-23T17:38:49.840 INFO:teuthology.orchestra.run.smithi152.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-03-23T17:38:49.840 INFO:teuthology.orchestra.run.smithi152.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-03-23T17:38:49.840 INFO:teuthology.orchestra.run.smithi152.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-03-23T17:38:49.840 INFO:teuthology.orchestra.run.smithi152.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-03-23T17:38:49.840 INFO:teuthology.orchestra.run.smithi152.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-03-23T17:38:49.840 INFO:teuthology.orchestra.run.smithi152.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2024-03-23T17:38:49.840 INFO:teuthology.orchestra.run.smithi152.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-03-23T17:38:49.840 INFO:teuthology.orchestra.run.smithi152.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2024-03-23T17:38:49.840 INFO:teuthology.orchestra.run.smithi152.stdout:Enabling module streams: 2024-03-23T17:38:49.840 INFO:teuthology.orchestra.run.smithi152.stdout: postgresql 10 2024-03-23T17:38:49.840 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-23T17:38:49.840 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2024-03-23T17:38:49.841 INFO:teuthology.orchestra.run.smithi152.stdout:=================================================================================================================================== 2024-03-23T17:38:49.841 INFO:teuthology.orchestra.run.smithi152.stdout:Install 130 Packages 2024-03-23T17:38:49.841 INFO:teuthology.orchestra.run.smithi152.stdout:Upgrade 16 Packages 2024-03-23T17:38:49.841 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-23T17:38:49.841 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 213 M 2024-03-23T17:38:49.841 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2024-03-23T17:38:49.991 INFO:teuthology.orchestra.run.smithi092.stdout:(1/146): ceph-18.2.2-954.g82316b04.el8.x86_64.r 19 kB/s | 6.4 kB 00:00 2024-03-23T17:38:50.267 INFO:teuthology.orchestra.run.smithi152.stdout:(1/146): ceph-18.2.2-954.g82316b04.el8.x86_64.r 18 kB/s | 6.4 kB 00:00 2024-03-23T17:38:50.824 INFO:teuthology.orchestra.run.smithi092.stdout:(2/146): ceph-fuse-18.2.2-954.g82316b04.el8.x86 1.0 MB/s | 853 kB 00:00 2024-03-23T17:38:50.875 INFO:teuthology.orchestra.run.smithi152.stdout:(2/146): ceph-fuse-18.2.2-954.g82316b04.el8.x86 1.4 MB/s | 853 kB 00:00 2024-03-23T17:38:50.992 INFO:teuthology.orchestra.run.smithi152.stdout:(3/146): ceph-immutable-object-cache-18.2.2-954 1.4 MB/s | 165 kB 00:00 2024-03-23T17:38:51.016 INFO:teuthology.orchestra.run.smithi092.stdout:(3/146): ceph-immutable-object-cache-18.2.2-954 864 kB/s | 165 kB 00:00 2024-03-23T17:38:51.476 INFO:teuthology.orchestra.run.smithi152.stdout:(4/146): ceph-mds-18.2.2-954.g82316b04.el8.x86_ 4.7 MB/s | 2.3 MB 00:00 2024-03-23T17:38:51.877 INFO:teuthology.orchestra.run.smithi152.stdout:(5/146): ceph-mgr-18.2.2-954.g82316b04.el8.x86_ 3.9 MB/s | 1.6 MB 00:00 2024-03-23T17:38:52.020 INFO:teuthology.orchestra.run.smithi086.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2024-03-23T17:38:52.021 INFO:teuthology.orchestra.run.smithi086.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2024-03-23T17:38:52.041 INFO:teuthology.orchestra.run.smithi086.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2024-03-23T17:38:52.043 INFO:teuthology.orchestra.run.smithi086.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2024-03-23T17:38:52.044 INFO:teuthology.orchestra.run.smithi086.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2024-03-23T17:38:52.046 INFO:teuthology.orchestra.run.smithi086.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2024-03-23T17:38:52.049 INFO:teuthology.orchestra.run.smithi086.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2024-03-23T17:38:52.054 INFO:teuthology.orchestra.run.smithi086.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2024-03-23T17:38:52.055 INFO:teuthology.orchestra.run.smithi086.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2024-03-23T17:38:52.074 INFO:teuthology.orchestra.run.smithi092.stdout:(4/146): ceph-mds-18.2.2-954.g82316b04.el8.x86_ 2.2 MB/s | 2.3 MB 00:01 2024-03-23T17:38:52.280 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-03-23T17:38:52.294 INFO:teuthology.orchestra.run.smithi086.stdout:=================================================================================================================================== 2024-03-23T17:38:52.294 INFO:teuthology.orchestra.run.smithi086.stdout: Package Arch Version Repository Size 2024-03-23T17:38:52.294 INFO:teuthology.orchestra.run.smithi086.stdout:=================================================================================================================================== 2024-03-23T17:38:52.294 INFO:teuthology.orchestra.run.smithi086.stdout:Installing: 2024-03-23T17:38:52.295 INFO:teuthology.orchestra.run.smithi086.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-13.el8_9 rhel-8-for-x86_64-baseos-rpms 144 k 2024-03-23T17:38:52.295 INFO:teuthology.orchestra.run.smithi086.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2024-03-23T17:38:52.295 INFO:teuthology.orchestra.run.smithi086.stdout: ceph x86_64 2:18.2.2-954.g82316b04.el8 ceph 6.4 k 2024-03-23T17:38:52.295 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-base x86_64 2:18.2.2-954.g82316b04.el8 ceph 5.5 M 2024-03-23T17:38:52.295 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-fuse x86_64 2:18.2.2-954.g82316b04.el8 ceph 853 k 2024-03-23T17:38:52.295 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-954.g82316b04.el8 ceph 165 k 2024-03-23T17:38:52.295 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr x86_64 2:18.2.2-954.g82316b04.el8 ceph 1.6 M 2024-03-23T17:38:52.295 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-cephadm noarch 2:18.2.2-954.g82316b04.el8 ceph-noarch 138 k 2024-03-23T17:38:52.295 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-dashboard noarch 2:18.2.2-954.g82316b04.el8 ceph-noarch 1.7 M 2024-03-23T17:38:52.295 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-954.g82316b04.el8 ceph-noarch 7.4 M 2024-03-23T17:38:52.295 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-rook noarch 2:18.2.2-954.g82316b04.el8 ceph-noarch 49 k 2024-03-23T17:38:52.295 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-radosgw x86_64 2:18.2.2-954.g82316b04.el8 ceph 13 M 2024-03-23T17:38:52.295 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-test x86_64 2:18.2.2-954.g82316b04.el8 ceph 47 M 2024-03-23T17:38:52.296 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-volume noarch 2:18.2.2-954.g82316b04.el8 ceph-noarch 263 k 2024-03-23T17:38:52.296 INFO:teuthology.orchestra.run.smithi086.stdout: cephadm noarch 2:18.2.2-954.g82316b04.el8 ceph-noarch 172 k 2024-03-23T17:38:52.296 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-mirror x86_64 2:18.2.2-954.g82316b04.el8 ceph 261 k 2024-03-23T17:38:52.296 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-top noarch 2:18.2.2-954.g82316b04.el8 ceph-noarch 18 k 2024-03-23T17:38:52.296 INFO:teuthology.orchestra.run.smithi086.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2024-03-23T17:38:52.296 INFO:teuthology.orchestra.run.smithi086.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2024-03-23T17:38:52.296 INFO:teuthology.orchestra.run.smithi086.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2024-03-23T17:38:52.296 INFO:teuthology.orchestra.run.smithi086.stdout: libcephfs-devel x86_64 2:18.2.2-954.g82316b04.el8 ceph 31 k 2024-03-23T17:38:52.296 INFO:teuthology.orchestra.run.smithi086.stdout: libcephfs2 x86_64 2:18.2.2-954.g82316b04.el8 ceph 775 k 2024-03-23T17:38:52.296 INFO:teuthology.orchestra.run.smithi086.stdout: librados-devel x86_64 2:18.2.2-954.g82316b04.el8 ceph 139 k 2024-03-23T17:38:52.296 INFO:teuthology.orchestra.run.smithi086.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2024-03-23T17:38:52.296 INFO:teuthology.orchestra.run.smithi086.stdout: libuuid-devel x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 100 k 2024-03-23T17:38:52.296 INFO:teuthology.orchestra.run.smithi086.stdout: openssl-devel x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 2.3 M 2024-03-23T17:38:52.297 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 1.5 M 2024-03-23T17:38:52.297 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-contrib x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 812 k 2024-03-23T17:38:52.297 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-server x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 5.1 M 2024-03-23T17:38:52.297 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cephfs x86_64 2:18.2.2-954.g82316b04.el8 ceph 197 k 2024-03-23T17:38:52.297 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rados x86_64 2:18.2.2-954.g82316b04.el8 ceph 371 k 2024-03-23T17:38:52.297 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rbd x86_64 2:18.2.2-954.g82316b04.el8 ceph 365 k 2024-03-23T17:38:52.297 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rgw x86_64 2:18.2.2-954.g82316b04.el8 ceph 107 k 2024-03-23T17:38:52.297 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-fuse x86_64 2:18.2.2-954.g82316b04.el8 ceph 95 k 2024-03-23T17:38:52.297 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-mirror x86_64 2:18.2.2-954.g82316b04.el8 ceph 3.5 M 2024-03-23T17:38:52.297 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-nbd x86_64 2:18.2.2-954.g82316b04.el8 ceph 181 k 2024-03-23T17:38:52.297 INFO:teuthology.orchestra.run.smithi086.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2024-03-23T17:38:52.297 INFO:teuthology.orchestra.run.smithi086.stdout:Upgrading: 2024-03-23T17:38:52.297 INFO:teuthology.orchestra.run.smithi086.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2024-03-23T17:38:52.297 INFO:teuthology.orchestra.run.smithi086.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2024-03-23T17:38:52.298 INFO:teuthology.orchestra.run.smithi086.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2024-03-23T17:38:52.298 INFO:teuthology.orchestra.run.smithi086.stdout: libblkid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-03-23T17:38:52.298 INFO:teuthology.orchestra.run.smithi086.stdout: libfdisk x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 254 k 2024-03-23T17:38:52.298 INFO:teuthology.orchestra.run.smithi086.stdout: libmount x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 237 k 2024-03-23T17:38:52.298 INFO:teuthology.orchestra.run.smithi086.stdout: librados2 x86_64 2:18.2.2-954.g82316b04.el8 ceph 3.7 M 2024-03-23T17:38:52.298 INFO:teuthology.orchestra.run.smithi086.stdout: librbd1 x86_64 2:18.2.2-954.g82316b04.el8 ceph 3.6 M 2024-03-23T17:38:52.298 INFO:teuthology.orchestra.run.smithi086.stdout: libsmartcols x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 179 k 2024-03-23T17:38:52.298 INFO:teuthology.orchestra.run.smithi086.stdout: libuuid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 99 k 2024-03-23T17:38:52.298 INFO:teuthology.orchestra.run.smithi086.stdout: openssl x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 711 k 2024-03-23T17:38:52.298 INFO:teuthology.orchestra.run.smithi086.stdout: openssl-libs x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 1.5 M 2024-03-23T17:38:52.298 INFO:teuthology.orchestra.run.smithi086.stdout: util-linux x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 2.5 M 2024-03-23T17:38:52.298 INFO:teuthology.orchestra.run.smithi086.stdout: util-linux-user x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 103 k 2024-03-23T17:38:52.298 INFO:teuthology.orchestra.run.smithi086.stdout: xfsdump x86_64 3.1.8-7.el8_9 rhel-8-for-x86_64-baseos-rpms 333 k 2024-03-23T17:38:52.298 INFO:teuthology.orchestra.run.smithi086.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2024-03-23T17:38:52.299 INFO:teuthology.orchestra.run.smithi086.stdout:Installing dependencies: 2024-03-23T17:38:52.299 INFO:teuthology.orchestra.run.smithi086.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2024-03-23T17:38:52.299 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-common x86_64 2:18.2.2-954.g82316b04.el8 ceph 22 M 2024-03-23T17:38:52.299 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-grafana-dashboards noarch 2:18.2.2-954.g82316b04.el8 ceph-noarch 23 k 2024-03-23T17:38:52.299 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mds x86_64 2:18.2.2-954.g82316b04.el8 ceph 2.3 M 2024-03-23T17:38:52.299 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-modules-core noarch 2:18.2.2-954.g82316b04.el8 ceph-noarch 245 k 2024-03-23T17:38:52.299 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mon x86_64 2:18.2.2-954.g82316b04.el8 ceph 4.8 M 2024-03-23T17:38:52.299 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-osd x86_64 2:18.2.2-954.g82316b04.el8 ceph 18 M 2024-03-23T17:38:52.299 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-prometheus-alerts noarch 2:18.2.2-954.g82316b04.el8 ceph-noarch 15 k 2024-03-23T17:38:52.299 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-selinux x86_64 2:18.2.2-954.g82316b04.el8 ceph 24 k 2024-03-23T17:38:52.299 INFO:teuthology.orchestra.run.smithi086.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-03-23T17:38:52.299 INFO:teuthology.orchestra.run.smithi086.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-03-23T17:38:52.299 INFO:teuthology.orchestra.run.smithi086.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-03-23T17:38:52.299 INFO:teuthology.orchestra.run.smithi086.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2024-03-23T17:38:52.300 INFO:teuthology.orchestra.run.smithi086.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2024-03-23T17:38:52.300 INFO:teuthology.orchestra.run.smithi086.stdout: libcephsqlite x86_64 2:18.2.2-954.g82316b04.el8 ceph 142 k 2024-03-23T17:38:52.300 INFO:teuthology.orchestra.run.smithi086.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2024-03-23T17:38:52.300 INFO:teuthology.orchestra.run.smithi086.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-03-23T17:38:52.300 INFO:teuthology.orchestra.run.smithi086.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-03-23T17:38:52.300 INFO:teuthology.orchestra.run.smithi086.stdout: libpq x86_64 13.11-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2024-03-23T17:38:52.300 INFO:teuthology.orchestra.run.smithi086.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-03-23T17:38:52.300 INFO:teuthology.orchestra.run.smithi086.stdout: librabbitmq x86_64 0.9.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 47 k 2024-03-23T17:38:52.300 INFO:teuthology.orchestra.run.smithi086.stdout: libradosstriper1 x86_64 2:18.2.2-954.g82316b04.el8 ceph 474 k 2024-03-23T17:38:52.300 INFO:teuthology.orchestra.run.smithi086.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-03-23T17:38:52.300 INFO:teuthology.orchestra.run.smithi086.stdout: librgw2 x86_64 2:18.2.2-954.g82316b04.el8 ceph 6.5 M 2024-03-23T17:38:52.300 INFO:teuthology.orchestra.run.smithi086.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2024-03-23T17:38:52.300 INFO:teuthology.orchestra.run.smithi086.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2024-03-23T17:38:52.300 INFO:teuthology.orchestra.run.smithi086.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-03-23T17:38:52.300 INFO:teuthology.orchestra.run.smithi086.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2024-03-23T17:38:52.300 INFO:teuthology.orchestra.run.smithi086.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2024-03-23T17:38:52.301 INFO:teuthology.orchestra.run.smithi086.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-03-23T17:38:52.301 INFO:teuthology.orchestra.run.smithi086.stdout: oniguruma x86_64 6.8.2-2.1.el8_9 rhel-8-for-x86_64-appstream-rpms 188 k 2024-03-23T17:38:52.301 INFO:teuthology.orchestra.run.smithi086.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-03-23T17:38:52.301 INFO:teuthology.orchestra.run.smithi086.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-03-23T17:38:52.301 INFO:teuthology.orchestra.run.smithi086.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2024-03-23T17:38:52.301 INFO:teuthology.orchestra.run.smithi086.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2024-03-23T17:38:52.301 INFO:teuthology.orchestra.run.smithi086.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-03-23T17:38:52.301 INFO:teuthology.orchestra.run.smithi086.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-03-23T17:38:52.301 INFO:teuthology.orchestra.run.smithi086.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-03-23T17:38:52.301 INFO:teuthology.orchestra.run.smithi086.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-03-23T17:38:52.301 INFO:teuthology.orchestra.run.smithi086.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-03-23T17:38:52.301 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-03-23T17:38:52.301 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ceph-argparse x86_64 2:18.2.2-954.g82316b04.el8 ceph 45 k 2024-03-23T17:38:52.301 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ceph-common x86_64 2:18.2.2-954.g82316b04.el8 ceph 126 k 2024-03-23T17:38:52.301 INFO:teuthology.orchestra.run.smithi086.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-03-23T17:38:52.302 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-03-23T17:38:52.302 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-03-23T17:38:52.302 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-03-23T17:38:52.302 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cryptography x86_64 3.2.1-7.el8_9 rhel-8-for-x86_64-baseos-rpms 559 k 2024-03-23T17:38:52.302 INFO:teuthology.orchestra.run.smithi086.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-03-23T17:38:52.302 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-03-23T17:38:52.302 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-03-23T17:38:52.302 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-03-23T17:38:52.302 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-03-23T17:38:52.302 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-03-23T17:38:52.302 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-03-23T17:38:52.302 INFO:teuthology.orchestra.run.smithi086.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-03-23T17:38:52.302 INFO:teuthology.orchestra.run.smithi086.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-03-23T17:38:52.303 INFO:teuthology.orchestra.run.smithi086.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-03-23T17:38:52.303 INFO:teuthology.orchestra.run.smithi086.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-03-23T17:38:52.303 INFO:teuthology.orchestra.run.smithi086.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-03-23T17:38:52.303 INFO:teuthology.orchestra.run.smithi086.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-03-23T17:38:52.303 INFO:teuthology.orchestra.run.smithi086.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-03-23T17:38:52.303 INFO:teuthology.orchestra.run.smithi086.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-03-23T17:38:52.303 INFO:teuthology.orchestra.run.smithi086.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-03-23T17:38:52.303 INFO:teuthology.orchestra.run.smithi086.stdout: python3-packaging noarch 16.8-9.el8 codeready-builder-for-rhel-8-x86_64-rpms 48 k 2024-03-23T17:38:52.303 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-03-23T17:38:52.303 INFO:teuthology.orchestra.run.smithi086.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-03-23T17:38:52.303 INFO:teuthology.orchestra.run.smithi086.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-03-23T17:38:52.303 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-03-23T17:38:52.303 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-03-23T17:38:52.303 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-03-23T17:38:52.303 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-03-23T17:38:52.304 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-03-23T17:38:52.304 INFO:teuthology.orchestra.run.smithi086.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-03-23T17:38:52.304 INFO:teuthology.orchestra.run.smithi086.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-03-23T17:38:52.304 INFO:teuthology.orchestra.run.smithi086.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-03-23T17:38:52.304 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-03-23T17:38:52.304 INFO:teuthology.orchestra.run.smithi086.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-03-23T17:38:52.304 INFO:teuthology.orchestra.run.smithi086.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-03-23T17:38:52.304 INFO:teuthology.orchestra.run.smithi086.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-03-23T17:38:52.304 INFO:teuthology.orchestra.run.smithi086.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-03-23T17:38:52.304 INFO:teuthology.orchestra.run.smithi086.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-03-23T17:38:52.304 INFO:teuthology.orchestra.run.smithi086.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-03-23T17:38:52.304 INFO:teuthology.orchestra.run.smithi086.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-03-23T17:38:52.304 INFO:teuthology.orchestra.run.smithi086.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-03-23T17:38:52.304 INFO:teuthology.orchestra.run.smithi086.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-03-23T17:38:52.304 INFO:teuthology.orchestra.run.smithi086.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-03-23T17:38:52.305 INFO:teuthology.orchestra.run.smithi086.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-03-23T17:38:52.305 INFO:teuthology.orchestra.run.smithi086.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-03-23T17:38:52.305 INFO:teuthology.orchestra.run.smithi086.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-03-23T17:38:52.305 INFO:teuthology.orchestra.run.smithi086.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2024-03-23T17:38:52.305 INFO:teuthology.orchestra.run.smithi086.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-03-23T17:38:52.305 INFO:teuthology.orchestra.run.smithi086.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2024-03-23T17:38:52.305 INFO:teuthology.orchestra.run.smithi086.stdout:Enabling module streams: 2024-03-23T17:38:52.305 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql 10 2024-03-23T17:38:52.305 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:38:52.305 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction Summary 2024-03-23T17:38:52.305 INFO:teuthology.orchestra.run.smithi086.stdout:=================================================================================================================================== 2024-03-23T17:38:52.305 INFO:teuthology.orchestra.run.smithi086.stdout:Install 130 Packages 2024-03-23T17:38:52.305 INFO:teuthology.orchestra.run.smithi086.stdout:Upgrade 16 Packages 2024-03-23T17:38:52.305 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:38:52.306 INFO:teuthology.orchestra.run.smithi086.stdout:Total download size: 213 M 2024-03-23T17:38:52.306 INFO:teuthology.orchestra.run.smithi086.stdout:Downloading Packages: 2024-03-23T17:38:52.819 INFO:teuthology.orchestra.run.smithi086.stdout:(1/146): ceph-18.2.2-954.g82316b04.el8.x86_64.r 15 kB/s | 6.4 kB 00:00 2024-03-23T17:38:52.833 INFO:teuthology.orchestra.run.smithi092.stdout:(5/146): ceph-mgr-18.2.2-954.g82316b04.el8.x86_ 2.1 MB/s | 1.6 MB 00:00 2024-03-23T17:38:53.443 INFO:teuthology.orchestra.run.smithi152.stdout:(6/146): ceph-mon-18.2.2-954.g82316b04.el8.x86_ 3.1 MB/s | 4.8 MB 00:01 2024-03-23T17:38:53.827 INFO:teuthology.orchestra.run.smithi086.stdout:(2/146): ceph-fuse-18.2.2-954.g82316b04.el8.x86 846 kB/s | 853 kB 00:01 2024-03-23T17:38:53.875 INFO:teuthology.orchestra.run.smithi092.stdout:(6/146): ceph-base-18.2.2-954.g82316b04.el8.x86 1.3 MB/s | 5.5 MB 00:04 2024-03-23T17:38:54.094 INFO:teuthology.orchestra.run.smithi086.stdout:(3/146): ceph-immutable-object-cache-18.2.2-954 622 kB/s | 165 kB 00:00 2024-03-23T17:38:55.053 INFO:teuthology.orchestra.run.smithi086.stdout:(4/146): ceph-base-18.2.2-954.g82316b04.el8.x86 2.1 MB/s | 5.5 MB 00:02 2024-03-23T17:38:55.620 INFO:teuthology.orchestra.run.smithi086.stdout:(5/146): ceph-mgr-18.2.2-954.g82316b04.el8.x86_ 2.7 MB/s | 1.6 MB 00:00 2024-03-23T17:38:55.641 INFO:teuthology.orchestra.run.smithi092.stdout:(7/146): ceph-mon-18.2.2-954.g82316b04.el8.x86_ 1.7 MB/s | 4.8 MB 00:02 2024-03-23T17:38:55.870 INFO:teuthology.orchestra.run.smithi086.stdout:(6/146): ceph-mds-18.2.2-954.g82316b04.el8.x86_ 1.3 MB/s | 2.3 MB 00:01 2024-03-23T17:38:57.278 INFO:teuthology.orchestra.run.smithi086.stdout:(7/146): ceph-mon-18.2.2-954.g82316b04.el8.x86_ 2.9 MB/s | 4.8 MB 00:01 2024-03-23T17:39:00.387 INFO:teuthology.orchestra.run.smithi086.stdout:(8/146): ceph-common-18.2.2-954.g82316b04.el8.x 2.7 MB/s | 22 MB 00:08 2024-03-23T17:39:00.459 INFO:teuthology.orchestra.run.smithi152.stdout:(7/146): ceph-base-18.2.2-954.g82316b04.el8.x86 536 kB/s | 5.5 MB 00:10 2024-03-23T17:39:00.503 INFO:teuthology.orchestra.run.smithi086.stdout:(9/146): ceph-selinux-18.2.2-954.g82316b04.el8. 209 kB/s | 24 kB 00:00 2024-03-23T17:39:01.989 INFO:teuthology.orchestra.run.smithi086.stdout:(10/146): ceph-radosgw-18.2.2-954.g82316b04.el8 2.8 MB/s | 13 MB 00:04 2024-03-23T17:39:02.212 INFO:teuthology.orchestra.run.smithi086.stdout:(11/146): cephfs-mirror-18.2.2-954.g82316b04.el 1.1 MB/s | 261 kB 00:00 2024-03-23T17:39:02.321 INFO:teuthology.orchestra.run.smithi086.stdout:(12/146): libcephfs-devel-18.2.2-954.g82316b04. 286 kB/s | 31 kB 00:00 2024-03-23T17:39:02.696 INFO:teuthology.orchestra.run.smithi086.stdout:(13/146): libcephfs2-18.2.2-954.g82316b04.el8.x 2.0 MB/s | 775 kB 00:00 2024-03-23T17:39:02.813 INFO:teuthology.orchestra.run.smithi086.stdout:(14/146): libcephsqlite-18.2.2-954.g82316b04.el 1.2 MB/s | 142 kB 00:00 2024-03-23T17:39:02.932 INFO:teuthology.orchestra.run.smithi092.stdout:(8/146): ceph-common-18.2.2-954.g82316b04.el8.x 1.6 MB/s | 22 MB 00:13 2024-03-23T17:39:02.938 INFO:teuthology.orchestra.run.smithi086.stdout:(15/146): librados-devel-18.2.2-954.g82316b04.e 1.1 MB/s | 139 kB 00:00 2024-03-23T17:39:03.024 INFO:teuthology.orchestra.run.smithi092.stdout:(9/146): ceph-selinux-18.2.2-954.g82316b04.el8. 266 kB/s | 24 kB 00:00 2024-03-23T17:39:03.424 INFO:teuthology.orchestra.run.smithi086.stdout:(16/146): libradosstriper1-18.2.2-954.g82316b04 977 kB/s | 474 kB 00:00 2024-03-23T17:39:03.882 INFO:teuthology.orchestra.run.smithi092.stdout:(10/146): ceph-radosgw-18.2.2-954.g82316b04.el8 1.6 MB/s | 13 MB 00:08 2024-03-23T17:39:04.166 INFO:teuthology.orchestra.run.smithi092.stdout:(11/146): cephfs-mirror-18.2.2-954.g82316b04.el 922 kB/s | 261 kB 00:00 2024-03-23T17:39:04.274 INFO:teuthology.orchestra.run.smithi092.stdout:(12/146): libcephfs-devel-18.2.2-954.g82316b04. 286 kB/s | 31 kB 00:00 2024-03-23T17:39:04.758 INFO:teuthology.orchestra.run.smithi092.stdout:(13/146): libcephfs2-18.2.2-954.g82316b04.el8.x 1.6 MB/s | 775 kB 00:00 2024-03-23T17:39:04.883 INFO:teuthology.orchestra.run.smithi092.stdout:(14/146): libcephsqlite-18.2.2-954.g82316b04.el 1.1 MB/s | 142 kB 00:00 2024-03-23T17:39:05.008 INFO:teuthology.orchestra.run.smithi092.stdout:(15/146): librados-devel-18.2.2-954.g82316b04.e 1.1 MB/s | 139 kB 00:00 2024-03-23T17:39:05.255 INFO:teuthology.orchestra.run.smithi086.stdout:(17/146): ceph-osd-18.2.2-954.g82316b04.el8.x86 2.0 MB/s | 18 MB 00:09 2024-03-23T17:39:05.308 INFO:teuthology.orchestra.run.smithi092.stdout:(16/146): libradosstriper1-18.2.2-954.g82316b04 1.5 MB/s | 474 kB 00:00 2024-03-23T17:39:05.372 INFO:teuthology.orchestra.run.smithi086.stdout:(18/146): python3-ceph-argparse-18.2.2-954.g823 384 kB/s | 45 kB 00:00 2024-03-23T17:39:05.497 INFO:teuthology.orchestra.run.smithi086.stdout:(19/146): python3-ceph-common-18.2.2-954.g82316 1.0 MB/s | 126 kB 00:00 2024-03-23T17:39:05.623 INFO:teuthology.orchestra.run.smithi086.stdout:(20/146): python3-cephfs-18.2.2-954.g82316b04.e 1.5 MB/s | 197 kB 00:00 2024-03-23T17:39:05.898 INFO:teuthology.orchestra.run.smithi086.stdout:(21/146): python3-rados-18.2.2-954.g82316b04.el 1.3 MB/s | 371 kB 00:00 2024-03-23T17:39:06.082 INFO:teuthology.orchestra.run.smithi086.stdout:(22/146): python3-rbd-18.2.2-954.g82316b04.el8. 1.9 MB/s | 365 kB 00:00 2024-03-23T17:39:06.190 INFO:teuthology.orchestra.run.smithi086.stdout:(23/146): python3-rgw-18.2.2-954.g82316b04.el8. 984 kB/s | 107 kB 00:00 2024-03-23T17:39:06.299 INFO:teuthology.orchestra.run.smithi086.stdout:(24/146): rbd-fuse-18.2.2-954.g82316b04.el8.x86 880 kB/s | 95 kB 00:00 2024-03-23T17:39:06.399 INFO:teuthology.orchestra.run.smithi086.stdout:(25/146): librgw2-18.2.2-954.g82316b04.el8.x86_ 2.2 MB/s | 6.5 MB 00:02 2024-03-23T17:39:06.524 INFO:teuthology.orchestra.run.smithi086.stdout:(26/146): rbd-nbd-18.2.2-954.g82316b04.el8.x86_ 1.4 MB/s | 181 kB 00:00 2024-03-23T17:39:06.625 INFO:teuthology.orchestra.run.smithi086.stdout:(27/146): ceph-grafana-dashboards-18.2.2-954.g8 232 kB/s | 23 kB 00:00 2024-03-23T17:39:06.733 INFO:teuthology.orchestra.run.smithi086.stdout:(28/146): ceph-mgr-cephadm-18.2.2-954.g82316b04 1.2 MB/s | 138 kB 00:00 2024-03-23T17:39:07.559 INFO:teuthology.orchestra.run.smithi086.stdout:(29/146): ceph-mgr-dashboard-18.2.2-954.g82316b 2.1 MB/s | 1.7 MB 00:00 2024-03-23T17:39:07.642 INFO:teuthology.orchestra.run.smithi086.stdout:(30/146): rbd-mirror-18.2.2-954.g82316b04.el8.x 2.6 MB/s | 3.5 MB 00:01 2024-03-23T17:39:07.834 INFO:teuthology.orchestra.run.smithi086.stdout:(31/146): ceph-mgr-modules-core-18.2.2-954.g823 1.2 MB/s | 245 kB 00:00 2024-03-23T17:39:07.943 INFO:teuthology.orchestra.run.smithi086.stdout:(32/146): ceph-mgr-rook-18.2.2-954.g82316b04.el 454 kB/s | 49 kB 00:00 2024-03-23T17:39:08.051 INFO:teuthology.orchestra.run.smithi086.stdout:(33/146): ceph-prometheus-alerts-18.2.2-954.g82 136 kB/s | 15 kB 00:00 2024-03-23T17:39:08.318 INFO:teuthology.orchestra.run.smithi086.stdout:(34/146): ceph-volume-18.2.2-954.g82316b04.el8. 989 kB/s | 263 kB 00:00 2024-03-23T17:39:08.435 INFO:teuthology.orchestra.run.smithi086.stdout:(35/146): cephadm-18.2.2-954.g82316b04.el8.noar 1.4 MB/s | 172 kB 00:00 2024-03-23T17:39:08.544 INFO:teuthology.orchestra.run.smithi086.stdout:(36/146): cephfs-top-18.2.2-954.g82316b04.el8.n 167 kB/s | 18 kB 00:00 2024-03-23T17:39:08.644 INFO:teuthology.orchestra.run.smithi086.stdout:(37/146): gperftools-libs-2.7-9.el8.x86_64.rpm 3.0 MB/s | 306 kB 00:00 2024-03-23T17:39:08.678 INFO:teuthology.orchestra.run.smithi086.stdout:(38/146): liboath-2.6.2-3.el8.x86_64.rpm 1.7 MB/s | 59 kB 00:00 2024-03-23T17:39:08.700 INFO:teuthology.orchestra.run.smithi092.stdout:(17/146): librgw2-18.2.2-954.g82316b04.el8.x86_ 1.9 MB/s | 6.5 MB 00:03 2024-03-23T17:39:08.712 INFO:teuthology.orchestra.run.smithi086.stdout:(39/146): libunwind-1.3.1-3.el8.x86_64.rpm 2.2 MB/s | 75 kB 00:00 2024-03-23T17:39:08.778 INFO:teuthology.orchestra.run.smithi086.stdout:(40/146): python3-asyncssh-2.7.0-2.el8.noarch.r 7.0 MB/s | 476 kB 00:00 2024-03-23T17:39:08.812 INFO:teuthology.orchestra.run.smithi086.stdout:(41/146): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 1.3 MB/s | 44 kB 00:00 2024-03-23T17:39:08.817 INFO:teuthology.orchestra.run.smithi092.stdout:(18/146): python3-ceph-argparse-18.2.2-954.g823 383 kB/s | 45 kB 00:00 2024-03-23T17:39:08.846 INFO:teuthology.orchestra.run.smithi086.stdout:(42/146): python3-beautifulsoup4-4.6.3-2.el8.1. 5.4 MB/s | 185 kB 00:00 2024-03-23T17:39:08.879 INFO:teuthology.orchestra.run.smithi086.stdout:(43/146): python3-cachetools-3.1.1-4.el8.noarch 987 kB/s | 33 kB 00:00 2024-03-23T17:39:08.913 INFO:teuthology.orchestra.run.smithi086.stdout:(44/146): python3-certifi-2018.10.15-7.el8.noar 489 kB/s | 16 kB 00:00 2024-03-23T17:39:08.934 INFO:teuthology.orchestra.run.smithi092.stdout:(19/146): python3-ceph-common-18.2.2-954.g82316 1.1 MB/s | 126 kB 00:00 2024-03-23T17:39:08.946 INFO:teuthology.orchestra.run.smithi086.stdout:(45/146): python3-cheroot-8.5.2-1.el8.noarch.rp 5.1 MB/s | 173 kB 00:00 2024-03-23T17:39:09.022 INFO:teuthology.orchestra.run.smithi086.stdout:(46/146): python3-cherrypy-18.4.0-1.el8.noarch. 5.0 MB/s | 384 kB 00:00 2024-03-23T17:39:09.056 INFO:teuthology.orchestra.run.smithi086.stdout:(47/146): python3-google-auth-1.1.1-10.el8.noar 2.4 MB/s | 81 kB 00:00 2024-03-23T17:39:09.090 INFO:teuthology.orchestra.run.smithi086.stdout:(48/146): python3-jaraco-6.2-6.el8.noarch.rpm 332 kB/s | 11 kB 00:00 2024-03-23T17:39:09.123 INFO:teuthology.orchestra.run.smithi086.stdout:(49/146): python3-jaraco-functools-2.0-4.el8.no 527 kB/s | 18 kB 00:00 2024-03-23T17:39:09.142 INFO:teuthology.orchestra.run.smithi092.stdout:(20/146): python3-cephfs-18.2.2-954.g82316b04.e 945 kB/s | 197 kB 00:00 2024-03-23T17:39:09.182 INFO:teuthology.orchestra.run.smithi086.stdout:(50/146): python3-kubernetes-11.0.0-6.el8.noarc 20 MB/s | 1.2 MB 00:00 2024-03-23T17:39:09.216 INFO:teuthology.orchestra.run.smithi086.stdout:(51/146): python3-logutils-0.3.5-11.el8.noarch. 1.4 MB/s | 49 kB 00:00 2024-03-23T17:39:09.249 INFO:teuthology.orchestra.run.smithi086.stdout:(52/146): python3-more-itertools-7.2.0-3.el8.no 1.7 MB/s | 59 kB 00:00 2024-03-23T17:39:09.291 INFO:teuthology.orchestra.run.smithi086.stdout:(53/146): python3-natsort-7.1.1-2.el8.noarch.rp 1.4 MB/s | 60 kB 00:00 2024-03-23T17:39:09.333 INFO:teuthology.orchestra.run.smithi086.stdout:(54/146): python3-pecan-1.3.2-9.el8.noarch.rpm 6.6 MB/s | 283 kB 00:00 2024-03-23T17:39:09.367 INFO:teuthology.orchestra.run.smithi086.stdout:(55/146): python3-portend-2.6-1.el8.noarch.rpm 469 kB/s | 16 kB 00:00 2024-03-23T17:39:09.400 INFO:teuthology.orchestra.run.smithi086.stdout:(56/146): python3-repoze-lru-0.7-6.el8.noarch.r 1.0 MB/s | 34 kB 00:00 2024-03-23T17:39:09.434 INFO:teuthology.orchestra.run.smithi086.stdout:(57/146): python3-routes-2.4.1-12.el8.noarch.rp 5.8 MB/s | 196 kB 00:00 2024-03-23T17:39:09.499 INFO:teuthology.orchestra.run.smithi086.stdout:(58/146): python3-rsa-4.9-2.el8.noarch.rpm 991 kB/s | 64 kB 00:00 2024-03-23T17:39:09.543 INFO:teuthology.orchestra.run.smithi086.stdout:(59/146): python3-simplegeneric-0.8.1-17.el8.no 430 kB/s | 19 kB 00:00 2024-03-23T17:39:09.576 INFO:teuthology.orchestra.run.smithi092.stdout:(21/146): ceph-osd-18.2.2-954.g82316b04.el8.x86 1.2 MB/s | 18 MB 00:15 2024-03-23T17:39:09.577 INFO:teuthology.orchestra.run.smithi086.stdout:(60/146): python3-singledispatch-3.4.0.3-18.el8 713 kB/s | 24 kB 00:00 2024-03-23T17:39:09.593 INFO:teuthology.orchestra.run.smithi092.stdout:(22/146): python3-rados-18.2.2-954.g82316b04.el 825 kB/s | 371 kB 00:00 2024-03-23T17:39:09.610 INFO:teuthology.orchestra.run.smithi086.stdout:(61/146): python3-tempora-1.14.1-5.el8.noarch.r 884 kB/s | 29 kB 00:00 2024-03-23T17:39:09.644 INFO:teuthology.orchestra.run.smithi086.stdout:(62/146): python3-trustme-0.6.0-4.el8.noarch.rp 822 kB/s | 27 kB 00:00 2024-03-23T17:39:09.678 INFO:teuthology.orchestra.run.smithi086.stdout:(63/146): python3-waitress-1.4.3-1.el8.noarch.r 7.5 MB/s | 255 kB 00:00 2024-03-23T17:39:09.711 INFO:teuthology.orchestra.run.smithi086.stdout:(64/146): python3-webob-1.8.5-1.el8.1.noarch.rp 7.4 MB/s | 251 kB 00:00 2024-03-23T17:39:09.718 INFO:teuthology.orchestra.run.smithi092.stdout:(23/146): python3-rgw-18.2.2-954.g82316b04.el8. 854 kB/s | 107 kB 00:00 2024-03-23T17:39:09.736 INFO:teuthology.orchestra.run.smithi086.stdout:(65/146): python3-websocket-client-0.56.0-5.el8 2.4 MB/s | 61 kB 00:00 2024-03-23T17:39:09.762 INFO:teuthology.orchestra.run.smithi086.stdout:(66/146): python3-webtest-2.0.33-1.el8.noarch.r 3.3 MB/s | 85 kB 00:00 2024-03-23T17:39:09.787 INFO:teuthology.orchestra.run.smithi086.stdout:(67/146): python3-zc-lockfile-2.0-2.el8.noarch. 931 kB/s | 23 kB 00:00 2024-03-23T17:39:09.835 INFO:teuthology.orchestra.run.smithi092.stdout:(24/146): rbd-fuse-18.2.2-954.g82316b04.el8.x86 815 kB/s | 95 kB 00:00 2024-03-23T17:39:09.854 INFO:teuthology.orchestra.run.smithi086.stdout:(68/146): thrift-0.13.0-2.el8.x86_64.rpm 26 MB/s | 1.7 MB 00:00 2024-03-23T17:39:09.860 INFO:teuthology.orchestra.run.smithi092.stdout:(25/146): python3-rbd-18.2.2-954.g82316b04.el8. 1.3 MB/s | 365 kB 00:00 2024-03-23T17:39:09.879 INFO:teuthology.orchestra.run.smithi086.stdout:(69/146): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2024-03-23T17:39:10.030 INFO:teuthology.orchestra.run.smithi086.stdout:(70/146): python3-packaging-16.8-9.el8.noarch.r 322 kB/s | 48 kB 00:00 2024-03-23T17:39:10.077 INFO:teuthology.orchestra.run.smithi092.stdout:(26/146): rbd-nbd-18.2.2-954.g82316b04.el8.x86_ 838 kB/s | 181 kB 00:00 2024-03-23T17:39:10.174 INFO:teuthology.orchestra.run.smithi086.stdout:(71/146): python3-jsonpointer-1.10-11.el8.noarc 141 kB/s | 20 kB 00:00 2024-03-23T17:39:10.186 INFO:teuthology.orchestra.run.smithi092.stdout:(27/146): ceph-grafana-dashboards-18.2.2-954.g8 215 kB/s | 23 kB 00:00 2024-03-23T17:39:10.319 INFO:teuthology.orchestra.run.smithi092.stdout:(28/146): ceph-mgr-cephadm-18.2.2-954.g82316b04 1.0 MB/s | 138 kB 00:00 2024-03-23T17:39:10.330 INFO:teuthology.orchestra.run.smithi086.stdout:(72/146): python3-prettytable-0.7.2-14.el8.noar 277 kB/s | 44 kB 00:00 2024-03-23T17:39:10.515 INFO:teuthology.orchestra.run.smithi086.stdout:(73/146): python3-pyasn1-modules-0.3.7-6.el8.no 597 kB/s | 110 kB 00:00 2024-03-23T17:39:10.656 INFO:teuthology.orchestra.run.smithi086.stdout:(74/146): python3-jsonpatch-1.21-2.el8.noarch.r 188 kB/s | 27 kB 00:00 2024-03-23T17:39:10.773 INFO:teuthology.orchestra.run.smithi086.stdout:(75/146): ceph-mgr-diskprediction-local-18.2.2- 2.3 MB/s | 7.4 MB 00:03 2024-03-23T17:39:10.798 INFO:teuthology.orchestra.run.smithi086.stdout:(76/146): python3-pyasn1-0.3.7-6.el8.noarch.rpm 890 kB/s | 126 kB 00:00 2024-03-23T17:39:10.924 INFO:teuthology.orchestra.run.smithi086.stdout:(77/146): bison-3.0.4-10.el8.x86_64.rpm 5.6 MB/s | 688 kB 00:00 2024-03-23T17:39:10.957 INFO:teuthology.orchestra.run.smithi086.stdout:(78/146): python3-werkzeug-0.12.2-4.el8.noarch. 2.4 MB/s | 457 kB 00:00 2024-03-23T17:39:11.083 INFO:teuthology.orchestra.run.smithi086.stdout:(79/146): flex-2.6.1-9.el8.x86_64.rpm 2.0 MB/s | 320 kB 00:00 2024-03-23T17:39:11.133 INFO:teuthology.orchestra.run.smithi086.stdout:(80/146): libtool-2.4.6-25.el8.x86_64.rpm 3.9 MB/s | 709 kB 00:00 2024-03-23T17:39:11.183 INFO:teuthology.orchestra.run.smithi086.stdout:(81/146): python3-markupsafe-0.23-19.el8.x86_64 387 kB/s | 39 kB 00:00 2024-03-23T17:39:11.317 INFO:teuthology.orchestra.run.smithi086.stdout:(82/146): uuid-1.6.2-43.el8.x86_64.rpm 480 kB/s | 64 kB 00:00 2024-03-23T17:39:11.345 INFO:teuthology.orchestra.run.smithi092.stdout:(29/146): ceph-mgr-dashboard-18.2.2-954.g82316b 1.7 MB/s | 1.7 MB 00:01 2024-03-23T17:39:11.350 INFO:teuthology.orchestra.run.smithi086.stdout:(83/146): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.2 MB/s | 259 kB 00:00 2024-03-23T17:39:11.409 INFO:teuthology.orchestra.run.smithi086.stdout:(84/146): python3-pyOpenSSL-19.0.0-1.el8.noarch 1.1 MB/s | 103 kB 00:00 2024-03-23T17:39:11.593 INFO:teuthology.orchestra.run.smithi086.stdout:(85/146): python3-babel-2.5.1-7.el8.noarch.rpm 20 MB/s | 4.8 MB 00:00 2024-03-23T17:39:11.645 INFO:teuthology.orchestra.run.smithi086.stdout:(86/146): python3-jinja2-2.10.1-3.el8.noarch.rp 2.4 MB/s | 538 kB 00:00 2024-03-23T17:39:11.785 INFO:teuthology.orchestra.run.smithi086.stdout:(87/146): python3-numpy-1.14.3-10.el8.x86_64.rp 19 MB/s | 3.7 MB 00:00 2024-03-23T17:39:11.827 INFO:teuthology.orchestra.run.smithi086.stdout:(88/146): socat-1.7.4.1-1.el8.x86_64.rpm 1.6 MB/s | 323 kB 00:00 2024-03-23T17:39:11.919 INFO:teuthology.orchestra.run.smithi086.stdout:(89/146): python3-numpy-f2py-1.14.3-10.el8.x86_ 1.6 MB/s | 225 kB 00:00 2024-03-23T17:39:11.953 INFO:teuthology.orchestra.run.smithi086.stdout:(90/146): librdkafka-0.11.4-3.el8.x86_64.rpm 2.8 MB/s | 354 kB 00:00 2024-03-23T17:39:11.978 INFO:teuthology.orchestra.run.smithi092.stdout:(30/146): rbd-mirror-18.2.2-954.g82316b04.el8.x 1.6 MB/s | 3.5 MB 00:02 2024-03-23T17:39:12.095 INFO:teuthology.orchestra.run.smithi086.stdout:(91/146): python3-mako-1.0.6-14.el8.noarch.rpm 1.1 MB/s | 157 kB 00:00 2024-03-23T17:39:12.150 INFO:teuthology.orchestra.run.smithi152.stdout:(8/146): ceph-osd-18.2.2-954.g82316b04.el8.x86_ 1.0 MB/s | 18 MB 00:18 2024-03-23T17:39:12.203 INFO:teuthology.orchestra.run.smithi086.stdout:(92/146): openblas-threads-0.3.15-6.el8.x86_64. 18 MB/s | 5.2 MB 00:00 2024-03-23T17:39:12.212 INFO:teuthology.orchestra.run.smithi092.stdout:(31/146): ceph-mgr-modules-core-18.2.2-954.g823 1.0 MB/s | 245 kB 00:00 2024-03-23T17:39:12.242 INFO:teuthology.orchestra.run.smithi152.stdout:(9/146): ceph-selinux-18.2.2-954.g82316b04.el8. 266 kB/s | 24 kB 00:00 2024-03-23T17:39:12.312 INFO:teuthology.orchestra.run.smithi092.stdout:(32/146): ceph-mgr-rook-18.2.2-954.g82316b04.el 492 kB/s | 49 kB 00:00 2024-03-23T17:39:12.320 INFO:teuthology.orchestra.run.smithi086.stdout:(93/146): openblas-0.3.15-6.el8.x86_64.rpm 22 MB/s | 5.0 MB 00:00 2024-03-23T17:39:12.354 INFO:teuthology.orchestra.run.smithi086.stdout:(94/146): automake-1.16.1-8.el8.noarch.rpm 4.6 MB/s | 714 kB 00:00 2024-03-23T17:39:12.404 INFO:teuthology.orchestra.run.smithi086.stdout:(95/146): libpq-13.11-1.el8.x86_64.rpm 2.3 MB/s | 198 kB 00:00 2024-03-23T17:39:12.429 INFO:teuthology.orchestra.run.smithi092.stdout:(33/146): ceph-prometheus-alerts-18.2.2-954.g82 126 kB/s | 15 kB 00:00 2024-03-23T17:39:12.471 INFO:teuthology.orchestra.run.smithi086.stdout:(96/146): jq-1.6-7.el8.x86_64.rpm 1.7 MB/s | 202 kB 00:00 2024-03-23T17:39:12.504 INFO:teuthology.orchestra.run.smithi086.stdout:(97/146): python3-pytz-2017.2-11.el8.noarch.rpm 541 kB/s | 54 kB 00:00 2024-03-23T17:39:12.638 INFO:teuthology.orchestra.run.smithi086.stdout:(98/146): oniguruma-6.8.2-2.1.el8_9.x86_64.rpm 1.4 MB/s | 188 kB 00:00 2024-03-23T17:39:12.721 INFO:teuthology.orchestra.run.smithi092.stdout:(34/146): ceph-volume-18.2.2-954.g82316b04.el8. 902 kB/s | 263 kB 00:00 2024-03-23T17:39:12.780 INFO:teuthology.orchestra.run.smithi086.stdout:(99/146): postgresql-10.23-4.module+el8.9.0+213 11 MB/s | 1.5 MB 00:00 2024-03-23T17:39:12.929 INFO:teuthology.orchestra.run.smithi092.stdout:(35/146): cephadm-18.2.2-954.g82316b04.el8.noar 829 kB/s | 172 kB 00:00 2024-03-23T17:39:12.980 INFO:teuthology.orchestra.run.smithi086.stdout:(100/146): python3-scipy-1.0.0-21.module+el8.9. 27 MB/s | 14 MB 00:00 2024-03-23T17:39:13.054 INFO:teuthology.orchestra.run.smithi086.stdout:(101/146): postgresql-contrib-10.23-4.module+el 2.9 MB/s | 812 kB 00:00 2024-03-23T17:39:13.055 INFO:teuthology.orchestra.run.smithi092.stdout:(36/146): cephfs-top-18.2.2-954.g82316b04.el8.n 145 kB/s | 18 kB 00:00 2024-03-23T17:39:13.155 INFO:teuthology.orchestra.run.smithi092.stdout:(37/146): gperftools-libs-2.7-9.el8.x86_64.rpm 3.0 MB/s | 306 kB 00:00 2024-03-23T17:39:13.181 INFO:teuthology.orchestra.run.smithi086.stdout:(102/146): libverto-devel-0.3.0-5.el8.x86_64.rp 142 kB/s | 18 kB 00:00 2024-03-23T17:39:13.188 INFO:teuthology.orchestra.run.smithi092.stdout:(38/146): liboath-2.6.2-3.el8.x86_64.rpm 1.7 MB/s | 59 kB 00:00 2024-03-23T17:39:13.222 INFO:teuthology.orchestra.run.smithi092.stdout:(39/146): libunwind-1.3.1-3.el8.x86_64.rpm 2.2 MB/s | 75 kB 00:00 2024-03-23T17:39:13.256 INFO:teuthology.orchestra.run.smithi086.stdout:(103/146): postgresql-server-10.23-4.module+el8 18 MB/s | 5.1 MB 00:00 2024-03-23T17:39:13.273 INFO:teuthology.orchestra.run.smithi092.stdout:(40/146): python3-asyncssh-2.7.0-2.el8.noarch.r 9.3 MB/s | 476 kB 00:00 2024-03-23T17:39:13.306 INFO:teuthology.orchestra.run.smithi092.stdout:(41/146): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 1.3 MB/s | 44 kB 00:00 2024-03-23T17:39:13.340 INFO:teuthology.orchestra.run.smithi092.stdout:(42/146): python3-beautifulsoup4-4.6.3-2.el8.1. 5.5 MB/s | 185 kB 00:00 2024-03-23T17:39:13.348 INFO:teuthology.orchestra.run.smithi086.stdout:(104/146): python3-jwt-1.6.1-2.el8.noarch.rpm 255 kB/s | 43 kB 00:00 2024-03-23T17:39:13.373 INFO:teuthology.orchestra.run.smithi092.stdout:(43/146): python3-cachetools-3.1.1-4.el8.noarch 993 kB/s | 33 kB 00:00 2024-03-23T17:39:13.438 INFO:teuthology.orchestra.run.smithi092.stdout:(44/146): python3-certifi-2018.10.15-7.el8.noar 255 kB/s | 16 kB 00:00 2024-03-23T17:39:13.440 INFO:teuthology.orchestra.run.smithi086.stdout:(105/146): python3-oauthlib-2.1.0-1.el8.noarch. 844 kB/s | 155 kB 00:00 2024-03-23T17:39:13.466 INFO:teuthology.orchestra.run.smithi086.stdout:(106/146): python3-pycparser-2.14-14.el8.noarch 932 kB/s | 109 kB 00:00 2024-03-23T17:39:13.482 INFO:teuthology.orchestra.run.smithi092.stdout:(45/146): python3-cheroot-8.5.2-1.el8.noarch.rp 3.8 MB/s | 173 kB 00:00 2024-03-23T17:39:13.524 INFO:teuthology.orchestra.run.smithi092.stdout:(46/146): python3-cherrypy-18.4.0-1.el8.noarch. 9.1 MB/s | 384 kB 00:00 2024-03-23T17:39:13.558 INFO:teuthology.orchestra.run.smithi092.stdout:(47/146): python3-google-auth-1.1.1-10.el8.noar 2.4 MB/s | 81 kB 00:00 2024-03-23T17:39:13.591 INFO:teuthology.orchestra.run.smithi086.stdout:(107/146): python3-requests-oauthlib-1.0.0-1.el 284 kB/s | 43 kB 00:00 2024-03-23T17:39:13.592 INFO:teuthology.orchestra.run.smithi092.stdout:(48/146): python3-jaraco-6.2-6.el8.noarch.rpm 334 kB/s | 11 kB 00:00 2024-03-23T17:39:13.616 INFO:teuthology.orchestra.run.smithi086.stdout:(108/146): libaio-devel-0.3.112-1.el8.x86_64.rp 124 kB/s | 19 kB 00:00 2024-03-23T17:39:13.625 INFO:teuthology.orchestra.run.smithi092.stdout:(49/146): python3-jaraco-functools-2.0-4.el8.no 527 kB/s | 18 kB 00:00 2024-03-23T17:39:13.684 INFO:teuthology.orchestra.run.smithi092.stdout:(50/146): python3-kubernetes-11.0.0-6.el8.noarc 20 MB/s | 1.2 MB 00:00 2024-03-23T17:39:13.717 INFO:teuthology.orchestra.run.smithi092.stdout:(51/146): python3-logutils-0.3.5-11.el8.noarch. 1.4 MB/s | 49 kB 00:00 2024-03-23T17:39:13.734 INFO:teuthology.orchestra.run.smithi086.stdout:(109/146): pcre2-devel-10.32-2.el8.x86_64.rpm 5.1 MB/s | 605 kB 00:00 2024-03-23T17:39:13.758 INFO:teuthology.orchestra.run.smithi086.stdout:(110/146): libzstd-devel-1.4.4-1.el8.x86_64.rpm 263 kB/s | 44 kB 00:00 2024-03-23T17:39:13.801 INFO:teuthology.orchestra.run.smithi092.stdout:(52/146): python3-more-itertools-7.2.0-3.el8.no 704 kB/s | 59 kB 00:00 2024-03-23T17:39:13.835 INFO:teuthology.orchestra.run.smithi092.stdout:(53/146): python3-natsort-7.1.1-2.el8.noarch.rp 1.8 MB/s | 60 kB 00:00 2024-03-23T17:39:13.867 INFO:teuthology.orchestra.run.smithi086.stdout:(111/146): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.7 MB/s | 229 kB 00:00 2024-03-23T17:39:13.877 INFO:teuthology.orchestra.run.smithi092.stdout:(54/146): python3-pecan-1.3.2-9.el8.noarch.rpm 6.6 MB/s | 283 kB 00:00 2024-03-23T17:39:13.910 INFO:teuthology.orchestra.run.smithi092.stdout:(55/146): python3-portend-2.6-1.el8.noarch.rpm 467 kB/s | 16 kB 00:00 2024-03-23T17:39:13.934 INFO:teuthology.orchestra.run.smithi086.stdout:(112/146): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.2 MB/s | 220 kB 00:00 2024-03-23T17:39:13.944 INFO:teuthology.orchestra.run.smithi092.stdout:(56/146): python3-repoze-lru-0.7-6.el8.noarch.r 1.0 MB/s | 34 kB 00:00 2024-03-23T17:39:13.978 INFO:teuthology.orchestra.run.smithi092.stdout:(57/146): python3-routes-2.4.1-12.el8.noarch.rp 5.8 MB/s | 196 kB 00:00 2024-03-23T17:39:14.011 INFO:teuthology.orchestra.run.smithi092.stdout:(58/146): python3-rsa-4.9-2.el8.noarch.rpm 1.9 MB/s | 64 kB 00:00 2024-03-23T17:39:14.034 INFO:teuthology.orchestra.run.smithi086.stdout:(113/146): libselinux-devel-2.9-5.el8.x86_64.rp 1.2 MB/s | 200 kB 00:00 2024-03-23T17:39:14.071 INFO:teuthology.orchestra.run.smithi092.stdout:(59/146): python3-simplegeneric-0.8.1-17.el8.no 313 kB/s | 19 kB 00:00 2024-03-23T17:39:14.093 INFO:teuthology.orchestra.run.smithi086.stdout:(114/146): keyutils-libs-devel-1.5.10-9.el8.x86 303 kB/s | 48 kB 00:00 2024-03-23T17:39:14.112 INFO:teuthology.orchestra.run.smithi092.stdout:(60/146): python3-singledispatch-3.4.0.3-18.el8 593 kB/s | 24 kB 00:00 2024-03-23T17:39:14.145 INFO:teuthology.orchestra.run.smithi092.stdout:(61/146): python3-tempora-1.14.1-5.el8.noarch.r 877 kB/s | 29 kB 00:00 2024-03-23T17:39:14.177 INFO:teuthology.orchestra.run.smithi086.stdout:(115/146): libsepol-devel-2.9-3.el8.x86_64.rpm 613 kB/s | 87 kB 00:00 2024-03-23T17:39:14.252 INFO:teuthology.orchestra.run.smithi086.stdout:(116/146): zlib-devel-1.2.11-18.el8_5.x86_64.rp 364 kB/s | 58 kB 00:00 2024-03-23T17:39:14.294 INFO:teuthology.orchestra.run.smithi086.stdout:(117/146): libcom_err-devel-1.45.6-4.el8.x86_64 330 kB/s | 39 kB 00:00 2024-03-23T17:39:14.369 INFO:teuthology.orchestra.run.smithi086.stdout:(118/146): fuse-2.9.7-17.el8.x86_64.rpm 711 kB/s | 83 kB 00:00 2024-03-23T17:39:14.387 INFO:teuthology.orchestra.run.smithi092.stdout:(62/146): python3-trustme-0.6.0-4.el8.noarch.rp 114 kB/s | 27 kB 00:00 2024-03-23T17:39:14.461 INFO:teuthology.orchestra.run.smithi086.stdout:(119/146): libgfortran-8.5.0-20.el8.x86_64.rpm 3.8 MB/s | 645 kB 00:00 2024-03-23T17:39:14.536 INFO:teuthology.orchestra.run.smithi086.stdout:(120/146): libquadmath-8.5.0-20.el8.x86_64.rpm 1.0 MB/s | 172 kB 00:00 2024-03-23T17:39:14.603 INFO:teuthology.orchestra.run.smithi086.stdout:(121/146): libuuid-devel-2.32.1-43.el8.x86_64.r 703 kB/s | 100 kB 00:00 2024-03-23T17:39:14.662 INFO:teuthology.orchestra.run.smithi086.stdout:(122/146): xfsprogs-devel-5.0.0-12.el8.x86_64.r 1.2 MB/s | 73 kB 00:00 2024-03-23T17:39:14.684 INFO:teuthology.orchestra.run.smithi152.stdout:(10/146): ceph-radosgw-18.2.2-954.g82316b04.el8 942 kB/s | 13 MB 00:14 2024-03-23T17:39:14.687 INFO:teuthology.orchestra.run.smithi086.stdout:(123/146): python3-cffi-1.11.5-6.el8.x86_64.rpm 1.5 MB/s | 238 kB 00:00 2024-03-23T17:39:14.712 INFO:teuthology.orchestra.run.smithi092.stdout:(63/146): python3-waitress-1.4.3-1.el8.noarch.r 788 kB/s | 255 kB 00:00 2024-03-23T17:39:14.772 INFO:teuthology.orchestra.run.smithi092.stdout:(64/146): python3-webob-1.8.5-1.el8.1.noarch.rp 4.1 MB/s | 251 kB 00:00 2024-03-23T17:39:14.779 INFO:teuthology.orchestra.run.smithi086.stdout:(124/146): elfutils-libelf-devel-0.189-3.el8.x8 674 kB/s | 62 kB 00:00 2024-03-23T17:39:14.797 INFO:teuthology.orchestra.run.smithi092.stdout:(65/146): python3-websocket-client-0.56.0-5.el8 2.4 MB/s | 61 kB 00:00 2024-03-23T17:39:14.805 INFO:teuthology.orchestra.run.smithi086.stdout:(125/146): NetworkManager-initscripts-updown-1. 1.0 MB/s | 144 kB 00:00 2024-03-23T17:39:14.830 INFO:teuthology.orchestra.run.smithi092.stdout:(66/146): python3-webtest-2.0.33-1.el8.noarch.r 2.5 MB/s | 85 kB 00:00 2024-03-23T17:39:14.856 INFO:teuthology.orchestra.run.smithi092.stdout:(67/146): python3-zc-lockfile-2.0-2.el8.noarch. 923 kB/s | 23 kB 00:00 2024-03-23T17:39:14.921 INFO:teuthology.orchestra.run.smithi086.stdout:(126/146): krb5-devel-1.18.2-26.el8_9.x86_64.rp 3.9 MB/s | 562 kB 00:00 2024-03-23T17:39:14.931 INFO:teuthology.orchestra.run.smithi092.stdout:(68/146): thrift-0.13.0-2.el8.x86_64.rpm 23 MB/s | 1.7 MB 00:00 2024-03-23T17:39:14.956 INFO:teuthology.orchestra.run.smithi092.stdout:(69/146): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.8 MB/s | 70 kB 00:00 2024-03-23T17:39:15.047 INFO:teuthology.orchestra.run.smithi086.stdout:(127/146): openssl-devel-1.1.1k-12.el8_9.x86_64 9.6 MB/s | 2.3 MB 00:00 2024-03-23T17:39:15.090 INFO:teuthology.orchestra.run.smithi092.stdout:(70/146): libverto-devel-0.3.0-5.el8.x86_64.rpm 134 kB/s | 18 kB 00:00 2024-03-23T17:39:15.091 INFO:teuthology.orchestra.run.smithi086.stdout:(128/146): librabbitmq-0.9.0-5.el8_9.x86_64.rpm 279 kB/s | 47 kB 00:00 2024-03-23T17:39:15.139 INFO:teuthology.orchestra.run.smithi086.stdout:(129/146): python3-cryptography-3.2.1-7.el8_9.x 5.9 MB/s | 559 kB 00:00 2024-03-23T17:39:15.209 INFO:teuthology.orchestra.run.smithi152.stdout:(11/146): cephfs-mirror-18.2.2-954.g82316b04.el 497 kB/s | 261 kB 00:00 2024-03-23T17:39:15.248 INFO:teuthology.orchestra.run.smithi092.stdout:(71/146): python3-jwt-1.6.1-2.el8.noarch.rpm 269 kB/s | 43 kB 00:00 2024-03-23T17:39:15.318 INFO:teuthology.orchestra.run.smithi152.stdout:(12/146): libcephfs-devel-18.2.2-954.g82316b04. 286 kB/s | 31 kB 00:00 2024-03-23T17:39:15.374 INFO:teuthology.orchestra.run.smithi092.stdout:(72/146): python3-oauthlib-2.1.0-1.el8.noarch.r 1.2 MB/s | 155 kB 00:00 2024-03-23T17:39:15.491 INFO:teuthology.orchestra.run.smithi092.stdout:(73/146): python3-pycparser-2.14-14.el8.noarch. 935 kB/s | 109 kB 00:00 2024-03-23T17:39:15.649 INFO:teuthology.orchestra.run.smithi092.stdout:(74/146): python3-requests-oauthlib-1.0.0-1.el8 270 kB/s | 43 kB 00:00 2024-03-23T17:39:15.791 INFO:teuthology.orchestra.run.smithi092.stdout:(75/146): libaio-devel-0.3.112-1.el8.x86_64.rpm 132 kB/s | 19 kB 00:00 2024-03-23T17:39:16.007 INFO:teuthology.orchestra.run.smithi092.stdout:(76/146): ceph-mgr-diskprediction-local-18.2.2- 1.6 MB/s | 7.4 MB 00:04 2024-03-23T17:39:16.042 INFO:teuthology.orchestra.run.smithi092.stdout:(77/146): libzstd-devel-1.4.4-1.el8.x86_64.rpm 175 kB/s | 44 kB 00:00 2024-03-23T17:39:16.209 INFO:teuthology.orchestra.run.smithi092.stdout:(78/146): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.4 MB/s | 229 kB 00:00 2024-03-23T17:39:16.242 INFO:teuthology.orchestra.run.smithi092.stdout:(79/146): pcre2-devel-10.32-2.el8.x86_64.rpm 2.5 MB/s | 605 kB 00:00 2024-03-23T17:39:16.351 INFO:teuthology.orchestra.run.smithi092.stdout:(80/146): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.5 MB/s | 220 kB 00:00 2024-03-23T17:39:16.427 INFO:teuthology.orchestra.run.smithi092.stdout:(81/146): libselinux-devel-2.9-5.el8.x86_64.rpm 1.1 MB/s | 200 kB 00:00 2024-03-23T17:39:16.460 INFO:teuthology.orchestra.run.smithi092.stdout:(82/146): keyutils-libs-devel-1.5.10-9.el8.x86_ 442 kB/s | 48 kB 00:00 2024-03-23T17:39:16.577 INFO:teuthology.orchestra.run.smithi092.stdout:(83/146): libsepol-devel-2.9-3.el8.x86_64.rpm 579 kB/s | 87 kB 00:00 2024-03-23T17:39:16.593 INFO:teuthology.orchestra.run.smithi152.stdout:(13/146): ceph-common-18.2.2-954.g82316b04.el8. 834 kB/s | 22 MB 00:26 2024-03-23T17:39:16.626 INFO:teuthology.orchestra.run.smithi152.stdout:(14/146): libcephfs2-18.2.2-954.g82316b04.el8.x 593 kB/s | 775 kB 00:01 2024-03-23T17:39:16.633 INFO:teuthology.orchestra.run.smithi092.stdout:(84/146): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 333 kB/s | 58 kB 00:00 2024-03-23T17:39:16.686 INFO:teuthology.orchestra.run.smithi092.stdout:(85/146): libcom_err-devel-1.45.6-4.el8.x86_64. 355 kB/s | 39 kB 00:00 2024-03-23T17:39:16.703 INFO:teuthology.orchestra.run.smithi092.stdout:(86/146): fuse-2.9.7-17.el8.x86_64.rpm 1.2 MB/s | 83 kB 00:00 2024-03-23T17:39:16.781 INFO:teuthology.orchestra.run.smithi086.stdout:(130/146): librbd1-18.2.2-954.g82316b04.el8.x86 2.2 MB/s | 3.6 MB 00:01 2024-03-23T17:39:16.862 INFO:teuthology.orchestra.run.smithi092.stdout:(87/146): libgfortran-8.5.0-20.el8.x86_64.rpm 3.6 MB/s | 645 kB 00:00 2024-03-23T17:39:16.885 INFO:teuthology.orchestra.run.smithi152.stdout:(15/146): libcephsqlite-18.2.2-954.g82316b04.el 487 kB/s | 142 kB 00:00 2024-03-23T17:39:16.903 INFO:teuthology.orchestra.run.smithi092.stdout:(88/146): libquadmath-8.5.0-20.el8.x86_64.rpm 861 kB/s | 172 kB 00:00 2024-03-23T17:39:16.918 INFO:teuthology.orchestra.run.smithi152.stdout:(16/146): librados-devel-18.2.2-954.g82316b04.e 476 kB/s | 139 kB 00:00 2024-03-23T17:39:16.933 INFO:teuthology.orchestra.run.smithi086.stdout:(131/146): elfutils-libelf-0.189-3.el8.x86_64.r 1.5 MB/s | 232 kB 00:00 2024-03-23T17:39:17.012 INFO:teuthology.orchestra.run.smithi092.stdout:(89/146): libuuid-devel-2.32.1-43.el8.x86_64.rp 664 kB/s | 100 kB 00:00 2024-03-23T17:39:17.062 INFO:teuthology.orchestra.run.smithi092.stdout:(90/146): python3-cffi-1.11.5-6.el8.x86_64.rpm 1.5 MB/s | 238 kB 00:00 2024-03-23T17:39:17.148 INFO:teuthology.orchestra.run.smithi086.stdout:(132/146): elfutils-libs-0.189-3.el8.x86_64.rpm 1.4 MB/s | 303 kB 00:00 2024-03-23T17:39:17.163 INFO:teuthology.orchestra.run.smithi092.stdout:(91/146): xfsprogs-devel-5.0.0-12.el8.x86_64.rp 483 kB/s | 73 kB 00:00 2024-03-23T17:39:17.188 INFO:teuthology.orchestra.run.smithi092.stdout:(92/146): NetworkManager-initscripts-updown-1.4 1.1 MB/s | 144 kB 00:00 2024-03-23T17:39:17.299 INFO:teuthology.orchestra.run.smithi086.stdout:(133/146): libmount-2.32.1-43.el8.x86_64.rpm 1.5 MB/s | 237 kB 00:00 2024-03-23T17:39:17.313 INFO:teuthology.orchestra.run.smithi092.stdout:(93/146): elfutils-libelf-devel-0.189-3.el8.x86 411 kB/s | 62 kB 00:00 2024-03-23T17:39:17.372 INFO:teuthology.orchestra.run.smithi092.stdout:(94/146): krb5-devel-1.18.2-26.el8_9.x86_64.rpm 3.0 MB/s | 562 kB 00:00 2024-03-23T17:39:17.385 INFO:teuthology.orchestra.run.smithi152.stdout:(17/146): libradosstriper1-18.2.2-954.g82316b04 948 kB/s | 474 kB 00:00 2024-03-23T17:39:17.457 INFO:teuthology.orchestra.run.smithi086.stdout:(134/146): librados2-18.2.2-954.g82316b04.el8.x 1.6 MB/s | 3.7 MB 00:02 2024-03-23T17:39:17.483 INFO:teuthology.orchestra.run.smithi086.stdout:(135/146): libsmartcols-2.32.1-43.el8.x86_64.rp 979 kB/s | 179 kB 00:00 2024-03-23T17:39:17.494 INFO:teuthology.orchestra.run.smithi152.stdout:(18/146): python3-ceph-argparse-18.2.2-954.g823 411 kB/s | 45 kB 00:00 2024-03-23T17:39:17.505 INFO:teuthology.orchestra.run.smithi092.stdout:(95/146): openssl-devel-1.1.1k-12.el8_9.x86_64. 12 MB/s | 2.3 MB 00:00 2024-03-23T17:39:17.531 INFO:teuthology.orchestra.run.smithi092.stdout:(96/146): librabbitmq-0.9.0-5.el8_9.x86_64.rpm 297 kB/s | 47 kB 00:00 2024-03-23T17:39:17.592 INFO:teuthology.orchestra.run.smithi086.stdout:(136/146): util-linux-2.32.1-43.el8.x86_64.rpm 23 MB/s | 2.5 MB 00:00 2024-03-23T17:39:17.639 INFO:teuthology.orchestra.run.smithi092.stdout:(97/146): python3-cryptography-3.2.1-7.el8_9.x8 4.1 MB/s | 559 kB 00:00 2024-03-23T17:39:17.650 INFO:teuthology.orchestra.run.smithi086.stdout:(137/146): libuuid-2.32.1-43.el8.x86_64.rpm 515 kB/s | 99 kB 00:00 2024-03-23T17:39:17.656 INFO:teuthology.orchestra.run.smithi092.stdout:(98/146): python3-jsonpointer-1.10-11.el8.noarc 160 kB/s | 20 kB 00:00 2024-03-23T17:39:17.717 INFO:teuthology.orchestra.run.smithi086.stdout:(138/146): util-linux-user-2.32.1-43.el8.x86_64 822 kB/s | 103 kB 00:00 2024-03-23T17:39:17.729 INFO:teuthology.orchestra.run.smithi152.stdout:(19/146): python3-ceph-common-18.2.2-954.g82316 539 kB/s | 126 kB 00:00 2024-03-23T17:39:17.767 INFO:teuthology.orchestra.run.smithi086.stdout:(139/146): iproute-6.2.0-5.el8_9.x86_64.rpm 7.1 MB/s | 854 kB 00:00 2024-03-23T17:39:17.773 INFO:teuthology.orchestra.run.smithi092.stdout:(99/146): python3-pyasn1-modules-0.3.7-6.el8.no 944 kB/s | 110 kB 00:00 2024-03-23T17:39:17.790 INFO:teuthology.orchestra.run.smithi092.stdout:(100/146): python3-prettytable-0.7.2-14.el8.noa 292 kB/s | 44 kB 00:00 2024-03-23T17:39:17.801 INFO:teuthology.orchestra.run.smithi086.stdout:(140/146): libblkid-2.32.1-43.el8.x86_64.rpm 2.6 MB/s | 220 kB 00:00 2024-03-23T17:39:17.951 INFO:teuthology.orchestra.run.smithi086.stdout:(141/146): libfdisk-2.32.1-43.el8.x86_64.rpm 1.4 MB/s | 254 kB 00:00 2024-03-23T17:39:17.965 INFO:teuthology.orchestra.run.smithi092.stdout:(101/146): python3-pyasn1-0.3.7-6.el8.noarch.rp 723 kB/s | 126 kB 00:00 2024-03-23T17:39:17.970 INFO:teuthology.orchestra.run.smithi152.stdout:(20/146): python3-cephfs-18.2.2-954.g82316b04.e 814 kB/s | 197 kB 00:00 2024-03-23T17:39:17.982 INFO:teuthology.orchestra.run.smithi092.stdout:(102/146): python3-jsonpatch-1.21-2.el8.noarch. 127 kB/s | 27 kB 00:00 2024-03-23T17:39:17.993 INFO:teuthology.orchestra.run.smithi086.stdout:(142/146): openssl-1.1.1k-12.el8_9.x86_64.rpm 3.6 MB/s | 711 kB 00:00 2024-03-23T17:39:18.141 INFO:teuthology.orchestra.run.smithi086.stdout:(143/146): openssl-libs-1.1.1k-12.el8_9.x86_64. 7.8 MB/s | 1.5 MB 00:00 2024-03-23T17:39:18.149 INFO:teuthology.orchestra.run.smithi092.stdout:(103/146): bison-3.0.4-10.el8.x86_64.rpm 4.0 MB/s | 688 kB 00:00 2024-03-23T17:39:18.177 INFO:teuthology.orchestra.run.smithi086.stdout:(144/146): xfsdump-3.1.8-7.el8_9.x86_64.rpm 1.8 MB/s | 333 kB 00:00 2024-03-23T17:39:18.213 INFO:teuthology.orchestra.run.smithi092.stdout:(104/146): python3-werkzeug-0.12.2-4.el8.noarch 1.8 MB/s | 457 kB 00:00 2024-03-23T17:39:18.291 INFO:teuthology.orchestra.run.smithi092.stdout:(105/146): flex-2.6.1-9.el8.x86_64.rpm 2.2 MB/s | 320 kB 00:00 2024-03-23T17:39:18.310 INFO:teuthology.orchestra.run.smithi086.stdout:(145/146): xfsprogs-5.0.0-12.el8.x86_64.rpm 6.3 MB/s | 1.1 MB 00:00 2024-03-23T17:39:18.370 INFO:teuthology.orchestra.run.smithi152.stdout:(21/146): python3-rados-18.2.2-954.g82316b04.el 929 kB/s | 371 kB 00:00 2024-03-23T17:39:18.375 INFO:teuthology.orchestra.run.smithi092.stdout:(106/146): libtool-2.4.6-25.el8.x86_64.rpm 4.3 MB/s | 709 kB 00:00 2024-03-23T17:39:18.417 INFO:teuthology.orchestra.run.smithi092.stdout:(107/146): python3-markupsafe-0.23-19.el8.x86_6 310 kB/s | 39 kB 00:00 2024-03-23T17:39:18.492 INFO:teuthology.orchestra.run.smithi092.stdout:(108/146): lttng-ust-2.8.1-11.el8.x86_64.rpm 2.2 MB/s | 259 kB 00:00 2024-03-23T17:39:18.568 INFO:teuthology.orchestra.run.smithi092.stdout:(109/146): uuid-1.6.2-43.el8.x86_64.rpm 426 kB/s | 64 kB 00:00 2024-03-23T17:39:18.668 INFO:teuthology.orchestra.run.smithi092.stdout:(110/146): python3-pyOpenSSL-19.0.0-1.el8.noarc 590 kB/s | 103 kB 00:00 2024-03-23T17:39:18.703 INFO:teuthology.orchestra.run.smithi152.stdout:(22/146): python3-rbd-18.2.2-954.g82316b04.el8. 1.1 MB/s | 365 kB 00:00 2024-03-23T17:39:18.793 INFO:teuthology.orchestra.run.smithi092.stdout:(111/146): python3-babel-2.5.1-7.el8.noarch.rpm 21 MB/s | 4.8 MB 00:00 2024-03-23T17:39:18.812 INFO:teuthology.orchestra.run.smithi152.stdout:(23/146): python3-rgw-18.2.2-954.g82316b04.el8. 984 kB/s | 107 kB 00:00 2024-03-23T17:39:18.864 INFO:teuthology.orchestra.run.smithi092.stdout:(112/146): python3-jinja2-2.10.1-3.el8.noarch.r 2.7 MB/s | 538 kB 00:00 2024-03-23T17:39:18.946 INFO:teuthology.orchestra.run.smithi152.stdout:(24/146): rbd-fuse-18.2.2-954.g82316b04.el8.x86 715 kB/s | 95 kB 00:00 2024-03-23T17:39:18.986 INFO:teuthology.orchestra.run.smithi092.stdout:(113/146): socat-1.7.4.1-1.el8.x86_64.rpm 2.6 MB/s | 323 kB 00:00 2024-03-23T17:39:19.202 INFO:teuthology.orchestra.run.smithi092.stdout:(114/146): python3-numpy-1.14.3-10.el8.x86_64.r 9.1 MB/s | 3.7 MB 00:00 2024-03-23T17:39:19.228 INFO:teuthology.orchestra.run.smithi092.stdout:(115/146): python3-numpy-f2py-1.14.3-10.el8.x86 931 kB/s | 225 kB 00:00 2024-03-23T17:39:19.328 INFO:teuthology.orchestra.run.smithi092.stdout:(116/146): librdkafka-0.11.4-3.el8.x86_64.rpm 2.8 MB/s | 354 kB 00:00 2024-03-23T17:39:19.453 INFO:teuthology.orchestra.run.smithi092.stdout:(117/146): openblas-threads-0.3.15-6.el8.x86_64 23 MB/s | 5.2 MB 00:00 2024-03-23T17:39:19.520 INFO:teuthology.orchestra.run.smithi092.stdout:(118/146): python3-mako-1.0.6-14.el8.noarch.rpm 818 kB/s | 157 kB 00:00 2024-03-23T17:39:19.704 INFO:teuthology.orchestra.run.smithi092.stdout:(119/146): openblas-0.3.15-6.el8.x86_64.rpm 20 MB/s | 5.0 MB 00:00 2024-03-23T17:39:19.737 INFO:teuthology.orchestra.run.smithi092.stdout:(120/146): automake-1.16.1-8.el8.noarch.rpm 3.2 MB/s | 714 kB 00:00 2024-03-23T17:39:19.871 INFO:teuthology.orchestra.run.smithi092.stdout:(121/146): jq-1.6-7.el8.x86_64.rpm 1.5 MB/s | 202 kB 00:00 2024-03-23T17:39:19.896 INFO:teuthology.orchestra.run.smithi092.stdout:(122/146): libpq-13.11-1.el8.x86_64.rpm 1.0 MB/s | 198 kB 00:00 2024-03-23T17:39:19.988 INFO:teuthology.orchestra.run.smithi092.stdout:(123/146): python3-pytz-2017.2-11.el8.noarch.rp 463 kB/s | 54 kB 00:00 2024-03-23T17:39:20.139 INFO:teuthology.orchestra.run.smithi092.stdout:(124/146): oniguruma-6.8.2-2.1.el8_9.x86_64.rpm 1.2 MB/s | 188 kB 00:00 2024-03-23T17:39:20.305 INFO:teuthology.orchestra.run.smithi092.stdout:(125/146): python3-scipy-1.0.0-21.module+el8.9. 34 MB/s | 14 MB 00:00 2024-03-23T17:39:20.389 INFO:teuthology.orchestra.run.smithi092.stdout:(126/146): postgresql-10.23-4.module+el8.9.0+21 6.0 MB/s | 1.5 MB 00:00 2024-03-23T17:39:20.464 INFO:teuthology.orchestra.run.smithi092.stdout:(127/146): postgresql-contrib-10.23-4.module+el 5.0 MB/s | 812 kB 00:00 2024-03-23T17:39:20.640 INFO:teuthology.orchestra.run.smithi092.stdout:(128/146): python3-packaging-16.8-9.el8.noarch. 275 kB/s | 48 kB 00:00 2024-03-23T17:39:20.773 INFO:teuthology.orchestra.run.smithi092.stdout:(129/146): postgresql-server-10.23-4.module+el8 13 MB/s | 5.1 MB 00:00 2024-03-23T17:39:20.804 INFO:teuthology.orchestra.run.smithi152.stdout:(25/146): rbd-mirror-18.2.2-954.g82316b04.el8.x 1.9 MB/s | 3.5 MB 00:01 2024-03-23T17:39:20.852 INFO:teuthology.orchestra.run.smithi086.stdout:(146/146): ceph-test-18.2.2-954.g82316b04.el8.x 2.3 MB/s | 47 MB 00:20 2024-03-23T17:39:20.856 INFO:teuthology.orchestra.run.smithi086.stdout:-------------------------------------------------------------------------------- 2024-03-23T17:39:20.856 INFO:teuthology.orchestra.run.smithi086.stdout:Total 7.5 MB/s | 213 MB 00:28 2024-03-23T17:39:20.971 INFO:teuthology.orchestra.run.smithi152.stdout:(26/146): rbd-nbd-18.2.2-954.g82316b04.el8.x86_ 1.1 MB/s | 181 kB 00:00 2024-03-23T17:39:21.072 INFO:teuthology.orchestra.run.smithi152.stdout:(27/146): ceph-grafana-dashboards-18.2.2-954.g8 233 kB/s | 23 kB 00:00 2024-03-23T17:39:21.295 INFO:teuthology.orchestra.run.smithi152.stdout:(28/146): ceph-mgr-cephadm-18.2.2-954.g82316b04 619 kB/s | 138 kB 00:00 2024-03-23T17:39:22.414 INFO:teuthology.orchestra.run.smithi152.stdout:(29/146): ceph-mgr-dashboard-18.2.2-954.g82316b 1.6 MB/s | 1.7 MB 00:01 2024-03-23T17:39:22.756 INFO:teuthology.orchestra.run.smithi152.stdout:(30/146): librgw2-18.2.2-954.g82316b04.el8.x86_ 1.1 MB/s | 6.5 MB 00:05 2024-03-23T17:39:22.942 INFO:teuthology.orchestra.run.smithi092.stdout:(130/146): ceph-test-18.2.2-954.g82316b04.el8.x 2.4 MB/s | 47 MB 00:19 2024-03-23T17:39:22.996 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction check 2024-03-23T17:39:23.006 INFO:teuthology.orchestra.run.smithi152.stdout:(31/146): ceph-mgr-modules-core-18.2.2-954.g823 983 kB/s | 245 kB 00:00 2024-03-23T17:39:23.015 INFO:teuthology.orchestra.run.smithi092.stdout:(131/146): librbd1-18.2.2-954.g82316b04.el8.x86 1.6 MB/s | 3.6 MB 00:02 2024-03-23T17:39:23.073 INFO:teuthology.orchestra.run.smithi092.stdout:(132/146): librados2-18.2.2-954.g82316b04.el8.x 1.5 MB/s | 3.7 MB 00:02 2024-03-23T17:39:23.124 INFO:teuthology.orchestra.run.smithi092.stdout:(133/146): elfutils-libs-0.189-3.el8.x86_64.rpm 2.7 MB/s | 303 kB 00:00 2024-03-23T17:39:23.131 INFO:teuthology.orchestra.run.smithi152.stdout:(32/146): ceph-mgr-rook-18.2.2-954.g82316b04.el 394 kB/s | 49 kB 00:00 2024-03-23T17:39:23.166 INFO:teuthology.orchestra.run.smithi092.stdout:(134/146): elfutils-libelf-0.189-3.el8.x86_64.r 1.0 MB/s | 232 kB 00:00 2024-03-23T17:39:23.191 INFO:teuthology.orchestra.run.smithi092.stdout:(135/146): libmount-2.32.1-43.el8.x86_64.rpm 2.0 MB/s | 237 kB 00:00 2024-03-23T17:39:23.231 INFO:teuthology.orchestra.run.smithi152.stdout:(33/146): ceph-prometheus-alerts-18.2.2-954.g82 147 kB/s | 15 kB 00:00 2024-03-23T17:39:23.283 INFO:teuthology.orchestra.run.smithi092.stdout:(136/146): libsmartcols-2.32.1-43.el8.x86_64.rp 1.1 MB/s | 179 kB 00:00 2024-03-23T17:39:23.400 INFO:teuthology.orchestra.run.smithi092.stdout:(137/146): util-linux-2.32.1-43.el8.x86_64.rpm 12 MB/s | 2.5 MB 00:00 2024-03-23T17:39:23.434 INFO:teuthology.orchestra.run.smithi092.stdout:(138/146): libuuid-2.32.1-43.el8.x86_64.rpm 370 kB/s | 99 kB 00:00 2024-03-23T17:39:23.435 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction check succeeded. 2024-03-23T17:39:23.435 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction test 2024-03-23T17:39:23.451 INFO:teuthology.orchestra.run.smithi092.stdout:(139/146): util-linux-user-2.32.1-43.el8.x86_64 614 kB/s | 103 kB 00:00 2024-03-23T17:39:23.490 INFO:teuthology.orchestra.run.smithi152.stdout:(34/146): ceph-volume-18.2.2-954.g82316b04.el8. 1.0 MB/s | 263 kB 00:00 2024-03-23T17:39:23.517 INFO:teuthology.orchestra.run.smithi092.stdout:(140/146): iproute-6.2.0-5.el8_9.x86_64.rpm 7.1 MB/s | 854 kB 00:00 2024-03-23T17:39:23.575 INFO:teuthology.orchestra.run.smithi092.stdout:(141/146): libblkid-2.32.1-43.el8.x86_64.rpm 1.5 MB/s | 220 kB 00:00 2024-03-23T17:39:23.660 INFO:teuthology.orchestra.run.smithi092.stdout:(142/146): libfdisk-2.32.1-43.el8.x86_64.rpm 1.2 MB/s | 254 kB 00:00 2024-03-23T17:39:23.699 INFO:teuthology.orchestra.run.smithi152.stdout:(35/146): cephadm-18.2.2-954.g82316b04.el8.noar 828 kB/s | 172 kB 00:00 2024-03-23T17:39:23.710 INFO:teuthology.orchestra.run.smithi092.stdout:(143/146): openssl-libs-1.1.1k-12.el8_9.x86_64. 11 MB/s | 1.5 MB 00:00 2024-03-23T17:39:23.743 INFO:teuthology.orchestra.run.smithi092.stdout:(144/146): openssl-1.1.1k-12.el8_9.x86_64.rpm 3.1 MB/s | 711 kB 00:00 2024-03-23T17:39:23.785 INFO:teuthology.orchestra.run.smithi092.stdout:(145/146): xfsdump-3.1.8-7.el8_9.x86_64.rpm 2.6 MB/s | 333 kB 00:00 2024-03-23T17:39:23.799 INFO:teuthology.orchestra.run.smithi152.stdout:(36/146): cephfs-top-18.2.2-954.g82316b04.el8.n 181 kB/s | 18 kB 00:00 2024-03-23T17:39:23.852 INFO:teuthology.orchestra.run.smithi092.stdout:(146/146): xfsprogs-5.0.0-12.el8.x86_64.rpm 7.5 MB/s | 1.1 MB 00:00 2024-03-23T17:39:23.855 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-03-23T17:39:23.855 INFO:teuthology.orchestra.run.smithi092.stdout:Total 6.2 MB/s | 213 MB 00:34 2024-03-23T17:39:24.224 INFO:teuthology.orchestra.run.smithi152.stdout:(37/146): gperftools-libs-2.7-9.el8.x86_64.rpm 720 kB/s | 306 kB 00:00 2024-03-23T17:39:24.308 INFO:teuthology.orchestra.run.smithi152.stdout:(38/146): liboath-2.6.2-3.el8.x86_64.rpm 707 kB/s | 59 kB 00:00 2024-03-23T17:39:24.383 INFO:teuthology.orchestra.run.smithi152.stdout:(39/146): libunwind-1.3.1-3.el8.x86_64.rpm 1.0 MB/s | 75 kB 00:00 2024-03-23T17:39:24.525 INFO:teuthology.orchestra.run.smithi152.stdout:(40/146): python3-asyncssh-2.7.0-2.el8.noarch.r 3.3 MB/s | 476 kB 00:00 2024-03-23T17:39:24.617 INFO:teuthology.orchestra.run.smithi152.stdout:(41/146): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 476 kB/s | 44 kB 00:00 2024-03-23T17:39:24.718 INFO:teuthology.orchestra.run.smithi152.stdout:(42/146): python3-beautifulsoup4-4.6.3-2.el8.1. 1.8 MB/s | 185 kB 00:00 2024-03-23T17:39:24.793 INFO:teuthology.orchestra.run.smithi152.stdout:(43/146): python3-cachetools-3.1.1-4.el8.noarch 440 kB/s | 33 kB 00:00 2024-03-23T17:39:24.877 INFO:teuthology.orchestra.run.smithi152.stdout:(44/146): python3-certifi-2018.10.15-7.el8.noar 195 kB/s | 16 kB 00:00 2024-03-23T17:39:24.885 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-03-23T17:39:24.986 INFO:teuthology.orchestra.run.smithi152.stdout:(45/146): python3-cheroot-8.5.2-1.el8.noarch.rp 1.6 MB/s | 173 kB 00:00 2024-03-23T17:39:25.033 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction test succeeded. 2024-03-23T17:39:25.045 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction 2024-03-23T17:39:25.077 INFO:teuthology.orchestra.run.smithi152.stdout:(46/146): python3-cherrypy-18.4.0-1.el8.noarch. 4.1 MB/s | 384 kB 00:00 2024-03-23T17:39:25.170 INFO:teuthology.orchestra.run.smithi152.stdout:(47/146): python3-google-auth-1.1.1-10.el8.noar 886 kB/s | 81 kB 00:00 2024-03-23T17:39:25.278 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-03-23T17:39:25.278 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-03-23T17:39:25.278 INFO:teuthology.orchestra.run.smithi152.stdout:(48/146): python3-jaraco-6.2-6.el8.noarch.rpm 103 kB/s | 11 kB 00:00 2024-03-23T17:39:25.353 INFO:teuthology.orchestra.run.smithi152.stdout:(49/146): python3-jaraco-functools-2.0-4.el8.no 234 kB/s | 18 kB 00:00 2024-03-23T17:39:25.604 INFO:teuthology.orchestra.run.smithi152.stdout:(50/146): python3-kubernetes-11.0.0-6.el8.noarc 4.7 MB/s | 1.2 MB 00:00 2024-03-23T17:39:25.679 INFO:teuthology.orchestra.run.smithi152.stdout:(51/146): python3-logutils-0.3.5-11.el8.noarch. 649 kB/s | 49 kB 00:00 2024-03-23T17:39:25.755 INFO:teuthology.orchestra.run.smithi152.stdout:(52/146): python3-more-itertools-7.2.0-3.el8.no 783 kB/s | 59 kB 00:00 2024-03-23T17:39:25.829 INFO:teuthology.orchestra.run.smithi152.stdout:(53/146): python3-natsort-7.1.1-2.el8.noarch.rp 805 kB/s | 60 kB 00:00 2024-03-23T17:39:25.938 INFO:teuthology.orchestra.run.smithi152.stdout:(54/146): python3-pecan-1.3.2-9.el8.noarch.rpm 2.5 MB/s | 283 kB 00:00 2024-03-23T17:39:26.014 INFO:teuthology.orchestra.run.smithi152.stdout:(55/146): python3-portend-2.6-1.el8.noarch.rpm 208 kB/s | 16 kB 00:00 2024-03-23T17:39:26.097 INFO:teuthology.orchestra.run.smithi152.stdout:(56/146): python3-repoze-lru-0.7-6.el8.noarch.r 404 kB/s | 34 kB 00:00 2024-03-23T17:39:26.189 INFO:teuthology.orchestra.run.smithi152.stdout:(57/146): python3-routes-2.4.1-12.el8.noarch.rp 2.1 MB/s | 196 kB 00:00 2024-03-23T17:39:26.273 INFO:teuthology.orchestra.run.smithi152.stdout:(58/146): python3-rsa-4.9-2.el8.noarch.rpm 771 kB/s | 64 kB 00:00 2024-03-23T17:39:26.349 INFO:teuthology.orchestra.run.smithi152.stdout:(59/146): python3-simplegeneric-0.8.1-17.el8.no 249 kB/s | 19 kB 00:00 2024-03-23T17:39:26.424 INFO:teuthology.orchestra.run.smithi152.stdout:(60/146): python3-singledispatch-3.4.0.3-18.el8 319 kB/s | 24 kB 00:00 2024-03-23T17:39:26.487 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-03-23T17:39:26.497 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-03-23T17:39:26.524 INFO:teuthology.orchestra.run.smithi152.stdout:(61/146): ceph-mgr-diskprediction-local-18.2.2- 1.8 MB/s | 7.4 MB 00:04 2024-03-23T17:39:26.583 INFO:teuthology.orchestra.run.smithi152.stdout:(62/146): python3-tempora-1.14.1-5.el8.noarch.r 184 kB/s | 29 kB 00:00 2024-03-23T17:39:26.658 INFO:teuthology.orchestra.run.smithi152.stdout:(63/146): python3-waitress-1.4.3-1.el8.noarch.r 3.3 MB/s | 255 kB 00:00 2024-03-23T17:39:26.708 INFO:teuthology.orchestra.run.smithi152.stdout:(64/146): python3-trustme-0.6.0-4.el8.noarch.rp 150 kB/s | 27 kB 00:00 2024-03-23T17:39:26.733 INFO:teuthology.orchestra.run.smithi152.stdout:(65/146): python3-webob-1.8.5-1.el8.1.noarch.rp 3.3 MB/s | 251 kB 00:00 2024-03-23T17:39:26.800 INFO:teuthology.orchestra.run.smithi152.stdout:(66/146): python3-webtest-2.0.33-1.el8.noarch.r 1.3 MB/s | 85 kB 00:00 2024-03-23T17:39:26.834 INFO:teuthology.orchestra.run.smithi152.stdout:(67/146): python3-websocket-client-0.56.0-5.el8 488 kB/s | 61 kB 00:00 2024-03-23T17:39:26.968 INFO:teuthology.orchestra.run.smithi152.stdout:(68/146): python3-zc-lockfile-2.0-2.el8.noarch. 137 kB/s | 23 kB 00:00 2024-03-23T17:39:27.002 INFO:teuthology.orchestra.run.smithi152.stdout:(69/146): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.0 MB/s | 70 kB 00:00 2024-03-23T17:39:27.162 INFO:teuthology.orchestra.run.smithi152.stdout:(70/146): thrift-0.13.0-2.el8.x86_64.rpm 5.3 MB/s | 1.7 MB 00:00 2024-03-23T17:39:27.186 INFO:teuthology.orchestra.run.smithi152.stdout:(71/146): libverto-devel-0.3.0-5.el8.x86_64.rpm 97 kB/s | 18 kB 00:00 2024-03-23T17:39:27.344 INFO:teuthology.orchestra.run.smithi152.stdout:(72/146): python3-oauthlib-2.1.0-1.el8.noarch.r 1.0 MB/s | 155 kB 00:00 2024-03-23T17:39:27.369 INFO:teuthology.orchestra.run.smithi152.stdout:(73/146): python3-jwt-1.6.1-2.el8.noarch.rpm 204 kB/s | 43 kB 00:00 2024-03-23T17:39:27.470 INFO:teuthology.orchestra.run.smithi152.stdout:(74/146): python3-pycparser-2.14-14.el8.noarch. 871 kB/s | 109 kB 00:00 2024-03-23T17:39:27.587 INFO:teuthology.orchestra.run.smithi152.stdout:(75/146): python3-requests-oauthlib-1.0.0-1.el8 197 kB/s | 43 kB 00:00 2024-03-23T17:39:27.612 INFO:teuthology.orchestra.run.smithi152.stdout:(76/146): libaio-devel-0.3.112-1.el8.x86_64.rpm 132 kB/s | 19 kB 00:00 2024-03-23T17:39:27.712 INFO:teuthology.orchestra.run.smithi152.stdout:(77/146): libzstd-devel-1.4.4-1.el8.x86_64.rpm 350 kB/s | 44 kB 00:00 2024-03-23T17:39:27.746 INFO:teuthology.orchestra.run.smithi152.stdout:(78/146): pcre2-devel-10.32-2.el8.x86_64.rpm 4.4 MB/s | 605 kB 00:00 2024-03-23T17:39:27.763 INFO:teuthology.orchestra.run.smithi152.stdout:(79/146): pcre2-utf16-10.32-2.el8.x86_64.rpm 4.5 MB/s | 229 kB 00:00 2024-03-23T17:39:27.913 INFO:teuthology.orchestra.run.smithi152.stdout:(80/146): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.3 MB/s | 220 kB 00:00 2024-03-23T17:39:27.980 INFO:teuthology.orchestra.run.smithi152.stdout:(81/146): libselinux-devel-2.9-5.el8.x86_64.rpm 922 kB/s | 200 kB 00:00 2024-03-23T17:39:28.022 INFO:teuthology.orchestra.run.smithi152.stdout:(82/146): keyutils-libs-devel-1.5.10-9.el8.x86_ 443 kB/s | 48 kB 00:00 2024-03-23T17:39:28.081 INFO:teuthology.orchestra.run.smithi152.stdout:(83/146): libsepol-devel-2.9-3.el8.x86_64.rpm 867 kB/s | 87 kB 00:00 2024-03-23T17:39:28.123 INFO:teuthology.orchestra.run.smithi152.stdout:(84/146): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 575 kB/s | 58 kB 00:00 2024-03-23T17:39:28.210 INFO:teuthology.orchestra.run.smithi152.stdout:(85/146): libcom_err-devel-1.45.6-4.el8.x86_64. 299 kB/s | 39 kB 00:00 2024-03-23T17:39:28.274 INFO:teuthology.orchestra.run.smithi152.stdout:(86/146): fuse-2.9.7-17.el8.x86_64.rpm 552 kB/s | 83 kB 00:00 2024-03-23T17:39:28.324 INFO:teuthology.orchestra.run.smithi152.stdout:(87/146): libgfortran-8.5.0-20.el8.x86_64.rpm 5.6 MB/s | 645 kB 00:00 2024-03-23T17:39:28.416 INFO:teuthology.orchestra.run.smithi152.stdout:(88/146): libquadmath-8.5.0-20.el8.x86_64.rpm 1.2 MB/s | 172 kB 00:00 2024-03-23T17:39:28.483 INFO:teuthology.orchestra.run.smithi152.stdout:(89/146): libuuid-devel-2.32.1-43.el8.x86_64.rp 629 kB/s | 100 kB 00:00 2024-03-23T17:39:28.575 INFO:teuthology.orchestra.run.smithi152.stdout:(90/146): python3-cffi-1.11.5-6.el8.x86_64.rpm 1.5 MB/s | 238 kB 00:00 2024-03-23T17:39:28.633 INFO:teuthology.orchestra.run.smithi152.stdout:(91/146): xfsprogs-devel-5.0.0-12.el8.x86_64.rp 483 kB/s | 73 kB 00:00 2024-03-23T17:39:28.717 INFO:teuthology.orchestra.run.smithi152.stdout:(92/146): NetworkManager-initscripts-updown-1.4 1.0 MB/s | 144 kB 00:00 2024-03-23T17:39:28.800 INFO:teuthology.orchestra.run.smithi152.stdout:(93/146): elfutils-libelf-devel-0.189-3.el8.x86 370 kB/s | 62 kB 00:00 2024-03-23T17:39:28.909 INFO:teuthology.orchestra.run.smithi152.stdout:(94/146): krb5-devel-1.18.2-26.el8_9.x86_64.rpm 2.9 MB/s | 562 kB 00:00 2024-03-23T17:39:28.985 INFO:teuthology.orchestra.run.smithi152.stdout:(95/146): openssl-devel-1.1.1k-12.el8_9.x86_64. 13 MB/s | 2.3 MB 00:00 2024-03-23T17:39:29.052 INFO:teuthology.orchestra.run.smithi152.stdout:(96/146): librabbitmq-0.9.0-5.el8_9.x86_64.rpm 331 kB/s | 47 kB 00:00 2024-03-23T17:39:29.144 INFO:teuthology.orchestra.run.smithi152.stdout:(97/146): python3-cryptography-3.2.1-7.el8_9.x8 3.4 MB/s | 559 kB 00:00 2024-03-23T17:39:29.204 INFO:teuthology.orchestra.run.smithi152.stdout:(98/146): python3-packaging-16.8-9.el8.noarch.r 317 kB/s | 48 kB 00:00 2024-03-23T17:39:29.252 INFO:teuthology.orchestra.run.smithi152.stdout:(99/146): python3-jsonpointer-1.10-11.el8.noarc 184 kB/s | 20 kB 00:00 2024-03-23T17:39:29.336 INFO:teuthology.orchestra.run.smithi152.stdout:(100/146): python3-prettytable-0.7.2-14.el8.noa 333 kB/s | 44 kB 00:00 2024-03-23T17:39:29.411 INFO:teuthology.orchestra.run.smithi152.stdout:(101/146): python3-pyasn1-modules-0.3.7-6.el8.n 694 kB/s | 110 kB 00:00 2024-03-23T17:39:29.545 INFO:teuthology.orchestra.run.smithi152.stdout:(102/146): python3-jsonpatch-1.21-2.el8.noarch. 127 kB/s | 27 kB 00:00 2024-03-23T17:39:29.570 INFO:teuthology.orchestra.run.smithi152.stdout:(103/146): python3-pyasn1-0.3.7-6.el8.noarch.rp 796 kB/s | 126 kB 00:00 2024-03-23T17:39:29.729 INFO:teuthology.orchestra.run.smithi152.stdout:(104/146): python3-werkzeug-0.12.2-4.el8.noarch 2.4 MB/s | 457 kB 00:00 2024-03-23T17:39:29.763 INFO:teuthology.orchestra.run.smithi152.stdout:(105/146): bison-3.0.4-10.el8.x86_64.rpm 3.5 MB/s | 688 kB 00:00 2024-03-23T17:39:29.888 INFO:teuthology.orchestra.run.smithi152.stdout:(106/146): libtool-2.4.6-25.el8.x86_64.rpm 5.5 MB/s | 709 kB 00:00 2024-03-23T17:39:29.922 INFO:teuthology.orchestra.run.smithi152.stdout:(107/146): flex-2.6.1-9.el8.x86_64.rpm 1.6 MB/s | 320 kB 00:00 2024-03-23T17:39:30.022 INFO:teuthology.orchestra.run.smithi152.stdout:(108/146): python3-markupsafe-0.23-19.el8.x86_6 291 kB/s | 39 kB 00:00 2024-03-23T17:39:30.047 INFO:teuthology.orchestra.run.smithi152.stdout:(109/146): lttng-ust-2.8.1-11.el8.x86_64.rpm 2.0 MB/s | 259 kB 00:00 2024-03-23T17:39:30.206 INFO:teuthology.orchestra.run.smithi152.stdout:(110/146): uuid-1.6.2-43.el8.x86_64.rpm 349 kB/s | 64 kB 00:00 2024-03-23T17:39:30.231 INFO:teuthology.orchestra.run.smithi152.stdout:(111/146): python3-pyOpenSSL-19.0.0-1.el8.noarc 564 kB/s | 103 kB 00:00 2024-03-23T17:39:30.457 INFO:teuthology.orchestra.run.smithi152.stdout:(112/146): python3-babel-2.5.1-7.el8.noarch.rpm 19 MB/s | 4.8 MB 00:00 2024-03-23T17:39:30.482 INFO:teuthology.orchestra.run.smithi152.stdout:(113/146): python3-jinja2-2.10.1-3.el8.noarch.r 2.1 MB/s | 538 kB 00:00 2024-03-23T17:39:30.636 INFO:teuthology.orchestra.run.smithi086.stdout: Preparing : 1/1 2024-03-23T17:39:30.690 INFO:teuthology.orchestra.run.smithi152.stdout:(114/146): python3-numpy-1.14.3-10.el8.x86_64.r 16 MB/s | 3.7 MB 00:00 2024-03-23T17:39:30.715 INFO:teuthology.orchestra.run.smithi152.stdout:(115/146): socat-1.7.4.1-1.el8.x86_64.rpm 1.4 MB/s | 323 kB 00:00 2024-03-23T17:39:30.827 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/1 2024-03-23T17:39:30.874 INFO:teuthology.orchestra.run.smithi152.stdout:(116/146): python3-numpy-f2py-1.14.3-10.el8.x86 1.2 MB/s | 225 kB 00:00 2024-03-23T17:39:30.958 INFO:teuthology.orchestra.run.smithi152.stdout:(117/146): librdkafka-0.11.4-3.el8.x86_64.rpm 1.4 MB/s | 354 kB 00:00 2024-03-23T17:39:31.073 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-03-23T17:39:31.117 INFO:teuthology.orchestra.run.smithi152.stdout:(118/146): openblas-threads-0.3.15-6.el8.x86_64 22 MB/s | 5.2 MB 00:00 2024-03-23T17:39:31.185 INFO:teuthology.orchestra.run.smithi152.stdout:(119/146): python3-mako-1.0.6-14.el8.noarch.rpm 693 kB/s | 157 kB 00:00 2024-03-23T17:39:31.349 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-03-23T17:39:31.376 INFO:teuthology.orchestra.run.smithi152.stdout:(120/146): openblas-0.3.15-6.el8.x86_64.rpm 20 MB/s | 5.0 MB 00:00 2024-03-23T17:39:31.409 INFO:teuthology.orchestra.run.smithi152.stdout:(121/146): automake-1.16.1-8.el8.noarch.rpm 3.1 MB/s | 714 kB 00:00 2024-03-23T17:39:31.451 INFO:teuthology.orchestra.run.smithi152.stdout:(122/146): libpq-13.11-1.el8.x86_64.rpm 2.6 MB/s | 198 kB 00:00 2024-03-23T17:39:31.535 INFO:teuthology.orchestra.run.smithi152.stdout:(123/146): jq-1.6-7.el8.x86_64.rpm 1.6 MB/s | 202 kB 00:00 2024-03-23T17:39:31.560 INFO:teuthology.orchestra.run.smithi152.stdout:(124/146): python3-pytz-2017.2-11.el8.noarch.rp 499 kB/s | 54 kB 00:00 2024-03-23T17:39:31.566 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/162 2024-03-23T17:39:31.744 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : libuuid-2.32.1-43.el8.x86_64 3/162 2024-03-23T17:39:31.752 INFO:teuthology.orchestra.run.smithi152.stdout:(125/146): oniguruma-6.8.2-2.1.el8_9.x86_64.rpm 982 kB/s | 188 kB 00:00 2024-03-23T17:39:31.851 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libuuid-2.32.1-43.el8.x86_64 3/162 2024-03-23T17:39:31.944 INFO:teuthology.orchestra.run.smithi152.stdout:(126/146): python3-scipy-1.0.0-21.module+el8.9. 34 MB/s | 14 MB 00:00 2024-03-23T17:39:32.003 INFO:teuthology.orchestra.run.smithi152.stdout:(127/146): postgresql-10.23-4.module+el8.9.0+21 6.0 MB/s | 1.5 MB 00:00 2024-03-23T17:39:32.060 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : libblkid-2.32.1-43.el8.x86_64 4/162 2024-03-23T17:39:32.103 INFO:teuthology.orchestra.run.smithi152.stdout:(128/146): postgresql-contrib-10.23-4.module+el 5.0 MB/s | 812 kB 00:00 2024-03-23T17:39:32.146 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libblkid-2.32.1-43.el8.x86_64 4/162 2024-03-23T17:39:32.278 INFO:teuthology.orchestra.run.smithi152.stdout:(129/146): postgresql-server-10.23-4.module+el8 18 MB/s | 5.1 MB 00:00 2024-03-23T17:39:32.352 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-03-23T17:39:32.362 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/162 2024-03-23T17:39:32.487 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/1 2024-03-23T17:39:32.577 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 6/162 2024-03-23T17:39:32.669 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 6/162 2024-03-23T17:39:32.790 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-03-23T17:39:32.893 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : librabbitmq-0.9.0-5.el8_9.x86_64 7/162 2024-03-23T17:39:32.999 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-03-23T17:39:33.085 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-03-23T17:39:33.212 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/162 2024-03-23T17:39:33.430 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : libuuid-2.32.1-43.el8.x86_64 3/162 2024-03-23T17:39:33.504 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libuuid-2.32.1-43.el8.x86_64 3/162 2024-03-23T17:39:33.527 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-03-23T17:39:33.697 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : libblkid-2.32.1-43.el8.x86_64 4/162 2024-03-23T17:39:33.710 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : librados2-2:18.2.2-954.g82316b04.el8.x86_64 9/162 2024-03-23T17:39:33.764 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libblkid-2.32.1-43.el8.x86_64 4/162 2024-03-23T17:39:33.993 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/162 2024-03-23T17:39:34.155 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librados2-2:18.2.2-954.g82316b04.el8.x86_64 9/162 2024-03-23T17:39:34.179 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librabbitmq-0.9.0-5.el8_9.x86_64 6/162 2024-03-23T17:39:34.368 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : librbd1-2:18.2.2-954.g82316b04.el8.x86_64 10/162 2024-03-23T17:39:34.380 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 7/162 2024-03-23T17:39:34.480 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 7/162 2024-03-23T17:39:34.522 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librbd1-2:18.2.2-954.g82316b04.el8.x86_64 10/162 2024-03-23T17:39:34.688 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-03-23T17:39:34.702 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libcephfs2-2:18.2.2-954.g82316b04.el8.x86_64 11/162 2024-03-23T17:39:34.824 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libcephfs2-2:18.2.2-954.g82316b04.el8.x86_64 11/162 2024-03-23T17:39:35.098 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-03-23T17:39:35.210 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-rados-2:18.2.2-954.g82316b04.el8.x86_64 12/162 2024-03-23T17:39:35.338 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : librados2-2:18.2.2-954.g82316b04.el8.x86_64 9/162 2024-03-23T17:39:35.412 INFO:teuthology.orchestra.run.smithi152.stdout:(130/146): librbd1-18.2.2-954.g82316b04.el8.x86 1.1 MB/s | 3.6 MB 00:03 2024-03-23T17:39:35.469 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-03-23T17:39:35.553 INFO:teuthology.orchestra.run.smithi152.stdout:(131/146): elfutils-libelf-0.189-3.el8.x86_64.r 1.6 MB/s | 232 kB 00:00 2024-03-23T17:39:35.595 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-03-23T17:39:35.726 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librados2-2:18.2.2-954.g82316b04.el8.x86_64 9/162 2024-03-23T17:39:35.754 INFO:teuthology.orchestra.run.smithi152.stdout:(132/146): elfutils-libs-0.189-3.el8.x86_64.rpm 1.5 MB/s | 303 kB 00:00 2024-03-23T17:39:35.833 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libpq-13.11-1.el8.x86_64 14/162 2024-03-23T17:39:35.921 INFO:teuthology.orchestra.run.smithi152.stdout:(133/146): libmount-2.32.1-43.el8.x86_64.rpm 1.4 MB/s | 237 kB 00:00 2024-03-23T17:39:35.955 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : librbd1-2:18.2.2-954.g82316b04.el8.x86_64 10/162 2024-03-23T17:39:36.079 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librbd1-2:18.2.2-954.g82316b04.el8.x86_64 10/162 2024-03-23T17:39:36.088 INFO:teuthology.orchestra.run.smithi152.stdout:(134/146): libsmartcols-2.32.1-43.el8.x86_64.rp 1.1 MB/s | 179 kB 00:00 2024-03-23T17:39:36.141 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 15/162 2024-03-23T17:39:36.239 INFO:teuthology.orchestra.run.smithi152.stdout:(135/146): libuuid-2.32.1-43.el8.x86_64.rpm 660 kB/s | 99 kB 00:00 2024-03-23T17:39:36.296 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcephfs2-2:18.2.2-954.g82316b04.el8.x86_64 11/162 2024-03-23T17:39:36.332 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/162 2024-03-23T17:39:36.392 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libcephfs2-2:18.2.2-954.g82316b04.el8.x86_64 11/162 2024-03-23T17:39:36.438 INFO:teuthology.orchestra.run.smithi152.stdout:(136/146): util-linux-2.32.1-43.el8.x86_64.rpm 12 MB/s | 2.5 MB 00:00 2024-03-23T17:39:36.564 INFO:teuthology.orchestra.run.smithi152.stdout:(137/146): util-linux-user-2.32.1-43.el8.x86_64 822 kB/s | 103 kB 00:00 2024-03-23T17:39:36.706 INFO:teuthology.orchestra.run.smithi152.stdout:(138/146): iproute-6.2.0-5.el8_9.x86_64.rpm 5.9 MB/s | 854 kB 00:00 2024-03-23T17:39:36.723 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rados-2:18.2.2-954.g82316b04.el8.x86_64 12/162 2024-03-23T17:39:36.756 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 17/162 2024-03-23T17:39:36.856 INFO:teuthology.orchestra.run.smithi152.stdout:(139/146): libblkid-2.32.1-43.el8.x86_64.rpm 1.4 MB/s | 220 kB 00:00 2024-03-23T17:39:36.973 INFO:teuthology.orchestra.run.smithi152.stdout:(140/146): libfdisk-2.32.1-43.el8.x86_64.rpm 2.1 MB/s | 254 kB 00:00 2024-03-23T17:39:36.988 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 18/162 2024-03-23T17:39:37.024 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-03-23T17:39:37.073 INFO:teuthology.orchestra.run.smithi152.stdout:(141/146): openssl-1.1.1k-12.el8_9.x86_64.rpm 6.9 MB/s | 711 kB 00:00 2024-03-23T17:39:37.108 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-03-23T17:39:37.168 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libcephsqlite-2:18.2.2-954.g82316b04.el8.x86_64 19/162 2024-03-23T17:39:37.232 INFO:teuthology.orchestra.run.smithi152.stdout:(142/146): openssl-libs-1.1.1k-12.el8_9.x86_64. 9.3 MB/s | 1.5 MB 00:00 2024-03-23T17:39:37.293 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libcephsqlite-2:18.2.2-954.g82316b04.el8.x86_64 19/162 2024-03-23T17:39:37.311 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libpq-13.11-1.el8.x86_64 14/162 2024-03-23T17:39:37.357 INFO:teuthology.orchestra.run.smithi152.stdout:(143/146): xfsdump-3.1.8-7.el8_9.x86_64.rpm 2.6 MB/s | 333 kB 00:00 2024-03-23T17:39:37.426 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libradosstriper1-2:18.2.2-954.g82316b04.el8.x86_ 20/162 2024-03-23T17:39:37.529 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libradosstriper1-2:18.2.2-954.g82316b04.el8.x86_ 20/162 2024-03-23T17:39:37.541 INFO:teuthology.orchestra.run.smithi152.stdout:(144/146): xfsprogs-5.0.0-12.el8.x86_64.rpm 5.8 MB/s | 1.1 MB 00:00 2024-03-23T17:39:37.601 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 15/162 2024-03-23T17:39:37.735 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 21/162 2024-03-23T17:39:37.816 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/162 2024-03-23T17:39:37.902 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 21/162 2024-03-23T17:39:38.088 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 22/162 2024-03-23T17:39:38.174 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 17/162 2024-03-23T17:39:38.469 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 18/162 2024-03-23T17:39:38.613 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcephsqlite-2:18.2.2-954.g82316b04.el8.x86_64 19/162 2024-03-23T17:39:38.718 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libcephsqlite-2:18.2.2-954.g82316b04.el8.x86_64 19/162 2024-03-23T17:39:38.771 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 22/162 2024-03-23T17:39:38.862 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libradosstriper1-2:18.2.2-954.g82316b04.el8.x86_ 20/162 2024-03-23T17:39:38.935 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 23/162 2024-03-23T17:39:38.940 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libradosstriper1-2:18.2.2-954.g82316b04.el8.x86_ 20/162 2024-03-23T17:39:39.149 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 21/162 2024-03-23T17:39:39.396 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 22/162 2024-03-23T17:39:39.594 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 23/162 2024-03-23T17:39:39.608 INFO:teuthology.orchestra.run.smithi152.stdout:(145/146): librados2-18.2.2-954.g82316b04.el8.x 501 kB/s | 3.7 MB 00:07 2024-03-23T17:39:39.609 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 23/162 2024-03-23T17:39:39.785 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : openblas-0.3.15-6.el8.x86_64 24/162 2024-03-23T17:39:39.787 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 24/162 2024-03-23T17:39:39.933 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 24/162 2024-03-23T17:39:40.104 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 25/162 2024-03-23T17:39:40.598 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 24/162 2024-03-23T17:39:40.728 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 25/162 2024-03-23T17:39:40.867 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 25/162 2024-03-23T17:39:40.895 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 26/162 2024-03-23T17:39:41.009 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 26/162 2024-03-23T17:39:41.190 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 27/162 2024-03-23T17:39:41.449 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 28/162 2024-03-23T17:39:41.534 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 26/162 2024-03-23T17:39:41.750 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 29/162 2024-03-23T17:39:41.753 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : openblas-0.3.15-6.el8.x86_64 27/162 2024-03-23T17:39:41.977 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 30/162 2024-03-23T17:39:42.175 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-ceph-common-2:18.2.2-954.g82316b04.el8.x 31/162 2024-03-23T17:39:42.307 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-ceph-argparse-2:18.2.2-954.g82316b04.el8 32/162 2024-03-23T17:39:42.439 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 27/162 2024-03-23T17:39:42.548 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cephfs-2:18.2.2-954.g82316b04.el8.x86_64 33/162 2024-03-23T17:39:42.709 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 28/162 2024-03-23T17:39:42.942 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 29/162 2024-03-23T17:39:43.247 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 30/162 2024-03-23T17:39:43.434 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-ceph-common-2:18.2.2-954.g82316b04.el8.x 31/162 2024-03-23T17:39:43.606 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-ceph-argparse-2:18.2.2-954.g82316b04.el8 32/162 2024-03-23T17:39:43.608 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 34/162 2024-03-23T17:39:43.887 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cephfs-2:18.2.2-954.g82316b04.el8.x86_64 33/162 2024-03-23T17:39:43.976 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 35/162 2024-03-23T17:39:44.086 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 34/162 2024-03-23T17:39:44.233 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 36/162 2024-03-23T17:39:44.409 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 37/162 2024-03-23T17:39:45.827 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 34/162 2024-03-23T17:39:46.196 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 35/162 2024-03-23T17:39:46.411 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 37/162 2024-03-23T17:39:46.715 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 38/162 2024-03-23T17:39:46.977 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-rsa-4.9-2.el8.noarch 39/162 2024-03-23T17:39:47.180 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/162 2024-03-23T17:39:47.197 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 36/162 2024-03-23T17:39:47.374 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-rbd-2:18.2.2-954.g82316b04.el8.x86_64 41/162 2024-03-23T17:39:47.525 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : librados-devel-2:18.2.2-954.g82316b04.el8.x86_64 42/162 2024-03-23T17:39:47.549 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 37/162 2024-03-23T17:39:47.769 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : libmount-2.32.1-43.el8.x86_64 43/162 2024-03-23T17:39:47.828 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 38/162 2024-03-23T17:39:47.875 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libmount-2.32.1-43.el8.x86_64 43/162 2024-03-23T17:39:48.097 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rsa-4.9-2.el8.noarch 39/162 2024-03-23T17:39:48.100 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : libfdisk-2.32.1-43.el8.x86_64 44/162 2024-03-23T17:39:48.175 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libfdisk-2.32.1-43.el8.x86_64 44/162 2024-03-23T17:39:48.371 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/162 2024-03-23T17:39:48.393 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libuuid-devel-2.32.1-43.el8.x86_64 45/162 2024-03-23T17:39:48.558 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rbd-2:18.2.2-954.g82316b04.el8.x86_64 41/162 2024-03-23T17:39:48.579 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 46/162 2024-03-23T17:39:48.750 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librados-devel-2:18.2.2-954.g82316b04.el8.x86_64 42/162 2024-03-23T17:39:48.800 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : libsmartcols-2.32.1-43.el8.x86_64 47/162 2024-03-23T17:39:48.996 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : libmount-2.32.1-43.el8.x86_64 43/162 2024-03-23T17:39:49.074 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libmount-2.32.1-43.el8.x86_64 43/162 2024-03-23T17:39:49.310 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : libfdisk-2.32.1-43.el8.x86_64 44/162 2024-03-23T17:39:49.382 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libfdisk-2.32.1-43.el8.x86_64 44/162 2024-03-23T17:39:49.388 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libsmartcols-2.32.1-43.el8.x86_64 47/162 2024-03-23T17:39:49.627 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libuuid-devel-2.32.1-43.el8.x86_64 45/162 2024-03-23T17:39:49.717 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : util-linux-2.32.1-43.el8.x86_64 48/162 2024-03-23T17:39:49.749 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: util-linux-2.32.1-43.el8.x86_64 48/162 2024-03-23T17:39:49.837 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 46/162 2024-03-23T17:39:49.955 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : fuse-2.9.7-17.el8.x86_64 49/162 2024-03-23T17:39:50.051 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : libsmartcols-2.32.1-43.el8.x86_64 47/162 2024-03-23T17:39:50.142 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 50/162 2024-03-23T17:39:50.382 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 51/162 2024-03-23T17:39:50.481 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libsmartcols-2.32.1-43.el8.x86_64 47/162 2024-03-23T17:39:50.529 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 52/162 2024-03-23T17:39:50.697 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 53/162 2024-03-23T17:39:50.777 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : util-linux-2.32.1-43.el8.x86_64 48/162 2024-03-23T17:39:50.802 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: util-linux-2.32.1-43.el8.x86_64 48/162 2024-03-23T17:39:51.012 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-packaging-16.8-9.el8.noarch 49/162 2024-03-23T17:39:51.027 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 54/162 2024-03-23T17:39:51.202 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : oniguruma-6.8.2-2.1.el8_9.x86_64 50/162 2024-03-23T17:39:51.286 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: oniguruma-6.8.2-2.1.el8_9.x86_64 50/162 2024-03-23T17:39:51.298 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 55/162 2024-03-23T17:39:51.349 INFO:teuthology.orchestra.run.smithi152.stdout:(146/146): ceph-test-18.2.2-954.g82316b04.el8.x 1.2 MB/s | 47 MB 00:39 2024-03-23T17:39:51.351 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2024-03-23T17:39:51.352 INFO:teuthology.orchestra.run.smithi152.stdout:Total 3.5 MB/s | 213 MB 01:01 2024-03-23T17:39:51.531 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 56/162 2024-03-23T17:39:51.596 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : jq-1.6-7.el8.x86_64 51/162 2024-03-23T17:39:51.701 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 57/162 2024-03-23T17:39:51.835 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : automake-1.16.1-8.el8.noarch 52/162 2024-03-23T17:39:51.934 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 58/162 2024-03-23T17:39:52.010 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : uuid-1.6.2-43.el8.x86_64 53/162 2024-03-23T17:39:52.071 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 53/162 2024-03-23T17:39:52.282 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 54/162 2024-03-23T17:39:52.347 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 59/162 2024-03-23T17:39:52.392 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2024-03-23T17:39:52.508 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 55/162 2024-03-23T17:39:52.614 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cryptography-3.2.1-7.el8_9.x86_64 60/162 2024-03-23T17:39:52.669 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 56/162 2024-03-23T17:39:52.786 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2024-03-23T17:39:52.787 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2024-03-23T17:39:52.897 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : fuse-2.9.7-17.el8.x86_64 57/162 2024-03-23T17:39:52.920 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 61/162 2024-03-23T17:39:53.124 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 58/162 2024-03-23T17:39:53.151 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 62/162 2024-03-23T17:39:53.297 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 59/162 2024-03-23T17:39:53.340 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 63/162 2024-03-23T17:39:53.521 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 60/162 2024-03-23T17:39:53.567 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 64/162 2024-03-23T17:39:53.705 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 61/162 2024-03-23T17:39:53.792 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 65/162 2024-03-23T17:39:53.994 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 66/162 2024-03-23T17:39:53.995 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2024-03-23T17:39:54.005 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2024-03-23T17:39:54.027 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 62/162 2024-03-23T17:39:54.169 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 67/162 2024-03-23T17:39:54.321 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 63/162 2024-03-23T17:39:54.346 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 68/162 2024-03-23T17:39:54.556 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 64/162 2024-03-23T17:39:54.647 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 69/162 2024-03-23T17:39:54.738 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 65/162 2024-03-23T17:39:54.799 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : oniguruma-6.8.2-2.1.el8_9.x86_64 70/162 2024-03-23T17:39:54.919 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 66/162 2024-03-23T17:39:54.932 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: oniguruma-6.8.2-2.1.el8_9.x86_64 70/162 2024-03-23T17:39:55.215 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : jq-1.6-7.el8.x86_64 71/162 2024-03-23T17:39:55.323 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 67/162 2024-03-23T17:39:55.460 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : automake-1.16.1-8.el8.noarch 72/162 2024-03-23T17:39:55.596 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cryptography-3.2.1-7.el8_9.x86_64 68/162 2024-03-23T17:39:55.616 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : uuid-1.6.2-43.el8.x86_64 73/162 2024-03-23T17:39:55.701 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 73/162 2024-03-23T17:39:55.849 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 74/162 2024-03-23T17:39:55.965 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 69/162 2024-03-23T17:39:56.058 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 75/162 2024-03-23T17:39:56.166 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 70/162 2024-03-23T17:39:56.228 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 76/162 2024-03-23T17:39:56.360 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 71/162 2024-03-23T17:39:56.443 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-packaging-16.8-9.el8.noarch 77/162 2024-03-23T17:39:56.600 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 78/162 2024-03-23T17:39:56.629 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 72/162 2024-03-23T17:39:56.789 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 79/162 2024-03-23T17:39:56.904 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 73/162 2024-03-23T17:39:57.080 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 80/162 2024-03-23T17:39:57.092 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 74/162 2024-03-23T17:39:57.290 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 81/162 2024-03-23T17:39:57.319 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 75/162 2024-03-23T17:39:57.471 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 82/162 2024-03-23T17:39:57.496 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 76/162 2024-03-23T17:39:57.628 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 83/162 2024-03-23T17:39:57.800 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 77/162 2024-03-23T17:39:57.836 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 84/162 2024-03-23T17:39:57.999 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 78/162 2024-03-23T17:39:58.064 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 85/162 2024-03-23T17:39:58.211 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 79/162 2024-03-23T17:39:58.237 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 86/162 2024-03-23T17:39:58.386 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 87/162 2024-03-23T17:39:58.543 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 80/162 2024-03-23T17:39:58.556 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 88/162 2024-03-23T17:39:58.755 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 89/162 2024-03-23T17:39:58.823 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 81/162 2024-03-23T17:39:58.921 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 90/162 2024-03-23T17:39:59.012 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 82/162 2024-03-23T17:39:59.122 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 91/162 2024-03-23T17:39:59.202 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 83/162 2024-03-23T17:39:59.401 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-portend-2.6-1.el8.noarch 92/162 2024-03-23T17:39:59.407 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 84/162 2024-03-23T17:39:59.602 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 85/162 2024-03-23T17:39:59.779 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2024-03-23T17:39:59.860 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 86/162 2024-03-23T17:39:59.913 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/1 2024-03-23T17:40:00.060 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 93/162 2024-03-23T17:40:00.069 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 87/162 2024-03-23T17:40:00.191 INFO:teuthology.orchestra.run.smithi152.stdout: Upgrading : openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-03-23T17:40:00.213 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 88/162 2024-03-23T17:40:00.235 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 94/162 2024-03-23T17:40:00.442 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-03-23T17:40:00.474 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 95/162 2024-03-23T17:40:00.485 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 89/162 2024-03-23T17:40:00.659 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/162 2024-03-23T17:40:00.719 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 90/162 2024-03-23T17:40:00.837 INFO:teuthology.orchestra.run.smithi152.stdout: Upgrading : libuuid-2.32.1-43.el8.x86_64 3/162 2024-03-23T17:40:00.888 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 91/162 2024-03-23T17:40:00.916 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: libuuid-2.32.1-43.el8.x86_64 3/162 2024-03-23T17:40:01.129 INFO:teuthology.orchestra.run.smithi152.stdout: Upgrading : libblkid-2.32.1-43.el8.x86_64 4/162 2024-03-23T17:40:01.167 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-portend-2.6-1.el8.noarch 92/162 2024-03-23T17:40:01.186 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: libblkid-2.32.1-43.el8.x86_64 4/162 2024-03-23T17:40:01.408 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/162 2024-03-23T17:40:01.431 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 93/162 2024-03-23T17:40:01.589 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 94/162 2024-03-23T17:40:01.619 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : librabbitmq-0.9.0-5.el8_9.x86_64 6/162 2024-03-23T17:40:01.750 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 96/162 2024-03-23T17:40:01.846 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 7/162 2024-03-23T17:40:01.855 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 95/162 2024-03-23T17:40:01.934 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 7/162 2024-03-23T17:40:02.050 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 97/162 2024-03-23T17:40:02.129 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-03-23T17:40:02.318 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 98/162 2024-03-23T17:40:02.535 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-03-23T17:40:02.641 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 99/162 2024-03-23T17:40:02.779 INFO:teuthology.orchestra.run.smithi152.stdout: Upgrading : librados2-2:18.2.2-954.g82316b04.el8.x86_64 9/162 2024-03-23T17:40:02.902 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 100/162 2024-03-23T17:40:02.967 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 96/162 2024-03-23T17:40:03.150 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 101/162 2024-03-23T17:40:03.166 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: librados2-2:18.2.2-954.g82316b04.el8.x86_64 9/162 2024-03-23T17:40:03.294 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 97/162 2024-03-23T17:40:03.403 INFO:teuthology.orchestra.run.smithi152.stdout: Upgrading : librbd1-2:18.2.2-954.g82316b04.el8.x86_64 10/162 2024-03-23T17:40:03.523 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 98/162 2024-03-23T17:40:03.528 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: librbd1-2:18.2.2-954.g82316b04.el8.x86_64 10/162 2024-03-23T17:40:03.728 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libcephfs2-2:18.2.2-954.g82316b04.el8.x86_64 11/162 2024-03-23T17:40:03.824 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: libcephfs2-2:18.2.2-954.g82316b04.el8.x86_64 11/162 2024-03-23T17:40:03.841 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 99/162 2024-03-23T17:40:03.976 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 102/162 2024-03-23T17:40:04.108 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 100/162 2024-03-23T17:40:04.165 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-rados-2:18.2.2-954.g82316b04.el8.x86_64 12/162 2024-03-23T17:40:04.215 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : librgw2-2:18.2.2-954.g82316b04.el8.x86_64 103/162 2024-03-23T17:40:04.313 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librgw2-2:18.2.2-954.g82316b04.el8.x86_64 103/162 2024-03-23T17:40:04.331 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 101/162 2024-03-23T17:40:04.473 INFO:teuthology.orchestra.run.smithi152.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-03-23T17:40:04.474 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-rgw-2:18.2.2-954.g82316b04.el8.x86_64 104/162 2024-03-23T17:40:04.541 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-03-23T17:40:04.793 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libpq-13.11-1.el8.x86_64 14/162 2024-03-23T17:40:05.057 INFO:teuthology.orchestra.run.smithi152.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 15/162 2024-03-23T17:40:05.130 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 102/162 2024-03-23T17:40:05.231 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/162 2024-03-23T17:40:05.273 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librgw2-2:18.2.2-954.g82316b04.el8.x86_64 103/162 2024-03-23T17:40:05.351 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librgw2-2:18.2.2-954.g82316b04.el8.x86_64 103/162 2024-03-23T17:40:05.574 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rgw-2:18.2.2-954.g82316b04.el8.x86_64 104/162 2024-03-23T17:40:05.642 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 17/162 2024-03-23T17:40:05.960 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 18/162 2024-03-23T17:40:06.137 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libcephsqlite-2:18.2.2-954.g82316b04.el8.x86_64 19/162 2024-03-23T17:40:06.243 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: libcephsqlite-2:18.2.2-954.g82316b04.el8.x86_64 19/162 2024-03-23T17:40:06.428 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libradosstriper1-2:18.2.2-954.g82316b04.el8.x86_ 20/162 2024-03-23T17:40:06.491 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: libradosstriper1-2:18.2.2-954.g82316b04.el8.x86_ 20/162 2024-03-23T17:40:06.657 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 21/162 2024-03-23T17:40:06.972 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 22/162 2024-03-23T17:40:07.225 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 23/162 2024-03-23T17:40:07.428 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 24/162 2024-03-23T17:40:07.556 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 24/162 2024-03-23T17:40:07.736 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 25/162 2024-03-23T17:40:08.343 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 25/162 2024-03-23T17:40:08.520 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 26/162 2024-03-23T17:40:09.108 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 26/162 2024-03-23T17:40:09.286 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : openblas-0.3.15-6.el8.x86_64 27/162 2024-03-23T17:40:09.447 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-common-2:18.2.2-954.g82316b04.el8.x86_64 105/162 2024-03-23T17:40:09.482 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-common-2:18.2.2-954.g82316b04.el8.x86_64 105/162 2024-03-23T17:40:09.673 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-common-2:18.2.2-954.g82316b04.el8.x86_64 105/162 2024-03-23T17:40:09.690 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-common-2:18.2.2-954.g82316b04.el8.x86_64 105/162 2024-03-23T17:40:10.015 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 27/162 2024-03-23T17:40:10.264 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-common-2:18.2.2-954.g82316b04.el8.x86_64 105/162 2024-03-23T17:40:10.284 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 28/162 2024-03-23T17:40:10.372 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-common-2:18.2.2-954.g82316b04.el8.x86_64 105/162 2024-03-23T17:40:10.516 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 29/162 2024-03-23T17:40:10.582 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-base-2:18.2.2-954.g82316b04.el8.x86_64 106/162 2024-03-23T17:40:10.660 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-base-2:18.2.2-954.g82316b04.el8.x86_64 106/162 2024-03-23T17:40:10.796 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 30/162 2024-03-23T17:40:10.889 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-selinux-2:18.2.2-954.g82316b04.el8.x86_64 107/162 2024-03-23T17:40:10.993 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-ceph-common-2:18.2.2-954.g82316b04.el8.x 31/162 2024-03-23T17:40:11.155 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-ceph-argparse-2:18.2.2-954.g82316b04.el8 32/162 2024-03-23T17:40:11.344 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-base-2:18.2.2-954.g82316b04.el8.x86_64 106/162 2024-03-23T17:40:11.466 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-cephfs-2:18.2.2-954.g82316b04.el8.x86_64 33/162 2024-03-23T17:40:11.512 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-base-2:18.2.2-954.g82316b04.el8.x86_64 106/162 2024-03-23T17:40:11.672 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-selinux-2:18.2.2-954.g82316b04.el8.x86_64 107/162 2024-03-23T17:40:11.693 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 34/162 2024-03-23T17:40:13.450 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 34/162 2024-03-23T17:40:13.829 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 35/162 2024-03-23T17:40:14.849 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 36/162 2024-03-23T17:40:15.224 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 37/162 2024-03-23T17:40:15.494 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 38/162 2024-03-23T17:40:15.713 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-rsa-4.9-2.el8.noarch 39/162 2024-03-23T17:40:15.946 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/162 2024-03-23T17:40:16.166 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-rbd-2:18.2.2-954.g82316b04.el8.x86_64 41/162 2024-03-23T17:40:16.316 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : librados-devel-2:18.2.2-954.g82316b04.el8.x86_64 42/162 2024-03-23T17:40:16.575 INFO:teuthology.orchestra.run.smithi152.stdout: Upgrading : libmount-2.32.1-43.el8.x86_64 43/162 2024-03-23T17:40:16.643 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: libmount-2.32.1-43.el8.x86_64 43/162 2024-03-23T17:40:16.877 INFO:teuthology.orchestra.run.smithi152.stdout: Upgrading : libfdisk-2.32.1-43.el8.x86_64 44/162 2024-03-23T17:40:16.953 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: libfdisk-2.32.1-43.el8.x86_64 44/162 2024-03-23T17:40:17.207 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libuuid-devel-2.32.1-43.el8.x86_64 45/162 2024-03-23T17:40:17.821 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 46/162 2024-03-23T17:40:18.018 INFO:teuthology.orchestra.run.smithi152.stdout: Upgrading : libsmartcols-2.32.1-43.el8.x86_64 47/162 2024-03-23T17:40:18.477 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: libsmartcols-2.32.1-43.el8.x86_64 47/162 2024-03-23T17:40:18.843 INFO:teuthology.orchestra.run.smithi152.stdout: Upgrading : util-linux-2.32.1-43.el8.x86_64 48/162 2024-03-23T17:40:18.871 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: util-linux-2.32.1-43.el8.x86_64 48/162 2024-03-23T17:40:19.051 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : oniguruma-6.8.2-2.1.el8_9.x86_64 49/162 2024-03-23T17:40:19.166 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: oniguruma-6.8.2-2.1.el8_9.x86_64 49/162 2024-03-23T17:40:19.419 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : jq-1.6-7.el8.x86_64 50/162 2024-03-23T17:40:19.643 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : automake-1.16.1-8.el8.noarch 51/162 2024-03-23T17:40:19.843 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : uuid-1.6.2-43.el8.x86_64 52/162 2024-03-23T17:40:19.901 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 52/162 2024-03-23T17:40:20.066 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 53/162 2024-03-23T17:40:20.258 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 54/162 2024-03-23T17:40:20.443 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 55/162 2024-03-23T17:40:20.652 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-packaging-16.8-9.el8.noarch 56/162 2024-03-23T17:40:20.872 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : fuse-2.9.7-17.el8.x86_64 57/162 2024-03-23T17:40:21.081 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 58/162 2024-03-23T17:40:21.306 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 59/162 2024-03-23T17:40:21.530 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 60/162 2024-03-23T17:40:21.721 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 61/162 2024-03-23T17:40:22.046 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 62/162 2024-03-23T17:40:22.356 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 63/162 2024-03-23T17:40:22.623 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 64/162 2024-03-23T17:40:22.780 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 65/162 2024-03-23T17:40:23.085 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 66/162 2024-03-23T17:40:23.472 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 67/162 2024-03-23T17:40:23.787 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-cryptography-3.2.1-7.el8_9.x86_64 68/162 2024-03-23T17:40:24.114 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 69/162 2024-03-23T17:40:24.341 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 70/162 2024-03-23T17:40:24.564 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 71/162 2024-03-23T17:40:24.811 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 72/162 2024-03-23T17:40:25.104 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 73/162 2024-03-23T17:40:25.334 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 74/162 2024-03-23T17:40:25.527 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 75/162 2024-03-23T17:40:25.747 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 76/162 2024-03-23T17:40:26.052 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 77/162 2024-03-23T17:40:26.266 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 78/162 2024-03-23T17:40:26.428 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 79/162 2024-03-23T17:40:26.759 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 80/162 2024-03-23T17:40:27.023 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 81/162 2024-03-23T17:40:27.204 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 82/162 2024-03-23T17:40:27.410 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 83/162 2024-03-23T17:40:27.663 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 84/162 2024-03-23T17:40:27.861 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 85/162 2024-03-23T17:40:28.135 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 86/162 2024-03-23T17:40:28.353 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 87/162 2024-03-23T17:40:28.522 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 88/162 2024-03-23T17:40:28.794 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 89/162 2024-03-23T17:40:29.045 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 90/162 2024-03-23T17:40:29.205 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 91/162 2024-03-23T17:40:29.477 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-portend-2.6-1.el8.noarch 92/162 2024-03-23T17:40:29.732 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 93/162 2024-03-23T17:40:29.915 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 94/162 2024-03-23T17:40:30.183 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 95/162 2024-03-23T17:40:30.386 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-selinux-2:18.2.2-954.g82316b04.el8.x86_64 107/162 2024-03-23T17:40:30.386 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /sys 2024-03-23T17:40:30.386 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /proc 2024-03-23T17:40:30.386 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /dev 2024-03-23T17:40:30.386 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /run 2024-03-23T17:40:30.386 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /mnt 2024-03-23T17:40:30.387 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /var/tmp 2024-03-23T17:40:30.387 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /home 2024-03-23T17:40:30.387 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /tmp 2024-03-23T17:40:30.387 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /dev 2024-03-23T17:40:30.387 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-23T17:40:30.776 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-volume-2:18.2.2-954.g82316b04.el8.noarch 108/162 2024-03-23T17:40:31.300 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 96/162 2024-03-23T17:40:31.611 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 97/162 2024-03-23T17:40:31.890 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 98/162 2024-03-23T17:40:32.234 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 99/162 2024-03-23T17:40:32.406 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-volume-2:18.2.2-954.g82316b04.el8.noarch 108/162 2024-03-23T17:40:32.471 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-selinux-2:18.2.2-954.g82316b04.el8.x86_64 107/162 2024-03-23T17:40:32.471 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /sys 2024-03-23T17:40:32.471 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /proc 2024-03-23T17:40:32.471 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /dev 2024-03-23T17:40:32.471 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /run 2024-03-23T17:40:32.472 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /mnt 2024-03-23T17:40:32.472 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /var/tmp 2024-03-23T17:40:32.472 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /home 2024-03-23T17:40:32.472 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /tmp 2024-03-23T17:40:32.472 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /dev 2024-03-23T17:40:32.472 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:40:32.516 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 100/162 2024-03-23T17:40:32.610 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-osd-2:18.2.2-954.g82316b04.el8.x86_64 109/162 2024-03-23T17:40:32.724 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 101/162 2024-03-23T17:40:32.830 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-osd-2:18.2.2-954.g82316b04.el8.x86_64 109/162 2024-03-23T17:40:32.870 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-volume-2:18.2.2-954.g82316b04.el8.noarch 108/162 2024-03-23T17:40:33.017 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mds-2:18.2.2-954.g82316b04.el8.x86_64 110/162 2024-03-23T17:40:33.490 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 102/162 2024-03-23T17:40:33.492 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mds-2:18.2.2-954.g82316b04.el8.x86_64 110/162 2024-03-23T17:40:33.692 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mon-2:18.2.2-954.g82316b04.el8.x86_64 111/162 2024-03-23T17:40:33.710 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mon-2:18.2.2-954.g82316b04.el8.x86_64 111/162 2024-03-23T17:40:33.715 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : librgw2-2:18.2.2-954.g82316b04.el8.x86_64 103/162 2024-03-23T17:40:33.773 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: librgw2-2:18.2.2-954.g82316b04.el8.x86_64 103/162 2024-03-23T17:40:33.958 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-rgw-2:18.2.2-954.g82316b04.el8.x86_64 104/162 2024-03-23T17:40:34.670 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-volume-2:18.2.2-954.g82316b04.el8.noarch 108/162 2024-03-23T17:40:34.824 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-osd-2:18.2.2-954.g82316b04.el8.x86_64 109/162 2024-03-23T17:40:35.089 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-osd-2:18.2.2-954.g82316b04.el8.x86_64 109/162 2024-03-23T17:40:35.253 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mds-2:18.2.2-954.g82316b04.el8.x86_64 110/162 2024-03-23T17:40:35.463 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: cephadm-2:18.2.2-954.g82316b04.el8.noarch 112/162 2024-03-23T17:40:35.621 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : cephadm-2:18.2.2-954.g82316b04.el8.noarch 112/162 2024-03-23T17:40:35.788 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mds-2:18.2.2-954.g82316b04.el8.x86_64 110/162 2024-03-23T17:40:35.789 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-954.g82316b04.el 113/162 2024-03-23T17:40:35.950 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-954.g82316b04.e 114/162 2024-03-23T17:40:35.969 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mon-2:18.2.2-954.g82316b04.el8.x86_64 111/162 2024-03-23T17:40:36.011 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mon-2:18.2.2-954.g82316b04.el8.x86_64 111/162 2024-03-23T17:40:36.183 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-954.g82316b04.el8.noar 115/162 2024-03-23T17:40:36.411 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-954.g82316b04.el8.noar 115/162 2024-03-23T17:40:36.768 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-954.g82316b04.el8.no 116/162 2024-03-23T17:40:37.733 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-954.g82316b04.el8.no 116/162 2024-03-23T17:40:37.735 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-common-2:18.2.2-954.g82316b04.el8.x86_64 105/162 2024-03-23T17:40:37.899 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-954.g8231 117/162 2024-03-23T17:40:37.991 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-common-2:18.2.2-954.g82316b04.el8.x86_64 105/162 2024-03-23T17:40:37.992 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-954.g8231 117/162 2024-03-23T17:40:38.329 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-954.g82316b04.el8 118/162 2024-03-23T17:40:38.484 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-2:18.2.2-954.g82316b04.el8.x86_64 119/162 2024-03-23T17:40:38.523 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-2:18.2.2-954.g82316b04.el8.x86_64 119/162 2024-03-23T17:40:38.582 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-common-2:18.2.2-954.g82316b04.el8.x86_64 105/162 2024-03-23T17:40:38.722 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-rook-2:18.2.2-954.g82316b04.el8.noarch 120/162 2024-03-23T17:40:38.734 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-954.g82316b04.el8.noarch 120/162 2024-03-23T17:40:38.825 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: cephadm-2:18.2.2-954.g82316b04.el8.noarch 112/162 2024-03-23T17:40:38.960 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : cephadm-2:18.2.2-954.g82316b04.el8.noarch 112/162 2024-03-23T17:40:39.120 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-954.g82316b04.el 113/162 2024-03-23T17:40:39.313 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-954.g82316b04.e 114/162 2024-03-23T17:40:39.455 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-base-2:18.2.2-954.g82316b04.el8.x86_64 106/162 2024-03-23T17:40:39.543 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-954.g82316b04.el8.noar 115/162 2024-03-23T17:40:39.816 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-954.g82316b04.el8.noar 115/162 2024-03-23T17:40:39.976 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-base-2:18.2.2-954.g82316b04.el8.x86_64 106/162 2024-03-23T17:40:39.993 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-2:18.2.2-954.g82316b04.el8.x86_64 121/162 2024-03-23T17:40:40.073 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-954.g82316b04.el8.no 116/162 2024-03-23T17:40:40.225 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-radosgw-2:18.2.2-954.g82316b04.el8.x86_64 122/162 2024-03-23T17:40:40.260 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-954.g82316b04.el8.x86_64 122/162 2024-03-23T17:40:40.441 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-954.g82316b 123/162 2024-03-23T17:40:40.481 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-selinux-2:18.2.2-954.g82316b04.el8.x86_64 107/162 2024-03-23T17:40:40.484 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-954.g82316b 123/162 2024-03-23T17:40:40.707 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : cephfs-mirror-2:18.2.2-954.g82316b04.el8.x86_64 124/162 2024-03-23T17:40:41.056 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: cephfs-mirror-2:18.2.2-954.g82316b04.el8.x86_64 124/162 2024-03-23T17:40:41.151 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-954.g82316b04.el8.no 116/162 2024-03-23T17:40:41.266 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rbd-mirror-2:18.2.2-954.g82316b04.el8.x86_64 125/162 2024-03-23T17:40:41.310 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-954.g8231 117/162 2024-03-23T17:40:41.440 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-954.g8231 117/162 2024-03-23T17:40:41.845 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-954.g82316b04.el8 118/162 2024-03-23T17:40:42.015 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-2:18.2.2-954.g82316b04.el8.x86_64 119/162 2024-03-23T17:40:42.084 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-2:18.2.2-954.g82316b04.el8.x86_64 119/162 2024-03-23T17:40:42.260 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-rook-2:18.2.2-954.g82316b04.el8.noarch 120/162 2024-03-23T17:40:42.289 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-954.g82316b04.el8.noarch 120/162 2024-03-23T17:40:43.603 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-2:18.2.2-954.g82316b04.el8.x86_64 121/162 2024-03-23T17:40:43.823 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-radosgw-2:18.2.2-954.g82316b04.el8.x86_64 122/162 2024-03-23T17:40:43.885 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-954.g82316b04.el8.x86_64 122/162 2024-03-23T17:40:44.052 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-954.g82316b 123/162 2024-03-23T17:40:44.123 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-954.g82316b 123/162 2024-03-23T17:40:44.327 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : cephfs-mirror-2:18.2.2-954.g82316b04.el8.x86_64 124/162 2024-03-23T17:40:44.727 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: cephfs-mirror-2:18.2.2-954.g82316b04.el8.x86_64 124/162 2024-03-23T17:40:44.932 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : rbd-mirror-2:18.2.2-954.g82316b04.el8.x86_64 125/162 2024-03-23T17:40:45.749 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: rbd-mirror-2:18.2.2-954.g82316b04.el8.x86_64 125/162 2024-03-23T17:40:46.747 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-test-2:18.2.2-954.g82316b04.el8.x86_64 126/162 2024-03-23T17:40:47.069 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : openssl-devel-1:1.1.1k-12.el8_9.x86_64 127/162 2024-03-23T17:40:47.380 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 128/162 2024-03-23T17:40:47.833 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-fuse-2:18.2.2-954.g82316b04.el8.x86_64 129/162 2024-03-23T17:40:48.174 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 130/162 2024-03-23T17:40:48.412 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libtool-2.4.6-25.el8.x86_64 131/162 2024-03-23T17:40:48.425 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 131/162 2024-03-23T17:40:48.650 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : util-linux-user-2.32.1-43.el8.x86_64 132/162 2024-03-23T17:40:48.869 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 133/162 2024-03-23T17:40:49.081 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcephfs-devel-2:18.2.2-954.g82316b04.el8.x86_6 134/162 2024-03-23T17:40:49.731 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: rbd-mirror-2:18.2.2-954.g82316b04.el8.x86_64 125/162 2024-03-23T17:40:50.263 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-test-2:18.2.2-954.g82316b04.el8.x86_64 126/162 2024-03-23T17:40:50.605 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 127/162 2024-03-23T17:40:50.851 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libtool-2.4.6-25.el8.x86_64 128/162 2024-03-23T17:40:51.331 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-03-23T17:40:51.639 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-03-23T17:40:51.676 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-03-23T17:40:51.857 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 128/162 2024-03-23T17:40:52.098 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 136/162 2024-03-23T17:40:52.156 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : openssl-devel-1:1.1.1k-12.el8_9.x86_64 129/162 2024-03-23T17:40:52.446 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 130/162 2024-03-23T17:40:52.470 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 137/162 2024-03-23T17:40:52.650 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-fuse-2:18.2.2-954.g82316b04.el8.x86_64 131/162 2024-03-23T17:40:52.742 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : xfsdump-3.1.8-7.el8_9.x86_64 138/162 2024-03-23T17:40:52.906 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : util-linux-user-2.32.1-43.el8.x86_64 132/162 2024-03-23T17:40:52.938 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : cephfs-top-2:18.2.2-954.g82316b04.el8.noarch 139/162 2024-03-23T17:40:53.114 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 133/162 2024-03-23T17:40:53.196 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rbd-fuse-2:18.2.2-954.g82316b04.el8.x86_64 140/162 2024-03-23T17:40:53.351 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libcephfs-devel-2:18.2.2-954.g82316b04.el8.x86_6 134/162 2024-03-23T17:40:53.482 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rbd-nbd-2:18.2.2-954.g82316b04.el8.x86_64 141/162 2024-03-23T17:40:53.858 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : openssl-1:1.1.1k-12.el8_9.x86_64 142/162 2024-03-23T17:40:54.054 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flex-2.6.1-9.el8.x86_64 143/162 2024-03-23T17:40:54.133 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 143/162 2024-03-23T17:40:54.329 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : bison-3.0.4-10.el8.x86_64 144/162 2024-03-23T17:40:54.343 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 144/162 2024-03-23T17:40:54.554 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-13.e 145/162 2024-03-23T17:40:54.563 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-13.e 145/162 2024-03-23T17:40:54.730 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 146/162 2024-03-23T17:40:54.909 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 147/162 2024-03-23T17:40:55.123 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 148/162 2024-03-23T17:40:55.299 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 149/162 2024-03-23T17:40:55.299 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-03-23T17:40:55.598 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-03-23T17:40:55.599 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-03-23T17:40:55.858 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-03-23T17:40:55.858 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 152/162 2024-03-23T17:40:56.118 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 152/162 2024-03-23T17:40:56.364 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 153/162 2024-03-23T17:40:56.529 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 154/162 2024-03-23T17:40:56.700 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-03-23T17:40:56.781 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 155/162 2024-03-23T17:40:56.781 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 156/162 2024-03-23T17:40:57.009 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-03-23T17:40:57.048 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 156/162 2024-03-23T17:40:57.048 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 157/162 2024-03-23T17:40:57.072 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-03-23T17:40:57.306 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 157/162 2024-03-23T17:40:57.306 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 158/162 2024-03-23T17:40:57.523 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 158/162 2024-03-23T17:40:57.523 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 159/162 2024-03-23T17:40:57.546 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 136/162 2024-03-23T17:40:57.772 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 159/162 2024-03-23T17:40:57.773 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-03-23T17:40:57.910 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 137/162 2024-03-23T17:40:58.014 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-03-23T17:40:58.148 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 161/162 2024-03-23T17:40:58.148 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-03-23T17:40:58.196 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : xfsdump-3.1.8-7.el8_9.x86_64 138/162 2024-03-23T17:40:58.426 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : cephfs-top-2:18.2.2-954.g82316b04.el8.noarch 139/162 2024-03-23T17:40:58.651 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : rbd-fuse-2:18.2.2-954.g82316b04.el8.x86_64 140/162 2024-03-23T17:40:58.952 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : rbd-nbd-2:18.2.2-954.g82316b04.el8.x86_64 141/162 2024-03-23T17:40:59.036 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-03-23T17:40:59.036 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-2:18.2.2-954.g82316b04.el8.x86_64 1/162 2024-03-23T17:40:59.037 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-base-2:18.2.2-954.g82316b04.el8.x86_64 2/162 2024-03-23T17:40:59.037 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-common-2:18.2.2-954.g82316b04.el8.x86_64 3/162 2024-03-23T17:40:59.037 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-fuse-2:18.2.2-954.g82316b04.el8.x86_64 4/162 2024-03-23T17:40:59.037 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-954.g82316b 5/162 2024-03-23T17:40:59.037 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mds-2:18.2.2-954.g82316b04.el8.x86_64 6/162 2024-03-23T17:40:59.037 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-2:18.2.2-954.g82316b04.el8.x86_64 7/162 2024-03-23T17:40:59.037 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mon-2:18.2.2-954.g82316b04.el8.x86_64 8/162 2024-03-23T17:40:59.037 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-osd-2:18.2.2-954.g82316b04.el8.x86_64 9/162 2024-03-23T17:40:59.037 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-radosgw-2:18.2.2-954.g82316b04.el8.x86_64 10/162 2024-03-23T17:40:59.037 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-selinux-2:18.2.2-954.g82316b04.el8.x86_64 11/162 2024-03-23T17:40:59.037 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-test-2:18.2.2-954.g82316b04.el8.x86_64 12/162 2024-03-23T17:40:59.037 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : cephfs-mirror-2:18.2.2-954.g82316b04.el8.x86_64 13/162 2024-03-23T17:40:59.037 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcephfs-devel-2:18.2.2-954.g82316b04.el8.x86_6 14/162 2024-03-23T17:40:59.038 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcephfs2-2:18.2.2-954.g82316b04.el8.x86_64 15/162 2024-03-23T17:40:59.038 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcephsqlite-2:18.2.2-954.g82316b04.el8.x86_64 16/162 2024-03-23T17:40:59.038 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librados-devel-2:18.2.2-954.g82316b04.el8.x86_64 17/162 2024-03-23T17:40:59.038 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libradosstriper1-2:18.2.2-954.g82316b04.el8.x86_ 18/162 2024-03-23T17:40:59.038 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librgw2-2:18.2.2-954.g82316b04.el8.x86_64 19/162 2024-03-23T17:40:59.038 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-ceph-argparse-2:18.2.2-954.g82316b04.el8 20/162 2024-03-23T17:40:59.038 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-ceph-common-2:18.2.2-954.g82316b04.el8.x 21/162 2024-03-23T17:40:59.038 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cephfs-2:18.2.2-954.g82316b04.el8.x86_64 22/162 2024-03-23T17:40:59.038 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rados-2:18.2.2-954.g82316b04.el8.x86_64 23/162 2024-03-23T17:40:59.038 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rbd-2:18.2.2-954.g82316b04.el8.x86_64 24/162 2024-03-23T17:40:59.038 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rgw-2:18.2.2-954.g82316b04.el8.x86_64 25/162 2024-03-23T17:40:59.038 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rbd-fuse-2:18.2.2-954.g82316b04.el8.x86_64 26/162 2024-03-23T17:40:59.038 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rbd-mirror-2:18.2.2-954.g82316b04.el8.x86_64 27/162 2024-03-23T17:40:59.051 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rbd-nbd-2:18.2.2-954.g82316b04.el8.x86_64 28/162 2024-03-23T17:40:59.051 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-954.g82316b04.e 29/162 2024-03-23T17:40:59.051 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-954.g82316b04.el8.noar 30/162 2024-03-23T17:40:59.051 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-954.g82316b04.el8.no 31/162 2024-03-23T17:40:59.051 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-954.g8231 32/162 2024-03-23T17:40:59.051 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-954.g82316b04.el8 33/162 2024-03-23T17:40:59.051 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-rook-2:18.2.2-954.g82316b04.el8.noarch 34/162 2024-03-23T17:40:59.051 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-954.g82316b04.el 35/162 2024-03-23T17:40:59.051 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-volume-2:18.2.2-954.g82316b04.el8.noarch 36/162 2024-03-23T17:40:59.051 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : cephadm-2:18.2.2-954.g82316b04.el8.noarch 37/162 2024-03-23T17:40:59.051 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : cephfs-top-2:18.2.2-954.g82316b04.el8.noarch 38/162 2024-03-23T17:40:59.051 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/162 2024-03-23T17:40:59.052 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/162 2024-03-23T17:40:59.052 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/162 2024-03-23T17:40:59.052 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/162 2024-03-23T17:40:59.052 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/162 2024-03-23T17:40:59.052 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/162 2024-03-23T17:40:59.052 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/162 2024-03-23T17:40:59.052 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/162 2024-03-23T17:40:59.052 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/162 2024-03-23T17:40:59.052 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/162 2024-03-23T17:40:59.052 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/162 2024-03-23T17:40:59.052 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/162 2024-03-23T17:40:59.052 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/162 2024-03-23T17:40:59.052 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/162 2024-03-23T17:40:59.052 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/162 2024-03-23T17:40:59.053 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/162 2024-03-23T17:40:59.053 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/162 2024-03-23T17:40:59.053 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/162 2024-03-23T17:40:59.053 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/162 2024-03-23T17:40:59.053 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/162 2024-03-23T17:40:59.053 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/162 2024-03-23T17:40:59.053 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/162 2024-03-23T17:40:59.053 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/162 2024-03-23T17:40:59.053 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/162 2024-03-23T17:40:59.053 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/162 2024-03-23T17:40:59.053 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/162 2024-03-23T17:40:59.053 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/162 2024-03-23T17:40:59.053 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/162 2024-03-23T17:40:59.053 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/162 2024-03-23T17:40:59.054 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/162 2024-03-23T17:40:59.054 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/162 2024-03-23T17:40:59.054 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/162 2024-03-23T17:40:59.054 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/162 2024-03-23T17:40:59.054 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 72/162 2024-03-23T17:40:59.054 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 73/162 2024-03-23T17:40:59.054 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 74/162 2024-03-23T17:40:59.054 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 75/162 2024-03-23T17:40:59.054 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 76/162 2024-03-23T17:40:59.054 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 77/162 2024-03-23T17:40:59.054 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 78/162 2024-03-23T17:40:59.054 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 79/162 2024-03-23T17:40:59.054 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/162 2024-03-23T17:40:59.055 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 81/162 2024-03-23T17:40:59.055 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 82/162 2024-03-23T17:40:59.055 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 83/162 2024-03-23T17:40:59.055 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 84/162 2024-03-23T17:40:59.055 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 85/162 2024-03-23T17:40:59.055 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 86/162 2024-03-23T17:40:59.055 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 87/162 2024-03-23T17:40:59.055 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 88/162 2024-03-23T17:40:59.055 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 89/162 2024-03-23T17:40:59.055 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libuuid-devel-2.32.1-43.el8.x86_64 90/162 2024-03-23T17:40:59.055 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 91/162 2024-03-23T17:40:59.055 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 92/162 2024-03-23T17:40:59.055 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-13.e 93/162 2024-03-23T17:40:59.055 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 94/162 2024-03-23T17:40:59.056 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 95/162 2024-03-23T17:40:59.056 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openssl-devel-1:1.1.1k-12.el8_9.x86_64 96/162 2024-03-23T17:40:59.056 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librabbitmq-0.9.0-5.el8_9.x86_64 97/162 2024-03-23T17:40:59.056 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cryptography-3.2.1-7.el8_9.x86_64 98/162 2024-03-23T17:40:59.056 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 99/162 2024-03-23T17:40:59.056 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 100/162 2024-03-23T17:40:59.056 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 101/162 2024-03-23T17:40:59.056 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 102/162 2024-03-23T17:40:59.056 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 103/162 2024-03-23T17:40:59.056 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 104/162 2024-03-23T17:40:59.056 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : bison-3.0.4-10.el8.x86_64 105/162 2024-03-23T17:40:59.056 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flex-2.6.1-9.el8.x86_64 106/162 2024-03-23T17:40:59.056 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 107/162 2024-03-23T17:40:59.056 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 108/162 2024-03-23T17:40:59.056 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 109/162 2024-03-23T17:40:59.057 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 110/162 2024-03-23T17:40:59.057 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 111/162 2024-03-23T17:40:59.057 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 112/162 2024-03-23T17:40:59.057 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 113/162 2024-03-23T17:40:59.057 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 114/162 2024-03-23T17:40:59.057 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 115/162 2024-03-23T17:40:59.057 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 116/162 2024-03-23T17:40:59.057 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 117/162 2024-03-23T17:40:59.057 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 118/162 2024-03-23T17:40:59.057 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 119/162 2024-03-23T17:40:59.057 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 120/162 2024-03-23T17:40:59.057 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : automake-1.16.1-8.el8.noarch 121/162 2024-03-23T17:40:59.057 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libpq-13.11-1.el8.x86_64 122/162 2024-03-23T17:40:59.057 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : jq-1.6-7.el8.x86_64 123/162 2024-03-23T17:40:59.058 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 124/162 2024-03-23T17:40:59.058 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 125/162 2024-03-23T17:40:59.058 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : oniguruma-6.8.2-2.1.el8_9.x86_64 126/162 2024-03-23T17:40:59.058 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 127/162 2024-03-23T17:40:59.058 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 128/162 2024-03-23T17:40:59.058 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : postgresql-server-10.23-4.module+el8.9.0+21317+d 129/162 2024-03-23T17:40:59.058 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-packaging-16.8-9.el8.noarch 130/162 2024-03-23T17:40:59.058 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librados2-2:18.2.2-954.g82316b04.el8.x86_64 131/162 2024-03-23T17:40:59.058 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 132/162 2024-03-23T17:40:59.058 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librbd1-2:18.2.2-954.g82316b04.el8.x86_64 133/162 2024-03-23T17:40:59.058 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 134/162 2024-03-23T17:40:59.058 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 135/162 2024-03-23T17:40:59.058 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 136/162 2024-03-23T17:40:59.059 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 137/162 2024-03-23T17:40:59.059 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 138/162 2024-03-23T17:40:59.059 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libmount-2.32.1-43.el8.x86_64 139/162 2024-03-23T17:40:59.059 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 140/162 2024-03-23T17:40:59.059 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libsmartcols-2.32.1-43.el8.x86_64 141/162 2024-03-23T17:40:59.059 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 142/162 2024-03-23T17:40:59.059 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libuuid-2.32.1-43.el8.x86_64 143/162 2024-03-23T17:40:59.059 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 144/162 2024-03-23T17:40:59.059 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : util-linux-2.32.1-43.el8.x86_64 145/162 2024-03-23T17:40:59.059 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 146/162 2024-03-23T17:40:59.059 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : util-linux-user-2.32.1-43.el8.x86_64 147/162 2024-03-23T17:40:59.059 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 148/162 2024-03-23T17:40:59.059 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 149/162 2024-03-23T17:40:59.059 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 150/162 2024-03-23T17:40:59.059 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libblkid-2.32.1-43.el8.x86_64 151/162 2024-03-23T17:40:59.060 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 152/162 2024-03-23T17:40:59.060 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libfdisk-2.32.1-43.el8.x86_64 153/162 2024-03-23T17:40:59.060 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 154/162 2024-03-23T17:40:59.060 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openssl-1:1.1.1k-12.el8_9.x86_64 155/162 2024-03-23T17:40:59.060 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 156/162 2024-03-23T17:40:59.060 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openssl-libs-1:1.1.1k-12.el8_9.x86_64 157/162 2024-03-23T17:40:59.060 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 158/162 2024-03-23T17:40:59.060 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : xfsdump-3.1.8-7.el8_9.x86_64 159/162 2024-03-23T17:40:59.060 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 160/162 2024-03-23T17:40:59.060 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 161/162 2024-03-23T17:40:59.268 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : openssl-1:1.1.1k-12.el8_9.x86_64 142/162 2024-03-23T17:40:59.466 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-13.e 143/162 2024-03-23T17:40:59.492 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-13.e 143/162 2024-03-23T17:40:59.672 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 144/162 2024-03-23T17:40:59.893 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-selinux-2:18.2.2-954.g82316b04.el8.x86_64 107/162 2024-03-23T17:40:59.893 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /sys 2024-03-23T17:40:59.893 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /proc 2024-03-23T17:40:59.893 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /dev 2024-03-23T17:40:59.893 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /run 2024-03-23T17:40:59.893 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /mnt 2024-03-23T17:40:59.893 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /var/tmp 2024-03-23T17:40:59.893 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /home 2024-03-23T17:40:59.893 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /tmp 2024-03-23T17:40:59.893 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /dev 2024-03-23T17:40:59.894 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-23T17:40:59.960 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : flex-2.6.1-9.el8.x86_64 145/162 2024-03-23T17:41:00.075 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 145/162 2024-03-23T17:41:00.291 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : bison-3.0.4-10.el8.x86_64 146/162 2024-03-23T17:41:00.311 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-volume-2:18.2.2-954.g82316b04.el8.noarch 108/162 2024-03-23T17:41:00.320 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 146/162 2024-03-23T17:41:00.525 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 147/162 2024-03-23T17:41:00.693 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 148/162 2024-03-23T17:41:00.870 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 149/162 2024-03-23T17:41:00.870 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-03-23T17:41:01.170 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-03-23T17:41:01.170 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-03-23T17:41:01.380 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-03-23T17:41:01.381 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 152/162 2024-03-23T17:41:01.683 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 152/162 2024-03-23T17:41:01.906 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 153/162 2024-03-23T17:41:01.942 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-volume-2:18.2.2-954.g82316b04.el8.noarch 108/162 2024-03-23T17:41:02.099 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 154/162 2024-03-23T17:41:02.148 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-osd-2:18.2.2-954.g82316b04.el8.x86_64 109/162 2024-03-23T17:41:02.309 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 155/162 2024-03-23T17:41:02.309 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 156/162 2024-03-23T17:41:02.368 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-osd-2:18.2.2-954.g82316b04.el8.x86_64 109/162 2024-03-23T17:41:02.585 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 156/162 2024-03-23T17:41:02.585 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 157/162 2024-03-23T17:41:02.586 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mds-2:18.2.2-954.g82316b04.el8.x86_64 110/162 2024-03-23T17:41:02.844 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 157/162 2024-03-23T17:41:02.844 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 158/162 2024-03-23T17:41:03.061 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mds-2:18.2.2-954.g82316b04.el8.x86_64 110/162 2024-03-23T17:41:03.135 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 158/162 2024-03-23T17:41:03.135 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 159/162 2024-03-23T17:41:03.278 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mon-2:18.2.2-954.g82316b04.el8.x86_64 111/162 2024-03-23T17:41:03.295 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mon-2:18.2.2-954.g82316b04.el8.x86_64 111/162 2024-03-23T17:41:03.385 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 159/162 2024-03-23T17:41:03.385 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-03-23T17:41:03.660 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-03-23T17:41:03.793 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 161/162 2024-03-23T17:41:03.794 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-03-23T17:41:04.385 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-03-23T17:41:04.385 INFO:teuthology.orchestra.run.smithi086.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-03-23T17:41:04.386 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:41:05.041 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-2:18.2.2-954.g82316b04.el8.x86_64 1/162 2024-03-23T17:41:05.041 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-base-2:18.2.2-954.g82316b04.el8.x86_64 2/162 2024-03-23T17:41:05.042 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-common-2:18.2.2-954.g82316b04.el8.x86_64 3/162 2024-03-23T17:41:05.042 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-fuse-2:18.2.2-954.g82316b04.el8.x86_64 4/162 2024-03-23T17:41:05.042 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-954.g82316b 5/162 2024-03-23T17:41:05.042 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mds-2:18.2.2-954.g82316b04.el8.x86_64 6/162 2024-03-23T17:41:05.042 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-2:18.2.2-954.g82316b04.el8.x86_64 7/162 2024-03-23T17:41:05.042 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mon-2:18.2.2-954.g82316b04.el8.x86_64 8/162 2024-03-23T17:41:05.042 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-osd-2:18.2.2-954.g82316b04.el8.x86_64 9/162 2024-03-23T17:41:05.042 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-radosgw-2:18.2.2-954.g82316b04.el8.x86_64 10/162 2024-03-23T17:41:05.042 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-selinux-2:18.2.2-954.g82316b04.el8.x86_64 11/162 2024-03-23T17:41:05.042 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-test-2:18.2.2-954.g82316b04.el8.x86_64 12/162 2024-03-23T17:41:05.042 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : cephfs-mirror-2:18.2.2-954.g82316b04.el8.x86_64 13/162 2024-03-23T17:41:05.042 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libcephfs-devel-2:18.2.2-954.g82316b04.el8.x86_6 14/162 2024-03-23T17:41:05.043 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libcephfs2-2:18.2.2-954.g82316b04.el8.x86_64 15/162 2024-03-23T17:41:05.043 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libcephsqlite-2:18.2.2-954.g82316b04.el8.x86_64 16/162 2024-03-23T17:41:05.043 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librados-devel-2:18.2.2-954.g82316b04.el8.x86_64 17/162 2024-03-23T17:41:05.043 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libradosstriper1-2:18.2.2-954.g82316b04.el8.x86_ 18/162 2024-03-23T17:41:05.043 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librgw2-2:18.2.2-954.g82316b04.el8.x86_64 19/162 2024-03-23T17:41:05.043 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-ceph-argparse-2:18.2.2-954.g82316b04.el8 20/162 2024-03-23T17:41:05.043 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-ceph-common-2:18.2.2-954.g82316b04.el8.x 21/162 2024-03-23T17:41:05.043 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cephfs-2:18.2.2-954.g82316b04.el8.x86_64 22/162 2024-03-23T17:41:05.043 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-rados-2:18.2.2-954.g82316b04.el8.x86_64 23/162 2024-03-23T17:41:05.043 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-rbd-2:18.2.2-954.g82316b04.el8.x86_64 24/162 2024-03-23T17:41:05.043 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-rgw-2:18.2.2-954.g82316b04.el8.x86_64 25/162 2024-03-23T17:41:05.043 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : rbd-fuse-2:18.2.2-954.g82316b04.el8.x86_64 26/162 2024-03-23T17:41:05.043 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : rbd-mirror-2:18.2.2-954.g82316b04.el8.x86_64 27/162 2024-03-23T17:41:05.043 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : rbd-nbd-2:18.2.2-954.g82316b04.el8.x86_64 28/162 2024-03-23T17:41:05.044 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-954.g82316b04.e 29/162 2024-03-23T17:41:05.056 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-954.g82316b04.el8.noar 30/162 2024-03-23T17:41:05.056 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-954.g82316b04.el8.no 31/162 2024-03-23T17:41:05.056 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-954.g8231 32/162 2024-03-23T17:41:05.056 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-954.g82316b04.el8 33/162 2024-03-23T17:41:05.056 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-rook-2:18.2.2-954.g82316b04.el8.noarch 34/162 2024-03-23T17:41:05.056 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-954.g82316b04.el 35/162 2024-03-23T17:41:05.056 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-volume-2:18.2.2-954.g82316b04.el8.noarch 36/162 2024-03-23T17:41:05.056 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : cephadm-2:18.2.2-954.g82316b04.el8.noarch 37/162 2024-03-23T17:41:05.057 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : cephfs-top-2:18.2.2-954.g82316b04.el8.noarch 38/162 2024-03-23T17:41:05.057 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/162 2024-03-23T17:41:05.057 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/162 2024-03-23T17:41:05.057 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/162 2024-03-23T17:41:05.057 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/162 2024-03-23T17:41:05.057 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/162 2024-03-23T17:41:05.057 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/162 2024-03-23T17:41:05.057 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/162 2024-03-23T17:41:05.057 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/162 2024-03-23T17:41:05.057 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/162 2024-03-23T17:41:05.057 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/162 2024-03-23T17:41:05.057 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/162 2024-03-23T17:41:05.057 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/162 2024-03-23T17:41:05.057 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/162 2024-03-23T17:41:05.057 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/162 2024-03-23T17:41:05.058 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/162 2024-03-23T17:41:05.058 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/162 2024-03-23T17:41:05.058 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/162 2024-03-23T17:41:05.058 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/162 2024-03-23T17:41:05.058 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/162 2024-03-23T17:41:05.058 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/162 2024-03-23T17:41:05.058 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/162 2024-03-23T17:41:05.058 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/162 2024-03-23T17:41:05.058 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/162 2024-03-23T17:41:05.058 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/162 2024-03-23T17:41:05.058 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/162 2024-03-23T17:41:05.058 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/162 2024-03-23T17:41:05.058 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/162 2024-03-23T17:41:05.058 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/162 2024-03-23T17:41:05.059 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/162 2024-03-23T17:41:05.059 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/162 2024-03-23T17:41:05.059 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/162 2024-03-23T17:41:05.059 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/162 2024-03-23T17:41:05.059 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/162 2024-03-23T17:41:05.059 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-packaging-16.8-9.el8.noarch 72/162 2024-03-23T17:41:05.059 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 73/162 2024-03-23T17:41:05.059 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 74/162 2024-03-23T17:41:05.059 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 75/162 2024-03-23T17:41:05.059 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 76/162 2024-03-23T17:41:05.059 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 77/162 2024-03-23T17:41:05.059 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 78/162 2024-03-23T17:41:05.059 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : bison-3.0.4-10.el8.x86_64 79/162 2024-03-23T17:41:05.060 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : flex-2.6.1-9.el8.x86_64 80/162 2024-03-23T17:41:05.060 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 81/162 2024-03-23T17:41:05.060 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 82/162 2024-03-23T17:41:05.060 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 83/162 2024-03-23T17:41:05.060 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 84/162 2024-03-23T17:41:05.060 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 85/162 2024-03-23T17:41:05.060 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 86/162 2024-03-23T17:41:05.060 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 87/162 2024-03-23T17:41:05.060 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 88/162 2024-03-23T17:41:05.060 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 89/162 2024-03-23T17:41:05.060 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 90/162 2024-03-23T17:41:05.060 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 91/162 2024-03-23T17:41:05.061 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 92/162 2024-03-23T17:41:05.061 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 93/162 2024-03-23T17:41:05.061 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 94/162 2024-03-23T17:41:05.061 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : automake-1.16.1-8.el8.noarch 95/162 2024-03-23T17:41:05.061 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libpq-13.11-1.el8.x86_64 96/162 2024-03-23T17:41:05.061 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : jq-1.6-7.el8.x86_64 97/162 2024-03-23T17:41:05.061 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 98/162 2024-03-23T17:41:05.061 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 99/162 2024-03-23T17:41:05.061 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : oniguruma-6.8.2-2.1.el8_9.x86_64 100/162 2024-03-23T17:41:05.061 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 101/162 2024-03-23T17:41:05.061 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 102/162 2024-03-23T17:41:05.061 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : postgresql-server-10.23-4.module+el8.9.0+21317+d 103/162 2024-03-23T17:41:05.061 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 104/162 2024-03-23T17:41:05.061 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/162 2024-03-23T17:41:05.061 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 106/162 2024-03-23T17:41:05.061 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 107/162 2024-03-23T17:41:05.062 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 108/162 2024-03-23T17:41:05.062 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 109/162 2024-03-23T17:41:05.062 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 110/162 2024-03-23T17:41:05.062 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 111/162 2024-03-23T17:41:05.062 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 112/162 2024-03-23T17:41:05.062 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 113/162 2024-03-23T17:41:05.062 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 114/162 2024-03-23T17:41:05.062 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 115/162 2024-03-23T17:41:05.062 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 116/162 2024-03-23T17:41:05.062 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 117/162 2024-03-23T17:41:05.062 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 118/162 2024-03-23T17:41:05.062 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 119/162 2024-03-23T17:41:05.062 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 120/162 2024-03-23T17:41:05.062 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 121/162 2024-03-23T17:41:05.063 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libuuid-devel-2.32.1-43.el8.x86_64 122/162 2024-03-23T17:41:05.063 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 123/162 2024-03-23T17:41:05.063 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 124/162 2024-03-23T17:41:05.063 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-13.e 125/162 2024-03-23T17:41:05.063 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 126/162 2024-03-23T17:41:05.063 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 127/162 2024-03-23T17:41:05.063 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openssl-devel-1:1.1.1k-12.el8_9.x86_64 128/162 2024-03-23T17:41:05.063 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librabbitmq-0.9.0-5.el8_9.x86_64 129/162 2024-03-23T17:41:05.063 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cryptography-3.2.1-7.el8_9.x86_64 130/162 2024-03-23T17:41:05.063 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librados2-2:18.2.2-954.g82316b04.el8.x86_64 131/162 2024-03-23T17:41:05.063 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 132/162 2024-03-23T17:41:05.063 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librbd1-2:18.2.2-954.g82316b04.el8.x86_64 133/162 2024-03-23T17:41:05.063 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 134/162 2024-03-23T17:41:05.063 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 135/162 2024-03-23T17:41:05.063 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 136/162 2024-03-23T17:41:05.063 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 137/162 2024-03-23T17:41:05.064 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 138/162 2024-03-23T17:41:05.064 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libmount-2.32.1-43.el8.x86_64 139/162 2024-03-23T17:41:05.064 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 140/162 2024-03-23T17:41:05.064 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libsmartcols-2.32.1-43.el8.x86_64 141/162 2024-03-23T17:41:05.064 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 142/162 2024-03-23T17:41:05.064 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libuuid-2.32.1-43.el8.x86_64 143/162 2024-03-23T17:41:05.064 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 144/162 2024-03-23T17:41:05.064 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : util-linux-2.32.1-43.el8.x86_64 145/162 2024-03-23T17:41:05.064 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 146/162 2024-03-23T17:41:05.064 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : util-linux-user-2.32.1-43.el8.x86_64 147/162 2024-03-23T17:41:05.064 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 148/162 2024-03-23T17:41:05.064 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 149/162 2024-03-23T17:41:05.064 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 150/162 2024-03-23T17:41:05.064 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libblkid-2.32.1-43.el8.x86_64 151/162 2024-03-23T17:41:05.064 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 152/162 2024-03-23T17:41:05.064 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libfdisk-2.32.1-43.el8.x86_64 153/162 2024-03-23T17:41:05.065 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 154/162 2024-03-23T17:41:05.065 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openssl-1:1.1.1k-12.el8_9.x86_64 155/162 2024-03-23T17:41:05.065 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 156/162 2024-03-23T17:41:05.065 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openssl-libs-1:1.1.1k-12.el8_9.x86_64 157/162 2024-03-23T17:41:05.065 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 158/162 2024-03-23T17:41:05.065 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : xfsdump-3.1.8-7.el8_9.x86_64 159/162 2024-03-23T17:41:05.065 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 160/162 2024-03-23T17:41:05.065 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 161/162 2024-03-23T17:41:05.130 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: cephadm-2:18.2.2-954.g82316b04.el8.noarch 112/162 2024-03-23T17:41:05.333 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : cephadm-2:18.2.2-954.g82316b04.el8.noarch 112/162 2024-03-23T17:41:05.488 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-954.g82316b04.el 113/162 2024-03-23T17:41:05.713 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-954.g82316b04.e 114/162 2024-03-23T17:41:05.968 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-954.g82316b04.el8.noar 115/162 2024-03-23T17:41:06.195 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-954.g82316b04.el8.noar 115/162 2024-03-23T17:41:06.505 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-954.g82316b04.el8.no 116/162 2024-03-23T17:41:07.472 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-954.g82316b04.el8.no 116/162 2024-03-23T17:41:07.668 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-954.g8231 117/162 2024-03-23T17:41:07.763 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-954.g8231 117/162 2024-03-23T17:41:08.151 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-954.g82316b04.el8 118/162 2024-03-23T17:41:08.372 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mgr-2:18.2.2-954.g82316b04.el8.x86_64 119/162 2024-03-23T17:41:08.410 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mgr-2:18.2.2-954.g82316b04.el8.x86_64 119/162 2024-03-23T17:41:08.572 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mgr-rook-2:18.2.2-954.g82316b04.el8.noarch 120/162 2024-03-23T17:41:08.583 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-954.g82316b04.el8.noarch 120/162 2024-03-23T17:41:09.885 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-2:18.2.2-954.g82316b04.el8.x86_64 121/162 2024-03-23T17:41:10.105 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-radosgw-2:18.2.2-954.g82316b04.el8.x86_64 122/162 2024-03-23T17:41:10.140 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-954.g82316b04.el8.x86_64 122/162 2024-03-23T17:41:10.360 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-954.g82316b 123/162 2024-03-23T17:41:10.403 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-954.g82316b 123/162 2024-03-23T17:41:10.585 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : cephfs-mirror-2:18.2.2-954.g82316b04.el8.x86_64 124/162 2024-03-23T17:41:10.932 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: cephfs-mirror-2:18.2.2-954.g82316b04.el8.x86_64 124/162 2024-03-23T17:41:11.127 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : rbd-mirror-2:18.2.2-954.g82316b04.el8.x86_64 125/162 2024-03-23T17:41:12.575 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 162/162 2024-03-23T17:41:12.575 INFO:teuthology.orchestra.run.smithi092.stdout:Installed products updated. 2024-03-23T17:41:13.178 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-23T17:41:13.179 INFO:teuthology.orchestra.run.smithi092.stdout:Upgraded: 2024-03-23T17:41:13.179 INFO:teuthology.orchestra.run.smithi092.stdout: elfutils-libelf-0.189-3.el8.x86_64 2024-03-23T17:41:13.179 INFO:teuthology.orchestra.run.smithi092.stdout: elfutils-libs-0.189-3.el8.x86_64 2024-03-23T17:41:13.179 INFO:teuthology.orchestra.run.smithi092.stdout: iproute-6.2.0-5.el8_9.x86_64 2024-03-23T17:41:13.179 INFO:teuthology.orchestra.run.smithi092.stdout: libblkid-2.32.1-43.el8.x86_64 2024-03-23T17:41:13.179 INFO:teuthology.orchestra.run.smithi092.stdout: libfdisk-2.32.1-43.el8.x86_64 2024-03-23T17:41:13.179 INFO:teuthology.orchestra.run.smithi092.stdout: libmount-2.32.1-43.el8.x86_64 2024-03-23T17:41:13.179 INFO:teuthology.orchestra.run.smithi092.stdout: librados2-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:13.179 INFO:teuthology.orchestra.run.smithi092.stdout: librbd1-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:13.179 INFO:teuthology.orchestra.run.smithi092.stdout: libsmartcols-2.32.1-43.el8.x86_64 2024-03-23T17:41:13.179 INFO:teuthology.orchestra.run.smithi092.stdout: libuuid-2.32.1-43.el8.x86_64 2024-03-23T17:41:13.179 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-1:1.1.1k-12.el8_9.x86_64 2024-03-23T17:41:13.180 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-libs-1:1.1.1k-12.el8_9.x86_64 2024-03-23T17:41:13.180 INFO:teuthology.orchestra.run.smithi092.stdout: util-linux-2.32.1-43.el8.x86_64 2024-03-23T17:41:13.180 INFO:teuthology.orchestra.run.smithi092.stdout: util-linux-user-2.32.1-43.el8.x86_64 2024-03-23T17:41:13.180 INFO:teuthology.orchestra.run.smithi092.stdout: xfsdump-3.1.8-7.el8_9.x86_64 2024-03-23T17:41:13.180 INFO:teuthology.orchestra.run.smithi092.stdout: xfsprogs-5.0.0-12.el8.x86_64 2024-03-23T17:41:13.180 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-03-23T17:41:13.180 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager-initscripts-updown-1:1.40.16-13.el8_9.noarch 2024-03-23T17:41:13.180 INFO:teuthology.orchestra.run.smithi092.stdout: automake-1.16.1-8.el8.noarch 2024-03-23T17:41:13.180 INFO:teuthology.orchestra.run.smithi092.stdout: bison-3.0.4-10.el8.x86_64 2024-03-23T17:41:13.180 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:13.180 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-base-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:13.180 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-common-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:13.180 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-fuse-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:13.180 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-grafana-dashboards-2:18.2.2-954.g82316b04.el8.noarch 2024-03-23T17:41:13.180 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-immutable-object-cache-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:13.181 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mds-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:13.181 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:13.181 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-cephadm-2:18.2.2-954.g82316b04.el8.noarch 2024-03-23T17:41:13.181 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-dashboard-2:18.2.2-954.g82316b04.el8.noarch 2024-03-23T17:41:13.181 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-diskprediction-local-2:18.2.2-954.g82316b04.el8.noarch 2024-03-23T17:41:13.181 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-modules-core-2:18.2.2-954.g82316b04.el8.noarch 2024-03-23T17:41:13.181 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-rook-2:18.2.2-954.g82316b04.el8.noarch 2024-03-23T17:41:13.181 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mon-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:13.181 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-osd-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:13.181 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-prometheus-alerts-2:18.2.2-954.g82316b04.el8.noarch 2024-03-23T17:41:13.181 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-radosgw-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:13.181 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-selinux-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:13.181 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-test-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:13.181 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-volume-2:18.2.2-954.g82316b04.el8.noarch 2024-03-23T17:41:13.182 INFO:teuthology.orchestra.run.smithi092.stdout: cephadm-2:18.2.2-954.g82316b04.el8.noarch 2024-03-23T17:41:13.182 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-mirror-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:13.182 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-top-2:18.2.2-954.g82316b04.el8.noarch 2024-03-23T17:41:13.182 INFO:teuthology.orchestra.run.smithi092.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2024-03-23T17:41:13.182 INFO:teuthology.orchestra.run.smithi092.stdout: flex-2.6.1-9.el8.x86_64 2024-03-23T17:41:13.182 INFO:teuthology.orchestra.run.smithi092.stdout: fuse-2.9.7-17.el8.x86_64 2024-03-23T17:41:13.182 INFO:teuthology.orchestra.run.smithi092.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-03-23T17:41:13.182 INFO:teuthology.orchestra.run.smithi092.stdout: jq-1.6-7.el8.x86_64 2024-03-23T17:41:13.182 INFO:teuthology.orchestra.run.smithi092.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2024-03-23T17:41:13.182 INFO:teuthology.orchestra.run.smithi092.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2024-03-23T17:41:13.182 INFO:teuthology.orchestra.run.smithi092.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-03-23T17:41:13.182 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs-devel-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:13.182 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs2-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:13.182 INFO:teuthology.orchestra.run.smithi092.stdout: libcephsqlite-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:13.183 INFO:teuthology.orchestra.run.smithi092.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2024-03-23T17:41:13.183 INFO:teuthology.orchestra.run.smithi092.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-03-23T17:41:13.183 INFO:teuthology.orchestra.run.smithi092.stdout: liboath-2.6.2-3.el8.x86_64 2024-03-23T17:41:13.183 INFO:teuthology.orchestra.run.smithi092.stdout: libpq-13.11-1.el8.x86_64 2024-03-23T17:41:13.183 INFO:teuthology.orchestra.run.smithi092.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-03-23T17:41:13.183 INFO:teuthology.orchestra.run.smithi092.stdout: librabbitmq-0.9.0-5.el8_9.x86_64 2024-03-23T17:41:13.183 INFO:teuthology.orchestra.run.smithi092.stdout: librados-devel-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:13.183 INFO:teuthology.orchestra.run.smithi092.stdout: libradosstriper1-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:13.183 INFO:teuthology.orchestra.run.smithi092.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-03-23T17:41:13.183 INFO:teuthology.orchestra.run.smithi092.stdout: librgw2-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:13.183 INFO:teuthology.orchestra.run.smithi092.stdout: libselinux-devel-2.9-5.el8.x86_64 2024-03-23T17:41:13.183 INFO:teuthology.orchestra.run.smithi092.stdout: libsepol-devel-2.9-3.el8.x86_64 2024-03-23T17:41:13.183 INFO:teuthology.orchestra.run.smithi092.stdout: libtool-2.4.6-25.el8.x86_64 2024-03-23T17:41:13.183 INFO:teuthology.orchestra.run.smithi092.stdout: libunwind-1.3.1-3.el8.x86_64 2024-03-23T17:41:13.184 INFO:teuthology.orchestra.run.smithi092.stdout: libuuid-devel-2.32.1-43.el8.x86_64 2024-03-23T17:41:13.184 INFO:teuthology.orchestra.run.smithi092.stdout: libverto-devel-0.3.0-5.el8.x86_64 2024-03-23T17:41:13.184 INFO:teuthology.orchestra.run.smithi092.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2024-03-23T17:41:13.184 INFO:teuthology.orchestra.run.smithi092.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-03-23T17:41:13.184 INFO:teuthology.orchestra.run.smithi092.stdout: oniguruma-6.8.2-2.1.el8_9.x86_64 2024-03-23T17:41:13.184 INFO:teuthology.orchestra.run.smithi092.stdout: openblas-0.3.15-6.el8.x86_64 2024-03-23T17:41:13.184 INFO:teuthology.orchestra.run.smithi092.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-03-23T17:41:13.184 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-devel-1:1.1.1k-12.el8_9.x86_64 2024-03-23T17:41:13.184 INFO:teuthology.orchestra.run.smithi092.stdout: pcre2-devel-10.32-2.el8.x86_64 2024-03-23T17:41:13.185 INFO:teuthology.orchestra.run.smithi092.stdout: pcre2-utf16-10.32-2.el8.x86_64 2024-03-23T17:41:13.185 INFO:teuthology.orchestra.run.smithi092.stdout: pcre2-utf32-10.32-2.el8.x86_64 2024-03-23T17:41:13.185 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-03-23T17:41:13.185 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-contrib-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-03-23T17:41:13.185 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-server-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-03-23T17:41:13.185 INFO:teuthology.orchestra.run.smithi092.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-03-23T17:41:13.185 INFO:teuthology.orchestra.run.smithi092.stdout: python3-babel-2.5.1-7.el8.noarch 2024-03-23T17:41:13.185 INFO:teuthology.orchestra.run.smithi092.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-03-23T17:41:13.185 INFO:teuthology.orchestra.run.smithi092.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-03-23T17:41:13.185 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-03-23T17:41:13.186 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-argparse-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:13.186 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-common-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:13.186 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cephfs-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:13.186 INFO:teuthology.orchestra.run.smithi092.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-03-23T17:41:13.187 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-03-23T17:41:13.187 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-03-23T17:41:13.187 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-03-23T17:41:13.187 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cryptography-3.2.1-7.el8_9.x86_64 2024-03-23T17:41:13.187 INFO:teuthology.orchestra.run.smithi092.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-03-23T17:41:13.187 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-6.2-6.el8.noarch 2024-03-23T17:41:13.187 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-03-23T17:41:13.187 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-03-23T17:41:13.187 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-03-23T17:41:13.187 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-03-23T17:41:13.187 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-03-23T17:41:13.187 INFO:teuthology.orchestra.run.smithi092.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-03-23T17:41:13.187 INFO:teuthology.orchestra.run.smithi092.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-03-23T17:41:13.187 INFO:teuthology.orchestra.run.smithi092.stdout: python3-mako-1.0.6-14.el8.noarch 2024-03-23T17:41:13.188 INFO:teuthology.orchestra.run.smithi092.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-03-23T17:41:13.188 INFO:teuthology.orchestra.run.smithi092.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-03-23T17:41:13.188 INFO:teuthology.orchestra.run.smithi092.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-03-23T17:41:13.188 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-03-23T17:41:13.188 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-03-23T17:41:13.188 INFO:teuthology.orchestra.run.smithi092.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-03-23T17:41:13.188 INFO:teuthology.orchestra.run.smithi092.stdout: python3-packaging-16.8-9.el8.noarch 2024-03-23T17:41:13.188 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-03-23T17:41:13.188 INFO:teuthology.orchestra.run.smithi092.stdout: python3-portend-2.6-1.el8.noarch 2024-03-23T17:41:13.188 INFO:teuthology.orchestra.run.smithi092.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-03-23T17:41:13.188 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-03-23T17:41:13.188 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-03-23T17:41:13.188 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-03-23T17:41:13.188 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pycparser-2.14-14.el8.noarch 2024-03-23T17:41:13.189 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pytz-2017.2-11.el8.noarch 2024-03-23T17:41:13.189 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rados-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:13.189 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rbd-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:13.189 INFO:teuthology.orchestra.run.smithi092.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-03-23T17:41:13.189 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-03-23T17:41:13.189 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rgw-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:13.189 INFO:teuthology.orchestra.run.smithi092.stdout: python3-routes-2.4.1-12.el8.noarch 2024-03-23T17:41:13.189 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rsa-4.9-2.el8.noarch 2024-03-23T17:41:13.189 INFO:teuthology.orchestra.run.smithi092.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-03-23T17:41:13.189 INFO:teuthology.orchestra.run.smithi092.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-03-23T17:41:13.189 INFO:teuthology.orchestra.run.smithi092.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-03-23T17:41:13.189 INFO:teuthology.orchestra.run.smithi092.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-03-23T17:41:13.189 INFO:teuthology.orchestra.run.smithi092.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-03-23T17:41:13.189 INFO:teuthology.orchestra.run.smithi092.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-03-23T17:41:13.189 INFO:teuthology.orchestra.run.smithi092.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-03-23T17:41:13.190 INFO:teuthology.orchestra.run.smithi092.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-03-23T17:41:13.190 INFO:teuthology.orchestra.run.smithi092.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-03-23T17:41:13.190 INFO:teuthology.orchestra.run.smithi092.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-03-23T17:41:13.190 INFO:teuthology.orchestra.run.smithi092.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-03-23T17:41:13.190 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-fuse-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:13.190 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-mirror-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:13.190 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-nbd-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:13.190 INFO:teuthology.orchestra.run.smithi092.stdout: socat-1.7.4.1-1.el8.x86_64 2024-03-23T17:41:13.190 INFO:teuthology.orchestra.run.smithi092.stdout: thrift-0.13.0-2.el8.x86_64 2024-03-23T17:41:13.190 INFO:teuthology.orchestra.run.smithi092.stdout: uuid-1.6.2-43.el8.x86_64 2024-03-23T17:41:13.190 INFO:teuthology.orchestra.run.smithi092.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2024-03-23T17:41:13.190 INFO:teuthology.orchestra.run.smithi092.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-03-23T17:41:13.191 INFO:teuthology.orchestra.run.smithi092.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2024-03-23T17:41:13.191 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-23T17:41:13.191 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-03-23T17:41:13.423 DEBUG:teuthology.parallel:result is None 2024-03-23T17:41:15.601 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: rbd-mirror-2:18.2.2-954.g82316b04.el8.x86_64 125/162 2024-03-23T17:41:16.597 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-test-2:18.2.2-954.g82316b04.el8.x86_64 126/162 2024-03-23T17:41:16.930 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : openssl-devel-1:1.1.1k-12.el8_9.x86_64 127/162 2024-03-23T17:41:17.208 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 128/162 2024-03-23T17:41:17.610 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-fuse-2:18.2.2-954.g82316b04.el8.x86_64 129/162 2024-03-23T17:41:17.952 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 130/162 2024-03-23T17:41:18.192 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libtool-2.4.6-25.el8.x86_64 131/162 2024-03-23T17:41:18.205 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 131/162 2024-03-23T17:41:18.303 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 162/162 2024-03-23T17:41:18.304 INFO:teuthology.orchestra.run.smithi086.stdout:Installed products updated. 2024-03-23T17:41:18.461 INFO:teuthology.orchestra.run.smithi152.stdout: Upgrading : util-linux-user-2.32.1-43.el8.x86_64 132/162 2024-03-23T17:41:18.664 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 133/162 2024-03-23T17:41:18.901 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libcephfs-devel-2:18.2.2-954.g82316b04.el8.x86_6 134/162 2024-03-23T17:41:20.694 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:41:20.695 INFO:teuthology.orchestra.run.smithi086.stdout:Upgraded: 2024-03-23T17:41:20.695 INFO:teuthology.orchestra.run.smithi086.stdout: elfutils-libelf-0.189-3.el8.x86_64 2024-03-23T17:41:20.695 INFO:teuthology.orchestra.run.smithi086.stdout: elfutils-libs-0.189-3.el8.x86_64 2024-03-23T17:41:20.695 INFO:teuthology.orchestra.run.smithi086.stdout: iproute-6.2.0-5.el8_9.x86_64 2024-03-23T17:41:20.695 INFO:teuthology.orchestra.run.smithi086.stdout: libblkid-2.32.1-43.el8.x86_64 2024-03-23T17:41:20.695 INFO:teuthology.orchestra.run.smithi086.stdout: libfdisk-2.32.1-43.el8.x86_64 2024-03-23T17:41:20.695 INFO:teuthology.orchestra.run.smithi086.stdout: libmount-2.32.1-43.el8.x86_64 2024-03-23T17:41:20.695 INFO:teuthology.orchestra.run.smithi086.stdout: librados2-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:20.695 INFO:teuthology.orchestra.run.smithi086.stdout: librbd1-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:20.695 INFO:teuthology.orchestra.run.smithi086.stdout: libsmartcols-2.32.1-43.el8.x86_64 2024-03-23T17:41:20.696 INFO:teuthology.orchestra.run.smithi086.stdout: libuuid-2.32.1-43.el8.x86_64 2024-03-23T17:41:20.696 INFO:teuthology.orchestra.run.smithi086.stdout: openssl-1:1.1.1k-12.el8_9.x86_64 2024-03-23T17:41:20.696 INFO:teuthology.orchestra.run.smithi086.stdout: openssl-libs-1:1.1.1k-12.el8_9.x86_64 2024-03-23T17:41:20.696 INFO:teuthology.orchestra.run.smithi086.stdout: util-linux-2.32.1-43.el8.x86_64 2024-03-23T17:41:20.696 INFO:teuthology.orchestra.run.smithi086.stdout: util-linux-user-2.32.1-43.el8.x86_64 2024-03-23T17:41:20.696 INFO:teuthology.orchestra.run.smithi086.stdout: xfsdump-3.1.8-7.el8_9.x86_64 2024-03-23T17:41:20.696 INFO:teuthology.orchestra.run.smithi086.stdout: xfsprogs-5.0.0-12.el8.x86_64 2024-03-23T17:41:20.696 INFO:teuthology.orchestra.run.smithi086.stdout:Installed: 2024-03-23T17:41:20.696 INFO:teuthology.orchestra.run.smithi086.stdout: NetworkManager-initscripts-updown-1:1.40.16-13.el8_9.noarch 2024-03-23T17:41:20.697 INFO:teuthology.orchestra.run.smithi086.stdout: automake-1.16.1-8.el8.noarch 2024-03-23T17:41:20.697 INFO:teuthology.orchestra.run.smithi086.stdout: bison-3.0.4-10.el8.x86_64 2024-03-23T17:41:20.697 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:20.697 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-base-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:20.697 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-common-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:20.697 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-fuse-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:20.697 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-grafana-dashboards-2:18.2.2-954.g82316b04.el8.noarch 2024-03-23T17:41:20.697 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-immutable-object-cache-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:20.697 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mds-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:20.697 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:20.697 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-cephadm-2:18.2.2-954.g82316b04.el8.noarch 2024-03-23T17:41:20.698 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-dashboard-2:18.2.2-954.g82316b04.el8.noarch 2024-03-23T17:41:20.698 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-diskprediction-local-2:18.2.2-954.g82316b04.el8.noarch 2024-03-23T17:41:20.698 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-modules-core-2:18.2.2-954.g82316b04.el8.noarch 2024-03-23T17:41:20.698 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-rook-2:18.2.2-954.g82316b04.el8.noarch 2024-03-23T17:41:20.698 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mon-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:20.698 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-osd-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:20.698 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-prometheus-alerts-2:18.2.2-954.g82316b04.el8.noarch 2024-03-23T17:41:20.698 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-radosgw-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:20.698 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-selinux-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:20.698 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-test-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:20.698 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-volume-2:18.2.2-954.g82316b04.el8.noarch 2024-03-23T17:41:20.698 INFO:teuthology.orchestra.run.smithi086.stdout: cephadm-2:18.2.2-954.g82316b04.el8.noarch 2024-03-23T17:41:20.699 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-mirror-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:20.699 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-top-2:18.2.2-954.g82316b04.el8.noarch 2024-03-23T17:41:20.699 INFO:teuthology.orchestra.run.smithi086.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2024-03-23T17:41:20.699 INFO:teuthology.orchestra.run.smithi086.stdout: flex-2.6.1-9.el8.x86_64 2024-03-23T17:41:20.699 INFO:teuthology.orchestra.run.smithi086.stdout: fuse-2.9.7-17.el8.x86_64 2024-03-23T17:41:20.699 INFO:teuthology.orchestra.run.smithi086.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-03-23T17:41:20.699 INFO:teuthology.orchestra.run.smithi086.stdout: jq-1.6-7.el8.x86_64 2024-03-23T17:41:20.699 INFO:teuthology.orchestra.run.smithi086.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2024-03-23T17:41:20.699 INFO:teuthology.orchestra.run.smithi086.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2024-03-23T17:41:20.699 INFO:teuthology.orchestra.run.smithi086.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-03-23T17:41:20.699 INFO:teuthology.orchestra.run.smithi086.stdout: libcephfs-devel-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:20.699 INFO:teuthology.orchestra.run.smithi086.stdout: libcephfs2-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:20.700 INFO:teuthology.orchestra.run.smithi086.stdout: libcephsqlite-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:20.700 INFO:teuthology.orchestra.run.smithi086.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2024-03-23T17:41:20.700 INFO:teuthology.orchestra.run.smithi086.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-03-23T17:41:20.700 INFO:teuthology.orchestra.run.smithi086.stdout: liboath-2.6.2-3.el8.x86_64 2024-03-23T17:41:20.700 INFO:teuthology.orchestra.run.smithi086.stdout: libpq-13.11-1.el8.x86_64 2024-03-23T17:41:20.700 INFO:teuthology.orchestra.run.smithi086.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-03-23T17:41:20.700 INFO:teuthology.orchestra.run.smithi086.stdout: librabbitmq-0.9.0-5.el8_9.x86_64 2024-03-23T17:41:20.700 INFO:teuthology.orchestra.run.smithi086.stdout: librados-devel-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:20.700 INFO:teuthology.orchestra.run.smithi086.stdout: libradosstriper1-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:20.700 INFO:teuthology.orchestra.run.smithi086.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-03-23T17:41:20.700 INFO:teuthology.orchestra.run.smithi086.stdout: librgw2-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:20.700 INFO:teuthology.orchestra.run.smithi086.stdout: libselinux-devel-2.9-5.el8.x86_64 2024-03-23T17:41:20.700 INFO:teuthology.orchestra.run.smithi086.stdout: libsepol-devel-2.9-3.el8.x86_64 2024-03-23T17:41:20.700 INFO:teuthology.orchestra.run.smithi086.stdout: libtool-2.4.6-25.el8.x86_64 2024-03-23T17:41:20.701 INFO:teuthology.orchestra.run.smithi086.stdout: libunwind-1.3.1-3.el8.x86_64 2024-03-23T17:41:20.701 INFO:teuthology.orchestra.run.smithi086.stdout: libuuid-devel-2.32.1-43.el8.x86_64 2024-03-23T17:41:20.701 INFO:teuthology.orchestra.run.smithi086.stdout: libverto-devel-0.3.0-5.el8.x86_64 2024-03-23T17:41:20.701 INFO:teuthology.orchestra.run.smithi086.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2024-03-23T17:41:20.701 INFO:teuthology.orchestra.run.smithi086.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-03-23T17:41:20.701 INFO:teuthology.orchestra.run.smithi086.stdout: oniguruma-6.8.2-2.1.el8_9.x86_64 2024-03-23T17:41:20.701 INFO:teuthology.orchestra.run.smithi086.stdout: openblas-0.3.15-6.el8.x86_64 2024-03-23T17:41:20.701 INFO:teuthology.orchestra.run.smithi086.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-03-23T17:41:20.701 INFO:teuthology.orchestra.run.smithi086.stdout: openssl-devel-1:1.1.1k-12.el8_9.x86_64 2024-03-23T17:41:20.701 INFO:teuthology.orchestra.run.smithi086.stdout: pcre2-devel-10.32-2.el8.x86_64 2024-03-23T17:41:20.701 INFO:teuthology.orchestra.run.smithi086.stdout: pcre2-utf16-10.32-2.el8.x86_64 2024-03-23T17:41:20.702 INFO:teuthology.orchestra.run.smithi086.stdout: pcre2-utf32-10.32-2.el8.x86_64 2024-03-23T17:41:20.702 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-03-23T17:41:20.702 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-contrib-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-03-23T17:41:20.702 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-server-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-03-23T17:41:20.702 INFO:teuthology.orchestra.run.smithi086.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-03-23T17:41:20.702 INFO:teuthology.orchestra.run.smithi086.stdout: python3-babel-2.5.1-7.el8.noarch 2024-03-23T17:41:20.702 INFO:teuthology.orchestra.run.smithi086.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-03-23T17:41:20.702 INFO:teuthology.orchestra.run.smithi086.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-03-23T17:41:20.702 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-03-23T17:41:20.702 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ceph-argparse-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:20.702 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ceph-common-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:20.703 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cephfs-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:20.703 INFO:teuthology.orchestra.run.smithi086.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-03-23T17:41:20.703 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-03-23T17:41:20.703 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-03-23T17:41:20.703 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-03-23T17:41:20.703 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cryptography-3.2.1-7.el8_9.x86_64 2024-03-23T17:41:20.703 INFO:teuthology.orchestra.run.smithi086.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-03-23T17:41:20.703 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-6.2-6.el8.noarch 2024-03-23T17:41:20.703 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-03-23T17:41:20.703 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-03-23T17:41:20.704 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-03-23T17:41:20.704 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-03-23T17:41:20.704 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-03-23T17:41:20.704 INFO:teuthology.orchestra.run.smithi086.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-03-23T17:41:20.704 INFO:teuthology.orchestra.run.smithi086.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-03-23T17:41:20.706 INFO:teuthology.orchestra.run.smithi086.stdout: python3-mako-1.0.6-14.el8.noarch 2024-03-23T17:41:20.706 INFO:teuthology.orchestra.run.smithi086.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-03-23T17:41:20.707 INFO:teuthology.orchestra.run.smithi086.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-03-23T17:41:20.707 INFO:teuthology.orchestra.run.smithi086.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-03-23T17:41:20.707 INFO:teuthology.orchestra.run.smithi086.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-03-23T17:41:20.707 INFO:teuthology.orchestra.run.smithi086.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-03-23T17:41:20.707 INFO:teuthology.orchestra.run.smithi086.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-03-23T17:41:20.707 INFO:teuthology.orchestra.run.smithi086.stdout: python3-packaging-16.8-9.el8.noarch 2024-03-23T17:41:20.707 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-03-23T17:41:20.707 INFO:teuthology.orchestra.run.smithi086.stdout: python3-portend-2.6-1.el8.noarch 2024-03-23T17:41:20.707 INFO:teuthology.orchestra.run.smithi086.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-03-23T17:41:20.707 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-03-23T17:41:20.708 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-03-23T17:41:20.708 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-03-23T17:41:20.708 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pycparser-2.14-14.el8.noarch 2024-03-23T17:41:20.708 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pytz-2017.2-11.el8.noarch 2024-03-23T17:41:20.708 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rados-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:20.708 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rbd-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:20.708 INFO:teuthology.orchestra.run.smithi086.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-03-23T17:41:20.708 INFO:teuthology.orchestra.run.smithi086.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-03-23T17:41:20.708 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rgw-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:20.708 INFO:teuthology.orchestra.run.smithi086.stdout: python3-routes-2.4.1-12.el8.noarch 2024-03-23T17:41:20.708 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rsa-4.9-2.el8.noarch 2024-03-23T17:41:20.708 INFO:teuthology.orchestra.run.smithi086.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-03-23T17:41:20.708 INFO:teuthology.orchestra.run.smithi086.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-03-23T17:41:20.708 INFO:teuthology.orchestra.run.smithi086.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-03-23T17:41:20.709 INFO:teuthology.orchestra.run.smithi086.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-03-23T17:41:20.709 INFO:teuthology.orchestra.run.smithi086.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-03-23T17:41:20.709 INFO:teuthology.orchestra.run.smithi086.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-03-23T17:41:20.709 INFO:teuthology.orchestra.run.smithi086.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-03-23T17:41:20.709 INFO:teuthology.orchestra.run.smithi086.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-03-23T17:41:20.709 INFO:teuthology.orchestra.run.smithi086.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-03-23T17:41:20.709 INFO:teuthology.orchestra.run.smithi086.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-03-23T17:41:20.709 INFO:teuthology.orchestra.run.smithi086.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-03-23T17:41:20.709 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-fuse-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:20.709 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-mirror-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:20.709 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-nbd-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:20.710 INFO:teuthology.orchestra.run.smithi086.stdout: socat-1.7.4.1-1.el8.x86_64 2024-03-23T17:41:20.710 INFO:teuthology.orchestra.run.smithi086.stdout: thrift-0.13.0-2.el8.x86_64 2024-03-23T17:41:20.710 INFO:teuthology.orchestra.run.smithi086.stdout: uuid-1.6.2-43.el8.x86_64 2024-03-23T17:41:20.710 INFO:teuthology.orchestra.run.smithi086.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2024-03-23T17:41:20.710 INFO:teuthology.orchestra.run.smithi086.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-03-23T17:41:20.710 INFO:teuthology.orchestra.run.smithi086.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2024-03-23T17:41:20.711 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:41:20.711 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-03-23T17:41:21.042 DEBUG:teuthology.parallel:result is None 2024-03-23T17:41:21.218 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-03-23T17:41:21.542 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-03-23T17:41:21.580 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-03-23T17:41:22.041 INFO:teuthology.orchestra.run.smithi152.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 136/162 2024-03-23T17:41:22.457 INFO:teuthology.orchestra.run.smithi152.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 137/162 2024-03-23T17:41:22.704 INFO:teuthology.orchestra.run.smithi152.stdout: Upgrading : xfsdump-3.1.8-7.el8_9.x86_64 138/162 2024-03-23T17:41:22.916 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : cephfs-top-2:18.2.2-954.g82316b04.el8.noarch 139/162 2024-03-23T17:41:23.099 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : rbd-fuse-2:18.2.2-954.g82316b04.el8.x86_64 140/162 2024-03-23T17:41:23.447 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : rbd-nbd-2:18.2.2-954.g82316b04.el8.x86_64 141/162 2024-03-23T17:41:23.804 INFO:teuthology.orchestra.run.smithi152.stdout: Upgrading : openssl-1:1.1.1k-12.el8_9.x86_64 142/162 2024-03-23T17:41:24.007 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : flex-2.6.1-9.el8.x86_64 143/162 2024-03-23T17:41:24.089 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 143/162 2024-03-23T17:41:24.316 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : bison-3.0.4-10.el8.x86_64 144/162 2024-03-23T17:41:24.330 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 144/162 2024-03-23T17:41:24.532 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-13.e 145/162 2024-03-23T17:41:24.542 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-13.e 145/162 2024-03-23T17:41:24.685 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 146/162 2024-03-23T17:41:24.882 INFO:teuthology.orchestra.run.smithi152.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 147/162 2024-03-23T17:41:25.068 INFO:teuthology.orchestra.run.smithi152.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 148/162 2024-03-23T17:41:25.278 INFO:teuthology.orchestra.run.smithi152.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 149/162 2024-03-23T17:41:25.278 INFO:teuthology.orchestra.run.smithi152.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-03-23T17:41:25.585 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-03-23T17:41:25.585 INFO:teuthology.orchestra.run.smithi152.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-03-23T17:41:25.853 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-03-23T17:41:25.854 INFO:teuthology.orchestra.run.smithi152.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 152/162 2024-03-23T17:41:26.072 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 152/162 2024-03-23T17:41:26.292 INFO:teuthology.orchestra.run.smithi152.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 153/162 2024-03-23T17:41:26.501 INFO:teuthology.orchestra.run.smithi152.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 154/162 2024-03-23T17:41:26.753 INFO:teuthology.orchestra.run.smithi152.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 155/162 2024-03-23T17:41:26.754 INFO:teuthology.orchestra.run.smithi152.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 156/162 2024-03-23T17:41:27.026 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 156/162 2024-03-23T17:41:27.026 INFO:teuthology.orchestra.run.smithi152.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 157/162 2024-03-23T17:41:27.301 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 157/162 2024-03-23T17:41:27.301 INFO:teuthology.orchestra.run.smithi152.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 158/162 2024-03-23T17:41:27.551 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 158/162 2024-03-23T17:41:27.551 INFO:teuthology.orchestra.run.smithi152.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 159/162 2024-03-23T17:41:27.784 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 159/162 2024-03-23T17:41:27.784 INFO:teuthology.orchestra.run.smithi152.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-03-23T17:41:28.059 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-03-23T17:41:28.251 INFO:teuthology.orchestra.run.smithi152.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 161/162 2024-03-23T17:41:28.251 INFO:teuthology.orchestra.run.smithi152.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-03-23T17:41:29.170 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-03-23T17:41:29.170 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-2:18.2.2-954.g82316b04.el8.x86_64 1/162 2024-03-23T17:41:29.170 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-base-2:18.2.2-954.g82316b04.el8.x86_64 2/162 2024-03-23T17:41:29.171 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-common-2:18.2.2-954.g82316b04.el8.x86_64 3/162 2024-03-23T17:41:29.171 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-fuse-2:18.2.2-954.g82316b04.el8.x86_64 4/162 2024-03-23T17:41:29.171 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-954.g82316b 5/162 2024-03-23T17:41:29.171 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mds-2:18.2.2-954.g82316b04.el8.x86_64 6/162 2024-03-23T17:41:29.171 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mgr-2:18.2.2-954.g82316b04.el8.x86_64 7/162 2024-03-23T17:41:29.171 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mon-2:18.2.2-954.g82316b04.el8.x86_64 8/162 2024-03-23T17:41:29.171 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-osd-2:18.2.2-954.g82316b04.el8.x86_64 9/162 2024-03-23T17:41:29.171 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-radosgw-2:18.2.2-954.g82316b04.el8.x86_64 10/162 2024-03-23T17:41:29.171 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-selinux-2:18.2.2-954.g82316b04.el8.x86_64 11/162 2024-03-23T17:41:29.171 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-test-2:18.2.2-954.g82316b04.el8.x86_64 12/162 2024-03-23T17:41:29.171 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : cephfs-mirror-2:18.2.2-954.g82316b04.el8.x86_64 13/162 2024-03-23T17:41:29.171 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libcephfs-devel-2:18.2.2-954.g82316b04.el8.x86_6 14/162 2024-03-23T17:41:29.172 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libcephfs2-2:18.2.2-954.g82316b04.el8.x86_64 15/162 2024-03-23T17:41:29.172 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libcephsqlite-2:18.2.2-954.g82316b04.el8.x86_64 16/162 2024-03-23T17:41:29.172 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librados-devel-2:18.2.2-954.g82316b04.el8.x86_64 17/162 2024-03-23T17:41:29.172 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libradosstriper1-2:18.2.2-954.g82316b04.el8.x86_ 18/162 2024-03-23T17:41:29.172 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librgw2-2:18.2.2-954.g82316b04.el8.x86_64 19/162 2024-03-23T17:41:29.172 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-ceph-argparse-2:18.2.2-954.g82316b04.el8 20/162 2024-03-23T17:41:29.172 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-ceph-common-2:18.2.2-954.g82316b04.el8.x 21/162 2024-03-23T17:41:29.172 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-cephfs-2:18.2.2-954.g82316b04.el8.x86_64 22/162 2024-03-23T17:41:29.172 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-rados-2:18.2.2-954.g82316b04.el8.x86_64 23/162 2024-03-23T17:41:29.172 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-rbd-2:18.2.2-954.g82316b04.el8.x86_64 24/162 2024-03-23T17:41:29.172 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-rgw-2:18.2.2-954.g82316b04.el8.x86_64 25/162 2024-03-23T17:41:29.172 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : rbd-fuse-2:18.2.2-954.g82316b04.el8.x86_64 26/162 2024-03-23T17:41:29.172 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : rbd-mirror-2:18.2.2-954.g82316b04.el8.x86_64 27/162 2024-03-23T17:41:29.185 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : rbd-nbd-2:18.2.2-954.g82316b04.el8.x86_64 28/162 2024-03-23T17:41:29.185 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-954.g82316b04.e 29/162 2024-03-23T17:41:29.185 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-954.g82316b04.el8.noar 30/162 2024-03-23T17:41:29.185 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-954.g82316b04.el8.no 31/162 2024-03-23T17:41:29.185 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-954.g8231 32/162 2024-03-23T17:41:29.185 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-954.g82316b04.el8 33/162 2024-03-23T17:41:29.185 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mgr-rook-2:18.2.2-954.g82316b04.el8.noarch 34/162 2024-03-23T17:41:29.185 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-954.g82316b04.el 35/162 2024-03-23T17:41:29.185 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-volume-2:18.2.2-954.g82316b04.el8.noarch 36/162 2024-03-23T17:41:29.185 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : cephadm-2:18.2.2-954.g82316b04.el8.noarch 37/162 2024-03-23T17:41:29.185 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : cephfs-top-2:18.2.2-954.g82316b04.el8.noarch 38/162 2024-03-23T17:41:29.185 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/162 2024-03-23T17:41:29.186 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/162 2024-03-23T17:41:29.186 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/162 2024-03-23T17:41:29.186 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/162 2024-03-23T17:41:29.186 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/162 2024-03-23T17:41:29.186 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/162 2024-03-23T17:41:29.186 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/162 2024-03-23T17:41:29.186 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/162 2024-03-23T17:41:29.186 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/162 2024-03-23T17:41:29.186 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/162 2024-03-23T17:41:29.186 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/162 2024-03-23T17:41:29.186 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/162 2024-03-23T17:41:29.186 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/162 2024-03-23T17:41:29.186 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/162 2024-03-23T17:41:29.187 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/162 2024-03-23T17:41:29.187 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/162 2024-03-23T17:41:29.187 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/162 2024-03-23T17:41:29.187 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/162 2024-03-23T17:41:29.187 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/162 2024-03-23T17:41:29.187 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/162 2024-03-23T17:41:29.187 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/162 2024-03-23T17:41:29.187 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/162 2024-03-23T17:41:29.187 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/162 2024-03-23T17:41:29.187 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/162 2024-03-23T17:41:29.187 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/162 2024-03-23T17:41:29.187 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/162 2024-03-23T17:41:29.187 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/162 2024-03-23T17:41:29.187 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/162 2024-03-23T17:41:29.188 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/162 2024-03-23T17:41:29.188 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/162 2024-03-23T17:41:29.188 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/162 2024-03-23T17:41:29.188 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/162 2024-03-23T17:41:29.188 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/162 2024-03-23T17:41:29.188 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 72/162 2024-03-23T17:41:29.188 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 73/162 2024-03-23T17:41:29.188 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 74/162 2024-03-23T17:41:29.188 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 75/162 2024-03-23T17:41:29.188 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 76/162 2024-03-23T17:41:29.188 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 77/162 2024-03-23T17:41:29.188 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 78/162 2024-03-23T17:41:29.188 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 79/162 2024-03-23T17:41:29.189 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/162 2024-03-23T17:41:29.189 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 81/162 2024-03-23T17:41:29.189 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 82/162 2024-03-23T17:41:29.189 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 83/162 2024-03-23T17:41:29.189 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 84/162 2024-03-23T17:41:29.189 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 85/162 2024-03-23T17:41:29.189 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 86/162 2024-03-23T17:41:29.189 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 87/162 2024-03-23T17:41:29.189 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 88/162 2024-03-23T17:41:29.189 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 89/162 2024-03-23T17:41:29.189 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libuuid-devel-2.32.1-43.el8.x86_64 90/162 2024-03-23T17:41:29.189 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 91/162 2024-03-23T17:41:29.189 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 92/162 2024-03-23T17:41:29.189 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-13.e 93/162 2024-03-23T17:41:29.190 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 94/162 2024-03-23T17:41:29.190 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 95/162 2024-03-23T17:41:29.190 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : openssl-devel-1:1.1.1k-12.el8_9.x86_64 96/162 2024-03-23T17:41:29.190 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librabbitmq-0.9.0-5.el8_9.x86_64 97/162 2024-03-23T17:41:29.190 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-cryptography-3.2.1-7.el8_9.x86_64 98/162 2024-03-23T17:41:29.190 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-packaging-16.8-9.el8.noarch 99/162 2024-03-23T17:41:29.190 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 100/162 2024-03-23T17:41:29.190 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 101/162 2024-03-23T17:41:29.190 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 102/162 2024-03-23T17:41:29.190 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 103/162 2024-03-23T17:41:29.190 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 104/162 2024-03-23T17:41:29.190 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 105/162 2024-03-23T17:41:29.190 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : bison-3.0.4-10.el8.x86_64 106/162 2024-03-23T17:41:29.191 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : flex-2.6.1-9.el8.x86_64 107/162 2024-03-23T17:41:29.191 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 108/162 2024-03-23T17:41:29.191 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 109/162 2024-03-23T17:41:29.191 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 110/162 2024-03-23T17:41:29.191 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 111/162 2024-03-23T17:41:29.191 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 112/162 2024-03-23T17:41:29.191 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 113/162 2024-03-23T17:41:29.191 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 114/162 2024-03-23T17:41:29.191 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 115/162 2024-03-23T17:41:29.191 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 116/162 2024-03-23T17:41:29.191 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 117/162 2024-03-23T17:41:29.191 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 118/162 2024-03-23T17:41:29.191 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 119/162 2024-03-23T17:41:29.191 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 120/162 2024-03-23T17:41:29.192 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 121/162 2024-03-23T17:41:29.192 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : automake-1.16.1-8.el8.noarch 122/162 2024-03-23T17:41:29.192 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libpq-13.11-1.el8.x86_64 123/162 2024-03-23T17:41:29.192 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : jq-1.6-7.el8.x86_64 124/162 2024-03-23T17:41:29.192 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 125/162 2024-03-23T17:41:29.192 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 126/162 2024-03-23T17:41:29.192 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : oniguruma-6.8.2-2.1.el8_9.x86_64 127/162 2024-03-23T17:41:29.192 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 128/162 2024-03-23T17:41:29.192 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 129/162 2024-03-23T17:41:29.192 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : postgresql-server-10.23-4.module+el8.9.0+21317+d 130/162 2024-03-23T17:41:29.192 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librados2-2:18.2.2-954.g82316b04.el8.x86_64 131/162 2024-03-23T17:41:29.192 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 132/162 2024-03-23T17:41:29.192 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librbd1-2:18.2.2-954.g82316b04.el8.x86_64 133/162 2024-03-23T17:41:29.192 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 134/162 2024-03-23T17:41:29.193 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 135/162 2024-03-23T17:41:29.193 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 136/162 2024-03-23T17:41:29.193 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 137/162 2024-03-23T17:41:29.193 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 138/162 2024-03-23T17:41:29.193 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libmount-2.32.1-43.el8.x86_64 139/162 2024-03-23T17:41:29.193 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 140/162 2024-03-23T17:41:29.193 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libsmartcols-2.32.1-43.el8.x86_64 141/162 2024-03-23T17:41:29.193 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 142/162 2024-03-23T17:41:29.193 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libuuid-2.32.1-43.el8.x86_64 143/162 2024-03-23T17:41:29.193 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 144/162 2024-03-23T17:41:29.193 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : util-linux-2.32.1-43.el8.x86_64 145/162 2024-03-23T17:41:29.193 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 146/162 2024-03-23T17:41:29.193 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : util-linux-user-2.32.1-43.el8.x86_64 147/162 2024-03-23T17:41:29.193 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 148/162 2024-03-23T17:41:29.194 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 149/162 2024-03-23T17:41:29.194 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 150/162 2024-03-23T17:41:29.194 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libblkid-2.32.1-43.el8.x86_64 151/162 2024-03-23T17:41:29.194 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 152/162 2024-03-23T17:41:29.194 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libfdisk-2.32.1-43.el8.x86_64 153/162 2024-03-23T17:41:29.194 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 154/162 2024-03-23T17:41:29.194 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : openssl-1:1.1.1k-12.el8_9.x86_64 155/162 2024-03-23T17:41:29.194 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 156/162 2024-03-23T17:41:29.194 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : openssl-libs-1:1.1.1k-12.el8_9.x86_64 157/162 2024-03-23T17:41:29.194 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 158/162 2024-03-23T17:41:29.194 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : xfsdump-3.1.8-7.el8_9.x86_64 159/162 2024-03-23T17:41:29.194 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 160/162 2024-03-23T17:41:29.194 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 161/162 2024-03-23T17:41:41.487 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 162/162 2024-03-23T17:41:41.487 INFO:teuthology.orchestra.run.smithi152.stdout:Installed products updated. 2024-03-23T17:41:42.095 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-23T17:41:42.095 INFO:teuthology.orchestra.run.smithi152.stdout:Upgraded: 2024-03-23T17:41:42.095 INFO:teuthology.orchestra.run.smithi152.stdout: elfutils-libelf-0.189-3.el8.x86_64 2024-03-23T17:41:42.095 INFO:teuthology.orchestra.run.smithi152.stdout: elfutils-libs-0.189-3.el8.x86_64 2024-03-23T17:41:42.096 INFO:teuthology.orchestra.run.smithi152.stdout: iproute-6.2.0-5.el8_9.x86_64 2024-03-23T17:41:42.096 INFO:teuthology.orchestra.run.smithi152.stdout: libblkid-2.32.1-43.el8.x86_64 2024-03-23T17:41:42.096 INFO:teuthology.orchestra.run.smithi152.stdout: libfdisk-2.32.1-43.el8.x86_64 2024-03-23T17:41:42.096 INFO:teuthology.orchestra.run.smithi152.stdout: libmount-2.32.1-43.el8.x86_64 2024-03-23T17:41:42.096 INFO:teuthology.orchestra.run.smithi152.stdout: librados2-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:42.096 INFO:teuthology.orchestra.run.smithi152.stdout: librbd1-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:42.096 INFO:teuthology.orchestra.run.smithi152.stdout: libsmartcols-2.32.1-43.el8.x86_64 2024-03-23T17:41:42.096 INFO:teuthology.orchestra.run.smithi152.stdout: libuuid-2.32.1-43.el8.x86_64 2024-03-23T17:41:42.096 INFO:teuthology.orchestra.run.smithi152.stdout: openssl-1:1.1.1k-12.el8_9.x86_64 2024-03-23T17:41:42.096 INFO:teuthology.orchestra.run.smithi152.stdout: openssl-libs-1:1.1.1k-12.el8_9.x86_64 2024-03-23T17:41:42.096 INFO:teuthology.orchestra.run.smithi152.stdout: util-linux-2.32.1-43.el8.x86_64 2024-03-23T17:41:42.096 INFO:teuthology.orchestra.run.smithi152.stdout: util-linux-user-2.32.1-43.el8.x86_64 2024-03-23T17:41:42.097 INFO:teuthology.orchestra.run.smithi152.stdout: xfsdump-3.1.8-7.el8_9.x86_64 2024-03-23T17:41:42.097 INFO:teuthology.orchestra.run.smithi152.stdout: xfsprogs-5.0.0-12.el8.x86_64 2024-03-23T17:41:42.097 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2024-03-23T17:41:42.097 INFO:teuthology.orchestra.run.smithi152.stdout: NetworkManager-initscripts-updown-1:1.40.16-13.el8_9.noarch 2024-03-23T17:41:42.097 INFO:teuthology.orchestra.run.smithi152.stdout: automake-1.16.1-8.el8.noarch 2024-03-23T17:41:42.097 INFO:teuthology.orchestra.run.smithi152.stdout: bison-3.0.4-10.el8.x86_64 2024-03-23T17:41:42.097 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:42.097 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-base-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:42.097 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-common-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:42.097 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-fuse-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:42.097 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-grafana-dashboards-2:18.2.2-954.g82316b04.el8.noarch 2024-03-23T17:41:42.097 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-immutable-object-cache-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:42.098 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mds-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:42.098 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:42.098 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-cephadm-2:18.2.2-954.g82316b04.el8.noarch 2024-03-23T17:41:42.098 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-dashboard-2:18.2.2-954.g82316b04.el8.noarch 2024-03-23T17:41:42.098 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-diskprediction-local-2:18.2.2-954.g82316b04.el8.noarch 2024-03-23T17:41:42.098 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-modules-core-2:18.2.2-954.g82316b04.el8.noarch 2024-03-23T17:41:42.098 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-rook-2:18.2.2-954.g82316b04.el8.noarch 2024-03-23T17:41:42.098 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mon-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:42.098 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-osd-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:42.098 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-prometheus-alerts-2:18.2.2-954.g82316b04.el8.noarch 2024-03-23T17:41:42.098 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-radosgw-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:42.098 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-selinux-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:42.098 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-test-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:42.099 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-volume-2:18.2.2-954.g82316b04.el8.noarch 2024-03-23T17:41:42.099 INFO:teuthology.orchestra.run.smithi152.stdout: cephadm-2:18.2.2-954.g82316b04.el8.noarch 2024-03-23T17:41:42.099 INFO:teuthology.orchestra.run.smithi152.stdout: cephfs-mirror-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:42.099 INFO:teuthology.orchestra.run.smithi152.stdout: cephfs-top-2:18.2.2-954.g82316b04.el8.noarch 2024-03-23T17:41:42.099 INFO:teuthology.orchestra.run.smithi152.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2024-03-23T17:41:42.099 INFO:teuthology.orchestra.run.smithi152.stdout: flex-2.6.1-9.el8.x86_64 2024-03-23T17:41:42.099 INFO:teuthology.orchestra.run.smithi152.stdout: fuse-2.9.7-17.el8.x86_64 2024-03-23T17:41:42.099 INFO:teuthology.orchestra.run.smithi152.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-03-23T17:41:42.099 INFO:teuthology.orchestra.run.smithi152.stdout: jq-1.6-7.el8.x86_64 2024-03-23T17:41:42.099 INFO:teuthology.orchestra.run.smithi152.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2024-03-23T17:41:42.099 INFO:teuthology.orchestra.run.smithi152.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2024-03-23T17:41:42.099 INFO:teuthology.orchestra.run.smithi152.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-03-23T17:41:42.100 INFO:teuthology.orchestra.run.smithi152.stdout: libcephfs-devel-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:42.100 INFO:teuthology.orchestra.run.smithi152.stdout: libcephfs2-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:42.100 INFO:teuthology.orchestra.run.smithi152.stdout: libcephsqlite-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:42.100 INFO:teuthology.orchestra.run.smithi152.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2024-03-23T17:41:42.100 INFO:teuthology.orchestra.run.smithi152.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-03-23T17:41:42.100 INFO:teuthology.orchestra.run.smithi152.stdout: liboath-2.6.2-3.el8.x86_64 2024-03-23T17:41:42.100 INFO:teuthology.orchestra.run.smithi152.stdout: libpq-13.11-1.el8.x86_64 2024-03-23T17:41:42.100 INFO:teuthology.orchestra.run.smithi152.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-03-23T17:41:42.100 INFO:teuthology.orchestra.run.smithi152.stdout: librabbitmq-0.9.0-5.el8_9.x86_64 2024-03-23T17:41:42.100 INFO:teuthology.orchestra.run.smithi152.stdout: librados-devel-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:42.100 INFO:teuthology.orchestra.run.smithi152.stdout: libradosstriper1-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:42.100 INFO:teuthology.orchestra.run.smithi152.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-03-23T17:41:42.101 INFO:teuthology.orchestra.run.smithi152.stdout: librgw2-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:42.101 INFO:teuthology.orchestra.run.smithi152.stdout: libselinux-devel-2.9-5.el8.x86_64 2024-03-23T17:41:42.101 INFO:teuthology.orchestra.run.smithi152.stdout: libsepol-devel-2.9-3.el8.x86_64 2024-03-23T17:41:42.101 INFO:teuthology.orchestra.run.smithi152.stdout: libtool-2.4.6-25.el8.x86_64 2024-03-23T17:41:42.101 INFO:teuthology.orchestra.run.smithi152.stdout: libunwind-1.3.1-3.el8.x86_64 2024-03-23T17:41:42.101 INFO:teuthology.orchestra.run.smithi152.stdout: libuuid-devel-2.32.1-43.el8.x86_64 2024-03-23T17:41:42.101 INFO:teuthology.orchestra.run.smithi152.stdout: libverto-devel-0.3.0-5.el8.x86_64 2024-03-23T17:41:42.101 INFO:teuthology.orchestra.run.smithi152.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2024-03-23T17:41:42.101 INFO:teuthology.orchestra.run.smithi152.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-03-23T17:41:42.101 INFO:teuthology.orchestra.run.smithi152.stdout: oniguruma-6.8.2-2.1.el8_9.x86_64 2024-03-23T17:41:42.101 INFO:teuthology.orchestra.run.smithi152.stdout: openblas-0.3.15-6.el8.x86_64 2024-03-23T17:41:42.101 INFO:teuthology.orchestra.run.smithi152.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-03-23T17:41:42.101 INFO:teuthology.orchestra.run.smithi152.stdout: openssl-devel-1:1.1.1k-12.el8_9.x86_64 2024-03-23T17:41:42.101 INFO:teuthology.orchestra.run.smithi152.stdout: pcre2-devel-10.32-2.el8.x86_64 2024-03-23T17:41:42.102 INFO:teuthology.orchestra.run.smithi152.stdout: pcre2-utf16-10.32-2.el8.x86_64 2024-03-23T17:41:42.102 INFO:teuthology.orchestra.run.smithi152.stdout: pcre2-utf32-10.32-2.el8.x86_64 2024-03-23T17:41:42.102 INFO:teuthology.orchestra.run.smithi152.stdout: postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-03-23T17:41:42.102 INFO:teuthology.orchestra.run.smithi152.stdout: postgresql-contrib-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-03-23T17:41:42.102 INFO:teuthology.orchestra.run.smithi152.stdout: postgresql-server-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-03-23T17:41:42.102 INFO:teuthology.orchestra.run.smithi152.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-03-23T17:41:42.102 INFO:teuthology.orchestra.run.smithi152.stdout: python3-babel-2.5.1-7.el8.noarch 2024-03-23T17:41:42.102 INFO:teuthology.orchestra.run.smithi152.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-03-23T17:41:42.102 INFO:teuthology.orchestra.run.smithi152.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-03-23T17:41:42.102 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-03-23T17:41:42.102 INFO:teuthology.orchestra.run.smithi152.stdout: python3-ceph-argparse-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:42.102 INFO:teuthology.orchestra.run.smithi152.stdout: python3-ceph-common-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:42.102 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cephfs-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:42.103 INFO:teuthology.orchestra.run.smithi152.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-03-23T17:41:42.103 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-03-23T17:41:42.103 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-03-23T17:41:42.103 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-03-23T17:41:42.103 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cryptography-3.2.1-7.el8_9.x86_64 2024-03-23T17:41:42.103 INFO:teuthology.orchestra.run.smithi152.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-03-23T17:41:42.103 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-6.2-6.el8.noarch 2024-03-23T17:41:42.103 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-03-23T17:41:42.103 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-03-23T17:41:42.103 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-03-23T17:41:42.103 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-03-23T17:41:42.103 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-03-23T17:41:42.103 INFO:teuthology.orchestra.run.smithi152.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-03-23T17:41:42.104 INFO:teuthology.orchestra.run.smithi152.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-03-23T17:41:42.104 INFO:teuthology.orchestra.run.smithi152.stdout: python3-mako-1.0.6-14.el8.noarch 2024-03-23T17:41:42.104 INFO:teuthology.orchestra.run.smithi152.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-03-23T17:41:42.104 INFO:teuthology.orchestra.run.smithi152.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-03-23T17:41:42.104 INFO:teuthology.orchestra.run.smithi152.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-03-23T17:41:42.104 INFO:teuthology.orchestra.run.smithi152.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-03-23T17:41:42.104 INFO:teuthology.orchestra.run.smithi152.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-03-23T17:41:42.104 INFO:teuthology.orchestra.run.smithi152.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-03-23T17:41:42.104 INFO:teuthology.orchestra.run.smithi152.stdout: python3-packaging-16.8-9.el8.noarch 2024-03-23T17:41:42.104 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-03-23T17:41:42.104 INFO:teuthology.orchestra.run.smithi152.stdout: python3-portend-2.6-1.el8.noarch 2024-03-23T17:41:42.104 INFO:teuthology.orchestra.run.smithi152.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-03-23T17:41:42.104 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-03-23T17:41:42.104 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-03-23T17:41:42.105 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-03-23T17:41:42.105 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pycparser-2.14-14.el8.noarch 2024-03-23T17:41:42.105 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pytz-2017.2-11.el8.noarch 2024-03-23T17:41:42.105 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rados-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:42.105 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rbd-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:42.105 INFO:teuthology.orchestra.run.smithi152.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-03-23T17:41:42.105 INFO:teuthology.orchestra.run.smithi152.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-03-23T17:41:42.105 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rgw-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:42.105 INFO:teuthology.orchestra.run.smithi152.stdout: python3-routes-2.4.1-12.el8.noarch 2024-03-23T17:41:42.105 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rsa-4.9-2.el8.noarch 2024-03-23T17:41:42.105 INFO:teuthology.orchestra.run.smithi152.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-03-23T17:41:42.105 INFO:teuthology.orchestra.run.smithi152.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-03-23T17:41:42.105 INFO:teuthology.orchestra.run.smithi152.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-03-23T17:41:42.106 INFO:teuthology.orchestra.run.smithi152.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-03-23T17:41:42.106 INFO:teuthology.orchestra.run.smithi152.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-03-23T17:41:42.106 INFO:teuthology.orchestra.run.smithi152.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-03-23T17:41:42.106 INFO:teuthology.orchestra.run.smithi152.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-03-23T17:41:42.106 INFO:teuthology.orchestra.run.smithi152.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-03-23T17:41:42.106 INFO:teuthology.orchestra.run.smithi152.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-03-23T17:41:42.106 INFO:teuthology.orchestra.run.smithi152.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-03-23T17:41:42.106 INFO:teuthology.orchestra.run.smithi152.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-03-23T17:41:42.106 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-fuse-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:42.106 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-mirror-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:42.106 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-nbd-2:18.2.2-954.g82316b04.el8.x86_64 2024-03-23T17:41:42.106 INFO:teuthology.orchestra.run.smithi152.stdout: socat-1.7.4.1-1.el8.x86_64 2024-03-23T17:41:42.107 INFO:teuthology.orchestra.run.smithi152.stdout: thrift-0.13.0-2.el8.x86_64 2024-03-23T17:41:42.107 INFO:teuthology.orchestra.run.smithi152.stdout: uuid-1.6.2-43.el8.x86_64 2024-03-23T17:41:42.107 INFO:teuthology.orchestra.run.smithi152.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2024-03-23T17:41:42.107 INFO:teuthology.orchestra.run.smithi152.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-03-23T17:41:42.107 INFO:teuthology.orchestra.run.smithi152.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2024-03-23T17:41:42.107 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-23T17:41:42.107 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2024-03-23T17:41:42.320 DEBUG:teuthology.parallel:result is None 2024-03-23T17:41:42.321 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=82316b04821a6c2c1d458a881920454bbadb8837 2024-03-23T17:41:42.474 DEBUG:teuthology.orchestra.run.smithi086:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-23T17:41:42.532 INFO:teuthology.orchestra.run.smithi086.stdout:18.2.2-954.g82316b04.el8 2024-03-23T17:41:42.533 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-954.g82316b04.el8 2024-03-23T17:41:42.533 INFO:teuthology.task.install:The correct ceph version 18.2.2-954.g82316b04 is installed. 2024-03-23T17:41:42.535 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=82316b04821a6c2c1d458a881920454bbadb8837 2024-03-23T17:41:42.748 DEBUG:teuthology.orchestra.run.smithi092:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-23T17:41:42.778 INFO:teuthology.orchestra.run.smithi092.stdout:18.2.2-954.g82316b04.el8 2024-03-23T17:41:42.778 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-954.g82316b04.el8 2024-03-23T17:41:42.778 INFO:teuthology.task.install:The correct ceph version 18.2.2-954.g82316b04 is installed. 2024-03-23T17:41:42.780 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=82316b04821a6c2c1d458a881920454bbadb8837 2024-03-23T17:41:43.000 DEBUG:teuthology.orchestra.run.smithi152:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-23T17:41:43.026 INFO:teuthology.orchestra.run.smithi152.stdout:18.2.2-954.g82316b04.el8 2024-03-23T17:41:43.026 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-954.g82316b04.el8 2024-03-23T17:41:43.026 INFO:teuthology.task.install:The correct ceph version 18.2.2-954.g82316b04 is installed. 2024-03-23T17:41:43.028 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-23T17:41:43.028 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-23T17:41:43.028 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-23T17:41:43.122 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-03-23T17:41:43.122 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-23T17:41:43.170 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-03-23T17:41:43.171 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-23T17:41:43.217 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-23T17:41:43.217 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-23T17:41:43.217 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/usr/bin/daemon-helper 2024-03-23T17:41:43.284 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-23T17:41:43.389 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-03-23T17:41:43.389 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/daemon-helper 2024-03-23T17:41:43.423 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-23T17:41:43.495 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-03-23T17:41:43.495 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/usr/bin/daemon-helper 2024-03-23T17:41:43.524 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-23T17:41:43.592 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-23T17:41:43.593 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-23T17:41:43.593 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-23T17:41:43.657 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-23T17:41:43.760 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-03-23T17:41:43.761 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-23T17:41:43.793 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-23T17:41:43.863 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-03-23T17:41:43.863 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-23T17:41:43.894 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-23T17:41:43.963 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-23T17:41:43.963 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-23T17:41:43.963 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/usr/bin/stdin-killer 2024-03-23T17:41:44.028 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-23T17:41:44.132 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-03-23T17:41:44.132 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/stdin-killer 2024-03-23T17:41:44.165 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-23T17:41:44.237 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-03-23T17:41:44.237 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/usr/bin/stdin-killer 2024-03-23T17:41:44.267 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-23T17:41:44.338 INFO:teuthology.run_tasks:Running task cephadm... 2024-03-23T17:41:44.417 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '82316b04821a6c2c1d458a881920454bbadb8837', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-03-23T17:41:44.417 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 2024-03-23T17:41:44.418 INFO:tasks.cephadm:Cluster fsid is 9d76c9e8-e93c-11ee-95cd-87774f69a715 2024-03-23T17:41:44.418 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-03-23T17:41:44.418 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.86', 'mon.b': '172.21.15.92', 'mon.c': '172.21.15.152'} 2024-03-23T17:41:44.418 INFO:tasks.cephadm:First mon is mon.a on smithi086 2024-03-23T17:41:44.418 INFO:tasks.cephadm:First mgr is x 2024-03-23T17:41:44.418 INFO:tasks.cephadm:Normalizing hostnames... 2024-03-23T17:41:44.419 DEBUG:teuthology.orchestra.run.smithi086:> sudo hostname $(hostname -s) 2024-03-23T17:41:44.485 DEBUG:teuthology.orchestra.run.smithi092:> sudo hostname $(hostname -s) 2024-03-23T17:41:44.519 DEBUG:teuthology.orchestra.run.smithi152:> sudo hostname $(hostname -s) 2024-03-23T17:41:44.554 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-03-23T17:41:44.554 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=82316b04821a6c2c1d458a881920454bbadb8837 2024-03-23T17:41:44.774 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '82316b04821a6c2c1d458a881920454bbadb8837', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/77868/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.2.2-954-g82316b04', 'node_name': '172.21.2.5+braggi05', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.2.2-954.g82316b04'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-yuri11-testing-2024-03-21-0851-reef/82316b04821a6c2c1d458a881920454bbadb8837/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2024-03-21 22:55:38.454766', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-yuri11-testing-2024-03-21-0851-reef', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-yuri11-testing-2024-03-21-0851-reef/82316b04821a6c2c1d458a881920454bbadb8837/centos/8/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-03-23T17:41:44.990 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref wip-yuri11-testing-2024-03-21-0851-reef, sha1 82316b04821a6c2c1d458a881920454bbadb8837 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F8%2Fx86_64&flavor=default&sha1=82316b04821a6c2c1d458a881920454bbadb8837 2024-03-23T17:41:44.992 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/wip-yuri11-testing-2024-03-21-0851-reef/82316b04821a6c2c1d458a881920454bbadb8837/centos/8/x86_64/flavors/default/cephadm 2024-03-23T17:41:44.992 DEBUG:teuthology.orchestra.run.smithi086:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-yuri11-testing-2024-03-21-0851-reef/82316b04821a6c2c1d458a881920454bbadb8837/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-23T17:41:45.452 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-rw-r--. 1 ubuntu ubuntu 702420 Mar 23 17:41 /home/ubuntu/cephtest/cephadm 2024-03-23T17:41:45.452 DEBUG:teuthology.orchestra.run.smithi092:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-yuri11-testing-2024-03-21-0851-reef/82316b04821a6c2c1d458a881920454bbadb8837/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-23T17:41:46.022 INFO:teuthology.orchestra.run.smithi092.stdout:-rw-rw-r--. 1 ubuntu ubuntu 702420 Mar 23 17:41 /home/ubuntu/cephtest/cephadm 2024-03-23T17:41:46.023 DEBUG:teuthology.orchestra.run.smithi152:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-yuri11-testing-2024-03-21-0851-reef/82316b04821a6c2c1d458a881920454bbadb8837/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-23T17:41:46.452 INFO:teuthology.orchestra.run.smithi152.stdout:-rw-rw-r--. 1 ubuntu ubuntu 702420 Mar 23 17:41 /home/ubuntu/cephtest/cephadm 2024-03-23T17:41:46.453 DEBUG:teuthology.orchestra.run.smithi086:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-23T17:41:46.502 DEBUG:teuthology.orchestra.run.smithi092:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-23T17:41:46.523 DEBUG:teuthology.orchestra.run.smithi152:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-23T17:41:46.561 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 on all hosts... 2024-03-23T17:41:46.562 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 pull 2024-03-23T17:41:46.566 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 pull 2024-03-23T17:41:46.568 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 pull 2024-03-23T17:41:47.542 INFO:teuthology.orchestra.run.smithi086.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837... 2024-03-23T17:41:47.574 INFO:teuthology.orchestra.run.smithi092.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837... 2024-03-23T17:41:50.001 INFO:teuthology.orchestra.run.smithi152.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837... 2024-03-23T17:42:14.035 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-03-23T17:42:14.035 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_version": "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)", 2024-03-23T17:42:14.036 INFO:teuthology.orchestra.run.smithi092.stdout: "image_id": "b24e1baba6370feeb9f066519871168c5c5655398bc2be05a927affe239d493d", 2024-03-23T17:42:14.036 INFO:teuthology.orchestra.run.smithi092.stdout: "repo_digests": [ 2024-03-23T17:42:14.036 INFO:teuthology.orchestra.run.smithi092.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4026324c507519c5a82fd7010cfd052f0faff768cf9b4dba85fa6998f59d8d2f" 2024-03-23T17:42:14.036 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-03-23T17:42:14.036 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-03-23T17:42:18.335 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-03-23T17:42:18.335 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_version": "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)", 2024-03-23T17:42:18.335 INFO:teuthology.orchestra.run.smithi152.stdout: "image_id": "b24e1baba6370feeb9f066519871168c5c5655398bc2be05a927affe239d493d", 2024-03-23T17:42:18.335 INFO:teuthology.orchestra.run.smithi152.stdout: "repo_digests": [ 2024-03-23T17:42:18.335 INFO:teuthology.orchestra.run.smithi152.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4026324c507519c5a82fd7010cfd052f0faff768cf9b4dba85fa6998f59d8d2f" 2024-03-23T17:42:18.336 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-03-23T17:42:18.336 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-03-23T17:42:22.133 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-23T17:42:22.133 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_version": "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)", 2024-03-23T17:42:22.133 INFO:teuthology.orchestra.run.smithi086.stdout: "image_id": "b24e1baba6370feeb9f066519871168c5c5655398bc2be05a927affe239d493d", 2024-03-23T17:42:22.134 INFO:teuthology.orchestra.run.smithi086.stdout: "repo_digests": [ 2024-03-23T17:42:22.134 INFO:teuthology.orchestra.run.smithi086.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4026324c507519c5a82fd7010cfd052f0faff768cf9b4dba85fa6998f59d8d2f" 2024-03-23T17:42:22.134 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-03-23T17:42:22.134 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-23T17:42:22.160 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /etc/ceph 2024-03-23T17:42:22.228 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /etc/ceph 2024-03-23T17:42:22.266 DEBUG:teuthology.orchestra.run.smithi152:> sudo mkdir -p /etc/ceph 2024-03-23T17:42:22.304 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 777 /etc/ceph 2024-03-23T17:42:22.368 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 777 /etc/ceph 2024-03-23T17:42:22.400 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod 777 /etc/ceph 2024-03-23T17:42:22.432 INFO:tasks.cephadm:Writing seed config... 2024-03-23T17:42:22.434 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-03-23T17:42:22.434 INFO:tasks.cephadm: override: [client] debug client = 20 2024-03-23T17:42:22.434 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-03-23T17:42:22.434 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-03-23T17:42:22.434 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-03-23T17:42:22.434 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-03-23T17:42:22.434 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-03-23T17:42:22.434 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-03-23T17:42:22.434 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-03-23T17:42:22.434 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-03-23T17:42:22.435 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-03-23T17:42:22.435 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-03-23T17:42:22.435 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-03-23T17:42:22.435 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-03-23T17:42:22.435 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-03-23T17:42:22.435 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-03-23T17:42:22.435 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-03-23T17:42:22.435 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-03-23T17:42:22.435 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-03-23T17:42:22.435 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-03-23T17:42:22.435 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-03-23T17:42:22.435 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-03-23T17:42:22.435 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-03-23T17:42:22.435 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-03-23T17:42:22.436 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-03-23T17:42:22.436 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-03-23T17:42:22.436 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-03-23T17:42:22.436 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-03-23T17:42:22.436 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-03-23T17:42:22.436 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-03-23T17:42:22.436 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-03-23T17:42:22.436 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-03-23T17:42:22.436 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-03-23T17:42:22.436 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-03-23T17:42:22.436 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-03-23T17:42:22.436 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-03-23T17:42:22.436 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-03-23T17:42:22.436 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-03-23T17:42:22.436 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-03-23T17:42:22.437 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-03-23T17:42:22.437 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-03-23T17:42:22.437 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-03-23T17:42:22.437 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-03-23T17:42:22.437 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-03-23T17:42:22.437 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-03-23T17:42:22.437 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-03-23T17:42:22.438 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-23T17:42:22.438 DEBUG:teuthology.orchestra.run.smithi086:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-03-23T17:42:22.479 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 9d76c9e8-e93c-11ee-95cd-87774f69a715 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-03-23T17:42:22.480 DEBUG:teuthology.orchestra.run.smithi086:mon.a> sudo journalctl -f -n 0 -u ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@mon.a.service 2024-03-23T17:42:22.523 DEBUG:teuthology.orchestra.run.smithi086:mgr.x> sudo journalctl -f -n 0 -u ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@mgr.x.service 2024-03-23T17:42:22.568 INFO:tasks.cephadm:Bootstrapping... 2024-03-23T17:42:22.568 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 -v bootstrap --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.86 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-03-23T17:42:22.586 INFO:journalctl@ceph.mon.a.smithi086.stdout:-- Logs begin at Sat 2024-03-23 17:25:35 UTC. -- 2024-03-23T17:42:22.631 INFO:journalctl@ceph.mgr.x.smithi086.stdout:-- Logs begin at Sat 2024-03-23 17:25:35 UTC. -- 2024-03-23T17:42:22.779 INFO:teuthology.orchestra.run.smithi086.stdout:-------------------------------------------------------------------------------- 2024-03-23T17:42:22.779 INFO:teuthology.orchestra.run.smithi086.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837', '-v', 'bootstrap', '--fsid', '9d76c9e8-e93c-11ee-95cd-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.86', '--skip-admin-label'] 2024-03-23T17:42:22.824 INFO:teuthology.orchestra.run.smithi086.stdout:/bin/podman: stdout 4.6.1 2024-03-23T17:42:22.824 INFO:teuthology.orchestra.run.smithi086.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-03-23T17:42:22.825 INFO:teuthology.orchestra.run.smithi086.stdout:Verifying podman|docker is present... 2024-03-23T17:42:22.868 INFO:teuthology.orchestra.run.smithi086.stdout:/bin/podman: stdout 4.6.1 2024-03-23T17:42:22.868 INFO:teuthology.orchestra.run.smithi086.stdout:Verifying lvm2 is present... 2024-03-23T17:42:22.868 INFO:teuthology.orchestra.run.smithi086.stdout:Verifying time synchronization is in place... 2024-03-23T17:42:22.882 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-23T17:42:22.882 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-23T17:42:22.894 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-23T17:42:22.894 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout inactive 2024-03-23T17:42:22.906 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout enabled 2024-03-23T17:42:22.918 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout active 2024-03-23T17:42:22.918 INFO:teuthology.orchestra.run.smithi086.stdout:Unit chronyd.service is enabled and running 2024-03-23T17:42:22.918 INFO:teuthology.orchestra.run.smithi086.stdout:Repeating the final host check... 2024-03-23T17:42:22.961 INFO:teuthology.orchestra.run.smithi086.stdout:/bin/podman: stdout 4.6.1 2024-03-23T17:42:22.961 INFO:teuthology.orchestra.run.smithi086.stdout:podman (/bin/podman) version 4.6.1 is present 2024-03-23T17:42:22.961 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl is present 2024-03-23T17:42:22.961 INFO:teuthology.orchestra.run.smithi086.stdout:lvcreate is present 2024-03-23T17:42:22.973 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-23T17:42:22.973 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-23T17:42:22.985 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-23T17:42:22.985 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout inactive 2024-03-23T17:42:22.998 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout enabled 2024-03-23T17:42:23.009 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout active 2024-03-23T17:42:23.010 INFO:teuthology.orchestra.run.smithi086.stdout:Unit chronyd.service is enabled and running 2024-03-23T17:42:23.010 INFO:teuthology.orchestra.run.smithi086.stdout:Host looks OK 2024-03-23T17:42:23.010 INFO:teuthology.orchestra.run.smithi086.stdout:Cluster fsid: 9d76c9e8-e93c-11ee-95cd-87774f69a715 2024-03-23T17:42:23.010 INFO:teuthology.orchestra.run.smithi086.stdout:Acquiring lock 140586826705384 on /run/cephadm/9d76c9e8-e93c-11ee-95cd-87774f69a715.lock 2024-03-23T17:42:23.010 INFO:teuthology.orchestra.run.smithi086.stdout:Lock 140586826705384 acquired on /run/cephadm/9d76c9e8-e93c-11ee-95cd-87774f69a715.lock 2024-03-23T17:42:23.011 INFO:teuthology.orchestra.run.smithi086.stdout:Verifying IP 172.21.15.86 port 3300 ... 2024-03-23T17:42:23.011 INFO:teuthology.orchestra.run.smithi086.stdout:Verifying IP 172.21.15.86 port 6789 ... 2024-03-23T17:42:23.012 INFO:teuthology.orchestra.run.smithi086.stdout:Base mon IP(s) is [172.21.15.86:3300, 172.21.15.86:6789], mon addrv is [v2:172.21.15.86:3300,v1:172.21.15.86:6789] 2024-03-23T17:42:23.022 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp metric 100 2024-03-23T17:42:23.022 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.86 metric 100 2024-03-23T17:42:23.026 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-03-23T17:42:23.027 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-03-23T17:42:23.027 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1782sec hoplimit 64 pref medium 2024-03-23T17:42:23.033 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-03-23T17:42:23.033 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-03-23T17:42:23.033 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-23T17:42:23.033 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-03-23T17:42:23.034 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2024-03-23T17:42:23.034 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-23T17:42:23.034 INFO:teuthology.orchestra.run.smithi086.stdout:Mon IP `172.21.15.86` is in CIDR network `172.21.0.0/20` 2024-03-23T17:42:23.034 INFO:teuthology.orchestra.run.smithi086.stdout:Mon IP `172.21.15.86` is in CIDR network `172.21.0.0/20` 2024-03-23T17:42:23.035 INFO:teuthology.orchestra.run.smithi086.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-03-23T17:42:23.035 INFO:teuthology.orchestra.run.smithi086.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-03-23T17:42:23.036 INFO:teuthology.orchestra.run.smithi086.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837... 2024-03-23T17:42:23.436 INFO:teuthology.orchestra.run.smithi086.stdout:/bin/podman: stdout b24e1baba6370feeb9f066519871168c5c5655398bc2be05a927affe239d493d 2024-03-23T17:42:23.436 INFO:teuthology.orchestra.run.smithi086.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837... 2024-03-23T17:42:23.436 INFO:teuthology.orchestra.run.smithi086.stdout:/bin/podman: stderr Getting image source signatures 2024-03-23T17:42:23.436 INFO:teuthology.orchestra.run.smithi086.stdout:/bin/podman: stderr Copying blob sha256:9516588f908cdb14b5e4062a6063db2d9291b355c3f32e2a99864608a4a7b649 2024-03-23T17:42:23.436 INFO:teuthology.orchestra.run.smithi086.stdout:/bin/podman: stderr Copying blob sha256:5f25cee497d7c88a73a15cc28d4fa8858206cd95950d35eef0550a60530b7012 2024-03-23T17:42:23.436 INFO:teuthology.orchestra.run.smithi086.stdout:/bin/podman: stderr Copying config sha256:b24e1baba6370feeb9f066519871168c5c5655398bc2be05a927affe239d493d 2024-03-23T17:42:23.436 INFO:teuthology.orchestra.run.smithi086.stdout:/bin/podman: stderr Writing manifest to image destination 2024-03-23T17:42:24.199 INFO:teuthology.orchestra.run.smithi086.stdout:ceph: stdout ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable) 2024-03-23T17:42:24.199 INFO:teuthology.orchestra.run.smithi086.stdout:Ceph version: ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable) 2024-03-23T17:42:24.199 INFO:teuthology.orchestra.run.smithi086.stdout:Extracting ceph user uid/gid from container image... 2024-03-23T17:42:24.779 INFO:teuthology.orchestra.run.smithi086.stdout:stat: stdout 167 167 2024-03-23T17:42:24.779 INFO:teuthology.orchestra.run.smithi086.stdout:Creating initial keys... 2024-03-23T17:42:25.377 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph-authtool: stdout AQCBFP9ly7lrAxAAC/MkI0pREoPw0AiieFxXoA== 2024-03-23T17:42:26.003 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph-authtool: stdout AQCBFP9l5kStJhAAJ8Kwh91xfMJTM7zDF+8cKw== 2024-03-23T17:42:26.593 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph-authtool: stdout AQCCFP9lcUTRDxAA6h43V/PxRzAOuRrCpBAj9w== 2024-03-23T17:42:26.594 INFO:teuthology.orchestra.run.smithi086.stdout:Creating initial monmap... 2024-03-23T17:42:27.767 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-23T17:42:27.768 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2024-03-23T17:42:27.768 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 9d76c9e8-e93c-11ee-95cd-87774f69a715 2024-03-23T17:42:27.768 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-23T17:42:27.768 INFO:teuthology.orchestra.run.smithi086.stdout:monmaptool for a [v2:172.21.15.86:3300,v1:172.21.15.86:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-23T17:42:27.768 INFO:teuthology.orchestra.run.smithi086.stdout:setting min_mon_release = pacific 2024-03-23T17:42:27.768 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/monmaptool: set fsid to 9d76c9e8-e93c-11ee-95cd-87774f69a715 2024-03-23T17:42:27.768 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-23T17:42:27.768 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:42:27.768 INFO:teuthology.orchestra.run.smithi086.stdout:Creating mon... 2024-03-23T17:42:28.363 INFO:teuthology.orchestra.run.smithi086.stdout:create mon.a on 2024-03-23T17:42:28.596 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2024-03-23T17:42:28.797 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-03-23T17:42:29.026 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715.target → /etc/systemd/system/ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715.target. 2024-03-23T17:42:29.027 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715.target → /etc/systemd/system/ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715.target. 2024-03-23T17:42:29.368 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@mon.a 2024-03-23T17:42:29.368 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stderr Failed to reset failed state of unit ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@mon.a.service: Unit ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@mon.a.service not loaded. 2024-03-23T17:42:29.553 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715.target.wants/ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@mon.a.service → /etc/systemd/system/ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@.service. 2024-03-23T17:42:29.641 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:29 smithi086 systemd[1]: Starting Ceph mon.a for 9d76c9e8-e93c-11ee-95cd-87774f69a715... 2024-03-23T17:42:30.085 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-23T17:42:30.086 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout disabled 2024-03-23T17:42:30.104 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:30 smithi086 bash[37685]: f262260c883b7a85a144c7d68e149dd7fb9ed78133f9c6d890f1045a13d9d669 2024-03-23T17:42:30.104 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:30 smithi086 systemd[1]: Started Ceph mon.a for 9d76c9e8-e93c-11ee-95cd-87774f69a715. 2024-03-23T17:42:30.104 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:30 smithi086 ceph-mon[37723]: mkfs 9d76c9e8-e93c-11ee-95cd-87774f69a715 2024-03-23T17:42:30.105 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-23T17:42:30.105 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout inactive 2024-03-23T17:42:30.105 INFO:teuthology.orchestra.run.smithi086.stdout:firewalld.service is not enabled 2024-03-23T17:42:30.105 INFO:teuthology.orchestra.run.smithi086.stdout:Not possible to enable service . firewalld.service is not available 2024-03-23T17:42:30.105 INFO:teuthology.orchestra.run.smithi086.stdout:Waiting for mon to start... 2024-03-23T17:42:30.105 INFO:teuthology.orchestra.run.smithi086.stdout:Waiting for mon... 2024-03-23T17:42:30.326 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:30 smithi086 ceph-mon[37723]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-23T17:42:30.957 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout cluster: 2024-03-23T17:42:30.963 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout id: 9d76c9e8-e93c-11ee-95cd-87774f69a715 2024-03-23T17:42:30.963 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-03-23T17:42:30.963 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-03-23T17:42:30.963 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout services: 2024-03-23T17:42:30.963 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.538693s) 2024-03-23T17:42:30.964 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-03-23T17:42:30.964 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-03-23T17:42:30.964 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-03-23T17:42:30.964 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout data: 2024-03-23T17:42:30.964 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-03-23T17:42:30.964 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-03-23T17:42:30.964 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-03-23T17:42:30.964 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout pgs: 2024-03-23T17:42:30.964 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-03-23T17:42:30.964 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:30.552+0000 7f2252c00700 1 Processor -- start 2024-03-23T17:42:30.964 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:30.552+0000 7f2252c00700 1 -- start start 2024-03-23T17:42:30.964 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:30.552+0000 7f2252c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f224c07ad90 0x7f224c0791f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:42:30.964 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:30.552+0000 7f2252c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f224c0797c0 con 0x7f224c07ad90 2024-03-23T17:42:30.964 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:30.553+0000 7f2251800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f224c07ad90 0x7f224c0791f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:42:30.965 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:30.553+0000 7f2251800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f224c07ad90 0x7f224c0791f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60408/0 (socket says 172.21.15.86:60408) 2024-03-23T17:42:30.965 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:30.553+0000 7f2251800700 1 -- 172.21.15.86:0/240600267 learned_addr learned my addr 172.21.15.86:0/240600267 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:42:30.965 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:30.553+0000 7f2251800700 1 -- 172.21.15.86:0/240600267 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f224c079900 con 0x7f224c07ad90 2024-03-23T17:42:30.965 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:30.554+0000 7f2251800700 1 --2- 172.21.15.86:0/240600267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f224c07ad90 0x7f224c0791f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f22400098a0 tx=0x7f2240009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a45a2ea2a7f1583f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:42:30.965 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:30.554+0000 7f224be00700 1 -- 172.21.15.86:0/240600267 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2240004160 con 0x7f224c07ad90 2024-03-23T17:42:30.965 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:30.554+0000 7f224be00700 1 -- 172.21.15.86:0/240600267 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22400042c0 con 0x7f224c07ad90 2024-03-23T17:42:30.965 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:30.554+0000 7f224be00700 1 -- 172.21.15.86:0/240600267 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f22400045a0 con 0x7f224c07ad90 2024-03-23T17:42:30.965 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:30.554+0000 7f2252c00700 1 -- 172.21.15.86:0/240600267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f224c07ad90 msgr2=0x7f224c0791f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:42:30.965 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:30.554+0000 7f2252c00700 1 --2- 172.21.15.86:0/240600267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f224c07ad90 0x7f224c0791f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f22400098a0 tx=0x7f2240009bb0 comp rx=0 tx=0).stop 2024-03-23T17:42:30.965 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:30.555+0000 7f2252c00700 1 -- 172.21.15.86:0/240600267 shutdown_connections 2024-03-23T17:42:30.965 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:30.555+0000 7f2252c00700 1 --2- 172.21.15.86:0/240600267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f224c07ad90 0x7f224c0791f0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:42:30.965 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:30.555+0000 7f2252c00700 1 -- 172.21.15.86:0/240600267 >> 172.21.15.86:0/240600267 conn(0x7f224c1019b0 msgr2=0x7f224c103dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:42:30.965 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:30.555+0000 7f2252c00700 1 -- 172.21.15.86:0/240600267 shutdown_connections 2024-03-23T17:42:30.966 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:30.555+0000 7f2252c00700 1 -- 172.21.15.86:0/240600267 wait complete. 2024-03-23T17:42:30.966 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:30.556+0000 7f2252c00700 1 Processor -- start 2024-03-23T17:42:30.966 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:30.556+0000 7f2252c00700 1 -- start start 2024-03-23T17:42:30.966 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:30.556+0000 7f2252c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f224c07ad90 0x7f224c19bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:42:30.966 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:30.556+0000 7f2252c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f224c19c060 con 0x7f224c07ad90 2024-03-23T17:42:30.966 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:30.557+0000 7f2251800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f224c07ad90 0x7f224c19bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:42:30.966 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:30.557+0000 7f2251800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f224c07ad90 0x7f224c19bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60412/0 (socket says 172.21.15.86:60412) 2024-03-23T17:42:30.966 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:30.557+0000 7f2251800700 1 -- 172.21.15.86:0/1097666302 learned_addr learned my addr 172.21.15.86:0/1097666302 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:42:30.966 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:30.557+0000 7f2251800700 1 -- 172.21.15.86:0/1097666302 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2240009550 con 0x7f224c07ad90 2024-03-23T17:42:30.966 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:30.557+0000 7f2251800700 1 --2- 172.21.15.86:0/1097666302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f224c07ad90 0x7f224c19bb20 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f224c07a260 tx=0x7f2240004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:42:30.966 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:30.557+0000 7f224a000700 1 -- 172.21.15.86:0/1097666302 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2240011630 con 0x7f224c07ad90 2024-03-23T17:42:30.966 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:30.558+0000 7f224a000700 1 -- 172.21.15.86:0/1097666302 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2240011790 con 0x7f224c07ad90 2024-03-23T17:42:30.966 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:30.558+0000 7f224a000700 1 -- 172.21.15.86:0/1097666302 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2240011a30 con 0x7f224c07ad90 2024-03-23T17:42:30.967 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:30.558+0000 7f2252c00700 1 -- 172.21.15.86:0/1097666302 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f224c19c260 con 0x7f224c07ad90 2024-03-23T17:42:30.967 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:30.558+0000 7f2252c00700 1 -- 172.21.15.86:0/1097666302 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f224c19c700 con 0x7f224c07ad90 2024-03-23T17:42:30.967 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:30.558+0000 7f224a000700 1 -- 172.21.15.86:0/1097666302 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f2240021070 con 0x7f224c07ad90 2024-03-23T17:42:30.967 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:30.558+0000 7f224a000700 1 -- 172.21.15.86:0/1097666302 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f224001b440 con 0x7f224c07ad90 2024-03-23T17:42:30.967 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:30.559+0000 7f2252c00700 1 -- 172.21.15.86:0/1097666302 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f224c1959b0 con 0x7f224c07ad90 2024-03-23T17:42:30.967 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:30.562+0000 7f224a000700 1 -- 172.21.15.86:0/1097666302 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f224001a460 con 0x7f224c07ad90 2024-03-23T17:42:30.967 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:30.633+0000 7f2252c00700 1 -- 172.21.15.86:0/1097666302 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f224c079f20 con 0x7f224c07ad90 2024-03-23T17:42:30.967 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:30.633+0000 7f224a000700 1 -- 172.21.15.86:0/1097666302 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7f224001a600 con 0x7f224c07ad90 2024-03-23T17:42:30.967 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:30.634+0000 7f2252c00700 1 -- 172.21.15.86:0/1097666302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f224c07ad90 msgr2=0x7f224c19bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:42:30.967 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:30.634+0000 7f2252c00700 1 --2- 172.21.15.86:0/1097666302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f224c07ad90 0x7f224c19bb20 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f224c07a260 tx=0x7f2240004750 comp rx=0 tx=0).stop 2024-03-23T17:42:30.967 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:30.634+0000 7f2252c00700 1 -- 172.21.15.86:0/1097666302 shutdown_connections 2024-03-23T17:42:30.967 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:30.634+0000 7f2252c00700 1 --2- 172.21.15.86:0/1097666302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f224c07ad90 0x7f224c19bb20 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:42:30.967 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:30.634+0000 7f2252c00700 1 -- 172.21.15.86:0/1097666302 >> 172.21.15.86:0/1097666302 conn(0x7f224c1019b0 msgr2=0x7f224c101d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:42:30.967 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:30.634+0000 7f2252c00700 1 -- 172.21.15.86:0/1097666302 shutdown_connections 2024-03-23T17:42:30.968 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:30.634+0000 7f2252c00700 1 -- 172.21.15.86:0/1097666302 wait complete. 2024-03-23T17:42:30.968 INFO:teuthology.orchestra.run.smithi086.stdout:mon is available 2024-03-23T17:42:30.968 INFO:teuthology.orchestra.run.smithi086.stdout:Assimilating anything we can from ceph.conf... 2024-03-23T17:42:31.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:31 smithi086 ceph-mon[37723]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-23T17:42:31.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:31 smithi086 ceph-mon[37723]: monmap e1: 1 mons at {a=[v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-23T17:42:31.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:31 smithi086 ceph-mon[37723]: fsmap 2024-03-23T17:42:31.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:31 smithi086 ceph-mon[37723]: osdmap e1: 0 total, 0 up, 0 in 2024-03-23T17:42:31.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:31 smithi086 ceph-mon[37723]: mgrmap e1: no daemons active 2024-03-23T17:42:31.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:31 smithi086 ceph-mon[37723]: from='client.? 172.21.15.86:0/1097666302' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-03-23T17:42:31.848 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-03-23T17:42:31.848 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout [global] 2024-03-23T17:42:31.848 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout fsid = 9d76c9e8-e93c-11ee-95cd-87774f69a715 2024-03-23T17:42:31.848 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.86:3300,v1:172.21.15.86:6789] 2024-03-23T17:42:31.848 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-23T17:42:31.848 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-23T17:42:31.848 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-23T17:42:31.848 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-23T17:42:31.849 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-03-23T17:42:31.849 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-23T17:42:31.849 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-23T17:42:31.849 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-03-23T17:42:31.849 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout [osd] 2024-03-23T17:42:31.849 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-23T17:42:31.849 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-23T17:42:31.849 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-23T17:42:31.849 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-23T17:42:31.849 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-23T17:42:31.849 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:31.392+0000 7f5b20e00700 1 Processor -- start 2024-03-23T17:42:31.849 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:31.392+0000 7f5b20e00700 1 -- start start 2024-03-23T17:42:31.849 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:31.392+0000 7f5b20e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b1c07adb0 0x7f5b1c079210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:42:31.849 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:31.392+0000 7f5b20e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1c0797e0 con 0x7f5b1c07adb0 2024-03-23T17:42:31.849 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:31.393+0000 7f5b1b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b1c07adb0 0x7f5b1c079210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:42:31.850 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:31.393+0000 7f5b1b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b1c07adb0 0x7f5b1c079210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60424/0 (socket says 172.21.15.86:60424) 2024-03-23T17:42:31.850 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:31.393+0000 7f5b1b400700 1 -- 172.21.15.86:0/3416191381 learned_addr learned my addr 172.21.15.86:0/3416191381 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:42:31.850 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:31.393+0000 7f5b1b400700 1 -- 172.21.15.86:0/3416191381 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b1c079920 con 0x7f5b1c07adb0 2024-03-23T17:42:31.850 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:31.393+0000 7f5b1b400700 1 --2- 172.21.15.86:0/3416191381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b1c07adb0 0x7f5b1c079210 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c0098a0 tx=0x7f5b0c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff54947f9674450b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:42:31.850 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:31.393+0000 7f5b1a000700 1 -- 172.21.15.86:0/3416191381 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5b0c004160 con 0x7f5b1c07adb0 2024-03-23T17:42:31.850 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:31.394+0000 7f5b1a000700 1 -- 172.21.15.86:0/3416191381 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b0c0042c0 con 0x7f5b1c07adb0 2024-03-23T17:42:31.850 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:31.394+0000 7f5b1a000700 1 -- 172.21.15.86:0/3416191381 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5b0c0045a0 con 0x7f5b1c07adb0 2024-03-23T17:42:31.850 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:31.394+0000 7f5b20e00700 1 -- 172.21.15.86:0/3416191381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b1c07adb0 msgr2=0x7f5b1c079210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:42:31.850 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:31.394+0000 7f5b20e00700 1 --2- 172.21.15.86:0/3416191381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b1c07adb0 0x7f5b1c079210 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c0098a0 tx=0x7f5b0c009bb0 comp rx=0 tx=0).stop 2024-03-23T17:42:31.850 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:31.394+0000 7f5b20e00700 1 -- 172.21.15.86:0/3416191381 shutdown_connections 2024-03-23T17:42:31.850 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:31.394+0000 7f5b20e00700 1 --2- 172.21.15.86:0/3416191381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b1c07adb0 0x7f5b1c079210 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:42:31.850 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:31.394+0000 7f5b20e00700 1 -- 172.21.15.86:0/3416191381 >> 172.21.15.86:0/3416191381 conn(0x7f5b1c1019b0 msgr2=0x7f5b1c103dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:42:31.851 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:31.394+0000 7f5b20e00700 1 -- 172.21.15.86:0/3416191381 shutdown_connections 2024-03-23T17:42:31.851 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:31.394+0000 7f5b20e00700 1 -- 172.21.15.86:0/3416191381 wait complete. 2024-03-23T17:42:31.851 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:31.396+0000 7f5b20e00700 1 Processor -- start 2024-03-23T17:42:31.851 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:31.396+0000 7f5b20e00700 1 -- start start 2024-03-23T17:42:31.851 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:31.396+0000 7f5b20e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b1c07adb0 0x7f5b1c19bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:42:31.851 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:31.396+0000 7f5b20e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1c19c090 con 0x7f5b1c07adb0 2024-03-23T17:42:31.851 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:31.396+0000 7f5b1b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b1c07adb0 0x7f5b1c19bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:42:31.851 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:31.397+0000 7f5b1b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b1c07adb0 0x7f5b1c19bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60430/0 (socket says 172.21.15.86:60430) 2024-03-23T17:42:31.851 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:31.397+0000 7f5b1b400700 1 -- 172.21.15.86:0/1688610801 learned_addr learned my addr 172.21.15.86:0/1688610801 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:42:31.851 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:31.397+0000 7f5b1b400700 1 -- 172.21.15.86:0/1688610801 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b0c009550 con 0x7f5b1c07adb0 2024-03-23T17:42:31.851 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:31.397+0000 7f5b1b400700 1 --2- 172.21.15.86:0/1688610801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b1c07adb0 0x7f5b1c19bb50 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c004000 tx=0x7f5b0c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:42:31.851 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:31.397+0000 7f5b13e00700 1 -- 172.21.15.86:0/1688610801 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5b0c011630 con 0x7f5b1c07adb0 2024-03-23T17:42:31.851 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:31.397+0000 7f5b13e00700 1 -- 172.21.15.86:0/1688610801 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b0c011790 con 0x7f5b1c07adb0 2024-03-23T17:42:31.852 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:31.397+0000 7f5b13e00700 1 -- 172.21.15.86:0/1688610801 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5b0c011a30 con 0x7f5b1c07adb0 2024-03-23T17:42:31.852 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:31.397+0000 7f5b20e00700 1 -- 172.21.15.86:0/1688610801 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b1c19c290 con 0x7f5b1c07adb0 2024-03-23T17:42:31.852 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:31.397+0000 7f5b20e00700 1 -- 172.21.15.86:0/1688610801 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b1c19c730 con 0x7f5b1c07adb0 2024-03-23T17:42:31.852 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:31.398+0000 7f5b13e00700 1 -- 172.21.15.86:0/1688610801 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f5b0c021070 con 0x7f5b1c07adb0 2024-03-23T17:42:31.852 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:31.398+0000 7f5b13e00700 1 -- 172.21.15.86:0/1688610801 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f5b0c01b440 con 0x7f5b1c07adb0 2024-03-23T17:42:31.852 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:31.399+0000 7f5b20e00700 1 -- 172.21.15.86:0/1688610801 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b1c195900 con 0x7f5b1c07adb0 2024-03-23T17:42:31.852 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:31.401+0000 7f5b13e00700 1 -- 172.21.15.86:0/1688610801 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f5b0c01a460 con 0x7f5b1c07adb0 2024-03-23T17:42:31.852 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:31.470+0000 7f5b20e00700 1 -- 172.21.15.86:0/1688610801 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f5b1c079ec0 con 0x7f5b1c07adb0 2024-03-23T17:42:31.852 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:31.475+0000 7f5b13e00700 1 -- 172.21.15.86:0/1688610801 <== mon.0 v2:172.21.15.86:3300/0 7 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5b0c01a600 con 0x7f5b1c07adb0 2024-03-23T17:42:31.852 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:31.475+0000 7f5b13e00700 1 -- 172.21.15.86:0/1688610801 <== mon.0 v2:172.21.15.86:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7f5b0c035590 con 0x7f5b1c07adb0 2024-03-23T17:42:31.852 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:31.476+0000 7f5b20e00700 1 -- 172.21.15.86:0/1688610801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b1c07adb0 msgr2=0x7f5b1c19bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:42:31.852 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:31.476+0000 7f5b20e00700 1 --2- 172.21.15.86:0/1688610801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b1c07adb0 0x7f5b1c19bb50 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c004000 tx=0x7f5b0c004750 comp rx=0 tx=0).stop 2024-03-23T17:42:31.852 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:31.476+0000 7f5b20e00700 1 -- 172.21.15.86:0/1688610801 shutdown_connections 2024-03-23T17:42:31.853 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:31.476+0000 7f5b20e00700 1 --2- 172.21.15.86:0/1688610801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b1c07adb0 0x7f5b1c19bb50 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:42:31.853 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:31.476+0000 7f5b20e00700 1 -- 172.21.15.86:0/1688610801 >> 172.21.15.86:0/1688610801 conn(0x7f5b1c1019b0 msgr2=0x7f5b1c103ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:42:31.853 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:31.477+0000 7f5b20e00700 1 -- 172.21.15.86:0/1688610801 shutdown_connections 2024-03-23T17:42:31.853 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:31.477+0000 7f5b20e00700 1 -- 172.21.15.86:0/1688610801 wait complete. 2024-03-23T17:42:31.853 INFO:teuthology.orchestra.run.smithi086.stdout:Generating new minimal ceph.conf... 2024-03-23T17:42:32.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:32 smithi086 ceph-mon[37723]: from='client.? 172.21.15.86:0/1688610801' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-23T17:42:32.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:32 smithi086 ceph-mon[37723]: from='client.? 172.21.15.86:0/1688610801' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-03-23T17:42:33.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:33 smithi086 ceph-mon[37723]: from='client.? 172.21.15.86:0/772497805' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:42:33.959 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:32.266+0000 7fa7ea000700 1 Processor -- start 2024-03-23T17:42:33.959 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:32.266+0000 7fa7ea000700 1 -- start start 2024-03-23T17:42:33.960 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:32.266+0000 7fa7ea000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7e41063d0 0x7fa7e41067f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:42:33.960 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:32.266+0000 7fa7ea000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e4106dc0 con 0x7fa7e41063d0 2024-03-23T17:42:33.960 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:32.266+0000 7fa7e8c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7e41063d0 0x7fa7e41067f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:42:33.960 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:32.266+0000 7fa7e8c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7e41063d0 0x7fa7e41067f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60434/0 (socket says 172.21.15.86:60434) 2024-03-23T17:42:33.960 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:32.266+0000 7fa7e8c00700 1 -- 172.21.15.86:0/216553009 learned_addr learned my addr 172.21.15.86:0/216553009 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:42:33.960 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:32.267+0000 7fa7e8c00700 1 -- 172.21.15.86:0/216553009 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7e41075d0 con 0x7fa7e41063d0 2024-03-23T17:42:33.960 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:32.267+0000 7fa7e8c00700 1 --2- 172.21.15.86:0/216553009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7e41063d0 0x7fa7e41067f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa7d80098a0 tx=0x7fa7d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e4e470d30fd87ad server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:42:33.960 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:32.267+0000 7fa7e3400700 1 -- 172.21.15.86:0/216553009 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa7d8004160 con 0x7fa7e41063d0 2024-03-23T17:42:33.960 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:32.267+0000 7fa7e3400700 1 -- 172.21.15.86:0/216553009 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa7d80042c0 con 0x7fa7e41063d0 2024-03-23T17:42:33.960 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:32.267+0000 7fa7e3400700 1 -- 172.21.15.86:0/216553009 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa7d8011a00 con 0x7fa7e41063d0 2024-03-23T17:42:33.960 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:32.268+0000 7fa7ea000700 1 -- 172.21.15.86:0/216553009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7e41063d0 msgr2=0x7fa7e41067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:42:33.961 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:32.268+0000 7fa7ea000700 1 --2- 172.21.15.86:0/216553009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7e41063d0 0x7fa7e41067f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa7d80098a0 tx=0x7fa7d8009bb0 comp rx=0 tx=0).stop 2024-03-23T17:42:33.961 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:32.268+0000 7fa7ea000700 1 -- 172.21.15.86:0/216553009 shutdown_connections 2024-03-23T17:42:33.961 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:32.268+0000 7fa7ea000700 1 --2- 172.21.15.86:0/216553009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7e41063d0 0x7fa7e41067f0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:42:33.961 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:32.268+0000 7fa7ea000700 1 -- 172.21.15.86:0/216553009 >> 172.21.15.86:0/216553009 conn(0x7fa7e41019b0 msgr2=0x7fa7e4103dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:42:33.961 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:32.268+0000 7fa7ea000700 1 -- 172.21.15.86:0/216553009 shutdown_connections 2024-03-23T17:42:33.961 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:32.268+0000 7fa7ea000700 1 -- 172.21.15.86:0/216553009 wait complete. 2024-03-23T17:42:33.961 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:32.269+0000 7fa7ea000700 1 Processor -- start 2024-03-23T17:42:33.961 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:32.269+0000 7fa7ea000700 1 -- start start 2024-03-23T17:42:33.961 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:32.269+0000 7fa7ea000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7e41063d0 0x7fa7e419c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:42:33.961 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:32.269+0000 7fa7ea000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e419c930 con 0x7fa7e41063d0 2024-03-23T17:42:33.961 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:32.270+0000 7fa7e8c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7e41063d0 0x7fa7e419c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:42:33.962 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:32.270+0000 7fa7e8c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7e41063d0 0x7fa7e419c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60436/0 (socket says 172.21.15.86:60436) 2024-03-23T17:42:33.962 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:32.270+0000 7fa7e8c00700 1 -- 172.21.15.86:0/772497805 learned_addr learned my addr 172.21.15.86:0/772497805 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:42:33.962 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:32.270+0000 7fa7e8c00700 1 -- 172.21.15.86:0/772497805 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7d8009550 con 0x7fa7e41063d0 2024-03-23T17:42:33.962 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:32.270+0000 7fa7e8c00700 1 --2- 172.21.15.86:0/772497805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7e41063d0 0x7fa7e419c3f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa7d8011710 tx=0x7fa7d8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:42:33.962 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:32.270+0000 7fa7e1600700 1 -- 172.21.15.86:0/772497805 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa7d8003aa0 con 0x7fa7e41063d0 2024-03-23T17:42:33.962 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:32.271+0000 7fa7e1600700 1 -- 172.21.15.86:0/772497805 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa7d8003c00 con 0x7fa7e41063d0 2024-03-23T17:42:33.962 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:32.271+0000 7fa7ea000700 1 -- 172.21.15.86:0/772497805 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7e419cb30 con 0x7fa7e41063d0 2024-03-23T17:42:33.962 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:32.271+0000 7fa7e1600700 1 -- 172.21.15.86:0/772497805 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa7d8024800 con 0x7fa7e41063d0 2024-03-23T17:42:33.962 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:32.271+0000 7fa7ea000700 1 -- 172.21.15.86:0/772497805 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7e419cfd0 con 0x7fa7e41063d0 2024-03-23T17:42:33.962 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:32.271+0000 7fa7e1600700 1 -- 172.21.15.86:0/772497805 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fa7d8024c60 con 0x7fa7e41063d0 2024-03-23T17:42:33.963 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:32.271+0000 7fa7e1600700 1 -- 172.21.15.86:0/772497805 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fa7d80193c0 con 0x7fa7e41063d0 2024-03-23T17:42:33.963 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:32.272+0000 7fa7ea000700 1 -- 172.21.15.86:0/772497805 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7e40623e0 con 0x7fa7e41063d0 2024-03-23T17:42:33.963 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:32.274+0000 7fa7e1600700 1 -- 172.21.15.86:0/772497805 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7fa7d801abd0 con 0x7fa7e41063d0 2024-03-23T17:42:33.963 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:32.344+0000 7fa7ea000700 1 -- 172.21.15.86:0/772497805 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fa7e410a800 con 0x7fa7e41063d0 2024-03-23T17:42:33.963 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:32.344+0000 7fa7e1600700 1 -- 172.21.15.86:0/772497805 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7fa7d801ad70 con 0x7fa7e41063d0 2024-03-23T17:42:33.963 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:32.345+0000 7fa7ea000700 1 -- 172.21.15.86:0/772497805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7e41063d0 msgr2=0x7fa7e419c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:42:33.963 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:32.345+0000 7fa7ea000700 1 --2- 172.21.15.86:0/772497805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7e41063d0 0x7fa7e419c3f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa7d8011710 tx=0x7fa7d8011ad0 comp rx=0 tx=0).stop 2024-03-23T17:42:33.963 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:32.345+0000 7fa7ea000700 1 -- 172.21.15.86:0/772497805 shutdown_connections 2024-03-23T17:42:33.963 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:32.345+0000 7fa7ea000700 1 --2- 172.21.15.86:0/772497805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7e41063d0 0x7fa7e419c3f0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:42:33.963 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:32.345+0000 7fa7ea000700 1 -- 172.21.15.86:0/772497805 >> 172.21.15.86:0/772497805 conn(0x7fa7e41019b0 msgr2=0x7fa7e4102580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:42:33.963 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:32.345+0000 7fa7ea000700 1 -- 172.21.15.86:0/772497805 shutdown_connections 2024-03-23T17:42:33.963 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:32.345+0000 7fa7ea000700 1 -- 172.21.15.86:0/772497805 wait complete. 2024-03-23T17:42:33.964 INFO:teuthology.orchestra.run.smithi086.stdout:Restarting the monitor... 2024-03-23T17:42:34.258 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:33 smithi086 systemd[1]: Stopping Ceph mon.a for 9d76c9e8-e93c-11ee-95cd-87774f69a715... 2024-03-23T17:42:34.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:34 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mon-a[37699]: 2024-03-23T17:42:34.256+0000 7fc097800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T17:42:34.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:34 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mon-a[37699]: 2024-03-23T17:42:34.256+0000 7fc097800700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-03-23T17:42:34.993 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:34 smithi086 bash[38059]: ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mon-a 2024-03-23T17:42:35.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 systemd[1]: ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@mon.a.service: Succeeded. 2024-03-23T17:42:35.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 systemd[1]: Stopped Ceph mon.a for 9d76c9e8-e93c-11ee-95cd-87774f69a715. 2024-03-23T17:42:35.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 systemd[1]: Starting Ceph mon.a for 9d76c9e8-e93c-11ee-95cd-87774f69a715... 2024-03-23T17:42:35.456 INFO:teuthology.orchestra.run.smithi086.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-03-23T17:42:35.768 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 bash[38182]: 37a05cedce31d1c2b90ef577d9faf143a58fa87062f19219c3cd6ea29d5a282b 2024-03-23T17:42:35.768 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 systemd[1]: Started Ceph mon.a for 9d76c9e8-e93c-11ee-95cd-87774f69a715. 2024-03-23T17:42:35.768 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: set uid:gid to 167:167 (ceph:ceph) 2024-03-23T17:42:35.768 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable), process ceph-mon, pid 7 2024-03-23T17:42:35.768 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: pidfile_write: ignore empty --pid-file 2024-03-23T17:42:35.768 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: load: jerasure load: lrc 2024-03-23T17:42:35.768 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: RocksDB version: 7.9.2 2024-03-23T17:42:35.768 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Git sha 0 2024-03-23T17:42:35.768 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Compile date 2024-03-21 21:38:23 2024-03-23T17:42:35.769 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: DB SUMMARY 2024-03-23T17:42:35.769 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: DB Session ID: CIESKYP9OLNUJOEEVBAW 2024-03-23T17:42:35.769 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: CURRENT file: CURRENT 2024-03-23T17:42:35.769 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: IDENTITY file: IDENTITY 2024-03-23T17:42:35.769 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-03-23T17:42:35.769 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-03-23T17:42:35.769 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 99631 ; 2024-03-23T17:42:35.769 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.error_if_exists: 0 2024-03-23T17:42:35.769 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.create_if_missing: 0 2024-03-23T17:42:35.769 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.paranoid_checks: 1 2024-03-23T17:42:35.769 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-23T17:42:35.769 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-23T17:42:35.769 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-23T17:42:35.769 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.env: 0x562317b2d720 2024-03-23T17:42:35.769 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.fs: PosixFileSystem 2024-03-23T17:42:35.770 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.info_log: 0x5623199e5520 2024-03-23T17:42:35.770 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.max_file_opening_threads: 16 2024-03-23T17:42:35.770 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.statistics: (nil) 2024-03-23T17:42:35.770 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.use_fsync: 0 2024-03-23T17:42:35.770 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.max_log_file_size: 0 2024-03-23T17:42:35.770 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-23T17:42:35.770 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-23T17:42:35.770 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.keep_log_file_num: 1000 2024-03-23T17:42:35.770 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.recycle_log_file_num: 0 2024-03-23T17:42:35.770 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.allow_fallocate: 1 2024-03-23T17:42:35.770 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.allow_mmap_reads: 0 2024-03-23T17:42:35.770 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.allow_mmap_writes: 0 2024-03-23T17:42:35.770 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.use_direct_reads: 0 2024-03-23T17:42:35.771 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-23T17:42:35.771 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.create_missing_column_families: 0 2024-03-23T17:42:35.771 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.db_log_dir: 2024-03-23T17:42:35.771 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.wal_dir: 2024-03-23T17:42:35.771 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-23T17:42:35.771 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-23T17:42:35.771 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-23T17:42:35.771 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-23T17:42:35.771 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-23T17:42:35.771 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-23T17:42:35.771 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.advise_random_on_open: 1 2024-03-23T17:42:35.771 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.db_write_buffer_size: 0 2024-03-23T17:42:35.771 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.write_buffer_manager: 0x562318c7c320 2024-03-23T17:42:35.771 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-23T17:42:35.772 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-23T17:42:35.772 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-23T17:42:35.772 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.rate_limiter: (nil) 2024-03-23T17:42:35.772 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-23T17:42:35.772 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.wal_recovery_mode: 2 2024-03-23T17:42:35.772 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.enable_thread_tracking: 0 2024-03-23T17:42:35.772 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.enable_pipelined_write: 0 2024-03-23T17:42:35.772 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.unordered_write: 0 2024-03-23T17:42:35.772 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-23T17:42:35.772 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-23T17:42:35.772 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-23T17:42:35.772 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-23T17:42:35.772 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.row_cache: None 2024-03-23T17:42:35.772 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.wal_filter: None 2024-03-23T17:42:35.773 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-23T17:42:35.773 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.allow_ingest_behind: 0 2024-03-23T17:42:35.773 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.two_write_queues: 0 2024-03-23T17:42:35.773 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.manual_wal_flush: 0 2024-03-23T17:42:35.773 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.wal_compression: 0 2024-03-23T17:42:35.773 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.atomic_flush: 0 2024-03-23T17:42:35.773 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-23T17:42:35.773 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-23T17:42:35.773 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-23T17:42:35.773 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.log_readahead_size: 0 2024-03-23T17:42:35.773 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-23T17:42:35.773 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.best_efforts_recovery: 0 2024-03-23T17:42:35.773 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-23T17:42:35.773 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-23T17:42:35.773 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.allow_data_in_errors: 0 2024-03-23T17:42:35.774 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.db_host_id: __hostname__ 2024-03-23T17:42:35.774 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-23T17:42:35.774 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.max_background_jobs: 2 2024-03-23T17:42:35.774 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.max_background_compactions: -1 2024-03-23T17:42:35.774 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.max_subcompactions: 1 2024-03-23T17:42:35.774 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-23T17:42:35.774 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-23T17:42:35.774 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-23T17:42:35.774 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.max_total_wal_size: 0 2024-03-23T17:42:35.774 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-23T17:42:35.774 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-23T17:42:35.774 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-23T17:42:35.774 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-23T17:42:35.775 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.max_open_files: -1 2024-03-23T17:42:35.775 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.bytes_per_sync: 0 2024-03-23T17:42:35.775 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-23T17:42:35.775 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-23T17:42:35.775 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.compaction_readahead_size: 0 2024-03-23T17:42:35.775 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.max_background_flushes: -1 2024-03-23T17:42:35.775 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Compression algorithms supported: 2024-03-23T17:42:35.775 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-23T17:42:35.775 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: kZSTD supported: 0 2024-03-23T17:42:35.775 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: kXpressCompression supported: 0 2024-03-23T17:42:35.775 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: kLZ4HCCompression supported: 1 2024-03-23T17:42:35.775 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: kZlibCompression supported: 1 2024-03-23T17:42:35.775 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: kSnappyCompression supported: 1 2024-03-23T17:42:35.775 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: kLZ4Compression supported: 1 2024-03-23T17:42:35.775 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: kBZip2Compression supported: 0 2024-03-23T17:42:35.776 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-23T17:42:35.776 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-23T17:42:35.776 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-03-23T17:42:35.776 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-23T17:42:35.776 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-23T17:42:35.776 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.merge_operator: 2024-03-23T17:42:35.776 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.compaction_filter: None 2024-03-23T17:42:35.776 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.compaction_filter_factory: None 2024-03-23T17:42:35.776 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.sst_partitioner_factory: None 2024-03-23T17:42:35.776 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-23T17:42:35.776 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-23T17:42:35.776 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5623199e5620) 2024-03-23T17:42:35.776 INFO:journalctl@ceph.mon.a.smithi086.stdout: cache_index_and_filter_blocks: 1 2024-03-23T17:42:35.776 INFO:journalctl@ceph.mon.a.smithi086.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-23T17:42:35.777 INFO:journalctl@ceph.mon.a.smithi086.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-23T17:42:35.777 INFO:journalctl@ceph.mon.a.smithi086.stdout: pin_top_level_index_and_filter: 1 2024-03-23T17:42:35.777 INFO:journalctl@ceph.mon.a.smithi086.stdout: index_type: 0 2024-03-23T17:42:35.777 INFO:journalctl@ceph.mon.a.smithi086.stdout: data_block_index_type: 0 2024-03-23T17:42:35.777 INFO:journalctl@ceph.mon.a.smithi086.stdout: index_shortening: 1 2024-03-23T17:42:35.777 INFO:journalctl@ceph.mon.a.smithi086.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-23T17:42:35.777 INFO:journalctl@ceph.mon.a.smithi086.stdout: checksum: 4 2024-03-23T17:42:35.777 INFO:journalctl@ceph.mon.a.smithi086.stdout: no_block_cache: 0 2024-03-23T17:42:35.777 INFO:journalctl@ceph.mon.a.smithi086.stdout: block_cache: 0x562318cfd1f0 2024-03-23T17:42:35.777 INFO:journalctl@ceph.mon.a.smithi086.stdout: block_cache_name: BinnedLRUCache 2024-03-23T17:42:35.777 INFO:journalctl@ceph.mon.a.smithi086.stdout: block_cache_options: 2024-03-23T17:42:35.777 INFO:journalctl@ceph.mon.a.smithi086.stdout: capacity : 536870912 2024-03-23T17:42:35.777 INFO:journalctl@ceph.mon.a.smithi086.stdout: num_shard_bits : 4 2024-03-23T17:42:35.777 INFO:journalctl@ceph.mon.a.smithi086.stdout: strict_capacity_limit : 0 2024-03-23T17:42:35.778 INFO:journalctl@ceph.mon.a.smithi086.stdout: high_pri_pool_ratio: 0.000 2024-03-23T17:42:35.778 INFO:journalctl@ceph.mon.a.smithi086.stdout: block_cache_compressed: (nil) 2024-03-23T17:42:35.778 INFO:journalctl@ceph.mon.a.smithi086.stdout: persistent_cache: (nil) 2024-03-23T17:42:35.778 INFO:journalctl@ceph.mon.a.smithi086.stdout: block_size: 4096 2024-03-23T17:42:35.778 INFO:journalctl@ceph.mon.a.smithi086.stdout: block_size_deviation: 10 2024-03-23T17:42:35.778 INFO:journalctl@ceph.mon.a.smithi086.stdout: block_restart_interval: 16 2024-03-23T17:42:35.778 INFO:journalctl@ceph.mon.a.smithi086.stdout: index_block_restart_interval: 1 2024-03-23T17:42:35.778 INFO:journalctl@ceph.mon.a.smithi086.stdout: metadata_block_size: 4096 2024-03-23T17:42:35.778 INFO:journalctl@ceph.mon.a.smithi086.stdout: partition_filters: 0 2024-03-23T17:42:35.778 INFO:journalctl@ceph.mon.a.smithi086.stdout: use_delta_encoding: 1 2024-03-23T17:42:35.778 INFO:journalctl@ceph.mon.a.smithi086.stdout: filter_policy: bloomfilter 2024-03-23T17:42:35.778 INFO:journalctl@ceph.mon.a.smithi086.stdout: whole_key_filtering: 1 2024-03-23T17:42:35.778 INFO:journalctl@ceph.mon.a.smithi086.stdout: verify_compression: 0 2024-03-23T17:42:35.778 INFO:journalctl@ceph.mon.a.smithi086.stdout: read_amp_bytes_per_bit: 0 2024-03-23T17:42:35.778 INFO:journalctl@ceph.mon.a.smithi086.stdout: format_version: 5 2024-03-23T17:42:35.779 INFO:journalctl@ceph.mon.a.smithi086.stdout: enable_index_compression: 1 2024-03-23T17:42:35.779 INFO:journalctl@ceph.mon.a.smithi086.stdout: block_align: 0 2024-03-23T17:42:35.779 INFO:journalctl@ceph.mon.a.smithi086.stdout: max_auto_readahead_size: 262144 2024-03-23T17:42:35.779 INFO:journalctl@ceph.mon.a.smithi086.stdout: prepopulate_block_cache: 0 2024-03-23T17:42:35.779 INFO:journalctl@ceph.mon.a.smithi086.stdout: initial_auto_readahead_size: 8192 2024-03-23T17:42:35.779 INFO:journalctl@ceph.mon.a.smithi086.stdout: num_file_reads_for_auto_readahead: 2 2024-03-23T17:42:35.779 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.write_buffer_size: 33554432 2024-03-23T17:42:35.779 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.max_write_buffer_number: 2 2024-03-23T17:42:35.779 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.compression: NoCompression 2024-03-23T17:42:35.779 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.bottommost_compression: Disabled 2024-03-23T17:42:35.779 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.prefix_extractor: nullptr 2024-03-23T17:42:35.779 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-23T17:42:35.779 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.num_levels: 7 2024-03-23T17:42:35.779 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-23T17:42:35.780 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-23T17:42:35.780 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-23T17:42:35.780 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-23T17:42:35.780 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-23T17:42:35.780 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-23T17:42:35.780 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-23T17:42:35.780 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-23T17:42:35.780 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-23T17:42:35.780 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-23T17:42:35.780 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-23T17:42:35.780 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-23T17:42:35.780 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-23T17:42:35.780 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.compression_opts.level: 32767 2024-03-23T17:42:35.780 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.compression_opts.strategy: 0 2024-03-23T17:42:35.781 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-23T17:42:35.781 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-23T17:42:35.781 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-23T17:42:35.781 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-23T17:42:35.781 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.compression_opts.enabled: false 2024-03-23T17:42:35.781 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-23T17:42:35.781 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-23T17:42:35.781 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-23T17:42:35.781 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-23T17:42:35.781 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.target_file_size_base: 67108864 2024-03-23T17:42:35.781 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-23T17:42:35.781 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-23T17:42:35.781 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-23T17:42:35.781 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-23T17:42:35.782 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-23T17:42:35.782 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-23T17:42:35.782 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-23T17:42:35.782 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-23T17:42:35.782 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-23T17:42:35.782 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-23T17:42:35.782 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-23T17:42:35.782 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-23T17:42:35.782 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-23T17:42:35.782 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-23T17:42:35.782 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.arena_block_size: 1048576 2024-03-23T17:42:35.782 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-23T17:42:35.782 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-23T17:42:35.782 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.disable_auto_compactions: 0 2024-03-23T17:42:35.782 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-23T17:42:35.783 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-23T17:42:35.783 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-23T17:42:35.783 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-23T17:42:35.784 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-23T17:42:35.784 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-23T17:42:35.784 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-23T17:42:35.784 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-23T17:42:35.784 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-23T17:42:35.784 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-23T17:42:35.784 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.table_properties_collectors: 2024-03-23T17:42:35.784 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.inplace_update_support: 0 2024-03-23T17:42:35.784 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-23T17:42:35.784 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-23T17:42:35.784 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-23T17:42:35.784 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-23T17:42:35.784 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.bloom_locality: 0 2024-03-23T17:42:35.784 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.max_successive_merges: 0 2024-03-23T17:42:35.784 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-23T17:42:35.785 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.paranoid_file_checks: 0 2024-03-23T17:42:35.785 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.force_consistency_checks: 1 2024-03-23T17:42:35.785 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.report_bg_io_stats: 0 2024-03-23T17:42:35.785 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.ttl: 2592000 2024-03-23T17:42:35.785 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-23T17:42:35.785 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-23T17:42:35.785 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-23T17:42:35.785 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.enable_blob_files: false 2024-03-23T17:42:35.785 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.min_blob_size: 0 2024-03-23T17:42:35.785 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.blob_file_size: 268435456 2024-03-23T17:42:35.785 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-23T17:42:35.785 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-23T17:42:35.785 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-23T17:42:35.785 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-23T17:42:35.786 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-23T17:42:35.786 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.blob_file_starting_level: 0 2024-03-23T17:42:35.786 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-23T17:42:35.786 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-03-23T17:42:35.786 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-03-23T17:42:35.786 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 5c6727c4-5615-4c97-81a9-0ba33396d919 2024-03-23T17:42:35.786 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711215755473883, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-03-23T17:42:35.786 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-03-23T17:42:35.786 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711215755477130, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 94794, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 318, "table_properties": {"data_size": 92785, "index_size": 291, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13858, "raw_average_key_size": 50, "raw_value_size": 85244, "raw_average_value_size": 307, "num_data_blocks": 11, "num_entries": 277, "num_filter_entries": 277, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711215755, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "5c6727c4-5615-4c97-81a9-0ba33396d919", "db_session_id": "CIESKYP9OLNUJOEEVBAW", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-03-23T17:42:35.786 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711215755477321, "job": 1, "event": "recovery_finished"} 2024-03-23T17:42:35.786 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-03-23T17:42:35.786 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-23T17:42:35.786 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x562318d9a000 2024-03-23T17:42:35.787 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: DB pointer 0x562318d88000 2024-03-23T17:42:35.787 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-23T17:42:35.787 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-23T17:42:35.787 INFO:journalctl@ceph.mon.a.smithi086.stdout: ** DB Stats ** 2024-03-23T17:42:35.787 INFO:journalctl@ceph.mon.a.smithi086.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-23T17:42:35.787 INFO:journalctl@ceph.mon.a.smithi086.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-23T17:42:35.787 INFO:journalctl@ceph.mon.a.smithi086.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-23T17:42:35.787 INFO:journalctl@ceph.mon.a.smithi086.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-23T17:42:35.787 INFO:journalctl@ceph.mon.a.smithi086.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-23T17:42:35.787 INFO:journalctl@ceph.mon.a.smithi086.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-23T17:42:35.787 INFO:journalctl@ceph.mon.a.smithi086.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-23T17:42:35.787 INFO:journalctl@ceph.mon.a.smithi086.stdout: 2024-03-23T17:42:35.787 INFO:journalctl@ceph.mon.a.smithi086.stdout: ** Compaction Stats [default] ** 2024-03-23T17:42:35.787 INFO:journalctl@ceph.mon.a.smithi086.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-23T17:42:35.788 INFO:journalctl@ceph.mon.a.smithi086.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-23T17:42:35.788 INFO:journalctl@ceph.mon.a.smithi086.stdout: L0 2/0 94.41 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 43.6 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-03-23T17:42:35.788 INFO:journalctl@ceph.mon.a.smithi086.stdout: Sum 2/0 94.41 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 43.6 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-03-23T17:42:35.788 INFO:journalctl@ceph.mon.a.smithi086.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 43.6 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-03-23T17:42:35.788 INFO:journalctl@ceph.mon.a.smithi086.stdout: 2024-03-23T17:42:35.788 INFO:journalctl@ceph.mon.a.smithi086.stdout: ** Compaction Stats [default] ** 2024-03-23T17:42:35.788 INFO:journalctl@ceph.mon.a.smithi086.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-23T17:42:35.788 INFO:journalctl@ceph.mon.a.smithi086.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-23T17:42:35.788 INFO:journalctl@ceph.mon.a.smithi086.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 43.6 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-03-23T17:42:35.788 INFO:journalctl@ceph.mon.a.smithi086.stdout: 2024-03-23T17:42:35.788 INFO:journalctl@ceph.mon.a.smithi086.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-23T17:42:35.788 INFO:journalctl@ceph.mon.a.smithi086.stdout: 2024-03-23T17:42:35.788 INFO:journalctl@ceph.mon.a.smithi086.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-23T17:42:35.788 INFO:journalctl@ceph.mon.a.smithi086.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-23T17:42:35.788 INFO:journalctl@ceph.mon.a.smithi086.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-23T17:42:35.789 INFO:journalctl@ceph.mon.a.smithi086.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-23T17:42:35.789 INFO:journalctl@ceph.mon.a.smithi086.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-23T17:42:35.789 INFO:journalctl@ceph.mon.a.smithi086.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-23T17:42:35.789 INFO:journalctl@ceph.mon.a.smithi086.stdout: Cumulative compaction: 0.00 GB write, 9.59 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-23T17:42:35.789 INFO:journalctl@ceph.mon.a.smithi086.stdout: Interval compaction: 0.00 GB write, 9.59 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-23T17:42:35.789 INFO:journalctl@ceph.mon.a.smithi086.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-23T17:42:35.789 INFO:journalctl@ceph.mon.a.smithi086.stdout: Block cache BinnedLRUCache@0x562318cfd1f0#7 capacity: 512.00 MB usage: 1.39 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-03-23T17:42:35.789 INFO:journalctl@ceph.mon.a.smithi086.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-03-23T17:42:35.789 INFO:journalctl@ceph.mon.a.smithi086.stdout: 2024-03-23T17:42:35.789 INFO:journalctl@ceph.mon.a.smithi086.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-23T17:42:35.789 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: starting mon.a rank 0 at public addrs [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] at bind addrs [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 2024-03-23T17:42:35.789 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: mon.a@-1(???) e1 preinit fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 2024-03-23T17:42:35.789 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: mon.a@-1(???).mds e1 new map 2024-03-23T17:42:35.789 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: mon.a@-1(???).mds e1 print_map 2024-03-23T17:42:35.789 INFO:journalctl@ceph.mon.a.smithi086.stdout: e1 2024-03-23T17:42:35.790 INFO:journalctl@ceph.mon.a.smithi086.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-23T17:42:35.790 INFO:journalctl@ceph.mon.a.smithi086.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-23T17:42:35.790 INFO:journalctl@ceph.mon.a.smithi086.stdout: legacy client fscid: -1 2024-03-23T17:42:35.790 INFO:journalctl@ceph.mon.a.smithi086.stdout: 2024-03-23T17:42:35.790 INFO:journalctl@ceph.mon.a.smithi086.stdout: No filesystems configured 2024-03-23T17:42:35.790 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-23T17:42:35.790 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-23T17:42:35.790 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-23T17:42:35.790 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-23T17:42:35.790 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-03-23T17:42:35.790 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-23T17:42:35.790 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: monmap e1: 1 mons at {a=[v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-23T17:42:35.790 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: fsmap 2024-03-23T17:42:35.790 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: osdmap e1: 0 total, 0 up, 0 in 2024-03-23T17:42:35.790 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:35 smithi086 ceph-mon[38221]: mgrmap e1: no daemons active 2024-03-23T17:42:36.328 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:35.894+0000 7f13e6400700 1 Processor -- start 2024-03-23T17:42:36.328 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:35.894+0000 7f13e6400700 1 -- start start 2024-03-23T17:42:36.328 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:35.895+0000 7f13e6400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13e01065b0 0x7f13e01069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:42:36.329 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:35.895+0000 7f13e6400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e0106fa0 con 0x7f13e01065b0 2024-03-23T17:42:36.329 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:35.895+0000 7f13df400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13e01065b0 0x7f13e01069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:42:36.329 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:35.895+0000 7f13df400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13e01065b0 0x7f13e01069d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52726/0 (socket says 172.21.15.86:52726) 2024-03-23T17:42:36.329 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:35.896+0000 7f13df400700 1 -- 172.21.15.86:0/1173222135 learned_addr learned my addr 172.21.15.86:0/1173222135 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:42:36.329 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:35.896+0000 7f13df400700 1 -- 172.21.15.86:0/1173222135 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13e01077b0 con 0x7f13e01065b0 2024-03-23T17:42:36.329 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:35.896+0000 7f13df400700 1 --2- 172.21.15.86:0/1173222135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13e01065b0 0x7f13e01069d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f13d00098a0 tx=0x7f13d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea2744f3c7a64b34 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:42:36.329 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:35.896+0000 7f13dea00700 1 -- 172.21.15.86:0/1173222135 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f13d0004160 con 0x7f13e01065b0 2024-03-23T17:42:36.329 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:35.896+0000 7f13dea00700 1 -- 172.21.15.86:0/1173222135 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f13d00042c0 con 0x7f13e01065b0 2024-03-23T17:42:36.329 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:35.896+0000 7f13dea00700 1 -- 172.21.15.86:0/1173222135 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f13d0011a00 con 0x7f13e01065b0 2024-03-23T17:42:36.329 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:35.897+0000 7f13e6400700 1 -- 172.21.15.86:0/1173222135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13e01065b0 msgr2=0x7f13e01069d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:42:36.329 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:35.897+0000 7f13e6400700 1 --2- 172.21.15.86:0/1173222135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13e01065b0 0x7f13e01069d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f13d00098a0 tx=0x7f13d0009bb0 comp rx=0 tx=0).stop 2024-03-23T17:42:36.329 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:35.897+0000 7f13e6400700 1 -- 172.21.15.86:0/1173222135 shutdown_connections 2024-03-23T17:42:36.329 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:35.897+0000 7f13e6400700 1 --2- 172.21.15.86:0/1173222135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13e01065b0 0x7f13e01069d0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:42:36.329 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:35.897+0000 7f13e6400700 1 -- 172.21.15.86:0/1173222135 >> 172.21.15.86:0/1173222135 conn(0x7f13e0101b90 msgr2=0x7f13e0103fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:42:36.330 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:35.897+0000 7f13e6400700 1 -- 172.21.15.86:0/1173222135 shutdown_connections 2024-03-23T17:42:36.330 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:35.897+0000 7f13e6400700 1 -- 172.21.15.86:0/1173222135 wait complete. 2024-03-23T17:42:36.330 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:35.898+0000 7f13e6400700 1 Processor -- start 2024-03-23T17:42:36.330 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:35.898+0000 7f13e6400700 1 -- start start 2024-03-23T17:42:36.330 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:35.899+0000 7f13e6400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13e01065b0 0x7f13e019c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:42:36.330 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:35.899+0000 7f13e6400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e019ca60 con 0x7f13e01065b0 2024-03-23T17:42:36.330 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:35.899+0000 7f13df400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13e01065b0 0x7f13e019c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:42:36.330 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:35.899+0000 7f13df400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13e01065b0 0x7f13e019c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52734/0 (socket says 172.21.15.86:52734) 2024-03-23T17:42:36.330 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:35.899+0000 7f13df400700 1 -- 172.21.15.86:0/3221548847 learned_addr learned my addr 172.21.15.86:0/3221548847 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:42:36.330 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:35.899+0000 7f13df400700 1 -- 172.21.15.86:0/3221548847 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d0009550 con 0x7f13e01065b0 2024-03-23T17:42:36.330 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:35.900+0000 7f13df400700 1 --2- 172.21.15.86:0/3221548847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13e01065b0 0x7f13e019c520 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f13d0009fb0 tx=0x7f13d0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:42:36.330 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:35.900+0000 7f13dcc00700 1 -- 172.21.15.86:0/3221548847 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f13d0003a40 con 0x7f13e01065b0 2024-03-23T17:42:36.330 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:35.900+0000 7f13dcc00700 1 -- 172.21.15.86:0/3221548847 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f13d0003ba0 con 0x7f13e01065b0 2024-03-23T17:42:36.331 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:35.900+0000 7f13e6400700 1 -- 172.21.15.86:0/3221548847 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13e019cc60 con 0x7f13e01065b0 2024-03-23T17:42:36.331 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:35.900+0000 7f13dcc00700 1 -- 172.21.15.86:0/3221548847 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f13d001a750 con 0x7f13e01065b0 2024-03-23T17:42:36.331 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:35.900+0000 7f13e6400700 1 -- 172.21.15.86:0/3221548847 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13e019d100 con 0x7f13e01065b0 2024-03-23T17:42:36.331 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:35.900+0000 7f13dcc00700 1 -- 172.21.15.86:0/3221548847 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f13d001acd0 con 0x7f13e01065b0 2024-03-23T17:42:36.331 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:35.900+0000 7f13dcc00700 1 -- 172.21.15.86:0/3221548847 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f13d00193c0 con 0x7f13e01065b0 2024-03-23T17:42:36.331 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:35.901+0000 7f13e6400700 1 -- 172.21.15.86:0/3221548847 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13e00623e0 con 0x7f13e01065b0 2024-03-23T17:42:36.331 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:35.904+0000 7f13dcc00700 1 -- 172.21.15.86:0/3221548847 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f13d002de40 con 0x7f13e01065b0 2024-03-23T17:42:36.331 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:35.973+0000 7f13e6400700 1 -- 172.21.15.86:0/3221548847 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f13e019f990 con 0x7f13e01065b0 2024-03-23T17:42:36.331 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:35.975+0000 7f13dcc00700 1 -- 172.21.15.86:0/3221548847 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f13d0024b60 con 0x7f13e01065b0 2024-03-23T17:42:36.331 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:35.976+0000 7f13e6400700 1 -- 172.21.15.86:0/3221548847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13e01065b0 msgr2=0x7f13e019c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:42:36.331 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:35.976+0000 7f13e6400700 1 --2- 172.21.15.86:0/3221548847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13e01065b0 0x7f13e019c520 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f13d0009fb0 tx=0x7f13d0011ad0 comp rx=0 tx=0).stop 2024-03-23T17:42:36.331 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:35.976+0000 7f13e6400700 1 -- 172.21.15.86:0/3221548847 shutdown_connections 2024-03-23T17:42:36.331 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:35.976+0000 7f13e6400700 1 --2- 172.21.15.86:0/3221548847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13e01065b0 0x7f13e019c520 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:42:36.332 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:35.976+0000 7f13e6400700 1 -- 172.21.15.86:0/3221548847 >> 172.21.15.86:0/3221548847 conn(0x7f13e0101b90 msgr2=0x7f13e0102690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:42:36.332 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:35.976+0000 7f13e6400700 1 -- 172.21.15.86:0/3221548847 shutdown_connections 2024-03-23T17:42:36.332 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:35.976+0000 7f13e6400700 1 -- 172.21.15.86:0/3221548847 wait complete. 2024-03-23T17:42:36.332 INFO:teuthology.orchestra.run.smithi086.stdout:Wrote config to /etc/ceph/ceph.conf 2024-03-23T17:42:36.355 INFO:teuthology.orchestra.run.smithi086.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-03-23T17:42:36.356 INFO:teuthology.orchestra.run.smithi086.stdout:Creating mgr... 2024-03-23T17:42:36.356 INFO:teuthology.orchestra.run.smithi086.stdout:Verifying port 0.0.0.0:9283 ... 2024-03-23T17:42:36.356 INFO:teuthology.orchestra.run.smithi086.stdout:Verifying port 0.0.0.0:8765 ... 2024-03-23T17:42:36.734 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@mgr.x 2024-03-23T17:42:36.734 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stderr Failed to reset failed state of unit ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@mgr.x.service: Unit ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@mgr.x.service not loaded. 2024-03-23T17:42:36.929 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715.target.wants/ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@mgr.x.service → /etc/systemd/system/ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@.service. 2024-03-23T17:42:37.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:36 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/3221548847' entity='client.admin' 2024-03-23T17:42:37.017 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:42:36 smithi086 systemd[1]: Starting Ceph mgr.x for 9d76c9e8-e93c-11ee-95cd-87774f69a715... 2024-03-23T17:42:37.474 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-23T17:42:37.474 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout disabled 2024-03-23T17:42:37.489 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-23T17:42:37.489 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout inactive 2024-03-23T17:42:37.489 INFO:teuthology.orchestra.run.smithi086.stdout:firewalld.service is not enabled 2024-03-23T17:42:37.489 INFO:teuthology.orchestra.run.smithi086.stdout:Not possible to enable service . firewalld.service is not available 2024-03-23T17:42:37.506 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-23T17:42:37.506 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout disabled 2024-03-23T17:42:37.521 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-23T17:42:37.521 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout inactive 2024-03-23T17:42:37.522 INFO:teuthology.orchestra.run.smithi086.stdout:firewalld.service is not enabled 2024-03-23T17:42:37.522 INFO:teuthology.orchestra.run.smithi086.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-03-23T17:42:37.522 INFO:teuthology.orchestra.run.smithi086.stdout:Waiting for mgr to start... 2024-03-23T17:42:37.522 INFO:teuthology.orchestra.run.smithi086.stdout:Waiting for mgr... 2024-03-23T17:42:37.707 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:42:37 smithi086 bash[38457]: 9aeda55ba8073bd64db153c55478f51442f7cb1aff9cca4459cffea04648f3ee 2024-03-23T17:42:37.707 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:42:37 smithi086 systemd[1]: Started Ceph mgr.x for 9d76c9e8-e93c-11ee-95cd-87774f69a715. 2024-03-23T17:42:38.232 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:38 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/2963957055' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-23T17:42:38.232 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:42:37 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:42:37.966+0000 7faade567200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-23T17:42:38.232 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:42:38 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:42:38.092+0000 7faade567200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-23T17:42:38.232 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:42:38 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:42:38.214+0000 7faade567200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-23T17:42:38.369 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-03-23T17:42:38.372 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout { 2024-03-23T17:42:38.372 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "fsid": "9d76c9e8-e93c-11ee-95cd-87774f69a715", 2024-03-23T17:42:38.372 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "health": { 2024-03-23T17:42:38.372 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-23T17:42:38.372 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-23T17:42:38.372 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-23T17:42:38.372 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-03-23T17:42:38.372 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-23T17:42:38.372 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-23T17:42:38.372 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 0 2024-03-23T17:42:38.372 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ], 2024-03-23T17:42:38.372 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-23T17:42:38.372 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "a" 2024-03-23T17:42:38.372 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ], 2024-03-23T17:42:38.372 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-03-23T17:42:38.373 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-23T17:42:38.373 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-23T17:42:38.373 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-03-23T17:42:38.373 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-23T17:42:38.373 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-03-23T17:42:38.373 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-23T17:42:38.373 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-23T17:42:38.373 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-23T17:42:38.373 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-23T17:42:38.381 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-23T17:42:38.381 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-23T17:42:38.381 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-23T17:42:38.381 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-23T17:42:38.381 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-03-23T17:42:38.381 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-23T17:42:38.381 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-23T17:42:38.381 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-23T17:42:38.382 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-23T17:42:38.382 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-23T17:42:38.382 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-23T17:42:38.382 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-23T17:42:38.382 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-23T17:42:38.382 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-23T17:42:38.382 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-03-23T17:42:38.382 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-23T17:42:38.382 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-23T17:42:38.382 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-23T17:42:38.382 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-23T17:42:38.383 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-03-23T17:42:38.383 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-23T17:42:38.383 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-23T17:42:38.383 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-23T17:42:38.383 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-23T17:42:38.383 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-23T17:42:38.383 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-23T17:42:38.383 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "restful" 2024-03-23T17:42:38.383 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ], 2024-03-23T17:42:38.383 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-23T17:42:38.386 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-03-23T17:42:38.386 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-23T17:42:38.386 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-23T17:42:38.386 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "modified": "2024-03-23T17:42:30.098080+0000", 2024-03-23T17:42:38.386 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-23T17:42:38.386 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-03-23T17:42:38.386 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-23T17:42:38.386 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout } 2024-03-23T17:42:38.386 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:37.936+0000 7f4629a00700 1 Processor -- start 2024-03-23T17:42:38.386 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:37.936+0000 7f4629a00700 1 -- start start 2024-03-23T17:42:38.386 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:37.936+0000 7f4629a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46241065b0 0x7f46241069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:42:38.386 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:37.936+0000 7f4629a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4624106fa0 con 0x7f46241065b0 2024-03-23T17:42:38.386 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:37.937+0000 7f4622a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46241065b0 0x7f46241069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:42:38.386 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:37.937+0000 7f4622a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46241065b0 0x7f46241069d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52774/0 (socket says 172.21.15.86:52774) 2024-03-23T17:42:38.387 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:37.937+0000 7f4622a00700 1 -- 172.21.15.86:0/1624668510 learned_addr learned my addr 172.21.15.86:0/1624668510 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:42:38.387 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:37.937+0000 7f4622a00700 1 -- 172.21.15.86:0/1624668510 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46241077b0 con 0x7f46241065b0 2024-03-23T17:42:38.387 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:37.937+0000 7f4622a00700 1 --2- 172.21.15.86:0/1624668510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46241065b0 0x7f46241069d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f46140098a0 tx=0x7f4614009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a387c90fc125c1e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:42:38.387 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:37.938+0000 7f4621600700 1 -- 172.21.15.86:0/1624668510 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4614004160 con 0x7f46241065b0 2024-03-23T17:42:38.387 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:37.938+0000 7f4621600700 1 -- 172.21.15.86:0/1624668510 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f46140042c0 con 0x7f46241065b0 2024-03-23T17:42:38.387 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:37.938+0000 7f4621600700 1 -- 172.21.15.86:0/1624668510 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4614011a00 con 0x7f46241065b0 2024-03-23T17:42:38.387 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:37.938+0000 7f4629a00700 1 -- 172.21.15.86:0/1624668510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46241065b0 msgr2=0x7f46241069d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:42:38.387 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:37.938+0000 7f4629a00700 1 --2- 172.21.15.86:0/1624668510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46241065b0 0x7f46241069d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f46140098a0 tx=0x7f4614009bb0 comp rx=0 tx=0).stop 2024-03-23T17:42:38.387 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:37.938+0000 7f4629a00700 1 -- 172.21.15.86:0/1624668510 shutdown_connections 2024-03-23T17:42:38.388 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:37.938+0000 7f4629a00700 1 --2- 172.21.15.86:0/1624668510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46241065b0 0x7f46241069d0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:42:38.388 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:37.938+0000 7f4629a00700 1 -- 172.21.15.86:0/1624668510 >> 172.21.15.86:0/1624668510 conn(0x7f4624101b70 msgr2=0x7f4624103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:42:38.388 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:37.938+0000 7f4629a00700 1 -- 172.21.15.86:0/1624668510 shutdown_connections 2024-03-23T17:42:38.388 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:37.939+0000 7f4629a00700 1 -- 172.21.15.86:0/1624668510 wait complete. 2024-03-23T17:42:38.388 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:37.940+0000 7f4629a00700 1 Processor -- start 2024-03-23T17:42:38.388 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:37.940+0000 7f4629a00700 1 -- start start 2024-03-23T17:42:38.388 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:37.941+0000 7f4629a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46241065b0 0x7f462407aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:42:38.388 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:37.941+0000 7f4629a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f462407af70 con 0x7f46241065b0 2024-03-23T17:42:38.388 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:37.941+0000 7f4622a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46241065b0 0x7f462407aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:42:38.388 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:37.941+0000 7f4622a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46241065b0 0x7f462407aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52780/0 (socket says 172.21.15.86:52780) 2024-03-23T17:42:38.388 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:37.941+0000 7f4622a00700 1 -- 172.21.15.86:0/2963957055 learned_addr learned my addr 172.21.15.86:0/2963957055 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:42:38.388 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:37.941+0000 7f4622a00700 1 -- 172.21.15.86:0/2963957055 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4614009550 con 0x7f46241065b0 2024-03-23T17:42:38.388 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:37.941+0000 7f4622a00700 1 --2- 172.21.15.86:0/2963957055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46241065b0 0x7f462407aa30 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4614009130 tx=0x7f4614011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:42:38.388 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:37.942+0000 7f461b400700 1 -- 172.21.15.86:0/2963957055 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4614003ab0 con 0x7f46241065b0 2024-03-23T17:42:38.388 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:37.942+0000 7f4629a00700 1 -- 172.21.15.86:0/2963957055 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46240791b0 con 0x7f46241065b0 2024-03-23T17:42:38.389 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:37.942+0000 7f461b400700 1 -- 172.21.15.86:0/2963957055 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4614003c10 con 0x7f46241065b0 2024-03-23T17:42:38.389 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:37.942+0000 7f461b400700 1 -- 172.21.15.86:0/2963957055 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f46140247d0 con 0x7f46241065b0 2024-03-23T17:42:38.389 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:37.942+0000 7f4629a00700 1 -- 172.21.15.86:0/2963957055 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4624079650 con 0x7f46241065b0 2024-03-23T17:42:38.389 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:37.942+0000 7f461b400700 1 -- 172.21.15.86:0/2963957055 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f4614024d50 con 0x7f46241065b0 2024-03-23T17:42:38.389 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:37.942+0000 7f461b400700 1 -- 172.21.15.86:0/2963957055 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f46140193c0 con 0x7f46241065b0 2024-03-23T17:42:38.389 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:37.943+0000 7f4629a00700 1 -- 172.21.15.86:0/2963957055 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46240623e0 con 0x7f46241065b0 2024-03-23T17:42:38.389 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:37.945+0000 7f461b400700 1 -- 172.21.15.86:0/2963957055 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f461401abb0 con 0x7f46241065b0 2024-03-23T17:42:38.389 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:38.019+0000 7f4629a00700 1 -- 172.21.15.86:0/2963957055 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f462410a9e0 con 0x7f46241065b0 2024-03-23T17:42:38.389 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:38.019+0000 7f461b400700 1 -- 172.21.15.86:0/2963957055 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f461401ad50 con 0x7f46241065b0 2024-03-23T17:42:38.389 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:38.020+0000 7f4629a00700 1 -- 172.21.15.86:0/2963957055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46241065b0 msgr2=0x7f462407aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:42:38.389 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:38.020+0000 7f4629a00700 1 --2- 172.21.15.86:0/2963957055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46241065b0 0x7f462407aa30 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4614009130 tx=0x7f4614011ad0 comp rx=0 tx=0).stop 2024-03-23T17:42:38.389 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:38.020+0000 7f4629a00700 1 -- 172.21.15.86:0/2963957055 shutdown_connections 2024-03-23T17:42:38.389 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:38.020+0000 7f4629a00700 1 --2- 172.21.15.86:0/2963957055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46241065b0 0x7f462407aa30 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:42:38.389 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:38.020+0000 7f4629a00700 1 -- 172.21.15.86:0/2963957055 >> 172.21.15.86:0/2963957055 conn(0x7f4624101b70 msgr2=0x7f4624102790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:42:38.390 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:38.020+0000 7f4629a00700 1 -- 172.21.15.86:0/2963957055 shutdown_connections 2024-03-23T17:42:38.390 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:38.020+0000 7f4629a00700 1 -- 172.21.15.86:0/2963957055 wait complete. 2024-03-23T17:42:38.390 INFO:teuthology.orchestra.run.smithi086.stdout:mgr not available, waiting (1/15)... 2024-03-23T17:42:38.516 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:42:38 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:42:38.325+0000 7faade567200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-23T17:42:39.479 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:42:39 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:42:39.170+0000 7faade567200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-23T17:42:39.766 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:42:39 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:42:39.478+0000 7faade567200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-23T17:42:39.767 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:42:39 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:42:39.589+0000 7faade567200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-23T17:42:39.767 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:42:39 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:42:39.726+0000 7faade567200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-23T17:42:40.200 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:42:39 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:42:39.922+0000 7faade567200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-23T17:42:40.200 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:42:40 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:42:40.087+0000 7faade567200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-23T17:42:40.517 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:42:40 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:42:40.198+0000 7faade567200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-23T17:42:40.966 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:42:40 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:42:40.706+0000 7faade567200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-23T17:42:41.328 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-03-23T17:42:41.328 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout { 2024-03-23T17:42:41.329 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "fsid": "9d76c9e8-e93c-11ee-95cd-87774f69a715", 2024-03-23T17:42:41.329 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "health": { 2024-03-23T17:42:41.329 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-23T17:42:41.329 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-23T17:42:41.329 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-23T17:42:41.329 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-03-23T17:42:41.329 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-23T17:42:41.329 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-23T17:42:41.329 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 0 2024-03-23T17:42:41.329 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ], 2024-03-23T17:42:41.329 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-23T17:42:41.330 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "a" 2024-03-23T17:42:41.330 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ], 2024-03-23T17:42:41.330 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-03-23T17:42:41.330 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-23T17:42:41.330 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-23T17:42:41.330 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-03-23T17:42:41.330 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-23T17:42:41.330 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-03-23T17:42:41.330 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-23T17:42:41.330 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-23T17:42:41.330 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-23T17:42:41.330 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-23T17:42:41.330 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-23T17:42:41.330 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-23T17:42:41.331 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-23T17:42:41.331 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-23T17:42:41.331 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-03-23T17:42:41.331 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-23T17:42:41.331 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-23T17:42:41.331 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-23T17:42:41.331 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-23T17:42:41.331 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-23T17:42:41.331 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-23T17:42:41.331 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-23T17:42:41.331 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-23T17:42:41.331 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-23T17:42:41.331 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-03-23T17:42:41.331 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-23T17:42:41.331 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-23T17:42:41.332 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-23T17:42:41.332 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-23T17:42:41.332 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-03-23T17:42:41.332 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-23T17:42:41.332 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-23T17:42:41.332 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-23T17:42:41.332 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-23T17:42:41.332 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-23T17:42:41.333 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-23T17:42:41.333 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "restful" 2024-03-23T17:42:41.333 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ], 2024-03-23T17:42:41.333 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-23T17:42:41.333 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-03-23T17:42:41.333 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-23T17:42:41.333 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-23T17:42:41.333 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "modified": "2024-03-23T17:42:30.098080+0000", 2024-03-23T17:42:41.333 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-23T17:42:41.333 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-03-23T17:42:41.333 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-23T17:42:41.333 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout } 2024-03-23T17:42:41.334 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:40.859+0000 7f2c2f000700 1 Processor -- start 2024-03-23T17:42:41.334 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:40.859+0000 7f2c2f000700 1 -- start start 2024-03-23T17:42:41.334 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:40.860+0000 7f2c2f000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c2807adf0 0x7f2c28079250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:42:41.334 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:40.860+0000 7f2c2f000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c28079820 con 0x7f2c2807adf0 2024-03-23T17:42:41.334 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:40.860+0000 7f2c27e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c2807adf0 0x7f2c28079250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:42:41.334 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:40.860+0000 7f2c27e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c2807adf0 0x7f2c28079250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52792/0 (socket says 172.21.15.86:52792) 2024-03-23T17:42:41.334 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:40.860+0000 7f2c27e00700 1 -- 172.21.15.86:0/2488244905 learned_addr learned my addr 172.21.15.86:0/2488244905 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:42:41.334 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:40.861+0000 7f2c27e00700 1 -- 172.21.15.86:0/2488244905 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c28079960 con 0x7f2c2807adf0 2024-03-23T17:42:41.334 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:40.861+0000 7f2c27e00700 1 --2- 172.21.15.86:0/2488244905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c2807adf0 0x7f2c28079250 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2c180098a0 tx=0x7f2c18009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a8040ef2a4f6bb2 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:42:41.334 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:40.861+0000 7f2c26a00700 1 -- 172.21.15.86:0/2488244905 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2c18004160 con 0x7f2c2807adf0 2024-03-23T17:42:41.334 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:40.861+0000 7f2c26a00700 1 -- 172.21.15.86:0/2488244905 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2c180042c0 con 0x7f2c2807adf0 2024-03-23T17:42:41.334 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:40.861+0000 7f2c26a00700 1 -- 172.21.15.86:0/2488244905 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2c18011a00 con 0x7f2c2807adf0 2024-03-23T17:42:41.335 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:40.862+0000 7f2c2f000700 1 -- 172.21.15.86:0/2488244905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c2807adf0 msgr2=0x7f2c28079250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:42:41.335 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:40.862+0000 7f2c2f000700 1 --2- 172.21.15.86:0/2488244905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c2807adf0 0x7f2c28079250 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2c180098a0 tx=0x7f2c18009bb0 comp rx=0 tx=0).stop 2024-03-23T17:42:41.335 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:40.862+0000 7f2c2f000700 1 -- 172.21.15.86:0/2488244905 shutdown_connections 2024-03-23T17:42:41.335 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:40.862+0000 7f2c2f000700 1 --2- 172.21.15.86:0/2488244905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c2807adf0 0x7f2c28079250 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:42:41.335 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:40.862+0000 7f2c2f000700 1 -- 172.21.15.86:0/2488244905 >> 172.21.15.86:0/2488244905 conn(0x7f2c28101b70 msgr2=0x7f2c28103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:42:41.335 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:40.862+0000 7f2c2f000700 1 -- 172.21.15.86:0/2488244905 shutdown_connections 2024-03-23T17:42:41.335 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:40.862+0000 7f2c2f000700 1 -- 172.21.15.86:0/2488244905 wait complete. 2024-03-23T17:42:41.335 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:40.863+0000 7f2c2f000700 1 Processor -- start 2024-03-23T17:42:41.335 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:40.863+0000 7f2c2f000700 1 -- start start 2024-03-23T17:42:41.335 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:40.863+0000 7f2c2f000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c2807adf0 0x7f2c281a0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:42:41.335 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:40.863+0000 7f2c2f000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c281a0e50 con 0x7f2c2807adf0 2024-03-23T17:42:41.335 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:40.864+0000 7f2c27e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c2807adf0 0x7f2c281a0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:42:41.335 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:40.864+0000 7f2c27e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c2807adf0 0x7f2c281a0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52798/0 (socket says 172.21.15.86:52798) 2024-03-23T17:42:41.335 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:40.864+0000 7f2c27e00700 1 -- 172.21.15.86:0/1736444570 learned_addr learned my addr 172.21.15.86:0/1736444570 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:42:41.336 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:40.864+0000 7f2c27e00700 1 -- 172.21.15.86:0/1736444570 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c18009550 con 0x7f2c2807adf0 2024-03-23T17:42:41.336 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:40.864+0000 7f2c27e00700 1 --2- 172.21.15.86:0/1736444570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c2807adf0 0x7f2c281a0910 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2c18009130 tx=0x7f2c18011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:42:41.336 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:40.865+0000 7f2c24c00700 1 -- 172.21.15.86:0/1736444570 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2c18003ab0 con 0x7f2c2807adf0 2024-03-23T17:42:41.336 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:40.865+0000 7f2c24c00700 1 -- 172.21.15.86:0/1736444570 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2c18003c10 con 0x7f2c2807adf0 2024-03-23T17:42:41.336 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:40.865+0000 7f2c2f000700 1 -- 172.21.15.86:0/1736444570 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c281a1050 con 0x7f2c2807adf0 2024-03-23T17:42:41.336 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:40.865+0000 7f2c24c00700 1 -- 172.21.15.86:0/1736444570 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2c180247d0 con 0x7f2c2807adf0 2024-03-23T17:42:41.336 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:40.865+0000 7f2c2f000700 1 -- 172.21.15.86:0/1736444570 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c281a14f0 con 0x7f2c2807adf0 2024-03-23T17:42:41.336 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:40.865+0000 7f2c24c00700 1 -- 172.21.15.86:0/1736444570 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f2c18024d00 con 0x7f2c2807adf0 2024-03-23T17:42:41.336 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:40.865+0000 7f2c24c00700 1 -- 172.21.15.86:0/1736444570 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f2c180193c0 con 0x7f2c2807adf0 2024-03-23T17:42:41.336 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:40.866+0000 7f2c2f000700 1 -- 172.21.15.86:0/1736444570 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c28108580 con 0x7f2c2807adf0 2024-03-23T17:42:41.336 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:40.868+0000 7f2c24c00700 1 -- 172.21.15.86:0/1736444570 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f2c1801abb0 con 0x7f2c2807adf0 2024-03-23T17:42:41.336 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:40.943+0000 7f2c2f000700 1 -- 172.21.15.86:0/1736444570 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f2c28108770 con 0x7f2c2807adf0 2024-03-23T17:42:41.336 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:40.944+0000 7f2c24c00700 1 -- 172.21.15.86:0/1736444570 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f2c1801ad50 con 0x7f2c2807adf0 2024-03-23T17:42:41.336 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:40.945+0000 7f2c2f000700 1 -- 172.21.15.86:0/1736444570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c2807adf0 msgr2=0x7f2c281a0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:42:41.337 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:40.945+0000 7f2c2f000700 1 --2- 172.21.15.86:0/1736444570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c2807adf0 0x7f2c281a0910 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2c18009130 tx=0x7f2c18011ad0 comp rx=0 tx=0).stop 2024-03-23T17:42:41.337 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:40.945+0000 7f2c2f000700 1 -- 172.21.15.86:0/1736444570 shutdown_connections 2024-03-23T17:42:41.337 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:40.945+0000 7f2c2f000700 1 --2- 172.21.15.86:0/1736444570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c2807adf0 0x7f2c281a0910 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:42:41.337 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:40.945+0000 7f2c2f000700 1 -- 172.21.15.86:0/1736444570 >> 172.21.15.86:0/1736444570 conn(0x7f2c28101b70 msgr2=0x7f2c28103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:42:41.337 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:40.945+0000 7f2c2f000700 1 -- 172.21.15.86:0/1736444570 shutdown_connections 2024-03-23T17:42:41.337 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:40.945+0000 7f2c2f000700 1 -- 172.21.15.86:0/1736444570 wait complete. 2024-03-23T17:42:41.337 INFO:teuthology.orchestra.run.smithi086.stdout:mgr not available, waiting (2/15)... 2024-03-23T17:42:41.475 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:42:41 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:42:41.217+0000 7faade567200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-23T17:42:41.766 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:41 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/1736444570' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-23T17:42:41.767 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:42:41 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:42:41.473+0000 7faade567200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-23T17:42:41.767 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:42:41 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:42:41.594+0000 7faade567200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-23T17:42:42.848 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:42:42 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:42:42.532+0000 7faade567200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-23T17:42:42.848 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:42:42 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:42:42.658+0000 7faade567200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-23T17:42:43.266 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:42:42 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:42:42.846+0000 7faade567200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-23T17:42:43.767 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:42:43 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:42:43.672+0000 7faade567200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-23T17:42:44.170 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:43 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/222479219' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-23T17:42:44.171 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:42:43 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:42:43.921+0000 7faade567200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-23T17:42:44.516 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:42:44 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:42:44.169+0000 7faade567200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-23T17:42:45.682 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:42:45 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:42:45.306+0000 7faade567200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-23T17:42:46.016 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:42:45 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:42:45.680+0000 7faade567200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-23T17:42:46.017 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:42:45 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:42:45.800+0000 7faade567200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-23T17:42:46.493 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:42:46 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:42:46.056+0000 7faade567200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-23T17:42:46.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:46 smithi086 ceph-mon[38221]: Activating manager daemon x 2024-03-23T17:42:46.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:46 smithi086 ceph-mon[38221]: mgrmap e2: x(active, starting, since 0.00454191s) 2024-03-23T17:42:46.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:46 smithi086 ceph-mon[38221]: from='mgr.14100 172.21.15.86:0/2048195546' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-23T17:42:46.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:46 smithi086 ceph-mon[38221]: from='mgr.14100 172.21.15.86:0/2048195546' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-23T17:42:46.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:46 smithi086 ceph-mon[38221]: from='mgr.14100 172.21.15.86:0/2048195546' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-23T17:42:46.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:46 smithi086 ceph-mon[38221]: from='mgr.14100 172.21.15.86:0/2048195546' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-23T17:42:46.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:46 smithi086 ceph-mon[38221]: from='mgr.14100 172.21.15.86:0/2048195546' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-23T17:42:46.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:46 smithi086 ceph-mon[38221]: Manager daemon x is now available 2024-03-23T17:42:46.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:46 smithi086 ceph-mon[38221]: from='mgr.14100 172.21.15.86:0/2048195546' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-23T17:42:46.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:46 smithi086 ceph-mon[38221]: from='mgr.14100 172.21.15.86:0/2048195546' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-23T17:42:46.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:46 smithi086 ceph-mon[38221]: from='mgr.14100 172.21.15.86:0/2048195546' entity='mgr.x' 2024-03-23T17:42:46.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:46 smithi086 ceph-mon[38221]: from='mgr.14100 172.21.15.86:0/2048195546' entity='mgr.x' 2024-03-23T17:42:46.768 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:46 smithi086 ceph-mon[38221]: from='mgr.14100 172.21.15.86:0/2048195546' entity='mgr.x' 2024-03-23T17:42:48.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:48 smithi086 ceph-mon[38221]: mgrmap e3: x(active, since 1.00989s) 2024-03-23T17:42:49.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:49 smithi086 ceph-mon[38221]: mgrmap e4: x(active, since 2s) 2024-03-23T17:42:51.741 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-03-23T17:42:51.741 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout { 2024-03-23T17:42:51.742 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "fsid": "9d76c9e8-e93c-11ee-95cd-87774f69a715", 2024-03-23T17:42:51.742 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "health": { 2024-03-23T17:42:51.742 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-23T17:42:51.742 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-23T17:42:51.742 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-23T17:42:51.742 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-03-23T17:42:51.742 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-23T17:42:51.742 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-23T17:42:51.742 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 0 2024-03-23T17:42:51.742 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ], 2024-03-23T17:42:51.742 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-23T17:42:51.743 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "a" 2024-03-23T17:42:51.743 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ], 2024-03-23T17:42:51.743 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-03-23T17:42:51.743 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-23T17:42:51.743 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-23T17:42:51.743 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-03-23T17:42:51.743 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-23T17:42:51.743 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-03-23T17:42:51.743 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-23T17:42:51.744 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-23T17:42:51.744 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-23T17:42:51.744 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-23T17:42:51.744 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-23T17:42:51.744 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-23T17:42:51.744 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-23T17:42:51.744 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-23T17:42:51.744 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-03-23T17:42:51.745 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-23T17:42:51.745 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-23T17:42:51.745 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-23T17:42:51.745 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-23T17:42:51.745 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-23T17:42:51.745 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-23T17:42:51.745 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-23T17:42:51.745 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-23T17:42:51.745 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-23T17:42:51.745 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-03-23T17:42:51.745 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-23T17:42:51.745 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-23T17:42:51.746 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-23T17:42:51.746 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-23T17:42:51.746 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-03-23T17:42:51.746 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-23T17:42:51.746 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-23T17:42:51.746 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-23T17:42:51.746 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-23T17:42:51.746 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-23T17:42:51.746 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-23T17:42:51.747 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "restful" 2024-03-23T17:42:51.747 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ], 2024-03-23T17:42:51.747 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-23T17:42:51.747 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-03-23T17:42:51.747 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-23T17:42:51.747 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-23T17:42:51.747 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "modified": "2024-03-23T17:42:30.098080+0000", 2024-03-23T17:42:51.747 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-23T17:42:51.747 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-03-23T17:42:51.747 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-23T17:42:51.747 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout } 2024-03-23T17:42:51.748 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:43.716+0000 7f0373200700 1 Processor -- start 2024-03-23T17:42:51.748 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:43.717+0000 7f0373200700 1 -- start start 2024-03-23T17:42:51.748 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:43.717+0000 7f0373200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f037407adb0 0x7f0374079210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:42:51.748 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:43.717+0000 7f0373200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03740797e0 con 0x7f037407adb0 2024-03-23T17:42:51.748 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:43.717+0000 7f0371e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f037407adb0 0x7f0374079210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:42:51.748 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:43.717+0000 7f0371e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f037407adb0 0x7f0374079210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52814/0 (socket says 172.21.15.86:52814) 2024-03-23T17:42:51.748 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:43.717+0000 7f0371e00700 1 -- 172.21.15.86:0/1853076562 learned_addr learned my addr 172.21.15.86:0/1853076562 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:42:51.748 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:43.718+0000 7f0371e00700 1 -- 172.21.15.86:0/1853076562 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0374079920 con 0x7f037407adb0 2024-03-23T17:42:51.748 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:43.718+0000 7f0371e00700 1 --2- 172.21.15.86:0/1853076562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f037407adb0 0x7f0374079210 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f03640098a0 tx=0x7f0364009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cca9b1d2ff0fc26a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:42:51.749 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:43.718+0000 7f0370a00700 1 -- 172.21.15.86:0/1853076562 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0364004160 con 0x7f037407adb0 2024-03-23T17:42:51.749 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:43.719+0000 7f0370a00700 1 -- 172.21.15.86:0/1853076562 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f03640042c0 con 0x7f037407adb0 2024-03-23T17:42:51.749 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:43.719+0000 7f0370a00700 1 -- 172.21.15.86:0/1853076562 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0364011a00 con 0x7f037407adb0 2024-03-23T17:42:51.749 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:43.719+0000 7f0373200700 1 -- 172.21.15.86:0/1853076562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f037407adb0 msgr2=0x7f0374079210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:42:51.749 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:43.719+0000 7f0373200700 1 --2- 172.21.15.86:0/1853076562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f037407adb0 0x7f0374079210 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f03640098a0 tx=0x7f0364009bb0 comp rx=0 tx=0).stop 2024-03-23T17:42:51.749 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:43.719+0000 7f0373200700 1 -- 172.21.15.86:0/1853076562 shutdown_connections 2024-03-23T17:42:51.749 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:43.719+0000 7f0373200700 1 --2- 172.21.15.86:0/1853076562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f037407adb0 0x7f0374079210 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:42:51.749 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:43.719+0000 7f0373200700 1 -- 172.21.15.86:0/1853076562 >> 172.21.15.86:0/1853076562 conn(0x7f03741019b0 msgr2=0x7f0374103dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:42:51.749 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:43.719+0000 7f0373200700 1 -- 172.21.15.86:0/1853076562 shutdown_connections 2024-03-23T17:42:51.749 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:43.719+0000 7f0373200700 1 -- 172.21.15.86:0/1853076562 wait complete. 2024-03-23T17:42:51.750 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:43.720+0000 7f0373200700 1 Processor -- start 2024-03-23T17:42:51.750 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:43.720+0000 7f0373200700 1 -- start start 2024-03-23T17:42:51.750 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:43.721+0000 7f0373200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f037407adb0 0x7f037419c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:42:51.750 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:43.721+0000 7f0373200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f037419c8f0 con 0x7f037407adb0 2024-03-23T17:42:51.750 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:43.721+0000 7f0371e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f037407adb0 0x7f037419c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:42:51.750 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:43.721+0000 7f0371e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f037407adb0 0x7f037419c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52822/0 (socket says 172.21.15.86:52822) 2024-03-23T17:42:51.750 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:43.721+0000 7f0371e00700 1 -- 172.21.15.86:0/222479219 learned_addr learned my addr 172.21.15.86:0/222479219 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:42:51.750 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:43.721+0000 7f0371e00700 1 -- 172.21.15.86:0/222479219 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0364009550 con 0x7f037407adb0 2024-03-23T17:42:51.751 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:43.722+0000 7f0371e00700 1 --2- 172.21.15.86:0/222479219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f037407adb0 0x7f037419c3b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0364011710 tx=0x7f0364011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:42:51.751 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:43.722+0000 7f036aa00700 1 -- 172.21.15.86:0/222479219 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0364003aa0 con 0x7f037407adb0 2024-03-23T17:42:51.751 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:43.722+0000 7f036aa00700 1 -- 172.21.15.86:0/222479219 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0364003c00 con 0x7f037407adb0 2024-03-23T17:42:51.751 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:43.722+0000 7f036aa00700 1 -- 172.21.15.86:0/222479219 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0364024800 con 0x7f037407adb0 2024-03-23T17:42:51.751 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:43.722+0000 7f0373200700 1 -- 172.21.15.86:0/222479219 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f037419caf0 con 0x7f037407adb0 2024-03-23T17:42:51.751 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:43.722+0000 7f0373200700 1 -- 172.21.15.86:0/222479219 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f037419cf90 con 0x7f037407adb0 2024-03-23T17:42:51.751 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:43.723+0000 7f036aa00700 1 -- 172.21.15.86:0/222479219 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f0364024c60 con 0x7f037407adb0 2024-03-23T17:42:51.751 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:43.723+0000 7f036aa00700 1 -- 172.21.15.86:0/222479219 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f03640193c0 con 0x7f037407adb0 2024-03-23T17:42:51.752 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:43.723+0000 7f0373200700 1 -- 172.21.15.86:0/222479219 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f037410c7a0 con 0x7f037407adb0 2024-03-23T17:42:51.752 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:43.726+0000 7f036aa00700 1 -- 172.21.15.86:0/222479219 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f036402de40 con 0x7f037407adb0 2024-03-23T17:42:51.752 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:43.800+0000 7f0373200700 1 -- 172.21.15.86:0/222479219 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f037410c960 con 0x7f037407adb0 2024-03-23T17:42:51.752 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:43.801+0000 7f036aa00700 1 -- 172.21.15.86:0/222479219 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f036401abd0 con 0x7f037407adb0 2024-03-23T17:42:51.752 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:43.802+0000 7f0373200700 1 -- 172.21.15.86:0/222479219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f037407adb0 msgr2=0x7f037419c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:42:51.752 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:43.802+0000 7f0373200700 1 --2- 172.21.15.86:0/222479219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f037407adb0 0x7f037419c3b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0364011710 tx=0x7f0364011ad0 comp rx=0 tx=0).stop 2024-03-23T17:42:51.752 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:43.802+0000 7f0373200700 1 -- 172.21.15.86:0/222479219 shutdown_connections 2024-03-23T17:42:51.752 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:43.802+0000 7f0373200700 1 --2- 172.21.15.86:0/222479219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f037407adb0 0x7f037419c3b0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:42:51.752 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:43.802+0000 7f0373200700 1 -- 172.21.15.86:0/222479219 >> 172.21.15.86:0/222479219 conn(0x7f03741019b0 msgr2=0x7f03741025a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:42:51.753 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:43.802+0000 7f0373200700 1 -- 172.21.15.86:0/222479219 shutdown_connections 2024-03-23T17:42:51.753 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:43.802+0000 7f0373200700 1 -- 172.21.15.86:0/222479219 wait complete. 2024-03-23T17:42:51.753 INFO:teuthology.orchestra.run.smithi086.stdout:mgr not available, waiting (3/15)... 2024-03-23T17:42:54.766 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:54 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/3484058135' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-23T17:42:54.813 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-03-23T17:42:54.815 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout { 2024-03-23T17:42:54.815 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "fsid": "9d76c9e8-e93c-11ee-95cd-87774f69a715", 2024-03-23T17:42:54.816 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "health": { 2024-03-23T17:42:54.816 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-23T17:42:54.816 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-23T17:42:54.816 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-23T17:42:54.816 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-03-23T17:42:54.816 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-23T17:42:54.816 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-23T17:42:54.816 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 0 2024-03-23T17:42:54.816 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ], 2024-03-23T17:42:54.816 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-23T17:42:54.816 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "a" 2024-03-23T17:42:54.816 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ], 2024-03-23T17:42:54.816 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "quorum_age": 18, 2024-03-23T17:42:54.816 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-23T17:42:54.817 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-23T17:42:54.817 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-03-23T17:42:54.817 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-23T17:42:54.817 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-03-23T17:42:54.817 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-23T17:42:54.817 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-23T17:42:54.817 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-23T17:42:54.817 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-23T17:42:54.826 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-23T17:42:54.826 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-23T17:42:54.826 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-23T17:42:54.826 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-23T17:42:54.826 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-03-23T17:42:54.826 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-23T17:42:54.826 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-23T17:42:54.826 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-23T17:42:54.826 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-23T17:42:54.826 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-23T17:42:54.826 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-23T17:42:54.827 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-23T17:42:54.827 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-23T17:42:54.827 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-23T17:42:54.827 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-03-23T17:42:54.827 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-23T17:42:54.827 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-23T17:42:54.827 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-23T17:42:54.827 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-23T17:42:54.827 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-03-23T17:42:54.827 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-23T17:42:54.828 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-23T17:42:54.828 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-23T17:42:54.828 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-23T17:42:54.828 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-23T17:42:54.828 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-23T17:42:54.828 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "restful" 2024-03-23T17:42:54.828 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ], 2024-03-23T17:42:54.828 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-23T17:42:54.828 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-03-23T17:42:54.828 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-23T17:42:54.828 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-23T17:42:54.828 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "modified": "2024-03-23T17:42:30.098080+0000", 2024-03-23T17:42:54.828 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-23T17:42:54.829 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-03-23T17:42:54.829 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-23T17:42:54.829 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout } 2024-03-23T17:42:54.829 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:54.173+0000 7f6a0c200700 1 Processor -- start 2024-03-23T17:42:54.829 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:54.173+0000 7f6a0c200700 1 -- start start 2024-03-23T17:42:54.829 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:54.173+0000 7f6a0c200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a041065b0 0x7f6a041069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:42:54.829 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:54.173+0000 7f6a0c200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a04106fa0 con 0x7f6a041065b0 2024-03-23T17:42:54.829 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:54.174+0000 7f6a09800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a041065b0 0x7f6a041069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:42:54.829 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:54.174+0000 7f6a09800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a041065b0 0x7f6a041069d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46404/0 (socket says 172.21.15.86:46404) 2024-03-23T17:42:54.829 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:54.174+0000 7f6a09800700 1 -- 172.21.15.86:0/363730689 learned_addr learned my addr 172.21.15.86:0/363730689 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:42:54.829 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:54.174+0000 7f6a09800700 1 -- 172.21.15.86:0/363730689 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a041077b0 con 0x7f6a041065b0 2024-03-23T17:42:54.829 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:54.174+0000 7f6a09800700 1 --2- 172.21.15.86:0/363730689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a041065b0 0x7f6a041069d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f69f80098a0 tx=0x7f69f8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20ece6f53eaac6e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:42:54.829 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:54.175+0000 7f6a03e00700 1 -- 172.21.15.86:0/363730689 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f69f8004160 con 0x7f6a041065b0 2024-03-23T17:42:54.830 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:54.175+0000 7f6a03e00700 1 -- 172.21.15.86:0/363730689 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f69f80042c0 con 0x7f6a041065b0 2024-03-23T17:42:54.830 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:54.175+0000 7f6a03e00700 1 -- 172.21.15.86:0/363730689 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f69f8011a00 con 0x7f6a041065b0 2024-03-23T17:42:54.830 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:54.175+0000 7f6a0c200700 1 -- 172.21.15.86:0/363730689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a041065b0 msgr2=0x7f6a041069d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:42:54.830 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:54.175+0000 7f6a0c200700 1 --2- 172.21.15.86:0/363730689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a041065b0 0x7f6a041069d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f69f80098a0 tx=0x7f69f8009bb0 comp rx=0 tx=0).stop 2024-03-23T17:42:54.830 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:54.175+0000 7f6a0c200700 1 -- 172.21.15.86:0/363730689 shutdown_connections 2024-03-23T17:42:54.830 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:54.175+0000 7f6a0c200700 1 --2- 172.21.15.86:0/363730689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a041065b0 0x7f6a041069d0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:42:54.830 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:54.175+0000 7f6a0c200700 1 -- 172.21.15.86:0/363730689 >> 172.21.15.86:0/363730689 conn(0x7f6a04101b30 msgr2=0x7f6a04103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:42:54.830 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:54.175+0000 7f6a0c200700 1 -- 172.21.15.86:0/363730689 shutdown_connections 2024-03-23T17:42:54.830 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:54.175+0000 7f6a0c200700 1 -- 172.21.15.86:0/363730689 wait complete. 2024-03-23T17:42:54.830 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:54.176+0000 7f6a0c200700 1 Processor -- start 2024-03-23T17:42:54.830 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:54.176+0000 7f6a0c200700 1 -- start start 2024-03-23T17:42:54.830 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:54.177+0000 7f6a0c200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a041065b0 0x7f6a0419c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:42:54.830 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:54.177+0000 7f6a0c200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a0419ca60 con 0x7f6a041065b0 2024-03-23T17:42:54.831 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:54.177+0000 7f6a09800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a041065b0 0x7f6a0419c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:42:54.831 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:54.177+0000 7f6a09800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a041065b0 0x7f6a0419c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46406/0 (socket says 172.21.15.86:46406) 2024-03-23T17:42:54.831 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:54.177+0000 7f6a09800700 1 -- 172.21.15.86:0/3484058135 learned_addr learned my addr 172.21.15.86:0/3484058135 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:42:54.831 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:54.177+0000 7f6a09800700 1 -- 172.21.15.86:0/3484058135 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f8009550 con 0x7f6a041065b0 2024-03-23T17:42:54.831 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:54.177+0000 7f6a09800700 1 --2- 172.21.15.86:0/3484058135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a041065b0 0x7f6a0419c520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f69f8009130 tx=0x7f69f8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:42:54.831 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:54.178+0000 7f6a02000700 1 -- 172.21.15.86:0/3484058135 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f69f8003ab0 con 0x7f6a041065b0 2024-03-23T17:42:54.831 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:54.178+0000 7f6a02000700 1 -- 172.21.15.86:0/3484058135 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f69f8003c10 con 0x7f6a041065b0 2024-03-23T17:42:54.831 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:54.178+0000 7f6a0c200700 1 -- 172.21.15.86:0/3484058135 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a0419cc60 con 0x7f6a041065b0 2024-03-23T17:42:54.831 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:54.178+0000 7f6a02000700 1 -- 172.21.15.86:0/3484058135 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f69f80247d0 con 0x7f6a041065b0 2024-03-23T17:42:54.831 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:54.178+0000 7f6a0c200700 1 -- 172.21.15.86:0/3484058135 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a0419d100 con 0x7f6a041065b0 2024-03-23T17:42:54.831 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:54.179+0000 7f6a0c200700 1 -- 172.21.15.86:0/3484058135 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a0410a9e0 con 0x7f6a041065b0 2024-03-23T17:42:54.831 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:54.179+0000 7f6a02000700 1 -- 172.21.15.86:0/3484058135 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 4) v1 ==== 45522+0+0 (secure 0 0 0) 0x7f69f801abb0 con 0x7f6a041065b0 2024-03-23T17:42:54.831 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:54.179+0000 7f6a02000700 1 --2- 172.21.15.86:0/3484058135 >> [v2:172.21.15.86:6800/2313723492,v1:172.21.15.86:6801/2313723492] conn(0x7f69ec0386b0 0x7f69ec03ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:42:54.831 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:54.179+0000 7f6a02000700 1 -- 172.21.15.86:0/3484058135 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f69f804c450 con 0x7f6a041065b0 2024-03-23T17:42:54.832 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:54.180+0000 7f6a08e00700 1 --2- 172.21.15.86:0/3484058135 >> [v2:172.21.15.86:6800/2313723492,v1:172.21.15.86:6801/2313723492] conn(0x7f69ec0386b0 0x7f69ec03ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:42:54.832 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:54.180+0000 7f6a08e00700 1 --2- 172.21.15.86:0/3484058135 >> [v2:172.21.15.86:6800/2313723492,v1:172.21.15.86:6801/2313723492] conn(0x7f69ec0386b0 0x7f69ec03ab70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f69f0006fd0 tx=0x7f69f0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:42:54.832 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:54.185+0000 7f6a02000700 1 -- 172.21.15.86:0/3484058135 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f69f8018530 con 0x7f6a041065b0 2024-03-23T17:42:54.832 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:54.460+0000 7f6a0c200700 1 -- 172.21.15.86:0/3484058135 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f6a0419f9f0 con 0x7f6a041065b0 2024-03-23T17:42:54.832 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:54.460+0000 7f6a02000700 1 -- 172.21.15.86:0/3484058135 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f69f8018350 con 0x7f6a041065b0 2024-03-23T17:42:54.832 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:54.462+0000 7f6a0c200700 1 -- 172.21.15.86:0/3484058135 >> [v2:172.21.15.86:6800/2313723492,v1:172.21.15.86:6801/2313723492] conn(0x7f69ec0386b0 msgr2=0x7f69ec03ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:42:54.832 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:54.462+0000 7f6a0c200700 1 --2- 172.21.15.86:0/3484058135 >> [v2:172.21.15.86:6800/2313723492,v1:172.21.15.86:6801/2313723492] conn(0x7f69ec0386b0 0x7f69ec03ab70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f69f0006fd0 tx=0x7f69f0006e40 comp rx=0 tx=0).stop 2024-03-23T17:42:54.832 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:54.462+0000 7f6a0c200700 1 -- 172.21.15.86:0/3484058135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a041065b0 msgr2=0x7f6a0419c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:42:54.832 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:54.462+0000 7f6a0c200700 1 --2- 172.21.15.86:0/3484058135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a041065b0 0x7f6a0419c520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f69f8009130 tx=0x7f69f8011ad0 comp rx=0 tx=0).stop 2024-03-23T17:42:54.832 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:54.462+0000 7f6a0c200700 1 -- 172.21.15.86:0/3484058135 shutdown_connections 2024-03-23T17:42:54.832 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:54.462+0000 7f6a0c200700 1 --2- 172.21.15.86:0/3484058135 >> [v2:172.21.15.86:6800/2313723492,v1:172.21.15.86:6801/2313723492] conn(0x7f69ec0386b0 0x7f69ec03ab70 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:42:54.832 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:54.462+0000 7f6a0c200700 1 --2- 172.21.15.86:0/3484058135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a041065b0 0x7f6a0419c520 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:42:54.832 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:54.462+0000 7f6a0c200700 1 -- 172.21.15.86:0/3484058135 >> 172.21.15.86:0/3484058135 conn(0x7f6a04101b30 msgr2=0x7f6a041026f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:42:54.832 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:54.463+0000 7f6a0c200700 1 -- 172.21.15.86:0/3484058135 shutdown_connections 2024-03-23T17:42:54.833 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:54.463+0000 7f6a0c200700 1 -- 172.21.15.86:0/3484058135 wait complete. 2024-03-23T17:42:54.833 INFO:teuthology.orchestra.run.smithi086.stdout:mgr is available 2024-03-23T17:42:55.763 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:55 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/2719748560' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-23T17:42:55.770 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-03-23T17:42:55.770 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout [global] 2024-03-23T17:42:55.770 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout fsid = 9d76c9e8-e93c-11ee-95cd-87774f69a715 2024-03-23T17:42:55.770 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-23T17:42:55.770 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-23T17:42:55.770 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-23T17:42:55.770 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-23T17:42:55.771 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-03-23T17:42:55.771 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-23T17:42:55.771 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-23T17:42:55.771 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-03-23T17:42:55.771 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout [osd] 2024-03-23T17:42:55.771 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-23T17:42:55.771 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-23T17:42:55.771 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-23T17:42:55.771 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-23T17:42:55.771 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-23T17:42:55.771 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:55.204+0000 7f2cdf200700 1 Processor -- start 2024-03-23T17:42:55.771 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:55.205+0000 7f2cdf200700 1 -- start start 2024-03-23T17:42:55.771 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:55.205+0000 7f2cdf200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2cd807ad90 0x7f2cd80791f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:42:55.771 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:55.205+0000 7f2cdf200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd80797c0 con 0x7f2cd807ad90 2024-03-23T17:42:55.772 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:55.205+0000 7f2cd7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2cd807ad90 0x7f2cd80791f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:42:55.772 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:55.205+0000 7f2cd7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2cd807ad90 0x7f2cd80791f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35322/0 (socket says 172.21.15.86:35322) 2024-03-23T17:42:55.772 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:55.205+0000 7f2cd7e00700 1 -- 172.21.15.86:0/2985137597 learned_addr learned my addr 172.21.15.86:0/2985137597 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:42:55.772 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:55.206+0000 7f2cd7e00700 1 -- 172.21.15.86:0/2985137597 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cd8079900 con 0x7f2cd807ad90 2024-03-23T17:42:55.772 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:55.206+0000 7f2cd7e00700 1 --2- 172.21.15.86:0/2985137597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2cd807ad90 0x7f2cd80791f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2cc80098a0 tx=0x7f2cc8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab88aff607907c40 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:42:55.772 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:55.206+0000 7f2cd7400700 1 -- 172.21.15.86:0/2985137597 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2cc8004160 con 0x7f2cd807ad90 2024-03-23T17:42:55.772 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:55.206+0000 7f2cd7400700 1 -- 172.21.15.86:0/2985137597 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2cc80042c0 con 0x7f2cd807ad90 2024-03-23T17:42:55.772 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:55.206+0000 7f2cd7400700 1 -- 172.21.15.86:0/2985137597 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2cc8011a00 con 0x7f2cd807ad90 2024-03-23T17:42:55.772 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:55.207+0000 7f2cdf200700 1 -- 172.21.15.86:0/2985137597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2cd807ad90 msgr2=0x7f2cd80791f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:42:55.772 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:55.207+0000 7f2cdf200700 1 --2- 172.21.15.86:0/2985137597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2cd807ad90 0x7f2cd80791f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2cc80098a0 tx=0x7f2cc8009bb0 comp rx=0 tx=0).stop 2024-03-23T17:42:55.772 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:55.207+0000 7f2cdf200700 1 -- 172.21.15.86:0/2985137597 shutdown_connections 2024-03-23T17:42:55.772 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:55.207+0000 7f2cdf200700 1 --2- 172.21.15.86:0/2985137597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2cd807ad90 0x7f2cd80791f0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:42:55.772 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:55.207+0000 7f2cdf200700 1 -- 172.21.15.86:0/2985137597 >> 172.21.15.86:0/2985137597 conn(0x7f2cd8101b30 msgr2=0x7f2cd8103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:42:55.772 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:55.207+0000 7f2cdf200700 1 -- 172.21.15.86:0/2985137597 shutdown_connections 2024-03-23T17:42:55.772 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:55.207+0000 7f2cdf200700 1 -- 172.21.15.86:0/2985137597 wait complete. 2024-03-23T17:42:55.773 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:55.208+0000 7f2cdf200700 1 Processor -- start 2024-03-23T17:42:55.773 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:55.208+0000 7f2cdf200700 1 -- start start 2024-03-23T17:42:55.773 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:55.208+0000 7f2cdf200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2cd807ad90 0x7f2cd819c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:42:55.773 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:55.208+0000 7f2cdf200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd819ca10 con 0x7f2cd807ad90 2024-03-23T17:42:55.773 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:55.209+0000 7f2cd7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2cd807ad90 0x7f2cd819c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:42:55.773 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:55.209+0000 7f2cd7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2cd807ad90 0x7f2cd819c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35324/0 (socket says 172.21.15.86:35324) 2024-03-23T17:42:55.773 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:55.209+0000 7f2cd7e00700 1 -- 172.21.15.86:0/2719748560 learned_addr learned my addr 172.21.15.86:0/2719748560 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:42:55.773 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:55.209+0000 7f2cd7e00700 1 -- 172.21.15.86:0/2719748560 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cc8009550 con 0x7f2cd807ad90 2024-03-23T17:42:55.773 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:55.209+0000 7f2cd7e00700 1 --2- 172.21.15.86:0/2719748560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2cd807ad90 0x7f2cd819c4d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2cc8009130 tx=0x7f2cc8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:42:55.773 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:55.209+0000 7f2cd5600700 1 -- 172.21.15.86:0/2719748560 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2cc8003b70 con 0x7f2cd807ad90 2024-03-23T17:42:55.773 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:55.209+0000 7f2cd5600700 1 -- 172.21.15.86:0/2719748560 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2cc8003cd0 con 0x7f2cd807ad90 2024-03-23T17:42:55.773 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:55.209+0000 7f2cd5600700 1 -- 172.21.15.86:0/2719748560 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2cc80197d0 con 0x7f2cd807ad90 2024-03-23T17:42:55.773 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:55.209+0000 7f2cdf200700 1 -- 172.21.15.86:0/2719748560 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cd819cc10 con 0x7f2cd807ad90 2024-03-23T17:42:55.773 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:55.210+0000 7f2cdf200700 1 -- 172.21.15.86:0/2719748560 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cd819d0b0 con 0x7f2cd807ad90 2024-03-23T17:42:55.773 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:55.210+0000 7f2cd5600700 1 -- 172.21.15.86:0/2719748560 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 4) v1 ==== 45522+0+0 (secure 0 0 0) 0x7f2cc801abb0 con 0x7f2cd807ad90 2024-03-23T17:42:55.774 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:55.211+0000 7f2cdf200700 1 -- 172.21.15.86:0/2719748560 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cd810c9a0 con 0x7f2cd807ad90 2024-03-23T17:42:55.774 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:55.211+0000 7f2cd5600700 1 --2- 172.21.15.86:0/2719748560 >> [v2:172.21.15.86:6800/2313723492,v1:172.21.15.86:6801/2313723492] conn(0x7f2cbc038700 0x7f2cbc03abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:42:55.774 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:55.211+0000 7f2cd5600700 1 -- 172.21.15.86:0/2719748560 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f2cc804c890 con 0x7f2cd807ad90 2024-03-23T17:42:55.774 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:55.211+0000 7f2ccfe00700 1 --2- 172.21.15.86:0/2719748560 >> [v2:172.21.15.86:6800/2313723492,v1:172.21.15.86:6801/2313723492] conn(0x7f2cbc038700 0x7f2cbc03abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:42:55.774 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:55.212+0000 7f2ccfe00700 1 --2- 172.21.15.86:0/2719748560 >> [v2:172.21.15.86:6800/2313723492,v1:172.21.15.86:6801/2313723492] conn(0x7f2cbc038700 0x7f2cbc03abc0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0006fd0 tx=0x7f2cc0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:42:55.774 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:55.216+0000 7f2cd5600700 1 -- 172.21.15.86:0/2719748560 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f2cc801a430 con 0x7f2cd807ad90 2024-03-23T17:42:55.774 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:55.415+0000 7f2cdf200700 1 -- 172.21.15.86:0/2719748560 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f2cd810cbe0 con 0x7f2cd807ad90 2024-03-23T17:42:55.774 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:55.416+0000 7f2cd5600700 1 -- 172.21.15.86:0/2719748560 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f2cc8023370 con 0x7f2cd807ad90 2024-03-23T17:42:55.774 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:55.418+0000 7f2cdf200700 1 -- 172.21.15.86:0/2719748560 >> [v2:172.21.15.86:6800/2313723492,v1:172.21.15.86:6801/2313723492] conn(0x7f2cbc038700 msgr2=0x7f2cbc03abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:42:55.774 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:55.418+0000 7f2cdf200700 1 --2- 172.21.15.86:0/2719748560 >> [v2:172.21.15.86:6800/2313723492,v1:172.21.15.86:6801/2313723492] conn(0x7f2cbc038700 0x7f2cbc03abc0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0006fd0 tx=0x7f2cc0006e40 comp rx=0 tx=0).stop 2024-03-23T17:42:55.774 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:55.418+0000 7f2cdf200700 1 -- 172.21.15.86:0/2719748560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2cd807ad90 msgr2=0x7f2cd819c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:42:55.774 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:55.418+0000 7f2cdf200700 1 --2- 172.21.15.86:0/2719748560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2cd807ad90 0x7f2cd819c4d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2cc8009130 tx=0x7f2cc8011ad0 comp rx=0 tx=0).stop 2024-03-23T17:42:55.774 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:55.418+0000 7f2cdf200700 1 -- 172.21.15.86:0/2719748560 shutdown_connections 2024-03-23T17:42:55.774 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:55.418+0000 7f2cdf200700 1 --2- 172.21.15.86:0/2719748560 >> [v2:172.21.15.86:6800/2313723492,v1:172.21.15.86:6801/2313723492] conn(0x7f2cbc038700 0x7f2cbc03abc0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:42:55.775 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:55.418+0000 7f2cdf200700 1 --2- 172.21.15.86:0/2719748560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2cd807ad90 0x7f2cd819c4d0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:42:55.775 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:55.418+0000 7f2cdf200700 1 -- 172.21.15.86:0/2719748560 >> 172.21.15.86:0/2719748560 conn(0x7f2cd8101b30 msgr2=0x7f2cd8102700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:42:55.775 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:55.418+0000 7f2cdf200700 1 -- 172.21.15.86:0/2719748560 shutdown_connections 2024-03-23T17:42:55.775 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:55.418+0000 7f2cdf200700 1 -- 172.21.15.86:0/2719748560 wait complete. 2024-03-23T17:42:55.775 INFO:teuthology.orchestra.run.smithi086.stdout:Enabling cephadm module... 2024-03-23T17:42:56.766 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:56 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/4102339747' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-03-23T17:42:56.767 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:42:56 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: ignoring --setuser ceph since I am not root 2024-03-23T17:42:56.767 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:42:56 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: ignoring --setgroup ceph since I am not root 2024-03-23T17:42:56.938 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:56.174+0000 7f7948a00700 1 Processor -- start 2024-03-23T17:42:56.939 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:56.174+0000 7f7948a00700 1 -- start start 2024-03-23T17:42:56.939 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:56.174+0000 7f7948a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f794407ad90 0x7f79440791f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:42:56.939 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:56.174+0000 7f7948a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79440797c0 con 0x7f794407ad90 2024-03-23T17:42:56.939 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:56.174+0000 7f7942800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f794407ad90 0x7f79440791f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:42:56.939 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:56.175+0000 7f7942800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f794407ad90 0x7f79440791f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35334/0 (socket says 172.21.15.86:35334) 2024-03-23T17:42:56.939 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:56.175+0000 7f7942800700 1 -- 172.21.15.86:0/1957187047 learned_addr learned my addr 172.21.15.86:0/1957187047 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:42:56.939 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:56.175+0000 7f7942800700 1 -- 172.21.15.86:0/1957187047 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7944079900 con 0x7f794407ad90 2024-03-23T17:42:56.939 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:56.175+0000 7f7942800700 1 --2- 172.21.15.86:0/1957187047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f794407ad90 0x7f79440791f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f79340098a0 tx=0x7f7934009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=806523d1de92de4d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:42:56.939 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:56.175+0000 7f7941400700 1 -- 172.21.15.86:0/1957187047 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7934004160 con 0x7f794407ad90 2024-03-23T17:42:56.939 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:56.175+0000 7f7941400700 1 -- 172.21.15.86:0/1957187047 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f79340042c0 con 0x7f794407ad90 2024-03-23T17:42:56.939 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:56.175+0000 7f7941400700 1 -- 172.21.15.86:0/1957187047 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7934011a00 con 0x7f794407ad90 2024-03-23T17:42:56.939 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:56.176+0000 7f7948a00700 1 -- 172.21.15.86:0/1957187047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f794407ad90 msgr2=0x7f79440791f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:42:56.939 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:56.176+0000 7f7948a00700 1 --2- 172.21.15.86:0/1957187047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f794407ad90 0x7f79440791f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f79340098a0 tx=0x7f7934009bb0 comp rx=0 tx=0).stop 2024-03-23T17:42:56.939 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:56.176+0000 7f7948a00700 1 -- 172.21.15.86:0/1957187047 shutdown_connections 2024-03-23T17:42:56.940 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:56.176+0000 7f7948a00700 1 --2- 172.21.15.86:0/1957187047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f794407ad90 0x7f79440791f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:42:56.940 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:56.176+0000 7f7948a00700 1 -- 172.21.15.86:0/1957187047 >> 172.21.15.86:0/1957187047 conn(0x7f79441019b0 msgr2=0x7f7944103dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:42:56.940 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:56.176+0000 7f7948a00700 1 -- 172.21.15.86:0/1957187047 shutdown_connections 2024-03-23T17:42:56.940 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:56.176+0000 7f7948a00700 1 -- 172.21.15.86:0/1957187047 wait complete. 2024-03-23T17:42:56.940 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:56.177+0000 7f7948a00700 1 Processor -- start 2024-03-23T17:42:56.940 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:56.177+0000 7f7948a00700 1 -- start start 2024-03-23T17:42:56.940 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:56.177+0000 7f7948a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f794407ad90 0x7f794419c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:42:56.940 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:56.177+0000 7f7948a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f794419c8d0 con 0x7f794407ad90 2024-03-23T17:42:56.940 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:56.178+0000 7f7942800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f794407ad90 0x7f794419c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:42:56.940 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:56.178+0000 7f7942800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f794407ad90 0x7f794419c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35350/0 (socket says 172.21.15.86:35350) 2024-03-23T17:42:56.940 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:56.178+0000 7f7942800700 1 -- 172.21.15.86:0/4102339747 learned_addr learned my addr 172.21.15.86:0/4102339747 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:42:56.940 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:56.178+0000 7f7942800700 1 -- 172.21.15.86:0/4102339747 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7934009550 con 0x7f794407ad90 2024-03-23T17:42:56.940 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:56.178+0000 7f7942800700 1 --2- 172.21.15.86:0/4102339747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f794407ad90 0x7f794419c390 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7934011a20 tx=0x7f7934011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:42:56.940 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:56.178+0000 7f793b400700 1 -- 172.21.15.86:0/4102339747 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7934003af0 con 0x7f794407ad90 2024-03-23T17:42:56.940 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:56.179+0000 7f793b400700 1 -- 172.21.15.86:0/4102339747 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7934003c50 con 0x7f794407ad90 2024-03-23T17:42:56.941 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:56.179+0000 7f7948a00700 1 -- 172.21.15.86:0/4102339747 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f794419cad0 con 0x7f794407ad90 2024-03-23T17:42:56.941 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:56.179+0000 7f793b400700 1 -- 172.21.15.86:0/4102339747 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7934019760 con 0x7f794407ad90 2024-03-23T17:42:56.941 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:56.179+0000 7f7948a00700 1 -- 172.21.15.86:0/4102339747 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f794419cf70 con 0x7f794407ad90 2024-03-23T17:42:56.941 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:56.180+0000 7f793b400700 1 -- 172.21.15.86:0/4102339747 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 4) v1 ==== 45522+0+0 (secure 0 0 0) 0x7f793401a430 con 0x7f794407ad90 2024-03-23T17:42:56.941 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:56.180+0000 7f7948a00700 1 -- 172.21.15.86:0/4102339747 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f794410c7c0 con 0x7f794407ad90 2024-03-23T17:42:56.941 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:56.180+0000 7f793b400700 1 --2- 172.21.15.86:0/4102339747 >> [v2:172.21.15.86:6800/2313723492,v1:172.21.15.86:6801/2313723492] conn(0x7f79280386b0 0x7f792803ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:42:56.941 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:56.180+0000 7f793b400700 1 -- 172.21.15.86:0/4102339747 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f793404c900 con 0x7f794407ad90 2024-03-23T17:42:56.941 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:56.180+0000 7f7941e00700 1 --2- 172.21.15.86:0/4102339747 >> [v2:172.21.15.86:6800/2313723492,v1:172.21.15.86:6801/2313723492] conn(0x7f79280386b0 0x7f792803ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:42:56.941 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:56.181+0000 7f7941e00700 1 --2- 172.21.15.86:0/4102339747 >> [v2:172.21.15.86:6800/2313723492,v1:172.21.15.86:6801/2313723492] conn(0x7f79280386b0 0x7f792803ab70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7930006fd0 tx=0x7f7930006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:42:56.941 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:56.185+0000 7f793b400700 1 -- 172.21.15.86:0/4102339747 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f7934023370 con 0x7f794407ad90 2024-03-23T17:42:56.941 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:56.436+0000 7f7948a00700 1 -- 172.21.15.86:0/4102339747 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f794419f860 con 0x7f794407ad90 2024-03-23T17:42:56.941 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:56.517+0000 7f793b400700 1 -- 172.21.15.86:0/4102339747 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mgrmap(e 5) v1 ==== 45533+0+0 (secure 0 0 0) 0x7f79340199a0 con 0x7f794407ad90 2024-03-23T17:42:56.941 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:56.519+0000 7f793b400700 1 -- 172.21.15.86:0/4102339747 <== mon.0 v2:172.21.15.86:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f793404c320 con 0x7f794407ad90 2024-03-23T17:42:56.941 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:56.522+0000 7f7948a00700 1 -- 172.21.15.86:0/4102339747 >> [v2:172.21.15.86:6800/2313723492,v1:172.21.15.86:6801/2313723492] conn(0x7f79280386b0 msgr2=0x7f792803ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:42:56.942 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:56.522+0000 7f7948a00700 1 --2- 172.21.15.86:0/4102339747 >> [v2:172.21.15.86:6800/2313723492,v1:172.21.15.86:6801/2313723492] conn(0x7f79280386b0 0x7f792803ab70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7930006fd0 tx=0x7f7930006e40 comp rx=0 tx=0).stop 2024-03-23T17:42:56.942 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:56.522+0000 7f7948a00700 1 -- 172.21.15.86:0/4102339747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f794407ad90 msgr2=0x7f794419c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:42:56.942 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:56.522+0000 7f7948a00700 1 --2- 172.21.15.86:0/4102339747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f794407ad90 0x7f794419c390 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7934011a20 tx=0x7f7934011b00 comp rx=0 tx=0).stop 2024-03-23T17:42:56.942 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:56.522+0000 7f7948a00700 1 -- 172.21.15.86:0/4102339747 shutdown_connections 2024-03-23T17:42:56.942 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:56.522+0000 7f7948a00700 1 --2- 172.21.15.86:0/4102339747 >> [v2:172.21.15.86:6800/2313723492,v1:172.21.15.86:6801/2313723492] conn(0x7f79280386b0 0x7f792803ab70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:42:56.942 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:56.522+0000 7f7948a00700 1 --2- 172.21.15.86:0/4102339747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f794407ad90 0x7f794419c390 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:42:56.942 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:56.522+0000 7f7948a00700 1 -- 172.21.15.86:0/4102339747 >> 172.21.15.86:0/4102339747 conn(0x7f79441019b0 msgr2=0x7f79441025e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:42:56.942 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:56.522+0000 7f7948a00700 1 -- 172.21.15.86:0/4102339747 shutdown_connections 2024-03-23T17:42:56.942 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:56.522+0000 7f7948a00700 1 -- 172.21.15.86:0/4102339747 wait complete. 2024-03-23T17:42:57.375 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:42:57 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:42:57.133+0000 7f6d6aaaa200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-23T17:42:57.375 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:42:57 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:42:57.259+0000 7f6d6aaaa200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-23T17:42:57.650 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:57 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/4102339747' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-03-23T17:42:57.650 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:57 smithi086 ceph-mon[38221]: mgrmap e5: x(active, since 10s) 2024-03-23T17:42:57.651 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:42:57 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:42:57.375+0000 7f6d6aaaa200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-23T17:42:57.651 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:42:57 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:42:57.485+0000 7f6d6aaaa200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-23T17:42:58.001 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout { 2024-03-23T17:42:58.002 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-03-23T17:42:58.002 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-23T17:42:58.002 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-23T17:42:58.002 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-23T17:42:58.002 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout } 2024-03-23T17:42:58.002 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:57.356+0000 7fed0a400700 1 Processor -- start 2024-03-23T17:42:58.002 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:57.356+0000 7fed0a400700 1 -- start start 2024-03-23T17:42:58.002 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:57.356+0000 7fed0a400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed0407adf0 0x7fed04079250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:42:58.002 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:57.356+0000 7fed0a400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed04079820 con 0x7fed0407adf0 2024-03-23T17:42:58.002 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:57.357+0000 7fed09000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed0407adf0 0x7fed04079250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:42:58.002 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:57.357+0000 7fed09000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed0407adf0 0x7fed04079250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35370/0 (socket says 172.21.15.86:35370) 2024-03-23T17:42:58.002 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:57.357+0000 7fed09000700 1 -- 172.21.15.86:0/3300901323 learned_addr learned my addr 172.21.15.86:0/3300901323 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:42:58.003 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:57.357+0000 7fed09000700 1 -- 172.21.15.86:0/3300901323 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed04079960 con 0x7fed0407adf0 2024-03-23T17:42:58.003 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:57.357+0000 7fed09000700 1 --2- 172.21.15.86:0/3300901323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed0407adf0 0x7fed04079250 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fecf80098a0 tx=0x7fecf8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdfea47839c80469 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:42:58.003 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:57.357+0000 7fed03400700 1 -- 172.21.15.86:0/3300901323 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fecf8004160 con 0x7fed0407adf0 2024-03-23T17:42:58.003 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:57.358+0000 7fed03400700 1 -- 172.21.15.86:0/3300901323 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fecf80042c0 con 0x7fed0407adf0 2024-03-23T17:42:58.003 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:57.358+0000 7fed03400700 1 -- 172.21.15.86:0/3300901323 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fecf8011a00 con 0x7fed0407adf0 2024-03-23T17:42:58.003 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:57.358+0000 7fed0a400700 1 -- 172.21.15.86:0/3300901323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed0407adf0 msgr2=0x7fed04079250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:42:58.003 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:57.358+0000 7fed0a400700 1 --2- 172.21.15.86:0/3300901323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed0407adf0 0x7fed04079250 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fecf80098a0 tx=0x7fecf8009bb0 comp rx=0 tx=0).stop 2024-03-23T17:42:58.003 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:57.358+0000 7fed0a400700 1 -- 172.21.15.86:0/3300901323 shutdown_connections 2024-03-23T17:42:58.003 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:57.358+0000 7fed0a400700 1 --2- 172.21.15.86:0/3300901323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed0407adf0 0x7fed04079250 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:42:58.003 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:57.358+0000 7fed0a400700 1 -- 172.21.15.86:0/3300901323 >> 172.21.15.86:0/3300901323 conn(0x7fed04101990 msgr2=0x7fed04103df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:42:58.003 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:57.358+0000 7fed0a400700 1 -- 172.21.15.86:0/3300901323 shutdown_connections 2024-03-23T17:42:58.003 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:57.358+0000 7fed0a400700 1 -- 172.21.15.86:0/3300901323 wait complete. 2024-03-23T17:42:58.003 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:57.360+0000 7fed0a400700 1 Processor -- start 2024-03-23T17:42:58.003 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:57.360+0000 7fed0a400700 1 -- start start 2024-03-23T17:42:58.004 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:57.360+0000 7fed0a400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed0407adf0 0x7fed041a07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:42:58.004 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:57.360+0000 7fed0a400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed041a0d10 con 0x7fed0407adf0 2024-03-23T17:42:58.004 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:57.361+0000 7fed09000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed0407adf0 0x7fed041a07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:42:58.004 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:57.361+0000 7fed09000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed0407adf0 0x7fed041a07d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35380/0 (socket says 172.21.15.86:35380) 2024-03-23T17:42:58.004 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:57.361+0000 7fed09000700 1 -- 172.21.15.86:0/307891651 learned_addr learned my addr 172.21.15.86:0/307891651 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:42:58.004 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:57.361+0000 7fed09000700 1 -- 172.21.15.86:0/307891651 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecf8009550 con 0x7fed0407adf0 2024-03-23T17:42:58.004 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:57.361+0000 7fed09000700 1 --2- 172.21.15.86:0/307891651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed0407adf0 0x7fed041a07d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fecf8011710 tx=0x7fecf8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:42:58.004 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:57.361+0000 7fed01600700 1 -- 172.21.15.86:0/307891651 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fecf8003aa0 con 0x7fed0407adf0 2024-03-23T17:42:58.004 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:57.361+0000 7fed0a400700 1 -- 172.21.15.86:0/307891651 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed041a0f10 con 0x7fed0407adf0 2024-03-23T17:42:58.004 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:57.362+0000 7fed01600700 1 -- 172.21.15.86:0/307891651 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fecf8003c00 con 0x7fed0407adf0 2024-03-23T17:42:58.004 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:57.362+0000 7fed01600700 1 -- 172.21.15.86:0/307891651 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fecf8024800 con 0x7fed0407adf0 2024-03-23T17:42:58.004 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:57.362+0000 7fed0a400700 1 -- 172.21.15.86:0/307891651 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed041a13b0 con 0x7fed0407adf0 2024-03-23T17:42:58.004 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:57.363+0000 7fed01600700 1 -- 172.21.15.86:0/307891651 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) v1 ==== 45533+0+0 (secure 0 0 0) 0x7fecf801abd0 con 0x7fed0407adf0 2024-03-23T17:42:58.004 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:57.363+0000 7fed0a400700 1 -- 172.21.15.86:0/307891651 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed041083e0 con 0x7fed0407adf0 2024-03-23T17:42:58.004 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:57.363+0000 7fed01600700 1 --2- 172.21.15.86:0/307891651 >> [v2:172.21.15.86:6800/2313723492,v1:172.21.15.86:6801/2313723492] conn(0x7fecec038750 0x7fecec03ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:42:58.005 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:57.363+0000 7fed01600700 1 -- 172.21.15.86:0/307891651 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fecf804c240 con 0x7fed0407adf0 2024-03-23T17:42:58.005 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:57.363+0000 7fed03e00700 1 -- 172.21.15.86:0/307891651 >> [v2:172.21.15.86:6800/2313723492,v1:172.21.15.86:6801/2313723492] conn(0x7fecec038750 msgr2=0x7fecec03ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/2313723492 2024-03-23T17:42:58.005 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:57.363+0000 7fed03e00700 1 --2- 172.21.15.86:0/307891651 >> [v2:172.21.15.86:6800/2313723492,v1:172.21.15.86:6801/2313723492] conn(0x7fecec038750 0x7fecec03ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-23T17:42:58.005 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:57.368+0000 7fed01600700 1 -- 172.21.15.86:0/307891651 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fecf8024960 con 0x7fed0407adf0 2024-03-23T17:42:58.005 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:57.564+0000 7fed03e00700 1 -- 172.21.15.86:0/307891651 >> [v2:172.21.15.86:6800/2313723492,v1:172.21.15.86:6801/2313723492] conn(0x7fecec038750 msgr2=0x7fecec03ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/2313723492 2024-03-23T17:42:58.005 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:57.564+0000 7fed03e00700 1 --2- 172.21.15.86:0/307891651 >> [v2:172.21.15.86:6800/2313723492,v1:172.21.15.86:6801/2313723492] conn(0x7fecec038750 0x7fecec03ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-23T17:42:58.005 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:57.626+0000 7fed0a400700 1 -- 172.21.15.86:0/307891651 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fed04199f10 con 0x7fed0407adf0 2024-03-23T17:42:58.005 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:57.626+0000 7fed01600700 1 -- 172.21.15.86:0/307891651 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7fecf802c350 con 0x7fed0407adf0 2024-03-23T17:42:58.005 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:57.629+0000 7fed0a400700 1 -- 172.21.15.86:0/307891651 >> [v2:172.21.15.86:6800/2313723492,v1:172.21.15.86:6801/2313723492] conn(0x7fecec038750 msgr2=0x7fecec03ac10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-23T17:42:58.005 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:57.629+0000 7fed0a400700 1 --2- 172.21.15.86:0/307891651 >> [v2:172.21.15.86:6800/2313723492,v1:172.21.15.86:6801/2313723492] conn(0x7fecec038750 0x7fecec03ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:42:58.005 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:57.629+0000 7fed0a400700 1 -- 172.21.15.86:0/307891651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed0407adf0 msgr2=0x7fed041a07d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:42:58.005 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:57.629+0000 7fed0a400700 1 --2- 172.21.15.86:0/307891651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed0407adf0 0x7fed041a07d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fecf8011710 tx=0x7fecf8011ad0 comp rx=0 tx=0).stop 2024-03-23T17:42:58.005 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:57.629+0000 7fed0a400700 1 -- 172.21.15.86:0/307891651 shutdown_connections 2024-03-23T17:42:58.006 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:57.629+0000 7fed0a400700 1 --2- 172.21.15.86:0/307891651 >> [v2:172.21.15.86:6800/2313723492,v1:172.21.15.86:6801/2313723492] conn(0x7fecec038750 0x7fecec03ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:42:58.006 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:57.629+0000 7fed0a400700 1 --2- 172.21.15.86:0/307891651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed0407adf0 0x7fed041a07d0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:42:58.006 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:57.629+0000 7fed0a400700 1 -- 172.21.15.86:0/307891651 >> 172.21.15.86:0/307891651 conn(0x7fed04101990 msgr2=0x7fed04103df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:42:58.006 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:57.629+0000 7fed0a400700 1 -- 172.21.15.86:0/307891651 shutdown_connections 2024-03-23T17:42:58.006 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:57.629+0000 7fed0a400700 1 -- 172.21.15.86:0/307891651 wait complete. 2024-03-23T17:42:58.006 INFO:teuthology.orchestra.run.smithi086.stdout:Waiting for the mgr to restart... 2024-03-23T17:42:58.006 INFO:teuthology.orchestra.run.smithi086.stdout:Waiting for mgr epoch 5... 2024-03-23T17:42:58.655 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:42:58 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/307891651' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-23T17:42:58.655 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:42:58 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:42:58.340+0000 7f6d6aaaa200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-23T17:42:59.016 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:42:58 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:42:58.654+0000 7f6d6aaaa200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-23T17:42:59.017 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:42:58 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:42:58.762+0000 7f6d6aaaa200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-23T17:42:59.017 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:42:58 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:42:58.899+0000 7f6d6aaaa200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-23T17:42:59.369 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:42:59 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:42:59.091+0000 7f6d6aaaa200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-23T17:42:59.369 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:42:59 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:42:59.256+0000 7f6d6aaaa200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-23T17:42:59.767 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:42:59 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:42:59.368+0000 7f6d6aaaa200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-23T17:43:00.266 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:42:59 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:42:59.833+0000 7f6d6aaaa200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-23T17:43:00.670 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:43:00 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:43:00.308+0000 7f6d6aaaa200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-23T17:43:00.670 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:43:00 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:43:00.550+0000 7f6d6aaaa200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-23T17:43:01.016 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:43:00 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:43:00.669+0000 7f6d6aaaa200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-23T17:43:01.912 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:43:01 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:43:01.597+0000 7f6d6aaaa200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-23T17:43:01.912 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:43:01 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:43:01.720+0000 7f6d6aaaa200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-23T17:43:02.266 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:43:01 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:43:01.911+0000 7f6d6aaaa200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-23T17:43:03.016 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:43:02 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:43:02.704+0000 7f6d6aaaa200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-23T17:43:03.017 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:43:02 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:43:02.940+0000 7f6d6aaaa200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-23T17:43:03.516 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:43:03 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:43:03.188+0000 7f6d6aaaa200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-23T17:43:04.679 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:43:04 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:43:04.310+0000 7f6d6aaaa200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-23T17:43:05.016 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:43:04 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:43:04.679+0000 7f6d6aaaa200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-23T17:43:05.017 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:43:04 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:43:04.793+0000 7f6d6aaaa200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-23T17:43:05.516 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:43:05 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:43:05.041+0000 7f6d6aaaa200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-23T17:43:05.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:05 smithi086 ceph-mon[38221]: Active manager daemon x restarted 2024-03-23T17:43:05.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:05 smithi086 ceph-mon[38221]: Activating manager daemon x 2024-03-23T17:43:05.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:05 smithi086 ceph-mon[38221]: osdmap e2: 0 total, 0 up, 0 in 2024-03-23T17:43:05.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:05 smithi086 ceph-mon[38221]: mgrmap e6: x(active, starting, since 0.00401104s) 2024-03-23T17:43:05.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:05 smithi086 ceph-mon[38221]: from='mgr.14120 172.21.15.86:0/1645921427' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-23T17:43:05.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:05 smithi086 ceph-mon[38221]: from='mgr.14120 172.21.15.86:0/1645921427' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-23T17:43:05.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:05 smithi086 ceph-mon[38221]: from='mgr.14120 172.21.15.86:0/1645921427' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-23T17:43:05.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:05 smithi086 ceph-mon[38221]: from='mgr.14120 172.21.15.86:0/1645921427' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-23T17:43:05.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:05 smithi086 ceph-mon[38221]: from='mgr.14120 172.21.15.86:0/1645921427' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-23T17:43:05.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:05 smithi086 ceph-mon[38221]: Manager daemon x is now available 2024-03-23T17:43:05.518 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:05 smithi086 ceph-mon[38221]: from='mgr.14120 172.21.15.86:0/1645921427' entity='mgr.x' 2024-03-23T17:43:05.518 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:05 smithi086 ceph-mon[38221]: from='mgr.14120 172.21.15.86:0/1645921427' entity='mgr.x' 2024-03-23T17:43:05.518 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:05 smithi086 ceph-mon[38221]: from='mgr.14120 172.21.15.86:0/1645921427' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:43:05.518 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:05 smithi086 ceph-mon[38221]: from='mgr.14120 172.21.15.86:0/1645921427' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:43:06.405 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:06 smithi086 ceph-mon[38221]: Found migration_current of "None". Setting to last migration. 2024-03-23T17:43:06.405 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:06 smithi086 ceph-mon[38221]: from='mgr.14120 172.21.15.86:0/1645921427' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:43:06.406 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:06 smithi086 ceph-mon[38221]: from='mgr.14120 172.21.15.86:0/1645921427' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-23T17:43:06.406 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:06 smithi086 ceph-mon[38221]: from='mgr.14120 172.21.15.86:0/1645921427' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-23T17:43:06.406 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:06 smithi086 ceph-mon[38221]: from='mgr.14120 172.21.15.86:0/1645921427' entity='mgr.x' 2024-03-23T17:43:06.406 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:06 smithi086 ceph-mon[38221]: from='mgr.14120 172.21.15.86:0/1645921427' entity='mgr.x' 2024-03-23T17:43:06.406 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:06 smithi086 ceph-mon[38221]: mgrmap e7: x(active, since 1.00588s) 2024-03-23T17:43:06.417 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout { 2024-03-23T17:43:06.417 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-03-23T17:43:06.417 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-23T17:43:06.417 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout } 2024-03-23T17:43:06.417 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:58.441+0000 7ff3ca400700 1 Processor -- start 2024-03-23T17:43:06.417 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:58.442+0000 7ff3ca400700 1 -- start start 2024-03-23T17:43:06.417 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:58.442+0000 7ff3ca400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3c407adb0 0x7ff3c4079210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:43:06.417 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:58.442+0000 7ff3ca400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c40797e0 con 0x7ff3c407adb0 2024-03-23T17:43:06.417 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:58.442+0000 7ff3c3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3c407adb0 0x7ff3c4079210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:43:06.417 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:58.442+0000 7ff3c3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3c407adb0 0x7ff3c4079210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35384/0 (socket says 172.21.15.86:35384) 2024-03-23T17:43:06.417 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:58.442+0000 7ff3c3400700 1 -- 172.21.15.86:0/3065704629 learned_addr learned my addr 172.21.15.86:0/3065704629 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:43:06.417 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:58.443+0000 7ff3c3400700 1 -- 172.21.15.86:0/3065704629 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3c4079920 con 0x7ff3c407adb0 2024-03-23T17:43:06.417 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:58.443+0000 7ff3c3400700 1 --2- 172.21.15.86:0/3065704629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3c407adb0 0x7ff3c4079210 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff3b40098a0 tx=0x7ff3b4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3a0b3a5a86463c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:43:06.418 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:58.443+0000 7ff3c2a00700 1 -- 172.21.15.86:0/3065704629 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff3b4004160 con 0x7ff3c407adb0 2024-03-23T17:43:06.418 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:58.443+0000 7ff3c2a00700 1 -- 172.21.15.86:0/3065704629 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff3b40042c0 con 0x7ff3c407adb0 2024-03-23T17:43:06.418 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:58.443+0000 7ff3c2a00700 1 -- 172.21.15.86:0/3065704629 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff3b4011a00 con 0x7ff3c407adb0 2024-03-23T17:43:06.418 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:58.444+0000 7ff3ca400700 1 -- 172.21.15.86:0/3065704629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3c407adb0 msgr2=0x7ff3c4079210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:43:06.418 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:58.444+0000 7ff3ca400700 1 --2- 172.21.15.86:0/3065704629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3c407adb0 0x7ff3c4079210 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff3b40098a0 tx=0x7ff3b4009bb0 comp rx=0 tx=0).stop 2024-03-23T17:43:06.418 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:58.444+0000 7ff3ca400700 1 -- 172.21.15.86:0/3065704629 shutdown_connections 2024-03-23T17:43:06.418 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:58.444+0000 7ff3ca400700 1 --2- 172.21.15.86:0/3065704629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3c407adb0 0x7ff3c4079210 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:06.418 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:58.444+0000 7ff3ca400700 1 -- 172.21.15.86:0/3065704629 >> 172.21.15.86:0/3065704629 conn(0x7ff3c4101b30 msgr2=0x7ff3c4103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:43:06.418 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:58.444+0000 7ff3ca400700 1 -- 172.21.15.86:0/3065704629 shutdown_connections 2024-03-23T17:43:06.418 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:58.444+0000 7ff3ca400700 1 -- 172.21.15.86:0/3065704629 wait complete. 2024-03-23T17:43:06.418 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:58.445+0000 7ff3ca400700 1 Processor -- start 2024-03-23T17:43:06.418 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:58.446+0000 7ff3ca400700 1 -- start start 2024-03-23T17:43:06.418 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:58.446+0000 7ff3ca400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3c407adb0 0x7ff3c419c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:43:06.419 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:58.446+0000 7ff3ca400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c419ca90 con 0x7ff3c407adb0 2024-03-23T17:43:06.419 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:58.446+0000 7ff3c3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3c407adb0 0x7ff3c419c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:43:06.419 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:58.446+0000 7ff3c3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3c407adb0 0x7ff3c419c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35392/0 (socket says 172.21.15.86:35392) 2024-03-23T17:43:06.419 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:58.446+0000 7ff3c3400700 1 -- 172.21.15.86:0/4128581560 learned_addr learned my addr 172.21.15.86:0/4128581560 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:43:06.419 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:58.446+0000 7ff3c3400700 1 -- 172.21.15.86:0/4128581560 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3b4009550 con 0x7ff3c407adb0 2024-03-23T17:43:06.419 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:58.447+0000 7ff3c3400700 1 --2- 172.21.15.86:0/4128581560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3c407adb0 0x7ff3c419c550 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff3b4009130 tx=0x7ff3b4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:43:06.419 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:58.447+0000 7ff3c0c00700 1 -- 172.21.15.86:0/4128581560 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff3b4003ab0 con 0x7ff3c407adb0 2024-03-23T17:43:06.419 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:58.447+0000 7ff3c0c00700 1 -- 172.21.15.86:0/4128581560 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff3b4003c10 con 0x7ff3c407adb0 2024-03-23T17:43:06.419 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:58.447+0000 7ff3c0c00700 1 -- 172.21.15.86:0/4128581560 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff3b40247d0 con 0x7ff3c407adb0 2024-03-23T17:43:06.419 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:58.447+0000 7ff3ca400700 1 -- 172.21.15.86:0/4128581560 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3c419cc90 con 0x7ff3c407adb0 2024-03-23T17:43:06.419 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:58.447+0000 7ff3ca400700 1 -- 172.21.15.86:0/4128581560 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3c419d130 con 0x7ff3c407adb0 2024-03-23T17:43:06.419 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:58.448+0000 7ff3c0c00700 1 -- 172.21.15.86:0/4128581560 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) v1 ==== 45533+0+0 (secure 0 0 0) 0x7ff3b401a430 con 0x7ff3c407adb0 2024-03-23T17:43:06.419 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:58.449+0000 7ff3c0c00700 1 --2- 172.21.15.86:0/4128581560 >> [v2:172.21.15.86:6800/2313723492,v1:172.21.15.86:6801/2313723492] conn(0x7ff3a8038750 0x7ff3a803ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:43:06.419 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:58.449+0000 7ff3c0c00700 1 -- 172.21.15.86:0/4128581560 --> [v2:172.21.15.86:6800/2313723492,v1:172.21.15.86:6801/2313723492] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff3a803b320 con 0x7ff3a8038750 2024-03-23T17:43:06.420 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:58.449+0000 7ff3c0c00700 1 -- 172.21.15.86:0/4128581560 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff3b404c440 con 0x7ff3c407adb0 2024-03-23T17:43:06.420 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:58.449+0000 7ff3bbe00700 1 -- 172.21.15.86:0/4128581560 >> [v2:172.21.15.86:6800/2313723492,v1:172.21.15.86:6801/2313723492] conn(0x7ff3a8038750 msgr2=0x7ff3a803ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/2313723492 2024-03-23T17:43:06.420 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:58.449+0000 7ff3bbe00700 1 --2- 172.21.15.86:0/4128581560 >> [v2:172.21.15.86:6800/2313723492,v1:172.21.15.86:6801/2313723492] conn(0x7ff3a8038750 0x7ff3a803ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-23T17:43:06.420 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:58.650+0000 7ff3bbe00700 1 -- 172.21.15.86:0/4128581560 >> [v2:172.21.15.86:6800/2313723492,v1:172.21.15.86:6801/2313723492] conn(0x7ff3a8038750 msgr2=0x7ff3a803ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/2313723492 2024-03-23T17:43:06.420 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:58.650+0000 7ff3bbe00700 1 --2- 172.21.15.86:0/4128581560 >> [v2:172.21.15.86:6800/2313723492,v1:172.21.15.86:6801/2313723492] conn(0x7ff3a8038750 0x7ff3a803ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-23T17:43:06.420 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:59.051+0000 7ff3bbe00700 1 -- 172.21.15.86:0/4128581560 >> [v2:172.21.15.86:6800/2313723492,v1:172.21.15.86:6801/2313723492] conn(0x7ff3a8038750 msgr2=0x7ff3a803ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/2313723492 2024-03-23T17:43:06.420 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:59.051+0000 7ff3bbe00700 1 --2- 172.21.15.86:0/4128581560 >> [v2:172.21.15.86:6800/2313723492,v1:172.21.15.86:6801/2313723492] conn(0x7ff3a8038750 0x7ff3a803ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-23T17:43:06.420 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:59.852+0000 7ff3bbe00700 1 -- 172.21.15.86:0/4128581560 >> [v2:172.21.15.86:6800/2313723492,v1:172.21.15.86:6801/2313723492] conn(0x7ff3a8038750 msgr2=0x7ff3a803ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/2313723492 2024-03-23T17:43:06.420 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:42:59.852+0000 7ff3bbe00700 1 --2- 172.21.15.86:0/4128581560 >> [v2:172.21.15.86:6800/2313723492,v1:172.21.15.86:6801/2313723492] conn(0x7ff3a8038750 0x7ff3a803ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-23T17:43:06.420 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:01.454+0000 7ff3bbe00700 1 -- 172.21.15.86:0/4128581560 >> [v2:172.21.15.86:6800/2313723492,v1:172.21.15.86:6801/2313723492] conn(0x7ff3a8038750 msgr2=0x7ff3a803ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/2313723492 2024-03-23T17:43:06.420 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:01.454+0000 7ff3bbe00700 1 --2- 172.21.15.86:0/4128581560 >> [v2:172.21.15.86:6800/2313723492,v1:172.21.15.86:6801/2313723492] conn(0x7ff3a8038750 0x7ff3a803ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-23T17:43:06.420 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:04.657+0000 7ff3bbe00700 1 -- 172.21.15.86:0/4128581560 >> [v2:172.21.15.86:6800/2313723492,v1:172.21.15.86:6801/2313723492] conn(0x7ff3a8038750 msgr2=0x7ff3a803ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/2313723492 2024-03-23T17:43:06.420 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:04.657+0000 7ff3bbe00700 1 --2- 172.21.15.86:0/4128581560 >> [v2:172.21.15.86:6800/2313723492,v1:172.21.15.86:6801/2313723492] conn(0x7ff3a8038750 0x7ff3a803ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-03-23T17:43:06.421 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:05.045+0000 7ff3c0c00700 1 -- 172.21.15.86:0/4128581560 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mgrmap(e 6) v1 ==== 45300+0+0 (secure 0 0 0) 0x7ff3b401abb0 con 0x7ff3c407adb0 2024-03-23T17:43:06.421 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:05.045+0000 7ff3c0c00700 1 -- 172.21.15.86:0/4128581560 >> [v2:172.21.15.86:6800/2313723492,v1:172.21.15.86:6801/2313723492] conn(0x7ff3a8038750 msgr2=0x7ff3a803ac10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-23T17:43:06.421 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:05.045+0000 7ff3c0c00700 1 --2- 172.21.15.86:0/4128581560 >> [v2:172.21.15.86:6800/2313723492,v1:172.21.15.86:6801/2313723492] conn(0x7ff3a8038750 0x7ff3a803ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:06.421 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:06.047+0000 7ff3c0c00700 1 -- 172.21.15.86:0/4128581560 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mgrmap(e 7) v1 ==== 45427+0+0 (secure 0 0 0) 0x7ff3b404e1f0 con 0x7ff3c407adb0 2024-03-23T17:43:06.421 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:06.047+0000 7ff3c0c00700 1 --2- 172.21.15.86:0/4128581560 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7ff3a803bfe0 0x7ff3a803e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:43:06.421 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:06.047+0000 7ff3c0c00700 1 -- 172.21.15.86:0/4128581560 --> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff3a803b320 con 0x7ff3a803bfe0 2024-03-23T17:43:06.421 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:06.048+0000 7ff3bbe00700 1 --2- 172.21.15.86:0/4128581560 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7ff3a803bfe0 0x7ff3a803e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:43:06.421 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:06.050+0000 7ff3bbe00700 1 --2- 172.21.15.86:0/4128581560 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7ff3a803bfe0 0x7ff3a803e3d0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac003a10 tx=0x7ff3ac0092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:43:06.421 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:06.050+0000 7ff3c0c00700 1 -- 172.21.15.86:0/4128581560 <== mgr.14120 v2:172.21.15.86:6800/3600278228 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7ff3a803b320 con 0x7ff3a803bfe0 2024-03-23T17:43:06.421 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:06.057+0000 7ff3ca400700 1 -- 172.21.15.86:0/4128581560 --> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7ff3c410c980 con 0x7ff3a803bfe0 2024-03-23T17:43:06.421 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:06.057+0000 7ff3c0c00700 1 -- 172.21.15.86:0/4128581560 <== mgr.14120 v2:172.21.15.86:6800/3600278228 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7ff3c410c980 con 0x7ff3a803bfe0 2024-03-23T17:43:06.421 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:06.057+0000 7ff3ca400700 1 -- 172.21.15.86:0/4128581560 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7ff3a803bfe0 msgr2=0x7ff3a803e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:43:06.421 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:06.057+0000 7ff3ca400700 1 --2- 172.21.15.86:0/4128581560 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7ff3a803bfe0 0x7ff3a803e3d0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac003a10 tx=0x7ff3ac0092b0 comp rx=0 tx=0).stop 2024-03-23T17:43:06.421 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:06.057+0000 7ff3ca400700 1 -- 172.21.15.86:0/4128581560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3c407adb0 msgr2=0x7ff3c419c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:43:06.422 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:06.057+0000 7ff3ca400700 1 --2- 172.21.15.86:0/4128581560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3c407adb0 0x7ff3c419c550 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff3b4009130 tx=0x7ff3b4011ad0 comp rx=0 tx=0).stop 2024-03-23T17:43:06.422 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:06.057+0000 7ff3ca400700 1 -- 172.21.15.86:0/4128581560 shutdown_connections 2024-03-23T17:43:06.422 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:06.057+0000 7ff3ca400700 1 --2- 172.21.15.86:0/4128581560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3c407adb0 0x7ff3c419c550 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:06.422 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:06.057+0000 7ff3ca400700 1 --2- 172.21.15.86:0/4128581560 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7ff3a803bfe0 0x7ff3a803e3d0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:06.422 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:06.057+0000 7ff3ca400700 1 --2- 172.21.15.86:0/4128581560 >> [v2:172.21.15.86:6800/2313723492,v1:172.21.15.86:6801/2313723492] conn(0x7ff3a8038750 0x7ff3a803ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:06.422 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:06.057+0000 7ff3ca400700 1 -- 172.21.15.86:0/4128581560 >> 172.21.15.86:0/4128581560 conn(0x7ff3c4101b30 msgr2=0x7ff3c4102730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:43:06.422 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:06.057+0000 7ff3ca400700 1 -- 172.21.15.86:0/4128581560 shutdown_connections 2024-03-23T17:43:06.422 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:06.058+0000 7ff3ca400700 1 -- 172.21.15.86:0/4128581560 wait complete. 2024-03-23T17:43:06.422 INFO:teuthology.orchestra.run.smithi086.stdout:mgr epoch 5 is available 2024-03-23T17:43:06.422 INFO:teuthology.orchestra.run.smithi086.stdout:Setting orchestrator backend to cephadm... 2024-03-23T17:43:07.419 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:06.815+0000 7f2503400700 1 Processor -- start 2024-03-23T17:43:07.420 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:06.815+0000 7f2503400700 1 -- start start 2024-03-23T17:43:07.420 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:06.815+0000 7f2503400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24fc07ad90 0x7f24fc0791f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:43:07.420 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:06.815+0000 7f2503400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24fc0797c0 con 0x7f24fc07ad90 2024-03-23T17:43:07.420 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:06.815+0000 7f2500a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24fc07ad90 0x7f24fc0791f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:43:07.420 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:06.815+0000 7f2500a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24fc07ad90 0x7f24fc0791f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42674/0 (socket says 172.21.15.86:42674) 2024-03-23T17:43:07.420 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:06.815+0000 7f2500a00700 1 -- 172.21.15.86:0/2803076573 learned_addr learned my addr 172.21.15.86:0/2803076573 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:43:07.420 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:06.816+0000 7f2500a00700 1 -- 172.21.15.86:0/2803076573 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24fc079900 con 0x7f24fc07ad90 2024-03-23T17:43:07.420 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:06.816+0000 7f2500a00700 1 --2- 172.21.15.86:0/2803076573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24fc07ad90 0x7f24fc0791f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f24f00098a0 tx=0x7f24f0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eaef46fa5a9c291 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:43:07.420 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:06.816+0000 7f24fb400700 1 -- 172.21.15.86:0/2803076573 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f24f0004160 con 0x7f24fc07ad90 2024-03-23T17:43:07.420 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:06.816+0000 7f24fb400700 1 -- 172.21.15.86:0/2803076573 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f24f00042c0 con 0x7f24fc07ad90 2024-03-23T17:43:07.420 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:06.816+0000 7f24fb400700 1 -- 172.21.15.86:0/2803076573 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f24f0011a00 con 0x7f24fc07ad90 2024-03-23T17:43:07.420 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:06.817+0000 7f2503400700 1 -- 172.21.15.86:0/2803076573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24fc07ad90 msgr2=0x7f24fc0791f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:43:07.421 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:06.817+0000 7f2503400700 1 --2- 172.21.15.86:0/2803076573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24fc07ad90 0x7f24fc0791f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f24f00098a0 tx=0x7f24f0009bb0 comp rx=0 tx=0).stop 2024-03-23T17:43:07.421 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:06.817+0000 7f2503400700 1 -- 172.21.15.86:0/2803076573 shutdown_connections 2024-03-23T17:43:07.421 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:06.817+0000 7f2503400700 1 --2- 172.21.15.86:0/2803076573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24fc07ad90 0x7f24fc0791f0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:07.421 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:06.817+0000 7f2503400700 1 -- 172.21.15.86:0/2803076573 >> 172.21.15.86:0/2803076573 conn(0x7f24fc101b90 msgr2=0x7f24fc103fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:43:07.421 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:06.817+0000 7f2503400700 1 -- 172.21.15.86:0/2803076573 shutdown_connections 2024-03-23T17:43:07.421 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:06.817+0000 7f2503400700 1 -- 172.21.15.86:0/2803076573 wait complete. 2024-03-23T17:43:07.421 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:06.818+0000 7f2503400700 1 Processor -- start 2024-03-23T17:43:07.421 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:06.818+0000 7f2503400700 1 -- start start 2024-03-23T17:43:07.421 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:06.819+0000 7f2503400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24fc07ad90 0x7f24fc19c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:43:07.421 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:06.819+0000 7f2503400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24fc19ca30 con 0x7f24fc07ad90 2024-03-23T17:43:07.421 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:06.819+0000 7f2500a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24fc07ad90 0x7f24fc19c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:43:07.421 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:06.819+0000 7f2500a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24fc07ad90 0x7f24fc19c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42678/0 (socket says 172.21.15.86:42678) 2024-03-23T17:43:07.421 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:06.819+0000 7f2500a00700 1 -- 172.21.15.86:0/3843517534 learned_addr learned my addr 172.21.15.86:0/3843517534 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:43:07.421 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:06.819+0000 7f2500a00700 1 -- 172.21.15.86:0/3843517534 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24f0009550 con 0x7f24fc07ad90 2024-03-23T17:43:07.421 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:06.819+0000 7f2500a00700 1 --2- 172.21.15.86:0/3843517534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24fc07ad90 0x7f24fc19c4f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f24f0009130 tx=0x7f24f0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:43:07.422 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:06.820+0000 7f24f9600700 1 -- 172.21.15.86:0/3843517534 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f24f0003b70 con 0x7f24fc07ad90 2024-03-23T17:43:07.422 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:06.820+0000 7f24f9600700 1 -- 172.21.15.86:0/3843517534 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f24f0003cd0 con 0x7f24fc07ad90 2024-03-23T17:43:07.422 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:06.820+0000 7f2503400700 1 -- 172.21.15.86:0/3843517534 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24fc19cc30 con 0x7f24fc07ad90 2024-03-23T17:43:07.422 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:06.820+0000 7f24f9600700 1 -- 172.21.15.86:0/3843517534 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f24f00197d0 con 0x7f24fc07ad90 2024-03-23T17:43:07.422 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:06.820+0000 7f2503400700 1 -- 172.21.15.86:0/3843517534 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24fc19d0d0 con 0x7f24fc07ad90 2024-03-23T17:43:07.422 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:06.821+0000 7f24f9600700 1 -- 172.21.15.86:0/3843517534 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 7) v1 ==== 45427+0+0 (secure 0 0 0) 0x7f24f001a430 con 0x7f24fc07ad90 2024-03-23T17:43:07.422 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:06.821+0000 7f2503400700 1 -- 172.21.15.86:0/3843517534 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24fc10c9a0 con 0x7f24fc07ad90 2024-03-23T17:43:07.422 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:06.821+0000 7f24f9600700 1 --2- 172.21.15.86:0/3843517534 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7f24e4038620 0x7f24e403aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:43:07.422 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:06.822+0000 7f24f9600700 1 -- 172.21.15.86:0/3843517534 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f24f004c720 con 0x7f24fc07ad90 2024-03-23T17:43:07.422 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:06.822+0000 7f24fbe00700 1 --2- 172.21.15.86:0/3843517534 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7f24e4038620 0x7f24e403aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:43:07.422 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:06.822+0000 7f24fbe00700 1 --2- 172.21.15.86:0/3843517534 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7f24e4038620 0x7f24e403aae0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f24e8006fd0 tx=0x7f24e8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:43:07.422 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:06.827+0000 7f24f9600700 1 -- 172.21.15.86:0/3843517534 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f24f002d9a0 con 0x7f24fc07ad90 2024-03-23T17:43:07.422 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:07.044+0000 7f2503400700 1 -- 172.21.15.86:0/3843517534 --> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f24fc102e10 con 0x7f24e4038620 2024-03-23T17:43:07.423 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:07.052+0000 7f24f9600700 1 -- 172.21.15.86:0/3843517534 <== mgr.14120 v2:172.21.15.86:6800/3600278228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f24fc102e10 con 0x7f24e4038620 2024-03-23T17:43:07.423 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:07.055+0000 7f2503400700 1 -- 172.21.15.86:0/3843517534 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7f24e4038620 msgr2=0x7f24e403aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:43:07.423 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:07.055+0000 7f2503400700 1 --2- 172.21.15.86:0/3843517534 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7f24e4038620 0x7f24e403aae0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f24e8006fd0 tx=0x7f24e8006e40 comp rx=0 tx=0).stop 2024-03-23T17:43:07.423 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:07.055+0000 7f2503400700 1 -- 172.21.15.86:0/3843517534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24fc07ad90 msgr2=0x7f24fc19c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:43:07.423 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:07.055+0000 7f2503400700 1 --2- 172.21.15.86:0/3843517534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24fc07ad90 0x7f24fc19c4f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f24f0009130 tx=0x7f24f0011ad0 comp rx=0 tx=0).stop 2024-03-23T17:43:07.423 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:07.055+0000 7f2503400700 1 -- 172.21.15.86:0/3843517534 shutdown_connections 2024-03-23T17:43:07.423 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:07.055+0000 7f2503400700 1 --2- 172.21.15.86:0/3843517534 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7f24e4038620 0x7f24e403aae0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:07.423 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:07.055+0000 7f2503400700 1 --2- 172.21.15.86:0/3843517534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24fc07ad90 0x7f24fc19c4f0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:07.423 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:07.055+0000 7f2503400700 1 -- 172.21.15.86:0/3843517534 >> 172.21.15.86:0/3843517534 conn(0x7f24fc101b90 msgr2=0x7f24fc102700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:43:07.423 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:07.055+0000 7f2503400700 1 -- 172.21.15.86:0/3843517534 shutdown_connections 2024-03-23T17:43:07.423 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:07.055+0000 7f2503400700 1 -- 172.21.15.86:0/3843517534 wait complete. 2024-03-23T17:43:07.424 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:07 smithi086 ceph-mon[38221]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-23T17:43:07.424 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:07 smithi086 ceph-mon[38221]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-23T17:43:07.424 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:07 smithi086 ceph-mon[38221]: [23/Mar/2024:17:43:06] ENGINE Bus STARTING 2024-03-23T17:43:07.424 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:07 smithi086 ceph-mon[38221]: from='mgr.14120 172.21.15.86:0/1645921427' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:43:07.424 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:07 smithi086 ceph-mon[38221]: from='mgr.14120 172.21.15.86:0/1645921427' entity='mgr.x' 2024-03-23T17:43:07.424 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:07 smithi086 ceph-mon[38221]: from='mgr.14120 172.21.15.86:0/1645921427' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:43:08.369 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout value unchanged 2024-03-23T17:43:08.369 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:07.797+0000 7f62d7000700 1 Processor -- start 2024-03-23T17:43:08.369 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:07.797+0000 7f62d7000700 1 -- start start 2024-03-23T17:43:08.370 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:07.797+0000 7f62d7000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62d01065b0 0x7f62d01069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:43:08.370 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:08 smithi086 ceph-mon[38221]: [23/Mar/2024:17:43:06] ENGINE Serving on https://172.21.15.86:7150 2024-03-23T17:43:08.370 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:08 smithi086 ceph-mon[38221]: [23/Mar/2024:17:43:06] ENGINE Serving on http://172.21.15.86:8765 2024-03-23T17:43:08.370 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:08 smithi086 ceph-mon[38221]: [23/Mar/2024:17:43:06] ENGINE Bus STARTED 2024-03-23T17:43:08.370 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:08 smithi086 ceph-mon[38221]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:43:08.370 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:08 smithi086 ceph-mon[38221]: mgrmap e8: x(active, since 2s) 2024-03-23T17:43:08.375 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:07.797+0000 7f62d7000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d0106fa0 con 0x7f62d01065b0 2024-03-23T17:43:08.375 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:07.797+0000 7f62cfe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62d01065b0 0x7f62d01069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:43:08.375 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:07.798+0000 7f62cfe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62d01065b0 0x7f62d01069d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42690/0 (socket says 172.21.15.86:42690) 2024-03-23T17:43:08.375 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:07.798+0000 7f62cfe00700 1 -- 172.21.15.86:0/568076441 learned_addr learned my addr 172.21.15.86:0/568076441 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:43:08.375 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:07.798+0000 7f62cfe00700 1 -- 172.21.15.86:0/568076441 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62d01077b0 con 0x7f62d01065b0 2024-03-23T17:43:08.375 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:07.798+0000 7f62cfe00700 1 --2- 172.21.15.86:0/568076441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62d01065b0 0x7f62d01069d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f62c00098a0 tx=0x7f62c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6918c0c48bcdfa59 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:43:08.375 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:07.798+0000 7f62cea00700 1 -- 172.21.15.86:0/568076441 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f62c0004160 con 0x7f62d01065b0 2024-03-23T17:43:08.375 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:07.798+0000 7f62cea00700 1 -- 172.21.15.86:0/568076441 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f62c00042c0 con 0x7f62d01065b0 2024-03-23T17:43:08.375 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:07.798+0000 7f62cea00700 1 -- 172.21.15.86:0/568076441 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f62c0011a00 con 0x7f62d01065b0 2024-03-23T17:43:08.376 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:07.799+0000 7f62d7000700 1 -- 172.21.15.86:0/568076441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62d01065b0 msgr2=0x7f62d01069d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:43:08.376 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:07.799+0000 7f62d7000700 1 --2- 172.21.15.86:0/568076441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62d01065b0 0x7f62d01069d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f62c00098a0 tx=0x7f62c0009bb0 comp rx=0 tx=0).stop 2024-03-23T17:43:08.376 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:07.799+0000 7f62d7000700 1 -- 172.21.15.86:0/568076441 shutdown_connections 2024-03-23T17:43:08.376 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:07.799+0000 7f62d7000700 1 --2- 172.21.15.86:0/568076441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62d01065b0 0x7f62d01069d0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:08.376 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:07.799+0000 7f62d7000700 1 -- 172.21.15.86:0/568076441 >> 172.21.15.86:0/568076441 conn(0x7f62d0101b70 msgr2=0x7f62d0103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:43:08.376 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:07.799+0000 7f62d7000700 1 -- 172.21.15.86:0/568076441 shutdown_connections 2024-03-23T17:43:08.376 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:07.799+0000 7f62d7000700 1 -- 172.21.15.86:0/568076441 wait complete. 2024-03-23T17:43:08.376 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:07.800+0000 7f62d7000700 1 Processor -- start 2024-03-23T17:43:08.376 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:07.800+0000 7f62d7000700 1 -- start start 2024-03-23T17:43:08.376 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:07.800+0000 7f62d7000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62d01065b0 0x7f62d019c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:43:08.376 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:07.800+0000 7f62d7000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d019ca90 con 0x7f62d01065b0 2024-03-23T17:43:08.376 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:07.801+0000 7f62cfe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62d01065b0 0x7f62d019c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:43:08.376 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:07.801+0000 7f62cfe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62d01065b0 0x7f62d019c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42704/0 (socket says 172.21.15.86:42704) 2024-03-23T17:43:08.376 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:07.801+0000 7f62cfe00700 1 -- 172.21.15.86:0/114346931 learned_addr learned my addr 172.21.15.86:0/114346931 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:43:08.377 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:07.801+0000 7f62cfe00700 1 -- 172.21.15.86:0/114346931 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62c0009550 con 0x7f62d01065b0 2024-03-23T17:43:08.377 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:07.801+0000 7f62cfe00700 1 --2- 172.21.15.86:0/114346931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62d01065b0 0x7f62d019c550 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f62c0009130 tx=0x7f62c0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:43:08.377 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:07.801+0000 7f62ccc00700 1 -- 172.21.15.86:0/114346931 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f62c0003ab0 con 0x7f62d01065b0 2024-03-23T17:43:08.377 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:07.802+0000 7f62ccc00700 1 -- 172.21.15.86:0/114346931 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f62c0003c10 con 0x7f62d01065b0 2024-03-23T17:43:08.377 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:07.802+0000 7f62d7000700 1 -- 172.21.15.86:0/114346931 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62d019cc90 con 0x7f62d01065b0 2024-03-23T17:43:08.377 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:07.802+0000 7f62ccc00700 1 -- 172.21.15.86:0/114346931 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f62c00247d0 con 0x7f62d01065b0 2024-03-23T17:43:08.377 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:07.802+0000 7f62d7000700 1 -- 172.21.15.86:0/114346931 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62d019d130 con 0x7f62d01065b0 2024-03-23T17:43:08.377 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:07.803+0000 7f62ccc00700 1 -- 172.21.15.86:0/114346931 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 8) v1 ==== 45533+0+0 (secure 0 0 0) 0x7f62c001a430 con 0x7f62d01065b0 2024-03-23T17:43:08.377 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:07.803+0000 7f62d7000700 1 -- 172.21.15.86:0/114346931 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62d010a9c0 con 0x7f62d01065b0 2024-03-23T17:43:08.377 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:07.803+0000 7f62ccc00700 1 --2- 172.21.15.86:0/114346931 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7f62b4038700 0x7f62b403abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:43:08.377 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:07.803+0000 7f62ccc00700 1 -- 172.21.15.86:0/114346931 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f62c004c8d0 con 0x7f62d01065b0 2024-03-23T17:43:08.377 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:07.803+0000 7f62cf400700 1 --2- 172.21.15.86:0/114346931 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7f62b4038700 0x7f62b403abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:43:08.377 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:07.804+0000 7f62cf400700 1 --2- 172.21.15.86:0/114346931 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7f62b4038700 0x7f62b403abc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f62bc006fd0 tx=0x7f62bc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:43:08.377 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:07.808+0000 7f62ccc00700 1 -- 172.21.15.86:0/114346931 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f62c0018530 con 0x7f62d01065b0 2024-03-23T17:43:08.378 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:08.009+0000 7f62d7000700 1 -- 172.21.15.86:0/114346931 --> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f62d0102e10 con 0x7f62b4038700 2024-03-23T17:43:08.378 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:08.010+0000 7f62ccc00700 1 -- 172.21.15.86:0/114346931 <== mgr.14120 v2:172.21.15.86:6800/3600278228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f62d0102e10 con 0x7f62b4038700 2024-03-23T17:43:08.378 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:08.012+0000 7f62d7000700 1 -- 172.21.15.86:0/114346931 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7f62b4038700 msgr2=0x7f62b403abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:43:08.378 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:08.012+0000 7f62d7000700 1 --2- 172.21.15.86:0/114346931 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7f62b4038700 0x7f62b403abc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f62bc006fd0 tx=0x7f62bc006e40 comp rx=0 tx=0).stop 2024-03-23T17:43:08.378 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:08.012+0000 7f62d7000700 1 -- 172.21.15.86:0/114346931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62d01065b0 msgr2=0x7f62d019c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:43:08.378 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:08.012+0000 7f62d7000700 1 --2- 172.21.15.86:0/114346931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62d01065b0 0x7f62d019c550 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f62c0009130 tx=0x7f62c0011ad0 comp rx=0 tx=0).stop 2024-03-23T17:43:08.378 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:08.013+0000 7f62d7000700 1 -- 172.21.15.86:0/114346931 shutdown_connections 2024-03-23T17:43:08.378 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:08.013+0000 7f62d7000700 1 --2- 172.21.15.86:0/114346931 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7f62b4038700 0x7f62b403abc0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:08.378 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:08.013+0000 7f62d7000700 1 --2- 172.21.15.86:0/114346931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62d01065b0 0x7f62d019c550 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:08.378 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:08.013+0000 7f62d7000700 1 -- 172.21.15.86:0/114346931 >> 172.21.15.86:0/114346931 conn(0x7f62d0101b70 msgr2=0x7f62d0102700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:43:08.378 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:08.013+0000 7f62d7000700 1 -- 172.21.15.86:0/114346931 shutdown_connections 2024-03-23T17:43:08.378 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:08.013+0000 7f62d7000700 1 -- 172.21.15.86:0/114346931 wait complete. 2024-03-23T17:43:08.378 INFO:teuthology.orchestra.run.smithi086.stdout:Generating ssh key... 2024-03-23T17:43:09.420 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:08.760+0000 7f2f01400700 1 Processor -- start 2024-03-23T17:43:09.420 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:08.761+0000 7f2f01400700 1 -- start start 2024-03-23T17:43:09.421 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:08.761+0000 7f2f01400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2efc07adb0 0x7f2efc079210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:43:09.421 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:08.761+0000 7f2f01400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2efc0797e0 con 0x7f2efc07adb0 2024-03-23T17:43:09.421 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:08.761+0000 7f2efa800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2efc07adb0 0x7f2efc079210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:43:09.421 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:08.761+0000 7f2efa800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2efc07adb0 0x7f2efc079210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42712/0 (socket says 172.21.15.86:42712) 2024-03-23T17:43:09.421 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:08.761+0000 7f2efa800700 1 -- 172.21.15.86:0/3240092825 learned_addr learned my addr 172.21.15.86:0/3240092825 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:43:09.421 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:08.762+0000 7f2efa800700 1 -- 172.21.15.86:0/3240092825 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2efc079920 con 0x7f2efc07adb0 2024-03-23T17:43:09.421 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:08.762+0000 7f2efa800700 1 --2- 172.21.15.86:0/3240092825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2efc07adb0 0x7f2efc079210 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2eec0098a0 tx=0x7f2eec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9140b76e1a94b055 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:43:09.421 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:08.762+0000 7f2ef9e00700 1 -- 172.21.15.86:0/3240092825 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2eec004160 con 0x7f2efc07adb0 2024-03-23T17:43:09.421 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:08.762+0000 7f2ef9e00700 1 -- 172.21.15.86:0/3240092825 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2eec0042c0 con 0x7f2efc07adb0 2024-03-23T17:43:09.421 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:08.762+0000 7f2ef9e00700 1 -- 172.21.15.86:0/3240092825 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2eec011a00 con 0x7f2efc07adb0 2024-03-23T17:43:09.421 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:08.763+0000 7f2f01400700 1 -- 172.21.15.86:0/3240092825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2efc07adb0 msgr2=0x7f2efc079210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:43:09.421 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:08.763+0000 7f2f01400700 1 --2- 172.21.15.86:0/3240092825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2efc07adb0 0x7f2efc079210 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2eec0098a0 tx=0x7f2eec009bb0 comp rx=0 tx=0).stop 2024-03-23T17:43:09.422 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:08.763+0000 7f2f01400700 1 -- 172.21.15.86:0/3240092825 shutdown_connections 2024-03-23T17:43:09.422 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:08.763+0000 7f2f01400700 1 --2- 172.21.15.86:0/3240092825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2efc07adb0 0x7f2efc079210 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:09.422 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:08.763+0000 7f2f01400700 1 -- 172.21.15.86:0/3240092825 >> 172.21.15.86:0/3240092825 conn(0x7f2efc101b30 msgr2=0x7f2efc103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:43:09.422 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:08.763+0000 7f2f01400700 1 -- 172.21.15.86:0/3240092825 shutdown_connections 2024-03-23T17:43:09.422 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:08.763+0000 7f2f01400700 1 -- 172.21.15.86:0/3240092825 wait complete. 2024-03-23T17:43:09.422 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:08.764+0000 7f2f01400700 1 Processor -- start 2024-03-23T17:43:09.422 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:08.764+0000 7f2f01400700 1 -- start start 2024-03-23T17:43:09.422 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:08.764+0000 7f2f01400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2efc07adb0 0x7f2efc19c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:43:09.422 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:08.764+0000 7f2f01400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2efc19ca90 con 0x7f2efc07adb0 2024-03-23T17:43:09.422 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:08.765+0000 7f2efa800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2efc07adb0 0x7f2efc19c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:43:09.422 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:08.765+0000 7f2efa800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2efc07adb0 0x7f2efc19c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42716/0 (socket says 172.21.15.86:42716) 2024-03-23T17:43:09.422 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:08.765+0000 7f2efa800700 1 -- 172.21.15.86:0/3574018093 learned_addr learned my addr 172.21.15.86:0/3574018093 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:43:09.422 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:08.765+0000 7f2efa800700 1 -- 172.21.15.86:0/3574018093 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eec009550 con 0x7f2efc07adb0 2024-03-23T17:43:09.423 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:08.765+0000 7f2efa800700 1 --2- 172.21.15.86:0/3574018093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2efc07adb0 0x7f2efc19c550 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2eec009130 tx=0x7f2eec011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:43:09.423 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:08.765+0000 7f2ef3400700 1 -- 172.21.15.86:0/3574018093 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2eec003ab0 con 0x7f2efc07adb0 2024-03-23T17:43:09.423 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:08.765+0000 7f2ef3400700 1 -- 172.21.15.86:0/3574018093 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2eec003c10 con 0x7f2efc07adb0 2024-03-23T17:43:09.423 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:08.765+0000 7f2ef3400700 1 -- 172.21.15.86:0/3574018093 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2eec0247d0 con 0x7f2efc07adb0 2024-03-23T17:43:09.423 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:08.765+0000 7f2f01400700 1 -- 172.21.15.86:0/3574018093 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2efc19cc90 con 0x7f2efc07adb0 2024-03-23T17:43:09.423 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:08.766+0000 7f2f01400700 1 -- 172.21.15.86:0/3574018093 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2efc19d130 con 0x7f2efc07adb0 2024-03-23T17:43:09.423 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:08.767+0000 7f2f01400700 1 -- 172.21.15.86:0/3574018093 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2efc10c980 con 0x7f2efc07adb0 2024-03-23T17:43:09.423 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:08.767+0000 7f2ef3400700 1 -- 172.21.15.86:0/3574018093 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 8) v1 ==== 45533+0+0 (secure 0 0 0) 0x7f2eec01abb0 con 0x7f2efc07adb0 2024-03-23T17:43:09.423 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:08.767+0000 7f2ef3400700 1 --2- 172.21.15.86:0/3574018093 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7f2ee0038750 0x7f2ee003ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:43:09.423 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:08.767+0000 7f2ef3400700 1 -- 172.21.15.86:0/3574018093 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f2eec04c7c0 con 0x7f2efc07adb0 2024-03-23T17:43:09.423 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:08.768+0000 7f2ef3e00700 1 --2- 172.21.15.86:0/3574018093 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7f2ee0038750 0x7f2ee003ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:43:09.423 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:08.768+0000 7f2ef3e00700 1 --2- 172.21.15.86:0/3574018093 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7f2ee0038750 0x7f2ee003ac10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2ee4006fd0 tx=0x7f2ee4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:43:09.423 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:08.772+0000 7f2ef3400700 1 -- 172.21.15.86:0/3574018093 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f2eec018530 con 0x7f2efc07adb0 2024-03-23T17:43:09.424 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:08.972+0000 7f2f01400700 1 -- 172.21.15.86:0/3574018093 --> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f2efc102e40 con 0x7f2ee0038750 2024-03-23T17:43:09.424 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:09.058+0000 7f2ef3400700 1 -- 172.21.15.86:0/3574018093 <== mgr.14120 v2:172.21.15.86:6800/3600278228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f2efc102e40 con 0x7f2ee0038750 2024-03-23T17:43:09.424 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:09.060+0000 7f2f01400700 1 -- 172.21.15.86:0/3574018093 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7f2ee0038750 msgr2=0x7f2ee003ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:43:09.424 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:09.060+0000 7f2f01400700 1 --2- 172.21.15.86:0/3574018093 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7f2ee0038750 0x7f2ee003ac10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2ee4006fd0 tx=0x7f2ee4006e40 comp rx=0 tx=0).stop 2024-03-23T17:43:09.424 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:09.060+0000 7f2f01400700 1 -- 172.21.15.86:0/3574018093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2efc07adb0 msgr2=0x7f2efc19c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:43:09.424 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:09.060+0000 7f2f01400700 1 --2- 172.21.15.86:0/3574018093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2efc07adb0 0x7f2efc19c550 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2eec009130 tx=0x7f2eec011ad0 comp rx=0 tx=0).stop 2024-03-23T17:43:09.424 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:09.060+0000 7f2f01400700 1 -- 172.21.15.86:0/3574018093 shutdown_connections 2024-03-23T17:43:09.424 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:09.060+0000 7f2f01400700 1 --2- 172.21.15.86:0/3574018093 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7f2ee0038750 0x7f2ee003ac10 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:09.424 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:09.060+0000 7f2f01400700 1 --2- 172.21.15.86:0/3574018093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2efc07adb0 0x7f2efc19c550 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:09.424 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:09.060+0000 7f2f01400700 1 -- 172.21.15.86:0/3574018093 >> 172.21.15.86:0/3574018093 conn(0x7f2efc101b30 msgr2=0x7f2efc102730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:43:09.424 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:09.061+0000 7f2f01400700 1 -- 172.21.15.86:0/3574018093 shutdown_connections 2024-03-23T17:43:09.424 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:09.061+0000 7f2f01400700 1 -- 172.21.15.86:0/3574018093 wait complete. 2024-03-23T17:43:09.425 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:09 smithi086 ceph-mon[38221]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:43:09.425 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:09 smithi086 ceph-mon[38221]: from='mgr.14120 172.21.15.86:0/1645921427' entity='mgr.x' 2024-03-23T17:43:09.425 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:09 smithi086 ceph-mon[38221]: from='mgr.14120 172.21.15.86:0/1645921427' entity='mgr.x' 2024-03-23T17:43:09.425 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:43:09 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: Generating public/private rsa key pair. 2024-03-23T17:43:09.425 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:43:09 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: Your identification has been saved in /tmp/tmp1x_oxzmw/key. 2024-03-23T17:43:09.425 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:43:09 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: Your public key has been saved in /tmp/tmp1x_oxzmw/key.pub. 2024-03-23T17:43:09.425 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:43:09 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: The key fingerprint is: 2024-03-23T17:43:09.425 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:43:09 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: SHA256:1iOZGq0yIEmWnn24r0VB+YULI13GXV+WncuOQyWmfxg ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715 2024-03-23T17:43:09.425 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:43:09 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: The key's randomart image is: 2024-03-23T17:43:09.426 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:43:09 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: +---[RSA 3072]----+ 2024-03-23T17:43:09.426 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:43:09 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: | ..+oo .. o+| 2024-03-23T17:43:09.426 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:43:09 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: | ...=.o o .oo+.| 2024-03-23T17:43:09.426 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:43:09 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: | + ..+ o o.+ .| 2024-03-23T17:43:09.426 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:43:09 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: |+.o . .+ + . E o | 2024-03-23T17:43:09.426 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:43:09 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: |oo.o o. S o o = | 2024-03-23T17:43:09.426 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:43:09 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: | . .+ = . . = o | 2024-03-23T17:43:09.426 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:43:09 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: | .o.o o | 2024-03-23T17:43:09.426 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:43:09 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: | oo | 2024-03-23T17:43:09.426 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:43:09 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: | ... | 2024-03-23T17:43:09.426 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:43:09 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: +----[SHA256]-----+ 2024-03-23T17:43:10.384 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDYwXYO+EWZJ5hYLn4SJSleOGOPqplSRytD9ctra/UPcQx4z6u4uWRI2RYuIvPwdv+b9wd9q/CcS8nXbYnf8VZ/oDAcDm8F3yTCiJ56j/sg6qmkA1LPw0yohSRHVnfRW/U+x25zLr0eFoSjUZ/+7N2EV3pQLjIJTBPwCdx3A+GO1+9YapjmyA8WmnxhWxUpWiWQc7Dr/n3vTGOvHKfNXeYkwgmGPSz3Ju2k7AqhAsaa/R2IqgkoFfBgC+0v4jvpCJFOUQjaABaMzTN4/TNz6kR0VYYQlReooODjZ1PQXuGOzsywtAD8A9/IS3dvA6t23r1fsZqHwF8JAiwTv9aH0LueNZVYRrJVEOHlx6SExQ4lkHTwEBdwko2jUjV2gZzfbO/Yf7u9mRVhrYfpuPi5hb/D6CW3l381rPUHpqDJt7qZ2pFlNvgzX+lCDGpSNsfZ8RBY6p6P7SkbsLtl2rk7tAPjuqKAM1lOxG/SaspjRTcWEd4h+Tu2TcoQk+oTCQoleGE= ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715 2024-03-23T17:43:10.384 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:09.803+0000 7f79efe00700 1 Processor -- start 2024-03-23T17:43:10.384 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:09.804+0000 7f79efe00700 1 -- start start 2024-03-23T17:43:10.384 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:09.804+0000 7f79efe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79e807ad70 0x7f79e80791d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:43:10.384 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:09.804+0000 7f79efe00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79e80797a0 con 0x7f79e807ad70 2024-03-23T17:43:10.384 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:09.804+0000 7f79ed400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79e807ad70 0x7f79e80791d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:43:10.384 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:09.804+0000 7f79ed400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79e807ad70 0x7f79e80791d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42728/0 (socket says 172.21.15.86:42728) 2024-03-23T17:43:10.384 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:09.804+0000 7f79ed400700 1 -- 172.21.15.86:0/270933088 learned_addr learned my addr 172.21.15.86:0/270933088 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:43:10.385 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:09.805+0000 7f79ed400700 1 -- 172.21.15.86:0/270933088 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79e80798e0 con 0x7f79e807ad70 2024-03-23T17:43:10.385 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:09.805+0000 7f79ed400700 1 --2- 172.21.15.86:0/270933088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79e807ad70 0x7f79e80791d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f79dc0098a0 tx=0x7f79dc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=295757c52529dba6 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:43:10.385 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:09.805+0000 7f79e7e00700 1 -- 172.21.15.86:0/270933088 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f79dc004160 con 0x7f79e807ad70 2024-03-23T17:43:10.385 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:09.805+0000 7f79e7e00700 1 -- 172.21.15.86:0/270933088 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f79dc0042c0 con 0x7f79e807ad70 2024-03-23T17:43:10.385 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:09.805+0000 7f79e7e00700 1 -- 172.21.15.86:0/270933088 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f79dc011a00 con 0x7f79e807ad70 2024-03-23T17:43:10.385 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:09.806+0000 7f79efe00700 1 -- 172.21.15.86:0/270933088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79e807ad70 msgr2=0x7f79e80791d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:43:10.385 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:09.806+0000 7f79efe00700 1 --2- 172.21.15.86:0/270933088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79e807ad70 0x7f79e80791d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f79dc0098a0 tx=0x7f79dc009bb0 comp rx=0 tx=0).stop 2024-03-23T17:43:10.385 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:09.806+0000 7f79efe00700 1 -- 172.21.15.86:0/270933088 shutdown_connections 2024-03-23T17:43:10.385 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:09.806+0000 7f79efe00700 1 --2- 172.21.15.86:0/270933088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79e807ad70 0x7f79e80791d0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:10.385 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:09.806+0000 7f79efe00700 1 -- 172.21.15.86:0/270933088 >> 172.21.15.86:0/270933088 conn(0x7f79e8101b30 msgr2=0x7f79e8103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:43:10.385 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:09.806+0000 7f79efe00700 1 -- 172.21.15.86:0/270933088 shutdown_connections 2024-03-23T17:43:10.385 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:09.806+0000 7f79efe00700 1 -- 172.21.15.86:0/270933088 wait complete. 2024-03-23T17:43:10.385 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:09.807+0000 7f79efe00700 1 Processor -- start 2024-03-23T17:43:10.385 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:09.807+0000 7f79efe00700 1 -- start start 2024-03-23T17:43:10.385 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:09.807+0000 7f79efe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79e807ad70 0x7f79e819c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:43:10.386 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:09.807+0000 7f79efe00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79e819ca10 con 0x7f79e807ad70 2024-03-23T17:43:10.386 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:09.808+0000 7f79ed400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79e807ad70 0x7f79e819c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:43:10.386 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:09.808+0000 7f79ed400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79e807ad70 0x7f79e819c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42744/0 (socket says 172.21.15.86:42744) 2024-03-23T17:43:10.386 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:09.808+0000 7f79ed400700 1 -- 172.21.15.86:0/1493536372 learned_addr learned my addr 172.21.15.86:0/1493536372 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:43:10.386 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:09.808+0000 7f79ed400700 1 -- 172.21.15.86:0/1493536372 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79dc009550 con 0x7f79e807ad70 2024-03-23T17:43:10.386 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:09.808+0000 7f79ed400700 1 --2- 172.21.15.86:0/1493536372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79e807ad70 0x7f79e819c4d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f79dc009130 tx=0x7f79dc011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:43:10.386 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:09.808+0000 7f79e6000700 1 -- 172.21.15.86:0/1493536372 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f79dc003b70 con 0x7f79e807ad70 2024-03-23T17:43:10.386 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:10 smithi086 ceph-mon[38221]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:43:10.386 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:10 smithi086 ceph-mon[38221]: Generating ssh key... 2024-03-23T17:43:10.389 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:09.808+0000 7f79e6000700 1 -- 172.21.15.86:0/1493536372 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f79dc003cd0 con 0x7f79e807ad70 2024-03-23T17:43:10.389 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:09.808+0000 7f79efe00700 1 -- 172.21.15.86:0/1493536372 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79e819cc10 con 0x7f79e807ad70 2024-03-23T17:43:10.389 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:09.809+0000 7f79e6000700 1 -- 172.21.15.86:0/1493536372 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f79dc0197d0 con 0x7f79e807ad70 2024-03-23T17:43:10.390 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:09.809+0000 7f79efe00700 1 -- 172.21.15.86:0/1493536372 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79e819d0b0 con 0x7f79e807ad70 2024-03-23T17:43:10.390 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:09.810+0000 7f79e6000700 1 -- 172.21.15.86:0/1493536372 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 8) v1 ==== 45533+0+0 (secure 0 0 0) 0x7f79dc01abb0 con 0x7f79e807ad70 2024-03-23T17:43:10.390 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:09.810+0000 7f79efe00700 1 -- 172.21.15.86:0/1493536372 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79e810c960 con 0x7f79e807ad70 2024-03-23T17:43:10.390 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:09.810+0000 7f79e6000700 1 --2- 172.21.15.86:0/1493536372 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7f79d0038750 0x7f79d003ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:43:10.390 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:09.810+0000 7f79e6000700 1 -- 172.21.15.86:0/1493536372 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f79dc04c990 con 0x7f79e807ad70 2024-03-23T17:43:10.390 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:09.810+0000 7f79eca00700 1 --2- 172.21.15.86:0/1493536372 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7f79d0038750 0x7f79d003ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:43:10.390 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:09.811+0000 7f79eca00700 1 --2- 172.21.15.86:0/1493536372 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7f79d0038750 0x7f79d003ac10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f79d4006fd0 tx=0x7f79d4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:43:10.390 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:09.815+0000 7f79e6000700 1 -- 172.21.15.86:0/1493536372 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f79dc01a430 con 0x7f79e807ad70 2024-03-23T17:43:10.390 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:10.016+0000 7f79efe00700 1 -- 172.21.15.86:0/1493536372 --> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f79e8102df0 con 0x7f79d0038750 2024-03-23T17:43:10.390 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:10.017+0000 7f79e6000700 1 -- 172.21.15.86:0/1493536372 <== mgr.14120 v2:172.21.15.86:6800/3600278228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f79e8102df0 con 0x7f79d0038750 2024-03-23T17:43:10.390 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:10.019+0000 7f79efe00700 1 -- 172.21.15.86:0/1493536372 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7f79d0038750 msgr2=0x7f79d003ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:43:10.390 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:10.019+0000 7f79efe00700 1 --2- 172.21.15.86:0/1493536372 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7f79d0038750 0x7f79d003ac10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f79d4006fd0 tx=0x7f79d4006e40 comp rx=0 tx=0).stop 2024-03-23T17:43:10.390 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:10.019+0000 7f79efe00700 1 -- 172.21.15.86:0/1493536372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79e807ad70 msgr2=0x7f79e819c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:43:10.391 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:10.019+0000 7f79efe00700 1 --2- 172.21.15.86:0/1493536372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79e807ad70 0x7f79e819c4d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f79dc009130 tx=0x7f79dc011ad0 comp rx=0 tx=0).stop 2024-03-23T17:43:10.391 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:10.020+0000 7f79efe00700 1 -- 172.21.15.86:0/1493536372 shutdown_connections 2024-03-23T17:43:10.391 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:10.020+0000 7f79efe00700 1 --2- 172.21.15.86:0/1493536372 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7f79d0038750 0x7f79d003ac10 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:10.391 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:10.020+0000 7f79efe00700 1 --2- 172.21.15.86:0/1493536372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79e807ad70 0x7f79e819c4d0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:10.391 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:10.020+0000 7f79efe00700 1 -- 172.21.15.86:0/1493536372 >> 172.21.15.86:0/1493536372 conn(0x7f79e8101b30 msgr2=0x7f79e81026e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:43:10.391 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:10.020+0000 7f79efe00700 1 -- 172.21.15.86:0/1493536372 shutdown_connections 2024-03-23T17:43:10.391 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:10.020+0000 7f79efe00700 1 -- 172.21.15.86:0/1493536372 wait complete. 2024-03-23T17:43:10.391 INFO:teuthology.orchestra.run.smithi086.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-03-23T17:43:10.391 INFO:teuthology.orchestra.run.smithi086.stdout:Adding key to root@localhost authorized_keys... 2024-03-23T17:43:10.391 INFO:teuthology.orchestra.run.smithi086.stdout:Adding host smithi086... 2024-03-23T17:43:11.631 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:11 smithi086 ceph-mon[38221]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:43:12.766 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:12 smithi086 ceph-mon[38221]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi086", "addr": "172.21.15.86", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:43:13.766 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:13 smithi086 ceph-mon[38221]: Deploying cephadm binary to smithi086 2024-03-23T17:43:14.238 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout Added host 'smithi086' with addr '172.21.15.86' 2024-03-23T17:43:14.239 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:10.765+0000 7f4213e00700 1 Processor -- start 2024-03-23T17:43:14.239 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:10.765+0000 7f4213e00700 1 -- start start 2024-03-23T17:43:14.239 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:10.765+0000 7f4213e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f421407add0 0x7f4214079230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:43:14.239 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:10.765+0000 7f4213e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4214079800 con 0x7f421407add0 2024-03-23T17:43:14.239 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:10.766+0000 7f4212a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f421407add0 0x7f4214079230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:43:14.239 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:10.766+0000 7f4212a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f421407add0 0x7f4214079230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42746/0 (socket says 172.21.15.86:42746) 2024-03-23T17:43:14.239 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:10.766+0000 7f4212a00700 1 -- 172.21.15.86:0/2462330162 learned_addr learned my addr 172.21.15.86:0/2462330162 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:43:14.240 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:10.766+0000 7f4212a00700 1 -- 172.21.15.86:0/2462330162 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4214079940 con 0x7f421407add0 2024-03-23T17:43:14.240 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:10.766+0000 7f4212a00700 1 --2- 172.21.15.86:0/2462330162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f421407add0 0x7f4214079230 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f42040098a0 tx=0x7f4204009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73a1e0d842db86f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:43:14.240 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:10.766+0000 7f4211600700 1 -- 172.21.15.86:0/2462330162 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4204004160 con 0x7f421407add0 2024-03-23T17:43:14.240 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:10.766+0000 7f4211600700 1 -- 172.21.15.86:0/2462330162 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f42040042c0 con 0x7f421407add0 2024-03-23T17:43:14.240 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:10.766+0000 7f4211600700 1 -- 172.21.15.86:0/2462330162 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4204011a00 con 0x7f421407add0 2024-03-23T17:43:14.240 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:10.767+0000 7f4213e00700 1 -- 172.21.15.86:0/2462330162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f421407add0 msgr2=0x7f4214079230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:43:14.241 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:10.767+0000 7f4213e00700 1 --2- 172.21.15.86:0/2462330162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f421407add0 0x7f4214079230 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f42040098a0 tx=0x7f4204009bb0 comp rx=0 tx=0).stop 2024-03-23T17:43:14.241 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:10.767+0000 7f4213e00700 1 -- 172.21.15.86:0/2462330162 shutdown_connections 2024-03-23T17:43:14.241 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:10.767+0000 7f4213e00700 1 --2- 172.21.15.86:0/2462330162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f421407add0 0x7f4214079230 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:14.241 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:10.767+0000 7f4213e00700 1 -- 172.21.15.86:0/2462330162 >> 172.21.15.86:0/2462330162 conn(0x7f42141019b0 msgr2=0x7f4214103dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:43:14.241 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:10.767+0000 7f4213e00700 1 -- 172.21.15.86:0/2462330162 shutdown_connections 2024-03-23T17:43:14.241 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:10.767+0000 7f4213e00700 1 -- 172.21.15.86:0/2462330162 wait complete. 2024-03-23T17:43:14.241 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:10.768+0000 7f4213e00700 1 Processor -- start 2024-03-23T17:43:14.242 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:10.768+0000 7f4213e00700 1 -- start start 2024-03-23T17:43:14.242 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:10.769+0000 7f4213e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f421407add0 0x7f421419c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:43:14.242 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:10.769+0000 7f4213e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f421419c8b0 con 0x7f421407add0 2024-03-23T17:43:14.242 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:10.769+0000 7f4212a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f421407add0 0x7f421419c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:43:14.242 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:10.769+0000 7f4212a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f421407add0 0x7f421419c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42756/0 (socket says 172.21.15.86:42756) 2024-03-23T17:43:14.242 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:10.769+0000 7f4212a00700 1 -- 172.21.15.86:0/1782591503 learned_addr learned my addr 172.21.15.86:0/1782591503 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:43:14.242 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:10.769+0000 7f4212a00700 1 -- 172.21.15.86:0/1782591503 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4204009550 con 0x7f421407add0 2024-03-23T17:43:14.243 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:10.769+0000 7f4212a00700 1 --2- 172.21.15.86:0/1782591503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f421407add0 0x7f421419c370 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4204011710 tx=0x7f4204011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:43:14.243 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:10.770+0000 7f420b400700 1 -- 172.21.15.86:0/1782591503 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4204003aa0 con 0x7f421407add0 2024-03-23T17:43:14.243 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:10.770+0000 7f420b400700 1 -- 172.21.15.86:0/1782591503 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4204003c00 con 0x7f421407add0 2024-03-23T17:43:14.243 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:10.770+0000 7f4213e00700 1 -- 172.21.15.86:0/1782591503 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f421419cab0 con 0x7f421407add0 2024-03-23T17:43:14.243 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:10.770+0000 7f420b400700 1 -- 172.21.15.86:0/1782591503 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4204024800 con 0x7f421407add0 2024-03-23T17:43:14.243 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:10.770+0000 7f4213e00700 1 -- 172.21.15.86:0/1782591503 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f421419cf50 con 0x7f421407add0 2024-03-23T17:43:14.243 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:10.771+0000 7f420b400700 1 -- 172.21.15.86:0/1782591503 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 8) v1 ==== 45533+0+0 (secure 0 0 0) 0x7f420401a430 con 0x7f421407add0 2024-03-23T17:43:14.244 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:10.771+0000 7f4213e00700 1 -- 172.21.15.86:0/1782591503 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f421410c780 con 0x7f421407add0 2024-03-23T17:43:14.244 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:10.771+0000 7f420b400700 1 --2- 172.21.15.86:0/1782591503 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7f41f8038750 0x7f41f803ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:43:14.244 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:10.771+0000 7f420b400700 1 -- 172.21.15.86:0/1782591503 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f420404c620 con 0x7f421407add0 2024-03-23T17:43:14.244 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:10.771+0000 7f4212000700 1 --2- 172.21.15.86:0/1782591503 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7f41f8038750 0x7f41f803ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:43:14.244 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:10.772+0000 7f4212000700 1 --2- 172.21.15.86:0/1782591503 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7f41f8038750 0x7f41f803ac10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4200006fd0 tx=0x7f4200006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:43:14.244 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:10.776+0000 7f420b400700 1 -- 172.21.15.86:0/1782591503 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f420402cb40 con 0x7f421407add0 2024-03-23T17:43:14.244 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:10.978+0000 7f4213e00700 1 -- 172.21.15.86:0/1782591503 --> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi086", "addr": "172.21.15.86", "target": ["mon-mgr", ""]}) v1 -- 0x7f4214102c90 con 0x7f41f8038750 2024-03-23T17:43:14.244 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:13.896+0000 7f420b400700 1 -- 172.21.15.86:0/1782591503 <== mgr.14120 v2:172.21.15.86:6800/3600278228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f4214102c90 con 0x7f41f8038750 2024-03-23T17:43:14.245 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:13.899+0000 7f4213e00700 1 -- 172.21.15.86:0/1782591503 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7f41f8038750 msgr2=0x7f41f803ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:43:14.245 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:13.899+0000 7f4213e00700 1 --2- 172.21.15.86:0/1782591503 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7f41f8038750 0x7f41f803ac10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4200006fd0 tx=0x7f4200006e40 comp rx=0 tx=0).stop 2024-03-23T17:43:14.245 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:13.899+0000 7f4213e00700 1 -- 172.21.15.86:0/1782591503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f421407add0 msgr2=0x7f421419c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:43:14.245 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:13.899+0000 7f4213e00700 1 --2- 172.21.15.86:0/1782591503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f421407add0 0x7f421419c370 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4204011710 tx=0x7f4204011ad0 comp rx=0 tx=0).stop 2024-03-23T17:43:14.245 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:13.899+0000 7f4213e00700 1 -- 172.21.15.86:0/1782591503 shutdown_connections 2024-03-23T17:43:14.245 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:13.899+0000 7f4213e00700 1 --2- 172.21.15.86:0/1782591503 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7f41f8038750 0x7f41f803ac10 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:14.246 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:13.899+0000 7f4213e00700 1 --2- 172.21.15.86:0/1782591503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f421407add0 0x7f421419c370 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:14.246 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:13.899+0000 7f4213e00700 1 -- 172.21.15.86:0/1782591503 >> 172.21.15.86:0/1782591503 conn(0x7f42141019b0 msgr2=0x7f4214102580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:43:14.246 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:13.899+0000 7f4213e00700 1 -- 172.21.15.86:0/1782591503 shutdown_connections 2024-03-23T17:43:14.246 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:13.899+0000 7f4213e00700 1 -- 172.21.15.86:0/1782591503 wait complete. 2024-03-23T17:43:14.246 INFO:teuthology.orchestra.run.smithi086.stdout:Deploying unmanaged mon service... 2024-03-23T17:43:15.087 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:14 smithi086 ceph-mon[38221]: from='mgr.14120 172.21.15.86:0/1645921427' entity='mgr.x' 2024-03-23T17:43:15.087 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:14 smithi086 ceph-mon[38221]: Added host smithi086 2024-03-23T17:43:15.087 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:14 smithi086 ceph-mon[38221]: from='mgr.14120 172.21.15.86:0/1645921427' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:43:15.515 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-03-23T17:43:15.515 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:14.711+0000 7f9261a00700 1 Processor -- start 2024-03-23T17:43:15.515 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:14.711+0000 7f9261a00700 1 -- start start 2024-03-23T17:43:15.515 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:14.711+0000 7f9261a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f925c07add0 0x7f925c079230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:43:15.515 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:14.711+0000 7f9261a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f925c079800 con 0x7f925c07add0 2024-03-23T17:43:15.515 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:14.712+0000 7f925aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f925c07add0 0x7f925c079230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:43:15.515 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:14.712+0000 7f925aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f925c07add0 0x7f925c079230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42772/0 (socket says 172.21.15.86:42772) 2024-03-23T17:43:15.515 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:14.712+0000 7f925aa00700 1 -- 172.21.15.86:0/497612304 learned_addr learned my addr 172.21.15.86:0/497612304 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:43:15.515 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:14.712+0000 7f925aa00700 1 -- 172.21.15.86:0/497612304 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f925c079940 con 0x7f925c07add0 2024-03-23T17:43:15.515 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:14.712+0000 7f925aa00700 1 --2- 172.21.15.86:0/497612304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f925c07add0 0x7f925c079230 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f924c0098a0 tx=0x7f924c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4fd5e06585dfcc1 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:43:15.515 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:14.713+0000 7f9259600700 1 -- 172.21.15.86:0/497612304 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f924c004160 con 0x7f925c07add0 2024-03-23T17:43:15.516 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:14.713+0000 7f9259600700 1 -- 172.21.15.86:0/497612304 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f924c0042c0 con 0x7f925c07add0 2024-03-23T17:43:15.516 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:14.713+0000 7f9259600700 1 -- 172.21.15.86:0/497612304 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f924c011a00 con 0x7f925c07add0 2024-03-23T17:43:15.516 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:14.713+0000 7f9261a00700 1 -- 172.21.15.86:0/497612304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f925c07add0 msgr2=0x7f925c079230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:43:15.516 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:14.713+0000 7f9261a00700 1 --2- 172.21.15.86:0/497612304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f925c07add0 0x7f925c079230 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f924c0098a0 tx=0x7f924c009bb0 comp rx=0 tx=0).stop 2024-03-23T17:43:15.516 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:14.713+0000 7f9261a00700 1 -- 172.21.15.86:0/497612304 shutdown_connections 2024-03-23T17:43:15.516 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:14.713+0000 7f9261a00700 1 --2- 172.21.15.86:0/497612304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f925c07add0 0x7f925c079230 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:15.516 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:14.713+0000 7f9261a00700 1 -- 172.21.15.86:0/497612304 >> 172.21.15.86:0/497612304 conn(0x7f925c101b90 msgr2=0x7f925c103fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:43:15.516 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:14.714+0000 7f9261a00700 1 -- 172.21.15.86:0/497612304 shutdown_connections 2024-03-23T17:43:15.516 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:14.714+0000 7f9261a00700 1 -- 172.21.15.86:0/497612304 wait complete. 2024-03-23T17:43:15.516 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:14.715+0000 7f9261a00700 1 Processor -- start 2024-03-23T17:43:15.516 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:14.715+0000 7f9261a00700 1 -- start start 2024-03-23T17:43:15.516 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:14.716+0000 7f9261a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f925c07add0 0x7f925c19c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:43:15.516 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:14.716+0000 7f9261a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f925c19ca10 con 0x7f925c07add0 2024-03-23T17:43:15.516 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:14.716+0000 7f925aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f925c07add0 0x7f925c19c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:43:15.517 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:14.716+0000 7f925aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f925c07add0 0x7f925c19c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42786/0 (socket says 172.21.15.86:42786) 2024-03-23T17:43:15.517 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:14.716+0000 7f925aa00700 1 -- 172.21.15.86:0/1677017608 learned_addr learned my addr 172.21.15.86:0/1677017608 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:43:15.517 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:14.716+0000 7f925aa00700 1 -- 172.21.15.86:0/1677017608 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f924c009550 con 0x7f925c07add0 2024-03-23T17:43:15.517 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:14.716+0000 7f925aa00700 1 --2- 172.21.15.86:0/1677017608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f925c07add0 0x7f925c19c4d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f924c009130 tx=0x7f924c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:43:15.517 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:14.717+0000 7f9253400700 1 -- 172.21.15.86:0/1677017608 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f924c003ab0 con 0x7f925c07add0 2024-03-23T17:43:15.517 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:14.717+0000 7f9253400700 1 -- 172.21.15.86:0/1677017608 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f924c003c10 con 0x7f925c07add0 2024-03-23T17:43:15.517 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:14.717+0000 7f9261a00700 1 -- 172.21.15.86:0/1677017608 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f925c19cc10 con 0x7f925c07add0 2024-03-23T17:43:15.517 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:14.717+0000 7f9253400700 1 -- 172.21.15.86:0/1677017608 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f924c0247d0 con 0x7f925c07add0 2024-03-23T17:43:15.517 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:14.717+0000 7f9261a00700 1 -- 172.21.15.86:0/1677017608 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f925c19d0b0 con 0x7f925c07add0 2024-03-23T17:43:15.517 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:14.718+0000 7f9253400700 1 -- 172.21.15.86:0/1677017608 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 8) v1 ==== 45533+0+0 (secure 0 0 0) 0x7f924c02d3f0 con 0x7f925c07add0 2024-03-23T17:43:15.517 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:14.718+0000 7f9261a00700 1 -- 172.21.15.86:0/1677017608 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f925c10c960 con 0x7f925c07add0 2024-03-23T17:43:15.517 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:14.719+0000 7f9253400700 1 --2- 172.21.15.86:0/1677017608 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7f92400387a0 0x7f924003ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:43:15.517 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:14.719+0000 7f9253400700 1 -- 172.21.15.86:0/1677017608 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f924c04c860 con 0x7f925c07add0 2024-03-23T17:43:15.518 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:14.719+0000 7f925a000700 1 --2- 172.21.15.86:0/1677017608 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7f92400387a0 0x7f924003ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:43:15.518 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:14.720+0000 7f925a000700 1 --2- 172.21.15.86:0/1677017608 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7f92400387a0 0x7f924003ac60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9248006fd0 tx=0x7f9248006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:43:15.518 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:14.724+0000 7f9253400700 1 -- 172.21.15.86:0/1677017608 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f924c010970 con 0x7f925c07add0 2024-03-23T17:43:15.518 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:14.956+0000 7f9261a00700 1 -- 172.21.15.86:0/1677017608 --> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f925c102de0 con 0x7f92400387a0 2024-03-23T17:43:15.518 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:14.962+0000 7f9253400700 1 -- 172.21.15.86:0/1677017608 <== mgr.14120 v2:172.21.15.86:6800/3600278228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f925c102de0 con 0x7f92400387a0 2024-03-23T17:43:15.518 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:14.964+0000 7f9261a00700 1 -- 172.21.15.86:0/1677017608 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7f92400387a0 msgr2=0x7f924003ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:43:15.518 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:14.964+0000 7f9261a00700 1 --2- 172.21.15.86:0/1677017608 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7f92400387a0 0x7f924003ac60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9248006fd0 tx=0x7f9248006e40 comp rx=0 tx=0).stop 2024-03-23T17:43:15.518 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:14.964+0000 7f9261a00700 1 -- 172.21.15.86:0/1677017608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f925c07add0 msgr2=0x7f925c19c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:43:15.518 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:14.964+0000 7f9261a00700 1 --2- 172.21.15.86:0/1677017608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f925c07add0 0x7f925c19c4d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f924c009130 tx=0x7f924c011ad0 comp rx=0 tx=0).stop 2024-03-23T17:43:15.518 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:14.964+0000 7f9261a00700 1 -- 172.21.15.86:0/1677017608 shutdown_connections 2024-03-23T17:43:15.518 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:14.964+0000 7f9261a00700 1 --2- 172.21.15.86:0/1677017608 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7f92400387a0 0x7f924003ac60 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:15.518 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:14.964+0000 7f9261a00700 1 --2- 172.21.15.86:0/1677017608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f925c07add0 0x7f925c19c4d0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:15.519 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:14.964+0000 7f9261a00700 1 -- 172.21.15.86:0/1677017608 >> 172.21.15.86:0/1677017608 conn(0x7f925c101b90 msgr2=0x7f925c1026d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:43:15.519 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:14.964+0000 7f9261a00700 1 -- 172.21.15.86:0/1677017608 shutdown_connections 2024-03-23T17:43:15.519 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:14.964+0000 7f9261a00700 1 -- 172.21.15.86:0/1677017608 wait complete. 2024-03-23T17:43:15.519 INFO:teuthology.orchestra.run.smithi086.stdout:Deploying unmanaged mgr service... 2024-03-23T17:43:16.215 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:15 smithi086 ceph-mon[38221]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:43:16.215 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:15 smithi086 ceph-mon[38221]: Saving service mon spec with placement count:5 2024-03-23T17:43:16.215 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:15 smithi086 ceph-mon[38221]: from='mgr.14120 172.21.15.86:0/1645921427' entity='mgr.x' 2024-03-23T17:43:16.216 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:15 smithi086 ceph-mon[38221]: from='mgr.14120 172.21.15.86:0/1645921427' entity='mgr.x' 2024-03-23T17:43:16.216 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:15 smithi086 ceph-mon[38221]: from='mgr.14120 172.21.15.86:0/1645921427' entity='mgr.x' 2024-03-23T17:43:16.560 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-03-23T17:43:16.560 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:15.941+0000 7fcfef000700 1 Processor -- start 2024-03-23T17:43:16.560 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:15.941+0000 7fcfef000700 1 -- start start 2024-03-23T17:43:16.560 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:15.941+0000 7fcfef000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfe807ad90 0x7fcfe80791f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:43:16.561 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:15.941+0000 7fcfef000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe80797c0 con 0x7fcfe807ad90 2024-03-23T17:43:16.561 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:15.942+0000 7fcfe7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfe807ad90 0x7fcfe80791f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:43:16.561 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:15.942+0000 7fcfe7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfe807ad90 0x7fcfe80791f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49562/0 (socket says 172.21.15.86:49562) 2024-03-23T17:43:16.561 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:15.942+0000 7fcfe7e00700 1 -- 172.21.15.86:0/2288162282 learned_addr learned my addr 172.21.15.86:0/2288162282 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:43:16.561 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:15.942+0000 7fcfe7e00700 1 -- 172.21.15.86:0/2288162282 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfe8079900 con 0x7fcfe807ad90 2024-03-23T17:43:16.561 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:15.943+0000 7fcfe7e00700 1 --2- 172.21.15.86:0/2288162282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfe807ad90 0x7fcfe80791f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcfd80098a0 tx=0x7fcfd8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cd509491957140a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:43:16.561 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:15.943+0000 7fcfe6a00700 1 -- 172.21.15.86:0/2288162282 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcfd8004160 con 0x7fcfe807ad90 2024-03-23T17:43:16.561 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:15.943+0000 7fcfe6a00700 1 -- 172.21.15.86:0/2288162282 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcfd80042c0 con 0x7fcfe807ad90 2024-03-23T17:43:16.561 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:15.943+0000 7fcfe6a00700 1 -- 172.21.15.86:0/2288162282 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcfd80119b0 con 0x7fcfe807ad90 2024-03-23T17:43:16.561 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:15.944+0000 7fcfef000700 1 -- 172.21.15.86:0/2288162282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfe807ad90 msgr2=0x7fcfe80791f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:43:16.561 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:15.944+0000 7fcfef000700 1 --2- 172.21.15.86:0/2288162282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfe807ad90 0x7fcfe80791f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcfd80098a0 tx=0x7fcfd8009bb0 comp rx=0 tx=0).stop 2024-03-23T17:43:16.561 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:15.944+0000 7fcfef000700 1 -- 172.21.15.86:0/2288162282 shutdown_connections 2024-03-23T17:43:16.561 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:15.944+0000 7fcfef000700 1 --2- 172.21.15.86:0/2288162282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfe807ad90 0x7fcfe80791f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:16.561 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:15.944+0000 7fcfef000700 1 -- 172.21.15.86:0/2288162282 >> 172.21.15.86:0/2288162282 conn(0x7fcfe8101b90 msgr2=0x7fcfe8103fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:43:16.562 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:15.944+0000 7fcfef000700 1 -- 172.21.15.86:0/2288162282 shutdown_connections 2024-03-23T17:43:16.562 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:15.944+0000 7fcfef000700 1 -- 172.21.15.86:0/2288162282 wait complete. 2024-03-23T17:43:16.562 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:15.945+0000 7fcfef000700 1 Processor -- start 2024-03-23T17:43:16.562 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:15.945+0000 7fcfef000700 1 -- start start 2024-03-23T17:43:16.562 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:15.945+0000 7fcfef000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfe807ad90 0x7fcfe819c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:43:16.562 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:15.945+0000 7fcfef000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe819ca30 con 0x7fcfe807ad90 2024-03-23T17:43:16.562 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:15.946+0000 7fcfe7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfe807ad90 0x7fcfe819c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:43:16.562 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:15.946+0000 7fcfe7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfe807ad90 0x7fcfe819c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49564/0 (socket says 172.21.15.86:49564) 2024-03-23T17:43:16.562 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:15.946+0000 7fcfe7e00700 1 -- 172.21.15.86:0/3799068781 learned_addr learned my addr 172.21.15.86:0/3799068781 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:43:16.562 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:15.946+0000 7fcfe7e00700 1 -- 172.21.15.86:0/3799068781 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfd8009550 con 0x7fcfe807ad90 2024-03-23T17:43:16.562 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:15.946+0000 7fcfe7e00700 1 --2- 172.21.15.86:0/3799068781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfe807ad90 0x7fcfe819c4f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcfd8011670 tx=0x7fcfd8011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:43:16.562 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:15.946+0000 7fcfe4c00700 1 -- 172.21.15.86:0/3799068781 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcfd8003b20 con 0x7fcfe807ad90 2024-03-23T17:43:16.563 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:15.946+0000 7fcfe4c00700 1 -- 172.21.15.86:0/3799068781 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcfd8003c80 con 0x7fcfe807ad90 2024-03-23T17:43:16.563 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:15.946+0000 7fcfef000700 1 -- 172.21.15.86:0/3799068781 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfe819cc30 con 0x7fcfe807ad90 2024-03-23T17:43:16.563 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:15.946+0000 7fcfe4c00700 1 -- 172.21.15.86:0/3799068781 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcfd8019770 con 0x7fcfe807ad90 2024-03-23T17:43:16.563 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:15.947+0000 7fcfef000700 1 -- 172.21.15.86:0/3799068781 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfe819d0d0 con 0x7fcfe807ad90 2024-03-23T17:43:16.563 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:15.948+0000 7fcfef000700 1 -- 172.21.15.86:0/3799068781 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfe810c9f0 con 0x7fcfe807ad90 2024-03-23T17:43:16.563 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:15.948+0000 7fcfe4c00700 1 -- 172.21.15.86:0/3799068781 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 8) v1 ==== 45533+0+0 (secure 0 0 0) 0x7fcfd801a430 con 0x7fcfe807ad90 2024-03-23T17:43:16.563 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:15.948+0000 7fcfe4c00700 1 --2- 172.21.15.86:0/3799068781 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7fcfcc038700 0x7fcfcc03abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:43:16.563 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:15.948+0000 7fcfe4c00700 1 -- 172.21.15.86:0/3799068781 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fcfd804ca00 con 0x7fcfe807ad90 2024-03-23T17:43:16.563 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:15.948+0000 7fcfe7400700 1 --2- 172.21.15.86:0/3799068781 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7fcfcc038700 0x7fcfcc03abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:43:16.563 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:15.949+0000 7fcfe7400700 1 --2- 172.21.15.86:0/3799068781 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7fcfcc038700 0x7fcfcc03abc0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fcfd4006fd0 tx=0x7fcfd4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:43:16.563 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:15.953+0000 7fcfe4c00700 1 -- 172.21.15.86:0/3799068781 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fcfd802d960 con 0x7fcfe807ad90 2024-03-23T17:43:16.564 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:16.182+0000 7fcfef000700 1 -- 172.21.15.86:0/3799068781 --> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fcfe819f960 con 0x7fcfcc038700 2024-03-23T17:43:16.564 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:16.190+0000 7fcfe4c00700 1 -- 172.21.15.86:0/3799068781 <== mgr.14120 v2:172.21.15.86:6800/3600278228 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fcfe819f960 con 0x7fcfcc038700 2024-03-23T17:43:16.564 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:16.192+0000 7fcfef000700 1 -- 172.21.15.86:0/3799068781 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7fcfcc038700 msgr2=0x7fcfcc03abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:43:16.564 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:16.192+0000 7fcfef000700 1 --2- 172.21.15.86:0/3799068781 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7fcfcc038700 0x7fcfcc03abc0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fcfd4006fd0 tx=0x7fcfd4006e40 comp rx=0 tx=0).stop 2024-03-23T17:43:16.564 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:16.192+0000 7fcfef000700 1 -- 172.21.15.86:0/3799068781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfe807ad90 msgr2=0x7fcfe819c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:43:16.564 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:16.192+0000 7fcfef000700 1 --2- 172.21.15.86:0/3799068781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfe807ad90 0x7fcfe819c4f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcfd8011670 tx=0x7fcfd8011a30 comp rx=0 tx=0).stop 2024-03-23T17:43:16.564 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:16.192+0000 7fcfef000700 1 -- 172.21.15.86:0/3799068781 shutdown_connections 2024-03-23T17:43:16.564 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:16.192+0000 7fcfef000700 1 --2- 172.21.15.86:0/3799068781 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7fcfcc038700 0x7fcfcc03abc0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:16.564 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:16.192+0000 7fcfef000700 1 --2- 172.21.15.86:0/3799068781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfe807ad90 0x7fcfe819c4f0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:16.564 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:16.192+0000 7fcfef000700 1 -- 172.21.15.86:0/3799068781 >> 172.21.15.86:0/3799068781 conn(0x7fcfe8101b90 msgr2=0x7fcfe81026b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:43:16.564 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:16.192+0000 7fcfef000700 1 -- 172.21.15.86:0/3799068781 shutdown_connections 2024-03-23T17:43:16.564 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:16.192+0000 7fcfef000700 1 -- 172.21.15.86:0/3799068781 wait complete. 2024-03-23T17:43:17.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:17 smithi086 ceph-mon[38221]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:43:17.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:17 smithi086 ceph-mon[38221]: Saving service mgr spec with placement count:2 2024-03-23T17:43:17.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:17 smithi086 ceph-mon[38221]: from='mgr.14120 172.21.15.86:0/1645921427' entity='mgr.x' 2024-03-23T17:43:17.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:17 smithi086 ceph-mon[38221]: from='mgr.14120 172.21.15.86:0/1645921427' entity='mgr.x' 2024-03-23T17:43:17.722 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:17.162+0000 7f4a53800700 1 Processor -- start 2024-03-23T17:43:17.723 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:17.163+0000 7f4a53800700 1 -- start start 2024-03-23T17:43:17.723 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:17.163+0000 7f4a53800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a4c07adf0 0x7f4a4c079250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:43:17.723 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:17.163+0000 7f4a53800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a4c079820 con 0x7f4a4c07adf0 2024-03-23T17:43:17.723 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:17.163+0000 7f4a50e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a4c07adf0 0x7f4a4c079250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:43:17.723 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:17.163+0000 7f4a50e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a4c07adf0 0x7f4a4c079250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49570/0 (socket says 172.21.15.86:49570) 2024-03-23T17:43:17.723 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:17.163+0000 7f4a50e00700 1 -- 172.21.15.86:0/1646976148 learned_addr learned my addr 172.21.15.86:0/1646976148 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:43:17.723 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:17.164+0000 7f4a50e00700 1 -- 172.21.15.86:0/1646976148 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a4c079960 con 0x7f4a4c07adf0 2024-03-23T17:43:17.723 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:17.164+0000 7f4a50e00700 1 --2- 172.21.15.86:0/1646976148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a4c07adf0 0x7f4a4c079250 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4a400098a0 tx=0x7f4a40009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1fb23fbd604cc8a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:43:17.723 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:17.164+0000 7f4a4b400700 1 -- 172.21.15.86:0/1646976148 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4a40004160 con 0x7f4a4c07adf0 2024-03-23T17:43:17.723 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:17.164+0000 7f4a4b400700 1 -- 172.21.15.86:0/1646976148 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a400042c0 con 0x7f4a4c07adf0 2024-03-23T17:43:17.723 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:17.164+0000 7f4a4b400700 1 -- 172.21.15.86:0/1646976148 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4a400119b0 con 0x7f4a4c07adf0 2024-03-23T17:43:17.723 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:17.165+0000 7f4a53800700 1 -- 172.21.15.86:0/1646976148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a4c07adf0 msgr2=0x7f4a4c079250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:43:17.723 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:17.165+0000 7f4a53800700 1 --2- 172.21.15.86:0/1646976148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a4c07adf0 0x7f4a4c079250 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4a400098a0 tx=0x7f4a40009bb0 comp rx=0 tx=0).stop 2024-03-23T17:43:17.724 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:17.165+0000 7f4a53800700 1 -- 172.21.15.86:0/1646976148 shutdown_connections 2024-03-23T17:43:17.724 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:17.165+0000 7f4a53800700 1 --2- 172.21.15.86:0/1646976148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a4c07adf0 0x7f4a4c079250 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:17.724 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:17.165+0000 7f4a53800700 1 -- 172.21.15.86:0/1646976148 >> 172.21.15.86:0/1646976148 conn(0x7f4a4c101b70 msgr2=0x7f4a4c103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:43:17.724 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:17.165+0000 7f4a53800700 1 -- 172.21.15.86:0/1646976148 shutdown_connections 2024-03-23T17:43:17.724 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:17.165+0000 7f4a53800700 1 -- 172.21.15.86:0/1646976148 wait complete. 2024-03-23T17:43:17.724 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:17.167+0000 7f4a53800700 1 Processor -- start 2024-03-23T17:43:17.724 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:17.167+0000 7f4a53800700 1 -- start start 2024-03-23T17:43:17.724 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:17.167+0000 7f4a53800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a4c07adf0 0x7f4a4c1a0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:43:17.724 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:17.167+0000 7f4a53800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a4c1a0e70 con 0x7f4a4c07adf0 2024-03-23T17:43:17.724 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:17.168+0000 7f4a50e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a4c07adf0 0x7f4a4c1a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:43:17.724 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:17.168+0000 7f4a50e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a4c07adf0 0x7f4a4c1a0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49586/0 (socket says 172.21.15.86:49586) 2024-03-23T17:43:17.724 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:17.168+0000 7f4a50e00700 1 -- 172.21.15.86:0/3008670791 learned_addr learned my addr 172.21.15.86:0/3008670791 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:43:17.724 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:17.168+0000 7f4a50e00700 1 -- 172.21.15.86:0/3008670791 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a40009550 con 0x7f4a4c07adf0 2024-03-23T17:43:17.724 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:17.168+0000 7f4a50e00700 1 --2- 172.21.15.86:0/3008670791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a4c07adf0 0x7f4a4c1a0930 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4a40003e60 tx=0x7f4a40011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:43:17.725 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:17.168+0000 7f4a49600700 1 -- 172.21.15.86:0/3008670791 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4a40003a60 con 0x7f4a4c07adf0 2024-03-23T17:43:17.725 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:17.168+0000 7f4a49600700 1 -- 172.21.15.86:0/3008670791 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a40003bc0 con 0x7f4a4c07adf0 2024-03-23T17:43:17.725 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:17.168+0000 7f4a53800700 1 -- 172.21.15.86:0/3008670791 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a4c1a1070 con 0x7f4a4c07adf0 2024-03-23T17:43:17.725 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:17.168+0000 7f4a49600700 1 -- 172.21.15.86:0/3008670791 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4a400196c0 con 0x7f4a4c07adf0 2024-03-23T17:43:17.725 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:17.169+0000 7f4a53800700 1 -- 172.21.15.86:0/3008670791 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a4c1a1510 con 0x7f4a4c07adf0 2024-03-23T17:43:17.725 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:17.170+0000 7f4a53800700 1 -- 172.21.15.86:0/3008670791 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a4c108580 con 0x7f4a4c07adf0 2024-03-23T17:43:17.725 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:17.170+0000 7f4a49600700 1 -- 172.21.15.86:0/3008670791 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 8) v1 ==== 45533+0+0 (secure 0 0 0) 0x7f4a4001ab60 con 0x7f4a4c07adf0 2024-03-23T17:43:17.725 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:17.170+0000 7f4a49600700 1 --2- 172.21.15.86:0/3008670791 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7f4a34038730 0x7f4a3403abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:43:17.725 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:17.170+0000 7f4a49600700 1 -- 172.21.15.86:0/3008670791 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f4a4004c9a0 con 0x7f4a4c07adf0 2024-03-23T17:43:17.725 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:17.170+0000 7f4a4be00700 1 --2- 172.21.15.86:0/3008670791 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7f4a34038730 0x7f4a3403abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:43:17.725 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:17.171+0000 7f4a4be00700 1 --2- 172.21.15.86:0/3008670791 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7f4a34038730 0x7f4a3403abf0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4a38006fd0 tx=0x7f4a38006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:43:17.725 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:17.176+0000 7f4a49600700 1 -- 172.21.15.86:0/3008670791 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f4a40019950 con 0x7f4a4c07adf0 2024-03-23T17:43:17.725 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:17.387+0000 7f4a53800700 1 -- 172.21.15.86:0/3008670791 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f4a4c1087c0 con 0x7f4a4c07adf0 2024-03-23T17:43:17.725 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:17.393+0000 7f4a49600700 1 -- 172.21.15.86:0/3008670791 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f4a40018350 con 0x7f4a4c07adf0 2024-03-23T17:43:17.726 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:17.396+0000 7f4a53800700 1 -- 172.21.15.86:0/3008670791 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7f4a34038730 msgr2=0x7f4a3403abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:43:17.726 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:17.396+0000 7f4a53800700 1 --2- 172.21.15.86:0/3008670791 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7f4a34038730 0x7f4a3403abf0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4a38006fd0 tx=0x7f4a38006e40 comp rx=0 tx=0).stop 2024-03-23T17:43:17.726 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:17.396+0000 7f4a53800700 1 -- 172.21.15.86:0/3008670791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a4c07adf0 msgr2=0x7f4a4c1a0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:43:17.726 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:17.396+0000 7f4a53800700 1 --2- 172.21.15.86:0/3008670791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a4c07adf0 0x7f4a4c1a0930 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4a40003e60 tx=0x7f4a40011a30 comp rx=0 tx=0).stop 2024-03-23T17:43:17.726 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:17.396+0000 7f4a53800700 1 -- 172.21.15.86:0/3008670791 shutdown_connections 2024-03-23T17:43:17.726 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:17.396+0000 7f4a53800700 1 --2- 172.21.15.86:0/3008670791 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7f4a34038730 0x7f4a3403abf0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:17.726 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:17.396+0000 7f4a53800700 1 --2- 172.21.15.86:0/3008670791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a4c07adf0 0x7f4a4c1a0930 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:17.726 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:17.396+0000 7f4a53800700 1 -- 172.21.15.86:0/3008670791 >> 172.21.15.86:0/3008670791 conn(0x7f4a4c101b70 msgr2=0x7f4a4c103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:43:17.726 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:17.397+0000 7f4a53800700 1 -- 172.21.15.86:0/3008670791 shutdown_connections 2024-03-23T17:43:17.726 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:17.397+0000 7f4a53800700 1 -- 172.21.15.86:0/3008670791 wait complete. 2024-03-23T17:43:18.646 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:18 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/3008670791' entity='client.admin' 2024-03-23T17:43:18.646 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:18 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/4000895042' entity='client.admin' 2024-03-23T17:43:18.815 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:18.157+0000 7fd4e6600700 1 Processor -- start 2024-03-23T17:43:18.815 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:18.157+0000 7fd4e6600700 1 -- start start 2024-03-23T17:43:18.815 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:18.157+0000 7fd4e6600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4e007adb0 0x7fd4e0079210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:43:18.815 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:18.157+0000 7fd4e6600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e00797e0 con 0x7fd4e007adb0 2024-03-23T17:43:18.815 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:18.158+0000 7fd4df400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4e007adb0 0x7fd4e0079210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:43:18.815 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:18.158+0000 7fd4df400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4e007adb0 0x7fd4e0079210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49598/0 (socket says 172.21.15.86:49598) 2024-03-23T17:43:18.815 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:18.158+0000 7fd4df400700 1 -- 172.21.15.86:0/1068250570 learned_addr learned my addr 172.21.15.86:0/1068250570 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:43:18.815 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:18.158+0000 7fd4df400700 1 -- 172.21.15.86:0/1068250570 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4e0079920 con 0x7fd4e007adb0 2024-03-23T17:43:18.815 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:18.158+0000 7fd4df400700 1 --2- 172.21.15.86:0/1068250570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4e007adb0 0x7fd4e0079210 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd4d00098a0 tx=0x7fd4d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dda55045aa04e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:43:18.815 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:18.159+0000 7fd4de000700 1 -- 172.21.15.86:0/1068250570 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd4d0004160 con 0x7fd4e007adb0 2024-03-23T17:43:18.815 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:18.159+0000 7fd4de000700 1 -- 172.21.15.86:0/1068250570 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd4d00042c0 con 0x7fd4e007adb0 2024-03-23T17:43:18.815 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:18.159+0000 7fd4de000700 1 -- 172.21.15.86:0/1068250570 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd4d00119b0 con 0x7fd4e007adb0 2024-03-23T17:43:18.816 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:18.159+0000 7fd4e6600700 1 -- 172.21.15.86:0/1068250570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4e007adb0 msgr2=0x7fd4e0079210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:43:18.816 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:18.159+0000 7fd4e6600700 1 --2- 172.21.15.86:0/1068250570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4e007adb0 0x7fd4e0079210 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd4d00098a0 tx=0x7fd4d0009bb0 comp rx=0 tx=0).stop 2024-03-23T17:43:18.816 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:18.159+0000 7fd4e6600700 1 -- 172.21.15.86:0/1068250570 shutdown_connections 2024-03-23T17:43:18.816 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:18.159+0000 7fd4e6600700 1 --2- 172.21.15.86:0/1068250570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4e007adb0 0x7fd4e0079210 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:18.816 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:18.159+0000 7fd4e6600700 1 -- 172.21.15.86:0/1068250570 >> 172.21.15.86:0/1068250570 conn(0x7fd4e0101b90 msgr2=0x7fd4e0103fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:43:18.816 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:18.160+0000 7fd4e6600700 1 -- 172.21.15.86:0/1068250570 shutdown_connections 2024-03-23T17:43:18.816 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:18.160+0000 7fd4e6600700 1 -- 172.21.15.86:0/1068250570 wait complete. 2024-03-23T17:43:18.816 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:18.161+0000 7fd4e6600700 1 Processor -- start 2024-03-23T17:43:18.816 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:18.161+0000 7fd4e6600700 1 -- start start 2024-03-23T17:43:18.816 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:18.161+0000 7fd4e6600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4e007adb0 0x7fd4e019c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:43:18.816 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:18.161+0000 7fd4e6600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e019ca60 con 0x7fd4e007adb0 2024-03-23T17:43:18.816 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:18.161+0000 7fd4df400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4e007adb0 0x7fd4e019c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:43:18.816 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:18.161+0000 7fd4df400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4e007adb0 0x7fd4e019c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49600/0 (socket says 172.21.15.86:49600) 2024-03-23T17:43:18.816 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:18.161+0000 7fd4df400700 1 -- 172.21.15.86:0/4000895042 learned_addr learned my addr 172.21.15.86:0/4000895042 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:43:18.816 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:18.162+0000 7fd4df400700 1 -- 172.21.15.86:0/4000895042 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4d0009550 con 0x7fd4e007adb0 2024-03-23T17:43:18.817 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:18.162+0000 7fd4df400700 1 --2- 172.21.15.86:0/4000895042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4e007adb0 0x7fd4e019c520 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0003e60 tx=0x7fd4d0011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:43:18.817 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:18.162+0000 7fd4d7e00700 1 -- 172.21.15.86:0/4000895042 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd4d0003a60 con 0x7fd4e007adb0 2024-03-23T17:43:18.817 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:18.162+0000 7fd4e6600700 1 -- 172.21.15.86:0/4000895042 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4e019cc60 con 0x7fd4e007adb0 2024-03-23T17:43:18.817 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:18.162+0000 7fd4d7e00700 1 -- 172.21.15.86:0/4000895042 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd4d0003bc0 con 0x7fd4e007adb0 2024-03-23T17:43:18.817 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:18.162+0000 7fd4d7e00700 1 -- 172.21.15.86:0/4000895042 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd4d00196c0 con 0x7fd4e007adb0 2024-03-23T17:43:18.817 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:18.162+0000 7fd4e6600700 1 -- 172.21.15.86:0/4000895042 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4e019d100 con 0x7fd4e007adb0 2024-03-23T17:43:18.817 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:18.163+0000 7fd4d7e00700 1 -- 172.21.15.86:0/4000895042 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 8) v1 ==== 45533+0+0 (secure 0 0 0) 0x7fd4d001a430 con 0x7fd4e007adb0 2024-03-23T17:43:18.817 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:18.163+0000 7fd4e6600700 1 -- 172.21.15.86:0/4000895042 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4e010c9d0 con 0x7fd4e007adb0 2024-03-23T17:43:18.817 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:18.164+0000 7fd4d7e00700 1 --2- 172.21.15.86:0/4000895042 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7fd4c40386e0 0x7fd4c403aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:43:18.817 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:18.164+0000 7fd4d7e00700 1 -- 172.21.15.86:0/4000895042 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fd4d004c9d0 con 0x7fd4e007adb0 2024-03-23T17:43:18.817 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:18.164+0000 7fd4dea00700 1 --2- 172.21.15.86:0/4000895042 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7fd4c40386e0 0x7fd4c403aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:43:18.817 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:18.165+0000 7fd4dea00700 1 --2- 172.21.15.86:0/4000895042 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7fd4c40386e0 0x7fd4c403aba0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc006fd0 tx=0x7fd4cc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:43:18.817 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:18.169+0000 7fd4d7e00700 1 -- 172.21.15.86:0/4000895042 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fd4d0010560 con 0x7fd4e007adb0 2024-03-23T17:43:18.817 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:18.378+0000 7fd4e6600700 1 -- 172.21.15.86:0/4000895042 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7fd4e010cbe0 con 0x7fd4e007adb0 2024-03-23T17:43:18.817 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:18.382+0000 7fd4d7e00700 1 -- 172.21.15.86:0/4000895042 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7fd4d0010380 con 0x7fd4e007adb0 2024-03-23T17:43:18.818 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:18.385+0000 7fd4e6600700 1 -- 172.21.15.86:0/4000895042 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7fd4c40386e0 msgr2=0x7fd4c403aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:43:18.818 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:18.385+0000 7fd4e6600700 1 --2- 172.21.15.86:0/4000895042 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7fd4c40386e0 0x7fd4c403aba0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc006fd0 tx=0x7fd4cc006e40 comp rx=0 tx=0).stop 2024-03-23T17:43:18.818 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:18.385+0000 7fd4e6600700 1 -- 172.21.15.86:0/4000895042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4e007adb0 msgr2=0x7fd4e019c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:43:18.818 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:18.385+0000 7fd4e6600700 1 --2- 172.21.15.86:0/4000895042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4e007adb0 0x7fd4e019c520 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0003e60 tx=0x7fd4d0011a30 comp rx=0 tx=0).stop 2024-03-23T17:43:18.818 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:18.385+0000 7fd4e6600700 1 -- 172.21.15.86:0/4000895042 shutdown_connections 2024-03-23T17:43:18.818 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:18.385+0000 7fd4e6600700 1 --2- 172.21.15.86:0/4000895042 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7fd4c40386e0 0x7fd4c403aba0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:18.818 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:18.385+0000 7fd4e6600700 1 --2- 172.21.15.86:0/4000895042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4e007adb0 0x7fd4e019c520 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:18.818 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:18.385+0000 7fd4e6600700 1 -- 172.21.15.86:0/4000895042 >> 172.21.15.86:0/4000895042 conn(0x7fd4e0101b90 msgr2=0x7fd4e0102790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:43:18.818 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:18.385+0000 7fd4e6600700 1 -- 172.21.15.86:0/4000895042 shutdown_connections 2024-03-23T17:43:18.818 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:18.386+0000 7fd4e6600700 1 -- 172.21.15.86:0/4000895042 wait complete. 2024-03-23T17:43:18.819 INFO:teuthology.orchestra.run.smithi086.stdout:Enabling the dashboard module... 2024-03-23T17:43:19.892 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:19 smithi086 ceph-mon[38221]: from='mgr.14120 172.21.15.86:0/1645921427' entity='mgr.x' 2024-03-23T17:43:19.892 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:19 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/3743625416' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-03-23T17:43:19.892 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:43:19 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: ignoring --setuser ceph since I am not root 2024-03-23T17:43:19.892 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:43:19 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: ignoring --setgroup ceph since I am not root 2024-03-23T17:43:20.034 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:19.244+0000 7ffa5bc00700 1 Processor -- start 2024-03-23T17:43:20.034 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:19.244+0000 7ffa5bc00700 1 -- start start 2024-03-23T17:43:20.034 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:19.245+0000 7ffa5bc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa541087e0 0x7ffa54108c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:43:20.034 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:19.245+0000 7ffa5bc00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa541091d0 con 0x7ffa541087e0 2024-03-23T17:43:20.034 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:19.245+0000 7ffa59200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa541087e0 0x7ffa54108c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:43:20.034 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:19.245+0000 7ffa59200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa541087e0 0x7ffa54108c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49610/0 (socket says 172.21.15.86:49610) 2024-03-23T17:43:20.034 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:19.245+0000 7ffa59200700 1 -- 172.21.15.86:0/3732423051 learned_addr learned my addr 172.21.15.86:0/3732423051 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:43:20.035 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:19.245+0000 7ffa59200700 1 -- 172.21.15.86:0/3732423051 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa54109990 con 0x7ffa541087e0 2024-03-23T17:43:20.035 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:19.246+0000 7ffa59200700 1 --2- 172.21.15.86:0/3732423051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa541087e0 0x7ffa54108c00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ffa480098a0 tx=0x7ffa48009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=275bc857d80d5b76 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:43:20.035 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:19.246+0000 7ffa53400700 1 -- 172.21.15.86:0/3732423051 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffa48004160 con 0x7ffa541087e0 2024-03-23T17:43:20.035 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:19.246+0000 7ffa53400700 1 -- 172.21.15.86:0/3732423051 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa480042c0 con 0x7ffa541087e0 2024-03-23T17:43:20.035 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:19.246+0000 7ffa53400700 1 -- 172.21.15.86:0/3732423051 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffa480119b0 con 0x7ffa541087e0 2024-03-23T17:43:20.035 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:19.247+0000 7ffa5bc00700 1 -- 172.21.15.86:0/3732423051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa541087e0 msgr2=0x7ffa54108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:43:20.035 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:19.247+0000 7ffa5bc00700 1 --2- 172.21.15.86:0/3732423051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa541087e0 0x7ffa54108c00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ffa480098a0 tx=0x7ffa48009bb0 comp rx=0 tx=0).stop 2024-03-23T17:43:20.035 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:19.247+0000 7ffa5bc00700 1 -- 172.21.15.86:0/3732423051 shutdown_connections 2024-03-23T17:43:20.035 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:19.247+0000 7ffa5bc00700 1 --2- 172.21.15.86:0/3732423051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa541087e0 0x7ffa54108c00 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:20.035 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:19.247+0000 7ffa5bc00700 1 -- 172.21.15.86:0/3732423051 >> 172.21.15.86:0/3732423051 conn(0x7ffa5407bc30 msgr2=0x7ffa541064f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:43:20.035 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:19.247+0000 7ffa5bc00700 1 -- 172.21.15.86:0/3732423051 shutdown_connections 2024-03-23T17:43:20.035 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:19.247+0000 7ffa5bc00700 1 -- 172.21.15.86:0/3732423051 wait complete. 2024-03-23T17:43:20.036 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:19.248+0000 7ffa5bc00700 1 Processor -- start 2024-03-23T17:43:20.036 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:19.248+0000 7ffa5bc00700 1 -- start start 2024-03-23T17:43:20.036 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:19.248+0000 7ffa5bc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa541087e0 0x7ffa5419c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:43:20.036 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:19.248+0000 7ffa5bc00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5419ca20 con 0x7ffa541087e0 2024-03-23T17:43:20.036 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:19.249+0000 7ffa59200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa541087e0 0x7ffa5419c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:43:20.036 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:19.249+0000 7ffa59200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa541087e0 0x7ffa5419c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49622/0 (socket says 172.21.15.86:49622) 2024-03-23T17:43:20.036 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:19.249+0000 7ffa59200700 1 -- 172.21.15.86:0/3743625416 learned_addr learned my addr 172.21.15.86:0/3743625416 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:43:20.036 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:19.249+0000 7ffa59200700 1 -- 172.21.15.86:0/3743625416 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa48009550 con 0x7ffa541087e0 2024-03-23T17:43:20.036 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:19.249+0000 7ffa59200700 1 --2- 172.21.15.86:0/3743625416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa541087e0 0x7ffa5419c4e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ffa48003e60 tx=0x7ffa48011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:43:20.036 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:19.249+0000 7ffa51600700 1 -- 172.21.15.86:0/3743625416 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffa48003a60 con 0x7ffa541087e0 2024-03-23T17:43:20.036 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:19.249+0000 7ffa51600700 1 -- 172.21.15.86:0/3743625416 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa48003bc0 con 0x7ffa541087e0 2024-03-23T17:43:20.036 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:19.249+0000 7ffa5bc00700 1 -- 172.21.15.86:0/3743625416 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa5419cc20 con 0x7ffa541087e0 2024-03-23T17:43:20.036 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:19.249+0000 7ffa51600700 1 -- 172.21.15.86:0/3743625416 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffa480196c0 con 0x7ffa541087e0 2024-03-23T17:43:20.036 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:19.250+0000 7ffa5bc00700 1 -- 172.21.15.86:0/3743625416 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa5419d0c0 con 0x7ffa541087e0 2024-03-23T17:43:20.037 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:19.251+0000 7ffa51600700 1 -- 172.21.15.86:0/3743625416 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 8) v1 ==== 45533+0+0 (secure 0 0 0) 0x7ffa4801a430 con 0x7ffa541087e0 2024-03-23T17:43:20.037 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:19.251+0000 7ffa51600700 1 --2- 172.21.15.86:0/3743625416 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7ffa3c038730 0x7ffa3c03abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:43:20.037 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:19.251+0000 7ffa51600700 1 -- 172.21.15.86:0/3743625416 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7ffa4804c990 con 0x7ffa541087e0 2024-03-23T17:43:20.037 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:19.251+0000 7ffa5bc00700 1 -- 172.21.15.86:0/3743625416 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa5410cbc0 con 0x7ffa541087e0 2024-03-23T17:43:20.037 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:19.251+0000 7ffa53e00700 1 --2- 172.21.15.86:0/3743625416 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7ffa3c038730 0x7ffa3c03abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:43:20.037 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:19.252+0000 7ffa53e00700 1 --2- 172.21.15.86:0/3743625416 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7ffa3c038730 0x7ffa3c03abf0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ffa40006fd0 tx=0x7ffa40006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:43:20.037 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:19.256+0000 7ffa51600700 1 -- 172.21.15.86:0/3743625416 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ffa4802c3a0 con 0x7ffa541087e0 2024-03-23T17:43:20.037 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:19.529+0000 7ffa5bc00700 1 -- 172.21.15.86:0/3743625416 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7ffa5419fbb0 con 0x7ffa541087e0 2024-03-23T17:43:20.037 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:19.623+0000 7ffa51600700 1 -- 172.21.15.86:0/3743625416 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mgrmap(e 9) v1 ==== 45546+0+0 (secure 0 0 0) 0x7ffa48047020 con 0x7ffa541087e0 2024-03-23T17:43:20.037 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:19.625+0000 7ffa51600700 1 -- 172.21.15.86:0/3743625416 <== mon.0 v2:172.21.15.86:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7ffa4804c2e0 con 0x7ffa541087e0 2024-03-23T17:43:20.037 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:19.629+0000 7ffa5bc00700 1 -- 172.21.15.86:0/3743625416 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7ffa3c038730 msgr2=0x7ffa3c03abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:43:20.037 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:19.629+0000 7ffa5bc00700 1 --2- 172.21.15.86:0/3743625416 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7ffa3c038730 0x7ffa3c03abf0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ffa40006fd0 tx=0x7ffa40006e40 comp rx=0 tx=0).stop 2024-03-23T17:43:20.037 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:19.629+0000 7ffa5bc00700 1 -- 172.21.15.86:0/3743625416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa541087e0 msgr2=0x7ffa5419c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:43:20.038 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:19.629+0000 7ffa5bc00700 1 --2- 172.21.15.86:0/3743625416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa541087e0 0x7ffa5419c4e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ffa48003e60 tx=0x7ffa48011a30 comp rx=0 tx=0).stop 2024-03-23T17:43:20.038 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:19.629+0000 7ffa5bc00700 1 -- 172.21.15.86:0/3743625416 shutdown_connections 2024-03-23T17:43:20.038 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:19.629+0000 7ffa5bc00700 1 --2- 172.21.15.86:0/3743625416 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7ffa3c038730 0x7ffa3c03abf0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:20.038 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:19.629+0000 7ffa5bc00700 1 --2- 172.21.15.86:0/3743625416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa541087e0 0x7ffa5419c4e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:20.038 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:19.629+0000 7ffa5bc00700 1 -- 172.21.15.86:0/3743625416 >> 172.21.15.86:0/3743625416 conn(0x7ffa5407bc30 msgr2=0x7ffa54105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:43:20.038 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:19.629+0000 7ffa5bc00700 1 -- 172.21.15.86:0/3743625416 shutdown_connections 2024-03-23T17:43:20.038 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:19.629+0000 7ffa5bc00700 1 -- 172.21.15.86:0/3743625416 wait complete. 2024-03-23T17:43:20.515 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:43:20 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:43:20.264+0000 7f141576e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-23T17:43:20.516 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:43:20 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:43:20.395+0000 7f141576e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-23T17:43:20.766 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:20 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/3743625416' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-03-23T17:43:20.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:20 smithi086 ceph-mon[38221]: mgrmap e9: x(active, since 14s) 2024-03-23T17:43:20.767 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:43:20 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:43:20.514+0000 7f141576e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-23T17:43:20.767 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:43:20 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:43:20.640+0000 7f141576e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-23T17:43:21.260 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout { 2024-03-23T17:43:21.260 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-03-23T17:43:21.260 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-23T17:43:21.261 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-23T17:43:21.261 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-23T17:43:21.261 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout } 2024-03-23T17:43:21.261 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:20.602+0000 7fa0eb600700 1 Processor -- start 2024-03-23T17:43:21.261 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:20.602+0000 7fa0eb600700 1 -- start start 2024-03-23T17:43:21.261 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:20.602+0000 7fa0eb600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0e41065b0 0x7fa0e41069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:43:21.261 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:20.602+0000 7fa0eb600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e4106fa0 con 0x7fa0e41065b0 2024-03-23T17:43:21.261 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:20.603+0000 7fa0e8c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0e41065b0 0x7fa0e41069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:43:21.261 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:20.603+0000 7fa0e8c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0e41065b0 0x7fa0e41069d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49654/0 (socket says 172.21.15.86:49654) 2024-03-23T17:43:21.261 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:20.603+0000 7fa0e8c00700 1 -- 172.21.15.86:0/2766912623 learned_addr learned my addr 172.21.15.86:0/2766912623 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:43:21.261 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:20.603+0000 7fa0e8c00700 1 -- 172.21.15.86:0/2766912623 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0e41077b0 con 0x7fa0e41065b0 2024-03-23T17:43:21.261 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:20.603+0000 7fa0e8c00700 1 --2- 172.21.15.86:0/2766912623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0e41065b0 0x7fa0e41069d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa0d80098a0 tx=0x7fa0d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeae7bac47bbc5bc server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:43:21.261 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:20.604+0000 7fa0e3400700 1 -- 172.21.15.86:0/2766912623 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa0d8004160 con 0x7fa0e41065b0 2024-03-23T17:43:21.261 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:20.604+0000 7fa0e3400700 1 -- 172.21.15.86:0/2766912623 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0d80042c0 con 0x7fa0e41065b0 2024-03-23T17:43:21.262 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:20.604+0000 7fa0e3400700 1 -- 172.21.15.86:0/2766912623 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa0d80119b0 con 0x7fa0e41065b0 2024-03-23T17:43:21.262 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:20.604+0000 7fa0eb600700 1 -- 172.21.15.86:0/2766912623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0e41065b0 msgr2=0x7fa0e41069d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:43:21.262 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:20.604+0000 7fa0eb600700 1 --2- 172.21.15.86:0/2766912623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0e41065b0 0x7fa0e41069d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa0d80098a0 tx=0x7fa0d8009bb0 comp rx=0 tx=0).stop 2024-03-23T17:43:21.262 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:20.604+0000 7fa0eb600700 1 -- 172.21.15.86:0/2766912623 shutdown_connections 2024-03-23T17:43:21.262 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:20.604+0000 7fa0eb600700 1 --2- 172.21.15.86:0/2766912623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0e41065b0 0x7fa0e41069d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:21.262 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:20.604+0000 7fa0eb600700 1 -- 172.21.15.86:0/2766912623 >> 172.21.15.86:0/2766912623 conn(0x7fa0e4101b30 msgr2=0x7fa0e4103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:43:21.262 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:20.604+0000 7fa0eb600700 1 -- 172.21.15.86:0/2766912623 shutdown_connections 2024-03-23T17:43:21.262 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:20.604+0000 7fa0eb600700 1 -- 172.21.15.86:0/2766912623 wait complete. 2024-03-23T17:43:21.262 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:20.605+0000 7fa0eb600700 1 Processor -- start 2024-03-23T17:43:21.262 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:20.605+0000 7fa0eb600700 1 -- start start 2024-03-23T17:43:21.262 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:20.606+0000 7fa0eb600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0e41065b0 0x7fa0e419c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:43:21.262 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:20.606+0000 7fa0eb600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e419cab0 con 0x7fa0e41065b0 2024-03-23T17:43:21.262 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:20.606+0000 7fa0e8c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0e41065b0 0x7fa0e419c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:43:21.262 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:20.606+0000 7fa0e8c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0e41065b0 0x7fa0e419c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49660/0 (socket says 172.21.15.86:49660) 2024-03-23T17:43:21.263 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:20.606+0000 7fa0e8c00700 1 -- 172.21.15.86:0/1547484315 learned_addr learned my addr 172.21.15.86:0/1547484315 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:43:21.263 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:20.606+0000 7fa0e8c00700 1 -- 172.21.15.86:0/1547484315 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0d8009550 con 0x7fa0e41065b0 2024-03-23T17:43:21.263 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:20.607+0000 7fa0e8c00700 1 --2- 172.21.15.86:0/1547484315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0e41065b0 0x7fa0e419c570 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa0d8006af0 tx=0x7fa0d8011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:43:21.263 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:20.607+0000 7fa0e1600700 1 -- 172.21.15.86:0/1547484315 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa0d8003a40 con 0x7fa0e41065b0 2024-03-23T17:43:21.263 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:20.607+0000 7fa0e1600700 1 -- 172.21.15.86:0/1547484315 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0d8003ba0 con 0x7fa0e41065b0 2024-03-23T17:43:21.263 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:20.607+0000 7fa0eb600700 1 -- 172.21.15.86:0/1547484315 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0e419ccb0 con 0x7fa0e41065b0 2024-03-23T17:43:21.263 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:20.607+0000 7fa0e1600700 1 -- 172.21.15.86:0/1547484315 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa0d8019690 con 0x7fa0e41065b0 2024-03-23T17:43:21.263 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:20.607+0000 7fa0eb600700 1 -- 172.21.15.86:0/1547484315 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0e419d150 con 0x7fa0e41065b0 2024-03-23T17:43:21.263 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:20.608+0000 7fa0e1600700 1 -- 172.21.15.86:0/1547484315 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 9) v1 ==== 45546+0+0 (secure 0 0 0) 0x7fa0d8019900 con 0x7fa0e41065b0 2024-03-23T17:43:21.263 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:20.608+0000 7fa0eb600700 1 -- 172.21.15.86:0/1547484315 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0e410a9c0 con 0x7fa0e41065b0 2024-03-23T17:43:21.263 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:20.608+0000 7fa0e1600700 1 --2- 172.21.15.86:0/1547484315 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7fa0cc038750 0x7fa0cc03ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:43:21.263 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:20.608+0000 7fa0e1600700 1 -- 172.21.15.86:0/1547484315 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fa0d804c680 con 0x7fa0e41065b0 2024-03-23T17:43:21.263 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:20.609+0000 7fa0e3e00700 1 -- 172.21.15.86:0/1547484315 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7fa0cc038750 msgr2=0x7fa0cc03ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/3600278228 2024-03-23T17:43:21.263 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:20.609+0000 7fa0e3e00700 1 --2- 172.21.15.86:0/1547484315 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7fa0cc038750 0x7fa0cc03ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-23T17:43:21.263 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:20.614+0000 7fa0e1600700 1 -- 172.21.15.86:0/1547484315 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa0d8018760 con 0x7fa0e41065b0 2024-03-23T17:43:21.264 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:20.810+0000 7fa0e3e00700 1 -- 172.21.15.86:0/1547484315 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7fa0cc038750 msgr2=0x7fa0cc03ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/3600278228 2024-03-23T17:43:21.264 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:20.810+0000 7fa0e3e00700 1 --2- 172.21.15.86:0/1547484315 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7fa0cc038750 0x7fa0cc03ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-23T17:43:21.264 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:20.900+0000 7fa0eb600700 1 -- 172.21.15.86:0/1547484315 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fa0e419f9e0 con 0x7fa0e41065b0 2024-03-23T17:43:21.264 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:20.901+0000 7fa0e1600700 1 -- 172.21.15.86:0/1547484315 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7fa0d8023390 con 0x7fa0e41065b0 2024-03-23T17:43:21.264 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:20.903+0000 7fa0eb600700 1 -- 172.21.15.86:0/1547484315 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7fa0cc038750 msgr2=0x7fa0cc03ac10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-23T17:43:21.264 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:20.903+0000 7fa0eb600700 1 --2- 172.21.15.86:0/1547484315 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7fa0cc038750 0x7fa0cc03ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:21.264 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:20.903+0000 7fa0eb600700 1 -- 172.21.15.86:0/1547484315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0e41065b0 msgr2=0x7fa0e419c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:43:21.264 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:20.903+0000 7fa0eb600700 1 --2- 172.21.15.86:0/1547484315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0e41065b0 0x7fa0e419c570 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa0d8006af0 tx=0x7fa0d8011a10 comp rx=0 tx=0).stop 2024-03-23T17:43:21.264 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:20.903+0000 7fa0eb600700 1 -- 172.21.15.86:0/1547484315 shutdown_connections 2024-03-23T17:43:21.264 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:20.903+0000 7fa0eb600700 1 --2- 172.21.15.86:0/1547484315 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7fa0cc038750 0x7fa0cc03ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:21.264 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:20.903+0000 7fa0eb600700 1 --2- 172.21.15.86:0/1547484315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0e41065b0 0x7fa0e419c570 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:21.264 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:20.903+0000 7fa0eb600700 1 -- 172.21.15.86:0/1547484315 >> 172.21.15.86:0/1547484315 conn(0x7fa0e4101b30 msgr2=0x7fa0e41027a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:43:21.264 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:20.903+0000 7fa0eb600700 1 -- 172.21.15.86:0/1547484315 shutdown_connections 2024-03-23T17:43:21.264 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:20.903+0000 7fa0eb600700 1 -- 172.21.15.86:0/1547484315 wait complete. 2024-03-23T17:43:21.264 INFO:teuthology.orchestra.run.smithi086.stdout:Waiting for the mgr to restart... 2024-03-23T17:43:21.265 INFO:teuthology.orchestra.run.smithi086.stdout:Waiting for mgr epoch 9... 2024-03-23T17:43:21.766 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:21 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/1547484315' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-23T17:43:21.767 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:43:21 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:43:21.512+0000 7f141576e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-23T17:43:22.266 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:43:21 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:43:21.831+0000 7f141576e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-23T17:43:22.267 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:43:21 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:43:21.941+0000 7f141576e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-23T17:43:22.267 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:43:22 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:43:22.080+0000 7f141576e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-23T17:43:22.553 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:43:22 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:43:22.275+0000 7f141576e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-23T17:43:22.553 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:43:22 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:43:22.441+0000 7f141576e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-23T17:43:23.016 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:43:22 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:43:22.552+0000 7f141576e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-23T17:43:23.498 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:43:23 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:43:23.017+0000 7f141576e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-23T17:43:23.766 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:43:23 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:43:23.498+0000 7f141576e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-23T17:43:23.767 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:43:23 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:43:23.743+0000 7f141576e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-23T17:43:24.266 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:43:23 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:43:23.865+0000 7f141576e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-23T17:43:25.110 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:43:24 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:43:24.797+0000 7f141576e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-23T17:43:25.110 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:43:24 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:43:24.920+0000 7f141576e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-23T17:43:26.266 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:43:25 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:43:25.109+0000 7f141576e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-23T17:43:26.267 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:43:25 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:43:25.901+0000 7f141576e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-23T17:43:26.267 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:43:26 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:43:26.139+0000 7f141576e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-23T17:43:26.766 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:43:26 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:43:26.389+0000 7f141576e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-23T17:43:27.766 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:43:27 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:43:27.506+0000 7f141576e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-23T17:43:28.240 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:43:27 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:43:27.875+0000 7f141576e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-23T17:43:28.240 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:43:27 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:43:27.990+0000 7f141576e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-23T17:43:28.240 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:43:28 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:43:28.238+0000 7f141576e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-23T17:43:28.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:28 smithi086 ceph-mon[38221]: Active manager daemon x restarted 2024-03-23T17:43:28.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:28 smithi086 ceph-mon[38221]: Activating manager daemon x 2024-03-23T17:43:28.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:28 smithi086 ceph-mon[38221]: osdmap e3: 0 total, 0 up, 0 in 2024-03-23T17:43:28.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:28 smithi086 ceph-mon[38221]: mgrmap e10: x(active, starting, since 0.00415358s) 2024-03-23T17:43:28.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:28 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-23T17:43:28.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:28 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-23T17:43:28.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:28 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-23T17:43:28.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:28 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-23T17:43:28.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:28 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-23T17:43:28.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:28 smithi086 ceph-mon[38221]: Manager daemon x is now available 2024-03-23T17:43:28.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:28 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:43:28.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:28 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:43:29.612 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:29 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-23T17:43:29.612 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:29 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-23T17:43:29.612 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:29 smithi086 ceph-mon[38221]: [23/Mar/2024:17:43:28] ENGINE Bus STARTING 2024-03-23T17:43:29.612 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:29 smithi086 ceph-mon[38221]: mgrmap e11: x(active, since 1.00747s) 2024-03-23T17:43:29.612 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout { 2024-03-23T17:43:29.620 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-03-23T17:43:29.621 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-23T17:43:29.621 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout } 2024-03-23T17:43:29.621 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:21.674+0000 7f1c42600700 1 Processor -- start 2024-03-23T17:43:29.621 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:21.674+0000 7f1c42600700 1 -- start start 2024-03-23T17:43:29.621 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:21.674+0000 7f1c42600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c3c1065b0 0x7f1c3c1069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:43:29.621 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:21.674+0000 7f1c42600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c3c106fa0 con 0x7f1c3c1065b0 2024-03-23T17:43:29.621 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:21.675+0000 7f1c3b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c3c1065b0 0x7f1c3c1069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:43:29.621 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:21.675+0000 7f1c3b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c3c1065b0 0x7f1c3c1069d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49666/0 (socket says 172.21.15.86:49666) 2024-03-23T17:43:29.621 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:21.675+0000 7f1c3b400700 1 -- 172.21.15.86:0/114062386 learned_addr learned my addr 172.21.15.86:0/114062386 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:43:29.621 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:21.675+0000 7f1c3b400700 1 -- 172.21.15.86:0/114062386 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c3c1077b0 con 0x7f1c3c1065b0 2024-03-23T17:43:29.621 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:21.676+0000 7f1c3b400700 1 --2- 172.21.15.86:0/114062386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c3c1065b0 0x7f1c3c1069d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c0098a0 tx=0x7f1c2c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4004f20f094ced9a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:43:29.621 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:21.676+0000 7f1c3a000700 1 -- 172.21.15.86:0/114062386 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1c2c004160 con 0x7f1c3c1065b0 2024-03-23T17:43:29.622 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:21.676+0000 7f1c3a000700 1 -- 172.21.15.86:0/114062386 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c2c0042c0 con 0x7f1c3c1065b0 2024-03-23T17:43:29.622 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:21.676+0000 7f1c3a000700 1 -- 172.21.15.86:0/114062386 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1c2c0119b0 con 0x7f1c3c1065b0 2024-03-23T17:43:29.622 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:21.676+0000 7f1c42600700 1 -- 172.21.15.86:0/114062386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c3c1065b0 msgr2=0x7f1c3c1069d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:43:29.622 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:21.676+0000 7f1c42600700 1 --2- 172.21.15.86:0/114062386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c3c1065b0 0x7f1c3c1069d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c0098a0 tx=0x7f1c2c009bb0 comp rx=0 tx=0).stop 2024-03-23T17:43:29.622 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:21.676+0000 7f1c42600700 1 -- 172.21.15.86:0/114062386 shutdown_connections 2024-03-23T17:43:29.622 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:21.676+0000 7f1c42600700 1 --2- 172.21.15.86:0/114062386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c3c1065b0 0x7f1c3c1069d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:29.622 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:21.676+0000 7f1c42600700 1 -- 172.21.15.86:0/114062386 >> 172.21.15.86:0/114062386 conn(0x7f1c3c101b70 msgr2=0x7f1c3c103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:43:29.622 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:21.677+0000 7f1c42600700 1 -- 172.21.15.86:0/114062386 shutdown_connections 2024-03-23T17:43:29.622 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:21.677+0000 7f1c42600700 1 -- 172.21.15.86:0/114062386 wait complete. 2024-03-23T17:43:29.622 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:21.678+0000 7f1c42600700 1 Processor -- start 2024-03-23T17:43:29.622 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:21.678+0000 7f1c42600700 1 -- start start 2024-03-23T17:43:29.622 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:21.679+0000 7f1c42600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c3c1065b0 0x7f1c3c19a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:43:29.622 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:21.679+0000 7f1c42600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c3c19a880 con 0x7f1c3c1065b0 2024-03-23T17:43:29.622 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:21.679+0000 7f1c3b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c3c1065b0 0x7f1c3c19a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:43:29.623 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:21.679+0000 7f1c3b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c3c1065b0 0x7f1c3c19a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49680/0 (socket says 172.21.15.86:49680) 2024-03-23T17:43:29.623 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:21.679+0000 7f1c3b400700 1 -- 172.21.15.86:0/183088160 learned_addr learned my addr 172.21.15.86:0/183088160 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:43:29.623 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:21.679+0000 7f1c3b400700 1 -- 172.21.15.86:0/183088160 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c2c009550 con 0x7f1c3c1065b0 2024-03-23T17:43:29.623 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:21.679+0000 7f1c3b400700 1 --2- 172.21.15.86:0/183088160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c3c1065b0 0x7f1c3c19a340 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c003e60 tx=0x7f1c2c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:43:29.623 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:21.680+0000 7f1c33e00700 1 -- 172.21.15.86:0/183088160 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1c2c003a60 con 0x7f1c3c1065b0 2024-03-23T17:43:29.623 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:21.680+0000 7f1c33e00700 1 -- 172.21.15.86:0/183088160 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c2c003bc0 con 0x7f1c3c1065b0 2024-03-23T17:43:29.623 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:21.680+0000 7f1c42600700 1 -- 172.21.15.86:0/183088160 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c3c19aa80 con 0x7f1c3c1065b0 2024-03-23T17:43:29.623 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:21.680+0000 7f1c33e00700 1 -- 172.21.15.86:0/183088160 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1c2c0196c0 con 0x7f1c3c1065b0 2024-03-23T17:43:29.623 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:21.680+0000 7f1c42600700 1 -- 172.21.15.86:0/183088160 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c3c19af20 con 0x7f1c3c1065b0 2024-03-23T17:43:29.623 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:21.681+0000 7f1c33e00700 1 -- 172.21.15.86:0/183088160 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 9) v1 ==== 45546+0+0 (secure 0 0 0) 0x7f1c2c01a430 con 0x7f1c3c1065b0 2024-03-23T17:43:29.623 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:21.681+0000 7f1c33e00700 1 --2- 172.21.15.86:0/183088160 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7f1c20038780 0x7f1c2003ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:43:29.623 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:21.681+0000 7f1c33e00700 1 -- 172.21.15.86:0/183088160 --> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1c2003b350 con 0x7f1c20038780 2024-03-23T17:43:29.623 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:21.682+0000 7f1c33e00700 1 -- 172.21.15.86:0/183088160 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f1c2c04c9e0 con 0x7f1c3c1065b0 2024-03-23T17:43:29.624 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:21.682+0000 7f1c3aa00700 1 -- 172.21.15.86:0/183088160 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7f1c20038780 msgr2=0x7f1c2003ac40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/3600278228 2024-03-23T17:43:29.624 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:21.682+0000 7f1c3aa00700 1 --2- 172.21.15.86:0/183088160 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7f1c20038780 0x7f1c2003ac40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-23T17:43:29.624 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:21.883+0000 7f1c3aa00700 1 -- 172.21.15.86:0/183088160 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7f1c20038780 msgr2=0x7f1c2003ac40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/3600278228 2024-03-23T17:43:29.624 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:21.883+0000 7f1c3aa00700 1 --2- 172.21.15.86:0/183088160 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7f1c20038780 0x7f1c2003ac40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-23T17:43:29.624 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:22.284+0000 7f1c3aa00700 1 -- 172.21.15.86:0/183088160 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7f1c20038780 msgr2=0x7f1c2003ac40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/3600278228 2024-03-23T17:43:29.624 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:22.284+0000 7f1c3aa00700 1 --2- 172.21.15.86:0/183088160 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7f1c20038780 0x7f1c2003ac40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-23T17:43:29.624 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:23.085+0000 7f1c3aa00700 1 -- 172.21.15.86:0/183088160 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7f1c20038780 msgr2=0x7f1c2003ac40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/3600278228 2024-03-23T17:43:29.624 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:23.085+0000 7f1c3aa00700 1 --2- 172.21.15.86:0/183088160 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7f1c20038780 0x7f1c2003ac40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-23T17:43:29.624 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:24.686+0000 7f1c3aa00700 1 -- 172.21.15.86:0/183088160 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7f1c20038780 msgr2=0x7f1c2003ac40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/3600278228 2024-03-23T17:43:29.624 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:24.687+0000 7f1c3aa00700 1 --2- 172.21.15.86:0/183088160 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7f1c20038780 0x7f1c2003ac40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-23T17:43:29.624 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:27.889+0000 7f1c3aa00700 1 -- 172.21.15.86:0/183088160 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7f1c20038780 msgr2=0x7f1c2003ac40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/3600278228 2024-03-23T17:43:29.624 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:27.889+0000 7f1c3aa00700 1 --2- 172.21.15.86:0/183088160 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7f1c20038780 0x7f1c2003ac40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-03-23T17:43:29.624 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:28.243+0000 7f1c33e00700 1 -- 172.21.15.86:0/183088160 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mgrmap(e 10) v1 ==== 45313+0+0 (secure 0 0 0) 0x7f1c2c019a20 con 0x7f1c3c1065b0 2024-03-23T17:43:29.624 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:28.244+0000 7f1c33e00700 1 -- 172.21.15.86:0/183088160 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7f1c20038780 msgr2=0x7f1c2003ac40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-23T17:43:29.625 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:28.244+0000 7f1c33e00700 1 --2- 172.21.15.86:0/183088160 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7f1c20038780 0x7f1c2003ac40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:29.625 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:29.245+0000 7f1c33e00700 1 -- 172.21.15.86:0/183088160 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mgrmap(e 11) v1 ==== 45440+0+0 (secure 0 0 0) 0x7f1c2c04e110 con 0x7f1c3c1065b0 2024-03-23T17:43:29.625 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:29.246+0000 7f1c33e00700 1 --2- 172.21.15.86:0/183088160 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f1c2003bff0 0x7f1c2003e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:43:29.625 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:29.246+0000 7f1c33e00700 1 -- 172.21.15.86:0/183088160 --> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1c2003b350 con 0x7f1c2003bff0 2024-03-23T17:43:29.625 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:29.246+0000 7f1c3aa00700 1 --2- 172.21.15.86:0/183088160 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f1c2003bff0 0x7f1c2003e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:43:29.625 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:29.247+0000 7f1c3aa00700 1 --2- 172.21.15.86:0/183088160 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f1c2003bff0 0x7f1c2003e3e0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f1c28003a10 tx=0x7f1c28009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:43:29.625 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:29.248+0000 7f1c33e00700 1 -- 172.21.15.86:0/183088160 <== mgr.14152 v2:172.21.15.86:6800/2734784244 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7f1c2003b350 con 0x7f1c2003bff0 2024-03-23T17:43:29.625 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:29.254+0000 7f1c42600700 1 -- 172.21.15.86:0/183088160 --> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f1c3c10a9e0 con 0x7f1c2003bff0 2024-03-23T17:43:29.625 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:29.255+0000 7f1c33e00700 1 -- 172.21.15.86:0/183088160 <== mgr.14152 v2:172.21.15.86:6800/2734784244 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f1c3c10a9e0 con 0x7f1c2003bff0 2024-03-23T17:43:29.625 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:29.255+0000 7f1c42600700 1 -- 172.21.15.86:0/183088160 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f1c2003bff0 msgr2=0x7f1c2003e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:43:29.625 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:29.255+0000 7f1c42600700 1 --2- 172.21.15.86:0/183088160 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f1c2003bff0 0x7f1c2003e3e0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f1c28003a10 tx=0x7f1c28009270 comp rx=0 tx=0).stop 2024-03-23T17:43:29.625 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:29.255+0000 7f1c42600700 1 -- 172.21.15.86:0/183088160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c3c1065b0 msgr2=0x7f1c3c19a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:43:29.625 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:29.255+0000 7f1c42600700 1 --2- 172.21.15.86:0/183088160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c3c1065b0 0x7f1c3c19a340 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c003e60 tx=0x7f1c2c011a30 comp rx=0 tx=0).stop 2024-03-23T17:43:29.625 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:29.255+0000 7f1c42600700 1 -- 172.21.15.86:0/183088160 shutdown_connections 2024-03-23T17:43:29.626 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:29.255+0000 7f1c42600700 1 --2- 172.21.15.86:0/183088160 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f1c2003bff0 0x7f1c2003e3e0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:29.626 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:29.255+0000 7f1c42600700 1 --2- 172.21.15.86:0/183088160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c3c1065b0 0x7f1c3c19a340 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:29.626 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:29.255+0000 7f1c42600700 1 --2- 172.21.15.86:0/183088160 >> [v2:172.21.15.86:6800/3600278228,v1:172.21.15.86:6801/3600278228] conn(0x7f1c20038780 0x7f1c2003ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:29.626 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:29.255+0000 7f1c42600700 1 -- 172.21.15.86:0/183088160 >> 172.21.15.86:0/183088160 conn(0x7f1c3c101b70 msgr2=0x7f1c3c102790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:43:29.626 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:29.255+0000 7f1c42600700 1 -- 172.21.15.86:0/183088160 shutdown_connections 2024-03-23T17:43:29.626 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:29.255+0000 7f1c42600700 1 -- 172.21.15.86:0/183088160 wait complete. 2024-03-23T17:43:29.626 INFO:teuthology.orchestra.run.smithi086.stdout:mgr epoch 9 is available 2024-03-23T17:43:29.626 INFO:teuthology.orchestra.run.smithi086.stdout:Generating a dashboard self-signed certificate... 2024-03-23T17:43:30.708 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-03-23T17:43:30.709 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:30.040+0000 7f59b1600700 1 Processor -- start 2024-03-23T17:43:30.709 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:30.040+0000 7f59b1600700 1 -- start start 2024-03-23T17:43:30.709 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:30.040+0000 7f59b1600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59ac07adb0 0x7f59ac079210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:43:30.709 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:30.040+0000 7f59b1600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59ac0797e0 con 0x7f59ac07adb0 2024-03-23T17:43:30.709 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:30.040+0000 7f59aaa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59ac07adb0 0x7f59ac079210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:43:30.709 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:30 smithi086 ceph-mon[38221]: [23/Mar/2024:17:43:28] ENGINE Serving on https://172.21.15.86:7150 2024-03-23T17:43:30.709 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:30 smithi086 ceph-mon[38221]: [23/Mar/2024:17:43:28] ENGINE Serving on http://172.21.15.86:8765 2024-03-23T17:43:30.709 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:30 smithi086 ceph-mon[38221]: [23/Mar/2024:17:43:28] ENGINE Bus STARTED 2024-03-23T17:43:30.709 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:30 smithi086 ceph-mon[38221]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-23T17:43:30.710 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:30 smithi086 ceph-mon[38221]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-23T17:43:30.710 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:30 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:43:30.710 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:30 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:43:30.710 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:30 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:43:30.714 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:30.041+0000 7f59aaa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59ac07adb0 0x7f59ac079210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38896/0 (socket says 172.21.15.86:38896) 2024-03-23T17:43:30.714 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:30.041+0000 7f59aaa00700 1 -- 172.21.15.86:0/12223531 learned_addr learned my addr 172.21.15.86:0/12223531 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:43:30.715 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:30.041+0000 7f59aaa00700 1 -- 172.21.15.86:0/12223531 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59ac079920 con 0x7f59ac07adb0 2024-03-23T17:43:30.715 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:30.041+0000 7f59aaa00700 1 --2- 172.21.15.86:0/12223531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59ac07adb0 0x7f59ac079210 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f599c0098a0 tx=0x7f599c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e8e23a0f15306b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:43:30.715 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:30.041+0000 7f59a9600700 1 -- 172.21.15.86:0/12223531 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f599c004160 con 0x7f59ac07adb0 2024-03-23T17:43:30.715 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:30.041+0000 7f59a9600700 1 -- 172.21.15.86:0/12223531 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f599c0042c0 con 0x7f59ac07adb0 2024-03-23T17:43:30.715 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:30.042+0000 7f59a9600700 1 -- 172.21.15.86:0/12223531 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f599c0119b0 con 0x7f59ac07adb0 2024-03-23T17:43:30.715 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:30.042+0000 7f59b1600700 1 -- 172.21.15.86:0/12223531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59ac07adb0 msgr2=0x7f59ac079210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:43:30.715 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:30.042+0000 7f59b1600700 1 --2- 172.21.15.86:0/12223531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59ac07adb0 0x7f59ac079210 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f599c0098a0 tx=0x7f599c009bb0 comp rx=0 tx=0).stop 2024-03-23T17:43:30.715 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:30.042+0000 7f59b1600700 1 -- 172.21.15.86:0/12223531 shutdown_connections 2024-03-23T17:43:30.715 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:30.042+0000 7f59b1600700 1 --2- 172.21.15.86:0/12223531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59ac07adb0 0x7f59ac079210 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:30.715 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:30.042+0000 7f59b1600700 1 -- 172.21.15.86:0/12223531 >> 172.21.15.86:0/12223531 conn(0x7f59ac101b30 msgr2=0x7f59ac103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:43:30.715 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:30.042+0000 7f59b1600700 1 -- 172.21.15.86:0/12223531 shutdown_connections 2024-03-23T17:43:30.715 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:30.042+0000 7f59b1600700 1 -- 172.21.15.86:0/12223531 wait complete. 2024-03-23T17:43:30.715 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:30.043+0000 7f59b1600700 1 Processor -- start 2024-03-23T17:43:30.716 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:30.044+0000 7f59b1600700 1 -- start start 2024-03-23T17:43:30.716 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:30.044+0000 7f59b1600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59ac07adb0 0x7f59ac19c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:43:30.716 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:30.044+0000 7f59b1600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59ac19cab0 con 0x7f59ac07adb0 2024-03-23T17:43:30.716 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:30.044+0000 7f59aaa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59ac07adb0 0x7f59ac19c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:43:30.716 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:30.044+0000 7f59aaa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59ac07adb0 0x7f59ac19c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38910/0 (socket says 172.21.15.86:38910) 2024-03-23T17:43:30.716 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:30.044+0000 7f59aaa00700 1 -- 172.21.15.86:0/1076193113 learned_addr learned my addr 172.21.15.86:0/1076193113 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:43:30.716 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:30.045+0000 7f59aaa00700 1 -- 172.21.15.86:0/1076193113 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f599c009550 con 0x7f59ac07adb0 2024-03-23T17:43:30.716 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:30.045+0000 7f59aaa00700 1 --2- 172.21.15.86:0/1076193113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59ac07adb0 0x7f59ac19c570 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f599c003e60 tx=0x7f599c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:43:30.716 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:30.045+0000 7f59a3400700 1 -- 172.21.15.86:0/1076193113 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f599c003a60 con 0x7f59ac07adb0 2024-03-23T17:43:30.716 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:30.045+0000 7f59a3400700 1 -- 172.21.15.86:0/1076193113 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f599c003bc0 con 0x7f59ac07adb0 2024-03-23T17:43:30.716 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:30.045+0000 7f59b1600700 1 -- 172.21.15.86:0/1076193113 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59ac19ccb0 con 0x7f59ac07adb0 2024-03-23T17:43:30.716 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:30.045+0000 7f59a3400700 1 -- 172.21.15.86:0/1076193113 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f599c0196c0 con 0x7f59ac07adb0 2024-03-23T17:43:30.716 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:30.045+0000 7f59b1600700 1 -- 172.21.15.86:0/1076193113 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59ac19d150 con 0x7f59ac07adb0 2024-03-23T17:43:30.716 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:30.046+0000 7f59a3400700 1 -- 172.21.15.86:0/1076193113 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 11) v1 ==== 45440+0+0 (secure 0 0 0) 0x7f599c01a430 con 0x7f59ac07adb0 2024-03-23T17:43:30.717 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:30.047+0000 7f59b1600700 1 -- 172.21.15.86:0/1076193113 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59ac10c9d0 con 0x7f59ac07adb0 2024-03-23T17:43:30.717 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:30.047+0000 7f59a3400700 1 --2- 172.21.15.86:0/1076193113 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f5990038620 0x7f599003aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:43:30.717 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:30.047+0000 7f59a3400700 1 -- 172.21.15.86:0/1076193113 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f599c04c8f0 con 0x7f59ac07adb0 2024-03-23T17:43:30.717 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:30.047+0000 7f59aa000700 1 --2- 172.21.15.86:0/1076193113 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f5990038620 0x7f599003aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:43:30.717 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:30.047+0000 7f59aa000700 1 --2- 172.21.15.86:0/1076193113 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f5990038620 0x7f599003aae0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5998006fd0 tx=0x7f5998006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:43:30.717 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:30.052+0000 7f59a3400700 1 -- 172.21.15.86:0/1076193113 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f599c01ab60 con 0x7f59ac07adb0 2024-03-23T17:43:30.717 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:30.284+0000 7f59b1600700 1 -- 172.21.15.86:0/1076193113 --> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f59ac19f9e0 con 0x7f5990038620 2024-03-23T17:43:30.717 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:30.343+0000 7f59a3400700 1 -- 172.21.15.86:0/1076193113 <== mgr.14152 v2:172.21.15.86:6800/2734784244 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f59ac19f9e0 con 0x7f5990038620 2024-03-23T17:43:30.717 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:30.347+0000 7f59b1600700 1 -- 172.21.15.86:0/1076193113 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f5990038620 msgr2=0x7f599003aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:43:30.717 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:30.347+0000 7f59b1600700 1 --2- 172.21.15.86:0/1076193113 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f5990038620 0x7f599003aae0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5998006fd0 tx=0x7f5998006e40 comp rx=0 tx=0).stop 2024-03-23T17:43:30.717 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:30.347+0000 7f59b1600700 1 -- 172.21.15.86:0/1076193113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59ac07adb0 msgr2=0x7f59ac19c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:43:30.717 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:30.347+0000 7f59b1600700 1 --2- 172.21.15.86:0/1076193113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59ac07adb0 0x7f59ac19c570 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f599c003e60 tx=0x7f599c011a30 comp rx=0 tx=0).stop 2024-03-23T17:43:30.717 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:30.347+0000 7f59b1600700 1 -- 172.21.15.86:0/1076193113 shutdown_connections 2024-03-23T17:43:30.717 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:30.348+0000 7f59b1600700 1 --2- 172.21.15.86:0/1076193113 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f5990038620 0x7f599003aae0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:30.718 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:30.348+0000 7f59b1600700 1 --2- 172.21.15.86:0/1076193113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59ac07adb0 0x7f59ac19c570 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:30.718 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:30.348+0000 7f59b1600700 1 -- 172.21.15.86:0/1076193113 >> 172.21.15.86:0/1076193113 conn(0x7f59ac101b30 msgr2=0x7f59ac102730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:43:30.718 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:30.348+0000 7f59b1600700 1 -- 172.21.15.86:0/1076193113 shutdown_connections 2024-03-23T17:43:30.718 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:30.348+0000 7f59b1600700 1 -- 172.21.15.86:0/1076193113 wait complete. 2024-03-23T17:43:30.718 INFO:teuthology.orchestra.run.smithi086.stdout:Creating initial admin user... 2024-03-23T17:43:31.766 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:31 smithi086 ceph-mon[38221]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:43:31.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:31 smithi086 ceph-mon[38221]: mgrmap e12: x(active, since 2s) 2024-03-23T17:43:31.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:31 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:43:32.034 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$cweeOuGOTgItBEcnfaUueOwcnlTnwSEKo3Ms52DJbYLqWgcqpbOU6", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1711215811, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-03-23T17:43:32.034 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:31.127+0000 7fd888200700 1 Processor -- start 2024-03-23T17:43:32.034 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:31.127+0000 7fd888200700 1 -- start start 2024-03-23T17:43:32.035 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:31.128+0000 7fd888200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd88007ad90 0x7fd8800791f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:43:32.035 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:31.128+0000 7fd888200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8800797c0 con 0x7fd88007ad90 2024-03-23T17:43:32.035 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:31.128+0000 7fd885800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd88007ad90 0x7fd8800791f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:43:32.035 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:31.128+0000 7fd885800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd88007ad90 0x7fd8800791f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38920/0 (socket says 172.21.15.86:38920) 2024-03-23T17:43:32.035 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:31.128+0000 7fd885800700 1 -- 172.21.15.86:0/916165188 learned_addr learned my addr 172.21.15.86:0/916165188 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:43:32.035 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:31.129+0000 7fd885800700 1 -- 172.21.15.86:0/916165188 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd880079900 con 0x7fd88007ad90 2024-03-23T17:43:32.035 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:31.129+0000 7fd885800700 1 --2- 172.21.15.86:0/916165188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd88007ad90 0x7fd8800791f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd8740098a0 tx=0x7fd874009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91160da2b95c78d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:43:32.035 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:31.129+0000 7fd87fe00700 1 -- 172.21.15.86:0/916165188 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd874004160 con 0x7fd88007ad90 2024-03-23T17:43:32.035 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:31.129+0000 7fd87fe00700 1 -- 172.21.15.86:0/916165188 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd8740042c0 con 0x7fd88007ad90 2024-03-23T17:43:32.035 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:31.129+0000 7fd87fe00700 1 -- 172.21.15.86:0/916165188 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd8740119b0 con 0x7fd88007ad90 2024-03-23T17:43:32.035 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:31.130+0000 7fd888200700 1 -- 172.21.15.86:0/916165188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd88007ad90 msgr2=0x7fd8800791f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:43:32.035 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:31.130+0000 7fd888200700 1 --2- 172.21.15.86:0/916165188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd88007ad90 0x7fd8800791f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd8740098a0 tx=0x7fd874009bb0 comp rx=0 tx=0).stop 2024-03-23T17:43:32.035 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:31.130+0000 7fd888200700 1 -- 172.21.15.86:0/916165188 shutdown_connections 2024-03-23T17:43:32.036 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:31.130+0000 7fd888200700 1 --2- 172.21.15.86:0/916165188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd88007ad90 0x7fd8800791f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:32.036 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:31.130+0000 7fd888200700 1 -- 172.21.15.86:0/916165188 >> 172.21.15.86:0/916165188 conn(0x7fd880101b30 msgr2=0x7fd880103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:43:32.036 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:31.130+0000 7fd888200700 1 -- 172.21.15.86:0/916165188 shutdown_connections 2024-03-23T17:43:32.036 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:31.130+0000 7fd888200700 1 -- 172.21.15.86:0/916165188 wait complete. 2024-03-23T17:43:32.036 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:31.131+0000 7fd888200700 1 Processor -- start 2024-03-23T17:43:32.036 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:31.131+0000 7fd888200700 1 -- start start 2024-03-23T17:43:32.036 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:31.132+0000 7fd888200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd88007ad90 0x7fd88019c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:43:32.036 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:31.132+0000 7fd888200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd88019ca40 con 0x7fd88007ad90 2024-03-23T17:43:32.036 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:31.132+0000 7fd885800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd88007ad90 0x7fd88019c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:43:32.036 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:31.132+0000 7fd885800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd88007ad90 0x7fd88019c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38928/0 (socket says 172.21.15.86:38928) 2024-03-23T17:43:32.036 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:31.132+0000 7fd885800700 1 -- 172.21.15.86:0/3331611603 learned_addr learned my addr 172.21.15.86:0/3331611603 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:43:32.036 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:31.132+0000 7fd885800700 1 -- 172.21.15.86:0/3331611603 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd874009550 con 0x7fd88007ad90 2024-03-23T17:43:32.036 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:31.133+0000 7fd885800700 1 --2- 172.21.15.86:0/3331611603 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd88007ad90 0x7fd88019c500 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd874011670 tx=0x7fd874011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:43:32.036 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:31.133+0000 7fd87e000700 1 -- 172.21.15.86:0/3331611603 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd874003b20 con 0x7fd88007ad90 2024-03-23T17:43:32.037 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:31.133+0000 7fd87e000700 1 -- 172.21.15.86:0/3331611603 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd874003c80 con 0x7fd88007ad90 2024-03-23T17:43:32.037 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:31.133+0000 7fd888200700 1 -- 172.21.15.86:0/3331611603 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd88019cc40 con 0x7fd88007ad90 2024-03-23T17:43:32.037 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:31.133+0000 7fd87e000700 1 -- 172.21.15.86:0/3331611603 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd874019770 con 0x7fd88007ad90 2024-03-23T17:43:32.037 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:31.133+0000 7fd888200700 1 -- 172.21.15.86:0/3331611603 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd88019d0e0 con 0x7fd88007ad90 2024-03-23T17:43:32.037 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:31.134+0000 7fd87e000700 1 -- 172.21.15.86:0/3331611603 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 12) v1 ==== 45546+0+0 (secure 0 0 0) 0x7fd87401ab60 con 0x7fd88007ad90 2024-03-23T17:43:32.037 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:31.134+0000 7fd888200700 1 -- 172.21.15.86:0/3331611603 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd88010c9f0 con 0x7fd88007ad90 2024-03-23T17:43:32.037 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:31.135+0000 7fd87e000700 1 --2- 172.21.15.86:0/3331611603 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fd8680387a0 0x7fd86803ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:43:32.037 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:31.135+0000 7fd87e000700 1 -- 172.21.15.86:0/3331611603 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd87404cb60 con 0x7fd88007ad90 2024-03-23T17:43:32.037 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:31.135+0000 7fd884e00700 1 --2- 172.21.15.86:0/3331611603 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fd8680387a0 0x7fd86803ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:43:32.037 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:31.135+0000 7fd884e00700 1 --2- 172.21.15.86:0/3331611603 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fd8680387a0 0x7fd86803ac60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd86c006fd0 tx=0x7fd86c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:43:32.037 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:31.140+0000 7fd87e000700 1 -- 172.21.15.86:0/3331611603 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fd87401a430 con 0x7fd88007ad90 2024-03-23T17:43:32.037 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:31.376+0000 7fd888200700 1 -- 172.21.15.86:0/3331611603 --> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fd880062200 con 0x7fd8680387a0 2024-03-23T17:43:32.037 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:31.607+0000 7fd87e000700 1 -- 172.21.15.86:0/3331611603 <== mgr.14152 v2:172.21.15.86:6800/2734784244 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7fd880062200 con 0x7fd8680387a0 2024-03-23T17:43:32.037 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:31.610+0000 7fd888200700 1 -- 172.21.15.86:0/3331611603 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fd8680387a0 msgr2=0x7fd86803ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:43:32.038 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:31.610+0000 7fd888200700 1 --2- 172.21.15.86:0/3331611603 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fd8680387a0 0x7fd86803ac60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd86c006fd0 tx=0x7fd86c006e40 comp rx=0 tx=0).stop 2024-03-23T17:43:32.038 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:31.610+0000 7fd888200700 1 -- 172.21.15.86:0/3331611603 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd88007ad90 msgr2=0x7fd88019c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:43:32.038 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:31.610+0000 7fd888200700 1 --2- 172.21.15.86:0/3331611603 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd88007ad90 0x7fd88019c500 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd874011670 tx=0x7fd874011a30 comp rx=0 tx=0).stop 2024-03-23T17:43:32.038 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:31.610+0000 7fd888200700 1 -- 172.21.15.86:0/3331611603 shutdown_connections 2024-03-23T17:43:32.038 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:31.610+0000 7fd888200700 1 --2- 172.21.15.86:0/3331611603 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fd8680387a0 0x7fd86803ac60 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:32.038 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:31.610+0000 7fd888200700 1 --2- 172.21.15.86:0/3331611603 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd88007ad90 0x7fd88019c500 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:32.038 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:31.610+0000 7fd888200700 1 -- 172.21.15.86:0/3331611603 >> 172.21.15.86:0/3331611603 conn(0x7fd880101b30 msgr2=0x7fd8801026d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:43:32.038 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:31.610+0000 7fd888200700 1 -- 172.21.15.86:0/3331611603 shutdown_connections 2024-03-23T17:43:32.038 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:31.610+0000 7fd888200700 1 -- 172.21.15.86:0/3331611603 wait complete. 2024-03-23T17:43:32.038 INFO:teuthology.orchestra.run.smithi086.stdout:Fetching dashboard port number... 2024-03-23T17:43:32.925 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:32 smithi086 ceph-mon[38221]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:43:32.925 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:32 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:43:33.079 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 8443 2024-03-23T17:43:33.079 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:32.537+0000 7f8348e00700 1 Processor -- start 2024-03-23T17:43:33.080 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:32.537+0000 7f8348e00700 1 -- start start 2024-03-23T17:43:33.080 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:32.537+0000 7f8348e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83441065b0 0x7f83441069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:43:33.080 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:32.537+0000 7f8348e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8344106fa0 con 0x7f83441065b0 2024-03-23T17:43:33.080 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:32.537+0000 7f8341e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83441065b0 0x7f83441069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:43:33.080 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:32.537+0000 7f8341e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83441065b0 0x7f83441069d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38940/0 (socket says 172.21.15.86:38940) 2024-03-23T17:43:33.080 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:32.537+0000 7f8341e00700 1 -- 172.21.15.86:0/1391638550 learned_addr learned my addr 172.21.15.86:0/1391638550 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:43:33.080 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:32.537+0000 7f8341e00700 1 -- 172.21.15.86:0/1391638550 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83441077b0 con 0x7f83441065b0 2024-03-23T17:43:33.080 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:32.537+0000 7f8341e00700 1 --2- 172.21.15.86:0/1391638550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83441065b0 0x7f83441069d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f83340098a0 tx=0x7f8334009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8315de32153aa8 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:43:33.080 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:32.537+0000 7f8340a00700 1 -- 172.21.15.86:0/1391638550 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8334004160 con 0x7f83441065b0 2024-03-23T17:43:33.080 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:32.538+0000 7f8340a00700 1 -- 172.21.15.86:0/1391638550 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83340042c0 con 0x7f83441065b0 2024-03-23T17:43:33.080 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:32.538+0000 7f8340a00700 1 -- 172.21.15.86:0/1391638550 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f83340119b0 con 0x7f83441065b0 2024-03-23T17:43:33.080 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:32.538+0000 7f8348e00700 1 -- 172.21.15.86:0/1391638550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83441065b0 msgr2=0x7f83441069d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:43:33.080 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:32.538+0000 7f8348e00700 1 --2- 172.21.15.86:0/1391638550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83441065b0 0x7f83441069d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f83340098a0 tx=0x7f8334009bb0 comp rx=0 tx=0).stop 2024-03-23T17:43:33.080 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:32.538+0000 7f8348e00700 1 -- 172.21.15.86:0/1391638550 shutdown_connections 2024-03-23T17:43:33.081 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:32.538+0000 7f8348e00700 1 --2- 172.21.15.86:0/1391638550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83441065b0 0x7f83441069d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:33.081 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:32.538+0000 7f8348e00700 1 -- 172.21.15.86:0/1391638550 >> 172.21.15.86:0/1391638550 conn(0x7f8344101b90 msgr2=0x7f8344103fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:43:33.081 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:32.538+0000 7f8348e00700 1 -- 172.21.15.86:0/1391638550 shutdown_connections 2024-03-23T17:43:33.081 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:32.538+0000 7f8348e00700 1 -- 172.21.15.86:0/1391638550 wait complete. 2024-03-23T17:43:33.081 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:32.539+0000 7f8348e00700 1 Processor -- start 2024-03-23T17:43:33.081 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:32.539+0000 7f8348e00700 1 -- start start 2024-03-23T17:43:33.081 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:32.540+0000 7f8348e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83441065b0 0x7f834407aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:43:33.081 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:32.540+0000 7f8348e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f834407af70 con 0x7f83441065b0 2024-03-23T17:43:33.081 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:32.540+0000 7f8341e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83441065b0 0x7f834407aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:43:33.082 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:32.540+0000 7f8341e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83441065b0 0x7f834407aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38948/0 (socket says 172.21.15.86:38948) 2024-03-23T17:43:33.082 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:32.540+0000 7f8341e00700 1 -- 172.21.15.86:0/1209405889 learned_addr learned my addr 172.21.15.86:0/1209405889 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:43:33.082 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:32.540+0000 7f8341e00700 1 -- 172.21.15.86:0/1209405889 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8334009550 con 0x7f83441065b0 2024-03-23T17:43:33.082 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:32.540+0000 7f8341e00700 1 --2- 172.21.15.86:0/1209405889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83441065b0 0x7f834407aa30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8334003e60 tx=0x7f8334011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:43:33.082 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:32.541+0000 7f833aa00700 1 -- 172.21.15.86:0/1209405889 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8334003a60 con 0x7f83441065b0 2024-03-23T17:43:33.082 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:32.541+0000 7f833aa00700 1 -- 172.21.15.86:0/1209405889 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8334003bc0 con 0x7f83441065b0 2024-03-23T17:43:33.082 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:32.541+0000 7f833aa00700 1 -- 172.21.15.86:0/1209405889 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f83340196c0 con 0x7f83441065b0 2024-03-23T17:43:33.082 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:32.541+0000 7f8348e00700 1 -- 172.21.15.86:0/1209405889 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83440791b0 con 0x7f83441065b0 2024-03-23T17:43:33.082 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:32.541+0000 7f8348e00700 1 -- 172.21.15.86:0/1209405889 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8344079650 con 0x7f83441065b0 2024-03-23T17:43:33.082 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:32.542+0000 7f833aa00700 1 -- 172.21.15.86:0/1209405889 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 12) v1 ==== 45546+0+0 (secure 0 0 0) 0x7f833401a430 con 0x7f83441065b0 2024-03-23T17:43:33.082 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:32.542+0000 7f8348e00700 1 -- 172.21.15.86:0/1209405889 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f834410a9e0 con 0x7f83441065b0 2024-03-23T17:43:33.082 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:32.542+0000 7f833aa00700 1 --2- 172.21.15.86:0/1209405889 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f8328038780 0x7f832803ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:43:33.082 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:32.542+0000 7f833aa00700 1 -- 172.21.15.86:0/1209405889 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f833404cad0 con 0x7f83441065b0 2024-03-23T17:43:33.082 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:32.542+0000 7f8341400700 1 --2- 172.21.15.86:0/1209405889 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f8328038780 0x7f832803ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:43:33.083 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:32.543+0000 7f8341400700 1 --2- 172.21.15.86:0/1209405889 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f8328038780 0x7f832803ac40 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8330006fd0 tx=0x7f8330006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:43:33.083 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:32.547+0000 7f833aa00700 1 -- 172.21.15.86:0/1209405889 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f8334010380 con 0x7f83441065b0 2024-03-23T17:43:33.083 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:32.748+0000 7f8348e00700 1 -- 172.21.15.86:0/1209405889 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f834410ac20 con 0x7f83441065b0 2024-03-23T17:43:33.083 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:32.749+0000 7f833aa00700 1 -- 172.21.15.86:0/1209405889 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f8334010380 con 0x7f83441065b0 2024-03-23T17:43:33.083 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:32.751+0000 7f8348e00700 1 -- 172.21.15.86:0/1209405889 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f8328038780 msgr2=0x7f832803ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:43:33.083 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:32.751+0000 7f8348e00700 1 --2- 172.21.15.86:0/1209405889 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f8328038780 0x7f832803ac40 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8330006fd0 tx=0x7f8330006e40 comp rx=0 tx=0).stop 2024-03-23T17:43:33.083 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:32.751+0000 7f8348e00700 1 -- 172.21.15.86:0/1209405889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83441065b0 msgr2=0x7f834407aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:43:33.083 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:32.751+0000 7f8348e00700 1 --2- 172.21.15.86:0/1209405889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83441065b0 0x7f834407aa30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8334003e60 tx=0x7f8334011a30 comp rx=0 tx=0).stop 2024-03-23T17:43:33.083 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:32.752+0000 7f8348e00700 1 -- 172.21.15.86:0/1209405889 shutdown_connections 2024-03-23T17:43:33.083 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:32.752+0000 7f8348e00700 1 --2- 172.21.15.86:0/1209405889 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f8328038780 0x7f832803ac40 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:33.083 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:32.752+0000 7f8348e00700 1 --2- 172.21.15.86:0/1209405889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83441065b0 0x7f834407aa30 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:33.083 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:32.752+0000 7f8348e00700 1 -- 172.21.15.86:0/1209405889 >> 172.21.15.86:0/1209405889 conn(0x7f8344101b90 msgr2=0x7f83441026f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:43:33.083 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:32.752+0000 7f8348e00700 1 -- 172.21.15.86:0/1209405889 shutdown_connections 2024-03-23T17:43:33.084 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:32.752+0000 7f8348e00700 1 -- 172.21.15.86:0/1209405889 wait complete. 2024-03-23T17:43:33.095 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-23T17:43:33.095 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout disabled 2024-03-23T17:43:33.110 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-23T17:43:33.110 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout inactive 2024-03-23T17:43:33.110 INFO:teuthology.orchestra.run.smithi086.stdout:firewalld.service is not enabled 2024-03-23T17:43:33.110 INFO:teuthology.orchestra.run.smithi086.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-03-23T17:43:33.113 INFO:teuthology.orchestra.run.smithi086.stdout:Ceph Dashboard is now available at: 2024-03-23T17:43:33.113 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:43:33.113 INFO:teuthology.orchestra.run.smithi086.stdout: URL: https://smithi086.front.sepia.ceph.com:8443/ 2024-03-23T17:43:33.113 INFO:teuthology.orchestra.run.smithi086.stdout: User: admin 2024-03-23T17:43:33.113 INFO:teuthology.orchestra.run.smithi086.stdout: Password: sdibi9ljqy 2024-03-23T17:43:33.113 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:43:33.113 INFO:teuthology.orchestra.run.smithi086.stdout:Saving cluster configuration to /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/config directory 2024-03-23T17:43:33.114 INFO:teuthology.orchestra.run.smithi086.stdout:Enabling autotune for osd_memory_target 2024-03-23T17:43:33.942 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:33 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/1209405889' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-03-23T17:43:34.104 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:33.546+0000 7fd2d9c00700 1 Processor -- start 2024-03-23T17:43:34.104 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:33.547+0000 7fd2d9c00700 1 -- start start 2024-03-23T17:43:34.104 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:33.547+0000 7fd2d9c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2d407ad70 0x7fd2d40791d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:43:34.104 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:33.547+0000 7fd2d9c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d40797a0 con 0x7fd2d407ad70 2024-03-23T17:43:34.104 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:33.547+0000 7fd2d2a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2d407ad70 0x7fd2d40791d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:43:34.104 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:33.547+0000 7fd2d2a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2d407ad70 0x7fd2d40791d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38958/0 (socket says 172.21.15.86:38958) 2024-03-23T17:43:34.104 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:33.548+0000 7fd2d2a00700 1 -- 172.21.15.86:0/2990352545 learned_addr learned my addr 172.21.15.86:0/2990352545 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:43:34.104 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:33.548+0000 7fd2d2a00700 1 -- 172.21.15.86:0/2990352545 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2d40798e0 con 0x7fd2d407ad70 2024-03-23T17:43:34.104 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:33.548+0000 7fd2d2a00700 1 --2- 172.21.15.86:0/2990352545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2d407ad70 0x7fd2d40791d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd2c40098a0 tx=0x7fd2c4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5efc8444acdb91d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:43:34.105 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:33.548+0000 7fd2d1600700 1 -- 172.21.15.86:0/2990352545 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd2c4004160 con 0x7fd2d407ad70 2024-03-23T17:43:34.105 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:33.548+0000 7fd2d1600700 1 -- 172.21.15.86:0/2990352545 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2c40042c0 con 0x7fd2d407ad70 2024-03-23T17:43:34.105 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:33.548+0000 7fd2d1600700 1 -- 172.21.15.86:0/2990352545 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd2c40119b0 con 0x7fd2d407ad70 2024-03-23T17:43:34.105 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:33.549+0000 7fd2d9c00700 1 -- 172.21.15.86:0/2990352545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2d407ad70 msgr2=0x7fd2d40791d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:43:34.105 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:33.549+0000 7fd2d9c00700 1 --2- 172.21.15.86:0/2990352545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2d407ad70 0x7fd2d40791d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd2c40098a0 tx=0x7fd2c4009bb0 comp rx=0 tx=0).stop 2024-03-23T17:43:34.105 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:33.549+0000 7fd2d9c00700 1 -- 172.21.15.86:0/2990352545 shutdown_connections 2024-03-23T17:43:34.105 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:33.549+0000 7fd2d9c00700 1 --2- 172.21.15.86:0/2990352545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2d407ad70 0x7fd2d40791d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:34.105 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:33.549+0000 7fd2d9c00700 1 -- 172.21.15.86:0/2990352545 >> 172.21.15.86:0/2990352545 conn(0x7fd2d4101b90 msgr2=0x7fd2d4103fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:43:34.105 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:33.549+0000 7fd2d9c00700 1 -- 172.21.15.86:0/2990352545 shutdown_connections 2024-03-23T17:43:34.105 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:33.549+0000 7fd2d9c00700 1 -- 172.21.15.86:0/2990352545 wait complete. 2024-03-23T17:43:34.105 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:33.550+0000 7fd2d9c00700 1 Processor -- start 2024-03-23T17:43:34.105 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:33.550+0000 7fd2d9c00700 1 -- start start 2024-03-23T17:43:34.105 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:33.550+0000 7fd2d9c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2d407ad70 0x7fd2d419c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:43:34.105 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:33.550+0000 7fd2d9c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d419ca60 con 0x7fd2d407ad70 2024-03-23T17:43:34.105 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:33.551+0000 7fd2d2a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2d407ad70 0x7fd2d419c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:43:34.106 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:33.551+0000 7fd2d2a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2d407ad70 0x7fd2d419c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38968/0 (socket says 172.21.15.86:38968) 2024-03-23T17:43:34.106 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:33.551+0000 7fd2d2a00700 1 -- 172.21.15.86:0/3314145281 learned_addr learned my addr 172.21.15.86:0/3314145281 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:43:34.106 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:33.551+0000 7fd2d2a00700 1 -- 172.21.15.86:0/3314145281 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2c4009550 con 0x7fd2d407ad70 2024-03-23T17:43:34.106 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:33.551+0000 7fd2d2a00700 1 --2- 172.21.15.86:0/3314145281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2d407ad70 0x7fd2d419c520 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd2c4011670 tx=0x7fd2c4011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:43:34.106 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:33.551+0000 7fd2cb400700 1 -- 172.21.15.86:0/3314145281 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd2c4003b20 con 0x7fd2d407ad70 2024-03-23T17:43:34.106 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:33.551+0000 7fd2cb400700 1 -- 172.21.15.86:0/3314145281 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2c4003c80 con 0x7fd2d407ad70 2024-03-23T17:43:34.106 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:33.551+0000 7fd2d9c00700 1 -- 172.21.15.86:0/3314145281 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2d419cc60 con 0x7fd2d407ad70 2024-03-23T17:43:34.106 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:33.551+0000 7fd2cb400700 1 -- 172.21.15.86:0/3314145281 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd2c4019770 con 0x7fd2d407ad70 2024-03-23T17:43:34.106 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:33.552+0000 7fd2d9c00700 1 -- 172.21.15.86:0/3314145281 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2d419d100 con 0x7fd2d407ad70 2024-03-23T17:43:34.106 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:33.553+0000 7fd2cb400700 1 -- 172.21.15.86:0/3314145281 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 12) v1 ==== 45546+0+0 (secure 0 0 0) 0x7fd2c401ab60 con 0x7fd2d407ad70 2024-03-23T17:43:34.106 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:33.553+0000 7fd2d9c00700 1 -- 172.21.15.86:0/3314145281 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2d410c9b0 con 0x7fd2d407ad70 2024-03-23T17:43:34.106 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:33.553+0000 7fd2cb400700 1 --2- 172.21.15.86:0/3314145281 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fd2b8038750 0x7fd2b803ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:43:34.106 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:33.553+0000 7fd2cb400700 1 -- 172.21.15.86:0/3314145281 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd2c404cae0 con 0x7fd2d407ad70 2024-03-23T17:43:34.107 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:33.553+0000 7fd2d2000700 1 --2- 172.21.15.86:0/3314145281 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fd2b8038750 0x7fd2b803ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:43:34.107 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:33.554+0000 7fd2d2000700 1 --2- 172.21.15.86:0/3314145281 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fd2b8038750 0x7fd2b803ac10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd2c0006fd0 tx=0x7fd2c0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:43:34.107 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:33.558+0000 7fd2cb400700 1 -- 172.21.15.86:0/3314145281 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fd2c401a430 con 0x7fd2d407ad70 2024-03-23T17:43:34.107 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:33.760+0000 7fd2d9c00700 1 -- 172.21.15.86:0/3314145281 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7fd2d410cbf0 con 0x7fd2d407ad70 2024-03-23T17:43:34.107 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:33.761+0000 7fd2cb400700 1 -- 172.21.15.86:0/3314145281 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7fd2c402db00 con 0x7fd2d407ad70 2024-03-23T17:43:34.107 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:33.763+0000 7fd2d9c00700 1 -- 172.21.15.86:0/3314145281 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fd2b8038750 msgr2=0x7fd2b803ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:43:34.107 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:33.763+0000 7fd2d9c00700 1 --2- 172.21.15.86:0/3314145281 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fd2b8038750 0x7fd2b803ac10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd2c0006fd0 tx=0x7fd2c0006e40 comp rx=0 tx=0).stop 2024-03-23T17:43:34.107 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:33.763+0000 7fd2d9c00700 1 -- 172.21.15.86:0/3314145281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2d407ad70 msgr2=0x7fd2d419c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:43:34.107 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:33.763+0000 7fd2d9c00700 1 --2- 172.21.15.86:0/3314145281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2d407ad70 0x7fd2d419c520 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd2c4011670 tx=0x7fd2c4011a30 comp rx=0 tx=0).stop 2024-03-23T17:43:34.107 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:33.763+0000 7fd2d9c00700 1 -- 172.21.15.86:0/3314145281 shutdown_connections 2024-03-23T17:43:34.107 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:33.763+0000 7fd2d9c00700 1 --2- 172.21.15.86:0/3314145281 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fd2b8038750 0x7fd2b803ac10 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:34.107 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:33.763+0000 7fd2d9c00700 1 --2- 172.21.15.86:0/3314145281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2d407ad70 0x7fd2d419c520 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:34.107 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:33.763+0000 7fd2d9c00700 1 -- 172.21.15.86:0/3314145281 >> 172.21.15.86:0/3314145281 conn(0x7fd2d4101b90 msgr2=0x7fd2d4102700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:43:34.107 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:33.763+0000 7fd2d9c00700 1 -- 172.21.15.86:0/3314145281 shutdown_connections 2024-03-23T17:43:34.107 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:33.763+0000 7fd2d9c00700 1 -- 172.21.15.86:0/3314145281 wait complete. 2024-03-23T17:43:35.213 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:34.554+0000 7fa345c00700 1 Processor -- start 2024-03-23T17:43:35.213 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:34.554+0000 7fa345c00700 1 -- start start 2024-03-23T17:43:35.213 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:34.554+0000 7fa345c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa34007ad90 0x7fa3400791f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:43:35.213 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:34.554+0000 7fa345c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3400797c0 con 0x7fa34007ad90 2024-03-23T17:43:35.214 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:34.555+0000 7fa33ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa34007ad90 0x7fa3400791f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:43:35.214 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:34.555+0000 7fa33ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa34007ad90 0x7fa3400791f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38974/0 (socket says 172.21.15.86:38974) 2024-03-23T17:43:35.214 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:34.555+0000 7fa33ea00700 1 -- 172.21.15.86:0/1407291691 learned_addr learned my addr 172.21.15.86:0/1407291691 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:43:35.214 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:34.555+0000 7fa33ea00700 1 -- 172.21.15.86:0/1407291691 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa340079900 con 0x7fa34007ad90 2024-03-23T17:43:35.214 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:34.555+0000 7fa33ea00700 1 --2- 172.21.15.86:0/1407291691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa34007ad90 0x7fa3400791f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa3300098a0 tx=0x7fa330009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=140ca146c22e49be server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:43:35.214 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:34.556+0000 7fa33d600700 1 -- 172.21.15.86:0/1407291691 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa330004160 con 0x7fa34007ad90 2024-03-23T17:43:35.214 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:34.556+0000 7fa33d600700 1 -- 172.21.15.86:0/1407291691 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa3300042c0 con 0x7fa34007ad90 2024-03-23T17:43:35.214 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:34.556+0000 7fa33d600700 1 -- 172.21.15.86:0/1407291691 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa3300119b0 con 0x7fa34007ad90 2024-03-23T17:43:35.214 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:34.556+0000 7fa345c00700 1 -- 172.21.15.86:0/1407291691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa34007ad90 msgr2=0x7fa3400791f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:43:35.214 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:34.556+0000 7fa345c00700 1 --2- 172.21.15.86:0/1407291691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa34007ad90 0x7fa3400791f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa3300098a0 tx=0x7fa330009bb0 comp rx=0 tx=0).stop 2024-03-23T17:43:35.214 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:34.556+0000 7fa345c00700 1 -- 172.21.15.86:0/1407291691 shutdown_connections 2024-03-23T17:43:35.214 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:34.556+0000 7fa345c00700 1 --2- 172.21.15.86:0/1407291691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa34007ad90 0x7fa3400791f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:35.214 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:34.556+0000 7fa345c00700 1 -- 172.21.15.86:0/1407291691 >> 172.21.15.86:0/1407291691 conn(0x7fa340101b90 msgr2=0x7fa340103fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:43:35.215 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:34.556+0000 7fa345c00700 1 -- 172.21.15.86:0/1407291691 shutdown_connections 2024-03-23T17:43:35.215 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:34.557+0000 7fa345c00700 1 -- 172.21.15.86:0/1407291691 wait complete. 2024-03-23T17:43:35.215 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:34.557+0000 7fa345c00700 1 Processor -- start 2024-03-23T17:43:35.215 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:34.558+0000 7fa345c00700 1 -- start start 2024-03-23T17:43:35.215 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:34.558+0000 7fa345c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa34007ad90 0x7fa34019c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:43:35.215 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:34.558+0000 7fa345c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa34019ca30 con 0x7fa34007ad90 2024-03-23T17:43:35.215 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:34.558+0000 7fa33ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa34007ad90 0x7fa34019c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:43:35.215 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:34.558+0000 7fa33ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa34007ad90 0x7fa34019c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38986/0 (socket says 172.21.15.86:38986) 2024-03-23T17:43:35.215 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:34.558+0000 7fa33ea00700 1 -- 172.21.15.86:0/47188027 learned_addr learned my addr 172.21.15.86:0/47188027 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:43:35.215 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:34.558+0000 7fa33ea00700 1 -- 172.21.15.86:0/47188027 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa330009550 con 0x7fa34007ad90 2024-03-23T17:43:35.215 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:34.559+0000 7fa33ea00700 1 --2- 172.21.15.86:0/47188027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa34007ad90 0x7fa34019c4f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa330006af0 tx=0x7fa330011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:43:35.215 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:34.559+0000 7fa337400700 1 -- 172.21.15.86:0/47188027 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa330003a40 con 0x7fa34007ad90 2024-03-23T17:43:35.215 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:34.559+0000 7fa337400700 1 -- 172.21.15.86:0/47188027 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa330003ba0 con 0x7fa34007ad90 2024-03-23T17:43:35.215 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:34.559+0000 7fa337400700 1 -- 172.21.15.86:0/47188027 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa3300196c0 con 0x7fa34007ad90 2024-03-23T17:43:35.216 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:34.559+0000 7fa345c00700 1 -- 172.21.15.86:0/47188027 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa34019cc30 con 0x7fa34007ad90 2024-03-23T17:43:35.216 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:34.559+0000 7fa345c00700 1 -- 172.21.15.86:0/47188027 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa34019d0d0 con 0x7fa34007ad90 2024-03-23T17:43:35.216 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:34.560+0000 7fa337400700 1 -- 172.21.15.86:0/47188027 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 12) v1 ==== 45546+0+0 (secure 0 0 0) 0x7fa33001aa80 con 0x7fa34007ad90 2024-03-23T17:43:35.216 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:34.560+0000 7fa345c00700 1 -- 172.21.15.86:0/47188027 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa34010c9f0 con 0x7fa34007ad90 2024-03-23T17:43:35.216 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:34.560+0000 7fa337400700 1 --2- 172.21.15.86:0/47188027 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fa324038750 0x7fa32403ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:43:35.216 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:34.560+0000 7fa337400700 1 -- 172.21.15.86:0/47188027 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa33004cbf0 con 0x7fa34007ad90 2024-03-23T17:43:35.216 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:34.561+0000 7fa33e000700 1 --2- 172.21.15.86:0/47188027 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fa324038750 0x7fa32403ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:43:35.216 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:34.561+0000 7fa33e000700 1 --2- 172.21.15.86:0/47188027 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fa324038750 0x7fa32403ac10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa32c006fd0 tx=0x7fa32c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:43:35.216 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:34.565+0000 7fa337400700 1 -- 172.21.15.86:0/47188027 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa33001a430 con 0x7fa34007ad90 2024-03-23T17:43:35.216 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:34.610+0000 7fa337400700 1 -- 172.21.15.86:0/47188027 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mgrmap(e 13) v1 ==== 45589+0+0 (secure 0 0 0) 0x7fa33002c3e0 con 0x7fa34007ad90 2024-03-23T17:43:35.216 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:34.873+0000 7fa345c00700 1 -- 172.21.15.86:0/47188027 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7fa34010cc30 con 0x7fa34007ad90 2024-03-23T17:43:35.216 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:34.875+0000 7fa337400700 1 -- 172.21.15.86:0/47188027 <== mon.0 v2:172.21.15.86:3300/0 8 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7fa33001a430 con 0x7fa34007ad90 2024-03-23T17:43:35.216 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:34.878+0000 7fa345c00700 1 -- 172.21.15.86:0/47188027 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fa324038750 msgr2=0x7fa32403ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:43:35.216 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:34.878+0000 7fa345c00700 1 --2- 172.21.15.86:0/47188027 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fa324038750 0x7fa32403ac10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa32c006fd0 tx=0x7fa32c006e40 comp rx=0 tx=0).stop 2024-03-23T17:43:35.217 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:34.878+0000 7fa345c00700 1 -- 172.21.15.86:0/47188027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa34007ad90 msgr2=0x7fa34019c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:43:35.217 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:34.878+0000 7fa345c00700 1 --2- 172.21.15.86:0/47188027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa34007ad90 0x7fa34019c4f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa330006af0 tx=0x7fa330011a10 comp rx=0 tx=0).stop 2024-03-23T17:43:35.217 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:34.878+0000 7fa345c00700 1 -- 172.21.15.86:0/47188027 shutdown_connections 2024-03-23T17:43:35.217 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:34.878+0000 7fa345c00700 1 --2- 172.21.15.86:0/47188027 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fa324038750 0x7fa32403ac10 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:35.217 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:34.878+0000 7fa345c00700 1 --2- 172.21.15.86:0/47188027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa34007ad90 0x7fa34019c4f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:35.217 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:34.878+0000 7fa345c00700 1 -- 172.21.15.86:0/47188027 >> 172.21.15.86:0/47188027 conn(0x7fa340101b90 msgr2=0x7fa3401026b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:43:35.217 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:34.878+0000 7fa345c00700 1 -- 172.21.15.86:0/47188027 shutdown_connections 2024-03-23T17:43:35.217 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-03-23T17:43:34.878+0000 7fa345c00700 1 -- 172.21.15.86:0/47188027 wait complete. 2024-03-23T17:43:35.217 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-03-23T17:43:35.217 INFO:teuthology.orchestra.run.smithi086.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-03-23T17:43:35.217 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:43:35.217 INFO:teuthology.orchestra.run.smithi086.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-03-23T17:43:35.217 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:43:35.217 INFO:teuthology.orchestra.run.smithi086.stdout:Or, if you are only running a single cluster on this host: 2024-03-23T17:43:35.217 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:43:35.218 INFO:teuthology.orchestra.run.smithi086.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-03-23T17:43:35.218 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:43:35.218 INFO:teuthology.orchestra.run.smithi086.stdout:Please consider enabling telemetry to help improve Ceph: 2024-03-23T17:43:35.218 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:43:35.218 INFO:teuthology.orchestra.run.smithi086.stdout: ceph telemetry on 2024-03-23T17:43:35.218 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:43:35.218 INFO:teuthology.orchestra.run.smithi086.stdout:For more information see: 2024-03-23T17:43:35.218 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:43:35.218 INFO:teuthology.orchestra.run.smithi086.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-03-23T17:43:35.218 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:43:35.218 INFO:teuthology.orchestra.run.smithi086.stdout:Bootstrap complete. 2024-03-23T17:43:35.268 INFO:tasks.cephadm:Fetching config... 2024-03-23T17:43:35.269 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-23T17:43:35.269 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-03-23T17:43:35.313 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-03-23T17:43:35.313 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-23T17:43:35.313 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-03-23T17:43:35.396 INFO:tasks.cephadm:Fetching mon keyring... 2024-03-23T17:43:35.396 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-23T17:43:35.396 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.a/keyring of=/dev/stdout 2024-03-23T17:43:35.521 INFO:tasks.cephadm:Fetching pub ssh key... 2024-03-23T17:43:35.521 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-23T17:43:35.521 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-03-23T17:43:35.605 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-03-23T17:43:35.605 DEBUG:teuthology.orchestra.run.smithi086:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDYwXYO+EWZJ5hYLn4SJSleOGOPqplSRytD9ctra/UPcQx4z6u4uWRI2RYuIvPwdv+b9wd9q/CcS8nXbYnf8VZ/oDAcDm8F3yTCiJ56j/sg6qmkA1LPw0yohSRHVnfRW/U+x25zLr0eFoSjUZ/+7N2EV3pQLjIJTBPwCdx3A+GO1+9YapjmyA8WmnxhWxUpWiWQc7Dr/n3vTGOvHKfNXeYkwgmGPSz3Ju2k7AqhAsaa/R2IqgkoFfBgC+0v4jvpCJFOUQjaABaMzTN4/TNz6kR0VYYQlReooODjZ1PQXuGOzsywtAD8A9/IS3dvA6t23r1fsZqHwF8JAiwTv9aH0LueNZVYRrJVEOHlx6SExQ4lkHTwEBdwko2jUjV2gZzfbO/Yf7u9mRVhrYfpuPi5hb/D6CW3l381rPUHpqDJt7qZ2pFlNvgzX+lCDGpSNsfZ8RBY6p6P7SkbsLtl2rk7tAPjuqKAM1lOxG/SaspjRTcWEd4h+Tu2TcoQk+oTCQoleGE= ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-23T17:43:35.735 INFO:teuthology.orchestra.run.smithi086.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDYwXYO+EWZJ5hYLn4SJSleOGOPqplSRytD9ctra/UPcQx4z6u4uWRI2RYuIvPwdv+b9wd9q/CcS8nXbYnf8VZ/oDAcDm8F3yTCiJ56j/sg6qmkA1LPw0yohSRHVnfRW/U+x25zLr0eFoSjUZ/+7N2EV3pQLjIJTBPwCdx3A+GO1+9YapjmyA8WmnxhWxUpWiWQc7Dr/n3vTGOvHKfNXeYkwgmGPSz3Ju2k7AqhAsaa/R2IqgkoFfBgC+0v4jvpCJFOUQjaABaMzTN4/TNz6kR0VYYQlReooODjZ1PQXuGOzsywtAD8A9/IS3dvA6t23r1fsZqHwF8JAiwTv9aH0LueNZVYRrJVEOHlx6SExQ4lkHTwEBdwko2jUjV2gZzfbO/Yf7u9mRVhrYfpuPi5hb/D6CW3l381rPUHpqDJt7qZ2pFlNvgzX+lCDGpSNsfZ8RBY6p6P7SkbsLtl2rk7tAPjuqKAM1lOxG/SaspjRTcWEd4h+Tu2TcoQk+oTCQoleGE= ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715 2024-03-23T17:43:35.755 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:35 smithi086 ceph-mon[38221]: mgrmap e13: x(active, since 6s) 2024-03-23T17:43:35.755 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:35 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/47188027' entity='client.admin' 2024-03-23T17:43:35.763 DEBUG:teuthology.orchestra.run.smithi092:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDYwXYO+EWZJ5hYLn4SJSleOGOPqplSRytD9ctra/UPcQx4z6u4uWRI2RYuIvPwdv+b9wd9q/CcS8nXbYnf8VZ/oDAcDm8F3yTCiJ56j/sg6qmkA1LPw0yohSRHVnfRW/U+x25zLr0eFoSjUZ/+7N2EV3pQLjIJTBPwCdx3A+GO1+9YapjmyA8WmnxhWxUpWiWQc7Dr/n3vTGOvHKfNXeYkwgmGPSz3Ju2k7AqhAsaa/R2IqgkoFfBgC+0v4jvpCJFOUQjaABaMzTN4/TNz6kR0VYYQlReooODjZ1PQXuGOzsywtAD8A9/IS3dvA6t23r1fsZqHwF8JAiwTv9aH0LueNZVYRrJVEOHlx6SExQ4lkHTwEBdwko2jUjV2gZzfbO/Yf7u9mRVhrYfpuPi5hb/D6CW3l381rPUHpqDJt7qZ2pFlNvgzX+lCDGpSNsfZ8RBY6p6P7SkbsLtl2rk7tAPjuqKAM1lOxG/SaspjRTcWEd4h+Tu2TcoQk+oTCQoleGE= ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-23T17:43:35.812 INFO:teuthology.orchestra.run.smithi092.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDYwXYO+EWZJ5hYLn4SJSleOGOPqplSRytD9ctra/UPcQx4z6u4uWRI2RYuIvPwdv+b9wd9q/CcS8nXbYnf8VZ/oDAcDm8F3yTCiJ56j/sg6qmkA1LPw0yohSRHVnfRW/U+x25zLr0eFoSjUZ/+7N2EV3pQLjIJTBPwCdx3A+GO1+9YapjmyA8WmnxhWxUpWiWQc7Dr/n3vTGOvHKfNXeYkwgmGPSz3Ju2k7AqhAsaa/R2IqgkoFfBgC+0v4jvpCJFOUQjaABaMzTN4/TNz6kR0VYYQlReooODjZ1PQXuGOzsywtAD8A9/IS3dvA6t23r1fsZqHwF8JAiwTv9aH0LueNZVYRrJVEOHlx6SExQ4lkHTwEBdwko2jUjV2gZzfbO/Yf7u9mRVhrYfpuPi5hb/D6CW3l381rPUHpqDJt7qZ2pFlNvgzX+lCDGpSNsfZ8RBY6p6P7SkbsLtl2rk7tAPjuqKAM1lOxG/SaspjRTcWEd4h+Tu2TcoQk+oTCQoleGE= ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715 2024-03-23T17:43:35.826 DEBUG:teuthology.orchestra.run.smithi152:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDYwXYO+EWZJ5hYLn4SJSleOGOPqplSRytD9ctra/UPcQx4z6u4uWRI2RYuIvPwdv+b9wd9q/CcS8nXbYnf8VZ/oDAcDm8F3yTCiJ56j/sg6qmkA1LPw0yohSRHVnfRW/U+x25zLr0eFoSjUZ/+7N2EV3pQLjIJTBPwCdx3A+GO1+9YapjmyA8WmnxhWxUpWiWQc7Dr/n3vTGOvHKfNXeYkwgmGPSz3Ju2k7AqhAsaa/R2IqgkoFfBgC+0v4jvpCJFOUQjaABaMzTN4/TNz6kR0VYYQlReooODjZ1PQXuGOzsywtAD8A9/IS3dvA6t23r1fsZqHwF8JAiwTv9aH0LueNZVYRrJVEOHlx6SExQ4lkHTwEBdwko2jUjV2gZzfbO/Yf7u9mRVhrYfpuPi5hb/D6CW3l381rPUHpqDJt7qZ2pFlNvgzX+lCDGpSNsfZ8RBY6p6P7SkbsLtl2rk7tAPjuqKAM1lOxG/SaspjRTcWEd4h+Tu2TcoQk+oTCQoleGE= ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-23T17:43:35.873 INFO:teuthology.orchestra.run.smithi152.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDYwXYO+EWZJ5hYLn4SJSleOGOPqplSRytD9ctra/UPcQx4z6u4uWRI2RYuIvPwdv+b9wd9q/CcS8nXbYnf8VZ/oDAcDm8F3yTCiJ56j/sg6qmkA1LPw0yohSRHVnfRW/U+x25zLr0eFoSjUZ/+7N2EV3pQLjIJTBPwCdx3A+GO1+9YapjmyA8WmnxhWxUpWiWQc7Dr/n3vTGOvHKfNXeYkwgmGPSz3Ju2k7AqhAsaa/R2IqgkoFfBgC+0v4jvpCJFOUQjaABaMzTN4/TNz6kR0VYYQlReooODjZ1PQXuGOzsywtAD8A9/IS3dvA6t23r1fsZqHwF8JAiwTv9aH0LueNZVYRrJVEOHlx6SExQ4lkHTwEBdwko2jUjV2gZzfbO/Yf7u9mRVhrYfpuPi5hb/D6CW3l381rPUHpqDJt7qZ2pFlNvgzX+lCDGpSNsfZ8RBY6p6P7SkbsLtl2rk7tAPjuqKAM1lOxG/SaspjRTcWEd4h+Tu2TcoQk+oTCQoleGE= ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715 2024-03-23T17:43:35.886 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-03-23T17:43:36.236 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T17:43:37.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:37.962+0000 7f0367200700 1 -- 172.21.15.86:0/314896148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0360100730 msgr2=0x7f0360100b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:43:37.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:37.962+0000 7f0367200700 1 --2- 172.21.15.86:0/314896148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0360100730 0x7f0360100b50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0350009910 tx=0x7f0350009c20 comp rx=0 tx=0).stop 2024-03-23T17:43:37.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:37.962+0000 7f0367200700 1 -- 172.21.15.86:0/314896148 shutdown_connections 2024-03-23T17:43:37.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:37.962+0000 7f0367200700 1 --2- 172.21.15.86:0/314896148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0360100730 0x7f0360100b50 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:37.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:37.962+0000 7f0367200700 1 -- 172.21.15.86:0/314896148 >> 172.21.15.86:0/314896148 conn(0x7f03600fbcd0 msgr2=0x7f03600fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:43:37.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:37.963+0000 7f0367200700 1 -- 172.21.15.86:0/314896148 shutdown_connections 2024-03-23T17:43:37.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:37.963+0000 7f0367200700 1 -- 172.21.15.86:0/314896148 wait complete. 2024-03-23T17:43:37.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:37.964+0000 7f0367200700 1 Processor -- start 2024-03-23T17:43:37.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:37.964+0000 7f0367200700 1 -- start start 2024-03-23T17:43:37.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:37.964+0000 7f0367200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0360100730 0x7f0360197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:43:37.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:37.964+0000 7f0367200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0360198340 con 0x7f0360100730 2024-03-23T17:43:37.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:37.964+0000 7f035fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0360100730 0x7f0360197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:43:37.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:37.964+0000 7f035fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0360100730 0x7f0360197e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48578/0 (socket says 172.21.15.86:48578) 2024-03-23T17:43:37.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:37.964+0000 7f035fe00700 1 -- 172.21.15.86:0/71801204 learned_addr learned my addr 172.21.15.86:0/71801204 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:43:37.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:37.965+0000 7f035fe00700 1 -- 172.21.15.86:0/71801204 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03500095f0 con 0x7f0360100730 2024-03-23T17:43:37.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:37.965+0000 7f035fe00700 1 --2- 172.21.15.86:0/71801204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0360100730 0x7f0360197e00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0350005950 tx=0x7f03500057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:43:37.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:37.965+0000 7f035cc00700 1 -- 172.21.15.86:0/71801204 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f035001d070 con 0x7f0360100730 2024-03-23T17:43:37.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:37.965+0000 7f035cc00700 1 -- 172.21.15.86:0/71801204 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f035000bc10 con 0x7f0360100730 2024-03-23T17:43:37.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:37.965+0000 7f0367200700 1 -- 172.21.15.86:0/71801204 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0360198540 con 0x7f0360100730 2024-03-23T17:43:37.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:37.965+0000 7f035cc00700 1 -- 172.21.15.86:0/71801204 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0350003a10 con 0x7f0360100730 2024-03-23T17:43:37.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:37.965+0000 7f0367200700 1 -- 172.21.15.86:0/71801204 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03601989e0 con 0x7f0360100730 2024-03-23T17:43:37.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:37.966+0000 7f035cc00700 1 -- 172.21.15.86:0/71801204 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) v1 ==== 45589+0+0 (secure 0 0 0) 0x7f0350003cd0 con 0x7f0360100730 2024-03-23T17:43:37.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:37.967+0000 7f0367200700 1 -- 172.21.15.86:0/71801204 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0360191850 con 0x7f0360100730 2024-03-23T17:43:37.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:37.967+0000 7f035cc00700 1 --2- 172.21.15.86:0/71801204 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f03440387a0 0x7f034403ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:43:37.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:37.967+0000 7f035cc00700 1 -- 172.21.15.86:0/71801204 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f035004c5e0 con 0x7f0360100730 2024-03-23T17:43:37.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:37.967+0000 7f035f400700 1 --2- 172.21.15.86:0/71801204 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f03440387a0 0x7f034403ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:43:37.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:37.968+0000 7f035f400700 1 --2- 172.21.15.86:0/71801204 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f03440387a0 0x7f034403ac60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f034c006fd0 tx=0x7f034c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:43:37.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:37.972+0000 7f035cc00700 1 -- 172.21.15.86:0/71801204 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f035000ba70 con 0x7f0360100730 2024-03-23T17:43:38.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:38.177+0000 7f0367200700 1 -- 172.21.15.86:0/71801204 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f0360106730 con 0x7f0360100730 2024-03-23T17:43:38.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:38.181+0000 7f035cc00700 1 -- 172.21.15.86:0/71801204 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f0350026070 con 0x7f0360100730 2024-03-23T17:43:38.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:38.184+0000 7f0367200700 1 -- 172.21.15.86:0/71801204 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f03440387a0 msgr2=0x7f034403ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:43:38.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:38.184+0000 7f0367200700 1 --2- 172.21.15.86:0/71801204 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f03440387a0 0x7f034403ac60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f034c006fd0 tx=0x7f034c006e40 comp rx=0 tx=0).stop 2024-03-23T17:43:38.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:38.184+0000 7f0367200700 1 -- 172.21.15.86:0/71801204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0360100730 msgr2=0x7f0360197e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:43:38.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:38.184+0000 7f0367200700 1 --2- 172.21.15.86:0/71801204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0360100730 0x7f0360197e00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0350005950 tx=0x7f03500057c0 comp rx=0 tx=0).stop 2024-03-23T17:43:38.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:38.184+0000 7f0367200700 1 -- 172.21.15.86:0/71801204 shutdown_connections 2024-03-23T17:43:38.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:38.184+0000 7f0367200700 1 --2- 172.21.15.86:0/71801204 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f03440387a0 0x7f034403ac60 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:38.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:38.184+0000 7f0367200700 1 --2- 172.21.15.86:0/71801204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0360100730 0x7f0360197e00 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:38.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:38.184+0000 7f0367200700 1 -- 172.21.15.86:0/71801204 >> 172.21.15.86:0/71801204 conn(0x7f03600fbcd0 msgr2=0x7f03600fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:43:38.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:38.184+0000 7f0367200700 1 -- 172.21.15.86:0/71801204 shutdown_connections 2024-03-23T17:43:38.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:38.184+0000 7f0367200700 1 -- 172.21.15.86:0/71801204 wait complete. 2024-03-23T17:43:38.565 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-03-23T17:43:38.565 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-03-23T17:43:38.938 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T17:43:39.416 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:39 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/71801204' entity='client.admin' 2024-03-23T17:43:40.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:40.252+0000 7f1e2d400700 1 -- 172.21.15.86:0/1110787069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e28100720 msgr2=0x7f1e28100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:43:40.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:40.252+0000 7f1e2d400700 1 --2- 172.21.15.86:0/1110787069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e28100720 0x7f1e28100b40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1e18009910 tx=0x7f1e18009c20 comp rx=0 tx=0).stop 2024-03-23T17:43:40.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:40.252+0000 7f1e2d400700 1 -- 172.21.15.86:0/1110787069 shutdown_connections 2024-03-23T17:43:40.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:40.252+0000 7f1e2d400700 1 --2- 172.21.15.86:0/1110787069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e28100720 0x7f1e28100b40 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:40.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:40.252+0000 7f1e2d400700 1 -- 172.21.15.86:0/1110787069 >> 172.21.15.86:0/1110787069 conn(0x7f1e280fbcc0 msgr2=0x7f1e280fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:43:40.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:40.252+0000 7f1e2d400700 1 -- 172.21.15.86:0/1110787069 shutdown_connections 2024-03-23T17:43:40.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:40.252+0000 7f1e2d400700 1 -- 172.21.15.86:0/1110787069 wait complete. 2024-03-23T17:43:40.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:40.253+0000 7f1e2d400700 1 Processor -- start 2024-03-23T17:43:40.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:40.253+0000 7f1e2d400700 1 -- start start 2024-03-23T17:43:40.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:40.253+0000 7f1e2d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e28100720 0x7f1e28074a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:43:40.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:40.253+0000 7f1e2d400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e28074f60 con 0x7f1e28100720 2024-03-23T17:43:40.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:40.254+0000 7f1e26800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e28100720 0x7f1e28074a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:43:40.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:40.254+0000 7f1e26800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e28100720 0x7f1e28074a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48588/0 (socket says 172.21.15.86:48588) 2024-03-23T17:43:40.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:40.254+0000 7f1e26800700 1 -- 172.21.15.86:0/776044133 learned_addr learned my addr 172.21.15.86:0/776044133 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:43:40.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:40.254+0000 7f1e26800700 1 -- 172.21.15.86:0/776044133 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e180095f0 con 0x7f1e28100720 2024-03-23T17:43:40.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:40.254+0000 7f1e26800700 1 --2- 172.21.15.86:0/776044133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e28100720 0x7f1e28074a20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1e18006010 tx=0x7f1e18005af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:43:40.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:40.254+0000 7f1e1f400700 1 -- 172.21.15.86:0/776044133 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1e1801d070 con 0x7f1e28100720 2024-03-23T17:43:40.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:40.254+0000 7f1e1f400700 1 -- 172.21.15.86:0/776044133 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e1800bc10 con 0x7f1e28100720 2024-03-23T17:43:40.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:40.254+0000 7f1e2d400700 1 -- 172.21.15.86:0/776044133 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e280731a0 con 0x7f1e28100720 2024-03-23T17:43:40.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:40.254+0000 7f1e1f400700 1 -- 172.21.15.86:0/776044133 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1e18003930 con 0x7f1e28100720 2024-03-23T17:43:40.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:40.255+0000 7f1e2d400700 1 -- 172.21.15.86:0/776044133 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e28073640 con 0x7f1e28100720 2024-03-23T17:43:40.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:40.255+0000 7f1e1f400700 1 -- 172.21.15.86:0/776044133 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) v1 ==== 45589+0+0 (secure 0 0 0) 0x7f1e18003ba0 con 0x7f1e28100720 2024-03-23T17:43:40.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:40.255+0000 7f1e2d400700 1 -- 172.21.15.86:0/776044133 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e28106d70 con 0x7f1e28100720 2024-03-23T17:43:40.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:40.256+0000 7f1e1f400700 1 --2- 172.21.15.86:0/776044133 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f1e0c0387f0 0x7f1e0c03acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:43:40.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:40.256+0000 7f1e1f400700 1 -- 172.21.15.86:0/776044133 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f1e1804c580 con 0x7f1e28100720 2024-03-23T17:43:40.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:40.256+0000 7f1e1fe00700 1 --2- 172.21.15.86:0/776044133 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f1e0c0387f0 0x7f1e0c03acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:43:40.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:40.257+0000 7f1e1fe00700 1 --2- 172.21.15.86:0/776044133 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f1e0c0387f0 0x7f1e0c03acb0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1e10006fd0 tx=0x7f1e10006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:43:40.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:40.261+0000 7f1e1f400700 1 -- 172.21.15.86:0/776044133 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f1e1800ba70 con 0x7f1e28100720 2024-03-23T17:43:40.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:40.467+0000 7f1e2d400700 1 -- 172.21.15.86:0/776044133 --> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f1e28073880 con 0x7f1e0c0387f0 2024-03-23T17:43:40.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:40.471+0000 7f1e1f400700 1 -- 172.21.15.86:0/776044133 <== mgr.14152 v2:172.21.15.86:6800/2734784244 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f1e28073880 con 0x7f1e0c0387f0 2024-03-23T17:43:40.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:40.474+0000 7f1e2d400700 1 -- 172.21.15.86:0/776044133 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f1e0c0387f0 msgr2=0x7f1e0c03acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:43:40.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:40.474+0000 7f1e2d400700 1 --2- 172.21.15.86:0/776044133 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f1e0c0387f0 0x7f1e0c03acb0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1e10006fd0 tx=0x7f1e10006e40 comp rx=0 tx=0).stop 2024-03-23T17:43:40.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:40.474+0000 7f1e2d400700 1 -- 172.21.15.86:0/776044133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e28100720 msgr2=0x7f1e28074a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:43:40.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:40.474+0000 7f1e2d400700 1 --2- 172.21.15.86:0/776044133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e28100720 0x7f1e28074a20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1e18006010 tx=0x7f1e18005af0 comp rx=0 tx=0).stop 2024-03-23T17:43:40.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:40.474+0000 7f1e2d400700 1 -- 172.21.15.86:0/776044133 shutdown_connections 2024-03-23T17:43:40.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:40.474+0000 7f1e2d400700 1 --2- 172.21.15.86:0/776044133 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f1e0c0387f0 0x7f1e0c03acb0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:40.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:40.474+0000 7f1e2d400700 1 --2- 172.21.15.86:0/776044133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e28100720 0x7f1e28074a20 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:40.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:40.474+0000 7f1e2d400700 1 -- 172.21.15.86:0/776044133 >> 172.21.15.86:0/776044133 conn(0x7f1e280fbcc0 msgr2=0x7f1e280fc850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:43:40.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:40.474+0000 7f1e2d400700 1 -- 172.21.15.86:0/776044133 shutdown_connections 2024-03-23T17:43:40.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:40.474+0000 7f1e2d400700 1 -- 172.21.15.86:0/776044133 wait complete. 2024-03-23T17:43:40.813 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi092 2024-03-23T17:43:40.813 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-03-23T17:43:40.813 DEBUG:teuthology.orchestra.run.smithi092:> dd of=/etc/ceph/ceph.conf 2024-03-23T17:43:40.834 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-03-23T17:43:40.834 DEBUG:teuthology.orchestra.run.smithi092:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-23T17:43:40.894 INFO:tasks.cephadm:Adding host smithi092 to orchestrator... 2024-03-23T17:43:40.895 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph orch host add smithi092 2024-03-23T17:43:41.259 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T17:43:41.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:40 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:43:41.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:40 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:43:41.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:40 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi086", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:43:41.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:40 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:43:41.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:40 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:43:41.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:40 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:43:41.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:40 smithi086 ceph-mon[38221]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:43:41.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:40 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:43:41.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:40 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:43:41.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:40 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:43:41.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:40 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:43:41.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:40 smithi086 ceph-mon[38221]: Updating smithi086:/etc/ceph/ceph.conf 2024-03-23T17:43:42.220 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:41 smithi086 ceph-mon[38221]: Updating smithi086:/var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T17:43:42.220 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:41 smithi086 ceph-mon[38221]: Updating smithi086:/etc/ceph/ceph.client.admin.keyring 2024-03-23T17:43:42.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:42.452+0000 7f9aeb200700 1 -- 172.21.15.86:0/3489506834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ae40752e0 msgr2=0x7f9ae40756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:43:42.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:42.452+0000 7f9aeb200700 1 --2- 172.21.15.86:0/3489506834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ae40752e0 0x7f9ae40756c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4009910 tx=0x7f9ad4009c20 comp rx=0 tx=0).stop 2024-03-23T17:43:42.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:42.453+0000 7f9aeb200700 1 -- 172.21.15.86:0/3489506834 shutdown_connections 2024-03-23T17:43:42.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:42.453+0000 7f9aeb200700 1 --2- 172.21.15.86:0/3489506834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ae40752e0 0x7f9ae40756c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:42.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:42.453+0000 7f9aeb200700 1 -- 172.21.15.86:0/3489506834 >> 172.21.15.86:0/3489506834 conn(0x7f9ae40fe3d0 msgr2=0x7f9ae41007f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:43:42.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:42.453+0000 7f9aeb200700 1 -- 172.21.15.86:0/3489506834 shutdown_connections 2024-03-23T17:43:42.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:42.453+0000 7f9aeb200700 1 -- 172.21.15.86:0/3489506834 wait complete. 2024-03-23T17:43:42.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:42.454+0000 7f9aeb200700 1 Processor -- start 2024-03-23T17:43:42.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:42.454+0000 7f9aeb200700 1 -- start start 2024-03-23T17:43:42.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:42.454+0000 7f9aeb200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ae40752e0 0x7f9ae419c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:43:42.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:42.454+0000 7f9aeb200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae419cad0 con 0x7f9ae40752e0 2024-03-23T17:43:42.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:42.454+0000 7f9ae3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ae40752e0 0x7f9ae419c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:43:42.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:42.455+0000 7f9ae3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ae40752e0 0x7f9ae419c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48596/0 (socket says 172.21.15.86:48596) 2024-03-23T17:43:42.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:42.455+0000 7f9ae3e00700 1 -- 172.21.15.86:0/2780307972 learned_addr learned my addr 172.21.15.86:0/2780307972 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:43:42.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:42.455+0000 7f9ae3e00700 1 -- 172.21.15.86:0/2780307972 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ad40095f0 con 0x7f9ae40752e0 2024-03-23T17:43:42.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:42.455+0000 7f9ae3e00700 1 --2- 172.21.15.86:0/2780307972 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ae40752e0 0x7f9ae419c590 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4006010 tx=0x7f9ad4004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:43:42.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:42.455+0000 7f9ae0c00700 1 -- 172.21.15.86:0/2780307972 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9ad401d070 con 0x7f9ae40752e0 2024-03-23T17:43:42.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:42.455+0000 7f9ae0c00700 1 -- 172.21.15.86:0/2780307972 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ad400bc10 con 0x7f9ae40752e0 2024-03-23T17:43:42.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:42.455+0000 7f9aeb200700 1 -- 172.21.15.86:0/2780307972 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ae419ccd0 con 0x7f9ae40752e0 2024-03-23T17:43:42.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:42.455+0000 7f9ae0c00700 1 -- 172.21.15.86:0/2780307972 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9ad40038a0 con 0x7f9ae40752e0 2024-03-23T17:43:42.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:42.456+0000 7f9aeb200700 1 -- 172.21.15.86:0/2780307972 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ae419d0f0 con 0x7f9ae40752e0 2024-03-23T17:43:42.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:42.457+0000 7f9ae0c00700 1 -- 172.21.15.86:0/2780307972 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) v1 ==== 45589+0+0 (secure 0 0 0) 0x7f9ad4003a00 con 0x7f9ae40752e0 2024-03-23T17:43:42.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:42.457+0000 7f9aeb200700 1 -- 172.21.15.86:0/2780307972 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ae4076a20 con 0x7f9ae40752e0 2024-03-23T17:43:42.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:42.457+0000 7f9ae0c00700 1 --2- 172.21.15.86:0/2780307972 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f9ac80387f0 0x7f9ac803acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:43:42.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:42.457+0000 7f9ae0c00700 1 -- 172.21.15.86:0/2780307972 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f9ad404c380 con 0x7f9ae40752e0 2024-03-23T17:43:42.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:42.457+0000 7f9ae3400700 1 --2- 172.21.15.86:0/2780307972 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f9ac80387f0 0x7f9ac803acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:43:42.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:42.458+0000 7f9ae3400700 1 --2- 172.21.15.86:0/2780307972 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f9ac80387f0 0x7f9ac803acb0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9ad0006fd0 tx=0x7f9ad0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:43:42.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:42.462+0000 7f9ae0c00700 1 -- 172.21.15.86:0/2780307972 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f9ad4029770 con 0x7f9ae40752e0 2024-03-23T17:43:42.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:42.693+0000 7f9aeb200700 1 -- 172.21.15.86:0/2780307972 --> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi092", "target": ["mon-mgr", ""]}) v1 -- 0x7f9ae4111030 con 0x7f9ac80387f0 2024-03-23T17:43:44.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:43 smithi086 ceph-mon[38221]: Updating smithi086:/var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-23T17:43:44.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:43 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:43:44.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:43 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:43:44.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:43 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:43:44.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:43 smithi086 ceph-mon[38221]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi092", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:43:44.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:44.879+0000 7f9ae0c00700 1 -- 172.21.15.86:0/2780307972 <== mgr.14152 v2:172.21.15.86:6800/2734784244 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f9ae4111030 con 0x7f9ac80387f0 2024-03-23T17:43:44.879 INFO:teuthology.orchestra.run.smithi086.stdout:Added host 'smithi092' with addr '172.21.15.92' 2024-03-23T17:43:44.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:44.881+0000 7f9aeb200700 1 -- 172.21.15.86:0/2780307972 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f9ac80387f0 msgr2=0x7f9ac803acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:43:44.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:44.881+0000 7f9aeb200700 1 --2- 172.21.15.86:0/2780307972 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f9ac80387f0 0x7f9ac803acb0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9ad0006fd0 tx=0x7f9ad0006e40 comp rx=0 tx=0).stop 2024-03-23T17:43:44.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:44.881+0000 7f9aeb200700 1 -- 172.21.15.86:0/2780307972 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ae40752e0 msgr2=0x7f9ae419c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:43:44.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:44.881+0000 7f9aeb200700 1 --2- 172.21.15.86:0/2780307972 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ae40752e0 0x7f9ae419c590 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4006010 tx=0x7f9ad4004ff0 comp rx=0 tx=0).stop 2024-03-23T17:43:44.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:44.881+0000 7f9aeb200700 1 -- 172.21.15.86:0/2780307972 shutdown_connections 2024-03-23T17:43:44.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:44.881+0000 7f9aeb200700 1 --2- 172.21.15.86:0/2780307972 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f9ac80387f0 0x7f9ac803acb0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:44.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:44.881+0000 7f9aeb200700 1 --2- 172.21.15.86:0/2780307972 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ae40752e0 0x7f9ae419c590 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:44.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:44.881+0000 7f9aeb200700 1 -- 172.21.15.86:0/2780307972 >> 172.21.15.86:0/2780307972 conn(0x7f9ae40fe3d0 msgr2=0x7f9ae40ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:43:44.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:44.881+0000 7f9aeb200700 1 -- 172.21.15.86:0/2780307972 shutdown_connections 2024-03-23T17:43:44.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:44.881+0000 7f9aeb200700 1 -- 172.21.15.86:0/2780307972 wait complete. 2024-03-23T17:43:44.904 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:44 smithi086 ceph-mon[38221]: Deploying cephadm binary to smithi092 2024-03-23T17:43:45.256 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph orch host ls --format=json 2024-03-23T17:43:45.603 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T17:43:46.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:45 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:43:46.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:45 smithi086 ceph-mon[38221]: Added host smithi092 2024-03-23T17:43:46.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:45 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:43:46.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:45 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:43:46.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:45 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:43:46.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:46.730+0000 7f9329e00700 1 -- 172.21.15.86:0/944960841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93241066c0 msgr2=0x7f9324106aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:43:46.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:46.730+0000 7f9329e00700 1 --2- 172.21.15.86:0/944960841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93241066c0 0x7f9324106aa0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9314009910 tx=0x7f9314009c20 comp rx=0 tx=0).stop 2024-03-23T17:43:46.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:46.730+0000 7f9329e00700 1 -- 172.21.15.86:0/944960841 shutdown_connections 2024-03-23T17:43:46.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:46.730+0000 7f9329e00700 1 --2- 172.21.15.86:0/944960841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93241066c0 0x7f9324106aa0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:46.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:46.730+0000 7f9329e00700 1 -- 172.21.15.86:0/944960841 >> 172.21.15.86:0/944960841 conn(0x7f93240fc1b0 msgr2=0x7f93240fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:43:46.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:46.730+0000 7f9329e00700 1 -- 172.21.15.86:0/944960841 shutdown_connections 2024-03-23T17:43:46.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:46.730+0000 7f9329e00700 1 -- 172.21.15.86:0/944960841 wait complete. 2024-03-23T17:43:46.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:46.732+0000 7f9329e00700 1 Processor -- start 2024-03-23T17:43:46.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:46.732+0000 7f9329e00700 1 -- start start 2024-03-23T17:43:46.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:46.732+0000 7f9329e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93241066c0 0x7f9324195f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:43:46.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:46.732+0000 7f9329e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93241964a0 con 0x7f93241066c0 2024-03-23T17:43:46.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:46.733+0000 7f9322a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93241066c0 0x7f9324195f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:43:46.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:46.733+0000 7f9322a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93241066c0 0x7f9324195f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54744/0 (socket says 172.21.15.86:54744) 2024-03-23T17:43:46.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:46.733+0000 7f9322a00700 1 -- 172.21.15.86:0/2559505517 learned_addr learned my addr 172.21.15.86:0/2559505517 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:43:46.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:46.733+0000 7f9322a00700 1 -- 172.21.15.86:0/2559505517 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93140095f0 con 0x7f93241066c0 2024-03-23T17:43:46.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:46.733+0000 7f9322a00700 1 --2- 172.21.15.86:0/2559505517 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93241066c0 0x7f9324195f60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9314006010 tx=0x7f9314004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:43:46.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:46.733+0000 7f9328a00700 1 -- 172.21.15.86:0/2559505517 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f931401d070 con 0x7f93241066c0 2024-03-23T17:43:46.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:46.733+0000 7f9328a00700 1 -- 172.21.15.86:0/2559505517 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f931400bbb0 con 0x7f93241066c0 2024-03-23T17:43:46.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:46.733+0000 7f9329e00700 1 -- 172.21.15.86:0/2559505517 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9324196700 con 0x7f93241066c0 2024-03-23T17:43:46.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:46.733+0000 7f9328a00700 1 -- 172.21.15.86:0/2559505517 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9314003810 con 0x7f93241066c0 2024-03-23T17:43:46.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:46.734+0000 7f9329e00700 1 -- 172.21.15.86:0/2559505517 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9324196b80 con 0x7f93241066c0 2024-03-23T17:43:46.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:46.734+0000 7f9329e00700 1 -- 172.21.15.86:0/2559505517 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9324101e00 con 0x7f93241066c0 2024-03-23T17:43:46.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:46.735+0000 7f9328a00700 1 -- 172.21.15.86:0/2559505517 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) v1 ==== 45589+0+0 (secure 0 0 0) 0x7f931400bd20 con 0x7f93241066c0 2024-03-23T17:43:46.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:46.736+0000 7f9328a00700 1 --2- 172.21.15.86:0/2559505517 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f93080387a0 0x7f930803ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:43:46.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:46.736+0000 7f9328a00700 1 -- 172.21.15.86:0/2559505517 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f931404ca00 con 0x7f93241066c0 2024-03-23T17:43:46.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:46.736+0000 7f931be00700 1 --2- 172.21.15.86:0/2559505517 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f93080387a0 0x7f930803ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:43:46.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:46.736+0000 7f931be00700 1 --2- 172.21.15.86:0/2559505517 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f93080387a0 0x7f930803ac60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f930c006fd0 tx=0x7f930c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:43:46.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:46.740+0000 7f9328a00700 1 -- 172.21.15.86:0/2559505517 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f931400e3e0 con 0x7f93241066c0 2024-03-23T17:43:46.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:46.950+0000 7f9329e00700 1 -- 172.21.15.86:0/2559505517 --> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f932410a9c0 con 0x7f93080387a0 2024-03-23T17:43:46.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:46.951+0000 7f9328a00700 1 -- 172.21.15.86:0/2559505517 <== mgr.14152 v2:172.21.15.86:6800/2734784244 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+159 (secure 0 0 0) 0x7f932410a9c0 con 0x7f93080387a0 2024-03-23T17:43:46.952 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:43:46.952 INFO:teuthology.orchestra.run.smithi086.stdout:[{"addr": "172.21.15.86", "hostname": "smithi086", "labels": [], "status": ""}, {"addr": "172.21.15.92", "hostname": "smithi092", "labels": [], "status": ""}] 2024-03-23T17:43:46.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:46.954+0000 7f9329e00700 1 -- 172.21.15.86:0/2559505517 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f93080387a0 msgr2=0x7f930803ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:43:46.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:46.954+0000 7f9329e00700 1 --2- 172.21.15.86:0/2559505517 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f93080387a0 0x7f930803ac60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f930c006fd0 tx=0x7f930c006e40 comp rx=0 tx=0).stop 2024-03-23T17:43:46.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:46.954+0000 7f9329e00700 1 -- 172.21.15.86:0/2559505517 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93241066c0 msgr2=0x7f9324195f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:43:46.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:46.954+0000 7f9329e00700 1 --2- 172.21.15.86:0/2559505517 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93241066c0 0x7f9324195f60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9314006010 tx=0x7f9314004ff0 comp rx=0 tx=0).stop 2024-03-23T17:43:46.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:46.954+0000 7f9329e00700 1 -- 172.21.15.86:0/2559505517 shutdown_connections 2024-03-23T17:43:46.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:46.954+0000 7f9329e00700 1 --2- 172.21.15.86:0/2559505517 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f93080387a0 0x7f930803ac60 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:46.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:46.954+0000 7f9329e00700 1 --2- 172.21.15.86:0/2559505517 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93241066c0 0x7f9324195f60 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:46.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:46.954+0000 7f9329e00700 1 -- 172.21.15.86:0/2559505517 >> 172.21.15.86:0/2559505517 conn(0x7f93240fc1b0 msgr2=0x7f93241082f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:43:46.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:46.954+0000 7f9329e00700 1 -- 172.21.15.86:0/2559505517 shutdown_connections 2024-03-23T17:43:46.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:46.954+0000 7f9329e00700 1 -- 172.21.15.86:0/2559505517 wait complete. 2024-03-23T17:43:47.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:47 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:43:47.305 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi152 2024-03-23T17:43:47.306 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-03-23T17:43:47.306 DEBUG:teuthology.orchestra.run.smithi152:> dd of=/etc/ceph/ceph.conf 2024-03-23T17:43:47.326 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-03-23T17:43:47.327 DEBUG:teuthology.orchestra.run.smithi152:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-23T17:43:47.385 INFO:tasks.cephadm:Adding host smithi152 to orchestrator... 2024-03-23T17:43:47.386 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph orch host add smithi152 2024-03-23T17:43:47.727 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T17:43:48.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:48 smithi086 ceph-mon[38221]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T17:43:48.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:48.852+0000 7f8e9d800700 1 -- 172.21.15.86:0/2157264274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e981066c0 msgr2=0x7f8e98106aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:43:48.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:48.852+0000 7f8e9d800700 1 --2- 172.21.15.86:0/2157264274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e981066c0 0x7f8e98106aa0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8e88009910 tx=0x7f8e88009c20 comp rx=0 tx=0).stop 2024-03-23T17:43:48.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:48.852+0000 7f8e9d800700 1 -- 172.21.15.86:0/2157264274 shutdown_connections 2024-03-23T17:43:48.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:48.852+0000 7f8e9d800700 1 --2- 172.21.15.86:0/2157264274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e981066c0 0x7f8e98106aa0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:48.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:48.852+0000 7f8e9d800700 1 -- 172.21.15.86:0/2157264274 >> 172.21.15.86:0/2157264274 conn(0x7f8e980fc1b0 msgr2=0x7f8e980fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:43:48.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:48.852+0000 7f8e9d800700 1 -- 172.21.15.86:0/2157264274 shutdown_connections 2024-03-23T17:43:48.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:48.852+0000 7f8e9d800700 1 -- 172.21.15.86:0/2157264274 wait complete. 2024-03-23T17:43:48.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:48.853+0000 7f8e9d800700 1 Processor -- start 2024-03-23T17:43:48.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:48.853+0000 7f8e9d800700 1 -- start start 2024-03-23T17:43:48.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:48.854+0000 7f8e9d800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e981066c0 0x7f8e980730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:43:48.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:48.854+0000 7f8e9d800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e98074f70 con 0x7f8e981066c0 2024-03-23T17:43:48.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:48.854+0000 7f8e96a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e981066c0 0x7f8e980730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:43:48.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:48.854+0000 7f8e96a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e981066c0 0x7f8e980730b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54760/0 (socket says 172.21.15.86:54760) 2024-03-23T17:43:48.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:48.854+0000 7f8e96a00700 1 -- 172.21.15.86:0/1130364931 learned_addr learned my addr 172.21.15.86:0/1130364931 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:43:48.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:48.854+0000 7f8e96a00700 1 -- 172.21.15.86:0/1130364931 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e880095f0 con 0x7f8e981066c0 2024-03-23T17:43:48.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:48.854+0000 7f8e96a00700 1 --2- 172.21.15.86:0/1130364931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e981066c0 0x7f8e980730b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8e88006010 tx=0x7f8e88004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:43:48.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:48.855+0000 7f8e8f400700 1 -- 172.21.15.86:0/1130364931 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8e8801d070 con 0x7f8e981066c0 2024-03-23T17:43:48.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:48.855+0000 7f8e8f400700 1 -- 172.21.15.86:0/1130364931 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e8800bbb0 con 0x7f8e981066c0 2024-03-23T17:43:48.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:48.855+0000 7f8e8f400700 1 -- 172.21.15.86:0/1130364931 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8e88003810 con 0x7f8e981066c0 2024-03-23T17:43:48.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:48.855+0000 7f8e9d800700 1 -- 172.21.15.86:0/1130364931 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e980735f0 con 0x7f8e981066c0 2024-03-23T17:43:48.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:48.855+0000 7f8e9d800700 1 -- 172.21.15.86:0/1130364931 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e98073a70 con 0x7f8e981066c0 2024-03-23T17:43:48.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:48.856+0000 7f8e8f400700 1 -- 172.21.15.86:0/1130364931 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) v1 ==== 45589+0+0 (secure 0 0 0) 0x7f8e8800bd20 con 0x7f8e981066c0 2024-03-23T17:43:48.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:48.856+0000 7f8e9d800700 1 -- 172.21.15.86:0/1130364931 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e98101e00 con 0x7f8e981066c0 2024-03-23T17:43:48.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:48.856+0000 7f8e8f400700 1 --2- 172.21.15.86:0/1130364931 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f8e7c0387f0 0x7f8e7c03acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:43:48.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:48.856+0000 7f8e8f400700 1 -- 172.21.15.86:0/1130364931 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8e8804c410 con 0x7f8e981066c0 2024-03-23T17:43:48.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:48.856+0000 7f8e96000700 1 --2- 172.21.15.86:0/1130364931 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f8e7c0387f0 0x7f8e7c03acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:43:48.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:48.857+0000 7f8e96000700 1 --2- 172.21.15.86:0/1130364931 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f8e7c0387f0 0x7f8e7c03acb0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8e84006fd0 tx=0x7f8e84006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:43:48.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:48.861+0000 7f8e8f400700 1 -- 172.21.15.86:0/1130364931 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f8e88029360 con 0x7f8e981066c0 2024-03-23T17:43:49.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:49.073+0000 7f8e9d800700 1 -- 172.21.15.86:0/1130364931 --> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi152", "target": ["mon-mgr", ""]}) v1 -- 0x7f8e98073cb0 con 0x7f8e7c0387f0 2024-03-23T17:43:50.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:49 smithi086 ceph-mon[38221]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:43:50.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:49 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:43:50.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:49 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:43:50.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:49 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:43:50.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:49 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:43:50.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:49 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi092", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:43:50.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:49 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:43:50.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:49 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:43:50.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:49 smithi086 ceph-mon[38221]: Updating smithi092:/etc/ceph/ceph.conf 2024-03-23T17:43:50.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:49 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:43:50.018 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:49 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:43:50.018 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:49 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:43:51.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:50 smithi086 ceph-mon[38221]: Updating smithi092:/var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T17:43:51.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:50 smithi086 ceph-mon[38221]: Updating smithi092:/etc/ceph/ceph.client.admin.keyring 2024-03-23T17:43:51.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:50 smithi086 ceph-mon[38221]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi152", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:43:51.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:50 smithi086 ceph-mon[38221]: Updating smithi092:/var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-23T17:43:51.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:50 smithi086 ceph-mon[38221]: Deploying cephadm binary to smithi152 2024-03-23T17:43:51.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:51.287+0000 7f8e8f400700 1 -- 172.21.15.86:0/1130364931 <== mgr.14152 v2:172.21.15.86:6800/2734784244 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f8e98073cb0 con 0x7f8e7c0387f0 2024-03-23T17:43:51.288 INFO:teuthology.orchestra.run.smithi086.stdout:Added host 'smithi152' with addr '172.21.15.152' 2024-03-23T17:43:51.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:51.289+0000 7f8e9d800700 1 -- 172.21.15.86:0/1130364931 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f8e7c0387f0 msgr2=0x7f8e7c03acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:43:51.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:51.289+0000 7f8e9d800700 1 --2- 172.21.15.86:0/1130364931 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f8e7c0387f0 0x7f8e7c03acb0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8e84006fd0 tx=0x7f8e84006e40 comp rx=0 tx=0).stop 2024-03-23T17:43:51.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:51.289+0000 7f8e9d800700 1 -- 172.21.15.86:0/1130364931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e981066c0 msgr2=0x7f8e980730b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:43:51.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:51.289+0000 7f8e9d800700 1 --2- 172.21.15.86:0/1130364931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e981066c0 0x7f8e980730b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8e88006010 tx=0x7f8e88004ff0 comp rx=0 tx=0).stop 2024-03-23T17:43:51.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:51.289+0000 7f8e9d800700 1 -- 172.21.15.86:0/1130364931 shutdown_connections 2024-03-23T17:43:51.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:51.289+0000 7f8e9d800700 1 --2- 172.21.15.86:0/1130364931 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f8e7c0387f0 0x7f8e7c03acb0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:51.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:51.289+0000 7f8e9d800700 1 --2- 172.21.15.86:0/1130364931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e981066c0 0x7f8e980730b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:51.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:51.289+0000 7f8e9d800700 1 -- 172.21.15.86:0/1130364931 >> 172.21.15.86:0/1130364931 conn(0x7f8e980fc1b0 msgr2=0x7f8e981082f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:43:51.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:51.290+0000 7f8e9d800700 1 -- 172.21.15.86:0/1130364931 shutdown_connections 2024-03-23T17:43:51.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:51.290+0000 7f8e9d800700 1 -- 172.21.15.86:0/1130364931 wait complete. 2024-03-23T17:43:51.640 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:51 smithi086 ceph-mon[38221]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:43:51.640 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:51 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:43:51.640 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:51 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:43:51.666 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph orch host ls --format=json 2024-03-23T17:43:52.009 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T17:43:52.766 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:52 smithi086 ceph-mon[38221]: Added host smithi152 2024-03-23T17:43:52.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:52 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:43:52.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:52 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:43:53.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:53.095+0000 7f707dc00700 1 -- 172.21.15.86:0/1177962136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70781066c0 msgr2=0x7f7078106aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:43:53.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:53.095+0000 7f707dc00700 1 --2- 172.21.15.86:0/1177962136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70781066c0 0x7f7078106aa0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7068009910 tx=0x7f7068009c20 comp rx=0 tx=0).stop 2024-03-23T17:43:53.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:53.095+0000 7f707dc00700 1 -- 172.21.15.86:0/1177962136 shutdown_connections 2024-03-23T17:43:53.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:53.095+0000 7f707dc00700 1 --2- 172.21.15.86:0/1177962136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70781066c0 0x7f7078106aa0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:53.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:53.095+0000 7f707dc00700 1 -- 172.21.15.86:0/1177962136 >> 172.21.15.86:0/1177962136 conn(0x7f70780fc1b0 msgr2=0x7f70780fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:43:53.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:53.095+0000 7f707dc00700 1 -- 172.21.15.86:0/1177962136 shutdown_connections 2024-03-23T17:43:53.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:53.095+0000 7f707dc00700 1 -- 172.21.15.86:0/1177962136 wait complete. 2024-03-23T17:43:53.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:53.096+0000 7f707dc00700 1 Processor -- start 2024-03-23T17:43:53.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:53.096+0000 7f707dc00700 1 -- start start 2024-03-23T17:43:53.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:53.096+0000 7f707dc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70781066c0 0x7f7078195f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:43:53.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:53.096+0000 7f707dc00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70781964a0 con 0x7f70781066c0 2024-03-23T17:43:53.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:53.097+0000 7f7076a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70781066c0 0x7f7078195f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:43:53.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:53.097+0000 7f7076a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70781066c0 0x7f7078195f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54788/0 (socket says 172.21.15.86:54788) 2024-03-23T17:43:53.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:53.097+0000 7f7076a00700 1 -- 172.21.15.86:0/1085298830 learned_addr learned my addr 172.21.15.86:0/1085298830 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:43:53.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:53.097+0000 7f7076a00700 1 -- 172.21.15.86:0/1085298830 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70680095f0 con 0x7f70781066c0 2024-03-23T17:43:53.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:53.097+0000 7f7076a00700 1 --2- 172.21.15.86:0/1085298830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70781066c0 0x7f7078195f60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7068006010 tx=0x7f7068004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:43:53.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:53.097+0000 7f706f400700 1 -- 172.21.15.86:0/1085298830 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f706801d070 con 0x7f70781066c0 2024-03-23T17:43:53.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:53.097+0000 7f706f400700 1 -- 172.21.15.86:0/1085298830 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f706800bbb0 con 0x7f70781066c0 2024-03-23T17:43:53.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:53.097+0000 7f706f400700 1 -- 172.21.15.86:0/1085298830 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7068003810 con 0x7f70781066c0 2024-03-23T17:43:53.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:53.097+0000 7f707dc00700 1 -- 172.21.15.86:0/1085298830 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7078196700 con 0x7f70781066c0 2024-03-23T17:43:53.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:53.098+0000 7f707dc00700 1 -- 172.21.15.86:0/1085298830 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7078196b80 con 0x7f70781066c0 2024-03-23T17:43:53.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:53.098+0000 7f707dc00700 1 -- 172.21.15.86:0/1085298830 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7078101e00 con 0x7f70781066c0 2024-03-23T17:43:53.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:53.099+0000 7f706f400700 1 -- 172.21.15.86:0/1085298830 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) v1 ==== 45589+0+0 (secure 0 0 0) 0x7f706800bd20 con 0x7f70781066c0 2024-03-23T17:43:53.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:53.099+0000 7f706f400700 1 --2- 172.21.15.86:0/1085298830 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f705c0387f0 0x7f705c03acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:43:53.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:53.099+0000 7f706f400700 1 -- 172.21.15.86:0/1085298830 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f706804c370 con 0x7f70781066c0 2024-03-23T17:43:53.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:53.099+0000 7f706fe00700 1 --2- 172.21.15.86:0/1085298830 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f705c0387f0 0x7f705c03acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:43:53.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:53.100+0000 7f706fe00700 1 --2- 172.21.15.86:0/1085298830 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f705c0387f0 0x7f705c03acb0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7060006fd0 tx=0x7f7060006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:43:53.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:53.104+0000 7f706f400700 1 -- 172.21.15.86:0/1085298830 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f706802aba0 con 0x7f70781066c0 2024-03-23T17:43:53.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:53.309+0000 7f707dc00700 1 -- 172.21.15.86:0/1085298830 --> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f707810a9c0 con 0x7f705c0387f0 2024-03-23T17:43:53.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:53.310+0000 7f706f400700 1 -- 172.21.15.86:0/1085298830 <== mgr.14152 v2:172.21.15.86:6800/2734784244 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+239 (secure 0 0 0) 0x7f707810a9c0 con 0x7f705c0387f0 2024-03-23T17:43:53.311 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:43:53.311 INFO:teuthology.orchestra.run.smithi086.stdout:[{"addr": "172.21.15.86", "hostname": "smithi086", "labels": [], "status": ""}, {"addr": "172.21.15.92", "hostname": "smithi092", "labels": [], "status": ""}, {"addr": "172.21.15.152", "hostname": "smithi152", "labels": [], "status": ""}] 2024-03-23T17:43:53.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:53.313+0000 7f707dc00700 1 -- 172.21.15.86:0/1085298830 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f705c0387f0 msgr2=0x7f705c03acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:43:53.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:53.313+0000 7f707dc00700 1 --2- 172.21.15.86:0/1085298830 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f705c0387f0 0x7f705c03acb0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7060006fd0 tx=0x7f7060006e40 comp rx=0 tx=0).stop 2024-03-23T17:43:53.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:53.313+0000 7f707dc00700 1 -- 172.21.15.86:0/1085298830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70781066c0 msgr2=0x7f7078195f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:43:53.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:53.313+0000 7f707dc00700 1 --2- 172.21.15.86:0/1085298830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70781066c0 0x7f7078195f60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7068006010 tx=0x7f7068004ff0 comp rx=0 tx=0).stop 2024-03-23T17:43:53.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:53.313+0000 7f707dc00700 1 -- 172.21.15.86:0/1085298830 shutdown_connections 2024-03-23T17:43:53.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:53.313+0000 7f707dc00700 1 --2- 172.21.15.86:0/1085298830 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f705c0387f0 0x7f705c03acb0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:53.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:53.313+0000 7f707dc00700 1 --2- 172.21.15.86:0/1085298830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70781066c0 0x7f7078195f60 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:53.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:53.313+0000 7f707dc00700 1 -- 172.21.15.86:0/1085298830 >> 172.21.15.86:0/1085298830 conn(0x7f70780fc1b0 msgr2=0x7f70781082f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:43:53.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:53.313+0000 7f707dc00700 1 -- 172.21.15.86:0/1085298830 shutdown_connections 2024-03-23T17:43:53.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:53.313+0000 7f707dc00700 1 -- 172.21.15.86:0/1085298830 wait complete. 2024-03-23T17:43:53.681 INFO:tasks.cephadm:Setting crush tunables to default 2024-03-23T17:43:53.682 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph osd crush tunables default 2024-03-23T17:43:54.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:53 smithi086 ceph-mon[38221]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:43:54.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:53 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:43:54.026 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T17:43:54.766 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:54 smithi086 ceph-mon[38221]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T17:43:55.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:55.130+0000 7f735d400700 1 -- 172.21.15.86:0/664746806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73581066c0 msgr2=0x7f7358106aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:43:55.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:55.130+0000 7f735d400700 1 --2- 172.21.15.86:0/664746806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73581066c0 0x7f7358106aa0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7348009910 tx=0x7f7348009c20 comp rx=0 tx=0).stop 2024-03-23T17:43:55.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:55.130+0000 7f735d400700 1 -- 172.21.15.86:0/664746806 shutdown_connections 2024-03-23T17:43:55.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:55.130+0000 7f735d400700 1 --2- 172.21.15.86:0/664746806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73581066c0 0x7f7358106aa0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:55.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:55.130+0000 7f735d400700 1 -- 172.21.15.86:0/664746806 >> 172.21.15.86:0/664746806 conn(0x7f73580fc1b0 msgr2=0x7f73580fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:43:55.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:55.130+0000 7f735d400700 1 -- 172.21.15.86:0/664746806 shutdown_connections 2024-03-23T17:43:55.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:55.130+0000 7f735d400700 1 -- 172.21.15.86:0/664746806 wait complete. 2024-03-23T17:43:55.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:55.131+0000 7f735d400700 1 Processor -- start 2024-03-23T17:43:55.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:55.131+0000 7f735d400700 1 -- start start 2024-03-23T17:43:55.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:55.132+0000 7f735d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73581066c0 0x7f7358198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:43:55.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:55.132+0000 7f735d400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73581986a0 con 0x7f73581066c0 2024-03-23T17:43:55.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:55.132+0000 7f7356800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73581066c0 0x7f7358198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:43:55.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:55.132+0000 7f7356800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73581066c0 0x7f7358198160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57630/0 (socket says 172.21.15.86:57630) 2024-03-23T17:43:55.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:55.132+0000 7f7356800700 1 -- 172.21.15.86:0/1041978632 learned_addr learned my addr 172.21.15.86:0/1041978632 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:43:55.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:55.132+0000 7f7356800700 1 -- 172.21.15.86:0/1041978632 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73480095f0 con 0x7f73581066c0 2024-03-23T17:43:55.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:55.132+0000 7f7356800700 1 --2- 172.21.15.86:0/1041978632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73581066c0 0x7f7358198160 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7348006010 tx=0x7f7348004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:43:55.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:55.133+0000 7f734f400700 1 -- 172.21.15.86:0/1041978632 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f734801d070 con 0x7f73581066c0 2024-03-23T17:43:55.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:55.133+0000 7f734f400700 1 -- 172.21.15.86:0/1041978632 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f734800bbb0 con 0x7f73581066c0 2024-03-23T17:43:55.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:55.133+0000 7f734f400700 1 -- 172.21.15.86:0/1041978632 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7348003810 con 0x7f73581066c0 2024-03-23T17:43:55.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:55.133+0000 7f735d400700 1 -- 172.21.15.86:0/1041978632 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7358198900 con 0x7f73581066c0 2024-03-23T17:43:55.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:55.133+0000 7f735d400700 1 -- 172.21.15.86:0/1041978632 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7358198d80 con 0x7f73581066c0 2024-03-23T17:43:55.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:55.134+0000 7f734f400700 1 -- 172.21.15.86:0/1041978632 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) v1 ==== 45589+0+0 (secure 0 0 0) 0x7f734800bd20 con 0x7f73581066c0 2024-03-23T17:43:55.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:55.134+0000 7f735d400700 1 -- 172.21.15.86:0/1041978632 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7358101e00 con 0x7f73581066c0 2024-03-23T17:43:55.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:55.134+0000 7f734f400700 1 --2- 172.21.15.86:0/1041978632 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f733c0387f0 0x7f733c03acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:43:55.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:55.134+0000 7f734f400700 1 -- 172.21.15.86:0/1041978632 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f734804c410 con 0x7f73581066c0 2024-03-23T17:43:55.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:55.134+0000 7f734fe00700 1 --2- 172.21.15.86:0/1041978632 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f733c0387f0 0x7f733c03acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:43:55.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:55.135+0000 7f734fe00700 1 --2- 172.21.15.86:0/1041978632 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f733c0387f0 0x7f733c03acb0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7340006fd0 tx=0x7f7340006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:43:55.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:55.139+0000 7f734f400700 1 -- 172.21.15.86:0/1041978632 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f7348029360 con 0x7f73581066c0 2024-03-23T17:43:55.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:55.345+0000 7f735d400700 1 -- 172.21.15.86:0/1041978632 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f735810a9c0 con 0x7f73581066c0 2024-03-23T17:43:56.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:56.002+0000 7f734f400700 1 -- 172.21.15.86:0/1041978632 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f7348026020 con 0x7f73581066c0 2024-03-23T17:43:56.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:56.004+0000 7f735d400700 1 -- 172.21.15.86:0/1041978632 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f733c0387f0 msgr2=0x7f733c03acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:43:56.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:56.004+0000 7f735d400700 1 --2- 172.21.15.86:0/1041978632 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f733c0387f0 0x7f733c03acb0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7340006fd0 tx=0x7f7340006e40 comp rx=0 tx=0).stop 2024-03-23T17:43:56.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:56.004+0000 7f735d400700 1 -- 172.21.15.86:0/1041978632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73581066c0 msgr2=0x7f7358198160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:43:56.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:56.004+0000 7f735d400700 1 --2- 172.21.15.86:0/1041978632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73581066c0 0x7f7358198160 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7348006010 tx=0x7f7348004ff0 comp rx=0 tx=0).stop 2024-03-23T17:43:56.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:56.005+0000 7f735d400700 1 -- 172.21.15.86:0/1041978632 shutdown_connections 2024-03-23T17:43:56.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:56.005+0000 7f735d400700 1 --2- 172.21.15.86:0/1041978632 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f733c0387f0 0x7f733c03acb0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:56.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:56.005+0000 7f735d400700 1 --2- 172.21.15.86:0/1041978632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73581066c0 0x7f7358198160 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:56.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:56.005+0000 7f735d400700 1 -- 172.21.15.86:0/1041978632 >> 172.21.15.86:0/1041978632 conn(0x7f73580fc1b0 msgr2=0x7f7358108260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:43:56.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:56.005+0000 7f735d400700 1 -- 172.21.15.86:0/1041978632 shutdown_connections 2024-03-23T17:43:56.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:43:56.005+0000 7f735d400700 1 -- 172.21.15.86:0/1041978632 wait complete. 2024-03-23T17:43:56.006 INFO:teuthology.orchestra.run.smithi086.stderr:adjusted tunables profile to default 2024-03-23T17:43:56.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:55 smithi086 ceph-mon[38221]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:43:56.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:55 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:43:56.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:55 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:43:56.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:55 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:43:56.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:55 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:43:56.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:55 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi152", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:43:56.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:55 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:43:56.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:55 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:43:56.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:55 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/1041978632' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-23T17:43:56.408 INFO:tasks.cephadm:Adding mon.a on smithi086 2024-03-23T17:43:56.409 INFO:tasks.cephadm:Adding mon.b on smithi092 2024-03-23T17:43:56.409 INFO:tasks.cephadm:Adding mon.c on smithi152 2024-03-23T17:43:56.409 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph orch apply mon '3;smithi086:172.21.15.86=a;smithi092:172.21.15.92=b;smithi152:172.21.15.152=c' 2024-03-23T17:43:56.646 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T17:43:57.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:56 smithi086 ceph-mon[38221]: Updating smithi152:/etc/ceph/ceph.conf 2024-03-23T17:43:57.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:56 smithi086 ceph-mon[38221]: Updating smithi152:/var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T17:43:57.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:56 smithi086 ceph-mon[38221]: Updating smithi152:/etc/ceph/ceph.client.admin.keyring 2024-03-23T17:43:57.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:56 smithi086 ceph-mon[38221]: Updating smithi152:/var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-23T17:43:57.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:56 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:43:57.018 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:56 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:43:57.018 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:56 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:43:57.018 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:56 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/1041978632' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-23T17:43:57.018 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:56 smithi086 ceph-mon[38221]: osdmap e4: 0 total, 0 up, 0 in 2024-03-23T17:43:57.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:43:57.580+0000 7f3a8117d700 1 -- 172.21.15.152:0/1448797634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a7c1066f0 msgr2=0x7f3a7c106ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:43:57.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:43:57.580+0000 7f3a8117d700 1 --2- 172.21.15.152:0/1448797634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a7c1066f0 0x7f3a7c106ad0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c009910 tx=0x7f3a6c009c20 comp rx=0 tx=0).stop 2024-03-23T17:43:57.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:43:57.580+0000 7f3a8117d700 1 -- 172.21.15.152:0/1448797634 shutdown_connections 2024-03-23T17:43:57.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:43:57.580+0000 7f3a8117d700 1 --2- 172.21.15.152:0/1448797634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a7c1066f0 0x7f3a7c106ad0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:57.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:43:57.580+0000 7f3a8117d700 1 -- 172.21.15.152:0/1448797634 >> 172.21.15.152:0/1448797634 conn(0x7f3a7c0fc220 msgr2=0x7f3a7c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:43:57.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:43:57.580+0000 7f3a8117d700 1 -- 172.21.15.152:0/1448797634 shutdown_connections 2024-03-23T17:43:57.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:43:57.580+0000 7f3a8117d700 1 -- 172.21.15.152:0/1448797634 wait complete. 2024-03-23T17:43:57.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:43:57.581+0000 7f3a8117d700 1 Processor -- start 2024-03-23T17:43:57.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:43:57.581+0000 7f3a8117d700 1 -- start start 2024-03-23T17:43:57.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:43:57.581+0000 7f3a8117d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a7c1066f0 0x7f3a7c198140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:43:57.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:43:57.581+0000 7f3a8117d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7c198680 con 0x7f3a7c1066f0 2024-03-23T17:43:57.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:43:57.582+0000 7f3a7ad9d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a7c1066f0 0x7f3a7c198140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:43:57.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:43:57.582+0000 7f3a7ad9d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a7c1066f0 0x7f3a7c198140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.152:45550/0 (socket says 172.21.15.152:45550) 2024-03-23T17:43:57.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:43:57.582+0000 7f3a7ad9d700 1 -- 172.21.15.152:0/1503006620 learned_addr learned my addr 172.21.15.152:0/1503006620 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-03-23T17:43:57.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:43:57.582+0000 7f3a7ad9d700 1 -- 172.21.15.152:0/1503006620 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a6c0095f0 con 0x7f3a7c1066f0 2024-03-23T17:43:57.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:43:57.582+0000 7f3a7ad9d700 1 --2- 172.21.15.152:0/1503006620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a7c1066f0 0x7f3a7c198140 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c006010 tx=0x7f3a6c004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:43:57.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:43:57.582+0000 7f3a63fff700 1 -- 172.21.15.152:0/1503006620 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3a6c01d070 con 0x7f3a7c1066f0 2024-03-23T17:43:57.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:43:57.582+0000 7f3a8117d700 1 -- 172.21.15.152:0/1503006620 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a7c1988e0 con 0x7f3a7c1066f0 2024-03-23T17:43:57.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:43:57.582+0000 7f3a63fff700 1 -- 172.21.15.152:0/1503006620 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a6c00bbb0 con 0x7f3a7c1066f0 2024-03-23T17:43:57.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:43:57.582+0000 7f3a63fff700 1 -- 172.21.15.152:0/1503006620 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3a6c003810 con 0x7f3a7c1066f0 2024-03-23T17:43:57.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:43:57.583+0000 7f3a8117d700 1 -- 172.21.15.152:0/1503006620 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a7c198d60 con 0x7f3a7c1066f0 2024-03-23T17:43:57.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:43:57.583+0000 7f3a63fff700 1 -- 172.21.15.152:0/1503006620 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) v1 ==== 45589+0+0 (secure 0 0 0) 0x7f3a6c003a10 con 0x7f3a7c1066f0 2024-03-23T17:43:57.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:43:57.583+0000 7f3a8117d700 1 -- 172.21.15.152:0/1503006620 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a7c101e30 con 0x7f3a7c1066f0 2024-03-23T17:43:57.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:43:57.584+0000 7f3a63fff700 1 --2- 172.21.15.152:0/1503006620 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f3a640387a0 0x7f3a6403ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:43:57.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:43:57.584+0000 7f3a63fff700 1 -- 172.21.15.152:0/1503006620 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f3a6c04c6e0 con 0x7f3a7c1066f0 2024-03-23T17:43:57.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:43:57.584+0000 7f3a7a59c700 1 --2- 172.21.15.152:0/1503006620 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f3a640387a0 0x7f3a6403ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:43:57.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:43:57.584+0000 7f3a7a59c700 1 --2- 172.21.15.152:0/1503006620 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f3a640387a0 0x7f3a6403ac60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3a70006fd0 tx=0x7f3a70006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:43:57.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:43:57.589+0000 7f3a63fff700 1 -- 172.21.15.152:0/1503006620 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f3a6c029770 con 0x7f3a7c1066f0 2024-03-23T17:43:57.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:43:57.788+0000 7f3a8117d700 1 -- 172.21.15.152:0/1503006620 --> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi086:172.21.15.86=a;smithi092:172.21.15.92=b;smithi152:172.21.15.152=c", "target": ["mon-mgr", ""]}) v1 -- 0x7f3a7c10aab0 con 0x7f3a640387a0 2024-03-23T17:43:57.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:43:57.798+0000 7f3a63fff700 1 -- 172.21.15.152:0/1503006620 <== mgr.14152 v2:172.21.15.86:6800/2734784244 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f3a7c10aab0 con 0x7f3a640387a0 2024-03-23T17:43:57.799 INFO:teuthology.orchestra.run.smithi152.stdout:Scheduled mon update... 2024-03-23T17:43:57.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:43:57.800+0000 7f3a8117d700 1 -- 172.21.15.152:0/1503006620 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f3a640387a0 msgr2=0x7f3a6403ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:43:57.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:43:57.800+0000 7f3a8117d700 1 --2- 172.21.15.152:0/1503006620 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f3a640387a0 0x7f3a6403ac60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3a70006fd0 tx=0x7f3a70006e40 comp rx=0 tx=0).stop 2024-03-23T17:43:57.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:43:57.800+0000 7f3a8117d700 1 -- 172.21.15.152:0/1503006620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a7c1066f0 msgr2=0x7f3a7c198140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:43:57.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:43:57.800+0000 7f3a8117d700 1 --2- 172.21.15.152:0/1503006620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a7c1066f0 0x7f3a7c198140 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c006010 tx=0x7f3a6c004ff0 comp rx=0 tx=0).stop 2024-03-23T17:43:57.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:43:57.800+0000 7f3a8117d700 1 -- 172.21.15.152:0/1503006620 shutdown_connections 2024-03-23T17:43:57.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:43:57.800+0000 7f3a8117d700 1 --2- 172.21.15.152:0/1503006620 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f3a640387a0 0x7f3a6403ac60 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:57.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:43:57.800+0000 7f3a8117d700 1 --2- 172.21.15.152:0/1503006620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a7c1066f0 0x7f3a7c198140 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:43:57.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:43:57.800+0000 7f3a8117d700 1 -- 172.21.15.152:0/1503006620 >> 172.21.15.152:0/1503006620 conn(0x7f3a7c0fc220 msgr2=0x7f3a7c108340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:43:57.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:43:57.800+0000 7f3a8117d700 1 -- 172.21.15.152:0/1503006620 shutdown_connections 2024-03-23T17:43:57.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:43:57.800+0000 7f3a8117d700 1 -- 172.21.15.152:0/1503006620 wait complete. 2024-03-23T17:43:58.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:57 smithi086 ceph-mon[38221]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:43:58.222 DEBUG:teuthology.orchestra.run.smithi092:mon.b> sudo journalctl -f -n 0 -u ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@mon.b.service 2024-03-23T17:43:58.225 DEBUG:teuthology.orchestra.run.smithi152:mon.c> sudo journalctl -f -n 0 -u ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@mon.c.service 2024-03-23T17:43:58.227 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-23T17:43:58.227 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-23T17:43:58.256 INFO:journalctl@ceph.mon.c.smithi152.stdout:-- Logs begin at Sat 2024-03-23 17:19:34 UTC. -- 2024-03-23T17:43:58.261 INFO:journalctl@ceph.mon.b.smithi092.stdout:-- Logs begin at Sat 2024-03-23 17:19:16 UTC. -- 2024-03-23T17:43:58.599 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T17:43:59.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:58 smithi086 ceph-mon[38221]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi086:172.21.15.86=a;smithi092:172.21.15.92=b;smithi152:172.21.15.152=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:43:59.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:58 smithi086 ceph-mon[38221]: Saving service mon spec with placement smithi086:172.21.15.86=a;smithi092:172.21.15.92=b;smithi152:172.21.15.152=c;count:3 2024-03-23T17:43:59.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:58 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:43:59.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:58 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:43:59.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:58 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:43:59.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:58 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:43:59.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:58 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:43:59.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:58 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-23T17:43:59.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:58 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:43:59.268 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:58 smithi086 ceph-mon[38221]: Deploying daemon mon.c on smithi152 2024-03-23T17:43:59.268 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:43:58 smithi086 ceph-mon[38221]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:44:00.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:00.088+0000 7f4236b27700 1 -- 172.21.15.152:0/1722874936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42301018c0 msgr2=0x7f4230101ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:44:00.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:00.088+0000 7f4236b27700 1 --2- 172.21.15.152:0/1722874936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42301018c0 0x7f4230101ca0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4224009910 tx=0x7f4224009c20 comp rx=0 tx=0).stop 2024-03-23T17:44:00.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:00.088+0000 7f4236b27700 1 -- 172.21.15.152:0/1722874936 shutdown_connections 2024-03-23T17:44:00.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:00.088+0000 7f4236b27700 1 --2- 172.21.15.152:0/1722874936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42301018c0 0x7f4230101ca0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:44:00.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:00.088+0000 7f4236b27700 1 -- 172.21.15.152:0/1722874936 >> 172.21.15.152:0/1722874936 conn(0x7f42300fb150 msgr2=0x7f42300fd570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:44:00.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:00.088+0000 7f4236b27700 1 -- 172.21.15.152:0/1722874936 shutdown_connections 2024-03-23T17:44:00.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:00.088+0000 7f4236b27700 1 -- 172.21.15.152:0/1722874936 wait complete. 2024-03-23T17:44:00.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:00.088+0000 7f4236b27700 1 Processor -- start 2024-03-23T17:44:00.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:00.088+0000 7f4236b27700 1 -- start start 2024-03-23T17:44:00.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:00.088+0000 7f4236b27700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42301018c0 0x7f4230196000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:44:00.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:00.088+0000 7f4236b27700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4230196540 con 0x7f42301018c0 2024-03-23T17:44:00.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:00.089+0000 7f42348c3700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42301018c0 0x7f4230196000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:44:00.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:00.089+0000 7f42348c3700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42301018c0 0x7f4230196000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.152:45576/0 (socket says 172.21.15.152:45576) 2024-03-23T17:44:00.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:00.089+0000 7f42348c3700 1 -- 172.21.15.152:0/1254979391 learned_addr learned my addr 172.21.15.152:0/1254979391 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-03-23T17:44:00.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:00.089+0000 7f42348c3700 1 -- 172.21.15.152:0/1254979391 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42240095f0 con 0x7f42301018c0 2024-03-23T17:44:00.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:00.090+0000 7f42348c3700 1 --2- 172.21.15.152:0/1254979391 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42301018c0 0x7f4230196000 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4224006010 tx=0x7f4224004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:44:00.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:00.090+0000 7f422dffb700 1 -- 172.21.15.152:0/1254979391 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f422401d070 con 0x7f42301018c0 2024-03-23T17:44:00.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:00.090+0000 7f4236b27700 1 -- 172.21.15.152:0/1254979391 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4230196740 con 0x7f42301018c0 2024-03-23T17:44:00.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:00.090+0000 7f422dffb700 1 -- 172.21.15.152:0/1254979391 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f422400bc10 con 0x7f42301018c0 2024-03-23T17:44:00.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:00.090+0000 7f422dffb700 1 -- 172.21.15.152:0/1254979391 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f42240038f0 con 0x7f42301018c0 2024-03-23T17:44:00.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:00.090+0000 7f4236b27700 1 -- 172.21.15.152:0/1254979391 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4230196b60 con 0x7f42301018c0 2024-03-23T17:44:00.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:00.091+0000 7f422dffb700 1 -- 172.21.15.152:0/1254979391 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) v1 ==== 45589+0+0 (secure 0 0 0) 0x7f4224003b60 con 0x7f42301018c0 2024-03-23T17:44:00.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:00.091+0000 7f4236b27700 1 -- 172.21.15.152:0/1254979391 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4230069380 con 0x7f42301018c0 2024-03-23T17:44:00.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:00.091+0000 7f422dffb700 1 --2- 172.21.15.152:0/1254979391 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f42200387a0 0x7f422003ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:44:00.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:00.091+0000 7f422dffb700 1 -- 172.21.15.152:0/1254979391 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f422404c6b0 con 0x7f42301018c0 2024-03-23T17:44:00.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:00.091+0000 7f422ffff700 1 --2- 172.21.15.152:0/1254979391 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f42200387a0 0x7f422003ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:44:00.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:00.092+0000 7f422ffff700 1 --2- 172.21.15.152:0/1254979391 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f42200387a0 0x7f422003ac60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f421c006fd0 tx=0x7f421c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:44:00.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:00.097+0000 7f422dffb700 1 -- 172.21.15.152:0/1254979391 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f422402aba0 con 0x7f42301018c0 2024-03-23T17:44:00.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:00.386+0000 7f4236b27700 1 -- 172.21.15.152:0/1254979391 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f4230197460 con 0x7f42301018c0 2024-03-23T17:44:00.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:00.387+0000 7f422dffb700 1 -- 172.21.15.152:0/1254979391 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+736 (secure 0 0 0) 0x7f4224026070 con 0x7f42301018c0 2024-03-23T17:44:00.388 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-23T17:44:00.388 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":1,"fsid":"9d76c9e8-e93c-11ee-95cd-87774f69a715","modified":"2024-03-23T17:42:26.888040Z","created":"2024-03-23T17:42:26.888040Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:3300","nonce":0},{"type":"v1","addr":"172.21.15.86:6789","nonce":0}]},"addr":"172.21.15.86:6789/0","public_addr":"172.21.15.86:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-23T17:44:00.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:00.390+0000 7f4236b27700 1 -- 172.21.15.152:0/1254979391 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f42200387a0 msgr2=0x7f422003ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:44:00.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:00.390+0000 7f4236b27700 1 --2- 172.21.15.152:0/1254979391 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f42200387a0 0x7f422003ac60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f421c006fd0 tx=0x7f421c006e40 comp rx=0 tx=0).stop 2024-03-23T17:44:00.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:00.390+0000 7f4236b27700 1 -- 172.21.15.152:0/1254979391 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42301018c0 msgr2=0x7f4230196000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:44:00.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:00.390+0000 7f4236b27700 1 --2- 172.21.15.152:0/1254979391 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42301018c0 0x7f4230196000 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4224006010 tx=0x7f4224004ff0 comp rx=0 tx=0).stop 2024-03-23T17:44:00.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:00.390+0000 7f4236b27700 1 -- 172.21.15.152:0/1254979391 shutdown_connections 2024-03-23T17:44:00.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:00.390+0000 7f4236b27700 1 --2- 172.21.15.152:0/1254979391 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f42200387a0 0x7f422003ac60 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:44:00.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:00.390+0000 7f4236b27700 1 --2- 172.21.15.152:0/1254979391 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42301018c0 0x7f4230196000 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:44:00.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:00.390+0000 7f4236b27700 1 -- 172.21.15.152:0/1254979391 >> 172.21.15.152:0/1254979391 conn(0x7f42300fb150 msgr2=0x7f4230104680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:44:00.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:00.390+0000 7f4236b27700 1 -- 172.21.15.152:0/1254979391 shutdown_connections 2024-03-23T17:44:00.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:00.390+0000 7f4236b27700 1 -- 172.21.15.152:0/1254979391 wait complete. 2024-03-23T17:44:00.392 INFO:teuthology.orchestra.run.smithi152.stderr:dumped monmap epoch 1 2024-03-23T17:44:01.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:01 smithi086 ceph-mon[38221]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:44:01.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:01 smithi086 ceph-mon[38221]: from='client.? 172.21.15.152:0/1254979391' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-23T17:44:01.797 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-23T17:44:01.798 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-23T17:44:01.828 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:01 smithi152 systemd[1]: Started Ceph mon.c for 9d76c9e8-e93c-11ee-95cd-87774f69a715. 2024-03-23T17:44:01.828 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:01 smithi152 ceph-mon[47949]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-23T17:44:01.828 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:01 smithi152 ceph-mon[47949]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-03-23T17:44:01.829 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:01 smithi152 ceph-mon[47949]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 8924597c-470d-401d-84cf-155a4a6ad76c 2024-03-23T17:44:01.829 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:01 smithi152 ceph-mon[47949]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711215841371438, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-03-23T17:44:01.829 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:01 smithi152 ceph-mon[47949]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-03-23T17:44:01.829 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:01 smithi152 ceph-mon[47949]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711215841372037, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711215841, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "8924597c-470d-401d-84cf-155a4a6ad76c", "db_session_id": "IUH0VX8M5ZS2YXWH8QXA", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-03-23T17:44:01.829 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:01 smithi152 ceph-mon[47949]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711215841372129, "job": 1, "event": "recovery_finished"} 2024-03-23T17:44:01.829 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:01 smithi152 ceph-mon[47949]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-03-23T17:44:01.829 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:01 smithi152 ceph-mon[47949]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-23T17:44:01.829 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:01 smithi152 ceph-mon[47949]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55f452cce000 2024-03-23T17:44:01.829 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:01 smithi152 ceph-mon[47949]: rocksdb: DB pointer 0x55f452cbc000 2024-03-23T17:44:01.829 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:01 smithi152 ceph-mon[47949]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-23T17:44:01.829 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:01 smithi152 ceph-mon[47949]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-23T17:44:01.829 INFO:journalctl@ceph.mon.c.smithi152.stdout: ** DB Stats ** 2024-03-23T17:44:01.829 INFO:journalctl@ceph.mon.c.smithi152.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-23T17:44:01.829 INFO:journalctl@ceph.mon.c.smithi152.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-23T17:44:01.830 INFO:journalctl@ceph.mon.c.smithi152.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-23T17:44:01.830 INFO:journalctl@ceph.mon.c.smithi152.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-23T17:44:01.830 INFO:journalctl@ceph.mon.c.smithi152.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-23T17:44:01.830 INFO:journalctl@ceph.mon.c.smithi152.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-23T17:44:01.830 INFO:journalctl@ceph.mon.c.smithi152.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-23T17:44:01.830 INFO:journalctl@ceph.mon.c.smithi152.stdout: 2024-03-23T17:44:01.830 INFO:journalctl@ceph.mon.c.smithi152.stdout: ** Compaction Stats [default] ** 2024-03-23T17:44:01.830 INFO:journalctl@ceph.mon.c.smithi152.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-23T17:44:01.830 INFO:journalctl@ceph.mon.c.smithi152.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-23T17:44:01.830 INFO:journalctl@ceph.mon.c.smithi152.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-23T17:44:01.830 INFO:journalctl@ceph.mon.c.smithi152.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-23T17:44:01.830 INFO:journalctl@ceph.mon.c.smithi152.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-23T17:44:01.831 INFO:journalctl@ceph.mon.c.smithi152.stdout: 2024-03-23T17:44:01.831 INFO:journalctl@ceph.mon.c.smithi152.stdout: ** Compaction Stats [default] ** 2024-03-23T17:44:01.831 INFO:journalctl@ceph.mon.c.smithi152.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-23T17:44:01.831 INFO:journalctl@ceph.mon.c.smithi152.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-23T17:44:01.831 INFO:journalctl@ceph.mon.c.smithi152.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-23T17:44:01.831 INFO:journalctl@ceph.mon.c.smithi152.stdout: 2024-03-23T17:44:01.831 INFO:journalctl@ceph.mon.c.smithi152.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-23T17:44:01.831 INFO:journalctl@ceph.mon.c.smithi152.stdout: 2024-03-23T17:44:01.831 INFO:journalctl@ceph.mon.c.smithi152.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-23T17:44:01.831 INFO:journalctl@ceph.mon.c.smithi152.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-23T17:44:01.831 INFO:journalctl@ceph.mon.c.smithi152.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-23T17:44:01.831 INFO:journalctl@ceph.mon.c.smithi152.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-23T17:44:01.831 INFO:journalctl@ceph.mon.c.smithi152.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-23T17:44:01.831 INFO:journalctl@ceph.mon.c.smithi152.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-23T17:44:01.831 INFO:journalctl@ceph.mon.c.smithi152.stdout: Cumulative compaction: 0.00 GB write, 0.44 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-23T17:44:01.832 INFO:journalctl@ceph.mon.c.smithi152.stdout: Interval compaction: 0.00 GB write, 0.44 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-23T17:44:01.832 INFO:journalctl@ceph.mon.c.smithi152.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-23T17:44:01.832 INFO:journalctl@ceph.mon.c.smithi152.stdout: Block cache BinnedLRUCache@0x55f452c311f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-03-23T17:44:01.832 INFO:journalctl@ceph.mon.c.smithi152.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-03-23T17:44:01.832 INFO:journalctl@ceph.mon.c.smithi152.stdout: 2024-03-23T17:44:01.832 INFO:journalctl@ceph.mon.c.smithi152.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-23T17:44:01.832 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:01 smithi152 ceph-mon[47949]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-03-23T17:44:01.832 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:01 smithi152 ceph-mon[47949]: using public_addr v2:172.21.15.152:0/0 -> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] 2024-03-23T17:44:01.832 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:01 smithi152 ceph-mon[47949]: starting mon.c rank -1 at public addrs [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] at bind addrs [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 2024-03-23T17:44:01.832 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:01 smithi152 ceph-mon[47949]: mon.c@-1(???) e0 preinit fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 2024-03-23T17:44:01.832 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:01 smithi152 ceph-mon[47949]: mon.c@-1(synchronizing).mds e1 new map 2024-03-23T17:44:01.832 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:01 smithi152 ceph-mon[47949]: mon.c@-1(synchronizing).mds e1 print_map 2024-03-23T17:44:01.832 INFO:journalctl@ceph.mon.c.smithi152.stdout: e1 2024-03-23T17:44:01.832 INFO:journalctl@ceph.mon.c.smithi152.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-23T17:44:01.833 INFO:journalctl@ceph.mon.c.smithi152.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-23T17:44:01.833 INFO:journalctl@ceph.mon.c.smithi152.stdout: legacy client fscid: -1 2024-03-23T17:44:01.833 INFO:journalctl@ceph.mon.c.smithi152.stdout: 2024-03-23T17:44:01.833 INFO:journalctl@ceph.mon.c.smithi152.stdout: No filesystems configured 2024-03-23T17:44:01.833 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:01 smithi152 ceph-mon[47949]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-03-23T17:44:01.833 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:01 smithi152 ceph-mon[47949]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-03-23T17:44:01.833 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:01 smithi152 ceph-mon[47949]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-03-23T17:44:01.833 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:01 smithi152 ceph-mon[47949]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-03-23T17:44:01.833 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:01 smithi152 ceph-mon[47949]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-03-23T17:44:01.833 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:01 smithi152 ceph-mon[47949]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-03-23T17:44:01.833 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:01 smithi152 ceph-mon[47949]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-23T17:44:01.833 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:01 smithi152 ceph-mon[47949]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-23T17:44:01.833 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:01 smithi152 ceph-mon[47949]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-23T17:44:01.834 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:01 smithi152 ceph-mon[47949]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-23T17:44:01.834 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:01 smithi152 ceph-mon[47949]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:44:01.834 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:01 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:01.834 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:01 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:44:01.834 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:01 smithi152 ceph-mon[47949]: Added host smithi152 2024-03-23T17:44:01.834 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:01 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:01.834 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:01 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:01.834 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:01 smithi152 ceph-mon[47949]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:44:01.834 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:01 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:01.834 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:01 smithi152 ceph-mon[47949]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T17:44:01.834 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:01 smithi152 ceph-mon[47949]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:44:01.834 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:01 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:01.834 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:01 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:01.834 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:01 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:01.834 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:01 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:01.835 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:01 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi152", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:44:01.835 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:01 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:44:01.835 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:01 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:44:01.835 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:01 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/1041978632' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-23T17:44:01.835 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:01 smithi152 ceph-mon[47949]: Updating smithi152:/etc/ceph/ceph.conf 2024-03-23T17:44:01.835 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:01 smithi152 ceph-mon[47949]: Updating smithi152:/var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T17:44:01.835 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:01 smithi152 ceph-mon[47949]: Updating smithi152:/etc/ceph/ceph.client.admin.keyring 2024-03-23T17:44:01.835 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:01 smithi152 ceph-mon[47949]: Updating smithi152:/var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-23T17:44:01.835 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:01 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:01.835 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:01 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:01.835 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:01 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:01.835 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:01 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/1041978632' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-23T17:44:01.835 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:01 smithi152 ceph-mon[47949]: osdmap e4: 0 total, 0 up, 0 in 2024-03-23T17:44:01.835 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:01 smithi152 ceph-mon[47949]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:44:01.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:01 smithi152 ceph-mon[47949]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi086:172.21.15.86=a;smithi092:172.21.15.92=b;smithi152:172.21.15.152=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:44:01.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:01 smithi152 ceph-mon[47949]: Saving service mon spec with placement smithi086:172.21.15.86=a;smithi092:172.21.15.92=b;smithi152:172.21.15.152=c;count:3 2024-03-23T17:44:01.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:01 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:01.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:01 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:44:01.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:01 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:44:01.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:01 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:44:01.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:01 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:01.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:01 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-23T17:44:01.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:01 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:44:01.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:01 smithi152 ceph-mon[47949]: Deploying daemon mon.c on smithi152 2024-03-23T17:44:01.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:01 smithi152 ceph-mon[47949]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:44:01.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:01 smithi152 ceph-mon[47949]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:44:01.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:01 smithi152 ceph-mon[47949]: from='client.? 172.21.15.152:0/1254979391' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-23T17:44:01.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:01 smithi152 ceph-mon[47949]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-03-23T17:44:02.019 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.c/config 2024-03-23T17:44:06.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:06.514+0000 7fee6a134700 1 -- 172.21.15.152:0/3071456848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee641056f0 msgr2=0x7fee4c007820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:44:06.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:06.514+0000 7fee6a134700 1 --2- 172.21.15.152:0/3071456848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee641056f0 0x7fee4c007820 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fee58005fa0 tx=0x7fee5800f4a0 comp rx=0 tx=0).stop 2024-03-23T17:44:06.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:06.515+0000 7fee6a134700 1 -- 172.21.15.152:0/3071456848 shutdown_connections 2024-03-23T17:44:06.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:06.515+0000 7fee6a134700 1 --2- 172.21.15.152:0/3071456848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee641056f0 0x7fee4c007820 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:44:06.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:06.515+0000 7fee6a134700 1 -- 172.21.15.152:0/3071456848 >> 172.21.15.152:0/3071456848 conn(0x7fee64076390 msgr2=0x7fee640fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:44:06.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:06.515+0000 7fee6a134700 1 -- 172.21.15.152:0/3071456848 shutdown_connections 2024-03-23T17:44:06.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:06.515+0000 7fee6a134700 1 -- 172.21.15.152:0/3071456848 wait complete. 2024-03-23T17:44:06.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:06.515+0000 7fee6a134700 1 Processor -- start 2024-03-23T17:44:06.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:06.516+0000 7fee6a134700 1 -- start start 2024-03-23T17:44:06.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:06.516+0000 7fee6a134700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee641056f0 0x7fee641982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:44:06.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:06.516+0000 7fee6a134700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee641987f0 0x7fee6419cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:44:06.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:06.516+0000 7fee6a134700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee64198d80 con 0x7fee641987f0 2024-03-23T17:44:06.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:06.516+0000 7fee6a134700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee64198ef0 con 0x7fee641056f0 2024-03-23T17:44:06.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:06.516+0000 7fee637fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee641056f0 0x7fee641982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:44:06.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:06.516+0000 7fee637fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee641056f0 0x7fee641982b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:47628/0 (socket says 172.21.15.152:47628) 2024-03-23T17:44:06.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:06.516+0000 7fee637fe700 1 -- 172.21.15.152:0/1996134700 learned_addr learned my addr 172.21.15.152:0/1996134700 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-03-23T17:44:06.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:06.517+0000 7fee62ffd700 1 --2- 172.21.15.152:0/1996134700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee641987f0 0x7fee6419cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:44:06.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:06.517+0000 7fee637fe700 1 -- 172.21.15.152:0/1996134700 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee641056f0 msgr2=0x7fee641982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-23T17:44:06.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:06.517+0000 7fee637fe700 1 -- 172.21.15.152:0/1996134700 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee641056f0 msgr2=0x7fee641982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-23T17:44:06.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:06.517+0000 7fee637fe700 1 --2- 172.21.15.152:0/1996134700 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee641056f0 0x7fee641982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-23T17:44:06.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:06.517+0000 7fee637fe700 1 --2- 172.21.15.152:0/1996134700 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee641056f0 0x7fee641982b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-23T17:44:06.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:06.517+0000 7fee62ffd700 1 -- 172.21.15.152:0/1996134700 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee641056f0 msgr2=0x7fee641982b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-23T17:44:06.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:06.517+0000 7fee62ffd700 1 --2- 172.21.15.152:0/1996134700 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee641056f0 0x7fee641982b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:44:06.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:06.517+0000 7fee62ffd700 1 -- 172.21.15.152:0/1996134700 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee580149d0 con 0x7fee641987f0 2024-03-23T17:44:06.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:06.517+0000 7fee62ffd700 1 --2- 172.21.15.152:0/1996134700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee641987f0 0x7fee6419cba0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fee5400b4d0 tx=0x7fee5400b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:44:06.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:06.518+0000 7fee60ff9700 1 -- 172.21.15.152:0/1996134700 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fee54011630 con 0x7fee641987f0 2024-03-23T17:44:06.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:06.518+0000 7fee60ff9700 1 -- 172.21.15.152:0/1996134700 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee54011c50 con 0x7fee641987f0 2024-03-23T17:44:06.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:06.518+0000 7fee6a134700 1 -- 172.21.15.152:0/1996134700 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee6419d140 con 0x7fee641987f0 2024-03-23T17:44:06.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:06.518+0000 7fee60ff9700 1 -- 172.21.15.152:0/1996134700 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fee54010d90 con 0x7fee641987f0 2024-03-23T17:44:06.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:06.518+0000 7fee6a134700 1 -- 172.21.15.152:0/1996134700 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee6419d690 con 0x7fee641987f0 2024-03-23T17:44:06.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:06.519+0000 7fee60ff9700 1 -- 172.21.15.152:0/1996134700 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) v1 ==== 45589+0+0 (secure 0 0 0) 0x7fee5401a400 con 0x7fee641987f0 2024-03-23T17:44:06.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:06.519+0000 7fee60ff9700 1 --2- 172.21.15.152:0/1996134700 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fee500385e0 0x7fee5003aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:44:06.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:06.519+0000 7fee60ff9700 1 -- 172.21.15.152:0/1996134700 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fee5404ada0 con 0x7fee641987f0 2024-03-23T17:44:06.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:06.519+0000 7fee6a134700 1 -- 172.21.15.152:0/1996134700 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee6418dcd0 con 0x7fee641987f0 2024-03-23T17:44:06.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:06.519+0000 7fee637fe700 1 --2- 172.21.15.152:0/1996134700 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fee500385e0 0x7fee5003aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:44:06.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:06.520+0000 7fee637fe700 1 --2- 172.21.15.152:0/1996134700 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fee500385e0 0x7fee5003aaa0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fee58005ed0 tx=0x7fee58005e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:44:06.526 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:06.526+0000 7fee60ff9700 1 -- 172.21.15.152:0/1996134700 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fee540193b0 con 0x7fee641987f0 2024-03-23T17:44:06.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:06 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-23T17:44:06.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:06 smithi086 ceph-mon[38221]: mon.a calling monitor election 2024-03-23T17:44:06.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:06 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T17:44:06.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:06 smithi086 ceph-mon[38221]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:44:06.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:06 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T17:44:06.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:06 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T17:44:06.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:06 smithi086 ceph-mon[38221]: mon.c calling monitor election 2024-03-23T17:44:06.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:06 smithi086 ceph-mon[38221]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:44:06.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:06 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T17:44:06.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:06 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T17:44:06.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:06 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T17:44:06.768 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:06 smithi086 ceph-mon[38221]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-23T17:44:06.768 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:06 smithi086 ceph-mon[38221]: monmap e2: 2 mons at {a=[v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0],c=[v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-23T17:44:06.768 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:06 smithi086 ceph-mon[38221]: fsmap 2024-03-23T17:44:06.768 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:06 smithi086 ceph-mon[38221]: osdmap e4: 0 total, 0 up, 0 in 2024-03-23T17:44:06.768 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:06 smithi086 ceph-mon[38221]: mgrmap e13: x(active, since 38s) 2024-03-23T17:44:06.768 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:06 smithi086 ceph-mon[38221]: overall HEALTH_OK 2024-03-23T17:44:06.768 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:06 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:06.768 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:06 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:06.768 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:06 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:06.768 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:06 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-23T17:44:06.768 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:06 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:44:06.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:06.800+0000 7fee6a134700 1 -- 172.21.15.152:0/1996134700 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fee640623e0 con 0x7fee641987f0 2024-03-23T17:44:06.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:06.800+0000 7fee60ff9700 1 -- 172.21.15.152:0/1996134700 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1006 (secure 0 0 0) 0x7fee540172c0 con 0x7fee641987f0 2024-03-23T17:44:06.801 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-23T17:44:06.801 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":2,"fsid":"9d76c9e8-e93c-11ee-95cd-87774f69a715","modified":"2024-03-23T17:44:01.402789Z","created":"2024-03-23T17:42:26.888040Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:3300","nonce":0},{"type":"v1","addr":"172.21.15.86:6789","nonce":0}]},"addr":"172.21.15.86:6789/0","public_addr":"172.21.15.86:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:3300","nonce":0},{"type":"v1","addr":"172.21.15.152:6789","nonce":0}]},"addr":"172.21.15.152:6789/0","public_addr":"172.21.15.152:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-23T17:44:06.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:06.802+0000 7fee6a134700 1 -- 172.21.15.152:0/1996134700 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fee500385e0 msgr2=0x7fee5003aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:44:06.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:06.802+0000 7fee6a134700 1 --2- 172.21.15.152:0/1996134700 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fee500385e0 0x7fee5003aaa0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fee58005ed0 tx=0x7fee58005e40 comp rx=0 tx=0).stop 2024-03-23T17:44:06.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:06.802+0000 7fee6a134700 1 -- 172.21.15.152:0/1996134700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee641987f0 msgr2=0x7fee6419cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:44:06.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:06.802+0000 7fee6a134700 1 --2- 172.21.15.152:0/1996134700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee641987f0 0x7fee6419cba0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fee5400b4d0 tx=0x7fee5400b890 comp rx=0 tx=0).stop 2024-03-23T17:44:06.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:06.802+0000 7fee6a134700 1 -- 172.21.15.152:0/1996134700 shutdown_connections 2024-03-23T17:44:06.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:06.802+0000 7fee6a134700 1 --2- 172.21.15.152:0/1996134700 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee641056f0 0x7fee641982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:44:06.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:06.802+0000 7fee6a134700 1 --2- 172.21.15.152:0/1996134700 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fee500385e0 0x7fee5003aaa0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:44:06.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:06.802+0000 7fee6a134700 1 --2- 172.21.15.152:0/1996134700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee641987f0 0x7fee6419cba0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:44:06.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:06.802+0000 7fee6a134700 1 -- 172.21.15.152:0/1996134700 >> 172.21.15.152:0/1996134700 conn(0x7fee64076390 msgr2=0x7fee640fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:44:06.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:06.803+0000 7fee6a134700 1 -- 172.21.15.152:0/1996134700 shutdown_connections 2024-03-23T17:44:06.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:06.803+0000 7fee6a134700 1 -- 172.21.15.152:0/1996134700 wait complete. 2024-03-23T17:44:06.804 INFO:teuthology.orchestra.run.smithi152.stderr:dumped monmap epoch 2 2024-03-23T17:44:06.823 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:06 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-23T17:44:06.823 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:06 smithi152 ceph-mon[47949]: mon.a calling monitor election 2024-03-23T17:44:06.823 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:06 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T17:44:06.823 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:06 smithi152 ceph-mon[47949]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:44:06.824 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:06 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T17:44:06.824 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:06 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T17:44:06.824 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:06 smithi152 ceph-mon[47949]: mon.c calling monitor election 2024-03-23T17:44:06.824 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:06 smithi152 ceph-mon[47949]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:44:06.824 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:06 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T17:44:06.824 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:06 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T17:44:06.824 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:06 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T17:44:06.824 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:06 smithi152 ceph-mon[47949]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-23T17:44:06.824 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:06 smithi152 ceph-mon[47949]: monmap e2: 2 mons at {a=[v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0],c=[v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-23T17:44:06.824 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:06 smithi152 ceph-mon[47949]: fsmap 2024-03-23T17:44:06.824 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:06 smithi152 ceph-mon[47949]: osdmap e4: 0 total, 0 up, 0 in 2024-03-23T17:44:06.824 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:06 smithi152 ceph-mon[47949]: mgrmap e13: x(active, since 38s) 2024-03-23T17:44:06.824 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:06 smithi152 ceph-mon[47949]: overall HEALTH_OK 2024-03-23T17:44:06.824 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:06 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:06.824 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:06 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:06.825 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:06 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:06.825 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:06 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-23T17:44:06.825 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:06 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:44:07.766 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:07 smithi086 ceph-mon[38221]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:44:07.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:07 smithi086 ceph-mon[38221]: Deploying daemon mon.b on smithi092 2024-03-23T17:44:07.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:07 smithi086 ceph-mon[38221]: from='client.? 172.21.15.152:0/1996134700' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-23T17:44:07.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:07 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T17:44:07.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:07 smithi152 ceph-mon[47949]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:44:07.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:07 smithi152 ceph-mon[47949]: Deploying daemon mon.b on smithi092 2024-03-23T17:44:07.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:07 smithi152 ceph-mon[47949]: from='client.? 172.21.15.152:0/1996134700' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-23T17:44:07.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:07 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T17:44:08.199 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-23T17:44:08.199 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-23T17:44:08.428 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.c/config 2024-03-23T17:44:09.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:09 smithi092 bash[79051]: 0cc843b3a2dbb0ae8f07cf17923f5e79794cfb4d9b5e8f00fed0263b25678e18 2024-03-23T17:44:09.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:09 smithi092 systemd[1]: Started Ceph mon.b for 9d76c9e8-e93c-11ee-95cd-87774f69a715. 2024-03-23T17:44:14.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:14 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-23T17:44:14.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:14 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T17:44:14.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:14 smithi086 ceph-mon[38221]: mon.a calling monitor election 2024-03-23T17:44:14.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:14 smithi086 ceph-mon[38221]: mon.c calling monitor election 2024-03-23T17:44:14.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:14 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T17:44:14.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:14 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T17:44:14.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:14 smithi086 ceph-mon[38221]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:44:14.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:14 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T17:44:14.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:14 smithi086 ceph-mon[38221]: mon.b calling monitor election 2024-03-23T17:44:14.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:14 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T17:44:14.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:14 smithi086 ceph-mon[38221]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:44:14.518 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:14 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T17:44:14.518 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:14 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T17:44:14.518 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:14 smithi086 ceph-mon[38221]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-23T17:44:14.518 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:14 smithi086 ceph-mon[38221]: monmap e3: 3 mons at {a=[v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0],b=[v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0],c=[v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-23T17:44:14.518 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:14 smithi086 ceph-mon[38221]: fsmap 2024-03-23T17:44:14.518 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:14 smithi086 ceph-mon[38221]: osdmap e4: 0 total, 0 up, 0 in 2024-03-23T17:44:14.518 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:14 smithi086 ceph-mon[38221]: mgrmap e13: x(active, since 45s) 2024-03-23T17:44:14.518 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:14 smithi086 ceph-mon[38221]: overall HEALTH_OK 2024-03-23T17:44:14.518 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:14 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:14.518 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:14 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:14.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:14 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-23T17:44:14.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:14 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T17:44:14.558 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:14 smithi092 ceph-mon[79089]: mon.a calling monitor election 2024-03-23T17:44:14.558 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:14 smithi092 ceph-mon[79089]: mon.c calling monitor election 2024-03-23T17:44:14.558 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:14 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T17:44:14.558 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:14 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T17:44:14.558 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:14 smithi092 ceph-mon[79089]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:44:14.558 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:14 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T17:44:14.558 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:14 smithi092 ceph-mon[79089]: mon.b calling monitor election 2024-03-23T17:44:14.558 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:14 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T17:44:14.558 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:14 smithi092 ceph-mon[79089]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:44:14.558 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:14 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T17:44:14.558 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:14 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T17:44:14.558 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:14 smithi092 ceph-mon[79089]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-23T17:44:14.558 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:14 smithi092 ceph-mon[79089]: monmap e3: 3 mons at {a=[v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0],b=[v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0],c=[v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-23T17:44:14.558 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:14 smithi092 ceph-mon[79089]: fsmap 2024-03-23T17:44:14.559 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:14 smithi092 ceph-mon[79089]: osdmap e4: 0 total, 0 up, 0 in 2024-03-23T17:44:14.559 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:14 smithi092 ceph-mon[79089]: mgrmap e13: x(active, since 45s) 2024-03-23T17:44:14.559 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:14 smithi092 ceph-mon[79089]: overall HEALTH_OK 2024-03-23T17:44:14.559 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:14 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:14.559 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:14 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:14.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:14 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-23T17:44:14.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:14 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T17:44:14.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:14 smithi152 ceph-mon[47949]: mon.a calling monitor election 2024-03-23T17:44:14.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:14 smithi152 ceph-mon[47949]: mon.c calling monitor election 2024-03-23T17:44:14.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:14 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T17:44:14.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:14 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T17:44:14.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:14 smithi152 ceph-mon[47949]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:44:14.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:14 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T17:44:14.587 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:14 smithi152 ceph-mon[47949]: mon.b calling monitor election 2024-03-23T17:44:14.587 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:14 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T17:44:14.587 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:14 smithi152 ceph-mon[47949]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:44:14.587 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:14 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T17:44:14.587 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:14 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T17:44:14.587 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:14 smithi152 ceph-mon[47949]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-23T17:44:14.587 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:14 smithi152 ceph-mon[47949]: monmap e3: 3 mons at {a=[v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0],b=[v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0],c=[v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-23T17:44:14.587 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:14 smithi152 ceph-mon[47949]: fsmap 2024-03-23T17:44:14.587 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:14 smithi152 ceph-mon[47949]: osdmap e4: 0 total, 0 up, 0 in 2024-03-23T17:44:14.587 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:14 smithi152 ceph-mon[47949]: mgrmap e13: x(active, since 45s) 2024-03-23T17:44:14.587 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:14 smithi152 ceph-mon[47949]: overall HEALTH_OK 2024-03-23T17:44:14.587 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:14 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:14.587 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:14 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:15.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:15.453+0000 7f0a7489f700 1 -- 172.21.15.152:0/4087590805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a580062a0 msgr2=0x7f0a58008710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:44:15.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:15.453+0000 7f0a7489f700 1 --2- 172.21.15.152:0/4087590805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a580062a0 0x7f0a58008710 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0a70100790 tx=0x7f0a6400fe90 comp rx=0 tx=0).stop 2024-03-23T17:44:15.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:15.453+0000 7f0a7489f700 1 -- 172.21.15.152:0/4087590805 shutdown_connections 2024-03-23T17:44:15.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:15.453+0000 7f0a7489f700 1 --2- 172.21.15.152:0/4087590805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a580062a0 0x7f0a58008710 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:44:15.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:15.453+0000 7f0a7489f700 1 -- 172.21.15.152:0/4087590805 >> 172.21.15.152:0/4087590805 conn(0x7f0a700fb1c0 msgr2=0x7f0a700fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:44:15.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:15.453+0000 7f0a7489f700 1 -- 172.21.15.152:0/4087590805 shutdown_connections 2024-03-23T17:44:15.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:15.453+0000 7f0a7489f700 1 -- 172.21.15.152:0/4087590805 wait complete. 2024-03-23T17:44:15.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:15.454+0000 7f0a7489f700 1 Processor -- start 2024-03-23T17:44:15.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:15.454+0000 7f0a7489f700 1 -- start start 2024-03-23T17:44:15.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:15.455+0000 7f0a7489f700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0a580062a0 0x7f0a70198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:44:15.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:15.455+0000 7f0a7489f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a701989a0 0x7f0a7019cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:44:15.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:15.455+0000 7f0a7489f700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a7019d290 0x7f0a7019d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:44:15.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:15.455+0000 7f0a7489f700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a70199090 con 0x7f0a7019d290 2024-03-23T17:44:15.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:15.455+0000 7f0a7489f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a701a5000 con 0x7f0a701989a0 2024-03-23T17:44:15.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:15.455+0000 7f0a7489f700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a701a5140 con 0x7f0a580062a0 2024-03-23T17:44:15.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:15.455+0000 7f0a6e59c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0a580062a0 0x7f0a70198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:44:15.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:15.455+0000 7f0a6e59c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0a580062a0 0x7f0a70198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:56650/0 (socket says 172.21.15.152:56650) 2024-03-23T17:44:15.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:15.455+0000 7f0a6e59c700 1 -- 172.21.15.152:0/2409185122 learned_addr learned my addr 172.21.15.152:0/2409185122 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-03-23T17:44:15.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:15.455+0000 7f0a6dd9b700 1 --2- 172.21.15.152:0/2409185122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a701989a0 0x7f0a7019cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:44:15.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:15.455+0000 7f0a6e59c700 1 -- 172.21.15.152:0/2409185122 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0a580062a0 msgr2=0x7f0a70198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-23T17:44:15.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:15.455+0000 7f0a6e59c700 1 -- 172.21.15.152:0/2409185122 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0a580062a0 msgr2=0x7f0a70198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-23T17:44:15.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:15.455+0000 7f0a6e59c700 1 --2- 172.21.15.152:0/2409185122 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0a580062a0 0x7f0a70198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-23T17:44:15.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:15.455+0000 7f0a6e59c700 1 --2- 172.21.15.152:0/2409185122 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0a580062a0 0x7f0a70198460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-23T17:44:15.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:15.455+0000 7f0a6ed9d700 1 --2- 172.21.15.152:0/2409185122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a7019d290 0x7f0a7019d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:44:15.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:15.455+0000 7f0a6dd9b700 1 -- 172.21.15.152:0/2409185122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a701989a0 msgr2=0x7f0a7019cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-03-23T17:44:15.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:15.455+0000 7f0a6dd9b700 1 -- 172.21.15.152:0/2409185122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a701989a0 msgr2=0x7f0a7019cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-23T17:44:15.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:15.455+0000 7f0a6dd9b700 1 --2- 172.21.15.152:0/2409185122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a701989a0 0x7f0a7019cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-23T17:44:15.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:15.455+0000 7f0a6dd9b700 1 --2- 172.21.15.152:0/2409185122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a701989a0 0x7f0a7019cd50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-23T17:44:15.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:15.455+0000 7f0a6ed9d700 1 -- 172.21.15.152:0/2409185122 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0a580062a0 msgr2=0x7f0a70198460 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-23T17:44:15.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:15.455+0000 7f0a6ed9d700 1 --2- 172.21.15.152:0/2409185122 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0a580062a0 0x7f0a70198460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:44:15.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:15.455+0000 7f0a6ed9d700 1 -- 172.21.15.152:0/2409185122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a701989a0 msgr2=0x7f0a7019cd50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-23T17:44:15.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:15.455+0000 7f0a6ed9d700 1 --2- 172.21.15.152:0/2409185122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a701989a0 0x7f0a7019cd50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:44:15.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:15.455+0000 7f0a6ed9d700 1 -- 172.21.15.152:0/2409185122 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a6400f8e0 con 0x7f0a7019d290 2024-03-23T17:44:15.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:15.455+0000 7f0a6ed9d700 1 --2- 172.21.15.152:0/2409185122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a7019d290 0x7f0a7019d740 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0a6800bd50 tx=0x7f0a6800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:44:15.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:15.456+0000 7f0a5f7fe700 1 -- 172.21.15.152:0/2409185122 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a6800cd10 con 0x7f0a7019d290 2024-03-23T17:44:15.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:15.456+0000 7f0a7489f700 1 -- 172.21.15.152:0/2409185122 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a701a5420 con 0x7f0a7019d290 2024-03-23T17:44:15.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:15.456+0000 7f0a5f7fe700 1 -- 172.21.15.152:0/2409185122 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a6800ce70 con 0x7f0a7019d290 2024-03-23T17:44:15.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:15.456+0000 7f0a5f7fe700 1 -- 172.21.15.152:0/2409185122 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a68012840 con 0x7f0a7019d290 2024-03-23T17:44:15.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:15.456+0000 7f0a7489f700 1 -- 172.21.15.152:0/2409185122 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a701a58d0 con 0x7f0a7019d290 2024-03-23T17:44:15.461 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:15.457+0000 7f0a5f7fe700 1 -- 172.21.15.152:0/2409185122 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) v1 ==== 45589+0+0 (secure 0 0 0) 0x7f0a680129c0 con 0x7f0a7019d290 2024-03-23T17:44:15.461 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:15.457+0000 7f0a7489f700 1 -- 172.21.15.152:0/2409185122 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a70103980 con 0x7f0a7019d290 2024-03-23T17:44:15.461 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:15.457+0000 7f0a5f7fe700 1 --2- 172.21.15.152:0/2409185122 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f0a50038c20 0x7f0a5003b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:44:15.461 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:15.457+0000 7f0a5f7fe700 1 -- 172.21.15.152:0/2409185122 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f0a6804ae20 con 0x7f0a7019d290 2024-03-23T17:44:15.461 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:15.457+0000 7f0a6e59c700 1 --2- 172.21.15.152:0/2409185122 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f0a50038c20 0x7f0a5003b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:44:15.461 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:15.458+0000 7f0a6e59c700 1 --2- 172.21.15.152:0/2409185122 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f0a50038c20 0x7f0a5003b0e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0a64005b40 tx=0x7f0a64005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:44:15.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:15.462+0000 7f0a5f7fe700 1 -- 172.21.15.152:0/2409185122 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f0a6801d3b0 con 0x7f0a7019d290 2024-03-23T17:44:15.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:15 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:15.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:15 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:15.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:15 smithi086 ceph-mon[38221]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:44:15.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:15 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:44:15.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:15 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T17:44:15.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:15 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:15.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:15 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:15.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:15 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:15.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:15 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:15.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:15 smithi092 ceph-mon[79089]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:44:15.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:15 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:44:15.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:15 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T17:44:15.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:15 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:15.558 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:15 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:15.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:15 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:15.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:15 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:15.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:15 smithi152 ceph-mon[47949]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:44:15.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:15 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:44:15.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:15 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T17:44:15.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:15 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:15.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:15 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:15.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:15.769+0000 7f0a7489f700 1 -- 172.21.15.152:0/2409185122 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f0a70100e10 con 0x7f0a7019d290 2024-03-23T17:44:15.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:15.771+0000 7f0a5f7fe700 1 -- 172.21.15.152:0/2409185122 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1272 (secure 0 0 0) 0x7f0a680112a0 con 0x7f0a7019d290 2024-03-23T17:44:15.771 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-03-23T17:44:15.772 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":3,"fsid":"9d76c9e8-e93c-11ee-95cd-87774f69a715","modified":"2024-03-23T17:44:09.097471Z","created":"2024-03-23T17:42:26.888040Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:3300","nonce":0},{"type":"v1","addr":"172.21.15.86:6789","nonce":0}]},"addr":"172.21.15.86:6789/0","public_addr":"172.21.15.86:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:3300","nonce":0},{"type":"v1","addr":"172.21.15.152:6789","nonce":0}]},"addr":"172.21.15.152:6789/0","public_addr":"172.21.15.152:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:3300","nonce":0},{"type":"v1","addr":"172.21.15.92:6789","nonce":0}]},"addr":"172.21.15.92:6789/0","public_addr":"172.21.15.92:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-03-23T17:44:15.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:15.773+0000 7f0a7489f700 1 -- 172.21.15.152:0/2409185122 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f0a50038c20 msgr2=0x7f0a5003b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:44:15.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:15.773+0000 7f0a7489f700 1 --2- 172.21.15.152:0/2409185122 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f0a50038c20 0x7f0a5003b0e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0a64005b40 tx=0x7f0a64005a90 comp rx=0 tx=0).stop 2024-03-23T17:44:15.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:15.773+0000 7f0a7489f700 1 -- 172.21.15.152:0/2409185122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a7019d290 msgr2=0x7f0a7019d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:44:15.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:15.773+0000 7f0a7489f700 1 --2- 172.21.15.152:0/2409185122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a7019d290 0x7f0a7019d740 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0a6800bd50 tx=0x7f0a6800be30 comp rx=0 tx=0).stop 2024-03-23T17:44:15.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:15.773+0000 7f0a7489f700 1 -- 172.21.15.152:0/2409185122 shutdown_connections 2024-03-23T17:44:15.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:15.773+0000 7f0a7489f700 1 --2- 172.21.15.152:0/2409185122 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f0a50038c20 0x7f0a5003b0e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:44:15.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:15.773+0000 7f0a7489f700 1 --2- 172.21.15.152:0/2409185122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a7019d290 0x7f0a7019d740 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:44:15.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:15.773+0000 7f0a7489f700 1 --2- 172.21.15.152:0/2409185122 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0a580062a0 0x7f0a70198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:44:15.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:15.773+0000 7f0a7489f700 1 --2- 172.21.15.152:0/2409185122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a701989a0 0x7f0a7019cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:44:15.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:15.773+0000 7f0a7489f700 1 -- 172.21.15.152:0/2409185122 >> 172.21.15.152:0/2409185122 conn(0x7f0a700fb1c0 msgr2=0x7f0a700fc440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:44:15.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:15.773+0000 7f0a7489f700 1 -- 172.21.15.152:0/2409185122 shutdown_connections 2024-03-23T17:44:15.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:15.773+0000 7f0a7489f700 1 -- 172.21.15.152:0/2409185122 wait complete. 2024-03-23T17:44:15.775 INFO:teuthology.orchestra.run.smithi152.stderr:dumped monmap epoch 3 2024-03-23T17:44:16.154 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-03-23T17:44:16.155 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph config generate-minimal-conf 2024-03-23T17:44:16.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:16 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:16.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:16 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:16.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:16 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:44:16.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:16 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:44:16.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:16 smithi152 ceph-mon[47949]: Updating smithi086:/etc/ceph/ceph.conf 2024-03-23T17:44:16.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:16 smithi152 ceph-mon[47949]: Updating smithi092:/etc/ceph/ceph.conf 2024-03-23T17:44:16.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:16 smithi152 ceph-mon[47949]: Updating smithi152:/etc/ceph/ceph.conf 2024-03-23T17:44:16.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:16 smithi152 ceph-mon[47949]: Updating smithi152:/var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T17:44:16.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:16 smithi152 ceph-mon[47949]: Updating smithi092:/var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T17:44:16.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:16 smithi152 ceph-mon[47949]: Updating smithi086:/var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T17:44:16.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:16 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:16.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:16 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:16.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:16 smithi152 ceph-mon[47949]: from='client.? 172.21.15.152:0/2409185122' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-23T17:44:16.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:16 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:16.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:16 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:16.513 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T17:44:16.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:16 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:16.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:16 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:16.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:16 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:44:16.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:16 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:44:16.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:16 smithi086 ceph-mon[38221]: Updating smithi086:/etc/ceph/ceph.conf 2024-03-23T17:44:16.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:16 smithi086 ceph-mon[38221]: Updating smithi092:/etc/ceph/ceph.conf 2024-03-23T17:44:16.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:16 smithi086 ceph-mon[38221]: Updating smithi152:/etc/ceph/ceph.conf 2024-03-23T17:44:16.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:16 smithi086 ceph-mon[38221]: Updating smithi152:/var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T17:44:16.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:16 smithi086 ceph-mon[38221]: Updating smithi092:/var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T17:44:16.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:16 smithi086 ceph-mon[38221]: Updating smithi086:/var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T17:44:16.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:16 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:16.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:16 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:16.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:16 smithi086 ceph-mon[38221]: from='client.? 172.21.15.152:0/2409185122' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-23T17:44:16.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:16 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:16.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:16 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:16.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:16 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:16.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:16 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:16.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:16 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:44:16.558 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:16 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:44:16.558 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:16 smithi092 ceph-mon[79089]: Updating smithi086:/etc/ceph/ceph.conf 2024-03-23T17:44:16.558 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:16 smithi092 ceph-mon[79089]: Updating smithi092:/etc/ceph/ceph.conf 2024-03-23T17:44:16.558 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:16 smithi092 ceph-mon[79089]: Updating smithi152:/etc/ceph/ceph.conf 2024-03-23T17:44:16.558 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:16 smithi092 ceph-mon[79089]: Updating smithi152:/var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T17:44:16.558 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:16 smithi092 ceph-mon[79089]: Updating smithi092:/var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T17:44:16.558 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:16 smithi092 ceph-mon[79089]: Updating smithi086:/var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T17:44:16.558 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:16 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:16.558 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:16 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:16.558 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:16 smithi092 ceph-mon[79089]: from='client.? 172.21.15.152:0/2409185122' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-23T17:44:16.558 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:16 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:16.558 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:16 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:17.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:17 smithi092 ceph-mon[79089]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:44:17.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:17 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:17.558 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:17 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:17.558 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:17 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:17.558 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:17 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:17.558 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:17 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:17.558 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:17 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:17.558 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:17 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:17.558 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:17 smithi092 ceph-mon[79089]: Reconfiguring mon.a (unknown last config time)... 2024-03-23T17:44:17.558 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:17 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-23T17:44:17.558 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:17 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-23T17:44:17.558 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:17 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:44:17.558 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:17 smithi092 ceph-mon[79089]: Reconfiguring daemon mon.a on smithi086 2024-03-23T17:44:17.583 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:17 smithi086 ceph-mon[38221]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:44:17.583 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:17 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:17.583 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:17 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:17.583 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:17 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:17.583 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:17 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:17.583 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:17 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:17.583 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:17 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:17.583 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:17 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:17.583 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:17 smithi086 ceph-mon[38221]: Reconfiguring mon.a (unknown last config time)... 2024-03-23T17:44:17.583 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:17 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-23T17:44:17.584 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:17 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-23T17:44:17.584 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:17 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:44:17.584 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:17 smithi086 ceph-mon[38221]: Reconfiguring daemon mon.a on smithi086 2024-03-23T17:44:17.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:17 smithi152 ceph-mon[47949]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:44:17.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:17 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:17.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:17 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:17.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:17 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:17.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:17 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:17.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:17 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:17.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:17 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:17.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:17 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:17.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:17 smithi152 ceph-mon[47949]: Reconfiguring mon.a (unknown last config time)... 2024-03-23T17:44:17.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:17 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-23T17:44:17.587 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:17 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-23T17:44:17.587 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:17 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:44:17.587 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:17 smithi152 ceph-mon[47949]: Reconfiguring daemon mon.a on smithi086 2024-03-23T17:44:18.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:18.017+0000 7f0079200700 1 -- 172.21.15.86:0/3505410485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00741066c0 msgr2=0x7f0074106aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:44:18.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:18.017+0000 7f0079200700 1 --2- 172.21.15.86:0/3505410485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00741066c0 0x7f0074106aa0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0064009910 tx=0x7f0064009c20 comp rx=0 tx=0).stop 2024-03-23T17:44:18.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:18.017+0000 7f0079200700 1 -- 172.21.15.86:0/3505410485 shutdown_connections 2024-03-23T17:44:18.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:18.017+0000 7f0079200700 1 --2- 172.21.15.86:0/3505410485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00741066c0 0x7f0074106aa0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:44:18.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:18.017+0000 7f0079200700 1 -- 172.21.15.86:0/3505410485 >> 172.21.15.86:0/3505410485 conn(0x7f00740fc1b0 msgr2=0x7f00740fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:44:18.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:18.017+0000 7f0079200700 1 -- 172.21.15.86:0/3505410485 shutdown_connections 2024-03-23T17:44:18.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:18.017+0000 7f0079200700 1 -- 172.21.15.86:0/3505410485 wait complete. 2024-03-23T17:44:18.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:18.018+0000 7f0079200700 1 Processor -- start 2024-03-23T17:44:18.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:18.018+0000 7f0079200700 1 -- start start 2024-03-23T17:44:18.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:18.018+0000 7f0079200700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f00741066c0 0x7f00741982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:44:18.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:18.018+0000 7f0079200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0074198800 0x7f007419cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:44:18.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:18.018+0000 7f0079200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f007419d0f0 0x7f007419d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:44:18.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:18.018+0000 7f0079200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0074198ef0 con 0x7f0074198800 2024-03-23T17:44:18.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:18.019+0000 7f0079200700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00741a4e60 con 0x7f007419d0f0 2024-03-23T17:44:18.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:18.019+0000 7f0079200700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00741a4fa0 con 0x7f00741066c0 2024-03-23T17:44:18.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:18.019+0000 7f0071400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0074198800 0x7f007419cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:44:18.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:18.019+0000 7f0071400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0074198800 0x7f007419cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59688/0 (socket says 172.21.15.86:59688) 2024-03-23T17:44:18.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:18.019+0000 7f0071400700 1 -- 172.21.15.86:0/4127144466 learned_addr learned my addr 172.21.15.86:0/4127144466 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:44:18.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:18.019+0000 7f0072800700 1 --2- 172.21.15.86:0/4127144466 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f007419d0f0 0x7f007419d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:44:18.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:18.019+0000 7f0071e00700 1 --2- 172.21.15.86:0/4127144466 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f00741066c0 0x7f00741982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:44:18.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:18.019+0000 7f0071400700 1 -- 172.21.15.86:0/4127144466 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f00741066c0 msgr2=0x7f00741982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:44:18.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:18.019+0000 7f0071400700 1 --2- 172.21.15.86:0/4127144466 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f00741066c0 0x7f00741982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:44:18.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:18.019+0000 7f0071400700 1 -- 172.21.15.86:0/4127144466 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f007419d0f0 msgr2=0x7f007419d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:44:18.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:18.019+0000 7f0071400700 1 --2- 172.21.15.86:0/4127144466 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f007419d0f0 0x7f007419d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:44:18.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:18.019+0000 7f0071400700 1 -- 172.21.15.86:0/4127144466 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00640095f0 con 0x7f0074198800 2024-03-23T17:44:18.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:18.019+0000 7f0071400700 1 --2- 172.21.15.86:0/4127144466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0074198800 0x7f007419cbb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f006000b780 tx=0x7f006000ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:44:18.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:18.020+0000 7f006aa00700 1 -- 172.21.15.86:0/4127144466 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0060004230 con 0x7f0074198800 2024-03-23T17:44:18.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:18.020+0000 7f006aa00700 1 -- 172.21.15.86:0/4127144466 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0060004390 con 0x7f0074198800 2024-03-23T17:44:18.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:18.020+0000 7f0079200700 1 -- 172.21.15.86:0/4127144466 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00741a5280 con 0x7f0074198800 2024-03-23T17:44:18.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:18.020+0000 7f006aa00700 1 -- 172.21.15.86:0/4127144466 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0060011b90 con 0x7f0074198800 2024-03-23T17:44:18.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:18.020+0000 7f0079200700 1 -- 172.21.15.86:0/4127144466 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00741a57d0 con 0x7f0074198800 2024-03-23T17:44:18.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:18.021+0000 7f006aa00700 1 -- 172.21.15.86:0/4127144466 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) v1 ==== 45589+0+0 (secure 0 0 0) 0x7f0060011cf0 con 0x7f0074198800 2024-03-23T17:44:18.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:18.021+0000 7f0079200700 1 -- 172.21.15.86:0/4127144466 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0074101e00 con 0x7f0074198800 2024-03-23T17:44:18.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:18.021+0000 7f006aa00700 1 --2- 172.21.15.86:0/4127144466 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f0058038c70 0x7f005803b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:44:18.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:18.021+0000 7f006aa00700 1 -- 172.21.15.86:0/4127144466 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f006004a030 con 0x7f0074198800 2024-03-23T17:44:18.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:18.022+0000 7f0071e00700 1 --2- 172.21.15.86:0/4127144466 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f0058038c70 0x7f005803b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:44:18.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:18.022+0000 7f0071e00700 1 --2- 172.21.15.86:0/4127144466 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f0058038c70 0x7f005803b130 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0064004690 tx=0x7f0064005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:44:18.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:18.027+0000 7f006aa00700 1 -- 172.21.15.86:0/4127144466 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f0060003bf0 con 0x7f0074198800 2024-03-23T17:44:18.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:18.226+0000 7f0079200700 1 -- 172.21.15.86:0/4127144466 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f00740623e0 con 0x7f0074198800 2024-03-23T17:44:18.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:18.227+0000 7f006aa00700 1 -- 172.21.15.86:0/4127144466 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+273 (secure 0 0 0) 0x7f0060016030 con 0x7f0074198800 2024-03-23T17:44:18.228 INFO:teuthology.orchestra.run.smithi086.stdout:# minimal ceph.conf for 9d76c9e8-e93c-11ee-95cd-87774f69a715 2024-03-23T17:44:18.228 INFO:teuthology.orchestra.run.smithi086.stdout:[global] 2024-03-23T17:44:18.228 INFO:teuthology.orchestra.run.smithi086.stdout: fsid = 9d76c9e8-e93c-11ee-95cd-87774f69a715 2024-03-23T17:44:18.229 INFO:teuthology.orchestra.run.smithi086.stdout: mon_host = [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] 2024-03-23T17:44:18.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:18.229+0000 7f0079200700 1 -- 172.21.15.86:0/4127144466 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f0058038c70 msgr2=0x7f005803b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:44:18.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:18.229+0000 7f0079200700 1 --2- 172.21.15.86:0/4127144466 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f0058038c70 0x7f005803b130 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0064004690 tx=0x7f0064005fb0 comp rx=0 tx=0).stop 2024-03-23T17:44:18.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:18.229+0000 7f0079200700 1 -- 172.21.15.86:0/4127144466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0074198800 msgr2=0x7f007419cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:44:18.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:18.229+0000 7f0079200700 1 --2- 172.21.15.86:0/4127144466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0074198800 0x7f007419cbb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f006000b780 tx=0x7f006000ba90 comp rx=0 tx=0).stop 2024-03-23T17:44:18.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:18.229+0000 7f0079200700 1 -- 172.21.15.86:0/4127144466 shutdown_connections 2024-03-23T17:44:18.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:18.229+0000 7f0079200700 1 --2- 172.21.15.86:0/4127144466 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f007419d0f0 0x7f007419d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:44:18.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:18.229+0000 7f0079200700 1 --2- 172.21.15.86:0/4127144466 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f00741066c0 0x7f00741982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:44:18.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:18.229+0000 7f0079200700 1 --2- 172.21.15.86:0/4127144466 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f0058038c70 0x7f005803b130 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:44:18.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:18.229+0000 7f0079200700 1 --2- 172.21.15.86:0/4127144466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0074198800 0x7f007419cbb0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:44:18.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:18.229+0000 7f0079200700 1 -- 172.21.15.86:0/4127144466 >> 172.21.15.86:0/4127144466 conn(0x7f00740fc1b0 msgr2=0x7f00740737a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:44:18.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:18.229+0000 7f0079200700 1 -- 172.21.15.86:0/4127144466 shutdown_connections 2024-03-23T17:44:18.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:18.229+0000 7f0079200700 1 -- 172.21.15.86:0/4127144466 wait complete. 2024-03-23T17:44:18.605 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-03-23T17:44:18.606 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-23T17:44:18.606 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/ceph/ceph.conf 2024-03-23T17:44:18.673 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-23T17:44:18.673 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-23T17:44:18.779 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-03-23T17:44:18.779 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/ceph/ceph.conf 2024-03-23T17:44:18.811 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-03-23T17:44:18.811 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-23T17:44:18.879 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-03-23T17:44:18.879 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/etc/ceph/ceph.conf 2024-03-23T17:44:18.912 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-03-23T17:44:18.912 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-23T17:44:18.980 INFO:tasks.cephadm:Adding mgr.x on smithi086 2024-03-23T17:44:18.980 INFO:tasks.cephadm:Adding mgr.y on smithi092 2024-03-23T17:44:18.980 INFO:tasks.cephadm:Adding mgr.z on smithi152 2024-03-23T17:44:18.980 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph orch apply mgr '3;smithi086=x;smithi092=y;smithi152=z' 2024-03-23T17:44:19.159 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:18 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:19.159 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:18 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:19.159 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:18 smithi152 ceph-mon[47949]: Reconfiguring mon.b (monmap changed)... 2024-03-23T17:44:19.159 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:18 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-23T17:44:19.159 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:18 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-23T17:44:19.159 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:18 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:44:19.159 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:18 smithi152 ceph-mon[47949]: Reconfiguring daemon mon.b on smithi092 2024-03-23T17:44:19.159 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:18 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/4127144466' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:44:19.159 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:18 smithi152 ceph-mon[47949]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:44:19.160 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:18 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:19.160 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:18 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:19.160 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:18 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:19.160 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:18 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-23T17:44:19.160 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:18 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-23T17:44:19.160 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:18 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:44:19.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:18 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:19.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:18 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:19.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:18 smithi086 ceph-mon[38221]: Reconfiguring mon.b (monmap changed)... 2024-03-23T17:44:19.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:18 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-23T17:44:19.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:18 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-23T17:44:19.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:18 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:44:19.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:18 smithi086 ceph-mon[38221]: Reconfiguring daemon mon.b on smithi092 2024-03-23T17:44:19.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:18 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/4127144466' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:44:19.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:18 smithi086 ceph-mon[38221]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:44:19.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:18 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:19.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:18 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:19.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:18 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:19.268 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:18 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-23T17:44:19.268 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:18 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-23T17:44:19.268 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:18 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:44:19.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:18 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:19.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:18 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:19.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:18 smithi092 ceph-mon[79089]: Reconfiguring mon.b (monmap changed)... 2024-03-23T17:44:19.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:18 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-23T17:44:19.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:18 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-23T17:44:19.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:18 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:44:19.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:18 smithi092 ceph-mon[79089]: Reconfiguring daemon mon.b on smithi092 2024-03-23T17:44:19.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:18 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/4127144466' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:44:19.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:18 smithi092 ceph-mon[79089]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:44:19.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:18 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:19.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:18 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:19.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:18 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:19.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:18 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-23T17:44:19.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:18 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-23T17:44:19.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:18 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:44:19.343 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.c/config 2024-03-23T17:44:20.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:20.544+0000 7f57406eb700 1 -- 172.21.15.152:0/720753466 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f57380ffcf0 msgr2=0x7f5738100110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:44:20.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:20.544+0000 7f57406eb700 1 --2- 172.21.15.152:0/720753466 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f57380ffcf0 0x7f5738100110 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5728009990 tx=0x7f5728009ca0 comp rx=0 tx=0).stop 2024-03-23T17:44:20.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:20.544+0000 7f57406eb700 1 -- 172.21.15.152:0/720753466 shutdown_connections 2024-03-23T17:44:20.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:20.544+0000 7f57406eb700 1 --2- 172.21.15.152:0/720753466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5738101890 0x7f5738108140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:44:20.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:20.544+0000 7f57406eb700 1 --2- 172.21.15.152:0/720753466 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f57380ffcf0 0x7f5738100110 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:44:20.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:20.544+0000 7f57406eb700 1 --2- 172.21.15.152:0/720753466 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5738100ef0 0x7f5738101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:44:20.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:20.544+0000 7f57406eb700 1 -- 172.21.15.152:0/720753466 >> 172.21.15.152:0/720753466 conn(0x7f57380fb2b0 msgr2=0x7f57380fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:44:20.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:20.544+0000 7f57406eb700 1 -- 172.21.15.152:0/720753466 shutdown_connections 2024-03-23T17:44:20.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:20.544+0000 7f57406eb700 1 -- 172.21.15.152:0/720753466 wait complete. 2024-03-23T17:44:20.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:20.545+0000 7f57406eb700 1 Processor -- start 2024-03-23T17:44:20.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:20.545+0000 7f57406eb700 1 -- start start 2024-03-23T17:44:20.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:20.545+0000 7f57406eb700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f57380ffcf0 0x7f57380691c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:44:20.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:20.545+0000 7f57406eb700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5738100ef0 0x7f573819bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:44:20.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:20.545+0000 7f57406eb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5738101890 0x7f573819e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:44:20.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:20.545+0000 7f57406eb700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f573819e8c0 con 0x7f5738100ef0 2024-03-23T17:44:20.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:20.545+0000 7f57406eb700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f573819ea30 con 0x7f5738101890 2024-03-23T17:44:20.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:20.545+0000 7f57406eb700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f573819ebf0 con 0x7f57380ffcf0 2024-03-23T17:44:20.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:20.545+0000 7f573e487700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f57380ffcf0 0x7f57380691c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:44:20.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:20.545+0000 7f573e487700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f57380ffcf0 0x7f57380691c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:56666/0 (socket says 172.21.15.152:56666) 2024-03-23T17:44:20.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:20.545+0000 7f573e487700 1 -- 172.21.15.152:0/2313371903 learned_addr learned my addr 172.21.15.152:0/2313371903 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-03-23T17:44:20.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:20.545+0000 7f573ec88700 1 --2- 172.21.15.152:0/2313371903 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5738101890 0x7f573819e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:44:20.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:20.546+0000 7f573dc86700 1 --2- 172.21.15.152:0/2313371903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5738100ef0 0x7f573819bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:44:20.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:20.546+0000 7f573e487700 1 -- 172.21.15.152:0/2313371903 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5738101890 msgr2=0x7f573819e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:44:20.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:20.546+0000 7f573e487700 1 --2- 172.21.15.152:0/2313371903 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5738101890 0x7f573819e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:44:20.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:20.546+0000 7f573e487700 1 -- 172.21.15.152:0/2313371903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5738100ef0 msgr2=0x7f573819bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:44:20.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:20.546+0000 7f573e487700 1 --2- 172.21.15.152:0/2313371903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5738100ef0 0x7f573819bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:44:20.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:20.546+0000 7f573e487700 1 -- 172.21.15.152:0/2313371903 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57280095f0 con 0x7f57380ffcf0 2024-03-23T17:44:20.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:20.546+0000 7f573dc86700 1 --2- 172.21.15.152:0/2313371903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5738100ef0 0x7f573819bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T17:44:20.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:20.546+0000 7f573e487700 1 --2- 172.21.15.152:0/2313371903 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f57380ffcf0 0x7f57380691c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5728003790 tx=0x7f572800f670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:44:20.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:20.546+0000 7f572f7fe700 1 -- 172.21.15.152:0/2313371903 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f572801d070 con 0x7f57380ffcf0 2024-03-23T17:44:20.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:20.546+0000 7f572f7fe700 1 -- 172.21.15.152:0/2313371903 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f572800fc50 con 0x7f57380ffcf0 2024-03-23T17:44:20.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:20.546+0000 7f57406eb700 1 -- 172.21.15.152:0/2313371903 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5738195a80 con 0x7f57380ffcf0 2024-03-23T17:44:20.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:20.546+0000 7f572f7fe700 1 -- 172.21.15.152:0/2313371903 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5728017650 con 0x7f57380ffcf0 2024-03-23T17:44:20.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:20.546+0000 7f57406eb700 1 -- 172.21.15.152:0/2313371903 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5738195f70 con 0x7f57380ffcf0 2024-03-23T17:44:20.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:20.547+0000 7f57406eb700 1 -- 172.21.15.152:0/2313371903 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5738107220 con 0x7f57380ffcf0 2024-03-23T17:44:20.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:20.547+0000 7f572f7fe700 1 -- 172.21.15.152:0/2313371903 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 13) v1 ==== 45589+0+0 (secure 0 0 0) 0x7f5728022470 con 0x7f57380ffcf0 2024-03-23T17:44:20.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:20.547+0000 7f572f7fe700 1 --2- 172.21.15.152:0/2313371903 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f57240389d0 0x7f572403ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:44:20.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:20.548+0000 7f572f7fe700 1 -- 172.21.15.152:0/2313371903 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f572804cc40 con 0x7f57380ffcf0 2024-03-23T17:44:20.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:20.548+0000 7f573dc86700 1 --2- 172.21.15.152:0/2313371903 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f57240389d0 0x7f572403ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:44:20.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:20.548+0000 7f573dc86700 1 --2- 172.21.15.152:0/2313371903 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f57240389d0 0x7f572403ae90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5734005fd0 tx=0x7f5734005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:44:20.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:20.552+0000 7f572f7fe700 1 -- 172.21.15.152:0/2313371903 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f5728022720 con 0x7f57380ffcf0 2024-03-23T17:44:20.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:20.752+0000 7f57406eb700 1 -- 172.21.15.152:0/2313371903 --> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi086=x;smithi092=y;smithi152=z", "target": ["mon-mgr", ""]}) v1 -- 0x7f57380611f0 con 0x7f57240389d0 2024-03-23T17:44:20.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:20.759+0000 7f572f7fe700 1 -- 172.21.15.152:0/2313371903 <== mgr.14152 v2:172.21.15.86:6800/2734784244 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f57380611f0 con 0x7f57240389d0 2024-03-23T17:44:20.760 INFO:teuthology.orchestra.run.smithi152.stdout:Scheduled mgr update... 2024-03-23T17:44:20.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:20.762+0000 7f57406eb700 1 -- 172.21.15.152:0/2313371903 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f57240389d0 msgr2=0x7f572403ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:44:20.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:20.762+0000 7f57406eb700 1 --2- 172.21.15.152:0/2313371903 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f57240389d0 0x7f572403ae90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5734005fd0 tx=0x7f5734005dc0 comp rx=0 tx=0).stop 2024-03-23T17:44:20.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:20.762+0000 7f57406eb700 1 -- 172.21.15.152:0/2313371903 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f57380ffcf0 msgr2=0x7f57380691c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:44:20.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:20.762+0000 7f57406eb700 1 --2- 172.21.15.152:0/2313371903 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f57380ffcf0 0x7f57380691c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5728003790 tx=0x7f572800f670 comp rx=0 tx=0).stop 2024-03-23T17:44:20.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:20.762+0000 7f57406eb700 1 -- 172.21.15.152:0/2313371903 shutdown_connections 2024-03-23T17:44:20.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:20.762+0000 7f57406eb700 1 --2- 172.21.15.152:0/2313371903 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5738101890 0x7f573819e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:44:20.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:20.762+0000 7f57406eb700 1 --2- 172.21.15.152:0/2313371903 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f57380ffcf0 0x7f57380691c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:44:20.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:20.762+0000 7f57406eb700 1 --2- 172.21.15.152:0/2313371903 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f57240389d0 0x7f572403ae90 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:44:20.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:20.762+0000 7f57406eb700 1 --2- 172.21.15.152:0/2313371903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5738100ef0 0x7f573819bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:44:20.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:20.762+0000 7f57406eb700 1 -- 172.21.15.152:0/2313371903 >> 172.21.15.152:0/2313371903 conn(0x7f57380fb2b0 msgr2=0x7f5738104360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:44:20.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:20.762+0000 7f57406eb700 1 -- 172.21.15.152:0/2313371903 shutdown_connections 2024-03-23T17:44:20.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:44:20.762+0000 7f57406eb700 1 -- 172.21.15.152:0/2313371903 wait complete. 2024-03-23T17:44:21.044 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:20 smithi152 ceph-mon[47949]: Reconfiguring mon.c (monmap changed)... 2024-03-23T17:44:21.044 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:20 smithi152 ceph-mon[47949]: Reconfiguring daemon mon.c on smithi152 2024-03-23T17:44:21.045 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:20 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:21.045 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:20 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:21.045 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:20 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:44:21.045 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:20 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:44:21.045 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:20 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:44:21.045 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:20 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:21.045 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:20 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:21.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:20 smithi092 ceph-mon[79089]: Reconfiguring mon.c (monmap changed)... 2024-03-23T17:44:21.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:20 smithi092 ceph-mon[79089]: Reconfiguring daemon mon.c on smithi152 2024-03-23T17:44:21.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:20 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:21.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:20 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:21.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:20 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:44:21.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:20 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:44:21.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:20 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:44:21.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:20 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:21.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:20 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:21.150 DEBUG:teuthology.orchestra.run.smithi092:mgr.y> sudo journalctl -f -n 0 -u ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@mgr.y.service 2024-03-23T17:44:21.153 DEBUG:teuthology.orchestra.run.smithi152:mgr.z> sudo journalctl -f -n 0 -u ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@mgr.z.service 2024-03-23T17:44:21.155 INFO:tasks.cephadm:Deploying OSDs... 2024-03-23T17:44:21.155 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-23T17:44:21.155 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/scratch_devs of=/dev/stdout 2024-03-23T17:44:21.185 INFO:journalctl@ceph.mgr.z.smithi152.stdout:-- Logs begin at Sat 2024-03-23 17:19:34 UTC. -- 2024-03-23T17:44:21.187 INFO:journalctl@ceph.mgr.y.smithi092.stdout:-- Logs begin at Sat 2024-03-23 17:19:16 UTC. -- 2024-03-23T17:44:21.201 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-23T17:44:21.201 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_1 2024-03-23T17:44:21.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:20 smithi086 ceph-mon[38221]: Reconfiguring mon.c (monmap changed)... 2024-03-23T17:44:21.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:20 smithi086 ceph-mon[38221]: Reconfiguring daemon mon.c on smithi152 2024-03-23T17:44:21.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:20 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:21.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:20 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:21.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:20 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:44:21.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:20 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:44:21.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:20 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:44:21.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:20 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:21.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:20 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:21.287 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-23T17:44:21.287 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T17:44:21.287 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 5h/5d Inode: 452 Links: 1 2024-03-23T17:44:21.287 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T17:44:21.287 INFO:teuthology.orchestra.run.smithi086.stdout:Context: system_u:object_r:device_t:s0 2024-03-23T17:44:21.287 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-03-23 17:43:38.666717387 +0000 2024-03-23T17:44:21.287 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-03-23 17:36:24.484494576 +0000 2024-03-23T17:44:21.287 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-03-23 17:36:24.484494576 +0000 2024-03-23T17:44:21.288 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: 2024-03-23 17:36:24.456495163 +0000 2024-03-23T17:44:21.288 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-23T17:44:21.391 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-03-23T17:44:21.391 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-03-23T17:44:21.391 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000320448 s, 1.6 MB/s 2024-03-23T17:44:21.394 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-23T17:44:21.437 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_2 2024-03-23T17:44:21.520 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-23T17:44:21.521 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T17:44:21.521 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 5h/5d Inode: 458 Links: 1 2024-03-23T17:44:21.521 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T17:44:21.521 INFO:teuthology.orchestra.run.smithi086.stdout:Context: system_u:object_r:device_t:s0 2024-03-23T17:44:21.521 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-03-23 17:43:38.730716036 +0000 2024-03-23T17:44:21.521 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-03-23 17:36:24.945484917 +0000 2024-03-23T17:44:21.521 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-03-23 17:36:24.945484917 +0000 2024-03-23T17:44:21.521 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: 2024-03-23 17:36:24.922485398 +0000 2024-03-23T17:44:21.521 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-23T17:44:21.623 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-03-23T17:44:21.623 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-03-23T17:44:21.623 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000264664 s, 1.9 MB/s 2024-03-23T17:44:21.626 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-23T17:44:21.668 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_3 2024-03-23T17:44:21.751 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-23T17:44:21.751 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T17:44:21.751 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 5h/5d Inode: 464 Links: 1 2024-03-23T17:44:21.751 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T17:44:21.751 INFO:teuthology.orchestra.run.smithi086.stdout:Context: system_u:object_r:device_t:s0 2024-03-23T17:44:21.751 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-03-23 17:43:38.797714622 +0000 2024-03-23T17:44:21.751 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-03-23 17:36:25.414475089 +0000 2024-03-23T17:44:21.752 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-03-23 17:36:25.414475089 +0000 2024-03-23T17:44:21.752 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: 2024-03-23 17:36:25.392475550 +0000 2024-03-23T17:44:21.752 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-23T17:44:21.853 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-03-23T17:44:21.853 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-03-23T17:44:21.853 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000262192 s, 2.0 MB/s 2024-03-23T17:44:21.855 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-23T17:44:21.898 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_4 2024-03-23T17:44:21.981 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-23T17:44:21.981 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T17:44:21.981 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 5h/5d Inode: 470 Links: 1 2024-03-23T17:44:21.982 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T17:44:21.982 INFO:teuthology.orchestra.run.smithi086.stdout:Context: system_u:object_r:device_t:s0 2024-03-23T17:44:21.982 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-03-23 17:43:38.862713250 +0000 2024-03-23T17:44:21.982 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-03-23 17:36:25.889465136 +0000 2024-03-23T17:44:21.982 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-03-23 17:36:25.889465136 +0000 2024-03-23T17:44:21.982 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: 2024-03-23 17:36:25.866465618 +0000 2024-03-23T17:44:21.982 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-23T17:44:21.987 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:21 smithi152 ceph-mon[47949]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:44:21.987 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:21 smithi152 ceph-mon[47949]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi086=x;smithi092=y;smithi152=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:44:21.988 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:21 smithi152 ceph-mon[47949]: Saving service mgr spec with placement smithi086=x;smithi092=y;smithi152=z;count:3 2024-03-23T17:44:21.988 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:21 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:44:21.988 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:21 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:44:21.988 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:21 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:44:21.988 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:21 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:21.988 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:21 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-23T17:44:21.988 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:21 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-23T17:44:21.988 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:21 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-23T17:44:21.988 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:21 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:44:22.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:21 smithi092 ceph-mon[79089]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:44:22.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:21 smithi092 ceph-mon[79089]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi086=x;smithi092=y;smithi152=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:44:22.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:21 smithi092 ceph-mon[79089]: Saving service mgr spec with placement smithi086=x;smithi092=y;smithi152=z;count:3 2024-03-23T17:44:22.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:21 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:44:22.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:21 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:44:22.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:21 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:44:22.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:21 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:22.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:21 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-23T17:44:22.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:21 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-23T17:44:22.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:21 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-23T17:44:22.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:21 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:44:22.078 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:21 smithi086 ceph-mon[38221]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:44:22.078 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:21 smithi086 ceph-mon[38221]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi086=x;smithi092=y;smithi152=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:44:22.078 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:21 smithi086 ceph-mon[38221]: Saving service mgr spec with placement smithi086=x;smithi092=y;smithi152=z;count:3 2024-03-23T17:44:22.078 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:21 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:44:22.078 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:21 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:44:22.078 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:21 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:44:22.079 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:21 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:22.079 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:21 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-23T17:44:22.079 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:21 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-23T17:44:22.079 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:21 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-23T17:44:22.079 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:21 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:44:22.084 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-03-23T17:44:22.084 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-03-23T17:44:22.084 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000292669 s, 1.7 MB/s 2024-03-23T17:44:22.087 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-23T17:44:22.129 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-03-23T17:44:22.130 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/scratch_devs of=/dev/stdout 2024-03-23T17:44:22.151 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-23T17:44:22.152 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_1 2024-03-23T17:44:22.213 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-23T17:44:22.213 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T17:44:22.213 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 6h/6d Inode: 205608 Links: 1 2024-03-23T17:44:22.213 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T17:44:22.214 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2024-03-23T17:44:22.214 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-03-23 17:44:08.684573744 +0000 2024-03-23T17:44:22.214 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-03-23 17:35:12.302936544 +0000 2024-03-23T17:44:22.214 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-03-23 17:35:12.302936544 +0000 2024-03-23T17:44:22.214 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: - 2024-03-23T17:44:22.214 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-23T17:44:22.285 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-03-23T17:44:22.285 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-03-23T17:44:22.285 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000266896 s, 1.9 MB/s 2024-03-23T17:44:22.286 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-23T17:44:22.350 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_2 2024-03-23T17:44:22.413 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-23T17:44:22.413 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T17:44:22.413 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 6h/6d Inode: 210579 Links: 1 2024-03-23T17:44:22.413 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T17:44:22.413 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2024-03-23T17:44:22.413 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-03-23 17:44:08.684573744 +0000 2024-03-23T17:44:22.413 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-03-23 17:35:12.643926718 +0000 2024-03-23T17:44:22.413 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-03-23 17:35:12.643926718 +0000 2024-03-23T17:44:22.413 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: - 2024-03-23T17:44:22.414 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-23T17:44:22.483 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-03-23T17:44:22.483 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-03-23T17:44:22.483 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000281322 s, 1.8 MB/s 2024-03-23T17:44:22.484 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-23T17:44:22.545 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_3 2024-03-23T17:44:22.587 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Mar 23 17:44:22 smithi152 bash[49791]: 37a09083e9be4f8213a3919c12a0fa8eb2f202a9eb20ce4b6fa4ab0b81b598d6 2024-03-23T17:44:22.587 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Mar 23 17:44:22 smithi152 systemd[1]: Started Ceph mgr.z for 9d76c9e8-e93c-11ee-95cd-87774f69a715. 2024-03-23T17:44:22.604 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-23T17:44:22.604 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T17:44:22.604 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 6h/6d Inode: 210693 Links: 1 2024-03-23T17:44:22.604 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T17:44:22.604 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2024-03-23T17:44:22.604 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-03-23 17:44:08.684573744 +0000 2024-03-23T17:44:22.605 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-03-23 17:35:12.974917179 +0000 2024-03-23T17:44:22.605 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-03-23 17:35:12.974917179 +0000 2024-03-23T17:44:22.605 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: - 2024-03-23T17:44:22.605 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-23T17:44:22.671 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-03-23T17:44:22.671 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-03-23T17:44:22.671 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000198566 s, 2.6 MB/s 2024-03-23T17:44:22.672 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-23T17:44:22.730 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_4 2024-03-23T17:44:22.788 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-23T17:44:22.789 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T17:44:22.789 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 6h/6d Inode: 205775 Links: 1 2024-03-23T17:44:22.789 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T17:44:22.789 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2024-03-23T17:44:22.789 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-03-23 17:44:08.684573744 +0000 2024-03-23T17:44:22.789 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-03-23 17:35:13.361906028 +0000 2024-03-23T17:44:22.789 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-03-23 17:35:13.361906028 +0000 2024-03-23T17:44:22.789 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: - 2024-03-23T17:44:22.789 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-23T17:44:22.856 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-03-23T17:44:22.856 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-03-23T17:44:22.856 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000245342 s, 2.1 MB/s 2024-03-23T17:44:22.857 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-23T17:44:22.916 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-03-23T17:44:22.916 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/scratch_devs of=/dev/stdout 2024-03-23T17:44:22.934 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-23T17:44:22.934 DEBUG:teuthology.orchestra.run.smithi152:> stat /dev/vg_nvme/lv_1 2024-03-23T17:44:22.945 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Mar 23 17:44:22 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-z[49806]: 2024-03-23T17:44:22.851+0000 7fd538928200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-23T17:44:22.945 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:22 smithi152 ceph-mon[47949]: Deploying daemon mgr.z on smithi152 2024-03-23T17:44:22.946 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:22 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:22.946 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:22 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:22.946 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:22 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:22.946 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:22 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-23T17:44:22.946 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:22 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-23T17:44:22.946 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:22 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-23T17:44:22.946 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:22 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:44:22.963 INFO:teuthology.orchestra.run.smithi152.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-23T17:44:22.963 INFO:teuthology.orchestra.run.smithi152.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T17:44:22.963 INFO:teuthology.orchestra.run.smithi152.stdout:Device: 6h/6d Inode: 177678 Links: 1 2024-03-23T17:44:22.963 INFO:teuthology.orchestra.run.smithi152.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T17:44:22.963 INFO:teuthology.orchestra.run.smithi152.stdout:Context: system_u:object_r:device_t:s0 2024-03-23T17:44:22.963 INFO:teuthology.orchestra.run.smithi152.stdout:Access: 2024-03-23 17:44:22.035573881 +0000 2024-03-23T17:44:22.964 INFO:teuthology.orchestra.run.smithi152.stdout:Modify: 2024-03-23 17:35:15.974382981 +0000 2024-03-23T17:44:22.964 INFO:teuthology.orchestra.run.smithi152.stdout:Change: 2024-03-23 17:35:15.974382981 +0000 2024-03-23T17:44:22.964 INFO:teuthology.orchestra.run.smithi152.stdout: Birth: - 2024-03-23T17:44:22.964 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-23T17:44:23.006 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:22 smithi092 ceph-mon[79089]: Deploying daemon mgr.z on smithi152 2024-03-23T17:44:23.006 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:22 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:23.006 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:22 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:23.006 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:22 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:23.007 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:22 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-23T17:44:23.007 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:22 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-23T17:44:23.007 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:22 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-23T17:44:23.007 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:22 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:44:23.030 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Mar 23 17:44:22 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-z[49806]: 2024-03-23T17:44:22.944+0000 7fd538928200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-23T17:44:23.033 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records in 2024-03-23T17:44:23.033 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records out 2024-03-23T17:44:23.033 INFO:teuthology.orchestra.run.smithi152.stderr:512 bytes copied, 0.000200336 s, 2.6 MB/s 2024-03-23T17:44:23.034 DEBUG:teuthology.orchestra.run.smithi152:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-23T17:44:23.092 DEBUG:teuthology.orchestra.run.smithi152:> stat /dev/vg_nvme/lv_2 2024-03-23T17:44:23.150 INFO:teuthology.orchestra.run.smithi152.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-23T17:44:23.151 INFO:teuthology.orchestra.run.smithi152.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T17:44:23.151 INFO:teuthology.orchestra.run.smithi152.stdout:Device: 6h/6d Inode: 181541 Links: 1 2024-03-23T17:44:23.151 INFO:teuthology.orchestra.run.smithi152.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T17:44:23.151 INFO:teuthology.orchestra.run.smithi152.stdout:Context: system_u:object_r:device_t:s0 2024-03-23T17:44:23.151 INFO:teuthology.orchestra.run.smithi152.stdout:Access: 2024-03-23 17:44:22.035573881 +0000 2024-03-23T17:44:23.151 INFO:teuthology.orchestra.run.smithi152.stdout:Modify: 2024-03-23 17:35:16.349378818 +0000 2024-03-23T17:44:23.151 INFO:teuthology.orchestra.run.smithi152.stdout:Change: 2024-03-23 17:35:16.349378818 +0000 2024-03-23T17:44:23.151 INFO:teuthology.orchestra.run.smithi152.stdout: Birth: - 2024-03-23T17:44:23.151 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-23T17:44:23.214 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Mar 23 17:44:23 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-z[49806]: 2024-03-23T17:44:23.037+0000 7fd538928200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-23T17:44:23.214 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Mar 23 17:44:23 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-z[49806]: 2024-03-23T17:44:23.124+0000 7fd538928200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-23T17:44:23.216 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records in 2024-03-23T17:44:23.216 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records out 2024-03-23T17:44:23.216 INFO:teuthology.orchestra.run.smithi152.stderr:512 bytes copied, 0.000209062 s, 2.4 MB/s 2024-03-23T17:44:23.217 DEBUG:teuthology.orchestra.run.smithi152:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-23T17:44:23.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:22 smithi086 ceph-mon[38221]: Deploying daemon mgr.z on smithi152 2024-03-23T17:44:23.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:22 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:23.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:22 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:23.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:22 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:23.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:22 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-23T17:44:23.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:22 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-23T17:44:23.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:22 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-23T17:44:23.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:22 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:44:23.275 DEBUG:teuthology.orchestra.run.smithi152:> stat /dev/vg_nvme/lv_3 2024-03-23T17:44:23.332 INFO:teuthology.orchestra.run.smithi152.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-23T17:44:23.333 INFO:teuthology.orchestra.run.smithi152.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T17:44:23.333 INFO:teuthology.orchestra.run.smithi152.stdout:Device: 6h/6d Inode: 179790 Links: 1 2024-03-23T17:44:23.333 INFO:teuthology.orchestra.run.smithi152.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T17:44:23.333 INFO:teuthology.orchestra.run.smithi152.stdout:Context: system_u:object_r:device_t:s0 2024-03-23T17:44:23.333 INFO:teuthology.orchestra.run.smithi152.stdout:Access: 2024-03-23 17:44:22.035573881 +0000 2024-03-23T17:44:23.333 INFO:teuthology.orchestra.run.smithi152.stdout:Modify: 2024-03-23 17:35:16.689375042 +0000 2024-03-23T17:44:23.333 INFO:teuthology.orchestra.run.smithi152.stdout:Change: 2024-03-23 17:35:16.689375042 +0000 2024-03-23T17:44:23.333 INFO:teuthology.orchestra.run.smithi152.stdout: Birth: - 2024-03-23T17:44:23.333 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-23T17:44:23.398 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records in 2024-03-23T17:44:23.398 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records out 2024-03-23T17:44:23.398 INFO:teuthology.orchestra.run.smithi152.stderr:512 bytes copied, 0.000205763 s, 2.5 MB/s 2024-03-23T17:44:23.399 DEBUG:teuthology.orchestra.run.smithi152:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-23T17:44:23.458 DEBUG:teuthology.orchestra.run.smithi152:> stat /dev/vg_nvme/lv_4 2024-03-23T17:44:23.516 INFO:teuthology.orchestra.run.smithi152.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-23T17:44:23.517 INFO:teuthology.orchestra.run.smithi152.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T17:44:23.517 INFO:teuthology.orchestra.run.smithi152.stdout:Device: 6h/6d Inode: 177798 Links: 1 2024-03-23T17:44:23.517 INFO:teuthology.orchestra.run.smithi152.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T17:44:23.517 INFO:teuthology.orchestra.run.smithi152.stdout:Context: system_u:object_r:device_t:s0 2024-03-23T17:44:23.517 INFO:teuthology.orchestra.run.smithi152.stdout:Access: 2024-03-23 17:44:22.035573881 +0000 2024-03-23T17:44:23.517 INFO:teuthology.orchestra.run.smithi152.stdout:Modify: 2024-03-23 17:35:17.048371057 +0000 2024-03-23T17:44:23.517 INFO:teuthology.orchestra.run.smithi152.stdout:Change: 2024-03-23 17:35:17.048371057 +0000 2024-03-23T17:44:23.517 INFO:teuthology.orchestra.run.smithi152.stdout: Birth: - 2024-03-23T17:44:23.517 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-23T17:44:23.590 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records in 2024-03-23T17:44:23.591 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records out 2024-03-23T17:44:23.591 INFO:teuthology.orchestra.run.smithi152.stderr:512 bytes copied, 0.00023177 s, 2.2 MB/s 2024-03-23T17:44:23.592 DEBUG:teuthology.orchestra.run.smithi152:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-23T17:44:23.650 INFO:tasks.cephadm:Deploying osd.0 on smithi086 with /dev/vg_nvme/lv_4... 2024-03-23T17:44:23.650 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-23T17:44:24.001 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T17:44:24.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:23 smithi086 ceph-mon[38221]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:44:24.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:23 smithi086 ceph-mon[38221]: Deploying daemon mgr.y on smithi092 2024-03-23T17:44:24.030 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:23 smithi092 ceph-mon[79089]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:44:24.030 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:23 smithi092 ceph-mon[79089]: Deploying daemon mgr.y on smithi092 2024-03-23T17:44:24.031 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Mar 23 17:44:23 smithi092 systemd[1]: Starting Ceph mgr.y for 9d76c9e8-e93c-11ee-95cd-87774f69a715... 2024-03-23T17:44:24.032 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Mar 23 17:44:23 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-z[49806]: 2024-03-23T17:44:23.783+0000 7fd538928200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-23T17:44:24.033 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:23 smithi152 ceph-mon[47949]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:44:24.033 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:23 smithi152 ceph-mon[47949]: Deploying daemon mgr.y on smithi092 2024-03-23T17:44:24.307 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Mar 23 17:44:24 smithi092 ceph-mgr[80588]: -- 172.21.15.92:0/2325796077 <== mon.2 v2:172.21.15.92:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5633daae01a0 con 0x5633daab6800 2024-03-23T17:44:24.307 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Mar 23 17:44:24 smithi092 bash[80550]: f220bcd9b7cb33dd1f38677a87140a6e45b58cda020785f6c409383845d55d5b 2024-03-23T17:44:24.307 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Mar 23 17:44:24 smithi092 systemd[1]: Started Ceph mgr.y for 9d76c9e8-e93c-11ee-95cd-87774f69a715. 2024-03-23T17:44:24.336 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Mar 23 17:44:24 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-z[49806]: 2024-03-23T17:44:24.032+0000 7fd538928200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-23T17:44:24.336 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Mar 23 17:44:24 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-z[49806]: 2024-03-23T17:44:24.117+0000 7fd538928200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-23T17:44:24.336 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Mar 23 17:44:24 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-z[49806]: 2024-03-23T17:44:24.226+0000 7fd538928200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-23T17:44:24.694 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Mar 23 17:44:24 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-y[80565]: 2024-03-23T17:44:24.597+0000 7fc937591200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-23T17:44:24.836 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Mar 23 17:44:24 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-z[49806]: 2024-03-23T17:44:24.386+0000 7fd538928200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-23T17:44:24.836 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Mar 23 17:44:24 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-z[49806]: 2024-03-23T17:44:24.515+0000 7fd538928200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-23T17:44:24.836 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Mar 23 17:44:24 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-z[49806]: 2024-03-23T17:44:24.607+0000 7fd538928200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-23T17:44:24.887 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Mar 23 17:44:24 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-y[80565]: 2024-03-23T17:44:24.693+0000 7fc937591200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-23T17:44:24.888 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Mar 23 17:44:24 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-y[80565]: 2024-03-23T17:44:24.794+0000 7fc937591200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-23T17:44:24.888 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Mar 23 17:44:24 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-y[80565]: 2024-03-23T17:44:24.887+0000 7fc937591200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-23T17:44:25.266 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Mar 23 17:44:25 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-z[49806]: 2024-03-23T17:44:25.014+0000 7fd538928200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-23T17:44:25.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:25 smithi086 ceph-mon[38221]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:44:25.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:25 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:25.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:25 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:25.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:25 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:25.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:25 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:25.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:25 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:44:25.552 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Mar 23 17:44:25 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-y[80565]: 2024-03-23T17:44:25.551+0000 7fc937591200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-23T17:44:25.552 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:25 smithi092 ceph-mon[79089]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:44:25.552 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:25 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:25.552 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:25 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:25.553 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:25 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:25.553 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:25 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:25.553 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:25 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:44:25.586 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Mar 23 17:44:25 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-z[49806]: 2024-03-23T17:44:25.423+0000 7fd538928200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-23T17:44:25.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:25 smithi152 ceph-mon[47949]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:44:25.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:25 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:25.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:25 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:25.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:25 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:25.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:25 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:25.587 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:25 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:44:25.660 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-23T17:44:25.661 INFO:teuthology.orchestra.run.smithi086.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-23T17:44:25.661 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 10+0 records in 2024-03-23T17:44:25.661 INFO:teuthology.orchestra.run.smithi086.stderr:10+0 records out 2024-03-23T17:44:25.661 INFO:teuthology.orchestra.run.smithi086.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0315316 s, 333 MB/s 2024-03-23T17:44:25.661 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping successful for: 2024-03-23T17:44:25.807 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Mar 23 17:44:25 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-y[80565]: 2024-03-23T17:44:25.797+0000 7fc937591200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-23T17:44:26.016 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi086:vg_nvme/lv_4 2024-03-23T17:44:26.086 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Mar 23 17:44:25 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-z[49806]: 2024-03-23T17:44:25.620+0000 7fd538928200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-23T17:44:26.086 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Mar 23 17:44:25 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-z[49806]: 2024-03-23T17:44:25.712+0000 7fd538928200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-23T17:44:26.144 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Mar 23 17:44:25 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-y[80565]: 2024-03-23T17:44:25.881+0000 7fc937591200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-23T17:44:26.144 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Mar 23 17:44:25 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-y[80565]: 2024-03-23T17:44:25.989+0000 7fc937591200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-23T17:44:26.144 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Mar 23 17:44:26 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-y[80565]: 2024-03-23T17:44:26.143+0000 7fc937591200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-23T17:44:26.557 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Mar 23 17:44:26 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-y[80565]: 2024-03-23T17:44:26.269+0000 7fc937591200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-23T17:44:26.557 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Mar 23 17:44:26 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-y[80565]: 2024-03-23T17:44:26.355+0000 7fc937591200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-23T17:44:26.558 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:26 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:26.558 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:26 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:26.558 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:26 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:26.558 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:26 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:26.558 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:26 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:44:26.558 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:26 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:44:26.558 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:26 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:26.558 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:26 smithi092 ceph-mon[79089]: Reconfiguring mgr.x (unknown last config time)... 2024-03-23T17:44:26.558 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:26 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-23T17:44:26.558 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:26 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-23T17:44:26.558 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:26 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:44:26.559 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:26 smithi092 ceph-mon[79089]: Reconfiguring daemon mgr.x on smithi086 2024-03-23T17:44:26.567 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Mar 23 17:44:26 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-z[49806]: 2024-03-23T17:44:26.469+0000 7fd538928200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-23T17:44:26.567 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Mar 23 17:44:26 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-z[49806]: 2024-03-23T17:44:26.566+0000 7fd538928200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-23T17:44:26.567 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:26 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:26.567 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:26 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:26.567 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:26 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:26.567 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:26 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:26.567 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:26 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:44:26.567 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:26 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:44:26.568 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:26 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:26.568 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:26 smithi152 ceph-mon[47949]: Reconfiguring mgr.x (unknown last config time)... 2024-03-23T17:44:26.568 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:26 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-23T17:44:26.568 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:26 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-23T17:44:26.568 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:26 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:44:26.568 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:26 smithi152 ceph-mon[47949]: Reconfiguring daemon mgr.x on smithi086 2024-03-23T17:44:26.634 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:26 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:26.635 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:26 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:26.635 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:26 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:26.635 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:26 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:26.635 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:26 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:44:26.635 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:26 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:44:26.635 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:26 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:26.635 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:26 smithi086 ceph-mon[38221]: Reconfiguring mgr.x (unknown last config time)... 2024-03-23T17:44:26.635 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:26 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-23T17:44:26.635 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:26 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-23T17:44:26.635 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:26 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:44:26.635 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:26 smithi086 ceph-mon[38221]: Reconfiguring daemon mgr.x on smithi086 2024-03-23T17:44:26.678 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T17:44:26.836 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Mar 23 17:44:26 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-z[49806]: 2024-03-23T17:44:26.714+0000 7fd538928200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-23T17:44:27.057 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Mar 23 17:44:26 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-y[80565]: 2024-03-23T17:44:26.724+0000 7fc937591200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-23T17:44:27.392 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Mar 23 17:44:27 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-y[80565]: 2024-03-23T17:44:27.110+0000 7fc937591200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-23T17:44:27.392 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Mar 23 17:44:27 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-y[80565]: 2024-03-23T17:44:27.300+0000 7fc937591200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-23T17:44:27.392 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Mar 23 17:44:27 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-y[80565]: 2024-03-23T17:44:27.391+0000 7fc937591200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-23T17:44:27.565 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:27 smithi086 ceph-mon[38221]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:44:27.565 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:27 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:27.565 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:27 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:27.565 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:27 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:44:27.565 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:27 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:44:27.565 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:27 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:44:27.565 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:27 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:27.718 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Mar 23 17:44:27 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-z[49806]: 2024-03-23T17:44:27.343+0000 7fd538928200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-23T17:44:27.718 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Mar 23 17:44:27 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-z[49806]: 2024-03-23T17:44:27.524+0000 7fd538928200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-23T17:44:27.718 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Mar 23 17:44:27 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-z[49806]: 2024-03-23T17:44:27.717+0000 7fd538928200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-23T17:44:27.718 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:27 smithi152 ceph-mon[47949]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:44:27.718 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:27 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:27.718 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:27 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:27.718 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:27 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:44:27.719 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:27 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:44:27.719 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:27 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:44:27.719 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:27 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:27.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:27.799+0000 7fb606e00700 1 -- 172.21.15.86:0/1497555785 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb600101f60 msgr2=0x7fb6001023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:44:27.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:27.799+0000 7fb606e00700 1 --2- 172.21.15.86:0/1497555785 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb600101f60 0x7fb6001023e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec009990 tx=0x7fb5ec009ca0 comp rx=0 tx=0).stop 2024-03-23T17:44:27.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:27.799+0000 7fb606e00700 1 -- 172.21.15.86:0/1497555785 shutdown_connections 2024-03-23T17:44:27.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:27.799+0000 7fb606e00700 1 --2- 172.21.15.86:0/1497555785 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb600102920 0x7fb600109240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:44:27.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:27.799+0000 7fb606e00700 1 --2- 172.21.15.86:0/1497555785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb600100e00 0x7fb600101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:44:27.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:27.799+0000 7fb606e00700 1 --2- 172.21.15.86:0/1497555785 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb600101f60 0x7fb6001023e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:44:27.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:27.799+0000 7fb606e00700 1 -- 172.21.15.86:0/1497555785 >> 172.21.15.86:0/1497555785 conn(0x7fb6000fc3c0 msgr2=0x7fb6000fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:44:27.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:27.799+0000 7fb606e00700 1 -- 172.21.15.86:0/1497555785 shutdown_connections 2024-03-23T17:44:27.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:27.799+0000 7fb606e00700 1 -- 172.21.15.86:0/1497555785 wait complete. 2024-03-23T17:44:27.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:27.800+0000 7fb606e00700 1 Processor -- start 2024-03-23T17:44:27.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:27.800+0000 7fb606e00700 1 -- start start 2024-03-23T17:44:27.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:27.801+0000 7fb606e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb600100e00 0x7fb600074b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:44:27.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:27.801+0000 7fb606e00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb600101f60 0x7fb6000731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:44:27.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:27.801+0000 7fb606e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb600102920 0x7fb6000736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:44:27.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:27.801+0000 7fb606e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb600073e80 con 0x7fb600102920 2024-03-23T17:44:27.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:27.801+0000 7fb606e00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb600073ff0 con 0x7fb600100e00 2024-03-23T17:44:27.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:27.801+0000 7fb606e00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6001a0da0 con 0x7fb600101f60 2024-03-23T17:44:27.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:27.801+0000 7fb5ff400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb600101f60 0x7fb6000731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:44:27.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:27.801+0000 7fb5ffe00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb600100e00 0x7fb600074b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:44:27.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:27.801+0000 7fb604e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb600102920 0x7fb6000736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:44:27.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:27.801+0000 7fb5ffe00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb600100e00 0x7fb600074b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:48410/0 (socket says 172.21.15.86:48410) 2024-03-23T17:44:27.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:27.801+0000 7fb5ff400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb600101f60 0x7fb6000731a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:56878/0 (socket says 172.21.15.86:56878) 2024-03-23T17:44:27.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:27.801+0000 7fb5ffe00700 1 -- 172.21.15.86:0/2449685400 learned_addr learned my addr 172.21.15.86:0/2449685400 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:44:27.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:27.801+0000 7fb5ffe00700 1 -- 172.21.15.86:0/2449685400 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb600101f60 msgr2=0x7fb6000731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:44:27.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:27.801+0000 7fb5ffe00700 1 --2- 172.21.15.86:0/2449685400 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb600101f60 0x7fb6000731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:44:27.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:27.801+0000 7fb5ffe00700 1 -- 172.21.15.86:0/2449685400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb600102920 msgr2=0x7fb6000736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:44:27.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:27.801+0000 7fb5ffe00700 1 --2- 172.21.15.86:0/2449685400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb600102920 0x7fb6000736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:44:27.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:27.801+0000 7fb5ffe00700 1 -- 172.21.15.86:0/2449685400 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5ec0095f0 con 0x7fb600100e00 2024-03-23T17:44:27.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:27.802+0000 7fb5ffe00700 1 --2- 172.21.15.86:0/2449685400 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb600100e00 0x7fb600074b00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb5f000e9a0 tx=0x7fb5f000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:44:27.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:27.802+0000 7fb5fcc00700 1 -- 172.21.15.86:0/2449685400 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f0009790 con 0x7fb600100e00 2024-03-23T17:44:27.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:27.802+0000 7fb5fcc00700 1 -- 172.21.15.86:0/2449685400 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb5f0010430 con 0x7fb600100e00 2024-03-23T17:44:27.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:27.802+0000 7fb606e00700 1 -- 172.21.15.86:0/2449685400 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6001a0fd0 con 0x7fb600100e00 2024-03-23T17:44:27.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:27.802+0000 7fb5fcc00700 1 -- 172.21.15.86:0/2449685400 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f0018680 con 0x7fb600100e00 2024-03-23T17:44:27.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:27.802+0000 7fb606e00700 1 -- 172.21.15.86:0/2449685400 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6001a1490 con 0x7fb600100e00 2024-03-23T17:44:27.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:27.803+0000 7fb5fcc00700 1 -- 172.21.15.86:0/2449685400 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 13) v1 ==== 45589+0+0 (secure 0 0 0) 0x7fb5f00187e0 con 0x7fb600100e00 2024-03-23T17:44:27.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:27.803+0000 7fb606e00700 1 -- 172.21.15.86:0/2449685400 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb600108280 con 0x7fb600100e00 2024-03-23T17:44:27.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:27.803+0000 7fb5fcc00700 1 --2- 172.21.15.86:0/2449685400 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fb5e40389d0 0x7fb5e403ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:44:27.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:27.803+0000 7fb5fcc00700 1 -- 172.21.15.86:0/2449685400 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb5f0014070 con 0x7fb600100e00 2024-03-23T17:44:27.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:27.804+0000 7fb5ff400700 1 --2- 172.21.15.86:0/2449685400 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fb5e40389d0 0x7fb5e403ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:44:27.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:27.804+0000 7fb5ff400700 1 --2- 172.21.15.86:0/2449685400 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fb5e40389d0 0x7fb5e403ae90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec00b2e0 tx=0x7fb5ec005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:44:27.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:27 smithi092 ceph-mon[79089]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:44:27.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:27 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:27.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:27 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:27.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:27 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:44:27.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:27 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:44:27.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:27 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:44:27.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:27 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:27.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:27.809+0000 7fb5fcc00700 1 -- 172.21.15.86:0/2449685400 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fb5f001f900 con 0x7fb600100e00 2024-03-23T17:44:28.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:28.020+0000 7fb606e00700 1 -- 172.21.15.86:0/2449685400 --> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fb60002cdc0 con 0x7fb5e40389d0 2024-03-23T17:44:28.516 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Mar 23 17:44:28 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-y[80565]: 2024-03-23T17:44:28.142+0000 7fc937591200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-23T17:44:28.516 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Mar 23 17:44:28 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-y[80565]: 2024-03-23T17:44:28.237+0000 7fc937591200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-23T17:44:28.516 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Mar 23 17:44:28 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-y[80565]: 2024-03-23T17:44:28.383+0000 7fc937591200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-23T17:44:28.766 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:28 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T17:44:28.766 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:28 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T17:44:28.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:28 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:44:28.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:28 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:28.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:28 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T17:44:28.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:28 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T17:44:28.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:28 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:44:28.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:28 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:28.836 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Mar 23 17:44:28 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-z[49806]: 2024-03-23T17:44:28.645+0000 7fd538928200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-23T17:44:28.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:28 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T17:44:28.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:28 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T17:44:28.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:28 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:44:28.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:28 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:29.208 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Mar 23 17:44:28 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-z[49806]: 2024-03-23T17:44:28.928+0000 7fd538928200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-23T17:44:29.208 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Mar 23 17:44:29 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-z[49806]: 2024-03-23T17:44:29.016+0000 7fd538928200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-23T17:44:29.208 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Mar 23 17:44:29 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-z[49806]: 2024-03-23T17:44:29.207+0000 7fd538928200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-23T17:44:29.307 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Mar 23 17:44:29 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-y[80565]: 2024-03-23T17:44:28.998+0000 7fc937591200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-23T17:44:29.307 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Mar 23 17:44:29 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-y[80565]: 2024-03-23T17:44:29.177+0000 7fc937591200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-23T17:44:29.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:29.524+0000 7fb5fcc00700 1 -- 172.21.15.86:0/2449685400 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mgrmap(e 14) v1 ==== 90699+0+0 (secure 0 0 0) 0x7fb5f00105a0 con 0x7fb600100e00 2024-03-23T17:44:29.766 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:29 smithi086 ceph-mon[38221]: from='client.24122 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:44:29.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:29 smithi086 ceph-mon[38221]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:44:29.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:29 smithi086 ceph-mon[38221]: Standby manager daemon z started 2024-03-23T17:44:29.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:29 smithi086 ceph-mon[38221]: from='mgr.? 172.21.15.152:0/2919942568' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-23T17:44:29.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:29 smithi086 ceph-mon[38221]: from='mgr.? 172.21.15.152:0/2919942568' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-23T17:44:29.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:29 smithi086 ceph-mon[38221]: from='mgr.? 172.21.15.152:0/2919942568' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-23T17:44:29.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:29 smithi086 ceph-mon[38221]: from='mgr.? 172.21.15.152:0/2919942568' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-23T17:44:29.807 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Mar 23 17:44:29 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-y[80565]: 2024-03-23T17:44:29.367+0000 7fc937591200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-23T17:44:29.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:29 smithi092 ceph-mon[79089]: from='client.24122 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:44:29.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:29 smithi092 ceph-mon[79089]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:44:29.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:29 smithi092 ceph-mon[79089]: Standby manager daemon z started 2024-03-23T17:44:29.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:29 smithi092 ceph-mon[79089]: from='mgr.? 172.21.15.152:0/2919942568' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-23T17:44:29.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:29 smithi092 ceph-mon[79089]: from='mgr.? 172.21.15.152:0/2919942568' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-23T17:44:29.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:29 smithi092 ceph-mon[79089]: from='mgr.? 172.21.15.152:0/2919942568' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-23T17:44:29.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:29 smithi092 ceph-mon[79089]: from='mgr.? 172.21.15.152:0/2919942568' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-23T17:44:29.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:29 smithi152 ceph-mon[47949]: from='client.24122 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:44:29.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:29 smithi152 ceph-mon[47949]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:44:29.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:29 smithi152 ceph-mon[47949]: Standby manager daemon z started 2024-03-23T17:44:29.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:29 smithi152 ceph-mon[47949]: from='mgr.? 172.21.15.152:0/2919942568' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-23T17:44:29.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:29 smithi152 ceph-mon[47949]: from='mgr.? 172.21.15.152:0/2919942568' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-23T17:44:29.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:29 smithi152 ceph-mon[47949]: from='mgr.? 172.21.15.152:0/2919942568' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-23T17:44:29.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:29 smithi152 ceph-mon[47949]: from='mgr.? 172.21.15.152:0/2919942568' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-23T17:44:30.557 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Mar 23 17:44:30 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-y[80565]: 2024-03-23T17:44:30.282+0000 7fc937591200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-23T17:44:30.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:30 smithi092 ceph-mon[79089]: mgrmap e14: x(active, since 61s), standbys: z 2024-03-23T17:44:30.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:30 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-23T17:44:30.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:30 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/4043828852' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "656655c6-1c7f-4640-92c4-2f42de820143"}]: dispatch 2024-03-23T17:44:30.558 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:30 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/4043828852' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "656655c6-1c7f-4640-92c4-2f42de820143"}]': finished 2024-03-23T17:44:30.558 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:30 smithi092 ceph-mon[79089]: osdmap e5: 1 total, 0 up, 1 in 2024-03-23T17:44:30.558 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:30 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T17:44:30.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:30 smithi152 ceph-mon[47949]: mgrmap e14: x(active, since 61s), standbys: z 2024-03-23T17:44:30.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:30 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-23T17:44:30.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:30 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/4043828852' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "656655c6-1c7f-4640-92c4-2f42de820143"}]: dispatch 2024-03-23T17:44:30.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:30 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/4043828852' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "656655c6-1c7f-4640-92c4-2f42de820143"}]': finished 2024-03-23T17:44:30.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:30 smithi152 ceph-mon[47949]: osdmap e5: 1 total, 0 up, 1 in 2024-03-23T17:44:30.837 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:30 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T17:44:30.842 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Mar 23 17:44:30 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-y[80565]: 2024-03-23T17:44:30.566+0000 7fc937591200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-23T17:44:30.842 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Mar 23 17:44:30 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-y[80565]: 2024-03-23T17:44:30.653+0000 7fc937591200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-23T17:44:31.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:30 smithi086 ceph-mon[38221]: mgrmap e14: x(active, since 61s), standbys: z 2024-03-23T17:44:31.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:30 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-23T17:44:31.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:30 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/4043828852' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "656655c6-1c7f-4640-92c4-2f42de820143"}]: dispatch 2024-03-23T17:44:31.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:30 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/4043828852' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "656655c6-1c7f-4640-92c4-2f42de820143"}]': finished 2024-03-23T17:44:31.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:30 smithi086 ceph-mon[38221]: osdmap e5: 1 total, 0 up, 1 in 2024-03-23T17:44:31.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:30 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T17:44:31.307 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Mar 23 17:44:30 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-y[80565]: 2024-03-23T17:44:30.841+0000 7fc937591200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-23T17:44:31.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:31.620+0000 7fb5fcc00700 1 -- 172.21.15.86:0/2449685400 <== mon.2 v2:172.21.15.92:3300/0 8 ==== mgrmap(e 15) v1 ==== 135809+0+0 (secure 0 0 0) 0x7fb5f0010880 con 0x7fb600100e00 2024-03-23T17:44:31.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:31 smithi092 ceph-mon[79089]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:44:31.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:31 smithi092 ceph-mon[79089]: Standby manager daemon y started 2024-03-23T17:44:31.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:31 smithi092 ceph-mon[79089]: from='mgr.? 172.21.15.92:0/3934817225' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-23T17:44:31.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:31 smithi092 ceph-mon[79089]: from='mgr.? 172.21.15.92:0/3934817225' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-23T17:44:31.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:31 smithi092 ceph-mon[79089]: from='mgr.? 172.21.15.92:0/3934817225' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-23T17:44:31.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:31 smithi092 ceph-mon[79089]: from='mgr.? 172.21.15.92:0/3934817225' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-23T17:44:31.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:31 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/360172348' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T17:44:31.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:31 smithi152 ceph-mon[47949]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:44:31.837 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:31 smithi152 ceph-mon[47949]: Standby manager daemon y started 2024-03-23T17:44:31.837 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:31 smithi152 ceph-mon[47949]: from='mgr.? 172.21.15.92:0/3934817225' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-23T17:44:31.837 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:31 smithi152 ceph-mon[47949]: from='mgr.? 172.21.15.92:0/3934817225' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-23T17:44:31.837 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:31 smithi152 ceph-mon[47949]: from='mgr.? 172.21.15.92:0/3934817225' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-23T17:44:31.837 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:31 smithi152 ceph-mon[47949]: from='mgr.? 172.21.15.92:0/3934817225' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-23T17:44:31.837 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:31 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/360172348' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T17:44:32.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:31 smithi086 ceph-mon[38221]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:44:32.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:31 smithi086 ceph-mon[38221]: Standby manager daemon y started 2024-03-23T17:44:32.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:31 smithi086 ceph-mon[38221]: from='mgr.? 172.21.15.92:0/3934817225' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-23T17:44:32.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:31 smithi086 ceph-mon[38221]: from='mgr.? 172.21.15.92:0/3934817225' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-23T17:44:32.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:31 smithi086 ceph-mon[38221]: from='mgr.? 172.21.15.92:0/3934817225' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-23T17:44:32.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:31 smithi086 ceph-mon[38221]: from='mgr.? 172.21.15.92:0/3934817225' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-23T17:44:32.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:31 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/360172348' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T17:44:33.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:32 smithi086 ceph-mon[38221]: mgrmap e15: x(active, since 63s), standbys: z, y 2024-03-23T17:44:33.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:32 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-23T17:44:33.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:32 smithi092 ceph-mon[79089]: mgrmap e15: x(active, since 63s), standbys: z, y 2024-03-23T17:44:33.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:32 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-23T17:44:33.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:32 smithi152 ceph-mon[47949]: mgrmap e15: x(active, since 63s), standbys: z, y 2024-03-23T17:44:33.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:32 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-23T17:44:34.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:33 smithi086 ceph-mon[38221]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:44:34.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:33 smithi092 ceph-mon[79089]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:44:34.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:33 smithi152 ceph-mon[47949]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:44:35.908 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:35 smithi086 ceph-mon[38221]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:44:36.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:35 smithi092 ceph-mon[79089]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:44:36.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:35 smithi152 ceph-mon[47949]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:44:37.897 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:37 smithi086 ceph-mon[38221]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:44:37.897 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:37 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-23T17:44:37.898 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:37 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:44:38.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:37 smithi092 ceph-mon[79089]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:44:38.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:37 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-23T17:44:38.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:37 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:44:38.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:37 smithi152 ceph-mon[47949]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:44:38.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:37 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-23T17:44:38.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:37 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:44:39.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:38 smithi086 ceph-mon[38221]: Deploying daemon osd.0 on smithi086 2024-03-23T17:44:39.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:38 smithi092 ceph-mon[79089]: Deploying daemon osd.0 on smithi086 2024-03-23T17:44:39.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:38 smithi152 ceph-mon[47949]: Deploying daemon osd.0 on smithi086 2024-03-23T17:44:39.753 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:39 smithi086 ceph-mon[38221]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:44:40.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:39 smithi092 ceph-mon[79089]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:44:40.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:39 smithi152 ceph-mon[47949]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:44:42.018 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:41 smithi086 ceph-mon[38221]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:44:42.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:41 smithi092 ceph-mon[79089]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:44:42.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:41 smithi152 ceph-mon[47949]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:44:43.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:43 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:43.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:43 smithi086 ceph-mon[38221]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:44:43.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:43 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:43.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:43 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:44:43.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:43 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:43.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:43 smithi092 ceph-mon[79089]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:44:43.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:43 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:43.558 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:43 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:44:43.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:43 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:43.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:43 smithi152 ceph-mon[47949]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:44:43.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:43 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:43.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:43 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:44:45.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:44 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:45.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:44 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:45.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:44 smithi086 ceph-mon[38221]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:44:45.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:44 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:44:45.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:44 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:44:45.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:44 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:45.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:44 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:45.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:44 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:45.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:44 smithi092 ceph-mon[79089]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:44:45.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:44 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:44:45.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:44 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:44:45.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:44 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:45.333 INFO:teuthology.orchestra.run.smithi086.stdout:Created osd(s) 0 on host 'smithi086' 2024-03-23T17:44:45.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:45.331+0000 7fb5fcc00700 1 -- 172.21.15.86:0/2449685400 <== mgr.14152 v2:172.21.15.86:6800/2734784244 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fb60002cdc0 con 0x7fb5e40389d0 2024-03-23T17:44:45.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:45.334+0000 7fb606e00700 1 -- 172.21.15.86:0/2449685400 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fb5e40389d0 msgr2=0x7fb5e403ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:44:45.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:45.334+0000 7fb606e00700 1 --2- 172.21.15.86:0/2449685400 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fb5e40389d0 0x7fb5e403ae90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec00b2e0 tx=0x7fb5ec005fd0 comp rx=0 tx=0).stop 2024-03-23T17:44:45.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:45.334+0000 7fb606e00700 1 -- 172.21.15.86:0/2449685400 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb600100e00 msgr2=0x7fb600074b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:44:45.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:45.334+0000 7fb606e00700 1 --2- 172.21.15.86:0/2449685400 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb600100e00 0x7fb600074b00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb5f000e9a0 tx=0x7fb5f000ecb0 comp rx=0 tx=0).stop 2024-03-23T17:44:45.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:45.334+0000 7fb606e00700 1 -- 172.21.15.86:0/2449685400 shutdown_connections 2024-03-23T17:44:45.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:45.334+0000 7fb606e00700 1 --2- 172.21.15.86:0/2449685400 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fb5e40389d0 0x7fb5e403ae90 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:44:45.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:45.334+0000 7fb606e00700 1 --2- 172.21.15.86:0/2449685400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb600102920 0x7fb6000736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:44:45.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:45.334+0000 7fb606e00700 1 --2- 172.21.15.86:0/2449685400 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb600100e00 0x7fb600074b00 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:44:45.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:45.334+0000 7fb606e00700 1 --2- 172.21.15.86:0/2449685400 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb600101f60 0x7fb6000731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:44:45.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:45.334+0000 7fb606e00700 1 -- 172.21.15.86:0/2449685400 >> 172.21.15.86:0/2449685400 conn(0x7fb6000fc3c0 msgr2=0x7fb6001053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:44:45.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:45.334+0000 7fb606e00700 1 -- 172.21.15.86:0/2449685400 shutdown_connections 2024-03-23T17:44:45.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:44:45.334+0000 7fb606e00700 1 -- 172.21.15.86:0/2449685400 wait complete. 2024-03-23T17:44:45.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:44 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:45.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:44 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:45.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:44 smithi152 ceph-mon[47949]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:44:45.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:44 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:44:45.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:44 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:44:45.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:44 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:45.711 DEBUG:teuthology.orchestra.run.smithi086:osd.0> sudo journalctl -f -n 0 -u ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@osd.0.service 2024-03-23T17:44:45.715 INFO:tasks.cephadm:Deploying osd.1 on smithi092 with /dev/vg_nvme/lv_4... 2024-03-23T17:44:45.715 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-23T17:44:45.788 INFO:journalctl@ceph.osd.0.smithi086.stdout:-- Logs begin at Sat 2024-03-23 17:25:35 UTC. -- 2024-03-23T17:44:45.946 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.b/config 2024-03-23T17:44:46.357 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:46 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:46.358 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:46 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:46.358 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:46 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:44:46.358 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:46 smithi092 ceph-mon[79089]: from='osd.0 [v2:172.21.15.86:6802/2423505491,v1:172.21.15.86:6803/2423505491]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-23T17:44:46.358 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:46 smithi092 ceph-mon[79089]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-23T17:44:46.448 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:46 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:46.449 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:46 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:46.449 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:46 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:44:46.449 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:46 smithi086 ceph-mon[38221]: from='osd.0 [v2:172.21.15.86:6802/2423505491,v1:172.21.15.86:6803/2423505491]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-23T17:44:46.449 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:46 smithi086 ceph-mon[38221]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-23T17:44:46.449 INFO:journalctl@ceph.osd.0.smithi086.stdout:Mar 23 17:44:46 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-0[56695]: 2024-03-23T17:44:46.268+0000 7fbab9044640 -1 osd.0 0 log_to_monitors true 2024-03-23T17:44:46.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:46 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:46.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:46 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:46.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:46 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:44:46.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:46 smithi152 ceph-mon[47949]: from='osd.0 [v2:172.21.15.86:6802/2423505491,v1:172.21.15.86:6803/2423505491]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-23T17:44:46.587 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:46 smithi152 ceph-mon[47949]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-23T17:44:47.235 INFO:teuthology.orchestra.run.smithi092.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-23T17:44:47.235 INFO:teuthology.orchestra.run.smithi092.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-23T17:44:47.235 INFO:teuthology.orchestra.run.smithi092.stderr: stderr: 10+0 records in 2024-03-23T17:44:47.236 INFO:teuthology.orchestra.run.smithi092.stderr:10+0 records out 2024-03-23T17:44:47.236 INFO:teuthology.orchestra.run.smithi092.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0128852 s, 814 MB/s 2024-03-23T17:44:47.236 INFO:teuthology.orchestra.run.smithi092.stderr: stderr: 2024-03-23T17:44:47.236 INFO:teuthology.orchestra.run.smithi092.stderr:--> Zapping successful for: 2024-03-23T17:44:47.567 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi092:vg_nvme/lv_4 2024-03-23T17:44:47.782 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.b/config 2024-03-23T17:44:47.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:47 smithi092 ceph-mon[79089]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:44:47.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:47 smithi092 ceph-mon[79089]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-23T17:44:47.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:47 smithi092 ceph-mon[79089]: osdmap e6: 1 total, 0 up, 1 in 2024-03-23T17:44:47.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:47 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T17:44:47.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:47 smithi092 ceph-mon[79089]: from='osd.0 [v2:172.21.15.86:6802/2423505491,v1:172.21.15.86:6803/2423505491]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-03-23T17:44:47.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:47 smithi092 ceph-mon[79089]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-03-23T17:44:47.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:47 smithi152 ceph-mon[47949]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:44:47.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:47 smithi152 ceph-mon[47949]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-23T17:44:47.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:47 smithi152 ceph-mon[47949]: osdmap e6: 1 total, 0 up, 1 in 2024-03-23T17:44:47.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:47 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T17:44:47.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:47 smithi152 ceph-mon[47949]: from='osd.0 [v2:172.21.15.86:6802/2423505491,v1:172.21.15.86:6803/2423505491]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-03-23T17:44:47.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:47 smithi152 ceph-mon[47949]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-03-23T17:44:48.016 INFO:journalctl@ceph.osd.0.smithi086.stdout:Mar 23 17:44:47 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-0[56695]: 2024-03-23T17:44:47.539+0000 7fbaacc00700 -1 osd.0 0 waiting for initial osdmap 2024-03-23T17:44:48.017 INFO:journalctl@ceph.osd.0.smithi086.stdout:Mar 23 17:44:47 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-0[56695]: 2024-03-23T17:44:47.559+0000 7fbaa8400700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T17:44:48.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:47 smithi086 ceph-mon[38221]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:44:48.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:47 smithi086 ceph-mon[38221]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-23T17:44:48.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:47 smithi086 ceph-mon[38221]: osdmap e6: 1 total, 0 up, 1 in 2024-03-23T17:44:48.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:47 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T17:44:48.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:47 smithi086 ceph-mon[38221]: from='osd.0 [v2:172.21.15.86:6802/2423505491,v1:172.21.15.86:6803/2423505491]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-03-23T17:44:48.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:47 smithi086 ceph-mon[38221]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-03-23T17:44:48.720 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:44:48.719+0000 7f57165d7700 1 -- 172.21.15.92:0/67133620 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5710102150 msgr2=0x7f57101025b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:44:48.720 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:44:48.719+0000 7f57165d7700 1 --2- 172.21.15.92:0/67133620 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5710102150 0x7f57101025b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5704009990 tx=0x7f5704009ca0 comp rx=0 tx=0).stop 2024-03-23T17:44:48.720 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:44:48.719+0000 7f57165d7700 1 -- 172.21.15.92:0/67133620 shutdown_connections 2024-03-23T17:44:48.720 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:44:48.719+0000 7f57165d7700 1 --2- 172.21.15.92:0/67133620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5710102af0 0x7f57101093a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:44:48.720 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:44:48.719+0000 7f57165d7700 1 --2- 172.21.15.92:0/67133620 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f57100ff0d0 0x7f57100ff4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:44:48.720 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:44:48.719+0000 7f57165d7700 1 --2- 172.21.15.92:0/67133620 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5710102150 0x7f57101025b0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:44:48.720 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:44:48.719+0000 7f57165d7700 1 -- 172.21.15.92:0/67133620 >> 172.21.15.92:0/67133620 conn(0x7f57100faa10 msgr2=0x7f57100fce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:44:48.720 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:44:48.719+0000 7f57165d7700 1 -- 172.21.15.92:0/67133620 shutdown_connections 2024-03-23T17:44:48.720 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:44:48.719+0000 7f57165d7700 1 -- 172.21.15.92:0/67133620 wait complete. 2024-03-23T17:44:48.722 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:44:48.720+0000 7f57165d7700 1 Processor -- start 2024-03-23T17:44:48.723 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:44:48.720+0000 7f57165d7700 1 -- start start 2024-03-23T17:44:48.723 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:44:48.720+0000 7f57165d7700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f57100ff0d0 0x7f5710101a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:44:48.723 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:44:48.720+0000 7f57165d7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5710102150 0x7f5710100100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:44:48.723 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:44:48.720+0000 7f57165d7700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5710102af0 0x7f5710100640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:44:48.723 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:44:48.720+0000 7f57165d7700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5710100de0 con 0x7f5710102af0 2024-03-23T17:44:48.723 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:44:48.720+0000 7f57165d7700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5710100f50 con 0x7f5710102150 2024-03-23T17:44:48.723 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:44:48.720+0000 7f57165d7700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57101a0d60 con 0x7f57100ff0d0 2024-03-23T17:44:48.723 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:44:48.720+0000 7f570f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5710102150 0x7f5710100100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:44:48.723 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:44:48.720+0000 7f570f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5710102150 0x7f5710100100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56066/0 (socket says 172.21.15.92:56066) 2024-03-23T17:44:48.723 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:44:48.720+0000 7f570f7fe700 1 -- 172.21.15.92:0/78569984 learned_addr learned my addr 172.21.15.92:0/78569984 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-23T17:44:48.723 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:44:48.720+0000 7f570ffff700 1 --2- 172.21.15.92:0/78569984 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f57100ff0d0 0x7f5710101a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:44:48.723 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:44:48.721+0000 7f5714b74700 1 --2- 172.21.15.92:0/78569984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5710102af0 0x7f5710100640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:44:48.723 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:44:48.721+0000 7f570f7fe700 1 -- 172.21.15.92:0/78569984 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f57100ff0d0 msgr2=0x7f5710101a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:44:48.723 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:44:48.721+0000 7f570f7fe700 1 --2- 172.21.15.92:0/78569984 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f57100ff0d0 0x7f5710101a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:44:48.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:44:48.721+0000 7f570f7fe700 1 -- 172.21.15.92:0/78569984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5710102af0 msgr2=0x7f5710100640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:44:48.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:44:48.721+0000 7f570f7fe700 1 --2- 172.21.15.92:0/78569984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5710102af0 0x7f5710100640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:44:48.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:44:48.721+0000 7f570f7fe700 1 -- 172.21.15.92:0/78569984 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57040095f0 con 0x7f5710102150 2024-03-23T17:44:48.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:44:48.721+0000 7f5714b74700 1 --2- 172.21.15.92:0/78569984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5710102af0 0x7f5710100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:44:48.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:44:48.721+0000 7f570f7fe700 1 --2- 172.21.15.92:0/78569984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5710102150 0x7f5710100100 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f57040095c0 tx=0x7f570400f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:44:48.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:44:48.721+0000 7f570d7fa700 1 -- 172.21.15.92:0/78569984 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f570401d070 con 0x7f5710102150 2024-03-23T17:44:48.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:44:48.721+0000 7f570d7fa700 1 -- 172.21.15.92:0/78569984 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5704022470 con 0x7f5710102150 2024-03-23T17:44:48.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:44:48.721+0000 7f57165d7700 1 -- 172.21.15.92:0/78569984 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57101a0f90 con 0x7f5710102150 2024-03-23T17:44:48.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:44:48.721+0000 7f570d7fa700 1 -- 172.21.15.92:0/78569984 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5704017650 con 0x7f5710102150 2024-03-23T17:44:48.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:44:48.721+0000 7f57165d7700 1 -- 172.21.15.92:0/78569984 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57101a13a0 con 0x7f5710102150 2024-03-23T17:44:48.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:44:48.722+0000 7f57165d7700 1 -- 172.21.15.92:0/78569984 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5710108380 con 0x7f5710102150 2024-03-23T17:44:48.725 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:44:48.723+0000 7f570d7fa700 1 -- 172.21.15.92:0/78569984 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 15) v1 ==== 135809+0+0 (secure 0 0 0) 0x7f5704022850 con 0x7f5710102150 2024-03-23T17:44:48.725 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:44:48.724+0000 7f570d7fa700 1 --2- 172.21.15.92:0/78569984 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f56f80a1110 0x7f56f80a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:44:48.725 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:44:48.724+0000 7f570d7fa700 1 -- 172.21.15.92:0/78569984 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1767+0+0 (secure 0 0 0) 0x7f57040cc550 con 0x7f5710102150 2024-03-23T17:44:48.725 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:44:48.724+0000 7f570ffff700 1 --2- 172.21.15.92:0/78569984 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f56f80a1110 0x7f56f80a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:44:48.725 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:44:48.725+0000 7f570ffff700 1 --2- 172.21.15.92:0/78569984 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f56f80a1110 0x7f56f80a35d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5710068490 tx=0x7f570000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:44:48.728 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:44:48.728+0000 7f570d7fa700 1 -- 172.21.15.92:0/78569984 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f57040977e0 con 0x7f5710102150 2024-03-23T17:44:48.766 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:48 smithi086 ceph-mon[38221]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-03-23T17:44:48.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:48 smithi086 ceph-mon[38221]: osdmap e7: 1 total, 0 up, 1 in 2024-03-23T17:44:48.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:48 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T17:44:48.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:48 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T17:44:48.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:48 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T17:44:48.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:48 smithi092 ceph-mon[79089]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-03-23T17:44:48.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:48 smithi092 ceph-mon[79089]: osdmap e7: 1 total, 0 up, 1 in 2024-03-23T17:44:48.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:48 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T17:44:48.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:48 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T17:44:48.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:48 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T17:44:48.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:48 smithi152 ceph-mon[47949]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-03-23T17:44:48.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:48 smithi152 ceph-mon[47949]: osdmap e7: 1 total, 0 up, 1 in 2024-03-23T17:44:48.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:48 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T17:44:48.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:48 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T17:44:48.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:48 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T17:44:48.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:44:48.929+0000 7f57165d7700 1 -- 172.21.15.92:0/78569984 --> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f571002cd20 con 0x7f56f80a1110 2024-03-23T17:44:49.707 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:49 smithi092 ceph-mon[79089]: purged_snaps scrub starts 2024-03-23T17:44:49.707 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:49 smithi092 ceph-mon[79089]: purged_snaps scrub ok 2024-03-23T17:44:49.707 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:49 smithi092 ceph-mon[79089]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:44:49.708 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:49 smithi092 ceph-mon[79089]: osd.0 [v2:172.21.15.86:6802/2423505491,v1:172.21.15.86:6803/2423505491] boot 2024-03-23T17:44:49.708 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:49 smithi092 ceph-mon[79089]: osdmap e8: 1 total, 1 up, 1 in 2024-03-23T17:44:49.708 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:49 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T17:44:49.708 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:49 smithi092 ceph-mon[79089]: Detected new or changed devices on smithi086 2024-03-23T17:44:49.708 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:49 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:49.708 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:49 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:49.708 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:49 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:44:49.708 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:49 smithi092 ceph-mon[79089]: Adjusting osd_memory_target on smithi086 to 17270M 2024-03-23T17:44:49.708 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:49 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:49.708 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:49 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:44:49.708 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:49 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:44:49.708 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:49 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:49.708 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:49 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T17:44:49.708 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:49 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T17:44:49.709 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:49 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:44:49.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:49 smithi152 ceph-mon[47949]: purged_snaps scrub starts 2024-03-23T17:44:49.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:49 smithi152 ceph-mon[47949]: purged_snaps scrub ok 2024-03-23T17:44:49.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:49 smithi152 ceph-mon[47949]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:44:49.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:49 smithi152 ceph-mon[47949]: osd.0 [v2:172.21.15.86:6802/2423505491,v1:172.21.15.86:6803/2423505491] boot 2024-03-23T17:44:49.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:49 smithi152 ceph-mon[47949]: osdmap e8: 1 total, 1 up, 1 in 2024-03-23T17:44:49.837 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:49 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T17:44:49.837 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:49 smithi152 ceph-mon[47949]: Detected new or changed devices on smithi086 2024-03-23T17:44:49.837 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:49 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:49.837 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:49 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:49.837 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:49 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:44:49.837 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:49 smithi152 ceph-mon[47949]: Adjusting osd_memory_target on smithi086 to 17270M 2024-03-23T17:44:49.837 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:49 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:49.837 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:49 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:44:49.837 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:49 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:44:49.837 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:49 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:49.837 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:49 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T17:44:49.837 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:49 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T17:44:49.837 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:49 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:44:50.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:49 smithi086 ceph-mon[38221]: purged_snaps scrub starts 2024-03-23T17:44:50.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:49 smithi086 ceph-mon[38221]: purged_snaps scrub ok 2024-03-23T17:44:50.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:49 smithi086 ceph-mon[38221]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T17:44:50.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:49 smithi086 ceph-mon[38221]: osd.0 [v2:172.21.15.86:6802/2423505491,v1:172.21.15.86:6803/2423505491] boot 2024-03-23T17:44:50.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:49 smithi086 ceph-mon[38221]: osdmap e8: 1 total, 1 up, 1 in 2024-03-23T17:44:50.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:49 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T17:44:50.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:49 smithi086 ceph-mon[38221]: Detected new or changed devices on smithi086 2024-03-23T17:44:50.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:49 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:50.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:49 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:50.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:49 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:44:50.018 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:49 smithi086 ceph-mon[38221]: Adjusting osd_memory_target on smithi086 to 17270M 2024-03-23T17:44:50.018 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:49 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:50.018 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:49 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:44:50.018 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:49 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:44:50.018 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:49 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:44:50.018 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:49 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T17:44:50.018 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:49 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T17:44:50.018 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:49 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:44:51.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:50 smithi086 ceph-mon[38221]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:44:51.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:50 smithi086 ceph-mon[38221]: osdmap e9: 1 total, 1 up, 1 in 2024-03-23T17:44:51.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:50 smithi092 ceph-mon[79089]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:44:51.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:50 smithi092 ceph-mon[79089]: osdmap e9: 1 total, 1 up, 1 in 2024-03-23T17:44:51.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:50 smithi152 ceph-mon[47949]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:44:51.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:50 smithi152 ceph-mon[47949]: osdmap e9: 1 total, 1 up, 1 in 2024-03-23T17:44:52.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:51 smithi086 ceph-mon[38221]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T17:44:52.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:51 smithi086 ceph-mon[38221]: from='client.? 172.21.15.92:0/3719861138' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3e86dc65-20a6-4de8-b9ce-fcf01f68862b"}]: dispatch 2024-03-23T17:44:52.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:51 smithi086 ceph-mon[38221]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3e86dc65-20a6-4de8-b9ce-fcf01f68862b"}]: dispatch 2024-03-23T17:44:52.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:51 smithi086 ceph-mon[38221]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3e86dc65-20a6-4de8-b9ce-fcf01f68862b"}]': finished 2024-03-23T17:44:52.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:51 smithi086 ceph-mon[38221]: osdmap e10: 2 total, 1 up, 2 in 2024-03-23T17:44:52.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:51 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T17:44:52.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:51 smithi086 ceph-mon[38221]: from='client.? 172.21.15.92:0/708901984' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T17:44:52.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:51 smithi092 ceph-mon[79089]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T17:44:52.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:51 smithi092 ceph-mon[79089]: from='client.? 172.21.15.92:0/3719861138' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3e86dc65-20a6-4de8-b9ce-fcf01f68862b"}]: dispatch 2024-03-23T17:44:52.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:51 smithi092 ceph-mon[79089]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3e86dc65-20a6-4de8-b9ce-fcf01f68862b"}]: dispatch 2024-03-23T17:44:52.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:51 smithi092 ceph-mon[79089]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3e86dc65-20a6-4de8-b9ce-fcf01f68862b"}]': finished 2024-03-23T17:44:52.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:51 smithi092 ceph-mon[79089]: osdmap e10: 2 total, 1 up, 2 in 2024-03-23T17:44:52.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:51 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T17:44:52.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:51 smithi092 ceph-mon[79089]: from='client.? 172.21.15.92:0/708901984' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T17:44:52.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:51 smithi152 ceph-mon[47949]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T17:44:52.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:51 smithi152 ceph-mon[47949]: from='client.? 172.21.15.92:0/3719861138' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3e86dc65-20a6-4de8-b9ce-fcf01f68862b"}]: dispatch 2024-03-23T17:44:52.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:51 smithi152 ceph-mon[47949]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3e86dc65-20a6-4de8-b9ce-fcf01f68862b"}]: dispatch 2024-03-23T17:44:52.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:51 smithi152 ceph-mon[47949]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3e86dc65-20a6-4de8-b9ce-fcf01f68862b"}]': finished 2024-03-23T17:44:52.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:51 smithi152 ceph-mon[47949]: osdmap e10: 2 total, 1 up, 2 in 2024-03-23T17:44:52.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:51 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T17:44:52.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:51 smithi152 ceph-mon[47949]: from='client.? 172.21.15.92:0/708901984' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T17:44:54.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:53 smithi086 ceph-mon[38221]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T17:44:54.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:53 smithi092 ceph-mon[79089]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T17:44:54.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:53 smithi152 ceph-mon[47949]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T17:44:55.772 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:55 smithi092 ceph-mon[79089]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T17:44:56.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:55 smithi086 ceph-mon[38221]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T17:44:56.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:55 smithi152 ceph-mon[47949]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T17:44:57.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:57 smithi092 ceph-mon[79089]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T17:44:57.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:57 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-23T17:44:57.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:57 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:44:58.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:57 smithi086 ceph-mon[38221]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T17:44:58.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:57 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-23T17:44:58.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:57 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:44:58.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:57 smithi152 ceph-mon[47949]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T17:44:58.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:57 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-23T17:44:58.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:57 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:44:58.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:58 smithi092 ceph-mon[79089]: Deploying daemon osd.1 on smithi092 2024-03-23T17:44:59.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:58 smithi086 ceph-mon[38221]: Deploying daemon osd.1 on smithi092 2024-03-23T17:44:59.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:58 smithi152 ceph-mon[47949]: Deploying daemon osd.1 on smithi092 2024-03-23T17:45:00.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:44:59 smithi086 ceph-mon[38221]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T17:45:00.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:44:59 smithi092 ceph-mon[79089]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T17:45:00.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:44:59 smithi152 ceph-mon[47949]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T17:45:01.666 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:01 smithi092 ceph-mon[79089]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T17:45:01.666 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:01 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:01.666 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:01 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:01.666 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:01 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:45:01.766 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:01 smithi086 ceph-mon[38221]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T17:45:01.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:01 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:01.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:01 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:01.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:01 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:45:01.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:01 smithi152 ceph-mon[47949]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T17:45:01.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:01 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:01.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:01 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:01.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:01 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:45:02.683 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:02 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:02.683 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:02 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:02.683 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:02 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:45:02.683 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:02 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:45:02.683 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:02 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:02.766 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:02 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:02.766 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:02 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:02.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:02 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:45:02.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:02 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:45:02.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:02 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:02.787 INFO:teuthology.orchestra.run.smithi092.stdout:Created osd(s) 1 on host 'smithi092' 2024-03-23T17:45:02.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:45:02.787+0000 7f570d7fa700 1 -- 172.21.15.92:0/78569984 <== mgr.14152 v2:172.21.15.86:6800/2734784244 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f571002cd20 con 0x7f56f80a1110 2024-03-23T17:45:02.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:45:02.789+0000 7f57165d7700 1 -- 172.21.15.92:0/78569984 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f56f80a1110 msgr2=0x7f56f80a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:45:02.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:45:02.789+0000 7f57165d7700 1 --2- 172.21.15.92:0/78569984 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f56f80a1110 0x7f56f80a35d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5710068490 tx=0x7f570000b410 comp rx=0 tx=0).stop 2024-03-23T17:45:02.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:45:02.789+0000 7f57165d7700 1 -- 172.21.15.92:0/78569984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5710102150 msgr2=0x7f5710100100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:45:02.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:45:02.789+0000 7f57165d7700 1 --2- 172.21.15.92:0/78569984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5710102150 0x7f5710100100 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f57040095c0 tx=0x7f570400f690 comp rx=0 tx=0).stop 2024-03-23T17:45:02.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:45:02.789+0000 7f57165d7700 1 -- 172.21.15.92:0/78569984 shutdown_connections 2024-03-23T17:45:02.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:45:02.789+0000 7f57165d7700 1 --2- 172.21.15.92:0/78569984 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f56f80a1110 0x7f56f80a35d0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:45:02.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:45:02.789+0000 7f57165d7700 1 --2- 172.21.15.92:0/78569984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5710102af0 0x7f5710100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:45:02.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:45:02.789+0000 7f57165d7700 1 --2- 172.21.15.92:0/78569984 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f57100ff0d0 0x7f5710101a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:45:02.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:45:02.789+0000 7f57165d7700 1 --2- 172.21.15.92:0/78569984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5710102150 0x7f5710100100 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:45:02.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:45:02.789+0000 7f57165d7700 1 -- 172.21.15.92:0/78569984 >> 172.21.15.92:0/78569984 conn(0x7f57100faa10 msgr2=0x7f57101054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:45:02.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:45:02.789+0000 7f57165d7700 1 -- 172.21.15.92:0/78569984 shutdown_connections 2024-03-23T17:45:02.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:45:02.789+0000 7f57165d7700 1 -- 172.21.15.92:0/78569984 wait complete. 2024-03-23T17:45:02.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:02 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:02.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:02 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:02.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:02 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:45:02.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:02 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:45:02.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:02 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:03.149 DEBUG:teuthology.orchestra.run.smithi092:osd.1> sudo journalctl -f -n 0 -u ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@osd.1.service 2024-03-23T17:45:03.151 INFO:tasks.cephadm:Deploying osd.2 on smithi152 with /dev/vg_nvme/lv_4... 2024-03-23T17:45:03.151 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-23T17:45:03.182 INFO:journalctl@ceph.osd.1.smithi092.stdout:-- Logs begin at Sat 2024-03-23 17:19:16 UTC. -- 2024-03-23T17:45:03.378 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.c/config 2024-03-23T17:45:03.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:03 smithi092 ceph-mon[79089]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T17:45:03.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:03 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:45:03.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:03 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:03.558 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:03 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:03.766 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:03 smithi086 ceph-mon[38221]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T17:45:03.766 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:03 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:45:03.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:03 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:03.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:03 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:03.834 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:03 smithi152 ceph-mon[47949]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T17:45:03.834 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:03 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:45:03.834 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:03 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:03.834 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:03 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:04.729 INFO:teuthology.orchestra.run.smithi152.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-23T17:45:04.730 INFO:teuthology.orchestra.run.smithi152.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-23T17:45:04.730 INFO:teuthology.orchestra.run.smithi152.stderr: stderr: 10+0 records in 2024-03-23T17:45:04.730 INFO:teuthology.orchestra.run.smithi152.stderr:10+0 records out 2024-03-23T17:45:04.730 INFO:teuthology.orchestra.run.smithi152.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0116287 s, 902 MB/s 2024-03-23T17:45:04.730 INFO:teuthology.orchestra.run.smithi152.stderr:--> Zapping successful for: 2024-03-23T17:45:04.766 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:04 smithi086 ceph-mon[38221]: from='osd.1 [v2:172.21.15.92:6800/3776252597,v1:172.21.15.92:6801/3776252597]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-23T17:45:04.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:04 smithi086 ceph-mon[38221]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-23T17:45:04.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:04 smithi092 ceph-mon[79089]: from='osd.1 [v2:172.21.15.92:6800/3776252597,v1:172.21.15.92:6801/3776252597]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-23T17:45:04.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:04 smithi092 ceph-mon[79089]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-23T17:45:04.808 INFO:journalctl@ceph.osd.1.smithi092.stdout:Mar 23 17:45:04 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-1[84104]: 2024-03-23T17:45:04.399+0000 7fb83b384640 -1 osd.1 0 log_to_monitors true 2024-03-23T17:45:04.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:04 smithi152 ceph-mon[47949]: from='osd.1 [v2:172.21.15.92:6800/3776252597,v1:172.21.15.92:6801/3776252597]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-23T17:45:04.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:04 smithi152 ceph-mon[47949]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-23T17:45:05.113 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi152:vg_nvme/lv_4 2024-03-23T17:45:05.330 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.c/config 2024-03-23T17:45:05.807 INFO:journalctl@ceph.osd.1.smithi092.stdout:Mar 23 17:45:05 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-1[84104]: 2024-03-23T17:45:05.537+0000 7fb8301e7700 -1 osd.1 0 waiting for initial osdmap 2024-03-23T17:45:05.808 INFO:journalctl@ceph.osd.1.smithi092.stdout:Mar 23 17:45:05 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-1[84104]: 2024-03-23T17:45:05.541+0000 7fb82c7dd700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T17:45:05.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:05 smithi092 ceph-mon[79089]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T17:45:05.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:05 smithi092 ceph-mon[79089]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-23T17:45:05.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:05 smithi092 ceph-mon[79089]: osdmap e11: 2 total, 1 up, 2 in 2024-03-23T17:45:05.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:05 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T17:45:05.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:05 smithi092 ceph-mon[79089]: from='osd.1 [v2:172.21.15.92:6800/3776252597,v1:172.21.15.92:6801/3776252597]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-03-23T17:45:05.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:05 smithi092 ceph-mon[79089]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-03-23T17:45:05.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:05 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:05.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:05 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:05.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:05 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:45:05.809 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:05 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:05.809 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:05 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:45:05.809 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:05 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:45:05.809 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:05 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:05.809 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:05 smithi152 ceph-mon[47949]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T17:45:05.809 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:05 smithi152 ceph-mon[47949]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-23T17:45:05.809 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:05 smithi152 ceph-mon[47949]: osdmap e11: 2 total, 1 up, 2 in 2024-03-23T17:45:05.809 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:05 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T17:45:05.809 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:05 smithi152 ceph-mon[47949]: from='osd.1 [v2:172.21.15.92:6800/3776252597,v1:172.21.15.92:6801/3776252597]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-03-23T17:45:05.810 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:05 smithi152 ceph-mon[47949]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-03-23T17:45:05.810 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:05 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:05.810 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:05 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:05.810 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:05 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:45:05.810 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:05 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:05.810 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:05 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:45:05.810 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:05 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:45:05.810 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:05 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:06.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:05 smithi086 ceph-mon[38221]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T17:45:06.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:05 smithi086 ceph-mon[38221]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-23T17:45:06.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:05 smithi086 ceph-mon[38221]: osdmap e11: 2 total, 1 up, 2 in 2024-03-23T17:45:06.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:05 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T17:45:06.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:05 smithi086 ceph-mon[38221]: from='osd.1 [v2:172.21.15.92:6800/3776252597,v1:172.21.15.92:6801/3776252597]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-03-23T17:45:06.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:05 smithi086 ceph-mon[38221]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-03-23T17:45:06.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:05 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:06.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:05 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:06.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:05 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:45:06.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:05 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:06.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:05 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:45:06.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:05 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:45:06.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:05 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:06.398 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:45:06.397+0000 7fb3c2294700 1 -- 172.21.15.152:0/1876693552 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb3bc0ffcf0 msgr2=0x7fb3bc100110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:45:06.398 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:45:06.397+0000 7fb3c2294700 1 --2- 172.21.15.152:0/1876693552 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb3bc0ffcf0 0x7fb3bc100110 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0009990 tx=0x7fb3b0009ca0 comp rx=0 tx=0).stop 2024-03-23T17:45:06.398 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:45:06.397+0000 7fb3c2294700 1 -- 172.21.15.152:0/1876693552 shutdown_connections 2024-03-23T17:45:06.399 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:45:06.397+0000 7fb3c2294700 1 --2- 172.21.15.152:0/1876693552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3bc101890 0x7fb3bc108140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:45:06.399 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:45:06.397+0000 7fb3c2294700 1 --2- 172.21.15.152:0/1876693552 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb3bc0ffcf0 0x7fb3bc100110 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:45:06.399 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:45:06.397+0000 7fb3c2294700 1 --2- 172.21.15.152:0/1876693552 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3bc100ef0 0x7fb3bc101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:45:06.399 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:45:06.397+0000 7fb3c2294700 1 -- 172.21.15.152:0/1876693552 >> 172.21.15.152:0/1876693552 conn(0x7fb3bc0fb2b0 msgr2=0x7fb3bc0fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:45:06.399 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:45:06.397+0000 7fb3c2294700 1 -- 172.21.15.152:0/1876693552 shutdown_connections 2024-03-23T17:45:06.399 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:45:06.397+0000 7fb3c2294700 1 -- 172.21.15.152:0/1876693552 wait complete. 2024-03-23T17:45:06.399 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:45:06.397+0000 7fb3c2294700 1 Processor -- start 2024-03-23T17:45:06.399 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:45:06.397+0000 7fb3c2294700 1 -- start start 2024-03-23T17:45:06.399 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:45:06.398+0000 7fb3c2294700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3bc0ffcf0 0x7fb3bc103c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:45:06.399 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:45:06.398+0000 7fb3c2294700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb3bc100ef0 0x7fb3bc1022d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:45:06.399 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:45:06.398+0000 7fb3c2294700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3bc101890 0x7fb3bc102810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:45:06.399 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:45:06.398+0000 7fb3c2294700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3bc102fb0 con 0x7fb3bc0ffcf0 2024-03-23T17:45:06.399 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:45:06.398+0000 7fb3c2294700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3bc103120 con 0x7fb3bc101890 2024-03-23T17:45:06.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:45:06.398+0000 7fb3c2294700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3bc1a0f30 con 0x7fb3bc100ef0 2024-03-23T17:45:06.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:45:06.398+0000 7fb3bb7fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb3bc100ef0 0x7fb3bc1022d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:45:06.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:45:06.398+0000 7fb3bb7fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb3bc100ef0 0x7fb3bc1022d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:39862/0 (socket says 172.21.15.152:39862) 2024-03-23T17:45:06.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:45:06.398+0000 7fb3bb7fe700 1 -- 172.21.15.152:0/3765581185 learned_addr learned my addr 172.21.15.152:0/3765581185 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-03-23T17:45:06.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:45:06.398+0000 7fb3c0831700 1 --2- 172.21.15.152:0/3765581185 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3bc101890 0x7fb3bc102810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:45:06.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:45:06.398+0000 7fb3bb7fe700 1 -- 172.21.15.152:0/3765581185 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3bc101890 msgr2=0x7fb3bc102810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:45:06.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:45:06.398+0000 7fb3bb7fe700 1 --2- 172.21.15.152:0/3765581185 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3bc101890 0x7fb3bc102810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:45:06.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:45:06.398+0000 7fb3bb7fe700 1 -- 172.21.15.152:0/3765581185 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3bc0ffcf0 msgr2=0x7fb3bc103c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:45:06.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:45:06.398+0000 7fb3bb7fe700 1 --2- 172.21.15.152:0/3765581185 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3bc0ffcf0 0x7fb3bc103c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:45:06.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:45:06.398+0000 7fb3bb7fe700 1 -- 172.21.15.152:0/3765581185 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3b00095f0 con 0x7fb3bc100ef0 2024-03-23T17:45:06.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:45:06.398+0000 7fb3bb7fe700 1 --2- 172.21.15.152:0/3765581185 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb3bc100ef0 0x7fb3bc1022d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac00b530 tx=0x7fb3ac00b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:45:06.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:45:06.399+0000 7fb3b97fa700 1 -- 172.21.15.152:0/3765581185 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac011830 con 0x7fb3bc100ef0 2024-03-23T17:45:06.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:45:06.399+0000 7fb3c2294700 1 -- 172.21.15.152:0/3765581185 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3bc1a11c0 con 0x7fb3bc100ef0 2024-03-23T17:45:06.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:45:06.399+0000 7fb3b97fa700 1 -- 172.21.15.152:0/3765581185 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb3ac011e50 con 0x7fb3bc100ef0 2024-03-23T17:45:06.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:45:06.399+0000 7fb3b97fa700 1 -- 172.21.15.152:0/3765581185 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac00f2e0 con 0x7fb3bc100ef0 2024-03-23T17:45:06.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:45:06.399+0000 7fb3c2294700 1 -- 172.21.15.152:0/3765581185 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3bc1a1740 con 0x7fb3bc100ef0 2024-03-23T17:45:06.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:45:06.400+0000 7fb3c2294700 1 -- 172.21.15.152:0/3765581185 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3bc04ea80 con 0x7fb3bc100ef0 2024-03-23T17:45:06.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:45:06.401+0000 7fb3b97fa700 1 -- 172.21.15.152:0/3765581185 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 15) v1 ==== 135809+0+0 (secure 0 0 0) 0x7fb3ac010380 con 0x7fb3bc100ef0 2024-03-23T17:45:06.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:45:06.401+0000 7fb3b97fa700 1 --2- 172.21.15.152:0/3765581185 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fb3a40a1090 0x7fb3a40a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:45:06.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:45:06.401+0000 7fb3b97fa700 1 -- 172.21.15.152:0/3765581185 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2029+0+0 (secure 0 0 0) 0x7fb3ac0cb060 con 0x7fb3bc100ef0 2024-03-23T17:45:06.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:45:06.402+0000 7fb3bbfff700 1 --2- 172.21.15.152:0/3765581185 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fb3a40a1090 0x7fb3a40a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:45:06.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:45:06.402+0000 7fb3bbfff700 1 --2- 172.21.15.152:0/3765581185 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fb3a40a1090 0x7fb3a40a3550 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0003790 tx=0x7fb3b000bee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:45:06.404 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:45:06.405+0000 7fb3b97fa700 1 -- 172.21.15.152:0/3765581185 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fb3ac096160 con 0x7fb3bc100ef0 2024-03-23T17:45:06.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:45:06.609+0000 7fb3c2294700 1 -- 172.21.15.152:0/3765581185 --> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fb3bc068ed0 con 0x7fb3a40a1090 2024-03-23T17:45:06.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:06 smithi092 ceph-mon[79089]: Detected new or changed devices on smithi092 2024-03-23T17:45:06.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:06 smithi092 ceph-mon[79089]: Adjusting osd_memory_target on smithi092 to 17114M 2024-03-23T17:45:06.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:06 smithi092 ceph-mon[79089]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-03-23T17:45:06.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:06 smithi092 ceph-mon[79089]: osdmap e12: 2 total, 1 up, 2 in 2024-03-23T17:45:06.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:06 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T17:45:06.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:06 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T17:45:06.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:06 smithi152 ceph-mon[47949]: Detected new or changed devices on smithi092 2024-03-23T17:45:06.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:06 smithi152 ceph-mon[47949]: Adjusting osd_memory_target on smithi092 to 17114M 2024-03-23T17:45:06.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:06 smithi152 ceph-mon[47949]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-03-23T17:45:06.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:06 smithi152 ceph-mon[47949]: osdmap e12: 2 total, 1 up, 2 in 2024-03-23T17:45:06.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:06 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T17:45:06.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:06 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T17:45:07.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:06 smithi086 ceph-mon[38221]: Detected new or changed devices on smithi092 2024-03-23T17:45:07.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:06 smithi086 ceph-mon[38221]: Adjusting osd_memory_target on smithi092 to 17114M 2024-03-23T17:45:07.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:06 smithi086 ceph-mon[38221]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-03-23T17:45:07.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:06 smithi086 ceph-mon[38221]: osdmap e12: 2 total, 1 up, 2 in 2024-03-23T17:45:07.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:06 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T17:45:07.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:06 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T17:45:07.717 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:07 smithi152 ceph-mon[47949]: purged_snaps scrub starts 2024-03-23T17:45:07.718 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:07 smithi152 ceph-mon[47949]: purged_snaps scrub ok 2024-03-23T17:45:07.718 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:07 smithi152 ceph-mon[47949]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T17:45:07.718 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:07 smithi152 ceph-mon[47949]: osd.1 [v2:172.21.15.92:6800/3776252597,v1:172.21.15.92:6801/3776252597] boot 2024-03-23T17:45:07.718 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:07 smithi152 ceph-mon[47949]: osdmap e13: 2 total, 2 up, 2 in 2024-03-23T17:45:07.718 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:07 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T17:45:07.718 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:07 smithi152 ceph-mon[47949]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:45:07.718 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:07 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T17:45:07.718 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:07 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T17:45:07.718 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:07 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:45:07.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:07 smithi092 ceph-mon[79089]: purged_snaps scrub starts 2024-03-23T17:45:07.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:07 smithi092 ceph-mon[79089]: purged_snaps scrub ok 2024-03-23T17:45:07.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:07 smithi092 ceph-mon[79089]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T17:45:07.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:07 smithi092 ceph-mon[79089]: osd.1 [v2:172.21.15.92:6800/3776252597,v1:172.21.15.92:6801/3776252597] boot 2024-03-23T17:45:07.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:07 smithi092 ceph-mon[79089]: osdmap e13: 2 total, 2 up, 2 in 2024-03-23T17:45:07.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:07 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T17:45:07.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:07 smithi092 ceph-mon[79089]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:45:07.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:07 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T17:45:07.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:07 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T17:45:07.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:07 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:45:08.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:07 smithi086 ceph-mon[38221]: purged_snaps scrub starts 2024-03-23T17:45:08.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:07 smithi086 ceph-mon[38221]: purged_snaps scrub ok 2024-03-23T17:45:08.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:07 smithi086 ceph-mon[38221]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T17:45:08.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:07 smithi086 ceph-mon[38221]: osd.1 [v2:172.21.15.92:6800/3776252597,v1:172.21.15.92:6801/3776252597] boot 2024-03-23T17:45:08.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:07 smithi086 ceph-mon[38221]: osdmap e13: 2 total, 2 up, 2 in 2024-03-23T17:45:08.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:07 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T17:45:08.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:07 smithi086 ceph-mon[38221]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:45:08.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:07 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T17:45:08.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:07 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T17:45:08.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:07 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:45:08.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:08 smithi092 ceph-mon[79089]: osdmap e14: 2 total, 2 up, 2 in 2024-03-23T17:45:08.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:08 smithi092 ceph-mon[79089]: from='client.? 172.21.15.152:0/2822499042' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09808786-3bd3-4d11-8c7b-1736119dfbb4"}]: dispatch 2024-03-23T17:45:08.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:08 smithi092 ceph-mon[79089]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09808786-3bd3-4d11-8c7b-1736119dfbb4"}]: dispatch 2024-03-23T17:45:08.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:08 smithi092 ceph-mon[79089]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "09808786-3bd3-4d11-8c7b-1736119dfbb4"}]': finished 2024-03-23T17:45:08.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:08 smithi092 ceph-mon[79089]: osdmap e15: 3 total, 2 up, 3 in 2024-03-23T17:45:08.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:08 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T17:45:08.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:08 smithi152 ceph-mon[47949]: osdmap e14: 2 total, 2 up, 2 in 2024-03-23T17:45:08.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:08 smithi152 ceph-mon[47949]: from='client.? 172.21.15.152:0/2822499042' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09808786-3bd3-4d11-8c7b-1736119dfbb4"}]: dispatch 2024-03-23T17:45:08.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:08 smithi152 ceph-mon[47949]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09808786-3bd3-4d11-8c7b-1736119dfbb4"}]: dispatch 2024-03-23T17:45:08.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:08 smithi152 ceph-mon[47949]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "09808786-3bd3-4d11-8c7b-1736119dfbb4"}]': finished 2024-03-23T17:45:08.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:08 smithi152 ceph-mon[47949]: osdmap e15: 3 total, 2 up, 3 in 2024-03-23T17:45:08.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:08 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T17:45:09.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:08 smithi086 ceph-mon[38221]: osdmap e14: 2 total, 2 up, 2 in 2024-03-23T17:45:09.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:08 smithi086 ceph-mon[38221]: from='client.? 172.21.15.152:0/2822499042' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09808786-3bd3-4d11-8c7b-1736119dfbb4"}]: dispatch 2024-03-23T17:45:09.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:08 smithi086 ceph-mon[38221]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09808786-3bd3-4d11-8c7b-1736119dfbb4"}]: dispatch 2024-03-23T17:45:09.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:08 smithi086 ceph-mon[38221]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "09808786-3bd3-4d11-8c7b-1736119dfbb4"}]': finished 2024-03-23T17:45:09.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:08 smithi086 ceph-mon[38221]: osdmap e15: 3 total, 2 up, 3 in 2024-03-23T17:45:09.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:08 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T17:45:09.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:09 smithi152 ceph-mon[47949]: pgmap v54: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-23T17:45:09.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:09 smithi152 ceph-mon[47949]: from='client.? 172.21.15.152:0/2118871853' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T17:45:10.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:09 smithi086 ceph-mon[38221]: pgmap v54: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-23T17:45:10.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:09 smithi086 ceph-mon[38221]: from='client.? 172.21.15.152:0/2118871853' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T17:45:10.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:09 smithi092 ceph-mon[79089]: pgmap v54: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-23T17:45:10.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:09 smithi092 ceph-mon[79089]: from='client.? 172.21.15.152:0/2118871853' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T17:45:11.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:11 smithi152 ceph-mon[47949]: pgmap v56: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-23T17:45:12.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:11 smithi086 ceph-mon[38221]: pgmap v56: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-23T17:45:12.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:11 smithi092 ceph-mon[79089]: pgmap v56: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-23T17:45:13.790 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:13 smithi152 ceph-mon[47949]: pgmap v57: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-23T17:45:14.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:13 smithi086 ceph-mon[38221]: pgmap v57: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-23T17:45:14.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:13 smithi092 ceph-mon[79089]: pgmap v57: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-23T17:45:15.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:15 smithi152 ceph-mon[47949]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-23T17:45:15.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:15 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-23T17:45:15.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:15 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:45:15.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:15 smithi152 ceph-mon[47949]: Deploying daemon osd.2 on smithi152 2024-03-23T17:45:16.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:15 smithi086 ceph-mon[38221]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-23T17:45:16.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:15 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-23T17:45:16.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:15 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:45:16.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:15 smithi086 ceph-mon[38221]: Deploying daemon osd.2 on smithi152 2024-03-23T17:45:16.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:15 smithi092 ceph-mon[79089]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-23T17:45:16.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:15 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-23T17:45:16.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:15 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:45:16.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:15 smithi092 ceph-mon[79089]: Deploying daemon osd.2 on smithi152 2024-03-23T17:45:17.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:17 smithi152 ceph-mon[47949]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-23T17:45:18.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:17 smithi086 ceph-mon[38221]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-23T17:45:18.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:17 smithi092 ceph-mon[79089]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-23T17:45:19.499 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:19 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:19.499 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:19 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:19.499 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:19 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:45:19.499 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:19 smithi152 ceph-mon[47949]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-23T17:45:19.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:19 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:19.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:19 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:19.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:19 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:45:19.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:19 smithi086 ceph-mon[38221]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-23T17:45:19.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:19 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:19.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:19 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:19.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:19 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:45:19.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:19 smithi092 ceph-mon[79089]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-23T17:45:20.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:45:20.416+0000 7fb3b97fa700 1 -- 172.21.15.152:0/3765581185 <== mgr.14152 v2:172.21.15.86:6800/2734784244 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fb3bc068ed0 con 0x7fb3a40a1090 2024-03-23T17:45:20.417 INFO:teuthology.orchestra.run.smithi152.stdout:Created osd(s) 2 on host 'smithi152' 2024-03-23T17:45:20.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:45:20.418+0000 7fb3c2294700 1 -- 172.21.15.152:0/3765581185 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fb3a40a1090 msgr2=0x7fb3a40a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:45:20.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:45:20.418+0000 7fb3c2294700 1 --2- 172.21.15.152:0/3765581185 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fb3a40a1090 0x7fb3a40a3550 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0003790 tx=0x7fb3b000bee0 comp rx=0 tx=0).stop 2024-03-23T17:45:20.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:45:20.418+0000 7fb3c2294700 1 -- 172.21.15.152:0/3765581185 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb3bc100ef0 msgr2=0x7fb3bc1022d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:45:20.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:45:20.418+0000 7fb3c2294700 1 --2- 172.21.15.152:0/3765581185 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb3bc100ef0 0x7fb3bc1022d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac00b530 tx=0x7fb3ac00b840 comp rx=0 tx=0).stop 2024-03-23T17:45:20.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:45:20.418+0000 7fb3c2294700 1 -- 172.21.15.152:0/3765581185 shutdown_connections 2024-03-23T17:45:20.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:45:20.418+0000 7fb3c2294700 1 --2- 172.21.15.152:0/3765581185 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3bc101890 0x7fb3bc102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:45:20.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:45:20.418+0000 7fb3c2294700 1 --2- 172.21.15.152:0/3765581185 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fb3a40a1090 0x7fb3a40a3550 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:45:20.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:45:20.418+0000 7fb3c2294700 1 --2- 172.21.15.152:0/3765581185 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3bc0ffcf0 0x7fb3bc103c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:45:20.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:45:20.418+0000 7fb3c2294700 1 --2- 172.21.15.152:0/3765581185 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb3bc100ef0 0x7fb3bc1022d0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:45:20.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:45:20.418+0000 7fb3c2294700 1 -- 172.21.15.152:0/3765581185 >> 172.21.15.152:0/3765581185 conn(0x7fb3bc0fb2b0 msgr2=0x7fb3bc0687c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:45:20.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:45:20.418+0000 7fb3c2294700 1 -- 172.21.15.152:0/3765581185 shutdown_connections 2024-03-23T17:45:20.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:45:20.418+0000 7fb3c2294700 1 -- 172.21.15.152:0/3765581185 wait complete. 2024-03-23T17:45:20.581 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:20 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:20.581 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:20 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:20.581 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:20 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:45:20.581 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:20 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:45:20.582 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:20 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:20.582 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:20 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:20.582 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:20 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:45:20.582 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:20 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:20.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:20 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:20.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:20 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:20.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:20 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:45:20.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:20 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:45:20.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:20 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:20.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:20 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:20.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:20 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:45:20.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:20 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:20.783 DEBUG:teuthology.orchestra.run.smithi152:osd.2> sudo journalctl -f -n 0 -u ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@osd.2.service 2024-03-23T17:45:20.785 INFO:tasks.cephadm:Deploying osd.3 on smithi086 with /dev/vg_nvme/lv_3... 2024-03-23T17:45:20.785 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-23T17:45:20.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:20 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:20.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:20 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:20.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:20 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:45:20.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:20 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:45:20.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:20 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:20.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:20 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:20.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:20 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:45:20.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:20 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:20.814 INFO:journalctl@ceph.osd.2.smithi152.stdout:-- Logs begin at Sat 2024-03-23 17:19:34 UTC. -- 2024-03-23T17:45:21.146 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T17:45:21.669 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:21 smithi086 ceph-mon[38221]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-23T17:45:21.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:21 smithi092 ceph-mon[79089]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-23T17:45:21.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:21 smithi152 ceph-mon[47949]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-23T17:45:22.532 INFO:journalctl@ceph.osd.2.smithi152.stdout:Mar 23 17:45:22 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-2[53792]: 2024-03-23T17:45:22.194+0000 7f5ed2c8b640 -1 osd.2 0 log_to_monitors true 2024-03-23T17:45:22.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:22 smithi092 ceph-mon[79089]: from='osd.2 [v2:172.21.15.152:6800/1601265891,v1:172.21.15.152:6801/1601265891]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-23T17:45:22.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:22 smithi092 ceph-mon[79089]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-23T17:45:22.817 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:22 smithi152 ceph-mon[47949]: from='osd.2 [v2:172.21.15.152:6800/1601265891,v1:172.21.15.152:6801/1601265891]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-23T17:45:22.817 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:22 smithi152 ceph-mon[47949]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-23T17:45:22.908 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-23T17:45:22.908 INFO:teuthology.orchestra.run.smithi086.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-23T17:45:22.908 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 10+0 records in 2024-03-23T17:45:22.908 INFO:teuthology.orchestra.run.smithi086.stderr:10+0 records out 2024-03-23T17:45:22.909 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0298178 s, 352 MB/s 2024-03-23T17:45:22.909 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping successful for: 2024-03-23T17:45:22.945 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:22 smithi086 ceph-mon[38221]: from='osd.2 [v2:172.21.15.152:6800/1601265891,v1:172.21.15.152:6801/1601265891]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-23T17:45:22.946 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:22 smithi086 ceph-mon[38221]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-23T17:45:23.289 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi086:vg_nvme/lv_3 2024-03-23T17:45:23.638 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T17:45:23.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:23 smithi092 ceph-mon[79089]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-23T17:45:23.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:23 smithi092 ceph-mon[79089]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-23T17:45:23.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:23 smithi092 ceph-mon[79089]: osdmap e16: 3 total, 2 up, 3 in 2024-03-23T17:45:23.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:23 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T17:45:23.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:23 smithi092 ceph-mon[79089]: from='osd.2 [v2:172.21.15.152:6800/1601265891,v1:172.21.15.152:6801/1601265891]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-03-23T17:45:23.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:23 smithi092 ceph-mon[79089]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-03-23T17:45:23.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:23 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:23.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:23 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:23.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:23 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:45:23.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:23 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:23.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:23 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:45:23.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:23 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:45:23.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:23 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:23.817 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:23 smithi086 ceph-mon[38221]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-23T17:45:23.818 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:23 smithi086 ceph-mon[38221]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-23T17:45:23.818 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:23 smithi086 ceph-mon[38221]: osdmap e16: 3 total, 2 up, 3 in 2024-03-23T17:45:23.818 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:23 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T17:45:23.818 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:23 smithi086 ceph-mon[38221]: from='osd.2 [v2:172.21.15.152:6800/1601265891,v1:172.21.15.152:6801/1601265891]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-03-23T17:45:23.818 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:23 smithi086 ceph-mon[38221]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-03-23T17:45:23.818 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:23 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:23.818 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:23 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:23.818 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:23 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:45:23.818 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:23 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:23.818 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:23 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:45:23.818 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:23 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:45:23.818 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:23 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:23.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:23 smithi152 ceph-mon[47949]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-23T17:45:23.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:23 smithi152 ceph-mon[47949]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-23T17:45:23.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:23 smithi152 ceph-mon[47949]: osdmap e16: 3 total, 2 up, 3 in 2024-03-23T17:45:23.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:23 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T17:45:23.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:23 smithi152 ceph-mon[47949]: from='osd.2 [v2:172.21.15.152:6800/1601265891,v1:172.21.15.152:6801/1601265891]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-03-23T17:45:23.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:23 smithi152 ceph-mon[47949]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-03-23T17:45:23.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:23 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:23.837 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:23 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:23.837 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:23 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:45:23.837 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:23 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:23.837 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:23 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:45:23.837 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:23 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:45:23.837 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:23 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:24.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:45:24.743+0000 7f62cf000700 1 -- 172.21.15.86:0/1500210117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62c8073810 msgr2=0x7f62c8073c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:45:24.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:45:24.743+0000 7f62cf000700 1 --2- 172.21.15.86:0/1500210117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62c8073810 0x7f62c8073c90 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f62b4009990 tx=0x7f62b4009ca0 comp rx=0 tx=0).stop 2024-03-23T17:45:24.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:45:24.743+0000 7f62cf000700 1 -- 172.21.15.86:0/1500210117 shutdown_connections 2024-03-23T17:45:24.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:45:24.743+0000 7f62cf000700 1 --2- 172.21.15.86:0/1500210117 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62c8108cd0 0x7f62c810b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:45:24.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:45:24.743+0000 7f62cf000700 1 --2- 172.21.15.86:0/1500210117 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f62c8074de0 0x7f62c8073240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:45:24.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:45:24.743+0000 7f62cf000700 1 --2- 172.21.15.86:0/1500210117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62c8073810 0x7f62c8073c90 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:45:24.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:45:24.743+0000 7f62cf000700 1 -- 172.21.15.86:0/1500210117 >> 172.21.15.86:0/1500210117 conn(0x7f62c80fc3c0 msgr2=0x7f62c80fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:45:24.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:45:24.743+0000 7f62cf000700 1 -- 172.21.15.86:0/1500210117 shutdown_connections 2024-03-23T17:45:24.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:45:24.743+0000 7f62cf000700 1 -- 172.21.15.86:0/1500210117 wait complete. 2024-03-23T17:45:24.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:45:24.744+0000 7f62cf000700 1 Processor -- start 2024-03-23T17:45:24.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:45:24.744+0000 7f62cf000700 1 -- start start 2024-03-23T17:45:24.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:45:24.744+0000 7f62cf000700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f62c8073810 0x7f62c8198fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:45:24.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:45:24.744+0000 7f62cf000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62c8074de0 0x7f62c8199500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:45:24.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:45:24.744+0000 7f62cf000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62c8108cd0 0x7f62c81a0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:45:24.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:45:24.744+0000 7f62cf000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62c81a0b00 con 0x7f62c8074de0 2024-03-23T17:45:24.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:45:24.744+0000 7f62cf000700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62c81a0c40 con 0x7f62c8108cd0 2024-03-23T17:45:24.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:45:24.744+0000 7f62cf000700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62c81a0dd0 con 0x7f62c8073810 2024-03-23T17:45:24.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:45:24.745+0000 7f62c7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62c8074de0 0x7f62c8199500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:45:24.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:45:24.745+0000 7f62c7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62c8074de0 0x7f62c8199500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39400/0 (socket says 172.21.15.86:39400) 2024-03-23T17:45:24.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:45:24.745+0000 7f62c7400700 1 -- 172.21.15.86:0/1034987609 learned_addr learned my addr 172.21.15.86:0/1034987609 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:45:24.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:45:24.745+0000 7f62c7e00700 1 --2- 172.21.15.86:0/1034987609 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f62c8073810 0x7f62c8198fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:45:24.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:45:24.745+0000 7f62cd000700 1 --2- 172.21.15.86:0/1034987609 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62c8108cd0 0x7f62c81a0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:45:24.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:45:24.745+0000 7f62c7400700 1 -- 172.21.15.86:0/1034987609 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f62c8073810 msgr2=0x7f62c8198fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:45:24.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:45:24.745+0000 7f62c7400700 1 --2- 172.21.15.86:0/1034987609 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f62c8073810 0x7f62c8198fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:45:24.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:45:24.745+0000 7f62c7400700 1 -- 172.21.15.86:0/1034987609 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62c8108cd0 msgr2=0x7f62c81a0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:45:24.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:45:24.745+0000 7f62c7400700 1 --2- 172.21.15.86:0/1034987609 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62c8108cd0 0x7f62c81a0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:45:24.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:45:24.745+0000 7f62c7400700 1 -- 172.21.15.86:0/1034987609 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62b40095f0 con 0x7f62c8074de0 2024-03-23T17:45:24.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:45:24.745+0000 7f62c7400700 1 --2- 172.21.15.86:0/1034987609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62c8074de0 0x7f62c8199500 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f62b40037f0 tx=0x7f62b400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:45:24.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:45:24.746+0000 7f62c4c00700 1 -- 172.21.15.86:0/1034987609 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62b401d070 con 0x7f62c8074de0 2024-03-23T17:45:24.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:45:24.746+0000 7f62c4c00700 1 -- 172.21.15.86:0/1034987609 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f62b4005030 con 0x7f62c8074de0 2024-03-23T17:45:24.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:45:24.746+0000 7f62cf000700 1 -- 172.21.15.86:0/1034987609 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62c81a1050 con 0x7f62c8074de0 2024-03-23T17:45:24.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:45:24.746+0000 7f62c4c00700 1 -- 172.21.15.86:0/1034987609 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62b4004b40 con 0x7f62c8074de0 2024-03-23T17:45:24.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:45:24.746+0000 7f62cf000700 1 -- 172.21.15.86:0/1034987609 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62c81a14c0 con 0x7f62c8074de0 2024-03-23T17:45:24.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:45:24.747+0000 7f62cf000700 1 -- 172.21.15.86:0/1034987609 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62c8107a90 con 0x7f62c8074de0 2024-03-23T17:45:24.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:45:24.749+0000 7f62c4c00700 1 -- 172.21.15.86:0/1034987609 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 15) v1 ==== 135809+0+0 (secure 0 0 0) 0x7f62b4004ca0 con 0x7f62c8074de0 2024-03-23T17:45:24.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:45:24.750+0000 7f62c4c00700 1 --2- 172.21.15.86:0/1034987609 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f62ac0a1110 0x7f62ac0a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:45:24.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:45:24.750+0000 7f62c4c00700 1 -- 172.21.15.86:0/1034987609 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 2571+0+0 (secure 0 0 0) 0x7f62b40cc450 con 0x7f62c8074de0 2024-03-23T17:45:24.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:45:24.750+0000 7f62c7e00700 1 --2- 172.21.15.86:0/1034987609 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f62ac0a1110 0x7f62ac0a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:45:24.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:45:24.751+0000 7f62c7e00700 1 --2- 172.21.15.86:0/1034987609 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f62ac0a1110 0x7f62ac0a35d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f62b8005950 tx=0x7f62b800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:45:24.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:45:24.753+0000 7f62c4c00700 1 -- 172.21.15.86:0/1034987609 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f62b4097320 con 0x7f62c8074de0 2024-03-23T17:45:24.766 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:24 smithi086 ceph-mon[38221]: Detected new or changed devices on smithi152 2024-03-23T17:45:24.766 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:24 smithi086 ceph-mon[38221]: Adjusting osd_memory_target on smithi152 to 17114M 2024-03-23T17:45:24.766 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:24 smithi086 ceph-mon[38221]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]': finished 2024-03-23T17:45:24.766 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:24 smithi086 ceph-mon[38221]: osdmap e17: 3 total, 2 up, 3 in 2024-03-23T17:45:24.766 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:24 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T17:45:24.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:24 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T17:45:24.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:24 smithi086 ceph-mon[38221]: from='osd.2 ' entity='osd.2' 2024-03-23T17:45:24.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:24 smithi092 ceph-mon[79089]: Detected new or changed devices on smithi152 2024-03-23T17:45:24.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:24 smithi092 ceph-mon[79089]: Adjusting osd_memory_target on smithi152 to 17114M 2024-03-23T17:45:24.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:24 smithi092 ceph-mon[79089]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]': finished 2024-03-23T17:45:24.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:24 smithi092 ceph-mon[79089]: osdmap e17: 3 total, 2 up, 3 in 2024-03-23T17:45:24.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:24 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T17:45:24.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:24 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T17:45:24.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:24 smithi092 ceph-mon[79089]: from='osd.2 ' entity='osd.2' 2024-03-23T17:45:24.836 INFO:journalctl@ceph.osd.2.smithi152.stdout:Mar 23 17:45:24 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-2[53792]: 2024-03-23T17:45:24.390+0000 7f5ec7aee700 -1 osd.2 0 waiting for initial osdmap 2024-03-23T17:45:24.836 INFO:journalctl@ceph.osd.2.smithi152.stdout:Mar 23 17:45:24 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-2[53792]: 2024-03-23T17:45:24.398+0000 7f5ec00dc700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T17:45:24.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:24 smithi152 ceph-mon[47949]: Detected new or changed devices on smithi152 2024-03-23T17:45:24.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:24 smithi152 ceph-mon[47949]: Adjusting osd_memory_target on smithi152 to 17114M 2024-03-23T17:45:24.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:24 smithi152 ceph-mon[47949]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]': finished 2024-03-23T17:45:24.837 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:24 smithi152 ceph-mon[47949]: osdmap e17: 3 total, 2 up, 3 in 2024-03-23T17:45:24.837 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:24 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T17:45:24.837 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:24 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T17:45:24.837 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:24 smithi152 ceph-mon[47949]: from='osd.2 ' entity='osd.2' 2024-03-23T17:45:24.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:45:24.965+0000 7f62cf000700 1 -- 172.21.15.86:0/1034987609 --> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f62c80611f0 con 0x7f62ac0a1110 2024-03-23T17:45:25.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:25 smithi152 ceph-mon[47949]: purged_snaps scrub starts 2024-03-23T17:45:25.837 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:25 smithi152 ceph-mon[47949]: purged_snaps scrub ok 2024-03-23T17:45:25.837 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:25 smithi152 ceph-mon[47949]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-23T17:45:25.837 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:25 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T17:45:25.837 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:25 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T17:45:25.837 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:25 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T17:45:25.837 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:25 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:45:25.837 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:25 smithi152 ceph-mon[47949]: osd.2 [v2:172.21.15.152:6800/1601265891,v1:172.21.15.152:6801/1601265891] boot 2024-03-23T17:45:25.837 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:25 smithi152 ceph-mon[47949]: osdmap e18: 3 total, 3 up, 3 in 2024-03-23T17:45:25.837 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:25 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T17:45:25.852 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:25 smithi086 ceph-mon[38221]: purged_snaps scrub starts 2024-03-23T17:45:25.852 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:25 smithi086 ceph-mon[38221]: purged_snaps scrub ok 2024-03-23T17:45:25.852 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:25 smithi086 ceph-mon[38221]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-23T17:45:25.852 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:25 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T17:45:25.852 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:25 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T17:45:25.852 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:25 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T17:45:25.852 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:25 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:45:25.852 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:25 smithi086 ceph-mon[38221]: osd.2 [v2:172.21.15.152:6800/1601265891,v1:172.21.15.152:6801/1601265891] boot 2024-03-23T17:45:25.853 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:25 smithi086 ceph-mon[38221]: osdmap e18: 3 total, 3 up, 3 in 2024-03-23T17:45:25.853 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:25 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T17:45:26.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:25 smithi092 ceph-mon[79089]: purged_snaps scrub starts 2024-03-23T17:45:26.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:25 smithi092 ceph-mon[79089]: purged_snaps scrub ok 2024-03-23T17:45:26.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:25 smithi092 ceph-mon[79089]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-23T17:45:26.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:25 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T17:45:26.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:25 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T17:45:26.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:25 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T17:45:26.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:25 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:45:26.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:25 smithi092 ceph-mon[79089]: osd.2 [v2:172.21.15.152:6800/1601265891,v1:172.21.15.152:6801/1601265891] boot 2024-03-23T17:45:26.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:25 smithi092 ceph-mon[79089]: osdmap e18: 3 total, 3 up, 3 in 2024-03-23T17:45:26.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:25 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T17:45:27.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:26 smithi086 ceph-mon[38221]: from='client.14259 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:45:27.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:26 smithi092 ceph-mon[79089]: from='client.14259 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:45:27.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:26 smithi152 ceph-mon[47949]: from='client.14259 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:45:28.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:27 smithi086 ceph-mon[38221]: pgmap v67: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-23T17:45:28.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:27 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-23T17:45:28.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:27 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-23T17:45:28.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:27 smithi086 ceph-mon[38221]: osdmap e19: 3 total, 3 up, 3 in 2024-03-23T17:45:28.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:27 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-23T17:45:28.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:27 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/2023552224' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79c61910-4b53-424c-b150-6f5385a2cd68"}]: dispatch 2024-03-23T17:45:28.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:27 smithi086 ceph-mon[38221]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79c61910-4b53-424c-b150-6f5385a2cd68"}]: dispatch 2024-03-23T17:45:28.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:27 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-23T17:45:28.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:27 smithi086 ceph-mon[38221]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "79c61910-4b53-424c-b150-6f5385a2cd68"}]': finished 2024-03-23T17:45:28.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:27 smithi086 ceph-mon[38221]: osdmap e20: 4 total, 3 up, 4 in 2024-03-23T17:45:28.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:27 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T17:45:28.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:27 smithi092 ceph-mon[79089]: pgmap v67: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-23T17:45:28.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:27 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-23T17:45:28.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:27 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-23T17:45:28.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:27 smithi092 ceph-mon[79089]: osdmap e19: 3 total, 3 up, 3 in 2024-03-23T17:45:28.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:27 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-23T17:45:28.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:27 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/2023552224' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79c61910-4b53-424c-b150-6f5385a2cd68"}]: dispatch 2024-03-23T17:45:28.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:27 smithi092 ceph-mon[79089]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79c61910-4b53-424c-b150-6f5385a2cd68"}]: dispatch 2024-03-23T17:45:28.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:27 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-23T17:45:28.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:27 smithi092 ceph-mon[79089]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "79c61910-4b53-424c-b150-6f5385a2cd68"}]': finished 2024-03-23T17:45:28.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:27 smithi092 ceph-mon[79089]: osdmap e20: 4 total, 3 up, 4 in 2024-03-23T17:45:28.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:27 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T17:45:28.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:27 smithi152 ceph-mon[47949]: pgmap v67: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-23T17:45:28.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:27 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-23T17:45:28.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:27 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-23T17:45:28.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:27 smithi152 ceph-mon[47949]: osdmap e19: 3 total, 3 up, 3 in 2024-03-23T17:45:28.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:27 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-23T17:45:28.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:27 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/2023552224' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79c61910-4b53-424c-b150-6f5385a2cd68"}]: dispatch 2024-03-23T17:45:28.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:27 smithi152 ceph-mon[47949]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79c61910-4b53-424c-b150-6f5385a2cd68"}]: dispatch 2024-03-23T17:45:28.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:27 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-23T17:45:28.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:27 smithi152 ceph-mon[47949]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "79c61910-4b53-424c-b150-6f5385a2cd68"}]': finished 2024-03-23T17:45:28.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:27 smithi152 ceph-mon[47949]: osdmap e20: 4 total, 3 up, 4 in 2024-03-23T17:45:28.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:27 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T17:45:29.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:28 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/2043755279' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T17:45:29.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:28 smithi086 ceph-mon[38221]: pgmap v70: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-23T17:45:29.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:28 smithi086 ceph-mon[38221]: osdmap e21: 4 total, 3 up, 4 in 2024-03-23T17:45:29.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:28 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T17:45:29.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:28 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/2043755279' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T17:45:29.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:28 smithi092 ceph-mon[79089]: pgmap v70: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-23T17:45:29.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:28 smithi092 ceph-mon[79089]: osdmap e21: 4 total, 3 up, 4 in 2024-03-23T17:45:29.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:28 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T17:45:29.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:28 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/2043755279' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T17:45:29.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:28 smithi152 ceph-mon[47949]: pgmap v70: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-23T17:45:29.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:28 smithi152 ceph-mon[47949]: osdmap e21: 4 total, 3 up, 4 in 2024-03-23T17:45:29.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:28 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T17:45:29.766 INFO:journalctl@ceph.osd.0.smithi086.stdout:Mar 23 17:45:29 smithi086 sudo[63335]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-23T17:45:29.767 INFO:journalctl@ceph.osd.0.smithi086.stdout:Mar 23 17:45:29 smithi086 sudo[63335]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-23T17:45:29.767 INFO:journalctl@ceph.osd.0.smithi086.stdout:Mar 23 17:45:29 smithi086 sudo[63335]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-23T17:45:29.767 INFO:journalctl@ceph.osd.0.smithi086.stdout:Mar 23 17:45:29 smithi086 sudo[63335]: pam_unix(sudo:session): session closed for user root 2024-03-23T17:45:30.266 INFO:journalctl@ceph.osd.0.smithi086.stdout:Mar 23 17:45:29 smithi086 sudo[63711]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-23T17:45:30.267 INFO:journalctl@ceph.osd.0.smithi086.stdout:Mar 23 17:45:29 smithi086 sudo[63711]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-23T17:45:30.267 INFO:journalctl@ceph.osd.0.smithi086.stdout:Mar 23 17:45:29 smithi086 sudo[63711]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-23T17:45:30.267 INFO:journalctl@ceph.osd.0.smithi086.stdout:Mar 23 17:45:29 smithi086 sudo[63711]: pam_unix(sudo:session): session closed for user root 2024-03-23T17:45:30.369 INFO:journalctl@ceph.osd.1.smithi092.stdout:Mar 23 17:45:30 smithi092 sudo[89718]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-23T17:45:30.369 INFO:journalctl@ceph.osd.1.smithi092.stdout:Mar 23 17:45:30 smithi092 sudo[89718]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-23T17:45:30.369 INFO:journalctl@ceph.osd.1.smithi092.stdout:Mar 23 17:45:30 smithi092 sudo[89718]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-23T17:45:30.369 INFO:journalctl@ceph.osd.1.smithi092.stdout:Mar 23 17:45:30 smithi092 sudo[89718]: pam_unix(sudo:session): session closed for user root 2024-03-23T17:45:30.369 INFO:journalctl@ceph.osd.1.smithi092.stdout:Mar 23 17:45:30 smithi092 sudo[89721]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-23T17:45:30.370 INFO:journalctl@ceph.osd.1.smithi092.stdout:Mar 23 17:45:30 smithi092 sudo[89721]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-23T17:45:30.370 INFO:journalctl@ceph.osd.1.smithi092.stdout:Mar 23 17:45:30 smithi092 sudo[89721]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-23T17:45:30.370 INFO:journalctl@ceph.osd.1.smithi092.stdout:Mar 23 17:45:30 smithi092 sudo[89721]: pam_unix(sudo:session): session closed for user root 2024-03-23T17:45:30.631 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:30 smithi152 ceph-mon[47949]: osdmap e22: 4 total, 3 up, 4 in 2024-03-23T17:45:30.631 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:30 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T17:45:30.631 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:30 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:30.766 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:30 smithi086 ceph-mon[38221]: osdmap e22: 4 total, 3 up, 4 in 2024-03-23T17:45:30.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:30 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T17:45:30.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:30 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:30.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:30 smithi092 ceph-mon[79089]: osdmap e22: 4 total, 3 up, 4 in 2024-03-23T17:45:30.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:30 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T17:45:30.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:30 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:30.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:45:30.895+0000 7f62c4c00700 1 -- 172.21.15.86:0/1034987609 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f62b40976f0 con 0x7f62c8074de0 2024-03-23T17:45:31.086 INFO:journalctl@ceph.osd.2.smithi152.stdout:Mar 23 17:45:30 smithi152 sudo[58473]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-23T17:45:31.086 INFO:journalctl@ceph.osd.2.smithi152.stdout:Mar 23 17:45:30 smithi152 sudo[58473]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-23T17:45:31.086 INFO:journalctl@ceph.osd.2.smithi152.stdout:Mar 23 17:45:30 smithi152 sudo[58473]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-23T17:45:31.086 INFO:journalctl@ceph.osd.2.smithi152.stdout:Mar 23 17:45:30 smithi152 sudo[58473]: pam_unix(sudo:session): session closed for user root 2024-03-23T17:45:31.086 INFO:journalctl@ceph.osd.2.smithi152.stdout:Mar 23 17:45:30 smithi152 sudo[58476]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-23T17:45:31.086 INFO:journalctl@ceph.osd.2.smithi152.stdout:Mar 23 17:45:30 smithi152 sudo[58476]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-23T17:45:31.086 INFO:journalctl@ceph.osd.2.smithi152.stdout:Mar 23 17:45:30 smithi152 sudo[58476]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-23T17:45:31.086 INFO:journalctl@ceph.osd.2.smithi152.stdout:Mar 23 17:45:30 smithi152 sudo[58476]: pam_unix(sudo:session): session closed for user root 2024-03-23T17:45:31.751 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:31 smithi086 sudo[64293]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-23T17:45:31.751 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:31 smithi086 sudo[64293]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-23T17:45:31.751 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:31 smithi086 sudo[64293]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-23T17:45:31.751 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:31 smithi086 sudo[64293]: pam_unix(sudo:session): session closed for user root 2024-03-23T17:45:32.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:32 smithi086 sudo[64477]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-23T17:45:32.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:32 smithi086 sudo[64477]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-23T17:45:32.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:32 smithi086 sudo[64477]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-23T17:45:32.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:32 smithi086 sudo[64477]: pam_unix(sudo:session): session closed for user root 2024-03-23T17:45:32.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:32 smithi086 ceph-mon[38221]: pgmap v73: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-23T17:45:32.268 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:32 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:32.268 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:32 smithi086 ceph-mon[38221]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-23T17:45:32.268 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:32 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:32.268 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:32 smithi086 ceph-mon[38221]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-23T17:45:32.278 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:32 smithi152 ceph-mon[47949]: pgmap v73: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-23T17:45:32.279 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:32 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:32.279 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:32 smithi152 ceph-mon[47949]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-23T17:45:32.279 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:32 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:32.279 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:32 smithi152 ceph-mon[47949]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-23T17:45:32.279 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:32 smithi152 sudo[58479]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-23T17:45:32.279 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:32 smithi152 sudo[58479]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-23T17:45:32.280 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:32 smithi152 sudo[58479]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-23T17:45:32.280 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:32 smithi152 sudo[58479]: pam_unix(sudo:session): session closed for user root 2024-03-23T17:45:32.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:32 smithi092 ceph-mon[79089]: pgmap v73: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-23T17:45:32.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:32 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:32.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:32 smithi092 ceph-mon[79089]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-23T17:45:32.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:32 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:32.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:32 smithi092 ceph-mon[79089]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-23T17:45:32.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:32 smithi152 sudo[58482]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-23T17:45:32.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:32 smithi152 sudo[58482]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-23T17:45:32.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:32 smithi152 sudo[58482]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-23T17:45:32.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:32 smithi152 sudo[58482]: pam_unix(sudo:session): session closed for user root 2024-03-23T17:45:33.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:32 smithi092 sudo[89724]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-23T17:45:33.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:32 smithi092 sudo[89724]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-23T17:45:33.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:32 smithi092 sudo[89724]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-23T17:45:33.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:32 smithi092 sudo[89724]: pam_unix(sudo:session): session closed for user root 2024-03-23T17:45:33.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:32 smithi092 sudo[89727]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-23T17:45:33.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:32 smithi092 sudo[89727]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-23T17:45:33.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:32 smithi092 sudo[89727]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-23T17:45:33.032 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:32 smithi092 sudo[89727]: pam_unix(sudo:session): session closed for user root 2024-03-23T17:45:33.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:33 smithi086 ceph-mon[38221]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-23T17:45:33.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:33 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-23T17:45:33.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:33 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T17:45:33.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:33 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T17:45:33.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:33 smithi086 ceph-mon[38221]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-23T17:45:33.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:33 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-23T17:45:33.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:33 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T17:45:33.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:33 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T17:45:33.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:33 smithi086 ceph-mon[38221]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-03-23T17:45:33.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:33 smithi086 ceph-mon[38221]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-23T17:45:33.268 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:33 smithi086 ceph-mon[38221]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-23T17:45:33.268 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:33 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-23T17:45:33.268 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:33 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T17:45:33.268 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:33 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T17:45:33.268 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:33 smithi086 ceph-mon[38221]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-23T17:45:33.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:33 smithi092 ceph-mon[79089]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-23T17:45:33.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:33 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-23T17:45:33.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:33 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T17:45:33.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:33 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T17:45:33.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:33 smithi092 ceph-mon[79089]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-23T17:45:33.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:33 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-23T17:45:33.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:33 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T17:45:33.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:33 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T17:45:33.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:33 smithi092 ceph-mon[79089]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-03-23T17:45:33.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:33 smithi092 ceph-mon[79089]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-23T17:45:33.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:33 smithi092 ceph-mon[79089]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-23T17:45:33.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:33 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-23T17:45:33.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:33 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T17:45:33.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:33 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T17:45:33.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:33 smithi092 ceph-mon[79089]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-23T17:45:33.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:33 smithi152 ceph-mon[47949]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-23T17:45:33.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:33 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-23T17:45:33.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:33 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T17:45:33.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:33 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T17:45:33.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:33 smithi152 ceph-mon[47949]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-23T17:45:33.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:33 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-23T17:45:33.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:33 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T17:45:33.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:33 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T17:45:33.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:33 smithi152 ceph-mon[47949]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-03-23T17:45:33.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:33 smithi152 ceph-mon[47949]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-23T17:45:33.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:33 smithi152 ceph-mon[47949]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-23T17:45:33.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:33 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-23T17:45:33.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:33 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T17:45:33.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:33 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T17:45:33.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:33 smithi152 ceph-mon[47949]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-23T17:45:34.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:34 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-23T17:45:34.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:34 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:45:34.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:34 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-23T17:45:34.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:34 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:45:34.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:34 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-23T17:45:34.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:34 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:45:35.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:35 smithi086 ceph-mon[38221]: Deploying daemon osd.3 on smithi086 2024-03-23T17:45:35.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:35 smithi086 ceph-mon[38221]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-03-23T17:45:35.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:35 smithi092 ceph-mon[79089]: Deploying daemon osd.3 on smithi086 2024-03-23T17:45:35.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:35 smithi092 ceph-mon[79089]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-03-23T17:45:35.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:35 smithi152 ceph-mon[47949]: Deploying daemon osd.3 on smithi086 2024-03-23T17:45:35.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:35 smithi152 ceph-mon[47949]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-03-23T17:45:37.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:37 smithi086 ceph-mon[38221]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-03-23T17:45:37.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:37 smithi092 ceph-mon[79089]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-03-23T17:45:37.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:37 smithi152 ceph-mon[47949]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-03-23T17:45:39.791 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:39 smithi086 ceph-mon[38221]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-03-23T17:45:39.791 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:39 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:39.791 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:39 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:39.791 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:39 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:45:39.809 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:39 smithi092 ceph-mon[79089]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-03-23T17:45:39.809 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:39 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:39.809 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:39 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:39.810 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:39 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:45:39.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:39 smithi152 ceph-mon[47949]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-03-23T17:45:39.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:39 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:39.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:39 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:39.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:39 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:45:41.781 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:41 smithi086 ceph-mon[38221]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-03-23T17:45:41.782 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:41 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:41.782 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:41 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:41.782 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:41 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:45:41.782 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:41 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:45:41.782 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:41 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:41.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:41 smithi092 ceph-mon[79089]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-03-23T17:45:41.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:41 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:41.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:41 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:41.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:41 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:45:41.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:41 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:45:41.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:41 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:41.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:41 smithi152 ceph-mon[47949]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-03-23T17:45:41.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:41 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:41.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:41 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:41.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:41 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:45:41.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:41 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:45:41.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:41 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:41.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:45:41.963+0000 7f62c4c00700 1 -- 172.21.15.86:0/1034987609 <== mgr.14152 v2:172.21.15.86:6800/2734784244 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f62c80611f0 con 0x7f62ac0a1110 2024-03-23T17:45:41.963 INFO:teuthology.orchestra.run.smithi086.stdout:Created osd(s) 3 on host 'smithi086' 2024-03-23T17:45:41.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:45:41.965+0000 7f62cf000700 1 -- 172.21.15.86:0/1034987609 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f62ac0a1110 msgr2=0x7f62ac0a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:45:41.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:45:41.965+0000 7f62cf000700 1 --2- 172.21.15.86:0/1034987609 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f62ac0a1110 0x7f62ac0a35d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f62b8005950 tx=0x7f62b800b410 comp rx=0 tx=0).stop 2024-03-23T17:45:41.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:45:41.965+0000 7f62cf000700 1 -- 172.21.15.86:0/1034987609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62c8074de0 msgr2=0x7f62c8199500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:45:41.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:45:41.965+0000 7f62cf000700 1 --2- 172.21.15.86:0/1034987609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62c8074de0 0x7f62c8199500 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f62b40037f0 tx=0x7f62b400f690 comp rx=0 tx=0).stop 2024-03-23T17:45:41.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:45:41.965+0000 7f62cf000700 1 -- 172.21.15.86:0/1034987609 shutdown_connections 2024-03-23T17:45:41.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:45:41.965+0000 7f62cf000700 1 --2- 172.21.15.86:0/1034987609 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62c8108cd0 0x7f62c81a0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:45:41.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:45:41.965+0000 7f62cf000700 1 --2- 172.21.15.86:0/1034987609 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f62c8073810 0x7f62c8198fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:45:41.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:45:41.965+0000 7f62cf000700 1 --2- 172.21.15.86:0/1034987609 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f62ac0a1110 0x7f62ac0a35d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:45:41.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:45:41.965+0000 7f62cf000700 1 --2- 172.21.15.86:0/1034987609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62c8074de0 0x7f62c8199500 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:45:41.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:45:41.965+0000 7f62cf000700 1 -- 172.21.15.86:0/1034987609 >> 172.21.15.86:0/1034987609 conn(0x7f62c80fc3c0 msgr2=0x7f62c8104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:45:41.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:45:41.965+0000 7f62cf000700 1 -- 172.21.15.86:0/1034987609 shutdown_connections 2024-03-23T17:45:41.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:45:41.966+0000 7f62cf000700 1 -- 172.21.15.86:0/1034987609 wait complete. 2024-03-23T17:45:42.365 DEBUG:teuthology.orchestra.run.smithi086:osd.3> sudo journalctl -f -n 0 -u ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@osd.3.service 2024-03-23T17:45:42.370 INFO:tasks.cephadm:Deploying osd.4 on smithi092 with /dev/vg_nvme/lv_3... 2024-03-23T17:45:42.370 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-23T17:45:42.445 INFO:journalctl@ceph.osd.3.smithi086.stdout:-- Logs begin at Sat 2024-03-23 17:25:35 UTC. -- 2024-03-23T17:45:42.623 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.b/config 2024-03-23T17:45:43.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:42 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:43.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:42 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:43.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:42 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:45:43.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:42 smithi092 ceph-mon[79089]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-03-23T17:45:43.075 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:42 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:43.075 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:42 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:43.075 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:42 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:45:43.075 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:42 smithi086 ceph-mon[38221]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-03-23T17:45:43.334 INFO:journalctl@ceph.osd.3.smithi086.stdout:Mar 23 17:45:43 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-3[65335]: 2024-03-23T17:45:43.180+0000 7f7b0c194640 -1 osd.3 0 log_to_monitors true 2024-03-23T17:45:43.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:42 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:43.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:42 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:43.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:42 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:45:43.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:42 smithi152 ceph-mon[47949]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-03-23T17:45:43.997 INFO:teuthology.orchestra.run.smithi092.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-23T17:45:43.997 INFO:teuthology.orchestra.run.smithi092.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-23T17:45:43.997 INFO:teuthology.orchestra.run.smithi092.stderr: stderr: 10+0 records in 2024-03-23T17:45:43.997 INFO:teuthology.orchestra.run.smithi092.stderr:10+0 records out 2024-03-23T17:45:43.998 INFO:teuthology.orchestra.run.smithi092.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113638 s, 923 MB/s 2024-03-23T17:45:43.998 INFO:teuthology.orchestra.run.smithi092.stderr:--> Zapping successful for: 2024-03-23T17:45:44.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:43 smithi086 ceph-mon[38221]: from='osd.3 [v2:172.21.15.86:6810/694846466,v1:172.21.15.86:6811/694846466]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-23T17:45:44.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:43 smithi086 ceph-mon[38221]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-23T17:45:44.298 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:43 smithi092 ceph-mon[79089]: from='osd.3 [v2:172.21.15.86:6810/694846466,v1:172.21.15.86:6811/694846466]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-23T17:45:44.298 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:43 smithi092 ceph-mon[79089]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-23T17:45:44.335 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi092:vg_nvme/lv_3 2024-03-23T17:45:44.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:43 smithi152 ceph-mon[47949]: from='osd.3 [v2:172.21.15.86:6810/694846466,v1:172.21.15.86:6811/694846466]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-23T17:45:44.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:43 smithi152 ceph-mon[47949]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-23T17:45:44.551 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.b/config 2024-03-23T17:45:45.137 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:44 smithi086 ceph-mon[38221]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-23T17:45:45.137 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:44 smithi086 ceph-mon[38221]: osdmap e23: 4 total, 3 up, 4 in 2024-03-23T17:45:45.137 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:44 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T17:45:45.137 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:44 smithi086 ceph-mon[38221]: from='osd.3 [v2:172.21.15.86:6810/694846466,v1:172.21.15.86:6811/694846466]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-03-23T17:45:45.137 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:44 smithi086 ceph-mon[38221]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-03-23T17:45:45.138 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:44 smithi086 ceph-mon[38221]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-03-23T17:45:45.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:44 smithi092 ceph-mon[79089]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-23T17:45:45.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:44 smithi092 ceph-mon[79089]: osdmap e23: 4 total, 3 up, 4 in 2024-03-23T17:45:45.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:44 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T17:45:45.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:44 smithi092 ceph-mon[79089]: from='osd.3 [v2:172.21.15.86:6810/694846466,v1:172.21.15.86:6811/694846466]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-03-23T17:45:45.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:44 smithi092 ceph-mon[79089]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-03-23T17:45:45.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:44 smithi092 ceph-mon[79089]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-03-23T17:45:45.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:44 smithi152 ceph-mon[47949]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-23T17:45:45.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:44 smithi152 ceph-mon[47949]: osdmap e23: 4 total, 3 up, 4 in 2024-03-23T17:45:45.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:44 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T17:45:45.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:44 smithi152 ceph-mon[47949]: from='osd.3 [v2:172.21.15.86:6810/694846466,v1:172.21.15.86:6811/694846466]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-03-23T17:45:45.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:44 smithi152 ceph-mon[47949]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-03-23T17:45:45.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:44 smithi152 ceph-mon[47949]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-03-23T17:45:45.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:45:45.540+0000 7f3797b9b700 1 -- 172.21.15.92:0/435490538 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3790100ef0 msgr2=0x7f3790101350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:45:45.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:45:45.540+0000 7f3797b9b700 1 --2- 172.21.15.92:0/435490538 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3790100ef0 0x7f3790101350 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f37800099c0 tx=0x7f3780009cd0 comp rx=0 tx=0).stop 2024-03-23T17:45:45.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:45:45.541+0000 7f3797b9b700 1 -- 172.21.15.92:0/435490538 shutdown_connections 2024-03-23T17:45:45.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:45:45.541+0000 7f3797b9b700 1 --2- 172.21.15.92:0/435490538 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3790101890 0x7f3790108140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:45:45.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:45:45.541+0000 7f3797b9b700 1 --2- 172.21.15.92:0/435490538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37900ffcf0 0x7f3790100110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:45:45.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:45:45.541+0000 7f3797b9b700 1 --2- 172.21.15.92:0/435490538 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3790100ef0 0x7f3790101350 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:45:45.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:45:45.541+0000 7f3797b9b700 1 -- 172.21.15.92:0/435490538 >> 172.21.15.92:0/435490538 conn(0x7f37900fb2b0 msgr2=0x7f37900fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:45:45.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:45:45.541+0000 7f3797b9b700 1 -- 172.21.15.92:0/435490538 shutdown_connections 2024-03-23T17:45:45.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:45:45.541+0000 7f3797b9b700 1 -- 172.21.15.92:0/435490538 wait complete. 2024-03-23T17:45:45.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:45:45.542+0000 7f3797b9b700 1 Processor -- start 2024-03-23T17:45:45.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:45:45.542+0000 7f3797b9b700 1 -- start start 2024-03-23T17:45:45.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:45:45.542+0000 7f3797b9b700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37900ffcf0 0x7f3790195c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:45:45.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:45:45.542+0000 7f3797b9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3790100ef0 0x7f3790196140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:45:45.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:45:45.542+0000 7f3797b9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3790101890 0x7f379019d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:45:45.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:45:45.542+0000 7f3797b9b700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f379019d740 con 0x7f37900ffcf0 2024-03-23T17:45:45.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:45:45.542+0000 7f3797b9b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f379019d880 con 0x7f3790101890 2024-03-23T17:45:45.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:45:45.542+0000 7f3797b9b700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f379019da10 con 0x7f3790100ef0 2024-03-23T17:45:45.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:45:45.542+0000 7f3796138700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3790101890 0x7f379019d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:45:45.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:45:45.542+0000 7f3796138700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3790101890 0x7f379019d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53386/0 (socket says 172.21.15.92:53386) 2024-03-23T17:45:45.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:45:45.542+0000 7f3796138700 1 -- 172.21.15.92:0/1122646553 learned_addr learned my addr 172.21.15.92:0/1122646553 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-23T17:45:45.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:45:45.542+0000 7f3795136700 1 --2- 172.21.15.92:0/1122646553 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3790100ef0 0x7f3790196140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:45:45.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:45:45.542+0000 7f3795937700 1 --2- 172.21.15.92:0/1122646553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37900ffcf0 0x7f3790195c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:45:45.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:45:45.543+0000 7f3796138700 1 -- 172.21.15.92:0/1122646553 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3790100ef0 msgr2=0x7f3790196140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:45:45.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:45:45.543+0000 7f3796138700 1 --2- 172.21.15.92:0/1122646553 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3790100ef0 0x7f3790196140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:45:45.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:45:45.543+0000 7f3796138700 1 -- 172.21.15.92:0/1122646553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37900ffcf0 msgr2=0x7f3790195c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:45:45.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:45:45.543+0000 7f3796138700 1 --2- 172.21.15.92:0/1122646553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37900ffcf0 0x7f3790195c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:45:45.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:45:45.543+0000 7f3796138700 1 -- 172.21.15.92:0/1122646553 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37800095f0 con 0x7f3790101890 2024-03-23T17:45:45.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:45:45.543+0000 7f3796138700 1 --2- 172.21.15.92:0/1122646553 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3790101890 0x7f379019d170 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3788007ee0 tx=0x7f378800d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:45:45.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:45:45.543+0000 7f3786ffd700 1 -- 172.21.15.92:0/1122646553 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3788017070 con 0x7f3790101890 2024-03-23T17:45:45.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:45:45.543+0000 7f3786ffd700 1 -- 172.21.15.92:0/1122646553 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f378800f040 con 0x7f3790101890 2024-03-23T17:45:45.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:45:45.543+0000 7f3797b9b700 1 -- 172.21.15.92:0/1122646553 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f379019dcf0 con 0x7f3790101890 2024-03-23T17:45:45.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:45:45.543+0000 7f3786ffd700 1 -- 172.21.15.92:0/1122646553 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3788005610 con 0x7f3790101890 2024-03-23T17:45:45.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:45:45.543+0000 7f3797b9b700 1 -- 172.21.15.92:0/1122646553 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f379019e1a0 con 0x7f3790101890 2024-03-23T17:45:45.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:45:45.544+0000 7f3797b9b700 1 -- 172.21.15.92:0/1122646553 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3790107180 con 0x7f3790101890 2024-03-23T17:45:45.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:45:45.545+0000 7f3786ffd700 1 -- 172.21.15.92:0/1122646553 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f3788004c40 con 0x7f3790101890 2024-03-23T17:45:45.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:45:45.545+0000 7f3786ffd700 1 --2- 172.21.15.92:0/1122646553 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f377c0a1120 0x7f377c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:45:45.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:45:45.545+0000 7f3786ffd700 1 -- 172.21.15.92:0/1122646553 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 3422+0+0 (secure 0 0 0) 0x7f37880cac70 con 0x7f3790101890 2024-03-23T17:45:45.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:45:45.545+0000 7f3795937700 1 --2- 172.21.15.92:0/1122646553 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f377c0a1120 0x7f377c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:45:45.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:45:45.546+0000 7f3795937700 1 --2- 172.21.15.92:0/1122646553 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f377c0a1120 0x7f377c0a35e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3790100d50 tx=0x7f378c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:45:45.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:45:45.550+0000 7f3786ffd700 1 -- 172.21.15.92:0/1122646553 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f3788095720 con 0x7f3790101890 2024-03-23T17:45:45.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:45:45.747+0000 7f3797b9b700 1 -- 172.21.15.92:0/1122646553 --> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f37900611f0 con 0x7f377c0a1120 2024-03-23T17:45:46.266 INFO:journalctl@ceph.osd.3.smithi086.stdout:Mar 23 17:45:45 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-3[65335]: 2024-03-23T17:45:45.939+0000 7f7affe00700 -1 osd.3 0 waiting for initial osdmap 2024-03-23T17:45:46.267 INFO:journalctl@ceph.osd.3.smithi086.stdout:Mar 23 17:45:45 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-3[65335]: 2024-03-23T17:45:45.956+0000 7f7af6600700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T17:45:46.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:45 smithi086 ceph-mon[38221]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-03-23T17:45:46.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:45 smithi086 ceph-mon[38221]: osdmap e24: 4 total, 3 up, 4 in 2024-03-23T17:45:46.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:45 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T17:45:46.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:45 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T17:45:46.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:45 smithi086 ceph-mon[38221]: Detected new or changed devices on smithi086 2024-03-23T17:45:46.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:45 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:46.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:45 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:46.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:45 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:45:46.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:45 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:45:46.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:45 smithi086 ceph-mon[38221]: Adjusting osd_memory_target on smithi086 to 8635M 2024-03-23T17:45:46.268 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:45 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:46.268 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:45 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:45:46.268 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:45 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:45:46.268 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:45 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:46.268 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:45 smithi086 ceph-mon[38221]: osdmap e25: 4 total, 3 up, 4 in 2024-03-23T17:45:46.268 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:45 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T17:45:46.268 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:45 smithi086 ceph-mon[38221]: from='client.24233 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:45:46.268 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:45 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T17:45:46.268 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:45 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T17:45:46.268 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:45 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:45:46.268 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:45 smithi086 ceph-mon[38221]: from='osd.3 ' entity='osd.3' 2024-03-23T17:45:46.297 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:45 smithi092 ceph-mon[79089]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-03-23T17:45:46.297 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:45 smithi092 ceph-mon[79089]: osdmap e24: 4 total, 3 up, 4 in 2024-03-23T17:45:46.297 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:45 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T17:45:46.297 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:45 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T17:45:46.297 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:45 smithi092 ceph-mon[79089]: Detected new or changed devices on smithi086 2024-03-23T17:45:46.297 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:45 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:46.297 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:45 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:46.297 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:45 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:45:46.297 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:45 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:45:46.297 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:45 smithi092 ceph-mon[79089]: Adjusting osd_memory_target on smithi086 to 8635M 2024-03-23T17:45:46.298 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:45 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:46.298 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:45 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:45:46.298 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:45 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:45:46.298 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:45 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:46.298 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:45 smithi092 ceph-mon[79089]: osdmap e25: 4 total, 3 up, 4 in 2024-03-23T17:45:46.298 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:45 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T17:45:46.298 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:45 smithi092 ceph-mon[79089]: from='client.24233 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:45:46.298 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:45 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T17:45:46.298 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:45 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T17:45:46.298 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:45 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:45:46.298 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:45 smithi092 ceph-mon[79089]: from='osd.3 ' entity='osd.3' 2024-03-23T17:45:46.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:45 smithi152 ceph-mon[47949]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-03-23T17:45:46.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:45 smithi152 ceph-mon[47949]: osdmap e24: 4 total, 3 up, 4 in 2024-03-23T17:45:46.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:45 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T17:45:46.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:45 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T17:45:46.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:45 smithi152 ceph-mon[47949]: Detected new or changed devices on smithi086 2024-03-23T17:45:46.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:45 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:46.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:45 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:46.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:45 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:45:46.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:45 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:45:46.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:45 smithi152 ceph-mon[47949]: Adjusting osd_memory_target on smithi086 to 8635M 2024-03-23T17:45:46.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:45 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:46.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:45 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:45:46.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:45 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:45:46.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:45 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:46.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:45 smithi152 ceph-mon[47949]: osdmap e25: 4 total, 3 up, 4 in 2024-03-23T17:45:46.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:45 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T17:45:46.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:45 smithi152 ceph-mon[47949]: from='client.24233 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:45:46.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:45 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T17:45:46.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:45 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T17:45:46.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:45 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:45:46.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:45 smithi152 ceph-mon[47949]: from='osd.3 ' entity='osd.3' 2024-03-23T17:45:47.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:46 smithi086 ceph-mon[38221]: purged_snaps scrub starts 2024-03-23T17:45:47.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:46 smithi086 ceph-mon[38221]: purged_snaps scrub ok 2024-03-23T17:45:47.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:46 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T17:45:47.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:46 smithi086 ceph-mon[38221]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-03-23T17:45:47.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:46 smithi086 ceph-mon[38221]: osd.3 [v2:172.21.15.86:6810/694846466,v1:172.21.15.86:6811/694846466] boot 2024-03-23T17:45:47.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:46 smithi086 ceph-mon[38221]: osdmap e26: 4 total, 4 up, 4 in 2024-03-23T17:45:47.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:46 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T17:45:47.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:46 smithi092 ceph-mon[79089]: purged_snaps scrub starts 2024-03-23T17:45:47.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:46 smithi092 ceph-mon[79089]: purged_snaps scrub ok 2024-03-23T17:45:47.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:46 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T17:45:47.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:46 smithi092 ceph-mon[79089]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-03-23T17:45:47.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:46 smithi092 ceph-mon[79089]: osd.3 [v2:172.21.15.86:6810/694846466,v1:172.21.15.86:6811/694846466] boot 2024-03-23T17:45:47.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:46 smithi092 ceph-mon[79089]: osdmap e26: 4 total, 4 up, 4 in 2024-03-23T17:45:47.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:46 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T17:45:47.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:46 smithi152 ceph-mon[47949]: purged_snaps scrub starts 2024-03-23T17:45:47.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:46 smithi152 ceph-mon[47949]: purged_snaps scrub ok 2024-03-23T17:45:47.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:46 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T17:45:47.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:46 smithi152 ceph-mon[47949]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-03-23T17:45:47.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:46 smithi152 ceph-mon[47949]: osd.3 [v2:172.21.15.86:6810/694846466,v1:172.21.15.86:6811/694846466] boot 2024-03-23T17:45:47.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:46 smithi152 ceph-mon[47949]: osdmap e26: 4 total, 4 up, 4 in 2024-03-23T17:45:47.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:46 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T17:45:48.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:48 smithi152 ceph-mon[47949]: osdmap e27: 4 total, 4 up, 4 in 2024-03-23T17:45:48.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:48 smithi152 ceph-mon[47949]: from='client.? 172.21.15.92:0/1584060317' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a2878035-0179-469f-bbd4-8ce2d3f6bddc"}]: dispatch 2024-03-23T17:45:48.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:48 smithi152 ceph-mon[47949]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a2878035-0179-469f-bbd4-8ce2d3f6bddc"}]: dispatch 2024-03-23T17:45:48.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:48 smithi152 ceph-mon[47949]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a2878035-0179-469f-bbd4-8ce2d3f6bddc"}]': finished 2024-03-23T17:45:48.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:48 smithi152 ceph-mon[47949]: osdmap e28: 5 total, 4 up, 5 in 2024-03-23T17:45:48.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:48 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T17:45:48.837 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:48 smithi152 ceph-mon[47949]: from='client.? 172.21.15.92:0/1837296340' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T17:45:49.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:48 smithi086 ceph-mon[38221]: osdmap e27: 4 total, 4 up, 4 in 2024-03-23T17:45:49.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:48 smithi086 ceph-mon[38221]: from='client.? 172.21.15.92:0/1584060317' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a2878035-0179-469f-bbd4-8ce2d3f6bddc"}]: dispatch 2024-03-23T17:45:49.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:48 smithi086 ceph-mon[38221]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a2878035-0179-469f-bbd4-8ce2d3f6bddc"}]: dispatch 2024-03-23T17:45:49.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:48 smithi086 ceph-mon[38221]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a2878035-0179-469f-bbd4-8ce2d3f6bddc"}]': finished 2024-03-23T17:45:49.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:48 smithi086 ceph-mon[38221]: osdmap e28: 5 total, 4 up, 5 in 2024-03-23T17:45:49.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:48 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T17:45:49.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:48 smithi086 ceph-mon[38221]: from='client.? 172.21.15.92:0/1837296340' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T17:45:49.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:48 smithi092 ceph-mon[79089]: osdmap e27: 4 total, 4 up, 4 in 2024-03-23T17:45:49.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:48 smithi092 ceph-mon[79089]: from='client.? 172.21.15.92:0/1584060317' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a2878035-0179-469f-bbd4-8ce2d3f6bddc"}]: dispatch 2024-03-23T17:45:49.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:48 smithi092 ceph-mon[79089]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a2878035-0179-469f-bbd4-8ce2d3f6bddc"}]: dispatch 2024-03-23T17:45:49.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:48 smithi092 ceph-mon[79089]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a2878035-0179-469f-bbd4-8ce2d3f6bddc"}]': finished 2024-03-23T17:45:49.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:48 smithi092 ceph-mon[79089]: osdmap e28: 5 total, 4 up, 5 in 2024-03-23T17:45:49.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:48 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T17:45:49.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:48 smithi092 ceph-mon[79089]: from='client.? 172.21.15.92:0/1837296340' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T17:45:50.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:49 smithi086 ceph-mon[38221]: pgmap v88: 1 pgs: 1 remapped+peering; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail 2024-03-23T17:45:50.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:49 smithi086 ceph-mon[38221]: osdmap e29: 5 total, 4 up, 5 in 2024-03-23T17:45:50.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:49 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T17:45:50.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:49 smithi092 ceph-mon[79089]: pgmap v88: 1 pgs: 1 remapped+peering; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail 2024-03-23T17:45:50.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:49 smithi092 ceph-mon[79089]: osdmap e29: 5 total, 4 up, 5 in 2024-03-23T17:45:50.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:49 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T17:45:50.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:49 smithi152 ceph-mon[47949]: pgmap v88: 1 pgs: 1 remapped+peering; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail 2024-03-23T17:45:50.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:49 smithi152 ceph-mon[47949]: osdmap e29: 5 total, 4 up, 5 in 2024-03-23T17:45:50.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:49 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T17:45:51.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:50 smithi086 ceph-mon[38221]: osdmap e30: 5 total, 4 up, 5 in 2024-03-23T17:45:51.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:50 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T17:45:51.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:50 smithi092 ceph-mon[79089]: osdmap e30: 5 total, 4 up, 5 in 2024-03-23T17:45:51.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:50 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T17:45:51.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:50 smithi152 ceph-mon[47949]: osdmap e30: 5 total, 4 up, 5 in 2024-03-23T17:45:51.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:50 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T17:45:51.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:51 smithi092 ceph-mon[79089]: pgmap v91: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-23T17:45:52.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:51 smithi086 ceph-mon[38221]: pgmap v91: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-23T17:45:52.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:51 smithi152 ceph-mon[47949]: pgmap v91: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-23T17:45:54.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:53 smithi086 ceph-mon[38221]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-03-23T17:45:54.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:53 smithi092 ceph-mon[79089]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-03-23T17:45:54.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:53 smithi152 ceph-mon[47949]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-03-23T17:45:54.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:54 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-23T17:45:54.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:54 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:45:54.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:54 smithi092 ceph-mon[79089]: Deploying daemon osd.4 on smithi092 2024-03-23T17:45:55.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:54 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-23T17:45:55.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:54 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:45:55.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:54 smithi086 ceph-mon[38221]: Deploying daemon osd.4 on smithi092 2024-03-23T17:45:55.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:54 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-23T17:45:55.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:54 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:45:55.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:54 smithi152 ceph-mon[47949]: Deploying daemon osd.4 on smithi092 2024-03-23T17:45:56.004 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:55 smithi092 ceph-mon[79089]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-03-23T17:45:56.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:55 smithi086 ceph-mon[38221]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-03-23T17:45:56.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:55 smithi152 ceph-mon[47949]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-03-23T17:45:57.974 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:57 smithi092 ceph-mon[79089]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-23T17:45:57.974 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:57 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:57.974 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:57 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:57.974 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:57 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:45:58.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:57 smithi086 ceph-mon[38221]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-23T17:45:58.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:57 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:58.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:57 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:58.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:57 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:45:58.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:57 smithi152 ceph-mon[47949]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-23T17:45:58.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:57 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:58.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:57 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:45:58.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:57 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:45:59.703 INFO:teuthology.orchestra.run.smithi092.stdout:Created osd(s) 4 on host 'smithi092' 2024-03-23T17:45:59.703 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:45:59.702+0000 7f3786ffd700 1 -- 172.21.15.92:0/1122646553 <== mgr.14152 v2:172.21.15.86:6800/2734784244 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f37900611f0 con 0x7f377c0a1120 2024-03-23T17:45:59.705 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:45:59.704+0000 7f3797b9b700 1 -- 172.21.15.92:0/1122646553 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f377c0a1120 msgr2=0x7f377c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:45:59.705 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:45:59.704+0000 7f3797b9b700 1 --2- 172.21.15.92:0/1122646553 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f377c0a1120 0x7f377c0a35e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3790100d50 tx=0x7f378c009260 comp rx=0 tx=0).stop 2024-03-23T17:45:59.705 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:45:59.704+0000 7f3797b9b700 1 -- 172.21.15.92:0/1122646553 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3790101890 msgr2=0x7f379019d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:45:59.705 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:45:59.704+0000 7f3797b9b700 1 --2- 172.21.15.92:0/1122646553 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3790101890 0x7f379019d170 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3788007ee0 tx=0x7f378800d3b0 comp rx=0 tx=0).stop 2024-03-23T17:45:59.705 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:45:59.704+0000 7f3797b9b700 1 -- 172.21.15.92:0/1122646553 shutdown_connections 2024-03-23T17:45:59.705 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:45:59.704+0000 7f3797b9b700 1 --2- 172.21.15.92:0/1122646553 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3790101890 0x7f379019d170 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:45:59.705 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:45:59.704+0000 7f3797b9b700 1 --2- 172.21.15.92:0/1122646553 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f377c0a1120 0x7f377c0a35e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:45:59.705 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:45:59.704+0000 7f3797b9b700 1 --2- 172.21.15.92:0/1122646553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37900ffcf0 0x7f3790195c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:45:59.705 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:45:59.704+0000 7f3797b9b700 1 --2- 172.21.15.92:0/1122646553 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3790100ef0 0x7f3790196140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:45:59.705 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:45:59.704+0000 7f3797b9b700 1 -- 172.21.15.92:0/1122646553 >> 172.21.15.92:0/1122646553 conn(0x7f37900fb2b0 msgr2=0x7f37901042c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:45:59.706 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:45:59.705+0000 7f3797b9b700 1 -- 172.21.15.92:0/1122646553 shutdown_connections 2024-03-23T17:45:59.706 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:45:59.705+0000 7f3797b9b700 1 -- 172.21.15.92:0/1122646553 wait complete. 2024-03-23T17:46:00.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:59 smithi092 ceph-mon[79089]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-03-23T17:46:00.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:59 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:00.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:59 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:00.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:59 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:46:00.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:59 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:46:00.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:59 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:00.000 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:59 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:00.001 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:45:59 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:00.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:59 smithi086 ceph-mon[38221]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-03-23T17:46:00.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:59 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:00.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:59 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:00.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:59 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:46:00.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:59 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:46:00.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:59 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:00.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:59 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:00.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:45:59 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:00.061 DEBUG:teuthology.orchestra.run.smithi092:osd.4> sudo journalctl -f -n 0 -u ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@osd.4.service 2024-03-23T17:46:00.063 INFO:tasks.cephadm:Deploying osd.5 on smithi152 with /dev/vg_nvme/lv_3... 2024-03-23T17:46:00.063 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-23T17:46:00.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:59 smithi152 ceph-mon[47949]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-03-23T17:46:00.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:59 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:00.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:59 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:00.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:59 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:46:00.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:59 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:46:00.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:59 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:00.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:59 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:00.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:45:59 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:00.094 INFO:journalctl@ceph.osd.4.smithi092.stdout:-- Logs begin at Sat 2024-03-23 17:19:16 UTC. -- 2024-03-23T17:46:00.296 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.c/config 2024-03-23T17:46:00.968 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:00 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:46:01.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:00 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:46:01.060 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:00 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:46:01.598 INFO:teuthology.orchestra.run.smithi152.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-23T17:46:01.598 INFO:teuthology.orchestra.run.smithi152.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-23T17:46:01.598 INFO:teuthology.orchestra.run.smithi152.stderr: stderr: 10+0 records in 2024-03-23T17:46:01.598 INFO:teuthology.orchestra.run.smithi152.stderr:10+0 records out 2024-03-23T17:46:01.598 INFO:teuthology.orchestra.run.smithi152.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0131939 s, 795 MB/s 2024-03-23T17:46:01.598 INFO:teuthology.orchestra.run.smithi152.stderr: stderr: 2024-03-23T17:46:01.598 INFO:teuthology.orchestra.run.smithi152.stderr:--> Zapping successful for: 2024-03-23T17:46:01.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:01 smithi092 ceph-mon[79089]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-03-23T17:46:01.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:01 smithi092 ceph-mon[79089]: from='osd.4 [v2:172.21.15.92:6808/1422821471,v1:172.21.15.92:6809/1422821471]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-23T17:46:01.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:01 smithi092 ceph-mon[79089]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-23T17:46:01.808 INFO:journalctl@ceph.osd.4.smithi092.stdout:Mar 23 17:46:01 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-4[92840]: 2024-03-23T17:46:01.526+0000 7f65b1c2c640 -1 osd.4 0 log_to_monitors true 2024-03-23T17:46:01.911 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:01 smithi152 ceph-mon[47949]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-03-23T17:46:01.912 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:01 smithi152 ceph-mon[47949]: from='osd.4 [v2:172.21.15.92:6808/1422821471,v1:172.21.15.92:6809/1422821471]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-23T17:46:01.912 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:01 smithi152 ceph-mon[47949]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-23T17:46:01.970 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi152:vg_nvme/lv_3 2024-03-23T17:46:02.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:01 smithi086 ceph-mon[38221]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-03-23T17:46:02.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:01 smithi086 ceph-mon[38221]: from='osd.4 [v2:172.21.15.92:6808/1422821471,v1:172.21.15.92:6809/1422821471]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-23T17:46:02.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:01 smithi086 ceph-mon[38221]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-23T17:46:02.229 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.c/config 2024-03-23T17:46:03.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:02 smithi086 ceph-mon[38221]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-23T17:46:03.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:02 smithi086 ceph-mon[38221]: osdmap e31: 5 total, 4 up, 5 in 2024-03-23T17:46:03.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:02 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T17:46:03.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:02 smithi086 ceph-mon[38221]: from='osd.4 [v2:172.21.15.92:6808/1422821471,v1:172.21.15.92:6809/1422821471]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-03-23T17:46:03.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:02 smithi086 ceph-mon[38221]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-03-23T17:46:03.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:02 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:03.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:02 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:03.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:02 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:46:03.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:02 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:46:03.018 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:02 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:03.018 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:02 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:46:03.018 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:02 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:46:03.018 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:02 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:03.025 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:02 smithi152 ceph-mon[47949]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-23T17:46:03.026 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:02 smithi152 ceph-mon[47949]: osdmap e31: 5 total, 4 up, 5 in 2024-03-23T17:46:03.026 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:02 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T17:46:03.026 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:02 smithi152 ceph-mon[47949]: from='osd.4 [v2:172.21.15.92:6808/1422821471,v1:172.21.15.92:6809/1422821471]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-03-23T17:46:03.026 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:02 smithi152 ceph-mon[47949]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-03-23T17:46:03.026 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:02 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:03.026 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:02 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:03.026 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:02 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:46:03.026 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:02 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:46:03.026 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:02 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:03.026 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:02 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:46:03.026 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:02 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:46:03.026 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:02 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:03.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:02 smithi092 ceph-mon[79089]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-23T17:46:03.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:02 smithi092 ceph-mon[79089]: osdmap e31: 5 total, 4 up, 5 in 2024-03-23T17:46:03.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:02 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T17:46:03.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:02 smithi092 ceph-mon[79089]: from='osd.4 [v2:172.21.15.92:6808/1422821471,v1:172.21.15.92:6809/1422821471]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-03-23T17:46:03.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:02 smithi092 ceph-mon[79089]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-03-23T17:46:03.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:02 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:03.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:02 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:03.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:02 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:46:03.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:02 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:46:03.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:02 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:03.059 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:02 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:46:03.059 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:02 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:46:03.059 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:02 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:03.059 INFO:journalctl@ceph.osd.4.smithi092.stdout:Mar 23 17:46:02 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-4[92840]: 2024-03-23T17:46:02.709+0000 7f65a6a8f700 -1 osd.4 0 waiting for initial osdmap 2024-03-23T17:46:03.059 INFO:journalctl@ceph.osd.4.smithi092.stdout:Mar 23 17:46:02 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-4[92840]: 2024-03-23T17:46:02.715+0000 7f65a3085700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T17:46:03.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:46:03.233+0000 7fc0fd5ee700 1 -- 172.21.15.152:0/3156681565 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc0f81018b0 msgr2=0x7fc0f8108160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:46:03.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:46:03.233+0000 7fc0fd5ee700 1 --2- 172.21.15.152:0/3156681565 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc0f81018b0 0x7fc0f8108160 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc0f000b0a0 tx=0x7fc0f000b3b0 comp rx=0 tx=0).stop 2024-03-23T17:46:03.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:46:03.233+0000 7fc0fd5ee700 1 -- 172.21.15.152:0/3156681565 shutdown_connections 2024-03-23T17:46:03.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:46:03.233+0000 7fc0fd5ee700 1 --2- 172.21.15.152:0/3156681565 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc0f81018b0 0x7fc0f8108160 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:46:03.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:46:03.233+0000 7fc0fd5ee700 1 --2- 172.21.15.152:0/3156681565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0f80ffd10 0x7fc0f8100130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:46:03.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:46:03.233+0000 7fc0fd5ee700 1 --2- 172.21.15.152:0/3156681565 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0f8100f10 0x7fc0f8101370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:46:03.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:46:03.233+0000 7fc0fd5ee700 1 -- 172.21.15.152:0/3156681565 >> 172.21.15.152:0/3156681565 conn(0x7fc0f80fb2d0 msgr2=0x7fc0f80fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:46:03.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:46:03.233+0000 7fc0fd5ee700 1 -- 172.21.15.152:0/3156681565 shutdown_connections 2024-03-23T17:46:03.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:46:03.233+0000 7fc0fd5ee700 1 -- 172.21.15.152:0/3156681565 wait complete. 2024-03-23T17:46:03.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:46:03.233+0000 7fc0fd5ee700 1 Processor -- start 2024-03-23T17:46:03.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:46:03.234+0000 7fc0fd5ee700 1 -- start start 2024-03-23T17:46:03.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:46:03.234+0000 7fc0fd5ee700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc0f80ffd10 0x7fc0f8194c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:46:03.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:46:03.234+0000 7fc0fd5ee700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0f8100f10 0x7fc0f8195140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:46:03.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:46:03.234+0000 7fc0fd5ee700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0f81018b0 0x7fc0f819c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:46:03.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:46:03.234+0000 7fc0fd5ee700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f819c740 con 0x7fc0f81018b0 2024-03-23T17:46:03.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:46:03.234+0000 7fc0fd5ee700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f819c880 con 0x7fc0f8100f10 2024-03-23T17:46:03.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:46:03.234+0000 7fc0fd5ee700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f819ca10 con 0x7fc0f80ffd10 2024-03-23T17:46:03.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:46:03.234+0000 7fc0f6ffd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc0f80ffd10 0x7fc0f8194c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:46:03.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:46:03.234+0000 7fc0f6ffd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc0f80ffd10 0x7fc0f8194c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46752/0 (socket says 172.21.15.152:46752) 2024-03-23T17:46:03.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:46:03.234+0000 7fc0f6ffd700 1 -- 172.21.15.152:0/1502614645 learned_addr learned my addr 172.21.15.152:0/1502614645 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-03-23T17:46:03.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:46:03.234+0000 7fc0f6ffd700 1 -- 172.21.15.152:0/1502614645 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0f8100f10 msgr2=0x7fc0f8195140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:46:03.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:46:03.234+0000 7fc0f67fc700 1 --2- 172.21.15.152:0/1502614645 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0f8100f10 0x7fc0f8195140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:46:03.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:46:03.234+0000 7fc0f77fe700 1 --2- 172.21.15.152:0/1502614645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0f81018b0 0x7fc0f819c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:46:03.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:46:03.234+0000 7fc0f6ffd700 1 --2- 172.21.15.152:0/1502614645 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0f8100f10 0x7fc0f8195140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:46:03.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:46:03.234+0000 7fc0f6ffd700 1 -- 172.21.15.152:0/1502614645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0f81018b0 msgr2=0x7fc0f819c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:46:03.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:46:03.234+0000 7fc0f6ffd700 1 --2- 172.21.15.152:0/1502614645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0f81018b0 0x7fc0f819c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:46:03.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:46:03.234+0000 7fc0f6ffd700 1 -- 172.21.15.152:0/1502614645 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0f0009d00 con 0x7fc0f80ffd10 2024-03-23T17:46:03.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:46:03.234+0000 7fc0f67fc700 1 --2- 172.21.15.152:0/1502614645 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0f8100f10 0x7fc0f8195140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T17:46:03.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:46:03.234+0000 7fc0f77fe700 1 --2- 172.21.15.152:0/1502614645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0f81018b0 0x7fc0f819c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T17:46:03.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:46:03.235+0000 7fc0f6ffd700 1 --2- 172.21.15.152:0/1502614645 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc0f80ffd10 0x7fc0f8194c00 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc0e800e9a0 tx=0x7fc0e800ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:46:03.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:46:03.235+0000 7fc0dffff700 1 -- 172.21.15.152:0/1502614645 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e8009790 con 0x7fc0f80ffd10 2024-03-23T17:46:03.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:46:03.235+0000 7fc0fd5ee700 1 -- 172.21.15.152:0/1502614645 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0f819ccf0 con 0x7fc0f80ffd10 2024-03-23T17:46:03.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:46:03.235+0000 7fc0dffff700 1 -- 172.21.15.152:0/1502614645 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0e80044e0 con 0x7fc0f80ffd10 2024-03-23T17:46:03.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:46:03.235+0000 7fc0dffff700 1 -- 172.21.15.152:0/1502614645 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e8003d70 con 0x7fc0f80ffd10 2024-03-23T17:46:03.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:46:03.235+0000 7fc0fd5ee700 1 -- 172.21.15.152:0/1502614645 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0f819d1a0 con 0x7fc0f80ffd10 2024-03-23T17:46:03.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:46:03.236+0000 7fc0fd5ee700 1 -- 172.21.15.152:0/1502614645 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0f8107150 con 0x7fc0f80ffd10 2024-03-23T17:46:03.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:46:03.237+0000 7fc0dffff700 1 -- 172.21.15.152:0/1502614645 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fc0e8003f10 con 0x7fc0f80ffd10 2024-03-23T17:46:03.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:46:03.237+0000 7fc0dffff700 1 --2- 172.21.15.152:0/1502614645 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fc0e00a3330 0x7fc0e00a57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:46:03.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:46:03.238+0000 7fc0dffff700 1 -- 172.21.15.152:0/1502614645 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 3821+0+0 (secure 0 0 0) 0x7fc0e8014070 con 0x7fc0f80ffd10 2024-03-23T17:46:03.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:46:03.238+0000 7fc0f67fc700 1 --2- 172.21.15.152:0/1502614645 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fc0e00a3330 0x7fc0e00a57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:46:03.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:46:03.238+0000 7fc0f67fc700 1 --2- 172.21.15.152:0/1502614645 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fc0e00a3330 0x7fc0e00a57f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec009be0 tx=0x7fc0ec009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:46:03.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:46:03.242+0000 7fc0dffff700 1 -- 172.21.15.152:0/1502614645 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc0e80963a0 con 0x7fc0f80ffd10 2024-03-23T17:46:03.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:46:03.444+0000 7fc0fd5ee700 1 -- 172.21.15.152:0/1502614645 --> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fc0f802cdc0 con 0x7fc0e00a3330 2024-03-23T17:46:04.002 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:03 smithi152 ceph-mon[47949]: Detected new or changed devices on smithi092 2024-03-23T17:46:04.002 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:03 smithi152 ceph-mon[47949]: Adjusting osd_memory_target on smithi092 to 8557M 2024-03-23T17:46:04.002 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:03 smithi152 ceph-mon[47949]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-23T17:46:04.002 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:03 smithi152 ceph-mon[47949]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-03-23T17:46:04.002 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:03 smithi152 ceph-mon[47949]: osdmap e32: 5 total, 4 up, 5 in 2024-03-23T17:46:04.002 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:03 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T17:46:04.002 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:03 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T17:46:04.003 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:03 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T17:46:04.003 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:03 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T17:46:04.003 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:03 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:46:04.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:03 smithi086 ceph-mon[38221]: Detected new or changed devices on smithi092 2024-03-23T17:46:04.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:03 smithi086 ceph-mon[38221]: Adjusting osd_memory_target on smithi092 to 8557M 2024-03-23T17:46:04.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:03 smithi086 ceph-mon[38221]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-23T17:46:04.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:03 smithi086 ceph-mon[38221]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-03-23T17:46:04.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:03 smithi086 ceph-mon[38221]: osdmap e32: 5 total, 4 up, 5 in 2024-03-23T17:46:04.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:03 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T17:46:04.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:03 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T17:46:04.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:03 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T17:46:04.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:03 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T17:46:04.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:03 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:46:04.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:03 smithi092 ceph-mon[79089]: Detected new or changed devices on smithi092 2024-03-23T17:46:04.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:03 smithi092 ceph-mon[79089]: Adjusting osd_memory_target on smithi092 to 8557M 2024-03-23T17:46:04.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:03 smithi092 ceph-mon[79089]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-23T17:46:04.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:03 smithi092 ceph-mon[79089]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-03-23T17:46:04.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:03 smithi092 ceph-mon[79089]: osdmap e32: 5 total, 4 up, 5 in 2024-03-23T17:46:04.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:03 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T17:46:04.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:03 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T17:46:04.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:03 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T17:46:04.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:03 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T17:46:04.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:03 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:46:05.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:04 smithi086 ceph-mon[38221]: purged_snaps scrub starts 2024-03-23T17:46:05.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:04 smithi086 ceph-mon[38221]: purged_snaps scrub ok 2024-03-23T17:46:05.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:04 smithi086 ceph-mon[38221]: from='client.24253 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:46:05.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:04 smithi086 ceph-mon[38221]: osd.4 [v2:172.21.15.92:6808/1422821471,v1:172.21.15.92:6809/1422821471] boot 2024-03-23T17:46:05.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:04 smithi086 ceph-mon[38221]: osdmap e33: 5 total, 5 up, 5 in 2024-03-23T17:46:05.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:04 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T17:46:05.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:04 smithi092 ceph-mon[79089]: purged_snaps scrub starts 2024-03-23T17:46:05.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:04 smithi092 ceph-mon[79089]: purged_snaps scrub ok 2024-03-23T17:46:05.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:04 smithi092 ceph-mon[79089]: from='client.24253 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:46:05.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:04 smithi092 ceph-mon[79089]: osd.4 [v2:172.21.15.92:6808/1422821471,v1:172.21.15.92:6809/1422821471] boot 2024-03-23T17:46:05.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:04 smithi092 ceph-mon[79089]: osdmap e33: 5 total, 5 up, 5 in 2024-03-23T17:46:05.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:04 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T17:46:05.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:04 smithi152 ceph-mon[47949]: purged_snaps scrub starts 2024-03-23T17:46:05.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:04 smithi152 ceph-mon[47949]: purged_snaps scrub ok 2024-03-23T17:46:05.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:04 smithi152 ceph-mon[47949]: from='client.24253 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:46:05.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:04 smithi152 ceph-mon[47949]: osd.4 [v2:172.21.15.92:6808/1422821471,v1:172.21.15.92:6809/1422821471] boot 2024-03-23T17:46:05.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:04 smithi152 ceph-mon[47949]: osdmap e33: 5 total, 5 up, 5 in 2024-03-23T17:46:05.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:04 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T17:46:06.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:05 smithi086 ceph-mon[38221]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-23T17:46:06.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:05 smithi086 ceph-mon[38221]: osdmap e34: 5 total, 5 up, 5 in 2024-03-23T17:46:06.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:05 smithi086 ceph-mon[38221]: from='client.? 172.21.15.152:0/2523879647' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "837366aa-ce25-42ca-b157-5bbb50159d0a"}]: dispatch 2024-03-23T17:46:06.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:05 smithi086 ceph-mon[38221]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "837366aa-ce25-42ca-b157-5bbb50159d0a"}]: dispatch 2024-03-23T17:46:06.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:05 smithi086 ceph-mon[38221]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "837366aa-ce25-42ca-b157-5bbb50159d0a"}]': finished 2024-03-23T17:46:06.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:05 smithi086 ceph-mon[38221]: osdmap e35: 6 total, 5 up, 6 in 2024-03-23T17:46:06.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:05 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T17:46:06.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:05 smithi092 ceph-mon[79089]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-23T17:46:06.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:05 smithi092 ceph-mon[79089]: osdmap e34: 5 total, 5 up, 5 in 2024-03-23T17:46:06.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:05 smithi092 ceph-mon[79089]: from='client.? 172.21.15.152:0/2523879647' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "837366aa-ce25-42ca-b157-5bbb50159d0a"}]: dispatch 2024-03-23T17:46:06.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:05 smithi092 ceph-mon[79089]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "837366aa-ce25-42ca-b157-5bbb50159d0a"}]: dispatch 2024-03-23T17:46:06.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:05 smithi092 ceph-mon[79089]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "837366aa-ce25-42ca-b157-5bbb50159d0a"}]': finished 2024-03-23T17:46:06.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:05 smithi092 ceph-mon[79089]: osdmap e35: 6 total, 5 up, 6 in 2024-03-23T17:46:06.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:05 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T17:46:06.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:05 smithi152 ceph-mon[47949]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-23T17:46:06.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:05 smithi152 ceph-mon[47949]: osdmap e34: 5 total, 5 up, 5 in 2024-03-23T17:46:06.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:05 smithi152 ceph-mon[47949]: from='client.? 172.21.15.152:0/2523879647' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "837366aa-ce25-42ca-b157-5bbb50159d0a"}]: dispatch 2024-03-23T17:46:06.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:05 smithi152 ceph-mon[47949]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "837366aa-ce25-42ca-b157-5bbb50159d0a"}]: dispatch 2024-03-23T17:46:06.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:05 smithi152 ceph-mon[47949]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "837366aa-ce25-42ca-b157-5bbb50159d0a"}]': finished 2024-03-23T17:46:06.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:05 smithi152 ceph-mon[47949]: osdmap e35: 6 total, 5 up, 6 in 2024-03-23T17:46:06.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:05 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T17:46:07.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:06 smithi086 ceph-mon[38221]: from='client.? 172.21.15.152:0/977058519' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T17:46:07.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:06 smithi092 ceph-mon[79089]: from='client.? 172.21.15.152:0/977058519' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T17:46:07.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:06 smithi152 ceph-mon[47949]: from='client.? 172.21.15.152:0/977058519' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T17:46:08.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:07 smithi086 ceph-mon[38221]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-23T17:46:08.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:07 smithi092 ceph-mon[79089]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-23T17:46:08.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:07 smithi152 ceph-mon[47949]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-23T17:46:09.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:09 smithi152 ceph-mon[47949]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-23T17:46:10.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:09 smithi086 ceph-mon[38221]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-23T17:46:10.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:09 smithi092 ceph-mon[79089]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-23T17:46:12.010 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:11 smithi152 ceph-mon[47949]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-23T17:46:12.010 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:11 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-23T17:46:12.010 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:11 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:46:12.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:11 smithi086 ceph-mon[38221]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-23T17:46:12.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:11 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-23T17:46:12.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:11 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:46:12.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:11 smithi092 ceph-mon[79089]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-23T17:46:12.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:11 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-23T17:46:12.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:11 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:46:12.935 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:12 smithi152 ceph-mon[47949]: Deploying daemon osd.5 on smithi152 2024-03-23T17:46:13.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:12 smithi086 ceph-mon[38221]: Deploying daemon osd.5 on smithi152 2024-03-23T17:46:13.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:12 smithi092 ceph-mon[79089]: Deploying daemon osd.5 on smithi152 2024-03-23T17:46:14.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:13 smithi086 ceph-mon[38221]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-23T17:46:14.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:13 smithi092 ceph-mon[79089]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-23T17:46:14.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:13 smithi152 ceph-mon[47949]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-23T17:46:15.837 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:15 smithi152 ceph-mon[47949]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-23T17:46:15.837 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:15 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:15.837 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:15 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:15.837 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:15 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:46:16.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:15 smithi086 ceph-mon[38221]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-23T17:46:16.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:15 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:16.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:15 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:16.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:15 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:46:16.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:15 smithi092 ceph-mon[79089]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-23T17:46:16.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:15 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:16.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:15 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:16.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:15 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:46:17.551 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:17 smithi152 ceph-mon[47949]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-23T17:46:17.551 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:17 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:17.551 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:17 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:17.551 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:17 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:46:17.551 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:17 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:46:17.551 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:17 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:17.612 INFO:teuthology.orchestra.run.smithi152.stdout:Created osd(s) 5 on host 'smithi152' 2024-03-23T17:46:17.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:46:17.611+0000 7fc0dffff700 1 -- 172.21.15.152:0/1502614645 <== mgr.14152 v2:172.21.15.86:6800/2734784244 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fc0f802cdc0 con 0x7fc0e00a3330 2024-03-23T17:46:17.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:46:17.613+0000 7fc0fd5ee700 1 -- 172.21.15.152:0/1502614645 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fc0e00a3330 msgr2=0x7fc0e00a57f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:46:17.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:46:17.613+0000 7fc0fd5ee700 1 --2- 172.21.15.152:0/1502614645 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fc0e00a3330 0x7fc0e00a57f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec009be0 tx=0x7fc0ec009260 comp rx=0 tx=0).stop 2024-03-23T17:46:17.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:46:17.614+0000 7fc0fd5ee700 1 -- 172.21.15.152:0/1502614645 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc0f80ffd10 msgr2=0x7fc0f8194c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:46:17.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:46:17.614+0000 7fc0fd5ee700 1 --2- 172.21.15.152:0/1502614645 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc0f80ffd10 0x7fc0f8194c00 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc0e800e9a0 tx=0x7fc0e800ed60 comp rx=0 tx=0).stop 2024-03-23T17:46:17.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:46:17.614+0000 7fc0fd5ee700 1 -- 172.21.15.152:0/1502614645 shutdown_connections 2024-03-23T17:46:17.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:46:17.614+0000 7fc0fd5ee700 1 --2- 172.21.15.152:0/1502614645 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fc0e00a3330 0x7fc0e00a57f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:46:17.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:46:17.614+0000 7fc0fd5ee700 1 --2- 172.21.15.152:0/1502614645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0f81018b0 0x7fc0f819c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:46:17.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:46:17.614+0000 7fc0fd5ee700 1 --2- 172.21.15.152:0/1502614645 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc0f80ffd10 0x7fc0f8194c00 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:46:17.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:46:17.614+0000 7fc0fd5ee700 1 --2- 172.21.15.152:0/1502614645 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0f8100f10 0x7fc0f8195140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:46:17.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:46:17.614+0000 7fc0fd5ee700 1 -- 172.21.15.152:0/1502614645 >> 172.21.15.152:0/1502614645 conn(0x7fc0f80fb2d0 msgr2=0x7fc0f8104290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:46:17.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:46:17.614+0000 7fc0fd5ee700 1 -- 172.21.15.152:0/1502614645 shutdown_connections 2024-03-23T17:46:17.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:46:17.614+0000 7fc0fd5ee700 1 -- 172.21.15.152:0/1502614645 wait complete. 2024-03-23T17:46:17.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:17 smithi086 ceph-mon[38221]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-23T17:46:17.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:17 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:17.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:17 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:17.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:17 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:46:17.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:17 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:46:17.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:17 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:17.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:17 smithi092 ceph-mon[79089]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-23T17:46:17.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:17 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:17.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:17 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:17.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:17 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:46:17.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:17 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:46:17.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:17 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:18.008 DEBUG:teuthology.orchestra.run.smithi152:osd.5> sudo journalctl -f -n 0 -u ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@osd.5.service 2024-03-23T17:46:18.011 INFO:tasks.cephadm:Deploying osd.6 on smithi086 with /dev/vg_nvme/lv_2... 2024-03-23T17:46:18.011 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-23T17:46:18.042 INFO:journalctl@ceph.osd.5.smithi152.stdout:-- Logs begin at Sat 2024-03-23 17:19:34 UTC. -- 2024-03-23T17:46:18.370 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T17:46:18.819 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:18 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:18.820 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:18 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:46:18.820 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:18 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:18.870 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:18 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:18.870 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:18 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:46:18.870 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:18 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:19.059 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:18 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:19.059 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:18 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:46:19.059 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:18 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:19.586 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 23 17:46:19 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-5[61654]: 2024-03-23T17:46:19.154+0000 7ff41c6a5640 -1 osd.5 0 log_to_monitors true 2024-03-23T17:46:19.947 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:19 smithi152 ceph-mon[47949]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-23T17:46:19.947 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:19 smithi152 ceph-mon[47949]: from='osd.5 [v2:172.21.15.152:6808/1039375239,v1:172.21.15.152:6809/1039375239]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-23T17:46:19.947 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:19 smithi152 ceph-mon[47949]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-23T17:46:20.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:19 smithi086 ceph-mon[38221]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-23T17:46:20.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:19 smithi086 ceph-mon[38221]: from='osd.5 [v2:172.21.15.152:6808/1039375239,v1:172.21.15.152:6809/1039375239]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-23T17:46:20.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:19 smithi086 ceph-mon[38221]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-23T17:46:20.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:19 smithi092 ceph-mon[79089]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-23T17:46:20.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:19 smithi092 ceph-mon[79089]: from='osd.5 [v2:172.21.15.152:6808/1039375239,v1:172.21.15.152:6809/1039375239]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-23T17:46:20.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:19 smithi092 ceph-mon[79089]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-23T17:46:20.083 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-23T17:46:20.083 INFO:teuthology.orchestra.run.smithi086.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-23T17:46:20.084 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 10+0 records in 2024-03-23T17:46:20.084 INFO:teuthology.orchestra.run.smithi086.stderr:10+0 records out 2024-03-23T17:46:20.084 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0309303 s, 339 MB/s 2024-03-23T17:46:20.084 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping successful for: 2024-03-23T17:46:20.481 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi086:vg_nvme/lv_2 2024-03-23T17:46:20.766 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:20 smithi086 ceph-mon[38221]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-23T17:46:20.766 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:20 smithi086 ceph-mon[38221]: osdmap e36: 6 total, 5 up, 6 in 2024-03-23T17:46:20.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:20 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T17:46:20.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:20 smithi086 ceph-mon[38221]: from='osd.5 [v2:172.21.15.152:6808/1039375239,v1:172.21.15.152:6809/1039375239]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-03-23T17:46:20.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:20 smithi086 ceph-mon[38221]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-03-23T17:46:20.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:20 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:20.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:20 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:20.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:20 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:46:20.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:20 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:46:20.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:20 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:20.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:20 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:46:20.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:20 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:46:20.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:20 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:20.832 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T17:46:21.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:20 smithi092 ceph-mon[79089]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-23T17:46:21.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:20 smithi092 ceph-mon[79089]: osdmap e36: 6 total, 5 up, 6 in 2024-03-23T17:46:21.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:20 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T17:46:21.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:20 smithi092 ceph-mon[79089]: from='osd.5 [v2:172.21.15.152:6808/1039375239,v1:172.21.15.152:6809/1039375239]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-03-23T17:46:21.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:20 smithi092 ceph-mon[79089]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-03-23T17:46:21.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:20 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:21.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:20 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:21.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:20 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:46:21.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:20 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:46:21.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:20 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:21.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:20 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:46:21.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:20 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:46:21.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:20 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:21.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:20 smithi152 ceph-mon[47949]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-23T17:46:21.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:20 smithi152 ceph-mon[47949]: osdmap e36: 6 total, 5 up, 6 in 2024-03-23T17:46:21.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:20 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T17:46:21.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:20 smithi152 ceph-mon[47949]: from='osd.5 [v2:172.21.15.152:6808/1039375239,v1:172.21.15.152:6809/1039375239]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-03-23T17:46:21.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:20 smithi152 ceph-mon[47949]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-03-23T17:46:21.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:20 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:21.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:20 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:21.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:20 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:46:21.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:20 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:46:21.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:20 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:21.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:20 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:46:21.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:20 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:46:21.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:20 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:21.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:21 smithi152 ceph-mon[47949]: Detected new or changed devices on smithi152 2024-03-23T17:46:21.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:21 smithi152 ceph-mon[47949]: Adjusting osd_memory_target on smithi152 to 8557M 2024-03-23T17:46:21.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:21 smithi152 ceph-mon[47949]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-23T17:46:21.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:21 smithi152 ceph-mon[47949]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]': finished 2024-03-23T17:46:21.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:21 smithi152 ceph-mon[47949]: osdmap e37: 6 total, 5 up, 6 in 2024-03-23T17:46:21.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:21 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T17:46:21.837 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:21 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T17:46:21.837 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:21 smithi152 ceph-mon[47949]: from='osd.5 ' entity='osd.5' 2024-03-23T17:46:21.837 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 23 17:46:21 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-5[61654]: 2024-03-23T17:46:21.421+0000 7ff411508700 -1 osd.5 0 waiting for initial osdmap 2024-03-23T17:46:21.837 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 23 17:46:21 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-5[61654]: 2024-03-23T17:46:21.430+0000 7ff409af6700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T17:46:21.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:46:21.979+0000 7f9dbbe00700 1 -- 172.21.15.86:0/2194290133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9dbc1027c0 msgr2=0x7f9dbc109070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:46:21.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:46:21.979+0000 7f9dbbe00700 1 --2- 172.21.15.86:0/2194290133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9dbc1027c0 0x7f9dbc109070 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9db400b0d0 tx=0x7f9db400b3e0 comp rx=0 tx=0).stop 2024-03-23T17:46:21.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:46:21.979+0000 7f9dbbe00700 1 -- 172.21.15.86:0/2194290133 shutdown_connections 2024-03-23T17:46:21.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:46:21.979+0000 7f9dbbe00700 1 --2- 172.21.15.86:0/2194290133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9dbc1027c0 0x7f9dbc109070 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:46:21.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:46:21.979+0000 7f9dbbe00700 1 --2- 172.21.15.86:0/2194290133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9dbc100c20 0x7f9dbc101040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:46:21.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:46:21.979+0000 7f9dbbe00700 1 --2- 172.21.15.86:0/2194290133 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9dbc101e20 0x7f9dbc102280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:46:21.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:46:21.979+0000 7f9dbbe00700 1 -- 172.21.15.86:0/2194290133 >> 172.21.15.86:0/2194290133 conn(0x7f9dbc0fc1c0 msgr2=0x7f9dbc0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:46:21.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:46:21.980+0000 7f9dbbe00700 1 -- 172.21.15.86:0/2194290133 shutdown_connections 2024-03-23T17:46:21.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:46:21.980+0000 7f9dbbe00700 1 -- 172.21.15.86:0/2194290133 wait complete. 2024-03-23T17:46:21.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:46:21.981+0000 7f9dbbe00700 1 Processor -- start 2024-03-23T17:46:21.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:46:21.981+0000 7f9dbbe00700 1 -- start start 2024-03-23T17:46:21.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:46:21.981+0000 7f9dbbe00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9dbc100c20 0x7f9dbc198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:46:21.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:46:21.981+0000 7f9dbbe00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9dbc101e20 0x7f9dbc1993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:46:21.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:46:21.981+0000 7f9dbbe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9dbc1027c0 0x7f9dbc1a0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:46:21.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:46:21.981+0000 7f9dbbe00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dbc1a09f0 con 0x7f9dbc1027c0 2024-03-23T17:46:21.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:46:21.981+0000 7f9dbbe00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dbc1a0b30 con 0x7f9dbc100c20 2024-03-23T17:46:21.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:46:21.981+0000 7f9dbbe00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dbc1a0cc0 con 0x7f9dbc101e20 2024-03-23T17:46:21.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:46:21.981+0000 7f9dbb400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9dbc1027c0 0x7f9dbc1a0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:46:21.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:46:21.981+0000 7f9dbb400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9dbc1027c0 0x7f9dbc1a0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49970/0 (socket says 172.21.15.86:49970) 2024-03-23T17:46:21.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:46:21.981+0000 7f9dbb400700 1 -- 172.21.15.86:0/1094831657 learned_addr learned my addr 172.21.15.86:0/1094831657 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:46:21.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:46:21.981+0000 7f9dba000700 1 --2- 172.21.15.86:0/1094831657 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9dbc101e20 0x7f9dbc1993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:46:21.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:46:21.981+0000 7f9dbaa00700 1 --2- 172.21.15.86:0/1094831657 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9dbc100c20 0x7f9dbc198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:46:21.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:46:21.982+0000 7f9dbb400700 1 -- 172.21.15.86:0/1094831657 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9dbc101e20 msgr2=0x7f9dbc1993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:46:21.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:46:21.982+0000 7f9dbb400700 1 --2- 172.21.15.86:0/1094831657 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9dbc101e20 0x7f9dbc1993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:46:21.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:46:21.982+0000 7f9dbb400700 1 -- 172.21.15.86:0/1094831657 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9dbc100c20 msgr2=0x7f9dbc198eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:46:21.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:46:21.982+0000 7f9dbb400700 1 --2- 172.21.15.86:0/1094831657 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9dbc100c20 0x7f9dbc198eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:46:21.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:46:21.982+0000 7f9dbb400700 1 -- 172.21.15.86:0/1094831657 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9db4009d00 con 0x7f9dbc1027c0 2024-03-23T17:46:21.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:46:21.982+0000 7f9dbb400700 1 --2- 172.21.15.86:0/1094831657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9dbc1027c0 0x7f9dbc1a0420 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9db4008e90 tx=0x7f9db4008ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:46:21.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:46:21.982+0000 7f9db3400700 1 -- 172.21.15.86:0/1094831657 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db400e040 con 0x7f9dbc1027c0 2024-03-23T17:46:21.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:46:21.982+0000 7f9db3400700 1 -- 172.21.15.86:0/1094831657 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9db401d930 con 0x7f9dbc1027c0 2024-03-23T17:46:21.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:46:21.982+0000 7f9dbbe00700 1 -- 172.21.15.86:0/1094831657 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dbc1a0f40 con 0x7f9dbc1027c0 2024-03-23T17:46:21.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:46:21.982+0000 7f9db3400700 1 -- 172.21.15.86:0/1094831657 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db401b960 con 0x7f9dbc1027c0 2024-03-23T17:46:21.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:46:21.982+0000 7f9dbbe00700 1 -- 172.21.15.86:0/1094831657 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dbc1a1300 con 0x7f9dbc1027c0 2024-03-23T17:46:21.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:46:21.983+0000 7f9dbbe00700 1 -- 172.21.15.86:0/1094831657 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dbc04ea80 con 0x7f9dbc1027c0 2024-03-23T17:46:21.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:46:21.985+0000 7f9db3400700 1 -- 172.21.15.86:0/1094831657 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f9db401daa0 con 0x7f9dbc1027c0 2024-03-23T17:46:21.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:46:21.987+0000 7f9db3400700 1 --2- 172.21.15.86:0/1094831657 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f9da00a1120 0x7f9da00a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:46:21.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:46:21.987+0000 7f9db3400700 1 -- 172.21.15.86:0/1094831657 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4566+0+0 (secure 0 0 0) 0x7f9db40cd6a0 con 0x7f9dbc1027c0 2024-03-23T17:46:21.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:46:21.987+0000 7f9dbaa00700 1 --2- 172.21.15.86:0/1094831657 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f9da00a1120 0x7f9da00a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:46:21.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:46:21.988+0000 7f9dbaa00700 1 --2- 172.21.15.86:0/1094831657 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f9da00a1120 0x7f9da00a35e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc101c80 tx=0x7f9dac00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:46:21.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:46:21.990+0000 7f9db3400700 1 -- 172.21.15.86:0/1094831657 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f9db4097cc0 con 0x7f9dbc1027c0 2024-03-23T17:46:22.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:21 smithi086 ceph-mon[38221]: Detected new or changed devices on smithi152 2024-03-23T17:46:22.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:21 smithi086 ceph-mon[38221]: Adjusting osd_memory_target on smithi152 to 8557M 2024-03-23T17:46:22.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:21 smithi086 ceph-mon[38221]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-23T17:46:22.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:21 smithi086 ceph-mon[38221]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]': finished 2024-03-23T17:46:22.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:21 smithi086 ceph-mon[38221]: osdmap e37: 6 total, 5 up, 6 in 2024-03-23T17:46:22.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:21 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T17:46:22.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:21 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T17:46:22.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:21 smithi086 ceph-mon[38221]: from='osd.5 ' entity='osd.5' 2024-03-23T17:46:22.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:21 smithi092 ceph-mon[79089]: Detected new or changed devices on smithi152 2024-03-23T17:46:22.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:21 smithi092 ceph-mon[79089]: Adjusting osd_memory_target on smithi152 to 8557M 2024-03-23T17:46:22.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:21 smithi092 ceph-mon[79089]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-23T17:46:22.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:21 smithi092 ceph-mon[79089]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]': finished 2024-03-23T17:46:22.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:21 smithi092 ceph-mon[79089]: osdmap e37: 6 total, 5 up, 6 in 2024-03-23T17:46:22.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:21 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T17:46:22.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:21 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T17:46:22.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:21 smithi092 ceph-mon[79089]: from='osd.5 ' entity='osd.5' 2024-03-23T17:46:22.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:46:22.204+0000 7f9dbbe00700 1 -- 172.21.15.86:0/1094831657 --> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f9dbc074700 con 0x7f9da00a1120 2024-03-23T17:46:22.932 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:22 smithi086 ceph-mon[38221]: purged_snaps scrub starts 2024-03-23T17:46:22.932 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:22 smithi086 ceph-mon[38221]: purged_snaps scrub ok 2024-03-23T17:46:22.932 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:22 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T17:46:22.932 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:22 smithi086 ceph-mon[38221]: osd.5 [v2:172.21.15.152:6808/1039375239,v1:172.21.15.152:6809/1039375239] boot 2024-03-23T17:46:22.932 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:22 smithi086 ceph-mon[38221]: osdmap e38: 6 total, 6 up, 6 in 2024-03-23T17:46:22.932 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:22 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T17:46:22.932 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:22 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T17:46:22.932 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:22 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T17:46:22.932 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:22 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:46:23.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:22 smithi092 ceph-mon[79089]: purged_snaps scrub starts 2024-03-23T17:46:23.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:22 smithi092 ceph-mon[79089]: purged_snaps scrub ok 2024-03-23T17:46:23.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:22 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T17:46:23.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:22 smithi092 ceph-mon[79089]: osd.5 [v2:172.21.15.152:6808/1039375239,v1:172.21.15.152:6809/1039375239] boot 2024-03-23T17:46:23.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:22 smithi092 ceph-mon[79089]: osdmap e38: 6 total, 6 up, 6 in 2024-03-23T17:46:23.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:22 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T17:46:23.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:22 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T17:46:23.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:22 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T17:46:23.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:22 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:46:23.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:22 smithi152 ceph-mon[47949]: purged_snaps scrub starts 2024-03-23T17:46:23.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:22 smithi152 ceph-mon[47949]: purged_snaps scrub ok 2024-03-23T17:46:23.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:22 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T17:46:23.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:22 smithi152 ceph-mon[47949]: osd.5 [v2:172.21.15.152:6808/1039375239,v1:172.21.15.152:6809/1039375239] boot 2024-03-23T17:46:23.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:22 smithi152 ceph-mon[47949]: osdmap e38: 6 total, 6 up, 6 in 2024-03-23T17:46:23.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:22 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T17:46:23.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:22 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T17:46:23.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:22 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T17:46:23.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:22 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:46:24.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:23 smithi086 ceph-mon[38221]: from='client.14310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:46:24.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:23 smithi086 ceph-mon[38221]: pgmap v115: 1 pgs: 1 peering; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-23T17:46:24.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:23 smithi086 ceph-mon[38221]: osdmap e39: 6 total, 6 up, 6 in 2024-03-23T17:46:24.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:23 smithi092 ceph-mon[79089]: from='client.14310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:46:24.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:23 smithi092 ceph-mon[79089]: pgmap v115: 1 pgs: 1 peering; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-23T17:46:24.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:23 smithi092 ceph-mon[79089]: osdmap e39: 6 total, 6 up, 6 in 2024-03-23T17:46:24.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:23 smithi152 ceph-mon[47949]: from='client.14310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:46:24.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:23 smithi152 ceph-mon[47949]: pgmap v115: 1 pgs: 1 peering; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-23T17:46:24.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:23 smithi152 ceph-mon[47949]: osdmap e39: 6 total, 6 up, 6 in 2024-03-23T17:46:25.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:24 smithi086 ceph-mon[38221]: osdmap e40: 6 total, 6 up, 6 in 2024-03-23T17:46:25.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:24 smithi092 ceph-mon[79089]: osdmap e40: 6 total, 6 up, 6 in 2024-03-23T17:46:25.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:24 smithi152 ceph-mon[47949]: osdmap e40: 6 total, 6 up, 6 in 2024-03-23T17:46:26.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:25 smithi086 ceph-mon[38221]: pgmap v118: 1 pgs: 1 peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-23T17:46:26.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:25 smithi086 ceph-mon[38221]: osdmap e41: 6 total, 6 up, 6 in 2024-03-23T17:46:26.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:25 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/4130506632' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "61024437-7b91-432f-a1af-6443d0ef5213"}]: dispatch 2024-03-23T17:46:26.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:25 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/4130506632' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "61024437-7b91-432f-a1af-6443d0ef5213"}]': finished 2024-03-23T17:46:26.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:25 smithi086 ceph-mon[38221]: osdmap e42: 7 total, 6 up, 7 in 2024-03-23T17:46:26.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:25 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-23T17:46:26.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:25 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/4027199076' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T17:46:26.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:25 smithi092 ceph-mon[79089]: pgmap v118: 1 pgs: 1 peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-23T17:46:26.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:25 smithi092 ceph-mon[79089]: osdmap e41: 6 total, 6 up, 6 in 2024-03-23T17:46:26.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:25 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/4130506632' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "61024437-7b91-432f-a1af-6443d0ef5213"}]: dispatch 2024-03-23T17:46:26.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:25 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/4130506632' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "61024437-7b91-432f-a1af-6443d0ef5213"}]': finished 2024-03-23T17:46:26.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:25 smithi092 ceph-mon[79089]: osdmap e42: 7 total, 6 up, 7 in 2024-03-23T17:46:26.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:25 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-23T17:46:26.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:25 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/4027199076' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T17:46:26.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:25 smithi152 ceph-mon[47949]: pgmap v118: 1 pgs: 1 peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-23T17:46:26.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:25 smithi152 ceph-mon[47949]: osdmap e41: 6 total, 6 up, 6 in 2024-03-23T17:46:26.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:25 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/4130506632' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "61024437-7b91-432f-a1af-6443d0ef5213"}]: dispatch 2024-03-23T17:46:26.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:25 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/4130506632' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "61024437-7b91-432f-a1af-6443d0ef5213"}]': finished 2024-03-23T17:46:26.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:25 smithi152 ceph-mon[47949]: osdmap e42: 7 total, 6 up, 7 in 2024-03-23T17:46:26.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:25 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-23T17:46:26.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:25 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/4027199076' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T17:46:28.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:27 smithi086 ceph-mon[38221]: pgmap v121: 1 pgs: 1 peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-23T17:46:28.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:27 smithi092 ceph-mon[79089]: pgmap v121: 1 pgs: 1 peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-23T17:46:28.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:27 smithi152 ceph-mon[47949]: pgmap v121: 1 pgs: 1 peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-23T17:46:29.996 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:29 smithi086 ceph-mon[38221]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-03-23T17:46:30.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:29 smithi092 ceph-mon[79089]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-03-23T17:46:30.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:29 smithi152 ceph-mon[47949]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-03-23T17:46:31.766 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:31 smithi086 ceph-mon[38221]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-03-23T17:46:31.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:31 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-23T17:46:31.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:31 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:46:32.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:31 smithi092 ceph-mon[79089]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-03-23T17:46:32.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:31 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-23T17:46:32.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:31 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:46:32.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:31 smithi152 ceph-mon[47949]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-03-23T17:46:32.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:31 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-23T17:46:32.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:31 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:46:33.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:32 smithi086 ceph-mon[38221]: Deploying daemon osd.6 on smithi086 2024-03-23T17:46:33.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:32 smithi092 ceph-mon[79089]: Deploying daemon osd.6 on smithi086 2024-03-23T17:46:33.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:32 smithi152 ceph-mon[47949]: Deploying daemon osd.6 on smithi086 2024-03-23T17:46:34.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:33 smithi086 ceph-mon[38221]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-23T17:46:34.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:33 smithi092 ceph-mon[79089]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-23T17:46:34.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:33 smithi152 ceph-mon[47949]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-23T17:46:35.960 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:35 smithi086 ceph-mon[38221]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-03-23T17:46:36.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:35 smithi092 ceph-mon[79089]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-03-23T17:46:36.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:35 smithi152 ceph-mon[47949]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-03-23T17:46:38.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:37 smithi086 ceph-mon[38221]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-03-23T17:46:38.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:37 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:38.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:37 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:46:38.009 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:37 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:38.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:37 smithi092 ceph-mon[79089]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-03-23T17:46:38.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:37 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:38.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:37 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:46:38.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:37 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:38.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:37 smithi152 ceph-mon[47949]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-03-23T17:46:38.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:37 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:38.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:37 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:46:38.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:37 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:39.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:46:39.506+0000 7f9db3400700 1 -- 172.21.15.86:0/1094831657 <== mgr.14152 v2:172.21.15.86:6800/2734784244 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f9dbc074700 con 0x7f9da00a1120 2024-03-23T17:46:39.508 INFO:teuthology.orchestra.run.smithi086.stdout:Created osd(s) 6 on host 'smithi086' 2024-03-23T17:46:39.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:46:39.508+0000 7f9dbbe00700 1 -- 172.21.15.86:0/1094831657 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f9da00a1120 msgr2=0x7f9da00a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:46:39.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:46:39.508+0000 7f9dbbe00700 1 --2- 172.21.15.86:0/1094831657 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f9da00a1120 0x7f9da00a35e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc101c80 tx=0x7f9dac00b410 comp rx=0 tx=0).stop 2024-03-23T17:46:39.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:46:39.508+0000 7f9dbbe00700 1 -- 172.21.15.86:0/1094831657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9dbc1027c0 msgr2=0x7f9dbc1a0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:46:39.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:46:39.508+0000 7f9dbbe00700 1 --2- 172.21.15.86:0/1094831657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9dbc1027c0 0x7f9dbc1a0420 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9db4008e90 tx=0x7f9db4008ec0 comp rx=0 tx=0).stop 2024-03-23T17:46:39.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:46:39.509+0000 7f9dbbe00700 1 -- 172.21.15.86:0/1094831657 shutdown_connections 2024-03-23T17:46:39.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:46:39.509+0000 7f9dbbe00700 1 --2- 172.21.15.86:0/1094831657 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f9da00a1120 0x7f9da00a35e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:46:39.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:46:39.509+0000 7f9dbbe00700 1 --2- 172.21.15.86:0/1094831657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9dbc1027c0 0x7f9dbc1a0420 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:46:39.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:46:39.509+0000 7f9dbbe00700 1 --2- 172.21.15.86:0/1094831657 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9dbc100c20 0x7f9dbc198eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:46:39.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:46:39.509+0000 7f9dbbe00700 1 --2- 172.21.15.86:0/1094831657 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9dbc101e20 0x7f9dbc1993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:46:39.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:46:39.509+0000 7f9dbbe00700 1 -- 172.21.15.86:0/1094831657 >> 172.21.15.86:0/1094831657 conn(0x7f9dbc0fc1c0 msgr2=0x7f9dbc072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:46:39.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:46:39.509+0000 7f9dbbe00700 1 -- 172.21.15.86:0/1094831657 shutdown_connections 2024-03-23T17:46:39.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:46:39.509+0000 7f9dbbe00700 1 -- 172.21.15.86:0/1094831657 wait complete. 2024-03-23T17:46:39.887 DEBUG:teuthology.orchestra.run.smithi086:osd.6> sudo journalctl -f -n 0 -u ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@osd.6.service 2024-03-23T17:46:39.891 INFO:tasks.cephadm:Deploying osd.7 on smithi092 with /dev/vg_nvme/lv_2... 2024-03-23T17:46:39.891 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-23T17:46:39.966 INFO:journalctl@ceph.osd.6.smithi086.stdout:-- Logs begin at Sat 2024-03-23 17:25:35 UTC. -- 2024-03-23T17:46:40.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:39 smithi086 ceph-mon[38221]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-23T17:46:40.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:39 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:40.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:39 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:40.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:39 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:46:40.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:39 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:46:40.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:39 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:40.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:39 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:40.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:39 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:40.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:39 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:46:40.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:39 smithi092 ceph-mon[79089]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-23T17:46:40.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:39 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:40.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:39 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:40.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:39 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:46:40.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:39 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:46:40.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:39 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:40.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:39 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:40.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:39 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:40.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:39 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:46:40.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:39 smithi152 ceph-mon[47949]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-23T17:46:40.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:39 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:40.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:39 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:40.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:39 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:46:40.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:39 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:46:40.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:39 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:40.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:39 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:40.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:39 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:40.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:39 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:46:40.145 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.b/config 2024-03-23T17:46:40.766 INFO:journalctl@ceph.osd.6.smithi086.stdout:Mar 23 17:46:40 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-6[74047]: 2024-03-23T17:46:40.717+0000 7f803007e640 -1 osd.6 0 log_to_monitors true 2024-03-23T17:46:41.460 INFO:teuthology.orchestra.run.smithi092.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-23T17:46:41.461 INFO:teuthology.orchestra.run.smithi092.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-23T17:46:41.461 INFO:teuthology.orchestra.run.smithi092.stderr: stderr: 10+0 records in 2024-03-23T17:46:41.461 INFO:teuthology.orchestra.run.smithi092.stderr:10+0 records out 2024-03-23T17:46:41.461 INFO:teuthology.orchestra.run.smithi092.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0116006 s, 904 MB/s 2024-03-23T17:46:41.461 INFO:teuthology.orchestra.run.smithi092.stderr:--> Zapping successful for: 2024-03-23T17:46:41.769 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:41 smithi092 ceph-mon[79089]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-23T17:46:41.769 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:41 smithi092 ceph-mon[79089]: from='osd.6 [v2:172.21.15.86:6818/4080452707,v1:172.21.15.86:6819/4080452707]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-23T17:46:41.769 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:41 smithi092 ceph-mon[79089]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-23T17:46:41.792 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi092:vg_nvme/lv_2 2024-03-23T17:46:41.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:41 smithi152 ceph-mon[47949]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-23T17:46:41.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:41 smithi152 ceph-mon[47949]: from='osd.6 [v2:172.21.15.86:6818/4080452707,v1:172.21.15.86:6819/4080452707]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-23T17:46:41.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:41 smithi152 ceph-mon[47949]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-23T17:46:42.010 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.b/config 2024-03-23T17:46:42.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:41 smithi086 ceph-mon[38221]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-23T17:46:42.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:41 smithi086 ceph-mon[38221]: from='osd.6 [v2:172.21.15.86:6818/4080452707,v1:172.21.15.86:6819/4080452707]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-23T17:46:42.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:41 smithi086 ceph-mon[38221]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-23T17:46:42.786 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:42 smithi092 ceph-mon[79089]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-23T17:46:42.786 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:42 smithi092 ceph-mon[79089]: osdmap e43: 7 total, 6 up, 7 in 2024-03-23T17:46:42.786 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:42 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-23T17:46:42.786 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:42 smithi092 ceph-mon[79089]: from='osd.6 [v2:172.21.15.86:6818/4080452707,v1:172.21.15.86:6819/4080452707]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-03-23T17:46:42.786 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:42 smithi092 ceph-mon[79089]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-03-23T17:46:42.994 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:46:42.994+0000 7f15131af700 1 -- 172.21.15.92:0/3467635623 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f150c102a30 msgr2=0x7f150c1092e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:46:42.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:46:42.994+0000 7f15131af700 1 --2- 172.21.15.92:0/3467635623 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f150c102a30 0x7f150c1092e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f150400b0a0 tx=0x7f150400b3b0 comp rx=0 tx=0).stop 2024-03-23T17:46:42.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:46:42.994+0000 7f15131af700 1 -- 172.21.15.92:0/3467635623 shutdown_connections 2024-03-23T17:46:42.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:46:42.994+0000 7f15131af700 1 --2- 172.21.15.92:0/3467635623 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f150c102a30 0x7f150c1092e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:46:42.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:46:42.994+0000 7f15131af700 1 --2- 172.21.15.92:0/3467635623 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f150c0ff0b0 0x7f150c0ff4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:46:42.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:46:42.994+0000 7f15131af700 1 --2- 172.21.15.92:0/3467635623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f150c102090 0x7f150c1024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:46:42.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:46:42.994+0000 7f15131af700 1 -- 172.21.15.92:0/3467635623 >> 172.21.15.92:0/3467635623 conn(0x7f150c0faa10 msgr2=0x7f150c0fce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:46:42.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:46:42.994+0000 7f15131af700 1 -- 172.21.15.92:0/3467635623 shutdown_connections 2024-03-23T17:46:42.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:46:42.994+0000 7f15131af700 1 -- 172.21.15.92:0/3467635623 wait complete. 2024-03-23T17:46:42.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:46:42.995+0000 7f15131af700 1 Processor -- start 2024-03-23T17:46:42.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:46:42.995+0000 7f15131af700 1 -- start start 2024-03-23T17:46:42.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:46:42.995+0000 7f15131af700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f150c0ff0b0 0x7f150c196e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:46:42.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:46:42.995+0000 7f15131af700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f150c102090 0x7f150c197370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:46:42.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:46:42.995+0000 7f15131af700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f150c102a30 0x7f150c19e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:46:42.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:46:42.995+0000 7f15131af700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150c19e970 con 0x7f150c102a30 2024-03-23T17:46:42.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:46:42.995+0000 7f15131af700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150c19eab0 con 0x7f150c0ff0b0 2024-03-23T17:46:42.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:46:42.995+0000 7f15131af700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150c19ec40 con 0x7f150c102090 2024-03-23T17:46:42.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:46:42.995+0000 7f1510f4b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f150c0ff0b0 0x7f150c196e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:46:42.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:46:42.995+0000 7f1510f4b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f150c0ff0b0 0x7f150c196e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50776/0 (socket says 172.21.15.92:50776) 2024-03-23T17:46:42.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:46:42.995+0000 7f1510f4b700 1 -- 172.21.15.92:0/3518131934 learned_addr learned my addr 172.21.15.92:0/3518131934 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-23T17:46:42.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:46:42.995+0000 7f151174c700 1 --2- 172.21.15.92:0/3518131934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f150c102a30 0x7f150c19e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:46:42.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:46:42.995+0000 7f150bfff700 1 --2- 172.21.15.92:0/3518131934 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f150c102090 0x7f150c197370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:46:42.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:46:42.995+0000 7f1510f4b700 1 -- 172.21.15.92:0/3518131934 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f150c102090 msgr2=0x7f150c197370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:46:42.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:46:42.995+0000 7f1510f4b700 1 --2- 172.21.15.92:0/3518131934 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f150c102090 0x7f150c197370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:46:42.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:46:42.995+0000 7f1510f4b700 1 -- 172.21.15.92:0/3518131934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f150c102a30 msgr2=0x7f150c19e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:46:42.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:46:42.995+0000 7f1510f4b700 1 --2- 172.21.15.92:0/3518131934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f150c102a30 0x7f150c19e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:46:42.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:46:42.995+0000 7f1510f4b700 1 -- 172.21.15.92:0/3518131934 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1504009d00 con 0x7f150c0ff0b0 2024-03-23T17:46:42.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:46:42.995+0000 7f151174c700 1 --2- 172.21.15.92:0/3518131934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f150c102a30 0x7f150c19e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:46:42.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:46:42.995+0000 7f1510f4b700 1 --2- 172.21.15.92:0/3518131934 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f150c0ff0b0 0x7f150c196e30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f150000b500 tx=0x7f150000b8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:46:42.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:46:42.995+0000 7f1509ffb700 1 -- 172.21.15.92:0/3518131934 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1500011830 con 0x7f150c0ff0b0 2024-03-23T17:46:42.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:46:42.996+0000 7f1509ffb700 1 -- 172.21.15.92:0/3518131934 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1500011e50 con 0x7f150c0ff0b0 2024-03-23T17:46:42.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:46:42.996+0000 7f15131af700 1 -- 172.21.15.92:0/3518131934 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f150c19ef20 con 0x7f150c0ff0b0 2024-03-23T17:46:43.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:46:42.996+0000 7f1509ffb700 1 -- 172.21.15.92:0/3518131934 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f150000f2e0 con 0x7f150c0ff0b0 2024-03-23T17:46:43.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:46:42.996+0000 7f15131af700 1 -- 172.21.15.92:0/3518131934 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f150c19f330 con 0x7f150c0ff0b0 2024-03-23T17:46:43.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:46:42.997+0000 7f15131af700 1 -- 172.21.15.92:0/3518131934 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f150c1082c0 con 0x7f150c0ff0b0 2024-03-23T17:46:43.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:46:42.998+0000 7f1509ffb700 1 -- 172.21.15.92:0/3518131934 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f1500010580 con 0x7f150c0ff0b0 2024-03-23T17:46:43.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:46:42.998+0000 7f1509ffb700 1 --2- 172.21.15.92:0/3518131934 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f14fc0a1120 0x7f14fc0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:46:43.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:46:42.998+0000 7f1509ffb700 1 -- 172.21.15.92:0/3518131934 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 4685+0+0 (secure 0 0 0) 0x7f15000cb520 con 0x7f150c0ff0b0 2024-03-23T17:46:43.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:46:42.998+0000 7f150bfff700 1 --2- 172.21.15.92:0/3518131934 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f14fc0a1120 0x7f14fc0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:46:43.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:46:42.999+0000 7f150bfff700 1 --2- 172.21.15.92:0/3518131934 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f14fc0a1120 0x7f14fc0a35e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f14f8005950 tx=0x7f14f80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:46:43.002 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:46:43.002+0000 7f1509ffb700 1 -- 172.21.15.92:0/3518131934 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f1500095ae0 con 0x7f150c0ff0b0 2024-03-23T17:46:43.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:42 smithi086 ceph-mon[38221]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-23T17:46:43.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:42 smithi086 ceph-mon[38221]: osdmap e43: 7 total, 6 up, 7 in 2024-03-23T17:46:43.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:42 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-23T17:46:43.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:42 smithi086 ceph-mon[38221]: from='osd.6 [v2:172.21.15.86:6818/4080452707,v1:172.21.15.86:6819/4080452707]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-03-23T17:46:43.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:42 smithi086 ceph-mon[38221]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-03-23T17:46:43.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:42 smithi152 ceph-mon[47949]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-23T17:46:43.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:42 smithi152 ceph-mon[47949]: osdmap e43: 7 total, 6 up, 7 in 2024-03-23T17:46:43.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:42 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-23T17:46:43.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:42 smithi152 ceph-mon[47949]: from='osd.6 [v2:172.21.15.86:6818/4080452707,v1:172.21.15.86:6819/4080452707]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-03-23T17:46:43.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:42 smithi152 ceph-mon[47949]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-03-23T17:46:43.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:46:43.208+0000 7f15131af700 1 -- 172.21.15.92:0/3518131934 --> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f150c02ce10 con 0x7f14fc0a1120 2024-03-23T17:46:43.989 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:43 smithi092 ceph-mon[79089]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-23T17:46:43.990 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:43 smithi092 ceph-mon[79089]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-03-23T17:46:43.990 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:43 smithi092 ceph-mon[79089]: osdmap e44: 7 total, 6 up, 7 in 2024-03-23T17:46:43.990 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:43 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-23T17:46:43.990 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:43 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-23T17:46:43.990 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:43 smithi092 ceph-mon[79089]: Detected new or changed devices on smithi086 2024-03-23T17:46:43.990 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:43 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:43.990 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:43 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:43.990 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:43 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:46:43.990 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:43 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:46:43.990 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:43 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:46:43.990 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:43 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:43.991 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:43 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:46:43.991 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:43 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:46:43.991 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:43 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:43.991 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:43 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T17:46:43.991 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:43 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T17:46:43.991 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:43 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:46:43.991 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:43 smithi092 ceph-mon[79089]: from='osd.6 ' entity='osd.6' 2024-03-23T17:46:44.016 INFO:journalctl@ceph.osd.6.smithi086.stdout:Mar 23 17:46:43 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-6[74047]: 2024-03-23T17:46:43.551+0000 7f8023c00700 -1 osd.6 0 waiting for initial osdmap 2024-03-23T17:46:44.016 INFO:journalctl@ceph.osd.6.smithi086.stdout:Mar 23 17:46:43 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-6[74047]: 2024-03-23T17:46:43.579+0000 7f801a400700 -1 osd.6 44 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T17:46:44.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:43 smithi086 ceph-mon[38221]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-23T17:46:44.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:43 smithi086 ceph-mon[38221]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-03-23T17:46:44.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:43 smithi086 ceph-mon[38221]: osdmap e44: 7 total, 6 up, 7 in 2024-03-23T17:46:44.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:43 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-23T17:46:44.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:43 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-23T17:46:44.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:43 smithi086 ceph-mon[38221]: Detected new or changed devices on smithi086 2024-03-23T17:46:44.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:43 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:44.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:43 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:44.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:43 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:46:44.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:43 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:46:44.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:43 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:46:44.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:43 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:44.018 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:43 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:46:44.018 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:43 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:46:44.018 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:43 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:44.018 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:43 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T17:46:44.018 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:43 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T17:46:44.018 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:43 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:46:44.018 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:43 smithi086 ceph-mon[38221]: from='osd.6 ' entity='osd.6' 2024-03-23T17:46:44.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:43 smithi152 ceph-mon[47949]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-23T17:46:44.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:43 smithi152 ceph-mon[47949]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-03-23T17:46:44.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:43 smithi152 ceph-mon[47949]: osdmap e44: 7 total, 6 up, 7 in 2024-03-23T17:46:44.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:43 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-23T17:46:44.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:43 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-23T17:46:44.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:43 smithi152 ceph-mon[47949]: Detected new or changed devices on smithi086 2024-03-23T17:46:44.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:43 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:44.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:43 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:44.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:43 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:46:44.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:43 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:46:44.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:43 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:46:44.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:43 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:44.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:43 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:46:44.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:43 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:46:44.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:43 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:44.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:43 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T17:46:44.088 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:43 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T17:46:44.088 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:43 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:46:44.088 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:43 smithi152 ceph-mon[47949]: from='osd.6 ' entity='osd.6' 2024-03-23T17:46:45.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:44 smithi086 ceph-mon[38221]: purged_snaps scrub starts 2024-03-23T17:46:45.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:44 smithi086 ceph-mon[38221]: purged_snaps scrub ok 2024-03-23T17:46:45.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:44 smithi086 ceph-mon[38221]: Adjusting osd_memory_target on smithi086 to 5756M 2024-03-23T17:46:45.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:44 smithi086 ceph-mon[38221]: from='client.24299 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:46:45.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:44 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-23T17:46:45.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:44 smithi086 ceph-mon[38221]: osd.6 [v2:172.21.15.86:6818/4080452707,v1:172.21.15.86:6819/4080452707] boot 2024-03-23T17:46:45.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:44 smithi086 ceph-mon[38221]: osdmap e45: 7 total, 7 up, 7 in 2024-03-23T17:46:45.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:44 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-23T17:46:45.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:44 smithi092 ceph-mon[79089]: purged_snaps scrub starts 2024-03-23T17:46:45.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:44 smithi092 ceph-mon[79089]: purged_snaps scrub ok 2024-03-23T17:46:45.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:44 smithi092 ceph-mon[79089]: Adjusting osd_memory_target on smithi086 to 5756M 2024-03-23T17:46:45.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:44 smithi092 ceph-mon[79089]: from='client.24299 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:46:45.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:44 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-23T17:46:45.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:44 smithi092 ceph-mon[79089]: osd.6 [v2:172.21.15.86:6818/4080452707,v1:172.21.15.86:6819/4080452707] boot 2024-03-23T17:46:45.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:44 smithi092 ceph-mon[79089]: osdmap e45: 7 total, 7 up, 7 in 2024-03-23T17:46:45.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:44 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-23T17:46:45.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:44 smithi152 ceph-mon[47949]: purged_snaps scrub starts 2024-03-23T17:46:45.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:44 smithi152 ceph-mon[47949]: purged_snaps scrub ok 2024-03-23T17:46:45.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:44 smithi152 ceph-mon[47949]: Adjusting osd_memory_target on smithi086 to 5756M 2024-03-23T17:46:45.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:44 smithi152 ceph-mon[47949]: from='client.24299 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:46:45.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:44 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-23T17:46:45.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:44 smithi152 ceph-mon[47949]: osd.6 [v2:172.21.15.86:6818/4080452707,v1:172.21.15.86:6819/4080452707] boot 2024-03-23T17:46:45.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:44 smithi152 ceph-mon[47949]: osdmap e45: 7 total, 7 up, 7 in 2024-03-23T17:46:45.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:44 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-23T17:46:46.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:45 smithi086 ceph-mon[38221]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-23T17:46:46.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:45 smithi086 ceph-mon[38221]: from='client.? 172.21.15.92:0/3220558059' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2684bdae-6b30-4afc-af68-4ce41b3a75b6"}]: dispatch 2024-03-23T17:46:46.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:45 smithi086 ceph-mon[38221]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2684bdae-6b30-4afc-af68-4ce41b3a75b6"}]: dispatch 2024-03-23T17:46:46.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:45 smithi086 ceph-mon[38221]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2684bdae-6b30-4afc-af68-4ce41b3a75b6"}]': finished 2024-03-23T17:46:46.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:45 smithi086 ceph-mon[38221]: osdmap e46: 8 total, 7 up, 8 in 2024-03-23T17:46:46.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:45 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-23T17:46:46.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:45 smithi092 ceph-mon[79089]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-23T17:46:46.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:45 smithi092 ceph-mon[79089]: from='client.? 172.21.15.92:0/3220558059' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2684bdae-6b30-4afc-af68-4ce41b3a75b6"}]: dispatch 2024-03-23T17:46:46.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:45 smithi092 ceph-mon[79089]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2684bdae-6b30-4afc-af68-4ce41b3a75b6"}]: dispatch 2024-03-23T17:46:46.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:45 smithi092 ceph-mon[79089]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2684bdae-6b30-4afc-af68-4ce41b3a75b6"}]': finished 2024-03-23T17:46:46.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:45 smithi092 ceph-mon[79089]: osdmap e46: 8 total, 7 up, 8 in 2024-03-23T17:46:46.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:45 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-23T17:46:46.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:45 smithi152 ceph-mon[47949]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-23T17:46:46.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:45 smithi152 ceph-mon[47949]: from='client.? 172.21.15.92:0/3220558059' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2684bdae-6b30-4afc-af68-4ce41b3a75b6"}]: dispatch 2024-03-23T17:46:46.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:45 smithi152 ceph-mon[47949]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2684bdae-6b30-4afc-af68-4ce41b3a75b6"}]: dispatch 2024-03-23T17:46:46.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:45 smithi152 ceph-mon[47949]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2684bdae-6b30-4afc-af68-4ce41b3a75b6"}]': finished 2024-03-23T17:46:46.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:45 smithi152 ceph-mon[47949]: osdmap e46: 8 total, 7 up, 8 in 2024-03-23T17:46:46.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:45 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-23T17:46:47.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:46 smithi086 ceph-mon[38221]: from='client.? 172.21.15.92:0/3644205324' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T17:46:47.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:46 smithi092 ceph-mon[79089]: from='client.? 172.21.15.92:0/3644205324' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T17:46:47.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:46 smithi152 ceph-mon[47949]: from='client.? 172.21.15.92:0/3644205324' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T17:46:48.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:47 smithi086 ceph-mon[38221]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-03-23T17:46:48.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:47 smithi092 ceph-mon[79089]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-03-23T17:46:48.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:47 smithi152 ceph-mon[47949]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-03-23T17:46:49.872 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:49 smithi092 ceph-mon[79089]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-03-23T17:46:50.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:49 smithi086 ceph-mon[38221]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-03-23T17:46:50.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:49 smithi152 ceph-mon[47949]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-03-23T17:46:51.763 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:51 smithi092 ceph-mon[79089]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-03-23T17:46:51.763 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:51 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-23T17:46:51.763 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:51 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:46:52.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:51 smithi086 ceph-mon[38221]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-03-23T17:46:52.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:51 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-23T17:46:52.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:51 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:46:52.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:51 smithi152 ceph-mon[47949]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-03-23T17:46:52.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:51 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-23T17:46:52.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:51 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:46:52.741 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:52 smithi092 ceph-mon[79089]: Deploying daemon osd.7 on smithi092 2024-03-23T17:46:53.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:52 smithi086 ceph-mon[38221]: Deploying daemon osd.7 on smithi092 2024-03-23T17:46:53.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:52 smithi152 ceph-mon[47949]: Deploying daemon osd.7 on smithi092 2024-03-23T17:46:53.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:53 smithi092 ceph-mon[79089]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-03-23T17:46:54.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:53 smithi086 ceph-mon[38221]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-03-23T17:46:54.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:53 smithi152 ceph-mon[47949]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-03-23T17:46:55.957 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:55 smithi092 ceph-mon[79089]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-03-23T17:46:55.957 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:55 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:55.957 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:55 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:55.957 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:55 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:46:56.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:55 smithi086 ceph-mon[38221]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-03-23T17:46:56.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:55 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:56.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:55 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:56.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:55 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:46:56.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:55 smithi152 ceph-mon[47949]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-03-23T17:46:56.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:55 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:56.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:55 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:56.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:55 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:46:56.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:46:56.813+0000 7f1509ffb700 1 -- 172.21.15.92:0/3518131934 <== mgr.14152 v2:172.21.15.86:6800/2734784244 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f150c02ce10 con 0x7f14fc0a1120 2024-03-23T17:46:56.813 INFO:teuthology.orchestra.run.smithi092.stdout:Created osd(s) 7 on host 'smithi092' 2024-03-23T17:46:56.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:46:56.815+0000 7f15131af700 1 -- 172.21.15.92:0/3518131934 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f14fc0a1120 msgr2=0x7f14fc0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:46:56.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:46:56.815+0000 7f15131af700 1 --2- 172.21.15.92:0/3518131934 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f14fc0a1120 0x7f14fc0a35e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f14f8005950 tx=0x7f14f80058e0 comp rx=0 tx=0).stop 2024-03-23T17:46:56.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:46:56.815+0000 7f15131af700 1 -- 172.21.15.92:0/3518131934 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f150c0ff0b0 msgr2=0x7f150c196e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:46:56.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:46:56.815+0000 7f15131af700 1 --2- 172.21.15.92:0/3518131934 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f150c0ff0b0 0x7f150c196e30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f150000b500 tx=0x7f150000b8c0 comp rx=0 tx=0).stop 2024-03-23T17:46:56.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:46:56.815+0000 7f15131af700 1 -- 172.21.15.92:0/3518131934 shutdown_connections 2024-03-23T17:46:56.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:46:56.815+0000 7f15131af700 1 --2- 172.21.15.92:0/3518131934 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f14fc0a1120 0x7f14fc0a35e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:46:56.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:46:56.815+0000 7f15131af700 1 --2- 172.21.15.92:0/3518131934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f150c102a30 0x7f150c19e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:46:56.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:46:56.815+0000 7f15131af700 1 --2- 172.21.15.92:0/3518131934 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f150c0ff0b0 0x7f150c196e30 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:46:56.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:46:56.815+0000 7f15131af700 1 --2- 172.21.15.92:0/3518131934 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f150c102090 0x7f150c197370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:46:56.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:46:56.815+0000 7f15131af700 1 -- 172.21.15.92:0/3518131934 >> 172.21.15.92:0/3518131934 conn(0x7f150c0faa10 msgr2=0x7f150c105400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:46:56.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:46:56.815+0000 7f15131af700 1 -- 172.21.15.92:0/3518131934 shutdown_connections 2024-03-23T17:46:56.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:46:56.815+0000 7f15131af700 1 -- 172.21.15.92:0/3518131934 wait complete. 2024-03-23T17:46:57.189 DEBUG:teuthology.orchestra.run.smithi092:osd.7> sudo journalctl -f -n 0 -u ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@osd.7.service 2024-03-23T17:46:57.192 INFO:tasks.cephadm:Deploying osd.8 on smithi152 with /dev/vg_nvme/lv_2... 2024-03-23T17:46:57.192 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-23T17:46:57.236 INFO:journalctl@ceph.osd.7.smithi092.stdout:-- Logs begin at Sat 2024-03-23 17:19:16 UTC. -- 2024-03-23T17:46:57.438 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.c/config 2024-03-23T17:46:57.461 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:57 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:57.461 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:57 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:57.461 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:57 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:46:57.461 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:57 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:46:57.461 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:57 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:57.461 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:57 smithi092 ceph-mon[79089]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-03-23T17:46:57.461 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:57 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:57.461 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:57 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:57.461 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:57 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:46:57.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:57 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:57.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:57 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:57.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:57 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:46:57.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:57 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:46:57.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:57 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:57.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:57 smithi086 ceph-mon[38221]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-03-23T17:46:57.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:57 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:57.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:57 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:57.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:57 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:46:57.550 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:57 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:57.550 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:57 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:57.550 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:57 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:46:57.550 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:57 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:46:57.550 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:57 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:57.550 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:57 smithi152 ceph-mon[47949]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-03-23T17:46:57.550 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:57 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:57.550 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:57 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:57.550 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:57 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:46:58.721 INFO:teuthology.orchestra.run.smithi152.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-23T17:46:58.721 INFO:teuthology.orchestra.run.smithi152.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-23T17:46:58.721 INFO:teuthology.orchestra.run.smithi152.stderr: stderr: 10+0 records in 2024-03-23T17:46:58.721 INFO:teuthology.orchestra.run.smithi152.stderr:10+0 records out 2024-03-23T17:46:58.721 INFO:teuthology.orchestra.run.smithi152.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112265 s, 934 MB/s 2024-03-23T17:46:58.721 INFO:teuthology.orchestra.run.smithi152.stderr:--> Zapping successful for: 2024-03-23T17:46:59.037 INFO:journalctl@ceph.osd.7.smithi092.stdout:Mar 23 17:46:58 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-7[100796]: 2024-03-23T17:46:58.691+0000 7fb54ce15640 -1 osd.7 0 log_to_monitors true 2024-03-23T17:46:59.095 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi152:vg_nvme/lv_2 2024-03-23T17:46:59.312 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.c/config 2024-03-23T17:46:59.766 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:59 smithi086 ceph-mon[38221]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-03-23T17:46:59.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:59 smithi086 ceph-mon[38221]: from='osd.7 [v2:172.21.15.92:6816/4187707622,v1:172.21.15.92:6817/4187707622]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-23T17:46:59.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:59 smithi086 ceph-mon[38221]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-23T17:46:59.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:59 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:59.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:59 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:59.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:59 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:46:59.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:59 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:46:59.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:59 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:46:59.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:59 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:59.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:59 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:46:59.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:59 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:46:59.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:46:59 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:59.782 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:59 smithi152 ceph-mon[47949]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-03-23T17:46:59.782 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:59 smithi152 ceph-mon[47949]: from='osd.7 [v2:172.21.15.92:6816/4187707622,v1:172.21.15.92:6817/4187707622]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-23T17:46:59.782 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:59 smithi152 ceph-mon[47949]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-23T17:46:59.783 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:59 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:59.783 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:59 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:59.783 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:59 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:46:59.783 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:59 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:46:59.783 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:59 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:46:59.783 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:59 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:59.783 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:59 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:46:59.783 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:59 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:46:59.783 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:46:59 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:59.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:59 smithi092 ceph-mon[79089]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-03-23T17:46:59.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:59 smithi092 ceph-mon[79089]: from='osd.7 [v2:172.21.15.92:6816/4187707622,v1:172.21.15.92:6817/4187707622]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-23T17:46:59.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:59 smithi092 ceph-mon[79089]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-23T17:46:59.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:59 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:59.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:59 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:59.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:59 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:46:59.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:59 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:46:59.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:59 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:46:59.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:59 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:46:59.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:59 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:46:59.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:59 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:46:59.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:46:59 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:00.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:00.326+0000 7f772e350700 1 -- 172.21.15.152:0/406346612 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7728100de0 msgr2=0x7f7728101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:47:00.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:00.327+0000 7f772e350700 1 --2- 172.21.15.152:0/406346612 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7728100de0 0x7f7728101200 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7718009990 tx=0x7f7718009ca0 comp rx=0 tx=0).stop 2024-03-23T17:47:00.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:00.327+0000 7f772e350700 1 -- 172.21.15.152:0/406346612 shutdown_connections 2024-03-23T17:47:00.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:00.327+0000 7f772e350700 1 --2- 172.21.15.152:0/406346612 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7728102980 0x7f7728109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:47:00.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:00.327+0000 7f772e350700 1 --2- 172.21.15.152:0/406346612 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7728100de0 0x7f7728101200 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:47:00.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:00.327+0000 7f772e350700 1 --2- 172.21.15.152:0/406346612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7728101fe0 0x7f7728102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:47:00.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:00.327+0000 7f772e350700 1 -- 172.21.15.152:0/406346612 >> 172.21.15.152:0/406346612 conn(0x7f77280fc380 msgr2=0x7f77280fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:47:00.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:00.327+0000 7f772e350700 1 -- 172.21.15.152:0/406346612 shutdown_connections 2024-03-23T17:47:00.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:00.327+0000 7f772e350700 1 -- 172.21.15.152:0/406346612 wait complete. 2024-03-23T17:47:00.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:00.327+0000 7f772e350700 1 Processor -- start 2024-03-23T17:47:00.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:00.327+0000 7f772e350700 1 -- start start 2024-03-23T17:47:00.330 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:00.328+0000 7f772e350700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7728100de0 0x7f7728196e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:47:00.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:00.328+0000 7f772e350700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7728101fe0 0x7f7728197390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:47:00.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:00.328+0000 7f772e350700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7728102980 0x7f772819e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:47:00.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:00.328+0000 7f772e350700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f772819e8e0 con 0x7f7728102980 2024-03-23T17:47:00.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:00.328+0000 7f772e350700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f772819ea50 con 0x7f7728101fe0 2024-03-23T17:47:00.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:00.328+0000 7f772e350700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f772819ec10 con 0x7f7728100de0 2024-03-23T17:47:00.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:00.328+0000 7f7727fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7728100de0 0x7f7728196e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:47:00.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:00.328+0000 7f7727fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7728100de0 0x7f7728196e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52624/0 (socket says 172.21.15.152:52624) 2024-03-23T17:47:00.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:00.328+0000 7f7727fff700 1 -- 172.21.15.152:0/2372208471 learned_addr learned my addr 172.21.15.152:0/2372208471 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-03-23T17:47:00.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:00.328+0000 7f77277fe700 1 --2- 172.21.15.152:0/2372208471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7728101fe0 0x7f7728197390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:47:00.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:00.328+0000 7f7727fff700 1 -- 172.21.15.152:0/2372208471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7728101fe0 msgr2=0x7f7728197390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:47:00.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:00.328+0000 7f7727fff700 1 --2- 172.21.15.152:0/2372208471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7728101fe0 0x7f7728197390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:47:00.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:00.328+0000 7f7727fff700 1 -- 172.21.15.152:0/2372208471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7728102980 msgr2=0x7f772819e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:47:00.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:00.328+0000 7f772c8ed700 1 --2- 172.21.15.152:0/2372208471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7728102980 0x7f772819e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:47:00.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:00.328+0000 7f7727fff700 1 --2- 172.21.15.152:0/2372208471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7728102980 0x7f772819e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:47:00.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:00.328+0000 7f7727fff700 1 -- 172.21.15.152:0/2372208471 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77180095f0 con 0x7f7728100de0 2024-03-23T17:47:00.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:00.328+0000 7f772c8ed700 1 --2- 172.21.15.152:0/2372208471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7728102980 0x7f772819e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T17:47:00.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:00.328+0000 7f7727fff700 1 --2- 172.21.15.152:0/2372208471 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7728100de0 0x7f7728196e30 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7718000c00 tx=0x7f771800f670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:47:00.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:00.328+0000 7f77257fa700 1 -- 172.21.15.152:0/2372208471 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f771801d070 con 0x7f7728100de0 2024-03-23T17:47:00.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:00.329+0000 7f772e350700 1 -- 172.21.15.152:0/2372208471 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f772819ee30 con 0x7f7728100de0 2024-03-23T17:47:00.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:00.329+0000 7f77257fa700 1 -- 172.21.15.152:0/2372208471 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f771800fc50 con 0x7f7728100de0 2024-03-23T17:47:00.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:00.329+0000 7f77257fa700 1 -- 172.21.15.152:0/2372208471 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7718017650 con 0x7f7728100de0 2024-03-23T17:47:00.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:00.329+0000 7f772e350700 1 -- 172.21.15.152:0/2372208471 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f772819f240 con 0x7f7728100de0 2024-03-23T17:47:00.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:00.330+0000 7f772e350700 1 -- 172.21.15.152:0/2372208471 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f772804eaf0 con 0x7f7728100de0 2024-03-23T17:47:00.333 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:00.331+0000 7f77257fa700 1 -- 172.21.15.152:0/2372208471 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f7718022470 con 0x7f7728100de0 2024-03-23T17:47:00.333 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:00.332+0000 7f77257fa700 1 --2- 172.21.15.152:0/2372208471 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f77100a1120 0x7f77100a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:47:00.333 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:00.332+0000 7f77257fa700 1 -- 172.21.15.152:0/2372208471 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5101+0+0 (secure 0 0 0) 0x7f77180cd280 con 0x7f7728100de0 2024-03-23T17:47:00.333 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:00.332+0000 7f77277fe700 1 --2- 172.21.15.152:0/2372208471 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f77100a1120 0x7f77100a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:47:00.333 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:00.332+0000 7f77277fe700 1 --2- 172.21.15.152:0/2372208471 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f77100a1120 0x7f77100a35e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7728197de0 tx=0x7f771c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:47:00.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:00.335+0000 7f77257fa700 1 -- 172.21.15.152:0/2372208471 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f7718097730 con 0x7f7728100de0 2024-03-23T17:47:00.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:00.536+0000 7f772e350700 1 -- 172.21.15.152:0/2372208471 --> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f772804ea80 con 0x7f77100a1120 2024-03-23T17:47:00.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:00 smithi152 ceph-mon[47949]: Detected new or changed devices on smithi092 2024-03-23T17:47:00.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:00 smithi152 ceph-mon[47949]: Adjusting osd_memory_target on smithi092 to 5704M 2024-03-23T17:47:00.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:00 smithi152 ceph-mon[47949]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-23T17:47:00.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:00 smithi152 ceph-mon[47949]: osdmap e47: 8 total, 7 up, 8 in 2024-03-23T17:47:00.587 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:00 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-23T17:47:00.587 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:00 smithi152 ceph-mon[47949]: from='osd.7 [v2:172.21.15.92:6816/4187707622,v1:172.21.15.92:6817/4187707622]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-03-23T17:47:00.587 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:00 smithi152 ceph-mon[47949]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-03-23T17:47:00.766 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:00 smithi086 ceph-mon[38221]: Detected new or changed devices on smithi092 2024-03-23T17:47:00.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:00 smithi086 ceph-mon[38221]: Adjusting osd_memory_target on smithi092 to 5704M 2024-03-23T17:47:00.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:00 smithi086 ceph-mon[38221]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-23T17:47:00.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:00 smithi086 ceph-mon[38221]: osdmap e47: 8 total, 7 up, 8 in 2024-03-23T17:47:00.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:00 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-23T17:47:00.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:00 smithi086 ceph-mon[38221]: from='osd.7 [v2:172.21.15.92:6816/4187707622,v1:172.21.15.92:6817/4187707622]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-03-23T17:47:00.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:00 smithi086 ceph-mon[38221]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-03-23T17:47:00.807 INFO:journalctl@ceph.osd.7.smithi092.stdout:Mar 23 17:47:00 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-7[100796]: 2024-03-23T17:47:00.358+0000 7fb541c78700 -1 osd.7 0 waiting for initial osdmap 2024-03-23T17:47:00.807 INFO:journalctl@ceph.osd.7.smithi092.stdout:Mar 23 17:47:00 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-7[100796]: 2024-03-23T17:47:00.370+0000 7fb53e26e700 -1 osd.7 48 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T17:47:00.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:00 smithi092 ceph-mon[79089]: Detected new or changed devices on smithi092 2024-03-23T17:47:00.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:00 smithi092 ceph-mon[79089]: Adjusting osd_memory_target on smithi092 to 5704M 2024-03-23T17:47:00.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:00 smithi092 ceph-mon[79089]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-23T17:47:00.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:00 smithi092 ceph-mon[79089]: osdmap e47: 8 total, 7 up, 8 in 2024-03-23T17:47:00.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:00 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-23T17:47:00.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:00 smithi092 ceph-mon[79089]: from='osd.7 [v2:172.21.15.92:6816/4187707622,v1:172.21.15.92:6817/4187707622]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-03-23T17:47:00.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:00 smithi092 ceph-mon[79089]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-03-23T17:47:01.557 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:01 smithi152 ceph-mon[47949]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-03-23T17:47:01.557 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:01 smithi152 ceph-mon[47949]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-03-23T17:47:01.557 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:01 smithi152 ceph-mon[47949]: osdmap e48: 8 total, 7 up, 8 in 2024-03-23T17:47:01.557 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:01 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-23T17:47:01.557 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:01 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-23T17:47:01.557 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:01 smithi152 ceph-mon[47949]: from='client.24325 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:47:01.557 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:01 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T17:47:01.557 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:01 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T17:47:01.557 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:01 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:47:01.557 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:01 smithi152 ceph-mon[47949]: osd.7 [v2:172.21.15.92:6816/4187707622,v1:172.21.15.92:6817/4187707622] boot 2024-03-23T17:47:01.557 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:01 smithi152 ceph-mon[47949]: osdmap e49: 8 total, 8 up, 8 in 2024-03-23T17:47:01.557 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:01 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-23T17:47:01.766 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:01 smithi086 ceph-mon[38221]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-03-23T17:47:01.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:01 smithi086 ceph-mon[38221]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-03-23T17:47:01.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:01 smithi086 ceph-mon[38221]: osdmap e48: 8 total, 7 up, 8 in 2024-03-23T17:47:01.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:01 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-23T17:47:01.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:01 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-23T17:47:01.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:01 smithi086 ceph-mon[38221]: from='client.24325 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:47:01.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:01 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T17:47:01.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:01 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T17:47:01.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:01 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:47:01.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:01 smithi086 ceph-mon[38221]: osd.7 [v2:172.21.15.92:6816/4187707622,v1:172.21.15.92:6817/4187707622] boot 2024-03-23T17:47:01.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:01 smithi086 ceph-mon[38221]: osdmap e49: 8 total, 8 up, 8 in 2024-03-23T17:47:01.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:01 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-23T17:47:01.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:01 smithi092 ceph-mon[79089]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-03-23T17:47:01.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:01 smithi092 ceph-mon[79089]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-03-23T17:47:01.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:01 smithi092 ceph-mon[79089]: osdmap e48: 8 total, 7 up, 8 in 2024-03-23T17:47:01.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:01 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-23T17:47:01.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:01 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-23T17:47:01.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:01 smithi092 ceph-mon[79089]: from='client.24325 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:47:01.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:01 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T17:47:01.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:01 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T17:47:01.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:01 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:47:01.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:01 smithi092 ceph-mon[79089]: osd.7 [v2:172.21.15.92:6816/4187707622,v1:172.21.15.92:6817/4187707622] boot 2024-03-23T17:47:01.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:01 smithi092 ceph-mon[79089]: osdmap e49: 8 total, 8 up, 8 in 2024-03-23T17:47:01.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:01 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-23T17:47:03.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:02 smithi086 ceph-mon[38221]: purged_snaps scrub starts 2024-03-23T17:47:03.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:02 smithi086 ceph-mon[38221]: purged_snaps scrub ok 2024-03-23T17:47:03.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:02 smithi086 ceph-mon[38221]: osdmap e50: 8 total, 8 up, 8 in 2024-03-23T17:47:03.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:02 smithi086 ceph-mon[38221]: from='client.? 172.21.15.152:0/1355274130' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f03bbf0d-1281-4bcf-9773-9854388e65ed"}]: dispatch 2024-03-23T17:47:03.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:02 smithi086 ceph-mon[38221]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f03bbf0d-1281-4bcf-9773-9854388e65ed"}]: dispatch 2024-03-23T17:47:03.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:02 smithi086 ceph-mon[38221]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f03bbf0d-1281-4bcf-9773-9854388e65ed"}]': finished 2024-03-23T17:47:03.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:02 smithi086 ceph-mon[38221]: osdmap e51: 9 total, 8 up, 9 in 2024-03-23T17:47:03.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:02 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-23T17:47:03.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:02 smithi092 ceph-mon[79089]: purged_snaps scrub starts 2024-03-23T17:47:03.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:02 smithi092 ceph-mon[79089]: purged_snaps scrub ok 2024-03-23T17:47:03.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:02 smithi092 ceph-mon[79089]: osdmap e50: 8 total, 8 up, 8 in 2024-03-23T17:47:03.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:02 smithi092 ceph-mon[79089]: from='client.? 172.21.15.152:0/1355274130' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f03bbf0d-1281-4bcf-9773-9854388e65ed"}]: dispatch 2024-03-23T17:47:03.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:02 smithi092 ceph-mon[79089]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f03bbf0d-1281-4bcf-9773-9854388e65ed"}]: dispatch 2024-03-23T17:47:03.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:02 smithi092 ceph-mon[79089]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f03bbf0d-1281-4bcf-9773-9854388e65ed"}]': finished 2024-03-23T17:47:03.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:02 smithi092 ceph-mon[79089]: osdmap e51: 9 total, 8 up, 9 in 2024-03-23T17:47:03.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:02 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-23T17:47:03.085 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:02 smithi152 ceph-mon[47949]: purged_snaps scrub starts 2024-03-23T17:47:03.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:02 smithi152 ceph-mon[47949]: purged_snaps scrub ok 2024-03-23T17:47:03.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:02 smithi152 ceph-mon[47949]: osdmap e50: 8 total, 8 up, 8 in 2024-03-23T17:47:03.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:02 smithi152 ceph-mon[47949]: from='client.? 172.21.15.152:0/1355274130' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f03bbf0d-1281-4bcf-9773-9854388e65ed"}]: dispatch 2024-03-23T17:47:03.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:02 smithi152 ceph-mon[47949]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f03bbf0d-1281-4bcf-9773-9854388e65ed"}]: dispatch 2024-03-23T17:47:03.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:02 smithi152 ceph-mon[47949]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f03bbf0d-1281-4bcf-9773-9854388e65ed"}]': finished 2024-03-23T17:47:03.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:02 smithi152 ceph-mon[47949]: osdmap e51: 9 total, 8 up, 9 in 2024-03-23T17:47:03.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:02 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-23T17:47:04.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:03 smithi086 ceph-mon[38221]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-23T17:47:04.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:03 smithi086 ceph-mon[38221]: from='client.? 172.21.15.152:0/2984816621' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T17:47:04.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:03 smithi086 ceph-mon[38221]: osdmap e52: 9 total, 8 up, 9 in 2024-03-23T17:47:04.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:03 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-23T17:47:04.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:03 smithi092 ceph-mon[79089]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-23T17:47:04.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:03 smithi092 ceph-mon[79089]: from='client.? 172.21.15.152:0/2984816621' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T17:47:04.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:03 smithi092 ceph-mon[79089]: osdmap e52: 9 total, 8 up, 9 in 2024-03-23T17:47:04.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:03 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-23T17:47:04.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:03 smithi152 ceph-mon[47949]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-23T17:47:04.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:03 smithi152 ceph-mon[47949]: from='client.? 172.21.15.152:0/2984816621' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T17:47:04.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:03 smithi152 ceph-mon[47949]: osdmap e52: 9 total, 8 up, 9 in 2024-03-23T17:47:04.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:03 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-23T17:47:05.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:05 smithi086 ceph-mon[38221]: pgmap v150: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-23T17:47:05.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:05 smithi086 ceph-mon[38221]: osdmap e53: 9 total, 8 up, 9 in 2024-03-23T17:47:05.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:05 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-23T17:47:05.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:05 smithi092 ceph-mon[79089]: pgmap v150: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-23T17:47:05.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:05 smithi092 ceph-mon[79089]: osdmap e53: 9 total, 8 up, 9 in 2024-03-23T17:47:05.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:05 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-23T17:47:05.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:05 smithi152 ceph-mon[47949]: pgmap v150: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-23T17:47:05.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:05 smithi152 ceph-mon[47949]: osdmap e53: 9 total, 8 up, 9 in 2024-03-23T17:47:05.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:05 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-23T17:47:07.815 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:07 smithi152 ceph-mon[47949]: pgmap v152: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-03-23T17:47:08.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:07 smithi086 ceph-mon[38221]: pgmap v152: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-03-23T17:47:08.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:07 smithi092 ceph-mon[79089]: pgmap v152: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-03-23T17:47:10.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:09 smithi086 ceph-mon[38221]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-03-23T17:47:10.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:09 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-23T17:47:10.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:09 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:47:10.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:09 smithi086 ceph-mon[38221]: Deploying daemon osd.8 on smithi152 2024-03-23T17:47:10.019 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:09 smithi152 ceph-mon[47949]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-03-23T17:47:10.019 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:09 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-23T17:47:10.019 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:09 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:47:10.019 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:09 smithi152 ceph-mon[47949]: Deploying daemon osd.8 on smithi152 2024-03-23T17:47:10.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:09 smithi092 ceph-mon[79089]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-03-23T17:47:10.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:09 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-23T17:47:10.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:09 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:47:10.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:09 smithi092 ceph-mon[79089]: Deploying daemon osd.8 on smithi152 2024-03-23T17:47:11.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:11 smithi152 ceph-mon[47949]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-03-23T17:47:12.018 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:11 smithi086 ceph-mon[38221]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-03-23T17:47:12.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:11 smithi092 ceph-mon[79089]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-03-23T17:47:13.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:13 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:13.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:13 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:13.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:13 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:47:13.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:13 smithi086 ceph-mon[38221]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-03-23T17:47:13.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:13 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:13.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:13 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:13.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:13 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:47:13.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:13 smithi092 ceph-mon[79089]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-03-23T17:47:13.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:13 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:13.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:13 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:13.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:13 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:47:13.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:13 smithi152 ceph-mon[47949]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-03-23T17:47:14.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:14.644+0000 7f77257fa700 1 -- 172.21.15.152:0/2372208471 <== mgr.14152 v2:172.21.15.86:6800/2734784244 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f772804ea80 con 0x7f77100a1120 2024-03-23T17:47:14.646 INFO:teuthology.orchestra.run.smithi152.stdout:Created osd(s) 8 on host 'smithi152' 2024-03-23T17:47:14.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:14.647+0000 7f772e350700 1 -- 172.21.15.152:0/2372208471 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f77100a1120 msgr2=0x7f77100a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:47:14.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:14.647+0000 7f772e350700 1 --2- 172.21.15.152:0/2372208471 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f77100a1120 0x7f77100a35e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7728197de0 tx=0x7f771c009190 comp rx=0 tx=0).stop 2024-03-23T17:47:14.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:14.647+0000 7f772e350700 1 -- 172.21.15.152:0/2372208471 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7728100de0 msgr2=0x7f7728196e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:47:14.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:14.647+0000 7f772e350700 1 --2- 172.21.15.152:0/2372208471 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7728100de0 0x7f7728196e30 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7718000c00 tx=0x7f771800f670 comp rx=0 tx=0).stop 2024-03-23T17:47:14.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:14.647+0000 7f772e350700 1 -- 172.21.15.152:0/2372208471 shutdown_connections 2024-03-23T17:47:14.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:14.647+0000 7f772e350700 1 --2- 172.21.15.152:0/2372208471 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f77100a1120 0x7f77100a35e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:47:14.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:14.647+0000 7f772e350700 1 --2- 172.21.15.152:0/2372208471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7728102980 0x7f772819e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:47:14.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:14.647+0000 7f772e350700 1 --2- 172.21.15.152:0/2372208471 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7728100de0 0x7f7728196e30 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:47:14.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:14.647+0000 7f772e350700 1 --2- 172.21.15.152:0/2372208471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7728101fe0 0x7f7728197390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:47:14.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:14.647+0000 7f772e350700 1 -- 172.21.15.152:0/2372208471 >> 172.21.15.152:0/2372208471 conn(0x7f77280fc380 msgr2=0x7f7728105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:47:14.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:14.647+0000 7f772e350700 1 -- 172.21.15.152:0/2372208471 shutdown_connections 2024-03-23T17:47:14.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:14.647+0000 7f772e350700 1 -- 172.21.15.152:0/2372208471 wait complete. 2024-03-23T17:47:14.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:14 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:14.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:14 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:14.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:14 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:47:14.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:14 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:47:14.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:14 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:15.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:14 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:15.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:14 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:15.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:14 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:47:15.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:14 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:47:15.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:14 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:15.054 DEBUG:teuthology.orchestra.run.smithi152:osd.8> sudo journalctl -f -n 0 -u ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@osd.8.service 2024-03-23T17:47:15.056 INFO:tasks.cephadm:Deploying osd.9 on smithi086 with /dev/vg_nvme/lv_1... 2024-03-23T17:47:15.057 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-23T17:47:15.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:14 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:15.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:14 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:15.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:14 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:47:15.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:14 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:47:15.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:14 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:15.090 INFO:journalctl@ceph.osd.8.smithi152.stdout:-- Logs begin at Sat 2024-03-23 17:19:34 UTC. -- 2024-03-23T17:47:15.435 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T17:47:15.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:15 smithi152 ceph-mon[47949]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-03-23T17:47:15.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:15 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:15.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:15 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:15.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:15 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:47:15.963 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:15 smithi086 ceph-mon[38221]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-03-23T17:47:15.963 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:15 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:15.963 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:15 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:15.963 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:15 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:47:16.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:15 smithi092 ceph-mon[79089]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-03-23T17:47:16.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:15 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:16.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:15 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:16.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:15 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:47:16.238 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 17:47:16 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T17:47:16.235+0000 7f723a726640 -1 osd.8 0 log_to_monitors true 2024-03-23T17:47:16.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:16 smithi086 ceph-mon[38221]: from='osd.8 [v2:172.21.15.152:6816/3237612748,v1:172.21.15.152:6817/3237612748]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-23T17:47:16.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:16 smithi086 ceph-mon[38221]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-23T17:47:16.988 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:16 smithi152 ceph-mon[47949]: from='osd.8 [v2:172.21.15.152:6816/3237612748,v1:172.21.15.152:6817/3237612748]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-23T17:47:16.988 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:16 smithi152 ceph-mon[47949]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-23T17:47:17.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:16 smithi092 ceph-mon[79089]: from='osd.8 [v2:172.21.15.152:6816/3237612748,v1:172.21.15.152:6817/3237612748]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-23T17:47:17.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:16 smithi092 ceph-mon[79089]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-23T17:47:17.208 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-23T17:47:17.208 INFO:teuthology.orchestra.run.smithi086.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-23T17:47:17.208 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 10+0 records in 2024-03-23T17:47:17.208 INFO:teuthology.orchestra.run.smithi086.stderr:10+0 records out 2024-03-23T17:47:17.209 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0471524 s, 222 MB/s 2024-03-23T17:47:17.209 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping successful for: 2024-03-23T17:47:17.648 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi086:vg_nvme/lv_1 2024-03-23T17:47:18.003 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T17:47:18.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:17 smithi086 ceph-mon[38221]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-03-23T17:47:18.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:17 smithi086 ceph-mon[38221]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-23T17:47:18.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:17 smithi086 ceph-mon[38221]: osdmap e54: 9 total, 8 up, 9 in 2024-03-23T17:47:18.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:17 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-23T17:47:18.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:17 smithi086 ceph-mon[38221]: from='osd.8 [v2:172.21.15.152:6816/3237612748,v1:172.21.15.152:6817/3237612748]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-03-23T17:47:18.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:17 smithi086 ceph-mon[38221]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-03-23T17:47:18.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:17 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:18.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:17 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:18.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:17 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:47:18.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:17 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:47:18.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:17 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:47:18.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:17 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:18.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:17 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:47:18.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:17 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:47:18.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:17 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:18.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:17 smithi092 ceph-mon[79089]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-03-23T17:47:18.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:17 smithi092 ceph-mon[79089]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-23T17:47:18.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:17 smithi092 ceph-mon[79089]: osdmap e54: 9 total, 8 up, 9 in 2024-03-23T17:47:18.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:17 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-23T17:47:18.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:17 smithi092 ceph-mon[79089]: from='osd.8 [v2:172.21.15.152:6816/3237612748,v1:172.21.15.152:6817/3237612748]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-03-23T17:47:18.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:17 smithi092 ceph-mon[79089]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-03-23T17:47:18.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:17 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:18.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:17 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:18.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:17 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:47:18.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:17 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:47:18.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:17 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:47:18.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:17 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:18.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:17 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:47:18.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:17 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:47:18.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:17 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:18.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:17 smithi152 ceph-mon[47949]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-03-23T17:47:18.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:17 smithi152 ceph-mon[47949]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-23T17:47:18.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:17 smithi152 ceph-mon[47949]: osdmap e54: 9 total, 8 up, 9 in 2024-03-23T17:47:18.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:17 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-23T17:47:18.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:17 smithi152 ceph-mon[47949]: from='osd.8 [v2:172.21.15.152:6816/3237612748,v1:172.21.15.152:6817/3237612748]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-03-23T17:47:18.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:17 smithi152 ceph-mon[47949]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-03-23T17:47:18.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:17 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:18.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:17 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:18.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:17 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:47:18.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:17 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:47:18.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:17 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:47:18.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:17 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:18.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:17 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:47:18.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:17 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:47:18.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:17 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:18.087 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 17:47:17 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T17:47:17.626+0000 7f722f589700 -1 osd.8 0 waiting for initial osdmap 2024-03-23T17:47:18.087 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 17:47:17 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T17:47:17.639+0000 7f722bb7f700 -1 osd.8 55 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T17:47:18.956 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:18 smithi086 ceph-mon[38221]: Detected new or changed devices on smithi152 2024-03-23T17:47:18.957 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:18 smithi086 ceph-mon[38221]: Adjusting osd_memory_target on smithi152 to 5704M 2024-03-23T17:47:18.957 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:18 smithi086 ceph-mon[38221]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]': finished 2024-03-23T17:47:18.957 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:18 smithi086 ceph-mon[38221]: osdmap e55: 9 total, 8 up, 9 in 2024-03-23T17:47:18.957 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:18 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-23T17:47:18.957 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:18 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-23T17:47:18.957 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:18 smithi086 ceph-mon[38221]: osd.8 [v2:172.21.15.152:6816/3237612748,v1:172.21.15.152:6817/3237612748] boot 2024-03-23T17:47:18.957 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:18 smithi086 ceph-mon[38221]: osdmap e56: 9 total, 9 up, 9 in 2024-03-23T17:47:18.957 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:18 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-23T17:47:19.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:18 smithi092 ceph-mon[79089]: Detected new or changed devices on smithi152 2024-03-23T17:47:19.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:18 smithi092 ceph-mon[79089]: Adjusting osd_memory_target on smithi152 to 5704M 2024-03-23T17:47:19.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:18 smithi092 ceph-mon[79089]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]': finished 2024-03-23T17:47:19.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:18 smithi092 ceph-mon[79089]: osdmap e55: 9 total, 8 up, 9 in 2024-03-23T17:47:19.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:18 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-23T17:47:19.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:18 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-23T17:47:19.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:18 smithi092 ceph-mon[79089]: osd.8 [v2:172.21.15.152:6816/3237612748,v1:172.21.15.152:6817/3237612748] boot 2024-03-23T17:47:19.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:18 smithi092 ceph-mon[79089]: osdmap e56: 9 total, 9 up, 9 in 2024-03-23T17:47:19.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:18 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-23T17:47:19.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:18 smithi152 ceph-mon[47949]: Detected new or changed devices on smithi152 2024-03-23T17:47:19.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:18 smithi152 ceph-mon[47949]: Adjusting osd_memory_target on smithi152 to 5704M 2024-03-23T17:47:19.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:18 smithi152 ceph-mon[47949]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]': finished 2024-03-23T17:47:19.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:18 smithi152 ceph-mon[47949]: osdmap e55: 9 total, 8 up, 9 in 2024-03-23T17:47:19.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:18 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-23T17:47:19.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:18 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-23T17:47:19.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:18 smithi152 ceph-mon[47949]: osd.8 [v2:172.21.15.152:6816/3237612748,v1:172.21.15.152:6817/3237612748] boot 2024-03-23T17:47:19.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:18 smithi152 ceph-mon[47949]: osdmap e56: 9 total, 9 up, 9 in 2024-03-23T17:47:19.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:18 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-23T17:47:19.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:47:19.196+0000 7f1ae6c00700 1 -- 172.21.15.86:0/4107739217 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1ae0101f60 msgr2=0x7f1ae01023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:47:19.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:47:19.196+0000 7f1ae6c00700 1 --2- 172.21.15.86:0/4107739217 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1ae0101f60 0x7f1ae01023e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1acc009990 tx=0x7f1acc009ca0 comp rx=0 tx=0).stop 2024-03-23T17:47:19.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:47:19.196+0000 7f1ae6c00700 1 -- 172.21.15.86:0/4107739217 shutdown_connections 2024-03-23T17:47:19.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:47:19.196+0000 7f1ae6c00700 1 --2- 172.21.15.86:0/4107739217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ae0102920 0x7f1ae0109240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:47:19.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:47:19.196+0000 7f1ae6c00700 1 --2- 172.21.15.86:0/4107739217 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ae0100e00 0x7f1ae0101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:47:19.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:47:19.196+0000 7f1ae6c00700 1 --2- 172.21.15.86:0/4107739217 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1ae0101f60 0x7f1ae01023e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:47:19.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:47:19.196+0000 7f1ae6c00700 1 -- 172.21.15.86:0/4107739217 >> 172.21.15.86:0/4107739217 conn(0x7f1ae00fc3c0 msgr2=0x7f1ae00fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:47:19.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:47:19.196+0000 7f1ae6c00700 1 -- 172.21.15.86:0/4107739217 shutdown_connections 2024-03-23T17:47:19.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:47:19.196+0000 7f1ae6c00700 1 -- 172.21.15.86:0/4107739217 wait complete. 2024-03-23T17:47:19.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:47:19.197+0000 7f1ae6c00700 1 Processor -- start 2024-03-23T17:47:19.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:47:19.198+0000 7f1ae6c00700 1 -- start start 2024-03-23T17:47:19.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:47:19.198+0000 7f1ae6c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ae0100e00 0x7f1ae0074b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:47:19.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:47:19.198+0000 7f1ae6c00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1ae0101f60 0x7f1ae00731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:47:19.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:47:19.198+0000 7f1ae6c00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ae0102920 0x7f1ae00736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:47:19.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:47:19.198+0000 7f1ae6c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae0073e80 con 0x7f1ae0100e00 2024-03-23T17:47:19.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:47:19.198+0000 7f1ae6c00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae0073ff0 con 0x7f1ae0102920 2024-03-23T17:47:19.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:47:19.198+0000 7f1ae6c00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae01a0d40 con 0x7f1ae0101f60 2024-03-23T17:47:19.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:47:19.198+0000 7f1adfe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ae0100e00 0x7f1ae0074b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:47:19.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:47:19.198+0000 7f1adfe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ae0100e00 0x7f1ae0074b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44532/0 (socket says 172.21.15.86:44532) 2024-03-23T17:47:19.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:47:19.198+0000 7f1adfe00700 1 -- 172.21.15.86:0/2668892619 learned_addr learned my addr 172.21.15.86:0/2668892619 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:47:19.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:47:19.198+0000 7f1adf400700 1 --2- 172.21.15.86:0/2668892619 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1ae0101f60 0x7f1ae00731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:47:19.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:47:19.198+0000 7f1ae4c00700 1 --2- 172.21.15.86:0/2668892619 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ae0102920 0x7f1ae00736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:47:19.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:47:19.199+0000 7f1adfe00700 1 -- 172.21.15.86:0/2668892619 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1ae0101f60 msgr2=0x7f1ae00731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:47:19.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:47:19.199+0000 7f1adfe00700 1 --2- 172.21.15.86:0/2668892619 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1ae0101f60 0x7f1ae00731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:47:19.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:47:19.199+0000 7f1adfe00700 1 -- 172.21.15.86:0/2668892619 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ae0102920 msgr2=0x7f1ae00736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:47:19.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:47:19.199+0000 7f1adfe00700 1 --2- 172.21.15.86:0/2668892619 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ae0102920 0x7f1ae00736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:47:19.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:47:19.199+0000 7f1adfe00700 1 -- 172.21.15.86:0/2668892619 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1acc0095f0 con 0x7f1ae0100e00 2024-03-23T17:47:19.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:47:19.199+0000 7f1adfe00700 1 --2- 172.21.15.86:0/2668892619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ae0100e00 0x7f1ae0074b00 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1ad000e9a0 tx=0x7f1ad000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:47:19.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:47:19.199+0000 7f1adcc00700 1 -- 172.21.15.86:0/2668892619 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad0009790 con 0x7f1ae0100e00 2024-03-23T17:47:19.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:47:19.199+0000 7f1adcc00700 1 -- 172.21.15.86:0/2668892619 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ad0010430 con 0x7f1ae0100e00 2024-03-23T17:47:19.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:47:19.199+0000 7f1ae6c00700 1 -- 172.21.15.86:0/2668892619 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ae01a0fd0 con 0x7f1ae0100e00 2024-03-23T17:47:19.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:47:19.199+0000 7f1adcc00700 1 -- 172.21.15.86:0/2668892619 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad0018680 con 0x7f1ae0100e00 2024-03-23T17:47:19.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:47:19.199+0000 7f1ae6c00700 1 -- 172.21.15.86:0/2668892619 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ae01a13e0 con 0x7f1ae0100e00 2024-03-23T17:47:19.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:47:19.201+0000 7f1ae6c00700 1 -- 172.21.15.86:0/2668892619 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ae0108280 con 0x7f1ae0100e00 2024-03-23T17:47:19.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:47:19.202+0000 7f1adcc00700 1 -- 172.21.15.86:0/2668892619 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f1ad00105a0 con 0x7f1ae0100e00 2024-03-23T17:47:19.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:47:19.204+0000 7f1adcc00700 1 --2- 172.21.15.86:0/2668892619 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f1ac40a1120 0x7f1ac40a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:47:19.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:47:19.204+0000 7f1adcc00700 1 -- 172.21.15.86:0/2668892619 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5862+0+0 (secure 0 0 0) 0x7f1ad0014070 con 0x7f1ae0100e00 2024-03-23T17:47:19.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:47:19.204+0000 7f1adf400700 1 --2- 172.21.15.86:0/2668892619 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f1ac40a1120 0x7f1ac40a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:47:19.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:47:19.205+0000 7f1adf400700 1 --2- 172.21.15.86:0/2668892619 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f1ac40a1120 0x7f1ac40a35e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1acc00b2e0 tx=0x7f1acc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:47:19.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:47:19.208+0000 7f1adcc00700 1 -- 172.21.15.86:0/2668892619 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f1ad00529f0 con 0x7f1ae0100e00 2024-03-23T17:47:19.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:47:19.422+0000 7f1ae6c00700 1 -- 172.21.15.86:0/2668892619 --> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f1ae002cdc0 con 0x7f1ac40a1120 2024-03-23T17:47:20.010 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:19 smithi086 ceph-mon[38221]: purged_snaps scrub starts 2024-03-23T17:47:20.010 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:19 smithi086 ceph-mon[38221]: purged_snaps scrub ok 2024-03-23T17:47:20.010 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:19 smithi086 ceph-mon[38221]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-03-23T17:47:20.010 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:19 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T17:47:20.010 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:19 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T17:47:20.010 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:19 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:47:20.010 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:19 smithi086 ceph-mon[38221]: osdmap e57: 9 total, 9 up, 9 in 2024-03-23T17:47:20.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:19 smithi092 ceph-mon[79089]: purged_snaps scrub starts 2024-03-23T17:47:20.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:19 smithi092 ceph-mon[79089]: purged_snaps scrub ok 2024-03-23T17:47:20.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:19 smithi092 ceph-mon[79089]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-03-23T17:47:20.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:19 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T17:47:20.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:19 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T17:47:20.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:19 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:47:20.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:19 smithi092 ceph-mon[79089]: osdmap e57: 9 total, 9 up, 9 in 2024-03-23T17:47:20.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:19 smithi152 ceph-mon[47949]: purged_snaps scrub starts 2024-03-23T17:47:20.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:19 smithi152 ceph-mon[47949]: purged_snaps scrub ok 2024-03-23T17:47:20.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:19 smithi152 ceph-mon[47949]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-03-23T17:47:20.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:19 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T17:47:20.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:19 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T17:47:20.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:19 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:47:20.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:19 smithi152 ceph-mon[47949]: osdmap e57: 9 total, 9 up, 9 in 2024-03-23T17:47:21.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:20 smithi086 ceph-mon[38221]: from='client.14349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:47:21.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:20 smithi086 ceph-mon[38221]: osdmap e58: 9 total, 9 up, 9 in 2024-03-23T17:47:21.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:20 smithi092 ceph-mon[79089]: from='client.14349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:47:21.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:20 smithi092 ceph-mon[79089]: osdmap e58: 9 total, 9 up, 9 in 2024-03-23T17:47:21.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:20 smithi152 ceph-mon[47949]: from='client.14349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:47:21.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:20 smithi152 ceph-mon[47949]: osdmap e58: 9 total, 9 up, 9 in 2024-03-23T17:47:22.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:21 smithi086 ceph-mon[38221]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-23T17:47:22.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:21 smithi092 ceph-mon[79089]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-23T17:47:22.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:21 smithi152 ceph-mon[47949]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-23T17:47:23.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:22 smithi086 ceph-mon[38221]: osdmap e59: 9 total, 9 up, 9 in 2024-03-23T17:47:23.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:22 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/957626253' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7e8b775-99ff-48a4-a23a-91e7ab6622bb"}]: dispatch 2024-03-23T17:47:23.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:22 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/957626253' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e7e8b775-99ff-48a4-a23a-91e7ab6622bb"}]': finished 2024-03-23T17:47:23.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:22 smithi086 ceph-mon[38221]: osdmap e60: 10 total, 9 up, 10 in 2024-03-23T17:47:23.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:22 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-23T17:47:23.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:22 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/2965058219' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T17:47:23.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:22 smithi092 ceph-mon[79089]: osdmap e59: 9 total, 9 up, 9 in 2024-03-23T17:47:23.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:22 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/957626253' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7e8b775-99ff-48a4-a23a-91e7ab6622bb"}]: dispatch 2024-03-23T17:47:23.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:22 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/957626253' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e7e8b775-99ff-48a4-a23a-91e7ab6622bb"}]': finished 2024-03-23T17:47:23.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:22 smithi092 ceph-mon[79089]: osdmap e60: 10 total, 9 up, 10 in 2024-03-23T17:47:23.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:22 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-23T17:47:23.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:22 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/2965058219' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T17:47:23.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:22 smithi152 ceph-mon[47949]: osdmap e59: 9 total, 9 up, 9 in 2024-03-23T17:47:23.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:22 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/957626253' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7e8b775-99ff-48a4-a23a-91e7ab6622bb"}]: dispatch 2024-03-23T17:47:23.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:22 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/957626253' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e7e8b775-99ff-48a4-a23a-91e7ab6622bb"}]': finished 2024-03-23T17:47:23.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:22 smithi152 ceph-mon[47949]: osdmap e60: 10 total, 9 up, 10 in 2024-03-23T17:47:23.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:22 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-23T17:47:23.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:22 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/2965058219' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T17:47:24.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:23 smithi086 ceph-mon[38221]: pgmap v167: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 157 KiB/s, 0 objects/s recovering 2024-03-23T17:47:24.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:23 smithi092 ceph-mon[79089]: pgmap v167: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 157 KiB/s, 0 objects/s recovering 2024-03-23T17:47:24.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:23 smithi152 ceph-mon[47949]: pgmap v167: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 157 KiB/s, 0 objects/s recovering 2024-03-23T17:47:26.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:25 smithi086 ceph-mon[38221]: pgmap v168: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 123 KiB/s, 0 objects/s recovering 2024-03-23T17:47:26.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:25 smithi092 ceph-mon[79089]: pgmap v168: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 123 KiB/s, 0 objects/s recovering 2024-03-23T17:47:26.085 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:25 smithi152 ceph-mon[47949]: pgmap v168: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 123 KiB/s, 0 objects/s recovering 2024-03-23T17:47:28.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:27 smithi086 ceph-mon[38221]: pgmap v169: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-23T17:47:28.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:27 smithi092 ceph-mon[79089]: pgmap v169: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-23T17:47:28.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:27 smithi152 ceph-mon[47949]: pgmap v169: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-23T17:47:28.517 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:47:28 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:47:28.263+0000 7f13a3e00700 -1 calc_pg_upmaps abort due to max <= 0 2024-03-23T17:47:30.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:29 smithi086 ceph-mon[38221]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-03-23T17:47:30.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:29 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-23T17:47:30.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:29 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:47:30.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:29 smithi086 ceph-mon[38221]: Deploying daemon osd.9 on smithi086 2024-03-23T17:47:30.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:29 smithi092 ceph-mon[79089]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-03-23T17:47:30.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:29 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-23T17:47:30.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:29 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:47:30.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:29 smithi092 ceph-mon[79089]: Deploying daemon osd.9 on smithi086 2024-03-23T17:47:30.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:29 smithi152 ceph-mon[47949]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-03-23T17:47:30.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:29 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-23T17:47:30.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:29 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:47:30.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:29 smithi152 ceph-mon[47949]: Deploying daemon osd.9 on smithi086 2024-03-23T17:47:32.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:31 smithi086 ceph-mon[38221]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-23T17:47:32.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:31 smithi092 ceph-mon[79089]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-23T17:47:32.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:31 smithi152 ceph-mon[47949]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-23T17:47:34.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:33 smithi086 ceph-mon[38221]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-23T17:47:34.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:33 smithi092 ceph-mon[79089]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-23T17:47:34.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:33 smithi152 ceph-mon[47949]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-23T17:47:34.990 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:34 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:34.990 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:34 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:34.990 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:34 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:47:34.990 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:34 smithi086 ceph-mon[38221]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-23T17:47:35.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:34 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:35.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:34 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:35.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:34 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:47:35.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:34 smithi092 ceph-mon[79089]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-23T17:47:35.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:34 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:35.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:34 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:35.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:34 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:47:35.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:34 smithi152 ceph-mon[47949]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-23T17:47:36.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:47:36.739+0000 7f1adcc00700 1 -- 172.21.15.86:0/2668892619 <== mgr.14152 v2:172.21.15.86:6800/2734784244 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f1ae002cdc0 con 0x7f1ac40a1120 2024-03-23T17:47:36.740 INFO:teuthology.orchestra.run.smithi086.stdout:Created osd(s) 9 on host 'smithi086' 2024-03-23T17:47:36.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:47:36.743+0000 7f1ae6c00700 1 -- 172.21.15.86:0/2668892619 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f1ac40a1120 msgr2=0x7f1ac40a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:47:36.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:47:36.743+0000 7f1ae6c00700 1 --2- 172.21.15.86:0/2668892619 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f1ac40a1120 0x7f1ac40a35e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1acc00b2e0 tx=0x7f1acc005fb0 comp rx=0 tx=0).stop 2024-03-23T17:47:36.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:47:36.743+0000 7f1ae6c00700 1 -- 172.21.15.86:0/2668892619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ae0100e00 msgr2=0x7f1ae0074b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:47:36.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:47:36.743+0000 7f1ae6c00700 1 --2- 172.21.15.86:0/2668892619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ae0100e00 0x7f1ae0074b00 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1ad000e9a0 tx=0x7f1ad000ecb0 comp rx=0 tx=0).stop 2024-03-23T17:47:36.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:47:36.743+0000 7f1ae6c00700 1 -- 172.21.15.86:0/2668892619 shutdown_connections 2024-03-23T17:47:36.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:47:36.743+0000 7f1ae6c00700 1 --2- 172.21.15.86:0/2668892619 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ae0102920 0x7f1ae00736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:47:36.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:47:36.743+0000 7f1ae6c00700 1 --2- 172.21.15.86:0/2668892619 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f1ac40a1120 0x7f1ac40a35e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:47:36.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:47:36.743+0000 7f1ae6c00700 1 --2- 172.21.15.86:0/2668892619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ae0100e00 0x7f1ae0074b00 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:47:36.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:47:36.743+0000 7f1ae6c00700 1 --2- 172.21.15.86:0/2668892619 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1ae0101f60 0x7f1ae00731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:47:36.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:47:36.743+0000 7f1ae6c00700 1 -- 172.21.15.86:0/2668892619 >> 172.21.15.86:0/2668892619 conn(0x7f1ae00fc3c0 msgr2=0x7f1ae01053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:47:36.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:47:36.744+0000 7f1ae6c00700 1 -- 172.21.15.86:0/2668892619 shutdown_connections 2024-03-23T17:47:36.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:47:36.744+0000 7f1ae6c00700 1 -- 172.21.15.86:0/2668892619 wait complete. 2024-03-23T17:47:36.960 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:36 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:36.960 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:36 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:36.960 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:36 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:47:36.960 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:36 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:47:36.960 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:36 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:36.960 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:36 smithi086 ceph-mon[38221]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-23T17:47:36.960 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:36 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:36.961 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:36 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:36.961 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:36 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:47:37.169 DEBUG:teuthology.orchestra.run.smithi086:osd.9> sudo journalctl -f -n 0 -u ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@osd.9.service 2024-03-23T17:47:37.174 INFO:tasks.cephadm:Deploying osd.10 on smithi092 with /dev/vg_nvme/lv_1... 2024-03-23T17:47:37.174 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-23T17:47:37.208 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:36 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:37.208 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:36 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:37.208 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:36 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:47:37.208 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:36 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:47:37.208 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:36 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:37.208 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:36 smithi092 ceph-mon[79089]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-23T17:47:37.208 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:36 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:37.208 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:36 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:37.208 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:36 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:47:37.247 INFO:journalctl@ceph.osd.9.smithi086.stdout:-- Logs begin at Sat 2024-03-23 17:25:35 UTC. -- 2024-03-23T17:47:37.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:36 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:37.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:36 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:37.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:36 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:47:37.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:36 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:47:37.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:36 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:37.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:36 smithi152 ceph-mon[47949]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-23T17:47:37.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:36 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:37.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:36 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:37.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:36 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:47:37.405 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.b/config 2024-03-23T17:47:38.211 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:38 smithi092 ceph-mon[79089]: from='osd.9 [v2:172.21.15.86:6826/2835218294,v1:172.21.15.86:6827/2835218294]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-23T17:47:38.223 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 17:47:37 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T17:47:37.938+0000 7fa32a58a640 -1 osd.9 0 log_to_monitors true 2024-03-23T17:47:38.223 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:38 smithi086 ceph-mon[38221]: from='osd.9 [v2:172.21.15.86:6826/2835218294,v1:172.21.15.86:6827/2835218294]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-23T17:47:38.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:38 smithi152 ceph-mon[47949]: from='osd.9 [v2:172.21.15.86:6826/2835218294,v1:172.21.15.86:6827/2835218294]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-23T17:47:38.742 INFO:teuthology.orchestra.run.smithi092.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-23T17:47:38.742 INFO:teuthology.orchestra.run.smithi092.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-23T17:47:38.742 INFO:teuthology.orchestra.run.smithi092.stderr: stderr: 10+0 records in 2024-03-23T17:47:38.742 INFO:teuthology.orchestra.run.smithi092.stderr:10+0 records out 2024-03-23T17:47:38.742 INFO:teuthology.orchestra.run.smithi092.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0131168 s, 799 MB/s 2024-03-23T17:47:38.743 INFO:teuthology.orchestra.run.smithi092.stderr:--> Zapping successful for: 2024-03-23T17:47:39.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:39 smithi092 ceph-mon[79089]: from='osd.9 [v2:172.21.15.86:6826/2835218294,v1:172.21.15.86:6827/2835218294]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-23T17:47:39.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:39 smithi092 ceph-mon[79089]: osdmap e61: 10 total, 9 up, 10 in 2024-03-23T17:47:39.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:39 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-23T17:47:39.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:39 smithi092 ceph-mon[79089]: from='osd.9 [v2:172.21.15.86:6826/2835218294,v1:172.21.15.86:6827/2835218294]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-03-23T17:47:39.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:39 smithi092 ceph-mon[79089]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-23T17:47:39.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:39 smithi092 ceph-mon[79089]: from='osd.9 [v2:172.21.15.86:6826/2835218294,v1:172.21.15.86:6827/2835218294]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-03-23T17:47:39.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:39 smithi092 ceph-mon[79089]: osdmap e62: 10 total, 9 up, 10 in 2024-03-23T17:47:39.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:39 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-23T17:47:39.091 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi092:vg_nvme/lv_1 2024-03-23T17:47:39.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:39 smithi086 ceph-mon[38221]: from='osd.9 [v2:172.21.15.86:6826/2835218294,v1:172.21.15.86:6827/2835218294]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-23T17:47:39.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:39 smithi086 ceph-mon[38221]: osdmap e61: 10 total, 9 up, 10 in 2024-03-23T17:47:39.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:39 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-23T17:47:39.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:39 smithi086 ceph-mon[38221]: from='osd.9 [v2:172.21.15.86:6826/2835218294,v1:172.21.15.86:6827/2835218294]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-03-23T17:47:39.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:39 smithi086 ceph-mon[38221]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-23T17:47:39.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:39 smithi086 ceph-mon[38221]: from='osd.9 [v2:172.21.15.86:6826/2835218294,v1:172.21.15.86:6827/2835218294]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-03-23T17:47:39.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:39 smithi086 ceph-mon[38221]: osdmap e62: 10 total, 9 up, 10 in 2024-03-23T17:47:39.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:39 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-23T17:47:39.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:39 smithi152 ceph-mon[47949]: from='osd.9 [v2:172.21.15.86:6826/2835218294,v1:172.21.15.86:6827/2835218294]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-23T17:47:39.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:39 smithi152 ceph-mon[47949]: osdmap e61: 10 total, 9 up, 10 in 2024-03-23T17:47:39.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:39 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-23T17:47:39.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:39 smithi152 ceph-mon[47949]: from='osd.9 [v2:172.21.15.86:6826/2835218294,v1:172.21.15.86:6827/2835218294]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-03-23T17:47:39.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:39 smithi152 ceph-mon[47949]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-23T17:47:39.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:39 smithi152 ceph-mon[47949]: from='osd.9 [v2:172.21.15.86:6826/2835218294,v1:172.21.15.86:6827/2835218294]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-03-23T17:47:39.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:39 smithi152 ceph-mon[47949]: osdmap e62: 10 total, 9 up, 10 in 2024-03-23T17:47:39.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:39 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-23T17:47:39.349 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.b/config 2024-03-23T17:47:40.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:40 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-23T17:47:40.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:40 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-23T17:47:40.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:40 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-23T17:47:40.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:40 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-23T17:47:40.267 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 17:47:40 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T17:47:40.024+0000 7fa31e200700 -1 osd.9 0 waiting for initial osdmap 2024-03-23T17:47:40.267 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 17:47:40 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T17:47:40.052+0000 7fa314a00700 -1 osd.9 62 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T17:47:40.288 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:47:40.288+0000 7f398d170700 1 -- 172.21.15.92:0/2837517444 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39880ffd50 msgr2=0x7f3988100170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:47:40.288 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:47:40.288+0000 7f398d170700 1 --2- 172.21.15.92:0/2837517444 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39880ffd50 0x7f3988100170 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f39700099c0 tx=0x7f3970009cd0 comp rx=0 tx=0).stop 2024-03-23T17:47:40.289 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:47:40.288+0000 7f398d170700 1 -- 172.21.15.92:0/2837517444 shutdown_connections 2024-03-23T17:47:40.289 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:47:40.288+0000 7f398d170700 1 --2- 172.21.15.92:0/2837517444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39881018f0 0x7f39881081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:47:40.289 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:47:40.288+0000 7f398d170700 1 --2- 172.21.15.92:0/2837517444 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39880ffd50 0x7f3988100170 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:47:40.289 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:47:40.288+0000 7f398d170700 1 --2- 172.21.15.92:0/2837517444 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3988100f50 0x7f39881013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:47:40.289 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:47:40.288+0000 7f398d170700 1 -- 172.21.15.92:0/2837517444 >> 172.21.15.92:0/2837517444 conn(0x7f39880fb310 msgr2=0x7f39880fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:47:40.289 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:47:40.288+0000 7f398d170700 1 -- 172.21.15.92:0/2837517444 shutdown_connections 2024-03-23T17:47:40.289 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:47:40.288+0000 7f398d170700 1 -- 172.21.15.92:0/2837517444 wait complete. 2024-03-23T17:47:40.291 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:47:40.289+0000 7f398d170700 1 Processor -- start 2024-03-23T17:47:40.291 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:47:40.289+0000 7f398d170700 1 -- start start 2024-03-23T17:47:40.291 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:47:40.289+0000 7f398d170700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f39880ffd50 0x7f3988195c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:47:40.291 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:47:40.290+0000 7f398d170700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3988100f50 0x7f39881961c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:47:40.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:47:40.290+0000 7f398d170700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39881018f0 0x7f398819d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:47:40.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:47:40.290+0000 7f398d170700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f398819d770 con 0x7f39881018f0 2024-03-23T17:47:40.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:47:40.290+0000 7f398d170700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f398819d8b0 con 0x7f3988100f50 2024-03-23T17:47:40.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:47:40.290+0000 7f398d170700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f398819da40 con 0x7f39880ffd50 2024-03-23T17:47:40.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:47:40.290+0000 7f398659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3988100f50 0x7f39881961c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:47:40.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:47:40.290+0000 7f398659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3988100f50 0x7f39881961c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35934/0 (socket says 172.21.15.92:35934) 2024-03-23T17:47:40.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:47:40.290+0000 7f398659c700 1 -- 172.21.15.92:0/2520216643 learned_addr learned my addr 172.21.15.92:0/2520216643 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-23T17:47:40.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:47:40.290+0000 7f3986d9d700 1 --2- 172.21.15.92:0/2520216643 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f39880ffd50 0x7f3988195c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:47:40.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:47:40.290+0000 7f398759e700 1 --2- 172.21.15.92:0/2520216643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39881018f0 0x7f398819d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:47:40.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:47:40.290+0000 7f398659c700 1 -- 172.21.15.92:0/2520216643 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f39880ffd50 msgr2=0x7f3988195c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:47:40.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:47:40.290+0000 7f398659c700 1 --2- 172.21.15.92:0/2520216643 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f39880ffd50 0x7f3988195c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:47:40.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:47:40.290+0000 7f398659c700 1 -- 172.21.15.92:0/2520216643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39881018f0 msgr2=0x7f398819d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:47:40.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:47:40.290+0000 7f398659c700 1 --2- 172.21.15.92:0/2520216643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39881018f0 0x7f398819d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:47:40.293 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:47:40.290+0000 7f398659c700 1 -- 172.21.15.92:0/2520216643 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39700095f0 con 0x7f3988100f50 2024-03-23T17:47:40.293 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:47:40.290+0000 7f398659c700 1 --2- 172.21.15.92:0/2520216643 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3988100f50 0x7f39881961c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f397800c920 tx=0x7f397800cce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:47:40.293 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:47:40.290+0000 7f397ffff700 1 -- 172.21.15.92:0/2520216643 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3978007b00 con 0x7f3988100f50 2024-03-23T17:47:40.293 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:47:40.290+0000 7f398d170700 1 -- 172.21.15.92:0/2520216643 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f398819dd20 con 0x7f3988100f50 2024-03-23T17:47:40.293 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:47:40.290+0000 7f397ffff700 1 -- 172.21.15.92:0/2520216643 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3978004cf0 con 0x7f3988100f50 2024-03-23T17:47:40.293 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:47:40.290+0000 7f397ffff700 1 -- 172.21.15.92:0/2520216643 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3978005520 con 0x7f3988100f50 2024-03-23T17:47:40.293 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:47:40.290+0000 7f398d170700 1 -- 172.21.15.92:0/2520216643 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f398819e210 con 0x7f3988100f50 2024-03-23T17:47:40.293 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:47:40.291+0000 7f398d170700 1 -- 172.21.15.92:0/2520216643 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39881072d0 con 0x7f3988100f50 2024-03-23T17:47:40.294 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:47:40.292+0000 7f397ffff700 1 -- 172.21.15.92:0/2520216643 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f3978004750 con 0x7f3988100f50 2024-03-23T17:47:40.294 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:47:40.293+0000 7f397ffff700 1 --2- 172.21.15.92:0/2520216643 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f39740a1120 0x7f39740a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:47:40.294 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:47:40.293+0000 7f397ffff700 1 -- 172.21.15.92:0/2520216643 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5981+0+0 (secure 0 0 0) 0x7f39780cbc60 con 0x7f3988100f50 2024-03-23T17:47:40.294 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:47:40.293+0000 7f3986d9d700 1 --2- 172.21.15.92:0/2520216643 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f39740a1120 0x7f39740a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:47:40.295 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:47:40.294+0000 7f3986d9d700 1 --2- 172.21.15.92:0/2520216643 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f39740a1120 0x7f39740a35e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f397000b580 tx=0x7f3970005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:47:40.298 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:47:40.298+0000 7f397ffff700 1 -- 172.21.15.92:0/2520216643 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f3978095d10 con 0x7f3988100f50 2024-03-23T17:47:40.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:40 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-23T17:47:40.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:40 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-23T17:47:40.501 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:47:40.501+0000 7f398d170700 1 -- 172.21.15.92:0/2520216643 --> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f398802cd70 con 0x7f39740a1120 2024-03-23T17:47:41.227 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:41 smithi092 ceph-mon[79089]: purged_snaps scrub starts 2024-03-23T17:47:41.227 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:41 smithi092 ceph-mon[79089]: purged_snaps scrub ok 2024-03-23T17:47:41.227 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:41 smithi092 ceph-mon[79089]: from='osd.9 [v2:172.21.15.86:6826/2835218294,v1:172.21.15.86:6827/2835218294]' entity='osd.9' 2024-03-23T17:47:41.227 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:41 smithi092 ceph-mon[79089]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-23T17:47:41.228 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:41 smithi092 ceph-mon[79089]: Detected new or changed devices on smithi086 2024-03-23T17:47:41.228 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:41 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:41.228 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:41 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:41.228 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:41 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:47:41.228 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:41 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:47:41.228 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:41 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:47:41.228 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:41 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:47:41.228 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:41 smithi092 ceph-mon[79089]: Adjusting osd_memory_target on smithi086 to 4317M 2024-03-23T17:47:41.228 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:41 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:41.228 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:41 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:47:41.228 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:41 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:47:41.229 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:41 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:41.229 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:41 smithi092 ceph-mon[79089]: from='client.24374 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:47:41.229 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:41 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T17:47:41.229 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:41 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T17:47:41.229 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:41 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:47:41.229 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:41 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-23T17:47:41.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:41 smithi152 ceph-mon[47949]: purged_snaps scrub starts 2024-03-23T17:47:41.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:41 smithi152 ceph-mon[47949]: purged_snaps scrub ok 2024-03-23T17:47:41.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:41 smithi152 ceph-mon[47949]: from='osd.9 [v2:172.21.15.86:6826/2835218294,v1:172.21.15.86:6827/2835218294]' entity='osd.9' 2024-03-23T17:47:41.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:41 smithi152 ceph-mon[47949]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-23T17:47:41.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:41 smithi152 ceph-mon[47949]: Detected new or changed devices on smithi086 2024-03-23T17:47:41.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:41 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:41.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:41 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:41.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:41 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:47:41.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:41 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:47:41.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:41 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:47:41.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:41 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:47:41.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:41 smithi152 ceph-mon[47949]: Adjusting osd_memory_target on smithi086 to 4317M 2024-03-23T17:47:41.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:41 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:41.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:41 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:47:41.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:41 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:47:41.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:41 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:41.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:41 smithi152 ceph-mon[47949]: from='client.24374 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:47:41.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:41 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T17:47:41.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:41 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T17:47:41.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:41 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:47:41.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:41 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-23T17:47:41.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:41 smithi086 ceph-mon[38221]: purged_snaps scrub starts 2024-03-23T17:47:41.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:41 smithi086 ceph-mon[38221]: purged_snaps scrub ok 2024-03-23T17:47:41.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:41 smithi086 ceph-mon[38221]: from='osd.9 [v2:172.21.15.86:6826/2835218294,v1:172.21.15.86:6827/2835218294]' entity='osd.9' 2024-03-23T17:47:41.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:41 smithi086 ceph-mon[38221]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-23T17:47:41.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:41 smithi086 ceph-mon[38221]: Detected new or changed devices on smithi086 2024-03-23T17:47:41.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:41 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:41.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:41 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:41.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:41 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:47:41.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:41 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:47:41.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:41 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:47:41.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:41 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:47:41.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:41 smithi086 ceph-mon[38221]: Adjusting osd_memory_target on smithi086 to 4317M 2024-03-23T17:47:41.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:41 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:41.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:41 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:47:41.518 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:41 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:47:41.518 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:41 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:41.518 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:41 smithi086 ceph-mon[38221]: from='client.24374 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:47:41.518 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:41 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T17:47:41.518 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:41 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T17:47:41.518 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:41 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:47:41.518 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:41 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-23T17:47:42.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:42 smithi092 ceph-mon[79089]: osd.9 [v2:172.21.15.86:6826/2835218294,v1:172.21.15.86:6827/2835218294] boot 2024-03-23T17:47:42.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:42 smithi092 ceph-mon[79089]: osdmap e63: 10 total, 10 up, 10 in 2024-03-23T17:47:42.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:42 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-23T17:47:42.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:42 smithi152 ceph-mon[47949]: osd.9 [v2:172.21.15.86:6826/2835218294,v1:172.21.15.86:6827/2835218294] boot 2024-03-23T17:47:42.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:42 smithi152 ceph-mon[47949]: osdmap e63: 10 total, 10 up, 10 in 2024-03-23T17:47:42.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:42 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-23T17:47:42.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:42 smithi086 ceph-mon[38221]: osd.9 [v2:172.21.15.86:6826/2835218294,v1:172.21.15.86:6827/2835218294] boot 2024-03-23T17:47:42.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:42 smithi086 ceph-mon[38221]: osdmap e63: 10 total, 10 up, 10 in 2024-03-23T17:47:42.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:42 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-23T17:47:43.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:43 smithi092 ceph-mon[79089]: osdmap e64: 10 total, 10 up, 10 in 2024-03-23T17:47:43.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:43 smithi092 ceph-mon[79089]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-03-23T17:47:43.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:43 smithi092 ceph-mon[79089]: from='client.? 172.21.15.92:0/456548097' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d9d616cd-a229-44de-8768-02a4415df115"}]: dispatch 2024-03-23T17:47:43.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:43 smithi092 ceph-mon[79089]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d9d616cd-a229-44de-8768-02a4415df115"}]: dispatch 2024-03-23T17:47:43.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:43 smithi092 ceph-mon[79089]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d9d616cd-a229-44de-8768-02a4415df115"}]': finished 2024-03-23T17:47:43.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:43 smithi092 ceph-mon[79089]: osdmap e65: 11 total, 10 up, 11 in 2024-03-23T17:47:43.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:43 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-23T17:47:43.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:43 smithi092 ceph-mon[79089]: from='client.? 172.21.15.92:0/4123560187' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T17:47:43.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:43 smithi152 ceph-mon[47949]: osdmap e64: 10 total, 10 up, 10 in 2024-03-23T17:47:43.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:43 smithi152 ceph-mon[47949]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-03-23T17:47:43.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:43 smithi152 ceph-mon[47949]: from='client.? 172.21.15.92:0/456548097' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d9d616cd-a229-44de-8768-02a4415df115"}]: dispatch 2024-03-23T17:47:43.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:43 smithi152 ceph-mon[47949]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d9d616cd-a229-44de-8768-02a4415df115"}]: dispatch 2024-03-23T17:47:43.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:43 smithi152 ceph-mon[47949]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d9d616cd-a229-44de-8768-02a4415df115"}]': finished 2024-03-23T17:47:43.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:43 smithi152 ceph-mon[47949]: osdmap e65: 11 total, 10 up, 11 in 2024-03-23T17:47:43.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:43 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-23T17:47:43.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:43 smithi152 ceph-mon[47949]: from='client.? 172.21.15.92:0/4123560187' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T17:47:43.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:43 smithi086 ceph-mon[38221]: osdmap e64: 10 total, 10 up, 10 in 2024-03-23T17:47:43.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:43 smithi086 ceph-mon[38221]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-03-23T17:47:43.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:43 smithi086 ceph-mon[38221]: from='client.? 172.21.15.92:0/456548097' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d9d616cd-a229-44de-8768-02a4415df115"}]: dispatch 2024-03-23T17:47:43.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:43 smithi086 ceph-mon[38221]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d9d616cd-a229-44de-8768-02a4415df115"}]: dispatch 2024-03-23T17:47:43.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:43 smithi086 ceph-mon[38221]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d9d616cd-a229-44de-8768-02a4415df115"}]': finished 2024-03-23T17:47:43.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:43 smithi086 ceph-mon[38221]: osdmap e65: 11 total, 10 up, 11 in 2024-03-23T17:47:43.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:43 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-23T17:47:43.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:43 smithi086 ceph-mon[38221]: from='client.? 172.21.15.92:0/4123560187' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T17:47:45.766 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:45 smithi086 ceph-mon[38221]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-03-23T17:47:45.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:45 smithi092 ceph-mon[79089]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-03-23T17:47:45.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:45 smithi152 ceph-mon[47949]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-03-23T17:47:48.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:47 smithi086 ceph-mon[38221]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-03-23T17:47:48.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:47 smithi092 ceph-mon[79089]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-03-23T17:47:48.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:47 smithi152 ceph-mon[47949]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-03-23T17:47:48.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:48 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-23T17:47:48.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:48 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:47:49.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:48 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-23T17:47:49.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:48 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:47:49.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:48 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-23T17:47:49.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:48 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:47:49.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:49 smithi092 ceph-mon[79089]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-03-23T17:47:49.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:49 smithi092 ceph-mon[79089]: Deploying daemon osd.10 on smithi092 2024-03-23T17:47:50.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:49 smithi086 ceph-mon[38221]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-03-23T17:47:50.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:49 smithi086 ceph-mon[38221]: Deploying daemon osd.10 on smithi092 2024-03-23T17:47:50.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:49 smithi152 ceph-mon[47949]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-03-23T17:47:50.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:49 smithi152 ceph-mon[47949]: Deploying daemon osd.10 on smithi092 2024-03-23T17:47:51.708 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:51 smithi092 ceph-mon[79089]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-03-23T17:47:52.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:51 smithi086 ceph-mon[38221]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-03-23T17:47:52.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:51 smithi152 ceph-mon[47949]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-03-23T17:47:52.901 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:52 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:47:52.901 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:52 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:52.901 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:52 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:53.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:52 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:47:53.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:52 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:53.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:52 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:53.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:52 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:47:53.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:52 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:53.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:52 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:54.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:53 smithi086 ceph-mon[38221]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-03-23T17:47:54.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:53 smithi092 ceph-mon[79089]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-03-23T17:47:54.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:53 smithi152 ceph-mon[47949]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-03-23T17:47:54.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:47:54.657+0000 7f397ffff700 1 -- 172.21.15.92:0/2520216643 <== mgr.14152 v2:172.21.15.86:6800/2734784244 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f398802cd70 con 0x7f39740a1120 2024-03-23T17:47:54.658 INFO:teuthology.orchestra.run.smithi092.stdout:Created osd(s) 10 on host 'smithi092' 2024-03-23T17:47:54.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:47:54.660+0000 7f398d170700 1 -- 172.21.15.92:0/2520216643 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f39740a1120 msgr2=0x7f39740a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:47:54.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:47:54.660+0000 7f398d170700 1 --2- 172.21.15.92:0/2520216643 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f39740a1120 0x7f39740a35e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f397000b580 tx=0x7f3970005fb0 comp rx=0 tx=0).stop 2024-03-23T17:47:54.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:47:54.660+0000 7f398d170700 1 -- 172.21.15.92:0/2520216643 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3988100f50 msgr2=0x7f39881961c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:47:54.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:47:54.660+0000 7f398d170700 1 --2- 172.21.15.92:0/2520216643 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3988100f50 0x7f39881961c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f397800c920 tx=0x7f397800cce0 comp rx=0 tx=0).stop 2024-03-23T17:47:54.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:47:54.660+0000 7f398d170700 1 -- 172.21.15.92:0/2520216643 shutdown_connections 2024-03-23T17:47:54.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:47:54.660+0000 7f398d170700 1 --2- 172.21.15.92:0/2520216643 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f39740a1120 0x7f39740a35e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:47:54.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:47:54.660+0000 7f398d170700 1 --2- 172.21.15.92:0/2520216643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39881018f0 0x7f398819d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:47:54.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:47:54.660+0000 7f398d170700 1 --2- 172.21.15.92:0/2520216643 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f39880ffd50 0x7f3988195c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:47:54.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:47:54.660+0000 7f398d170700 1 --2- 172.21.15.92:0/2520216643 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3988100f50 0x7f39881961c0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:47:54.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:47:54.660+0000 7f398d170700 1 -- 172.21.15.92:0/2520216643 >> 172.21.15.92:0/2520216643 conn(0x7f39880fb310 msgr2=0x7f3988104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:47:54.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:47:54.660+0000 7f398d170700 1 -- 172.21.15.92:0/2520216643 shutdown_connections 2024-03-23T17:47:54.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-23T17:47:54.660+0000 7f398d170700 1 -- 172.21.15.92:0/2520216643 wait complete. 2024-03-23T17:47:55.082 DEBUG:teuthology.orchestra.run.smithi092:osd.10> sudo journalctl -f -n 0 -u ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@osd.10.service 2024-03-23T17:47:55.084 INFO:tasks.cephadm:Deploying osd.11 on smithi152 with /dev/vg_nvme/lv_1... 2024-03-23T17:47:55.084 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-23T17:47:55.114 INFO:journalctl@ceph.osd.10.smithi092.stdout:-- Logs begin at Sat 2024-03-23 17:19:16 UTC. -- 2024-03-23T17:47:55.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:55 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:55.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:55 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:55.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:55 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:47:55.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:55 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:47:55.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:55 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:55.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:55 smithi092 ceph-mon[79089]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-03-23T17:47:55.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:55 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:55.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:55 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:55.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:55 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:47:55.321 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.c/config 2024-03-23T17:47:55.422 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:55 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:55.423 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:55 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:55.423 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:55 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:47:55.423 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:55 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:47:55.423 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:55 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:55.423 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:55 smithi152 ceph-mon[47949]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-03-23T17:47:55.423 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:55 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:55.423 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:55 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:55.423 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:55 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:47:55.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:55 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:55.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:55 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:55.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:55 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:47:55.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:55 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:47:55.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:55 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:55.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:55 smithi086 ceph-mon[38221]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-03-23T17:47:55.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:55 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:55.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:55 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:55.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:55 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:47:56.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 17:47:56 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T17:47:56.279+0000 7f7abbc32640 -1 osd.10 0 log_to_monitors true 2024-03-23T17:47:56.652 INFO:teuthology.orchestra.run.smithi152.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-23T17:47:56.652 INFO:teuthology.orchestra.run.smithi152.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-23T17:47:56.652 INFO:teuthology.orchestra.run.smithi152.stderr: stderr: 10+0 records in 2024-03-23T17:47:56.653 INFO:teuthology.orchestra.run.smithi152.stderr:10+0 records out 2024-03-23T17:47:56.653 INFO:teuthology.orchestra.run.smithi152.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111806 s, 938 MB/s 2024-03-23T17:47:56.653 INFO:teuthology.orchestra.run.smithi152.stderr:--> Zapping successful for: 2024-03-23T17:47:56.881 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:56 smithi152 ceph-mon[47949]: from='osd.10 [v2:172.21.15.92:6824/4259465100,v1:172.21.15.92:6825/4259465100]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-23T17:47:56.881 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:56 smithi152 ceph-mon[47949]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-23T17:47:57.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:56 smithi086 ceph-mon[38221]: from='osd.10 [v2:172.21.15.92:6824/4259465100,v1:172.21.15.92:6825/4259465100]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-23T17:47:57.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:56 smithi086 ceph-mon[38221]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-23T17:47:57.020 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi152:vg_nvme/lv_1 2024-03-23T17:47:57.024 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:56 smithi092 ceph-mon[79089]: from='osd.10 [v2:172.21.15.92:6824/4259465100,v1:172.21.15.92:6825/4259465100]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-23T17:47:57.024 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:56 smithi092 ceph-mon[79089]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-23T17:47:57.239 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.c/config 2024-03-23T17:47:58.016 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:57 smithi152 ceph-mon[47949]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-03-23T17:47:58.016 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:57 smithi152 ceph-mon[47949]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-23T17:47:58.016 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:57 smithi152 ceph-mon[47949]: osdmap e66: 11 total, 10 up, 11 in 2024-03-23T17:47:58.016 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:57 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-23T17:47:58.016 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:57 smithi152 ceph-mon[47949]: from='osd.10 [v2:172.21.15.92:6824/4259465100,v1:172.21.15.92:6825/4259465100]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-03-23T17:47:58.016 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:57 smithi152 ceph-mon[47949]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-03-23T17:47:58.016 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:57 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:58.016 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:57 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:58.017 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:57 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:47:58.017 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:57 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:47:58.017 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:57 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:47:58.017 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:57 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:47:58.017 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:57 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:58.017 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:57 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:47:58.017 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:57 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:47:58.017 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:57 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:58.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:57 smithi086 ceph-mon[38221]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-03-23T17:47:58.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:57 smithi086 ceph-mon[38221]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-23T17:47:58.018 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:57 smithi086 ceph-mon[38221]: osdmap e66: 11 total, 10 up, 11 in 2024-03-23T17:47:58.018 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:57 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-23T17:47:58.018 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:57 smithi086 ceph-mon[38221]: from='osd.10 [v2:172.21.15.92:6824/4259465100,v1:172.21.15.92:6825/4259465100]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-03-23T17:47:58.018 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:57 smithi086 ceph-mon[38221]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-03-23T17:47:58.018 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:57 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:58.018 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:57 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:58.018 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:57 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:47:58.018 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:57 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:47:58.018 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:57 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:47:58.018 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:57 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:47:58.018 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:57 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:58.018 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:57 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:47:58.018 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:57 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:47:58.018 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:57 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:58.057 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 17:47:57 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T17:47:57.620+0000 7f7ab0a95700 -1 osd.10 0 waiting for initial osdmap 2024-03-23T17:47:58.057 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 17:47:57 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T17:47:57.634+0000 7f7aad08b700 -1 osd.10 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T17:47:58.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:57 smithi092 ceph-mon[79089]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-03-23T17:47:58.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:57 smithi092 ceph-mon[79089]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-23T17:47:58.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:57 smithi092 ceph-mon[79089]: osdmap e66: 11 total, 10 up, 11 in 2024-03-23T17:47:58.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:57 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-23T17:47:58.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:57 smithi092 ceph-mon[79089]: from='osd.10 [v2:172.21.15.92:6824/4259465100,v1:172.21.15.92:6825/4259465100]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-03-23T17:47:58.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:57 smithi092 ceph-mon[79089]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-03-23T17:47:58.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:57 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:58.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:57 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:58.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:57 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:47:58.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:57 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:47:58.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:57 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:47:58.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:57 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:47:58.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:57 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:58.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:57 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:47:58.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:57 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:47:58.059 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:57 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:47:58.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:58.195+0000 7f98fdcc7700 1 -- 172.21.15.152:0/494997152 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f98f8101910 msgr2=0x7f98f8103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:47:58.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:58.195+0000 7f98fdcc7700 1 --2- 172.21.15.152:0/494997152 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f98f8101910 0x7f98f8103d00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f98ec009990 tx=0x7f98ec009ca0 comp rx=0 tx=0).stop 2024-03-23T17:47:58.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:58.195+0000 7f98fdcc7700 1 -- 172.21.15.152:0/494997152 shutdown_connections 2024-03-23T17:47:58.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:58.195+0000 7f98fdcc7700 1 --2- 172.21.15.152:0/494997152 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98f8106b70 0x7f98f8108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:47:58.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:58.195+0000 7f98fdcc7700 1 --2- 172.21.15.152:0/494997152 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f98f8101910 0x7f98f8103d00 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:47:58.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:58.195+0000 7f98fdcc7700 1 --2- 172.21.15.152:0/494997152 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98f8104240 0x7f98f8106630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:47:58.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:58.195+0000 7f98fdcc7700 1 -- 172.21.15.152:0/494997152 >> 172.21.15.152:0/494997152 conn(0x7f98f80fb2c0 msgr2=0x7f98f80fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:47:58.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:58.195+0000 7f98fdcc7700 1 -- 172.21.15.152:0/494997152 shutdown_connections 2024-03-23T17:47:58.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:58.195+0000 7f98fdcc7700 1 -- 172.21.15.152:0/494997152 wait complete. 2024-03-23T17:47:58.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:58.195+0000 7f98fdcc7700 1 Processor -- start 2024-03-23T17:47:58.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:58.196+0000 7f98fdcc7700 1 -- start start 2024-03-23T17:47:58.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:58.196+0000 7f98fdcc7700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f98f8101910 0x7f98f8196d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:47:58.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:58.196+0000 7f98fdcc7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98f8104240 0x7f98f8197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:47:58.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:58.196+0000 7f98fdcc7700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98f8106b70 0x7f98f819e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:47:58.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:58.196+0000 7f98fdcc7700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f819e830 con 0x7f98f8106b70 2024-03-23T17:47:58.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:58.196+0000 7f98fdcc7700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f819e970 con 0x7f98f8104240 2024-03-23T17:47:58.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:58.196+0000 7f98fdcc7700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f819eb30 con 0x7f98f8101910 2024-03-23T17:47:58.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:58.196+0000 7f98f77fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f98f8101910 0x7f98f8196d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:47:58.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:58.196+0000 7f98f77fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f98f8101910 0x7f98f8196d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:42134/0 (socket says 172.21.15.152:42134) 2024-03-23T17:47:58.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:58.196+0000 7f98f77fe700 1 -- 172.21.15.152:0/24423745 learned_addr learned my addr 172.21.15.152:0/24423745 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-03-23T17:47:58.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:58.196+0000 7f98f6ffd700 1 --2- 172.21.15.152:0/24423745 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98f8104240 0x7f98f8197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:47:58.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:58.196+0000 7f98f7fff700 1 --2- 172.21.15.152:0/24423745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98f8106b70 0x7f98f819e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:47:58.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:58.196+0000 7f98f77fe700 1 -- 172.21.15.152:0/24423745 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98f8104240 msgr2=0x7f98f8197240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:47:58.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:58.196+0000 7f98f77fe700 1 --2- 172.21.15.152:0/24423745 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98f8104240 0x7f98f8197240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:47:58.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:58.196+0000 7f98f77fe700 1 -- 172.21.15.152:0/24423745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98f8106b70 msgr2=0x7f98f819e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:47:58.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:58.196+0000 7f98f77fe700 1 --2- 172.21.15.152:0/24423745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98f8106b70 0x7f98f819e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:47:58.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:58.196+0000 7f98f77fe700 1 -- 172.21.15.152:0/24423745 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98ec0095f0 con 0x7f98f8101910 2024-03-23T17:47:58.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:58.196+0000 7f98f7fff700 1 --2- 172.21.15.152:0/24423745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98f8106b70 0x7f98f819e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T17:47:58.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:58.196+0000 7f98f77fe700 1 --2- 172.21.15.152:0/24423745 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f98f8101910 0x7f98f8196d00 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f98ec0095c0 tx=0x7f98ec00f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:47:58.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:58.196+0000 7f98f4ff9700 1 -- 172.21.15.152:0/24423745 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ec01d070 con 0x7f98f8101910 2024-03-23T17:47:58.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:58.196+0000 7f98f4ff9700 1 -- 172.21.15.152:0/24423745 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f98ec005510 con 0x7f98f8101910 2024-03-23T17:47:58.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:58.196+0000 7f98fdcc7700 1 -- 172.21.15.152:0/24423745 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98f819edb0 con 0x7f98f8101910 2024-03-23T17:47:58.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:58.196+0000 7f98f4ff9700 1 -- 172.21.15.152:0/24423745 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ec004c20 con 0x7f98f8101910 2024-03-23T17:47:58.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:58.197+0000 7f98fdcc7700 1 -- 172.21.15.152:0/24423745 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98f819f1c0 con 0x7f98f8101910 2024-03-23T17:47:58.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:58.198+0000 7f98fdcc7700 1 -- 172.21.15.152:0/24423745 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98f80fce40 con 0x7f98f8101910 2024-03-23T17:47:58.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:58.198+0000 7f98f4ff9700 1 -- 172.21.15.152:0/24423745 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f98ec00fb90 con 0x7f98f8101910 2024-03-23T17:47:58.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:58.199+0000 7f98f4ff9700 1 --2- 172.21.15.152:0/24423745 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f98e00a1120 0x7f98e00a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:47:58.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:58.199+0000 7f98f4ff9700 1 -- 172.21.15.152:0/24423745 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f98ec0cd150 con 0x7f98f8101910 2024-03-23T17:47:58.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:58.199+0000 7f98f6ffd700 1 --2- 172.21.15.152:0/24423745 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f98e00a1120 0x7f98e00a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:47:58.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:58.200+0000 7f98f6ffd700 1 --2- 172.21.15.152:0/24423745 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f98e00a1120 0x7f98e00a35e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f98f81981c0 tx=0x7f98e80073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:47:58.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:58.203+0000 7f98f4ff9700 1 -- 172.21.15.152:0/24423745 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f98ec0970e0 con 0x7f98f8101910 2024-03-23T17:47:58.404 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:47:58.405+0000 7f98fdcc7700 1 -- 172.21.15.152:0/24423745 --> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f98f802cdc0 con 0x7f98e00a1120 2024-03-23T17:47:58.950 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:58 smithi152 ceph-mon[47949]: Detected new or changed devices on smithi092 2024-03-23T17:47:58.950 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:58 smithi152 ceph-mon[47949]: Adjusting osd_memory_target on smithi092 to 4278M 2024-03-23T17:47:58.950 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:58 smithi152 ceph-mon[47949]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-03-23T17:47:58.950 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:58 smithi152 ceph-mon[47949]: osdmap e67: 11 total, 10 up, 11 in 2024-03-23T17:47:58.950 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:58 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-23T17:47:58.950 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:58 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-23T17:47:58.950 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:58 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T17:47:58.951 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:58 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T17:47:58.951 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:58 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:47:59.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:58 smithi086 ceph-mon[38221]: Detected new or changed devices on smithi092 2024-03-23T17:47:59.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:58 smithi086 ceph-mon[38221]: Adjusting osd_memory_target on smithi092 to 4278M 2024-03-23T17:47:59.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:58 smithi086 ceph-mon[38221]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-03-23T17:47:59.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:58 smithi086 ceph-mon[38221]: osdmap e67: 11 total, 10 up, 11 in 2024-03-23T17:47:59.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:58 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-23T17:47:59.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:58 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-23T17:47:59.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:58 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T17:47:59.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:58 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T17:47:59.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:58 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:47:59.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:58 smithi092 ceph-mon[79089]: Detected new or changed devices on smithi092 2024-03-23T17:47:59.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:58 smithi092 ceph-mon[79089]: Adjusting osd_memory_target on smithi092 to 4278M 2024-03-23T17:47:59.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:58 smithi092 ceph-mon[79089]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-03-23T17:47:59.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:58 smithi092 ceph-mon[79089]: osdmap e67: 11 total, 10 up, 11 in 2024-03-23T17:47:59.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:58 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-23T17:47:59.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:58 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-23T17:47:59.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:58 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T17:47:59.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:58 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T17:47:59.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:58 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:48:00.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:59 smithi086 ceph-mon[38221]: purged_snaps scrub starts 2024-03-23T17:48:00.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:59 smithi086 ceph-mon[38221]: purged_snaps scrub ok 2024-03-23T17:48:00.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:59 smithi086 ceph-mon[38221]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-03-23T17:48:00.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:59 smithi086 ceph-mon[38221]: from='client.24400 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:48:00.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:59 smithi086 ceph-mon[38221]: osd.10 [v2:172.21.15.92:6824/4259465100,v1:172.21.15.92:6825/4259465100] boot 2024-03-23T17:48:00.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:59 smithi086 ceph-mon[38221]: osdmap e68: 11 total, 11 up, 11 in 2024-03-23T17:48:00.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:47:59 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-23T17:48:00.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:59 smithi092 ceph-mon[79089]: purged_snaps scrub starts 2024-03-23T17:48:00.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:59 smithi092 ceph-mon[79089]: purged_snaps scrub ok 2024-03-23T17:48:00.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:59 smithi092 ceph-mon[79089]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-03-23T17:48:00.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:59 smithi092 ceph-mon[79089]: from='client.24400 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:48:00.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:59 smithi092 ceph-mon[79089]: osd.10 [v2:172.21.15.92:6824/4259465100,v1:172.21.15.92:6825/4259465100] boot 2024-03-23T17:48:00.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:59 smithi092 ceph-mon[79089]: osdmap e68: 11 total, 11 up, 11 in 2024-03-23T17:48:00.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:47:59 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-23T17:48:00.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:59 smithi152 ceph-mon[47949]: purged_snaps scrub starts 2024-03-23T17:48:00.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:59 smithi152 ceph-mon[47949]: purged_snaps scrub ok 2024-03-23T17:48:00.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:59 smithi152 ceph-mon[47949]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-03-23T17:48:00.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:59 smithi152 ceph-mon[47949]: from='client.24400 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:48:00.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:59 smithi152 ceph-mon[47949]: osd.10 [v2:172.21.15.92:6824/4259465100,v1:172.21.15.92:6825/4259465100] boot 2024-03-23T17:48:00.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:59 smithi152 ceph-mon[47949]: osdmap e68: 11 total, 11 up, 11 in 2024-03-23T17:48:00.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:47:59 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-23T17:48:01.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:00 smithi086 ceph-mon[38221]: osdmap e69: 11 total, 11 up, 11 in 2024-03-23T17:48:01.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:00 smithi086 ceph-mon[38221]: from='client.? 172.21.15.152:0/2482935369' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7c0b865-ccf5-4233-8d22-0d9b6dbe5c97"}]: dispatch 2024-03-23T17:48:01.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:00 smithi086 ceph-mon[38221]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7c0b865-ccf5-4233-8d22-0d9b6dbe5c97"}]: dispatch 2024-03-23T17:48:01.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:00 smithi086 ceph-mon[38221]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e7c0b865-ccf5-4233-8d22-0d9b6dbe5c97"}]': finished 2024-03-23T17:48:01.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:00 smithi086 ceph-mon[38221]: osdmap e70: 12 total, 11 up, 12 in 2024-03-23T17:48:01.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:00 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T17:48:01.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:00 smithi086 ceph-mon[38221]: osdmap e71: 12 total, 11 up, 12 in 2024-03-23T17:48:01.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:00 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T17:48:01.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:00 smithi092 ceph-mon[79089]: osdmap e69: 11 total, 11 up, 11 in 2024-03-23T17:48:01.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:00 smithi092 ceph-mon[79089]: from='client.? 172.21.15.152:0/2482935369' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7c0b865-ccf5-4233-8d22-0d9b6dbe5c97"}]: dispatch 2024-03-23T17:48:01.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:00 smithi092 ceph-mon[79089]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7c0b865-ccf5-4233-8d22-0d9b6dbe5c97"}]: dispatch 2024-03-23T17:48:01.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:00 smithi092 ceph-mon[79089]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e7c0b865-ccf5-4233-8d22-0d9b6dbe5c97"}]': finished 2024-03-23T17:48:01.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:00 smithi092 ceph-mon[79089]: osdmap e70: 12 total, 11 up, 12 in 2024-03-23T17:48:01.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:00 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T17:48:01.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:00 smithi092 ceph-mon[79089]: osdmap e71: 12 total, 11 up, 12 in 2024-03-23T17:48:01.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:00 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T17:48:01.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:00 smithi152 ceph-mon[47949]: osdmap e69: 11 total, 11 up, 11 in 2024-03-23T17:48:01.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:00 smithi152 ceph-mon[47949]: from='client.? 172.21.15.152:0/2482935369' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7c0b865-ccf5-4233-8d22-0d9b6dbe5c97"}]: dispatch 2024-03-23T17:48:01.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:00 smithi152 ceph-mon[47949]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7c0b865-ccf5-4233-8d22-0d9b6dbe5c97"}]: dispatch 2024-03-23T17:48:01.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:00 smithi152 ceph-mon[47949]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e7c0b865-ccf5-4233-8d22-0d9b6dbe5c97"}]': finished 2024-03-23T17:48:01.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:00 smithi152 ceph-mon[47949]: osdmap e70: 12 total, 11 up, 12 in 2024-03-23T17:48:01.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:00 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T17:48:01.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:00 smithi152 ceph-mon[47949]: osdmap e71: 12 total, 11 up, 12 in 2024-03-23T17:48:01.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:00 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T17:48:02.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:01 smithi086 ceph-mon[38221]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-03-23T17:48:02.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:01 smithi086 ceph-mon[38221]: from='client.? 172.21.15.152:0/3479629654' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T17:48:02.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:01 smithi086 ceph-mon[38221]: osdmap e72: 12 total, 11 up, 12 in 2024-03-23T17:48:02.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:01 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T17:48:02.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:01 smithi092 ceph-mon[79089]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-03-23T17:48:02.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:01 smithi092 ceph-mon[79089]: from='client.? 172.21.15.152:0/3479629654' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T17:48:02.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:01 smithi092 ceph-mon[79089]: osdmap e72: 12 total, 11 up, 12 in 2024-03-23T17:48:02.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:01 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T17:48:02.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:01 smithi152 ceph-mon[47949]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-03-23T17:48:02.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:01 smithi152 ceph-mon[47949]: from='client.? 172.21.15.152:0/3479629654' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T17:48:02.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:01 smithi152 ceph-mon[47949]: osdmap e72: 12 total, 11 up, 12 in 2024-03-23T17:48:02.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:01 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T17:48:04.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:03 smithi086 ceph-mon[38221]: pgmap v199: 1 pgs: 1 peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 157 KiB/s, 0 objects/s recovering 2024-03-23T17:48:04.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:03 smithi092 ceph-mon[79089]: pgmap v199: 1 pgs: 1 peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 157 KiB/s, 0 objects/s recovering 2024-03-23T17:48:04.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:03 smithi152 ceph-mon[47949]: pgmap v199: 1 pgs: 1 peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 157 KiB/s, 0 objects/s recovering 2024-03-23T17:48:06.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:05 smithi086 ceph-mon[38221]: pgmap v200: 1 pgs: 1 peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 123 KiB/s, 0 objects/s recovering 2024-03-23T17:48:06.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:05 smithi092 ceph-mon[79089]: pgmap v200: 1 pgs: 1 peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 123 KiB/s, 0 objects/s recovering 2024-03-23T17:48:06.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:05 smithi152 ceph-mon[47949]: pgmap v200: 1 pgs: 1 peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 123 KiB/s, 0 objects/s recovering 2024-03-23T17:48:06.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:06 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-23T17:48:06.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:06 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:48:07.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:06 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-23T17:48:07.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:06 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:48:07.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:06 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-23T17:48:07.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:06 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:48:08.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:07 smithi086 ceph-mon[38221]: pgmap v201: 1 pgs: 1 peering; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-03-23T17:48:08.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:07 smithi086 ceph-mon[38221]: Deploying daemon osd.11 on smithi152 2024-03-23T17:48:08.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:07 smithi092 ceph-mon[79089]: pgmap v201: 1 pgs: 1 peering; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-03-23T17:48:08.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:07 smithi092 ceph-mon[79089]: Deploying daemon osd.11 on smithi152 2024-03-23T17:48:08.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:07 smithi152 ceph-mon[47949]: pgmap v201: 1 pgs: 1 peering; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-03-23T17:48:08.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:07 smithi152 ceph-mon[47949]: Deploying daemon osd.11 on smithi152 2024-03-23T17:48:09.905 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:09 smithi152 ceph-mon[47949]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-23T17:48:10.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:09 smithi086 ceph-mon[38221]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-23T17:48:10.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:09 smithi092 ceph-mon[79089]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-23T17:48:11.724 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:11 smithi152 ceph-mon[47949]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-03-23T17:48:11.725 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:11 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:11.725 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:11 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:11.725 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:11 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:48:11.766 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:11 smithi086 ceph-mon[38221]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-03-23T17:48:11.766 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:11 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:11.766 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:11 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:11.766 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:11 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:48:11.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:11 smithi092 ceph-mon[79089]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-03-23T17:48:11.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:11 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:11.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:11 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:11.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:11 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:48:12.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:48:12.814+0000 7f98f4ff9700 1 -- 172.21.15.152:0/24423745 <== mgr.14152 v2:172.21.15.86:6800/2734784244 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f98f802cdc0 con 0x7f98e00a1120 2024-03-23T17:48:12.815 INFO:teuthology.orchestra.run.smithi152.stdout:Created osd(s) 11 on host 'smithi152' 2024-03-23T17:48:12.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:48:12.817+0000 7f98fdcc7700 1 -- 172.21.15.152:0/24423745 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f98e00a1120 msgr2=0x7f98e00a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:12.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:48:12.817+0000 7f98fdcc7700 1 --2- 172.21.15.152:0/24423745 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f98e00a1120 0x7f98e00a35e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f98f81981c0 tx=0x7f98e80073d0 comp rx=0 tx=0).stop 2024-03-23T17:48:12.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:48:12.817+0000 7f98fdcc7700 1 -- 172.21.15.152:0/24423745 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f98f8101910 msgr2=0x7f98f8196d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:12.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:48:12.817+0000 7f98fdcc7700 1 --2- 172.21.15.152:0/24423745 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f98f8101910 0x7f98f8196d00 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f98ec0095c0 tx=0x7f98ec00f690 comp rx=0 tx=0).stop 2024-03-23T17:48:12.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:48:12.817+0000 7f98fdcc7700 1 -- 172.21.15.152:0/24423745 shutdown_connections 2024-03-23T17:48:12.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:48:12.817+0000 7f98fdcc7700 1 --2- 172.21.15.152:0/24423745 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f98e00a1120 0x7f98e00a35e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:12.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:48:12.817+0000 7f98fdcc7700 1 --2- 172.21.15.152:0/24423745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98f8106b70 0x7f98f819e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:12.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:48:12.817+0000 7f98fdcc7700 1 --2- 172.21.15.152:0/24423745 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f98f8101910 0x7f98f8196d00 secure :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f98ec0095c0 tx=0x7f98ec00f690 comp rx=0 tx=0).stop 2024-03-23T17:48:12.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:48:12.817+0000 7f98fdcc7700 1 --2- 172.21.15.152:0/24423745 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98f8104240 0x7f98f8197240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:12.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:48:12.817+0000 7f98fdcc7700 1 -- 172.21.15.152:0/24423745 >> 172.21.15.152:0/24423745 conn(0x7f98f80fb2c0 msgr2=0x7f98f8102250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:48:12.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:48:12.817+0000 7f98fdcc7700 1 -- 172.21.15.152:0/24423745 shutdown_connections 2024-03-23T17:48:12.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:48:12.817+0000 7f98fdcc7700 1 -- 172.21.15.152:0/24423745 wait complete. 2024-03-23T17:48:13.213 DEBUG:teuthology.orchestra.run.smithi152:osd.11> sudo journalctl -f -n 0 -u ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@osd.11.service 2024-03-23T17:48:13.216 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-03-23T17:48:13.216 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph osd stat -f json 2024-03-23T17:48:13.241 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:13 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:13.241 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:13 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:13.242 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:13 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:48:13.242 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:13 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:48:13.242 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:13 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:13.242 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:13 smithi152 ceph-mon[47949]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-03-23T17:48:13.242 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:13 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:13.242 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:13 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:13.242 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:13 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:48:13.248 INFO:journalctl@ceph.osd.11.smithi152.stdout:-- Logs begin at Sat 2024-03-23 17:19:34 UTC. -- 2024-03-23T17:48:13.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:13 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:13.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:13 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:13.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:13 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:48:13.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:13 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:48:13.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:13 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:13.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:13 smithi086 ceph-mon[38221]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-03-23T17:48:13.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:13 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:13.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:13 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:13.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:13 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:48:13.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:13 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:13.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:13 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:13.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:13 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:48:13.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:13 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:48:13.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:13 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:13.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:13 smithi092 ceph-mon[79089]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-03-23T17:48:13.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:13 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:13.558 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:13 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:13.558 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:13 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:48:13.583 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T17:48:14.836 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 17:48:14 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T17:48:14.416+0000 7fcad1919640 -1 osd.11 0 log_to_monitors true 2024-03-23T17:48:14.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:14.842+0000 7f4385200700 1 -- 172.21.15.86:0/4160772531 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4380074de0 msgr2=0x7f4380073240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:14.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:14.842+0000 7f4385200700 1 --2- 172.21.15.86:0/4160772531 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4380074de0 0x7f4380073240 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4370009990 tx=0x7f4370009ca0 comp rx=0 tx=0).stop 2024-03-23T17:48:14.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:14.842+0000 7f4385200700 1 -- 172.21.15.86:0/4160772531 shutdown_connections 2024-03-23T17:48:14.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:14.842+0000 7f4385200700 1 --2- 172.21.15.86:0/4160772531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4380108cd0 0x7f438010b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:14.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:14.842+0000 7f4385200700 1 --2- 172.21.15.86:0/4160772531 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4380074de0 0x7f4380073240 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:14.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:14.842+0000 7f4385200700 1 --2- 172.21.15.86:0/4160772531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4380073810 0x7f4380073c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:14.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:14.842+0000 7f4385200700 1 -- 172.21.15.86:0/4160772531 >> 172.21.15.86:0/4160772531 conn(0x7f43800fc3c0 msgr2=0x7f43800fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:48:14.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:14.842+0000 7f4385200700 1 -- 172.21.15.86:0/4160772531 shutdown_connections 2024-03-23T17:48:14.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:14.842+0000 7f4385200700 1 -- 172.21.15.86:0/4160772531 wait complete. 2024-03-23T17:48:14.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:14.843+0000 7f4385200700 1 Processor -- start 2024-03-23T17:48:14.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:14.843+0000 7f4385200700 1 -- start start 2024-03-23T17:48:14.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:14.844+0000 7f4385200700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4380073810 0x7f4380198fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:14.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:14.844+0000 7f4385200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4380074de0 0x7f4380199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:14.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:14.844+0000 7f4385200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4380108cd0 0x7f43801a04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:14.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:14.844+0000 7f4385200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43801a0ac0 con 0x7f4380108cd0 2024-03-23T17:48:14.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:14.844+0000 7f4385200700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43801a0c00 con 0x7f4380074de0 2024-03-23T17:48:14.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:14.844+0000 7f4385200700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43801a0d90 con 0x7f4380073810 2024-03-23T17:48:14.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:14.844+0000 7f437de00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4380073810 0x7f4380198fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:14.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:14.844+0000 7f437de00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4380073810 0x7f4380198fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:48642/0 (socket says 172.21.15.86:48642) 2024-03-23T17:48:14.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:14.844+0000 7f437de00700 1 -- 172.21.15.86:0/2910459330 learned_addr learned my addr 172.21.15.86:0/2910459330 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:48:14.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:14.844+0000 7f437e800700 1 --2- 172.21.15.86:0/2910459330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4380108cd0 0x7f43801a04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:14.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:14.844+0000 7f437d400700 1 --2- 172.21.15.86:0/2910459330 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4380074de0 0x7f4380199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:14.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:14.844+0000 7f437de00700 1 -- 172.21.15.86:0/2910459330 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4380074de0 msgr2=0x7f4380199510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:14.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:14.844+0000 7f437de00700 1 --2- 172.21.15.86:0/2910459330 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4380074de0 0x7f4380199510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:14.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:14.844+0000 7f437de00700 1 -- 172.21.15.86:0/2910459330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4380108cd0 msgr2=0x7f43801a04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:14.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:14.844+0000 7f437de00700 1 --2- 172.21.15.86:0/2910459330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4380108cd0 0x7f43801a04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:14.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:14.844+0000 7f437de00700 1 -- 172.21.15.86:0/2910459330 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43700095f0 con 0x7f4380073810 2024-03-23T17:48:14.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:14.844+0000 7f437de00700 1 --2- 172.21.15.86:0/2910459330 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4380073810 0x7f4380198fd0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4370005740 tx=0x7f4370009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:48:14.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:14.845+0000 7f4376a00700 1 -- 172.21.15.86:0/2910459330 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f437001d070 con 0x7f4380073810 2024-03-23T17:48:14.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:14.845+0000 7f4376a00700 1 -- 172.21.15.86:0/2910459330 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4370021410 con 0x7f4380073810 2024-03-23T17:48:14.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:14.845+0000 7f4376a00700 1 -- 172.21.15.86:0/2910459330 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4370017650 con 0x7f4380073810 2024-03-23T17:48:14.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:14.845+0000 7f4385200700 1 -- 172.21.15.86:0/2910459330 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43801a1010 con 0x7f4380073810 2024-03-23T17:48:14.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:14.845+0000 7f4385200700 1 -- 172.21.15.86:0/2910459330 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43801a1420 con 0x7f4380073810 2024-03-23T17:48:14.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:14.846+0000 7f4385200700 1 -- 172.21.15.86:0/2910459330 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4380107a90 con 0x7f4380073810 2024-03-23T17:48:14.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:14.848+0000 7f4376a00700 1 -- 172.21.15.86:0/2910459330 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f4370021580 con 0x7f4380073810 2024-03-23T17:48:14.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:14.849+0000 7f4376a00700 1 --2- 172.21.15.86:0/2910459330 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f43640a11a0 0x7f43640a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:14.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:14.849+0000 7f4376a00700 1 -- 172.21.15.86:0/2910459330 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f43700cd690 con 0x7f4380073810 2024-03-23T17:48:14.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:14.849+0000 7f437d400700 1 --2- 172.21.15.86:0/2910459330 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f43640a11a0 0x7f43640a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:14.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:14.850+0000 7f437d400700 1 --2- 172.21.15.86:0/2910459330 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f43640a11a0 0x7f43640a3660 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4380074b10 tx=0x7f436c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:48:14.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:14.853+0000 7f4376a00700 1 -- 172.21.15.86:0/2910459330 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f43700973f0 con 0x7f4380073810 2024-03-23T17:48:15.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:15.057+0000 7f4385200700 1 -- 172.21.15.86:0/2910459330 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f438004ea80 con 0x7f4380073810 2024-03-23T17:48:15.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:15.058+0000 7f4376a00700 1 -- 172.21.15.86:0/2910459330 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v72) v1 ==== 74+0+133 (secure 0 0 0) 0x7f437009aa10 con 0x7f4380073810 2024-03-23T17:48:15.059 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:48:15.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:15.060+0000 7f4385200700 1 -- 172.21.15.86:0/2910459330 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f43640a11a0 msgr2=0x7f43640a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:15.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:15.060+0000 7f4385200700 1 --2- 172.21.15.86:0/2910459330 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f43640a11a0 0x7f43640a3660 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4380074b10 tx=0x7f436c009260 comp rx=0 tx=0).stop 2024-03-23T17:48:15.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:15.060+0000 7f4385200700 1 -- 172.21.15.86:0/2910459330 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4380073810 msgr2=0x7f4380198fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:15.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:15.060+0000 7f4385200700 1 --2- 172.21.15.86:0/2910459330 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4380073810 0x7f4380198fd0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4370005740 tx=0x7f4370009580 comp rx=0 tx=0).stop 2024-03-23T17:48:15.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:15.060+0000 7f4385200700 1 -- 172.21.15.86:0/2910459330 shutdown_connections 2024-03-23T17:48:15.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:15.060+0000 7f4385200700 1 --2- 172.21.15.86:0/2910459330 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f43640a11a0 0x7f43640a3660 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:15.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:15.060+0000 7f4385200700 1 --2- 172.21.15.86:0/2910459330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4380108cd0 0x7f43801a04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:15.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:15.060+0000 7f4385200700 1 --2- 172.21.15.86:0/2910459330 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4380073810 0x7f4380198fd0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:15.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:15.060+0000 7f4385200700 1 --2- 172.21.15.86:0/2910459330 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4380074de0 0x7f4380199510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:15.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:15.060+0000 7f4385200700 1 -- 172.21.15.86:0/2910459330 >> 172.21.15.86:0/2910459330 conn(0x7f43800fc3c0 msgr2=0x7f4380104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:48:15.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:15.060+0000 7f4385200700 1 -- 172.21.15.86:0/2910459330 shutdown_connections 2024-03-23T17:48:15.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:15.060+0000 7f4385200700 1 -- 172.21.15.86:0/2910459330 wait complete. 2024-03-23T17:48:15.497 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":72,"num_osds":12,"num_up_osds":11,"osd_up_since":1711216078,"num_in_osds":12,"osd_in_since":1711216080,"num_remapped_pgs":0} 2024-03-23T17:48:15.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:15 smithi152 ceph-mon[47949]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-03-23T17:48:15.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:15 smithi152 ceph-mon[47949]: from='osd.11 [v2:172.21.15.152:6824/194662760,v1:172.21.15.152:6825/194662760]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-23T17:48:15.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:15 smithi152 ceph-mon[47949]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-23T17:48:15.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:15 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/2910459330' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-23T17:48:15.766 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:15 smithi086 ceph-mon[38221]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-03-23T17:48:15.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:15 smithi086 ceph-mon[38221]: from='osd.11 [v2:172.21.15.152:6824/194662760,v1:172.21.15.152:6825/194662760]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-23T17:48:15.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:15 smithi086 ceph-mon[38221]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-23T17:48:15.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:15 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/2910459330' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-23T17:48:15.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:15 smithi092 ceph-mon[79089]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-03-23T17:48:15.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:15 smithi092 ceph-mon[79089]: from='osd.11 [v2:172.21.15.152:6824/194662760,v1:172.21.15.152:6825/194662760]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-23T17:48:15.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:15 smithi092 ceph-mon[79089]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-23T17:48:15.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:15 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/2910459330' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-23T17:48:16.497 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph osd stat -f json 2024-03-23T17:48:16.766 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:16 smithi086 ceph-mon[38221]: Detected new or changed devices on smithi152 2024-03-23T17:48:16.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:16 smithi086 ceph-mon[38221]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-23T17:48:16.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:16 smithi086 ceph-mon[38221]: osdmap e73: 12 total, 11 up, 12 in 2024-03-23T17:48:16.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:16 smithi086 ceph-mon[38221]: from='osd.11 [v2:172.21.15.152:6824/194662760,v1:172.21.15.152:6825/194662760]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-03-23T17:48:16.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:16 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T17:48:16.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:16 smithi086 ceph-mon[38221]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-03-23T17:48:16.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:16 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:16.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:16 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:16.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:16 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:48:16.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:16 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:48:16.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:16 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:48:16.768 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:16 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:48:16.768 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:16 smithi086 ceph-mon[38221]: Adjusting osd_memory_target on smithi152 to 4278M 2024-03-23T17:48:16.768 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:16 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:16.768 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:16 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:48:16.768 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:16 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:48:16.768 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:16 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:16.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:16 smithi092 ceph-mon[79089]: Detected new or changed devices on smithi152 2024-03-23T17:48:16.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:16 smithi092 ceph-mon[79089]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-23T17:48:16.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:16 smithi092 ceph-mon[79089]: osdmap e73: 12 total, 11 up, 12 in 2024-03-23T17:48:16.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:16 smithi092 ceph-mon[79089]: from='osd.11 [v2:172.21.15.152:6824/194662760,v1:172.21.15.152:6825/194662760]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-03-23T17:48:16.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:16 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T17:48:16.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:16 smithi092 ceph-mon[79089]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-03-23T17:48:16.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:16 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:16.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:16 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:16.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:16 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:48:16.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:16 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:48:16.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:16 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:48:16.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:16 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:48:16.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:16 smithi092 ceph-mon[79089]: Adjusting osd_memory_target on smithi152 to 4278M 2024-03-23T17:48:16.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:16 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:16.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:16 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:48:16.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:16 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:48:16.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:16 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:16.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:16 smithi152 ceph-mon[47949]: Detected new or changed devices on smithi152 2024-03-23T17:48:16.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:16 smithi152 ceph-mon[47949]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-23T17:48:16.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:16 smithi152 ceph-mon[47949]: osdmap e73: 12 total, 11 up, 12 in 2024-03-23T17:48:16.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:16 smithi152 ceph-mon[47949]: from='osd.11 [v2:172.21.15.152:6824/194662760,v1:172.21.15.152:6825/194662760]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-03-23T17:48:16.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:16 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T17:48:16.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:16 smithi152 ceph-mon[47949]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-03-23T17:48:16.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:16 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:16.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:16 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:16.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:16 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:48:16.837 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:16 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:48:16.837 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:16 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:48:16.837 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:16 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:48:16.837 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:16 smithi152 ceph-mon[47949]: Adjusting osd_memory_target on smithi152 to 4278M 2024-03-23T17:48:16.837 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:16 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:16.837 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:16 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:48:16.837 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:16 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:48:16.837 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:16 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:16.854 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T17:48:17.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 17:48:17 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T17:48:17.214+0000 7fcac677c700 -1 osd.11 0 waiting for initial osdmap 2024-03-23T17:48:17.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 17:48:17 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T17:48:17.234+0000 7fcabed6a700 -1 osd.11 74 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T17:48:17.587 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:17 smithi152 ceph-mon[47949]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-03-23T17:48:17.587 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:17 smithi152 ceph-mon[47949]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]': finished 2024-03-23T17:48:17.587 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:17 smithi152 ceph-mon[47949]: osdmap e74: 12 total, 11 up, 12 in 2024-03-23T17:48:17.587 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:17 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T17:48:17.587 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:17 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T17:48:17.587 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:17 smithi152 ceph-mon[47949]: from='osd.11 ' entity='osd.11' 2024-03-23T17:48:17.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:17 smithi086 ceph-mon[38221]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-03-23T17:48:17.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:17 smithi086 ceph-mon[38221]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]': finished 2024-03-23T17:48:17.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:17 smithi086 ceph-mon[38221]: osdmap e74: 12 total, 11 up, 12 in 2024-03-23T17:48:17.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:17 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T17:48:17.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:17 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T17:48:17.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:17 smithi086 ceph-mon[38221]: from='osd.11 ' entity='osd.11' 2024-03-23T17:48:17.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:17 smithi092 ceph-mon[79089]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-03-23T17:48:17.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:17 smithi092 ceph-mon[79089]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]': finished 2024-03-23T17:48:17.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:17 smithi092 ceph-mon[79089]: osdmap e74: 12 total, 11 up, 12 in 2024-03-23T17:48:17.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:17 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T17:48:17.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:17 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T17:48:17.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:17 smithi092 ceph-mon[79089]: from='osd.11 ' entity='osd.11' 2024-03-23T17:48:18.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:18.136+0000 7f919ac00700 1 -- 172.21.15.86:0/2227375615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9194100e00 msgr2=0x7f9194101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:18.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:18.136+0000 7f919ac00700 1 --2- 172.21.15.86:0/2227375615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9194100e00 0x7f9194101220 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9184009990 tx=0x7f9184009ca0 comp rx=0 tx=0).stop 2024-03-23T17:48:18.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:18.136+0000 7f919ac00700 1 -- 172.21.15.86:0/2227375615 shutdown_connections 2024-03-23T17:48:18.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:18.136+0000 7f919ac00700 1 --2- 172.21.15.86:0/2227375615 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9194102920 0x7f9194109240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:18.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:18.136+0000 7f919ac00700 1 --2- 172.21.15.86:0/2227375615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9194100e00 0x7f9194101220 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:18.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:18.136+0000 7f919ac00700 1 --2- 172.21.15.86:0/2227375615 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9194101f60 0x7f91941023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:18.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:18.136+0000 7f919ac00700 1 -- 172.21.15.86:0/2227375615 >> 172.21.15.86:0/2227375615 conn(0x7f91940fc3c0 msgr2=0x7f91940fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:48:18.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:18.136+0000 7f919ac00700 1 -- 172.21.15.86:0/2227375615 shutdown_connections 2024-03-23T17:48:18.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:18.136+0000 7f919ac00700 1 -- 172.21.15.86:0/2227375615 wait complete. 2024-03-23T17:48:18.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:18.137+0000 7f919ac00700 1 Processor -- start 2024-03-23T17:48:18.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:18.137+0000 7f919ac00700 1 -- start start 2024-03-23T17:48:18.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:18.137+0000 7f919ac00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9194100e00 0x7f9194198f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:18.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:18.137+0000 7f919ac00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9194101f60 0x7f9194199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:18.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:18.137+0000 7f919ac00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9194102920 0x7f91941a0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:18.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:18.137+0000 7f919ac00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91941a0a60 con 0x7f9194101f60 2024-03-23T17:48:18.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:18.137+0000 7f919ac00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91941a0ba0 con 0x7f9194102920 2024-03-23T17:48:18.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:18.137+0000 7f919ac00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91941a0d30 con 0x7f9194100e00 2024-03-23T17:48:18.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:18.138+0000 7f9193400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9194101f60 0x7f9194199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:18.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:18.138+0000 7f9193400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9194101f60 0x7f9194199460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55832/0 (socket says 172.21.15.86:55832) 2024-03-23T17:48:18.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:18.138+0000 7f9193400700 1 -- 172.21.15.86:0/1953730674 learned_addr learned my addr 172.21.15.86:0/1953730674 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:48:18.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:18.138+0000 7f9198c00700 1 --2- 172.21.15.86:0/1953730674 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9194102920 0x7f91941a0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:18.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:18.138+0000 7f9193e00700 1 --2- 172.21.15.86:0/1953730674 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9194100e00 0x7f9194198f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:18.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:18.138+0000 7f9193400700 1 -- 172.21.15.86:0/1953730674 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9194100e00 msgr2=0x7f9194198f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:18.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:18.138+0000 7f9193400700 1 --2- 172.21.15.86:0/1953730674 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9194100e00 0x7f9194198f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:18.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:18.138+0000 7f9193400700 1 -- 172.21.15.86:0/1953730674 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9194102920 msgr2=0x7f91941a0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:18.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:18.138+0000 7f9193400700 1 --2- 172.21.15.86:0/1953730674 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9194102920 0x7f91941a0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:18.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:18.138+0000 7f9193400700 1 -- 172.21.15.86:0/1953730674 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91840095f0 con 0x7f9194101f60 2024-03-23T17:48:18.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:18.138+0000 7f9193400700 1 --2- 172.21.15.86:0/1953730674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9194101f60 0x7f9194199460 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f918000d9b0 tx=0x7f918000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:48:18.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:18.138+0000 7f9190c00700 1 -- 172.21.15.86:0/1953730674 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9180004220 con 0x7f9194101f60 2024-03-23T17:48:18.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:18.139+0000 7f9190c00700 1 -- 172.21.15.86:0/1953730674 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9180007640 con 0x7f9194101f60 2024-03-23T17:48:18.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:18.139+0000 7f9190c00700 1 -- 172.21.15.86:0/1953730674 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9180005590 con 0x7f9194101f60 2024-03-23T17:48:18.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:18.139+0000 7f919ac00700 1 -- 172.21.15.86:0/1953730674 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91941a0fb0 con 0x7f9194101f60 2024-03-23T17:48:18.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:18.139+0000 7f919ac00700 1 -- 172.21.15.86:0/1953730674 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91941a1430 con 0x7f9194101f60 2024-03-23T17:48:18.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:18.140+0000 7f919ac00700 1 -- 172.21.15.86:0/1953730674 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f919404ea80 con 0x7f9194101f60 2024-03-23T17:48:18.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:18.142+0000 7f9190c00700 1 -- 172.21.15.86:0/1953730674 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f9180004750 con 0x7f9194101f60 2024-03-23T17:48:18.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:18.143+0000 7f9190c00700 1 --2- 172.21.15.86:0/1953730674 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f91780a1120 0x7f91780a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:18.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:18.143+0000 7f9190c00700 1 -- 172.21.15.86:0/1953730674 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6847+0+0 (secure 0 0 0) 0x7f91800cc150 con 0x7f9194101f60 2024-03-23T17:48:18.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:18.143+0000 7f9193e00700 1 --2- 172.21.15.86:0/1953730674 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f91780a1120 0x7f91780a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:18.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:18.144+0000 7f9193e00700 1 --2- 172.21.15.86:0/1953730674 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f91780a1120 0x7f91780a35e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f918400b2e0 tx=0x7f918400b6f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:48:18.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:18.147+0000 7f9190c00700 1 -- 172.21.15.86:0/1953730674 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f9180095e00 con 0x7f9194101f60 2024-03-23T17:48:18.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:18.355+0000 7f919ac00700 1 -- 172.21.15.86:0/1953730674 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f919410c630 con 0x7f9194101f60 2024-03-23T17:48:18.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:18.356+0000 7f9190c00700 1 -- 172.21.15.86:0/1953730674 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v75) v1 ==== 74+0+133 (secure 0 0 0) 0x7f9180099420 con 0x7f9194101f60 2024-03-23T17:48:18.356 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:48:18.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:18.358+0000 7f919ac00700 1 -- 172.21.15.86:0/1953730674 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f91780a1120 msgr2=0x7f91780a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:18.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:18.358+0000 7f919ac00700 1 --2- 172.21.15.86:0/1953730674 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f91780a1120 0x7f91780a35e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f918400b2e0 tx=0x7f918400b6f0 comp rx=0 tx=0).stop 2024-03-23T17:48:18.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:18.358+0000 7f919ac00700 1 -- 172.21.15.86:0/1953730674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9194101f60 msgr2=0x7f9194199460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:18.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:18.358+0000 7f919ac00700 1 --2- 172.21.15.86:0/1953730674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9194101f60 0x7f9194199460 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f918000d9b0 tx=0x7f918000dcc0 comp rx=0 tx=0).stop 2024-03-23T17:48:18.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:18.358+0000 7f919ac00700 1 -- 172.21.15.86:0/1953730674 shutdown_connections 2024-03-23T17:48:18.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:18.358+0000 7f919ac00700 1 --2- 172.21.15.86:0/1953730674 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9194102920 0x7f91941a0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:18.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:18.358+0000 7f919ac00700 1 --2- 172.21.15.86:0/1953730674 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9194100e00 0x7f9194198f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:18.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:18.358+0000 7f919ac00700 1 --2- 172.21.15.86:0/1953730674 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f91780a1120 0x7f91780a35e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:18.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:18.358+0000 7f919ac00700 1 --2- 172.21.15.86:0/1953730674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9194101f60 0x7f9194199460 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:18.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:18.358+0000 7f919ac00700 1 -- 172.21.15.86:0/1953730674 >> 172.21.15.86:0/1953730674 conn(0x7f91940fc3c0 msgr2=0x7f9194072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:48:18.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:18.358+0000 7f919ac00700 1 -- 172.21.15.86:0/1953730674 shutdown_connections 2024-03-23T17:48:18.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:18.358+0000 7f919ac00700 1 -- 172.21.15.86:0/1953730674 wait complete. 2024-03-23T17:48:18.766 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:18 smithi086 ceph-mon[38221]: purged_snaps scrub starts 2024-03-23T17:48:18.766 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:18 smithi086 ceph-mon[38221]: purged_snaps scrub ok 2024-03-23T17:48:18.766 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:18 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T17:48:18.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:18 smithi086 ceph-mon[38221]: osd.11 [v2:172.21.15.152:6824/194662760,v1:172.21.15.152:6825/194662760] boot 2024-03-23T17:48:18.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:18 smithi086 ceph-mon[38221]: osdmap e75: 12 total, 12 up, 12 in 2024-03-23T17:48:18.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:18 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T17:48:18.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:18 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/1953730674' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-23T17:48:18.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:18 smithi092 ceph-mon[79089]: purged_snaps scrub starts 2024-03-23T17:48:18.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:18 smithi092 ceph-mon[79089]: purged_snaps scrub ok 2024-03-23T17:48:18.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:18 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T17:48:18.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:18 smithi092 ceph-mon[79089]: osd.11 [v2:172.21.15.152:6824/194662760,v1:172.21.15.152:6825/194662760] boot 2024-03-23T17:48:18.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:18 smithi092 ceph-mon[79089]: osdmap e75: 12 total, 12 up, 12 in 2024-03-23T17:48:18.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:18 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T17:48:18.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:18 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/1953730674' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-23T17:48:18.820 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":75,"num_osds":12,"num_up_osds":12,"osd_up_since":1711216098,"num_in_osds":12,"osd_in_since":1711216080,"num_remapped_pgs":0} 2024-03-23T17:48:18.820 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph osd dump --format=json 2024-03-23T17:48:18.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:18 smithi152 ceph-mon[47949]: purged_snaps scrub starts 2024-03-23T17:48:18.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:18 smithi152 ceph-mon[47949]: purged_snaps scrub ok 2024-03-23T17:48:18.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:18 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T17:48:18.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:18 smithi152 ceph-mon[47949]: osd.11 [v2:172.21.15.152:6824/194662760,v1:172.21.15.152:6825/194662760] boot 2024-03-23T17:48:18.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:18 smithi152 ceph-mon[47949]: osdmap e75: 12 total, 12 up, 12 in 2024-03-23T17:48:18.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:18 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T17:48:18.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:18 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/1953730674' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-23T17:48:19.170 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T17:48:19.721 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:19 smithi086 ceph-mon[38221]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-03-23T17:48:19.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:19 smithi092 ceph-mon[79089]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-03-23T17:48:19.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:19 smithi152 ceph-mon[47949]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-03-23T17:48:20.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:20.389+0000 7f026fc00700 1 -- 172.21.15.86:0/755847189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0268100e60 msgr2=0x7f02681012e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:20.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:20.389+0000 7f026fc00700 1 --2- 172.21.15.86:0/755847189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0268100e60 0x7f02681012e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f02540099c0 tx=0x7f0254009cd0 comp rx=0 tx=0).stop 2024-03-23T17:48:20.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:20.389+0000 7f026fc00700 1 -- 172.21.15.86:0/755847189 shutdown_connections 2024-03-23T17:48:20.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:20.389+0000 7f026fc00700 1 --2- 172.21.15.86:0/755847189 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0268101820 0x7f026810dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:20.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:20.389+0000 7f026fc00700 1 --2- 172.21.15.86:0/755847189 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0268106ef0 0x7f02681072d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:20.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:20.389+0000 7f026fc00700 1 --2- 172.21.15.86:0/755847189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0268100e60 0x7f02681012e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:20.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:20.389+0000 7f026fc00700 1 -- 172.21.15.86:0/755847189 >> 172.21.15.86:0/755847189 conn(0x7f0268077fa0 msgr2=0x7f02680783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:48:20.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:20.389+0000 7f026fc00700 1 -- 172.21.15.86:0/755847189 shutdown_connections 2024-03-23T17:48:20.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:20.389+0000 7f026fc00700 1 -- 172.21.15.86:0/755847189 wait complete. 2024-03-23T17:48:20.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:20.391+0000 7f026fc00700 1 Processor -- start 2024-03-23T17:48:20.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:20.391+0000 7f026fc00700 1 -- start start 2024-03-23T17:48:20.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:20.391+0000 7f026fc00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0268100e60 0x7f02681971e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:20.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:20.391+0000 7f026fc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0268101820 0x7f0268197720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:20.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:20.391+0000 7f026fc00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0268106ef0 0x7f026819bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:20.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:20.391+0000 7f026fc00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f026819c0e0 con 0x7f0268101820 2024-03-23T17:48:20.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:20.391+0000 7f026fc00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f026819c250 con 0x7f0268100e60 2024-03-23T17:48:20.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:20.391+0000 7f026fc00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f026819c410 con 0x7f0268106ef0 2024-03-23T17:48:20.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:20.392+0000 7f0267e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0268101820 0x7f0268197720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:20.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:20.392+0000 7f026d200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0268100e60 0x7f02681971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:20.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:20.392+0000 7f026d200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0268100e60 0x7f02681971e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:60734/0 (socket says 172.21.15.86:60734) 2024-03-23T17:48:20.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:20.392+0000 7f026d200700 1 -- 172.21.15.86:0/2188178155 learned_addr learned my addr 172.21.15.86:0/2188178155 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:48:20.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:20.392+0000 7f026dc00700 1 --2- 172.21.15.86:0/2188178155 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0268106ef0 0x7f026819bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:20.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:20.392+0000 7f0267e00700 1 -- 172.21.15.86:0/2188178155 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0268106ef0 msgr2=0x7f026819bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:20.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:20.392+0000 7f0267e00700 1 --2- 172.21.15.86:0/2188178155 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0268106ef0 0x7f026819bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:20.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:20.392+0000 7f0267e00700 1 -- 172.21.15.86:0/2188178155 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0268100e60 msgr2=0x7f02681971e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:20.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:20.392+0000 7f0267e00700 1 --2- 172.21.15.86:0/2188178155 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0268100e60 0x7f02681971e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:20.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:20.392+0000 7f0267e00700 1 -- 172.21.15.86:0/2188178155 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02540095f0 con 0x7f0268101820 2024-03-23T17:48:20.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:20.392+0000 7f026d200700 1 --2- 172.21.15.86:0/2188178155 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0268100e60 0x7f02681971e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:48:20.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:20.392+0000 7f0267e00700 1 --2- 172.21.15.86:0/2188178155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0268101820 0x7f0268197720 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f02540098e0 tx=0x7f025400b9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:48:20.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:20.396+0000 7f0265600700 1 -- 172.21.15.86:0/2188178155 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f025401d070 con 0x7f0268101820 2024-03-23T17:48:20.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:20.396+0000 7f0265600700 1 -- 172.21.15.86:0/2188178155 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f025400be00 con 0x7f0268101820 2024-03-23T17:48:20.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:20.396+0000 7f026fc00700 1 -- 172.21.15.86:0/2188178155 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f026819c690 con 0x7f0268101820 2024-03-23T17:48:20.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:20.396+0000 7f0265600700 1 -- 172.21.15.86:0/2188178155 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f025400f9a0 con 0x7f0268101820 2024-03-23T17:48:20.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:20.396+0000 7f026fc00700 1 -- 172.21.15.86:0/2188178155 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02681a3f50 con 0x7f0268101820 2024-03-23T17:48:20.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:20.397+0000 7f026fc00700 1 -- 172.21.15.86:0/2188178155 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02681092f0 con 0x7f0268101820 2024-03-23T17:48:20.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:20.399+0000 7f0265600700 1 -- 172.21.15.86:0/2188178155 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f0254004c60 con 0x7f0268101820 2024-03-23T17:48:20.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:20.400+0000 7f0265600700 1 --2- 172.21.15.86:0/2188178155 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f02500a11a0 0x7f02500a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:20.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:20.400+0000 7f0265600700 1 -- 172.21.15.86:0/2188178155 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f02540ccfa0 con 0x7f0268101820 2024-03-23T17:48:20.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:20.400+0000 7f026d200700 1 --2- 172.21.15.86:0/2188178155 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f02500a11a0 0x7f02500a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:20.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:20.401+0000 7f026d200700 1 --2- 172.21.15.86:0/2188178155 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f02500a11a0 0x7f02500a3660 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f025c011550 tx=0x7f025c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:48:20.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:20.404+0000 7f0265600700 1 -- 172.21.15.86:0/2188178155 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f0254096bc0 con 0x7f0268101820 2024-03-23T17:48:20.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:20 smithi086 ceph-mon[38221]: osdmap e76: 12 total, 12 up, 12 in 2024-03-23T17:48:20.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:20.608+0000 7f026fc00700 1 -- 172.21.15.86:0/2188178155 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f02681984f0 con 0x7f0268101820 2024-03-23T17:48:20.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:20.609+0000 7f0265600700 1 -- 172.21.15.86:0/2188178155 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+18386 (secure 0 0 0) 0x7f025409a1e0 con 0x7f0268101820 2024-03-23T17:48:20.611 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:48:20.611 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":76,"fsid":"9d76c9e8-e93c-11ee-95cd-87774f69a715","created":"2024-03-23T17:42:30.097233+0000","modified":"2024-03-23T17:48:19.377461+0000","last_up_change":"2024-03-23T17:48:18.217601+0000","last_in_change":"2024-03-23T17:48:00.250579+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T17:45:26.848397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"656655c6-1c7f-4640-92c4-2f42de820143","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6803","nonce":2423505491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6805","nonce":2423505491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6809","nonce":2423505491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6807","nonce":2423505491}]},"public_addr":"172.21.15.86:6803/2423505491","cluster_addr":"172.21.15.86:6805/2423505491","heartbeat_back_addr":"172.21.15.86:6809/2423505491","heartbeat_front_addr":"172.21.15.86:6807/2423505491","state":["exists","up"]},{"osd":1,"uuid":"3e86dc65-20a6-4de8-b9ce-fcf01f68862b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6801","nonce":3776252597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6803","nonce":3776252597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6807","nonce":3776252597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6805","nonce":3776252597}]},"public_addr":"172.21.15.92:6801/3776252597","cluster_addr":"172.21.15.92:6803/3776252597","heartbeat_back_addr":"172.21.15.92:6807/3776252597","heartbeat_front_addr":"172.21.15.92:6805/3776252597","state":["exists","up"]},{"osd":2,"uuid":"09808786-3bd3-4d11-8c7b-1736119dfbb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6801","nonce":1601265891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6803","nonce":1601265891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6807","nonce":1601265891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6805","nonce":1601265891}]},"public_addr":"172.21.15.152:6801/1601265891","cluster_addr":"172.21.15.152:6803/1601265891","heartbeat_back_addr":"172.21.15.152:6807/1601265891","heartbeat_front_addr":"172.21.15.152:6805/1601265891","state":["exists","up"]},{"osd":3,"uuid":"79c61910-4b53-424c-b150-6f5385a2cd68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6811","nonce":694846466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6813","nonce":694846466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6817","nonce":694846466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6815","nonce":694846466}]},"public_addr":"172.21.15.86:6811/694846466","cluster_addr":"172.21.15.86:6813/694846466","heartbeat_back_addr":"172.21.15.86:6817/694846466","heartbeat_front_addr":"172.21.15.86:6815/694846466","state":["exists","up"]},{"osd":4,"uuid":"a2878035-0179-469f-bbd4-8ce2d3f6bddc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6809","nonce":1422821471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6811","nonce":1422821471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6815","nonce":1422821471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6813","nonce":1422821471}]},"public_addr":"172.21.15.92:6809/1422821471","cluster_addr":"172.21.15.92:6811/1422821471","heartbeat_back_addr":"172.21.15.92:6815/1422821471","heartbeat_front_addr":"172.21.15.92:6813/1422821471","state":["exists","up"]},{"osd":5,"uuid":"837366aa-ce25-42ca-b157-5bbb50159d0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6809","nonce":1039375239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6811","nonce":1039375239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6815","nonce":1039375239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6813","nonce":1039375239}]},"public_addr":"172.21.15.152:6809/1039375239","cluster_addr":"172.21.15.152:6811/1039375239","heartbeat_back_addr":"172.21.15.152:6815/1039375239","heartbeat_front_addr":"172.21.15.152:6813/1039375239","state":["exists","up"]},{"osd":6,"uuid":"61024437-7b91-432f-a1af-6443d0ef5213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6819","nonce":4080452707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6821","nonce":4080452707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6825","nonce":4080452707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6823","nonce":4080452707}]},"public_addr":"172.21.15.86:6819/4080452707","cluster_addr":"172.21.15.86:6821/4080452707","heartbeat_back_addr":"172.21.15.86:6825/4080452707","heartbeat_front_addr":"172.21.15.86:6823/4080452707","state":["exists","up"]},{"osd":7,"uuid":"2684bdae-6b30-4afc-af68-4ce41b3a75b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6817","nonce":4187707622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6819","nonce":4187707622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6823","nonce":4187707622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6821","nonce":4187707622}]},"public_addr":"172.21.15.92:6817/4187707622","cluster_addr":"172.21.15.92:6819/4187707622","heartbeat_back_addr":"172.21.15.92:6823/4187707622","heartbeat_front_addr":"172.21.15.92:6821/4187707622","state":["exists","up"]},{"osd":8,"uuid":"f03bbf0d-1281-4bcf-9773-9854388e65ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6817","nonce":3237612748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6819","nonce":3237612748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6823","nonce":3237612748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6821","nonce":3237612748}]},"public_addr":"172.21.15.152:6817/3237612748","cluster_addr":"172.21.15.152:6819/3237612748","heartbeat_back_addr":"172.21.15.152:6823/3237612748","heartbeat_front_addr":"172.21.15.152:6821/3237612748","state":["exists","up"]},{"osd":9,"uuid":"e7e8b775-99ff-48a4-a23a-91e7ab6622bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6827","nonce":2835218294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6829","nonce":2835218294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6833","nonce":2835218294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6831","nonce":2835218294}]},"public_addr":"172.21.15.86:6827/2835218294","cluster_addr":"172.21.15.86:6829/2835218294","heartbeat_back_addr":"172.21.15.86:6833/2835218294","heartbeat_front_addr":"172.21.15.86:6831/2835218294","state":["exists","up"]},{"osd":10,"uuid":"d9d616cd-a229-44de-8768-02a4415df115","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6825","nonce":4259465100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6827","nonce":4259465100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6831","nonce":4259465100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6829","nonce":4259465100}]},"public_addr":"172.21.15.92:6825/4259465100","cluster_addr":"172.21.15.92:6827/4259465100","heartbeat_back_addr":"172.21.15.92:6831/4259465100","heartbeat_front_addr":"172.21.15.92:6829/4259465100","state":["exists","up"]},{"osd":11,"uuid":"e7c0b865-ccf5-4233-8d22-0d9b6dbe5c97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6825","nonce":194662760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6827","nonce":194662760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6831","nonce":194662760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6829","nonce":194662760}]},"public_addr":"172.21.15.152:6825/194662760","cluster_addr":"172.21.15.152:6827/194662760","heartbeat_back_addr":"172.21.15.152:6831/194662760","heartbeat_front_addr":"172.21.15.152:6829/194662760","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:44:47.269167+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:45:05.363655+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:45:23.192677+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:45:44.156228+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:02.532971+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:20.153295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:41.713888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:59.676516+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:47:17.207087+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:47:38.970316+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:47:57.282525+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:48:15.437606+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/3553500912":"2024-03-24T17:43:28.242492+0000","172.21.15.86:6801/3600278228":"2024-03-24T17:43:28.242492+0000","172.21.15.86:0/887262329":"2024-03-24T17:43:28.242492+0000","172.21.15.86:6800/2313723492":"2024-03-24T17:43:05.043910+0000","172.21.15.86:6800/3600278228":"2024-03-24T17:43:28.242492+0000","172.21.15.86:0/3815705642":"2024-03-24T17:43:05.043910+0000","172.21.15.86:0/1614943062":"2024-03-24T17:43:28.242492+0000","172.21.15.86:6801/2313723492":"2024-03-24T17:43:05.043910+0000","172.21.15.86:0/2416321716":"2024-03-24T17:43:05.043910+0000","172.21.15.86:0/433588188":"2024-03-24T17:43:05.043910+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T17:48:20.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:20.611+0000 7f026fc00700 1 -- 172.21.15.86:0/2188178155 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f02500a11a0 msgr2=0x7f02500a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:20.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:20.611+0000 7f026fc00700 1 --2- 172.21.15.86:0/2188178155 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f02500a11a0 0x7f02500a3660 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f025c011550 tx=0x7f025c009260 comp rx=0 tx=0).stop 2024-03-23T17:48:20.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:20.611+0000 7f026fc00700 1 -- 172.21.15.86:0/2188178155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0268101820 msgr2=0x7f0268197720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:20.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:20.611+0000 7f026fc00700 1 --2- 172.21.15.86:0/2188178155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0268101820 0x7f0268197720 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f02540098e0 tx=0x7f025400b9c0 comp rx=0 tx=0).stop 2024-03-23T17:48:20.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:20.611+0000 7f026fc00700 1 -- 172.21.15.86:0/2188178155 shutdown_connections 2024-03-23T17:48:20.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:20.611+0000 7f026fc00700 1 --2- 172.21.15.86:0/2188178155 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0268106ef0 0x7f026819bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:20.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:20.611+0000 7f026fc00700 1 --2- 172.21.15.86:0/2188178155 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0268100e60 0x7f02681971e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:20.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:20.611+0000 7f026fc00700 1 --2- 172.21.15.86:0/2188178155 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f02500a11a0 0x7f02500a3660 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:20.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:20.611+0000 7f026fc00700 1 --2- 172.21.15.86:0/2188178155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0268101820 0x7f0268197720 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:20.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:20.611+0000 7f026fc00700 1 -- 172.21.15.86:0/2188178155 >> 172.21.15.86:0/2188178155 conn(0x7f0268077fa0 msgr2=0x7f02680ffad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:48:20.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:20.611+0000 7f026fc00700 1 -- 172.21.15.86:0/2188178155 shutdown_connections 2024-03-23T17:48:20.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:20.611+0000 7f026fc00700 1 -- 172.21.15.86:0/2188178155 wait complete. 2024-03-23T17:48:20.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:20 smithi092 ceph-mon[79089]: osdmap e76: 12 total, 12 up, 12 in 2024-03-23T17:48:20.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:20 smithi152 ceph-mon[47949]: osdmap e76: 12 total, 12 up, 12 in 2024-03-23T17:48:21.028 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T17:45:26.848397+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T17:48:21.029 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-03-23T17:48:21.381 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T17:48:21.666 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:21 smithi086 ceph-mon[38221]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T17:48:21.666 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:21 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/2188178155' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T17:48:21.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:21 smithi092 ceph-mon[79089]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T17:48:21.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:21 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/2188178155' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T17:48:21.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:21 smithi152 ceph-mon[47949]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T17:48:21.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:21 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/2188178155' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T17:48:22.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:22.539+0000 7fa0a7200700 1 -- 172.21.15.86:0/3873670885 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0a0073520 msgr2=0x7fa0a00739a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:22.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:22.539+0000 7fa0a7200700 1 --2- 172.21.15.86:0/3873670885 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0a0073520 0x7fa0a00739a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa08c009990 tx=0x7fa08c009ca0 comp rx=0 tx=0).stop 2024-03-23T17:48:22.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:22.539+0000 7fa0a7200700 1 -- 172.21.15.86:0/3873670885 shutdown_connections 2024-03-23T17:48:22.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:22.539+0000 7fa0a7200700 1 --2- 172.21.15.86:0/3873670885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0a0111b30 0x7fa0a0113f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:22.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:22.539+0000 7fa0a7200700 1 --2- 172.21.15.86:0/3873670885 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0a0074d30 0x7fa0a0072fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:22.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:22.539+0000 7fa0a7200700 1 --2- 172.21.15.86:0/3873670885 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0a0073520 0x7fa0a00739a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:22.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:22.539+0000 7fa0a7200700 1 -- 172.21.15.86:0/3873670885 >> 172.21.15.86:0/3873670885 conn(0x7fa0a0077fa0 msgr2=0x7fa0a00783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:48:22.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:22.539+0000 7fa0a7200700 1 -- 172.21.15.86:0/3873670885 shutdown_connections 2024-03-23T17:48:22.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:22.539+0000 7fa0a7200700 1 -- 172.21.15.86:0/3873670885 wait complete. 2024-03-23T17:48:22.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:22.540+0000 7fa0a7200700 1 Processor -- start 2024-03-23T17:48:22.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:22.541+0000 7fa0a7200700 1 -- start start 2024-03-23T17:48:22.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:22.541+0000 7fa0a7200700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0a0073520 0x7fa0a01a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:22.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:22.541+0000 7fa0a7200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0a0074d30 0x7fa0a01a3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:22.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:22.541+0000 7fa0a7200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0a0111b30 0x7fa0a019d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:22.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:22.541+0000 7fa0a7200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a019dd70 con 0x7fa0a0074d30 2024-03-23T17:48:22.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:22.541+0000 7fa0a7200700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a019dee0 con 0x7fa0a0111b30 2024-03-23T17:48:22.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:22.541+0000 7fa0a7200700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a019e0a0 con 0x7fa0a0073520 2024-03-23T17:48:22.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:22.541+0000 7fa09fe00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0a0073520 0x7fa0a01a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:22.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:22.541+0000 7fa09f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0a0074d30 0x7fa0a01a3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:22.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:22.541+0000 7fa09fe00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0a0073520 0x7fa0a01a35d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:48696/0 (socket says 172.21.15.86:48696) 2024-03-23T17:48:22.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:22.541+0000 7fa09fe00700 1 -- 172.21.15.86:0/3873769897 learned_addr learned my addr 172.21.15.86:0/3873769897 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:48:22.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:22.542+0000 7fa0a5200700 1 --2- 172.21.15.86:0/3873769897 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0a0111b30 0x7fa0a019d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:22.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:22.542+0000 7fa09fe00700 1 -- 172.21.15.86:0/3873769897 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0a0111b30 msgr2=0x7fa0a019d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:22.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:22.542+0000 7fa09fe00700 1 --2- 172.21.15.86:0/3873769897 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0a0111b30 0x7fa0a019d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:22.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:22.542+0000 7fa09fe00700 1 -- 172.21.15.86:0/3873769897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0a0074d30 msgr2=0x7fa0a01a3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:22.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:22.542+0000 7fa09fe00700 1 --2- 172.21.15.86:0/3873769897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0a0074d30 0x7fa0a01a3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:22.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:22.542+0000 7fa09fe00700 1 -- 172.21.15.86:0/3873769897 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa098009c40 con 0x7fa0a0073520 2024-03-23T17:48:22.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:22.542+0000 7fa09fe00700 1 --2- 172.21.15.86:0/3873769897 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0a0073520 0x7fa0a01a35d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa09000e9a0 tx=0x7fa09000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:48:22.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:22.542+0000 7fa09cc00700 1 -- 172.21.15.86:0/3873769897 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa090009790 con 0x7fa0a0073520 2024-03-23T17:48:22.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:22.542+0000 7fa09cc00700 1 -- 172.21.15.86:0/3873769897 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa090010430 con 0x7fa0a0073520 2024-03-23T17:48:22.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:22.542+0000 7fa0a7200700 1 -- 172.21.15.86:0/3873769897 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa08c0095f0 con 0x7fa0a0073520 2024-03-23T17:48:22.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:22.542+0000 7fa09cc00700 1 -- 172.21.15.86:0/3873769897 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa090018680 con 0x7fa0a0073520 2024-03-23T17:48:22.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:22.543+0000 7fa0a7200700 1 -- 172.21.15.86:0/3873769897 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0a01aa3f0 con 0x7fa0a0073520 2024-03-23T17:48:22.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:22.543+0000 7fa0a7200700 1 -- 172.21.15.86:0/3873769897 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0a0074670 con 0x7fa0a0073520 2024-03-23T17:48:22.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:22.545+0000 7fa09cc00700 1 -- 172.21.15.86:0/3873769897 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fa0900105a0 con 0x7fa0a0073520 2024-03-23T17:48:22.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:22.547+0000 7fa09cc00700 1 --2- 172.21.15.86:0/3873769897 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fa0840a1120 0x7fa0840a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:22.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:22.547+0000 7fa09cc00700 1 -- 172.21.15.86:0/3873769897 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa090014070 con 0x7fa0a0073520 2024-03-23T17:48:22.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:22.547+0000 7fa09f400700 1 --2- 172.21.15.86:0/3873769897 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fa0840a1120 0x7fa0840a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:22.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:22.548+0000 7fa09f400700 1 --2- 172.21.15.86:0/3873769897 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fa0840a1120 0x7fa0840a35e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa0a019ee20 tx=0x7fa08c00b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:48:22.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:22.549+0000 7fa09cc00700 1 -- 172.21.15.86:0/3873769897 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa0900961e0 con 0x7fa0a0073520 2024-03-23T17:48:22.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:22.749+0000 7fa0a7200700 1 -- 172.21.15.86:0/3873769897 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa0a019ec70 con 0x7fa0a0073520 2024-03-23T17:48:22.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:22.749+0000 7fa09cc00700 1 -- 172.21.15.86:0/3873769897 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v76) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa090099800 con 0x7fa0a0073520 2024-03-23T17:48:22.750 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-03-23T17:48:22.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:22.752+0000 7fa0a7200700 1 -- 172.21.15.86:0/3873769897 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fa0840a1120 msgr2=0x7fa0840a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:22.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:22.752+0000 7fa0a7200700 1 --2- 172.21.15.86:0/3873769897 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fa0840a1120 0x7fa0840a35e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa0a019ee20 tx=0x7fa08c00b260 comp rx=0 tx=0).stop 2024-03-23T17:48:22.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:22.752+0000 7fa0a7200700 1 -- 172.21.15.86:0/3873769897 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0a0073520 msgr2=0x7fa0a01a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:22.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:22.752+0000 7fa0a7200700 1 --2- 172.21.15.86:0/3873769897 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0a0073520 0x7fa0a01a35d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa09000e9a0 tx=0x7fa09000ecb0 comp rx=0 tx=0).stop 2024-03-23T17:48:22.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:22.752+0000 7fa0a7200700 1 -- 172.21.15.86:0/3873769897 shutdown_connections 2024-03-23T17:48:22.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:22.752+0000 7fa0a7200700 1 --2- 172.21.15.86:0/3873769897 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0a0111b30 0x7fa0a019d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:22.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:22.752+0000 7fa0a7200700 1 --2- 172.21.15.86:0/3873769897 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0a0073520 0x7fa0a01a35d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:22.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:22.752+0000 7fa0a7200700 1 --2- 172.21.15.86:0/3873769897 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fa0840a1120 0x7fa0840a35e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:22.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:22.752+0000 7fa0a7200700 1 --2- 172.21.15.86:0/3873769897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0a0074d30 0x7fa0a01a3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:22.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:22.752+0000 7fa0a7200700 1 -- 172.21.15.86:0/3873769897 >> 172.21.15.86:0/3873769897 conn(0x7fa0a0077fa0 msgr2=0x7fa0a0110800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:48:22.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:22.752+0000 7fa0a7200700 1 -- 172.21.15.86:0/3873769897 shutdown_connections 2024-03-23T17:48:22.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:22.752+0000 7fa0a7200700 1 -- 172.21.15.86:0/3873769897 wait complete. 2024-03-23T17:48:23.173 INFO:tasks.cephadm:Adding mds.a on smithi086 2024-03-23T17:48:23.173 INFO:tasks.cephadm:Adding mds.d on smithi086 2024-03-23T17:48:23.173 INFO:tasks.cephadm:Adding mds.g on smithi086 2024-03-23T17:48:23.173 INFO:tasks.cephadm:Adding mds.j on smithi086 2024-03-23T17:48:23.173 INFO:tasks.cephadm:Adding mds.b on smithi092 2024-03-23T17:48:23.173 INFO:tasks.cephadm:Adding mds.e on smithi092 2024-03-23T17:48:23.173 INFO:tasks.cephadm:Adding mds.h on smithi092 2024-03-23T17:48:23.173 INFO:tasks.cephadm:Adding mds.k on smithi092 2024-03-23T17:48:23.173 INFO:tasks.cephadm:Adding mds.c on smithi152 2024-03-23T17:48:23.173 INFO:tasks.cephadm:Adding mds.f on smithi152 2024-03-23T17:48:23.173 INFO:tasks.cephadm:Adding mds.i on smithi152 2024-03-23T17:48:23.173 INFO:tasks.cephadm:Adding mds.l on smithi152 2024-03-23T17:48:23.174 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph orch apply mds all '12;smithi086=a;smithi086=d;smithi086=g;smithi086=j;smithi092=b;smithi092=e;smithi092=h;smithi092=k;smithi152=c;smithi152=f;smithi152=i;smithi152=l' 2024-03-23T17:48:23.409 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.c/config 2024-03-23T17:48:23.519 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:23 smithi152 ceph-mon[47949]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T17:48:23.519 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:23 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/3873769897' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T17:48:23.766 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:23 smithi086 ceph-mon[38221]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T17:48:23.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:23 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/3873769897' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T17:48:23.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:23 smithi092 ceph-mon[79089]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T17:48:23.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:23 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/3873769897' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T17:48:24.526 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:48:24.523+0000 7fd34859c700 1 -- 172.21.15.152:0/742041440 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3400ffcf0 msgr2=0x7fd340100110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:24.526 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:48:24.523+0000 7fd34859c700 1 --2- 172.21.15.152:0/742041440 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3400ffcf0 0x7fd340100110 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd330009990 tx=0x7fd330009ca0 comp rx=0 tx=0).stop 2024-03-23T17:48:24.526 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:48:24.523+0000 7fd34859c700 1 -- 172.21.15.152:0/742041440 shutdown_connections 2024-03-23T17:48:24.526 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:48:24.523+0000 7fd34859c700 1 --2- 172.21.15.152:0/742041440 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd340101890 0x7fd340108140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:24.526 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:48:24.523+0000 7fd34859c700 1 --2- 172.21.15.152:0/742041440 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3400ffcf0 0x7fd340100110 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:24.526 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:48:24.523+0000 7fd34859c700 1 --2- 172.21.15.152:0/742041440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd340100ef0 0x7fd340101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:24.526 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:48:24.523+0000 7fd34859c700 1 -- 172.21.15.152:0/742041440 >> 172.21.15.152:0/742041440 conn(0x7fd3400fb2b0 msgr2=0x7fd3400fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:48:24.526 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:48:24.523+0000 7fd34859c700 1 -- 172.21.15.152:0/742041440 shutdown_connections 2024-03-23T17:48:24.526 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:48:24.523+0000 7fd34859c700 1 -- 172.21.15.152:0/742041440 wait complete. 2024-03-23T17:48:24.526 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:48:24.523+0000 7fd34859c700 1 Processor -- start 2024-03-23T17:48:24.526 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:48:24.523+0000 7fd34859c700 1 -- start start 2024-03-23T17:48:24.527 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:48:24.523+0000 7fd34859c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3400ffcf0 0x7fd340103c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:24.527 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:48:24.524+0000 7fd34859c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd340100ef0 0x7fd3401022d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:24.527 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:48:24.524+0000 7fd34859c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd340101890 0x7fd340102810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:24.527 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:48:24.524+0000 7fd34859c700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd340102fb0 con 0x7fd340100ef0 2024-03-23T17:48:24.527 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:48:24.524+0000 7fd34859c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd340103120 con 0x7fd340101890 2024-03-23T17:48:24.527 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:48:24.524+0000 7fd34859c700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3401a0d20 con 0x7fd3400ffcf0 2024-03-23T17:48:24.527 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:48:24.524+0000 7fd346338700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3400ffcf0 0x7fd340103c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:24.527 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:48:24.524+0000 7fd346338700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3400ffcf0 0x7fd340103c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:39384/0 (socket says 172.21.15.152:39384) 2024-03-23T17:48:24.527 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:48:24.524+0000 7fd346338700 1 -- 172.21.15.152:0/1808301011 learned_addr learned my addr 172.21.15.152:0/1808301011 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-03-23T17:48:24.527 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:48:24.524+0000 7fd346b39700 1 --2- 172.21.15.152:0/1808301011 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd340101890 0x7fd340102810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:24.527 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:48:24.524+0000 7fd345b37700 1 --2- 172.21.15.152:0/1808301011 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd340100ef0 0x7fd3401022d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:24.527 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:48:24.524+0000 7fd346338700 1 -- 172.21.15.152:0/1808301011 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd340101890 msgr2=0x7fd340102810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:24.527 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:48:24.524+0000 7fd346338700 1 --2- 172.21.15.152:0/1808301011 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd340101890 0x7fd340102810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:24.528 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:48:24.524+0000 7fd346338700 1 -- 172.21.15.152:0/1808301011 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd340100ef0 msgr2=0x7fd3401022d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:24.528 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:48:24.524+0000 7fd346338700 1 --2- 172.21.15.152:0/1808301011 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd340100ef0 0x7fd3401022d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:24.528 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:48:24.524+0000 7fd346338700 1 -- 172.21.15.152:0/1808301011 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3300095f0 con 0x7fd3400ffcf0 2024-03-23T17:48:24.528 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:48:24.524+0000 7fd346b39700 1 --2- 172.21.15.152:0/1808301011 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd340101890 0x7fd340102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T17:48:24.528 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:48:24.524+0000 7fd345b37700 1 --2- 172.21.15.152:0/1808301011 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd340100ef0 0x7fd3401022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T17:48:24.528 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:48:24.524+0000 7fd346338700 1 --2- 172.21.15.152:0/1808301011 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3400ffcf0 0x7fd340103c30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd330000c00 tx=0x7fd330005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:48:24.528 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:48:24.524+0000 7fd3377fe700 1 -- 172.21.15.152:0/1808301011 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd33001d070 con 0x7fd3400ffcf0 2024-03-23T17:48:24.528 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:48:24.524+0000 7fd34859c700 1 -- 172.21.15.152:0/1808301011 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3401a0f50 con 0x7fd3400ffcf0 2024-03-23T17:48:24.528 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:48:24.524+0000 7fd3377fe700 1 -- 172.21.15.152:0/1808301011 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd33000fc80 con 0x7fd3400ffcf0 2024-03-23T17:48:24.528 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:48:24.524+0000 7fd3377fe700 1 -- 172.21.15.152:0/1808301011 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd330017650 con 0x7fd3400ffcf0 2024-03-23T17:48:24.528 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:48:24.524+0000 7fd34859c700 1 -- 172.21.15.152:0/1808301011 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3401a1300 con 0x7fd3400ffcf0 2024-03-23T17:48:24.528 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:48:24.525+0000 7fd34859c700 1 -- 172.21.15.152:0/1808301011 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd34004ea80 con 0x7fd3400ffcf0 2024-03-23T17:48:24.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:48:24.526+0000 7fd3377fe700 1 -- 172.21.15.152:0/1808301011 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fd330021410 con 0x7fd3400ffcf0 2024-03-23T17:48:24.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:48:24.527+0000 7fd3377fe700 1 --2- 172.21.15.152:0/1808301011 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fd32c0a1120 0x7fd32c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:24.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:48:24.527+0000 7fd3377fe700 1 -- 172.21.15.152:0/1808301011 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd3300cd9b0 con 0x7fd3400ffcf0 2024-03-23T17:48:24.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:48:24.527+0000 7fd345b37700 1 --2- 172.21.15.152:0/1808301011 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fd32c0a1120 0x7fd32c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:24.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:48:24.528+0000 7fd345b37700 1 --2- 172.21.15.152:0/1808301011 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fd32c0a1120 0x7fd32c0a35e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd33c005fd0 tx=0x7fd33c005e20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:48:24.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:48:24.531+0000 7fd3377fe700 1 -- 172.21.15.152:0/1808301011 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fd3300975d0 con 0x7fd3400ffcf0 2024-03-23T17:48:24.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:48:24.732+0000 7fd34859c700 1 -- 172.21.15.152:0/1808301011 --> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi086=a;smithi086=d;smithi086=g;smithi086=j;smithi092=b;smithi092=e;smithi092=h;smithi092=k;smithi152=c;smithi152=f;smithi152=i;smithi152=l", "target": ["mon-mgr", ""]}) v1 -- 0x7fd3400fd5b0 con 0x7fd32c0a1120 2024-03-23T17:48:24.743 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:48:24.743+0000 7fd3377fe700 1 -- 172.21.15.152:0/1808301011 <== mgr.14152 v2:172.21.15.86:6800/2734784244 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7fd3400fd5b0 con 0x7fd32c0a1120 2024-03-23T17:48:24.820 INFO:teuthology.orchestra.run.smithi152.stdout:Scheduled mds.all update... 2024-03-23T17:48:24.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:48:24.745+0000 7fd34859c700 1 -- 172.21.15.152:0/1808301011 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fd32c0a1120 msgr2=0x7fd32c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:24.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:48:24.745+0000 7fd34859c700 1 --2- 172.21.15.152:0/1808301011 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fd32c0a1120 0x7fd32c0a35e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd33c005fd0 tx=0x7fd33c005e20 comp rx=0 tx=0).stop 2024-03-23T17:48:24.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:48:24.745+0000 7fd34859c700 1 -- 172.21.15.152:0/1808301011 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3400ffcf0 msgr2=0x7fd340103c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:24.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:48:24.745+0000 7fd34859c700 1 --2- 172.21.15.152:0/1808301011 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3400ffcf0 0x7fd340103c30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd330000c00 tx=0x7fd330005e80 comp rx=0 tx=0).stop 2024-03-23T17:48:24.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:48:24.745+0000 7fd34859c700 1 -- 172.21.15.152:0/1808301011 shutdown_connections 2024-03-23T17:48:24.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:48:24.745+0000 7fd34859c700 1 --2- 172.21.15.152:0/1808301011 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd340101890 0x7fd340102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:24.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:48:24.745+0000 7fd34859c700 1 --2- 172.21.15.152:0/1808301011 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3400ffcf0 0x7fd340103c30 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:24.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:48:24.745+0000 7fd34859c700 1 --2- 172.21.15.152:0/1808301011 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fd32c0a1120 0x7fd32c0a35e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:24.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:48:24.745+0000 7fd34859c700 1 --2- 172.21.15.152:0/1808301011 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd340100ef0 0x7fd3401022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:24.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:48:24.745+0000 7fd34859c700 1 -- 172.21.15.152:0/1808301011 >> 172.21.15.152:0/1808301011 conn(0x7fd3400fb2b0 msgr2=0x7fd3400fcda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:48:24.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:48:24.745+0000 7fd34859c700 1 -- 172.21.15.152:0/1808301011 shutdown_connections 2024-03-23T17:48:24.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-03-23T17:48:24.745+0000 7fd34859c700 1 -- 172.21.15.152:0/1808301011 wait complete. 2024-03-23T17:48:25.122 DEBUG:teuthology.orchestra.run.smithi086:mds.a> sudo journalctl -f -n 0 -u ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@mds.a.service 2024-03-23T17:48:25.127 DEBUG:teuthology.orchestra.run.smithi086:mds.d> sudo journalctl -f -n 0 -u ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@mds.d.service 2024-03-23T17:48:25.171 DEBUG:teuthology.orchestra.run.smithi086:mds.g> sudo journalctl -f -n 0 -u ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@mds.g.service 2024-03-23T17:48:25.195 INFO:journalctl@ceph.mds.a.smithi086.stdout:-- Logs begin at Sat 2024-03-23 17:25:35 UTC. -- 2024-03-23T17:48:25.199 DEBUG:teuthology.orchestra.run.smithi086:mds.j> sudo journalctl -f -n 0 -u ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@mds.j.service 2024-03-23T17:48:25.238 INFO:journalctl@ceph.mds.d.smithi086.stdout:-- Logs begin at Sat 2024-03-23 17:25:35 UTC. -- 2024-03-23T17:48:25.241 DEBUG:teuthology.orchestra.run.smithi092:mds.b> sudo journalctl -f -n 0 -u ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@mds.b.service 2024-03-23T17:48:25.245 DEBUG:teuthology.orchestra.run.smithi092:mds.e> sudo journalctl -f -n 0 -u ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@mds.e.service 2024-03-23T17:48:25.264 INFO:journalctl@ceph.mds.g.smithi086.stdout:-- Logs begin at Sat 2024-03-23 17:25:35 UTC. -- 2024-03-23T17:48:25.283 INFO:journalctl@ceph.mds.b.smithi092.stdout:-- Logs begin at Sat 2024-03-23 17:19:16 UTC. -- 2024-03-23T17:48:25.285 DEBUG:teuthology.orchestra.run.smithi092:mds.h> sudo journalctl -f -n 0 -u ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@mds.h.service 2024-03-23T17:48:25.306 INFO:journalctl@ceph.mds.j.smithi086.stdout:-- Logs begin at Sat 2024-03-23 17:25:35 UTC. -- 2024-03-23T17:48:25.317 INFO:journalctl@ceph.mds.e.smithi092.stdout:-- Logs begin at Sat 2024-03-23 17:19:16 UTC. -- 2024-03-23T17:48:25.318 DEBUG:teuthology.orchestra.run.smithi092:mds.k> sudo journalctl -f -n 0 -u ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@mds.k.service 2024-03-23T17:48:25.351 INFO:journalctl@ceph.mds.h.smithi092.stdout:-- Logs begin at Sat 2024-03-23 17:19:16 UTC. -- 2024-03-23T17:48:25.352 DEBUG:teuthology.orchestra.run.smithi152:mds.c> sudo journalctl -f -n 0 -u ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@mds.c.service 2024-03-23T17:48:25.355 DEBUG:teuthology.orchestra.run.smithi152:mds.f> sudo journalctl -f -n 0 -u ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@mds.f.service 2024-03-23T17:48:25.384 INFO:journalctl@ceph.mds.k.smithi092.stdout:-- Logs begin at Sat 2024-03-23 17:19:16 UTC. -- 2024-03-23T17:48:25.385 INFO:journalctl@ceph.mds.c.smithi152.stdout:-- Logs begin at Sat 2024-03-23 17:19:34 UTC. -- 2024-03-23T17:48:25.387 DEBUG:teuthology.orchestra.run.smithi152:mds.i> sudo journalctl -f -n 0 -u ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@mds.i.service 2024-03-23T17:48:25.413 INFO:journalctl@ceph.mds.f.smithi152.stdout:-- Logs begin at Sat 2024-03-23 17:19:34 UTC. -- 2024-03-23T17:48:25.414 DEBUG:teuthology.orchestra.run.smithi152:mds.l> sudo journalctl -f -n 0 -u ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@mds.l.service 2024-03-23T17:48:25.439 INFO:journalctl@ceph.mds.i.smithi152.stdout:-- Logs begin at Sat 2024-03-23 17:19:34 UTC. -- 2024-03-23T17:48:25.440 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-03-23T17:48:25.441 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T17:48:25.466 INFO:journalctl@ceph.mds.l.smithi152.stdout:-- Logs begin at Sat 2024-03-23 17:19:34 UTC. -- 2024-03-23T17:48:25.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:25.648+0000 7f27ea800700 1 -- 172.21.15.86:0/1385998794 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f27e4073810 msgr2=0x7f27e4073c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:25.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:25.648+0000 7f27ea800700 1 --2- 172.21.15.86:0/1385998794 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f27e4073810 0x7f27e4073c90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f27d0009990 tx=0x7f27d0009ca0 comp rx=0 tx=0).stop 2024-03-23T17:48:25.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:25.648+0000 7f27ea800700 1 -- 172.21.15.86:0/1385998794 shutdown_connections 2024-03-23T17:48:25.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:25.648+0000 7f27ea800700 1 --2- 172.21.15.86:0/1385998794 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27e4108b40 0x7f27e410afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:25.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:25.648+0000 7f27ea800700 1 --2- 172.21.15.86:0/1385998794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27e4074df0 0x7f27e4073240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:25.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:25.648+0000 7f27ea800700 1 --2- 172.21.15.86:0/1385998794 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f27e4073810 0x7f27e4073c90 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:25.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:25.648+0000 7f27ea800700 1 -- 172.21.15.86:0/1385998794 >> 172.21.15.86:0/1385998794 conn(0x7f27e40fc1f0 msgr2=0x7f27e40fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:48:25.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:25.648+0000 7f27ea800700 1 -- 172.21.15.86:0/1385998794 shutdown_connections 2024-03-23T17:48:25.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:25.648+0000 7f27ea800700 1 -- 172.21.15.86:0/1385998794 wait complete. 2024-03-23T17:48:25.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:25.649+0000 7f27ea800700 1 Processor -- start 2024-03-23T17:48:25.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:25.649+0000 7f27ea800700 1 -- start start 2024-03-23T17:48:25.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:25.650+0000 7f27ea800700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f27e4073810 0x7f27e4198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:25.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:25.650+0000 7f27ea800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27e4074df0 0x7f27e41993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:25.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:25.650+0000 7f27ea800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27e4108b40 0x7f27e41a03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:25.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:25.650+0000 7f27ea800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e41a0980 con 0x7f27e4074df0 2024-03-23T17:48:25.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:25.650+0000 7f27ea800700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e41a0ac0 con 0x7f27e4108b40 2024-03-23T17:48:25.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:25.650+0000 7f27ea800700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e41a0c50 con 0x7f27e4073810 2024-03-23T17:48:25.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:25.650+0000 7f27e3400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f27e4073810 0x7f27e4198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:25.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:25.650+0000 7f27e2a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27e4074df0 0x7f27e41993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:25.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:25.650+0000 7f27e3400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f27e4073810 0x7f27e4198ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:57014/0 (socket says 172.21.15.86:57014) 2024-03-23T17:48:25.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:25.650+0000 7f27e3400700 1 -- 172.21.15.86:0/3273346792 learned_addr learned my addr 172.21.15.86:0/3273346792 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:48:25.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:25.650+0000 7f27e3e00700 1 --2- 172.21.15.86:0/3273346792 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27e4108b40 0x7f27e41a03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:25.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:25.650+0000 7f27e3400700 1 -- 172.21.15.86:0/3273346792 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27e4108b40 msgr2=0x7f27e41a03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:25.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:25.650+0000 7f27e3400700 1 --2- 172.21.15.86:0/3273346792 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27e4108b40 0x7f27e41a03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:25.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:25.650+0000 7f27e3400700 1 -- 172.21.15.86:0/3273346792 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27e4074df0 msgr2=0x7f27e41993e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:25.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:25.650+0000 7f27e3400700 1 --2- 172.21.15.86:0/3273346792 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27e4074df0 0x7f27e41993e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:25.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:25.650+0000 7f27e3400700 1 -- 172.21.15.86:0/3273346792 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27d00095f0 con 0x7f27e4073810 2024-03-23T17:48:25.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:25.651+0000 7f27e3400700 1 --2- 172.21.15.86:0/3273346792 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f27e4073810 0x7f27e4198ea0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f27d400e9a0 tx=0x7f27d400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:48:25.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:25.651+0000 7f27dbe00700 1 -- 172.21.15.86:0/3273346792 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d4009790 con 0x7f27e4073810 2024-03-23T17:48:25.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:25.651+0000 7f27dbe00700 1 -- 172.21.15.86:0/3273346792 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27d4010430 con 0x7f27e4073810 2024-03-23T17:48:25.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:25.651+0000 7f27ea800700 1 -- 172.21.15.86:0/3273346792 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27e41a0f30 con 0x7f27e4073810 2024-03-23T17:48:25.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:25.651+0000 7f27dbe00700 1 -- 172.21.15.86:0/3273346792 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d4018680 con 0x7f27e4073810 2024-03-23T17:48:25.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:25.651+0000 7f27ea800700 1 -- 172.21.15.86:0/3273346792 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27e41a1360 con 0x7f27e4073810 2024-03-23T17:48:25.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:25.652+0000 7f27ea800700 1 -- 172.21.15.86:0/3273346792 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27e4107900 con 0x7f27e4073810 2024-03-23T17:48:25.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:25.654+0000 7f27dbe00700 1 -- 172.21.15.86:0/3273346792 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f27d40105a0 con 0x7f27e4073810 2024-03-23T17:48:25.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:25.655+0000 7f27dbe00700 1 --2- 172.21.15.86:0/3273346792 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f27c80a11a0 0x7f27c80a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:25.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:25.655+0000 7f27dbe00700 1 -- 172.21.15.86:0/3273346792 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f27d4014070 con 0x7f27e4073810 2024-03-23T17:48:25.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:25.655+0000 7f27e2a00700 1 --2- 172.21.15.86:0/3273346792 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f27c80a11a0 0x7f27c80a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:25.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:25.656+0000 7f27e2a00700 1 --2- 172.21.15.86:0/3273346792 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f27c80a11a0 0x7f27c80a3660 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f27d000b680 tx=0x7f27d0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:48:25.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:25.659+0000 7f27dbe00700 1 -- 172.21.15.86:0/3273346792 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f27d40529f0 con 0x7f27e4073810 2024-03-23T17:48:25.766 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:25 smithi086 ceph-mon[38221]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T17:48:25.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:25 smithi086 ceph-mon[38221]: from='client.24451 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi086=a;smithi086=d;smithi086=g;smithi086=j;smithi092=b;smithi092=e;smithi092=h;smithi092=k;smithi152=c;smithi152=f;smithi152=i;smithi152=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:48:25.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:25 smithi086 ceph-mon[38221]: Saving service mds.all spec with placement smithi086=a;smithi086=d;smithi086=g;smithi086=j;smithi092=b;smithi092=e;smithi092=h;smithi092=k;smithi152=c;smithi152=f;smithi152=i;smithi152=l;count:12 2024-03-23T17:48:25.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:25 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:25.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:25 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:48:25.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:25 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:48:25.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:25 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:48:25.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:25 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:25.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:25 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T17:48:25.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:25 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T17:48:25.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:25 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:48:25.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:25.904+0000 7f27ea800700 1 -- 172.21.15.86:0/3273346792 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f27e4067a10 con 0x7f27e4073810 2024-03-23T17:48:25.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:25.905+0000 7f27dbe00700 1 -- 172.21.15.86:0/3273346792 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+18386 (secure 0 0 0) 0x7f27d40961a0 con 0x7f27e4073810 2024-03-23T17:48:25.906 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:48:25.906 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":76,"fsid":"9d76c9e8-e93c-11ee-95cd-87774f69a715","created":"2024-03-23T17:42:30.097233+0000","modified":"2024-03-23T17:48:19.377461+0000","last_up_change":"2024-03-23T17:48:18.217601+0000","last_in_change":"2024-03-23T17:48:00.250579+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T17:45:26.848397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"656655c6-1c7f-4640-92c4-2f42de820143","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6803","nonce":2423505491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6805","nonce":2423505491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6809","nonce":2423505491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6807","nonce":2423505491}]},"public_addr":"172.21.15.86:6803/2423505491","cluster_addr":"172.21.15.86:6805/2423505491","heartbeat_back_addr":"172.21.15.86:6809/2423505491","heartbeat_front_addr":"172.21.15.86:6807/2423505491","state":["exists","up"]},{"osd":1,"uuid":"3e86dc65-20a6-4de8-b9ce-fcf01f68862b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6801","nonce":3776252597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6803","nonce":3776252597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6807","nonce":3776252597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6805","nonce":3776252597}]},"public_addr":"172.21.15.92:6801/3776252597","cluster_addr":"172.21.15.92:6803/3776252597","heartbeat_back_addr":"172.21.15.92:6807/3776252597","heartbeat_front_addr":"172.21.15.92:6805/3776252597","state":["exists","up"]},{"osd":2,"uuid":"09808786-3bd3-4d11-8c7b-1736119dfbb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6801","nonce":1601265891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6803","nonce":1601265891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6807","nonce":1601265891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6805","nonce":1601265891}]},"public_addr":"172.21.15.152:6801/1601265891","cluster_addr":"172.21.15.152:6803/1601265891","heartbeat_back_addr":"172.21.15.152:6807/1601265891","heartbeat_front_addr":"172.21.15.152:6805/1601265891","state":["exists","up"]},{"osd":3,"uuid":"79c61910-4b53-424c-b150-6f5385a2cd68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6811","nonce":694846466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6813","nonce":694846466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6817","nonce":694846466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6815","nonce":694846466}]},"public_addr":"172.21.15.86:6811/694846466","cluster_addr":"172.21.15.86:6813/694846466","heartbeat_back_addr":"172.21.15.86:6817/694846466","heartbeat_front_addr":"172.21.15.86:6815/694846466","state":["exists","up"]},{"osd":4,"uuid":"a2878035-0179-469f-bbd4-8ce2d3f6bddc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6809","nonce":1422821471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6811","nonce":1422821471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6815","nonce":1422821471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6813","nonce":1422821471}]},"public_addr":"172.21.15.92:6809/1422821471","cluster_addr":"172.21.15.92:6811/1422821471","heartbeat_back_addr":"172.21.15.92:6815/1422821471","heartbeat_front_addr":"172.21.15.92:6813/1422821471","state":["exists","up"]},{"osd":5,"uuid":"837366aa-ce25-42ca-b157-5bbb50159d0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6809","nonce":1039375239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6811","nonce":1039375239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6815","nonce":1039375239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6813","nonce":1039375239}]},"public_addr":"172.21.15.152:6809/1039375239","cluster_addr":"172.21.15.152:6811/1039375239","heartbeat_back_addr":"172.21.15.152:6815/1039375239","heartbeat_front_addr":"172.21.15.152:6813/1039375239","state":["exists","up"]},{"osd":6,"uuid":"61024437-7b91-432f-a1af-6443d0ef5213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6819","nonce":4080452707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6821","nonce":4080452707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6825","nonce":4080452707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6823","nonce":4080452707}]},"public_addr":"172.21.15.86:6819/4080452707","cluster_addr":"172.21.15.86:6821/4080452707","heartbeat_back_addr":"172.21.15.86:6825/4080452707","heartbeat_front_addr":"172.21.15.86:6823/4080452707","state":["exists","up"]},{"osd":7,"uuid":"2684bdae-6b30-4afc-af68-4ce41b3a75b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6817","nonce":4187707622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6819","nonce":4187707622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6823","nonce":4187707622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6821","nonce":4187707622}]},"public_addr":"172.21.15.92:6817/4187707622","cluster_addr":"172.21.15.92:6819/4187707622","heartbeat_back_addr":"172.21.15.92:6823/4187707622","heartbeat_front_addr":"172.21.15.92:6821/4187707622","state":["exists","up"]},{"osd":8,"uuid":"f03bbf0d-1281-4bcf-9773-9854388e65ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6817","nonce":3237612748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6819","nonce":3237612748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6823","nonce":3237612748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6821","nonce":3237612748}]},"public_addr":"172.21.15.152:6817/3237612748","cluster_addr":"172.21.15.152:6819/3237612748","heartbeat_back_addr":"172.21.15.152:6823/3237612748","heartbeat_front_addr":"172.21.15.152:6821/3237612748","state":["exists","up"]},{"osd":9,"uuid":"e7e8b775-99ff-48a4-a23a-91e7ab6622bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6827","nonce":2835218294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6829","nonce":2835218294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6833","nonce":2835218294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6831","nonce":2835218294}]},"public_addr":"172.21.15.86:6827/2835218294","cluster_addr":"172.21.15.86:6829/2835218294","heartbeat_back_addr":"172.21.15.86:6833/2835218294","heartbeat_front_addr":"172.21.15.86:6831/2835218294","state":["exists","up"]},{"osd":10,"uuid":"d9d616cd-a229-44de-8768-02a4415df115","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6825","nonce":4259465100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6827","nonce":4259465100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6831","nonce":4259465100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6829","nonce":4259465100}]},"public_addr":"172.21.15.92:6825/4259465100","cluster_addr":"172.21.15.92:6827/4259465100","heartbeat_back_addr":"172.21.15.92:6831/4259465100","heartbeat_front_addr":"172.21.15.92:6829/4259465100","state":["exists","up"]},{"osd":11,"uuid":"e7c0b865-ccf5-4233-8d22-0d9b6dbe5c97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6825","nonce":194662760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6827","nonce":194662760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6831","nonce":194662760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6829","nonce":194662760}]},"public_addr":"172.21.15.152:6825/194662760","cluster_addr":"172.21.15.152:6827/194662760","heartbeat_back_addr":"172.21.15.152:6831/194662760","heartbeat_front_addr":"172.21.15.152:6829/194662760","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:44:47.269167+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:45:05.363655+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:45:23.192677+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:45:44.156228+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:02.532971+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:20.153295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:41.713888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:59.676516+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:47:17.207087+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:47:38.970316+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:47:57.282525+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:48:15.437606+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/3553500912":"2024-03-24T17:43:28.242492+0000","172.21.15.86:6801/3600278228":"2024-03-24T17:43:28.242492+0000","172.21.15.86:0/887262329":"2024-03-24T17:43:28.242492+0000","172.21.15.86:6800/2313723492":"2024-03-24T17:43:05.043910+0000","172.21.15.86:6800/3600278228":"2024-03-24T17:43:28.242492+0000","172.21.15.86:0/3815705642":"2024-03-24T17:43:05.043910+0000","172.21.15.86:0/1614943062":"2024-03-24T17:43:28.242492+0000","172.21.15.86:6801/2313723492":"2024-03-24T17:43:05.043910+0000","172.21.15.86:0/2416321716":"2024-03-24T17:43:05.043910+0000","172.21.15.86:0/433588188":"2024-03-24T17:43:05.043910+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T17:48:25.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:25.908+0000 7f27ea800700 1 -- 172.21.15.86:0/3273346792 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f27c80a11a0 msgr2=0x7f27c80a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:25.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:25.908+0000 7f27ea800700 1 --2- 172.21.15.86:0/3273346792 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f27c80a11a0 0x7f27c80a3660 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f27d000b680 tx=0x7f27d0005fd0 comp rx=0 tx=0).stop 2024-03-23T17:48:25.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:25.908+0000 7f27ea800700 1 -- 172.21.15.86:0/3273346792 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f27e4073810 msgr2=0x7f27e4198ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:25.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:25.908+0000 7f27ea800700 1 --2- 172.21.15.86:0/3273346792 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f27e4073810 0x7f27e4198ea0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f27d400e9a0 tx=0x7f27d400ecb0 comp rx=0 tx=0).stop 2024-03-23T17:48:25.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:25.908+0000 7f27ea800700 1 -- 172.21.15.86:0/3273346792 shutdown_connections 2024-03-23T17:48:25.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:25.908+0000 7f27ea800700 1 --2- 172.21.15.86:0/3273346792 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27e4108b40 0x7f27e41a03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:25.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:25.908+0000 7f27ea800700 1 --2- 172.21.15.86:0/3273346792 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f27e4073810 0x7f27e4198ea0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:25.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:25.908+0000 7f27ea800700 1 --2- 172.21.15.86:0/3273346792 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f27c80a11a0 0x7f27c80a3660 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:25.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:25.908+0000 7f27ea800700 1 --2- 172.21.15.86:0/3273346792 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27e4074df0 0x7f27e41993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:25.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:25.908+0000 7f27ea800700 1 -- 172.21.15.86:0/3273346792 >> 172.21.15.86:0/3273346792 conn(0x7f27e40fc1f0 msgr2=0x7f27e4104a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:48:25.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:25.908+0000 7f27ea800700 1 -- 172.21.15.86:0/3273346792 shutdown_connections 2024-03-23T17:48:25.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:25.908+0000 7f27ea800700 1 -- 172.21.15.86:0/3273346792 wait complete. 2024-03-23T17:48:25.928 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T17:45:26.848397+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T17:48:25.928 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T17:48:26.033 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:25 smithi152 ceph-mon[47949]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T17:48:26.033 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:25 smithi152 ceph-mon[47949]: from='client.24451 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi086=a;smithi086=d;smithi086=g;smithi086=j;smithi092=b;smithi092=e;smithi092=h;smithi092=k;smithi152=c;smithi152=f;smithi152=i;smithi152=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:48:26.033 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:25 smithi152 ceph-mon[47949]: Saving service mds.all spec with placement smithi086=a;smithi086=d;smithi086=g;smithi086=j;smithi092=b;smithi092=e;smithi092=h;smithi092=k;smithi152=c;smithi152=f;smithi152=i;smithi152=l;count:12 2024-03-23T17:48:26.034 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:25 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:26.034 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:25 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:48:26.034 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:25 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:48:26.034 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:25 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:48:26.034 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:25 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:26.034 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:25 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T17:48:26.034 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:25 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T17:48:26.034 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:25 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:48:26.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:25 smithi092 ceph-mon[79089]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T17:48:26.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:25 smithi092 ceph-mon[79089]: from='client.24451 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi086=a;smithi086=d;smithi086=g;smithi086=j;smithi092=b;smithi092=e;smithi092=h;smithi092=k;smithi152=c;smithi152=f;smithi152=i;smithi152=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:48:26.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:25 smithi092 ceph-mon[79089]: Saving service mds.all spec with placement smithi086=a;smithi086=d;smithi086=g;smithi086=j;smithi092=b;smithi092=e;smithi092=h;smithi092=k;smithi152=c;smithi152=f;smithi152=i;smithi152=l;count:12 2024-03-23T17:48:26.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:25 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:26.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:25 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:48:26.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:25 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:48:26.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:25 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:48:26.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:25 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:26.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:25 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T17:48:26.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:25 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T17:48:26.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:25 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:48:26.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.124+0000 7f11a3800700 1 -- 172.21.15.86:0/1082833732 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f119c100d80 msgr2=0x7f119c1011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:26.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.124+0000 7f11a3800700 1 --2- 172.21.15.86:0/1082833732 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f119c100d80 0x7f119c1011a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f11900098f0 tx=0x7f1190009c00 comp rx=0 tx=0).stop 2024-03-23T17:48:26.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.124+0000 7f11a3800700 1 -- 172.21.15.86:0/1082833732 shutdown_connections 2024-03-23T17:48:26.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.124+0000 7f11a3800700 1 --2- 172.21.15.86:0/1082833732 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f119c102910 0x7f119c1091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:26.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.124+0000 7f11a3800700 1 --2- 172.21.15.86:0/1082833732 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f119c100d80 0x7f119c1011a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:26.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.124+0000 7f11a3800700 1 --2- 172.21.15.86:0/1082833732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f119c101f70 0x7f119c1023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:26.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.124+0000 7f11a3800700 1 -- 172.21.15.86:0/1082833732 >> 172.21.15.86:0/1082833732 conn(0x7f119c0fc2f0 msgr2=0x7f119c0fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:48:26.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.124+0000 7f11a3800700 1 -- 172.21.15.86:0/1082833732 shutdown_connections 2024-03-23T17:48:26.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.124+0000 7f11a3800700 1 -- 172.21.15.86:0/1082833732 wait complete. 2024-03-23T17:48:26.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.125+0000 7f11a3800700 1 Processor -- start 2024-03-23T17:48:26.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.125+0000 7f11a3800700 1 -- start start 2024-03-23T17:48:26.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.125+0000 7f11a3800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f119c100d80 0x7f119c198fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:26.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.126+0000 7f11a3800700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f119c101f70 0x7f119c199500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:26.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.126+0000 7f11a3800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f119c102910 0x7f119c1a0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:26.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.126+0000 7f11a3800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f119c1a0af0 con 0x7f119c100d80 2024-03-23T17:48:26.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.126+0000 7f11a3800700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f119c1a0c30 con 0x7f119c102910 2024-03-23T17:48:26.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.126+0000 7f11a3800700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f119c1a0dc0 con 0x7f119c101f70 2024-03-23T17:48:26.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.126+0000 7f11a0e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f119c100d80 0x7f119c198fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:26.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.126+0000 7f119be00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f119c101f70 0x7f119c199500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:26.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.126+0000 7f11a0e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f119c100d80 0x7f119c198fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59250/0 (socket says 172.21.15.86:59250) 2024-03-23T17:48:26.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.126+0000 7f11a0e00700 1 -- 172.21.15.86:0/103884813 learned_addr learned my addr 172.21.15.86:0/103884813 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:48:26.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.126+0000 7f11a1800700 1 --2- 172.21.15.86:0/103884813 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f119c102910 0x7f119c1a0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:26.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.126+0000 7f11a0e00700 1 -- 172.21.15.86:0/103884813 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f119c101f70 msgr2=0x7f119c199500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:26.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.126+0000 7f11a0e00700 1 --2- 172.21.15.86:0/103884813 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f119c101f70 0x7f119c199500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:26.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.126+0000 7f11a0e00700 1 -- 172.21.15.86:0/103884813 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f119c102910 msgr2=0x7f119c1a0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:26.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.126+0000 7f11a0e00700 1 --2- 172.21.15.86:0/103884813 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f119c102910 0x7f119c1a0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:26.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.126+0000 7f11a0e00700 1 -- 172.21.15.86:0/103884813 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1190009520 con 0x7f119c100d80 2024-03-23T17:48:26.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.126+0000 7f119be00700 1 --2- 172.21.15.86:0/103884813 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f119c101f70 0x7f119c199500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:48:26.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.126+0000 7f11a1800700 1 --2- 172.21.15.86:0/103884813 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f119c102910 0x7f119c1a0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:48:26.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.126+0000 7f11a0e00700 1 --2- 172.21.15.86:0/103884813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f119c100d80 0x7f119c198fc0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1190009810 tx=0x7f119000bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:48:26.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.127+0000 7f1199600700 1 -- 172.21.15.86:0/103884813 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f119001c070 con 0x7f119c100d80 2024-03-23T17:48:26.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.127+0000 7f11a3800700 1 -- 172.21.15.86:0/103884813 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f119c1a1040 con 0x7f119c100d80 2024-03-23T17:48:26.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.127+0000 7f1199600700 1 -- 172.21.15.86:0/103884813 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f119000bba0 con 0x7f119c100d80 2024-03-23T17:48:26.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.127+0000 7f1199600700 1 -- 172.21.15.86:0/103884813 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1190003900 con 0x7f119c100d80 2024-03-23T17:48:26.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.127+0000 7f11a3800700 1 -- 172.21.15.86:0/103884813 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f119c1a14c0 con 0x7f119c100d80 2024-03-23T17:48:26.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.128+0000 7f11a3800700 1 -- 172.21.15.86:0/103884813 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f119c067af0 con 0x7f119c100d80 2024-03-23T17:48:26.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.130+0000 7f1199600700 1 -- 172.21.15.86:0/103884813 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f1190020030 con 0x7f119c100d80 2024-03-23T17:48:26.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.131+0000 7f1199600700 1 --2- 172.21.15.86:0/103884813 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f11840a1120 0x7f11840a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:26.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.131+0000 7f1199600700 1 -- 172.21.15.86:0/103884813 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f11900cd3b0 con 0x7f119c100d80 2024-03-23T17:48:26.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.131+0000 7f119be00700 1 --2- 172.21.15.86:0/103884813 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f11840a1120 0x7f11840a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:26.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.132+0000 7f119be00700 1 --2- 172.21.15.86:0/103884813 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f11840a1120 0x7f11840a35e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1188011490 tx=0x7f1188009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:48:26.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.135+0000 7f1199600700 1 -- 172.21.15.86:0/103884813 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f1190096fd0 con 0x7f119c100d80 2024-03-23T17:48:26.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.338+0000 7f11a3800700 1 -- 172.21.15.86:0/103884813 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f119c1a16e0 con 0x7f119c100d80 2024-03-23T17:48:26.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.339+0000 7f1199600700 1 -- 172.21.15.86:0/103884813 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v76) v1 ==== 93+0+10 (secure 0 0 0) 0x7f119009a5f0 con 0x7f119c100d80 2024-03-23T17:48:26.339 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-03-23T17:48:26.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.341+0000 7f11a3800700 1 -- 172.21.15.86:0/103884813 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f11840a1120 msgr2=0x7f11840a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:26.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.341+0000 7f11a3800700 1 --2- 172.21.15.86:0/103884813 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f11840a1120 0x7f11840a35e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1188011490 tx=0x7f1188009260 comp rx=0 tx=0).stop 2024-03-23T17:48:26.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.341+0000 7f11a3800700 1 -- 172.21.15.86:0/103884813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f119c100d80 msgr2=0x7f119c198fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:26.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.341+0000 7f11a3800700 1 --2- 172.21.15.86:0/103884813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f119c100d80 0x7f119c198fc0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1190009810 tx=0x7f119000bf60 comp rx=0 tx=0).stop 2024-03-23T17:48:26.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.341+0000 7f11a3800700 1 -- 172.21.15.86:0/103884813 shutdown_connections 2024-03-23T17:48:26.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.341+0000 7f11a3800700 1 --2- 172.21.15.86:0/103884813 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f119c102910 0x7f119c1a0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:26.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.341+0000 7f11a3800700 1 --2- 172.21.15.86:0/103884813 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f11840a1120 0x7f11840a35e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:26.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.341+0000 7f11a3800700 1 --2- 172.21.15.86:0/103884813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f119c100d80 0x7f119c198fc0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:26.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.341+0000 7f11a3800700 1 --2- 172.21.15.86:0/103884813 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f119c101f70 0x7f119c199500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:26.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.341+0000 7f11a3800700 1 -- 172.21.15.86:0/103884813 >> 172.21.15.86:0/103884813 conn(0x7f119c0fc2f0 msgr2=0x7f119c107790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:48:26.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.341+0000 7f11a3800700 1 -- 172.21.15.86:0/103884813 shutdown_connections 2024-03-23T17:48:26.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.341+0000 7f11a3800700 1 -- 172.21.15.86:0/103884813 wait complete. 2024-03-23T17:48:26.361 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T17:48:26.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.551+0000 7f1580c00700 1 -- 172.21.15.86:0/4244310535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f157c100c80 msgr2=0x7f157c1010a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:26.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.551+0000 7f1580c00700 1 --2- 172.21.15.86:0/4244310535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f157c100c80 0x7f157c1010a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f156c0099c0 tx=0x7f156c009cd0 comp rx=0 tx=0).stop 2024-03-23T17:48:26.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.551+0000 7f1580c00700 1 -- 172.21.15.86:0/4244310535 shutdown_connections 2024-03-23T17:48:26.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.551+0000 7f1580c00700 1 --2- 172.21.15.86:0/4244310535 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f157c102810 0x7f157c1090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:26.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.551+0000 7f1580c00700 1 --2- 172.21.15.86:0/4244310535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f157c100c80 0x7f157c1010a0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:26.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.551+0000 7f1580c00700 1 --2- 172.21.15.86:0/4244310535 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f157c101e70 0x7f157c1022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:26.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.551+0000 7f1580c00700 1 -- 172.21.15.86:0/4244310535 >> 172.21.15.86:0/4244310535 conn(0x7f157c0fc1f0 msgr2=0x7f157c0fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:48:26.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.552+0000 7f1580c00700 1 -- 172.21.15.86:0/4244310535 shutdown_connections 2024-03-23T17:48:26.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.552+0000 7f1580c00700 1 -- 172.21.15.86:0/4244310535 wait complete. 2024-03-23T17:48:26.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.552+0000 7f1580c00700 1 Processor -- start 2024-03-23T17:48:26.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.553+0000 7f1580c00700 1 -- start start 2024-03-23T17:48:26.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.553+0000 7f1580c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f157c100c80 0x7f157c198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:26.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.553+0000 7f1580c00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f157c101e70 0x7f157c1993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:26.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.553+0000 7f1580c00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f157c102810 0x7f157c1a03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:26.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.553+0000 7f1580c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f157c1a0990 con 0x7f157c100c80 2024-03-23T17:48:26.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.553+0000 7f1580c00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f157c1a0ad0 con 0x7f157c102810 2024-03-23T17:48:26.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.553+0000 7f1580c00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f157c1a0c60 con 0x7f157c101e70 2024-03-23T17:48:26.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.553+0000 7f1579e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f157c100c80 0x7f157c198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:26.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.553+0000 7f1579400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f157c101e70 0x7f157c1993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:26.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.553+0000 7f1579e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f157c100c80 0x7f157c198eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59274/0 (socket says 172.21.15.86:59274) 2024-03-23T17:48:26.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.553+0000 7f1579e00700 1 -- 172.21.15.86:0/4209860420 learned_addr learned my addr 172.21.15.86:0/4209860420 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:48:26.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.553+0000 7f157a800700 1 --2- 172.21.15.86:0/4209860420 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f157c102810 0x7f157c1a03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:26.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.553+0000 7f1579e00700 1 -- 172.21.15.86:0/4209860420 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f157c101e70 msgr2=0x7f157c1993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:26.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.553+0000 7f1579e00700 1 --2- 172.21.15.86:0/4209860420 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f157c101e70 0x7f157c1993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:26.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.553+0000 7f1579e00700 1 -- 172.21.15.86:0/4209860420 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f157c102810 msgr2=0x7f157c1a03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:26.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.553+0000 7f1579e00700 1 --2- 172.21.15.86:0/4209860420 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f157c102810 0x7f157c1a03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:26.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.553+0000 7f1579e00700 1 -- 172.21.15.86:0/4209860420 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f156c0095f0 con 0x7f157c100c80 2024-03-23T17:48:26.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.554+0000 7f1579e00700 1 --2- 172.21.15.86:0/4209860420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f157c100c80 0x7f157c198eb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f156c003730 tx=0x7f156c00fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:48:26.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.554+0000 7f1572a00700 1 -- 172.21.15.86:0/4209860420 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156c01d070 con 0x7f157c100c80 2024-03-23T17:48:26.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.554+0000 7f1572a00700 1 -- 172.21.15.86:0/4209860420 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f156c00fe70 con 0x7f157c100c80 2024-03-23T17:48:26.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.554+0000 7f1580c00700 1 -- 172.21.15.86:0/4209860420 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f157c1a0ee0 con 0x7f157c100c80 2024-03-23T17:48:26.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.554+0000 7f1572a00700 1 -- 172.21.15.86:0/4209860420 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156c004b30 con 0x7f157c100c80 2024-03-23T17:48:26.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.554+0000 7f1580c00700 1 -- 172.21.15.86:0/4209860420 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f157c1a1250 con 0x7f157c100c80 2024-03-23T17:48:26.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.555+0000 7f1580c00700 1 -- 172.21.15.86:0/4209860420 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f157c067a80 con 0x7f157c100c80 2024-03-23T17:48:26.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.557+0000 7f1572a00700 1 -- 172.21.15.86:0/4209860420 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f156c004cf0 con 0x7f157c100c80 2024-03-23T17:48:26.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.559+0000 7f1572a00700 1 --2- 172.21.15.86:0/4209860420 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f15600a1120 0x7f15600a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:26.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.559+0000 7f1579400700 1 --2- 172.21.15.86:0/4209860420 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f15600a1120 0x7f15600a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:26.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.559+0000 7f1572a00700 1 -- 172.21.15.86:0/4209860420 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f156c0cd3b0 con 0x7f157c100c80 2024-03-23T17:48:26.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.560+0000 7f1579400700 1 --2- 172.21.15.86:0/4209860420 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f15600a1120 0x7f15600a35e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1568007c60 tx=0x7f15680073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:48:26.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.563+0000 7f1572a00700 1 -- 172.21.15.86:0/4209860420 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f157c067a80 con 0x7f157c100c80 2024-03-23T17:48:26.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.815+0000 7f1580c00700 1 -- 172.21.15.86:0/4209860420 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f157c10c4e0 con 0x7f157c100c80 2024-03-23T17:48:26.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.815+0000 7f1572a00700 1 -- 172.21.15.86:0/4209860420 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+501 (secure 0 0 0) 0x7f156c096fd0 con 0x7f157c100c80 2024-03-23T17:48:26.816 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:48:26.816 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-03-23T17:48:26.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.818+0000 7f1580c00700 1 -- 172.21.15.86:0/4209860420 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f15600a1120 msgr2=0x7f15600a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:26.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.818+0000 7f1580c00700 1 --2- 172.21.15.86:0/4209860420 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f15600a1120 0x7f15600a35e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1568007c60 tx=0x7f15680073d0 comp rx=0 tx=0).stop 2024-03-23T17:48:26.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.818+0000 7f1580c00700 1 -- 172.21.15.86:0/4209860420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f157c100c80 msgr2=0x7f157c198eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:26.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.818+0000 7f1580c00700 1 --2- 172.21.15.86:0/4209860420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f157c100c80 0x7f157c198eb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f156c003730 tx=0x7f156c00fa00 comp rx=0 tx=0).stop 2024-03-23T17:48:26.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.818+0000 7f1580c00700 1 -- 172.21.15.86:0/4209860420 shutdown_connections 2024-03-23T17:48:26.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.818+0000 7f1580c00700 1 --2- 172.21.15.86:0/4209860420 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f157c102810 0x7f157c1a03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:26.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.818+0000 7f1580c00700 1 --2- 172.21.15.86:0/4209860420 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f15600a1120 0x7f15600a35e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:26.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.818+0000 7f1580c00700 1 --2- 172.21.15.86:0/4209860420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f157c100c80 0x7f157c198eb0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:26.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.818+0000 7f1580c00700 1 --2- 172.21.15.86:0/4209860420 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f157c101e70 0x7f157c1993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:26.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.818+0000 7f1580c00700 1 -- 172.21.15.86:0/4209860420 >> 172.21.15.86:0/4209860420 conn(0x7f157c0fc1f0 msgr2=0x7f157c072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:48:26.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.818+0000 7f1580c00700 1 -- 172.21.15.86:0/4209860420 shutdown_connections 2024-03-23T17:48:26.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:26.818+0000 7f1580c00700 1 -- 172.21.15.86:0/4209860420 wait complete. 2024-03-23T17:48:26.820 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 1 2024-03-23T17:48:27.002 INFO:journalctl@ceph.mds.l.smithi152.stdout:Mar 23 17:48:26 smithi152 systemd[1]: Starting Ceph mds.l for 9d76c9e8-e93c-11ee-95cd-87774f69a715... 2024-03-23T17:48:27.002 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:26 smithi152 ceph-mon[47949]: Deploying daemon mds.l on smithi152 2024-03-23T17:48:27.002 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:26 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/3273346792' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T17:48:27.002 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:26 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/103884813' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T17:48:27.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:26 smithi086 ceph-mon[38221]: Deploying daemon mds.l on smithi152 2024-03-23T17:48:27.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:26 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/3273346792' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T17:48:27.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:26 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/103884813' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T17:48:27.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:26 smithi092 ceph-mon[79089]: Deploying daemon mds.l on smithi152 2024-03-23T17:48:27.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:26 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/3273346792' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T17:48:27.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:26 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/103884813' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T17:48:27.586 INFO:journalctl@ceph.mds.l.smithi152.stdout:Mar 23 17:48:27 smithi152 ceph-mds[83883]: -- 172.21.15.152:0/958185593 <== mon.1 v2:172.21.15.152:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56163be801a0 con 0x56163be56c00 2024-03-23T17:48:27.586 INFO:journalctl@ceph.mds.l.smithi152.stdout:Mar 23 17:48:27 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-l[83861]: starting mds.l at 2024-03-23T17:48:27.586 INFO:journalctl@ceph.mds.l.smithi152.stdout:Mar 23 17:48:27 smithi152 bash[83846]: 05c687668ebe318ba3534cc73de6012216eca689e4591ae440f3a42eed9cb427 2024-03-23T17:48:27.586 INFO:journalctl@ceph.mds.l.smithi152.stdout:Mar 23 17:48:27 smithi152 systemd[1]: Started Ceph mds.l for 9d76c9e8-e93c-11ee-95cd-87774f69a715. 2024-03-23T17:48:28.016 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:27 smithi092 ceph-mon[79089]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T17:48:28.017 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:27 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/4209860420' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T17:48:28.017 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:27 smithi092 ceph-mon[79089]: mds.? [v2:172.21.15.152:6832/3075554623,v1:172.21.15.152:6833/3075554623] up:boot 2024-03-23T17:48:28.017 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:27 smithi092 ceph-mon[79089]: fsmap 1 up:standby 2024-03-23T17:48:28.017 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:27 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-23T17:48:28.017 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:27 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:28.017 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:27 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:28.017 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:27 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:28.017 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:27 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T17:48:28.017 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:27 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T17:48:28.017 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:27 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:48:28.018 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:27 smithi086 ceph-mon[38221]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T17:48:28.018 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:27 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/4209860420' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T17:48:28.018 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:27 smithi086 ceph-mon[38221]: mds.? [v2:172.21.15.152:6832/3075554623,v1:172.21.15.152:6833/3075554623] up:boot 2024-03-23T17:48:28.018 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:27 smithi086 ceph-mon[38221]: fsmap 1 up:standby 2024-03-23T17:48:28.018 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:27 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-23T17:48:28.018 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:27 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:28.018 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:27 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:28.018 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:27 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:28.018 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:27 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T17:48:28.018 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:27 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T17:48:28.018 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:27 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:48:28.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:27 smithi152 ceph-mon[47949]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T17:48:28.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:27 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/4209860420' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T17:48:28.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:27 smithi152 ceph-mon[47949]: mds.? [v2:172.21.15.152:6832/3075554623,v1:172.21.15.152:6833/3075554623] up:boot 2024-03-23T17:48:28.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:27 smithi152 ceph-mon[47949]: fsmap 1 up:standby 2024-03-23T17:48:28.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:27 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-23T17:48:28.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:27 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:28.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:27 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:28.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:27 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:28.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:27 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T17:48:28.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:27 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T17:48:28.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:27 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:48:28.552 INFO:journalctl@ceph.mds.k.smithi092.stdout:Mar 23 17:48:28 smithi092 systemd[1]: Starting Ceph mds.k for 9d76c9e8-e93c-11ee-95cd-87774f69a715... 2024-03-23T17:48:28.837 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T17:48:29.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:28 smithi086 ceph-mon[38221]: Deploying daemon mds.k on smithi092 2024-03-23T17:48:29.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:29.042+0000 7f752d800700 1 -- 172.21.15.86:0/4072376159 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7528074df0 msgr2=0x7f7528073240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:29.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:29.042+0000 7f752d800700 1 --2- 172.21.15.86:0/4072376159 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7528074df0 0x7f7528073240 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f75180098f0 tx=0x7f7518009c00 comp rx=0 tx=0).stop 2024-03-23T17:48:29.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:29.042+0000 7f752d800700 1 -- 172.21.15.86:0/4072376159 shutdown_connections 2024-03-23T17:48:29.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:29.042+0000 7f752d800700 1 --2- 172.21.15.86:0/4072376159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7528108b40 0x7f752810afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:29.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:29.042+0000 7f752d800700 1 --2- 172.21.15.86:0/4072376159 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7528074df0 0x7f7528073240 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:29.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:29.042+0000 7f752d800700 1 --2- 172.21.15.86:0/4072376159 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7528073810 0x7f7528073c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:29.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:29.042+0000 7f752d800700 1 -- 172.21.15.86:0/4072376159 >> 172.21.15.86:0/4072376159 conn(0x7f75280fc1f0 msgr2=0x7f75280fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:48:29.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:29.042+0000 7f752d800700 1 -- 172.21.15.86:0/4072376159 shutdown_connections 2024-03-23T17:48:29.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:29.042+0000 7f752d800700 1 -- 172.21.15.86:0/4072376159 wait complete. 2024-03-23T17:48:29.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:29.043+0000 7f752d800700 1 Processor -- start 2024-03-23T17:48:29.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:29.043+0000 7f752d800700 1 -- start start 2024-03-23T17:48:29.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:29.044+0000 7f752d800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7528073810 0x7f7528198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:29.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:29.044+0000 7f752d800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7528074df0 0x7f75281993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:29.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:29.044+0000 7f752d800700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7528108b40 0x7f75281a0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:29.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:29.044+0000 7f752d800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75281a09d0 con 0x7f7528073810 2024-03-23T17:48:29.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:29.044+0000 7f752d800700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75281a0b10 con 0x7f7528074df0 2024-03-23T17:48:29.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:29.044+0000 7f752d800700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75281a0ca0 con 0x7f7528108b40 2024-03-23T17:48:29.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:29.044+0000 7f7526a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7528073810 0x7f7528198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:29.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:29.044+0000 7f7526a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7528073810 0x7f7528198ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59284/0 (socket says 172.21.15.86:59284) 2024-03-23T17:48:29.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:29.044+0000 7f7526a00700 1 -- 172.21.15.86:0/974422635 learned_addr learned my addr 172.21.15.86:0/974422635 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:48:29.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:29.044+0000 7f7526000700 1 --2- 172.21.15.86:0/974422635 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7528074df0 0x7f75281993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:29.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:29.044+0000 7f7527400700 1 --2- 172.21.15.86:0/974422635 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7528108b40 0x7f75281a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:29.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:29.044+0000 7f7526a00700 1 -- 172.21.15.86:0/974422635 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7528108b40 msgr2=0x7f75281a0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:29.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:29.044+0000 7f7526a00700 1 --2- 172.21.15.86:0/974422635 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7528108b40 0x7f75281a0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:29.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:29.044+0000 7f7526a00700 1 -- 172.21.15.86:0/974422635 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7528074df0 msgr2=0x7f75281993e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:29.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:29.044+0000 7f7526a00700 1 --2- 172.21.15.86:0/974422635 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7528074df0 0x7f75281993e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:29.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:29.044+0000 7f7526a00700 1 -- 172.21.15.86:0/974422635 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7520009d00 con 0x7f7528073810 2024-03-23T17:48:29.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:29.044+0000 7f7526a00700 1 --2- 172.21.15.86:0/974422635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7528073810 0x7f7528198ea0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7518009810 tx=0x7f751800b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:48:29.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:29.045+0000 7f751f400700 1 -- 172.21.15.86:0/974422635 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f751801c070 con 0x7f7528073810 2024-03-23T17:48:29.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:29.045+0000 7f752d800700 1 -- 172.21.15.86:0/974422635 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7518009520 con 0x7f7528073810 2024-03-23T17:48:29.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:29.045+0000 7f751f400700 1 -- 172.21.15.86:0/974422635 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f751800f7d0 con 0x7f7528073810 2024-03-23T17:48:29.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:29.045+0000 7f751f400700 1 -- 172.21.15.86:0/974422635 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75180039f0 con 0x7f7528073810 2024-03-23T17:48:29.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:29.045+0000 7f752d800700 1 -- 172.21.15.86:0/974422635 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75281a11c0 con 0x7f7528073810 2024-03-23T17:48:29.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:29.046+0000 7f752d800700 1 -- 172.21.15.86:0/974422635 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7528107900 con 0x7f7528073810 2024-03-23T17:48:29.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:29.048+0000 7f751f400700 1 -- 172.21.15.86:0/974422635 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f7518020030 con 0x7f7528073810 2024-03-23T17:48:29.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:29.049+0000 7f751f400700 1 --2- 172.21.15.86:0/974422635 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f750c0a1120 0x7f750c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:29.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:29.049+0000 7f751f400700 1 -- 172.21.15.86:0/974422635 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f75180cd450 con 0x7f7528073810 2024-03-23T17:48:29.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:29.049+0000 7f7526000700 1 --2- 172.21.15.86:0/974422635 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f750c0a1120 0x7f750c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:29.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:29.050+0000 7f7526000700 1 --2- 172.21.15.86:0/974422635 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f750c0a1120 0x7f750c0a35e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7528074b10 tx=0x7f7514009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:48:29.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:29.053+0000 7f751f400700 1 -- 172.21.15.86:0/974422635 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f7518097070 con 0x7f7528073810 2024-03-23T17:48:29.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:28 smithi092 ceph-mon[79089]: Deploying daemon mds.k on smithi092 2024-03-23T17:48:29.057 INFO:journalctl@ceph.mds.k.smithi092.stdout:Mar 23 17:48:28 smithi092 ceph-mds[114114]: -- 172.21.15.92:0/3612952078 <== mon.2 v2:172.21.15.92:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5640e33781a0 con 0x5640e334e800 2024-03-23T17:48:29.057 INFO:journalctl@ceph.mds.k.smithi092.stdout:Mar 23 17:48:28 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-k[114091]: starting mds.k at 2024-03-23T17:48:29.057 INFO:journalctl@ceph.mds.k.smithi092.stdout:Mar 23 17:48:28 smithi092 bash[114076]: 5022702cb9bfa9dcf40b2949496e210bc59549c3fbdf3779fde6a4281fd88e8c 2024-03-23T17:48:29.057 INFO:journalctl@ceph.mds.k.smithi092.stdout:Mar 23 17:48:28 smithi092 systemd[1]: Started Ceph mds.k for 9d76c9e8-e93c-11ee-95cd-87774f69a715. 2024-03-23T17:48:29.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:28 smithi152 ceph-mon[47949]: Deploying daemon mds.k on smithi092 2024-03-23T17:48:29.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:29.328+0000 7f752d800700 1 -- 172.21.15.86:0/974422635 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7528067a10 con 0x7f7528073810 2024-03-23T17:48:29.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:29.328+0000 7f751f400700 1 -- 172.21.15.86:0/974422635 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1260 (secure 0 0 0) 0x7f751809a690 con 0x7f7528073810 2024-03-23T17:48:29.329 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:48:29.329 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24475,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/3075554623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3075554623},{"type":"v1","addr":"172.21.15.152:6833","nonce":3075554623}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2024-03-23T17:48:29.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:29.331+0000 7f752d800700 1 -- 172.21.15.86:0/974422635 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f750c0a1120 msgr2=0x7f750c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:29.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:29.331+0000 7f752d800700 1 --2- 172.21.15.86:0/974422635 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f750c0a1120 0x7f750c0a35e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7528074b10 tx=0x7f7514009260 comp rx=0 tx=0).stop 2024-03-23T17:48:29.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:29.331+0000 7f752d800700 1 -- 172.21.15.86:0/974422635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7528073810 msgr2=0x7f7528198ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:29.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:29.331+0000 7f752d800700 1 --2- 172.21.15.86:0/974422635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7528073810 0x7f7528198ea0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7518009810 tx=0x7f751800b790 comp rx=0 tx=0).stop 2024-03-23T17:48:29.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:29.331+0000 7f752d800700 1 -- 172.21.15.86:0/974422635 shutdown_connections 2024-03-23T17:48:29.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:29.331+0000 7f752d800700 1 --2- 172.21.15.86:0/974422635 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7528108b40 0x7f75281a0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:29.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:29.331+0000 7f752d800700 1 --2- 172.21.15.86:0/974422635 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f750c0a1120 0x7f750c0a35e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:29.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:29.331+0000 7f752d800700 1 --2- 172.21.15.86:0/974422635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7528073810 0x7f7528198ea0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:29.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:29.331+0000 7f752d800700 1 --2- 172.21.15.86:0/974422635 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7528074df0 0x7f75281993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:29.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:29.331+0000 7f752d800700 1 -- 172.21.15.86:0/974422635 >> 172.21.15.86:0/974422635 conn(0x7f75280fc1f0 msgr2=0x7f7528104a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:48:29.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:29.331+0000 7f752d800700 1 -- 172.21.15.86:0/974422635 shutdown_connections 2024-03-23T17:48:29.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:29.331+0000 7f752d800700 1 -- 172.21.15.86:0/974422635 wait complete. 2024-03-23T17:48:29.332 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 2 2024-03-23T17:48:30.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:29 smithi086 ceph-mon[38221]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T17:48:30.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:29 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:30.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:29 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:30.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:29 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:30.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:29 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T17:48:30.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:29 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T17:48:30.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:29 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:48:30.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:29 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/974422635' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T17:48:30.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:29 smithi092 ceph-mon[79089]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T17:48:30.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:29 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:30.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:29 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:30.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:29 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:30.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:29 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T17:48:30.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:29 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T17:48:30.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:29 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:48:30.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:29 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/974422635' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T17:48:30.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:29 smithi152 ceph-mon[47949]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T17:48:30.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:29 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:30.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:29 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:30.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:29 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:30.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:29 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T17:48:30.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:29 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T17:48:30.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:29 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:48:30.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:29 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/974422635' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T17:48:31.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:30 smithi086 ceph-mon[38221]: Deploying daemon mds.j on smithi086 2024-03-23T17:48:31.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:30 smithi086 ceph-mon[38221]: mds.? [v2:172.21.15.92:6832/1467091186,v1:172.21.15.92:6833/1467091186] up:boot 2024-03-23T17:48:31.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:30 smithi086 ceph-mon[38221]: fsmap 2 up:standby 2024-03-23T17:48:31.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:30 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-23T17:48:31.017 INFO:journalctl@ceph.mds.j.smithi086.stdout:Mar 23 17:48:30 smithi086 systemd[1]: Starting Ceph mds.j for 9d76c9e8-e93c-11ee-95cd-87774f69a715... 2024-03-23T17:48:31.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:30 smithi092 ceph-mon[79089]: Deploying daemon mds.j on smithi086 2024-03-23T17:48:31.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:30 smithi092 ceph-mon[79089]: mds.? [v2:172.21.15.92:6832/1467091186,v1:172.21.15.92:6833/1467091186] up:boot 2024-03-23T17:48:31.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:30 smithi092 ceph-mon[79089]: fsmap 2 up:standby 2024-03-23T17:48:31.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:30 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-23T17:48:31.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:30 smithi152 ceph-mon[47949]: Deploying daemon mds.j on smithi086 2024-03-23T17:48:31.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:30 smithi152 ceph-mon[47949]: mds.? [v2:172.21.15.92:6832/1467091186,v1:172.21.15.92:6833/1467091186] up:boot 2024-03-23T17:48:31.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:30 smithi152 ceph-mon[47949]: fsmap 2 up:standby 2024-03-23T17:48:31.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:30 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-23T17:48:31.352 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T17:48:31.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:31.563+0000 7f44d1400700 1 -- 172.21.15.86:0/3104173692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44cc10d250 msgr2=0x7f44cc10f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:31.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:31.563+0000 7f44d1400700 1 --2- 172.21.15.86:0/3104173692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44cc10d250 0x7f44cc10f640 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f44c400b0a0 tx=0x7f44c400b3b0 comp rx=0 tx=0).stop 2024-03-23T17:48:31.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:31.564+0000 7f44d1400700 1 -- 172.21.15.86:0/3104173692 shutdown_connections 2024-03-23T17:48:31.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:31.564+0000 7f44d1400700 1 --2- 172.21.15.86:0/3104173692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44cc10d250 0x7f44cc10f640 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:31.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:31.564+0000 7f44d1400700 1 --2- 172.21.15.86:0/3104173692 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f44cc074df0 0x7f44cc073240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:31.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:31.564+0000 7f44d1400700 1 --2- 172.21.15.86:0/3104173692 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44cc073810 0x7f44cc073c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:31.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:31.564+0000 7f44d1400700 1 -- 172.21.15.86:0/3104173692 >> 172.21.15.86:0/3104173692 conn(0x7f44cc0fc1f0 msgr2=0x7f44cc0fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:48:31.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:31.564+0000 7f44d1400700 1 -- 172.21.15.86:0/3104173692 shutdown_connections 2024-03-23T17:48:31.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:31.564+0000 7f44d1400700 1 -- 172.21.15.86:0/3104173692 wait complete. 2024-03-23T17:48:31.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:31.565+0000 7f44d1400700 1 Processor -- start 2024-03-23T17:48:31.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:31.565+0000 7f44d1400700 1 -- start start 2024-03-23T17:48:31.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:31.565+0000 7f44d1400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44cc073810 0x7f44cc19d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:31.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:31.565+0000 7f44d1400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44cc074df0 0x7f44cc19d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:31.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:31.565+0000 7f44d1400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f44cc10d250 0x7f44cc1a4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:31.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:31.565+0000 7f44d1400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44cc1a4e40 con 0x7f44cc073810 2024-03-23T17:48:31.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:31.565+0000 7f44d1400700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44cc1a4f80 con 0x7f44cc074df0 2024-03-23T17:48:31.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:31.565+0000 7f44d1400700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44cc1a5110 con 0x7f44cc10d250 2024-03-23T17:48:31.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:31.565+0000 7f44ca800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44cc073810 0x7f44cc19d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:31.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:31.565+0000 7f44ca800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44cc073810 0x7f44cc19d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59314/0 (socket says 172.21.15.86:59314) 2024-03-23T17:48:31.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:31.565+0000 7f44ca800700 1 -- 172.21.15.86:0/577328175 learned_addr learned my addr 172.21.15.86:0/577328175 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:48:31.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:31.565+0000 7f44c9e00700 1 --2- 172.21.15.86:0/577328175 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44cc074df0 0x7f44cc19d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:31.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:31.565+0000 7f44cb200700 1 --2- 172.21.15.86:0/577328175 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f44cc10d250 0x7f44cc1a4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:31.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:31.566+0000 7f44ca800700 1 -- 172.21.15.86:0/577328175 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f44cc10d250 msgr2=0x7f44cc1a4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:31.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:31.566+0000 7f44ca800700 1 --2- 172.21.15.86:0/577328175 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f44cc10d250 0x7f44cc1a4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:31.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:31.566+0000 7f44ca800700 1 -- 172.21.15.86:0/577328175 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44cc074df0 msgr2=0x7f44cc19d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:31.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:31.566+0000 7f44ca800700 1 --2- 172.21.15.86:0/577328175 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44cc074df0 0x7f44cc19d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:31.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:31.566+0000 7f44ca800700 1 -- 172.21.15.86:0/577328175 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44c4009d00 con 0x7f44cc073810 2024-03-23T17:48:31.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:31.566+0000 7f44ca800700 1 --2- 172.21.15.86:0/577328175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44cc073810 0x7f44cc19d310 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f44bc00e9a0 tx=0x7f44bc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:48:31.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:31.566+0000 7f44c3400700 1 -- 172.21.15.86:0/577328175 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44bc009790 con 0x7f44cc073810 2024-03-23T17:48:31.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:31.566+0000 7f44c3400700 1 -- 172.21.15.86:0/577328175 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f44bc010430 con 0x7f44cc073810 2024-03-23T17:48:31.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:31.566+0000 7f44c3400700 1 -- 172.21.15.86:0/577328175 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44bc018680 con 0x7f44cc073810 2024-03-23T17:48:31.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:31.566+0000 7f44d1400700 1 -- 172.21.15.86:0/577328175 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44cc1a53f0 con 0x7f44cc073810 2024-03-23T17:48:31.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:31.566+0000 7f44d1400700 1 -- 172.21.15.86:0/577328175 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44cc1a58b0 con 0x7f44cc073810 2024-03-23T17:48:31.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:31.568+0000 7f44d1400700 1 -- 172.21.15.86:0/577328175 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44cc1034b0 con 0x7f44cc073810 2024-03-23T17:48:31.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:31.570+0000 7f44c3400700 1 -- 172.21.15.86:0/577328175 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f44bc0105a0 con 0x7f44cc073810 2024-03-23T17:48:31.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:31.571+0000 7f44c3400700 1 --2- 172.21.15.86:0/577328175 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f44b00a1120 0x7f44b00a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:31.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:31.571+0000 7f44c3400700 1 -- 172.21.15.86:0/577328175 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f44bc014070 con 0x7f44cc073810 2024-03-23T17:48:31.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:31.571+0000 7f44c9e00700 1 --2- 172.21.15.86:0/577328175 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f44b00a1120 0x7f44b00a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:31.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:31.572+0000 7f44c9e00700 1 --2- 172.21.15.86:0/577328175 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f44b00a1120 0x7f44b00a35e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f44cc074b10 tx=0x7f44b8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:48:31.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:31.575+0000 7f44c3400700 1 -- 172.21.15.86:0/577328175 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f44bc0529f0 con 0x7f44cc073810 2024-03-23T17:48:31.766 INFO:journalctl@ceph.mds.j.smithi086.stdout:Mar 23 17:48:31 smithi086 bash[89109]: 72bdeac7764e28768ec24cdb636ac6b70c2a437826153f0fcb7223fb51166b57 2024-03-23T17:48:31.766 INFO:journalctl@ceph.mds.j.smithi086.stdout:Mar 23 17:48:31 smithi086 systemd[1]: Started Ceph mds.j for 9d76c9e8-e93c-11ee-95cd-87774f69a715. 2024-03-23T17:48:31.767 INFO:journalctl@ceph.mds.j.smithi086.stdout:Mar 23 17:48:31 smithi086 ceph-mds[89147]: -- 172.21.15.86:0/607176504 <== mon.0 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55db0db7e1a0 con 0x55db0db56c00 2024-03-23T17:48:31.767 INFO:journalctl@ceph.mds.j.smithi086.stdout:Mar 23 17:48:31 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-j[89124]: starting mds.j at 2024-03-23T17:48:31.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:31.848+0000 7f44d1400700 1 -- 172.21.15.86:0/577328175 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f44cc067a10 con 0x7f44cc073810 2024-03-23T17:48:31.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:31.848+0000 7f44c3400700 1 -- 172.21.15.86:0/577328175 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2774 (secure 0 0 0) 0x7f44bc096220 con 0x7f44cc073810 2024-03-23T17:48:31.849 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:48:31.849 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14445,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1433140219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1433140219},{"type":"v1","addr":"172.21.15.86:6835","nonce":1433140219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/1467091186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1467091186},{"type":"v1","addr":"172.21.15.92:6833","nonce":1467091186}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24475,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/3075554623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3075554623},{"type":"v1","addr":"172.21.15.152:6833","nonce":3075554623}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2024-03-23T17:48:31.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:31.850+0000 7f44d1400700 1 -- 172.21.15.86:0/577328175 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f44b00a1120 msgr2=0x7f44b00a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:31.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:31.850+0000 7f44d1400700 1 --2- 172.21.15.86:0/577328175 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f44b00a1120 0x7f44b00a35e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f44cc074b10 tx=0x7f44b8009260 comp rx=0 tx=0).stop 2024-03-23T17:48:31.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:31.851+0000 7f44d1400700 1 -- 172.21.15.86:0/577328175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44cc073810 msgr2=0x7f44cc19d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:31.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:31.851+0000 7f44d1400700 1 --2- 172.21.15.86:0/577328175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44cc073810 0x7f44cc19d310 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f44bc00e9a0 tx=0x7f44bc00ecb0 comp rx=0 tx=0).stop 2024-03-23T17:48:31.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:31.851+0000 7f44d1400700 1 -- 172.21.15.86:0/577328175 shutdown_connections 2024-03-23T17:48:31.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:31.851+0000 7f44d1400700 1 --2- 172.21.15.86:0/577328175 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f44cc10d250 0x7f44cc1a4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:31.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:31.851+0000 7f44d1400700 1 --2- 172.21.15.86:0/577328175 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f44b00a1120 0x7f44b00a35e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:31.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:31.851+0000 7f44d1400700 1 --2- 172.21.15.86:0/577328175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44cc073810 0x7f44cc19d310 secure :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f44bc00e9a0 tx=0x7f44bc00ecb0 comp rx=0 tx=0).stop 2024-03-23T17:48:31.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:31.851+0000 7f44d1400700 1 --2- 172.21.15.86:0/577328175 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44cc074df0 0x7f44cc19d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:31.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:31.851+0000 7f44d1400700 1 -- 172.21.15.86:0/577328175 >> 172.21.15.86:0/577328175 conn(0x7f44cc0fc1f0 msgr2=0x7f44cc1005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:48:31.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:31.851+0000 7f44d1400700 1 -- 172.21.15.86:0/577328175 shutdown_connections 2024-03-23T17:48:31.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:31.851+0000 7f44d1400700 1 -- 172.21.15.86:0/577328175 wait complete. 2024-03-23T17:48:31.852 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 4 2024-03-23T17:48:32.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:31 smithi092 ceph-mon[79089]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T17:48:32.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:31 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:32.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:31 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:32.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:31 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:32.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:31 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T17:48:32.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:31 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T17:48:32.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:31 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:48:32.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:31 smithi152 ceph-mon[47949]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T17:48:32.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:31 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:32.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:31 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:32.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:31 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:32.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:31 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T17:48:32.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:31 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T17:48:32.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:31 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:48:32.114 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:31 smithi086 ceph-mon[38221]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T17:48:32.114 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:31 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:32.114 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:31 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:32.114 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:31 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:32.114 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:31 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T17:48:32.114 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:31 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T17:48:32.114 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:31 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:48:33.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:32 smithi086 ceph-mon[38221]: Deploying daemon mds.d on smithi086 2024-03-23T17:48:33.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:32 smithi086 ceph-mon[38221]: mds.? [v2:172.21.15.86:6834/1433140219,v1:172.21.15.86:6835/1433140219] up:boot 2024-03-23T17:48:33.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:32 smithi086 ceph-mon[38221]: fsmap 3 up:standby 2024-03-23T17:48:33.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:32 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-23T17:48:33.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:32 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/577328175' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T17:48:33.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:32 smithi092 ceph-mon[79089]: Deploying daemon mds.d on smithi086 2024-03-23T17:48:33.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:32 smithi092 ceph-mon[79089]: mds.? [v2:172.21.15.86:6834/1433140219,v1:172.21.15.86:6835/1433140219] up:boot 2024-03-23T17:48:33.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:32 smithi092 ceph-mon[79089]: fsmap 3 up:standby 2024-03-23T17:48:33.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:32 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-23T17:48:33.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:32 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/577328175' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T17:48:33.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:32 smithi152 ceph-mon[47949]: Deploying daemon mds.d on smithi086 2024-03-23T17:48:33.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:32 smithi152 ceph-mon[47949]: mds.? [v2:172.21.15.86:6834/1433140219,v1:172.21.15.86:6835/1433140219] up:boot 2024-03-23T17:48:33.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:32 smithi152 ceph-mon[47949]: fsmap 3 up:standby 2024-03-23T17:48:33.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:32 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-23T17:48:33.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:32 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/577328175' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T17:48:33.672 INFO:journalctl@ceph.mds.d.smithi086.stdout:Mar 23 17:48:33 smithi086 systemd[1]: Starting Ceph mds.d for 9d76c9e8-e93c-11ee-95cd-87774f69a715... 2024-03-23T17:48:33.871 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T17:48:34.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:33 smithi092 ceph-mon[79089]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T17:48:34.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:34.073+0000 7f57c5c00700 1 -- 172.21.15.86:0/886178221 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57c0102810 msgr2=0x7f57c01090c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:34.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:34.073+0000 7f57c5c00700 1 --2- 172.21.15.86:0/886178221 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57c0102810 0x7f57c01090c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f57b800b0d0 tx=0x7f57b800b3e0 comp rx=0 tx=0).stop 2024-03-23T17:48:34.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:34.073+0000 7f57c5c00700 1 -- 172.21.15.86:0/886178221 shutdown_connections 2024-03-23T17:48:34.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:34.073+0000 7f57c5c00700 1 --2- 172.21.15.86:0/886178221 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57c0102810 0x7f57c01090c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:34.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:34.073+0000 7f57c5c00700 1 --2- 172.21.15.86:0/886178221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57c0100c80 0x7f57c01010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:34.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:34.073+0000 7f57c5c00700 1 --2- 172.21.15.86:0/886178221 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f57c0101e70 0x7f57c01022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:34.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:34.073+0000 7f57c5c00700 1 -- 172.21.15.86:0/886178221 >> 172.21.15.86:0/886178221 conn(0x7f57c00fc1f0 msgr2=0x7f57c00fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:48:34.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:34.073+0000 7f57c5c00700 1 -- 172.21.15.86:0/886178221 shutdown_connections 2024-03-23T17:48:34.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:34.073+0000 7f57c5c00700 1 -- 172.21.15.86:0/886178221 wait complete. 2024-03-23T17:48:34.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:34.074+0000 7f57c5c00700 1 Processor -- start 2024-03-23T17:48:34.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:34.074+0000 7f57c5c00700 1 -- start start 2024-03-23T17:48:34.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:34.075+0000 7f57c5c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57c0100c80 0x7f57c0198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:34.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:34.075+0000 7f57c5c00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57c0101e70 0x7f57c0199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:34.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:34.075+0000 7f57c5c00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f57c0102810 0x7f57c01a0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:34.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:34.075+0000 7f57c5c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c01a0a30 con 0x7f57c0100c80 2024-03-23T17:48:34.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:34.075+0000 7f57c5c00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c01a0b70 con 0x7f57c0101e70 2024-03-23T17:48:34.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:34.075+0000 7f57c5c00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c01a0d00 con 0x7f57c0102810 2024-03-23T17:48:34.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:34.075+0000 7f57bea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57c0100c80 0x7f57c0198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:34.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:34.075+0000 7f57bea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57c0100c80 0x7f57c0198f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59352/0 (socket says 172.21.15.86:59352) 2024-03-23T17:48:34.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:34.075+0000 7f57bf400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f57c0102810 0x7f57c01a0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:34.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:34.075+0000 7f57bea00700 1 -- 172.21.15.86:0/811871601 learned_addr learned my addr 172.21.15.86:0/811871601 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:48:34.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:34.075+0000 7f57be000700 1 --2- 172.21.15.86:0/811871601 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57c0101e70 0x7f57c0199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:34.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:34.075+0000 7f57bf400700 1 -- 172.21.15.86:0/811871601 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57c0101e70 msgr2=0x7f57c0199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:34.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:34.075+0000 7f57bf400700 1 --2- 172.21.15.86:0/811871601 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57c0101e70 0x7f57c0199440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:34.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:34.075+0000 7f57bf400700 1 -- 172.21.15.86:0/811871601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57c0100c80 msgr2=0x7f57c0198f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:34.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:34.075+0000 7f57bf400700 1 --2- 172.21.15.86:0/811871601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57c0100c80 0x7f57c0198f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:34.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:34.075+0000 7f57bf400700 1 -- 172.21.15.86:0/811871601 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57b8009d00 con 0x7f57c0102810 2024-03-23T17:48:34.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:34.076+0000 7f57bf400700 1 --2- 172.21.15.86:0/811871601 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f57c0102810 0x7f57c01a0460 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f57b8007b60 tx=0x7f57b8008e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:48:34.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:34.076+0000 7f57b7400700 1 -- 172.21.15.86:0/811871601 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b800e040 con 0x7f57c0102810 2024-03-23T17:48:34.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:34.076+0000 7f57b7400700 1 -- 172.21.15.86:0/811871601 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f57b8005070 con 0x7f57c0102810 2024-03-23T17:48:34.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:34.076+0000 7f57b7400700 1 -- 172.21.15.86:0/811871601 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b801d6d0 con 0x7f57c0102810 2024-03-23T17:48:34.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:34.076+0000 7f57c5c00700 1 -- 172.21.15.86:0/811871601 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57c01a0f80 con 0x7f57c0102810 2024-03-23T17:48:34.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:34.076+0000 7f57c5c00700 1 -- 172.21.15.86:0/811871601 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57c01a12f0 con 0x7f57c0102810 2024-03-23T17:48:34.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:34.077+0000 7f57c5c00700 1 -- 172.21.15.86:0/811871601 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57c0067a80 con 0x7f57c0102810 2024-03-23T17:48:34.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:34.079+0000 7f57b7400700 1 -- 172.21.15.86:0/811871601 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f57b80051e0 con 0x7f57c0102810 2024-03-23T17:48:34.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:34.080+0000 7f57b7400700 1 --2- 172.21.15.86:0/811871601 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f57a40a11a0 0x7f57a40a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:34.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:34.080+0000 7f57b7400700 1 -- 172.21.15.86:0/811871601 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f57b80cd7b0 con 0x7f57c0102810 2024-03-23T17:48:34.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:34.080+0000 7f57bea00700 1 --2- 172.21.15.86:0/811871601 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f57a40a11a0 0x7f57a40a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:34.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:34.081+0000 7f57bea00700 1 --2- 172.21.15.86:0/811871601 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f57a40a11a0 0x7f57a40a3660 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f57b0005950 tx=0x7f57b0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:48:34.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:34.083+0000 7f57b7400700 1 -- 172.21.15.86:0/811871601 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f57b80973d0 con 0x7f57c0102810 2024-03-23T17:48:34.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:33 smithi152 ceph-mon[47949]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T17:48:34.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:33 smithi086 ceph-mon[38221]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T17:48:34.267 INFO:journalctl@ceph.mds.d.smithi086.stdout:Mar 23 17:48:33 smithi086 bash[89495]: 81be26271251152f6c522cbb8552584f9c042a9710633526f5d583ef14bfc87b 2024-03-23T17:48:34.267 INFO:journalctl@ceph.mds.d.smithi086.stdout:Mar 23 17:48:33 smithi086 systemd[1]: Started Ceph mds.d for 9d76c9e8-e93c-11ee-95cd-87774f69a715. 2024-03-23T17:48:34.267 INFO:journalctl@ceph.mds.d.smithi086.stdout:Mar 23 17:48:33 smithi086 ceph-mds[89533]: -- 172.21.15.86:0/4258910794 <== mon.1 v2:172.21.15.152:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x557c454161a0 con 0x557c453ee800 2024-03-23T17:48:34.267 INFO:journalctl@ceph.mds.d.smithi086.stdout:Mar 23 17:48:33 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-d[89510]: starting mds.d at 2024-03-23T17:48:34.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:34.336+0000 7f57c5c00700 1 -- 172.21.15.86:0/811871601 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f57c010c740 con 0x7f57c0102810 2024-03-23T17:48:34.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:34.337+0000 7f57b7400700 1 -- 172.21.15.86:0/811871601 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2774 (secure 0 0 0) 0x7f57b809a9f0 con 0x7f57c0102810 2024-03-23T17:48:34.337 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:48:34.337 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14445,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1433140219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1433140219},{"type":"v1","addr":"172.21.15.86:6835","nonce":1433140219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/1467091186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1467091186},{"type":"v1","addr":"172.21.15.92:6833","nonce":1467091186}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24475,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/3075554623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3075554623},{"type":"v1","addr":"172.21.15.152:6833","nonce":3075554623}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2024-03-23T17:48:34.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:34.339+0000 7f57c5c00700 1 -- 172.21.15.86:0/811871601 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f57a40a11a0 msgr2=0x7f57a40a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:34.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:34.339+0000 7f57c5c00700 1 --2- 172.21.15.86:0/811871601 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f57a40a11a0 0x7f57a40a3660 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f57b0005950 tx=0x7f57b0009260 comp rx=0 tx=0).stop 2024-03-23T17:48:34.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:34.339+0000 7f57c5c00700 1 -- 172.21.15.86:0/811871601 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f57c0102810 msgr2=0x7f57c01a0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:34.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:34.339+0000 7f57c5c00700 1 --2- 172.21.15.86:0/811871601 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f57c0102810 0x7f57c01a0460 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f57b8007b60 tx=0x7f57b8008e40 comp rx=0 tx=0).stop 2024-03-23T17:48:34.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:34.339+0000 7f57c5c00700 1 -- 172.21.15.86:0/811871601 shutdown_connections 2024-03-23T17:48:34.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:34.339+0000 7f57c5c00700 1 --2- 172.21.15.86:0/811871601 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f57c0102810 0x7f57c01a0460 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:34.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:34.339+0000 7f57c5c00700 1 --2- 172.21.15.86:0/811871601 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f57a40a11a0 0x7f57a40a3660 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:34.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:34.339+0000 7f57c5c00700 1 --2- 172.21.15.86:0/811871601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57c0100c80 0x7f57c0198f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:34.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:34.339+0000 7f57c5c00700 1 --2- 172.21.15.86:0/811871601 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57c0101e70 0x7f57c0199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:34.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:34.339+0000 7f57c5c00700 1 -- 172.21.15.86:0/811871601 >> 172.21.15.86:0/811871601 conn(0x7f57c00fc1f0 msgr2=0x7f57c0072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:48:34.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:34.340+0000 7f57c5c00700 1 -- 172.21.15.86:0/811871601 shutdown_connections 2024-03-23T17:48:34.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:34.340+0000 7f57c5c00700 1 -- 172.21.15.86:0/811871601 wait complete. 2024-03-23T17:48:34.341 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 4 2024-03-23T17:48:34.947 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:34 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:34.947 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:34 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:34.948 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:34 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:34.948 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:34 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T17:48:34.948 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:34 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T17:48:34.948 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:34 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:48:34.948 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:34 smithi092 ceph-mon[79089]: Deploying daemon mds.e on smithi092 2024-03-23T17:48:34.948 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:34 smithi092 ceph-mon[79089]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T17:48:34.948 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:34 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/811871601' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T17:48:34.948 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:34 smithi092 ceph-mon[79089]: mds.? [v2:172.21.15.86:6836/2933881328,v1:172.21.15.86:6837/2933881328] up:boot 2024-03-23T17:48:34.948 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:34 smithi092 ceph-mon[79089]: fsmap 4 up:standby 2024-03-23T17:48:34.948 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:34 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-23T17:48:35.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:34 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:35.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:34 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:35.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:34 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:35.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:34 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T17:48:35.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:34 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T17:48:35.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:34 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:48:35.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:34 smithi086 ceph-mon[38221]: Deploying daemon mds.e on smithi092 2024-03-23T17:48:35.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:34 smithi086 ceph-mon[38221]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T17:48:35.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:34 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/811871601' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T17:48:35.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:34 smithi086 ceph-mon[38221]: mds.? [v2:172.21.15.86:6836/2933881328,v1:172.21.15.86:6837/2933881328] up:boot 2024-03-23T17:48:35.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:34 smithi086 ceph-mon[38221]: fsmap 4 up:standby 2024-03-23T17:48:35.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:34 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-23T17:48:35.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:34 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:35.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:34 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:35.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:34 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:35.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:34 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T17:48:35.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:34 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T17:48:35.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:34 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:48:35.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:34 smithi152 ceph-mon[47949]: Deploying daemon mds.e on smithi092 2024-03-23T17:48:35.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:34 smithi152 ceph-mon[47949]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T17:48:35.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:34 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/811871601' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T17:48:35.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:34 smithi152 ceph-mon[47949]: mds.? [v2:172.21.15.86:6836/2933881328,v1:172.21.15.86:6837/2933881328] up:boot 2024-03-23T17:48:35.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:34 smithi152 ceph-mon[47949]: fsmap 4 up:standby 2024-03-23T17:48:35.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:34 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-23T17:48:35.557 INFO:journalctl@ceph.mds.e.smithi092.stdout:Mar 23 17:48:35 smithi092 systemd[1]: Starting Ceph mds.e for 9d76c9e8-e93c-11ee-95cd-87774f69a715... 2024-03-23T17:48:36.057 INFO:journalctl@ceph.mds.e.smithi092.stdout:Mar 23 17:48:35 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-e[114438]: starting mds.e at 2024-03-23T17:48:36.057 INFO:journalctl@ceph.mds.e.smithi092.stdout:Mar 23 17:48:35 smithi092 bash[114423]: c7e375d6898d9e8dbf28739ac3dae0991c2aa7382f78d3e6bc71cd79d588f1f6 2024-03-23T17:48:36.057 INFO:journalctl@ceph.mds.e.smithi092.stdout:Mar 23 17:48:35 smithi092 systemd[1]: Started Ceph mds.e for 9d76c9e8-e93c-11ee-95cd-87774f69a715. 2024-03-23T17:48:36.359 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T17:48:36.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:36.563+0000 7f2e94a00700 1 -- 172.21.15.86:0/2461164034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e90101e70 msgr2=0x7f2e901022d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:36.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:36.563+0000 7f2e94a00700 1 --2- 172.21.15.86:0/2461164034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e90101e70 0x7f2e901022d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c0099c0 tx=0x7f2e7c009cd0 comp rx=0 tx=0).stop 2024-03-23T17:48:36.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:36.563+0000 7f2e94a00700 1 -- 172.21.15.86:0/2461164034 shutdown_connections 2024-03-23T17:48:36.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:36.563+0000 7f2e94a00700 1 --2- 172.21.15.86:0/2461164034 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e90102810 0x7f2e901090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:36.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:36.563+0000 7f2e94a00700 1 --2- 172.21.15.86:0/2461164034 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e90100c80 0x7f2e901010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:36.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:36.563+0000 7f2e94a00700 1 --2- 172.21.15.86:0/2461164034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e90101e70 0x7f2e901022d0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:36.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:36.563+0000 7f2e94a00700 1 -- 172.21.15.86:0/2461164034 >> 172.21.15.86:0/2461164034 conn(0x7f2e900fc1f0 msgr2=0x7f2e900fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:48:36.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:36.563+0000 7f2e94a00700 1 -- 172.21.15.86:0/2461164034 shutdown_connections 2024-03-23T17:48:36.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:36.563+0000 7f2e94a00700 1 -- 172.21.15.86:0/2461164034 wait complete. 2024-03-23T17:48:36.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:36.564+0000 7f2e94a00700 1 Processor -- start 2024-03-23T17:48:36.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:36.565+0000 7f2e94a00700 1 -- start start 2024-03-23T17:48:36.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:36.565+0000 7f2e94a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e90100c80 0x7f2e90074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:36.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:36.565+0000 7f2e94a00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e90101e70 0x7f2e900731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:36.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:36.565+0000 7f2e94a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e90102810 0x7f2e900736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:36.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:36.565+0000 7f2e94a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e90073e80 con 0x7f2e90100c80 2024-03-23T17:48:36.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:36.565+0000 7f2e94a00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e90073ff0 con 0x7f2e90102810 2024-03-23T17:48:36.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:36.565+0000 7f2e94a00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e901a0c20 con 0x7f2e90101e70 2024-03-23T17:48:36.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:36.565+0000 7f2e8de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e90100c80 0x7f2e90074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:36.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:36.565+0000 7f2e8de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e90100c80 0x7f2e90074b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54944/0 (socket says 172.21.15.86:54944) 2024-03-23T17:48:36.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:36.565+0000 7f2e8d400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e90101e70 0x7f2e900731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:36.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:36.565+0000 7f2e8de00700 1 -- 172.21.15.86:0/3331429788 learned_addr learned my addr 172.21.15.86:0/3331429788 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:48:36.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:36.565+0000 7f2e8e800700 1 --2- 172.21.15.86:0/3331429788 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e90102810 0x7f2e900736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:36.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:36.565+0000 7f2e8de00700 1 -- 172.21.15.86:0/3331429788 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e90101e70 msgr2=0x7f2e900731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:36.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:36.566+0000 7f2e8de00700 1 --2- 172.21.15.86:0/3331429788 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e90101e70 0x7f2e900731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:36.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:36.566+0000 7f2e8de00700 1 -- 172.21.15.86:0/3331429788 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e90102810 msgr2=0x7f2e900736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:36.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:36.566+0000 7f2e8de00700 1 --2- 172.21.15.86:0/3331429788 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e90102810 0x7f2e900736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:36.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:36.566+0000 7f2e8de00700 1 -- 172.21.15.86:0/3331429788 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e7c0095f0 con 0x7f2e90100c80 2024-03-23T17:48:36.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:36.566+0000 7f2e8d400700 1 --2- 172.21.15.86:0/3331429788 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e90101e70 0x7f2e900731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:48:36.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:36.566+0000 7f2e8e800700 1 --2- 172.21.15.86:0/3331429788 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e90102810 0x7f2e900736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:48:36.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:36.566+0000 7f2e8de00700 1 --2- 172.21.15.86:0/3331429788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e90100c80 0x7f2e90074b10 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2e8000d980 tx=0x7f2e8000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:48:36.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:36.566+0000 7f2e86a00700 1 -- 172.21.15.86:0/3331429788 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e80004020 con 0x7f2e90100c80 2024-03-23T17:48:36.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:36.566+0000 7f2e86a00700 1 -- 172.21.15.86:0/3331429788 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e800096d0 con 0x7f2e90100c80 2024-03-23T17:48:36.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:36.566+0000 7f2e86a00700 1 -- 172.21.15.86:0/3331429788 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e80003a80 con 0x7f2e90100c80 2024-03-23T17:48:36.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:36.566+0000 7f2e94a00700 1 -- 172.21.15.86:0/3331429788 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e901a0eb0 con 0x7f2e90100c80 2024-03-23T17:48:36.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:36.566+0000 7f2e94a00700 1 -- 172.21.15.86:0/3331429788 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e901a1280 con 0x7f2e90100c80 2024-03-23T17:48:36.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:36.567+0000 7f2e94a00700 1 -- 172.21.15.86:0/3331429788 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e90067a80 con 0x7f2e90100c80 2024-03-23T17:48:36.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:36.569+0000 7f2e86a00700 1 -- 172.21.15.86:0/3331429788 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f2e80009840 con 0x7f2e90100c80 2024-03-23T17:48:36.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:36.571+0000 7f2e86a00700 1 --2- 172.21.15.86:0/3331429788 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f2e740a11a0 0x7f2e740a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:36.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:36.571+0000 7f2e86a00700 1 -- 172.21.15.86:0/3331429788 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f2e800cc270 con 0x7f2e90100c80 2024-03-23T17:48:36.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:36.572+0000 7f2e8d400700 1 --2- 172.21.15.86:0/3331429788 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f2e740a11a0 0x7f2e740a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:36.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:36.572+0000 7f2e8d400700 1 --2- 172.21.15.86:0/3331429788 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f2e740a11a0 0x7f2e740a3660 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c006010 tx=0x7f2e7c00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:48:36.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:36.574+0000 7f2e86a00700 1 -- 172.21.15.86:0/3331429788 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f2e80095e80 con 0x7f2e90100c80 2024-03-23T17:48:36.742 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:36 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:36.742 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:36 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:36.743 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:36 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:36.743 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:36 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T17:48:36.743 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:36 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T17:48:36.743 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:36 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:48:36.743 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:36 smithi152 ceph-mon[47949]: Deploying daemon mds.f on smithi152 2024-03-23T17:48:36.743 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:36 smithi152 ceph-mon[47949]: mds.? [v2:172.21.15.92:6834/908900027,v1:172.21.15.92:6835/908900027] up:boot 2024-03-23T17:48:36.743 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:36 smithi152 ceph-mon[47949]: fsmap 5 up:standby 2024-03-23T17:48:36.743 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:36 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-23T17:48:36.766 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:36 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:36.766 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:36 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:36.766 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:36 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:36.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:36 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T17:48:36.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:36 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T17:48:36.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:36 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:48:36.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:36 smithi086 ceph-mon[38221]: Deploying daemon mds.f on smithi152 2024-03-23T17:48:36.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:36 smithi086 ceph-mon[38221]: mds.? [v2:172.21.15.92:6834/908900027,v1:172.21.15.92:6835/908900027] up:boot 2024-03-23T17:48:36.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:36 smithi086 ceph-mon[38221]: fsmap 5 up:standby 2024-03-23T17:48:36.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:36 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-23T17:48:36.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:36.829+0000 7f2e94a00700 1 -- 172.21.15.86:0/3331429788 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2e90108030 con 0x7f2e90100c80 2024-03-23T17:48:36.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:36.829+0000 7f2e86a00700 1 -- 172.21.15.86:0/3331429788 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4285 (secure 0 0 0) 0x7f2e800994a0 con 0x7f2e90100c80 2024-03-23T17:48:36.830 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:48:36.830 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14445,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1433140219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1433140219},{"type":"v1","addr":"172.21.15.86:6835","nonce":1433140219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/1467091186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1467091186},{"type":"v1","addr":"172.21.15.92:6833","nonce":1467091186}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24475,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/3075554623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3075554623},{"type":"v1","addr":"172.21.15.152:6833","nonce":3075554623}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/908900027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":908900027},{"type":"v1","addr":"172.21.15.92:6835","nonce":908900027}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24505,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2933881328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2933881328},{"type":"v1","addr":"172.21.15.86:6837","nonce":2933881328}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[]} 2024-03-23T17:48:36.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:36.831+0000 7f2e94a00700 1 -- 172.21.15.86:0/3331429788 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f2e740a11a0 msgr2=0x7f2e740a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:36.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:36.831+0000 7f2e94a00700 1 --2- 172.21.15.86:0/3331429788 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f2e740a11a0 0x7f2e740a3660 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c006010 tx=0x7f2e7c00b540 comp rx=0 tx=0).stop 2024-03-23T17:48:36.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:36.831+0000 7f2e94a00700 1 -- 172.21.15.86:0/3331429788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e90100c80 msgr2=0x7f2e90074b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:36.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:36.831+0000 7f2e94a00700 1 --2- 172.21.15.86:0/3331429788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e90100c80 0x7f2e90074b10 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2e8000d980 tx=0x7f2e8000dc90 comp rx=0 tx=0).stop 2024-03-23T17:48:36.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:36.832+0000 7f2e94a00700 1 -- 172.21.15.86:0/3331429788 shutdown_connections 2024-03-23T17:48:36.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:36.832+0000 7f2e94a00700 1 --2- 172.21.15.86:0/3331429788 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e90102810 0x7f2e900736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:36.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:36.832+0000 7f2e94a00700 1 --2- 172.21.15.86:0/3331429788 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f2e740a11a0 0x7f2e740a3660 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:36.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:36.832+0000 7f2e94a00700 1 --2- 172.21.15.86:0/3331429788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e90100c80 0x7f2e90074b10 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:36.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:36.832+0000 7f2e94a00700 1 --2- 172.21.15.86:0/3331429788 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e90101e70 0x7f2e900731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:36.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:36.832+0000 7f2e94a00700 1 -- 172.21.15.86:0/3331429788 >> 172.21.15.86:0/3331429788 conn(0x7f2e900fc1f0 msgr2=0x7f2e90105170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:48:36.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:36.832+0000 7f2e94a00700 1 -- 172.21.15.86:0/3331429788 shutdown_connections 2024-03-23T17:48:36.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:36.832+0000 7f2e94a00700 1 -- 172.21.15.86:0/3331429788 wait complete. 2024-03-23T17:48:36.833 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 6 2024-03-23T17:48:37.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:36 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:37.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:36 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:37.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:36 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:37.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:36 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T17:48:37.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:36 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T17:48:37.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:36 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:48:37.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:36 smithi092 ceph-mon[79089]: Deploying daemon mds.f on smithi152 2024-03-23T17:48:37.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:36 smithi092 ceph-mon[79089]: mds.? [v2:172.21.15.92:6834/908900027,v1:172.21.15.92:6835/908900027] up:boot 2024-03-23T17:48:37.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:36 smithi092 ceph-mon[79089]: fsmap 5 up:standby 2024-03-23T17:48:37.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:36 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-23T17:48:37.086 INFO:journalctl@ceph.mds.f.smithi152.stdout:Mar 23 17:48:36 smithi152 systemd[1]: Starting Ceph mds.f for 9d76c9e8-e93c-11ee-95cd-87774f69a715... 2024-03-23T17:48:37.833 INFO:journalctl@ceph.mds.f.smithi152.stdout:Mar 23 17:48:37 smithi152 ceph-mds[84232]: -- 172.21.15.152:0/1576078888 <== mon.1 v2:172.21.15.152:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560d3f3281a0 con 0x560d3f2fe800 2024-03-23T17:48:37.834 INFO:journalctl@ceph.mds.f.smithi152.stdout:Mar 23 17:48:37 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-f[84209]: starting mds.f at 2024-03-23T17:48:37.834 INFO:journalctl@ceph.mds.f.smithi152.stdout:Mar 23 17:48:37 smithi152 bash[84194]: 2f03768518bd87bf25b443b33f4090899bf076aa4f49f0cff035ae34361085b0 2024-03-23T17:48:37.834 INFO:journalctl@ceph.mds.f.smithi152.stdout:Mar 23 17:48:37 smithi152 systemd[1]: Started Ceph mds.f for 9d76c9e8-e93c-11ee-95cd-87774f69a715. 2024-03-23T17:48:37.834 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:37 smithi152 ceph-mon[47949]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T17:48:37.834 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:37 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/3331429788' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T17:48:37.834 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:37 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:37.834 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:37 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:37.834 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:37 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:37.834 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:37 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T17:48:37.834 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:37 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T17:48:37.834 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:37 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:48:38.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:37 smithi086 ceph-mon[38221]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T17:48:38.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:37 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/3331429788' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T17:48:38.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:37 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:38.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:37 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:38.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:37 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:38.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:37 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T17:48:38.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:37 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T17:48:38.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:37 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:48:38.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:37 smithi092 ceph-mon[79089]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T17:48:38.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:37 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/3331429788' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T17:48:38.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:37 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:38.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:37 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:38.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:37 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:38.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:37 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T17:48:38.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:37 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T17:48:38.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:37 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:48:38.852 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T17:48:38.915 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:38 smithi086 ceph-mon[38221]: Deploying daemon mds.c on smithi152 2024-03-23T17:48:38.916 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:38 smithi086 ceph-mon[38221]: mds.? [v2:172.21.15.152:6834/3137085021,v1:172.21.15.152:6835/3137085021] up:boot 2024-03-23T17:48:38.916 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:38 smithi086 ceph-mon[38221]: fsmap 6 up:standby 2024-03-23T17:48:38.916 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:38 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-23T17:48:38.950 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:38 smithi152 ceph-mon[47949]: Deploying daemon mds.c on smithi152 2024-03-23T17:48:38.950 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:38 smithi152 ceph-mon[47949]: mds.? [v2:172.21.15.152:6834/3137085021,v1:172.21.15.152:6835/3137085021] up:boot 2024-03-23T17:48:38.950 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:38 smithi152 ceph-mon[47949]: fsmap 6 up:standby 2024-03-23T17:48:38.950 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:38 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-23T17:48:38.951 INFO:journalctl@ceph.mds.c.smithi152.stdout:Mar 23 17:48:38 smithi152 systemd[1]: Starting Ceph mds.c for 9d76c9e8-e93c-11ee-95cd-87774f69a715... 2024-03-23T17:48:39.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:39.056+0000 7f797f400700 1 -- 172.21.15.86:0/1387255535 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7978101e70 msgr2=0x7f79781022d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:39.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:39.056+0000 7f797f400700 1 --2- 172.21.15.86:0/1387255535 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7978101e70 0x7f79781022d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7964009990 tx=0x7f7964009ca0 comp rx=0 tx=0).stop 2024-03-23T17:48:39.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:39.056+0000 7f797f400700 1 -- 172.21.15.86:0/1387255535 shutdown_connections 2024-03-23T17:48:39.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:39.056+0000 7f797f400700 1 --2- 172.21.15.86:0/1387255535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7978102810 0x7f79781090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:39.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:39.056+0000 7f797f400700 1 --2- 172.21.15.86:0/1387255535 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7978100c80 0x7f79781010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:39.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:39.056+0000 7f797f400700 1 --2- 172.21.15.86:0/1387255535 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7978101e70 0x7f79781022d0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:39.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:39.056+0000 7f797f400700 1 -- 172.21.15.86:0/1387255535 >> 172.21.15.86:0/1387255535 conn(0x7f79780fc1f0 msgr2=0x7f79780fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:48:39.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:39.056+0000 7f797f400700 1 -- 172.21.15.86:0/1387255535 shutdown_connections 2024-03-23T17:48:39.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:39.056+0000 7f797f400700 1 -- 172.21.15.86:0/1387255535 wait complete. 2024-03-23T17:48:39.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:38 smithi092 ceph-mon[79089]: Deploying daemon mds.c on smithi152 2024-03-23T17:48:39.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:38 smithi092 ceph-mon[79089]: mds.? [v2:172.21.15.152:6834/3137085021,v1:172.21.15.152:6835/3137085021] up:boot 2024-03-23T17:48:39.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:38 smithi092 ceph-mon[79089]: fsmap 6 up:standby 2024-03-23T17:48:39.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:38 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-23T17:48:39.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:39.057+0000 7f797f400700 1 Processor -- start 2024-03-23T17:48:39.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:39.057+0000 7f797f400700 1 -- start start 2024-03-23T17:48:39.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:39.058+0000 7f797f400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7978100c80 0x7f7978196c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:39.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:39.058+0000 7f797f400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7978101e70 0x7f7978197180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:39.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:39.058+0000 7f797f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7978102810 0x7f797819e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:39.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:39.058+0000 7f797f400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f797819e770 con 0x7f7978102810 2024-03-23T17:48:39.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:39.058+0000 7f797f400700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f797819e8b0 con 0x7f7978100c80 2024-03-23T17:48:39.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:39.058+0000 7f797f400700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f797819ea40 con 0x7f7978101e70 2024-03-23T17:48:39.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:39.058+0000 7f797d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7978102810 0x7f797819e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:39.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:39.058+0000 7f797ca00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7978100c80 0x7f7978196c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:39.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:39.058+0000 7f797d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7978102810 0x7f797819e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54968/0 (socket says 172.21.15.86:54968) 2024-03-23T17:48:39.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:39.058+0000 7f797d400700 1 -- 172.21.15.86:0/4154231897 learned_addr learned my addr 172.21.15.86:0/4154231897 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:48:39.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:39.058+0000 7f7977e00700 1 --2- 172.21.15.86:0/4154231897 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7978101e70 0x7f7978197180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:39.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:39.058+0000 7f7977e00700 1 -- 172.21.15.86:0/4154231897 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7978100c80 msgr2=0x7f7978196c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:39.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:39.058+0000 7f7977e00700 1 --2- 172.21.15.86:0/4154231897 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7978100c80 0x7f7978196c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:39.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:39.058+0000 7f7977e00700 1 -- 172.21.15.86:0/4154231897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7978102810 msgr2=0x7f797819e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:39.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:39.058+0000 7f7977e00700 1 --2- 172.21.15.86:0/4154231897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7978102810 0x7f797819e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:39.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:39.058+0000 7f7977e00700 1 -- 172.21.15.86:0/4154231897 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7970009c40 con 0x7f7978101e70 2024-03-23T17:48:39.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:39.059+0000 7f7977e00700 1 --2- 172.21.15.86:0/4154231897 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7978101e70 0x7f7978197180 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f79640098e0 tx=0x7f7964009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:48:39.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:39.059+0000 7f7975600700 1 -- 172.21.15.86:0/4154231897 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f796401d070 con 0x7f7978101e70 2024-03-23T17:48:39.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:39.059+0000 7f797f400700 1 -- 172.21.15.86:0/4154231897 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79640095f0 con 0x7f7978101e70 2024-03-23T17:48:39.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:39.059+0000 7f7975600700 1 -- 172.21.15.86:0/4154231897 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7964022470 con 0x7f7978101e70 2024-03-23T17:48:39.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:39.059+0000 7f7975600700 1 -- 172.21.15.86:0/4154231897 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7964017650 con 0x7f7978101e70 2024-03-23T17:48:39.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:39.059+0000 7f797f400700 1 -- 172.21.15.86:0/4154231897 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f797819ef60 con 0x7f7978101e70 2024-03-23T17:48:39.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:39.060+0000 7f797f400700 1 -- 172.21.15.86:0/4154231897 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7978108150 con 0x7f7978101e70 2024-03-23T17:48:39.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:39.062+0000 7f7975600700 1 -- 172.21.15.86:0/4154231897 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f7964022890 con 0x7f7978101e70 2024-03-23T17:48:39.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:39.063+0000 7f7975600700 1 --2- 172.21.15.86:0/4154231897 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f79600a1120 0x7f79600a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:39.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:39.063+0000 7f7975600700 1 -- 172.21.15.86:0/4154231897 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f79640cd670 con 0x7f7978101e70 2024-03-23T17:48:39.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:39.063+0000 7f797ca00700 1 --2- 172.21.15.86:0/4154231897 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f79600a1120 0x7f79600a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:39.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:39.064+0000 7f797ca00700 1 --2- 172.21.15.86:0/4154231897 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f79600a1120 0x7f79600a35e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7978101cd0 tx=0x7f796c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:48:39.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:39.067+0000 7f7975600700 1 -- 172.21.15.86:0/4154231897 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f796405d9f0 con 0x7f7978101e70 2024-03-23T17:48:39.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:39.325+0000 7f797f400700 1 -- 172.21.15.86:0/4154231897 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7978067a10 con 0x7f7978101e70 2024-03-23T17:48:39.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:39.326+0000 7f7975600700 1 -- 172.21.15.86:0/4154231897 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5045 (secure 0 0 0) 0x7f7964097290 con 0x7f7978101e70 2024-03-23T17:48:39.326 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:48:39.327 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14445,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1433140219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1433140219},{"type":"v1","addr":"172.21.15.86:6835","nonce":1433140219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/1467091186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1467091186},{"type":"v1","addr":"172.21.15.92:6833","nonce":1467091186}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24475,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/3075554623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3075554623},{"type":"v1","addr":"172.21.15.152:6833","nonce":3075554623}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/908900027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":908900027},{"type":"v1","addr":"172.21.15.92:6835","nonce":908900027}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24505,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2933881328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2933881328},{"type":"v1","addr":"172.21.15.86:6837","nonce":2933881328}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/3137085021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3137085021},{"type":"v1","addr":"172.21.15.152:6835","nonce":3137085021}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7}],"filesystems":[]} 2024-03-23T17:48:39.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:39.329+0000 7f797f400700 1 -- 172.21.15.86:0/4154231897 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f79600a1120 msgr2=0x7f79600a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:39.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:39.329+0000 7f797f400700 1 --2- 172.21.15.86:0/4154231897 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f79600a1120 0x7f79600a35e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7978101cd0 tx=0x7f796c009260 comp rx=0 tx=0).stop 2024-03-23T17:48:39.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:39.329+0000 7f797f400700 1 -- 172.21.15.86:0/4154231897 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7978101e70 msgr2=0x7f7978197180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:39.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:39.329+0000 7f797f400700 1 --2- 172.21.15.86:0/4154231897 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7978101e70 0x7f7978197180 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f79640098e0 tx=0x7f7964009580 comp rx=0 tx=0).stop 2024-03-23T17:48:39.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:39.329+0000 7f797f400700 1 -- 172.21.15.86:0/4154231897 shutdown_connections 2024-03-23T17:48:39.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:39.329+0000 7f797f400700 1 --2- 172.21.15.86:0/4154231897 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f79600a1120 0x7f79600a35e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:39.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:39.329+0000 7f797f400700 1 --2- 172.21.15.86:0/4154231897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7978102810 0x7f797819e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:39.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:39.329+0000 7f797f400700 1 --2- 172.21.15.86:0/4154231897 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7978100c80 0x7f7978196c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:39.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:39.329+0000 7f797f400700 1 --2- 172.21.15.86:0/4154231897 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7978101e70 0x7f7978197180 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:39.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:39.329+0000 7f797f400700 1 -- 172.21.15.86:0/4154231897 >> 172.21.15.86:0/4154231897 conn(0x7f79780fc1f0 msgr2=0x7f7978105290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:48:39.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:39.329+0000 7f797f400700 1 -- 172.21.15.86:0/4154231897 shutdown_connections 2024-03-23T17:48:39.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:39.329+0000 7f797f400700 1 -- 172.21.15.86:0/4154231897 wait complete. 2024-03-23T17:48:39.330 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 7 2024-03-23T17:48:39.586 INFO:journalctl@ceph.mds.c.smithi152.stdout:Mar 23 17:48:39 smithi152 ceph-mds[84575]: -- 172.21.15.152:0/3977181516 <== mon.1 v2:172.21.15.152:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e404b421a0 con 0x55e404b19000 2024-03-23T17:48:39.586 INFO:journalctl@ceph.mds.c.smithi152.stdout:Mar 23 17:48:39 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-c[84552]: starting mds.c at 2024-03-23T17:48:39.586 INFO:journalctl@ceph.mds.c.smithi152.stdout:Mar 23 17:48:39 smithi152 bash[84537]: ff682e0248704412cb07618b25ba3a5e23c891efe0bec3f6ee66808a60cfe299 2024-03-23T17:48:39.586 INFO:journalctl@ceph.mds.c.smithi152.stdout:Mar 23 17:48:39 smithi152 systemd[1]: Started Ceph mds.c for 9d76c9e8-e93c-11ee-95cd-87774f69a715. 2024-03-23T17:48:39.947 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:39 smithi092 ceph-mon[79089]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T17:48:39.947 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:39 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:39.947 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:39 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:39.947 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:39 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:39.948 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:39 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T17:48:39.948 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:39 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T17:48:39.948 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:39 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:48:39.948 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:39 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/4154231897' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T17:48:40.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:39 smithi086 ceph-mon[38221]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T17:48:40.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:39 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:40.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:39 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:40.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:39 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:40.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:39 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T17:48:40.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:39 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T17:48:40.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:39 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:48:40.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:39 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/4154231897' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T17:48:40.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:39 smithi152 ceph-mon[47949]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T17:48:40.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:39 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:40.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:39 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:40.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:39 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:40.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:39 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T17:48:40.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:39 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T17:48:40.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:39 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:48:40.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:39 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/4154231897' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T17:48:40.491 INFO:journalctl@ceph.mds.h.smithi092.stdout:Mar 23 17:48:40 smithi092 systemd[1]: Starting Ceph mds.h for 9d76c9e8-e93c-11ee-95cd-87774f69a715... 2024-03-23T17:48:41.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:40 smithi086 ceph-mon[38221]: Deploying daemon mds.h on smithi092 2024-03-23T17:48:41.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:40 smithi086 ceph-mon[38221]: mds.? [v2:172.21.15.152:6836/1151208965,v1:172.21.15.152:6837/1151208965] up:boot 2024-03-23T17:48:41.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:40 smithi086 ceph-mon[38221]: fsmap 7 up:standby 2024-03-23T17:48:41.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:40 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-23T17:48:41.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:40 smithi092 ceph-mon[79089]: Deploying daemon mds.h on smithi092 2024-03-23T17:48:41.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:40 smithi092 ceph-mon[79089]: mds.? [v2:172.21.15.152:6836/1151208965,v1:172.21.15.152:6837/1151208965] up:boot 2024-03-23T17:48:41.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:40 smithi092 ceph-mon[79089]: fsmap 7 up:standby 2024-03-23T17:48:41.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:40 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-23T17:48:41.058 INFO:journalctl@ceph.mds.h.smithi092.stdout:Mar 23 17:48:40 smithi092 ceph-mds[114804]: -- 172.21.15.92:0/619365443 <== mon.2 v2:172.21.15.92:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5605635b41a0 con 0x56056358ac00 2024-03-23T17:48:41.058 INFO:journalctl@ceph.mds.h.smithi092.stdout:Mar 23 17:48:40 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-h[114780]: starting mds.h at 2024-03-23T17:48:41.058 INFO:journalctl@ceph.mds.h.smithi092.stdout:Mar 23 17:48:40 smithi092 bash[114765]: 5dd6eda4695d419392710fa10a9f08898d6187abba41b7447895f12b5115be63 2024-03-23T17:48:41.058 INFO:journalctl@ceph.mds.h.smithi092.stdout:Mar 23 17:48:40 smithi092 systemd[1]: Started Ceph mds.h for 9d76c9e8-e93c-11ee-95cd-87774f69a715. 2024-03-23T17:48:41.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:40 smithi152 ceph-mon[47949]: Deploying daemon mds.h on smithi092 2024-03-23T17:48:41.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:40 smithi152 ceph-mon[47949]: mds.? [v2:172.21.15.152:6836/1151208965,v1:172.21.15.152:6837/1151208965] up:boot 2024-03-23T17:48:41.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:40 smithi152 ceph-mon[47949]: fsmap 7 up:standby 2024-03-23T17:48:41.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:40 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-23T17:48:41.348 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T17:48:41.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:41.550+0000 7fd818c00700 1 -- 172.21.15.86:0/1717994655 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd814104070 msgr2=0x7fd814106460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:41.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:41.550+0000 7fd818c00700 1 --2- 172.21.15.86:0/1717994655 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd814104070 0x7fd814106460 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd800009990 tx=0x7fd800009ca0 comp rx=0 tx=0).stop 2024-03-23T17:48:41.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:41.550+0000 7fd818c00700 1 -- 172.21.15.86:0/1717994655 shutdown_connections 2024-03-23T17:48:41.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:41.550+0000 7fd818c00700 1 --2- 172.21.15.86:0/1717994655 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8141069a0 0x7fd814108db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:41.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:41.550+0000 7fd818c00700 1 --2- 172.21.15.86:0/1717994655 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd814101740 0x7fd814103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:41.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:41.550+0000 7fd818c00700 1 --2- 172.21.15.86:0/1717994655 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd814104070 0x7fd814106460 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:41.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:41.550+0000 7fd818c00700 1 -- 172.21.15.86:0/1717994655 >> 172.21.15.86:0/1717994655 conn(0x7fd8140fb0d0 msgr2=0x7fd8140fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:48:41.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:41.550+0000 7fd818c00700 1 -- 172.21.15.86:0/1717994655 shutdown_connections 2024-03-23T17:48:41.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:41.550+0000 7fd818c00700 1 -- 172.21.15.86:0/1717994655 wait complete. 2024-03-23T17:48:41.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:41.551+0000 7fd818c00700 1 Processor -- start 2024-03-23T17:48:41.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:41.551+0000 7fd818c00700 1 -- start start 2024-03-23T17:48:41.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:41.551+0000 7fd818c00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd814101740 0x7fd814198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:41.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:41.551+0000 7fd818c00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd814104070 0x7fd814199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:41.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:41.551+0000 7fd818c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8141069a0 0x7fd8141a0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:41.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:41.551+0000 7fd818c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8141a0950 con 0x7fd8141069a0 2024-03-23T17:48:41.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:41.551+0000 7fd818c00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8141a0a90 con 0x7fd814101740 2024-03-23T17:48:41.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:41.551+0000 7fd818c00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8141a0c20 con 0x7fd814104070 2024-03-23T17:48:41.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:41.552+0000 7fd811400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd814104070 0x7fd814199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:41.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:41.552+0000 7fd811400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd814104070 0x7fd814199360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:39260/0 (socket says 172.21.15.86:39260) 2024-03-23T17:48:41.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:41.552+0000 7fd811400700 1 -- 172.21.15.86:0/1363482562 learned_addr learned my addr 172.21.15.86:0/1363482562 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:48:41.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:41.552+0000 7fd811e00700 1 --2- 172.21.15.86:0/1363482562 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd814101740 0x7fd814198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:41.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:41.552+0000 7fd812800700 1 --2- 172.21.15.86:0/1363482562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8141069a0 0x7fd8141a0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:41.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:41.552+0000 7fd811e00700 1 -- 172.21.15.86:0/1363482562 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd814104070 msgr2=0x7fd814199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:41.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:41.552+0000 7fd811e00700 1 --2- 172.21.15.86:0/1363482562 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd814104070 0x7fd814199360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:41.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:41.552+0000 7fd811e00700 1 -- 172.21.15.86:0/1363482562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8141069a0 msgr2=0x7fd8141a0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:41.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:41.552+0000 7fd811e00700 1 --2- 172.21.15.86:0/1363482562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8141069a0 0x7fd8141a0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:41.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:41.552+0000 7fd811e00700 1 -- 172.21.15.86:0/1363482562 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8000095f0 con 0x7fd814101740 2024-03-23T17:48:41.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:41.552+0000 7fd811e00700 1 --2- 172.21.15.86:0/1363482562 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd814101740 0x7fd814198e20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd80400e9f0 tx=0x7fd80400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:48:41.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:41.553+0000 7fd80aa00700 1 -- 172.21.15.86:0/1363482562 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd80400c9c0 con 0x7fd814101740 2024-03-23T17:48:41.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:41.553+0000 7fd80aa00700 1 -- 172.21.15.86:0/1363482562 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd804010430 con 0x7fd814101740 2024-03-23T17:48:41.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:41.553+0000 7fd80aa00700 1 -- 172.21.15.86:0/1363482562 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd804018680 con 0x7fd814101740 2024-03-23T17:48:41.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:41.553+0000 7fd818c00700 1 -- 172.21.15.86:0/1363482562 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8141a0f00 con 0x7fd814101740 2024-03-23T17:48:41.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:41.553+0000 7fd818c00700 1 -- 172.21.15.86:0/1363482562 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8141a12d0 con 0x7fd814101740 2024-03-23T17:48:41.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:41.554+0000 7fd818c00700 1 -- 172.21.15.86:0/1363482562 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd814108540 con 0x7fd814101740 2024-03-23T17:48:41.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:41.555+0000 7fd80aa00700 1 -- 172.21.15.86:0/1363482562 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fd804010860 con 0x7fd814101740 2024-03-23T17:48:41.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:41.557+0000 7fd80aa00700 1 --2- 172.21.15.86:0/1363482562 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fd7f80a11a0 0x7fd7f80a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:41.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:41.557+0000 7fd80aa00700 1 -- 172.21.15.86:0/1363482562 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd804014070 con 0x7fd814101740 2024-03-23T17:48:41.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:41.557+0000 7fd811400700 1 --2- 172.21.15.86:0/1363482562 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fd7f80a11a0 0x7fd7f80a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:41.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:41.557+0000 7fd811400700 1 --2- 172.21.15.86:0/1363482562 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fd7f80a11a0 0x7fd7f80a3660 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd800005b00 tx=0x7fd800005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:48:41.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:41.560+0000 7fd80aa00700 1 -- 172.21.15.86:0/1363482562 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fd804096070 con 0x7fd814101740 2024-03-23T17:48:41.766 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:41 smithi086 ceph-mon[38221]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T17:48:41.766 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:41 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:41.766 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:41 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:41.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:41 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:41.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:41 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T17:48:41.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:41 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T17:48:41.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:41 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:48:41.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:41.814+0000 7fd818c00700 1 -- 172.21.15.86:0/1363482562 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd814067a10 con 0x7fd814101740 2024-03-23T17:48:41.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:41.815+0000 7fd80aa00700 1 -- 172.21.15.86:0/1363482562 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6562 (secure 0 0 0) 0x7fd804099690 con 0x7fd814101740 2024-03-23T17:48:41.816 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:48:41.816 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14445,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1433140219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1433140219},{"type":"v1","addr":"172.21.15.86:6835","nonce":1433140219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/1467091186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1467091186},{"type":"v1","addr":"172.21.15.92:6833","nonce":1467091186}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24475,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/3075554623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3075554623},{"type":"v1","addr":"172.21.15.152:6833","nonce":3075554623}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/908900027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":908900027},{"type":"v1","addr":"172.21.15.92:6835","nonce":908900027}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24505,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2933881328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2933881328},{"type":"v1","addr":"172.21.15.86:6837","nonce":2933881328}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24518,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/3967837948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":3967837948},{"type":"v1","addr":"172.21.15.92:6837","nonce":3967837948}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/3137085021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3137085021},{"type":"v1","addr":"172.21.15.152:6835","nonce":3137085021}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24538,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/1151208965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":1151208965},{"type":"v1","addr":"172.21.15.152:6837","nonce":1151208965}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8}],"filesystems":[]} 2024-03-23T17:48:41.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:41.818+0000 7fd818c00700 1 -- 172.21.15.86:0/1363482562 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fd7f80a11a0 msgr2=0x7fd7f80a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:41.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:41.818+0000 7fd818c00700 1 --2- 172.21.15.86:0/1363482562 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fd7f80a11a0 0x7fd7f80a3660 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd800005b00 tx=0x7fd800005a90 comp rx=0 tx=0).stop 2024-03-23T17:48:41.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:41.818+0000 7fd818c00700 1 -- 172.21.15.86:0/1363482562 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd814101740 msgr2=0x7fd814198e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:41.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:41.818+0000 7fd818c00700 1 --2- 172.21.15.86:0/1363482562 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd814101740 0x7fd814198e20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd80400e9f0 tx=0x7fd80400ed00 comp rx=0 tx=0).stop 2024-03-23T17:48:41.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:41.818+0000 7fd818c00700 1 -- 172.21.15.86:0/1363482562 shutdown_connections 2024-03-23T17:48:41.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:41.818+0000 7fd818c00700 1 --2- 172.21.15.86:0/1363482562 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fd7f80a11a0 0x7fd7f80a3660 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:41.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:41.818+0000 7fd818c00700 1 --2- 172.21.15.86:0/1363482562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8141069a0 0x7fd8141a0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:41.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:41.818+0000 7fd818c00700 1 --2- 172.21.15.86:0/1363482562 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd814101740 0x7fd814198e20 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:41.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:41.818+0000 7fd818c00700 1 --2- 172.21.15.86:0/1363482562 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd814104070 0x7fd814199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:41.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:41.818+0000 7fd818c00700 1 -- 172.21.15.86:0/1363482562 >> 172.21.15.86:0/1363482562 conn(0x7fd8140fb0d0 msgr2=0x7fd814101000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:48:41.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:41.818+0000 7fd818c00700 1 -- 172.21.15.86:0/1363482562 shutdown_connections 2024-03-23T17:48:41.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:41.818+0000 7fd818c00700 1 -- 172.21.15.86:0/1363482562 wait complete. 2024-03-23T17:48:41.821 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 9 2024-03-23T17:48:42.051 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:41 smithi152 ceph-mon[47949]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T17:48:42.051 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:41 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:42.051 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:41 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:42.051 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:41 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:42.051 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:41 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T17:48:42.051 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:41 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T17:48:42.051 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:41 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:48:42.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:41 smithi092 ceph-mon[79089]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T17:48:42.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:41 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:42.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:41 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:42.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:41 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:42.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:41 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T17:48:42.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:41 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T17:48:42.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:41 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:48:42.336 INFO:journalctl@ceph.mds.i.smithi152.stdout:Mar 23 17:48:42 smithi152 systemd[1]: Starting Ceph mds.i for 9d76c9e8-e93c-11ee-95cd-87774f69a715... 2024-03-23T17:48:42.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:42 smithi152 ceph-mon[47949]: Deploying daemon mds.i on smithi152 2024-03-23T17:48:42.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:42 smithi152 ceph-mon[47949]: mds.? [v2:172.21.15.92:6836/3967837948,v1:172.21.15.92:6837/3967837948] up:boot 2024-03-23T17:48:42.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:42 smithi152 ceph-mon[47949]: fsmap 8 up:standby 2024-03-23T17:48:42.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:42 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-23T17:48:42.837 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:42 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/1363482562' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T17:48:42.837 INFO:journalctl@ceph.mds.i.smithi152.stdout:Mar 23 17:48:42 smithi152 ceph-mds[84917]: -- 172.21.15.152:0/3121643630 <== mon.1 v2:172.21.15.152:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f11f18a1a0 con 0x55f11f160800 2024-03-23T17:48:42.837 INFO:journalctl@ceph.mds.i.smithi152.stdout:Mar 23 17:48:42 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-i[84894]: starting mds.i at 2024-03-23T17:48:42.837 INFO:journalctl@ceph.mds.i.smithi152.stdout:Mar 23 17:48:42 smithi152 bash[84879]: 3e9be4dddbb64744d763400b1cb48b5a067187e8eff1ae8c4606677d2e5b1b72 2024-03-23T17:48:42.837 INFO:journalctl@ceph.mds.i.smithi152.stdout:Mar 23 17:48:42 smithi152 systemd[1]: Started Ceph mds.i for 9d76c9e8-e93c-11ee-95cd-87774f69a715. 2024-03-23T17:48:43.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:42 smithi086 ceph-mon[38221]: Deploying daemon mds.i on smithi152 2024-03-23T17:48:43.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:42 smithi086 ceph-mon[38221]: mds.? [v2:172.21.15.92:6836/3967837948,v1:172.21.15.92:6837/3967837948] up:boot 2024-03-23T17:48:43.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:42 smithi086 ceph-mon[38221]: fsmap 8 up:standby 2024-03-23T17:48:43.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:42 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-23T17:48:43.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:42 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/1363482562' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T17:48:43.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:42 smithi092 ceph-mon[79089]: Deploying daemon mds.i on smithi152 2024-03-23T17:48:43.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:42 smithi092 ceph-mon[79089]: mds.? [v2:172.21.15.92:6836/3967837948,v1:172.21.15.92:6837/3967837948] up:boot 2024-03-23T17:48:43.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:42 smithi092 ceph-mon[79089]: fsmap 8 up:standby 2024-03-23T17:48:43.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:42 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-23T17:48:43.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:42 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/1363482562' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T17:48:43.838 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T17:48:44.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:44.038+0000 7f6c90000700 1 -- 172.21.15.86:0/1346554363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c88101e70 msgr2=0x7f6c881022d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:44.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:44.038+0000 7f6c90000700 1 --2- 172.21.15.86:0/1346554363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c88101e70 0x7f6c881022d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6c740099c0 tx=0x7f6c74009cd0 comp rx=0 tx=0).stop 2024-03-23T17:48:44.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:44.039+0000 7f6c90000700 1 -- 172.21.15.86:0/1346554363 shutdown_connections 2024-03-23T17:48:44.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:44.039+0000 7f6c90000700 1 --2- 172.21.15.86:0/1346554363 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c88102810 0x7f6c881090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:44.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:44.039+0000 7f6c90000700 1 --2- 172.21.15.86:0/1346554363 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6c88100c80 0x7f6c881010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:44.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:44.039+0000 7f6c90000700 1 --2- 172.21.15.86:0/1346554363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c88101e70 0x7f6c881022d0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:44.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:44.039+0000 7f6c90000700 1 -- 172.21.15.86:0/1346554363 >> 172.21.15.86:0/1346554363 conn(0x7f6c880fc1f0 msgr2=0x7f6c880fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:48:44.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:44.039+0000 7f6c90000700 1 -- 172.21.15.86:0/1346554363 shutdown_connections 2024-03-23T17:48:44.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:44.039+0000 7f6c90000700 1 -- 172.21.15.86:0/1346554363 wait complete. 2024-03-23T17:48:44.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:44.040+0000 7f6c90000700 1 Processor -- start 2024-03-23T17:48:44.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:44.040+0000 7f6c90000700 1 -- start start 2024-03-23T17:48:44.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:44.040+0000 7f6c90000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c88100c80 0x7f6c88074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:44.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:44.040+0000 7f6c90000700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6c88101e70 0x7f6c880731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:44.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:44.040+0000 7f6c90000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c88102810 0x7f6c880736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:44.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:44.040+0000 7f6c90000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c88073e80 con 0x7f6c88102810 2024-03-23T17:48:44.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:44.040+0000 7f6c90000700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c88073ff0 con 0x7f6c88100c80 2024-03-23T17:48:44.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:44.040+0000 7f6c90000700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c881a0c40 con 0x7f6c88101e70 2024-03-23T17:48:44.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:44.040+0000 7f6c8d600700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c88100c80 0x7f6c88074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:44.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:44.040+0000 7f6c8cc00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6c88101e70 0x7f6c880731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:44.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:44.040+0000 7f6c8d600700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c88100c80 0x7f6c88074b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:57302/0 (socket says 172.21.15.86:57302) 2024-03-23T17:48:44.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:44.040+0000 7f6c8d600700 1 -- 172.21.15.86:0/2139074508 learned_addr learned my addr 172.21.15.86:0/2139074508 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:48:44.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:44.041+0000 7f6c8e000700 1 --2- 172.21.15.86:0/2139074508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c88102810 0x7f6c880736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:44.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:44.041+0000 7f6c8d600700 1 -- 172.21.15.86:0/2139074508 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6c88101e70 msgr2=0x7f6c880731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:44.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:44.041+0000 7f6c8d600700 1 --2- 172.21.15.86:0/2139074508 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6c88101e70 0x7f6c880731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:44.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:44.041+0000 7f6c8d600700 1 -- 172.21.15.86:0/2139074508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c88102810 msgr2=0x7f6c880736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:44.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:44.041+0000 7f6c8d600700 1 --2- 172.21.15.86:0/2139074508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c88102810 0x7f6c880736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:44.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:44.041+0000 7f6c8d600700 1 -- 172.21.15.86:0/2139074508 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c740095f0 con 0x7f6c88100c80 2024-03-23T17:48:44.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:44.041+0000 7f6c8cc00700 1 --2- 172.21.15.86:0/2139074508 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6c88101e70 0x7f6c880731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:48:44.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:44.041+0000 7f6c8d600700 1 --2- 172.21.15.86:0/2139074508 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c88100c80 0x7f6c88074b10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c00e970 tx=0x7f6c7c00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:48:44.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:44.041+0000 7f6c86000700 1 -- 172.21.15.86:0/2139074508 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c7c009790 con 0x7f6c88100c80 2024-03-23T17:48:44.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:44.042+0000 7f6c90000700 1 -- 172.21.15.86:0/2139074508 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c881a0ed0 con 0x7f6c88100c80 2024-03-23T17:48:44.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:44.042+0000 7f6c86000700 1 -- 172.21.15.86:0/2139074508 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c7c00c780 con 0x7f6c88100c80 2024-03-23T17:48:44.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:44.042+0000 7f6c86000700 1 -- 172.21.15.86:0/2139074508 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c7c003be0 con 0x7f6c88100c80 2024-03-23T17:48:44.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:44.042+0000 7f6c90000700 1 -- 172.21.15.86:0/2139074508 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c881a1300 con 0x7f6c88100c80 2024-03-23T17:48:44.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:44.043+0000 7f6c90000700 1 -- 172.21.15.86:0/2139074508 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c881080b0 con 0x7f6c88100c80 2024-03-23T17:48:44.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:44.045+0000 7f6c86000700 1 -- 172.21.15.86:0/2139074508 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f6c7c00cbf0 con 0x7f6c88100c80 2024-03-23T17:48:44.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:44.046+0000 7f6c86000700 1 --2- 172.21.15.86:0/2139074508 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f6c700a1120 0x7f6c700a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:44.047 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:43 smithi086 ceph-mon[38221]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T17:48:44.047 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:43 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:44.047 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:43 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:44.047 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:43 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:44.047 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:43 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T17:48:44.047 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:43 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T17:48:44.047 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:43 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:48:44.047 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:43 smithi086 ceph-mon[38221]: Deploying daemon mds.g on smithi086 2024-03-23T17:48:44.047 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:43 smithi086 ceph-mon[38221]: mds.? [v2:172.21.15.152:6838/1100723992,v1:172.21.15.152:6839/1100723992] up:boot 2024-03-23T17:48:44.047 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:43 smithi086 ceph-mon[38221]: fsmap 9 up:standby 2024-03-23T17:48:44.048 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:43 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-23T17:48:44.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:44.047+0000 7f6c86000700 1 -- 172.21.15.86:0/2139074508 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f6c7c014070 con 0x7f6c88100c80 2024-03-23T17:48:44.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:44.047+0000 7f6c8cc00700 1 --2- 172.21.15.86:0/2139074508 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f6c700a1120 0x7f6c700a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:44.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:44.048+0000 7f6c8cc00700 1 --2- 172.21.15.86:0/2139074508 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f6c700a1120 0x7f6c700a35e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6c74006010 tx=0x7f6c7400b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:48:44.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:44.049+0000 7f6c86000700 1 -- 172.21.15.86:0/2139074508 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f6c7c0968c0 con 0x7f6c88100c80 2024-03-23T17:48:44.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:43 smithi092 ceph-mon[79089]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T17:48:44.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:43 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:44.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:43 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:44.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:43 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:44.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:43 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T17:48:44.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:43 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T17:48:44.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:43 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:48:44.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:43 smithi092 ceph-mon[79089]: Deploying daemon mds.g on smithi086 2024-03-23T17:48:44.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:43 smithi092 ceph-mon[79089]: mds.? [v2:172.21.15.152:6838/1100723992,v1:172.21.15.152:6839/1100723992] up:boot 2024-03-23T17:48:44.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:43 smithi092 ceph-mon[79089]: fsmap 9 up:standby 2024-03-23T17:48:44.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:43 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-23T17:48:44.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:43 smithi152 ceph-mon[47949]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T17:48:44.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:43 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:44.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:43 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:44.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:43 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:44.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:43 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T17:48:44.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:43 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T17:48:44.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:43 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:48:44.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:43 smithi152 ceph-mon[47949]: Deploying daemon mds.g on smithi086 2024-03-23T17:48:44.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:43 smithi152 ceph-mon[47949]: mds.? [v2:172.21.15.152:6838/1100723992,v1:172.21.15.152:6839/1100723992] up:boot 2024-03-23T17:48:44.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:43 smithi152 ceph-mon[47949]: fsmap 9 up:standby 2024-03-23T17:48:44.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:43 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-23T17:48:44.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:44.337+0000 7f6c90000700 1 -- 172.21.15.86:0/2139074508 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6c8810c790 con 0x7f6c88100c80 2024-03-23T17:48:44.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:44.338+0000 7f6c86000700 1 -- 172.21.15.86:0/2139074508 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7324 (secure 0 0 0) 0x7f6c7c099ee0 con 0x7f6c88100c80 2024-03-23T17:48:44.339 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:48:44.339 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14445,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1433140219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1433140219},{"type":"v1","addr":"172.21.15.86:6835","nonce":1433140219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/1467091186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1467091186},{"type":"v1","addr":"172.21.15.92:6833","nonce":1467091186}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24475,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/3075554623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3075554623},{"type":"v1","addr":"172.21.15.152:6833","nonce":3075554623}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/908900027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":908900027},{"type":"v1","addr":"172.21.15.92:6835","nonce":908900027}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24505,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2933881328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2933881328},{"type":"v1","addr":"172.21.15.86:6837","nonce":2933881328}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24518,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/3967837948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":3967837948},{"type":"v1","addr":"172.21.15.92:6837","nonce":3967837948}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/3137085021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3137085021},{"type":"v1","addr":"172.21.15.152:6835","nonce":3137085021}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24538,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/1151208965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":1151208965},{"type":"v1","addr":"172.21.15.152:6837","nonce":1151208965}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24553,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6839/1100723992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":1100723992},{"type":"v1","addr":"172.21.15.152:6839","nonce":1100723992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2024-03-23T17:48:44.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:44.341+0000 7f6c90000700 1 -- 172.21.15.86:0/2139074508 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f6c700a1120 msgr2=0x7f6c700a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:44.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:44.341+0000 7f6c90000700 1 --2- 172.21.15.86:0/2139074508 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f6c700a1120 0x7f6c700a35e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6c74006010 tx=0x7f6c7400b540 comp rx=0 tx=0).stop 2024-03-23T17:48:44.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:44.341+0000 7f6c90000700 1 -- 172.21.15.86:0/2139074508 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c88100c80 msgr2=0x7f6c88074b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:44.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:44.341+0000 7f6c90000700 1 --2- 172.21.15.86:0/2139074508 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c88100c80 0x7f6c88074b10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c00e970 tx=0x7f6c7c00ec80 comp rx=0 tx=0).stop 2024-03-23T17:48:44.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:44.341+0000 7f6c90000700 1 -- 172.21.15.86:0/2139074508 shutdown_connections 2024-03-23T17:48:44.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:44.341+0000 7f6c90000700 1 --2- 172.21.15.86:0/2139074508 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f6c700a1120 0x7f6c700a35e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:44.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:44.341+0000 7f6c90000700 1 --2- 172.21.15.86:0/2139074508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c88102810 0x7f6c880736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:44.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:44.341+0000 7f6c90000700 1 --2- 172.21.15.86:0/2139074508 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c88100c80 0x7f6c88074b10 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:44.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:44.341+0000 7f6c90000700 1 --2- 172.21.15.86:0/2139074508 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6c88101e70 0x7f6c880731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:44.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:44.341+0000 7f6c90000700 1 -- 172.21.15.86:0/2139074508 >> 172.21.15.86:0/2139074508 conn(0x7f6c880fc1f0 msgr2=0x7f6c881051f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:48:44.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:44.341+0000 7f6c90000700 1 -- 172.21.15.86:0/2139074508 shutdown_connections 2024-03-23T17:48:44.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:44.341+0000 7f6c90000700 1 -- 172.21.15.86:0/2139074508 wait complete. 2024-03-23T17:48:44.343 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 10 2024-03-23T17:48:45.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:44 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/2139074508' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T17:48:45.017 INFO:journalctl@ceph.mds.g.smithi086.stdout:Mar 23 17:48:44 smithi086 systemd[1]: Starting Ceph mds.g for 9d76c9e8-e93c-11ee-95cd-87774f69a715... 2024-03-23T17:48:45.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:44 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/2139074508' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T17:48:45.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:44 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/2139074508' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T17:48:45.516 INFO:journalctl@ceph.mds.g.smithi086.stdout:Mar 23 17:48:45 smithi086 ceph-mds[90108]: -- 172.21.15.86:0/4231801107 <== mon.0 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562a773141a0 con 0x562a772ecc00 2024-03-23T17:48:45.516 INFO:journalctl@ceph.mds.g.smithi086.stdout:Mar 23 17:48:45 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-g[90085]: starting mds.g at 2024-03-23T17:48:45.517 INFO:journalctl@ceph.mds.g.smithi086.stdout:Mar 23 17:48:45 smithi086 bash[90070]: 1bf4fce38dbcb2ca903bd0d02c81030e46abdae88f3423d9ef32217d14310533 2024-03-23T17:48:45.517 INFO:journalctl@ceph.mds.g.smithi086.stdout:Mar 23 17:48:45 smithi086 systemd[1]: Started Ceph mds.g for 9d76c9e8-e93c-11ee-95cd-87774f69a715. 2024-03-23T17:48:46.035 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:45 smithi086 ceph-mon[38221]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T17:48:46.035 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:45 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:46.035 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:45 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:46.035 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:45 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:46.035 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:45 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T17:48:46.035 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:45 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T17:48:46.035 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:45 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:48:46.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:45 smithi092 ceph-mon[79089]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T17:48:46.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:45 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:46.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:45 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:46.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:45 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:46.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:45 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T17:48:46.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:45 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T17:48:46.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:45 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:48:46.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:45 smithi152 ceph-mon[47949]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T17:48:46.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:45 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:46.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:45 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:46.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:45 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:46.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:45 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T17:48:46.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:45 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T17:48:46.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:45 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:48:46.361 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T17:48:46.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:46.558+0000 7fc00a000700 1 -- 172.21.15.86:0/452601565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc004102810 msgr2=0x7fc0041090c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:46.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:46.558+0000 7fc00a000700 1 --2- 172.21.15.86:0/452601565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc004102810 0x7fc0041090c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbffc00b0d0 tx=0x7fbffc00b3e0 comp rx=0 tx=0).stop 2024-03-23T17:48:46.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:46.559+0000 7fc00a000700 1 -- 172.21.15.86:0/452601565 shutdown_connections 2024-03-23T17:48:46.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:46.559+0000 7fc00a000700 1 --2- 172.21.15.86:0/452601565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc004102810 0x7fc0041090c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:46.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:46.559+0000 7fc00a000700 1 --2- 172.21.15.86:0/452601565 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc004100c80 0x7fc0041010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:46.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:46.559+0000 7fc00a000700 1 --2- 172.21.15.86:0/452601565 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc004101e70 0x7fc0041022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:46.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:46.559+0000 7fc00a000700 1 -- 172.21.15.86:0/452601565 >> 172.21.15.86:0/452601565 conn(0x7fc0040fc1f0 msgr2=0x7fc0040fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:48:46.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:46.559+0000 7fc00a000700 1 -- 172.21.15.86:0/452601565 shutdown_connections 2024-03-23T17:48:46.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:46.559+0000 7fc00a000700 1 -- 172.21.15.86:0/452601565 wait complete. 2024-03-23T17:48:46.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:46.560+0000 7fc00a000700 1 Processor -- start 2024-03-23T17:48:46.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:46.561+0000 7fc00a000700 1 -- start start 2024-03-23T17:48:46.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:46.561+0000 7fc00a000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc004100c80 0x7fc004198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:46.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:46.561+0000 7fc00a000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc004101e70 0x7fc0041993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:46.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:46.561+0000 7fc00a000700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc004102810 0x7fc0041a0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:46.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:46.561+0000 7fc00a000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0041a09d0 con 0x7fc004100c80 2024-03-23T17:48:46.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:46.561+0000 7fc00a000700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0041a0b10 con 0x7fc004101e70 2024-03-23T17:48:46.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:46.561+0000 7fc00a000700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0041a0ca0 con 0x7fc004102810 2024-03-23T17:48:46.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:46.561+0000 7fc003400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc004100c80 0x7fc004198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:46.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:46.561+0000 7fc003400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc004100c80 0x7fc004198ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58136/0 (socket says 172.21.15.86:58136) 2024-03-23T17:48:46.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:46.561+0000 7fc003400700 1 -- 172.21.15.86:0/3895898999 learned_addr learned my addr 172.21.15.86:0/3895898999 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:48:46.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:46.561+0000 7fc002a00700 1 --2- 172.21.15.86:0/3895898999 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc004101e70 0x7fc0041993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:46.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:46.562+0000 7fc003e00700 1 --2- 172.21.15.86:0/3895898999 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc004102810 0x7fc0041a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:46.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:46.562+0000 7fc003400700 1 -- 172.21.15.86:0/3895898999 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc004102810 msgr2=0x7fc0041a0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:46.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:46.562+0000 7fc003400700 1 --2- 172.21.15.86:0/3895898999 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc004102810 0x7fc0041a0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:46.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:46.562+0000 7fc003400700 1 -- 172.21.15.86:0/3895898999 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc004101e70 msgr2=0x7fc0041993e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:46.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:46.562+0000 7fc003400700 1 --2- 172.21.15.86:0/3895898999 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc004101e70 0x7fc0041993e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:46.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:46.562+0000 7fc003400700 1 -- 172.21.15.86:0/3895898999 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbffc009d00 con 0x7fc004100c80 2024-03-23T17:48:46.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:46.562+0000 7fc002a00700 1 --2- 172.21.15.86:0/3895898999 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc004101e70 0x7fc0041993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:48:46.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:46.562+0000 7fc003400700 1 --2- 172.21.15.86:0/3895898999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc004100c80 0x7fc004198ea0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbff400e970 tx=0x7fbff400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:48:46.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:46.562+0000 7fbffbe00700 1 -- 172.21.15.86:0/3895898999 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff4009790 con 0x7fc004100c80 2024-03-23T17:48:46.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:46.562+0000 7fbffbe00700 1 -- 172.21.15.86:0/3895898999 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbff40044e0 con 0x7fc004100c80 2024-03-23T17:48:46.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:46.562+0000 7fc00a000700 1 -- 172.21.15.86:0/3895898999 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0041a0f80 con 0x7fc004100c80 2024-03-23T17:48:46.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:46.562+0000 7fbffbe00700 1 -- 172.21.15.86:0/3895898999 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff4003e30 con 0x7fc004100c80 2024-03-23T17:48:46.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:46.562+0000 7fc00a000700 1 -- 172.21.15.86:0/3895898999 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0041a13b0 con 0x7fc004100c80 2024-03-23T17:48:46.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:46.564+0000 7fc00a000700 1 -- 172.21.15.86:0/3895898999 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc004067a10 con 0x7fc004100c80 2024-03-23T17:48:46.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:46.567+0000 7fbffbe00700 1 -- 172.21.15.86:0/3895898999 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fbff4003f90 con 0x7fc004100c80 2024-03-23T17:48:46.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:46.568+0000 7fbffbe00700 1 --2- 172.21.15.86:0/3895898999 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fbfe80a11a0 0x7fbfe80a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:46.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:46.568+0000 7fbffbe00700 1 -- 172.21.15.86:0/3895898999 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fbff4014070 con 0x7fc004100c80 2024-03-23T17:48:46.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:46.569+0000 7fc002a00700 1 --2- 172.21.15.86:0/3895898999 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fbfe80a11a0 0x7fbfe80a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:46.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:46.570+0000 7fc002a00700 1 --2- 172.21.15.86:0/3895898999 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fbfe80a11a0 0x7fbfe80a3660 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbff0009cb0 tx=0x7fbff0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:48:46.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:46.575+0000 7fbffbe00700 1 -- 172.21.15.86:0/3895898999 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fbff40962c0 con 0x7fc004100c80 2024-03-23T17:48:46.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:46.862+0000 7fc00a000700 1 -- 172.21.15.86:0/3895898999 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc0041a1630 con 0x7fc004100c80 2024-03-23T17:48:46.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:46.863+0000 7fbffbe00700 1 -- 172.21.15.86:0/3895898999 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8082 (secure 0 0 0) 0x7fbff40998e0 con 0x7fc004100c80 2024-03-23T17:48:46.863 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:48:46.863 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14445,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1433140219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1433140219},{"type":"v1","addr":"172.21.15.86:6835","nonce":1433140219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6839/1879457730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1879457730},{"type":"v1","addr":"172.21.15.86:6839","nonce":1879457730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/1467091186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1467091186},{"type":"v1","addr":"172.21.15.92:6833","nonce":1467091186}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24475,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/3075554623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3075554623},{"type":"v1","addr":"172.21.15.152:6833","nonce":3075554623}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/908900027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":908900027},{"type":"v1","addr":"172.21.15.92:6835","nonce":908900027}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24505,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2933881328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2933881328},{"type":"v1","addr":"172.21.15.86:6837","nonce":2933881328}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24518,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/3967837948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":3967837948},{"type":"v1","addr":"172.21.15.92:6837","nonce":3967837948}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/3137085021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3137085021},{"type":"v1","addr":"172.21.15.152:6835","nonce":3137085021}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24538,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/1151208965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":1151208965},{"type":"v1","addr":"172.21.15.152:6837","nonce":1151208965}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24553,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6839/1100723992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":1100723992},{"type":"v1","addr":"172.21.15.152:6839","nonce":1100723992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2024-03-23T17:48:46.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:46.865+0000 7fc00a000700 1 -- 172.21.15.86:0/3895898999 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fbfe80a11a0 msgr2=0x7fbfe80a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:46.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:46.865+0000 7fc00a000700 1 --2- 172.21.15.86:0/3895898999 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fbfe80a11a0 0x7fbfe80a3660 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbff0009cb0 tx=0x7fbff0009260 comp rx=0 tx=0).stop 2024-03-23T17:48:46.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:46.866+0000 7fc00a000700 1 -- 172.21.15.86:0/3895898999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc004100c80 msgr2=0x7fc004198ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:46.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:46.866+0000 7fc00a000700 1 --2- 172.21.15.86:0/3895898999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc004100c80 0x7fc004198ea0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbff400e970 tx=0x7fbff400ed30 comp rx=0 tx=0).stop 2024-03-23T17:48:46.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:46.866+0000 7fc00a000700 1 -- 172.21.15.86:0/3895898999 shutdown_connections 2024-03-23T17:48:46.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:46.866+0000 7fc00a000700 1 --2- 172.21.15.86:0/3895898999 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc004102810 0x7fc0041a0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:46.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:46.866+0000 7fc00a000700 1 --2- 172.21.15.86:0/3895898999 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fbfe80a11a0 0x7fbfe80a3660 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:46.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:46.866+0000 7fc00a000700 1 --2- 172.21.15.86:0/3895898999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc004100c80 0x7fc004198ea0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:46.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:46.866+0000 7fc00a000700 1 --2- 172.21.15.86:0/3895898999 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc004101e70 0x7fc0041993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:46.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:46.866+0000 7fc00a000700 1 -- 172.21.15.86:0/3895898999 >> 172.21.15.86:0/3895898999 conn(0x7fc0040fc1f0 msgr2=0x7fc004072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:48:46.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:46.866+0000 7fc00a000700 1 -- 172.21.15.86:0/3895898999 shutdown_connections 2024-03-23T17:48:46.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:46.866+0000 7fc00a000700 1 -- 172.21.15.86:0/3895898999 wait complete. 2024-03-23T17:48:46.867 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 11 2024-03-23T17:48:46.984 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:46 smithi086 ceph-mon[38221]: Deploying daemon mds.a on smithi086 2024-03-23T17:48:46.984 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:46 smithi086 ceph-mon[38221]: mds.? [v2:172.21.15.86:6838/1879457730,v1:172.21.15.86:6839/1879457730] up:boot 2024-03-23T17:48:46.985 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:46 smithi086 ceph-mon[38221]: fsmap 10 up:standby 2024-03-23T17:48:46.985 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:46 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-23T17:48:47.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:46 smithi092 ceph-mon[79089]: Deploying daemon mds.a on smithi086 2024-03-23T17:48:47.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:46 smithi092 ceph-mon[79089]: mds.? [v2:172.21.15.86:6838/1879457730,v1:172.21.15.86:6839/1879457730] up:boot 2024-03-23T17:48:47.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:46 smithi092 ceph-mon[79089]: fsmap 10 up:standby 2024-03-23T17:48:47.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:46 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-23T17:48:47.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:46 smithi152 ceph-mon[47949]: Deploying daemon mds.a on smithi086 2024-03-23T17:48:47.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:46 smithi152 ceph-mon[47949]: mds.? [v2:172.21.15.86:6838/1879457730,v1:172.21.15.86:6839/1879457730] up:boot 2024-03-23T17:48:47.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:46 smithi152 ceph-mon[47949]: fsmap 10 up:standby 2024-03-23T17:48:47.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:46 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-23T17:48:47.266 INFO:journalctl@ceph.mds.a.smithi086.stdout:Mar 23 17:48:47 smithi086 systemd[1]: Starting Ceph mds.a for 9d76c9e8-e93c-11ee-95cd-87774f69a715... 2024-03-23T17:48:48.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:47 smithi086 ceph-mon[38221]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T17:48:48.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:47 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/3895898999' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T17:48:48.017 INFO:journalctl@ceph.mds.a.smithi086.stdout:Mar 23 17:48:47 smithi086 ceph-mds[90497]: -- 172.21.15.86:0/2360834706 <== mon.0 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5633c19921a0 con 0x5633c196ac00 2024-03-23T17:48:48.017 INFO:journalctl@ceph.mds.a.smithi086.stdout:Mar 23 17:48:47 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-a[90474]: starting mds.a at 2024-03-23T17:48:48.017 INFO:journalctl@ceph.mds.a.smithi086.stdout:Mar 23 17:48:47 smithi086 bash[90459]: 409beb7eb2e7145bc18fd501d602f85f8c50fa06aa615f81e483680fe8f8274c 2024-03-23T17:48:48.017 INFO:journalctl@ceph.mds.a.smithi086.stdout:Mar 23 17:48:47 smithi086 systemd[1]: Started Ceph mds.a for 9d76c9e8-e93c-11ee-95cd-87774f69a715. 2024-03-23T17:48:48.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:47 smithi092 ceph-mon[79089]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T17:48:48.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:47 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/3895898999' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T17:48:48.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:47 smithi152 ceph-mon[47949]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T17:48:48.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:47 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/3895898999' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T17:48:48.886 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T17:48:48.969 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:48 smithi092 ceph-mon[79089]: mds.? [v2:172.21.15.86:6840/3026943866,v1:172.21.15.86:6841/3026943866] up:boot 2024-03-23T17:48:48.970 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:48 smithi092 ceph-mon[79089]: fsmap 11 up:standby 2024-03-23T17:48:48.970 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:48 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-23T17:48:48.970 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:48 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:48.970 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:48 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:48.970 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:48 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:48.970 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:48 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T17:48:48.970 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:48 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T17:48:48.970 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:48 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:48:49.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:48 smithi152 ceph-mon[47949]: mds.? [v2:172.21.15.86:6840/3026943866,v1:172.21.15.86:6841/3026943866] up:boot 2024-03-23T17:48:49.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:48 smithi152 ceph-mon[47949]: fsmap 11 up:standby 2024-03-23T17:48:49.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:48 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-23T17:48:49.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:48 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:49.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:48 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:49.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:48 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:49.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:48 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T17:48:49.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:48 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T17:48:49.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:48 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:48:49.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:49.096+0000 7f0f1d000700 1 -- 172.21.15.86:0/3545431579 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f18100c80 msgr2=0x7f0f181010a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:49.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:49.096+0000 7f0f1d000700 1 --2- 172.21.15.86:0/3545431579 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f18100c80 0x7f0f181010a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0f080099c0 tx=0x7f0f08009cd0 comp rx=0 tx=0).stop 2024-03-23T17:48:49.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:49.096+0000 7f0f1d000700 1 -- 172.21.15.86:0/3545431579 shutdown_connections 2024-03-23T17:48:49.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:49.096+0000 7f0f1d000700 1 --2- 172.21.15.86:0/3545431579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f18102810 0x7f0f181090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:49.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:49.096+0000 7f0f1d000700 1 --2- 172.21.15.86:0/3545431579 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f18100c80 0x7f0f181010a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:49.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:49.096+0000 7f0f1d000700 1 --2- 172.21.15.86:0/3545431579 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f18101e70 0x7f0f181022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:49.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:49.096+0000 7f0f1d000700 1 -- 172.21.15.86:0/3545431579 >> 172.21.15.86:0/3545431579 conn(0x7f0f180fc1f0 msgr2=0x7f0f180fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:48:49.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:49.096+0000 7f0f1d000700 1 -- 172.21.15.86:0/3545431579 shutdown_connections 2024-03-23T17:48:49.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:49.096+0000 7f0f1d000700 1 -- 172.21.15.86:0/3545431579 wait complete. 2024-03-23T17:48:49.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:49.097+0000 7f0f1d000700 1 Processor -- start 2024-03-23T17:48:49.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:49.098+0000 7f0f1d000700 1 -- start start 2024-03-23T17:48:49.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:49.098+0000 7f0f1d000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f18100c80 0x7f0f18074ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:49.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:49.098+0000 7f0f1d000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f18101e70 0x7f0f180731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:49.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:49.098+0000 7f0f1d000700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f18102810 0x7f0f180736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:49.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:49.098+0000 7f0f1d000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f18073de0 con 0x7f0f18101e70 2024-03-23T17:48:49.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:49.098+0000 7f0f1d000700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f18073f50 con 0x7f0f18100c80 2024-03-23T17:48:49.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:49.098+0000 7f0f1d000700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f181a0d10 con 0x7f0f18102810 2024-03-23T17:48:49.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:49.098+0000 7f0f15e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f18100c80 0x7f0f18074ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:49.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:49.098+0000 7f0f15400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f18101e70 0x7f0f180731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:49.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:49.098+0000 7f0f15e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f18100c80 0x7f0f18074ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:35652/0 (socket says 172.21.15.86:35652) 2024-03-23T17:48:49.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:49.098+0000 7f0f15e00700 1 -- 172.21.15.86:0/2655708361 learned_addr learned my addr 172.21.15.86:0/2655708361 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:48:49.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:49.098+0000 7f0f16800700 1 --2- 172.21.15.86:0/2655708361 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f18102810 0x7f0f180736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:49.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:49.098+0000 7f0f15e00700 1 -- 172.21.15.86:0/2655708361 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f18102810 msgr2=0x7f0f180736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:49.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:49.098+0000 7f0f15e00700 1 --2- 172.21.15.86:0/2655708361 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f18102810 0x7f0f180736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:49.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:49.098+0000 7f0f15e00700 1 -- 172.21.15.86:0/2655708361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f18101e70 msgr2=0x7f0f180731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:49.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:49.098+0000 7f0f15e00700 1 --2- 172.21.15.86:0/2655708361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f18101e70 0x7f0f180731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:49.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:49.098+0000 7f0f15e00700 1 -- 172.21.15.86:0/2655708361 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f080095f0 con 0x7f0f18100c80 2024-03-23T17:48:49.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:49.098+0000 7f0f15400700 1 --2- 172.21.15.86:0/2655708361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f18101e70 0x7f0f180731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:48:49.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:49.099+0000 7f0f15e00700 1 --2- 172.21.15.86:0/2655708361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f18100c80 0x7f0f18074ac0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0f08006010 tx=0x7f0f0800f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:48:49.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:49.099+0000 7f0f0ea00700 1 -- 172.21.15.86:0/2655708361 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f0801d070 con 0x7f0f18100c80 2024-03-23T17:48:49.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:49.099+0000 7f0f0ea00700 1 -- 172.21.15.86:0/2655708361 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f08005020 con 0x7f0f18100c80 2024-03-23T17:48:49.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:49.099+0000 7f0f1d000700 1 -- 172.21.15.86:0/2655708361 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f181a0f10 con 0x7f0f18100c80 2024-03-23T17:48:49.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:49.099+0000 7f0f0ea00700 1 -- 172.21.15.86:0/2655708361 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f08004bb0 con 0x7f0f18100c80 2024-03-23T17:48:49.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:49.099+0000 7f0f1d000700 1 -- 172.21.15.86:0/2655708361 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f181a1280 con 0x7f0f18100c80 2024-03-23T17:48:49.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:49.100+0000 7f0f1d000700 1 -- 172.21.15.86:0/2655708361 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f18067a80 con 0x7f0f18100c80 2024-03-23T17:48:49.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:49.102+0000 7f0f0ea00700 1 -- 172.21.15.86:0/2655708361 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f0f08005190 con 0x7f0f18100c80 2024-03-23T17:48:49.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:49.103+0000 7f0f0ea00700 1 --2- 172.21.15.86:0/2655708361 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f0efc0a1120 0x7f0efc0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:49.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:49.103+0000 7f0f0ea00700 1 -- 172.21.15.86:0/2655708361 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f0f080ccf70 con 0x7f0f18100c80 2024-03-23T17:48:49.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:49.103+0000 7f0f15400700 1 --2- 172.21.15.86:0/2655708361 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f0efc0a1120 0x7f0efc0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:49.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:49.104+0000 7f0f15400700 1 --2- 172.21.15.86:0/2655708361 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f0efc0a1120 0x7f0efc0a35e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0f04011490 tx=0x7f0f04009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:48:49.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:49.108+0000 7f0f0ea00700 1 -- 172.21.15.86:0/2655708361 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f0f18067a80 con 0x7f0f18100c80 2024-03-23T17:48:49.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:48 smithi086 ceph-mon[38221]: mds.? [v2:172.21.15.86:6840/3026943866,v1:172.21.15.86:6841/3026943866] up:boot 2024-03-23T17:48:49.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:48 smithi086 ceph-mon[38221]: fsmap 11 up:standby 2024-03-23T17:48:49.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:48 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-23T17:48:49.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:48 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:49.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:48 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:49.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:48 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:49.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:48 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T17:48:49.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:48 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T17:48:49.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:48 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:48:49.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:49.360+0000 7f0f1d000700 1 -- 172.21.15.86:0/2655708361 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0f18067a10 con 0x7f0f18100c80 2024-03-23T17:48:49.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:49.361+0000 7f0f0ea00700 1 -- 172.21.15.86:0/2655708361 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+8840 (secure 0 0 0) 0x7f0f08096b90 con 0x7f0f18100c80 2024-03-23T17:48:49.361 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:48:49.361 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14445,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1433140219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1433140219},{"type":"v1","addr":"172.21.15.86:6835","nonce":1433140219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6839/1879457730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1879457730},{"type":"v1","addr":"172.21.15.86:6839","nonce":1879457730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6841/3026943866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3026943866},{"type":"v1","addr":"172.21.15.86:6841","nonce":3026943866}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/1467091186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1467091186},{"type":"v1","addr":"172.21.15.92:6833","nonce":1467091186}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24475,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/3075554623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3075554623},{"type":"v1","addr":"172.21.15.152:6833","nonce":3075554623}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/908900027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":908900027},{"type":"v1","addr":"172.21.15.92:6835","nonce":908900027}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24505,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2933881328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2933881328},{"type":"v1","addr":"172.21.15.86:6837","nonce":2933881328}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24518,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/3967837948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":3967837948},{"type":"v1","addr":"172.21.15.92:6837","nonce":3967837948}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/3137085021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3137085021},{"type":"v1","addr":"172.21.15.152:6835","nonce":3137085021}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24538,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/1151208965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":1151208965},{"type":"v1","addr":"172.21.15.152:6837","nonce":1151208965}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24553,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6839/1100723992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":1100723992},{"type":"v1","addr":"172.21.15.152:6839","nonce":1100723992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2024-03-23T17:48:49.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:49.363+0000 7f0f1d000700 1 -- 172.21.15.86:0/2655708361 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f0efc0a1120 msgr2=0x7f0efc0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:49.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:49.363+0000 7f0f1d000700 1 --2- 172.21.15.86:0/2655708361 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f0efc0a1120 0x7f0efc0a35e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0f04011490 tx=0x7f0f04009260 comp rx=0 tx=0).stop 2024-03-23T17:48:49.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:49.363+0000 7f0f1d000700 1 -- 172.21.15.86:0/2655708361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f18100c80 msgr2=0x7f0f18074ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:49.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:49.363+0000 7f0f1d000700 1 --2- 172.21.15.86:0/2655708361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f18100c80 0x7f0f18074ac0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0f08006010 tx=0x7f0f0800f690 comp rx=0 tx=0).stop 2024-03-23T17:48:49.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:49.363+0000 7f0f1d000700 1 -- 172.21.15.86:0/2655708361 shutdown_connections 2024-03-23T17:48:49.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:49.363+0000 7f0f1d000700 1 --2- 172.21.15.86:0/2655708361 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f18102810 0x7f0f180736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:49.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:49.363+0000 7f0f1d000700 1 --2- 172.21.15.86:0/2655708361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f18100c80 0x7f0f18074ac0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:49.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:49.363+0000 7f0f1d000700 1 --2- 172.21.15.86:0/2655708361 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f0efc0a1120 0x7f0efc0a35e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:49.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:49.363+0000 7f0f1d000700 1 --2- 172.21.15.86:0/2655708361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f18101e70 0x7f0f180731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:49.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:49.363+0000 7f0f1d000700 1 -- 172.21.15.86:0/2655708361 >> 172.21.15.86:0/2655708361 conn(0x7f0f180fc1f0 msgr2=0x7f0f181051e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:48:49.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:49.363+0000 7f0f1d000700 1 -- 172.21.15.86:0/2655708361 shutdown_connections 2024-03-23T17:48:49.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:49.363+0000 7f0f1d000700 1 -- 172.21.15.86:0/2655708361 wait complete. 2024-03-23T17:48:49.365 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 12 2024-03-23T17:48:49.557 INFO:journalctl@ceph.mds.b.smithi092.stdout:Mar 23 17:48:49 smithi092 systemd[1]: Starting Ceph mds.b for 9d76c9e8-e93c-11ee-95cd-87774f69a715... 2024-03-23T17:48:50.057 INFO:journalctl@ceph.mds.b.smithi092.stdout:Mar 23 17:48:49 smithi092 ceph-mds[115145]: -- 172.21.15.92:0/1000967379 <== mon.2 v2:172.21.15.92:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c8422121a0 con 0x55c8421e8800 2024-03-23T17:48:50.057 INFO:journalctl@ceph.mds.b.smithi092.stdout:Mar 23 17:48:49 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-b[115122]: starting mds.b at 2024-03-23T17:48:50.057 INFO:journalctl@ceph.mds.b.smithi092.stdout:Mar 23 17:48:49 smithi092 bash[115107]: 60fd5234d59f09cb2b2ed0bbc254c5495536bd872fa7e9fc3962007a31d6e6d9 2024-03-23T17:48:50.057 INFO:journalctl@ceph.mds.b.smithi092.stdout:Mar 23 17:48:49 smithi092 systemd[1]: Started Ceph mds.b for 9d76c9e8-e93c-11ee-95cd-87774f69a715. 2024-03-23T17:48:50.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:49 smithi092 ceph-mon[79089]: Deploying daemon mds.b on smithi092 2024-03-23T17:48:50.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:49 smithi092 ceph-mon[79089]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T17:48:50.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:49 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/2655708361' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T17:48:50.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:49 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:50.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:49 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:50.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:49 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:50.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:49 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:50.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:49 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:50.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:49 smithi152 ceph-mon[47949]: Deploying daemon mds.b on smithi092 2024-03-23T17:48:50.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:49 smithi152 ceph-mon[47949]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T17:48:50.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:49 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/2655708361' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T17:48:50.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:49 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:50.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:49 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:50.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:49 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:50.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:49 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:50.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:49 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:50.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:49 smithi086 ceph-mon[38221]: Deploying daemon mds.b on smithi092 2024-03-23T17:48:50.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:49 smithi086 ceph-mon[38221]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T17:48:50.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:49 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/2655708361' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T17:48:50.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:49 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:50.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:49 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:50.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:49 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:50.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:49 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:50.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:49 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:51.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:50 smithi086 ceph-mon[38221]: mds.? [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] up:boot 2024-03-23T17:48:51.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:50 smithi086 ceph-mon[38221]: fsmap 12 up:standby 2024-03-23T17:48:51.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:50 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-23T17:48:51.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:50 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:48:51.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:50 smithi086 ceph-mon[38221]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T17:48:51.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:50 smithi092 ceph-mon[79089]: mds.? [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] up:boot 2024-03-23T17:48:51.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:50 smithi092 ceph-mon[79089]: fsmap 12 up:standby 2024-03-23T17:48:51.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:50 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-23T17:48:51.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:50 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:48:51.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:50 smithi092 ceph-mon[79089]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T17:48:51.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:50 smithi152 ceph-mon[47949]: mds.? [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] up:boot 2024-03-23T17:48:51.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:50 smithi152 ceph-mon[47949]: fsmap 12 up:standby 2024-03-23T17:48:51.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:50 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-23T17:48:51.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:50 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:48:51.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:50 smithi152 ceph-mon[47949]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T17:48:51.383 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T17:48:51.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:51.597+0000 7f5605400700 1 -- 172.21.15.86:0/4256380749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f560010d250 msgr2=0x7f560010f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:51.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:51.597+0000 7f5605400700 1 --2- 172.21.15.86:0/4256380749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f560010d250 0x7f560010f640 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f55f800b0a0 tx=0x7f55f800b3b0 comp rx=0 tx=0).stop 2024-03-23T17:48:51.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:51.597+0000 7f5605400700 1 -- 172.21.15.86:0/4256380749 shutdown_connections 2024-03-23T17:48:51.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:51.597+0000 7f5605400700 1 --2- 172.21.15.86:0/4256380749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f560010d250 0x7f560010f640 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:51.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:51.597+0000 7f5605400700 1 --2- 172.21.15.86:0/4256380749 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5600074df0 0x7f5600073240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:51.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:51.597+0000 7f5605400700 1 --2- 172.21.15.86:0/4256380749 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5600073810 0x7f5600073c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:51.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:51.597+0000 7f5605400700 1 -- 172.21.15.86:0/4256380749 >> 172.21.15.86:0/4256380749 conn(0x7f56000fc1f0 msgr2=0x7f56000fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:48:51.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:51.597+0000 7f5605400700 1 -- 172.21.15.86:0/4256380749 shutdown_connections 2024-03-23T17:48:51.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:51.597+0000 7f5605400700 1 -- 172.21.15.86:0/4256380749 wait complete. 2024-03-23T17:48:51.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:51.598+0000 7f5605400700 1 Processor -- start 2024-03-23T17:48:51.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:51.599+0000 7f5605400700 1 -- start start 2024-03-23T17:48:51.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:51.599+0000 7f5605400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5600073810 0x7f560019d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:51.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:51.599+0000 7f5605400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5600074df0 0x7f560019d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:51.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:51.599+0000 7f5605400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f560010d250 0x7f56001a4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:51.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:51.599+0000 7f5605400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56001a4e30 con 0x7f560010d250 2024-03-23T17:48:51.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:51.599+0000 7f5605400700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56001a4f70 con 0x7f5600074df0 2024-03-23T17:48:51.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:51.599+0000 7f5605400700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56001a5100 con 0x7f5600073810 2024-03-23T17:48:51.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:51.599+0000 7f55fe800700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5600073810 0x7f560019d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:51.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:51.599+0000 7f55ff200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f560010d250 0x7f56001a4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:51.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:51.599+0000 7f55fe800700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5600073810 0x7f560019d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:32798/0 (socket says 172.21.15.86:32798) 2024-03-23T17:48:51.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:51.599+0000 7f55fe800700 1 -- 172.21.15.86:0/2769607491 learned_addr learned my addr 172.21.15.86:0/2769607491 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:48:51.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:51.599+0000 7f55fde00700 1 --2- 172.21.15.86:0/2769607491 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5600074df0 0x7f560019d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:51.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:51.600+0000 7f55fe800700 1 -- 172.21.15.86:0/2769607491 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5600074df0 msgr2=0x7f560019d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:51.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:51.600+0000 7f55fe800700 1 --2- 172.21.15.86:0/2769607491 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5600074df0 0x7f560019d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:51.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:51.600+0000 7f55fe800700 1 -- 172.21.15.86:0/2769607491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f560010d250 msgr2=0x7f56001a4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:51.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:51.600+0000 7f55fe800700 1 --2- 172.21.15.86:0/2769607491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f560010d250 0x7f56001a4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:51.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:51.600+0000 7f55fe800700 1 -- 172.21.15.86:0/2769607491 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55f8009d00 con 0x7f5600073810 2024-03-23T17:48:51.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:51.600+0000 7f55fe800700 1 --2- 172.21.15.86:0/2769607491 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5600073810 0x7f560019d300 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f55f000e9a0 tx=0x7f55f000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:48:51.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:51.600+0000 7f55f7400700 1 -- 172.21.15.86:0/2769607491 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f0009790 con 0x7f5600073810 2024-03-23T17:48:51.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:51.600+0000 7f5605400700 1 -- 172.21.15.86:0/2769607491 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56001a53e0 con 0x7f5600073810 2024-03-23T17:48:51.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:51.600+0000 7f55f7400700 1 -- 172.21.15.86:0/2769607491 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55f0010430 con 0x7f5600073810 2024-03-23T17:48:51.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:51.600+0000 7f55f7400700 1 -- 172.21.15.86:0/2769607491 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f0018680 con 0x7f5600073810 2024-03-23T17:48:51.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:51.600+0000 7f5605400700 1 -- 172.21.15.86:0/2769607491 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56001a58f0 con 0x7f5600073810 2024-03-23T17:48:51.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:51.602+0000 7f5605400700 1 -- 172.21.15.86:0/2769607491 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56001034b0 con 0x7f5600073810 2024-03-23T17:48:51.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:51.603+0000 7f55f7400700 1 -- 172.21.15.86:0/2769607491 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f55f00105a0 con 0x7f5600073810 2024-03-23T17:48:51.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:51.604+0000 7f55f7400700 1 --2- 172.21.15.86:0/2769607491 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f55e40a1120 0x7f55e40a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:51.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:51.605+0000 7f55f7400700 1 -- 172.21.15.86:0/2769607491 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f55f0014070 con 0x7f5600073810 2024-03-23T17:48:51.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:51.605+0000 7f55fde00700 1 --2- 172.21.15.86:0/2769607491 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f55e40a1120 0x7f55e40a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:51.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:51.605+0000 7f55fde00700 1 --2- 172.21.15.86:0/2769607491 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f55e40a1120 0x7f55e40a35e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5600074b10 tx=0x7f55ec009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:48:51.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:51.608+0000 7f55f7400700 1 -- 172.21.15.86:0/2769607491 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f55f00529f0 con 0x7f5600073810 2024-03-23T17:48:51.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:51.873+0000 7f5605400700 1 -- 172.21.15.86:0/2769607491 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5600067a10 con 0x7f5600073810 2024-03-23T17:48:51.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:51.874+0000 7f55f7400700 1 -- 172.21.15.86:0/2769607491 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+9598 (secure 0 0 0) 0x7f55f0096220 con 0x7f5600073810 2024-03-23T17:48:51.874 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:48:51.874 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14445,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1433140219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1433140219},{"type":"v1","addr":"172.21.15.86:6835","nonce":1433140219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6839/1879457730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1879457730},{"type":"v1","addr":"172.21.15.86:6839","nonce":1879457730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6841/3026943866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3026943866},{"type":"v1","addr":"172.21.15.86:6841","nonce":3026943866}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/1467091186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1467091186},{"type":"v1","addr":"172.21.15.92:6833","nonce":1467091186}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24475,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/3075554623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3075554623},{"type":"v1","addr":"172.21.15.152:6833","nonce":3075554623}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/908900027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":908900027},{"type":"v1","addr":"172.21.15.92:6835","nonce":908900027}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24505,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2933881328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2933881328},{"type":"v1","addr":"172.21.15.86:6837","nonce":2933881328}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24518,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/3967837948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":3967837948},{"type":"v1","addr":"172.21.15.92:6837","nonce":3967837948}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/3137085021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3137085021},{"type":"v1","addr":"172.21.15.152:6835","nonce":3137085021}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24538,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/1151208965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":1151208965},{"type":"v1","addr":"172.21.15.152:6837","nonce":1151208965}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24553,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6839/1100723992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":1100723992},{"type":"v1","addr":"172.21.15.152:6839","nonce":1100723992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24563,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6839/1394139887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1394139887},{"type":"v1","addr":"172.21.15.92:6839","nonce":1394139887}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[]} 2024-03-23T17:48:51.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:51.876+0000 7f5605400700 1 -- 172.21.15.86:0/2769607491 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f55e40a1120 msgr2=0x7f55e40a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:51.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:51.876+0000 7f5605400700 1 --2- 172.21.15.86:0/2769607491 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f55e40a1120 0x7f55e40a35e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5600074b10 tx=0x7f55ec009260 comp rx=0 tx=0).stop 2024-03-23T17:48:51.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:51.876+0000 7f5605400700 1 -- 172.21.15.86:0/2769607491 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5600073810 msgr2=0x7f560019d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:51.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:51.876+0000 7f5605400700 1 --2- 172.21.15.86:0/2769607491 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5600073810 0x7f560019d300 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f55f000e9a0 tx=0x7f55f000ecb0 comp rx=0 tx=0).stop 2024-03-23T17:48:51.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:51.876+0000 7f5605400700 1 -- 172.21.15.86:0/2769607491 shutdown_connections 2024-03-23T17:48:51.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:51.876+0000 7f5605400700 1 --2- 172.21.15.86:0/2769607491 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f55e40a1120 0x7f55e40a35e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:51.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:51.876+0000 7f5605400700 1 --2- 172.21.15.86:0/2769607491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f560010d250 0x7f56001a4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:51.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:51.876+0000 7f5605400700 1 --2- 172.21.15.86:0/2769607491 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5600073810 0x7f560019d300 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:51.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:51.876+0000 7f5605400700 1 --2- 172.21.15.86:0/2769607491 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5600074df0 0x7f560019d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:51.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:51.876+0000 7f5605400700 1 -- 172.21.15.86:0/2769607491 >> 172.21.15.86:0/2769607491 conn(0x7f56000fc1f0 msgr2=0x7f56001005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:48:51.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:51.876+0000 7f5605400700 1 -- 172.21.15.86:0/2769607491 shutdown_connections 2024-03-23T17:48:51.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:51.876+0000 7f5605400700 1 -- 172.21.15.86:0/2769607491 wait complete. 2024-03-23T17:48:51.878 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 13 2024-03-23T17:48:51.896 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T17:48:52.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.102+0000 7f77f7200700 1 -- 172.21.15.86:0/3033868817 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f77f0100c80 msgr2=0x7f77f01010a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:52.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.102+0000 7f77f7200700 1 --2- 172.21.15.86:0/3033868817 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f77f0100c80 0x7f77f01010a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f77e0009990 tx=0x7f77e0009ca0 comp rx=0 tx=0).stop 2024-03-23T17:48:52.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.102+0000 7f77f7200700 1 -- 172.21.15.86:0/3033868817 shutdown_connections 2024-03-23T17:48:52.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.102+0000 7f77f7200700 1 --2- 172.21.15.86:0/3033868817 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77f0102810 0x7f77f01090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:52.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.102+0000 7f77f7200700 1 --2- 172.21.15.86:0/3033868817 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f77f0100c80 0x7f77f01010a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:52.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.102+0000 7f77f7200700 1 --2- 172.21.15.86:0/3033868817 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77f0101e70 0x7f77f01022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:52.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.102+0000 7f77f7200700 1 -- 172.21.15.86:0/3033868817 >> 172.21.15.86:0/3033868817 conn(0x7f77f00fc1f0 msgr2=0x7f77f00fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:48:52.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.102+0000 7f77f7200700 1 -- 172.21.15.86:0/3033868817 shutdown_connections 2024-03-23T17:48:52.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.102+0000 7f77f7200700 1 -- 172.21.15.86:0/3033868817 wait complete. 2024-03-23T17:48:52.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.103+0000 7f77f7200700 1 Processor -- start 2024-03-23T17:48:52.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.103+0000 7f77f7200700 1 -- start start 2024-03-23T17:48:52.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.103+0000 7f77f7200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77f0100c80 0x7f77f0198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:52.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.104+0000 7f77f7200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77f0101e70 0x7f77f0199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:52.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.104+0000 7f77f7200700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f77f0102810 0x7f77f01a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:52.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.104+0000 7f77f7200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f01a08f0 con 0x7f77f0100c80 2024-03-23T17:48:52.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.104+0000 7f77f7200700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f01a0a30 con 0x7f77f0101e70 2024-03-23T17:48:52.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.104+0000 7f77f7200700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f01a0bc0 con 0x7f77f0102810 2024-03-23T17:48:52.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.104+0000 7f77efe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77f0100c80 0x7f77f0198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:52.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.104+0000 7f77ef400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77f0101e70 0x7f77f0199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:52.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.104+0000 7f77f5200700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f77f0102810 0x7f77f01a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:52.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.104+0000 7f77ef400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77f0101e70 0x7f77f0199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:35698/0 (socket says 172.21.15.86:35698) 2024-03-23T17:48:52.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.104+0000 7f77ef400700 1 -- 172.21.15.86:0/89932496 learned_addr learned my addr 172.21.15.86:0/89932496 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:48:52.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.104+0000 7f77ef400700 1 -- 172.21.15.86:0/89932496 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f77f0102810 msgr2=0x7f77f01a0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:52.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.104+0000 7f77ef400700 1 --2- 172.21.15.86:0/89932496 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f77f0102810 0x7f77f01a0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:52.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.104+0000 7f77ef400700 1 -- 172.21.15.86:0/89932496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77f0100c80 msgr2=0x7f77f0198dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:52.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.104+0000 7f77ef400700 1 --2- 172.21.15.86:0/89932496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77f0100c80 0x7f77f0198dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:52.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.104+0000 7f77ef400700 1 -- 172.21.15.86:0/89932496 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77e00095f0 con 0x7f77f0101e70 2024-03-23T17:48:52.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.104+0000 7f77efe00700 1 --2- 172.21.15.86:0/89932496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77f0100c80 0x7f77f0198dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:48:52.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.104+0000 7f77ef400700 1 --2- 172.21.15.86:0/89932496 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77f0101e70 0x7f77f0199300 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f77dc00e9a0 tx=0x7f77dc00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:48:52.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.105+0000 7f77ecc00700 1 -- 172.21.15.86:0/89932496 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77dc009790 con 0x7f77f0101e70 2024-03-23T17:48:52.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.105+0000 7f77ecc00700 1 -- 172.21.15.86:0/89932496 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77dc010430 con 0x7f77f0101e70 2024-03-23T17:48:52.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.105+0000 7f77ecc00700 1 -- 172.21.15.86:0/89932496 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77dc018680 con 0x7f77f0101e70 2024-03-23T17:48:52.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.105+0000 7f77f7200700 1 -- 172.21.15.86:0/89932496 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77f01a0ea0 con 0x7f77f0101e70 2024-03-23T17:48:52.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.105+0000 7f77f7200700 1 -- 172.21.15.86:0/89932496 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77f01a12d0 con 0x7f77f0101e70 2024-03-23T17:48:52.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.106+0000 7f77f7200700 1 -- 172.21.15.86:0/89932496 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77f0067a10 con 0x7f77f0101e70 2024-03-23T17:48:52.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.108+0000 7f77ecc00700 1 -- 172.21.15.86:0/89932496 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f77dc0105a0 con 0x7f77f0101e70 2024-03-23T17:48:52.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.109+0000 7f77ecc00700 1 --2- 172.21.15.86:0/89932496 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f77d40a11a0 0x7f77d40a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:52.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.109+0000 7f77ecc00700 1 -- 172.21.15.86:0/89932496 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f77dc014070 con 0x7f77f0101e70 2024-03-23T17:48:52.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.109+0000 7f77efe00700 1 --2- 172.21.15.86:0/89932496 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f77d40a11a0 0x7f77d40a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:52.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.110+0000 7f77efe00700 1 --2- 172.21.15.86:0/89932496 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f77d40a11a0 0x7f77d40a3660 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f77e0000c00 tx=0x7f77e0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:48:52.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.112+0000 7f77ecc00700 1 -- 172.21.15.86:0/89932496 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f77dc0961d0 con 0x7f77f0101e70 2024-03-23T17:48:52.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:51 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:52.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:51 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:52.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:51 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:52.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:51 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:52.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:51 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:52.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:51 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:52.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:51 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:48:52.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:51 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:48:52.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:51 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:52.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:51 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/2769607491' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T17:48:52.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:51 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:48:52.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:51 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:52.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:51 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:52.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:51 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:52.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:51 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:52.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:51 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:52.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:51 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:52.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:51 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:48:52.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:51 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:48:52.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:51 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:52.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:51 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/2769607491' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T17:48:52.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:51 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:48:52.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:51 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:52.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:51 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:52.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:51 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:52.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:51 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:52.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:51 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:52.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:51 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:52.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:51 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:48:52.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:51 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:48:52.338 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:51 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:52.338 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:51 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/2769607491' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T17:48:52.338 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:51 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:48:52.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.364+0000 7f77f7200700 1 -- 172.21.15.86:0/89932496 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f77f010c6e0 con 0x7f77f0101e70 2024-03-23T17:48:52.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.365+0000 7f77ecc00700 1 -- 172.21.15.86:0/89932496 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+18386 (secure 0 0 0) 0x7f77dc0997f0 con 0x7f77f0101e70 2024-03-23T17:48:52.366 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:48:52.366 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":76,"fsid":"9d76c9e8-e93c-11ee-95cd-87774f69a715","created":"2024-03-23T17:42:30.097233+0000","modified":"2024-03-23T17:48:19.377461+0000","last_up_change":"2024-03-23T17:48:18.217601+0000","last_in_change":"2024-03-23T17:48:00.250579+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T17:45:26.848397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"656655c6-1c7f-4640-92c4-2f42de820143","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6803","nonce":2423505491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6805","nonce":2423505491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6809","nonce":2423505491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6807","nonce":2423505491}]},"public_addr":"172.21.15.86:6803/2423505491","cluster_addr":"172.21.15.86:6805/2423505491","heartbeat_back_addr":"172.21.15.86:6809/2423505491","heartbeat_front_addr":"172.21.15.86:6807/2423505491","state":["exists","up"]},{"osd":1,"uuid":"3e86dc65-20a6-4de8-b9ce-fcf01f68862b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6801","nonce":3776252597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6803","nonce":3776252597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6807","nonce":3776252597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6805","nonce":3776252597}]},"public_addr":"172.21.15.92:6801/3776252597","cluster_addr":"172.21.15.92:6803/3776252597","heartbeat_back_addr":"172.21.15.92:6807/3776252597","heartbeat_front_addr":"172.21.15.92:6805/3776252597","state":["exists","up"]},{"osd":2,"uuid":"09808786-3bd3-4d11-8c7b-1736119dfbb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6801","nonce":1601265891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6803","nonce":1601265891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6807","nonce":1601265891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6805","nonce":1601265891}]},"public_addr":"172.21.15.152:6801/1601265891","cluster_addr":"172.21.15.152:6803/1601265891","heartbeat_back_addr":"172.21.15.152:6807/1601265891","heartbeat_front_addr":"172.21.15.152:6805/1601265891","state":["exists","up"]},{"osd":3,"uuid":"79c61910-4b53-424c-b150-6f5385a2cd68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6811","nonce":694846466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6813","nonce":694846466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6817","nonce":694846466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6815","nonce":694846466}]},"public_addr":"172.21.15.86:6811/694846466","cluster_addr":"172.21.15.86:6813/694846466","heartbeat_back_addr":"172.21.15.86:6817/694846466","heartbeat_front_addr":"172.21.15.86:6815/694846466","state":["exists","up"]},{"osd":4,"uuid":"a2878035-0179-469f-bbd4-8ce2d3f6bddc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6809","nonce":1422821471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6811","nonce":1422821471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6815","nonce":1422821471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6813","nonce":1422821471}]},"public_addr":"172.21.15.92:6809/1422821471","cluster_addr":"172.21.15.92:6811/1422821471","heartbeat_back_addr":"172.21.15.92:6815/1422821471","heartbeat_front_addr":"172.21.15.92:6813/1422821471","state":["exists","up"]},{"osd":5,"uuid":"837366aa-ce25-42ca-b157-5bbb50159d0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6809","nonce":1039375239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6811","nonce":1039375239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6815","nonce":1039375239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6813","nonce":1039375239}]},"public_addr":"172.21.15.152:6809/1039375239","cluster_addr":"172.21.15.152:6811/1039375239","heartbeat_back_addr":"172.21.15.152:6815/1039375239","heartbeat_front_addr":"172.21.15.152:6813/1039375239","state":["exists","up"]},{"osd":6,"uuid":"61024437-7b91-432f-a1af-6443d0ef5213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6819","nonce":4080452707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6821","nonce":4080452707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6825","nonce":4080452707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6823","nonce":4080452707}]},"public_addr":"172.21.15.86:6819/4080452707","cluster_addr":"172.21.15.86:6821/4080452707","heartbeat_back_addr":"172.21.15.86:6825/4080452707","heartbeat_front_addr":"172.21.15.86:6823/4080452707","state":["exists","up"]},{"osd":7,"uuid":"2684bdae-6b30-4afc-af68-4ce41b3a75b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6817","nonce":4187707622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6819","nonce":4187707622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6823","nonce":4187707622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6821","nonce":4187707622}]},"public_addr":"172.21.15.92:6817/4187707622","cluster_addr":"172.21.15.92:6819/4187707622","heartbeat_back_addr":"172.21.15.92:6823/4187707622","heartbeat_front_addr":"172.21.15.92:6821/4187707622","state":["exists","up"]},{"osd":8,"uuid":"f03bbf0d-1281-4bcf-9773-9854388e65ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6817","nonce":3237612748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6819","nonce":3237612748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6823","nonce":3237612748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6821","nonce":3237612748}]},"public_addr":"172.21.15.152:6817/3237612748","cluster_addr":"172.21.15.152:6819/3237612748","heartbeat_back_addr":"172.21.15.152:6823/3237612748","heartbeat_front_addr":"172.21.15.152:6821/3237612748","state":["exists","up"]},{"osd":9,"uuid":"e7e8b775-99ff-48a4-a23a-91e7ab6622bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6827","nonce":2835218294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6829","nonce":2835218294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6833","nonce":2835218294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6831","nonce":2835218294}]},"public_addr":"172.21.15.86:6827/2835218294","cluster_addr":"172.21.15.86:6829/2835218294","heartbeat_back_addr":"172.21.15.86:6833/2835218294","heartbeat_front_addr":"172.21.15.86:6831/2835218294","state":["exists","up"]},{"osd":10,"uuid":"d9d616cd-a229-44de-8768-02a4415df115","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6825","nonce":4259465100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6827","nonce":4259465100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6831","nonce":4259465100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6829","nonce":4259465100}]},"public_addr":"172.21.15.92:6825/4259465100","cluster_addr":"172.21.15.92:6827/4259465100","heartbeat_back_addr":"172.21.15.92:6831/4259465100","heartbeat_front_addr":"172.21.15.92:6829/4259465100","state":["exists","up"]},{"osd":11,"uuid":"e7c0b865-ccf5-4233-8d22-0d9b6dbe5c97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6825","nonce":194662760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6827","nonce":194662760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6831","nonce":194662760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6829","nonce":194662760}]},"public_addr":"172.21.15.152:6825/194662760","cluster_addr":"172.21.15.152:6827/194662760","heartbeat_back_addr":"172.21.15.152:6831/194662760","heartbeat_front_addr":"172.21.15.152:6829/194662760","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:44:47.269167+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:45:05.363655+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:45:23.192677+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:45:44.156228+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:02.532971+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:20.153295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:41.713888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:59.676516+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:47:17.207087+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:47:38.970316+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:47:57.282525+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:48:15.437606+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/3553500912":"2024-03-24T17:43:28.242492+0000","172.21.15.86:6801/3600278228":"2024-03-24T17:43:28.242492+0000","172.21.15.86:0/887262329":"2024-03-24T17:43:28.242492+0000","172.21.15.86:6800/2313723492":"2024-03-24T17:43:05.043910+0000","172.21.15.86:6800/3600278228":"2024-03-24T17:43:28.242492+0000","172.21.15.86:0/3815705642":"2024-03-24T17:43:05.043910+0000","172.21.15.86:0/1614943062":"2024-03-24T17:43:28.242492+0000","172.21.15.86:6801/2313723492":"2024-03-24T17:43:05.043910+0000","172.21.15.86:0/2416321716":"2024-03-24T17:43:05.043910+0000","172.21.15.86:0/433588188":"2024-03-24T17:43:05.043910+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T17:48:52.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.368+0000 7f77f7200700 1 -- 172.21.15.86:0/89932496 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f77d40a11a0 msgr2=0x7f77d40a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:52.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.368+0000 7f77f7200700 1 --2- 172.21.15.86:0/89932496 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f77d40a11a0 0x7f77d40a3660 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f77e0000c00 tx=0x7f77e0005fb0 comp rx=0 tx=0).stop 2024-03-23T17:48:52.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.368+0000 7f77f7200700 1 -- 172.21.15.86:0/89932496 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77f0101e70 msgr2=0x7f77f0199300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:52.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.368+0000 7f77f7200700 1 --2- 172.21.15.86:0/89932496 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77f0101e70 0x7f77f0199300 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f77dc00e9a0 tx=0x7f77dc00ecb0 comp rx=0 tx=0).stop 2024-03-23T17:48:52.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.368+0000 7f77f7200700 1 -- 172.21.15.86:0/89932496 shutdown_connections 2024-03-23T17:48:52.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.368+0000 7f77f7200700 1 --2- 172.21.15.86:0/89932496 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f77f0102810 0x7f77f01a0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:52.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.368+0000 7f77f7200700 1 --2- 172.21.15.86:0/89932496 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f77d40a11a0 0x7f77d40a3660 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:52.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.368+0000 7f77f7200700 1 --2- 172.21.15.86:0/89932496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77f0100c80 0x7f77f0198dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:52.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.368+0000 7f77f7200700 1 --2- 172.21.15.86:0/89932496 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77f0101e70 0x7f77f0199300 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:52.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.368+0000 7f77f7200700 1 -- 172.21.15.86:0/89932496 >> 172.21.15.86:0/89932496 conn(0x7f77f00fc1f0 msgr2=0x7f77f0072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:48:52.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.368+0000 7f77f7200700 1 -- 172.21.15.86:0/89932496 shutdown_connections 2024-03-23T17:48:52.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.368+0000 7f77f7200700 1 -- 172.21.15.86:0/89932496 wait complete. 2024-03-23T17:48:52.389 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T17:45:26.848397+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T17:48:52.389 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T17:48:52.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.591+0000 7fba6ba00700 1 -- 172.21.15.86:0/76926877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba64104b20 msgr2=0x7fba6410b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:52.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.591+0000 7fba6ba00700 1 --2- 172.21.15.86:0/76926877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba64104b20 0x7fba6410b3d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fba5c00b0d0 tx=0x7fba5c00b3e0 comp rx=0 tx=0).stop 2024-03-23T17:48:52.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.591+0000 7fba6ba00700 1 -- 172.21.15.86:0/76926877 shutdown_connections 2024-03-23T17:48:52.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.591+0000 7fba6ba00700 1 --2- 172.21.15.86:0/76926877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba64104b20 0x7fba6410b3d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:52.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.591+0000 7fba6ba00700 1 --2- 172.21.15.86:0/76926877 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba64102f90 0x7fba641033b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:52.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.591+0000 7fba6ba00700 1 --2- 172.21.15.86:0/76926877 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fba64104180 0x7fba641045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:52.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.591+0000 7fba6ba00700 1 -- 172.21.15.86:0/76926877 >> 172.21.15.86:0/76926877 conn(0x7fba640fe500 msgr2=0x7fba64100940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:48:52.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.591+0000 7fba6ba00700 1 -- 172.21.15.86:0/76926877 shutdown_connections 2024-03-23T17:48:52.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.591+0000 7fba6ba00700 1 -- 172.21.15.86:0/76926877 wait complete. 2024-03-23T17:48:52.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.592+0000 7fba6ba00700 1 Processor -- start 2024-03-23T17:48:52.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.592+0000 7fba6ba00700 1 -- start start 2024-03-23T17:48:52.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.592+0000 7fba6ba00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba64102f90 0x7fba64198fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:52.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.592+0000 7fba6ba00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fba64104180 0x7fba64199520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:52.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.592+0000 7fba6ba00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba64104b20 0x7fba641a0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:52.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.592+0000 7fba6ba00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba641a0b10 con 0x7fba64102f90 2024-03-23T17:48:52.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.592+0000 7fba6ba00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba641a0c50 con 0x7fba64104b20 2024-03-23T17:48:52.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.592+0000 7fba6ba00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba641a0de0 con 0x7fba64104180 2024-03-23T17:48:52.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.593+0000 7fba69000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba64102f90 0x7fba64198fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:52.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.593+0000 7fba69000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba64102f90 0x7fba64198fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58214/0 (socket says 172.21.15.86:58214) 2024-03-23T17:48:52.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.593+0000 7fba69000700 1 -- 172.21.15.86:0/1842913566 learned_addr learned my addr 172.21.15.86:0/1842913566 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:48:52.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.593+0000 7fba63e00700 1 --2- 172.21.15.86:0/1842913566 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fba64104180 0x7fba64199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:52.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.593+0000 7fba69a00700 1 --2- 172.21.15.86:0/1842913566 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba64104b20 0x7fba641a0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:52.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.593+0000 7fba69000700 1 -- 172.21.15.86:0/1842913566 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fba64104180 msgr2=0x7fba64199520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:52.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.593+0000 7fba69000700 1 --2- 172.21.15.86:0/1842913566 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fba64104180 0x7fba64199520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:52.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.593+0000 7fba69000700 1 -- 172.21.15.86:0/1842913566 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba64104b20 msgr2=0x7fba641a0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:52.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.593+0000 7fba69000700 1 --2- 172.21.15.86:0/1842913566 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba64104b20 0x7fba641a0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:52.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.593+0000 7fba69000700 1 -- 172.21.15.86:0/1842913566 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba5c009d00 con 0x7fba64102f90 2024-03-23T17:48:52.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.593+0000 7fba63e00700 1 --2- 172.21.15.86:0/1842913566 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fba64104180 0x7fba64199520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:48:52.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.593+0000 7fba69000700 1 --2- 172.21.15.86:0/1842913566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba64102f90 0x7fba64198fe0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fba5800e970 tx=0x7fba5800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:48:52.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.593+0000 7fba61600700 1 -- 172.21.15.86:0/1842913566 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba58009790 con 0x7fba64102f90 2024-03-23T17:48:52.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.594+0000 7fba61600700 1 -- 172.21.15.86:0/1842913566 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba5800c7e0 con 0x7fba64102f90 2024-03-23T17:48:52.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.594+0000 7fba6ba00700 1 -- 172.21.15.86:0/1842913566 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba641a10c0 con 0x7fba64102f90 2024-03-23T17:48:52.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.594+0000 7fba61600700 1 -- 172.21.15.86:0/1842913566 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba58003c70 con 0x7fba64102f90 2024-03-23T17:48:52.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.594+0000 7fba6ba00700 1 -- 172.21.15.86:0/1842913566 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba641a15d0 con 0x7fba64102f90 2024-03-23T17:48:52.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.594+0000 7fba6ba00700 1 -- 172.21.15.86:0/1842913566 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba6410a340 con 0x7fba64102f90 2024-03-23T17:48:52.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.596+0000 7fba61600700 1 -- 172.21.15.86:0/1842913566 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fba5800cc50 con 0x7fba64102f90 2024-03-23T17:48:52.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.598+0000 7fba61600700 1 --2- 172.21.15.86:0/1842913566 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fba4c0a1120 0x7fba4c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:52.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.599+0000 7fba61600700 1 -- 172.21.15.86:0/1842913566 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fba58014070 con 0x7fba64102f90 2024-03-23T17:48:52.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.599+0000 7fba63e00700 1 --2- 172.21.15.86:0/1842913566 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fba4c0a1120 0x7fba4c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:52.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.599+0000 7fba63e00700 1 --2- 172.21.15.86:0/1842913566 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fba4c0a1120 0x7fba4c0a35e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fba50011490 tx=0x7fba50009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:48:52.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.603+0000 7fba61600700 1 -- 172.21.15.86:0/1842913566 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fba58096230 con 0x7fba64102f90 2024-03-23T17:48:52.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.832+0000 7fba6ba00700 1 -- 172.21.15.86:0/1842913566 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fba6410ea30 con 0x7fba64102f90 2024-03-23T17:48:52.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.833+0000 7fba61600700 1 -- 172.21.15.86:0/1842913566 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v76) v1 ==== 93+0+10 (secure 0 0 0) 0x7fba58003dd0 con 0x7fba64102f90 2024-03-23T17:48:52.833 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-03-23T17:48:52.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.835+0000 7fba6ba00700 1 -- 172.21.15.86:0/1842913566 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fba4c0a1120 msgr2=0x7fba4c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:52.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.835+0000 7fba6ba00700 1 --2- 172.21.15.86:0/1842913566 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fba4c0a1120 0x7fba4c0a35e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fba50011490 tx=0x7fba50009260 comp rx=0 tx=0).stop 2024-03-23T17:48:52.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.835+0000 7fba6ba00700 1 -- 172.21.15.86:0/1842913566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba64102f90 msgr2=0x7fba64198fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:52.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.835+0000 7fba6ba00700 1 --2- 172.21.15.86:0/1842913566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba64102f90 0x7fba64198fe0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fba5800e970 tx=0x7fba5800ec80 comp rx=0 tx=0).stop 2024-03-23T17:48:52.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.835+0000 7fba6ba00700 1 -- 172.21.15.86:0/1842913566 shutdown_connections 2024-03-23T17:48:52.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.835+0000 7fba6ba00700 1 --2- 172.21.15.86:0/1842913566 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba64104b20 0x7fba641a0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:52.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.835+0000 7fba6ba00700 1 --2- 172.21.15.86:0/1842913566 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fba4c0a1120 0x7fba4c0a35e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:52.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.835+0000 7fba6ba00700 1 --2- 172.21.15.86:0/1842913566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba64102f90 0x7fba64198fe0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:52.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.835+0000 7fba6ba00700 1 --2- 172.21.15.86:0/1842913566 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fba64104180 0x7fba64199520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:52.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.835+0000 7fba6ba00700 1 -- 172.21.15.86:0/1842913566 >> 172.21.15.86:0/1842913566 conn(0x7fba640fe500 msgr2=0x7fba64107480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:48:52.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.835+0000 7fba6ba00700 1 -- 172.21.15.86:0/1842913566 shutdown_connections 2024-03-23T17:48:52.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:52.835+0000 7fba6ba00700 1 -- 172.21.15.86:0/1842913566 wait complete. 2024-03-23T17:48:52.854 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-23T17:48:53.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.067+0000 7ff8e5c00700 1 -- 172.21.15.86:0/2511162168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8e0100c80 msgr2=0x7ff8e01010a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:53.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.067+0000 7ff8e5c00700 1 --2- 172.21.15.86:0/2511162168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8e0100c80 0x7ff8e01010a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff8d00099c0 tx=0x7ff8d0009cd0 comp rx=0 tx=0).stop 2024-03-23T17:48:53.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.067+0000 7ff8e5c00700 1 -- 172.21.15.86:0/2511162168 shutdown_connections 2024-03-23T17:48:53.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.067+0000 7ff8e5c00700 1 --2- 172.21.15.86:0/2511162168 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff8e0102810 0x7ff8e01090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:53.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.067+0000 7ff8e5c00700 1 --2- 172.21.15.86:0/2511162168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8e0100c80 0x7ff8e01010a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:53.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.067+0000 7ff8e5c00700 1 --2- 172.21.15.86:0/2511162168 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8e0101e70 0x7ff8e01022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:53.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.067+0000 7ff8e5c00700 1 -- 172.21.15.86:0/2511162168 >> 172.21.15.86:0/2511162168 conn(0x7ff8e00fc1f0 msgr2=0x7ff8e00fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:48:53.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.067+0000 7ff8e5c00700 1 -- 172.21.15.86:0/2511162168 shutdown_connections 2024-03-23T17:48:53.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.068+0000 7ff8e5c00700 1 -- 172.21.15.86:0/2511162168 wait complete. 2024-03-23T17:48:53.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.068+0000 7ff8e5c00700 1 Processor -- start 2024-03-23T17:48:53.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.069+0000 7ff8e5c00700 1 -- start start 2024-03-23T17:48:53.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.069+0000 7ff8e5c00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8e0100c80 0x7ff8e0198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:53.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.069+0000 7ff8e5c00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff8e0101e70 0x7ff8e01992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:53.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.069+0000 7ff8e5c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8e0102810 0x7ff8e01a0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:53.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.069+0000 7ff8e5c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e01a08e0 con 0x7ff8e0102810 2024-03-23T17:48:53.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.069+0000 7ff8e5c00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e01a0a20 con 0x7ff8e0100c80 2024-03-23T17:48:53.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.069+0000 7ff8e5c00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e01a0bb0 con 0x7ff8e0101e70 2024-03-23T17:48:53.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.069+0000 7ff8de000700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff8e0101e70 0x7ff8e01992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:53.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.069+0000 7ff8de000700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff8e0101e70 0x7ff8e01992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:32874/0 (socket says 172.21.15.86:32874) 2024-03-23T17:48:53.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.069+0000 7ff8de000700 1 -- 172.21.15.86:0/1876426331 learned_addr learned my addr 172.21.15.86:0/1876426331 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:48:53.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.069+0000 7ff8df400700 1 --2- 172.21.15.86:0/1876426331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8e0102810 0x7ff8e01a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:53.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.069+0000 7ff8dea00700 1 --2- 172.21.15.86:0/1876426331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8e0100c80 0x7ff8e0198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:53.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.070+0000 7ff8de000700 1 -- 172.21.15.86:0/1876426331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8e0100c80 msgr2=0x7ff8e0198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:53.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.070+0000 7ff8de000700 1 --2- 172.21.15.86:0/1876426331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8e0100c80 0x7ff8e0198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:53.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.070+0000 7ff8de000700 1 -- 172.21.15.86:0/1876426331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8e0102810 msgr2=0x7ff8e01a0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:53.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.070+0000 7ff8de000700 1 --2- 172.21.15.86:0/1876426331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8e0102810 0x7ff8e01a0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:53.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.070+0000 7ff8de000700 1 -- 172.21.15.86:0/1876426331 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8cc009520 con 0x7ff8e0101e70 2024-03-23T17:48:53.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.070+0000 7ff8de000700 1 --2- 172.21.15.86:0/1876426331 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff8e0101e70 0x7ff8e01992f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc00eb30 tx=0x7ff8cc00ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:48:53.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.070+0000 7ff8d7400700 1 -- 172.21.15.86:0/1876426331 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8cc009880 con 0x7ff8e0101e70 2024-03-23T17:48:53.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.070+0000 7ff8d7400700 1 -- 172.21.15.86:0/1876426331 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff8cc0044e0 con 0x7ff8e0101e70 2024-03-23T17:48:53.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.070+0000 7ff8e5c00700 1 -- 172.21.15.86:0/1876426331 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8d00095f0 con 0x7ff8e0101e70 2024-03-23T17:48:53.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.070+0000 7ff8d7400700 1 -- 172.21.15.86:0/1876426331 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8cc003d80 con 0x7ff8e0101e70 2024-03-23T17:48:53.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.070+0000 7ff8e5c00700 1 -- 172.21.15.86:0/1876426331 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8e01a1200 con 0x7ff8e0101e70 2024-03-23T17:48:53.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.071+0000 7ff8e5c00700 1 -- 172.21.15.86:0/1876426331 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8e0067a10 con 0x7ff8e0101e70 2024-03-23T17:48:53.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.074+0000 7ff8d7400700 1 -- 172.21.15.86:0/1876426331 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7ff8cc004020 con 0x7ff8e0101e70 2024-03-23T17:48:53.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.075+0000 7ff8d7400700 1 --2- 172.21.15.86:0/1876426331 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7ff8c40a11a0 0x7ff8c40a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:53.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.075+0000 7ff8d7400700 1 -- 172.21.15.86:0/1876426331 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff8cc014070 con 0x7ff8e0101e70 2024-03-23T17:48:53.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.075+0000 7ff8dea00700 1 --2- 172.21.15.86:0/1876426331 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7ff8c40a11a0 0x7ff8c40a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:53.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.076+0000 7ff8dea00700 1 --2- 172.21.15.86:0/1876426331 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7ff8c40a11a0 0x7ff8c40a3660 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0003730 tx=0x7ff8d0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:48:53.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.079+0000 7ff8d7400700 1 -- 172.21.15.86:0/1876426331 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ff8cc096530 con 0x7ff8e0101e70 2024-03-23T17:48:53.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:52 smithi086 ceph-mon[38221]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T17:48:53.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:52 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/89932496' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T17:48:53.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:52 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/1842913566' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T17:48:53.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:52 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:53.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:52 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:53.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:52 smithi092 ceph-mon[79089]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T17:48:53.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:52 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/89932496' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T17:48:53.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:52 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/1842913566' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T17:48:53.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:52 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:53.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:52 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:53.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:52 smithi152 ceph-mon[47949]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T17:48:53.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:52 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/89932496' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T17:48:53.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:52 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/1842913566' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T17:48:53.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:52 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:53.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:52 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:53.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.353+0000 7ff8e5c00700 1 -- 172.21.15.86:0/1876426331 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7ff8e010c640 con 0x7ff8e0101e70 2024-03-23T17:48:53.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.354+0000 7ff8d7400700 1 -- 172.21.15.86:0/1876426331 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v76) v1 ==== 84+0+65 (secure 0 0 0) 0x7ff8cc099b50 con 0x7ff8e0101e70 2024-03-23T17:48:53.354 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:48:53.355 INFO:teuthology.orchestra.run.smithi086.stdout:[ 2024-03-23T17:48:53.355 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T17:48:53.355 INFO:teuthology.orchestra.run.smithi086.stdout: "poolnum": 1, 2024-03-23T17:48:53.355 INFO:teuthology.orchestra.run.smithi086.stdout: "poolname": ".mgr" 2024-03-23T17:48:53.355 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T17:48:53.355 INFO:teuthology.orchestra.run.smithi086.stdout:] 2024-03-23T17:48:53.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.357+0000 7ff8e5c00700 1 -- 172.21.15.86:0/1876426331 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7ff8c40a11a0 msgr2=0x7ff8c40a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:53.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.357+0000 7ff8e5c00700 1 --2- 172.21.15.86:0/1876426331 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7ff8c40a11a0 0x7ff8c40a3660 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0003730 tx=0x7ff8d0000bc0 comp rx=0 tx=0).stop 2024-03-23T17:48:53.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.357+0000 7ff8e5c00700 1 -- 172.21.15.86:0/1876426331 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff8e0101e70 msgr2=0x7ff8e01992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:53.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.357+0000 7ff8e5c00700 1 --2- 172.21.15.86:0/1876426331 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff8e0101e70 0x7ff8e01992f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc00eb30 tx=0x7ff8cc00ee40 comp rx=0 tx=0).stop 2024-03-23T17:48:53.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.357+0000 7ff8e5c00700 1 -- 172.21.15.86:0/1876426331 shutdown_connections 2024-03-23T17:48:53.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.357+0000 7ff8e5c00700 1 --2- 172.21.15.86:0/1876426331 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7ff8c40a11a0 0x7ff8c40a3660 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:53.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.357+0000 7ff8e5c00700 1 --2- 172.21.15.86:0/1876426331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8e0102810 0x7ff8e01a0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:53.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.357+0000 7ff8e5c00700 1 --2- 172.21.15.86:0/1876426331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8e0100c80 0x7ff8e0198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:53.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.357+0000 7ff8e5c00700 1 --2- 172.21.15.86:0/1876426331 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff8e0101e70 0x7ff8e01992f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:53.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.357+0000 7ff8e5c00700 1 -- 172.21.15.86:0/1876426331 >> 172.21.15.86:0/1876426331 conn(0x7ff8e00fc1f0 msgr2=0x7ff8e0107640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:48:53.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.357+0000 7ff8e5c00700 1 -- 172.21.15.86:0/1876426331 shutdown_connections 2024-03-23T17:48:53.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.357+0000 7ff8e5c00700 1 -- 172.21.15.86:0/1876426331 wait complete. 2024-03-23T17:48:53.378 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-23T17:48:53.378 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-23T17:48:53.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.584+0000 7feeb7600700 1 -- 172.21.15.86:0/3284195448 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feeb01017f0 msgr2=0x7feeb0103be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:53.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.584+0000 7feeb7600700 1 --2- 172.21.15.86:0/3284195448 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feeb01017f0 0x7feeb0103be0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7feea4009990 tx=0x7feea4009ca0 comp rx=0 tx=0).stop 2024-03-23T17:48:53.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.584+0000 7feeb7600700 1 -- 172.21.15.86:0/3284195448 shutdown_connections 2024-03-23T17:48:53.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.584+0000 7feeb7600700 1 --2- 172.21.15.86:0/3284195448 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feeb0106a50 0x7feeb0108ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:53.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.584+0000 7feeb7600700 1 --2- 172.21.15.86:0/3284195448 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feeb01017f0 0x7feeb0103be0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:53.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.584+0000 7feeb7600700 1 --2- 172.21.15.86:0/3284195448 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feeb0104120 0x7feeb0106510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:53.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.584+0000 7feeb7600700 1 -- 172.21.15.86:0/3284195448 >> 172.21.15.86:0/3284195448 conn(0x7feeb00fb200 msgr2=0x7feeb00fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:48:53.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.584+0000 7feeb7600700 1 -- 172.21.15.86:0/3284195448 shutdown_connections 2024-03-23T17:48:53.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.584+0000 7feeb7600700 1 -- 172.21.15.86:0/3284195448 wait complete. 2024-03-23T17:48:53.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.585+0000 7feeb7600700 1 Processor -- start 2024-03-23T17:48:53.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.585+0000 7feeb7600700 1 -- start start 2024-03-23T17:48:53.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.585+0000 7feeb7600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feeb01017f0 0x7feeb0198f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:53.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.585+0000 7feeb7600700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feeb0104120 0x7feeb01994a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:53.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.585+0000 7feeb7600700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feeb0106a50 0x7feeb01a04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:53.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.585+0000 7feeb7600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeb01a0a90 con 0x7feeb01017f0 2024-03-23T17:48:53.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.585+0000 7feeb7600700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeb01a0bd0 con 0x7feeb0104120 2024-03-23T17:48:53.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.585+0000 7feeb7600700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeb01a0d60 con 0x7feeb0106a50 2024-03-23T17:48:53.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.586+0000 7feeb4c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feeb01017f0 0x7feeb0198f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:53.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.586+0000 7feeafe00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feeb0104120 0x7feeb01994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:53.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.586+0000 7feeafe00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feeb0104120 0x7feeb01994a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:35740/0 (socket says 172.21.15.86:35740) 2024-03-23T17:48:53.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.586+0000 7feeafe00700 1 -- 172.21.15.86:0/223058190 learned_addr learned my addr 172.21.15.86:0/223058190 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:48:53.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.586+0000 7feeb5600700 1 --2- 172.21.15.86:0/223058190 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feeb0106a50 0x7feeb01a04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:53.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.586+0000 7feeafe00700 1 -- 172.21.15.86:0/223058190 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feeb0106a50 msgr2=0x7feeb01a04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:53.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.586+0000 7feeafe00700 1 --2- 172.21.15.86:0/223058190 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feeb0106a50 0x7feeb01a04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:53.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.586+0000 7feeafe00700 1 -- 172.21.15.86:0/223058190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feeb01017f0 msgr2=0x7feeb0198f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:53.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.586+0000 7feeafe00700 1 --2- 172.21.15.86:0/223058190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feeb01017f0 0x7feeb0198f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:53.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.586+0000 7feeafe00700 1 -- 172.21.15.86:0/223058190 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feea40095f0 con 0x7feeb0104120 2024-03-23T17:48:53.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.586+0000 7feeafe00700 1 --2- 172.21.15.86:0/223058190 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feeb0104120 0x7feeb01994a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fee9c00e9a0 tx=0x7fee9c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:48:53.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.586+0000 7feead600700 1 -- 172.21.15.86:0/223058190 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee9c009790 con 0x7feeb0104120 2024-03-23T17:48:53.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.586+0000 7feead600700 1 -- 172.21.15.86:0/223058190 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee9c010430 con 0x7feeb0104120 2024-03-23T17:48:53.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.586+0000 7feead600700 1 -- 172.21.15.86:0/223058190 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee9c018680 con 0x7feeb0104120 2024-03-23T17:48:53.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.587+0000 7feeb7600700 1 -- 172.21.15.86:0/223058190 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feeb01a0fe0 con 0x7feeb0104120 2024-03-23T17:48:53.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.587+0000 7feeb7600700 1 -- 172.21.15.86:0/223058190 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feeb01a14c0 con 0x7feeb0104120 2024-03-23T17:48:53.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.588+0000 7feeb7600700 1 -- 172.21.15.86:0/223058190 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feeb0067f40 con 0x7feeb0104120 2024-03-23T17:48:53.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.591+0000 7feead600700 1 -- 172.21.15.86:0/223058190 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fee9c0105a0 con 0x7feeb0104120 2024-03-23T17:48:53.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.593+0000 7feead600700 1 --2- 172.21.15.86:0/223058190 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fee980a11a0 0x7fee980a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:53.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.593+0000 7feead600700 1 -- 172.21.15.86:0/223058190 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fee9c014070 con 0x7feeb0104120 2024-03-23T17:48:53.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.593+0000 7feeb4c00700 1 --2- 172.21.15.86:0/223058190 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fee980a11a0 0x7fee980a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:53.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.593+0000 7feead600700 1 -- 172.21.15.86:0/223058190 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fee9c099380 con 0x7feeb0104120 2024-03-23T17:48:53.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.594+0000 7feeb4c00700 1 --2- 172.21.15.86:0/223058190 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fee980a11a0 0x7fee980a3660 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7feea400b2e0 tx=0x7feea4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:48:53.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:53.828+0000 7feeb7600700 1 -- 172.21.15.86:0/223058190 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7feeb010c5a0 con 0x7feeb0104120 2024-03-23T17:48:54.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:54 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:54.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:54 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:54.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:54 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:54.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:54 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/1876426331' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-23T17:48:54.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:54 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:54.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:54 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:54.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:54 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:48:54.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:54 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:48:54.587 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:54 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:54.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:54.699+0000 7feead600700 1 -- 172.21.15.86:0/223058190 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v77) v1 ==== 138+0+0 (secure 0 0 0) 0x7fee9c099610 con 0x7feeb0104120 2024-03-23T17:48:54.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:54.705+0000 7feeb7600700 1 -- 172.21.15.86:0/223058190 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fee980a11a0 msgr2=0x7fee980a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:54.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:54.705+0000 7feeb7600700 1 --2- 172.21.15.86:0/223058190 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fee980a11a0 0x7fee980a3660 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7feea400b2e0 tx=0x7feea4005fb0 comp rx=0 tx=0).stop 2024-03-23T17:48:54.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:54.705+0000 7feeb7600700 1 -- 172.21.15.86:0/223058190 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feeb0104120 msgr2=0x7feeb01994a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:54.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:54.705+0000 7feeb7600700 1 --2- 172.21.15.86:0/223058190 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feeb0104120 0x7feeb01994a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fee9c00e9a0 tx=0x7fee9c00ecb0 comp rx=0 tx=0).stop 2024-03-23T17:48:54.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:54.705+0000 7feeb7600700 1 -- 172.21.15.86:0/223058190 shutdown_connections 2024-03-23T17:48:54.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:54.705+0000 7feeb7600700 1 --2- 172.21.15.86:0/223058190 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feeb0106a50 0x7feeb01a04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:54.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:54.705+0000 7feeb7600700 1 --2- 172.21.15.86:0/223058190 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fee980a11a0 0x7fee980a3660 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:54.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:54.705+0000 7feeb7600700 1 --2- 172.21.15.86:0/223058190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feeb01017f0 0x7feeb0198f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:54.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:54.705+0000 7feeb7600700 1 --2- 172.21.15.86:0/223058190 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feeb0104120 0x7feeb01994a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:54.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:54.705+0000 7feeb7600700 1 -- 172.21.15.86:0/223058190 >> 172.21.15.86:0/223058190 conn(0x7feeb00fb200 msgr2=0x7feeb00ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:48:54.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:54.705+0000 7feeb7600700 1 -- 172.21.15.86:0/223058190 shutdown_connections 2024-03-23T17:48:54.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:54.705+0000 7feeb7600700 1 -- 172.21.15.86:0/223058190 wait complete. 2024-03-23T17:48:54.707 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_metadata' created 2024-03-23T17:48:54.722 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:54 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:54.722 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:54 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:54.723 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:54 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:54.723 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:54 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/1876426331' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-23T17:48:54.723 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:54 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:54.723 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:54 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:54.723 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:54 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:48:54.723 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:54 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:48:54.723 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:54 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:54.725 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-23T17:48:54.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:54 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:54.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:54 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:54.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:54 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:54.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:54 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/1876426331' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-23T17:48:54.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:54 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:54.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:54 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:54.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:54 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:48:54.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:54 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:48:54.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:54 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:48:54.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:54.924+0000 7fd209200700 1 -- 172.21.15.86:0/1793977727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2041018d0 msgr2=0x7fd204103cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:54.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:54.924+0000 7fd209200700 1 --2- 172.21.15.86:0/1793977727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2041018d0 0x7fd204103cc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd1f4009990 tx=0x7fd1f4009ca0 comp rx=0 tx=0).stop 2024-03-23T17:48:54.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:54.924+0000 7fd209200700 1 -- 172.21.15.86:0/1793977727 shutdown_connections 2024-03-23T17:48:54.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:54.924+0000 7fd209200700 1 --2- 172.21.15.86:0/1793977727 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd204106b30 0x7fd204108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:54.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:54.924+0000 7fd209200700 1 --2- 172.21.15.86:0/1793977727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2041018d0 0x7fd204103cc0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:54.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:54.924+0000 7fd209200700 1 --2- 172.21.15.86:0/1793977727 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd204104200 0x7fd2041065f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:54.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:54.924+0000 7fd209200700 1 -- 172.21.15.86:0/1793977727 >> 172.21.15.86:0/1793977727 conn(0x7fd2040fb200 msgr2=0x7fd2040fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:48:54.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:54.924+0000 7fd209200700 1 -- 172.21.15.86:0/1793977727 shutdown_connections 2024-03-23T17:48:54.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:54.924+0000 7fd209200700 1 -- 172.21.15.86:0/1793977727 wait complete. 2024-03-23T17:48:54.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:54.925+0000 7fd209200700 1 Processor -- start 2024-03-23T17:48:54.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:54.925+0000 7fd209200700 1 -- start start 2024-03-23T17:48:54.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:54.925+0000 7fd209200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2041018d0 0x7fd2041011f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:54.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:54.925+0000 7fd209200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd204104200 0x7fd2040ff880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:54.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:54.926+0000 7fd209200700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd204106b30 0x7fd2040ffdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:54.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:54.926+0000 7fd209200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd204100560 con 0x7fd204104200 2024-03-23T17:48:54.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:54.926+0000 7fd209200700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2041006d0 con 0x7fd2041018d0 2024-03-23T17:48:54.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:54.926+0000 7fd209200700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2041a0d90 con 0x7fd204106b30 2024-03-23T17:48:54.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:54.926+0000 7fd201e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2041018d0 0x7fd2041011f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:54.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:54.926+0000 7fd201e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2041018d0 0x7fd2041011f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:49264/0 (socket says 172.21.15.86:49264) 2024-03-23T17:48:54.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:54.926+0000 7fd201e00700 1 -- 172.21.15.86:0/1904638533 learned_addr learned my addr 172.21.15.86:0/1904638533 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:48:54.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:54.926+0000 7fd201400700 1 --2- 172.21.15.86:0/1904638533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd204104200 0x7fd2040ff880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:54.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:54.926+0000 7fd202800700 1 --2- 172.21.15.86:0/1904638533 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd204106b30 0x7fd2040ffdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:54.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:54.926+0000 7fd201e00700 1 -- 172.21.15.86:0/1904638533 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd204106b30 msgr2=0x7fd2040ffdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:54.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:54.926+0000 7fd201e00700 1 --2- 172.21.15.86:0/1904638533 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd204106b30 0x7fd2040ffdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:54.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:54.926+0000 7fd201e00700 1 -- 172.21.15.86:0/1904638533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd204104200 msgr2=0x7fd2040ff880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:54.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:54.926+0000 7fd201e00700 1 --2- 172.21.15.86:0/1904638533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd204104200 0x7fd2040ff880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:54.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:54.926+0000 7fd201e00700 1 -- 172.21.15.86:0/1904638533 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1f0009520 con 0x7fd2041018d0 2024-03-23T17:48:54.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:54.927+0000 7fd201e00700 1 --2- 172.21.15.86:0/1904638533 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2041018d0 0x7fd2041011f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd1f400b680 tx=0x7fd1f4009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:48:54.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:54.927+0000 7fd1faa00700 1 -- 172.21.15.86:0/1904638533 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f401d070 con 0x7fd2041018d0 2024-03-23T17:48:54.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:54.927+0000 7fd1faa00700 1 -- 172.21.15.86:0/1904638533 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd1f4022470 con 0x7fd2041018d0 2024-03-23T17:48:54.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:54.927+0000 7fd209200700 1 -- 172.21.15.86:0/1904638533 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1f40095f0 con 0x7fd2041018d0 2024-03-23T17:48:54.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:54.927+0000 7fd1faa00700 1 -- 172.21.15.86:0/1904638533 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f4017650 con 0x7fd2041018d0 2024-03-23T17:48:54.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:54.927+0000 7fd209200700 1 -- 172.21.15.86:0/1904638533 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2041a11a0 con 0x7fd2041018d0 2024-03-23T17:48:54.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:54.928+0000 7fd209200700 1 -- 172.21.15.86:0/1904638533 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2040fcd80 con 0x7fd2041018d0 2024-03-23T17:48:54.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:54.930+0000 7fd1faa00700 1 -- 172.21.15.86:0/1904638533 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fd1f40225e0 con 0x7fd2041018d0 2024-03-23T17:48:54.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:54.931+0000 7fd1faa00700 1 --2- 172.21.15.86:0/1904638533 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fd1e80a1120 0x7fd1e80a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:54.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:54.931+0000 7fd1faa00700 1 -- 172.21.15.86:0/1904638533 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7497+0+0 (secure 0 0 0) 0x7fd1f40cdf40 con 0x7fd2041018d0 2024-03-23T17:48:54.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:54.931+0000 7fd201400700 1 --2- 172.21.15.86:0/1904638533 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fd1e80a1120 0x7fd1e80a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:54.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:54.932+0000 7fd201400700 1 --2- 172.21.15.86:0/1904638533 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fd1e80a1120 0x7fd1e80a35e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd1f0009df0 tx=0x7fd1f0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:48:54.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:54.934+0000 7fd1faa00700 1 -- 172.21.15.86:0/1904638533 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fd1f40979f0 con 0x7fd2041018d0 2024-03-23T17:48:55.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:55.140+0000 7fd209200700 1 -- 172.21.15.86:0/1904638533 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fd204067260 con 0x7fd2041018d0 2024-03-23T17:48:55.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:55.705+0000 7fd1faa00700 1 -- 172.21.15.86:0/1904638533 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v78) v1 ==== 170+0+0 (secure 0 0 0) 0x7fd1f409b010 con 0x7fd2041018d0 2024-03-23T17:48:55.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:55.713+0000 7fd209200700 1 -- 172.21.15.86:0/1904638533 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fd1e80a1120 msgr2=0x7fd1e80a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:55.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:55.713+0000 7fd209200700 1 --2- 172.21.15.86:0/1904638533 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fd1e80a1120 0x7fd1e80a35e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd1f0009df0 tx=0x7fd1f0009260 comp rx=0 tx=0).stop 2024-03-23T17:48:55.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:55.713+0000 7fd209200700 1 -- 172.21.15.86:0/1904638533 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2041018d0 msgr2=0x7fd2041011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:55.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:55.713+0000 7fd209200700 1 --2- 172.21.15.86:0/1904638533 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2041018d0 0x7fd2041011f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd1f400b680 tx=0x7fd1f4009580 comp rx=0 tx=0).stop 2024-03-23T17:48:55.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:55.713+0000 7fd209200700 1 -- 172.21.15.86:0/1904638533 shutdown_connections 2024-03-23T17:48:55.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:55.713+0000 7fd209200700 1 --2- 172.21.15.86:0/1904638533 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd204106b30 0x7fd2040ffdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:55.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:55.713+0000 7fd209200700 1 --2- 172.21.15.86:0/1904638533 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2041018d0 0x7fd2041011f0 secure :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd1f400b680 tx=0x7fd1f4009580 comp rx=0 tx=0).stop 2024-03-23T17:48:55.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:55.713+0000 7fd209200700 1 --2- 172.21.15.86:0/1904638533 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fd1e80a1120 0x7fd1e80a35e0 secure :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd1f0009df0 tx=0x7fd1f0009260 comp rx=0 tx=0).stop 2024-03-23T17:48:55.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:55.713+0000 7fd209200700 1 --2- 172.21.15.86:0/1904638533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd204104200 0x7fd2040ff880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:55.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:55.713+0000 7fd209200700 1 -- 172.21.15.86:0/1904638533 >> 172.21.15.86:0/1904638533 conn(0x7fd2040fb200 msgr2=0x7fd204103760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:48:55.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:55.714+0000 7fd209200700 1 -- 172.21.15.86:0/1904638533 shutdown_connections 2024-03-23T17:48:55.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:55.714+0000 7fd209200700 1 -- 172.21.15.86:0/1904638533 wait complete. 2024-03-23T17:48:55.716 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_data' created 2024-03-23T17:48:55.742 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-23T17:48:55.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:55.938+0000 7fb8cc400700 1 -- 172.21.15.86:0/3584282464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8c4101f70 msgr2=0x7fb8c41023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:55.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:55.938+0000 7fb8cc400700 1 --2- 172.21.15.86:0/3584282464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8c4101f70 0x7fb8c41023d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb8b00099c0 tx=0x7fb8b0009cd0 comp rx=0 tx=0).stop 2024-03-23T17:48:55.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:55.938+0000 7fb8cc400700 1 -- 172.21.15.86:0/3584282464 shutdown_connections 2024-03-23T17:48:55.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:55.938+0000 7fb8cc400700 1 --2- 172.21.15.86:0/3584282464 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb8c4102910 0x7fb8c41091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:55.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:55.938+0000 7fb8cc400700 1 --2- 172.21.15.86:0/3584282464 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb8c4100d80 0x7fb8c41011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:55.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:55.938+0000 7fb8cc400700 1 --2- 172.21.15.86:0/3584282464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8c4101f70 0x7fb8c41023d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:55.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:55.938+0000 7fb8cc400700 1 -- 172.21.15.86:0/3584282464 >> 172.21.15.86:0/3584282464 conn(0x7fb8c40fc2f0 msgr2=0x7fb8c40fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:48:55.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:55.938+0000 7fb8cc400700 1 -- 172.21.15.86:0/3584282464 shutdown_connections 2024-03-23T17:48:55.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:55.938+0000 7fb8cc400700 1 -- 172.21.15.86:0/3584282464 wait complete. 2024-03-23T17:48:55.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:55.939+0000 7fb8cc400700 1 Processor -- start 2024-03-23T17:48:55.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:55.939+0000 7fb8cc400700 1 -- start start 2024-03-23T17:48:55.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:55.940+0000 7fb8cc400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb8c4100d80 0x7fb8c4074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:55.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:55.940+0000 7fb8cc400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb8c4101f70 0x7fb8c40731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:55.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:55.940+0000 7fb8cc400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8c4102910 0x7fb8c40736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:55.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:55.940+0000 7fb8cc400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c4073e80 con 0x7fb8c4102910 2024-03-23T17:48:55.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:55.940+0000 7fb8cc400700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c4073ff0 con 0x7fb8c4100d80 2024-03-23T17:48:55.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:55.940+0000 7fb8cc400700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c41a0d60 con 0x7fb8c4101f70 2024-03-23T17:48:55.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:55.940+0000 7fb8ca400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8c4102910 0x7fb8c40736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:55.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:55.940+0000 7fb8c9a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb8c4100d80 0x7fb8c4074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:55.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:55.940+0000 7fb8c9a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb8c4100d80 0x7fb8c4074b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:49280/0 (socket says 172.21.15.86:49280) 2024-03-23T17:48:55.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:55.940+0000 7fb8c9a00700 1 -- 172.21.15.86:0/3157344438 learned_addr learned my addr 172.21.15.86:0/3157344438 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:48:55.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:55.940+0000 7fb8c9000700 1 --2- 172.21.15.86:0/3157344438 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb8c4101f70 0x7fb8c40731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:55.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:55.940+0000 7fb8ca400700 1 -- 172.21.15.86:0/3157344438 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb8c4101f70 msgr2=0x7fb8c40731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:55.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:55.940+0000 7fb8ca400700 1 --2- 172.21.15.86:0/3157344438 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb8c4101f70 0x7fb8c40731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:55.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:55.940+0000 7fb8ca400700 1 -- 172.21.15.86:0/3157344438 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb8c4100d80 msgr2=0x7fb8c4074b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:55.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:55.940+0000 7fb8ca400700 1 --2- 172.21.15.86:0/3157344438 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb8c4100d80 0x7fb8c4074b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:55.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:55.940+0000 7fb8ca400700 1 -- 172.21.15.86:0/3157344438 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8b00095f0 con 0x7fb8c4102910 2024-03-23T17:48:55.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:55.940+0000 7fb8ca400700 1 --2- 172.21.15.86:0/3157344438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8c4102910 0x7fb8c40736e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc00efc0 tx=0x7fb8bc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:48:55.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:55.941+0000 7fb8c2000700 1 -- 172.21.15.86:0/3157344438 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8bc018070 con 0x7fb8c4102910 2024-03-23T17:48:55.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:55.941+0000 7fb8c2000700 1 -- 172.21.15.86:0/3157344438 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb8bc00f040 con 0x7fb8c4102910 2024-03-23T17:48:55.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:55.941+0000 7fb8cc400700 1 -- 172.21.15.86:0/3157344438 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8c41a0f90 con 0x7fb8c4102910 2024-03-23T17:48:55.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:55.941+0000 7fb8c2000700 1 -- 172.21.15.86:0/3157344438 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8bc00cce0 con 0x7fb8c4102910 2024-03-23T17:48:55.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:55.941+0000 7fb8cc400700 1 -- 172.21.15.86:0/3157344438 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8c41a1410 con 0x7fb8c4102910 2024-03-23T17:48:55.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:55.942+0000 7fb8cc400700 1 -- 172.21.15.86:0/3157344438 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8c4108250 con 0x7fb8c4102910 2024-03-23T17:48:55.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:55.944+0000 7fb8c2000700 1 -- 172.21.15.86:0/3157344438 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fb8bc003a00 con 0x7fb8c4102910 2024-03-23T17:48:55.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:55.945+0000 7fb8c2000700 1 --2- 172.21.15.86:0/3157344438 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fb8ac0a1120 0x7fb8ac0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:55.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:55.946+0000 7fb8c2000700 1 -- 172.21.15.86:0/3157344438 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7879+0+0 (secure 0 0 0) 0x7fb8bc0ccbc0 con 0x7fb8c4102910 2024-03-23T17:48:55.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:55.946+0000 7fb8c9a00700 1 --2- 172.21.15.86:0/3157344438 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fb8ac0a1120 0x7fb8ac0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:55.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:55.946+0000 7fb8c9a00700 1 --2- 172.21.15.86:0/3157344438 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fb8ac0a1120 0x7fb8ac0a35e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb8b8007cf0 tx=0x7fb8b8007480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:48:55.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:55.948+0000 7fb8c2000700 1 -- 172.21.15.86:0/3157344438 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fb8bc0964f0 con 0x7fb8c4102910 2024-03-23T17:48:56.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:55 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/223058190' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-23T17:48:56.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:55 smithi086 ceph-mon[38221]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-23T17:48:56.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:55 smithi086 ceph-mon[38221]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T17:48:56.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:55 smithi086 ceph-mon[38221]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-23T17:48:56.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:55 smithi086 ceph-mon[38221]: osdmap e77: 12 total, 12 up, 12 in 2024-03-23T17:48:56.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:55 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/223058190' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-23T17:48:56.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:55 smithi092 ceph-mon[79089]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-23T17:48:56.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:55 smithi092 ceph-mon[79089]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T17:48:56.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:55 smithi092 ceph-mon[79089]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-23T17:48:56.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:55 smithi092 ceph-mon[79089]: osdmap e77: 12 total, 12 up, 12 in 2024-03-23T17:48:56.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:55 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/223058190' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-23T17:48:56.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:55 smithi152 ceph-mon[47949]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-23T17:48:56.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:55 smithi152 ceph-mon[47949]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T17:48:56.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:55 smithi152 ceph-mon[47949]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-23T17:48:56.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:55 smithi152 ceph-mon[47949]: osdmap e77: 12 total, 12 up, 12 in 2024-03-23T17:48:56.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.200+0000 7fb8cc400700 1 -- 172.21.15.86:0/3157344438 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fb8c410c5e0 con 0x7fb8c4102910 2024-03-23T17:48:56.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.207+0000 7fb8c2000700 1 -- 172.21.15.86:0/3157344438 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7fb8bc099b10 con 0x7fb8c4102910 2024-03-23T17:48:56.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.212+0000 7fb8cc400700 1 -- 172.21.15.86:0/3157344438 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fb8ac0a1120 msgr2=0x7fb8ac0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:56.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.212+0000 7fb8cc400700 1 --2- 172.21.15.86:0/3157344438 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fb8ac0a1120 0x7fb8ac0a35e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb8b8007cf0 tx=0x7fb8b8007480 comp rx=0 tx=0).stop 2024-03-23T17:48:56.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.212+0000 7fb8cc400700 1 -- 172.21.15.86:0/3157344438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8c4102910 msgr2=0x7fb8c40736e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:56.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.212+0000 7fb8cc400700 1 --2- 172.21.15.86:0/3157344438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8c4102910 0x7fb8c40736e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc00efc0 tx=0x7fb8bc00c3a0 comp rx=0 tx=0).stop 2024-03-23T17:48:56.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.212+0000 7fb8cc400700 1 -- 172.21.15.86:0/3157344438 shutdown_connections 2024-03-23T17:48:56.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.212+0000 7fb8cc400700 1 --2- 172.21.15.86:0/3157344438 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fb8ac0a1120 0x7fb8ac0a35e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:56.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.212+0000 7fb8cc400700 1 --2- 172.21.15.86:0/3157344438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8c4102910 0x7fb8c40736e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:56.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.212+0000 7fb8cc400700 1 --2- 172.21.15.86:0/3157344438 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb8c4100d80 0x7fb8c4074b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:56.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.212+0000 7fb8cc400700 1 --2- 172.21.15.86:0/3157344438 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb8c4101f70 0x7fb8c40731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:56.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.212+0000 7fb8cc400700 1 -- 172.21.15.86:0/3157344438 >> 172.21.15.86:0/3157344438 conn(0x7fb8c40fc2f0 msgr2=0x7fb8c4105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:48:56.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.213+0000 7fb8cc400700 1 -- 172.21.15.86:0/3157344438 shutdown_connections 2024-03-23T17:48:56.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.213+0000 7fb8cc400700 1 -- 172.21.15.86:0/3157344438 wait complete. 2024-03-23T17:48:56.217 INFO:teuthology.orchestra.run.smithi086.stderr:new fs with metadata pool 2 and data pool 3 2024-03-23T17:48:56.246 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T17:48:56.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.438+0000 7fa477800700 1 -- 172.21.15.86:0/3169843268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa470101e70 msgr2=0x7fa4701022d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:56.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.438+0000 7fa477800700 1 --2- 172.21.15.86:0/3169843268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa470101e70 0x7fa4701022d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa45c0099c0 tx=0x7fa45c009cd0 comp rx=0 tx=0).stop 2024-03-23T17:48:56.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.438+0000 7fa477800700 1 -- 172.21.15.86:0/3169843268 shutdown_connections 2024-03-23T17:48:56.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.438+0000 7fa477800700 1 --2- 172.21.15.86:0/3169843268 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa470102810 0x7fa4701090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:56.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.438+0000 7fa477800700 1 --2- 172.21.15.86:0/3169843268 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa470100c80 0x7fa4701010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:56.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.438+0000 7fa477800700 1 --2- 172.21.15.86:0/3169843268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa470101e70 0x7fa4701022d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:56.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.438+0000 7fa477800700 1 -- 172.21.15.86:0/3169843268 >> 172.21.15.86:0/3169843268 conn(0x7fa4700fc1f0 msgr2=0x7fa4700fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:48:56.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.438+0000 7fa477800700 1 -- 172.21.15.86:0/3169843268 shutdown_connections 2024-03-23T17:48:56.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.438+0000 7fa477800700 1 -- 172.21.15.86:0/3169843268 wait complete. 2024-03-23T17:48:56.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.439+0000 7fa477800700 1 Processor -- start 2024-03-23T17:48:56.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.439+0000 7fa477800700 1 -- start start 2024-03-23T17:48:56.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.440+0000 7fa477800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa470100c80 0x7fa470198e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:56.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.440+0000 7fa477800700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa470101e70 0x7fa470199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:56.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.440+0000 7fa477800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa470102810 0x7fa4701a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:56.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.440+0000 7fa477800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4701a08f0 con 0x7fa470100c80 2024-03-23T17:48:56.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.440+0000 7fa477800700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4701a0a60 con 0x7fa470102810 2024-03-23T17:48:56.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.440+0000 7fa477800700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4701a0c20 con 0x7fa470101e70 2024-03-23T17:48:56.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.440+0000 7fa474e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa470100c80 0x7fa470198e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:56.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.440+0000 7fa474e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa470100c80 0x7fa470198e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36662/0 (socket says 172.21.15.86:36662) 2024-03-23T17:48:56.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.440+0000 7fa474e00700 1 -- 172.21.15.86:0/2821120106 learned_addr learned my addr 172.21.15.86:0/2821120106 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:48:56.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.440+0000 7fa46fe00700 1 --2- 172.21.15.86:0/2821120106 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa470101e70 0x7fa470199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:56.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.440+0000 7fa475800700 1 --2- 172.21.15.86:0/2821120106 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa470102810 0x7fa4701a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:56.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.440+0000 7fa474e00700 1 -- 172.21.15.86:0/2821120106 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa470101e70 msgr2=0x7fa470199390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:56.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.440+0000 7fa474e00700 1 --2- 172.21.15.86:0/2821120106 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa470101e70 0x7fa470199390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:56.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.440+0000 7fa474e00700 1 -- 172.21.15.86:0/2821120106 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa470102810 msgr2=0x7fa4701a0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:56.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.440+0000 7fa474e00700 1 --2- 172.21.15.86:0/2821120106 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa470102810 0x7fa4701a0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:56.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.440+0000 7fa474e00700 1 -- 172.21.15.86:0/2821120106 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa45c0095f0 con 0x7fa470100c80 2024-03-23T17:48:56.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.441+0000 7fa474e00700 1 --2- 172.21.15.86:0/2821120106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa470100c80 0x7fa470198e50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa46400e970 tx=0x7fa46400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:48:56.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.441+0000 7fa46d600700 1 -- 172.21.15.86:0/2821120106 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa464009790 con 0x7fa470100c80 2024-03-23T17:48:56.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.441+0000 7fa46d600700 1 -- 172.21.15.86:0/2821120106 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa46400eec0 con 0x7fa470100c80 2024-03-23T17:48:56.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.441+0000 7fa477800700 1 -- 172.21.15.86:0/2821120106 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4701a0f00 con 0x7fa470100c80 2024-03-23T17:48:56.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.441+0000 7fa46d600700 1 -- 172.21.15.86:0/2821120106 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa464003cf0 con 0x7fa470100c80 2024-03-23T17:48:56.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.441+0000 7fa477800700 1 -- 172.21.15.86:0/2821120106 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4701a1320 con 0x7fa470100c80 2024-03-23T17:48:56.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.442+0000 7fa477800700 1 -- 172.21.15.86:0/2821120106 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa470067a10 con 0x7fa470100c80 2024-03-23T17:48:56.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.444+0000 7fa46d600700 1 -- 172.21.15.86:0/2821120106 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fa46400ccd0 con 0x7fa470100c80 2024-03-23T17:48:56.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.445+0000 7fa46d600700 1 --2- 172.21.15.86:0/2821120106 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fa4580a1120 0x7fa4580a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:56.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.445+0000 7fa46d600700 1 -- 172.21.15.86:0/2821120106 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa464014070 con 0x7fa470100c80 2024-03-23T17:48:56.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.445+0000 7fa46fe00700 1 --2- 172.21.15.86:0/2821120106 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fa4580a1120 0x7fa4580a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:56.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.446+0000 7fa46fe00700 1 --2- 172.21.15.86:0/2821120106 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fa4580a1120 0x7fa4580a35e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa45c003730 tx=0x7fa45c00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:48:56.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.450+0000 7fa46d600700 1 -- 172.21.15.86:0/2821120106 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa4640962b0 con 0x7fa470100c80 2024-03-23T17:48:56.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.695+0000 7fa477800700 1 -- 172.21.15.86:0/2821120106 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa47010c790 con 0x7fa470100c80 2024-03-23T17:48:56.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.696+0000 7fa46d600700 1 -- 172.21.15.86:0/2821120106 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+22096 (secure 0 0 0) 0x7fa4640998d0 con 0x7fa470100c80 2024-03-23T17:48:56.697 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:48:56.697 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":79,"fsid":"9d76c9e8-e93c-11ee-95cd-87774f69a715","created":"2024-03-23T17:42:30.097233+0000","modified":"2024-03-23T17:48:56.201953+0000","last_up_change":"2024-03-23T17:48:18.217601+0000","last_in_change":"2024-03-23T17:48:00.250579+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T17:45:26.848397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T17:48:54.331894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T17:48:55.640693+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"656655c6-1c7f-4640-92c4-2f42de820143","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6803","nonce":2423505491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6805","nonce":2423505491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6809","nonce":2423505491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6807","nonce":2423505491}]},"public_addr":"172.21.15.86:6803/2423505491","cluster_addr":"172.21.15.86:6805/2423505491","heartbeat_back_addr":"172.21.15.86:6809/2423505491","heartbeat_front_addr":"172.21.15.86:6807/2423505491","state":["exists","up"]},{"osd":1,"uuid":"3e86dc65-20a6-4de8-b9ce-fcf01f68862b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6801","nonce":3776252597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6803","nonce":3776252597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6807","nonce":3776252597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6805","nonce":3776252597}]},"public_addr":"172.21.15.92:6801/3776252597","cluster_addr":"172.21.15.92:6803/3776252597","heartbeat_back_addr":"172.21.15.92:6807/3776252597","heartbeat_front_addr":"172.21.15.92:6805/3776252597","state":["exists","up"]},{"osd":2,"uuid":"09808786-3bd3-4d11-8c7b-1736119dfbb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6801","nonce":1601265891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6803","nonce":1601265891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6807","nonce":1601265891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6805","nonce":1601265891}]},"public_addr":"172.21.15.152:6801/1601265891","cluster_addr":"172.21.15.152:6803/1601265891","heartbeat_back_addr":"172.21.15.152:6807/1601265891","heartbeat_front_addr":"172.21.15.152:6805/1601265891","state":["exists","up"]},{"osd":3,"uuid":"79c61910-4b53-424c-b150-6f5385a2cd68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6811","nonce":694846466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6813","nonce":694846466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6817","nonce":694846466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6815","nonce":694846466}]},"public_addr":"172.21.15.86:6811/694846466","cluster_addr":"172.21.15.86:6813/694846466","heartbeat_back_addr":"172.21.15.86:6817/694846466","heartbeat_front_addr":"172.21.15.86:6815/694846466","state":["exists","up"]},{"osd":4,"uuid":"a2878035-0179-469f-bbd4-8ce2d3f6bddc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6809","nonce":1422821471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6811","nonce":1422821471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6815","nonce":1422821471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6813","nonce":1422821471}]},"public_addr":"172.21.15.92:6809/1422821471","cluster_addr":"172.21.15.92:6811/1422821471","heartbeat_back_addr":"172.21.15.92:6815/1422821471","heartbeat_front_addr":"172.21.15.92:6813/1422821471","state":["exists","up"]},{"osd":5,"uuid":"837366aa-ce25-42ca-b157-5bbb50159d0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6809","nonce":1039375239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6811","nonce":1039375239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6815","nonce":1039375239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6813","nonce":1039375239}]},"public_addr":"172.21.15.152:6809/1039375239","cluster_addr":"172.21.15.152:6811/1039375239","heartbeat_back_addr":"172.21.15.152:6815/1039375239","heartbeat_front_addr":"172.21.15.152:6813/1039375239","state":["exists","up"]},{"osd":6,"uuid":"61024437-7b91-432f-a1af-6443d0ef5213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6819","nonce":4080452707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6821","nonce":4080452707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6825","nonce":4080452707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6823","nonce":4080452707}]},"public_addr":"172.21.15.86:6819/4080452707","cluster_addr":"172.21.15.86:6821/4080452707","heartbeat_back_addr":"172.21.15.86:6825/4080452707","heartbeat_front_addr":"172.21.15.86:6823/4080452707","state":["exists","up"]},{"osd":7,"uuid":"2684bdae-6b30-4afc-af68-4ce41b3a75b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6817","nonce":4187707622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6819","nonce":4187707622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6823","nonce":4187707622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6821","nonce":4187707622}]},"public_addr":"172.21.15.92:6817/4187707622","cluster_addr":"172.21.15.92:6819/4187707622","heartbeat_back_addr":"172.21.15.92:6823/4187707622","heartbeat_front_addr":"172.21.15.92:6821/4187707622","state":["exists","up"]},{"osd":8,"uuid":"f03bbf0d-1281-4bcf-9773-9854388e65ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6817","nonce":3237612748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6819","nonce":3237612748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6823","nonce":3237612748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6821","nonce":3237612748}]},"public_addr":"172.21.15.152:6817/3237612748","cluster_addr":"172.21.15.152:6819/3237612748","heartbeat_back_addr":"172.21.15.152:6823/3237612748","heartbeat_front_addr":"172.21.15.152:6821/3237612748","state":["exists","up"]},{"osd":9,"uuid":"e7e8b775-99ff-48a4-a23a-91e7ab6622bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6827","nonce":2835218294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6829","nonce":2835218294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6833","nonce":2835218294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6831","nonce":2835218294}]},"public_addr":"172.21.15.86:6827/2835218294","cluster_addr":"172.21.15.86:6829/2835218294","heartbeat_back_addr":"172.21.15.86:6833/2835218294","heartbeat_front_addr":"172.21.15.86:6831/2835218294","state":["exists","up"]},{"osd":10,"uuid":"d9d616cd-a229-44de-8768-02a4415df115","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6825","nonce":4259465100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6827","nonce":4259465100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6831","nonce":4259465100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6829","nonce":4259465100}]},"public_addr":"172.21.15.92:6825/4259465100","cluster_addr":"172.21.15.92:6827/4259465100","heartbeat_back_addr":"172.21.15.92:6831/4259465100","heartbeat_front_addr":"172.21.15.92:6829/4259465100","state":["exists","up"]},{"osd":11,"uuid":"e7c0b865-ccf5-4233-8d22-0d9b6dbe5c97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6825","nonce":194662760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6827","nonce":194662760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6831","nonce":194662760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6829","nonce":194662760}]},"public_addr":"172.21.15.152:6825/194662760","cluster_addr":"172.21.15.152:6827/194662760","heartbeat_back_addr":"172.21.15.152:6831/194662760","heartbeat_front_addr":"172.21.15.152:6829/194662760","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:44:47.269167+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:45:05.363655+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:45:23.192677+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:45:44.156228+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:02.532971+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:20.153295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:41.713888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:59.676516+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:47:17.207087+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:47:38.970316+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:47:57.282525+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:48:15.437606+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/3553500912":"2024-03-24T17:43:28.242492+0000","172.21.15.86:6801/3600278228":"2024-03-24T17:43:28.242492+0000","172.21.15.86:0/887262329":"2024-03-24T17:43:28.242492+0000","172.21.15.86:6800/2313723492":"2024-03-24T17:43:05.043910+0000","172.21.15.86:6800/3600278228":"2024-03-24T17:43:28.242492+0000","172.21.15.86:0/3815705642":"2024-03-24T17:43:05.043910+0000","172.21.15.86:0/1614943062":"2024-03-24T17:43:28.242492+0000","172.21.15.86:6801/2313723492":"2024-03-24T17:43:05.043910+0000","172.21.15.86:0/2416321716":"2024-03-24T17:43:05.043910+0000","172.21.15.86:0/433588188":"2024-03-24T17:43:05.043910+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T17:48:56.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.699+0000 7fa477800700 1 -- 172.21.15.86:0/2821120106 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fa4580a1120 msgr2=0x7fa4580a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:56.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.699+0000 7fa477800700 1 --2- 172.21.15.86:0/2821120106 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fa4580a1120 0x7fa4580a35e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa45c003730 tx=0x7fa45c00b540 comp rx=0 tx=0).stop 2024-03-23T17:48:56.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.699+0000 7fa477800700 1 -- 172.21.15.86:0/2821120106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa470100c80 msgr2=0x7fa470198e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:56.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.699+0000 7fa477800700 1 --2- 172.21.15.86:0/2821120106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa470100c80 0x7fa470198e50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa46400e970 tx=0x7fa46400ec80 comp rx=0 tx=0).stop 2024-03-23T17:48:56.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.699+0000 7fa477800700 1 -- 172.21.15.86:0/2821120106 shutdown_connections 2024-03-23T17:48:56.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.699+0000 7fa477800700 1 --2- 172.21.15.86:0/2821120106 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa470102810 0x7fa4701a0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:56.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.699+0000 7fa477800700 1 --2- 172.21.15.86:0/2821120106 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fa4580a1120 0x7fa4580a35e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:56.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.699+0000 7fa477800700 1 --2- 172.21.15.86:0/2821120106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa470100c80 0x7fa470198e50 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:56.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.699+0000 7fa477800700 1 --2- 172.21.15.86:0/2821120106 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa470101e70 0x7fa470199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:56.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.699+0000 7fa477800700 1 -- 172.21.15.86:0/2821120106 >> 172.21.15.86:0/2821120106 conn(0x7fa4700fc1f0 msgr2=0x7fa470072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:48:56.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.699+0000 7fa477800700 1 -- 172.21.15.86:0/2821120106 shutdown_connections 2024-03-23T17:48:56.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.699+0000 7fa477800700 1 -- 172.21.15.86:0/2821120106 wait complete. 2024-03-23T17:48:56.718 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T17:48:56.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.910+0000 7f1a1f600700 1 -- 172.21.15.86:0/863420029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a18101e70 msgr2=0x7f1a181022d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:56.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.910+0000 7f1a1f600700 1 --2- 172.21.15.86:0/863420029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a18101e70 0x7f1a181022d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1a040099c0 tx=0x7f1a04009cd0 comp rx=0 tx=0).stop 2024-03-23T17:48:56.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.910+0000 7f1a1f600700 1 -- 172.21.15.86:0/863420029 shutdown_connections 2024-03-23T17:48:56.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.910+0000 7f1a1f600700 1 --2- 172.21.15.86:0/863420029 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1a18102810 0x7f1a181090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:56.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.910+0000 7f1a1f600700 1 --2- 172.21.15.86:0/863420029 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a18100c80 0x7f1a181010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:56.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.910+0000 7f1a1f600700 1 --2- 172.21.15.86:0/863420029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a18101e70 0x7f1a181022d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:56.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.910+0000 7f1a1f600700 1 -- 172.21.15.86:0/863420029 >> 172.21.15.86:0/863420029 conn(0x7f1a180fc1f0 msgr2=0x7f1a180fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:48:56.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.910+0000 7f1a1f600700 1 -- 172.21.15.86:0/863420029 shutdown_connections 2024-03-23T17:48:56.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.910+0000 7f1a1f600700 1 -- 172.21.15.86:0/863420029 wait complete. 2024-03-23T17:48:56.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.911+0000 7f1a1f600700 1 Processor -- start 2024-03-23T17:48:56.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.912+0000 7f1a1f600700 1 -- start start 2024-03-23T17:48:56.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.912+0000 7f1a1f600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a18100c80 0x7f1a18198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:56.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.912+0000 7f1a1f600700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a18101e70 0x7f1a181993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:56.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.912+0000 7f1a1f600700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1a18102810 0x7f1a181a03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:56.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.912+0000 7f1a1f600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a181a0990 con 0x7f1a18100c80 2024-03-23T17:48:56.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.912+0000 7f1a1f600700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a181a0ad0 con 0x7f1a18101e70 2024-03-23T17:48:56.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.912+0000 7f1a1f600700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a181a0c60 con 0x7f1a18102810 2024-03-23T17:48:56.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.912+0000 7f1a1cc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a18100c80 0x7f1a18198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:56.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.912+0000 7f1a1cc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a18100c80 0x7f1a18198e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36668/0 (socket says 172.21.15.86:36668) 2024-03-23T17:48:56.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.912+0000 7f1a1cc00700 1 -- 172.21.15.86:0/862734456 learned_addr learned my addr 172.21.15.86:0/862734456 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:48:56.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.912+0000 7f1a17e00700 1 --2- 172.21.15.86:0/862734456 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a18101e70 0x7f1a181993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:56.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.912+0000 7f1a1d600700 1 --2- 172.21.15.86:0/862734456 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1a18102810 0x7f1a181a03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:56.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.912+0000 7f1a1cc00700 1 -- 172.21.15.86:0/862734456 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1a18102810 msgr2=0x7f1a181a03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:56.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.912+0000 7f1a1cc00700 1 --2- 172.21.15.86:0/862734456 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1a18102810 0x7f1a181a03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:56.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.912+0000 7f1a1cc00700 1 -- 172.21.15.86:0/862734456 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a18101e70 msgr2=0x7f1a181993a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:56.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.912+0000 7f1a1cc00700 1 --2- 172.21.15.86:0/862734456 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a18101e70 0x7f1a181993a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:56.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.912+0000 7f1a1cc00700 1 -- 172.21.15.86:0/862734456 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a040095f0 con 0x7f1a18100c80 2024-03-23T17:48:56.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.913+0000 7f1a1d600700 1 --2- 172.21.15.86:0/862734456 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1a18102810 0x7f1a181a03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T17:48:56.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.913+0000 7f1a17e00700 1 --2- 172.21.15.86:0/862734456 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a18101e70 0x7f1a181993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:48:56.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.913+0000 7f1a1cc00700 1 --2- 172.21.15.86:0/862734456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a18100c80 0x7f1a18198e60 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c00e970 tx=0x7f1a0c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:48:56.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.913+0000 7f1a15600700 1 -- 172.21.15.86:0/862734456 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a0c009790 con 0x7f1a18100c80 2024-03-23T17:48:56.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.913+0000 7f1a15600700 1 -- 172.21.15.86:0/862734456 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a0c0044e0 con 0x7f1a18100c80 2024-03-23T17:48:56.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.913+0000 7f1a15600700 1 -- 172.21.15.86:0/862734456 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a0c003e30 con 0x7f1a18100c80 2024-03-23T17:48:56.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.913+0000 7f1a1f600700 1 -- 172.21.15.86:0/862734456 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a181a0f40 con 0x7f1a18100c80 2024-03-23T17:48:56.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.913+0000 7f1a1f600700 1 -- 172.21.15.86:0/862734456 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a181a12b0 con 0x7f1a18100c80 2024-03-23T17:48:56.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.914+0000 7f1a1f600700 1 -- 172.21.15.86:0/862734456 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a18067a10 con 0x7f1a18100c80 2024-03-23T17:48:56.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.916+0000 7f1a15600700 1 -- 172.21.15.86:0/862734456 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f1a0c003f90 con 0x7f1a18100c80 2024-03-23T17:48:56.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.917+0000 7f1a15600700 1 --2- 172.21.15.86:0/862734456 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f1a000a1120 0x7f1a000a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:56.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.917+0000 7f1a15600700 1 -- 172.21.15.86:0/862734456 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f1a0c014070 con 0x7f1a18100c80 2024-03-23T17:48:56.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.918+0000 7f1a17e00700 1 --2- 172.21.15.86:0/862734456 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f1a000a1120 0x7f1a000a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:56.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.918+0000 7f1a17e00700 1 --2- 172.21.15.86:0/862734456 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f1a000a1120 0x7f1a000a35e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1a0400b580 tx=0x7f1a0401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:48:56.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:56.922+0000 7f1a15600700 1 -- 172.21.15.86:0/862734456 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f1a0c096350 con 0x7f1a18100c80 2024-03-23T17:48:57.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:56 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/1904638533' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-23T17:48:57.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:56 smithi086 ceph-mon[38221]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-23T17:48:57.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:56 smithi086 ceph-mon[38221]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-23T17:48:57.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:56 smithi086 ceph-mon[38221]: osdmap e78: 12 total, 12 up, 12 in 2024-03-23T17:48:57.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:56 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/3157344438' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-23T17:48:57.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:56 smithi086 ceph-mon[38221]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-23T17:48:57.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:56 smithi086 ceph-mon[38221]: osdmap e79: 12 total, 12 up, 12 in 2024-03-23T17:48:57.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:56 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/3157344438' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-23T17:48:57.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:56 smithi086 ceph-mon[38221]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-23T17:48:57.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:56 smithi086 ceph-mon[38221]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-23T17:48:57.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:56 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/1904638533' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-23T17:48:57.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:56 smithi092 ceph-mon[79089]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-23T17:48:57.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:56 smithi092 ceph-mon[79089]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-23T17:48:57.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:56 smithi092 ceph-mon[79089]: osdmap e78: 12 total, 12 up, 12 in 2024-03-23T17:48:57.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:56 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/3157344438' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-23T17:48:57.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:56 smithi092 ceph-mon[79089]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-23T17:48:57.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:56 smithi092 ceph-mon[79089]: osdmap e79: 12 total, 12 up, 12 in 2024-03-23T17:48:57.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:56 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/3157344438' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-23T17:48:57.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:56 smithi092 ceph-mon[79089]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-23T17:48:57.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:56 smithi092 ceph-mon[79089]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-23T17:48:57.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:56 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/1904638533' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-23T17:48:57.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:56 smithi152 ceph-mon[47949]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-23T17:48:57.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:56 smithi152 ceph-mon[47949]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-23T17:48:57.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:56 smithi152 ceph-mon[47949]: osdmap e78: 12 total, 12 up, 12 in 2024-03-23T17:48:57.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:56 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/3157344438' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-23T17:48:57.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:56 smithi152 ceph-mon[47949]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-23T17:48:57.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:56 smithi152 ceph-mon[47949]: osdmap e79: 12 total, 12 up, 12 in 2024-03-23T17:48:57.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:56 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/3157344438' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-23T17:48:57.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:56 smithi152 ceph-mon[47949]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-23T17:48:57.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:56 smithi152 ceph-mon[47949]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-23T17:48:57.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:57.167+0000 7f1a1f600700 1 -- 172.21.15.86:0/862734456 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1a1810c4e0 con 0x7f1a18100c80 2024-03-23T17:48:57.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:57.168+0000 7f1a15600700 1 -- 172.21.15.86:0/862734456 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+22096 (secure 0 0 0) 0x7f1a0c099970 con 0x7f1a18100c80 2024-03-23T17:48:57.170 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:48:57.170 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":79,"fsid":"9d76c9e8-e93c-11ee-95cd-87774f69a715","created":"2024-03-23T17:42:30.097233+0000","modified":"2024-03-23T17:48:56.201953+0000","last_up_change":"2024-03-23T17:48:18.217601+0000","last_in_change":"2024-03-23T17:48:00.250579+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T17:45:26.848397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T17:48:54.331894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T17:48:55.640693+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"656655c6-1c7f-4640-92c4-2f42de820143","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6803","nonce":2423505491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6805","nonce":2423505491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6809","nonce":2423505491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6807","nonce":2423505491}]},"public_addr":"172.21.15.86:6803/2423505491","cluster_addr":"172.21.15.86:6805/2423505491","heartbeat_back_addr":"172.21.15.86:6809/2423505491","heartbeat_front_addr":"172.21.15.86:6807/2423505491","state":["exists","up"]},{"osd":1,"uuid":"3e86dc65-20a6-4de8-b9ce-fcf01f68862b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6801","nonce":3776252597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6803","nonce":3776252597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6807","nonce":3776252597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6805","nonce":3776252597}]},"public_addr":"172.21.15.92:6801/3776252597","cluster_addr":"172.21.15.92:6803/3776252597","heartbeat_back_addr":"172.21.15.92:6807/3776252597","heartbeat_front_addr":"172.21.15.92:6805/3776252597","state":["exists","up"]},{"osd":2,"uuid":"09808786-3bd3-4d11-8c7b-1736119dfbb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6801","nonce":1601265891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6803","nonce":1601265891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6807","nonce":1601265891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6805","nonce":1601265891}]},"public_addr":"172.21.15.152:6801/1601265891","cluster_addr":"172.21.15.152:6803/1601265891","heartbeat_back_addr":"172.21.15.152:6807/1601265891","heartbeat_front_addr":"172.21.15.152:6805/1601265891","state":["exists","up"]},{"osd":3,"uuid":"79c61910-4b53-424c-b150-6f5385a2cd68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6811","nonce":694846466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6813","nonce":694846466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6817","nonce":694846466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6815","nonce":694846466}]},"public_addr":"172.21.15.86:6811/694846466","cluster_addr":"172.21.15.86:6813/694846466","heartbeat_back_addr":"172.21.15.86:6817/694846466","heartbeat_front_addr":"172.21.15.86:6815/694846466","state":["exists","up"]},{"osd":4,"uuid":"a2878035-0179-469f-bbd4-8ce2d3f6bddc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6809","nonce":1422821471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6811","nonce":1422821471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6815","nonce":1422821471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6813","nonce":1422821471}]},"public_addr":"172.21.15.92:6809/1422821471","cluster_addr":"172.21.15.92:6811/1422821471","heartbeat_back_addr":"172.21.15.92:6815/1422821471","heartbeat_front_addr":"172.21.15.92:6813/1422821471","state":["exists","up"]},{"osd":5,"uuid":"837366aa-ce25-42ca-b157-5bbb50159d0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6809","nonce":1039375239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6811","nonce":1039375239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6815","nonce":1039375239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6813","nonce":1039375239}]},"public_addr":"172.21.15.152:6809/1039375239","cluster_addr":"172.21.15.152:6811/1039375239","heartbeat_back_addr":"172.21.15.152:6815/1039375239","heartbeat_front_addr":"172.21.15.152:6813/1039375239","state":["exists","up"]},{"osd":6,"uuid":"61024437-7b91-432f-a1af-6443d0ef5213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6819","nonce":4080452707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6821","nonce":4080452707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6825","nonce":4080452707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6823","nonce":4080452707}]},"public_addr":"172.21.15.86:6819/4080452707","cluster_addr":"172.21.15.86:6821/4080452707","heartbeat_back_addr":"172.21.15.86:6825/4080452707","heartbeat_front_addr":"172.21.15.86:6823/4080452707","state":["exists","up"]},{"osd":7,"uuid":"2684bdae-6b30-4afc-af68-4ce41b3a75b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6817","nonce":4187707622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6819","nonce":4187707622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6823","nonce":4187707622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6821","nonce":4187707622}]},"public_addr":"172.21.15.92:6817/4187707622","cluster_addr":"172.21.15.92:6819/4187707622","heartbeat_back_addr":"172.21.15.92:6823/4187707622","heartbeat_front_addr":"172.21.15.92:6821/4187707622","state":["exists","up"]},{"osd":8,"uuid":"f03bbf0d-1281-4bcf-9773-9854388e65ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6817","nonce":3237612748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6819","nonce":3237612748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6823","nonce":3237612748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6821","nonce":3237612748}]},"public_addr":"172.21.15.152:6817/3237612748","cluster_addr":"172.21.15.152:6819/3237612748","heartbeat_back_addr":"172.21.15.152:6823/3237612748","heartbeat_front_addr":"172.21.15.152:6821/3237612748","state":["exists","up"]},{"osd":9,"uuid":"e7e8b775-99ff-48a4-a23a-91e7ab6622bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6827","nonce":2835218294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6829","nonce":2835218294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6833","nonce":2835218294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6831","nonce":2835218294}]},"public_addr":"172.21.15.86:6827/2835218294","cluster_addr":"172.21.15.86:6829/2835218294","heartbeat_back_addr":"172.21.15.86:6833/2835218294","heartbeat_front_addr":"172.21.15.86:6831/2835218294","state":["exists","up"]},{"osd":10,"uuid":"d9d616cd-a229-44de-8768-02a4415df115","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6825","nonce":4259465100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6827","nonce":4259465100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6831","nonce":4259465100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6829","nonce":4259465100}]},"public_addr":"172.21.15.92:6825/4259465100","cluster_addr":"172.21.15.92:6827/4259465100","heartbeat_back_addr":"172.21.15.92:6831/4259465100","heartbeat_front_addr":"172.21.15.92:6829/4259465100","state":["exists","up"]},{"osd":11,"uuid":"e7c0b865-ccf5-4233-8d22-0d9b6dbe5c97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6825","nonce":194662760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6827","nonce":194662760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6831","nonce":194662760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6829","nonce":194662760}]},"public_addr":"172.21.15.152:6825/194662760","cluster_addr":"172.21.15.152:6827/194662760","heartbeat_back_addr":"172.21.15.152:6831/194662760","heartbeat_front_addr":"172.21.15.152:6829/194662760","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:44:47.269167+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:45:05.363655+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:45:23.192677+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:45:44.156228+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:02.532971+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:20.153295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:41.713888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:59.676516+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:47:17.207087+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:47:38.970316+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:47:57.282525+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:48:15.437606+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/3553500912":"2024-03-24T17:43:28.242492+0000","172.21.15.86:6801/3600278228":"2024-03-24T17:43:28.242492+0000","172.21.15.86:0/887262329":"2024-03-24T17:43:28.242492+0000","172.21.15.86:6800/2313723492":"2024-03-24T17:43:05.043910+0000","172.21.15.86:6800/3600278228":"2024-03-24T17:43:28.242492+0000","172.21.15.86:0/3815705642":"2024-03-24T17:43:05.043910+0000","172.21.15.86:0/1614943062":"2024-03-24T17:43:28.242492+0000","172.21.15.86:6801/2313723492":"2024-03-24T17:43:05.043910+0000","172.21.15.86:0/2416321716":"2024-03-24T17:43:05.043910+0000","172.21.15.86:0/433588188":"2024-03-24T17:43:05.043910+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T17:48:57.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:57.171+0000 7f1a1f600700 1 -- 172.21.15.86:0/862734456 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f1a000a1120 msgr2=0x7f1a000a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:57.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:57.171+0000 7f1a1f600700 1 --2- 172.21.15.86:0/862734456 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f1a000a1120 0x7f1a000a35e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1a0400b580 tx=0x7f1a0401a040 comp rx=0 tx=0).stop 2024-03-23T17:48:57.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:57.171+0000 7f1a1f600700 1 -- 172.21.15.86:0/862734456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a18100c80 msgr2=0x7f1a18198e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:57.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:57.171+0000 7f1a1f600700 1 --2- 172.21.15.86:0/862734456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a18100c80 0x7f1a18198e60 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c00e970 tx=0x7f1a0c00ed30 comp rx=0 tx=0).stop 2024-03-23T17:48:57.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:57.171+0000 7f1a1f600700 1 -- 172.21.15.86:0/862734456 shutdown_connections 2024-03-23T17:48:57.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:57.171+0000 7f1a1f600700 1 --2- 172.21.15.86:0/862734456 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1a18102810 0x7f1a181a03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:57.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:57.171+0000 7f1a1f600700 1 --2- 172.21.15.86:0/862734456 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f1a000a1120 0x7f1a000a35e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:57.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:57.171+0000 7f1a1f600700 1 --2- 172.21.15.86:0/862734456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a18100c80 0x7f1a18198e60 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:57.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:57.171+0000 7f1a1f600700 1 --2- 172.21.15.86:0/862734456 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a18101e70 0x7f1a181993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:57.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:57.171+0000 7f1a1f600700 1 -- 172.21.15.86:0/862734456 >> 172.21.15.86:0/862734456 conn(0x7f1a180fc1f0 msgr2=0x7f1a18072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:48:57.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:57.171+0000 7f1a1f600700 1 -- 172.21.15.86:0/862734456 shutdown_connections 2024-03-23T17:48:57.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:57.171+0000 7f1a1f600700 1 -- 172.21.15.86:0/862734456 wait complete. 2024-03-23T17:48:57.192 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-23T17:48:57.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:57.389+0000 7f783ce00700 1 -- 172.21.15.86:0/895035123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7838100d80 msgr2=0x7f78381011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:57.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:57.389+0000 7f783ce00700 1 --2- 172.21.15.86:0/895035123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7838100d80 0x7f78381011a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f78280099c0 tx=0x7f7828009cd0 comp rx=0 tx=0).stop 2024-03-23T17:48:57.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:57.389+0000 7f783ce00700 1 -- 172.21.15.86:0/895035123 shutdown_connections 2024-03-23T17:48:57.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:57.389+0000 7f783ce00700 1 --2- 172.21.15.86:0/895035123 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7838102910 0x7f78381091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:57.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:57.389+0000 7f783ce00700 1 --2- 172.21.15.86:0/895035123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7838100d80 0x7f78381011a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:57.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:57.389+0000 7f783ce00700 1 --2- 172.21.15.86:0/895035123 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7838101f70 0x7f78381023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:57.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:57.389+0000 7f783ce00700 1 -- 172.21.15.86:0/895035123 >> 172.21.15.86:0/895035123 conn(0x7f78380fc2f0 msgr2=0x7f78380fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:48:57.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:57.389+0000 7f783ce00700 1 -- 172.21.15.86:0/895035123 shutdown_connections 2024-03-23T17:48:57.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:57.389+0000 7f783ce00700 1 -- 172.21.15.86:0/895035123 wait complete. 2024-03-23T17:48:57.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:57.390+0000 7f783ce00700 1 Processor -- start 2024-03-23T17:48:57.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:57.391+0000 7f783ce00700 1 -- start start 2024-03-23T17:48:57.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:57.391+0000 7f783ce00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7838100d80 0x7f7838196d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:57.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:57.391+0000 7f783ce00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7838101f70 0x7f7838197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:57.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:57.391+0000 7f783ce00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7838102910 0x7f783819e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:57.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:57.391+0000 7f783ce00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f783819e840 con 0x7f7838101f70 2024-03-23T17:48:57.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:57.391+0000 7f783ce00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f783819e9b0 con 0x7f7838100d80 2024-03-23T17:48:57.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:57.391+0000 7f783ce00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f783819eb70 con 0x7f7838102910 2024-03-23T17:48:57.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:57.391+0000 7f7835400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7838101f70 0x7f7838197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:57.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:57.391+0000 7f7835400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7838101f70 0x7f7838197290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36688/0 (socket says 172.21.15.86:36688) 2024-03-23T17:48:57.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:57.391+0000 7f7835400700 1 -- 172.21.15.86:0/3388198101 learned_addr learned my addr 172.21.15.86:0/3388198101 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:48:57.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:57.391+0000 7f7836800700 1 --2- 172.21.15.86:0/3388198101 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7838102910 0x7f783819e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:57.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:57.391+0000 7f7835e00700 1 --2- 172.21.15.86:0/3388198101 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7838100d80 0x7f7838196d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:57.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:57.392+0000 7f7835400700 1 -- 172.21.15.86:0/3388198101 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7838102910 msgr2=0x7f783819e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:57.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:57.392+0000 7f7835400700 1 --2- 172.21.15.86:0/3388198101 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7838102910 0x7f783819e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:57.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:57.392+0000 7f7835400700 1 -- 172.21.15.86:0/3388198101 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7838100d80 msgr2=0x7f7838196d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:57.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:57.392+0000 7f7835400700 1 --2- 172.21.15.86:0/3388198101 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7838100d80 0x7f7838196d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:57.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:57.392+0000 7f7835400700 1 -- 172.21.15.86:0/3388198101 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78280095f0 con 0x7f7838101f70 2024-03-23T17:48:57.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:57.392+0000 7f7835400700 1 --2- 172.21.15.86:0/3388198101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7838101f70 0x7f7838197290 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f782400d950 tx=0x7f782400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:48:57.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:57.392+0000 7f782ea00700 1 -- 172.21.15.86:0/3388198101 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f782400f890 con 0x7f7838101f70 2024-03-23T17:48:57.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:57.392+0000 7f783ce00700 1 -- 172.21.15.86:0/3388198101 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f783819ee50 con 0x7f7838101f70 2024-03-23T17:48:57.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:57.392+0000 7f782ea00700 1 -- 172.21.15.86:0/3388198101 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7824010460 con 0x7f7838101f70 2024-03-23T17:48:57.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:57.392+0000 7f782ea00700 1 -- 172.21.15.86:0/3388198101 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f782401b650 con 0x7f7838101f70 2024-03-23T17:48:57.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:57.392+0000 7f783ce00700 1 -- 172.21.15.86:0/3388198101 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f783819f280 con 0x7f7838101f70 2024-03-23T17:48:57.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:57.393+0000 7f783ce00700 1 -- 172.21.15.86:0/3388198101 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78381081a0 con 0x7f7838101f70 2024-03-23T17:48:57.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:57.395+0000 7f782ea00700 1 -- 172.21.15.86:0/3388198101 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f78240108d0 con 0x7f7838101f70 2024-03-23T17:48:57.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:57.396+0000 7f782ea00700 1 --2- 172.21.15.86:0/3388198101 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f781c0a1120 0x7f781c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:57.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:57.396+0000 7f782ea00700 1 -- 172.21.15.86:0/3388198101 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f78240cc920 con 0x7f7838101f70 2024-03-23T17:48:57.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:57.397+0000 7f7835e00700 1 --2- 172.21.15.86:0/3388198101 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f781c0a1120 0x7f781c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:57.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:57.397+0000 7f7835e00700 1 --2- 172.21.15.86:0/3388198101 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f781c0a1120 0x7f781c0a35e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7828009990 tx=0x7f782801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:48:57.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:57.400+0000 7f782ea00700 1 -- 172.21.15.86:0/3388198101 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f7824096160 con 0x7f7838101f70 2024-03-23T17:48:57.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:57.651+0000 7f783ce00700 1 -- 172.21.15.86:0/3388198101 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f7838067af0 con 0x7f7838101f70 2024-03-23T17:48:58.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:57 smithi086 ceph-mon[38221]: pgmap v233: 97 pgs: 6 creating+peering, 1 active+clean, 90 unknown; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T17:48:58.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:57 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/2821120106' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T17:48:58.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:57 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/862734456' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T17:48:58.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:57 smithi086 ceph-mon[38221]: mds.? [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] up:active 2024-03-23T17:48:58.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:57 smithi086 ceph-mon[38221]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-23T17:48:58.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:57 smithi086 ceph-mon[38221]: osdmap e80: 12 total, 12 up, 12 in 2024-03-23T17:48:58.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:57 smithi086 ceph-mon[38221]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-23T17:48:58.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:57 smithi092 ceph-mon[79089]: pgmap v233: 97 pgs: 6 creating+peering, 1 active+clean, 90 unknown; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T17:48:58.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:57 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/2821120106' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T17:48:58.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:57 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/862734456' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T17:48:58.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:57 smithi092 ceph-mon[79089]: mds.? [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] up:active 2024-03-23T17:48:58.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:57 smithi092 ceph-mon[79089]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-23T17:48:58.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:57 smithi092 ceph-mon[79089]: osdmap e80: 12 total, 12 up, 12 in 2024-03-23T17:48:58.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:57 smithi092 ceph-mon[79089]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-23T17:48:58.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:57 smithi152 ceph-mon[47949]: pgmap v233: 97 pgs: 6 creating+peering, 1 active+clean, 90 unknown; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T17:48:58.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:57 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/2821120106' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T17:48:58.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:57 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/862734456' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T17:48:58.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:57 smithi152 ceph-mon[47949]: mds.? [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] up:active 2024-03-23T17:48:58.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:57 smithi152 ceph-mon[47949]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-23T17:48:58.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:57 smithi152 ceph-mon[47949]: osdmap e80: 12 total, 12 up, 12 in 2024-03-23T17:48:58.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:57 smithi152 ceph-mon[47949]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-23T17:48:58.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:58.656+0000 7f782ea00700 1 -- 172.21.15.86:0/3388198101 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) v1 ==== 118+0+0 (secure 0 0 0) 0x7f7824099780 con 0x7f7838101f70 2024-03-23T17:48:58.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:58.659+0000 7f783ce00700 1 -- 172.21.15.86:0/3388198101 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f781c0a1120 msgr2=0x7f781c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:58.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:58.659+0000 7f783ce00700 1 --2- 172.21.15.86:0/3388198101 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f781c0a1120 0x7f781c0a35e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7828009990 tx=0x7f782801a040 comp rx=0 tx=0).stop 2024-03-23T17:48:58.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:58.659+0000 7f783ce00700 1 -- 172.21.15.86:0/3388198101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7838101f70 msgr2=0x7f7838197290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:58.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:58.659+0000 7f783ce00700 1 --2- 172.21.15.86:0/3388198101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7838101f70 0x7f7838197290 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f782400d950 tx=0x7f782400dc60 comp rx=0 tx=0).stop 2024-03-23T17:48:58.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:58.660+0000 7f783ce00700 1 -- 172.21.15.86:0/3388198101 shutdown_connections 2024-03-23T17:48:58.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:58.660+0000 7f783ce00700 1 --2- 172.21.15.86:0/3388198101 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7838102910 0x7f783819e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:58.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:58.660+0000 7f783ce00700 1 --2- 172.21.15.86:0/3388198101 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7838100d80 0x7f7838196d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:58.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:58.660+0000 7f783ce00700 1 --2- 172.21.15.86:0/3388198101 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f781c0a1120 0x7f781c0a35e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:58.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:58.660+0000 7f783ce00700 1 --2- 172.21.15.86:0/3388198101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7838101f70 0x7f7838197290 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:58.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:58.660+0000 7f783ce00700 1 -- 172.21.15.86:0/3388198101 >> 172.21.15.86:0/3388198101 conn(0x7f78380fc2f0 msgr2=0x7f78381052e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:48:58.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:58.660+0000 7f783ce00700 1 -- 172.21.15.86:0/3388198101 shutdown_connections 2024-03-23T17:48:58.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:58.660+0000 7f783ce00700 1 -- 172.21.15.86:0/3388198101 wait complete. 2024-03-23T17:48:58.682 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-03-23T17:48:58.682 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-03-23T17:48:58.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:58.882+0000 7f1fa6000700 1 -- 172.21.15.86:0/1857662698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fa0102910 msgr2=0x7f1fa01091c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:58.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:58.882+0000 7f1fa6000700 1 --2- 172.21.15.86:0/1857662698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fa0102910 0x7f1fa01091c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1f9800b0d0 tx=0x7f1f9800b3e0 comp rx=0 tx=0).stop 2024-03-23T17:48:58.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:58.882+0000 7f1fa6000700 1 -- 172.21.15.86:0/1857662698 shutdown_connections 2024-03-23T17:48:58.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:58.882+0000 7f1fa6000700 1 --2- 172.21.15.86:0/1857662698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fa0102910 0x7f1fa01091c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:58.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:58.882+0000 7f1fa6000700 1 --2- 172.21.15.86:0/1857662698 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fa0100d80 0x7f1fa01011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:58.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:58.882+0000 7f1fa6000700 1 --2- 172.21.15.86:0/1857662698 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1fa0101f70 0x7f1fa01023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:58.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:58.882+0000 7f1fa6000700 1 -- 172.21.15.86:0/1857662698 >> 172.21.15.86:0/1857662698 conn(0x7f1fa00fc2f0 msgr2=0x7f1fa00fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:48:58.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:58.882+0000 7f1fa6000700 1 -- 172.21.15.86:0/1857662698 shutdown_connections 2024-03-23T17:48:58.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:58.882+0000 7f1fa6000700 1 -- 172.21.15.86:0/1857662698 wait complete. 2024-03-23T17:48:58.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:58.883+0000 7f1fa6000700 1 Processor -- start 2024-03-23T17:48:58.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:58.883+0000 7f1fa6000700 1 -- start start 2024-03-23T17:48:58.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:58.883+0000 7f1fa6000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fa0100d80 0x7f1fa0074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:58.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:58.884+0000 7f1fa6000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fa0101f70 0x7f1fa00731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:58.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:58.884+0000 7f1fa6000700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1fa0102910 0x7f1fa00736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:58.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:58.884+0000 7f1fa6000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa0073e80 con 0x7f1fa0101f70 2024-03-23T17:48:58.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:58.884+0000 7f1fa6000700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa0073ff0 con 0x7f1fa0100d80 2024-03-23T17:48:58.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:58.884+0000 7f1fa6000700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa01a0e50 con 0x7f1fa0102910 2024-03-23T17:48:58.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:58.884+0000 7f1f9ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fa0101f70 0x7f1fa00731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:58.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:58.884+0000 7f1f9ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fa0101f70 0x7f1fa00731a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36696/0 (socket says 172.21.15.86:36696) 2024-03-23T17:48:58.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:58.884+0000 7f1f9ea00700 1 -- 172.21.15.86:0/397646856 learned_addr learned my addr 172.21.15.86:0/397646856 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:48:58.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:58.884+0000 7f1f9f400700 1 --2- 172.21.15.86:0/397646856 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fa0100d80 0x7f1fa0074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:58.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:58.884+0000 7f1f9fe00700 1 --2- 172.21.15.86:0/397646856 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1fa0102910 0x7f1fa00736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:58.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:58.884+0000 7f1f9ea00700 1 -- 172.21.15.86:0/397646856 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1fa0102910 msgr2=0x7f1fa00736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:58.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:58.884+0000 7f1f9ea00700 1 --2- 172.21.15.86:0/397646856 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1fa0102910 0x7f1fa00736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:58.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:58.884+0000 7f1f9ea00700 1 -- 172.21.15.86:0/397646856 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fa0100d80 msgr2=0x7f1fa0074b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:58.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:58.884+0000 7f1f9ea00700 1 --2- 172.21.15.86:0/397646856 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fa0100d80 0x7f1fa0074b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:58.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:58.884+0000 7f1f9ea00700 1 -- 172.21.15.86:0/397646856 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f98009d00 con 0x7f1fa0101f70 2024-03-23T17:48:58.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:58.884+0000 7f1f9f400700 1 --2- 172.21.15.86:0/397646856 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fa0100d80 0x7f1fa0074b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:48:58.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:58.884+0000 7f1f9ea00700 1 --2- 172.21.15.86:0/397646856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fa0101f70 0x7f1fa00731a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c00e9a0 tx=0x7f1f8c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:48:58.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:58.885+0000 7f1f97e00700 1 -- 172.21.15.86:0/397646856 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c009790 con 0x7f1fa0101f70 2024-03-23T17:48:58.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:58.885+0000 7f1f97e00700 1 -- 172.21.15.86:0/397646856 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f8c00ce70 con 0x7f1fa0101f70 2024-03-23T17:48:58.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:58.885+0000 7f1fa6000700 1 -- 172.21.15.86:0/397646856 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fa01a10e0 con 0x7f1fa0101f70 2024-03-23T17:48:58.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:58.885+0000 7f1f97e00700 1 -- 172.21.15.86:0/397646856 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c018420 con 0x7f1fa0101f70 2024-03-23T17:48:58.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:58.885+0000 7f1fa6000700 1 -- 172.21.15.86:0/397646856 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fa01a1640 con 0x7f1fa0101f70 2024-03-23T17:48:58.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:58.886+0000 7f1fa6000700 1 -- 172.21.15.86:0/397646856 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fa0108130 con 0x7f1fa0101f70 2024-03-23T17:48:58.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:58.888+0000 7f1f97e00700 1 -- 172.21.15.86:0/397646856 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f1f8c0106f0 con 0x7f1fa0101f70 2024-03-23T17:48:58.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:58.890+0000 7f1f97e00700 1 --2- 172.21.15.86:0/397646856 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f1f840a11a0 0x7f1f840a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:58.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:58.890+0000 7f1f97e00700 1 -- 172.21.15.86:0/397646856 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f1f8c014070 con 0x7f1fa0101f70 2024-03-23T17:48:58.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:58.890+0000 7f1f9f400700 1 --2- 172.21.15.86:0/397646856 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f1f840a11a0 0x7f1f840a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:58.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:58.890+0000 7f1f9f400700 1 --2- 172.21.15.86:0/397646856 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f1f840a11a0 0x7f1f840a3660 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0101dd0 tx=0x7f1f90009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:48:58.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:58.894+0000 7f1f97e00700 1 -- 172.21.15.86:0/397646856 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f1f8c0529f0 con 0x7f1fa0101f70 2024-03-23T17:48:59.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:58 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/3388198101' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-23T17:48:59.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:58 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/3388198101' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-23T17:48:59.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:58 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/3388198101' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-23T17:48:59.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:59.149+0000 7f1fa6000700 1 -- 172.21.15.86:0/397646856 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) v1 -- 0x7f1fa010c820 con 0x7f1fa0101f70 2024-03-23T17:48:59.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:59.658+0000 7f1f97e00700 1 -- 172.21.15.86:0/397646856 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) v1 ==== 105+0+0 (secure 0 0 0) 0x7f1f8c099be0 con 0x7f1fa0101f70 2024-03-23T17:48:59.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:59.661+0000 7f1fa6000700 1 -- 172.21.15.86:0/397646856 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f1f840a11a0 msgr2=0x7f1f840a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:59.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:59.661+0000 7f1fa6000700 1 --2- 172.21.15.86:0/397646856 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f1f840a11a0 0x7f1f840a3660 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0101dd0 tx=0x7f1f90009260 comp rx=0 tx=0).stop 2024-03-23T17:48:59.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:59.661+0000 7f1fa6000700 1 -- 172.21.15.86:0/397646856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fa0101f70 msgr2=0x7f1fa00731a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:59.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:59.661+0000 7f1fa6000700 1 --2- 172.21.15.86:0/397646856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fa0101f70 0x7f1fa00731a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c00e9a0 tx=0x7f1f8c00ecb0 comp rx=0 tx=0).stop 2024-03-23T17:48:59.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:59.662+0000 7f1fa6000700 1 -- 172.21.15.86:0/397646856 shutdown_connections 2024-03-23T17:48:59.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:59.662+0000 7f1fa6000700 1 --2- 172.21.15.86:0/397646856 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1fa0102910 0x7f1fa00736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:59.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:59.662+0000 7f1fa6000700 1 --2- 172.21.15.86:0/397646856 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fa0100d80 0x7f1fa0074b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:59.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:59.662+0000 7f1fa6000700 1 --2- 172.21.15.86:0/397646856 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f1f840a11a0 0x7f1f840a3660 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:59.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:59.662+0000 7f1fa6000700 1 --2- 172.21.15.86:0/397646856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fa0101f70 0x7f1fa00731a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:59.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:59.662+0000 7f1fa6000700 1 -- 172.21.15.86:0/397646856 >> 172.21.15.86:0/397646856 conn(0x7f1fa00fc2f0 msgr2=0x7f1fa0105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:48:59.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:59.662+0000 7f1fa6000700 1 -- 172.21.15.86:0/397646856 shutdown_connections 2024-03-23T17:48:59.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:59.662+0000 7f1fa6000700 1 -- 172.21.15.86:0/397646856 wait complete. 2024-03-23T17:48:59.688 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-03-23T17:48:59.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:59.883+0000 7fba6bc00700 1 -- 172.21.15.86:0/705033519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba640ff680 msgr2=0x7fba640ffaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:59.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:59.883+0000 7fba6bc00700 1 --2- 172.21.15.86:0/705033519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba640ff680 0x7fba640ffaa0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fba58009990 tx=0x7fba58009ca0 comp rx=0 tx=0).stop 2024-03-23T17:48:59.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:59.883+0000 7fba6bc00700 1 -- 172.21.15.86:0/705033519 shutdown_connections 2024-03-23T17:48:59.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:59.883+0000 7fba6bc00700 1 --2- 172.21.15.86:0/705033519 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba641009a0 0x7fba64100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:59.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:59.883+0000 7fba6bc00700 1 --2- 172.21.15.86:0/705033519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba640ff680 0x7fba640ffaa0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:59.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:59.883+0000 7fba6bc00700 1 --2- 172.21.15.86:0/705033519 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fba640fffe0 0x7fba64100460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:59.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:59.883+0000 7fba6bc00700 1 -- 172.21.15.86:0/705033519 >> 172.21.15.86:0/705033519 conn(0x7fba640fb200 msgr2=0x7fba640fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:48:59.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:59.883+0000 7fba6bc00700 1 -- 172.21.15.86:0/705033519 shutdown_connections 2024-03-23T17:48:59.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:59.883+0000 7fba6bc00700 1 -- 172.21.15.86:0/705033519 wait complete. 2024-03-23T17:48:59.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:59.884+0000 7fba6bc00700 1 Processor -- start 2024-03-23T17:48:59.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:59.884+0000 7fba6bc00700 1 -- start start 2024-03-23T17:48:59.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:59.884+0000 7fba6bc00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba640ff680 0x7fba64198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:59.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:59.884+0000 7fba6bc00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fba640fffe0 0x7fba64199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:59.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:59.884+0000 7fba6bc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba641009a0 0x7fba641a0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:59.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:59.884+0000 7fba6bc00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba641a0a40 con 0x7fba641009a0 2024-03-23T17:48:59.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:59.884+0000 7fba6bc00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba641a0b80 con 0x7fba640ff680 2024-03-23T17:48:59.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:59.884+0000 7fba6bc00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba641a0d10 con 0x7fba640fffe0 2024-03-23T17:48:59.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:59.885+0000 7fba69c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba641009a0 0x7fba641a0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:59.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:59.885+0000 7fba69c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba641009a0 0x7fba641a0470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36708/0 (socket says 172.21.15.86:36708) 2024-03-23T17:48:59.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:59.885+0000 7fba69c00700 1 -- 172.21.15.86:0/1082904011 learned_addr learned my addr 172.21.15.86:0/1082904011 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:48:59.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:59.885+0000 7fba69200700 1 --2- 172.21.15.86:0/1082904011 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba640ff680 0x7fba64198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:59.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:59.885+0000 7fba63e00700 1 --2- 172.21.15.86:0/1082904011 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fba640fffe0 0x7fba64199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:59.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:59.885+0000 7fba69c00700 1 -- 172.21.15.86:0/1082904011 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fba640fffe0 msgr2=0x7fba64199450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:59.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:59.885+0000 7fba69c00700 1 --2- 172.21.15.86:0/1082904011 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fba640fffe0 0x7fba64199450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:59.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:59.885+0000 7fba69c00700 1 -- 172.21.15.86:0/1082904011 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba640ff680 msgr2=0x7fba64198f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:48:59.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:59.885+0000 7fba69c00700 1 --2- 172.21.15.86:0/1082904011 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba640ff680 0x7fba64198f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:48:59.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:59.885+0000 7fba69c00700 1 -- 172.21.15.86:0/1082904011 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba580095f0 con 0x7fba641009a0 2024-03-23T17:48:59.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:59.885+0000 7fba69c00700 1 --2- 172.21.15.86:0/1082904011 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba641009a0 0x7fba641a0470 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fba5c007ee0 tx=0x7fba5c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:48:59.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:59.885+0000 7fba61600700 1 -- 172.21.15.86:0/1082904011 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba5c00ddd0 con 0x7fba641009a0 2024-03-23T17:48:59.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:59.886+0000 7fba61600700 1 -- 172.21.15.86:0/1082904011 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba5c00f040 con 0x7fba641009a0 2024-03-23T17:48:59.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:59.886+0000 7fba6bc00700 1 -- 172.21.15.86:0/1082904011 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba641a0ff0 con 0x7fba641009a0 2024-03-23T17:48:59.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:59.886+0000 7fba61600700 1 -- 172.21.15.86:0/1082904011 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba5c0048e0 con 0x7fba641009a0 2024-03-23T17:48:59.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:59.886+0000 7fba6bc00700 1 -- 172.21.15.86:0/1082904011 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba641a14c0 con 0x7fba641009a0 2024-03-23T17:48:59.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:59.887+0000 7fba6bc00700 1 -- 172.21.15.86:0/1082904011 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba6410a740 con 0x7fba641009a0 2024-03-23T17:48:59.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:59.889+0000 7fba61600700 1 -- 172.21.15.86:0/1082904011 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fba5c003a00 con 0x7fba641009a0 2024-03-23T17:48:59.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:59.890+0000 7fba61600700 1 --2- 172.21.15.86:0/1082904011 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fba4c0a11a0 0x7fba4c0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:48:59.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:59.890+0000 7fba61600700 1 -- 172.21.15.86:0/1082904011 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fba5c0cbfd0 con 0x7fba641009a0 2024-03-23T17:48:59.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:59.890+0000 7fba69200700 1 --2- 172.21.15.86:0/1082904011 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fba4c0a11a0 0x7fba4c0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:48:59.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:59.891+0000 7fba69200700 1 --2- 172.21.15.86:0/1082904011 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fba4c0a11a0 0x7fba4c0a3660 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fba5800b5c0 tx=0x7fba58003790 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:48:59.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:48:59.894+0000 7fba61600700 1 -- 172.21.15.86:0/1082904011 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fba5c0958b0 con 0x7fba641009a0 2024-03-23T17:49:00.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:59 smithi086 ceph-mon[38221]: pgmap v235: 97 pgs: 44 active+clean, 6 creating+peering, 47 unknown; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 2 op/s 2024-03-23T17:49:00.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:59 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/3388198101' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-23T17:49:00.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:59 smithi086 ceph-mon[38221]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-23T17:49:00.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:48:59 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/397646856' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-23T17:49:00.057 INFO:journalctl@ceph.mds.b.smithi092.stdout:Mar 23 17:48:59 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-b[115122]: 2024-03-23T17:48:59.880+0000 7fd62c913700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-23T17:49:00.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:59 smithi092 ceph-mon[79089]: pgmap v235: 97 pgs: 44 active+clean, 6 creating+peering, 47 unknown; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 2 op/s 2024-03-23T17:49:00.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:59 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/3388198101' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-23T17:49:00.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:59 smithi092 ceph-mon[79089]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-23T17:49:00.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:48:59 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/397646856' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-23T17:49:00.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:59 smithi152 ceph-mon[47949]: pgmap v235: 97 pgs: 44 active+clean, 6 creating+peering, 47 unknown; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 2 op/s 2024-03-23T17:49:00.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:59 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/3388198101' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-23T17:49:00.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:59 smithi152 ceph-mon[47949]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-23T17:49:00.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:48:59 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/397646856' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-23T17:49:00.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:00.145+0000 7fba6bc00700 1 -- 172.21.15.86:0/1082904011 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7fba64067260 con 0x7fba641009a0 2024-03-23T17:49:00.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:00.647+0000 7fba61600700 1 -- 172.21.15.86:0/1082904011 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v20) v1 ==== 121+0+0 (secure 0 0 0) 0x7fba5c098ed0 con 0x7fba641009a0 2024-03-23T17:49:00.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:00.650+0000 7fba6bc00700 1 -- 172.21.15.86:0/1082904011 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fba4c0a11a0 msgr2=0x7fba4c0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:00.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:00.650+0000 7fba6bc00700 1 --2- 172.21.15.86:0/1082904011 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fba4c0a11a0 0x7fba4c0a3660 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fba5800b5c0 tx=0x7fba58003790 comp rx=0 tx=0).stop 2024-03-23T17:49:00.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:00.650+0000 7fba6bc00700 1 -- 172.21.15.86:0/1082904011 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba641009a0 msgr2=0x7fba641a0470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:00.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:00.650+0000 7fba6bc00700 1 --2- 172.21.15.86:0/1082904011 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba641009a0 0x7fba641a0470 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fba5c007ee0 tx=0x7fba5c00d3b0 comp rx=0 tx=0).stop 2024-03-23T17:49:00.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:00.650+0000 7fba6bc00700 1 -- 172.21.15.86:0/1082904011 shutdown_connections 2024-03-23T17:49:00.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:00.650+0000 7fba6bc00700 1 --2- 172.21.15.86:0/1082904011 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fba4c0a11a0 0x7fba4c0a3660 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:00.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:00.650+0000 7fba6bc00700 1 --2- 172.21.15.86:0/1082904011 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba641009a0 0x7fba641a0470 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:00.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:00.650+0000 7fba6bc00700 1 --2- 172.21.15.86:0/1082904011 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba640ff680 0x7fba64198f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:00.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:00.650+0000 7fba6bc00700 1 --2- 172.21.15.86:0/1082904011 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fba640fffe0 0x7fba64199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:00.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:00.650+0000 7fba6bc00700 1 -- 172.21.15.86:0/1082904011 >> 172.21.15.86:0/1082904011 conn(0x7fba640fb200 msgr2=0x7fba640fca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:00.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:00.650+0000 7fba6bc00700 1 -- 172.21.15.86:0/1082904011 shutdown_connections 2024-03-23T17:49:00.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:00.650+0000 7fba6bc00700 1 -- 172.21.15.86:0/1082904011 wait complete. 2024-03-23T17:49:00.676 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-03-23T17:49:00.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:00.871+0000 7f529b800700 1 -- 172.21.15.86:0/2374491270 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5294074d90 msgr2=0x7f52940731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:00.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:00.871+0000 7f529b800700 1 --2- 172.21.15.86:0/2374491270 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5294074d90 0x7f52940731e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f52880098f0 tx=0x7f5288009c00 comp rx=0 tx=0).stop 2024-03-23T17:49:00.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:00.871+0000 7f529b800700 1 -- 172.21.15.86:0/2374491270 shutdown_connections 2024-03-23T17:49:00.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:00.871+0000 7f529b800700 1 --2- 172.21.15.86:0/2374491270 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5294108c50 0x7f529410b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:00.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:00.871+0000 7f529b800700 1 --2- 172.21.15.86:0/2374491270 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5294074d90 0x7f52940731e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:00.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:00.871+0000 7f529b800700 1 --2- 172.21.15.86:0/2374491270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52940737b0 0x7f5294073c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:00.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:00.871+0000 7f529b800700 1 -- 172.21.15.86:0/2374491270 >> 172.21.15.86:0/2374491270 conn(0x7f52940fc2f0 msgr2=0x7f52940fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:00.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:00.871+0000 7f529b800700 1 -- 172.21.15.86:0/2374491270 shutdown_connections 2024-03-23T17:49:00.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:00.871+0000 7f529b800700 1 -- 172.21.15.86:0/2374491270 wait complete. 2024-03-23T17:49:00.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:00.872+0000 7f529b800700 1 Processor -- start 2024-03-23T17:49:00.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:00.872+0000 7f529b800700 1 -- start start 2024-03-23T17:49:00.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:00.873+0000 7f529b800700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f52940737b0 0x7f5294198f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:00.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:00.873+0000 7f529b800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5294074d90 0x7f52941994a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:00.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:00.873+0000 7f529b800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5294108c50 0x7f52941a04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:00.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:00.873+0000 7f529b800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52941a0a90 con 0x7f5294074d90 2024-03-23T17:49:00.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:00.873+0000 7f529b800700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52941a0bd0 con 0x7f5294108c50 2024-03-23T17:49:00.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:00.873+0000 7f529b800700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52941a0d60 con 0x7f52940737b0 2024-03-23T17:49:00.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:00.873+0000 7f5293e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5294074d90 0x7f52941994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:00.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:00.873+0000 7f5299800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5294108c50 0x7f52941a04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:00.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:00.873+0000 7f5293e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5294074d90 0x7f52941994a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36724/0 (socket says 172.21.15.86:36724) 2024-03-23T17:49:00.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:00.873+0000 7f5293e00700 1 -- 172.21.15.86:0/60040392 learned_addr learned my addr 172.21.15.86:0/60040392 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:00.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:00.873+0000 7f5299800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5294108c50 0x7f52941a04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:49434/0 (socket says 172.21.15.86:49434) 2024-03-23T17:49:00.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:00.873+0000 7f5298e00700 1 --2- 172.21.15.86:0/60040392 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f52940737b0 0x7f5294198f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:00.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:00.873+0000 7f5299800700 1 -- 172.21.15.86:0/60040392 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f52940737b0 msgr2=0x7f5294198f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:00.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:00.873+0000 7f5299800700 1 --2- 172.21.15.86:0/60040392 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f52940737b0 0x7f5294198f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:00.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:00.873+0000 7f5299800700 1 -- 172.21.15.86:0/60040392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5294074d90 msgr2=0x7f52941994a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:00.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:00.873+0000 7f5299800700 1 --2- 172.21.15.86:0/60040392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5294074d90 0x7f52941994a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:00.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:00.873+0000 7f5299800700 1 -- 172.21.15.86:0/60040392 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52800095f0 con 0x7f5294108c50 2024-03-23T17:49:00.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:00.873+0000 7f5293e00700 1 --2- 172.21.15.86:0/60040392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5294074d90 0x7f52941994a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:49:00.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:00.874+0000 7f5299800700 1 --2- 172.21.15.86:0/60040392 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5294108c50 0x7f52941a04c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f528c00efc0 tx=0x7f528c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:00.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:00.874+0000 7f5291600700 1 -- 172.21.15.86:0/60040392 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f528c018070 con 0x7f5294108c50 2024-03-23T17:49:00.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:00.874+0000 7f5291600700 1 -- 172.21.15.86:0/60040392 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f528c00f040 con 0x7f5294108c50 2024-03-23T17:49:00.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:00.874+0000 7f5291600700 1 -- 172.21.15.86:0/60040392 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f528c00cd40 con 0x7f5294108c50 2024-03-23T17:49:00.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:00.874+0000 7f529b800700 1 -- 172.21.15.86:0/60040392 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5288009520 con 0x7f5294108c50 2024-03-23T17:49:00.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:00.874+0000 7f529b800700 1 -- 172.21.15.86:0/60040392 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52941a1280 con 0x7f5294108c50 2024-03-23T17:49:00.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:00.875+0000 7f529b800700 1 -- 172.21.15.86:0/60040392 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5294107a60 con 0x7f5294108c50 2024-03-23T17:49:00.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:00.877+0000 7f5291600700 1 -- 172.21.15.86:0/60040392 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f528c003a00 con 0x7f5294108c50 2024-03-23T17:49:00.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:00.878+0000 7f5291600700 1 --2- 172.21.15.86:0/60040392 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f527c0a1120 0x7f527c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:00.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:00.878+0000 7f5291600700 1 -- 172.21.15.86:0/60040392 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f528c0cc3a0 con 0x7f5294108c50 2024-03-23T17:49:00.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:00.878+0000 7f5298e00700 1 --2- 172.21.15.86:0/60040392 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f527c0a1120 0x7f527c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:00.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:00.878+0000 7f5298e00700 1 --2- 172.21.15.86:0/60040392 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f527c0a1120 0x7f527c0a35e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f528800fa40 tx=0x7f5288019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:00.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:00.882+0000 7f5291600700 1 -- 172.21.15.86:0/60040392 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f528c095c70 con 0x7f5294108c50 2024-03-23T17:49:01.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:00 smithi086 ceph-mon[38221]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-23T17:49:01.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:00 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/397646856' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-03-23T17:49:01.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:00 smithi086 ceph-mon[38221]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-23T17:49:01.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:00 smithi086 ceph-mon[38221]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-23T17:49:01.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:00 smithi086 ceph-mon[38221]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-03-23T17:49:01.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:00 smithi086 ceph-mon[38221]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-03-23T17:49:01.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:00 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/1082904011' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-23T17:49:01.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:00 smithi086 ceph-mon[38221]: mds.? [v2:172.21.15.152:6838/1100723992,v1:172.21.15.152:6839/1100723992] up:active 2024-03-23T17:49:01.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:00 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/1082904011' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-23T17:49:01.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:00 smithi086 ceph-mon[38221]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-23T17:49:01.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:00 smithi086 ceph-mon[38221]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 2 up:standby-replay 8 up:standby 2024-03-23T17:49:01.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:00 smithi086 ceph-mon[38221]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 2 up:standby-replay 7 up:standby 2024-03-23T17:49:01.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:00 smithi092 ceph-mon[79089]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-23T17:49:01.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:00 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/397646856' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-03-23T17:49:01.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:00 smithi092 ceph-mon[79089]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-23T17:49:01.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:00 smithi092 ceph-mon[79089]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-23T17:49:01.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:00 smithi092 ceph-mon[79089]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-03-23T17:49:01.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:00 smithi092 ceph-mon[79089]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-03-23T17:49:01.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:00 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/1082904011' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-23T17:49:01.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:00 smithi092 ceph-mon[79089]: mds.? [v2:172.21.15.152:6838/1100723992,v1:172.21.15.152:6839/1100723992] up:active 2024-03-23T17:49:01.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:00 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/1082904011' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-23T17:49:01.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:00 smithi092 ceph-mon[79089]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-23T17:49:01.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:00 smithi092 ceph-mon[79089]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 2 up:standby-replay 8 up:standby 2024-03-23T17:49:01.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:00 smithi092 ceph-mon[79089]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 2 up:standby-replay 7 up:standby 2024-03-23T17:49:01.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:00 smithi152 ceph-mon[47949]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-23T17:49:01.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:00 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/397646856' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-03-23T17:49:01.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:00 smithi152 ceph-mon[47949]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-23T17:49:01.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:00 smithi152 ceph-mon[47949]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-23T17:49:01.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:00 smithi152 ceph-mon[47949]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-03-23T17:49:01.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:00 smithi152 ceph-mon[47949]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-03-23T17:49:01.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:00 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/1082904011' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-23T17:49:01.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:00 smithi152 ceph-mon[47949]: mds.? [v2:172.21.15.152:6838/1100723992,v1:172.21.15.152:6839/1100723992] up:active 2024-03-23T17:49:01.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:00 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/1082904011' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-23T17:49:01.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:00 smithi152 ceph-mon[47949]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-23T17:49:01.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:00 smithi152 ceph-mon[47949]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 2 up:standby-replay 8 up:standby 2024-03-23T17:49:01.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:00 smithi152 ceph-mon[47949]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 2 up:standby-replay 7 up:standby 2024-03-23T17:49:01.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:01.138+0000 7f529b800700 1 -- 172.21.15.86:0/60040392 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f5294067af0 con 0x7f5294108c50 2024-03-23T17:49:01.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:01.668+0000 7f5291600700 1 -- 172.21.15.86:0/60040392 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v22) v1 ==== 115+0+0 (secure 0 0 0) 0x7f528c099290 con 0x7f5294108c50 2024-03-23T17:49:01.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:01.672+0000 7f529b800700 1 -- 172.21.15.86:0/60040392 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f527c0a1120 msgr2=0x7f527c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:01.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:01.672+0000 7f529b800700 1 --2- 172.21.15.86:0/60040392 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f527c0a1120 0x7f527c0a35e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f528800fa40 tx=0x7f5288019040 comp rx=0 tx=0).stop 2024-03-23T17:49:01.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:01.672+0000 7f529b800700 1 -- 172.21.15.86:0/60040392 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5294108c50 msgr2=0x7f52941a04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:01.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:01.672+0000 7f529b800700 1 --2- 172.21.15.86:0/60040392 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5294108c50 0x7f52941a04c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f528c00efc0 tx=0x7f528c00c3a0 comp rx=0 tx=0).stop 2024-03-23T17:49:01.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:01.672+0000 7f529b800700 1 -- 172.21.15.86:0/60040392 shutdown_connections 2024-03-23T17:49:01.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:01.672+0000 7f529b800700 1 --2- 172.21.15.86:0/60040392 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5294108c50 0x7f52941a04c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:01.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:01.672+0000 7f529b800700 1 --2- 172.21.15.86:0/60040392 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f52940737b0 0x7f5294198f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:01.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:01.672+0000 7f529b800700 1 --2- 172.21.15.86:0/60040392 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f527c0a1120 0x7f527c0a35e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:01.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:01.672+0000 7f529b800700 1 --2- 172.21.15.86:0/60040392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5294074d90 0x7f52941994a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:01.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:01.672+0000 7f529b800700 1 -- 172.21.15.86:0/60040392 >> 172.21.15.86:0/60040392 conn(0x7f52940fc2f0 msgr2=0x7f5294104ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:01.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:01.672+0000 7f529b800700 1 -- 172.21.15.86:0/60040392 shutdown_connections 2024-03-23T17:49:01.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:01.672+0000 7f529b800700 1 -- 172.21.15.86:0/60040392 wait complete. 2024-03-23T17:49:01.700 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-03-23T17:49:01.700 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-03-23T17:49:01.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:01.896+0000 7f1993200700 1 -- 172.21.15.86:0/2494015303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f198c104140 msgr2=0x7f198c106530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:01.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:01.896+0000 7f1993200700 1 --2- 172.21.15.86:0/2494015303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f198c104140 0x7f198c106530 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1978009990 tx=0x7f1978009ca0 comp rx=0 tx=0).stop 2024-03-23T17:49:01.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:01.896+0000 7f1993200700 1 -- 172.21.15.86:0/2494015303 shutdown_connections 2024-03-23T17:49:01.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:01.896+0000 7f1993200700 1 --2- 172.21.15.86:0/2494015303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f198c106a70 0x7f198c108ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:01.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:01.896+0000 7f1993200700 1 --2- 172.21.15.86:0/2494015303 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f198c101810 0x7f198c103c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:01.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:01.896+0000 7f1993200700 1 --2- 172.21.15.86:0/2494015303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f198c104140 0x7f198c106530 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:01.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:01.896+0000 7f1993200700 1 -- 172.21.15.86:0/2494015303 >> 172.21.15.86:0/2494015303 conn(0x7f198c0fb200 msgr2=0x7f198c0fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:01.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:01.896+0000 7f1993200700 1 -- 172.21.15.86:0/2494015303 shutdown_connections 2024-03-23T17:49:01.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:01.896+0000 7f1993200700 1 -- 172.21.15.86:0/2494015303 wait complete. 2024-03-23T17:49:01.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:01.897+0000 7f1993200700 1 Processor -- start 2024-03-23T17:49:01.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:01.897+0000 7f1993200700 1 -- start start 2024-03-23T17:49:01.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:01.897+0000 7f1993200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f198c101810 0x7f198c198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:01.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:01.898+0000 7f1993200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f198c104140 0x7f198c199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:01.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:01.898+0000 7f1993200700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f198c106a70 0x7f198c1a0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:01.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:01.898+0000 7f1993200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f198c1a0a20 con 0x7f198c101810 2024-03-23T17:49:01.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:01.898+0000 7f1993200700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f198c1a0b60 con 0x7f198c104140 2024-03-23T17:49:01.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:01.898+0000 7f1993200700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f198c1a0cf0 con 0x7f198c106a70 2024-03-23T17:49:01.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:01.898+0000 7f198be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f198c101810 0x7f198c198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:01.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:01.898+0000 7f198be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f198c101810 0x7f198c198ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36742/0 (socket says 172.21.15.86:36742) 2024-03-23T17:49:01.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:01.898+0000 7f198be00700 1 -- 172.21.15.86:0/1817588791 learned_addr learned my addr 172.21.15.86:0/1817588791 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:01.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:01.898+0000 7f198b400700 1 --2- 172.21.15.86:0/1817588791 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f198c104140 0x7f198c199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:01.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:01.898+0000 7f1991200700 1 --2- 172.21.15.86:0/1817588791 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f198c106a70 0x7f198c1a0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:01.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:01.898+0000 7f198b400700 1 -- 172.21.15.86:0/1817588791 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f198c106a70 msgr2=0x7f198c1a0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:01.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:01.898+0000 7f198b400700 1 --2- 172.21.15.86:0/1817588791 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f198c106a70 0x7f198c1a0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:01.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:01.898+0000 7f198b400700 1 -- 172.21.15.86:0/1817588791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f198c101810 msgr2=0x7f198c198ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:01.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:01.898+0000 7f198b400700 1 --2- 172.21.15.86:0/1817588791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f198c101810 0x7f198c198ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:01.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:01.898+0000 7f198b400700 1 -- 172.21.15.86:0/1817588791 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19780095f0 con 0x7f198c104140 2024-03-23T17:49:01.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:01.898+0000 7f198be00700 1 --2- 172.21.15.86:0/1817588791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f198c101810 0x7f198c198ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:49:01.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:01.898+0000 7f198b400700 1 --2- 172.21.15.86:0/1817588791 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f198c104140 0x7f198c199430 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f197800b680 tx=0x7f1978004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:01.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:01.899+0000 7f1988c00700 1 -- 172.21.15.86:0/1817588791 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f197801d070 con 0x7f198c104140 2024-03-23T17:49:01.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:01.899+0000 7f1988c00700 1 -- 172.21.15.86:0/1817588791 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f197800fbd0 con 0x7f198c104140 2024-03-23T17:49:01.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:01.899+0000 7f1993200700 1 -- 172.21.15.86:0/1817588791 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f198c1a0f70 con 0x7f198c104140 2024-03-23T17:49:01.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:01.899+0000 7f1988c00700 1 -- 172.21.15.86:0/1817588791 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19780173f0 con 0x7f198c104140 2024-03-23T17:49:01.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:01.899+0000 7f1993200700 1 -- 172.21.15.86:0/1817588791 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f198c1a1330 con 0x7f198c104140 2024-03-23T17:49:01.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:01.900+0000 7f1993200700 1 -- 172.21.15.86:0/1817588791 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f198c108640 con 0x7f198c104140 2024-03-23T17:49:01.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:01.902+0000 7f1988c00700 1 -- 172.21.15.86:0/1817588791 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f1978021410 con 0x7f198c104140 2024-03-23T17:49:01.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:01.903+0000 7f1988c00700 1 --2- 172.21.15.86:0/1817588791 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f19700a1120 0x7f19700a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:01.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:01.903+0000 7f1988c00700 1 -- 172.21.15.86:0/1817588791 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f19780cdef0 con 0x7f198c104140 2024-03-23T17:49:01.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:01.903+0000 7f198be00700 1 --2- 172.21.15.86:0/1817588791 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f19700a1120 0x7f19700a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:01.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:01.904+0000 7f198be00700 1 --2- 172.21.15.86:0/1817588791 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f19700a1120 0x7f19700a35e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f197c0104e0 tx=0x7f197c007480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:01.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:01.905+0000 7f1988c00700 1 -- 172.21.15.86:0/1817588791 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f19780977c0 con 0x7f198c104140 2024-03-23T17:49:02.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:01 smithi086 ceph-mon[38221]: pgmap v236: 97 pgs: 90 active+clean, 3 creating+peering, 4 unknown; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 8 op/s 2024-03-23T17:49:02.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:01 smithi086 ceph-mon[38221]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-03-23T17:49:02.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:01 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/60040392' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-23T17:49:02.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:01 smithi086 ceph-mon[38221]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-23T17:49:02.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:01 smithi092 ceph-mon[79089]: pgmap v236: 97 pgs: 90 active+clean, 3 creating+peering, 4 unknown; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 8 op/s 2024-03-23T17:49:02.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:01 smithi092 ceph-mon[79089]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-03-23T17:49:02.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:01 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/60040392' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-23T17:49:02.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:01 smithi092 ceph-mon[79089]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-23T17:49:02.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:01 smithi152 ceph-mon[47949]: pgmap v236: 97 pgs: 90 active+clean, 3 creating+peering, 4 unknown; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 8 op/s 2024-03-23T17:49:02.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:01 smithi152 ceph-mon[47949]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-03-23T17:49:02.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:01 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/60040392' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-23T17:49:02.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:01 smithi152 ceph-mon[47949]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-23T17:49:02.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.143+0000 7f1993200700 1 -- 172.21.15.86:0/1817588791 --> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f198c0611f0 con 0x7f19700a1120 2024-03-23T17:49:02.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.164+0000 7f1988c00700 1 -- 172.21.15.86:0/1817588791 <== mgr.14152 v2:172.21.15.86:6800/2734784244 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f198c0611f0 con 0x7f19700a1120 2024-03-23T17:49:02.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.166+0000 7f1993200700 1 -- 172.21.15.86:0/1817588791 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f19700a1120 msgr2=0x7f19700a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:02.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.166+0000 7f1993200700 1 --2- 172.21.15.86:0/1817588791 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f19700a1120 0x7f19700a35e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f197c0104e0 tx=0x7f197c007480 comp rx=0 tx=0).stop 2024-03-23T17:49:02.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.166+0000 7f1993200700 1 -- 172.21.15.86:0/1817588791 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f198c104140 msgr2=0x7f198c199430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:02.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.166+0000 7f1993200700 1 --2- 172.21.15.86:0/1817588791 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f198c104140 0x7f198c199430 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f197800b680 tx=0x7f1978004730 comp rx=0 tx=0).stop 2024-03-23T17:49:02.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.167+0000 7f1993200700 1 -- 172.21.15.86:0/1817588791 shutdown_connections 2024-03-23T17:49:02.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.167+0000 7f1993200700 1 --2- 172.21.15.86:0/1817588791 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f198c106a70 0x7f198c1a0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:02.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.167+0000 7f1993200700 1 --2- 172.21.15.86:0/1817588791 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f19700a1120 0x7f19700a35e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:02.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.167+0000 7f1993200700 1 --2- 172.21.15.86:0/1817588791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f198c101810 0x7f198c198ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:02.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.167+0000 7f1993200700 1 --2- 172.21.15.86:0/1817588791 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f198c104140 0x7f198c199430 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:02.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.167+0000 7f1993200700 1 -- 172.21.15.86:0/1817588791 >> 172.21.15.86:0/1817588791 conn(0x7f198c0fb200 msgr2=0x7f198c1010c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:02.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.167+0000 7f1993200700 1 -- 172.21.15.86:0/1817588791 shutdown_connections 2024-03-23T17:49:02.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.167+0000 7f1993200700 1 -- 172.21.15.86:0/1817588791 wait complete. 2024-03-23T17:49:02.186 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-03-23T17:49:02.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.383+0000 7fb751e00700 1 -- 172.21.15.86:0/2137551099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb74c101f70 msgr2=0x7fb74c1023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:02.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.383+0000 7fb751e00700 1 --2- 172.21.15.86:0/2137551099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb74c101f70 0x7fb74c1023d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb7380099c0 tx=0x7fb738009cd0 comp rx=0 tx=0).stop 2024-03-23T17:49:02.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.383+0000 7fb751e00700 1 -- 172.21.15.86:0/2137551099 shutdown_connections 2024-03-23T17:49:02.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.383+0000 7fb751e00700 1 --2- 172.21.15.86:0/2137551099 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb74c102910 0x7fb74c1091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:02.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.383+0000 7fb751e00700 1 --2- 172.21.15.86:0/2137551099 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb74c100d80 0x7fb74c1011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:02.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.383+0000 7fb751e00700 1 --2- 172.21.15.86:0/2137551099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb74c101f70 0x7fb74c1023d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:02.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.383+0000 7fb751e00700 1 -- 172.21.15.86:0/2137551099 >> 172.21.15.86:0/2137551099 conn(0x7fb74c0fc2f0 msgr2=0x7fb74c0fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:02.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.383+0000 7fb751e00700 1 -- 172.21.15.86:0/2137551099 shutdown_connections 2024-03-23T17:49:02.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.383+0000 7fb751e00700 1 -- 172.21.15.86:0/2137551099 wait complete. 2024-03-23T17:49:02.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.384+0000 7fb751e00700 1 Processor -- start 2024-03-23T17:49:02.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.384+0000 7fb751e00700 1 -- start start 2024-03-23T17:49:02.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.385+0000 7fb751e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb74c100d80 0x7fb74c074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:02.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.385+0000 7fb751e00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb74c101f70 0x7fb74c0731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:02.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.385+0000 7fb751e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb74c102910 0x7fb74c0736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:02.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.385+0000 7fb751e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb74c073e80 con 0x7fb74c100d80 2024-03-23T17:49:02.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.385+0000 7fb751e00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb74c073ff0 con 0x7fb74c102910 2024-03-23T17:49:02.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.385+0000 7fb751e00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb74c1a0d70 con 0x7fb74c101f70 2024-03-23T17:49:02.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.385+0000 7fb74aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb74c100d80 0x7fb74c074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:02.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.385+0000 7fb74aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb74c100d80 0x7fb74c074b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36780/0 (socket says 172.21.15.86:36780) 2024-03-23T17:49:02.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.385+0000 7fb74aa00700 1 -- 172.21.15.86:0/3294588960 learned_addr learned my addr 172.21.15.86:0/3294588960 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:02.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.385+0000 7fb74a000700 1 --2- 172.21.15.86:0/3294588960 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb74c101f70 0x7fb74c0731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:02.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.385+0000 7fb74b400700 1 --2- 172.21.15.86:0/3294588960 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb74c102910 0x7fb74c0736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:02.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.385+0000 7fb74aa00700 1 -- 172.21.15.86:0/3294588960 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb74c101f70 msgr2=0x7fb74c0731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:02.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.385+0000 7fb74aa00700 1 --2- 172.21.15.86:0/3294588960 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb74c101f70 0x7fb74c0731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:02.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.385+0000 7fb74aa00700 1 -- 172.21.15.86:0/3294588960 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb74c102910 msgr2=0x7fb74c0736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:02.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.385+0000 7fb74aa00700 1 --2- 172.21.15.86:0/3294588960 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb74c102910 0x7fb74c0736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:02.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.385+0000 7fb74aa00700 1 -- 172.21.15.86:0/3294588960 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7380095f0 con 0x7fb74c100d80 2024-03-23T17:49:02.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.385+0000 7fb74b400700 1 --2- 172.21.15.86:0/3294588960 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb74c102910 0x7fb74c0736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:49:02.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.385+0000 7fb74aa00700 1 --2- 172.21.15.86:0/3294588960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb74c100d80 0x7fb74c074b10 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb73c00e970 tx=0x7fb73c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:02.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.386+0000 7fb750a00700 1 -- 172.21.15.86:0/3294588960 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb73c009790 con 0x7fb74c100d80 2024-03-23T17:49:02.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.386+0000 7fb750a00700 1 -- 172.21.15.86:0/3294588960 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb73c00eec0 con 0x7fb74c100d80 2024-03-23T17:49:02.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.386+0000 7fb751e00700 1 -- 172.21.15.86:0/3294588960 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb74c1a0fa0 con 0x7fb74c100d80 2024-03-23T17:49:02.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.386+0000 7fb750a00700 1 -- 172.21.15.86:0/3294588960 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb73c003c90 con 0x7fb74c100d80 2024-03-23T17:49:02.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.386+0000 7fb751e00700 1 -- 172.21.15.86:0/3294588960 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb74c1a1420 con 0x7fb74c100d80 2024-03-23T17:49:02.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.387+0000 7fb751e00700 1 -- 172.21.15.86:0/3294588960 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb74c108250 con 0x7fb74c100d80 2024-03-23T17:49:02.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.389+0000 7fb750a00700 1 -- 172.21.15.86:0/3294588960 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fb73c00cc70 con 0x7fb74c100d80 2024-03-23T17:49:02.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.390+0000 7fb750a00700 1 --2- 172.21.15.86:0/3294588960 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fb7300a1120 0x7fb7300a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:02.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.390+0000 7fb750a00700 1 -- 172.21.15.86:0/3294588960 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb73c014070 con 0x7fb74c100d80 2024-03-23T17:49:02.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.390+0000 7fb74a000700 1 --2- 172.21.15.86:0/3294588960 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fb7300a1120 0x7fb7300a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:02.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.391+0000 7fb74a000700 1 --2- 172.21.15.86:0/3294588960 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fb7300a1120 0x7fb7300a35e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb738003730 tx=0x7fb73800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:02.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.394+0000 7fb750a00700 1 -- 172.21.15.86:0/3294588960 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fb73c0962b0 con 0x7fb74c100d80 2024-03-23T17:49:02.516 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:49:02 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:49:02.149+0000 7f136b600700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-23T17:49:02.517 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:49:02 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:49:02.149+0000 7f136b600700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-23T17:49:02.517 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:49:02 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:49:02.149+0000 7f136b600700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-23T17:49:02.517 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:49:02 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:49:02.149+0000 7f136b600700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-23T17:49:02.517 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:49:02 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:49:02.149+0000 7f136b600700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-23T17:49:02.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.640+0000 7fb751e00700 1 -- 172.21.15.86:0/3294588960 --> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7fb74c02cdc0 con 0x7fb7300a1120 2024-03-23T17:49:02.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.662+0000 7fb750a00700 1 -- 172.21.15.86:0/3294588960 <== mgr.14152 v2:172.21.15.86:6800/2734784244 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fb74c02cdc0 con 0x7fb7300a1120 2024-03-23T17:49:02.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.664+0000 7fb751e00700 1 -- 172.21.15.86:0/3294588960 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fb7300a1120 msgr2=0x7fb7300a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:02.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.664+0000 7fb751e00700 1 --2- 172.21.15.86:0/3294588960 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fb7300a1120 0x7fb7300a35e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb738003730 tx=0x7fb73800b540 comp rx=0 tx=0).stop 2024-03-23T17:49:02.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.664+0000 7fb751e00700 1 -- 172.21.15.86:0/3294588960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb74c100d80 msgr2=0x7fb74c074b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:02.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.664+0000 7fb751e00700 1 --2- 172.21.15.86:0/3294588960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb74c100d80 0x7fb74c074b10 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb73c00e970 tx=0x7fb73c00ec80 comp rx=0 tx=0).stop 2024-03-23T17:49:02.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.665+0000 7fb751e00700 1 -- 172.21.15.86:0/3294588960 shutdown_connections 2024-03-23T17:49:02.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.665+0000 7fb751e00700 1 --2- 172.21.15.86:0/3294588960 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb74c102910 0x7fb74c0736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:02.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.665+0000 7fb751e00700 1 --2- 172.21.15.86:0/3294588960 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fb7300a1120 0x7fb7300a35e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:02.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.665+0000 7fb751e00700 1 --2- 172.21.15.86:0/3294588960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb74c100d80 0x7fb74c074b10 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:02.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.665+0000 7fb751e00700 1 --2- 172.21.15.86:0/3294588960 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb74c101f70 0x7fb74c0731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:02.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.665+0000 7fb751e00700 1 -- 172.21.15.86:0/3294588960 >> 172.21.15.86:0/3294588960 conn(0x7fb74c0fc2f0 msgr2=0x7fb74c105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:02.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.665+0000 7fb751e00700 1 -- 172.21.15.86:0/3294588960 shutdown_connections 2024-03-23T17:49:02.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.665+0000 7fb751e00700 1 -- 172.21.15.86:0/3294588960 wait complete. 2024-03-23T17:49:02.685 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-03-23T17:49:02.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.889+0000 7f87f4400700 1 -- 172.21.15.86:0/4134281356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87ec100d80 msgr2=0x7f87ec1011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:02.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.889+0000 7f87f4400700 1 --2- 172.21.15.86:0/4134281356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87ec100d80 0x7f87ec1011a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f87e00099c0 tx=0x7f87e0009cd0 comp rx=0 tx=0).stop 2024-03-23T17:49:02.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.889+0000 7f87f4400700 1 -- 172.21.15.86:0/4134281356 shutdown_connections 2024-03-23T17:49:02.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.889+0000 7f87f4400700 1 --2- 172.21.15.86:0/4134281356 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87ec102910 0x7f87ec1091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:02.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.889+0000 7f87f4400700 1 --2- 172.21.15.86:0/4134281356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87ec100d80 0x7f87ec1011a0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:02.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.889+0000 7f87f4400700 1 --2- 172.21.15.86:0/4134281356 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87ec101f70 0x7f87ec1023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:02.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.889+0000 7f87f4400700 1 -- 172.21.15.86:0/4134281356 >> 172.21.15.86:0/4134281356 conn(0x7f87ec0fc2f0 msgr2=0x7f87ec0fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:02.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.889+0000 7f87f4400700 1 -- 172.21.15.86:0/4134281356 shutdown_connections 2024-03-23T17:49:02.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.889+0000 7f87f4400700 1 -- 172.21.15.86:0/4134281356 wait complete. 2024-03-23T17:49:02.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.890+0000 7f87f4400700 1 Processor -- start 2024-03-23T17:49:02.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.890+0000 7f87f4400700 1 -- start start 2024-03-23T17:49:02.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.891+0000 7f87f4400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87ec100d80 0x7f87ec198f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:02.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.891+0000 7f87f4400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87ec101f70 0x7f87ec1994a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:02.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.891+0000 7f87f4400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87ec102910 0x7f87ec1a04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:02.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.891+0000 7f87f4400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87ec1a0a90 con 0x7f87ec101f70 2024-03-23T17:49:02.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.891+0000 7f87f4400700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87ec1a0bd0 con 0x7f87ec102910 2024-03-23T17:49:02.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.891+0000 7f87f4400700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87ec1a0d60 con 0x7f87ec100d80 2024-03-23T17:49:02.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.891+0000 7f87f1000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87ec101f70 0x7f87ec1994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:02.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.891+0000 7f87f1a00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87ec100d80 0x7f87ec198f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:02.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.891+0000 7f87f1a00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87ec100d80 0x7f87ec198f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:37594/0 (socket says 172.21.15.86:37594) 2024-03-23T17:49:02.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.891+0000 7f87f1a00700 1 -- 172.21.15.86:0/2100314173 learned_addr learned my addr 172.21.15.86:0/2100314173 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:02.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.891+0000 7f87f1a00700 1 -- 172.21.15.86:0/2100314173 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87ec102910 msgr2=0x7f87ec1a04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:02.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.891+0000 7f87f1a00700 1 --2- 172.21.15.86:0/2100314173 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87ec102910 0x7f87ec1a04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:02.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.891+0000 7f87f1a00700 1 -- 172.21.15.86:0/2100314173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87ec101f70 msgr2=0x7f87ec1994a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:02.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.891+0000 7f87f1a00700 1 --2- 172.21.15.86:0/2100314173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87ec101f70 0x7f87ec1994a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:02.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.891+0000 7f87f1a00700 1 -- 172.21.15.86:0/2100314173 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87e00095f0 con 0x7f87ec100d80 2024-03-23T17:49:02.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.892+0000 7f87f1a00700 1 --2- 172.21.15.86:0/2100314173 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87ec100d80 0x7f87ec198f60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f87e0006010 tx=0x7f87e000f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:02.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.892+0000 7f87ea000700 1 -- 172.21.15.86:0/2100314173 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e001d070 con 0x7f87ec100d80 2024-03-23T17:49:02.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.892+0000 7f87ea000700 1 -- 172.21.15.86:0/2100314173 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f87e0005020 con 0x7f87ec100d80 2024-03-23T17:49:02.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.892+0000 7f87f4400700 1 -- 172.21.15.86:0/2100314173 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87ec1a0fe0 con 0x7f87ec100d80 2024-03-23T17:49:02.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.892+0000 7f87ea000700 1 -- 172.21.15.86:0/2100314173 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e0004af0 con 0x7f87ec100d80 2024-03-23T17:49:02.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.892+0000 7f87f4400700 1 -- 172.21.15.86:0/2100314173 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87ec1a1350 con 0x7f87ec100d80 2024-03-23T17:49:02.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.893+0000 7f87f4400700 1 -- 172.21.15.86:0/2100314173 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87ec067af0 con 0x7f87ec100d80 2024-03-23T17:49:02.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.895+0000 7f87ea000700 1 -- 172.21.15.86:0/2100314173 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f87e0005190 con 0x7f87ec100d80 2024-03-23T17:49:02.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.896+0000 7f87ea000700 1 --2- 172.21.15.86:0/2100314173 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f87d40a11b0 0x7f87d40a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:02.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.896+0000 7f87ea000700 1 -- 172.21.15.86:0/2100314173 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f87e00cd3e0 con 0x7f87ec100d80 2024-03-23T17:49:02.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.896+0000 7f87f1000700 1 --2- 172.21.15.86:0/2100314173 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f87d40a11b0 0x7f87d40a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:02.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.897+0000 7f87f1000700 1 --2- 172.21.15.86:0/2100314173 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f87d40a11b0 0x7f87d40a3670 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f87d8009c60 tx=0x7f87d8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:02.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:02.900+0000 7f87ea000700 1 -- 172.21.15.86:0/2100314173 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f87e0096b50 con 0x7f87ec100d80 2024-03-23T17:49:03.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:02 smithi086 ceph-mon[38221]: mds.? [v2:172.21.15.92:6836/3967837948,v1:172.21.15.92:6837/3967837948] up:active 2024-03-23T17:49:03.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:02 smithi086 ceph-mon[38221]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-23T17:49:03.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:02 smithi086 ceph-mon[38221]: daemon mds.d assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-03-23T17:49:03.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:02 smithi086 ceph-mon[38221]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-03-23T17:49:03.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:02 smithi086 ceph-mon[38221]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=d=up:creating} 2 up:standby-replay 6 up:standby 2024-03-23T17:49:03.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:02 smithi086 ceph-mon[38221]: daemon mds.d is now active in filesystem cephfs as rank 3 2024-03-23T17:49:03.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:02 smithi092 ceph-mon[79089]: mds.? [v2:172.21.15.92:6836/3967837948,v1:172.21.15.92:6837/3967837948] up:active 2024-03-23T17:49:03.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:02 smithi092 ceph-mon[79089]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-23T17:49:03.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:02 smithi092 ceph-mon[79089]: daemon mds.d assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-03-23T17:49:03.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:02 smithi092 ceph-mon[79089]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-03-23T17:49:03.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:02 smithi092 ceph-mon[79089]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=d=up:creating} 2 up:standby-replay 6 up:standby 2024-03-23T17:49:03.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:02 smithi092 ceph-mon[79089]: daemon mds.d is now active in filesystem cephfs as rank 3 2024-03-23T17:49:03.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:02 smithi152 ceph-mon[47949]: mds.? [v2:172.21.15.92:6836/3967837948,v1:172.21.15.92:6837/3967837948] up:active 2024-03-23T17:49:03.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:02 smithi152 ceph-mon[47949]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-23T17:49:03.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:02 smithi152 ceph-mon[47949]: daemon mds.d assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-03-23T17:49:03.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:02 smithi152 ceph-mon[47949]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-03-23T17:49:03.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:02 smithi152 ceph-mon[47949]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=d=up:creating} 2 up:standby-replay 6 up:standby 2024-03-23T17:49:03.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:02 smithi152 ceph-mon[47949]: daemon mds.d is now active in filesystem cephfs as rank 3 2024-03-23T17:49:03.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.145+0000 7f87f4400700 1 -- 172.21.15.86:0/2100314173 --> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7f87ec108e60 con 0x7f87d40a11b0 2024-03-23T17:49:03.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.150+0000 7f87ea000700 1 -- 172.21.15.86:0/2100314173 <== mgr.14152 v2:172.21.15.86:6800/2734784244 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7f87ec108e60 con 0x7f87d40a11b0 2024-03-23T17:49:03.151 INFO:teuthology.orchestra.run.smithi086.stdout:/volumes/qa/sv_0/50ba1f49-08bd-4f31-bec2-889a257755ef 2024-03-23T17:49:03.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.152+0000 7f87f4400700 1 -- 172.21.15.86:0/2100314173 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f87d40a11b0 msgr2=0x7f87d40a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:03.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.152+0000 7f87f4400700 1 --2- 172.21.15.86:0/2100314173 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f87d40a11b0 0x7f87d40a3670 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f87d8009c60 tx=0x7f87d8009260 comp rx=0 tx=0).stop 2024-03-23T17:49:03.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.152+0000 7f87f4400700 1 -- 172.21.15.86:0/2100314173 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87ec100d80 msgr2=0x7f87ec198f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:03.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.152+0000 7f87f4400700 1 --2- 172.21.15.86:0/2100314173 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87ec100d80 0x7f87ec198f60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f87e0006010 tx=0x7f87e000f690 comp rx=0 tx=0).stop 2024-03-23T17:49:03.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.152+0000 7f87f4400700 1 -- 172.21.15.86:0/2100314173 shutdown_connections 2024-03-23T17:49:03.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.152+0000 7f87f4400700 1 --2- 172.21.15.86:0/2100314173 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87ec102910 0x7f87ec1a04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:03.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.152+0000 7f87f4400700 1 --2- 172.21.15.86:0/2100314173 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87ec100d80 0x7f87ec198f60 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:03.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.152+0000 7f87f4400700 1 --2- 172.21.15.86:0/2100314173 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f87d40a11b0 0x7f87d40a3670 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:03.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.152+0000 7f87f4400700 1 --2- 172.21.15.86:0/2100314173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87ec101f70 0x7f87ec1994a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:03.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.152+0000 7f87f4400700 1 -- 172.21.15.86:0/2100314173 >> 172.21.15.86:0/2100314173 conn(0x7f87ec0fc2f0 msgr2=0x7f87ec107740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:03.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.152+0000 7f87f4400700 1 -- 172.21.15.86:0/2100314173 shutdown_connections 2024-03-23T17:49:03.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.152+0000 7f87f4400700 1 -- 172.21.15.86:0/2100314173 wait complete. 2024-03-23T17:49:03.173 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-03-23T17:49:03.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.369+0000 7f7c85200700 1 -- 172.21.15.86:0/1385529834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c80101f70 msgr2=0x7f7c801023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:03.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.369+0000 7f7c85200700 1 --2- 172.21.15.86:0/1385529834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c80101f70 0x7f7c801023d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c0099c0 tx=0x7f7c6c009cd0 comp rx=0 tx=0).stop 2024-03-23T17:49:03.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.369+0000 7f7c85200700 1 -- 172.21.15.86:0/1385529834 shutdown_connections 2024-03-23T17:49:03.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.369+0000 7f7c85200700 1 --2- 172.21.15.86:0/1385529834 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c80102910 0x7f7c801091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:03.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.369+0000 7f7c85200700 1 --2- 172.21.15.86:0/1385529834 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c80100d80 0x7f7c801011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:03.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.369+0000 7f7c85200700 1 --2- 172.21.15.86:0/1385529834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c80101f70 0x7f7c801023d0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:03.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.369+0000 7f7c85200700 1 -- 172.21.15.86:0/1385529834 >> 172.21.15.86:0/1385529834 conn(0x7f7c800fc2f0 msgr2=0x7f7c800fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:03.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.369+0000 7f7c85200700 1 -- 172.21.15.86:0/1385529834 shutdown_connections 2024-03-23T17:49:03.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.369+0000 7f7c85200700 1 -- 172.21.15.86:0/1385529834 wait complete. 2024-03-23T17:49:03.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.370+0000 7f7c85200700 1 Processor -- start 2024-03-23T17:49:03.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.370+0000 7f7c85200700 1 -- start start 2024-03-23T17:49:03.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.370+0000 7f7c85200700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c80100d80 0x7f7c80198f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:03.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.370+0000 7f7c85200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c80101f70 0x7f7c80199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:03.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.371+0000 7f7c85200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c80102910 0x7f7c801a0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:03.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.371+0000 7f7c85200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c801a0a60 con 0x7f7c80101f70 2024-03-23T17:49:03.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.371+0000 7f7c85200700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c801a0ba0 con 0x7f7c80102910 2024-03-23T17:49:03.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.371+0000 7f7c85200700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c801a0d30 con 0x7f7c80100d80 2024-03-23T17:49:03.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.371+0000 7f7c7d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c80101f70 0x7f7c80199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:03.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.371+0000 7f7c7de00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c80100d80 0x7f7c80198f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:03.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.371+0000 7f7c7de00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c80100d80 0x7f7c80198f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:37612/0 (socket says 172.21.15.86:37612) 2024-03-23T17:49:03.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.371+0000 7f7c7de00700 1 -- 172.21.15.86:0/44695261 learned_addr learned my addr 172.21.15.86:0/44695261 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:03.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.371+0000 7f7c7d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c80101f70 0x7f7c80199470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36824/0 (socket says 172.21.15.86:36824) 2024-03-23T17:49:03.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.371+0000 7f7c7e800700 1 --2- 172.21.15.86:0/44695261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c80102910 0x7f7c801a0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:03.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.371+0000 7f7c7de00700 1 -- 172.21.15.86:0/44695261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c80102910 msgr2=0x7f7c801a0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:03.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.371+0000 7f7c7de00700 1 --2- 172.21.15.86:0/44695261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c80102910 0x7f7c801a0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:03.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.371+0000 7f7c7de00700 1 -- 172.21.15.86:0/44695261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c80101f70 msgr2=0x7f7c80199470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:03.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.371+0000 7f7c7de00700 1 --2- 172.21.15.86:0/44695261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c80101f70 0x7f7c80199470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:03.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.371+0000 7f7c7de00700 1 -- 172.21.15.86:0/44695261 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c6c0095f0 con 0x7f7c80100d80 2024-03-23T17:49:03.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.371+0000 7f7c7d400700 1 --2- 172.21.15.86:0/44695261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c80101f70 0x7f7c80199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:49:03.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.371+0000 7f7c7de00700 1 --2- 172.21.15.86:0/44695261 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c80100d80 0x7f7c80198f30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7c7000e970 tx=0x7f7c7000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:03.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.372+0000 7f7c76a00700 1 -- 172.21.15.86:0/44695261 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c70009790 con 0x7f7c80100d80 2024-03-23T17:49:03.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.372+0000 7f7c76a00700 1 -- 172.21.15.86:0/44695261 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c7000c7e0 con 0x7f7c80100d80 2024-03-23T17:49:03.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.372+0000 7f7c85200700 1 -- 172.21.15.86:0/44695261 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c801a0fb0 con 0x7f7c80100d80 2024-03-23T17:49:03.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.372+0000 7f7c76a00700 1 -- 172.21.15.86:0/44695261 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c70003c70 con 0x7f7c80100d80 2024-03-23T17:49:03.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.372+0000 7f7c85200700 1 -- 172.21.15.86:0/44695261 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c801a1430 con 0x7f7c80100d80 2024-03-23T17:49:03.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.373+0000 7f7c85200700 1 -- 172.21.15.86:0/44695261 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c80067af0 con 0x7f7c80100d80 2024-03-23T17:49:03.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.375+0000 7f7c76a00700 1 -- 172.21.15.86:0/44695261 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f7c7000cc50 con 0x7f7c80100d80 2024-03-23T17:49:03.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.376+0000 7f7c76a00700 1 --2- 172.21.15.86:0/44695261 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f7c640a11b0 0x7f7c640a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:03.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.376+0000 7f7c76a00700 1 -- 172.21.15.86:0/44695261 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f7c70014070 con 0x7f7c80100d80 2024-03-23T17:49:03.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.376+0000 7f7c7d400700 1 --2- 172.21.15.86:0/44695261 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f7c640a11b0 0x7f7c640a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:03.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.377+0000 7f7c7d400700 1 --2- 172.21.15.86:0/44695261 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f7c640a11b0 0x7f7c640a3670 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c006010 tx=0x7f7c6c00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:03.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.378+0000 7f7c76a00700 1 -- 172.21.15.86:0/44695261 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f7c70096340 con 0x7f7c80100d80 2024-03-23T17:49:03.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.621+0000 7f7c85200700 1 -- 172.21.15.86:0/44695261 --> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f7c80074700 con 0x7f7c640a11b0 2024-03-23T17:49:03.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.644+0000 7f7c76a00700 1 -- 172.21.15.86:0/44695261 <== mgr.14152 v2:172.21.15.86:6800/2734784244 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f7c80074700 con 0x7f7c640a11b0 2024-03-23T17:49:03.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.646+0000 7f7c85200700 1 -- 172.21.15.86:0/44695261 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f7c640a11b0 msgr2=0x7f7c640a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:03.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.646+0000 7f7c85200700 1 --2- 172.21.15.86:0/44695261 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f7c640a11b0 0x7f7c640a3670 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c006010 tx=0x7f7c6c00b540 comp rx=0 tx=0).stop 2024-03-23T17:49:03.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.646+0000 7f7c85200700 1 -- 172.21.15.86:0/44695261 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c80100d80 msgr2=0x7f7c80198f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:03.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.646+0000 7f7c85200700 1 --2- 172.21.15.86:0/44695261 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c80100d80 0x7f7c80198f30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7c7000e970 tx=0x7f7c7000ec80 comp rx=0 tx=0).stop 2024-03-23T17:49:03.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.646+0000 7f7c85200700 1 -- 172.21.15.86:0/44695261 shutdown_connections 2024-03-23T17:49:03.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.646+0000 7f7c85200700 1 --2- 172.21.15.86:0/44695261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c80102910 0x7f7c801a0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:03.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.646+0000 7f7c85200700 1 --2- 172.21.15.86:0/44695261 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c80100d80 0x7f7c80198f30 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:03.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.646+0000 7f7c85200700 1 --2- 172.21.15.86:0/44695261 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f7c640a11b0 0x7f7c640a3670 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:03.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.646+0000 7f7c85200700 1 --2- 172.21.15.86:0/44695261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c80101f70 0x7f7c80199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:03.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.646+0000 7f7c85200700 1 -- 172.21.15.86:0/44695261 >> 172.21.15.86:0/44695261 conn(0x7f7c800fc2f0 msgr2=0x7f7c80072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:03.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.646+0000 7f7c85200700 1 -- 172.21.15.86:0/44695261 shutdown_connections 2024-03-23T17:49:03.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.647+0000 7f7c85200700 1 -- 172.21.15.86:0/44695261 wait complete. 2024-03-23T17:49:03.667 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-03-23T17:49:03.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.868+0000 7f00aa400700 1 -- 172.21.15.86:0/2953027248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00a4101f70 msgr2=0x7f00a41023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:03.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.868+0000 7f00aa400700 1 --2- 172.21.15.86:0/2953027248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00a4101f70 0x7f00a41023d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f00900099c0 tx=0x7f0090009cd0 comp rx=0 tx=0).stop 2024-03-23T17:49:03.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.868+0000 7f00aa400700 1 -- 172.21.15.86:0/2953027248 shutdown_connections 2024-03-23T17:49:03.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.868+0000 7f00aa400700 1 --2- 172.21.15.86:0/2953027248 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00a4102910 0x7f00a41091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:03.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.868+0000 7f00aa400700 1 --2- 172.21.15.86:0/2953027248 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f00a4100d80 0x7f00a41011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:03.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.868+0000 7f00aa400700 1 --2- 172.21.15.86:0/2953027248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00a4101f70 0x7f00a41023d0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:03.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.868+0000 7f00aa400700 1 -- 172.21.15.86:0/2953027248 >> 172.21.15.86:0/2953027248 conn(0x7f00a40fc2f0 msgr2=0x7f00a40fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:03.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.868+0000 7f00aa400700 1 -- 172.21.15.86:0/2953027248 shutdown_connections 2024-03-23T17:49:03.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.868+0000 7f00aa400700 1 -- 172.21.15.86:0/2953027248 wait complete. 2024-03-23T17:49:03.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.869+0000 7f00aa400700 1 Processor -- start 2024-03-23T17:49:03.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.869+0000 7f00aa400700 1 -- start start 2024-03-23T17:49:03.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.870+0000 7f00aa400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f00a4100d80 0x7f00a4198f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:03.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.870+0000 7f00aa400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00a4101f70 0x7f00a41994c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:03.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.870+0000 7f00aa400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00a4102910 0x7f00a41a0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:03.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.870+0000 7f00aa400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a41a0a60 con 0x7f00a4101f70 2024-03-23T17:49:03.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.870+0000 7f00aa400700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a41a0ba0 con 0x7f00a4102910 2024-03-23T17:49:03.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.870+0000 7f00aa400700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a41a0d30 con 0x7f00a4100d80 2024-03-23T17:49:03.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.870+0000 7f00a2a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00a4101f70 0x7f00a41994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:03.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.870+0000 7f00a2a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00a4101f70 0x7f00a41994c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36844/0 (socket says 172.21.15.86:36844) 2024-03-23T17:49:03.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.870+0000 7f00a2a00700 1 -- 172.21.15.86:0/4051260191 learned_addr learned my addr 172.21.15.86:0/4051260191 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:03.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.870+0000 7f00a3400700 1 --2- 172.21.15.86:0/4051260191 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f00a4100d80 0x7f00a4198f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:03.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.870+0000 7f00a3e00700 1 --2- 172.21.15.86:0/4051260191 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00a4102910 0x7f00a41a0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:03.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.870+0000 7f00a2a00700 1 -- 172.21.15.86:0/4051260191 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f00a4100d80 msgr2=0x7f00a4198f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:03.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.870+0000 7f00a2a00700 1 --2- 172.21.15.86:0/4051260191 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f00a4100d80 0x7f00a4198f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:03.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.870+0000 7f00a2a00700 1 -- 172.21.15.86:0/4051260191 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00a4102910 msgr2=0x7f00a41a0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:03.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.870+0000 7f00a2a00700 1 --2- 172.21.15.86:0/4051260191 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00a4102910 0x7f00a41a0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:03.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.870+0000 7f00a2a00700 1 -- 172.21.15.86:0/4051260191 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00900095f0 con 0x7f00a4101f70 2024-03-23T17:49:03.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.871+0000 7f00a2a00700 1 --2- 172.21.15.86:0/4051260191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00a4101f70 0x7f00a41994c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f00900036f0 tx=0x7f0090009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:03.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.871+0000 7f009be00700 1 -- 172.21.15.86:0/4051260191 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f009001d070 con 0x7f00a4101f70 2024-03-23T17:49:03.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.871+0000 7f009be00700 1 -- 172.21.15.86:0/4051260191 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0090004050 con 0x7f00a4101f70 2024-03-23T17:49:03.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.871+0000 7f00aa400700 1 -- 172.21.15.86:0/4051260191 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00a41a0fb0 con 0x7f00a4101f70 2024-03-23T17:49:03.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.871+0000 7f009be00700 1 -- 172.21.15.86:0/4051260191 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0090022cf0 con 0x7f00a4101f70 2024-03-23T17:49:03.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.871+0000 7f00aa400700 1 -- 172.21.15.86:0/4051260191 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00a41a13d0 con 0x7f00a4101f70 2024-03-23T17:49:03.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.872+0000 7f00aa400700 1 -- 172.21.15.86:0/4051260191 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00a4067af0 con 0x7f00a4101f70 2024-03-23T17:49:03.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.874+0000 7f009be00700 1 -- 172.21.15.86:0/4051260191 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f0090022470 con 0x7f00a4101f70 2024-03-23T17:49:03.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.875+0000 7f009be00700 1 --2- 172.21.15.86:0/4051260191 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f00880a1230 0x7f00880a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:03.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.875+0000 7f009be00700 1 -- 172.21.15.86:0/4051260191 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f00900cda40 con 0x7f00a4101f70 2024-03-23T17:49:03.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.875+0000 7f00a3400700 1 --2- 172.21.15.86:0/4051260191 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f00880a1230 0x7f00880a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:03.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.876+0000 7f00a3400700 1 --2- 172.21.15.86:0/4051260191 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f00880a1230 0x7f00880a36f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f00a4101dd0 tx=0x7f0094007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:03.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:03.879+0000 7f009be00700 1 -- 172.21.15.86:0/4051260191 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f009005d9f0 con 0x7f00a4101f70 2024-03-23T17:49:04.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:03 smithi086 ceph-mon[38221]: from='client.24647 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:49:04.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:03 smithi086 ceph-mon[38221]: pgmap v237: 97 pgs: 97 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 154 B/s rd, 3.2 KiB/s wr, 13 op/s 2024-03-23T17:49:04.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:03 smithi086 ceph-mon[38221]: from='client.14619 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:49:04.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:03 smithi086 ceph-mon[38221]: mds.? [v2:172.21.15.86:6836/2933881328,v1:172.21.15.86:6837/2933881328] up:active 2024-03-23T17:49:04.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:03 smithi086 ceph-mon[38221]: daemon mds.e assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-03-23T17:49:04.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:03 smithi086 ceph-mon[38221]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-23T17:49:04.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:03 smithi086 ceph-mon[38221]: Cluster is now healthy 2024-03-23T17:49:04.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:03 smithi086 ceph-mon[38221]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=d=up:active} 2 up:standby-replay 6 up:standby 2024-03-23T17:49:04.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:03 smithi086 ceph-mon[38221]: mgrmap e17: x(active, since 5m), standbys: z, y 2024-03-23T17:49:04.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:03 smithi086 ceph-mon[38221]: fsmap cephfs:5 2 up:standby-replay 1 up:creating 5 up:standby 4 up:active 2024-03-23T17:49:04.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:03 smithi086 ceph-mon[38221]: daemon mds.e is now active in filesystem cephfs as rank 4 2024-03-23T17:49:04.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:03 smithi092 ceph-mon[79089]: from='client.24647 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:49:04.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:03 smithi092 ceph-mon[79089]: pgmap v237: 97 pgs: 97 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 154 B/s rd, 3.2 KiB/s wr, 13 op/s 2024-03-23T17:49:04.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:03 smithi092 ceph-mon[79089]: from='client.14619 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:49:04.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:03 smithi092 ceph-mon[79089]: mds.? [v2:172.21.15.86:6836/2933881328,v1:172.21.15.86:6837/2933881328] up:active 2024-03-23T17:49:04.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:03 smithi092 ceph-mon[79089]: daemon mds.e assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-03-23T17:49:04.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:03 smithi092 ceph-mon[79089]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-23T17:49:04.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:03 smithi092 ceph-mon[79089]: Cluster is now healthy 2024-03-23T17:49:04.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:03 smithi092 ceph-mon[79089]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=d=up:active} 2 up:standby-replay 6 up:standby 2024-03-23T17:49:04.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:03 smithi092 ceph-mon[79089]: mgrmap e17: x(active, since 5m), standbys: z, y 2024-03-23T17:49:04.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:03 smithi092 ceph-mon[79089]: fsmap cephfs:5 2 up:standby-replay 1 up:creating 5 up:standby 4 up:active 2024-03-23T17:49:04.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:03 smithi092 ceph-mon[79089]: daemon mds.e is now active in filesystem cephfs as rank 4 2024-03-23T17:49:04.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:03 smithi152 ceph-mon[47949]: from='client.24647 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:49:04.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:03 smithi152 ceph-mon[47949]: pgmap v237: 97 pgs: 97 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 154 B/s rd, 3.2 KiB/s wr, 13 op/s 2024-03-23T17:49:04.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:03 smithi152 ceph-mon[47949]: from='client.14619 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:49:04.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:03 smithi152 ceph-mon[47949]: mds.? [v2:172.21.15.86:6836/2933881328,v1:172.21.15.86:6837/2933881328] up:active 2024-03-23T17:49:04.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:03 smithi152 ceph-mon[47949]: daemon mds.e assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-03-23T17:49:04.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:03 smithi152 ceph-mon[47949]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-23T17:49:04.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:03 smithi152 ceph-mon[47949]: Cluster is now healthy 2024-03-23T17:49:04.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:03 smithi152 ceph-mon[47949]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=d=up:active} 2 up:standby-replay 6 up:standby 2024-03-23T17:49:04.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:03 smithi152 ceph-mon[47949]: mgrmap e17: x(active, since 5m), standbys: z, y 2024-03-23T17:49:04.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:03 smithi152 ceph-mon[47949]: fsmap cephfs:5 2 up:standby-replay 1 up:creating 5 up:standby 4 up:active 2024-03-23T17:49:04.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:03 smithi152 ceph-mon[47949]: daemon mds.e is now active in filesystem cephfs as rank 4 2024-03-23T17:49:04.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.122+0000 7f00aa400700 1 -- 172.21.15.86:0/4051260191 --> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f00a4108eb0 con 0x7f00880a1230 2024-03-23T17:49:04.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.127+0000 7f009be00700 1 -- 172.21.15.86:0/4051260191 <== mgr.14152 v2:172.21.15.86:6800/2734784244 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7f00a4108eb0 con 0x7f00880a1230 2024-03-23T17:49:04.128 INFO:teuthology.orchestra.run.smithi086.stdout:/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc 2024-03-23T17:49:04.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.130+0000 7f00aa400700 1 -- 172.21.15.86:0/4051260191 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f00880a1230 msgr2=0x7f00880a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:04.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.130+0000 7f00aa400700 1 --2- 172.21.15.86:0/4051260191 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f00880a1230 0x7f00880a36f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f00a4101dd0 tx=0x7f0094007400 comp rx=0 tx=0).stop 2024-03-23T17:49:04.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.130+0000 7f00aa400700 1 -- 172.21.15.86:0/4051260191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00a4101f70 msgr2=0x7f00a41994c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:04.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.130+0000 7f00aa400700 1 --2- 172.21.15.86:0/4051260191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00a4101f70 0x7f00a41994c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f00900036f0 tx=0x7f0090009580 comp rx=0 tx=0).stop 2024-03-23T17:49:04.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.130+0000 7f00aa400700 1 -- 172.21.15.86:0/4051260191 shutdown_connections 2024-03-23T17:49:04.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.130+0000 7f00aa400700 1 --2- 172.21.15.86:0/4051260191 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00a4102910 0x7f00a41a0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:04.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.130+0000 7f00aa400700 1 --2- 172.21.15.86:0/4051260191 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f00a4100d80 0x7f00a4198f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:04.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.130+0000 7f00aa400700 1 --2- 172.21.15.86:0/4051260191 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f00880a1230 0x7f00880a36f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:04.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.130+0000 7f00aa400700 1 --2- 172.21.15.86:0/4051260191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00a4101f70 0x7f00a41994c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:04.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.130+0000 7f00aa400700 1 -- 172.21.15.86:0/4051260191 >> 172.21.15.86:0/4051260191 conn(0x7f00a40fc2f0 msgr2=0x7f00a4107790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:04.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.130+0000 7f00aa400700 1 -- 172.21.15.86:0/4051260191 shutdown_connections 2024-03-23T17:49:04.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.130+0000 7f00aa400700 1 -- 172.21.15.86:0/4051260191 wait complete. 2024-03-23T17:49:04.149 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T17:49:04.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.344+0000 7f2352200700 1 -- 172.21.15.86:0/3776491321 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f234c102810 msgr2=0x7f234c1090c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:04.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.344+0000 7f2352200700 1 --2- 172.21.15.86:0/3776491321 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f234c102810 0x7f234c1090c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f234400b0a0 tx=0x7f234400b3b0 comp rx=0 tx=0).stop 2024-03-23T17:49:04.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.344+0000 7f2352200700 1 -- 172.21.15.86:0/3776491321 shutdown_connections 2024-03-23T17:49:04.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.344+0000 7f2352200700 1 --2- 172.21.15.86:0/3776491321 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f234c102810 0x7f234c1090c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:04.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.344+0000 7f2352200700 1 --2- 172.21.15.86:0/3776491321 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f234c100c80 0x7f234c1010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:04.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.344+0000 7f2352200700 1 --2- 172.21.15.86:0/3776491321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f234c101e70 0x7f234c1022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:04.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.344+0000 7f2352200700 1 -- 172.21.15.86:0/3776491321 >> 172.21.15.86:0/3776491321 conn(0x7f234c0fc1f0 msgr2=0x7f234c0fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:04.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.344+0000 7f2352200700 1 -- 172.21.15.86:0/3776491321 shutdown_connections 2024-03-23T17:49:04.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.344+0000 7f2352200700 1 -- 172.21.15.86:0/3776491321 wait complete. 2024-03-23T17:49:04.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.345+0000 7f2352200700 1 Processor -- start 2024-03-23T17:49:04.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.345+0000 7f2352200700 1 -- start start 2024-03-23T17:49:04.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.346+0000 7f2352200700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f234c100c80 0x7f234c198f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:04.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.346+0000 7f2352200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f234c101e70 0x7f234c1994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:04.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.346+0000 7f2352200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f234c102810 0x7f234c1a04d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:04.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.346+0000 7f2352200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f234c1a0aa0 con 0x7f234c101e70 2024-03-23T17:49:04.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.346+0000 7f2352200700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f234c1a0be0 con 0x7f234c102810 2024-03-23T17:49:04.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.346+0000 7f2352200700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f234c1a0d70 con 0x7f234c100c80 2024-03-23T17:49:04.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.346+0000 7f234aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f234c101e70 0x7f234c1994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:04.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.346+0000 7f234aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f234c101e70 0x7f234c1994b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36870/0 (socket says 172.21.15.86:36870) 2024-03-23T17:49:04.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.346+0000 7f234aa00700 1 -- 172.21.15.86:0/748463261 learned_addr learned my addr 172.21.15.86:0/748463261 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:04.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.346+0000 7f234be00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f234c102810 0x7f234c1a04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:04.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.346+0000 7f234b400700 1 --2- 172.21.15.86:0/748463261 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f234c100c80 0x7f234c198f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:04.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.346+0000 7f234aa00700 1 -- 172.21.15.86:0/748463261 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f234c100c80 msgr2=0x7f234c198f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:04.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.346+0000 7f234aa00700 1 --2- 172.21.15.86:0/748463261 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f234c100c80 0x7f234c198f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:04.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.346+0000 7f234aa00700 1 -- 172.21.15.86:0/748463261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f234c102810 msgr2=0x7f234c1a04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:04.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.346+0000 7f234aa00700 1 --2- 172.21.15.86:0/748463261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f234c102810 0x7f234c1a04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:04.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.346+0000 7f234aa00700 1 -- 172.21.15.86:0/748463261 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2344009d00 con 0x7f234c101e70 2024-03-23T17:49:04.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.347+0000 7f234aa00700 1 --2- 172.21.15.86:0/748463261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f234c101e70 0x7f234c1994b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f233800e9a0 tx=0x7f233800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:04.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.347+0000 7f2343e00700 1 -- 172.21.15.86:0/748463261 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2338009790 con 0x7f234c101e70 2024-03-23T17:49:04.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.347+0000 7f2343e00700 1 -- 172.21.15.86:0/748463261 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f233800cbd0 con 0x7f234c101e70 2024-03-23T17:49:04.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.347+0000 7f2352200700 1 -- 172.21.15.86:0/748463261 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f234c1a0ff0 con 0x7f234c101e70 2024-03-23T17:49:04.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.347+0000 7f2343e00700 1 -- 172.21.15.86:0/748463261 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2338018680 con 0x7f234c101e70 2024-03-23T17:49:04.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.347+0000 7f2352200700 1 -- 172.21.15.86:0/748463261 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f234c1a14d0 con 0x7f234c101e70 2024-03-23T17:49:04.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.348+0000 7f2352200700 1 -- 172.21.15.86:0/748463261 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f234c067a10 con 0x7f234c101e70 2024-03-23T17:49:04.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.350+0000 7f2343e00700 1 -- 172.21.15.86:0/748463261 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f233800cd40 con 0x7f234c101e70 2024-03-23T17:49:04.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.351+0000 7f2343e00700 1 --2- 172.21.15.86:0/748463261 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f23300a11b0 0x7f23300a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:04.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.351+0000 7f2343e00700 1 -- 172.21.15.86:0/748463261 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f2338014070 con 0x7f234c101e70 2024-03-23T17:49:04.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.351+0000 7f234b400700 1 --2- 172.21.15.86:0/748463261 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f23300a11b0 0x7f23300a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:04.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.352+0000 7f234b400700 1 --2- 172.21.15.86:0/748463261 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f23300a11b0 0x7f23300a3670 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f234c101cd0 tx=0x7f233c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:04.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.356+0000 7f2343e00700 1 -- 172.21.15.86:0/748463261 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f23380529f0 con 0x7f234c101e70 2024-03-23T17:49:04.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.613+0000 7f2352200700 1 -- 172.21.15.86:0/748463261 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f234c10c4f0 con 0x7f234c101e70 2024-03-23T17:49:04.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.614+0000 7f2343e00700 1 -- 172.21.15.86:0/748463261 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10913 (secure 0 0 0) 0x7f23380962f0 con 0x7f234c101e70 2024-03-23T17:49:04.616 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:49:04.617 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14445,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1433140219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1433140219},{"type":"v1","addr":"172.21.15.86:6835","nonce":1433140219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6839/1879457730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1879457730},{"type":"v1","addr":"172.21.15.86:6839","nonce":1879457730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-23T17:48:56.202323+0000","modified":"2024-03-23T17:49:03.703276+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24553,"mds_2":24518,"mds_3":24505,"mds_4":24500},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14514":{"gid":14514,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/3026943866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3026943866},{"type":"v1","addr":"172.21.15.86:6841","nonce":3026943866}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24464":{"gid":24464,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6833/1467091186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1467091186},{"type":"v1","addr":"172.21.15.92:6833","nonce":1467091186}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6833/3075554623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3075554623},{"type":"v1","addr":"172.21.15.152:6833","nonce":3075554623}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.92:6835/908900027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":908900027},{"type":"v1","addr":"172.21.15.92:6835","nonce":908900027}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"d","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.86:6837/2933881328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2933881328},{"type":"v1","addr":"172.21.15.86:6837","nonce":2933881328}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6837/3967837948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":3967837948},{"type":"v1","addr":"172.21.15.92:6837","nonce":3967837948}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6835/3137085021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3137085021},{"type":"v1","addr":"172.21.15.152:6835","nonce":3137085021}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24538":{"gid":24538,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6837/1151208965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":1151208965},{"type":"v1","addr":"172.21.15.152:6837","nonce":1151208965}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.152:6839/1100723992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":1100723992},{"type":"v1","addr":"172.21.15.152:6839","nonce":1100723992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6839/1394139887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1394139887},{"type":"v1","addr":"172.21.15.92:6839","nonce":1394139887}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-23T17:49:04.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.617+0000 7f2352200700 1 -- 172.21.15.86:0/748463261 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f23300a11b0 msgr2=0x7f23300a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:04.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.617+0000 7f2352200700 1 --2- 172.21.15.86:0/748463261 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f23300a11b0 0x7f23300a3670 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f234c101cd0 tx=0x7f233c009260 comp rx=0 tx=0).stop 2024-03-23T17:49:04.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.617+0000 7f2352200700 1 -- 172.21.15.86:0/748463261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f234c101e70 msgr2=0x7f234c1994b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:04.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.617+0000 7f2352200700 1 --2- 172.21.15.86:0/748463261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f234c101e70 0x7f234c1994b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f233800e9a0 tx=0x7f233800ecb0 comp rx=0 tx=0).stop 2024-03-23T17:49:04.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.617+0000 7f2352200700 1 -- 172.21.15.86:0/748463261 shutdown_connections 2024-03-23T17:49:04.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.617+0000 7f2352200700 1 --2- 172.21.15.86:0/748463261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f234c102810 0x7f234c1a04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:04.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.617+0000 7f2352200700 1 --2- 172.21.15.86:0/748463261 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f234c100c80 0x7f234c198f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:04.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.617+0000 7f2352200700 1 --2- 172.21.15.86:0/748463261 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f23300a11b0 0x7f23300a3670 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:04.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.617+0000 7f2352200700 1 --2- 172.21.15.86:0/748463261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f234c101e70 0x7f234c1994b0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:04.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.617+0000 7f2352200700 1 -- 172.21.15.86:0/748463261 >> 172.21.15.86:0/748463261 conn(0x7f234c0fc1f0 msgr2=0x7f234c072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:04.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.617+0000 7f2352200700 1 -- 172.21.15.86:0/748463261 shutdown_connections 2024-03-23T17:49:04.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.617+0000 7f2352200700 1 -- 172.21.15.86:0/748463261 wait complete. 2024-03-23T17:49:04.619 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 27 2024-03-23T17:49:04.637 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T17:49:04.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.829+0000 7fe85cc00700 1 -- 172.21.15.86:0/3240734206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe858100c80 msgr2=0x7fe8581010a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:04.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.829+0000 7fe85cc00700 1 --2- 172.21.15.86:0/3240734206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe858100c80 0x7fe8581010a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe8480099c0 tx=0x7fe848009cd0 comp rx=0 tx=0).stop 2024-03-23T17:49:04.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.829+0000 7fe85cc00700 1 -- 172.21.15.86:0/3240734206 shutdown_connections 2024-03-23T17:49:04.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.829+0000 7fe85cc00700 1 --2- 172.21.15.86:0/3240734206 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe858102810 0x7fe8581090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:04.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.829+0000 7fe85cc00700 1 --2- 172.21.15.86:0/3240734206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe858100c80 0x7fe8581010a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:04.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.829+0000 7fe85cc00700 1 --2- 172.21.15.86:0/3240734206 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe858101e70 0x7fe8581022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:04.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.829+0000 7fe85cc00700 1 -- 172.21.15.86:0/3240734206 >> 172.21.15.86:0/3240734206 conn(0x7fe8580fc1f0 msgr2=0x7fe8580fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:04.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.829+0000 7fe85cc00700 1 -- 172.21.15.86:0/3240734206 shutdown_connections 2024-03-23T17:49:04.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.830+0000 7fe85cc00700 1 -- 172.21.15.86:0/3240734206 wait complete. 2024-03-23T17:49:04.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.830+0000 7fe85cc00700 1 Processor -- start 2024-03-23T17:49:04.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.831+0000 7fe85cc00700 1 -- start start 2024-03-23T17:49:04.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.831+0000 7fe85cc00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe858100c80 0x7fe858074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:04.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.831+0000 7fe85cc00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe858101e70 0x7fe8580731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:04.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.831+0000 7fe85cc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe858102810 0x7fe8580736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:04.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.831+0000 7fe85cc00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe858073e80 con 0x7fe858102810 2024-03-23T17:49:04.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.831+0000 7fe85cc00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe858073ff0 con 0x7fe858101e70 2024-03-23T17:49:04.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.831+0000 7fe85cc00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8581a0cc0 con 0x7fe858100c80 2024-03-23T17:49:04.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.831+0000 7fe856800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe858102810 0x7fe8580736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:04.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.831+0000 7fe856800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe858102810 0x7fe8580736e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36878/0 (socket says 172.21.15.86:36878) 2024-03-23T17:49:04.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.831+0000 7fe856800700 1 -- 172.21.15.86:0/1253825365 learned_addr learned my addr 172.21.15.86:0/1253825365 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:04.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.831+0000 7fe855e00700 1 --2- 172.21.15.86:0/1253825365 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe858100c80 0x7fe858074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:04.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.831+0000 7fe855400700 1 --2- 172.21.15.86:0/1253825365 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe858101e70 0x7fe8580731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:04.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.831+0000 7fe856800700 1 -- 172.21.15.86:0/1253825365 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe858100c80 msgr2=0x7fe858074b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:04.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.831+0000 7fe856800700 1 --2- 172.21.15.86:0/1253825365 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe858100c80 0x7fe858074b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:04.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.831+0000 7fe856800700 1 -- 172.21.15.86:0/1253825365 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe858101e70 msgr2=0x7fe8580731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:04.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.831+0000 7fe856800700 1 --2- 172.21.15.86:0/1253825365 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe858101e70 0x7fe8580731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:04.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.831+0000 7fe856800700 1 -- 172.21.15.86:0/1253825365 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8480095f0 con 0x7fe858102810 2024-03-23T17:49:04.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.831+0000 7fe855e00700 1 --2- 172.21.15.86:0/1253825365 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe858100c80 0x7fe858074b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:49:04.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.831+0000 7fe855400700 1 --2- 172.21.15.86:0/1253825365 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe858101e70 0x7fe8580731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:49:04.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.832+0000 7fe856800700 1 --2- 172.21.15.86:0/1253825365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe858102810 0x7fe8580736e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe85000ef30 tx=0x7fe85000c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:04.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.832+0000 7fe84ea00700 1 -- 172.21.15.86:0/1253825365 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe850018070 con 0x7fe858102810 2024-03-23T17:49:04.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.832+0000 7fe84ea00700 1 -- 172.21.15.86:0/1253825365 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe85000f040 con 0x7fe858102810 2024-03-23T17:49:04.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.832+0000 7fe84ea00700 1 -- 172.21.15.86:0/1253825365 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe85000ce00 con 0x7fe858102810 2024-03-23T17:49:04.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.832+0000 7fe85cc00700 1 -- 172.21.15.86:0/1253825365 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8581a0ef0 con 0x7fe858102810 2024-03-23T17:49:04.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.832+0000 7fe85cc00700 1 -- 172.21.15.86:0/1253825365 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8581a1320 con 0x7fe858102810 2024-03-23T17:49:04.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.833+0000 7fe85cc00700 1 -- 172.21.15.86:0/1253825365 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe858108150 con 0x7fe858102810 2024-03-23T17:49:04.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.835+0000 7fe84ea00700 1 -- 172.21.15.86:0/1253825365 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fe850003a00 con 0x7fe858102810 2024-03-23T17:49:04.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.836+0000 7fe84ea00700 1 --2- 172.21.15.86:0/1253825365 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fe83c0a1230 0x7fe83c0a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:04.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.836+0000 7fe84ea00700 1 -- 172.21.15.86:0/1253825365 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fe8500cc430 con 0x7fe858102810 2024-03-23T17:49:04.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.837+0000 7fe855e00700 1 --2- 172.21.15.86:0/1253825365 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fe83c0a1230 0x7fe83c0a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:04.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.837+0000 7fe855e00700 1 --2- 172.21.15.86:0/1253825365 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fe83c0a1230 0x7fe83c0a36f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe848009990 tx=0x7fe84801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:04.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:04.840+0000 7fe84ea00700 1 -- 172.21.15.86:0/1253825365 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fe850095ca0 con 0x7fe858102810 2024-03-23T17:49:05.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:04 smithi086 ceph-mon[38221]: from='client.24685 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:49:05.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:04 smithi086 ceph-mon[38221]: from='client.24691 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:49:05.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:04 smithi086 ceph-mon[38221]: mds.? [v2:172.21.15.92:6834/908900027,v1:172.21.15.92:6835/908900027] up:active 2024-03-23T17:49:05.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:04 smithi086 ceph-mon[38221]: fsmap cephfs:5 2 up:standby-replay 5 up:standby 5 up:active 2024-03-23T17:49:05.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:04 smithi086 ceph-mon[38221]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T17:49:05.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:04 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/748463261' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T17:49:05.057 INFO:journalctl@ceph.mds.b.smithi092.stdout:Mar 23 17:49:04 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-b[115122]: 2024-03-23T17:49:04.879+0000 7fd62c913700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-03-23T17:49:05.058 INFO:journalctl@ceph.mds.b.smithi092.stdout:Mar 23 17:49:04 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-b[115122]: 2024-03-23T17:49:04.879+0000 7fd62c913700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-03-23T17:49:05.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:04 smithi092 ceph-mon[79089]: from='client.24685 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:49:05.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:04 smithi092 ceph-mon[79089]: from='client.24691 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:49:05.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:04 smithi092 ceph-mon[79089]: mds.? [v2:172.21.15.92:6834/908900027,v1:172.21.15.92:6835/908900027] up:active 2024-03-23T17:49:05.059 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:04 smithi092 ceph-mon[79089]: fsmap cephfs:5 2 up:standby-replay 5 up:standby 5 up:active 2024-03-23T17:49:05.059 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:04 smithi092 ceph-mon[79089]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T17:49:05.059 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:04 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/748463261' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T17:49:05.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.080+0000 7fe85cc00700 1 -- 172.21.15.86:0/1253825365 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe85810c570 con 0x7fe858102810 2024-03-23T17:49:05.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.081+0000 7fe84ea00700 1 -- 172.21.15.86:0/1253825365 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+22083 (secure 0 0 0) 0x7fe8500992c0 con 0x7fe858102810 2024-03-23T17:49:05.083 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:49:05.083 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":80,"fsid":"9d76c9e8-e93c-11ee-95cd-87774f69a715","created":"2024-03-23T17:42:30.097233+0000","modified":"2024-03-23T17:48:57.204624+0000","last_up_change":"2024-03-23T17:48:18.217601+0000","last_in_change":"2024-03-23T17:48:00.250579+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T17:45:26.848397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T17:48:54.331894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T17:48:55.640693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"656655c6-1c7f-4640-92c4-2f42de820143","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6803","nonce":2423505491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6805","nonce":2423505491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6809","nonce":2423505491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6807","nonce":2423505491}]},"public_addr":"172.21.15.86:6803/2423505491","cluster_addr":"172.21.15.86:6805/2423505491","heartbeat_back_addr":"172.21.15.86:6809/2423505491","heartbeat_front_addr":"172.21.15.86:6807/2423505491","state":["exists","up"]},{"osd":1,"uuid":"3e86dc65-20a6-4de8-b9ce-fcf01f68862b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6801","nonce":3776252597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6803","nonce":3776252597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6807","nonce":3776252597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6805","nonce":3776252597}]},"public_addr":"172.21.15.92:6801/3776252597","cluster_addr":"172.21.15.92:6803/3776252597","heartbeat_back_addr":"172.21.15.92:6807/3776252597","heartbeat_front_addr":"172.21.15.92:6805/3776252597","state":["exists","up"]},{"osd":2,"uuid":"09808786-3bd3-4d11-8c7b-1736119dfbb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6801","nonce":1601265891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6803","nonce":1601265891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6807","nonce":1601265891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6805","nonce":1601265891}]},"public_addr":"172.21.15.152:6801/1601265891","cluster_addr":"172.21.15.152:6803/1601265891","heartbeat_back_addr":"172.21.15.152:6807/1601265891","heartbeat_front_addr":"172.21.15.152:6805/1601265891","state":["exists","up"]},{"osd":3,"uuid":"79c61910-4b53-424c-b150-6f5385a2cd68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6811","nonce":694846466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6813","nonce":694846466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6817","nonce":694846466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6815","nonce":694846466}]},"public_addr":"172.21.15.86:6811/694846466","cluster_addr":"172.21.15.86:6813/694846466","heartbeat_back_addr":"172.21.15.86:6817/694846466","heartbeat_front_addr":"172.21.15.86:6815/694846466","state":["exists","up"]},{"osd":4,"uuid":"a2878035-0179-469f-bbd4-8ce2d3f6bddc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6809","nonce":1422821471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6811","nonce":1422821471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6815","nonce":1422821471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6813","nonce":1422821471}]},"public_addr":"172.21.15.92:6809/1422821471","cluster_addr":"172.21.15.92:6811/1422821471","heartbeat_back_addr":"172.21.15.92:6815/1422821471","heartbeat_front_addr":"172.21.15.92:6813/1422821471","state":["exists","up"]},{"osd":5,"uuid":"837366aa-ce25-42ca-b157-5bbb50159d0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6809","nonce":1039375239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6811","nonce":1039375239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6815","nonce":1039375239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6813","nonce":1039375239}]},"public_addr":"172.21.15.152:6809/1039375239","cluster_addr":"172.21.15.152:6811/1039375239","heartbeat_back_addr":"172.21.15.152:6815/1039375239","heartbeat_front_addr":"172.21.15.152:6813/1039375239","state":["exists","up"]},{"osd":6,"uuid":"61024437-7b91-432f-a1af-6443d0ef5213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6819","nonce":4080452707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6821","nonce":4080452707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6825","nonce":4080452707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6823","nonce":4080452707}]},"public_addr":"172.21.15.86:6819/4080452707","cluster_addr":"172.21.15.86:6821/4080452707","heartbeat_back_addr":"172.21.15.86:6825/4080452707","heartbeat_front_addr":"172.21.15.86:6823/4080452707","state":["exists","up"]},{"osd":7,"uuid":"2684bdae-6b30-4afc-af68-4ce41b3a75b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6817","nonce":4187707622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6819","nonce":4187707622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6823","nonce":4187707622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6821","nonce":4187707622}]},"public_addr":"172.21.15.92:6817/4187707622","cluster_addr":"172.21.15.92:6819/4187707622","heartbeat_back_addr":"172.21.15.92:6823/4187707622","heartbeat_front_addr":"172.21.15.92:6821/4187707622","state":["exists","up"]},{"osd":8,"uuid":"f03bbf0d-1281-4bcf-9773-9854388e65ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6817","nonce":3237612748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6819","nonce":3237612748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6823","nonce":3237612748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6821","nonce":3237612748}]},"public_addr":"172.21.15.152:6817/3237612748","cluster_addr":"172.21.15.152:6819/3237612748","heartbeat_back_addr":"172.21.15.152:6823/3237612748","heartbeat_front_addr":"172.21.15.152:6821/3237612748","state":["exists","up"]},{"osd":9,"uuid":"e7e8b775-99ff-48a4-a23a-91e7ab6622bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6827","nonce":2835218294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6829","nonce":2835218294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6833","nonce":2835218294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6831","nonce":2835218294}]},"public_addr":"172.21.15.86:6827/2835218294","cluster_addr":"172.21.15.86:6829/2835218294","heartbeat_back_addr":"172.21.15.86:6833/2835218294","heartbeat_front_addr":"172.21.15.86:6831/2835218294","state":["exists","up"]},{"osd":10,"uuid":"d9d616cd-a229-44de-8768-02a4415df115","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6825","nonce":4259465100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6827","nonce":4259465100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6831","nonce":4259465100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6829","nonce":4259465100}]},"public_addr":"172.21.15.92:6825/4259465100","cluster_addr":"172.21.15.92:6827/4259465100","heartbeat_back_addr":"172.21.15.92:6831/4259465100","heartbeat_front_addr":"172.21.15.92:6829/4259465100","state":["exists","up"]},{"osd":11,"uuid":"e7c0b865-ccf5-4233-8d22-0d9b6dbe5c97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6825","nonce":194662760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6827","nonce":194662760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6831","nonce":194662760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6829","nonce":194662760}]},"public_addr":"172.21.15.152:6825/194662760","cluster_addr":"172.21.15.152:6827/194662760","heartbeat_back_addr":"172.21.15.152:6831/194662760","heartbeat_front_addr":"172.21.15.152:6829/194662760","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:44:47.269167+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:45:05.363655+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:45:23.192677+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:45:44.156228+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:02.532971+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:20.153295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:41.713888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:59.676516+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:47:17.207087+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:47:38.970316+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:47:57.282525+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:48:15.437606+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/3553500912":"2024-03-24T17:43:28.242492+0000","172.21.15.86:6801/3600278228":"2024-03-24T17:43:28.242492+0000","172.21.15.86:0/887262329":"2024-03-24T17:43:28.242492+0000","172.21.15.86:6800/2313723492":"2024-03-24T17:43:05.043910+0000","172.21.15.86:6800/3600278228":"2024-03-24T17:43:28.242492+0000","172.21.15.86:0/3815705642":"2024-03-24T17:43:05.043910+0000","172.21.15.86:0/1614943062":"2024-03-24T17:43:28.242492+0000","172.21.15.86:6801/2313723492":"2024-03-24T17:43:05.043910+0000","172.21.15.86:0/2416321716":"2024-03-24T17:43:05.043910+0000","172.21.15.86:0/433588188":"2024-03-24T17:43:05.043910+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T17:49:05.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.084+0000 7fe85cc00700 1 -- 172.21.15.86:0/1253825365 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fe83c0a1230 msgr2=0x7fe83c0a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:05.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.084+0000 7fe85cc00700 1 --2- 172.21.15.86:0/1253825365 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fe83c0a1230 0x7fe83c0a36f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe848009990 tx=0x7fe84801a040 comp rx=0 tx=0).stop 2024-03-23T17:49:05.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.084+0000 7fe85cc00700 1 -- 172.21.15.86:0/1253825365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe858102810 msgr2=0x7fe8580736e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:05.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.084+0000 7fe85cc00700 1 --2- 172.21.15.86:0/1253825365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe858102810 0x7fe8580736e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe85000ef30 tx=0x7fe85000c450 comp rx=0 tx=0).stop 2024-03-23T17:49:05.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.084+0000 7fe85cc00700 1 -- 172.21.15.86:0/1253825365 shutdown_connections 2024-03-23T17:49:05.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.084+0000 7fe85cc00700 1 --2- 172.21.15.86:0/1253825365 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fe83c0a1230 0x7fe83c0a36f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:05.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.084+0000 7fe85cc00700 1 --2- 172.21.15.86:0/1253825365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe858102810 0x7fe8580736e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:05.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.084+0000 7fe85cc00700 1 --2- 172.21.15.86:0/1253825365 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe858100c80 0x7fe858074b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:05.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.084+0000 7fe85cc00700 1 --2- 172.21.15.86:0/1253825365 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe858101e70 0x7fe8580731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:05.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.084+0000 7fe85cc00700 1 -- 172.21.15.86:0/1253825365 >> 172.21.15.86:0/1253825365 conn(0x7fe8580fc1f0 msgr2=0x7fe858105290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:05.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.084+0000 7fe85cc00700 1 -- 172.21.15.86:0/1253825365 shutdown_connections 2024-03-23T17:49:05.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.084+0000 7fe85cc00700 1 -- 172.21.15.86:0/1253825365 wait complete. 2024-03-23T17:49:05.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:04 smithi152 ceph-mon[47949]: from='client.24685 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:49:05.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:04 smithi152 ceph-mon[47949]: from='client.24691 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:49:05.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:04 smithi152 ceph-mon[47949]: mds.? [v2:172.21.15.92:6834/908900027,v1:172.21.15.92:6835/908900027] up:active 2024-03-23T17:49:05.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:04 smithi152 ceph-mon[47949]: fsmap cephfs:5 2 up:standby-replay 5 up:standby 5 up:active 2024-03-23T17:49:05.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:04 smithi152 ceph-mon[47949]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T17:49:05.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:04 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/748463261' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T17:49:05.104 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-23T17:49:05.105 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T17:49:05.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.295+0000 7ff5ce400700 1 -- 172.21.15.86:0/124931329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5c8108b40 msgr2=0x7ff5c810afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:05.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.295+0000 7ff5ce400700 1 --2- 172.21.15.86:0/124931329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5c8108b40 0x7ff5c810afd0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff5c000b0a0 tx=0x7ff5c000b3b0 comp rx=0 tx=0).stop 2024-03-23T17:49:05.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.295+0000 7ff5ce400700 1 -- 172.21.15.86:0/124931329 shutdown_connections 2024-03-23T17:49:05.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.295+0000 7ff5ce400700 1 --2- 172.21.15.86:0/124931329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5c8108b40 0x7ff5c810afd0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:05.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.295+0000 7ff5ce400700 1 --2- 172.21.15.86:0/124931329 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff5c8074df0 0x7ff5c8073240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:05.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.295+0000 7ff5ce400700 1 --2- 172.21.15.86:0/124931329 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5c8073810 0x7ff5c8073c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:05.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.295+0000 7ff5ce400700 1 -- 172.21.15.86:0/124931329 >> 172.21.15.86:0/124931329 conn(0x7ff5c80fc1f0 msgr2=0x7ff5c80fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:05.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.295+0000 7ff5ce400700 1 -- 172.21.15.86:0/124931329 shutdown_connections 2024-03-23T17:49:05.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.295+0000 7ff5ce400700 1 -- 172.21.15.86:0/124931329 wait complete. 2024-03-23T17:49:05.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.296+0000 7ff5ce400700 1 Processor -- start 2024-03-23T17:49:05.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.296+0000 7ff5ce400700 1 -- start start 2024-03-23T17:49:05.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.296+0000 7ff5ce400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5c8073810 0x7ff5c8198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:05.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.296+0000 7ff5ce400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5c8074df0 0x7ff5c8199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:05.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.296+0000 7ff5ce400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff5c8108b40 0x7ff5c81a0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:05.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.296+0000 7ff5ce400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c81a0a20 con 0x7ff5c8074df0 2024-03-23T17:49:05.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.296+0000 7ff5ce400700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c81a0b60 con 0x7ff5c8073810 2024-03-23T17:49:05.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.296+0000 7ff5ce400700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c81a0cf0 con 0x7ff5c8108b40 2024-03-23T17:49:05.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.297+0000 7ff5c6a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5c8074df0 0x7ff5c8199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:05.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.297+0000 7ff5c6a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5c8074df0 0x7ff5c8199430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34744/0 (socket says 172.21.15.86:34744) 2024-03-23T17:49:05.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.297+0000 7ff5c6a00700 1 -- 172.21.15.86:0/694221527 learned_addr learned my addr 172.21.15.86:0/694221527 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:05.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.297+0000 7ff5c7400700 1 --2- 172.21.15.86:0/694221527 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5c8073810 0x7ff5c8198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:05.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.297+0000 7ff5c7e00700 1 --2- 172.21.15.86:0/694221527 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff5c8108b40 0x7ff5c81a0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:05.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.297+0000 7ff5c6a00700 1 -- 172.21.15.86:0/694221527 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff5c8108b40 msgr2=0x7ff5c81a0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:05.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.297+0000 7ff5c6a00700 1 --2- 172.21.15.86:0/694221527 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff5c8108b40 0x7ff5c81a0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:05.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.297+0000 7ff5c6a00700 1 -- 172.21.15.86:0/694221527 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5c8073810 msgr2=0x7ff5c8198ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:05.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.297+0000 7ff5c6a00700 1 --2- 172.21.15.86:0/694221527 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5c8073810 0x7ff5c8198ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:05.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.297+0000 7ff5c6a00700 1 -- 172.21.15.86:0/694221527 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5c0009d00 con 0x7ff5c8074df0 2024-03-23T17:49:05.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.297+0000 7ff5c6a00700 1 --2- 172.21.15.86:0/694221527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5c8074df0 0x7ff5c8199430 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff5b400d980 tx=0x7ff5b400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:05.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.298+0000 7ff5bfe00700 1 -- 172.21.15.86:0/694221527 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b400f890 con 0x7ff5c8074df0 2024-03-23T17:49:05.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.298+0000 7ff5bfe00700 1 -- 172.21.15.86:0/694221527 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff5b4010460 con 0x7ff5c8074df0 2024-03-23T17:49:05.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.298+0000 7ff5ce400700 1 -- 172.21.15.86:0/694221527 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5c81a0fd0 con 0x7ff5c8074df0 2024-03-23T17:49:05.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.298+0000 7ff5bfe00700 1 -- 172.21.15.86:0/694221527 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b40195c0 con 0x7ff5c8074df0 2024-03-23T17:49:05.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.298+0000 7ff5ce400700 1 -- 172.21.15.86:0/694221527 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5c81a1450 con 0x7ff5c8074df0 2024-03-23T17:49:05.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.300+0000 7ff5ce400700 1 -- 172.21.15.86:0/694221527 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5c8107900 con 0x7ff5c8074df0 2024-03-23T17:49:05.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.301+0000 7ff5bfe00700 1 -- 172.21.15.86:0/694221527 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7ff5b40105d0 con 0x7ff5c8074df0 2024-03-23T17:49:05.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.302+0000 7ff5bfe00700 1 --2- 172.21.15.86:0/694221527 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7ff5ac0a11e0 0x7ff5ac0a36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:05.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.302+0000 7ff5bfe00700 1 -- 172.21.15.86:0/694221527 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff5b40cc6e0 con 0x7ff5c8074df0 2024-03-23T17:49:05.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.302+0000 7ff5c7400700 1 --2- 172.21.15.86:0/694221527 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7ff5ac0a11e0 0x7ff5ac0a36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:05.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.303+0000 7ff5c7400700 1 --2- 172.21.15.86:0/694221527 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7ff5ac0a11e0 0x7ff5ac0a36a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff5b8009be0 tx=0x7ff5b8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:05.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.306+0000 7ff5bfe00700 1 -- 172.21.15.86:0/694221527 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ff5b4095dc0 con 0x7ff5c8074df0 2024-03-23T17:49:05.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.509+0000 7ff5ce400700 1 -- 172.21.15.86:0/694221527 --> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff5c802cdf0 con 0x7ff5ac0a11e0 2024-03-23T17:49:05.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.521+0000 7ff5bfe00700 1 -- 172.21.15.86:0/694221527 <== mgr.14152 v2:172.21.15.86:6800/2734784244 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+329969 (secure 0 0 0) 0x7ff5c802cdf0 con 0x7ff5ac0a11e0 2024-03-23T17:49:05.528 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:49:05.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.526+0000 7ff5ce400700 1 -- 172.21.15.86:0/694221527 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7ff5ac0a11e0 msgr2=0x7ff5ac0a36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:05.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.526+0000 7ff5ce400700 1 --2- 172.21.15.86:0/694221527 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7ff5ac0a11e0 0x7ff5ac0a36a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff5b8009be0 tx=0x7ff5b8009260 comp rx=0 tx=0).stop 2024-03-23T17:49:05.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.526+0000 7ff5ce400700 1 -- 172.21.15.86:0/694221527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5c8074df0 msgr2=0x7ff5c8199430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:05.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.526+0000 7ff5ce400700 1 --2- 172.21.15.86:0/694221527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5c8074df0 0x7ff5c8199430 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff5b400d980 tx=0x7ff5b400dc90 comp rx=0 tx=0).stop 2024-03-23T17:49:05.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.526+0000 7ff5ce400700 1 -- 172.21.15.86:0/694221527 shutdown_connections 2024-03-23T17:49:05.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.526+0000 7ff5ce400700 1 --2- 172.21.15.86:0/694221527 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff5c8108b40 0x7ff5c81a0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:05.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.526+0000 7ff5ce400700 1 --2- 172.21.15.86:0/694221527 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5c8073810 0x7ff5c8198ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:05.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.526+0000 7ff5ce400700 1 --2- 172.21.15.86:0/694221527 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7ff5ac0a11e0 0x7ff5ac0a36a0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:05.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.526+0000 7ff5ce400700 1 --2- 172.21.15.86:0/694221527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5c8074df0 0x7ff5c8199430 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:05.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.526+0000 7ff5ce400700 1 -- 172.21.15.86:0/694221527 >> 172.21.15.86:0/694221527 conn(0x7ff5c80fc1f0 msgr2=0x7ff5c8104a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:05.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.527+0000 7ff5ce400700 1 -- 172.21.15.86:0/694221527 shutdown_connections 2024-03-23T17:49:05.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.527+0000 7ff5ce400700 1 -- 172.21.15.86:0/694221527 wait complete. 2024-03-23T17:49:05.533 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-03-23T17:49:05.546 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":238,"stamp":"2024-03-23T17:49:04.309563+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595454,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":147,"num_read_kb":128,"num_write":311,"num_write_kb":2981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":250,"ondisk_log_size":250,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":372988,"kb_used_data":11900,"kb_used_omap":18,"kb_used_meta":356845,"kb_avail":1124568836,"statfs":{"total":1151940427776,"available":1151558488064,"internally_reserved":0,"allocated":12185600,"data_stored":6708543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19130,"internal_metadata":365409606},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5086,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":11,"num_read_kb":9,"num_write":130,"num_write_kb":43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.104470"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.359640+0000","last_change":"2024-03-23T17:48:56.239724+0000","last_active":"2024-03-23T17:48:57.359640+0000","last_peered":"2024-03-23T17:48:57.359640+0000","last_clean":"2024-03-23T17:48:57.359640+0000","last_became_active":"2024-03-23T17:48:56.239451+0000","last_became_peered":"2024-03-23T17:48:56.239451+0000","last_unstale":"2024-03-23T17:48:57.359640+0000","last_undegraded":"2024-03-23T17:48:57.359640+0000","last_fullsized":"2024-03-23T17:48:57.359640+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:51:38.687863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.259420+0000","last_change":"2024-03-23T17:48:56.232415+0000","last_active":"2024-03-23T17:48:57.259420+0000","last_peered":"2024-03-23T17:48:57.259420+0000","last_clean":"2024-03-23T17:48:57.259420+0000","last_became_active":"2024-03-23T17:48:56.231777+0000","last_became_peered":"2024-03-23T17:48:56.231777+0000","last_unstale":"2024-03-23T17:48:57.259420+0000","last_undegraded":"2024-03-23T17:48:57.259420+0000","last_fullsized":"2024-03-23T17:48:57.259420+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:16:22.631654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.259288+0000","last_change":"2024-03-23T17:48:56.243413+0000","last_active":"2024-03-23T17:48:57.259288+0000","last_peered":"2024-03-23T17:48:57.259288+0000","last_clean":"2024-03-23T17:48:57.259288+0000","last_became_active":"2024-03-23T17:48:56.241281+0000","last_became_peered":"2024-03-23T17:48:56.241281+0000","last_unstale":"2024-03-23T17:48:57.259288+0000","last_undegraded":"2024-03-23T17:48:57.259288+0000","last_fullsized":"2024-03-23T17:48:57.259288+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:52:48.521689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.260410+0000","last_change":"2024-03-23T17:48:56.226010+0000","last_active":"2024-03-23T17:48:57.260410+0000","last_peered":"2024-03-23T17:48:57.260410+0000","last_clean":"2024-03-23T17:48:57.260410+0000","last_became_active":"2024-03-23T17:48:56.225562+0000","last_became_peered":"2024-03-23T17:48:56.225562+0000","last_unstale":"2024-03-23T17:48:57.260410+0000","last_undegraded":"2024-03-23T17:48:57.260410+0000","last_fullsized":"2024-03-23T17:48:57.260410+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:17:47.597335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.259300+0000","last_change":"2024-03-23T17:48:56.223249+0000","last_active":"2024-03-23T17:48:57.259300+0000","last_peered":"2024-03-23T17:48:57.259300+0000","last_clean":"2024-03-23T17:48:57.259300+0000","last_became_active":"2024-03-23T17:48:56.222902+0000","last_became_peered":"2024-03-23T17:48:56.222902+0000","last_unstale":"2024-03-23T17:48:57.259300+0000","last_undegraded":"2024-03-23T17:48:57.259300+0000","last_fullsized":"2024-03-23T17:48:57.259300+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:51:49.029628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.260827+0000","last_change":"2024-03-23T17:48:56.229229+0000","last_active":"2024-03-23T17:48:57.260827+0000","last_peered":"2024-03-23T17:48:57.260827+0000","last_clean":"2024-03-23T17:48:57.260827+0000","last_became_active":"2024-03-23T17:48:56.228725+0000","last_became_peered":"2024-03-23T17:48:56.228725+0000","last_unstale":"2024-03-23T17:48:57.260827+0000","last_undegraded":"2024-03-23T17:48:57.260827+0000","last_fullsized":"2024-03-23T17:48:57.260827+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:53:44.118762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-23T17:48:56.370934+0000","last_change":"2024-03-23T17:48:56.229212+0000","last_active":"2024-03-23T17:48:56.370934+0000","last_peered":"2024-03-23T17:48:56.370934+0000","last_clean":"2024-03-23T17:48:56.370934+0000","last_became_active":"2024-03-23T17:48:56.228888+0000","last_became_peered":"2024-03-23T17:48:56.228888+0000","last_unstale":"2024-03-23T17:48:56.370934+0000","last_undegraded":"2024-03-23T17:48:56.370934+0000","last_fullsized":"2024-03-23T17:48:56.370934+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:47:21.509554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-23T17:48:56.984356+0000","last_change":"2024-03-23T17:48:56.222925+0000","last_active":"2024-03-23T17:48:56.984356+0000","last_peered":"2024-03-23T17:48:56.984356+0000","last_clean":"2024-03-23T17:48:56.984356+0000","last_became_active":"2024-03-23T17:48:56.221504+0000","last_became_peered":"2024-03-23T17:48:56.221504+0000","last_unstale":"2024-03-23T17:48:56.984356+0000","last_undegraded":"2024-03-23T17:48:56.984356+0000","last_fullsized":"2024-03-23T17:48:56.984356+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:14:34.327705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-23T17:48:56.562349+0000","last_change":"2024-03-23T17:48:56.230530+0000","last_active":"2024-03-23T17:48:56.562349+0000","last_peered":"2024-03-23T17:48:56.562349+0000","last_clean":"2024-03-23T17:48:56.562349+0000","last_became_active":"2024-03-23T17:48:56.230224+0000","last_became_peered":"2024-03-23T17:48:56.230224+0000","last_unstale":"2024-03-23T17:48:56.562349+0000","last_undegraded":"2024-03-23T17:48:56.562349+0000","last_fullsized":"2024-03-23T17:48:56.562349+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:10:28.031956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.259603+0000","last_change":"2024-03-23T17:48:56.229673+0000","last_active":"2024-03-23T17:48:57.259603+0000","last_peered":"2024-03-23T17:48:57.259603+0000","last_clean":"2024-03-23T17:48:57.259603+0000","last_became_active":"2024-03-23T17:48:56.229000+0000","last_became_peered":"2024-03-23T17:48:56.229000+0000","last_unstale":"2024-03-23T17:48:57.259603+0000","last_undegraded":"2024-03-23T17:48:57.259603+0000","last_fullsized":"2024-03-23T17:48:57.259603+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:23:28.057256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-23T17:48:56.371001+0000","last_change":"2024-03-23T17:48:56.224508+0000","last_active":"2024-03-23T17:48:56.371001+0000","last_peered":"2024-03-23T17:48:56.371001+0000","last_clean":"2024-03-23T17:48:56.371001+0000","last_became_active":"2024-03-23T17:48:56.224172+0000","last_became_peered":"2024-03-23T17:48:56.224172+0000","last_unstale":"2024-03-23T17:48:56.371001+0000","last_undegraded":"2024-03-23T17:48:56.371001+0000","last_fullsized":"2024-03-23T17:48:56.371001+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:03:57.063472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.260352+0000","last_change":"2024-03-23T17:48:56.234978+0000","last_active":"2024-03-23T17:48:57.260352+0000","last_peered":"2024-03-23T17:48:57.260352+0000","last_clean":"2024-03-23T17:48:57.260352+0000","last_became_active":"2024-03-23T17:48:56.234641+0000","last_became_peered":"2024-03-23T17:48:56.234641+0000","last_unstale":"2024-03-23T17:48:57.260352+0000","last_undegraded":"2024-03-23T17:48:57.260352+0000","last_fullsized":"2024-03-23T17:48:57.260352+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:34:30.745991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.260030+0000","last_change":"2024-03-23T17:48:56.218534+0000","last_active":"2024-03-23T17:48:57.260030+0000","last_peered":"2024-03-23T17:48:57.260030+0000","last_clean":"2024-03-23T17:48:57.260030+0000","last_became_active":"2024-03-23T17:48:56.217424+0000","last_became_peered":"2024-03-23T17:48:56.217424+0000","last_unstale":"2024-03-23T17:48:57.260030+0000","last_undegraded":"2024-03-23T17:48:57.260030+0000","last_fullsized":"2024-03-23T17:48:57.260030+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:32:05.618057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.259005+0000","last_change":"2024-03-23T17:48:56.235525+0000","last_active":"2024-03-23T17:48:57.259005+0000","last_peered":"2024-03-23T17:48:57.259005+0000","last_clean":"2024-03-23T17:48:57.259005+0000","last_became_active":"2024-03-23T17:48:56.234591+0000","last_became_peered":"2024-03-23T17:48:56.234591+0000","last_unstale":"2024-03-23T17:48:57.259005+0000","last_undegraded":"2024-03-23T17:48:57.259005+0000","last_fullsized":"2024-03-23T17:48:57.259005+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:19:38.381878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.258904+0000","last_change":"2024-03-23T17:48:56.243137+0000","last_active":"2024-03-23T17:48:57.258904+0000","last_peered":"2024-03-23T17:48:57.258904+0000","last_clean":"2024-03-23T17:48:57.258904+0000","last_became_active":"2024-03-23T17:48:56.242516+0000","last_became_peered":"2024-03-23T17:48:56.242516+0000","last_unstale":"2024-03-23T17:48:57.258904+0000","last_undegraded":"2024-03-23T17:48:57.258904+0000","last_fullsized":"2024-03-23T17:48:57.258904+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:54:59.849467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-23T17:48:56.562254+0000","last_change":"2024-03-23T17:48:56.236026+0000","last_active":"2024-03-23T17:48:56.562254+0000","last_peered":"2024-03-23T17:48:56.562254+0000","last_clean":"2024-03-23T17:48:56.562254+0000","last_became_active":"2024-03-23T17:48:56.235645+0000","last_became_peered":"2024-03-23T17:48:56.235645+0000","last_unstale":"2024-03-23T17:48:56.562254+0000","last_undegraded":"2024-03-23T17:48:56.562254+0000","last_fullsized":"2024-03-23T17:48:56.562254+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:47:01.076737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.359659+0000","last_change":"2024-03-23T17:48:56.239689+0000","last_active":"2024-03-23T17:48:57.359659+0000","last_peered":"2024-03-23T17:48:57.359659+0000","last_clean":"2024-03-23T17:48:57.359659+0000","last_became_active":"2024-03-23T17:48:56.239331+0000","last_became_peered":"2024-03-23T17:48:56.239331+0000","last_unstale":"2024-03-23T17:48:57.359659+0000","last_undegraded":"2024-03-23T17:48:57.359659+0000","last_fullsized":"2024-03-23T17:48:57.359659+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:21:15.287627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.014188+0000","last_change":"2024-03-23T17:48:56.229934+0000","last_active":"2024-03-23T17:48:57.014188+0000","last_peered":"2024-03-23T17:48:57.014188+0000","last_clean":"2024-03-23T17:48:57.014188+0000","last_became_active":"2024-03-23T17:48:56.229244+0000","last_became_peered":"2024-03-23T17:48:56.229244+0000","last_unstale":"2024-03-23T17:48:57.014188+0000","last_undegraded":"2024-03-23T17:48:57.014188+0000","last_fullsized":"2024-03-23T17:48:57.014188+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:02:40.254799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.259964+0000","last_change":"2024-03-23T17:48:56.233786+0000","last_active":"2024-03-23T17:48:57.259964+0000","last_peered":"2024-03-23T17:48:57.259964+0000","last_clean":"2024-03-23T17:48:57.259964+0000","last_became_active":"2024-03-23T17:48:56.233137+0000","last_became_peered":"2024-03-23T17:48:56.233137+0000","last_unstale":"2024-03-23T17:48:57.259964+0000","last_undegraded":"2024-03-23T17:48:57.259964+0000","last_fullsized":"2024-03-23T17:48:57.259964+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:33:35.202788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'11","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:49:01.684392+0000","last_change":"2024-03-23T17:48:55.712184+0000","last_active":"2024-03-23T17:49:01.684392+0000","last_peered":"2024-03-23T17:49:01.684392+0000","last_clean":"2024-03-23T17:49:01.684392+0000","last_became_active":"2024-03-23T17:48:55.711860+0000","last_became_peered":"2024-03-23T17:48:55.711860+0000","last_unstale":"2024-03-23T17:49:01.684392+0000","last_undegraded":"2024-03-23T17:49:01.684392+0000","last_fullsized":"2024-03-23T17:49:01.684392+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:59:01.133652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-23T17:48:56.984412+0000","last_change":"2024-03-23T17:48:56.228317+0000","last_active":"2024-03-23T17:48:56.984412+0000","last_peered":"2024-03-23T17:48:56.984412+0000","last_clean":"2024-03-23T17:48:56.984412+0000","last_became_active":"2024-03-23T17:48:56.227197+0000","last_became_peered":"2024-03-23T17:48:56.227197+0000","last_unstale":"2024-03-23T17:48:56.984412+0000","last_undegraded":"2024-03-23T17:48:56.984412+0000","last_fullsized":"2024-03-23T17:48:56.984412+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:33:55.677013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.260645+0000","last_change":"2024-03-23T17:48:55.725351+0000","last_active":"2024-03-23T17:48:57.260645+0000","last_peered":"2024-03-23T17:48:57.260645+0000","last_clean":"2024-03-23T17:48:57.260645+0000","last_became_active":"2024-03-23T17:48:55.724566+0000","last_became_peered":"2024-03-23T17:48:55.724566+0000","last_unstale":"2024-03-23T17:48:57.260645+0000","last_undegraded":"2024-03-23T17:48:57.260645+0000","last_fullsized":"2024-03-23T17:48:57.260645+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:58:21.088166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.260283+0000","last_change":"2024-03-23T17:48:56.243526+0000","last_active":"2024-03-23T17:48:57.260283+0000","last_peered":"2024-03-23T17:48:57.260283+0000","last_clean":"2024-03-23T17:48:57.260283+0000","last_became_active":"2024-03-23T17:48:56.242468+0000","last_became_peered":"2024-03-23T17:48:56.242468+0000","last_unstale":"2024-03-23T17:48:57.260283+0000","last_undegraded":"2024-03-23T17:48:57.260283+0000","last_fullsized":"2024-03-23T17:48:57.260283+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:16:50.392770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'11","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:49:02.709260+0000","last_change":"2024-03-23T17:48:55.720237+0000","last_active":"2024-03-23T17:49:02.709260+0000","last_peered":"2024-03-23T17:49:02.709260+0000","last_clean":"2024-03-23T17:49:02.709260+0000","last_became_active":"2024-03-23T17:48:55.719843+0000","last_became_peered":"2024-03-23T17:48:55.719843+0000","last_unstale":"2024-03-23T17:49:02.709260+0000","last_undegraded":"2024-03-23T17:49:02.709260+0000","last_fullsized":"2024-03-23T17:49:02.709260+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:07:26.973511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-23T17:48:56.562302+0000","last_change":"2024-03-23T17:48:56.235642+0000","last_active":"2024-03-23T17:48:56.562302+0000","last_peered":"2024-03-23T17:48:56.562302+0000","last_clean":"2024-03-23T17:48:56.562302+0000","last_became_active":"2024-03-23T17:48:56.235326+0000","last_became_peered":"2024-03-23T17:48:56.235326+0000","last_unstale":"2024-03-23T17:48:56.562302+0000","last_undegraded":"2024-03-23T17:48:56.562302+0000","last_fullsized":"2024-03-23T17:48:56.562302+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:21:59.934620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:49:02.710352+0000","last_change":"2024-03-23T17:48:55.718579+0000","last_active":"2024-03-23T17:49:02.710352+0000","last_peered":"2024-03-23T17:49:02.710352+0000","last_clean":"2024-03-23T17:49:02.710352+0000","last_became_active":"2024-03-23T17:48:55.718220+0000","last_became_peered":"2024-03-23T17:48:55.718220+0000","last_unstale":"2024-03-23T17:49:02.710352+0000","last_undegraded":"2024-03-23T17:49:02.710352+0000","last_fullsized":"2024-03-23T17:49:02.710352+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:37:56.417367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.214206+0000","last_change":"2024-03-23T17:48:56.224842+0000","last_active":"2024-03-23T17:48:57.214206+0000","last_peered":"2024-03-23T17:48:57.214206+0000","last_clean":"2024-03-23T17:48:57.214206+0000","last_became_active":"2024-03-23T17:48:56.224427+0000","last_became_peered":"2024-03-23T17:48:56.224427+0000","last_unstale":"2024-03-23T17:48:57.214206+0000","last_undegraded":"2024-03-23T17:48:57.214206+0000","last_fullsized":"2024-03-23T17:48:57.214206+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:16:22.684369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"80'5","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:49:00.656886+0000","last_change":"2024-03-23T17:48:55.713996+0000","last_active":"2024-03-23T17:49:00.656886+0000","last_peered":"2024-03-23T17:49:00.656886+0000","last_clean":"2024-03-23T17:49:00.656886+0000","last_became_active":"2024-03-23T17:48:55.713647+0000","last_became_peered":"2024-03-23T17:48:55.713647+0000","last_unstale":"2024-03-23T17:49:00.656886+0000","last_undegraded":"2024-03-23T17:49:00.656886+0000","last_fullsized":"2024-03-23T17:49:00.656886+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:12:22.251533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.259235+0000","last_change":"2024-03-23T17:48:56.219281+0000","last_active":"2024-03-23T17:48:57.259235+0000","last_peered":"2024-03-23T17:48:57.259235+0000","last_clean":"2024-03-23T17:48:57.259235+0000","last_became_active":"2024-03-23T17:48:56.218924+0000","last_became_peered":"2024-03-23T17:48:56.218924+0000","last_unstale":"2024-03-23T17:48:57.259235+0000","last_undegraded":"2024-03-23T17:48:57.259235+0000","last_fullsized":"2024-03-23T17:48:57.259235+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:49:04.793088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"79'1","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.260763+0000","last_change":"2024-03-23T17:48:55.718131+0000","last_active":"2024-03-23T17:48:57.260763+0000","last_peered":"2024-03-23T17:48:57.260763+0000","last_clean":"2024-03-23T17:48:57.260763+0000","last_became_active":"2024-03-23T17:48:55.717760+0000","last_became_peered":"2024-03-23T17:48:55.717760+0000","last_unstale":"2024-03-23T17:48:57.260763+0000","last_undegraded":"2024-03-23T17:48:57.260763+0000","last_fullsized":"2024-03-23T17:48:57.260763+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:16:54.355844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.214056+0000","last_change":"2024-03-23T17:48:56.225965+0000","last_active":"2024-03-23T17:48:57.214056+0000","last_peered":"2024-03-23T17:48:57.214056+0000","last_clean":"2024-03-23T17:48:57.214056+0000","last_became_active":"2024-03-23T17:48:56.225101+0000","last_became_peered":"2024-03-23T17:48:56.225101+0000","last_unstale":"2024-03-23T17:48:57.214056+0000","last_undegraded":"2024-03-23T17:48:57.214056+0000","last_fullsized":"2024-03-23T17:48:57.214056+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:52:14.302306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.259871+0000","last_change":"2024-03-23T17:48:55.722289+0000","last_active":"2024-03-23T17:48:57.259871+0000","last_peered":"2024-03-23T17:48:57.259871+0000","last_clean":"2024-03-23T17:48:57.259871+0000","last_became_active":"2024-03-23T17:48:55.721927+0000","last_became_peered":"2024-03-23T17:48:55.721927+0000","last_unstale":"2024-03-23T17:48:57.259871+0000","last_undegraded":"2024-03-23T17:48:57.259871+0000","last_fullsized":"2024-03-23T17:48:57.259871+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:20:08.354897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-23T17:48:56.984339+0000","last_change":"2024-03-23T17:48:56.232231+0000","last_active":"2024-03-23T17:48:56.984339+0000","last_peered":"2024-03-23T17:48:56.984339+0000","last_clean":"2024-03-23T17:48:56.984339+0000","last_became_active":"2024-03-23T17:48:56.227335+0000","last_became_peered":"2024-03-23T17:48:56.227335+0000","last_unstale":"2024-03-23T17:48:56.984339+0000","last_undegraded":"2024-03-23T17:48:56.984339+0000","last_fullsized":"2024-03-23T17:48:56.984339+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:49:01.460775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'2","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:59.672427+0000","last_change":"2024-03-23T17:48:55.717615+0000","last_active":"2024-03-23T17:48:59.672427+0000","last_peered":"2024-03-23T17:48:59.672427+0000","last_clean":"2024-03-23T17:48:59.672427+0000","last_became_active":"2024-03-23T17:48:55.717266+0000","last_became_peered":"2024-03-23T17:48:55.717266+0000","last_unstale":"2024-03-23T17:48:59.672427+0000","last_undegraded":"2024-03-23T17:48:59.672427+0000","last_fullsized":"2024-03-23T17:48:59.672427+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:40:07.988182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.014083+0000","last_change":"2024-03-23T17:48:56.229787+0000","last_active":"2024-03-23T17:48:57.014083+0000","last_peered":"2024-03-23T17:48:57.014083+0000","last_clean":"2024-03-23T17:48:57.014083+0000","last_became_active":"2024-03-23T17:48:56.229471+0000","last_became_peered":"2024-03-23T17:48:56.229471+0000","last_unstale":"2024-03-23T17:48:57.014083+0000","last_undegraded":"2024-03-23T17:48:57.014083+0000","last_fullsized":"2024-03-23T17:48:57.014083+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:44:09.358809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.259940+0000","last_change":"2024-03-23T17:48:55.717130+0000","last_active":"2024-03-23T17:48:57.259940+0000","last_peered":"2024-03-23T17:48:57.259940+0000","last_clean":"2024-03-23T17:48:57.259940+0000","last_became_active":"2024-03-23T17:48:55.716733+0000","last_became_peered":"2024-03-23T17:48:55.716733+0000","last_unstale":"2024-03-23T17:48:57.259940+0000","last_undegraded":"2024-03-23T17:48:57.259940+0000","last_fullsized":"2024-03-23T17:48:57.259940+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:22:43.229694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-23T17:48:56.562338+0000","last_change":"2024-03-23T17:48:56.235565+0000","last_active":"2024-03-23T17:48:56.562338+0000","last_peered":"2024-03-23T17:48:56.562338+0000","last_clean":"2024-03-23T17:48:56.562338+0000","last_became_active":"2024-03-23T17:48:56.235267+0000","last_became_peered":"2024-03-23T17:48:56.235267+0000","last_unstale":"2024-03-23T17:48:56.562338+0000","last_undegraded":"2024-03-23T17:48:56.562338+0000","last_fullsized":"2024-03-23T17:48:56.562338+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:38:45.906545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'2","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:59.672296+0000","last_change":"2024-03-23T17:48:55.719096+0000","last_active":"2024-03-23T17:48:59.672296+0000","last_peered":"2024-03-23T17:48:59.672296+0000","last_clean":"2024-03-23T17:48:59.672296+0000","last_became_active":"2024-03-23T17:48:55.718721+0000","last_became_peered":"2024-03-23T17:48:55.718721+0000","last_unstale":"2024-03-23T17:48:59.672296+0000","last_undegraded":"2024-03-23T17:48:59.672296+0000","last_fullsized":"2024-03-23T17:48:59.672296+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:43:05.532263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.260207+0000","last_change":"2024-03-23T17:48:56.235549+0000","last_active":"2024-03-23T17:48:57.260207+0000","last_peered":"2024-03-23T17:48:57.260207+0000","last_clean":"2024-03-23T17:48:57.260207+0000","last_became_active":"2024-03-23T17:48:56.234572+0000","last_became_peered":"2024-03-23T17:48:56.234572+0000","last_unstale":"2024-03-23T17:48:57.260207+0000","last_undegraded":"2024-03-23T17:48:57.260207+0000","last_fullsized":"2024-03-23T17:48:57.260207+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:19:58.639181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:49:00.656075+0000","last_change":"2024-03-23T17:48:55.724688+0000","last_active":"2024-03-23T17:49:00.656075+0000","last_peered":"2024-03-23T17:49:00.656075+0000","last_clean":"2024-03-23T17:49:00.656075+0000","last_became_active":"2024-03-23T17:48:55.724385+0000","last_became_peered":"2024-03-23T17:48:55.724385+0000","last_unstale":"2024-03-23T17:49:00.656075+0000","last_undegraded":"2024-03-23T17:49:00.656075+0000","last_fullsized":"2024-03-23T17:49:00.656075+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:17:54.338059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-23T17:48:56.562325+0000","last_change":"2024-03-23T17:48:56.242983+0000","last_active":"2024-03-23T17:48:56.562325+0000","last_peered":"2024-03-23T17:48:56.562325+0000","last_clean":"2024-03-23T17:48:56.562325+0000","last_became_active":"2024-03-23T17:48:56.242596+0000","last_became_peered":"2024-03-23T17:48:56.242596+0000","last_unstale":"2024-03-23T17:48:56.562325+0000","last_undegraded":"2024-03-23T17:48:56.562325+0000","last_fullsized":"2024-03-23T17:48:56.562325+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:55:51.971834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"79'6","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.259839+0000","last_change":"2024-03-23T17:48:55.717631+0000","last_active":"2024-03-23T17:48:57.259839+0000","last_peered":"2024-03-23T17:48:57.259839+0000","last_clean":"2024-03-23T17:48:57.259839+0000","last_became_active":"2024-03-23T17:48:55.717082+0000","last_became_peered":"2024-03-23T17:48:55.717082+0000","last_unstale":"2024-03-23T17:48:57.259839+0000","last_undegraded":"2024-03-23T17:48:57.259839+0000","last_fullsized":"2024-03-23T17:48:57.259839+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:36:25.323269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.259542+0000","last_change":"2024-03-23T17:48:56.226174+0000","last_active":"2024-03-23T17:48:57.259542+0000","last_peered":"2024-03-23T17:48:57.259542+0000","last_clean":"2024-03-23T17:48:57.259542+0000","last_became_active":"2024-03-23T17:48:56.225834+0000","last_became_peered":"2024-03-23T17:48:56.225834+0000","last_unstale":"2024-03-23T17:48:57.259542+0000","last_undegraded":"2024-03-23T17:48:57.259542+0000","last_fullsized":"2024-03-23T17:48:57.259542+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:50:16.793071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'4","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:59.674040+0000","last_change":"2024-03-23T17:48:55.723059+0000","last_active":"2024-03-23T17:48:59.674040+0000","last_peered":"2024-03-23T17:48:59.674040+0000","last_clean":"2024-03-23T17:48:59.674040+0000","last_became_active":"2024-03-23T17:48:55.722402+0000","last_became_peered":"2024-03-23T17:48:55.722402+0000","last_unstale":"2024-03-23T17:48:59.674040+0000","last_undegraded":"2024-03-23T17:48:59.674040+0000","last_fullsized":"2024-03-23T17:48:59.674040+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:40:28.322650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.259860+0000","last_change":"2024-03-23T17:48:56.228459+0000","last_active":"2024-03-23T17:48:57.259860+0000","last_peered":"2024-03-23T17:48:57.259860+0000","last_clean":"2024-03-23T17:48:57.259860+0000","last_became_active":"2024-03-23T17:48:56.228179+0000","last_became_peered":"2024-03-23T17:48:56.228179+0000","last_unstale":"2024-03-23T17:48:57.259860+0000","last_undegraded":"2024-03-23T17:48:57.259860+0000","last_fullsized":"2024-03-23T17:48:57.259860+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:28:41.613432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'4","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:59.672767+0000","last_change":"2024-03-23T17:48:55.721533+0000","last_active":"2024-03-23T17:48:59.672767+0000","last_peered":"2024-03-23T17:48:59.672767+0000","last_clean":"2024-03-23T17:48:59.672767+0000","last_became_active":"2024-03-23T17:48:55.721100+0000","last_became_peered":"2024-03-23T17:48:55.721100+0000","last_unstale":"2024-03-23T17:48:59.672767+0000","last_undegraded":"2024-03-23T17:48:59.672767+0000","last_fullsized":"2024-03-23T17:48:59.672767+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:56:58.112649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-23T17:48:56.370970+0000","last_change":"2024-03-23T17:48:56.226508+0000","last_active":"2024-03-23T17:48:56.370970+0000","last_peered":"2024-03-23T17:48:56.370970+0000","last_clean":"2024-03-23T17:48:56.370970+0000","last_became_active":"2024-03-23T17:48:56.226082+0000","last_became_peered":"2024-03-23T17:48:56.226082+0000","last_unstale":"2024-03-23T17:48:56.370970+0000","last_undegraded":"2024-03-23T17:48:56.370970+0000","last_fullsized":"2024-03-23T17:48:56.370970+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:38:38.133029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-23T17:48:56.220140+0000","last_change":"2024-03-23T17:48:55.715205+0000","last_active":"2024-03-23T17:48:56.220140+0000","last_peered":"2024-03-23T17:48:56.220140+0000","last_clean":"2024-03-23T17:48:56.220140+0000","last_became_active":"2024-03-23T17:48:55.714810+0000","last_became_peered":"2024-03-23T17:48:55.714810+0000","last_unstale":"2024-03-23T17:48:56.220140+0000","last_undegraded":"2024-03-23T17:48:56.220140+0000","last_fullsized":"2024-03-23T17:48:56.220140+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:40:08.768998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-23T17:48:56.370884+0000","last_change":"2024-03-23T17:48:56.241434+0000","last_active":"2024-03-23T17:48:56.370884+0000","last_peered":"2024-03-23T17:48:56.370884+0000","last_clean":"2024-03-23T17:48:56.370884+0000","last_became_active":"2024-03-23T17:48:56.240532+0000","last_became_peered":"2024-03-23T17:48:56.240532+0000","last_unstale":"2024-03-23T17:48:56.370884+0000","last_undegraded":"2024-03-23T17:48:56.370884+0000","last_fullsized":"2024-03-23T17:48:56.370884+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:56:15.970864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:49:01.685147+0000","last_change":"2024-03-23T17:48:55.721653+0000","last_active":"2024-03-23T17:49:01.685147+0000","last_peered":"2024-03-23T17:49:01.685147+0000","last_clean":"2024-03-23T17:49:01.685147+0000","last_became_active":"2024-03-23T17:48:55.721317+0000","last_became_peered":"2024-03-23T17:48:55.721317+0000","last_unstale":"2024-03-23T17:49:01.685147+0000","last_undegraded":"2024-03-23T17:49:01.685147+0000","last_fullsized":"2024-03-23T17:49:01.685147+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:31:40.085230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.213925+0000","last_change":"2024-03-23T17:48:56.225624+0000","last_active":"2024-03-23T17:48:57.213925+0000","last_peered":"2024-03-23T17:48:57.213925+0000","last_clean":"2024-03-23T17:48:57.213925+0000","last_became_active":"2024-03-23T17:48:56.223798+0000","last_became_peered":"2024-03-23T17:48:56.223798+0000","last_unstale":"2024-03-23T17:48:57.213925+0000","last_undegraded":"2024-03-23T17:48:57.213925+0000","last_fullsized":"2024-03-23T17:48:57.213925+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:00:20.547553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"79'1","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:58.639902+0000","last_change":"2024-03-23T17:48:55.717792+0000","last_active":"2024-03-23T17:48:58.639902+0000","last_peered":"2024-03-23T17:48:58.639902+0000","last_clean":"2024-03-23T17:48:58.639902+0000","last_became_active":"2024-03-23T17:48:55.717479+0000","last_became_peered":"2024-03-23T17:48:55.717479+0000","last_unstale":"2024-03-23T17:48:58.639902+0000","last_undegraded":"2024-03-23T17:48:58.639902+0000","last_fullsized":"2024-03-23T17:48:58.639902+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:49:43.483785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.359512+0000","last_change":"2024-03-23T17:48:56.219135+0000","last_active":"2024-03-23T17:48:57.359512+0000","last_peered":"2024-03-23T17:48:57.359512+0000","last_clean":"2024-03-23T17:48:57.359512+0000","last_became_active":"2024-03-23T17:48:56.218824+0000","last_became_peered":"2024-03-23T17:48:56.218824+0000","last_unstale":"2024-03-23T17:48:57.359512+0000","last_undegraded":"2024-03-23T17:48:57.359512+0000","last_fullsized":"2024-03-23T17:48:57.359512+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:20:35.190781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.215711+0000","last_change":"2024-03-23T17:48:55.717530+0000","last_active":"2024-03-23T17:48:57.215711+0000","last_peered":"2024-03-23T17:48:57.215711+0000","last_clean":"2024-03-23T17:48:57.215711+0000","last_became_active":"2024-03-23T17:48:55.717213+0000","last_became_peered":"2024-03-23T17:48:55.717213+0000","last_unstale":"2024-03-23T17:48:57.215711+0000","last_undegraded":"2024-03-23T17:48:57.215711+0000","last_fullsized":"2024-03-23T17:48:57.215711+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:40:09.803021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.260518+0000","last_change":"2024-03-23T17:48:56.222237+0000","last_active":"2024-03-23T17:48:57.260518+0000","last_peered":"2024-03-23T17:48:57.260518+0000","last_clean":"2024-03-23T17:48:57.260518+0000","last_became_active":"2024-03-23T17:48:56.221916+0000","last_became_peered":"2024-03-23T17:48:56.221916+0000","last_unstale":"2024-03-23T17:48:57.260518+0000","last_undegraded":"2024-03-23T17:48:57.260518+0000","last_fullsized":"2024-03-23T17:48:57.260518+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:35:59.213482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.259186+0000","last_change":"2024-03-23T17:48:55.718143+0000","last_active":"2024-03-23T17:48:57.259186+0000","last_peered":"2024-03-23T17:48:57.259186+0000","last_clean":"2024-03-23T17:48:57.259186+0000","last_became_active":"2024-03-23T17:48:55.717834+0000","last_became_peered":"2024-03-23T17:48:55.717834+0000","last_unstale":"2024-03-23T17:48:57.259186+0000","last_undegraded":"2024-03-23T17:48:57.259186+0000","last_fullsized":"2024-03-23T17:48:57.259186+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:29:08.879636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.259713+0000","last_change":"2024-03-23T17:48:56.237136+0000","last_active":"2024-03-23T17:48:57.259713+0000","last_peered":"2024-03-23T17:48:57.259713+0000","last_clean":"2024-03-23T17:48:57.259713+0000","last_became_active":"2024-03-23T17:48:56.236795+0000","last_became_peered":"2024-03-23T17:48:56.236795+0000","last_unstale":"2024-03-23T17:48:57.259713+0000","last_undegraded":"2024-03-23T17:48:57.259713+0000","last_fullsized":"2024-03-23T17:48:57.259713+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:15:31.356382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.214259+0000","last_change":"2024-03-23T17:48:56.224772+0000","last_active":"2024-03-23T17:48:57.214259+0000","last_peered":"2024-03-23T17:48:57.214259+0000","last_clean":"2024-03-23T17:48:57.214259+0000","last_became_active":"2024-03-23T17:48:56.223685+0000","last_became_peered":"2024-03-23T17:48:56.223685+0000","last_unstale":"2024-03-23T17:48:57.214259+0000","last_undegraded":"2024-03-23T17:48:57.214259+0000","last_fullsized":"2024-03-23T17:48:57.214259+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:07:53.765749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.259768+0000","last_change":"2024-03-23T17:48:55.722968+0000","last_active":"2024-03-23T17:48:57.259768+0000","last_peered":"2024-03-23T17:48:57.259768+0000","last_clean":"2024-03-23T17:48:57.259768+0000","last_became_active":"2024-03-23T17:48:55.722578+0000","last_became_peered":"2024-03-23T17:48:55.722578+0000","last_unstale":"2024-03-23T17:48:57.259768+0000","last_undegraded":"2024-03-23T17:48:57.259768+0000","last_fullsized":"2024-03-23T17:48:57.259768+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:45:42.344283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-23T17:48:56.984393+0000","last_change":"2024-03-23T17:48:56.228000+0000","last_active":"2024-03-23T17:48:56.984393+0000","last_peered":"2024-03-23T17:48:56.984393+0000","last_clean":"2024-03-23T17:48:56.984393+0000","last_became_active":"2024-03-23T17:48:56.225645+0000","last_became_peered":"2024-03-23T17:48:56.225645+0000","last_unstale":"2024-03-23T17:48:56.984393+0000","last_undegraded":"2024-03-23T17:48:56.984393+0000","last_fullsized":"2024-03-23T17:48:56.984393+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:44:25.197761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'2","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:59.672875+0000","last_change":"2024-03-23T17:48:55.718307+0000","last_active":"2024-03-23T17:48:59.672875+0000","last_peered":"2024-03-23T17:48:59.672875+0000","last_clean":"2024-03-23T17:48:59.672875+0000","last_became_active":"2024-03-23T17:48:55.717839+0000","last_became_peered":"2024-03-23T17:48:55.717839+0000","last_unstale":"2024-03-23T17:48:59.672875+0000","last_undegraded":"2024-03-23T17:48:59.672875+0000","last_fullsized":"2024-03-23T17:48:59.672875+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:11:56.053940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":332,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-23T17:48:56.218676+0000","last_change":"2024-03-23T17:48:01.618019+0000","last_active":"2024-03-23T17:48:56.218676+0000","last_peered":"2024-03-23T17:48:56.218676+0000","last_clean":"2024-03-23T17:48:56.218676+0000","last_became_active":"2024-03-23T17:48:01.617553+0000","last_became_peered":"2024-03-23T17:48:01.617553+0000","last_unstale":"2024-03-23T17:48:56.218676+0000","last_undegraded":"2024-03-23T17:48:56.218676+0000","last_fullsized":"2024-03-23T17:48:56.218676+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:45:26.849029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:45:26.849029+0000","last_clean_scrub_stamp":"2024-03-23T17:45:26.849029+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:05:44.823364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.259679+0000","last_change":"2024-03-23T17:48:56.224285+0000","last_active":"2024-03-23T17:48:57.259679+0000","last_peered":"2024-03-23T17:48:57.259679+0000","last_clean":"2024-03-23T17:48:57.259679+0000","last_became_active":"2024-03-23T17:48:56.220943+0000","last_became_peered":"2024-03-23T17:48:56.220943+0000","last_unstale":"2024-03-23T17:48:57.259679+0000","last_undegraded":"2024-03-23T17:48:57.259679+0000","last_fullsized":"2024-03-23T17:48:57.259679+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:36:05.458243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:49:00.661819+0000","last_change":"2024-03-23T17:48:55.715877+0000","last_active":"2024-03-23T17:49:00.661819+0000","last_peered":"2024-03-23T17:49:00.661819+0000","last_clean":"2024-03-23T17:49:00.661819+0000","last_became_active":"2024-03-23T17:48:55.715550+0000","last_became_peered":"2024-03-23T17:48:55.715550+0000","last_unstale":"2024-03-23T17:49:00.661819+0000","last_undegraded":"2024-03-23T17:49:00.661819+0000","last_fullsized":"2024-03-23T17:49:00.661819+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:31:57.668495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:49:01.685625+0000","last_change":"2024-03-23T17:48:55.724618+0000","last_active":"2024-03-23T17:49:01.685625+0000","last_peered":"2024-03-23T17:49:01.685625+0000","last_clean":"2024-03-23T17:49:01.685625+0000","last_became_active":"2024-03-23T17:48:55.724278+0000","last_became_peered":"2024-03-23T17:48:55.724278+0000","last_unstale":"2024-03-23T17:49:01.685625+0000","last_undegraded":"2024-03-23T17:49:01.685625+0000","last_fullsized":"2024-03-23T17:49:01.685625+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:27:05.065115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.260132+0000","last_change":"2024-03-23T17:48:56.216141+0000","last_active":"2024-03-23T17:48:57.260132+0000","last_peered":"2024-03-23T17:48:57.260132+0000","last_clean":"2024-03-23T17:48:57.260132+0000","last_became_active":"2024-03-23T17:48:56.215653+0000","last_became_peered":"2024-03-23T17:48:56.215653+0000","last_unstale":"2024-03-23T17:48:57.260132+0000","last_undegraded":"2024-03-23T17:48:57.260132+0000","last_fullsized":"2024-03-23T17:48:57.260132+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:02:19.611056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'7","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:49:01.686750+0000","last_change":"2024-03-23T17:48:55.727425+0000","last_active":"2024-03-23T17:49:01.686750+0000","last_peered":"2024-03-23T17:49:01.686750+0000","last_clean":"2024-03-23T17:49:01.686750+0000","last_became_active":"2024-03-23T17:48:55.727071+0000","last_became_peered":"2024-03-23T17:48:55.727071+0000","last_unstale":"2024-03-23T17:49:01.686750+0000","last_undegraded":"2024-03-23T17:49:01.686750+0000","last_fullsized":"2024-03-23T17:49:01.686750+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:30:31.885441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-23T17:48:56.984444+0000","last_change":"2024-03-23T17:48:56.228439+0000","last_active":"2024-03-23T17:48:56.984444+0000","last_peered":"2024-03-23T17:48:56.984444+0000","last_clean":"2024-03-23T17:48:56.984444+0000","last_became_active":"2024-03-23T17:48:56.226677+0000","last_became_peered":"2024-03-23T17:48:56.226677+0000","last_unstale":"2024-03-23T17:48:56.984444+0000","last_undegraded":"2024-03-23T17:48:56.984444+0000","last_fullsized":"2024-03-23T17:48:56.984444+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:46:00.063266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'4","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:59.671887+0000","last_change":"2024-03-23T17:48:55.724872+0000","last_active":"2024-03-23T17:48:59.671887+0000","last_peered":"2024-03-23T17:48:59.671887+0000","last_clean":"2024-03-23T17:48:59.671887+0000","last_became_active":"2024-03-23T17:48:55.724468+0000","last_became_peered":"2024-03-23T17:48:55.724468+0000","last_unstale":"2024-03-23T17:48:59.671887+0000","last_undegraded":"2024-03-23T17:48:59.671887+0000","last_fullsized":"2024-03-23T17:48:59.671887+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:28:34.918056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.260034+0000","last_change":"2024-03-23T17:48:56.233730+0000","last_active":"2024-03-23T17:48:57.260034+0000","last_peered":"2024-03-23T17:48:57.260034+0000","last_clean":"2024-03-23T17:48:57.260034+0000","last_became_active":"2024-03-23T17:48:56.233189+0000","last_became_peered":"2024-03-23T17:48:56.233189+0000","last_unstale":"2024-03-23T17:48:57.260034+0000","last_undegraded":"2024-03-23T17:48:57.260034+0000","last_fullsized":"2024-03-23T17:48:57.260034+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:20:30.764344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:59.673121+0000","last_change":"2024-03-23T17:48:55.724633+0000","last_active":"2024-03-23T17:48:59.673121+0000","last_peered":"2024-03-23T17:48:59.673121+0000","last_clean":"2024-03-23T17:48:59.673121+0000","last_became_active":"2024-03-23T17:48:55.724301+0000","last_became_peered":"2024-03-23T17:48:55.724301+0000","last_unstale":"2024-03-23T17:48:59.673121+0000","last_undegraded":"2024-03-23T17:48:59.673121+0000","last_fullsized":"2024-03-23T17:48:59.673121+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:45:08.392668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.258996+0000","last_change":"2024-03-23T17:48:56.230981+0000","last_active":"2024-03-23T17:48:57.258996+0000","last_peered":"2024-03-23T17:48:57.258996+0000","last_clean":"2024-03-23T17:48:57.258996+0000","last_became_active":"2024-03-23T17:48:56.230644+0000","last_became_peered":"2024-03-23T17:48:56.230644+0000","last_unstale":"2024-03-23T17:48:57.258996+0000","last_undegraded":"2024-03-23T17:48:57.258996+0000","last_fullsized":"2024-03-23T17:48:57.258996+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:37:51.310938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'5","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:49:01.685039+0000","last_change":"2024-03-23T17:48:55.719112+0000","last_active":"2024-03-23T17:49:01.685039+0000","last_peered":"2024-03-23T17:49:01.685039+0000","last_clean":"2024-03-23T17:49:01.685039+0000","last_became_active":"2024-03-23T17:48:55.718756+0000","last_became_peered":"2024-03-23T17:48:55.718756+0000","last_unstale":"2024-03-23T17:49:01.685039+0000","last_undegraded":"2024-03-23T17:49:01.685039+0000","last_fullsized":"2024-03-23T17:49:01.685039+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:41:22.164706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.359316+0000","last_change":"2024-03-23T17:48:56.240289+0000","last_active":"2024-03-23T17:48:57.359316+0000","last_peered":"2024-03-23T17:48:57.359316+0000","last_clean":"2024-03-23T17:48:57.359316+0000","last_became_active":"2024-03-23T17:48:56.239896+0000","last_became_peered":"2024-03-23T17:48:56.239896+0000","last_unstale":"2024-03-23T17:48:57.359316+0000","last_undegraded":"2024-03-23T17:48:57.359316+0000","last_fullsized":"2024-03-23T17:48:57.359316+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:49:10.505576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-23T17:48:56.212920+0000","last_change":"2024-03-23T17:48:55.723330+0000","last_active":"2024-03-23T17:48:56.212920+0000","last_peered":"2024-03-23T17:48:56.212920+0000","last_clean":"2024-03-23T17:48:56.212920+0000","last_became_active":"2024-03-23T17:48:55.722879+0000","last_became_peered":"2024-03-23T17:48:55.722879+0000","last_unstale":"2024-03-23T17:48:56.212920+0000","last_undegraded":"2024-03-23T17:48:56.212920+0000","last_fullsized":"2024-03-23T17:48:56.212920+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:34:47.173411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-23T17:48:56.984403+0000","last_change":"2024-03-23T17:48:56.237042+0000","last_active":"2024-03-23T17:48:56.984403+0000","last_peered":"2024-03-23T17:48:56.984403+0000","last_clean":"2024-03-23T17:48:56.984403+0000","last_became_active":"2024-03-23T17:48:56.236722+0000","last_became_peered":"2024-03-23T17:48:56.236722+0000","last_unstale":"2024-03-23T17:48:56.984403+0000","last_undegraded":"2024-03-23T17:48:56.984403+0000","last_fullsized":"2024-03-23T17:48:56.984403+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:27:29.656120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:59.672459+0000","last_change":"2024-03-23T17:48:55.719223+0000","last_active":"2024-03-23T17:48:59.672459+0000","last_peered":"2024-03-23T17:48:59.672459+0000","last_clean":"2024-03-23T17:48:59.672459+0000","last_became_active":"2024-03-23T17:48:55.718864+0000","last_became_peered":"2024-03-23T17:48:55.718864+0000","last_unstale":"2024-03-23T17:48:59.672459+0000","last_undegraded":"2024-03-23T17:48:59.672459+0000","last_fullsized":"2024-03-23T17:48:59.672459+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:52:18.169557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-23T17:48:56.370959+0000","last_change":"2024-03-23T17:48:56.224885+0000","last_active":"2024-03-23T17:48:56.370959+0000","last_peered":"2024-03-23T17:48:56.370959+0000","last_clean":"2024-03-23T17:48:56.370959+0000","last_became_active":"2024-03-23T17:48:56.224160+0000","last_became_peered":"2024-03-23T17:48:56.224160+0000","last_unstale":"2024-03-23T17:48:56.370959+0000","last_undegraded":"2024-03-23T17:48:56.370959+0000","last_fullsized":"2024-03-23T17:48:56.370959+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:35:34.538313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-23T17:48:56.214095+0000","last_change":"2024-03-23T17:48:55.720518+0000","last_active":"2024-03-23T17:48:56.214095+0000","last_peered":"2024-03-23T17:48:56.214095+0000","last_clean":"2024-03-23T17:48:56.214095+0000","last_became_active":"2024-03-23T17:48:55.720045+0000","last_became_peered":"2024-03-23T17:48:55.720045+0000","last_unstale":"2024-03-23T17:48:56.214095+0000","last_undegraded":"2024-03-23T17:48:56.214095+0000","last_fullsized":"2024-03-23T17:48:56.214095+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:20:54.819406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-23T17:48:56.371015+0000","last_change":"2024-03-23T17:48:56.224171+0000","last_active":"2024-03-23T17:48:56.371015+0000","last_peered":"2024-03-23T17:48:56.371015+0000","last_clean":"2024-03-23T17:48:56.371015+0000","last_became_active":"2024-03-23T17:48:56.223737+0000","last_became_peered":"2024-03-23T17:48:56.223737+0000","last_unstale":"2024-03-23T17:48:56.371015+0000","last_undegraded":"2024-03-23T17:48:56.371015+0000","last_fullsized":"2024-03-23T17:48:56.371015+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:40:04.353945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'2","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:49:00.653260+0000","last_change":"2024-03-23T17:48:55.721734+0000","last_active":"2024-03-23T17:49:00.653260+0000","last_peered":"2024-03-23T17:49:00.653260+0000","last_clean":"2024-03-23T17:49:00.653260+0000","last_became_active":"2024-03-23T17:48:55.721417+0000","last_became_peered":"2024-03-23T17:48:55.721417+0000","last_unstale":"2024-03-23T17:49:00.653260+0000","last_undegraded":"2024-03-23T17:49:00.653260+0000","last_fullsized":"2024-03-23T17:49:00.653260+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:36:36.842740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-23T17:48:56.370980+0000","last_change":"2024-03-23T17:48:56.227211+0000","last_active":"2024-03-23T17:48:56.370980+0000","last_peered":"2024-03-23T17:48:56.370980+0000","last_clean":"2024-03-23T17:48:56.370980+0000","last_became_active":"2024-03-23T17:48:56.226881+0000","last_became_peered":"2024-03-23T17:48:56.226881+0000","last_unstale":"2024-03-23T17:48:56.370980+0000","last_undegraded":"2024-03-23T17:48:56.370980+0000","last_fullsized":"2024-03-23T17:48:56.370980+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:15:02.937666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.260341+0000","last_change":"2024-03-23T17:48:55.728679+0000","last_active":"2024-03-23T17:48:57.260341+0000","last_peered":"2024-03-23T17:48:57.260341+0000","last_clean":"2024-03-23T17:48:57.260341+0000","last_became_active":"2024-03-23T17:48:55.727754+0000","last_became_peered":"2024-03-23T17:48:55.727754+0000","last_unstale":"2024-03-23T17:48:57.260341+0000","last_undegraded":"2024-03-23T17:48:57.260341+0000","last_fullsized":"2024-03-23T17:48:57.260341+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:34:39.720488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-23T17:48:56.371037+0000","last_change":"2024-03-23T17:48:56.225214+0000","last_active":"2024-03-23T17:48:56.371037+0000","last_peered":"2024-03-23T17:48:56.371037+0000","last_clean":"2024-03-23T17:48:56.371037+0000","last_became_active":"2024-03-23T17:48:56.224635+0000","last_became_peered":"2024-03-23T17:48:56.224635+0000","last_unstale":"2024-03-23T17:48:56.371037+0000","last_undegraded":"2024-03-23T17:48:56.371037+0000","last_fullsized":"2024-03-23T17:48:56.371037+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:57:14.935434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.359428+0000","last_change":"2024-03-23T17:48:56.229835+0000","last_active":"2024-03-23T17:48:57.359428+0000","last_peered":"2024-03-23T17:48:57.359428+0000","last_clean":"2024-03-23T17:48:57.359428+0000","last_became_active":"2024-03-23T17:48:56.229437+0000","last_became_peered":"2024-03-23T17:48:56.229437+0000","last_unstale":"2024-03-23T17:48:57.359428+0000","last_undegraded":"2024-03-23T17:48:57.359428+0000","last_fullsized":"2024-03-23T17:48:57.359428+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:43:51.544867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.259118+0000","last_change":"2024-03-23T17:48:56.233552+0000","last_active":"2024-03-23T17:48:57.259118+0000","last_peered":"2024-03-23T17:48:57.259118+0000","last_clean":"2024-03-23T17:48:57.259118+0000","last_became_active":"2024-03-23T17:48:56.232941+0000","last_became_peered":"2024-03-23T17:48:56.232941+0000","last_unstale":"2024-03-23T17:48:57.259118+0000","last_undegraded":"2024-03-23T17:48:57.259118+0000","last_fullsized":"2024-03-23T17:48:57.259118+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:03:32.325496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.259155+0000","last_change":"2024-03-23T17:48:56.238238+0000","last_active":"2024-03-23T17:48:57.259155+0000","last_peered":"2024-03-23T17:48:57.259155+0000","last_clean":"2024-03-23T17:48:57.259155+0000","last_became_active":"2024-03-23T17:48:56.237882+0000","last_became_peered":"2024-03-23T17:48:56.237882+0000","last_unstale":"2024-03-23T17:48:57.259155+0000","last_undegraded":"2024-03-23T17:48:57.259155+0000","last_fullsized":"2024-03-23T17:48:57.259155+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:27:13.896205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.258990+0000","last_change":"2024-03-23T17:48:56.242895+0000","last_active":"2024-03-23T17:48:57.258990+0000","last_peered":"2024-03-23T17:48:57.258990+0000","last_clean":"2024-03-23T17:48:57.258990+0000","last_became_active":"2024-03-23T17:48:56.242515+0000","last_became_peered":"2024-03-23T17:48:56.242515+0000","last_unstale":"2024-03-23T17:48:57.258990+0000","last_undegraded":"2024-03-23T17:48:57.258990+0000","last_fullsized":"2024-03-23T17:48:57.258990+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:45:01.978843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.260154+0000","last_change":"2024-03-23T17:48:56.237029+0000","last_active":"2024-03-23T17:48:57.260154+0000","last_peered":"2024-03-23T17:48:57.260154+0000","last_clean":"2024-03-23T17:48:57.260154+0000","last_became_active":"2024-03-23T17:48:56.236692+0000","last_became_peered":"2024-03-23T17:48:56.236692+0000","last_unstale":"2024-03-23T17:48:57.260154+0000","last_undegraded":"2024-03-23T17:48:57.260154+0000","last_fullsized":"2024-03-23T17:48:57.260154+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:42:31.855469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.359525+0000","last_change":"2024-03-23T17:48:56.223236+0000","last_active":"2024-03-23T17:48:57.359525+0000","last_peered":"2024-03-23T17:48:57.359525+0000","last_clean":"2024-03-23T17:48:57.359525+0000","last_became_active":"2024-03-23T17:48:56.222824+0000","last_became_peered":"2024-03-23T17:48:56.222824+0000","last_unstale":"2024-03-23T17:48:57.359525+0000","last_undegraded":"2024-03-23T17:48:57.359525+0000","last_fullsized":"2024-03-23T17:48:57.359525+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:54:45.096969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.260287+0000","last_change":"2024-03-23T17:48:56.224471+0000","last_active":"2024-03-23T17:48:57.260287+0000","last_peered":"2024-03-23T17:48:57.260287+0000","last_clean":"2024-03-23T17:48:57.260287+0000","last_became_active":"2024-03-23T17:48:56.223140+0000","last_became_peered":"2024-03-23T17:48:56.223140+0000","last_unstale":"2024-03-23T17:48:57.260287+0000","last_undegraded":"2024-03-23T17:48:57.260287+0000","last_fullsized":"2024-03-23T17:48:57.260287+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:38:53.906820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-23T17:48:56.984433+0000","last_change":"2024-03-23T17:48:56.223564+0000","last_active":"2024-03-23T17:48:56.984433+0000","last_peered":"2024-03-23T17:48:56.984433+0000","last_clean":"2024-03-23T17:48:56.984433+0000","last_became_active":"2024-03-23T17:48:56.223217+0000","last_became_peered":"2024-03-23T17:48:56.223217+0000","last_unstale":"2024-03-23T17:48:56.984433+0000","last_undegraded":"2024-03-23T17:48:56.984433+0000","last_fullsized":"2024-03-23T17:48:56.984433+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:43:53.250082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.214184+0000","last_change":"2024-03-23T17:48:56.226283+0000","last_active":"2024-03-23T17:48:57.214184+0000","last_peered":"2024-03-23T17:48:57.214184+0000","last_clean":"2024-03-23T17:48:57.214184+0000","last_became_active":"2024-03-23T17:48:56.224211+0000","last_became_peered":"2024-03-23T17:48:56.224211+0000","last_unstale":"2024-03-23T17:48:57.214184+0000","last_undegraded":"2024-03-23T17:48:57.214184+0000","last_fullsized":"2024-03-23T17:48:57.214184+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:15:20.518950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-23T17:48:56.984368+0000","last_change":"2024-03-23T17:48:56.223557+0000","last_active":"2024-03-23T17:48:56.984368+0000","last_peered":"2024-03-23T17:48:56.984368+0000","last_clean":"2024-03-23T17:48:56.984368+0000","last_became_active":"2024-03-23T17:48:56.222792+0000","last_became_peered":"2024-03-23T17:48:56.222792+0000","last_unstale":"2024-03-23T17:48:56.984368+0000","last_undegraded":"2024-03-23T17:48:56.984368+0000","last_fullsized":"2024-03-23T17:48:56.984368+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:49:17.587817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-23T17:48:56.562286+0000","last_change":"2024-03-23T17:48:56.236041+0000","last_active":"2024-03-23T17:48:56.562286+0000","last_peered":"2024-03-23T17:48:56.562286+0000","last_clean":"2024-03-23T17:48:56.562286+0000","last_became_active":"2024-03-23T17:48:56.234259+0000","last_became_peered":"2024-03-23T17:48:56.234259+0000","last_unstale":"2024-03-23T17:48:56.562286+0000","last_undegraded":"2024-03-23T17:48:56.562286+0000","last_fullsized":"2024-03-23T17:48:56.562286+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:32:43.786783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.260007+0000","last_change":"2024-03-23T17:48:56.239283+0000","last_active":"2024-03-23T17:48:57.260007+0000","last_peered":"2024-03-23T17:48:57.260007+0000","last_clean":"2024-03-23T17:48:57.260007+0000","last_became_active":"2024-03-23T17:48:56.238952+0000","last_became_peered":"2024-03-23T17:48:56.238952+0000","last_unstale":"2024-03-23T17:48:57.260007+0000","last_undegraded":"2024-03-23T17:48:57.260007+0000","last_fullsized":"2024-03-23T17:48:57.260007+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:31:50.383284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.359422+0000","last_change":"2024-03-23T17:48:56.226684+0000","last_active":"2024-03-23T17:48:57.359422+0000","last_peered":"2024-03-23T17:48:57.359422+0000","last_clean":"2024-03-23T17:48:57.359422+0000","last_became_active":"2024-03-23T17:48:56.226367+0000","last_became_peered":"2024-03-23T17:48:56.226367+0000","last_unstale":"2024-03-23T17:48:57.359422+0000","last_undegraded":"2024-03-23T17:48:57.359422+0000","last_fullsized":"2024-03-23T17:48:57.359422+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:12:29.053330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5086,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":11,"num_read_kb":9,"num_write":130,"num_write_kb":43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":221184,"data_stored":13452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":113,"ondisk_log_size":113,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547212,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31956,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713196,"statfs":{"total":95995035648,"available":95962312704,"internally_reserved":0,"allocated":864256,"data_stored":411663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31848901},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40699999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]}]},{"osd":10,"up_from":68,"seq":292057776142,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28452,"kb_used_data":1432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716700,"statfs":{"total":95995035648,"available":95965900800,"internally_reserved":0,"allocated":1466368,"data_stored":1002314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66500000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85099999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83499999999999996}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86799999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.877}]}]},{"osd":9,"up_from":63,"seq":270582939668,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36060,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709092,"statfs":{"total":95995035648,"available":95958110208,"internally_reserved":0,"allocated":872448,"data_stored":411337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":36043201},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 17:48:42 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.39700000000000002}]},{"osd":1,"last update":"Sat Mar 23 17:48:42 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.68500000000000005}]},{"osd":2,"last update":"Sat Mar 23 17:48:42 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.55600000000000005}]},{"osd":3,"last update":"Sat Mar 23 17:48:42 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.44900000000000001}]},{"osd":4,"last update":"Sat Mar 23 17:48:42 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.69799999999999995}]},{"osd":5,"last update":"Sat Mar 23 17:48:42 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.48,"5min":1.48,"15min":1.48},"last":0.505}]},{"osd":6,"last update":"Sat Mar 23 17:48:42 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.49099999999999999}]},{"osd":7,"last update":"Sat Mar 23 17:48:42 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.373,"5min":1.373,"15min":1.373},"last":0.66800000000000004}]},{"osd":8,"last update":"Sat Mar 23 17:48:42 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.63900000000000001}]},{"osd":10,"last update":"Sat Mar 23 17:48:58 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.70499999999999996}]}]},{"osd":8,"up_from":56,"seq":240518168598,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":884736,"data_stored":412019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 17:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.502}]},{"osd":1,"last update":"Sat Mar 23 17:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.64800000000000002}]},{"osd":2,"last update":"Sat Mar 23 17:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.46999999999999997}]},{"osd":3,"last update":"Sat Mar 23 17:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.56699999999999995}]},{"osd":4,"last update":"Sat Mar 23 17:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.62}]},{"osd":5,"last update":"Sat Mar 23 17:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.53800000000000003}]},{"osd":6,"last update":"Sat Mar 23 17:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.73399999999999999}]},{"osd":7,"last update":"Sat Mar 23 17:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.66000000000000003}]},{"osd":9,"last update":"Sat Mar 23 17:48:42 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.71699999999999997}]},{"osd":10,"last update":"Sat Mar 23 17:49:01 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.63600000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77200000000000002}]}]},{"osd":7,"up_from":49,"seq":210453397530,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28444,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716708,"statfs":{"total":95995035648,"available":95965908992,"internally_reserved":0,"allocated":1462272,"data_stored":1002583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 17:48:02 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.54800000000000004}]},{"osd":1,"last update":"Sat Mar 23 17:48:02 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.371,"5min":0.371,"15min":0.371},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"last":0.46000000000000002}]},{"osd":2,"last update":"Sat Mar 23 17:48:02 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.60799999999999998}]},{"osd":3,"last update":"Sat Mar 23 17:48:02 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.65500000000000003}]},{"osd":4,"last update":"Sat Mar 23 17:48:02 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.51000000000000001}]},{"osd":5,"last update":"Sat Mar 23 17:48:02 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.76800000000000002}]},{"osd":6,"last update":"Sat Mar 23 17:48:02 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.751},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.71899999999999997}]},{"osd":8,"last update":"Sat Mar 23 17:48:21 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.78200000000000003}]},{"osd":9,"last update":"Sat Mar 23 17:48:42 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.81100000000000005}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82099999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84799999999999998}]}]},{"osd":6,"up_from":45,"seq":193273528350,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36048,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93709104,"statfs":{"total":95995035648,"available":95958122496,"internally_reserved":0,"allocated":847872,"data_stored":410246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 17:48:45 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.47499999999999998,"15min":0.47599999999999998},"min":{"1min":0.35599999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.52800000000000002,"15min":0.52400000000000002},"min":{"1min":0.33700000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.41699999999999998}]},{"osd":1,"last update":"Sat Mar 23 17:48:45 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.64400000000000002,"15min":0.64500000000000002},"min":{"1min":0.48399999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.91800000000000004,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.63300000000000001}]},{"osd":2,"last update":"Sat Mar 23 17:48:45 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.67900000000000005,"15min":0.68200000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.67400000000000004,"15min":0.67800000000000005},"min":{"1min":0.47099999999999997,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.91900000000000004,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.65200000000000002}]},{"osd":3,"last update":"Sat Mar 23 17:48:45 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.61399999999999999,"15min":0.61899999999999999},"min":{"1min":0.34399999999999997,"5min":0.316,"15min":0.316},"max":{"1min":0.89700000000000002,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.66400000000000003,"15min":0.66600000000000004},"min":{"1min":0.45600000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.88300000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.50700000000000001}]},{"osd":4,"last update":"Sat Mar 23 17:48:45 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.65700000000000003,"15min":0.65800000000000003},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.72999999999999998,"15min":0.73699999999999999},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.86399999999999999,"5min":0.998,"15min":0.998},"last":0.68400000000000005}]},{"osd":5,"last update":"Sat Mar 23 17:48:45 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.77900000000000003,"15min":0.78300000000000003},"min":{"1min":0.57199999999999995,"5min":0.436,"15min":0.436},"max":{"1min":0.90600000000000003,"5min":1.375,"15min":1.375},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.75,"5min":0.78100000000000003,"15min":0.78600000000000003},"min":{"1min":0.59199999999999997,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.67000000000000004}]},{"osd":7,"last update":"Sat Mar 23 17:48:03 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.78200000000000003}]},{"osd":8,"last update":"Sat Mar 23 17:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.747}]},{"osd":9,"last update":"Sat Mar 23 17:48:45 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.52900000000000003}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574896,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":864256,"data_stored":412083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 17:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.52400000000000002,"15min":0.51200000000000001},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.79200000000000004,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.502},{"interface":"front","average":{"1min":1.0680000000000001,"5min":0.61599999999999999,"15min":0.52800000000000002},"min":{"1min":0.38,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":5.8399999999999999,"5min":5.8399999999999999,"15min":5.8399999999999999},"last":0.52900000000000003}]},{"osd":2,"last update":"Sat Mar 23 17:48:28 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.48699999999999999,"15min":0.46800000000000003},"min":{"1min":0.40200000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.49399999999999999,"15min":0.48399999999999999},"min":{"1min":0.34499999999999997,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.76200000000000001,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.77400000000000002}]},{"osd":3,"last update":"Sat Mar 23 17:48:51 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.55000000000000004,"15min":0.53200000000000003},"min":{"1min":0.377,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.55100000000000005,"15min":0.53900000000000003},"min":{"1min":0.39100000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.66600000000000004}]},{"osd":4,"last update":"Sat Mar 23 17:48:08 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.40200000000000002,"15min":0.39200000000000002},"min":{"1min":0.30199999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.432,"5min":0.39300000000000002,"15min":0.38700000000000001},"min":{"1min":0.29199999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.64200000000000002}]},{"osd":5,"last update":"Sat Mar 23 17:48:28 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.55600000000000005,"15min":0.55000000000000004},"min":{"1min":0.35999999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.57599999999999996,"15min":0.56599999999999995},"min":{"1min":0.436,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.82399999999999995}]},{"osd":6,"last update":"Sat Mar 23 17:48:45 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.58799999999999997,"15min":0.56899999999999995},"min":{"1min":0.52100000000000002,"5min":0.433,"15min":0.433},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.63600000000000001,"15min":0.621},"min":{"1min":0.47399999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.78800000000000003}]},{"osd":7,"last update":"Sat Mar 23 17:48:02 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.84099999999999997}]},{"osd":8,"last update":"Sat Mar 23 17:48:20 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.80500000000000005}]},{"osd":9,"last update":"Sat Mar 23 17:48:45 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.90300000000000002}]},{"osd":10,"last update":"Sat Mar 23 17:48:59 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.877}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95099999999999996}]}]},{"osd":0,"up_from":8,"seq":34359738421,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":851968,"data_stored":410336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 23 17:48:14 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.68799999999999994,"15min":0.73799999999999999},"min":{"1min":0.45400000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":1.101,"5min":6.2110000000000003,"15min":6.2110000000000003},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.52800000000000002,"15min":0.502},"min":{"1min":0.41899999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.84499999999999997}]},{"osd":2,"last update":"Sat Mar 23 17:48:29 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.56399999999999995,"15min":0.54600000000000004},"min":{"1min":0.47099999999999997,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.57699999999999996,"15min":0.55200000000000005},"min":{"1min":0.52700000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.68700000000000006}]},{"osd":3,"last update":"Sat Mar 23 17:47:52 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.498,"15min":0.49399999999999999},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.78000000000000003,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.34499999999999997,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.79100000000000004,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.63400000000000001}]},{"osd":4,"last update":"Sat Mar 23 17:48:14 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.629,"15min":0.61899999999999999},"min":{"1min":0.48499999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.91300000000000003,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.59299999999999997,"15min":0.58599999999999997},"min":{"1min":0.46899999999999997,"5min":0.438,"15min":0.438},"max":{"1min":0.89500000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.77700000000000002}]},{"osd":5,"last update":"Sat Mar 23 17:48:24 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.65200000000000002,"15min":0.63600000000000001},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.629,"15min":0.61099999999999999},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.73299999999999998}]},{"osd":6,"last update":"Sat Mar 23 17:48:47 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.60699999999999998,"15min":0.60099999999999998},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.67900000000000005,"15min":0.66800000000000004},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.85699999999999998}]},{"osd":7,"last update":"Sat Mar 23 17:48:02 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.495},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.58599999999999997}]},{"osd":8,"last update":"Sat Mar 23 17:48:21 2024","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.89000000000000001}]},{"osd":9,"last update":"Sat Mar 23 17:48:42 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":0.91100000000000003}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95499999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92600000000000005}]}]},{"osd":2,"up_from":18,"seq":77309411373,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36060,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709092,"statfs":{"total":95995035648,"available":95958110208,"internally_reserved":0,"allocated":872448,"data_stored":412418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 17:48:28 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.51700000000000002,"15min":0.50600000000000001},"min":{"1min":0.435,"5min":0.37,"15min":0.37},"max":{"1min":0.73599999999999999,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.56100000000000005,"15min":0.54700000000000004},"min":{"1min":0.434,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.81299999999999994,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.67400000000000004}]},{"osd":1,"last update":"Sat Mar 23 17:48:28 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.51500000000000001,"15min":0.48999999999999999},"min":{"1min":0.33400000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.85899999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.53000000000000003,"15min":0.50800000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.83999999999999997,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.57399999999999995}]},{"osd":3,"last update":"Sat Mar 23 17:48:54 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.54400000000000004,"15min":0.52800000000000002},"min":{"1min":0.36699999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.58299999999999996,"15min":0.56499999999999995},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.69899999999999995}]},{"osd":4,"last update":"Sat Mar 23 17:48:06 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.51900000000000002,"15min":0.51100000000000001},"min":{"1min":0.41099999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.628,"5min":0.58199999999999996,"15min":0.57399999999999995},"min":{"1min":0.42599999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.44400000000000001}]},{"osd":5,"last update":"Sat Mar 23 17:48:24 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.46100000000000002,"15min":0.45100000000000001},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.504,"5min":0.46700000000000003,"15min":0.46100000000000002},"min":{"1min":0.314,"5min":0.308,"15min":0.308},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.46999999999999997}]},{"osd":6,"last update":"Sat Mar 23 17:48:48 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.623,"15min":0.61699999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.64400000000000002,"15min":0.63800000000000001},"min":{"1min":0.46100000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.73199999999999998}]},{"osd":7,"last update":"Sat Mar 23 17:48:01 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.745}]},{"osd":8,"last update":"Sat Mar 23 17:48:21 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.64900000000000002}]},{"osd":9,"last update":"Sat Mar 23 17:48:43 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.746},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.80600000000000005}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]}]},{"osd":3,"up_from":26,"seq":111669149737,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31956,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713196,"statfs":{"total":95995035648,"available":95962312704,"internally_reserved":0,"allocated":864256,"data_stored":410530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 17:48:58 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.41299999999999998,"15min":0.39300000000000002},"min":{"1min":0.34599999999999997,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.499,"5min":0.42499999999999999,"15min":0.40000000000000002},"min":{"1min":0.33500000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.94899999999999995,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.36799999999999999}]},{"osd":1,"last update":"Sat Mar 23 17:48:58 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.54200000000000004,"15min":0.51500000000000001},"min":{"1min":0.379,"5min":0.371,"15min":0.371},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.54100000000000004,"15min":0.50700000000000001},"min":{"1min":0.40699999999999997,"5min":0.371,"15min":0.371},"max":{"1min":0.91500000000000004,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.81299999999999994}]},{"osd":2,"last update":"Sat Mar 23 17:48:58 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.60999999999999999,"15min":0.58299999999999996},"min":{"1min":0.496,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.626,"15min":0.61199999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.90500000000000003,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.69199999999999995}]},{"osd":4,"last update":"Sat Mar 23 17:48:08 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.61499999999999999,"15min":0.60299999999999998},"min":{"1min":0.46899999999999997,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.61899999999999999,"15min":0.60699999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.86399999999999999}]},{"osd":5,"last update":"Sat Mar 23 17:48:26 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.66200000000000003,"15min":0.64400000000000002},"min":{"1min":0.52900000000000003,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.68400000000000005,"15min":0.66700000000000004},"min":{"1min":0.64400000000000002,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.78900000000000003}]},{"osd":6,"last update":"Sat Mar 23 17:48:45 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.64800000000000002,"15min":0.64400000000000002},"min":{"1min":0.504,"5min":0.38,"15min":0.38},"max":{"1min":0.88600000000000001,"5min":1.024,"15min":1.024},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.66900000000000004,"15min":0.66400000000000003},"min":{"1min":0.48099999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.93400000000000005,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.51300000000000001}]},{"osd":7,"last update":"Sat Mar 23 17:48:04 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.52900000000000003}]},{"osd":8,"last update":"Sat Mar 23 17:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.88},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.85099999999999998}]},{"osd":9,"last update":"Sat Mar 23 17:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":1.046}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]}]},{"osd":4,"up_from":33,"seq":141733920806,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":851968,"data_stored":410268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 17:48:06 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.53700000000000003,"15min":0.53400000000000003},"min":{"1min":0.41799999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.77600000000000002,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.53700000000000003,"15min":0.52500000000000002},"min":{"1min":0.45300000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.52000000000000002}]},{"osd":1,"last update":"Sat Mar 23 17:48:06 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.33300000000000002,"15min":0.31900000000000001},"min":{"1min":0.27700000000000002,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.436,"5min":0.35199999999999998,"15min":0.33800000000000002},"min":{"1min":0.29399999999999998,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.33700000000000002}]},{"osd":2,"last update":"Sat Mar 23 17:48:06 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.56200000000000006,"15min":0.55300000000000005},"min":{"1min":0.439,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.84899999999999998,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.55500000000000005,"15min":0.54200000000000004},"min":{"1min":0.42099999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.85299999999999998,"5min":0.88,"15min":0.88},"last":0.47599999999999998}]},{"osd":3,"last update":"Sat Mar 23 17:48:06 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.56599999999999995,"15min":0.55800000000000005},"min":{"1min":0.40799999999999997,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84099999999999997,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.57199999999999995,"15min":0.56399999999999995},"min":{"1min":0.46800000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.89300000000000002,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.60599999999999998}]},{"osd":5,"last update":"Sat Mar 23 17:48:28 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.56599999999999995,"15min":0.56100000000000005},"min":{"1min":0.42799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.81799999999999995,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.58299999999999996,"15min":0.57399999999999995},"min":{"1min":0.46000000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.82299999999999995,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.64500000000000002}]},{"osd":6,"last update":"Sat Mar 23 17:48:53 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.47699999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.78200000000000003,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.58999999999999997,"15min":0.58899999999999997},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.55900000000000005}]},{"osd":7,"last update":"Sat Mar 23 17:48:01 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.45100000000000001}]},{"osd":8,"last update":"Sat Mar 23 17:48:22 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.68200000000000005}]},{"osd":9,"last update":"Sat Mar 23 17:48:42 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.73699999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71099999999999997}]}]},{"osd":5,"up_from":38,"seq":163208757282,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32560,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712592,"statfs":{"total":95995035648,"available":95961694208,"internally_reserved":0,"allocated":1482752,"data_stored":1002746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 17:48:25 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.53900000000000003,"15min":0.53800000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.54400000000000004,"15min":0.54800000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.751}]},{"osd":1,"last update":"Sat Mar 23 17:48:25 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.52500000000000002,"15min":0.51900000000000002},"min":{"1min":0.38,"5min":0.37,"15min":0.37},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.63},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.55700000000000005,"15min":0.55500000000000005},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.66300000000000003,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.73199999999999998}]},{"osd":2,"last update":"Sat Mar 23 17:48:25 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.39100000000000001,"15min":0.39000000000000001},"min":{"1min":0.29999999999999999,"5min":0.251,"15min":0.251},"max":{"1min":0.52800000000000002,"5min":0.62,"15min":0.62},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.39600000000000002,"15min":0.39400000000000002},"min":{"1min":0.315,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.55300000000000005,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.438}]},{"osd":3,"last update":"Sat Mar 23 17:48:25 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.58299999999999996,"15min":0.58499999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.57499999999999996,"15min":0.58499999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.70399999999999996,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.497}]},{"osd":4,"last update":"Sat Mar 23 17:48:25 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.57999999999999996,"15min":0.57299999999999995},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.55600000000000005,"15min":0.55100000000000005},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.83099999999999996}]},{"osd":6,"last update":"Sat Mar 23 17:48:48 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.627,"15min":0.625},"min":{"1min":0.53400000000000003,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.88500000000000001,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.441,"5min":0.434,"15min":0.434},"max":{"1min":0.79600000000000004,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.58799999999999997}]},{"osd":7,"last update":"Sat Mar 23 17:48:02 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.79700000000000004}]},{"osd":8,"last update":"Sat Mar 23 17:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.53600000000000003}]},{"osd":9,"last update":"Sat Mar 23 17:48:43 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.65600000000000003}]},{"osd":10,"last update":"Sat Mar 23 17:49:00 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.878}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T17:49:05.553 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T17:49:05.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.782+0000 7fcb1c400700 1 -- 172.21.15.86:0/630073686 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb14100c80 msgr2=0x7fcb141010a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:05.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.782+0000 7fcb1c400700 1 --2- 172.21.15.86:0/630073686 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb14100c80 0x7fcb141010a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fcb080099c0 tx=0x7fcb08009cd0 comp rx=0 tx=0).stop 2024-03-23T17:49:05.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.782+0000 7fcb1c400700 1 -- 172.21.15.86:0/630073686 shutdown_connections 2024-03-23T17:49:05.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.782+0000 7fcb1c400700 1 --2- 172.21.15.86:0/630073686 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcb14102810 0x7fcb141090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:05.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.782+0000 7fcb1c400700 1 --2- 172.21.15.86:0/630073686 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb14100c80 0x7fcb141010a0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:05.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.782+0000 7fcb1c400700 1 --2- 172.21.15.86:0/630073686 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb14101e70 0x7fcb141022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:05.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.782+0000 7fcb1c400700 1 -- 172.21.15.86:0/630073686 >> 172.21.15.86:0/630073686 conn(0x7fcb140fc1f0 msgr2=0x7fcb140fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:05.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.782+0000 7fcb1c400700 1 -- 172.21.15.86:0/630073686 shutdown_connections 2024-03-23T17:49:05.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.782+0000 7fcb1c400700 1 -- 172.21.15.86:0/630073686 wait complete. 2024-03-23T17:49:05.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.783+0000 7fcb1c400700 1 Processor -- start 2024-03-23T17:49:05.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.783+0000 7fcb1c400700 1 -- start start 2024-03-23T17:49:05.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.784+0000 7fcb1c400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb14100c80 0x7fcb14074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:05.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.784+0000 7fcb1c400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcb14101e70 0x7fcb140731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:05.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.784+0000 7fcb1c400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb14102810 0x7fcb140736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:05.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.784+0000 7fcb1c400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb14073e80 con 0x7fcb14102810 2024-03-23T17:49:05.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.784+0000 7fcb1c400700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb14073ff0 con 0x7fcb14100c80 2024-03-23T17:49:05.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.784+0000 7fcb1c400700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb141a0c80 con 0x7fcb14101e70 2024-03-23T17:49:05.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.784+0000 7fcb1a400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb14102810 0x7fcb140736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:05.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.784+0000 7fcb1a400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb14102810 0x7fcb140736e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34768/0 (socket says 172.21.15.86:34768) 2024-03-23T17:49:05.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.784+0000 7fcb19a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb14100c80 0x7fcb14074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:05.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.784+0000 7fcb1a400700 1 -- 172.21.15.86:0/1489866194 learned_addr learned my addr 172.21.15.86:0/1489866194 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:05.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.784+0000 7fcb19000700 1 --2- 172.21.15.86:0/1489866194 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcb14101e70 0x7fcb140731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:05.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.784+0000 7fcb1a400700 1 -- 172.21.15.86:0/1489866194 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcb14101e70 msgr2=0x7fcb140731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:05.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.784+0000 7fcb1a400700 1 --2- 172.21.15.86:0/1489866194 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcb14101e70 0x7fcb140731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:05.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.784+0000 7fcb1a400700 1 -- 172.21.15.86:0/1489866194 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb14100c80 msgr2=0x7fcb14074b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:05.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.784+0000 7fcb1a400700 1 --2- 172.21.15.86:0/1489866194 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb14100c80 0x7fcb14074b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:05.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.784+0000 7fcb1a400700 1 -- 172.21.15.86:0/1489866194 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb080095f0 con 0x7fcb14102810 2024-03-23T17:49:05.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.784+0000 7fcb19000700 1 --2- 172.21.15.86:0/1489866194 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcb14101e70 0x7fcb140731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:49:05.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.785+0000 7fcb1a400700 1 --2- 172.21.15.86:0/1489866194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb14102810 0x7fcb140736e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c007ee0 tx=0x7fcb0c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:05.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.785+0000 7fcb12000700 1 -- 172.21.15.86:0/1489866194 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c017070 con 0x7fcb14102810 2024-03-23T17:49:05.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.785+0000 7fcb12000700 1 -- 172.21.15.86:0/1489866194 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb0c00f040 con 0x7fcb14102810 2024-03-23T17:49:05.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.785+0000 7fcb12000700 1 -- 172.21.15.86:0/1489866194 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c005740 con 0x7fcb14102810 2024-03-23T17:49:05.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.785+0000 7fcb1c400700 1 -- 172.21.15.86:0/1489866194 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb141a0f10 con 0x7fcb14102810 2024-03-23T17:49:05.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.785+0000 7fcb1c400700 1 -- 172.21.15.86:0/1489866194 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb141a1340 con 0x7fcb14102810 2024-03-23T17:49:05.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.786+0000 7fcb1c400700 1 -- 172.21.15.86:0/1489866194 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb141080d0 con 0x7fcb14102810 2024-03-23T17:49:05.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.788+0000 7fcb12000700 1 -- 172.21.15.86:0/1489866194 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fcb0c004c40 con 0x7fcb14102810 2024-03-23T17:49:05.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.789+0000 7fcb12000700 1 --2- 172.21.15.86:0/1489866194 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fcafc0a1230 0x7fcafc0a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:05.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.789+0000 7fcb12000700 1 -- 172.21.15.86:0/1489866194 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fcb0c0cbff0 con 0x7fcb14102810 2024-03-23T17:49:05.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.789+0000 7fcb19a00700 1 --2- 172.21.15.86:0/1489866194 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fcafc0a1230 0x7fcafc0a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:05.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.790+0000 7fcb19a00700 1 --2- 172.21.15.86:0/1489866194 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fcafc0a1230 0x7fcafc0a36f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fcb08009f50 tx=0x7fcb08005ea0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:05.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.794+0000 7fcb12000700 1 -- 172.21.15.86:0/1489866194 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fcb0c0ff9f0 con 0x7fcb14102810 2024-03-23T17:49:05.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:05.996+0000 7fcb1c400700 1 -- 172.21.15.86:0/1489866194 --> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcb1402ce30 con 0x7fcafc0a1230 2024-03-23T17:49:06.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:06.006+0000 7fcb12000700 1 -- 172.21.15.86:0/1489866194 <== mgr.14152 v2:172.21.15.86:6800/2734784244 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+329969 (secure 0 0 0) 0x7fcb1402ce30 con 0x7fcafc0a1230 2024-03-23T17:49:06.011 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:49:06.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:06.011+0000 7fcb1c400700 1 -- 172.21.15.86:0/1489866194 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fcafc0a1230 msgr2=0x7fcafc0a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:06.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:06.011+0000 7fcb1c400700 1 --2- 172.21.15.86:0/1489866194 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fcafc0a1230 0x7fcafc0a36f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fcb08009f50 tx=0x7fcb08005ea0 comp rx=0 tx=0).stop 2024-03-23T17:49:06.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:06.011+0000 7fcb1c400700 1 -- 172.21.15.86:0/1489866194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb14102810 msgr2=0x7fcb140736e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:06.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:06.011+0000 7fcb1c400700 1 --2- 172.21.15.86:0/1489866194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb14102810 0x7fcb140736e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c007ee0 tx=0x7fcb0c00d3b0 comp rx=0 tx=0).stop 2024-03-23T17:49:06.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:06.011+0000 7fcb1c400700 1 -- 172.21.15.86:0/1489866194 shutdown_connections 2024-03-23T17:49:06.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:06.011+0000 7fcb1c400700 1 --2- 172.21.15.86:0/1489866194 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fcafc0a1230 0x7fcafc0a36f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:06.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:06.011+0000 7fcb1c400700 1 --2- 172.21.15.86:0/1489866194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb14102810 0x7fcb140736e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:06.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:06.011+0000 7fcb1c400700 1 --2- 172.21.15.86:0/1489866194 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb14100c80 0x7fcb14074b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:06.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:06.011+0000 7fcb1c400700 1 --2- 172.21.15.86:0/1489866194 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcb14101e70 0x7fcb140731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:06.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:06.011+0000 7fcb1c400700 1 -- 172.21.15.86:0/1489866194 >> 172.21.15.86:0/1489866194 conn(0x7fcb140fc1f0 msgr2=0x7fcb14105210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:06.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:06.012+0000 7fcb1c400700 1 -- 172.21.15.86:0/1489866194 shutdown_connections 2024-03-23T17:49:06.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:06.012+0000 7fcb1c400700 1 -- 172.21.15.86:0/1489866194 wait complete. 2024-03-23T17:49:06.015 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-03-23T17:49:06.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:05 smithi086 ceph-mon[38221]: from='client.14637 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:49:06.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:05 smithi086 ceph-mon[38221]: pgmap v238: 97 pgs: 97 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 5.3 KiB/s wr, 17 op/s 2024-03-23T17:49:06.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:05 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/1253825365' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T17:49:06.030 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":238,"stamp":"2024-03-23T17:49:04.309563+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595454,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":147,"num_read_kb":128,"num_write":311,"num_write_kb":2981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":250,"ondisk_log_size":250,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":372988,"kb_used_data":11900,"kb_used_omap":18,"kb_used_meta":356845,"kb_avail":1124568836,"statfs":{"total":1151940427776,"available":1151558488064,"internally_reserved":0,"allocated":12185600,"data_stored":6708543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19130,"internal_metadata":365409606},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5086,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":11,"num_read_kb":9,"num_write":130,"num_write_kb":43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.104470"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.359640+0000","last_change":"2024-03-23T17:48:56.239724+0000","last_active":"2024-03-23T17:48:57.359640+0000","last_peered":"2024-03-23T17:48:57.359640+0000","last_clean":"2024-03-23T17:48:57.359640+0000","last_became_active":"2024-03-23T17:48:56.239451+0000","last_became_peered":"2024-03-23T17:48:56.239451+0000","last_unstale":"2024-03-23T17:48:57.359640+0000","last_undegraded":"2024-03-23T17:48:57.359640+0000","last_fullsized":"2024-03-23T17:48:57.359640+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:51:38.687863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.259420+0000","last_change":"2024-03-23T17:48:56.232415+0000","last_active":"2024-03-23T17:48:57.259420+0000","last_peered":"2024-03-23T17:48:57.259420+0000","last_clean":"2024-03-23T17:48:57.259420+0000","last_became_active":"2024-03-23T17:48:56.231777+0000","last_became_peered":"2024-03-23T17:48:56.231777+0000","last_unstale":"2024-03-23T17:48:57.259420+0000","last_undegraded":"2024-03-23T17:48:57.259420+0000","last_fullsized":"2024-03-23T17:48:57.259420+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:16:22.631654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.259288+0000","last_change":"2024-03-23T17:48:56.243413+0000","last_active":"2024-03-23T17:48:57.259288+0000","last_peered":"2024-03-23T17:48:57.259288+0000","last_clean":"2024-03-23T17:48:57.259288+0000","last_became_active":"2024-03-23T17:48:56.241281+0000","last_became_peered":"2024-03-23T17:48:56.241281+0000","last_unstale":"2024-03-23T17:48:57.259288+0000","last_undegraded":"2024-03-23T17:48:57.259288+0000","last_fullsized":"2024-03-23T17:48:57.259288+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:52:48.521689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.260410+0000","last_change":"2024-03-23T17:48:56.226010+0000","last_active":"2024-03-23T17:48:57.260410+0000","last_peered":"2024-03-23T17:48:57.260410+0000","last_clean":"2024-03-23T17:48:57.260410+0000","last_became_active":"2024-03-23T17:48:56.225562+0000","last_became_peered":"2024-03-23T17:48:56.225562+0000","last_unstale":"2024-03-23T17:48:57.260410+0000","last_undegraded":"2024-03-23T17:48:57.260410+0000","last_fullsized":"2024-03-23T17:48:57.260410+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:17:47.597335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.259300+0000","last_change":"2024-03-23T17:48:56.223249+0000","last_active":"2024-03-23T17:48:57.259300+0000","last_peered":"2024-03-23T17:48:57.259300+0000","last_clean":"2024-03-23T17:48:57.259300+0000","last_became_active":"2024-03-23T17:48:56.222902+0000","last_became_peered":"2024-03-23T17:48:56.222902+0000","last_unstale":"2024-03-23T17:48:57.259300+0000","last_undegraded":"2024-03-23T17:48:57.259300+0000","last_fullsized":"2024-03-23T17:48:57.259300+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:51:49.029628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.260827+0000","last_change":"2024-03-23T17:48:56.229229+0000","last_active":"2024-03-23T17:48:57.260827+0000","last_peered":"2024-03-23T17:48:57.260827+0000","last_clean":"2024-03-23T17:48:57.260827+0000","last_became_active":"2024-03-23T17:48:56.228725+0000","last_became_peered":"2024-03-23T17:48:56.228725+0000","last_unstale":"2024-03-23T17:48:57.260827+0000","last_undegraded":"2024-03-23T17:48:57.260827+0000","last_fullsized":"2024-03-23T17:48:57.260827+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:53:44.118762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-23T17:48:56.370934+0000","last_change":"2024-03-23T17:48:56.229212+0000","last_active":"2024-03-23T17:48:56.370934+0000","last_peered":"2024-03-23T17:48:56.370934+0000","last_clean":"2024-03-23T17:48:56.370934+0000","last_became_active":"2024-03-23T17:48:56.228888+0000","last_became_peered":"2024-03-23T17:48:56.228888+0000","last_unstale":"2024-03-23T17:48:56.370934+0000","last_undegraded":"2024-03-23T17:48:56.370934+0000","last_fullsized":"2024-03-23T17:48:56.370934+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:47:21.509554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-23T17:48:56.984356+0000","last_change":"2024-03-23T17:48:56.222925+0000","last_active":"2024-03-23T17:48:56.984356+0000","last_peered":"2024-03-23T17:48:56.984356+0000","last_clean":"2024-03-23T17:48:56.984356+0000","last_became_active":"2024-03-23T17:48:56.221504+0000","last_became_peered":"2024-03-23T17:48:56.221504+0000","last_unstale":"2024-03-23T17:48:56.984356+0000","last_undegraded":"2024-03-23T17:48:56.984356+0000","last_fullsized":"2024-03-23T17:48:56.984356+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:14:34.327705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-23T17:48:56.562349+0000","last_change":"2024-03-23T17:48:56.230530+0000","last_active":"2024-03-23T17:48:56.562349+0000","last_peered":"2024-03-23T17:48:56.562349+0000","last_clean":"2024-03-23T17:48:56.562349+0000","last_became_active":"2024-03-23T17:48:56.230224+0000","last_became_peered":"2024-03-23T17:48:56.230224+0000","last_unstale":"2024-03-23T17:48:56.562349+0000","last_undegraded":"2024-03-23T17:48:56.562349+0000","last_fullsized":"2024-03-23T17:48:56.562349+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:10:28.031956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.259603+0000","last_change":"2024-03-23T17:48:56.229673+0000","last_active":"2024-03-23T17:48:57.259603+0000","last_peered":"2024-03-23T17:48:57.259603+0000","last_clean":"2024-03-23T17:48:57.259603+0000","last_became_active":"2024-03-23T17:48:56.229000+0000","last_became_peered":"2024-03-23T17:48:56.229000+0000","last_unstale":"2024-03-23T17:48:57.259603+0000","last_undegraded":"2024-03-23T17:48:57.259603+0000","last_fullsized":"2024-03-23T17:48:57.259603+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:23:28.057256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-23T17:48:56.371001+0000","last_change":"2024-03-23T17:48:56.224508+0000","last_active":"2024-03-23T17:48:56.371001+0000","last_peered":"2024-03-23T17:48:56.371001+0000","last_clean":"2024-03-23T17:48:56.371001+0000","last_became_active":"2024-03-23T17:48:56.224172+0000","last_became_peered":"2024-03-23T17:48:56.224172+0000","last_unstale":"2024-03-23T17:48:56.371001+0000","last_undegraded":"2024-03-23T17:48:56.371001+0000","last_fullsized":"2024-03-23T17:48:56.371001+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:03:57.063472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.260352+0000","last_change":"2024-03-23T17:48:56.234978+0000","last_active":"2024-03-23T17:48:57.260352+0000","last_peered":"2024-03-23T17:48:57.260352+0000","last_clean":"2024-03-23T17:48:57.260352+0000","last_became_active":"2024-03-23T17:48:56.234641+0000","last_became_peered":"2024-03-23T17:48:56.234641+0000","last_unstale":"2024-03-23T17:48:57.260352+0000","last_undegraded":"2024-03-23T17:48:57.260352+0000","last_fullsized":"2024-03-23T17:48:57.260352+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:34:30.745991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.260030+0000","last_change":"2024-03-23T17:48:56.218534+0000","last_active":"2024-03-23T17:48:57.260030+0000","last_peered":"2024-03-23T17:48:57.260030+0000","last_clean":"2024-03-23T17:48:57.260030+0000","last_became_active":"2024-03-23T17:48:56.217424+0000","last_became_peered":"2024-03-23T17:48:56.217424+0000","last_unstale":"2024-03-23T17:48:57.260030+0000","last_undegraded":"2024-03-23T17:48:57.260030+0000","last_fullsized":"2024-03-23T17:48:57.260030+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:32:05.618057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.259005+0000","last_change":"2024-03-23T17:48:56.235525+0000","last_active":"2024-03-23T17:48:57.259005+0000","last_peered":"2024-03-23T17:48:57.259005+0000","last_clean":"2024-03-23T17:48:57.259005+0000","last_became_active":"2024-03-23T17:48:56.234591+0000","last_became_peered":"2024-03-23T17:48:56.234591+0000","last_unstale":"2024-03-23T17:48:57.259005+0000","last_undegraded":"2024-03-23T17:48:57.259005+0000","last_fullsized":"2024-03-23T17:48:57.259005+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:19:38.381878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.258904+0000","last_change":"2024-03-23T17:48:56.243137+0000","last_active":"2024-03-23T17:48:57.258904+0000","last_peered":"2024-03-23T17:48:57.258904+0000","last_clean":"2024-03-23T17:48:57.258904+0000","last_became_active":"2024-03-23T17:48:56.242516+0000","last_became_peered":"2024-03-23T17:48:56.242516+0000","last_unstale":"2024-03-23T17:48:57.258904+0000","last_undegraded":"2024-03-23T17:48:57.258904+0000","last_fullsized":"2024-03-23T17:48:57.258904+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:54:59.849467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-23T17:48:56.562254+0000","last_change":"2024-03-23T17:48:56.236026+0000","last_active":"2024-03-23T17:48:56.562254+0000","last_peered":"2024-03-23T17:48:56.562254+0000","last_clean":"2024-03-23T17:48:56.562254+0000","last_became_active":"2024-03-23T17:48:56.235645+0000","last_became_peered":"2024-03-23T17:48:56.235645+0000","last_unstale":"2024-03-23T17:48:56.562254+0000","last_undegraded":"2024-03-23T17:48:56.562254+0000","last_fullsized":"2024-03-23T17:48:56.562254+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:47:01.076737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.359659+0000","last_change":"2024-03-23T17:48:56.239689+0000","last_active":"2024-03-23T17:48:57.359659+0000","last_peered":"2024-03-23T17:48:57.359659+0000","last_clean":"2024-03-23T17:48:57.359659+0000","last_became_active":"2024-03-23T17:48:56.239331+0000","last_became_peered":"2024-03-23T17:48:56.239331+0000","last_unstale":"2024-03-23T17:48:57.359659+0000","last_undegraded":"2024-03-23T17:48:57.359659+0000","last_fullsized":"2024-03-23T17:48:57.359659+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:21:15.287627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.014188+0000","last_change":"2024-03-23T17:48:56.229934+0000","last_active":"2024-03-23T17:48:57.014188+0000","last_peered":"2024-03-23T17:48:57.014188+0000","last_clean":"2024-03-23T17:48:57.014188+0000","last_became_active":"2024-03-23T17:48:56.229244+0000","last_became_peered":"2024-03-23T17:48:56.229244+0000","last_unstale":"2024-03-23T17:48:57.014188+0000","last_undegraded":"2024-03-23T17:48:57.014188+0000","last_fullsized":"2024-03-23T17:48:57.014188+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:02:40.254799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.259964+0000","last_change":"2024-03-23T17:48:56.233786+0000","last_active":"2024-03-23T17:48:57.259964+0000","last_peered":"2024-03-23T17:48:57.259964+0000","last_clean":"2024-03-23T17:48:57.259964+0000","last_became_active":"2024-03-23T17:48:56.233137+0000","last_became_peered":"2024-03-23T17:48:56.233137+0000","last_unstale":"2024-03-23T17:48:57.259964+0000","last_undegraded":"2024-03-23T17:48:57.259964+0000","last_fullsized":"2024-03-23T17:48:57.259964+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:33:35.202788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'11","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:49:01.684392+0000","last_change":"2024-03-23T17:48:55.712184+0000","last_active":"2024-03-23T17:49:01.684392+0000","last_peered":"2024-03-23T17:49:01.684392+0000","last_clean":"2024-03-23T17:49:01.684392+0000","last_became_active":"2024-03-23T17:48:55.711860+0000","last_became_peered":"2024-03-23T17:48:55.711860+0000","last_unstale":"2024-03-23T17:49:01.684392+0000","last_undegraded":"2024-03-23T17:49:01.684392+0000","last_fullsized":"2024-03-23T17:49:01.684392+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:59:01.133652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-23T17:48:56.984412+0000","last_change":"2024-03-23T17:48:56.228317+0000","last_active":"2024-03-23T17:48:56.984412+0000","last_peered":"2024-03-23T17:48:56.984412+0000","last_clean":"2024-03-23T17:48:56.984412+0000","last_became_active":"2024-03-23T17:48:56.227197+0000","last_became_peered":"2024-03-23T17:48:56.227197+0000","last_unstale":"2024-03-23T17:48:56.984412+0000","last_undegraded":"2024-03-23T17:48:56.984412+0000","last_fullsized":"2024-03-23T17:48:56.984412+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:33:55.677013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.260645+0000","last_change":"2024-03-23T17:48:55.725351+0000","last_active":"2024-03-23T17:48:57.260645+0000","last_peered":"2024-03-23T17:48:57.260645+0000","last_clean":"2024-03-23T17:48:57.260645+0000","last_became_active":"2024-03-23T17:48:55.724566+0000","last_became_peered":"2024-03-23T17:48:55.724566+0000","last_unstale":"2024-03-23T17:48:57.260645+0000","last_undegraded":"2024-03-23T17:48:57.260645+0000","last_fullsized":"2024-03-23T17:48:57.260645+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:58:21.088166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.260283+0000","last_change":"2024-03-23T17:48:56.243526+0000","last_active":"2024-03-23T17:48:57.260283+0000","last_peered":"2024-03-23T17:48:57.260283+0000","last_clean":"2024-03-23T17:48:57.260283+0000","last_became_active":"2024-03-23T17:48:56.242468+0000","last_became_peered":"2024-03-23T17:48:56.242468+0000","last_unstale":"2024-03-23T17:48:57.260283+0000","last_undegraded":"2024-03-23T17:48:57.260283+0000","last_fullsized":"2024-03-23T17:48:57.260283+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:16:50.392770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'11","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:49:02.709260+0000","last_change":"2024-03-23T17:48:55.720237+0000","last_active":"2024-03-23T17:49:02.709260+0000","last_peered":"2024-03-23T17:49:02.709260+0000","last_clean":"2024-03-23T17:49:02.709260+0000","last_became_active":"2024-03-23T17:48:55.719843+0000","last_became_peered":"2024-03-23T17:48:55.719843+0000","last_unstale":"2024-03-23T17:49:02.709260+0000","last_undegraded":"2024-03-23T17:49:02.709260+0000","last_fullsized":"2024-03-23T17:49:02.709260+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:07:26.973511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-23T17:48:56.562302+0000","last_change":"2024-03-23T17:48:56.235642+0000","last_active":"2024-03-23T17:48:56.562302+0000","last_peered":"2024-03-23T17:48:56.562302+0000","last_clean":"2024-03-23T17:48:56.562302+0000","last_became_active":"2024-03-23T17:48:56.235326+0000","last_became_peered":"2024-03-23T17:48:56.235326+0000","last_unstale":"2024-03-23T17:48:56.562302+0000","last_undegraded":"2024-03-23T17:48:56.562302+0000","last_fullsized":"2024-03-23T17:48:56.562302+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:21:59.934620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:49:02.710352+0000","last_change":"2024-03-23T17:48:55.718579+0000","last_active":"2024-03-23T17:49:02.710352+0000","last_peered":"2024-03-23T17:49:02.710352+0000","last_clean":"2024-03-23T17:49:02.710352+0000","last_became_active":"2024-03-23T17:48:55.718220+0000","last_became_peered":"2024-03-23T17:48:55.718220+0000","last_unstale":"2024-03-23T17:49:02.710352+0000","last_undegraded":"2024-03-23T17:49:02.710352+0000","last_fullsized":"2024-03-23T17:49:02.710352+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:37:56.417367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.214206+0000","last_change":"2024-03-23T17:48:56.224842+0000","last_active":"2024-03-23T17:48:57.214206+0000","last_peered":"2024-03-23T17:48:57.214206+0000","last_clean":"2024-03-23T17:48:57.214206+0000","last_became_active":"2024-03-23T17:48:56.224427+0000","last_became_peered":"2024-03-23T17:48:56.224427+0000","last_unstale":"2024-03-23T17:48:57.214206+0000","last_undegraded":"2024-03-23T17:48:57.214206+0000","last_fullsized":"2024-03-23T17:48:57.214206+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:16:22.684369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"80'5","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:49:00.656886+0000","last_change":"2024-03-23T17:48:55.713996+0000","last_active":"2024-03-23T17:49:00.656886+0000","last_peered":"2024-03-23T17:49:00.656886+0000","last_clean":"2024-03-23T17:49:00.656886+0000","last_became_active":"2024-03-23T17:48:55.713647+0000","last_became_peered":"2024-03-23T17:48:55.713647+0000","last_unstale":"2024-03-23T17:49:00.656886+0000","last_undegraded":"2024-03-23T17:49:00.656886+0000","last_fullsized":"2024-03-23T17:49:00.656886+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:12:22.251533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.259235+0000","last_change":"2024-03-23T17:48:56.219281+0000","last_active":"2024-03-23T17:48:57.259235+0000","last_peered":"2024-03-23T17:48:57.259235+0000","last_clean":"2024-03-23T17:48:57.259235+0000","last_became_active":"2024-03-23T17:48:56.218924+0000","last_became_peered":"2024-03-23T17:48:56.218924+0000","last_unstale":"2024-03-23T17:48:57.259235+0000","last_undegraded":"2024-03-23T17:48:57.259235+0000","last_fullsized":"2024-03-23T17:48:57.259235+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:49:04.793088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"79'1","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.260763+0000","last_change":"2024-03-23T17:48:55.718131+0000","last_active":"2024-03-23T17:48:57.260763+0000","last_peered":"2024-03-23T17:48:57.260763+0000","last_clean":"2024-03-23T17:48:57.260763+0000","last_became_active":"2024-03-23T17:48:55.717760+0000","last_became_peered":"2024-03-23T17:48:55.717760+0000","last_unstale":"2024-03-23T17:48:57.260763+0000","last_undegraded":"2024-03-23T17:48:57.260763+0000","last_fullsized":"2024-03-23T17:48:57.260763+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:16:54.355844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.214056+0000","last_change":"2024-03-23T17:48:56.225965+0000","last_active":"2024-03-23T17:48:57.214056+0000","last_peered":"2024-03-23T17:48:57.214056+0000","last_clean":"2024-03-23T17:48:57.214056+0000","last_became_active":"2024-03-23T17:48:56.225101+0000","last_became_peered":"2024-03-23T17:48:56.225101+0000","last_unstale":"2024-03-23T17:48:57.214056+0000","last_undegraded":"2024-03-23T17:48:57.214056+0000","last_fullsized":"2024-03-23T17:48:57.214056+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:52:14.302306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.259871+0000","last_change":"2024-03-23T17:48:55.722289+0000","last_active":"2024-03-23T17:48:57.259871+0000","last_peered":"2024-03-23T17:48:57.259871+0000","last_clean":"2024-03-23T17:48:57.259871+0000","last_became_active":"2024-03-23T17:48:55.721927+0000","last_became_peered":"2024-03-23T17:48:55.721927+0000","last_unstale":"2024-03-23T17:48:57.259871+0000","last_undegraded":"2024-03-23T17:48:57.259871+0000","last_fullsized":"2024-03-23T17:48:57.259871+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:20:08.354897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-23T17:48:56.984339+0000","last_change":"2024-03-23T17:48:56.232231+0000","last_active":"2024-03-23T17:48:56.984339+0000","last_peered":"2024-03-23T17:48:56.984339+0000","last_clean":"2024-03-23T17:48:56.984339+0000","last_became_active":"2024-03-23T17:48:56.227335+0000","last_became_peered":"2024-03-23T17:48:56.227335+0000","last_unstale":"2024-03-23T17:48:56.984339+0000","last_undegraded":"2024-03-23T17:48:56.984339+0000","last_fullsized":"2024-03-23T17:48:56.984339+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:49:01.460775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'2","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:59.672427+0000","last_change":"2024-03-23T17:48:55.717615+0000","last_active":"2024-03-23T17:48:59.672427+0000","last_peered":"2024-03-23T17:48:59.672427+0000","last_clean":"2024-03-23T17:48:59.672427+0000","last_became_active":"2024-03-23T17:48:55.717266+0000","last_became_peered":"2024-03-23T17:48:55.717266+0000","last_unstale":"2024-03-23T17:48:59.672427+0000","last_undegraded":"2024-03-23T17:48:59.672427+0000","last_fullsized":"2024-03-23T17:48:59.672427+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:40:07.988182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.014083+0000","last_change":"2024-03-23T17:48:56.229787+0000","last_active":"2024-03-23T17:48:57.014083+0000","last_peered":"2024-03-23T17:48:57.014083+0000","last_clean":"2024-03-23T17:48:57.014083+0000","last_became_active":"2024-03-23T17:48:56.229471+0000","last_became_peered":"2024-03-23T17:48:56.229471+0000","last_unstale":"2024-03-23T17:48:57.014083+0000","last_undegraded":"2024-03-23T17:48:57.014083+0000","last_fullsized":"2024-03-23T17:48:57.014083+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:44:09.358809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.259940+0000","last_change":"2024-03-23T17:48:55.717130+0000","last_active":"2024-03-23T17:48:57.259940+0000","last_peered":"2024-03-23T17:48:57.259940+0000","last_clean":"2024-03-23T17:48:57.259940+0000","last_became_active":"2024-03-23T17:48:55.716733+0000","last_became_peered":"2024-03-23T17:48:55.716733+0000","last_unstale":"2024-03-23T17:48:57.259940+0000","last_undegraded":"2024-03-23T17:48:57.259940+0000","last_fullsized":"2024-03-23T17:48:57.259940+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:22:43.229694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-23T17:48:56.562338+0000","last_change":"2024-03-23T17:48:56.235565+0000","last_active":"2024-03-23T17:48:56.562338+0000","last_peered":"2024-03-23T17:48:56.562338+0000","last_clean":"2024-03-23T17:48:56.562338+0000","last_became_active":"2024-03-23T17:48:56.235267+0000","last_became_peered":"2024-03-23T17:48:56.235267+0000","last_unstale":"2024-03-23T17:48:56.562338+0000","last_undegraded":"2024-03-23T17:48:56.562338+0000","last_fullsized":"2024-03-23T17:48:56.562338+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:38:45.906545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'2","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:59.672296+0000","last_change":"2024-03-23T17:48:55.719096+0000","last_active":"2024-03-23T17:48:59.672296+0000","last_peered":"2024-03-23T17:48:59.672296+0000","last_clean":"2024-03-23T17:48:59.672296+0000","last_became_active":"2024-03-23T17:48:55.718721+0000","last_became_peered":"2024-03-23T17:48:55.718721+0000","last_unstale":"2024-03-23T17:48:59.672296+0000","last_undegraded":"2024-03-23T17:48:59.672296+0000","last_fullsized":"2024-03-23T17:48:59.672296+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:43:05.532263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.260207+0000","last_change":"2024-03-23T17:48:56.235549+0000","last_active":"2024-03-23T17:48:57.260207+0000","last_peered":"2024-03-23T17:48:57.260207+0000","last_clean":"2024-03-23T17:48:57.260207+0000","last_became_active":"2024-03-23T17:48:56.234572+0000","last_became_peered":"2024-03-23T17:48:56.234572+0000","last_unstale":"2024-03-23T17:48:57.260207+0000","last_undegraded":"2024-03-23T17:48:57.260207+0000","last_fullsized":"2024-03-23T17:48:57.260207+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:19:58.639181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:49:00.656075+0000","last_change":"2024-03-23T17:48:55.724688+0000","last_active":"2024-03-23T17:49:00.656075+0000","last_peered":"2024-03-23T17:49:00.656075+0000","last_clean":"2024-03-23T17:49:00.656075+0000","last_became_active":"2024-03-23T17:48:55.724385+0000","last_became_peered":"2024-03-23T17:48:55.724385+0000","last_unstale":"2024-03-23T17:49:00.656075+0000","last_undegraded":"2024-03-23T17:49:00.656075+0000","last_fullsized":"2024-03-23T17:49:00.656075+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:17:54.338059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-23T17:48:56.562325+0000","last_change":"2024-03-23T17:48:56.242983+0000","last_active":"2024-03-23T17:48:56.562325+0000","last_peered":"2024-03-23T17:48:56.562325+0000","last_clean":"2024-03-23T17:48:56.562325+0000","last_became_active":"2024-03-23T17:48:56.242596+0000","last_became_peered":"2024-03-23T17:48:56.242596+0000","last_unstale":"2024-03-23T17:48:56.562325+0000","last_undegraded":"2024-03-23T17:48:56.562325+0000","last_fullsized":"2024-03-23T17:48:56.562325+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:55:51.971834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"79'6","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.259839+0000","last_change":"2024-03-23T17:48:55.717631+0000","last_active":"2024-03-23T17:48:57.259839+0000","last_peered":"2024-03-23T17:48:57.259839+0000","last_clean":"2024-03-23T17:48:57.259839+0000","last_became_active":"2024-03-23T17:48:55.717082+0000","last_became_peered":"2024-03-23T17:48:55.717082+0000","last_unstale":"2024-03-23T17:48:57.259839+0000","last_undegraded":"2024-03-23T17:48:57.259839+0000","last_fullsized":"2024-03-23T17:48:57.259839+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:36:25.323269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.259542+0000","last_change":"2024-03-23T17:48:56.226174+0000","last_active":"2024-03-23T17:48:57.259542+0000","last_peered":"2024-03-23T17:48:57.259542+0000","last_clean":"2024-03-23T17:48:57.259542+0000","last_became_active":"2024-03-23T17:48:56.225834+0000","last_became_peered":"2024-03-23T17:48:56.225834+0000","last_unstale":"2024-03-23T17:48:57.259542+0000","last_undegraded":"2024-03-23T17:48:57.259542+0000","last_fullsized":"2024-03-23T17:48:57.259542+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:50:16.793071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'4","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:59.674040+0000","last_change":"2024-03-23T17:48:55.723059+0000","last_active":"2024-03-23T17:48:59.674040+0000","last_peered":"2024-03-23T17:48:59.674040+0000","last_clean":"2024-03-23T17:48:59.674040+0000","last_became_active":"2024-03-23T17:48:55.722402+0000","last_became_peered":"2024-03-23T17:48:55.722402+0000","last_unstale":"2024-03-23T17:48:59.674040+0000","last_undegraded":"2024-03-23T17:48:59.674040+0000","last_fullsized":"2024-03-23T17:48:59.674040+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:40:28.322650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.259860+0000","last_change":"2024-03-23T17:48:56.228459+0000","last_active":"2024-03-23T17:48:57.259860+0000","last_peered":"2024-03-23T17:48:57.259860+0000","last_clean":"2024-03-23T17:48:57.259860+0000","last_became_active":"2024-03-23T17:48:56.228179+0000","last_became_peered":"2024-03-23T17:48:56.228179+0000","last_unstale":"2024-03-23T17:48:57.259860+0000","last_undegraded":"2024-03-23T17:48:57.259860+0000","last_fullsized":"2024-03-23T17:48:57.259860+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:28:41.613432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'4","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:59.672767+0000","last_change":"2024-03-23T17:48:55.721533+0000","last_active":"2024-03-23T17:48:59.672767+0000","last_peered":"2024-03-23T17:48:59.672767+0000","last_clean":"2024-03-23T17:48:59.672767+0000","last_became_active":"2024-03-23T17:48:55.721100+0000","last_became_peered":"2024-03-23T17:48:55.721100+0000","last_unstale":"2024-03-23T17:48:59.672767+0000","last_undegraded":"2024-03-23T17:48:59.672767+0000","last_fullsized":"2024-03-23T17:48:59.672767+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:56:58.112649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-23T17:48:56.370970+0000","last_change":"2024-03-23T17:48:56.226508+0000","last_active":"2024-03-23T17:48:56.370970+0000","last_peered":"2024-03-23T17:48:56.370970+0000","last_clean":"2024-03-23T17:48:56.370970+0000","last_became_active":"2024-03-23T17:48:56.226082+0000","last_became_peered":"2024-03-23T17:48:56.226082+0000","last_unstale":"2024-03-23T17:48:56.370970+0000","last_undegraded":"2024-03-23T17:48:56.370970+0000","last_fullsized":"2024-03-23T17:48:56.370970+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:38:38.133029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-23T17:48:56.220140+0000","last_change":"2024-03-23T17:48:55.715205+0000","last_active":"2024-03-23T17:48:56.220140+0000","last_peered":"2024-03-23T17:48:56.220140+0000","last_clean":"2024-03-23T17:48:56.220140+0000","last_became_active":"2024-03-23T17:48:55.714810+0000","last_became_peered":"2024-03-23T17:48:55.714810+0000","last_unstale":"2024-03-23T17:48:56.220140+0000","last_undegraded":"2024-03-23T17:48:56.220140+0000","last_fullsized":"2024-03-23T17:48:56.220140+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:40:08.768998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-23T17:48:56.370884+0000","last_change":"2024-03-23T17:48:56.241434+0000","last_active":"2024-03-23T17:48:56.370884+0000","last_peered":"2024-03-23T17:48:56.370884+0000","last_clean":"2024-03-23T17:48:56.370884+0000","last_became_active":"2024-03-23T17:48:56.240532+0000","last_became_peered":"2024-03-23T17:48:56.240532+0000","last_unstale":"2024-03-23T17:48:56.370884+0000","last_undegraded":"2024-03-23T17:48:56.370884+0000","last_fullsized":"2024-03-23T17:48:56.370884+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:56:15.970864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:49:01.685147+0000","last_change":"2024-03-23T17:48:55.721653+0000","last_active":"2024-03-23T17:49:01.685147+0000","last_peered":"2024-03-23T17:49:01.685147+0000","last_clean":"2024-03-23T17:49:01.685147+0000","last_became_active":"2024-03-23T17:48:55.721317+0000","last_became_peered":"2024-03-23T17:48:55.721317+0000","last_unstale":"2024-03-23T17:49:01.685147+0000","last_undegraded":"2024-03-23T17:49:01.685147+0000","last_fullsized":"2024-03-23T17:49:01.685147+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:31:40.085230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.213925+0000","last_change":"2024-03-23T17:48:56.225624+0000","last_active":"2024-03-23T17:48:57.213925+0000","last_peered":"2024-03-23T17:48:57.213925+0000","last_clean":"2024-03-23T17:48:57.213925+0000","last_became_active":"2024-03-23T17:48:56.223798+0000","last_became_peered":"2024-03-23T17:48:56.223798+0000","last_unstale":"2024-03-23T17:48:57.213925+0000","last_undegraded":"2024-03-23T17:48:57.213925+0000","last_fullsized":"2024-03-23T17:48:57.213925+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:00:20.547553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"79'1","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:58.639902+0000","last_change":"2024-03-23T17:48:55.717792+0000","last_active":"2024-03-23T17:48:58.639902+0000","last_peered":"2024-03-23T17:48:58.639902+0000","last_clean":"2024-03-23T17:48:58.639902+0000","last_became_active":"2024-03-23T17:48:55.717479+0000","last_became_peered":"2024-03-23T17:48:55.717479+0000","last_unstale":"2024-03-23T17:48:58.639902+0000","last_undegraded":"2024-03-23T17:48:58.639902+0000","last_fullsized":"2024-03-23T17:48:58.639902+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:49:43.483785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.359512+0000","last_change":"2024-03-23T17:48:56.219135+0000","last_active":"2024-03-23T17:48:57.359512+0000","last_peered":"2024-03-23T17:48:57.359512+0000","last_clean":"2024-03-23T17:48:57.359512+0000","last_became_active":"2024-03-23T17:48:56.218824+0000","last_became_peered":"2024-03-23T17:48:56.218824+0000","last_unstale":"2024-03-23T17:48:57.359512+0000","last_undegraded":"2024-03-23T17:48:57.359512+0000","last_fullsized":"2024-03-23T17:48:57.359512+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:20:35.190781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.215711+0000","last_change":"2024-03-23T17:48:55.717530+0000","last_active":"2024-03-23T17:48:57.215711+0000","last_peered":"2024-03-23T17:48:57.215711+0000","last_clean":"2024-03-23T17:48:57.215711+0000","last_became_active":"2024-03-23T17:48:55.717213+0000","last_became_peered":"2024-03-23T17:48:55.717213+0000","last_unstale":"2024-03-23T17:48:57.215711+0000","last_undegraded":"2024-03-23T17:48:57.215711+0000","last_fullsized":"2024-03-23T17:48:57.215711+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:40:09.803021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.260518+0000","last_change":"2024-03-23T17:48:56.222237+0000","last_active":"2024-03-23T17:48:57.260518+0000","last_peered":"2024-03-23T17:48:57.260518+0000","last_clean":"2024-03-23T17:48:57.260518+0000","last_became_active":"2024-03-23T17:48:56.221916+0000","last_became_peered":"2024-03-23T17:48:56.221916+0000","last_unstale":"2024-03-23T17:48:57.260518+0000","last_undegraded":"2024-03-23T17:48:57.260518+0000","last_fullsized":"2024-03-23T17:48:57.260518+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:35:59.213482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.259186+0000","last_change":"2024-03-23T17:48:55.718143+0000","last_active":"2024-03-23T17:48:57.259186+0000","last_peered":"2024-03-23T17:48:57.259186+0000","last_clean":"2024-03-23T17:48:57.259186+0000","last_became_active":"2024-03-23T17:48:55.717834+0000","last_became_peered":"2024-03-23T17:48:55.717834+0000","last_unstale":"2024-03-23T17:48:57.259186+0000","last_undegraded":"2024-03-23T17:48:57.259186+0000","last_fullsized":"2024-03-23T17:48:57.259186+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:29:08.879636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.259713+0000","last_change":"2024-03-23T17:48:56.237136+0000","last_active":"2024-03-23T17:48:57.259713+0000","last_peered":"2024-03-23T17:48:57.259713+0000","last_clean":"2024-03-23T17:48:57.259713+0000","last_became_active":"2024-03-23T17:48:56.236795+0000","last_became_peered":"2024-03-23T17:48:56.236795+0000","last_unstale":"2024-03-23T17:48:57.259713+0000","last_undegraded":"2024-03-23T17:48:57.259713+0000","last_fullsized":"2024-03-23T17:48:57.259713+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:15:31.356382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.214259+0000","last_change":"2024-03-23T17:48:56.224772+0000","last_active":"2024-03-23T17:48:57.214259+0000","last_peered":"2024-03-23T17:48:57.214259+0000","last_clean":"2024-03-23T17:48:57.214259+0000","last_became_active":"2024-03-23T17:48:56.223685+0000","last_became_peered":"2024-03-23T17:48:56.223685+0000","last_unstale":"2024-03-23T17:48:57.214259+0000","last_undegraded":"2024-03-23T17:48:57.214259+0000","last_fullsized":"2024-03-23T17:48:57.214259+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:07:53.765749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.259768+0000","last_change":"2024-03-23T17:48:55.722968+0000","last_active":"2024-03-23T17:48:57.259768+0000","last_peered":"2024-03-23T17:48:57.259768+0000","last_clean":"2024-03-23T17:48:57.259768+0000","last_became_active":"2024-03-23T17:48:55.722578+0000","last_became_peered":"2024-03-23T17:48:55.722578+0000","last_unstale":"2024-03-23T17:48:57.259768+0000","last_undegraded":"2024-03-23T17:48:57.259768+0000","last_fullsized":"2024-03-23T17:48:57.259768+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:45:42.344283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-23T17:48:56.984393+0000","last_change":"2024-03-23T17:48:56.228000+0000","last_active":"2024-03-23T17:48:56.984393+0000","last_peered":"2024-03-23T17:48:56.984393+0000","last_clean":"2024-03-23T17:48:56.984393+0000","last_became_active":"2024-03-23T17:48:56.225645+0000","last_became_peered":"2024-03-23T17:48:56.225645+0000","last_unstale":"2024-03-23T17:48:56.984393+0000","last_undegraded":"2024-03-23T17:48:56.984393+0000","last_fullsized":"2024-03-23T17:48:56.984393+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:44:25.197761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'2","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:59.672875+0000","last_change":"2024-03-23T17:48:55.718307+0000","last_active":"2024-03-23T17:48:59.672875+0000","last_peered":"2024-03-23T17:48:59.672875+0000","last_clean":"2024-03-23T17:48:59.672875+0000","last_became_active":"2024-03-23T17:48:55.717839+0000","last_became_peered":"2024-03-23T17:48:55.717839+0000","last_unstale":"2024-03-23T17:48:59.672875+0000","last_undegraded":"2024-03-23T17:48:59.672875+0000","last_fullsized":"2024-03-23T17:48:59.672875+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:11:56.053940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":332,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-23T17:48:56.218676+0000","last_change":"2024-03-23T17:48:01.618019+0000","last_active":"2024-03-23T17:48:56.218676+0000","last_peered":"2024-03-23T17:48:56.218676+0000","last_clean":"2024-03-23T17:48:56.218676+0000","last_became_active":"2024-03-23T17:48:01.617553+0000","last_became_peered":"2024-03-23T17:48:01.617553+0000","last_unstale":"2024-03-23T17:48:56.218676+0000","last_undegraded":"2024-03-23T17:48:56.218676+0000","last_fullsized":"2024-03-23T17:48:56.218676+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:45:26.849029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:45:26.849029+0000","last_clean_scrub_stamp":"2024-03-23T17:45:26.849029+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:05:44.823364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.259679+0000","last_change":"2024-03-23T17:48:56.224285+0000","last_active":"2024-03-23T17:48:57.259679+0000","last_peered":"2024-03-23T17:48:57.259679+0000","last_clean":"2024-03-23T17:48:57.259679+0000","last_became_active":"2024-03-23T17:48:56.220943+0000","last_became_peered":"2024-03-23T17:48:56.220943+0000","last_unstale":"2024-03-23T17:48:57.259679+0000","last_undegraded":"2024-03-23T17:48:57.259679+0000","last_fullsized":"2024-03-23T17:48:57.259679+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:36:05.458243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:49:00.661819+0000","last_change":"2024-03-23T17:48:55.715877+0000","last_active":"2024-03-23T17:49:00.661819+0000","last_peered":"2024-03-23T17:49:00.661819+0000","last_clean":"2024-03-23T17:49:00.661819+0000","last_became_active":"2024-03-23T17:48:55.715550+0000","last_became_peered":"2024-03-23T17:48:55.715550+0000","last_unstale":"2024-03-23T17:49:00.661819+0000","last_undegraded":"2024-03-23T17:49:00.661819+0000","last_fullsized":"2024-03-23T17:49:00.661819+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:31:57.668495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:49:01.685625+0000","last_change":"2024-03-23T17:48:55.724618+0000","last_active":"2024-03-23T17:49:01.685625+0000","last_peered":"2024-03-23T17:49:01.685625+0000","last_clean":"2024-03-23T17:49:01.685625+0000","last_became_active":"2024-03-23T17:48:55.724278+0000","last_became_peered":"2024-03-23T17:48:55.724278+0000","last_unstale":"2024-03-23T17:49:01.685625+0000","last_undegraded":"2024-03-23T17:49:01.685625+0000","last_fullsized":"2024-03-23T17:49:01.685625+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:27:05.065115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.260132+0000","last_change":"2024-03-23T17:48:56.216141+0000","last_active":"2024-03-23T17:48:57.260132+0000","last_peered":"2024-03-23T17:48:57.260132+0000","last_clean":"2024-03-23T17:48:57.260132+0000","last_became_active":"2024-03-23T17:48:56.215653+0000","last_became_peered":"2024-03-23T17:48:56.215653+0000","last_unstale":"2024-03-23T17:48:57.260132+0000","last_undegraded":"2024-03-23T17:48:57.260132+0000","last_fullsized":"2024-03-23T17:48:57.260132+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:02:19.611056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'7","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:49:01.686750+0000","last_change":"2024-03-23T17:48:55.727425+0000","last_active":"2024-03-23T17:49:01.686750+0000","last_peered":"2024-03-23T17:49:01.686750+0000","last_clean":"2024-03-23T17:49:01.686750+0000","last_became_active":"2024-03-23T17:48:55.727071+0000","last_became_peered":"2024-03-23T17:48:55.727071+0000","last_unstale":"2024-03-23T17:49:01.686750+0000","last_undegraded":"2024-03-23T17:49:01.686750+0000","last_fullsized":"2024-03-23T17:49:01.686750+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:30:31.885441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-23T17:48:56.984444+0000","last_change":"2024-03-23T17:48:56.228439+0000","last_active":"2024-03-23T17:48:56.984444+0000","last_peered":"2024-03-23T17:48:56.984444+0000","last_clean":"2024-03-23T17:48:56.984444+0000","last_became_active":"2024-03-23T17:48:56.226677+0000","last_became_peered":"2024-03-23T17:48:56.226677+0000","last_unstale":"2024-03-23T17:48:56.984444+0000","last_undegraded":"2024-03-23T17:48:56.984444+0000","last_fullsized":"2024-03-23T17:48:56.984444+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:46:00.063266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'4","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:59.671887+0000","last_change":"2024-03-23T17:48:55.724872+0000","last_active":"2024-03-23T17:48:59.671887+0000","last_peered":"2024-03-23T17:48:59.671887+0000","last_clean":"2024-03-23T17:48:59.671887+0000","last_became_active":"2024-03-23T17:48:55.724468+0000","last_became_peered":"2024-03-23T17:48:55.724468+0000","last_unstale":"2024-03-23T17:48:59.671887+0000","last_undegraded":"2024-03-23T17:48:59.671887+0000","last_fullsized":"2024-03-23T17:48:59.671887+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:28:34.918056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.260034+0000","last_change":"2024-03-23T17:48:56.233730+0000","last_active":"2024-03-23T17:48:57.260034+0000","last_peered":"2024-03-23T17:48:57.260034+0000","last_clean":"2024-03-23T17:48:57.260034+0000","last_became_active":"2024-03-23T17:48:56.233189+0000","last_became_peered":"2024-03-23T17:48:56.233189+0000","last_unstale":"2024-03-23T17:48:57.260034+0000","last_undegraded":"2024-03-23T17:48:57.260034+0000","last_fullsized":"2024-03-23T17:48:57.260034+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:20:30.764344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:59.673121+0000","last_change":"2024-03-23T17:48:55.724633+0000","last_active":"2024-03-23T17:48:59.673121+0000","last_peered":"2024-03-23T17:48:59.673121+0000","last_clean":"2024-03-23T17:48:59.673121+0000","last_became_active":"2024-03-23T17:48:55.724301+0000","last_became_peered":"2024-03-23T17:48:55.724301+0000","last_unstale":"2024-03-23T17:48:59.673121+0000","last_undegraded":"2024-03-23T17:48:59.673121+0000","last_fullsized":"2024-03-23T17:48:59.673121+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:45:08.392668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.258996+0000","last_change":"2024-03-23T17:48:56.230981+0000","last_active":"2024-03-23T17:48:57.258996+0000","last_peered":"2024-03-23T17:48:57.258996+0000","last_clean":"2024-03-23T17:48:57.258996+0000","last_became_active":"2024-03-23T17:48:56.230644+0000","last_became_peered":"2024-03-23T17:48:56.230644+0000","last_unstale":"2024-03-23T17:48:57.258996+0000","last_undegraded":"2024-03-23T17:48:57.258996+0000","last_fullsized":"2024-03-23T17:48:57.258996+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:37:51.310938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'5","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:49:01.685039+0000","last_change":"2024-03-23T17:48:55.719112+0000","last_active":"2024-03-23T17:49:01.685039+0000","last_peered":"2024-03-23T17:49:01.685039+0000","last_clean":"2024-03-23T17:49:01.685039+0000","last_became_active":"2024-03-23T17:48:55.718756+0000","last_became_peered":"2024-03-23T17:48:55.718756+0000","last_unstale":"2024-03-23T17:49:01.685039+0000","last_undegraded":"2024-03-23T17:49:01.685039+0000","last_fullsized":"2024-03-23T17:49:01.685039+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:41:22.164706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.359316+0000","last_change":"2024-03-23T17:48:56.240289+0000","last_active":"2024-03-23T17:48:57.359316+0000","last_peered":"2024-03-23T17:48:57.359316+0000","last_clean":"2024-03-23T17:48:57.359316+0000","last_became_active":"2024-03-23T17:48:56.239896+0000","last_became_peered":"2024-03-23T17:48:56.239896+0000","last_unstale":"2024-03-23T17:48:57.359316+0000","last_undegraded":"2024-03-23T17:48:57.359316+0000","last_fullsized":"2024-03-23T17:48:57.359316+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:49:10.505576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-23T17:48:56.212920+0000","last_change":"2024-03-23T17:48:55.723330+0000","last_active":"2024-03-23T17:48:56.212920+0000","last_peered":"2024-03-23T17:48:56.212920+0000","last_clean":"2024-03-23T17:48:56.212920+0000","last_became_active":"2024-03-23T17:48:55.722879+0000","last_became_peered":"2024-03-23T17:48:55.722879+0000","last_unstale":"2024-03-23T17:48:56.212920+0000","last_undegraded":"2024-03-23T17:48:56.212920+0000","last_fullsized":"2024-03-23T17:48:56.212920+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:34:47.173411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-23T17:48:56.984403+0000","last_change":"2024-03-23T17:48:56.237042+0000","last_active":"2024-03-23T17:48:56.984403+0000","last_peered":"2024-03-23T17:48:56.984403+0000","last_clean":"2024-03-23T17:48:56.984403+0000","last_became_active":"2024-03-23T17:48:56.236722+0000","last_became_peered":"2024-03-23T17:48:56.236722+0000","last_unstale":"2024-03-23T17:48:56.984403+0000","last_undegraded":"2024-03-23T17:48:56.984403+0000","last_fullsized":"2024-03-23T17:48:56.984403+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:27:29.656120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:59.672459+0000","last_change":"2024-03-23T17:48:55.719223+0000","last_active":"2024-03-23T17:48:59.672459+0000","last_peered":"2024-03-23T17:48:59.672459+0000","last_clean":"2024-03-23T17:48:59.672459+0000","last_became_active":"2024-03-23T17:48:55.718864+0000","last_became_peered":"2024-03-23T17:48:55.718864+0000","last_unstale":"2024-03-23T17:48:59.672459+0000","last_undegraded":"2024-03-23T17:48:59.672459+0000","last_fullsized":"2024-03-23T17:48:59.672459+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:52:18.169557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-23T17:48:56.370959+0000","last_change":"2024-03-23T17:48:56.224885+0000","last_active":"2024-03-23T17:48:56.370959+0000","last_peered":"2024-03-23T17:48:56.370959+0000","last_clean":"2024-03-23T17:48:56.370959+0000","last_became_active":"2024-03-23T17:48:56.224160+0000","last_became_peered":"2024-03-23T17:48:56.224160+0000","last_unstale":"2024-03-23T17:48:56.370959+0000","last_undegraded":"2024-03-23T17:48:56.370959+0000","last_fullsized":"2024-03-23T17:48:56.370959+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:35:34.538313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-23T17:48:56.214095+0000","last_change":"2024-03-23T17:48:55.720518+0000","last_active":"2024-03-23T17:48:56.214095+0000","last_peered":"2024-03-23T17:48:56.214095+0000","last_clean":"2024-03-23T17:48:56.214095+0000","last_became_active":"2024-03-23T17:48:55.720045+0000","last_became_peered":"2024-03-23T17:48:55.720045+0000","last_unstale":"2024-03-23T17:48:56.214095+0000","last_undegraded":"2024-03-23T17:48:56.214095+0000","last_fullsized":"2024-03-23T17:48:56.214095+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:20:54.819406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-23T17:48:56.371015+0000","last_change":"2024-03-23T17:48:56.224171+0000","last_active":"2024-03-23T17:48:56.371015+0000","last_peered":"2024-03-23T17:48:56.371015+0000","last_clean":"2024-03-23T17:48:56.371015+0000","last_became_active":"2024-03-23T17:48:56.223737+0000","last_became_peered":"2024-03-23T17:48:56.223737+0000","last_unstale":"2024-03-23T17:48:56.371015+0000","last_undegraded":"2024-03-23T17:48:56.371015+0000","last_fullsized":"2024-03-23T17:48:56.371015+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:40:04.353945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'2","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:49:00.653260+0000","last_change":"2024-03-23T17:48:55.721734+0000","last_active":"2024-03-23T17:49:00.653260+0000","last_peered":"2024-03-23T17:49:00.653260+0000","last_clean":"2024-03-23T17:49:00.653260+0000","last_became_active":"2024-03-23T17:48:55.721417+0000","last_became_peered":"2024-03-23T17:48:55.721417+0000","last_unstale":"2024-03-23T17:49:00.653260+0000","last_undegraded":"2024-03-23T17:49:00.653260+0000","last_fullsized":"2024-03-23T17:49:00.653260+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:36:36.842740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-23T17:48:56.370980+0000","last_change":"2024-03-23T17:48:56.227211+0000","last_active":"2024-03-23T17:48:56.370980+0000","last_peered":"2024-03-23T17:48:56.370980+0000","last_clean":"2024-03-23T17:48:56.370980+0000","last_became_active":"2024-03-23T17:48:56.226881+0000","last_became_peered":"2024-03-23T17:48:56.226881+0000","last_unstale":"2024-03-23T17:48:56.370980+0000","last_undegraded":"2024-03-23T17:48:56.370980+0000","last_fullsized":"2024-03-23T17:48:56.370980+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:15:02.937666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.260341+0000","last_change":"2024-03-23T17:48:55.728679+0000","last_active":"2024-03-23T17:48:57.260341+0000","last_peered":"2024-03-23T17:48:57.260341+0000","last_clean":"2024-03-23T17:48:57.260341+0000","last_became_active":"2024-03-23T17:48:55.727754+0000","last_became_peered":"2024-03-23T17:48:55.727754+0000","last_unstale":"2024-03-23T17:48:57.260341+0000","last_undegraded":"2024-03-23T17:48:57.260341+0000","last_fullsized":"2024-03-23T17:48:57.260341+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:34:39.720488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-23T17:48:56.371037+0000","last_change":"2024-03-23T17:48:56.225214+0000","last_active":"2024-03-23T17:48:56.371037+0000","last_peered":"2024-03-23T17:48:56.371037+0000","last_clean":"2024-03-23T17:48:56.371037+0000","last_became_active":"2024-03-23T17:48:56.224635+0000","last_became_peered":"2024-03-23T17:48:56.224635+0000","last_unstale":"2024-03-23T17:48:56.371037+0000","last_undegraded":"2024-03-23T17:48:56.371037+0000","last_fullsized":"2024-03-23T17:48:56.371037+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:57:14.935434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.359428+0000","last_change":"2024-03-23T17:48:56.229835+0000","last_active":"2024-03-23T17:48:57.359428+0000","last_peered":"2024-03-23T17:48:57.359428+0000","last_clean":"2024-03-23T17:48:57.359428+0000","last_became_active":"2024-03-23T17:48:56.229437+0000","last_became_peered":"2024-03-23T17:48:56.229437+0000","last_unstale":"2024-03-23T17:48:57.359428+0000","last_undegraded":"2024-03-23T17:48:57.359428+0000","last_fullsized":"2024-03-23T17:48:57.359428+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:43:51.544867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.259118+0000","last_change":"2024-03-23T17:48:56.233552+0000","last_active":"2024-03-23T17:48:57.259118+0000","last_peered":"2024-03-23T17:48:57.259118+0000","last_clean":"2024-03-23T17:48:57.259118+0000","last_became_active":"2024-03-23T17:48:56.232941+0000","last_became_peered":"2024-03-23T17:48:56.232941+0000","last_unstale":"2024-03-23T17:48:57.259118+0000","last_undegraded":"2024-03-23T17:48:57.259118+0000","last_fullsized":"2024-03-23T17:48:57.259118+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:03:32.325496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.259155+0000","last_change":"2024-03-23T17:48:56.238238+0000","last_active":"2024-03-23T17:48:57.259155+0000","last_peered":"2024-03-23T17:48:57.259155+0000","last_clean":"2024-03-23T17:48:57.259155+0000","last_became_active":"2024-03-23T17:48:56.237882+0000","last_became_peered":"2024-03-23T17:48:56.237882+0000","last_unstale":"2024-03-23T17:48:57.259155+0000","last_undegraded":"2024-03-23T17:48:57.259155+0000","last_fullsized":"2024-03-23T17:48:57.259155+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:27:13.896205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.258990+0000","last_change":"2024-03-23T17:48:56.242895+0000","last_active":"2024-03-23T17:48:57.258990+0000","last_peered":"2024-03-23T17:48:57.258990+0000","last_clean":"2024-03-23T17:48:57.258990+0000","last_became_active":"2024-03-23T17:48:56.242515+0000","last_became_peered":"2024-03-23T17:48:56.242515+0000","last_unstale":"2024-03-23T17:48:57.258990+0000","last_undegraded":"2024-03-23T17:48:57.258990+0000","last_fullsized":"2024-03-23T17:48:57.258990+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:45:01.978843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.260154+0000","last_change":"2024-03-23T17:48:56.237029+0000","last_active":"2024-03-23T17:48:57.260154+0000","last_peered":"2024-03-23T17:48:57.260154+0000","last_clean":"2024-03-23T17:48:57.260154+0000","last_became_active":"2024-03-23T17:48:56.236692+0000","last_became_peered":"2024-03-23T17:48:56.236692+0000","last_unstale":"2024-03-23T17:48:57.260154+0000","last_undegraded":"2024-03-23T17:48:57.260154+0000","last_fullsized":"2024-03-23T17:48:57.260154+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:42:31.855469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.359525+0000","last_change":"2024-03-23T17:48:56.223236+0000","last_active":"2024-03-23T17:48:57.359525+0000","last_peered":"2024-03-23T17:48:57.359525+0000","last_clean":"2024-03-23T17:48:57.359525+0000","last_became_active":"2024-03-23T17:48:56.222824+0000","last_became_peered":"2024-03-23T17:48:56.222824+0000","last_unstale":"2024-03-23T17:48:57.359525+0000","last_undegraded":"2024-03-23T17:48:57.359525+0000","last_fullsized":"2024-03-23T17:48:57.359525+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:54:45.096969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.260287+0000","last_change":"2024-03-23T17:48:56.224471+0000","last_active":"2024-03-23T17:48:57.260287+0000","last_peered":"2024-03-23T17:48:57.260287+0000","last_clean":"2024-03-23T17:48:57.260287+0000","last_became_active":"2024-03-23T17:48:56.223140+0000","last_became_peered":"2024-03-23T17:48:56.223140+0000","last_unstale":"2024-03-23T17:48:57.260287+0000","last_undegraded":"2024-03-23T17:48:57.260287+0000","last_fullsized":"2024-03-23T17:48:57.260287+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:38:53.906820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-23T17:48:56.984433+0000","last_change":"2024-03-23T17:48:56.223564+0000","last_active":"2024-03-23T17:48:56.984433+0000","last_peered":"2024-03-23T17:48:56.984433+0000","last_clean":"2024-03-23T17:48:56.984433+0000","last_became_active":"2024-03-23T17:48:56.223217+0000","last_became_peered":"2024-03-23T17:48:56.223217+0000","last_unstale":"2024-03-23T17:48:56.984433+0000","last_undegraded":"2024-03-23T17:48:56.984433+0000","last_fullsized":"2024-03-23T17:48:56.984433+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:43:53.250082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.214184+0000","last_change":"2024-03-23T17:48:56.226283+0000","last_active":"2024-03-23T17:48:57.214184+0000","last_peered":"2024-03-23T17:48:57.214184+0000","last_clean":"2024-03-23T17:48:57.214184+0000","last_became_active":"2024-03-23T17:48:56.224211+0000","last_became_peered":"2024-03-23T17:48:56.224211+0000","last_unstale":"2024-03-23T17:48:57.214184+0000","last_undegraded":"2024-03-23T17:48:57.214184+0000","last_fullsized":"2024-03-23T17:48:57.214184+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:15:20.518950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-23T17:48:56.984368+0000","last_change":"2024-03-23T17:48:56.223557+0000","last_active":"2024-03-23T17:48:56.984368+0000","last_peered":"2024-03-23T17:48:56.984368+0000","last_clean":"2024-03-23T17:48:56.984368+0000","last_became_active":"2024-03-23T17:48:56.222792+0000","last_became_peered":"2024-03-23T17:48:56.222792+0000","last_unstale":"2024-03-23T17:48:56.984368+0000","last_undegraded":"2024-03-23T17:48:56.984368+0000","last_fullsized":"2024-03-23T17:48:56.984368+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:49:17.587817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-23T17:48:56.562286+0000","last_change":"2024-03-23T17:48:56.236041+0000","last_active":"2024-03-23T17:48:56.562286+0000","last_peered":"2024-03-23T17:48:56.562286+0000","last_clean":"2024-03-23T17:48:56.562286+0000","last_became_active":"2024-03-23T17:48:56.234259+0000","last_became_peered":"2024-03-23T17:48:56.234259+0000","last_unstale":"2024-03-23T17:48:56.562286+0000","last_undegraded":"2024-03-23T17:48:56.562286+0000","last_fullsized":"2024-03-23T17:48:56.562286+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:32:43.786783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.260007+0000","last_change":"2024-03-23T17:48:56.239283+0000","last_active":"2024-03-23T17:48:57.260007+0000","last_peered":"2024-03-23T17:48:57.260007+0000","last_clean":"2024-03-23T17:48:57.260007+0000","last_became_active":"2024-03-23T17:48:56.238952+0000","last_became_peered":"2024-03-23T17:48:56.238952+0000","last_unstale":"2024-03-23T17:48:57.260007+0000","last_undegraded":"2024-03-23T17:48:57.260007+0000","last_fullsized":"2024-03-23T17:48:57.260007+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:31:50.383284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-23T17:48:57.359422+0000","last_change":"2024-03-23T17:48:56.226684+0000","last_active":"2024-03-23T17:48:57.359422+0000","last_peered":"2024-03-23T17:48:57.359422+0000","last_clean":"2024-03-23T17:48:57.359422+0000","last_became_active":"2024-03-23T17:48:56.226367+0000","last_became_peered":"2024-03-23T17:48:56.226367+0000","last_unstale":"2024-03-23T17:48:57.359422+0000","last_undegraded":"2024-03-23T17:48:57.359422+0000","last_fullsized":"2024-03-23T17:48:57.359422+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:12:29.053330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5086,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":11,"num_read_kb":9,"num_write":130,"num_write_kb":43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":221184,"data_stored":13452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":113,"ondisk_log_size":113,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547212,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31956,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713196,"statfs":{"total":95995035648,"available":95962312704,"internally_reserved":0,"allocated":864256,"data_stored":411663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31848901},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40699999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]}]},{"osd":10,"up_from":68,"seq":292057776142,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28452,"kb_used_data":1432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716700,"statfs":{"total":95995035648,"available":95965900800,"internally_reserved":0,"allocated":1466368,"data_stored":1002314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66500000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85099999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83499999999999996}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86799999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.877}]}]},{"osd":9,"up_from":63,"seq":270582939668,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36060,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709092,"statfs":{"total":95995035648,"available":95958110208,"internally_reserved":0,"allocated":872448,"data_stored":411337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":36043201},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 17:48:42 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.39700000000000002}]},{"osd":1,"last update":"Sat Mar 23 17:48:42 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.68500000000000005}]},{"osd":2,"last update":"Sat Mar 23 17:48:42 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.55600000000000005}]},{"osd":3,"last update":"Sat Mar 23 17:48:42 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.44900000000000001}]},{"osd":4,"last update":"Sat Mar 23 17:48:42 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.69799999999999995}]},{"osd":5,"last update":"Sat Mar 23 17:48:42 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.48,"5min":1.48,"15min":1.48},"last":0.505}]},{"osd":6,"last update":"Sat Mar 23 17:48:42 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.49099999999999999}]},{"osd":7,"last update":"Sat Mar 23 17:48:42 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.373,"5min":1.373,"15min":1.373},"last":0.66800000000000004}]},{"osd":8,"last update":"Sat Mar 23 17:48:42 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.63900000000000001}]},{"osd":10,"last update":"Sat Mar 23 17:48:58 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.70499999999999996}]}]},{"osd":8,"up_from":56,"seq":240518168598,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":884736,"data_stored":412019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 17:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.502}]},{"osd":1,"last update":"Sat Mar 23 17:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.64800000000000002}]},{"osd":2,"last update":"Sat Mar 23 17:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.46999999999999997}]},{"osd":3,"last update":"Sat Mar 23 17:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.56699999999999995}]},{"osd":4,"last update":"Sat Mar 23 17:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.62}]},{"osd":5,"last update":"Sat Mar 23 17:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.53800000000000003}]},{"osd":6,"last update":"Sat Mar 23 17:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.73399999999999999}]},{"osd":7,"last update":"Sat Mar 23 17:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.66000000000000003}]},{"osd":9,"last update":"Sat Mar 23 17:48:42 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.71699999999999997}]},{"osd":10,"last update":"Sat Mar 23 17:49:01 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.63600000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77200000000000002}]}]},{"osd":7,"up_from":49,"seq":210453397530,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28444,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716708,"statfs":{"total":95995035648,"available":95965908992,"internally_reserved":0,"allocated":1462272,"data_stored":1002583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 17:48:02 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.54800000000000004}]},{"osd":1,"last update":"Sat Mar 23 17:48:02 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.371,"5min":0.371,"15min":0.371},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"last":0.46000000000000002}]},{"osd":2,"last update":"Sat Mar 23 17:48:02 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.60799999999999998}]},{"osd":3,"last update":"Sat Mar 23 17:48:02 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.65500000000000003}]},{"osd":4,"last update":"Sat Mar 23 17:48:02 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.51000000000000001}]},{"osd":5,"last update":"Sat Mar 23 17:48:02 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.76800000000000002}]},{"osd":6,"last update":"Sat Mar 23 17:48:02 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.751},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.71899999999999997}]},{"osd":8,"last update":"Sat Mar 23 17:48:21 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.78200000000000003}]},{"osd":9,"last update":"Sat Mar 23 17:48:42 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.81100000000000005}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82099999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84799999999999998}]}]},{"osd":6,"up_from":45,"seq":193273528350,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36048,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93709104,"statfs":{"total":95995035648,"available":95958122496,"internally_reserved":0,"allocated":847872,"data_stored":410246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 17:48:45 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.47499999999999998,"15min":0.47599999999999998},"min":{"1min":0.35599999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.52800000000000002,"15min":0.52400000000000002},"min":{"1min":0.33700000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.41699999999999998}]},{"osd":1,"last update":"Sat Mar 23 17:48:45 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.64400000000000002,"15min":0.64500000000000002},"min":{"1min":0.48399999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.91800000000000004,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.63300000000000001}]},{"osd":2,"last update":"Sat Mar 23 17:48:45 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.67900000000000005,"15min":0.68200000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.67400000000000004,"15min":0.67800000000000005},"min":{"1min":0.47099999999999997,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.91900000000000004,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.65200000000000002}]},{"osd":3,"last update":"Sat Mar 23 17:48:45 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.61399999999999999,"15min":0.61899999999999999},"min":{"1min":0.34399999999999997,"5min":0.316,"15min":0.316},"max":{"1min":0.89700000000000002,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.66400000000000003,"15min":0.66600000000000004},"min":{"1min":0.45600000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.88300000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.50700000000000001}]},{"osd":4,"last update":"Sat Mar 23 17:48:45 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.65700000000000003,"15min":0.65800000000000003},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.72999999999999998,"15min":0.73699999999999999},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.86399999999999999,"5min":0.998,"15min":0.998},"last":0.68400000000000005}]},{"osd":5,"last update":"Sat Mar 23 17:48:45 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.77900000000000003,"15min":0.78300000000000003},"min":{"1min":0.57199999999999995,"5min":0.436,"15min":0.436},"max":{"1min":0.90600000000000003,"5min":1.375,"15min":1.375},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.75,"5min":0.78100000000000003,"15min":0.78600000000000003},"min":{"1min":0.59199999999999997,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.67000000000000004}]},{"osd":7,"last update":"Sat Mar 23 17:48:03 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.78200000000000003}]},{"osd":8,"last update":"Sat Mar 23 17:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.747}]},{"osd":9,"last update":"Sat Mar 23 17:48:45 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.52900000000000003}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574896,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":864256,"data_stored":412083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 17:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.52400000000000002,"15min":0.51200000000000001},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.79200000000000004,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.502},{"interface":"front","average":{"1min":1.0680000000000001,"5min":0.61599999999999999,"15min":0.52800000000000002},"min":{"1min":0.38,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":5.8399999999999999,"5min":5.8399999999999999,"15min":5.8399999999999999},"last":0.52900000000000003}]},{"osd":2,"last update":"Sat Mar 23 17:48:28 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.48699999999999999,"15min":0.46800000000000003},"min":{"1min":0.40200000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.49399999999999999,"15min":0.48399999999999999},"min":{"1min":0.34499999999999997,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.76200000000000001,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.77400000000000002}]},{"osd":3,"last update":"Sat Mar 23 17:48:51 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.55000000000000004,"15min":0.53200000000000003},"min":{"1min":0.377,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.55100000000000005,"15min":0.53900000000000003},"min":{"1min":0.39100000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.66600000000000004}]},{"osd":4,"last update":"Sat Mar 23 17:48:08 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.40200000000000002,"15min":0.39200000000000002},"min":{"1min":0.30199999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.432,"5min":0.39300000000000002,"15min":0.38700000000000001},"min":{"1min":0.29199999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.64200000000000002}]},{"osd":5,"last update":"Sat Mar 23 17:48:28 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.55600000000000005,"15min":0.55000000000000004},"min":{"1min":0.35999999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.57599999999999996,"15min":0.56599999999999995},"min":{"1min":0.436,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.82399999999999995}]},{"osd":6,"last update":"Sat Mar 23 17:48:45 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.58799999999999997,"15min":0.56899999999999995},"min":{"1min":0.52100000000000002,"5min":0.433,"15min":0.433},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.63600000000000001,"15min":0.621},"min":{"1min":0.47399999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.78800000000000003}]},{"osd":7,"last update":"Sat Mar 23 17:48:02 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.84099999999999997}]},{"osd":8,"last update":"Sat Mar 23 17:48:20 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.80500000000000005}]},{"osd":9,"last update":"Sat Mar 23 17:48:45 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.90300000000000002}]},{"osd":10,"last update":"Sat Mar 23 17:48:59 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.877}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95099999999999996}]}]},{"osd":0,"up_from":8,"seq":34359738421,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":851968,"data_stored":410336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 23 17:48:14 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.68799999999999994,"15min":0.73799999999999999},"min":{"1min":0.45400000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":1.101,"5min":6.2110000000000003,"15min":6.2110000000000003},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.52800000000000002,"15min":0.502},"min":{"1min":0.41899999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.84499999999999997}]},{"osd":2,"last update":"Sat Mar 23 17:48:29 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.56399999999999995,"15min":0.54600000000000004},"min":{"1min":0.47099999999999997,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.57699999999999996,"15min":0.55200000000000005},"min":{"1min":0.52700000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.68700000000000006}]},{"osd":3,"last update":"Sat Mar 23 17:47:52 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.498,"15min":0.49399999999999999},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.78000000000000003,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.34499999999999997,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.79100000000000004,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.63400000000000001}]},{"osd":4,"last update":"Sat Mar 23 17:48:14 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.629,"15min":0.61899999999999999},"min":{"1min":0.48499999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.91300000000000003,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.59299999999999997,"15min":0.58599999999999997},"min":{"1min":0.46899999999999997,"5min":0.438,"15min":0.438},"max":{"1min":0.89500000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.77700000000000002}]},{"osd":5,"last update":"Sat Mar 23 17:48:24 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.65200000000000002,"15min":0.63600000000000001},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.629,"15min":0.61099999999999999},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.73299999999999998}]},{"osd":6,"last update":"Sat Mar 23 17:48:47 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.60699999999999998,"15min":0.60099999999999998},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.67900000000000005,"15min":0.66800000000000004},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.85699999999999998}]},{"osd":7,"last update":"Sat Mar 23 17:48:02 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.495},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.58599999999999997}]},{"osd":8,"last update":"Sat Mar 23 17:48:21 2024","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.89000000000000001}]},{"osd":9,"last update":"Sat Mar 23 17:48:42 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":0.91100000000000003}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95499999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92600000000000005}]}]},{"osd":2,"up_from":18,"seq":77309411373,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36060,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709092,"statfs":{"total":95995035648,"available":95958110208,"internally_reserved":0,"allocated":872448,"data_stored":412418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 17:48:28 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.51700000000000002,"15min":0.50600000000000001},"min":{"1min":0.435,"5min":0.37,"15min":0.37},"max":{"1min":0.73599999999999999,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.56100000000000005,"15min":0.54700000000000004},"min":{"1min":0.434,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.81299999999999994,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.67400000000000004}]},{"osd":1,"last update":"Sat Mar 23 17:48:28 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.51500000000000001,"15min":0.48999999999999999},"min":{"1min":0.33400000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.85899999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.53000000000000003,"15min":0.50800000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.83999999999999997,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.57399999999999995}]},{"osd":3,"last update":"Sat Mar 23 17:48:54 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.54400000000000004,"15min":0.52800000000000002},"min":{"1min":0.36699999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.58299999999999996,"15min":0.56499999999999995},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.69899999999999995}]},{"osd":4,"last update":"Sat Mar 23 17:48:06 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.51900000000000002,"15min":0.51100000000000001},"min":{"1min":0.41099999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.628,"5min":0.58199999999999996,"15min":0.57399999999999995},"min":{"1min":0.42599999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.44400000000000001}]},{"osd":5,"last update":"Sat Mar 23 17:48:24 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.46100000000000002,"15min":0.45100000000000001},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.504,"5min":0.46700000000000003,"15min":0.46100000000000002},"min":{"1min":0.314,"5min":0.308,"15min":0.308},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.46999999999999997}]},{"osd":6,"last update":"Sat Mar 23 17:48:48 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.623,"15min":0.61699999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.64400000000000002,"15min":0.63800000000000001},"min":{"1min":0.46100000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.73199999999999998}]},{"osd":7,"last update":"Sat Mar 23 17:48:01 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.745}]},{"osd":8,"last update":"Sat Mar 23 17:48:21 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.64900000000000002}]},{"osd":9,"last update":"Sat Mar 23 17:48:43 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.746},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.80600000000000005}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]}]},{"osd":3,"up_from":26,"seq":111669149737,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31956,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713196,"statfs":{"total":95995035648,"available":95962312704,"internally_reserved":0,"allocated":864256,"data_stored":410530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 17:48:58 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.41299999999999998,"15min":0.39300000000000002},"min":{"1min":0.34599999999999997,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.499,"5min":0.42499999999999999,"15min":0.40000000000000002},"min":{"1min":0.33500000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.94899999999999995,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.36799999999999999}]},{"osd":1,"last update":"Sat Mar 23 17:48:58 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.54200000000000004,"15min":0.51500000000000001},"min":{"1min":0.379,"5min":0.371,"15min":0.371},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.54100000000000004,"15min":0.50700000000000001},"min":{"1min":0.40699999999999997,"5min":0.371,"15min":0.371},"max":{"1min":0.91500000000000004,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.81299999999999994}]},{"osd":2,"last update":"Sat Mar 23 17:48:58 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.60999999999999999,"15min":0.58299999999999996},"min":{"1min":0.496,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.626,"15min":0.61199999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.90500000000000003,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.69199999999999995}]},{"osd":4,"last update":"Sat Mar 23 17:48:08 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.61499999999999999,"15min":0.60299999999999998},"min":{"1min":0.46899999999999997,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.61899999999999999,"15min":0.60699999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.86399999999999999}]},{"osd":5,"last update":"Sat Mar 23 17:48:26 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.66200000000000003,"15min":0.64400000000000002},"min":{"1min":0.52900000000000003,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.68400000000000005,"15min":0.66700000000000004},"min":{"1min":0.64400000000000002,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.78900000000000003}]},{"osd":6,"last update":"Sat Mar 23 17:48:45 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.64800000000000002,"15min":0.64400000000000002},"min":{"1min":0.504,"5min":0.38,"15min":0.38},"max":{"1min":0.88600000000000001,"5min":1.024,"15min":1.024},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.66900000000000004,"15min":0.66400000000000003},"min":{"1min":0.48099999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.93400000000000005,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.51300000000000001}]},{"osd":7,"last update":"Sat Mar 23 17:48:04 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.52900000000000003}]},{"osd":8,"last update":"Sat Mar 23 17:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.88},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.85099999999999998}]},{"osd":9,"last update":"Sat Mar 23 17:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":1.046}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]}]},{"osd":4,"up_from":33,"seq":141733920806,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":851968,"data_stored":410268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 17:48:06 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.53700000000000003,"15min":0.53400000000000003},"min":{"1min":0.41799999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.77600000000000002,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.53700000000000003,"15min":0.52500000000000002},"min":{"1min":0.45300000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.52000000000000002}]},{"osd":1,"last update":"Sat Mar 23 17:48:06 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.33300000000000002,"15min":0.31900000000000001},"min":{"1min":0.27700000000000002,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.436,"5min":0.35199999999999998,"15min":0.33800000000000002},"min":{"1min":0.29399999999999998,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.33700000000000002}]},{"osd":2,"last update":"Sat Mar 23 17:48:06 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.56200000000000006,"15min":0.55300000000000005},"min":{"1min":0.439,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.84899999999999998,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.55500000000000005,"15min":0.54200000000000004},"min":{"1min":0.42099999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.85299999999999998,"5min":0.88,"15min":0.88},"last":0.47599999999999998}]},{"osd":3,"last update":"Sat Mar 23 17:48:06 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.56599999999999995,"15min":0.55800000000000005},"min":{"1min":0.40799999999999997,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84099999999999997,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.57199999999999995,"15min":0.56399999999999995},"min":{"1min":0.46800000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.89300000000000002,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.60599999999999998}]},{"osd":5,"last update":"Sat Mar 23 17:48:28 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.56599999999999995,"15min":0.56100000000000005},"min":{"1min":0.42799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.81799999999999995,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.58299999999999996,"15min":0.57399999999999995},"min":{"1min":0.46000000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.82299999999999995,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.64500000000000002}]},{"osd":6,"last update":"Sat Mar 23 17:48:53 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.47699999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.78200000000000003,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.58999999999999997,"15min":0.58899999999999997},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.55900000000000005}]},{"osd":7,"last update":"Sat Mar 23 17:48:01 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.45100000000000001}]},{"osd":8,"last update":"Sat Mar 23 17:48:22 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.68200000000000005}]},{"osd":9,"last update":"Sat Mar 23 17:48:42 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.73699999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71099999999999997}]}]},{"osd":5,"up_from":38,"seq":163208757282,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32560,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712592,"statfs":{"total":95995035648,"available":95961694208,"internally_reserved":0,"allocated":1482752,"data_stored":1002746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 17:48:25 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.53900000000000003,"15min":0.53800000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.54400000000000004,"15min":0.54800000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.751}]},{"osd":1,"last update":"Sat Mar 23 17:48:25 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.52500000000000002,"15min":0.51900000000000002},"min":{"1min":0.38,"5min":0.37,"15min":0.37},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.63},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.55700000000000005,"15min":0.55500000000000005},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.66300000000000003,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.73199999999999998}]},{"osd":2,"last update":"Sat Mar 23 17:48:25 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.39100000000000001,"15min":0.39000000000000001},"min":{"1min":0.29999999999999999,"5min":0.251,"15min":0.251},"max":{"1min":0.52800000000000002,"5min":0.62,"15min":0.62},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.39600000000000002,"15min":0.39400000000000002},"min":{"1min":0.315,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.55300000000000005,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.438}]},{"osd":3,"last update":"Sat Mar 23 17:48:25 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.58299999999999996,"15min":0.58499999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.57499999999999996,"15min":0.58499999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.70399999999999996,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.497}]},{"osd":4,"last update":"Sat Mar 23 17:48:25 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.57999999999999996,"15min":0.57299999999999995},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.55600000000000005,"15min":0.55100000000000005},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.83099999999999996}]},{"osd":6,"last update":"Sat Mar 23 17:48:48 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.627,"15min":0.625},"min":{"1min":0.53400000000000003,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.88500000000000001,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.441,"5min":0.434,"15min":0.434},"max":{"1min":0.79600000000000004,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.58799999999999997}]},{"osd":7,"last update":"Sat Mar 23 17:48:02 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.79700000000000004}]},{"osd":8,"last update":"Sat Mar 23 17:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.53600000000000003}]},{"osd":9,"last update":"Sat Mar 23 17:48:43 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.65600000000000003}]},{"osd":10,"last update":"Sat Mar 23 17:49:00 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.878}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T17:49:06.036 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-23T17:49:06.036 INFO:tasks.cephadm:Setting up client nodes... 2024-03-23T17:49:06.037 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-23T17:49:06.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:05 smithi092 ceph-mon[79089]: from='client.14637 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:49:06.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:05 smithi092 ceph-mon[79089]: pgmap v238: 97 pgs: 97 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 5.3 KiB/s wr, 17 op/s 2024-03-23T17:49:06.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:05 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/1253825365' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T17:49:06.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:05 smithi152 ceph-mon[47949]: from='client.14637 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:49:06.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:05 smithi152 ceph-mon[47949]: pgmap v238: 97 pgs: 97 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 5.3 KiB/s wr, 17 op/s 2024-03-23T17:49:06.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:05 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/1253825365' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T17:49:06.423 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T17:49:06.948 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:06 smithi086 ceph-mon[38221]: from='client.14655 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T17:49:07.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:06 smithi092 ceph-mon[79089]: from='client.14655 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T17:49:07.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:06 smithi152 ceph-mon[47949]: from='client.14655 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T17:49:07.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:07.606+0000 7f5638600700 1 -- 172.21.15.86:0/4131899635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5630100dc0 msgr2=0x7f56301011e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:07.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:07.606+0000 7f5638600700 1 --2- 172.21.15.86:0/4131899635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5630100dc0 0x7f56301011e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f56240099c0 tx=0x7f5624009cd0 comp rx=0 tx=0).stop 2024-03-23T17:49:07.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:07.606+0000 7f5638600700 1 -- 172.21.15.86:0/4131899635 shutdown_connections 2024-03-23T17:49:07.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:07.606+0000 7f5638600700 1 --2- 172.21.15.86:0/4131899635 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5630102960 0x7f5630109210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:07.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:07.606+0000 7f5638600700 1 --2- 172.21.15.86:0/4131899635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5630100dc0 0x7f56301011e0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:07.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:07.606+0000 7f5638600700 1 --2- 172.21.15.86:0/4131899635 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5630101fc0 0x7f5630102420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:07.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:07.606+0000 7f5638600700 1 -- 172.21.15.86:0/4131899635 >> 172.21.15.86:0/4131899635 conn(0x7f56300fc360 msgr2=0x7f56300fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:07.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:07.606+0000 7f5638600700 1 -- 172.21.15.86:0/4131899635 shutdown_connections 2024-03-23T17:49:07.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:07.606+0000 7f5638600700 1 -- 172.21.15.86:0/4131899635 wait complete. 2024-03-23T17:49:07.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:07.607+0000 7f5638600700 1 Processor -- start 2024-03-23T17:49:07.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:07.607+0000 7f5638600700 1 -- start start 2024-03-23T17:49:07.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:07.608+0000 7f5638600700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5630100dc0 0x7f5630196d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:07.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:07.608+0000 7f5638600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5630101fc0 0x7f5630197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:07.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:07.608+0000 7f5638600700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5630102960 0x7f563019e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:07.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:07.608+0000 7f5638600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f563019e830 con 0x7f5630101fc0 2024-03-23T17:49:07.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:07.608+0000 7f5638600700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f563019e970 con 0x7f5630102960 2024-03-23T17:49:07.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:07.608+0000 7f5638600700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f563019eb30 con 0x7f5630100dc0 2024-03-23T17:49:07.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:07.608+0000 7f5635200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5630101fc0 0x7f5630197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:07.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:07.608+0000 7f5635200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5630101fc0 0x7f5630197240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34788/0 (socket says 172.21.15.86:34788) 2024-03-23T17:49:07.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:07.608+0000 7f5635200700 1 -- 172.21.15.86:0/1390786353 learned_addr learned my addr 172.21.15.86:0/1390786353 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:07.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:07.608+0000 7f5635c00700 1 --2- 172.21.15.86:0/1390786353 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5630100dc0 0x7f5630196d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:07.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:07.608+0000 7f5636600700 1 --2- 172.21.15.86:0/1390786353 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5630102960 0x7f563019e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:07.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:07.608+0000 7f5635200700 1 -- 172.21.15.86:0/1390786353 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5630100dc0 msgr2=0x7f5630196d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:07.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:07.608+0000 7f5635200700 1 --2- 172.21.15.86:0/1390786353 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5630100dc0 0x7f5630196d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:07.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:07.608+0000 7f5635200700 1 -- 172.21.15.86:0/1390786353 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5630102960 msgr2=0x7f563019e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:07.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:07.608+0000 7f5635200700 1 --2- 172.21.15.86:0/1390786353 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5630102960 0x7f563019e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:07.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:07.608+0000 7f5635200700 1 -- 172.21.15.86:0/1390786353 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56240095f0 con 0x7f5630101fc0 2024-03-23T17:49:07.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:07.608+0000 7f5636600700 1 --2- 172.21.15.86:0/1390786353 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5630102960 0x7f563019e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:49:07.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:07.608+0000 7f5635200700 1 --2- 172.21.15.86:0/1390786353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5630101fc0 0x7f5630197240 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f561c00d980 tx=0x7f561c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:07.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:07.609+0000 7f562e000700 1 -- 172.21.15.86:0/1390786353 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f561c004220 con 0x7f5630101fc0 2024-03-23T17:49:07.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:07.609+0000 7f562e000700 1 -- 172.21.15.86:0/1390786353 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f561c007640 con 0x7f5630101fc0 2024-03-23T17:49:07.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:07.609+0000 7f5638600700 1 -- 172.21.15.86:0/1390786353 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5630073040 con 0x7f5630101fc0 2024-03-23T17:49:07.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:07.609+0000 7f562e000700 1 -- 172.21.15.86:0/1390786353 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f561c005610 con 0x7f5630101fc0 2024-03-23T17:49:07.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:07.609+0000 7f5638600700 1 -- 172.21.15.86:0/1390786353 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5630073510 con 0x7f5630101fc0 2024-03-23T17:49:07.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:07.610+0000 7f5638600700 1 -- 172.21.15.86:0/1390786353 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5630108290 con 0x7f5630101fc0 2024-03-23T17:49:07.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:07.612+0000 7f562e000700 1 -- 172.21.15.86:0/1390786353 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f561c004750 con 0x7f5630101fc0 2024-03-23T17:49:07.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:07.614+0000 7f562e000700 1 --2- 172.21.15.86:0/1390786353 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f56180a11b0 0x7f56180a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:07.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:07.614+0000 7f562e000700 1 -- 172.21.15.86:0/1390786353 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f561c0cc890 con 0x7f5630101fc0 2024-03-23T17:49:07.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:07.614+0000 7f5635c00700 1 --2- 172.21.15.86:0/1390786353 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f56180a11b0 0x7f56180a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:07.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:07.614+0000 7f5635c00700 1 --2- 172.21.15.86:0/1390786353 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f56180a11b0 0x7f56180a3670 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5624006010 tx=0x7f5624000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:07.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:07.618+0000 7f562e000700 1 -- 172.21.15.86:0/1390786353 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f561c095f70 con 0x7f5630101fc0 2024-03-23T17:49:07.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:07.907+0000 7f5638600700 1 -- 172.21.15.86:0/1390786353 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f563004ea80 con 0x7f5630101fc0 2024-03-23T17:49:07.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:07.912+0000 7f562e000700 1 -- 172.21.15.86:0/1390786353 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v31) v1 ==== 170+0+59 (secure 0 0 0) 0x7f561c016020 con 0x7f5630101fc0 2024-03-23T17:49:07.913 INFO:teuthology.orchestra.run.smithi086.stdout:[client.0] 2024-03-23T17:49:07.913 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQATFv9luTIxNhAAgvkFSUpJHVkzVNdFatM3ug== 2024-03-23T17:49:07.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:07.915+0000 7f5638600700 1 -- 172.21.15.86:0/1390786353 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f56180a11b0 msgr2=0x7f56180a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:07.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:07.915+0000 7f5638600700 1 --2- 172.21.15.86:0/1390786353 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f56180a11b0 0x7f56180a3670 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5624006010 tx=0x7f5624000bc0 comp rx=0 tx=0).stop 2024-03-23T17:49:07.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:07.915+0000 7f5638600700 1 -- 172.21.15.86:0/1390786353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5630101fc0 msgr2=0x7f5630197240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:07.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:07.915+0000 7f5638600700 1 --2- 172.21.15.86:0/1390786353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5630101fc0 0x7f5630197240 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f561c00d980 tx=0x7f561c00dc90 comp rx=0 tx=0).stop 2024-03-23T17:49:07.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:07.915+0000 7f5638600700 1 -- 172.21.15.86:0/1390786353 shutdown_connections 2024-03-23T17:49:07.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:07.915+0000 7f5638600700 1 --2- 172.21.15.86:0/1390786353 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5630102960 0x7f563019e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:07.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:07.915+0000 7f5638600700 1 --2- 172.21.15.86:0/1390786353 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5630100dc0 0x7f5630196d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:07.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:07.915+0000 7f5638600700 1 --2- 172.21.15.86:0/1390786353 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f56180a11b0 0x7f56180a3670 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:07.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:07.915+0000 7f5638600700 1 --2- 172.21.15.86:0/1390786353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5630101fc0 0x7f5630197240 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:07.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:07.915+0000 7f5638600700 1 -- 172.21.15.86:0/1390786353 >> 172.21.15.86:0/1390786353 conn(0x7f56300fc360 msgr2=0x7f56301053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:07.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:07.915+0000 7f5638600700 1 -- 172.21.15.86:0/1390786353 shutdown_connections 2024-03-23T17:49:07.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:07.915+0000 7f5638600700 1 -- 172.21.15.86:0/1390786353 wait complete. 2024-03-23T17:49:08.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:07 smithi086 ceph-mon[38221]: from='client.14661 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T17:49:08.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:07 smithi086 ceph-mon[38221]: pgmap v239: 97 pgs: 97 active+clean; 624 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 10 KiB/s wr, 24 op/s 2024-03-23T17:49:08.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:07 smithi092 ceph-mon[79089]: from='client.14661 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T17:49:08.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:07 smithi092 ceph-mon[79089]: pgmap v239: 97 pgs: 97 active+clean; 624 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 10 KiB/s wr, 24 op/s 2024-03-23T17:49:08.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:07 smithi152 ceph-mon[47949]: from='client.14661 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T17:49:08.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:07 smithi152 ceph-mon[47949]: pgmap v239: 97 pgs: 97 active+clean; 624 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 10 KiB/s wr, 24 op/s 2024-03-23T17:49:08.287 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-23T17:49:08.287 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-03-23T17:49:08.287 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-23T17:49:08.384 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-23T17:49:08.384 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-03-23T17:49:08.384 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph mgr dump --format=json 2024-03-23T17:49:08.781 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T17:49:08.972 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:08 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/1390786353' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-23T17:49:08.973 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:08 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/1390786353' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-23T17:49:09.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:08 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/1390786353' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-23T17:49:09.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:08 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/1390786353' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-23T17:49:09.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:08 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/1390786353' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-23T17:49:09.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:08 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/1390786353' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-23T17:49:09.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:09.912+0000 7fada0000700 1 -- 172.21.15.86:0/2490969817 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad981060c0 msgr2=0x7fad9810dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:09.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:09.912+0000 7fada0000700 1 --2- 172.21.15.86:0/2490969817 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad981060c0 0x7fad9810dc90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fad9000b0a0 tx=0x7fad9000b3b0 comp rx=0 tx=0).stop 2024-03-23T17:49:09.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:09.913+0000 7fada0000700 1 -- 172.21.15.86:0/2490969817 shutdown_connections 2024-03-23T17:49:09.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:09.913+0000 7fada0000700 1 --2- 172.21.15.86:0/2490969817 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad981060c0 0x7fad9810dc90 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:09.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:09.913+0000 7fada0000700 1 --2- 172.21.15.86:0/2490969817 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad98069240 0x7fad98069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:09.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:09.913+0000 7fada0000700 1 --2- 172.21.15.86:0/2490969817 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad98069bf0 0x7fad981059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:09.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:09.913+0000 7fada0000700 1 -- 172.21.15.86:0/2490969817 >> 172.21.15.86:0/2490969817 conn(0x7fad98075d50 msgr2=0x7fad98076160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:09.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:09.913+0000 7fada0000700 1 -- 172.21.15.86:0/2490969817 shutdown_connections 2024-03-23T17:49:09.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:09.913+0000 7fada0000700 1 -- 172.21.15.86:0/2490969817 wait complete. 2024-03-23T17:49:09.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:09.914+0000 7fada0000700 1 Processor -- start 2024-03-23T17:49:09.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:09.914+0000 7fada0000700 1 -- start start 2024-03-23T17:49:09.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:09.914+0000 7fada0000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad98069240 0x7fad981972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:09.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:09.914+0000 7fada0000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad98069bf0 0x7fad98197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:09.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:09.914+0000 7fada0000700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad981060c0 0x7fad9819bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:09.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:09.914+0000 7fada0000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad9819c1e0 con 0x7fad98069240 2024-03-23T17:49:09.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:09.914+0000 7fada0000700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad9819c350 con 0x7fad98069bf0 2024-03-23T17:49:09.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:09.914+0000 7fada0000700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad9819c510 con 0x7fad981060c0 2024-03-23T17:49:09.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:09.914+0000 7fad9d600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad98069240 0x7fad981972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:09.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:09.915+0000 7fad9d600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad98069240 0x7fad981972e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34808/0 (socket says 172.21.15.86:34808) 2024-03-23T17:49:09.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:09.915+0000 7fad9d600700 1 -- 172.21.15.86:0/3780357292 learned_addr learned my addr 172.21.15.86:0/3780357292 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:09.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:09.915+0000 7fad9cc00700 1 --2- 172.21.15.86:0/3780357292 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad98069bf0 0x7fad98197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:09.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:09.915+0000 7fad9e000700 1 --2- 172.21.15.86:0/3780357292 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad981060c0 0x7fad9819bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:09.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:09.915+0000 7fad9d600700 1 -- 172.21.15.86:0/3780357292 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad981060c0 msgr2=0x7fad9819bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:09.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:09.915+0000 7fad9d600700 1 --2- 172.21.15.86:0/3780357292 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad981060c0 0x7fad9819bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:09.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:09.915+0000 7fad9d600700 1 -- 172.21.15.86:0/3780357292 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad98069bf0 msgr2=0x7fad98197820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:09.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:09.915+0000 7fad9d600700 1 --2- 172.21.15.86:0/3780357292 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad98069bf0 0x7fad98197820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:09.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:09.915+0000 7fad9d600700 1 -- 172.21.15.86:0/3780357292 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad90009d00 con 0x7fad98069240 2024-03-23T17:49:09.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:09.915+0000 7fad9d600700 1 --2- 172.21.15.86:0/3780357292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad98069240 0x7fad981972e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fad8c00d950 tx=0x7fad8c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:09.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:09.915+0000 7fad96000700 1 -- 172.21.15.86:0/3780357292 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad8c00f890 con 0x7fad98069240 2024-03-23T17:49:09.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:09.915+0000 7fad96000700 1 -- 172.21.15.86:0/3780357292 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad8c010460 con 0x7fad98069240 2024-03-23T17:49:09.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:09.915+0000 7fada0000700 1 -- 172.21.15.86:0/3780357292 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad9819c7f0 con 0x7fad98069240 2024-03-23T17:49:09.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:09.915+0000 7fad96000700 1 -- 172.21.15.86:0/3780357292 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad8c00e610 con 0x7fad98069240 2024-03-23T17:49:09.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:09.916+0000 7fada0000700 1 -- 172.21.15.86:0/3780357292 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad981a4050 con 0x7fad98069240 2024-03-23T17:49:09.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:09.917+0000 7fada0000700 1 -- 172.21.15.86:0/3780357292 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad98109350 con 0x7fad98069240 2024-03-23T17:49:09.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:09.918+0000 7fad96000700 1 -- 172.21.15.86:0/3780357292 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fad8c0108d0 con 0x7fad98069240 2024-03-23T17:49:09.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:09.920+0000 7fad96000700 1 --2- 172.21.15.86:0/3780357292 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fad800a12b0 0x7fad800a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:09.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:09.920+0000 7fad96000700 1 -- 172.21.15.86:0/3780357292 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fad8c0cc6e0 con 0x7fad98069240 2024-03-23T17:49:09.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:09.920+0000 7fad9cc00700 1 --2- 172.21.15.86:0/3780357292 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fad800a12b0 0x7fad800a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:09.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:09.920+0000 7fad9cc00700 1 --2- 172.21.15.86:0/3780357292 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fad800a12b0 0x7fad800a3770 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fad98198840 tx=0x7fad840073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:09.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:09.923+0000 7fad96000700 1 -- 172.21.15.86:0/3780357292 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fad8c095dc0 con 0x7fad98069240 2024-03-23T17:49:10.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:09 smithi086 ceph-mon[38221]: pgmap v240: 97 pgs: 97 active+clean; 625 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 9.7 KiB/s wr, 24 op/s 2024-03-23T17:49:10.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:09 smithi092 ceph-mon[79089]: pgmap v240: 97 pgs: 97 active+clean; 625 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 9.7 KiB/s wr, 24 op/s 2024-03-23T17:49:10.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:09 smithi152 ceph-mon[47949]: pgmap v240: 97 pgs: 97 active+clean; 625 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 9.7 KiB/s wr, 24 op/s 2024-03-23T17:49:10.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:10.182+0000 7fada0000700 1 -- 172.21.15.86:0/3780357292 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fad981024c0 con 0x7fad98069240 2024-03-23T17:49:10.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:10.187+0000 7fad96000700 1 -- 172.21.15.86:0/3780357292 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+260274 (secure 0 0 0) 0x7fad8c0993e0 con 0x7fad98069240 2024-03-23T17:49:10.192 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:49:10.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:10.192+0000 7fada0000700 1 -- 172.21.15.86:0/3780357292 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fad800a12b0 msgr2=0x7fad800a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:10.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:10.192+0000 7fada0000700 1 --2- 172.21.15.86:0/3780357292 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fad800a12b0 0x7fad800a3770 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fad98198840 tx=0x7fad840073d0 comp rx=0 tx=0).stop 2024-03-23T17:49:10.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:10.192+0000 7fada0000700 1 -- 172.21.15.86:0/3780357292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad98069240 msgr2=0x7fad981972e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:10.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:10.192+0000 7fada0000700 1 --2- 172.21.15.86:0/3780357292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad98069240 0x7fad981972e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fad8c00d950 tx=0x7fad8c00dc60 comp rx=0 tx=0).stop 2024-03-23T17:49:10.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:10.192+0000 7fada0000700 1 -- 172.21.15.86:0/3780357292 shutdown_connections 2024-03-23T17:49:10.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:10.192+0000 7fada0000700 1 --2- 172.21.15.86:0/3780357292 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad981060c0 0x7fad9819bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:10.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:10.192+0000 7fada0000700 1 --2- 172.21.15.86:0/3780357292 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fad800a12b0 0x7fad800a3770 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:10.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:10.192+0000 7fada0000700 1 --2- 172.21.15.86:0/3780357292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad98069240 0x7fad981972e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:10.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:10.192+0000 7fada0000700 1 --2- 172.21.15.86:0/3780357292 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad98069bf0 0x7fad98197820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:10.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:10.192+0000 7fada0000700 1 -- 172.21.15.86:0/3780357292 >> 172.21.15.86:0/3780357292 conn(0x7fad98075d50 msgr2=0x7fad98107240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:10.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:10.192+0000 7fada0000700 1 -- 172.21.15.86:0/3780357292 shutdown_connections 2024-03-23T17:49:10.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:10.192+0000 7fada0000700 1 -- 172.21.15.86:0/3780357292 wait complete. 2024-03-23T17:49:10.605 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":17,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2734784244},{"type":"v1","addr":"172.21.15.86:6801","nonce":2734784244}]},"active_addr":"172.21.15.86:6801/2734784244","active_change":"2024-03-23T17:43:28.242598+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24112,"name":"z","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24116,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.86:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.86:0","nonce":2354115349}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.86:0","nonce":883052430}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.86:0","nonce":2938296862}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.86:0","nonce":734928365}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.86:0","nonce":2385800549}]}]} 2024-03-23T17:49:10.611 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-03-23T17:49:10.611 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-03-23T17:49:10.611 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph osd dump --format=json 2024-03-23T17:49:10.973 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T17:49:11.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:10 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/3780357292' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-23T17:49:11.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:10 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/3780357292' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-23T17:49:11.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:10 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/3780357292' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-23T17:49:12.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:11 smithi086 ceph-mon[38221]: pgmap v241: 97 pgs: 97 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 10 KiB/s wr, 26 op/s 2024-03-23T17:49:12.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:11 smithi092 ceph-mon[79089]: pgmap v241: 97 pgs: 97 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 10 KiB/s wr, 26 op/s 2024-03-23T17:49:12.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:12.069+0000 7fbc90800700 1 -- 172.21.15.86:0/3191247194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc88074d30 msgr2=0x7fbc88072fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:12.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:12.069+0000 7fbc90800700 1 --2- 172.21.15.86:0/3191247194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc88074d30 0x7fbc88072fe0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c009990 tx=0x7fbc7c009ca0 comp rx=0 tx=0).stop 2024-03-23T17:49:12.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:12.069+0000 7fbc90800700 1 -- 172.21.15.86:0/3191247194 shutdown_connections 2024-03-23T17:49:12.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:12.069+0000 7fbc90800700 1 --2- 172.21.15.86:0/3191247194 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc88111b30 0x7fbc88113f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:12.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:12.069+0000 7fbc90800700 1 --2- 172.21.15.86:0/3191247194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc88074d30 0x7fbc88072fe0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:12.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:12.069+0000 7fbc90800700 1 --2- 172.21.15.86:0/3191247194 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc88073520 0x7fbc880739a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:12.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:12.069+0000 7fbc90800700 1 -- 172.21.15.86:0/3191247194 >> 172.21.15.86:0/3191247194 conn(0x7fbc88077fa0 msgr2=0x7fbc880783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:12.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:12.069+0000 7fbc90800700 1 -- 172.21.15.86:0/3191247194 shutdown_connections 2024-03-23T17:49:12.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:12.069+0000 7fbc90800700 1 -- 172.21.15.86:0/3191247194 wait complete. 2024-03-23T17:49:12.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:12.070+0000 7fbc90800700 1 Processor -- start 2024-03-23T17:49:12.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:12.070+0000 7fbc90800700 1 -- start start 2024-03-23T17:49:12.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:12.071+0000 7fbc90800700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc88073520 0x7fbc881a3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:12.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:12.071+0000 7fbc90800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc88074d30 0x7fbc881a3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:12.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:12.071+0000 7fbc90800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc88111b30 0x7fbc8819d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:12.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:12.071+0000 7fbc90800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc8819dd30 con 0x7fbc88111b30 2024-03-23T17:49:12.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:12.071+0000 7fbc90800700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc8819dea0 con 0x7fbc88074d30 2024-03-23T17:49:12.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:12.071+0000 7fbc90800700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc8819e060 con 0x7fbc88073520 2024-03-23T17:49:12.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:12.071+0000 7fbc8e800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc88111b30 0x7fbc8819d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:12.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:12.071+0000 7fbc8e800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc88111b30 0x7fbc8819d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34836/0 (socket says 172.21.15.86:34836) 2024-03-23T17:49:12.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:12.071+0000 7fbc8e800700 1 -- 172.21.15.86:0/818582920 learned_addr learned my addr 172.21.15.86:0/818582920 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:12.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:12.071+0000 7fbc8de00700 1 --2- 172.21.15.86:0/818582920 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc88073520 0x7fbc881a3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:12.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:12.071+0000 7fbc8d400700 1 --2- 172.21.15.86:0/818582920 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc88074d30 0x7fbc881a3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:12.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:12.071+0000 7fbc8e800700 1 -- 172.21.15.86:0/818582920 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc88073520 msgr2=0x7fbc881a3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:12.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:12.071+0000 7fbc8e800700 1 --2- 172.21.15.86:0/818582920 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc88073520 0x7fbc881a3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:12.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:12.071+0000 7fbc8e800700 1 -- 172.21.15.86:0/818582920 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc88074d30 msgr2=0x7fbc881a3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:12.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:12.071+0000 7fbc8e800700 1 --2- 172.21.15.86:0/818582920 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc88074d30 0x7fbc881a3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:12.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:12.071+0000 7fbc8e800700 1 -- 172.21.15.86:0/818582920 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc7c0095f0 con 0x7fbc88111b30 2024-03-23T17:49:12.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:12.072+0000 7fbc8e800700 1 --2- 172.21.15.86:0/818582920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc88111b30 0x7fbc8819d6d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fbc84007f10 tx=0x7fbc8400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:12.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:12.072+0000 7fbc82a00700 1 -- 172.21.15.86:0/818582920 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc84017070 con 0x7fbc88111b30 2024-03-23T17:49:12.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:12.072+0000 7fbc82a00700 1 -- 172.21.15.86:0/818582920 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc8400f040 con 0x7fbc88111b30 2024-03-23T17:49:12.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:12.072+0000 7fbc90800700 1 -- 172.21.15.86:0/818582920 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc8819e340 con 0x7fbc88111b30 2024-03-23T17:49:12.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:12.072+0000 7fbc82a00700 1 -- 172.21.15.86:0/818582920 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc84004020 con 0x7fbc88111b30 2024-03-23T17:49:12.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:12.072+0000 7fbc90800700 1 -- 172.21.15.86:0/818582920 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc881aa580 con 0x7fbc88111b30 2024-03-23T17:49:12.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:12.073+0000 7fbc90800700 1 -- 172.21.15.86:0/818582920 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc88074670 con 0x7fbc88111b30 2024-03-23T17:49:12.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:12.075+0000 7fbc82a00700 1 -- 172.21.15.86:0/818582920 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fbc8400db20 con 0x7fbc88111b30 2024-03-23T17:49:12.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:12.076+0000 7fbc82a00700 1 --2- 172.21.15.86:0/818582920 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fbc700a1230 0x7fbc700a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:12.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:12.076+0000 7fbc82a00700 1 -- 172.21.15.86:0/818582920 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fbc840cbdf0 con 0x7fbc88111b30 2024-03-23T17:49:12.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:12.077+0000 7fbc8de00700 1 --2- 172.21.15.86:0/818582920 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fbc700a1230 0x7fbc700a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:12.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:12.077+0000 7fbc8de00700 1 --2- 172.21.15.86:0/818582920 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fbc700a1230 0x7fbc700a36f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c0098e0 tx=0x7fbc7c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:12.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:12.081+0000 7fbc82a00700 1 -- 172.21.15.86:0/818582920 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fbc84095560 con 0x7fbc88111b30 2024-03-23T17:49:12.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:11 smithi152 ceph-mon[47949]: pgmap v241: 97 pgs: 97 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 10 KiB/s wr, 26 op/s 2024-03-23T17:49:12.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:12.289+0000 7fbc90800700 1 -- 172.21.15.86:0/818582920 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbc8804ea80 con 0x7fbc88111b30 2024-03-23T17:49:12.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:12.290+0000 7fbc82a00700 1 -- 172.21.15.86:0/818582920 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+22083 (secure 0 0 0) 0x7fbc84098b80 con 0x7fbc88111b30 2024-03-23T17:49:12.291 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:49:12.291 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":80,"fsid":"9d76c9e8-e93c-11ee-95cd-87774f69a715","created":"2024-03-23T17:42:30.097233+0000","modified":"2024-03-23T17:48:57.204624+0000","last_up_change":"2024-03-23T17:48:18.217601+0000","last_in_change":"2024-03-23T17:48:00.250579+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T17:45:26.848397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T17:48:54.331894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T17:48:55.640693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"656655c6-1c7f-4640-92c4-2f42de820143","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6803","nonce":2423505491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6805","nonce":2423505491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6809","nonce":2423505491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6807","nonce":2423505491}]},"public_addr":"172.21.15.86:6803/2423505491","cluster_addr":"172.21.15.86:6805/2423505491","heartbeat_back_addr":"172.21.15.86:6809/2423505491","heartbeat_front_addr":"172.21.15.86:6807/2423505491","state":["exists","up"]},{"osd":1,"uuid":"3e86dc65-20a6-4de8-b9ce-fcf01f68862b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6801","nonce":3776252597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6803","nonce":3776252597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6807","nonce":3776252597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6805","nonce":3776252597}]},"public_addr":"172.21.15.92:6801/3776252597","cluster_addr":"172.21.15.92:6803/3776252597","heartbeat_back_addr":"172.21.15.92:6807/3776252597","heartbeat_front_addr":"172.21.15.92:6805/3776252597","state":["exists","up"]},{"osd":2,"uuid":"09808786-3bd3-4d11-8c7b-1736119dfbb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6801","nonce":1601265891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6803","nonce":1601265891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6807","nonce":1601265891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6805","nonce":1601265891}]},"public_addr":"172.21.15.152:6801/1601265891","cluster_addr":"172.21.15.152:6803/1601265891","heartbeat_back_addr":"172.21.15.152:6807/1601265891","heartbeat_front_addr":"172.21.15.152:6805/1601265891","state":["exists","up"]},{"osd":3,"uuid":"79c61910-4b53-424c-b150-6f5385a2cd68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6811","nonce":694846466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6813","nonce":694846466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6817","nonce":694846466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6815","nonce":694846466}]},"public_addr":"172.21.15.86:6811/694846466","cluster_addr":"172.21.15.86:6813/694846466","heartbeat_back_addr":"172.21.15.86:6817/694846466","heartbeat_front_addr":"172.21.15.86:6815/694846466","state":["exists","up"]},{"osd":4,"uuid":"a2878035-0179-469f-bbd4-8ce2d3f6bddc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6809","nonce":1422821471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6811","nonce":1422821471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6815","nonce":1422821471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6813","nonce":1422821471}]},"public_addr":"172.21.15.92:6809/1422821471","cluster_addr":"172.21.15.92:6811/1422821471","heartbeat_back_addr":"172.21.15.92:6815/1422821471","heartbeat_front_addr":"172.21.15.92:6813/1422821471","state":["exists","up"]},{"osd":5,"uuid":"837366aa-ce25-42ca-b157-5bbb50159d0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6809","nonce":1039375239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6811","nonce":1039375239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6815","nonce":1039375239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6813","nonce":1039375239}]},"public_addr":"172.21.15.152:6809/1039375239","cluster_addr":"172.21.15.152:6811/1039375239","heartbeat_back_addr":"172.21.15.152:6815/1039375239","heartbeat_front_addr":"172.21.15.152:6813/1039375239","state":["exists","up"]},{"osd":6,"uuid":"61024437-7b91-432f-a1af-6443d0ef5213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6819","nonce":4080452707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6821","nonce":4080452707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6825","nonce":4080452707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6823","nonce":4080452707}]},"public_addr":"172.21.15.86:6819/4080452707","cluster_addr":"172.21.15.86:6821/4080452707","heartbeat_back_addr":"172.21.15.86:6825/4080452707","heartbeat_front_addr":"172.21.15.86:6823/4080452707","state":["exists","up"]},{"osd":7,"uuid":"2684bdae-6b30-4afc-af68-4ce41b3a75b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6817","nonce":4187707622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6819","nonce":4187707622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6823","nonce":4187707622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6821","nonce":4187707622}]},"public_addr":"172.21.15.92:6817/4187707622","cluster_addr":"172.21.15.92:6819/4187707622","heartbeat_back_addr":"172.21.15.92:6823/4187707622","heartbeat_front_addr":"172.21.15.92:6821/4187707622","state":["exists","up"]},{"osd":8,"uuid":"f03bbf0d-1281-4bcf-9773-9854388e65ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6817","nonce":3237612748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6819","nonce":3237612748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6823","nonce":3237612748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6821","nonce":3237612748}]},"public_addr":"172.21.15.152:6817/3237612748","cluster_addr":"172.21.15.152:6819/3237612748","heartbeat_back_addr":"172.21.15.152:6823/3237612748","heartbeat_front_addr":"172.21.15.152:6821/3237612748","state":["exists","up"]},{"osd":9,"uuid":"e7e8b775-99ff-48a4-a23a-91e7ab6622bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6827","nonce":2835218294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6829","nonce":2835218294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6833","nonce":2835218294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6831","nonce":2835218294}]},"public_addr":"172.21.15.86:6827/2835218294","cluster_addr":"172.21.15.86:6829/2835218294","heartbeat_back_addr":"172.21.15.86:6833/2835218294","heartbeat_front_addr":"172.21.15.86:6831/2835218294","state":["exists","up"]},{"osd":10,"uuid":"d9d616cd-a229-44de-8768-02a4415df115","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6825","nonce":4259465100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6827","nonce":4259465100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6831","nonce":4259465100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6829","nonce":4259465100}]},"public_addr":"172.21.15.92:6825/4259465100","cluster_addr":"172.21.15.92:6827/4259465100","heartbeat_back_addr":"172.21.15.92:6831/4259465100","heartbeat_front_addr":"172.21.15.92:6829/4259465100","state":["exists","up"]},{"osd":11,"uuid":"e7c0b865-ccf5-4233-8d22-0d9b6dbe5c97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6825","nonce":194662760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6827","nonce":194662760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6831","nonce":194662760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6829","nonce":194662760}]},"public_addr":"172.21.15.152:6825/194662760","cluster_addr":"172.21.15.152:6827/194662760","heartbeat_back_addr":"172.21.15.152:6831/194662760","heartbeat_front_addr":"172.21.15.152:6829/194662760","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:44:47.269167+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:45:05.363655+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:45:23.192677+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:45:44.156228+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:02.532971+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:20.153295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:41.713888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:59.676516+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:47:17.207087+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:47:38.970316+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:47:57.282525+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:48:15.437606+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/3553500912":"2024-03-24T17:43:28.242492+0000","172.21.15.86:6801/3600278228":"2024-03-24T17:43:28.242492+0000","172.21.15.86:0/887262329":"2024-03-24T17:43:28.242492+0000","172.21.15.86:6800/2313723492":"2024-03-24T17:43:05.043910+0000","172.21.15.86:6800/3600278228":"2024-03-24T17:43:28.242492+0000","172.21.15.86:0/3815705642":"2024-03-24T17:43:05.043910+0000","172.21.15.86:0/1614943062":"2024-03-24T17:43:28.242492+0000","172.21.15.86:6801/2313723492":"2024-03-24T17:43:05.043910+0000","172.21.15.86:0/2416321716":"2024-03-24T17:43:05.043910+0000","172.21.15.86:0/433588188":"2024-03-24T17:43:05.043910+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T17:49:12.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:12.293+0000 7fbc90800700 1 -- 172.21.15.86:0/818582920 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fbc700a1230 msgr2=0x7fbc700a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:12.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:12.293+0000 7fbc90800700 1 --2- 172.21.15.86:0/818582920 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fbc700a1230 0x7fbc700a36f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c0098e0 tx=0x7fbc7c005960 comp rx=0 tx=0).stop 2024-03-23T17:49:12.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:12.293+0000 7fbc90800700 1 -- 172.21.15.86:0/818582920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc88111b30 msgr2=0x7fbc8819d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:12.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:12.293+0000 7fbc90800700 1 --2- 172.21.15.86:0/818582920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc88111b30 0x7fbc8819d6d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fbc84007f10 tx=0x7fbc8400d3b0 comp rx=0 tx=0).stop 2024-03-23T17:49:12.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:12.293+0000 7fbc90800700 1 -- 172.21.15.86:0/818582920 shutdown_connections 2024-03-23T17:49:12.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:12.293+0000 7fbc90800700 1 --2- 172.21.15.86:0/818582920 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fbc700a1230 0x7fbc700a36f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:12.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:12.293+0000 7fbc90800700 1 --2- 172.21.15.86:0/818582920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc88111b30 0x7fbc8819d6d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:12.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:12.293+0000 7fbc90800700 1 --2- 172.21.15.86:0/818582920 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc88073520 0x7fbc881a3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:12.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:12.293+0000 7fbc90800700 1 --2- 172.21.15.86:0/818582920 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc88074d30 0x7fbc881a3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:12.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:12.293+0000 7fbc90800700 1 -- 172.21.15.86:0/818582920 >> 172.21.15.86:0/818582920 conn(0x7fbc88077fa0 msgr2=0x7fbc881107c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:12.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:12.293+0000 7fbc90800700 1 -- 172.21.15.86:0/818582920 shutdown_connections 2024-03-23T17:49:12.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:12.293+0000 7fbc90800700 1 -- 172.21.15.86:0/818582920 wait complete. 2024-03-23T17:49:12.687 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-03-23T17:49:12.687 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph osd dump --format=json 2024-03-23T17:49:13.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:12 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/818582920' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T17:49:13.046 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T17:49:13.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:12 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/818582920' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T17:49:13.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:12 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/818582920' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T17:49:14.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:13 smithi086 ceph-mon[38221]: pgmap v242: 97 pgs: 97 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 9.5 KiB/s wr, 24 op/s 2024-03-23T17:49:14.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:13 smithi092 ceph-mon[79089]: pgmap v242: 97 pgs: 97 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 9.5 KiB/s wr, 24 op/s 2024-03-23T17:49:14.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:13 smithi152 ceph-mon[47949]: pgmap v242: 97 pgs: 97 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 9.5 KiB/s wr, 24 op/s 2024-03-23T17:49:14.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:14.137+0000 7f3962600700 1 -- 172.21.15.86:0/3752236354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f395c100e10 msgr2=0x7f395c1011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:14.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:14.137+0000 7f3962600700 1 --2- 172.21.15.86:0/3752236354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f395c100e10 0x7f395c1011f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3950009990 tx=0x7f3950009ca0 comp rx=0 tx=0).stop 2024-03-23T17:49:14.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:14.137+0000 7f3962600700 1 -- 172.21.15.86:0/3752236354 shutdown_connections 2024-03-23T17:49:14.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:14.137+0000 7f3962600700 1 --2- 172.21.15.86:0/3752236354 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f395c105ef0 0x7f395c10dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:14.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:14.137+0000 7f3962600700 1 --2- 172.21.15.86:0/3752236354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f395c100e10 0x7f395c1011f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:14.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:14.137+0000 7f3962600700 1 --2- 172.21.15.86:0/3752236354 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f395c1017c0 0x7f395c105810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:14.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:14.137+0000 7f3962600700 1 -- 172.21.15.86:0/3752236354 >> 172.21.15.86:0/3752236354 conn(0x7f395c0fc6e0 msgr2=0x7f395c0feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:14.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:14.137+0000 7f3962600700 1 -- 172.21.15.86:0/3752236354 shutdown_connections 2024-03-23T17:49:14.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:14.137+0000 7f3962600700 1 -- 172.21.15.86:0/3752236354 wait complete. 2024-03-23T17:49:14.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:14.138+0000 7f3962600700 1 Processor -- start 2024-03-23T17:49:14.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:14.138+0000 7f3962600700 1 -- start start 2024-03-23T17:49:14.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:14.139+0000 7f3962600700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f395c100e10 0x7f395c1992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:14.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:14.139+0000 7f3962600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f395c1017c0 0x7f395c199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:14.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:14.139+0000 7f3962600700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f395c105ef0 0x7f395c19dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:14.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:14.139+0000 7f3962600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f395c19e1f0 con 0x7f395c1017c0 2024-03-23T17:49:14.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:14.139+0000 7f3962600700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f395c19e360 con 0x7f395c100e10 2024-03-23T17:49:14.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:14.139+0000 7f3962600700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f395c19e520 con 0x7f395c105ef0 2024-03-23T17:49:14.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:14.139+0000 7f395be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f395c1017c0 0x7f395c199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:14.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:14.139+0000 7f395be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f395c1017c0 0x7f395c199830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34850/0 (socket says 172.21.15.86:34850) 2024-03-23T17:49:14.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:14.139+0000 7f395be00700 1 -- 172.21.15.86:0/1609595991 learned_addr learned my addr 172.21.15.86:0/1609595991 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:14.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:14.139+0000 7f3961200700 1 --2- 172.21.15.86:0/1609595991 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f395c100e10 0x7f395c1992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:14.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:14.139+0000 7f3961c00700 1 --2- 172.21.15.86:0/1609595991 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f395c105ef0 0x7f395c19dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:14.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:14.139+0000 7f395be00700 1 -- 172.21.15.86:0/1609595991 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f395c105ef0 msgr2=0x7f395c19dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:14.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:14.139+0000 7f395be00700 1 --2- 172.21.15.86:0/1609595991 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f395c105ef0 0x7f395c19dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:14.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:14.139+0000 7f395be00700 1 -- 172.21.15.86:0/1609595991 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f395c100e10 msgr2=0x7f395c1992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:14.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:14.139+0000 7f395be00700 1 --2- 172.21.15.86:0/1609595991 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f395c100e10 0x7f395c1992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:14.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:14.140+0000 7f395be00700 1 -- 172.21.15.86:0/1609595991 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39500095f0 con 0x7f395c1017c0 2024-03-23T17:49:14.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:14.140+0000 7f395be00700 1 --2- 172.21.15.86:0/1609595991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f395c1017c0 0x7f395c199830 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f394800d950 tx=0x7f394800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:14.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:14.140+0000 7f3959600700 1 -- 172.21.15.86:0/1609595991 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3948004220 con 0x7f395c1017c0 2024-03-23T17:49:14.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:14.140+0000 7f3959600700 1 -- 172.21.15.86:0/1609595991 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3948007620 con 0x7f395c1017c0 2024-03-23T17:49:14.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:14.140+0000 7f3959600700 1 -- 172.21.15.86:0/1609595991 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39480056c0 con 0x7f395c1017c0 2024-03-23T17:49:14.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:14.140+0000 7f3962600700 1 -- 172.21.15.86:0/1609595991 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f395c19e800 con 0x7f395c1017c0 2024-03-23T17:49:14.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:14.140+0000 7f3962600700 1 -- 172.21.15.86:0/1609595991 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f395c1a6060 con 0x7f395c1017c0 2024-03-23T17:49:14.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:14.141+0000 7f3962600700 1 -- 172.21.15.86:0/1609595991 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f395c1022f0 con 0x7f395c1017c0 2024-03-23T17:49:14.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:14.143+0000 7f3959600700 1 -- 172.21.15.86:0/1609595991 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f3948004380 con 0x7f395c1017c0 2024-03-23T17:49:14.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:14.144+0000 7f3959600700 1 --2- 172.21.15.86:0/1609595991 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f39440a1230 0x7f39440a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:14.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:14.144+0000 7f3959600700 1 -- 172.21.15.86:0/1609595991 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f39480cc630 con 0x7f395c1017c0 2024-03-23T17:49:14.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:14.144+0000 7f3961200700 1 --2- 172.21.15.86:0/1609595991 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f39440a1230 0x7f39440a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:14.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:14.145+0000 7f3961200700 1 --2- 172.21.15.86:0/1609595991 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f39440a1230 0x7f39440a36f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f395000b5c0 tx=0x7f3950005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:14.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:14.148+0000 7f3959600700 1 -- 172.21.15.86:0/1609595991 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f3948095d10 con 0x7f395c1017c0 2024-03-23T17:49:14.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:14.350+0000 7f3962600700 1 -- 172.21.15.86:0/1609595991 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f395c04ea80 con 0x7f395c1017c0 2024-03-23T17:49:14.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:14.351+0000 7f3959600700 1 -- 172.21.15.86:0/1609595991 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+22083 (secure 0 0 0) 0x7f3948099330 con 0x7f395c1017c0 2024-03-23T17:49:14.352 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:49:14.352 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":80,"fsid":"9d76c9e8-e93c-11ee-95cd-87774f69a715","created":"2024-03-23T17:42:30.097233+0000","modified":"2024-03-23T17:48:57.204624+0000","last_up_change":"2024-03-23T17:48:18.217601+0000","last_in_change":"2024-03-23T17:48:00.250579+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T17:45:26.848397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T17:48:54.331894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T17:48:55.640693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"656655c6-1c7f-4640-92c4-2f42de820143","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6803","nonce":2423505491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6805","nonce":2423505491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6809","nonce":2423505491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6807","nonce":2423505491}]},"public_addr":"172.21.15.86:6803/2423505491","cluster_addr":"172.21.15.86:6805/2423505491","heartbeat_back_addr":"172.21.15.86:6809/2423505491","heartbeat_front_addr":"172.21.15.86:6807/2423505491","state":["exists","up"]},{"osd":1,"uuid":"3e86dc65-20a6-4de8-b9ce-fcf01f68862b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6801","nonce":3776252597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6803","nonce":3776252597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6807","nonce":3776252597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6805","nonce":3776252597}]},"public_addr":"172.21.15.92:6801/3776252597","cluster_addr":"172.21.15.92:6803/3776252597","heartbeat_back_addr":"172.21.15.92:6807/3776252597","heartbeat_front_addr":"172.21.15.92:6805/3776252597","state":["exists","up"]},{"osd":2,"uuid":"09808786-3bd3-4d11-8c7b-1736119dfbb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6801","nonce":1601265891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6803","nonce":1601265891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6807","nonce":1601265891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6805","nonce":1601265891}]},"public_addr":"172.21.15.152:6801/1601265891","cluster_addr":"172.21.15.152:6803/1601265891","heartbeat_back_addr":"172.21.15.152:6807/1601265891","heartbeat_front_addr":"172.21.15.152:6805/1601265891","state":["exists","up"]},{"osd":3,"uuid":"79c61910-4b53-424c-b150-6f5385a2cd68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6811","nonce":694846466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6813","nonce":694846466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6817","nonce":694846466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6815","nonce":694846466}]},"public_addr":"172.21.15.86:6811/694846466","cluster_addr":"172.21.15.86:6813/694846466","heartbeat_back_addr":"172.21.15.86:6817/694846466","heartbeat_front_addr":"172.21.15.86:6815/694846466","state":["exists","up"]},{"osd":4,"uuid":"a2878035-0179-469f-bbd4-8ce2d3f6bddc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6809","nonce":1422821471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6811","nonce":1422821471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6815","nonce":1422821471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6813","nonce":1422821471}]},"public_addr":"172.21.15.92:6809/1422821471","cluster_addr":"172.21.15.92:6811/1422821471","heartbeat_back_addr":"172.21.15.92:6815/1422821471","heartbeat_front_addr":"172.21.15.92:6813/1422821471","state":["exists","up"]},{"osd":5,"uuid":"837366aa-ce25-42ca-b157-5bbb50159d0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6809","nonce":1039375239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6811","nonce":1039375239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6815","nonce":1039375239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6813","nonce":1039375239}]},"public_addr":"172.21.15.152:6809/1039375239","cluster_addr":"172.21.15.152:6811/1039375239","heartbeat_back_addr":"172.21.15.152:6815/1039375239","heartbeat_front_addr":"172.21.15.152:6813/1039375239","state":["exists","up"]},{"osd":6,"uuid":"61024437-7b91-432f-a1af-6443d0ef5213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6819","nonce":4080452707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6821","nonce":4080452707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6825","nonce":4080452707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6823","nonce":4080452707}]},"public_addr":"172.21.15.86:6819/4080452707","cluster_addr":"172.21.15.86:6821/4080452707","heartbeat_back_addr":"172.21.15.86:6825/4080452707","heartbeat_front_addr":"172.21.15.86:6823/4080452707","state":["exists","up"]},{"osd":7,"uuid":"2684bdae-6b30-4afc-af68-4ce41b3a75b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6817","nonce":4187707622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6819","nonce":4187707622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6823","nonce":4187707622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6821","nonce":4187707622}]},"public_addr":"172.21.15.92:6817/4187707622","cluster_addr":"172.21.15.92:6819/4187707622","heartbeat_back_addr":"172.21.15.92:6823/4187707622","heartbeat_front_addr":"172.21.15.92:6821/4187707622","state":["exists","up"]},{"osd":8,"uuid":"f03bbf0d-1281-4bcf-9773-9854388e65ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6817","nonce":3237612748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6819","nonce":3237612748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6823","nonce":3237612748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6821","nonce":3237612748}]},"public_addr":"172.21.15.152:6817/3237612748","cluster_addr":"172.21.15.152:6819/3237612748","heartbeat_back_addr":"172.21.15.152:6823/3237612748","heartbeat_front_addr":"172.21.15.152:6821/3237612748","state":["exists","up"]},{"osd":9,"uuid":"e7e8b775-99ff-48a4-a23a-91e7ab6622bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6827","nonce":2835218294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6829","nonce":2835218294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6833","nonce":2835218294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6831","nonce":2835218294}]},"public_addr":"172.21.15.86:6827/2835218294","cluster_addr":"172.21.15.86:6829/2835218294","heartbeat_back_addr":"172.21.15.86:6833/2835218294","heartbeat_front_addr":"172.21.15.86:6831/2835218294","state":["exists","up"]},{"osd":10,"uuid":"d9d616cd-a229-44de-8768-02a4415df115","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6825","nonce":4259465100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6827","nonce":4259465100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6831","nonce":4259465100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6829","nonce":4259465100}]},"public_addr":"172.21.15.92:6825/4259465100","cluster_addr":"172.21.15.92:6827/4259465100","heartbeat_back_addr":"172.21.15.92:6831/4259465100","heartbeat_front_addr":"172.21.15.92:6829/4259465100","state":["exists","up"]},{"osd":11,"uuid":"e7c0b865-ccf5-4233-8d22-0d9b6dbe5c97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6825","nonce":194662760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6827","nonce":194662760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6831","nonce":194662760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6829","nonce":194662760}]},"public_addr":"172.21.15.152:6825/194662760","cluster_addr":"172.21.15.152:6827/194662760","heartbeat_back_addr":"172.21.15.152:6831/194662760","heartbeat_front_addr":"172.21.15.152:6829/194662760","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:44:47.269167+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:45:05.363655+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:45:23.192677+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:45:44.156228+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:02.532971+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:20.153295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:41.713888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:59.676516+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:47:17.207087+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:47:38.970316+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:47:57.282525+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:48:15.437606+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/3553500912":"2024-03-24T17:43:28.242492+0000","172.21.15.86:6801/3600278228":"2024-03-24T17:43:28.242492+0000","172.21.15.86:0/887262329":"2024-03-24T17:43:28.242492+0000","172.21.15.86:6800/2313723492":"2024-03-24T17:43:05.043910+0000","172.21.15.86:6800/3600278228":"2024-03-24T17:43:28.242492+0000","172.21.15.86:0/3815705642":"2024-03-24T17:43:05.043910+0000","172.21.15.86:0/1614943062":"2024-03-24T17:43:28.242492+0000","172.21.15.86:6801/2313723492":"2024-03-24T17:43:05.043910+0000","172.21.15.86:0/2416321716":"2024-03-24T17:43:05.043910+0000","172.21.15.86:0/433588188":"2024-03-24T17:43:05.043910+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T17:49:14.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:14.354+0000 7f3962600700 1 -- 172.21.15.86:0/1609595991 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f39440a1230 msgr2=0x7f39440a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:14.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:14.354+0000 7f3962600700 1 --2- 172.21.15.86:0/1609595991 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f39440a1230 0x7f39440a36f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f395000b5c0 tx=0x7f3950005fd0 comp rx=0 tx=0).stop 2024-03-23T17:49:14.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:14.354+0000 7f3962600700 1 -- 172.21.15.86:0/1609595991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f395c1017c0 msgr2=0x7f395c199830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:14.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:14.354+0000 7f3962600700 1 --2- 172.21.15.86:0/1609595991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f395c1017c0 0x7f395c199830 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f394800d950 tx=0x7f394800dc60 comp rx=0 tx=0).stop 2024-03-23T17:49:14.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:14.354+0000 7f3962600700 1 -- 172.21.15.86:0/1609595991 shutdown_connections 2024-03-23T17:49:14.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:14.354+0000 7f3962600700 1 --2- 172.21.15.86:0/1609595991 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f395c105ef0 0x7f395c19dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:14.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:14.354+0000 7f3962600700 1 --2- 172.21.15.86:0/1609595991 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f395c100e10 0x7f395c1992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:14.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:14.354+0000 7f3962600700 1 --2- 172.21.15.86:0/1609595991 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f39440a1230 0x7f39440a36f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:14.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:14.354+0000 7f3962600700 1 --2- 172.21.15.86:0/1609595991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f395c1017c0 0x7f395c199830 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:14.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:14.354+0000 7f3962600700 1 -- 172.21.15.86:0/1609595991 >> 172.21.15.86:0/1609595991 conn(0x7f395c0fc6e0 msgr2=0x7f395c0fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:14.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:14.354+0000 7f3962600700 1 -- 172.21.15.86:0/1609595991 shutdown_connections 2024-03-23T17:49:14.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:14.354+0000 7f3962600700 1 -- 172.21.15.86:0/1609595991 wait complete. 2024-03-23T17:49:14.795 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-03-23T17:49:14.796 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-03-23T17:49:14.796 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-03-23T17:49:14.797 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-03-23T17:49:14.797 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-03-23T17:49:14.797 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-03-23T17:49:14.798 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-03-23T17:49:14.798 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-03-23T17:49:14.799 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph tell osd.8 flush_pg_stats 2024-03-23T17:49:14.799 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph tell osd.9 flush_pg_stats 2024-03-23T17:49:14.800 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph tell osd.10 flush_pg_stats 2024-03-23T17:49:14.800 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph tell osd.11 flush_pg_stats 2024-03-23T17:49:14.990 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:14 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/1609595991' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T17:49:15.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:14 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/1609595991' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T17:49:15.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:14 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/1609595991' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T17:49:16.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:15 smithi086 ceph-mon[38221]: pgmap v243: 97 pgs: 97 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 8.8 KiB/s wr, 21 op/s 2024-03-23T17:49:16.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:15 smithi092 ceph-mon[79089]: pgmap v243: 97 pgs: 97 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 8.8 KiB/s wr, 21 op/s 2024-03-23T17:49:16.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:15 smithi152 ceph-mon[47949]: pgmap v243: 97 pgs: 97 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 8.8 KiB/s wr, 21 op/s 2024-03-23T17:49:16.185 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T17:49:16.530 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T17:49:16.531 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T17:49:16.546 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T17:49:16.561 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T17:49:16.947 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T17:49:17.203 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T17:49:17.653 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T17:49:17.779 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T17:49:18.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:17 smithi086 ceph-mon[38221]: pgmap v244: 97 pgs: 97 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.0 KiB/s wr, 19 op/s 2024-03-23T17:49:18.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:17 smithi092 ceph-mon[79089]: pgmap v244: 97 pgs: 97 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.0 KiB/s wr, 19 op/s 2024-03-23T17:49:18.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:17 smithi152 ceph-mon[47949]: pgmap v244: 97 pgs: 97 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.0 KiB/s wr, 19 op/s 2024-03-23T17:49:19.017 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T17:49:19.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:19.509+0000 7f2e49000700 1 -- 172.21.15.86:0/3781616210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e44072fe0 msgr2=0x7f2e440733c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:19.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:19.509+0000 7f2e49000700 1 --2- 172.21.15.86:0/3781616210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e44072fe0 0x7f2e440733c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2e34009990 tx=0x7f2e34009ca0 comp rx=0 tx=0).stop 2024-03-23T17:49:19.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:19.509+0000 7f2e49000700 1 -- 172.21.15.86:0/3781616210 shutdown_connections 2024-03-23T17:49:19.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:19.509+0000 7f2e49000700 1 --2- 172.21.15.86:0/3781616210 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e4410d810 0x7f2e4410fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:19.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:19.509+0000 7f2e49000700 1 --2- 172.21.15.86:0/3781616210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e44072fe0 0x7f2e440733c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:19.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:19.509+0000 7f2e49000700 1 --2- 172.21.15.86:0/3781616210 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e44073900 0x7f2e4410d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:19.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:19.509+0000 7f2e49000700 1 -- 172.21.15.86:0/3781616210 >> 172.21.15.86:0/3781616210 conn(0x7f2e44077fa0 msgr2=0x7f2e440783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:19.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:19.509+0000 7f2e49000700 1 -- 172.21.15.86:0/3781616210 shutdown_connections 2024-03-23T17:49:19.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:19.509+0000 7f2e49000700 1 -- 172.21.15.86:0/3781616210 wait complete. 2024-03-23T17:49:19.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:19.510+0000 7f2e49000700 1 Processor -- start 2024-03-23T17:49:19.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:19.510+0000 7f2e49000700 1 -- start start 2024-03-23T17:49:19.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:19.511+0000 7f2e49000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e44072fe0 0x7f2e44072360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:19.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:19.511+0000 7f2e49000700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e44073900 0x7f2e440728a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:19.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:19.511+0000 7f2e49000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e4410d810 0x7f2e4406fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:19.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:19.511+0000 7f2e49000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e44070670 con 0x7f2e44072fe0 2024-03-23T17:49:19.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:19.511+0000 7f2e49000700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e440707e0 con 0x7f2e4410d810 2024-03-23T17:49:19.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:19.511+0000 7f2e49000700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e440709a0 con 0x7f2e44073900 2024-03-23T17:49:19.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:19.511+0000 7f2e41e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e44072fe0 0x7f2e44072360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:19.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:19.511+0000 7f2e41e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e44072fe0 0x7f2e44072360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60086/0 (socket says 172.21.15.86:60086) 2024-03-23T17:49:19.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:19.511+0000 7f2e41e00700 1 -- 172.21.15.86:0/3238822969 learned_addr learned my addr 172.21.15.86:0/3238822969 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:19.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:19.511+0000 7f2e41400700 1 --2- 172.21.15.86:0/3238822969 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e44073900 0x7f2e440728a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:19.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:19.511+0000 7f2e42800700 1 --2- 172.21.15.86:0/3238822969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e4410d810 0x7f2e4406fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:19.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:19.512+0000 7f2e41e00700 1 -- 172.21.15.86:0/3238822969 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e44073900 msgr2=0x7f2e440728a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:19.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:19.512+0000 7f2e41e00700 1 --2- 172.21.15.86:0/3238822969 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e44073900 0x7f2e440728a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:19.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:19.512+0000 7f2e41e00700 1 -- 172.21.15.86:0/3238822969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e4410d810 msgr2=0x7f2e4406fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:19.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:19.512+0000 7f2e41e00700 1 --2- 172.21.15.86:0/3238822969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e4410d810 0x7f2e4406fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:19.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:19.512+0000 7f2e41e00700 1 -- 172.21.15.86:0/3238822969 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e340095f0 con 0x7f2e44072fe0 2024-03-23T17:49:19.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:19.512+0000 7f2e41e00700 1 --2- 172.21.15.86:0/3238822969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e44072fe0 0x7f2e44072360 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2e34000c00 tx=0x7f2e34004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:19.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:19.512+0000 7f2e3aa00700 1 -- 172.21.15.86:0/3238822969 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3401d070 con 0x7f2e44072fe0 2024-03-23T17:49:19.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:19.512+0000 7f2e49000700 1 -- 172.21.15.86:0/3238822969 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e44070c20 con 0x7f2e44072fe0 2024-03-23T17:49:19.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:19.512+0000 7f2e3aa00700 1 -- 172.21.15.86:0/3238822969 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e34022470 con 0x7f2e44072fe0 2024-03-23T17:49:19.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:19.512+0000 7f2e3aa00700 1 -- 172.21.15.86:0/3238822969 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e34017650 con 0x7f2e44072fe0 2024-03-23T17:49:19.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:19.512+0000 7f2e49000700 1 -- 172.21.15.86:0/3238822969 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e4406b690 con 0x7f2e44072fe0 2024-03-23T17:49:19.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:19.513+0000 7f2e49000700 1 -- 172.21.15.86:0/3238822969 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2e44113250 con 0x7f2e44072fe0 2024-03-23T17:49:19.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:19.515+0000 7f2e3aa00700 1 -- 172.21.15.86:0/3238822969 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f2e340225e0 con 0x7f2e44072fe0 2024-03-23T17:49:19.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:19.517+0000 7f2e3aa00700 1 --2- 172.21.15.86:0/3238822969 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f2e280a11b0 0x7f2e280a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:19.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:19.517+0000 7f2e3aa00700 1 -- 172.21.15.86:0/3238822969 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f2e340ce390 con 0x7f2e44072fe0 2024-03-23T17:49:19.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:19.517+0000 7f2e3aa00700 1 --2- 172.21.15.86:0/3238822969 >> [v2:172.21.15.86:6802/2423505491,v1:172.21.15.86:6803/2423505491] conn(0x7f2e280aa990 0x7f2e280acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:19.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:19.518+0000 7f2e3aa00700 1 -- 172.21.15.86:0/3238822969 --> [v2:172.21.15.86:6802/2423505491,v1:172.21.15.86:6803/2423505491] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2e280ad460 con 0x7f2e280aa990 2024-03-23T17:49:19.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:19.518+0000 7f2e3aa00700 1 -- 172.21.15.86:0/3238822969 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2e340ce700 con 0x7f2e44072fe0 2024-03-23T17:49:19.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:19.518+0000 7f2e41400700 1 --2- 172.21.15.86:0/3238822969 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f2e280a11b0 0x7f2e280a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:19.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:19.518+0000 7f2e42800700 1 --2- 172.21.15.86:0/3238822969 >> [v2:172.21.15.86:6802/2423505491,v1:172.21.15.86:6803/2423505491] conn(0x7f2e280aa990 0x7f2e280acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:19.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:19.519+0000 7f2e41400700 1 --2- 172.21.15.86:0/3238822969 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f2e280a11b0 0x7f2e280a3670 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2e4406cb30 tx=0x7f2e30009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:19.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:19.519+0000 7f2e42800700 1 --2- 172.21.15.86:0/3238822969 >> [v2:172.21.15.86:6802/2423505491,v1:172.21.15.86:6803/2423505491] conn(0x7f2e280aa990 0x7f2e280acdb0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:19.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:19.520+0000 7f2e3aa00700 1 -- 172.21.15.86:0/3238822969 <== osd.0 v2:172.21.15.86:6802/2423505491 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f2e280ad460 con 0x7f2e280aa990 2024-03-23T17:49:19.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:19.538+0000 7f2e49000700 1 -- 172.21.15.86:0/3238822969 --> [v2:172.21.15.86:6802/2423505491,v1:172.21.15.86:6803/2423505491] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2e44074670 con 0x7f2e280aa990 2024-03-23T17:49:19.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:19.539+0000 7f2e3aa00700 1 -- 172.21.15.86:0/3238822969 <== osd.0 v2:172.21.15.86:6802/2423505491 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2e44074670 con 0x7f2e280aa990 2024-03-23T17:49:19.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:19.539+0000 7f2e49000700 1 -- 172.21.15.86:0/3238822969 >> [v2:172.21.15.86:6802/2423505491,v1:172.21.15.86:6803/2423505491] conn(0x7f2e280aa990 msgr2=0x7f2e280acdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:19.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:19.539+0000 7f2e49000700 1 --2- 172.21.15.86:0/3238822969 >> [v2:172.21.15.86:6802/2423505491,v1:172.21.15.86:6803/2423505491] conn(0x7f2e280aa990 0x7f2e280acdb0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:19.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:19.539+0000 7f2e49000700 1 -- 172.21.15.86:0/3238822969 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f2e280a11b0 msgr2=0x7f2e280a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:19.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:19.539+0000 7f2e49000700 1 --2- 172.21.15.86:0/3238822969 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f2e280a11b0 0x7f2e280a3670 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2e4406cb30 tx=0x7f2e30009260 comp rx=0 tx=0).stop 2024-03-23T17:49:19.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:19.539+0000 7f2e49000700 1 -- 172.21.15.86:0/3238822969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e44072fe0 msgr2=0x7f2e44072360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:19.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:19.539+0000 7f2e49000700 1 --2- 172.21.15.86:0/3238822969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e44072fe0 0x7f2e44072360 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2e34000c00 tx=0x7f2e34004730 comp rx=0 tx=0).stop 2024-03-23T17:49:19.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:19.539+0000 7f2e42800700 1 -- 172.21.15.86:0/3238822969 reap_dead start 2024-03-23T17:49:19.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:19.539+0000 7f2e49000700 1 -- 172.21.15.86:0/3238822969 shutdown_connections 2024-03-23T17:49:19.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:19.539+0000 7f2e49000700 1 -- 172.21.15.86:0/3238822969 >> 172.21.15.86:0/3238822969 conn(0x7f2e44077fa0 msgr2=0x7f2e440fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:19.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:19.539+0000 7f2e49000700 1 -- 172.21.15.86:0/3238822969 shutdown_connections 2024-03-23T17:49:19.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:19.539+0000 7f2e49000700 1 -- 172.21.15.86:0/3238822969 wait complete. 2024-03-23T17:49:19.595 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T17:49:19.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:19 smithi086 ceph-mon[38221]: pgmap v245: 97 pgs: 97 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-03-23T17:49:20.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.044+0000 7fbae7800700 1 -- 172.21.15.86:0/1601705235 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbae0106ef0 msgr2=0x7fbae01072d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:20.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.044+0000 7fbae7800700 1 --2- 172.21.15.86:0/1601705235 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbae0106ef0 0x7fbae01072d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fbad40099c0 tx=0x7fbad4009cd0 comp rx=0 tx=0).stop 2024-03-23T17:49:20.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.044+0000 7fbae7800700 1 -- 172.21.15.86:0/1601705235 shutdown_connections 2024-03-23T17:49:20.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.044+0000 7fbae7800700 1 --2- 172.21.15.86:0/1601705235 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbae0101820 0x7fbae010dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:20.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.044+0000 7fbae7800700 1 --2- 172.21.15.86:0/1601705235 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbae0106ef0 0x7fbae01072d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:20.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.044+0000 7fbae7800700 1 --2- 172.21.15.86:0/1601705235 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbae0100e60 0x7fbae01012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:20.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.044+0000 7fbae7800700 1 -- 172.21.15.86:0/1601705235 >> 172.21.15.86:0/1601705235 conn(0x7fbae0077fa0 msgr2=0x7fbae00783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:20.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.044+0000 7fbae7800700 1 -- 172.21.15.86:0/1601705235 shutdown_connections 2024-03-23T17:49:20.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.044+0000 7fbae7800700 1 -- 172.21.15.86:0/1601705235 wait complete. 2024-03-23T17:49:20.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.045+0000 7fbae7800700 1 Processor -- start 2024-03-23T17:49:20.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.045+0000 7fbae7800700 1 -- start start 2024-03-23T17:49:20.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.045+0000 7fbae7800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbae0100e60 0x7fbae00730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:20.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.045+0000 7fbae7800700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbae0101820 0x7fbae00735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:20.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.045+0000 7fbae7800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbae0106ef0 0x7fbae0073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:20.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.045+0000 7fbae7800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae0075020 con 0x7fbae0100e60 2024-03-23T17:49:20.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.046+0000 7fbae7800700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae01a9bd0 con 0x7fbae0106ef0 2024-03-23T17:49:20.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.046+0000 7fbae7800700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae01a9d40 con 0x7fbae0101820 2024-03-23T17:49:20.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.046+0000 7fbae4e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbae0100e60 0x7fbae00730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:20.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.046+0000 7fbae4e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbae0100e60 0x7fbae00730b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60116/0 (socket says 172.21.15.86:60116) 2024-03-23T17:49:20.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.046+0000 7fbae4e00700 1 -- 172.21.15.86:0/1904453887 learned_addr learned my addr 172.21.15.86:0/1904453887 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:20.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.046+0000 7fbadfe00700 1 --2- 172.21.15.86:0/1904453887 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbae0101820 0x7fbae00735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:20.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.046+0000 7fbae5800700 1 --2- 172.21.15.86:0/1904453887 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbae0106ef0 0x7fbae0073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:20.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.046+0000 7fbae4e00700 1 -- 172.21.15.86:0/1904453887 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbae0101820 msgr2=0x7fbae00735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:20.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.046+0000 7fbae4e00700 1 --2- 172.21.15.86:0/1904453887 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbae0101820 0x7fbae00735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:20.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.046+0000 7fbae4e00700 1 -- 172.21.15.86:0/1904453887 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbae0106ef0 msgr2=0x7fbae0073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:20.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.046+0000 7fbae4e00700 1 --2- 172.21.15.86:0/1904453887 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbae0106ef0 0x7fbae0073b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:20.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.046+0000 7fbae4e00700 1 -- 172.21.15.86:0/1904453887 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbad40095f0 con 0x7fbae0100e60 2024-03-23T17:49:20.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.046+0000 7fbae5800700 1 --2- 172.21.15.86:0/1904453887 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbae0106ef0 0x7fbae0073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T17:49:20.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.046+0000 7fbae4e00700 1 --2- 172.21.15.86:0/1904453887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbae0100e60 0x7fbae00730b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fbad400b580 tx=0x7fbad400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:20.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.047+0000 7fbadd600700 1 -- 172.21.15.86:0/1904453887 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad401c070 con 0x7fbae0100e60 2024-03-23T17:49:20.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.047+0000 7fbadd600700 1 -- 172.21.15.86:0/1904453887 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbad400fe30 con 0x7fbae0100e60 2024-03-23T17:49:20.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.047+0000 7fbae7800700 1 -- 172.21.15.86:0/1904453887 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbae01a9fc0 con 0x7fbae0100e60 2024-03-23T17:49:20.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.047+0000 7fbadd600700 1 -- 172.21.15.86:0/1904453887 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad4022d40 con 0x7fbae0100e60 2024-03-23T17:49:20.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.047+0000 7fbae7800700 1 -- 172.21.15.86:0/1904453887 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbae01aa320 con 0x7fbae0100e60 2024-03-23T17:49:20.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.048+0000 7fbae7800700 1 -- 172.21.15.86:0/1904453887 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbae01024f0 con 0x7fbae0100e60 2024-03-23T17:49:20.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.050+0000 7fbadd600700 1 -- 172.21.15.86:0/1904453887 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fbad4020070 con 0x7fbae0100e60 2024-03-23T17:49:20.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.051+0000 7fbadd600700 1 --2- 172.21.15.86:0/1904453887 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fbac80a11b0 0x7fbac80a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:20.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.051+0000 7fbadd600700 1 -- 172.21.15.86:0/1904453887 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fbad40cdb90 con 0x7fbae0100e60 2024-03-23T17:49:20.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.051+0000 7fbadfe00700 1 --2- 172.21.15.86:0/1904453887 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fbac80a11b0 0x7fbac80a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:20.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.051+0000 7fbadd600700 1 --2- 172.21.15.86:0/1904453887 >> [v2:172.21.15.92:6816/4187707622,v1:172.21.15.92:6817/4187707622] conn(0x7fbac80aa990 0x7fbac80acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:20.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.051+0000 7fbadd600700 1 -- 172.21.15.86:0/1904453887 --> [v2:172.21.15.92:6816/4187707622,v1:172.21.15.92:6817/4187707622] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbac80ad460 con 0x7fbac80aa990 2024-03-23T17:49:20.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.051+0000 7fbadd600700 1 -- 172.21.15.86:0/1904453887 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbad40cdf00 con 0x7fbae0100e60 2024-03-23T17:49:20.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.052+0000 7fbadfe00700 1 --2- 172.21.15.86:0/1904453887 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fbac80a11b0 0x7fbac80a3670 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fbae0074790 tx=0x7fbacc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:20.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.052+0000 7fbae5800700 1 --2- 172.21.15.86:0/1904453887 >> [v2:172.21.15.92:6816/4187707622,v1:172.21.15.92:6817/4187707622] conn(0x7fbac80aa990 0x7fbac80acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:20.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.052+0000 7fbae5800700 1 --2- 172.21.15.86:0/1904453887 >> [v2:172.21.15.92:6816/4187707622,v1:172.21.15.92:6817/4187707622] conn(0x7fbac80aa990 0x7fbac80acdb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:20.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.053+0000 7fbadd600700 1 -- 172.21.15.86:0/1904453887 <== osd.7 v2:172.21.15.92:6816/4187707622 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fbac80ad460 con 0x7fbac80aa990 2024-03-23T17:49:20.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:19 smithi092 ceph-mon[79089]: pgmap v245: 97 pgs: 97 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-03-23T17:49:20.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.072+0000 7fbae7800700 1 -- 172.21.15.86:0/1904453887 --> [v2:172.21.15.92:6816/4187707622,v1:172.21.15.92:6817/4187707622] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbae0110fd0 con 0x7fbac80aa990 2024-03-23T17:49:20.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.073+0000 7fbadd600700 1 -- 172.21.15.86:0/1904453887 <== osd.7 v2:172.21.15.92:6816/4187707622 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fbae0110fd0 con 0x7fbac80aa990 2024-03-23T17:49:20.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.073+0000 7fbae7800700 1 -- 172.21.15.86:0/1904453887 >> [v2:172.21.15.92:6816/4187707622,v1:172.21.15.92:6817/4187707622] conn(0x7fbac80aa990 msgr2=0x7fbac80acdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:20.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.073+0000 7fbae7800700 1 --2- 172.21.15.86:0/1904453887 >> [v2:172.21.15.92:6816/4187707622,v1:172.21.15.92:6817/4187707622] conn(0x7fbac80aa990 0x7fbac80acdb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:20.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.073+0000 7fbae7800700 1 -- 172.21.15.86:0/1904453887 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fbac80a11b0 msgr2=0x7fbac80a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:20.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.073+0000 7fbae7800700 1 --2- 172.21.15.86:0/1904453887 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fbac80a11b0 0x7fbac80a3670 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fbae0074790 tx=0x7fbacc009260 comp rx=0 tx=0).stop 2024-03-23T17:49:20.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.073+0000 7fbae7800700 1 -- 172.21.15.86:0/1904453887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbae0100e60 msgr2=0x7fbae00730b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:20.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.073+0000 7fbae7800700 1 --2- 172.21.15.86:0/1904453887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbae0100e60 0x7fbae00730b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fbad400b580 tx=0x7fbad400f690 comp rx=0 tx=0).stop 2024-03-23T17:49:20.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.073+0000 7fbae5800700 1 -- 172.21.15.86:0/1904453887 reap_dead start 2024-03-23T17:49:20.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.073+0000 7fbae7800700 1 -- 172.21.15.86:0/1904453887 shutdown_connections 2024-03-23T17:49:20.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.073+0000 7fbae7800700 1 -- 172.21.15.86:0/1904453887 >> 172.21.15.86:0/1904453887 conn(0x7fbae0077fa0 msgr2=0x7fbae00ffc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:20.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.074+0000 7fbae7800700 1 -- 172.21.15.86:0/1904453887 shutdown_connections 2024-03-23T17:49:20.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.074+0000 7fbae7800700 1 -- 172.21.15.86:0/1904453887 wait complete. 2024-03-23T17:49:20.085 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:19 smithi152 ceph-mon[47949]: pgmap v245: 97 pgs: 97 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-03-23T17:49:20.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.718+0000 7f06fa400700 1 -- 172.21.15.86:0/328034307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06f40731e0 msgr2=0x7f06f40735c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:20.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.718+0000 7f06fa400700 1 --2- 172.21.15.86:0/328034307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06f40731e0 0x7f06f40735c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f06e8009990 tx=0x7f06e8009ca0 comp rx=0 tx=0).stop 2024-03-23T17:49:20.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.718+0000 7f06fa400700 1 -- 172.21.15.86:0/328034307 shutdown_connections 2024-03-23T17:49:20.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.718+0000 7f06fa400700 1 --2- 172.21.15.86:0/328034307 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f06f410d3f0 0x7f06f410f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:20.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.718+0000 7f06fa400700 1 --2- 172.21.15.86:0/328034307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06f40731e0 0x7f06f40735c0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:20.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.718+0000 7f06fa400700 1 --2- 172.21.15.86:0/328034307 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f06f4073b00 0x7f06f410ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:20.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.718+0000 7f06fa400700 1 -- 172.21.15.86:0/328034307 >> 172.21.15.86:0/328034307 conn(0x7f06f40fc6e0 msgr2=0x7f06f40feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:20.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.718+0000 7f06fa400700 1 -- 172.21.15.86:0/328034307 shutdown_connections 2024-03-23T17:49:20.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.718+0000 7f06fa400700 1 -- 172.21.15.86:0/328034307 wait complete. 2024-03-23T17:49:20.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.719+0000 7f06fa400700 1 Processor -- start 2024-03-23T17:49:20.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.719+0000 7f06fa400700 1 -- start start 2024-03-23T17:49:20.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.720+0000 7f06fa400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06f40731e0 0x7f06f4199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:20.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.720+0000 7f06fa400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f06f4073b00 0x7f06f41997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:20.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.720+0000 7f06fa400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f06f410d3f0 0x7f06f419db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:20.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.720+0000 7f06fa400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06f419e160 con 0x7f06f40731e0 2024-03-23T17:49:20.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.720+0000 7f06fa400700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06f419e2d0 con 0x7f06f4073b00 2024-03-23T17:49:20.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.720+0000 7f06fa400700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06f419e490 con 0x7f06f410d3f0 2024-03-23T17:49:20.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.720+0000 7f06f9000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06f40731e0 0x7f06f4199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:20.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.720+0000 7f06f9000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06f40731e0 0x7f06f4199260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60128/0 (socket says 172.21.15.86:60128) 2024-03-23T17:49:20.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.720+0000 7f06f3e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f06f4073b00 0x7f06f41997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:20.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.720+0000 7f06f9000700 1 -- 172.21.15.86:0/3550402372 learned_addr learned my addr 172.21.15.86:0/3550402372 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:20.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.720+0000 7f06f9a00700 1 --2- 172.21.15.86:0/3550402372 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f06f410d3f0 0x7f06f419db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:20.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.720+0000 7f06f3e00700 1 -- 172.21.15.86:0/3550402372 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f06f410d3f0 msgr2=0x7f06f419db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:20.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.720+0000 7f06f3e00700 1 --2- 172.21.15.86:0/3550402372 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f06f410d3f0 0x7f06f419db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:20.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.720+0000 7f06f3e00700 1 -- 172.21.15.86:0/3550402372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06f40731e0 msgr2=0x7f06f4199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:20.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.720+0000 7f06f3e00700 1 --2- 172.21.15.86:0/3550402372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06f40731e0 0x7f06f4199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:20.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.720+0000 7f06f3e00700 1 -- 172.21.15.86:0/3550402372 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06e80095f0 con 0x7f06f4073b00 2024-03-23T17:49:20.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.720+0000 7f06f9000700 1 --2- 172.21.15.86:0/3550402372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06f40731e0 0x7f06f4199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:49:20.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.721+0000 7f06f3e00700 1 --2- 172.21.15.86:0/3550402372 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f06f4073b00 0x7f06f41997a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f06e000d950 tx=0x7f06e000dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:20.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.721+0000 7f06f1600700 1 -- 172.21.15.86:0/3550402372 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e000f890 con 0x7f06f4073b00 2024-03-23T17:49:20.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.721+0000 7f06fa400700 1 -- 172.21.15.86:0/3550402372 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06f419e770 con 0x7f06f4073b00 2024-03-23T17:49:20.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.721+0000 7f06f1600700 1 -- 172.21.15.86:0/3550402372 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f06e0010460 con 0x7f06f4073b00 2024-03-23T17:49:20.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.721+0000 7f06f1600700 1 -- 172.21.15.86:0/3550402372 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e000e610 con 0x7f06f4073b00 2024-03-23T17:49:20.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.721+0000 7f06fa400700 1 -- 172.21.15.86:0/3550402372 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06f41a5fd0 con 0x7f06f4073b00 2024-03-23T17:49:20.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.722+0000 7f06fa400700 1 -- 172.21.15.86:0/3550402372 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f06f4112fd0 con 0x7f06f4073b00 2024-03-23T17:49:20.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.724+0000 7f06f1600700 1 -- 172.21.15.86:0/3550402372 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f06e00105d0 con 0x7f06f4073b00 2024-03-23T17:49:20.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.725+0000 7f06f1600700 1 --2- 172.21.15.86:0/3550402372 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f06dc0a1230 0x7f06dc0a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:20.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.725+0000 7f06f1600700 1 -- 172.21.15.86:0/3550402372 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f06e00cc670 con 0x7f06f4073b00 2024-03-23T17:49:20.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.725+0000 7f06f9000700 1 --2- 172.21.15.86:0/3550402372 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f06dc0a1230 0x7f06dc0a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:20.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.725+0000 7f06f1600700 1 --2- 172.21.15.86:0/3550402372 >> [v2:172.21.15.86:6818/4080452707,v1:172.21.15.86:6819/4080452707] conn(0x7f06dc0aaa10 0x7f06dc0ace30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:20.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.725+0000 7f06f1600700 1 -- 172.21.15.86:0/3550402372 --> [v2:172.21.15.86:6818/4080452707,v1:172.21.15.86:6819/4080452707] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f06dc0ad4e0 con 0x7f06dc0aaa10 2024-03-23T17:49:20.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.725+0000 7f06f1600700 1 -- 172.21.15.86:0/3550402372 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f06e00cc9e0 con 0x7f06f4073b00 2024-03-23T17:49:20.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.726+0000 7f06f9a00700 1 --2- 172.21.15.86:0/3550402372 >> [v2:172.21.15.86:6818/4080452707,v1:172.21.15.86:6819/4080452707] conn(0x7f06dc0aaa10 0x7f06dc0ace30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:20.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.726+0000 7f06f9000700 1 --2- 172.21.15.86:0/3550402372 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f06dc0a1230 0x7f06dc0a36f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f06e800bc60 tx=0x7f06e801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:20.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.726+0000 7f06f9a00700 1 --2- 172.21.15.86:0/3550402372 >> [v2:172.21.15.86:6818/4080452707,v1:172.21.15.86:6819/4080452707] conn(0x7f06dc0aaa10 0x7f06dc0ace30 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:20.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.727+0000 7f06f1600700 1 -- 172.21.15.86:0/3550402372 <== osd.6 v2:172.21.15.86:6818/4080452707 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f06dc0ad4e0 con 0x7f06dc0aaa10 2024-03-23T17:49:20.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.743+0000 7f06fa400700 1 -- 172.21.15.86:0/3550402372 --> [v2:172.21.15.86:6818/4080452707,v1:172.21.15.86:6819/4080452707] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f06f4061430 con 0x7f06dc0aaa10 2024-03-23T17:49:20.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.744+0000 7f06f1600700 1 -- 172.21.15.86:0/3550402372 <== osd.6 v2:172.21.15.86:6818/4080452707 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f06f4061430 con 0x7f06dc0aaa10 2024-03-23T17:49:20.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.744+0000 7f06fa400700 1 -- 172.21.15.86:0/3550402372 >> [v2:172.21.15.86:6818/4080452707,v1:172.21.15.86:6819/4080452707] conn(0x7f06dc0aaa10 msgr2=0x7f06dc0ace30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:20.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.744+0000 7f06fa400700 1 --2- 172.21.15.86:0/3550402372 >> [v2:172.21.15.86:6818/4080452707,v1:172.21.15.86:6819/4080452707] conn(0x7f06dc0aaa10 0x7f06dc0ace30 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:20.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.744+0000 7f06fa400700 1 -- 172.21.15.86:0/3550402372 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f06dc0a1230 msgr2=0x7f06dc0a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:20.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.744+0000 7f06fa400700 1 --2- 172.21.15.86:0/3550402372 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f06dc0a1230 0x7f06dc0a36f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f06e800bc60 tx=0x7f06e801a040 comp rx=0 tx=0).stop 2024-03-23T17:49:20.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.744+0000 7f06fa400700 1 -- 172.21.15.86:0/3550402372 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f06f4073b00 msgr2=0x7f06f41997a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:20.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.744+0000 7f06fa400700 1 --2- 172.21.15.86:0/3550402372 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f06f4073b00 0x7f06f41997a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f06e000d950 tx=0x7f06e000dc60 comp rx=0 tx=0).stop 2024-03-23T17:49:20.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.744+0000 7f06f9a00700 1 -- 172.21.15.86:0/3550402372 reap_dead start 2024-03-23T17:49:20.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.745+0000 7f06fa400700 1 -- 172.21.15.86:0/3550402372 shutdown_connections 2024-03-23T17:49:20.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.745+0000 7f06fa400700 1 -- 172.21.15.86:0/3550402372 >> 172.21.15.86:0/3550402372 conn(0x7f06f40fc6e0 msgr2=0x7f06f40fd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:20.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.745+0000 7f06fa400700 1 -- 172.21.15.86:0/3550402372 shutdown_connections 2024-03-23T17:49:20.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:20.745+0000 7f06fa400700 1 -- 172.21.15.86:0/3550402372 wait complete. 2024-03-23T17:49:21.191 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T17:49:21.298 INFO:teuthology.orchestra.run.smithi086.stdout:34359738425 2024-03-23T17:49:21.299 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-03-23T17:49:21.848 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:21 smithi086 ceph-mon[38221]: pgmap v246: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-03-23T17:49:22.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:21 smithi092 ceph-mon[79089]: pgmap v246: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-03-23T17:49:22.062 INFO:teuthology.orchestra.run.smithi086.stdout:210453397534 2024-03-23T17:49:22.062 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-03-23T17:49:22.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:21 smithi152 ceph-mon[47949]: pgmap v246: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-03-23T17:49:22.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.097+0000 7f445c000700 1 -- 172.21.15.86:0/1265741436 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f445410d810 msgr2=0x7f445410fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:22.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.097+0000 7f445c000700 1 --2- 172.21.15.86:0/1265741436 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f445410d810 0x7f445410fc00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f444c009f90 tx=0x7f444c00b330 comp rx=0 tx=0).stop 2024-03-23T17:49:22.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.097+0000 7f445c000700 1 -- 172.21.15.86:0/1265741436 shutdown_connections 2024-03-23T17:49:22.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.097+0000 7f445c000700 1 --2- 172.21.15.86:0/1265741436 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f445410d810 0x7f445410fc00 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:22.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.097+0000 7f445c000700 1 --2- 172.21.15.86:0/1265741436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4454072fe0 0x7f44540733c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:22.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.097+0000 7f445c000700 1 --2- 172.21.15.86:0/1265741436 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4454073900 0x7f445410d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:22.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.097+0000 7f445c000700 1 -- 172.21.15.86:0/1265741436 >> 172.21.15.86:0/1265741436 conn(0x7f4454077fa0 msgr2=0x7f44540783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:22.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.097+0000 7f445c000700 1 -- 172.21.15.86:0/1265741436 shutdown_connections 2024-03-23T17:49:22.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.097+0000 7f445c000700 1 -- 172.21.15.86:0/1265741436 wait complete. 2024-03-23T17:49:22.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.098+0000 7f445c000700 1 Processor -- start 2024-03-23T17:49:22.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.099+0000 7f445c000700 1 -- start start 2024-03-23T17:49:22.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.099+0000 7f445c000700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4454072fe0 0x7f4454199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:22.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.099+0000 7f445c000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4454073900 0x7f4454199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:22.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.099+0000 7f445c000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f445410d810 0x7f445419df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:22.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.099+0000 7f445c000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f445419e520 con 0x7f4454073900 2024-03-23T17:49:22.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.099+0000 7f445c000700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f445419e690 con 0x7f445410d810 2024-03-23T17:49:22.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.099+0000 7f445c000700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f445419e850 con 0x7f4454072fe0 2024-03-23T17:49:22.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.099+0000 7f4459600700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4454072fe0 0x7f4454199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:22.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.099+0000 7f4458c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4454073900 0x7f4454199b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:22.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.099+0000 7f4459600700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4454072fe0 0x7f4454199620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:60072/0 (socket says 172.21.15.86:60072) 2024-03-23T17:49:22.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.099+0000 7f4459600700 1 -- 172.21.15.86:0/639022551 learned_addr learned my addr 172.21.15.86:0/639022551 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:22.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.100+0000 7f445a000700 1 --2- 172.21.15.86:0/639022551 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f445410d810 0x7f445419df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:22.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.100+0000 7f4458c00700 1 -- 172.21.15.86:0/639022551 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4454072fe0 msgr2=0x7f4454199620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:22.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.100+0000 7f4458c00700 1 --2- 172.21.15.86:0/639022551 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4454072fe0 0x7f4454199620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:22.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.100+0000 7f4458c00700 1 -- 172.21.15.86:0/639022551 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f445410d810 msgr2=0x7f445419df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:22.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.100+0000 7f4458c00700 1 --2- 172.21.15.86:0/639022551 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f445410d810 0x7f445419df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:22.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.100+0000 7f4458c00700 1 -- 172.21.15.86:0/639022551 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44400095f0 con 0x7f4454073900 2024-03-23T17:49:22.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.100+0000 7f445a000700 1 --2- 172.21.15.86:0/639022551 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f445410d810 0x7f445419df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:49:22.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.100+0000 7f4459600700 1 --2- 172.21.15.86:0/639022551 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4454072fe0 0x7f4454199620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:49:22.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.100+0000 7f4458c00700 1 --2- 172.21.15.86:0/639022551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4454073900 0x7f4454199b60 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f444000eb90 tx=0x7f444000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:22.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.101+0000 7f4452000700 1 -- 172.21.15.86:0/639022551 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4440009970 con 0x7f4454073900 2024-03-23T17:49:22.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.101+0000 7f4452000700 1 -- 172.21.15.86:0/639022551 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f44400044e0 con 0x7f4454073900 2024-03-23T17:49:22.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.101+0000 7f4452000700 1 -- 172.21.15.86:0/639022551 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4440003e50 con 0x7f4454073900 2024-03-23T17:49:22.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.101+0000 7f445c000700 1 -- 172.21.15.86:0/639022551 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f444c009c40 con 0x7f4454073900 2024-03-23T17:49:22.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.101+0000 7f445c000700 1 -- 172.21.15.86:0/639022551 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44541a61c0 con 0x7f4454073900 2024-03-23T17:49:22.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.102+0000 7f445c000700 1 -- 172.21.15.86:0/639022551 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4454113290 con 0x7f4454073900 2024-03-23T17:49:22.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.104+0000 7f4452000700 1 -- 172.21.15.86:0/639022551 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f4440004000 con 0x7f4454073900 2024-03-23T17:49:22.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.105+0000 7f4452000700 1 --2- 172.21.15.86:0/639022551 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f443c0a11b0 0x7f443c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:22.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.106+0000 7f4452000700 1 -- 172.21.15.86:0/639022551 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f4440014070 con 0x7f4454073900 2024-03-23T17:49:22.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.106+0000 7f4459600700 1 --2- 172.21.15.86:0/639022551 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f443c0a11b0 0x7f443c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:22.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.106+0000 7f4452000700 1 --2- 172.21.15.86:0/639022551 >> [v2:172.21.15.92:6808/1422821471,v1:172.21.15.92:6809/1422821471] conn(0x7f443c0aa990 0x7f443c0acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:22.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.106+0000 7f4452000700 1 -- 172.21.15.86:0/639022551 --> [v2:172.21.15.92:6808/1422821471,v1:172.21.15.92:6809/1422821471] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f443c0ad460 con 0x7f443c0aa990 2024-03-23T17:49:22.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.106+0000 7f4452000700 1 -- 172.21.15.86:0/639022551 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f44400d1220 con 0x7f4454073900 2024-03-23T17:49:22.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.106+0000 7f445a000700 1 --2- 172.21.15.86:0/639022551 >> [v2:172.21.15.92:6808/1422821471,v1:172.21.15.92:6809/1422821471] conn(0x7f443c0aa990 0x7f443c0acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:22.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.106+0000 7f4459600700 1 --2- 172.21.15.86:0/639022551 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f443c0a11b0 0x7f443c0a3670 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4448011550 tx=0x7f4448009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:22.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.107+0000 7f445a000700 1 --2- 172.21.15.86:0/639022551 >> [v2:172.21.15.92:6808/1422821471,v1:172.21.15.92:6809/1422821471] conn(0x7f443c0aa990 0x7f443c0acdb0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:22.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.108+0000 7f4452000700 1 -- 172.21.15.86:0/639022551 <== osd.4 v2:172.21.15.92:6808/1422821471 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f443c0ad460 con 0x7f443c0aa990 2024-03-23T17:49:22.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.126+0000 7f445c000700 1 -- 172.21.15.86:0/639022551 --> [v2:172.21.15.92:6808/1422821471,v1:172.21.15.92:6809/1422821471] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4454061430 con 0x7f443c0aa990 2024-03-23T17:49:22.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.126+0000 7f4452000700 1 -- 172.21.15.86:0/639022551 <== osd.4 v2:172.21.15.92:6808/1422821471 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f4454061430 con 0x7f443c0aa990 2024-03-23T17:49:22.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.127+0000 7f445c000700 1 -- 172.21.15.86:0/639022551 >> [v2:172.21.15.92:6808/1422821471,v1:172.21.15.92:6809/1422821471] conn(0x7f443c0aa990 msgr2=0x7f443c0acdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:22.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.127+0000 7f445c000700 1 --2- 172.21.15.86:0/639022551 >> [v2:172.21.15.92:6808/1422821471,v1:172.21.15.92:6809/1422821471] conn(0x7f443c0aa990 0x7f443c0acdb0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:22.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.127+0000 7f445c000700 1 -- 172.21.15.86:0/639022551 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f443c0a11b0 msgr2=0x7f443c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:22.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.127+0000 7f445c000700 1 --2- 172.21.15.86:0/639022551 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f443c0a11b0 0x7f443c0a3670 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4448011550 tx=0x7f4448009260 comp rx=0 tx=0).stop 2024-03-23T17:49:22.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.127+0000 7f445c000700 1 -- 172.21.15.86:0/639022551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4454073900 msgr2=0x7f4454199b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:22.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.127+0000 7f445c000700 1 --2- 172.21.15.86:0/639022551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4454073900 0x7f4454199b60 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f444000eb90 tx=0x7f444000eea0 comp rx=0 tx=0).stop 2024-03-23T17:49:22.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.127+0000 7f445a000700 1 -- 172.21.15.86:0/639022551 reap_dead start 2024-03-23T17:49:22.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.127+0000 7f445c000700 1 -- 172.21.15.86:0/639022551 shutdown_connections 2024-03-23T17:49:22.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.127+0000 7f445c000700 1 -- 172.21.15.86:0/639022551 >> 172.21.15.86:0/639022551 conn(0x7f4454077fa0 msgr2=0x7f44540fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:22.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.127+0000 7f445c000700 1 -- 172.21.15.86:0/639022551 shutdown_connections 2024-03-23T17:49:22.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.128+0000 7f445c000700 1 -- 172.21.15.86:0/639022551 wait complete. 2024-03-23T17:49:22.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.221+0000 7f8322200700 1 -- 172.21.15.86:0/4184301260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f831c106ef0 msgr2=0x7f831c1072d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:22.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.221+0000 7f8322200700 1 --2- 172.21.15.86:0/4184301260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f831c106ef0 0x7f831c1072d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f830c009920 tx=0x7f830c009c30 comp rx=0 tx=0).stop 2024-03-23T17:49:22.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.221+0000 7f8322200700 1 -- 172.21.15.86:0/4184301260 shutdown_connections 2024-03-23T17:49:22.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.221+0000 7f8322200700 1 --2- 172.21.15.86:0/4184301260 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f831c101820 0x7f831c10dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:22.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.221+0000 7f8322200700 1 --2- 172.21.15.86:0/4184301260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f831c106ef0 0x7f831c1072d0 secure :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f830c009920 tx=0x7f830c009c30 comp rx=0 tx=0).stop 2024-03-23T17:49:22.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.221+0000 7f8322200700 1 --2- 172.21.15.86:0/4184301260 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f831c100e60 0x7f831c1012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:22.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.221+0000 7f8322200700 1 -- 172.21.15.86:0/4184301260 >> 172.21.15.86:0/4184301260 conn(0x7f831c077fa0 msgr2=0x7f831c0783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:22.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.222+0000 7f8322200700 1 -- 172.21.15.86:0/4184301260 shutdown_connections 2024-03-23T17:49:22.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.222+0000 7f8322200700 1 -- 172.21.15.86:0/4184301260 wait complete. 2024-03-23T17:49:22.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.224+0000 7f8322200700 1 Processor -- start 2024-03-23T17:49:22.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.224+0000 7f8322200700 1 -- start start 2024-03-23T17:49:22.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.224+0000 7f8322200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f831c100e60 0x7f831c0730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:22.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.224+0000 7f8322200700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f831c101820 0x7f831c0735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:22.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.225+0000 7f8322200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f831c073b30 0x7f831c1a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:22.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.225+0000 7f8322200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f831c1a5c40 con 0x7f831c100e60 2024-03-23T17:49:22.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.225+0000 7f8322200700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f831c1a5db0 con 0x7f831c073b30 2024-03-23T17:49:22.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.225+0000 7f8322200700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f831c1a5f70 con 0x7f831c101820 2024-03-23T17:49:22.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.225+0000 7f831be00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f831c073b30 0x7f831c1a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:22.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.225+0000 7f831aa00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f831c101820 0x7f831c0735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:22.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.225+0000 7f831be00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f831c073b30 0x7f831c1a5670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:52524/0 (socket says 172.21.15.86:52524) 2024-03-23T17:49:22.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.225+0000 7f831be00700 1 -- 172.21.15.86:0/3127954958 learned_addr learned my addr 172.21.15.86:0/3127954958 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:22.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.225+0000 7f831be00700 1 -- 172.21.15.86:0/3127954958 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f831c101820 msgr2=0x7f831c0735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:22.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.226+0000 7f831be00700 1 --2- 172.21.15.86:0/3127954958 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f831c101820 0x7f831c0735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:22.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.226+0000 7f831be00700 1 -- 172.21.15.86:0/3127954958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f831c100e60 msgr2=0x7f831c0730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:22.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.226+0000 7f831b400700 1 --2- 172.21.15.86:0/3127954958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f831c100e60 0x7f831c0730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:22.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.226+0000 7f831be00700 1 --2- 172.21.15.86:0/3127954958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f831c100e60 0x7f831c0730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:22.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.226+0000 7f831be00700 1 -- 172.21.15.86:0/3127954958 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f830c009520 con 0x7f831c073b30 2024-03-23T17:49:22.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.226+0000 7f831aa00700 1 --2- 172.21.15.86:0/3127954958 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f831c101820 0x7f831c0735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:49:22.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.226+0000 7f831b400700 1 --2- 172.21.15.86:0/3127954958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f831c100e60 0x7f831c0730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T17:49:22.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.226+0000 7f831be00700 1 --2- 172.21.15.86:0/3127954958 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f831c073b30 0x7f831c1a5670 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f831400efc0 tx=0x7f831400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:22.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.227+0000 7f8313e00700 1 -- 172.21.15.86:0/3127954958 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8314018070 con 0x7f831c073b30 2024-03-23T17:49:22.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.227+0000 7f8313e00700 1 -- 172.21.15.86:0/3127954958 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f831400f040 con 0x7f831c073b30 2024-03-23T17:49:22.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.227+0000 7f8313e00700 1 -- 172.21.15.86:0/3127954958 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831400ccc0 con 0x7f831c073b30 2024-03-23T17:49:22.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.227+0000 7f8322200700 1 -- 172.21.15.86:0/3127954958 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f831c1a61f0 con 0x7f831c073b30 2024-03-23T17:49:22.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.227+0000 7f8322200700 1 -- 172.21.15.86:0/3127954958 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f831c197480 con 0x7f831c073b30 2024-03-23T17:49:22.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.228+0000 7f8322200700 1 -- 172.21.15.86:0/3127954958 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f831c1024f0 con 0x7f831c073b30 2024-03-23T17:49:22.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.232+0000 7f8313e00700 1 -- 172.21.15.86:0/3127954958 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f8314003a00 con 0x7f831c073b30 2024-03-23T17:49:22.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.233+0000 7f8313e00700 1 --2- 172.21.15.86:0/3127954958 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f83000a12a0 0x7f83000a3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:22.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.233+0000 7f8313e00700 1 -- 172.21.15.86:0/3127954958 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f83140ccbc0 con 0x7f831c073b30 2024-03-23T17:49:22.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.234+0000 7f8313e00700 1 --2- 172.21.15.86:0/3127954958 >> [v2:172.21.15.152:6816/3237612748,v1:172.21.15.152:6817/3237612748] conn(0x7f83000aaa80 0x7f83000acea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:22.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.234+0000 7f8313e00700 1 -- 172.21.15.86:0/3127954958 --> [v2:172.21.15.152:6816/3237612748,v1:172.21.15.152:6817/3237612748] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f83000ad550 con 0x7f83000aaa80 2024-03-23T17:49:22.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.234+0000 7f8313e00700 1 -- 172.21.15.86:0/3127954958 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f83140cd060 con 0x7f831c073b30 2024-03-23T17:49:22.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.234+0000 7f831aa00700 1 --2- 172.21.15.86:0/3127954958 >> [v2:172.21.15.152:6816/3237612748,v1:172.21.15.152:6817/3237612748] conn(0x7f83000aaa80 0x7f83000acea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:22.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.235+0000 7f831b400700 1 --2- 172.21.15.86:0/3127954958 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f83000a12a0 0x7f83000a3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:22.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.236+0000 7f831b400700 1 --2- 172.21.15.86:0/3127954958 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f83000a12a0 0x7f83000a3760 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f830c005950 tx=0x7f830c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:22.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.237+0000 7f831aa00700 1 --2- 172.21.15.86:0/3127954958 >> [v2:172.21.15.152:6816/3237612748,v1:172.21.15.152:6817/3237612748] conn(0x7f83000aaa80 0x7f83000acea0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:22.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.238+0000 7f8313e00700 1 -- 172.21.15.86:0/3127954958 <== osd.8 v2:172.21.15.152:6816/3237612748 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f83000ad550 con 0x7f83000aaa80 2024-03-23T17:49:22.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.259+0000 7f8322200700 1 -- 172.21.15.86:0/3127954958 --> [v2:172.21.15.152:6816/3237612748,v1:172.21.15.152:6817/3237612748] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f831c111100 con 0x7f83000aaa80 2024-03-23T17:49:22.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.260+0000 7f8313e00700 1 -- 172.21.15.86:0/3127954958 <== osd.8 v2:172.21.15.152:6816/3237612748 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f831c111100 con 0x7f83000aaa80 2024-03-23T17:49:22.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.260+0000 7f8322200700 1 -- 172.21.15.86:0/3127954958 >> [v2:172.21.15.152:6816/3237612748,v1:172.21.15.152:6817/3237612748] conn(0x7f83000aaa80 msgr2=0x7f83000acea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:22.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.260+0000 7f8322200700 1 --2- 172.21.15.86:0/3127954958 >> [v2:172.21.15.152:6816/3237612748,v1:172.21.15.152:6817/3237612748] conn(0x7f83000aaa80 0x7f83000acea0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:22.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.260+0000 7f8322200700 1 -- 172.21.15.86:0/3127954958 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f83000a12a0 msgr2=0x7f83000a3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:22.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.260+0000 7f8322200700 1 --2- 172.21.15.86:0/3127954958 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f83000a12a0 0x7f83000a3760 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f830c005950 tx=0x7f830c0058e0 comp rx=0 tx=0).stop 2024-03-23T17:49:22.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.260+0000 7f8322200700 1 -- 172.21.15.86:0/3127954958 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f831c073b30 msgr2=0x7f831c1a5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:22.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.260+0000 7f8322200700 1 --2- 172.21.15.86:0/3127954958 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f831c073b30 0x7f831c1a5670 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f831400efc0 tx=0x7f831400c3a0 comp rx=0 tx=0).stop 2024-03-23T17:49:22.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.260+0000 7f831be00700 1 -- 172.21.15.86:0/3127954958 reap_dead start 2024-03-23T17:49:22.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.260+0000 7f8322200700 1 -- 172.21.15.86:0/3127954958 shutdown_connections 2024-03-23T17:49:22.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.260+0000 7f8322200700 1 -- 172.21.15.86:0/3127954958 >> 172.21.15.86:0/3127954958 conn(0x7f831c077fa0 msgr2=0x7f831c0ffc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:22.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.261+0000 7f8322200700 1 -- 172.21.15.86:0/3127954958 shutdown_connections 2024-03-23T17:49:22.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.261+0000 7f8322200700 1 -- 172.21.15.86:0/3127954958 wait complete. 2024-03-23T17:49:22.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.406+0000 7fde84400700 1 -- 172.21.15.86:0/3701334171 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fde7c1010f0 msgr2=0x7fde7c1014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:22.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.406+0000 7fde84400700 1 --2- 172.21.15.86:0/3701334171 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fde7c1010f0 0x7fde7c1014d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fde70009990 tx=0x7fde70009ca0 comp rx=0 tx=0).stop 2024-03-23T17:49:22.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.408+0000 7fde84400700 1 -- 172.21.15.86:0/3701334171 shutdown_connections 2024-03-23T17:49:22.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.408+0000 7fde84400700 1 --2- 172.21.15.86:0/3701334171 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde7c1060c0 0x7fde7c10dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:22.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.408+0000 7fde84400700 1 --2- 172.21.15.86:0/3701334171 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fde7c1010f0 0x7fde7c1014d0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:22.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.408+0000 7fde84400700 1 --2- 172.21.15.86:0/3701334171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde7c101aa0 0x7fde7c1059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:22.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.408+0000 7fde84400700 1 -- 172.21.15.86:0/3701334171 >> 172.21.15.86:0/3701334171 conn(0x7fde7c077e90 msgr2=0x7fde7c0782a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:22.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.408+0000 7fde84400700 1 -- 172.21.15.86:0/3701334171 shutdown_connections 2024-03-23T17:49:22.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.408+0000 7fde84400700 1 -- 172.21.15.86:0/3701334171 wait complete. 2024-03-23T17:49:22.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.409+0000 7fde84400700 1 Processor -- start 2024-03-23T17:49:22.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.409+0000 7fde84400700 1 -- start start 2024-03-23T17:49:22.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.410+0000 7fde84400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde7c1010f0 0x7fde7c10da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:22.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.410+0000 7fde84400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde7c101aa0 0x7fde7c10aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:22.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.410+0000 7fde84400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fde7c1060c0 0x7fde7c10b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:22.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.410+0000 7fde84400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde7c10b7b0 con 0x7fde7c101aa0 2024-03-23T17:49:22.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.410+0000 7fde84400700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde7c10b920 con 0x7fde7c1010f0 2024-03-23T17:49:22.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.410+0000 7fde84400700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde7c1a9d50 con 0x7fde7c1060c0 2024-03-23T17:49:22.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.410+0000 7fde82400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fde7c1060c0 0x7fde7c10b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:22.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.410+0000 7fde81a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde7c1010f0 0x7fde7c10da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:22.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.410+0000 7fde81a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde7c1010f0 0x7fde7c10da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:52542/0 (socket says 172.21.15.86:52542) 2024-03-23T17:49:22.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.410+0000 7fde82400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fde7c1060c0 0x7fde7c10b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:60102/0 (socket says 172.21.15.86:60102) 2024-03-23T17:49:22.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.410+0000 7fde81a00700 1 -- 172.21.15.86:0/3260487211 learned_addr learned my addr 172.21.15.86:0/3260487211 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:22.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.410+0000 7fde81000700 1 --2- 172.21.15.86:0/3260487211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde7c101aa0 0x7fde7c10aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:22.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.411+0000 7fde81a00700 1 -- 172.21.15.86:0/3260487211 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fde7c1060c0 msgr2=0x7fde7c10b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:22.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.411+0000 7fde81a00700 1 --2- 172.21.15.86:0/3260487211 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fde7c1060c0 0x7fde7c10b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:22.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.411+0000 7fde81a00700 1 -- 172.21.15.86:0/3260487211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde7c101aa0 msgr2=0x7fde7c10aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:22.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.411+0000 7fde81a00700 1 --2- 172.21.15.86:0/3260487211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde7c101aa0 0x7fde7c10aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:22.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.411+0000 7fde81a00700 1 -- 172.21.15.86:0/3260487211 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde700095f0 con 0x7fde7c1010f0 2024-03-23T17:49:22.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.411+0000 7fde81000700 1 --2- 172.21.15.86:0/3260487211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde7c101aa0 0x7fde7c10aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:49:22.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.411+0000 7fde81a00700 1 --2- 172.21.15.86:0/3260487211 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde7c1010f0 0x7fde7c10da10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fde70000c00 tx=0x7fde70005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:22.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.411+0000 7fde7a000700 1 -- 172.21.15.86:0/3260487211 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde7001d070 con 0x7fde7c1010f0 2024-03-23T17:49:22.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.411+0000 7fde84400700 1 -- 172.21.15.86:0/3260487211 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde7c1a9f50 con 0x7fde7c1010f0 2024-03-23T17:49:22.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.411+0000 7fde84400700 1 -- 172.21.15.86:0/3260487211 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde7c1aa310 con 0x7fde7c1010f0 2024-03-23T17:49:22.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.412+0000 7fde7a000700 1 -- 172.21.15.86:0/3260487211 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde70022470 con 0x7fde7c1010f0 2024-03-23T17:49:22.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.412+0000 7fde7a000700 1 -- 172.21.15.86:0/3260487211 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde70017650 con 0x7fde7c1010f0 2024-03-23T17:49:22.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.412+0000 7fde84400700 1 -- 172.21.15.86:0/3260487211 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fde5c000fc0 con 0x7fde7c1010f0 2024-03-23T17:49:22.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.415+0000 7fde7a000700 1 -- 172.21.15.86:0/3260487211 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fde7000fb00 con 0x7fde7c1010f0 2024-03-23T17:49:22.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.416+0000 7fde7a000700 1 --2- 172.21.15.86:0/3260487211 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fde640a1230 0x7fde640a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:22.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.416+0000 7fde7a000700 1 -- 172.21.15.86:0/3260487211 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fde700ce620 con 0x7fde7c1010f0 2024-03-23T17:49:22.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.416+0000 7fde81000700 1 --2- 172.21.15.86:0/3260487211 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fde640a1230 0x7fde640a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:22.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.416+0000 7fde7a000700 1 --2- 172.21.15.86:0/3260487211 >> [v2:172.21.15.86:6826/2835218294,v1:172.21.15.86:6827/2835218294] conn(0x7fde640aaa10 0x7fde640ace30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:22.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.416+0000 7fde7a000700 1 -- 172.21.15.86:0/3260487211 --> [v2:172.21.15.86:6826/2835218294,v1:172.21.15.86:6827/2835218294] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fde640ad4e0 con 0x7fde640aaa10 2024-03-23T17:49:22.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.416+0000 7fde7a000700 1 -- 172.21.15.86:0/3260487211 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde700ce990 con 0x7fde7c1010f0 2024-03-23T17:49:22.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.417+0000 7fde82400700 1 --2- 172.21.15.86:0/3260487211 >> [v2:172.21.15.86:6826/2835218294,v1:172.21.15.86:6827/2835218294] conn(0x7fde640aaa10 0x7fde640ace30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:22.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.417+0000 7fde81000700 1 --2- 172.21.15.86:0/3260487211 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fde640a1230 0x7fde640a36f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fde7c10c220 tx=0x7fde6800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:22.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.418+0000 7fde82400700 1 --2- 172.21.15.86:0/3260487211 >> [v2:172.21.15.86:6826/2835218294,v1:172.21.15.86:6827/2835218294] conn(0x7fde640aaa10 0x7fde640ace30 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:22.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.419+0000 7fde7a000700 1 -- 172.21.15.86:0/3260487211 <== osd.9 v2:172.21.15.86:6826/2835218294 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fde640ad4e0 con 0x7fde640aaa10 2024-03-23T17:49:22.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.437+0000 7fde84400700 1 -- 172.21.15.86:0/3260487211 --> [v2:172.21.15.86:6826/2835218294,v1:172.21.15.86:6827/2835218294] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fde5c002d40 con 0x7fde640aaa10 2024-03-23T17:49:22.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.438+0000 7fde7a000700 1 -- 172.21.15.86:0/3260487211 <== osd.9 v2:172.21.15.86:6826/2835218294 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fde5c002d40 con 0x7fde640aaa10 2024-03-23T17:49:22.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.438+0000 7fde84400700 1 -- 172.21.15.86:0/3260487211 >> [v2:172.21.15.86:6826/2835218294,v1:172.21.15.86:6827/2835218294] conn(0x7fde640aaa10 msgr2=0x7fde640ace30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:22.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.438+0000 7fde84400700 1 --2- 172.21.15.86:0/3260487211 >> [v2:172.21.15.86:6826/2835218294,v1:172.21.15.86:6827/2835218294] conn(0x7fde640aaa10 0x7fde640ace30 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:22.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.438+0000 7fde84400700 1 -- 172.21.15.86:0/3260487211 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fde640a1230 msgr2=0x7fde640a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:22.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.438+0000 7fde84400700 1 --2- 172.21.15.86:0/3260487211 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fde640a1230 0x7fde640a36f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fde7c10c220 tx=0x7fde6800b410 comp rx=0 tx=0).stop 2024-03-23T17:49:22.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.438+0000 7fde84400700 1 -- 172.21.15.86:0/3260487211 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde7c1010f0 msgr2=0x7fde7c10da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:22.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.438+0000 7fde84400700 1 --2- 172.21.15.86:0/3260487211 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde7c1010f0 0x7fde7c10da10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fde70000c00 tx=0x7fde70005740 comp rx=0 tx=0).stop 2024-03-23T17:49:22.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.438+0000 7fde82400700 1 -- 172.21.15.86:0/3260487211 reap_dead start 2024-03-23T17:49:22.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.438+0000 7fde84400700 1 -- 172.21.15.86:0/3260487211 shutdown_connections 2024-03-23T17:49:22.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.438+0000 7fde84400700 1 -- 172.21.15.86:0/3260487211 >> 172.21.15.86:0/3260487211 conn(0x7fde7c077e90 msgr2=0x7fde7c0ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:22.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.439+0000 7fde84400700 1 -- 172.21.15.86:0/3260487211 shutdown_connections 2024-03-23T17:49:22.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.439+0000 7fde84400700 1 -- 172.21.15.86:0/3260487211 wait complete. 2024-03-23T17:49:22.730 INFO:teuthology.orchestra.run.smithi086.stdout:193273528355 2024-03-23T17:49:22.730 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-03-23T17:49:23.058 INFO:teuthology.orchestra.run.smithi086.stdout:141733920811 2024-03-23T17:49:23.059 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-03-23T17:49:23.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.681+0000 7fb333600700 1 -- 172.21.15.86:0/3943806558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb32c072fe0 msgr2=0x7fb32c0733c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:23.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.681+0000 7fb333600700 1 --2- 172.21.15.86:0/3943806558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb32c072fe0 0x7fb32c0733c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb320009990 tx=0x7fb320009ca0 comp rx=0 tx=0).stop 2024-03-23T17:49:23.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.681+0000 7fb333600700 1 -- 172.21.15.86:0/3943806558 shutdown_connections 2024-03-23T17:49:23.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.681+0000 7fb333600700 1 --2- 172.21.15.86:0/3943806558 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb32c10d810 0x7fb32c10fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:23.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.681+0000 7fb333600700 1 --2- 172.21.15.86:0/3943806558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb32c072fe0 0x7fb32c0733c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:23.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.681+0000 7fb333600700 1 --2- 172.21.15.86:0/3943806558 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb32c073900 0x7fb32c10d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:23.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.681+0000 7fb333600700 1 -- 172.21.15.86:0/3943806558 >> 172.21.15.86:0/3943806558 conn(0x7fb32c077fa0 msgr2=0x7fb32c0783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:23.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.681+0000 7fb333600700 1 -- 172.21.15.86:0/3943806558 shutdown_connections 2024-03-23T17:49:23.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.682+0000 7fb333600700 1 -- 172.21.15.86:0/3943806558 wait complete. 2024-03-23T17:49:23.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.683+0000 7fb333600700 1 Processor -- start 2024-03-23T17:49:23.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.683+0000 7fb333600700 1 -- start start 2024-03-23T17:49:23.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.684+0000 7fb333600700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb32c072fe0 0x7fb32c1994d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:23.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.684+0000 7fb333600700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb32c073900 0x7fb32c199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:23.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.684+0000 7fb333600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb32c10d810 0x7fb32c19dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:23.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.684+0000 7fb333600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb32c19e380 con 0x7fb32c10d810 2024-03-23T17:49:23.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.684+0000 7fb333600700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb32c19e4f0 con 0x7fb32c073900 2024-03-23T17:49:23.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.684+0000 7fb333600700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb32c19e6b0 con 0x7fb32c072fe0 2024-03-23T17:49:23.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.684+0000 7fb331600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb32c10d810 0x7fb32c19dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:23.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.684+0000 7fb331600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb32c10d810 0x7fb32c19dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60188/0 (socket says 172.21.15.86:60188) 2024-03-23T17:49:23.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.684+0000 7fb331600700 1 -- 172.21.15.86:0/2285535197 learned_addr learned my addr 172.21.15.86:0/2285535197 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:23.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.684+0000 7fb32be00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb32c073900 0x7fb32c199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:23.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.684+0000 7fb330c00700 1 --2- 172.21.15.86:0/2285535197 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb32c072fe0 0x7fb32c1994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:23.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.684+0000 7fb32be00700 1 -- 172.21.15.86:0/2285535197 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb32c072fe0 msgr2=0x7fb32c1994d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:23.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.684+0000 7fb32be00700 1 --2- 172.21.15.86:0/2285535197 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb32c072fe0 0x7fb32c1994d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:23.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.684+0000 7fb32be00700 1 -- 172.21.15.86:0/2285535197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb32c10d810 msgr2=0x7fb32c19dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:23.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.684+0000 7fb32be00700 1 --2- 172.21.15.86:0/2285535197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb32c10d810 0x7fb32c19dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:23.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.684+0000 7fb32be00700 1 -- 172.21.15.86:0/2285535197 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3200095f0 con 0x7fb32c073900 2024-03-23T17:49:23.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.685+0000 7fb32be00700 1 --2- 172.21.15.86:0/2285535197 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb32c073900 0x7fb32c199a10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb31800d750 tx=0x7fb31800da60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:23.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.685+0000 7fb329600700 1 -- 172.21.15.86:0/2285535197 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb318004a30 con 0x7fb32c073900 2024-03-23T17:49:23.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.685+0000 7fb333600700 1 -- 172.21.15.86:0/2285535197 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb32c19e990 con 0x7fb32c073900 2024-03-23T17:49:23.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.685+0000 7fb329600700 1 -- 172.21.15.86:0/2285535197 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb3180054e0 con 0x7fb32c073900 2024-03-23T17:49:23.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.685+0000 7fb329600700 1 -- 172.21.15.86:0/2285535197 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb31800fb50 con 0x7fb32c073900 2024-03-23T17:49:23.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.685+0000 7fb333600700 1 -- 172.21.15.86:0/2285535197 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb32c1a61f0 con 0x7fb32c073900 2024-03-23T17:49:23.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.686+0000 7fb333600700 1 -- 172.21.15.86:0/2285535197 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb32c113280 con 0x7fb32c073900 2024-03-23T17:49:23.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.688+0000 7fb329600700 1 -- 172.21.15.86:0/2285535197 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fb318004b90 con 0x7fb32c073900 2024-03-23T17:49:23.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.689+0000 7fb329600700 1 --2- 172.21.15.86:0/2285535197 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fb3140a11b0 0x7fb3140a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:23.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.689+0000 7fb329600700 1 -- 172.21.15.86:0/2285535197 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb3180cc340 con 0x7fb32c073900 2024-03-23T17:49:23.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.690+0000 7fb330c00700 1 --2- 172.21.15.86:0/2285535197 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fb3140a11b0 0x7fb3140a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:23.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.690+0000 7fb329600700 1 --2- 172.21.15.86:0/2285535197 >> [v2:172.21.15.92:6800/3776252597,v1:172.21.15.92:6801/3776252597] conn(0x7fb3140aa990 0x7fb3140acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:23.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.690+0000 7fb329600700 1 -- 172.21.15.86:0/2285535197 --> [v2:172.21.15.92:6800/3776252597,v1:172.21.15.92:6801/3776252597] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb3140ad460 con 0x7fb3140aa990 2024-03-23T17:49:23.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.690+0000 7fb329600700 1 -- 172.21.15.86:0/2285535197 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb3180ce920 con 0x7fb32c073900 2024-03-23T17:49:23.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.690+0000 7fb330c00700 1 --2- 172.21.15.86:0/2285535197 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fb3140a11b0 0x7fb3140a3670 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb3200098e0 tx=0x7fb320005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:23.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.691+0000 7fb331600700 1 --2- 172.21.15.86:0/2285535197 >> [v2:172.21.15.92:6800/3776252597,v1:172.21.15.92:6801/3776252597] conn(0x7fb3140aa990 0x7fb3140acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:23.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.691+0000 7fb331600700 1 --2- 172.21.15.86:0/2285535197 >> [v2:172.21.15.92:6800/3776252597,v1:172.21.15.92:6801/3776252597] conn(0x7fb3140aa990 0x7fb3140acdb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:23.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.692+0000 7fb329600700 1 -- 172.21.15.86:0/2285535197 <== osd.1 v2:172.21.15.92:6800/3776252597 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fb3140ad460 con 0x7fb3140aa990 2024-03-23T17:49:23.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.715+0000 7fb333600700 1 -- 172.21.15.86:0/2285535197 --> [v2:172.21.15.92:6800/3776252597,v1:172.21.15.92:6801/3776252597] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb32c061430 con 0x7fb3140aa990 2024-03-23T17:49:23.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.715+0000 7fb329600700 1 -- 172.21.15.86:0/2285535197 <== osd.1 v2:172.21.15.92:6800/3776252597 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb32c061430 con 0x7fb3140aa990 2024-03-23T17:49:23.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.716+0000 7fb333600700 1 -- 172.21.15.86:0/2285535197 >> [v2:172.21.15.92:6800/3776252597,v1:172.21.15.92:6801/3776252597] conn(0x7fb3140aa990 msgr2=0x7fb3140acdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:23.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.716+0000 7fb333600700 1 --2- 172.21.15.86:0/2285535197 >> [v2:172.21.15.92:6800/3776252597,v1:172.21.15.92:6801/3776252597] conn(0x7fb3140aa990 0x7fb3140acdb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:23.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.716+0000 7fb333600700 1 -- 172.21.15.86:0/2285535197 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fb3140a11b0 msgr2=0x7fb3140a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:23.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.716+0000 7fb333600700 1 --2- 172.21.15.86:0/2285535197 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fb3140a11b0 0x7fb3140a3670 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb3200098e0 tx=0x7fb320005960 comp rx=0 tx=0).stop 2024-03-23T17:49:23.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.716+0000 7fb333600700 1 -- 172.21.15.86:0/2285535197 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb32c073900 msgr2=0x7fb32c199a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:23.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.716+0000 7fb333600700 1 --2- 172.21.15.86:0/2285535197 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb32c073900 0x7fb32c199a10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb31800d750 tx=0x7fb31800da60 comp rx=0 tx=0).stop 2024-03-23T17:49:23.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.716+0000 7fb331600700 1 -- 172.21.15.86:0/2285535197 reap_dead start 2024-03-23T17:49:23.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.717+0000 7fb333600700 1 -- 172.21.15.86:0/2285535197 shutdown_connections 2024-03-23T17:49:23.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.717+0000 7fb333600700 1 -- 172.21.15.86:0/2285535197 >> 172.21.15.86:0/2285535197 conn(0x7fb32c077fa0 msgr2=0x7fb32c0fe0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:23.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.717+0000 7fb333600700 1 -- 172.21.15.86:0/2285535197 shutdown_connections 2024-03-23T17:49:23.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:22.717+0000 7fb333600700 1 -- 172.21.15.86:0/2285535197 wait complete. 2024-03-23T17:49:23.634 INFO:teuthology.orchestra.run.smithi086.stdout:270582939673 2024-03-23T17:49:23.634 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.9 2024-03-23T17:49:24.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:23 smithi086 ceph-mon[38221]: pgmap v247: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-23T17:49:24.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:23 smithi092 ceph-mon[79089]: pgmap v247: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-23T17:49:24.085 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:23 smithi152 ceph-mon[47949]: pgmap v247: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-23T17:49:24.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.193+0000 7fc38e400700 1 -- 172.21.15.86:0/2055812206 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc388074d30 msgr2=0x7fc388072fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:24.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.193+0000 7fc38e400700 1 --2- 172.21.15.86:0/2055812206 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc388074d30 0x7fc388072fe0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc3780098f0 tx=0x7fc378009c00 comp rx=0 tx=0).stop 2024-03-23T17:49:24.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.193+0000 7fc38e400700 1 -- 172.21.15.86:0/2055812206 shutdown_connections 2024-03-23T17:49:24.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.193+0000 7fc38e400700 1 --2- 172.21.15.86:0/2055812206 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc388111b30 0x7fc388113f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:24.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.193+0000 7fc38e400700 1 --2- 172.21.15.86:0/2055812206 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc388074d30 0x7fc388072fe0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:24.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.193+0000 7fc38e400700 1 --2- 172.21.15.86:0/2055812206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc388073520 0x7fc3880739a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:24.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.193+0000 7fc38e400700 1 -- 172.21.15.86:0/2055812206 >> 172.21.15.86:0/2055812206 conn(0x7fc388077fa0 msgr2=0x7fc3880783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:24.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.193+0000 7fc38e400700 1 -- 172.21.15.86:0/2055812206 shutdown_connections 2024-03-23T17:49:24.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.193+0000 7fc38e400700 1 -- 172.21.15.86:0/2055812206 wait complete. 2024-03-23T17:49:24.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.195+0000 7fc38e400700 1 Processor -- start 2024-03-23T17:49:24.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.195+0000 7fc38e400700 1 -- start start 2024-03-23T17:49:24.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.196+0000 7fc38e400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc388073520 0x7fc3881a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:24.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.196+0000 7fc38e400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc388074d30 0x7fc3881a3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:24.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.196+0000 7fc38e400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc388111b30 0x7fc38819d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:24.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.196+0000 7fc38e400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc38819ddc0 con 0x7fc388074d30 2024-03-23T17:49:24.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.196+0000 7fc38e400700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc38819df30 con 0x7fc388111b30 2024-03-23T17:49:24.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.196+0000 7fc38e400700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc38819e0f0 con 0x7fc388073520 2024-03-23T17:49:24.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.196+0000 7fc386a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc388074d30 0x7fc3881a3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:24.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.196+0000 7fc387400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc388073520 0x7fc3881a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:24.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.196+0000 7fc387e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc388111b30 0x7fc38819d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:24.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.196+0000 7fc387e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc388111b30 0x7fc38819d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:52590/0 (socket says 172.21.15.86:52590) 2024-03-23T17:49:24.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.196+0000 7fc387e00700 1 -- 172.21.15.86:0/1521787476 learned_addr learned my addr 172.21.15.86:0/1521787476 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:24.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.196+0000 7fc386a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc388074d30 0x7fc3881a3b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60208/0 (socket says 172.21.15.86:60208) 2024-03-23T17:49:24.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.197+0000 7fc387400700 1 -- 172.21.15.86:0/1521787476 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc388111b30 msgr2=0x7fc38819d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:24.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.197+0000 7fc387400700 1 --2- 172.21.15.86:0/1521787476 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc388111b30 0x7fc38819d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:24.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.197+0000 7fc387400700 1 -- 172.21.15.86:0/1521787476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc388074d30 msgr2=0x7fc3881a3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:24.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.197+0000 7fc387400700 1 --2- 172.21.15.86:0/1521787476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc388074d30 0x7fc3881a3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:24.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.197+0000 7fc387400700 1 -- 172.21.15.86:0/1521787476 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3740095f0 con 0x7fc388073520 2024-03-23T17:49:24.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.197+0000 7fc387400700 1 --2- 172.21.15.86:0/1521787476 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc388073520 0x7fc3881a35d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc378009fd0 tx=0x7fc37800fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:24.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.197+0000 7fc37fe00700 1 -- 172.21.15.86:0/1521787476 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc37801c070 con 0x7fc388073520 2024-03-23T17:49:24.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.197+0000 7fc37fe00700 1 -- 172.21.15.86:0/1521787476 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc37800baa0 con 0x7fc388073520 2024-03-23T17:49:24.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.197+0000 7fc38e400700 1 -- 172.21.15.86:0/1521787476 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc378009520 con 0x7fc388073520 2024-03-23T17:49:24.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.197+0000 7fc37fe00700 1 -- 172.21.15.86:0/1521787476 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3780038e0 con 0x7fc388073520 2024-03-23T17:49:24.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.197+0000 7fc38e400700 1 -- 172.21.15.86:0/1521787476 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3881aa3f0 con 0x7fc388073520 2024-03-23T17:49:24.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.198+0000 7fc38e400700 1 -- 172.21.15.86:0/1521787476 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc388117700 con 0x7fc388073520 2024-03-23T17:49:24.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.200+0000 7fc37fe00700 1 -- 172.21.15.86:0/1521787476 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fc378020030 con 0x7fc388073520 2024-03-23T17:49:24.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.201+0000 7fc37fe00700 1 --2- 172.21.15.86:0/1521787476 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fc36c0a11b0 0x7fc36c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:24.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.201+0000 7fc37fe00700 1 -- 172.21.15.86:0/1521787476 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc3780cdc80 con 0x7fc388073520 2024-03-23T17:49:24.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.202+0000 7fc37fe00700 1 --2- 172.21.15.86:0/1521787476 >> [v2:172.21.15.152:6824/194662760,v1:172.21.15.152:6825/194662760] conn(0x7fc36c0aa860 0x7fc36c0acc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:24.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.202+0000 7fc37fe00700 1 -- 172.21.15.86:0/1521787476 --> [v2:172.21.15.152:6824/194662760,v1:172.21.15.152:6825/194662760] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc36c0ad330 con 0x7fc36c0aa860 2024-03-23T17:49:24.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.202+0000 7fc37fe00700 1 -- 172.21.15.86:0/1521787476 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc3780cdff0 con 0x7fc388073520 2024-03-23T17:49:24.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.202+0000 7fc386a00700 1 --2- 172.21.15.86:0/1521787476 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fc36c0a11b0 0x7fc36c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:24.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.202+0000 7fc387e00700 1 --2- 172.21.15.86:0/1521787476 >> [v2:172.21.15.152:6824/194662760,v1:172.21.15.152:6825/194662760] conn(0x7fc36c0aa860 0x7fc36c0acc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:24.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.203+0000 7fc387e00700 1 --2- 172.21.15.86:0/1521787476 >> [v2:172.21.15.152:6824/194662760,v1:172.21.15.152:6825/194662760] conn(0x7fc36c0aa860 0x7fc36c0acc80 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:24.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.203+0000 7fc386a00700 1 --2- 172.21.15.86:0/1521787476 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fc36c0a11b0 0x7fc36c0a3670 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc38819ee20 tx=0x7fc374009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:24.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.204+0000 7fc37fe00700 1 -- 172.21.15.86:0/1521787476 <== osd.11 v2:172.21.15.152:6824/194662760 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fc36c0ad330 con 0x7fc36c0aa860 2024-03-23T17:49:24.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.221+0000 7fc38e400700 1 -- 172.21.15.86:0/1521787476 --> [v2:172.21.15.152:6824/194662760,v1:172.21.15.152:6825/194662760] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc388074670 con 0x7fc36c0aa860 2024-03-23T17:49:24.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.221+0000 7fc37fe00700 1 -- 172.21.15.86:0/1521787476 <== osd.11 v2:172.21.15.152:6824/194662760 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fc388074670 con 0x7fc36c0aa860 2024-03-23T17:49:24.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.221+0000 7fc38e400700 1 -- 172.21.15.86:0/1521787476 >> [v2:172.21.15.152:6824/194662760,v1:172.21.15.152:6825/194662760] conn(0x7fc36c0aa860 msgr2=0x7fc36c0acc80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:24.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.221+0000 7fc38e400700 1 --2- 172.21.15.86:0/1521787476 >> [v2:172.21.15.152:6824/194662760,v1:172.21.15.152:6825/194662760] conn(0x7fc36c0aa860 0x7fc36c0acc80 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:24.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.221+0000 7fc38e400700 1 -- 172.21.15.86:0/1521787476 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fc36c0a11b0 msgr2=0x7fc36c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:24.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.222+0000 7fc38e400700 1 --2- 172.21.15.86:0/1521787476 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fc36c0a11b0 0x7fc36c0a3670 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc38819ee20 tx=0x7fc374009310 comp rx=0 tx=0).stop 2024-03-23T17:49:24.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.222+0000 7fc38e400700 1 -- 172.21.15.86:0/1521787476 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc388073520 msgr2=0x7fc3881a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:24.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.222+0000 7fc38e400700 1 --2- 172.21.15.86:0/1521787476 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc388073520 0x7fc3881a35d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc378009fd0 tx=0x7fc37800fa00 comp rx=0 tx=0).stop 2024-03-23T17:49:24.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.222+0000 7fc387e00700 1 -- 172.21.15.86:0/1521787476 reap_dead start 2024-03-23T17:49:24.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.222+0000 7fc38e400700 1 -- 172.21.15.86:0/1521787476 shutdown_connections 2024-03-23T17:49:24.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.222+0000 7fc38e400700 1 -- 172.21.15.86:0/1521787476 >> 172.21.15.86:0/1521787476 conn(0x7fc388077fa0 msgr2=0x7fc3881107a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:24.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.222+0000 7fc38e400700 1 -- 172.21.15.86:0/1521787476 shutdown_connections 2024-03-23T17:49:24.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.222+0000 7fc38e400700 1 -- 172.21.15.86:0/1521787476 wait complete. 2024-03-23T17:49:24.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.103+0000 7fbe5c600700 1 -- 172.21.15.86:0/3271253702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe5410d810 msgr2=0x7fbe5410fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:24.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.103+0000 7fbe5c600700 1 --2- 172.21.15.86:0/3271253702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe5410d810 0x7fbe5410fc00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c00b0a0 tx=0x7fbe4c00b3b0 comp rx=0 tx=0).stop 2024-03-23T17:49:24.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.103+0000 7fbe5c600700 1 -- 172.21.15.86:0/3271253702 shutdown_connections 2024-03-23T17:49:24.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.103+0000 7fbe5c600700 1 --2- 172.21.15.86:0/3271253702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe5410d810 0x7fbe5410fc00 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:24.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.103+0000 7fbe5c600700 1 --2- 172.21.15.86:0/3271253702 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe54072fe0 0x7fbe540733c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:24.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.103+0000 7fbe5c600700 1 --2- 172.21.15.86:0/3271253702 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe54073900 0x7fbe5410d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:24.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.103+0000 7fbe5c600700 1 -- 172.21.15.86:0/3271253702 >> 172.21.15.86:0/3271253702 conn(0x7fbe54077ed0 msgr2=0x7fbe540782e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:24.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.103+0000 7fbe5c600700 1 -- 172.21.15.86:0/3271253702 shutdown_connections 2024-03-23T17:49:24.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.103+0000 7fbe5c600700 1 -- 172.21.15.86:0/3271253702 wait complete. 2024-03-23T17:49:24.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.104+0000 7fbe5c600700 1 Processor -- start 2024-03-23T17:49:24.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.104+0000 7fbe5c600700 1 -- start start 2024-03-23T17:49:24.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.105+0000 7fbe5c600700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe54072fe0 0x7fbe54199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:24.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.105+0000 7fbe5c600700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe54073900 0x7fbe541999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:24.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.105+0000 7fbe5c600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe5410d810 0x7fbe5419dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:24.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.105+0000 7fbe5c600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5419e310 con 0x7fbe5410d810 2024-03-23T17:49:24.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.105+0000 7fbe5c600700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5419e480 con 0x7fbe54072fe0 2024-03-23T17:49:24.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.105+0000 7fbe5c600700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5419e640 con 0x7fbe54073900 2024-03-23T17:49:24.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.105+0000 7fbe5a600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe5410d810 0x7fbe5419dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:24.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.105+0000 7fbe59c00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe54072fe0 0x7fbe54199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:24.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.105+0000 7fbe59c00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe54072fe0 0x7fbe54199460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:52566/0 (socket says 172.21.15.86:52566) 2024-03-23T17:49:24.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.105+0000 7fbe59c00700 1 -- 172.21.15.86:0/841212418 learned_addr learned my addr 172.21.15.86:0/841212418 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:24.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.105+0000 7fbe59200700 1 --2- 172.21.15.86:0/841212418 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe54073900 0x7fbe541999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:24.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.105+0000 7fbe5a600700 1 -- 172.21.15.86:0/841212418 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe54073900 msgr2=0x7fbe541999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:24.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.105+0000 7fbe5a600700 1 --2- 172.21.15.86:0/841212418 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe54073900 0x7fbe541999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:24.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.105+0000 7fbe5a600700 1 -- 172.21.15.86:0/841212418 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe54072fe0 msgr2=0x7fbe54199460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:24.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.105+0000 7fbe5a600700 1 --2- 172.21.15.86:0/841212418 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe54072fe0 0x7fbe54199460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:24.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.105+0000 7fbe5a600700 1 -- 172.21.15.86:0/841212418 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe48009520 con 0x7fbe5410d810 2024-03-23T17:49:24.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.105+0000 7fbe59c00700 1 --2- 172.21.15.86:0/841212418 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe54072fe0 0x7fbe54199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:49:24.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.106+0000 7fbe5a600700 1 --2- 172.21.15.86:0/841212418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe5410d810 0x7fbe5419dd00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbe5406a4d0 tx=0x7fbe4c00bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:24.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.106+0000 7fbe52000700 1 -- 172.21.15.86:0/841212418 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c00e040 con 0x7fbe5410d810 2024-03-23T17:49:24.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.106+0000 7fbe5c600700 1 -- 172.21.15.86:0/841212418 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe4c009d00 con 0x7fbe5410d810 2024-03-23T17:49:24.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.106+0000 7fbe52000700 1 -- 172.21.15.86:0/841212418 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe4c01d470 con 0x7fbe5410d810 2024-03-23T17:49:24.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.106+0000 7fbe52000700 1 -- 172.21.15.86:0/841212418 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c0079d0 con 0x7fbe5410d810 2024-03-23T17:49:24.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.106+0000 7fbe5c600700 1 -- 172.21.15.86:0/841212418 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe541a5fb0 con 0x7fbe5410d810 2024-03-23T17:49:24.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.107+0000 7fbe5c600700 1 -- 172.21.15.86:0/841212418 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbe54074610 con 0x7fbe5410d810 2024-03-23T17:49:24.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.110+0000 7fbe52000700 1 -- 172.21.15.86:0/841212418 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fbe4c01d5e0 con 0x7fbe5410d810 2024-03-23T17:49:24.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.112+0000 7fbe52000700 1 --2- 172.21.15.86:0/841212418 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fbe3c0a11b0 0x7fbe3c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:24.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.112+0000 7fbe52000700 1 -- 172.21.15.86:0/841212418 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fbe4c0cdea0 con 0x7fbe5410d810 2024-03-23T17:49:24.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.112+0000 7fbe59c00700 1 --2- 172.21.15.86:0/841212418 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fbe3c0a11b0 0x7fbe3c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:24.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.112+0000 7fbe52000700 1 --2- 172.21.15.86:0/841212418 >> [v2:172.21.15.86:6810/694846466,v1:172.21.15.86:6811/694846466] conn(0x7fbe3c0aa990 0x7fbe3c0acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:24.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.112+0000 7fbe52000700 1 -- 172.21.15.86:0/841212418 --> [v2:172.21.15.86:6810/694846466,v1:172.21.15.86:6811/694846466] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbe3c0ad460 con 0x7fbe3c0aa990 2024-03-23T17:49:24.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.112+0000 7fbe52000700 1 -- 172.21.15.86:0/841212418 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe4c0ce210 con 0x7fbe5410d810 2024-03-23T17:49:24.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.112+0000 7fbe59200700 1 --2- 172.21.15.86:0/841212418 >> [v2:172.21.15.86:6810/694846466,v1:172.21.15.86:6811/694846466] conn(0x7fbe3c0aa990 0x7fbe3c0acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:24.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.112+0000 7fbe59c00700 1 --2- 172.21.15.86:0/841212418 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fbe3c0a11b0 0x7fbe3c0a3670 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbe48009fd0 tx=0x7fbe48009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:24.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.113+0000 7fbe59200700 1 --2- 172.21.15.86:0/841212418 >> [v2:172.21.15.86:6810/694846466,v1:172.21.15.86:6811/694846466] conn(0x7fbe3c0aa990 0x7fbe3c0acdb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:24.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.114+0000 7fbe52000700 1 -- 172.21.15.86:0/841212418 <== osd.3 v2:172.21.15.86:6810/694846466 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fbe3c0ad460 con 0x7fbe3c0aa990 2024-03-23T17:49:24.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.130+0000 7fbe5c600700 1 -- 172.21.15.86:0/841212418 --> [v2:172.21.15.86:6810/694846466,v1:172.21.15.86:6811/694846466] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbe54113130 con 0x7fbe3c0aa990 2024-03-23T17:49:24.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.131+0000 7fbe52000700 1 -- 172.21.15.86:0/841212418 <== osd.3 v2:172.21.15.86:6810/694846466 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fbe54113130 con 0x7fbe3c0aa990 2024-03-23T17:49:24.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.131+0000 7fbe5c600700 1 -- 172.21.15.86:0/841212418 >> [v2:172.21.15.86:6810/694846466,v1:172.21.15.86:6811/694846466] conn(0x7fbe3c0aa990 msgr2=0x7fbe3c0acdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:24.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.131+0000 7fbe5c600700 1 --2- 172.21.15.86:0/841212418 >> [v2:172.21.15.86:6810/694846466,v1:172.21.15.86:6811/694846466] conn(0x7fbe3c0aa990 0x7fbe3c0acdb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:24.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.131+0000 7fbe5c600700 1 -- 172.21.15.86:0/841212418 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fbe3c0a11b0 msgr2=0x7fbe3c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:24.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.131+0000 7fbe5c600700 1 --2- 172.21.15.86:0/841212418 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fbe3c0a11b0 0x7fbe3c0a3670 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbe48009fd0 tx=0x7fbe48009260 comp rx=0 tx=0).stop 2024-03-23T17:49:24.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.131+0000 7fbe5c600700 1 -- 172.21.15.86:0/841212418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe5410d810 msgr2=0x7fbe5419dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:24.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.131+0000 7fbe5c600700 1 --2- 172.21.15.86:0/841212418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe5410d810 0x7fbe5419dd00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbe5406a4d0 tx=0x7fbe4c00bdd0 comp rx=0 tx=0).stop 2024-03-23T17:49:24.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.131+0000 7fbe5a600700 1 -- 172.21.15.86:0/841212418 reap_dead start 2024-03-23T17:49:24.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.131+0000 7fbe5c600700 1 -- 172.21.15.86:0/841212418 shutdown_connections 2024-03-23T17:49:24.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.131+0000 7fbe5c600700 1 -- 172.21.15.86:0/841212418 >> 172.21.15.86:0/841212418 conn(0x7fbe54077ed0 msgr2=0x7fbe540fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:24.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.132+0000 7fbe5c600700 1 -- 172.21.15.86:0/841212418 shutdown_connections 2024-03-23T17:49:24.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:24.132+0000 7fbe5c600700 1 -- 172.21.15.86:0/841212418 wait complete. 2024-03-23T17:49:24.705 INFO:teuthology.orchestra.run.smithi086.stdout:55834574901 2024-03-23T17:49:24.706 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-03-23T17:49:25.119 INFO:teuthology.orchestra.run.smithi086.stdout:111669149742 2024-03-23T17:49:25.119 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-03-23T17:49:25.393 INFO:teuthology.orchestra.run.smithi086.stdout:322122547217 2024-03-23T17:49:25.393 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.11 2024-03-23T17:49:25.707 INFO:teuthology.orchestra.run.smithi086.stdout:240518168602 2024-03-23T17:49:25.707 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.8 2024-03-23T17:49:26.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:25 smithi086 ceph-mon[38221]: pgmap v248: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-23T17:49:26.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.051+0000 7f3d79c00700 1 -- 172.21.15.86:0/1125493721 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3d74103b80 msgr2=0x7f3d74107bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:26.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.051+0000 7f3d79c00700 1 --2- 172.21.15.86:0/1125493721 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3d74103b80 0x7f3d74107bd0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3d600098f0 tx=0x7f3d60009c00 comp rx=0 tx=0).stop 2024-03-23T17:49:26.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.051+0000 7f3d79c00700 1 -- 172.21.15.86:0/1125493721 shutdown_connections 2024-03-23T17:49:26.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.051+0000 7f3d79c00700 1 --2- 172.21.15.86:0/1125493721 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d741082b0 0x7f3d7410fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:26.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.051+0000 7f3d79c00700 1 --2- 172.21.15.86:0/1125493721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d741031d0 0x7f3d741035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:26.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.051+0000 7f3d79c00700 1 --2- 172.21.15.86:0/1125493721 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3d74103b80 0x7f3d74107bd0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:26.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.051+0000 7f3d79c00700 1 -- 172.21.15.86:0/1125493721 >> 172.21.15.86:0/1125493721 conn(0x7f3d740fea60 msgr2=0x7f3d74100e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:26.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.051+0000 7f3d79c00700 1 -- 172.21.15.86:0/1125493721 shutdown_connections 2024-03-23T17:49:26.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.051+0000 7f3d79c00700 1 -- 172.21.15.86:0/1125493721 wait complete. 2024-03-23T17:49:26.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.052+0000 7f3d79c00700 1 Processor -- start 2024-03-23T17:49:26.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.052+0000 7f3d79c00700 1 -- start start 2024-03-23T17:49:26.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.053+0000 7f3d79c00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3d741031d0 0x7f3d741994a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:26.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.053+0000 7f3d79c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d74103b80 0x7f3d741999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:26.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.053+0000 7f3d79c00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d741082b0 0x7f3d7419dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:26.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.053+0000 7f3d79c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7419e350 con 0x7f3d74103b80 2024-03-23T17:49:26.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.053+0000 7f3d79c00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7419e4c0 con 0x7f3d741082b0 2024-03-23T17:49:26.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.053+0000 7f3d79c00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7419e680 con 0x7f3d741031d0 2024-03-23T17:49:26.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.053+0000 7f3d72000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d74103b80 0x7f3d741999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:26.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.053+0000 7f3d72000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d74103b80 0x7f3d741999e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58758/0 (socket says 172.21.15.86:58758) 2024-03-23T17:49:26.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.053+0000 7f3d72000700 1 -- 172.21.15.86:0/228099979 learned_addr learned my addr 172.21.15.86:0/228099979 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:26.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.053+0000 7f3d72a00700 1 --2- 172.21.15.86:0/228099979 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3d741031d0 0x7f3d741994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:26.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.053+0000 7f3d73400700 1 --2- 172.21.15.86:0/228099979 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d741082b0 0x7f3d7419dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:26.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.054+0000 7f3d72000700 1 -- 172.21.15.86:0/228099979 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3d741031d0 msgr2=0x7f3d741994a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:26.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.054+0000 7f3d72000700 1 --2- 172.21.15.86:0/228099979 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3d741031d0 0x7f3d741994a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:26.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.054+0000 7f3d72000700 1 -- 172.21.15.86:0/228099979 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d741082b0 msgr2=0x7f3d7419dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:26.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.054+0000 7f3d72000700 1 --2- 172.21.15.86:0/228099979 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d741082b0 0x7f3d7419dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:26.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.054+0000 7f3d72000700 1 -- 172.21.15.86:0/228099979 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d6c009d00 con 0x7f3d74103b80 2024-03-23T17:49:26.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.054+0000 7f3d72000700 1 --2- 172.21.15.86:0/228099979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d74103b80 0x7f3d741999e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3d60009230 tx=0x7f3d6000fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:26.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.054+0000 7f3d6b400700 1 -- 172.21.15.86:0/228099979 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6001c070 con 0x7f3d74103b80 2024-03-23T17:49:26.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.054+0000 7f3d6b400700 1 -- 172.21.15.86:0/228099979 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d6000ba20 con 0x7f3d74103b80 2024-03-23T17:49:26.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.054+0000 7f3d79c00700 1 -- 172.21.15.86:0/228099979 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d60009520 con 0x7f3d74103b80 2024-03-23T17:49:26.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.054+0000 7f3d6b400700 1 -- 172.21.15.86:0/228099979 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d600038e0 con 0x7f3d74103b80 2024-03-23T17:49:26.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.054+0000 7f3d79c00700 1 -- 172.21.15.86:0/228099979 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d7419eb80 con 0x7f3d74103b80 2024-03-23T17:49:26.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.055+0000 7f3d68c00700 1 -- 172.21.15.86:0/228099979 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3d4c000fc0 con 0x7f3d74103b80 2024-03-23T17:49:26.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:25 smithi092 ceph-mon[79089]: pgmap v248: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-23T17:49:26.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.057+0000 7f3d6b400700 1 -- 172.21.15.86:0/228099979 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f3d60020070 con 0x7f3d74103b80 2024-03-23T17:49:26.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.058+0000 7f3d6b400700 1 --2- 172.21.15.86:0/228099979 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f3d580a1230 0x7f3d580a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:26.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.058+0000 7f3d6b400700 1 -- 172.21.15.86:0/228099979 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f3d600cd450 con 0x7f3d74103b80 2024-03-23T17:49:26.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.059+0000 7f3d72a00700 1 --2- 172.21.15.86:0/228099979 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f3d580a1230 0x7f3d580a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:26.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.059+0000 7f3d6b400700 1 --2- 172.21.15.86:0/228099979 >> [v2:172.21.15.152:6808/1039375239,v1:172.21.15.152:6809/1039375239] conn(0x7f3d580aaa10 0x7f3d580ace30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:26.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.059+0000 7f3d6b400700 1 -- 172.21.15.86:0/228099979 --> [v2:172.21.15.152:6808/1039375239,v1:172.21.15.152:6809/1039375239] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3d580ad4e0 con 0x7f3d580aaa10 2024-03-23T17:49:26.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.059+0000 7f3d6b400700 1 -- 172.21.15.86:0/228099979 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3d600cd7c0 con 0x7f3d74103b80 2024-03-23T17:49:26.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.059+0000 7f3d73400700 1 --2- 172.21.15.86:0/228099979 >> [v2:172.21.15.152:6808/1039375239,v1:172.21.15.152:6809/1039375239] conn(0x7f3d580aaa10 0x7f3d580ace30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:26.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.059+0000 7f3d72a00700 1 --2- 172.21.15.86:0/228099979 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f3d580a1230 0x7f3d580a36f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3d64009c60 tx=0x7f3d64009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:26.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.060+0000 7f3d73400700 1 --2- 172.21.15.86:0/228099979 >> [v2:172.21.15.152:6808/1039375239,v1:172.21.15.152:6809/1039375239] conn(0x7f3d580aaa10 0x7f3d580ace30 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:26.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.061+0000 7f3d6b400700 1 -- 172.21.15.86:0/228099979 <== osd.5 v2:172.21.15.152:6808/1039375239 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f3d580ad4e0 con 0x7f3d580aaa10 2024-03-23T17:49:26.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.077+0000 7f3d68c00700 1 -- 172.21.15.86:0/228099979 --> [v2:172.21.15.152:6808/1039375239,v1:172.21.15.152:6809/1039375239] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3d4c002d40 con 0x7f3d580aaa10 2024-03-23T17:49:26.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.078+0000 7f3d6b400700 1 -- 172.21.15.86:0/228099979 <== osd.5 v2:172.21.15.152:6808/1039375239 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f3d4c002d40 con 0x7f3d580aaa10 2024-03-23T17:49:26.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.078+0000 7f3d68c00700 1 -- 172.21.15.86:0/228099979 >> [v2:172.21.15.152:6808/1039375239,v1:172.21.15.152:6809/1039375239] conn(0x7f3d580aaa10 msgr2=0x7f3d580ace30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:26.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.078+0000 7f3d68c00700 1 --2- 172.21.15.86:0/228099979 >> [v2:172.21.15.152:6808/1039375239,v1:172.21.15.152:6809/1039375239] conn(0x7f3d580aaa10 0x7f3d580ace30 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:26.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.078+0000 7f3d68c00700 1 -- 172.21.15.86:0/228099979 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f3d580a1230 msgr2=0x7f3d580a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:26.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.078+0000 7f3d68c00700 1 --2- 172.21.15.86:0/228099979 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f3d580a1230 0x7f3d580a36f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3d64009c60 tx=0x7f3d64009260 comp rx=0 tx=0).stop 2024-03-23T17:49:26.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.078+0000 7f3d68c00700 1 -- 172.21.15.86:0/228099979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d74103b80 msgr2=0x7f3d741999e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:26.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.078+0000 7f3d68c00700 1 --2- 172.21.15.86:0/228099979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d74103b80 0x7f3d741999e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3d60009230 tx=0x7f3d6000fa00 comp rx=0 tx=0).stop 2024-03-23T17:49:26.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.078+0000 7f3d73400700 1 -- 172.21.15.86:0/228099979 reap_dead start 2024-03-23T17:49:26.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.078+0000 7f3d68c00700 1 -- 172.21.15.86:0/228099979 shutdown_connections 2024-03-23T17:49:26.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.078+0000 7f3d68c00700 1 -- 172.21.15.86:0/228099979 >> 172.21.15.86:0/228099979 conn(0x7f3d740fea60 msgr2=0x7f3d740ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:26.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.079+0000 7f3d68c00700 1 -- 172.21.15.86:0/228099979 shutdown_connections 2024-03-23T17:49:26.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.079+0000 7f3d68c00700 1 -- 172.21.15.86:0/228099979 wait complete. 2024-03-23T17:49:26.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:25 smithi152 ceph-mon[47949]: pgmap v248: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-23T17:49:26.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.486+0000 7faf8ec00700 1 -- 172.21.15.86:0/2850963971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf88111b30 msgr2=0x7faf88113f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:26.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.486+0000 7faf8ec00700 1 --2- 172.21.15.86:0/2850963971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf88111b30 0x7faf88113f60 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7faf8000b0a0 tx=0x7faf8000b3b0 comp rx=0 tx=0).stop 2024-03-23T17:49:26.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.486+0000 7faf8ec00700 1 -- 172.21.15.86:0/2850963971 shutdown_connections 2024-03-23T17:49:26.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.486+0000 7faf8ec00700 1 --2- 172.21.15.86:0/2850963971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf88111b30 0x7faf88113f60 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:26.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.486+0000 7faf8ec00700 1 --2- 172.21.15.86:0/2850963971 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf88074d30 0x7faf88072fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:26.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.486+0000 7faf8ec00700 1 --2- 172.21.15.86:0/2850963971 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf88073520 0x7faf880739a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:26.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.486+0000 7faf8ec00700 1 -- 172.21.15.86:0/2850963971 >> 172.21.15.86:0/2850963971 conn(0x7faf88077fa0 msgr2=0x7faf880783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:26.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.486+0000 7faf8ec00700 1 -- 172.21.15.86:0/2850963971 shutdown_connections 2024-03-23T17:49:26.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.486+0000 7faf8ec00700 1 -- 172.21.15.86:0/2850963971 wait complete. 2024-03-23T17:49:26.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.487+0000 7faf8ec00700 1 Processor -- start 2024-03-23T17:49:26.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.487+0000 7faf8ec00700 1 -- start start 2024-03-23T17:49:26.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.487+0000 7faf8ec00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf88073520 0x7faf881a36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:26.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.487+0000 7faf8ec00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf88074d30 0x7faf881a3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:26.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.487+0000 7faf8ec00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf88111b30 0x7faf8819d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:26.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.487+0000 7faf8ec00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf8819dea0 con 0x7faf88073520 2024-03-23T17:49:26.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.487+0000 7faf8ec00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf8819e010 con 0x7faf88074d30 2024-03-23T17:49:26.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.487+0000 7faf8ec00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf8819e1d0 con 0x7faf88111b30 2024-03-23T17:49:26.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.488+0000 7faf87e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf88073520 0x7faf881a36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:26.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.488+0000 7faf87e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf88073520 0x7faf881a36a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58786/0 (socket says 172.21.15.86:58786) 2024-03-23T17:49:26.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.488+0000 7faf87e00700 1 -- 172.21.15.86:0/3002377295 learned_addr learned my addr 172.21.15.86:0/3002377295 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:26.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.488+0000 7faf7fe00700 1 --2- 172.21.15.86:0/3002377295 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf88074d30 0x7faf881a3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:26.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.488+0000 7faf8cc00700 1 --2- 172.21.15.86:0/3002377295 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf88111b30 0x7faf8819d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:26.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.488+0000 7faf87e00700 1 -- 172.21.15.86:0/3002377295 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf88111b30 msgr2=0x7faf8819d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:26.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.488+0000 7faf87e00700 1 --2- 172.21.15.86:0/3002377295 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf88111b30 0x7faf8819d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:26.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.488+0000 7faf87e00700 1 -- 172.21.15.86:0/3002377295 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf88074d30 msgr2=0x7faf881a3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:26.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.488+0000 7faf87e00700 1 --2- 172.21.15.86:0/3002377295 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf88074d30 0x7faf881a3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:26.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.488+0000 7faf87e00700 1 -- 172.21.15.86:0/3002377295 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf80009d00 con 0x7faf88073520 2024-03-23T17:49:26.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.488+0000 7faf7fe00700 1 --2- 172.21.15.86:0/3002377295 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf88074d30 0x7faf881a3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:49:26.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.488+0000 7faf87e00700 1 --2- 172.21.15.86:0/3002377295 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf88073520 0x7faf881a36a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7faf7800e9a0 tx=0x7faf7800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:26.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.489+0000 7faf85600700 1 -- 172.21.15.86:0/3002377295 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf78009790 con 0x7faf88073520 2024-03-23T17:49:26.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.489+0000 7faf8ec00700 1 -- 172.21.15.86:0/3002377295 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf8819e4b0 con 0x7faf88073520 2024-03-23T17:49:26.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.489+0000 7faf85600700 1 -- 172.21.15.86:0/3002377295 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf78010430 con 0x7faf88073520 2024-03-23T17:49:26.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.489+0000 7faf85600700 1 -- 172.21.15.86:0/3002377295 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf78018680 con 0x7faf88073520 2024-03-23T17:49:26.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.489+0000 7faf8ec00700 1 -- 172.21.15.86:0/3002377295 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf881aa690 con 0x7faf88073520 2024-03-23T17:49:26.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.490+0000 7faf8ec00700 1 -- 172.21.15.86:0/3002377295 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7faf881176d0 con 0x7faf88073520 2024-03-23T17:49:26.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.492+0000 7faf85600700 1 -- 172.21.15.86:0/3002377295 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7faf780105a0 con 0x7faf88073520 2024-03-23T17:49:26.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.493+0000 7faf85600700 1 --2- 172.21.15.86:0/3002377295 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7faf6c0a11b0 0x7faf6c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:26.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.493+0000 7faf85600700 1 -- 172.21.15.86:0/3002377295 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7faf78014070 con 0x7faf88073520 2024-03-23T17:49:26.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.493+0000 7faf7fe00700 1 --2- 172.21.15.86:0/3002377295 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7faf6c0a11b0 0x7faf6c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:26.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.493+0000 7faf85600700 1 --2- 172.21.15.86:0/3002377295 >> [v2:172.21.15.92:6824/4259465100,v1:172.21.15.92:6825/4259465100] conn(0x7faf6c0aa990 0x7faf6c0acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:26.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.493+0000 7faf85600700 1 -- 172.21.15.86:0/3002377295 --> [v2:172.21.15.92:6824/4259465100,v1:172.21.15.92:6825/4259465100] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7faf6c0ad460 con 0x7faf6c0aa990 2024-03-23T17:49:26.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.493+0000 7faf85600700 1 -- 172.21.15.86:0/3002377295 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf780cd200 con 0x7faf88073520 2024-03-23T17:49:26.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.494+0000 7faf7fe00700 1 --2- 172.21.15.86:0/3002377295 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7faf6c0a11b0 0x7faf6c0a3670 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7faf8819ec00 tx=0x7faf70009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:26.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.494+0000 7faf8cc00700 1 --2- 172.21.15.86:0/3002377295 >> [v2:172.21.15.92:6824/4259465100,v1:172.21.15.92:6825/4259465100] conn(0x7faf6c0aa990 0x7faf6c0acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:26.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.494+0000 7faf8cc00700 1 --2- 172.21.15.86:0/3002377295 >> [v2:172.21.15.92:6824/4259465100,v1:172.21.15.92:6825/4259465100] conn(0x7faf6c0aa990 0x7faf6c0acdb0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:26.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.495+0000 7faf85600700 1 -- 172.21.15.86:0/3002377295 <== osd.10 v2:172.21.15.92:6824/4259465100 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7faf6c0ad460 con 0x7faf6c0aa990 2024-03-23T17:49:26.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.512+0000 7faf8ec00700 1 -- 172.21.15.86:0/3002377295 --> [v2:172.21.15.92:6824/4259465100,v1:172.21.15.92:6825/4259465100] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7faf881110f0 con 0x7faf6c0aa990 2024-03-23T17:49:26.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.513+0000 7faf85600700 1 -- 172.21.15.86:0/3002377295 <== osd.10 v2:172.21.15.92:6824/4259465100 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7faf881110f0 con 0x7faf6c0aa990 2024-03-23T17:49:26.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.513+0000 7faf8ec00700 1 -- 172.21.15.86:0/3002377295 >> [v2:172.21.15.92:6824/4259465100,v1:172.21.15.92:6825/4259465100] conn(0x7faf6c0aa990 msgr2=0x7faf6c0acdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:26.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.513+0000 7faf8ec00700 1 --2- 172.21.15.86:0/3002377295 >> [v2:172.21.15.92:6824/4259465100,v1:172.21.15.92:6825/4259465100] conn(0x7faf6c0aa990 0x7faf6c0acdb0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:26.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.513+0000 7faf8ec00700 1 -- 172.21.15.86:0/3002377295 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7faf6c0a11b0 msgr2=0x7faf6c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:26.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.513+0000 7faf8ec00700 1 --2- 172.21.15.86:0/3002377295 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7faf6c0a11b0 0x7faf6c0a3670 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7faf8819ec00 tx=0x7faf70009260 comp rx=0 tx=0).stop 2024-03-23T17:49:26.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.513+0000 7faf8ec00700 1 -- 172.21.15.86:0/3002377295 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf88073520 msgr2=0x7faf881a36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:26.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.513+0000 7faf8ec00700 1 --2- 172.21.15.86:0/3002377295 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf88073520 0x7faf881a36a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7faf7800e9a0 tx=0x7faf7800ecb0 comp rx=0 tx=0).stop 2024-03-23T17:49:26.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.513+0000 7faf8cc00700 1 -- 172.21.15.86:0/3002377295 reap_dead start 2024-03-23T17:49:26.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.513+0000 7faf8ec00700 1 -- 172.21.15.86:0/3002377295 shutdown_connections 2024-03-23T17:49:26.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.513+0000 7faf8ec00700 1 -- 172.21.15.86:0/3002377295 >> 172.21.15.86:0/3002377295 conn(0x7faf88077fa0 msgr2=0x7faf881107a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:26.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.514+0000 7faf8ec00700 1 -- 172.21.15.86:0/3002377295 shutdown_connections 2024-03-23T17:49:26.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.514+0000 7faf8ec00700 1 -- 172.21.15.86:0/3002377295 wait complete. 2024-03-23T17:49:26.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.651+0000 7f1c7d800700 1 -- 172.21.15.86:0/236697465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c781031e0 msgr2=0x7f1c781035c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:26.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.651+0000 7f1c7d800700 1 --2- 172.21.15.86:0/236697465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c781031e0 0x7f1c781035c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1c68009990 tx=0x7f1c68009ca0 comp rx=0 tx=0).stop 2024-03-23T17:49:26.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.651+0000 7f1c7d800700 1 -- 172.21.15.86:0/236697465 shutdown_connections 2024-03-23T17:49:26.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.651+0000 7f1c7d800700 1 --2- 172.21.15.86:0/236697465 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c781082c0 0x7f1c7810fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:26.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.651+0000 7f1c7d800700 1 --2- 172.21.15.86:0/236697465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c781031e0 0x7f1c781035c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:26.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.651+0000 7f1c7d800700 1 --2- 172.21.15.86:0/236697465 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1c78103b90 0x7f1c78107be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:26.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.651+0000 7f1c7d800700 1 -- 172.21.15.86:0/236697465 >> 172.21.15.86:0/236697465 conn(0x7f1c780fea70 msgr2=0x7f1c78100e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:26.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.651+0000 7f1c7d800700 1 -- 172.21.15.86:0/236697465 shutdown_connections 2024-03-23T17:49:26.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.651+0000 7f1c7d800700 1 -- 172.21.15.86:0/236697465 wait complete. 2024-03-23T17:49:26.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.652+0000 7f1c7d800700 1 Processor -- start 2024-03-23T17:49:26.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.652+0000 7f1c7d800700 1 -- start start 2024-03-23T17:49:26.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.653+0000 7f1c7d800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1c781031e0 0x7f1c78199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:26.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.653+0000 7f1c7d800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c78103b90 0x7f1c78199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:26.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.653+0000 7f1c7d800700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c781082c0 0x7f1c7819dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:26.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.653+0000 7f1c7d800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7819e330 con 0x7f1c78103b90 2024-03-23T17:49:26.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.653+0000 7f1c7d800700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7819e4a0 con 0x7f1c781031e0 2024-03-23T17:49:26.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.653+0000 7f1c7d800700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7819e660 con 0x7f1c781082c0 2024-03-23T17:49:26.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.653+0000 7f1c76000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c78103b90 0x7f1c78199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:26.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.653+0000 7f1c76a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1c781031e0 0x7f1c78199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:26.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.653+0000 7f1c76a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1c781031e0 0x7f1c78199430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:44160/0 (socket says 172.21.15.86:44160) 2024-03-23T17:49:26.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.653+0000 7f1c76a00700 1 -- 172.21.15.86:0/1558210034 learned_addr learned my addr 172.21.15.86:0/1558210034 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:26.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.653+0000 7f1c77400700 1 --2- 172.21.15.86:0/1558210034 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c781082c0 0x7f1c7819dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:26.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.654+0000 7f1c76000700 1 -- 172.21.15.86:0/1558210034 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c781082c0 msgr2=0x7f1c7819dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:26.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.654+0000 7f1c76000700 1 --2- 172.21.15.86:0/1558210034 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c781082c0 0x7f1c7819dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:26.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.654+0000 7f1c76000700 1 -- 172.21.15.86:0/1558210034 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1c781031e0 msgr2=0x7f1c78199430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:26.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.654+0000 7f1c76000700 1 --2- 172.21.15.86:0/1558210034 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1c781031e0 0x7f1c78199430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:26.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.654+0000 7f1c76000700 1 -- 172.21.15.86:0/1558210034 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c64009520 con 0x7f1c78103b90 2024-03-23T17:49:26.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.654+0000 7f1c77400700 1 --2- 172.21.15.86:0/1558210034 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c781082c0 0x7f1c7819dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:49:26.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.654+0000 7f1c76a00700 1 --2- 172.21.15.86:0/1558210034 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1c781031e0 0x7f1c78199430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:49:26.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.654+0000 7f1c76000700 1 --2- 172.21.15.86:0/1558210034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c78103b90 0x7f1c78199970 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1c6400e9c0 tx=0x7f1c6400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:26.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.654+0000 7f1c6f400700 1 -- 172.21.15.86:0/1558210034 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6400cb70 con 0x7f1c78103b90 2024-03-23T17:49:26.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.654+0000 7f1c6f400700 1 -- 172.21.15.86:0/1558210034 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c6400ee60 con 0x7f1c78103b90 2024-03-23T17:49:26.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.654+0000 7f1c7d800700 1 -- 172.21.15.86:0/1558210034 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c680095f0 con 0x7f1c78103b90 2024-03-23T17:49:26.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.655+0000 7f1c6f400700 1 -- 172.21.15.86:0/1558210034 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c64003c60 con 0x7f1c78103b90 2024-03-23T17:49:26.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.655+0000 7f1c7d800700 1 -- 172.21.15.86:0/1558210034 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c781a5fd0 con 0x7f1c78103b90 2024-03-23T17:49:26.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.655+0000 7f1c7d800700 1 -- 172.21.15.86:0/1558210034 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1c781131a0 con 0x7f1c78103b90 2024-03-23T17:49:26.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.659+0000 7f1c6f400700 1 -- 172.21.15.86:0/1558210034 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f1c64003e00 con 0x7f1c78103b90 2024-03-23T17:49:26.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.660+0000 7f1c6f400700 1 --2- 172.21.15.86:0/1558210034 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f1c5c0a1230 0x7f1c5c0a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:26.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.660+0000 7f1c6f400700 1 -- 172.21.15.86:0/1558210034 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f1c64014070 con 0x7f1c78103b90 2024-03-23T17:49:26.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.661+0000 7f1c76a00700 1 --2- 172.21.15.86:0/1558210034 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f1c5c0a1230 0x7f1c5c0a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:26.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.661+0000 7f1c6f400700 1 --2- 172.21.15.86:0/1558210034 >> [v2:172.21.15.152:6800/1601265891,v1:172.21.15.152:6801/1601265891] conn(0x7f1c5c0aaa10 0x7f1c5c0ace30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:26.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.661+0000 7f1c6f400700 1 -- 172.21.15.86:0/1558210034 --> [v2:172.21.15.152:6800/1601265891,v1:172.21.15.152:6801/1601265891] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1c5c0ad4e0 con 0x7f1c5c0aaa10 2024-03-23T17:49:26.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.661+0000 7f1c6f400700 1 -- 172.21.15.86:0/1558210034 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c640d12a0 con 0x7f1c78103b90 2024-03-23T17:49:26.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.661+0000 7f1c77400700 1 --2- 172.21.15.86:0/1558210034 >> [v2:172.21.15.152:6800/1601265891,v1:172.21.15.152:6801/1601265891] conn(0x7f1c5c0aaa10 0x7f1c5c0ace30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:26.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.661+0000 7f1c76a00700 1 --2- 172.21.15.86:0/1558210034 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f1c5c0a1230 0x7f1c5c0a36f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1c68006010 tx=0x7f1c6800b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:26.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.662+0000 7f1c77400700 1 --2- 172.21.15.86:0/1558210034 >> [v2:172.21.15.152:6800/1601265891,v1:172.21.15.152:6801/1601265891] conn(0x7f1c5c0aaa10 0x7f1c5c0ace30 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:26.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.663+0000 7f1c6f400700 1 -- 172.21.15.86:0/1558210034 <== osd.2 v2:172.21.15.152:6800/1601265891 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f1c5c0ad4e0 con 0x7f1c5c0aaa10 2024-03-23T17:49:26.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.685+0000 7f1c7d800700 1 -- 172.21.15.86:0/1558210034 --> [v2:172.21.15.152:6800/1601265891,v1:172.21.15.152:6801/1601265891] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1c78061430 con 0x7f1c5c0aaa10 2024-03-23T17:49:26.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.685+0000 7f1c6f400700 1 -- 172.21.15.86:0/1558210034 <== osd.2 v2:172.21.15.152:6800/1601265891 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f1c78061430 con 0x7f1c5c0aaa10 2024-03-23T17:49:26.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.686+0000 7f1c7d800700 1 -- 172.21.15.86:0/1558210034 >> [v2:172.21.15.152:6800/1601265891,v1:172.21.15.152:6801/1601265891] conn(0x7f1c5c0aaa10 msgr2=0x7f1c5c0ace30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:26.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.686+0000 7f1c7d800700 1 --2- 172.21.15.86:0/1558210034 >> [v2:172.21.15.152:6800/1601265891,v1:172.21.15.152:6801/1601265891] conn(0x7f1c5c0aaa10 0x7f1c5c0ace30 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:26.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.686+0000 7f1c7d800700 1 -- 172.21.15.86:0/1558210034 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f1c5c0a1230 msgr2=0x7f1c5c0a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:26.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.686+0000 7f1c7d800700 1 --2- 172.21.15.86:0/1558210034 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f1c5c0a1230 0x7f1c5c0a36f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1c68006010 tx=0x7f1c6800b260 comp rx=0 tx=0).stop 2024-03-23T17:49:26.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.686+0000 7f1c7d800700 1 -- 172.21.15.86:0/1558210034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c78103b90 msgr2=0x7f1c78199970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:26.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.686+0000 7f1c7d800700 1 --2- 172.21.15.86:0/1558210034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c78103b90 0x7f1c78199970 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1c6400e9c0 tx=0x7f1c6400ecd0 comp rx=0 tx=0).stop 2024-03-23T17:49:26.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.686+0000 7f1c77400700 1 -- 172.21.15.86:0/1558210034 reap_dead start 2024-03-23T17:49:26.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.686+0000 7f1c7d800700 1 -- 172.21.15.86:0/1558210034 shutdown_connections 2024-03-23T17:49:26.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.686+0000 7f1c7d800700 1 -- 172.21.15.86:0/1558210034 >> 172.21.15.86:0/1558210034 conn(0x7f1c780fea70 msgr2=0x7f1c780fff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:26.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.686+0000 7f1c7d800700 1 -- 172.21.15.86:0/1558210034 shutdown_connections 2024-03-23T17:49:26.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:26.686+0000 7f1c7d800700 1 -- 172.21.15.86:0/1558210034 wait complete. 2024-03-23T17:49:26.959 INFO:teuthology.orchestra.run.smithi086.stdout:292057776148 2024-03-23T17:49:26.960 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.10 2024-03-23T17:49:27.070 INFO:teuthology.orchestra.run.smithi086.stdout:163208757287 2024-03-23T17:49:27.070 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-03-23T17:49:27.397 INFO:teuthology.orchestra.run.smithi086.stdout:77309411379 2024-03-23T17:49:27.398 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-03-23T17:49:27.748 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T17:49:27.765 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:27 smithi086 ceph-mon[38221]: pgmap v249: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-23T17:49:27.900 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T17:49:28.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:27 smithi092 ceph-mon[79089]: pgmap v249: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-23T17:49:28.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:27 smithi152 ceph-mon[47949]: pgmap v249: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-23T17:49:28.849 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:28 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-03-23T17:49:28.849 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:28 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-03-23T17:49:28.849 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:28 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-03-23T17:49:28.849 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:28 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]: dispatch 2024-03-23T17:49:28.849 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:28 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-03-23T17:49:28.849 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:28 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-03-23T17:49:28.849 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:28 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-03-23T17:49:28.958 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T17:49:29.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:28 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-03-23T17:49:29.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:28 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-03-23T17:49:29.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:28 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-03-23T17:49:29.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:28 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]: dispatch 2024-03-23T17:49:29.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:28 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-03-23T17:49:29.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:28 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-03-23T17:49:29.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:28 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-03-23T17:49:29.061 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T17:49:29.085 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:28 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-03-23T17:49:29.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:28 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-03-23T17:49:29.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:28 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-03-23T17:49:29.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:28 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]: dispatch 2024-03-23T17:49:29.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:28 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-03-23T17:49:29.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:28 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-03-23T17:49:29.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:28 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-03-23T17:49:29.369 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T17:49:29.372 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T17:49:29.376 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T17:49:29.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.377+0000 7ff3bb800700 1 -- 172.21.15.86:0/1744852244 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3b410f550 msgr2=0x7ff3b4107db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:29.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.377+0000 7ff3bb800700 1 --2- 172.21.15.86:0/1744852244 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3b410f550 0x7ff3b4107db0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4009990 tx=0x7ff3a4009ca0 comp rx=0 tx=0).stop 2024-03-23T17:49:29.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.378+0000 7ff3bb800700 1 -- 172.21.15.86:0/1744852244 shutdown_connections 2024-03-23T17:49:29.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.378+0000 7ff3bb800700 1 --2- 172.21.15.86:0/1744852244 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff3b411a590 0x7ff3b411c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:29.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.378+0000 7ff3bb800700 1 --2- 172.21.15.86:0/1744852244 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3b410f550 0x7ff3b4107db0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:29.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.378+0000 7ff3bb800700 1 --2- 172.21.15.86:0/1744852244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3b41082f0 0x7ff3b4108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:29.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.378+0000 7ff3bb800700 1 -- 172.21.15.86:0/1744852244 >> 172.21.15.86:0/1744852244 conn(0x7ff3b406be80 msgr2=0x7ff3b406c290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:29.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.379+0000 7ff3bb800700 1 -- 172.21.15.86:0/1744852244 shutdown_connections 2024-03-23T17:49:29.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.379+0000 7ff3bb800700 1 -- 172.21.15.86:0/1744852244 wait complete. 2024-03-23T17:49:29.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.381+0000 7ff3bb800700 1 Processor -- start 2024-03-23T17:49:29.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.383+0000 7ff3bb800700 1 -- start start 2024-03-23T17:49:29.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.384+0000 7ff3bb800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3b41082f0 0x7ff3b41a6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:29.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.384+0000 7ff3bb800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3b410f550 0x7ff3b41a65c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:29.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.384+0000 7ff3bb800700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff3b411a590 0x7ff3b41aa970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:29.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.384+0000 7ff3bb800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b41aaf80 con 0x7ff3b410f550 2024-03-23T17:49:29.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.384+0000 7ff3bb800700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b41ab0f0 con 0x7ff3b41082f0 2024-03-23T17:49:29.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.384+0000 7ff3bb800700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b41ab2b0 con 0x7ff3b411a590 2024-03-23T17:49:29.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.385+0000 7ff3b9800700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff3b411a590 0x7ff3b41aa970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:29.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.385+0000 7ff3b9800700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff3b411a590 0x7ff3b41aa970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:56806/0 (socket says 172.21.15.86:56806) 2024-03-23T17:49:29.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.385+0000 7ff3b9800700 1 -- 172.21.15.86:0/2598237971 learned_addr learned my addr 172.21.15.86:0/2598237971 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:29.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.385+0000 7ff3b3e00700 1 --2- 172.21.15.86:0/2598237971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3b410f550 0x7ff3b41a65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:29.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.385+0000 7ff3b9800700 1 -- 172.21.15.86:0/2598237971 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3b41082f0 msgr2=0x7ff3b41a6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:29.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.385+0000 7ff3b9800700 1 --2- 172.21.15.86:0/2598237971 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3b41082f0 0x7ff3b41a6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:29.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.385+0000 7ff3b9800700 1 -- 172.21.15.86:0/2598237971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3b410f550 msgr2=0x7ff3b41a65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:29.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.385+0000 7ff3b9800700 1 --2- 172.21.15.86:0/2598237971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3b410f550 0x7ff3b41a65c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:29.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.385+0000 7ff3b9800700 1 -- 172.21.15.86:0/2598237971 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3a40095f0 con 0x7ff3b411a590 2024-03-23T17:49:29.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.385+0000 7ff3b9800700 1 --2- 172.21.15.86:0/2598237971 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff3b411a590 0x7ff3b41aa970 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff3a000c8c0 tx=0x7ff3a000cc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:29.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.386+0000 7ff3b1600700 1 -- 172.21.15.86:0/2598237971 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a0007b00 con 0x7ff3b411a590 2024-03-23T17:49:29.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.386+0000 7ff3bb800700 1 -- 172.21.15.86:0/2598237971 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3b41ab590 con 0x7ff3b411a590 2024-03-23T17:49:29.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.386+0000 7ff3bb800700 1 -- 172.21.15.86:0/2598237971 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3b410ff40 con 0x7ff3b411a590 2024-03-23T17:49:29.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.386+0000 7ff3b1600700 1 -- 172.21.15.86:0/2598237971 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff3a0004cf0 con 0x7ff3b411a590 2024-03-23T17:49:29.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.386+0000 7ff3b1600700 1 -- 172.21.15.86:0/2598237971 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a00055c0 con 0x7ff3b411a590 2024-03-23T17:49:29.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.390+0000 7ff3bb800700 1 -- 172.21.15.86:0/2598237971 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3940052f0 con 0x7ff3b411a590 2024-03-23T17:49:29.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.394+0000 7ff3b1600700 1 -- 172.21.15.86:0/2598237971 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7ff3a0004750 con 0x7ff3b411a590 2024-03-23T17:49:29.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.396+0000 7ff3b1600700 1 --2- 172.21.15.86:0/2598237971 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7ff3980a12f0 0x7ff3980a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:29.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.397+0000 7ff3b1600700 1 -- 172.21.15.86:0/2598237971 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8182+0+0 (secure 0 0 0) 0x7ff3a00cc680 con 0x7ff3b411a590 2024-03-23T17:49:29.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.398+0000 7ff3b8e00700 1 --2- 172.21.15.86:0/2598237971 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7ff3980a12f0 0x7ff3980a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:29.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.399+0000 7ff3b1600700 1 -- 172.21.15.86:0/2598237971 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ff3a0095c80 con 0x7ff3b411a590 2024-03-23T17:49:29.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.403+0000 7ff3b8e00700 1 --2- 172.21.15.86:0/2598237971 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7ff3980a12f0 0x7ff3980a37b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff3a400b5c0 tx=0x7ff3a4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:29.584 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T17:49:29.586 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T17:49:29.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.672+0000 7ff3bb800700 1 -- 172.21.15.86:0/2598237971 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7ff394005160 con 0x7ff3b411a590 2024-03-23T17:49:29.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.673+0000 7ff3b1600700 1 -- 172.21.15.86:0/2598237971 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff3a00992a0 con 0x7ff3b411a590 2024-03-23T17:49:29.673 INFO:teuthology.orchestra.run.smithi086.stdout:141733920812 2024-03-23T17:49:29.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.675+0000 7ff3bb800700 1 -- 172.21.15.86:0/2598237971 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7ff3980a12f0 msgr2=0x7ff3980a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:29.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.675+0000 7ff3bb800700 1 --2- 172.21.15.86:0/2598237971 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7ff3980a12f0 0x7ff3980a37b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff3a400b5c0 tx=0x7ff3a4005fb0 comp rx=0 tx=0).stop 2024-03-23T17:49:29.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.675+0000 7ff3bb800700 1 -- 172.21.15.86:0/2598237971 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff3b411a590 msgr2=0x7ff3b41aa970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:29.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.675+0000 7ff3bb800700 1 --2- 172.21.15.86:0/2598237971 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff3b411a590 0x7ff3b41aa970 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff3a000c8c0 tx=0x7ff3a000cc80 comp rx=0 tx=0).stop 2024-03-23T17:49:29.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.675+0000 7ff3bb800700 1 -- 172.21.15.86:0/2598237971 shutdown_connections 2024-03-23T17:49:29.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.675+0000 7ff3bb800700 1 --2- 172.21.15.86:0/2598237971 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff3b411a590 0x7ff3b41aa970 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:29.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.675+0000 7ff3bb800700 1 --2- 172.21.15.86:0/2598237971 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3b41082f0 0x7ff3b41a6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:29.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.675+0000 7ff3bb800700 1 --2- 172.21.15.86:0/2598237971 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7ff3980a12f0 0x7ff3980a37b0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:29.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.675+0000 7ff3bb800700 1 --2- 172.21.15.86:0/2598237971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3b410f550 0x7ff3b41a65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:29.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.675+0000 7ff3bb800700 1 -- 172.21.15.86:0/2598237971 >> 172.21.15.86:0/2598237971 conn(0x7ff3b406be80 msgr2=0x7ff3b41187c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:29.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.675+0000 7ff3bb800700 1 -- 172.21.15.86:0/2598237971 shutdown_connections 2024-03-23T17:49:29.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.675+0000 7ff3bb800700 1 -- 172.21.15.86:0/2598237971 wait complete. 2024-03-23T17:49:29.785 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:29 smithi086 ceph-mon[38221]: pgmap v250: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-23T17:49:29.785 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:29 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-03-23T17:49:29.786 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:29 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-03-23T17:49:29.786 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:29 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-03-23T17:49:29.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.879+0000 7fcb1e000700 1 -- 172.21.15.86:0/2323900793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb18069ff0 msgr2=0x7fcb181059e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:29.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.879+0000 7fcb1e000700 1 --2- 172.21.15.86:0/2323900793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb18069ff0 0x7fcb181059e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fcb04009990 tx=0x7fcb04009ca0 comp rx=0 tx=0).stop 2024-03-23T17:49:29.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.880+0000 7fcb1e000700 1 -- 172.21.15.86:0/2323900793 shutdown_connections 2024-03-23T17:49:29.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.880+0000 7fcb1e000700 1 --2- 172.21.15.86:0/2323900793 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb181060c0 0x7fcb1810dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:29.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.880+0000 7fcb1e000700 1 --2- 172.21.15.86:0/2323900793 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcb18069640 0x7fcb18069a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:29.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.880+0000 7fcb1e000700 1 --2- 172.21.15.86:0/2323900793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb18069ff0 0x7fcb181059e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:29.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.880+0000 7fcb1e000700 1 -- 172.21.15.86:0/2323900793 >> 172.21.15.86:0/2323900793 conn(0x7fcb18075d90 msgr2=0x7fcb180761a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:29.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.880+0000 7fcb1e000700 1 -- 172.21.15.86:0/2323900793 shutdown_connections 2024-03-23T17:49:29.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.880+0000 7fcb1e000700 1 -- 172.21.15.86:0/2323900793 wait complete. 2024-03-23T17:49:29.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.881+0000 7fcb1e000700 1 Processor -- start 2024-03-23T17:49:29.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.881+0000 7fcb1e000700 1 -- start start 2024-03-23T17:49:29.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.881+0000 7fcb1e000700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcb18069640 0x7fcb18199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:29.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.881+0000 7fcb1e000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb18069ff0 0x7fcb181998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:29.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.881+0000 7fcb1e000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb181060c0 0x7fcb1819dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:29.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.881+0000 7fcb1e000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb1819e280 con 0x7fcb181060c0 2024-03-23T17:49:29.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.881+0000 7fcb1e000700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb1819e3f0 con 0x7fcb18069ff0 2024-03-23T17:49:29.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.881+0000 7fcb1e000700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb1819e5b0 con 0x7fcb18069640 2024-03-23T17:49:29.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.881+0000 7fcb17400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcb18069640 0x7fcb18199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:29.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.881+0000 7fcb17400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcb18069640 0x7fcb18199380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:56824/0 (socket says 172.21.15.86:56824) 2024-03-23T17:49:29.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.881+0000 7fcb17400700 1 -- 172.21.15.86:0/1668999935 learned_addr learned my addr 172.21.15.86:0/1668999935 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:29.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.881+0000 7fcb17e00700 1 --2- 172.21.15.86:0/1668999935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb181060c0 0x7fcb1819dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:29.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.882+0000 7fcb16a00700 1 --2- 172.21.15.86:0/1668999935 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb18069ff0 0x7fcb181998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:29.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.882+0000 7fcb17400700 1 -- 172.21.15.86:0/1668999935 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb18069ff0 msgr2=0x7fcb181998c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:29.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.882+0000 7fcb17400700 1 --2- 172.21.15.86:0/1668999935 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb18069ff0 0x7fcb181998c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:29.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.882+0000 7fcb17400700 1 -- 172.21.15.86:0/1668999935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb181060c0 msgr2=0x7fcb1819dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:29.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.882+0000 7fcb17400700 1 --2- 172.21.15.86:0/1668999935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb181060c0 0x7fcb1819dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:29.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.882+0000 7fcb17400700 1 -- 172.21.15.86:0/1668999935 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb040095f0 con 0x7fcb18069640 2024-03-23T17:49:29.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.882+0000 7fcb17400700 1 --2- 172.21.15.86:0/1668999935 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcb18069640 0x7fcb18199380 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcb0800e9a0 tx=0x7fcb0800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:29.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.882+0000 7fcb0fe00700 1 -- 172.21.15.86:0/1668999935 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb08009790 con 0x7fcb18069640 2024-03-23T17:49:29.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.882+0000 7fcb0fe00700 1 -- 172.21.15.86:0/1668999935 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb08010430 con 0x7fcb18069640 2024-03-23T17:49:29.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.883+0000 7fcb1e000700 1 -- 172.21.15.86:0/1668999935 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb1819e890 con 0x7fcb18069640 2024-03-23T17:49:29.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.883+0000 7fcb0fe00700 1 -- 172.21.15.86:0/1668999935 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb08018680 con 0x7fcb18069640 2024-03-23T17:49:29.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.883+0000 7fcb1e000700 1 -- 172.21.15.86:0/1668999935 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb181a60f0 con 0x7fcb18069640 2024-03-23T17:49:29.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.884+0000 7fcb1e000700 1 -- 172.21.15.86:0/1668999935 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb181024c0 con 0x7fcb18069640 2024-03-23T17:49:29.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.886+0000 7fcb0fe00700 1 -- 172.21.15.86:0/1668999935 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fcb080105a0 con 0x7fcb18069640 2024-03-23T17:49:29.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.888+0000 7fcb0fe00700 1 --2- 172.21.15.86:0/1668999935 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fcafc0a11b0 0x7fcafc0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:29.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.888+0000 7fcb0fe00700 1 -- 172.21.15.86:0/1668999935 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fcb08014070 con 0x7fcb18069640 2024-03-23T17:49:29.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.888+0000 7fcb16a00700 1 --2- 172.21.15.86:0/1668999935 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fcafc0a11b0 0x7fcafc0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:29.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.888+0000 7fcb16a00700 1 --2- 172.21.15.86:0/1668999935 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fcafc0a11b0 0x7fcafc0a3670 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fcb04006010 tx=0x7fcb04005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:29.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:29.889+0000 7fcb0fe00700 1 -- 172.21.15.86:0/1668999935 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fcb08096220 con 0x7fcb18069640 2024-03-23T17:49:29.975 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T17:49:29.976 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T17:49:30.055 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:29 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]': finished 2024-03-23T17:49:30.055 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:29 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-03-23T17:49:30.055 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:29 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-03-23T17:49:30.055 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:29 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-03-23T17:49:30.055 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:29 smithi086 ceph-mon[38221]: osdmap e81: 12 total, 12 up, 12 in 2024-03-23T17:49:30.055 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:29 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/2598237971' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-23T17:49:30.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:29 smithi092 ceph-mon[79089]: pgmap v250: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-23T17:49:30.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:29 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-03-23T17:49:30.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:29 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-03-23T17:49:30.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:29 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-03-23T17:49:30.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:29 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]': finished 2024-03-23T17:49:30.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:29 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-03-23T17:49:30.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:29 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-03-23T17:49:30.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:29 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-03-23T17:49:30.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:29 smithi092 ceph-mon[79089]: osdmap e81: 12 total, 12 up, 12 in 2024-03-23T17:49:30.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:29 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/2598237971' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-23T17:49:30.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:29 smithi152 ceph-mon[47949]: pgmap v250: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-23T17:49:30.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:29 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-03-23T17:49:30.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:29 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-03-23T17:49:30.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:29 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-03-23T17:49:30.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:29 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]': finished 2024-03-23T17:49:30.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:29 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-03-23T17:49:30.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:29 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-03-23T17:49:30.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:29 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-03-23T17:49:30.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:29 smithi152 ceph-mon[47949]: osdmap e81: 12 total, 12 up, 12 in 2024-03-23T17:49:30.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:29 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/2598237971' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-23T17:49:30.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:30.129+0000 7fcb1e000700 1 -- 172.21.15.86:0/1668999935 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fcb1819a730 con 0x7fcb18069640 2024-03-23T17:49:30.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:30.129+0000 7fcb0fe00700 1 -- 172.21.15.86:0/1668999935 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcb08099840 con 0x7fcb18069640 2024-03-23T17:49:30.130 INFO:teuthology.orchestra.run.smithi086.stdout:210453397536 2024-03-23T17:49:30.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:30.132+0000 7fcb1e000700 1 -- 172.21.15.86:0/1668999935 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fcafc0a11b0 msgr2=0x7fcafc0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:30.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:30.132+0000 7fcb1e000700 1 --2- 172.21.15.86:0/1668999935 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fcafc0a11b0 0x7fcafc0a3670 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fcb04006010 tx=0x7fcb04005960 comp rx=0 tx=0).stop 2024-03-23T17:49:30.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:30.132+0000 7fcb1e000700 1 -- 172.21.15.86:0/1668999935 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcb18069640 msgr2=0x7fcb18199380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:30.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:30.132+0000 7fcb1e000700 1 --2- 172.21.15.86:0/1668999935 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcb18069640 0x7fcb18199380 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcb0800e9a0 tx=0x7fcb0800ecb0 comp rx=0 tx=0).stop 2024-03-23T17:49:30.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:30.132+0000 7fcb1e000700 1 -- 172.21.15.86:0/1668999935 shutdown_connections 2024-03-23T17:49:30.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:30.132+0000 7fcb1e000700 1 --2- 172.21.15.86:0/1668999935 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fcafc0a11b0 0x7fcafc0a3670 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:30.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:30.132+0000 7fcb1e000700 1 --2- 172.21.15.86:0/1668999935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb181060c0 0x7fcb1819dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:30.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:30.132+0000 7fcb1e000700 1 --2- 172.21.15.86:0/1668999935 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcb18069640 0x7fcb18199380 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:30.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:30.132+0000 7fcb1e000700 1 --2- 172.21.15.86:0/1668999935 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb18069ff0 0x7fcb181998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:30.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:30.132+0000 7fcb1e000700 1 -- 172.21.15.86:0/1668999935 >> 172.21.15.86:0/1668999935 conn(0x7fcb18075d90 msgr2=0x7fcb1810bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:30.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:30.132+0000 7fcb1e000700 1 -- 172.21.15.86:0/1668999935 shutdown_connections 2024-03-23T17:49:30.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:30.132+0000 7fcb1e000700 1 -- 172.21.15.86:0/1668999935 wait complete. 2024-03-23T17:49:30.619 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920811 got 141733920812 for osd.4 2024-03-23T17:49:30.619 DEBUG:teuthology.parallel:result is None 2024-03-23T17:49:30.877 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:30 smithi086 ceph-mon[38221]: osdmap e82: 12 total, 12 up, 12 in 2024-03-23T17:49:30.877 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:30 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/1668999935' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-23T17:49:31.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:30 smithi092 ceph-mon[79089]: osdmap e82: 12 total, 12 up, 12 in 2024-03-23T17:49:31.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:30 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/1668999935' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-23T17:49:31.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:30 smithi152 ceph-mon[47949]: osdmap e82: 12 total, 12 up, 12 in 2024-03-23T17:49:31.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:30 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/1668999935' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-23T17:49:31.844 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:31 smithi086 ceph-mon[38221]: pgmap v253: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-03-23T17:49:32.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:31 smithi092 ceph-mon[79089]: pgmap v253: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-03-23T17:49:32.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:31 smithi152 ceph-mon[47949]: pgmap v253: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-03-23T17:49:32.958 INFO:tasks.cephadm.ceph_manager.ceph:need seq 210453397534 got 210453397536 for osd.7 2024-03-23T17:49:32.959 DEBUG:teuthology.parallel:result is None 2024-03-23T17:49:33.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:33.122+0000 7fae54e00700 1 -- 172.21.15.86:0/2155213661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae50107db0 msgr2=0x7fae50108190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:33.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:33.122+0000 7fae54e00700 1 --2- 172.21.15.86:0/2155213661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae50107db0 0x7fae50108190 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fae400099c0 tx=0x7fae40009cd0 comp rx=0 tx=0).stop 2024-03-23T17:49:33.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:33.122+0000 7fae54e00700 1 -- 172.21.15.86:0/2155213661 shutdown_connections 2024-03-23T17:49:33.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:33.122+0000 7fae54e00700 1 --2- 172.21.15.86:0/2155213661 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae50113f00 0x7fae501162f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:33.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:33.122+0000 7fae54e00700 1 --2- 172.21.15.86:0/2155213661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae50107db0 0x7fae50108190 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:33.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:33.122+0000 7fae54e00700 1 --2- 172.21.15.86:0/2155213661 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae501086d0 0x7fae501139c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:33.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:33.122+0000 7fae54e00700 1 -- 172.21.15.86:0/2155213661 >> 172.21.15.86:0/2155213661 conn(0x7fae5006be80 msgr2=0x7fae5006c290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:33.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:33.122+0000 7fae54e00700 1 -- 172.21.15.86:0/2155213661 shutdown_connections 2024-03-23T17:49:33.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:33.122+0000 7fae54e00700 1 -- 172.21.15.86:0/2155213661 wait complete. 2024-03-23T17:49:33.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:33.123+0000 7fae54e00700 1 Processor -- start 2024-03-23T17:49:33.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:33.124+0000 7fae54e00700 1 -- start start 2024-03-23T17:49:33.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:33.124+0000 7fae54e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae50107db0 0x7fae501a7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:33.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:33.124+0000 7fae54e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae501086d0 0x7fae501a7e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:33.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:33.124+0000 7fae54e00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae50113f00 0x7fae501a1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:33.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:33.124+0000 7fae54e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae501a2100 con 0x7fae501086d0 2024-03-23T17:49:33.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:33.124+0000 7fae54e00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae501a2270 con 0x7fae50107db0 2024-03-23T17:49:33.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:33.124+0000 7fae54e00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae501a2430 con 0x7fae50113f00 2024-03-23T17:49:33.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:33.125+0000 7fae4de00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae50107db0 0x7fae501a7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:33.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:33.125+0000 7fae4d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae501086d0 0x7fae501a7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:33.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:33.125+0000 7fae4e800700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae50113f00 0x7fae501a1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:33.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:33.125+0000 7fae4d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae501086d0 0x7fae501a7e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58866/0 (socket says 172.21.15.86:58866) 2024-03-23T17:49:33.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:33.125+0000 7fae4d400700 1 -- 172.21.15.86:0/398165345 learned_addr learned my addr 172.21.15.86:0/398165345 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:33.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:33.125+0000 7fae4de00700 1 -- 172.21.15.86:0/398165345 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae50113f00 msgr2=0x7fae501a1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:33.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:33.125+0000 7fae4de00700 1 --2- 172.21.15.86:0/398165345 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae50113f00 0x7fae501a1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:33.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:33.125+0000 7fae4de00700 1 -- 172.21.15.86:0/398165345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae501086d0 msgr2=0x7fae501a7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:33.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:33.125+0000 7fae4de00700 1 --2- 172.21.15.86:0/398165345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae501086d0 0x7fae501a7e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:33.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:33.125+0000 7fae4de00700 1 -- 172.21.15.86:0/398165345 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae400095f0 con 0x7fae50107db0 2024-03-23T17:49:33.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:33.125+0000 7fae4d400700 1 --2- 172.21.15.86:0/398165345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae501086d0 0x7fae501a7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:49:33.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:33.125+0000 7fae4de00700 1 --2- 172.21.15.86:0/398165345 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae50107db0 0x7fae501a7910 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fae400098e0 tx=0x7fae4000b9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:33.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:33.126+0000 7fae46a00700 1 -- 172.21.15.86:0/398165345 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae4001d070 con 0x7fae50107db0 2024-03-23T17:49:33.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:33.126+0000 7fae46a00700 1 -- 172.21.15.86:0/398165345 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae4000be00 con 0x7fae50107db0 2024-03-23T17:49:33.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:33.126+0000 7fae54e00700 1 -- 172.21.15.86:0/398165345 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae501a26b0 con 0x7fae50107db0 2024-03-23T17:49:33.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:33.126+0000 7fae46a00700 1 -- 172.21.15.86:0/398165345 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae4000f9a0 con 0x7fae50107db0 2024-03-23T17:49:33.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:33.126+0000 7fae54e00700 1 -- 172.21.15.86:0/398165345 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae501ae900 con 0x7fae50107db0 2024-03-23T17:49:33.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:33.127+0000 7fae54e00700 1 -- 172.21.15.86:0/398165345 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae5006d450 con 0x7fae50107db0 2024-03-23T17:49:33.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:33.128+0000 7fae46a00700 1 -- 172.21.15.86:0/398165345 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fae40004c60 con 0x7fae50107db0 2024-03-23T17:49:33.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:33.130+0000 7fae46a00700 1 --2- 172.21.15.86:0/398165345 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fae340a11b0 0x7fae340a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:33.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:33.130+0000 7fae46a00700 1 -- 172.21.15.86:0/398165345 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fae400cd5d0 con 0x7fae50107db0 2024-03-23T17:49:33.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:33.130+0000 7fae4d400700 1 --2- 172.21.15.86:0/398165345 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fae340a11b0 0x7fae340a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:33.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:33.130+0000 7fae4d400700 1 --2- 172.21.15.86:0/398165345 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fae340a11b0 0x7fae340a3670 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fae501a2c10 tx=0x7fae3c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:33.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:33.134+0000 7fae46a00700 1 -- 172.21.15.86:0/398165345 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fae40096cc0 con 0x7fae50107db0 2024-03-23T17:49:33.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:33.361+0000 7fae54e00700 1 -- 172.21.15.86:0/398165345 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fae50119520 con 0x7fae50107db0 2024-03-23T17:49:33.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:33.362+0000 7fae46a00700 1 -- 172.21.15.86:0/398165345 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fae4009a2e0 con 0x7fae50107db0 2024-03-23T17:49:33.362 INFO:teuthology.orchestra.run.smithi086.stdout:34359738428 2024-03-23T17:49:33.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:33.365+0000 7fae54e00700 1 -- 172.21.15.86:0/398165345 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fae340a11b0 msgr2=0x7fae340a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:33.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:33.365+0000 7fae54e00700 1 --2- 172.21.15.86:0/398165345 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fae340a11b0 0x7fae340a3670 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fae501a2c10 tx=0x7fae3c009260 comp rx=0 tx=0).stop 2024-03-23T17:49:33.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:33.365+0000 7fae54e00700 1 -- 172.21.15.86:0/398165345 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae50107db0 msgr2=0x7fae501a7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:33.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:33.365+0000 7fae54e00700 1 --2- 172.21.15.86:0/398165345 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae50107db0 0x7fae501a7910 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fae400098e0 tx=0x7fae4000b9c0 comp rx=0 tx=0).stop 2024-03-23T17:49:33.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:33.365+0000 7fae54e00700 1 -- 172.21.15.86:0/398165345 shutdown_connections 2024-03-23T17:49:33.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:33.365+0000 7fae54e00700 1 --2- 172.21.15.86:0/398165345 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae50113f00 0x7fae501a1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:33.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:33.365+0000 7fae54e00700 1 --2- 172.21.15.86:0/398165345 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae50107db0 0x7fae501a7910 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:33.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:33.365+0000 7fae54e00700 1 --2- 172.21.15.86:0/398165345 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fae340a11b0 0x7fae340a3670 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:33.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:33.365+0000 7fae54e00700 1 --2- 172.21.15.86:0/398165345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae501086d0 0x7fae501a7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:33.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:33.365+0000 7fae54e00700 1 -- 172.21.15.86:0/398165345 >> 172.21.15.86:0/398165345 conn(0x7fae5006be80 msgr2=0x7fae501097f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:33.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:33.365+0000 7fae54e00700 1 -- 172.21.15.86:0/398165345 shutdown_connections 2024-03-23T17:49:33.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:33.365+0000 7fae54e00700 1 -- 172.21.15.86:0/398165345 wait complete. 2024-03-23T17:49:33.991 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T17:49:33.995 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:33 smithi086 ceph-mon[38221]: pgmap v254: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-23T17:49:33.995 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:33 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/398165345' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-23T17:49:34.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:33 smithi092 ceph-mon[79089]: pgmap v254: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-23T17:49:34.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:33 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/398165345' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-23T17:49:34.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:33 smithi152 ceph-mon[47949]: pgmap v254: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-23T17:49:34.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:33 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/398165345' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-23T17:49:34.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.574+0000 7fb853a00700 1 -- 172.21.15.86:0/2589376399 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb84c0ff890 msgr2=0x7fb84c10dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:34.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.574+0000 7fb853a00700 1 --2- 172.21.15.86:0/2589376399 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb84c0ff890 0x7fb84c10dc40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb844009f90 tx=0x7fb84400b330 comp rx=0 tx=0).stop 2024-03-23T17:49:34.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.574+0000 7fb853a00700 1 -- 172.21.15.86:0/2589376399 shutdown_connections 2024-03-23T17:49:34.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.574+0000 7fb853a00700 1 --2- 172.21.15.86:0/2589376399 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb84c0ff890 0x7fb84c10dc40 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:34.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.574+0000 7fb853a00700 1 --2- 172.21.15.86:0/2589376399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb84c106ef0 0x7fb84c1072d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:34.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.574+0000 7fb853a00700 1 --2- 172.21.15.86:0/2589376399 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb84c0feed0 0x7fb84c0ff350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:34.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.574+0000 7fb853a00700 1 -- 172.21.15.86:0/2589376399 >> 172.21.15.86:0/2589376399 conn(0x7fb84c075e80 msgr2=0x7fb84c0fd200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:34.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.574+0000 7fb853a00700 1 -- 172.21.15.86:0/2589376399 shutdown_connections 2024-03-23T17:49:34.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.574+0000 7fb853a00700 1 -- 172.21.15.86:0/2589376399 wait complete. 2024-03-23T17:49:34.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.576+0000 7fb853a00700 1 Processor -- start 2024-03-23T17:49:34.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.576+0000 7fb853a00700 1 -- start start 2024-03-23T17:49:34.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.576+0000 7fb853a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb84c0feed0 0x7fb84c199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:34.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.576+0000 7fb853a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb84c0ff890 0x7fb84c199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:34.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.576+0000 7fb853a00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb84c106ef0 0x7fb84c19dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:34.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.576+0000 7fb853a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb84c19e330 con 0x7fb84c0ff890 2024-03-23T17:49:34.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.576+0000 7fb853a00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb84c19e4a0 con 0x7fb84c0feed0 2024-03-23T17:49:34.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.576+0000 7fb853a00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb84c19e660 con 0x7fb84c106ef0 2024-03-23T17:49:34.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.577+0000 7fb84be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb84c0ff890 0x7fb84c199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:34.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.577+0000 7fb851000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb84c0feed0 0x7fb84c199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:34.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.577+0000 7fb851000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb84c0feed0 0x7fb84c199430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:44236/0 (socket says 172.21.15.86:44236) 2024-03-23T17:49:34.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.577+0000 7fb851000700 1 -- 172.21.15.86:0/2637515001 learned_addr learned my addr 172.21.15.86:0/2637515001 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:34.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.577+0000 7fb851a00700 1 --2- 172.21.15.86:0/2637515001 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb84c106ef0 0x7fb84c19dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:34.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.577+0000 7fb851000700 1 -- 172.21.15.86:0/2637515001 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb84c106ef0 msgr2=0x7fb84c19dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:34.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.577+0000 7fb851000700 1 --2- 172.21.15.86:0/2637515001 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb84c106ef0 0x7fb84c19dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:34.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.577+0000 7fb851000700 1 -- 172.21.15.86:0/2637515001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb84c0ff890 msgr2=0x7fb84c199970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:34.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.577+0000 7fb851000700 1 --2- 172.21.15.86:0/2637515001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb84c0ff890 0x7fb84c199970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:34.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.577+0000 7fb851000700 1 -- 172.21.15.86:0/2637515001 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb844009c40 con 0x7fb84c0feed0 2024-03-23T17:49:34.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.578+0000 7fb851000700 1 --2- 172.21.15.86:0/2637515001 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb84c0feed0 0x7fb84c199430 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb84000d980 tx=0x7fb84000dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:34.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.578+0000 7fb849600700 1 -- 172.21.15.86:0/2637515001 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb84000f890 con 0x7fb84c0feed0 2024-03-23T17:49:34.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.578+0000 7fb853a00700 1 -- 172.21.15.86:0/2637515001 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb84c19e940 con 0x7fb84c0feed0 2024-03-23T17:49:34.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.578+0000 7fb849600700 1 -- 172.21.15.86:0/2637515001 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb840010460 con 0x7fb84c0feed0 2024-03-23T17:49:34.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.578+0000 7fb849600700 1 -- 172.21.15.86:0/2637515001 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb840009600 con 0x7fb84c0feed0 2024-03-23T17:49:34.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.578+0000 7fb853a00700 1 -- 172.21.15.86:0/2637515001 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb84c1a61a0 con 0x7fb84c0feed0 2024-03-23T17:49:34.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.579+0000 7fb853a00700 1 -- 172.21.15.86:0/2637515001 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb84c1002a0 con 0x7fb84c0feed0 2024-03-23T17:49:34.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.581+0000 7fb849600700 1 -- 172.21.15.86:0/2637515001 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fb8400105d0 con 0x7fb84c0feed0 2024-03-23T17:49:34.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.582+0000 7fb849600700 1 --2- 172.21.15.86:0/2637515001 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fb8340a1230 0x7fb8340a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:34.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.583+0000 7fb849600700 1 -- 172.21.15.86:0/2637515001 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fb8400cc750 con 0x7fb84c0feed0 2024-03-23T17:49:34.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.583+0000 7fb84be00700 1 --2- 172.21.15.86:0/2637515001 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fb8340a1230 0x7fb8340a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:34.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.583+0000 7fb84be00700 1 --2- 172.21.15.86:0/2637515001 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fb8340a1230 0x7fb8340a36f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb84c19a990 tx=0x7fb838009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:34.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.586+0000 7fb849600700 1 -- 172.21.15.86:0/2637515001 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fb840095d50 con 0x7fb84c0feed0 2024-03-23T17:49:34.834 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:34 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/3376701695' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-23T17:49:34.863 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738425 got 34359738428 for osd.0 2024-03-23T17:49:34.863 DEBUG:teuthology.parallel:result is None 2024-03-23T17:49:34.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.854+0000 7fb4d4a00700 1 -- 172.21.15.86:0/1561189195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4d0105cb0 msgr2=0x7fb4d0068750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:34.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.854+0000 7fb4d4a00700 1 --2- 172.21.15.86:0/1561189195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4d0105cb0 0x7fb4d0068750 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0009990 tx=0x7fb4c0009ca0 comp rx=0 tx=0).stop 2024-03-23T17:49:34.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.855+0000 7fb4d4a00700 1 -- 172.21.15.86:0/1561189195 shutdown_connections 2024-03-23T17:49:34.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.855+0000 7fb4d4a00700 1 --2- 172.21.15.86:0/1561189195 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4d0103810 0x7fb4d0103cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:34.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.855+0000 7fb4d4a00700 1 --2- 172.21.15.86:0/1561189195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4d0105cb0 0x7fb4d0068750 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:34.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.855+0000 7fb4d4a00700 1 --2- 172.21.15.86:0/1561189195 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4d0068c90 0x7fb4d0069110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:34.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.855+0000 7fb4d4a00700 1 -- 172.21.15.86:0/1561189195 >> 172.21.15.86:0/1561189195 conn(0x7fb4d0075d50 msgr2=0x7fb4d0076160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:34.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.855+0000 7fb4d4a00700 1 -- 172.21.15.86:0/1561189195 shutdown_connections 2024-03-23T17:49:34.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.855+0000 7fb4d4a00700 1 -- 172.21.15.86:0/1561189195 wait complete. 2024-03-23T17:49:34.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.859+0000 7fb4d4a00700 1 Processor -- start 2024-03-23T17:49:34.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.860+0000 7fb4d4a00700 1 -- start start 2024-03-23T17:49:34.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.861+0000 7fb4d4a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4d0068c90 0x7fb4d019d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:34.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.862+0000 7fb4d4a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4d0103810 0x7fb4d019dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:34.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.862+0000 7fb4cde00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4d0068c90 0x7fb4d019d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:34.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.862+0000 7fb4d4a00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4d0105cb0 0x7fb4d01a0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:34.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.862+0000 7fb4d4a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d01a15e0 con 0x7fb4d0103810 2024-03-23T17:49:34.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.862+0000 7fb4d4a00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d01a1750 con 0x7fb4d0068c90 2024-03-23T17:49:34.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.862+0000 7fb4d4a00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d01a1910 con 0x7fb4d0105cb0 2024-03-23T17:49:34.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.862+0000 7fb4cd400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4d0103810 0x7fb4d019dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:34.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.863+0000 7fb4ce800700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4d0105cb0 0x7fb4d01a0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:34.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.863+0000 7fb4cd400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4d0103810 0x7fb4d019dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40342/0 (socket says 172.21.15.86:40342) 2024-03-23T17:49:34.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.863+0000 7fb4cd400700 1 -- 172.21.15.86:0/347350383 learned_addr learned my addr 172.21.15.86:0/347350383 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:34.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.862+0000 7fb4cde00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4d0068c90 0x7fb4d019d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:50380/0 (socket says 172.21.15.86:50380) 2024-03-23T17:49:34.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.863+0000 7fb4cd400700 1 -- 172.21.15.86:0/347350383 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4d0105cb0 msgr2=0x7fb4d01a0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:34.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.863+0000 7fb4cd400700 1 --2- 172.21.15.86:0/347350383 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4d0105cb0 0x7fb4d01a0fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:34.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.863+0000 7fb4cd400700 1 -- 172.21.15.86:0/347350383 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4d0068c90 msgr2=0x7fb4d019d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:34.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.863+0000 7fb4cd400700 1 --2- 172.21.15.86:0/347350383 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4d0068c90 0x7fb4d019d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:34.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.863+0000 7fb4cd400700 1 -- 172.21.15.86:0/347350383 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4c00095f0 con 0x7fb4d0103810 2024-03-23T17:49:34.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.864+0000 7fb4cd400700 1 --2- 172.21.15.86:0/347350383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4d0103810 0x7fb4d019dce0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc00cce0 tx=0x7fb4bc0074a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:34.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.865+0000 7fb4c6a00700 1 -- 172.21.15.86:0/347350383 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc007af0 con 0x7fb4d0103810 2024-03-23T17:49:34.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.865+0000 7fb4d4a00700 1 -- 172.21.15.86:0/347350383 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4d01a1bf0 con 0x7fb4d0103810 2024-03-23T17:49:34.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.865+0000 7fb4d4a00700 1 -- 172.21.15.86:0/347350383 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4d01aa540 con 0x7fb4d0103810 2024-03-23T17:49:34.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.866+0000 7fb4c6a00700 1 -- 172.21.15.86:0/347350383 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb4bc004cf0 con 0x7fb4d0103810 2024-03-23T17:49:34.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.866+0000 7fb4c6a00700 1 -- 172.21.15.86:0/347350383 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc007af0 con 0x7fb4d0103810 2024-03-23T17:49:34.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.867+0000 7fb4d4a00700 1 -- 172.21.15.86:0/347350383 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4d01002f0 con 0x7fb4d0103810 2024-03-23T17:49:34.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.871+0000 7fb4c6a00700 1 -- 172.21.15.86:0/347350383 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fb4bc0053c0 con 0x7fb4d0103810 2024-03-23T17:49:34.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.874+0000 7fb4c6a00700 1 --2- 172.21.15.86:0/347350383 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fb4b40a1290 0x7fb4b40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:34.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.874+0000 7fb4c6a00700 1 -- 172.21.15.86:0/347350383 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fb4bc0cc270 con 0x7fb4d0103810 2024-03-23T17:49:34.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.875+0000 7fb4cde00700 1 --2- 172.21.15.86:0/347350383 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fb4b40a1290 0x7fb4b40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:34.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.877+0000 7fb4cde00700 1 --2- 172.21.15.86:0/347350383 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fb4b40a1290 0x7fb4b40a3750 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb4c000b2e0 tx=0x7fb4c000b610 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:34.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.878+0000 7fb4c6a00700 1 -- 172.21.15.86:0/347350383 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fb4bc099300 con 0x7fb4d0103810 2024-03-23T17:49:34.970 INFO:teuthology.orchestra.run.smithi086.stdout:292057776149 2024-03-23T17:49:34.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.969+0000 7fb853a00700 1 -- 172.21.15.86:0/2637515001 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7fb84c04ea80 con 0x7fb84c0feed0 2024-03-23T17:49:34.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.969+0000 7fb849600700 1 -- 172.21.15.86:0/2637515001 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7fb840099370 con 0x7fb84c0feed0 2024-03-23T17:49:34.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.972+0000 7fb853a00700 1 -- 172.21.15.86:0/2637515001 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fb8340a1230 msgr2=0x7fb8340a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:34.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.972+0000 7fb853a00700 1 --2- 172.21.15.86:0/2637515001 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fb8340a1230 0x7fb8340a36f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb84c19a990 tx=0x7fb838009260 comp rx=0 tx=0).stop 2024-03-23T17:49:34.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.972+0000 7fb853a00700 1 -- 172.21.15.86:0/2637515001 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb84c0feed0 msgr2=0x7fb84c199430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:34.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.972+0000 7fb853a00700 1 --2- 172.21.15.86:0/2637515001 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb84c0feed0 0x7fb84c199430 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb84000d980 tx=0x7fb84000dc90 comp rx=0 tx=0).stop 2024-03-23T17:49:34.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.972+0000 7fb853a00700 1 -- 172.21.15.86:0/2637515001 shutdown_connections 2024-03-23T17:49:34.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.972+0000 7fb853a00700 1 --2- 172.21.15.86:0/2637515001 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb84c106ef0 0x7fb84c19dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:34.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.972+0000 7fb853a00700 1 --2- 172.21.15.86:0/2637515001 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb84c0feed0 0x7fb84c199430 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:34.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.972+0000 7fb853a00700 1 --2- 172.21.15.86:0/2637515001 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fb8340a1230 0x7fb8340a36f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:34.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.972+0000 7fb853a00700 1 --2- 172.21.15.86:0/2637515001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb84c0ff890 0x7fb84c199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:34.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.972+0000 7fb853a00700 1 -- 172.21.15.86:0/2637515001 >> 172.21.15.86:0/2637515001 conn(0x7fb84c075e80 msgr2=0x7fb84c0fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:34.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.972+0000 7fb853a00700 1 -- 172.21.15.86:0/2637515001 shutdown_connections 2024-03-23T17:49:34.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.972+0000 7fb853a00700 1 -- 172.21.15.86:0/2637515001 wait complete. 2024-03-23T17:49:34.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.538+0000 7f031fe00700 1 -- 172.21.15.86:0/3048806660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0320100e10 msgr2=0x7f03201011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:34.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.538+0000 7f031fe00700 1 --2- 172.21.15.86:0/3048806660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0320100e10 0x7f03201011f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0310009990 tx=0x7f0310009ca0 comp rx=0 tx=0).stop 2024-03-23T17:49:34.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.539+0000 7f031fe00700 1 -- 172.21.15.86:0/3048806660 shutdown_connections 2024-03-23T17:49:34.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.539+0000 7f031fe00700 1 --2- 172.21.15.86:0/3048806660 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0320105ef0 0x7f032010dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:34.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.539+0000 7f031fe00700 1 --2- 172.21.15.86:0/3048806660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0320100e10 0x7f03201011f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:34.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.539+0000 7f031fe00700 1 --2- 172.21.15.86:0/3048806660 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03201017c0 0x7f0320105810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:34.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.539+0000 7f031fe00700 1 -- 172.21.15.86:0/3048806660 >> 172.21.15.86:0/3048806660 conn(0x7f03200fc6e0 msgr2=0x7f03200feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:34.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.539+0000 7f031fe00700 1 -- 172.21.15.86:0/3048806660 shutdown_connections 2024-03-23T17:49:34.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.539+0000 7f031fe00700 1 -- 172.21.15.86:0/3048806660 wait complete. 2024-03-23T17:49:34.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.540+0000 7f031fe00700 1 Processor -- start 2024-03-23T17:49:34.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.540+0000 7f031fe00700 1 -- start start 2024-03-23T17:49:34.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.540+0000 7f031fe00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0320100e10 0x7f0320199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:34.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.540+0000 7f031fe00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f03201017c0 0x7f0320199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:34.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.540+0000 7f031fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0320105ef0 0x7f032019dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:34.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.540+0000 7f031fe00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f032019e220 con 0x7f0320105ef0 2024-03-23T17:49:34.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.540+0000 7f031fe00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f032019e390 con 0x7f03201017c0 2024-03-23T17:49:34.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.540+0000 7f031fe00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f032019e550 con 0x7f0320100e10 2024-03-23T17:49:34.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.541+0000 7f031f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0320105ef0 0x7f032019dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:34.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.541+0000 7f031ea00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0320100e10 0x7f0320199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:34.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.541+0000 7f031f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0320105ef0 0x7f032019dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58884/0 (socket says 172.21.15.86:58884) 2024-03-23T17:49:34.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.541+0000 7f031f400700 1 -- 172.21.15.86:0/3376701695 learned_addr learned my addr 172.21.15.86:0/3376701695 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:34.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.541+0000 7f031e000700 1 --2- 172.21.15.86:0/3376701695 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f03201017c0 0x7f0320199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:34.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.541+0000 7f031f400700 1 -- 172.21.15.86:0/3376701695 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0320100e10 msgr2=0x7f0320199410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:34.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.541+0000 7f031f400700 1 --2- 172.21.15.86:0/3376701695 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0320100e10 0x7f0320199410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:34.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.541+0000 7f031f400700 1 -- 172.21.15.86:0/3376701695 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f03201017c0 msgr2=0x7f0320199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:34.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.541+0000 7f031f400700 1 --2- 172.21.15.86:0/3376701695 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f03201017c0 0x7f0320199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:34.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.541+0000 7f031f400700 1 -- 172.21.15.86:0/3376701695 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0318009c40 con 0x7f0320105ef0 2024-03-23T17:49:34.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.541+0000 7f031ea00700 1 --2- 172.21.15.86:0/3376701695 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0320100e10 0x7f0320199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:49:34.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.541+0000 7f031f400700 1 --2- 172.21.15.86:0/3376701695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0320105ef0 0x7f032019dc10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f031800c370 tx=0x7f031800c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:34.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.542+0000 7f0317400700 1 -- 172.21.15.86:0/3376701695 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0318017070 con 0x7f0320105ef0 2024-03-23T17:49:34.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.542+0000 7f0317400700 1 -- 172.21.15.86:0/3376701695 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f031800f040 con 0x7f0320105ef0 2024-03-23T17:49:34.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.542+0000 7f031fe00700 1 -- 172.21.15.86:0/3376701695 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03100095f0 con 0x7f0320105ef0 2024-03-23T17:49:34.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.542+0000 7f0317400700 1 -- 172.21.15.86:0/3376701695 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f031800e050 con 0x7f0320105ef0 2024-03-23T17:49:34.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.542+0000 7f031fe00700 1 -- 172.21.15.86:0/3376701695 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03201a5ec0 con 0x7f0320105ef0 2024-03-23T17:49:34.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.544+0000 7f031fe00700 1 -- 172.21.15.86:0/3376701695 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03201022f0 con 0x7f0320105ef0 2024-03-23T17:49:34.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.545+0000 7f0317400700 1 -- 172.21.15.86:0/3376701695 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f031800cb60 con 0x7f0320105ef0 2024-03-23T17:49:34.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.547+0000 7f0317400700 1 --2- 172.21.15.86:0/3376701695 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f03040a1230 0x7f03040a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:34.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.547+0000 7f0317400700 1 -- 172.21.15.86:0/3376701695 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f03180cc2a0 con 0x7f0320105ef0 2024-03-23T17:49:34.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.547+0000 7f031ea00700 1 --2- 172.21.15.86:0/3376701695 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f03040a1230 0x7f03040a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:34.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.548+0000 7f031ea00700 1 --2- 172.21.15.86:0/3376701695 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f03040a1230 0x7f03040a36f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0310005b20 tx=0x7f0310005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:34.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.551+0000 7f0317400700 1 -- 172.21.15.86:0/3376701695 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f03180958a0 con 0x7f0320105ef0 2024-03-23T17:49:34.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.789+0000 7f031fe00700 1 -- 172.21.15.86:0/3376701695 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7f032019a770 con 0x7f0320105ef0 2024-03-23T17:49:34.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.790+0000 7f0317400700 1 -- 172.21.15.86:0/3376701695 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0318095070 con 0x7f0320105ef0 2024-03-23T17:49:34.988 INFO:teuthology.orchestra.run.smithi086.stdout:270582939675 2024-03-23T17:49:34.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.793+0000 7f031fe00700 1 -- 172.21.15.86:0/3376701695 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f03040a1230 msgr2=0x7f03040a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:34.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.793+0000 7f031fe00700 1 --2- 172.21.15.86:0/3376701695 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f03040a1230 0x7f03040a36f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0310005b20 tx=0x7f0310005a90 comp rx=0 tx=0).stop 2024-03-23T17:49:34.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.793+0000 7f031fe00700 1 -- 172.21.15.86:0/3376701695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0320105ef0 msgr2=0x7f032019dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:34.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.793+0000 7f031fe00700 1 --2- 172.21.15.86:0/3376701695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0320105ef0 0x7f032019dc10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f031800c370 tx=0x7f031800c680 comp rx=0 tx=0).stop 2024-03-23T17:49:34.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.793+0000 7f031fe00700 1 -- 172.21.15.86:0/3376701695 shutdown_connections 2024-03-23T17:49:34.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.793+0000 7f031fe00700 1 --2- 172.21.15.86:0/3376701695 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f03040a1230 0x7f03040a36f0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:34.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.793+0000 7f031fe00700 1 --2- 172.21.15.86:0/3376701695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0320105ef0 0x7f032019dc10 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:34.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.793+0000 7f031fe00700 1 --2- 172.21.15.86:0/3376701695 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0320100e10 0x7f0320199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:34.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.793+0000 7f031fe00700 1 --2- 172.21.15.86:0/3376701695 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f03201017c0 0x7f0320199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:34.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.793+0000 7f031fe00700 1 -- 172.21.15.86:0/3376701695 >> 172.21.15.86:0/3376701695 conn(0x7f03200fc6e0 msgr2=0x7f03200fdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:34.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.793+0000 7f031fe00700 1 -- 172.21.15.86:0/3376701695 shutdown_connections 2024-03-23T17:49:34.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.794+0000 7f031fe00700 1 -- 172.21.15.86:0/3376701695 wait complete. 2024-03-23T17:49:35.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.018+0000 7f475a000700 1 -- 172.21.15.86:0/105044617 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4754073520 msgr2=0x7f47540739a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:35.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.018+0000 7f475a000700 1 --2- 172.21.15.86:0/105044617 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4754073520 0x7f47540739a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4740009990 tx=0x7f4740009ca0 comp rx=0 tx=0).stop 2024-03-23T17:49:35.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.019+0000 7f475a000700 1 -- 172.21.15.86:0/105044617 shutdown_connections 2024-03-23T17:49:35.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.019+0000 7f475a000700 1 --2- 172.21.15.86:0/105044617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4754111b30 0x7f4754113f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:35.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.019+0000 7f475a000700 1 --2- 172.21.15.86:0/105044617 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4754074d30 0x7f4754072fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:35.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.019+0000 7f475a000700 1 --2- 172.21.15.86:0/105044617 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4754073520 0x7f47540739a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:35.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.019+0000 7f475a000700 1 -- 172.21.15.86:0/105044617 >> 172.21.15.86:0/105044617 conn(0x7f4754077fa0 msgr2=0x7f47540783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:35.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.019+0000 7f475a000700 1 -- 172.21.15.86:0/105044617 shutdown_connections 2024-03-23T17:49:35.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.019+0000 7f475a000700 1 -- 172.21.15.86:0/105044617 wait complete. 2024-03-23T17:49:35.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.020+0000 7f475a000700 1 Processor -- start 2024-03-23T17:49:35.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.020+0000 7f475a000700 1 -- start start 2024-03-23T17:49:35.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.020+0000 7f475a000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4754073520 0x7f47541a35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:35.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.020+0000 7f475a000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4754074d30 0x7f47541a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:35.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.020+0000 7f475a000700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4754111b30 0x7f475419d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:35.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.020+0000 7f475a000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f475419dd90 con 0x7f4754073520 2024-03-23T17:49:35.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.020+0000 7f475a000700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f475419df00 con 0x7f4754074d30 2024-03-23T17:49:35.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.020+0000 7f475a000700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f475419e0c0 con 0x7f4754111b30 2024-03-23T17:49:35.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.021+0000 7f4753400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4754073520 0x7f47541a35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:35.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.021+0000 7f4752a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4754074d30 0x7f47541a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:35.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.021+0000 7f4752a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4754074d30 0x7f47541a3b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:50424/0 (socket says 172.21.15.86:50424) 2024-03-23T17:49:35.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.021+0000 7f4752a00700 1 -- 172.21.15.86:0/203290162 learned_addr learned my addr 172.21.15.86:0/203290162 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:35.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.021+0000 7f4753e00700 1 --2- 172.21.15.86:0/203290162 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4754111b30 0x7f475419d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:35.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.021+0000 7f4752a00700 1 -- 172.21.15.86:0/203290162 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4754111b30 msgr2=0x7f475419d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:35.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.021+0000 7f4752a00700 1 --2- 172.21.15.86:0/203290162 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4754111b30 0x7f475419d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:35.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.021+0000 7f4752a00700 1 -- 172.21.15.86:0/203290162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4754073520 msgr2=0x7f47541a35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:35.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.021+0000 7f4752a00700 1 --2- 172.21.15.86:0/203290162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4754073520 0x7f47541a35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:35.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.021+0000 7f4752a00700 1 -- 172.21.15.86:0/203290162 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4744009520 con 0x7f4754074d30 2024-03-23T17:49:35.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.021+0000 7f4752a00700 1 --2- 172.21.15.86:0/203290162 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4754074d30 0x7f47541a3b30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4740005910 tx=0x7f4740004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:35.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.022+0000 7f474be00700 1 -- 172.21.15.86:0/203290162 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f474001d070 con 0x7f4754074d30 2024-03-23T17:49:35.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.022+0000 7f474be00700 1 -- 172.21.15.86:0/203290162 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4740022470 con 0x7f4754074d30 2024-03-23T17:49:35.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.022+0000 7f475a000700 1 -- 172.21.15.86:0/203290162 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47400095f0 con 0x7f4754074d30 2024-03-23T17:49:35.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.022+0000 7f474be00700 1 -- 172.21.15.86:0/203290162 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4740017650 con 0x7f4754074d30 2024-03-23T17:49:35.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.022+0000 7f475a000700 1 -- 172.21.15.86:0/203290162 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47541aa410 con 0x7f4754074d30 2024-03-23T17:49:35.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.023+0000 7f475a000700 1 -- 172.21.15.86:0/203290162 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4754074670 con 0x7f4754074d30 2024-03-23T17:49:35.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.025+0000 7f474be00700 1 -- 172.21.15.86:0/203290162 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f47400225e0 con 0x7f4754074d30 2024-03-23T17:49:35.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.026+0000 7f474be00700 1 --2- 172.21.15.86:0/203290162 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f47380a11b0 0x7f47380a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:35.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.026+0000 7f474be00700 1 -- 172.21.15.86:0/203290162 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f47400cdd90 con 0x7f4754074d30 2024-03-23T17:49:35.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.026+0000 7f4753400700 1 --2- 172.21.15.86:0/203290162 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f47380a11b0 0x7f47380a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:35.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.027+0000 7f4753400700 1 --2- 172.21.15.86:0/203290162 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f47380a11b0 0x7f47380a3670 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4744009df0 tx=0x7f4744009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:35.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.028+0000 7f474be00700 1 -- 172.21.15.86:0/203290162 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f4740097420 con 0x7f4754074d30 2024-03-23T17:49:35.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:34 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/3376701695' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-23T17:49:35.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.133+0000 7fb4d4a00700 1 -- 172.21.15.86:0/347350383 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fb4d0117330 con 0x7fb4d0103810 2024-03-23T17:49:35.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.133+0000 7fb4c6a00700 1 -- 172.21.15.86:0/347350383 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb4bc01d020 con 0x7fb4d0103810 2024-03-23T17:49:35.134 INFO:teuthology.orchestra.run.smithi086.stdout:163208757289 2024-03-23T17:49:35.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.136+0000 7fb4d4a00700 1 -- 172.21.15.86:0/347350383 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fb4b40a1290 msgr2=0x7fb4b40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:35.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.136+0000 7fb4d4a00700 1 --2- 172.21.15.86:0/347350383 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fb4b40a1290 0x7fb4b40a3750 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb4c000b2e0 tx=0x7fb4c000b610 comp rx=0 tx=0).stop 2024-03-23T17:49:35.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.136+0000 7fb4d4a00700 1 -- 172.21.15.86:0/347350383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4d0103810 msgr2=0x7fb4d019dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:35.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.136+0000 7fb4d4a00700 1 --2- 172.21.15.86:0/347350383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4d0103810 0x7fb4d019dce0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc00cce0 tx=0x7fb4bc0074a0 comp rx=0 tx=0).stop 2024-03-23T17:49:35.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.136+0000 7fb4d4a00700 1 -- 172.21.15.86:0/347350383 shutdown_connections 2024-03-23T17:49:35.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.136+0000 7fb4d4a00700 1 --2- 172.21.15.86:0/347350383 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4d0105cb0 0x7fb4d01a0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:35.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.136+0000 7fb4d4a00700 1 --2- 172.21.15.86:0/347350383 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4d0068c90 0x7fb4d019d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:35.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.136+0000 7fb4d4a00700 1 --2- 172.21.15.86:0/347350383 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fb4b40a1290 0x7fb4b40a3750 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:35.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.136+0000 7fb4d4a00700 1 --2- 172.21.15.86:0/347350383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4d0103810 0x7fb4d019dce0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:35.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.136+0000 7fb4d4a00700 1 -- 172.21.15.86:0/347350383 >> 172.21.15.86:0/347350383 conn(0x7fb4d0075d50 msgr2=0x7fb4d00fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:35.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.136+0000 7fb4d4a00700 1 -- 172.21.15.86:0/347350383 shutdown_connections 2024-03-23T17:49:35.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.136+0000 7fb4d4a00700 1 -- 172.21.15.86:0/347350383 wait complete. 2024-03-23T17:49:35.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.999+0000 7f62bbe00700 1 -- 172.21.15.86:0/714335028 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62b4068c90 msgr2=0x7f62b4069110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:35.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.999+0000 7f62bbe00700 1 --2- 172.21.15.86:0/714335028 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62b4068c90 0x7f62b4069110 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f62a0009990 tx=0x7f62a0009ca0 comp rx=0 tx=0).stop 2024-03-23T17:49:35.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.999+0000 7f62bbe00700 1 -- 172.21.15.86:0/714335028 shutdown_connections 2024-03-23T17:49:35.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.999+0000 7f62bbe00700 1 --2- 172.21.15.86:0/714335028 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f62b4103810 0x7f62b4103cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:35.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.999+0000 7f62bbe00700 1 --2- 172.21.15.86:0/714335028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62b4105cb0 0x7f62b4068750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:35.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.999+0000 7f62bbe00700 1 --2- 172.21.15.86:0/714335028 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62b4068c90 0x7f62b4069110 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:35.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.999+0000 7f62bbe00700 1 -- 172.21.15.86:0/714335028 >> 172.21.15.86:0/714335028 conn(0x7f62b4075d50 msgr2=0x7f62b4076160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:35.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.999+0000 7f62bbe00700 1 -- 172.21.15.86:0/714335028 shutdown_connections 2024-03-23T17:49:35.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:34.999+0000 7f62bbe00700 1 -- 172.21.15.86:0/714335028 wait complete. 2024-03-23T17:49:35.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.000+0000 7f62bbe00700 1 Processor -- start 2024-03-23T17:49:35.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.000+0000 7f62bbe00700 1 -- start start 2024-03-23T17:49:35.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.001+0000 7f62bbe00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62b4068c90 0x7f62b419d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:35.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.001+0000 7f62bbe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62b4103810 0x7f62b419dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:35.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.001+0000 7f62bbe00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f62b4105cb0 0x7f62b41a0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:35.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.001+0000 7f62bbe00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b41a15e0 con 0x7f62b4103810 2024-03-23T17:49:35.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.001+0000 7f62bbe00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b41a1750 con 0x7f62b4068c90 2024-03-23T17:49:35.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.001+0000 7f62bbe00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b41a1910 con 0x7f62b4105cb0 2024-03-23T17:49:35.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.001+0000 7f62b8a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62b4103810 0x7f62b419dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:35.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.001+0000 7f62b8a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62b4103810 0x7f62b419dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40370/0 (socket says 172.21.15.86:40370) 2024-03-23T17:49:35.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.001+0000 7f62b8a00700 1 -- 172.21.15.86:0/3121380689 learned_addr learned my addr 172.21.15.86:0/3121380689 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:35.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.001+0000 7f62b9400700 1 --2- 172.21.15.86:0/3121380689 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62b4068c90 0x7f62b419d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:35.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.001+0000 7f62b9e00700 1 --2- 172.21.15.86:0/3121380689 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f62b4105cb0 0x7f62b41a0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:35.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.002+0000 7f62b9e00700 1 -- 172.21.15.86:0/3121380689 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62b4068c90 msgr2=0x7f62b419d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:35.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.002+0000 7f62b9e00700 1 --2- 172.21.15.86:0/3121380689 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62b4068c90 0x7f62b419d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:35.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.002+0000 7f62b9e00700 1 -- 172.21.15.86:0/3121380689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62b4103810 msgr2=0x7f62b419dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:35.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.002+0000 7f62b9e00700 1 --2- 172.21.15.86:0/3121380689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62b4103810 0x7f62b419dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:35.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.002+0000 7f62b9e00700 1 -- 172.21.15.86:0/3121380689 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62a8009520 con 0x7f62b4105cb0 2024-03-23T17:49:35.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.002+0000 7f62b8a00700 1 --2- 172.21.15.86:0/3121380689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62b4103810 0x7f62b419dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:49:35.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.002+0000 7f62b9e00700 1 --2- 172.21.15.86:0/3121380689 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f62b4105cb0 0x7f62b41a0fe0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f62ac00efc0 tx=0x7f62ac00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:35.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.002+0000 7f62b2000700 1 -- 172.21.15.86:0/3121380689 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62ac018070 con 0x7f62b4105cb0 2024-03-23T17:49:35.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.002+0000 7f62bbe00700 1 -- 172.21.15.86:0/3121380689 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62a00095f0 con 0x7f62b4105cb0 2024-03-23T17:49:35.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.002+0000 7f62bbe00700 1 -- 172.21.15.86:0/3121380689 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62b41aa370 con 0x7f62b4105cb0 2024-03-23T17:49:35.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.002+0000 7f62b2000700 1 -- 172.21.15.86:0/3121380689 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f62ac00f040 con 0x7f62b4105cb0 2024-03-23T17:49:35.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.002+0000 7f62b2000700 1 -- 172.21.15.86:0/3121380689 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62ac00cd60 con 0x7f62b4105cb0 2024-03-23T17:49:35.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.004+0000 7f62bbe00700 1 -- 172.21.15.86:0/3121380689 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62940052f0 con 0x7f62b4105cb0 2024-03-23T17:49:35.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.005+0000 7f62b2000700 1 -- 172.21.15.86:0/3121380689 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f62ac004ad0 con 0x7f62b4105cb0 2024-03-23T17:49:35.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.007+0000 7f62b2000700 1 --2- 172.21.15.86:0/3121380689 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f629c0a1230 0x7f629c0a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:35.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.007+0000 7f62b2000700 1 -- 172.21.15.86:0/3121380689 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f62ac0cc3b0 con 0x7f62b4105cb0 2024-03-23T17:49:35.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.007+0000 7f62b9400700 1 --2- 172.21.15.86:0/3121380689 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f629c0a1230 0x7f629c0a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:35.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.008+0000 7f62b9400700 1 --2- 172.21.15.86:0/3121380689 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f629c0a1230 0x7f629c0a36f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f62a8009d70 tx=0x7f62a8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:35.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.011+0000 7f62b2000700 1 -- 172.21.15.86:0/3121380689 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f62ac095aa0 con 0x7f62b4105cb0 2024-03-23T17:49:35.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.266+0000 7f62bbe00700 1 -- 172.21.15.86:0/3121380689 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f6294005160 con 0x7f62b4105cb0 2024-03-23T17:49:35.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.267+0000 7f62b2000700 1 -- 172.21.15.86:0/3121380689 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f62ac0990c0 con 0x7f62b4105cb0 2024-03-23T17:49:35.267 INFO:teuthology.orchestra.run.smithi086.stdout:193273528357 2024-03-23T17:49:35.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.268+0000 7f475a000700 1 -- 172.21.15.86:0/203290162 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7f475419ebf0 con 0x7f4754074d30 2024-03-23T17:49:35.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.268+0000 7f474be00700 1 -- 172.21.15.86:0/203290162 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f474009aa40 con 0x7f4754074d30 2024-03-23T17:49:35.269 INFO:teuthology.orchestra.run.smithi086.stdout:322122547219 2024-03-23T17:49:35.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.269+0000 7f62bbe00700 1 -- 172.21.15.86:0/3121380689 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f629c0a1230 msgr2=0x7f629c0a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:35.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.269+0000 7f62bbe00700 1 --2- 172.21.15.86:0/3121380689 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f629c0a1230 0x7f629c0a36f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f62a8009d70 tx=0x7f62a8009260 comp rx=0 tx=0).stop 2024-03-23T17:49:35.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.270+0000 7f62bbe00700 1 -- 172.21.15.86:0/3121380689 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f62b4105cb0 msgr2=0x7f62b41a0fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:35.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.270+0000 7f62bbe00700 1 --2- 172.21.15.86:0/3121380689 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f62b4105cb0 0x7f62b41a0fe0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f62ac00efc0 tx=0x7f62ac00c3a0 comp rx=0 tx=0).stop 2024-03-23T17:49:35.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.270+0000 7f62bbe00700 1 -- 172.21.15.86:0/3121380689 shutdown_connections 2024-03-23T17:49:35.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.270+0000 7f62bbe00700 1 --2- 172.21.15.86:0/3121380689 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f62b4105cb0 0x7f62b41a0fe0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:35.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.270+0000 7f62bbe00700 1 --2- 172.21.15.86:0/3121380689 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62b4068c90 0x7f62b419d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:35.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.270+0000 7f62bbe00700 1 --2- 172.21.15.86:0/3121380689 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f629c0a1230 0x7f629c0a36f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:35.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.270+0000 7f62bbe00700 1 --2- 172.21.15.86:0/3121380689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62b4103810 0x7f62b419dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:35.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.270+0000 7f62bbe00700 1 -- 172.21.15.86:0/3121380689 >> 172.21.15.86:0/3121380689 conn(0x7f62b4075d50 msgr2=0x7f62b41007e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:35.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.270+0000 7f62bbe00700 1 -- 172.21.15.86:0/3121380689 shutdown_connections 2024-03-23T17:49:35.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.270+0000 7f62bbe00700 1 -- 172.21.15.86:0/3121380689 wait complete. 2024-03-23T17:49:35.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.271+0000 7f475a000700 1 -- 172.21.15.86:0/203290162 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f47380a11b0 msgr2=0x7f47380a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:35.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.271+0000 7f475a000700 1 --2- 172.21.15.86:0/203290162 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f47380a11b0 0x7f47380a3670 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4744009df0 tx=0x7f4744009260 comp rx=0 tx=0).stop 2024-03-23T17:49:35.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.271+0000 7f475a000700 1 -- 172.21.15.86:0/203290162 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4754074d30 msgr2=0x7f47541a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:35.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.271+0000 7f475a000700 1 --2- 172.21.15.86:0/203290162 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4754074d30 0x7f47541a3b30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4740005910 tx=0x7f4740004730 comp rx=0 tx=0).stop 2024-03-23T17:49:35.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.271+0000 7f475a000700 1 -- 172.21.15.86:0/203290162 shutdown_connections 2024-03-23T17:49:35.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.271+0000 7f475a000700 1 --2- 172.21.15.86:0/203290162 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4754111b30 0x7f475419d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:35.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.271+0000 7f475a000700 1 --2- 172.21.15.86:0/203290162 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f47380a11b0 0x7f47380a3670 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:35.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.271+0000 7f475a000700 1 --2- 172.21.15.86:0/203290162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4754073520 0x7f47541a35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:35.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.271+0000 7f475a000700 1 --2- 172.21.15.86:0/203290162 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4754074d30 0x7f47541a3b30 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:35.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.271+0000 7f475a000700 1 -- 172.21.15.86:0/203290162 >> 172.21.15.86:0/203290162 conn(0x7f4754077fa0 msgr2=0x7f4754110760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:35.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.272+0000 7f475a000700 1 -- 172.21.15.86:0/203290162 shutdown_connections 2024-03-23T17:49:35.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.272+0000 7f475a000700 1 -- 172.21.15.86:0/203290162 wait complete. 2024-03-23T17:49:35.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:34 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/3376701695' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-23T17:49:35.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.396+0000 7fc8c0400700 1 -- 172.21.15.86:0/2257559580 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8b8105cb0 msgr2=0x7fc8b8068750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:35.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.396+0000 7fc8c0400700 1 --2- 172.21.15.86:0/2257559580 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8b8105cb0 0x7fc8b8068750 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac009990 tx=0x7fc8ac009ca0 comp rx=0 tx=0).stop 2024-03-23T17:49:35.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.396+0000 7fc8c0400700 1 -- 172.21.15.86:0/2257559580 shutdown_connections 2024-03-23T17:49:35.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.396+0000 7fc8c0400700 1 --2- 172.21.15.86:0/2257559580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8b8103810 0x7fc8b8103cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:35.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.396+0000 7fc8c0400700 1 --2- 172.21.15.86:0/2257559580 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8b8105cb0 0x7fc8b8068750 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:35.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.396+0000 7fc8c0400700 1 --2- 172.21.15.86:0/2257559580 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8b8068c90 0x7fc8b8069110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:35.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.396+0000 7fc8c0400700 1 -- 172.21.15.86:0/2257559580 >> 172.21.15.86:0/2257559580 conn(0x7fc8b8075d50 msgr2=0x7fc8b8076160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:35.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.396+0000 7fc8c0400700 1 -- 172.21.15.86:0/2257559580 shutdown_connections 2024-03-23T17:49:35.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.396+0000 7fc8c0400700 1 -- 172.21.15.86:0/2257559580 wait complete. 2024-03-23T17:49:35.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.397+0000 7fc8c0400700 1 Processor -- start 2024-03-23T17:49:35.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.397+0000 7fc8c0400700 1 -- start start 2024-03-23T17:49:35.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.398+0000 7fc8c0400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8b8068c90 0x7fc8b81a3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:35.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.398+0000 7fc8c0400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8b8103810 0x7fc8b81a3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:35.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.398+0000 7fc8c0400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8b8105cb0 0x7fc8b819d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:35.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.398+0000 7fc8c0400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b819dd30 con 0x7fc8b8103810 2024-03-23T17:49:35.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.398+0000 7fc8c0400700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b819dea0 con 0x7fc8b8105cb0 2024-03-23T17:49:35.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.398+0000 7fc8c0400700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b819e060 con 0x7fc8b8068c90 2024-03-23T17:49:35.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.398+0000 7fc8bd000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8b8103810 0x7fc8b81a3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:35.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.398+0000 7fc8bda00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8b8068c90 0x7fc8b81a3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:35.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.398+0000 7fc8bd000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8b8103810 0x7fc8b81a3ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40416/0 (socket says 172.21.15.86:40416) 2024-03-23T17:49:35.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.398+0000 7fc8bda00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8b8068c90 0x7fc8b81a3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:42332/0 (socket says 172.21.15.86:42332) 2024-03-23T17:49:35.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.398+0000 7fc8bd000700 1 -- 172.21.15.86:0/2811428143 learned_addr learned my addr 172.21.15.86:0/2811428143 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:35.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.398+0000 7fc8be400700 1 --2- 172.21.15.86:0/2811428143 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8b8105cb0 0x7fc8b819d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:35.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.398+0000 7fc8bda00700 1 -- 172.21.15.86:0/2811428143 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8b8105cb0 msgr2=0x7fc8b819d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:35.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.398+0000 7fc8bda00700 1 --2- 172.21.15.86:0/2811428143 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8b8105cb0 0x7fc8b819d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:35.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.398+0000 7fc8bda00700 1 -- 172.21.15.86:0/2811428143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8b8103810 msgr2=0x7fc8b81a3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:35.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.398+0000 7fc8bda00700 1 --2- 172.21.15.86:0/2811428143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8b8103810 0x7fc8b81a3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:35.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.398+0000 7fc8bda00700 1 -- 172.21.15.86:0/2811428143 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8ac0095f0 con 0x7fc8b8068c90 2024-03-23T17:49:35.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.398+0000 7fc8be400700 1 --2- 172.21.15.86:0/2811428143 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8b8105cb0 0x7fc8b819d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:49:35.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.398+0000 7fc8bd000700 1 --2- 172.21.15.86:0/2811428143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8b8103810 0x7fc8b81a3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:49:35.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.399+0000 7fc8bda00700 1 --2- 172.21.15.86:0/2811428143 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8b8068c90 0x7fc8b81a3590 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac005950 tx=0x7fc8ac005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:35.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.399+0000 7fc8b6000700 1 -- 172.21.15.86:0/2811428143 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8ac01d070 con 0x7fc8b8068c90 2024-03-23T17:49:35.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.399+0000 7fc8b6000700 1 -- 172.21.15.86:0/2811428143 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc8ac005da0 con 0x7fc8b8068c90 2024-03-23T17:49:35.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.399+0000 7fc8b6000700 1 -- 172.21.15.86:0/2811428143 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8ac021ea0 con 0x7fc8b8068c90 2024-03-23T17:49:35.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.399+0000 7fc8c0400700 1 -- 172.21.15.86:0/2811428143 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8b819e2e0 con 0x7fc8b8068c90 2024-03-23T17:49:35.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.399+0000 7fc8c0400700 1 -- 172.21.15.86:0/2811428143 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8b81aa580 con 0x7fc8b8068c90 2024-03-23T17:49:35.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.400+0000 7fc8c0400700 1 -- 172.21.15.86:0/2811428143 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8b81002f0 con 0x7fc8b8068c90 2024-03-23T17:49:35.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.404+0000 7fc8b6000700 1 -- 172.21.15.86:0/2811428143 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fc8ac0215a0 con 0x7fc8b8068c90 2024-03-23T17:49:35.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.405+0000 7fc8b6000700 1 --2- 172.21.15.86:0/2811428143 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fc8a00a11b0 0x7fc8a00a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:35.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.405+0000 7fc8b6000700 1 -- 172.21.15.86:0/2811428143 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fc8ac0cd980 con 0x7fc8b8068c90 2024-03-23T17:49:35.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.406+0000 7fc8bd000700 1 --2- 172.21.15.86:0/2811428143 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fc8a00a11b0 0x7fc8a00a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:35.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.406+0000 7fc8bd000700 1 --2- 172.21.15.86:0/2811428143 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fc8a00a11b0 0x7fc8a00a3670 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc8b819ede0 tx=0x7fc8a4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:35.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.409+0000 7fc8b6000700 1 -- 172.21.15.86:0/2811428143 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc8ac05c9f0 con 0x7fc8b8068c90 2024-03-23T17:49:35.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.625+0000 7fc8c0400700 1 -- 172.21.15.86:0/2811428143 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7fc8b804ea80 con 0x7fc8b8068c90 2024-03-23T17:49:35.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.626+0000 7fc8b6000700 1 -- 172.21.15.86:0/2811428143 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc8ac026020 con 0x7fc8b8068c90 2024-03-23T17:49:35.626 INFO:teuthology.orchestra.run.smithi086.stdout:240518168605 2024-03-23T17:49:35.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.628+0000 7fc8c0400700 1 -- 172.21.15.86:0/2811428143 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fc8a00a11b0 msgr2=0x7fc8a00a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:35.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.628+0000 7fc8c0400700 1 --2- 172.21.15.86:0/2811428143 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fc8a00a11b0 0x7fc8a00a3670 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc8b819ede0 tx=0x7fc8a4009260 comp rx=0 tx=0).stop 2024-03-23T17:49:35.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.628+0000 7fc8c0400700 1 -- 172.21.15.86:0/2811428143 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8b8068c90 msgr2=0x7fc8b81a3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:35.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.628+0000 7fc8c0400700 1 --2- 172.21.15.86:0/2811428143 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8b8068c90 0x7fc8b81a3590 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac005950 tx=0x7fc8ac005740 comp rx=0 tx=0).stop 2024-03-23T17:49:35.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.628+0000 7fc8c0400700 1 -- 172.21.15.86:0/2811428143 shutdown_connections 2024-03-23T17:49:35.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.628+0000 7fc8c0400700 1 --2- 172.21.15.86:0/2811428143 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8b8105cb0 0x7fc8b819d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:35.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.628+0000 7fc8c0400700 1 --2- 172.21.15.86:0/2811428143 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8b8068c90 0x7fc8b81a3590 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:35.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.628+0000 7fc8c0400700 1 --2- 172.21.15.86:0/2811428143 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fc8a00a11b0 0x7fc8a00a3670 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:35.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.628+0000 7fc8c0400700 1 --2- 172.21.15.86:0/2811428143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8b8103810 0x7fc8b81a3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:35.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.628+0000 7fc8c0400700 1 -- 172.21.15.86:0/2811428143 >> 172.21.15.86:0/2811428143 conn(0x7fc8b8075d50 msgr2=0x7fc8b80fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:35.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.629+0000 7fc8c0400700 1 -- 172.21.15.86:0/2811428143 shutdown_connections 2024-03-23T17:49:35.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.629+0000 7fc8c0400700 1 -- 172.21.15.86:0/2811428143 wait complete. 2024-03-23T17:49:35.861 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:35 smithi086 ceph-mon[38221]: pgmap v255: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 22 B/s, 2 keys/s, 2 objects/s recovering 2024-03-23T17:49:35.861 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:35 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/2637515001' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-23T17:49:35.861 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:35 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/347350383' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-23T17:49:35.861 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:35 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/3121380689' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-23T17:49:35.861 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:35 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/203290162' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-23T17:49:35.861 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:35 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/2811428143' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-23T17:49:36.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.949+0000 7f1433200700 1 -- 172.21.15.86:0/3581330512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f142c1003b0 msgr2=0x7f142c10cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:36.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.949+0000 7f1433200700 1 --2- 172.21.15.86:0/3581330512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f142c1003b0 0x7f142c10cfe0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f141c009990 tx=0x7f141c009ca0 comp rx=0 tx=0).stop 2024-03-23T17:49:36.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.949+0000 7f1433200700 1 -- 172.21.15.86:0/3581330512 shutdown_connections 2024-03-23T17:49:36.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.949+0000 7f1433200700 1 --2- 172.21.15.86:0/3581330512 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f142c10d520 0x7f142c10f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:36.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.949+0000 7f1433200700 1 --2- 172.21.15.86:0/3581330512 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f142c0ffa00 0x7f142c0ffde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:36.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.949+0000 7f1433200700 1 --2- 172.21.15.86:0/3581330512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f142c1003b0 0x7f142c10cfe0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:36.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.949+0000 7f1433200700 1 -- 172.21.15.86:0/3581330512 >> 172.21.15.86:0/3581330512 conn(0x7f142c0fb5f0 msgr2=0x7f142c0fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:36.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.949+0000 7f1433200700 1 -- 172.21.15.86:0/3581330512 shutdown_connections 2024-03-23T17:49:36.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.949+0000 7f1433200700 1 -- 172.21.15.86:0/3581330512 wait complete. 2024-03-23T17:49:36.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.950+0000 7f1433200700 1 Processor -- start 2024-03-23T17:49:36.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.951+0000 7f1433200700 1 -- start start 2024-03-23T17:49:36.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.951+0000 7f1433200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f142c0ffa00 0x7f142c199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:36.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.951+0000 7f1433200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f142c1003b0 0x7f142c1998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:36.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.951+0000 7f1433200700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f142c10d520 0x7f142c19dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:36.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.951+0000 7f1433200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f142c19e260 con 0x7f142c0ffa00 2024-03-23T17:49:36.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.951+0000 7f1433200700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f142c19e3d0 con 0x7f142c1003b0 2024-03-23T17:49:36.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.951+0000 7f1433200700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f142c19e590 con 0x7f142c10d520 2024-03-23T17:49:36.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.951+0000 7f1431e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f142c0ffa00 0x7f142c199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:36.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.951+0000 7f1432800700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f142c10d520 0x7f142c19dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:36.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.951+0000 7f1431e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f142c0ffa00 0x7f142c199360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40444/0 (socket says 172.21.15.86:40444) 2024-03-23T17:49:36.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.951+0000 7f1431e00700 1 -- 172.21.15.86:0/2447803547 learned_addr learned my addr 172.21.15.86:0/2447803547 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:36.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.951+0000 7f1431400700 1 --2- 172.21.15.86:0/2447803547 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f142c1003b0 0x7f142c1998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:36.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.952+0000 7f1432800700 1 -- 172.21.15.86:0/2447803547 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f142c1003b0 msgr2=0x7f142c1998a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:36.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.952+0000 7f1432800700 1 --2- 172.21.15.86:0/2447803547 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f142c1003b0 0x7f142c1998a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:36.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.952+0000 7f1432800700 1 -- 172.21.15.86:0/2447803547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f142c0ffa00 msgr2=0x7f142c199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:36.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.952+0000 7f1432800700 1 --2- 172.21.15.86:0/2447803547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f142c0ffa00 0x7f142c199360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:36.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.952+0000 7f1432800700 1 -- 172.21.15.86:0/2447803547 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1420009520 con 0x7f142c10d520 2024-03-23T17:49:36.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.952+0000 7f1431e00700 1 --2- 172.21.15.86:0/2447803547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f142c0ffa00 0x7f142c199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:49:36.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.952+0000 7f1432800700 1 --2- 172.21.15.86:0/2447803547 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f142c10d520 0x7f142c19dc50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f142800ef90 tx=0x7f142800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:36.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.952+0000 7f1426a00700 1 -- 172.21.15.86:0/2447803547 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1428018070 con 0x7f142c10d520 2024-03-23T17:49:36.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.952+0000 7f1426a00700 1 -- 172.21.15.86:0/2447803547 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f142800f040 con 0x7f142c10d520 2024-03-23T17:49:36.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.952+0000 7f1433200700 1 -- 172.21.15.86:0/2447803547 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f141c0095f0 con 0x7f142c10d520 2024-03-23T17:49:36.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.952+0000 7f1426a00700 1 -- 172.21.15.86:0/2447803547 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f142800cc30 con 0x7f142c10d520 2024-03-23T17:49:36.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.953+0000 7f1433200700 1 -- 172.21.15.86:0/2447803547 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f142c1a5f00 con 0x7f142c10d520 2024-03-23T17:49:36.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.954+0000 7f1433200700 1 -- 172.21.15.86:0/2447803547 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f142c108740 con 0x7f142c10d520 2024-03-23T17:49:36.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.955+0000 7f1426a00700 1 -- 172.21.15.86:0/2447803547 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f1428003a00 con 0x7f142c10d520 2024-03-23T17:49:36.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.957+0000 7f1426a00700 1 --2- 172.21.15.86:0/2447803547 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f14140a1230 0x7f14140a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:36.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.957+0000 7f1426a00700 1 -- 172.21.15.86:0/2447803547 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f14280cc510 con 0x7f142c10d520 2024-03-23T17:49:36.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.957+0000 7f1431e00700 1 --2- 172.21.15.86:0/2447803547 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f14140a1230 0x7f14140a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:36.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.957+0000 7f1431e00700 1 --2- 172.21.15.86:0/2447803547 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f14140a1230 0x7f14140a36f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1420009fd0 tx=0x7f1420009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:36.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:35.961+0000 7f1426a00700 1 -- 172.21.15.86:0/2447803547 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f1428095ba0 con 0x7f142c10d520 2024-03-23T17:49:36.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:35 smithi152 ceph-mon[47949]: pgmap v255: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 22 B/s, 2 keys/s, 2 objects/s recovering 2024-03-23T17:49:36.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:35 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/2637515001' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-23T17:49:36.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:35 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/347350383' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-23T17:49:36.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:35 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/3121380689' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-23T17:49:36.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:35 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/203290162' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-23T17:49:36.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:35 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/2811428143' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-23T17:49:36.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:36.193+0000 7f1433200700 1 -- 172.21.15.86:0/2447803547 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f142c04ea80 con 0x7f142c10d520 2024-03-23T17:49:36.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:36.194+0000 7f1426a00700 1 -- 172.21.15.86:0/2447803547 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f14280991c0 con 0x7f142c10d520 2024-03-23T17:49:36.194 INFO:teuthology.orchestra.run.smithi086.stdout:55834574903 2024-03-23T17:49:36.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:36.196+0000 7f1433200700 1 -- 172.21.15.86:0/2447803547 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f14140a1230 msgr2=0x7f14140a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:36.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:36.196+0000 7f1433200700 1 --2- 172.21.15.86:0/2447803547 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f14140a1230 0x7f14140a36f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1420009fd0 tx=0x7f1420009260 comp rx=0 tx=0).stop 2024-03-23T17:49:36.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:36.196+0000 7f1433200700 1 -- 172.21.15.86:0/2447803547 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f142c10d520 msgr2=0x7f142c19dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:36.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:36.196+0000 7f1433200700 1 --2- 172.21.15.86:0/2447803547 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f142c10d520 0x7f142c19dc50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f142800ef90 tx=0x7f142800c3a0 comp rx=0 tx=0).stop 2024-03-23T17:49:36.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:36.196+0000 7f1433200700 1 -- 172.21.15.86:0/2447803547 shutdown_connections 2024-03-23T17:49:36.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:36.196+0000 7f1433200700 1 --2- 172.21.15.86:0/2447803547 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f142c10d520 0x7f142c19dc50 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:36.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:36.196+0000 7f1433200700 1 --2- 172.21.15.86:0/2447803547 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f14140a1230 0x7f14140a36f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:36.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:36.196+0000 7f1433200700 1 --2- 172.21.15.86:0/2447803547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f142c0ffa00 0x7f142c199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:36.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:36.196+0000 7f1433200700 1 --2- 172.21.15.86:0/2447803547 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f142c1003b0 0x7f142c1998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:36.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:36.196+0000 7f1433200700 1 -- 172.21.15.86:0/2447803547 >> 172.21.15.86:0/2447803547 conn(0x7f142c0fb5f0 msgr2=0x7f142c0fd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:36.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:36.197+0000 7f1433200700 1 -- 172.21.15.86:0/2447803547 shutdown_connections 2024-03-23T17:49:36.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:36.197+0000 7f1433200700 1 -- 172.21.15.86:0/2447803547 wait complete. 2024-03-23T17:49:36.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:35 smithi092 ceph-mon[79089]: pgmap v255: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 22 B/s, 2 keys/s, 2 objects/s recovering 2024-03-23T17:49:36.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:35 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/2637515001' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-23T17:49:36.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:35 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/347350383' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-23T17:49:36.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:35 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/3121380689' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-23T17:49:36.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:35 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/203290162' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-23T17:49:36.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:35 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/2811428143' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-23T17:49:36.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:36.260+0000 7f88ab000700 1 -- 172.21.15.86:0/3704228117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88a4106ef0 msgr2=0x7f88a41072d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:36.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:36.260+0000 7f88ab000700 1 --2- 172.21.15.86:0/3704228117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88a4106ef0 0x7f88a41072d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f88940099c0 tx=0x7f8894009cd0 comp rx=0 tx=0).stop 2024-03-23T17:49:36.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:36.260+0000 7f88ab000700 1 -- 172.21.15.86:0/3704228117 shutdown_connections 2024-03-23T17:49:36.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:36.260+0000 7f88ab000700 1 --2- 172.21.15.86:0/3704228117 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88a4101820 0x7f88a410dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:36.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:36.260+0000 7f88ab000700 1 --2- 172.21.15.86:0/3704228117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88a4106ef0 0x7f88a41072d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:36.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:36.260+0000 7f88ab000700 1 --2- 172.21.15.86:0/3704228117 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88a4100e60 0x7f88a41012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:36.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:36.260+0000 7f88ab000700 1 -- 172.21.15.86:0/3704228117 >> 172.21.15.86:0/3704228117 conn(0x7f88a4077fa0 msgr2=0x7f88a40783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:36.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:36.260+0000 7f88ab000700 1 -- 172.21.15.86:0/3704228117 shutdown_connections 2024-03-23T17:49:36.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:36.260+0000 7f88ab000700 1 -- 172.21.15.86:0/3704228117 wait complete. 2024-03-23T17:49:36.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:36.261+0000 7f88ab000700 1 Processor -- start 2024-03-23T17:49:36.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:36.262+0000 7f88ab000700 1 -- start start 2024-03-23T17:49:36.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:36.262+0000 7f88ab000700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88a4100e60 0x7f88a4199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:36.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:36.262+0000 7f88ab000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88a4101820 0x7f88a4199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:36.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:36.262+0000 7f88ab000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88a4106ef0 0x7f88a419dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:36.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:36.262+0000 7f88ab000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88a419e300 con 0x7f88a4106ef0 2024-03-23T17:49:36.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:36.262+0000 7f88ab000700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88a419e470 con 0x7f88a4101820 2024-03-23T17:49:36.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:36.262+0000 7f88ab000700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88a419e630 con 0x7f88a4100e60 2024-03-23T17:49:36.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:36.263+0000 7f88a3e00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88a4100e60 0x7f88a4199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:36.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:36.263+0000 7f88a9000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88a4106ef0 0x7f88a419dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:36.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:36.263+0000 7f88a3e00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88a4100e60 0x7f88a4199400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:42362/0 (socket says 172.21.15.86:42362) 2024-03-23T17:49:36.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:36.263+0000 7f88a3e00700 1 -- 172.21.15.86:0/1208754714 learned_addr learned my addr 172.21.15.86:0/1208754714 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:36.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:36.263+0000 7f88a3400700 1 --2- 172.21.15.86:0/1208754714 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88a4101820 0x7f88a4199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:36.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:36.263+0000 7f88a3e00700 1 -- 172.21.15.86:0/1208754714 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88a4101820 msgr2=0x7f88a4199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:36.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:36.263+0000 7f88a3e00700 1 --2- 172.21.15.86:0/1208754714 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88a4101820 0x7f88a4199940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:36.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:36.263+0000 7f88a3e00700 1 -- 172.21.15.86:0/1208754714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88a4106ef0 msgr2=0x7f88a419dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:36.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:36.263+0000 7f88a3e00700 1 --2- 172.21.15.86:0/1208754714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88a4106ef0 0x7f88a419dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:36.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:36.263+0000 7f88a3e00700 1 -- 172.21.15.86:0/1208754714 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88940095f0 con 0x7f88a4100e60 2024-03-23T17:49:36.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:36.263+0000 7f88a3e00700 1 --2- 172.21.15.86:0/1208754714 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88a4100e60 0x7f88a4199400 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f889400b580 tx=0x7f889400f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:36.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:36.263+0000 7f88a0c00700 1 -- 172.21.15.86:0/1208754714 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f889401c070 con 0x7f88a4100e60 2024-03-23T17:49:36.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:36.264+0000 7f88a0c00700 1 -- 172.21.15.86:0/1208754714 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f889400fd80 con 0x7f88a4100e60 2024-03-23T17:49:36.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:36.264+0000 7f88ab000700 1 -- 172.21.15.86:0/1208754714 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88a419e8b0 con 0x7f88a4100e60 2024-03-23T17:49:36.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:36.264+0000 7f88a0c00700 1 -- 172.21.15.86:0/1208754714 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8894022c40 con 0x7f88a4100e60 2024-03-23T17:49:36.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:36.264+0000 7f88ab000700 1 -- 172.21.15.86:0/1208754714 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88a41a6170 con 0x7f88a4100e60 2024-03-23T17:49:36.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:36.266+0000 7f88ab000700 1 -- 172.21.15.86:0/1208754714 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88840052f0 con 0x7f88a4100e60 2024-03-23T17:49:36.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:36.267+0000 7f88a0c00700 1 -- 172.21.15.86:0/1208754714 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f8894020070 con 0x7f88a4100e60 2024-03-23T17:49:36.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:36.268+0000 7f88a0c00700 1 --2- 172.21.15.86:0/1208754714 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f88880a11b0 0x7f88880a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:36.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:36.268+0000 7f88a0c00700 1 -- 172.21.15.86:0/1208754714 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f88940cd6f0 con 0x7f88a4100e60 2024-03-23T17:49:36.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:36.269+0000 7f88a3400700 1 --2- 172.21.15.86:0/1208754714 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f88880a11b0 0x7f88880a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:36.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:36.269+0000 7f88a3400700 1 --2- 172.21.15.86:0/1208754714 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f88880a11b0 0x7f88880a3670 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f88a419a960 tx=0x7f8890009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:36.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:36.272+0000 7f88a0c00700 1 -- 172.21.15.86:0/1208754714 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f8894096d80 con 0x7f88a4100e60 2024-03-23T17:49:36.405 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776148 got 292057776149 for osd.10 2024-03-23T17:49:36.405 DEBUG:teuthology.parallel:result is None 2024-03-23T17:49:36.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:36.510+0000 7f88ab000700 1 -- 172.21.15.86:0/1208754714 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f8884005160 con 0x7f88a4100e60 2024-03-23T17:49:36.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:36.510+0000 7f88a0c00700 1 -- 172.21.15.86:0/1208754714 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f889409a3a0 con 0x7f88a4100e60 2024-03-23T17:49:36.511 INFO:teuthology.orchestra.run.smithi086.stdout:111669149744 2024-03-23T17:49:36.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:36.513+0000 7f88ab000700 1 -- 172.21.15.86:0/1208754714 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f88880a11b0 msgr2=0x7f88880a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:36.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:36.513+0000 7f88ab000700 1 --2- 172.21.15.86:0/1208754714 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f88880a11b0 0x7f88880a3670 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f88a419a960 tx=0x7f8890009260 comp rx=0 tx=0).stop 2024-03-23T17:49:36.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:36.513+0000 7f88ab000700 1 -- 172.21.15.86:0/1208754714 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88a4100e60 msgr2=0x7f88a4199400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:36.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:36.513+0000 7f88ab000700 1 --2- 172.21.15.86:0/1208754714 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88a4100e60 0x7f88a4199400 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f889400b580 tx=0x7f889400f690 comp rx=0 tx=0).stop 2024-03-23T17:49:36.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:36.513+0000 7f88ab000700 1 -- 172.21.15.86:0/1208754714 shutdown_connections 2024-03-23T17:49:36.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:36.513+0000 7f88ab000700 1 --2- 172.21.15.86:0/1208754714 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f88880a11b0 0x7f88880a3670 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:36.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:36.513+0000 7f88ab000700 1 --2- 172.21.15.86:0/1208754714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88a4106ef0 0x7f88a419dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:36.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:36.513+0000 7f88ab000700 1 --2- 172.21.15.86:0/1208754714 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88a4100e60 0x7f88a4199400 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:36.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:36.513+0000 7f88ab000700 1 --2- 172.21.15.86:0/1208754714 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88a4101820 0x7f88a4199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:36.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:36.513+0000 7f88ab000700 1 -- 172.21.15.86:0/1208754714 >> 172.21.15.86:0/1208754714 conn(0x7f88a4077fa0 msgr2=0x7f88a410c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:36.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:36.513+0000 7f88ab000700 1 -- 172.21.15.86:0/1208754714 shutdown_connections 2024-03-23T17:49:36.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:36.513+0000 7f88ab000700 1 -- 172.21.15.86:0/1208754714 wait complete. 2024-03-23T17:49:36.550 INFO:tasks.cephadm.ceph_manager.ceph:need seq 240518168602 got 240518168605 for osd.8 2024-03-23T17:49:36.551 DEBUG:teuthology.parallel:result is None 2024-03-23T17:49:36.744 INFO:tasks.cephadm.ceph_manager.ceph:need seq 270582939673 got 270582939675 for osd.9 2024-03-23T17:49:36.744 DEBUG:teuthology.parallel:result is None 2024-03-23T17:49:36.949 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757287 got 163208757289 for osd.5 2024-03-23T17:49:36.950 DEBUG:teuthology.parallel:result is None 2024-03-23T17:49:37.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:36 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/2447803547' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-23T17:49:37.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:36 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/1208754714' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-23T17:49:37.095 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:36 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/2447803547' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-23T17:49:37.095 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:36 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/1208754714' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-23T17:49:37.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:36 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/2447803547' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-23T17:49:37.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:36 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/1208754714' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-23T17:49:37.390 INFO:tasks.cephadm.ceph_manager.ceph:need seq 322122547217 got 322122547219 for osd.11 2024-03-23T17:49:37.391 DEBUG:teuthology.parallel:result is None 2024-03-23T17:49:37.736 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149742 got 111669149744 for osd.3 2024-03-23T17:49:37.737 DEBUG:teuthology.parallel:result is None 2024-03-23T17:49:37.900 INFO:tasks.cephadm.ceph_manager.ceph:need seq 193273528355 got 193273528357 for osd.6 2024-03-23T17:49:37.900 DEBUG:teuthology.parallel:result is None 2024-03-23T17:49:37.995 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:37 smithi086 ceph-mon[38221]: pgmap v256: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s; 22 B/s, 2 keys/s, 2 objects/s recovering 2024-03-23T17:49:38.021 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574901 got 55834574903 for osd.1 2024-03-23T17:49:38.021 DEBUG:teuthology.parallel:result is None 2024-03-23T17:49:38.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:37 smithi152 ceph-mon[47949]: pgmap v256: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s; 22 B/s, 2 keys/s, 2 objects/s recovering 2024-03-23T17:49:38.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:37 smithi092 ceph-mon[79089]: pgmap v256: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s; 22 B/s, 2 keys/s, 2 objects/s recovering 2024-03-23T17:49:38.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:38.342+0000 7f735b200700 1 -- 172.21.15.86:0/2495737342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7354106ef0 msgr2=0x7f73541072d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:38.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:38.342+0000 7f735b200700 1 --2- 172.21.15.86:0/2495737342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7354106ef0 0x7f73541072d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f73440099c0 tx=0x7f7344009cd0 comp rx=0 tx=0).stop 2024-03-23T17:49:38.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:38.342+0000 7f735b200700 1 -- 172.21.15.86:0/2495737342 shutdown_connections 2024-03-23T17:49:38.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:38.342+0000 7f735b200700 1 --2- 172.21.15.86:0/2495737342 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7354101820 0x7f735410dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:38.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:38.342+0000 7f735b200700 1 --2- 172.21.15.86:0/2495737342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7354106ef0 0x7f73541072d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:38.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:38.342+0000 7f735b200700 1 --2- 172.21.15.86:0/2495737342 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7354100e60 0x7f73541012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:38.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:38.342+0000 7f735b200700 1 -- 172.21.15.86:0/2495737342 >> 172.21.15.86:0/2495737342 conn(0x7f7354077fa0 msgr2=0x7f73540783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:38.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:38.342+0000 7f735b200700 1 -- 172.21.15.86:0/2495737342 shutdown_connections 2024-03-23T17:49:38.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:38.342+0000 7f735b200700 1 -- 172.21.15.86:0/2495737342 wait complete. 2024-03-23T17:49:38.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:38.343+0000 7f735b200700 1 Processor -- start 2024-03-23T17:49:38.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:38.343+0000 7f735b200700 1 -- start start 2024-03-23T17:49:38.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:38.344+0000 7f735b200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7354100e60 0x7f73541993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:38.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:38.344+0000 7f735b200700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7354101820 0x7f7354199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:38.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:38.344+0000 7f735b200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7354106ef0 0x7f735419dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:38.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:38.344+0000 7f735b200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f735419e2c0 con 0x7f7354100e60 2024-03-23T17:49:38.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:38.344+0000 7f735b200700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f735419e430 con 0x7f7354106ef0 2024-03-23T17:49:38.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:38.344+0000 7f735b200700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f735419e5f0 con 0x7f7354101820 2024-03-23T17:49:38.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:38.344+0000 7f7353e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7354100e60 0x7f73541993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:38.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:38.344+0000 7f7353e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7354100e60 0x7f73541993c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40478/0 (socket says 172.21.15.86:40478) 2024-03-23T17:49:38.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:38.344+0000 7f7353e00700 1 -- 172.21.15.86:0/2419888903 learned_addr learned my addr 172.21.15.86:0/2419888903 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:38.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:38.344+0000 7f7353400700 1 --2- 172.21.15.86:0/2419888903 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7354101820 0x7f7354199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:38.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:38.344+0000 7f7359200700 1 --2- 172.21.15.86:0/2419888903 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7354106ef0 0x7f735419dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:38.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:38.344+0000 7f7353e00700 1 -- 172.21.15.86:0/2419888903 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7354101820 msgr2=0x7f7354199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:38.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:38.344+0000 7f7353e00700 1 --2- 172.21.15.86:0/2419888903 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7354101820 0x7f7354199900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:38.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:38.344+0000 7f7353e00700 1 -- 172.21.15.86:0/2419888903 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7354106ef0 msgr2=0x7f735419dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:38.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:38.344+0000 7f7353e00700 1 --2- 172.21.15.86:0/2419888903 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7354106ef0 0x7f735419dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:38.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:38.345+0000 7f7353e00700 1 -- 172.21.15.86:0/2419888903 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73440095f0 con 0x7f7354100e60 2024-03-23T17:49:38.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:38.345+0000 7f7353e00700 1 --2- 172.21.15.86:0/2419888903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7354100e60 0x7f73541993c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f734400b580 tx=0x7f734400bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:38.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:38.345+0000 7f7350c00700 1 -- 172.21.15.86:0/2419888903 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f734401c070 con 0x7f7354100e60 2024-03-23T17:49:38.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:38.345+0000 7f7350c00700 1 -- 172.21.15.86:0/2419888903 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f734400fd40 con 0x7f7354100e60 2024-03-23T17:49:38.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:38.345+0000 7f735b200700 1 -- 172.21.15.86:0/2419888903 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f735419e870 con 0x7f7354100e60 2024-03-23T17:49:38.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:38.345+0000 7f7350c00700 1 -- 172.21.15.86:0/2419888903 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7344022b80 con 0x7f7354100e60 2024-03-23T17:49:38.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:38.345+0000 7f735b200700 1 -- 172.21.15.86:0/2419888903 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73541a6130 con 0x7f7354100e60 2024-03-23T17:49:38.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:38.346+0000 7f735b200700 1 -- 172.21.15.86:0/2419888903 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73541024f0 con 0x7f7354100e60 2024-03-23T17:49:38.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:38.348+0000 7f7350c00700 1 -- 172.21.15.86:0/2419888903 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f7344020070 con 0x7f7354100e60 2024-03-23T17:49:38.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:38.349+0000 7f7350c00700 1 --2- 172.21.15.86:0/2419888903 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f73380a1230 0x7f73380a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:38.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:38.349+0000 7f7350c00700 1 -- 172.21.15.86:0/2419888903 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f73440cd730 con 0x7f7354100e60 2024-03-23T17:49:38.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:38.349+0000 7f7353400700 1 --2- 172.21.15.86:0/2419888903 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f73380a1230 0x7f73380a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:38.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:38.350+0000 7f7353400700 1 --2- 172.21.15.86:0/2419888903 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f73380a1230 0x7f73380a36f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f735419a920 tx=0x7f7340009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:38.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:38.354+0000 7f7350c00700 1 -- 172.21.15.86:0/2419888903 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f7344096dc0 con 0x7f7354100e60 2024-03-23T17:49:38.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:38.558+0000 7f735b200700 1 -- 172.21.15.86:0/2419888903 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f735404ea80 con 0x7f7354100e60 2024-03-23T17:49:38.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:38.558+0000 7f7350c00700 1 -- 172.21.15.86:0/2419888903 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f734409a3e0 con 0x7f7354100e60 2024-03-23T17:49:38.558 INFO:teuthology.orchestra.run.smithi086.stdout:77309411381 2024-03-23T17:49:38.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:38.560+0000 7f735b200700 1 -- 172.21.15.86:0/2419888903 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f73380a1230 msgr2=0x7f73380a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:38.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:38.560+0000 7f735b200700 1 --2- 172.21.15.86:0/2419888903 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f73380a1230 0x7f73380a36f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f735419a920 tx=0x7f7340009260 comp rx=0 tx=0).stop 2024-03-23T17:49:38.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:38.560+0000 7f735b200700 1 -- 172.21.15.86:0/2419888903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7354100e60 msgr2=0x7f73541993c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:38.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:38.560+0000 7f735b200700 1 --2- 172.21.15.86:0/2419888903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7354100e60 0x7f73541993c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f734400b580 tx=0x7f734400bfd0 comp rx=0 tx=0).stop 2024-03-23T17:49:38.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:38.560+0000 7f735b200700 1 -- 172.21.15.86:0/2419888903 shutdown_connections 2024-03-23T17:49:38.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:38.560+0000 7f735b200700 1 --2- 172.21.15.86:0/2419888903 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7354106ef0 0x7f735419dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:38.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:38.560+0000 7f735b200700 1 --2- 172.21.15.86:0/2419888903 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f73380a1230 0x7f73380a36f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:38.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:38.560+0000 7f735b200700 1 --2- 172.21.15.86:0/2419888903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7354100e60 0x7f73541993c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:38.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:38.560+0000 7f735b200700 1 --2- 172.21.15.86:0/2419888903 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7354101820 0x7f7354199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:38.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:38.560+0000 7f735b200700 1 -- 172.21.15.86:0/2419888903 >> 172.21.15.86:0/2419888903 conn(0x7f7354077fa0 msgr2=0x7f735410c260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:38.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:38.561+0000 7f735b200700 1 -- 172.21.15.86:0/2419888903 shutdown_connections 2024-03-23T17:49:38.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:38.561+0000 7f735b200700 1 -- 172.21.15.86:0/2419888903 wait complete. 2024-03-23T17:49:38.969 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:38 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/2419888903' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-23T17:49:38.996 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411379 got 77309411381 for osd.2 2024-03-23T17:49:38.996 DEBUG:teuthology.parallel:result is None 2024-03-23T17:49:38.996 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-03-23T17:49:38.996 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph pg dump --format=json 2024-03-23T17:49:39.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:38 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/2419888903' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-23T17:49:39.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:38 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/2419888903' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-23T17:49:39.354 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T17:49:40.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:39 smithi152 ceph-mon[47949]: pgmap v257: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 18 B/s, 1 keys/s, 1 objects/s recovering 2024-03-23T17:49:40.227 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:39 smithi086 ceph-mon[38221]: pgmap v257: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 18 B/s, 1 keys/s, 1 objects/s recovering 2024-03-23T17:49:40.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:39 smithi092 ceph-mon[79089]: pgmap v257: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 18 B/s, 1 keys/s, 1 objects/s recovering 2024-03-23T17:49:40.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:40.448+0000 7fe6ae400700 1 -- 172.21.15.86:0/98908882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6a8106ef0 msgr2=0x7fe6a81072d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:40.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:40.448+0000 7fe6ae400700 1 --2- 172.21.15.86:0/98908882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6a8106ef0 0x7fe6a81072d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe6980099c0 tx=0x7fe698009cd0 comp rx=0 tx=0).stop 2024-03-23T17:49:40.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:40.448+0000 7fe6ae400700 1 -- 172.21.15.86:0/98908882 shutdown_connections 2024-03-23T17:49:40.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:40.448+0000 7fe6ae400700 1 --2- 172.21.15.86:0/98908882 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6a8101820 0x7fe6a810dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:40.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:40.448+0000 7fe6ae400700 1 --2- 172.21.15.86:0/98908882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6a8106ef0 0x7fe6a81072d0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:40.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:40.448+0000 7fe6ae400700 1 --2- 172.21.15.86:0/98908882 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe6a8100e60 0x7fe6a81012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:40.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:40.448+0000 7fe6ae400700 1 -- 172.21.15.86:0/98908882 >> 172.21.15.86:0/98908882 conn(0x7fe6a8077fa0 msgr2=0x7fe6a80783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:40.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:40.448+0000 7fe6ae400700 1 -- 172.21.15.86:0/98908882 shutdown_connections 2024-03-23T17:49:40.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:40.448+0000 7fe6ae400700 1 -- 172.21.15.86:0/98908882 wait complete. 2024-03-23T17:49:40.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:40.450+0000 7fe6ae400700 1 Processor -- start 2024-03-23T17:49:40.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:40.450+0000 7fe6ae400700 1 -- start start 2024-03-23T17:49:40.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:40.451+0000 7fe6ae400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe6a8100e60 0x7fe6a80730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:40.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:40.451+0000 7fe6ae400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6a8101820 0x7fe6a80735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:40.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:40.451+0000 7fe6ae400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6a8106ef0 0x7fe6a8073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:40.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:40.451+0000 7fe6ae400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a81a5580 con 0x7fe6a8101820 2024-03-23T17:49:40.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:40.451+0000 7fe6ae400700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a81a56f0 con 0x7fe6a8100e60 2024-03-23T17:49:40.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:40.451+0000 7fe6ae400700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a81a58b0 con 0x7fe6a8106ef0 2024-03-23T17:49:40.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:40.451+0000 7fe6a6a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6a8101820 0x7fe6a80735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:40.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:40.451+0000 7fe6a7400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe6a8100e60 0x7fe6a80730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:40.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:40.451+0000 7fe6a6a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6a8101820 0x7fe6a80735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40500/0 (socket says 172.21.15.86:40500) 2024-03-23T17:49:40.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:40.451+0000 7fe6a7400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe6a8100e60 0x7fe6a80730b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:50532/0 (socket says 172.21.15.86:50532) 2024-03-23T17:49:40.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:40.451+0000 7fe6a6a00700 1 -- 172.21.15.86:0/2479978637 learned_addr learned my addr 172.21.15.86:0/2479978637 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:40.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:40.451+0000 7fe6a6a00700 1 -- 172.21.15.86:0/2479978637 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6a8106ef0 msgr2=0x7fe6a8073b30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-23T17:49:40.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:40.451+0000 7fe6a6a00700 1 --2- 172.21.15.86:0/2479978637 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6a8106ef0 0x7fe6a8073b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:40.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:40.451+0000 7fe6a6a00700 1 -- 172.21.15.86:0/2479978637 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe6a8100e60 msgr2=0x7fe6a80730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:40.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:40.451+0000 7fe6a6a00700 1 --2- 172.21.15.86:0/2479978637 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe6a8100e60 0x7fe6a80730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:40.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:40.451+0000 7fe6a6a00700 1 -- 172.21.15.86:0/2479978637 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6980095f0 con 0x7fe6a8101820 2024-03-23T17:49:40.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:40.451+0000 7fe6a7400700 1 --2- 172.21.15.86:0/2479978637 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe6a8100e60 0x7fe6a80730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:49:40.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:40.452+0000 7fe6a6a00700 1 --2- 172.21.15.86:0/2479978637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6a8101820 0x7fe6a80735f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe69400e9a0 tx=0x7fe69400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:40.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:40.452+0000 7fe69fe00700 1 -- 172.21.15.86:0/2479978637 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe694009790 con 0x7fe6a8101820 2024-03-23T17:49:40.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:40.452+0000 7fe69fe00700 1 -- 172.21.15.86:0/2479978637 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe69400cbd0 con 0x7fe6a8101820 2024-03-23T17:49:40.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:40.452+0000 7fe6ae400700 1 -- 172.21.15.86:0/2479978637 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6a81a5b90 con 0x7fe6a8101820 2024-03-23T17:49:40.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:40.452+0000 7fe69fe00700 1 -- 172.21.15.86:0/2479978637 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe694018680 con 0x7fe6a8101820 2024-03-23T17:49:40.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:40.452+0000 7fe6ae400700 1 -- 172.21.15.86:0/2479978637 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6a81a60e0 con 0x7fe6a8101820 2024-03-23T17:49:40.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:40.453+0000 7fe6ae400700 1 -- 172.21.15.86:0/2479978637 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6a81092d0 con 0x7fe6a8101820 2024-03-23T17:49:40.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:40.456+0000 7fe69fe00700 1 -- 172.21.15.86:0/2479978637 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fe69400cd40 con 0x7fe6a8101820 2024-03-23T17:49:40.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:40.457+0000 7fe69fe00700 1 --2- 172.21.15.86:0/2479978637 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fe68c0a1230 0x7fe68c0a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:40.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:40.457+0000 7fe69fe00700 1 -- 172.21.15.86:0/2479978637 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fe694014070 con 0x7fe6a8101820 2024-03-23T17:49:40.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:40.457+0000 7fe6a7400700 1 --2- 172.21.15.86:0/2479978637 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fe68c0a1230 0x7fe68c0a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:40.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:40.458+0000 7fe6a7400700 1 --2- 172.21.15.86:0/2479978637 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fe68c0a1230 0x7fe68c0a36f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe698009990 tx=0x7fe698005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:40.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:40.461+0000 7fe69fe00700 1 -- 172.21.15.86:0/2479978637 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fe6940529f0 con 0x7fe6a8101820 2024-03-23T17:49:40.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:40.659+0000 7fe6ae400700 1 -- 172.21.15.86:0/2479978637 --> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe6a8074590 con 0x7fe68c0a1230 2024-03-23T17:49:40.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:40.670+0000 7fe69fe00700 1 -- 172.21.15.86:0/2479978637 <== mgr.14152 v2:172.21.15.86:6800/2734784244 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+341373 (secure 0 0 0) 0x7fe6a8074590 con 0x7fe68c0a1230 2024-03-23T17:49:40.678 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:49:40.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:40.675+0000 7fe6ae400700 1 -- 172.21.15.86:0/2479978637 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fe68c0a1230 msgr2=0x7fe68c0a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:40.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:40.676+0000 7fe6ae400700 1 --2- 172.21.15.86:0/2479978637 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fe68c0a1230 0x7fe68c0a36f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe698009990 tx=0x7fe698005fb0 comp rx=0 tx=0).stop 2024-03-23T17:49:40.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:40.676+0000 7fe6ae400700 1 -- 172.21.15.86:0/2479978637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6a8101820 msgr2=0x7fe6a80735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:40.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:40.676+0000 7fe6ae400700 1 --2- 172.21.15.86:0/2479978637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6a8101820 0x7fe6a80735f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe69400e9a0 tx=0x7fe69400ecb0 comp rx=0 tx=0).stop 2024-03-23T17:49:40.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:40.676+0000 7fe6ae400700 1 -- 172.21.15.86:0/2479978637 shutdown_connections 2024-03-23T17:49:40.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:40.676+0000 7fe6ae400700 1 --2- 172.21.15.86:0/2479978637 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6a8106ef0 0x7fe6a8073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:40.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:40.676+0000 7fe6ae400700 1 --2- 172.21.15.86:0/2479978637 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe6a8100e60 0x7fe6a80730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:40.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:40.676+0000 7fe6ae400700 1 --2- 172.21.15.86:0/2479978637 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fe68c0a1230 0x7fe68c0a36f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:40.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:40.676+0000 7fe6ae400700 1 --2- 172.21.15.86:0/2479978637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6a8101820 0x7fe6a80735f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:40.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:40.676+0000 7fe6ae400700 1 -- 172.21.15.86:0/2479978637 >> 172.21.15.86:0/2479978637 conn(0x7fe6a8077fa0 msgr2=0x7fe6a80ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:40.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:40.676+0000 7fe6ae400700 1 -- 172.21.15.86:0/2479978637 shutdown_connections 2024-03-23T17:49:40.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:40.676+0000 7fe6ae400700 1 -- 172.21.15.86:0/2479978637 wait complete. 2024-03-23T17:49:40.681 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-03-23T17:49:41.100 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":258,"stamp":"2024-03-23T17:49:40.318047+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656250,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":534,"num_read_kb":394,"num_write":411,"num_write_kb":3068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":1180916,"num_keys_recovered":17,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":339,"ondisk_log_size":339,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":373684,"kb_used_data":12560,"kb_used_omap":18,"kb_used_meta":356845,"kb_avail":1124568140,"statfs":{"total":1151940427776,"available":1151557775360,"internally_reserved":0,"allocated":12861440,"data_stored":7097035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19130,"internal_metadata":365409606},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":99,"num_read_kb":50,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":180,"num_keys_recovered":17,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.541615"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.782853+0000","last_change":"2024-03-23T17:48:56.239724+0000","last_active":"2024-03-23T17:49:28.782853+0000","last_peered":"2024-03-23T17:49:28.782853+0000","last_clean":"2024-03-23T17:49:28.782853+0000","last_became_active":"2024-03-23T17:48:56.239451+0000","last_became_peered":"2024-03-23T17:48:56.239451+0000","last_unstale":"2024-03-23T17:49:28.782853+0000","last_undegraded":"2024-03-23T17:49:28.782853+0000","last_fullsized":"2024-03-23T17:49:28.782853+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:51:38.687863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.783538+0000","last_change":"2024-03-23T17:48:56.232415+0000","last_active":"2024-03-23T17:49:28.783538+0000","last_peered":"2024-03-23T17:49:28.783538+0000","last_clean":"2024-03-23T17:49:28.783538+0000","last_became_active":"2024-03-23T17:48:56.231777+0000","last_became_peered":"2024-03-23T17:48:56.231777+0000","last_unstale":"2024-03-23T17:49:28.783538+0000","last_undegraded":"2024-03-23T17:49:28.783538+0000","last_fullsized":"2024-03-23T17:49:28.783538+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:16:22.631654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.784174+0000","last_change":"2024-03-23T17:48:56.243413+0000","last_active":"2024-03-23T17:49:28.784174+0000","last_peered":"2024-03-23T17:49:28.784174+0000","last_clean":"2024-03-23T17:49:28.784174+0000","last_became_active":"2024-03-23T17:48:56.241281+0000","last_became_peered":"2024-03-23T17:48:56.241281+0000","last_unstale":"2024-03-23T17:49:28.784174+0000","last_undegraded":"2024-03-23T17:49:28.784174+0000","last_fullsized":"2024-03-23T17:49:28.784174+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:52:48.521689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.784355+0000","last_change":"2024-03-23T17:48:56.226010+0000","last_active":"2024-03-23T17:49:28.784355+0000","last_peered":"2024-03-23T17:49:28.784355+0000","last_clean":"2024-03-23T17:49:28.784355+0000","last_became_active":"2024-03-23T17:48:56.225562+0000","last_became_peered":"2024-03-23T17:48:56.225562+0000","last_unstale":"2024-03-23T17:49:28.784355+0000","last_undegraded":"2024-03-23T17:49:28.784355+0000","last_fullsized":"2024-03-23T17:49:28.784355+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:17:47.597335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.780757+0000","last_change":"2024-03-23T17:48:56.223249+0000","last_active":"2024-03-23T17:49:28.780757+0000","last_peered":"2024-03-23T17:49:28.780757+0000","last_clean":"2024-03-23T17:49:28.780757+0000","last_became_active":"2024-03-23T17:48:56.222902+0000","last_became_peered":"2024-03-23T17:48:56.222902+0000","last_unstale":"2024-03-23T17:49:28.780757+0000","last_undegraded":"2024-03-23T17:49:28.780757+0000","last_fullsized":"2024-03-23T17:49:28.780757+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:51:49.029628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.781946+0000","last_change":"2024-03-23T17:48:56.229229+0000","last_active":"2024-03-23T17:49:28.781946+0000","last_peered":"2024-03-23T17:49:28.781946+0000","last_clean":"2024-03-23T17:49:28.781946+0000","last_became_active":"2024-03-23T17:48:56.228725+0000","last_became_peered":"2024-03-23T17:48:56.228725+0000","last_unstale":"2024-03-23T17:49:28.781946+0000","last_undegraded":"2024-03-23T17:49:28.781946+0000","last_fullsized":"2024-03-23T17:49:28.781946+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:53:44.118762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.780711+0000","last_change":"2024-03-23T17:48:56.229212+0000","last_active":"2024-03-23T17:49:28.780711+0000","last_peered":"2024-03-23T17:49:28.780711+0000","last_clean":"2024-03-23T17:49:28.780711+0000","last_became_active":"2024-03-23T17:48:56.228888+0000","last_became_peered":"2024-03-23T17:48:56.228888+0000","last_unstale":"2024-03-23T17:49:28.780711+0000","last_undegraded":"2024-03-23T17:49:28.780711+0000","last_fullsized":"2024-03-23T17:49:28.780711+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:47:21.509554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.779498+0000","last_change":"2024-03-23T17:48:56.222925+0000","last_active":"2024-03-23T17:49:28.779498+0000","last_peered":"2024-03-23T17:49:28.779498+0000","last_clean":"2024-03-23T17:49:28.779498+0000","last_became_active":"2024-03-23T17:48:56.221504+0000","last_became_peered":"2024-03-23T17:48:56.221504+0000","last_unstale":"2024-03-23T17:49:28.779498+0000","last_undegraded":"2024-03-23T17:49:28.779498+0000","last_fullsized":"2024-03-23T17:49:28.779498+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:14:34.327705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.922478+0000","last_change":"2024-03-23T17:48:56.230530+0000","last_active":"2024-03-23T17:49:28.922478+0000","last_peered":"2024-03-23T17:49:28.922478+0000","last_clean":"2024-03-23T17:49:28.922478+0000","last_became_active":"2024-03-23T17:48:56.230224+0000","last_became_peered":"2024-03-23T17:48:56.230224+0000","last_unstale":"2024-03-23T17:49:28.922478+0000","last_undegraded":"2024-03-23T17:49:28.922478+0000","last_fullsized":"2024-03-23T17:49:28.922478+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:10:28.031956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.781514+0000","last_change":"2024-03-23T17:48:56.229673+0000","last_active":"2024-03-23T17:49:28.781514+0000","last_peered":"2024-03-23T17:49:28.781514+0000","last_clean":"2024-03-23T17:49:28.781514+0000","last_became_active":"2024-03-23T17:48:56.229000+0000","last_became_peered":"2024-03-23T17:48:56.229000+0000","last_unstale":"2024-03-23T17:49:28.781514+0000","last_undegraded":"2024-03-23T17:49:28.781514+0000","last_fullsized":"2024-03-23T17:49:28.781514+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:23:28.057256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.781035+0000","last_change":"2024-03-23T17:48:56.224508+0000","last_active":"2024-03-23T17:49:28.781035+0000","last_peered":"2024-03-23T17:49:28.781035+0000","last_clean":"2024-03-23T17:49:28.781035+0000","last_became_active":"2024-03-23T17:48:56.224172+0000","last_became_peered":"2024-03-23T17:48:56.224172+0000","last_unstale":"2024-03-23T17:49:28.781035+0000","last_undegraded":"2024-03-23T17:49:28.781035+0000","last_fullsized":"2024-03-23T17:49:28.781035+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:03:57.063472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.784254+0000","last_change":"2024-03-23T17:48:56.234978+0000","last_active":"2024-03-23T17:49:28.784254+0000","last_peered":"2024-03-23T17:49:28.784254+0000","last_clean":"2024-03-23T17:49:28.784254+0000","last_became_active":"2024-03-23T17:48:56.234641+0000","last_became_peered":"2024-03-23T17:48:56.234641+0000","last_unstale":"2024-03-23T17:49:28.784254+0000","last_undegraded":"2024-03-23T17:49:28.784254+0000","last_fullsized":"2024-03-23T17:49:28.784254+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:34:30.745991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.777608+0000","last_change":"2024-03-23T17:48:56.218534+0000","last_active":"2024-03-23T17:49:28.777608+0000","last_peered":"2024-03-23T17:49:28.777608+0000","last_clean":"2024-03-23T17:49:28.777608+0000","last_became_active":"2024-03-23T17:48:56.217424+0000","last_became_peered":"2024-03-23T17:48:56.217424+0000","last_unstale":"2024-03-23T17:49:28.777608+0000","last_undegraded":"2024-03-23T17:49:28.777608+0000","last_fullsized":"2024-03-23T17:49:28.777608+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:32:05.618057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.783444+0000","last_change":"2024-03-23T17:48:56.235525+0000","last_active":"2024-03-23T17:49:28.783444+0000","last_peered":"2024-03-23T17:49:28.783444+0000","last_clean":"2024-03-23T17:49:28.783444+0000","last_became_active":"2024-03-23T17:48:56.234591+0000","last_became_peered":"2024-03-23T17:48:56.234591+0000","last_unstale":"2024-03-23T17:49:28.783444+0000","last_undegraded":"2024-03-23T17:49:28.783444+0000","last_fullsized":"2024-03-23T17:49:28.783444+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:19:38.381878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.783446+0000","last_change":"2024-03-23T17:48:56.243137+0000","last_active":"2024-03-23T17:49:28.783446+0000","last_peered":"2024-03-23T17:49:28.783446+0000","last_clean":"2024-03-23T17:49:28.783446+0000","last_became_active":"2024-03-23T17:48:56.242516+0000","last_became_peered":"2024-03-23T17:48:56.242516+0000","last_unstale":"2024-03-23T17:49:28.783446+0000","last_undegraded":"2024-03-23T17:49:28.783446+0000","last_fullsized":"2024-03-23T17:49:28.783446+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:54:59.849467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.922041+0000","last_change":"2024-03-23T17:48:56.236026+0000","last_active":"2024-03-23T17:49:28.922041+0000","last_peered":"2024-03-23T17:49:28.922041+0000","last_clean":"2024-03-23T17:49:28.922041+0000","last_became_active":"2024-03-23T17:48:56.235645+0000","last_became_peered":"2024-03-23T17:48:56.235645+0000","last_unstale":"2024-03-23T17:49:28.922041+0000","last_undegraded":"2024-03-23T17:49:28.922041+0000","last_fullsized":"2024-03-23T17:49:28.922041+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:47:01.076737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.782865+0000","last_change":"2024-03-23T17:48:56.239689+0000","last_active":"2024-03-23T17:49:28.782865+0000","last_peered":"2024-03-23T17:49:28.782865+0000","last_clean":"2024-03-23T17:49:28.782865+0000","last_became_active":"2024-03-23T17:48:56.239331+0000","last_became_peered":"2024-03-23T17:48:56.239331+0000","last_unstale":"2024-03-23T17:49:28.782865+0000","last_undegraded":"2024-03-23T17:49:28.782865+0000","last_fullsized":"2024-03-23T17:49:28.782865+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:21:15.287627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.777987+0000","last_change":"2024-03-23T17:48:56.229934+0000","last_active":"2024-03-23T17:49:28.777987+0000","last_peered":"2024-03-23T17:49:28.777987+0000","last_clean":"2024-03-23T17:49:28.777987+0000","last_became_active":"2024-03-23T17:48:56.229244+0000","last_became_peered":"2024-03-23T17:48:56.229244+0000","last_unstale":"2024-03-23T17:49:28.777987+0000","last_undegraded":"2024-03-23T17:49:28.777987+0000","last_fullsized":"2024-03-23T17:49:28.777987+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:02:40.254799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.784936+0000","last_change":"2024-03-23T17:48:56.233786+0000","last_active":"2024-03-23T17:49:28.784936+0000","last_peered":"2024-03-23T17:49:28.784936+0000","last_clean":"2024-03-23T17:49:28.784936+0000","last_became_active":"2024-03-23T17:48:56.233137+0000","last_became_peered":"2024-03-23T17:48:56.233137+0000","last_unstale":"2024-03-23T17:49:28.784936+0000","last_undegraded":"2024-03-23T17:49:28.784936+0000","last_fullsized":"2024-03-23T17:49:28.784936+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:33:35.202788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'13","reported_seq":62,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-23T17:49:35.753377+0000","last_change":"2024-03-23T17:48:55.712184+0000","last_active":"2024-03-23T17:49:35.753377+0000","last_peered":"2024-03-23T17:49:35.753377+0000","last_clean":"2024-03-23T17:49:35.753377+0000","last_became_active":"2024-03-23T17:48:55.711860+0000","last_became_peered":"2024-03-23T17:48:55.711860+0000","last_unstale":"2024-03-23T17:49:35.753377+0000","last_undegraded":"2024-03-23T17:49:35.753377+0000","last_fullsized":"2024-03-23T17:49:35.753377+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:59:01.133652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":35,"num_read_kb":35,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-23T17:49:29.789705+0000","last_change":"2024-03-23T17:49:29.789057+0000","last_active":"2024-03-23T17:49:29.789705+0000","last_peered":"2024-03-23T17:49:29.789705+0000","last_clean":"2024-03-23T17:49:29.789705+0000","last_became_active":"2024-03-23T17:49:29.788796+0000","last_became_peered":"2024-03-23T17:49:29.788796+0000","last_unstale":"2024-03-23T17:49:29.789705+0000","last_undegraded":"2024-03-23T17:49:29.789705+0000","last_fullsized":"2024-03-23T17:49:29.789705+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:30:53.062926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.780493+0000","last_change":"2024-03-23T17:48:55.725351+0000","last_active":"2024-03-23T17:49:28.780493+0000","last_peered":"2024-03-23T17:49:28.780493+0000","last_clean":"2024-03-23T17:49:28.780493+0000","last_became_active":"2024-03-23T17:48:55.724566+0000","last_became_peered":"2024-03-23T17:48:55.724566+0000","last_unstale":"2024-03-23T17:49:28.780493+0000","last_undegraded":"2024-03-23T17:49:28.780493+0000","last_fullsized":"2024-03-23T17:49:28.780493+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:58:21.088166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.784151+0000","last_change":"2024-03-23T17:48:56.243526+0000","last_active":"2024-03-23T17:49:28.784151+0000","last_peered":"2024-03-23T17:49:28.784151+0000","last_clean":"2024-03-23T17:49:28.784151+0000","last_became_active":"2024-03-23T17:48:56.242468+0000","last_became_peered":"2024-03-23T17:48:56.242468+0000","last_unstale":"2024-03-23T17:49:28.784151+0000","last_undegraded":"2024-03-23T17:49:28.784151+0000","last_fullsized":"2024-03-23T17:49:28.784151+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:16:50.392770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.781056+0000","last_change":"2024-03-23T17:48:55.720237+0000","last_active":"2024-03-23T17:49:28.781056+0000","last_peered":"2024-03-23T17:49:28.781056+0000","last_clean":"2024-03-23T17:49:28.781056+0000","last_became_active":"2024-03-23T17:48:55.719843+0000","last_became_peered":"2024-03-23T17:48:55.719843+0000","last_unstale":"2024-03-23T17:49:28.781056+0000","last_undegraded":"2024-03-23T17:49:28.781056+0000","last_fullsized":"2024-03-23T17:49:28.781056+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:07:26.973511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.922177+0000","last_change":"2024-03-23T17:48:56.235642+0000","last_active":"2024-03-23T17:49:28.922177+0000","last_peered":"2024-03-23T17:49:28.922177+0000","last_clean":"2024-03-23T17:49:28.922177+0000","last_became_active":"2024-03-23T17:48:56.235326+0000","last_became_peered":"2024-03-23T17:48:56.235326+0000","last_unstale":"2024-03-23T17:49:28.922177+0000","last_undegraded":"2024-03-23T17:49:28.922177+0000","last_fullsized":"2024-03-23T17:49:28.922177+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:21:59.934620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.781362+0000","last_change":"2024-03-23T17:48:55.718579+0000","last_active":"2024-03-23T17:49:28.781362+0000","last_peered":"2024-03-23T17:49:28.781362+0000","last_clean":"2024-03-23T17:49:28.781362+0000","last_became_active":"2024-03-23T17:48:55.718220+0000","last_became_peered":"2024-03-23T17:48:55.718220+0000","last_unstale":"2024-03-23T17:49:28.781362+0000","last_undegraded":"2024-03-23T17:49:28.781362+0000","last_fullsized":"2024-03-23T17:49:28.781362+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:37:56.417367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.785628+0000","last_change":"2024-03-23T17:48:56.224842+0000","last_active":"2024-03-23T17:49:28.785628+0000","last_peered":"2024-03-23T17:49:28.785628+0000","last_clean":"2024-03-23T17:49:28.785628+0000","last_became_active":"2024-03-23T17:48:56.224427+0000","last_became_peered":"2024-03-23T17:48:56.224427+0000","last_unstale":"2024-03-23T17:49:28.785628+0000","last_undegraded":"2024-03-23T17:49:28.785628+0000","last_fullsized":"2024-03-23T17:49:28.785628+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:16:22.684369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"80'5","reported_seq":58,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-23T17:49:35.712872+0000","last_change":"2024-03-23T17:48:55.713996+0000","last_active":"2024-03-23T17:49:35.712872+0000","last_peered":"2024-03-23T17:49:35.712872+0000","last_clean":"2024-03-23T17:49:35.712872+0000","last_became_active":"2024-03-23T17:48:55.713647+0000","last_became_peered":"2024-03-23T17:48:55.713647+0000","last_unstale":"2024-03-23T17:49:35.712872+0000","last_undegraded":"2024-03-23T17:49:35.712872+0000","last_fullsized":"2024-03-23T17:49:35.712872+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:12:22.251533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":39,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.781148+0000","last_change":"2024-03-23T17:48:56.219281+0000","last_active":"2024-03-23T17:49:28.781148+0000","last_peered":"2024-03-23T17:49:28.781148+0000","last_clean":"2024-03-23T17:49:28.781148+0000","last_became_active":"2024-03-23T17:48:56.218924+0000","last_became_peered":"2024-03-23T17:48:56.218924+0000","last_unstale":"2024-03-23T17:49:28.781148+0000","last_undegraded":"2024-03-23T17:49:28.781148+0000","last_fullsized":"2024-03-23T17:49:28.781148+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:49:04.793088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.781333+0000","last_change":"2024-03-23T17:48:55.718131+0000","last_active":"2024-03-23T17:49:28.781333+0000","last_peered":"2024-03-23T17:49:28.781333+0000","last_clean":"2024-03-23T17:49:28.781333+0000","last_became_active":"2024-03-23T17:48:55.717760+0000","last_became_peered":"2024-03-23T17:48:55.717760+0000","last_unstale":"2024-03-23T17:49:28.781333+0000","last_undegraded":"2024-03-23T17:49:28.781333+0000","last_fullsized":"2024-03-23T17:49:28.781333+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:16:54.355844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.784309+0000","last_change":"2024-03-23T17:48:56.225965+0000","last_active":"2024-03-23T17:49:28.784309+0000","last_peered":"2024-03-23T17:49:28.784309+0000","last_clean":"2024-03-23T17:49:28.784309+0000","last_became_active":"2024-03-23T17:48:56.225101+0000","last_became_peered":"2024-03-23T17:48:56.225101+0000","last_unstale":"2024-03-23T17:49:28.784309+0000","last_undegraded":"2024-03-23T17:49:28.784309+0000","last_fullsized":"2024-03-23T17:49:28.784309+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:52:14.302306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.776847+0000","last_change":"2024-03-23T17:48:55.722289+0000","last_active":"2024-03-23T17:49:28.776847+0000","last_peered":"2024-03-23T17:49:28.776847+0000","last_clean":"2024-03-23T17:49:28.776847+0000","last_became_active":"2024-03-23T17:48:55.721927+0000","last_became_peered":"2024-03-23T17:48:55.721927+0000","last_unstale":"2024-03-23T17:49:28.776847+0000","last_undegraded":"2024-03-23T17:49:28.776847+0000","last_fullsized":"2024-03-23T17:49:28.776847+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:20:08.354897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.779465+0000","last_change":"2024-03-23T17:48:56.232231+0000","last_active":"2024-03-23T17:49:28.779465+0000","last_peered":"2024-03-23T17:49:28.779465+0000","last_clean":"2024-03-23T17:49:28.779465+0000","last_became_active":"2024-03-23T17:48:56.227335+0000","last_became_peered":"2024-03-23T17:48:56.227335+0000","last_unstale":"2024-03-23T17:49:28.779465+0000","last_undegraded":"2024-03-23T17:49:28.779465+0000","last_fullsized":"2024-03-23T17:49:28.779465+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:49:01.460775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.785071+0000","last_change":"2024-03-23T17:48:55.717615+0000","last_active":"2024-03-23T17:49:28.785071+0000","last_peered":"2024-03-23T17:49:28.785071+0000","last_clean":"2024-03-23T17:49:28.785071+0000","last_became_active":"2024-03-23T17:48:55.717266+0000","last_became_peered":"2024-03-23T17:48:55.717266+0000","last_unstale":"2024-03-23T17:49:28.785071+0000","last_undegraded":"2024-03-23T17:49:28.785071+0000","last_fullsized":"2024-03-23T17:49:28.785071+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:40:07.988182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.776405+0000","last_change":"2024-03-23T17:48:56.229787+0000","last_active":"2024-03-23T17:49:28.776405+0000","last_peered":"2024-03-23T17:49:28.776405+0000","last_clean":"2024-03-23T17:49:28.776405+0000","last_became_active":"2024-03-23T17:48:56.229471+0000","last_became_peered":"2024-03-23T17:48:56.229471+0000","last_unstale":"2024-03-23T17:49:28.776405+0000","last_undegraded":"2024-03-23T17:49:28.776405+0000","last_fullsized":"2024-03-23T17:49:28.776405+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:44:09.358809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'4","reported_seq":93,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-23T17:49:38.717008+0000","last_change":"2024-03-23T17:48:55.717130+0000","last_active":"2024-03-23T17:49:38.717008+0000","last_peered":"2024-03-23T17:49:38.717008+0000","last_clean":"2024-03-23T17:49:38.717008+0000","last_became_active":"2024-03-23T17:48:55.716733+0000","last_became_peered":"2024-03-23T17:48:55.716733+0000","last_unstale":"2024-03-23T17:49:38.717008+0000","last_undegraded":"2024-03-23T17:49:38.717008+0000","last_fullsized":"2024-03-23T17:49:38.717008+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:22:43.229694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":75,"num_read_kb":75,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.922446+0000","last_change":"2024-03-23T17:48:56.235565+0000","last_active":"2024-03-23T17:49:28.922446+0000","last_peered":"2024-03-23T17:49:28.922446+0000","last_clean":"2024-03-23T17:49:28.922446+0000","last_became_active":"2024-03-23T17:48:56.235267+0000","last_became_peered":"2024-03-23T17:48:56.235267+0000","last_unstale":"2024-03-23T17:49:28.922446+0000","last_undegraded":"2024-03-23T17:49:28.922446+0000","last_fullsized":"2024-03-23T17:49:28.922446+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:38:45.906545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.780900+0000","last_change":"2024-03-23T17:48:55.719096+0000","last_active":"2024-03-23T17:49:28.780900+0000","last_peered":"2024-03-23T17:49:28.780900+0000","last_clean":"2024-03-23T17:49:28.780900+0000","last_became_active":"2024-03-23T17:48:55.718721+0000","last_became_peered":"2024-03-23T17:48:55.718721+0000","last_unstale":"2024-03-23T17:49:28.780900+0000","last_undegraded":"2024-03-23T17:49:28.780900+0000","last_fullsized":"2024-03-23T17:49:28.780900+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:43:05.532263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.784108+0000","last_change":"2024-03-23T17:48:56.235549+0000","last_active":"2024-03-23T17:49:28.784108+0000","last_peered":"2024-03-23T17:49:28.784108+0000","last_clean":"2024-03-23T17:49:28.784108+0000","last_became_active":"2024-03-23T17:48:56.234572+0000","last_became_peered":"2024-03-23T17:48:56.234572+0000","last_unstale":"2024-03-23T17:49:28.784108+0000","last_undegraded":"2024-03-23T17:49:28.784108+0000","last_fullsized":"2024-03-23T17:49:28.784108+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:19:58.639181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.785732+0000","last_change":"2024-03-23T17:48:55.724688+0000","last_active":"2024-03-23T17:49:28.785732+0000","last_peered":"2024-03-23T17:49:28.785732+0000","last_clean":"2024-03-23T17:49:28.785732+0000","last_became_active":"2024-03-23T17:48:55.724385+0000","last_became_peered":"2024-03-23T17:48:55.724385+0000","last_unstale":"2024-03-23T17:49:28.785732+0000","last_undegraded":"2024-03-23T17:49:28.785732+0000","last_fullsized":"2024-03-23T17:49:28.785732+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:17:54.338059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.922287+0000","last_change":"2024-03-23T17:48:56.242983+0000","last_active":"2024-03-23T17:49:28.922287+0000","last_peered":"2024-03-23T17:49:28.922287+0000","last_clean":"2024-03-23T17:49:28.922287+0000","last_became_active":"2024-03-23T17:48:56.242596+0000","last_became_peered":"2024-03-23T17:48:56.242596+0000","last_unstale":"2024-03-23T17:49:28.922287+0000","last_undegraded":"2024-03-23T17:49:28.922287+0000","last_fullsized":"2024-03-23T17:49:28.922287+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:55:51.971834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'29","reported_seq":128,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-23T17:49:38.730186+0000","last_change":"2024-03-23T17:48:55.717631+0000","last_active":"2024-03-23T17:49:38.730186+0000","last_peered":"2024-03-23T17:49:38.730186+0000","last_clean":"2024-03-23T17:49:38.730186+0000","last_became_active":"2024-03-23T17:48:55.717082+0000","last_became_peered":"2024-03-23T17:48:55.717082+0000","last_unstale":"2024-03-23T17:49:38.730186+0000","last_undegraded":"2024-03-23T17:49:38.730186+0000","last_fullsized":"2024-03-23T17:49:38.730186+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:36:25.323269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59418,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":85,"num_read_kb":100,"num_write":31,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.780792+0000","last_change":"2024-03-23T17:48:56.226174+0000","last_active":"2024-03-23T17:49:28.780792+0000","last_peered":"2024-03-23T17:49:28.780792+0000","last_clean":"2024-03-23T17:49:28.780792+0000","last_became_active":"2024-03-23T17:48:56.225834+0000","last_became_peered":"2024-03-23T17:48:56.225834+0000","last_unstale":"2024-03-23T17:49:28.780792+0000","last_undegraded":"2024-03-23T17:49:28.780792+0000","last_fullsized":"2024-03-23T17:49:28.780792+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:50:16.793071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-23T17:49:29.804494+0000","last_change":"2024-03-23T17:49:29.804099+0000","last_active":"2024-03-23T17:49:29.804494+0000","last_peered":"2024-03-23T17:49:29.804494+0000","last_clean":"2024-03-23T17:49:29.804494+0000","last_became_active":"2024-03-23T17:49:29.787483+0000","last_became_peered":"2024-03-23T17:49:29.787483+0000","last_unstale":"2024-03-23T17:49:29.804494+0000","last_undegraded":"2024-03-23T17:49:29.804494+0000","last_fullsized":"2024-03-23T17:49:29.804494+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:13:16.463738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.776667+0000","last_change":"2024-03-23T17:48:56.228459+0000","last_active":"2024-03-23T17:49:28.776667+0000","last_peered":"2024-03-23T17:49:28.776667+0000","last_clean":"2024-03-23T17:49:28.776667+0000","last_became_active":"2024-03-23T17:48:56.228179+0000","last_became_peered":"2024-03-23T17:48:56.228179+0000","last_unstale":"2024-03-23T17:49:28.776667+0000","last_undegraded":"2024-03-23T17:49:28.776667+0000","last_fullsized":"2024-03-23T17:49:28.776667+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:28:41.613432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.785318+0000","last_change":"2024-03-23T17:48:55.721533+0000","last_active":"2024-03-23T17:49:28.785318+0000","last_peered":"2024-03-23T17:49:28.785318+0000","last_clean":"2024-03-23T17:49:28.785318+0000","last_became_active":"2024-03-23T17:48:55.721100+0000","last_became_peered":"2024-03-23T17:48:55.721100+0000","last_unstale":"2024-03-23T17:49:28.785318+0000","last_undegraded":"2024-03-23T17:49:28.785318+0000","last_fullsized":"2024-03-23T17:49:28.785318+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:56:58.112649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-23T17:49:30.003965+0000","last_change":"2024-03-23T17:49:30.003965+0000","last_active":"2024-03-23T17:49:30.003965+0000","last_peered":"2024-03-23T17:49:30.003965+0000","last_clean":"2024-03-23T17:49:30.003965+0000","last_became_active":"2024-03-23T17:49:30.003643+0000","last_became_peered":"2024-03-23T17:49:30.003643+0000","last_unstale":"2024-03-23T17:49:30.003965+0000","last_undegraded":"2024-03-23T17:49:30.003965+0000","last_fullsized":"2024-03-23T17:49:30.003965+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:47:39.892782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'5","reported_seq":57,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-23T17:49:37.757640+0000","last_change":"2024-03-23T17:48:55.715205+0000","last_active":"2024-03-23T17:49:37.757640+0000","last_peered":"2024-03-23T17:49:37.757640+0000","last_clean":"2024-03-23T17:49:37.757640+0000","last_became_active":"2024-03-23T17:48:55.714810+0000","last_became_peered":"2024-03-23T17:48:55.714810+0000","last_unstale":"2024-03-23T17:49:37.757640+0000","last_undegraded":"2024-03-23T17:49:37.757640+0000","last_fullsized":"2024-03-23T17:49:37.757640+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:40:08.768998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":39,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-23T17:49:29.783025+0000","last_change":"2024-03-23T17:49:29.783025+0000","last_active":"2024-03-23T17:49:29.783025+0000","last_peered":"2024-03-23T17:49:29.783025+0000","last_clean":"2024-03-23T17:49:29.783025+0000","last_became_active":"2024-03-23T17:49:29.782541+0000","last_became_peered":"2024-03-23T17:49:29.782541+0000","last_unstale":"2024-03-23T17:49:29.783025+0000","last_undegraded":"2024-03-23T17:49:29.783025+0000","last_fullsized":"2024-03-23T17:49:29.783025+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:50:29.754074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":59,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-23T17:49:30.616960+0000","last_change":"2024-03-23T17:49:30.616960+0000","last_active":"2024-03-23T17:49:30.616960+0000","last_peered":"2024-03-23T17:49:30.616960+0000","last_clean":"2024-03-23T17:49:30.616960+0000","last_became_active":"2024-03-23T17:49:30.604464+0000","last_became_peered":"2024-03-23T17:49:30.604464+0000","last_unstale":"2024-03-23T17:49:30.616960+0000","last_undegraded":"2024-03-23T17:49:30.616960+0000","last_fullsized":"2024-03-23T17:49:30.616960+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:24:13.862841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.785079+0000","last_change":"2024-03-23T17:48:56.225624+0000","last_active":"2024-03-23T17:49:28.785079+0000","last_peered":"2024-03-23T17:49:28.785079+0000","last_clean":"2024-03-23T17:49:28.785079+0000","last_became_active":"2024-03-23T17:48:56.223798+0000","last_became_peered":"2024-03-23T17:48:56.223798+0000","last_unstale":"2024-03-23T17:49:28.785079+0000","last_undegraded":"2024-03-23T17:49:28.785079+0000","last_fullsized":"2024-03-23T17:49:28.785079+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:00:20.547553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.781323+0000","last_change":"2024-03-23T17:48:55.717792+0000","last_active":"2024-03-23T17:49:28.781323+0000","last_peered":"2024-03-23T17:49:28.781323+0000","last_clean":"2024-03-23T17:49:28.781323+0000","last_became_active":"2024-03-23T17:48:55.717479+0000","last_became_peered":"2024-03-23T17:48:55.717479+0000","last_unstale":"2024-03-23T17:49:28.781323+0000","last_undegraded":"2024-03-23T17:49:28.781323+0000","last_fullsized":"2024-03-23T17:49:28.781323+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:49:43.483785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.782572+0000","last_change":"2024-03-23T17:48:56.219135+0000","last_active":"2024-03-23T17:49:28.782572+0000","last_peered":"2024-03-23T17:49:28.782572+0000","last_clean":"2024-03-23T17:49:28.782572+0000","last_became_active":"2024-03-23T17:48:56.218824+0000","last_became_peered":"2024-03-23T17:48:56.218824+0000","last_unstale":"2024-03-23T17:49:28.782572+0000","last_undegraded":"2024-03-23T17:49:28.782572+0000","last_fullsized":"2024-03-23T17:49:28.782572+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:20:35.190781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.776774+0000","last_change":"2024-03-23T17:48:55.717530+0000","last_active":"2024-03-23T17:49:28.776774+0000","last_peered":"2024-03-23T17:49:28.776774+0000","last_clean":"2024-03-23T17:49:28.776774+0000","last_became_active":"2024-03-23T17:48:55.717213+0000","last_became_peered":"2024-03-23T17:48:55.717213+0000","last_unstale":"2024-03-23T17:49:28.776774+0000","last_undegraded":"2024-03-23T17:49:28.776774+0000","last_fullsized":"2024-03-23T17:49:28.776774+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:40:09.803021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'1","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.781617+0000","last_change":"2024-03-23T17:48:56.222237+0000","last_active":"2024-03-23T17:49:28.781617+0000","last_peered":"2024-03-23T17:49:28.781617+0000","last_clean":"2024-03-23T17:49:28.781617+0000","last_became_active":"2024-03-23T17:48:56.221916+0000","last_became_peered":"2024-03-23T17:48:56.221916+0000","last_unstale":"2024-03-23T17:49:28.781617+0000","last_undegraded":"2024-03-23T17:49:28.781617+0000","last_fullsized":"2024-03-23T17:49:28.781617+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:35:59.213482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'5","reported_seq":53,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-23T17:49:34.752427+0000","last_change":"2024-03-23T17:48:55.718143+0000","last_active":"2024-03-23T17:49:34.752427+0000","last_peered":"2024-03-23T17:49:34.752427+0000","last_clean":"2024-03-23T17:49:34.752427+0000","last_became_active":"2024-03-23T17:48:55.717834+0000","last_became_peered":"2024-03-23T17:48:55.717834+0000","last_unstale":"2024-03-23T17:49:34.752427+0000","last_undegraded":"2024-03-23T17:49:34.752427+0000","last_fullsized":"2024-03-23T17:49:34.752427+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:29:08.879636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.781385+0000","last_change":"2024-03-23T17:48:56.237136+0000","last_active":"2024-03-23T17:49:28.781385+0000","last_peered":"2024-03-23T17:49:28.781385+0000","last_clean":"2024-03-23T17:49:28.781385+0000","last_became_active":"2024-03-23T17:48:56.236795+0000","last_became_peered":"2024-03-23T17:48:56.236795+0000","last_unstale":"2024-03-23T17:49:28.781385+0000","last_undegraded":"2024-03-23T17:49:28.781385+0000","last_fullsized":"2024-03-23T17:49:28.781385+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:15:31.356382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-23T17:49:29.789433+0000","last_change":"2024-03-23T17:49:29.789433+0000","last_active":"2024-03-23T17:49:29.789433+0000","last_peered":"2024-03-23T17:49:29.789433+0000","last_clean":"2024-03-23T17:49:29.789433+0000","last_became_active":"2024-03-23T17:49:29.788956+0000","last_became_peered":"2024-03-23T17:49:29.788956+0000","last_unstale":"2024-03-23T17:49:29.789433+0000","last_undegraded":"2024-03-23T17:49:29.789433+0000","last_fullsized":"2024-03-23T17:49:29.789433+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:43:30.929796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"80'5","reported_seq":56,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-23T17:49:37.757029+0000","last_change":"2024-03-23T17:48:55.722968+0000","last_active":"2024-03-23T17:49:37.757029+0000","last_peered":"2024-03-23T17:49:37.757029+0000","last_clean":"2024-03-23T17:49:37.757029+0000","last_became_active":"2024-03-23T17:48:55.722578+0000","last_became_peered":"2024-03-23T17:48:55.722578+0000","last_unstale":"2024-03-23T17:49:37.757029+0000","last_undegraded":"2024-03-23T17:49:37.757029+0000","last_fullsized":"2024-03-23T17:49:37.757029+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:45:42.344283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":37,"num_read_kb":37,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.779906+0000","last_change":"2024-03-23T17:48:56.228000+0000","last_active":"2024-03-23T17:49:28.779906+0000","last_peered":"2024-03-23T17:49:28.779906+0000","last_clean":"2024-03-23T17:49:28.779906+0000","last_became_active":"2024-03-23T17:48:56.225645+0000","last_became_peered":"2024-03-23T17:48:56.225645+0000","last_unstale":"2024-03-23T17:49:28.779906+0000","last_undegraded":"2024-03-23T17:49:28.779906+0000","last_fullsized":"2024-03-23T17:49:28.779906+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:44:25.197761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.784417+0000","last_change":"2024-03-23T17:48:55.718307+0000","last_active":"2024-03-23T17:49:28.784417+0000","last_peered":"2024-03-23T17:49:28.784417+0000","last_clean":"2024-03-23T17:49:28.784417+0000","last_became_active":"2024-03-23T17:48:55.717839+0000","last_became_peered":"2024-03-23T17:48:55.717839+0000","last_unstale":"2024-03-23T17:49:28.784417+0000","last_undegraded":"2024-03-23T17:49:28.784417+0000","last_fullsized":"2024-03-23T17:49:28.784417+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:11:56.053940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":335,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.780763+0000","last_change":"2024-03-23T17:48:01.618019+0000","last_active":"2024-03-23T17:49:28.780763+0000","last_peered":"2024-03-23T17:49:28.780763+0000","last_clean":"2024-03-23T17:49:28.780763+0000","last_became_active":"2024-03-23T17:48:01.617553+0000","last_became_peered":"2024-03-23T17:48:01.617553+0000","last_unstale":"2024-03-23T17:49:28.780763+0000","last_undegraded":"2024-03-23T17:49:28.780763+0000","last_fullsized":"2024-03-23T17:49:28.780763+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:45:26.849029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:45:26.849029+0000","last_clean_scrub_stamp":"2024-03-23T17:45:26.849029+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:05:44.823364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.776693+0000","last_change":"2024-03-23T17:48:56.224285+0000","last_active":"2024-03-23T17:49:28.776693+0000","last_peered":"2024-03-23T17:49:28.776693+0000","last_clean":"2024-03-23T17:49:28.776693+0000","last_became_active":"2024-03-23T17:48:56.220943+0000","last_became_peered":"2024-03-23T17:48:56.220943+0000","last_unstale":"2024-03-23T17:49:28.776693+0000","last_undegraded":"2024-03-23T17:49:28.776693+0000","last_fullsized":"2024-03-23T17:49:28.776693+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:36:05.458243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.780994+0000","last_change":"2024-03-23T17:48:55.715877+0000","last_active":"2024-03-23T17:49:28.780994+0000","last_peered":"2024-03-23T17:49:28.780994+0000","last_clean":"2024-03-23T17:49:28.780994+0000","last_became_active":"2024-03-23T17:48:55.715550+0000","last_became_peered":"2024-03-23T17:48:55.715550+0000","last_unstale":"2024-03-23T17:49:28.780994+0000","last_undegraded":"2024-03-23T17:49:28.780994+0000","last_fullsized":"2024-03-23T17:49:28.780994+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:31:57.668495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.780911+0000","last_change":"2024-03-23T17:48:55.724618+0000","last_active":"2024-03-23T17:49:28.780911+0000","last_peered":"2024-03-23T17:49:28.780911+0000","last_clean":"2024-03-23T17:49:28.780911+0000","last_became_active":"2024-03-23T17:48:55.724278+0000","last_became_peered":"2024-03-23T17:48:55.724278+0000","last_unstale":"2024-03-23T17:49:28.780911+0000","last_undegraded":"2024-03-23T17:49:28.780911+0000","last_fullsized":"2024-03-23T17:49:28.780911+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:27:05.065115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.776532+0000","last_change":"2024-03-23T17:48:56.216141+0000","last_active":"2024-03-23T17:49:28.776532+0000","last_peered":"2024-03-23T17:49:28.776532+0000","last_clean":"2024-03-23T17:49:28.776532+0000","last_became_active":"2024-03-23T17:48:56.215653+0000","last_became_peered":"2024-03-23T17:48:56.215653+0000","last_unstale":"2024-03-23T17:49:28.776532+0000","last_undegraded":"2024-03-23T17:49:28.776532+0000","last_fullsized":"2024-03-23T17:49:28.776532+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:02:19.611056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'9","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-23T17:49:29.794454+0000","last_change":"2024-03-23T17:49:29.794454+0000","last_active":"2024-03-23T17:49:29.794454+0000","last_peered":"2024-03-23T17:49:29.794454+0000","last_clean":"2024-03-23T17:49:29.794454+0000","last_became_active":"2024-03-23T17:49:29.787238+0000","last_became_peered":"2024-03-23T17:49:29.787238+0000","last_unstale":"2024-03-23T17:49:29.794454+0000","last_undegraded":"2024-03-23T17:49:29.794454+0000","last_fullsized":"2024-03-23T17:49:29.794454+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:33:34.917283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.780480+0000","last_change":"2024-03-23T17:48:56.228439+0000","last_active":"2024-03-23T17:49:28.780480+0000","last_peered":"2024-03-23T17:49:28.780480+0000","last_clean":"2024-03-23T17:49:28.780480+0000","last_became_active":"2024-03-23T17:48:56.226677+0000","last_became_peered":"2024-03-23T17:48:56.226677+0000","last_unstale":"2024-03-23T17:49:28.780480+0000","last_undegraded":"2024-03-23T17:49:28.780480+0000","last_fullsized":"2024-03-23T17:49:28.780480+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:46:00.063266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.784329+0000","last_change":"2024-03-23T17:48:55.724872+0000","last_active":"2024-03-23T17:49:28.784329+0000","last_peered":"2024-03-23T17:49:28.784329+0000","last_clean":"2024-03-23T17:49:28.784329+0000","last_became_active":"2024-03-23T17:48:55.724468+0000","last_became_peered":"2024-03-23T17:48:55.724468+0000","last_unstale":"2024-03-23T17:49:28.784329+0000","last_undegraded":"2024-03-23T17:49:28.784329+0000","last_fullsized":"2024-03-23T17:49:28.784329+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:28:34.918056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.783623+0000","last_change":"2024-03-23T17:48:56.233730+0000","last_active":"2024-03-23T17:49:28.783623+0000","last_peered":"2024-03-23T17:49:28.783623+0000","last_clean":"2024-03-23T17:49:28.783623+0000","last_became_active":"2024-03-23T17:48:56.233189+0000","last_became_peered":"2024-03-23T17:48:56.233189+0000","last_unstale":"2024-03-23T17:49:28.783623+0000","last_undegraded":"2024-03-23T17:49:28.783623+0000","last_fullsized":"2024-03-23T17:49:28.783623+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:20:30.764344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.922070+0000","last_change":"2024-03-23T17:48:55.724633+0000","last_active":"2024-03-23T17:49:28.922070+0000","last_peered":"2024-03-23T17:49:28.922070+0000","last_clean":"2024-03-23T17:49:28.922070+0000","last_became_active":"2024-03-23T17:48:55.724301+0000","last_became_peered":"2024-03-23T17:48:55.724301+0000","last_unstale":"2024-03-23T17:49:28.922070+0000","last_undegraded":"2024-03-23T17:49:28.922070+0000","last_fullsized":"2024-03-23T17:49:28.922070+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:45:08.392668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.780692+0000","last_change":"2024-03-23T17:48:56.230981+0000","last_active":"2024-03-23T17:49:28.780692+0000","last_peered":"2024-03-23T17:49:28.780692+0000","last_clean":"2024-03-23T17:49:28.780692+0000","last_became_active":"2024-03-23T17:48:56.230644+0000","last_became_peered":"2024-03-23T17:48:56.230644+0000","last_unstale":"2024-03-23T17:49:28.780692+0000","last_undegraded":"2024-03-23T17:49:28.780692+0000","last_fullsized":"2024-03-23T17:49:28.780692+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:37:51.310938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.779873+0000","last_change":"2024-03-23T17:48:55.719112+0000","last_active":"2024-03-23T17:49:28.779873+0000","last_peered":"2024-03-23T17:49:28.779873+0000","last_clean":"2024-03-23T17:49:28.779873+0000","last_became_active":"2024-03-23T17:48:55.718756+0000","last_became_peered":"2024-03-23T17:48:55.718756+0000","last_unstale":"2024-03-23T17:49:28.779873+0000","last_undegraded":"2024-03-23T17:49:28.779873+0000","last_fullsized":"2024-03-23T17:49:28.779873+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:41:22.164706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.781982+0000","last_change":"2024-03-23T17:48:56.240289+0000","last_active":"2024-03-23T17:49:28.781982+0000","last_peered":"2024-03-23T17:49:28.781982+0000","last_clean":"2024-03-23T17:49:28.781982+0000","last_became_active":"2024-03-23T17:48:56.239896+0000","last_became_peered":"2024-03-23T17:48:56.239896+0000","last_unstale":"2024-03-23T17:49:28.781982+0000","last_undegraded":"2024-03-23T17:49:28.781982+0000","last_fullsized":"2024-03-23T17:49:28.781982+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:49:10.505576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'3","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.785515+0000","last_change":"2024-03-23T17:48:55.723330+0000","last_active":"2024-03-23T17:49:28.785515+0000","last_peered":"2024-03-23T17:49:28.785515+0000","last_clean":"2024-03-23T17:49:28.785515+0000","last_became_active":"2024-03-23T17:48:55.722879+0000","last_became_peered":"2024-03-23T17:48:55.722879+0000","last_unstale":"2024-03-23T17:49:28.785515+0000","last_undegraded":"2024-03-23T17:49:28.785515+0000","last_fullsized":"2024-03-23T17:49:28.785515+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:34:47.173411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.779841+0000","last_change":"2024-03-23T17:48:56.237042+0000","last_active":"2024-03-23T17:49:28.779841+0000","last_peered":"2024-03-23T17:49:28.779841+0000","last_clean":"2024-03-23T17:49:28.779841+0000","last_became_active":"2024-03-23T17:48:56.236722+0000","last_became_peered":"2024-03-23T17:48:56.236722+0000","last_unstale":"2024-03-23T17:49:28.779841+0000","last_undegraded":"2024-03-23T17:49:28.779841+0000","last_fullsized":"2024-03-23T17:49:28.779841+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:27:29.656120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.780900+0000","last_change":"2024-03-23T17:48:55.719223+0000","last_active":"2024-03-23T17:49:28.780900+0000","last_peered":"2024-03-23T17:49:28.780900+0000","last_clean":"2024-03-23T17:49:28.780900+0000","last_became_active":"2024-03-23T17:48:55.718864+0000","last_became_peered":"2024-03-23T17:48:55.718864+0000","last_unstale":"2024-03-23T17:49:28.780900+0000","last_undegraded":"2024-03-23T17:49:28.780900+0000","last_fullsized":"2024-03-23T17:49:28.780900+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:52:18.169557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.781086+0000","last_change":"2024-03-23T17:48:56.224885+0000","last_active":"2024-03-23T17:49:28.781086+0000","last_peered":"2024-03-23T17:49:28.781086+0000","last_clean":"2024-03-23T17:49:28.781086+0000","last_became_active":"2024-03-23T17:48:56.224160+0000","last_became_peered":"2024-03-23T17:48:56.224160+0000","last_unstale":"2024-03-23T17:49:28.781086+0000","last_undegraded":"2024-03-23T17:49:28.781086+0000","last_fullsized":"2024-03-23T17:49:28.781086+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:35:34.538313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'5","reported_seq":53,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-23T17:49:33.751651+0000","last_change":"2024-03-23T17:48:55.720518+0000","last_active":"2024-03-23T17:49:33.751651+0000","last_peered":"2024-03-23T17:49:33.751651+0000","last_clean":"2024-03-23T17:49:33.751651+0000","last_became_active":"2024-03-23T17:48:55.720045+0000","last_became_peered":"2024-03-23T17:48:55.720045+0000","last_unstale":"2024-03-23T17:49:33.751651+0000","last_undegraded":"2024-03-23T17:49:33.751651+0000","last_fullsized":"2024-03-23T17:49:33.751651+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:20:54.819406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":34,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.781093+0000","last_change":"2024-03-23T17:48:56.224171+0000","last_active":"2024-03-23T17:49:28.781093+0000","last_peered":"2024-03-23T17:49:28.781093+0000","last_clean":"2024-03-23T17:49:28.781093+0000","last_became_active":"2024-03-23T17:48:56.223737+0000","last_became_peered":"2024-03-23T17:48:56.223737+0000","last_unstale":"2024-03-23T17:49:28.781093+0000","last_undegraded":"2024-03-23T17:49:28.781093+0000","last_fullsized":"2024-03-23T17:49:28.781093+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:40:04.353945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'3","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.785531+0000","last_change":"2024-03-23T17:48:55.721734+0000","last_active":"2024-03-23T17:49:28.785531+0000","last_peered":"2024-03-23T17:49:28.785531+0000","last_clean":"2024-03-23T17:49:28.785531+0000","last_became_active":"2024-03-23T17:48:55.721417+0000","last_became_peered":"2024-03-23T17:48:55.721417+0000","last_unstale":"2024-03-23T17:49:28.785531+0000","last_undegraded":"2024-03-23T17:49:28.785531+0000","last_fullsized":"2024-03-23T17:49:28.785531+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:36:36.842740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'1","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.780817+0000","last_change":"2024-03-23T17:48:56.227211+0000","last_active":"2024-03-23T17:49:28.780817+0000","last_peered":"2024-03-23T17:49:28.780817+0000","last_clean":"2024-03-23T17:49:28.780817+0000","last_became_active":"2024-03-23T17:48:56.226881+0000","last_became_peered":"2024-03-23T17:48:56.226881+0000","last_unstale":"2024-03-23T17:49:28.780817+0000","last_undegraded":"2024-03-23T17:49:28.780817+0000","last_fullsized":"2024-03-23T17:49:28.780817+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:15:02.937666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.776701+0000","last_change":"2024-03-23T17:48:55.728679+0000","last_active":"2024-03-23T17:49:28.776701+0000","last_peered":"2024-03-23T17:49:28.776701+0000","last_clean":"2024-03-23T17:49:28.776701+0000","last_became_active":"2024-03-23T17:48:55.727754+0000","last_became_peered":"2024-03-23T17:48:55.727754+0000","last_unstale":"2024-03-23T17:49:28.776701+0000","last_undegraded":"2024-03-23T17:49:28.776701+0000","last_fullsized":"2024-03-23T17:49:28.776701+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:34:39.720488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.781328+0000","last_change":"2024-03-23T17:48:56.225214+0000","last_active":"2024-03-23T17:49:28.781328+0000","last_peered":"2024-03-23T17:49:28.781328+0000","last_clean":"2024-03-23T17:49:28.781328+0000","last_became_active":"2024-03-23T17:48:56.224635+0000","last_became_peered":"2024-03-23T17:48:56.224635+0000","last_unstale":"2024-03-23T17:49:28.781328+0000","last_undegraded":"2024-03-23T17:49:28.781328+0000","last_fullsized":"2024-03-23T17:49:28.781328+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:57:14.935434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.782109+0000","last_change":"2024-03-23T17:48:56.229835+0000","last_active":"2024-03-23T17:49:28.782109+0000","last_peered":"2024-03-23T17:49:28.782109+0000","last_clean":"2024-03-23T17:49:28.782109+0000","last_became_active":"2024-03-23T17:48:56.229437+0000","last_became_peered":"2024-03-23T17:48:56.229437+0000","last_unstale":"2024-03-23T17:49:28.782109+0000","last_undegraded":"2024-03-23T17:49:28.782109+0000","last_fullsized":"2024-03-23T17:49:28.782109+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:43:51.544867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.784779+0000","last_change":"2024-03-23T17:48:56.233552+0000","last_active":"2024-03-23T17:49:28.784779+0000","last_peered":"2024-03-23T17:49:28.784779+0000","last_clean":"2024-03-23T17:49:28.784779+0000","last_became_active":"2024-03-23T17:48:56.232941+0000","last_became_peered":"2024-03-23T17:48:56.232941+0000","last_unstale":"2024-03-23T17:49:28.784779+0000","last_undegraded":"2024-03-23T17:49:28.784779+0000","last_fullsized":"2024-03-23T17:49:28.784779+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:03:32.325496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.785402+0000","last_change":"2024-03-23T17:48:56.238238+0000","last_active":"2024-03-23T17:49:28.785402+0000","last_peered":"2024-03-23T17:49:28.785402+0000","last_clean":"2024-03-23T17:49:28.785402+0000","last_became_active":"2024-03-23T17:48:56.237882+0000","last_became_peered":"2024-03-23T17:48:56.237882+0000","last_unstale":"2024-03-23T17:49:28.785402+0000","last_undegraded":"2024-03-23T17:49:28.785402+0000","last_fullsized":"2024-03-23T17:49:28.785402+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:27:13.896205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.783881+0000","last_change":"2024-03-23T17:48:56.242895+0000","last_active":"2024-03-23T17:49:28.783881+0000","last_peered":"2024-03-23T17:49:28.783881+0000","last_clean":"2024-03-23T17:49:28.783881+0000","last_became_active":"2024-03-23T17:48:56.242515+0000","last_became_peered":"2024-03-23T17:48:56.242515+0000","last_unstale":"2024-03-23T17:49:28.783881+0000","last_undegraded":"2024-03-23T17:49:28.783881+0000","last_fullsized":"2024-03-23T17:49:28.783881+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:45:01.978843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.776603+0000","last_change":"2024-03-23T17:48:56.237029+0000","last_active":"2024-03-23T17:49:28.776603+0000","last_peered":"2024-03-23T17:49:28.776603+0000","last_clean":"2024-03-23T17:49:28.776603+0000","last_became_active":"2024-03-23T17:48:56.236692+0000","last_became_peered":"2024-03-23T17:48:56.236692+0000","last_unstale":"2024-03-23T17:49:28.776603+0000","last_undegraded":"2024-03-23T17:49:28.776603+0000","last_fullsized":"2024-03-23T17:49:28.776603+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:42:31.855469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.782393+0000","last_change":"2024-03-23T17:48:56.223236+0000","last_active":"2024-03-23T17:49:28.782393+0000","last_peered":"2024-03-23T17:49:28.782393+0000","last_clean":"2024-03-23T17:49:28.782393+0000","last_became_active":"2024-03-23T17:48:56.222824+0000","last_became_peered":"2024-03-23T17:48:56.222824+0000","last_unstale":"2024-03-23T17:49:28.782393+0000","last_undegraded":"2024-03-23T17:49:28.782393+0000","last_fullsized":"2024-03-23T17:49:28.782393+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:54:45.096969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.785731+0000","last_change":"2024-03-23T17:48:56.224471+0000","last_active":"2024-03-23T17:49:28.785731+0000","last_peered":"2024-03-23T17:49:28.785731+0000","last_clean":"2024-03-23T17:49:28.785731+0000","last_became_active":"2024-03-23T17:48:56.223140+0000","last_became_peered":"2024-03-23T17:48:56.223140+0000","last_unstale":"2024-03-23T17:49:28.785731+0000","last_undegraded":"2024-03-23T17:49:28.785731+0000","last_fullsized":"2024-03-23T17:49:28.785731+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:38:53.906820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.780461+0000","last_change":"2024-03-23T17:48:56.223564+0000","last_active":"2024-03-23T17:49:28.780461+0000","last_peered":"2024-03-23T17:49:28.780461+0000","last_clean":"2024-03-23T17:49:28.780461+0000","last_became_active":"2024-03-23T17:48:56.223217+0000","last_became_peered":"2024-03-23T17:48:56.223217+0000","last_unstale":"2024-03-23T17:49:28.780461+0000","last_undegraded":"2024-03-23T17:49:28.780461+0000","last_fullsized":"2024-03-23T17:49:28.780461+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:43:53.250082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.785081+0000","last_change":"2024-03-23T17:48:56.226283+0000","last_active":"2024-03-23T17:49:28.785081+0000","last_peered":"2024-03-23T17:49:28.785081+0000","last_clean":"2024-03-23T17:49:28.785081+0000","last_became_active":"2024-03-23T17:48:56.224211+0000","last_became_peered":"2024-03-23T17:48:56.224211+0000","last_unstale":"2024-03-23T17:49:28.785081+0000","last_undegraded":"2024-03-23T17:49:28.785081+0000","last_fullsized":"2024-03-23T17:49:28.785081+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:15:20.518950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.779571+0000","last_change":"2024-03-23T17:48:56.223557+0000","last_active":"2024-03-23T17:49:28.779571+0000","last_peered":"2024-03-23T17:49:28.779571+0000","last_clean":"2024-03-23T17:49:28.779571+0000","last_became_active":"2024-03-23T17:48:56.222792+0000","last_became_peered":"2024-03-23T17:48:56.222792+0000","last_unstale":"2024-03-23T17:49:28.779571+0000","last_undegraded":"2024-03-23T17:49:28.779571+0000","last_fullsized":"2024-03-23T17:49:28.779571+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:49:17.587817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.922230+0000","last_change":"2024-03-23T17:48:56.236041+0000","last_active":"2024-03-23T17:49:28.922230+0000","last_peered":"2024-03-23T17:49:28.922230+0000","last_clean":"2024-03-23T17:49:28.922230+0000","last_became_active":"2024-03-23T17:48:56.234259+0000","last_became_peered":"2024-03-23T17:48:56.234259+0000","last_unstale":"2024-03-23T17:49:28.922230+0000","last_undegraded":"2024-03-23T17:49:28.922230+0000","last_fullsized":"2024-03-23T17:49:28.922230+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:32:43.786783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.777528+0000","last_change":"2024-03-23T17:48:56.239283+0000","last_active":"2024-03-23T17:49:28.777528+0000","last_peered":"2024-03-23T17:49:28.777528+0000","last_clean":"2024-03-23T17:49:28.777528+0000","last_became_active":"2024-03-23T17:48:56.238952+0000","last_became_peered":"2024-03-23T17:48:56.238952+0000","last_unstale":"2024-03-23T17:49:28.777528+0000","last_undegraded":"2024-03-23T17:49:28.777528+0000","last_fullsized":"2024-03-23T17:49:28.777528+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:31:50.383284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.782143+0000","last_change":"2024-03-23T17:48:56.226684+0000","last_active":"2024-03-23T17:49:28.782143+0000","last_peered":"2024-03-23T17:49:28.782143+0000","last_clean":"2024-03-23T17:49:28.782143+0000","last_became_active":"2024-03-23T17:48:56.226367+0000","last_became_peered":"2024-03-23T17:48:56.226367+0000","last_unstale":"2024-03-23T17:49:28.782143+0000","last_undegraded":"2024-03-23T17:49:28.782143+0000","last_fullsized":"2024-03-23T17:49:28.782143+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:12:29.053330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65648,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":398,"num_read_kb":275,"num_write":228,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":180,"num_keys_recovered":17,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":196944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":200,"ondisk_log_size":200,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547220,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31992,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713160,"statfs":{"total":95995035648,"available":95962275840,"internally_reserved":0,"allocated":897024,"data_stored":428738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31848901},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 17:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.64400000000000002}]},{"osd":1,"last update":"Sat Mar 23 17:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.65600000000000003}]},{"osd":2,"last update":"Sat Mar 23 17:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.42599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":4,"last update":"Sat Mar 23 17:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.748}]},{"osd":5,"last update":"Sat Mar 23 17:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.50600000000000001}]},{"osd":6,"last update":"Sat Mar 23 17:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.84499999999999997}]},{"osd":7,"last update":"Sat Mar 23 17:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.63},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.69899999999999995}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":9,"last update":"Sat Mar 23 17:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.68799999999999994}]},{"osd":10,"last update":"Sat Mar 23 17:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.80000000000000004}]}]},{"osd":10,"up_from":68,"seq":292057776151,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28568,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716584,"statfs":{"total":95995035648,"available":95965782016,"internally_reserved":0,"allocated":1585152,"data_stored":1078786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":1,"last update":"Sat Mar 23 17:49:02 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.432}]},{"osd":2,"last update":"Sat Mar 23 17:49:02 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.64700000000000002}]},{"osd":3,"last update":"Sat Mar 23 17:49:02 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.61599999999999999}]},{"osd":4,"last update":"Sat Mar 23 17:49:02 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.54800000000000004}]},{"osd":5,"last update":"Sat Mar 23 17:49:02 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.69199999999999995}]},{"osd":6,"last update":"Sat Mar 23 17:49:02 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.66200000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":8,"last update":"Sat Mar 23 17:49:02 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.63700000000000001}]},{"osd":9,"last update":"Sat Mar 23 17:49:02 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.67200000000000004}]},{"osd":11,"last update":"Sat Mar 23 17:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.70799999999999996}]}]},{"osd":9,"up_from":63,"seq":270582939676,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36108,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709044,"statfs":{"total":95995035648,"available":95958061056,"internally_reserved":0,"allocated":913408,"data_stored":428648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":36043201},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 17:48:42 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.40100000000000002}]},{"osd":1,"last update":"Sat Mar 23 17:48:42 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.51800000000000002}]},{"osd":2,"last update":"Sat Mar 23 17:48:42 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.56200000000000006}]},{"osd":3,"last update":"Sat Mar 23 17:48:42 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.52900000000000003}]},{"osd":4,"last update":"Sat Mar 23 17:48:42 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.61099999999999999}]},{"osd":5,"last update":"Sat Mar 23 17:48:42 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.48,"5min":1.48,"15min":1.48},"last":0.751}]},{"osd":6,"last update":"Sat Mar 23 17:48:42 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.499},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.59899999999999998}]},{"osd":7,"last update":"Sat Mar 23 17:48:42 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.373,"5min":1.373,"15min":1.373},"last":0.84499999999999997}]},{"osd":8,"last update":"Sat Mar 23 17:48:42 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.66500000000000004}]},{"osd":10,"last update":"Sat Mar 23 17:48:58 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.79600000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81599999999999995}]}]},{"osd":8,"up_from":56,"seq":240518168606,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":429026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 17:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.57399999999999995,"15min":0.57999999999999996},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.78000000000000003,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.504},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.54800000000000004,"15min":0.55100000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.67300000000000004,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.52900000000000003}]},{"osd":1,"last update":"Sat Mar 23 17:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.59099999999999997,"15min":0.58599999999999997},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.77700000000000002,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.65700000000000003}]},{"osd":2,"last update":"Sat Mar 23 17:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.45600000000000002,"15min":0.45400000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.46500000000000002,"15min":0.46200000000000002},"min":{"1min":0.36799999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.61099999999999999,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.55600000000000005}]},{"osd":3,"last update":"Sat Mar 23 17:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.58499999999999996,"15min":0.59099999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.76800000000000002,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.65300000000000002,"15min":0.66200000000000003},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.83199999999999996,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.82299999999999995}]},{"osd":4,"last update":"Sat Mar 23 17:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.59499999999999997,"15min":0.58799999999999997},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.63800000000000001,"15min":0.63500000000000001},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.746}]},{"osd":5,"last update":"Sat Mar 23 17:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.54300000000000004,"15min":0.54700000000000004},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.69599999999999995,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.56200000000000006,"15min":0.55800000000000005},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.79400000000000004,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.80700000000000005}]},{"osd":6,"last update":"Sat Mar 23 17:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.70499999999999996,"15min":0.71099999999999997},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.91900000000000004,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.68500000000000005,"15min":0.68999999999999995},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.78700000000000003}]},{"osd":7,"last update":"Sat Mar 23 17:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68100000000000005,"15min":0.67900000000000005},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.876,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.70299999999999996,"15min":0.69899999999999995},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.90200000000000002,"5min":1.04,"15min":1.04},"last":0.84399999999999997}]},{"osd":9,"last update":"Sat Mar 23 17:48:42 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.875},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.88500000000000001}]},{"osd":10,"last update":"Sat Mar 23 17:49:01 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.92000000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86299999999999999}]}]},{"osd":7,"up_from":49,"seq":210453397538,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28492,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716660,"statfs":{"total":95995035648,"available":95965859840,"internally_reserved":0,"allocated":1507328,"data_stored":1021009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 17:48:02 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.54800000000000004}]},{"osd":1,"last update":"Sat Mar 23 17:49:03 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.40200000000000002,"15min":0.39600000000000002},"min":{"1min":0.33800000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.58399999999999996,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.39200000000000002,"15min":0.378},"min":{"1min":0.28299999999999997,"5min":0.248,"15min":0.248},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.41199999999999998}]},{"osd":2,"last update":"Sat Mar 23 17:49:03 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.56200000000000006,"15min":0.55400000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.56200000000000006,"15min":0.55600000000000005},"min":{"1min":0.45500000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.47499999999999998}]},{"osd":3,"last update":"Sat Mar 23 17:49:03 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.54800000000000004,"15min":0.54400000000000004},"min":{"1min":0.42499999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.70599999999999996,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.627,"5min":0.59499999999999997,"15min":0.58999999999999997},"min":{"1min":0.44600000000000001,"5min":0.432,"15min":0.432},"max":{"1min":0.74099999999999999,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.61199999999999999}]},{"osd":4,"last update":"Sat Mar 23 17:49:03 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.48799999999999999,"15min":0.47699999999999998},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.47899999999999998,"15min":0.47099999999999997},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.68500000000000005,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.51500000000000001}]},{"osd":5,"last update":"Sat Mar 23 17:49:03 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.59499999999999997,"15min":0.58599999999999997},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.77500000000000002,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.625,"5min":0.58199999999999996,"15min":0.57499999999999996},"min":{"1min":0.41999999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.57799999999999996}]},{"osd":6,"last update":"Sat Mar 23 17:49:03 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.67500000000000004,"15min":0.66700000000000004},"min":{"1min":0.54600000000000004,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.95499999999999996,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.45500000000000002,"5min":0.433,"15min":0.433},"max":{"1min":0.81899999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.65200000000000002}]},{"osd":8,"last update":"Sat Mar 23 17:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.64900000000000002,"15min":0.63300000000000001},"min":{"1min":0.47799999999999998,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.66600000000000004,"15min":0.64700000000000002},"min":{"1min":0.52700000000000002,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.747}]},{"osd":9,"last update":"Sat Mar 23 17:48:42 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.629}]},{"osd":10,"last update":"Sat Mar 23 17:49:01 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.68600000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]}]},{"osd":6,"up_from":45,"seq":193273528359,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36088,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93709064,"statfs":{"total":95995035648,"available":95958081536,"internally_reserved":0,"allocated":892928,"data_stored":428044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 17:48:45 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.47499999999999998,"15min":0.47599999999999998},"min":{"1min":0.35599999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.52800000000000002,"15min":0.52400000000000002},"min":{"1min":0.33700000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":1.256}]},{"osd":1,"last update":"Sat Mar 23 17:48:45 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.64400000000000002,"15min":0.64500000000000002},"min":{"1min":0.48399999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.91800000000000004,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.88700000000000001}]},{"osd":2,"last update":"Sat Mar 23 17:48:45 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.67900000000000005,"15min":0.68200000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":1.4710000000000001},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.67400000000000004,"15min":0.67800000000000005},"min":{"1min":0.47099999999999997,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.91900000000000004,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.59299999999999997}]},{"osd":3,"last update":"Sat Mar 23 17:48:45 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.61399999999999999,"15min":0.61899999999999999},"min":{"1min":0.34399999999999997,"5min":0.316,"15min":0.316},"max":{"1min":0.89700000000000002,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.66400000000000003,"15min":0.66600000000000004},"min":{"1min":0.45600000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.88300000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":1.1140000000000001}]},{"osd":4,"last update":"Sat Mar 23 17:48:45 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.65700000000000003,"15min":0.65800000000000003},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":1.3740000000000001},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.72999999999999998,"15min":0.73699999999999999},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.86399999999999999,"5min":0.998,"15min":0.998},"last":0.68300000000000005}]},{"osd":5,"last update":"Sat Mar 23 17:48:45 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.77900000000000003,"15min":0.78300000000000003},"min":{"1min":0.57199999999999995,"5min":0.436,"15min":0.436},"max":{"1min":0.90600000000000003,"5min":1.375,"15min":1.375},"last":1.2170000000000001},{"interface":"front","average":{"1min":0.75,"5min":0.78100000000000003,"15min":0.78600000000000003},"min":{"1min":0.59199999999999997,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":1.504}]},{"osd":7,"last update":"Sat Mar 23 17:49:03 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.73699999999999999,"15min":0.73899999999999999},"min":{"1min":0.53800000000000003,"5min":0.5,"15min":0.5},"max":{"1min":0.97299999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":1.347},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.77000000000000002,"15min":0.78600000000000003},"min":{"1min":0.505,"5min":0.495,"15min":0.495},"max":{"1min":0.91100000000000003,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":1.4339999999999999}]},{"osd":8,"last update":"Sat Mar 23 17:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.85599999999999998,"15min":0.876},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.92400000000000004,"5min":1.085,"15min":1.085},"last":1.5409999999999999},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.81499999999999995,"15min":0.82399999999999995},"min":{"1min":0.63200000000000001,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":0.91200000000000003,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.66500000000000004}]},{"osd":9,"last update":"Sat Mar 23 17:48:45 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":1.3100000000000001},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.83699999999999997}]},{"osd":10,"last update":"Sat Mar 23 17:49:01 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":1.369},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":1.573}]}]},{"osd":1,"up_from":13,"seq":55834574904,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":488648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 17:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.52600000000000002,"15min":0.51300000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.85699999999999998,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.63100000000000001,"15min":0.53300000000000003},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.73699999999999999,"5min":5.8399999999999999,"15min":5.8399999999999999},"last":0.504}]},{"osd":2,"last update":"Sat Mar 23 17:49:29 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.52700000000000002,"15min":0.48099999999999998},"min":{"1min":0.46600000000000003,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.81399999999999995,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.623,"5min":0.52200000000000002,"15min":0.49299999999999999},"min":{"1min":0.45900000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.77400000000000002,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.68200000000000005}]},{"osd":3,"last update":"Sat Mar 23 17:48:51 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.55000000000000004,"15min":0.53200000000000003},"min":{"1min":0.377,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.55100000000000005,"15min":0.53900000000000003},"min":{"1min":0.39100000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.53600000000000003}]},{"osd":4,"last update":"Sat Mar 23 17:49:08 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.432,"15min":0.40200000000000002},"min":{"1min":0.29499999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.371},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.42299999999999999,"15min":0.39700000000000002},"min":{"1min":0.33800000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.75900000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.42799999999999999}]},{"osd":5,"last update":"Sat Mar 23 17:49:29 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.59499999999999997,"15min":0.56299999999999994},"min":{"1min":0.55500000000000005,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.86599999999999999,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.61199999999999999,"15min":0.57799999999999996},"min":{"1min":0.55600000000000005,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.85399999999999998,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.73899999999999999}]},{"osd":6,"last update":"Sat Mar 23 17:48:45 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.58799999999999997,"15min":0.56899999999999995},"min":{"1min":0.52100000000000002,"5min":0.433,"15min":0.433},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.628},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.63600000000000001,"15min":0.621},"min":{"1min":0.47399999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.40799999999999997}]},{"osd":7,"last update":"Sat Mar 23 17:49:03 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.52700000000000002,"15min":0.51100000000000001},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.82899999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.56200000000000006,"15min":0.53800000000000003},"min":{"1min":0.45000000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.48299999999999998}]},{"osd":8,"last update":"Sat Mar 23 17:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.68000000000000005,"15min":0.66800000000000004},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.96099999999999997,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.64400000000000002,"15min":0.63200000000000001},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.57199999999999995}]},{"osd":9,"last update":"Sat Mar 23 17:48:45 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.72199999999999998}]},{"osd":10,"last update":"Sat Mar 23 17:48:59 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.76500000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80800000000000005}]}]},{"osd":0,"up_from":8,"seq":34359738430,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":913408,"data_stored":429383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 23 17:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.64600000000000002,"15min":0.72399999999999998},"min":{"1min":0.30099999999999999,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.85899999999999999,"5min":6.2110000000000003,"15min":6.2110000000000003},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.54300000000000004,"15min":0.50700000000000001},"min":{"1min":0.38500000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.84499999999999997,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.432}]},{"osd":2,"last update":"Sat Mar 23 17:49:31 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.57799999999999996,"15min":0.55100000000000005},"min":{"1min":0.38100000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.82999999999999996,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.59999999999999998,"15min":0.56000000000000005},"min":{"1min":0.41299999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.63400000000000001}]},{"osd":3,"last update":"Sat Mar 23 17:47:52 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.498,"15min":0.49399999999999999},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.78000000000000003,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.34499999999999997,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.79100000000000004,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.68200000000000005}]},{"osd":4,"last update":"Sat Mar 23 17:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.628,"15min":0.61799999999999999},"min":{"1min":0.42199999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59599999999999997,"15min":0.58699999999999997},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.88100000000000001,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.46100000000000002}]},{"osd":5,"last update":"Sat Mar 23 17:49:28 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65700000000000003,"15min":0.63700000000000001},"min":{"1min":0.44700000000000001,"5min":0.44,"15min":0.44},"max":{"1min":1.018,"5min":1.167,"15min":1.167},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.64500000000000002,"15min":0.61599999999999999},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.65700000000000003}]},{"osd":6,"last update":"Sat Mar 23 17:48:47 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.60699999999999998,"15min":0.60099999999999998},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.67900000000000005,"15min":0.66800000000000004},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.749}]},{"osd":7,"last update":"Sat Mar 23 17:48:02 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.495},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.58599999999999997}]},{"osd":8,"last update":"Sat Mar 23 17:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.83299999999999996,"15min":0.84899999999999998},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.091,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.81299999999999994,"15min":0.82599999999999996},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.70099999999999996}]},{"osd":9,"last update":"Sat Mar 23 17:48:42 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":0.82199999999999995}]},{"osd":10,"last update":"Sat Mar 23 17:49:04 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.79600000000000004}]},{"osd":11,"last update":"Sat Mar 23 17:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.82899999999999996}]}]},{"osd":2,"up_from":18,"seq":77309411381,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36148,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709004,"statfs":{"total":95995035648,"available":95958020096,"internally_reserved":0,"allocated":958464,"data_stored":488367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 17:49:30 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.54000000000000004,"15min":0.51400000000000001},"min":{"1min":0.42199999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.63,"5min":0.57899999999999996,"15min":0.55300000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.88700000000000001,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.47199999999999998}]},{"osd":1,"last update":"Sat Mar 23 17:49:30 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.54500000000000004,"15min":0.5},"min":{"1min":0.42199999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.80900000000000005,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.55800000000000005,"15min":0.51800000000000002},"min":{"1min":0.41999999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.79300000000000004,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.71399999999999997}]},{"osd":3,"last update":"Sat Mar 23 17:48:54 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.54400000000000004,"15min":0.52800000000000002},"min":{"1min":0.36699999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.58299999999999996,"15min":0.56499999999999995},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.50800000000000001}]},{"osd":4,"last update":"Sat Mar 23 17:48:06 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.51900000000000002,"15min":0.51100000000000001},"min":{"1min":0.41099999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.628,"5min":0.58199999999999996,"15min":0.57399999999999995},"min":{"1min":0.42599999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.66700000000000004}]},{"osd":5,"last update":"Sat Mar 23 17:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.48299999999999998,"15min":0.45900000000000002},"min":{"1min":0.32600000000000001,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.495,"15min":0.46999999999999997},"min":{"1min":0.34100000000000003,"5min":0.308,"15min":0.308},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.34699999999999998}]},{"osd":6,"last update":"Sat Mar 23 17:48:48 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.623,"15min":0.61699999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.64400000000000002,"15min":0.63800000000000001},"min":{"1min":0.46100000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.60499999999999998}]},{"osd":7,"last update":"Sat Mar 23 17:48:01 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.745}]},{"osd":8,"last update":"Sat Mar 23 17:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.51000000000000001,"15min":0.50600000000000001},"min":{"1min":0.35599999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.59399999999999997,"15min":0.58599999999999997},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.78100000000000003,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.45200000000000001}]},{"osd":9,"last update":"Sat Mar 23 17:48:43 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.68400000000000005}]},{"osd":10,"last update":"Sat Mar 23 17:49:03 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.73999999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]}]},{"osd":3,"up_from":26,"seq":111669149745,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32008,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713144,"statfs":{"total":95995035648,"available":95962259456,"internally_reserved":0,"allocated":909312,"data_stored":427917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 17:48:58 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.41299999999999998,"15min":0.39300000000000002},"min":{"1min":0.34599999999999997,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.499,"5min":0.42499999999999999,"15min":0.40000000000000002},"min":{"1min":0.33500000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.94899999999999995,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.46500000000000002}]},{"osd":1,"last update":"Sat Mar 23 17:48:58 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.54200000000000004,"15min":0.51500000000000001},"min":{"1min":0.379,"5min":0.371,"15min":0.371},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.54100000000000004,"15min":0.50700000000000001},"min":{"1min":0.40699999999999997,"5min":0.371,"15min":0.371},"max":{"1min":0.91500000000000004,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.63300000000000001}]},{"osd":2,"last update":"Sat Mar 23 17:48:58 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.60999999999999999,"15min":0.58299999999999996},"min":{"1min":0.496,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.626,"15min":0.61199999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.90500000000000003,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.52000000000000002}]},{"osd":4,"last update":"Sat Mar 23 17:49:09 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.63600000000000001,"15min":0.60999999999999999},"min":{"1min":0.46700000000000003,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.63700000000000001,"15min":0.61299999999999999},"min":{"1min":0.45600000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.71599999999999997}]},{"osd":5,"last update":"Sat Mar 23 17:49:31 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.67800000000000005,"15min":0.65000000000000002},"min":{"1min":0.53500000000000003,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.01,"5min":1.036,"15min":1.036},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.70099999999999996,"15min":0.67300000000000004},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.66100000000000003}]},{"osd":6,"last update":"Sat Mar 23 17:48:45 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.64800000000000002,"15min":0.64400000000000002},"min":{"1min":0.504,"5min":0.38,"15min":0.38},"max":{"1min":0.88600000000000001,"5min":1.024,"15min":1.024},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.66900000000000004,"15min":0.66400000000000003},"min":{"1min":0.48099999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.93400000000000005,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.95399999999999996}]},{"osd":7,"last update":"Sat Mar 23 17:49:06 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.76200000000000001,"15min":0.76700000000000002},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.995,"5min":1.038,"15min":1.038},"last":1.016},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.71099999999999997,"15min":0.70799999999999996},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.79600000000000004}]},{"osd":8,"last update":"Sat Mar 23 17:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.88},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.85099999999999998}]},{"osd":9,"last update":"Sat Mar 23 17:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.59499999999999997}]},{"osd":10,"last update":"Sat Mar 23 17:49:02 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":1.046},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.85299999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82399999999999995}]}]},{"osd":4,"up_from":33,"seq":141733920815,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":901120,"data_stored":428017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 17:48:06 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.53700000000000003,"15min":0.53400000000000003},"min":{"1min":0.41799999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.77600000000000002,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.53700000000000003,"15min":0.52500000000000002},"min":{"1min":0.45300000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.52000000000000002}]},{"osd":1,"last update":"Sat Mar 23 17:49:07 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.34999999999999998,"15min":0.32500000000000001},"min":{"1min":0.27700000000000002,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.60799999999999998,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.36499999999999999,"15min":0.34300000000000003},"min":{"1min":0.28599999999999998,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.56299999999999994,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.41499999999999998}]},{"osd":2,"last update":"Sat Mar 23 17:49:07 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.56899999999999995,"15min":0.55500000000000005},"min":{"1min":0.379,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.80700000000000005,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.56599999999999995,"15min":0.54600000000000004},"min":{"1min":0.42499999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.78000000000000003,"5min":0.88,"15min":0.88},"last":0.54700000000000004}]},{"osd":3,"last update":"Sat Mar 23 17:49:07 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.56899999999999995,"15min":0.55900000000000005},"min":{"1min":0.40300000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.751,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.56799999999999995,"15min":0.56299999999999994},"min":{"1min":0.38400000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.76600000000000001,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.47799999999999998}]},{"osd":5,"last update":"Sat Mar 23 17:49:29 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.57999999999999996,"15min":0.56599999999999995},"min":{"1min":0.41299999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.80600000000000005,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.629,"5min":0.59499999999999997,"15min":0.57799999999999996},"min":{"1min":0.44600000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.77700000000000002,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.57099999999999995}]},{"osd":6,"last update":"Sat Mar 23 17:48:53 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.47699999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.78200000000000003,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.58999999999999997,"15min":0.58899999999999997},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.73599999999999999}]},{"osd":7,"last update":"Sat Mar 23 17:49:03 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.54400000000000004,"15min":0.54000000000000004},"min":{"1min":0.39300000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.54000000000000004,"15min":0.53400000000000003},"min":{"1min":0.36299999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.70099999999999996,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.51500000000000001}]},{"osd":8,"last update":"Sat Mar 23 17:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.65800000000000003,"15min":0.65400000000000003},"min":{"1min":0.501,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.71699999999999997,"15min":0.71799999999999997},"min":{"1min":0.48599999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.86599999999999999,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.67900000000000005}]},{"osd":9,"last update":"Sat Mar 23 17:48:42 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.623}]},{"osd":10,"last update":"Sat Mar 23 17:49:01 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.749},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.66700000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]}]},{"osd":5,"up_from":38,"seq":163208757290,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32600,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712552,"statfs":{"total":95995035648,"available":95961653248,"internally_reserved":0,"allocated":1515520,"data_stored":1020452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 17:49:28 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.54900000000000004,"15min":0.54100000000000004},"min":{"1min":0.435,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.502},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.54400000000000004,"15min":0.54800000000000004},"min":{"1min":0.36299999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.83499999999999996,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.441}]},{"osd":1,"last update":"Sat Mar 23 17:49:28 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.55100000000000005,"15min":0.52700000000000002},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.58799999999999997,"15min":0.56499999999999995},"min":{"1min":0.495,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.61399999999999999}]},{"osd":2,"last update":"Sat Mar 23 17:49:28 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.40400000000000003,"15min":0.39400000000000002},"min":{"1min":0.311,"5min":0.251,"15min":0.251},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.503,"5min":0.41799999999999998,"15min":0.40100000000000002},"min":{"1min":0.30399999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.38400000000000001}]},{"osd":3,"last update":"Sat Mar 23 17:49:28 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58299999999999996,"15min":0.58499999999999996},"min":{"1min":0.38100000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.80100000000000005,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.57699999999999996,"15min":0.58499999999999996},"min":{"1min":0.40899999999999997,"5min":0.373,"15min":0.373},"max":{"1min":0.71899999999999997,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.52000000000000002}]},{"osd":4,"last update":"Sat Mar 23 17:49:28 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.60599999999999998,"15min":0.58199999999999996},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.89500000000000002,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.58199999999999996,"15min":0.56000000000000005},"min":{"1min":0.44400000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.877,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.71199999999999997}]},{"osd":6,"last update":"Sat Mar 23 17:48:48 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.627,"15min":0.625},"min":{"1min":0.53400000000000003,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.88500000000000001,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.441,"5min":0.434,"15min":0.434},"max":{"1min":0.79600000000000004,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.56499999999999995}]},{"osd":7,"last update":"Sat Mar 23 17:49:03 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.64400000000000002,"15min":0.64000000000000001},"min":{"1min":0.41999999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.89300000000000002,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.63500000000000001,"15min":0.63},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.86599999999999999,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.79200000000000004}]},{"osd":8,"last update":"Sat Mar 23 17:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.60499999999999998,"15min":0.59099999999999997},"min":{"1min":0.39600000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.56999999999999995,"15min":0.55800000000000005},"min":{"1min":0.42999999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.53900000000000003}]},{"osd":9,"last update":"Sat Mar 23 17:48:43 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.64800000000000002}]},{"osd":10,"last update":"Sat Mar 23 17:49:00 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.754},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.80500000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":61114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":61106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T17:49:41.107 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph pg dump --format=json 2024-03-23T17:49:41.466 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T17:49:42.002 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:41 smithi086 ceph-mon[38221]: pgmap v258: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s; 17 B/s, 1 keys/s, 1 objects/s recovering 2024-03-23T17:49:42.002 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:41 smithi086 ceph-mon[38221]: from='client.14826 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T17:49:42.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:41 smithi152 ceph-mon[47949]: pgmap v258: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s; 17 B/s, 1 keys/s, 1 objects/s recovering 2024-03-23T17:49:42.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:41 smithi152 ceph-mon[47949]: from='client.14826 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T17:49:42.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:41 smithi092 ceph-mon[79089]: pgmap v258: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s; 17 B/s, 1 keys/s, 1 objects/s recovering 2024-03-23T17:49:42.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:41 smithi092 ceph-mon[79089]: from='client.14826 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T17:49:42.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:42.665+0000 7f55d5400700 1 -- 172.21.15.86:0/4236208731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55d0101820 msgr2=0x7f55d010dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:42.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:42.665+0000 7f55d5400700 1 --2- 172.21.15.86:0/4236208731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55d0101820 0x7f55d010dc10 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f55c800b0d0 tx=0x7f55c800b3e0 comp rx=0 tx=0).stop 2024-03-23T17:49:42.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:42.665+0000 7f55d5400700 1 -- 172.21.15.86:0/4236208731 shutdown_connections 2024-03-23T17:49:42.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:42.665+0000 7f55d5400700 1 --2- 172.21.15.86:0/4236208731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55d0101820 0x7f55d010dc10 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:42.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:42.665+0000 7f55d5400700 1 --2- 172.21.15.86:0/4236208731 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55d0106ef0 0x7f55d01072d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:42.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:42.665+0000 7f55d5400700 1 --2- 172.21.15.86:0/4236208731 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f55d0100e60 0x7f55d01012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:42.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:42.665+0000 7f55d5400700 1 -- 172.21.15.86:0/4236208731 >> 172.21.15.86:0/4236208731 conn(0x7f55d0077fa0 msgr2=0x7f55d00783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:42.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:42.666+0000 7f55d5400700 1 -- 172.21.15.86:0/4236208731 shutdown_connections 2024-03-23T17:49:42.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:42.666+0000 7f55d5400700 1 -- 172.21.15.86:0/4236208731 wait complete. 2024-03-23T17:49:42.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:42.667+0000 7f55d5400700 1 Processor -- start 2024-03-23T17:49:42.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:42.667+0000 7f55d5400700 1 -- start start 2024-03-23T17:49:42.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:42.667+0000 7f55d5400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f55d0100e60 0x7f55d01994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:42.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:42.667+0000 7f55d5400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55d0101820 0x7f55d0199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:42.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:42.667+0000 7f55d5400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55d0106ef0 0x7f55d019ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:42.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:42.667+0000 7f55d5400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55d019e3e0 con 0x7f55d0101820 2024-03-23T17:49:42.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:42.667+0000 7f55d5400700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55d019e550 con 0x7f55d0106ef0 2024-03-23T17:49:42.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:42.667+0000 7f55d5400700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55d019e710 con 0x7f55d0100e60 2024-03-23T17:49:42.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:42.667+0000 7f55cde00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55d0101820 0x7f55d0199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:42.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:42.667+0000 7f55ce800700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f55d0100e60 0x7f55d01994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:42.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:42.667+0000 7f55cde00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55d0101820 0x7f55d0199a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40524/0 (socket says 172.21.15.86:40524) 2024-03-23T17:49:42.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:42.667+0000 7f55cde00700 1 -- 172.21.15.86:0/1586105682 learned_addr learned my addr 172.21.15.86:0/1586105682 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:42.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:42.667+0000 7f55cf200700 1 --2- 172.21.15.86:0/1586105682 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55d0106ef0 0x7f55d019ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:42.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:42.668+0000 7f55cde00700 1 -- 172.21.15.86:0/1586105682 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f55d0100e60 msgr2=0x7f55d01994e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:42.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:42.668+0000 7f55cde00700 1 --2- 172.21.15.86:0/1586105682 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f55d0100e60 0x7f55d01994e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:42.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:42.668+0000 7f55cde00700 1 -- 172.21.15.86:0/1586105682 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55d0106ef0 msgr2=0x7f55d019ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:42.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:42.668+0000 7f55cde00700 1 --2- 172.21.15.86:0/1586105682 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55d0106ef0 0x7f55d019ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:42.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:42.668+0000 7f55cde00700 1 -- 172.21.15.86:0/1586105682 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55c8009d00 con 0x7f55d0101820 2024-03-23T17:49:42.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:42.668+0000 7f55cde00700 1 --2- 172.21.15.86:0/1586105682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55d0101820 0x7f55d0199a20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f55bc00e9a0 tx=0x7f55bc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:42.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:42.668+0000 7f55c7400700 1 -- 172.21.15.86:0/1586105682 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55bc009790 con 0x7f55d0101820 2024-03-23T17:49:42.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:42.668+0000 7f55d5400700 1 -- 172.21.15.86:0/1586105682 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55d019e9f0 con 0x7f55d0101820 2024-03-23T17:49:42.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:42.668+0000 7f55c7400700 1 -- 172.21.15.86:0/1586105682 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55bc010430 con 0x7f55d0101820 2024-03-23T17:49:42.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:42.668+0000 7f55c7400700 1 -- 172.21.15.86:0/1586105682 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55bc018680 con 0x7f55d0101820 2024-03-23T17:49:42.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:42.669+0000 7f55d5400700 1 -- 172.21.15.86:0/1586105682 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55d01a6250 con 0x7f55d0101820 2024-03-23T17:49:42.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:42.670+0000 7f55d5400700 1 -- 172.21.15.86:0/1586105682 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55d01024f0 con 0x7f55d0101820 2024-03-23T17:49:42.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:42.671+0000 7f55c7400700 1 -- 172.21.15.86:0/1586105682 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f55bc0105a0 con 0x7f55d0101820 2024-03-23T17:49:42.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:42.672+0000 7f55c7400700 1 --2- 172.21.15.86:0/1586105682 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f55b40a11b0 0x7f55b40a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:42.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:42.673+0000 7f55c7400700 1 -- 172.21.15.86:0/1586105682 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f55bc014070 con 0x7f55d0101820 2024-03-23T17:49:42.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:42.673+0000 7f55ce800700 1 --2- 172.21.15.86:0/1586105682 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f55b40a11b0 0x7f55b40a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:42.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:42.673+0000 7f55ce800700 1 --2- 172.21.15.86:0/1586105682 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f55b40a11b0 0x7f55b40a3670 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f55c0009c60 tx=0x7f55c0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:42.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:42.677+0000 7f55c7400700 1 -- 172.21.15.86:0/1586105682 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f55bc0529f0 con 0x7f55d0101820 2024-03-23T17:49:42.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:42.876+0000 7f55d5400700 1 -- 172.21.15.86:0/1586105682 --> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f55d019a890 con 0x7f55b40a11b0 2024-03-23T17:49:42.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:42.886+0000 7f55c7400700 1 -- 172.21.15.86:0/1586105682 <== mgr.14152 v2:172.21.15.86:6800/2734784244 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+341371 (secure 0 0 0) 0x7f55d019a890 con 0x7f55b40a11b0 2024-03-23T17:49:42.891 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:49:42.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:42.890+0000 7f55d5400700 1 -- 172.21.15.86:0/1586105682 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f55b40a11b0 msgr2=0x7f55b40a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:42.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:42.890+0000 7f55d5400700 1 --2- 172.21.15.86:0/1586105682 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f55b40a11b0 0x7f55b40a3670 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f55c0009c60 tx=0x7f55c0009260 comp rx=0 tx=0).stop 2024-03-23T17:49:42.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:42.890+0000 7f55d5400700 1 -- 172.21.15.86:0/1586105682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55d0101820 msgr2=0x7f55d0199a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:42.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:42.890+0000 7f55d5400700 1 --2- 172.21.15.86:0/1586105682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55d0101820 0x7f55d0199a20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f55bc00e9a0 tx=0x7f55bc00ecb0 comp rx=0 tx=0).stop 2024-03-23T17:49:42.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:42.891+0000 7f55d5400700 1 -- 172.21.15.86:0/1586105682 shutdown_connections 2024-03-23T17:49:42.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:42.891+0000 7f55d5400700 1 --2- 172.21.15.86:0/1586105682 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55d0106ef0 0x7f55d019ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:42.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:42.891+0000 7f55d5400700 1 --2- 172.21.15.86:0/1586105682 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f55d0100e60 0x7f55d01994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:42.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:42.891+0000 7f55d5400700 1 --2- 172.21.15.86:0/1586105682 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f55b40a11b0 0x7f55b40a3670 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:42.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:42.891+0000 7f55d5400700 1 --2- 172.21.15.86:0/1586105682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55d0101820 0x7f55d0199a20 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:42.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:42.891+0000 7f55d5400700 1 -- 172.21.15.86:0/1586105682 >> 172.21.15.86:0/1586105682 conn(0x7f55d0077fa0 msgr2=0x7f55d010c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:42.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:42.891+0000 7f55d5400700 1 -- 172.21.15.86:0/1586105682 shutdown_connections 2024-03-23T17:49:42.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:42.891+0000 7f55d5400700 1 -- 172.21.15.86:0/1586105682 wait complete. 2024-03-23T17:49:42.894 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-03-23T17:49:43.263 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":259,"stamp":"2024-03-23T17:49:42.318570+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656250,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":549,"num_read_kb":399,"num_write":411,"num_write_kb":3068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":1180916,"num_keys_recovered":17,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":339,"ondisk_log_size":339,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":373684,"kb_used_data":12560,"kb_used_omap":18,"kb_used_meta":356845,"kb_avail":1124568140,"statfs":{"total":1151940427776,"available":1151557775360,"internally_reserved":0,"allocated":12861440,"data_stored":7097035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19130,"internal_metadata":365409606},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":114,"num_read_kb":55,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":180,"num_keys_recovered":17,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002695"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.782853+0000","last_change":"2024-03-23T17:48:56.239724+0000","last_active":"2024-03-23T17:49:28.782853+0000","last_peered":"2024-03-23T17:49:28.782853+0000","last_clean":"2024-03-23T17:49:28.782853+0000","last_became_active":"2024-03-23T17:48:56.239451+0000","last_became_peered":"2024-03-23T17:48:56.239451+0000","last_unstale":"2024-03-23T17:49:28.782853+0000","last_undegraded":"2024-03-23T17:49:28.782853+0000","last_fullsized":"2024-03-23T17:49:28.782853+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:51:38.687863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.783538+0000","last_change":"2024-03-23T17:48:56.232415+0000","last_active":"2024-03-23T17:49:28.783538+0000","last_peered":"2024-03-23T17:49:28.783538+0000","last_clean":"2024-03-23T17:49:28.783538+0000","last_became_active":"2024-03-23T17:48:56.231777+0000","last_became_peered":"2024-03-23T17:48:56.231777+0000","last_unstale":"2024-03-23T17:49:28.783538+0000","last_undegraded":"2024-03-23T17:49:28.783538+0000","last_fullsized":"2024-03-23T17:49:28.783538+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:16:22.631654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.784174+0000","last_change":"2024-03-23T17:48:56.243413+0000","last_active":"2024-03-23T17:49:28.784174+0000","last_peered":"2024-03-23T17:49:28.784174+0000","last_clean":"2024-03-23T17:49:28.784174+0000","last_became_active":"2024-03-23T17:48:56.241281+0000","last_became_peered":"2024-03-23T17:48:56.241281+0000","last_unstale":"2024-03-23T17:49:28.784174+0000","last_undegraded":"2024-03-23T17:49:28.784174+0000","last_fullsized":"2024-03-23T17:49:28.784174+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:52:48.521689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.784355+0000","last_change":"2024-03-23T17:48:56.226010+0000","last_active":"2024-03-23T17:49:28.784355+0000","last_peered":"2024-03-23T17:49:28.784355+0000","last_clean":"2024-03-23T17:49:28.784355+0000","last_became_active":"2024-03-23T17:48:56.225562+0000","last_became_peered":"2024-03-23T17:48:56.225562+0000","last_unstale":"2024-03-23T17:49:28.784355+0000","last_undegraded":"2024-03-23T17:49:28.784355+0000","last_fullsized":"2024-03-23T17:49:28.784355+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:17:47.597335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.780757+0000","last_change":"2024-03-23T17:48:56.223249+0000","last_active":"2024-03-23T17:49:28.780757+0000","last_peered":"2024-03-23T17:49:28.780757+0000","last_clean":"2024-03-23T17:49:28.780757+0000","last_became_active":"2024-03-23T17:48:56.222902+0000","last_became_peered":"2024-03-23T17:48:56.222902+0000","last_unstale":"2024-03-23T17:49:28.780757+0000","last_undegraded":"2024-03-23T17:49:28.780757+0000","last_fullsized":"2024-03-23T17:49:28.780757+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:51:49.029628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.781946+0000","last_change":"2024-03-23T17:48:56.229229+0000","last_active":"2024-03-23T17:49:28.781946+0000","last_peered":"2024-03-23T17:49:28.781946+0000","last_clean":"2024-03-23T17:49:28.781946+0000","last_became_active":"2024-03-23T17:48:56.228725+0000","last_became_peered":"2024-03-23T17:48:56.228725+0000","last_unstale":"2024-03-23T17:49:28.781946+0000","last_undegraded":"2024-03-23T17:49:28.781946+0000","last_fullsized":"2024-03-23T17:49:28.781946+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:53:44.118762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.780711+0000","last_change":"2024-03-23T17:48:56.229212+0000","last_active":"2024-03-23T17:49:28.780711+0000","last_peered":"2024-03-23T17:49:28.780711+0000","last_clean":"2024-03-23T17:49:28.780711+0000","last_became_active":"2024-03-23T17:48:56.228888+0000","last_became_peered":"2024-03-23T17:48:56.228888+0000","last_unstale":"2024-03-23T17:49:28.780711+0000","last_undegraded":"2024-03-23T17:49:28.780711+0000","last_fullsized":"2024-03-23T17:49:28.780711+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:47:21.509554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.779498+0000","last_change":"2024-03-23T17:48:56.222925+0000","last_active":"2024-03-23T17:49:28.779498+0000","last_peered":"2024-03-23T17:49:28.779498+0000","last_clean":"2024-03-23T17:49:28.779498+0000","last_became_active":"2024-03-23T17:48:56.221504+0000","last_became_peered":"2024-03-23T17:48:56.221504+0000","last_unstale":"2024-03-23T17:49:28.779498+0000","last_undegraded":"2024-03-23T17:49:28.779498+0000","last_fullsized":"2024-03-23T17:49:28.779498+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:14:34.327705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.922478+0000","last_change":"2024-03-23T17:48:56.230530+0000","last_active":"2024-03-23T17:49:28.922478+0000","last_peered":"2024-03-23T17:49:28.922478+0000","last_clean":"2024-03-23T17:49:28.922478+0000","last_became_active":"2024-03-23T17:48:56.230224+0000","last_became_peered":"2024-03-23T17:48:56.230224+0000","last_unstale":"2024-03-23T17:49:28.922478+0000","last_undegraded":"2024-03-23T17:49:28.922478+0000","last_fullsized":"2024-03-23T17:49:28.922478+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:10:28.031956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.781514+0000","last_change":"2024-03-23T17:48:56.229673+0000","last_active":"2024-03-23T17:49:28.781514+0000","last_peered":"2024-03-23T17:49:28.781514+0000","last_clean":"2024-03-23T17:49:28.781514+0000","last_became_active":"2024-03-23T17:48:56.229000+0000","last_became_peered":"2024-03-23T17:48:56.229000+0000","last_unstale":"2024-03-23T17:49:28.781514+0000","last_undegraded":"2024-03-23T17:49:28.781514+0000","last_fullsized":"2024-03-23T17:49:28.781514+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:23:28.057256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.781035+0000","last_change":"2024-03-23T17:48:56.224508+0000","last_active":"2024-03-23T17:49:28.781035+0000","last_peered":"2024-03-23T17:49:28.781035+0000","last_clean":"2024-03-23T17:49:28.781035+0000","last_became_active":"2024-03-23T17:48:56.224172+0000","last_became_peered":"2024-03-23T17:48:56.224172+0000","last_unstale":"2024-03-23T17:49:28.781035+0000","last_undegraded":"2024-03-23T17:49:28.781035+0000","last_fullsized":"2024-03-23T17:49:28.781035+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:03:57.063472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.784254+0000","last_change":"2024-03-23T17:48:56.234978+0000","last_active":"2024-03-23T17:49:28.784254+0000","last_peered":"2024-03-23T17:49:28.784254+0000","last_clean":"2024-03-23T17:49:28.784254+0000","last_became_active":"2024-03-23T17:48:56.234641+0000","last_became_peered":"2024-03-23T17:48:56.234641+0000","last_unstale":"2024-03-23T17:49:28.784254+0000","last_undegraded":"2024-03-23T17:49:28.784254+0000","last_fullsized":"2024-03-23T17:49:28.784254+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:34:30.745991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.777608+0000","last_change":"2024-03-23T17:48:56.218534+0000","last_active":"2024-03-23T17:49:28.777608+0000","last_peered":"2024-03-23T17:49:28.777608+0000","last_clean":"2024-03-23T17:49:28.777608+0000","last_became_active":"2024-03-23T17:48:56.217424+0000","last_became_peered":"2024-03-23T17:48:56.217424+0000","last_unstale":"2024-03-23T17:49:28.777608+0000","last_undegraded":"2024-03-23T17:49:28.777608+0000","last_fullsized":"2024-03-23T17:49:28.777608+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:32:05.618057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.783444+0000","last_change":"2024-03-23T17:48:56.235525+0000","last_active":"2024-03-23T17:49:28.783444+0000","last_peered":"2024-03-23T17:49:28.783444+0000","last_clean":"2024-03-23T17:49:28.783444+0000","last_became_active":"2024-03-23T17:48:56.234591+0000","last_became_peered":"2024-03-23T17:48:56.234591+0000","last_unstale":"2024-03-23T17:49:28.783444+0000","last_undegraded":"2024-03-23T17:49:28.783444+0000","last_fullsized":"2024-03-23T17:49:28.783444+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:19:38.381878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.783446+0000","last_change":"2024-03-23T17:48:56.243137+0000","last_active":"2024-03-23T17:49:28.783446+0000","last_peered":"2024-03-23T17:49:28.783446+0000","last_clean":"2024-03-23T17:49:28.783446+0000","last_became_active":"2024-03-23T17:48:56.242516+0000","last_became_peered":"2024-03-23T17:48:56.242516+0000","last_unstale":"2024-03-23T17:49:28.783446+0000","last_undegraded":"2024-03-23T17:49:28.783446+0000","last_fullsized":"2024-03-23T17:49:28.783446+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:54:59.849467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.922041+0000","last_change":"2024-03-23T17:48:56.236026+0000","last_active":"2024-03-23T17:49:28.922041+0000","last_peered":"2024-03-23T17:49:28.922041+0000","last_clean":"2024-03-23T17:49:28.922041+0000","last_became_active":"2024-03-23T17:48:56.235645+0000","last_became_peered":"2024-03-23T17:48:56.235645+0000","last_unstale":"2024-03-23T17:49:28.922041+0000","last_undegraded":"2024-03-23T17:49:28.922041+0000","last_fullsized":"2024-03-23T17:49:28.922041+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:47:01.076737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.782865+0000","last_change":"2024-03-23T17:48:56.239689+0000","last_active":"2024-03-23T17:49:28.782865+0000","last_peered":"2024-03-23T17:49:28.782865+0000","last_clean":"2024-03-23T17:49:28.782865+0000","last_became_active":"2024-03-23T17:48:56.239331+0000","last_became_peered":"2024-03-23T17:48:56.239331+0000","last_unstale":"2024-03-23T17:49:28.782865+0000","last_undegraded":"2024-03-23T17:49:28.782865+0000","last_fullsized":"2024-03-23T17:49:28.782865+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:21:15.287627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.777987+0000","last_change":"2024-03-23T17:48:56.229934+0000","last_active":"2024-03-23T17:49:28.777987+0000","last_peered":"2024-03-23T17:49:28.777987+0000","last_clean":"2024-03-23T17:49:28.777987+0000","last_became_active":"2024-03-23T17:48:56.229244+0000","last_became_peered":"2024-03-23T17:48:56.229244+0000","last_unstale":"2024-03-23T17:49:28.777987+0000","last_undegraded":"2024-03-23T17:49:28.777987+0000","last_fullsized":"2024-03-23T17:49:28.777987+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:02:40.254799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.784936+0000","last_change":"2024-03-23T17:48:56.233786+0000","last_active":"2024-03-23T17:49:28.784936+0000","last_peered":"2024-03-23T17:49:28.784936+0000","last_clean":"2024-03-23T17:49:28.784936+0000","last_became_active":"2024-03-23T17:48:56.233137+0000","last_became_peered":"2024-03-23T17:48:56.233137+0000","last_unstale":"2024-03-23T17:49:28.784936+0000","last_undegraded":"2024-03-23T17:49:28.784936+0000","last_fullsized":"2024-03-23T17:49:28.784936+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:33:35.202788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'13","reported_seq":67,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-23T17:49:40.759669+0000","last_change":"2024-03-23T17:48:55.712184+0000","last_active":"2024-03-23T17:49:40.759669+0000","last_peered":"2024-03-23T17:49:40.759669+0000","last_clean":"2024-03-23T17:49:40.759669+0000","last_became_active":"2024-03-23T17:48:55.711860+0000","last_became_peered":"2024-03-23T17:48:55.711860+0000","last_unstale":"2024-03-23T17:49:40.759669+0000","last_undegraded":"2024-03-23T17:49:40.759669+0000","last_fullsized":"2024-03-23T17:49:40.759669+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:59:01.133652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":40,"num_read_kb":40,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-23T17:49:29.789705+0000","last_change":"2024-03-23T17:49:29.789057+0000","last_active":"2024-03-23T17:49:29.789705+0000","last_peered":"2024-03-23T17:49:29.789705+0000","last_clean":"2024-03-23T17:49:29.789705+0000","last_became_active":"2024-03-23T17:49:29.788796+0000","last_became_peered":"2024-03-23T17:49:29.788796+0000","last_unstale":"2024-03-23T17:49:29.789705+0000","last_undegraded":"2024-03-23T17:49:29.789705+0000","last_fullsized":"2024-03-23T17:49:29.789705+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:30:53.062926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.780493+0000","last_change":"2024-03-23T17:48:55.725351+0000","last_active":"2024-03-23T17:49:28.780493+0000","last_peered":"2024-03-23T17:49:28.780493+0000","last_clean":"2024-03-23T17:49:28.780493+0000","last_became_active":"2024-03-23T17:48:55.724566+0000","last_became_peered":"2024-03-23T17:48:55.724566+0000","last_unstale":"2024-03-23T17:49:28.780493+0000","last_undegraded":"2024-03-23T17:49:28.780493+0000","last_fullsized":"2024-03-23T17:49:28.780493+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:58:21.088166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.784151+0000","last_change":"2024-03-23T17:48:56.243526+0000","last_active":"2024-03-23T17:49:28.784151+0000","last_peered":"2024-03-23T17:49:28.784151+0000","last_clean":"2024-03-23T17:49:28.784151+0000","last_became_active":"2024-03-23T17:48:56.242468+0000","last_became_peered":"2024-03-23T17:48:56.242468+0000","last_unstale":"2024-03-23T17:49:28.784151+0000","last_undegraded":"2024-03-23T17:49:28.784151+0000","last_fullsized":"2024-03-23T17:49:28.784151+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:16:50.392770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.781056+0000","last_change":"2024-03-23T17:48:55.720237+0000","last_active":"2024-03-23T17:49:28.781056+0000","last_peered":"2024-03-23T17:49:28.781056+0000","last_clean":"2024-03-23T17:49:28.781056+0000","last_became_active":"2024-03-23T17:48:55.719843+0000","last_became_peered":"2024-03-23T17:48:55.719843+0000","last_unstale":"2024-03-23T17:49:28.781056+0000","last_undegraded":"2024-03-23T17:49:28.781056+0000","last_fullsized":"2024-03-23T17:49:28.781056+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:07:26.973511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.922177+0000","last_change":"2024-03-23T17:48:56.235642+0000","last_active":"2024-03-23T17:49:28.922177+0000","last_peered":"2024-03-23T17:49:28.922177+0000","last_clean":"2024-03-23T17:49:28.922177+0000","last_became_active":"2024-03-23T17:48:56.235326+0000","last_became_peered":"2024-03-23T17:48:56.235326+0000","last_unstale":"2024-03-23T17:49:28.922177+0000","last_undegraded":"2024-03-23T17:49:28.922177+0000","last_fullsized":"2024-03-23T17:49:28.922177+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:21:59.934620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.781362+0000","last_change":"2024-03-23T17:48:55.718579+0000","last_active":"2024-03-23T17:49:28.781362+0000","last_peered":"2024-03-23T17:49:28.781362+0000","last_clean":"2024-03-23T17:49:28.781362+0000","last_became_active":"2024-03-23T17:48:55.718220+0000","last_became_peered":"2024-03-23T17:48:55.718220+0000","last_unstale":"2024-03-23T17:49:28.781362+0000","last_undegraded":"2024-03-23T17:49:28.781362+0000","last_fullsized":"2024-03-23T17:49:28.781362+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:37:56.417367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.785628+0000","last_change":"2024-03-23T17:48:56.224842+0000","last_active":"2024-03-23T17:49:28.785628+0000","last_peered":"2024-03-23T17:49:28.785628+0000","last_clean":"2024-03-23T17:49:28.785628+0000","last_became_active":"2024-03-23T17:48:56.224427+0000","last_became_peered":"2024-03-23T17:48:56.224427+0000","last_unstale":"2024-03-23T17:49:28.785628+0000","last_undegraded":"2024-03-23T17:49:28.785628+0000","last_fullsized":"2024-03-23T17:49:28.785628+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:16:22.684369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"80'5","reported_seq":58,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-23T17:49:35.712872+0000","last_change":"2024-03-23T17:48:55.713996+0000","last_active":"2024-03-23T17:49:35.712872+0000","last_peered":"2024-03-23T17:49:35.712872+0000","last_clean":"2024-03-23T17:49:35.712872+0000","last_became_active":"2024-03-23T17:48:55.713647+0000","last_became_peered":"2024-03-23T17:48:55.713647+0000","last_unstale":"2024-03-23T17:49:35.712872+0000","last_undegraded":"2024-03-23T17:49:35.712872+0000","last_fullsized":"2024-03-23T17:49:35.712872+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:12:22.251533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":39,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.781148+0000","last_change":"2024-03-23T17:48:56.219281+0000","last_active":"2024-03-23T17:49:28.781148+0000","last_peered":"2024-03-23T17:49:28.781148+0000","last_clean":"2024-03-23T17:49:28.781148+0000","last_became_active":"2024-03-23T17:48:56.218924+0000","last_became_peered":"2024-03-23T17:48:56.218924+0000","last_unstale":"2024-03-23T17:49:28.781148+0000","last_undegraded":"2024-03-23T17:49:28.781148+0000","last_fullsized":"2024-03-23T17:49:28.781148+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:49:04.793088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.781333+0000","last_change":"2024-03-23T17:48:55.718131+0000","last_active":"2024-03-23T17:49:28.781333+0000","last_peered":"2024-03-23T17:49:28.781333+0000","last_clean":"2024-03-23T17:49:28.781333+0000","last_became_active":"2024-03-23T17:48:55.717760+0000","last_became_peered":"2024-03-23T17:48:55.717760+0000","last_unstale":"2024-03-23T17:49:28.781333+0000","last_undegraded":"2024-03-23T17:49:28.781333+0000","last_fullsized":"2024-03-23T17:49:28.781333+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:16:54.355844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.784309+0000","last_change":"2024-03-23T17:48:56.225965+0000","last_active":"2024-03-23T17:49:28.784309+0000","last_peered":"2024-03-23T17:49:28.784309+0000","last_clean":"2024-03-23T17:49:28.784309+0000","last_became_active":"2024-03-23T17:48:56.225101+0000","last_became_peered":"2024-03-23T17:48:56.225101+0000","last_unstale":"2024-03-23T17:49:28.784309+0000","last_undegraded":"2024-03-23T17:49:28.784309+0000","last_fullsized":"2024-03-23T17:49:28.784309+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:52:14.302306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.776847+0000","last_change":"2024-03-23T17:48:55.722289+0000","last_active":"2024-03-23T17:49:28.776847+0000","last_peered":"2024-03-23T17:49:28.776847+0000","last_clean":"2024-03-23T17:49:28.776847+0000","last_became_active":"2024-03-23T17:48:55.721927+0000","last_became_peered":"2024-03-23T17:48:55.721927+0000","last_unstale":"2024-03-23T17:49:28.776847+0000","last_undegraded":"2024-03-23T17:49:28.776847+0000","last_fullsized":"2024-03-23T17:49:28.776847+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:20:08.354897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.779465+0000","last_change":"2024-03-23T17:48:56.232231+0000","last_active":"2024-03-23T17:49:28.779465+0000","last_peered":"2024-03-23T17:49:28.779465+0000","last_clean":"2024-03-23T17:49:28.779465+0000","last_became_active":"2024-03-23T17:48:56.227335+0000","last_became_peered":"2024-03-23T17:48:56.227335+0000","last_unstale":"2024-03-23T17:49:28.779465+0000","last_undegraded":"2024-03-23T17:49:28.779465+0000","last_fullsized":"2024-03-23T17:49:28.779465+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:49:01.460775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.785071+0000","last_change":"2024-03-23T17:48:55.717615+0000","last_active":"2024-03-23T17:49:28.785071+0000","last_peered":"2024-03-23T17:49:28.785071+0000","last_clean":"2024-03-23T17:49:28.785071+0000","last_became_active":"2024-03-23T17:48:55.717266+0000","last_became_peered":"2024-03-23T17:48:55.717266+0000","last_unstale":"2024-03-23T17:49:28.785071+0000","last_undegraded":"2024-03-23T17:49:28.785071+0000","last_fullsized":"2024-03-23T17:49:28.785071+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:40:07.988182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.776405+0000","last_change":"2024-03-23T17:48:56.229787+0000","last_active":"2024-03-23T17:49:28.776405+0000","last_peered":"2024-03-23T17:49:28.776405+0000","last_clean":"2024-03-23T17:49:28.776405+0000","last_became_active":"2024-03-23T17:48:56.229471+0000","last_became_peered":"2024-03-23T17:48:56.229471+0000","last_unstale":"2024-03-23T17:49:28.776405+0000","last_undegraded":"2024-03-23T17:49:28.776405+0000","last_fullsized":"2024-03-23T17:49:28.776405+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:44:09.358809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'4","reported_seq":93,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-23T17:49:38.717008+0000","last_change":"2024-03-23T17:48:55.717130+0000","last_active":"2024-03-23T17:49:38.717008+0000","last_peered":"2024-03-23T17:49:38.717008+0000","last_clean":"2024-03-23T17:49:38.717008+0000","last_became_active":"2024-03-23T17:48:55.716733+0000","last_became_peered":"2024-03-23T17:48:55.716733+0000","last_unstale":"2024-03-23T17:49:38.717008+0000","last_undegraded":"2024-03-23T17:49:38.717008+0000","last_fullsized":"2024-03-23T17:49:38.717008+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:22:43.229694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":75,"num_read_kb":75,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.922446+0000","last_change":"2024-03-23T17:48:56.235565+0000","last_active":"2024-03-23T17:49:28.922446+0000","last_peered":"2024-03-23T17:49:28.922446+0000","last_clean":"2024-03-23T17:49:28.922446+0000","last_became_active":"2024-03-23T17:48:56.235267+0000","last_became_peered":"2024-03-23T17:48:56.235267+0000","last_unstale":"2024-03-23T17:49:28.922446+0000","last_undegraded":"2024-03-23T17:49:28.922446+0000","last_fullsized":"2024-03-23T17:49:28.922446+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:38:45.906545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.780900+0000","last_change":"2024-03-23T17:48:55.719096+0000","last_active":"2024-03-23T17:49:28.780900+0000","last_peered":"2024-03-23T17:49:28.780900+0000","last_clean":"2024-03-23T17:49:28.780900+0000","last_became_active":"2024-03-23T17:48:55.718721+0000","last_became_peered":"2024-03-23T17:48:55.718721+0000","last_unstale":"2024-03-23T17:49:28.780900+0000","last_undegraded":"2024-03-23T17:49:28.780900+0000","last_fullsized":"2024-03-23T17:49:28.780900+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:43:05.532263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.784108+0000","last_change":"2024-03-23T17:48:56.235549+0000","last_active":"2024-03-23T17:49:28.784108+0000","last_peered":"2024-03-23T17:49:28.784108+0000","last_clean":"2024-03-23T17:49:28.784108+0000","last_became_active":"2024-03-23T17:48:56.234572+0000","last_became_peered":"2024-03-23T17:48:56.234572+0000","last_unstale":"2024-03-23T17:49:28.784108+0000","last_undegraded":"2024-03-23T17:49:28.784108+0000","last_fullsized":"2024-03-23T17:49:28.784108+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:19:58.639181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.785732+0000","last_change":"2024-03-23T17:48:55.724688+0000","last_active":"2024-03-23T17:49:28.785732+0000","last_peered":"2024-03-23T17:49:28.785732+0000","last_clean":"2024-03-23T17:49:28.785732+0000","last_became_active":"2024-03-23T17:48:55.724385+0000","last_became_peered":"2024-03-23T17:48:55.724385+0000","last_unstale":"2024-03-23T17:49:28.785732+0000","last_undegraded":"2024-03-23T17:49:28.785732+0000","last_fullsized":"2024-03-23T17:49:28.785732+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:17:54.338059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.922287+0000","last_change":"2024-03-23T17:48:56.242983+0000","last_active":"2024-03-23T17:49:28.922287+0000","last_peered":"2024-03-23T17:49:28.922287+0000","last_clean":"2024-03-23T17:49:28.922287+0000","last_became_active":"2024-03-23T17:48:56.242596+0000","last_became_peered":"2024-03-23T17:48:56.242596+0000","last_unstale":"2024-03-23T17:49:28.922287+0000","last_undegraded":"2024-03-23T17:49:28.922287+0000","last_fullsized":"2024-03-23T17:49:28.922287+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:55:51.971834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'29","reported_seq":128,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-23T17:49:38.730186+0000","last_change":"2024-03-23T17:48:55.717631+0000","last_active":"2024-03-23T17:49:38.730186+0000","last_peered":"2024-03-23T17:49:38.730186+0000","last_clean":"2024-03-23T17:49:38.730186+0000","last_became_active":"2024-03-23T17:48:55.717082+0000","last_became_peered":"2024-03-23T17:48:55.717082+0000","last_unstale":"2024-03-23T17:49:38.730186+0000","last_undegraded":"2024-03-23T17:49:38.730186+0000","last_fullsized":"2024-03-23T17:49:38.730186+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:36:25.323269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59418,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":85,"num_read_kb":100,"num_write":31,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.780792+0000","last_change":"2024-03-23T17:48:56.226174+0000","last_active":"2024-03-23T17:49:28.780792+0000","last_peered":"2024-03-23T17:49:28.780792+0000","last_clean":"2024-03-23T17:49:28.780792+0000","last_became_active":"2024-03-23T17:48:56.225834+0000","last_became_peered":"2024-03-23T17:48:56.225834+0000","last_unstale":"2024-03-23T17:49:28.780792+0000","last_undegraded":"2024-03-23T17:49:28.780792+0000","last_fullsized":"2024-03-23T17:49:28.780792+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:50:16.793071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-23T17:49:29.804494+0000","last_change":"2024-03-23T17:49:29.804099+0000","last_active":"2024-03-23T17:49:29.804494+0000","last_peered":"2024-03-23T17:49:29.804494+0000","last_clean":"2024-03-23T17:49:29.804494+0000","last_became_active":"2024-03-23T17:49:29.787483+0000","last_became_peered":"2024-03-23T17:49:29.787483+0000","last_unstale":"2024-03-23T17:49:29.804494+0000","last_undegraded":"2024-03-23T17:49:29.804494+0000","last_fullsized":"2024-03-23T17:49:29.804494+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:13:16.463738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.776667+0000","last_change":"2024-03-23T17:48:56.228459+0000","last_active":"2024-03-23T17:49:28.776667+0000","last_peered":"2024-03-23T17:49:28.776667+0000","last_clean":"2024-03-23T17:49:28.776667+0000","last_became_active":"2024-03-23T17:48:56.228179+0000","last_became_peered":"2024-03-23T17:48:56.228179+0000","last_unstale":"2024-03-23T17:49:28.776667+0000","last_undegraded":"2024-03-23T17:49:28.776667+0000","last_fullsized":"2024-03-23T17:49:28.776667+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:28:41.613432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.785318+0000","last_change":"2024-03-23T17:48:55.721533+0000","last_active":"2024-03-23T17:49:28.785318+0000","last_peered":"2024-03-23T17:49:28.785318+0000","last_clean":"2024-03-23T17:49:28.785318+0000","last_became_active":"2024-03-23T17:48:55.721100+0000","last_became_peered":"2024-03-23T17:48:55.721100+0000","last_unstale":"2024-03-23T17:49:28.785318+0000","last_undegraded":"2024-03-23T17:49:28.785318+0000","last_fullsized":"2024-03-23T17:49:28.785318+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:56:58.112649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-23T17:49:30.003965+0000","last_change":"2024-03-23T17:49:30.003965+0000","last_active":"2024-03-23T17:49:30.003965+0000","last_peered":"2024-03-23T17:49:30.003965+0000","last_clean":"2024-03-23T17:49:30.003965+0000","last_became_active":"2024-03-23T17:49:30.003643+0000","last_became_peered":"2024-03-23T17:49:30.003643+0000","last_unstale":"2024-03-23T17:49:30.003965+0000","last_undegraded":"2024-03-23T17:49:30.003965+0000","last_fullsized":"2024-03-23T17:49:30.003965+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:47:39.892782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'5","reported_seq":57,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-23T17:49:37.757640+0000","last_change":"2024-03-23T17:48:55.715205+0000","last_active":"2024-03-23T17:49:37.757640+0000","last_peered":"2024-03-23T17:49:37.757640+0000","last_clean":"2024-03-23T17:49:37.757640+0000","last_became_active":"2024-03-23T17:48:55.714810+0000","last_became_peered":"2024-03-23T17:48:55.714810+0000","last_unstale":"2024-03-23T17:49:37.757640+0000","last_undegraded":"2024-03-23T17:49:37.757640+0000","last_fullsized":"2024-03-23T17:49:37.757640+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:40:08.768998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":39,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-23T17:49:29.783025+0000","last_change":"2024-03-23T17:49:29.783025+0000","last_active":"2024-03-23T17:49:29.783025+0000","last_peered":"2024-03-23T17:49:29.783025+0000","last_clean":"2024-03-23T17:49:29.783025+0000","last_became_active":"2024-03-23T17:49:29.782541+0000","last_became_peered":"2024-03-23T17:49:29.782541+0000","last_unstale":"2024-03-23T17:49:29.783025+0000","last_undegraded":"2024-03-23T17:49:29.783025+0000","last_fullsized":"2024-03-23T17:49:29.783025+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:50:29.754074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":59,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-23T17:49:30.616960+0000","last_change":"2024-03-23T17:49:30.616960+0000","last_active":"2024-03-23T17:49:30.616960+0000","last_peered":"2024-03-23T17:49:30.616960+0000","last_clean":"2024-03-23T17:49:30.616960+0000","last_became_active":"2024-03-23T17:49:30.604464+0000","last_became_peered":"2024-03-23T17:49:30.604464+0000","last_unstale":"2024-03-23T17:49:30.616960+0000","last_undegraded":"2024-03-23T17:49:30.616960+0000","last_fullsized":"2024-03-23T17:49:30.616960+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:24:13.862841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.785079+0000","last_change":"2024-03-23T17:48:56.225624+0000","last_active":"2024-03-23T17:49:28.785079+0000","last_peered":"2024-03-23T17:49:28.785079+0000","last_clean":"2024-03-23T17:49:28.785079+0000","last_became_active":"2024-03-23T17:48:56.223798+0000","last_became_peered":"2024-03-23T17:48:56.223798+0000","last_unstale":"2024-03-23T17:49:28.785079+0000","last_undegraded":"2024-03-23T17:49:28.785079+0000","last_fullsized":"2024-03-23T17:49:28.785079+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:00:20.547553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.781323+0000","last_change":"2024-03-23T17:48:55.717792+0000","last_active":"2024-03-23T17:49:28.781323+0000","last_peered":"2024-03-23T17:49:28.781323+0000","last_clean":"2024-03-23T17:49:28.781323+0000","last_became_active":"2024-03-23T17:48:55.717479+0000","last_became_peered":"2024-03-23T17:48:55.717479+0000","last_unstale":"2024-03-23T17:49:28.781323+0000","last_undegraded":"2024-03-23T17:49:28.781323+0000","last_fullsized":"2024-03-23T17:49:28.781323+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:49:43.483785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.782572+0000","last_change":"2024-03-23T17:48:56.219135+0000","last_active":"2024-03-23T17:49:28.782572+0000","last_peered":"2024-03-23T17:49:28.782572+0000","last_clean":"2024-03-23T17:49:28.782572+0000","last_became_active":"2024-03-23T17:48:56.218824+0000","last_became_peered":"2024-03-23T17:48:56.218824+0000","last_unstale":"2024-03-23T17:49:28.782572+0000","last_undegraded":"2024-03-23T17:49:28.782572+0000","last_fullsized":"2024-03-23T17:49:28.782572+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:20:35.190781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.776774+0000","last_change":"2024-03-23T17:48:55.717530+0000","last_active":"2024-03-23T17:49:28.776774+0000","last_peered":"2024-03-23T17:49:28.776774+0000","last_clean":"2024-03-23T17:49:28.776774+0000","last_became_active":"2024-03-23T17:48:55.717213+0000","last_became_peered":"2024-03-23T17:48:55.717213+0000","last_unstale":"2024-03-23T17:49:28.776774+0000","last_undegraded":"2024-03-23T17:49:28.776774+0000","last_fullsized":"2024-03-23T17:49:28.776774+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:40:09.803021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'1","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.781617+0000","last_change":"2024-03-23T17:48:56.222237+0000","last_active":"2024-03-23T17:49:28.781617+0000","last_peered":"2024-03-23T17:49:28.781617+0000","last_clean":"2024-03-23T17:49:28.781617+0000","last_became_active":"2024-03-23T17:48:56.221916+0000","last_became_peered":"2024-03-23T17:48:56.221916+0000","last_unstale":"2024-03-23T17:49:28.781617+0000","last_undegraded":"2024-03-23T17:49:28.781617+0000","last_fullsized":"2024-03-23T17:49:28.781617+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:35:59.213482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'5","reported_seq":58,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-23T17:49:39.758958+0000","last_change":"2024-03-23T17:48:55.718143+0000","last_active":"2024-03-23T17:49:39.758958+0000","last_peered":"2024-03-23T17:49:39.758958+0000","last_clean":"2024-03-23T17:49:39.758958+0000","last_became_active":"2024-03-23T17:48:55.717834+0000","last_became_peered":"2024-03-23T17:48:55.717834+0000","last_unstale":"2024-03-23T17:49:39.758958+0000","last_undegraded":"2024-03-23T17:49:39.758958+0000","last_fullsized":"2024-03-23T17:49:39.758958+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:29:08.879636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":39,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.781385+0000","last_change":"2024-03-23T17:48:56.237136+0000","last_active":"2024-03-23T17:49:28.781385+0000","last_peered":"2024-03-23T17:49:28.781385+0000","last_clean":"2024-03-23T17:49:28.781385+0000","last_became_active":"2024-03-23T17:48:56.236795+0000","last_became_peered":"2024-03-23T17:48:56.236795+0000","last_unstale":"2024-03-23T17:49:28.781385+0000","last_undegraded":"2024-03-23T17:49:28.781385+0000","last_fullsized":"2024-03-23T17:49:28.781385+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:15:31.356382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-23T17:49:29.789433+0000","last_change":"2024-03-23T17:49:29.789433+0000","last_active":"2024-03-23T17:49:29.789433+0000","last_peered":"2024-03-23T17:49:29.789433+0000","last_clean":"2024-03-23T17:49:29.789433+0000","last_became_active":"2024-03-23T17:49:29.788956+0000","last_became_peered":"2024-03-23T17:49:29.788956+0000","last_unstale":"2024-03-23T17:49:29.789433+0000","last_undegraded":"2024-03-23T17:49:29.789433+0000","last_fullsized":"2024-03-23T17:49:29.789433+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:43:30.929796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"80'5","reported_seq":56,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-23T17:49:37.757029+0000","last_change":"2024-03-23T17:48:55.722968+0000","last_active":"2024-03-23T17:49:37.757029+0000","last_peered":"2024-03-23T17:49:37.757029+0000","last_clean":"2024-03-23T17:49:37.757029+0000","last_became_active":"2024-03-23T17:48:55.722578+0000","last_became_peered":"2024-03-23T17:48:55.722578+0000","last_unstale":"2024-03-23T17:49:37.757029+0000","last_undegraded":"2024-03-23T17:49:37.757029+0000","last_fullsized":"2024-03-23T17:49:37.757029+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:45:42.344283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":37,"num_read_kb":37,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.779906+0000","last_change":"2024-03-23T17:48:56.228000+0000","last_active":"2024-03-23T17:49:28.779906+0000","last_peered":"2024-03-23T17:49:28.779906+0000","last_clean":"2024-03-23T17:49:28.779906+0000","last_became_active":"2024-03-23T17:48:56.225645+0000","last_became_peered":"2024-03-23T17:48:56.225645+0000","last_unstale":"2024-03-23T17:49:28.779906+0000","last_undegraded":"2024-03-23T17:49:28.779906+0000","last_fullsized":"2024-03-23T17:49:28.779906+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:44:25.197761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.784417+0000","last_change":"2024-03-23T17:48:55.718307+0000","last_active":"2024-03-23T17:49:28.784417+0000","last_peered":"2024-03-23T17:49:28.784417+0000","last_clean":"2024-03-23T17:49:28.784417+0000","last_became_active":"2024-03-23T17:48:55.717839+0000","last_became_peered":"2024-03-23T17:48:55.717839+0000","last_unstale":"2024-03-23T17:49:28.784417+0000","last_undegraded":"2024-03-23T17:49:28.784417+0000","last_fullsized":"2024-03-23T17:49:28.784417+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:11:56.053940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":335,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.780763+0000","last_change":"2024-03-23T17:48:01.618019+0000","last_active":"2024-03-23T17:49:28.780763+0000","last_peered":"2024-03-23T17:49:28.780763+0000","last_clean":"2024-03-23T17:49:28.780763+0000","last_became_active":"2024-03-23T17:48:01.617553+0000","last_became_peered":"2024-03-23T17:48:01.617553+0000","last_unstale":"2024-03-23T17:49:28.780763+0000","last_undegraded":"2024-03-23T17:49:28.780763+0000","last_fullsized":"2024-03-23T17:49:28.780763+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:45:26.849029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:45:26.849029+0000","last_clean_scrub_stamp":"2024-03-23T17:45:26.849029+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:05:44.823364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.776693+0000","last_change":"2024-03-23T17:48:56.224285+0000","last_active":"2024-03-23T17:49:28.776693+0000","last_peered":"2024-03-23T17:49:28.776693+0000","last_clean":"2024-03-23T17:49:28.776693+0000","last_became_active":"2024-03-23T17:48:56.220943+0000","last_became_peered":"2024-03-23T17:48:56.220943+0000","last_unstale":"2024-03-23T17:49:28.776693+0000","last_undegraded":"2024-03-23T17:49:28.776693+0000","last_fullsized":"2024-03-23T17:49:28.776693+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:36:05.458243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.780994+0000","last_change":"2024-03-23T17:48:55.715877+0000","last_active":"2024-03-23T17:49:28.780994+0000","last_peered":"2024-03-23T17:49:28.780994+0000","last_clean":"2024-03-23T17:49:28.780994+0000","last_became_active":"2024-03-23T17:48:55.715550+0000","last_became_peered":"2024-03-23T17:48:55.715550+0000","last_unstale":"2024-03-23T17:49:28.780994+0000","last_undegraded":"2024-03-23T17:49:28.780994+0000","last_fullsized":"2024-03-23T17:49:28.780994+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:31:57.668495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.780911+0000","last_change":"2024-03-23T17:48:55.724618+0000","last_active":"2024-03-23T17:49:28.780911+0000","last_peered":"2024-03-23T17:49:28.780911+0000","last_clean":"2024-03-23T17:49:28.780911+0000","last_became_active":"2024-03-23T17:48:55.724278+0000","last_became_peered":"2024-03-23T17:48:55.724278+0000","last_unstale":"2024-03-23T17:49:28.780911+0000","last_undegraded":"2024-03-23T17:49:28.780911+0000","last_fullsized":"2024-03-23T17:49:28.780911+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:27:05.065115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.776532+0000","last_change":"2024-03-23T17:48:56.216141+0000","last_active":"2024-03-23T17:49:28.776532+0000","last_peered":"2024-03-23T17:49:28.776532+0000","last_clean":"2024-03-23T17:49:28.776532+0000","last_became_active":"2024-03-23T17:48:56.215653+0000","last_became_peered":"2024-03-23T17:48:56.215653+0000","last_unstale":"2024-03-23T17:49:28.776532+0000","last_undegraded":"2024-03-23T17:49:28.776532+0000","last_fullsized":"2024-03-23T17:49:28.776532+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:02:19.611056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'9","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-23T17:49:29.794454+0000","last_change":"2024-03-23T17:49:29.794454+0000","last_active":"2024-03-23T17:49:29.794454+0000","last_peered":"2024-03-23T17:49:29.794454+0000","last_clean":"2024-03-23T17:49:29.794454+0000","last_became_active":"2024-03-23T17:49:29.787238+0000","last_became_peered":"2024-03-23T17:49:29.787238+0000","last_unstale":"2024-03-23T17:49:29.794454+0000","last_undegraded":"2024-03-23T17:49:29.794454+0000","last_fullsized":"2024-03-23T17:49:29.794454+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:33:34.917283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.780480+0000","last_change":"2024-03-23T17:48:56.228439+0000","last_active":"2024-03-23T17:49:28.780480+0000","last_peered":"2024-03-23T17:49:28.780480+0000","last_clean":"2024-03-23T17:49:28.780480+0000","last_became_active":"2024-03-23T17:48:56.226677+0000","last_became_peered":"2024-03-23T17:48:56.226677+0000","last_unstale":"2024-03-23T17:49:28.780480+0000","last_undegraded":"2024-03-23T17:49:28.780480+0000","last_fullsized":"2024-03-23T17:49:28.780480+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:46:00.063266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.784329+0000","last_change":"2024-03-23T17:48:55.724872+0000","last_active":"2024-03-23T17:49:28.784329+0000","last_peered":"2024-03-23T17:49:28.784329+0000","last_clean":"2024-03-23T17:49:28.784329+0000","last_became_active":"2024-03-23T17:48:55.724468+0000","last_became_peered":"2024-03-23T17:48:55.724468+0000","last_unstale":"2024-03-23T17:49:28.784329+0000","last_undegraded":"2024-03-23T17:49:28.784329+0000","last_fullsized":"2024-03-23T17:49:28.784329+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:28:34.918056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.783623+0000","last_change":"2024-03-23T17:48:56.233730+0000","last_active":"2024-03-23T17:49:28.783623+0000","last_peered":"2024-03-23T17:49:28.783623+0000","last_clean":"2024-03-23T17:49:28.783623+0000","last_became_active":"2024-03-23T17:48:56.233189+0000","last_became_peered":"2024-03-23T17:48:56.233189+0000","last_unstale":"2024-03-23T17:49:28.783623+0000","last_undegraded":"2024-03-23T17:49:28.783623+0000","last_fullsized":"2024-03-23T17:49:28.783623+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:20:30.764344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.922070+0000","last_change":"2024-03-23T17:48:55.724633+0000","last_active":"2024-03-23T17:49:28.922070+0000","last_peered":"2024-03-23T17:49:28.922070+0000","last_clean":"2024-03-23T17:49:28.922070+0000","last_became_active":"2024-03-23T17:48:55.724301+0000","last_became_peered":"2024-03-23T17:48:55.724301+0000","last_unstale":"2024-03-23T17:49:28.922070+0000","last_undegraded":"2024-03-23T17:49:28.922070+0000","last_fullsized":"2024-03-23T17:49:28.922070+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:45:08.392668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.780692+0000","last_change":"2024-03-23T17:48:56.230981+0000","last_active":"2024-03-23T17:49:28.780692+0000","last_peered":"2024-03-23T17:49:28.780692+0000","last_clean":"2024-03-23T17:49:28.780692+0000","last_became_active":"2024-03-23T17:48:56.230644+0000","last_became_peered":"2024-03-23T17:48:56.230644+0000","last_unstale":"2024-03-23T17:49:28.780692+0000","last_undegraded":"2024-03-23T17:49:28.780692+0000","last_fullsized":"2024-03-23T17:49:28.780692+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:37:51.310938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.779873+0000","last_change":"2024-03-23T17:48:55.719112+0000","last_active":"2024-03-23T17:49:28.779873+0000","last_peered":"2024-03-23T17:49:28.779873+0000","last_clean":"2024-03-23T17:49:28.779873+0000","last_became_active":"2024-03-23T17:48:55.718756+0000","last_became_peered":"2024-03-23T17:48:55.718756+0000","last_unstale":"2024-03-23T17:49:28.779873+0000","last_undegraded":"2024-03-23T17:49:28.779873+0000","last_fullsized":"2024-03-23T17:49:28.779873+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:41:22.164706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.781982+0000","last_change":"2024-03-23T17:48:56.240289+0000","last_active":"2024-03-23T17:49:28.781982+0000","last_peered":"2024-03-23T17:49:28.781982+0000","last_clean":"2024-03-23T17:49:28.781982+0000","last_became_active":"2024-03-23T17:48:56.239896+0000","last_became_peered":"2024-03-23T17:48:56.239896+0000","last_unstale":"2024-03-23T17:49:28.781982+0000","last_undegraded":"2024-03-23T17:49:28.781982+0000","last_fullsized":"2024-03-23T17:49:28.781982+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:49:10.505576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'3","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.785515+0000","last_change":"2024-03-23T17:48:55.723330+0000","last_active":"2024-03-23T17:49:28.785515+0000","last_peered":"2024-03-23T17:49:28.785515+0000","last_clean":"2024-03-23T17:49:28.785515+0000","last_became_active":"2024-03-23T17:48:55.722879+0000","last_became_peered":"2024-03-23T17:48:55.722879+0000","last_unstale":"2024-03-23T17:49:28.785515+0000","last_undegraded":"2024-03-23T17:49:28.785515+0000","last_fullsized":"2024-03-23T17:49:28.785515+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:34:47.173411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.779841+0000","last_change":"2024-03-23T17:48:56.237042+0000","last_active":"2024-03-23T17:49:28.779841+0000","last_peered":"2024-03-23T17:49:28.779841+0000","last_clean":"2024-03-23T17:49:28.779841+0000","last_became_active":"2024-03-23T17:48:56.236722+0000","last_became_peered":"2024-03-23T17:48:56.236722+0000","last_unstale":"2024-03-23T17:49:28.779841+0000","last_undegraded":"2024-03-23T17:49:28.779841+0000","last_fullsized":"2024-03-23T17:49:28.779841+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:27:29.656120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.780900+0000","last_change":"2024-03-23T17:48:55.719223+0000","last_active":"2024-03-23T17:49:28.780900+0000","last_peered":"2024-03-23T17:49:28.780900+0000","last_clean":"2024-03-23T17:49:28.780900+0000","last_became_active":"2024-03-23T17:48:55.718864+0000","last_became_peered":"2024-03-23T17:48:55.718864+0000","last_unstale":"2024-03-23T17:49:28.780900+0000","last_undegraded":"2024-03-23T17:49:28.780900+0000","last_fullsized":"2024-03-23T17:49:28.780900+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:52:18.169557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.781086+0000","last_change":"2024-03-23T17:48:56.224885+0000","last_active":"2024-03-23T17:49:28.781086+0000","last_peered":"2024-03-23T17:49:28.781086+0000","last_clean":"2024-03-23T17:49:28.781086+0000","last_became_active":"2024-03-23T17:48:56.224160+0000","last_became_peered":"2024-03-23T17:48:56.224160+0000","last_unstale":"2024-03-23T17:49:28.781086+0000","last_undegraded":"2024-03-23T17:49:28.781086+0000","last_fullsized":"2024-03-23T17:49:28.781086+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:35:34.538313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'5","reported_seq":58,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-23T17:49:38.758323+0000","last_change":"2024-03-23T17:48:55.720518+0000","last_active":"2024-03-23T17:49:38.758323+0000","last_peered":"2024-03-23T17:49:38.758323+0000","last_clean":"2024-03-23T17:49:38.758323+0000","last_became_active":"2024-03-23T17:48:55.720045+0000","last_became_peered":"2024-03-23T17:48:55.720045+0000","last_unstale":"2024-03-23T17:49:38.758323+0000","last_undegraded":"2024-03-23T17:49:38.758323+0000","last_fullsized":"2024-03-23T17:49:38.758323+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:20:54.819406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":39,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.781093+0000","last_change":"2024-03-23T17:48:56.224171+0000","last_active":"2024-03-23T17:49:28.781093+0000","last_peered":"2024-03-23T17:49:28.781093+0000","last_clean":"2024-03-23T17:49:28.781093+0000","last_became_active":"2024-03-23T17:48:56.223737+0000","last_became_peered":"2024-03-23T17:48:56.223737+0000","last_unstale":"2024-03-23T17:49:28.781093+0000","last_undegraded":"2024-03-23T17:49:28.781093+0000","last_fullsized":"2024-03-23T17:49:28.781093+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:40:04.353945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'3","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.785531+0000","last_change":"2024-03-23T17:48:55.721734+0000","last_active":"2024-03-23T17:49:28.785531+0000","last_peered":"2024-03-23T17:49:28.785531+0000","last_clean":"2024-03-23T17:49:28.785531+0000","last_became_active":"2024-03-23T17:48:55.721417+0000","last_became_peered":"2024-03-23T17:48:55.721417+0000","last_unstale":"2024-03-23T17:49:28.785531+0000","last_undegraded":"2024-03-23T17:49:28.785531+0000","last_fullsized":"2024-03-23T17:49:28.785531+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:36:36.842740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'1","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.780817+0000","last_change":"2024-03-23T17:48:56.227211+0000","last_active":"2024-03-23T17:49:28.780817+0000","last_peered":"2024-03-23T17:49:28.780817+0000","last_clean":"2024-03-23T17:49:28.780817+0000","last_became_active":"2024-03-23T17:48:56.226881+0000","last_became_peered":"2024-03-23T17:48:56.226881+0000","last_unstale":"2024-03-23T17:49:28.780817+0000","last_undegraded":"2024-03-23T17:49:28.780817+0000","last_fullsized":"2024-03-23T17:49:28.780817+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:15:02.937666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.776701+0000","last_change":"2024-03-23T17:48:55.728679+0000","last_active":"2024-03-23T17:49:28.776701+0000","last_peered":"2024-03-23T17:49:28.776701+0000","last_clean":"2024-03-23T17:49:28.776701+0000","last_became_active":"2024-03-23T17:48:55.727754+0000","last_became_peered":"2024-03-23T17:48:55.727754+0000","last_unstale":"2024-03-23T17:49:28.776701+0000","last_undegraded":"2024-03-23T17:49:28.776701+0000","last_fullsized":"2024-03-23T17:49:28.776701+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:34:39.720488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.781328+0000","last_change":"2024-03-23T17:48:56.225214+0000","last_active":"2024-03-23T17:49:28.781328+0000","last_peered":"2024-03-23T17:49:28.781328+0000","last_clean":"2024-03-23T17:49:28.781328+0000","last_became_active":"2024-03-23T17:48:56.224635+0000","last_became_peered":"2024-03-23T17:48:56.224635+0000","last_unstale":"2024-03-23T17:49:28.781328+0000","last_undegraded":"2024-03-23T17:49:28.781328+0000","last_fullsized":"2024-03-23T17:49:28.781328+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:57:14.935434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.782109+0000","last_change":"2024-03-23T17:48:56.229835+0000","last_active":"2024-03-23T17:49:28.782109+0000","last_peered":"2024-03-23T17:49:28.782109+0000","last_clean":"2024-03-23T17:49:28.782109+0000","last_became_active":"2024-03-23T17:48:56.229437+0000","last_became_peered":"2024-03-23T17:48:56.229437+0000","last_unstale":"2024-03-23T17:49:28.782109+0000","last_undegraded":"2024-03-23T17:49:28.782109+0000","last_fullsized":"2024-03-23T17:49:28.782109+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:43:51.544867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.784779+0000","last_change":"2024-03-23T17:48:56.233552+0000","last_active":"2024-03-23T17:49:28.784779+0000","last_peered":"2024-03-23T17:49:28.784779+0000","last_clean":"2024-03-23T17:49:28.784779+0000","last_became_active":"2024-03-23T17:48:56.232941+0000","last_became_peered":"2024-03-23T17:48:56.232941+0000","last_unstale":"2024-03-23T17:49:28.784779+0000","last_undegraded":"2024-03-23T17:49:28.784779+0000","last_fullsized":"2024-03-23T17:49:28.784779+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:03:32.325496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.785402+0000","last_change":"2024-03-23T17:48:56.238238+0000","last_active":"2024-03-23T17:49:28.785402+0000","last_peered":"2024-03-23T17:49:28.785402+0000","last_clean":"2024-03-23T17:49:28.785402+0000","last_became_active":"2024-03-23T17:48:56.237882+0000","last_became_peered":"2024-03-23T17:48:56.237882+0000","last_unstale":"2024-03-23T17:49:28.785402+0000","last_undegraded":"2024-03-23T17:49:28.785402+0000","last_fullsized":"2024-03-23T17:49:28.785402+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:27:13.896205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.783881+0000","last_change":"2024-03-23T17:48:56.242895+0000","last_active":"2024-03-23T17:49:28.783881+0000","last_peered":"2024-03-23T17:49:28.783881+0000","last_clean":"2024-03-23T17:49:28.783881+0000","last_became_active":"2024-03-23T17:48:56.242515+0000","last_became_peered":"2024-03-23T17:48:56.242515+0000","last_unstale":"2024-03-23T17:49:28.783881+0000","last_undegraded":"2024-03-23T17:49:28.783881+0000","last_fullsized":"2024-03-23T17:49:28.783881+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:45:01.978843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.776603+0000","last_change":"2024-03-23T17:48:56.237029+0000","last_active":"2024-03-23T17:49:28.776603+0000","last_peered":"2024-03-23T17:49:28.776603+0000","last_clean":"2024-03-23T17:49:28.776603+0000","last_became_active":"2024-03-23T17:48:56.236692+0000","last_became_peered":"2024-03-23T17:48:56.236692+0000","last_unstale":"2024-03-23T17:49:28.776603+0000","last_undegraded":"2024-03-23T17:49:28.776603+0000","last_fullsized":"2024-03-23T17:49:28.776603+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:42:31.855469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.782393+0000","last_change":"2024-03-23T17:48:56.223236+0000","last_active":"2024-03-23T17:49:28.782393+0000","last_peered":"2024-03-23T17:49:28.782393+0000","last_clean":"2024-03-23T17:49:28.782393+0000","last_became_active":"2024-03-23T17:48:56.222824+0000","last_became_peered":"2024-03-23T17:48:56.222824+0000","last_unstale":"2024-03-23T17:49:28.782393+0000","last_undegraded":"2024-03-23T17:49:28.782393+0000","last_fullsized":"2024-03-23T17:49:28.782393+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:54:45.096969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.785731+0000","last_change":"2024-03-23T17:48:56.224471+0000","last_active":"2024-03-23T17:49:28.785731+0000","last_peered":"2024-03-23T17:49:28.785731+0000","last_clean":"2024-03-23T17:49:28.785731+0000","last_became_active":"2024-03-23T17:48:56.223140+0000","last_became_peered":"2024-03-23T17:48:56.223140+0000","last_unstale":"2024-03-23T17:49:28.785731+0000","last_undegraded":"2024-03-23T17:49:28.785731+0000","last_fullsized":"2024-03-23T17:49:28.785731+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:38:53.906820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.780461+0000","last_change":"2024-03-23T17:48:56.223564+0000","last_active":"2024-03-23T17:49:28.780461+0000","last_peered":"2024-03-23T17:49:28.780461+0000","last_clean":"2024-03-23T17:49:28.780461+0000","last_became_active":"2024-03-23T17:48:56.223217+0000","last_became_peered":"2024-03-23T17:48:56.223217+0000","last_unstale":"2024-03-23T17:49:28.780461+0000","last_undegraded":"2024-03-23T17:49:28.780461+0000","last_fullsized":"2024-03-23T17:49:28.780461+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:43:53.250082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.785081+0000","last_change":"2024-03-23T17:48:56.226283+0000","last_active":"2024-03-23T17:49:28.785081+0000","last_peered":"2024-03-23T17:49:28.785081+0000","last_clean":"2024-03-23T17:49:28.785081+0000","last_became_active":"2024-03-23T17:48:56.224211+0000","last_became_peered":"2024-03-23T17:48:56.224211+0000","last_unstale":"2024-03-23T17:49:28.785081+0000","last_undegraded":"2024-03-23T17:49:28.785081+0000","last_fullsized":"2024-03-23T17:49:28.785081+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:15:20.518950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.779571+0000","last_change":"2024-03-23T17:48:56.223557+0000","last_active":"2024-03-23T17:49:28.779571+0000","last_peered":"2024-03-23T17:49:28.779571+0000","last_clean":"2024-03-23T17:49:28.779571+0000","last_became_active":"2024-03-23T17:48:56.222792+0000","last_became_peered":"2024-03-23T17:48:56.222792+0000","last_unstale":"2024-03-23T17:49:28.779571+0000","last_undegraded":"2024-03-23T17:49:28.779571+0000","last_fullsized":"2024-03-23T17:49:28.779571+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:49:17.587817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.922230+0000","last_change":"2024-03-23T17:48:56.236041+0000","last_active":"2024-03-23T17:49:28.922230+0000","last_peered":"2024-03-23T17:49:28.922230+0000","last_clean":"2024-03-23T17:49:28.922230+0000","last_became_active":"2024-03-23T17:48:56.234259+0000","last_became_peered":"2024-03-23T17:48:56.234259+0000","last_unstale":"2024-03-23T17:49:28.922230+0000","last_undegraded":"2024-03-23T17:49:28.922230+0000","last_fullsized":"2024-03-23T17:49:28.922230+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:32:43.786783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.777528+0000","last_change":"2024-03-23T17:48:56.239283+0000","last_active":"2024-03-23T17:49:28.777528+0000","last_peered":"2024-03-23T17:49:28.777528+0000","last_clean":"2024-03-23T17:49:28.777528+0000","last_became_active":"2024-03-23T17:48:56.238952+0000","last_became_peered":"2024-03-23T17:48:56.238952+0000","last_unstale":"2024-03-23T17:49:28.777528+0000","last_undegraded":"2024-03-23T17:49:28.777528+0000","last_fullsized":"2024-03-23T17:49:28.777528+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:31:50.383284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.782143+0000","last_change":"2024-03-23T17:48:56.226684+0000","last_active":"2024-03-23T17:49:28.782143+0000","last_peered":"2024-03-23T17:49:28.782143+0000","last_clean":"2024-03-23T17:49:28.782143+0000","last_became_active":"2024-03-23T17:48:56.226367+0000","last_became_peered":"2024-03-23T17:48:56.226367+0000","last_unstale":"2024-03-23T17:49:28.782143+0000","last_undegraded":"2024-03-23T17:49:28.782143+0000","last_fullsized":"2024-03-23T17:49:28.782143+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:12:29.053330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65648,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":413,"num_read_kb":280,"num_write":228,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":180,"num_keys_recovered":17,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":196944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":200,"ondisk_log_size":200,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547220,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31992,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713160,"statfs":{"total":95995035648,"available":95962275840,"internally_reserved":0,"allocated":897024,"data_stored":428738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31848901},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 17:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.64400000000000002}]},{"osd":1,"last update":"Sat Mar 23 17:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.65600000000000003}]},{"osd":2,"last update":"Sat Mar 23 17:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.42599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":4,"last update":"Sat Mar 23 17:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.748}]},{"osd":5,"last update":"Sat Mar 23 17:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.50600000000000001}]},{"osd":6,"last update":"Sat Mar 23 17:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.84499999999999997}]},{"osd":7,"last update":"Sat Mar 23 17:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.63},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.69899999999999995}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":9,"last update":"Sat Mar 23 17:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.68799999999999994}]},{"osd":10,"last update":"Sat Mar 23 17:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.80000000000000004}]}]},{"osd":10,"up_from":68,"seq":292057776151,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28568,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716584,"statfs":{"total":95995035648,"available":95965782016,"internally_reserved":0,"allocated":1585152,"data_stored":1078786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":1,"last update":"Sat Mar 23 17:49:02 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.432}]},{"osd":2,"last update":"Sat Mar 23 17:49:02 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.64700000000000002}]},{"osd":3,"last update":"Sat Mar 23 17:49:02 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.61599999999999999}]},{"osd":4,"last update":"Sat Mar 23 17:49:02 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.54800000000000004}]},{"osd":5,"last update":"Sat Mar 23 17:49:02 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.69199999999999995}]},{"osd":6,"last update":"Sat Mar 23 17:49:02 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.66200000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":8,"last update":"Sat Mar 23 17:49:02 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.63700000000000001}]},{"osd":9,"last update":"Sat Mar 23 17:49:02 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.67200000000000004}]},{"osd":11,"last update":"Sat Mar 23 17:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.70799999999999996}]}]},{"osd":9,"up_from":63,"seq":270582939677,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36108,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709044,"statfs":{"total":95995035648,"available":95958061056,"internally_reserved":0,"allocated":913408,"data_stored":428648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":36043201},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 17:48:42 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.59999999999999998}]},{"osd":1,"last update":"Sat Mar 23 17:48:42 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.56499999999999995}]},{"osd":2,"last update":"Sat Mar 23 17:48:42 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.67400000000000004}]},{"osd":3,"last update":"Sat Mar 23 17:48:42 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.45100000000000001}]},{"osd":4,"last update":"Sat Mar 23 17:48:42 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.61099999999999999}]},{"osd":5,"last update":"Sat Mar 23 17:48:42 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":1.1180000000000001},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.48,"5min":1.48,"15min":1.48},"last":0.80100000000000005}]},{"osd":6,"last update":"Sat Mar 23 17:48:42 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.48899999999999999}]},{"osd":7,"last update":"Sat Mar 23 17:48:42 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.373,"5min":1.373,"15min":1.373},"last":0.65500000000000003}]},{"osd":8,"last update":"Sat Mar 23 17:48:42 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":1.2270000000000001},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.78800000000000003}]},{"osd":10,"last update":"Sat Mar 23 17:48:58 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.69699999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74299999999999999}]}]},{"osd":8,"up_from":56,"seq":240518168606,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":429026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 17:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.57399999999999995,"15min":0.57999999999999996},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.78000000000000003,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.504},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.54800000000000004,"15min":0.55100000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.67300000000000004,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.52900000000000003}]},{"osd":1,"last update":"Sat Mar 23 17:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.59099999999999997,"15min":0.58599999999999997},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.77700000000000002,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.65700000000000003}]},{"osd":2,"last update":"Sat Mar 23 17:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.45600000000000002,"15min":0.45400000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.46500000000000002,"15min":0.46200000000000002},"min":{"1min":0.36799999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.61099999999999999,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.55600000000000005}]},{"osd":3,"last update":"Sat Mar 23 17:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.58499999999999996,"15min":0.59099999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.76800000000000002,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.65300000000000002,"15min":0.66200000000000003},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.83199999999999996,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.82299999999999995}]},{"osd":4,"last update":"Sat Mar 23 17:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.59499999999999997,"15min":0.58799999999999997},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.63800000000000001,"15min":0.63500000000000001},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.746}]},{"osd":5,"last update":"Sat Mar 23 17:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.54300000000000004,"15min":0.54700000000000004},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.69599999999999995,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.56200000000000006,"15min":0.55800000000000005},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.79400000000000004,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.80700000000000005}]},{"osd":6,"last update":"Sat Mar 23 17:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.70499999999999996,"15min":0.71099999999999997},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.91900000000000004,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.68500000000000005,"15min":0.68999999999999995},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.78700000000000003}]},{"osd":7,"last update":"Sat Mar 23 17:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68100000000000005,"15min":0.67900000000000005},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.876,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.70299999999999996,"15min":0.69899999999999995},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.90200000000000002,"5min":1.04,"15min":1.04},"last":0.84399999999999997}]},{"osd":9,"last update":"Sat Mar 23 17:48:42 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.875},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.88500000000000001}]},{"osd":10,"last update":"Sat Mar 23 17:49:01 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.92000000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86299999999999999}]}]},{"osd":7,"up_from":49,"seq":210453397539,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28492,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716660,"statfs":{"total":95995035648,"available":95965859840,"internally_reserved":0,"allocated":1507328,"data_stored":1021009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 17:48:02 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.54800000000000004}]},{"osd":1,"last update":"Sat Mar 23 17:49:03 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.40200000000000002,"15min":0.39600000000000002},"min":{"1min":0.33800000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.58399999999999996,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.39200000000000002,"15min":0.378},"min":{"1min":0.28299999999999997,"5min":0.248,"15min":0.248},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.36499999999999999}]},{"osd":2,"last update":"Sat Mar 23 17:49:03 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.56200000000000006,"15min":0.55400000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.56200000000000006,"15min":0.55600000000000005},"min":{"1min":0.45500000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.70499999999999996}]},{"osd":3,"last update":"Sat Mar 23 17:49:03 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.54800000000000004,"15min":0.54400000000000004},"min":{"1min":0.42499999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.70599999999999996,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.627,"5min":0.59499999999999997,"15min":0.58999999999999997},"min":{"1min":0.44600000000000001,"5min":0.432,"15min":0.432},"max":{"1min":0.74099999999999999,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.58999999999999997}]},{"osd":4,"last update":"Sat Mar 23 17:49:03 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.48799999999999999,"15min":0.47699999999999998},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.498},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.47899999999999998,"15min":0.47099999999999997},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.68500000000000005,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.35299999999999998}]},{"osd":5,"last update":"Sat Mar 23 17:49:03 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.59499999999999997,"15min":0.58599999999999997},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.77500000000000002,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.625,"5min":0.58199999999999996,"15min":0.57499999999999996},"min":{"1min":0.41999999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.60699999999999998}]},{"osd":6,"last update":"Sat Mar 23 17:49:03 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.67500000000000004,"15min":0.66700000000000004},"min":{"1min":0.54600000000000004,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.95499999999999996,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.45500000000000002,"5min":0.433,"15min":0.433},"max":{"1min":0.81899999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.64300000000000002}]},{"osd":8,"last update":"Sat Mar 23 17:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.64900000000000002,"15min":0.63300000000000001},"min":{"1min":0.47799999999999998,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.66600000000000004,"15min":0.64700000000000002},"min":{"1min":0.52700000000000002,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.752}]},{"osd":9,"last update":"Sat Mar 23 17:48:42 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.67900000000000005}]},{"osd":10,"last update":"Sat Mar 23 17:49:01 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.72799999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]}]},{"osd":6,"up_from":45,"seq":193273528359,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36088,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93709064,"statfs":{"total":95995035648,"available":95958081536,"internally_reserved":0,"allocated":892928,"data_stored":428044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 17:48:45 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.47499999999999998,"15min":0.47599999999999998},"min":{"1min":0.35599999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.52800000000000002,"15min":0.52400000000000002},"min":{"1min":0.33700000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":1.256}]},{"osd":1,"last update":"Sat Mar 23 17:48:45 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.64400000000000002,"15min":0.64500000000000002},"min":{"1min":0.48399999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.91800000000000004,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.88700000000000001}]},{"osd":2,"last update":"Sat Mar 23 17:48:45 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.67900000000000005,"15min":0.68200000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":1.4710000000000001},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.67400000000000004,"15min":0.67800000000000005},"min":{"1min":0.47099999999999997,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.91900000000000004,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.59299999999999997}]},{"osd":3,"last update":"Sat Mar 23 17:48:45 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.61399999999999999,"15min":0.61899999999999999},"min":{"1min":0.34399999999999997,"5min":0.316,"15min":0.316},"max":{"1min":0.89700000000000002,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.66400000000000003,"15min":0.66600000000000004},"min":{"1min":0.45600000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.88300000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":1.1140000000000001}]},{"osd":4,"last update":"Sat Mar 23 17:48:45 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.65700000000000003,"15min":0.65800000000000003},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":1.3740000000000001},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.72999999999999998,"15min":0.73699999999999999},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.86399999999999999,"5min":0.998,"15min":0.998},"last":0.68300000000000005}]},{"osd":5,"last update":"Sat Mar 23 17:48:45 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.77900000000000003,"15min":0.78300000000000003},"min":{"1min":0.57199999999999995,"5min":0.436,"15min":0.436},"max":{"1min":0.90600000000000003,"5min":1.375,"15min":1.375},"last":1.2170000000000001},{"interface":"front","average":{"1min":0.75,"5min":0.78100000000000003,"15min":0.78600000000000003},"min":{"1min":0.59199999999999997,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":1.504}]},{"osd":7,"last update":"Sat Mar 23 17:49:03 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.73699999999999999,"15min":0.73899999999999999},"min":{"1min":0.53800000000000003,"5min":0.5,"15min":0.5},"max":{"1min":0.97299999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":1.347},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.77000000000000002,"15min":0.78600000000000003},"min":{"1min":0.505,"5min":0.495,"15min":0.495},"max":{"1min":0.91100000000000003,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":1.4339999999999999}]},{"osd":8,"last update":"Sat Mar 23 17:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.85599999999999998,"15min":0.876},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.92400000000000004,"5min":1.085,"15min":1.085},"last":1.5409999999999999},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.81499999999999995,"15min":0.82399999999999995},"min":{"1min":0.63200000000000001,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":0.91200000000000003,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.66500000000000004}]},{"osd":9,"last update":"Sat Mar 23 17:48:45 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":1.3100000000000001},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.83699999999999997}]},{"osd":10,"last update":"Sat Mar 23 17:49:01 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":1.369},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":1.573}]}]},{"osd":1,"up_from":13,"seq":55834574905,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":488648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 17:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.52600000000000002,"15min":0.51300000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.85699999999999998,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.63100000000000001,"15min":0.53300000000000003},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.73699999999999999,"5min":5.8399999999999999,"15min":5.8399999999999999},"last":0.68200000000000005}]},{"osd":2,"last update":"Sat Mar 23 17:49:29 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.52700000000000002,"15min":0.48099999999999998},"min":{"1min":0.46600000000000003,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.81399999999999995,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.52200000000000002,"15min":0.49299999999999999},"min":{"1min":0.45900000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.77400000000000002,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.55400000000000005}]},{"osd":3,"last update":"Sat Mar 23 17:48:51 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.55000000000000004,"15min":0.53200000000000003},"min":{"1min":0.377,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.55100000000000005,"15min":0.53900000000000003},"min":{"1min":0.39100000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.59199999999999997}]},{"osd":4,"last update":"Sat Mar 23 17:49:08 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.432,"15min":0.40200000000000002},"min":{"1min":0.29499999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.42299999999999999,"15min":0.39700000000000002},"min":{"1min":0.33800000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.75900000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.44900000000000001}]},{"osd":5,"last update":"Sat Mar 23 17:49:29 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.59499999999999997,"15min":0.56299999999999994},"min":{"1min":0.55500000000000005,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.86599999999999999,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.61199999999999999,"15min":0.57799999999999996},"min":{"1min":0.55600000000000005,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.85399999999999998,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.64500000000000002}]},{"osd":6,"last update":"Sat Mar 23 17:48:45 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.58799999999999997,"15min":0.56899999999999995},"min":{"1min":0.52100000000000002,"5min":0.433,"15min":0.433},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.63600000000000001,"15min":0.621},"min":{"1min":0.47399999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.66200000000000003}]},{"osd":7,"last update":"Sat Mar 23 17:49:03 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.52700000000000002,"15min":0.51100000000000001},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.82899999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.56200000000000006,"15min":0.53800000000000003},"min":{"1min":0.45000000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.48899999999999999}]},{"osd":8,"last update":"Sat Mar 23 17:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.68000000000000005,"15min":0.66800000000000004},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.96099999999999997,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.64400000000000002,"15min":0.63200000000000001},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.621}]},{"osd":9,"last update":"Sat Mar 23 17:48:45 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.72099999999999997}]},{"osd":10,"last update":"Sat Mar 23 17:48:59 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.70799999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76100000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738430,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":913408,"data_stored":429383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 23 17:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.64600000000000002,"15min":0.72399999999999998},"min":{"1min":0.30099999999999999,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.85899999999999999,"5min":6.2110000000000003,"15min":6.2110000000000003},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.54300000000000004,"15min":0.50700000000000001},"min":{"1min":0.38500000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.84499999999999997,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.432}]},{"osd":2,"last update":"Sat Mar 23 17:49:31 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.57799999999999996,"15min":0.55100000000000005},"min":{"1min":0.38100000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.82999999999999996,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.59999999999999998,"15min":0.56000000000000005},"min":{"1min":0.41299999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.63400000000000001}]},{"osd":3,"last update":"Sat Mar 23 17:47:52 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.498,"15min":0.49399999999999999},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.78000000000000003,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.34499999999999997,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.79100000000000004,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.68200000000000005}]},{"osd":4,"last update":"Sat Mar 23 17:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.628,"15min":0.61799999999999999},"min":{"1min":0.42199999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59599999999999997,"15min":0.58699999999999997},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.88100000000000001,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.46100000000000002}]},{"osd":5,"last update":"Sat Mar 23 17:49:28 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65700000000000003,"15min":0.63700000000000001},"min":{"1min":0.44700000000000001,"5min":0.44,"15min":0.44},"max":{"1min":1.018,"5min":1.167,"15min":1.167},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.64500000000000002,"15min":0.61599999999999999},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.65700000000000003}]},{"osd":6,"last update":"Sat Mar 23 17:48:47 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.60699999999999998,"15min":0.60099999999999998},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.67900000000000005,"15min":0.66800000000000004},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.749}]},{"osd":7,"last update":"Sat Mar 23 17:48:02 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.495},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.58599999999999997}]},{"osd":8,"last update":"Sat Mar 23 17:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.83299999999999996,"15min":0.84899999999999998},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.091,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.81299999999999994,"15min":0.82599999999999996},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.70099999999999996}]},{"osd":9,"last update":"Sat Mar 23 17:48:42 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":0.82199999999999995}]},{"osd":10,"last update":"Sat Mar 23 17:49:04 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.79600000000000004}]},{"osd":11,"last update":"Sat Mar 23 17:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.82899999999999996}]}]},{"osd":2,"up_from":18,"seq":77309411382,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36148,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709004,"statfs":{"total":95995035648,"available":95958020096,"internally_reserved":0,"allocated":958464,"data_stored":488367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 17:49:30 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.54000000000000004,"15min":0.51400000000000001},"min":{"1min":0.42199999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.63,"5min":0.57899999999999996,"15min":0.55300000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.88700000000000001,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.68799999999999994}]},{"osd":1,"last update":"Sat Mar 23 17:49:30 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.54500000000000004,"15min":0.5},"min":{"1min":0.42199999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.80900000000000005,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.55800000000000005,"15min":0.51800000000000002},"min":{"1min":0.41999999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.79300000000000004,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.65300000000000002}]},{"osd":3,"last update":"Sat Mar 23 17:48:54 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.54400000000000004,"15min":0.52800000000000002},"min":{"1min":0.36699999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.58299999999999996,"15min":0.56499999999999995},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.52600000000000002}]},{"osd":4,"last update":"Sat Mar 23 17:48:06 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.51900000000000002,"15min":0.51100000000000001},"min":{"1min":0.41099999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.628,"5min":0.58199999999999996,"15min":0.57399999999999995},"min":{"1min":0.42599999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.48099999999999998}]},{"osd":5,"last update":"Sat Mar 23 17:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.48299999999999998,"15min":0.45900000000000002},"min":{"1min":0.32600000000000001,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.433},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.495,"15min":0.46999999999999997},"min":{"1min":0.34100000000000003,"5min":0.308,"15min":0.308},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.72899999999999998}]},{"osd":6,"last update":"Sat Mar 23 17:48:48 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.623,"15min":0.61699999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.64400000000000002,"15min":0.63800000000000001},"min":{"1min":0.46100000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.63}]},{"osd":7,"last update":"Sat Mar 23 17:48:01 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.745}]},{"osd":8,"last update":"Sat Mar 23 17:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.51000000000000001,"15min":0.50600000000000001},"min":{"1min":0.35599999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.59399999999999997,"15min":0.58599999999999997},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.78100000000000003,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.498}]},{"osd":9,"last update":"Sat Mar 23 17:48:43 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.71599999999999997}]},{"osd":10,"last update":"Sat Mar 23 17:49:03 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.77400000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]}]},{"osd":3,"up_from":26,"seq":111669149746,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32008,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713144,"statfs":{"total":95995035648,"available":95962259456,"internally_reserved":0,"allocated":909312,"data_stored":427917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 17:48:58 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.41299999999999998,"15min":0.39300000000000002},"min":{"1min":0.34599999999999997,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.499,"5min":0.42499999999999999,"15min":0.40000000000000002},"min":{"1min":0.33500000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.94899999999999995,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.316}]},{"osd":1,"last update":"Sat Mar 23 17:48:58 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.54200000000000004,"15min":0.51500000000000001},"min":{"1min":0.379,"5min":0.371,"15min":0.371},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.54100000000000004,"15min":0.50700000000000001},"min":{"1min":0.40699999999999997,"5min":0.371,"15min":0.371},"max":{"1min":0.91500000000000004,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.60099999999999998}]},{"osd":2,"last update":"Sat Mar 23 17:48:58 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.60999999999999999,"15min":0.58299999999999996},"min":{"1min":0.496,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.626,"15min":0.61199999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.90500000000000003,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.48299999999999998}]},{"osd":4,"last update":"Sat Mar 23 17:49:09 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.63600000000000001,"15min":0.60999999999999999},"min":{"1min":0.46700000000000003,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.63700000000000001,"15min":0.61299999999999999},"min":{"1min":0.45600000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.63400000000000001}]},{"osd":5,"last update":"Sat Mar 23 17:49:31 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.67800000000000005,"15min":0.65000000000000002},"min":{"1min":0.53500000000000003,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.01,"5min":1.036,"15min":1.036},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.70099999999999996,"15min":0.67300000000000004},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.55900000000000005}]},{"osd":6,"last update":"Sat Mar 23 17:48:45 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.64800000000000002,"15min":0.64400000000000002},"min":{"1min":0.504,"5min":0.38,"15min":0.38},"max":{"1min":0.88600000000000001,"5min":1.024,"15min":1.024},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.66900000000000004,"15min":0.66400000000000003},"min":{"1min":0.48099999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.93400000000000005,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.52000000000000002}]},{"osd":7,"last update":"Sat Mar 23 17:49:06 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.76200000000000001,"15min":0.76700000000000002},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.995,"5min":1.038,"15min":1.038},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.71099999999999997,"15min":0.70799999999999996},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.66500000000000004}]},{"osd":8,"last update":"Sat Mar 23 17:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.88},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.85099999999999998}]},{"osd":9,"last update":"Sat Mar 23 17:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.58099999999999996}]},{"osd":10,"last update":"Sat Mar 23 17:49:02 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.61099999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]}]},{"osd":4,"up_from":33,"seq":141733920815,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":901120,"data_stored":428017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 17:48:06 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.53700000000000003,"15min":0.53400000000000003},"min":{"1min":0.41799999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.77600000000000002,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.53700000000000003,"15min":0.52500000000000002},"min":{"1min":0.45300000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.52000000000000002}]},{"osd":1,"last update":"Sat Mar 23 17:49:07 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.34999999999999998,"15min":0.32500000000000001},"min":{"1min":0.27700000000000002,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.60799999999999998,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.36499999999999999,"15min":0.34300000000000003},"min":{"1min":0.28599999999999998,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.56299999999999994,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.41499999999999998}]},{"osd":2,"last update":"Sat Mar 23 17:49:07 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.56899999999999995,"15min":0.55500000000000005},"min":{"1min":0.379,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.80700000000000005,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.56599999999999995,"15min":0.54600000000000004},"min":{"1min":0.42499999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.78000000000000003,"5min":0.88,"15min":0.88},"last":0.54700000000000004}]},{"osd":3,"last update":"Sat Mar 23 17:49:07 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.56899999999999995,"15min":0.55900000000000005},"min":{"1min":0.40300000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.751,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.56799999999999995,"15min":0.56299999999999994},"min":{"1min":0.38400000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.76600000000000001,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.47799999999999998}]},{"osd":5,"last update":"Sat Mar 23 17:49:29 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.57999999999999996,"15min":0.56599999999999995},"min":{"1min":0.41299999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.80600000000000005,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.629,"5min":0.59499999999999997,"15min":0.57799999999999996},"min":{"1min":0.44600000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.77700000000000002,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.57099999999999995}]},{"osd":6,"last update":"Sat Mar 23 17:48:53 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.47699999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.78200000000000003,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.58999999999999997,"15min":0.58899999999999997},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.73599999999999999}]},{"osd":7,"last update":"Sat Mar 23 17:49:03 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.54400000000000004,"15min":0.54000000000000004},"min":{"1min":0.39300000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.54000000000000004,"15min":0.53400000000000003},"min":{"1min":0.36299999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.70099999999999996,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.51500000000000001}]},{"osd":8,"last update":"Sat Mar 23 17:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.65800000000000003,"15min":0.65400000000000003},"min":{"1min":0.501,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.71699999999999997,"15min":0.71799999999999997},"min":{"1min":0.48599999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.86599999999999999,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.67900000000000005}]},{"osd":9,"last update":"Sat Mar 23 17:48:42 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.623}]},{"osd":10,"last update":"Sat Mar 23 17:49:01 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.749},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.66700000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]}]},{"osd":5,"up_from":38,"seq":163208757290,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32600,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712552,"statfs":{"total":95995035648,"available":95961653248,"internally_reserved":0,"allocated":1515520,"data_stored":1020452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 17:49:28 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.54900000000000004,"15min":0.54100000000000004},"min":{"1min":0.435,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.502},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.54400000000000004,"15min":0.54800000000000004},"min":{"1min":0.36299999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.83499999999999996,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.441}]},{"osd":1,"last update":"Sat Mar 23 17:49:28 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.55100000000000005,"15min":0.52700000000000002},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.58799999999999997,"15min":0.56499999999999995},"min":{"1min":0.495,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.61399999999999999}]},{"osd":2,"last update":"Sat Mar 23 17:49:28 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.40400000000000003,"15min":0.39400000000000002},"min":{"1min":0.311,"5min":0.251,"15min":0.251},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.503,"5min":0.41799999999999998,"15min":0.40100000000000002},"min":{"1min":0.30399999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.38400000000000001}]},{"osd":3,"last update":"Sat Mar 23 17:49:28 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58299999999999996,"15min":0.58499999999999996},"min":{"1min":0.38100000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.80100000000000005,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.57699999999999996,"15min":0.58499999999999996},"min":{"1min":0.40899999999999997,"5min":0.373,"15min":0.373},"max":{"1min":0.71899999999999997,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.52000000000000002}]},{"osd":4,"last update":"Sat Mar 23 17:49:28 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.60599999999999998,"15min":0.58199999999999996},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.89500000000000002,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.58199999999999996,"15min":0.56000000000000005},"min":{"1min":0.44400000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.877,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.71199999999999997}]},{"osd":6,"last update":"Sat Mar 23 17:48:48 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.627,"15min":0.625},"min":{"1min":0.53400000000000003,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.88500000000000001,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.441,"5min":0.434,"15min":0.434},"max":{"1min":0.79600000000000004,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.56499999999999995}]},{"osd":7,"last update":"Sat Mar 23 17:49:03 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.64400000000000002,"15min":0.64000000000000001},"min":{"1min":0.41999999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.89300000000000002,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.63500000000000001,"15min":0.63},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.86599999999999999,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.79200000000000004}]},{"osd":8,"last update":"Sat Mar 23 17:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.60499999999999998,"15min":0.59099999999999997},"min":{"1min":0.39600000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.56999999999999995,"15min":0.55800000000000005},"min":{"1min":0.42999999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.53900000000000003}]},{"osd":9,"last update":"Sat Mar 23 17:48:43 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.64800000000000002}]},{"osd":10,"last update":"Sat Mar 23 17:49:00 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.754},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.80500000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":61114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":61106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T17:49:43.269 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-03-23T17:49:43.269 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-23T17:49:43.269 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-03-23T17:49:43.270 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph health --format=json 2024-03-23T17:49:43.632 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T17:49:44.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:43 smithi152 ceph-mon[47949]: pgmap v259: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 14 B/s, 1 keys/s, 1 objects/s recovering 2024-03-23T17:49:44.094 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:43 smithi086 ceph-mon[38221]: pgmap v259: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 14 B/s, 1 keys/s, 1 objects/s recovering 2024-03-23T17:49:44.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:43 smithi092 ceph-mon[79089]: pgmap v259: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 14 B/s, 1 keys/s, 1 objects/s recovering 2024-03-23T17:49:44.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:44.773+0000 7f32c9800700 1 -- 172.21.15.86:0/1722190461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32c4072fe0 msgr2=0x7f32c40733c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:44.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:44.773+0000 7f32c9800700 1 --2- 172.21.15.86:0/1722190461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32c4072fe0 0x7f32c40733c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f32b4009990 tx=0x7f32b4009ca0 comp rx=0 tx=0).stop 2024-03-23T17:49:44.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:44.773+0000 7f32c9800700 1 -- 172.21.15.86:0/1722190461 shutdown_connections 2024-03-23T17:49:44.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:44.773+0000 7f32c9800700 1 --2- 172.21.15.86:0/1722190461 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f32c410d840 0x7f32c410fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:44.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:44.773+0000 7f32c9800700 1 --2- 172.21.15.86:0/1722190461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32c4072fe0 0x7f32c40733c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:44.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:44.773+0000 7f32c9800700 1 --2- 172.21.15.86:0/1722190461 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32c4073900 0x7f32c410d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:44.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:44.773+0000 7f32c9800700 1 -- 172.21.15.86:0/1722190461 >> 172.21.15.86:0/1722190461 conn(0x7f32c4077e90 msgr2=0x7f32c40782a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:44.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:44.773+0000 7f32c9800700 1 -- 172.21.15.86:0/1722190461 shutdown_connections 2024-03-23T17:49:44.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:44.773+0000 7f32c9800700 1 -- 172.21.15.86:0/1722190461 wait complete. 2024-03-23T17:49:44.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:44.774+0000 7f32c9800700 1 Processor -- start 2024-03-23T17:49:44.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:44.775+0000 7f32c9800700 1 -- start start 2024-03-23T17:49:44.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:44.775+0000 7f32c9800700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f32c4072fe0 0x7f32c41993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:44.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:44.775+0000 7f32c9800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32c4073900 0x7f32c4199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:44.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:44.775+0000 7f32c9800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32c410d840 0x7f32c419dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:44.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:44.775+0000 7f32c9800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c419e2d0 con 0x7f32c4073900 2024-03-23T17:49:44.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:44.775+0000 7f32c9800700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c419e440 con 0x7f32c410d840 2024-03-23T17:49:44.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:44.775+0000 7f32c9800700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c419e600 con 0x7f32c4072fe0 2024-03-23T17:49:44.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:44.775+0000 7f32c2a00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f32c4072fe0 0x7f32c41993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:44.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:44.775+0000 7f32c2000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32c4073900 0x7f32c4199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:44.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:44.775+0000 7f32c2a00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f32c4072fe0 0x7f32c41993d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:42440/0 (socket says 172.21.15.86:42440) 2024-03-23T17:49:44.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:44.775+0000 7f32c2a00700 1 -- 172.21.15.86:0/1404254946 learned_addr learned my addr 172.21.15.86:0/1404254946 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:44.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:44.775+0000 7f32c3400700 1 --2- 172.21.15.86:0/1404254946 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32c410d840 0x7f32c419dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:44.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:44.776+0000 7f32c2a00700 1 -- 172.21.15.86:0/1404254946 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32c410d840 msgr2=0x7f32c419dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:44.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:44.776+0000 7f32c2a00700 1 --2- 172.21.15.86:0/1404254946 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32c410d840 0x7f32c419dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:44.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:44.776+0000 7f32c2a00700 1 -- 172.21.15.86:0/1404254946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32c4073900 msgr2=0x7f32c4199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:44.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:44.776+0000 7f32c2a00700 1 --2- 172.21.15.86:0/1404254946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32c4073900 0x7f32c4199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:44.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:44.776+0000 7f32c2a00700 1 -- 172.21.15.86:0/1404254946 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32b40095f0 con 0x7f32c4072fe0 2024-03-23T17:49:44.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:44.776+0000 7f32c2000700 1 --2- 172.21.15.86:0/1404254946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32c4073900 0x7f32c4199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:49:44.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:44.776+0000 7f32c2a00700 1 --2- 172.21.15.86:0/1404254946 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f32c4072fe0 0x7f32c41993d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f32b4005910 tx=0x7f32b4004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:44.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:44.776+0000 7f32bb400700 1 -- 172.21.15.86:0/1404254946 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b401d070 con 0x7f32c4072fe0 2024-03-23T17:49:44.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:44.776+0000 7f32c9800700 1 -- 172.21.15.86:0/1404254946 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32c419e880 con 0x7f32c4072fe0 2024-03-23T17:49:44.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:44.776+0000 7f32bb400700 1 -- 172.21.15.86:0/1404254946 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f32b4022470 con 0x7f32c4072fe0 2024-03-23T17:49:44.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:44.776+0000 7f32bb400700 1 -- 172.21.15.86:0/1404254946 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b4017650 con 0x7f32c4072fe0 2024-03-23T17:49:44.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:44.776+0000 7f32c9800700 1 -- 172.21.15.86:0/1404254946 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32c41a6140 con 0x7f32c4072fe0 2024-03-23T17:49:44.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:44.777+0000 7f32c9800700 1 -- 172.21.15.86:0/1404254946 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32c4108a50 con 0x7f32c4072fe0 2024-03-23T17:49:44.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:44.779+0000 7f32bb400700 1 -- 172.21.15.86:0/1404254946 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f32b40225e0 con 0x7f32c4072fe0 2024-03-23T17:49:44.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:44.780+0000 7f32bb400700 1 --2- 172.21.15.86:0/1404254946 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f32a80a11b0 0x7f32a80a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:44.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:44.781+0000 7f32bb400700 1 -- 172.21.15.86:0/1404254946 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f32b40cdd80 con 0x7f32c4072fe0 2024-03-23T17:49:44.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:44.781+0000 7f32c2000700 1 --2- 172.21.15.86:0/1404254946 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f32a80a11b0 0x7f32a80a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:44.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:44.781+0000 7f32c2000700 1 --2- 172.21.15.86:0/1404254946 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f32a80a11b0 0x7f32a80a3670 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f32c419a930 tx=0x7f32b00073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:44.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:44.783+0000 7f32bb400700 1 -- 172.21.15.86:0/1404254946 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f32b4097410 con 0x7f32c4072fe0 2024-03-23T17:49:45.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:45.030+0000 7f32c9800700 1 -- 172.21.15.86:0/1404254946 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f32c40745d0 con 0x7f32c4072fe0 2024-03-23T17:49:45.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:45.031+0000 7f32bb400700 1 -- 172.21.15.86:0/1404254946 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f32b409aa30 con 0x7f32c4072fe0 2024-03-23T17:49:45.032 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:49:45.032 INFO:teuthology.orchestra.run.smithi086.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-23T17:49:45.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:45.033+0000 7f32c9800700 1 -- 172.21.15.86:0/1404254946 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f32a80a11b0 msgr2=0x7f32a80a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:45.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:45.033+0000 7f32c9800700 1 --2- 172.21.15.86:0/1404254946 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f32a80a11b0 0x7f32a80a3670 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f32c419a930 tx=0x7f32b00073d0 comp rx=0 tx=0).stop 2024-03-23T17:49:45.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:45.033+0000 7f32c9800700 1 -- 172.21.15.86:0/1404254946 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f32c4072fe0 msgr2=0x7f32c41993d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:45.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:45.033+0000 7f32c9800700 1 --2- 172.21.15.86:0/1404254946 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f32c4072fe0 0x7f32c41993d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f32b4005910 tx=0x7f32b4004730 comp rx=0 tx=0).stop 2024-03-23T17:49:45.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:45.033+0000 7f32c9800700 1 -- 172.21.15.86:0/1404254946 shutdown_connections 2024-03-23T17:49:45.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:45.033+0000 7f32c9800700 1 --2- 172.21.15.86:0/1404254946 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32c410d840 0x7f32c419dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:45.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:45.033+0000 7f32c9800700 1 --2- 172.21.15.86:0/1404254946 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f32c4072fe0 0x7f32c41993d0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:45.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:45.033+0000 7f32c9800700 1 --2- 172.21.15.86:0/1404254946 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f32a80a11b0 0x7f32a80a3670 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:45.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:45.033+0000 7f32c9800700 1 --2- 172.21.15.86:0/1404254946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32c4073900 0x7f32c4199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:45.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:45.033+0000 7f32c9800700 1 -- 172.21.15.86:0/1404254946 >> 172.21.15.86:0/1404254946 conn(0x7f32c4077e90 msgr2=0x7f32c40fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:45.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:45.033+0000 7f32c9800700 1 -- 172.21.15.86:0/1404254946 shutdown_connections 2024-03-23T17:49:45.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:45.033+0000 7f32c9800700 1 -- 172.21.15.86:0/1404254946 wait complete. 2024-03-23T17:49:45.218 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:44 smithi086 ceph-mon[38221]: from='client.14832 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T17:49:45.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:44 smithi092 ceph-mon[79089]: from='client.14832 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T17:49:45.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:44 smithi152 ceph-mon[47949]: from='client.14832 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T17:49:45.467 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-03-23T17:49:45.468 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T17:49:45.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:45.664+0000 7fb439600700 1 -- 172.21.15.86:0/3901222627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb434101e70 msgr2=0x7fb4341022d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:45.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:45.664+0000 7fb439600700 1 --2- 172.21.15.86:0/3901222627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb434101e70 0x7fb4341022d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb420009920 tx=0x7fb420009c30 comp rx=0 tx=0).stop 2024-03-23T17:49:45.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:45.664+0000 7fb439600700 1 -- 172.21.15.86:0/3901222627 shutdown_connections 2024-03-23T17:49:45.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:45.664+0000 7fb439600700 1 --2- 172.21.15.86:0/3901222627 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb434102810 0x7fb4341090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:45.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:45.664+0000 7fb439600700 1 --2- 172.21.15.86:0/3901222627 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb434100c80 0x7fb4341010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:45.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:45.664+0000 7fb439600700 1 --2- 172.21.15.86:0/3901222627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb434101e70 0x7fb4341022d0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:45.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:45.664+0000 7fb439600700 1 -- 172.21.15.86:0/3901222627 >> 172.21.15.86:0/3901222627 conn(0x7fb4340fc1f0 msgr2=0x7fb4340fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:45.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:45.664+0000 7fb439600700 1 -- 172.21.15.86:0/3901222627 shutdown_connections 2024-03-23T17:49:45.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:45.664+0000 7fb439600700 1 -- 172.21.15.86:0/3901222627 wait complete. 2024-03-23T17:49:45.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:45.665+0000 7fb439600700 1 Processor -- start 2024-03-23T17:49:45.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:45.665+0000 7fb439600700 1 -- start start 2024-03-23T17:49:45.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:45.665+0000 7fb439600700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb434100c80 0x7fb434196d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:45.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:45.665+0000 7fb439600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb434101e70 0x7fb434197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:45.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:45.665+0000 7fb439600700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb434102810 0x7fb43419e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:45.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:45.665+0000 7fb439600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb43419e7e0 con 0x7fb434101e70 2024-03-23T17:49:45.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:45.665+0000 7fb439600700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb43419e950 con 0x7fb434102810 2024-03-23T17:49:45.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:45.665+0000 7fb439600700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb43419eb10 con 0x7fb434100c80 2024-03-23T17:49:45.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:45.665+0000 7fb432000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb434101e70 0x7fb434197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:45.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:45.665+0000 7fb432000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb434101e70 0x7fb434197250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46830/0 (socket says 172.21.15.86:46830) 2024-03-23T17:49:45.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:45.665+0000 7fb432000700 1 -- 172.21.15.86:0/1636539247 learned_addr learned my addr 172.21.15.86:0/1636539247 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:45.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:45.666+0000 7fb432a00700 1 --2- 172.21.15.86:0/1636539247 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb434100c80 0x7fb434196d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:45.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:45.666+0000 7fb433400700 1 --2- 172.21.15.86:0/1636539247 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb434102810 0x7fb43419e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:45.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:45.666+0000 7fb432000700 1 -- 172.21.15.86:0/1636539247 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb434100c80 msgr2=0x7fb434196d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:45.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:45.666+0000 7fb432000700 1 --2- 172.21.15.86:0/1636539247 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb434100c80 0x7fb434196d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:45.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:45.666+0000 7fb432000700 1 -- 172.21.15.86:0/1636539247 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb434102810 msgr2=0x7fb43419e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:45.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:45.666+0000 7fb432000700 1 --2- 172.21.15.86:0/1636539247 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb434102810 0x7fb43419e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:45.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:45.666+0000 7fb432000700 1 -- 172.21.15.86:0/1636539247 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb42c009d00 con 0x7fb434101e70 2024-03-23T17:49:45.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:45.666+0000 7fb432000700 1 --2- 172.21.15.86:0/1636539247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb434101e70 0x7fb434197250 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb420009840 tx=0x7fb42000b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:45.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:45.668+0000 7fb42b400700 1 -- 172.21.15.86:0/1636539247 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb42001c070 con 0x7fb434101e70 2024-03-23T17:49:45.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:45.668+0000 7fb42b400700 1 -- 172.21.15.86:0/1636539247 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb42000bb90 con 0x7fb434101e70 2024-03-23T17:49:45.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:45.668+0000 7fb439600700 1 -- 172.21.15.86:0/1636539247 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb420009520 con 0x7fb434101e70 2024-03-23T17:49:45.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:45.668+0000 7fb42b400700 1 -- 172.21.15.86:0/1636539247 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb420003ab0 con 0x7fb434101e70 2024-03-23T17:49:45.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:45.668+0000 7fb439600700 1 -- 172.21.15.86:0/1636539247 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb43419efd0 con 0x7fb434101e70 2024-03-23T17:49:45.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:45.669+0000 7fb439600700 1 -- 172.21.15.86:0/1636539247 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4341080f0 con 0x7fb434101e70 2024-03-23T17:49:45.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:45.671+0000 7fb42b400700 1 -- 172.21.15.86:0/1636539247 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fb420003c90 con 0x7fb434101e70 2024-03-23T17:49:45.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:45.673+0000 7fb42b400700 1 --2- 172.21.15.86:0/1636539247 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fb4180a11b0 0x7fb4180a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:45.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:45.673+0000 7fb42b400700 1 -- 172.21.15.86:0/1636539247 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fb4200cd7c0 con 0x7fb434101e70 2024-03-23T17:49:45.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:45.673+0000 7fb432a00700 1 --2- 172.21.15.86:0/1636539247 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fb4180a11b0 0x7fb4180a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:45.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:45.673+0000 7fb432a00700 1 --2- 172.21.15.86:0/1636539247 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fb4180a11b0 0x7fb4180a3670 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb4340fe650 tx=0x7fb424009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:45.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:45.677+0000 7fb42b400700 1 -- 172.21.15.86:0/1636539247 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fb420096e50 con 0x7fb434101e70 2024-03-23T17:49:45.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:45.925+0000 7fb439600700 1 -- 172.21.15.86:0/1636539247 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb434067a10 con 0x7fb434101e70 2024-03-23T17:49:45.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:45.926+0000 7fb42b400700 1 -- 172.21.15.86:0/1636539247 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22414 (secure 0 0 0) 0x7fb42009a470 con 0x7fb434101e70 2024-03-23T17:49:45.928 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:49:45.928 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":82,"fsid":"9d76c9e8-e93c-11ee-95cd-87774f69a715","created":"2024-03-23T17:42:30.097233+0000","modified":"2024-03-23T17:49:29.773954+0000","last_up_change":"2024-03-23T17:48:18.217601+0000","last_in_change":"2024-03-23T17:48:00.250579+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T17:45:26.848397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T17:48:54.331894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T17:48:55.640693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"656655c6-1c7f-4640-92c4-2f42de820143","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6803","nonce":2423505491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6805","nonce":2423505491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6809","nonce":2423505491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6807","nonce":2423505491}]},"public_addr":"172.21.15.86:6803/2423505491","cluster_addr":"172.21.15.86:6805/2423505491","heartbeat_back_addr":"172.21.15.86:6809/2423505491","heartbeat_front_addr":"172.21.15.86:6807/2423505491","state":["exists","up"]},{"osd":1,"uuid":"3e86dc65-20a6-4de8-b9ce-fcf01f68862b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6801","nonce":3776252597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6803","nonce":3776252597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6807","nonce":3776252597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6805","nonce":3776252597}]},"public_addr":"172.21.15.92:6801/3776252597","cluster_addr":"172.21.15.92:6803/3776252597","heartbeat_back_addr":"172.21.15.92:6807/3776252597","heartbeat_front_addr":"172.21.15.92:6805/3776252597","state":["exists","up"]},{"osd":2,"uuid":"09808786-3bd3-4d11-8c7b-1736119dfbb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6801","nonce":1601265891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6803","nonce":1601265891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6807","nonce":1601265891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6805","nonce":1601265891}]},"public_addr":"172.21.15.152:6801/1601265891","cluster_addr":"172.21.15.152:6803/1601265891","heartbeat_back_addr":"172.21.15.152:6807/1601265891","heartbeat_front_addr":"172.21.15.152:6805/1601265891","state":["exists","up"]},{"osd":3,"uuid":"79c61910-4b53-424c-b150-6f5385a2cd68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6811","nonce":694846466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6813","nonce":694846466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6817","nonce":694846466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6815","nonce":694846466}]},"public_addr":"172.21.15.86:6811/694846466","cluster_addr":"172.21.15.86:6813/694846466","heartbeat_back_addr":"172.21.15.86:6817/694846466","heartbeat_front_addr":"172.21.15.86:6815/694846466","state":["exists","up"]},{"osd":4,"uuid":"a2878035-0179-469f-bbd4-8ce2d3f6bddc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6809","nonce":1422821471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6811","nonce":1422821471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6815","nonce":1422821471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6813","nonce":1422821471}]},"public_addr":"172.21.15.92:6809/1422821471","cluster_addr":"172.21.15.92:6811/1422821471","heartbeat_back_addr":"172.21.15.92:6815/1422821471","heartbeat_front_addr":"172.21.15.92:6813/1422821471","state":["exists","up"]},{"osd":5,"uuid":"837366aa-ce25-42ca-b157-5bbb50159d0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6809","nonce":1039375239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6811","nonce":1039375239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6815","nonce":1039375239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6813","nonce":1039375239}]},"public_addr":"172.21.15.152:6809/1039375239","cluster_addr":"172.21.15.152:6811/1039375239","heartbeat_back_addr":"172.21.15.152:6815/1039375239","heartbeat_front_addr":"172.21.15.152:6813/1039375239","state":["exists","up"]},{"osd":6,"uuid":"61024437-7b91-432f-a1af-6443d0ef5213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6819","nonce":4080452707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6821","nonce":4080452707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6825","nonce":4080452707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6823","nonce":4080452707}]},"public_addr":"172.21.15.86:6819/4080452707","cluster_addr":"172.21.15.86:6821/4080452707","heartbeat_back_addr":"172.21.15.86:6825/4080452707","heartbeat_front_addr":"172.21.15.86:6823/4080452707","state":["exists","up"]},{"osd":7,"uuid":"2684bdae-6b30-4afc-af68-4ce41b3a75b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6817","nonce":4187707622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6819","nonce":4187707622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6823","nonce":4187707622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6821","nonce":4187707622}]},"public_addr":"172.21.15.92:6817/4187707622","cluster_addr":"172.21.15.92:6819/4187707622","heartbeat_back_addr":"172.21.15.92:6823/4187707622","heartbeat_front_addr":"172.21.15.92:6821/4187707622","state":["exists","up"]},{"osd":8,"uuid":"f03bbf0d-1281-4bcf-9773-9854388e65ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6817","nonce":3237612748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6819","nonce":3237612748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6823","nonce":3237612748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6821","nonce":3237612748}]},"public_addr":"172.21.15.152:6817/3237612748","cluster_addr":"172.21.15.152:6819/3237612748","heartbeat_back_addr":"172.21.15.152:6823/3237612748","heartbeat_front_addr":"172.21.15.152:6821/3237612748","state":["exists","up"]},{"osd":9,"uuid":"e7e8b775-99ff-48a4-a23a-91e7ab6622bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6827","nonce":2835218294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6829","nonce":2835218294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6833","nonce":2835218294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6831","nonce":2835218294}]},"public_addr":"172.21.15.86:6827/2835218294","cluster_addr":"172.21.15.86:6829/2835218294","heartbeat_back_addr":"172.21.15.86:6833/2835218294","heartbeat_front_addr":"172.21.15.86:6831/2835218294","state":["exists","up"]},{"osd":10,"uuid":"d9d616cd-a229-44de-8768-02a4415df115","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6825","nonce":4259465100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6827","nonce":4259465100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6831","nonce":4259465100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6829","nonce":4259465100}]},"public_addr":"172.21.15.92:6825/4259465100","cluster_addr":"172.21.15.92:6827/4259465100","heartbeat_back_addr":"172.21.15.92:6831/4259465100","heartbeat_front_addr":"172.21.15.92:6829/4259465100","state":["exists","up"]},{"osd":11,"uuid":"e7c0b865-ccf5-4233-8d22-0d9b6dbe5c97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6825","nonce":194662760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6827","nonce":194662760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6831","nonce":194662760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6829","nonce":194662760}]},"public_addr":"172.21.15.152:6825/194662760","cluster_addr":"172.21.15.152:6827/194662760","heartbeat_back_addr":"172.21.15.152:6831/194662760","heartbeat_front_addr":"172.21.15.152:6829/194662760","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:44:47.269167+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:45:05.363655+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:45:23.192677+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:45:44.156228+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:02.532971+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:20.153295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:41.713888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:59.676516+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:47:17.207087+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:47:38.970316+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:47:57.282525+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:48:15.437606+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/3553500912":"2024-03-24T17:43:28.242492+0000","172.21.15.86:6801/3600278228":"2024-03-24T17:43:28.242492+0000","172.21.15.86:0/887262329":"2024-03-24T17:43:28.242492+0000","172.21.15.86:6800/2313723492":"2024-03-24T17:43:05.043910+0000","172.21.15.86:6800/3600278228":"2024-03-24T17:43:28.242492+0000","172.21.15.86:0/3815705642":"2024-03-24T17:43:05.043910+0000","172.21.15.86:0/1614943062":"2024-03-24T17:43:28.242492+0000","172.21.15.86:6801/2313723492":"2024-03-24T17:43:05.043910+0000","172.21.15.86:0/2416321716":"2024-03-24T17:43:05.043910+0000","172.21.15.86:0/433588188":"2024-03-24T17:43:05.043910+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T17:49:45.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:45.929+0000 7fb439600700 1 -- 172.21.15.86:0/1636539247 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fb4180a11b0 msgr2=0x7fb4180a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:45.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:45.929+0000 7fb439600700 1 --2- 172.21.15.86:0/1636539247 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fb4180a11b0 0x7fb4180a3670 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb4340fe650 tx=0x7fb424009260 comp rx=0 tx=0).stop 2024-03-23T17:49:45.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:45.929+0000 7fb439600700 1 -- 172.21.15.86:0/1636539247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb434101e70 msgr2=0x7fb434197250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:45.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:45.929+0000 7fb439600700 1 --2- 172.21.15.86:0/1636539247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb434101e70 0x7fb434197250 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb420009840 tx=0x7fb42000b790 comp rx=0 tx=0).stop 2024-03-23T17:49:45.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:45.929+0000 7fb439600700 1 -- 172.21.15.86:0/1636539247 shutdown_connections 2024-03-23T17:49:45.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:45.929+0000 7fb439600700 1 --2- 172.21.15.86:0/1636539247 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb434102810 0x7fb43419e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:45.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:45.929+0000 7fb439600700 1 --2- 172.21.15.86:0/1636539247 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb434100c80 0x7fb434196d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:45.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:45.929+0000 7fb439600700 1 --2- 172.21.15.86:0/1636539247 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fb4180a11b0 0x7fb4180a3670 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:45.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:45.929+0000 7fb439600700 1 --2- 172.21.15.86:0/1636539247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb434101e70 0x7fb434197250 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:45.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:45.929+0000 7fb439600700 1 -- 172.21.15.86:0/1636539247 >> 172.21.15.86:0/1636539247 conn(0x7fb4340fc1f0 msgr2=0x7fb434105230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:45.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:45.929+0000 7fb439600700 1 -- 172.21.15.86:0/1636539247 shutdown_connections 2024-03-23T17:49:45.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:45.929+0000 7fb439600700 1 -- 172.21.15.86:0/1636539247 wait complete. 2024-03-23T17:49:45.949 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T17:45:26.848397+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T17:48:54.331894+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T17:48:55.640693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T17:49:45.949 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T17:49:46.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.144+0000 7fa5c7400700 1 -- 172.21.15.86:0/3895537107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5c0100d80 msgr2=0x7fa5c01011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:46.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.144+0000 7fa5c7400700 1 --2- 172.21.15.86:0/3895537107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5c0100d80 0x7fa5c01011a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa5b40099c0 tx=0x7fa5b4009cd0 comp rx=0 tx=0).stop 2024-03-23T17:49:46.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.144+0000 7fa5c7400700 1 -- 172.21.15.86:0/3895537107 shutdown_connections 2024-03-23T17:49:46.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.144+0000 7fa5c7400700 1 --2- 172.21.15.86:0/3895537107 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5c0102910 0x7fa5c01091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:46.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.144+0000 7fa5c7400700 1 --2- 172.21.15.86:0/3895537107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5c0100d80 0x7fa5c01011a0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:46.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.144+0000 7fa5c7400700 1 --2- 172.21.15.86:0/3895537107 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa5c0101f70 0x7fa5c01023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:46.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.144+0000 7fa5c7400700 1 -- 172.21.15.86:0/3895537107 >> 172.21.15.86:0/3895537107 conn(0x7fa5c00fc2f0 msgr2=0x7fa5c00fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:46.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.144+0000 7fa5c7400700 1 -- 172.21.15.86:0/3895537107 shutdown_connections 2024-03-23T17:49:46.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.144+0000 7fa5c7400700 1 -- 172.21.15.86:0/3895537107 wait complete. 2024-03-23T17:49:46.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.145+0000 7fa5c7400700 1 Processor -- start 2024-03-23T17:49:46.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.145+0000 7fa5c7400700 1 -- start start 2024-03-23T17:49:46.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.145+0000 7fa5c7400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa5c0100d80 0x7fa5c0198ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:46.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.145+0000 7fa5c7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5c0101f70 0x7fa5c0199530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:46.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.145+0000 7fa5c7400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5c0102910 0x7fa5c01a0500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:46.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.145+0000 7fa5c7400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c01a0ad0 con 0x7fa5c0101f70 2024-03-23T17:49:46.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.145+0000 7fa5c7400700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c01a0c10 con 0x7fa5c0102910 2024-03-23T17:49:46.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.145+0000 7fa5c7400700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c01a0da0 con 0x7fa5c0100d80 2024-03-23T17:49:46.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.145+0000 7fa5bfe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5c0101f70 0x7fa5c0199530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:46.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.145+0000 7fa5bfe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5c0101f70 0x7fa5c0199530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46842/0 (socket says 172.21.15.86:46842) 2024-03-23T17:49:46.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.145+0000 7fa5bfe00700 1 -- 172.21.15.86:0/1220211748 learned_addr learned my addr 172.21.15.86:0/1220211748 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:46.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.146+0000 7fa5c4a00700 1 --2- 172.21.15.86:0/1220211748 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa5c0100d80 0x7fa5c0198ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:46.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.146+0000 7fa5c5400700 1 --2- 172.21.15.86:0/1220211748 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5c0102910 0x7fa5c01a0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:46.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.146+0000 7fa5bfe00700 1 -- 172.21.15.86:0/1220211748 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa5c0100d80 msgr2=0x7fa5c0198ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:46.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.146+0000 7fa5bfe00700 1 --2- 172.21.15.86:0/1220211748 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa5c0100d80 0x7fa5c0198ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:46.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.146+0000 7fa5bfe00700 1 -- 172.21.15.86:0/1220211748 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5c0102910 msgr2=0x7fa5c01a0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:46.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.146+0000 7fa5bfe00700 1 --2- 172.21.15.86:0/1220211748 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5c0102910 0x7fa5c01a0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:46.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.146+0000 7fa5bfe00700 1 -- 172.21.15.86:0/1220211748 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5b40095f0 con 0x7fa5c0101f70 2024-03-23T17:49:46.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.146+0000 7fa5bfe00700 1 --2- 172.21.15.86:0/1220211748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5c0101f70 0x7fa5c0199530 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac00e9a0 tx=0x7fa5ac00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:46.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.146+0000 7fa5bd600700 1 -- 172.21.15.86:0/1220211748 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac009790 con 0x7fa5c0101f70 2024-03-23T17:49:46.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.146+0000 7fa5bd600700 1 -- 172.21.15.86:0/1220211748 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa5ac010430 con 0x7fa5c0101f70 2024-03-23T17:49:46.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.146+0000 7fa5bd600700 1 -- 172.21.15.86:0/1220211748 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac018680 con 0x7fa5c0101f70 2024-03-23T17:49:46.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.146+0000 7fa5c7400700 1 -- 172.21.15.86:0/1220211748 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5c01a1080 con 0x7fa5c0101f70 2024-03-23T17:49:46.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.147+0000 7fa5c7400700 1 -- 172.21.15.86:0/1220211748 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5c01a1540 con 0x7fa5c0101f70 2024-03-23T17:49:46.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.147+0000 7fa5c7400700 1 -- 172.21.15.86:0/1220211748 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5c0067af0 con 0x7fa5c0101f70 2024-03-23T17:49:46.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.149+0000 7fa5bd600700 1 -- 172.21.15.86:0/1220211748 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fa5ac0105a0 con 0x7fa5c0101f70 2024-03-23T17:49:46.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.151+0000 7fa5bd600700 1 --2- 172.21.15.86:0/1220211748 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fa5a80a1230 0x7fa5a80a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:46.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.152+0000 7fa5bd600700 1 -- 172.21.15.86:0/1220211748 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fa5ac014070 con 0x7fa5c0101f70 2024-03-23T17:49:46.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.152+0000 7fa5c4a00700 1 --2- 172.21.15.86:0/1220211748 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fa5a80a1230 0x7fa5a80a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:46.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.152+0000 7fa5c4a00700 1 --2- 172.21.15.86:0/1220211748 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fa5a80a1230 0x7fa5a80a36f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa5b4009990 tx=0x7fa5b4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:46.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.155+0000 7fa5bd600700 1 -- 172.21.15.86:0/1220211748 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa5ac0529f0 con 0x7fa5c0101f70 2024-03-23T17:49:46.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:45 smithi086 ceph-mon[38221]: pgmap v260: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s; 14 B/s, 0 keys/s, 1 objects/s recovering 2024-03-23T17:49:46.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:45 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/1404254946' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-23T17:49:46.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:45 smithi092 ceph-mon[79089]: pgmap v260: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s; 14 B/s, 0 keys/s, 1 objects/s recovering 2024-03-23T17:49:46.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:45 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/1404254946' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-23T17:49:46.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:45 smithi152 ceph-mon[47949]: pgmap v260: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s; 14 B/s, 0 keys/s, 1 objects/s recovering 2024-03-23T17:49:46.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:45 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/1404254946' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-23T17:49:46.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.362+0000 7fa5c7400700 1 -- 172.21.15.86:0/1220211748 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa5c01a17c0 con 0x7fa5c0101f70 2024-03-23T17:49:46.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.363+0000 7fa5bd600700 1 -- 172.21.15.86:0/1220211748 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa5ac0961d0 con 0x7fa5c0101f70 2024-03-23T17:49:46.364 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-03-23T17:49:46.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.365+0000 7fa5c7400700 1 -- 172.21.15.86:0/1220211748 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fa5a80a1230 msgr2=0x7fa5a80a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:46.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.365+0000 7fa5c7400700 1 --2- 172.21.15.86:0/1220211748 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fa5a80a1230 0x7fa5a80a36f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa5b4009990 tx=0x7fa5b4000bc0 comp rx=0 tx=0).stop 2024-03-23T17:49:46.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.365+0000 7fa5c7400700 1 -- 172.21.15.86:0/1220211748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5c0101f70 msgr2=0x7fa5c0199530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:46.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.365+0000 7fa5c7400700 1 --2- 172.21.15.86:0/1220211748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5c0101f70 0x7fa5c0199530 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac00e9a0 tx=0x7fa5ac00ecb0 comp rx=0 tx=0).stop 2024-03-23T17:49:46.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.366+0000 7fa5c7400700 1 -- 172.21.15.86:0/1220211748 shutdown_connections 2024-03-23T17:49:46.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.366+0000 7fa5c7400700 1 --2- 172.21.15.86:0/1220211748 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5c0102910 0x7fa5c01a0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:46.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.366+0000 7fa5c7400700 1 --2- 172.21.15.86:0/1220211748 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa5c0100d80 0x7fa5c0198ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:46.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.366+0000 7fa5c7400700 1 --2- 172.21.15.86:0/1220211748 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fa5a80a1230 0x7fa5a80a36f0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:46.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.366+0000 7fa5c7400700 1 --2- 172.21.15.86:0/1220211748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5c0101f70 0x7fa5c0199530 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:46.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.366+0000 7fa5c7400700 1 -- 172.21.15.86:0/1220211748 >> 172.21.15.86:0/1220211748 conn(0x7fa5c00fc2f0 msgr2=0x7fa5c0072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:46.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.366+0000 7fa5c7400700 1 -- 172.21.15.86:0/1220211748 shutdown_connections 2024-03-23T17:49:46.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.366+0000 7fa5c7400700 1 -- 172.21.15.86:0/1220211748 wait complete. 2024-03-23T17:49:46.386 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T17:49:46.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.579+0000 7faaeca00700 1 -- 172.21.15.86:0/4121369143 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faae8101f70 msgr2=0x7faae81023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:46.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.579+0000 7faaeca00700 1 --2- 172.21.15.86:0/4121369143 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faae8101f70 0x7faae81023d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7faad4009920 tx=0x7faad4009c30 comp rx=0 tx=0).stop 2024-03-23T17:49:46.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.579+0000 7faaeca00700 1 -- 172.21.15.86:0/4121369143 shutdown_connections 2024-03-23T17:49:46.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.579+0000 7faaeca00700 1 --2- 172.21.15.86:0/4121369143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faae8102910 0x7faae81091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:46.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.579+0000 7faaeca00700 1 --2- 172.21.15.86:0/4121369143 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faae8100d80 0x7faae81011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:46.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.579+0000 7faaeca00700 1 --2- 172.21.15.86:0/4121369143 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faae8101f70 0x7faae81023d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:46.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.579+0000 7faaeca00700 1 -- 172.21.15.86:0/4121369143 >> 172.21.15.86:0/4121369143 conn(0x7faae80fc2f0 msgr2=0x7faae80fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:46.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.579+0000 7faaeca00700 1 -- 172.21.15.86:0/4121369143 shutdown_connections 2024-03-23T17:49:46.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.579+0000 7faaeca00700 1 -- 172.21.15.86:0/4121369143 wait complete. 2024-03-23T17:49:46.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.580+0000 7faaeca00700 1 Processor -- start 2024-03-23T17:49:46.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.580+0000 7faaeca00700 1 -- start start 2024-03-23T17:49:46.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.580+0000 7faaeca00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faae8100d80 0x7faae8196e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:46.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.580+0000 7faaeca00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faae8101f70 0x7faae81973a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:46.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.580+0000 7faaeca00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faae8102910 0x7faae819e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:46.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.580+0000 7faaeca00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faae819e900 con 0x7faae8101f70 2024-03-23T17:49:46.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.580+0000 7faaeca00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faae819ea70 con 0x7faae8102910 2024-03-23T17:49:46.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.580+0000 7faaeca00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faae819ec30 con 0x7faae8100d80 2024-03-23T17:49:46.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.581+0000 7faae5400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faae8101f70 0x7faae81973a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:46.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.581+0000 7faae5400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faae8101f70 0x7faae81973a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46858/0 (socket says 172.21.15.86:46858) 2024-03-23T17:49:46.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.581+0000 7faae5400700 1 -- 172.21.15.86:0/1963588199 learned_addr learned my addr 172.21.15.86:0/1963588199 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:46.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.581+0000 7faae5e00700 1 --2- 172.21.15.86:0/1963588199 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faae8100d80 0x7faae8196e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:46.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.581+0000 7faae6800700 1 --2- 172.21.15.86:0/1963588199 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faae8102910 0x7faae819e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:46.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.581+0000 7faae5400700 1 -- 172.21.15.86:0/1963588199 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faae8100d80 msgr2=0x7faae8196e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:46.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.581+0000 7faae5400700 1 --2- 172.21.15.86:0/1963588199 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faae8100d80 0x7faae8196e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:46.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.581+0000 7faae5400700 1 -- 172.21.15.86:0/1963588199 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faae8102910 msgr2=0x7faae819e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:46.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.581+0000 7faae5400700 1 --2- 172.21.15.86:0/1963588199 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faae8102910 0x7faae819e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:46.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.581+0000 7faae5400700 1 -- 172.21.15.86:0/1963588199 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faae0009d00 con 0x7faae8101f70 2024-03-23T17:49:46.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.581+0000 7faae6800700 1 --2- 172.21.15.86:0/1963588199 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faae8102910 0x7faae819e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:49:46.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.581+0000 7faae5400700 1 --2- 172.21.15.86:0/1963588199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faae8101f70 0x7faae81973a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7faad4009840 tx=0x7faad400b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:46.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.581+0000 7faadea00700 1 -- 172.21.15.86:0/1963588199 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad401c070 con 0x7faae8101f70 2024-03-23T17:49:46.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.581+0000 7faadea00700 1 -- 172.21.15.86:0/1963588199 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faad400f7d0 con 0x7faae8101f70 2024-03-23T17:49:46.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.581+0000 7faadea00700 1 -- 172.21.15.86:0/1963588199 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad4003bb0 con 0x7faae8101f70 2024-03-23T17:49:46.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.581+0000 7faaeca00700 1 -- 172.21.15.86:0/1963588199 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faad4009520 con 0x7faae8101f70 2024-03-23T17:49:46.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.582+0000 7faaeca00700 1 -- 172.21.15.86:0/1963588199 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faae819f0f0 con 0x7faae8101f70 2024-03-23T17:49:46.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.583+0000 7faaeca00700 1 -- 172.21.15.86:0/1963588199 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faae81082f0 con 0x7faae8101f70 2024-03-23T17:49:46.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.584+0000 7faadea00700 1 -- 172.21.15.86:0/1963588199 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7faad4003d50 con 0x7faae8101f70 2024-03-23T17:49:46.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.586+0000 7faadea00700 1 --2- 172.21.15.86:0/1963588199 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7faacc0a1230 0x7faacc0a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:46.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.586+0000 7faadea00700 1 -- 172.21.15.86:0/1963588199 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7faad40cd750 con 0x7faae8101f70 2024-03-23T17:49:46.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.586+0000 7faae5e00700 1 --2- 172.21.15.86:0/1963588199 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7faacc0a1230 0x7faacc0a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:46.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.586+0000 7faae5e00700 1 --2- 172.21.15.86:0/1963588199 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7faacc0a1230 0x7faacc0a36f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7faae8198290 tx=0x7faad8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:46.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.589+0000 7faadea00700 1 -- 172.21.15.86:0/1963588199 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7faad4096de0 con 0x7faae8101f70 2024-03-23T17:49:46.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.799+0000 7faaeca00700 1 -- 172.21.15.86:0/1963588199 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7faae8107cd0 con 0x7faae8101f70 2024-03-23T17:49:46.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.800+0000 7faadea00700 1 -- 172.21.15.86:0/1963588199 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v82) v1 ==== 104+0+11 (secure 0 0 0) 0x7faad409a400 con 0x7faae8101f70 2024-03-23T17:49:46.801 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-03-23T17:49:46.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.802+0000 7faaeca00700 1 -- 172.21.15.86:0/1963588199 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7faacc0a1230 msgr2=0x7faacc0a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:46.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.802+0000 7faaeca00700 1 --2- 172.21.15.86:0/1963588199 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7faacc0a1230 0x7faacc0a36f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7faae8198290 tx=0x7faad8009260 comp rx=0 tx=0).stop 2024-03-23T17:49:46.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.802+0000 7faaeca00700 1 -- 172.21.15.86:0/1963588199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faae8101f70 msgr2=0x7faae81973a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:46.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.802+0000 7faaeca00700 1 --2- 172.21.15.86:0/1963588199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faae8101f70 0x7faae81973a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7faad4009840 tx=0x7faad400b790 comp rx=0 tx=0).stop 2024-03-23T17:49:46.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.803+0000 7faaeca00700 1 -- 172.21.15.86:0/1963588199 shutdown_connections 2024-03-23T17:49:46.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.803+0000 7faaeca00700 1 --2- 172.21.15.86:0/1963588199 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faae8102910 0x7faae819e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:46.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.803+0000 7faaeca00700 1 --2- 172.21.15.86:0/1963588199 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faae8100d80 0x7faae8196e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:46.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.803+0000 7faaeca00700 1 --2- 172.21.15.86:0/1963588199 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7faacc0a1230 0x7faacc0a36f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:46.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.803+0000 7faaeca00700 1 --2- 172.21.15.86:0/1963588199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faae8101f70 0x7faae81973a0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:46.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.803+0000 7faaeca00700 1 -- 172.21.15.86:0/1963588199 >> 172.21.15.86:0/1963588199 conn(0x7faae80fc2f0 msgr2=0x7faae8105430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:46.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.803+0000 7faaeca00700 1 -- 172.21.15.86:0/1963588199 shutdown_connections 2024-03-23T17:49:46.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:46.803+0000 7faaeca00700 1 -- 172.21.15.86:0/1963588199 wait complete. 2024-03-23T17:49:46.822 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T17:49:47.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.016+0000 7fa59c200700 1 -- 172.21.15.86:0/1678989308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa594101f70 msgr2=0x7fa5941023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:47.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.016+0000 7fa59c200700 1 --2- 172.21.15.86:0/1678989308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa594101f70 0x7fa5941023d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa5800099c0 tx=0x7fa580009cd0 comp rx=0 tx=0).stop 2024-03-23T17:49:47.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.016+0000 7fa59c200700 1 -- 172.21.15.86:0/1678989308 shutdown_connections 2024-03-23T17:49:47.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.016+0000 7fa59c200700 1 --2- 172.21.15.86:0/1678989308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa594102910 0x7fa5941091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:47.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.016+0000 7fa59c200700 1 --2- 172.21.15.86:0/1678989308 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa594100d80 0x7fa5941011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:47.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.016+0000 7fa59c200700 1 --2- 172.21.15.86:0/1678989308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa594101f70 0x7fa5941023d0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:47.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.016+0000 7fa59c200700 1 -- 172.21.15.86:0/1678989308 >> 172.21.15.86:0/1678989308 conn(0x7fa5940fc2f0 msgr2=0x7fa5940fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:47.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.016+0000 7fa59c200700 1 -- 172.21.15.86:0/1678989308 shutdown_connections 2024-03-23T17:49:47.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.016+0000 7fa59c200700 1 -- 172.21.15.86:0/1678989308 wait complete. 2024-03-23T17:49:47.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.017+0000 7fa59c200700 1 Processor -- start 2024-03-23T17:49:47.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.017+0000 7fa59c200700 1 -- start start 2024-03-23T17:49:47.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.018+0000 7fa59c200700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa594100d80 0x7fa594074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:47.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.018+0000 7fa59c200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa594101f70 0x7fa5940731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:47.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.018+0000 7fa59c200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa594102910 0x7fa5940736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:47.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.018+0000 7fa59c200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa594073e80 con 0x7fa594102910 2024-03-23T17:49:47.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.018+0000 7fa59c200700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa594073ff0 con 0x7fa594101f70 2024-03-23T17:49:47.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.018+0000 7fa59c200700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5941a0d90 con 0x7fa594100d80 2024-03-23T17:49:47.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.018+0000 7fa598e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa594101f70 0x7fa5940731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:47.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.018+0000 7fa59a200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa594102910 0x7fa5940736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:47.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.018+0000 7fa598e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa594101f70 0x7fa5940731a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:42538/0 (socket says 172.21.15.86:42538) 2024-03-23T17:49:47.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.018+0000 7fa598e00700 1 -- 172.21.15.86:0/1024999656 learned_addr learned my addr 172.21.15.86:0/1024999656 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:47.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.018+0000 7fa599800700 1 --2- 172.21.15.86:0/1024999656 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa594100d80 0x7fa594074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:47.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.018+0000 7fa598e00700 1 -- 172.21.15.86:0/1024999656 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa594100d80 msgr2=0x7fa594074b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:47.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.018+0000 7fa598e00700 1 --2- 172.21.15.86:0/1024999656 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa594100d80 0x7fa594074b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:47.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.018+0000 7fa598e00700 1 -- 172.21.15.86:0/1024999656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa594102910 msgr2=0x7fa5940736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:47.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.018+0000 7fa598e00700 1 --2- 172.21.15.86:0/1024999656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa594102910 0x7fa5940736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:47.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.018+0000 7fa598e00700 1 -- 172.21.15.86:0/1024999656 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5800095f0 con 0x7fa594101f70 2024-03-23T17:49:47.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.019+0000 7fa598e00700 1 --2- 172.21.15.86:0/1024999656 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa594101f70 0x7fa5940731a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa580006010 tx=0x7fa58000f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:47.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.019+0000 7fa592000700 1 -- 172.21.15.86:0/1024999656 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58001d070 con 0x7fa594101f70 2024-03-23T17:49:47.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.019+0000 7fa592000700 1 -- 172.21.15.86:0/1024999656 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa580005030 con 0x7fa594101f70 2024-03-23T17:49:47.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.019+0000 7fa59c200700 1 -- 172.21.15.86:0/1024999656 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5941a0fc0 con 0x7fa594101f70 2024-03-23T17:49:47.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.019+0000 7fa592000700 1 -- 172.21.15.86:0/1024999656 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa580004b10 con 0x7fa594101f70 2024-03-23T17:49:47.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.019+0000 7fa59c200700 1 -- 172.21.15.86:0/1024999656 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5941a1330 con 0x7fa594101f70 2024-03-23T17:49:47.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.020+0000 7fa59c200700 1 -- 172.21.15.86:0/1024999656 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa594108220 con 0x7fa594101f70 2024-03-23T17:49:47.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.022+0000 7fa592000700 1 -- 172.21.15.86:0/1024999656 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fa5800051a0 con 0x7fa594101f70 2024-03-23T17:49:47.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.023+0000 7fa592000700 1 --2- 172.21.15.86:0/1024999656 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fa57c0a11b0 0x7fa57c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:47.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.023+0000 7fa592000700 1 -- 172.21.15.86:0/1024999656 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fa5800cd780 con 0x7fa594101f70 2024-03-23T17:49:47.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.023+0000 7fa599800700 1 --2- 172.21.15.86:0/1024999656 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fa57c0a11b0 0x7fa57c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:47.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.024+0000 7fa599800700 1 --2- 172.21.15.86:0/1024999656 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fa57c0a11b0 0x7fa57c0a3670 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa594101dd0 tx=0x7fa588009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:47.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.027+0000 7fa592000700 1 -- 172.21.15.86:0/1024999656 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa580096e10 con 0x7fa594101f70 2024-03-23T17:49:47.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.230+0000 7fa59c200700 1 -- 172.21.15.86:0/1024999656 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa59410c6f0 con 0x7fa594101f70 2024-03-23T17:49:47.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.231+0000 7fa592000700 1 -- 172.21.15.86:0/1024999656 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v82) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa58009a430 con 0x7fa594101f70 2024-03-23T17:49:47.232 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-03-23T17:49:47.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.234+0000 7fa59c200700 1 -- 172.21.15.86:0/1024999656 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fa57c0a11b0 msgr2=0x7fa57c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:47.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.234+0000 7fa59c200700 1 --2- 172.21.15.86:0/1024999656 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fa57c0a11b0 0x7fa57c0a3670 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa594101dd0 tx=0x7fa588009260 comp rx=0 tx=0).stop 2024-03-23T17:49:47.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.234+0000 7fa59c200700 1 -- 172.21.15.86:0/1024999656 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa594101f70 msgr2=0x7fa5940731a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:47.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.234+0000 7fa59c200700 1 --2- 172.21.15.86:0/1024999656 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa594101f70 0x7fa5940731a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa580006010 tx=0x7fa58000f690 comp rx=0 tx=0).stop 2024-03-23T17:49:47.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.234+0000 7fa59c200700 1 -- 172.21.15.86:0/1024999656 shutdown_connections 2024-03-23T17:49:47.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.234+0000 7fa59c200700 1 --2- 172.21.15.86:0/1024999656 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fa57c0a11b0 0x7fa57c0a3670 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:47.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.234+0000 7fa59c200700 1 --2- 172.21.15.86:0/1024999656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa594102910 0x7fa5940736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:47.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.234+0000 7fa59c200700 1 --2- 172.21.15.86:0/1024999656 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa594100d80 0x7fa594074b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:47.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.234+0000 7fa59c200700 1 --2- 172.21.15.86:0/1024999656 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa594101f70 0x7fa5940731a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:47.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.234+0000 7fa59c200700 1 -- 172.21.15.86:0/1024999656 >> 172.21.15.86:0/1024999656 conn(0x7fa5940fc2f0 msgr2=0x7fa594105360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:47.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.234+0000 7fa59c200700 1 -- 172.21.15.86:0/1024999656 shutdown_connections 2024-03-23T17:49:47.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.234+0000 7fa59c200700 1 -- 172.21.15.86:0/1024999656 wait complete. 2024-03-23T17:49:47.251 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:46 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/1636539247' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T17:49:47.251 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:46 smithi086 ceph-mon[38221]: pgmap v261: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-23T17:49:47.251 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:46 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/1220211748' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T17:49:47.251 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:46 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/1963588199' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T17:49:47.253 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T17:49:47.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:46 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/1636539247' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T17:49:47.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:46 smithi092 ceph-mon[79089]: pgmap v261: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-23T17:49:47.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:46 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/1220211748' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T17:49:47.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:46 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/1963588199' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T17:49:47.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:46 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/1636539247' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T17:49:47.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:46 smithi152 ceph-mon[47949]: pgmap v261: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-23T17:49:47.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:46 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/1220211748' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T17:49:47.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:46 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/1963588199' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T17:49:47.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.445+0000 7f097c200700 1 -- 172.21.15.86:0/748312528 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0974101710 msgr2=0x7f0974103b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:47.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.445+0000 7f097c200700 1 --2- 172.21.15.86:0/748312528 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0974101710 0x7f0974103b00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0968009990 tx=0x7f0968009ca0 comp rx=0 tx=0).stop 2024-03-23T17:49:47.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.445+0000 7f097c200700 1 -- 172.21.15.86:0/748312528 shutdown_connections 2024-03-23T17:49:47.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.445+0000 7f097c200700 1 --2- 172.21.15.86:0/748312528 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0974106970 0x7f0974108de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:47.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.445+0000 7f097c200700 1 --2- 172.21.15.86:0/748312528 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0974101710 0x7f0974103b00 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:47.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.445+0000 7f097c200700 1 --2- 172.21.15.86:0/748312528 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0974104040 0x7f0974106430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:47.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.445+0000 7f097c200700 1 -- 172.21.15.86:0/748312528 >> 172.21.15.86:0/748312528 conn(0x7f09740fb100 msgr2=0x7f09740fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:47.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.445+0000 7f097c200700 1 -- 172.21.15.86:0/748312528 shutdown_connections 2024-03-23T17:49:47.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.445+0000 7f097c200700 1 -- 172.21.15.86:0/748312528 wait complete. 2024-03-23T17:49:47.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.446+0000 7f097c200700 1 Processor -- start 2024-03-23T17:49:47.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.446+0000 7f097c200700 1 -- start start 2024-03-23T17:49:47.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.446+0000 7f097c200700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0974101710 0x7f0974198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:47.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.446+0000 7f097c200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0974104040 0x7f0974199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:47.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.446+0000 7f097c200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0974106970 0x7f09741a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:47.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.446+0000 7f097c200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09741a08f0 con 0x7f0974104040 2024-03-23T17:49:47.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.446+0000 7f097c200700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09741a0a30 con 0x7f0974106970 2024-03-23T17:49:47.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.446+0000 7f097c200700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09741a0bc0 con 0x7f0974101710 2024-03-23T17:49:47.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.447+0000 7f0978e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0974104040 0x7f0974199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:47.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.447+0000 7f0978e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0974104040 0x7f0974199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46904/0 (socket says 172.21.15.86:46904) 2024-03-23T17:49:47.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.447+0000 7f0978e00700 1 -- 172.21.15.86:0/2718642577 learned_addr learned my addr 172.21.15.86:0/2718642577 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:47.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.447+0000 7f097a200700 1 --2- 172.21.15.86:0/2718642577 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0974106970 0x7f09741a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:47.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.447+0000 7f0979800700 1 --2- 172.21.15.86:0/2718642577 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0974101710 0x7f0974198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:47.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.447+0000 7f0978e00700 1 -- 172.21.15.86:0/2718642577 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0974101710 msgr2=0x7f0974198dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:47.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.447+0000 7f0978e00700 1 --2- 172.21.15.86:0/2718642577 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0974101710 0x7f0974198dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:47.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.447+0000 7f0978e00700 1 -- 172.21.15.86:0/2718642577 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0974106970 msgr2=0x7f09741a0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:47.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.447+0000 7f0978e00700 1 --2- 172.21.15.86:0/2718642577 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0974106970 0x7f09741a0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:47.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.447+0000 7f0978e00700 1 -- 172.21.15.86:0/2718642577 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09680095f0 con 0x7f0974104040 2024-03-23T17:49:47.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.447+0000 7f0978e00700 1 --2- 172.21.15.86:0/2718642577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0974104040 0x7f0974199300 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f096000e9a0 tx=0x7f096000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:47.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.447+0000 7f0972000700 1 -- 172.21.15.86:0/2718642577 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0960009790 con 0x7f0974104040 2024-03-23T17:49:47.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.448+0000 7f0972000700 1 -- 172.21.15.86:0/2718642577 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0960010430 con 0x7f0974104040 2024-03-23T17:49:47.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.448+0000 7f0972000700 1 -- 172.21.15.86:0/2718642577 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0960018680 con 0x7f0974104040 2024-03-23T17:49:47.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.448+0000 7f097c200700 1 -- 172.21.15.86:0/2718642577 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09741a0ea0 con 0x7f0974104040 2024-03-23T17:49:47.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.448+0000 7f097c200700 1 -- 172.21.15.86:0/2718642577 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09741a1320 con 0x7f0974104040 2024-03-23T17:49:47.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.449+0000 7f097c200700 1 -- 172.21.15.86:0/2718642577 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0974108540 con 0x7f0974104040 2024-03-23T17:49:47.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.450+0000 7f0972000700 1 -- 172.21.15.86:0/2718642577 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f09600105a0 con 0x7f0974104040 2024-03-23T17:49:47.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.452+0000 7f0972000700 1 --2- 172.21.15.86:0/2718642577 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f095c0a1230 0x7f095c0a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:47.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.452+0000 7f0972000700 1 -- 172.21.15.86:0/2718642577 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f0960014070 con 0x7f0974104040 2024-03-23T17:49:47.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.452+0000 7f0979800700 1 --2- 172.21.15.86:0/2718642577 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f095c0a1230 0x7f095c0a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:47.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.453+0000 7f0979800700 1 --2- 172.21.15.86:0/2718642577 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f095c0a1230 0x7f095c0a36f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f096800b2e0 tx=0x7f0968005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:47.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.454+0000 7f0972000700 1 -- 172.21.15.86:0/2718642577 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f0960096440 con 0x7f0974104040 2024-03-23T17:49:47.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.707+0000 7f097c200700 1 -- 172.21.15.86:0/2718642577 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0974067180 con 0x7f0974104040 2024-03-23T17:49:47.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.708+0000 7f0972000700 1 -- 172.21.15.86:0/2718642577 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10913 (secure 0 0 0) 0x7f0960026070 con 0x7f0974104040 2024-03-23T17:49:47.709 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:49:47.710 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14445,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1433140219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1433140219},{"type":"v1","addr":"172.21.15.86:6835","nonce":1433140219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6839/1879457730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1879457730},{"type":"v1","addr":"172.21.15.86:6839","nonce":1879457730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-23T17:48:56.202323+0000","modified":"2024-03-23T17:49:03.703276+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24553,"mds_2":24518,"mds_3":24505,"mds_4":24500},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14514":{"gid":14514,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/3026943866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3026943866},{"type":"v1","addr":"172.21.15.86:6841","nonce":3026943866}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24464":{"gid":24464,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6833/1467091186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1467091186},{"type":"v1","addr":"172.21.15.92:6833","nonce":1467091186}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6833/3075554623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3075554623},{"type":"v1","addr":"172.21.15.152:6833","nonce":3075554623}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.92:6835/908900027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":908900027},{"type":"v1","addr":"172.21.15.92:6835","nonce":908900027}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"d","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.86:6837/2933881328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2933881328},{"type":"v1","addr":"172.21.15.86:6837","nonce":2933881328}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6837/3967837948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":3967837948},{"type":"v1","addr":"172.21.15.92:6837","nonce":3967837948}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6835/3137085021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3137085021},{"type":"v1","addr":"172.21.15.152:6835","nonce":3137085021}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24538":{"gid":24538,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6837/1151208965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":1151208965},{"type":"v1","addr":"172.21.15.152:6837","nonce":1151208965}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.152:6839/1100723992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":1100723992},{"type":"v1","addr":"172.21.15.152:6839","nonce":1100723992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6839/1394139887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1394139887},{"type":"v1","addr":"172.21.15.92:6839","nonce":1394139887}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-23T17:49:47.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.711+0000 7f097c200700 1 -- 172.21.15.86:0/2718642577 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f095c0a1230 msgr2=0x7f095c0a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:47.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.711+0000 7f097c200700 1 --2- 172.21.15.86:0/2718642577 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f095c0a1230 0x7f095c0a36f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f096800b2e0 tx=0x7f0968005fb0 comp rx=0 tx=0).stop 2024-03-23T17:49:47.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.711+0000 7f097c200700 1 -- 172.21.15.86:0/2718642577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0974104040 msgr2=0x7f0974199300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:47.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.711+0000 7f097c200700 1 --2- 172.21.15.86:0/2718642577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0974104040 0x7f0974199300 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f096000e9a0 tx=0x7f096000ecb0 comp rx=0 tx=0).stop 2024-03-23T17:49:47.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.711+0000 7f097c200700 1 -- 172.21.15.86:0/2718642577 shutdown_connections 2024-03-23T17:49:47.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.711+0000 7f097c200700 1 --2- 172.21.15.86:0/2718642577 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0974106970 0x7f09741a0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:47.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.711+0000 7f097c200700 1 --2- 172.21.15.86:0/2718642577 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0974101710 0x7f0974198dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:47.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.711+0000 7f097c200700 1 --2- 172.21.15.86:0/2718642577 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f095c0a1230 0x7f095c0a36f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:47.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.711+0000 7f097c200700 1 --2- 172.21.15.86:0/2718642577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0974104040 0x7f0974199300 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:47.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.711+0000 7f097c200700 1 -- 172.21.15.86:0/2718642577 >> 172.21.15.86:0/2718642577 conn(0x7f09740fb100 msgr2=0x7f0974100e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:47.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.711+0000 7f097c200700 1 -- 172.21.15.86:0/2718642577 shutdown_connections 2024-03-23T17:49:47.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.711+0000 7f097c200700 1 -- 172.21.15.86:0/2718642577 wait complete. 2024-03-23T17:49:47.714 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 27 2024-03-23T17:49:47.731 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T17:49:47.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.923+0000 7f4ad4e00700 1 -- 172.21.15.86:0/2192106727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ad0100c80 msgr2=0x7f4ad01010a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:47.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.923+0000 7f4ad4e00700 1 --2- 172.21.15.86:0/2192106727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ad0100c80 0x7f4ad01010a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4ac00099c0 tx=0x7f4ac0009cd0 comp rx=0 tx=0).stop 2024-03-23T17:49:47.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.923+0000 7f4ad4e00700 1 -- 172.21.15.86:0/2192106727 shutdown_connections 2024-03-23T17:49:47.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.923+0000 7f4ad4e00700 1 --2- 172.21.15.86:0/2192106727 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ad0102810 0x7f4ad01090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:47.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.923+0000 7f4ad4e00700 1 --2- 172.21.15.86:0/2192106727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ad0100c80 0x7f4ad01010a0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:47.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.923+0000 7f4ad4e00700 1 --2- 172.21.15.86:0/2192106727 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ad0101e70 0x7f4ad01022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:47.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.923+0000 7f4ad4e00700 1 -- 172.21.15.86:0/2192106727 >> 172.21.15.86:0/2192106727 conn(0x7f4ad00fc1f0 msgr2=0x7f4ad00fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:47.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.923+0000 7f4ad4e00700 1 -- 172.21.15.86:0/2192106727 shutdown_connections 2024-03-23T17:49:47.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.923+0000 7f4ad4e00700 1 -- 172.21.15.86:0/2192106727 wait complete. 2024-03-23T17:49:47.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.924+0000 7f4ad4e00700 1 Processor -- start 2024-03-23T17:49:47.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.925+0000 7f4ad4e00700 1 -- start start 2024-03-23T17:49:47.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.925+0000 7f4ad4e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ad0100c80 0x7f4ad0074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:47.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.925+0000 7f4ad4e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ad0101e70 0x7f4ad00731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:47.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.925+0000 7f4ad4e00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ad0102810 0x7f4ad00736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:47.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.925+0000 7f4ad4e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad0073e80 con 0x7f4ad0101e70 2024-03-23T17:49:47.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.925+0000 7f4ad4e00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad0073ff0 con 0x7f4ad0100c80 2024-03-23T17:49:47.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.925+0000 7f4ad4e00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad01a0c80 con 0x7f4ad0102810 2024-03-23T17:49:47.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.925+0000 7f4acd400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ad0101e70 0x7f4ad00731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:47.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.925+0000 7f4acde00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ad0100c80 0x7f4ad0074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:47.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.925+0000 7f4acde00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ad0100c80 0x7f4ad0074b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:42568/0 (socket says 172.21.15.86:42568) 2024-03-23T17:49:47.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.925+0000 7f4ace800700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ad0102810 0x7f4ad00736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:47.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.925+0000 7f4acd400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ad0101e70 0x7f4ad00731a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46930/0 (socket says 172.21.15.86:46930) 2024-03-23T17:49:47.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.925+0000 7f4acde00700 1 -- 172.21.15.86:0/444749903 learned_addr learned my addr 172.21.15.86:0/444749903 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:47.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.926+0000 7f4acde00700 1 -- 172.21.15.86:0/444749903 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ad0102810 msgr2=0x7f4ad00736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:47.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.926+0000 7f4acde00700 1 --2- 172.21.15.86:0/444749903 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ad0102810 0x7f4ad00736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:47.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.926+0000 7f4acde00700 1 -- 172.21.15.86:0/444749903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ad0101e70 msgr2=0x7f4ad00731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:47.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.926+0000 7f4acde00700 1 --2- 172.21.15.86:0/444749903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ad0101e70 0x7f4ad00731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:47.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.926+0000 7f4acde00700 1 -- 172.21.15.86:0/444749903 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ac00095f0 con 0x7f4ad0100c80 2024-03-23T17:49:47.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.926+0000 7f4acd400700 1 --2- 172.21.15.86:0/444749903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ad0101e70 0x7f4ad00731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:49:47.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.926+0000 7f4acde00700 1 --2- 172.21.15.86:0/444749903 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ad0100c80 0x7f4ad0074b10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0006010 tx=0x7f4ac000f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:47.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.926+0000 7f4ac6a00700 1 -- 172.21.15.86:0/444749903 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac001d070 con 0x7f4ad0100c80 2024-03-23T17:49:47.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.926+0000 7f4ac6a00700 1 -- 172.21.15.86:0/444749903 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ac0005020 con 0x7f4ad0100c80 2024-03-23T17:49:47.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.926+0000 7f4ad4e00700 1 -- 172.21.15.86:0/444749903 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ad01a0eb0 con 0x7f4ad0100c80 2024-03-23T17:49:47.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.926+0000 7f4ac6a00700 1 -- 172.21.15.86:0/444749903 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac0004b10 con 0x7f4ad0100c80 2024-03-23T17:49:47.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.926+0000 7f4ad4e00700 1 -- 172.21.15.86:0/444749903 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ad01a1220 con 0x7f4ad0100c80 2024-03-23T17:49:47.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.927+0000 7f4ad4e00700 1 -- 172.21.15.86:0/444749903 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ad0108100 con 0x7f4ad0100c80 2024-03-23T17:49:47.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.929+0000 7f4ac6a00700 1 -- 172.21.15.86:0/444749903 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f4ac0005190 con 0x7f4ad0100c80 2024-03-23T17:49:47.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.930+0000 7f4ac6a00700 1 --2- 172.21.15.86:0/444749903 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f4ab40a11b0 0x7f4ab40a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:47.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.930+0000 7f4ac6a00700 1 -- 172.21.15.86:0/444749903 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f4ac00cd4f0 con 0x7f4ad0100c80 2024-03-23T17:49:47.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.930+0000 7f4acd400700 1 --2- 172.21.15.86:0/444749903 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f4ab40a11b0 0x7f4ab40a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:47.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.931+0000 7f4acd400700 1 --2- 172.21.15.86:0/444749903 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f4ab40a11b0 0x7f4ab40a3670 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4abc011490 tx=0x7f4abc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:47.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:47.935+0000 7f4ac6a00700 1 -- 172.21.15.86:0/444749903 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f4ac0096b80 con 0x7f4ad0100c80 2024-03-23T17:49:48.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:47 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/1024999656' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T17:49:48.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:47 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/2718642577' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T17:49:48.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.174+0000 7f4ad4e00700 1 -- 172.21.15.86:0/444749903 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4ad010c4e0 con 0x7f4ad0100c80 2024-03-23T17:49:48.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.176+0000 7f4ac6a00700 1 -- 172.21.15.86:0/444749903 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22414 (secure 0 0 0) 0x7f4ac009a1a0 con 0x7f4ad0100c80 2024-03-23T17:49:48.177 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:49:48.177 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":82,"fsid":"9d76c9e8-e93c-11ee-95cd-87774f69a715","created":"2024-03-23T17:42:30.097233+0000","modified":"2024-03-23T17:49:29.773954+0000","last_up_change":"2024-03-23T17:48:18.217601+0000","last_in_change":"2024-03-23T17:48:00.250579+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T17:45:26.848397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T17:48:54.331894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T17:48:55.640693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"656655c6-1c7f-4640-92c4-2f42de820143","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6803","nonce":2423505491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6805","nonce":2423505491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6809","nonce":2423505491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6807","nonce":2423505491}]},"public_addr":"172.21.15.86:6803/2423505491","cluster_addr":"172.21.15.86:6805/2423505491","heartbeat_back_addr":"172.21.15.86:6809/2423505491","heartbeat_front_addr":"172.21.15.86:6807/2423505491","state":["exists","up"]},{"osd":1,"uuid":"3e86dc65-20a6-4de8-b9ce-fcf01f68862b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6801","nonce":3776252597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6803","nonce":3776252597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6807","nonce":3776252597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6805","nonce":3776252597}]},"public_addr":"172.21.15.92:6801/3776252597","cluster_addr":"172.21.15.92:6803/3776252597","heartbeat_back_addr":"172.21.15.92:6807/3776252597","heartbeat_front_addr":"172.21.15.92:6805/3776252597","state":["exists","up"]},{"osd":2,"uuid":"09808786-3bd3-4d11-8c7b-1736119dfbb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6801","nonce":1601265891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6803","nonce":1601265891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6807","nonce":1601265891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6805","nonce":1601265891}]},"public_addr":"172.21.15.152:6801/1601265891","cluster_addr":"172.21.15.152:6803/1601265891","heartbeat_back_addr":"172.21.15.152:6807/1601265891","heartbeat_front_addr":"172.21.15.152:6805/1601265891","state":["exists","up"]},{"osd":3,"uuid":"79c61910-4b53-424c-b150-6f5385a2cd68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6811","nonce":694846466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6813","nonce":694846466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6817","nonce":694846466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6815","nonce":694846466}]},"public_addr":"172.21.15.86:6811/694846466","cluster_addr":"172.21.15.86:6813/694846466","heartbeat_back_addr":"172.21.15.86:6817/694846466","heartbeat_front_addr":"172.21.15.86:6815/694846466","state":["exists","up"]},{"osd":4,"uuid":"a2878035-0179-469f-bbd4-8ce2d3f6bddc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6809","nonce":1422821471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6811","nonce":1422821471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6815","nonce":1422821471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6813","nonce":1422821471}]},"public_addr":"172.21.15.92:6809/1422821471","cluster_addr":"172.21.15.92:6811/1422821471","heartbeat_back_addr":"172.21.15.92:6815/1422821471","heartbeat_front_addr":"172.21.15.92:6813/1422821471","state":["exists","up"]},{"osd":5,"uuid":"837366aa-ce25-42ca-b157-5bbb50159d0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6809","nonce":1039375239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6811","nonce":1039375239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6815","nonce":1039375239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6813","nonce":1039375239}]},"public_addr":"172.21.15.152:6809/1039375239","cluster_addr":"172.21.15.152:6811/1039375239","heartbeat_back_addr":"172.21.15.152:6815/1039375239","heartbeat_front_addr":"172.21.15.152:6813/1039375239","state":["exists","up"]},{"osd":6,"uuid":"61024437-7b91-432f-a1af-6443d0ef5213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6819","nonce":4080452707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6821","nonce":4080452707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6825","nonce":4080452707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6823","nonce":4080452707}]},"public_addr":"172.21.15.86:6819/4080452707","cluster_addr":"172.21.15.86:6821/4080452707","heartbeat_back_addr":"172.21.15.86:6825/4080452707","heartbeat_front_addr":"172.21.15.86:6823/4080452707","state":["exists","up"]},{"osd":7,"uuid":"2684bdae-6b30-4afc-af68-4ce41b3a75b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6817","nonce":4187707622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6819","nonce":4187707622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6823","nonce":4187707622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6821","nonce":4187707622}]},"public_addr":"172.21.15.92:6817/4187707622","cluster_addr":"172.21.15.92:6819/4187707622","heartbeat_back_addr":"172.21.15.92:6823/4187707622","heartbeat_front_addr":"172.21.15.92:6821/4187707622","state":["exists","up"]},{"osd":8,"uuid":"f03bbf0d-1281-4bcf-9773-9854388e65ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6817","nonce":3237612748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6819","nonce":3237612748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6823","nonce":3237612748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6821","nonce":3237612748}]},"public_addr":"172.21.15.152:6817/3237612748","cluster_addr":"172.21.15.152:6819/3237612748","heartbeat_back_addr":"172.21.15.152:6823/3237612748","heartbeat_front_addr":"172.21.15.152:6821/3237612748","state":["exists","up"]},{"osd":9,"uuid":"e7e8b775-99ff-48a4-a23a-91e7ab6622bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6827","nonce":2835218294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6829","nonce":2835218294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6833","nonce":2835218294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6831","nonce":2835218294}]},"public_addr":"172.21.15.86:6827/2835218294","cluster_addr":"172.21.15.86:6829/2835218294","heartbeat_back_addr":"172.21.15.86:6833/2835218294","heartbeat_front_addr":"172.21.15.86:6831/2835218294","state":["exists","up"]},{"osd":10,"uuid":"d9d616cd-a229-44de-8768-02a4415df115","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6825","nonce":4259465100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6827","nonce":4259465100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6831","nonce":4259465100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6829","nonce":4259465100}]},"public_addr":"172.21.15.92:6825/4259465100","cluster_addr":"172.21.15.92:6827/4259465100","heartbeat_back_addr":"172.21.15.92:6831/4259465100","heartbeat_front_addr":"172.21.15.92:6829/4259465100","state":["exists","up"]},{"osd":11,"uuid":"e7c0b865-ccf5-4233-8d22-0d9b6dbe5c97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6825","nonce":194662760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6827","nonce":194662760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6831","nonce":194662760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6829","nonce":194662760}]},"public_addr":"172.21.15.152:6825/194662760","cluster_addr":"172.21.15.152:6827/194662760","heartbeat_back_addr":"172.21.15.152:6831/194662760","heartbeat_front_addr":"172.21.15.152:6829/194662760","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:44:47.269167+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:45:05.363655+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:45:23.192677+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:45:44.156228+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:02.532971+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:20.153295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:41.713888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:59.676516+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:47:17.207087+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:47:38.970316+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:47:57.282525+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:48:15.437606+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/3553500912":"2024-03-24T17:43:28.242492+0000","172.21.15.86:6801/3600278228":"2024-03-24T17:43:28.242492+0000","172.21.15.86:0/887262329":"2024-03-24T17:43:28.242492+0000","172.21.15.86:6800/2313723492":"2024-03-24T17:43:05.043910+0000","172.21.15.86:6800/3600278228":"2024-03-24T17:43:28.242492+0000","172.21.15.86:0/3815705642":"2024-03-24T17:43:05.043910+0000","172.21.15.86:0/1614943062":"2024-03-24T17:43:28.242492+0000","172.21.15.86:6801/2313723492":"2024-03-24T17:43:05.043910+0000","172.21.15.86:0/2416321716":"2024-03-24T17:43:05.043910+0000","172.21.15.86:0/433588188":"2024-03-24T17:43:05.043910+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T17:49:48.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.178+0000 7f4ad4e00700 1 -- 172.21.15.86:0/444749903 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f4ab40a11b0 msgr2=0x7f4ab40a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:48.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.178+0000 7f4ad4e00700 1 --2- 172.21.15.86:0/444749903 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f4ab40a11b0 0x7f4ab40a3670 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4abc011490 tx=0x7f4abc009260 comp rx=0 tx=0).stop 2024-03-23T17:49:48.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.178+0000 7f4ad4e00700 1 -- 172.21.15.86:0/444749903 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ad0100c80 msgr2=0x7f4ad0074b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:48.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.178+0000 7f4ad4e00700 1 --2- 172.21.15.86:0/444749903 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ad0100c80 0x7f4ad0074b10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0006010 tx=0x7f4ac000f690 comp rx=0 tx=0).stop 2024-03-23T17:49:48.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.178+0000 7f4ad4e00700 1 -- 172.21.15.86:0/444749903 shutdown_connections 2024-03-23T17:49:48.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.178+0000 7f4ad4e00700 1 --2- 172.21.15.86:0/444749903 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ad0102810 0x7f4ad00736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:48.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.178+0000 7f4ad4e00700 1 --2- 172.21.15.86:0/444749903 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ad0100c80 0x7f4ad0074b10 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:48.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.178+0000 7f4ad4e00700 1 --2- 172.21.15.86:0/444749903 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f4ab40a11b0 0x7f4ab40a3670 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:48.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.178+0000 7f4ad4e00700 1 --2- 172.21.15.86:0/444749903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ad0101e70 0x7f4ad00731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:48.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.178+0000 7f4ad4e00700 1 -- 172.21.15.86:0/444749903 >> 172.21.15.86:0/444749903 conn(0x7f4ad00fc1f0 msgr2=0x7f4ad0105240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:48.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.178+0000 7f4ad4e00700 1 -- 172.21.15.86:0/444749903 shutdown_connections 2024-03-23T17:49:48.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.178+0000 7f4ad4e00700 1 -- 172.21.15.86:0/444749903 wait complete. 2024-03-23T17:49:48.198 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T17:45:26.848397+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T17:48:54.331894+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T17:48:55.640693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T17:49:48.199 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T17:49:48.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:47 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/1024999656' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T17:49:48.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:47 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/2718642577' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T17:49:48.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:47 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/1024999656' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T17:49:48.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:47 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/2718642577' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T17:49:48.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.393+0000 7f8746e00700 1 -- 172.21.15.86:0/860609124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8740073810 msgr2=0x7f8740073c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:48.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.393+0000 7f8746e00700 1 --2- 172.21.15.86:0/860609124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8740073810 0x7f8740073c90 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f872c009990 tx=0x7f872c009ca0 comp rx=0 tx=0).stop 2024-03-23T17:49:48.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.393+0000 7f8746e00700 1 -- 172.21.15.86:0/860609124 shutdown_connections 2024-03-23T17:49:48.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.393+0000 7f8746e00700 1 --2- 172.21.15.86:0/860609124 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8740108c40 0x7f874010b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:48.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.393+0000 7f8746e00700 1 --2- 172.21.15.86:0/860609124 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8740074df0 0x7f8740073240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:48.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.393+0000 7f8746e00700 1 --2- 172.21.15.86:0/860609124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8740073810 0x7f8740073c90 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:48.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.393+0000 7f8746e00700 1 -- 172.21.15.86:0/860609124 >> 172.21.15.86:0/860609124 conn(0x7f87400fc2f0 msgr2=0x7f87400fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:48.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.393+0000 7f8746e00700 1 -- 172.21.15.86:0/860609124 shutdown_connections 2024-03-23T17:49:48.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.393+0000 7f8746e00700 1 -- 172.21.15.86:0/860609124 wait complete. 2024-03-23T17:49:48.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.394+0000 7f8746e00700 1 Processor -- start 2024-03-23T17:49:48.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.394+0000 7f8746e00700 1 -- start start 2024-03-23T17:49:48.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.394+0000 7f8746e00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8740073810 0x7f8740198f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:48.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.394+0000 7f8746e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8740074df0 0x7f87401994a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:48.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.394+0000 7f8746e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8740108c40 0x7f87401a04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:48.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.395+0000 7f8746e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87401a0a90 con 0x7f8740074df0 2024-03-23T17:49:48.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.395+0000 7f8746e00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87401a0bd0 con 0x7f8740108c40 2024-03-23T17:49:48.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.395+0000 7f8746e00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87401a0d60 con 0x7f8740073810 2024-03-23T17:49:48.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.395+0000 7f873f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8740074df0 0x7f87401994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:48.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.395+0000 7f873f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8740074df0 0x7f87401994a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46956/0 (socket says 172.21.15.86:46956) 2024-03-23T17:49:48.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.395+0000 7f873f400700 1 -- 172.21.15.86:0/1904571620 learned_addr learned my addr 172.21.15.86:0/1904571620 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:48.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.395+0000 7f8744e00700 1 --2- 172.21.15.86:0/1904571620 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8740108c40 0x7f87401a04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:48.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.395+0000 7f873fe00700 1 --2- 172.21.15.86:0/1904571620 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8740073810 0x7f8740198f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:48.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.395+0000 7f873f400700 1 -- 172.21.15.86:0/1904571620 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8740073810 msgr2=0x7f8740198f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:48.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.395+0000 7f873f400700 1 --2- 172.21.15.86:0/1904571620 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8740073810 0x7f8740198f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:48.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.395+0000 7f873f400700 1 -- 172.21.15.86:0/1904571620 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8740108c40 msgr2=0x7f87401a04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:48.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.395+0000 7f873f400700 1 --2- 172.21.15.86:0/1904571620 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8740108c40 0x7f87401a04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:48.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.395+0000 7f873f400700 1 -- 172.21.15.86:0/1904571620 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f872c0095f0 con 0x7f8740074df0 2024-03-23T17:49:48.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.395+0000 7f8744e00700 1 --2- 172.21.15.86:0/1904571620 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8740108c40 0x7f87401a04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:49:48.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.395+0000 7f873f400700 1 --2- 172.21.15.86:0/1904571620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8740074df0 0x7f87401994a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f872c0098e0 tx=0x7f872c00f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:48.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.396+0000 7f873cc00700 1 -- 172.21.15.86:0/1904571620 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872c01d070 con 0x7f8740074df0 2024-03-23T17:49:48.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.396+0000 7f873cc00700 1 -- 172.21.15.86:0/1904571620 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f872c005050 con 0x7f8740074df0 2024-03-23T17:49:48.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.396+0000 7f8746e00700 1 -- 172.21.15.86:0/1904571620 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87401a0fe0 con 0x7f8740074df0 2024-03-23T17:49:48.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.396+0000 7f873cc00700 1 -- 172.21.15.86:0/1904571620 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872c004bb0 con 0x7f8740074df0 2024-03-23T17:49:48.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.396+0000 7f8746e00700 1 -- 172.21.15.86:0/1904571620 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87401a13a0 con 0x7f8740074df0 2024-03-23T17:49:48.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.397+0000 7f8746e00700 1 -- 172.21.15.86:0/1904571620 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8740107a00 con 0x7f8740074df0 2024-03-23T17:49:48.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.399+0000 7f873cc00700 1 -- 172.21.15.86:0/1904571620 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f872c004d10 con 0x7f8740074df0 2024-03-23T17:49:48.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.400+0000 7f873cc00700 1 --2- 172.21.15.86:0/1904571620 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f87240a1230 0x7f87240a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:48.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.400+0000 7f873cc00700 1 -- 172.21.15.86:0/1904571620 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f872c0cdd70 con 0x7f8740074df0 2024-03-23T17:49:48.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.400+0000 7f873fe00700 1 --2- 172.21.15.86:0/1904571620 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f87240a1230 0x7f87240a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:48.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.401+0000 7f873fe00700 1 --2- 172.21.15.86:0/1904571620 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f87240a1230 0x7f87240a36f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8730005950 tx=0x7f873000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:48.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.402+0000 7f873cc00700 1 -- 172.21.15.86:0/1904571620 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f872c097520 con 0x7f8740074df0 2024-03-23T17:49:48.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.607+0000 7f8746e00700 1 -- 172.21.15.86:0/1904571620 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f874010e8d0 con 0x7f8740074df0 2024-03-23T17:49:48.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.607+0000 7f873cc00700 1 -- 172.21.15.86:0/1904571620 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f872c026020 con 0x7f8740074df0 2024-03-23T17:49:48.608 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-03-23T17:49:48.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.610+0000 7f8746e00700 1 -- 172.21.15.86:0/1904571620 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f87240a1230 msgr2=0x7f87240a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:48.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.610+0000 7f8746e00700 1 --2- 172.21.15.86:0/1904571620 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f87240a1230 0x7f87240a36f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8730005950 tx=0x7f873000b410 comp rx=0 tx=0).stop 2024-03-23T17:49:48.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.610+0000 7f8746e00700 1 -- 172.21.15.86:0/1904571620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8740074df0 msgr2=0x7f87401994a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:48.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.610+0000 7f8746e00700 1 --2- 172.21.15.86:0/1904571620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8740074df0 0x7f87401994a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f872c0098e0 tx=0x7f872c00f670 comp rx=0 tx=0).stop 2024-03-23T17:49:48.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.610+0000 7f8746e00700 1 -- 172.21.15.86:0/1904571620 shutdown_connections 2024-03-23T17:49:48.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.610+0000 7f8746e00700 1 --2- 172.21.15.86:0/1904571620 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8740108c40 0x7f87401a04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:48.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.610+0000 7f8746e00700 1 --2- 172.21.15.86:0/1904571620 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8740073810 0x7f8740198f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:48.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.610+0000 7f8746e00700 1 --2- 172.21.15.86:0/1904571620 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f87240a1230 0x7f87240a36f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:48.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.610+0000 7f8746e00700 1 --2- 172.21.15.86:0/1904571620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8740074df0 0x7f87401994a0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:48.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.610+0000 7f8746e00700 1 -- 172.21.15.86:0/1904571620 >> 172.21.15.86:0/1904571620 conn(0x7f87400fc2f0 msgr2=0x7f8740104b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:48.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.610+0000 7f8746e00700 1 -- 172.21.15.86:0/1904571620 shutdown_connections 2024-03-23T17:49:48.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.610+0000 7f8746e00700 1 -- 172.21.15.86:0/1904571620 wait complete. 2024-03-23T17:49:48.630 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T17:49:48.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.825+0000 7f5c5f600700 1 -- 172.21.15.86:0/2886543027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c58100d80 msgr2=0x7f5c581011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:48.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.825+0000 7f5c5f600700 1 --2- 172.21.15.86:0/2886543027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c58100d80 0x7f5c581011a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c0099c0 tx=0x7f5c4c009cd0 comp rx=0 tx=0).stop 2024-03-23T17:49:48.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.825+0000 7f5c5f600700 1 -- 172.21.15.86:0/2886543027 shutdown_connections 2024-03-23T17:49:48.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.825+0000 7f5c5f600700 1 --2- 172.21.15.86:0/2886543027 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5c58102910 0x7f5c581091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:48.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.825+0000 7f5c5f600700 1 --2- 172.21.15.86:0/2886543027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c58100d80 0x7f5c581011a0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:48.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.825+0000 7f5c5f600700 1 --2- 172.21.15.86:0/2886543027 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c58101f70 0x7f5c581023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:48.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.825+0000 7f5c5f600700 1 -- 172.21.15.86:0/2886543027 >> 172.21.15.86:0/2886543027 conn(0x7f5c580fc2f0 msgr2=0x7f5c580fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:48.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.826+0000 7f5c5f600700 1 -- 172.21.15.86:0/2886543027 shutdown_connections 2024-03-23T17:49:48.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.826+0000 7f5c5f600700 1 -- 172.21.15.86:0/2886543027 wait complete. 2024-03-23T17:49:48.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.827+0000 7f5c5f600700 1 Processor -- start 2024-03-23T17:49:48.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.827+0000 7f5c5f600700 1 -- start start 2024-03-23T17:49:48.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.827+0000 7f5c5f600700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c58100d80 0x7f5c58196d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:48.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.827+0000 7f5c5f600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c58101f70 0x7f5c581972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:48.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.827+0000 7f5c5f600700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5c58102910 0x7f5c5819e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:48.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.827+0000 7f5c5f600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c5819e870 con 0x7f5c58101f70 2024-03-23T17:49:48.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.827+0000 7f5c5f600700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c5819e9e0 con 0x7f5c58100d80 2024-03-23T17:49:48.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.827+0000 7f5c5f600700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c5819eba0 con 0x7f5c58102910 2024-03-23T17:49:48.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.827+0000 7f5c57e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c58101f70 0x7f5c581972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:48.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.827+0000 7f5c57e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c58101f70 0x7f5c581972c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46964/0 (socket says 172.21.15.86:46964) 2024-03-23T17:49:48.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.827+0000 7f5c57e00700 1 -- 172.21.15.86:0/1458883429 learned_addr learned my addr 172.21.15.86:0/1458883429 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:48.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.827+0000 7f5c5d600700 1 --2- 172.21.15.86:0/1458883429 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5c58102910 0x7f5c5819e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:48.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.827+0000 7f5c5cc00700 1 --2- 172.21.15.86:0/1458883429 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c58100d80 0x7f5c58196d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:48.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.828+0000 7f5c57e00700 1 -- 172.21.15.86:0/1458883429 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5c58102910 msgr2=0x7f5c5819e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:48.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.828+0000 7f5c57e00700 1 --2- 172.21.15.86:0/1458883429 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5c58102910 0x7f5c5819e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:48.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.828+0000 7f5c57e00700 1 -- 172.21.15.86:0/1458883429 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c58100d80 msgr2=0x7f5c58196d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:48.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.828+0000 7f5c57e00700 1 --2- 172.21.15.86:0/1458883429 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c58100d80 0x7f5c58196d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:48.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.828+0000 7f5c57e00700 1 -- 172.21.15.86:0/1458883429 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c4c0095f0 con 0x7f5c58101f70 2024-03-23T17:49:48.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.828+0000 7f5c5d600700 1 --2- 172.21.15.86:0/1458883429 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5c58102910 0x7f5c5819e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:49:48.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.828+0000 7f5c57e00700 1 --2- 172.21.15.86:0/1458883429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c58101f70 0x7f5c581972c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5c4400e9a0 tx=0x7f5c4400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:48.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.828+0000 7f5c55600700 1 -- 172.21.15.86:0/1458883429 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c44009790 con 0x7f5c58101f70 2024-03-23T17:49:48.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.828+0000 7f5c55600700 1 -- 172.21.15.86:0/1458883429 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c4400cbd0 con 0x7f5c58101f70 2024-03-23T17:49:48.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.828+0000 7f5c5f600700 1 -- 172.21.15.86:0/1458883429 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c5819ee80 con 0x7f5c58101f70 2024-03-23T17:49:48.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.828+0000 7f5c55600700 1 -- 172.21.15.86:0/1458883429 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c44018680 con 0x7f5c58101f70 2024-03-23T17:49:48.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.828+0000 7f5c5f600700 1 -- 172.21.15.86:0/1458883429 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c5819f2a0 con 0x7f5c58101f70 2024-03-23T17:49:48.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.829+0000 7f5c5f600700 1 -- 172.21.15.86:0/1458883429 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c58108250 con 0x7f5c58101f70 2024-03-23T17:49:48.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.831+0000 7f5c55600700 1 -- 172.21.15.86:0/1458883429 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f5c4400cd40 con 0x7f5c58101f70 2024-03-23T17:49:48.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.832+0000 7f5c55600700 1 --2- 172.21.15.86:0/1458883429 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f5c400a11b0 0x7f5c400a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:48.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.832+0000 7f5c55600700 1 -- 172.21.15.86:0/1458883429 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f5c44014070 con 0x7f5c58101f70 2024-03-23T17:49:48.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.833+0000 7f5c5cc00700 1 --2- 172.21.15.86:0/1458883429 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f5c400a11b0 0x7f5c400a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:48.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.833+0000 7f5c5cc00700 1 --2- 172.21.15.86:0/1458883429 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f5c400a11b0 0x7f5c400a3670 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c003730 tx=0x7f5c4c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:48.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:48.836+0000 7f5c55600700 1 -- 172.21.15.86:0/1458883429 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f5c440529f0 con 0x7f5c58101f70 2024-03-23T17:49:49.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:48 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/444749903' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T17:49:49.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:48 smithi086 ceph-mon[38221]: pgmap v262: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-23T17:49:49.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:48 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/1904571620' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T17:49:49.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.042+0000 7f5c5f600700 1 -- 172.21.15.86:0/1458883429 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5c5810c6f0 con 0x7f5c58101f70 2024-03-23T17:49:49.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.042+0000 7f5c55600700 1 -- 172.21.15.86:0/1458883429 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v82) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5c440962f0 con 0x7f5c58101f70 2024-03-23T17:49:49.044 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-03-23T17:49:49.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.045+0000 7f5c5f600700 1 -- 172.21.15.86:0/1458883429 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f5c400a11b0 msgr2=0x7f5c400a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:49.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.045+0000 7f5c5f600700 1 --2- 172.21.15.86:0/1458883429 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f5c400a11b0 0x7f5c400a3670 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c003730 tx=0x7f5c4c000bc0 comp rx=0 tx=0).stop 2024-03-23T17:49:49.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.045+0000 7f5c5f600700 1 -- 172.21.15.86:0/1458883429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c58101f70 msgr2=0x7f5c581972c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:49.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.045+0000 7f5c5f600700 1 --2- 172.21.15.86:0/1458883429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c58101f70 0x7f5c581972c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5c4400e9a0 tx=0x7f5c4400ecb0 comp rx=0 tx=0).stop 2024-03-23T17:49:49.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.045+0000 7f5c5f600700 1 -- 172.21.15.86:0/1458883429 shutdown_connections 2024-03-23T17:49:49.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.045+0000 7f5c5f600700 1 --2- 172.21.15.86:0/1458883429 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5c58102910 0x7f5c5819e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:49.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.045+0000 7f5c5f600700 1 --2- 172.21.15.86:0/1458883429 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c58100d80 0x7f5c58196d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:49.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.045+0000 7f5c5f600700 1 --2- 172.21.15.86:0/1458883429 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f5c400a11b0 0x7f5c400a3670 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:49.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.045+0000 7f5c5f600700 1 --2- 172.21.15.86:0/1458883429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c58101f70 0x7f5c581972c0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:49.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.045+0000 7f5c5f600700 1 -- 172.21.15.86:0/1458883429 >> 172.21.15.86:0/1458883429 conn(0x7f5c580fc2f0 msgr2=0x7f5c58105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:49.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.045+0000 7f5c5f600700 1 -- 172.21.15.86:0/1458883429 shutdown_connections 2024-03-23T17:49:49.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.045+0000 7f5c5f600700 1 -- 172.21.15.86:0/1458883429 wait complete. 2024-03-23T17:49:49.065 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T17:49:49.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.259+0000 7f8d06000700 1 -- 172.21.15.86:0/1035292844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d00101f70 msgr2=0x7f8d001023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:49.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.259+0000 7f8d06000700 1 --2- 172.21.15.86:0/1035292844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d00101f70 0x7f8d001023d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8cec0099c0 tx=0x7f8cec009cd0 comp rx=0 tx=0).stop 2024-03-23T17:49:49.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.259+0000 7f8d06000700 1 -- 172.21.15.86:0/1035292844 shutdown_connections 2024-03-23T17:49:49.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.259+0000 7f8d06000700 1 --2- 172.21.15.86:0/1035292844 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8d00102910 0x7f8d001091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:49.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.259+0000 7f8d06000700 1 --2- 172.21.15.86:0/1035292844 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d00100d80 0x7f8d001011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:49.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.259+0000 7f8d06000700 1 --2- 172.21.15.86:0/1035292844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d00101f70 0x7f8d001023d0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:49.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.259+0000 7f8d06000700 1 -- 172.21.15.86:0/1035292844 >> 172.21.15.86:0/1035292844 conn(0x7f8d000fc2f0 msgr2=0x7f8d000fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:49.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.259+0000 7f8d06000700 1 -- 172.21.15.86:0/1035292844 shutdown_connections 2024-03-23T17:49:49.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.259+0000 7f8d06000700 1 -- 172.21.15.86:0/1035292844 wait complete. 2024-03-23T17:49:49.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.260+0000 7f8d06000700 1 Processor -- start 2024-03-23T17:49:49.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.260+0000 7f8d06000700 1 -- start start 2024-03-23T17:49:49.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.261+0000 7f8d06000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d00100d80 0x7f8d00074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:49.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.261+0000 7f8d06000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d00101f70 0x7f8d000731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:49.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.261+0000 7f8d06000700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8d00102910 0x7f8d000736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:49.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.261+0000 7f8d06000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d00073e80 con 0x7f8d00100d80 2024-03-23T17:49:49.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.261+0000 7f8d06000700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d00073ff0 con 0x7f8d00101f70 2024-03-23T17:49:49.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.261+0000 7f8d06000700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d001a0d80 con 0x7f8d00102910 2024-03-23T17:49:49.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.261+0000 7f8cff400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d00100d80 0x7f8d00074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:49.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.261+0000 7f8cff400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d00100d80 0x7f8d00074b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46980/0 (socket says 172.21.15.86:46980) 2024-03-23T17:49:49.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.261+0000 7f8cff400700 1 -- 172.21.15.86:0/1433649396 learned_addr learned my addr 172.21.15.86:0/1433649396 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:49.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.261+0000 7f8cffe00700 1 --2- 172.21.15.86:0/1433649396 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8d00102910 0x7f8d000736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:49.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.261+0000 7f8cfea00700 1 --2- 172.21.15.86:0/1433649396 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d00101f70 0x7f8d000731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:49.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.261+0000 7f8cff400700 1 -- 172.21.15.86:0/1433649396 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8d00102910 msgr2=0x7f8d000736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:49.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.261+0000 7f8cff400700 1 --2- 172.21.15.86:0/1433649396 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8d00102910 0x7f8d000736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:49.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.261+0000 7f8cff400700 1 -- 172.21.15.86:0/1433649396 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d00101f70 msgr2=0x7f8d000731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:49.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.261+0000 7f8cff400700 1 --2- 172.21.15.86:0/1433649396 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d00101f70 0x7f8d000731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:49.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.261+0000 7f8cff400700 1 -- 172.21.15.86:0/1433649396 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cec0095f0 con 0x7f8d00100d80 2024-03-23T17:49:49.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.261+0000 7f8cffe00700 1 --2- 172.21.15.86:0/1433649396 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8d00102910 0x7f8d000736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:49:49.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.262+0000 7f8cff400700 1 --2- 172.21.15.86:0/1433649396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d00100d80 0x7f8d00074b10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8cf000e970 tx=0x7f8cf000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:49.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.262+0000 7f8cf7e00700 1 -- 172.21.15.86:0/1433649396 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf0009790 con 0x7f8d00100d80 2024-03-23T17:49:49.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.262+0000 7f8cf7e00700 1 -- 172.21.15.86:0/1433649396 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8cf000eec0 con 0x7f8d00100d80 2024-03-23T17:49:49.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.262+0000 7f8d06000700 1 -- 172.21.15.86:0/1433649396 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d001a0fb0 con 0x7f8d00100d80 2024-03-23T17:49:49.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.262+0000 7f8cf7e00700 1 -- 172.21.15.86:0/1433649396 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf0003c90 con 0x7f8d00100d80 2024-03-23T17:49:49.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.262+0000 7f8d06000700 1 -- 172.21.15.86:0/1433649396 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d001a1430 con 0x7f8d00100d80 2024-03-23T17:49:49.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.263+0000 7f8d06000700 1 -- 172.21.15.86:0/1433649396 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d00108200 con 0x7f8d00100d80 2024-03-23T17:49:49.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.265+0000 7f8cf7e00700 1 -- 172.21.15.86:0/1433649396 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f8cf000cc70 con 0x7f8d00100d80 2024-03-23T17:49:49.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.266+0000 7f8cf7e00700 1 --2- 172.21.15.86:0/1433649396 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f8ce40a11b0 0x7f8ce40a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:49.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.266+0000 7f8cf7e00700 1 -- 172.21.15.86:0/1433649396 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f8cf0014070 con 0x7f8d00100d80 2024-03-23T17:49:49.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.267+0000 7f8cfea00700 1 --2- 172.21.15.86:0/1433649396 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f8ce40a11b0 0x7f8ce40a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:49.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.267+0000 7f8cfea00700 1 --2- 172.21.15.86:0/1433649396 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f8ce40a11b0 0x7f8ce40a3670 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8cec000c00 tx=0x7f8cec019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:49.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.270+0000 7f8cf7e00700 1 -- 172.21.15.86:0/1433649396 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f8cf00962e0 con 0x7f8d00100d80 2024-03-23T17:49:49.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:48 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/444749903' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T17:49:49.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:48 smithi092 ceph-mon[79089]: pgmap v262: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-23T17:49:49.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:48 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/1904571620' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T17:49:49.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:48 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/444749903' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T17:49:49.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:48 smithi152 ceph-mon[47949]: pgmap v262: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-23T17:49:49.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:48 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/1904571620' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T17:49:49.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.478+0000 7f8d06000700 1 -- 172.21.15.86:0/1433649396 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8d0010c5e0 con 0x7f8d00100d80 2024-03-23T17:49:49.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.478+0000 7f8cf7e00700 1 -- 172.21.15.86:0/1433649396 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v82) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8cf0099900 con 0x7f8d00100d80 2024-03-23T17:49:49.479 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-03-23T17:49:49.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.480+0000 7f8d06000700 1 -- 172.21.15.86:0/1433649396 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f8ce40a11b0 msgr2=0x7f8ce40a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:49.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.481+0000 7f8d06000700 1 --2- 172.21.15.86:0/1433649396 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f8ce40a11b0 0x7f8ce40a3670 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8cec000c00 tx=0x7f8cec019040 comp rx=0 tx=0).stop 2024-03-23T17:49:49.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.481+0000 7f8d06000700 1 -- 172.21.15.86:0/1433649396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d00100d80 msgr2=0x7f8d00074b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:49.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.481+0000 7f8d06000700 1 --2- 172.21.15.86:0/1433649396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d00100d80 0x7f8d00074b10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8cf000e970 tx=0x7f8cf000ec80 comp rx=0 tx=0).stop 2024-03-23T17:49:49.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.481+0000 7f8d06000700 1 -- 172.21.15.86:0/1433649396 shutdown_connections 2024-03-23T17:49:49.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.481+0000 7f8d06000700 1 --2- 172.21.15.86:0/1433649396 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8d00102910 0x7f8d000736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:49.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.481+0000 7f8d06000700 1 --2- 172.21.15.86:0/1433649396 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f8ce40a11b0 0x7f8ce40a3670 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:49.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.481+0000 7f8d06000700 1 --2- 172.21.15.86:0/1433649396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d00100d80 0x7f8d00074b10 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:49.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.481+0000 7f8d06000700 1 --2- 172.21.15.86:0/1433649396 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d00101f70 0x7f8d000731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:49.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.481+0000 7f8d06000700 1 -- 172.21.15.86:0/1433649396 >> 172.21.15.86:0/1433649396 conn(0x7f8d000fc2f0 msgr2=0x7f8d00105340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:49.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.481+0000 7f8d06000700 1 -- 172.21.15.86:0/1433649396 shutdown_connections 2024-03-23T17:49:49.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.481+0000 7f8d06000700 1 -- 172.21.15.86:0/1433649396 wait complete. 2024-03-23T17:49:49.500 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T17:49:49.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.691+0000 7f9e7e200700 1 -- 172.21.15.86:0/3893708969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e78100c80 msgr2=0x7f9e781010a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:49.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.691+0000 7f9e7e200700 1 --2- 172.21.15.86:0/3893708969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e78100c80 0x7f9e781010a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9e680099c0 tx=0x7f9e68009cd0 comp rx=0 tx=0).stop 2024-03-23T17:49:49.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.691+0000 7f9e7e200700 1 -- 172.21.15.86:0/3893708969 shutdown_connections 2024-03-23T17:49:49.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.691+0000 7f9e7e200700 1 --2- 172.21.15.86:0/3893708969 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e78102810 0x7f9e781090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:49.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.691+0000 7f9e7e200700 1 --2- 172.21.15.86:0/3893708969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e78100c80 0x7f9e781010a0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:49.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.691+0000 7f9e7e200700 1 --2- 172.21.15.86:0/3893708969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e78101e70 0x7f9e781022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:49.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.691+0000 7f9e7e200700 1 -- 172.21.15.86:0/3893708969 >> 172.21.15.86:0/3893708969 conn(0x7f9e780fc210 msgr2=0x7f9e780fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:49.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.691+0000 7f9e7e200700 1 -- 172.21.15.86:0/3893708969 shutdown_connections 2024-03-23T17:49:49.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.691+0000 7f9e7e200700 1 -- 172.21.15.86:0/3893708969 wait complete. 2024-03-23T17:49:49.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.692+0000 7f9e7e200700 1 Processor -- start 2024-03-23T17:49:49.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.692+0000 7f9e7e200700 1 -- start start 2024-03-23T17:49:49.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.693+0000 7f9e7e200700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e78100c80 0x7f9e78198e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:49.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.693+0000 7f9e7e200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e78101e70 0x7f9e78199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:49.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.693+0000 7f9e7e200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e78102810 0x7f9e781a03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:49.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.693+0000 7f9e7e200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e781a0980 con 0x7f9e78101e70 2024-03-23T17:49:49.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.693+0000 7f9e7e200700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e781a0ac0 con 0x7f9e78102810 2024-03-23T17:49:49.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.693+0000 7f9e7e200700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e781a0c50 con 0x7f9e78100c80 2024-03-23T17:49:49.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.693+0000 7f9e76a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e78101e70 0x7f9e78199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:49.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.693+0000 7f9e76a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e78101e70 0x7f9e78199390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46984/0 (socket says 172.21.15.86:46984) 2024-03-23T17:49:49.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.693+0000 7f9e76a00700 1 -- 172.21.15.86:0/3791808422 learned_addr learned my addr 172.21.15.86:0/3791808422 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:49.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.693+0000 7f9e77e00700 1 --2- 172.21.15.86:0/3791808422 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e78102810 0x7f9e781a03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:49.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.693+0000 7f9e77400700 1 --2- 172.21.15.86:0/3791808422 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e78100c80 0x7f9e78198e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:49.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.693+0000 7f9e76a00700 1 -- 172.21.15.86:0/3791808422 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e78100c80 msgr2=0x7f9e78198e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:49.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.693+0000 7f9e76a00700 1 --2- 172.21.15.86:0/3791808422 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e78100c80 0x7f9e78198e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:49.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.693+0000 7f9e76a00700 1 -- 172.21.15.86:0/3791808422 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e78102810 msgr2=0x7f9e781a03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:49.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.693+0000 7f9e76a00700 1 --2- 172.21.15.86:0/3791808422 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e78102810 0x7f9e781a03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:49.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.693+0000 7f9e76a00700 1 -- 172.21.15.86:0/3791808422 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e680095f0 con 0x7f9e78101e70 2024-03-23T17:49:49.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.693+0000 7f9e76a00700 1 --2- 172.21.15.86:0/3791808422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e78101e70 0x7f9e78199390 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9e6400d980 tx=0x7f9e6400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:49.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.694+0000 7f9e6fe00700 1 -- 172.21.15.86:0/3791808422 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e64004220 con 0x7f9e78101e70 2024-03-23T17:49:49.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.694+0000 7f9e6fe00700 1 -- 172.21.15.86:0/3791808422 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e64007640 con 0x7f9e78101e70 2024-03-23T17:49:49.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.694+0000 7f9e7e200700 1 -- 172.21.15.86:0/3791808422 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e781a0f30 con 0x7f9e78101e70 2024-03-23T17:49:49.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.694+0000 7f9e6fe00700 1 -- 172.21.15.86:0/3791808422 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e64005610 con 0x7f9e78101e70 2024-03-23T17:49:49.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.694+0000 7f9e7e200700 1 -- 172.21.15.86:0/3791808422 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e781a1400 con 0x7f9e78101e70 2024-03-23T17:49:49.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.695+0000 7f9e7e200700 1 -- 172.21.15.86:0/3791808422 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e78067180 con 0x7f9e78101e70 2024-03-23T17:49:49.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.696+0000 7f9e6fe00700 1 -- 172.21.15.86:0/3791808422 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f9e64004750 con 0x7f9e78101e70 2024-03-23T17:49:49.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.698+0000 7f9e6fe00700 1 --2- 172.21.15.86:0/3791808422 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f9e5c0a1230 0x7f9e5c0a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:49.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.698+0000 7f9e6fe00700 1 -- 172.21.15.86:0/3791808422 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f9e640cc8b0 con 0x7f9e78101e70 2024-03-23T17:49:49.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.698+0000 7f9e77400700 1 --2- 172.21.15.86:0/3791808422 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f9e5c0a1230 0x7f9e5c0a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:49.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.698+0000 7f9e77400700 1 --2- 172.21.15.86:0/3791808422 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f9e5c0a1230 0x7f9e5c0a36f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9e68009990 tx=0x7f9e6801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:49.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.702+0000 7f9e6fe00700 1 -- 172.21.15.86:0/3791808422 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f9e64095eb0 con 0x7f9e78101e70 2024-03-23T17:49:49.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.959+0000 7f9e7e200700 1 -- 172.21.15.86:0/3791808422 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9e7810c4e0 con 0x7f9e78101e70 2024-03-23T17:49:49.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.960+0000 7f9e6fe00700 1 -- 172.21.15.86:0/3791808422 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10913 (secure 0 0 0) 0x7f9e64016020 con 0x7f9e78101e70 2024-03-23T17:49:49.961 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:49:49.961 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14445,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1433140219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1433140219},{"type":"v1","addr":"172.21.15.86:6835","nonce":1433140219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6839/1879457730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1879457730},{"type":"v1","addr":"172.21.15.86:6839","nonce":1879457730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-23T17:48:56.202323+0000","modified":"2024-03-23T17:49:03.703276+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24553,"mds_2":24518,"mds_3":24505,"mds_4":24500},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14514":{"gid":14514,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/3026943866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3026943866},{"type":"v1","addr":"172.21.15.86:6841","nonce":3026943866}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24464":{"gid":24464,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6833/1467091186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1467091186},{"type":"v1","addr":"172.21.15.92:6833","nonce":1467091186}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6833/3075554623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3075554623},{"type":"v1","addr":"172.21.15.152:6833","nonce":3075554623}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.92:6835/908900027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":908900027},{"type":"v1","addr":"172.21.15.92:6835","nonce":908900027}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"d","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.86:6837/2933881328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2933881328},{"type":"v1","addr":"172.21.15.86:6837","nonce":2933881328}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6837/3967837948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":3967837948},{"type":"v1","addr":"172.21.15.92:6837","nonce":3967837948}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6835/3137085021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3137085021},{"type":"v1","addr":"172.21.15.152:6835","nonce":3137085021}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24538":{"gid":24538,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6837/1151208965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":1151208965},{"type":"v1","addr":"172.21.15.152:6837","nonce":1151208965}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.152:6839/1100723992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":1100723992},{"type":"v1","addr":"172.21.15.152:6839","nonce":1100723992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6839/1394139887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1394139887},{"type":"v1","addr":"172.21.15.92:6839","nonce":1394139887}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-23T17:49:49.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.962+0000 7f9e7e200700 1 -- 172.21.15.86:0/3791808422 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f9e5c0a1230 msgr2=0x7f9e5c0a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:49.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.962+0000 7f9e7e200700 1 --2- 172.21.15.86:0/3791808422 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f9e5c0a1230 0x7f9e5c0a36f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9e68009990 tx=0x7f9e6801a040 comp rx=0 tx=0).stop 2024-03-23T17:49:49.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.962+0000 7f9e7e200700 1 -- 172.21.15.86:0/3791808422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e78101e70 msgr2=0x7f9e78199390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:49.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.962+0000 7f9e7e200700 1 --2- 172.21.15.86:0/3791808422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e78101e70 0x7f9e78199390 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9e6400d980 tx=0x7f9e6400dc90 comp rx=0 tx=0).stop 2024-03-23T17:49:49.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.963+0000 7f9e7e200700 1 -- 172.21.15.86:0/3791808422 shutdown_connections 2024-03-23T17:49:49.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.963+0000 7f9e7e200700 1 --2- 172.21.15.86:0/3791808422 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e78102810 0x7f9e781a03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:49.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.963+0000 7f9e7e200700 1 --2- 172.21.15.86:0/3791808422 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e78100c80 0x7f9e78198e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:49.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.963+0000 7f9e7e200700 1 --2- 172.21.15.86:0/3791808422 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f9e5c0a1230 0x7f9e5c0a36f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:49.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.963+0000 7f9e7e200700 1 --2- 172.21.15.86:0/3791808422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e78101e70 0x7f9e78199390 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:49.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.963+0000 7f9e7e200700 1 -- 172.21.15.86:0/3791808422 >> 172.21.15.86:0/3791808422 conn(0x7f9e780fc210 msgr2=0x7f9e78072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:49.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.963+0000 7f9e7e200700 1 -- 172.21.15.86:0/3791808422 shutdown_connections 2024-03-23T17:49:49.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:49.963+0000 7f9e7e200700 1 -- 172.21.15.86:0/3791808422 wait complete. 2024-03-23T17:49:49.965 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 27 2024-03-23T17:49:49.983 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T17:49:50.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.173+0000 7f0fd2e00700 1 -- 172.21.15.86:0/534874242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fcc101e70 msgr2=0x7f0fcc1022d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:50.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.173+0000 7f0fd2e00700 1 --2- 172.21.15.86:0/534874242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fcc101e70 0x7f0fcc1022d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0fb80099c0 tx=0x7f0fb8009cd0 comp rx=0 tx=0).stop 2024-03-23T17:49:50.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.174+0000 7f0fd2e00700 1 -- 172.21.15.86:0/534874242 shutdown_connections 2024-03-23T17:49:50.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.174+0000 7f0fd2e00700 1 --2- 172.21.15.86:0/534874242 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0fcc102810 0x7f0fcc1090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:50.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.174+0000 7f0fd2e00700 1 --2- 172.21.15.86:0/534874242 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0fcc100c80 0x7f0fcc1010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:50.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.174+0000 7f0fd2e00700 1 --2- 172.21.15.86:0/534874242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fcc101e70 0x7f0fcc1022d0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:50.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.174+0000 7f0fd2e00700 1 -- 172.21.15.86:0/534874242 >> 172.21.15.86:0/534874242 conn(0x7f0fcc0fc1f0 msgr2=0x7f0fcc0fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:50.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.174+0000 7f0fd2e00700 1 -- 172.21.15.86:0/534874242 shutdown_connections 2024-03-23T17:49:50.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.174+0000 7f0fd2e00700 1 -- 172.21.15.86:0/534874242 wait complete. 2024-03-23T17:49:50.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.175+0000 7f0fd2e00700 1 Processor -- start 2024-03-23T17:49:50.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.175+0000 7f0fd2e00700 1 -- start start 2024-03-23T17:49:50.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.175+0000 7f0fd2e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fcc100c80 0x7f0fcc198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:50.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.175+0000 7f0fd2e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0fcc101e70 0x7f0fcc199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:50.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.175+0000 7f0fd2e00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0fcc102810 0x7f0fcc1a0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:50.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.175+0000 7f0fd2e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fcc1a0950 con 0x7f0fcc100c80 2024-03-23T17:49:50.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.175+0000 7f0fd2e00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fcc1a0a90 con 0x7f0fcc101e70 2024-03-23T17:49:50.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.175+0000 7f0fd2e00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fcc1a0c20 con 0x7f0fcc102810 2024-03-23T17:49:50.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.175+0000 7f0fcbe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fcc100c80 0x7f0fcc198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:50.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.175+0000 7f0fcbe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fcc100c80 0x7f0fcc198e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47000/0 (socket says 172.21.15.86:47000) 2024-03-23T17:49:50.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.175+0000 7f0fcbe00700 1 -- 172.21.15.86:0/1590213400 learned_addr learned my addr 172.21.15.86:0/1590213400 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:50.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.175+0000 7f0fcb400700 1 --2- 172.21.15.86:0/1590213400 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0fcc101e70 0x7f0fcc199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:50.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.176+0000 7f0fd0e00700 1 --2- 172.21.15.86:0/1590213400 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0fcc102810 0x7f0fcc1a0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:50.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.176+0000 7f0fcbe00700 1 -- 172.21.15.86:0/1590213400 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0fcc102810 msgr2=0x7f0fcc1a0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:50.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.176+0000 7f0fcbe00700 1 --2- 172.21.15.86:0/1590213400 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0fcc102810 0x7f0fcc1a0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:50.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.176+0000 7f0fcbe00700 1 -- 172.21.15.86:0/1590213400 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0fcc101e70 msgr2=0x7f0fcc199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:50.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.176+0000 7f0fcbe00700 1 --2- 172.21.15.86:0/1590213400 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0fcc101e70 0x7f0fcc199360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:50.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.176+0000 7f0fcbe00700 1 -- 172.21.15.86:0/1590213400 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fb80095f0 con 0x7f0fcc100c80 2024-03-23T17:49:50.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.176+0000 7f0fcbe00700 1 --2- 172.21.15.86:0/1590213400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fcc100c80 0x7f0fcc198e20 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc00d980 tx=0x7f0fbc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:50.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.176+0000 7f0fc8c00700 1 -- 172.21.15.86:0/1590213400 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc004020 con 0x7f0fcc100c80 2024-03-23T17:49:50.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.176+0000 7f0fc8c00700 1 -- 172.21.15.86:0/1590213400 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0fbc00f690 con 0x7f0fcc100c80 2024-03-23T17:49:50.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.176+0000 7f0fc8c00700 1 -- 172.21.15.86:0/1590213400 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc003a60 con 0x7f0fcc100c80 2024-03-23T17:49:50.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.176+0000 7f0fd2e00700 1 -- 172.21.15.86:0/1590213400 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fcc1a0f00 con 0x7f0fcc100c80 2024-03-23T17:49:50.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.176+0000 7f0fd2e00700 1 -- 172.21.15.86:0/1590213400 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fcc1a1320 con 0x7f0fcc100c80 2024-03-23T17:49:50.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.177+0000 7f0fd2e00700 1 -- 172.21.15.86:0/1590213400 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fcc067a80 con 0x7f0fcc100c80 2024-03-23T17:49:50.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.179+0000 7f0fc8c00700 1 -- 172.21.15.86:0/1590213400 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f0fbc00fb00 con 0x7f0fcc100c80 2024-03-23T17:49:50.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.180+0000 7f0fc8c00700 1 --2- 172.21.15.86:0/1590213400 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f0fb00a1230 0x7f0fb00a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:50.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.180+0000 7f0fc8c00700 1 -- 172.21.15.86:0/1590213400 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f0fbc0cc500 con 0x7f0fcc100c80 2024-03-23T17:49:50.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.181+0000 7f0fcb400700 1 --2- 172.21.15.86:0/1590213400 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f0fb00a1230 0x7f0fb00a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:50.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.181+0000 7f0fcb400700 1 --2- 172.21.15.86:0/1590213400 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f0fb00a1230 0x7f0fb00a36f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0fb8006010 tx=0x7f0fb800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:50.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.184+0000 7f0fc8c00700 1 -- 172.21.15.86:0/1590213400 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f0fcc067a80 con 0x7f0fcc100c80 2024-03-23T17:49:50.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:49 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/1458883429' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T17:49:50.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:49 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/1433649396' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T17:49:50.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:49 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/1458883429' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T17:49:50.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:49 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/1433649396' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T17:49:50.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:49 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/1458883429' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T17:49:50.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:49 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/1433649396' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T17:49:50.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.426+0000 7f0fd2e00700 1 -- 172.21.15.86:0/1590213400 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0fcc10c790 con 0x7f0fcc100c80 2024-03-23T17:49:50.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.427+0000 7f0fc8c00700 1 -- 172.21.15.86:0/1590213400 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22414 (secure 0 0 0) 0x7f0fbc095b60 con 0x7f0fcc100c80 2024-03-23T17:49:50.429 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:49:50.429 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":82,"fsid":"9d76c9e8-e93c-11ee-95cd-87774f69a715","created":"2024-03-23T17:42:30.097233+0000","modified":"2024-03-23T17:49:29.773954+0000","last_up_change":"2024-03-23T17:48:18.217601+0000","last_in_change":"2024-03-23T17:48:00.250579+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T17:45:26.848397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T17:48:54.331894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T17:48:55.640693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"656655c6-1c7f-4640-92c4-2f42de820143","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6803","nonce":2423505491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6805","nonce":2423505491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6809","nonce":2423505491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6807","nonce":2423505491}]},"public_addr":"172.21.15.86:6803/2423505491","cluster_addr":"172.21.15.86:6805/2423505491","heartbeat_back_addr":"172.21.15.86:6809/2423505491","heartbeat_front_addr":"172.21.15.86:6807/2423505491","state":["exists","up"]},{"osd":1,"uuid":"3e86dc65-20a6-4de8-b9ce-fcf01f68862b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6801","nonce":3776252597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6803","nonce":3776252597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6807","nonce":3776252597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6805","nonce":3776252597}]},"public_addr":"172.21.15.92:6801/3776252597","cluster_addr":"172.21.15.92:6803/3776252597","heartbeat_back_addr":"172.21.15.92:6807/3776252597","heartbeat_front_addr":"172.21.15.92:6805/3776252597","state":["exists","up"]},{"osd":2,"uuid":"09808786-3bd3-4d11-8c7b-1736119dfbb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6801","nonce":1601265891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6803","nonce":1601265891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6807","nonce":1601265891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6805","nonce":1601265891}]},"public_addr":"172.21.15.152:6801/1601265891","cluster_addr":"172.21.15.152:6803/1601265891","heartbeat_back_addr":"172.21.15.152:6807/1601265891","heartbeat_front_addr":"172.21.15.152:6805/1601265891","state":["exists","up"]},{"osd":3,"uuid":"79c61910-4b53-424c-b150-6f5385a2cd68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6811","nonce":694846466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6813","nonce":694846466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6817","nonce":694846466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6815","nonce":694846466}]},"public_addr":"172.21.15.86:6811/694846466","cluster_addr":"172.21.15.86:6813/694846466","heartbeat_back_addr":"172.21.15.86:6817/694846466","heartbeat_front_addr":"172.21.15.86:6815/694846466","state":["exists","up"]},{"osd":4,"uuid":"a2878035-0179-469f-bbd4-8ce2d3f6bddc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6809","nonce":1422821471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6811","nonce":1422821471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6815","nonce":1422821471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6813","nonce":1422821471}]},"public_addr":"172.21.15.92:6809/1422821471","cluster_addr":"172.21.15.92:6811/1422821471","heartbeat_back_addr":"172.21.15.92:6815/1422821471","heartbeat_front_addr":"172.21.15.92:6813/1422821471","state":["exists","up"]},{"osd":5,"uuid":"837366aa-ce25-42ca-b157-5bbb50159d0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6809","nonce":1039375239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6811","nonce":1039375239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6815","nonce":1039375239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6813","nonce":1039375239}]},"public_addr":"172.21.15.152:6809/1039375239","cluster_addr":"172.21.15.152:6811/1039375239","heartbeat_back_addr":"172.21.15.152:6815/1039375239","heartbeat_front_addr":"172.21.15.152:6813/1039375239","state":["exists","up"]},{"osd":6,"uuid":"61024437-7b91-432f-a1af-6443d0ef5213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6819","nonce":4080452707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6821","nonce":4080452707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6825","nonce":4080452707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6823","nonce":4080452707}]},"public_addr":"172.21.15.86:6819/4080452707","cluster_addr":"172.21.15.86:6821/4080452707","heartbeat_back_addr":"172.21.15.86:6825/4080452707","heartbeat_front_addr":"172.21.15.86:6823/4080452707","state":["exists","up"]},{"osd":7,"uuid":"2684bdae-6b30-4afc-af68-4ce41b3a75b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6817","nonce":4187707622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6819","nonce":4187707622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6823","nonce":4187707622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6821","nonce":4187707622}]},"public_addr":"172.21.15.92:6817/4187707622","cluster_addr":"172.21.15.92:6819/4187707622","heartbeat_back_addr":"172.21.15.92:6823/4187707622","heartbeat_front_addr":"172.21.15.92:6821/4187707622","state":["exists","up"]},{"osd":8,"uuid":"f03bbf0d-1281-4bcf-9773-9854388e65ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6817","nonce":3237612748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6819","nonce":3237612748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6823","nonce":3237612748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6821","nonce":3237612748}]},"public_addr":"172.21.15.152:6817/3237612748","cluster_addr":"172.21.15.152:6819/3237612748","heartbeat_back_addr":"172.21.15.152:6823/3237612748","heartbeat_front_addr":"172.21.15.152:6821/3237612748","state":["exists","up"]},{"osd":9,"uuid":"e7e8b775-99ff-48a4-a23a-91e7ab6622bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6827","nonce":2835218294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6829","nonce":2835218294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6833","nonce":2835218294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6831","nonce":2835218294}]},"public_addr":"172.21.15.86:6827/2835218294","cluster_addr":"172.21.15.86:6829/2835218294","heartbeat_back_addr":"172.21.15.86:6833/2835218294","heartbeat_front_addr":"172.21.15.86:6831/2835218294","state":["exists","up"]},{"osd":10,"uuid":"d9d616cd-a229-44de-8768-02a4415df115","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6825","nonce":4259465100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6827","nonce":4259465100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6831","nonce":4259465100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6829","nonce":4259465100}]},"public_addr":"172.21.15.92:6825/4259465100","cluster_addr":"172.21.15.92:6827/4259465100","heartbeat_back_addr":"172.21.15.92:6831/4259465100","heartbeat_front_addr":"172.21.15.92:6829/4259465100","state":["exists","up"]},{"osd":11,"uuid":"e7c0b865-ccf5-4233-8d22-0d9b6dbe5c97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6825","nonce":194662760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6827","nonce":194662760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6831","nonce":194662760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6829","nonce":194662760}]},"public_addr":"172.21.15.152:6825/194662760","cluster_addr":"172.21.15.152:6827/194662760","heartbeat_back_addr":"172.21.15.152:6831/194662760","heartbeat_front_addr":"172.21.15.152:6829/194662760","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:44:47.269167+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:45:05.363655+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:45:23.192677+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:45:44.156228+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:02.532971+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:20.153295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:41.713888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:59.676516+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:47:17.207087+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:47:38.970316+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:47:57.282525+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:48:15.437606+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/3553500912":"2024-03-24T17:43:28.242492+0000","172.21.15.86:6801/3600278228":"2024-03-24T17:43:28.242492+0000","172.21.15.86:0/887262329":"2024-03-24T17:43:28.242492+0000","172.21.15.86:6800/2313723492":"2024-03-24T17:43:05.043910+0000","172.21.15.86:6800/3600278228":"2024-03-24T17:43:28.242492+0000","172.21.15.86:0/3815705642":"2024-03-24T17:43:05.043910+0000","172.21.15.86:0/1614943062":"2024-03-24T17:43:28.242492+0000","172.21.15.86:6801/2313723492":"2024-03-24T17:43:05.043910+0000","172.21.15.86:0/2416321716":"2024-03-24T17:43:05.043910+0000","172.21.15.86:0/433588188":"2024-03-24T17:43:05.043910+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T17:49:50.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.430+0000 7f0fd2e00700 1 -- 172.21.15.86:0/1590213400 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f0fb00a1230 msgr2=0x7f0fb00a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:50.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.430+0000 7f0fd2e00700 1 --2- 172.21.15.86:0/1590213400 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f0fb00a1230 0x7f0fb00a36f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0fb8006010 tx=0x7f0fb800b540 comp rx=0 tx=0).stop 2024-03-23T17:49:50.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.430+0000 7f0fd2e00700 1 -- 172.21.15.86:0/1590213400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fcc100c80 msgr2=0x7f0fcc198e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:50.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.430+0000 7f0fd2e00700 1 --2- 172.21.15.86:0/1590213400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fcc100c80 0x7f0fcc198e20 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc00d980 tx=0x7f0fbc00dc90 comp rx=0 tx=0).stop 2024-03-23T17:49:50.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.430+0000 7f0fd2e00700 1 -- 172.21.15.86:0/1590213400 shutdown_connections 2024-03-23T17:49:50.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.430+0000 7f0fd2e00700 1 --2- 172.21.15.86:0/1590213400 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0fcc102810 0x7f0fcc1a0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:50.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.430+0000 7f0fd2e00700 1 --2- 172.21.15.86:0/1590213400 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f0fb00a1230 0x7f0fb00a36f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:50.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.430+0000 7f0fd2e00700 1 --2- 172.21.15.86:0/1590213400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fcc100c80 0x7f0fcc198e20 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:50.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.430+0000 7f0fd2e00700 1 --2- 172.21.15.86:0/1590213400 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0fcc101e70 0x7f0fcc199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:50.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.430+0000 7f0fd2e00700 1 -- 172.21.15.86:0/1590213400 >> 172.21.15.86:0/1590213400 conn(0x7f0fcc0fc1f0 msgr2=0x7f0fcc072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:50.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.430+0000 7f0fd2e00700 1 -- 172.21.15.86:0/1590213400 shutdown_connections 2024-03-23T17:49:50.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.430+0000 7f0fd2e00700 1 -- 172.21.15.86:0/1590213400 wait complete. 2024-03-23T17:49:50.451 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T17:49:50.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.642+0000 7f46b5600700 1 -- 172.21.15.86:0/1700746679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46b0104040 msgr2=0x7f46b0106430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:50.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.642+0000 7f46b5600700 1 --2- 172.21.15.86:0/1700746679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46b0104040 0x7f46b0106430 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f469c009990 tx=0x7f469c009ca0 comp rx=0 tx=0).stop 2024-03-23T17:49:50.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.642+0000 7f46b5600700 1 -- 172.21.15.86:0/1700746679 shutdown_connections 2024-03-23T17:49:50.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.642+0000 7f46b5600700 1 --2- 172.21.15.86:0/1700746679 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46b0106970 0x7f46b0108de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:50.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.642+0000 7f46b5600700 1 --2- 172.21.15.86:0/1700746679 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46b0101710 0x7f46b0103b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:50.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.642+0000 7f46b5600700 1 --2- 172.21.15.86:0/1700746679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46b0104040 0x7f46b0106430 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:50.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.642+0000 7f46b5600700 1 -- 172.21.15.86:0/1700746679 >> 172.21.15.86:0/1700746679 conn(0x7f46b00fb100 msgr2=0x7f46b00fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:50.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.642+0000 7f46b5600700 1 -- 172.21.15.86:0/1700746679 shutdown_connections 2024-03-23T17:49:50.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.642+0000 7f46b5600700 1 -- 172.21.15.86:0/1700746679 wait complete. 2024-03-23T17:49:50.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.643+0000 7f46b5600700 1 Processor -- start 2024-03-23T17:49:50.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.643+0000 7f46b5600700 1 -- start start 2024-03-23T17:49:50.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.643+0000 7f46b5600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46b0101710 0x7f46b0198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:50.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.643+0000 7f46b5600700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46b0104040 0x7f46b0199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:50.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.643+0000 7f46b5600700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46b0106970 0x7f46b01a0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:50.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.643+0000 7f46b5600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46b01a0900 con 0x7f46b0101710 2024-03-23T17:49:50.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.643+0000 7f46b5600700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46b01a0a70 con 0x7f46b0104040 2024-03-23T17:49:50.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.643+0000 7f46b5600700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46b01a0c30 con 0x7f46b0106970 2024-03-23T17:49:50.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.643+0000 7f46aea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46b0101710 0x7f46b0198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:50.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.643+0000 7f46aea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46b0101710 0x7f46b0198e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47026/0 (socket says 172.21.15.86:47026) 2024-03-23T17:49:50.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.643+0000 7f46aea00700 1 -- 172.21.15.86:0/536892461 learned_addr learned my addr 172.21.15.86:0/536892461 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:50.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.644+0000 7f46ae000700 1 --2- 172.21.15.86:0/536892461 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46b0104040 0x7f46b0199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:50.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.644+0000 7f46af400700 1 --2- 172.21.15.86:0/536892461 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46b0106970 0x7f46b01a0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:50.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.644+0000 7f46aea00700 1 -- 172.21.15.86:0/536892461 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46b0106970 msgr2=0x7f46b01a0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:50.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.644+0000 7f46aea00700 1 --2- 172.21.15.86:0/536892461 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46b0106970 0x7f46b01a0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:50.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.644+0000 7f46aea00700 1 -- 172.21.15.86:0/536892461 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46b0104040 msgr2=0x7f46b0199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:50.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.644+0000 7f46aea00700 1 --2- 172.21.15.86:0/536892461 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46b0104040 0x7f46b0199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:50.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.644+0000 7f46aea00700 1 -- 172.21.15.86:0/536892461 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f469c0095f0 con 0x7f46b0101710 2024-03-23T17:49:50.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.644+0000 7f46aea00700 1 --2- 172.21.15.86:0/536892461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46b0101710 0x7f46b0198e10 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f46a000e9a0 tx=0x7f46a000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:50.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.644+0000 7f46a7400700 1 -- 172.21.15.86:0/536892461 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a0009790 con 0x7f46b0101710 2024-03-23T17:49:50.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.644+0000 7f46a7400700 1 -- 172.21.15.86:0/536892461 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f46a000cbd0 con 0x7f46b0101710 2024-03-23T17:49:50.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.644+0000 7f46b5600700 1 -- 172.21.15.86:0/536892461 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46b01a0f10 con 0x7f46b0101710 2024-03-23T17:49:50.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.644+0000 7f46a7400700 1 -- 172.21.15.86:0/536892461 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a0018680 con 0x7f46b0101710 2024-03-23T17:49:50.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.644+0000 7f46b5600700 1 -- 172.21.15.86:0/536892461 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46b01a1340 con 0x7f46b0101710 2024-03-23T17:49:50.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.645+0000 7f46b5600700 1 -- 172.21.15.86:0/536892461 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46b0108540 con 0x7f46b0101710 2024-03-23T17:49:50.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.647+0000 7f46a7400700 1 -- 172.21.15.86:0/536892461 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f46a000cd40 con 0x7f46b0101710 2024-03-23T17:49:50.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.648+0000 7f46a7400700 1 --2- 172.21.15.86:0/536892461 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f46940a1230 0x7f46940a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:50.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.648+0000 7f46a7400700 1 -- 172.21.15.86:0/536892461 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f46a0014070 con 0x7f46b0101710 2024-03-23T17:49:50.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.649+0000 7f46ae000700 1 --2- 172.21.15.86:0/536892461 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f46940a1230 0x7f46940a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:50.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.649+0000 7f46ae000700 1 --2- 172.21.15.86:0/536892461 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f46940a1230 0x7f46940a36f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f469c00b2e0 tx=0x7f469c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:50.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.651+0000 7f46a7400700 1 -- 172.21.15.86:0/536892461 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f46a0096570 con 0x7f46b0101710 2024-03-23T17:49:50.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.906+0000 7f46b5600700 1 -- 172.21.15.86:0/536892461 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f46b0067180 con 0x7f46b0101710 2024-03-23T17:49:50.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.907+0000 7f46a7400700 1 -- 172.21.15.86:0/536892461 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10913 (secure 0 0 0) 0x7f46a0099b90 con 0x7f46b0101710 2024-03-23T17:49:50.908 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:49:50.908 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14445,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1433140219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1433140219},{"type":"v1","addr":"172.21.15.86:6835","nonce":1433140219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6839/1879457730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1879457730},{"type":"v1","addr":"172.21.15.86:6839","nonce":1879457730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-23T17:48:56.202323+0000","modified":"2024-03-23T17:49:03.703276+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24553,"mds_2":24518,"mds_3":24505,"mds_4":24500},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14514":{"gid":14514,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/3026943866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3026943866},{"type":"v1","addr":"172.21.15.86:6841","nonce":3026943866}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24464":{"gid":24464,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6833/1467091186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1467091186},{"type":"v1","addr":"172.21.15.92:6833","nonce":1467091186}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6833/3075554623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3075554623},{"type":"v1","addr":"172.21.15.152:6833","nonce":3075554623}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.92:6835/908900027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":908900027},{"type":"v1","addr":"172.21.15.92:6835","nonce":908900027}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"d","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.86:6837/2933881328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2933881328},{"type":"v1","addr":"172.21.15.86:6837","nonce":2933881328}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6837/3967837948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":3967837948},{"type":"v1","addr":"172.21.15.92:6837","nonce":3967837948}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6835/3137085021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3137085021},{"type":"v1","addr":"172.21.15.152:6835","nonce":3137085021}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24538":{"gid":24538,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6837/1151208965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":1151208965},{"type":"v1","addr":"172.21.15.152:6837","nonce":1151208965}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.152:6839/1100723992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":1100723992},{"type":"v1","addr":"172.21.15.152:6839","nonce":1100723992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6839/1394139887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1394139887},{"type":"v1","addr":"172.21.15.92:6839","nonce":1394139887}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-23T17:49:50.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.909+0000 7f46b5600700 1 -- 172.21.15.86:0/536892461 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f46940a1230 msgr2=0x7f46940a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:50.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.909+0000 7f46b5600700 1 --2- 172.21.15.86:0/536892461 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f46940a1230 0x7f46940a36f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f469c00b2e0 tx=0x7f469c005fb0 comp rx=0 tx=0).stop 2024-03-23T17:49:50.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.909+0000 7f46b5600700 1 -- 172.21.15.86:0/536892461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46b0101710 msgr2=0x7f46b0198e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:50.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.909+0000 7f46b5600700 1 --2- 172.21.15.86:0/536892461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46b0101710 0x7f46b0198e10 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f46a000e9a0 tx=0x7f46a000ecb0 comp rx=0 tx=0).stop 2024-03-23T17:49:50.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.909+0000 7f46b5600700 1 -- 172.21.15.86:0/536892461 shutdown_connections 2024-03-23T17:49:50.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.909+0000 7f46b5600700 1 --2- 172.21.15.86:0/536892461 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46b0106970 0x7f46b01a0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:50.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.909+0000 7f46b5600700 1 --2- 172.21.15.86:0/536892461 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f46940a1230 0x7f46940a36f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:50.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.909+0000 7f46b5600700 1 --2- 172.21.15.86:0/536892461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46b0101710 0x7f46b0198e10 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:50.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.909+0000 7f46b5600700 1 --2- 172.21.15.86:0/536892461 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46b0104040 0x7f46b0199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:50.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.909+0000 7f46b5600700 1 -- 172.21.15.86:0/536892461 >> 172.21.15.86:0/536892461 conn(0x7f46b00fb100 msgr2=0x7f46b0100fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:50.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.909+0000 7f46b5600700 1 -- 172.21.15.86:0/536892461 shutdown_connections 2024-03-23T17:49:50.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:50.909+0000 7f46b5600700 1 -- 172.21.15.86:0/536892461 wait complete. 2024-03-23T17:49:50.912 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 27 2024-03-23T17:49:50.929 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-23T17:48:56.202323+0000', 'modified': '2024-03-23T17:49:03.703276+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24563, 'mds_1': 24553, 'mds_2': 24518, 'mds_3': 24505, 'mds_4': 24500}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14514': {'gid': 14514, 'name': 'a', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.86:6841/3026943866', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6840', 'nonce': 3026943866}, {'type': 'v1', 'addr': '172.21.15.86:6841', 'nonce': 3026943866}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24464': {'gid': 24464, 'name': 'k', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.92:6833/1467091186', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6832', 'nonce': 1467091186}, {'type': 'v1', 'addr': '172.21.15.92:6833', 'nonce': 1467091186}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24475': {'gid': 24475, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.152:6833/3075554623', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6832', 'nonce': 3075554623}, {'type': 'v1', 'addr': '172.21.15.152:6833', 'nonce': 3075554623}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24500': {'gid': 24500, 'name': 'e', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.92:6835/908900027', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6834', 'nonce': 908900027}, {'type': 'v1', 'addr': '172.21.15.92:6835', 'nonce': 908900027}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24505': {'gid': 24505, 'name': 'd', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.86:6837/2933881328', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 2933881328}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 2933881328}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24518': {'gid': 24518, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.92:6837/3967837948', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6836', 'nonce': 3967837948}, {'type': 'v1', 'addr': '172.21.15.92:6837', 'nonce': 3967837948}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24526': {'gid': 24526, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.152:6835/3137085021', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6834', 'nonce': 3137085021}, {'type': 'v1', 'addr': '172.21.15.152:6835', 'nonce': 3137085021}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24538': {'gid': 24538, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.152:6837/1151208965', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6836', 'nonce': 1151208965}, {'type': 'v1', 'addr': '172.21.15.152:6837', 'nonce': 1151208965}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24553': {'gid': 24553, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.152:6839/1100723992', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6838', 'nonce': 1100723992}, {'type': 'v1', 'addr': '172.21.15.152:6839', 'nonce': 1100723992}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24563': {'gid': 24563, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.92:6839/1394139887', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6838', 'nonce': 1394139887}, {'type': 'v1', 'addr': '172.21.15.92:6839', 'nonce': 1394139887}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-23T17:49:50.929 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-03-23T17:49:50.929 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-03-23T17:49:51.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.121+0000 7f4d5d400700 1 -- 172.21.15.86:0/1656150616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d58101f70 msgr2=0x7f4d581023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:51.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.121+0000 7f4d5d400700 1 --2- 172.21.15.86:0/1656150616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d58101f70 0x7f4d581023d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4d440099c0 tx=0x7f4d44009cd0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.121+0000 7f4d5d400700 1 -- 172.21.15.86:0/1656150616 shutdown_connections 2024-03-23T17:49:51.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.121+0000 7f4d5d400700 1 --2- 172.21.15.86:0/1656150616 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d58102910 0x7f4d581091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.121+0000 7f4d5d400700 1 --2- 172.21.15.86:0/1656150616 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d58100d80 0x7f4d581011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.121+0000 7f4d5d400700 1 --2- 172.21.15.86:0/1656150616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d58101f70 0x7f4d581023d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.121+0000 7f4d5d400700 1 -- 172.21.15.86:0/1656150616 >> 172.21.15.86:0/1656150616 conn(0x7f4d580fc2f0 msgr2=0x7f4d580fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:51.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.121+0000 7f4d5d400700 1 -- 172.21.15.86:0/1656150616 shutdown_connections 2024-03-23T17:49:51.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.121+0000 7f4d5d400700 1 -- 172.21.15.86:0/1656150616 wait complete. 2024-03-23T17:49:51.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.122+0000 7f4d5d400700 1 Processor -- start 2024-03-23T17:49:51.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.122+0000 7f4d5d400700 1 -- start start 2024-03-23T17:49:51.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.122+0000 7f4d5d400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d58100d80 0x7f4d58198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:51.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.122+0000 7f4d5d400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d58101f70 0x7f4d58199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:51.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.123+0000 7f4d5d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d58102910 0x7f4d581a0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:51.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.123+0000 7f4d5d400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d581a0a30 con 0x7f4d58102910 2024-03-23T17:49:51.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.123+0000 7f4d5d400700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d581a0b70 con 0x7f4d58100d80 2024-03-23T17:49:51.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.123+0000 7f4d5d400700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d581a0d00 con 0x7f4d58101f70 2024-03-23T17:49:51.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.123+0000 7f4d56800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d58100d80 0x7f4d58198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:51.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.123+0000 7f4d57200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d58102910 0x7f4d581a0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:51.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.123+0000 7f4d56800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d58100d80 0x7f4d58198f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:42682/0 (socket says 172.21.15.86:42682) 2024-03-23T17:49:51.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.123+0000 7f4d56800700 1 -- 172.21.15.86:0/2636483380 learned_addr learned my addr 172.21.15.86:0/2636483380 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:51.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.123+0000 7f4d55e00700 1 --2- 172.21.15.86:0/2636483380 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d58101f70 0x7f4d58199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:51.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.123+0000 7f4d56800700 1 -- 172.21.15.86:0/2636483380 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d58101f70 msgr2=0x7f4d58199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:51.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.123+0000 7f4d56800700 1 --2- 172.21.15.86:0/2636483380 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d58101f70 0x7f4d58199440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.123+0000 7f4d56800700 1 -- 172.21.15.86:0/2636483380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d58102910 msgr2=0x7f4d581a0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:51.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.123+0000 7f4d56800700 1 --2- 172.21.15.86:0/2636483380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d58102910 0x7f4d581a0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.123+0000 7f4d56800700 1 -- 172.21.15.86:0/2636483380 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d440095f0 con 0x7f4d58100d80 2024-03-23T17:49:51.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.123+0000 7f4d57200700 1 --2- 172.21.15.86:0/2636483380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d58102910 0x7f4d581a0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:49:51.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.123+0000 7f4d56800700 1 --2- 172.21.15.86:0/2636483380 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d58100d80 0x7f4d58198f00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4d4800e970 tx=0x7f4d4800ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:51.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.124+0000 7f4d4f400700 1 -- 172.21.15.86:0/2636483380 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d48009790 con 0x7f4d58100d80 2024-03-23T17:49:51.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.124+0000 7f4d4f400700 1 -- 172.21.15.86:0/2636483380 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d4800c7e0 con 0x7f4d58100d80 2024-03-23T17:49:51.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.124+0000 7f4d5d400700 1 -- 172.21.15.86:0/2636483380 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d581a0fe0 con 0x7f4d58100d80 2024-03-23T17:49:51.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.124+0000 7f4d4f400700 1 -- 172.21.15.86:0/2636483380 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d48003c70 con 0x7f4d58100d80 2024-03-23T17:49:51.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.124+0000 7f4d5d400700 1 -- 172.21.15.86:0/2636483380 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d581a1400 con 0x7f4d58100d80 2024-03-23T17:49:51.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.128+0000 7f4d4f400700 1 -- 172.21.15.86:0/2636483380 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f4d4800cc50 con 0x7f4d58100d80 2024-03-23T17:49:51.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.129+0000 7f4d70e63b80 1 Processor -- start 2024-03-23T17:49:51.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.129+0000 7f4d70e63b80 1 -- start start 2024-03-23T17:49:51.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.130+0000 7f4d70e63b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556458d6a780 0x556458d6aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:51.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.130+0000 7f4d70e63b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556458d6b0e0 0x556458d6b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:51.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.130+0000 7f4d70e63b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556458d6bf80 0x556458d6c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:51.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.130+0000 7f4d70e63b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x556458cc2940 con 0x556458d6a780 2024-03-23T17:49:51.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.130+0000 7f4d70e63b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x556458cc0300 con 0x556458d6bf80 2024-03-23T17:49:51.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.130+0000 7f4d70e63b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x556458cbbbf0 con 0x556458d6b0e0 2024-03-23T17:49:51.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.130+0000 7f4d4f400700 1 --2- 172.21.15.86:0/2636483380 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f4d3c0a11b0 0x7f4d3c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:51.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.130+0000 7f4d55e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556458d6bf80 0x556458d6c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:51.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.130+0000 7f4d55e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556458d6bf80 0x556458d6c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:42686/0 (socket says 172.21.15.86:42686) 2024-03-23T17:49:51.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.130+0000 7f4d55e00700 1 -- 172.21.15.86:0/1982726631 learned_addr learned my addr 172.21.15.86:0/1982726631 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:51.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.130+0000 7f4d57200700 1 --2- 172.21.15.86:0/1982726631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556458d6a780 0x556458d6aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:51.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.130+0000 7f4d4f400700 1 -- 172.21.15.86:0/2636483380 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f4d48014070 con 0x7f4d58100d80 2024-03-23T17:49:51.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.130+0000 7f4d56800700 1 --2- 172.21.15.86:0/1982726631 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556458d6b0e0 0x556458d6b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:51.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.130+0000 7f4d57200700 1 --2- 172.21.15.86:0/2636483380 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f4d3c0a11b0 0x7f4d3c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:51.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.130+0000 7f4d57200700 1 -- 172.21.15.86:0/1982726631 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556458d6b0e0 msgr2=0x556458d6b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:49:51.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.130+0000 7f4d57200700 1 --2- 172.21.15.86:0/1982726631 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556458d6b0e0 0x556458d6b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.130+0000 7f4d57200700 1 -- 172.21.15.86:0/1982726631 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556458d6bf80 msgr2=0x556458d6c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:49:51.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.130+0000 7f4d57200700 1 --2- 172.21.15.86:0/1982726631 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556458d6bf80 0x556458d6c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.130+0000 7f4d57200700 1 -- 172.21.15.86:0/1982726631 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556458b3d360 con 0x556458d6a780 2024-03-23T17:49:51.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.130+0000 7f4d56800700 1 --2- 172.21.15.86:0/1982726631 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556458d6b0e0 0x556458d6b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:49:51.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.131+0000 7f4d57200700 1 --2- 172.21.15.86:0/1982726631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556458d6a780 0x556458d6aba0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4d50019420 tx=0x7f4d50019730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bc05c62f4e2c254 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:51.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.131+0000 7f4d57200700 1 --2- 172.21.15.86:0/2636483380 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f4d3c0a11b0 0x7f4d3c0a3670 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4d50019a80 tx=0x7f4d5000d000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:51.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.131+0000 7f4d43400700 1 -- 172.21.15.86:0/1982726631 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d50007b30 con 0x556458d6a780 2024-03-23T17:49:51.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.131+0000 7f4d43400700 1 -- 172.21.15.86:0/1982726631 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d5000f040 con 0x556458d6a780 2024-03-23T17:49:51.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.131+0000 7f4d43400700 1 -- 172.21.15.86:0/1982726631 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d50005280 con 0x556458d6a780 2024-03-23T17:49:51.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.132+0000 7f4d70e63b80 1 -- 172.21.15.86:0/1982726631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556458d6a780 msgr2=0x556458d6aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:51.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.132+0000 7f4d70e63b80 1 --2- 172.21.15.86:0/1982726631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556458d6a780 0x556458d6aba0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4d50019420 tx=0x7f4d50019730 comp rx=0 tx=0).stop 2024-03-23T17:49:51.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.132+0000 7f4d70e63b80 1 -- 172.21.15.86:0/1982726631 shutdown_connections 2024-03-23T17:49:51.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.132+0000 7f4d70e63b80 1 --2- 172.21.15.86:0/1982726631 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556458d6bf80 0x556458d6c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.132+0000 7f4d70e63b80 1 --2- 172.21.15.86:0/1982726631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556458d6a780 0x556458d6aba0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.132+0000 7f4d70e63b80 1 --2- 172.21.15.86:0/1982726631 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556458d6b0e0 0x556458d6b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.132+0000 7f4d70e63b80 1 -- 172.21.15.86:0/1982726631 >> 172.21.15.86:0/1982726631 conn(0x556458cc9650 msgr2=0x556458d65950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:51.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.132+0000 7f4d70e63b80 1 -- 172.21.15.86:0/1982726631 shutdown_connections 2024-03-23T17:49:51.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.132+0000 7f4d70e63b80 1 -- 172.21.15.86:0/1982726631 wait complete. 2024-03-23T17:49:51.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.132+0000 7f4d70e63b80 1 -- start start 2024-03-23T17:49:51.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.132+0000 7f4d70e63b80 1 Processor -- start 2024-03-23T17:49:51.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.133+0000 7f4d70e63b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556458d6a780 0x556458d745f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:51.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.133+0000 7f4d70e63b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556458d6b0e0 0x556458d74b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:51.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.133+0000 7f4d70e63b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556458d6bf80 0x556458d7bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:51.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.133+0000 7f4d70e63b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x556458c57d30 con 0x556458d6bf80 2024-03-23T17:49:51.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.133+0000 7f4d70e63b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x556458c579d0 con 0x556458d6a780 2024-03-23T17:49:51.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.133+0000 7f4d70e63b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x556458b3f650 con 0x556458d6b0e0 2024-03-23T17:49:51.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.133+0000 7f4d70e63b80 10 client.0 _ll_register_callbacks cb 0x556458c61420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:49:51.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.133+0000 7f4d56800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556458d6a780 0x556458d745f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:51.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.133+0000 7f4d56800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556458d6a780 0x556458d745f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:42700/0 (socket says 172.21.15.86:42700) 2024-03-23T17:49:51.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.133+0000 7f4d56800700 1 -- 172.21.15.86:0/3598909783 learned_addr learned my addr 172.21.15.86:0/3598909783 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:51.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.133+0000 7f4d57200700 1 --2- 172.21.15.86:0/3598909783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556458d6bf80 0x556458d7bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:51.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.133+0000 7f4d55e00700 1 --2- 172.21.15.86:0/3598909783 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556458d6b0e0 0x556458d74b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:51.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.134+0000 7f4d56800700 1 -- 172.21.15.86:0/3598909783 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556458d6b0e0 msgr2=0x556458d74b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:49:51.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.134+0000 7f4d56800700 1 --2- 172.21.15.86:0/3598909783 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556458d6b0e0 0x556458d74b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.134+0000 7f4d56800700 1 -- 172.21.15.86:0/3598909783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556458d6bf80 msgr2=0x556458d7bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:49:51.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.134+0000 7f4d56800700 1 --2- 172.21.15.86:0/3598909783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556458d6bf80 0x556458d7bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.134+0000 7f4d56800700 1 -- 172.21.15.86:0/3598909783 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d581a1400 con 0x556458d6a780 2024-03-23T17:49:51.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.134+0000 7f4d56800700 1 --2- 172.21.15.86:0/3598909783 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556458d6a780 0x556458d745f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4d480ca5e0 tx=0x7f4d48099db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9558d646215626f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:51.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.134+0000 7f4d37e00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T17:49:51.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.134+0000 7f4d37e00700 1 -- 172.21.15.86:0/3598909783 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d4809a350 con 0x556458d6a780 2024-03-23T17:49:51.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.134+0000 7f4d37e00700 1 -- 172.21.15.86:0/3598909783 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f4d480a2370 con 0x556458d6a780 2024-03-23T17:49:51.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.134+0000 7f4d70e63b80 1 -- 172.21.15.86:0/3598909783 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556458c7c330 con 0x556458d6a780 2024-03-23T17:49:51.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.134+0000 7f4d37e00700 1 -- 172.21.15.86:0/3598909783 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4d480993e0 con 0x556458d6a780 2024-03-23T17:49:51.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.134+0000 7f4d70e63b80 10 client.24962 fetch_fsmap learned FSMap version 27 2024-03-23T17:49:51.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.134+0000 7f4d70e63b80 1 -- 172.21.15.86:0/3598909783 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f4d50011040 con 0x556458d6a780 2024-03-23T17:49:51.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.134+0000 7f4d37e00700 1 -- 172.21.15.86:0/3598909783 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d48063020 con 0x556458d6a780 2024-03-23T17:49:51.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.135+0000 7f4d37e00700 1 -- 172.21.15.86:0/3598909783 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f4d480a1fe0 con 0x556458d6a780 2024-03-23T17:49:51.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.135+0000 7f4d70e63b80 10 client.24962 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T17:49:51.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.135+0000 7f4d70e63b80 10 client.24962 resolve_mds: resolved name 'e' to daemon mds.e 2024-03-23T17:49:51.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.135+0000 7f4d70e63b80 20 client.24962 populate_metadata read hostname 'smithi086' 2024-03-23T17:49:51.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.135+0000 7f4d70e63b80 1 --2- 172.21.15.86:0/3598909783 >> [v2:172.21.15.92:6834/908900027,v1:172.21.15.92:6835/908900027] conn(0x556458d7f810 0x556458d84af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:51.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.135+0000 7f4d70e63b80 4 client.24962 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:49:51.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.135+0000 7f4d70e63b80 1 -- 172.21.15.86:0/3598909783 --> [v2:172.21.15.92:6834/908900027,v1:172.21.15.92:6835/908900027] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556458cb46b0 con 0x556458d7f810 2024-03-23T17:49:51.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.135+0000 7f4d55e00700 1 --2- 172.21.15.86:0/3598909783 >> [v2:172.21.15.92:6834/908900027,v1:172.21.15.92:6835/908900027] conn(0x556458d7f810 0x556458d84af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:51.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.136+0000 7f4d37e00700 10 client.24962 ms_handle_connect on v2:172.21.15.92:6834/908900027 2024-03-23T17:49:51.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.136+0000 7f4d55e00700 1 --2- 172.21.15.86:0/3598909783 >> [v2:172.21.15.92:6834/908900027,v1:172.21.15.92:6835/908900027] conn(0x556458d7f810 0x556458d84af0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a7861f850ee8e479 server_cookie=e03c6c653cf2bc57 in_seq=0 out_seq=0 2024-03-23T17:49:51.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.136+0000 7f4d37e00700 1 -- 172.21.15.86:0/3598909783 <== mds.4 v2:172.21.15.92:6834/908900027 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x556458cb46b0 con 0x556458d7f810 2024-03-23T17:49:51.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.136+0000 7f4d37e00700 10 client.24962 handle_command_reply: tid=0 2024-03-23T17:49:51.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.136+0000 7f4d70e63b80 1 client.24962 shutdown 2024-03-23T17:49:51.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.136+0000 7f4d70e63b80 1 -- 172.21.15.86:0/3598909783 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556458d6a780 msgr2=0x556458d745f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:51.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.136+0000 7f4d70e63b80 1 --2- 172.21.15.86:0/3598909783 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556458d6a780 0x556458d745f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4d480ca5e0 tx=0x7f4d48099db0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.136+0000 7f4d70e63b80 1 -- 172.21.15.86:0/3598909783 shutdown_connections 2024-03-23T17:49:51.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.137+0000 7f4d70e63b80 1 --2- 172.21.15.86:0/3598909783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556458d6bf80 0x556458d7bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.137+0000 7f4d70e63b80 1 --2- 172.21.15.86:0/3598909783 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556458d6a780 0x556458d745f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.137+0000 7f4d70e63b80 1 --2- 172.21.15.86:0/3598909783 >> [v2:172.21.15.92:6834/908900027,v1:172.21.15.92:6835/908900027] conn(0x556458d7f810 0x556458d84af0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.137+0000 7f4d70e63b80 1 --2- 172.21.15.86:0/3598909783 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556458d6b0e0 0x556458d74b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.137+0000 7f4d37e00700 0 client.24962 ms_handle_reset on v2:172.21.15.92:6834/908900027 2024-03-23T17:49:51.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.137+0000 7f4d70e63b80 1 -- 172.21.15.86:0/3598909783 >> 172.21.15.86:0/3598909783 conn(0x556458cc9650 msgr2=0x556458d69120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:51.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.137+0000 7f4d70e63b80 1 -- 172.21.15.86:0/3598909783 shutdown_connections 2024-03-23T17:49:51.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.137+0000 7f4d70e63b80 1 -- 172.21.15.86:0/3598909783 wait complete. 2024-03-23T17:49:51.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.137+0000 7f4d70e63b80 20 client.24962 trim_cache size 0 max 16384 2024-03-23T17:49:51.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.154+0000 7f4d70e63b80 1 Processor -- start 2024-03-23T17:49:51.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.154+0000 7f4d70e63b80 1 -- start start 2024-03-23T17:49:51.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.154+0000 7f4d70e63b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556458d6a780 0x556458d6ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:51.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.154+0000 7f4d70e63b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556458d6b0e0 0x556458d79b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:51.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.154+0000 7f4d70e63b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556458d6bf80 0x556458d6c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:51.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.154+0000 7f4d70e63b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x556458d6b830 con 0x556458d6a780 2024-03-23T17:49:51.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.154+0000 7f4d70e63b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x556458c79ad0 con 0x556458d6b0e0 2024-03-23T17:49:51.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.154+0000 7f4d70e63b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x556458cb4990 con 0x556458d6bf80 2024-03-23T17:49:51.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.155+0000 7f4d56800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556458d6a780 0x556458d6ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:51.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.155+0000 7f4d56800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556458d6a780 0x556458d6ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47074/0 (socket says 172.21.15.86:47074) 2024-03-23T17:49:51.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.155+0000 7f4d56800700 1 -- 172.21.15.86:0/3486142215 learned_addr learned my addr 172.21.15.86:0/3486142215 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:51.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.155+0000 7f4d55e00700 1 --2- 172.21.15.86:0/3486142215 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556458d6b0e0 0x556458d79b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:51.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.155+0000 7f4d57200700 1 --2- 172.21.15.86:0/3486142215 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556458d6bf80 0x556458d6c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:51.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.155+0000 7f4d56800700 1 -- 172.21.15.86:0/3486142215 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556458d6bf80 msgr2=0x556458d6c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:49:51.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.155+0000 7f4d56800700 1 --2- 172.21.15.86:0/3486142215 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556458d6bf80 0x556458d6c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.155+0000 7f4d56800700 1 -- 172.21.15.86:0/3486142215 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556458d6b0e0 msgr2=0x556458d79b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:49:51.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.155+0000 7f4d56800700 1 --2- 172.21.15.86:0/3486142215 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556458d6b0e0 0x556458d79b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.155+0000 7f4d56800700 1 -- 172.21.15.86:0/3486142215 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556458ae9b10 con 0x556458d6a780 2024-03-23T17:49:51.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.155+0000 7f4d56800700 1 --2- 172.21.15.86:0/3486142215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556458d6a780 0x556458d6ab60 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4d48096960 tx=0x7f4d4809dea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c44e6eb4e155b786 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:51.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.155+0000 7f4d43e00700 1 -- 172.21.15.86:0/3486142215 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d480a2370 con 0x556458d6a780 2024-03-23T17:49:51.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.155+0000 7f4d43e00700 1 -- 172.21.15.86:0/3486142215 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556458c7c330 con 0x556458d6a780 2024-03-23T17:49:51.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.155+0000 7f4d43e00700 1 -- 172.21.15.86:0/3486142215 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d4809a3b0 con 0x556458d6a780 2024-03-23T17:49:51.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.156+0000 7f4d70e63b80 1 -- 172.21.15.86:0/3486142215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556458d6a780 msgr2=0x556458d6ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:51.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.156+0000 7f4d70e63b80 1 --2- 172.21.15.86:0/3486142215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556458d6a780 0x556458d6ab60 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4d48096960 tx=0x7f4d4809dea0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.156+0000 7f4d70e63b80 1 -- 172.21.15.86:0/3486142215 shutdown_connections 2024-03-23T17:49:51.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.156+0000 7f4d70e63b80 1 --2- 172.21.15.86:0/3486142215 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556458d6bf80 0x556458d6c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.156+0000 7f4d70e63b80 1 --2- 172.21.15.86:0/3486142215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556458d6a780 0x556458d6ab60 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.156+0000 7f4d70e63b80 1 --2- 172.21.15.86:0/3486142215 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556458d6b0e0 0x556458d79b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.156+0000 7f4d70e63b80 1 -- 172.21.15.86:0/3486142215 >> 172.21.15.86:0/3486142215 conn(0x556458cc9650 msgr2=0x556458d724f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:51.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.156+0000 7f4d70e63b80 1 -- 172.21.15.86:0/3486142215 shutdown_connections 2024-03-23T17:49:51.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.156+0000 7f4d70e63b80 1 -- 172.21.15.86:0/3486142215 wait complete. 2024-03-23T17:49:51.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.156+0000 7f4d70e63b80 1 -- start start 2024-03-23T17:49:51.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.157+0000 7f4d70e63b80 1 Processor -- start 2024-03-23T17:49:51.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.157+0000 7f4d70e63b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556458d6a780 0x556458d78880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:51.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.157+0000 7f4d70e63b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556458d6b0e0 0x556458d78dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:51.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.157+0000 7f4d70e63b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556458d6bf80 0x556458dd1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:51.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.157+0000 7f4d70e63b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x556458d79630 con 0x556458d6b0e0 2024-03-23T17:49:51.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.157+0000 7f4d70e63b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x556458dd2310 con 0x556458d6bf80 2024-03-23T17:49:51.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.157+0000 7f4d70e63b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x556458dd2450 con 0x556458d6a780 2024-03-23T17:49:51.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.157+0000 7f4d70e63b80 10 client.0 _ll_register_callbacks cb 0x556458c61420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:49:51.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.157+0000 7f4d56800700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556458d6a780 0x556458d78880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:51.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.157+0000 7f4d55e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556458d6b0e0 0x556458d78dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:51.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.157+0000 7f4d56800700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556458d6a780 0x556458d78880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:56594/0 (socket says 172.21.15.86:56594) 2024-03-23T17:49:51.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.157+0000 7f4d57200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556458d6bf80 0x556458dd1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:51.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.157+0000 7f4d55e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556458d6b0e0 0x556458d78dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47080/0 (socket says 172.21.15.86:47080) 2024-03-23T17:49:51.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.157+0000 7f4d56800700 1 -- 172.21.15.86:0/678531466 learned_addr learned my addr 172.21.15.86:0/678531466 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:51.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.158+0000 7f4d55e00700 1 -- 172.21.15.86:0/678531466 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556458d6a780 msgr2=0x556458d78880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:49:51.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.158+0000 7f4d55e00700 1 --2- 172.21.15.86:0/678531466 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556458d6a780 0x556458d78880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.158+0000 7f4d55e00700 1 -- 172.21.15.86:0/678531466 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556458d6bf80 msgr2=0x556458dd1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:49:51.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.158+0000 7f4d55e00700 1 --2- 172.21.15.86:0/678531466 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556458d6bf80 0x556458dd1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.158+0000 7f4d55e00700 1 -- 172.21.15.86:0/678531466 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d50011040 con 0x556458d6b0e0 2024-03-23T17:49:51.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.158+0000 7f4d56800700 1 --2- 172.21.15.86:0/678531466 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556458d6a780 0x556458d78880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:49:51.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.158+0000 7f4d55e00700 1 --2- 172.21.15.86:0/678531466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556458d6b0e0 0x556458d78dc0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4d440095c0 tx=0x7f4d44005640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fe943ebb730df9c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:51.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.158+0000 7f4d37e00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-23T17:49:51.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.158+0000 7f4d37e00700 1 -- 172.21.15.86:0/678531466 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d440278f0 con 0x556458d6b0e0 2024-03-23T17:49:51.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.158+0000 7f4d37e00700 1 -- 172.21.15.86:0/678531466 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f4d44027a50 con 0x556458d6b0e0 2024-03-23T17:49:51.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.158+0000 7f4d70e63b80 1 -- 172.21.15.86:0/678531466 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4d5000f040 con 0x556458d6b0e0 2024-03-23T17:49:51.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.158+0000 7f4d37e00700 1 -- 172.21.15.86:0/678531466 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4d44027c10 con 0x556458d6b0e0 2024-03-23T17:49:51.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.158+0000 7f4d70e63b80 10 client.14928 fetch_fsmap learned FSMap version 27 2024-03-23T17:49:51.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.158+0000 7f4d70e63b80 1 -- 172.21.15.86:0/678531466 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x556458c81e50 con 0x556458d6b0e0 2024-03-23T17:49:51.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.159+0000 7f4d37e00700 1 -- 172.21.15.86:0/678531466 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d5000f040 con 0x556458d6b0e0 2024-03-23T17:49:51.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.159+0000 7f4d37e00700 1 -- 172.21.15.86:0/678531466 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f4d44003960 con 0x556458d6b0e0 2024-03-23T17:49:51.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.159+0000 7f4d70e63b80 10 client.14928 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T17:49:51.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.159+0000 7f4d70e63b80 10 client.14928 resolve_mds: resolved name 'e' to daemon mds.e 2024-03-23T17:49:51.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.159+0000 7f4d70e63b80 20 client.14928 populate_metadata read hostname 'smithi086' 2024-03-23T17:49:51.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.159+0000 7f4d70e63b80 1 --2- 172.21.15.86:0/678531466 >> [v2:172.21.15.92:6834/908900027,v1:172.21.15.92:6835/908900027] conn(0x556458dd7260 0x556458dd3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:51.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.159+0000 7f4d70e63b80 4 client.14928 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-23T17:49:51.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.159+0000 7f4d70e63b80 1 -- 172.21.15.86:0/678531466 --> [v2:172.21.15.92:6834/908900027,v1:172.21.15.92:6835/908900027] -- command(tid 0: {"prefix": "status"}) v1 -- 0x556458b0aca0 con 0x556458dd7260 2024-03-23T17:49:51.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.159+0000 7f4d56800700 1 --2- 172.21.15.86:0/678531466 >> [v2:172.21.15.92:6834/908900027,v1:172.21.15.92:6835/908900027] conn(0x556458dd7260 0x556458dd3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:51.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.160+0000 7f4d37e00700 10 client.14928 ms_handle_connect on v2:172.21.15.92:6834/908900027 2024-03-23T17:49:51.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.160+0000 7f4d56800700 1 --2- 172.21.15.86:0/678531466 >> [v2:172.21.15.92:6834/908900027,v1:172.21.15.92:6835/908900027] conn(0x556458dd7260 0x556458dd3b80 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b4e523550b537c50 server_cookie=c5a00ac185c77ba3 in_seq=0 out_seq=0 2024-03-23T17:49:51.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.160+0000 7f4d37e00700 1 -- 172.21.15.86:0/678531466 <== mds.4 v2:172.21.15.92:6834/908900027 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x556458b0aca0 con 0x556458dd7260 2024-03-23T17:49:51.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.160+0000 7f4d37e00700 10 client.14928 handle_command_reply: tid=0 2024-03-23T17:49:51.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.160+0000 7f4d70e63b80 1 client.14928 shutdown 2024-03-23T17:49:51.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.160+0000 7f4d70e63b80 1 -- 172.21.15.86:0/678531466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556458d6b0e0 msgr2=0x556458d78dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:51.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.160+0000 7f4d70e63b80 1 --2- 172.21.15.86:0/678531466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556458d6b0e0 0x556458d78dc0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4d440095c0 tx=0x7f4d44005640 comp rx=0 tx=0).stop 2024-03-23T17:49:51.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.160+0000 7f4d70e63b80 1 -- 172.21.15.86:0/678531466 shutdown_connections 2024-03-23T17:49:51.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.160+0000 7f4d70e63b80 1 --2- 172.21.15.86:0/678531466 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556458d6bf80 0x556458dd1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.160+0000 7f4d70e63b80 1 --2- 172.21.15.86:0/678531466 >> [v2:172.21.15.92:6834/908900027,v1:172.21.15.92:6835/908900027] conn(0x556458dd7260 0x556458dd3b80 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.160+0000 7f4d70e63b80 1 --2- 172.21.15.86:0/678531466 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556458d6a780 0x556458d78880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.160+0000 7f4d70e63b80 1 --2- 172.21.15.86:0/678531466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556458d6b0e0 0x556458d78dc0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.160+0000 7f4d37e00700 0 client.14928 ms_handle_reset on v2:172.21.15.92:6834/908900027 2024-03-23T17:49:51.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.160+0000 7f4d70e63b80 1 -- 172.21.15.86:0/678531466 >> 172.21.15.86:0/678531466 conn(0x556458cc9650 msgr2=0x556458d72360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:51.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.160+0000 7f4d70e63b80 1 -- 172.21.15.86:0/678531466 shutdown_connections 2024-03-23T17:49:51.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.160+0000 7f4d70e63b80 1 -- 172.21.15.86:0/678531466 wait complete. 2024-03-23T17:49:51.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.160+0000 7f4d70e63b80 20 client.14928 trim_cache size 0 max 16384 2024-03-23T17:49:51.168 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-23T17:49:51.168 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "9d76c9e8-e93c-11ee-95cd-87774f69a715", 2024-03-23T17:49:51.168 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 4, 2024-03-23T17:49:51.168 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24500, 2024-03-23T17:49:51.168 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-03-23T17:49:51.169 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-03-23T17:49:51.169 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-03-23T17:49:51.169 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 48.461855329000002, 2024-03-23T17:49:51.169 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 27, 2024-03-23T17:49:51.169 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 82, 2024-03-23T17:49:51.169 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 80, 2024-03-23T17:49:51.169 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 75.565828452999995 2024-03-23T17:49:51.169 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-23T17:49:51.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.161+0000 7f4d5d400700 1 -- 172.21.15.86:0/2636483380 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f4d3c0a11b0 msgr2=0x7f4d3c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:51.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.161+0000 7f4d5d400700 1 --2- 172.21.15.86:0/2636483380 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f4d3c0a11b0 0x7f4d3c0a3670 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4d50019a80 tx=0x7f4d5000d000 comp rx=0 tx=0).stop 2024-03-23T17:49:51.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.161+0000 7f4d5d400700 1 -- 172.21.15.86:0/2636483380 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d58100d80 msgr2=0x7f4d58198f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:51.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.161+0000 7f4d5d400700 1 --2- 172.21.15.86:0/2636483380 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d58100d80 0x7f4d58198f00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4d4800e970 tx=0x7f4d4800ec80 comp rx=0 tx=0).stop 2024-03-23T17:49:51.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.161+0000 7f4d5d400700 1 -- 172.21.15.86:0/2636483380 shutdown_connections 2024-03-23T17:49:51.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.161+0000 7f4d5d400700 1 --2- 172.21.15.86:0/2636483380 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f4d3c0a11b0 0x7f4d3c0a3670 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.161+0000 7f4d5d400700 1 --2- 172.21.15.86:0/2636483380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d58102910 0x7f4d581a0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.161+0000 7f4d5d400700 1 --2- 172.21.15.86:0/2636483380 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d58100d80 0x7f4d58198f00 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.161+0000 7f4d5d400700 1 --2- 172.21.15.86:0/2636483380 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d58101f70 0x7f4d58199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.161+0000 7f4d5d400700 1 -- 172.21.15.86:0/2636483380 >> 172.21.15.86:0/2636483380 conn(0x7f4d580fc2f0 msgr2=0x7f4d58072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:51.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.161+0000 7f4d5d400700 1 -- 172.21.15.86:0/2636483380 shutdown_connections 2024-03-23T17:49:51.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.161+0000 7f4d5d400700 1 -- 172.21.15.86:0/2636483380 wait complete. 2024-03-23T17:49:51.178 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:50 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/3791808422' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T17:49:51.178 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:50 smithi086 ceph-mon[38221]: pgmap v263: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-23T17:49:51.178 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:50 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/1590213400' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T17:49:51.181 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-03-23T17:49:51.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:50 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/3791808422' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T17:49:51.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:50 smithi092 ceph-mon[79089]: pgmap v263: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-23T17:49:51.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:50 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/1590213400' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T17:49:51.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:50 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/3791808422' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T17:49:51.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:50 smithi152 ceph-mon[47949]: pgmap v263: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-23T17:49:51.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:50 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/1590213400' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T17:49:51.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.372+0000 7f54d1a00700 1 -- 172.21.15.86:0/3960236860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54cc100d80 msgr2=0x7f54cc1011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:51.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.372+0000 7f54d1a00700 1 --2- 172.21.15.86:0/3960236860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54cc100d80 0x7f54cc1011a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f54bc0099c0 tx=0x7f54bc009cd0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.372+0000 7f54d1a00700 1 -- 172.21.15.86:0/3960236860 shutdown_connections 2024-03-23T17:49:51.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.372+0000 7f54d1a00700 1 --2- 172.21.15.86:0/3960236860 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f54cc102910 0x7f54cc1091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.372+0000 7f54d1a00700 1 --2- 172.21.15.86:0/3960236860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54cc100d80 0x7f54cc1011a0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.372+0000 7f54d1a00700 1 --2- 172.21.15.86:0/3960236860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54cc101f70 0x7f54cc1023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.372+0000 7f54d1a00700 1 -- 172.21.15.86:0/3960236860 >> 172.21.15.86:0/3960236860 conn(0x7f54cc0fc2f0 msgr2=0x7f54cc0fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:51.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.372+0000 7f54d1a00700 1 -- 172.21.15.86:0/3960236860 shutdown_connections 2024-03-23T17:49:51.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.372+0000 7f54d1a00700 1 -- 172.21.15.86:0/3960236860 wait complete. 2024-03-23T17:49:51.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.373+0000 7f54d1a00700 1 Processor -- start 2024-03-23T17:49:51.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.373+0000 7f54d1a00700 1 -- start start 2024-03-23T17:49:51.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.374+0000 7f54d1a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54cc100d80 0x7f54cc074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:51.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.374+0000 7f54d1a00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f54cc101f70 0x7f54cc0731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:51.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.374+0000 7f54d1a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54cc102910 0x7f54cc0736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:51.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.374+0000 7f54d1a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54cc073e80 con 0x7f54cc100d80 2024-03-23T17:49:51.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.374+0000 7f54d1a00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54cc073ff0 con 0x7f54cc102910 2024-03-23T17:49:51.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.374+0000 7f54d1a00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54cc1a0dc0 con 0x7f54cc101f70 2024-03-23T17:49:51.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.374+0000 7f54caa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54cc100d80 0x7f54cc074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:51.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.374+0000 7f54caa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54cc100d80 0x7f54cc074b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47096/0 (socket says 172.21.15.86:47096) 2024-03-23T17:49:51.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.374+0000 7f54caa00700 1 -- 172.21.15.86:0/2748155098 learned_addr learned my addr 172.21.15.86:0/2748155098 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:51.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.374+0000 7f54cb400700 1 --2- 172.21.15.86:0/2748155098 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54cc102910 0x7f54cc0736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:51.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.374+0000 7f54ca000700 1 --2- 172.21.15.86:0/2748155098 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f54cc101f70 0x7f54cc0731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:51.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.374+0000 7f54caa00700 1 -- 172.21.15.86:0/2748155098 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f54cc101f70 msgr2=0x7f54cc0731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:51.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.374+0000 7f54caa00700 1 --2- 172.21.15.86:0/2748155098 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f54cc101f70 0x7f54cc0731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.374+0000 7f54caa00700 1 -- 172.21.15.86:0/2748155098 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54cc102910 msgr2=0x7f54cc0736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:51.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.374+0000 7f54caa00700 1 --2- 172.21.15.86:0/2748155098 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54cc102910 0x7f54cc0736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.374+0000 7f54caa00700 1 -- 172.21.15.86:0/2748155098 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54bc0095f0 con 0x7f54cc100d80 2024-03-23T17:49:51.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.375+0000 7f54caa00700 1 --2- 172.21.15.86:0/2748155098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54cc100d80 0x7f54cc074b10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f54bc003730 tx=0x7f54bc00fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:51.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.375+0000 7f54c3400700 1 -- 172.21.15.86:0/2748155098 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54bc01d070 con 0x7f54cc100d80 2024-03-23T17:49:51.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.375+0000 7f54c3400700 1 -- 172.21.15.86:0/2748155098 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f54bc00bda0 con 0x7f54cc100d80 2024-03-23T17:49:51.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.375+0000 7f54d1a00700 1 -- 172.21.15.86:0/2748155098 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54cc1a0ff0 con 0x7f54cc100d80 2024-03-23T17:49:51.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.375+0000 7f54c3400700 1 -- 172.21.15.86:0/2748155098 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54bc004b30 con 0x7f54cc100d80 2024-03-23T17:49:51.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.375+0000 7f54d1a00700 1 -- 172.21.15.86:0/2748155098 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54cc1a1360 con 0x7f54cc100d80 2024-03-23T17:49:51.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.378+0000 7f54c3400700 1 -- 172.21.15.86:0/2748155098 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f54bc004c90 con 0x7f54cc100d80 2024-03-23T17:49:51.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.379+0000 7f54e5563b80 1 Processor -- start 2024-03-23T17:49:51.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.379+0000 7f54c3400700 1 --2- 172.21.15.86:0/2748155098 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f54b00a11b0 0x7f54b00a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:51.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.379+0000 7f54e5563b80 1 -- start start 2024-03-23T17:49:51.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.379+0000 7f54c3400700 1 -- 172.21.15.86:0/2748155098 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f54bc0cd920 con 0x7f54cc100d80 2024-03-23T17:49:51.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.379+0000 7f54cb400700 1 --2- 172.21.15.86:0/2748155098 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f54b00a11b0 0x7f54b00a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:51.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.380+0000 7f54e5563b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f7e840780 0x555f7e840ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:51.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.380+0000 7f54e5563b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555f7e8410e0 0x555f7e8419a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:51.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.380+0000 7f54e5563b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555f7e841f80 0x555f7e842400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:51.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.380+0000 7f54e5563b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f7e798940 con 0x555f7e840780 2024-03-23T17:49:51.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.380+0000 7f54e5563b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f7e796300 con 0x555f7e841f80 2024-03-23T17:49:51.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.380+0000 7f54e5563b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f7e791bf0 con 0x555f7e8410e0 2024-03-23T17:49:51.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.380+0000 7f54caa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f7e840780 0x555f7e840ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:51.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.380+0000 7f54caa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f7e840780 0x555f7e840ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47100/0 (socket says 172.21.15.86:47100) 2024-03-23T17:49:51.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.380+0000 7f54caa00700 1 -- 172.21.15.86:0/714296462 learned_addr learned my addr 172.21.15.86:0/714296462 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:51.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.380+0000 7f54cb400700 1 --2- 172.21.15.86:0/714296462 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555f7e841f80 0x555f7e842400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:51.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.380+0000 7f54ca000700 1 --2- 172.21.15.86:0/714296462 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555f7e8410e0 0x555f7e8419a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:51.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.380+0000 7f54cb400700 1 --2- 172.21.15.86:0/2748155098 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f54b00a11b0 0x7f54b00a3670 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f54c4003ee0 tx=0x7f54c4009130 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:51.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.380+0000 7f54caa00700 1 -- 172.21.15.86:0/714296462 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555f7e8410e0 msgr2=0x555f7e8419a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:49:51.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.380+0000 7f54caa00700 1 --2- 172.21.15.86:0/714296462 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555f7e8410e0 0x555f7e8419a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.380+0000 7f54caa00700 1 -- 172.21.15.86:0/714296462 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555f7e841f80 msgr2=0x555f7e842400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:49:51.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.380+0000 7f54caa00700 1 --2- 172.21.15.86:0/714296462 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555f7e841f80 0x555f7e842400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.380+0000 7f54caa00700 1 -- 172.21.15.86:0/714296462 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f7e613360 con 0x555f7e840780 2024-03-23T17:49:51.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.380+0000 7f54caa00700 1 --2- 172.21.15.86:0/714296462 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f7e840780 0x555f7e840ba0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f54bc097170 tx=0x7f54bc09a800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=158db62ebf310e59 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:51.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.381+0000 7f54b7e00700 1 -- 172.21.15.86:0/714296462 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54bc00fdc0 con 0x555f7e840780 2024-03-23T17:49:51.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.381+0000 7f54b7e00700 1 -- 172.21.15.86:0/714296462 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f54bc09b370 con 0x555f7e840780 2024-03-23T17:49:51.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.381+0000 7f54b7e00700 1 -- 172.21.15.86:0/714296462 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54bc09a160 con 0x555f7e840780 2024-03-23T17:49:51.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.381+0000 7f54e5563b80 1 -- 172.21.15.86:0/714296462 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f7e840780 msgr2=0x555f7e840ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:51.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.381+0000 7f54e5563b80 1 --2- 172.21.15.86:0/714296462 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f7e840780 0x555f7e840ba0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f54bc097170 tx=0x7f54bc09a800 comp rx=0 tx=0).stop 2024-03-23T17:49:51.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.381+0000 7f54e5563b80 1 -- 172.21.15.86:0/714296462 shutdown_connections 2024-03-23T17:49:51.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.381+0000 7f54e5563b80 1 --2- 172.21.15.86:0/714296462 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555f7e841f80 0x555f7e842400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.381+0000 7f54e5563b80 1 --2- 172.21.15.86:0/714296462 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f7e840780 0x555f7e840ba0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.381+0000 7f54e5563b80 1 --2- 172.21.15.86:0/714296462 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555f7e8410e0 0x555f7e8419a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.381+0000 7f54e5563b80 1 -- 172.21.15.86:0/714296462 >> 172.21.15.86:0/714296462 conn(0x555f7e79f650 msgr2=0x555f7e83b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:51.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.382+0000 7f54e5563b80 1 -- 172.21.15.86:0/714296462 shutdown_connections 2024-03-23T17:49:51.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.382+0000 7f54e5563b80 1 -- 172.21.15.86:0/714296462 wait complete. 2024-03-23T17:49:51.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.382+0000 7f54e5563b80 1 -- start start 2024-03-23T17:49:51.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.382+0000 7f54e5563b80 1 Processor -- start 2024-03-23T17:49:51.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.383+0000 7f54e5563b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555f7e840780 0x555f7e84a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:51.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.383+0000 7f54e5563b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f7e8410e0 0x555f7e84a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:51.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.383+0000 7f54e5563b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555f7e841f80 0x555f7e8517e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:51.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.383+0000 7f54e5563b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f7e72dd30 con 0x555f7e8410e0 2024-03-23T17:49:51.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.383+0000 7f54e5563b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f7e72d9d0 con 0x555f7e840780 2024-03-23T17:49:51.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.383+0000 7f54e5563b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f7e615650 con 0x555f7e841f80 2024-03-23T17:49:51.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.383+0000 7f54e5563b80 10 client.0 _ll_register_callbacks cb 0x555f7e737420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:49:51.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.383+0000 7f54caa00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555f7e840780 0x555f7e84a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:51.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.383+0000 7f54ca000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f7e8410e0 0x555f7e84a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:51.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.383+0000 7f54caa00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555f7e840780 0x555f7e84a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:42756/0 (socket says 172.21.15.86:42756) 2024-03-23T17:49:51.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.383+0000 7f54caa00700 1 -- 172.21.15.86:0/2479752627 learned_addr learned my addr 172.21.15.86:0/2479752627 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:51.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.383+0000 7f54cb400700 1 --2- 172.21.15.86:0/2479752627 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555f7e841f80 0x555f7e8517e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:51.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.383+0000 7f54caa00700 1 -- 172.21.15.86:0/2479752627 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555f7e841f80 msgr2=0x555f7e8517e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:49:51.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.383+0000 7f54caa00700 1 --2- 172.21.15.86:0/2479752627 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555f7e841f80 0x555f7e8517e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.383+0000 7f54caa00700 1 -- 172.21.15.86:0/2479752627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f7e8410e0 msgr2=0x555f7e84a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:49:51.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.383+0000 7f54caa00700 1 --2- 172.21.15.86:0/2479752627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f7e8410e0 0x555f7e84a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.383+0000 7f54caa00700 1 -- 172.21.15.86:0/2479752627 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54cc1a1360 con 0x555f7e840780 2024-03-23T17:49:51.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.383+0000 7f54cb400700 1 --2- 172.21.15.86:0/2479752627 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555f7e841f80 0x555f7e8517e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:49:51.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.383+0000 7f54caa00700 1 --2- 172.21.15.86:0/2479752627 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555f7e840780 0x555f7e84a260 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f54bc09b2d0 tx=0x7f54bc009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3fafef7070628ca5 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:51.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.383+0000 7f54b4c00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T17:49:51.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.384+0000 7f54b4c00700 1 -- 172.21.15.86:0/2479752627 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54bc09ac70 con 0x555f7e840780 2024-03-23T17:49:51.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.384+0000 7f54b4c00700 1 -- 172.21.15.86:0/2479752627 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f54bc00fdc0 con 0x555f7e840780 2024-03-23T17:49:51.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.384+0000 7f54e5563b80 1 -- 172.21.15.86:0/2479752627 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555f7e752330 con 0x555f7e840780 2024-03-23T17:49:51.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.384+0000 7f54b4c00700 1 -- 172.21.15.86:0/2479752627 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f54bc0a2de0 con 0x555f7e840780 2024-03-23T17:49:51.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.384+0000 7f54e5563b80 10 client.24980 fetch_fsmap learned FSMap version 27 2024-03-23T17:49:51.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.384+0000 7f54e5563b80 1 -- 172.21.15.86:0/2479752627 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x555f7e5bfb10 con 0x555f7e840780 2024-03-23T17:49:51.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.384+0000 7f54b4c00700 1 -- 172.21.15.86:0/2479752627 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f54bc07d080 con 0x555f7e840780 2024-03-23T17:49:51.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.384+0000 7f54b4c00700 1 -- 172.21.15.86:0/2479752627 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f54bc0aece0 con 0x555f7e840780 2024-03-23T17:49:51.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.384+0000 7f54e5563b80 10 client.24980 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T17:49:51.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.384+0000 7f54e5563b80 10 client.24980 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-23T17:49:51.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.384+0000 7f54e5563b80 20 client.24980 populate_metadata read hostname 'smithi086' 2024-03-23T17:49:51.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.384+0000 7f54e5563b80 1 --2- 172.21.15.86:0/2479752627 >> [v2:172.21.15.86:6836/2933881328,v1:172.21.15.86:6837/2933881328] conn(0x555f7e854fe0 0x555f7e85a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:51.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.384+0000 7f54e5563b80 4 client.24980 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:49:51.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.384+0000 7f54e5563b80 1 -- 172.21.15.86:0/2479752627 --> [v2:172.21.15.86:6836/2933881328,v1:172.21.15.86:6837/2933881328] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555f7e78a6b0 con 0x555f7e854fe0 2024-03-23T17:49:51.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.385+0000 7f54ca000700 1 --2- 172.21.15.86:0/2479752627 >> [v2:172.21.15.86:6836/2933881328,v1:172.21.15.86:6837/2933881328] conn(0x555f7e854fe0 0x555f7e85a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:51.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.385+0000 7f54b4c00700 10 client.24980 ms_handle_connect on v2:172.21.15.86:6836/2933881328 2024-03-23T17:49:51.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.385+0000 7f54ca000700 1 --2- 172.21.15.86:0/2479752627 >> [v2:172.21.15.86:6836/2933881328,v1:172.21.15.86:6837/2933881328] conn(0x555f7e854fe0 0x555f7e85a2a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f2754a129ad2abaf server_cookie=4821f9253c514448 in_seq=0 out_seq=0 2024-03-23T17:49:51.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.386+0000 7f54b4c00700 1 -- 172.21.15.86:0/2479752627 <== mds.3 v2:172.21.15.86:6836/2933881328 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x555f7e78a6b0 con 0x555f7e854fe0 2024-03-23T17:49:51.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.386+0000 7f54b4c00700 10 client.24980 handle_command_reply: tid=0 2024-03-23T17:49:51.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.386+0000 7f54e5563b80 1 client.24980 shutdown 2024-03-23T17:49:51.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.386+0000 7f54e5563b80 1 -- 172.21.15.86:0/2479752627 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555f7e840780 msgr2=0x555f7e84a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:51.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.386+0000 7f54e5563b80 1 --2- 172.21.15.86:0/2479752627 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555f7e840780 0x555f7e84a260 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f54bc09b2d0 tx=0x7f54bc009310 comp rx=0 tx=0).stop 2024-03-23T17:49:51.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.386+0000 7f54e5563b80 1 -- 172.21.15.86:0/2479752627 shutdown_connections 2024-03-23T17:49:51.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.386+0000 7f54e5563b80 1 --2- 172.21.15.86:0/2479752627 >> [v2:172.21.15.86:6836/2933881328,v1:172.21.15.86:6837/2933881328] conn(0x555f7e854fe0 0x555f7e85a2a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.386+0000 7f54e5563b80 1 --2- 172.21.15.86:0/2479752627 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555f7e841f80 0x555f7e8517e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.386+0000 7f54e5563b80 1 --2- 172.21.15.86:0/2479752627 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555f7e840780 0x555f7e84a260 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.386+0000 7f54e5563b80 1 --2- 172.21.15.86:0/2479752627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f7e8410e0 0x555f7e84a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.386+0000 7f54b4c00700 0 client.24980 ms_handle_reset on v2:172.21.15.86:6836/2933881328 2024-03-23T17:49:51.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.386+0000 7f54e5563b80 1 -- 172.21.15.86:0/2479752627 >> 172.21.15.86:0/2479752627 conn(0x555f7e79f650 msgr2=0x555f7e83f120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:51.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.386+0000 7f54e5563b80 1 -- 172.21.15.86:0/2479752627 shutdown_connections 2024-03-23T17:49:51.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.386+0000 7f54e5563b80 1 -- 172.21.15.86:0/2479752627 wait complete. 2024-03-23T17:49:51.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.386+0000 7f54e5563b80 20 client.24980 trim_cache size 0 max 16384 2024-03-23T17:49:51.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.404+0000 7f54e5563b80 1 Processor -- start 2024-03-23T17:49:51.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.404+0000 7f54e5563b80 1 -- start start 2024-03-23T17:49:51.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.404+0000 7f54e5563b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f7e840780 0x555f7e84f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:51.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.404+0000 7f54e5563b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555f7e8410e0 0x555f7e83f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:51.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.404+0000 7f54e5563b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555f7e841f80 0x555f7e83f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:51.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.404+0000 7f54e5563b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f7e72d9d0 con 0x555f7e840780 2024-03-23T17:49:51.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.404+0000 7f54e5563b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f7e796300 con 0x555f7e8410e0 2024-03-23T17:49:51.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.404+0000 7f54e5563b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f7e848e20 con 0x555f7e841f80 2024-03-23T17:49:51.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.404+0000 7f54caa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f7e840780 0x555f7e84f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:51.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.404+0000 7f54caa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f7e840780 0x555f7e84f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47126/0 (socket says 172.21.15.86:47126) 2024-03-23T17:49:51.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.404+0000 7f54caa00700 1 -- 172.21.15.86:0/3914838314 learned_addr learned my addr 172.21.15.86:0/3914838314 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:51.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.404+0000 7f54ca000700 1 --2- 172.21.15.86:0/3914838314 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555f7e8410e0 0x555f7e83f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:51.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.404+0000 7f54cb400700 1 --2- 172.21.15.86:0/3914838314 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555f7e841f80 0x555f7e83f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:51.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.405+0000 7f54caa00700 1 -- 172.21.15.86:0/3914838314 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555f7e841f80 msgr2=0x555f7e83f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:49:51.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.405+0000 7f54caa00700 1 --2- 172.21.15.86:0/3914838314 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555f7e841f80 0x555f7e83f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.405+0000 7f54caa00700 1 -- 172.21.15.86:0/3914838314 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555f7e8410e0 msgr2=0x555f7e83f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:49:51.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.405+0000 7f54caa00700 1 --2- 172.21.15.86:0/3914838314 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555f7e8410e0 0x555f7e83f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.405+0000 7f54caa00700 1 -- 172.21.15.86:0/3914838314 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f7e757e50 con 0x555f7e840780 2024-03-23T17:49:51.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.405+0000 7f54ca000700 1 --2- 172.21.15.86:0/3914838314 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555f7e8410e0 0x555f7e83f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:49:51.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.405+0000 7f54caa00700 1 --2- 172.21.15.86:0/3914838314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f7e840780 0x555f7e84f790 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f54bc09ec60 tx=0x7f54bc09b250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcbe3ab1a709805f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:51.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.405+0000 7f54c0c00700 1 -- 172.21.15.86:0/3914838314 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54bc09ac70 con 0x555f7e840780 2024-03-23T17:49:51.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.405+0000 7f54c0c00700 1 -- 172.21.15.86:0/3914838314 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555f7e752330 con 0x555f7e840780 2024-03-23T17:49:51.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.405+0000 7f54c0c00700 1 -- 172.21.15.86:0/3914838314 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54bc099010 con 0x555f7e840780 2024-03-23T17:49:51.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.406+0000 7f54e5563b80 1 -- 172.21.15.86:0/3914838314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f7e840780 msgr2=0x555f7e84f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:51.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.406+0000 7f54e5563b80 1 --2- 172.21.15.86:0/3914838314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f7e840780 0x555f7e84f790 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f54bc09ec60 tx=0x7f54bc09b250 comp rx=0 tx=0).stop 2024-03-23T17:49:51.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.406+0000 7f54e5563b80 1 -- 172.21.15.86:0/3914838314 shutdown_connections 2024-03-23T17:49:51.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.406+0000 7f54e5563b80 1 --2- 172.21.15.86:0/3914838314 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555f7e841f80 0x555f7e83f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.406+0000 7f54e5563b80 1 --2- 172.21.15.86:0/3914838314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f7e840780 0x555f7e84f790 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.406+0000 7f54e5563b80 1 --2- 172.21.15.86:0/3914838314 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555f7e8410e0 0x555f7e83f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.406+0000 7f54e5563b80 1 -- 172.21.15.86:0/3914838314 >> 172.21.15.86:0/3914838314 conn(0x555f7e79f650 msgr2=0x555f7e848440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:51.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.406+0000 7f54e5563b80 1 -- 172.21.15.86:0/3914838314 shutdown_connections 2024-03-23T17:49:51.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.406+0000 7f54e5563b80 1 -- 172.21.15.86:0/3914838314 wait complete. 2024-03-23T17:49:51.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.406+0000 7f54e5563b80 1 -- start start 2024-03-23T17:49:51.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.406+0000 7f54e5563b80 1 Processor -- start 2024-03-23T17:49:51.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.407+0000 7f54e5563b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f7e840780 0x555f7e849a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:51.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.407+0000 7f54e5563b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555f7e8410e0 0x555f7e84e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:51.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.407+0000 7f54e5563b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555f7e841f80 0x555f7e84ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:51.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.407+0000 7f54e5563b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f7e84f450 con 0x555f7e840780 2024-03-23T17:49:51.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.407+0000 7f54e5563b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f7e84f5c0 con 0x555f7e841f80 2024-03-23T17:49:51.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.407+0000 7f54e5563b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f7e8a75d0 con 0x555f7e8410e0 2024-03-23T17:49:51.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.407+0000 7f54e5563b80 10 client.0 _ll_register_callbacks cb 0x555f7e737420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:49:51.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.407+0000 7f54caa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f7e840780 0x555f7e849a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:51.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.407+0000 7f54caa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f7e840780 0x555f7e849a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47132/0 (socket says 172.21.15.86:47132) 2024-03-23T17:49:51.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.407+0000 7f54caa00700 1 -- 172.21.15.86:0/600806042 learned_addr learned my addr 172.21.15.86:0/600806042 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:51.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.407+0000 7f54ca000700 1 --2- 172.21.15.86:0/600806042 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555f7e8410e0 0x555f7e84e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:51.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.407+0000 7f54cb400700 1 --2- 172.21.15.86:0/600806042 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555f7e841f80 0x555f7e84ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:51.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.407+0000 7f54caa00700 1 -- 172.21.15.86:0/600806042 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555f7e8410e0 msgr2=0x555f7e84e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:49:51.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.407+0000 7f54caa00700 1 --2- 172.21.15.86:0/600806042 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555f7e8410e0 0x555f7e84e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.407+0000 7f54caa00700 1 -- 172.21.15.86:0/600806042 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555f7e841f80 msgr2=0x555f7e84ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:49:51.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.407+0000 7f54caa00700 1 --2- 172.21.15.86:0/600806042 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555f7e841f80 0x555f7e84ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.407+0000 7f54caa00700 1 -- 172.21.15.86:0/600806042 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54b8010430 con 0x555f7e840780 2024-03-23T17:49:51.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.407+0000 7f54cb400700 1 --2- 172.21.15.86:0/600806042 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555f7e841f80 0x555f7e84ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:49:51.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.408+0000 7f54caa00700 1 --2- 172.21.15.86:0/600806042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f7e840780 0x555f7e849a20 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f54bc009fd0 tx=0x7f54bc0cb350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=873d2403e11c89c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:51.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.408+0000 7f54b4c00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-23T17:49:51.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.408+0000 7f54b4c00700 1 -- 172.21.15.86:0/600806042 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54bc00fdc0 con 0x555f7e840780 2024-03-23T17:49:51.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.408+0000 7f54b4c00700 1 -- 172.21.15.86:0/600806042 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f54bc09eaa0 con 0x555f7e840780 2024-03-23T17:49:51.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.408+0000 7f54e5563b80 1 -- 172.21.15.86:0/600806042 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f54bc09b370 con 0x555f7e840780 2024-03-23T17:49:51.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.408+0000 7f54b4c00700 1 -- 172.21.15.86:0/600806042 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f54bc09ac70 con 0x555f7e840780 2024-03-23T17:49:51.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.408+0000 7f54e5563b80 10 client.14946 fetch_fsmap learned FSMap version 27 2024-03-23T17:49:51.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.408+0000 7f54e5563b80 1 -- 172.21.15.86:0/600806042 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x555f7e5bfb10 con 0x555f7e840780 2024-03-23T17:49:51.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.408+0000 7f54b4c00700 1 -- 172.21.15.86:0/600806042 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f54bc07d080 con 0x555f7e840780 2024-03-23T17:49:51.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.408+0000 7f54b4c00700 1 -- 172.21.15.86:0/600806042 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f54bc0993f0 con 0x555f7e840780 2024-03-23T17:49:51.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.408+0000 7f54e5563b80 10 client.14946 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T17:49:51.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.408+0000 7f54e5563b80 10 client.14946 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-23T17:49:51.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.408+0000 7f54e5563b80 20 client.14946 populate_metadata read hostname 'smithi086' 2024-03-23T17:49:51.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.409+0000 7f54e5563b80 1 --2- 172.21.15.86:0/600806042 >> [v2:172.21.15.86:6836/2933881328,v1:172.21.15.86:6837/2933881328] conn(0x555f7e8abca0 0x555f7e8a88c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:51.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.409+0000 7f54e5563b80 4 client.14946 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-23T17:49:51.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.409+0000 7f54e5563b80 1 -- 172.21.15.86:0/600806042 --> [v2:172.21.15.86:6836/2933881328,v1:172.21.15.86:6837/2933881328] -- command(tid 0: {"prefix": "status"}) v1 -- 0x555f7e5e0ca0 con 0x555f7e8abca0 2024-03-23T17:49:51.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.409+0000 7f54ca000700 1 --2- 172.21.15.86:0/600806042 >> [v2:172.21.15.86:6836/2933881328,v1:172.21.15.86:6837/2933881328] conn(0x555f7e8abca0 0x555f7e8a88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:51.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.409+0000 7f54ca000700 1 --2- 172.21.15.86:0/600806042 >> [v2:172.21.15.86:6836/2933881328,v1:172.21.15.86:6837/2933881328] conn(0x555f7e8abca0 0x555f7e8a88c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c6c83e3b6dbdcc17 server_cookie=a99462e17380b4d2 in_seq=0 out_seq=0 2024-03-23T17:49:51.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.409+0000 7f54b4c00700 10 client.14946 ms_handle_connect on v2:172.21.15.86:6836/2933881328 2024-03-23T17:49:51.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.410+0000 7f54b4c00700 1 -- 172.21.15.86:0/600806042 <== mds.3 v2:172.21.15.86:6836/2933881328 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x555f7e5e0ca0 con 0x555f7e8abca0 2024-03-23T17:49:51.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.410+0000 7f54b4c00700 10 client.14946 handle_command_reply: tid=0 2024-03-23T17:49:51.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.410+0000 7f54e5563b80 1 client.14946 shutdown 2024-03-23T17:49:51.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.410+0000 7f54e5563b80 1 -- 172.21.15.86:0/600806042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f7e840780 msgr2=0x555f7e849a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:51.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.410+0000 7f54e5563b80 1 --2- 172.21.15.86:0/600806042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f7e840780 0x555f7e849a20 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f54bc009fd0 tx=0x7f54bc0cb350 comp rx=0 tx=0).stop 2024-03-23T17:49:51.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.410+0000 7f54e5563b80 1 -- 172.21.15.86:0/600806042 shutdown_connections 2024-03-23T17:49:51.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.410+0000 7f54e5563b80 1 --2- 172.21.15.86:0/600806042 >> [v2:172.21.15.86:6836/2933881328,v1:172.21.15.86:6837/2933881328] conn(0x555f7e8abca0 0x555f7e8a88c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.410+0000 7f54e5563b80 1 --2- 172.21.15.86:0/600806042 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555f7e841f80 0x555f7e84ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.410+0000 7f54e5563b80 1 --2- 172.21.15.86:0/600806042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f7e840780 0x555f7e849a20 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.410+0000 7f54b4c00700 0 client.14946 ms_handle_reset on v2:172.21.15.86:6836/2933881328 2024-03-23T17:49:51.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.410+0000 7f54e5563b80 1 --2- 172.21.15.86:0/600806042 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555f7e8410e0 0x555f7e84e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.410+0000 7f54e5563b80 1 -- 172.21.15.86:0/600806042 >> 172.21.15.86:0/600806042 conn(0x555f7e79f650 msgr2=0x555f7e83f120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:51.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.410+0000 7f54e5563b80 1 -- 172.21.15.86:0/600806042 shutdown_connections 2024-03-23T17:49:51.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.410+0000 7f54e5563b80 1 -- 172.21.15.86:0/600806042 wait complete. 2024-03-23T17:49:51.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.410+0000 7f54e5563b80 20 client.14946 trim_cache size 0 max 16384 2024-03-23T17:49:51.417 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-23T17:49:51.417 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "9d76c9e8-e93c-11ee-95cd-87774f69a715", 2024-03-23T17:49:51.417 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 3, 2024-03-23T17:49:51.417 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24505, 2024-03-23T17:49:51.417 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-03-23T17:49:51.417 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-03-23T17:49:51.417 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-03-23T17:49:51.417 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 49.73959395, 2024-03-23T17:49:51.418 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 27, 2024-03-23T17:49:51.418 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 82, 2024-03-23T17:49:51.418 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 80, 2024-03-23T17:49:51.418 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 77.593272411000001 2024-03-23T17:49:51.418 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-23T17:49:51.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.411+0000 7f54d1a00700 1 -- 172.21.15.86:0/2748155098 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f54b00a11b0 msgr2=0x7f54b00a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:51.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.411+0000 7f54d1a00700 1 --2- 172.21.15.86:0/2748155098 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f54b00a11b0 0x7f54b00a3670 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f54c4003ee0 tx=0x7f54c4009130 comp rx=0 tx=0).stop 2024-03-23T17:49:51.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.411+0000 7f54d1a00700 1 -- 172.21.15.86:0/2748155098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54cc100d80 msgr2=0x7f54cc074b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:51.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.411+0000 7f54d1a00700 1 --2- 172.21.15.86:0/2748155098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54cc100d80 0x7f54cc074b10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f54bc003730 tx=0x7f54bc00fa00 comp rx=0 tx=0).stop 2024-03-23T17:49:51.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.411+0000 7f54d1a00700 1 -- 172.21.15.86:0/2748155098 shutdown_connections 2024-03-23T17:49:51.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.411+0000 7f54d1a00700 1 --2- 172.21.15.86:0/2748155098 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54cc102910 0x7f54cc0736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.411+0000 7f54d1a00700 1 --2- 172.21.15.86:0/2748155098 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f54b00a11b0 0x7f54b00a3670 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.411+0000 7f54d1a00700 1 --2- 172.21.15.86:0/2748155098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54cc100d80 0x7f54cc074b10 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.411+0000 7f54d1a00700 1 --2- 172.21.15.86:0/2748155098 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f54cc101f70 0x7f54cc0731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.411+0000 7f54d1a00700 1 -- 172.21.15.86:0/2748155098 >> 172.21.15.86:0/2748155098 conn(0x7f54cc0fc2f0 msgr2=0x7f54cc105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:51.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.411+0000 7f54d1a00700 1 -- 172.21.15.86:0/2748155098 shutdown_connections 2024-03-23T17:49:51.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.411+0000 7f54d1a00700 1 -- 172.21.15.86:0/2748155098 wait complete. 2024-03-23T17:49:51.430 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-03-23T17:49:51.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.620+0000 7f3b62400700 1 -- 172.21.15.86:0/1343943585 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b5c100d80 msgr2=0x7f3b5c1011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:51.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.620+0000 7f3b62400700 1 --2- 172.21.15.86:0/1343943585 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b5c100d80 0x7f3b5c1011a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c009920 tx=0x7f3b4c009c30 comp rx=0 tx=0).stop 2024-03-23T17:49:51.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.620+0000 7f3b62400700 1 -- 172.21.15.86:0/1343943585 shutdown_connections 2024-03-23T17:49:51.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.620+0000 7f3b62400700 1 --2- 172.21.15.86:0/1343943585 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3b5c102910 0x7f3b5c1091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.620+0000 7f3b62400700 1 --2- 172.21.15.86:0/1343943585 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b5c100d80 0x7f3b5c1011a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.620+0000 7f3b62400700 1 --2- 172.21.15.86:0/1343943585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b5c101f70 0x7f3b5c1023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.620+0000 7f3b62400700 1 -- 172.21.15.86:0/1343943585 >> 172.21.15.86:0/1343943585 conn(0x7f3b5c0fc2f0 msgr2=0x7f3b5c0fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:51.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.620+0000 7f3b62400700 1 -- 172.21.15.86:0/1343943585 shutdown_connections 2024-03-23T17:49:51.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.620+0000 7f3b62400700 1 -- 172.21.15.86:0/1343943585 wait complete. 2024-03-23T17:49:51.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.621+0000 7f3b62400700 1 Processor -- start 2024-03-23T17:49:51.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.621+0000 7f3b62400700 1 -- start start 2024-03-23T17:49:51.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.622+0000 7f3b62400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3b5c100d80 0x7f3b5c074ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:51.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.622+0000 7f3b62400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b5c101f70 0x7f3b5c0731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:51.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.622+0000 7f3b62400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b5c102910 0x7f3b5c0736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:51.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.622+0000 7f3b62400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b5c073de0 con 0x7f3b5c101f70 2024-03-23T17:49:51.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.622+0000 7f3b62400700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b5c073f50 con 0x7f3b5c102910 2024-03-23T17:49:51.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.622+0000 7f3b62400700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b5c1a0ed0 con 0x7f3b5c100d80 2024-03-23T17:49:51.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.622+0000 7f3b5aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b5c101f70 0x7f3b5c0731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:51.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.622+0000 7f3b5aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b5c101f70 0x7f3b5c0731a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47152/0 (socket says 172.21.15.86:47152) 2024-03-23T17:49:51.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.622+0000 7f3b5aa00700 1 -- 172.21.15.86:0/2720079607 learned_addr learned my addr 172.21.15.86:0/2720079607 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:51.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.622+0000 7f3b5b400700 1 --2- 172.21.15.86:0/2720079607 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3b5c100d80 0x7f3b5c074ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:51.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.622+0000 7f3b5be00700 1 --2- 172.21.15.86:0/2720079607 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b5c102910 0x7f3b5c0736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:51.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.622+0000 7f3b5aa00700 1 -- 172.21.15.86:0/2720079607 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3b5c100d80 msgr2=0x7f3b5c074ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:51.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.622+0000 7f3b5aa00700 1 --2- 172.21.15.86:0/2720079607 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3b5c100d80 0x7f3b5c074ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.622+0000 7f3b5aa00700 1 -- 172.21.15.86:0/2720079607 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b5c102910 msgr2=0x7f3b5c0736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:51.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.622+0000 7f3b5aa00700 1 --2- 172.21.15.86:0/2720079607 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b5c102910 0x7f3b5c0736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.622+0000 7f3b5aa00700 1 -- 172.21.15.86:0/2720079607 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b4c009520 con 0x7f3b5c101f70 2024-03-23T17:49:51.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.622+0000 7f3b5b400700 1 --2- 172.21.15.86:0/2720079607 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3b5c100d80 0x7f3b5c074ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:49:51.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.622+0000 7f3b5be00700 1 --2- 172.21.15.86:0/2720079607 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b5c102910 0x7f3b5c0736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:49:51.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.623+0000 7f3b5aa00700 1 --2- 172.21.15.86:0/2720079607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b5c101f70 0x7f3b5c0731a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3b4800e9c0 tx=0x7f3b4800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:51.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.623+0000 7f3b53e00700 1 -- 172.21.15.86:0/2720079607 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b48004a30 con 0x7f3b5c101f70 2024-03-23T17:49:51.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.623+0000 7f3b53e00700 1 -- 172.21.15.86:0/2720079607 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b4800cc80 con 0x7f3b5c101f70 2024-03-23T17:49:51.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.623+0000 7f3b53e00700 1 -- 172.21.15.86:0/2720079607 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b480054e0 con 0x7f3b5c101f70 2024-03-23T17:49:51.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.623+0000 7f3b62400700 1 -- 172.21.15.86:0/2720079607 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b5c1a1130 con 0x7f3b5c101f70 2024-03-23T17:49:51.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.623+0000 7f3b62400700 1 -- 172.21.15.86:0/2720079607 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b5c1a15f0 con 0x7f3b5c101f70 2024-03-23T17:49:51.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.626+0000 7f3b53e00700 1 -- 172.21.15.86:0/2720079607 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f3b48004b90 con 0x7f3b5c101f70 2024-03-23T17:49:51.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.627+0000 7f3b53e00700 1 --2- 172.21.15.86:0/2720079607 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f3b400a11b0 0x7f3b400a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:51.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.627+0000 7f3b75eafb80 1 Processor -- start 2024-03-23T17:49:51.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.627+0000 7f3b53e00700 1 -- 172.21.15.86:0/2720079607 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f3b48014070 con 0x7f3b5c101f70 2024-03-23T17:49:51.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.628+0000 7f3b75eafb80 1 -- start start 2024-03-23T17:49:51.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.628+0000 7f3b5b400700 1 --2- 172.21.15.86:0/2720079607 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f3b400a11b0 0x7f3b400a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:51.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.628+0000 7f3b75eafb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557d67a64780 0x557d67a64ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:51.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.628+0000 7f3b75eafb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d67a650e0 0x557d67a659a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:51.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.628+0000 7f3b75eafb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d67a65f80 0x557d67a66400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:51.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.628+0000 7f3b75eafb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d679bc940 con 0x557d67a65f80 2024-03-23T17:49:51.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.628+0000 7f3b75eafb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d679ba300 con 0x557d67a650e0 2024-03-23T17:49:51.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.628+0000 7f3b75eafb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d679b5bf0 con 0x557d67a64780 2024-03-23T17:49:51.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.628+0000 7f3b5b400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557d67a64780 0x557d67a64ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:51.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.628+0000 7f3b5be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d67a65f80 0x557d67a66400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:51.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.628+0000 7f3b5b400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557d67a64780 0x557d67a64ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:56678/0 (socket says 172.21.15.86:56678) 2024-03-23T17:49:51.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.628+0000 7f3b5be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d67a65f80 0x557d67a66400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47160/0 (socket says 172.21.15.86:47160) 2024-03-23T17:49:51.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.628+0000 7f3b5b400700 1 -- 172.21.15.86:0/3412985639 learned_addr learned my addr 172.21.15.86:0/3412985639 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:51.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.628+0000 7f3b5aa00700 1 --2- 172.21.15.86:0/3412985639 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d67a650e0 0x557d67a659a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:51.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.628+0000 7f3b5b400700 1 --2- 172.21.15.86:0/2720079607 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f3b400a11b0 0x7f3b400a3670 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c0098f0 tx=0x7f3b4c019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:51.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.629+0000 7f3b5b400700 1 -- 172.21.15.86:0/3412985639 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d67a650e0 msgr2=0x557d67a659a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:49:51.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.629+0000 7f3b5b400700 1 --2- 172.21.15.86:0/3412985639 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d67a650e0 0x557d67a659a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.629+0000 7f3b5b400700 1 -- 172.21.15.86:0/3412985639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d67a65f80 msgr2=0x557d67a66400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:49:51.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.629+0000 7f3b5b400700 1 --2- 172.21.15.86:0/3412985639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d67a65f80 0x557d67a66400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.629+0000 7f3b5b400700 1 -- 172.21.15.86:0/3412985639 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d67837360 con 0x557d67a64780 2024-03-23T17:49:51.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.629+0000 7f3b5aa00700 1 --2- 172.21.15.86:0/3412985639 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d67a650e0 0x557d67a659a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:49:51.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.629+0000 7f3b5be00700 1 --2- 172.21.15.86:0/3412985639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d67a65f80 0x557d67a66400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:49:51.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.629+0000 7f3b5b400700 1 --2- 172.21.15.86:0/3412985639 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557d67a64780 0x557d67a64ba0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c016000 tx=0x7f3b4c024000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e68144cb47958241 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:51.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.629+0000 7f3b50c00700 1 -- 172.21.15.86:0/3412985639 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b4c00b9a0 con 0x557d67a64780 2024-03-23T17:49:51.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.629+0000 7f3b50c00700 1 -- 172.21.15.86:0/3412985639 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b4c01d040 con 0x557d67a64780 2024-03-23T17:49:51.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.629+0000 7f3b50c00700 1 -- 172.21.15.86:0/3412985639 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b4c003b10 con 0x557d67a64780 2024-03-23T17:49:51.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.630+0000 7f3b75eafb80 1 -- 172.21.15.86:0/3412985639 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557d67a64780 msgr2=0x557d67a64ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:51.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.630+0000 7f3b75eafb80 1 --2- 172.21.15.86:0/3412985639 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557d67a64780 0x557d67a64ba0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c016000 tx=0x7f3b4c024000 comp rx=0 tx=0).stop 2024-03-23T17:49:51.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.630+0000 7f3b75eafb80 1 -- 172.21.15.86:0/3412985639 shutdown_connections 2024-03-23T17:49:51.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.630+0000 7f3b75eafb80 1 --2- 172.21.15.86:0/3412985639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d67a65f80 0x557d67a66400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.630+0000 7f3b75eafb80 1 --2- 172.21.15.86:0/3412985639 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557d67a64780 0x557d67a64ba0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.630+0000 7f3b75eafb80 1 --2- 172.21.15.86:0/3412985639 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d67a650e0 0x557d67a659a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.630+0000 7f3b75eafb80 1 -- 172.21.15.86:0/3412985639 >> 172.21.15.86:0/3412985639 conn(0x557d679c3650 msgr2=0x557d67a5f950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:51.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.630+0000 7f3b75eafb80 1 -- 172.21.15.86:0/3412985639 shutdown_connections 2024-03-23T17:49:51.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.630+0000 7f3b75eafb80 1 -- 172.21.15.86:0/3412985639 wait complete. 2024-03-23T17:49:51.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.630+0000 7f3b75eafb80 1 -- start start 2024-03-23T17:49:51.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.631+0000 7f3b75eafb80 1 Processor -- start 2024-03-23T17:49:51.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.631+0000 7f3b75eafb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d67a64780 0x557d67a6e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:51.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.631+0000 7f3b75eafb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d67a650e0 0x557d67a6e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:51.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.631+0000 7f3b75eafb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557d67a65f80 0x557d67a75a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:51.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.631+0000 7f3b75eafb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d67951d30 con 0x557d67a64780 2024-03-23T17:49:51.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.631+0000 7f3b75eafb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d679519d0 con 0x557d67a650e0 2024-03-23T17:49:51.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.631+0000 7f3b75eafb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d67839650 con 0x557d67a65f80 2024-03-23T17:49:51.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.631+0000 7f3b75eafb80 10 client.0 _ll_register_callbacks cb 0x557d6795b420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:49:51.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.631+0000 7f3b5b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d67a64780 0x557d67a6e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:51.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.631+0000 7f3b5b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d67a64780 0x557d67a6e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47170/0 (socket says 172.21.15.86:47170) 2024-03-23T17:49:51.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.631+0000 7f3b5b400700 1 -- 172.21.15.86:0/3919462294 learned_addr learned my addr 172.21.15.86:0/3919462294 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:51.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.631+0000 7f3b5aa00700 1 --2- 172.21.15.86:0/3919462294 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d67a650e0 0x557d67a6e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:51.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.631+0000 7f3b5be00700 1 --2- 172.21.15.86:0/3919462294 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557d67a65f80 0x557d67a75a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:51.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.632+0000 7f3b5b400700 1 -- 172.21.15.86:0/3919462294 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557d67a65f80 msgr2=0x557d67a75a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:49:51.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.632+0000 7f3b5b400700 1 --2- 172.21.15.86:0/3919462294 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557d67a65f80 0x557d67a75a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.632+0000 7f3b5b400700 1 -- 172.21.15.86:0/3919462294 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d67a650e0 msgr2=0x557d67a6e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:49:51.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.632+0000 7f3b5b400700 1 --2- 172.21.15.86:0/3919462294 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d67a650e0 0x557d67a6e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.632+0000 7f3b5b400700 1 -- 172.21.15.86:0/3919462294 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b54003a00 con 0x557d67a64780 2024-03-23T17:49:51.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.632+0000 7f3b5aa00700 1 --2- 172.21.15.86:0/3919462294 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d67a650e0 0x557d67a6e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:49:51.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.632+0000 7f3b5b400700 1 --2- 172.21.15.86:0/3919462294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d67a64780 0x557d67a6e490 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c00fb70 tx=0x7f3b4c02e510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=327a800f46f1aa0f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:51.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.632+0000 7f3b45600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-23T17:49:51.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.632+0000 7f3b45600700 1 -- 172.21.15.86:0/3919462294 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b4c003ca0 con 0x557d67a64780 2024-03-23T17:49:51.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.632+0000 7f3b75eafb80 1 -- 172.21.15.86:0/3919462294 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557d67976330 con 0x557d67a64780 2024-03-23T17:49:51.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.632+0000 7f3b45600700 1 -- 172.21.15.86:0/3919462294 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f3b4c028070 con 0x557d67a64780 2024-03-23T17:49:51.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.632+0000 7f3b45600700 1 -- 172.21.15.86:0/3919462294 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b4c003e00 con 0x557d67a64780 2024-03-23T17:49:51.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.632+0000 7f3b75eafb80 10 client.14958 fetch_fsmap learned FSMap version 27 2024-03-23T17:49:51.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.632+0000 7f3b75eafb80 1 -- 172.21.15.86:0/3919462294 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f3b5c109b10 con 0x557d67a64780 2024-03-23T17:49:51.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.633+0000 7f3b45600700 1 -- 172.21.15.86:0/3919462294 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557d67976330 con 0x557d67a64780 2024-03-23T17:49:51.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.633+0000 7f3b45600700 1 -- 172.21.15.86:0/3919462294 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3b4c042020 con 0x557d67a64780 2024-03-23T17:49:51.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.633+0000 7f3b75eafb80 10 client.14958 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T17:49:51.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.633+0000 7f3b75eafb80 10 client.14958 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-23T17:49:51.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.633+0000 7f3b75eafb80 20 client.14958 populate_metadata read hostname 'smithi086' 2024-03-23T17:49:51.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.633+0000 7f3b75eafb80 1 --2- 172.21.15.86:0/3919462294 >> [v2:172.21.15.92:6836/3967837948,v1:172.21.15.92:6837/3967837948] conn(0x557d67a791a0 0x557d67a7e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:51.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.633+0000 7f3b75eafb80 4 client.14958 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:49:51.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.633+0000 7f3b75eafb80 1 -- 172.21.15.86:0/3919462294 --> [v2:172.21.15.92:6836/3967837948,v1:172.21.15.92:6837/3967837948] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557d679ae6b0 con 0x557d67a791a0 2024-03-23T17:49:51.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.633+0000 7f3b5aa00700 1 --2- 172.21.15.86:0/3919462294 >> [v2:172.21.15.92:6836/3967837948,v1:172.21.15.92:6837/3967837948] conn(0x557d67a791a0 0x557d67a7e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:51.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.634+0000 7f3b45600700 10 client.14958 ms_handle_connect on v2:172.21.15.92:6836/3967837948 2024-03-23T17:49:51.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.634+0000 7f3b5aa00700 1 --2- 172.21.15.86:0/3919462294 >> [v2:172.21.15.92:6836/3967837948,v1:172.21.15.92:6837/3967837948] conn(0x557d67a791a0 0x557d67a7e460 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e6487978271f4309 server_cookie=bec35f659e4d965a in_seq=0 out_seq=0 2024-03-23T17:49:51.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.635+0000 7f3b45600700 1 -- 172.21.15.86:0/3919462294 <== mds.2 v2:172.21.15.92:6836/3967837948 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x557d679ae6b0 con 0x557d67a791a0 2024-03-23T17:49:51.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.635+0000 7f3b45600700 10 client.14958 handle_command_reply: tid=0 2024-03-23T17:49:51.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.635+0000 7f3b75eafb80 1 client.14958 shutdown 2024-03-23T17:49:51.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.635+0000 7f3b75eafb80 1 -- 172.21.15.86:0/3919462294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d67a64780 msgr2=0x557d67a6e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:51.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.635+0000 7f3b75eafb80 1 --2- 172.21.15.86:0/3919462294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d67a64780 0x557d67a6e490 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c00fb70 tx=0x7f3b4c02e510 comp rx=0 tx=0).stop 2024-03-23T17:49:51.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.635+0000 7f3b75eafb80 1 -- 172.21.15.86:0/3919462294 shutdown_connections 2024-03-23T17:49:51.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.635+0000 7f3b75eafb80 1 --2- 172.21.15.86:0/3919462294 >> [v2:172.21.15.92:6836/3967837948,v1:172.21.15.92:6837/3967837948] conn(0x557d67a791a0 0x557d67a7e460 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.635+0000 7f3b75eafb80 1 --2- 172.21.15.86:0/3919462294 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557d67a65f80 0x557d67a75a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.635+0000 7f3b75eafb80 1 --2- 172.21.15.86:0/3919462294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d67a64780 0x557d67a6e490 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.635+0000 7f3b45600700 0 client.14958 ms_handle_reset on v2:172.21.15.92:6836/3967837948 2024-03-23T17:49:51.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.635+0000 7f3b75eafb80 1 --2- 172.21.15.86:0/3919462294 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d67a650e0 0x557d67a6e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.635+0000 7f3b75eafb80 1 -- 172.21.15.86:0/3919462294 >> 172.21.15.86:0/3919462294 conn(0x557d679c3650 msgr2=0x557d67a63120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:51.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.635+0000 7f3b75eafb80 1 -- 172.21.15.86:0/3919462294 shutdown_connections 2024-03-23T17:49:51.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.635+0000 7f3b75eafb80 1 -- 172.21.15.86:0/3919462294 wait complete. 2024-03-23T17:49:51.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.635+0000 7f3b75eafb80 20 client.14958 trim_cache size 0 max 16384 2024-03-23T17:49:51.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.653+0000 7f3b75eafb80 1 Processor -- start 2024-03-23T17:49:51.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.653+0000 7f3b75eafb80 1 -- start start 2024-03-23T17:49:51.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.653+0000 7f3b75eafb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d67a64780 0x557d67a739c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:51.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.653+0000 7f3b75eafb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557d67a650e0 0x557d67a66360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:51.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.653+0000 7f3b75eafb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d67a65f80 0x557d67a6d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:51.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.653+0000 7f3b75eafb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d67a7abf0 con 0x557d67a65f80 2024-03-23T17:49:51.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.653+0000 7f3b75eafb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d67a6f390 con 0x557d67a64780 2024-03-23T17:49:51.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.653+0000 7f3b75eafb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d67976330 con 0x557d67a650e0 2024-03-23T17:49:51.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.653+0000 7f3b5be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d67a65f80 0x557d67a6d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:51.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.653+0000 7f3b5be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d67a65f80 0x557d67a6d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47186/0 (socket says 172.21.15.86:47186) 2024-03-23T17:49:51.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.653+0000 7f3b5be00700 1 -- 172.21.15.86:0/3005336573 learned_addr learned my addr 172.21.15.86:0/3005336573 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:51.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.653+0000 7f3b5aa00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557d67a650e0 0x557d67a66360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:51.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.653+0000 7f3b5b400700 1 --2- 172.21.15.86:0/3005336573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d67a64780 0x557d67a739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:51.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.654+0000 7f3b5be00700 1 -- 172.21.15.86:0/3005336573 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557d67a650e0 msgr2=0x557d67a66360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:49:51.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.654+0000 7f3b5be00700 1 --2- 172.21.15.86:0/3005336573 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557d67a650e0 0x557d67a66360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.654+0000 7f3b5be00700 1 -- 172.21.15.86:0/3005336573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d67a64780 msgr2=0x557d67a739c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:49:51.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.654+0000 7f3b5be00700 1 --2- 172.21.15.86:0/3005336573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d67a64780 0x557d67a739c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.654+0000 7f3b5be00700 1 -- 172.21.15.86:0/3005336573 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d677e3b10 con 0x557d67a65f80 2024-03-23T17:49:51.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.654+0000 7f3b5aa00700 1 --2- 172.21.15.86:0/3005336573 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557d67a650e0 0x557d67a66360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:49:51.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.654+0000 7f3b5be00700 1 --2- 172.21.15.86:0/3005336573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d67a65f80 0x557d67a6d670 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3b5400ea10 tx=0x7f3b54009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4da339e2f807224 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:51.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.654+0000 7f3b51600700 1 -- 172.21.15.86:0/3005336573 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b54017ad0 con 0x557d67a65f80 2024-03-23T17:49:51.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.654+0000 7f3b51600700 1 -- 172.21.15.86:0/3005336573 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b5400f040 con 0x557d67a65f80 2024-03-23T17:49:51.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.654+0000 7f3b51600700 1 -- 172.21.15.86:0/3005336573 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b54007640 con 0x557d67a65f80 2024-03-23T17:49:51.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.655+0000 7f3b75eafb80 1 -- 172.21.15.86:0/3005336573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d67a65f80 msgr2=0x557d67a6d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:51.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.655+0000 7f3b75eafb80 1 --2- 172.21.15.86:0/3005336573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d67a65f80 0x557d67a6d670 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3b5400ea10 tx=0x7f3b54009920 comp rx=0 tx=0).stop 2024-03-23T17:49:51.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.655+0000 7f3b75eafb80 1 -- 172.21.15.86:0/3005336573 shutdown_connections 2024-03-23T17:49:51.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.655+0000 7f3b75eafb80 1 --2- 172.21.15.86:0/3005336573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d67a65f80 0x557d67a6d670 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.655+0000 7f3b75eafb80 1 --2- 172.21.15.86:0/3005336573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d67a64780 0x557d67a739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.655+0000 7f3b75eafb80 1 --2- 172.21.15.86:0/3005336573 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557d67a650e0 0x557d67a66360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.655+0000 7f3b75eafb80 1 -- 172.21.15.86:0/3005336573 >> 172.21.15.86:0/3005336573 conn(0x557d679c3650 msgr2=0x557d67a82480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:51.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.655+0000 7f3b75eafb80 1 -- 172.21.15.86:0/3005336573 shutdown_connections 2024-03-23T17:49:51.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.655+0000 7f3b75eafb80 1 -- 172.21.15.86:0/3005336573 wait complete. 2024-03-23T17:49:51.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.655+0000 7f3b75eafb80 1 -- start start 2024-03-23T17:49:51.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.655+0000 7f3b75eafb80 1 Processor -- start 2024-03-23T17:49:51.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.656+0000 7f3b75eafb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d67a64780 0x557d67a73560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:51.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.656+0000 7f3b75eafb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d67a650e0 0x557d67a71c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:51.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.656+0000 7f3b75eafb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557d67a65f80 0x557d67a721c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:51.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.656+0000 7f3b75eafb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d67a73dc0 con 0x557d67a650e0 2024-03-23T17:49:51.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.656+0000 7f3b75eafb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d67a72730 con 0x557d67a64780 2024-03-23T17:49:51.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.656+0000 7f3b75eafb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d67a728f0 con 0x557d67a65f80 2024-03-23T17:49:51.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.656+0000 7f3b75eafb80 10 client.0 _ll_register_callbacks cb 0x557d6795b420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:49:51.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.656+0000 7f3b5aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d67a650e0 0x557d67a71c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:51.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.656+0000 7f3b5aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d67a650e0 0x557d67a71c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47200/0 (socket says 172.21.15.86:47200) 2024-03-23T17:49:51.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.656+0000 7f3b5aa00700 1 -- 172.21.15.86:0/2651118928 learned_addr learned my addr 172.21.15.86:0/2651118928 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:51.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.656+0000 7f3b5b400700 1 --2- 172.21.15.86:0/2651118928 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d67a64780 0x557d67a73560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:51.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.656+0000 7f3b5be00700 1 --2- 172.21.15.86:0/2651118928 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557d67a65f80 0x557d67a721c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:51.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.657+0000 7f3b5aa00700 1 -- 172.21.15.86:0/2651118928 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557d67a65f80 msgr2=0x557d67a721c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:49:51.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.657+0000 7f3b5aa00700 1 --2- 172.21.15.86:0/2651118928 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557d67a65f80 0x557d67a721c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.657+0000 7f3b5aa00700 1 -- 172.21.15.86:0/2651118928 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d67a64780 msgr2=0x557d67a73560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:49:51.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.657+0000 7f3b5aa00700 1 --2- 172.21.15.86:0/2651118928 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d67a64780 0x557d67a73560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.657+0000 7f3b5aa00700 1 -- 172.21.15.86:0/2651118928 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b5c1a1130 con 0x557d67a650e0 2024-03-23T17:49:51.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.657+0000 7f3b5b400700 1 --2- 172.21.15.86:0/2651118928 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d67a64780 0x557d67a73560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:49:51.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.657+0000 7f3b5aa00700 1 --2- 172.21.15.86:0/2651118928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d67a650e0 0x557d67a71c80 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3b48020000 tx=0x7f3b48096260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1c40bfc74d11425 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:51.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.657+0000 7f3b45600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-23T17:49:51.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.657+0000 7f3b45600700 1 -- 172.21.15.86:0/2651118928 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b48005640 con 0x557d67a650e0 2024-03-23T17:49:51.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.657+0000 7f3b45600700 1 -- 172.21.15.86:0/2651118928 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f3b4809a050 con 0x557d67a650e0 2024-03-23T17:49:51.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.657+0000 7f3b75eafb80 1 -- 172.21.15.86:0/2651118928 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3b4c01d040 con 0x557d67a650e0 2024-03-23T17:49:51.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.657+0000 7f3b45600700 1 -- 172.21.15.86:0/2651118928 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b4809a210 con 0x557d67a650e0 2024-03-23T17:49:51.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.657+0000 7f3b75eafb80 10 client.14964 fetch_fsmap learned FSMap version 27 2024-03-23T17:49:51.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.657+0000 7f3b75eafb80 1 -- 172.21.15.86:0/2651118928 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f3b5400e6c0 con 0x557d67a650e0 2024-03-23T17:49:51.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.657+0000 7f3b45600700 1 -- 172.21.15.86:0/2651118928 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b4c01d040 con 0x557d67a650e0 2024-03-23T17:49:51.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.658+0000 7f3b45600700 1 -- 172.21.15.86:0/2651118928 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3b480a1de0 con 0x557d67a650e0 2024-03-23T17:49:51.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.658+0000 7f3b75eafb80 10 client.14964 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T17:49:51.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.658+0000 7f3b75eafb80 10 client.14964 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-23T17:49:51.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.658+0000 7f3b75eafb80 20 client.14964 populate_metadata read hostname 'smithi086' 2024-03-23T17:49:51.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.658+0000 7f3b75eafb80 1 --2- 172.21.15.86:0/2651118928 >> [v2:172.21.15.92:6836/3967837948,v1:172.21.15.92:6837/3967837948] conn(0x557d67ad0190 0x557d67accae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:51.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.658+0000 7f3b75eafb80 4 client.14964 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-23T17:49:51.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.658+0000 7f3b75eafb80 1 -- 172.21.15.86:0/2651118928 --> [v2:172.21.15.92:6836/3967837948,v1:172.21.15.92:6837/3967837948] -- command(tid 0: {"prefix": "status"}) v1 -- 0x557d67804ca0 con 0x557d67ad0190 2024-03-23T17:49:51.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.658+0000 7f3b5b400700 1 --2- 172.21.15.86:0/2651118928 >> [v2:172.21.15.92:6836/3967837948,v1:172.21.15.92:6837/3967837948] conn(0x557d67ad0190 0x557d67accae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:51.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.659+0000 7f3b5b400700 1 --2- 172.21.15.86:0/2651118928 >> [v2:172.21.15.92:6836/3967837948,v1:172.21.15.92:6837/3967837948] conn(0x557d67ad0190 0x557d67accae0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bcefd85fd9107abe server_cookie=7ca15d5319687cb9 in_seq=0 out_seq=0 2024-03-23T17:49:51.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.659+0000 7f3b45600700 10 client.14964 ms_handle_connect on v2:172.21.15.92:6836/3967837948 2024-03-23T17:49:51.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.659+0000 7f3b45600700 1 -- 172.21.15.86:0/2651118928 <== mds.2 v2:172.21.15.92:6836/3967837948 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x557d67804ca0 con 0x557d67ad0190 2024-03-23T17:49:51.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.659+0000 7f3b45600700 10 client.14964 handle_command_reply: tid=0 2024-03-23T17:49:51.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.659+0000 7f3b75eafb80 1 client.14964 shutdown 2024-03-23T17:49:51.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.659+0000 7f3b75eafb80 1 -- 172.21.15.86:0/2651118928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d67a650e0 msgr2=0x557d67a71c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:51.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.659+0000 7f3b75eafb80 1 --2- 172.21.15.86:0/2651118928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d67a650e0 0x557d67a71c80 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3b48020000 tx=0x7f3b48096260 comp rx=0 tx=0).stop 2024-03-23T17:49:51.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.659+0000 7f3b75eafb80 1 -- 172.21.15.86:0/2651118928 shutdown_connections 2024-03-23T17:49:51.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.659+0000 7f3b75eafb80 1 --2- 172.21.15.86:0/2651118928 >> [v2:172.21.15.92:6836/3967837948,v1:172.21.15.92:6837/3967837948] conn(0x557d67ad0190 0x557d67accae0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.659+0000 7f3b75eafb80 1 --2- 172.21.15.86:0/2651118928 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557d67a65f80 0x557d67a721c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.659+0000 7f3b75eafb80 1 --2- 172.21.15.86:0/2651118928 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d67a64780 0x557d67a73560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.659+0000 7f3b75eafb80 1 --2- 172.21.15.86:0/2651118928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d67a650e0 0x557d67a71c80 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.659+0000 7f3b75eafb80 1 -- 172.21.15.86:0/2651118928 >> 172.21.15.86:0/2651118928 conn(0x557d679c3650 msgr2=0x557d67a6d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:51.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.659+0000 7f3b45600700 0 client.14964 ms_handle_reset on v2:172.21.15.92:6836/3967837948 2024-03-23T17:49:51.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.659+0000 7f3b75eafb80 1 -- 172.21.15.86:0/2651118928 shutdown_connections 2024-03-23T17:49:51.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.659+0000 7f3b75eafb80 1 -- 172.21.15.86:0/2651118928 wait complete. 2024-03-23T17:49:51.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.659+0000 7f3b75eafb80 20 client.14964 trim_cache size 0 max 16384 2024-03-23T17:49:51.666 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-23T17:49:51.666 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "9d76c9e8-e93c-11ee-95cd-87774f69a715", 2024-03-23T17:49:51.666 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 2, 2024-03-23T17:49:51.666 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24518, 2024-03-23T17:49:51.666 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-03-23T17:49:51.666 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-03-23T17:49:51.666 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-03-23T17:49:51.666 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 51.008167114000003, 2024-03-23T17:49:51.666 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 27, 2024-03-23T17:49:51.666 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 82, 2024-03-23T17:49:51.666 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 80, 2024-03-23T17:49:51.666 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 70.774360010999999 2024-03-23T17:49:51.666 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-23T17:49:51.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.660+0000 7f3b62400700 1 -- 172.21.15.86:0/2720079607 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f3b400a11b0 msgr2=0x7f3b400a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:51.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.660+0000 7f3b62400700 1 --2- 172.21.15.86:0/2720079607 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f3b400a11b0 0x7f3b400a3670 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c0098f0 tx=0x7f3b4c019040 comp rx=0 tx=0).stop 2024-03-23T17:49:51.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.660+0000 7f3b62400700 1 -- 172.21.15.86:0/2720079607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b5c101f70 msgr2=0x7f3b5c0731a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:51.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.660+0000 7f3b62400700 1 --2- 172.21.15.86:0/2720079607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b5c101f70 0x7f3b5c0731a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3b4800e9c0 tx=0x7f3b4800ecd0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.660+0000 7f3b62400700 1 -- 172.21.15.86:0/2720079607 shutdown_connections 2024-03-23T17:49:51.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.660+0000 7f3b62400700 1 --2- 172.21.15.86:0/2720079607 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b5c102910 0x7f3b5c0736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.660+0000 7f3b62400700 1 --2- 172.21.15.86:0/2720079607 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3b5c100d80 0x7f3b5c074ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.660+0000 7f3b62400700 1 --2- 172.21.15.86:0/2720079607 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f3b400a11b0 0x7f3b400a3670 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.660+0000 7f3b62400700 1 --2- 172.21.15.86:0/2720079607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b5c101f70 0x7f3b5c0731a0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.660+0000 7f3b62400700 1 -- 172.21.15.86:0/2720079607 >> 172.21.15.86:0/2720079607 conn(0x7f3b5c0fc2f0 msgr2=0x7f3b5c1053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:51.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.660+0000 7f3b62400700 1 -- 172.21.15.86:0/2720079607 shutdown_connections 2024-03-23T17:49:51.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.660+0000 7f3b62400700 1 -- 172.21.15.86:0/2720079607 wait complete. 2024-03-23T17:49:51.679 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-03-23T17:49:51.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.872+0000 7fec7fa00700 1 -- 172.21.15.86:0/1656979186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec78100d80 msgr2=0x7fec781011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:51.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.872+0000 7fec7fa00700 1 --2- 172.21.15.86:0/1656979186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec78100d80 0x7fec781011a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fec6c0099c0 tx=0x7fec6c009cd0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.872+0000 7fec7fa00700 1 -- 172.21.15.86:0/1656979186 shutdown_connections 2024-03-23T17:49:51.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.872+0000 7fec7fa00700 1 --2- 172.21.15.86:0/1656979186 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec78102910 0x7fec781091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.872+0000 7fec7fa00700 1 --2- 172.21.15.86:0/1656979186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec78100d80 0x7fec781011a0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.872+0000 7fec7fa00700 1 --2- 172.21.15.86:0/1656979186 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec78101f70 0x7fec781023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.872+0000 7fec7fa00700 1 -- 172.21.15.86:0/1656979186 >> 172.21.15.86:0/1656979186 conn(0x7fec780fc2f0 msgr2=0x7fec780fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:51.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.872+0000 7fec7fa00700 1 -- 172.21.15.86:0/1656979186 shutdown_connections 2024-03-23T17:49:51.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.872+0000 7fec7fa00700 1 -- 172.21.15.86:0/1656979186 wait complete. 2024-03-23T17:49:51.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.873+0000 7fec7fa00700 1 Processor -- start 2024-03-23T17:49:51.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.873+0000 7fec7fa00700 1 -- start start 2024-03-23T17:49:51.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.873+0000 7fec7fa00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec78100d80 0x7fec78198f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:51.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.873+0000 7fec7fa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec78101f70 0x7fec781994d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:51.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.873+0000 7fec7fa00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec78102910 0x7fec781a04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:51.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.873+0000 7fec7fa00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec781a0ac0 con 0x7fec78101f70 2024-03-23T17:49:51.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.873+0000 7fec7fa00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec781a0c00 con 0x7fec78102910 2024-03-23T17:49:51.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.873+0000 7fec7fa00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec781a0d90 con 0x7fec78100d80 2024-03-23T17:49:51.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.874+0000 7fec77e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec78101f70 0x7fec781994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:51.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.874+0000 7fec77e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec78101f70 0x7fec781994d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47212/0 (socket says 172.21.15.86:47212) 2024-03-23T17:49:51.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.874+0000 7fec77e00700 1 -- 172.21.15.86:0/802974332 learned_addr learned my addr 172.21.15.86:0/802974332 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:51.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.874+0000 7fec7d000700 1 --2- 172.21.15.86:0/802974332 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec78100d80 0x7fec78198f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:51.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.874+0000 7fec7da00700 1 --2- 172.21.15.86:0/802974332 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec78102910 0x7fec781a04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:51.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.874+0000 7fec77e00700 1 -- 172.21.15.86:0/802974332 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec78100d80 msgr2=0x7fec78198f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:51.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.874+0000 7fec77e00700 1 --2- 172.21.15.86:0/802974332 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec78100d80 0x7fec78198f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.874+0000 7fec77e00700 1 -- 172.21.15.86:0/802974332 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec78102910 msgr2=0x7fec781a04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:51.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.874+0000 7fec77e00700 1 --2- 172.21.15.86:0/802974332 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec78102910 0x7fec781a04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.874+0000 7fec77e00700 1 -- 172.21.15.86:0/802974332 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec6c0095f0 con 0x7fec78101f70 2024-03-23T17:49:51.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.874+0000 7fec7da00700 1 --2- 172.21.15.86:0/802974332 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec78102910 0x7fec781a04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:49:51.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.874+0000 7fec77e00700 1 --2- 172.21.15.86:0/802974332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec78101f70 0x7fec781994d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fec6400e9a0 tx=0x7fec6400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:51.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.874+0000 7fec75600700 1 -- 172.21.15.86:0/802974332 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec64009790 con 0x7fec78101f70 2024-03-23T17:49:51.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.874+0000 7fec75600700 1 -- 172.21.15.86:0/802974332 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fec6400cbd0 con 0x7fec78101f70 2024-03-23T17:49:51.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.874+0000 7fec7fa00700 1 -- 172.21.15.86:0/802974332 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec781a1070 con 0x7fec78101f70 2024-03-23T17:49:51.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.874+0000 7fec75600700 1 -- 172.21.15.86:0/802974332 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec64018680 con 0x7fec78101f70 2024-03-23T17:49:51.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.875+0000 7fec7fa00700 1 -- 172.21.15.86:0/802974332 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec781a1580 con 0x7fec78101f70 2024-03-23T17:49:51.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.878+0000 7fec75600700 1 -- 172.21.15.86:0/802974332 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fec6400cd40 con 0x7fec78101f70 2024-03-23T17:49:51.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.880+0000 7fec93600b80 1 Processor -- start 2024-03-23T17:49:51.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.880+0000 7fec75600700 1 --2- 172.21.15.86:0/802974332 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fec600a11b0 0x7fec600a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:51.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.880+0000 7fec93600b80 1 -- start start 2024-03-23T17:49:51.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.880+0000 7fec75600700 1 -- 172.21.15.86:0/802974332 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fec64014070 con 0x7fec78101f70 2024-03-23T17:49:51.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.880+0000 7fec7da00700 1 --2- 172.21.15.86:0/802974332 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fec600a11b0 0x7fec600a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:51.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.880+0000 7fec93600b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577cc17b780 0x5577cc17bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:51.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.880+0000 7fec93600b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5577cc17c0e0 0x5577cc17c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:51.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.880+0000 7fec93600b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5577cc17cf80 0x5577cc17d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:51.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.880+0000 7fec93600b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577cc0d3940 con 0x5577cc17b780 2024-03-23T17:49:51.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.880+0000 7fec93600b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577cc0d1300 con 0x5577cc17cf80 2024-03-23T17:49:51.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.880+0000 7fec93600b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577cc0ccbf0 con 0x5577cc17c0e0 2024-03-23T17:49:51.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.880+0000 7fec7d000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577cc17b780 0x5577cc17bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:51.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.881+0000 7fec7d000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577cc17b780 0x5577cc17bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47226/0 (socket says 172.21.15.86:47226) 2024-03-23T17:49:51.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.881+0000 7fec7d000700 1 -- 172.21.15.86:0/2781826280 learned_addr learned my addr 172.21.15.86:0/2781826280 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:51.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.881+0000 7fec7da00700 1 --2- 172.21.15.86:0/2781826280 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5577cc17cf80 0x5577cc17d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:51.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.881+0000 7fec77e00700 1 --2- 172.21.15.86:0/2781826280 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5577cc17c0e0 0x5577cc17c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:51.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.881+0000 7fec7da00700 1 --2- 172.21.15.86:0/802974332 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fec600a11b0 0x7fec600a3670 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fec700093f0 tx=0x7fec7000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:51.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.881+0000 7fec7d000700 1 -- 172.21.15.86:0/2781826280 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5577cc17c0e0 msgr2=0x5577cc17c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:49:51.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.881+0000 7fec7d000700 1 --2- 172.21.15.86:0/2781826280 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5577cc17c0e0 0x5577cc17c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.881+0000 7fec7d000700 1 -- 172.21.15.86:0/2781826280 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5577cc17cf80 msgr2=0x5577cc17d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:49:51.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.881+0000 7fec7d000700 1 --2- 172.21.15.86:0/2781826280 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5577cc17cf80 0x5577cc17d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.881+0000 7fec7d000700 1 -- 172.21.15.86:0/2781826280 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5577cbf4e360 con 0x5577cc17b780 2024-03-23T17:49:51.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.881+0000 7fec7da00700 1 --2- 172.21.15.86:0/2781826280 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5577cc17cf80 0x5577cc17d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:49:51.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.881+0000 7fec77e00700 1 --2- 172.21.15.86:0/2781826280 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5577cc17c0e0 0x5577cc17c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:49:51.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.881+0000 7fec7d000700 1 --2- 172.21.15.86:0/2781826280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577cc17b780 0x5577cc17bba0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fec6c006010 tx=0x7fec6c005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8b62e669e6ae96f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:51.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.881+0000 7fec6a000700 1 -- 172.21.15.86:0/2781826280 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec6c00ba90 con 0x5577cc17b780 2024-03-23T17:49:51.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.881+0000 7fec6a000700 1 -- 172.21.15.86:0/2781826280 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fec6c01e040 con 0x5577cc17b780 2024-03-23T17:49:51.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.881+0000 7fec6a000700 1 -- 172.21.15.86:0/2781826280 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec6c003bb0 con 0x5577cc17b780 2024-03-23T17:49:51.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.882+0000 7fec93600b80 1 -- 172.21.15.86:0/2781826280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577cc17b780 msgr2=0x5577cc17bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:51.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.882+0000 7fec93600b80 1 --2- 172.21.15.86:0/2781826280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577cc17b780 0x5577cc17bba0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fec6c006010 tx=0x7fec6c005bc0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.882+0000 7fec93600b80 1 -- 172.21.15.86:0/2781826280 shutdown_connections 2024-03-23T17:49:51.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.882+0000 7fec93600b80 1 --2- 172.21.15.86:0/2781826280 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5577cc17cf80 0x5577cc17d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.882+0000 7fec93600b80 1 --2- 172.21.15.86:0/2781826280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577cc17b780 0x5577cc17bba0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.882+0000 7fec93600b80 1 --2- 172.21.15.86:0/2781826280 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5577cc17c0e0 0x5577cc17c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.882+0000 7fec93600b80 1 -- 172.21.15.86:0/2781826280 >> 172.21.15.86:0/2781826280 conn(0x5577cc0da650 msgr2=0x5577cc176950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:51.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.882+0000 7fec93600b80 1 -- 172.21.15.86:0/2781826280 shutdown_connections 2024-03-23T17:49:51.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.882+0000 7fec93600b80 1 -- 172.21.15.86:0/2781826280 wait complete. 2024-03-23T17:49:51.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.883+0000 7fec93600b80 1 -- start start 2024-03-23T17:49:51.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.883+0000 7fec93600b80 1 Processor -- start 2024-03-23T17:49:51.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.883+0000 7fec93600b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577cc17b780 0x5577cc1855f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:51.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.883+0000 7fec93600b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5577cc17c0e0 0x5577cc185b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:51.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.883+0000 7fec93600b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5577cc17cf80 0x5577cc18cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:51.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.883+0000 7fec93600b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577cc068d30 con 0x5577cc17b780 2024-03-23T17:49:51.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.883+0000 7fec93600b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577cc0689d0 con 0x5577cc17cf80 2024-03-23T17:49:51.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.883+0000 7fec93600b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577cbf50650 con 0x5577cc17c0e0 2024-03-23T17:49:51.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.884+0000 7fec93600b80 10 client.0 _ll_register_callbacks cb 0x5577cc072420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:49:51.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.884+0000 7fec7d000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577cc17b780 0x5577cc1855f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:51.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.884+0000 7fec7d000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577cc17b780 0x5577cc1855f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47240/0 (socket says 172.21.15.86:47240) 2024-03-23T17:49:51.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.884+0000 7fec7d000700 1 -- 172.21.15.86:0/2697091049 learned_addr learned my addr 172.21.15.86:0/2697091049 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:51.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.884+0000 7fec7da00700 1 --2- 172.21.15.86:0/2697091049 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5577cc17cf80 0x5577cc18cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:51.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.884+0000 7fec77e00700 1 --2- 172.21.15.86:0/2697091049 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5577cc17c0e0 0x5577cc185b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:51.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.884+0000 7fec7d000700 1 -- 172.21.15.86:0/2697091049 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5577cc17c0e0 msgr2=0x5577cc185b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:49:51.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.884+0000 7fec7d000700 1 --2- 172.21.15.86:0/2697091049 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5577cc17c0e0 0x5577cc185b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.884+0000 7fec7d000700 1 -- 172.21.15.86:0/2697091049 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5577cc17cf80 msgr2=0x5577cc18cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:49:51.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.884+0000 7fec7d000700 1 --2- 172.21.15.86:0/2697091049 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5577cc17cf80 0x5577cc18cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.884+0000 7fec7d000700 1 -- 172.21.15.86:0/2697091049 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec78109b10 con 0x5577cc17b780 2024-03-23T17:49:51.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.884+0000 7fec77e00700 1 --2- 172.21.15.86:0/2697091049 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5577cc17c0e0 0x5577cc185b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:49:51.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.884+0000 7fec7d000700 1 --2- 172.21.15.86:0/2697091049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577cc17b780 0x5577cc1855f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fec6c0058e0 tx=0x7fec6c005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db7b0f776a81dc4 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:51.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.884+0000 7fec5ea00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-23T17:49:51.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.884+0000 7fec5ea00700 1 -- 172.21.15.86:0/2697091049 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec6c003d60 con 0x5577cc17b780 2024-03-23T17:49:51.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.884+0000 7fec5ea00700 1 -- 172.21.15.86:0/2697091049 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fec6c0050d0 con 0x5577cc17b780 2024-03-23T17:49:51.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.885+0000 7fec93600b80 1 -- 172.21.15.86:0/2697091049 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5577cc08d330 con 0x5577cc17b780 2024-03-23T17:49:51.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.885+0000 7fec5ea00700 1 -- 172.21.15.86:0/2697091049 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec6c005290 con 0x5577cc17b780 2024-03-23T17:49:51.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.885+0000 7fec93600b80 10 client.14976 fetch_fsmap learned FSMap version 27 2024-03-23T17:49:51.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.885+0000 7fec93600b80 1 -- 172.21.15.86:0/2697091049 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5577cbefab10 con 0x5577cc17b780 2024-03-23T17:49:51.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.885+0000 7fec5ea00700 1 -- 172.21.15.86:0/2697091049 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5577cc08d330 con 0x5577cc17b780 2024-03-23T17:49:51.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.885+0000 7fec5ea00700 1 -- 172.21.15.86:0/2697091049 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fec6c0223f0 con 0x5577cc17b780 2024-03-23T17:49:51.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.885+0000 7fec93600b80 10 client.14976 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T17:49:51.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.885+0000 7fec93600b80 10 client.14976 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-23T17:49:51.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.885+0000 7fec93600b80 20 client.14976 populate_metadata read hostname 'smithi086' 2024-03-23T17:49:51.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.885+0000 7fec93600b80 1 --2- 172.21.15.86:0/2697091049 >> [v2:172.21.15.152:6838/1100723992,v1:172.21.15.152:6839/1100723992] conn(0x5577cc190320 0x5577cc195640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:51.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.885+0000 7fec93600b80 4 client.14976 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:49:51.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.885+0000 7fec93600b80 1 -- 172.21.15.86:0/2697091049 --> [v2:172.21.15.152:6838/1100723992,v1:172.21.15.152:6839/1100723992] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5577cc0c56b0 con 0x5577cc190320 2024-03-23T17:49:51.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.886+0000 7fec77e00700 1 --2- 172.21.15.86:0/2697091049 >> [v2:172.21.15.152:6838/1100723992,v1:172.21.15.152:6839/1100723992] conn(0x5577cc190320 0x5577cc195640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:51.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.886+0000 7fec5ea00700 10 client.14976 ms_handle_connect on v2:172.21.15.152:6838/1100723992 2024-03-23T17:49:51.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.886+0000 7fec77e00700 1 --2- 172.21.15.86:0/2697091049 >> [v2:172.21.15.152:6838/1100723992,v1:172.21.15.152:6839/1100723992] conn(0x5577cc190320 0x5577cc195640 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7d5ea1fc2a40d155 server_cookie=72517277391ffa8d in_seq=0 out_seq=0 2024-03-23T17:49:51.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.887+0000 7fec5ea00700 1 -- 172.21.15.86:0/2697091049 <== mds.1 v2:172.21.15.152:6838/1100723992 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5577cc0c56b0 con 0x5577cc190320 2024-03-23T17:49:51.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.887+0000 7fec5ea00700 10 client.14976 handle_command_reply: tid=0 2024-03-23T17:49:51.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.887+0000 7fec93600b80 1 client.14976 shutdown 2024-03-23T17:49:51.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.887+0000 7fec93600b80 1 -- 172.21.15.86:0/2697091049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577cc17b780 msgr2=0x5577cc1855f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:51.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.887+0000 7fec93600b80 1 --2- 172.21.15.86:0/2697091049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577cc17b780 0x5577cc1855f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fec6c0058e0 tx=0x7fec6c005f00 comp rx=0 tx=0).stop 2024-03-23T17:49:51.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.887+0000 7fec93600b80 1 -- 172.21.15.86:0/2697091049 shutdown_connections 2024-03-23T17:49:51.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.887+0000 7fec93600b80 1 --2- 172.21.15.86:0/2697091049 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5577cc17cf80 0x5577cc18cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.887+0000 7fec93600b80 1 --2- 172.21.15.86:0/2697091049 >> [v2:172.21.15.152:6838/1100723992,v1:172.21.15.152:6839/1100723992] conn(0x5577cc190320 0x5577cc195640 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.887+0000 7fec93600b80 1 --2- 172.21.15.86:0/2697091049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577cc17b780 0x5577cc1855f0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.888+0000 7fec93600b80 1 --2- 172.21.15.86:0/2697091049 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5577cc17c0e0 0x5577cc185b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.888+0000 7fec5ea00700 0 client.14976 ms_handle_reset on v2:172.21.15.152:6838/1100723992 2024-03-23T17:49:51.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.888+0000 7fec93600b80 1 -- 172.21.15.86:0/2697091049 >> 172.21.15.86:0/2697091049 conn(0x5577cc0da650 msgr2=0x5577cc17a120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:51.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.888+0000 7fec93600b80 1 -- 172.21.15.86:0/2697091049 shutdown_connections 2024-03-23T17:49:51.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.888+0000 7fec93600b80 1 -- 172.21.15.86:0/2697091049 wait complete. 2024-03-23T17:49:51.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.888+0000 7fec93600b80 20 client.14976 trim_cache size 0 max 16384 2024-03-23T17:49:51.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.905+0000 7fec93600b80 1 Processor -- start 2024-03-23T17:49:51.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.905+0000 7fec93600b80 1 -- start start 2024-03-23T17:49:51.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.906+0000 7fec93600b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5577cc17c0e0 0x5577cc18ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:51.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.906+0000 7fec93600b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577cc17cf80 0x5577cc17d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:51.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.906+0000 7fec93600b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5577cc175ec0 0x5577cc180190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:51.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.906+0000 7fec93600b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577cc18e880 con 0x5577cc17cf80 2024-03-23T17:49:51.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.906+0000 7fec93600b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577cc1864f0 con 0x5577cc17c0e0 2024-03-23T17:49:51.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.906+0000 7fec93600b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577cc08d330 con 0x5577cc175ec0 2024-03-23T17:49:51.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.906+0000 7fec77e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577cc17cf80 0x5577cc17d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:51.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.906+0000 7fec77e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577cc17cf80 0x5577cc17d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47254/0 (socket says 172.21.15.86:47254) 2024-03-23T17:49:51.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.906+0000 7fec77e00700 1 -- 172.21.15.86:0/1053363584 learned_addr learned my addr 172.21.15.86:0/1053363584 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:51.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.906+0000 7fec7da00700 1 --2- 172.21.15.86:0/1053363584 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5577cc175ec0 0x5577cc180190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:51.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.906+0000 7fec7d000700 1 --2- 172.21.15.86:0/1053363584 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5577cc17c0e0 0x5577cc18ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:51.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.906+0000 7fec77e00700 1 -- 172.21.15.86:0/1053363584 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5577cc175ec0 msgr2=0x5577cc180190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:49:51.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.906+0000 7fec77e00700 1 --2- 172.21.15.86:0/1053363584 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5577cc175ec0 0x5577cc180190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.906+0000 7fec77e00700 1 -- 172.21.15.86:0/1053363584 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5577cc17c0e0 msgr2=0x5577cc18ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:49:51.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.906+0000 7fec77e00700 1 --2- 172.21.15.86:0/1053363584 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5577cc17c0e0 0x5577cc18ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.906+0000 7fec77e00700 1 -- 172.21.15.86:0/1053363584 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5577cc092e50 con 0x5577cc17cf80 2024-03-23T17:49:51.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.906+0000 7fec77e00700 1 --2- 172.21.15.86:0/1053363584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577cc17cf80 0x5577cc17d360 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fec6400e970 tx=0x7fec640cc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=534f1f5e11ed8739 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:51.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.907+0000 7fec6aa00700 1 -- 172.21.15.86:0/1053363584 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec640b1070 con 0x5577cc17cf80 2024-03-23T17:49:51.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.907+0000 7fec6aa00700 1 -- 172.21.15.86:0/1053363584 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fec640ade90 con 0x5577cc17cf80 2024-03-23T17:49:51.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.907+0000 7fec6aa00700 1 -- 172.21.15.86:0/1053363584 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec640994b0 con 0x5577cc17cf80 2024-03-23T17:49:51.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.907+0000 7fec93600b80 1 -- 172.21.15.86:0/1053363584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577cc17cf80 msgr2=0x5577cc17d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:51.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.907+0000 7fec93600b80 1 --2- 172.21.15.86:0/1053363584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577cc17cf80 0x5577cc17d360 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fec6400e970 tx=0x7fec640cc040 comp rx=0 tx=0).stop 2024-03-23T17:49:51.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.907+0000 7fec93600b80 1 -- 172.21.15.86:0/1053363584 shutdown_connections 2024-03-23T17:49:51.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.907+0000 7fec93600b80 1 --2- 172.21.15.86:0/1053363584 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5577cc175ec0 0x5577cc180190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.907+0000 7fec93600b80 1 --2- 172.21.15.86:0/1053363584 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5577cc17c0e0 0x5577cc18ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.907+0000 7fec93600b80 1 --2- 172.21.15.86:0/1053363584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577cc17cf80 0x5577cc17d360 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.907+0000 7fec93600b80 1 -- 172.21.15.86:0/1053363584 >> 172.21.15.86:0/1053363584 conn(0x5577cc0da650 msgr2=0x5577cc184b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:51.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.907+0000 7fec93600b80 1 -- 172.21.15.86:0/1053363584 shutdown_connections 2024-03-23T17:49:51.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.907+0000 7fec93600b80 1 -- 172.21.15.86:0/1053363584 wait complete. 2024-03-23T17:49:51.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.908+0000 7fec93600b80 1 -- start start 2024-03-23T17:49:51.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.908+0000 7fec93600b80 1 Processor -- start 2024-03-23T17:49:51.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.908+0000 7fec93600b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577cc175ec0 0x5577cc189d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:51.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.908+0000 7fec93600b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5577cc17c0e0 0x5577cc18a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:51.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.908+0000 7fec93600b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5577cc17cf80 0x5577cc1e2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:51.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.908+0000 7fec93600b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577cc1e2e60 con 0x5577cc175ec0 2024-03-23T17:49:51.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.908+0000 7fec93600b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577cc1e2fa0 con 0x5577cc17c0e0 2024-03-23T17:49:51.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.908+0000 7fec93600b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577cc1e3160 con 0x5577cc17cf80 2024-03-23T17:49:51.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.908+0000 7fec93600b80 10 client.0 _ll_register_callbacks cb 0x5577cc072420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:49:51.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.908+0000 7fec7d000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577cc175ec0 0x5577cc189d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:51.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.909+0000 7fec7d000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577cc175ec0 0x5577cc189d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47258/0 (socket says 172.21.15.86:47258) 2024-03-23T17:49:51.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.909+0000 7fec7d000700 1 -- 172.21.15.86:0/3958577564 learned_addr learned my addr 172.21.15.86:0/3958577564 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:51.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.909+0000 7fec7da00700 1 --2- 172.21.15.86:0/3958577564 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5577cc17cf80 0x5577cc1e2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:51.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.909+0000 7fec77e00700 1 --2- 172.21.15.86:0/3958577564 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5577cc17c0e0 0x5577cc18a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:51.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.909+0000 7fec7d000700 1 -- 172.21.15.86:0/3958577564 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5577cc17cf80 msgr2=0x5577cc1e2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:49:51.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.909+0000 7fec7d000700 1 --2- 172.21.15.86:0/3958577564 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5577cc17cf80 0x5577cc1e2920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.909+0000 7fec7d000700 1 -- 172.21.15.86:0/3958577564 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5577cc17c0e0 msgr2=0x5577cc18a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:49:51.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.909+0000 7fec7d000700 1 --2- 172.21.15.86:0/3958577564 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5577cc17c0e0 0x5577cc18a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.909+0000 7fec7d000700 1 -- 172.21.15.86:0/3958577564 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec781a1580 con 0x5577cc175ec0 2024-03-23T17:49:51.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.909+0000 7fec77e00700 1 --2- 172.21.15.86:0/3958577564 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5577cc17c0e0 0x5577cc18a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:49:51.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.909+0000 7fec7d000700 1 --2- 172.21.15.86:0/3958577564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577cc175ec0 0x5577cc189d10 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fec6c022780 tx=0x7fec6c00bf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2b674ea06944e25 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:51.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.909+0000 7fec5ea00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-23T17:49:51.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.909+0000 7fec5ea00700 1 -- 172.21.15.86:0/3958577564 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec6c003d60 con 0x5577cc175ec0 2024-03-23T17:49:51.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.909+0000 7fec5ea00700 1 -- 172.21.15.86:0/3958577564 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fec6c000e40 con 0x5577cc175ec0 2024-03-23T17:49:51.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.909+0000 7fec93600b80 1 -- 172.21.15.86:0/3958577564 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fec6c01e040 con 0x5577cc175ec0 2024-03-23T17:49:51.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.910+0000 7fec5ea00700 1 -- 172.21.15.86:0/3958577564 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec6c0037d0 con 0x5577cc175ec0 2024-03-23T17:49:51.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.910+0000 7fec93600b80 10 client.14982 fetch_fsmap learned FSMap version 27 2024-03-23T17:49:51.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.910+0000 7fec93600b80 1 -- 172.21.15.86:0/3958577564 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5577cbf52c80 con 0x5577cc175ec0 2024-03-23T17:49:51.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.910+0000 7fec5ea00700 1 -- 172.21.15.86:0/3958577564 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fec6c01e040 con 0x5577cc175ec0 2024-03-23T17:49:51.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.910+0000 7fec5ea00700 1 -- 172.21.15.86:0/3958577564 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fec6c0163a0 con 0x5577cc175ec0 2024-03-23T17:49:51.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.910+0000 7fec93600b80 10 client.14982 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T17:49:51.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.910+0000 7fec93600b80 10 client.14982 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-23T17:49:51.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.910+0000 7fec93600b80 20 client.14982 populate_metadata read hostname 'smithi086' 2024-03-23T17:49:51.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.910+0000 7fec93600b80 1 --2- 172.21.15.86:0/3958577564 >> [v2:172.21.15.152:6838/1100723992,v1:172.21.15.152:6839/1100723992] conn(0x5577cc1e7a80 0x5577cc1e4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:51.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.910+0000 7fec93600b80 4 client.14982 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-23T17:49:51.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.910+0000 7fec93600b80 1 -- 172.21.15.86:0/3958577564 --> [v2:172.21.15.152:6838/1100723992,v1:172.21.15.152:6839/1100723992] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5577cbf1bca0 con 0x5577cc1e7a80 2024-03-23T17:49:51.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.910+0000 7fec77e00700 1 --2- 172.21.15.86:0/3958577564 >> [v2:172.21.15.152:6838/1100723992,v1:172.21.15.152:6839/1100723992] conn(0x5577cc1e7a80 0x5577cc1e4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:51.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.911+0000 7fec77e00700 1 --2- 172.21.15.86:0/3958577564 >> [v2:172.21.15.152:6838/1100723992,v1:172.21.15.152:6839/1100723992] conn(0x5577cc1e7a80 0x5577cc1e4430 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b3b8fb819c2ebd79 server_cookie=947303d2c43677cf in_seq=0 out_seq=0 2024-03-23T17:49:51.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.911+0000 7fec5ea00700 10 client.14982 ms_handle_connect on v2:172.21.15.152:6838/1100723992 2024-03-23T17:49:51.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.911+0000 7fec5ea00700 1 -- 172.21.15.86:0/3958577564 <== mds.1 v2:172.21.15.152:6838/1100723992 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x5577cbf1bca0 con 0x5577cc1e7a80 2024-03-23T17:49:51.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.911+0000 7fec5ea00700 10 client.14982 handle_command_reply: tid=0 2024-03-23T17:49:51.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.911+0000 7fec93600b80 1 client.14982 shutdown 2024-03-23T17:49:51.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.911+0000 7fec93600b80 1 -- 172.21.15.86:0/3958577564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577cc175ec0 msgr2=0x5577cc189d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:51.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.911+0000 7fec93600b80 1 --2- 172.21.15.86:0/3958577564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577cc175ec0 0x5577cc189d10 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fec6c022780 tx=0x7fec6c00bf00 comp rx=0 tx=0).stop 2024-03-23T17:49:51.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.911+0000 7fec93600b80 1 -- 172.21.15.86:0/3958577564 shutdown_connections 2024-03-23T17:49:51.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.911+0000 7fec93600b80 1 --2- 172.21.15.86:0/3958577564 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5577cc17cf80 0x5577cc1e2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.911+0000 7fec93600b80 1 --2- 172.21.15.86:0/3958577564 >> [v2:172.21.15.152:6838/1100723992,v1:172.21.15.152:6839/1100723992] conn(0x5577cc1e7a80 0x5577cc1e4430 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.911+0000 7fec93600b80 1 --2- 172.21.15.86:0/3958577564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577cc175ec0 0x5577cc189d10 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.911+0000 7fec93600b80 1 --2- 172.21.15.86:0/3958577564 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5577cc17c0e0 0x5577cc18a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.911+0000 7fec5ea00700 0 client.14982 ms_handle_reset on v2:172.21.15.152:6838/1100723992 2024-03-23T17:49:51.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.911+0000 7fec93600b80 1 -- 172.21.15.86:0/3958577564 >> 172.21.15.86:0/3958577564 conn(0x5577cc0da650 msgr2=0x5577cc180190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:51.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.912+0000 7fec93600b80 1 -- 172.21.15.86:0/3958577564 shutdown_connections 2024-03-23T17:49:51.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.912+0000 7fec93600b80 1 -- 172.21.15.86:0/3958577564 wait complete. 2024-03-23T17:49:51.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.912+0000 7fec93600b80 20 client.14982 trim_cache size 0 max 16384 2024-03-23T17:49:51.960 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-23T17:49:51.960 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "9d76c9e8-e93c-11ee-95cd-87774f69a715", 2024-03-23T17:49:51.960 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 1, 2024-03-23T17:49:51.961 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24553, 2024-03-23T17:49:51.961 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-03-23T17:49:51.961 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-03-23T17:49:51.961 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-03-23T17:49:51.961 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 52.250099644999999, 2024-03-23T17:49:51.961 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 27, 2024-03-23T17:49:51.961 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 82, 2024-03-23T17:49:51.962 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 80, 2024-03-23T17:49:51.962 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 69.191085419999993 2024-03-23T17:49:51.962 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-23T17:49:51.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.912+0000 7fec7fa00700 1 -- 172.21.15.86:0/802974332 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fec600a11b0 msgr2=0x7fec600a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:51.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.912+0000 7fec7fa00700 1 --2- 172.21.15.86:0/802974332 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fec600a11b0 0x7fec600a3670 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fec700093f0 tx=0x7fec7000b040 comp rx=0 tx=0).stop 2024-03-23T17:49:51.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.912+0000 7fec7fa00700 1 -- 172.21.15.86:0/802974332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec78101f70 msgr2=0x7fec781994d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:51.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.912+0000 7fec7fa00700 1 --2- 172.21.15.86:0/802974332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec78101f70 0x7fec781994d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fec6400e9a0 tx=0x7fec6400ecb0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.912+0000 7fec7fa00700 1 -- 172.21.15.86:0/802974332 shutdown_connections 2024-03-23T17:49:51.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.912+0000 7fec7fa00700 1 --2- 172.21.15.86:0/802974332 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec78102910 0x7fec781a04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.912+0000 7fec7fa00700 1 --2- 172.21.15.86:0/802974332 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec78100d80 0x7fec78198f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.912+0000 7fec7fa00700 1 --2- 172.21.15.86:0/802974332 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fec600a11b0 0x7fec600a3670 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.912+0000 7fec7fa00700 1 --2- 172.21.15.86:0/802974332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec78101f70 0x7fec781994d0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:51.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.912+0000 7fec7fa00700 1 -- 172.21.15.86:0/802974332 >> 172.21.15.86:0/802974332 conn(0x7fec780fc2f0 msgr2=0x7fec78072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:51.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.912+0000 7fec7fa00700 1 -- 172.21.15.86:0/802974332 shutdown_connections 2024-03-23T17:49:51.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:51.913+0000 7fec7fa00700 1 -- 172.21.15.86:0/802974332 wait complete. 2024-03-23T17:49:51.968 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-23T17:49:52.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:51 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/536892461' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T17:49:52.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.161+0000 7fa1f3000700 1 -- 172.21.15.86:0/1064630279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1ec100d80 msgr2=0x7fa1ec1011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:52.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.161+0000 7fa1f3000700 1 --2- 172.21.15.86:0/1064630279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1ec100d80 0x7fa1ec1011a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc0099c0 tx=0x7fa1dc009cd0 comp rx=0 tx=0).stop 2024-03-23T17:49:52.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.161+0000 7fa1f3000700 1 -- 172.21.15.86:0/1064630279 shutdown_connections 2024-03-23T17:49:52.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.161+0000 7fa1f3000700 1 --2- 172.21.15.86:0/1064630279 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa1ec102910 0x7fa1ec1091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:52.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.161+0000 7fa1f3000700 1 --2- 172.21.15.86:0/1064630279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1ec100d80 0x7fa1ec1011a0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:52.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.161+0000 7fa1f3000700 1 --2- 172.21.15.86:0/1064630279 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa1ec101f70 0x7fa1ec1023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:52.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.161+0000 7fa1f3000700 1 -- 172.21.15.86:0/1064630279 >> 172.21.15.86:0/1064630279 conn(0x7fa1ec0fc2f0 msgr2=0x7fa1ec0fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:52.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.161+0000 7fa1f3000700 1 -- 172.21.15.86:0/1064630279 shutdown_connections 2024-03-23T17:49:52.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.161+0000 7fa1f3000700 1 -- 172.21.15.86:0/1064630279 wait complete. 2024-03-23T17:49:52.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.162+0000 7fa1f3000700 1 Processor -- start 2024-03-23T17:49:52.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.162+0000 7fa1f3000700 1 -- start start 2024-03-23T17:49:52.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.163+0000 7fa1f3000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1ec100d80 0x7fa1ec074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:52.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.163+0000 7fa1f3000700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa1ec101f70 0x7fa1ec0731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:52.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.163+0000 7fa1f3000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa1ec102910 0x7fa1ec0736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:52.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.163+0000 7fa1f3000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1ec073e80 con 0x7fa1ec100d80 2024-03-23T17:49:52.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.163+0000 7fa1f3000700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1ec073ff0 con 0x7fa1ec102910 2024-03-23T17:49:52.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.163+0000 7fa1f3000700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1ec1a0da0 con 0x7fa1ec101f70 2024-03-23T17:49:52.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.163+0000 7fa1ebe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1ec100d80 0x7fa1ec074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:52.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.163+0000 7fa1ebe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1ec100d80 0x7fa1ec074b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47266/0 (socket says 172.21.15.86:47266) 2024-03-23T17:49:52.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.163+0000 7fa1ebe00700 1 -- 172.21.15.86:0/867387128 learned_addr learned my addr 172.21.15.86:0/867387128 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:52.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.163+0000 7fa1f1000700 1 --2- 172.21.15.86:0/867387128 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa1ec102910 0x7fa1ec0736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:52.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.163+0000 7fa1eb400700 1 --2- 172.21.15.86:0/867387128 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa1ec101f70 0x7fa1ec0731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:52.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.163+0000 7fa1ebe00700 1 -- 172.21.15.86:0/867387128 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa1ec101f70 msgr2=0x7fa1ec0731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:52.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.163+0000 7fa1ebe00700 1 --2- 172.21.15.86:0/867387128 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa1ec101f70 0x7fa1ec0731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:52.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.163+0000 7fa1ebe00700 1 -- 172.21.15.86:0/867387128 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa1ec102910 msgr2=0x7fa1ec0736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:52.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.163+0000 7fa1ebe00700 1 --2- 172.21.15.86:0/867387128 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa1ec102910 0x7fa1ec0736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:52.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.163+0000 7fa1ebe00700 1 -- 172.21.15.86:0/867387128 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1e4009c40 con 0x7fa1ec100d80 2024-03-23T17:49:52.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.163+0000 7fa1f1000700 1 --2- 172.21.15.86:0/867387128 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa1ec102910 0x7fa1ec0736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:49:52.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.164+0000 7fa1ebe00700 1 --2- 172.21.15.86:0/867387128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1ec100d80 0x7fa1ec074b10 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc006010 tx=0x7fa1dc00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:52.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.164+0000 7fa1e8c00700 1 -- 172.21.15.86:0/867387128 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1dc01d070 con 0x7fa1ec100d80 2024-03-23T17:49:52.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.164+0000 7fa1e8c00700 1 -- 172.21.15.86:0/867387128 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa1dc005020 con 0x7fa1ec100d80 2024-03-23T17:49:52.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.164+0000 7fa1e8c00700 1 -- 172.21.15.86:0/867387128 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1dc004a50 con 0x7fa1ec100d80 2024-03-23T17:49:52.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.164+0000 7fa1f3000700 1 -- 172.21.15.86:0/867387128 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1dc0095f0 con 0x7fa1ec100d80 2024-03-23T17:49:52.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.164+0000 7fa1f3000700 1 -- 172.21.15.86:0/867387128 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1ec1a11b0 con 0x7fa1ec100d80 2024-03-23T17:49:52.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.167+0000 7fa1e8c00700 1 -- 172.21.15.86:0/867387128 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fa1dc005190 con 0x7fa1ec100d80 2024-03-23T17:49:52.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.168+0000 7fa1e8c00700 1 --2- 172.21.15.86:0/867387128 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fa1d00a1230 0x7fa1d00a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:52.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.168+0000 7fa206a85b80 1 Processor -- start 2024-03-23T17:49:52.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.168+0000 7fa1e8c00700 1 -- 172.21.15.86:0/867387128 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fa1dc0cd400 con 0x7fa1ec100d80 2024-03-23T17:49:52.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.168+0000 7fa206a85b80 1 -- start start 2024-03-23T17:49:52.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.168+0000 7fa1eb400700 1 --2- 172.21.15.86:0/867387128 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fa1d00a1230 0x7fa1d00a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:52.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.169+0000 7fa206a85b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4bb85a780 0x55c4bb85aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:52.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.169+0000 7fa206a85b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c4bb85b0e0 0x55c4bb85b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:52.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.169+0000 7fa206a85b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4bb85bf80 0x55c4bb85c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:52.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.169+0000 7fa206a85b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4bb7b2940 con 0x55c4bb85a780 2024-03-23T17:49:52.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.169+0000 7fa206a85b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4bb7b0300 con 0x55c4bb85bf80 2024-03-23T17:49:52.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.169+0000 7fa206a85b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4bb7abbf0 con 0x55c4bb85b0e0 2024-03-23T17:49:52.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.169+0000 7fa1ebe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4bb85a780 0x55c4bb85aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:52.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.169+0000 7fa1ebe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4bb85a780 0x55c4bb85aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47274/0 (socket says 172.21.15.86:47274) 2024-03-23T17:49:52.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.169+0000 7fa1eb400700 1 --2- 172.21.15.86:0/867387128 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fa1d00a1230 0x7fa1d00a36f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa1d8009c60 tx=0x7fa1d8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:52.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.169+0000 7fa1ebe00700 1 -- 172.21.15.86:0/2262740109 learned_addr learned my addr 172.21.15.86:0/2262740109 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:52.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.169+0000 7fa1f1000700 1 --2- 172.21.15.86:0/2262740109 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4bb85bf80 0x55c4bb85c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:52.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.169+0000 7fa1eb400700 1 --2- 172.21.15.86:0/2262740109 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c4bb85b0e0 0x55c4bb85b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:52.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.169+0000 7fa1ebe00700 1 -- 172.21.15.86:0/2262740109 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c4bb85b0e0 msgr2=0x55c4bb85b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:49:52.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.169+0000 7fa1ebe00700 1 --2- 172.21.15.86:0/2262740109 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c4bb85b0e0 0x55c4bb85b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:52.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.169+0000 7fa1ebe00700 1 -- 172.21.15.86:0/2262740109 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4bb85bf80 msgr2=0x55c4bb85c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:49:52.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.169+0000 7fa1ebe00700 1 --2- 172.21.15.86:0/2262740109 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4bb85bf80 0x55c4bb85c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:52.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.169+0000 7fa1ebe00700 1 -- 172.21.15.86:0/2262740109 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c4bb62d360 con 0x55c4bb85a780 2024-03-23T17:49:52.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.170+0000 7fa1ebe00700 1 --2- 172.21.15.86:0/2262740109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4bb85a780 0x55c4bb85aba0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc097220 tx=0x7fa1dc009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fde4499c35ac334e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:52.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.170+0000 7fa1e1600700 1 -- 172.21.15.86:0/2262740109 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1dc09a7b0 con 0x55c4bb85a780 2024-03-23T17:49:52.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.170+0000 7fa1e1600700 1 -- 172.21.15.86:0/2262740109 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa1dc09e5e0 con 0x55c4bb85a780 2024-03-23T17:49:52.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.170+0000 7fa1e1600700 1 -- 172.21.15.86:0/2262740109 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1dc09aa10 con 0x55c4bb85a780 2024-03-23T17:49:52.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.171+0000 7fa206a85b80 1 -- 172.21.15.86:0/2262740109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4bb85a780 msgr2=0x55c4bb85aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:52.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.171+0000 7fa206a85b80 1 --2- 172.21.15.86:0/2262740109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4bb85a780 0x55c4bb85aba0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc097220 tx=0x7fa1dc009310 comp rx=0 tx=0).stop 2024-03-23T17:49:52.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.171+0000 7fa206a85b80 1 -- 172.21.15.86:0/2262740109 shutdown_connections 2024-03-23T17:49:52.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.171+0000 7fa206a85b80 1 --2- 172.21.15.86:0/2262740109 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4bb85bf80 0x55c4bb85c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:52.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.171+0000 7fa206a85b80 1 --2- 172.21.15.86:0/2262740109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4bb85a780 0x55c4bb85aba0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:52.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.171+0000 7fa206a85b80 1 --2- 172.21.15.86:0/2262740109 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c4bb85b0e0 0x55c4bb85b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:52.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.171+0000 7fa206a85b80 1 -- 172.21.15.86:0/2262740109 >> 172.21.15.86:0/2262740109 conn(0x55c4bb7b9650 msgr2=0x55c4bb855950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:52.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.171+0000 7fa206a85b80 1 -- 172.21.15.86:0/2262740109 shutdown_connections 2024-03-23T17:49:52.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.171+0000 7fa206a85b80 1 -- 172.21.15.86:0/2262740109 wait complete. 2024-03-23T17:49:52.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.171+0000 7fa206a85b80 1 -- start start 2024-03-23T17:49:52.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.171+0000 7fa206a85b80 1 Processor -- start 2024-03-23T17:49:52.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.172+0000 7fa206a85b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4bb85a780 0x55c4bb8642d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:52.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.172+0000 7fa206a85b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c4bb85b0e0 0x55c4bb864810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:52.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.172+0000 7fa206a85b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4bb85bf80 0x55c4bb86b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:52.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.172+0000 7fa206a85b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4bb747d30 con 0x55c4bb85a780 2024-03-23T17:49:52.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.172+0000 7fa206a85b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4bb7479d0 con 0x55c4bb85bf80 2024-03-23T17:49:52.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.172+0000 7fa206a85b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4bb62f650 con 0x55c4bb85b0e0 2024-03-23T17:49:52.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.172+0000 7fa206a85b80 10 client.0 _ll_register_callbacks cb 0x55c4bb751420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:49:52.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.172+0000 7fa1ebe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4bb85a780 0x55c4bb8642d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:52.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.172+0000 7fa1eb400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c4bb85b0e0 0x55c4bb864810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:52.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.172+0000 7fa1ebe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4bb85a780 0x55c4bb8642d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47282/0 (socket says 172.21.15.86:47282) 2024-03-23T17:49:52.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.172+0000 7fa1ebe00700 1 -- 172.21.15.86:0/3421113279 learned_addr learned my addr 172.21.15.86:0/3421113279 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:52.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.172+0000 7fa1f1000700 1 --2- 172.21.15.86:0/3421113279 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4bb85bf80 0x55c4bb86b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:52.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.172+0000 7fa1ebe00700 1 -- 172.21.15.86:0/3421113279 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c4bb85b0e0 msgr2=0x55c4bb864810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:49:52.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.172+0000 7fa1ebe00700 1 --2- 172.21.15.86:0/3421113279 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c4bb85b0e0 0x55c4bb864810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:52.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.172+0000 7fa1ebe00700 1 -- 172.21.15.86:0/3421113279 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4bb85bf80 msgr2=0x55c4bb86b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:49:52.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.172+0000 7fa1ebe00700 1 --2- 172.21.15.86:0/3421113279 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4bb85bf80 0x55c4bb86b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:52.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.173+0000 7fa1ebe00700 1 -- 172.21.15.86:0/3421113279 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1ec1a11b0 con 0x55c4bb85a780 2024-03-23T17:49:52.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.173+0000 7fa1eb400700 1 --2- 172.21.15.86:0/3421113279 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c4bb85b0e0 0x55c4bb864810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:49:52.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.173+0000 7fa1ebe00700 1 --2- 172.21.15.86:0/3421113279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4bb85a780 0x55c4bb8642d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc009990 tx=0x7fa1dc09a240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a710353995f5f78 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:52.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.173+0000 7fa1d6000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-23T17:49:52.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.173+0000 7fa1d6000700 1 -- 172.21.15.86:0/3421113279 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1dc0b2070 con 0x55c4bb85a780 2024-03-23T17:49:52.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.173+0000 7fa1d6000700 1 -- 172.21.15.86:0/3421113279 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fa1dc09a7b0 con 0x55c4bb85a780 2024-03-23T17:49:52.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.173+0000 7fa206a85b80 1 -- 172.21.15.86:0/3421113279 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c4bb76c330 con 0x55c4bb85a780 2024-03-23T17:49:52.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.173+0000 7fa1d6000700 1 -- 172.21.15.86:0/3421113279 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa1dc09e170 con 0x55c4bb85a780 2024-03-23T17:49:52.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.173+0000 7fa206a85b80 10 client.14994 fetch_fsmap learned FSMap version 27 2024-03-23T17:49:52.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.173+0000 7fa206a85b80 1 -- 172.21.15.86:0/3421113279 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55c4bb5d9b10 con 0x55c4bb85a780 2024-03-23T17:49:52.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.173+0000 7fa1d6000700 1 -- 172.21.15.86:0/3421113279 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c4bb76c330 con 0x55c4bb85a780 2024-03-23T17:49:52.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.174+0000 7fa1d6000700 1 -- 172.21.15.86:0/3421113279 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa1dc0a2710 con 0x55c4bb85a780 2024-03-23T17:49:52.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.174+0000 7fa206a85b80 10 client.14994 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T17:49:52.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.174+0000 7fa206a85b80 10 client.14994 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-23T17:49:52.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.174+0000 7fa206a85b80 20 client.14994 populate_metadata read hostname 'smithi086' 2024-03-23T17:49:52.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.174+0000 7fa206a85b80 1 --2- 172.21.15.86:0/3421113279 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55c4bb86eee0 0x55c4bb86d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:52.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.174+0000 7fa206a85b80 4 client.14994 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:49:52.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.174+0000 7fa206a85b80 1 -- 172.21.15.86:0/3421113279 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c4bb7a46b0 con 0x55c4bb86eee0 2024-03-23T17:49:52.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.174+0000 7fa1eb400700 1 --2- 172.21.15.86:0/3421113279 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55c4bb86eee0 0x55c4bb86d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:52.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.175+0000 7fa1d6000700 10 client.14994 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:49:52.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.175+0000 7fa1eb400700 1 --2- 172.21.15.86:0/3421113279 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55c4bb86eee0 0x55c4bb86d6a0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbd12c2694122a3e server_cookie=84673da71a696cd in_seq=0 out_seq=0 2024-03-23T17:49:52.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.176+0000 7fa1d6000700 1 -- 172.21.15.86:0/3421113279 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55c4bb7a46b0 con 0x55c4bb86eee0 2024-03-23T17:49:52.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.176+0000 7fa1d6000700 10 client.14994 handle_command_reply: tid=0 2024-03-23T17:49:52.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.176+0000 7fa206a85b80 1 client.14994 shutdown 2024-03-23T17:49:52.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.176+0000 7fa206a85b80 1 -- 172.21.15.86:0/3421113279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4bb85a780 msgr2=0x55c4bb8642d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:52.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.176+0000 7fa206a85b80 1 --2- 172.21.15.86:0/3421113279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4bb85a780 0x55c4bb8642d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc009990 tx=0x7fa1dc09a240 comp rx=0 tx=0).stop 2024-03-23T17:49:52.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.176+0000 7fa206a85b80 1 -- 172.21.15.86:0/3421113279 shutdown_connections 2024-03-23T17:49:52.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.176+0000 7fa206a85b80 1 --2- 172.21.15.86:0/3421113279 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55c4bb86eee0 0x55c4bb86d6a0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:52.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.176+0000 7fa206a85b80 1 --2- 172.21.15.86:0/3421113279 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4bb85bf80 0x55c4bb86b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:52.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.176+0000 7fa206a85b80 1 --2- 172.21.15.86:0/3421113279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4bb85a780 0x55c4bb8642d0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:52.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.176+0000 7fa206a85b80 1 --2- 172.21.15.86:0/3421113279 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c4bb85b0e0 0x55c4bb864810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:52.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.176+0000 7fa206a85b80 1 -- 172.21.15.86:0/3421113279 >> 172.21.15.86:0/3421113279 conn(0x55c4bb7b9650 msgr2=0x55c4bb859120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:52.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.176+0000 7fa1d6000700 0 client.14994 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:49:52.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.176+0000 7fa206a85b80 1 -- 172.21.15.86:0/3421113279 shutdown_connections 2024-03-23T17:49:52.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.176+0000 7fa206a85b80 1 -- 172.21.15.86:0/3421113279 wait complete. 2024-03-23T17:49:52.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.177+0000 7fa206a85b80 20 client.14994 trim_cache size 0 max 16384 2024-03-23T17:49:52.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.195+0000 7fa206a85b80 1 Processor -- start 2024-03-23T17:49:52.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.195+0000 7fa206a85b80 1 -- start start 2024-03-23T17:49:52.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.195+0000 7fa206a85b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4bb85a780 0x55c4bb854ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:52.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.195+0000 7fa206a85b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4bb85b0e0 0x55c4bb879340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:52.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.195+0000 7fa206a85b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c4bb85bf80 0x55c4bb862ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:52.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.195+0000 7fa206a85b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4bb7a46b0 con 0x55c4bb85a780 2024-03-23T17:49:52.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.195+0000 7fa206a85b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4bb76c330 con 0x55c4bb85b0e0 2024-03-23T17:49:52.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.195+0000 7fa206a85b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4bb769ad0 con 0x55c4bb85bf80 2024-03-23T17:49:52.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.195+0000 7fa1ebe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4bb85a780 0x55c4bb854ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:52.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.195+0000 7fa1ebe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4bb85a780 0x55c4bb854ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47286/0 (socket says 172.21.15.86:47286) 2024-03-23T17:49:52.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.195+0000 7fa1ebe00700 1 -- 172.21.15.86:0/1712719261 learned_addr learned my addr 172.21.15.86:0/1712719261 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:52.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.195+0000 7fa1eb400700 1 --2- 172.21.15.86:0/1712719261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4bb85b0e0 0x55c4bb879340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:52.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.195+0000 7fa1f1000700 1 --2- 172.21.15.86:0/1712719261 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c4bb85bf80 0x55c4bb862ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:52.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.195+0000 7fa1ebe00700 1 -- 172.21.15.86:0/1712719261 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c4bb85bf80 msgr2=0x55c4bb862ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:49:52.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.195+0000 7fa1ebe00700 1 --2- 172.21.15.86:0/1712719261 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c4bb85bf80 0x55c4bb862ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:52.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.195+0000 7fa1ebe00700 1 -- 172.21.15.86:0/1712719261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4bb85b0e0 msgr2=0x55c4bb879340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:49:52.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.195+0000 7fa1ebe00700 1 --2- 172.21.15.86:0/1712719261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4bb85b0e0 0x55c4bb879340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:52.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.195+0000 7fa1ebe00700 1 -- 172.21.15.86:0/1712719261 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c4bb771e50 con 0x55c4bb85a780 2024-03-23T17:49:52.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.195+0000 7fa1ebe00700 1 --2- 172.21.15.86:0/1712719261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4bb85a780 0x55c4bb854ec0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc0cae10 tx=0x7fa1dc009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=668d9518186e22 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:52.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.196+0000 7fa1e2000700 1 -- 172.21.15.86:0/1712719261 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1dc0b2070 con 0x55c4bb85a780 2024-03-23T17:49:52.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.196+0000 7fa1e2000700 1 -- 172.21.15.86:0/1712719261 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa1dc0ba0d0 con 0x55c4bb85a780 2024-03-23T17:49:52.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.196+0000 7fa1e2000700 1 -- 172.21.15.86:0/1712719261 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1dc09a7b0 con 0x55c4bb85a780 2024-03-23T17:49:52.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.196+0000 7fa206a85b80 1 -- 172.21.15.86:0/1712719261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4bb85a780 msgr2=0x55c4bb854ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:52.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.196+0000 7fa206a85b80 1 --2- 172.21.15.86:0/1712719261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4bb85a780 0x55c4bb854ec0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc0cae10 tx=0x7fa1dc009310 comp rx=0 tx=0).stop 2024-03-23T17:49:52.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.196+0000 7fa206a85b80 1 -- 172.21.15.86:0/1712719261 shutdown_connections 2024-03-23T17:49:52.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.196+0000 7fa206a85b80 1 --2- 172.21.15.86:0/1712719261 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c4bb85bf80 0x55c4bb862ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:52.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.196+0000 7fa206a85b80 1 --2- 172.21.15.86:0/1712719261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4bb85a780 0x55c4bb854ec0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:52.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.196+0000 7fa206a85b80 1 --2- 172.21.15.86:0/1712719261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4bb85b0e0 0x55c4bb879340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:52.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.196+0000 7fa206a85b80 1 -- 172.21.15.86:0/1712719261 >> 172.21.15.86:0/1712719261 conn(0x55c4bb7b9650 msgr2=0x55c4bb84e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:52.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.196+0000 7fa206a85b80 1 -- 172.21.15.86:0/1712719261 shutdown_connections 2024-03-23T17:49:52.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.196+0000 7fa206a85b80 1 -- 172.21.15.86:0/1712719261 wait complete. 2024-03-23T17:49:52.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.197+0000 7fa206a85b80 1 -- start start 2024-03-23T17:49:52.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.197+0000 7fa206a85b80 1 Processor -- start 2024-03-23T17:49:52.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.197+0000 7fa206a85b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4bb85a780 0x55c4bb85ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:52.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.197+0000 7fa206a85b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4bb85b0e0 0x55c4bb85f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:52.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.197+0000 7fa206a85b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c4bb85bf80 0x55c4bb878380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:52.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.197+0000 7fa206a85b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4bb864980 con 0x55c4bb85b0e0 2024-03-23T17:49:52.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.197+0000 7fa206a85b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4bb864af0 con 0x55c4bb85a780 2024-03-23T17:49:52.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.197+0000 7fa206a85b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4bb864cb0 con 0x55c4bb85bf80 2024-03-23T17:49:52.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.197+0000 7fa206a85b80 10 client.0 _ll_register_callbacks cb 0x55c4bb751420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:49:52.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.197+0000 7fa1eb400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4bb85b0e0 0x55c4bb85f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:52.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.197+0000 7fa1ebe00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4bb85a780 0x55c4bb85ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:52.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.197+0000 7fa1eb400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4bb85b0e0 0x55c4bb85f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47288/0 (socket says 172.21.15.86:47288) 2024-03-23T17:49:52.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.198+0000 7fa1eb400700 1 -- 172.21.15.86:0/3802609942 learned_addr learned my addr 172.21.15.86:0/3802609942 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:52.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.198+0000 7fa1f1000700 1 --2- 172.21.15.86:0/3802609942 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c4bb85bf80 0x55c4bb878380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:52.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.198+0000 7fa1ebe00700 1 -- 172.21.15.86:0/3802609942 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c4bb85bf80 msgr2=0x55c4bb878380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:49:52.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.198+0000 7fa1ebe00700 1 --2- 172.21.15.86:0/3802609942 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c4bb85bf80 0x55c4bb878380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:52.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.198+0000 7fa1ebe00700 1 -- 172.21.15.86:0/3802609942 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4bb85b0e0 msgr2=0x55c4bb85f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:49:52.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.198+0000 7fa1ebe00700 1 --2- 172.21.15.86:0/3802609942 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4bb85b0e0 0x55c4bb85f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:52.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.198+0000 7fa1ebe00700 1 -- 172.21.15.86:0/3802609942 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c4bb5d9b10 con 0x55c4bb85a780 2024-03-23T17:49:52.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.198+0000 7fa1eb400700 1 --2- 172.21.15.86:0/3802609942 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4bb85b0e0 0x55c4bb85f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:49:52.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.198+0000 7fa1ebe00700 1 --2- 172.21.15.86:0/3802609942 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4bb85a780 0x55c4bb85ebd0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc0a2670 tx=0x7fa1dc09a3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=504760403ae3eb71 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:52.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.198+0000 7fa1d6000700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T17:49:52.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.198+0000 7fa1d6000700 1 -- 172.21.15.86:0/3802609942 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1dc0b2070 con 0x55c4bb85a780 2024-03-23T17:49:52.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.198+0000 7fa206a85b80 1 -- 172.21.15.86:0/3802609942 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa1dc09e5e0 con 0x55c4bb85a780 2024-03-23T17:49:52.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.198+0000 7fa1d6000700 1 -- 172.21.15.86:0/3802609942 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fa1dc09a7b0 con 0x55c4bb85a780 2024-03-23T17:49:52.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.198+0000 7fa1d6000700 1 -- 172.21.15.86:0/3802609942 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa1dc004ea0 con 0x55c4bb85a780 2024-03-23T17:49:52.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.198+0000 7fa206a85b80 10 client.25040 fetch_fsmap learned FSMap version 27 2024-03-23T17:49:52.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.198+0000 7fa206a85b80 1 -- 172.21.15.86:0/3802609942 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55c4bb631c80 con 0x55c4bb85a780 2024-03-23T17:49:52.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.199+0000 7fa1d6000700 1 -- 172.21.15.86:0/3802609942 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa1dc0a6080 con 0x55c4bb85a780 2024-03-23T17:49:52.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.199+0000 7fa1d6000700 1 -- 172.21.15.86:0/3802609942 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa1dc082030 con 0x55c4bb85a780 2024-03-23T17:49:52.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.199+0000 7fa206a85b80 10 client.25040 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T17:49:52.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.199+0000 7fa206a85b80 10 client.25040 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-23T17:49:52.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.199+0000 7fa206a85b80 20 client.25040 populate_metadata read hostname 'smithi086' 2024-03-23T17:49:52.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.199+0000 7fa206a85b80 1 --2- 172.21.15.86:0/3802609942 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55c4bb8c5600 0x55c4bb8c2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:52.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.199+0000 7fa206a85b80 4 client.25040 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-23T17:49:52.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.199+0000 7fa206a85b80 1 -- 172.21.15.86:0/3802609942 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c4bb5faca0 con 0x55c4bb8c5600 2024-03-23T17:49:52.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.199+0000 7fa1eb400700 1 --2- 172.21.15.86:0/3802609942 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55c4bb8c5600 0x55c4bb8c2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:52.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.200+0000 7fa1eb400700 1 --2- 172.21.15.86:0/3802609942 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55c4bb8c5600 0x55c4bb8c2180 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79f0cd72f8bf0191 server_cookie=115d0f6d23cb8ff8 in_seq=0 out_seq=0 2024-03-23T17:49:52.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.200+0000 7fa1d6000700 10 client.25040 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:49:52.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.200+0000 7fa1d6000700 1 -- 172.21.15.86:0/3802609942 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55c4bb5faca0 con 0x55c4bb8c5600 2024-03-23T17:49:52.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.200+0000 7fa1d6000700 10 client.25040 handle_command_reply: tid=0 2024-03-23T17:49:52.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.200+0000 7fa206a85b80 1 client.25040 shutdown 2024-03-23T17:49:52.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.200+0000 7fa206a85b80 1 -- 172.21.15.86:0/3802609942 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4bb85a780 msgr2=0x55c4bb85ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:52.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.200+0000 7fa206a85b80 1 --2- 172.21.15.86:0/3802609942 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4bb85a780 0x55c4bb85ebd0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc0a2670 tx=0x7fa1dc09a3a0 comp rx=0 tx=0).stop 2024-03-23T17:49:52.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.200+0000 7fa206a85b80 1 -- 172.21.15.86:0/3802609942 shutdown_connections 2024-03-23T17:49:52.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.200+0000 7fa206a85b80 1 --2- 172.21.15.86:0/3802609942 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55c4bb8c5600 0x55c4bb8c2180 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:52.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.200+0000 7fa206a85b80 1 --2- 172.21.15.86:0/3802609942 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c4bb85bf80 0x55c4bb878380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:52.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.200+0000 7fa206a85b80 1 --2- 172.21.15.86:0/3802609942 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4bb85a780 0x55c4bb85ebd0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:52.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.200+0000 7fa1d6000700 0 client.25040 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:49:52.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.200+0000 7fa206a85b80 1 --2- 172.21.15.86:0/3802609942 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4bb85b0e0 0x55c4bb85f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:52.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.200+0000 7fa206a85b80 1 -- 172.21.15.86:0/3802609942 >> 172.21.15.86:0/3802609942 conn(0x55c4bb7b9650 msgr2=0x55c4bb862ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:52.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.200+0000 7fa206a85b80 1 -- 172.21.15.86:0/3802609942 shutdown_connections 2024-03-23T17:49:52.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.200+0000 7fa206a85b80 1 -- 172.21.15.86:0/3802609942 wait complete. 2024-03-23T17:49:52.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.201+0000 7fa206a85b80 20 client.25040 trim_cache size 0 max 16384 2024-03-23T17:49:52.218 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-23T17:49:52.218 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "9d76c9e8-e93c-11ee-95cd-87774f69a715", 2024-03-23T17:49:52.218 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 0, 2024-03-23T17:49:52.218 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24563, 2024-03-23T17:49:52.219 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-03-23T17:49:52.219 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-03-23T17:49:52.219 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-03-23T17:49:52.219 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 55.996486310999998, 2024-03-23T17:49:52.219 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 27, 2024-03-23T17:49:52.219 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 82, 2024-03-23T17:49:52.219 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 80, 2024-03-23T17:49:52.219 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 62.609545969000003 2024-03-23T17:49:52.219 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-23T17:49:52.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.201+0000 7fa1f3000700 1 -- 172.21.15.86:0/867387128 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fa1d00a1230 msgr2=0x7fa1d00a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:52.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.201+0000 7fa1f3000700 1 --2- 172.21.15.86:0/867387128 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fa1d00a1230 0x7fa1d00a36f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa1d8009c60 tx=0x7fa1d8009260 comp rx=0 tx=0).stop 2024-03-23T17:49:52.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.201+0000 7fa1f3000700 1 -- 172.21.15.86:0/867387128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1ec100d80 msgr2=0x7fa1ec074b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:52.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.201+0000 7fa1f3000700 1 --2- 172.21.15.86:0/867387128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1ec100d80 0x7fa1ec074b10 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc006010 tx=0x7fa1dc00f690 comp rx=0 tx=0).stop 2024-03-23T17:49:52.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.201+0000 7fa1f3000700 1 -- 172.21.15.86:0/867387128 shutdown_connections 2024-03-23T17:49:52.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.201+0000 7fa1f3000700 1 --2- 172.21.15.86:0/867387128 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa1ec102910 0x7fa1ec0736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:52.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.201+0000 7fa1f3000700 1 --2- 172.21.15.86:0/867387128 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fa1d00a1230 0x7fa1d00a36f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:52.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.201+0000 7fa1f3000700 1 --2- 172.21.15.86:0/867387128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1ec100d80 0x7fa1ec074b10 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:52.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.201+0000 7fa1f3000700 1 --2- 172.21.15.86:0/867387128 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa1ec101f70 0x7fa1ec0731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:52.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.201+0000 7fa1f3000700 1 -- 172.21.15.86:0/867387128 >> 172.21.15.86:0/867387128 conn(0x7fa1ec0fc2f0 msgr2=0x7fa1ec105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:52.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.201+0000 7fa1f3000700 1 -- 172.21.15.86:0/867387128 shutdown_connections 2024-03-23T17:49:52.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:52.201+0000 7fa1f3000700 1 -- 172.21.15.86:0/867387128 wait complete. 2024-03-23T17:49:52.222 INFO:tasks.cephadm:Setup complete, yielding 2024-03-23T17:49:52.222 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-23T17:49:52.232 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi086.front.sepia.ceph.com 2024-03-23T17:49:52.233 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- bash -c 'ceph orch status' 2024-03-23T17:49:52.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:51 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/536892461' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T17:49:52.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:51 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/536892461' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T17:49:52.615 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T17:49:53.126 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:52 smithi086 ceph-mon[38221]: pgmap v264: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-23T17:49:53.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:52 smithi092 ceph-mon[79089]: pgmap v264: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-23T17:49:53.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:52 smithi152 ceph-mon[47949]: pgmap v264: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-23T17:49:53.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:53.835+0000 7f01ea000700 1 -- 172.21.15.86:0/1268824619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01e4102920 msgr2=0x7f01e4109240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:53.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:53.835+0000 7f01ea000700 1 --2- 172.21.15.86:0/1268824619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01e4102920 0x7f01e4109240 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f01dc00b0a0 tx=0x7f01dc00b3b0 comp rx=0 tx=0).stop 2024-03-23T17:49:53.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:53.835+0000 7f01ea000700 1 -- 172.21.15.86:0/1268824619 shutdown_connections 2024-03-23T17:49:53.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:53.835+0000 7f01ea000700 1 --2- 172.21.15.86:0/1268824619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01e4102920 0x7f01e4109240 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:53.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:53.835+0000 7f01ea000700 1 --2- 172.21.15.86:0/1268824619 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01e4100e00 0x7f01e4101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:53.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:53.835+0000 7f01ea000700 1 --2- 172.21.15.86:0/1268824619 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01e4101f60 0x7f01e41023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:53.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:53.835+0000 7f01ea000700 1 -- 172.21.15.86:0/1268824619 >> 172.21.15.86:0/1268824619 conn(0x7f01e40fc3c0 msgr2=0x7f01e40fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:53.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:53.836+0000 7f01ea000700 1 -- 172.21.15.86:0/1268824619 shutdown_connections 2024-03-23T17:49:53.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:53.836+0000 7f01ea000700 1 -- 172.21.15.86:0/1268824619 wait complete. 2024-03-23T17:49:53.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:53.837+0000 7f01ea000700 1 Processor -- start 2024-03-23T17:49:53.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:53.837+0000 7f01ea000700 1 -- start start 2024-03-23T17:49:53.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:53.837+0000 7f01ea000700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01e4100e00 0x7f01e4196e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:53.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:53.837+0000 7f01ea000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01e4101f60 0x7f01e41973b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:53.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:53.837+0000 7f01ea000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01e4102920 0x7f01e419e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:53.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:53.837+0000 7f01ea000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e419e960 con 0x7f01e4102920 2024-03-23T17:49:53.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:53.837+0000 7f01ea000700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e419eaa0 con 0x7f01e4101f60 2024-03-23T17:49:53.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:53.837+0000 7f01ea000700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e419ec30 con 0x7f01e4100e00 2024-03-23T17:49:53.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:53.837+0000 7f01e3400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01e4100e00 0x7f01e4196e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:53.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:53.838+0000 7f01e3400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01e4100e00 0x7f01e4196e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:56834/0 (socket says 172.21.15.86:56834) 2024-03-23T17:49:53.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:53.838+0000 7f01e3400700 1 -- 172.21.15.86:0/1105639145 learned_addr learned my addr 172.21.15.86:0/1105639145 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:53.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:53.838+0000 7f01e2a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01e4101f60 0x7f01e41973b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:53.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:53.838+0000 7f01e3e00700 1 --2- 172.21.15.86:0/1105639145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01e4102920 0x7f01e419e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:53.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:53.838+0000 7f01e2a00700 1 -- 172.21.15.86:0/1105639145 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01e4100e00 msgr2=0x7f01e4196e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:53.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:53.838+0000 7f01e2a00700 1 --2- 172.21.15.86:0/1105639145 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01e4100e00 0x7f01e4196e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:53.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:53.838+0000 7f01e2a00700 1 -- 172.21.15.86:0/1105639145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01e4102920 msgr2=0x7f01e419e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:53.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:53.838+0000 7f01e2a00700 1 --2- 172.21.15.86:0/1105639145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01e4102920 0x7f01e419e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:53.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:53.838+0000 7f01e2a00700 1 -- 172.21.15.86:0/1105639145 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01dc009d00 con 0x7f01e4101f60 2024-03-23T17:49:53.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:53.838+0000 7f01e3400700 1 --2- 172.21.15.86:0/1105639145 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01e4100e00 0x7f01e4196e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:49:53.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:53.838+0000 7f01e3e00700 1 --2- 172.21.15.86:0/1105639145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01e4102920 0x7f01e419e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:49:53.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:53.838+0000 7f01e2a00700 1 --2- 172.21.15.86:0/1105639145 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01e4101f60 0x7f01e41973b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f01d000d750 tx=0x7f01d000da60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:53.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:53.838+0000 7f01dbe00700 1 -- 172.21.15.86:0/1105639145 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d0004020 con 0x7f01e4101f60 2024-03-23T17:49:53.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:53.839+0000 7f01dbe00700 1 -- 172.21.15.86:0/1105639145 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f01d0004830 con 0x7f01e4101f60 2024-03-23T17:49:53.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:53.839+0000 7f01ea000700 1 -- 172.21.15.86:0/1105639145 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01e419ef10 con 0x7f01e4101f60 2024-03-23T17:49:53.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:53.839+0000 7f01dbe00700 1 -- 172.21.15.86:0/1105639145 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d0005520 con 0x7f01e4101f60 2024-03-23T17:49:53.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:53.839+0000 7f01ea000700 1 -- 172.21.15.86:0/1105639145 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01e419f380 con 0x7f01e4101f60 2024-03-23T17:49:53.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:53.840+0000 7f01ea000700 1 -- 172.21.15.86:0/1105639145 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01e41081b0 con 0x7f01e4101f60 2024-03-23T17:49:53.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:53.842+0000 7f01dbe00700 1 -- 172.21.15.86:0/1105639145 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f01d00049a0 con 0x7f01e4101f60 2024-03-23T17:49:53.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:53.843+0000 7f01dbe00700 1 --2- 172.21.15.86:0/1105639145 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f01c80a11b0 0x7f01c80a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:53.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:53.843+0000 7f01dbe00700 1 -- 172.21.15.86:0/1105639145 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f01d00cc620 con 0x7f01e4101f60 2024-03-23T17:49:53.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:53.843+0000 7f01e3400700 1 --2- 172.21.15.86:0/1105639145 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f01c80a11b0 0x7f01c80a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:53.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:53.844+0000 7f01e3400700 1 --2- 172.21.15.86:0/1105639145 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f01c80a11b0 0x7f01c80a3670 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f01d4011550 tx=0x7f01d4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:53.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:53.845+0000 7f01dbe00700 1 -- 172.21.15.86:0/1105639145 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f01d0095c20 con 0x7f01e4101f60 2024-03-23T17:49:54.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:54.074+0000 7f01ea000700 1 -- 172.21.15.86:0/1105639145 --> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f01e402cdc0 con 0x7f01c80a11b0 2024-03-23T17:49:54.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:54.076+0000 7f01dbe00700 1 -- 172.21.15.86:0/1105639145 <== mgr.14152 v2:172.21.15.86:6800/2734784244 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f01e402cdc0 con 0x7f01c80a11b0 2024-03-23T17:49:54.076 INFO:teuthology.orchestra.run.smithi086.stdout:Backend: cephadm 2024-03-23T17:49:54.076 INFO:teuthology.orchestra.run.smithi086.stdout:Available: Yes 2024-03-23T17:49:54.076 INFO:teuthology.orchestra.run.smithi086.stdout:Paused: No 2024-03-23T17:49:54.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:54.078+0000 7f01ea000700 1 -- 172.21.15.86:0/1105639145 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f01c80a11b0 msgr2=0x7f01c80a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:54.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:54.078+0000 7f01ea000700 1 --2- 172.21.15.86:0/1105639145 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f01c80a11b0 0x7f01c80a3670 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f01d4011550 tx=0x7f01d4009260 comp rx=0 tx=0).stop 2024-03-23T17:49:54.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:54.078+0000 7f01ea000700 1 -- 172.21.15.86:0/1105639145 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01e4101f60 msgr2=0x7f01e41973b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:54.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:54.078+0000 7f01ea000700 1 --2- 172.21.15.86:0/1105639145 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01e4101f60 0x7f01e41973b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f01d000d750 tx=0x7f01d000da60 comp rx=0 tx=0).stop 2024-03-23T17:49:54.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:54.078+0000 7f01ea000700 1 -- 172.21.15.86:0/1105639145 shutdown_connections 2024-03-23T17:49:54.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:54.078+0000 7f01ea000700 1 --2- 172.21.15.86:0/1105639145 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f01c80a11b0 0x7f01c80a3670 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:54.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:54.078+0000 7f01ea000700 1 --2- 172.21.15.86:0/1105639145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01e4102920 0x7f01e419e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:54.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:54.078+0000 7f01ea000700 1 --2- 172.21.15.86:0/1105639145 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01e4100e00 0x7f01e4196e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:54.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:54.078+0000 7f01ea000700 1 --2- 172.21.15.86:0/1105639145 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01e4101f60 0x7f01e41973b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:54.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:54.078+0000 7f01ea000700 1 -- 172.21.15.86:0/1105639145 >> 172.21.15.86:0/1105639145 conn(0x7f01e40fc3c0 msgr2=0x7f01e41052f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:54.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:54.078+0000 7f01ea000700 1 -- 172.21.15.86:0/1105639145 shutdown_connections 2024-03-23T17:49:54.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:54.078+0000 7f01ea000700 1 -- 172.21.15.86:0/1105639145 wait complete. 2024-03-23T17:49:54.259 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:53 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:49:54.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:53 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:49:54.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:53 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:49:54.444 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- bash -c 'ceph orch ps' 2024-03-23T17:49:55.141 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T17:49:55.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:54 smithi086 ceph-mon[38221]: from='client.25046 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:49:55.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:54 smithi086 ceph-mon[38221]: pgmap v265: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-23T17:49:55.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:54 smithi092 ceph-mon[79089]: from='client.25046 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:49:55.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:54 smithi092 ceph-mon[79089]: pgmap v265: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-23T17:49:55.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:55 smithi152 ceph-mon[47949]: from='client.25046 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:49:55.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:55 smithi152 ceph-mon[47949]: pgmap v265: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-23T17:49:56.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:56.287+0000 7f6992a00700 1 -- 172.21.15.86:0/1882315792 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f698c074de0 msgr2=0x7f698c073240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:56.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:56.287+0000 7f6992a00700 1 --2- 172.21.15.86:0/1882315792 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f698c074de0 0x7f698c073240 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f697c009990 tx=0x7f697c009ca0 comp rx=0 tx=0).stop 2024-03-23T17:49:56.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:56.287+0000 7f6992a00700 1 -- 172.21.15.86:0/1882315792 shutdown_connections 2024-03-23T17:49:56.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:56.287+0000 7f6992a00700 1 --2- 172.21.15.86:0/1882315792 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f698c10d3e0 0x7f698c10f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:56.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:56.287+0000 7f6992a00700 1 --2- 172.21.15.86:0/1882315792 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f698c074de0 0x7f698c073240 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:56.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:56.287+0000 7f6992a00700 1 --2- 172.21.15.86:0/1882315792 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f698c073810 0x7f698c073c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:56.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:56.287+0000 7f6992a00700 1 -- 172.21.15.86:0/1882315792 >> 172.21.15.86:0/1882315792 conn(0x7f698c0fc3c0 msgr2=0x7f698c0fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:56.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:56.287+0000 7f6992a00700 1 -- 172.21.15.86:0/1882315792 shutdown_connections 2024-03-23T17:49:56.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:56.287+0000 7f6992a00700 1 -- 172.21.15.86:0/1882315792 wait complete. 2024-03-23T17:49:56.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:56.288+0000 7f6992a00700 1 Processor -- start 2024-03-23T17:49:56.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:56.288+0000 7f6992a00700 1 -- start start 2024-03-23T17:49:56.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:56.289+0000 7f6992a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f698c073810 0x7f698c19d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:56.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:56.289+0000 7f6992a00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f698c074de0 0x7f698c19d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:56.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:56.289+0000 7f6992a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f698c10d3e0 0x7f698c1a48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:56.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:56.289+0000 7f6992a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f698c1a4eb0 con 0x7f698c10d3e0 2024-03-23T17:49:56.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:56.289+0000 7f6992a00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f698c1a4ff0 con 0x7f698c073810 2024-03-23T17:49:56.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:56.289+0000 7f6992a00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f698c1a5180 con 0x7f698c074de0 2024-03-23T17:49:56.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:56.289+0000 7f698be00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f698c073810 0x7f698c19d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:56.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:56.289+0000 7f698be00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f698c073810 0x7f698c19d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:45028/0 (socket says 172.21.15.86:45028) 2024-03-23T17:49:56.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:56.289+0000 7f698be00700 1 -- 172.21.15.86:0/2391152100 learned_addr learned my addr 172.21.15.86:0/2391152100 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:56.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:56.289+0000 7f698b400700 1 --2- 172.21.15.86:0/2391152100 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f698c074de0 0x7f698c19d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:56.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:56.289+0000 7f6990a00700 1 --2- 172.21.15.86:0/2391152100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f698c10d3e0 0x7f698c1a48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:56.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:56.289+0000 7f698be00700 1 -- 172.21.15.86:0/2391152100 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f698c074de0 msgr2=0x7f698c19d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:56.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:56.289+0000 7f698be00700 1 --2- 172.21.15.86:0/2391152100 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f698c074de0 0x7f698c19d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:56.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:56.289+0000 7f698be00700 1 -- 172.21.15.86:0/2391152100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f698c10d3e0 msgr2=0x7f698c1a48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:56.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:56.289+0000 7f698be00700 1 --2- 172.21.15.86:0/2391152100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f698c10d3e0 0x7f698c1a48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:56.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:56.289+0000 7f698be00700 1 -- 172.21.15.86:0/2391152100 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f697c0095f0 con 0x7f698c073810 2024-03-23T17:49:56.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:56.289+0000 7f698b400700 1 --2- 172.21.15.86:0/2391152100 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f698c074de0 0x7f698c19d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:49:56.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:56.290+0000 7f698be00700 1 --2- 172.21.15.86:0/2391152100 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f698c073810 0x7f698c19d370 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f697c000c00 tx=0x7f697c009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:56.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:56.290+0000 7f6988c00700 1 -- 172.21.15.86:0/2391152100 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f697c01d070 con 0x7f698c073810 2024-03-23T17:49:56.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:56.290+0000 7f6988c00700 1 -- 172.21.15.86:0/2391152100 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f697c021410 con 0x7f698c073810 2024-03-23T17:49:56.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:56.290+0000 7f6992a00700 1 -- 172.21.15.86:0/2391152100 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f698c1a5400 con 0x7f698c073810 2024-03-23T17:49:56.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:56.290+0000 7f6988c00700 1 -- 172.21.15.86:0/2391152100 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f697c017650 con 0x7f698c073810 2024-03-23T17:49:56.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:56.290+0000 7f6992a00700 1 -- 172.21.15.86:0/2391152100 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f698c1a5870 con 0x7f698c073810 2024-03-23T17:49:56.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:56.292+0000 7f6992a00700 1 -- 172.21.15.86:0/2391152100 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f698c103640 con 0x7f698c073810 2024-03-23T17:49:56.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:56.294+0000 7f6988c00700 1 -- 172.21.15.86:0/2391152100 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f697c021580 con 0x7f698c073810 2024-03-23T17:49:56.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:56.295+0000 7f6988c00700 1 --2- 172.21.15.86:0/2391152100 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f69700a1230 0x7f69700a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:56.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:56.295+0000 7f6988c00700 1 -- 172.21.15.86:0/2391152100 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f697c0cdd30 con 0x7f698c073810 2024-03-23T17:49:56.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:56.295+0000 7f698b400700 1 --2- 172.21.15.86:0/2391152100 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f69700a1230 0x7f69700a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:56.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:56.296+0000 7f698b400700 1 --2- 172.21.15.86:0/2391152100 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f69700a1230 0x7f69700a36f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f698c074b10 tx=0x7f6978009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:56.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:56.299+0000 7f6988c00700 1 -- 172.21.15.86:0/2391152100 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f697c097420 con 0x7f698c073810 2024-03-23T17:49:56.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:56 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:49:56.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:56 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:49:56.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:56 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:49:56.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:56 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:49:56.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:56 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:49:56.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:56 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:49:56.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:56 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:49:56.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:56 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:49:56.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:56 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:49:56.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:56 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:49:56.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:56 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:49:56.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:56 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:49:56.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:56.586+0000 7f6992a00700 1 -- 172.21.15.86:0/2391152100 --> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f698c0611f0 con 0x7f69700a1230 2024-03-23T17:49:56.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:56.602+0000 7f6988c00700 1 -- 172.21.15.86:0/2391152100 <== mgr.14152 v2:172.21.15.86:6800/2734784244 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4154 (secure 0 0 0) 0x7f698c0611f0 con 0x7f69700a1230 2024-03-23T17:49:56.602 INFO:teuthology.orchestra.run.smithi086.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-23T17:49:56.602 INFO:teuthology.orchestra.run.smithi086.stdout:mds.a smithi086 running (69s) 0s ago 68s 17.2M - 18.2.2-954-g82316b04 b24e1baba637 409beb7eb2e7 2024-03-23T17:49:56.602 INFO:teuthology.orchestra.run.smithi086.stdout:mds.b smithi092 running (67s) 1s ago 66s 32.7M - 18.2.2-954-g82316b04 b24e1baba637 60fd5234d59f 2024-03-23T17:49:56.602 INFO:teuthology.orchestra.run.smithi086.stdout:mds.c smithi152 running (77s) 1s ago 77s 19.8M - 18.2.2-954-g82316b04 b24e1baba637 ff682e024870 2024-03-23T17:49:56.602 INFO:teuthology.orchestra.run.smithi086.stdout:mds.d smithi086 running (83s) 0s ago 82s 17.9M - 18.2.2-954-g82316b04 b24e1baba637 81be26271251 2024-03-23T17:49:56.602 INFO:teuthology.orchestra.run.smithi086.stdout:mds.e smithi092 running (81s) 1s ago 80s 18.0M - 18.2.2-954-g82316b04 b24e1baba637 c7e375d6898d 2024-03-23T17:49:56.602 INFO:teuthology.orchestra.run.smithi086.stdout:mds.f smithi152 running (79s) 1s ago 79s 17.4M - 18.2.2-954-g82316b04 b24e1baba637 2f03768518bd 2024-03-23T17:49:56.603 INFO:teuthology.orchestra.run.smithi086.stdout:mds.g smithi086 running (71s) 0s ago 71s 13.7M - 18.2.2-954-g82316b04 b24e1baba637 1bf4fce38dbc 2024-03-23T17:49:56.603 INFO:teuthology.orchestra.run.smithi086.stdout:mds.h smithi092 running (75s) 1s ago 75s 21.0M - 18.2.2-954-g82316b04 b24e1baba637 5dd6eda4695d 2024-03-23T17:49:56.603 INFO:teuthology.orchestra.run.smithi086.stdout:mds.i smithi152 running (74s) 1s ago 73s 19.0M - 18.2.2-954-g82316b04 b24e1baba637 3e9be4dddbb6 2024-03-23T17:49:56.603 INFO:teuthology.orchestra.run.smithi086.stdout:mds.j smithi086 running (85s) 0s ago 85s 13.8M - 18.2.2-954-g82316b04 b24e1baba637 72bdeac7764e 2024-03-23T17:49:56.603 INFO:teuthology.orchestra.run.smithi086.stdout:mds.k smithi092 running (87s) 1s ago 87s 18.4M - 18.2.2-954-g82316b04 b24e1baba637 5022702cb9bf 2024-03-23T17:49:56.603 INFO:teuthology.orchestra.run.smithi086.stdout:mds.l smithi152 running (89s) 1s ago 89s 19.7M - 18.2.2-954-g82316b04 b24e1baba637 05c687668ebe 2024-03-23T17:49:56.603 INFO:teuthology.orchestra.run.smithi086.stdout:mgr.x smithi086 *:9283,8765 running (7m) 0s ago 7m 534M - 18.2.2-954-g82316b04 b24e1baba637 9aeda55ba807 2024-03-23T17:49:56.603 INFO:teuthology.orchestra.run.smithi086.stdout:mgr.y smithi092 *:8443,8765 running (5m) 1s ago 5m 458M - 18.2.2-954-g82316b04 b24e1baba637 f220bcd9b7cb 2024-03-23T17:49:56.603 INFO:teuthology.orchestra.run.smithi086.stdout:mgr.z smithi152 *:8443,8765 running (5m) 1s ago 5m 456M - 18.2.2-954-g82316b04 b24e1baba637 37a09083e9be 2024-03-23T17:49:56.603 INFO:teuthology.orchestra.run.smithi086.stdout:mon.a smithi086 running (7m) 0s ago 7m 89.8M 2048M 18.2.2-954-g82316b04 b24e1baba637 37a05cedce31 2024-03-23T17:49:56.603 INFO:teuthology.orchestra.run.smithi086.stdout:mon.b smithi092 running (5m) 1s ago 5m 63.4M 2048M 18.2.2-954-g82316b04 b24e1baba637 0cc843b3a2db 2024-03-23T17:49:56.603 INFO:teuthology.orchestra.run.smithi086.stdout:mon.c smithi152 running (5m) 1s ago 5m 60.2M 2048M 18.2.2-954-g82316b04 b24e1baba637 1b607df0a2eb 2024-03-23T17:49:56.603 INFO:teuthology.orchestra.run.smithi086.stdout:osd.0 smithi086 running (5m) 0s ago 5m 62.0M 4317M 18.2.2-954-g82316b04 b24e1baba637 0a6372e9d5dc 2024-03-23T17:49:56.603 INFO:teuthology.orchestra.run.smithi086.stdout:osd.1 smithi092 running (4m) 1s ago 4m 67.5M 4278M 18.2.2-954-g82316b04 b24e1baba637 6f5e0fb8493e 2024-03-23T17:49:56.604 INFO:teuthology.orchestra.run.smithi086.stdout:osd.2 smithi152 running (4m) 1s ago 4m 78.9M 4278M 18.2.2-954-g82316b04 b24e1baba637 00c2714a453c 2024-03-23T17:49:56.604 INFO:teuthology.orchestra.run.smithi086.stdout:osd.3 smithi086 running (4m) 0s ago 4m 69.4M 4317M 18.2.2-954-g82316b04 b24e1baba637 dda1539f7dff 2024-03-23T17:49:56.604 INFO:teuthology.orchestra.run.smithi086.stdout:osd.4 smithi092 running (3m) 1s ago 3m 57.7M 4278M 18.2.2-954-g82316b04 b24e1baba637 3401cd1311a8 2024-03-23T17:49:56.604 INFO:teuthology.orchestra.run.smithi086.stdout:osd.5 smithi152 running (3m) 1s ago 3m 71.6M 4278M 18.2.2-954-g82316b04 b24e1baba637 a15010ed2700 2024-03-23T17:49:56.604 INFO:teuthology.orchestra.run.smithi086.stdout:osd.6 smithi086 running (3m) 0s ago 3m 66.8M 4317M 18.2.2-954-g82316b04 b24e1baba637 b42e19cf4457 2024-03-23T17:49:56.604 INFO:teuthology.orchestra.run.smithi086.stdout:osd.7 smithi092 running (3m) 1s ago 3m 59.5M 4278M 18.2.2-954-g82316b04 b24e1baba637 2cff5cf13c46 2024-03-23T17:49:56.604 INFO:teuthology.orchestra.run.smithi086.stdout:osd.8 smithi152 running (2m) 1s ago 2m 59.5M 4278M 18.2.2-954-g82316b04 b24e1baba637 13f784951c58 2024-03-23T17:49:56.604 INFO:teuthology.orchestra.run.smithi086.stdout:osd.9 smithi086 running (2m) 0s ago 2m 63.8M 4317M 18.2.2-954-g82316b04 b24e1baba637 be27053a2027 2024-03-23T17:49:56.604 INFO:teuthology.orchestra.run.smithi086.stdout:osd.10 smithi092 running (2m) 1s ago 2m 62.5M 4278M 18.2.2-954-g82316b04 b24e1baba637 f8d2f47c1341 2024-03-23T17:49:56.604 INFO:teuthology.orchestra.run.smithi086.stdout:osd.11 smithi152 running (106s) 1s ago 106s 63.7M 4278M 18.2.2-954-g82316b04 b24e1baba637 93b8711100bf 2024-03-23T17:49:56.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:56.604+0000 7f6992a00700 1 -- 172.21.15.86:0/2391152100 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f69700a1230 msgr2=0x7f69700a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:56.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:56.604+0000 7f6992a00700 1 --2- 172.21.15.86:0/2391152100 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f69700a1230 0x7f69700a36f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f698c074b10 tx=0x7f6978009260 comp rx=0 tx=0).stop 2024-03-23T17:49:56.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:56.604+0000 7f6992a00700 1 -- 172.21.15.86:0/2391152100 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f698c073810 msgr2=0x7f698c19d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:56.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:56.604+0000 7f6992a00700 1 --2- 172.21.15.86:0/2391152100 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f698c073810 0x7f698c19d370 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f697c000c00 tx=0x7f697c009580 comp rx=0 tx=0).stop 2024-03-23T17:49:56.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:56.604+0000 7f6992a00700 1 -- 172.21.15.86:0/2391152100 shutdown_connections 2024-03-23T17:49:56.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:56.604+0000 7f6992a00700 1 --2- 172.21.15.86:0/2391152100 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f69700a1230 0x7f69700a36f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:56.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:56.604+0000 7f6992a00700 1 --2- 172.21.15.86:0/2391152100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f698c10d3e0 0x7f698c1a48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:56.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:56.604+0000 7f6992a00700 1 --2- 172.21.15.86:0/2391152100 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f698c073810 0x7f698c19d370 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:56.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:56.604+0000 7f6992a00700 1 --2- 172.21.15.86:0/2391152100 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f698c074de0 0x7f698c19d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:56.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:56.604+0000 7f6992a00700 1 -- 172.21.15.86:0/2391152100 >> 172.21.15.86:0/2391152100 conn(0x7f698c0fc3c0 msgr2=0x7f698c100780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:56.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:56.604+0000 7f6992a00700 1 -- 172.21.15.86:0/2391152100 shutdown_connections 2024-03-23T17:49:56.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:56.604+0000 7f6992a00700 1 -- 172.21.15.86:0/2391152100 wait complete. 2024-03-23T17:49:56.953 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- bash -c 'ceph orch ls' 2024-03-23T17:49:57.329 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T17:49:57.766 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:57 smithi086 ceph-mon[38221]: pgmap v266: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-23T17:49:57.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:57 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:49:57.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:57 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:49:57.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:57 smithi086 ceph-mon[38221]: from='client.25052 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:49:57.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:57 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:49:57.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:57 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:49:57.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:57 smithi086 ceph-mon[38221]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:49:57.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:57 smithi152 ceph-mon[47949]: pgmap v266: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-23T17:49:57.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:57 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:49:57.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:57 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:49:57.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:57 smithi152 ceph-mon[47949]: from='client.25052 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:49:57.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:57 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:49:57.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:57 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:49:57.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:57 smithi152 ceph-mon[47949]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:49:58.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:57 smithi092 ceph-mon[79089]: pgmap v266: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-23T17:49:58.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:57 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:49:58.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:57 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:49:58.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:57 smithi092 ceph-mon[79089]: from='client.25052 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:49:58.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:57 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:49:58.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:57 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:49:58.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:57 smithi092 ceph-mon[79089]: from='mgr.14152 172.21.15.86:0/2655565582' entity='mgr.x' 2024-03-23T17:49:58.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:58.475+0000 7f3add400700 1 -- 172.21.15.86:0/4115496374 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3ad8100e00 msgr2=0x7f3ad8101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:58.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:58.475+0000 7f3add400700 1 --2- 172.21.15.86:0/4115496374 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3ad8100e00 0x7f3ad8101220 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3ac80098f0 tx=0x7f3ac8009c00 comp rx=0 tx=0).stop 2024-03-23T17:49:58.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:58.475+0000 7f3add400700 1 -- 172.21.15.86:0/4115496374 shutdown_connections 2024-03-23T17:49:58.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:58.475+0000 7f3add400700 1 --2- 172.21.15.86:0/4115496374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ad8102920 0x7f3ad8109240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:58.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:58.475+0000 7f3add400700 1 --2- 172.21.15.86:0/4115496374 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3ad8100e00 0x7f3ad8101220 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:58.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:58.475+0000 7f3add400700 1 --2- 172.21.15.86:0/4115496374 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ad8101f60 0x7f3ad81023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:58.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:58.475+0000 7f3add400700 1 -- 172.21.15.86:0/4115496374 >> 172.21.15.86:0/4115496374 conn(0x7f3ad80fc3c0 msgr2=0x7f3ad80fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:58.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:58.475+0000 7f3add400700 1 -- 172.21.15.86:0/4115496374 shutdown_connections 2024-03-23T17:49:58.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:58.475+0000 7f3add400700 1 -- 172.21.15.86:0/4115496374 wait complete. 2024-03-23T17:49:58.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:58.476+0000 7f3add400700 1 Processor -- start 2024-03-23T17:49:58.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:58.476+0000 7f3add400700 1 -- start start 2024-03-23T17:49:58.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:58.476+0000 7f3add400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ad8100e00 0x7f3ad8074b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:58.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:58.476+0000 7f3add400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ad8101f60 0x7f3ad80731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:58.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:58.476+0000 7f3add400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3ad8102920 0x7f3ad80736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:58.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:58.476+0000 7f3add400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad8073e80 con 0x7f3ad8101f60 2024-03-23T17:49:58.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:58.476+0000 7f3add400700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad8073ff0 con 0x7f3ad8100e00 2024-03-23T17:49:58.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:58.476+0000 7f3add400700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad81a0d40 con 0x7f3ad8102920 2024-03-23T17:49:58.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:58.477+0000 7f3ad5e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ad8101f60 0x7f3ad80731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:58.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:58.477+0000 7f3ad5e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ad8101f60 0x7f3ad80731a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55476/0 (socket says 172.21.15.86:55476) 2024-03-23T17:49:58.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:58.477+0000 7f3ad5e00700 1 -- 172.21.15.86:0/3487338196 learned_addr learned my addr 172.21.15.86:0/3487338196 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:49:58.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:58.477+0000 7f3ad6800700 1 --2- 172.21.15.86:0/3487338196 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ad8100e00 0x7f3ad8074b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:58.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:58.477+0000 7f3ad7200700 1 --2- 172.21.15.86:0/3487338196 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3ad8102920 0x7f3ad80736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:58.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:58.477+0000 7f3ad5e00700 1 -- 172.21.15.86:0/3487338196 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3ad8102920 msgr2=0x7f3ad80736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:58.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:58.477+0000 7f3ad5e00700 1 --2- 172.21.15.86:0/3487338196 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3ad8102920 0x7f3ad80736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:58.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:58.477+0000 7f3ad5e00700 1 -- 172.21.15.86:0/3487338196 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ad8100e00 msgr2=0x7f3ad8074b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:58.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:58.477+0000 7f3ad5e00700 1 --2- 172.21.15.86:0/3487338196 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ad8100e00 0x7f3ad8074b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:58.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:58.477+0000 7f3ad5e00700 1 -- 172.21.15.86:0/3487338196 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ad0009d00 con 0x7f3ad8101f60 2024-03-23T17:49:58.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:58.477+0000 7f3ad5e00700 1 --2- 172.21.15.86:0/3487338196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ad8101f60 0x7f3ad80731a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3ac400e9a0 tx=0x7f3ac400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:58.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:58.478+0000 7f3acf400700 1 -- 172.21.15.86:0/3487338196 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac4009790 con 0x7f3ad8101f60 2024-03-23T17:49:58.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:58.478+0000 7f3acf400700 1 -- 172.21.15.86:0/3487338196 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ac4010430 con 0x7f3ad8101f60 2024-03-23T17:49:58.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:58.478+0000 7f3add400700 1 -- 172.21.15.86:0/3487338196 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ac8009520 con 0x7f3ad8101f60 2024-03-23T17:49:58.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:58.478+0000 7f3acf400700 1 -- 172.21.15.86:0/3487338196 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac4018680 con 0x7f3ad8101f60 2024-03-23T17:49:58.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:58.478+0000 7f3add400700 1 -- 172.21.15.86:0/3487338196 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ad81a12b0 con 0x7f3ad8101f60 2024-03-23T17:49:58.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:58.479+0000 7f3add400700 1 -- 172.21.15.86:0/3487338196 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ad81082d0 con 0x7f3ad8101f60 2024-03-23T17:49:58.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:58.481+0000 7f3acf400700 1 -- 172.21.15.86:0/3487338196 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f3ac40105a0 con 0x7f3ad8101f60 2024-03-23T17:49:58.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:58.483+0000 7f3acf400700 1 --2- 172.21.15.86:0/3487338196 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f3abc0a11b0 0x7f3abc0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:49:58.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:58.483+0000 7f3acf400700 1 -- 172.21.15.86:0/3487338196 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f3ac4014070 con 0x7f3ad8101f60 2024-03-23T17:49:58.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:58.483+0000 7f3ad6800700 1 --2- 172.21.15.86:0/3487338196 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f3abc0a11b0 0x7f3abc0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:49:58.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:58.484+0000 7f3ad6800700 1 --2- 172.21.15.86:0/3487338196 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f3abc0a11b0 0x7f3abc0a3670 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8005e90 tx=0x7f3ac8019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:49:58.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:58.487+0000 7f3acf400700 1 -- 172.21.15.86:0/3487338196 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f3ac40529f0 con 0x7f3ad8101f60 2024-03-23T17:49:58.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:58.700+0000 7f3add400700 1 -- 172.21.15.86:0/3487338196 --> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f3ad802cd20 con 0x7f3abc0a11b0 2024-03-23T17:49:58.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:58.706+0000 7f3acf400700 1 -- 172.21.15.86:0/3487338196 <== mgr.14152 v2:172.21.15.86:6800/2734784244 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7f3ad802cd20 con 0x7f3abc0a11b0 2024-03-23T17:49:58.706 INFO:teuthology.orchestra.run.smithi086.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-03-23T17:49:58.706 INFO:teuthology.orchestra.run.smithi086.stdout:mds.all 12/12 3s ago 93s smithi086=a;smithi086=d;smithi086=g;smithi086=j;smithi092=b;smithi092=e;smithi092=h;smithi092=k;smithi152=c;smithi152=f;smithi152=i;smithi152=l;count:12 2024-03-23T17:49:58.706 INFO:teuthology.orchestra.run.smithi086.stdout:mgr 3/3 3s ago 5m smithi086=x;smithi092=y;smithi152=z;count:3 2024-03-23T17:49:58.706 INFO:teuthology.orchestra.run.smithi086.stdout:mon 3/3 3s ago 6m smithi086:172.21.15.86=a;smithi092:172.21.15.92=b;smithi152:172.21.15.152=c;count:3 2024-03-23T17:49:58.706 INFO:teuthology.orchestra.run.smithi086.stdout:osd 12 3s ago - 2024-03-23T17:49:58.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:58.708+0000 7f3add400700 1 -- 172.21.15.86:0/3487338196 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f3abc0a11b0 msgr2=0x7f3abc0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:58.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:58.708+0000 7f3add400700 1 --2- 172.21.15.86:0/3487338196 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f3abc0a11b0 0x7f3abc0a3670 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8005e90 tx=0x7f3ac8019040 comp rx=0 tx=0).stop 2024-03-23T17:49:58.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:58.708+0000 7f3add400700 1 -- 172.21.15.86:0/3487338196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ad8101f60 msgr2=0x7f3ad80731a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:49:58.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:58.708+0000 7f3add400700 1 --2- 172.21.15.86:0/3487338196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ad8101f60 0x7f3ad80731a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3ac400e9a0 tx=0x7f3ac400ecb0 comp rx=0 tx=0).stop 2024-03-23T17:49:58.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:58.708+0000 7f3add400700 1 -- 172.21.15.86:0/3487338196 shutdown_connections 2024-03-23T17:49:58.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:58.708+0000 7f3add400700 1 --2- 172.21.15.86:0/3487338196 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3ad8102920 0x7f3ad80736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:58.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:58.708+0000 7f3add400700 1 --2- 172.21.15.86:0/3487338196 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ad8100e00 0x7f3ad8074b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:58.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:58.708+0000 7f3add400700 1 --2- 172.21.15.86:0/3487338196 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f3abc0a11b0 0x7f3abc0a3670 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:58.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:58.708+0000 7f3add400700 1 --2- 172.21.15.86:0/3487338196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ad8101f60 0x7f3ad80731a0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:49:58.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:58.708+0000 7f3add400700 1 -- 172.21.15.86:0/3487338196 >> 172.21.15.86:0/3487338196 conn(0x7f3ad80fc3c0 msgr2=0x7f3ad8105410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:49:58.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:58.708+0000 7f3add400700 1 -- 172.21.15.86:0/3487338196 shutdown_connections 2024-03-23T17:49:58.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:49:58.708+0000 7f3add400700 1 -- 172.21.15.86:0/3487338196 wait complete. 2024-03-23T17:49:59.086 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- bash -c 'ceph orch host ls' 2024-03-23T17:49:59.443 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T17:49:59.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:59 smithi152 ceph-mon[47949]: pgmap v267: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-23T17:49:59.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:49:59 smithi152 ceph-mon[47949]: from='client.15018 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:49:59.933 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:59 smithi086 ceph-mon[38221]: pgmap v267: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-23T17:49:59.933 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:49:59 smithi086 ceph-mon[38221]: from='client.15018 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:50:00.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:59 smithi092 ceph-mon[79089]: pgmap v267: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-23T17:50:00.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:49:59 smithi092 ceph-mon[79089]: from='client.15018 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:50:00.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:00.628+0000 7f099ae00700 1 -- 172.21.15.86:0/168671790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0994102920 msgr2=0x7f0994109240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:00.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:00.628+0000 7f099ae00700 1 --2- 172.21.15.86:0/168671790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0994102920 0x7f0994109240 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f098c00b0a0 tx=0x7f098c00b3b0 comp rx=0 tx=0).stop 2024-03-23T17:50:00.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:00.628+0000 7f099ae00700 1 -- 172.21.15.86:0/168671790 shutdown_connections 2024-03-23T17:50:00.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:00.628+0000 7f099ae00700 1 --2- 172.21.15.86:0/168671790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0994102920 0x7f0994109240 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:00.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:00.628+0000 7f099ae00700 1 --2- 172.21.15.86:0/168671790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0994100e00 0x7f0994101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:00.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:00.628+0000 7f099ae00700 1 --2- 172.21.15.86:0/168671790 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0994101f60 0x7f09941023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:00.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:00.628+0000 7f099ae00700 1 -- 172.21.15.86:0/168671790 >> 172.21.15.86:0/168671790 conn(0x7f09940fc3c0 msgr2=0x7f09940fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:00.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:00.628+0000 7f099ae00700 1 -- 172.21.15.86:0/168671790 shutdown_connections 2024-03-23T17:50:00.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:00.628+0000 7f099ae00700 1 -- 172.21.15.86:0/168671790 wait complete. 2024-03-23T17:50:00.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:00.629+0000 7f099ae00700 1 Processor -- start 2024-03-23T17:50:00.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:00.629+0000 7f099ae00700 1 -- start start 2024-03-23T17:50:00.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:00.629+0000 7f099ae00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0994100e00 0x7f0994199030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:00.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:00.629+0000 7f099ae00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0994101f60 0x7f0994199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:00.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:00.629+0000 7f099ae00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0994102920 0x7f09941a05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:00.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:00.629+0000 7f099ae00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09941a0b70 con 0x7f0994100e00 2024-03-23T17:50:00.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:00.629+0000 7f099ae00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09941a0cb0 con 0x7f0994102920 2024-03-23T17:50:00.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:00.629+0000 7f099ae00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09941a0e40 con 0x7f0994101f60 2024-03-23T17:50:00.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:00.630+0000 7f0993e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0994100e00 0x7f0994199030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:00.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:00.630+0000 7f0993e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0994100e00 0x7f0994199030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55494/0 (socket says 172.21.15.86:55494) 2024-03-23T17:50:00.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:00.630+0000 7f0993e00700 1 -- 172.21.15.86:0/1121267956 learned_addr learned my addr 172.21.15.86:0/1121267956 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:00.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:00.630+0000 7f0998e00700 1 --2- 172.21.15.86:0/1121267956 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0994102920 0x7f09941a05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:00.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:00.630+0000 7f0993400700 1 --2- 172.21.15.86:0/1121267956 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0994101f60 0x7f0994199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:00.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:00.630+0000 7f0993e00700 1 -- 172.21.15.86:0/1121267956 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0994101f60 msgr2=0x7f0994199570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:00.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:00.630+0000 7f0993e00700 1 --2- 172.21.15.86:0/1121267956 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0994101f60 0x7f0994199570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:00.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:00.630+0000 7f0993e00700 1 -- 172.21.15.86:0/1121267956 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0994102920 msgr2=0x7f09941a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:00.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:00.630+0000 7f0993e00700 1 --2- 172.21.15.86:0/1121267956 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0994102920 0x7f09941a05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:00.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:00.630+0000 7f0993e00700 1 -- 172.21.15.86:0/1121267956 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f098c009d00 con 0x7f0994100e00 2024-03-23T17:50:00.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:00.630+0000 7f0993e00700 1 --2- 172.21.15.86:0/1121267956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0994100e00 0x7f0994199030 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f098400e9d0 tx=0x7f098400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:00.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:00.630+0000 7f0990c00700 1 -- 172.21.15.86:0/1121267956 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0984009790 con 0x7f0994100e00 2024-03-23T17:50:00.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:00.630+0000 7f0990c00700 1 -- 172.21.15.86:0/1121267956 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f098400ee70 con 0x7f0994100e00 2024-03-23T17:50:00.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:00.630+0000 7f099ae00700 1 -- 172.21.15.86:0/1121267956 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09941a1120 con 0x7f0994100e00 2024-03-23T17:50:00.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:00.630+0000 7f0990c00700 1 -- 172.21.15.86:0/1121267956 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0984010ea0 con 0x7f0994100e00 2024-03-23T17:50:00.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:00.631+0000 7f099ae00700 1 -- 172.21.15.86:0/1121267956 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09941a1670 con 0x7f0994100e00 2024-03-23T17:50:00.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:00.632+0000 7f099ae00700 1 -- 172.21.15.86:0/1121267956 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f099404ea80 con 0x7f0994100e00 2024-03-23T17:50:00.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:00.633+0000 7f0990c00700 1 -- 172.21.15.86:0/1121267956 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f098400cbd0 con 0x7f0994100e00 2024-03-23T17:50:00.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:00.635+0000 7f0990c00700 1 --2- 172.21.15.86:0/1121267956 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f09780a1230 0x7f09780a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:00.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:00.635+0000 7f0990c00700 1 -- 172.21.15.86:0/1121267956 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f0984014070 con 0x7f0994100e00 2024-03-23T17:50:00.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:00.635+0000 7f0993400700 1 --2- 172.21.15.86:0/1121267956 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f09780a1230 0x7f09780a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:00.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:00.635+0000 7f0993400700 1 --2- 172.21.15.86:0/1121267956 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f09780a1230 0x7f09780a36f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0980009c60 tx=0x7f0980009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:00.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:00.637+0000 7f0990c00700 1 -- 172.21.15.86:0/1121267956 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f0984096640 con 0x7f0994100e00 2024-03-23T17:50:00.766 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:00 smithi086 ceph-mon[38221]: overall HEALTH_OK 2024-03-23T17:50:00.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:00 smithi152 ceph-mon[47949]: overall HEALTH_OK 2024-03-23T17:50:00.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:00.847+0000 7f099ae00700 1 -- 172.21.15.86:0/1121267956 --> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f0994074700 con 0x7f09780a1230 2024-03-23T17:50:00.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:00.849+0000 7f0990c00700 1 -- 172.21.15.86:0/1121267956 <== mgr.14152 v2:172.21.15.86:6800/2734784244 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7f0994074700 con 0x7f09780a1230 2024-03-23T17:50:00.849 INFO:teuthology.orchestra.run.smithi086.stdout:HOST ADDR LABELS STATUS 2024-03-23T17:50:00.849 INFO:teuthology.orchestra.run.smithi086.stdout:smithi086 172.21.15.86 2024-03-23T17:50:00.849 INFO:teuthology.orchestra.run.smithi086.stdout:smithi092 172.21.15.92 2024-03-23T17:50:00.849 INFO:teuthology.orchestra.run.smithi086.stdout:smithi152 172.21.15.152 2024-03-23T17:50:00.849 INFO:teuthology.orchestra.run.smithi086.stdout:3 hosts in cluster 2024-03-23T17:50:00.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:00.851+0000 7f099ae00700 1 -- 172.21.15.86:0/1121267956 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f09780a1230 msgr2=0x7f09780a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:00.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:00.851+0000 7f099ae00700 1 --2- 172.21.15.86:0/1121267956 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f09780a1230 0x7f09780a36f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0980009c60 tx=0x7f0980009260 comp rx=0 tx=0).stop 2024-03-23T17:50:00.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:00.851+0000 7f099ae00700 1 -- 172.21.15.86:0/1121267956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0994100e00 msgr2=0x7f0994199030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:00.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:00.851+0000 7f099ae00700 1 --2- 172.21.15.86:0/1121267956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0994100e00 0x7f0994199030 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f098400e9d0 tx=0x7f098400ece0 comp rx=0 tx=0).stop 2024-03-23T17:50:00.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:00.851+0000 7f099ae00700 1 -- 172.21.15.86:0/1121267956 shutdown_connections 2024-03-23T17:50:00.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:00.851+0000 7f099ae00700 1 --2- 172.21.15.86:0/1121267956 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0994102920 0x7f09941a05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:00.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:00.851+0000 7f099ae00700 1 --2- 172.21.15.86:0/1121267956 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f09780a1230 0x7f09780a36f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:00.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:00.851+0000 7f099ae00700 1 --2- 172.21.15.86:0/1121267956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0994100e00 0x7f0994199030 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:00.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:00.851+0000 7f099ae00700 1 --2- 172.21.15.86:0/1121267956 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0994101f60 0x7f0994199570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:00.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:00.851+0000 7f099ae00700 1 -- 172.21.15.86:0/1121267956 >> 172.21.15.86:0/1121267956 conn(0x7f09940fc3c0 msgr2=0x7f0994072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:00.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:00.851+0000 7f099ae00700 1 -- 172.21.15.86:0/1121267956 shutdown_connections 2024-03-23T17:50:00.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:00.852+0000 7f099ae00700 1 -- 172.21.15.86:0/1121267956 wait complete. 2024-03-23T17:50:01.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:00 smithi092 ceph-mon[79089]: overall HEALTH_OK 2024-03-23T17:50:01.238 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- bash -c 'ceph orch device ls' 2024-03-23T17:50:01.597 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T17:50:02.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:01 smithi086 ceph-mon[38221]: pgmap v268: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-23T17:50:02.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:01 smithi092 ceph-mon[79089]: pgmap v268: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-23T17:50:02.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:01 smithi152 ceph-mon[47949]: pgmap v268: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-23T17:50:02.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:02.804+0000 7fc2fd400700 1 -- 172.21.15.86:0/4134360459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2f8100e00 msgr2=0x7fc2f8101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:02.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:02.804+0000 7fc2fd400700 1 --2- 172.21.15.86:0/4134360459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2f8100e00 0x7fc2f8101220 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc2e8009990 tx=0x7fc2e8009ca0 comp rx=0 tx=0).stop 2024-03-23T17:50:02.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:02.804+0000 7fc2fd400700 1 -- 172.21.15.86:0/4134360459 shutdown_connections 2024-03-23T17:50:02.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:02.804+0000 7fc2fd400700 1 --2- 172.21.15.86:0/4134360459 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc2f8102920 0x7fc2f8109240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:02.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:02.804+0000 7fc2fd400700 1 --2- 172.21.15.86:0/4134360459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2f8100e00 0x7fc2f8101220 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:02.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:02.804+0000 7fc2fd400700 1 --2- 172.21.15.86:0/4134360459 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2f8101f60 0x7fc2f81023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:02.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:02.804+0000 7fc2fd400700 1 -- 172.21.15.86:0/4134360459 >> 172.21.15.86:0/4134360459 conn(0x7fc2f80fc3c0 msgr2=0x7fc2f80fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:02.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:02.804+0000 7fc2fd400700 1 -- 172.21.15.86:0/4134360459 shutdown_connections 2024-03-23T17:50:02.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:02.804+0000 7fc2fd400700 1 -- 172.21.15.86:0/4134360459 wait complete. 2024-03-23T17:50:02.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:02.805+0000 7fc2fd400700 1 Processor -- start 2024-03-23T17:50:02.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:02.805+0000 7fc2fd400700 1 -- start start 2024-03-23T17:50:02.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:02.806+0000 7fc2fd400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2f8100e00 0x7fc2f8196d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:02.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:02.806+0000 7fc2fd400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc2f8101f60 0x7fc2f81972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:02.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:02.806+0000 7fc2fd400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2f8102920 0x7fc2f819e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:02.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:02.806+0000 7fc2fd400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f819e870 con 0x7fc2f8102920 2024-03-23T17:50:02.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:02.806+0000 7fc2fd400700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f819e9b0 con 0x7fc2f8100e00 2024-03-23T17:50:02.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:02.806+0000 7fc2fd400700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f819eb70 con 0x7fc2f8101f60 2024-03-23T17:50:02.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:02.806+0000 7fc2f6800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2f8100e00 0x7fc2f8196d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:02.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:02.806+0000 7fc2f6800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2f8100e00 0x7fc2f8196d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:45078/0 (socket says 172.21.15.86:45078) 2024-03-23T17:50:02.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:02.806+0000 7fc2f5e00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc2f8101f60 0x7fc2f81972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:02.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:02.806+0000 7fc2f6800700 1 -- 172.21.15.86:0/2918474589 learned_addr learned my addr 172.21.15.86:0/2918474589 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:02.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:02.806+0000 7fc2f7200700 1 --2- 172.21.15.86:0/2918474589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2f8102920 0x7fc2f819e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:02.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:02.806+0000 7fc2f6800700 1 -- 172.21.15.86:0/2918474589 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc2f8101f60 msgr2=0x7fc2f81972b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:02.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:02.806+0000 7fc2f6800700 1 --2- 172.21.15.86:0/2918474589 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc2f8101f60 0x7fc2f81972b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:02.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:02.806+0000 7fc2f6800700 1 -- 172.21.15.86:0/2918474589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2f8102920 msgr2=0x7fc2f819e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:02.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:02.806+0000 7fc2f6800700 1 --2- 172.21.15.86:0/2918474589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2f8102920 0x7fc2f819e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:02.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:02.806+0000 7fc2f6800700 1 -- 172.21.15.86:0/2918474589 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2e80095f0 con 0x7fc2f8100e00 2024-03-23T17:50:02.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:02.807+0000 7fc2f6800700 1 --2- 172.21.15.86:0/2918474589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2f8100e00 0x7fc2f8196d70 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc2e8005910 tx=0x7fc2e8005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:02.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:02.807+0000 7fc2ef400700 1 -- 172.21.15.86:0/2918474589 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e801d070 con 0x7fc2f8100e00 2024-03-23T17:50:02.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:02.807+0000 7fc2ef400700 1 -- 172.21.15.86:0/2918474589 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc2e8021410 con 0x7fc2f8100e00 2024-03-23T17:50:02.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:02.807+0000 7fc2fd400700 1 -- 172.21.15.86:0/2918474589 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2f819edf0 con 0x7fc2f8100e00 2024-03-23T17:50:02.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:02.807+0000 7fc2ef400700 1 -- 172.21.15.86:0/2918474589 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e8017650 con 0x7fc2f8100e00 2024-03-23T17:50:02.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:02.807+0000 7fc2fd400700 1 -- 172.21.15.86:0/2918474589 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2f819f1b0 con 0x7fc2f8100e00 2024-03-23T17:50:02.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:02.808+0000 7fc2fd400700 1 -- 172.21.15.86:0/2918474589 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2f81082d0 con 0x7fc2f8100e00 2024-03-23T17:50:02.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:02.810+0000 7fc2ef400700 1 -- 172.21.15.86:0/2918474589 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fc2e8021580 con 0x7fc2f8100e00 2024-03-23T17:50:02.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:02.811+0000 7fc2ef400700 1 --2- 172.21.15.86:0/2918474589 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fc2dc0a11b0 0x7fc2dc0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:02.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:02.811+0000 7fc2ef400700 1 -- 172.21.15.86:0/2918474589 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fc2e80cdef0 con 0x7fc2f8100e00 2024-03-23T17:50:02.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:02.811+0000 7fc2f5e00700 1 --2- 172.21.15.86:0/2918474589 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fc2dc0a11b0 0x7fc2dc0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:02.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:02.812+0000 7fc2f5e00700 1 --2- 172.21.15.86:0/2918474589 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fc2dc0a11b0 0x7fc2dc0a3670 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc2e4009c60 tx=0x7fc2e4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:02.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:02.814+0000 7fc2ef400700 1 -- 172.21.15.86:0/2918474589 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc2e80fb9f0 con 0x7fc2f8100e00 2024-03-23T17:50:03.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:02 smithi086 ceph-mon[38221]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:50:03.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:03.022+0000 7fc2fd400700 1 -- 172.21.15.86:0/2918474589 --> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fc2f802cdc0 con 0x7fc2dc0a11b0 2024-03-23T17:50:03.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:03.024+0000 7fc2ef400700 1 -- 172.21.15.86:0/2918474589 <== mgr.14152 v2:172.21.15.86:6800/2734784244 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7fc2f802cdc0 con 0x7fc2dc0a11b0 2024-03-23T17:50:03.025 INFO:teuthology.orchestra.run.smithi086.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-03-23T17:50:03.025 INFO:teuthology.orchestra.run.smithi086.stdout:smithi086 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400UL400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-03-23T17:50:03.025 INFO:teuthology.orchestra.run.smithi086.stdout:smithi092 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400QM400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-03-23T17:50:03.025 INFO:teuthology.orchestra.run.smithi086.stdout:smithi152 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300JT400BGN 372G No 107s ago Has a FileSystem, LVM detected 2024-03-23T17:50:03.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:03.026+0000 7fc2fd400700 1 -- 172.21.15.86:0/2918474589 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fc2dc0a11b0 msgr2=0x7fc2dc0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:03.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:03.026+0000 7fc2fd400700 1 --2- 172.21.15.86:0/2918474589 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fc2dc0a11b0 0x7fc2dc0a3670 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc2e4009c60 tx=0x7fc2e4009260 comp rx=0 tx=0).stop 2024-03-23T17:50:03.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:03.026+0000 7fc2fd400700 1 -- 172.21.15.86:0/2918474589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2f8100e00 msgr2=0x7fc2f8196d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:03.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:03.026+0000 7fc2fd400700 1 --2- 172.21.15.86:0/2918474589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2f8100e00 0x7fc2f8196d70 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc2e8005910 tx=0x7fc2e8005740 comp rx=0 tx=0).stop 2024-03-23T17:50:03.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:03.026+0000 7fc2fd400700 1 -- 172.21.15.86:0/2918474589 shutdown_connections 2024-03-23T17:50:03.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:03.026+0000 7fc2fd400700 1 --2- 172.21.15.86:0/2918474589 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fc2dc0a11b0 0x7fc2dc0a3670 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:03.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:03.026+0000 7fc2fd400700 1 --2- 172.21.15.86:0/2918474589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2f8102920 0x7fc2f819e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:03.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:03.026+0000 7fc2fd400700 1 --2- 172.21.15.86:0/2918474589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2f8100e00 0x7fc2f8196d70 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:03.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:03.026+0000 7fc2fd400700 1 --2- 172.21.15.86:0/2918474589 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc2f8101f60 0x7fc2f81972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:03.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:03.026+0000 7fc2fd400700 1 -- 172.21.15.86:0/2918474589 >> 172.21.15.86:0/2918474589 conn(0x7fc2f80fc3c0 msgr2=0x7fc2f8105410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:03.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:03.027+0000 7fc2fd400700 1 -- 172.21.15.86:0/2918474589 shutdown_connections 2024-03-23T17:50:03.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:03.027+0000 7fc2fd400700 1 -- 172.21.15.86:0/2918474589 wait complete. 2024-03-23T17:50:03.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:02 smithi092 ceph-mon[79089]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:50:03.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:02 smithi152 ceph-mon[47949]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:50:03.465 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-23T17:50:03.475 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi086.front.sepia.ceph.com 2024-03-23T17:50:03.475 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-23T17:50:03.766 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:03 smithi086 ceph-mon[38221]: pgmap v269: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-23T17:50:03.834 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T17:50:04.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:03 smithi092 ceph-mon[79089]: pgmap v269: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-23T17:50:04.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:03 smithi152 ceph-mon[47949]: pgmap v269: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-23T17:50:05.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:04 smithi086 ceph-mon[38221]: from='client.25070 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:50:05.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:05.025+0000 7f19d1600700 1 -- 172.21.15.86:0/3870137570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19cc101e20 msgr2=0x7f19cc102280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:05.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:05.025+0000 7f19d1600700 1 --2- 172.21.15.86:0/3870137570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19cc101e20 0x7f19cc102280 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f19b80099c0 tx=0x7f19b8009cd0 comp rx=0 tx=0).stop 2024-03-23T17:50:05.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:05.025+0000 7f19d1600700 1 -- 172.21.15.86:0/3870137570 shutdown_connections 2024-03-23T17:50:05.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:05.025+0000 7f19d1600700 1 --2- 172.21.15.86:0/3870137570 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19cc1027c0 0x7f19cc109070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:05.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:05.025+0000 7f19d1600700 1 --2- 172.21.15.86:0/3870137570 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f19cc100c20 0x7f19cc101040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:05.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:05.025+0000 7f19d1600700 1 --2- 172.21.15.86:0/3870137570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19cc101e20 0x7f19cc102280 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:05.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:05.025+0000 7f19d1600700 1 -- 172.21.15.86:0/3870137570 >> 172.21.15.86:0/3870137570 conn(0x7f19cc0fc1c0 msgr2=0x7f19cc0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:05.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:05.025+0000 7f19d1600700 1 -- 172.21.15.86:0/3870137570 shutdown_connections 2024-03-23T17:50:05.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:05.025+0000 7f19d1600700 1 -- 172.21.15.86:0/3870137570 wait complete. 2024-03-23T17:50:05.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:05.026+0000 7f19d1600700 1 Processor -- start 2024-03-23T17:50:05.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:05.026+0000 7f19d1600700 1 -- start start 2024-03-23T17:50:05.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:05.026+0000 7f19d1600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19cc100c20 0x7f19cc074b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:05.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:05.026+0000 7f19d1600700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19cc101e20 0x7f19cc0731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:05.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:05.026+0000 7f19d1600700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f19cc1027c0 0x7f19cc0736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:05.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:05.026+0000 7f19d1600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19cc073e80 con 0x7f19cc100c20 2024-03-23T17:50:05.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:05.026+0000 7f19d1600700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19cc073ff0 con 0x7f19cc101e20 2024-03-23T17:50:05.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:05.026+0000 7f19d1600700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19cc1a0bc0 con 0x7f19cc1027c0 2024-03-23T17:50:05.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:05.027+0000 7f19cbe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19cc100c20 0x7f19cc074b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:05.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:05.027+0000 7f19cb400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19cc101e20 0x7f19cc0731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:05.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:05.027+0000 7f19cbe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19cc100c20 0x7f19cc074b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37908/0 (socket says 172.21.15.86:37908) 2024-03-23T17:50:05.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:05.027+0000 7f19cbe00700 1 -- 172.21.15.86:0/2095179463 learned_addr learned my addr 172.21.15.86:0/2095179463 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:05.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:05.027+0000 7f19d0c00700 1 --2- 172.21.15.86:0/2095179463 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f19cc1027c0 0x7f19cc0736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:05.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:05.027+0000 7f19cbe00700 1 -- 172.21.15.86:0/2095179463 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f19cc1027c0 msgr2=0x7f19cc0736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:05.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:05.027+0000 7f19cbe00700 1 --2- 172.21.15.86:0/2095179463 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f19cc1027c0 0x7f19cc0736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:05.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:05.027+0000 7f19cbe00700 1 -- 172.21.15.86:0/2095179463 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19cc101e20 msgr2=0x7f19cc0731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:05.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:05.027+0000 7f19cbe00700 1 --2- 172.21.15.86:0/2095179463 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19cc101e20 0x7f19cc0731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:05.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:05.027+0000 7f19cbe00700 1 -- 172.21.15.86:0/2095179463 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19b80095f0 con 0x7f19cc100c20 2024-03-23T17:50:05.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:05.027+0000 7f19d0c00700 1 --2- 172.21.15.86:0/2095179463 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f19cc1027c0 0x7f19cc0736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:50:05.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:05.027+0000 7f19cbe00700 1 --2- 172.21.15.86:0/2095179463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19cc100c20 0x7f19cc074b00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f19bc00d980 tx=0x7f19bc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:05.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:05.028+0000 7f19c8c00700 1 -- 172.21.15.86:0/2095179463 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19bc00b950 con 0x7f19cc100c20 2024-03-23T17:50:05.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:05.028+0000 7f19d1600700 1 -- 172.21.15.86:0/2095179463 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19cc1a0e50 con 0x7f19cc100c20 2024-03-23T17:50:05.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:05.028+0000 7f19c8c00700 1 -- 172.21.15.86:0/2095179463 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19bc007620 con 0x7f19cc100c20 2024-03-23T17:50:05.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:05.028+0000 7f19c8c00700 1 -- 172.21.15.86:0/2095179463 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19bc00f3c0 con 0x7f19cc100c20 2024-03-23T17:50:05.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:05.028+0000 7f19d1600700 1 -- 172.21.15.86:0/2095179463 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19cc1a1270 con 0x7f19cc100c20 2024-03-23T17:50:05.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:05.029+0000 7f19d1600700 1 -- 172.21.15.86:0/2095179463 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19cc108050 con 0x7f19cc100c20 2024-03-23T17:50:05.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:05.031+0000 7f19c8c00700 1 -- 172.21.15.86:0/2095179463 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f19bc010640 con 0x7f19cc100c20 2024-03-23T17:50:05.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:05.032+0000 7f19c8c00700 1 --2- 172.21.15.86:0/2095179463 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f19b00a11b0 0x7f19b00a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:05.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:05.032+0000 7f19c8c00700 1 -- 172.21.15.86:0/2095179463 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f19bc0cc930 con 0x7f19cc100c20 2024-03-23T17:50:05.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:05.032+0000 7f19cb400700 1 --2- 172.21.15.86:0/2095179463 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f19b00a11b0 0x7f19b00a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:05.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:05.033+0000 7f19cb400700 1 --2- 172.21.15.86:0/2095179463 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f19b00a11b0 0x7f19b00a3670 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f19b8006010 tx=0x7f19b80037c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:05.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:05.036+0000 7f19c8c00700 1 -- 172.21.15.86:0/2095179463 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f19bc095f30 con 0x7f19cc100c20 2024-03-23T17:50:05.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:04 smithi092 ceph-mon[79089]: from='client.25070 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:50:05.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:04 smithi152 ceph-mon[47949]: from='client.25070 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:50:05.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:05.283+0000 7f19d1600700 1 -- 172.21.15.86:0/2095179463 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f19cc02ccb0 con 0x7f19cc100c20 2024-03-23T17:50:05.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:05.283+0000 7f19c8c00700 1 -- 172.21.15.86:0/2095179463 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) v1 ==== 76+0+2923 (secure 0 0 0) 0x7f19bc016070 con 0x7f19cc100c20 2024-03-23T17:50:05.284 INFO:teuthology.orchestra.run.smithi086.stdout:e27 2024-03-23T17:50:05.284 INFO:teuthology.orchestra.run.smithi086.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-23T17:50:05.284 INFO:teuthology.orchestra.run.smithi086.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-23T17:50:05.284 INFO:teuthology.orchestra.run.smithi086.stdout:legacy client fscid: 1 2024-03-23T17:50:05.284 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:50:05.284 INFO:teuthology.orchestra.run.smithi086.stdout:Filesystem 'cephfs' (1) 2024-03-23T17:50:05.284 INFO:teuthology.orchestra.run.smithi086.stdout:fs_name cephfs 2024-03-23T17:50:05.284 INFO:teuthology.orchestra.run.smithi086.stdout:epoch 27 2024-03-23T17:50:05.284 INFO:teuthology.orchestra.run.smithi086.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-23T17:50:05.284 INFO:teuthology.orchestra.run.smithi086.stdout:created 2024-03-23T17:48:56.202323+0000 2024-03-23T17:50:05.284 INFO:teuthology.orchestra.run.smithi086.stdout:modified 2024-03-23T17:49:03.703276+0000 2024-03-23T17:50:05.284 INFO:teuthology.orchestra.run.smithi086.stdout:tableserver 0 2024-03-23T17:50:05.285 INFO:teuthology.orchestra.run.smithi086.stdout:root 0 2024-03-23T17:50:05.285 INFO:teuthology.orchestra.run.smithi086.stdout:session_timeout 300 2024-03-23T17:50:05.285 INFO:teuthology.orchestra.run.smithi086.stdout:session_autoclose 300 2024-03-23T17:50:05.285 INFO:teuthology.orchestra.run.smithi086.stdout:max_file_size 1099511627776 2024-03-23T17:50:05.285 INFO:teuthology.orchestra.run.smithi086.stdout:max_xattr_size 65536 2024-03-23T17:50:05.285 INFO:teuthology.orchestra.run.smithi086.stdout:required_client_features {} 2024-03-23T17:50:05.285 INFO:teuthology.orchestra.run.smithi086.stdout:last_failure 0 2024-03-23T17:50:05.285 INFO:teuthology.orchestra.run.smithi086.stdout:last_failure_osd_epoch 0 2024-03-23T17:50:05.285 INFO:teuthology.orchestra.run.smithi086.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-23T17:50:05.285 INFO:teuthology.orchestra.run.smithi086.stdout:max_mds 5 2024-03-23T17:50:05.285 INFO:teuthology.orchestra.run.smithi086.stdout:in 0,1,2,3,4 2024-03-23T17:50:05.285 INFO:teuthology.orchestra.run.smithi086.stdout:up {0=24563,1=24553,2=24518,3=24505,4=24500} 2024-03-23T17:50:05.285 INFO:teuthology.orchestra.run.smithi086.stdout:failed 2024-03-23T17:50:05.286 INFO:teuthology.orchestra.run.smithi086.stdout:damaged 2024-03-23T17:50:05.286 INFO:teuthology.orchestra.run.smithi086.stdout:stopped 2024-03-23T17:50:05.286 INFO:teuthology.orchestra.run.smithi086.stdout:data_pools [3] 2024-03-23T17:50:05.286 INFO:teuthology.orchestra.run.smithi086.stdout:metadata_pool 2 2024-03-23T17:50:05.286 INFO:teuthology.orchestra.run.smithi086.stdout:inline_data disabled 2024-03-23T17:50:05.286 INFO:teuthology.orchestra.run.smithi086.stdout:balancer 2024-03-23T17:50:05.286 INFO:teuthology.orchestra.run.smithi086.stdout:bal_rank_mask -1 2024-03-23T17:50:05.286 INFO:teuthology.orchestra.run.smithi086.stdout:standby_count_wanted 0 2024-03-23T17:50:05.286 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.b{0:24563} state up:active seq 3 addr [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] compat {c=[1],r=[1],i=[7ff]}] 2024-03-23T17:50:05.286 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.c{0:24538} state up:standby-replay seq 1 addr [v2:172.21.15.152:6836/1151208965,v1:172.21.15.152:6837/1151208965] compat {c=[1],r=[1],i=[7ff]}] 2024-03-23T17:50:05.286 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.i{1:24553} state up:active seq 6 addr [v2:172.21.15.152:6838/1100723992,v1:172.21.15.152:6839/1100723992] compat {c=[1],r=[1],i=[7ff]}] 2024-03-23T17:50:05.286 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.f{1:24526} state up:standby-replay seq 1 addr [v2:172.21.15.152:6834/3137085021,v1:172.21.15.152:6835/3137085021] compat {c=[1],r=[1],i=[7ff]}] 2024-03-23T17:50:05.286 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.h{2:24518} state up:active seq 6 addr [v2:172.21.15.92:6836/3967837948,v1:172.21.15.92:6837/3967837948] compat {c=[1],r=[1],i=[7ff]}] 2024-03-23T17:50:05.286 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.l{2:24475} state up:standby-replay seq 1 addr [v2:172.21.15.152:6832/3075554623,v1:172.21.15.152:6833/3075554623] compat {c=[1],r=[1],i=[7ff]}] 2024-03-23T17:50:05.287 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.d{3:24505} state up:active seq 8 addr [v2:172.21.15.86:6836/2933881328,v1:172.21.15.86:6837/2933881328] compat {c=[1],r=[1],i=[7ff]}] 2024-03-23T17:50:05.287 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.k{3:24464} state up:standby-replay seq 1 addr [v2:172.21.15.92:6832/1467091186,v1:172.21.15.92:6833/1467091186] compat {c=[1],r=[1],i=[7ff]}] 2024-03-23T17:50:05.287 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.e{4:24500} state up:active seq 8 addr [v2:172.21.15.92:6834/908900027,v1:172.21.15.92:6835/908900027] compat {c=[1],r=[1],i=[7ff]}] 2024-03-23T17:50:05.287 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.a{4:14514} state up:standby-replay seq 1 addr [v2:172.21.15.86:6840/3026943866,v1:172.21.15.86:6841/3026943866] compat {c=[1],r=[1],i=[7ff]}] 2024-03-23T17:50:05.287 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:50:05.287 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:50:05.287 INFO:teuthology.orchestra.run.smithi086.stdout:Standby daemons: 2024-03-23T17:50:05.287 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:50:05.287 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.j{-1:14445} state up:standby seq 1 addr [v2:172.21.15.86:6834/1433140219,v1:172.21.15.86:6835/1433140219] compat {c=[1],r=[1],i=[7ff]}] 2024-03-23T17:50:05.287 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.g{-1:14502} state up:standby seq 1 addr [v2:172.21.15.86:6838/1879457730,v1:172.21.15.86:6839/1879457730] compat {c=[1],r=[1],i=[7ff]}] 2024-03-23T17:50:05.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:05.285+0000 7f19d1600700 1 -- 172.21.15.86:0/2095179463 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f19b00a11b0 msgr2=0x7f19b00a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:05.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:05.285+0000 7f19d1600700 1 --2- 172.21.15.86:0/2095179463 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f19b00a11b0 0x7f19b00a3670 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f19b8006010 tx=0x7f19b80037c0 comp rx=0 tx=0).stop 2024-03-23T17:50:05.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:05.285+0000 7f19d1600700 1 -- 172.21.15.86:0/2095179463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19cc100c20 msgr2=0x7f19cc074b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:05.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:05.286+0000 7f19d1600700 1 --2- 172.21.15.86:0/2095179463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19cc100c20 0x7f19cc074b00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f19bc00d980 tx=0x7f19bc00dc90 comp rx=0 tx=0).stop 2024-03-23T17:50:05.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:05.286+0000 7f19d1600700 1 -- 172.21.15.86:0/2095179463 shutdown_connections 2024-03-23T17:50:05.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:05.286+0000 7f19d1600700 1 --2- 172.21.15.86:0/2095179463 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f19cc1027c0 0x7f19cc0736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:05.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:05.286+0000 7f19d1600700 1 --2- 172.21.15.86:0/2095179463 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f19b00a11b0 0x7f19b00a3670 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:05.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:05.286+0000 7f19d1600700 1 --2- 172.21.15.86:0/2095179463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19cc100c20 0x7f19cc074b00 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:05.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:05.286+0000 7f19d1600700 1 --2- 172.21.15.86:0/2095179463 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19cc101e20 0x7f19cc0731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:05.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:05.286+0000 7f19d1600700 1 -- 172.21.15.86:0/2095179463 >> 172.21.15.86:0/2095179463 conn(0x7f19cc0fc1c0 msgr2=0x7f19cc105190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:05.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:05.286+0000 7f19d1600700 1 -- 172.21.15.86:0/2095179463 shutdown_connections 2024-03-23T17:50:05.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:05.286+0000 7f19d1600700 1 -- 172.21.15.86:0/2095179463 wait complete. 2024-03-23T17:50:05.289 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 27 2024-03-23T17:50:05.654 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- bash -c 'ceph osd dump' 2024-03-23T17:50:06.015 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T17:50:06.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:05 smithi086 ceph-mon[38221]: pgmap v270: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-23T17:50:06.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:05 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/2095179463' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-23T17:50:06.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:05 smithi092 ceph-mon[79089]: pgmap v270: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-23T17:50:06.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:05 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/2095179463' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-23T17:50:06.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:05 smithi152 ceph-mon[47949]: pgmap v270: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-23T17:50:06.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:05 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/2095179463' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-23T17:50:07.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.107+0000 7efc6c400700 1 -- 172.21.15.86:0/2608055200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc64102920 msgr2=0x7efc64109240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:07.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.107+0000 7efc6c400700 1 --2- 172.21.15.86:0/2608055200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc64102920 0x7efc64109240 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7efc5c00b0a0 tx=0x7efc5c00b3b0 comp rx=0 tx=0).stop 2024-03-23T17:50:07.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.107+0000 7efc6c400700 1 -- 172.21.15.86:0/2608055200 shutdown_connections 2024-03-23T17:50:07.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.107+0000 7efc6c400700 1 --2- 172.21.15.86:0/2608055200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc64102920 0x7efc64109240 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:07.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.107+0000 7efc6c400700 1 --2- 172.21.15.86:0/2608055200 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc64100e00 0x7efc64101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:07.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.107+0000 7efc6c400700 1 --2- 172.21.15.86:0/2608055200 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc64101f60 0x7efc641023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:07.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.107+0000 7efc6c400700 1 -- 172.21.15.86:0/2608055200 >> 172.21.15.86:0/2608055200 conn(0x7efc640fc3c0 msgr2=0x7efc640fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:07.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.107+0000 7efc6c400700 1 -- 172.21.15.86:0/2608055200 shutdown_connections 2024-03-23T17:50:07.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.107+0000 7efc6c400700 1 -- 172.21.15.86:0/2608055200 wait complete. 2024-03-23T17:50:07.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.108+0000 7efc6c400700 1 Processor -- start 2024-03-23T17:50:07.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.108+0000 7efc6c400700 1 -- start start 2024-03-23T17:50:07.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.109+0000 7efc6c400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc64100e00 0x7efc64196e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:07.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.109+0000 7efc6c400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc64101f60 0x7efc641973d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:07.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.109+0000 7efc6c400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc64102920 0x7efc6419e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:07.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.109+0000 7efc6c400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc6419e980 con 0x7efc64101f60 2024-03-23T17:50:07.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.109+0000 7efc6c400700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc6419eac0 con 0x7efc64100e00 2024-03-23T17:50:07.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.109+0000 7efc6c400700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc6419ec50 con 0x7efc64102920 2024-03-23T17:50:07.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.109+0000 7efc69000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc64101f60 0x7efc641973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:07.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.109+0000 7efc69a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc64100e00 0x7efc64196e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:07.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.109+0000 7efc69000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc64101f60 0x7efc641973d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37912/0 (socket says 172.21.15.86:37912) 2024-03-23T17:50:07.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.109+0000 7efc69a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc64100e00 0x7efc64196e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:54396/0 (socket says 172.21.15.86:54396) 2024-03-23T17:50:07.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.109+0000 7efc69000700 1 -- 172.21.15.86:0/257606367 learned_addr learned my addr 172.21.15.86:0/257606367 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:07.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.109+0000 7efc6a400700 1 --2- 172.21.15.86:0/257606367 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc64102920 0x7efc6419e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:07.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.109+0000 7efc69a00700 1 -- 172.21.15.86:0/257606367 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc64102920 msgr2=0x7efc6419e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:07.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.109+0000 7efc69a00700 1 --2- 172.21.15.86:0/257606367 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc64102920 0x7efc6419e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:07.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.109+0000 7efc69a00700 1 -- 172.21.15.86:0/257606367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc64101f60 msgr2=0x7efc641973d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:07.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.109+0000 7efc69a00700 1 --2- 172.21.15.86:0/257606367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc64101f60 0x7efc641973d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:07.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.110+0000 7efc69a00700 1 -- 172.21.15.86:0/257606367 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc5c009d00 con 0x7efc64100e00 2024-03-23T17:50:07.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.110+0000 7efc69a00700 1 --2- 172.21.15.86:0/257606367 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc64100e00 0x7efc64196e90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7efc5800e9f0 tx=0x7efc5800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:07.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.110+0000 7efc62000700 1 -- 172.21.15.86:0/257606367 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc58004220 con 0x7efc64100e00 2024-03-23T17:50:07.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.110+0000 7efc6c400700 1 -- 172.21.15.86:0/257606367 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc6419ef30 con 0x7efc64100e00 2024-03-23T17:50:07.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.110+0000 7efc62000700 1 -- 172.21.15.86:0/257606367 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc58004cf0 con 0x7efc64100e00 2024-03-23T17:50:07.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.110+0000 7efc62000700 1 -- 172.21.15.86:0/257606367 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc58005020 con 0x7efc64100e00 2024-03-23T17:50:07.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.110+0000 7efc6c400700 1 -- 172.21.15.86:0/257606367 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc6419f340 con 0x7efc64100e00 2024-03-23T17:50:07.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.111+0000 7efc6c400700 1 -- 172.21.15.86:0/257606367 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc641081b0 con 0x7efc64100e00 2024-03-23T17:50:07.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.113+0000 7efc62000700 1 -- 172.21.15.86:0/257606367 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7efc58004750 con 0x7efc64100e00 2024-03-23T17:50:07.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.114+0000 7efc62000700 1 --2- 172.21.15.86:0/257606367 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7efc4c0a11b0 0x7efc4c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:07.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.115+0000 7efc62000700 1 -- 172.21.15.86:0/257606367 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7efc58014070 con 0x7efc64100e00 2024-03-23T17:50:07.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.115+0000 7efc69000700 1 --2- 172.21.15.86:0/257606367 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7efc4c0a11b0 0x7efc4c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:07.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.115+0000 7efc69000700 1 --2- 172.21.15.86:0/257606367 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7efc4c0a11b0 0x7efc4c0a3670 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7efc50009d20 tx=0x7efc50009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:07.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.117+0000 7efc62000700 1 -- 172.21.15.86:0/257606367 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7efc58095ff0 con 0x7efc64100e00 2024-03-23T17:50:07.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.321+0000 7efc6c400700 1 -- 172.21.15.86:0/257606367 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7efc6410c680 con 0x7efc64100e00 2024-03-23T17:50:07.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.322+0000 7efc62000700 1 -- 172.21.15.86:0/257606367 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v82) v1 ==== 56+0+5245 (secure 0 0 0) 0x7efc58004e60 con 0x7efc64100e00 2024-03-23T17:50:07.322 INFO:teuthology.orchestra.run.smithi086.stdout:epoch 82 2024-03-23T17:50:07.323 INFO:teuthology.orchestra.run.smithi086.stdout:fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 2024-03-23T17:50:07.323 INFO:teuthology.orchestra.run.smithi086.stdout:created 2024-03-23T17:42:30.097233+0000 2024-03-23T17:50:07.323 INFO:teuthology.orchestra.run.smithi086.stdout:modified 2024-03-23T17:49:29.773954+0000 2024-03-23T17:50:07.323 INFO:teuthology.orchestra.run.smithi086.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-03-23T17:50:07.323 INFO:teuthology.orchestra.run.smithi086.stdout:crush_version 26 2024-03-23T17:50:07.323 INFO:teuthology.orchestra.run.smithi086.stdout:full_ratio 0.95 2024-03-23T17:50:07.323 INFO:teuthology.orchestra.run.smithi086.stdout:backfillfull_ratio 0.9 2024-03-23T17:50:07.323 INFO:teuthology.orchestra.run.smithi086.stdout:nearfull_ratio 0.85 2024-03-23T17:50:07.323 INFO:teuthology.orchestra.run.smithi086.stdout:require_min_compat_client luminous 2024-03-23T17:50:07.323 INFO:teuthology.orchestra.run.smithi086.stdout:min_compat_client luminous 2024-03-23T17:50:07.323 INFO:teuthology.orchestra.run.smithi086.stdout:require_osd_release reef 2024-03-23T17:50:07.323 INFO:teuthology.orchestra.run.smithi086.stdout:stretch_mode_enabled false 2024-03-23T17:50:07.323 INFO:teuthology.orchestra.run.smithi086.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-03-23T17:50:07.324 INFO:teuthology.orchestra.run.smithi086.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 79 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2024-03-23T17:50:07.324 INFO:teuthology.orchestra.run.smithi086.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 80 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-03-23T17:50:07.324 INFO:teuthology.orchestra.run.smithi086.stdout:max_osd 12 2024-03-23T17:50:07.324 INFO:teuthology.orchestra.run.smithi086.stdout:osd.0 up in weight 1 up_from 8 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.86:6802/2423505491,v1:172.21.15.86:6803/2423505491] [v2:172.21.15.86:6804/2423505491,v1:172.21.15.86:6805/2423505491] exists,up 656655c6-1c7f-4640-92c4-2f42de820143 2024-03-23T17:50:07.324 INFO:teuthology.orchestra.run.smithi086.stdout:osd.1 up in weight 1 up_from 13 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6800/3776252597,v1:172.21.15.92:6801/3776252597] [v2:172.21.15.92:6802/3776252597,v1:172.21.15.92:6803/3776252597] exists,up 3e86dc65-20a6-4de8-b9ce-fcf01f68862b 2024-03-23T17:50:07.324 INFO:teuthology.orchestra.run.smithi086.stdout:osd.2 up in weight 1 up_from 18 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.152:6800/1601265891,v1:172.21.15.152:6801/1601265891] [v2:172.21.15.152:6802/1601265891,v1:172.21.15.152:6803/1601265891] exists,up 09808786-3bd3-4d11-8c7b-1736119dfbb4 2024-03-23T17:50:07.324 INFO:teuthology.orchestra.run.smithi086.stdout:osd.3 up in weight 1 up_from 26 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.86:6810/694846466,v1:172.21.15.86:6811/694846466] [v2:172.21.15.86:6812/694846466,v1:172.21.15.86:6813/694846466] exists,up 79c61910-4b53-424c-b150-6f5385a2cd68 2024-03-23T17:50:07.324 INFO:teuthology.orchestra.run.smithi086.stdout:osd.4 up in weight 1 up_from 33 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6808/1422821471,v1:172.21.15.92:6809/1422821471] [v2:172.21.15.92:6810/1422821471,v1:172.21.15.92:6811/1422821471] exists,up a2878035-0179-469f-bbd4-8ce2d3f6bddc 2024-03-23T17:50:07.324 INFO:teuthology.orchestra.run.smithi086.stdout:osd.5 up in weight 1 up_from 38 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.152:6808/1039375239,v1:172.21.15.152:6809/1039375239] [v2:172.21.15.152:6810/1039375239,v1:172.21.15.152:6811/1039375239] exists,up 837366aa-ce25-42ca-b157-5bbb50159d0a 2024-03-23T17:50:07.324 INFO:teuthology.orchestra.run.smithi086.stdout:osd.6 up in weight 1 up_from 45 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.86:6818/4080452707,v1:172.21.15.86:6819/4080452707] [v2:172.21.15.86:6820/4080452707,v1:172.21.15.86:6821/4080452707] exists,up 61024437-7b91-432f-a1af-6443d0ef5213 2024-03-23T17:50:07.324 INFO:teuthology.orchestra.run.smithi086.stdout:osd.7 up in weight 1 up_from 49 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6816/4187707622,v1:172.21.15.92:6817/4187707622] [v2:172.21.15.92:6818/4187707622,v1:172.21.15.92:6819/4187707622] exists,up 2684bdae-6b30-4afc-af68-4ce41b3a75b6 2024-03-23T17:50:07.324 INFO:teuthology.orchestra.run.smithi086.stdout:osd.8 up in weight 1 up_from 56 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.152:6816/3237612748,v1:172.21.15.152:6817/3237612748] [v2:172.21.15.152:6818/3237612748,v1:172.21.15.152:6819/3237612748] exists,up f03bbf0d-1281-4bcf-9773-9854388e65ed 2024-03-23T17:50:07.324 INFO:teuthology.orchestra.run.smithi086.stdout:osd.9 up in weight 1 up_from 63 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.86:6826/2835218294,v1:172.21.15.86:6827/2835218294] [v2:172.21.15.86:6828/2835218294,v1:172.21.15.86:6829/2835218294] exists,up e7e8b775-99ff-48a4-a23a-91e7ab6622bb 2024-03-23T17:50:07.324 INFO:teuthology.orchestra.run.smithi086.stdout:osd.10 up in weight 1 up_from 68 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6824/4259465100,v1:172.21.15.92:6825/4259465100] [v2:172.21.15.92:6826/4259465100,v1:172.21.15.92:6827/4259465100] exists,up d9d616cd-a229-44de-8768-02a4415df115 2024-03-23T17:50:07.325 INFO:teuthology.orchestra.run.smithi086.stdout:osd.11 up in weight 1 up_from 75 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.152:6824/194662760,v1:172.21.15.152:6825/194662760] [v2:172.21.15.152:6826/194662760,v1:172.21.15.152:6827/194662760] exists,up e7c0b865-ccf5-4233-8d22-0d9b6dbe5c97 2024-03-23T17:50:07.325 INFO:teuthology.orchestra.run.smithi086.stdout:pg_upmap_items 2.6 [8,11] 2024-03-23T17:50:07.325 INFO:teuthology.orchestra.run.smithi086.stdout:pg_upmap_items 2.10 [2,4] 2024-03-23T17:50:07.325 INFO:teuthology.orchestra.run.smithi086.stdout:pg_upmap_items 2.13 [2,11] 2024-03-23T17:50:07.325 INFO:teuthology.orchestra.run.smithi086.stdout:pg_upmap_items 3.0 [4,8] 2024-03-23T17:50:07.325 INFO:teuthology.orchestra.run.smithi086.stdout:pg_upmap_items 3.10 [5,9] 2024-03-23T17:50:07.325 INFO:teuthology.orchestra.run.smithi086.stdout:pg_upmap_items 3.13 [4,9] 2024-03-23T17:50:07.325 INFO:teuthology.orchestra.run.smithi086.stdout:pg_upmap_items 3.1e [3,9] 2024-03-23T17:50:07.325 INFO:teuthology.orchestra.run.smithi086.stdout:blocklist 172.21.15.86:0/3553500912 expires 2024-03-24T17:43:28.242492+0000 2024-03-23T17:50:07.325 INFO:teuthology.orchestra.run.smithi086.stdout:blocklist 172.21.15.86:6801/3600278228 expires 2024-03-24T17:43:28.242492+0000 2024-03-23T17:50:07.325 INFO:teuthology.orchestra.run.smithi086.stdout:blocklist 172.21.15.86:0/887262329 expires 2024-03-24T17:43:28.242492+0000 2024-03-23T17:50:07.325 INFO:teuthology.orchestra.run.smithi086.stdout:blocklist 172.21.15.86:6800/2313723492 expires 2024-03-24T17:43:05.043910+0000 2024-03-23T17:50:07.325 INFO:teuthology.orchestra.run.smithi086.stdout:blocklist 172.21.15.86:6800/3600278228 expires 2024-03-24T17:43:28.242492+0000 2024-03-23T17:50:07.325 INFO:teuthology.orchestra.run.smithi086.stdout:blocklist 172.21.15.86:0/3815705642 expires 2024-03-24T17:43:05.043910+0000 2024-03-23T17:50:07.326 INFO:teuthology.orchestra.run.smithi086.stdout:blocklist 172.21.15.86:0/1614943062 expires 2024-03-24T17:43:28.242492+0000 2024-03-23T17:50:07.326 INFO:teuthology.orchestra.run.smithi086.stdout:blocklist 172.21.15.86:6801/2313723492 expires 2024-03-24T17:43:05.043910+0000 2024-03-23T17:50:07.326 INFO:teuthology.orchestra.run.smithi086.stdout:blocklist 172.21.15.86:0/2416321716 expires 2024-03-24T17:43:05.043910+0000 2024-03-23T17:50:07.326 INFO:teuthology.orchestra.run.smithi086.stdout:blocklist 172.21.15.86:0/433588188 expires 2024-03-24T17:43:05.043910+0000 2024-03-23T17:50:07.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.324+0000 7efc6c400700 1 -- 172.21.15.86:0/257606367 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7efc4c0a11b0 msgr2=0x7efc4c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:07.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.324+0000 7efc6c400700 1 --2- 172.21.15.86:0/257606367 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7efc4c0a11b0 0x7efc4c0a3670 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7efc50009d20 tx=0x7efc50009260 comp rx=0 tx=0).stop 2024-03-23T17:50:07.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.324+0000 7efc6c400700 1 -- 172.21.15.86:0/257606367 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc64100e00 msgr2=0x7efc64196e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:07.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.324+0000 7efc6c400700 1 --2- 172.21.15.86:0/257606367 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc64100e00 0x7efc64196e90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7efc5800e9f0 tx=0x7efc5800ed00 comp rx=0 tx=0).stop 2024-03-23T17:50:07.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.325+0000 7efc6c400700 1 -- 172.21.15.86:0/257606367 shutdown_connections 2024-03-23T17:50:07.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.325+0000 7efc6c400700 1 --2- 172.21.15.86:0/257606367 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc64102920 0x7efc6419e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:07.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.325+0000 7efc6c400700 1 --2- 172.21.15.86:0/257606367 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc64100e00 0x7efc64196e90 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:07.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.325+0000 7efc6c400700 1 --2- 172.21.15.86:0/257606367 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7efc4c0a11b0 0x7efc4c0a3670 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:07.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.325+0000 7efc6c400700 1 --2- 172.21.15.86:0/257606367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc64101f60 0x7efc641973d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:07.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.325+0000 7efc6c400700 1 -- 172.21.15.86:0/257606367 >> 172.21.15.86:0/257606367 conn(0x7efc640fc3c0 msgr2=0x7efc641052f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:07.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.325+0000 7efc6c400700 1 -- 172.21.15.86:0/257606367 shutdown_connections 2024-03-23T17:50:07.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.325+0000 7efc6c400700 1 -- 172.21.15.86:0/257606367 wait complete. 2024-03-23T17:50:07.685 INFO:teuthology.run_tasks:Running task fs.ready... 2024-03-23T17:50:07.698 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T17:50:07.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.894+0000 7fdbe5e00700 1 -- 172.21.15.86:0/3405215642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbe01016e0 msgr2=0x7fdbe0103ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:07.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.894+0000 7fdbe5e00700 1 --2- 172.21.15.86:0/3405215642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbe01016e0 0x7fdbe0103ad0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0009990 tx=0x7fdbd0009ca0 comp rx=0 tx=0).stop 2024-03-23T17:50:07.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.894+0000 7fdbe5e00700 1 -- 172.21.15.86:0/3405215642 shutdown_connections 2024-03-23T17:50:07.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.894+0000 7fdbe5e00700 1 --2- 172.21.15.86:0/3405215642 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbe0106940 0x7fdbe0108db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:07.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.894+0000 7fdbe5e00700 1 --2- 172.21.15.86:0/3405215642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbe01016e0 0x7fdbe0103ad0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:07.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.894+0000 7fdbe5e00700 1 --2- 172.21.15.86:0/3405215642 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbe0104010 0x7fdbe0106400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:07.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.894+0000 7fdbe5e00700 1 -- 172.21.15.86:0/3405215642 >> 172.21.15.86:0/3405215642 conn(0x7fdbe00fb0d0 msgr2=0x7fdbe00fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:07.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.894+0000 7fdbe5e00700 1 -- 172.21.15.86:0/3405215642 shutdown_connections 2024-03-23T17:50:07.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.894+0000 7fdbe5e00700 1 -- 172.21.15.86:0/3405215642 wait complete. 2024-03-23T17:50:07.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.896+0000 7fdbe5e00700 1 Processor -- start 2024-03-23T17:50:07.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.896+0000 7fdbe5e00700 1 -- start start 2024-03-23T17:50:07.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.896+0000 7fdbe5e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbe01016e0 0x7fdbe0101000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:07.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.896+0000 7fdbe5e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbe0104010 0x7fdbe00ff690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:07.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.896+0000 7fdbe5e00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbe0106940 0x7fdbe00ffbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:07.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.896+0000 7fdbe5e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe0100370 con 0x7fdbe0104010 2024-03-23T17:50:07.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.896+0000 7fdbe5e00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe01004e0 con 0x7fdbe01016e0 2024-03-23T17:50:07.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.896+0000 7fdbe5e00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe01a0c40 con 0x7fdbe0106940 2024-03-23T17:50:07.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.896+0000 7fdbde000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbe0104010 0x7fdbe00ff690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:07.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.896+0000 7fdbde000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbe0104010 0x7fdbe00ff690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37922/0 (socket says 172.21.15.86:37922) 2024-03-23T17:50:07.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.896+0000 7fdbde000700 1 -- 172.21.15.86:0/2207054129 learned_addr learned my addr 172.21.15.86:0/2207054129 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:07.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.896+0000 7fdbdf400700 1 --2- 172.21.15.86:0/2207054129 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbe0106940 0x7fdbe00ffbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:07.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.896+0000 7fdbdea00700 1 --2- 172.21.15.86:0/2207054129 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbe01016e0 0x7fdbe0101000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:07.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.897+0000 7fdbde000700 1 -- 172.21.15.86:0/2207054129 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbe0106940 msgr2=0x7fdbe00ffbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:07.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.897+0000 7fdbde000700 1 --2- 172.21.15.86:0/2207054129 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbe0106940 0x7fdbe00ffbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:07.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.897+0000 7fdbde000700 1 -- 172.21.15.86:0/2207054129 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbe01016e0 msgr2=0x7fdbe0101000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:07.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.897+0000 7fdbde000700 1 --2- 172.21.15.86:0/2207054129 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbe01016e0 0x7fdbe0101000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:07.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.897+0000 7fdbde000700 1 -- 172.21.15.86:0/2207054129 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbd00095f0 con 0x7fdbe0104010 2024-03-23T17:50:07.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.897+0000 7fdbde000700 1 --2- 172.21.15.86:0/2207054129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbe0104010 0x7fdbe00ff690 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc00e9a0 tx=0x7fdbcc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:07.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.897+0000 7fdbe4a00700 1 -- 172.21.15.86:0/2207054129 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc009790 con 0x7fdbe0104010 2024-03-23T17:50:07.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.897+0000 7fdbe4a00700 1 -- 172.21.15.86:0/2207054129 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdbcc00cb70 con 0x7fdbe0104010 2024-03-23T17:50:07.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.897+0000 7fdbe5e00700 1 -- 172.21.15.86:0/2207054129 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbe01a0ed0 con 0x7fdbe0104010 2024-03-23T17:50:07.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.897+0000 7fdbe4a00700 1 -- 172.21.15.86:0/2207054129 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc010ea0 con 0x7fdbe0104010 2024-03-23T17:50:07.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.897+0000 7fdbe5e00700 1 -- 172.21.15.86:0/2207054129 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbe01a1350 con 0x7fdbe0104010 2024-03-23T17:50:07.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.898+0000 7fdbe5e00700 1 -- 172.21.15.86:0/2207054129 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbe00fcc50 con 0x7fdbe0104010 2024-03-23T17:50:07.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.900+0000 7fdbe4a00700 1 -- 172.21.15.86:0/2207054129 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fdbcc00cce0 con 0x7fdbe0104010 2024-03-23T17:50:07.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.901+0000 7fdbe4a00700 1 --2- 172.21.15.86:0/2207054129 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fdbc40a11b0 0x7fdbc40a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:07.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.902+0000 7fdbe4a00700 1 -- 172.21.15.86:0/2207054129 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fdbcc014070 con 0x7fdbe0104010 2024-03-23T17:50:07.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.902+0000 7fdbdea00700 1 --2- 172.21.15.86:0/2207054129 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fdbc40a11b0 0x7fdbc40a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:07.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.902+0000 7fdbdea00700 1 --2- 172.21.15.86:0/2207054129 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fdbc40a11b0 0x7fdbc40a3670 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0006010 tx=0x7fdbd00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:07.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:07.906+0000 7fdbe4a00700 1 -- 172.21.15.86:0/2207054129 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fdbcc0529f0 con 0x7fdbe0104010 2024-03-23T17:50:08.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:07 smithi086 ceph-mon[38221]: pgmap v271: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-23T17:50:08.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:07 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/257606367' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-23T17:50:08.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:07 smithi092 ceph-mon[79089]: pgmap v271: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-23T17:50:08.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:07 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/257606367' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-23T17:50:08.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:07 smithi152 ceph-mon[47949]: pgmap v271: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-23T17:50:08.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:07 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/257606367' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-23T17:50:08.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.150+0000 7fdbe5e00700 1 -- 172.21.15.86:0/2207054129 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdbe010c4e0 con 0x7fdbe0104010 2024-03-23T17:50:08.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.151+0000 7fdbe4a00700 1 -- 172.21.15.86:0/2207054129 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22414 (secure 0 0 0) 0x7fdbcc0962d0 con 0x7fdbe0104010 2024-03-23T17:50:08.152 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:50:08.152 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":82,"fsid":"9d76c9e8-e93c-11ee-95cd-87774f69a715","created":"2024-03-23T17:42:30.097233+0000","modified":"2024-03-23T17:49:29.773954+0000","last_up_change":"2024-03-23T17:48:18.217601+0000","last_in_change":"2024-03-23T17:48:00.250579+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T17:45:26.848397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T17:48:54.331894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T17:48:55.640693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"656655c6-1c7f-4640-92c4-2f42de820143","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6803","nonce":2423505491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6805","nonce":2423505491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6809","nonce":2423505491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6807","nonce":2423505491}]},"public_addr":"172.21.15.86:6803/2423505491","cluster_addr":"172.21.15.86:6805/2423505491","heartbeat_back_addr":"172.21.15.86:6809/2423505491","heartbeat_front_addr":"172.21.15.86:6807/2423505491","state":["exists","up"]},{"osd":1,"uuid":"3e86dc65-20a6-4de8-b9ce-fcf01f68862b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6801","nonce":3776252597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6803","nonce":3776252597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6807","nonce":3776252597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6805","nonce":3776252597}]},"public_addr":"172.21.15.92:6801/3776252597","cluster_addr":"172.21.15.92:6803/3776252597","heartbeat_back_addr":"172.21.15.92:6807/3776252597","heartbeat_front_addr":"172.21.15.92:6805/3776252597","state":["exists","up"]},{"osd":2,"uuid":"09808786-3bd3-4d11-8c7b-1736119dfbb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6801","nonce":1601265891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6803","nonce":1601265891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6807","nonce":1601265891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6805","nonce":1601265891}]},"public_addr":"172.21.15.152:6801/1601265891","cluster_addr":"172.21.15.152:6803/1601265891","heartbeat_back_addr":"172.21.15.152:6807/1601265891","heartbeat_front_addr":"172.21.15.152:6805/1601265891","state":["exists","up"]},{"osd":3,"uuid":"79c61910-4b53-424c-b150-6f5385a2cd68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6811","nonce":694846466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6813","nonce":694846466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6817","nonce":694846466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6815","nonce":694846466}]},"public_addr":"172.21.15.86:6811/694846466","cluster_addr":"172.21.15.86:6813/694846466","heartbeat_back_addr":"172.21.15.86:6817/694846466","heartbeat_front_addr":"172.21.15.86:6815/694846466","state":["exists","up"]},{"osd":4,"uuid":"a2878035-0179-469f-bbd4-8ce2d3f6bddc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6809","nonce":1422821471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6811","nonce":1422821471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6815","nonce":1422821471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6813","nonce":1422821471}]},"public_addr":"172.21.15.92:6809/1422821471","cluster_addr":"172.21.15.92:6811/1422821471","heartbeat_back_addr":"172.21.15.92:6815/1422821471","heartbeat_front_addr":"172.21.15.92:6813/1422821471","state":["exists","up"]},{"osd":5,"uuid":"837366aa-ce25-42ca-b157-5bbb50159d0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6809","nonce":1039375239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6811","nonce":1039375239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6815","nonce":1039375239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6813","nonce":1039375239}]},"public_addr":"172.21.15.152:6809/1039375239","cluster_addr":"172.21.15.152:6811/1039375239","heartbeat_back_addr":"172.21.15.152:6815/1039375239","heartbeat_front_addr":"172.21.15.152:6813/1039375239","state":["exists","up"]},{"osd":6,"uuid":"61024437-7b91-432f-a1af-6443d0ef5213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6819","nonce":4080452707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6821","nonce":4080452707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6825","nonce":4080452707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6823","nonce":4080452707}]},"public_addr":"172.21.15.86:6819/4080452707","cluster_addr":"172.21.15.86:6821/4080452707","heartbeat_back_addr":"172.21.15.86:6825/4080452707","heartbeat_front_addr":"172.21.15.86:6823/4080452707","state":["exists","up"]},{"osd":7,"uuid":"2684bdae-6b30-4afc-af68-4ce41b3a75b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6817","nonce":4187707622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6819","nonce":4187707622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6823","nonce":4187707622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6821","nonce":4187707622}]},"public_addr":"172.21.15.92:6817/4187707622","cluster_addr":"172.21.15.92:6819/4187707622","heartbeat_back_addr":"172.21.15.92:6823/4187707622","heartbeat_front_addr":"172.21.15.92:6821/4187707622","state":["exists","up"]},{"osd":8,"uuid":"f03bbf0d-1281-4bcf-9773-9854388e65ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6817","nonce":3237612748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6819","nonce":3237612748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6823","nonce":3237612748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6821","nonce":3237612748}]},"public_addr":"172.21.15.152:6817/3237612748","cluster_addr":"172.21.15.152:6819/3237612748","heartbeat_back_addr":"172.21.15.152:6823/3237612748","heartbeat_front_addr":"172.21.15.152:6821/3237612748","state":["exists","up"]},{"osd":9,"uuid":"e7e8b775-99ff-48a4-a23a-91e7ab6622bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6827","nonce":2835218294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6829","nonce":2835218294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6833","nonce":2835218294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6831","nonce":2835218294}]},"public_addr":"172.21.15.86:6827/2835218294","cluster_addr":"172.21.15.86:6829/2835218294","heartbeat_back_addr":"172.21.15.86:6833/2835218294","heartbeat_front_addr":"172.21.15.86:6831/2835218294","state":["exists","up"]},{"osd":10,"uuid":"d9d616cd-a229-44de-8768-02a4415df115","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6825","nonce":4259465100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6827","nonce":4259465100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6831","nonce":4259465100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6829","nonce":4259465100}]},"public_addr":"172.21.15.92:6825/4259465100","cluster_addr":"172.21.15.92:6827/4259465100","heartbeat_back_addr":"172.21.15.92:6831/4259465100","heartbeat_front_addr":"172.21.15.92:6829/4259465100","state":["exists","up"]},{"osd":11,"uuid":"e7c0b865-ccf5-4233-8d22-0d9b6dbe5c97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6825","nonce":194662760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6827","nonce":194662760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6831","nonce":194662760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6829","nonce":194662760}]},"public_addr":"172.21.15.152:6825/194662760","cluster_addr":"172.21.15.152:6827/194662760","heartbeat_back_addr":"172.21.15.152:6831/194662760","heartbeat_front_addr":"172.21.15.152:6829/194662760","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:44:47.269167+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:45:05.363655+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:45:23.192677+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:45:44.156228+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:02.532971+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:20.153295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:41.713888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:59.676516+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:47:17.207087+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:47:38.970316+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:47:57.282525+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:48:15.437606+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/3553500912":"2024-03-24T17:43:28.242492+0000","172.21.15.86:6801/3600278228":"2024-03-24T17:43:28.242492+0000","172.21.15.86:0/887262329":"2024-03-24T17:43:28.242492+0000","172.21.15.86:6800/2313723492":"2024-03-24T17:43:05.043910+0000","172.21.15.86:6800/3600278228":"2024-03-24T17:43:28.242492+0000","172.21.15.86:0/3815705642":"2024-03-24T17:43:05.043910+0000","172.21.15.86:0/1614943062":"2024-03-24T17:43:28.242492+0000","172.21.15.86:6801/2313723492":"2024-03-24T17:43:05.043910+0000","172.21.15.86:0/2416321716":"2024-03-24T17:43:05.043910+0000","172.21.15.86:0/433588188":"2024-03-24T17:43:05.043910+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T17:50:08.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.154+0000 7fdbe5e00700 1 -- 172.21.15.86:0/2207054129 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fdbc40a11b0 msgr2=0x7fdbc40a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:08.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.154+0000 7fdbe5e00700 1 --2- 172.21.15.86:0/2207054129 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fdbc40a11b0 0x7fdbc40a3670 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0006010 tx=0x7fdbd00058e0 comp rx=0 tx=0).stop 2024-03-23T17:50:08.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.154+0000 7fdbe5e00700 1 -- 172.21.15.86:0/2207054129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbe0104010 msgr2=0x7fdbe00ff690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:08.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.154+0000 7fdbe5e00700 1 --2- 172.21.15.86:0/2207054129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbe0104010 0x7fdbe00ff690 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc00e9a0 tx=0x7fdbcc00ecb0 comp rx=0 tx=0).stop 2024-03-23T17:50:08.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.154+0000 7fdbe5e00700 1 -- 172.21.15.86:0/2207054129 shutdown_connections 2024-03-23T17:50:08.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.154+0000 7fdbe5e00700 1 --2- 172.21.15.86:0/2207054129 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbe0106940 0x7fdbe00ffbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:08.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.154+0000 7fdbe5e00700 1 --2- 172.21.15.86:0/2207054129 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbe01016e0 0x7fdbe0101000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:08.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.154+0000 7fdbe5e00700 1 --2- 172.21.15.86:0/2207054129 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fdbc40a11b0 0x7fdbc40a3670 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:08.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.154+0000 7fdbe5e00700 1 --2- 172.21.15.86:0/2207054129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbe0104010 0x7fdbe00ff690 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:08.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.154+0000 7fdbe5e00700 1 -- 172.21.15.86:0/2207054129 >> 172.21.15.86:0/2207054129 conn(0x7fdbe00fb0d0 msgr2=0x7fdbe01035a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:08.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.154+0000 7fdbe5e00700 1 -- 172.21.15.86:0/2207054129 shutdown_connections 2024-03-23T17:50:08.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.154+0000 7fdbe5e00700 1 -- 172.21.15.86:0/2207054129 wait complete. 2024-03-23T17:50:08.173 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T17:45:26.848397+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T17:48:54.331894+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T17:48:55.640693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T17:50:08.174 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T17:50:08.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.373+0000 7f5809000700 1 -- 172.21.15.86:0/1857529206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5804106af0 msgr2=0x7f5804108ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:08.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.373+0000 7f5809000700 1 --2- 172.21.15.86:0/1857529206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5804106af0 0x7f5804108ee0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f57fc00b0a0 tx=0x7f57fc00b3b0 comp rx=0 tx=0).stop 2024-03-23T17:50:08.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.373+0000 7f5809000700 1 -- 172.21.15.86:0/1857529206 shutdown_connections 2024-03-23T17:50:08.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.373+0000 7f5809000700 1 --2- 172.21.15.86:0/1857529206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5804106af0 0x7f5804108ee0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:08.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.373+0000 7f5809000700 1 --2- 172.21.15.86:0/1857529206 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5804101890 0x7f5804103c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:08.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.373+0000 7f5809000700 1 --2- 172.21.15.86:0/1857529206 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58041041c0 0x7f58041065b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:08.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.373+0000 7f5809000700 1 -- 172.21.15.86:0/1857529206 >> 172.21.15.86:0/1857529206 conn(0x7f58040fb200 msgr2=0x7f58040fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:08.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.373+0000 7f5809000700 1 -- 172.21.15.86:0/1857529206 shutdown_connections 2024-03-23T17:50:08.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.373+0000 7f5809000700 1 -- 172.21.15.86:0/1857529206 wait complete. 2024-03-23T17:50:08.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.374+0000 7f5809000700 1 Processor -- start 2024-03-23T17:50:08.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.374+0000 7f5809000700 1 -- start start 2024-03-23T17:50:08.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.374+0000 7f5809000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5804101890 0x7f5804101160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:08.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.374+0000 7f5809000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58041041c0 0x7f58040ff840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:08.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.374+0000 7f5809000700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5804106af0 0x7f58040ffd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:08.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.374+0000 7f5809000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5804100480 con 0x7f5804101890 2024-03-23T17:50:08.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.374+0000 7f5809000700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58041005f0 con 0x7f58041041c0 2024-03-23T17:50:08.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.374+0000 7f5809000700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58041a0e80 con 0x7f5804106af0 2024-03-23T17:50:08.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.374+0000 7f5801e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5804101890 0x7f5804101160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:08.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.375+0000 7f5801e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5804101890 0x7f5804101160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37932/0 (socket says 172.21.15.86:37932) 2024-03-23T17:50:08.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.375+0000 7f5801e00700 1 -- 172.21.15.86:0/1921436245 learned_addr learned my addr 172.21.15.86:0/1921436245 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:08.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.375+0000 7f5801400700 1 --2- 172.21.15.86:0/1921436245 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58041041c0 0x7f58040ff840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:08.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.375+0000 7f5802800700 1 --2- 172.21.15.86:0/1921436245 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5804106af0 0x7f58040ffd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:08.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.375+0000 7f5801e00700 1 -- 172.21.15.86:0/1921436245 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5804106af0 msgr2=0x7f58040ffd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:08.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.375+0000 7f5801e00700 1 --2- 172.21.15.86:0/1921436245 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5804106af0 0x7f58040ffd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:08.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.375+0000 7f5801e00700 1 -- 172.21.15.86:0/1921436245 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58041041c0 msgr2=0x7f58040ff840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:08.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.375+0000 7f5801e00700 1 --2- 172.21.15.86:0/1921436245 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58041041c0 0x7f58040ff840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:08.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.375+0000 7f5801e00700 1 -- 172.21.15.86:0/1921436245 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57fc009d00 con 0x7f5804101890 2024-03-23T17:50:08.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.375+0000 7f5801e00700 1 --2- 172.21.15.86:0/1921436245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5804101890 0x7f5804101160 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f57f400e9a0 tx=0x7f57f400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:08.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.375+0000 7f57faa00700 1 -- 172.21.15.86:0/1921436245 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f4009790 con 0x7f5804101890 2024-03-23T17:50:08.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.375+0000 7f5809000700 1 -- 172.21.15.86:0/1921436245 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58041a10e0 con 0x7f5804101890 2024-03-23T17:50:08.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.375+0000 7f57faa00700 1 -- 172.21.15.86:0/1921436245 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f57f4010430 con 0x7f5804101890 2024-03-23T17:50:08.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.375+0000 7f57faa00700 1 -- 172.21.15.86:0/1921436245 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f4018680 con 0x7f5804101890 2024-03-23T17:50:08.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.375+0000 7f5809000700 1 -- 172.21.15.86:0/1921436245 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58041a15f0 con 0x7f5804101890 2024-03-23T17:50:08.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.377+0000 7f5809000700 1 -- 172.21.15.86:0/1921436245 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58040fcd80 con 0x7f5804101890 2024-03-23T17:50:08.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.378+0000 7f57faa00700 1 -- 172.21.15.86:0/1921436245 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f57f40105a0 con 0x7f5804101890 2024-03-23T17:50:08.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.380+0000 7f57faa00700 1 --2- 172.21.15.86:0/1921436245 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f57e80a11b0 0x7f57e80a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:08.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.380+0000 7f57faa00700 1 -- 172.21.15.86:0/1921436245 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f57f4014070 con 0x7f5804101890 2024-03-23T17:50:08.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.381+0000 7f5801400700 1 --2- 172.21.15.86:0/1921436245 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f57e80a11b0 0x7f57e80a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:08.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.381+0000 7f5801400700 1 --2- 172.21.15.86:0/1921436245 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f57e80a11b0 0x7f57e80a3670 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f57f0009c60 tx=0x7f57f0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:08.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.384+0000 7f57faa00700 1 -- 172.21.15.86:0/1921436245 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f57f40529f0 con 0x7f5804101890 2024-03-23T17:50:08.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.593+0000 7f5809000700 1 -- 172.21.15.86:0/1921436245 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f58040fc760 con 0x7f5804101890 2024-03-23T17:50:08.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.594+0000 7f57faa00700 1 -- 172.21.15.86:0/1921436245 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f57f4096250 con 0x7f5804101890 2024-03-23T17:50:08.594 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-03-23T17:50:08.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.596+0000 7f5809000700 1 -- 172.21.15.86:0/1921436245 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f57e80a11b0 msgr2=0x7f57e80a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:08.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.596+0000 7f5809000700 1 --2- 172.21.15.86:0/1921436245 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f57e80a11b0 0x7f57e80a3670 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f57f0009c60 tx=0x7f57f0009260 comp rx=0 tx=0).stop 2024-03-23T17:50:08.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.596+0000 7f5809000700 1 -- 172.21.15.86:0/1921436245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5804101890 msgr2=0x7f5804101160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:08.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.596+0000 7f5809000700 1 --2- 172.21.15.86:0/1921436245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5804101890 0x7f5804101160 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f57f400e9a0 tx=0x7f57f400ecb0 comp rx=0 tx=0).stop 2024-03-23T17:50:08.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.596+0000 7f5809000700 1 -- 172.21.15.86:0/1921436245 shutdown_connections 2024-03-23T17:50:08.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.596+0000 7f5809000700 1 --2- 172.21.15.86:0/1921436245 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5804106af0 0x7f58040ffd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:08.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.596+0000 7f5809000700 1 --2- 172.21.15.86:0/1921436245 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f57e80a11b0 0x7f57e80a3670 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:08.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.596+0000 7f5809000700 1 --2- 172.21.15.86:0/1921436245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5804101890 0x7f5804101160 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:08.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.596+0000 7f5809000700 1 --2- 172.21.15.86:0/1921436245 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58041041c0 0x7f58040ff840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:08.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.596+0000 7f5809000700 1 -- 172.21.15.86:0/1921436245 >> 172.21.15.86:0/1921436245 conn(0x7f58040fb200 msgr2=0x7f58041021d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:08.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.596+0000 7f5809000700 1 -- 172.21.15.86:0/1921436245 shutdown_connections 2024-03-23T17:50:08.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.596+0000 7f5809000700 1 -- 172.21.15.86:0/1921436245 wait complete. 2024-03-23T17:50:08.616 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T17:50:08.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.812+0000 7f813ba00700 1 -- 172.21.15.86:0/2271704249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8134108c40 msgr2=0x7f813410b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:08.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.812+0000 7f813ba00700 1 --2- 172.21.15.86:0/2271704249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8134108c40 0x7f813410b0d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f812c00b0a0 tx=0x7f812c00b3b0 comp rx=0 tx=0).stop 2024-03-23T17:50:08.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.812+0000 7f813ba00700 1 -- 172.21.15.86:0/2271704249 shutdown_connections 2024-03-23T17:50:08.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.812+0000 7f813ba00700 1 --2- 172.21.15.86:0/2271704249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8134108c40 0x7f813410b0d0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:08.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.812+0000 7f813ba00700 1 --2- 172.21.15.86:0/2271704249 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8134074df0 0x7f8134073240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:08.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.812+0000 7f813ba00700 1 --2- 172.21.15.86:0/2271704249 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8134073810 0x7f8134073c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:08.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.812+0000 7f813ba00700 1 -- 172.21.15.86:0/2271704249 >> 172.21.15.86:0/2271704249 conn(0x7f81340fc2f0 msgr2=0x7f81340fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:08.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.812+0000 7f813ba00700 1 -- 172.21.15.86:0/2271704249 shutdown_connections 2024-03-23T17:50:08.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.812+0000 7f813ba00700 1 -- 172.21.15.86:0/2271704249 wait complete. 2024-03-23T17:50:08.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.814+0000 7f813ba00700 1 Processor -- start 2024-03-23T17:50:08.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.814+0000 7f813ba00700 1 -- start start 2024-03-23T17:50:08.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.814+0000 7f813ba00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8134073810 0x7f8134199040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:08.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.814+0000 7f813ba00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8134074df0 0x7f8134199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:08.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.814+0000 7f813ba00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8134108c40 0x7f81341a05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:08.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.814+0000 7f813ba00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81341a0b70 con 0x7f8134073810 2024-03-23T17:50:08.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.814+0000 7f813ba00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81341a0cb0 con 0x7f8134074df0 2024-03-23T17:50:08.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.814+0000 7f813ba00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81341a0e40 con 0x7f8134108c40 2024-03-23T17:50:08.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.814+0000 7f8139000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8134073810 0x7f8134199040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:08.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.814+0000 7f8139000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8134073810 0x7f8134199040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37946/0 (socket says 172.21.15.86:37946) 2024-03-23T17:50:08.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.814+0000 7f8139000700 1 -- 172.21.15.86:0/947937446 learned_addr learned my addr 172.21.15.86:0/947937446 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:08.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.815+0000 7f8139a00700 1 --2- 172.21.15.86:0/947937446 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8134108c40 0x7f81341a05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:08.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.815+0000 7f8133e00700 1 --2- 172.21.15.86:0/947937446 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8134074df0 0x7f8134199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:08.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.815+0000 7f8139000700 1 -- 172.21.15.86:0/947937446 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8134108c40 msgr2=0x7f81341a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:08.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.815+0000 7f8139000700 1 --2- 172.21.15.86:0/947937446 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8134108c40 0x7f81341a05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:08.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.815+0000 7f8139000700 1 -- 172.21.15.86:0/947937446 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8134074df0 msgr2=0x7f8134199580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:08.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.815+0000 7f8139000700 1 --2- 172.21.15.86:0/947937446 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8134074df0 0x7f8134199580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:08.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.815+0000 7f8139000700 1 -- 172.21.15.86:0/947937446 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f812c009d00 con 0x7f8134073810 2024-03-23T17:50:08.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.815+0000 7f8139000700 1 --2- 172.21.15.86:0/947937446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8134073810 0x7f8134199040 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f812800e9a0 tx=0x7f812800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:08.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.815+0000 7f8131600700 1 -- 172.21.15.86:0/947937446 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8128009790 con 0x7f8134073810 2024-03-23T17:50:08.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.815+0000 7f813ba00700 1 -- 172.21.15.86:0/947937446 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81341a1120 con 0x7f8134073810 2024-03-23T17:50:08.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.815+0000 7f8131600700 1 -- 172.21.15.86:0/947937446 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8128010430 con 0x7f8134073810 2024-03-23T17:50:08.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.815+0000 7f813ba00700 1 -- 172.21.15.86:0/947937446 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81341a1680 con 0x7f8134073810 2024-03-23T17:50:08.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.815+0000 7f8131600700 1 -- 172.21.15.86:0/947937446 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8128018620 con 0x7f8134073810 2024-03-23T17:50:08.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.817+0000 7f813ba00700 1 -- 172.21.15.86:0/947937446 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8134107a00 con 0x7f8134073810 2024-03-23T17:50:08.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.818+0000 7f8131600700 1 -- 172.21.15.86:0/947937446 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f81280105a0 con 0x7f8134073810 2024-03-23T17:50:08.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.819+0000 7f8131600700 1 --2- 172.21.15.86:0/947937446 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f811c0a11b0 0x7f811c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:08.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.820+0000 7f8131600700 1 -- 172.21.15.86:0/947937446 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f8128014070 con 0x7f8134073810 2024-03-23T17:50:08.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.820+0000 7f8133e00700 1 --2- 172.21.15.86:0/947937446 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f811c0a11b0 0x7f811c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:08.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.820+0000 7f8133e00700 1 --2- 172.21.15.86:0/947937446 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f811c0a11b0 0x7f811c0a3670 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8134074b10 tx=0x7f8120009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:08.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:08.823+0000 7f8131600700 1 -- 172.21.15.86:0/947937446 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f81280529f0 con 0x7f8134073810 2024-03-23T17:50:09.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:08 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/2207054129' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T17:50:09.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:08 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/1921436245' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T17:50:09.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.029+0000 7f813ba00700 1 -- 172.21.15.86:0/947937446 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f813410ea00 con 0x7f8134073810 2024-03-23T17:50:09.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.030+0000 7f8131600700 1 -- 172.21.15.86:0/947937446 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v82) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8128096200 con 0x7f8134073810 2024-03-23T17:50:09.030 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-03-23T17:50:09.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.032+0000 7f813ba00700 1 -- 172.21.15.86:0/947937446 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f811c0a11b0 msgr2=0x7f811c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:09.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.032+0000 7f813ba00700 1 --2- 172.21.15.86:0/947937446 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f811c0a11b0 0x7f811c0a3670 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8134074b10 tx=0x7f8120009260 comp rx=0 tx=0).stop 2024-03-23T17:50:09.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.032+0000 7f813ba00700 1 -- 172.21.15.86:0/947937446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8134073810 msgr2=0x7f8134199040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:09.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.032+0000 7f813ba00700 1 --2- 172.21.15.86:0/947937446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8134073810 0x7f8134199040 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f812800e9a0 tx=0x7f812800ecb0 comp rx=0 tx=0).stop 2024-03-23T17:50:09.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.032+0000 7f813ba00700 1 -- 172.21.15.86:0/947937446 shutdown_connections 2024-03-23T17:50:09.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.032+0000 7f813ba00700 1 --2- 172.21.15.86:0/947937446 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8134108c40 0x7f81341a05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:09.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.032+0000 7f813ba00700 1 --2- 172.21.15.86:0/947937446 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f811c0a11b0 0x7f811c0a3670 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:09.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.032+0000 7f813ba00700 1 --2- 172.21.15.86:0/947937446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8134073810 0x7f8134199040 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:09.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.032+0000 7f813ba00700 1 --2- 172.21.15.86:0/947937446 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8134074df0 0x7f8134199580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:09.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.032+0000 7f813ba00700 1 -- 172.21.15.86:0/947937446 >> 172.21.15.86:0/947937446 conn(0x7f81340fc2f0 msgr2=0x7f8134104b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:09.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.032+0000 7f813ba00700 1 -- 172.21.15.86:0/947937446 shutdown_connections 2024-03-23T17:50:09.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.032+0000 7f813ba00700 1 -- 172.21.15.86:0/947937446 wait complete. 2024-03-23T17:50:09.051 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T17:50:09.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:08 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/2207054129' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T17:50:09.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:08 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/1921436245' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T17:50:09.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:08 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/2207054129' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T17:50:09.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:08 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/1921436245' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T17:50:09.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.246+0000 7fa0dea00700 1 -- 172.21.15.86:0/3175933200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0d8101f70 msgr2=0x7fa0d81023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:09.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.246+0000 7fa0dea00700 1 --2- 172.21.15.86:0/3175933200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0d8101f70 0x7fa0d81023d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa0c40099c0 tx=0x7fa0c4009cd0 comp rx=0 tx=0).stop 2024-03-23T17:50:09.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.246+0000 7fa0dea00700 1 -- 172.21.15.86:0/3175933200 shutdown_connections 2024-03-23T17:50:09.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.246+0000 7fa0dea00700 1 --2- 172.21.15.86:0/3175933200 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0d8102910 0x7fa0d81091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:09.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.246+0000 7fa0dea00700 1 --2- 172.21.15.86:0/3175933200 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0d8100d80 0x7fa0d81011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:09.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.246+0000 7fa0dea00700 1 --2- 172.21.15.86:0/3175933200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0d8101f70 0x7fa0d81023d0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:09.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.246+0000 7fa0dea00700 1 -- 172.21.15.86:0/3175933200 >> 172.21.15.86:0/3175933200 conn(0x7fa0d80fc2f0 msgr2=0x7fa0d80fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:09.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.246+0000 7fa0dea00700 1 -- 172.21.15.86:0/3175933200 shutdown_connections 2024-03-23T17:50:09.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.246+0000 7fa0dea00700 1 -- 172.21.15.86:0/3175933200 wait complete. 2024-03-23T17:50:09.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.247+0000 7fa0dea00700 1 Processor -- start 2024-03-23T17:50:09.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.247+0000 7fa0dea00700 1 -- start start 2024-03-23T17:50:09.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.247+0000 7fa0dea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0d8100d80 0x7fa0d8196d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:09.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.247+0000 7fa0dea00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0d8101f70 0x7fa0d8197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:09.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.247+0000 7fa0dea00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0d8102910 0x7fa0d819e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:09.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.247+0000 7fa0dea00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d819e840 con 0x7fa0d8100d80 2024-03-23T17:50:09.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.247+0000 7fa0dea00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d819e9b0 con 0x7fa0d8102910 2024-03-23T17:50:09.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.247+0000 7fa0dea00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d819eb70 con 0x7fa0d8101f70 2024-03-23T17:50:09.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.248+0000 7fa0d7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0d8100d80 0x7fa0d8196d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:09.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.248+0000 7fa0d7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0d8100d80 0x7fa0d8196d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37962/0 (socket says 172.21.15.86:37962) 2024-03-23T17:50:09.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.248+0000 7fa0d7e00700 1 -- 172.21.15.86:0/2247123991 learned_addr learned my addr 172.21.15.86:0/2247123991 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:09.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.248+0000 7fa0dca00700 1 --2- 172.21.15.86:0/2247123991 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0d8102910 0x7fa0d819e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:09.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.248+0000 7fa0d7400700 1 --2- 172.21.15.86:0/2247123991 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0d8101f70 0x7fa0d8197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:09.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.248+0000 7fa0d7e00700 1 -- 172.21.15.86:0/2247123991 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0d8101f70 msgr2=0x7fa0d8197290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:09.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.248+0000 7fa0d7e00700 1 --2- 172.21.15.86:0/2247123991 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0d8101f70 0x7fa0d8197290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:09.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.248+0000 7fa0d7e00700 1 -- 172.21.15.86:0/2247123991 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0d8102910 msgr2=0x7fa0d819e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:09.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.248+0000 7fa0d7e00700 1 --2- 172.21.15.86:0/2247123991 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0d8102910 0x7fa0d819e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:09.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.248+0000 7fa0d7e00700 1 -- 172.21.15.86:0/2247123991 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0c40095f0 con 0x7fa0d8100d80 2024-03-23T17:50:09.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.248+0000 7fa0dca00700 1 --2- 172.21.15.86:0/2247123991 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0d8102910 0x7fa0d819e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:50:09.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.248+0000 7fa0d7e00700 1 --2- 172.21.15.86:0/2247123991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0d8100d80 0x7fa0d8196d50 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa0c800e970 tx=0x7fa0c800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:09.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.249+0000 7fa0d4c00700 1 -- 172.21.15.86:0/2247123991 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c8009790 con 0x7fa0d8100d80 2024-03-23T17:50:09.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.249+0000 7fa0d4c00700 1 -- 172.21.15.86:0/2247123991 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0c800c7e0 con 0x7fa0d8100d80 2024-03-23T17:50:09.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.249+0000 7fa0dea00700 1 -- 172.21.15.86:0/2247123991 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0d819ee50 con 0x7fa0d8100d80 2024-03-23T17:50:09.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.249+0000 7fa0d4c00700 1 -- 172.21.15.86:0/2247123991 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c8003c70 con 0x7fa0d8100d80 2024-03-23T17:50:09.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.249+0000 7fa0dea00700 1 -- 172.21.15.86:0/2247123991 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0d819f270 con 0x7fa0d8100d80 2024-03-23T17:50:09.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.250+0000 7fa0dea00700 1 -- 172.21.15.86:0/2247123991 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0d8108200 con 0x7fa0d8100d80 2024-03-23T17:50:09.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.252+0000 7fa0d4c00700 1 -- 172.21.15.86:0/2247123991 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fa0c800cc50 con 0x7fa0d8100d80 2024-03-23T17:50:09.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.253+0000 7fa0d4c00700 1 --2- 172.21.15.86:0/2247123991 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fa0bc0a11b0 0x7fa0bc0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:09.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.253+0000 7fa0d4c00700 1 -- 172.21.15.86:0/2247123991 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fa0c8014070 con 0x7fa0d8100d80 2024-03-23T17:50:09.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.253+0000 7fa0d7400700 1 --2- 172.21.15.86:0/2247123991 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fa0bc0a11b0 0x7fa0bc0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:09.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.254+0000 7fa0d7400700 1 --2- 172.21.15.86:0/2247123991 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fa0bc0a11b0 0x7fa0bc0a3670 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa0c4003730 tx=0x7fa0c400b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:09.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.257+0000 7fa0d4c00700 1 -- 172.21.15.86:0/2247123991 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa0c8096230 con 0x7fa0d8100d80 2024-03-23T17:50:09.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.461+0000 7fa0dea00700 1 -- 172.21.15.86:0/2247123991 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa0d810c5e0 con 0x7fa0d8100d80 2024-03-23T17:50:09.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.461+0000 7fa0d4c00700 1 -- 172.21.15.86:0/2247123991 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v82) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa0c8099850 con 0x7fa0d8100d80 2024-03-23T17:50:09.462 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-03-23T17:50:09.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.464+0000 7fa0dea00700 1 -- 172.21.15.86:0/2247123991 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fa0bc0a11b0 msgr2=0x7fa0bc0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:09.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.464+0000 7fa0dea00700 1 --2- 172.21.15.86:0/2247123991 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fa0bc0a11b0 0x7fa0bc0a3670 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa0c4003730 tx=0x7fa0c400b540 comp rx=0 tx=0).stop 2024-03-23T17:50:09.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.464+0000 7fa0dea00700 1 -- 172.21.15.86:0/2247123991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0d8100d80 msgr2=0x7fa0d8196d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:09.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.464+0000 7fa0dea00700 1 --2- 172.21.15.86:0/2247123991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0d8100d80 0x7fa0d8196d50 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa0c800e970 tx=0x7fa0c800ec80 comp rx=0 tx=0).stop 2024-03-23T17:50:09.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.464+0000 7fa0dea00700 1 -- 172.21.15.86:0/2247123991 shutdown_connections 2024-03-23T17:50:09.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.464+0000 7fa0dea00700 1 --2- 172.21.15.86:0/2247123991 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0d8102910 0x7fa0d819e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:09.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.464+0000 7fa0dea00700 1 --2- 172.21.15.86:0/2247123991 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fa0bc0a11b0 0x7fa0bc0a3670 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:09.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.464+0000 7fa0dea00700 1 --2- 172.21.15.86:0/2247123991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0d8100d80 0x7fa0d8196d50 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:09.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.464+0000 7fa0dea00700 1 --2- 172.21.15.86:0/2247123991 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0d8101f70 0x7fa0d8197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:09.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.464+0000 7fa0dea00700 1 -- 172.21.15.86:0/2247123991 >> 172.21.15.86:0/2247123991 conn(0x7fa0d80fc2f0 msgr2=0x7fa0d8105340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:09.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.464+0000 7fa0dea00700 1 -- 172.21.15.86:0/2247123991 shutdown_connections 2024-03-23T17:50:09.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.464+0000 7fa0dea00700 1 -- 172.21.15.86:0/2247123991 wait complete. 2024-03-23T17:50:09.483 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T17:50:09.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.674+0000 7fc223e00700 1 -- 172.21.15.86:0/1920941412 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc21c1016e0 msgr2=0x7fc21c103ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:09.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.674+0000 7fc223e00700 1 --2- 172.21.15.86:0/1920941412 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc21c1016e0 0x7fc21c103ad0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc2100098f0 tx=0x7fc210009c00 comp rx=0 tx=0).stop 2024-03-23T17:50:09.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.675+0000 7fc223e00700 1 -- 172.21.15.86:0/1920941412 shutdown_connections 2024-03-23T17:50:09.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.675+0000 7fc223e00700 1 --2- 172.21.15.86:0/1920941412 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc21c106940 0x7fc21c108db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:09.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.675+0000 7fc223e00700 1 --2- 172.21.15.86:0/1920941412 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc21c1016e0 0x7fc21c103ad0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:09.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.675+0000 7fc223e00700 1 --2- 172.21.15.86:0/1920941412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc21c104010 0x7fc21c106400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:09.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.675+0000 7fc223e00700 1 -- 172.21.15.86:0/1920941412 >> 172.21.15.86:0/1920941412 conn(0x7fc21c0fb0d0 msgr2=0x7fc21c0fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:09.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.675+0000 7fc223e00700 1 -- 172.21.15.86:0/1920941412 shutdown_connections 2024-03-23T17:50:09.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.675+0000 7fc223e00700 1 -- 172.21.15.86:0/1920941412 wait complete. 2024-03-23T17:50:09.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.676+0000 7fc223e00700 1 Processor -- start 2024-03-23T17:50:09.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.676+0000 7fc223e00700 1 -- start start 2024-03-23T17:50:09.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.676+0000 7fc223e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc21c1016e0 0x7fc21c101000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:09.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.676+0000 7fc223e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc21c104010 0x7fc21c0ff690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:09.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.676+0000 7fc223e00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc21c106940 0x7fc21c0ffbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:09.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.676+0000 7fc223e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc21c100370 con 0x7fc21c104010 2024-03-23T17:50:09.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.676+0000 7fc223e00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc21c1004e0 con 0x7fc21c1016e0 2024-03-23T17:50:09.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.676+0000 7fc223e00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc21c1a0ca0 con 0x7fc21c106940 2024-03-23T17:50:09.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.676+0000 7fc220a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc21c104010 0x7fc21c0ff690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:09.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.676+0000 7fc221400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc21c1016e0 0x7fc21c101000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:09.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.676+0000 7fc220a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc21c104010 0x7fc21c0ff690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37990/0 (socket says 172.21.15.86:37990) 2024-03-23T17:50:09.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.676+0000 7fc220a00700 1 -- 172.21.15.86:0/4195254383 learned_addr learned my addr 172.21.15.86:0/4195254383 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:09.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.676+0000 7fc221e00700 1 --2- 172.21.15.86:0/4195254383 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc21c106940 0x7fc21c0ffbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:09.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.677+0000 7fc221400700 1 -- 172.21.15.86:0/4195254383 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc21c106940 msgr2=0x7fc21c0ffbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:09.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.677+0000 7fc221400700 1 --2- 172.21.15.86:0/4195254383 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc21c106940 0x7fc21c0ffbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:09.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.677+0000 7fc221400700 1 -- 172.21.15.86:0/4195254383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc21c104010 msgr2=0x7fc21c0ff690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:09.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.677+0000 7fc221400700 1 --2- 172.21.15.86:0/4195254383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc21c104010 0x7fc21c0ff690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:09.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.677+0000 7fc221400700 1 -- 172.21.15.86:0/4195254383 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc210009520 con 0x7fc21c1016e0 2024-03-23T17:50:09.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.677+0000 7fc220a00700 1 --2- 172.21.15.86:0/4195254383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc21c104010 0x7fc21c0ff690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:50:09.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.677+0000 7fc221400700 1 --2- 172.21.15.86:0/4195254383 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc21c1016e0 0x7fc21c101000 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc210009230 tx=0x7fc21000b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:09.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.677+0000 7fc21a000700 1 -- 172.21.15.86:0/4195254383 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc21001c070 con 0x7fc21c1016e0 2024-03-23T17:50:09.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.677+0000 7fc21a000700 1 -- 172.21.15.86:0/4195254383 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc21000f7d0 con 0x7fc21c1016e0 2024-03-23T17:50:09.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.677+0000 7fc21a000700 1 -- 172.21.15.86:0/4195254383 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2100038e0 con 0x7fc21c1016e0 2024-03-23T17:50:09.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.677+0000 7fc223e00700 1 -- 172.21.15.86:0/4195254383 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc21c1a0ed0 con 0x7fc21c1016e0 2024-03-23T17:50:09.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.677+0000 7fc223e00700 1 -- 172.21.15.86:0/4195254383 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc21c1a1240 con 0x7fc21c1016e0 2024-03-23T17:50:09.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.678+0000 7fc223e00700 1 -- 172.21.15.86:0/4195254383 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc21c0fcc50 con 0x7fc21c1016e0 2024-03-23T17:50:09.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.680+0000 7fc21a000700 1 -- 172.21.15.86:0/4195254383 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fc210020030 con 0x7fc21c1016e0 2024-03-23T17:50:09.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.681+0000 7fc21a000700 1 --2- 172.21.15.86:0/4195254383 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fc2040a11b0 0x7fc2040a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:09.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.682+0000 7fc21a000700 1 -- 172.21.15.86:0/4195254383 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fc2100cdd20 con 0x7fc21c1016e0 2024-03-23T17:50:09.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.682+0000 7fc220a00700 1 --2- 172.21.15.86:0/4195254383 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fc2040a11b0 0x7fc2040a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:09.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.682+0000 7fc220a00700 1 --2- 172.21.15.86:0/4195254383 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fc2040a11b0 0x7fc2040a3670 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc208005950 tx=0x7fc208009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:09.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.684+0000 7fc21a000700 1 -- 172.21.15.86:0/4195254383 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc2100974d0 con 0x7fc21c1016e0 2024-03-23T17:50:09.766 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:09 smithi086 ceph-mon[38221]: pgmap v272: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-23T17:50:09.766 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:09 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/947937446' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T17:50:09.766 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:09 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/2247123991' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T17:50:09.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.940+0000 7fc223e00700 1 -- 172.21.15.86:0/4195254383 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc21c067a10 con 0x7fc21c1016e0 2024-03-23T17:50:09.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.941+0000 7fc21a000700 1 -- 172.21.15.86:0/4195254383 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10913 (secure 0 0 0) 0x7fc21009aaf0 con 0x7fc21c1016e0 2024-03-23T17:50:09.942 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:50:09.942 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14445,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1433140219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1433140219},{"type":"v1","addr":"172.21.15.86:6835","nonce":1433140219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6839/1879457730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1879457730},{"type":"v1","addr":"172.21.15.86:6839","nonce":1879457730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-23T17:48:56.202323+0000","modified":"2024-03-23T17:49:03.703276+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24553,"mds_2":24518,"mds_3":24505,"mds_4":24500},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14514":{"gid":14514,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/3026943866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3026943866},{"type":"v1","addr":"172.21.15.86:6841","nonce":3026943866}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24464":{"gid":24464,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6833/1467091186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1467091186},{"type":"v1","addr":"172.21.15.92:6833","nonce":1467091186}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6833/3075554623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3075554623},{"type":"v1","addr":"172.21.15.152:6833","nonce":3075554623}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.92:6835/908900027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":908900027},{"type":"v1","addr":"172.21.15.92:6835","nonce":908900027}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"d","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.86:6837/2933881328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2933881328},{"type":"v1","addr":"172.21.15.86:6837","nonce":2933881328}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6837/3967837948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":3967837948},{"type":"v1","addr":"172.21.15.92:6837","nonce":3967837948}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6835/3137085021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3137085021},{"type":"v1","addr":"172.21.15.152:6835","nonce":3137085021}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24538":{"gid":24538,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6837/1151208965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":1151208965},{"type":"v1","addr":"172.21.15.152:6837","nonce":1151208965}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.152:6839/1100723992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":1100723992},{"type":"v1","addr":"172.21.15.152:6839","nonce":1100723992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6839/1394139887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1394139887},{"type":"v1","addr":"172.21.15.92:6839","nonce":1394139887}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-23T17:50:09.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.944+0000 7fc223e00700 1 -- 172.21.15.86:0/4195254383 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fc2040a11b0 msgr2=0x7fc2040a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:09.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.944+0000 7fc223e00700 1 --2- 172.21.15.86:0/4195254383 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fc2040a11b0 0x7fc2040a3670 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc208005950 tx=0x7fc208009310 comp rx=0 tx=0).stop 2024-03-23T17:50:09.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.944+0000 7fc223e00700 1 -- 172.21.15.86:0/4195254383 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc21c1016e0 msgr2=0x7fc21c101000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:09.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.944+0000 7fc223e00700 1 --2- 172.21.15.86:0/4195254383 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc21c1016e0 0x7fc21c101000 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc210009230 tx=0x7fc21000b790 comp rx=0 tx=0).stop 2024-03-23T17:50:09.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.944+0000 7fc223e00700 1 -- 172.21.15.86:0/4195254383 shutdown_connections 2024-03-23T17:50:09.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.944+0000 7fc223e00700 1 --2- 172.21.15.86:0/4195254383 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc21c106940 0x7fc21c0ffbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:09.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.944+0000 7fc223e00700 1 --2- 172.21.15.86:0/4195254383 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc21c1016e0 0x7fc21c101000 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:09.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.944+0000 7fc223e00700 1 --2- 172.21.15.86:0/4195254383 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fc2040a11b0 0x7fc2040a3670 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:09.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.944+0000 7fc223e00700 1 --2- 172.21.15.86:0/4195254383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc21c104010 0x7fc21c0ff690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:09.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.944+0000 7fc223e00700 1 -- 172.21.15.86:0/4195254383 >> 172.21.15.86:0/4195254383 conn(0x7fc21c0fb0d0 msgr2=0x7fc21c103610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:09.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.944+0000 7fc223e00700 1 -- 172.21.15.86:0/4195254383 shutdown_connections 2024-03-23T17:50:09.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:09.944+0000 7fc223e00700 1 -- 172.21.15.86:0/4195254383 wait complete. 2024-03-23T17:50:09.945 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 27 2024-03-23T17:50:09.966 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T17:50:10.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:09 smithi092 ceph-mon[79089]: pgmap v272: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-23T17:50:10.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:09 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/947937446' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T17:50:10.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:09 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/2247123991' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T17:50:10.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:09 smithi152 ceph-mon[47949]: pgmap v272: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-23T17:50:10.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:09 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/947937446' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T17:50:10.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:09 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/2247123991' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T17:50:10.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.156+0000 7f0753800700 1 -- 172.21.15.86:0/2389108685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f074c100c80 msgr2=0x7f074c1010a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:10.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.157+0000 7f0753800700 1 --2- 172.21.15.86:0/2389108685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f074c100c80 0x7f074c1010a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f07400099c0 tx=0x7f0740009cd0 comp rx=0 tx=0).stop 2024-03-23T17:50:10.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.157+0000 7f0753800700 1 -- 172.21.15.86:0/2389108685 shutdown_connections 2024-03-23T17:50:10.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.157+0000 7f0753800700 1 --2- 172.21.15.86:0/2389108685 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f074c102810 0x7f074c1090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:10.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.157+0000 7f0753800700 1 --2- 172.21.15.86:0/2389108685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f074c100c80 0x7f074c1010a0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:10.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.157+0000 7f0753800700 1 --2- 172.21.15.86:0/2389108685 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f074c101e70 0x7f074c1022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:10.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.157+0000 7f0753800700 1 -- 172.21.15.86:0/2389108685 >> 172.21.15.86:0/2389108685 conn(0x7f074c0fc1f0 msgr2=0x7f074c0fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:10.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.157+0000 7f0753800700 1 -- 172.21.15.86:0/2389108685 shutdown_connections 2024-03-23T17:50:10.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.157+0000 7f0753800700 1 -- 172.21.15.86:0/2389108685 wait complete. 2024-03-23T17:50:10.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.158+0000 7f0753800700 1 Processor -- start 2024-03-23T17:50:10.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.158+0000 7f0753800700 1 -- start start 2024-03-23T17:50:10.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.158+0000 7f0753800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f074c100c80 0x7f074c198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:10.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.158+0000 7f0753800700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f074c101e70 0x7f074c1993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:10.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.158+0000 7f0753800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f074c102810 0x7f074c1a03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:10.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.158+0000 7f0753800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f074c1a0990 con 0x7f074c102810 2024-03-23T17:50:10.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.158+0000 7f0753800700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f074c1a0ad0 con 0x7f074c100c80 2024-03-23T17:50:10.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.158+0000 7f0753800700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f074c1a0c60 con 0x7f074c101e70 2024-03-23T17:50:10.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.158+0000 7f0751800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f074c102810 0x7f074c1a03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:10.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.158+0000 7f0750e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f074c100c80 0x7f074c198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:10.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.158+0000 7f0751800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f074c102810 0x7f074c1a03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38006/0 (socket says 172.21.15.86:38006) 2024-03-23T17:50:10.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.158+0000 7f0751800700 1 -- 172.21.15.86:0/857146404 learned_addr learned my addr 172.21.15.86:0/857146404 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:10.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.158+0000 7f074be00700 1 --2- 172.21.15.86:0/857146404 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f074c101e70 0x7f074c1993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:10.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.159+0000 7f0750e00700 1 -- 172.21.15.86:0/857146404 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f074c101e70 msgr2=0x7f074c1993a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:10.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.159+0000 7f0750e00700 1 --2- 172.21.15.86:0/857146404 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f074c101e70 0x7f074c1993a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:10.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.159+0000 7f0750e00700 1 -- 172.21.15.86:0/857146404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f074c102810 msgr2=0x7f074c1a03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:10.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.159+0000 7f0750e00700 1 --2- 172.21.15.86:0/857146404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f074c102810 0x7f074c1a03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:10.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.159+0000 7f0750e00700 1 -- 172.21.15.86:0/857146404 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07400095f0 con 0x7f074c100c80 2024-03-23T17:50:10.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.159+0000 7f0751800700 1 --2- 172.21.15.86:0/857146404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f074c102810 0x7f074c1a03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:50:10.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.159+0000 7f0750e00700 1 --2- 172.21.15.86:0/857146404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f074c100c80 0x7f074c198e60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f07400098e0 tx=0x7f074000f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:10.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.159+0000 7f0749600700 1 -- 172.21.15.86:0/857146404 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f074001d070 con 0x7f074c100c80 2024-03-23T17:50:10.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.159+0000 7f0749600700 1 -- 172.21.15.86:0/857146404 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0740005050 con 0x7f074c100c80 2024-03-23T17:50:10.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.159+0000 7f0753800700 1 -- 172.21.15.86:0/857146404 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f074c1a0ee0 con 0x7f074c100c80 2024-03-23T17:50:10.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.159+0000 7f0749600700 1 -- 172.21.15.86:0/857146404 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0740004a50 con 0x7f074c100c80 2024-03-23T17:50:10.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.159+0000 7f0753800700 1 -- 172.21.15.86:0/857146404 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f074c1a1250 con 0x7f074c100c80 2024-03-23T17:50:10.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.160+0000 7f0753800700 1 -- 172.21.15.86:0/857146404 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f074c067a10 con 0x7f074c100c80 2024-03-23T17:50:10.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.162+0000 7f0749600700 1 -- 172.21.15.86:0/857146404 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f07400051c0 con 0x7f074c100c80 2024-03-23T17:50:10.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.164+0000 7f0749600700 1 --2- 172.21.15.86:0/857146404 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f07340a11b0 0x7f07340a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:10.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.164+0000 7f0749600700 1 -- 172.21.15.86:0/857146404 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f07400cd8f0 con 0x7f074c100c80 2024-03-23T17:50:10.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.164+0000 7f074be00700 1 --2- 172.21.15.86:0/857146404 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f07340a11b0 0x7f07340a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:10.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.164+0000 7f074be00700 1 --2- 172.21.15.86:0/857146404 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f07340a11b0 0x7f07340a3670 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0738009c60 tx=0x7f0738009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:10.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.166+0000 7f0749600700 1 -- 172.21.15.86:0/857146404 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f0740096f80 con 0x7f074c100c80 2024-03-23T17:50:10.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.402+0000 7f0753800700 1 -- 172.21.15.86:0/857146404 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f074c10c700 con 0x7f074c100c80 2024-03-23T17:50:10.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.404+0000 7f0749600700 1 -- 172.21.15.86:0/857146404 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22414 (secure 0 0 0) 0x7f074009a5a0 con 0x7f074c100c80 2024-03-23T17:50:10.405 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:50:10.405 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":82,"fsid":"9d76c9e8-e93c-11ee-95cd-87774f69a715","created":"2024-03-23T17:42:30.097233+0000","modified":"2024-03-23T17:49:29.773954+0000","last_up_change":"2024-03-23T17:48:18.217601+0000","last_in_change":"2024-03-23T17:48:00.250579+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T17:45:26.848397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T17:48:54.331894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T17:48:55.640693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"656655c6-1c7f-4640-92c4-2f42de820143","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6803","nonce":2423505491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6805","nonce":2423505491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6809","nonce":2423505491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6807","nonce":2423505491}]},"public_addr":"172.21.15.86:6803/2423505491","cluster_addr":"172.21.15.86:6805/2423505491","heartbeat_back_addr":"172.21.15.86:6809/2423505491","heartbeat_front_addr":"172.21.15.86:6807/2423505491","state":["exists","up"]},{"osd":1,"uuid":"3e86dc65-20a6-4de8-b9ce-fcf01f68862b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6801","nonce":3776252597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6803","nonce":3776252597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6807","nonce":3776252597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6805","nonce":3776252597}]},"public_addr":"172.21.15.92:6801/3776252597","cluster_addr":"172.21.15.92:6803/3776252597","heartbeat_back_addr":"172.21.15.92:6807/3776252597","heartbeat_front_addr":"172.21.15.92:6805/3776252597","state":["exists","up"]},{"osd":2,"uuid":"09808786-3bd3-4d11-8c7b-1736119dfbb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6801","nonce":1601265891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6803","nonce":1601265891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6807","nonce":1601265891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6805","nonce":1601265891}]},"public_addr":"172.21.15.152:6801/1601265891","cluster_addr":"172.21.15.152:6803/1601265891","heartbeat_back_addr":"172.21.15.152:6807/1601265891","heartbeat_front_addr":"172.21.15.152:6805/1601265891","state":["exists","up"]},{"osd":3,"uuid":"79c61910-4b53-424c-b150-6f5385a2cd68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6811","nonce":694846466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6813","nonce":694846466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6817","nonce":694846466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6815","nonce":694846466}]},"public_addr":"172.21.15.86:6811/694846466","cluster_addr":"172.21.15.86:6813/694846466","heartbeat_back_addr":"172.21.15.86:6817/694846466","heartbeat_front_addr":"172.21.15.86:6815/694846466","state":["exists","up"]},{"osd":4,"uuid":"a2878035-0179-469f-bbd4-8ce2d3f6bddc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6809","nonce":1422821471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6811","nonce":1422821471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6815","nonce":1422821471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6813","nonce":1422821471}]},"public_addr":"172.21.15.92:6809/1422821471","cluster_addr":"172.21.15.92:6811/1422821471","heartbeat_back_addr":"172.21.15.92:6815/1422821471","heartbeat_front_addr":"172.21.15.92:6813/1422821471","state":["exists","up"]},{"osd":5,"uuid":"837366aa-ce25-42ca-b157-5bbb50159d0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6809","nonce":1039375239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6811","nonce":1039375239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6815","nonce":1039375239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6813","nonce":1039375239}]},"public_addr":"172.21.15.152:6809/1039375239","cluster_addr":"172.21.15.152:6811/1039375239","heartbeat_back_addr":"172.21.15.152:6815/1039375239","heartbeat_front_addr":"172.21.15.152:6813/1039375239","state":["exists","up"]},{"osd":6,"uuid":"61024437-7b91-432f-a1af-6443d0ef5213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6819","nonce":4080452707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6821","nonce":4080452707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6825","nonce":4080452707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6823","nonce":4080452707}]},"public_addr":"172.21.15.86:6819/4080452707","cluster_addr":"172.21.15.86:6821/4080452707","heartbeat_back_addr":"172.21.15.86:6825/4080452707","heartbeat_front_addr":"172.21.15.86:6823/4080452707","state":["exists","up"]},{"osd":7,"uuid":"2684bdae-6b30-4afc-af68-4ce41b3a75b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6817","nonce":4187707622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6819","nonce":4187707622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6823","nonce":4187707622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6821","nonce":4187707622}]},"public_addr":"172.21.15.92:6817/4187707622","cluster_addr":"172.21.15.92:6819/4187707622","heartbeat_back_addr":"172.21.15.92:6823/4187707622","heartbeat_front_addr":"172.21.15.92:6821/4187707622","state":["exists","up"]},{"osd":8,"uuid":"f03bbf0d-1281-4bcf-9773-9854388e65ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6817","nonce":3237612748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6819","nonce":3237612748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6823","nonce":3237612748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6821","nonce":3237612748}]},"public_addr":"172.21.15.152:6817/3237612748","cluster_addr":"172.21.15.152:6819/3237612748","heartbeat_back_addr":"172.21.15.152:6823/3237612748","heartbeat_front_addr":"172.21.15.152:6821/3237612748","state":["exists","up"]},{"osd":9,"uuid":"e7e8b775-99ff-48a4-a23a-91e7ab6622bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6827","nonce":2835218294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6829","nonce":2835218294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6833","nonce":2835218294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6831","nonce":2835218294}]},"public_addr":"172.21.15.86:6827/2835218294","cluster_addr":"172.21.15.86:6829/2835218294","heartbeat_back_addr":"172.21.15.86:6833/2835218294","heartbeat_front_addr":"172.21.15.86:6831/2835218294","state":["exists","up"]},{"osd":10,"uuid":"d9d616cd-a229-44de-8768-02a4415df115","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6825","nonce":4259465100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6827","nonce":4259465100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6831","nonce":4259465100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6829","nonce":4259465100}]},"public_addr":"172.21.15.92:6825/4259465100","cluster_addr":"172.21.15.92:6827/4259465100","heartbeat_back_addr":"172.21.15.92:6831/4259465100","heartbeat_front_addr":"172.21.15.92:6829/4259465100","state":["exists","up"]},{"osd":11,"uuid":"e7c0b865-ccf5-4233-8d22-0d9b6dbe5c97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6825","nonce":194662760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6827","nonce":194662760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6831","nonce":194662760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6829","nonce":194662760}]},"public_addr":"172.21.15.152:6825/194662760","cluster_addr":"172.21.15.152:6827/194662760","heartbeat_back_addr":"172.21.15.152:6831/194662760","heartbeat_front_addr":"172.21.15.152:6829/194662760","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:44:47.269167+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:45:05.363655+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:45:23.192677+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:45:44.156228+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:02.532971+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:20.153295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:41.713888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:59.676516+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:47:17.207087+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:47:38.970316+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:47:57.282525+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:48:15.437606+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/3553500912":"2024-03-24T17:43:28.242492+0000","172.21.15.86:6801/3600278228":"2024-03-24T17:43:28.242492+0000","172.21.15.86:0/887262329":"2024-03-24T17:43:28.242492+0000","172.21.15.86:6800/2313723492":"2024-03-24T17:43:05.043910+0000","172.21.15.86:6800/3600278228":"2024-03-24T17:43:28.242492+0000","172.21.15.86:0/3815705642":"2024-03-24T17:43:05.043910+0000","172.21.15.86:0/1614943062":"2024-03-24T17:43:28.242492+0000","172.21.15.86:6801/2313723492":"2024-03-24T17:43:05.043910+0000","172.21.15.86:0/2416321716":"2024-03-24T17:43:05.043910+0000","172.21.15.86:0/433588188":"2024-03-24T17:43:05.043910+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T17:50:10.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.407+0000 7f0753800700 1 -- 172.21.15.86:0/857146404 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f07340a11b0 msgr2=0x7f07340a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:10.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.407+0000 7f0753800700 1 --2- 172.21.15.86:0/857146404 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f07340a11b0 0x7f07340a3670 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0738009c60 tx=0x7f0738009260 comp rx=0 tx=0).stop 2024-03-23T17:50:10.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.407+0000 7f0753800700 1 -- 172.21.15.86:0/857146404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f074c100c80 msgr2=0x7f074c198e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:10.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.407+0000 7f0753800700 1 --2- 172.21.15.86:0/857146404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f074c100c80 0x7f074c198e60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f07400098e0 tx=0x7f074000f690 comp rx=0 tx=0).stop 2024-03-23T17:50:10.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.407+0000 7f0753800700 1 -- 172.21.15.86:0/857146404 shutdown_connections 2024-03-23T17:50:10.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.407+0000 7f0753800700 1 --2- 172.21.15.86:0/857146404 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f07340a11b0 0x7f07340a3670 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:10.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.407+0000 7f0753800700 1 --2- 172.21.15.86:0/857146404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f074c102810 0x7f074c1a03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:10.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.407+0000 7f0753800700 1 --2- 172.21.15.86:0/857146404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f074c100c80 0x7f074c198e60 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:10.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.407+0000 7f0753800700 1 --2- 172.21.15.86:0/857146404 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f074c101e70 0x7f074c1993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:10.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.407+0000 7f0753800700 1 -- 172.21.15.86:0/857146404 >> 172.21.15.86:0/857146404 conn(0x7f074c0fc1f0 msgr2=0x7f074c072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:10.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.407+0000 7f0753800700 1 -- 172.21.15.86:0/857146404 shutdown_connections 2024-03-23T17:50:10.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.407+0000 7f0753800700 1 -- 172.21.15.86:0/857146404 wait complete. 2024-03-23T17:50:10.426 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T17:45:26.848397+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T17:48:54.331894+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T17:48:55.640693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T17:50:10.427 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T17:50:10.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.620+0000 7f400d200700 1 -- 172.21.15.86:0/1128440851 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4008100d80 msgr2=0x7f40081011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:10.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.620+0000 7f400d200700 1 --2- 172.21.15.86:0/1128440851 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4008100d80 0x7f40081011a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f3ff80099c0 tx=0x7f3ff8009cd0 comp rx=0 tx=0).stop 2024-03-23T17:50:10.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.620+0000 7f400d200700 1 -- 172.21.15.86:0/1128440851 shutdown_connections 2024-03-23T17:50:10.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.620+0000 7f400d200700 1 --2- 172.21.15.86:0/1128440851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4008102910 0x7f40081091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:10.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.620+0000 7f400d200700 1 --2- 172.21.15.86:0/1128440851 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4008100d80 0x7f40081011a0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:10.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.620+0000 7f400d200700 1 --2- 172.21.15.86:0/1128440851 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4008101f70 0x7f40081023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:10.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.620+0000 7f400d200700 1 -- 172.21.15.86:0/1128440851 >> 172.21.15.86:0/1128440851 conn(0x7f40080fc2f0 msgr2=0x7f40080fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:10.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.620+0000 7f400d200700 1 -- 172.21.15.86:0/1128440851 shutdown_connections 2024-03-23T17:50:10.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.621+0000 7f400d200700 1 -- 172.21.15.86:0/1128440851 wait complete. 2024-03-23T17:50:10.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.621+0000 7f400d200700 1 Processor -- start 2024-03-23T17:50:10.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.622+0000 7f400d200700 1 -- start start 2024-03-23T17:50:10.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.622+0000 7f400d200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4008100d80 0x7f4008198fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:10.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.622+0000 7f400d200700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4008101f70 0x7f40081994f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:10.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.622+0000 7f400d200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4008102910 0x7f40081a04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:10.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.622+0000 7f400d200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40081a0a90 con 0x7f4008102910 2024-03-23T17:50:10.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.622+0000 7f400d200700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40081a0bd0 con 0x7f4008100d80 2024-03-23T17:50:10.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.622+0000 7f400d200700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40081a0d60 con 0x7f4008101f70 2024-03-23T17:50:10.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.622+0000 7f4005e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4008100d80 0x7f4008198fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:10.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.622+0000 7f4005e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4008100d80 0x7f4008198fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:54508/0 (socket says 172.21.15.86:54508) 2024-03-23T17:50:10.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.622+0000 7f4005e00700 1 -- 172.21.15.86:0/4289479586 learned_addr learned my addr 172.21.15.86:0/4289479586 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:10.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.622+0000 7f4006800700 1 --2- 172.21.15.86:0/4289479586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4008102910 0x7f40081a04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:10.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.622+0000 7f4005400700 1 --2- 172.21.15.86:0/4289479586 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4008101f70 0x7f40081994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:10.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.622+0000 7f4005e00700 1 -- 172.21.15.86:0/4289479586 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4008101f70 msgr2=0x7f40081994f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:10.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.622+0000 7f4005e00700 1 --2- 172.21.15.86:0/4289479586 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4008101f70 0x7f40081994f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:10.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.622+0000 7f4005e00700 1 -- 172.21.15.86:0/4289479586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4008102910 msgr2=0x7f40081a04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:10.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.622+0000 7f4005e00700 1 --2- 172.21.15.86:0/4289479586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4008102910 0x7f40081a04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:10.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.622+0000 7f4005e00700 1 -- 172.21.15.86:0/4289479586 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ff80095f0 con 0x7f4008100d80 2024-03-23T17:50:10.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.623+0000 7f4005e00700 1 --2- 172.21.15.86:0/4289479586 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4008100d80 0x7f4008198fb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8006010 tx=0x7f3ff800f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:10.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.623+0000 7f3ffea00700 1 -- 172.21.15.86:0/4289479586 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff801d070 con 0x7f4008100d80 2024-03-23T17:50:10.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.623+0000 7f3ffea00700 1 -- 172.21.15.86:0/4289479586 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ff8005020 con 0x7f4008100d80 2024-03-23T17:50:10.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.623+0000 7f3ffea00700 1 -- 172.21.15.86:0/4289479586 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff8004b10 con 0x7f4008100d80 2024-03-23T17:50:10.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.623+0000 7f400d200700 1 -- 172.21.15.86:0/4289479586 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40081a0fe0 con 0x7f4008100d80 2024-03-23T17:50:10.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.623+0000 7f400d200700 1 -- 172.21.15.86:0/4289479586 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40081a1350 con 0x7f4008100d80 2024-03-23T17:50:10.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.624+0000 7f400d200700 1 -- 172.21.15.86:0/4289479586 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4008067af0 con 0x7f4008100d80 2024-03-23T17:50:10.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.626+0000 7f3ffea00700 1 -- 172.21.15.86:0/4289479586 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f3ff8005190 con 0x7f4008100d80 2024-03-23T17:50:10.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.627+0000 7f3ffea00700 1 --2- 172.21.15.86:0/4289479586 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f3fec0a11b0 0x7f3fec0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:10.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.627+0000 7f3ffea00700 1 -- 172.21.15.86:0/4289479586 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f3ff80cd670 con 0x7f4008100d80 2024-03-23T17:50:10.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.627+0000 7f4005400700 1 --2- 172.21.15.86:0/4289479586 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f3fec0a11b0 0x7f3fec0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:10.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.628+0000 7f4005400700 1 --2- 172.21.15.86:0/4289479586 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f3fec0a11b0 0x7f3fec0a3670 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3ff4009c60 tx=0x7f3ff4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:10.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.629+0000 7f3ffea00700 1 -- 172.21.15.86:0/4289479586 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f3ff8096d00 con 0x7f4008100d80 2024-03-23T17:50:10.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.837+0000 7f400d200700 1 -- 172.21.15.86:0/4289479586 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f400810c850 con 0x7f4008100d80 2024-03-23T17:50:10.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.838+0000 7f3ffea00700 1 -- 172.21.15.86:0/4289479586 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3ff809a320 con 0x7f4008100d80 2024-03-23T17:50:10.838 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-03-23T17:50:10.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.840+0000 7f400d200700 1 -- 172.21.15.86:0/4289479586 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f3fec0a11b0 msgr2=0x7f3fec0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:10.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.840+0000 7f400d200700 1 --2- 172.21.15.86:0/4289479586 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f3fec0a11b0 0x7f3fec0a3670 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3ff4009c60 tx=0x7f3ff4009260 comp rx=0 tx=0).stop 2024-03-23T17:50:10.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.840+0000 7f400d200700 1 -- 172.21.15.86:0/4289479586 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4008100d80 msgr2=0x7f4008198fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:10.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.840+0000 7f400d200700 1 --2- 172.21.15.86:0/4289479586 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4008100d80 0x7f4008198fb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8006010 tx=0x7f3ff800f690 comp rx=0 tx=0).stop 2024-03-23T17:50:10.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.840+0000 7f400d200700 1 -- 172.21.15.86:0/4289479586 shutdown_connections 2024-03-23T17:50:10.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.840+0000 7f400d200700 1 --2- 172.21.15.86:0/4289479586 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f3fec0a11b0 0x7f3fec0a3670 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:10.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.840+0000 7f400d200700 1 --2- 172.21.15.86:0/4289479586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4008102910 0x7f40081a04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:10.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.840+0000 7f400d200700 1 --2- 172.21.15.86:0/4289479586 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4008100d80 0x7f4008198fb0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:10.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.840+0000 7f400d200700 1 --2- 172.21.15.86:0/4289479586 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4008101f70 0x7f40081994f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:10.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.840+0000 7f400d200700 1 -- 172.21.15.86:0/4289479586 >> 172.21.15.86:0/4289479586 conn(0x7f40080fc2f0 msgr2=0x7f4008107790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:10.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.840+0000 7f400d200700 1 -- 172.21.15.86:0/4289479586 shutdown_connections 2024-03-23T17:50:10.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:10.840+0000 7f400d200700 1 -- 172.21.15.86:0/4289479586 wait complete. 2024-03-23T17:50:10.859 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T17:50:11.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:10 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/4195254383' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T17:50:11.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:10 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/857146404' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T17:50:11.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.054+0000 7fa0b3e00700 1 -- 172.21.15.86:0/2736558265 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0ac100d80 msgr2=0x7fa0ac1011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:11.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.054+0000 7fa0b3e00700 1 --2- 172.21.15.86:0/2736558265 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0ac100d80 0x7fa0ac1011a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa0a0009920 tx=0x7fa0a0009c30 comp rx=0 tx=0).stop 2024-03-23T17:50:11.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.054+0000 7fa0b3e00700 1 -- 172.21.15.86:0/2736558265 shutdown_connections 2024-03-23T17:50:11.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.054+0000 7fa0b3e00700 1 --2- 172.21.15.86:0/2736558265 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0ac102910 0x7fa0ac1091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:11.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.054+0000 7fa0b3e00700 1 --2- 172.21.15.86:0/2736558265 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0ac100d80 0x7fa0ac1011a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:11.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.054+0000 7fa0b3e00700 1 --2- 172.21.15.86:0/2736558265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0ac101f70 0x7fa0ac1023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:11.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.054+0000 7fa0b3e00700 1 -- 172.21.15.86:0/2736558265 >> 172.21.15.86:0/2736558265 conn(0x7fa0ac0fc2f0 msgr2=0x7fa0ac0fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:11.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.054+0000 7fa0b3e00700 1 -- 172.21.15.86:0/2736558265 shutdown_connections 2024-03-23T17:50:11.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.054+0000 7fa0b3e00700 1 -- 172.21.15.86:0/2736558265 wait complete. 2024-03-23T17:50:11.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.055+0000 7fa0b3e00700 1 Processor -- start 2024-03-23T17:50:11.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.056+0000 7fa0b3e00700 1 -- start start 2024-03-23T17:50:11.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.056+0000 7fa0b3e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0ac100d80 0x7fa0ac074ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:11.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.056+0000 7fa0b3e00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0ac101f70 0x7fa0ac0731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:11.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.056+0000 7fa0b3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0ac102910 0x7fa0ac0736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:11.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.056+0000 7fa0b3e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0ac073de0 con 0x7fa0ac102910 2024-03-23T17:50:11.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.056+0000 7fa0b3e00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0ac073f50 con 0x7fa0ac100d80 2024-03-23T17:50:11.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.056+0000 7fa0b3e00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0ac1a0e90 con 0x7fa0ac101f70 2024-03-23T17:50:11.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.056+0000 7fa0b1e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0ac102910 0x7fa0ac0736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:11.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.056+0000 7fa0b1e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0ac102910 0x7fa0ac0736e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38064/0 (socket says 172.21.15.86:38064) 2024-03-23T17:50:11.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.056+0000 7fa0b1e00700 1 -- 172.21.15.86:0/42080976 learned_addr learned my addr 172.21.15.86:0/42080976 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:11.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.056+0000 7fa0b0a00700 1 --2- 172.21.15.86:0/42080976 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0ac101f70 0x7fa0ac0731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:11.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.056+0000 7fa0b1400700 1 --2- 172.21.15.86:0/42080976 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0ac100d80 0x7fa0ac074ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:11.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.057+0000 7fa0b1e00700 1 -- 172.21.15.86:0/42080976 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0ac101f70 msgr2=0x7fa0ac0731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:11.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.057+0000 7fa0b1e00700 1 --2- 172.21.15.86:0/42080976 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0ac101f70 0x7fa0ac0731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:11.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.057+0000 7fa0b1e00700 1 -- 172.21.15.86:0/42080976 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0ac100d80 msgr2=0x7fa0ac074ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:11.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.057+0000 7fa0b1e00700 1 --2- 172.21.15.86:0/42080976 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0ac100d80 0x7fa0ac074ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:11.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.057+0000 7fa0b1e00700 1 -- 172.21.15.86:0/42080976 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0a0009520 con 0x7fa0ac102910 2024-03-23T17:50:11.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.057+0000 7fa0b1e00700 1 --2- 172.21.15.86:0/42080976 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0ac102910 0x7fa0ac0736e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa0a400ef30 tx=0x7fa0a400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:11.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.057+0000 7fa0aa000700 1 -- 172.21.15.86:0/42080976 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a4018070 con 0x7fa0ac102910 2024-03-23T17:50:11.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.057+0000 7fa0aa000700 1 -- 172.21.15.86:0/42080976 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0a400f040 con 0x7fa0ac102910 2024-03-23T17:50:11.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.057+0000 7fa0b3e00700 1 -- 172.21.15.86:0/42080976 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0ac1a1090 con 0x7fa0ac102910 2024-03-23T17:50:11.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.057+0000 7fa0aa000700 1 -- 172.21.15.86:0/42080976 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a400ccf0 con 0x7fa0ac102910 2024-03-23T17:50:11.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.057+0000 7fa0b3e00700 1 -- 172.21.15.86:0/42080976 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0ac1a1550 con 0x7fa0ac102910 2024-03-23T17:50:11.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.058+0000 7fa0b3e00700 1 -- 172.21.15.86:0/42080976 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0ac1082c0 con 0x7fa0ac102910 2024-03-23T17:50:11.061 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:10 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/4195254383' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T17:50:11.061 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:10 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/857146404' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T17:50:11.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.060+0000 7fa0aa000700 1 -- 172.21.15.86:0/42080976 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fa0a4003a00 con 0x7fa0ac102910 2024-03-23T17:50:11.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.061+0000 7fa0aa000700 1 --2- 172.21.15.86:0/42080976 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fa0940a11b0 0x7fa0940a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:11.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.061+0000 7fa0aa000700 1 -- 172.21.15.86:0/42080976 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fa0a40cc5d0 con 0x7fa0ac102910 2024-03-23T17:50:11.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.061+0000 7fa0b1400700 1 --2- 172.21.15.86:0/42080976 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fa0940a11b0 0x7fa0940a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:11.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.062+0000 7fa0b1400700 1 --2- 172.21.15.86:0/42080976 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fa0940a11b0 0x7fa0940a3670 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa0a00098f0 tx=0x7fa0a0019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:11.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.065+0000 7fa0aa000700 1 -- 172.21.15.86:0/42080976 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa0a4095c60 con 0x7fa0ac102910 2024-03-23T17:50:11.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:10 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/4195254383' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T17:50:11.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:10 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/857146404' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T17:50:11.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.270+0000 7fa0b3e00700 1 -- 172.21.15.86:0/42080976 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa0ac107ca0 con 0x7fa0ac102910 2024-03-23T17:50:11.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.271+0000 7fa0aa000700 1 -- 172.21.15.86:0/42080976 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v82) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa0a4099280 con 0x7fa0ac102910 2024-03-23T17:50:11.271 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-03-23T17:50:11.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.273+0000 7fa0b3e00700 1 -- 172.21.15.86:0/42080976 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fa0940a11b0 msgr2=0x7fa0940a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:11.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.273+0000 7fa0b3e00700 1 --2- 172.21.15.86:0/42080976 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fa0940a11b0 0x7fa0940a3670 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa0a00098f0 tx=0x7fa0a0019040 comp rx=0 tx=0).stop 2024-03-23T17:50:11.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.273+0000 7fa0b3e00700 1 -- 172.21.15.86:0/42080976 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0ac102910 msgr2=0x7fa0ac0736e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:11.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.273+0000 7fa0b3e00700 1 --2- 172.21.15.86:0/42080976 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0ac102910 0x7fa0ac0736e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa0a400ef30 tx=0x7fa0a400c3a0 comp rx=0 tx=0).stop 2024-03-23T17:50:11.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.274+0000 7fa0b3e00700 1 -- 172.21.15.86:0/42080976 shutdown_connections 2024-03-23T17:50:11.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.274+0000 7fa0b3e00700 1 --2- 172.21.15.86:0/42080976 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fa0940a11b0 0x7fa0940a3670 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:11.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.274+0000 7fa0b3e00700 1 --2- 172.21.15.86:0/42080976 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0ac102910 0x7fa0ac0736e0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:11.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.274+0000 7fa0b3e00700 1 --2- 172.21.15.86:0/42080976 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0ac100d80 0x7fa0ac074ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:11.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.274+0000 7fa0b3e00700 1 --2- 172.21.15.86:0/42080976 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0ac101f70 0x7fa0ac0731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:11.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.274+0000 7fa0b3e00700 1 -- 172.21.15.86:0/42080976 >> 172.21.15.86:0/42080976 conn(0x7fa0ac0fc2f0 msgr2=0x7fa0ac105400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:11.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.274+0000 7fa0b3e00700 1 -- 172.21.15.86:0/42080976 shutdown_connections 2024-03-23T17:50:11.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.274+0000 7fa0b3e00700 1 -- 172.21.15.86:0/42080976 wait complete. 2024-03-23T17:50:11.292 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T17:50:11.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.489+0000 7f58ab800700 1 -- 172.21.15.86:0/1592002420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58a4101f70 msgr2=0x7f58a41023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:11.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.489+0000 7f58ab800700 1 --2- 172.21.15.86:0/1592002420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58a4101f70 0x7f58a41023d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f58900099c0 tx=0x7f5890009cd0 comp rx=0 tx=0).stop 2024-03-23T17:50:11.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.489+0000 7f58ab800700 1 -- 172.21.15.86:0/1592002420 shutdown_connections 2024-03-23T17:50:11.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.489+0000 7f58ab800700 1 --2- 172.21.15.86:0/1592002420 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58a4102910 0x7f58a41091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:11.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.489+0000 7f58ab800700 1 --2- 172.21.15.86:0/1592002420 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58a4100d80 0x7f58a41011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:11.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.490+0000 7f58ab800700 1 --2- 172.21.15.86:0/1592002420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58a4101f70 0x7f58a41023d0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:11.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.490+0000 7f58ab800700 1 -- 172.21.15.86:0/1592002420 >> 172.21.15.86:0/1592002420 conn(0x7f58a40fc2f0 msgr2=0x7f58a40fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:11.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.490+0000 7f58ab800700 1 -- 172.21.15.86:0/1592002420 shutdown_connections 2024-03-23T17:50:11.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.490+0000 7f58ab800700 1 -- 172.21.15.86:0/1592002420 wait complete. 2024-03-23T17:50:11.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.491+0000 7f58ab800700 1 Processor -- start 2024-03-23T17:50:11.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.491+0000 7f58ab800700 1 -- start start 2024-03-23T17:50:11.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.491+0000 7f58ab800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58a4100d80 0x7f58a4074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:11.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.491+0000 7f58ab800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58a4101f70 0x7f58a40731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:11.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.491+0000 7f58ab800700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58a4102910 0x7f58a40736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:11.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.491+0000 7f58ab800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a4073e80 con 0x7f58a4100d80 2024-03-23T17:50:11.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.491+0000 7f58ab800700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a4073ff0 con 0x7f58a4101f70 2024-03-23T17:50:11.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.491+0000 7f58ab800700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a41a0d70 con 0x7f58a4102910 2024-03-23T17:50:11.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.491+0000 7f58a8e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58a4100d80 0x7f58a4074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:11.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.491+0000 7f58a8e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58a4100d80 0x7f58a4074b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38086/0 (socket says 172.21.15.86:38086) 2024-03-23T17:50:11.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.491+0000 7f58a8e00700 1 -- 172.21.15.86:0/4221395961 learned_addr learned my addr 172.21.15.86:0/4221395961 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:11.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.491+0000 7f58a3e00700 1 --2- 172.21.15.86:0/4221395961 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58a4101f70 0x7f58a40731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:11.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.491+0000 7f58a9800700 1 --2- 172.21.15.86:0/4221395961 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58a4102910 0x7f58a40736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:11.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.492+0000 7f58a8e00700 1 -- 172.21.15.86:0/4221395961 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58a4102910 msgr2=0x7f58a40736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:11.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.492+0000 7f58a8e00700 1 --2- 172.21.15.86:0/4221395961 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58a4102910 0x7f58a40736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:11.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.492+0000 7f58a8e00700 1 -- 172.21.15.86:0/4221395961 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58a4101f70 msgr2=0x7f58a40731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:11.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.492+0000 7f58a8e00700 1 --2- 172.21.15.86:0/4221395961 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58a4101f70 0x7f58a40731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:11.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.492+0000 7f58a8e00700 1 -- 172.21.15.86:0/4221395961 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58900095f0 con 0x7f58a4100d80 2024-03-23T17:50:11.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.492+0000 7f58a8e00700 1 --2- 172.21.15.86:0/4221395961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58a4100d80 0x7f58a4074b10 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f589800e970 tx=0x7f589800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:11.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.492+0000 7f58a1600700 1 -- 172.21.15.86:0/4221395961 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5898009790 con 0x7f58a4100d80 2024-03-23T17:50:11.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.492+0000 7f58a1600700 1 -- 172.21.15.86:0/4221395961 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f589800c780 con 0x7f58a4100d80 2024-03-23T17:50:11.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.492+0000 7f58a1600700 1 -- 172.21.15.86:0/4221395961 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5898003be0 con 0x7f58a4100d80 2024-03-23T17:50:11.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.492+0000 7f58ab800700 1 -- 172.21.15.86:0/4221395961 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58a41a0fa0 con 0x7f58a4100d80 2024-03-23T17:50:11.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.492+0000 7f58ab800700 1 -- 172.21.15.86:0/4221395961 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58a41a13d0 con 0x7f58a4100d80 2024-03-23T17:50:11.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.493+0000 7f58ab800700 1 -- 172.21.15.86:0/4221395961 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58a4108250 con 0x7f58a4100d80 2024-03-23T17:50:11.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.495+0000 7f58a1600700 1 -- 172.21.15.86:0/4221395961 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f589800cbf0 con 0x7f58a4100d80 2024-03-23T17:50:11.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.496+0000 7f58a1600700 1 --2- 172.21.15.86:0/4221395961 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f588c0a1230 0x7f588c0a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:11.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.497+0000 7f58a1600700 1 -- 172.21.15.86:0/4221395961 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f5898014070 con 0x7f58a4100d80 2024-03-23T17:50:11.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.497+0000 7f58a3e00700 1 --2- 172.21.15.86:0/4221395961 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f588c0a1230 0x7f588c0a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:11.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.497+0000 7f58a3e00700 1 --2- 172.21.15.86:0/4221395961 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f588c0a1230 0x7f588c0a36f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5890006010 tx=0x7f589000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:11.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.500+0000 7f58a1600700 1 -- 172.21.15.86:0/4221395961 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f5898096180 con 0x7f58a4100d80 2024-03-23T17:50:11.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.705+0000 7f58ab800700 1 -- 172.21.15.86:0/4221395961 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f58a410c5e0 con 0x7f58a4100d80 2024-03-23T17:50:11.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.706+0000 7f58a1600700 1 -- 172.21.15.86:0/4221395961 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v82) v1 ==== 100+0+11 (secure 0 0 0) 0x7f58980997a0 con 0x7f58a4100d80 2024-03-23T17:50:11.706 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-03-23T17:50:11.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.708+0000 7f58ab800700 1 -- 172.21.15.86:0/4221395961 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f588c0a1230 msgr2=0x7f588c0a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:11.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.708+0000 7f58ab800700 1 --2- 172.21.15.86:0/4221395961 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f588c0a1230 0x7f588c0a36f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5890006010 tx=0x7f589000b540 comp rx=0 tx=0).stop 2024-03-23T17:50:11.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.708+0000 7f58ab800700 1 -- 172.21.15.86:0/4221395961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58a4100d80 msgr2=0x7f58a4074b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:11.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.708+0000 7f58ab800700 1 --2- 172.21.15.86:0/4221395961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58a4100d80 0x7f58a4074b10 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f589800e970 tx=0x7f589800ec80 comp rx=0 tx=0).stop 2024-03-23T17:50:11.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.709+0000 7f58ab800700 1 -- 172.21.15.86:0/4221395961 shutdown_connections 2024-03-23T17:50:11.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.709+0000 7f58ab800700 1 --2- 172.21.15.86:0/4221395961 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58a4102910 0x7f58a40736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:11.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.709+0000 7f58ab800700 1 --2- 172.21.15.86:0/4221395961 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f588c0a1230 0x7f588c0a36f0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:11.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.709+0000 7f58ab800700 1 --2- 172.21.15.86:0/4221395961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58a4100d80 0x7f58a4074b10 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:11.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.709+0000 7f58ab800700 1 --2- 172.21.15.86:0/4221395961 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58a4101f70 0x7f58a40731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:11.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.709+0000 7f58ab800700 1 -- 172.21.15.86:0/4221395961 >> 172.21.15.86:0/4221395961 conn(0x7f58a40fc2f0 msgr2=0x7f58a4105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:11.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.709+0000 7f58ab800700 1 -- 172.21.15.86:0/4221395961 shutdown_connections 2024-03-23T17:50:11.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.709+0000 7f58ab800700 1 -- 172.21.15.86:0/4221395961 wait complete. 2024-03-23T17:50:11.727 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T17:50:11.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.918+0000 7f9c60a00700 1 -- 172.21.15.86:0/3607374601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c5c100c80 msgr2=0x7f9c5c1010a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:11.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.918+0000 7f9c60a00700 1 --2- 172.21.15.86:0/3607374601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c5c100c80 0x7f9c5c1010a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c0099c0 tx=0x7f9c4c009cd0 comp rx=0 tx=0).stop 2024-03-23T17:50:11.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.918+0000 7f9c60a00700 1 -- 172.21.15.86:0/3607374601 shutdown_connections 2024-03-23T17:50:11.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.918+0000 7f9c60a00700 1 --2- 172.21.15.86:0/3607374601 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c5c102810 0x7f9c5c1090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:11.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.918+0000 7f9c60a00700 1 --2- 172.21.15.86:0/3607374601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c5c100c80 0x7f9c5c1010a0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:11.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.918+0000 7f9c60a00700 1 --2- 172.21.15.86:0/3607374601 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c5c101e70 0x7f9c5c1022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:11.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.918+0000 7f9c60a00700 1 -- 172.21.15.86:0/3607374601 >> 172.21.15.86:0/3607374601 conn(0x7f9c5c0fc1f0 msgr2=0x7f9c5c0fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:11.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.918+0000 7f9c60a00700 1 -- 172.21.15.86:0/3607374601 shutdown_connections 2024-03-23T17:50:11.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.918+0000 7f9c60a00700 1 -- 172.21.15.86:0/3607374601 wait complete. 2024-03-23T17:50:11.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.919+0000 7f9c60a00700 1 Processor -- start 2024-03-23T17:50:11.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.919+0000 7f9c60a00700 1 -- start start 2024-03-23T17:50:11.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.919+0000 7f9c60a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c5c100c80 0x7f9c5c196c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:11.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.919+0000 7f9c60a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c5c101e70 0x7f9c5c197140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:11.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.919+0000 7f9c60a00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c5c102810 0x7f9c5c19e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:11.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.919+0000 7f9c60a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5c19e730 con 0x7f9c5c101e70 2024-03-23T17:50:11.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.919+0000 7f9c60a00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5c19e870 con 0x7f9c5c100c80 2024-03-23T17:50:11.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.919+0000 7f9c60a00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5c19ea00 con 0x7f9c5c102810 2024-03-23T17:50:11.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.920+0000 7f9c59400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c5c101e70 0x7f9c5c197140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:11.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.920+0000 7f9c59400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c5c101e70 0x7f9c5c197140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38110/0 (socket says 172.21.15.86:38110) 2024-03-23T17:50:11.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.920+0000 7f9c59400700 1 -- 172.21.15.86:0/2433426463 learned_addr learned my addr 172.21.15.86:0/2433426463 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:11.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.920+0000 7f9c59e00700 1 --2- 172.21.15.86:0/2433426463 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c5c100c80 0x7f9c5c196c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:11.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.920+0000 7f9c5a800700 1 --2- 172.21.15.86:0/2433426463 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c5c102810 0x7f9c5c19e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:11.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.920+0000 7f9c59400700 1 -- 172.21.15.86:0/2433426463 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c5c102810 msgr2=0x7f9c5c19e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:11.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.920+0000 7f9c59400700 1 --2- 172.21.15.86:0/2433426463 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c5c102810 0x7f9c5c19e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:11.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.920+0000 7f9c59400700 1 -- 172.21.15.86:0/2433426463 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c5c100c80 msgr2=0x7f9c5c196c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:11.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.920+0000 7f9c59400700 1 --2- 172.21.15.86:0/2433426463 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c5c100c80 0x7f9c5c196c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:11.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.920+0000 7f9c59400700 1 -- 172.21.15.86:0/2433426463 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c4c0095f0 con 0x7f9c5c101e70 2024-03-23T17:50:11.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.920+0000 7f9c59400700 1 --2- 172.21.15.86:0/2433426463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c5c101e70 0x7f9c5c197140 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9c4800d950 tx=0x7f9c4800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:11.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.920+0000 7f9c52a00700 1 -- 172.21.15.86:0/2433426463 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c48009940 con 0x7f9c5c101e70 2024-03-23T17:50:11.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.921+0000 7f9c52a00700 1 -- 172.21.15.86:0/2433426463 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c48010460 con 0x7f9c5c101e70 2024-03-23T17:50:11.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.921+0000 7f9c60a00700 1 -- 172.21.15.86:0/2433426463 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c5c19ece0 con 0x7f9c5c101e70 2024-03-23T17:50:11.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.921+0000 7f9c52a00700 1 -- 172.21.15.86:0/2433426463 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c4800f620 con 0x7f9c5c101e70 2024-03-23T17:50:11.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.921+0000 7f9c60a00700 1 -- 172.21.15.86:0/2433426463 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c5c19f1e0 con 0x7f9c5c101e70 2024-03-23T17:50:11.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.922+0000 7f9c60a00700 1 -- 172.21.15.86:0/2433426463 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c5c108100 con 0x7f9c5c101e70 2024-03-23T17:50:11.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.923+0000 7f9c52a00700 1 -- 172.21.15.86:0/2433426463 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f9c480108d0 con 0x7f9c5c101e70 2024-03-23T17:50:11.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.925+0000 7f9c52a00700 1 --2- 172.21.15.86:0/2433426463 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f9c400a1230 0x7f9c400a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:11.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.925+0000 7f9c52a00700 1 -- 172.21.15.86:0/2433426463 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f9c480cc890 con 0x7f9c5c101e70 2024-03-23T17:50:11.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.925+0000 7f9c59e00700 1 --2- 172.21.15.86:0/2433426463 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f9c400a1230 0x7f9c400a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:11.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.925+0000 7f9c59e00700 1 --2- 172.21.15.86:0/2433426463 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f9c400a1230 0x7f9c400a36f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c006010 tx=0x7f9c4c00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:11.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:11.928+0000 7f9c52a00700 1 -- 172.21.15.86:0/2433426463 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f9c48095e90 con 0x7f9c5c101e70 2024-03-23T17:50:12.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:11 smithi086 ceph-mon[38221]: pgmap v273: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-23T17:50:12.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:11 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/4289479586' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T17:50:12.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:11 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/42080976' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T17:50:12.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:11 smithi092 ceph-mon[79089]: pgmap v273: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-23T17:50:12.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:11 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/4289479586' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T17:50:12.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:11 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/42080976' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T17:50:12.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:11 smithi152 ceph-mon[47949]: pgmap v273: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-23T17:50:12.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:11 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/4289479586' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T17:50:12.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:11 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/42080976' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T17:50:12.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.181+0000 7f9c60a00700 1 -- 172.21.15.86:0/2433426463 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9c5c067a10 con 0x7f9c5c101e70 2024-03-23T17:50:12.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.181+0000 7f9c52a00700 1 -- 172.21.15.86:0/2433426463 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10913 (secure 0 0 0) 0x7f9c48016090 con 0x7f9c5c101e70 2024-03-23T17:50:12.181 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:50:12.182 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14445,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1433140219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1433140219},{"type":"v1","addr":"172.21.15.86:6835","nonce":1433140219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6839/1879457730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1879457730},{"type":"v1","addr":"172.21.15.86:6839","nonce":1879457730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-23T17:48:56.202323+0000","modified":"2024-03-23T17:49:03.703276+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24553,"mds_2":24518,"mds_3":24505,"mds_4":24500},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14514":{"gid":14514,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/3026943866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3026943866},{"type":"v1","addr":"172.21.15.86:6841","nonce":3026943866}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24464":{"gid":24464,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6833/1467091186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1467091186},{"type":"v1","addr":"172.21.15.92:6833","nonce":1467091186}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6833/3075554623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3075554623},{"type":"v1","addr":"172.21.15.152:6833","nonce":3075554623}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.92:6835/908900027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":908900027},{"type":"v1","addr":"172.21.15.92:6835","nonce":908900027}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"d","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.86:6837/2933881328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2933881328},{"type":"v1","addr":"172.21.15.86:6837","nonce":2933881328}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6837/3967837948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":3967837948},{"type":"v1","addr":"172.21.15.92:6837","nonce":3967837948}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6835/3137085021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3137085021},{"type":"v1","addr":"172.21.15.152:6835","nonce":3137085021}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24538":{"gid":24538,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6837/1151208965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":1151208965},{"type":"v1","addr":"172.21.15.152:6837","nonce":1151208965}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.152:6839/1100723992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":1100723992},{"type":"v1","addr":"172.21.15.152:6839","nonce":1100723992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6839/1394139887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1394139887},{"type":"v1","addr":"172.21.15.92:6839","nonce":1394139887}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-23T17:50:12.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.184+0000 7f9c60a00700 1 -- 172.21.15.86:0/2433426463 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f9c400a1230 msgr2=0x7f9c400a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:12.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.184+0000 7f9c60a00700 1 --2- 172.21.15.86:0/2433426463 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f9c400a1230 0x7f9c400a36f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c006010 tx=0x7f9c4c00b540 comp rx=0 tx=0).stop 2024-03-23T17:50:12.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.184+0000 7f9c60a00700 1 -- 172.21.15.86:0/2433426463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c5c101e70 msgr2=0x7f9c5c197140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:12.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.184+0000 7f9c60a00700 1 --2- 172.21.15.86:0/2433426463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c5c101e70 0x7f9c5c197140 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9c4800d950 tx=0x7f9c4800dc60 comp rx=0 tx=0).stop 2024-03-23T17:50:12.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.184+0000 7f9c60a00700 1 -- 172.21.15.86:0/2433426463 shutdown_connections 2024-03-23T17:50:12.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.184+0000 7f9c60a00700 1 --2- 172.21.15.86:0/2433426463 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c5c102810 0x7f9c5c19e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:12.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.184+0000 7f9c60a00700 1 --2- 172.21.15.86:0/2433426463 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c5c100c80 0x7f9c5c196c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:12.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.184+0000 7f9c60a00700 1 --2- 172.21.15.86:0/2433426463 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f9c400a1230 0x7f9c400a36f0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:12.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.184+0000 7f9c60a00700 1 --2- 172.21.15.86:0/2433426463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c5c101e70 0x7f9c5c197140 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:12.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.184+0000 7f9c60a00700 1 -- 172.21.15.86:0/2433426463 >> 172.21.15.86:0/2433426463 conn(0x7f9c5c0fc1f0 msgr2=0x7f9c5c105240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:12.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.184+0000 7f9c60a00700 1 -- 172.21.15.86:0/2433426463 shutdown_connections 2024-03-23T17:50:12.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.184+0000 7f9c60a00700 1 -- 172.21.15.86:0/2433426463 wait complete. 2024-03-23T17:50:12.185 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 27 2024-03-23T17:50:12.203 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T17:50:12.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.394+0000 7f6760a00700 1 -- 172.21.15.86:0/2909505923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f675c102e90 msgr2=0x7f675c1032b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:12.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.394+0000 7f6760a00700 1 --2- 172.21.15.86:0/2909505923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f675c102e90 0x7f675c1032b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f674c009990 tx=0x7f674c009ca0 comp rx=0 tx=0).stop 2024-03-23T17:50:12.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.394+0000 7f6760a00700 1 -- 172.21.15.86:0/2909505923 shutdown_connections 2024-03-23T17:50:12.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.394+0000 7f6760a00700 1 --2- 172.21.15.86:0/2909505923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f675c1049a0 0x7f675c10b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:12.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.394+0000 7f6760a00700 1 --2- 172.21.15.86:0/2909505923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f675c102e90 0x7f675c1032b0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:12.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.394+0000 7f6760a00700 1 --2- 172.21.15.86:0/2909505923 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f675c103fe0 0x7f675c104460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:12.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.394+0000 7f6760a00700 1 -- 172.21.15.86:0/2909505923 >> 172.21.15.86:0/2909505923 conn(0x7f675c0fe420 msgr2=0x7f675c100840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:12.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.394+0000 7f6760a00700 1 -- 172.21.15.86:0/2909505923 shutdown_connections 2024-03-23T17:50:12.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.394+0000 7f6760a00700 1 -- 172.21.15.86:0/2909505923 wait complete. 2024-03-23T17:50:12.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.395+0000 7f6760a00700 1 Processor -- start 2024-03-23T17:50:12.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.395+0000 7f6760a00700 1 -- start start 2024-03-23T17:50:12.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.396+0000 7f6760a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f675c102e90 0x7f675c198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:12.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.396+0000 7f6760a00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f675c103fe0 0x7f675c199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:12.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.396+0000 7f6760a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f675c1049a0 0x7f675c1a03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:12.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.396+0000 7f6760a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f675c1a0970 con 0x7f675c1049a0 2024-03-23T17:50:12.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.396+0000 7f6760a00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f675c1a0ab0 con 0x7f675c102e90 2024-03-23T17:50:12.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.396+0000 7f6760a00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f675c1a0c40 con 0x7f675c103fe0 2024-03-23T17:50:12.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.396+0000 7f6759e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f675c102e90 0x7f675c198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:12.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.396+0000 7f6759400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f675c103fe0 0x7f675c199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:12.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.396+0000 7f6759e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f675c102e90 0x7f675c198e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:54592/0 (socket says 172.21.15.86:54592) 2024-03-23T17:50:12.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.396+0000 7f675a800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f675c1049a0 0x7f675c1a03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:12.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.396+0000 7f6759e00700 1 -- 172.21.15.86:0/760963510 learned_addr learned my addr 172.21.15.86:0/760963510 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:12.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.396+0000 7f6759400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f675c103fe0 0x7f675c199380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:52078/0 (socket says 172.21.15.86:52078) 2024-03-23T17:50:12.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.396+0000 7f6759e00700 1 -- 172.21.15.86:0/760963510 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f675c103fe0 msgr2=0x7f675c199380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:12.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.396+0000 7f6759e00700 1 --2- 172.21.15.86:0/760963510 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f675c103fe0 0x7f675c199380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:12.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.396+0000 7f6759e00700 1 -- 172.21.15.86:0/760963510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f675c1049a0 msgr2=0x7f675c1a03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:12.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.397+0000 7f6759e00700 1 --2- 172.21.15.86:0/760963510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f675c1049a0 0x7f675c1a03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:12.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.397+0000 7f6759e00700 1 -- 172.21.15.86:0/760963510 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6754009c40 con 0x7f675c102e90 2024-03-23T17:50:12.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.397+0000 7f6759400700 1 --2- 172.21.15.86:0/760963510 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f675c103fe0 0x7f675c199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:50:12.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.397+0000 7f675a800700 1 --2- 172.21.15.86:0/760963510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f675c1049a0 0x7f675c1a03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:50:12.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.397+0000 7f6759e00700 1 --2- 172.21.15.86:0/760963510 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f675c102e90 0x7f675c198e40 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f674c00b620 tx=0x7f674c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:12.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.397+0000 7f6752a00700 1 -- 172.21.15.86:0/760963510 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f674c01d070 con 0x7f675c102e90 2024-03-23T17:50:12.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.397+0000 7f6760a00700 1 -- 172.21.15.86:0/760963510 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f674c0095f0 con 0x7f675c102e90 2024-03-23T17:50:12.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.397+0000 7f6752a00700 1 -- 172.21.15.86:0/760963510 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f674c022470 con 0x7f675c102e90 2024-03-23T17:50:12.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.397+0000 7f6752a00700 1 -- 172.21.15.86:0/760963510 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f674c017650 con 0x7f675c102e90 2024-03-23T17:50:12.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.397+0000 7f6760a00700 1 -- 172.21.15.86:0/760963510 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f675c1a11b0 con 0x7f675c102e90 2024-03-23T17:50:12.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.398+0000 7f6760a00700 1 -- 172.21.15.86:0/760963510 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f675c10a350 con 0x7f675c102e90 2024-03-23T17:50:12.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.400+0000 7f6752a00700 1 -- 172.21.15.86:0/760963510 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f674c0225e0 con 0x7f675c102e90 2024-03-23T17:50:12.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.401+0000 7f6752a00700 1 --2- 172.21.15.86:0/760963510 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f67400a11b0 0x7f67400a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:12.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.402+0000 7f6752a00700 1 -- 172.21.15.86:0/760963510 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f674c0ce040 con 0x7f675c102e90 2024-03-23T17:50:12.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.402+0000 7f6759400700 1 --2- 172.21.15.86:0/760963510 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f67400a11b0 0x7f67400a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:12.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.402+0000 7f6759400700 1 --2- 172.21.15.86:0/760963510 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f67400a11b0 0x7f67400a3670 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6748011490 tx=0x7f6748009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:12.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.404+0000 7f6752a00700 1 -- 172.21.15.86:0/760963510 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f674c097730 con 0x7f675c102e90 2024-03-23T17:50:12.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.642+0000 7f6760a00700 1 -- 172.21.15.86:0/760963510 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f675c10e6e0 con 0x7f675c102e90 2024-03-23T17:50:12.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.644+0000 7f6752a00700 1 -- 172.21.15.86:0/760963510 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22414 (secure 0 0 0) 0x7f674c09ad50 con 0x7f675c102e90 2024-03-23T17:50:12.644 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:50:12.644 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":82,"fsid":"9d76c9e8-e93c-11ee-95cd-87774f69a715","created":"2024-03-23T17:42:30.097233+0000","modified":"2024-03-23T17:49:29.773954+0000","last_up_change":"2024-03-23T17:48:18.217601+0000","last_in_change":"2024-03-23T17:48:00.250579+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T17:45:26.848397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T17:48:54.331894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T17:48:55.640693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"656655c6-1c7f-4640-92c4-2f42de820143","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6803","nonce":2423505491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6805","nonce":2423505491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6809","nonce":2423505491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6807","nonce":2423505491}]},"public_addr":"172.21.15.86:6803/2423505491","cluster_addr":"172.21.15.86:6805/2423505491","heartbeat_back_addr":"172.21.15.86:6809/2423505491","heartbeat_front_addr":"172.21.15.86:6807/2423505491","state":["exists","up"]},{"osd":1,"uuid":"3e86dc65-20a6-4de8-b9ce-fcf01f68862b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6801","nonce":3776252597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6803","nonce":3776252597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6807","nonce":3776252597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6805","nonce":3776252597}]},"public_addr":"172.21.15.92:6801/3776252597","cluster_addr":"172.21.15.92:6803/3776252597","heartbeat_back_addr":"172.21.15.92:6807/3776252597","heartbeat_front_addr":"172.21.15.92:6805/3776252597","state":["exists","up"]},{"osd":2,"uuid":"09808786-3bd3-4d11-8c7b-1736119dfbb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6801","nonce":1601265891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6803","nonce":1601265891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6807","nonce":1601265891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6805","nonce":1601265891}]},"public_addr":"172.21.15.152:6801/1601265891","cluster_addr":"172.21.15.152:6803/1601265891","heartbeat_back_addr":"172.21.15.152:6807/1601265891","heartbeat_front_addr":"172.21.15.152:6805/1601265891","state":["exists","up"]},{"osd":3,"uuid":"79c61910-4b53-424c-b150-6f5385a2cd68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6811","nonce":694846466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6813","nonce":694846466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6817","nonce":694846466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6815","nonce":694846466}]},"public_addr":"172.21.15.86:6811/694846466","cluster_addr":"172.21.15.86:6813/694846466","heartbeat_back_addr":"172.21.15.86:6817/694846466","heartbeat_front_addr":"172.21.15.86:6815/694846466","state":["exists","up"]},{"osd":4,"uuid":"a2878035-0179-469f-bbd4-8ce2d3f6bddc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6809","nonce":1422821471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6811","nonce":1422821471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6815","nonce":1422821471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6813","nonce":1422821471}]},"public_addr":"172.21.15.92:6809/1422821471","cluster_addr":"172.21.15.92:6811/1422821471","heartbeat_back_addr":"172.21.15.92:6815/1422821471","heartbeat_front_addr":"172.21.15.92:6813/1422821471","state":["exists","up"]},{"osd":5,"uuid":"837366aa-ce25-42ca-b157-5bbb50159d0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6809","nonce":1039375239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6811","nonce":1039375239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6815","nonce":1039375239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6813","nonce":1039375239}]},"public_addr":"172.21.15.152:6809/1039375239","cluster_addr":"172.21.15.152:6811/1039375239","heartbeat_back_addr":"172.21.15.152:6815/1039375239","heartbeat_front_addr":"172.21.15.152:6813/1039375239","state":["exists","up"]},{"osd":6,"uuid":"61024437-7b91-432f-a1af-6443d0ef5213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6819","nonce":4080452707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6821","nonce":4080452707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6825","nonce":4080452707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6823","nonce":4080452707}]},"public_addr":"172.21.15.86:6819/4080452707","cluster_addr":"172.21.15.86:6821/4080452707","heartbeat_back_addr":"172.21.15.86:6825/4080452707","heartbeat_front_addr":"172.21.15.86:6823/4080452707","state":["exists","up"]},{"osd":7,"uuid":"2684bdae-6b30-4afc-af68-4ce41b3a75b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6817","nonce":4187707622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6819","nonce":4187707622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6823","nonce":4187707622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6821","nonce":4187707622}]},"public_addr":"172.21.15.92:6817/4187707622","cluster_addr":"172.21.15.92:6819/4187707622","heartbeat_back_addr":"172.21.15.92:6823/4187707622","heartbeat_front_addr":"172.21.15.92:6821/4187707622","state":["exists","up"]},{"osd":8,"uuid":"f03bbf0d-1281-4bcf-9773-9854388e65ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6817","nonce":3237612748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6819","nonce":3237612748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6823","nonce":3237612748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6821","nonce":3237612748}]},"public_addr":"172.21.15.152:6817/3237612748","cluster_addr":"172.21.15.152:6819/3237612748","heartbeat_back_addr":"172.21.15.152:6823/3237612748","heartbeat_front_addr":"172.21.15.152:6821/3237612748","state":["exists","up"]},{"osd":9,"uuid":"e7e8b775-99ff-48a4-a23a-91e7ab6622bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6827","nonce":2835218294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6829","nonce":2835218294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6833","nonce":2835218294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6831","nonce":2835218294}]},"public_addr":"172.21.15.86:6827/2835218294","cluster_addr":"172.21.15.86:6829/2835218294","heartbeat_back_addr":"172.21.15.86:6833/2835218294","heartbeat_front_addr":"172.21.15.86:6831/2835218294","state":["exists","up"]},{"osd":10,"uuid":"d9d616cd-a229-44de-8768-02a4415df115","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6825","nonce":4259465100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6827","nonce":4259465100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6831","nonce":4259465100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6829","nonce":4259465100}]},"public_addr":"172.21.15.92:6825/4259465100","cluster_addr":"172.21.15.92:6827/4259465100","heartbeat_back_addr":"172.21.15.92:6831/4259465100","heartbeat_front_addr":"172.21.15.92:6829/4259465100","state":["exists","up"]},{"osd":11,"uuid":"e7c0b865-ccf5-4233-8d22-0d9b6dbe5c97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6825","nonce":194662760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6827","nonce":194662760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6831","nonce":194662760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6829","nonce":194662760}]},"public_addr":"172.21.15.152:6825/194662760","cluster_addr":"172.21.15.152:6827/194662760","heartbeat_back_addr":"172.21.15.152:6831/194662760","heartbeat_front_addr":"172.21.15.152:6829/194662760","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:44:47.269167+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:45:05.363655+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:45:23.192677+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:45:44.156228+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:02.532971+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:20.153295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:41.713888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:59.676516+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:47:17.207087+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:47:38.970316+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:47:57.282525+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:48:15.437606+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/3553500912":"2024-03-24T17:43:28.242492+0000","172.21.15.86:6801/3600278228":"2024-03-24T17:43:28.242492+0000","172.21.15.86:0/887262329":"2024-03-24T17:43:28.242492+0000","172.21.15.86:6800/2313723492":"2024-03-24T17:43:05.043910+0000","172.21.15.86:6800/3600278228":"2024-03-24T17:43:28.242492+0000","172.21.15.86:0/3815705642":"2024-03-24T17:43:05.043910+0000","172.21.15.86:0/1614943062":"2024-03-24T17:43:28.242492+0000","172.21.15.86:6801/2313723492":"2024-03-24T17:43:05.043910+0000","172.21.15.86:0/2416321716":"2024-03-24T17:43:05.043910+0000","172.21.15.86:0/433588188":"2024-03-24T17:43:05.043910+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T17:50:12.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.646+0000 7f6760a00700 1 -- 172.21.15.86:0/760963510 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f67400a11b0 msgr2=0x7f67400a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:12.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.646+0000 7f6760a00700 1 --2- 172.21.15.86:0/760963510 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f67400a11b0 0x7f67400a3670 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6748011490 tx=0x7f6748009260 comp rx=0 tx=0).stop 2024-03-23T17:50:12.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.646+0000 7f6760a00700 1 -- 172.21.15.86:0/760963510 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f675c102e90 msgr2=0x7f675c198e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:12.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.646+0000 7f6760a00700 1 --2- 172.21.15.86:0/760963510 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f675c102e90 0x7f675c198e40 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f674c00b620 tx=0x7f674c004730 comp rx=0 tx=0).stop 2024-03-23T17:50:12.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.646+0000 7f6760a00700 1 -- 172.21.15.86:0/760963510 shutdown_connections 2024-03-23T17:50:12.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.646+0000 7f6760a00700 1 --2- 172.21.15.86:0/760963510 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f67400a11b0 0x7f67400a3670 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:12.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.646+0000 7f6760a00700 1 --2- 172.21.15.86:0/760963510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f675c1049a0 0x7f675c1a03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:12.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.646+0000 7f6760a00700 1 --2- 172.21.15.86:0/760963510 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f675c102e90 0x7f675c198e40 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:12.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.646+0000 7f6760a00700 1 --2- 172.21.15.86:0/760963510 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f675c103fe0 0x7f675c199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:12.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.646+0000 7f6760a00700 1 -- 172.21.15.86:0/760963510 >> 172.21.15.86:0/760963510 conn(0x7f675c0fe420 msgr2=0x7f675c107490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:12.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.646+0000 7f6760a00700 1 -- 172.21.15.86:0/760963510 shutdown_connections 2024-03-23T17:50:12.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.646+0000 7f6760a00700 1 -- 172.21.15.86:0/760963510 wait complete. 2024-03-23T17:50:12.666 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-23T17:48:56.202323+0000', 'modified': '2024-03-23T17:49:03.703276+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24563, 'mds_1': 24553, 'mds_2': 24518, 'mds_3': 24505, 'mds_4': 24500}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14514': {'gid': 14514, 'name': 'a', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.86:6841/3026943866', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6840', 'nonce': 3026943866}, {'type': 'v1', 'addr': '172.21.15.86:6841', 'nonce': 3026943866}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24464': {'gid': 24464, 'name': 'k', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.92:6833/1467091186', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6832', 'nonce': 1467091186}, {'type': 'v1', 'addr': '172.21.15.92:6833', 'nonce': 1467091186}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24475': {'gid': 24475, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.152:6833/3075554623', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6832', 'nonce': 3075554623}, {'type': 'v1', 'addr': '172.21.15.152:6833', 'nonce': 3075554623}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24500': {'gid': 24500, 'name': 'e', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.92:6835/908900027', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6834', 'nonce': 908900027}, {'type': 'v1', 'addr': '172.21.15.92:6835', 'nonce': 908900027}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24505': {'gid': 24505, 'name': 'd', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.86:6837/2933881328', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 2933881328}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 2933881328}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24518': {'gid': 24518, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.92:6837/3967837948', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6836', 'nonce': 3967837948}, {'type': 'v1', 'addr': '172.21.15.92:6837', 'nonce': 3967837948}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24526': {'gid': 24526, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.152:6835/3137085021', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6834', 'nonce': 3137085021}, {'type': 'v1', 'addr': '172.21.15.152:6835', 'nonce': 3137085021}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24538': {'gid': 24538, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.152:6837/1151208965', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6836', 'nonce': 1151208965}, {'type': 'v1', 'addr': '172.21.15.152:6837', 'nonce': 1151208965}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24553': {'gid': 24553, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.152:6839/1100723992', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6838', 'nonce': 1100723992}, {'type': 'v1', 'addr': '172.21.15.152:6839', 'nonce': 1100723992}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24563': {'gid': 24563, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.92:6839/1394139887', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6838', 'nonce': 1394139887}, {'type': 'v1', 'addr': '172.21.15.92:6839', 'nonce': 1394139887}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-23T17:50:12.666 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-03-23T17:50:12.667 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-03-23T17:50:12.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.860+0000 7f5530e00700 1 -- 172.21.15.86:0/3632680314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f552c102910 msgr2=0x7f552c1091c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:12.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.860+0000 7f5530e00700 1 --2- 172.21.15.86:0/3632680314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f552c102910 0x7f552c1091c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f552400b0d0 tx=0x7f552400b3e0 comp rx=0 tx=0).stop 2024-03-23T17:50:12.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.860+0000 7f5530e00700 1 -- 172.21.15.86:0/3632680314 shutdown_connections 2024-03-23T17:50:12.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.860+0000 7f5530e00700 1 --2- 172.21.15.86:0/3632680314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f552c102910 0x7f552c1091c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:12.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.860+0000 7f5530e00700 1 --2- 172.21.15.86:0/3632680314 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f552c100d80 0x7f552c1011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:12.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.860+0000 7f5530e00700 1 --2- 172.21.15.86:0/3632680314 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f552c101f70 0x7f552c1023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:12.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.860+0000 7f5530e00700 1 -- 172.21.15.86:0/3632680314 >> 172.21.15.86:0/3632680314 conn(0x7f552c0fc2f0 msgr2=0x7f552c0fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:12.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.860+0000 7f5530e00700 1 -- 172.21.15.86:0/3632680314 shutdown_connections 2024-03-23T17:50:12.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.860+0000 7f5530e00700 1 -- 172.21.15.86:0/3632680314 wait complete. 2024-03-23T17:50:12.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.861+0000 7f5530e00700 1 Processor -- start 2024-03-23T17:50:12.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.861+0000 7f5530e00700 1 -- start start 2024-03-23T17:50:12.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.861+0000 7f5530e00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f552c100d80 0x7f552c074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:12.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.861+0000 7f5530e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f552c101f70 0x7f552c0731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:12.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.861+0000 7f5530e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f552c102910 0x7f552c0736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:12.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.861+0000 7f5530e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f552c073e80 con 0x7f552c101f70 2024-03-23T17:50:12.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.861+0000 7f5530e00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f552c073ff0 con 0x7f552c102910 2024-03-23T17:50:12.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.861+0000 7f5530e00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f552c1a0e90 con 0x7f552c100d80 2024-03-23T17:50:12.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.862+0000 7f5529400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f552c101f70 0x7f552c0731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:12.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.862+0000 7f5529400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f552c101f70 0x7f552c0731a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38158/0 (socket says 172.21.15.86:38158) 2024-03-23T17:50:12.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.862+0000 7f5529400700 1 -- 172.21.15.86:0/100248616 learned_addr learned my addr 172.21.15.86:0/100248616 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:12.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.862+0000 7f552a800700 1 --2- 172.21.15.86:0/100248616 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f552c102910 0x7f552c0736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:12.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.862+0000 7f5529e00700 1 --2- 172.21.15.86:0/100248616 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f552c100d80 0x7f552c074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:12.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.862+0000 7f552a800700 1 -- 172.21.15.86:0/100248616 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f552c100d80 msgr2=0x7f552c074b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:12.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.862+0000 7f552a800700 1 --2- 172.21.15.86:0/100248616 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f552c100d80 0x7f552c074b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:12.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.862+0000 7f552a800700 1 -- 172.21.15.86:0/100248616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f552c101f70 msgr2=0x7f552c0731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:12.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.862+0000 7f552a800700 1 --2- 172.21.15.86:0/100248616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f552c101f70 0x7f552c0731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:12.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.862+0000 7f552a800700 1 -- 172.21.15.86:0/100248616 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5524009d00 con 0x7f552c102910 2024-03-23T17:50:12.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.862+0000 7f5529400700 1 --2- 172.21.15.86:0/100248616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f552c101f70 0x7f552c0731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:50:12.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.862+0000 7f552a800700 1 --2- 172.21.15.86:0/100248616 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f552c102910 0x7f552c0736e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5524007b60 tx=0x7f5524008e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:12.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.863+0000 7f5522a00700 1 -- 172.21.15.86:0/100248616 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f552400e040 con 0x7f552c102910 2024-03-23T17:50:12.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.863+0000 7f5522a00700 1 -- 172.21.15.86:0/100248616 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5524005070 con 0x7f552c102910 2024-03-23T17:50:12.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.863+0000 7f5522a00700 1 -- 172.21.15.86:0/100248616 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f552401d6d0 con 0x7f552c102910 2024-03-23T17:50:12.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.863+0000 7f5530e00700 1 -- 172.21.15.86:0/100248616 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f552c1a10c0 con 0x7f552c102910 2024-03-23T17:50:12.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.863+0000 7f5530e00700 1 -- 172.21.15.86:0/100248616 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f552c1a1490 con 0x7f552c102910 2024-03-23T17:50:12.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.865+0000 7f5522a00700 1 -- 172.21.15.86:0/100248616 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f55240051e0 con 0x7f552c102910 2024-03-23T17:50:12.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.867+0000 7f55449a9b80 1 Processor -- start 2024-03-23T17:50:12.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.867+0000 7f5522a00700 1 --2- 172.21.15.86:0/100248616 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f55100a11b0 0x7f55100a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:12.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.867+0000 7f55449a9b80 1 -- start start 2024-03-23T17:50:12.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.867+0000 7f5522a00700 1 -- 172.21.15.86:0/100248616 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f55240cd900 con 0x7f552c102910 2024-03-23T17:50:12.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.867+0000 7f5529400700 1 --2- 172.21.15.86:0/100248616 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f55100a11b0 0x7f55100a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:12.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.867+0000 7f55449a9b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5629d65f2780 0x5629d65f2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:12.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.867+0000 7f55449a9b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629d65f30e0 0x5629d65f39a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:12.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.867+0000 7f55449a9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629d65f3f80 0x5629d65f4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:12.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.867+0000 7f55449a9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629d654a940 con 0x5629d65f3f80 2024-03-23T17:50:12.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.867+0000 7f55449a9b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629d6548300 con 0x5629d65f2780 2024-03-23T17:50:12.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.867+0000 7f55449a9b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629d6543bf0 con 0x5629d65f30e0 2024-03-23T17:50:12.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.868+0000 7f552a800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629d65f3f80 0x5629d65f4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:12.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.868+0000 7f5529400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629d65f30e0 0x5629d65f39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:12.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.868+0000 7f5529e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5629d65f2780 0x5629d65f2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:12.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.868+0000 7f5529400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629d65f30e0 0x5629d65f39a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:52118/0 (socket says 172.21.15.86:52118) 2024-03-23T17:50:12.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.868+0000 7f5529400700 1 -- 172.21.15.86:0/677862923 learned_addr learned my addr 172.21.15.86:0/677862923 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:12.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.868+0000 7f552a800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629d65f3f80 0x5629d65f4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38160/0 (socket says 172.21.15.86:38160) 2024-03-23T17:50:12.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.868+0000 7f5529400700 1 --2- 172.21.15.86:0/100248616 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f55100a11b0 0x7f55100a3670 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5518011590 tx=0x7f5518011950 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:12.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.868+0000 7f5529400700 1 -- 172.21.15.86:0/677862923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5629d65f2780 msgr2=0x5629d65f2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:50:12.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.868+0000 7f5529400700 1 --2- 172.21.15.86:0/677862923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5629d65f2780 0x5629d65f2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:12.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.868+0000 7f5529400700 1 -- 172.21.15.86:0/677862923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629d65f3f80 msgr2=0x5629d65f4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:50:12.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.868+0000 7f5529400700 1 --2- 172.21.15.86:0/677862923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629d65f3f80 0x5629d65f4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:12.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.868+0000 7f5529400700 1 -- 172.21.15.86:0/677862923 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5629d63c5360 con 0x5629d65f30e0 2024-03-23T17:50:12.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.868+0000 7f552a800700 1 --2- 172.21.15.86:0/677862923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629d65f3f80 0x5629d65f4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:50:12.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.868+0000 7f5529400700 1 --2- 172.21.15.86:0/677862923 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629d65f30e0 0x5629d65f39a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f551801d7e0 tx=0x7f551801c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4c2a2dbf99d01e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:12.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.868+0000 7f5517400700 1 -- 172.21.15.86:0/677862923 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551800f680 con 0x5629d65f30e0 2024-03-23T17:50:12.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.869+0000 7f5517400700 1 -- 172.21.15.86:0/677862923 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f551800fca0 con 0x5629d65f30e0 2024-03-23T17:50:12.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.869+0000 7f5517400700 1 -- 172.21.15.86:0/677862923 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5518026c90 con 0x5629d65f30e0 2024-03-23T17:50:12.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.869+0000 7f55449a9b80 1 -- 172.21.15.86:0/677862923 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629d65f30e0 msgr2=0x5629d65f39a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:12.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.869+0000 7f55449a9b80 1 --2- 172.21.15.86:0/677862923 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629d65f30e0 0x5629d65f39a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f551801d7e0 tx=0x7f551801c000 comp rx=0 tx=0).stop 2024-03-23T17:50:12.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.869+0000 7f55449a9b80 1 -- 172.21.15.86:0/677862923 shutdown_connections 2024-03-23T17:50:12.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.869+0000 7f55449a9b80 1 --2- 172.21.15.86:0/677862923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629d65f3f80 0x5629d65f4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:12.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.869+0000 7f55449a9b80 1 --2- 172.21.15.86:0/677862923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5629d65f2780 0x5629d65f2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:12.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.869+0000 7f55449a9b80 1 --2- 172.21.15.86:0/677862923 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629d65f30e0 0x5629d65f39a0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:12.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.869+0000 7f55449a9b80 1 -- 172.21.15.86:0/677862923 >> 172.21.15.86:0/677862923 conn(0x5629d6551650 msgr2=0x5629d65ed950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:12.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.869+0000 7f55449a9b80 1 -- 172.21.15.86:0/677862923 shutdown_connections 2024-03-23T17:50:12.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.869+0000 7f55449a9b80 1 -- 172.21.15.86:0/677862923 wait complete. 2024-03-23T17:50:12.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.870+0000 7f55449a9b80 1 -- start start 2024-03-23T17:50:12.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.870+0000 7f55449a9b80 1 Processor -- start 2024-03-23T17:50:12.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.870+0000 7f55449a9b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5629d65f2780 0x5629d65fc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:12.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.870+0000 7f55449a9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629d65f30e0 0x5629d65fcb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:12.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.870+0000 7f55449a9b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629d65f3f80 0x5629d6603b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:12.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.870+0000 7f55449a9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629d64dfd30 con 0x5629d65f30e0 2024-03-23T17:50:12.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.870+0000 7f55449a9b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629d64df9d0 con 0x5629d65f2780 2024-03-23T17:50:12.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.870+0000 7f55449a9b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629d63c7650 con 0x5629d65f3f80 2024-03-23T17:50:12.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.871+0000 7f55449a9b80 10 client.0 _ll_register_callbacks cb 0x5629d64e9420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:50:12.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.871+0000 7f5529400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629d65f30e0 0x5629d65fcb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:12.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.871+0000 7f5529400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629d65f30e0 0x5629d65fcb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38166/0 (socket says 172.21.15.86:38166) 2024-03-23T17:50:12.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.871+0000 7f5529e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5629d65f2780 0x5629d65fc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:12.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.871+0000 7f5529400700 1 -- 172.21.15.86:0/3348904952 learned_addr learned my addr 172.21.15.86:0/3348904952 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:12.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.871+0000 7f552a800700 1 --2- 172.21.15.86:0/3348904952 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629d65f3f80 0x5629d6603b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:12.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.871+0000 7f5529400700 1 -- 172.21.15.86:0/3348904952 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629d65f3f80 msgr2=0x5629d6603b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:50:12.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.871+0000 7f5529400700 1 --2- 172.21.15.86:0/3348904952 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629d65f3f80 0x5629d6603b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:12.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.871+0000 7f5529400700 1 -- 172.21.15.86:0/3348904952 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5629d65f2780 msgr2=0x5629d65fc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:50:12.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.871+0000 7f5529400700 1 --2- 172.21.15.86:0/3348904952 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5629d65f2780 0x5629d65fc5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:12.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.871+0000 7f5529400700 1 -- 172.21.15.86:0/3348904952 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f552c1a1490 con 0x5629d65f30e0 2024-03-23T17:50:12.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.871+0000 7f5529400700 1 --2- 172.21.15.86:0/3348904952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629d65f30e0 0x5629d65fcb20 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f551800fe90 tx=0x7f551800fec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf1df6d1b3f9f1ff server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:12.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.871+0000 7f550be00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-23T17:50:12.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.871+0000 7f550be00700 1 -- 172.21.15.86:0/3348904952 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551800f480 con 0x5629d65f30e0 2024-03-23T17:50:12.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.872+0000 7f55449a9b80 1 -- 172.21.15.86:0/3348904952 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5629d6504330 con 0x5629d65f30e0 2024-03-23T17:50:12.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.872+0000 7f550be00700 1 -- 172.21.15.86:0/3348904952 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f551800f5e0 con 0x5629d65f30e0 2024-03-23T17:50:12.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.872+0000 7f550be00700 1 -- 172.21.15.86:0/3348904952 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f551800f7e0 con 0x5629d65f30e0 2024-03-23T17:50:12.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.872+0000 7f55449a9b80 10 client.15120 fetch_fsmap learned FSMap version 27 2024-03-23T17:50:12.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.872+0000 7f55449a9b80 1 -- 172.21.15.86:0/3348904952 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f551801d550 con 0x5629d65f30e0 2024-03-23T17:50:12.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.872+0000 7f550be00700 1 -- 172.21.15.86:0/3348904952 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5629d6504330 con 0x5629d65f30e0 2024-03-23T17:50:12.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.872+0000 7f550be00700 1 -- 172.21.15.86:0/3348904952 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f55180046f0 con 0x5629d65f30e0 2024-03-23T17:50:12.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.872+0000 7f55449a9b80 10 client.15120 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T17:50:12.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.872+0000 7f55449a9b80 10 client.15120 resolve_mds: resolved name 'e' to daemon mds.e 2024-03-23T17:50:12.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.872+0000 7f55449a9b80 20 client.15120 populate_metadata read hostname 'smithi086' 2024-03-23T17:50:12.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.872+0000 7f55449a9b80 1 --2- 172.21.15.86:0/3348904952 >> [v2:172.21.15.92:6834/908900027,v1:172.21.15.92:6835/908900027] conn(0x5629d6607700 0x5629d660c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:12.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.872+0000 7f55449a9b80 4 client.15120 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:50:12.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.872+0000 7f55449a9b80 1 -- 172.21.15.86:0/3348904952 --> [v2:172.21.15.92:6834/908900027,v1:172.21.15.92:6835/908900027] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5629d653c6b0 con 0x5629d6607700 2024-03-23T17:50:12.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.873+0000 7f5529e00700 1 --2- 172.21.15.86:0/3348904952 >> [v2:172.21.15.92:6834/908900027,v1:172.21.15.92:6835/908900027] conn(0x5629d6607700 0x5629d660c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:12.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.873+0000 7f5529e00700 1 --2- 172.21.15.86:0/3348904952 >> [v2:172.21.15.92:6834/908900027,v1:172.21.15.92:6835/908900027] conn(0x5629d6607700 0x5629d660c9e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=81d583eaca3b7201 server_cookie=451f9e03a900c503 in_seq=0 out_seq=0 2024-03-23T17:50:12.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.873+0000 7f550be00700 10 client.15120 ms_handle_connect on v2:172.21.15.92:6834/908900027 2024-03-23T17:50:12.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.874+0000 7f550be00700 1 -- 172.21.15.86:0/3348904952 <== mds.4 v2:172.21.15.92:6834/908900027 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5629d653c6b0 con 0x5629d6607700 2024-03-23T17:50:12.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.874+0000 7f550be00700 10 client.15120 handle_command_reply: tid=0 2024-03-23T17:50:12.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.874+0000 7f55449a9b80 1 client.15120 shutdown 2024-03-23T17:50:12.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.874+0000 7f55449a9b80 1 -- 172.21.15.86:0/3348904952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629d65f30e0 msgr2=0x5629d65fcb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:12.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.874+0000 7f55449a9b80 1 --2- 172.21.15.86:0/3348904952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629d65f30e0 0x5629d65fcb20 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f551800fe90 tx=0x7f551800fec0 comp rx=0 tx=0).stop 2024-03-23T17:50:12.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.874+0000 7f55449a9b80 1 -- 172.21.15.86:0/3348904952 shutdown_connections 2024-03-23T17:50:12.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.874+0000 7f55449a9b80 1 --2- 172.21.15.86:0/3348904952 >> [v2:172.21.15.92:6834/908900027,v1:172.21.15.92:6835/908900027] conn(0x5629d6607700 0x5629d660c9e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:12.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.874+0000 7f55449a9b80 1 --2- 172.21.15.86:0/3348904952 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629d65f3f80 0x5629d6603b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:12.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.874+0000 7f55449a9b80 1 --2- 172.21.15.86:0/3348904952 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5629d65f2780 0x5629d65fc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:12.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.874+0000 7f550be00700 0 client.15120 ms_handle_reset on v2:172.21.15.92:6834/908900027 2024-03-23T17:50:12.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.874+0000 7f55449a9b80 1 --2- 172.21.15.86:0/3348904952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629d65f30e0 0x5629d65fcb20 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:12.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.874+0000 7f55449a9b80 1 -- 172.21.15.86:0/3348904952 >> 172.21.15.86:0/3348904952 conn(0x5629d6551650 msgr2=0x5629d65f1120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:12.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.874+0000 7f55449a9b80 1 -- 172.21.15.86:0/3348904952 shutdown_connections 2024-03-23T17:50:12.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.874+0000 7f55449a9b80 1 -- 172.21.15.86:0/3348904952 wait complete. 2024-03-23T17:50:12.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.874+0000 7f55449a9b80 20 client.15120 trim_cache size 0 max 16384 2024-03-23T17:50:12.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.892+0000 7f55449a9b80 1 Processor -- start 2024-03-23T17:50:12.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.892+0000 7f55449a9b80 1 -- start start 2024-03-23T17:50:12.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.892+0000 7f55449a9b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629d65f2780 0x5629d65f2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:12.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.892+0000 7f55449a9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629d65f30e0 0x5629d6601b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:12.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.892+0000 7f55449a9b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5629d65f3f80 0x5629d65f4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:12.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.892+0000 7f55449a9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629d65fd530 con 0x5629d65f30e0 2024-03-23T17:50:12.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.892+0000 7f55449a9b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629d6608f90 con 0x5629d65f3f80 2024-03-23T17:50:12.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.892+0000 7f55449a9b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629d653c6b0 con 0x5629d65f2780 2024-03-23T17:50:12.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.892+0000 7f5529400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629d65f30e0 0x5629d6601b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:12.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.893+0000 7f5529400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629d65f30e0 0x5629d6601b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38182/0 (socket says 172.21.15.86:38182) 2024-03-23T17:50:12.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.893+0000 7f5529400700 1 -- 172.21.15.86:0/4224669487 learned_addr learned my addr 172.21.15.86:0/4224669487 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:12.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.893+0000 7f5529e00700 1 --2- 172.21.15.86:0/4224669487 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629d65f2780 0x5629d65f2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:12.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.893+0000 7f552a800700 1 --2- 172.21.15.86:0/4224669487 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5629d65f3f80 0x5629d65f4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:12.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.893+0000 7f5529400700 1 -- 172.21.15.86:0/4224669487 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629d65f2780 msgr2=0x5629d65f2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:50:12.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.893+0000 7f5529400700 1 --2- 172.21.15.86:0/4224669487 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629d65f2780 0x5629d65f2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:12.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.893+0000 7f5529400700 1 -- 172.21.15.86:0/4224669487 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5629d65f3f80 msgr2=0x5629d65f4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:50:12.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.893+0000 7f5529400700 1 --2- 172.21.15.86:0/4224669487 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5629d65f3f80 0x5629d65f4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:12.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.893+0000 7f5529400700 1 -- 172.21.15.86:0/4224669487 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5629d6371b10 con 0x5629d65f30e0 2024-03-23T17:50:12.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.893+0000 7f5529400700 1 --2- 172.21.15.86:0/4224669487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629d65f30e0 0x5629d6601b10 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5518005e90 tx=0x7f5518005ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ffc25fb8653b3c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:12.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.893+0000 7f5517e00700 1 -- 172.21.15.86:0/4224669487 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55180048b0 con 0x5629d65f30e0 2024-03-23T17:50:12.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.893+0000 7f5517e00700 1 -- 172.21.15.86:0/4224669487 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5518026860 con 0x5629d65f30e0 2024-03-23T17:50:12.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.893+0000 7f5517e00700 1 -- 172.21.15.86:0/4224669487 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55180052a0 con 0x5629d65f30e0 2024-03-23T17:50:12.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.894+0000 7f55449a9b80 1 -- 172.21.15.86:0/4224669487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629d65f30e0 msgr2=0x5629d6601b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:12.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.894+0000 7f55449a9b80 1 --2- 172.21.15.86:0/4224669487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629d65f30e0 0x5629d6601b10 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5518005e90 tx=0x7f5518005ce0 comp rx=0 tx=0).stop 2024-03-23T17:50:12.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.894+0000 7f55449a9b80 1 -- 172.21.15.86:0/4224669487 shutdown_connections 2024-03-23T17:50:12.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.894+0000 7f55449a9b80 1 --2- 172.21.15.86:0/4224669487 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5629d65f3f80 0x5629d65f4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:12.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.894+0000 7f55449a9b80 1 --2- 172.21.15.86:0/4224669487 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629d65f2780 0x5629d65f2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:12.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.894+0000 7f55449a9b80 1 --2- 172.21.15.86:0/4224669487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629d65f30e0 0x5629d6601b10 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:12.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.894+0000 7f55449a9b80 1 -- 172.21.15.86:0/4224669487 >> 172.21.15.86:0/4224669487 conn(0x5629d6551650 msgr2=0x5629d65fbaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:12.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.894+0000 7f55449a9b80 1 -- 172.21.15.86:0/4224669487 shutdown_connections 2024-03-23T17:50:12.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.894+0000 7f55449a9b80 1 -- 172.21.15.86:0/4224669487 wait complete. 2024-03-23T17:50:12.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.894+0000 7f55449a9b80 1 -- start start 2024-03-23T17:50:12.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.895+0000 7f55449a9b80 1 Processor -- start 2024-03-23T17:50:12.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.895+0000 7f55449a9b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5629d65f2780 0x5629d66006f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:12.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.895+0000 7f55449a9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629d65f3f80 0x5629d6600c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:12.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.895+0000 7f55449a9b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629d6601170 0x5629d6659cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:12.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.895+0000 7f55449a9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629d665a200 con 0x5629d65f3f80 2024-03-23T17:50:12.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.895+0000 7f55449a9b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629d665a370 con 0x5629d65f2780 2024-03-23T17:50:12.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.895+0000 7f55449a9b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629d665a530 con 0x5629d6601170 2024-03-23T17:50:12.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.895+0000 7f55449a9b80 10 client.0 _ll_register_callbacks cb 0x5629d64e9420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:50:12.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.895+0000 7f5529400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629d65f3f80 0x5629d6600c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:12.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.895+0000 7f5529400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629d65f3f80 0x5629d6600c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38188/0 (socket says 172.21.15.86:38188) 2024-03-23T17:50:12.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.895+0000 7f5529400700 1 -- 172.21.15.86:0/1819601895 learned_addr learned my addr 172.21.15.86:0/1819601895 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:12.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.895+0000 7f5529e00700 1 --2- 172.21.15.86:0/1819601895 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5629d65f2780 0x5629d66006f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:12.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.895+0000 7f552a800700 1 --2- 172.21.15.86:0/1819601895 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629d6601170 0x5629d6659cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:12.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.895+0000 7f5529400700 1 -- 172.21.15.86:0/1819601895 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629d6601170 msgr2=0x5629d6659cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:50:12.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.895+0000 7f5529400700 1 --2- 172.21.15.86:0/1819601895 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629d6601170 0x5629d6659cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:12.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.895+0000 7f5529400700 1 -- 172.21.15.86:0/1819601895 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5629d65f2780 msgr2=0x5629d66006f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:50:12.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.895+0000 7f5529400700 1 --2- 172.21.15.86:0/1819601895 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5629d65f2780 0x5629d66006f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:12.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.896+0000 7f5529400700 1 -- 172.21.15.86:0/1819601895 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f551801d550 con 0x5629d65f3f80 2024-03-23T17:50:12.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.896+0000 7f5529400700 1 --2- 172.21.15.86:0/1819601895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629d65f3f80 0x5629d6600c30 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f5518004a80 tx=0x7f5518005b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e447da50c18c583 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:12.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.896+0000 7f550be00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-23T17:50:12.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.896+0000 7f550be00700 1 -- 172.21.15.86:0/1819601895 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551800f890 con 0x5629d65f3f80 2024-03-23T17:50:12.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.896+0000 7f550be00700 1 -- 172.21.15.86:0/1819601895 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f5518026ac0 con 0x5629d65f3f80 2024-03-23T17:50:12.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.896+0000 7f55449a9b80 1 -- 172.21.15.86:0/1819601895 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5629d65fb610 con 0x5629d65f3f80 2024-03-23T17:50:12.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.896+0000 7f550be00700 1 -- 172.21.15.86:0/1819601895 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f551800fe10 con 0x5629d65f3f80 2024-03-23T17:50:12.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.896+0000 7f55449a9b80 10 client.15126 fetch_fsmap learned FSMap version 27 2024-03-23T17:50:12.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.896+0000 7f55449a9b80 1 -- 172.21.15.86:0/1819601895 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5629d6509e50 con 0x5629d65f3f80 2024-03-23T17:50:12.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.896+0000 7f550be00700 1 -- 172.21.15.86:0/1819601895 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5629d65fb610 con 0x5629d65f3f80 2024-03-23T17:50:12.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.897+0000 7f550be00700 1 -- 172.21.15.86:0/1819601895 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f55180447a0 con 0x5629d65f3f80 2024-03-23T17:50:12.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.897+0000 7f55449a9b80 10 client.15126 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T17:50:12.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.897+0000 7f55449a9b80 10 client.15126 resolve_mds: resolved name 'e' to daemon mds.e 2024-03-23T17:50:12.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.897+0000 7f55449a9b80 20 client.15126 populate_metadata read hostname 'smithi086' 2024-03-23T17:50:12.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.897+0000 7f55449a9b80 1 --2- 172.21.15.86:0/1819601895 >> [v2:172.21.15.92:6834/908900027,v1:172.21.15.92:6835/908900027] conn(0x5629d665ee30 0x5629d665b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:12.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.897+0000 7f55449a9b80 4 client.15126 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-23T17:50:12.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.897+0000 7f55449a9b80 1 -- 172.21.15.86:0/1819601895 --> [v2:172.21.15.92:6834/908900027,v1:172.21.15.92:6835/908900027] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f551800fca0 con 0x5629d665ee30 2024-03-23T17:50:12.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.897+0000 7f5529e00700 1 --2- 172.21.15.86:0/1819601895 >> [v2:172.21.15.92:6834/908900027,v1:172.21.15.92:6835/908900027] conn(0x5629d665ee30 0x5629d665b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:12.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.897+0000 7f5529e00700 1 --2- 172.21.15.86:0/1819601895 >> [v2:172.21.15.92:6834/908900027,v1:172.21.15.92:6835/908900027] conn(0x5629d665ee30 0x5629d665b7e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b5365931b15d2586 server_cookie=cde5526b5d7a3943 in_seq=0 out_seq=0 2024-03-23T17:50:12.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.897+0000 7f550be00700 10 client.15126 ms_handle_connect on v2:172.21.15.92:6834/908900027 2024-03-23T17:50:12.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.898+0000 7f550be00700 1 -- 172.21.15.86:0/1819601895 <== mds.4 v2:172.21.15.92:6834/908900027 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x7f551800fca0 con 0x5629d665ee30 2024-03-23T17:50:12.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.898+0000 7f550be00700 10 client.15126 handle_command_reply: tid=0 2024-03-23T17:50:12.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.898+0000 7f55449a9b80 1 client.15126 shutdown 2024-03-23T17:50:12.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.898+0000 7f55449a9b80 1 -- 172.21.15.86:0/1819601895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629d65f3f80 msgr2=0x5629d6600c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:12.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.898+0000 7f55449a9b80 1 --2- 172.21.15.86:0/1819601895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629d65f3f80 0x5629d6600c30 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f5518004a80 tx=0x7f5518005b40 comp rx=0 tx=0).stop 2024-03-23T17:50:12.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.898+0000 7f55449a9b80 1 -- 172.21.15.86:0/1819601895 shutdown_connections 2024-03-23T17:50:12.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.898+0000 7f55449a9b80 1 --2- 172.21.15.86:0/1819601895 >> [v2:172.21.15.92:6834/908900027,v1:172.21.15.92:6835/908900027] conn(0x5629d665ee30 0x5629d665b7e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:12.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.898+0000 7f55449a9b80 1 --2- 172.21.15.86:0/1819601895 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629d6601170 0x5629d6659cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:12.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.898+0000 7f550be00700 0 client.15126 ms_handle_reset on v2:172.21.15.92:6834/908900027 2024-03-23T17:50:12.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.898+0000 7f55449a9b80 1 --2- 172.21.15.86:0/1819601895 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5629d65f2780 0x5629d66006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:12.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.898+0000 7f55449a9b80 1 --2- 172.21.15.86:0/1819601895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629d65f3f80 0x5629d6600c30 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:12.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.898+0000 7f55449a9b80 1 -- 172.21.15.86:0/1819601895 >> 172.21.15.86:0/1819601895 conn(0x5629d6551650 msgr2=0x5629d65fc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:12.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.898+0000 7f55449a9b80 1 -- 172.21.15.86:0/1819601895 shutdown_connections 2024-03-23T17:50:12.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.898+0000 7f55449a9b80 1 -- 172.21.15.86:0/1819601895 wait complete. 2024-03-23T17:50:12.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.898+0000 7f55449a9b80 20 client.15126 trim_cache size 0 max 16384 2024-03-23T17:50:12.903 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-23T17:50:12.903 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "9d76c9e8-e93c-11ee-95cd-87774f69a715", 2024-03-23T17:50:12.904 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 4, 2024-03-23T17:50:12.904 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24500, 2024-03-23T17:50:12.904 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-03-23T17:50:12.904 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-03-23T17:50:12.904 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-03-23T17:50:12.904 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 70.198622190999998, 2024-03-23T17:50:12.904 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 27, 2024-03-23T17:50:12.904 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 82, 2024-03-23T17:50:12.904 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 80, 2024-03-23T17:50:12.904 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 97.302596433000005 2024-03-23T17:50:12.904 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-23T17:50:12.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.899+0000 7f5530e00700 1 -- 172.21.15.86:0/100248616 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f55100a11b0 msgr2=0x7f55100a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:12.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.899+0000 7f5530e00700 1 --2- 172.21.15.86:0/100248616 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f55100a11b0 0x7f55100a3670 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5518011590 tx=0x7f5518011950 comp rx=0 tx=0).stop 2024-03-23T17:50:12.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.899+0000 7f5530e00700 1 -- 172.21.15.86:0/100248616 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f552c102910 msgr2=0x7f552c0736e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:12.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.899+0000 7f5530e00700 1 --2- 172.21.15.86:0/100248616 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f552c102910 0x7f552c0736e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5524007b60 tx=0x7f5524008e40 comp rx=0 tx=0).stop 2024-03-23T17:50:12.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.899+0000 7f5530e00700 1 -- 172.21.15.86:0/100248616 shutdown_connections 2024-03-23T17:50:12.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.899+0000 7f5530e00700 1 --2- 172.21.15.86:0/100248616 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f552c102910 0x7f552c0736e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:12.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.899+0000 7f5530e00700 1 --2- 172.21.15.86:0/100248616 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f552c100d80 0x7f552c074b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:12.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.899+0000 7f5530e00700 1 --2- 172.21.15.86:0/100248616 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f55100a11b0 0x7f55100a3670 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:12.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.899+0000 7f5530e00700 1 --2- 172.21.15.86:0/100248616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f552c101f70 0x7f552c0731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:12.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.899+0000 7f5530e00700 1 -- 172.21.15.86:0/100248616 >> 172.21.15.86:0/100248616 conn(0x7f552c0fc2f0 msgr2=0x7f552c105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:12.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.899+0000 7f5530e00700 1 -- 172.21.15.86:0/100248616 shutdown_connections 2024-03-23T17:50:12.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:12.899+0000 7f5530e00700 1 -- 172.21.15.86:0/100248616 wait complete. 2024-03-23T17:50:12.915 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:12 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/4221395961' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T17:50:12.915 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:12 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/2433426463' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T17:50:12.915 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:12 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/760963510' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T17:50:12.918 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-03-23T17:50:13.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:12 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/4221395961' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T17:50:13.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:12 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/2433426463' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T17:50:13.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:12 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/760963510' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T17:50:13.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:12 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/4221395961' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T17:50:13.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:12 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/2433426463' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T17:50:13.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:12 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/760963510' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T17:50:13.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.110+0000 7f2985000700 1 -- 172.21.15.86:0/175260110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2980100d80 msgr2=0x7f29801011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:13.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.110+0000 7f2985000700 1 --2- 172.21.15.86:0/175260110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2980100d80 0x7f29801011a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f29700099c0 tx=0x7f2970009cd0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.110+0000 7f2985000700 1 -- 172.21.15.86:0/175260110 shutdown_connections 2024-03-23T17:50:13.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.110+0000 7f2985000700 1 --2- 172.21.15.86:0/175260110 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2980102910 0x7f29801091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.110+0000 7f2985000700 1 --2- 172.21.15.86:0/175260110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2980100d80 0x7f29801011a0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.110+0000 7f2985000700 1 --2- 172.21.15.86:0/175260110 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2980101f70 0x7f29801023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.110+0000 7f2985000700 1 -- 172.21.15.86:0/175260110 >> 172.21.15.86:0/175260110 conn(0x7f29800fc2f0 msgr2=0x7f29800fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:13.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.110+0000 7f2985000700 1 -- 172.21.15.86:0/175260110 shutdown_connections 2024-03-23T17:50:13.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.110+0000 7f2985000700 1 -- 172.21.15.86:0/175260110 wait complete. 2024-03-23T17:50:13.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.111+0000 7f2985000700 1 Processor -- start 2024-03-23T17:50:13.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.111+0000 7f2985000700 1 -- start start 2024-03-23T17:50:13.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.112+0000 7f2985000700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2980100d80 0x7f2980198f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:13.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.112+0000 7f2985000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2980101f70 0x7f29801994d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:13.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.112+0000 7f2985000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2980102910 0x7f29801a04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:13.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.112+0000 7f2985000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29801a0ac0 con 0x7f2980102910 2024-03-23T17:50:13.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.112+0000 7f2985000700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29801a0c00 con 0x7f2980101f70 2024-03-23T17:50:13.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.112+0000 7f2985000700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29801a0d90 con 0x7f2980100d80 2024-03-23T17:50:13.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.112+0000 7f297e800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2980102910 0x7f29801a04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:13.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.112+0000 7f297e800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2980102910 0x7f29801a04f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38196/0 (socket says 172.21.15.86:38196) 2024-03-23T17:50:13.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.112+0000 7f297e800700 1 -- 172.21.15.86:0/180472519 learned_addr learned my addr 172.21.15.86:0/180472519 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:13.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.112+0000 7f297de00700 1 --2- 172.21.15.86:0/180472519 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2980100d80 0x7f2980198f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:13.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.112+0000 7f297d400700 1 --2- 172.21.15.86:0/180472519 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2980101f70 0x7f29801994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:13.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.112+0000 7f297e800700 1 -- 172.21.15.86:0/180472519 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2980100d80 msgr2=0x7f2980198f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:13.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.113+0000 7f297e800700 1 --2- 172.21.15.86:0/180472519 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2980100d80 0x7f2980198f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.113+0000 7f297e800700 1 -- 172.21.15.86:0/180472519 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2980101f70 msgr2=0x7f29801994d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:13.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.113+0000 7f297e800700 1 --2- 172.21.15.86:0/180472519 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2980101f70 0x7f29801994d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.113+0000 7f297e800700 1 -- 172.21.15.86:0/180472519 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29700095f0 con 0x7f2980102910 2024-03-23T17:50:13.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.113+0000 7f297d400700 1 --2- 172.21.15.86:0/180472519 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2980101f70 0x7f29801994d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:50:13.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.113+0000 7f297e800700 1 --2- 172.21.15.86:0/180472519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2980102910 0x7f29801a04f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f297800ef30 tx=0x7f297800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:13.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.113+0000 7f2976a00700 1 -- 172.21.15.86:0/180472519 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2978018070 con 0x7f2980102910 2024-03-23T17:50:13.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.113+0000 7f2976a00700 1 -- 172.21.15.86:0/180472519 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f297800f040 con 0x7f2980102910 2024-03-23T17:50:13.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.113+0000 7f2976a00700 1 -- 172.21.15.86:0/180472519 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f297800cd00 con 0x7f2980102910 2024-03-23T17:50:13.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.113+0000 7f2985000700 1 -- 172.21.15.86:0/180472519 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29801a1010 con 0x7f2980102910 2024-03-23T17:50:13.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.113+0000 7f2985000700 1 -- 172.21.15.86:0/180472519 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29801a14d0 con 0x7f2980102910 2024-03-23T17:50:13.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.116+0000 7f2976a00700 1 -- 172.21.15.86:0/180472519 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f2978003a00 con 0x7f2980102910 2024-03-23T17:50:13.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.117+0000 7f2998c0fb80 1 Processor -- start 2024-03-23T17:50:13.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.117+0000 7f2998c0fb80 1 -- start start 2024-03-23T17:50:13.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.118+0000 7f2976a00700 1 --2- 172.21.15.86:0/180472519 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f29640a11b0 0x7f29640a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:13.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.118+0000 7f2976a00700 1 -- 172.21.15.86:0/180472519 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f29780cc5c0 con 0x7f2980102910 2024-03-23T17:50:13.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.118+0000 7f297d400700 1 --2- 172.21.15.86:0/180472519 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f29640a11b0 0x7f29640a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:13.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.118+0000 7f2998c0fb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565e89a7780 0x5565e89a7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:13.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.118+0000 7f2998c0fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565e89a80e0 0x5565e89a89a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:13.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.118+0000 7f2998c0fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565e89a8f80 0x5565e89a9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:13.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.118+0000 7f2998c0fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565e88ff940 con 0x5565e89a8f80 2024-03-23T17:50:13.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.118+0000 7f2998c0fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565e88fd300 con 0x5565e89a80e0 2024-03-23T17:50:13.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.118+0000 7f2998c0fb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565e88f8bf0 con 0x5565e89a7780 2024-03-23T17:50:13.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.118+0000 7f297de00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565e89a7780 0x5565e89a7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:13.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.118+0000 7f297de00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565e89a7780 0x5565e89a7ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:52198/0 (socket says 172.21.15.86:52198) 2024-03-23T17:50:13.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.118+0000 7f297de00700 1 -- 172.21.15.86:0/1656615644 learned_addr learned my addr 172.21.15.86:0/1656615644 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:13.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.118+0000 7f297d400700 1 --2- 172.21.15.86:0/1656615644 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565e89a80e0 0x5565e89a89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:13.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.118+0000 7f297e800700 1 --2- 172.21.15.86:0/1656615644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565e89a8f80 0x5565e89a9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:13.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.118+0000 7f297d400700 1 --2- 172.21.15.86:0/180472519 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f29640a11b0 0x7f29640a3670 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f296c011550 tx=0x7f296c011860 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:13.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.119+0000 7f297d400700 1 -- 172.21.15.86:0/1656615644 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565e89a7780 msgr2=0x5565e89a7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:50:13.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.119+0000 7f297d400700 1 --2- 172.21.15.86:0/1656615644 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565e89a7780 0x5565e89a7ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.119+0000 7f297d400700 1 -- 172.21.15.86:0/1656615644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565e89a8f80 msgr2=0x5565e89a9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:50:13.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.119+0000 7f297d400700 1 --2- 172.21.15.86:0/1656615644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565e89a8f80 0x5565e89a9400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.119+0000 7f297d400700 1 -- 172.21.15.86:0/1656615644 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565e877a360 con 0x5565e89a80e0 2024-03-23T17:50:13.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.119+0000 7f297d400700 1 --2- 172.21.15.86:0/1656615644 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565e89a80e0 0x5565e89a89a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f296c011c60 tx=0x7f296c01a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7bb091371b5b27aa server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:13.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.119+0000 7f296b400700 1 -- 172.21.15.86:0/1656615644 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f296c00f480 con 0x5565e89a80e0 2024-03-23T17:50:13.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.119+0000 7f296b400700 1 -- 172.21.15.86:0/1656615644 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f296c00faa0 con 0x5565e89a80e0 2024-03-23T17:50:13.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.119+0000 7f296b400700 1 -- 172.21.15.86:0/1656615644 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f296c00eb60 con 0x5565e89a80e0 2024-03-23T17:50:13.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.120+0000 7f2998c0fb80 1 -- 172.21.15.86:0/1656615644 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565e89a80e0 msgr2=0x5565e89a89a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:13.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.120+0000 7f2998c0fb80 1 --2- 172.21.15.86:0/1656615644 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565e89a80e0 0x5565e89a89a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f296c011c60 tx=0x7f296c01a000 comp rx=0 tx=0).stop 2024-03-23T17:50:13.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.120+0000 7f2998c0fb80 1 -- 172.21.15.86:0/1656615644 shutdown_connections 2024-03-23T17:50:13.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.120+0000 7f2998c0fb80 1 --2- 172.21.15.86:0/1656615644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565e89a8f80 0x5565e89a9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.120+0000 7f2998c0fb80 1 --2- 172.21.15.86:0/1656615644 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565e89a7780 0x5565e89a7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.120+0000 7f2998c0fb80 1 --2- 172.21.15.86:0/1656615644 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565e89a80e0 0x5565e89a89a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.120+0000 7f2998c0fb80 1 -- 172.21.15.86:0/1656615644 >> 172.21.15.86:0/1656615644 conn(0x5565e8906650 msgr2=0x5565e89a2950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:13.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.120+0000 7f2998c0fb80 1 -- 172.21.15.86:0/1656615644 shutdown_connections 2024-03-23T17:50:13.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.120+0000 7f2998c0fb80 1 -- 172.21.15.86:0/1656615644 wait complete. 2024-03-23T17:50:13.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.120+0000 7f2998c0fb80 1 -- start start 2024-03-23T17:50:13.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.121+0000 7f2998c0fb80 1 Processor -- start 2024-03-23T17:50:13.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.121+0000 7f2998c0fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565e89a7780 0x5565e89b1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:13.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.121+0000 7f2998c0fb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565e89a80e0 0x5565e89b1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:13.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.121+0000 7f2998c0fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565e89a8f80 0x5565e89b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:13.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.121+0000 7f2998c0fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565e8894d30 con 0x5565e89a7780 2024-03-23T17:50:13.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.121+0000 7f2998c0fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565e88949d0 con 0x5565e89a8f80 2024-03-23T17:50:13.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.121+0000 7f2998c0fb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565e877c650 con 0x5565e89a80e0 2024-03-23T17:50:13.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.121+0000 7f2998c0fb80 10 client.0 _ll_register_callbacks cb 0x5565e889e420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:50:13.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.121+0000 7f297de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565e89a7780 0x5565e89b1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:13.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.121+0000 7f297de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565e89a7780 0x5565e89b1660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38204/0 (socket says 172.21.15.86:38204) 2024-03-23T17:50:13.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.121+0000 7f297de00700 1 -- 172.21.15.86:0/1675014133 learned_addr learned my addr 172.21.15.86:0/1675014133 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:13.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.121+0000 7f297e800700 1 --2- 172.21.15.86:0/1675014133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565e89a8f80 0x5565e89b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:13.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.122+0000 7f297d400700 1 --2- 172.21.15.86:0/1675014133 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565e89a80e0 0x5565e89b1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:13.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.122+0000 7f297de00700 1 -- 172.21.15.86:0/1675014133 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565e89a80e0 msgr2=0x5565e89b1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:50:13.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.122+0000 7f297de00700 1 --2- 172.21.15.86:0/1675014133 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565e89a80e0 0x5565e89b1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.122+0000 7f297de00700 1 -- 172.21.15.86:0/1675014133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565e89a8f80 msgr2=0x5565e89b8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:50:13.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.122+0000 7f297de00700 1 --2- 172.21.15.86:0/1675014133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565e89a8f80 0x5565e89b8b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.122+0000 7f297de00700 1 -- 172.21.15.86:0/1675014133 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f296c01d420 con 0x5565e89a7780 2024-03-23T17:50:13.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.122+0000 7f297de00700 1 --2- 172.21.15.86:0/1675014133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565e89a7780 0x5565e89b1660 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f29700058e0 tx=0x7f2970005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c750ba716cc1563f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:13.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.122+0000 7f295fe00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-23T17:50:13.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.122+0000 7f295fe00700 1 -- 172.21.15.86:0/1675014133 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2970003d20 con 0x5565e89a7780 2024-03-23T17:50:13.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.122+0000 7f295fe00700 1 -- 172.21.15.86:0/1675014133 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f2970005040 con 0x5565e89a7780 2024-03-23T17:50:13.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.122+0000 7f2998c0fb80 1 -- 172.21.15.86:0/1675014133 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5565e88b9330 con 0x5565e89a7780 2024-03-23T17:50:13.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.123+0000 7f295fe00700 1 -- 172.21.15.86:0/1675014133 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2970003e80 con 0x5565e89a7780 2024-03-23T17:50:13.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.123+0000 7f2998c0fb80 10 client.15138 fetch_fsmap learned FSMap version 27 2024-03-23T17:50:13.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.123+0000 7f2998c0fb80 1 -- 172.21.15.86:0/1675014133 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5565e8726b10 con 0x5565e89a7780 2024-03-23T17:50:13.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.123+0000 7f295fe00700 1 -- 172.21.15.86:0/1675014133 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5565e88b9330 con 0x5565e89a7780 2024-03-23T17:50:13.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.123+0000 7f295fe00700 1 -- 172.21.15.86:0/1675014133 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2970024b50 con 0x5565e89a7780 2024-03-23T17:50:13.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.123+0000 7f2998c0fb80 10 client.15138 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T17:50:13.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.123+0000 7f2998c0fb80 10 client.15138 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-23T17:50:13.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.123+0000 7f2998c0fb80 20 client.15138 populate_metadata read hostname 'smithi086' 2024-03-23T17:50:13.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.123+0000 7f2998c0fb80 1 --2- 172.21.15.86:0/1675014133 >> [v2:172.21.15.86:6836/2933881328,v1:172.21.15.86:6837/2933881328] conn(0x5565e89bc7f0 0x5565e89c1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:13.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.123+0000 7f2998c0fb80 4 client.15138 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:50:13.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.123+0000 7f2998c0fb80 1 -- 172.21.15.86:0/1675014133 --> [v2:172.21.15.86:6836/2933881328,v1:172.21.15.86:6837/2933881328] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5565e88f16b0 con 0x5565e89bc7f0 2024-03-23T17:50:13.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.123+0000 7f297d400700 1 --2- 172.21.15.86:0/1675014133 >> [v2:172.21.15.86:6836/2933881328,v1:172.21.15.86:6837/2933881328] conn(0x5565e89bc7f0 0x5565e89c1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:13.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.124+0000 7f295fe00700 10 client.15138 ms_handle_connect on v2:172.21.15.86:6836/2933881328 2024-03-23T17:50:13.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.124+0000 7f297d400700 1 --2- 172.21.15.86:0/1675014133 >> [v2:172.21.15.86:6836/2933881328,v1:172.21.15.86:6837/2933881328] conn(0x5565e89bc7f0 0x5565e89c1af0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=272516aa4b523697 server_cookie=b36b27d5325e8972 in_seq=0 out_seq=0 2024-03-23T17:50:13.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.125+0000 7f295fe00700 1 -- 172.21.15.86:0/1675014133 <== mds.3 v2:172.21.15.86:6836/2933881328 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5565e88f16b0 con 0x5565e89bc7f0 2024-03-23T17:50:13.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.125+0000 7f295fe00700 10 client.15138 handle_command_reply: tid=0 2024-03-23T17:50:13.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.125+0000 7f2998c0fb80 1 client.15138 shutdown 2024-03-23T17:50:13.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.125+0000 7f2998c0fb80 1 -- 172.21.15.86:0/1675014133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565e89a7780 msgr2=0x5565e89b1660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:13.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.125+0000 7f2998c0fb80 1 --2- 172.21.15.86:0/1675014133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565e89a7780 0x5565e89b1660 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f29700058e0 tx=0x7f2970005be0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.125+0000 7f2998c0fb80 1 -- 172.21.15.86:0/1675014133 shutdown_connections 2024-03-23T17:50:13.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.125+0000 7f2998c0fb80 1 --2- 172.21.15.86:0/1675014133 >> [v2:172.21.15.86:6836/2933881328,v1:172.21.15.86:6837/2933881328] conn(0x5565e89bc7f0 0x5565e89c1af0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.125+0000 7f2998c0fb80 1 --2- 172.21.15.86:0/1675014133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565e89a8f80 0x5565e89b8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.125+0000 7f2998c0fb80 1 --2- 172.21.15.86:0/1675014133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565e89a7780 0x5565e89b1660 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.125+0000 7f2998c0fb80 1 --2- 172.21.15.86:0/1675014133 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565e89a80e0 0x5565e89b1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.125+0000 7f295fe00700 0 client.15138 ms_handle_reset on v2:172.21.15.86:6836/2933881328 2024-03-23T17:50:13.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.125+0000 7f2998c0fb80 1 -- 172.21.15.86:0/1675014133 >> 172.21.15.86:0/1675014133 conn(0x5565e8906650 msgr2=0x5565e89a6120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:13.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.125+0000 7f2998c0fb80 1 -- 172.21.15.86:0/1675014133 shutdown_connections 2024-03-23T17:50:13.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.125+0000 7f2998c0fb80 1 -- 172.21.15.86:0/1675014133 wait complete. 2024-03-23T17:50:13.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.125+0000 7f2998c0fb80 20 client.15138 trim_cache size 0 max 16384 2024-03-23T17:50:13.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.143+0000 7f2998c0fb80 1 Processor -- start 2024-03-23T17:50:13.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.143+0000 7f2998c0fb80 1 -- start start 2024-03-23T17:50:13.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.143+0000 7f2998c0fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565e89a7780 0x5565e89a7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:13.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.143+0000 7f2998c0fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565e89a80e0 0x5565e89b6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:13.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.143+0000 7f2998c0fb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565e89a8f80 0x5565e89a9360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:13.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.143+0000 7f2998c0fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565e88b9330 con 0x5565e89a7780 2024-03-23T17:50:13.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.143+0000 7f2998c0fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565e88b6ad0 con 0x5565e89a80e0 2024-03-23T17:50:13.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.143+0000 7f2998c0fb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565e88f1990 con 0x5565e89a8f80 2024-03-23T17:50:13.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.143+0000 7f297de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565e89a7780 0x5565e89a7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:13.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.143+0000 7f297de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565e89a7780 0x5565e89a7b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38206/0 (socket says 172.21.15.86:38206) 2024-03-23T17:50:13.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.143+0000 7f297de00700 1 -- 172.21.15.86:0/719494422 learned_addr learned my addr 172.21.15.86:0/719494422 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:13.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.143+0000 7f297d400700 1 --2- 172.21.15.86:0/719494422 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565e89a80e0 0x5565e89b6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:13.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.144+0000 7f297e800700 1 --2- 172.21.15.86:0/719494422 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565e89a8f80 0x5565e89a9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:13.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.144+0000 7f297de00700 1 -- 172.21.15.86:0/719494422 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565e89a8f80 msgr2=0x5565e89a9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:50:13.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.144+0000 7f297de00700 1 --2- 172.21.15.86:0/719494422 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565e89a8f80 0x5565e89a9360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.144+0000 7f297de00700 1 -- 172.21.15.86:0/719494422 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565e89a80e0 msgr2=0x5565e89b6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:50:13.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.144+0000 7f297de00700 1 --2- 172.21.15.86:0/719494422 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565e89a80e0 0x5565e89b6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.144+0000 7f297de00700 1 -- 172.21.15.86:0/719494422 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565e88bee50 con 0x5565e89a7780 2024-03-23T17:50:13.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.144+0000 7f297d400700 1 --2- 172.21.15.86:0/719494422 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565e89a80e0 0x5565e89b6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:50:13.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.144+0000 7f297de00700 1 --2- 172.21.15.86:0/719494422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565e89a7780 0x5565e89a7b60 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f2970024ee0 tx=0x7f2970001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3645ead277605351 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:13.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.144+0000 7f296be00700 1 -- 172.21.15.86:0/719494422 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2970003d20 con 0x5565e89a7780 2024-03-23T17:50:13.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.144+0000 7f296be00700 1 -- 172.21.15.86:0/719494422 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f297001e040 con 0x5565e89a7780 2024-03-23T17:50:13.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.144+0000 7f296be00700 1 -- 172.21.15.86:0/719494422 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2970016db0 con 0x5565e89a7780 2024-03-23T17:50:13.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.145+0000 7f2998c0fb80 1 -- 172.21.15.86:0/719494422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565e89a7780 msgr2=0x5565e89a7b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:13.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.145+0000 7f2998c0fb80 1 --2- 172.21.15.86:0/719494422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565e89a7780 0x5565e89a7b60 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f2970024ee0 tx=0x7f2970001e50 comp rx=0 tx=0).stop 2024-03-23T17:50:13.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.145+0000 7f2998c0fb80 1 -- 172.21.15.86:0/719494422 shutdown_connections 2024-03-23T17:50:13.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.145+0000 7f2998c0fb80 1 --2- 172.21.15.86:0/719494422 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565e89a8f80 0x5565e89a9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.145+0000 7f2998c0fb80 1 --2- 172.21.15.86:0/719494422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565e89a7780 0x5565e89a7b60 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.145+0000 7f2998c0fb80 1 --2- 172.21.15.86:0/719494422 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565e89a80e0 0x5565e89b6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.145+0000 7f2998c0fb80 1 -- 172.21.15.86:0/719494422 >> 172.21.15.86:0/719494422 conn(0x5565e8906650 msgr2=0x5565e89af530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:13.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.145+0000 7f2998c0fb80 1 -- 172.21.15.86:0/719494422 shutdown_connections 2024-03-23T17:50:13.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.145+0000 7f2998c0fb80 1 -- 172.21.15.86:0/719494422 wait complete. 2024-03-23T17:50:13.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.145+0000 7f2998c0fb80 1 -- start start 2024-03-23T17:50:13.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.145+0000 7f2998c0fb80 1 Processor -- start 2024-03-23T17:50:13.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.146+0000 7f2998c0fb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565e89a7780 0x5565e89b6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:13.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.146+0000 7f2998c0fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565e89a80e0 0x5565e89b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:13.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.146+0000 7f2998c0fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565e89a8f80 0x5565e89b5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:13.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.146+0000 7f2998c0fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565e89b5960 con 0x5565e89a80e0 2024-03-23T17:50:13.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.146+0000 7f2998c0fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565e89b5aa0 con 0x5565e89a8f80 2024-03-23T17:50:13.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.146+0000 7f2998c0fb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565e89b5c60 con 0x5565e89a7780 2024-03-23T17:50:13.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.146+0000 7f2998c0fb80 10 client.0 _ll_register_callbacks cb 0x5565e889e420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:50:13.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.146+0000 7f297d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565e89a80e0 0x5565e89b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:13.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.146+0000 7f297d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565e89a80e0 0x5565e89b4ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38214/0 (socket says 172.21.15.86:38214) 2024-03-23T17:50:13.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.146+0000 7f297d400700 1 -- 172.21.15.86:0/517931526 learned_addr learned my addr 172.21.15.86:0/517931526 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:13.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.146+0000 7f297de00700 1 --2- 172.21.15.86:0/517931526 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565e89a7780 0x5565e89b6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:13.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.146+0000 7f297e800700 1 --2- 172.21.15.86:0/517931526 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565e89a8f80 0x5565e89b5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:13.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.147+0000 7f297e800700 1 -- 172.21.15.86:0/517931526 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565e89a7780 msgr2=0x5565e89b6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:50:13.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.147+0000 7f297e800700 1 --2- 172.21.15.86:0/517931526 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565e89a7780 0x5565e89b6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.147+0000 7f297e800700 1 -- 172.21.15.86:0/517931526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565e89a80e0 msgr2=0x5565e89b4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:50:13.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.147+0000 7f297e800700 1 --2- 172.21.15.86:0/517931526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565e89a80e0 0x5565e89b4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.147+0000 7f297e800700 1 -- 172.21.15.86:0/517931526 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5565e877a360 con 0x5565e89a8f80 2024-03-23T17:50:13.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.147+0000 7f297de00700 1 --2- 172.21.15.86:0/517931526 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565e89a7780 0x5565e89b6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:50:13.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.147+0000 7f297d400700 1 --2- 172.21.15.86:0/517931526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565e89a80e0 0x5565e89b4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:50:13.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.147+0000 7f297e800700 1 --2- 172.21.15.86:0/517931526 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565e89a8f80 0x5565e89b5420 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f29780c9f40 tx=0x7f2978004b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be96fb76da556616 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:13.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.147+0000 7f295fe00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T17:50:13.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.147+0000 7f295fe00700 1 -- 172.21.15.86:0/517931526 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f297800ce60 con 0x5565e89a8f80 2024-03-23T17:50:13.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.147+0000 7f295fe00700 1 -- 172.21.15.86:0/517931526 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f2978099910 con 0x5565e89a8f80 2024-03-23T17:50:13.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.147+0000 7f2998c0fb80 1 -- 172.21.15.86:0/517931526 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f296c00faa0 con 0x5565e89a8f80 2024-03-23T17:50:13.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.147+0000 7f295fe00700 1 -- 172.21.15.86:0/517931526 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f297809d740 con 0x5565e89a8f80 2024-03-23T17:50:13.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.147+0000 7f2998c0fb80 10 client.25184 fetch_fsmap learned FSMap version 27 2024-03-23T17:50:13.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.147+0000 7f2998c0fb80 1 -- 172.21.15.86:0/517931526 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5565e8726b10 con 0x5565e89a8f80 2024-03-23T17:50:13.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.147+0000 7f295fe00700 1 -- 172.21.15.86:0/517931526 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29780b6080 con 0x5565e89a8f80 2024-03-23T17:50:13.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.148+0000 7f295fe00700 1 -- 172.21.15.86:0/517931526 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2978098e00 con 0x5565e89a8f80 2024-03-23T17:50:13.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.148+0000 7f2998c0fb80 10 client.25184 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T17:50:13.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.148+0000 7f2998c0fb80 10 client.25184 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-23T17:50:13.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.148+0000 7f2998c0fb80 20 client.25184 populate_metadata read hostname 'smithi086' 2024-03-23T17:50:13.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.148+0000 7f2998c0fb80 1 --2- 172.21.15.86:0/517931526 >> [v2:172.21.15.86:6836/2933881328,v1:172.21.15.86:6837/2933881328] conn(0x5565e8a13a40 0x5565e8a10390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:13.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.148+0000 7f2998c0fb80 4 client.25184 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-23T17:50:13.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.148+0000 7f2998c0fb80 1 -- 172.21.15.86:0/517931526 --> [v2:172.21.15.86:6836/2933881328,v1:172.21.15.86:6837/2933881328] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5565e8747ca0 con 0x5565e8a13a40 2024-03-23T17:50:13.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.148+0000 7f297de00700 1 --2- 172.21.15.86:0/517931526 >> [v2:172.21.15.86:6836/2933881328,v1:172.21.15.86:6837/2933881328] conn(0x5565e8a13a40 0x5565e8a10390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:13.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.148+0000 7f295fe00700 10 client.25184 ms_handle_connect on v2:172.21.15.86:6836/2933881328 2024-03-23T17:50:13.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.148+0000 7f297de00700 1 --2- 172.21.15.86:0/517931526 >> [v2:172.21.15.86:6836/2933881328,v1:172.21.15.86:6837/2933881328] conn(0x5565e8a13a40 0x5565e8a10390 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9d6b7b51812ec713 server_cookie=e8a73cb5d46c2dbe in_seq=0 out_seq=0 2024-03-23T17:50:13.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.149+0000 7f295fe00700 1 -- 172.21.15.86:0/517931526 <== mds.3 v2:172.21.15.86:6836/2933881328 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x5565e8747ca0 con 0x5565e8a13a40 2024-03-23T17:50:13.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.149+0000 7f295fe00700 10 client.25184 handle_command_reply: tid=0 2024-03-23T17:50:13.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.149+0000 7f2998c0fb80 1 client.25184 shutdown 2024-03-23T17:50:13.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.149+0000 7f2998c0fb80 1 -- 172.21.15.86:0/517931526 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565e89a8f80 msgr2=0x5565e89b5420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:13.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.149+0000 7f2998c0fb80 1 --2- 172.21.15.86:0/517931526 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565e89a8f80 0x5565e89b5420 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f29780c9f40 tx=0x7f2978004b50 comp rx=0 tx=0).stop 2024-03-23T17:50:13.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.149+0000 7f2998c0fb80 1 -- 172.21.15.86:0/517931526 shutdown_connections 2024-03-23T17:50:13.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.149+0000 7f2998c0fb80 1 --2- 172.21.15.86:0/517931526 >> [v2:172.21.15.86:6836/2933881328,v1:172.21.15.86:6837/2933881328] conn(0x5565e8a13a40 0x5565e8a10390 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.149+0000 7f2998c0fb80 1 --2- 172.21.15.86:0/517931526 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565e89a8f80 0x5565e89b5420 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.149+0000 7f2998c0fb80 1 --2- 172.21.15.86:0/517931526 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565e89a7780 0x5565e89b6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.149+0000 7f2998c0fb80 1 --2- 172.21.15.86:0/517931526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565e89a80e0 0x5565e89b4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.149+0000 7f295fe00700 0 client.25184 ms_handle_reset on v2:172.21.15.86:6836/2933881328 2024-03-23T17:50:13.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.149+0000 7f2998c0fb80 1 -- 172.21.15.86:0/517931526 >> 172.21.15.86:0/517931526 conn(0x5565e8906650 msgr2=0x5565e89c5b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:13.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.149+0000 7f2998c0fb80 1 -- 172.21.15.86:0/517931526 shutdown_connections 2024-03-23T17:50:13.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.149+0000 7f2998c0fb80 1 -- 172.21.15.86:0/517931526 wait complete. 2024-03-23T17:50:13.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.149+0000 7f2998c0fb80 20 client.25184 trim_cache size 0 max 16384 2024-03-23T17:50:13.155 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-23T17:50:13.155 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "9d76c9e8-e93c-11ee-95cd-87774f69a715", 2024-03-23T17:50:13.155 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 3, 2024-03-23T17:50:13.155 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24505, 2024-03-23T17:50:13.155 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-03-23T17:50:13.156 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-03-23T17:50:13.156 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-03-23T17:50:13.156 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 71.477683251000002, 2024-03-23T17:50:13.156 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 27, 2024-03-23T17:50:13.156 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 82, 2024-03-23T17:50:13.156 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 80, 2024-03-23T17:50:13.156 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 99.331363512999999 2024-03-23T17:50:13.156 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-23T17:50:13.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.150+0000 7f2985000700 1 -- 172.21.15.86:0/180472519 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f29640a11b0 msgr2=0x7f29640a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:13.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.150+0000 7f2985000700 1 --2- 172.21.15.86:0/180472519 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f29640a11b0 0x7f29640a3670 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f296c011550 tx=0x7f296c011860 comp rx=0 tx=0).stop 2024-03-23T17:50:13.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.150+0000 7f2985000700 1 -- 172.21.15.86:0/180472519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2980102910 msgr2=0x7f29801a04f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:13.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.150+0000 7f2985000700 1 --2- 172.21.15.86:0/180472519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2980102910 0x7f29801a04f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f297800ef30 tx=0x7f297800c3a0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.150+0000 7f2985000700 1 -- 172.21.15.86:0/180472519 shutdown_connections 2024-03-23T17:50:13.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.150+0000 7f2985000700 1 --2- 172.21.15.86:0/180472519 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f29640a11b0 0x7f29640a3670 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.150+0000 7f2985000700 1 --2- 172.21.15.86:0/180472519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2980102910 0x7f29801a04f0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.150+0000 7f2985000700 1 --2- 172.21.15.86:0/180472519 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2980100d80 0x7f2980198f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.150+0000 7f2985000700 1 --2- 172.21.15.86:0/180472519 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2980101f70 0x7f29801994d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.150+0000 7f2985000700 1 -- 172.21.15.86:0/180472519 >> 172.21.15.86:0/180472519 conn(0x7f29800fc2f0 msgr2=0x7f2980072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:13.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.150+0000 7f2985000700 1 -- 172.21.15.86:0/180472519 shutdown_connections 2024-03-23T17:50:13.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.150+0000 7f2985000700 1 -- 172.21.15.86:0/180472519 wait complete. 2024-03-23T17:50:13.168 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-03-23T17:50:13.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.360+0000 7f1a04a00700 1 -- 172.21.15.86:0/336630480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a00101e80 msgr2=0x7f1a001022e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:13.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.360+0000 7f1a04a00700 1 --2- 172.21.15.86:0/336630480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a00101e80 0x7f1a001022e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f19ec0099c0 tx=0x7f19ec009cd0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.360+0000 7f1a04a00700 1 -- 172.21.15.86:0/336630480 shutdown_connections 2024-03-23T17:50:13.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.360+0000 7f1a04a00700 1 --2- 172.21.15.86:0/336630480 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a00102820 0x7f1a001090d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.360+0000 7f1a04a00700 1 --2- 172.21.15.86:0/336630480 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1a00100c90 0x7f1a001010b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.360+0000 7f1a04a00700 1 --2- 172.21.15.86:0/336630480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a00101e80 0x7f1a001022e0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.360+0000 7f1a04a00700 1 -- 172.21.15.86:0/336630480 >> 172.21.15.86:0/336630480 conn(0x7f1a000fc220 msgr2=0x7f1a000fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:13.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.360+0000 7f1a04a00700 1 -- 172.21.15.86:0/336630480 shutdown_connections 2024-03-23T17:50:13.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.360+0000 7f1a04a00700 1 -- 172.21.15.86:0/336630480 wait complete. 2024-03-23T17:50:13.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.361+0000 7f1a04a00700 1 Processor -- start 2024-03-23T17:50:13.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.361+0000 7f1a04a00700 1 -- start start 2024-03-23T17:50:13.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.361+0000 7f1a04a00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1a00100c90 0x7f1a00196c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:13.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.362+0000 7f1a04a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a00101e80 0x7f1a00197170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:13.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.362+0000 7f1a04a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a00102820 0x7f1a0019e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:13.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.362+0000 7f1a04a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0019e760 con 0x7f1a00102820 2024-03-23T17:50:13.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.362+0000 7f1a04a00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0019e8a0 con 0x7f1a00101e80 2024-03-23T17:50:13.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.362+0000 7f1a04a00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0019ea30 con 0x7f1a00100c90 2024-03-23T17:50:13.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.362+0000 7f19fe800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a00102820 0x7f1a0019e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:13.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.362+0000 7f19fe800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a00102820 0x7f1a0019e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38232/0 (socket says 172.21.15.86:38232) 2024-03-23T17:50:13.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.362+0000 7f19fd400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a00101e80 0x7f1a00197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:13.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.362+0000 7f19fe800700 1 -- 172.21.15.86:0/3222558616 learned_addr learned my addr 172.21.15.86:0/3222558616 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:13.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.362+0000 7f19fde00700 1 --2- 172.21.15.86:0/3222558616 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1a00100c90 0x7f1a00196c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:13.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.362+0000 7f19fe800700 1 -- 172.21.15.86:0/3222558616 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1a00100c90 msgr2=0x7f1a00196c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:13.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.362+0000 7f19fe800700 1 --2- 172.21.15.86:0/3222558616 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1a00100c90 0x7f1a00196c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.362+0000 7f19fe800700 1 -- 172.21.15.86:0/3222558616 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a00101e80 msgr2=0x7f1a00197170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:13.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.362+0000 7f19fe800700 1 --2- 172.21.15.86:0/3222558616 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a00101e80 0x7f1a00197170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.362+0000 7f19fe800700 1 -- 172.21.15.86:0/3222558616 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19ec0095f0 con 0x7f1a00102820 2024-03-23T17:50:13.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.362+0000 7f19fd400700 1 --2- 172.21.15.86:0/3222558616 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a00101e80 0x7f1a00197170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:50:13.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.362+0000 7f19fe800700 1 --2- 172.21.15.86:0/3222558616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a00102820 0x7f1a0019e190 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f19f800ef30 tx=0x7f19f800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:13.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.363+0000 7f19f6a00700 1 -- 172.21.15.86:0/3222558616 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f8018070 con 0x7f1a00102820 2024-03-23T17:50:13.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.363+0000 7f19f6a00700 1 -- 172.21.15.86:0/3222558616 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19f800f040 con 0x7f1a00102820 2024-03-23T17:50:13.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.363+0000 7f1a04a00700 1 -- 172.21.15.86:0/3222558616 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a0019ecb0 con 0x7f1a00102820 2024-03-23T17:50:13.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.363+0000 7f19f6a00700 1 -- 172.21.15.86:0/3222558616 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f800cd00 con 0x7f1a00102820 2024-03-23T17:50:13.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.363+0000 7f1a04a00700 1 -- 172.21.15.86:0/3222558616 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a0019f1c0 con 0x7f1a00102820 2024-03-23T17:50:13.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.366+0000 7f19f6a00700 1 -- 172.21.15.86:0/3222558616 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f19f8003a00 con 0x7f1a00102820 2024-03-23T17:50:13.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.367+0000 7f1a184a0b80 1 Processor -- start 2024-03-23T17:50:13.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.367+0000 7f19f6a00700 1 --2- 172.21.15.86:0/3222558616 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f19e40a1220 0x7f19e40a36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:13.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.367+0000 7f1a184a0b80 1 -- start start 2024-03-23T17:50:13.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.367+0000 7f19f6a00700 1 -- 172.21.15.86:0/3222558616 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f19f80cc4d0 con 0x7f1a00102820 2024-03-23T17:50:13.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.367+0000 7f19fd400700 1 --2- 172.21.15.86:0/3222558616 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f19e40a1220 0x7f19e40a36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:13.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.367+0000 7f1a184a0b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e60586a780 0x55e60586aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:13.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.367+0000 7f1a184a0b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e60586b0e0 0x55e60586b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:13.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.367+0000 7f1a184a0b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e60586bf80 0x55e60586c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:13.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.367+0000 7f1a184a0b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6057c2940 con 0x55e60586b0e0 2024-03-23T17:50:13.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.367+0000 7f1a184a0b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6057c0300 con 0x55e60586bf80 2024-03-23T17:50:13.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.367+0000 7f1a184a0b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6057bbbf0 con 0x55e60586a780 2024-03-23T17:50:13.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.368+0000 7f19fde00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e60586a780 0x55e60586aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:13.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.368+0000 7f19fd400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e60586b0e0 0x55e60586b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:13.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.368+0000 7f19fde00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e60586a780 0x55e60586aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:52262/0 (socket says 172.21.15.86:52262) 2024-03-23T17:50:13.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.368+0000 7f19fde00700 1 -- 172.21.15.86:0/2025940250 learned_addr learned my addr 172.21.15.86:0/2025940250 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:13.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.368+0000 7f19fe800700 1 --2- 172.21.15.86:0/2025940250 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e60586bf80 0x55e60586c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:13.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.368+0000 7f19fd400700 1 --2- 172.21.15.86:0/3222558616 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f19e40a1220 0x7f19e40a36e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f19ec006010 tx=0x7f19ec00bd20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:13.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.368+0000 7f19fde00700 1 -- 172.21.15.86:0/2025940250 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e60586bf80 msgr2=0x55e60586c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:50:13.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.368+0000 7f19fde00700 1 --2- 172.21.15.86:0/2025940250 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e60586bf80 0x55e60586c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.368+0000 7f19fde00700 1 -- 172.21.15.86:0/2025940250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e60586b0e0 msgr2=0x55e60586b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:50:13.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.368+0000 7f19fde00700 1 --2- 172.21.15.86:0/2025940250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e60586b0e0 0x55e60586b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.368+0000 7f19fde00700 1 -- 172.21.15.86:0/2025940250 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e60563d360 con 0x55e60586a780 2024-03-23T17:50:13.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.368+0000 7f19fd400700 1 --2- 172.21.15.86:0/2025940250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e60586b0e0 0x55e60586b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:50:13.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.368+0000 7f19fde00700 1 --2- 172.21.15.86:0/2025940250 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e60586a780 0x55e60586aba0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f19f0009d30 tx=0x7f19f0011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d92b5efbf83de5e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:13.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.369+0000 7f19eb000700 1 -- 172.21.15.86:0/2025940250 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f0004220 con 0x55e60586a780 2024-03-23T17:50:13.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.369+0000 7f19eb000700 1 -- 172.21.15.86:0/2025940250 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19f0004380 con 0x55e60586a780 2024-03-23T17:50:13.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.369+0000 7f19eb000700 1 -- 172.21.15.86:0/2025940250 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f00198d0 con 0x55e60586a780 2024-03-23T17:50:13.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.369+0000 7f1a184a0b80 1 -- 172.21.15.86:0/2025940250 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e60586a780 msgr2=0x55e60586aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:13.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.369+0000 7f1a184a0b80 1 --2- 172.21.15.86:0/2025940250 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e60586a780 0x55e60586aba0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f19f0009d30 tx=0x7f19f0011720 comp rx=0 tx=0).stop 2024-03-23T17:50:13.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.370+0000 7f1a184a0b80 1 -- 172.21.15.86:0/2025940250 shutdown_connections 2024-03-23T17:50:13.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.370+0000 7f1a184a0b80 1 --2- 172.21.15.86:0/2025940250 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e60586bf80 0x55e60586c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.370+0000 7f1a184a0b80 1 --2- 172.21.15.86:0/2025940250 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e60586a780 0x55e60586aba0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.370+0000 7f1a184a0b80 1 --2- 172.21.15.86:0/2025940250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e60586b0e0 0x55e60586b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.370+0000 7f1a184a0b80 1 -- 172.21.15.86:0/2025940250 >> 172.21.15.86:0/2025940250 conn(0x55e6057c9650 msgr2=0x55e605865950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:13.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.370+0000 7f1a184a0b80 1 -- 172.21.15.86:0/2025940250 shutdown_connections 2024-03-23T17:50:13.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.370+0000 7f1a184a0b80 1 -- 172.21.15.86:0/2025940250 wait complete. 2024-03-23T17:50:13.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.370+0000 7f1a184a0b80 1 -- start start 2024-03-23T17:50:13.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.370+0000 7f1a184a0b80 1 Processor -- start 2024-03-23T17:50:13.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.371+0000 7f1a184a0b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e60586a780 0x55e605878a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:13.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.371+0000 7f1a184a0b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e60586b0e0 0x55e605878fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:13.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.371+0000 7f1a184a0b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e60586bf80 0x55e60587fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:13.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.371+0000 7f1a184a0b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e605757d30 con 0x55e60586b0e0 2024-03-23T17:50:13.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.371+0000 7f1a184a0b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6057579d0 con 0x55e60586a780 2024-03-23T17:50:13.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.371+0000 7f1a184a0b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e60563f650 con 0x55e60586bf80 2024-03-23T17:50:13.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.371+0000 7f1a184a0b80 10 client.0 _ll_register_callbacks cb 0x55e605761420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:50:13.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.371+0000 7f19fde00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e60586a780 0x55e605878a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:13.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.371+0000 7f19fd400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e60586b0e0 0x55e605878fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:13.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.371+0000 7f19fde00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e60586a780 0x55e605878a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:54766/0 (socket says 172.21.15.86:54766) 2024-03-23T17:50:13.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.371+0000 7f19fde00700 1 -- 172.21.15.86:0/1654983303 learned_addr learned my addr 172.21.15.86:0/1654983303 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:13.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.371+0000 7f19fe800700 1 --2- 172.21.15.86:0/1654983303 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e60586bf80 0x55e60587fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:13.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.371+0000 7f19fde00700 1 -- 172.21.15.86:0/1654983303 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e60586bf80 msgr2=0x55e60587fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:50:13.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.371+0000 7f19fde00700 1 --2- 172.21.15.86:0/1654983303 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e60586bf80 0x55e60587fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.372+0000 7f19fde00700 1 -- 172.21.15.86:0/1654983303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e60586b0e0 msgr2=0x55e605878fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:50:13.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.372+0000 7f19fde00700 1 --2- 172.21.15.86:0/1654983303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e60586b0e0 0x55e605878fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.372+0000 7f19fde00700 1 -- 172.21.15.86:0/1654983303 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19ec020040 con 0x55e60586a780 2024-03-23T17:50:13.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.372+0000 7f19fd400700 1 --2- 172.21.15.86:0/1654983303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e60586b0e0 0x55e605878fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:50:13.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.372+0000 7f19fde00700 1 --2- 172.21.15.86:0/1654983303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e60586a780 0x55e605878a70 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f19f0005950 tx=0x7f19f0003e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d382f01b24e0964 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:13.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.372+0000 7f19df400700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T17:50:13.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.372+0000 7f19df400700 1 -- 172.21.15.86:0/1654983303 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f0003b10 con 0x55e60586a780 2024-03-23T17:50:13.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.372+0000 7f19df400700 1 -- 172.21.15.86:0/1654983303 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f19f0005040 con 0x55e60586a780 2024-03-23T17:50:13.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.372+0000 7f1a184a0b80 1 -- 172.21.15.86:0/1654983303 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e60577c330 con 0x55e60586a780 2024-03-23T17:50:13.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.372+0000 7f19df400700 1 -- 172.21.15.86:0/1654983303 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19f0005200 con 0x55e60586a780 2024-03-23T17:50:13.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.372+0000 7f1a184a0b80 10 client.25196 fetch_fsmap learned FSMap version 27 2024-03-23T17:50:13.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.372+0000 7f1a184a0b80 1 -- 172.21.15.86:0/1654983303 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f19f0009960 con 0x55e60586a780 2024-03-23T17:50:13.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.373+0000 7f19df400700 1 -- 172.21.15.86:0/1654983303 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e60577c330 con 0x55e60586a780 2024-03-23T17:50:13.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.373+0000 7f19df400700 1 -- 172.21.15.86:0/1654983303 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f19f0021040 con 0x55e60586a780 2024-03-23T17:50:13.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.373+0000 7f1a184a0b80 10 client.25196 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T17:50:13.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.373+0000 7f1a184a0b80 10 client.25196 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-23T17:50:13.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.373+0000 7f1a184a0b80 20 client.25196 populate_metadata read hostname 'smithi086' 2024-03-23T17:50:13.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.373+0000 7f1a184a0b80 1 --2- 172.21.15.86:0/1654983303 >> [v2:172.21.15.92:6836/3967837948,v1:172.21.15.92:6837/3967837948] conn(0x55e605883ba0 0x55e605888ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:13.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.373+0000 7f1a184a0b80 4 client.25196 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:50:13.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.373+0000 7f1a184a0b80 1 -- 172.21.15.86:0/1654983303 --> [v2:172.21.15.92:6836/3967837948,v1:172.21.15.92:6837/3967837948] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e6057b46b0 con 0x55e605883ba0 2024-03-23T17:50:13.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.373+0000 7f19fd400700 1 --2- 172.21.15.86:0/1654983303 >> [v2:172.21.15.92:6836/3967837948,v1:172.21.15.92:6837/3967837948] conn(0x55e605883ba0 0x55e605888ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:13.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.374+0000 7f19fd400700 1 --2- 172.21.15.86:0/1654983303 >> [v2:172.21.15.92:6836/3967837948,v1:172.21.15.92:6837/3967837948] conn(0x55e605883ba0 0x55e605888ea0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=867f5d1c0d789399 server_cookie=6623c2e3d3127146 in_seq=0 out_seq=0 2024-03-23T17:50:13.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.374+0000 7f19df400700 10 client.25196 ms_handle_connect on v2:172.21.15.92:6836/3967837948 2024-03-23T17:50:13.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.374+0000 7f19df400700 1 -- 172.21.15.86:0/1654983303 <== mds.2 v2:172.21.15.92:6836/3967837948 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55e6057b46b0 con 0x55e605883ba0 2024-03-23T17:50:13.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.374+0000 7f19df400700 10 client.25196 handle_command_reply: tid=0 2024-03-23T17:50:13.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.375+0000 7f1a184a0b80 1 client.25196 shutdown 2024-03-23T17:50:13.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.375+0000 7f1a184a0b80 1 -- 172.21.15.86:0/1654983303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e60586a780 msgr2=0x55e605878a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:13.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.375+0000 7f1a184a0b80 1 --2- 172.21.15.86:0/1654983303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e60586a780 0x55e605878a70 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f19f0005950 tx=0x7f19f0003e60 comp rx=0 tx=0).stop 2024-03-23T17:50:13.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.375+0000 7f1a184a0b80 1 -- 172.21.15.86:0/1654983303 shutdown_connections 2024-03-23T17:50:13.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.375+0000 7f1a184a0b80 1 --2- 172.21.15.86:0/1654983303 >> [v2:172.21.15.92:6836/3967837948,v1:172.21.15.92:6837/3967837948] conn(0x55e605883ba0 0x55e605888ea0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.375+0000 7f1a184a0b80 1 --2- 172.21.15.86:0/1654983303 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e60586bf80 0x55e60587fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.375+0000 7f19df400700 0 client.25196 ms_handle_reset on v2:172.21.15.92:6836/3967837948 2024-03-23T17:50:13.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.375+0000 7f1a184a0b80 1 --2- 172.21.15.86:0/1654983303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e60586a780 0x55e605878a70 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.375+0000 7f1a184a0b80 1 --2- 172.21.15.86:0/1654983303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e60586b0e0 0x55e605878fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.375+0000 7f1a184a0b80 1 -- 172.21.15.86:0/1654983303 >> 172.21.15.86:0/1654983303 conn(0x55e6057c9650 msgr2=0x55e605869120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:13.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.375+0000 7f1a184a0b80 1 -- 172.21.15.86:0/1654983303 shutdown_connections 2024-03-23T17:50:13.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.375+0000 7f1a184a0b80 1 -- 172.21.15.86:0/1654983303 wait complete. 2024-03-23T17:50:13.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.375+0000 7f1a184a0b80 20 client.25196 trim_cache size 0 max 16384 2024-03-23T17:50:13.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.393+0000 7f1a184a0b80 1 Processor -- start 2024-03-23T17:50:13.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.393+0000 7f1a184a0b80 1 -- start start 2024-03-23T17:50:13.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.393+0000 7f1a184a0b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e60586a780 0x55e60587dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:13.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.393+0000 7f1a184a0b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e60586b0e0 0x55e6058806a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:13.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.393+0000 7f1a184a0b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e60586bf80 0x55e60586c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:13.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.393+0000 7f1a184a0b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e60577c330 con 0x55e60586a780 2024-03-23T17:50:13.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.393+0000 7f1a184a0b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e605779ad0 con 0x55e60586bf80 2024-03-23T17:50:13.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.393+0000 7f1a184a0b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6057b4990 con 0x55e60586b0e0 2024-03-23T17:50:13.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.393+0000 7f19fde00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e60586a780 0x55e60587dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:13.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.393+0000 7f19fde00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e60586a780 0x55e60587dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38270/0 (socket says 172.21.15.86:38270) 2024-03-23T17:50:13.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.393+0000 7f19fde00700 1 -- 172.21.15.86:0/1339813631 learned_addr learned my addr 172.21.15.86:0/1339813631 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:13.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.393+0000 7f19fd400700 1 --2- 172.21.15.86:0/1339813631 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e60586b0e0 0x55e6058806a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:13.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.393+0000 7f19fe800700 1 --2- 172.21.15.86:0/1339813631 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e60586bf80 0x55e60586c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:13.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.393+0000 7f19fde00700 1 -- 172.21.15.86:0/1339813631 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e60586b0e0 msgr2=0x55e6058806a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:50:13.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.393+0000 7f19fde00700 1 --2- 172.21.15.86:0/1339813631 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e60586b0e0 0x55e6058806a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.393+0000 7f19fde00700 1 -- 172.21.15.86:0/1339813631 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e60586bf80 msgr2=0x55e60586c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:50:13.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.393+0000 7f19fde00700 1 --2- 172.21.15.86:0/1339813631 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e60586bf80 0x55e60586c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.393+0000 7f19fde00700 1 -- 172.21.15.86:0/1339813631 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e6055e9b10 con 0x55e60586a780 2024-03-23T17:50:13.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.393+0000 7f19fd400700 1 --2- 172.21.15.86:0/1339813631 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e60586b0e0 0x55e6058806a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:50:13.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.394+0000 7f19fde00700 1 --2- 172.21.15.86:0/1339813631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e60586a780 0x55e60587dfa0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f19f0009c50 tx=0x7f19f0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab9beb9ee4a7362e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:13.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.394+0000 7f19eba00700 1 -- 172.21.15.86:0/1339813631 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f0019430 con 0x55e60586a780 2024-03-23T17:50:13.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.394+0000 7f19eba00700 1 -- 172.21.15.86:0/1339813631 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19f0019890 con 0x55e60586a780 2024-03-23T17:50:13.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.394+0000 7f19eba00700 1 -- 172.21.15.86:0/1339813631 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f0005380 con 0x55e60586a780 2024-03-23T17:50:13.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.394+0000 7f1a184a0b80 1 -- 172.21.15.86:0/1339813631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e60586a780 msgr2=0x55e60587dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:13.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.394+0000 7f1a184a0b80 1 --2- 172.21.15.86:0/1339813631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e60586a780 0x55e60587dfa0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f19f0009c50 tx=0x7f19f0009260 comp rx=0 tx=0).stop 2024-03-23T17:50:13.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.394+0000 7f1a184a0b80 1 -- 172.21.15.86:0/1339813631 shutdown_connections 2024-03-23T17:50:13.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.394+0000 7f1a184a0b80 1 --2- 172.21.15.86:0/1339813631 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e60586bf80 0x55e60586c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.394+0000 7f1a184a0b80 1 --2- 172.21.15.86:0/1339813631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e60586a780 0x55e60587dfa0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.394+0000 7f1a184a0b80 1 --2- 172.21.15.86:0/1339813631 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e60586b0e0 0x55e6058806a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.394+0000 7f1a184a0b80 1 -- 172.21.15.86:0/1339813631 >> 172.21.15.86:0/1339813631 conn(0x55e6057c9650 msgr2=0x55e6058725b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:13.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.395+0000 7f1a184a0b80 1 -- 172.21.15.86:0/1339813631 shutdown_connections 2024-03-23T17:50:13.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.395+0000 7f1a184a0b80 1 -- 172.21.15.86:0/1339813631 wait complete. 2024-03-23T17:50:13.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.395+0000 7f1a184a0b80 1 -- start start 2024-03-23T17:50:13.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.395+0000 7f1a184a0b80 1 Processor -- start 2024-03-23T17:50:13.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.395+0000 7f1a184a0b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e60586a780 0x55e60587c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:13.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.396+0000 7f1a184a0b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e60586b0e0 0x55e60587c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:13.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.396+0000 7f1a184a0b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e60586bf80 0x55e60587d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:13.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.396+0000 7f1a184a0b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e60587d640 con 0x55e60586bf80 2024-03-23T17:50:13.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.396+0000 7f1a184a0b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e60587d7b0 con 0x55e60586a780 2024-03-23T17:50:13.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.396+0000 7f1a184a0b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e60587d970 con 0x55e60586b0e0 2024-03-23T17:50:13.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.396+0000 7f1a184a0b80 10 client.0 _ll_register_callbacks cb 0x55e605761420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:50:13.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.396+0000 7f19fde00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e60586a780 0x55e60587c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:13.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.396+0000 7f19fde00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e60586a780 0x55e60587c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:54784/0 (socket says 172.21.15.86:54784) 2024-03-23T17:50:13.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.396+0000 7f19fde00700 1 -- 172.21.15.86:0/3767927239 learned_addr learned my addr 172.21.15.86:0/3767927239 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:13.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.396+0000 7f19fe800700 1 --2- 172.21.15.86:0/3767927239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e60586bf80 0x55e60587d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:13.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.396+0000 7f19fd400700 1 --2- 172.21.15.86:0/3767927239 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e60586b0e0 0x55e60587c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:13.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.396+0000 7f19fde00700 1 -- 172.21.15.86:0/3767927239 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e60586b0e0 msgr2=0x55e60587c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:50:13.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.396+0000 7f19fde00700 1 --2- 172.21.15.86:0/3767927239 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e60586b0e0 0x55e60587c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.396+0000 7f19fde00700 1 -- 172.21.15.86:0/3767927239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e60586bf80 msgr2=0x55e60587d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:50:13.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.396+0000 7f19fde00700 1 --2- 172.21.15.86:0/3767927239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e60586bf80 0x55e60587d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.396+0000 7f19fde00700 1 -- 172.21.15.86:0/3767927239 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19f0009960 con 0x55e60586a780 2024-03-23T17:50:13.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.396+0000 7f19fe800700 1 --2- 172.21.15.86:0/3767927239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e60586bf80 0x55e60587d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:50:13.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.396+0000 7f19fde00700 1 --2- 172.21.15.86:0/3767927239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e60586a780 0x55e60587c140 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f19f0005d90 tx=0x7f19f0005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97de318019913652 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:13.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.397+0000 7f19df400700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T17:50:13.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.397+0000 7f19df400700 1 -- 172.21.15.86:0/3767927239 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f002a460 con 0x55e60586a780 2024-03-23T17:50:13.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.397+0000 7f1a184a0b80 1 -- 172.21.15.86:0/3767927239 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f19f0004380 con 0x55e60586a780 2024-03-23T17:50:13.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.397+0000 7f19df400700 1 -- 172.21.15.86:0/3767927239 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f19f0000e40 con 0x55e60586a780 2024-03-23T17:50:13.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.397+0000 7f19df400700 1 -- 172.21.15.86:0/3767927239 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19f0019ad0 con 0x55e60586a780 2024-03-23T17:50:13.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.397+0000 7f1a184a0b80 10 client.25202 fetch_fsmap learned FSMap version 27 2024-03-23T17:50:13.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.397+0000 7f1a184a0b80 1 -- 172.21.15.86:0/3767927239 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e605781e50 con 0x55e60586a780 2024-03-23T17:50:13.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.397+0000 7f19df400700 1 -- 172.21.15.86:0/3767927239 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19f004f020 con 0x55e60586a780 2024-03-23T17:50:13.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.398+0000 7f19df400700 1 -- 172.21.15.86:0/3767927239 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f19f0021030 con 0x55e60586a780 2024-03-23T17:50:13.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.398+0000 7f1a184a0b80 10 client.25202 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T17:50:13.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.398+0000 7f1a184a0b80 10 client.25202 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-23T17:50:13.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.398+0000 7f1a184a0b80 20 client.25202 populate_metadata read hostname 'smithi086' 2024-03-23T17:50:13.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.398+0000 7f1a184a0b80 1 --2- 172.21.15.86:0/3767927239 >> [v2:172.21.15.92:6836/3967837948,v1:172.21.15.92:6837/3967837948] conn(0x55e6058d9f30 0x55e6058d68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:13.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.398+0000 7f1a184a0b80 4 client.25202 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-23T17:50:13.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.398+0000 7f1a184a0b80 1 -- 172.21.15.86:0/3767927239 --> [v2:172.21.15.92:6836/3967837948,v1:172.21.15.92:6837/3967837948] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e60560aca0 con 0x55e6058d9f30 2024-03-23T17:50:13.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.398+0000 7f19fd400700 1 --2- 172.21.15.86:0/3767927239 >> [v2:172.21.15.92:6836/3967837948,v1:172.21.15.92:6837/3967837948] conn(0x55e6058d9f30 0x55e6058d68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:13.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.398+0000 7f19fd400700 1 --2- 172.21.15.86:0/3767927239 >> [v2:172.21.15.92:6836/3967837948,v1:172.21.15.92:6837/3967837948] conn(0x55e6058d9f30 0x55e6058d68a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=46fda3d4502826b2 server_cookie=bada202b21865bee in_seq=0 out_seq=0 2024-03-23T17:50:13.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.398+0000 7f19df400700 10 client.25202 ms_handle_connect on v2:172.21.15.92:6836/3967837948 2024-03-23T17:50:13.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.399+0000 7f19df400700 1 -- 172.21.15.86:0/3767927239 <== mds.2 v2:172.21.15.92:6836/3967837948 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55e60560aca0 con 0x55e6058d9f30 2024-03-23T17:50:13.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.399+0000 7f19df400700 10 client.25202 handle_command_reply: tid=0 2024-03-23T17:50:13.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.399+0000 7f1a184a0b80 1 client.25202 shutdown 2024-03-23T17:50:13.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.399+0000 7f1a184a0b80 1 -- 172.21.15.86:0/3767927239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e60586a780 msgr2=0x55e60587c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:13.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.399+0000 7f1a184a0b80 1 --2- 172.21.15.86:0/3767927239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e60586a780 0x55e60587c140 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f19f0005d90 tx=0x7f19f0005dc0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.399+0000 7f1a184a0b80 1 -- 172.21.15.86:0/3767927239 shutdown_connections 2024-03-23T17:50:13.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.399+0000 7f1a184a0b80 1 --2- 172.21.15.86:0/3767927239 >> [v2:172.21.15.92:6836/3967837948,v1:172.21.15.92:6837/3967837948] conn(0x55e6058d9f30 0x55e6058d68a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.399+0000 7f1a184a0b80 1 --2- 172.21.15.86:0/3767927239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e60586bf80 0x55e60587d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.399+0000 7f1a184a0b80 1 --2- 172.21.15.86:0/3767927239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e60586a780 0x55e60587c140 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.399+0000 7f19df400700 0 client.25202 ms_handle_reset on v2:172.21.15.92:6836/3967837948 2024-03-23T17:50:13.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.399+0000 7f1a184a0b80 1 --2- 172.21.15.86:0/3767927239 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e60586b0e0 0x55e60587c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.399+0000 7f1a184a0b80 1 -- 172.21.15.86:0/3767927239 >> 172.21.15.86:0/3767927239 conn(0x55e6057c9650 msgr2=0x55e60588cec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:13.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.399+0000 7f1a184a0b80 1 -- 172.21.15.86:0/3767927239 shutdown_connections 2024-03-23T17:50:13.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.399+0000 7f1a184a0b80 1 -- 172.21.15.86:0/3767927239 wait complete. 2024-03-23T17:50:13.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.399+0000 7f1a184a0b80 20 client.25202 trim_cache size 0 max 16384 2024-03-23T17:50:13.405 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-23T17:50:13.405 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "9d76c9e8-e93c-11ee-95cd-87774f69a715", 2024-03-23T17:50:13.405 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 2, 2024-03-23T17:50:13.405 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24518, 2024-03-23T17:50:13.405 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-03-23T17:50:13.405 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-03-23T17:50:13.406 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-03-23T17:50:13.406 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 72.746960100999999, 2024-03-23T17:50:13.406 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 27, 2024-03-23T17:50:13.406 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 82, 2024-03-23T17:50:13.406 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 80, 2024-03-23T17:50:13.406 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 92.513152923999996 2024-03-23T17:50:13.406 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-23T17:50:13.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.400+0000 7f1a04a00700 1 -- 172.21.15.86:0/3222558616 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f19e40a1220 msgr2=0x7f19e40a36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:13.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.400+0000 7f1a04a00700 1 --2- 172.21.15.86:0/3222558616 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f19e40a1220 0x7f19e40a36e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f19ec006010 tx=0x7f19ec00bd20 comp rx=0 tx=0).stop 2024-03-23T17:50:13.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.400+0000 7f1a04a00700 1 -- 172.21.15.86:0/3222558616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a00102820 msgr2=0x7f1a0019e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:13.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.400+0000 7f1a04a00700 1 --2- 172.21.15.86:0/3222558616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a00102820 0x7f1a0019e190 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f19f800ef30 tx=0x7f19f800c3a0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.400+0000 7f1a04a00700 1 -- 172.21.15.86:0/3222558616 shutdown_connections 2024-03-23T17:50:13.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.400+0000 7f1a04a00700 1 --2- 172.21.15.86:0/3222558616 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f19e40a1220 0x7f19e40a36e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.400+0000 7f1a04a00700 1 --2- 172.21.15.86:0/3222558616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a00102820 0x7f1a0019e190 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.400+0000 7f1a04a00700 1 --2- 172.21.15.86:0/3222558616 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1a00100c90 0x7f1a00196c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.400+0000 7f1a04a00700 1 --2- 172.21.15.86:0/3222558616 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a00101e80 0x7f1a00197170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.400+0000 7f1a04a00700 1 -- 172.21.15.86:0/3222558616 >> 172.21.15.86:0/3222558616 conn(0x7f1a000fc220 msgr2=0x7f1a00105250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:13.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.400+0000 7f1a04a00700 1 -- 172.21.15.86:0/3222558616 shutdown_connections 2024-03-23T17:50:13.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.400+0000 7f1a04a00700 1 -- 172.21.15.86:0/3222558616 wait complete. 2024-03-23T17:50:13.419 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-03-23T17:50:13.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.610+0000 7fbcc0000700 1 -- 172.21.15.86:0/1089541901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbcb8102910 msgr2=0x7fbcb81091c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:13.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.610+0000 7fbcc0000700 1 --2- 172.21.15.86:0/1089541901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbcb8102910 0x7fbcb81091c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fbcb000b0d0 tx=0x7fbcb000b3e0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.610+0000 7fbcc0000700 1 -- 172.21.15.86:0/1089541901 shutdown_connections 2024-03-23T17:50:13.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.610+0000 7fbcc0000700 1 --2- 172.21.15.86:0/1089541901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbcb8102910 0x7fbcb81091c0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.610+0000 7fbcc0000700 1 --2- 172.21.15.86:0/1089541901 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbcb8100d80 0x7fbcb81011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.610+0000 7fbcc0000700 1 --2- 172.21.15.86:0/1089541901 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbcb8101f70 0x7fbcb81023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.610+0000 7fbcc0000700 1 -- 172.21.15.86:0/1089541901 >> 172.21.15.86:0/1089541901 conn(0x7fbcb80fc2f0 msgr2=0x7fbcb80fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:13.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.610+0000 7fbcc0000700 1 -- 172.21.15.86:0/1089541901 shutdown_connections 2024-03-23T17:50:13.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.610+0000 7fbcc0000700 1 -- 172.21.15.86:0/1089541901 wait complete. 2024-03-23T17:50:13.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.611+0000 7fbcc0000700 1 Processor -- start 2024-03-23T17:50:13.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.611+0000 7fbcc0000700 1 -- start start 2024-03-23T17:50:13.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.612+0000 7fbcc0000700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbcb8100d80 0x7fbcb8074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:13.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.612+0000 7fbcc0000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbcb8101f70 0x7fbcb80731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:13.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.612+0000 7fbcc0000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbcb8102910 0x7fbcb80736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:13.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.612+0000 7fbcc0000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb8073e80 con 0x7fbcb8102910 2024-03-23T17:50:13.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.612+0000 7fbcc0000700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb8073ff0 con 0x7fbcb8101f70 2024-03-23T17:50:13.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.612+0000 7fbcc0000700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb81a0e90 con 0x7fbcb8100d80 2024-03-23T17:50:13.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.612+0000 7fbcbe000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbcb8102910 0x7fbcb80736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:13.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.612+0000 7fbcbcc00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbcb8101f70 0x7fbcb80731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:13.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.612+0000 7fbcbcc00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbcb8101f70 0x7fbcb80731a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:54798/0 (socket says 172.21.15.86:54798) 2024-03-23T17:50:13.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.612+0000 7fbcbcc00700 1 -- 172.21.15.86:0/847157984 learned_addr learned my addr 172.21.15.86:0/847157984 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:13.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.612+0000 7fbcbe000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbcb8102910 0x7fbcb80736e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38292/0 (socket says 172.21.15.86:38292) 2024-03-23T17:50:13.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.612+0000 7fbcbd600700 1 --2- 172.21.15.86:0/847157984 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbcb8100d80 0x7fbcb8074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:13.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.612+0000 7fbcbe000700 1 -- 172.21.15.86:0/847157984 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbcb8100d80 msgr2=0x7fbcb8074b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:13.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.612+0000 7fbcbe000700 1 --2- 172.21.15.86:0/847157984 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbcb8100d80 0x7fbcb8074b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.612+0000 7fbcbe000700 1 -- 172.21.15.86:0/847157984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbcb8101f70 msgr2=0x7fbcb80731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:13.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.612+0000 7fbcbe000700 1 --2- 172.21.15.86:0/847157984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbcb8101f70 0x7fbcb80731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.612+0000 7fbcbe000700 1 -- 172.21.15.86:0/847157984 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbca4009520 con 0x7fbcb8102910 2024-03-23T17:50:13.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.612+0000 7fbcbcc00700 1 --2- 172.21.15.86:0/847157984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbcb8101f70 0x7fbcb80731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:50:13.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.613+0000 7fbcbe000700 1 --2- 172.21.15.86:0/847157984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbcb8102910 0x7fbcb80736e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fbcb0007b60 tx=0x7fbcb0008e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:13.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.613+0000 7fbcb6000700 1 -- 172.21.15.86:0/847157984 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb000e040 con 0x7fbcb8102910 2024-03-23T17:50:13.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.613+0000 7fbcb6000700 1 -- 172.21.15.86:0/847157984 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbcb0005070 con 0x7fbcb8102910 2024-03-23T17:50:13.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.613+0000 7fbcb6000700 1 -- 172.21.15.86:0/847157984 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb001d6d0 con 0x7fbcb8102910 2024-03-23T17:50:13.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.613+0000 7fbcc0000700 1 -- 172.21.15.86:0/847157984 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcb0009d00 con 0x7fbcb8102910 2024-03-23T17:50:13.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.613+0000 7fbcc0000700 1 -- 172.21.15.86:0/847157984 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcb81a1300 con 0x7fbcb8102910 2024-03-23T17:50:13.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.616+0000 7fbcb6000700 1 -- 172.21.15.86:0/847157984 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fbcb00051e0 con 0x7fbcb8102910 2024-03-23T17:50:13.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.617+0000 7fbcd3a76b80 1 Processor -- start 2024-03-23T17:50:13.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.617+0000 7fbcb6000700 1 --2- 172.21.15.86:0/847157984 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fbca00a1230 0x7fbca00a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:13.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.618+0000 7fbcd3a76b80 1 -- start start 2024-03-23T17:50:13.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.618+0000 7fbcb6000700 1 -- 172.21.15.86:0/847157984 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fbcb00cd830 con 0x7fbcb8102910 2024-03-23T17:50:13.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.618+0000 7fbcbcc00700 1 --2- 172.21.15.86:0/847157984 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fbca00a1230 0x7fbca00a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:13.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.618+0000 7fbcd3a76b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563d102e9780 0x563d102e9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:13.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.618+0000 7fbcd3a76b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d102ea0e0 0x563d102ea9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:13.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.618+0000 7fbcd3a76b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563d102eaf80 0x563d102eb400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:13.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.618+0000 7fbcd3a76b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d10241940 con 0x563d102ea0e0 2024-03-23T17:50:13.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.618+0000 7fbcd3a76b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d1023f300 con 0x563d102eaf80 2024-03-23T17:50:13.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.618+0000 7fbcd3a76b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d1023abf0 con 0x563d102e9780 2024-03-23T17:50:13.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.618+0000 7fbcbd600700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563d102e9780 0x563d102e9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:13.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.618+0000 7fbcbcc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d102ea0e0 0x563d102ea9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:13.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.618+0000 7fbcbd600700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563d102e9780 0x563d102e9ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:52320/0 (socket says 172.21.15.86:52320) 2024-03-23T17:50:13.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.618+0000 7fbcbd600700 1 -- 172.21.15.86:0/3154358595 learned_addr learned my addr 172.21.15.86:0/3154358595 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:13.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.618+0000 7fbcbe000700 1 --2- 172.21.15.86:0/3154358595 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563d102eaf80 0x563d102eb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:13.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.619+0000 7fbcbcc00700 1 --2- 172.21.15.86:0/847157984 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fbca00a1230 0x7fbca00a36f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fbca4009fd0 tx=0x7fbca4011b30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:13.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.619+0000 7fbcbd600700 1 -- 172.21.15.86:0/3154358595 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563d102eaf80 msgr2=0x563d102eb400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:50:13.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.619+0000 7fbcbd600700 1 --2- 172.21.15.86:0/3154358595 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563d102eaf80 0x563d102eb400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.619+0000 7fbcbd600700 1 -- 172.21.15.86:0/3154358595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d102ea0e0 msgr2=0x563d102ea9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:50:13.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.619+0000 7fbcbd600700 1 --2- 172.21.15.86:0/3154358595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d102ea0e0 0x563d102ea9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.619+0000 7fbcbd600700 1 -- 172.21.15.86:0/3154358595 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d100bc360 con 0x563d102e9780 2024-03-23T17:50:13.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.619+0000 7fbcbcc00700 1 --2- 172.21.15.86:0/3154358595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d102ea0e0 0x563d102ea9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:50:13.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.619+0000 7fbcbd600700 1 --2- 172.21.15.86:0/3154358595 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563d102e9780 0x563d102e9ba0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbcac009d30 tx=0x7fbcac011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=983d4c235fb09477 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:13.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.619+0000 7fbcaa600700 1 -- 172.21.15.86:0/3154358595 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcac004220 con 0x563d102e9780 2024-03-23T17:50:13.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.619+0000 7fbcaa600700 1 -- 172.21.15.86:0/3154358595 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbcac004380 con 0x563d102e9780 2024-03-23T17:50:13.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.619+0000 7fbcaa600700 1 -- 172.21.15.86:0/3154358595 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcac019830 con 0x563d102e9780 2024-03-23T17:50:13.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.620+0000 7fbcd3a76b80 1 -- 172.21.15.86:0/3154358595 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563d102e9780 msgr2=0x563d102e9ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:13.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.620+0000 7fbcd3a76b80 1 --2- 172.21.15.86:0/3154358595 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563d102e9780 0x563d102e9ba0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbcac009d30 tx=0x7fbcac011720 comp rx=0 tx=0).stop 2024-03-23T17:50:13.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.620+0000 7fbcd3a76b80 1 -- 172.21.15.86:0/3154358595 shutdown_connections 2024-03-23T17:50:13.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.620+0000 7fbcd3a76b80 1 --2- 172.21.15.86:0/3154358595 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563d102eaf80 0x563d102eb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.620+0000 7fbcd3a76b80 1 --2- 172.21.15.86:0/3154358595 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563d102e9780 0x563d102e9ba0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.620+0000 7fbcd3a76b80 1 --2- 172.21.15.86:0/3154358595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d102ea0e0 0x563d102ea9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.620+0000 7fbcd3a76b80 1 -- 172.21.15.86:0/3154358595 >> 172.21.15.86:0/3154358595 conn(0x563d10248650 msgr2=0x563d102e4950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:13.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.620+0000 7fbcd3a76b80 1 -- 172.21.15.86:0/3154358595 shutdown_connections 2024-03-23T17:50:13.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.620+0000 7fbcd3a76b80 1 -- 172.21.15.86:0/3154358595 wait complete. 2024-03-23T17:50:13.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.621+0000 7fbcd3a76b80 1 -- start start 2024-03-23T17:50:13.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.621+0000 7fbcd3a76b80 1 Processor -- start 2024-03-23T17:50:13.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.621+0000 7fbcd3a76b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563d102e9780 0x563d102f35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:13.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.621+0000 7fbcd3a76b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d102ea0e0 0x563d102f3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:13.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.621+0000 7fbcd3a76b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563d102eaf80 0x563d102faae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:13.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.621+0000 7fbcd3a76b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d101d6d30 con 0x563d102ea0e0 2024-03-23T17:50:13.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.621+0000 7fbcd3a76b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d101d69d0 con 0x563d102eaf80 2024-03-23T17:50:13.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.621+0000 7fbcd3a76b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d100be650 con 0x563d102e9780 2024-03-23T17:50:13.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.621+0000 7fbcd3a76b80 10 client.0 _ll_register_callbacks cb 0x563d101e0420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:50:13.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.621+0000 7fbcbcc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d102ea0e0 0x563d102f3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:13.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.621+0000 7fbcbcc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d102ea0e0 0x563d102f3af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38300/0 (socket says 172.21.15.86:38300) 2024-03-23T17:50:13.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.621+0000 7fbcbcc00700 1 -- 172.21.15.86:0/4009738151 learned_addr learned my addr 172.21.15.86:0/4009738151 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:13.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.622+0000 7fbcbe000700 1 --2- 172.21.15.86:0/4009738151 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563d102eaf80 0x563d102faae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:13.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.622+0000 7fbcbd600700 1 --2- 172.21.15.86:0/4009738151 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563d102e9780 0x563d102f35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:13.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.622+0000 7fbcbcc00700 1 -- 172.21.15.86:0/4009738151 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563d102e9780 msgr2=0x563d102f35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:50:13.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.622+0000 7fbcbcc00700 1 --2- 172.21.15.86:0/4009738151 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563d102e9780 0x563d102f35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.622+0000 7fbcbcc00700 1 -- 172.21.15.86:0/4009738151 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563d102eaf80 msgr2=0x563d102faae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:50:13.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.622+0000 7fbcbcc00700 1 --2- 172.21.15.86:0/4009738151 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563d102eaf80 0x563d102faae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.622+0000 7fbcbcc00700 1 -- 172.21.15.86:0/4009738151 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbca400f630 con 0x563d102ea0e0 2024-03-23T17:50:13.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.622+0000 7fbcbcc00700 1 --2- 172.21.15.86:0/4009738151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d102ea0e0 0x563d102f3af0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fbca400ff50 tx=0x7fbca40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc51129d0bf91a73 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:13.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.622+0000 7fbc9ea00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-23T17:50:13.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.622+0000 7fbc9ea00700 1 -- 172.21.15.86:0/4009738151 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca40263f0 con 0x563d102ea0e0 2024-03-23T17:50:13.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.622+0000 7fbc9ea00700 1 -- 172.21.15.86:0/4009738151 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fbca4026550 con 0x563d102ea0e0 2024-03-23T17:50:13.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.622+0000 7fbcd3a76b80 1 -- 172.21.15.86:0/4009738151 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563d101fb330 con 0x563d102ea0e0 2024-03-23T17:50:13.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.623+0000 7fbc9ea00700 1 -- 172.21.15.86:0/4009738151 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbca4026710 con 0x563d102ea0e0 2024-03-23T17:50:13.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.623+0000 7fbcd3a76b80 10 client.15174 fetch_fsmap learned FSMap version 27 2024-03-23T17:50:13.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.623+0000 7fbcd3a76b80 1 -- 172.21.15.86:0/4009738151 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7fbcac009960 con 0x563d102ea0e0 2024-03-23T17:50:13.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.623+0000 7fbc9ea00700 1 -- 172.21.15.86:0/4009738151 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563d101fb330 con 0x563d102ea0e0 2024-03-23T17:50:13.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.623+0000 7fbc9ea00700 1 -- 172.21.15.86:0/4009738151 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fbca40318f0 con 0x563d102ea0e0 2024-03-23T17:50:13.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.623+0000 7fbcd3a76b80 10 client.15174 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T17:50:13.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.623+0000 7fbcd3a76b80 10 client.15174 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-23T17:50:13.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.623+0000 7fbcd3a76b80 20 client.15174 populate_metadata read hostname 'smithi086' 2024-03-23T17:50:13.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.623+0000 7fbcd3a76b80 1 --2- 172.21.15.86:0/4009738151 >> [v2:172.21.15.152:6838/1100723992,v1:172.21.15.152:6839/1100723992] conn(0x563d102fe800 0x563d10303b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:13.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.623+0000 7fbcd3a76b80 4 client.15174 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:50:13.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.623+0000 7fbcd3a76b80 1 -- 172.21.15.86:0/4009738151 --> [v2:172.21.15.152:6838/1100723992,v1:172.21.15.152:6839/1100723992] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563d102336b0 con 0x563d102fe800 2024-03-23T17:50:13.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.623+0000 7fbcbd600700 1 --2- 172.21.15.86:0/4009738151 >> [v2:172.21.15.152:6838/1100723992,v1:172.21.15.152:6839/1100723992] conn(0x563d102fe800 0x563d10303b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:13.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.624+0000 7fbc9ea00700 10 client.15174 ms_handle_connect on v2:172.21.15.152:6838/1100723992 2024-03-23T17:50:13.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.624+0000 7fbcbd600700 1 --2- 172.21.15.86:0/4009738151 >> [v2:172.21.15.152:6838/1100723992,v1:172.21.15.152:6839/1100723992] conn(0x563d102fe800 0x563d10303b20 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=90fc4431ffd3acb7 server_cookie=5f4b8b4f34215bd4 in_seq=0 out_seq=0 2024-03-23T17:50:13.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.625+0000 7fbc9ea00700 1 -- 172.21.15.86:0/4009738151 <== mds.1 v2:172.21.15.152:6838/1100723992 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x563d102336b0 con 0x563d102fe800 2024-03-23T17:50:13.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.625+0000 7fbc9ea00700 10 client.15174 handle_command_reply: tid=0 2024-03-23T17:50:13.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.625+0000 7fbcd3a76b80 1 client.15174 shutdown 2024-03-23T17:50:13.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.625+0000 7fbcd3a76b80 1 -- 172.21.15.86:0/4009738151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d102ea0e0 msgr2=0x563d102f3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:13.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.625+0000 7fbcd3a76b80 1 --2- 172.21.15.86:0/4009738151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d102ea0e0 0x563d102f3af0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fbca400ff50 tx=0x7fbca40047c0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.625+0000 7fbcd3a76b80 1 -- 172.21.15.86:0/4009738151 shutdown_connections 2024-03-23T17:50:13.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.625+0000 7fbcd3a76b80 1 --2- 172.21.15.86:0/4009738151 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563d102eaf80 0x563d102faae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.625+0000 7fbcd3a76b80 1 --2- 172.21.15.86:0/4009738151 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563d102e9780 0x563d102f35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.625+0000 7fbcd3a76b80 1 --2- 172.21.15.86:0/4009738151 >> [v2:172.21.15.152:6838/1100723992,v1:172.21.15.152:6839/1100723992] conn(0x563d102fe800 0x563d10303b20 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.625+0000 7fbcd3a76b80 1 --2- 172.21.15.86:0/4009738151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d102ea0e0 0x563d102f3af0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.625+0000 7fbc9ea00700 0 client.15174 ms_handle_reset on v2:172.21.15.152:6838/1100723992 2024-03-23T17:50:13.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.625+0000 7fbcd3a76b80 1 -- 172.21.15.86:0/4009738151 >> 172.21.15.86:0/4009738151 conn(0x563d10248650 msgr2=0x563d102e8120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:13.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.626+0000 7fbcd3a76b80 1 -- 172.21.15.86:0/4009738151 shutdown_connections 2024-03-23T17:50:13.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.626+0000 7fbcd3a76b80 1 -- 172.21.15.86:0/4009738151 wait complete. 2024-03-23T17:50:13.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.626+0000 7fbcd3a76b80 20 client.15174 trim_cache size 0 max 16384 2024-03-23T17:50:13.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.643+0000 7fbcd3a76b80 1 Processor -- start 2024-03-23T17:50:13.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.643+0000 7fbcd3a76b80 1 -- start start 2024-03-23T17:50:13.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.643+0000 7fbcd3a76b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563d102e9780 0x563d102e9b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:13.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.643+0000 7fbcd3a76b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d102ea0e0 0x563d102f8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:13.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.643+0000 7fbcd3a76b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563d102eaf80 0x563d102eb360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:13.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.643+0000 7fbcd3a76b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d103002b0 con 0x563d102ea0e0 2024-03-23T17:50:13.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.643+0000 7fbcd3a76b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d102fdf80 con 0x563d102e9780 2024-03-23T17:50:13.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.643+0000 7fbcd3a76b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d102336e0 con 0x563d102eaf80 2024-03-23T17:50:13.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.643+0000 7fbcbcc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d102ea0e0 0x563d102f8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:13.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.644+0000 7fbcbcc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d102ea0e0 0x563d102f8a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38304/0 (socket says 172.21.15.86:38304) 2024-03-23T17:50:13.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.644+0000 7fbcbcc00700 1 -- 172.21.15.86:0/2272083425 learned_addr learned my addr 172.21.15.86:0/2272083425 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:13.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.644+0000 7fbcbd600700 1 --2- 172.21.15.86:0/2272083425 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563d102e9780 0x563d102e9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:13.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.644+0000 7fbcbe000700 1 --2- 172.21.15.86:0/2272083425 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563d102eaf80 0x563d102eb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:13.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.644+0000 7fbcbcc00700 1 -- 172.21.15.86:0/2272083425 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563d102eaf80 msgr2=0x563d102eb360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:50:13.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.644+0000 7fbcbcc00700 1 --2- 172.21.15.86:0/2272083425 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563d102eaf80 0x563d102eb360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.644+0000 7fbcbcc00700 1 -- 172.21.15.86:0/2272083425 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563d102e9780 msgr2=0x563d102e9b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:50:13.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.644+0000 7fbcbcc00700 1 --2- 172.21.15.86:0/2272083425 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563d102e9780 0x563d102e9b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.644+0000 7fbcbcc00700 1 -- 172.21.15.86:0/2272083425 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d10068b10 con 0x563d102ea0e0 2024-03-23T17:50:13.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.644+0000 7fbcbcc00700 1 --2- 172.21.15.86:0/2272083425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d102ea0e0 0x563d102f8a90 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fbca4031c20 tx=0x7fbca40046f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73b6403cb58f8792 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:13.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.644+0000 7fbcab000700 1 -- 172.21.15.86:0/2272083425 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca401c3a0 con 0x563d102ea0e0 2024-03-23T17:50:13.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.644+0000 7fbcab000700 1 -- 172.21.15.86:0/2272083425 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbca4031e40 con 0x563d102ea0e0 2024-03-23T17:50:13.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.644+0000 7fbcab000700 1 -- 172.21.15.86:0/2272083425 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca401cc90 con 0x563d102ea0e0 2024-03-23T17:50:13.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.645+0000 7fbcd3a76b80 1 -- 172.21.15.86:0/2272083425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d102ea0e0 msgr2=0x563d102f8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:13.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.645+0000 7fbcd3a76b80 1 --2- 172.21.15.86:0/2272083425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d102ea0e0 0x563d102f8a90 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fbca4031c20 tx=0x7fbca40046f0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.645+0000 7fbcd3a76b80 1 -- 172.21.15.86:0/2272083425 shutdown_connections 2024-03-23T17:50:13.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.645+0000 7fbcd3a76b80 1 --2- 172.21.15.86:0/2272083425 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563d102eaf80 0x563d102eb360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.645+0000 7fbcd3a76b80 1 --2- 172.21.15.86:0/2272083425 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563d102e9780 0x563d102e9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.645+0000 7fbcd3a76b80 1 --2- 172.21.15.86:0/2272083425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d102ea0e0 0x563d102f8a90 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.645+0000 7fbcd3a76b80 1 -- 172.21.15.86:0/2272083425 >> 172.21.15.86:0/2272083425 conn(0x563d10248650 msgr2=0x563d102dd550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:13.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.645+0000 7fbcd3a76b80 1 -- 172.21.15.86:0/2272083425 shutdown_connections 2024-03-23T17:50:13.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.645+0000 7fbcd3a76b80 1 -- 172.21.15.86:0/2272083425 wait complete. 2024-03-23T17:50:13.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.645+0000 7fbcd3a76b80 1 -- start start 2024-03-23T17:50:13.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.646+0000 7fbcd3a76b80 1 Processor -- start 2024-03-23T17:50:13.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.646+0000 7fbcd3a76b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d102e9780 0x563d102f3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:13.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.646+0000 7fbcd3a76b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563d102ea0e0 0x563d102f7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:13.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.646+0000 7fbcd3a76b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563d102eaf80 0x563d102f8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:13.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.646+0000 7fbcd3a76b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d102f8790 con 0x563d102e9780 2024-03-23T17:50:13.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.646+0000 7fbcd3a76b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d102f8900 con 0x563d102eaf80 2024-03-23T17:50:13.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.646+0000 7fbcd3a76b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d10350e00 con 0x563d102ea0e0 2024-03-23T17:50:13.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.646+0000 7fbcd3a76b80 10 client.0 _ll_register_callbacks cb 0x563d101e0420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:50:13.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.646+0000 7fbcbd600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d102e9780 0x563d102f3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:13.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.646+0000 7fbcbd600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d102e9780 0x563d102f3bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38312/0 (socket says 172.21.15.86:38312) 2024-03-23T17:50:13.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.646+0000 7fbcbd600700 1 -- 172.21.15.86:0/3962249917 learned_addr learned my addr 172.21.15.86:0/3962249917 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:13.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.646+0000 7fbcbe000700 1 --2- 172.21.15.86:0/3962249917 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563d102eaf80 0x563d102f8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:13.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.646+0000 7fbcbcc00700 1 --2- 172.21.15.86:0/3962249917 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563d102ea0e0 0x563d102f7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:13.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.647+0000 7fbcbd600700 1 -- 172.21.15.86:0/3962249917 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563d102ea0e0 msgr2=0x563d102f7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:50:13.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.647+0000 7fbcbd600700 1 --2- 172.21.15.86:0/3962249917 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563d102ea0e0 0x563d102f7ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.647+0000 7fbcbd600700 1 -- 172.21.15.86:0/3962249917 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563d102eaf80 msgr2=0x563d102f8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:50:13.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.647+0000 7fbcbd600700 1 --2- 172.21.15.86:0/3962249917 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563d102eaf80 0x563d102f8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.647+0000 7fbcbd600700 1 -- 172.21.15.86:0/3962249917 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcac009960 con 0x563d102e9780 2024-03-23T17:50:13.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.647+0000 7fbcbe000700 1 --2- 172.21.15.86:0/3962249917 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563d102eaf80 0x563d102f8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:50:13.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.647+0000 7fbcbd600700 1 --2- 172.21.15.86:0/3962249917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d102e9780 0x563d102f3bd0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fbcac011a70 tx=0x7fbcac011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cd11ec45732d81b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:13.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.647+0000 7fbc9ea00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-23T17:50:13.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.647+0000 7fbc9ea00700 1 -- 172.21.15.86:0/3962249917 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcac029040 con 0x563d102e9780 2024-03-23T17:50:13.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.647+0000 7fbc9ea00700 1 -- 172.21.15.86:0/3962249917 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fbcac005240 con 0x563d102e9780 2024-03-23T17:50:13.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.647+0000 7fbcd3a76b80 1 -- 172.21.15.86:0/3962249917 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbcac004380 con 0x563d102e9780 2024-03-23T17:50:13.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.647+0000 7fbc9ea00700 1 -- 172.21.15.86:0/3962249917 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbcac005400 con 0x563d102e9780 2024-03-23T17:50:13.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.647+0000 7fbcd3a76b80 10 client.15180 fetch_fsmap learned FSMap version 27 2024-03-23T17:50:13.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.647+0000 7fbcd3a76b80 1 -- 172.21.15.86:0/3962249917 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x563d10200e50 con 0x563d102e9780 2024-03-23T17:50:13.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.648+0000 7fbc9ea00700 1 -- 172.21.15.86:0/3962249917 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbcac004380 con 0x563d102e9780 2024-03-23T17:50:13.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.648+0000 7fbc9ea00700 1 -- 172.21.15.86:0/3962249917 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fbcac011440 con 0x563d102e9780 2024-03-23T17:50:13.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.648+0000 7fbcd3a76b80 10 client.15180 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T17:50:13.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.648+0000 7fbcd3a76b80 10 client.15180 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-23T17:50:13.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.648+0000 7fbcd3a76b80 20 client.15180 populate_metadata read hostname 'smithi086' 2024-03-23T17:50:13.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.648+0000 7fbcd3a76b80 1 --2- 172.21.15.86:0/3962249917 >> [v2:172.21.15.152:6838/1100723992,v1:172.21.15.152:6839/1100723992] conn(0x563d10355b50 0x563d10352420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:13.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.648+0000 7fbcd3a76b80 4 client.15180 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-23T17:50:13.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.648+0000 7fbcd3a76b80 1 -- 172.21.15.86:0/3962249917 --> [v2:172.21.15.152:6838/1100723992,v1:172.21.15.152:6839/1100723992] -- command(tid 0: {"prefix": "status"}) v1 -- 0x563d10089ca0 con 0x563d10355b50 2024-03-23T17:50:13.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.648+0000 7fbcbcc00700 1 --2- 172.21.15.86:0/3962249917 >> [v2:172.21.15.152:6838/1100723992,v1:172.21.15.152:6839/1100723992] conn(0x563d10355b50 0x563d10352420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:13.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.648+0000 7fbcbcc00700 1 --2- 172.21.15.86:0/3962249917 >> [v2:172.21.15.152:6838/1100723992,v1:172.21.15.152:6839/1100723992] conn(0x563d10355b50 0x563d10352420 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8028063686c1fe41 server_cookie=f7e7c13bc0e35fc5 in_seq=0 out_seq=0 2024-03-23T17:50:13.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.648+0000 7fbc9ea00700 10 client.15180 ms_handle_connect on v2:172.21.15.152:6838/1100723992 2024-03-23T17:50:13.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.649+0000 7fbc9ea00700 1 -- 172.21.15.86:0/3962249917 <== mds.1 v2:172.21.15.152:6838/1100723992 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x563d10089ca0 con 0x563d10355b50 2024-03-23T17:50:13.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.649+0000 7fbc9ea00700 10 client.15180 handle_command_reply: tid=0 2024-03-23T17:50:13.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.649+0000 7fbcd3a76b80 1 client.15180 shutdown 2024-03-23T17:50:13.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.649+0000 7fbcd3a76b80 1 -- 172.21.15.86:0/3962249917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d102e9780 msgr2=0x563d102f3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:13.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.649+0000 7fbcd3a76b80 1 --2- 172.21.15.86:0/3962249917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d102e9780 0x563d102f3bd0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fbcac011a70 tx=0x7fbcac011b20 comp rx=0 tx=0).stop 2024-03-23T17:50:13.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.649+0000 7fbcd3a76b80 1 -- 172.21.15.86:0/3962249917 shutdown_connections 2024-03-23T17:50:13.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.649+0000 7fbcd3a76b80 1 --2- 172.21.15.86:0/3962249917 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563d102eaf80 0x563d102f8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.649+0000 7fbcd3a76b80 1 --2- 172.21.15.86:0/3962249917 >> [v2:172.21.15.152:6838/1100723992,v1:172.21.15.152:6839/1100723992] conn(0x563d10355b50 0x563d10352420 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.649+0000 7fbcd3a76b80 1 --2- 172.21.15.86:0/3962249917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d102e9780 0x563d102f3bd0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.649+0000 7fbcd3a76b80 1 --2- 172.21.15.86:0/3962249917 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563d102ea0e0 0x563d102f7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.649+0000 7fbcd3a76b80 1 -- 172.21.15.86:0/3962249917 >> 172.21.15.86:0/3962249917 conn(0x563d10248650 msgr2=0x563d102ee190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:13.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.649+0000 7fbc9ea00700 0 client.15180 ms_handle_reset on v2:172.21.15.152:6838/1100723992 2024-03-23T17:50:13.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.649+0000 7fbcd3a76b80 1 -- 172.21.15.86:0/3962249917 shutdown_connections 2024-03-23T17:50:13.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.649+0000 7fbcd3a76b80 1 -- 172.21.15.86:0/3962249917 wait complete. 2024-03-23T17:50:13.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.649+0000 7fbcd3a76b80 20 client.15180 trim_cache size 0 max 16384 2024-03-23T17:50:13.655 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-23T17:50:13.656 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "9d76c9e8-e93c-11ee-95cd-87774f69a715", 2024-03-23T17:50:13.656 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 1, 2024-03-23T17:50:13.656 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24553, 2024-03-23T17:50:13.656 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-03-23T17:50:13.656 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-03-23T17:50:13.656 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-03-23T17:50:13.656 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 73.986742293999995, 2024-03-23T17:50:13.656 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 27, 2024-03-23T17:50:13.656 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 82, 2024-03-23T17:50:13.656 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 80, 2024-03-23T17:50:13.656 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 90.927727031000003 2024-03-23T17:50:13.656 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-23T17:50:13.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.650+0000 7fbcc0000700 1 -- 172.21.15.86:0/847157984 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fbca00a1230 msgr2=0x7fbca00a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:13.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.650+0000 7fbcc0000700 1 --2- 172.21.15.86:0/847157984 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fbca00a1230 0x7fbca00a36f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fbca4009fd0 tx=0x7fbca4011b30 comp rx=0 tx=0).stop 2024-03-23T17:50:13.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.650+0000 7fbcc0000700 1 -- 172.21.15.86:0/847157984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbcb8102910 msgr2=0x7fbcb80736e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:13.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.650+0000 7fbcc0000700 1 --2- 172.21.15.86:0/847157984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbcb8102910 0x7fbcb80736e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fbcb0007b60 tx=0x7fbcb0008e40 comp rx=0 tx=0).stop 2024-03-23T17:50:13.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.650+0000 7fbcc0000700 1 -- 172.21.15.86:0/847157984 shutdown_connections 2024-03-23T17:50:13.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.650+0000 7fbcc0000700 1 --2- 172.21.15.86:0/847157984 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fbca00a1230 0x7fbca00a36f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.650+0000 7fbcc0000700 1 --2- 172.21.15.86:0/847157984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbcb8102910 0x7fbcb80736e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.650+0000 7fbcc0000700 1 --2- 172.21.15.86:0/847157984 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbcb8100d80 0x7fbcb8074b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.650+0000 7fbcc0000700 1 --2- 172.21.15.86:0/847157984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbcb8101f70 0x7fbcb80731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.650+0000 7fbcc0000700 1 -- 172.21.15.86:0/847157984 >> 172.21.15.86:0/847157984 conn(0x7fbcb80fc2f0 msgr2=0x7fbcb8105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:13.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.650+0000 7fbcc0000700 1 -- 172.21.15.86:0/847157984 shutdown_connections 2024-03-23T17:50:13.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.650+0000 7fbcc0000700 1 -- 172.21.15.86:0/847157984 wait complete. 2024-03-23T17:50:13.669 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-23T17:50:13.725 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:13 smithi086 ceph-mon[38221]: pgmap v274: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-23T17:50:13.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.862+0000 7fb4ff600700 1 -- 172.21.15.86:0/277556946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4f8100d80 msgr2=0x7fb4f81011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:13.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.862+0000 7fb4ff600700 1 --2- 172.21.15.86:0/277556946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4f8100d80 0x7fb4f81011a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec0099c0 tx=0x7fb4ec009cd0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.862+0000 7fb4ff600700 1 -- 172.21.15.86:0/277556946 shutdown_connections 2024-03-23T17:50:13.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.862+0000 7fb4ff600700 1 --2- 172.21.15.86:0/277556946 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4f8102910 0x7fb4f81091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.862+0000 7fb4ff600700 1 --2- 172.21.15.86:0/277556946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4f8100d80 0x7fb4f81011a0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.862+0000 7fb4ff600700 1 --2- 172.21.15.86:0/277556946 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4f8101f70 0x7fb4f81023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.862+0000 7fb4ff600700 1 -- 172.21.15.86:0/277556946 >> 172.21.15.86:0/277556946 conn(0x7fb4f80fc2f0 msgr2=0x7fb4f80fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:13.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.862+0000 7fb4ff600700 1 -- 172.21.15.86:0/277556946 shutdown_connections 2024-03-23T17:50:13.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.862+0000 7fb4ff600700 1 -- 172.21.15.86:0/277556946 wait complete. 2024-03-23T17:50:13.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.863+0000 7fb4ff600700 1 Processor -- start 2024-03-23T17:50:13.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.863+0000 7fb4ff600700 1 -- start start 2024-03-23T17:50:13.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.863+0000 7fb4ff600700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4f8100d80 0x7fb4f8198f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:13.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.863+0000 7fb4ff600700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4f8101f70 0x7fb4f81994c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:13.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.863+0000 7fb4ff600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4f8102910 0x7fb4f81a04e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:13.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.863+0000 7fb4ff600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f81a0ab0 con 0x7fb4f8102910 2024-03-23T17:50:13.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.863+0000 7fb4ff600700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f81a0bf0 con 0x7fb4f8100d80 2024-03-23T17:50:13.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.863+0000 7fb4ff600700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f81a0d80 con 0x7fb4f8101f70 2024-03-23T17:50:13.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.864+0000 7fb4fd600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4f8102910 0x7fb4f81a04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:13.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.864+0000 7fb4fd600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4f8102910 0x7fb4f81a04e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38328/0 (socket says 172.21.15.86:38328) 2024-03-23T17:50:13.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.864+0000 7fb4fd600700 1 -- 172.21.15.86:0/1738069210 learned_addr learned my addr 172.21.15.86:0/1738069210 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:13.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.864+0000 7fb4fcc00700 1 --2- 172.21.15.86:0/1738069210 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4f8100d80 0x7fb4f8198f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:13.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.864+0000 7fb4f7e00700 1 --2- 172.21.15.86:0/1738069210 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4f8101f70 0x7fb4f81994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:13.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.864+0000 7fb4fd600700 1 -- 172.21.15.86:0/1738069210 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4f8101f70 msgr2=0x7fb4f81994c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:13.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.864+0000 7fb4fd600700 1 --2- 172.21.15.86:0/1738069210 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4f8101f70 0x7fb4f81994c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.864+0000 7fb4fd600700 1 -- 172.21.15.86:0/1738069210 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4f8100d80 msgr2=0x7fb4f8198f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:13.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.864+0000 7fb4fd600700 1 --2- 172.21.15.86:0/1738069210 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4f8100d80 0x7fb4f8198f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.864+0000 7fb4fd600700 1 -- 172.21.15.86:0/1738069210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4e4009520 con 0x7fb4f8102910 2024-03-23T17:50:13.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.864+0000 7fb4fd600700 1 --2- 172.21.15.86:0/1738069210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4f8102910 0x7fb4f81a04e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb4f000ef30 tx=0x7fb4f000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:13.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.864+0000 7fb4f5600700 1 -- 172.21.15.86:0/1738069210 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f0018070 con 0x7fb4f8102910 2024-03-23T17:50:13.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.864+0000 7fb4f5600700 1 -- 172.21.15.86:0/1738069210 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb4f000f040 con 0x7fb4f8102910 2024-03-23T17:50:13.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.864+0000 7fb4f5600700 1 -- 172.21.15.86:0/1738069210 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f000cd00 con 0x7fb4f8102910 2024-03-23T17:50:13.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.865+0000 7fb4ff600700 1 -- 172.21.15.86:0/1738069210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4ec0095f0 con 0x7fb4f8102910 2024-03-23T17:50:13.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.865+0000 7fb4ff600700 1 -- 172.21.15.86:0/1738069210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4f81a1330 con 0x7fb4f8102910 2024-03-23T17:50:13.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.868+0000 7fb4f5600700 1 -- 172.21.15.86:0/1738069210 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fb4f0003a00 con 0x7fb4f8102910 2024-03-23T17:50:13.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.870+0000 7fb51309ab80 1 Processor -- start 2024-03-23T17:50:13.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.870+0000 7fb4f5600700 1 --2- 172.21.15.86:0/1738069210 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fb4e00a11b0 0x7fb4e00a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:13.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.870+0000 7fb51309ab80 1 -- start start 2024-03-23T17:50:13.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.870+0000 7fb4f5600700 1 -- 172.21.15.86:0/1738069210 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fb4f00cc5c0 con 0x7fb4f8102910 2024-03-23T17:50:13.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.870+0000 7fb4f7e00700 1 --2- 172.21.15.86:0/1738069210 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fb4e00a11b0 0x7fb4e00a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:13.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.870+0000 7fb51309ab80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561662595780 0x561662595ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:13.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.870+0000 7fb51309ab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5616625960e0 0x5616625969a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:13.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.870+0000 7fb51309ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561662596f80 0x561662597400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:13.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.870+0000 7fb51309ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616624ed940 con 0x561662596f80 2024-03-23T17:50:13.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.870+0000 7fb51309ab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616624eb300 con 0x5616625960e0 2024-03-23T17:50:13.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.870+0000 7fb51309ab80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616624e6bf0 con 0x561662595780 2024-03-23T17:50:13.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.871+0000 7fb4fcc00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561662595780 0x561662595ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:13.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.871+0000 7fb4fcc00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561662595780 0x561662595ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:52378/0 (socket says 172.21.15.86:52378) 2024-03-23T17:50:13.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.871+0000 7fb4fcc00700 1 -- 172.21.15.86:0/3844685969 learned_addr learned my addr 172.21.15.86:0/3844685969 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:13.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.871+0000 7fb4fd600700 1 --2- 172.21.15.86:0/3844685969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561662596f80 0x561662597400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:13.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.871+0000 7fb4f7e00700 1 --2- 172.21.15.86:0/3844685969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5616625960e0 0x5616625969a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:13.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.871+0000 7fb4fcc00700 1 -- 172.21.15.86:0/3844685969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5616625960e0 msgr2=0x5616625969a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:50:13.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.871+0000 7fb4fcc00700 1 --2- 172.21.15.86:0/3844685969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5616625960e0 0x5616625969a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.871+0000 7fb4fcc00700 1 -- 172.21.15.86:0/3844685969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561662596f80 msgr2=0x561662597400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:50:13.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.871+0000 7fb4fcc00700 1 --2- 172.21.15.86:0/3844685969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561662596f80 0x561662597400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.871+0000 7fb4fcc00700 1 -- 172.21.15.86:0/3844685969 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561662368360 con 0x561662595780 2024-03-23T17:50:13.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.871+0000 7fb4f7e00700 1 --2- 172.21.15.86:0/3844685969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5616625960e0 0x5616625969a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:50:13.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.871+0000 7fb4f7e00700 1 --2- 172.21.15.86:0/1738069210 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fb4e00a11b0 0x7fb4e00a3670 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb4e4009e00 tx=0x7fb4e4011720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:13.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.871+0000 7fb4fcc00700 1 --2- 172.21.15.86:0/3844685969 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561662595780 0x561662595ba0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec006010 tx=0x7fb4ec005bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=440851346bf66169 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:13.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.871+0000 7fb4e9c00700 1 -- 172.21.15.86:0/3844685969 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ec00ba10 con 0x561662595780 2024-03-23T17:50:13.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.871+0000 7fb4e9c00700 1 -- 172.21.15.86:0/3844685969 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb4ec01e040 con 0x561662595780 2024-03-23T17:50:13.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.871+0000 7fb4e9c00700 1 -- 172.21.15.86:0/3844685969 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ec003b30 con 0x561662595780 2024-03-23T17:50:13.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.872+0000 7fb51309ab80 1 -- 172.21.15.86:0/3844685969 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561662595780 msgr2=0x561662595ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:13.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.872+0000 7fb51309ab80 1 --2- 172.21.15.86:0/3844685969 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561662595780 0x561662595ba0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec006010 tx=0x7fb4ec005bc0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.872+0000 7fb51309ab80 1 -- 172.21.15.86:0/3844685969 shutdown_connections 2024-03-23T17:50:13.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.872+0000 7fb51309ab80 1 --2- 172.21.15.86:0/3844685969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561662596f80 0x561662597400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.872+0000 7fb51309ab80 1 --2- 172.21.15.86:0/3844685969 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561662595780 0x561662595ba0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.872+0000 7fb51309ab80 1 --2- 172.21.15.86:0/3844685969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5616625960e0 0x5616625969a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.872+0000 7fb51309ab80 1 -- 172.21.15.86:0/3844685969 >> 172.21.15.86:0/3844685969 conn(0x5616624f4650 msgr2=0x561662590950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:13.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.872+0000 7fb51309ab80 1 -- 172.21.15.86:0/3844685969 shutdown_connections 2024-03-23T17:50:13.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.872+0000 7fb51309ab80 1 -- 172.21.15.86:0/3844685969 wait complete. 2024-03-23T17:50:13.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.873+0000 7fb51309ab80 1 -- start start 2024-03-23T17:50:13.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.873+0000 7fb51309ab80 1 Processor -- start 2024-03-23T17:50:13.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.873+0000 7fb51309ab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561662595780 0x56166259f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:13.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.873+0000 7fb51309ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616625960e0 0x56166259faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:13.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.873+0000 7fb51309ab80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561662596f80 0x5616625a6b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:13.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.873+0000 7fb51309ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x561662482d30 con 0x5616625960e0 2024-03-23T17:50:13.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.873+0000 7fb51309ab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616624829d0 con 0x561662595780 2024-03-23T17:50:13.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.873+0000 7fb51309ab80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x56166236a650 con 0x561662596f80 2024-03-23T17:50:13.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.873+0000 7fb51309ab80 10 client.0 _ll_register_callbacks cb 0x56166248c420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:50:13.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.874+0000 7fb4f7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616625960e0 0x56166259faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:13.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.874+0000 7fb4f7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616625960e0 0x56166259faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38348/0 (socket says 172.21.15.86:38348) 2024-03-23T17:50:13.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.874+0000 7fb4f7e00700 1 -- 172.21.15.86:0/71532651 learned_addr learned my addr 172.21.15.86:0/71532651 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:13.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.874+0000 7fb4fcc00700 1 --2- 172.21.15.86:0/71532651 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561662595780 0x56166259f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:13.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.874+0000 7fb4fd600700 1 --2- 172.21.15.86:0/71532651 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561662596f80 0x5616625a6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:13.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.874+0000 7fb4f7e00700 1 -- 172.21.15.86:0/71532651 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561662596f80 msgr2=0x5616625a6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:50:13.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.874+0000 7fb4f7e00700 1 --2- 172.21.15.86:0/71532651 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561662596f80 0x5616625a6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.874+0000 7fb4f7e00700 1 -- 172.21.15.86:0/71532651 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561662595780 msgr2=0x56166259f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:50:13.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.874+0000 7fb4f7e00700 1 --2- 172.21.15.86:0/71532651 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561662595780 0x56166259f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.874+0000 7fb4f7e00700 1 -- 172.21.15.86:0/71532651 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4f8109b10 con 0x5616625960e0 2024-03-23T17:50:13.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.874+0000 7fb4fd600700 1 --2- 172.21.15.86:0/71532651 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561662596f80 0x5616625a6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:50:13.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.874+0000 7fb4f7e00700 1 --2- 172.21.15.86:0/71532651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616625960e0 0x56166259faf0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb4e400fb70 tx=0x7fb4e40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef528f2bed24ae57 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:13.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.874+0000 7fb4de000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-23T17:50:13.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.874+0000 7fb4de000700 1 -- 172.21.15.86:0/71532651 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e401ecf0 con 0x5616625960e0 2024-03-23T17:50:13.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.874+0000 7fb4de000700 1 -- 172.21.15.86:0/71532651 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fb4e40283f0 con 0x5616625960e0 2024-03-23T17:50:13.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.874+0000 7fb51309ab80 1 -- 172.21.15.86:0/71532651 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5616624a7330 con 0x5616625960e0 2024-03-23T17:50:13.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.875+0000 7fb4de000700 1 -- 172.21.15.86:0/71532651 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb4e401ee50 con 0x5616625960e0 2024-03-23T17:50:13.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.875+0000 7fb51309ab80 10 client.15192 fetch_fsmap learned FSMap version 27 2024-03-23T17:50:13.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.875+0000 7fb51309ab80 1 -- 172.21.15.86:0/71532651 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561662314b10 con 0x5616625960e0 2024-03-23T17:50:13.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.875+0000 7fb4de000700 1 -- 172.21.15.86:0/71532651 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5616624a7330 con 0x5616625960e0 2024-03-23T17:50:13.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.875+0000 7fb4de000700 1 -- 172.21.15.86:0/71532651 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb4e40346b0 con 0x5616625960e0 2024-03-23T17:50:13.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.875+0000 7fb51309ab80 10 client.15192 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T17:50:13.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.875+0000 7fb51309ab80 10 client.15192 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-23T17:50:13.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.875+0000 7fb51309ab80 20 client.15192 populate_metadata read hostname 'smithi086' 2024-03-23T17:50:13.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.875+0000 7fb51309ab80 1 --2- 172.21.15.86:0/71532651 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5616625a9ce0 0x5616625afb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:13.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.875+0000 7fb51309ab80 4 client.15192 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:50:13.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.875+0000 7fb51309ab80 1 -- 172.21.15.86:0/71532651 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5616624df6b0 con 0x5616625a9ce0 2024-03-23T17:50:13.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.876+0000 7fb4fcc00700 1 --2- 172.21.15.86:0/71532651 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5616625a9ce0 0x5616625afb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:13.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.876+0000 7fb4fcc00700 1 --2- 172.21.15.86:0/71532651 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5616625a9ce0 0x5616625afb20 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8298cc5170bad3ea server_cookie=e2a4dbdb602c58eb in_seq=0 out_seq=0 2024-03-23T17:50:13.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.876+0000 7fb4de000700 10 client.15192 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:50:13.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.877+0000 7fb4de000700 1 -- 172.21.15.86:0/71532651 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5616624df6b0 con 0x5616625a9ce0 2024-03-23T17:50:13.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.877+0000 7fb4de000700 10 client.15192 handle_command_reply: tid=0 2024-03-23T17:50:13.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.877+0000 7fb51309ab80 1 client.15192 shutdown 2024-03-23T17:50:13.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.877+0000 7fb51309ab80 1 -- 172.21.15.86:0/71532651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616625960e0 msgr2=0x56166259faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:13.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.877+0000 7fb51309ab80 1 --2- 172.21.15.86:0/71532651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616625960e0 0x56166259faf0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb4e400fb70 tx=0x7fb4e40047c0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.877+0000 7fb51309ab80 1 -- 172.21.15.86:0/71532651 shutdown_connections 2024-03-23T17:50:13.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.877+0000 7fb51309ab80 1 --2- 172.21.15.86:0/71532651 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5616625a9ce0 0x5616625afb20 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.877+0000 7fb51309ab80 1 --2- 172.21.15.86:0/71532651 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561662596f80 0x5616625a6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.877+0000 7fb51309ab80 1 --2- 172.21.15.86:0/71532651 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561662595780 0x56166259f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.877+0000 7fb4de000700 0 client.15192 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:50:13.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.877+0000 7fb51309ab80 1 --2- 172.21.15.86:0/71532651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616625960e0 0x56166259faf0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.877+0000 7fb51309ab80 1 -- 172.21.15.86:0/71532651 >> 172.21.15.86:0/71532651 conn(0x5616624f4650 msgr2=0x561662594120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:13.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.877+0000 7fb51309ab80 1 -- 172.21.15.86:0/71532651 shutdown_connections 2024-03-23T17:50:13.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.877+0000 7fb51309ab80 1 -- 172.21.15.86:0/71532651 wait complete. 2024-03-23T17:50:13.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.877+0000 7fb51309ab80 20 client.15192 trim_cache size 0 max 16384 2024-03-23T17:50:13.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.895+0000 7fb51309ab80 1 Processor -- start 2024-03-23T17:50:13.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.895+0000 7fb51309ab80 1 -- start start 2024-03-23T17:50:13.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.895+0000 7fb51309ab80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561662595780 0x5616625a4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:13.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.895+0000 7fb51309ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616625960e0 0x56166259eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:13.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.895+0000 7fb51309ab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561662596f80 0x561662597360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:13.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.895+0000 7fb51309ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616624a7330 con 0x5616625960e0 2024-03-23T17:50:13.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.895+0000 7fb51309ab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616625964c0 con 0x561662596f80 2024-03-23T17:50:13.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.895+0000 7fb51309ab80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x561662595420 con 0x561662595780 2024-03-23T17:50:13.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.896+0000 7fb4f7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616625960e0 0x56166259eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:13.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.896+0000 7fb4f7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616625960e0 0x56166259eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38362/0 (socket says 172.21.15.86:38362) 2024-03-23T17:50:13.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.896+0000 7fb4f7e00700 1 -- 172.21.15.86:0/2256392311 learned_addr learned my addr 172.21.15.86:0/2256392311 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:13.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.896+0000 7fb4fd600700 1 --2- 172.21.15.86:0/2256392311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561662596f80 0x561662597360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:13.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.896+0000 7fb4fcc00700 1 --2- 172.21.15.86:0/2256392311 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561662595780 0x5616625a4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:13.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.896+0000 7fb4f7e00700 1 -- 172.21.15.86:0/2256392311 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561662595780 msgr2=0x5616625a4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:50:13.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.896+0000 7fb4f7e00700 1 --2- 172.21.15.86:0/2256392311 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561662595780 0x5616625a4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.896+0000 7fb4f7e00700 1 -- 172.21.15.86:0/2256392311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561662596f80 msgr2=0x561662597360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:50:13.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.896+0000 7fb4f7e00700 1 --2- 172.21.15.86:0/2256392311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561662596f80 0x561662597360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.896+0000 7fb4f7e00700 1 -- 172.21.15.86:0/2256392311 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5616624ace50 con 0x5616625960e0 2024-03-23T17:50:13.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.896+0000 7fb4fd600700 1 --2- 172.21.15.86:0/2256392311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561662596f80 0x561662597360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:50:13.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.896+0000 7fb4f7e00700 1 --2- 172.21.15.86:0/2256392311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616625960e0 0x56166259eac0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb4e4035fd0 tx=0x7fb4e4009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfffe5e6f2a07be8 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:13.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.896+0000 7fb4ea600700 1 -- 172.21.15.86:0/2256392311 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e400e3b0 con 0x5616625960e0 2024-03-23T17:50:13.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.896+0000 7fb4ea600700 1 -- 172.21.15.86:0/2256392311 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb4e400e510 con 0x5616625960e0 2024-03-23T17:50:13.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.896+0000 7fb4ea600700 1 -- 172.21.15.86:0/2256392311 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e400ea50 con 0x5616625960e0 2024-03-23T17:50:13.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.897+0000 7fb51309ab80 1 -- 172.21.15.86:0/2256392311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616625960e0 msgr2=0x56166259eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:13.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.897+0000 7fb51309ab80 1 --2- 172.21.15.86:0/2256392311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616625960e0 0x56166259eac0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb4e4035fd0 tx=0x7fb4e4009f60 comp rx=0 tx=0).stop 2024-03-23T17:50:13.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.897+0000 7fb51309ab80 1 -- 172.21.15.86:0/2256392311 shutdown_connections 2024-03-23T17:50:13.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.897+0000 7fb51309ab80 1 --2- 172.21.15.86:0/2256392311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561662596f80 0x561662597360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.897+0000 7fb51309ab80 1 --2- 172.21.15.86:0/2256392311 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561662595780 0x5616625a4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.897+0000 7fb51309ab80 1 --2- 172.21.15.86:0/2256392311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616625960e0 0x56166259eac0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.897+0000 7fb51309ab80 1 -- 172.21.15.86:0/2256392311 >> 172.21.15.86:0/2256392311 conn(0x5616624f4650 msgr2=0x5616625b3b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:13.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.897+0000 7fb51309ab80 1 -- 172.21.15.86:0/2256392311 shutdown_connections 2024-03-23T17:50:13.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.897+0000 7fb51309ab80 1 -- 172.21.15.86:0/2256392311 wait complete. 2024-03-23T17:50:13.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.897+0000 7fb51309ab80 1 -- start start 2024-03-23T17:50:13.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.898+0000 7fb51309ab80 1 Processor -- start 2024-03-23T17:50:13.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.898+0000 7fb51309ab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561662595780 0x5616625a32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:13.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.898+0000 7fb51309ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616625960e0 0x5616625a3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:13.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.898+0000 7fb51309ab80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561662596f80 0x5616625a3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:13.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.898+0000 7fb51309ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616625a4ea0 con 0x5616625960e0 2024-03-23T17:50:13.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.898+0000 7fb51309ab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616625fce00 con 0x561662595780 2024-03-23T17:50:13.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.898+0000 7fb51309ab80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616625fcf70 con 0x561662596f80 2024-03-23T17:50:13.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.898+0000 7fb51309ab80 10 client.0 _ll_register_callbacks cb 0x56166248c420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:50:13.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.898+0000 7fb4f7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616625960e0 0x5616625a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:13.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.898+0000 7fb4f7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616625960e0 0x5616625a3810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38368/0 (socket says 172.21.15.86:38368) 2024-03-23T17:50:13.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.898+0000 7fb4f7e00700 1 -- 172.21.15.86:0/3951138148 learned_addr learned my addr 172.21.15.86:0/3951138148 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:13.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.898+0000 7fb4fcc00700 1 --2- 172.21.15.86:0/3951138148 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561662595780 0x5616625a32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:13.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.899+0000 7fb4fd600700 1 --2- 172.21.15.86:0/3951138148 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561662596f80 0x5616625a3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:13.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.899+0000 7fb4f7e00700 1 -- 172.21.15.86:0/3951138148 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561662596f80 msgr2=0x5616625a3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:50:13.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.899+0000 7fb4f7e00700 1 --2- 172.21.15.86:0/3951138148 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561662596f80 0x5616625a3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.899+0000 7fb4f7e00700 1 -- 172.21.15.86:0/3951138148 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561662595780 msgr2=0x5616625a32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:50:13.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.899+0000 7fb4f7e00700 1 --2- 172.21.15.86:0/3951138148 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561662595780 0x5616625a32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.899+0000 7fb4f7e00700 1 -- 172.21.15.86:0/3951138148 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4f0084040 con 0x5616625960e0 2024-03-23T17:50:13.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.899+0000 7fb4fcc00700 1 --2- 172.21.15.86:0/3951138148 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561662595780 0x5616625a32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:50:13.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.899+0000 7fb4f7e00700 1 --2- 172.21.15.86:0/3951138148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616625960e0 0x5616625a3810 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb4e400a010 tx=0x7fb4e40346b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa91c1bb8448c5db server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:13.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.899+0000 7fb4de000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-23T17:50:13.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.899+0000 7fb4de000700 1 -- 172.21.15.86:0/3951138148 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e4005030 con 0x5616625960e0 2024-03-23T17:50:13.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.899+0000 7fb4de000700 1 -- 172.21.15.86:0/3951138148 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fb4e4005190 con 0x5616625960e0 2024-03-23T17:50:13.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.899+0000 7fb51309ab80 1 -- 172.21.15.86:0/3951138148 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb4ec01e040 con 0x5616625960e0 2024-03-23T17:50:13.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.899+0000 7fb4de000700 1 -- 172.21.15.86:0/3951138148 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb4e4005350 con 0x5616625960e0 2024-03-23T17:50:13.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.899+0000 7fb51309ab80 10 client.15198 fetch_fsmap learned FSMap version 27 2024-03-23T17:50:13.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.899+0000 7fb51309ab80 1 -- 172.21.15.86:0/3951138148 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561662314b10 con 0x5616625960e0 2024-03-23T17:50:13.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.899+0000 7fb4de000700 1 -- 172.21.15.86:0/3951138148 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb4ec01e040 con 0x5616625960e0 2024-03-23T17:50:13.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.900+0000 7fb4de000700 1 -- 172.21.15.86:0/3951138148 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb4e4046020 con 0x5616625960e0 2024-03-23T17:50:13.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.900+0000 7fb51309ab80 10 client.15198 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T17:50:13.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.900+0000 7fb51309ab80 10 client.15198 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-23T17:50:13.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.900+0000 7fb51309ab80 20 client.15198 populate_metadata read hostname 'smithi086' 2024-03-23T17:50:13.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.900+0000 7fb51309ab80 1 --2- 172.21.15.86:0/3951138148 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x561662601a40 0x5616625fe3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:13.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.900+0000 7fb51309ab80 4 client.15198 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-23T17:50:13.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.900+0000 7fb51309ab80 1 -- 172.21.15.86:0/3951138148 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "status"}) v1 -- 0x561662335ca0 con 0x561662601a40 2024-03-23T17:50:13.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.900+0000 7fb4fcc00700 1 --2- 172.21.15.86:0/3951138148 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x561662601a40 0x5616625fe3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:13.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.901+0000 7fb4fcc00700 1 --2- 172.21.15.86:0/3951138148 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x561662601a40 0x5616625fe3b0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71e40a551d3638a3 server_cookie=ea3595ef1f3a6676 in_seq=0 out_seq=0 2024-03-23T17:50:13.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.901+0000 7fb4de000700 10 client.15198 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:50:13.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.901+0000 7fb4de000700 1 -- 172.21.15.86:0/3951138148 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x561662335ca0 con 0x561662601a40 2024-03-23T17:50:13.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.901+0000 7fb4de000700 10 client.15198 handle_command_reply: tid=0 2024-03-23T17:50:13.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.901+0000 7fb51309ab80 1 client.15198 shutdown 2024-03-23T17:50:13.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.901+0000 7fb51309ab80 1 -- 172.21.15.86:0/3951138148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616625960e0 msgr2=0x5616625a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:13.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.901+0000 7fb51309ab80 1 --2- 172.21.15.86:0/3951138148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616625960e0 0x5616625a3810 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb4e400a010 tx=0x7fb4e40346b0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.901+0000 7fb51309ab80 1 -- 172.21.15.86:0/3951138148 shutdown_connections 2024-03-23T17:50:13.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.901+0000 7fb51309ab80 1 --2- 172.21.15.86:0/3951138148 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x561662601a40 0x5616625fe3b0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.901+0000 7fb51309ab80 1 --2- 172.21.15.86:0/3951138148 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561662596f80 0x5616625a3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.901+0000 7fb51309ab80 1 --2- 172.21.15.86:0/3951138148 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561662595780 0x5616625a32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.901+0000 7fb4de000700 0 client.15198 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:50:13.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.901+0000 7fb51309ab80 1 --2- 172.21.15.86:0/3951138148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616625960e0 0x5616625a3810 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.901+0000 7fb51309ab80 1 -- 172.21.15.86:0/3951138148 >> 172.21.15.86:0/3951138148 conn(0x5616624f4650 msgr2=0x561662594120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:13.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.901+0000 7fb51309ab80 1 -- 172.21.15.86:0/3951138148 shutdown_connections 2024-03-23T17:50:13.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.901+0000 7fb51309ab80 1 -- 172.21.15.86:0/3951138148 wait complete. 2024-03-23T17:50:13.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.901+0000 7fb51309ab80 20 client.15198 trim_cache size 0 max 16384 2024-03-23T17:50:13.907 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-23T17:50:13.907 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "9d76c9e8-e93c-11ee-95cd-87774f69a715", 2024-03-23T17:50:13.907 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 0, 2024-03-23T17:50:13.907 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24563, 2024-03-23T17:50:13.907 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-03-23T17:50:13.907 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-03-23T17:50:13.907 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-03-23T17:50:13.907 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 77.696427978000003, 2024-03-23T17:50:13.908 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 27, 2024-03-23T17:50:13.908 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 82, 2024-03-23T17:50:13.908 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 80, 2024-03-23T17:50:13.908 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 84.309486901 2024-03-23T17:50:13.908 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-23T17:50:13.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.902+0000 7fb4ff600700 1 -- 172.21.15.86:0/1738069210 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fb4e00a11b0 msgr2=0x7fb4e00a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:13.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.902+0000 7fb4ff600700 1 --2- 172.21.15.86:0/1738069210 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fb4e00a11b0 0x7fb4e00a3670 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb4e4009e00 tx=0x7fb4e4011720 comp rx=0 tx=0).stop 2024-03-23T17:50:13.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.902+0000 7fb4ff600700 1 -- 172.21.15.86:0/1738069210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4f8102910 msgr2=0x7fb4f81a04e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:13.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.902+0000 7fb4ff600700 1 --2- 172.21.15.86:0/1738069210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4f8102910 0x7fb4f81a04e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb4f000ef30 tx=0x7fb4f000c3a0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.902+0000 7fb4ff600700 1 -- 172.21.15.86:0/1738069210 shutdown_connections 2024-03-23T17:50:13.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.902+0000 7fb4ff600700 1 --2- 172.21.15.86:0/1738069210 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fb4e00a11b0 0x7fb4e00a3670 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.902+0000 7fb4ff600700 1 --2- 172.21.15.86:0/1738069210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4f8102910 0x7fb4f81a04e0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.902+0000 7fb4ff600700 1 --2- 172.21.15.86:0/1738069210 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4f8100d80 0x7fb4f8198f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.902+0000 7fb4ff600700 1 --2- 172.21.15.86:0/1738069210 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4f8101f70 0x7fb4f81994c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:13.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.902+0000 7fb4ff600700 1 -- 172.21.15.86:0/1738069210 >> 172.21.15.86:0/1738069210 conn(0x7fb4f80fc2f0 msgr2=0x7fb4f8072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:13.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.902+0000 7fb4ff600700 1 -- 172.21.15.86:0/1738069210 shutdown_connections 2024-03-23T17:50:13.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:13.902+0000 7fb4ff600700 1 -- 172.21.15.86:0/1738069210 wait complete. 2024-03-23T17:50:13.920 INFO:teuthology.run_tasks:Running task kclient... 2024-03-23T17:50:13.943 INFO:tasks.kclient:Mounting kernel clients... 2024-03-23T17:50:13.943 INFO:tasks.kclient:config is {'client.0': {}} 2024-03-23T17:50:13.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T17:50:13.944 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-03-23T17:50:13.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T17:50:13.986 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link delete ceph-brx 2024-03-23T17:50:14.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:13 smithi092 ceph-mon[79089]: pgmap v274: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-23T17:50:14.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:13 smithi152 ceph-mon[47949]: pgmap v274: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-23T17:50:14.096 INFO:teuthology.orchestra.run.smithi086.stderr:Cannot find device "ceph-brx" 2024-03-23T17:50:14.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T17:50:14.099 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'nowsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-03-23T17:50:14.099 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-23T17:50:14.099 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-23T17:50:14.100 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph fs ls 2024-03-23T17:50:14.453 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T17:50:15.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:15.545+0000 7f1daf800700 1 -- 172.21.15.86:0/2565991847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1da8101820 msgr2=0x7f1da810dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:15.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:15.545+0000 7f1daf800700 1 --2- 172.21.15.86:0/2565991847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1da8101820 0x7f1da810dc10 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f1da000b0d0 tx=0x7f1da000b3e0 comp rx=0 tx=0).stop 2024-03-23T17:50:15.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:15.545+0000 7f1daf800700 1 -- 172.21.15.86:0/2565991847 shutdown_connections 2024-03-23T17:50:15.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:15.545+0000 7f1daf800700 1 --2- 172.21.15.86:0/2565991847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1da8101820 0x7f1da810dc10 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:15.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:15.545+0000 7f1daf800700 1 --2- 172.21.15.86:0/2565991847 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1da8106ef0 0x7f1da81072d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:15.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:15.545+0000 7f1daf800700 1 --2- 172.21.15.86:0/2565991847 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1da8100e60 0x7f1da81012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:15.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:15.545+0000 7f1daf800700 1 -- 172.21.15.86:0/2565991847 >> 172.21.15.86:0/2565991847 conn(0x7f1da8077fa0 msgr2=0x7f1da80783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:15.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:15.545+0000 7f1daf800700 1 -- 172.21.15.86:0/2565991847 shutdown_connections 2024-03-23T17:50:15.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:15.545+0000 7f1daf800700 1 -- 172.21.15.86:0/2565991847 wait complete. 2024-03-23T17:50:15.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:15.546+0000 7f1daf800700 1 Processor -- start 2024-03-23T17:50:15.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:15.546+0000 7f1daf800700 1 -- start start 2024-03-23T17:50:15.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:15.546+0000 7f1daf800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1da8100e60 0x7f1da81994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:15.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:15.546+0000 7f1daf800700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1da8101820 0x7f1da8199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:15.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:15.547+0000 7f1daf800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1da8106ef0 0x7f1da819dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:15.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:15.547+0000 7f1daf800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da819e370 con 0x7f1da8106ef0 2024-03-23T17:50:15.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:15.547+0000 7f1daf800700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da819e4e0 con 0x7f1da8100e60 2024-03-23T17:50:15.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:15.547+0000 7f1daf800700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da819e6a0 con 0x7f1da8101820 2024-03-23T17:50:15.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:15.547+0000 7f1dace00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1da8100e60 0x7f1da81994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:15.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:15.547+0000 7f1dad800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1da8106ef0 0x7f1da819dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:15.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:15.547+0000 7f1dad800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1da8106ef0 0x7f1da819dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34962/0 (socket says 172.21.15.86:34962) 2024-03-23T17:50:15.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:15.547+0000 7f1dad800700 1 -- 172.21.15.86:0/2175457104 learned_addr learned my addr 172.21.15.86:0/2175457104 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:15.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:15.547+0000 7f1da7e00700 1 --2- 172.21.15.86:0/2175457104 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1da8101820 0x7f1da8199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:15.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:15.547+0000 7f1dad800700 1 -- 172.21.15.86:0/2175457104 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1da8101820 msgr2=0x7f1da8199a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:15.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:15.547+0000 7f1dad800700 1 --2- 172.21.15.86:0/2175457104 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1da8101820 0x7f1da8199a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:15.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:15.547+0000 7f1dad800700 1 -- 172.21.15.86:0/2175457104 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1da8100e60 msgr2=0x7f1da81994e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:15.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:15.547+0000 7f1dad800700 1 --2- 172.21.15.86:0/2175457104 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1da8100e60 0x7f1da81994e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:15.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:15.547+0000 7f1dad800700 1 -- 172.21.15.86:0/2175457104 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1da0009d00 con 0x7f1da8106ef0 2024-03-23T17:50:15.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:15.547+0000 7f1da7e00700 1 --2- 172.21.15.86:0/2175457104 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1da8101820 0x7f1da8199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:50:15.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:15.547+0000 7f1dad800700 1 --2- 172.21.15.86:0/2175457104 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1da8106ef0 0x7f1da819dd60 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1da000bcd0 tx=0x7f1da000bd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:15.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:15.548+0000 7f1da5600700 1 -- 172.21.15.86:0/2175457104 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da000e040 con 0x7f1da8106ef0 2024-03-23T17:50:15.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:15.548+0000 7f1da5600700 1 -- 172.21.15.86:0/2175457104 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1da0005070 con 0x7f1da8106ef0 2024-03-23T17:50:15.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:15.548+0000 7f1da5600700 1 -- 172.21.15.86:0/2175457104 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da001b680 con 0x7f1da8106ef0 2024-03-23T17:50:15.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:15.548+0000 7f1daf800700 1 -- 172.21.15.86:0/2175457104 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1da819e920 con 0x7f1da8106ef0 2024-03-23T17:50:15.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:15.548+0000 7f1daf800700 1 -- 172.21.15.86:0/2175457104 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1da81a6010 con 0x7f1da8106ef0 2024-03-23T17:50:15.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:15.549+0000 7f1daf800700 1 -- 172.21.15.86:0/2175457104 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1da81024f0 con 0x7f1da8106ef0 2024-03-23T17:50:15.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:15.551+0000 7f1da5600700 1 -- 172.21.15.86:0/2175457104 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f1da0003a00 con 0x7f1da8106ef0 2024-03-23T17:50:15.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:15.552+0000 7f1da5600700 1 --2- 172.21.15.86:0/2175457104 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f1d900a1230 0x7f1d900a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:15.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:15.552+0000 7f1da5600700 1 -- 172.21.15.86:0/2175457104 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f1da00cd770 con 0x7f1da8106ef0 2024-03-23T17:50:15.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:15.552+0000 7f1dace00700 1 --2- 172.21.15.86:0/2175457104 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f1d900a1230 0x7f1d900a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:15.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:15.553+0000 7f1dace00700 1 --2- 172.21.15.86:0/2175457104 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f1d900a1230 0x7f1d900a36f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c005950 tx=0x7f1d9c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:15.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:15.556+0000 7f1da5600700 1 -- 172.21.15.86:0/2175457104 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f1da0096e00 con 0x7f1da8106ef0 2024-03-23T17:50:15.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:15.807+0000 7f1daf800700 1 -- 172.21.15.86:0/2175457104 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1da804ea80 con 0x7f1da8106ef0 2024-03-23T17:50:15.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:15.808+0000 7f1da5600700 1 -- 172.21.15.86:0/2175457104 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1da0003cb0 con 0x7f1da8106ef0 2024-03-23T17:50:15.809 INFO:teuthology.orchestra.run.smithi086.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-23T17:50:15.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:15.810+0000 7f1daf800700 1 -- 172.21.15.86:0/2175457104 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f1d900a1230 msgr2=0x7f1d900a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:15.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:15.810+0000 7f1daf800700 1 --2- 172.21.15.86:0/2175457104 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f1d900a1230 0x7f1d900a36f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c005950 tx=0x7f1d9c00b410 comp rx=0 tx=0).stop 2024-03-23T17:50:15.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:15.810+0000 7f1daf800700 1 -- 172.21.15.86:0/2175457104 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1da8106ef0 msgr2=0x7f1da819dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:15.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:15.810+0000 7f1daf800700 1 --2- 172.21.15.86:0/2175457104 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1da8106ef0 0x7f1da819dd60 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1da000bcd0 tx=0x7f1da000bd00 comp rx=0 tx=0).stop 2024-03-23T17:50:15.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:15.810+0000 7f1daf800700 1 -- 172.21.15.86:0/2175457104 shutdown_connections 2024-03-23T17:50:15.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:15.810+0000 7f1daf800700 1 --2- 172.21.15.86:0/2175457104 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f1d900a1230 0x7f1d900a36f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:15.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:15.810+0000 7f1daf800700 1 --2- 172.21.15.86:0/2175457104 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1da8106ef0 0x7f1da819dd60 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:15.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:15.810+0000 7f1daf800700 1 --2- 172.21.15.86:0/2175457104 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1da8100e60 0x7f1da81994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:15.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:15.810+0000 7f1daf800700 1 --2- 172.21.15.86:0/2175457104 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1da8101820 0x7f1da8199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:15.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:15.810+0000 7f1daf800700 1 -- 172.21.15.86:0/2175457104 >> 172.21.15.86:0/2175457104 conn(0x7f1da8077fa0 msgr2=0x7f1da810c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:15.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:15.810+0000 7f1daf800700 1 -- 172.21.15.86:0/2175457104 shutdown_connections 2024-03-23T17:50:15.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:15.810+0000 7f1daf800700 1 -- 172.21.15.86:0/2175457104 wait complete. 2024-03-23T17:50:16.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:15 smithi086 ceph-mon[38221]: pgmap v275: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-23T17:50:16.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:15 smithi092 ceph-mon[79089]: pgmap v275: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-23T17:50:16.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:15 smithi152 ceph-mon[47949]: pgmap v275: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-23T17:50:16.200 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-23T17:50:16.200 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-23T17:50:16.200 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2024-03-23T17:50:16.200 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-23T17:50:16.201 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-23T17:50:16.201 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-23T17:50:16.201 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-23T17:50:16.201 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-23T17:50:16.201 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-23T17:50:16.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T17:50:16.201 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2024-03-23T17:50:16.246 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-23T17:50:16.246 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-23T17:50:16.246 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-03-23T17:50:16.246 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-23T17:50:16.246 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-03-23T17:50:16.246 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-23T17:50:16.246 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T17:50:16.246 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2024-03-23T17:50:16.247 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2024-03-23T17:50:16.247 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T17:50:16.247 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2024-03-23T17:50:16.247 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2024-03-23T17:50:16.247 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T17:50:16.247 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2024-03-23T17:50:16.247 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-23T17:50:16.247 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2024-03-23T17:50:16.247 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-23T17:50:16.247 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 41739sec preferred_lft 41739sec 2024-03-23T17:50:16.247 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2024-03-23T17:50:16.247 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-03-23T17:50:16.248 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-23T17:50:16.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T17:50:16.249 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec bash -c ' 2024-03-23T17:50:16.249 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-03-23T17:50:16.249 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link add name ceph-brx type bridge 2024-03-23T17:50:16.249 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip addr flush dev ceph-brx 2024-03-23T17:50:16.249 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set ceph-brx up 2024-03-23T17:50:16.249 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-23T17:50:16.249 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-03-23T17:50:16.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T17:50:16.695 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-23T17:50:16.760 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-03-23T17:50:16.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T17:50:16.764 DEBUG:teuthology.orchestra.run.smithi086:> route 2024-03-23T17:50:16.825 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2024-03-23T17:50:16.826 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-23T17:50:16.826 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-23T17:50:16.826 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-23T17:50:16.826 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-23T17:50:16.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T17:50:16.827 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec bash -c ' 2024-03-23T17:50:16.827 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-03-23T17:50:16.827 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-23T17:50:16.827 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-23T17:50:16.827 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-23T17:50:16.827 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-03-23T17:50:16.923 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:16 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/2175457104' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-23T17:50:17.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:16 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/2175457104' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-23T17:50:17.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:16 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/2175457104' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-23T17:50:17.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T17:50:17.527 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-03-23T17:50:17.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T17:50:17.570 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list-id 2024-03-23T17:50:17.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T17:50:17.653 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec bash -c ' 2024-03-23T17:50:17.653 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-03-23T17:50:17.653 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-23T17:50:17.653 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-23T17:50:17.653 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-03-23T17:50:17.790 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:17 smithi086 ceph-mon[38221]: pgmap v276: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-23T17:50:17.831 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-23T17:50:17.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T17:50:17.831 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec bash -c ' 2024-03-23T17:50:17.832 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-03-23T17:50:17.832 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-23T17:50:17.832 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-23T17:50:17.832 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-23T17:50:17.832 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-23T17:50:17.832 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-23T17:50:17.832 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-03-23T17:50:18.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:17 smithi092 ceph-mon[79089]: pgmap v276: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-23T17:50:18.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:17 smithi152 ceph-mon[47949]: pgmap v276: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-23T17:50:18.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T17:50:18.113 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec bash -c ' 2024-03-23T17:50:18.113 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-03-23T17:50:18.113 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.0 up 2024-03-23T17:50:18.113 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set dev brx.0 master ceph-brx 2024-03-23T17:50:18.113 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-03-23T17:50:18.278 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-23T17:50:18.278 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-23T17:50:18.320 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-23T17:50:18.320 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-23T17:50:18.320 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-23T17:50:18.401 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-03-23T17:50:18.401 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-03-23T17:50:18.401 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc 2024-03-23T17:50:18.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T17:50:18.401 DEBUG:teuthology.orchestra.run.smithi086:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync 2024-03-23T17:50:18.541 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync 2024-03-23T17:50:18.541 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync". 2024-03-23T17:50:18.541 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-03-23T17:50:19.149 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync 2024-03-23T17:50:19.149 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync". 2024-03-23T17:50:19.149 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-03-23T17:50:19.149 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.92:3300,172.21.15.152:3300" 2024-03-23T17:50:19.149 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.86:3300,172.21.15.92:3300,172.21.15.152:3300:/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc 2024-03-23T17:50:19.150 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync,key=0,fsid=9d76c9e8-e93c-11ee-95cd-87774f69a715" will pass to kernel 2024-03-23T17:50:19.155 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-23T17:50:19.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T17:50:19.156 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-23T17:50:19.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T17:50:19.229 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-23T17:50:19.343 INFO:teuthology.orchestra.run.smithi086.stdout:client25258 2024-03-23T17:50:19.343 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-23T17:50:19.343 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T17:50:19.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T17:50:19.386 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-23T17:50:19.533 INFO:teuthology.orchestra.run.smithi086.stdout:client25258 2024-03-23T17:50:19.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T17:50:19.533 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-23T17:50:19.625 INFO:teuthology.orchestra.run.smithi086.stdout:client25258 2024-03-23T17:50:19.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T17:50:19.625 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715.client25258/status') 2024-03-23T17:50:19.727 INFO:teuthology.orchestra.run.smithi086.stdout:instance: client.25258 (3)172.21.15.86:0/855195935 2024-03-23T17:50:19.728 INFO:teuthology.orchestra.run.smithi086.stdout:blocklisted: false 2024-03-23T17:50:19.728 INFO:teuthology.orchestra.run.smithi086.stderr:0+1 records in 2024-03-23T17:50:19.728 INFO:teuthology.orchestra.run.smithi086.stderr:0+1 records out 2024-03-23T17:50:19.728 INFO:teuthology.orchestra.run.smithi086.stderr:70 bytes copied, 5.2067e-05 s, 1.3 MB/s 2024-03-23T17:50:19.731 INFO:teuthology.run_tasks:Running task exec... 2024-03-23T17:50:19.744 INFO:teuthology.task.exec:Executing custom commands... 2024-03-23T17:50:19.744 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi086.front.sepia.ceph.com 2024-03-23T17:50:19.745 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-03-23T17:50:19.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:19.934+0000 7f8c61c00700 1 -- 172.21.15.86:0/2629968007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c5c101f00 msgr2=0x7f8c5c102360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:19.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:19.934+0000 7f8c61c00700 1 --2- 172.21.15.86:0/2629968007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c5c101f00 0x7f8c5c102360 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f8c480099c0 tx=0x7f8c48009cd0 comp rx=0 tx=0).stop 2024-03-23T17:50:19.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:19.934+0000 7f8c61c00700 1 -- 172.21.15.86:0/2629968007 shutdown_connections 2024-03-23T17:50:19.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:19.934+0000 7f8c61c00700 1 --2- 172.21.15.86:0/2629968007 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c5c1028a0 0x7f8c5c109150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:19.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:19.934+0000 7f8c61c00700 1 --2- 172.21.15.86:0/2629968007 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c5c100d10 0x7f8c5c101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:19.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:19.934+0000 7f8c61c00700 1 --2- 172.21.15.86:0/2629968007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c5c101f00 0x7f8c5c102360 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:19.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:19.934+0000 7f8c61c00700 1 -- 172.21.15.86:0/2629968007 >> 172.21.15.86:0/2629968007 conn(0x7f8c5c0fc280 msgr2=0x7f8c5c0fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:19.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:19.934+0000 7f8c61c00700 1 -- 172.21.15.86:0/2629968007 shutdown_connections 2024-03-23T17:50:19.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:19.934+0000 7f8c61c00700 1 -- 172.21.15.86:0/2629968007 wait complete. 2024-03-23T17:50:19.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:19.935+0000 7f8c61c00700 1 Processor -- start 2024-03-23T17:50:19.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:19.935+0000 7f8c61c00700 1 -- start start 2024-03-23T17:50:19.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:19.936+0000 7f8c61c00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c5c100d10 0x7f8c5c196cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:19.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:19.936+0000 7f8c61c00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c5c101f00 0x7f8c5c197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:19.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:19.936+0000 7f8c61c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c5c1028a0 0x7f8c5c19e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:19.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:19.936+0000 7f8c61c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c5c19e7b0 con 0x7f8c5c1028a0 2024-03-23T17:50:19.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:19.936+0000 7f8c61c00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c5c19e920 con 0x7f8c5c100d10 2024-03-23T17:50:19.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:19.936+0000 7f8c61c00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c5c19eae0 con 0x7f8c5c101f00 2024-03-23T17:50:19.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:19.936+0000 7f8c5b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c5c1028a0 0x7f8c5c19e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:19.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:19.936+0000 7f8c5aa00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c5c100d10 0x7f8c5c196cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:19.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:19.936+0000 7f8c5a000700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c5c101f00 0x7f8c5c197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:19.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:19.936+0000 7f8c5aa00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c5c100d10 0x7f8c5c196cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:45200/0 (socket says 172.21.15.86:45200) 2024-03-23T17:50:19.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:19.936+0000 7f8c5aa00700 1 -- 172.21.15.86:0/2581660283 learned_addr learned my addr 172.21.15.86:0/2581660283 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:19.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:19.936+0000 7f8c5b400700 1 -- 172.21.15.86:0/2581660283 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c5c101f00 msgr2=0x7f8c5c197200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:19.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:19.936+0000 7f8c5b400700 1 --2- 172.21.15.86:0/2581660283 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c5c101f00 0x7f8c5c197200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:19.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:19.936+0000 7f8c5b400700 1 -- 172.21.15.86:0/2581660283 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c5c100d10 msgr2=0x7f8c5c196cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:19.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:19.936+0000 7f8c5b400700 1 --2- 172.21.15.86:0/2581660283 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c5c100d10 0x7f8c5c196cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:19.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:19.936+0000 7f8c5b400700 1 -- 172.21.15.86:0/2581660283 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c480095f0 con 0x7f8c5c1028a0 2024-03-23T17:50:19.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:19.937+0000 7f8c5b400700 1 --2- 172.21.15.86:0/2581660283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c5c1028a0 0x7f8c5c19e1e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f8c5400efc0 tx=0x7f8c5400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:19.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:19.937+0000 7f8c53400700 1 -- 172.21.15.86:0/2581660283 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c54018070 con 0x7f8c5c1028a0 2024-03-23T17:50:19.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:19.937+0000 7f8c61c00700 1 -- 172.21.15.86:0/2581660283 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c5c19ed60 con 0x7f8c5c1028a0 2024-03-23T17:50:19.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:19.937+0000 7f8c53400700 1 -- 172.21.15.86:0/2581660283 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c5400f040 con 0x7f8c5c1028a0 2024-03-23T17:50:19.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:19.937+0000 7f8c53400700 1 -- 172.21.15.86:0/2581660283 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c5400cce0 con 0x7f8c5c1028a0 2024-03-23T17:50:19.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:19.937+0000 7f8c61c00700 1 -- 172.21.15.86:0/2581660283 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c5c19f1b0 con 0x7f8c5c1028a0 2024-03-23T17:50:19.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:19.938+0000 7f8c61c00700 1 -- 172.21.15.86:0/2581660283 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c5c1081e0 con 0x7f8c5c1028a0 2024-03-23T17:50:19.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:19.940+0000 7f8c53400700 1 -- 172.21.15.86:0/2581660283 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f8c54003a00 con 0x7f8c5c1028a0 2024-03-23T17:50:19.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:19.941+0000 7f8c53400700 1 --2- 172.21.15.86:0/2581660283 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f8c400a11b0 0x7f8c400a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:19.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:19.941+0000 7f8c53400700 1 -- 172.21.15.86:0/2581660283 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f8c540cc5c0 con 0x7f8c5c1028a0 2024-03-23T17:50:19.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:19.942+0000 7f8c5aa00700 1 --2- 172.21.15.86:0/2581660283 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f8c400a11b0 0x7f8c400a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:19.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:19.942+0000 7f8c5aa00700 1 --2- 172.21.15.86:0/2581660283 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f8c400a11b0 0x7f8c400a3670 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c198140 tx=0x7f8c4c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:19.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:19.946+0000 7f8c53400700 1 -- 172.21.15.86:0/2581660283 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f8c54095c50 con 0x7f8c5c1028a0 2024-03-23T17:50:19.969 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:19 smithi086 ceph-mon[38221]: pgmap v277: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-23T17:50:20.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:19 smithi092 ceph-mon[79089]: pgmap v277: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-23T17:50:20.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:19 smithi152 ceph-mon[47949]: pgmap v277: 97 pgs: 97 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-23T17:50:20.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:20.201+0000 7f8c61c00700 1 -- 172.21.15.86:0/2581660283 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) v1 -- 0x7f8c5c10c5c0 con 0x7f8c5c1028a0 2024-03-23T17:50:20.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:20.727+0000 7f8c53400700 1 -- 172.21.15.86:0/2581660283 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v28) v1 ==== 117+0+0 (secure 0 0 0) 0x7f8c54099270 con 0x7f8c5c1028a0 2024-03-23T17:50:20.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:20.729+0000 7f8c61c00700 1 -- 172.21.15.86:0/2581660283 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f8c400a11b0 msgr2=0x7f8c400a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:20.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:20.729+0000 7f8c61c00700 1 --2- 172.21.15.86:0/2581660283 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f8c400a11b0 0x7f8c400a3670 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c198140 tx=0x7f8c4c009260 comp rx=0 tx=0).stop 2024-03-23T17:50:20.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:20.729+0000 7f8c61c00700 1 -- 172.21.15.86:0/2581660283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c5c1028a0 msgr2=0x7f8c5c19e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:20.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:20.729+0000 7f8c61c00700 1 --2- 172.21.15.86:0/2581660283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c5c1028a0 0x7f8c5c19e1e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f8c5400efc0 tx=0x7f8c5400c3a0 comp rx=0 tx=0).stop 2024-03-23T17:50:20.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:20.729+0000 7f8c61c00700 1 -- 172.21.15.86:0/2581660283 shutdown_connections 2024-03-23T17:50:20.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:20.729+0000 7f8c61c00700 1 --2- 172.21.15.86:0/2581660283 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f8c400a11b0 0x7f8c400a3670 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:20.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:20.729+0000 7f8c61c00700 1 --2- 172.21.15.86:0/2581660283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c5c1028a0 0x7f8c5c19e1e0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:20.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:20.729+0000 7f8c61c00700 1 --2- 172.21.15.86:0/2581660283 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c5c100d10 0x7f8c5c196cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:20.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:20.729+0000 7f8c61c00700 1 --2- 172.21.15.86:0/2581660283 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c5c101f00 0x7f8c5c197200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:20.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:20.729+0000 7f8c61c00700 1 -- 172.21.15.86:0/2581660283 >> 172.21.15.86:0/2581660283 conn(0x7f8c5c0fc280 msgr2=0x7f8c5c105320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:20.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:20.730+0000 7f8c61c00700 1 -- 172.21.15.86:0/2581660283 shutdown_connections 2024-03-23T17:50:20.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:20.730+0000 7f8c61c00700 1 -- 172.21.15.86:0/2581660283 wait complete. 2024-03-23T17:50:20.748 INFO:teuthology.run_tasks:Running task check-counter... 2024-03-23T17:50:20.759 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-03-23T17:50:20.798 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T17:50:20.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:20.992+0000 7f3ec2c00700 1 -- 172.21.15.86:0/3453696864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ebc101780 msgr2=0x7f3ebc103b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:20.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:20.992+0000 7f3ec2c00700 1 --2- 172.21.15.86:0/3453696864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ebc101780 0x7f3ebc103b70 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3eac009990 tx=0x7f3eac009ca0 comp rx=0 tx=0).stop 2024-03-23T17:50:20.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:20.992+0000 7f3ec2c00700 1 -- 172.21.15.86:0/3453696864 shutdown_connections 2024-03-23T17:50:20.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:20.992+0000 7f3ec2c00700 1 --2- 172.21.15.86:0/3453696864 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ebc1069e0 0x7f3ebc108dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:20.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:20.992+0000 7f3ec2c00700 1 --2- 172.21.15.86:0/3453696864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ebc101780 0x7f3ebc103b70 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:20.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:20.992+0000 7f3ec2c00700 1 --2- 172.21.15.86:0/3453696864 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3ebc1040b0 0x7f3ebc1064a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:20.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:20.992+0000 7f3ec2c00700 1 -- 172.21.15.86:0/3453696864 >> 172.21.15.86:0/3453696864 conn(0x7f3ebc0fb0d0 msgr2=0x7f3ebc0fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:20.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:20.992+0000 7f3ec2c00700 1 -- 172.21.15.86:0/3453696864 shutdown_connections 2024-03-23T17:50:20.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:20.992+0000 7f3ec2c00700 1 -- 172.21.15.86:0/3453696864 wait complete. 2024-03-23T17:50:20.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:20.993+0000 7f3ec2c00700 1 Processor -- start 2024-03-23T17:50:20.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:20.994+0000 7f3ec2c00700 1 -- start start 2024-03-23T17:50:20.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:20.994+0000 7f3ec2c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ebc101780 0x7f3ebc198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:20.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:20.994+0000 7f3ec2c00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3ebc1040b0 0x7f3ebc199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:20.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:20.994+0000 7f3ec2c00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ebc1069e0 0x7f3ebc1a0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:20.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:20.994+0000 7f3ec2c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ebc1a08e0 con 0x7f3ebc101780 2024-03-23T17:50:20.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:20.994+0000 7f3ec2c00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ebc1a0a50 con 0x7f3ebc1069e0 2024-03-23T17:50:20.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:20.994+0000 7f3ec2c00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ebc1a0c10 con 0x7f3ebc1040b0 2024-03-23T17:50:20.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:20.994+0000 7f3ebbe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ebc101780 0x7f3ebc198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:20.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:20.994+0000 7f3ebbe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ebc101780 0x7f3ebc198df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34986/0 (socket says 172.21.15.86:34986) 2024-03-23T17:50:20.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:20.994+0000 7f3ebbe00700 1 -- 172.21.15.86:0/3477703716 learned_addr learned my addr 172.21.15.86:0/3477703716 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:20.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:20.995+0000 7f3ebb400700 1 --2- 172.21.15.86:0/3477703716 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3ebc1040b0 0x7f3ebc199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:20.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:20.995+0000 7f3ec0c00700 1 --2- 172.21.15.86:0/3477703716 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ebc1069e0 0x7f3ebc1a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:20.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:20.995+0000 7f3ebbe00700 1 -- 172.21.15.86:0/3477703716 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3ebc1040b0 msgr2=0x7f3ebc199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:20.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:20.995+0000 7f3ebbe00700 1 --2- 172.21.15.86:0/3477703716 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3ebc1040b0 0x7f3ebc199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:20.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:20.995+0000 7f3ebbe00700 1 -- 172.21.15.86:0/3477703716 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ebc1069e0 msgr2=0x7f3ebc1a0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:20.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:20.995+0000 7f3ebbe00700 1 --2- 172.21.15.86:0/3477703716 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ebc1069e0 0x7f3ebc1a0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:20.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:20.995+0000 7f3ebbe00700 1 -- 172.21.15.86:0/3477703716 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eac0095f0 con 0x7f3ebc101780 2024-03-23T17:50:20.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:20.995+0000 7f3ebb400700 1 --2- 172.21.15.86:0/3477703716 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3ebc1040b0 0x7f3ebc199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:50:20.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:20.995+0000 7f3ec0c00700 1 --2- 172.21.15.86:0/3477703716 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ebc1069e0 0x7f3ebc1a0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:50:20.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:20.995+0000 7f3ebbe00700 1 --2- 172.21.15.86:0/3477703716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ebc101780 0x7f3ebc198df0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3eac00b680 tx=0x7f3eac009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:20.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:20.995+0000 7f3eb8c00700 1 -- 172.21.15.86:0/3477703716 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eac01d070 con 0x7f3ebc101780 2024-03-23T17:50:20.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:20.995+0000 7f3eb8c00700 1 -- 172.21.15.86:0/3477703716 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3eac022470 con 0x7f3ebc101780 2024-03-23T17:50:20.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:20.995+0000 7f3ec2c00700 1 -- 172.21.15.86:0/3477703716 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ebc1a0e90 con 0x7f3ebc101780 2024-03-23T17:50:21.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:20.995+0000 7f3eb8c00700 1 -- 172.21.15.86:0/3477703716 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eac017650 con 0x7f3ebc101780 2024-03-23T17:50:21.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:20.995+0000 7f3ec2c00700 1 -- 172.21.15.86:0/3477703716 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ebc1a1250 con 0x7f3ebc101780 2024-03-23T17:50:21.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:20.996+0000 7f3ec2c00700 1 -- 172.21.15.86:0/3477703716 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ebc108560 con 0x7f3ebc101780 2024-03-23T17:50:21.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:20.998+0000 7f3eb8c00700 1 -- 172.21.15.86:0/3477703716 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f3eac022870 con 0x7f3ebc101780 2024-03-23T17:50:21.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:20.999+0000 7f3eb8c00700 1 --2- 172.21.15.86:0/3477703716 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f3ea00a1230 0x7f3ea00a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:21.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:20.999+0000 7f3eb8c00700 1 -- 172.21.15.86:0/3477703716 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f3eac0cded0 con 0x7f3ebc101780 2024-03-23T17:50:21.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.000+0000 7f3ebb400700 1 --2- 172.21.15.86:0/3477703716 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f3ea00a1230 0x7f3ea00a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:21.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.000+0000 7f3ebb400700 1 --2- 172.21.15.86:0/3477703716 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f3ea00a1230 0x7f3ea00a36f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3ea8009cb0 tx=0x7f3ea8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:21.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.003+0000 7f3eb8c00700 1 -- 172.21.15.86:0/3477703716 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f3eac097560 con 0x7f3ebc101780 2024-03-23T17:50:21.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:20 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/2581660283' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-03-23T17:50:21.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:20 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/2581660283' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-03-23T17:50:21.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:20 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/2581660283' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-03-23T17:50:21.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.245+0000 7f3ec2c00700 1 -- 172.21.15.86:0/3477703716 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3ebc067a10 con 0x7f3ebc101780 2024-03-23T17:50:21.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.246+0000 7f3eb8c00700 1 -- 172.21.15.86:0/3477703716 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22414 (secure 0 0 0) 0x7f3eac09ab80 con 0x7f3ebc101780 2024-03-23T17:50:21.247 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:50:21.247 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":82,"fsid":"9d76c9e8-e93c-11ee-95cd-87774f69a715","created":"2024-03-23T17:42:30.097233+0000","modified":"2024-03-23T17:49:29.773954+0000","last_up_change":"2024-03-23T17:48:18.217601+0000","last_in_change":"2024-03-23T17:48:00.250579+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T17:45:26.848397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T17:48:54.331894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T17:48:55.640693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"656655c6-1c7f-4640-92c4-2f42de820143","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6803","nonce":2423505491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6805","nonce":2423505491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6809","nonce":2423505491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6807","nonce":2423505491}]},"public_addr":"172.21.15.86:6803/2423505491","cluster_addr":"172.21.15.86:6805/2423505491","heartbeat_back_addr":"172.21.15.86:6809/2423505491","heartbeat_front_addr":"172.21.15.86:6807/2423505491","state":["exists","up"]},{"osd":1,"uuid":"3e86dc65-20a6-4de8-b9ce-fcf01f68862b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6801","nonce":3776252597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6803","nonce":3776252597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6807","nonce":3776252597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6805","nonce":3776252597}]},"public_addr":"172.21.15.92:6801/3776252597","cluster_addr":"172.21.15.92:6803/3776252597","heartbeat_back_addr":"172.21.15.92:6807/3776252597","heartbeat_front_addr":"172.21.15.92:6805/3776252597","state":["exists","up"]},{"osd":2,"uuid":"09808786-3bd3-4d11-8c7b-1736119dfbb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6801","nonce":1601265891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6803","nonce":1601265891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6807","nonce":1601265891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6805","nonce":1601265891}]},"public_addr":"172.21.15.152:6801/1601265891","cluster_addr":"172.21.15.152:6803/1601265891","heartbeat_back_addr":"172.21.15.152:6807/1601265891","heartbeat_front_addr":"172.21.15.152:6805/1601265891","state":["exists","up"]},{"osd":3,"uuid":"79c61910-4b53-424c-b150-6f5385a2cd68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6811","nonce":694846466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6813","nonce":694846466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6817","nonce":694846466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6815","nonce":694846466}]},"public_addr":"172.21.15.86:6811/694846466","cluster_addr":"172.21.15.86:6813/694846466","heartbeat_back_addr":"172.21.15.86:6817/694846466","heartbeat_front_addr":"172.21.15.86:6815/694846466","state":["exists","up"]},{"osd":4,"uuid":"a2878035-0179-469f-bbd4-8ce2d3f6bddc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6809","nonce":1422821471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6811","nonce":1422821471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6815","nonce":1422821471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6813","nonce":1422821471}]},"public_addr":"172.21.15.92:6809/1422821471","cluster_addr":"172.21.15.92:6811/1422821471","heartbeat_back_addr":"172.21.15.92:6815/1422821471","heartbeat_front_addr":"172.21.15.92:6813/1422821471","state":["exists","up"]},{"osd":5,"uuid":"837366aa-ce25-42ca-b157-5bbb50159d0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6809","nonce":1039375239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6811","nonce":1039375239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6815","nonce":1039375239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6813","nonce":1039375239}]},"public_addr":"172.21.15.152:6809/1039375239","cluster_addr":"172.21.15.152:6811/1039375239","heartbeat_back_addr":"172.21.15.152:6815/1039375239","heartbeat_front_addr":"172.21.15.152:6813/1039375239","state":["exists","up"]},{"osd":6,"uuid":"61024437-7b91-432f-a1af-6443d0ef5213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6819","nonce":4080452707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6821","nonce":4080452707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6825","nonce":4080452707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6823","nonce":4080452707}]},"public_addr":"172.21.15.86:6819/4080452707","cluster_addr":"172.21.15.86:6821/4080452707","heartbeat_back_addr":"172.21.15.86:6825/4080452707","heartbeat_front_addr":"172.21.15.86:6823/4080452707","state":["exists","up"]},{"osd":7,"uuid":"2684bdae-6b30-4afc-af68-4ce41b3a75b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6817","nonce":4187707622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6819","nonce":4187707622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6823","nonce":4187707622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6821","nonce":4187707622}]},"public_addr":"172.21.15.92:6817/4187707622","cluster_addr":"172.21.15.92:6819/4187707622","heartbeat_back_addr":"172.21.15.92:6823/4187707622","heartbeat_front_addr":"172.21.15.92:6821/4187707622","state":["exists","up"]},{"osd":8,"uuid":"f03bbf0d-1281-4bcf-9773-9854388e65ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6817","nonce":3237612748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6819","nonce":3237612748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6823","nonce":3237612748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6821","nonce":3237612748}]},"public_addr":"172.21.15.152:6817/3237612748","cluster_addr":"172.21.15.152:6819/3237612748","heartbeat_back_addr":"172.21.15.152:6823/3237612748","heartbeat_front_addr":"172.21.15.152:6821/3237612748","state":["exists","up"]},{"osd":9,"uuid":"e7e8b775-99ff-48a4-a23a-91e7ab6622bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6827","nonce":2835218294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6829","nonce":2835218294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6833","nonce":2835218294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6831","nonce":2835218294}]},"public_addr":"172.21.15.86:6827/2835218294","cluster_addr":"172.21.15.86:6829/2835218294","heartbeat_back_addr":"172.21.15.86:6833/2835218294","heartbeat_front_addr":"172.21.15.86:6831/2835218294","state":["exists","up"]},{"osd":10,"uuid":"d9d616cd-a229-44de-8768-02a4415df115","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6825","nonce":4259465100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6827","nonce":4259465100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6831","nonce":4259465100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6829","nonce":4259465100}]},"public_addr":"172.21.15.92:6825/4259465100","cluster_addr":"172.21.15.92:6827/4259465100","heartbeat_back_addr":"172.21.15.92:6831/4259465100","heartbeat_front_addr":"172.21.15.92:6829/4259465100","state":["exists","up"]},{"osd":11,"uuid":"e7c0b865-ccf5-4233-8d22-0d9b6dbe5c97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6825","nonce":194662760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6827","nonce":194662760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6831","nonce":194662760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6829","nonce":194662760}]},"public_addr":"172.21.15.152:6825/194662760","cluster_addr":"172.21.15.152:6827/194662760","heartbeat_back_addr":"172.21.15.152:6831/194662760","heartbeat_front_addr":"172.21.15.152:6829/194662760","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:44:47.269167+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:45:05.363655+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:45:23.192677+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:45:44.156228+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:02.532971+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:20.153295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:41.713888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:59.676516+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:47:17.207087+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:47:38.970316+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:47:57.282525+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:48:15.437606+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/3553500912":"2024-03-24T17:43:28.242492+0000","172.21.15.86:6801/3600278228":"2024-03-24T17:43:28.242492+0000","172.21.15.86:0/887262329":"2024-03-24T17:43:28.242492+0000","172.21.15.86:6800/2313723492":"2024-03-24T17:43:05.043910+0000","172.21.15.86:6800/3600278228":"2024-03-24T17:43:28.242492+0000","172.21.15.86:0/3815705642":"2024-03-24T17:43:05.043910+0000","172.21.15.86:0/1614943062":"2024-03-24T17:43:28.242492+0000","172.21.15.86:6801/2313723492":"2024-03-24T17:43:05.043910+0000","172.21.15.86:0/2416321716":"2024-03-24T17:43:05.043910+0000","172.21.15.86:0/433588188":"2024-03-24T17:43:05.043910+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T17:50:21.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.249+0000 7f3ec2c00700 1 -- 172.21.15.86:0/3477703716 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f3ea00a1230 msgr2=0x7f3ea00a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:21.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.249+0000 7f3ec2c00700 1 --2- 172.21.15.86:0/3477703716 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f3ea00a1230 0x7f3ea00a36f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3ea8009cb0 tx=0x7f3ea8009260 comp rx=0 tx=0).stop 2024-03-23T17:50:21.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.249+0000 7f3ec2c00700 1 -- 172.21.15.86:0/3477703716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ebc101780 msgr2=0x7f3ebc198df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:21.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.249+0000 7f3ec2c00700 1 --2- 172.21.15.86:0/3477703716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ebc101780 0x7f3ebc198df0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3eac00b680 tx=0x7f3eac009580 comp rx=0 tx=0).stop 2024-03-23T17:50:21.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.249+0000 7f3ec2c00700 1 -- 172.21.15.86:0/3477703716 shutdown_connections 2024-03-23T17:50:21.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.249+0000 7f3ec2c00700 1 --2- 172.21.15.86:0/3477703716 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ebc1069e0 0x7f3ebc1a0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:21.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.249+0000 7f3ec2c00700 1 --2- 172.21.15.86:0/3477703716 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f3ea00a1230 0x7f3ea00a36f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:21.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.249+0000 7f3ec2c00700 1 --2- 172.21.15.86:0/3477703716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ebc101780 0x7f3ebc198df0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:21.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.249+0000 7f3ec2c00700 1 --2- 172.21.15.86:0/3477703716 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3ebc1040b0 0x7f3ebc199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:21.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.249+0000 7f3ec2c00700 1 -- 172.21.15.86:0/3477703716 >> 172.21.15.86:0/3477703716 conn(0x7f3ebc0fb0d0 msgr2=0x7f3ebc100ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:21.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.249+0000 7f3ec2c00700 1 -- 172.21.15.86:0/3477703716 shutdown_connections 2024-03-23T17:50:21.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.249+0000 7f3ec2c00700 1 -- 172.21.15.86:0/3477703716 wait complete. 2024-03-23T17:50:21.268 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T17:45:26.848397+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T17:48:54.331894+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T17:48:55.640693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T17:50:21.268 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T17:50:21.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.463+0000 7f70aa400700 1 -- 172.21.15.86:0/2871902633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70a4101f70 msgr2=0x7f70a41023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:21.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.463+0000 7f70aa400700 1 --2- 172.21.15.86:0/2871902633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70a4101f70 0x7f70a41023d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f70900099c0 tx=0x7f7090009cd0 comp rx=0 tx=0).stop 2024-03-23T17:50:21.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.463+0000 7f70aa400700 1 -- 172.21.15.86:0/2871902633 shutdown_connections 2024-03-23T17:50:21.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.463+0000 7f70aa400700 1 --2- 172.21.15.86:0/2871902633 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f70a4102910 0x7f70a41091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:21.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.463+0000 7f70aa400700 1 --2- 172.21.15.86:0/2871902633 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70a4100d80 0x7f70a41011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:21.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.463+0000 7f70aa400700 1 --2- 172.21.15.86:0/2871902633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70a4101f70 0x7f70a41023d0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:21.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.463+0000 7f70aa400700 1 -- 172.21.15.86:0/2871902633 >> 172.21.15.86:0/2871902633 conn(0x7f70a40fc2f0 msgr2=0x7f70a40fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:21.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.463+0000 7f70aa400700 1 -- 172.21.15.86:0/2871902633 shutdown_connections 2024-03-23T17:50:21.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.463+0000 7f70aa400700 1 -- 172.21.15.86:0/2871902633 wait complete. 2024-03-23T17:50:21.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.464+0000 7f70aa400700 1 Processor -- start 2024-03-23T17:50:21.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.464+0000 7f70aa400700 1 -- start start 2024-03-23T17:50:21.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.464+0000 7f70aa400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70a4100d80 0x7f70a4196d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:21.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.464+0000 7f70aa400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70a4101f70 0x7f70a4197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:21.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.464+0000 7f70aa400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f70a4102910 0x7f70a419e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:21.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.464+0000 7f70aa400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a419e840 con 0x7f70a4100d80 2024-03-23T17:50:21.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.464+0000 7f70aa400700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a419e9b0 con 0x7f70a4101f70 2024-03-23T17:50:21.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.464+0000 7f70aa400700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a419eb70 con 0x7f70a4102910 2024-03-23T17:50:21.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.465+0000 7f70a3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70a4100d80 0x7f70a4196d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:21.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.465+0000 7f70a3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70a4100d80 0x7f70a4196d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35006/0 (socket says 172.21.15.86:35006) 2024-03-23T17:50:21.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.465+0000 7f70a3400700 1 -- 172.21.15.86:0/1738428010 learned_addr learned my addr 172.21.15.86:0/1738428010 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:21.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.465+0000 7f70a2a00700 1 --2- 172.21.15.86:0/1738428010 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70a4101f70 0x7f70a4197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:21.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.465+0000 7f70a3e00700 1 --2- 172.21.15.86:0/1738428010 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f70a4102910 0x7f70a419e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:21.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.465+0000 7f70a3400700 1 -- 172.21.15.86:0/1738428010 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f70a4102910 msgr2=0x7f70a419e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:21.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.465+0000 7f70a3400700 1 --2- 172.21.15.86:0/1738428010 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f70a4102910 0x7f70a419e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:21.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.465+0000 7f70a3400700 1 -- 172.21.15.86:0/1738428010 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70a4101f70 msgr2=0x7f70a4197290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:21.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.465+0000 7f70a3400700 1 --2- 172.21.15.86:0/1738428010 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70a4101f70 0x7f70a4197290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:21.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.465+0000 7f70a3400700 1 -- 172.21.15.86:0/1738428010 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70900095f0 con 0x7f70a4100d80 2024-03-23T17:50:21.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.465+0000 7f70a3e00700 1 --2- 172.21.15.86:0/1738428010 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f70a4102910 0x7f70a419e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:50:21.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.465+0000 7f70a3400700 1 --2- 172.21.15.86:0/1738428010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70a4100d80 0x7f70a4196d50 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f709400e970 tx=0x7f709400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:21.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.466+0000 7f709be00700 1 -- 172.21.15.86:0/1738428010 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7094009790 con 0x7f70a4100d80 2024-03-23T17:50:21.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.466+0000 7f709be00700 1 -- 172.21.15.86:0/1738428010 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f709400eec0 con 0x7f70a4100d80 2024-03-23T17:50:21.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.466+0000 7f70aa400700 1 -- 172.21.15.86:0/1738428010 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70a419ee50 con 0x7f70a4100d80 2024-03-23T17:50:21.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.466+0000 7f709be00700 1 -- 172.21.15.86:0/1738428010 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7094003cf0 con 0x7f70a4100d80 2024-03-23T17:50:21.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.466+0000 7f70aa400700 1 -- 172.21.15.86:0/1738428010 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70a419f270 con 0x7f70a4100d80 2024-03-23T17:50:21.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.467+0000 7f70aa400700 1 -- 172.21.15.86:0/1738428010 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70a4108200 con 0x7f70a4100d80 2024-03-23T17:50:21.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.469+0000 7f709be00700 1 -- 172.21.15.86:0/1738428010 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f709400ccd0 con 0x7f70a4100d80 2024-03-23T17:50:21.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.470+0000 7f709be00700 1 --2- 172.21.15.86:0/1738428010 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f70880a11b0 0x7f70880a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:21.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.470+0000 7f709be00700 1 -- 172.21.15.86:0/1738428010 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f7094014070 con 0x7f70a4100d80 2024-03-23T17:50:21.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.470+0000 7f70a2a00700 1 --2- 172.21.15.86:0/1738428010 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f70880a11b0 0x7f70880a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:21.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.471+0000 7f70a2a00700 1 --2- 172.21.15.86:0/1738428010 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f70880a11b0 0x7f70880a3670 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7090003730 tx=0x7f709000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:21.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.474+0000 7f709be00700 1 -- 172.21.15.86:0/1738428010 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f70940962e0 con 0x7f70a4100d80 2024-03-23T17:50:21.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.680+0000 7f70aa400700 1 -- 172.21.15.86:0/1738428010 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f70a410c5e0 con 0x7f70a4100d80 2024-03-23T17:50:21.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.681+0000 7f709be00700 1 -- 172.21.15.86:0/1738428010 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7094099900 con 0x7f70a4100d80 2024-03-23T17:50:21.681 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-03-23T17:50:21.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.683+0000 7f70aa400700 1 -- 172.21.15.86:0/1738428010 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f70880a11b0 msgr2=0x7f70880a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:21.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.683+0000 7f70aa400700 1 --2- 172.21.15.86:0/1738428010 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f70880a11b0 0x7f70880a3670 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7090003730 tx=0x7f709000b540 comp rx=0 tx=0).stop 2024-03-23T17:50:21.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.683+0000 7f70aa400700 1 -- 172.21.15.86:0/1738428010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70a4100d80 msgr2=0x7f70a4196d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:21.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.683+0000 7f70aa400700 1 --2- 172.21.15.86:0/1738428010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70a4100d80 0x7f70a4196d50 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f709400e970 tx=0x7f709400ec80 comp rx=0 tx=0).stop 2024-03-23T17:50:21.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.683+0000 7f70aa400700 1 -- 172.21.15.86:0/1738428010 shutdown_connections 2024-03-23T17:50:21.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.683+0000 7f70aa400700 1 --2- 172.21.15.86:0/1738428010 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f70a4102910 0x7f70a419e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:21.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.683+0000 7f70aa400700 1 --2- 172.21.15.86:0/1738428010 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f70880a11b0 0x7f70880a3670 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:21.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.683+0000 7f70aa400700 1 --2- 172.21.15.86:0/1738428010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70a4100d80 0x7f70a4196d50 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:21.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.683+0000 7f70aa400700 1 --2- 172.21.15.86:0/1738428010 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70a4101f70 0x7f70a4197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:21.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.683+0000 7f70aa400700 1 -- 172.21.15.86:0/1738428010 >> 172.21.15.86:0/1738428010 conn(0x7f70a40fc2f0 msgr2=0x7f70a4105340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:21.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.684+0000 7f70aa400700 1 -- 172.21.15.86:0/1738428010 shutdown_connections 2024-03-23T17:50:21.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.684+0000 7f70aa400700 1 -- 172.21.15.86:0/1738428010 wait complete. 2024-03-23T17:50:21.703 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T17:50:21.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.900+0000 7fe276200700 1 -- 172.21.15.86:0/2754069900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2701017e0 msgr2=0x7fe270103bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:21.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.900+0000 7fe276200700 1 --2- 172.21.15.86:0/2754069900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2701017e0 0x7fe270103bd0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe260009990 tx=0x7fe260009ca0 comp rx=0 tx=0).stop 2024-03-23T17:50:21.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.900+0000 7fe276200700 1 -- 172.21.15.86:0/2754069900 shutdown_connections 2024-03-23T17:50:21.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.900+0000 7fe276200700 1 --2- 172.21.15.86:0/2754069900 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe270106a40 0x7fe270108eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:21.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.900+0000 7fe276200700 1 --2- 172.21.15.86:0/2754069900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2701017e0 0x7fe270103bd0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:21.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.900+0000 7fe276200700 1 --2- 172.21.15.86:0/2754069900 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe270104110 0x7fe270106500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:21.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.900+0000 7fe276200700 1 -- 172.21.15.86:0/2754069900 >> 172.21.15.86:0/2754069900 conn(0x7fe2700fb1d0 msgr2=0x7fe2700fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:21.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.900+0000 7fe276200700 1 -- 172.21.15.86:0/2754069900 shutdown_connections 2024-03-23T17:50:21.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.900+0000 7fe276200700 1 -- 172.21.15.86:0/2754069900 wait complete. 2024-03-23T17:50:21.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.901+0000 7fe276200700 1 Processor -- start 2024-03-23T17:50:21.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.901+0000 7fe276200700 1 -- start start 2024-03-23T17:50:21.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.901+0000 7fe276200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2701017e0 0x7fe270101100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:21.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.901+0000 7fe276200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe270104110 0x7fe2700ff790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:21.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.901+0000 7fe276200700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe270106a40 0x7fe2700ffcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:21.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.901+0000 7fe276200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe270100470 con 0x7fe270104110 2024-03-23T17:50:21.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.901+0000 7fe276200700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2701005e0 con 0x7fe2701017e0 2024-03-23T17:50:21.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.901+0000 7fe276200700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2701a0d90 con 0x7fe270106a40 2024-03-23T17:50:21.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.902+0000 7fe267e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe270104110 0x7fe2700ff790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:21.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.902+0000 7fe26f400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2701017e0 0x7fe270101100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:21.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.902+0000 7fe26f400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2701017e0 0x7fe270101100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:45224/0 (socket says 172.21.15.86:45224) 2024-03-23T17:50:21.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.902+0000 7fe26f400700 1 -- 172.21.15.86:0/1727152007 learned_addr learned my addr 172.21.15.86:0/1727152007 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:21.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.902+0000 7fe26fe00700 1 --2- 172.21.15.86:0/1727152007 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe270106a40 0x7fe2700ffcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:21.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.902+0000 7fe26f400700 1 -- 172.21.15.86:0/1727152007 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe270106a40 msgr2=0x7fe2700ffcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:21.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.902+0000 7fe26f400700 1 --2- 172.21.15.86:0/1727152007 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe270106a40 0x7fe2700ffcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:21.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.902+0000 7fe26f400700 1 -- 172.21.15.86:0/1727152007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe270104110 msgr2=0x7fe2700ff790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:21.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.902+0000 7fe26f400700 1 --2- 172.21.15.86:0/1727152007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe270104110 0x7fe2700ff790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:21.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.902+0000 7fe26f400700 1 -- 172.21.15.86:0/1727152007 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe268009c40 con 0x7fe2701017e0 2024-03-23T17:50:21.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.902+0000 7fe267e00700 1 --2- 172.21.15.86:0/1727152007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe270104110 0x7fe2700ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:50:21.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.902+0000 7fe26f400700 1 --2- 172.21.15.86:0/1727152007 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2701017e0 0x7fe270101100 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe260005910 tx=0x7fe260009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:21.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.902+0000 7fe26cc00700 1 -- 172.21.15.86:0/1727152007 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe26001d070 con 0x7fe2701017e0 2024-03-23T17:50:21.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.902+0000 7fe276200700 1 -- 172.21.15.86:0/1727152007 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2600095f0 con 0x7fe2701017e0 2024-03-23T17:50:21.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.903+0000 7fe26cc00700 1 -- 172.21.15.86:0/1727152007 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe260021410 con 0x7fe2701017e0 2024-03-23T17:50:21.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.903+0000 7fe26cc00700 1 -- 172.21.15.86:0/1727152007 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe260017650 con 0x7fe2701017e0 2024-03-23T17:50:21.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.903+0000 7fe276200700 1 -- 172.21.15.86:0/1727152007 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2701a11a0 con 0x7fe2701017e0 2024-03-23T17:50:21.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.903+0000 7fe276200700 1 -- 172.21.15.86:0/1727152007 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2700fcd50 con 0x7fe2701017e0 2024-03-23T17:50:21.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.906+0000 7fe26cc00700 1 -- 172.21.15.86:0/1727152007 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fe260021580 con 0x7fe2701017e0 2024-03-23T17:50:21.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.907+0000 7fe26cc00700 1 --2- 172.21.15.86:0/1727152007 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fe2540a11b0 0x7fe2540a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:21.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.907+0000 7fe26cc00700 1 -- 172.21.15.86:0/1727152007 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fe2600ce3d0 con 0x7fe2701017e0 2024-03-23T17:50:21.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.907+0000 7fe267e00700 1 --2- 172.21.15.86:0/1727152007 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fe2540a11b0 0x7fe2540a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:21.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.908+0000 7fe267e00700 1 --2- 172.21.15.86:0/1727152007 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fe2540a11b0 0x7fe2540a3670 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe258011490 tx=0x7fe258009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:21.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:21.909+0000 7fe26cc00700 1 -- 172.21.15.86:0/1727152007 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fe260097ac0 con 0x7fe2701017e0 2024-03-23T17:50:22.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:21 smithi086 ceph-mon[38221]: pgmap v278: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 597 B/s wr, 11 op/s 2024-03-23T17:50:22.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:21 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/2581660283' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-03-23T17:50:22.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:21 smithi086 ceph-mon[38221]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T17:50:22.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:21 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/3477703716' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T17:50:22.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:21 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/1738428010' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T17:50:22.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:21 smithi092 ceph-mon[79089]: pgmap v278: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 597 B/s wr, 11 op/s 2024-03-23T17:50:22.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:21 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/2581660283' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-03-23T17:50:22.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:21 smithi092 ceph-mon[79089]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T17:50:22.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:21 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/3477703716' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T17:50:22.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:21 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/1738428010' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T17:50:22.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:21 smithi152 ceph-mon[47949]: pgmap v278: 97 pgs: 97 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 597 B/s wr, 11 op/s 2024-03-23T17:50:22.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:21 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/2581660283' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-03-23T17:50:22.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:21 smithi152 ceph-mon[47949]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T17:50:22.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:21 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/3477703716' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T17:50:22.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:21 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/1738428010' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T17:50:22.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.114+0000 7fe276200700 1 -- 172.21.15.86:0/1727152007 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe27010c6a0 con 0x7fe2701017e0 2024-03-23T17:50:22.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.114+0000 7fe26cc00700 1 -- 172.21.15.86:0/1727152007 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v82) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe26009b0e0 con 0x7fe2701017e0 2024-03-23T17:50:22.115 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-03-23T17:50:22.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.117+0000 7fe276200700 1 -- 172.21.15.86:0/1727152007 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fe2540a11b0 msgr2=0x7fe2540a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:22.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.117+0000 7fe276200700 1 --2- 172.21.15.86:0/1727152007 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fe2540a11b0 0x7fe2540a3670 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe258011490 tx=0x7fe258009260 comp rx=0 tx=0).stop 2024-03-23T17:50:22.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.117+0000 7fe276200700 1 -- 172.21.15.86:0/1727152007 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2701017e0 msgr2=0x7fe270101100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:22.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.117+0000 7fe276200700 1 --2- 172.21.15.86:0/1727152007 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2701017e0 0x7fe270101100 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe260005910 tx=0x7fe260009580 comp rx=0 tx=0).stop 2024-03-23T17:50:22.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.117+0000 7fe276200700 1 -- 172.21.15.86:0/1727152007 shutdown_connections 2024-03-23T17:50:22.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.117+0000 7fe276200700 1 --2- 172.21.15.86:0/1727152007 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe270106a40 0x7fe2700ffcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:22.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.117+0000 7fe276200700 1 --2- 172.21.15.86:0/1727152007 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2701017e0 0x7fe270101100 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:22.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.117+0000 7fe276200700 1 --2- 172.21.15.86:0/1727152007 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fe2540a11b0 0x7fe2540a3670 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:22.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.117+0000 7fe276200700 1 --2- 172.21.15.86:0/1727152007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe270104110 0x7fe2700ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:22.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.117+0000 7fe276200700 1 -- 172.21.15.86:0/1727152007 >> 172.21.15.86:0/1727152007 conn(0x7fe2700fb1d0 msgr2=0x7fe270102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:22.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.117+0000 7fe276200700 1 -- 172.21.15.86:0/1727152007 shutdown_connections 2024-03-23T17:50:22.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.117+0000 7fe276200700 1 -- 172.21.15.86:0/1727152007 wait complete. 2024-03-23T17:50:22.137 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T17:50:22.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.331+0000 7f3117600700 1 -- 172.21.15.86:0/4018106457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3110100d80 msgr2=0x7f31101011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:22.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.331+0000 7f3117600700 1 --2- 172.21.15.86:0/4018106457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3110100d80 0x7f31101011a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f31040099c0 tx=0x7f3104009cd0 comp rx=0 tx=0).stop 2024-03-23T17:50:22.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.331+0000 7f3117600700 1 -- 172.21.15.86:0/4018106457 shutdown_connections 2024-03-23T17:50:22.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.331+0000 7f3117600700 1 --2- 172.21.15.86:0/4018106457 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3110102910 0x7f31101091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:22.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.331+0000 7f3117600700 1 --2- 172.21.15.86:0/4018106457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3110100d80 0x7f31101011a0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:22.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.331+0000 7f3117600700 1 --2- 172.21.15.86:0/4018106457 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3110101f70 0x7f31101023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:22.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.331+0000 7f3117600700 1 -- 172.21.15.86:0/4018106457 >> 172.21.15.86:0/4018106457 conn(0x7f31100fc2f0 msgr2=0x7f31100fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:22.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.331+0000 7f3117600700 1 -- 172.21.15.86:0/4018106457 shutdown_connections 2024-03-23T17:50:22.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.331+0000 7f3117600700 1 -- 172.21.15.86:0/4018106457 wait complete. 2024-03-23T17:50:22.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.332+0000 7f3117600700 1 Processor -- start 2024-03-23T17:50:22.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.333+0000 7f3117600700 1 -- start start 2024-03-23T17:50:22.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.333+0000 7f3117600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3110100d80 0x7f3110196d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:22.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.333+0000 7f3117600700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3110101f70 0x7f3110197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:22.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.333+0000 7f3117600700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3110102910 0x7f311019e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:22.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.333+0000 7f3117600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f311019e840 con 0x7f3110100d80 2024-03-23T17:50:22.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.333+0000 7f3117600700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f311019e9b0 con 0x7f3110102910 2024-03-23T17:50:22.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.333+0000 7f3117600700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f311019eb70 con 0x7f3110101f70 2024-03-23T17:50:22.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.333+0000 7f3114c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3110100d80 0x7f3110196d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:22.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.333+0000 7f3114c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3110100d80 0x7f3110196d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35046/0 (socket says 172.21.15.86:35046) 2024-03-23T17:50:22.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.333+0000 7f3114c00700 1 -- 172.21.15.86:0/1549084308 learned_addr learned my addr 172.21.15.86:0/1549084308 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:22.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.333+0000 7f310fe00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3110101f70 0x7f3110197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:22.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.333+0000 7f3115600700 1 --2- 172.21.15.86:0/1549084308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3110102910 0x7f311019e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:22.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.333+0000 7f3114c00700 1 -- 172.21.15.86:0/1549084308 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3110101f70 msgr2=0x7f3110197290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:22.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.333+0000 7f3114c00700 1 --2- 172.21.15.86:0/1549084308 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3110101f70 0x7f3110197290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:22.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.333+0000 7f3114c00700 1 -- 172.21.15.86:0/1549084308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3110102910 msgr2=0x7f311019e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:22.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.334+0000 7f3114c00700 1 --2- 172.21.15.86:0/1549084308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3110102910 0x7f311019e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:22.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.334+0000 7f3114c00700 1 -- 172.21.15.86:0/1549084308 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31040095f0 con 0x7f3110100d80 2024-03-23T17:50:22.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.334+0000 7f3115600700 1 --2- 172.21.15.86:0/1549084308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3110102910 0x7f311019e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:50:22.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.334+0000 7f3114c00700 1 --2- 172.21.15.86:0/1549084308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3110100d80 0x7f3110196d50 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3104006010 tx=0x7f3104004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:22.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.334+0000 7f310d600700 1 -- 172.21.15.86:0/1549084308 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f310401d070 con 0x7f3110100d80 2024-03-23T17:50:22.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.334+0000 7f310d600700 1 -- 172.21.15.86:0/1549084308 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3104003f90 con 0x7f3110100d80 2024-03-23T17:50:22.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.334+0000 7f310d600700 1 -- 172.21.15.86:0/1549084308 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f310400fd70 con 0x7f3110100d80 2024-03-23T17:50:22.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.334+0000 7f3117600700 1 -- 172.21.15.86:0/1549084308 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f311019edf0 con 0x7f3110100d80 2024-03-23T17:50:22.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.334+0000 7f3117600700 1 -- 172.21.15.86:0/1549084308 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f311019f1c0 con 0x7f3110100d80 2024-03-23T17:50:22.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.335+0000 7f3117600700 1 -- 172.21.15.86:0/1549084308 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3110108270 con 0x7f3110100d80 2024-03-23T17:50:22.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.337+0000 7f310d600700 1 -- 172.21.15.86:0/1549084308 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f310400f460 con 0x7f3110100d80 2024-03-23T17:50:22.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.339+0000 7f310d600700 1 --2- 172.21.15.86:0/1549084308 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f30f80a1230 0x7f30f80a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:22.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.339+0000 7f310d600700 1 -- 172.21.15.86:0/1549084308 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f31040cdc90 con 0x7f3110100d80 2024-03-23T17:50:22.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.339+0000 7f310fe00700 1 --2- 172.21.15.86:0/1549084308 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f30f80a1230 0x7f30f80a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:22.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.339+0000 7f310fe00700 1 --2- 172.21.15.86:0/1549084308 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f30f80a1230 0x7f30f80a36f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f30fc007c60 tx=0x7f30fc0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:22.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.341+0000 7f310d600700 1 -- 172.21.15.86:0/1549084308 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f3104097320 con 0x7f3110100d80 2024-03-23T17:50:22.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.546+0000 7f3117600700 1 -- 172.21.15.86:0/1549084308 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f311010c5e0 con 0x7f3110100d80 2024-03-23T17:50:22.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.547+0000 7f310d600700 1 -- 172.21.15.86:0/1549084308 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v82) v1 ==== 100+0+11 (secure 0 0 0) 0x7f310409a940 con 0x7f3110100d80 2024-03-23T17:50:22.547 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-03-23T17:50:22.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.549+0000 7f3117600700 1 -- 172.21.15.86:0/1549084308 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f30f80a1230 msgr2=0x7f30f80a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:22.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.549+0000 7f3117600700 1 --2- 172.21.15.86:0/1549084308 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f30f80a1230 0x7f30f80a36f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f30fc007c60 tx=0x7f30fc0073d0 comp rx=0 tx=0).stop 2024-03-23T17:50:22.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.549+0000 7f3117600700 1 -- 172.21.15.86:0/1549084308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3110100d80 msgr2=0x7f3110196d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:22.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.549+0000 7f3117600700 1 --2- 172.21.15.86:0/1549084308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3110100d80 0x7f3110196d50 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3104006010 tx=0x7f3104004730 comp rx=0 tx=0).stop 2024-03-23T17:50:22.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.550+0000 7f3117600700 1 -- 172.21.15.86:0/1549084308 shutdown_connections 2024-03-23T17:50:22.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.550+0000 7f3117600700 1 --2- 172.21.15.86:0/1549084308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3110102910 0x7f311019e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:22.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.550+0000 7f3117600700 1 --2- 172.21.15.86:0/1549084308 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f30f80a1230 0x7f30f80a36f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:22.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.550+0000 7f3117600700 1 --2- 172.21.15.86:0/1549084308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3110100d80 0x7f3110196d50 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:22.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.550+0000 7f3117600700 1 --2- 172.21.15.86:0/1549084308 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3110101f70 0x7f3110197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:22.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.550+0000 7f3117600700 1 -- 172.21.15.86:0/1549084308 >> 172.21.15.86:0/1549084308 conn(0x7f31100fc2f0 msgr2=0x7f31101053b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:22.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.550+0000 7f3117600700 1 -- 172.21.15.86:0/1549084308 shutdown_connections 2024-03-23T17:50:22.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.550+0000 7f3117600700 1 -- 172.21.15.86:0/1549084308 wait complete. 2024-03-23T17:50:22.568 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T17:50:22.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.762+0000 7f815fa00700 1 -- 172.21.15.86:0/2518461050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8158074df0 msgr2=0x7f8158073240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:22.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.762+0000 7f815fa00700 1 --2- 172.21.15.86:0/2518461050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8158074df0 0x7f8158073240 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f814c009990 tx=0x7f814c009ca0 comp rx=0 tx=0).stop 2024-03-23T17:50:22.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.762+0000 7f815fa00700 1 -- 172.21.15.86:0/2518461050 shutdown_connections 2024-03-23T17:50:22.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.762+0000 7f815fa00700 1 --2- 172.21.15.86:0/2518461050 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f815810d250 0x7f815810f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:22.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.762+0000 7f815fa00700 1 --2- 172.21.15.86:0/2518461050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8158074df0 0x7f8158073240 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:22.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.762+0000 7f815fa00700 1 --2- 172.21.15.86:0/2518461050 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8158073810 0x7f8158073c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:22.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.762+0000 7f815fa00700 1 -- 172.21.15.86:0/2518461050 >> 172.21.15.86:0/2518461050 conn(0x7f81580fc1f0 msgr2=0x7f81580fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:22.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.762+0000 7f815fa00700 1 -- 172.21.15.86:0/2518461050 shutdown_connections 2024-03-23T17:50:22.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.762+0000 7f815fa00700 1 -- 172.21.15.86:0/2518461050 wait complete. 2024-03-23T17:50:22.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.763+0000 7f815fa00700 1 Processor -- start 2024-03-23T17:50:22.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.763+0000 7f815fa00700 1 -- start start 2024-03-23T17:50:22.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.763+0000 7f815fa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8158073810 0x7f815819d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:22.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.763+0000 7f815fa00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8158074df0 0x7f815819d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:22.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.763+0000 7f815fa00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f815810d250 0x7f81581a47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:22.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.763+0000 7f815fa00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81581a4d80 con 0x7f8158073810 2024-03-23T17:50:22.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.763+0000 7f815fa00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81581a4ec0 con 0x7f815810d250 2024-03-23T17:50:22.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.763+0000 7f815fa00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81581a5050 con 0x7f8158074df0 2024-03-23T17:50:22.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.764+0000 7f815d000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8158073810 0x7f815819d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:22.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.764+0000 7f815d000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8158073810 0x7f815819d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35070/0 (socket says 172.21.15.86:35070) 2024-03-23T17:50:22.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.764+0000 7f815d000700 1 -- 172.21.15.86:0/3408614366 learned_addr learned my addr 172.21.15.86:0/3408614366 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:22.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.764+0000 7f8157e00700 1 --2- 172.21.15.86:0/3408614366 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8158074df0 0x7f815819d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:22.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.764+0000 7f815da00700 1 --2- 172.21.15.86:0/3408614366 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f815810d250 0x7f81581a47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:22.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.764+0000 7f815d000700 1 -- 172.21.15.86:0/3408614366 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8158074df0 msgr2=0x7f815819d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:22.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.764+0000 7f815d000700 1 --2- 172.21.15.86:0/3408614366 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8158074df0 0x7f815819d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:22.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.764+0000 7f815d000700 1 -- 172.21.15.86:0/3408614366 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f815810d250 msgr2=0x7f81581a47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:22.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.764+0000 7f815d000700 1 --2- 172.21.15.86:0/3408614366 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f815810d250 0x7f81581a47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:22.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.764+0000 7f815d000700 1 -- 172.21.15.86:0/3408614366 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f814c0095f0 con 0x7f8158073810 2024-03-23T17:50:22.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.764+0000 7f815d000700 1 --2- 172.21.15.86:0/3408614366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8158073810 0x7f815819d250 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f814c00b680 tx=0x7f814c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:22.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.765+0000 7f8155600700 1 -- 172.21.15.86:0/3408614366 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f814c01d070 con 0x7f8158073810 2024-03-23T17:50:22.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.765+0000 7f815fa00700 1 -- 172.21.15.86:0/3408614366 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81581a52d0 con 0x7f8158073810 2024-03-23T17:50:22.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.765+0000 7f8155600700 1 -- 172.21.15.86:0/3408614366 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f814c022470 con 0x7f8158073810 2024-03-23T17:50:22.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.765+0000 7f8155600700 1 -- 172.21.15.86:0/3408614366 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f814c017650 con 0x7f8158073810 2024-03-23T17:50:22.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.765+0000 7f815fa00700 1 -- 172.21.15.86:0/3408614366 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81581a5700 con 0x7f8158073810 2024-03-23T17:50:22.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.766+0000 7f815fa00700 1 -- 172.21.15.86:0/3408614366 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81581034b0 con 0x7f8158073810 2024-03-23T17:50:22.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.768+0000 7f8155600700 1 -- 172.21.15.86:0/3408614366 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f814c0225e0 con 0x7f8158073810 2024-03-23T17:50:22.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.769+0000 7f8155600700 1 --2- 172.21.15.86:0/3408614366 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f81400a11b0 0x7f81400a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:22.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.769+0000 7f8155600700 1 -- 172.21.15.86:0/3408614366 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f814c0ce2d0 con 0x7f8158073810 2024-03-23T17:50:22.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.769+0000 7f8157e00700 1 --2- 172.21.15.86:0/3408614366 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f81400a11b0 0x7f81400a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:22.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.770+0000 7f8157e00700 1 --2- 172.21.15.86:0/3408614366 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f81400a11b0 0x7f81400a3670 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8158074b10 tx=0x7f8144009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:22.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:22.771+0000 7f8155600700 1 -- 172.21.15.86:0/3408614366 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f814c097960 con 0x7f8158073810 2024-03-23T17:50:23.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.013+0000 7f815fa00700 1 -- 172.21.15.86:0/3408614366 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8158112ad0 con 0x7f8158073810 2024-03-23T17:50:23.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.015+0000 7f8155600700 1 -- 172.21.15.86:0/3408614366 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22414 (secure 0 0 0) 0x7f814c09af80 con 0x7f8158073810 2024-03-23T17:50:23.015 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:50:23.016 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":82,"fsid":"9d76c9e8-e93c-11ee-95cd-87774f69a715","created":"2024-03-23T17:42:30.097233+0000","modified":"2024-03-23T17:49:29.773954+0000","last_up_change":"2024-03-23T17:48:18.217601+0000","last_in_change":"2024-03-23T17:48:00.250579+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T17:45:26.848397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T17:48:54.331894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T17:48:55.640693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"656655c6-1c7f-4640-92c4-2f42de820143","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6803","nonce":2423505491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6805","nonce":2423505491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6809","nonce":2423505491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6807","nonce":2423505491}]},"public_addr":"172.21.15.86:6803/2423505491","cluster_addr":"172.21.15.86:6805/2423505491","heartbeat_back_addr":"172.21.15.86:6809/2423505491","heartbeat_front_addr":"172.21.15.86:6807/2423505491","state":["exists","up"]},{"osd":1,"uuid":"3e86dc65-20a6-4de8-b9ce-fcf01f68862b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6801","nonce":3776252597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6803","nonce":3776252597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6807","nonce":3776252597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6805","nonce":3776252597}]},"public_addr":"172.21.15.92:6801/3776252597","cluster_addr":"172.21.15.92:6803/3776252597","heartbeat_back_addr":"172.21.15.92:6807/3776252597","heartbeat_front_addr":"172.21.15.92:6805/3776252597","state":["exists","up"]},{"osd":2,"uuid":"09808786-3bd3-4d11-8c7b-1736119dfbb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6801","nonce":1601265891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6803","nonce":1601265891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6807","nonce":1601265891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6805","nonce":1601265891}]},"public_addr":"172.21.15.152:6801/1601265891","cluster_addr":"172.21.15.152:6803/1601265891","heartbeat_back_addr":"172.21.15.152:6807/1601265891","heartbeat_front_addr":"172.21.15.152:6805/1601265891","state":["exists","up"]},{"osd":3,"uuid":"79c61910-4b53-424c-b150-6f5385a2cd68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6811","nonce":694846466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6813","nonce":694846466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6817","nonce":694846466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6815","nonce":694846466}]},"public_addr":"172.21.15.86:6811/694846466","cluster_addr":"172.21.15.86:6813/694846466","heartbeat_back_addr":"172.21.15.86:6817/694846466","heartbeat_front_addr":"172.21.15.86:6815/694846466","state":["exists","up"]},{"osd":4,"uuid":"a2878035-0179-469f-bbd4-8ce2d3f6bddc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6809","nonce":1422821471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6811","nonce":1422821471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6815","nonce":1422821471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6813","nonce":1422821471}]},"public_addr":"172.21.15.92:6809/1422821471","cluster_addr":"172.21.15.92:6811/1422821471","heartbeat_back_addr":"172.21.15.92:6815/1422821471","heartbeat_front_addr":"172.21.15.92:6813/1422821471","state":["exists","up"]},{"osd":5,"uuid":"837366aa-ce25-42ca-b157-5bbb50159d0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6809","nonce":1039375239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6811","nonce":1039375239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6815","nonce":1039375239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6813","nonce":1039375239}]},"public_addr":"172.21.15.152:6809/1039375239","cluster_addr":"172.21.15.152:6811/1039375239","heartbeat_back_addr":"172.21.15.152:6815/1039375239","heartbeat_front_addr":"172.21.15.152:6813/1039375239","state":["exists","up"]},{"osd":6,"uuid":"61024437-7b91-432f-a1af-6443d0ef5213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6819","nonce":4080452707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6821","nonce":4080452707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6825","nonce":4080452707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6823","nonce":4080452707}]},"public_addr":"172.21.15.86:6819/4080452707","cluster_addr":"172.21.15.86:6821/4080452707","heartbeat_back_addr":"172.21.15.86:6825/4080452707","heartbeat_front_addr":"172.21.15.86:6823/4080452707","state":["exists","up"]},{"osd":7,"uuid":"2684bdae-6b30-4afc-af68-4ce41b3a75b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6817","nonce":4187707622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6819","nonce":4187707622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6823","nonce":4187707622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6821","nonce":4187707622}]},"public_addr":"172.21.15.92:6817/4187707622","cluster_addr":"172.21.15.92:6819/4187707622","heartbeat_back_addr":"172.21.15.92:6823/4187707622","heartbeat_front_addr":"172.21.15.92:6821/4187707622","state":["exists","up"]},{"osd":8,"uuid":"f03bbf0d-1281-4bcf-9773-9854388e65ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6817","nonce":3237612748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6819","nonce":3237612748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6823","nonce":3237612748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6821","nonce":3237612748}]},"public_addr":"172.21.15.152:6817/3237612748","cluster_addr":"172.21.15.152:6819/3237612748","heartbeat_back_addr":"172.21.15.152:6823/3237612748","heartbeat_front_addr":"172.21.15.152:6821/3237612748","state":["exists","up"]},{"osd":9,"uuid":"e7e8b775-99ff-48a4-a23a-91e7ab6622bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6827","nonce":2835218294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6829","nonce":2835218294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6833","nonce":2835218294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6831","nonce":2835218294}]},"public_addr":"172.21.15.86:6827/2835218294","cluster_addr":"172.21.15.86:6829/2835218294","heartbeat_back_addr":"172.21.15.86:6833/2835218294","heartbeat_front_addr":"172.21.15.86:6831/2835218294","state":["exists","up"]},{"osd":10,"uuid":"d9d616cd-a229-44de-8768-02a4415df115","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6825","nonce":4259465100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6827","nonce":4259465100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6831","nonce":4259465100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6829","nonce":4259465100}]},"public_addr":"172.21.15.92:6825/4259465100","cluster_addr":"172.21.15.92:6827/4259465100","heartbeat_back_addr":"172.21.15.92:6831/4259465100","heartbeat_front_addr":"172.21.15.92:6829/4259465100","state":["exists","up"]},{"osd":11,"uuid":"e7c0b865-ccf5-4233-8d22-0d9b6dbe5c97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6825","nonce":194662760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6827","nonce":194662760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6831","nonce":194662760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6829","nonce":194662760}]},"public_addr":"172.21.15.152:6825/194662760","cluster_addr":"172.21.15.152:6827/194662760","heartbeat_back_addr":"172.21.15.152:6831/194662760","heartbeat_front_addr":"172.21.15.152:6829/194662760","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:44:47.269167+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:45:05.363655+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:45:23.192677+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:45:44.156228+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:02.532971+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:20.153295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:41.713888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:59.676516+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:47:17.207087+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:47:38.970316+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:47:57.282525+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:48:15.437606+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/3553500912":"2024-03-24T17:43:28.242492+0000","172.21.15.86:6801/3600278228":"2024-03-24T17:43:28.242492+0000","172.21.15.86:0/887262329":"2024-03-24T17:43:28.242492+0000","172.21.15.86:6800/2313723492":"2024-03-24T17:43:05.043910+0000","172.21.15.86:6800/3600278228":"2024-03-24T17:43:28.242492+0000","172.21.15.86:0/3815705642":"2024-03-24T17:43:05.043910+0000","172.21.15.86:0/1614943062":"2024-03-24T17:43:28.242492+0000","172.21.15.86:6801/2313723492":"2024-03-24T17:43:05.043910+0000","172.21.15.86:0/2416321716":"2024-03-24T17:43:05.043910+0000","172.21.15.86:0/433588188":"2024-03-24T17:43:05.043910+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T17:50:23.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:22 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/1727152007' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T17:50:23.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:22 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/1549084308' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T17:50:23.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.017+0000 7f815fa00700 1 -- 172.21.15.86:0/3408614366 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f81400a11b0 msgr2=0x7f81400a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:23.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.017+0000 7f815fa00700 1 --2- 172.21.15.86:0/3408614366 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f81400a11b0 0x7f81400a3670 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8158074b10 tx=0x7f8144009260 comp rx=0 tx=0).stop 2024-03-23T17:50:23.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.017+0000 7f815fa00700 1 -- 172.21.15.86:0/3408614366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8158073810 msgr2=0x7f815819d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:23.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.017+0000 7f815fa00700 1 --2- 172.21.15.86:0/3408614366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8158073810 0x7f815819d250 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f814c00b680 tx=0x7f814c009580 comp rx=0 tx=0).stop 2024-03-23T17:50:23.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.018+0000 7f815fa00700 1 -- 172.21.15.86:0/3408614366 shutdown_connections 2024-03-23T17:50:23.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.018+0000 7f815fa00700 1 --2- 172.21.15.86:0/3408614366 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f815810d250 0x7f81581a47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:23.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.018+0000 7f815fa00700 1 --2- 172.21.15.86:0/3408614366 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f81400a11b0 0x7f81400a3670 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:23.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.018+0000 7f815fa00700 1 --2- 172.21.15.86:0/3408614366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8158073810 0x7f815819d250 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:23.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.018+0000 7f815fa00700 1 --2- 172.21.15.86:0/3408614366 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8158074df0 0x7f815819d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:23.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.018+0000 7f815fa00700 1 -- 172.21.15.86:0/3408614366 >> 172.21.15.86:0/3408614366 conn(0x7f81580fc1f0 msgr2=0x7f81581005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:23.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.018+0000 7f815fa00700 1 -- 172.21.15.86:0/3408614366 shutdown_connections 2024-03-23T17:50:23.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.018+0000 7f815fa00700 1 -- 172.21.15.86:0/3408614366 wait complete. 2024-03-23T17:50:23.037 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T17:45:26.848397+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T17:48:54.331894+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T17:48:55.640693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T17:50:23.038 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T17:50:23.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:22 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/1727152007' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T17:50:23.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:22 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/1549084308' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T17:50:23.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:22 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/1727152007' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T17:50:23.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:22 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/1549084308' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T17:50:23.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.233+0000 7fd45a400700 1 -- 172.21.15.86:0/2331617901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd454102910 msgr2=0x7fd4541091c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:23.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.233+0000 7fd45a400700 1 --2- 172.21.15.86:0/2331617901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd454102910 0x7fd4541091c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd44c00b0d0 tx=0x7fd44c00b3e0 comp rx=0 tx=0).stop 2024-03-23T17:50:23.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.233+0000 7fd45a400700 1 -- 172.21.15.86:0/2331617901 shutdown_connections 2024-03-23T17:50:23.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.233+0000 7fd45a400700 1 --2- 172.21.15.86:0/2331617901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd454102910 0x7fd4541091c0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:23.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.233+0000 7fd45a400700 1 --2- 172.21.15.86:0/2331617901 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd454100d80 0x7fd4541011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:23.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.233+0000 7fd45a400700 1 --2- 172.21.15.86:0/2331617901 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd454101f70 0x7fd4541023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:23.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.233+0000 7fd45a400700 1 -- 172.21.15.86:0/2331617901 >> 172.21.15.86:0/2331617901 conn(0x7fd4540fc2f0 msgr2=0x7fd4540fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:23.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.233+0000 7fd45a400700 1 -- 172.21.15.86:0/2331617901 shutdown_connections 2024-03-23T17:50:23.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.234+0000 7fd45a400700 1 -- 172.21.15.86:0/2331617901 wait complete. 2024-03-23T17:50:23.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.234+0000 7fd45a400700 1 Processor -- start 2024-03-23T17:50:23.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.235+0000 7fd45a400700 1 -- start start 2024-03-23T17:50:23.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.235+0000 7fd45a400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd454100d80 0x7fd454199020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:23.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.235+0000 7fd45a400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd454101f70 0x7fd454199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:23.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.235+0000 7fd45a400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd454102910 0x7fd4541a0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:23.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.235+0000 7fd45a400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4541a0b50 con 0x7fd454101f70 2024-03-23T17:50:23.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.235+0000 7fd45a400700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4541a0c90 con 0x7fd454100d80 2024-03-23T17:50:23.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.235+0000 7fd45a400700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4541a0e20 con 0x7fd454102910 2024-03-23T17:50:23.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.235+0000 7fd452a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd454101f70 0x7fd454199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:23.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.235+0000 7fd452a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd454101f70 0x7fd454199560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35098/0 (socket says 172.21.15.86:35098) 2024-03-23T17:50:23.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.235+0000 7fd452a00700 1 -- 172.21.15.86:0/3886089514 learned_addr learned my addr 172.21.15.86:0/3886089514 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:23.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.235+0000 7fd453400700 1 --2- 172.21.15.86:0/3886089514 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd454100d80 0x7fd454199020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:23.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.235+0000 7fd453e00700 1 --2- 172.21.15.86:0/3886089514 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd454102910 0x7fd4541a0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:23.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.235+0000 7fd452a00700 1 -- 172.21.15.86:0/3886089514 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd454102910 msgr2=0x7fd4541a0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:23.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.235+0000 7fd452a00700 1 --2- 172.21.15.86:0/3886089514 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd454102910 0x7fd4541a0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:23.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.235+0000 7fd452a00700 1 -- 172.21.15.86:0/3886089514 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd454100d80 msgr2=0x7fd454199020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:23.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.235+0000 7fd452a00700 1 --2- 172.21.15.86:0/3886089514 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd454100d80 0x7fd454199020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:23.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.236+0000 7fd452a00700 1 -- 172.21.15.86:0/3886089514 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd44c009d00 con 0x7fd454101f70 2024-03-23T17:50:23.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.236+0000 7fd453e00700 1 --2- 172.21.15.86:0/3886089514 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd454102910 0x7fd4541a0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T17:50:23.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.236+0000 7fd452a00700 1 --2- 172.21.15.86:0/3886089514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd454101f70 0x7fd454199560 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd44000e9a0 tx=0x7fd44000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:23.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.236+0000 7fd44be00700 1 -- 172.21.15.86:0/3886089514 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd440009790 con 0x7fd454101f70 2024-03-23T17:50:23.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.236+0000 7fd44be00700 1 -- 172.21.15.86:0/3886089514 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd44000cbd0 con 0x7fd454101f70 2024-03-23T17:50:23.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.236+0000 7fd44be00700 1 -- 172.21.15.86:0/3886089514 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd440018680 con 0x7fd454101f70 2024-03-23T17:50:23.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.236+0000 7fd45a400700 1 -- 172.21.15.86:0/3886089514 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4541a1100 con 0x7fd454101f70 2024-03-23T17:50:23.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.236+0000 7fd45a400700 1 -- 172.21.15.86:0/3886089514 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4541a15c0 con 0x7fd454101f70 2024-03-23T17:50:23.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.237+0000 7fd45a400700 1 -- 172.21.15.86:0/3886089514 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd454067af0 con 0x7fd454101f70 2024-03-23T17:50:23.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.239+0000 7fd44be00700 1 -- 172.21.15.86:0/3886089514 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fd44000cd40 con 0x7fd454101f70 2024-03-23T17:50:23.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.240+0000 7fd44be00700 1 --2- 172.21.15.86:0/3886089514 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fd4380a11b0 0x7fd4380a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:23.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.241+0000 7fd44be00700 1 -- 172.21.15.86:0/3886089514 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fd440014070 con 0x7fd454101f70 2024-03-23T17:50:23.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.241+0000 7fd453400700 1 --2- 172.21.15.86:0/3886089514 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fd4380a11b0 0x7fd4380a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:23.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.241+0000 7fd453400700 1 --2- 172.21.15.86:0/3886089514 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fd4380a11b0 0x7fd4380a3670 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd454101dd0 tx=0x7fd444009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:23.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.243+0000 7fd44be00700 1 -- 172.21.15.86:0/3886089514 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fd440096610 con 0x7fd454101f70 2024-03-23T17:50:23.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.447+0000 7fd45a400700 1 -- 172.21.15.86:0/3886089514 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd45410c860 con 0x7fd454101f70 2024-03-23T17:50:23.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.448+0000 7fd44be00700 1 -- 172.21.15.86:0/3886089514 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd440099c30 con 0x7fd454101f70 2024-03-23T17:50:23.448 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-03-23T17:50:23.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.450+0000 7fd45a400700 1 -- 172.21.15.86:0/3886089514 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fd4380a11b0 msgr2=0x7fd4380a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:23.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.450+0000 7fd45a400700 1 --2- 172.21.15.86:0/3886089514 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fd4380a11b0 0x7fd4380a3670 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd454101dd0 tx=0x7fd444009260 comp rx=0 tx=0).stop 2024-03-23T17:50:23.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.450+0000 7fd45a400700 1 -- 172.21.15.86:0/3886089514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd454101f70 msgr2=0x7fd454199560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:23.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.450+0000 7fd45a400700 1 --2- 172.21.15.86:0/3886089514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd454101f70 0x7fd454199560 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd44000e9a0 tx=0x7fd44000ecb0 comp rx=0 tx=0).stop 2024-03-23T17:50:23.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.450+0000 7fd45a400700 1 -- 172.21.15.86:0/3886089514 shutdown_connections 2024-03-23T17:50:23.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.450+0000 7fd45a400700 1 --2- 172.21.15.86:0/3886089514 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd454102910 0x7fd4541a0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:23.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.450+0000 7fd45a400700 1 --2- 172.21.15.86:0/3886089514 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd454100d80 0x7fd454199020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:23.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.450+0000 7fd45a400700 1 --2- 172.21.15.86:0/3886089514 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fd4380a11b0 0x7fd4380a3670 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:23.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.450+0000 7fd45a400700 1 --2- 172.21.15.86:0/3886089514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd454101f70 0x7fd454199560 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:23.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.450+0000 7fd45a400700 1 -- 172.21.15.86:0/3886089514 >> 172.21.15.86:0/3886089514 conn(0x7fd4540fc2f0 msgr2=0x7fd454072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:23.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.450+0000 7fd45a400700 1 -- 172.21.15.86:0/3886089514 shutdown_connections 2024-03-23T17:50:23.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.451+0000 7fd45a400700 1 -- 172.21.15.86:0/3886089514 wait complete. 2024-03-23T17:50:23.470 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T17:50:23.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.663+0000 7f7096a00700 1 -- 172.21.15.86:0/4252021132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7090073810 msgr2=0x7f7090073c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:23.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.663+0000 7f7096a00700 1 --2- 172.21.15.86:0/4252021132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7090073810 0x7f7090073c90 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f707c009990 tx=0x7f707c009ca0 comp rx=0 tx=0).stop 2024-03-23T17:50:23.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.663+0000 7f7096a00700 1 -- 172.21.15.86:0/4252021132 shutdown_connections 2024-03-23T17:50:23.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.663+0000 7f7096a00700 1 --2- 172.21.15.86:0/4252021132 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7090108c40 0x7f709010b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:23.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.663+0000 7f7096a00700 1 --2- 172.21.15.86:0/4252021132 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7090074df0 0x7f7090073240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:23.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.663+0000 7f7096a00700 1 --2- 172.21.15.86:0/4252021132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7090073810 0x7f7090073c90 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:23.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.663+0000 7f7096a00700 1 -- 172.21.15.86:0/4252021132 >> 172.21.15.86:0/4252021132 conn(0x7f70900fc2f0 msgr2=0x7f70900fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:23.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.663+0000 7f7096a00700 1 -- 172.21.15.86:0/4252021132 shutdown_connections 2024-03-23T17:50:23.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.663+0000 7f7096a00700 1 -- 172.21.15.86:0/4252021132 wait complete. 2024-03-23T17:50:23.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.664+0000 7f7096a00700 1 Processor -- start 2024-03-23T17:50:23.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.665+0000 7f7096a00700 1 -- start start 2024-03-23T17:50:23.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.665+0000 7f7096a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7090073810 0x7f7090198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:23.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.665+0000 7f7096a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7090074df0 0x7f7090199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:23.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.665+0000 7f7096a00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7090108c40 0x7f70901a0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:23.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.665+0000 7f7096a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70901a0a40 con 0x7f7090073810 2024-03-23T17:50:23.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.665+0000 7f7096a00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70901a0b80 con 0x7f7090074df0 2024-03-23T17:50:23.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.665+0000 7f7096a00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70901a0d10 con 0x7f7090108c40 2024-03-23T17:50:23.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.665+0000 7f708fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7090073810 0x7f7090198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:23.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.665+0000 7f708fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7090073810 0x7f7090198f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35114/0 (socket says 172.21.15.86:35114) 2024-03-23T17:50:23.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.665+0000 7f708fe00700 1 -- 172.21.15.86:0/291440702 learned_addr learned my addr 172.21.15.86:0/291440702 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:23.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.665+0000 7f708f400700 1 --2- 172.21.15.86:0/291440702 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7090074df0 0x7f7090199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:23.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.665+0000 7f7094a00700 1 --2- 172.21.15.86:0/291440702 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7090108c40 0x7f70901a0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:23.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.665+0000 7f708fe00700 1 -- 172.21.15.86:0/291440702 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7090108c40 msgr2=0x7f70901a0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:23.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.665+0000 7f708fe00700 1 --2- 172.21.15.86:0/291440702 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7090108c40 0x7f70901a0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:23.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.665+0000 7f708fe00700 1 -- 172.21.15.86:0/291440702 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7090074df0 msgr2=0x7f7090199450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:23.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.666+0000 7f708fe00700 1 --2- 172.21.15.86:0/291440702 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7090074df0 0x7f7090199450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:23.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.666+0000 7f708fe00700 1 -- 172.21.15.86:0/291440702 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f707c0095f0 con 0x7f7090073810 2024-03-23T17:50:23.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.666+0000 7f708fe00700 1 --2- 172.21.15.86:0/291440702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7090073810 0x7f7090198f10 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f708000e9a0 tx=0x7f708000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:23.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.666+0000 7f708cc00700 1 -- 172.21.15.86:0/291440702 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7080009790 con 0x7f7090073810 2024-03-23T17:50:23.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.666+0000 7f708cc00700 1 -- 172.21.15.86:0/291440702 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f708000cbd0 con 0x7f7090073810 2024-03-23T17:50:23.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.666+0000 7f708cc00700 1 -- 172.21.15.86:0/291440702 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7080018680 con 0x7f7090073810 2024-03-23T17:50:23.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.666+0000 7f7096a00700 1 -- 172.21.15.86:0/291440702 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70901a0ff0 con 0x7f7090073810 2024-03-23T17:50:23.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.666+0000 7f7096a00700 1 -- 172.21.15.86:0/291440702 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70901a13c0 con 0x7f7090073810 2024-03-23T17:50:23.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.667+0000 7f7096a00700 1 -- 172.21.15.86:0/291440702 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7090107a00 con 0x7f7090073810 2024-03-23T17:50:23.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.669+0000 7f708cc00700 1 -- 172.21.15.86:0/291440702 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f708000cd40 con 0x7f7090073810 2024-03-23T17:50:23.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.670+0000 7f708cc00700 1 --2- 172.21.15.86:0/291440702 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f70740a11b0 0x7f70740a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:23.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.671+0000 7f708cc00700 1 -- 172.21.15.86:0/291440702 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f7080014070 con 0x7f7090073810 2024-03-23T17:50:23.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.671+0000 7f708f400700 1 --2- 172.21.15.86:0/291440702 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f70740a11b0 0x7f70740a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:23.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.671+0000 7f708f400700 1 --2- 172.21.15.86:0/291440702 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f70740a11b0 0x7f70740a3670 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f707c00b680 tx=0x7f707c00b750 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:23.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.673+0000 7f708cc00700 1 -- 172.21.15.86:0/291440702 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f70800965e0 con 0x7f7090073810 2024-03-23T17:50:23.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.885+0000 7f7096a00700 1 -- 172.21.15.86:0/291440702 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f709010e9b0 con 0x7f7090073810 2024-03-23T17:50:23.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.886+0000 7f708cc00700 1 -- 172.21.15.86:0/291440702 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v82) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7080099c00 con 0x7f7090073810 2024-03-23T17:50:23.886 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-03-23T17:50:23.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.888+0000 7f7096a00700 1 -- 172.21.15.86:0/291440702 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f70740a11b0 msgr2=0x7f70740a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:23.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.888+0000 7f7096a00700 1 --2- 172.21.15.86:0/291440702 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f70740a11b0 0x7f70740a3670 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f707c00b680 tx=0x7f707c00b750 comp rx=0 tx=0).stop 2024-03-23T17:50:23.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.888+0000 7f7096a00700 1 -- 172.21.15.86:0/291440702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7090073810 msgr2=0x7f7090198f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:23.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.888+0000 7f7096a00700 1 --2- 172.21.15.86:0/291440702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7090073810 0x7f7090198f10 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f708000e9a0 tx=0x7f708000ecb0 comp rx=0 tx=0).stop 2024-03-23T17:50:23.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.889+0000 7f7096a00700 1 -- 172.21.15.86:0/291440702 shutdown_connections 2024-03-23T17:50:23.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.889+0000 7f7096a00700 1 --2- 172.21.15.86:0/291440702 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7090108c40 0x7f70901a0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:23.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.889+0000 7f7096a00700 1 --2- 172.21.15.86:0/291440702 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f70740a11b0 0x7f70740a3670 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:23.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.889+0000 7f7096a00700 1 --2- 172.21.15.86:0/291440702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7090073810 0x7f7090198f10 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:23.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.889+0000 7f7096a00700 1 --2- 172.21.15.86:0/291440702 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7090074df0 0x7f7090199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:23.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.889+0000 7f7096a00700 1 -- 172.21.15.86:0/291440702 >> 172.21.15.86:0/291440702 conn(0x7f70900fc2f0 msgr2=0x7f7090104b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:23.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.889+0000 7f7096a00700 1 -- 172.21.15.86:0/291440702 shutdown_connections 2024-03-23T17:50:23.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:23.889+0000 7f7096a00700 1 -- 172.21.15.86:0/291440702 wait complete. 2024-03-23T17:50:23.907 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T17:50:24.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:23 smithi086 ceph-mon[38221]: pgmap v279: 97 pgs: 97 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 10 op/s 2024-03-23T17:50:24.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:23 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/3408614366' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T17:50:24.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:23 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/3886089514' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T17:50:24.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:23 smithi092 ceph-mon[79089]: pgmap v279: 97 pgs: 97 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 10 op/s 2024-03-23T17:50:24.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:23 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/3408614366' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T17:50:24.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:23 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/3886089514' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T17:50:24.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:23 smithi152 ceph-mon[47949]: pgmap v279: 97 pgs: 97 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 10 op/s 2024-03-23T17:50:24.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:23 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/3408614366' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T17:50:24.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:23 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/3886089514' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T17:50:24.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.103+0000 7fc169600700 1 -- 172.21.15.86:0/1896002482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc164074df0 msgr2=0x7fc164073240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:24.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.103+0000 7fc169600700 1 --2- 172.21.15.86:0/1896002482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc164074df0 0x7fc164073240 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc154009990 tx=0x7fc154009ca0 comp rx=0 tx=0).stop 2024-03-23T17:50:24.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.103+0000 7fc169600700 1 -- 172.21.15.86:0/1896002482 shutdown_connections 2024-03-23T17:50:24.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.103+0000 7fc169600700 1 --2- 172.21.15.86:0/1896002482 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc164108c40 0x7fc16410b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:24.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.103+0000 7fc169600700 1 --2- 172.21.15.86:0/1896002482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc164074df0 0x7fc164073240 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:24.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.103+0000 7fc169600700 1 --2- 172.21.15.86:0/1896002482 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc164073810 0x7fc164073c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:24.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.103+0000 7fc169600700 1 -- 172.21.15.86:0/1896002482 >> 172.21.15.86:0/1896002482 conn(0x7fc1640fc2f0 msgr2=0x7fc1640fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:24.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.103+0000 7fc169600700 1 -- 172.21.15.86:0/1896002482 shutdown_connections 2024-03-23T17:50:24.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.103+0000 7fc169600700 1 -- 172.21.15.86:0/1896002482 wait complete. 2024-03-23T17:50:24.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.104+0000 7fc169600700 1 Processor -- start 2024-03-23T17:50:24.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.104+0000 7fc169600700 1 -- start start 2024-03-23T17:50:24.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.105+0000 7fc169600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc164073810 0x7fc164198f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:24.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.105+0000 7fc169600700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc164074df0 0x7fc1641994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:24.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.105+0000 7fc169600700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc164108c40 0x7fc1641a0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:24.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.105+0000 7fc169600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1641a0a50 con 0x7fc164073810 2024-03-23T17:50:24.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.105+0000 7fc169600700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1641a0b90 con 0x7fc164108c40 2024-03-23T17:50:24.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.105+0000 7fc169600700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1641a0d20 con 0x7fc164074df0 2024-03-23T17:50:24.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.105+0000 7fc162a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc164073810 0x7fc164198f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:24.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.105+0000 7fc162a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc164073810 0x7fc164198f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35146/0 (socket says 172.21.15.86:35146) 2024-03-23T17:50:24.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.105+0000 7fc162a00700 1 -- 172.21.15.86:0/4257459655 learned_addr learned my addr 172.21.15.86:0/4257459655 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:24.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.105+0000 7fc162000700 1 --2- 172.21.15.86:0/4257459655 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc164074df0 0x7fc1641994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:24.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.105+0000 7fc163400700 1 --2- 172.21.15.86:0/4257459655 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc164108c40 0x7fc1641a0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:24.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.105+0000 7fc162a00700 1 -- 172.21.15.86:0/4257459655 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc164074df0 msgr2=0x7fc1641994b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:24.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.105+0000 7fc162a00700 1 --2- 172.21.15.86:0/4257459655 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc164074df0 0x7fc1641994b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:24.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.105+0000 7fc162a00700 1 -- 172.21.15.86:0/4257459655 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc164108c40 msgr2=0x7fc1641a0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:24.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.105+0000 7fc162a00700 1 --2- 172.21.15.86:0/4257459655 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc164108c40 0x7fc1641a0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:24.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.105+0000 7fc162a00700 1 -- 172.21.15.86:0/4257459655 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1540095f0 con 0x7fc164073810 2024-03-23T17:50:24.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.106+0000 7fc162a00700 1 --2- 172.21.15.86:0/4257459655 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc164073810 0x7fc164198f70 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc154006010 tx=0x7fc154004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:24.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.106+0000 7fc15b400700 1 -- 172.21.15.86:0/4257459655 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc15401d070 con 0x7fc164073810 2024-03-23T17:50:24.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.106+0000 7fc15b400700 1 -- 172.21.15.86:0/4257459655 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc154005030 con 0x7fc164073810 2024-03-23T17:50:24.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.106+0000 7fc169600700 1 -- 172.21.15.86:0/4257459655 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1641a0fa0 con 0x7fc164073810 2024-03-23T17:50:24.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.106+0000 7fc15b400700 1 -- 172.21.15.86:0/4257459655 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc154004b40 con 0x7fc164073810 2024-03-23T17:50:24.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.106+0000 7fc169600700 1 -- 172.21.15.86:0/4257459655 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1641a1310 con 0x7fc164073810 2024-03-23T17:50:24.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.107+0000 7fc169600700 1 -- 172.21.15.86:0/4257459655 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc164107a00 con 0x7fc164073810 2024-03-23T17:50:24.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.109+0000 7fc15b400700 1 -- 172.21.15.86:0/4257459655 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fc154004d00 con 0x7fc164073810 2024-03-23T17:50:24.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.110+0000 7fc15b400700 1 --2- 172.21.15.86:0/4257459655 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fc1480a11b0 0x7fc1480a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:24.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.110+0000 7fc15b400700 1 -- 172.21.15.86:0/4257459655 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fc1540cdf00 con 0x7fc164073810 2024-03-23T17:50:24.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.110+0000 7fc162000700 1 --2- 172.21.15.86:0/4257459655 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fc1480a11b0 0x7fc1480a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:24.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.111+0000 7fc162000700 1 --2- 172.21.15.86:0/4257459655 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fc1480a11b0 0x7fc1480a3670 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc164074b10 tx=0x7fc150009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:24.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.113+0000 7fc15b400700 1 -- 172.21.15.86:0/4257459655 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc1540975f0 con 0x7fc164073810 2024-03-23T17:50:24.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.326+0000 7fc169600700 1 -- 172.21.15.86:0/4257459655 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc16410e7c0 con 0x7fc164073810 2024-03-23T17:50:24.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.327+0000 7fc15b400700 1 -- 172.21.15.86:0/4257459655 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v82) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc15409ac10 con 0x7fc164073810 2024-03-23T17:50:24.327 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-03-23T17:50:24.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.329+0000 7fc169600700 1 -- 172.21.15.86:0/4257459655 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fc1480a11b0 msgr2=0x7fc1480a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:24.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.329+0000 7fc169600700 1 --2- 172.21.15.86:0/4257459655 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fc1480a11b0 0x7fc1480a3670 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc164074b10 tx=0x7fc150009260 comp rx=0 tx=0).stop 2024-03-23T17:50:24.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.329+0000 7fc169600700 1 -- 172.21.15.86:0/4257459655 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc164073810 msgr2=0x7fc164198f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:24.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.329+0000 7fc169600700 1 --2- 172.21.15.86:0/4257459655 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc164073810 0x7fc164198f70 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc154006010 tx=0x7fc154004730 comp rx=0 tx=0).stop 2024-03-23T17:50:24.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.329+0000 7fc169600700 1 -- 172.21.15.86:0/4257459655 shutdown_connections 2024-03-23T17:50:24.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.329+0000 7fc169600700 1 --2- 172.21.15.86:0/4257459655 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc164108c40 0x7fc1641a0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:24.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.329+0000 7fc169600700 1 --2- 172.21.15.86:0/4257459655 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fc1480a11b0 0x7fc1480a3670 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:24.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.329+0000 7fc169600700 1 --2- 172.21.15.86:0/4257459655 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc164073810 0x7fc164198f70 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:24.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.329+0000 7fc169600700 1 --2- 172.21.15.86:0/4257459655 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc164074df0 0x7fc1641994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:24.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.329+0000 7fc169600700 1 -- 172.21.15.86:0/4257459655 >> 172.21.15.86:0/4257459655 conn(0x7fc1640fc2f0 msgr2=0x7fc164104b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:24.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.329+0000 7fc169600700 1 -- 172.21.15.86:0/4257459655 shutdown_connections 2024-03-23T17:50:24.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.329+0000 7fc169600700 1 -- 172.21.15.86:0/4257459655 wait complete. 2024-03-23T17:50:24.348 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-03-23T17:50:24.348 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T17:50:24.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.540+0000 7f7d81800700 1 -- 172.21.15.86:0/2155257051 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d7c102e90 msgr2=0x7f7d7c1032b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:24.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.540+0000 7f7d81800700 1 --2- 172.21.15.86:0/2155257051 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d7c102e90 0x7f7d7c1032b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c0098f0 tx=0x7f7d6c009c00 comp rx=0 tx=0).stop 2024-03-23T17:50:24.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.540+0000 7f7d81800700 1 -- 172.21.15.86:0/2155257051 shutdown_connections 2024-03-23T17:50:24.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.540+0000 7f7d81800700 1 --2- 172.21.15.86:0/2155257051 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d7c1049a0 0x7f7d7c10b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:24.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.540+0000 7f7d81800700 1 --2- 172.21.15.86:0/2155257051 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d7c102e90 0x7f7d7c1032b0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:24.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.540+0000 7f7d81800700 1 --2- 172.21.15.86:0/2155257051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d7c103fe0 0x7f7d7c104460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:24.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.540+0000 7f7d81800700 1 -- 172.21.15.86:0/2155257051 >> 172.21.15.86:0/2155257051 conn(0x7f7d7c0fe420 msgr2=0x7f7d7c100840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:24.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.540+0000 7f7d81800700 1 -- 172.21.15.86:0/2155257051 shutdown_connections 2024-03-23T17:50:24.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.540+0000 7f7d81800700 1 -- 172.21.15.86:0/2155257051 wait complete. 2024-03-23T17:50:24.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.541+0000 7f7d81800700 1 Processor -- start 2024-03-23T17:50:24.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.541+0000 7f7d81800700 1 -- start start 2024-03-23T17:50:24.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.541+0000 7f7d81800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d7c102e90 0x7f7d7c198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:24.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.541+0000 7f7d81800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d7c103fe0 0x7f7d7c199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:24.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.541+0000 7f7d81800700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d7c1049a0 0x7f7d7c1a0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:24.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.541+0000 7f7d81800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7c1a0960 con 0x7f7d7c102e90 2024-03-23T17:50:24.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.541+0000 7f7d81800700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7c1a0aa0 con 0x7f7d7c103fe0 2024-03-23T17:50:24.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.541+0000 7f7d81800700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7c1a0c30 con 0x7f7d7c1049a0 2024-03-23T17:50:24.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.542+0000 7f7d7aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d7c102e90 0x7f7d7c198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:24.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.542+0000 7f7d7aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d7c102e90 0x7f7d7c198e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35166/0 (socket says 172.21.15.86:35166) 2024-03-23T17:50:24.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.542+0000 7f7d7aa00700 1 -- 172.21.15.86:0/1885316573 learned_addr learned my addr 172.21.15.86:0/1885316573 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:24.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.542+0000 7f7d7a000700 1 --2- 172.21.15.86:0/1885316573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d7c103fe0 0x7f7d7c199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:24.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.542+0000 7f7d7b400700 1 --2- 172.21.15.86:0/1885316573 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d7c1049a0 0x7f7d7c1a0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:24.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.542+0000 7f7d7aa00700 1 -- 172.21.15.86:0/1885316573 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d7c1049a0 msgr2=0x7f7d7c1a0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:24.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.542+0000 7f7d7aa00700 1 --2- 172.21.15.86:0/1885316573 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d7c1049a0 0x7f7d7c1a0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:24.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.542+0000 7f7d7aa00700 1 -- 172.21.15.86:0/1885316573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d7c103fe0 msgr2=0x7f7d7c199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:24.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.542+0000 7f7d7aa00700 1 --2- 172.21.15.86:0/1885316573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d7c103fe0 0x7f7d7c199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:24.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.542+0000 7f7d7aa00700 1 -- 172.21.15.86:0/1885316573 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d6c009520 con 0x7f7d7c102e90 2024-03-23T17:50:24.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.542+0000 7f7d7b400700 1 --2- 172.21.15.86:0/1885316573 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d7c1049a0 0x7f7d7c1a0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:50:24.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.542+0000 7f7d7aa00700 1 --2- 172.21.15.86:0/1885316573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d7c102e90 0x7f7d7c198e30 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c005e90 tx=0x7f7d6c00b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:24.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.543+0000 7f7d73400700 1 -- 172.21.15.86:0/1885316573 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c01c070 con 0x7f7d7c102e90 2024-03-23T17:50:24.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.543+0000 7f7d73400700 1 -- 172.21.15.86:0/1885316573 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d6c00f7f0 con 0x7f7d7c102e90 2024-03-23T17:50:24.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.543+0000 7f7d81800700 1 -- 172.21.15.86:0/1885316573 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d7c1a0eb0 con 0x7f7d7c102e90 2024-03-23T17:50:24.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.543+0000 7f7d73400700 1 -- 172.21.15.86:0/1885316573 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c003a00 con 0x7f7d7c102e90 2024-03-23T17:50:24.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.543+0000 7f7d81800700 1 -- 172.21.15.86:0/1885316573 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d7c1a1220 con 0x7f7d7c102e90 2024-03-23T17:50:24.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.544+0000 7f7d81800700 1 -- 172.21.15.86:0/1885316573 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d7c067ed0 con 0x7f7d7c102e90 2024-03-23T17:50:24.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.546+0000 7f7d73400700 1 -- 172.21.15.86:0/1885316573 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f7d6c020030 con 0x7f7d7c102e90 2024-03-23T17:50:24.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.547+0000 7f7d73400700 1 --2- 172.21.15.86:0/1885316573 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f7d600a11b0 0x7f7d600a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:24.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.547+0000 7f7d73400700 1 -- 172.21.15.86:0/1885316573 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f7d6c0cda90 con 0x7f7d7c102e90 2024-03-23T17:50:24.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.547+0000 7f7d7a000700 1 --2- 172.21.15.86:0/1885316573 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f7d600a11b0 0x7f7d600a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:24.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.548+0000 7f7d7a000700 1 --2- 172.21.15.86:0/1885316573 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f7d600a11b0 0x7f7d600a3670 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7d68005fd0 tx=0x7f7d68009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:24.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.551+0000 7f7d73400700 1 -- 172.21.15.86:0/1885316573 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f7d7c067ed0 con 0x7f7d7c102e90 2024-03-23T17:50:24.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.808+0000 7f7d81800700 1 -- 172.21.15.86:0/1885316573 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7d7c067e60 con 0x7f7d7c102e90 2024-03-23T17:50:24.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.808+0000 7f7d73400700 1 -- 172.21.15.86:0/1885316573 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+10913 (secure 0 0 0) 0x7f7d6c097120 con 0x7f7d7c102e90 2024-03-23T17:50:24.808 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:50:24.809 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14445,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1433140219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1433140219},{"type":"v1","addr":"172.21.15.86:6835","nonce":1433140219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6839/1879457730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1879457730},{"type":"v1","addr":"172.21.15.86:6839","nonce":1879457730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-23T17:48:56.202323+0000","modified":"2024-03-23T17:50:20.718165+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24553,"mds_2":24518,"mds_3":24505,"mds_4":24500},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14514":{"gid":14514,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/3026943866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3026943866},{"type":"v1","addr":"172.21.15.86:6841","nonce":3026943866}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24464":{"gid":24464,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6833/1467091186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1467091186},{"type":"v1","addr":"172.21.15.92:6833","nonce":1467091186}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6833/3075554623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3075554623},{"type":"v1","addr":"172.21.15.152:6833","nonce":3075554623}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.92:6835/908900027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":908900027},{"type":"v1","addr":"172.21.15.92:6835","nonce":908900027}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"d","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.86:6837/2933881328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2933881328},{"type":"v1","addr":"172.21.15.86:6837","nonce":2933881328}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6837/3967837948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":3967837948},{"type":"v1","addr":"172.21.15.92:6837","nonce":3967837948}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6835/3137085021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3137085021},{"type":"v1","addr":"172.21.15.152:6835","nonce":3137085021}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24538":{"gid":24538,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6837/1151208965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":1151208965},{"type":"v1","addr":"172.21.15.152:6837","nonce":1151208965}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.152:6839/1100723992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":1100723992},{"type":"v1","addr":"172.21.15.152:6839","nonce":1100723992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6839/1394139887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1394139887},{"type":"v1","addr":"172.21.15.92:6839","nonce":1394139887}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-23T17:50:24.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.811+0000 7f7d81800700 1 -- 172.21.15.86:0/1885316573 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f7d600a11b0 msgr2=0x7f7d600a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:24.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.811+0000 7f7d81800700 1 --2- 172.21.15.86:0/1885316573 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f7d600a11b0 0x7f7d600a3670 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7d68005fd0 tx=0x7f7d68009310 comp rx=0 tx=0).stop 2024-03-23T17:50:24.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.811+0000 7f7d81800700 1 -- 172.21.15.86:0/1885316573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d7c102e90 msgr2=0x7f7d7c198e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:24.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.811+0000 7f7d81800700 1 --2- 172.21.15.86:0/1885316573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d7c102e90 0x7f7d7c198e30 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c005e90 tx=0x7f7d6c00b7e0 comp rx=0 tx=0).stop 2024-03-23T17:50:24.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.811+0000 7f7d81800700 1 -- 172.21.15.86:0/1885316573 shutdown_connections 2024-03-23T17:50:24.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.811+0000 7f7d81800700 1 --2- 172.21.15.86:0/1885316573 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d7c1049a0 0x7f7d7c1a0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:24.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.811+0000 7f7d81800700 1 --2- 172.21.15.86:0/1885316573 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f7d600a11b0 0x7f7d600a3670 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:24.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.811+0000 7f7d81800700 1 --2- 172.21.15.86:0/1885316573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d7c102e90 0x7f7d7c198e30 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:24.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.811+0000 7f7d81800700 1 --2- 172.21.15.86:0/1885316573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d7c103fe0 0x7f7d7c199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:24.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.811+0000 7f7d81800700 1 -- 172.21.15.86:0/1885316573 >> 172.21.15.86:0/1885316573 conn(0x7f7d7c0fe420 msgr2=0x7f7d7c107490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:24.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.811+0000 7f7d81800700 1 -- 172.21.15.86:0/1885316573 shutdown_connections 2024-03-23T17:50:24.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:24.811+0000 7f7d81800700 1 -- 172.21.15.86:0/1885316573 wait complete. 2024-03-23T17:50:24.812 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 28 2024-03-23T17:50:24.830 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-03-23T17:50:24.830 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-03-23T17:50:24.831 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T17:50:25.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:24 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/291440702' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T17:50:25.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:24 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/4257459655' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T17:50:25.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.024+0000 7fcde0c00700 1 -- 172.21.15.86:0/1826177038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcddc102810 msgr2=0x7fcddc1090c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:25.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.024+0000 7fcde0c00700 1 --2- 172.21.15.86:0/1826177038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcddc102810 0x7fcddc1090c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fcdd400b0d0 tx=0x7fcdd400b3e0 comp rx=0 tx=0).stop 2024-03-23T17:50:25.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.024+0000 7fcde0c00700 1 -- 172.21.15.86:0/1826177038 shutdown_connections 2024-03-23T17:50:25.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.024+0000 7fcde0c00700 1 --2- 172.21.15.86:0/1826177038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcddc102810 0x7fcddc1090c0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:25.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.024+0000 7fcde0c00700 1 --2- 172.21.15.86:0/1826177038 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcddc100c80 0x7fcddc1010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:25.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.024+0000 7fcde0c00700 1 --2- 172.21.15.86:0/1826177038 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcddc101e70 0x7fcddc1022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:25.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.024+0000 7fcde0c00700 1 -- 172.21.15.86:0/1826177038 >> 172.21.15.86:0/1826177038 conn(0x7fcddc0fc1f0 msgr2=0x7fcddc0fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:25.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.025+0000 7fcde0c00700 1 -- 172.21.15.86:0/1826177038 shutdown_connections 2024-03-23T17:50:25.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.025+0000 7fcde0c00700 1 -- 172.21.15.86:0/1826177038 wait complete. 2024-03-23T17:50:25.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.025+0000 7fcde0c00700 1 Processor -- start 2024-03-23T17:50:25.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.026+0000 7fcde0c00700 1 -- start start 2024-03-23T17:50:25.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.026+0000 7fcde0c00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcddc100c80 0x7fcddc074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:25.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.026+0000 7fcde0c00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcddc101e70 0x7fcddc0731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:25.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.026+0000 7fcde0c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcddc102810 0x7fcddc0736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:25.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.026+0000 7fcde0c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcddc19ab90 con 0x7fcddc102810 2024-03-23T17:50:25.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.026+0000 7fcde0c00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcddc073c20 con 0x7fcddc101e70 2024-03-23T17:50:25.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.026+0000 7fcde0c00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcddc073de0 con 0x7fcddc100c80 2024-03-23T17:50:25.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.026+0000 7fcdda800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcddc102810 0x7fcddc0736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:25.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.026+0000 7fcdda800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcddc102810 0x7fcddc0736e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43032/0 (socket says 172.21.15.86:43032) 2024-03-23T17:50:25.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.026+0000 7fcdda800700 1 -- 172.21.15.86:0/4019021870 learned_addr learned my addr 172.21.15.86:0/4019021870 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:25.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.026+0000 7fcdd9e00700 1 --2- 172.21.15.86:0/4019021870 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcddc100c80 0x7fcddc074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:25.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.026+0000 7fcdd9400700 1 --2- 172.21.15.86:0/4019021870 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcddc101e70 0x7fcddc0731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:25.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.027+0000 7fcdd9e00700 1 -- 172.21.15.86:0/4019021870 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcddc101e70 msgr2=0x7fcddc0731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:25.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.027+0000 7fcdd9e00700 1 --2- 172.21.15.86:0/4019021870 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcddc101e70 0x7fcddc0731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:25.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.027+0000 7fcdd9e00700 1 -- 172.21.15.86:0/4019021870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcddc102810 msgr2=0x7fcddc0736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:25.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.027+0000 7fcdd9e00700 1 --2- 172.21.15.86:0/4019021870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcddc102810 0x7fcddc0736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:25.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.027+0000 7fcdd9e00700 1 -- 172.21.15.86:0/4019021870 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdd4009d00 con 0x7fcddc100c80 2024-03-23T17:50:25.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.027+0000 7fcdda800700 1 --2- 172.21.15.86:0/4019021870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcddc102810 0x7fcddc0736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:50:25.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.027+0000 7fcdd9e00700 1 --2- 172.21.15.86:0/4019021870 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcddc100c80 0x7fcddc074b10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc00e9c0 tx=0x7fcdcc00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:25.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.027+0000 7fcdd2a00700 1 -- 172.21.15.86:0/4019021870 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc004220 con 0x7fcddc100c80 2024-03-23T17:50:25.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.028+0000 7fcde0c00700 1 -- 172.21.15.86:0/4019021870 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcddc1a5540 con 0x7fcddc100c80 2024-03-23T17:50:25.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.028+0000 7fcdd2a00700 1 -- 172.21.15.86:0/4019021870 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcdcc004380 con 0x7fcddc100c80 2024-03-23T17:50:25.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.028+0000 7fcdd2a00700 1 -- 172.21.15.86:0/4019021870 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc003be0 con 0x7fcddc100c80 2024-03-23T17:50:25.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.028+0000 7fcde0c00700 1 -- 172.21.15.86:0/4019021870 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcddc1a5960 con 0x7fcddc100c80 2024-03-23T17:50:25.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.029+0000 7fcde0c00700 1 -- 172.21.15.86:0/4019021870 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcddc108030 con 0x7fcddc100c80 2024-03-23T17:50:25.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.031+0000 7fcdd2a00700 1 -- 172.21.15.86:0/4019021870 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fcdcc003d40 con 0x7fcddc100c80 2024-03-23T17:50:25.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.032+0000 7fcdd2a00700 1 --2- 172.21.15.86:0/4019021870 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fcdc00a11b0 0x7fcdc00a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:25.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.032+0000 7fcdd2a00700 1 -- 172.21.15.86:0/4019021870 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fcdcc014070 con 0x7fcddc100c80 2024-03-23T17:50:25.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.032+0000 7fcdd9400700 1 --2- 172.21.15.86:0/4019021870 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fcdc00a11b0 0x7fcdc00a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:25.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.033+0000 7fcdd9400700 1 --2- 172.21.15.86:0/4019021870 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fcdc00a11b0 0x7fcdc00a3670 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8009c60 tx=0x7fcdc8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:25.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.034+0000 7fcdd2a00700 1 -- 172.21.15.86:0/4019021870 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fcdcc096490 con 0x7fcddc100c80 2024-03-23T17:50:25.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:24 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/291440702' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T17:50:25.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:24 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/4257459655' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T17:50:25.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:24 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/291440702' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T17:50:25.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:24 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/4257459655' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T17:50:25.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.238+0000 7fcde0c00700 1 -- 172.21.15.86:0/4019021870 --> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcddc0611f0 con 0x7fcdc00a11b0 2024-03-23T17:50:25.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.249+0000 7fcdd2a00700 1 -- 172.21.15.86:0/4019021870 <== mgr.14152 v2:172.21.15.86:6800/2734784244 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+344441 (secure 0 0 0) 0x7fcddc0611f0 con 0x7fcdc00a11b0 2024-03-23T17:50:25.258 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:50:25.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.254+0000 7fcde0c00700 1 -- 172.21.15.86:0/4019021870 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fcdc00a11b0 msgr2=0x7fcdc00a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:25.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.254+0000 7fcde0c00700 1 --2- 172.21.15.86:0/4019021870 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fcdc00a11b0 0x7fcdc00a3670 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8009c60 tx=0x7fcdc8009260 comp rx=0 tx=0).stop 2024-03-23T17:50:25.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.254+0000 7fcde0c00700 1 -- 172.21.15.86:0/4019021870 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcddc100c80 msgr2=0x7fcddc074b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:25.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.254+0000 7fcde0c00700 1 --2- 172.21.15.86:0/4019021870 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcddc100c80 0x7fcddc074b10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc00e9c0 tx=0x7fcdcc00ecd0 comp rx=0 tx=0).stop 2024-03-23T17:50:25.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.254+0000 7fcde0c00700 1 -- 172.21.15.86:0/4019021870 shutdown_connections 2024-03-23T17:50:25.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.254+0000 7fcde0c00700 1 --2- 172.21.15.86:0/4019021870 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fcdc00a11b0 0x7fcdc00a3670 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:25.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.254+0000 7fcde0c00700 1 --2- 172.21.15.86:0/4019021870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcddc102810 0x7fcddc0736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:25.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.254+0000 7fcde0c00700 1 --2- 172.21.15.86:0/4019021870 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcddc100c80 0x7fcddc074b10 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:25.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.254+0000 7fcde0c00700 1 --2- 172.21.15.86:0/4019021870 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcddc101e70 0x7fcddc0731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:25.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.254+0000 7fcde0c00700 1 -- 172.21.15.86:0/4019021870 >> 172.21.15.86:0/4019021870 conn(0x7fcddc0fc1f0 msgr2=0x7fcddc105170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:25.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.254+0000 7fcde0c00700 1 -- 172.21.15.86:0/4019021870 shutdown_connections 2024-03-23T17:50:25.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.254+0000 7fcde0c00700 1 -- 172.21.15.86:0/4019021870 wait complete. 2024-03-23T17:50:25.264 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-03-23T17:50:25.274 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":280,"stamp":"2024-03-23T17:50:24.329151+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661176,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":955,"num_read_kb":600,"num_write":418,"num_write_kb":3078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":1180916,"num_keys_recovered":17,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":345,"ondisk_log_size":345,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":373672,"kb_used_data":12568,"kb_used_omap":18,"kb_used_meta":356845,"kb_avail":1124568152,"statfs":{"total":1151940427776,"available":1151557787648,"internally_reserved":0,"allocated":12869632,"data_stored":7106887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19130,"internal_metadata":365409606},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4926,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":106,"num_read_kb":51,"num_write":7,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002979"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.782853+0000","last_change":"2024-03-23T17:48:56.239724+0000","last_active":"2024-03-23T17:49:28.782853+0000","last_peered":"2024-03-23T17:49:28.782853+0000","last_clean":"2024-03-23T17:49:28.782853+0000","last_became_active":"2024-03-23T17:48:56.239451+0000","last_became_peered":"2024-03-23T17:48:56.239451+0000","last_unstale":"2024-03-23T17:49:28.782853+0000","last_undegraded":"2024-03-23T17:49:28.782853+0000","last_fullsized":"2024-03-23T17:49:28.782853+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:51:38.687863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.783538+0000","last_change":"2024-03-23T17:48:56.232415+0000","last_active":"2024-03-23T17:49:28.783538+0000","last_peered":"2024-03-23T17:49:28.783538+0000","last_clean":"2024-03-23T17:49:28.783538+0000","last_became_active":"2024-03-23T17:48:56.231777+0000","last_became_peered":"2024-03-23T17:48:56.231777+0000","last_unstale":"2024-03-23T17:49:28.783538+0000","last_undegraded":"2024-03-23T17:49:28.783538+0000","last_fullsized":"2024-03-23T17:49:28.783538+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:16:22.631654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.784174+0000","last_change":"2024-03-23T17:48:56.243413+0000","last_active":"2024-03-23T17:49:28.784174+0000","last_peered":"2024-03-23T17:49:28.784174+0000","last_clean":"2024-03-23T17:49:28.784174+0000","last_became_active":"2024-03-23T17:48:56.241281+0000","last_became_peered":"2024-03-23T17:48:56.241281+0000","last_unstale":"2024-03-23T17:49:28.784174+0000","last_undegraded":"2024-03-23T17:49:28.784174+0000","last_fullsized":"2024-03-23T17:49:28.784174+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:52:48.521689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.784355+0000","last_change":"2024-03-23T17:48:56.226010+0000","last_active":"2024-03-23T17:49:28.784355+0000","last_peered":"2024-03-23T17:49:28.784355+0000","last_clean":"2024-03-23T17:49:28.784355+0000","last_became_active":"2024-03-23T17:48:56.225562+0000","last_became_peered":"2024-03-23T17:48:56.225562+0000","last_unstale":"2024-03-23T17:49:28.784355+0000","last_undegraded":"2024-03-23T17:49:28.784355+0000","last_fullsized":"2024-03-23T17:49:28.784355+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:17:47.597335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.780757+0000","last_change":"2024-03-23T17:48:56.223249+0000","last_active":"2024-03-23T17:49:28.780757+0000","last_peered":"2024-03-23T17:49:28.780757+0000","last_clean":"2024-03-23T17:49:28.780757+0000","last_became_active":"2024-03-23T17:48:56.222902+0000","last_became_peered":"2024-03-23T17:48:56.222902+0000","last_unstale":"2024-03-23T17:49:28.780757+0000","last_undegraded":"2024-03-23T17:49:28.780757+0000","last_fullsized":"2024-03-23T17:49:28.780757+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:51:49.029628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.781946+0000","last_change":"2024-03-23T17:48:56.229229+0000","last_active":"2024-03-23T17:49:28.781946+0000","last_peered":"2024-03-23T17:49:28.781946+0000","last_clean":"2024-03-23T17:49:28.781946+0000","last_became_active":"2024-03-23T17:48:56.228725+0000","last_became_peered":"2024-03-23T17:48:56.228725+0000","last_unstale":"2024-03-23T17:49:28.781946+0000","last_undegraded":"2024-03-23T17:49:28.781946+0000","last_fullsized":"2024-03-23T17:49:28.781946+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:53:44.118762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.780711+0000","last_change":"2024-03-23T17:48:56.229212+0000","last_active":"2024-03-23T17:49:28.780711+0000","last_peered":"2024-03-23T17:49:28.780711+0000","last_clean":"2024-03-23T17:49:28.780711+0000","last_became_active":"2024-03-23T17:48:56.228888+0000","last_became_peered":"2024-03-23T17:48:56.228888+0000","last_unstale":"2024-03-23T17:49:28.780711+0000","last_undegraded":"2024-03-23T17:49:28.780711+0000","last_fullsized":"2024-03-23T17:49:28.780711+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:47:21.509554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.779498+0000","last_change":"2024-03-23T17:48:56.222925+0000","last_active":"2024-03-23T17:49:28.779498+0000","last_peered":"2024-03-23T17:49:28.779498+0000","last_clean":"2024-03-23T17:49:28.779498+0000","last_became_active":"2024-03-23T17:48:56.221504+0000","last_became_peered":"2024-03-23T17:48:56.221504+0000","last_unstale":"2024-03-23T17:49:28.779498+0000","last_undegraded":"2024-03-23T17:49:28.779498+0000","last_fullsized":"2024-03-23T17:49:28.779498+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:14:34.327705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.922478+0000","last_change":"2024-03-23T17:48:56.230530+0000","last_active":"2024-03-23T17:49:28.922478+0000","last_peered":"2024-03-23T17:49:28.922478+0000","last_clean":"2024-03-23T17:49:28.922478+0000","last_became_active":"2024-03-23T17:48:56.230224+0000","last_became_peered":"2024-03-23T17:48:56.230224+0000","last_unstale":"2024-03-23T17:49:28.922478+0000","last_undegraded":"2024-03-23T17:49:28.922478+0000","last_fullsized":"2024-03-23T17:49:28.922478+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:10:28.031956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.781514+0000","last_change":"2024-03-23T17:48:56.229673+0000","last_active":"2024-03-23T17:49:28.781514+0000","last_peered":"2024-03-23T17:49:28.781514+0000","last_clean":"2024-03-23T17:49:28.781514+0000","last_became_active":"2024-03-23T17:48:56.229000+0000","last_became_peered":"2024-03-23T17:48:56.229000+0000","last_unstale":"2024-03-23T17:49:28.781514+0000","last_undegraded":"2024-03-23T17:49:28.781514+0000","last_fullsized":"2024-03-23T17:49:28.781514+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:23:28.057256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.781035+0000","last_change":"2024-03-23T17:48:56.224508+0000","last_active":"2024-03-23T17:49:28.781035+0000","last_peered":"2024-03-23T17:49:28.781035+0000","last_clean":"2024-03-23T17:49:28.781035+0000","last_became_active":"2024-03-23T17:48:56.224172+0000","last_became_peered":"2024-03-23T17:48:56.224172+0000","last_unstale":"2024-03-23T17:49:28.781035+0000","last_undegraded":"2024-03-23T17:49:28.781035+0000","last_fullsized":"2024-03-23T17:49:28.781035+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:03:57.063472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.784254+0000","last_change":"2024-03-23T17:48:56.234978+0000","last_active":"2024-03-23T17:49:28.784254+0000","last_peered":"2024-03-23T17:49:28.784254+0000","last_clean":"2024-03-23T17:49:28.784254+0000","last_became_active":"2024-03-23T17:48:56.234641+0000","last_became_peered":"2024-03-23T17:48:56.234641+0000","last_unstale":"2024-03-23T17:49:28.784254+0000","last_undegraded":"2024-03-23T17:49:28.784254+0000","last_fullsized":"2024-03-23T17:49:28.784254+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:34:30.745991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.777608+0000","last_change":"2024-03-23T17:48:56.218534+0000","last_active":"2024-03-23T17:49:28.777608+0000","last_peered":"2024-03-23T17:49:28.777608+0000","last_clean":"2024-03-23T17:49:28.777608+0000","last_became_active":"2024-03-23T17:48:56.217424+0000","last_became_peered":"2024-03-23T17:48:56.217424+0000","last_unstale":"2024-03-23T17:49:28.777608+0000","last_undegraded":"2024-03-23T17:49:28.777608+0000","last_fullsized":"2024-03-23T17:49:28.777608+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:32:05.618057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.783444+0000","last_change":"2024-03-23T17:48:56.235525+0000","last_active":"2024-03-23T17:49:28.783444+0000","last_peered":"2024-03-23T17:49:28.783444+0000","last_clean":"2024-03-23T17:49:28.783444+0000","last_became_active":"2024-03-23T17:48:56.234591+0000","last_became_peered":"2024-03-23T17:48:56.234591+0000","last_unstale":"2024-03-23T17:49:28.783444+0000","last_undegraded":"2024-03-23T17:49:28.783444+0000","last_fullsized":"2024-03-23T17:49:28.783444+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:19:38.381878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.783446+0000","last_change":"2024-03-23T17:48:56.243137+0000","last_active":"2024-03-23T17:49:28.783446+0000","last_peered":"2024-03-23T17:49:28.783446+0000","last_clean":"2024-03-23T17:49:28.783446+0000","last_became_active":"2024-03-23T17:48:56.242516+0000","last_became_peered":"2024-03-23T17:48:56.242516+0000","last_unstale":"2024-03-23T17:49:28.783446+0000","last_undegraded":"2024-03-23T17:49:28.783446+0000","last_fullsized":"2024-03-23T17:49:28.783446+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:54:59.849467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.922041+0000","last_change":"2024-03-23T17:48:56.236026+0000","last_active":"2024-03-23T17:49:28.922041+0000","last_peered":"2024-03-23T17:49:28.922041+0000","last_clean":"2024-03-23T17:49:28.922041+0000","last_became_active":"2024-03-23T17:48:56.235645+0000","last_became_peered":"2024-03-23T17:48:56.235645+0000","last_unstale":"2024-03-23T17:49:28.922041+0000","last_undegraded":"2024-03-23T17:49:28.922041+0000","last_fullsized":"2024-03-23T17:49:28.922041+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:47:01.076737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.782865+0000","last_change":"2024-03-23T17:48:56.239689+0000","last_active":"2024-03-23T17:49:28.782865+0000","last_peered":"2024-03-23T17:49:28.782865+0000","last_clean":"2024-03-23T17:49:28.782865+0000","last_became_active":"2024-03-23T17:48:56.239331+0000","last_became_peered":"2024-03-23T17:48:56.239331+0000","last_unstale":"2024-03-23T17:49:28.782865+0000","last_undegraded":"2024-03-23T17:49:28.782865+0000","last_fullsized":"2024-03-23T17:49:28.782865+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:21:15.287627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.777987+0000","last_change":"2024-03-23T17:48:56.229934+0000","last_active":"2024-03-23T17:49:28.777987+0000","last_peered":"2024-03-23T17:49:28.777987+0000","last_clean":"2024-03-23T17:49:28.777987+0000","last_became_active":"2024-03-23T17:48:56.229244+0000","last_became_peered":"2024-03-23T17:48:56.229244+0000","last_unstale":"2024-03-23T17:49:28.777987+0000","last_undegraded":"2024-03-23T17:49:28.777987+0000","last_fullsized":"2024-03-23T17:49:28.777987+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:02:40.254799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.784936+0000","last_change":"2024-03-23T17:48:56.233786+0000","last_active":"2024-03-23T17:49:28.784936+0000","last_peered":"2024-03-23T17:49:28.784936+0000","last_clean":"2024-03-23T17:49:28.784936+0000","last_became_active":"2024-03-23T17:48:56.233137+0000","last_became_peered":"2024-03-23T17:48:56.233137+0000","last_unstale":"2024-03-23T17:49:28.784936+0000","last_undegraded":"2024-03-23T17:49:28.784936+0000","last_fullsized":"2024-03-23T17:49:28.784936+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:33:35.202788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"82'14","reported_seq":108,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-23T17:50:20.814129+0000","last_change":"2024-03-23T17:48:55.712184+0000","last_active":"2024-03-23T17:50:20.814129+0000","last_peered":"2024-03-23T17:50:20.814129+0000","last_clean":"2024-03-23T17:50:20.814129+0000","last_became_active":"2024-03-23T17:48:55.711860+0000","last_became_peered":"2024-03-23T17:48:55.711860+0000","last_unstale":"2024-03-23T17:50:20.814129+0000","last_undegraded":"2024-03-23T17:50:20.814129+0000","last_fullsized":"2024-03-23T17:50:20.814129+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:59:01.133652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":80,"num_read_kb":80,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-23T17:49:29.789705+0000","last_change":"2024-03-23T17:49:29.789057+0000","last_active":"2024-03-23T17:49:29.789705+0000","last_peered":"2024-03-23T17:49:29.789705+0000","last_clean":"2024-03-23T17:49:29.789705+0000","last_became_active":"2024-03-23T17:49:29.788796+0000","last_became_peered":"2024-03-23T17:49:29.788796+0000","last_unstale":"2024-03-23T17:49:29.789705+0000","last_undegraded":"2024-03-23T17:49:29.789705+0000","last_fullsized":"2024-03-23T17:49:29.789705+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:30:53.062926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.780493+0000","last_change":"2024-03-23T17:48:55.725351+0000","last_active":"2024-03-23T17:49:28.780493+0000","last_peered":"2024-03-23T17:49:28.780493+0000","last_clean":"2024-03-23T17:49:28.780493+0000","last_became_active":"2024-03-23T17:48:55.724566+0000","last_became_peered":"2024-03-23T17:48:55.724566+0000","last_unstale":"2024-03-23T17:49:28.780493+0000","last_undegraded":"2024-03-23T17:49:28.780493+0000","last_fullsized":"2024-03-23T17:49:28.780493+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:58:21.088166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.784151+0000","last_change":"2024-03-23T17:48:56.243526+0000","last_active":"2024-03-23T17:49:28.784151+0000","last_peered":"2024-03-23T17:49:28.784151+0000","last_clean":"2024-03-23T17:49:28.784151+0000","last_became_active":"2024-03-23T17:48:56.242468+0000","last_became_peered":"2024-03-23T17:48:56.242468+0000","last_unstale":"2024-03-23T17:49:28.784151+0000","last_undegraded":"2024-03-23T17:49:28.784151+0000","last_fullsized":"2024-03-23T17:49:28.784151+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:16:50.392770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"82'13","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-23T17:49:29.781747+0000","last_change":"2024-03-23T17:48:55.720237+0000","last_active":"2024-03-23T17:49:29.781747+0000","last_peered":"2024-03-23T17:49:29.781747+0000","last_clean":"2024-03-23T17:49:29.781747+0000","last_became_active":"2024-03-23T17:48:55.719843+0000","last_became_peered":"2024-03-23T17:48:55.719843+0000","last_unstale":"2024-03-23T17:49:29.781747+0000","last_undegraded":"2024-03-23T17:49:29.781747+0000","last_fullsized":"2024-03-23T17:49:29.781747+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:07:26.973511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.922177+0000","last_change":"2024-03-23T17:48:56.235642+0000","last_active":"2024-03-23T17:49:28.922177+0000","last_peered":"2024-03-23T17:49:28.922177+0000","last_clean":"2024-03-23T17:49:28.922177+0000","last_became_active":"2024-03-23T17:48:56.235326+0000","last_became_peered":"2024-03-23T17:48:56.235326+0000","last_unstale":"2024-03-23T17:49:28.922177+0000","last_undegraded":"2024-03-23T17:49:28.922177+0000","last_fullsized":"2024-03-23T17:49:28.922177+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:21:59.934620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.781362+0000","last_change":"2024-03-23T17:48:55.718579+0000","last_active":"2024-03-23T17:49:28.781362+0000","last_peered":"2024-03-23T17:49:28.781362+0000","last_clean":"2024-03-23T17:49:28.781362+0000","last_became_active":"2024-03-23T17:48:55.718220+0000","last_became_peered":"2024-03-23T17:48:55.718220+0000","last_unstale":"2024-03-23T17:49:28.781362+0000","last_undegraded":"2024-03-23T17:49:28.781362+0000","last_fullsized":"2024-03-23T17:49:28.781362+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:37:56.417367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.785628+0000","last_change":"2024-03-23T17:48:56.224842+0000","last_active":"2024-03-23T17:49:28.785628+0000","last_peered":"2024-03-23T17:49:28.785628+0000","last_clean":"2024-03-23T17:49:28.785628+0000","last_became_active":"2024-03-23T17:48:56.224427+0000","last_became_peered":"2024-03-23T17:48:56.224427+0000","last_unstale":"2024-03-23T17:49:28.785628+0000","last_undegraded":"2024-03-23T17:49:28.785628+0000","last_fullsized":"2024-03-23T17:49:28.785628+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:16:22.684369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"80'5","reported_seq":103,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-23T17:50:20.781361+0000","last_change":"2024-03-23T17:48:55.713996+0000","last_active":"2024-03-23T17:50:20.781361+0000","last_peered":"2024-03-23T17:50:20.781361+0000","last_clean":"2024-03-23T17:50:20.781361+0000","last_became_active":"2024-03-23T17:48:55.713647+0000","last_became_peered":"2024-03-23T17:48:55.713647+0000","last_unstale":"2024-03-23T17:50:20.781361+0000","last_undegraded":"2024-03-23T17:50:20.781361+0000","last_fullsized":"2024-03-23T17:50:20.781361+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:12:22.251533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":84,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.781148+0000","last_change":"2024-03-23T17:48:56.219281+0000","last_active":"2024-03-23T17:49:28.781148+0000","last_peered":"2024-03-23T17:49:28.781148+0000","last_clean":"2024-03-23T17:49:28.781148+0000","last_became_active":"2024-03-23T17:48:56.218924+0000","last_became_peered":"2024-03-23T17:48:56.218924+0000","last_unstale":"2024-03-23T17:49:28.781148+0000","last_undegraded":"2024-03-23T17:49:28.781148+0000","last_fullsized":"2024-03-23T17:49:28.781148+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:49:04.793088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.781333+0000","last_change":"2024-03-23T17:48:55.718131+0000","last_active":"2024-03-23T17:49:28.781333+0000","last_peered":"2024-03-23T17:49:28.781333+0000","last_clean":"2024-03-23T17:49:28.781333+0000","last_became_active":"2024-03-23T17:48:55.717760+0000","last_became_peered":"2024-03-23T17:48:55.717760+0000","last_unstale":"2024-03-23T17:49:28.781333+0000","last_undegraded":"2024-03-23T17:49:28.781333+0000","last_fullsized":"2024-03-23T17:49:28.781333+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:16:54.355844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.784309+0000","last_change":"2024-03-23T17:48:56.225965+0000","last_active":"2024-03-23T17:49:28.784309+0000","last_peered":"2024-03-23T17:49:28.784309+0000","last_clean":"2024-03-23T17:49:28.784309+0000","last_became_active":"2024-03-23T17:48:56.225101+0000","last_became_peered":"2024-03-23T17:48:56.225101+0000","last_unstale":"2024-03-23T17:49:28.784309+0000","last_undegraded":"2024-03-23T17:49:28.784309+0000","last_fullsized":"2024-03-23T17:49:28.784309+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:52:14.302306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.776847+0000","last_change":"2024-03-23T17:48:55.722289+0000","last_active":"2024-03-23T17:49:28.776847+0000","last_peered":"2024-03-23T17:49:28.776847+0000","last_clean":"2024-03-23T17:49:28.776847+0000","last_became_active":"2024-03-23T17:48:55.721927+0000","last_became_peered":"2024-03-23T17:48:55.721927+0000","last_unstale":"2024-03-23T17:49:28.776847+0000","last_undegraded":"2024-03-23T17:49:28.776847+0000","last_fullsized":"2024-03-23T17:49:28.776847+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:20:08.354897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.779465+0000","last_change":"2024-03-23T17:48:56.232231+0000","last_active":"2024-03-23T17:49:28.779465+0000","last_peered":"2024-03-23T17:49:28.779465+0000","last_clean":"2024-03-23T17:49:28.779465+0000","last_became_active":"2024-03-23T17:48:56.227335+0000","last_became_peered":"2024-03-23T17:48:56.227335+0000","last_unstale":"2024-03-23T17:49:28.779465+0000","last_undegraded":"2024-03-23T17:49:28.779465+0000","last_fullsized":"2024-03-23T17:49:28.779465+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:49:01.460775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.785071+0000","last_change":"2024-03-23T17:48:55.717615+0000","last_active":"2024-03-23T17:49:28.785071+0000","last_peered":"2024-03-23T17:49:28.785071+0000","last_clean":"2024-03-23T17:49:28.785071+0000","last_became_active":"2024-03-23T17:48:55.717266+0000","last_became_peered":"2024-03-23T17:48:55.717266+0000","last_unstale":"2024-03-23T17:49:28.785071+0000","last_undegraded":"2024-03-23T17:49:28.785071+0000","last_fullsized":"2024-03-23T17:49:28.785071+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:40:07.988182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.776405+0000","last_change":"2024-03-23T17:48:56.229787+0000","last_active":"2024-03-23T17:49:28.776405+0000","last_peered":"2024-03-23T17:49:28.776405+0000","last_clean":"2024-03-23T17:49:28.776405+0000","last_became_active":"2024-03-23T17:48:56.229471+0000","last_became_peered":"2024-03-23T17:48:56.229471+0000","last_unstale":"2024-03-23T17:49:28.776405+0000","last_undegraded":"2024-03-23T17:49:28.776405+0000","last_fullsized":"2024-03-23T17:49:28.776405+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:44:09.358809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'4","reported_seq":173,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-23T17:50:18.777100+0000","last_change":"2024-03-23T17:48:55.717130+0000","last_active":"2024-03-23T17:50:18.777100+0000","last_peered":"2024-03-23T17:50:18.777100+0000","last_clean":"2024-03-23T17:50:18.777100+0000","last_became_active":"2024-03-23T17:48:55.716733+0000","last_became_peered":"2024-03-23T17:48:55.716733+0000","last_unstale":"2024-03-23T17:50:18.777100+0000","last_undegraded":"2024-03-23T17:50:18.777100+0000","last_fullsized":"2024-03-23T17:50:18.777100+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:22:43.229694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":155,"num_read_kb":155,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.922446+0000","last_change":"2024-03-23T17:48:56.235565+0000","last_active":"2024-03-23T17:49:28.922446+0000","last_peered":"2024-03-23T17:49:28.922446+0000","last_clean":"2024-03-23T17:49:28.922446+0000","last_became_active":"2024-03-23T17:48:56.235267+0000","last_became_peered":"2024-03-23T17:48:56.235267+0000","last_unstale":"2024-03-23T17:49:28.922446+0000","last_undegraded":"2024-03-23T17:49:28.922446+0000","last_fullsized":"2024-03-23T17:49:28.922446+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:38:45.906545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.780900+0000","last_change":"2024-03-23T17:48:55.719096+0000","last_active":"2024-03-23T17:49:28.780900+0000","last_peered":"2024-03-23T17:49:28.780900+0000","last_clean":"2024-03-23T17:49:28.780900+0000","last_became_active":"2024-03-23T17:48:55.718721+0000","last_became_peered":"2024-03-23T17:48:55.718721+0000","last_unstale":"2024-03-23T17:49:28.780900+0000","last_undegraded":"2024-03-23T17:49:28.780900+0000","last_fullsized":"2024-03-23T17:49:28.780900+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:43:05.532263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.784108+0000","last_change":"2024-03-23T17:48:56.235549+0000","last_active":"2024-03-23T17:49:28.784108+0000","last_peered":"2024-03-23T17:49:28.784108+0000","last_clean":"2024-03-23T17:49:28.784108+0000","last_became_active":"2024-03-23T17:48:56.234572+0000","last_became_peered":"2024-03-23T17:48:56.234572+0000","last_unstale":"2024-03-23T17:49:28.784108+0000","last_undegraded":"2024-03-23T17:49:28.784108+0000","last_fullsized":"2024-03-23T17:49:28.784108+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:19:58.639181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.785732+0000","last_change":"2024-03-23T17:48:55.724688+0000","last_active":"2024-03-23T17:49:28.785732+0000","last_peered":"2024-03-23T17:49:28.785732+0000","last_clean":"2024-03-23T17:49:28.785732+0000","last_became_active":"2024-03-23T17:48:55.724385+0000","last_became_peered":"2024-03-23T17:48:55.724385+0000","last_unstale":"2024-03-23T17:49:28.785732+0000","last_undegraded":"2024-03-23T17:49:28.785732+0000","last_fullsized":"2024-03-23T17:49:28.785732+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:17:54.338059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.922287+0000","last_change":"2024-03-23T17:48:56.242983+0000","last_active":"2024-03-23T17:49:28.922287+0000","last_peered":"2024-03-23T17:49:28.922287+0000","last_clean":"2024-03-23T17:49:28.922287+0000","last_became_active":"2024-03-23T17:48:56.242596+0000","last_became_peered":"2024-03-23T17:48:56.242596+0000","last_unstale":"2024-03-23T17:49:28.922287+0000","last_undegraded":"2024-03-23T17:49:28.922287+0000","last_fullsized":"2024-03-23T17:49:28.922287+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:55:51.971834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'32","reported_seq":211,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-23T17:50:19.225390+0000","last_change":"2024-03-23T17:48:55.717631+0000","last_active":"2024-03-23T17:50:19.225390+0000","last_peered":"2024-03-23T17:50:19.225390+0000","last_clean":"2024-03-23T17:50:19.225390+0000","last_became_active":"2024-03-23T17:48:55.717082+0000","last_became_peered":"2024-03-23T17:48:55.717082+0000","last_unstale":"2024-03-23T17:50:19.225390+0000","last_undegraded":"2024-03-23T17:50:19.225390+0000","last_fullsized":"2024-03-23T17:50:19.225390+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:36:25.323269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64038,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":165,"num_read_kb":140,"num_write":34,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.780792+0000","last_change":"2024-03-23T17:48:56.226174+0000","last_active":"2024-03-23T17:49:28.780792+0000","last_peered":"2024-03-23T17:49:28.780792+0000","last_clean":"2024-03-23T17:49:28.780792+0000","last_became_active":"2024-03-23T17:48:56.225834+0000","last_became_peered":"2024-03-23T17:48:56.225834+0000","last_unstale":"2024-03-23T17:49:28.780792+0000","last_undegraded":"2024-03-23T17:49:28.780792+0000","last_fullsized":"2024-03-23T17:49:28.780792+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:50:16.793071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-23T17:49:29.804494+0000","last_change":"2024-03-23T17:49:29.804099+0000","last_active":"2024-03-23T17:49:29.804494+0000","last_peered":"2024-03-23T17:49:29.804494+0000","last_clean":"2024-03-23T17:49:29.804494+0000","last_became_active":"2024-03-23T17:49:29.787483+0000","last_became_peered":"2024-03-23T17:49:29.787483+0000","last_unstale":"2024-03-23T17:49:29.804494+0000","last_undegraded":"2024-03-23T17:49:29.804494+0000","last_fullsized":"2024-03-23T17:49:29.804494+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:13:16.463738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.776667+0000","last_change":"2024-03-23T17:48:56.228459+0000","last_active":"2024-03-23T17:49:28.776667+0000","last_peered":"2024-03-23T17:49:28.776667+0000","last_clean":"2024-03-23T17:49:28.776667+0000","last_became_active":"2024-03-23T17:48:56.228179+0000","last_became_peered":"2024-03-23T17:48:56.228179+0000","last_unstale":"2024-03-23T17:49:28.776667+0000","last_undegraded":"2024-03-23T17:49:28.776667+0000","last_fullsized":"2024-03-23T17:49:28.776667+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:28:41.613432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.785318+0000","last_change":"2024-03-23T17:48:55.721533+0000","last_active":"2024-03-23T17:49:28.785318+0000","last_peered":"2024-03-23T17:49:28.785318+0000","last_clean":"2024-03-23T17:49:28.785318+0000","last_became_active":"2024-03-23T17:48:55.721100+0000","last_became_peered":"2024-03-23T17:48:55.721100+0000","last_unstale":"2024-03-23T17:49:28.785318+0000","last_undegraded":"2024-03-23T17:49:28.785318+0000","last_fullsized":"2024-03-23T17:49:28.785318+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:56:58.112649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-23T17:49:30.003965+0000","last_change":"2024-03-23T17:49:30.003965+0000","last_active":"2024-03-23T17:49:30.003965+0000","last_peered":"2024-03-23T17:49:30.003965+0000","last_clean":"2024-03-23T17:49:30.003965+0000","last_became_active":"2024-03-23T17:49:30.003643+0000","last_became_peered":"2024-03-23T17:49:30.003643+0000","last_unstale":"2024-03-23T17:49:30.003965+0000","last_undegraded":"2024-03-23T17:49:30.003965+0000","last_fullsized":"2024-03-23T17:49:30.003965+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:47:39.892782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'5","reported_seq":97,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-23T17:50:17.824512+0000","last_change":"2024-03-23T17:48:55.715205+0000","last_active":"2024-03-23T17:50:17.824512+0000","last_peered":"2024-03-23T17:50:17.824512+0000","last_clean":"2024-03-23T17:50:17.824512+0000","last_became_active":"2024-03-23T17:48:55.714810+0000","last_became_peered":"2024-03-23T17:48:55.714810+0000","last_unstale":"2024-03-23T17:50:17.824512+0000","last_undegraded":"2024-03-23T17:50:17.824512+0000","last_fullsized":"2024-03-23T17:50:17.824512+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:40:08.768998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":79,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-23T17:49:29.783025+0000","last_change":"2024-03-23T17:49:29.783025+0000","last_active":"2024-03-23T17:49:29.783025+0000","last_peered":"2024-03-23T17:49:29.783025+0000","last_clean":"2024-03-23T17:49:29.783025+0000","last_became_active":"2024-03-23T17:49:29.782541+0000","last_became_peered":"2024-03-23T17:49:29.782541+0000","last_unstale":"2024-03-23T17:49:29.783025+0000","last_undegraded":"2024-03-23T17:49:29.783025+0000","last_fullsized":"2024-03-23T17:49:29.783025+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:50:29.754074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":59,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-23T17:49:30.616960+0000","last_change":"2024-03-23T17:49:30.616960+0000","last_active":"2024-03-23T17:49:30.616960+0000","last_peered":"2024-03-23T17:49:30.616960+0000","last_clean":"2024-03-23T17:49:30.616960+0000","last_became_active":"2024-03-23T17:49:30.604464+0000","last_became_peered":"2024-03-23T17:49:30.604464+0000","last_unstale":"2024-03-23T17:49:30.616960+0000","last_undegraded":"2024-03-23T17:49:30.616960+0000","last_fullsized":"2024-03-23T17:49:30.616960+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:24:13.862841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.785079+0000","last_change":"2024-03-23T17:48:56.225624+0000","last_active":"2024-03-23T17:49:28.785079+0000","last_peered":"2024-03-23T17:49:28.785079+0000","last_clean":"2024-03-23T17:49:28.785079+0000","last_became_active":"2024-03-23T17:48:56.223798+0000","last_became_peered":"2024-03-23T17:48:56.223798+0000","last_unstale":"2024-03-23T17:49:28.785079+0000","last_undegraded":"2024-03-23T17:49:28.785079+0000","last_fullsized":"2024-03-23T17:49:28.785079+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:00:20.547553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.781323+0000","last_change":"2024-03-23T17:48:55.717792+0000","last_active":"2024-03-23T17:49:28.781323+0000","last_peered":"2024-03-23T17:49:28.781323+0000","last_clean":"2024-03-23T17:49:28.781323+0000","last_became_active":"2024-03-23T17:48:55.717479+0000","last_became_peered":"2024-03-23T17:48:55.717479+0000","last_unstale":"2024-03-23T17:49:28.781323+0000","last_undegraded":"2024-03-23T17:49:28.781323+0000","last_fullsized":"2024-03-23T17:49:28.781323+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:49:43.483785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.782572+0000","last_change":"2024-03-23T17:48:56.219135+0000","last_active":"2024-03-23T17:49:28.782572+0000","last_peered":"2024-03-23T17:49:28.782572+0000","last_clean":"2024-03-23T17:49:28.782572+0000","last_became_active":"2024-03-23T17:48:56.218824+0000","last_became_peered":"2024-03-23T17:48:56.218824+0000","last_unstale":"2024-03-23T17:49:28.782572+0000","last_undegraded":"2024-03-23T17:49:28.782572+0000","last_fullsized":"2024-03-23T17:49:28.782572+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:20:35.190781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.776774+0000","last_change":"2024-03-23T17:48:55.717530+0000","last_active":"2024-03-23T17:49:28.776774+0000","last_peered":"2024-03-23T17:49:28.776774+0000","last_clean":"2024-03-23T17:49:28.776774+0000","last_became_active":"2024-03-23T17:48:55.717213+0000","last_became_peered":"2024-03-23T17:48:55.717213+0000","last_unstale":"2024-03-23T17:49:28.776774+0000","last_undegraded":"2024-03-23T17:49:28.776774+0000","last_fullsized":"2024-03-23T17:49:28.776774+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:40:09.803021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'1","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.781617+0000","last_change":"2024-03-23T17:48:56.222237+0000","last_active":"2024-03-23T17:49:28.781617+0000","last_peered":"2024-03-23T17:49:28.781617+0000","last_clean":"2024-03-23T17:49:28.781617+0000","last_became_active":"2024-03-23T17:48:56.221916+0000","last_became_peered":"2024-03-23T17:48:56.221916+0000","last_unstale":"2024-03-23T17:49:28.781617+0000","last_undegraded":"2024-03-23T17:49:28.781617+0000","last_fullsized":"2024-03-23T17:49:28.781617+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:35:59.213482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"82'6","reported_seq":100,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-23T17:50:19.813475+0000","last_change":"2024-03-23T17:48:55.718143+0000","last_active":"2024-03-23T17:50:19.813475+0000","last_peered":"2024-03-23T17:50:19.813475+0000","last_clean":"2024-03-23T17:50:19.813475+0000","last_became_active":"2024-03-23T17:48:55.717834+0000","last_became_peered":"2024-03-23T17:48:55.717834+0000","last_unstale":"2024-03-23T17:50:19.813475+0000","last_undegraded":"2024-03-23T17:50:19.813475+0000","last_fullsized":"2024-03-23T17:50:19.813475+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:29:08.879636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":846,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":80,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.781385+0000","last_change":"2024-03-23T17:48:56.237136+0000","last_active":"2024-03-23T17:49:28.781385+0000","last_peered":"2024-03-23T17:49:28.781385+0000","last_clean":"2024-03-23T17:49:28.781385+0000","last_became_active":"2024-03-23T17:48:56.236795+0000","last_became_peered":"2024-03-23T17:48:56.236795+0000","last_unstale":"2024-03-23T17:49:28.781385+0000","last_undegraded":"2024-03-23T17:49:28.781385+0000","last_fullsized":"2024-03-23T17:49:28.781385+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:15:31.356382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-23T17:49:29.789433+0000","last_change":"2024-03-23T17:49:29.789433+0000","last_active":"2024-03-23T17:49:29.789433+0000","last_peered":"2024-03-23T17:49:29.789433+0000","last_clean":"2024-03-23T17:49:29.789433+0000","last_became_active":"2024-03-23T17:49:29.788956+0000","last_became_peered":"2024-03-23T17:49:29.788956+0000","last_unstale":"2024-03-23T17:49:29.789433+0000","last_undegraded":"2024-03-23T17:49:29.789433+0000","last_fullsized":"2024-03-23T17:49:29.789433+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:43:30.929796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"80'5","reported_seq":96,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-23T17:50:17.823614+0000","last_change":"2024-03-23T17:48:55.722968+0000","last_active":"2024-03-23T17:50:17.823614+0000","last_peered":"2024-03-23T17:50:17.823614+0000","last_clean":"2024-03-23T17:50:17.823614+0000","last_became_active":"2024-03-23T17:48:55.722578+0000","last_became_peered":"2024-03-23T17:48:55.722578+0000","last_unstale":"2024-03-23T17:50:17.823614+0000","last_undegraded":"2024-03-23T17:50:17.823614+0000","last_fullsized":"2024-03-23T17:50:17.823614+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:45:42.344283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":77,"num_read_kb":77,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.779906+0000","last_change":"2024-03-23T17:48:56.228000+0000","last_active":"2024-03-23T17:49:28.779906+0000","last_peered":"2024-03-23T17:49:28.779906+0000","last_clean":"2024-03-23T17:49:28.779906+0000","last_became_active":"2024-03-23T17:48:56.225645+0000","last_became_peered":"2024-03-23T17:48:56.225645+0000","last_unstale":"2024-03-23T17:49:28.779906+0000","last_undegraded":"2024-03-23T17:49:28.779906+0000","last_fullsized":"2024-03-23T17:49:28.779906+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:44:25.197761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.784417+0000","last_change":"2024-03-23T17:48:55.718307+0000","last_active":"2024-03-23T17:49:28.784417+0000","last_peered":"2024-03-23T17:49:28.784417+0000","last_clean":"2024-03-23T17:49:28.784417+0000","last_became_active":"2024-03-23T17:48:55.717839+0000","last_became_peered":"2024-03-23T17:48:55.717839+0000","last_unstale":"2024-03-23T17:49:28.784417+0000","last_undegraded":"2024-03-23T17:49:28.784417+0000","last_fullsized":"2024-03-23T17:49:28.784417+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:11:56.053940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":335,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.780763+0000","last_change":"2024-03-23T17:48:01.618019+0000","last_active":"2024-03-23T17:49:28.780763+0000","last_peered":"2024-03-23T17:49:28.780763+0000","last_clean":"2024-03-23T17:49:28.780763+0000","last_became_active":"2024-03-23T17:48:01.617553+0000","last_became_peered":"2024-03-23T17:48:01.617553+0000","last_unstale":"2024-03-23T17:49:28.780763+0000","last_undegraded":"2024-03-23T17:49:28.780763+0000","last_fullsized":"2024-03-23T17:49:28.780763+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:45:26.849029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:45:26.849029+0000","last_clean_scrub_stamp":"2024-03-23T17:45:26.849029+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:05:44.823364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.776693+0000","last_change":"2024-03-23T17:48:56.224285+0000","last_active":"2024-03-23T17:49:28.776693+0000","last_peered":"2024-03-23T17:49:28.776693+0000","last_clean":"2024-03-23T17:49:28.776693+0000","last_became_active":"2024-03-23T17:48:56.220943+0000","last_became_peered":"2024-03-23T17:48:56.220943+0000","last_unstale":"2024-03-23T17:49:28.776693+0000","last_undegraded":"2024-03-23T17:49:28.776693+0000","last_fullsized":"2024-03-23T17:49:28.776693+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:36:05.458243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.780994+0000","last_change":"2024-03-23T17:48:55.715877+0000","last_active":"2024-03-23T17:49:28.780994+0000","last_peered":"2024-03-23T17:49:28.780994+0000","last_clean":"2024-03-23T17:49:28.780994+0000","last_became_active":"2024-03-23T17:48:55.715550+0000","last_became_peered":"2024-03-23T17:48:55.715550+0000","last_unstale":"2024-03-23T17:49:28.780994+0000","last_undegraded":"2024-03-23T17:49:28.780994+0000","last_fullsized":"2024-03-23T17:49:28.780994+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:31:57.668495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.780911+0000","last_change":"2024-03-23T17:48:55.724618+0000","last_active":"2024-03-23T17:49:28.780911+0000","last_peered":"2024-03-23T17:49:28.780911+0000","last_clean":"2024-03-23T17:49:28.780911+0000","last_became_active":"2024-03-23T17:48:55.724278+0000","last_became_peered":"2024-03-23T17:48:55.724278+0000","last_unstale":"2024-03-23T17:49:28.780911+0000","last_undegraded":"2024-03-23T17:49:28.780911+0000","last_fullsized":"2024-03-23T17:49:28.780911+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:27:05.065115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.776532+0000","last_change":"2024-03-23T17:48:56.216141+0000","last_active":"2024-03-23T17:49:28.776532+0000","last_peered":"2024-03-23T17:49:28.776532+0000","last_clean":"2024-03-23T17:49:28.776532+0000","last_became_active":"2024-03-23T17:48:56.215653+0000","last_became_peered":"2024-03-23T17:48:56.215653+0000","last_unstale":"2024-03-23T17:49:28.776532+0000","last_undegraded":"2024-03-23T17:49:28.776532+0000","last_fullsized":"2024-03-23T17:49:28.776532+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:02:19.611056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'9","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-23T17:49:29.794454+0000","last_change":"2024-03-23T17:49:29.794454+0000","last_active":"2024-03-23T17:49:29.794454+0000","last_peered":"2024-03-23T17:49:29.794454+0000","last_clean":"2024-03-23T17:49:29.794454+0000","last_became_active":"2024-03-23T17:49:29.787238+0000","last_became_peered":"2024-03-23T17:49:29.787238+0000","last_unstale":"2024-03-23T17:49:29.794454+0000","last_undegraded":"2024-03-23T17:49:29.794454+0000","last_fullsized":"2024-03-23T17:49:29.794454+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:33:34.917283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.780480+0000","last_change":"2024-03-23T17:48:56.228439+0000","last_active":"2024-03-23T17:49:28.780480+0000","last_peered":"2024-03-23T17:49:28.780480+0000","last_clean":"2024-03-23T17:49:28.780480+0000","last_became_active":"2024-03-23T17:48:56.226677+0000","last_became_peered":"2024-03-23T17:48:56.226677+0000","last_unstale":"2024-03-23T17:49:28.780480+0000","last_undegraded":"2024-03-23T17:49:28.780480+0000","last_fullsized":"2024-03-23T17:49:28.780480+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:46:00.063266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.784329+0000","last_change":"2024-03-23T17:48:55.724872+0000","last_active":"2024-03-23T17:49:28.784329+0000","last_peered":"2024-03-23T17:49:28.784329+0000","last_clean":"2024-03-23T17:49:28.784329+0000","last_became_active":"2024-03-23T17:48:55.724468+0000","last_became_peered":"2024-03-23T17:48:55.724468+0000","last_unstale":"2024-03-23T17:49:28.784329+0000","last_undegraded":"2024-03-23T17:49:28.784329+0000","last_fullsized":"2024-03-23T17:49:28.784329+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:28:34.918056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.783623+0000","last_change":"2024-03-23T17:48:56.233730+0000","last_active":"2024-03-23T17:49:28.783623+0000","last_peered":"2024-03-23T17:49:28.783623+0000","last_clean":"2024-03-23T17:49:28.783623+0000","last_became_active":"2024-03-23T17:48:56.233189+0000","last_became_peered":"2024-03-23T17:48:56.233189+0000","last_unstale":"2024-03-23T17:49:28.783623+0000","last_undegraded":"2024-03-23T17:49:28.783623+0000","last_fullsized":"2024-03-23T17:49:28.783623+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:20:30.764344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.922070+0000","last_change":"2024-03-23T17:48:55.724633+0000","last_active":"2024-03-23T17:49:28.922070+0000","last_peered":"2024-03-23T17:49:28.922070+0000","last_clean":"2024-03-23T17:49:28.922070+0000","last_became_active":"2024-03-23T17:48:55.724301+0000","last_became_peered":"2024-03-23T17:48:55.724301+0000","last_unstale":"2024-03-23T17:49:28.922070+0000","last_undegraded":"2024-03-23T17:49:28.922070+0000","last_fullsized":"2024-03-23T17:49:28.922070+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:45:08.392668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.780692+0000","last_change":"2024-03-23T17:48:56.230981+0000","last_active":"2024-03-23T17:49:28.780692+0000","last_peered":"2024-03-23T17:49:28.780692+0000","last_clean":"2024-03-23T17:49:28.780692+0000","last_became_active":"2024-03-23T17:48:56.230644+0000","last_became_peered":"2024-03-23T17:48:56.230644+0000","last_unstale":"2024-03-23T17:49:28.780692+0000","last_undegraded":"2024-03-23T17:49:28.780692+0000","last_fullsized":"2024-03-23T17:49:28.780692+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:37:51.310938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.779873+0000","last_change":"2024-03-23T17:48:55.719112+0000","last_active":"2024-03-23T17:49:28.779873+0000","last_peered":"2024-03-23T17:49:28.779873+0000","last_clean":"2024-03-23T17:49:28.779873+0000","last_became_active":"2024-03-23T17:48:55.718756+0000","last_became_peered":"2024-03-23T17:48:55.718756+0000","last_unstale":"2024-03-23T17:49:28.779873+0000","last_undegraded":"2024-03-23T17:49:28.779873+0000","last_fullsized":"2024-03-23T17:49:28.779873+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:41:22.164706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.781982+0000","last_change":"2024-03-23T17:48:56.240289+0000","last_active":"2024-03-23T17:49:28.781982+0000","last_peered":"2024-03-23T17:49:28.781982+0000","last_clean":"2024-03-23T17:49:28.781982+0000","last_became_active":"2024-03-23T17:48:56.239896+0000","last_became_peered":"2024-03-23T17:48:56.239896+0000","last_unstale":"2024-03-23T17:49:28.781982+0000","last_undegraded":"2024-03-23T17:49:28.781982+0000","last_fullsized":"2024-03-23T17:49:28.781982+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:49:10.505576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'3","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.785515+0000","last_change":"2024-03-23T17:48:55.723330+0000","last_active":"2024-03-23T17:49:28.785515+0000","last_peered":"2024-03-23T17:49:28.785515+0000","last_clean":"2024-03-23T17:49:28.785515+0000","last_became_active":"2024-03-23T17:48:55.722879+0000","last_became_peered":"2024-03-23T17:48:55.722879+0000","last_unstale":"2024-03-23T17:49:28.785515+0000","last_undegraded":"2024-03-23T17:49:28.785515+0000","last_fullsized":"2024-03-23T17:49:28.785515+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:34:47.173411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.779841+0000","last_change":"2024-03-23T17:48:56.237042+0000","last_active":"2024-03-23T17:49:28.779841+0000","last_peered":"2024-03-23T17:49:28.779841+0000","last_clean":"2024-03-23T17:49:28.779841+0000","last_became_active":"2024-03-23T17:48:56.236722+0000","last_became_peered":"2024-03-23T17:48:56.236722+0000","last_unstale":"2024-03-23T17:49:28.779841+0000","last_undegraded":"2024-03-23T17:49:28.779841+0000","last_fullsized":"2024-03-23T17:49:28.779841+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:27:29.656120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.780900+0000","last_change":"2024-03-23T17:48:55.719223+0000","last_active":"2024-03-23T17:49:28.780900+0000","last_peered":"2024-03-23T17:49:28.780900+0000","last_clean":"2024-03-23T17:49:28.780900+0000","last_became_active":"2024-03-23T17:48:55.718864+0000","last_became_peered":"2024-03-23T17:48:55.718864+0000","last_unstale":"2024-03-23T17:49:28.780900+0000","last_undegraded":"2024-03-23T17:49:28.780900+0000","last_fullsized":"2024-03-23T17:49:28.780900+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:52:18.169557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.781086+0000","last_change":"2024-03-23T17:48:56.224885+0000","last_active":"2024-03-23T17:49:28.781086+0000","last_peered":"2024-03-23T17:49:28.781086+0000","last_clean":"2024-03-23T17:49:28.781086+0000","last_became_active":"2024-03-23T17:48:56.224160+0000","last_became_peered":"2024-03-23T17:48:56.224160+0000","last_unstale":"2024-03-23T17:49:28.781086+0000","last_undegraded":"2024-03-23T17:49:28.781086+0000","last_fullsized":"2024-03-23T17:49:28.781086+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:35:34.538313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'5","reported_seq":98,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-23T17:50:18.811876+0000","last_change":"2024-03-23T17:48:55.720518+0000","last_active":"2024-03-23T17:50:18.811876+0000","last_peered":"2024-03-23T17:50:18.811876+0000","last_clean":"2024-03-23T17:50:18.811876+0000","last_became_active":"2024-03-23T17:48:55.720045+0000","last_became_peered":"2024-03-23T17:48:55.720045+0000","last_unstale":"2024-03-23T17:50:18.811876+0000","last_undegraded":"2024-03-23T17:50:18.811876+0000","last_fullsized":"2024-03-23T17:50:18.811876+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:20:54.819406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":79,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.781093+0000","last_change":"2024-03-23T17:48:56.224171+0000","last_active":"2024-03-23T17:49:28.781093+0000","last_peered":"2024-03-23T17:49:28.781093+0000","last_clean":"2024-03-23T17:49:28.781093+0000","last_became_active":"2024-03-23T17:48:56.223737+0000","last_became_peered":"2024-03-23T17:48:56.223737+0000","last_unstale":"2024-03-23T17:49:28.781093+0000","last_undegraded":"2024-03-23T17:49:28.781093+0000","last_fullsized":"2024-03-23T17:49:28.781093+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:40:04.353945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'3","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.785531+0000","last_change":"2024-03-23T17:48:55.721734+0000","last_active":"2024-03-23T17:49:28.785531+0000","last_peered":"2024-03-23T17:49:28.785531+0000","last_clean":"2024-03-23T17:49:28.785531+0000","last_became_active":"2024-03-23T17:48:55.721417+0000","last_became_peered":"2024-03-23T17:48:55.721417+0000","last_unstale":"2024-03-23T17:49:28.785531+0000","last_undegraded":"2024-03-23T17:49:28.785531+0000","last_fullsized":"2024-03-23T17:49:28.785531+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:36:36.842740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'1","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.780817+0000","last_change":"2024-03-23T17:48:56.227211+0000","last_active":"2024-03-23T17:49:28.780817+0000","last_peered":"2024-03-23T17:49:28.780817+0000","last_clean":"2024-03-23T17:49:28.780817+0000","last_became_active":"2024-03-23T17:48:56.226881+0000","last_became_peered":"2024-03-23T17:48:56.226881+0000","last_unstale":"2024-03-23T17:49:28.780817+0000","last_undegraded":"2024-03-23T17:49:28.780817+0000","last_fullsized":"2024-03-23T17:49:28.780817+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:15:02.937666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.776701+0000","last_change":"2024-03-23T17:48:55.728679+0000","last_active":"2024-03-23T17:49:28.776701+0000","last_peered":"2024-03-23T17:49:28.776701+0000","last_clean":"2024-03-23T17:49:28.776701+0000","last_became_active":"2024-03-23T17:48:55.727754+0000","last_became_peered":"2024-03-23T17:48:55.727754+0000","last_unstale":"2024-03-23T17:49:28.776701+0000","last_undegraded":"2024-03-23T17:49:28.776701+0000","last_fullsized":"2024-03-23T17:49:28.776701+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:34:39.720488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.781328+0000","last_change":"2024-03-23T17:48:56.225214+0000","last_active":"2024-03-23T17:49:28.781328+0000","last_peered":"2024-03-23T17:49:28.781328+0000","last_clean":"2024-03-23T17:49:28.781328+0000","last_became_active":"2024-03-23T17:48:56.224635+0000","last_became_peered":"2024-03-23T17:48:56.224635+0000","last_unstale":"2024-03-23T17:49:28.781328+0000","last_undegraded":"2024-03-23T17:49:28.781328+0000","last_fullsized":"2024-03-23T17:49:28.781328+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:57:14.935434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.782109+0000","last_change":"2024-03-23T17:48:56.229835+0000","last_active":"2024-03-23T17:49:28.782109+0000","last_peered":"2024-03-23T17:49:28.782109+0000","last_clean":"2024-03-23T17:49:28.782109+0000","last_became_active":"2024-03-23T17:48:56.229437+0000","last_became_peered":"2024-03-23T17:48:56.229437+0000","last_unstale":"2024-03-23T17:49:28.782109+0000","last_undegraded":"2024-03-23T17:49:28.782109+0000","last_fullsized":"2024-03-23T17:49:28.782109+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:43:51.544867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.784779+0000","last_change":"2024-03-23T17:48:56.233552+0000","last_active":"2024-03-23T17:49:28.784779+0000","last_peered":"2024-03-23T17:49:28.784779+0000","last_clean":"2024-03-23T17:49:28.784779+0000","last_became_active":"2024-03-23T17:48:56.232941+0000","last_became_peered":"2024-03-23T17:48:56.232941+0000","last_unstale":"2024-03-23T17:49:28.784779+0000","last_undegraded":"2024-03-23T17:49:28.784779+0000","last_fullsized":"2024-03-23T17:49:28.784779+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:03:32.325496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.785402+0000","last_change":"2024-03-23T17:48:56.238238+0000","last_active":"2024-03-23T17:49:28.785402+0000","last_peered":"2024-03-23T17:49:28.785402+0000","last_clean":"2024-03-23T17:49:28.785402+0000","last_became_active":"2024-03-23T17:48:56.237882+0000","last_became_peered":"2024-03-23T17:48:56.237882+0000","last_unstale":"2024-03-23T17:49:28.785402+0000","last_undegraded":"2024-03-23T17:49:28.785402+0000","last_fullsized":"2024-03-23T17:49:28.785402+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:27:13.896205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.783881+0000","last_change":"2024-03-23T17:48:56.242895+0000","last_active":"2024-03-23T17:49:28.783881+0000","last_peered":"2024-03-23T17:49:28.783881+0000","last_clean":"2024-03-23T17:49:28.783881+0000","last_became_active":"2024-03-23T17:48:56.242515+0000","last_became_peered":"2024-03-23T17:48:56.242515+0000","last_unstale":"2024-03-23T17:49:28.783881+0000","last_undegraded":"2024-03-23T17:49:28.783881+0000","last_fullsized":"2024-03-23T17:49:28.783881+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:45:01.978843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.776603+0000","last_change":"2024-03-23T17:48:56.237029+0000","last_active":"2024-03-23T17:49:28.776603+0000","last_peered":"2024-03-23T17:49:28.776603+0000","last_clean":"2024-03-23T17:49:28.776603+0000","last_became_active":"2024-03-23T17:48:56.236692+0000","last_became_peered":"2024-03-23T17:48:56.236692+0000","last_unstale":"2024-03-23T17:49:28.776603+0000","last_undegraded":"2024-03-23T17:49:28.776603+0000","last_fullsized":"2024-03-23T17:49:28.776603+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:42:31.855469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.782393+0000","last_change":"2024-03-23T17:48:56.223236+0000","last_active":"2024-03-23T17:49:28.782393+0000","last_peered":"2024-03-23T17:49:28.782393+0000","last_clean":"2024-03-23T17:49:28.782393+0000","last_became_active":"2024-03-23T17:48:56.222824+0000","last_became_peered":"2024-03-23T17:48:56.222824+0000","last_unstale":"2024-03-23T17:49:28.782393+0000","last_undegraded":"2024-03-23T17:49:28.782393+0000","last_fullsized":"2024-03-23T17:49:28.782393+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:54:45.096969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.785731+0000","last_change":"2024-03-23T17:48:56.224471+0000","last_active":"2024-03-23T17:49:28.785731+0000","last_peered":"2024-03-23T17:49:28.785731+0000","last_clean":"2024-03-23T17:49:28.785731+0000","last_became_active":"2024-03-23T17:48:56.223140+0000","last_became_peered":"2024-03-23T17:48:56.223140+0000","last_unstale":"2024-03-23T17:49:28.785731+0000","last_undegraded":"2024-03-23T17:49:28.785731+0000","last_fullsized":"2024-03-23T17:49:28.785731+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:38:53.906820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.780461+0000","last_change":"2024-03-23T17:48:56.223564+0000","last_active":"2024-03-23T17:49:28.780461+0000","last_peered":"2024-03-23T17:49:28.780461+0000","last_clean":"2024-03-23T17:49:28.780461+0000","last_became_active":"2024-03-23T17:48:56.223217+0000","last_became_peered":"2024-03-23T17:48:56.223217+0000","last_unstale":"2024-03-23T17:49:28.780461+0000","last_undegraded":"2024-03-23T17:49:28.780461+0000","last_fullsized":"2024-03-23T17:49:28.780461+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:43:53.250082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.785081+0000","last_change":"2024-03-23T17:48:56.226283+0000","last_active":"2024-03-23T17:49:28.785081+0000","last_peered":"2024-03-23T17:49:28.785081+0000","last_clean":"2024-03-23T17:49:28.785081+0000","last_became_active":"2024-03-23T17:48:56.224211+0000","last_became_peered":"2024-03-23T17:48:56.224211+0000","last_unstale":"2024-03-23T17:49:28.785081+0000","last_undegraded":"2024-03-23T17:49:28.785081+0000","last_fullsized":"2024-03-23T17:49:28.785081+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:15:20.518950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.779571+0000","last_change":"2024-03-23T17:48:56.223557+0000","last_active":"2024-03-23T17:49:28.779571+0000","last_peered":"2024-03-23T17:49:28.779571+0000","last_clean":"2024-03-23T17:49:28.779571+0000","last_became_active":"2024-03-23T17:48:56.222792+0000","last_became_peered":"2024-03-23T17:48:56.222792+0000","last_unstale":"2024-03-23T17:49:28.779571+0000","last_undegraded":"2024-03-23T17:49:28.779571+0000","last_fullsized":"2024-03-23T17:49:28.779571+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:49:17.587817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.922230+0000","last_change":"2024-03-23T17:48:56.236041+0000","last_active":"2024-03-23T17:49:28.922230+0000","last_peered":"2024-03-23T17:49:28.922230+0000","last_clean":"2024-03-23T17:49:28.922230+0000","last_became_active":"2024-03-23T17:48:56.234259+0000","last_became_peered":"2024-03-23T17:48:56.234259+0000","last_unstale":"2024-03-23T17:49:28.922230+0000","last_undegraded":"2024-03-23T17:49:28.922230+0000","last_fullsized":"2024-03-23T17:49:28.922230+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:32:43.786783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.777528+0000","last_change":"2024-03-23T17:48:56.239283+0000","last_active":"2024-03-23T17:49:28.777528+0000","last_peered":"2024-03-23T17:49:28.777528+0000","last_clean":"2024-03-23T17:49:28.777528+0000","last_became_active":"2024-03-23T17:48:56.238952+0000","last_became_peered":"2024-03-23T17:48:56.238952+0000","last_unstale":"2024-03-23T17:49:28.777528+0000","last_undegraded":"2024-03-23T17:49:28.777528+0000","last_fullsized":"2024-03-23T17:49:28.777528+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:31:50.383284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.782143+0000","last_change":"2024-03-23T17:48:56.226684+0000","last_active":"2024-03-23T17:49:28.782143+0000","last_peered":"2024-03-23T17:49:28.782143+0000","last_clean":"2024-03-23T17:49:28.782143+0000","last_became_active":"2024-03-23T17:48:56.226367+0000","last_became_peered":"2024-03-23T17:48:56.226367+0000","last_unstale":"2024-03-23T17:49:28.782143+0000","last_undegraded":"2024-03-23T17:49:28.782143+0000","last_fullsized":"2024-03-23T17:49:28.782143+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:12:29.053330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70574,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":819,"num_read_kb":481,"num_write":235,"num_write_kb":138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":180,"num_keys_recovered":17,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":211722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":206,"ondisk_log_size":206,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547229,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31992,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713160,"statfs":{"total":95995035648,"available":95962275840,"internally_reserved":0,"allocated":897024,"data_stored":428738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31848901},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 17:50:19 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.53200000000000003,"15min":0.53900000000000003},"min":{"1min":0.38700000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.64300000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.60299999999999998,"15min":0.60899999999999999},"min":{"1min":0.435,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.73599999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.48399999999999999}]},{"osd":1,"last update":"Sat Mar 23 17:50:19 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.51600000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.749,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.63700000000000001,"15min":0.63600000000000001},"min":{"1min":0.503,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.85399999999999998,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.74099999999999999}]},{"osd":2,"last update":"Sat Mar 23 17:50:19 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.503,"15min":0.501},"min":{"1min":0.35399999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.47899999999999998,"15min":0.47299999999999998},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.51400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":4,"last update":"Sat Mar 23 17:50:19 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.64300000000000002,"15min":0.64500000000000002},"min":{"1min":0.44800000000000001,"5min":0.438,"15min":0.438},"max":{"1min":0.75700000000000001,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.66300000000000003,"15min":0.66100000000000003},"min":{"1min":0.505,"5min":0.441,"15min":0.441},"max":{"1min":0.83599999999999997,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.79000000000000004}]},{"osd":5,"last update":"Sat Mar 23 17:50:19 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.59599999999999997,"15min":0.58899999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.621,"5min":0.58899999999999997,"15min":0.58399999999999996},"min":{"1min":0.436,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.81299999999999994,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.61799999999999999}]},{"osd":6,"last update":"Sat Mar 23 17:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.84499999999999997}]},{"osd":7,"last update":"Sat Mar 23 17:50:19 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.71999999999999997,"15min":0.71899999999999997},"min":{"1min":0.54800000000000004,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.86399999999999999,"5min":1.002,"15min":1.002},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.753,"15min":0.75600000000000001},"min":{"1min":0.59299999999999997,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.89500000000000002,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.69299999999999995}]},{"osd":8,"last update":"Sat Mar 23 17:49:56 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.82999999999999996}]},{"osd":9,"last update":"Sat Mar 23 17:50:19 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.63200000000000001,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.93899999999999995,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.76300000000000001,"15min":0.76600000000000001},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.90200000000000002,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.80000000000000004}]},{"osd":10,"last update":"Sat Mar 23 17:50:19 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.78500000000000003,"15min":0.78600000000000003},"min":{"1min":0.66100000000000003,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.95299999999999996,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.79500000000000004,"15min":0.79400000000000004},"min":{"1min":0.69299999999999995,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.97299999999999998,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.86799999999999999}]}]},{"osd":10,"up_from":68,"seq":292057776159,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28568,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716584,"statfs":{"total":95995035648,"available":95965782016,"internally_reserved":0,"allocated":1585152,"data_stored":1078786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 17:49:56 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.438}]},{"osd":1,"last update":"Sat Mar 23 17:50:02 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.47099999999999997,"15min":0.47899999999999998},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.59599999999999997,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.44,"15min":0.44500000000000001},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.57199999999999995,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.41599999999999998}]},{"osd":2,"last update":"Sat Mar 23 17:50:02 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.58899999999999997,"15min":0.59399999999999997},"min":{"1min":0.41299999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.746,"5min":0.755,"15min":0.755},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.629,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.80700000000000005,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.623}]},{"osd":3,"last update":"Sat Mar 23 17:50:02 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.60899999999999999,"15min":0.61699999999999999},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.70999999999999996,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.626,"15min":0.63600000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.79000000000000004,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.58499999999999996}]},{"osd":4,"last update":"Sat Mar 23 17:50:02 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.56499999999999995,"15min":0.56899999999999995},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.72399999999999998,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.53900000000000003,"15min":0.55000000000000004},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.64900000000000002,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.55000000000000004}]},{"osd":5,"last update":"Sat Mar 23 17:50:02 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.64100000000000001,"15min":0.65100000000000002},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.77900000000000003,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.69899999999999995,"15min":0.70699999999999996},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.80500000000000005,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.72599999999999998}]},{"osd":6,"last update":"Sat Mar 23 17:50:02 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.68999999999999995,"15min":0.69599999999999995},"min":{"1min":0.47099999999999997,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.85099999999999998,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.71399999999999997,"15min":0.72599999999999998},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.82099999999999995,"5min":1.071,"15min":1.071},"last":0.70499999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":8,"last update":"Sat Mar 23 17:50:02 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.73899999999999999,"15min":0.748},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.86399999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.748,"15min":0.76000000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.82899999999999996,"5min":0.996,"15min":0.996},"last":0.67200000000000004}]},{"osd":9,"last update":"Sat Mar 23 17:50:02 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.76000000000000001,"15min":0.77000000000000002},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.92600000000000005,"5min":1.194,"15min":1.194},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.755,"15min":0.76900000000000002},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.879,"5min":1.042,"15min":1.042},"last":0.71499999999999997}]},{"osd":11,"last update":"Sat Mar 23 17:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.77500000000000002}]}]},{"osd":9,"up_from":63,"seq":270582939685,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36108,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709044,"statfs":{"total":95995035648,"available":95958061056,"internally_reserved":0,"allocated":913408,"data_stored":428648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":36043201},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 17:49:44 2024","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.54200000000000004,"15min":0.56200000000000006},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.64400000000000002,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.58499999999999996,"15min":0.60699999999999998},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.59999999999999998,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.51900000000000002}]},{"osd":1,"last update":"Sat Mar 23 17:49:44 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.78500000000000003,"15min":0.80500000000000005},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.70699999999999996,"15min":0.72499999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.78600000000000003,"5min":1.091,"15min":1.091},"last":0.45500000000000002}]},{"osd":2,"last update":"Sat Mar 23 17:49:44 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.78600000000000003,"15min":0.80100000000000005},"min":{"1min":0.47299999999999998,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.871,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.73599999999999999,"15min":0.752},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.97399999999999998,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.77700000000000002}]},{"osd":3,"last update":"Sat Mar 23 17:49:44 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.61499999999999999,"15min":0.63300000000000001},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.77900000000000003,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.71899999999999997,"15min":0.73699999999999999},"min":{"1min":0.44900000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.88900000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.79900000000000004}]},{"osd":4,"last update":"Sat Mar 23 17:48:42 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.61099999999999999}]},{"osd":5,"last update":"Sat Mar 23 17:49:44 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.73199999999999998,"15min":0.74199999999999999},"min":{"1min":0.54700000000000004,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.1180000000000001,"5min":1.167,"15min":1.167},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.85099999999999998,"15min":0.87},"min":{"1min":0.505,"5min":0.437,"15min":0.437},"max":{"1min":0.93100000000000005,"5min":1.48,"15min":1.48},"last":0.85799999999999998}]},{"osd":6,"last update":"Sat Mar 23 17:49:44 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.83099999999999996,"15min":0.85499999999999998},"min":{"1min":0.47099999999999997,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.877,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.78600000000000003,"15min":0.81200000000000006},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.85599999999999998,"5min":1.153,"15min":1.153},"last":0.69699999999999995}]},{"osd":7,"last update":"Sat Mar 23 17:49:44 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.81599999999999995,"15min":0.84399999999999997},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.78000000000000003,"5min":1.216,"15min":1.216},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.92200000000000004,"15min":0.94499999999999995},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.2969999999999999,"5min":1.373,"15min":1.373},"last":0.90400000000000003}]},{"osd":8,"last update":"Sat Mar 23 17:49:44 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.85999999999999999,"15min":0.878},"min":{"1min":0.57499999999999996,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.88200000000000001,"15min":0.90800000000000003},"min":{"1min":0.59099999999999997,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.95499999999999996,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.59699999999999998}]},{"osd":10,"last update":"Sat Mar 23 17:50:00 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.79500000000000004,"15min":0.81699999999999995},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.80300000000000005,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.89400000000000002,"15min":0.91300000000000003},"min":{"1min":0.67200000000000004,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":0.94299999999999995,"5min":1.339,"15min":1.339},"last":0.88200000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]}]},{"osd":8,"up_from":56,"seq":240518168615,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":429026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 17:50:23 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.57099999999999995,"15min":0.57899999999999996},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.79600000000000004,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55100000000000005,"15min":0.55200000000000005},"min":{"1min":0.38300000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.55600000000000005}]},{"osd":1,"last update":"Sat Mar 23 17:50:23 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.60799999999999998,"15min":0.59199999999999997},"min":{"1min":0.51800000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.60599999999999998,"15min":0.59899999999999998},"min":{"1min":0.44600000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.69399999999999995}]},{"osd":2,"last update":"Sat Mar 23 17:50:23 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.46899999999999997,"15min":0.45900000000000002},"min":{"1min":0.38300000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.71399999999999997,"5min":0.874,"15min":0.874},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.47499999999999998,"15min":0.46500000000000002},"min":{"1min":0.34399999999999997,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.747,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.65400000000000003}]},{"osd":3,"last update":"Sat Mar 23 17:50:23 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.58699999999999997,"15min":0.59099999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64800000000000002,"15min":0.66000000000000003},"min":{"1min":0.434,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.86299999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.60199999999999998}]},{"osd":4,"last update":"Sat Mar 23 17:50:23 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.61599999999999999,"15min":0.59499999999999997},"min":{"1min":0.55700000000000005,"5min":0.308,"15min":0.308},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.65400000000000003,"15min":0.64000000000000001},"min":{"1min":0.51300000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.82799999999999996}]},{"osd":5,"last update":"Sat Mar 23 17:50:23 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.55000000000000004,"15min":0.54900000000000004},"min":{"1min":0.45700000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.84599999999999997,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.56999999999999995,"15min":0.56000000000000005},"min":{"1min":0.39300000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.91800000000000004,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.61699999999999999}]},{"osd":6,"last update":"Sat Mar 23 17:50:23 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70299999999999996,"15min":0.70999999999999996},"min":{"1min":0.48199999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.94199999999999995,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.68700000000000006,"15min":0.69099999999999995},"min":{"1min":0.55300000000000005,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.89400000000000002,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.58699999999999997}]},{"osd":7,"last update":"Sat Mar 23 17:50:23 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.69299999999999995,"15min":0.68300000000000005},"min":{"1min":0.55300000000000005,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.71699999999999997,"15min":0.70399999999999996},"min":{"1min":0.60099999999999998,"5min":0.438,"15min":0.438},"max":{"1min":0.99199999999999999,"5min":1.04,"15min":1.04},"last":0.88800000000000001}]},{"osd":9,"last update":"Sat Mar 23 17:49:46 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.72899999999999998,"15min":0.73299999999999998},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.875,"5min":1.026,"15min":1.026},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.745,"5min":0.76100000000000001,"15min":0.76400000000000001},"min":{"1min":0.55500000000000005,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.94599999999999995,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.875}]},{"osd":10,"last update":"Sat Mar 23 17:50:02 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.72599999999999998,"15min":0.71599999999999997},"min":{"1min":0.61899999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.74299999999999999,"15min":0.73099999999999998},"min":{"1min":0.63600000000000001,"5min":0.496,"15min":0.496},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.93400000000000005}]},{"osd":11,"last update":"Sat Mar 23 17:49:57 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.91400000000000003}]}]},{"osd":7,"up_from":49,"seq":210453397547,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28488,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716664,"statfs":{"total":95995035648,"available":95965863936,"internally_reserved":0,"allocated":1507328,"data_stored":1021315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 17:48:02 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.54800000000000004}]},{"osd":1,"last update":"Sat Mar 23 17:50:08 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.39700000000000002},"min":{"1min":0.31,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.58499999999999996,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.434,"5min":0.40500000000000003,"15min":0.38200000000000001},"min":{"1min":0.314,"5min":0.248,"15min":0.248},"max":{"1min":0.64000000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.621}]},{"osd":2,"last update":"Sat Mar 23 17:50:08 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.57399999999999995,"15min":0.55800000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.57299999999999995,"15min":0.55900000000000005},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1,"5min":1,"15min":1},"last":0.64500000000000002}]},{"osd":3,"last update":"Sat Mar 23 17:50:08 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.55800000000000005,"15min":0.54700000000000004},"min":{"1min":0.41899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.59899999999999998,"15min":0.59099999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.77500000000000002,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.82899999999999996}]},{"osd":4,"last update":"Sat Mar 23 17:50:08 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.495,"15min":0.47899999999999998},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.72899999999999998,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.47699999999999998,"15min":0.47099999999999997},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.63700000000000001,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.69499999999999995}]},{"osd":5,"last update":"Sat Mar 23 17:50:08 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.60699999999999998,"15min":0.58999999999999997},"min":{"1min":0.46899999999999997,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.80400000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.58999999999999997,"15min":0.57799999999999996},"min":{"1min":0.41499999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.78100000000000003}]},{"osd":6,"last update":"Sat Mar 23 17:50:08 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.67300000000000004,"15min":0.66700000000000004},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.83899999999999997,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.879},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65200000000000002,"15min":0.65100000000000002},"min":{"1min":0.45000000000000001,"5min":0.433,"15min":0.433},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.80600000000000005}]},{"osd":8,"last update":"Sat Mar 23 17:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.64900000000000002,"15min":0.63300000000000001},"min":{"1min":0.47799999999999998,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.66600000000000004,"15min":0.64700000000000002},"min":{"1min":0.52700000000000002,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.92300000000000004}]},{"osd":9,"last update":"Sat Mar 23 17:49:43 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.71299999999999997,"15min":0.71699999999999997},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.88200000000000001,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.69299999999999995,"15min":0.69599999999999995},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.84499999999999997,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.84799999999999998}]},{"osd":10,"last update":"Sat Mar 23 17:50:02 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.73199999999999998,"15min":0.73999999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.92000000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.70499999999999996,"15min":0.70999999999999996},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.85999999999999999}]},{"osd":11,"last update":"Sat Mar 23 17:49:58 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.93500000000000005}]}]},{"osd":6,"up_from":45,"seq":193273528367,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36088,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93709064,"statfs":{"total":95995035648,"available":95958081536,"internally_reserved":0,"allocated":892928,"data_stored":428044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 17:49:47 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47499999999999998,"15min":0.47599999999999998},"min":{"1min":0.32200000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.51700000000000002,"15min":0.52100000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.48899999999999999}]},{"osd":1,"last update":"Sat Mar 23 17:49:47 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.59799999999999998,"15min":0.59399999999999997},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.83899999999999997,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.46999999999999997,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.95699999999999996,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.71799999999999997}]},{"osd":2,"last update":"Sat Mar 23 17:49:47 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.44900000000000001,"5min":0.374,"15min":0.374},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67200000000000004,"15min":0.67700000000000005},"min":{"1min":0.53000000000000003,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.88200000000000001,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.66400000000000003}]},{"osd":3,"last update":"Sat Mar 23 17:49:47 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.59799999999999998,"15min":0.61399999999999999},"min":{"1min":0.35399999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.878,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.65700000000000003,"15min":0.66300000000000003},"min":{"1min":0.376,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.1140000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.64600000000000002}]},{"osd":4,"last update":"Sat Mar 23 17:49:47 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.65200000000000002,"15min":0.65600000000000003},"min":{"1min":0.42999999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.71799999999999997,"15min":0.73299999999999998},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.90500000000000003,"5min":0.998,"15min":0.998},"last":0.69999999999999996}]},{"osd":5,"last update":"Sat Mar 23 17:49:47 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.77400000000000002,"15min":0.78100000000000003},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.77000000000000002,"15min":0.78200000000000003},"min":{"1min":0.49299999999999999,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.504,"5min":1.504,"15min":1.504},"last":0.69199999999999995}]},{"osd":7,"last update":"Sat Mar 23 17:50:08 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.745,"15min":0.74199999999999999},"min":{"1min":0.54900000000000004,"5min":0.5,"15min":0.5},"max":{"1min":1.448,"5min":1.448,"15min":1.448},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.749,"15min":0.77900000000000003},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.4339999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.621}]},{"osd":8,"last update":"Sat Mar 23 17:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.85599999999999998,"15min":0.876},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.92400000000000004,"5min":1.085,"15min":1.085},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.81499999999999995,"15min":0.82399999999999995},"min":{"1min":0.63200000000000001,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":0.91200000000000003,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.77600000000000002}]},{"osd":9,"last update":"Sat Mar 23 17:49:47 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.80500000000000005,"15min":0.80800000000000005},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.78400000000000003,"15min":0.79400000000000004},"min":{"1min":0.48399999999999999,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.93899999999999995,"5min":1.099,"15min":1.099},"last":0.81399999999999995}]},{"osd":10,"last update":"Sat Mar 23 17:50:02 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.80800000000000005,"15min":0.80200000000000005},"min":{"1min":0.67900000000000005,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.80300000000000005,"15min":0.80700000000000005},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.573,"5min":1.573,"15min":1.573},"last":0.83899999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":493574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 17:50:21 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53100000000000003,"15min":0.51500000000000001},"min":{"1min":0.41499999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.82299999999999995,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.63700000000000001,"15min":0.53500000000000003},"min":{"1min":0.39800000000000002,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.68200000000000005,"5min":5.8399999999999999,"15min":5.8399999999999999},"last":0.47699999999999998}]},{"osd":2,"last update":"Sat Mar 23 17:49:29 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.52700000000000002,"15min":0.48099999999999998},"min":{"1min":0.46600000000000003,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.81399999999999995,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.52200000000000002,"15min":0.49299999999999999},"min":{"1min":0.45900000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.77400000000000002,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.56899999999999995}]},{"osd":3,"last update":"Sat Mar 23 17:49:52 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55600000000000005,"15min":0.53400000000000003},"min":{"1min":0.38100000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.73799999999999999,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.55300000000000005,"15min":0.53900000000000003},"min":{"1min":0.38400000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.69199999999999995,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.503}]},{"osd":4,"last update":"Sat Mar 23 17:50:10 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.44400000000000001,"15min":0.40600000000000003},"min":{"1min":0.34300000000000003,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.59699999999999998,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.442,"15min":0.40300000000000002},"min":{"1min":0.36399999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.69799999999999995,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.39000000000000001}]},{"osd":5,"last update":"Sat Mar 23 17:49:29 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.59499999999999997,"15min":0.56299999999999994},"min":{"1min":0.55500000000000005,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.86599999999999999,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.61199999999999999,"15min":0.57799999999999996},"min":{"1min":0.55600000000000005,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.85399999999999998,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.71199999999999997}]},{"osd":6,"last update":"Sat Mar 23 17:49:45 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.59999999999999998,"15min":0.57299999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.80400000000000005,"5min":0.998,"15min":0.998},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.63600000000000001,"15min":0.621},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.78800000000000003,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.64100000000000001}]},{"osd":7,"last update":"Sat Mar 23 17:50:06 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53200000000000003,"15min":0.51300000000000001},"min":{"1min":0.38200000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.66000000000000003,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.372},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.57299999999999995,"15min":0.54100000000000004},"min":{"1min":0.39400000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.73399999999999999,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.65900000000000003}]},{"osd":8,"last update":"Sat Mar 23 17:50:21 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.69499999999999995,"15min":0.67300000000000004},"min":{"1min":0.57499999999999996,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.88,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.64800000000000002,"15min":0.63400000000000001},"min":{"1min":0.505,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.80600000000000005,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.59799999999999998}]},{"osd":9,"last update":"Sat Mar 23 17:49:45 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.75600000000000001,"15min":0.76200000000000001},"min":{"1min":0.48699999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.85399999999999998,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.63},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.78900000000000003,"15min":0.79900000000000004},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.90300000000000002,"5min":1.103,"15min":1.103},"last":0.76200000000000001}]},{"osd":10,"last update":"Sat Mar 23 17:50:00 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.70699999999999996,"15min":0.71399999999999997},"min":{"1min":0.56599999999999995,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.84199999999999997,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.76300000000000001,"15min":0.76800000000000002},"min":{"1min":0.46100000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.89100000000000001,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.747}]},{"osd":11,"last update":"Sat Mar 23 17:50:00 2024","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.78600000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738438,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":913408,"data_stored":429383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 23 17:50:17 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.61399999999999999,"15min":0.71399999999999997},"min":{"1min":0.38400000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.89800000000000002,"5min":6.2110000000000003,"15min":6.2110000000000003},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.628,"5min":0.56999999999999995,"15min":0.51600000000000001},"min":{"1min":0.41399999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.58099999999999996}]},{"osd":2,"last update":"Sat Mar 23 17:49:31 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.57799999999999996,"15min":0.55100000000000005},"min":{"1min":0.38100000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.82999999999999996,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.59999999999999998,"15min":0.56000000000000005},"min":{"1min":0.41299999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.751}]},{"osd":3,"last update":"Sat Mar 23 17:49:58 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.45800000000000002}]},{"osd":4,"last update":"Sat Mar 23 17:50:17 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.64700000000000002,"15min":0.625},"min":{"1min":0.45500000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.621},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.61099999999999999,"15min":0.59199999999999997},"min":{"1min":0.46100000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.873,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.79800000000000004}]},{"osd":5,"last update":"Sat Mar 23 17:49:28 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65700000000000003,"15min":0.63700000000000001},"min":{"1min":0.44700000000000001,"5min":0.44,"15min":0.44},"max":{"1min":1.018,"5min":1.167,"15min":1.167},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.64500000000000002,"15min":0.61599999999999999},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.65500000000000003}]},{"osd":6,"last update":"Sat Mar 23 17:49:48 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60099999999999998},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.68000000000000005,"15min":0.66800000000000004},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.0509999999999999,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.63800000000000001}]},{"osd":7,"last update":"Sat Mar 23 17:48:02 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.495},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.58599999999999997}]},{"osd":8,"last update":"Sat Mar 23 17:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.83299999999999996,"15min":0.84899999999999998},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.091,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.81299999999999994,"15min":0.82599999999999996},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.68100000000000005}]},{"osd":9,"last update":"Sat Mar 23 17:49:43 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.76900000000000002,"15min":0.77300000000000002},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.82899999999999996,"15min":0.83699999999999997},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.119,"5min":1.286,"15min":1.286},"last":0.78700000000000003}]},{"osd":10,"last update":"Sat Mar 23 17:50:04 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.81499999999999995,"15min":0.81100000000000005},"min":{"1min":0.59599999999999997,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.421,"5min":1.421,"15min":1.421},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.90000000000000002}]},{"osd":11,"last update":"Sat Mar 23 17:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.93400000000000005}]}]},{"osd":2,"up_from":18,"seq":77309411390,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36148,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709004,"statfs":{"total":95995035648,"available":95958020096,"internally_reserved":0,"allocated":962560,"data_stored":492987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 17:49:30 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.54000000000000004,"15min":0.51400000000000001},"min":{"1min":0.42199999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.5},{"interface":"front","average":{"1min":0.63,"5min":0.57899999999999996,"15min":0.55300000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.88700000000000001,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.35699999999999998}]},{"osd":1,"last update":"Sat Mar 23 17:49:30 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.54500000000000004,"15min":0.5},"min":{"1min":0.42199999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.80900000000000005,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.55800000000000005,"15min":0.51800000000000002},"min":{"1min":0.41999999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.79300000000000004,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.68600000000000005}]},{"osd":3,"last update":"Sat Mar 23 17:49:56 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.55900000000000005,"15min":0.53300000000000003},"min":{"1min":0.38200000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.438},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58899999999999997,"15min":0.56699999999999995},"min":{"1min":0.40300000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.82199999999999995,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.57299999999999995}]},{"osd":4,"last update":"Sat Mar 23 17:49:56 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.63300000000000001}]},{"osd":5,"last update":"Sat Mar 23 17:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.48299999999999998,"15min":0.45900000000000002},"min":{"1min":0.32600000000000001,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.495,"15min":0.46999999999999997},"min":{"1min":0.34100000000000003,"5min":0.308,"15min":0.308},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.61199999999999999}]},{"osd":6,"last update":"Sat Mar 23 17:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.63700000000000001,"15min":0.622},"min":{"1min":0.505,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.80800000000000005,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.65700000000000003,"15min":0.64200000000000002},"min":{"1min":0.51100000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.85099999999999998,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.52100000000000002}]},{"osd":7,"last update":"Sat Mar 23 17:48:01 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.745}]},{"osd":8,"last update":"Sat Mar 23 17:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.51000000000000001,"15min":0.50600000000000001},"min":{"1min":0.35599999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.59399999999999997,"15min":0.58599999999999997},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.78100000000000003,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.56000000000000005}]},{"osd":9,"last update":"Sat Mar 23 17:49:45 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.70399999999999996,"15min":0.69999999999999996},"min":{"1min":0.49399999999999999,"5min":0.433,"15min":0.433},"max":{"1min":0.84099999999999997,"5min":0.879,"15min":0.879},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.77300000000000002,"15min":0.77400000000000002},"min":{"1min":0.68400000000000005,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.94899999999999995,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.73599999999999999}]},{"osd":10,"last update":"Sat Mar 23 17:50:06 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68200000000000005,"15min":0.68000000000000005},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.79300000000000004,"15min":0.79100000000000004},"min":{"1min":0.68000000000000005,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.80800000000000005}]},{"osd":11,"last update":"Sat Mar 23 17:49:56 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.65900000000000003}]}]},{"osd":3,"up_from":26,"seq":111669149754,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32008,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713144,"statfs":{"total":95995035648,"available":95962259456,"internally_reserved":0,"allocated":909312,"data_stored":427917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 17:50:02 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.44,"15min":0.40200000000000002},"min":{"1min":0.32000000000000001,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.44500000000000001,"15min":0.40699999999999997},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.85899999999999999,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.39600000000000002}]},{"osd":1,"last update":"Sat Mar 23 17:50:02 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.58999999999999997,"15min":0.53100000000000003},"min":{"1min":0.38700000000000001,"5min":0.371,"15min":0.371},"max":{"1min":2.0649999999999999,"5min":2.0649999999999999,"15min":2.0649999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.57599999999999996,"15min":0.51900000000000002},"min":{"1min":0.48299999999999998,"5min":0.371,"15min":0.371},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.751}]},{"osd":2,"last update":"Sat Mar 23 17:50:02 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.629,"15min":0.58899999999999997},"min":{"1min":0.434,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.96699999999999997,"5min":1.02,"15min":1.02},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.64900000000000002,"15min":0.61899999999999999},"min":{"1min":0.47299999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":2.141,"5min":2.141,"15min":2.141},"last":0.61299999999999999}]},{"osd":4,"last update":"Sat Mar 23 17:50:12 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.67700000000000005,"15min":0.624},"min":{"1min":0.55300000000000005,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":2.2080000000000002,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.65900000000000003,"15min":0.621},"min":{"1min":0.45200000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.73099999999999998}]},{"osd":5,"last update":"Sat Mar 23 17:49:31 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.67800000000000005,"15min":0.65000000000000002},"min":{"1min":0.53500000000000003,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.01,"5min":1.036,"15min":1.036},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.70099999999999996,"15min":0.67300000000000004},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.56599999999999995}]},{"osd":6,"last update":"Sat Mar 23 17:49:45 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.66700000000000004,"15min":0.65100000000000002},"min":{"1min":0.41099999999999998,"5min":0.38,"15min":0.38},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.68000000000000005,"15min":0.66800000000000004},"min":{"1min":0.46300000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.64500000000000002}]},{"osd":7,"last update":"Sat Mar 23 17:50:08 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.75600000000000001,"15min":0.76500000000000001},"min":{"1min":0.54500000000000004,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.747,"15min":0.71999999999999997},"min":{"1min":0.55200000000000005,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":2.1720000000000002,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":0.89900000000000002}]},{"osd":8,"last update":"Sat Mar 23 17:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.88},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.85099999999999998}]},{"osd":9,"last update":"Sat Mar 23 17:49:45 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.78500000000000003,"15min":0.77300000000000002},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":2.2370000000000001,"5min":2.2370000000000001,"15min":2.2370000000000001},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.78400000000000003,"15min":0.78000000000000003},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.625}]},{"osd":10,"last update":"Sat Mar 23 17:50:03 2024","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.79600000000000004,"15min":0.78700000000000003},"min":{"1min":0.65400000000000003,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.46,"5min":1.46,"15min":1.46},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.85499999999999998,"15min":0.84099999999999997},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":2.6349999999999998,"5min":2.6349999999999998,"15min":2.6349999999999998},"last":0.82999999999999996}]},{"osd":11,"last update":"Sat Mar 23 17:49:57 2024","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":2.6819999999999999,"5min":2.6819999999999999,"15min":2.6819999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.51,"5min":1.51,"15min":1.51},"last":0.68100000000000005}]}]},{"osd":4,"up_from":33,"seq":141733920823,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":901120,"data_stored":428017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 17:48:06 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.53700000000000003,"15min":0.53400000000000003},"min":{"1min":0.41799999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.77600000000000002,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.53700000000000003,"15min":0.52500000000000002},"min":{"1min":0.45300000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.52000000000000002}]},{"osd":1,"last update":"Sat Mar 23 17:50:07 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.36799999999999999,"15min":0.33100000000000002},"min":{"1min":0.27500000000000002,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.62,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.38400000000000001,"15min":0.34899999999999998},"min":{"1min":0.28999999999999998,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.57399999999999995,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.52000000000000002}]},{"osd":2,"last update":"Sat Mar 23 17:50:07 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.57999999999999996,"15min":0.55900000000000005},"min":{"1min":0.379,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.83499999999999996,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.58599999999999997,"15min":0.55200000000000005},"min":{"1min":0.441,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.81100000000000005,"5min":0.88,"15min":0.88},"last":0.78200000000000003}]},{"osd":3,"last update":"Sat Mar 23 17:50:07 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57599999999999996,"15min":0.56100000000000005},"min":{"1min":0.46600000000000003,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.73999999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.56499999999999995,"15min":0.56200000000000006},"min":{"1min":0.39400000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.754,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.504}]},{"osd":5,"last update":"Sat Mar 23 17:49:29 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.57999999999999996,"15min":0.56599999999999995},"min":{"1min":0.41299999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.80600000000000005,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.629,"5min":0.59499999999999997,"15min":0.57799999999999996},"min":{"1min":0.44600000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.77700000000000002,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.82799999999999996}]},{"osd":6,"last update":"Sat Mar 23 17:49:53 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.73699999999999999,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59199999999999997,"15min":0.58999999999999997},"min":{"1min":0.41799999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.79700000000000004,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.54900000000000004}]},{"osd":7,"last update":"Sat Mar 23 17:50:03 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.55300000000000005,"15min":0.54300000000000004},"min":{"1min":0.39300000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.55300000000000005,"15min":0.53900000000000003},"min":{"1min":0.36599999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.80900000000000005}]},{"osd":8,"last update":"Sat Mar 23 17:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.65800000000000003,"15min":0.65400000000000003},"min":{"1min":0.501,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.75},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.71699999999999997,"15min":0.71799999999999997},"min":{"1min":0.48599999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.86599999999999999,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.90700000000000003}]},{"osd":9,"last update":"Sat Mar 23 17:49:43 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.71799999999999997,"15min":0.71599999999999997},"min":{"1min":0.60599999999999998,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.54500000000000004,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.65200000000000002}]},{"osd":10,"last update":"Sat Mar 23 17:50:03 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.69899999999999995,"15min":0.69499999999999995},"min":{"1min":0.52600000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.86599999999999999,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.87},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.68700000000000006,"15min":0.67900000000000005},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.89300000000000002}]},{"osd":11,"last update":"Sat Mar 23 17:49:59 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.92900000000000005}]}]},{"osd":5,"up_from":38,"seq":163208757299,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32600,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712552,"statfs":{"total":95995035648,"available":95961653248,"internally_reserved":0,"allocated":1515520,"data_stored":1020452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 17:49:28 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.54900000000000004,"15min":0.54100000000000004},"min":{"1min":0.435,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.54400000000000004,"15min":0.54800000000000004},"min":{"1min":0.36299999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.83499999999999996,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.435}]},{"osd":1,"last update":"Sat Mar 23 17:49:28 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.55100000000000005,"15min":0.52700000000000002},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.58799999999999997,"15min":0.56499999999999995},"min":{"1min":0.495,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.67200000000000004}]},{"osd":2,"last update":"Sat Mar 23 17:49:28 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.40400000000000003,"15min":0.39400000000000002},"min":{"1min":0.311,"5min":0.251,"15min":0.251},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.503,"5min":0.41799999999999998,"15min":0.40100000000000002},"min":{"1min":0.30399999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.374}]},{"osd":3,"last update":"Sat Mar 23 17:49:28 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58299999999999996,"15min":0.58499999999999996},"min":{"1min":0.38100000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.80100000000000005,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.57699999999999996,"15min":0.58499999999999996},"min":{"1min":0.40899999999999997,"5min":0.373,"15min":0.373},"max":{"1min":0.71899999999999997,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.41999999999999998}]},{"osd":4,"last update":"Sat Mar 23 17:49:28 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.60599999999999998,"15min":0.58199999999999996},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.89500000000000002,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.58199999999999996,"15min":0.56000000000000005},"min":{"1min":0.44400000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.877,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.64700000000000002}]},{"osd":6,"last update":"Sat Mar 23 17:49:50 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.64000000000000001,"15min":0.629},"min":{"1min":0.55100000000000005,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.61499999999999999,"15min":0.61199999999999999},"min":{"1min":0.436,"5min":0.434,"15min":0.434},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.47399999999999998}]},{"osd":7,"last update":"Sat Mar 23 17:50:06 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.67800000000000005,"15min":0.65200000000000002},"min":{"1min":0.623,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.66100000000000003,"15min":0.63900000000000001},"min":{"1min":0.58399999999999996,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.66100000000000003}]},{"osd":8,"last update":"Sat Mar 23 17:50:21 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.624,"15min":0.59799999999999998},"min":{"1min":0.45000000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.57699999999999996,"15min":0.56000000000000005},"min":{"1min":0.38800000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.59999999999999998}]},{"osd":9,"last update":"Sat Mar 23 17:49:44 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.91600000000000004,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.70599999999999996,"15min":0.70099999999999996},"min":{"1min":0.53700000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.61499999999999999}]},{"osd":10,"last update":"Sat Mar 23 17:50:01 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.748,"15min":0.73399999999999999},"min":{"1min":0.71399999999999997,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.71799999999999997,"15min":0.70699999999999996},"min":{"1min":0.65000000000000002,"5min":0.503,"15min":0.503},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.76900000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":66048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":65734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":65726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T17:50:25.281 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T17:50:25.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.507+0000 7f6b40e00700 1 -- 172.21.15.86:0/1925443180 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b3c073810 msgr2=0x7f6b3c073c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:25.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.507+0000 7f6b40e00700 1 --2- 172.21.15.86:0/1925443180 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b3c073810 0x7f6b3c073c90 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6b280098f0 tx=0x7f6b28009c00 comp rx=0 tx=0).stop 2024-03-23T17:50:25.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.507+0000 7f6b40e00700 1 -- 172.21.15.86:0/1925443180 shutdown_connections 2024-03-23T17:50:25.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.507+0000 7f6b40e00700 1 --2- 172.21.15.86:0/1925443180 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b3c108b40 0x7f6b3c10afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:25.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.507+0000 7f6b40e00700 1 --2- 172.21.15.86:0/1925443180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b3c074df0 0x7f6b3c073240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:25.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.507+0000 7f6b40e00700 1 --2- 172.21.15.86:0/1925443180 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b3c073810 0x7f6b3c073c90 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:25.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.507+0000 7f6b40e00700 1 -- 172.21.15.86:0/1925443180 >> 172.21.15.86:0/1925443180 conn(0x7f6b3c0fc1f0 msgr2=0x7f6b3c0fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:25.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.507+0000 7f6b40e00700 1 -- 172.21.15.86:0/1925443180 shutdown_connections 2024-03-23T17:50:25.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.507+0000 7f6b40e00700 1 -- 172.21.15.86:0/1925443180 wait complete. 2024-03-23T17:50:25.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.508+0000 7f6b40e00700 1 Processor -- start 2024-03-23T17:50:25.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.508+0000 7f6b40e00700 1 -- start start 2024-03-23T17:50:25.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.508+0000 7f6b40e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b3c073810 0x7f6b3c198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:25.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.508+0000 7f6b40e00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b3c074df0 0x7f6b3c1993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:25.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.508+0000 7f6b40e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b3c108b40 0x7f6b3c1a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:25.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.508+0000 7f6b40e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3c1a09e0 con 0x7f6b3c108b40 2024-03-23T17:50:25.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.508+0000 7f6b40e00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3c1a0b20 con 0x7f6b3c073810 2024-03-23T17:50:25.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.508+0000 7f6b40e00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3c1a0cb0 con 0x7f6b3c074df0 2024-03-23T17:50:25.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.509+0000 7f6b39400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b3c074df0 0x7f6b3c1993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:25.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.509+0000 7f6b39400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b3c074df0 0x7f6b3c1993f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:33266/0 (socket says 172.21.15.86:33266) 2024-03-23T17:50:25.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.509+0000 7f6b39400700 1 -- 172.21.15.86:0/2368442760 learned_addr learned my addr 172.21.15.86:0/2368442760 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:25.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.509+0000 7f6b39e00700 1 --2- 172.21.15.86:0/2368442760 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b3c073810 0x7f6b3c198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:25.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.509+0000 7f6b3a800700 1 --2- 172.21.15.86:0/2368442760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b3c108b40 0x7f6b3c1a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:25.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.509+0000 7f6b39400700 1 -- 172.21.15.86:0/2368442760 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b3c073810 msgr2=0x7f6b3c198eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:25.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.509+0000 7f6b39400700 1 --2- 172.21.15.86:0/2368442760 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b3c073810 0x7f6b3c198eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:25.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.509+0000 7f6b39400700 1 -- 172.21.15.86:0/2368442760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b3c108b40 msgr2=0x7f6b3c1a0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:25.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.509+0000 7f6b39400700 1 --2- 172.21.15.86:0/2368442760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b3c108b40 0x7f6b3c1a0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:25.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.509+0000 7f6b39400700 1 -- 172.21.15.86:0/2368442760 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b34009c40 con 0x7f6b3c074df0 2024-03-23T17:50:25.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.509+0000 7f6b39400700 1 --2- 172.21.15.86:0/2368442760 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b3c074df0 0x7f6b3c1993f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6b28009230 tx=0x7f6b2800bfa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:25.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.510+0000 7f6b32a00700 1 -- 172.21.15.86:0/2368442760 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b2801c070 con 0x7f6b3c074df0 2024-03-23T17:50:25.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.510+0000 7f6b32a00700 1 -- 172.21.15.86:0/2368442760 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b28005040 con 0x7f6b3c074df0 2024-03-23T17:50:25.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.510+0000 7f6b40e00700 1 -- 172.21.15.86:0/2368442760 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b28009520 con 0x7f6b3c074df0 2024-03-23T17:50:25.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.510+0000 7f6b32a00700 1 -- 172.21.15.86:0/2368442760 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b280038e0 con 0x7f6b3c074df0 2024-03-23T17:50:25.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.510+0000 7f6b40e00700 1 -- 172.21.15.86:0/2368442760 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b3c1a11c0 con 0x7f6b3c074df0 2024-03-23T17:50:25.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.511+0000 7f6b40e00700 1 -- 172.21.15.86:0/2368442760 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b3c107900 con 0x7f6b3c074df0 2024-03-23T17:50:25.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.513+0000 7f6b32a00700 1 -- 172.21.15.86:0/2368442760 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f6b28020030 con 0x7f6b3c074df0 2024-03-23T17:50:25.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.514+0000 7f6b32a00700 1 --2- 172.21.15.86:0/2368442760 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f6b200a11b0 0x7f6b200a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:25.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.514+0000 7f6b32a00700 1 -- 172.21.15.86:0/2368442760 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f6b280ce130 con 0x7f6b3c074df0 2024-03-23T17:50:25.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.514+0000 7f6b39e00700 1 --2- 172.21.15.86:0/2368442760 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f6b200a11b0 0x7f6b200a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:25.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.515+0000 7f6b39e00700 1 --2- 172.21.15.86:0/2368442760 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f6b200a11b0 0x7f6b200a3670 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c005fd0 tx=0x7f6b2c009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:25.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.516+0000 7f6b32a00700 1 -- 172.21.15.86:0/2368442760 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f6b280977c0 con 0x7f6b3c074df0 2024-03-23T17:50:25.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.718+0000 7f6b40e00700 1 -- 172.21.15.86:0/2368442760 --> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6b3c02cdc0 con 0x7f6b200a11b0 2024-03-23T17:50:25.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.729+0000 7f6b32a00700 1 -- 172.21.15.86:0/2368442760 <== mgr.14152 v2:172.21.15.86:6800/2734784244 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+344441 (secure 0 0 0) 0x7f6b3c02cdc0 con 0x7f6b200a11b0 2024-03-23T17:50:25.734 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:50:25.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.734+0000 7f6b40e00700 1 -- 172.21.15.86:0/2368442760 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f6b200a11b0 msgr2=0x7f6b200a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:25.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.734+0000 7f6b40e00700 1 --2- 172.21.15.86:0/2368442760 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f6b200a11b0 0x7f6b200a3670 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c005fd0 tx=0x7f6b2c009310 comp rx=0 tx=0).stop 2024-03-23T17:50:25.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.734+0000 7f6b40e00700 1 -- 172.21.15.86:0/2368442760 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b3c074df0 msgr2=0x7f6b3c1993f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:25.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.734+0000 7f6b40e00700 1 --2- 172.21.15.86:0/2368442760 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b3c074df0 0x7f6b3c1993f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6b28009230 tx=0x7f6b2800bfa0 comp rx=0 tx=0).stop 2024-03-23T17:50:25.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.734+0000 7f6b40e00700 1 -- 172.21.15.86:0/2368442760 shutdown_connections 2024-03-23T17:50:25.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.734+0000 7f6b40e00700 1 --2- 172.21.15.86:0/2368442760 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f6b200a11b0 0x7f6b200a3670 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:25.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.734+0000 7f6b40e00700 1 --2- 172.21.15.86:0/2368442760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b3c108b40 0x7f6b3c1a0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:25.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.734+0000 7f6b40e00700 1 --2- 172.21.15.86:0/2368442760 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b3c073810 0x7f6b3c198eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:25.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.734+0000 7f6b40e00700 1 --2- 172.21.15.86:0/2368442760 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b3c074df0 0x7f6b3c1993f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:25.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.734+0000 7f6b40e00700 1 -- 172.21.15.86:0/2368442760 >> 172.21.15.86:0/2368442760 conn(0x7f6b3c0fc1f0 msgr2=0x7f6b3c104a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:25.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.734+0000 7f6b40e00700 1 -- 172.21.15.86:0/2368442760 shutdown_connections 2024-03-23T17:50:25.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.735+0000 7f6b40e00700 1 -- 172.21.15.86:0/2368442760 wait complete. 2024-03-23T17:50:25.738 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-03-23T17:50:25.755 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":280,"stamp":"2024-03-23T17:50:24.329151+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661176,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":955,"num_read_kb":600,"num_write":418,"num_write_kb":3078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":1180916,"num_keys_recovered":17,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":345,"ondisk_log_size":345,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":373672,"kb_used_data":12568,"kb_used_omap":18,"kb_used_meta":356845,"kb_avail":1124568152,"statfs":{"total":1151940427776,"available":1151557787648,"internally_reserved":0,"allocated":12869632,"data_stored":7106887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19130,"internal_metadata":365409606},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4926,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":106,"num_read_kb":51,"num_write":7,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002979"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.782853+0000","last_change":"2024-03-23T17:48:56.239724+0000","last_active":"2024-03-23T17:49:28.782853+0000","last_peered":"2024-03-23T17:49:28.782853+0000","last_clean":"2024-03-23T17:49:28.782853+0000","last_became_active":"2024-03-23T17:48:56.239451+0000","last_became_peered":"2024-03-23T17:48:56.239451+0000","last_unstale":"2024-03-23T17:49:28.782853+0000","last_undegraded":"2024-03-23T17:49:28.782853+0000","last_fullsized":"2024-03-23T17:49:28.782853+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:51:38.687863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.783538+0000","last_change":"2024-03-23T17:48:56.232415+0000","last_active":"2024-03-23T17:49:28.783538+0000","last_peered":"2024-03-23T17:49:28.783538+0000","last_clean":"2024-03-23T17:49:28.783538+0000","last_became_active":"2024-03-23T17:48:56.231777+0000","last_became_peered":"2024-03-23T17:48:56.231777+0000","last_unstale":"2024-03-23T17:49:28.783538+0000","last_undegraded":"2024-03-23T17:49:28.783538+0000","last_fullsized":"2024-03-23T17:49:28.783538+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:16:22.631654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.784174+0000","last_change":"2024-03-23T17:48:56.243413+0000","last_active":"2024-03-23T17:49:28.784174+0000","last_peered":"2024-03-23T17:49:28.784174+0000","last_clean":"2024-03-23T17:49:28.784174+0000","last_became_active":"2024-03-23T17:48:56.241281+0000","last_became_peered":"2024-03-23T17:48:56.241281+0000","last_unstale":"2024-03-23T17:49:28.784174+0000","last_undegraded":"2024-03-23T17:49:28.784174+0000","last_fullsized":"2024-03-23T17:49:28.784174+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:52:48.521689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.784355+0000","last_change":"2024-03-23T17:48:56.226010+0000","last_active":"2024-03-23T17:49:28.784355+0000","last_peered":"2024-03-23T17:49:28.784355+0000","last_clean":"2024-03-23T17:49:28.784355+0000","last_became_active":"2024-03-23T17:48:56.225562+0000","last_became_peered":"2024-03-23T17:48:56.225562+0000","last_unstale":"2024-03-23T17:49:28.784355+0000","last_undegraded":"2024-03-23T17:49:28.784355+0000","last_fullsized":"2024-03-23T17:49:28.784355+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:17:47.597335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.780757+0000","last_change":"2024-03-23T17:48:56.223249+0000","last_active":"2024-03-23T17:49:28.780757+0000","last_peered":"2024-03-23T17:49:28.780757+0000","last_clean":"2024-03-23T17:49:28.780757+0000","last_became_active":"2024-03-23T17:48:56.222902+0000","last_became_peered":"2024-03-23T17:48:56.222902+0000","last_unstale":"2024-03-23T17:49:28.780757+0000","last_undegraded":"2024-03-23T17:49:28.780757+0000","last_fullsized":"2024-03-23T17:49:28.780757+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:51:49.029628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.781946+0000","last_change":"2024-03-23T17:48:56.229229+0000","last_active":"2024-03-23T17:49:28.781946+0000","last_peered":"2024-03-23T17:49:28.781946+0000","last_clean":"2024-03-23T17:49:28.781946+0000","last_became_active":"2024-03-23T17:48:56.228725+0000","last_became_peered":"2024-03-23T17:48:56.228725+0000","last_unstale":"2024-03-23T17:49:28.781946+0000","last_undegraded":"2024-03-23T17:49:28.781946+0000","last_fullsized":"2024-03-23T17:49:28.781946+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:53:44.118762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.780711+0000","last_change":"2024-03-23T17:48:56.229212+0000","last_active":"2024-03-23T17:49:28.780711+0000","last_peered":"2024-03-23T17:49:28.780711+0000","last_clean":"2024-03-23T17:49:28.780711+0000","last_became_active":"2024-03-23T17:48:56.228888+0000","last_became_peered":"2024-03-23T17:48:56.228888+0000","last_unstale":"2024-03-23T17:49:28.780711+0000","last_undegraded":"2024-03-23T17:49:28.780711+0000","last_fullsized":"2024-03-23T17:49:28.780711+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:47:21.509554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.779498+0000","last_change":"2024-03-23T17:48:56.222925+0000","last_active":"2024-03-23T17:49:28.779498+0000","last_peered":"2024-03-23T17:49:28.779498+0000","last_clean":"2024-03-23T17:49:28.779498+0000","last_became_active":"2024-03-23T17:48:56.221504+0000","last_became_peered":"2024-03-23T17:48:56.221504+0000","last_unstale":"2024-03-23T17:49:28.779498+0000","last_undegraded":"2024-03-23T17:49:28.779498+0000","last_fullsized":"2024-03-23T17:49:28.779498+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:14:34.327705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.922478+0000","last_change":"2024-03-23T17:48:56.230530+0000","last_active":"2024-03-23T17:49:28.922478+0000","last_peered":"2024-03-23T17:49:28.922478+0000","last_clean":"2024-03-23T17:49:28.922478+0000","last_became_active":"2024-03-23T17:48:56.230224+0000","last_became_peered":"2024-03-23T17:48:56.230224+0000","last_unstale":"2024-03-23T17:49:28.922478+0000","last_undegraded":"2024-03-23T17:49:28.922478+0000","last_fullsized":"2024-03-23T17:49:28.922478+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:10:28.031956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.781514+0000","last_change":"2024-03-23T17:48:56.229673+0000","last_active":"2024-03-23T17:49:28.781514+0000","last_peered":"2024-03-23T17:49:28.781514+0000","last_clean":"2024-03-23T17:49:28.781514+0000","last_became_active":"2024-03-23T17:48:56.229000+0000","last_became_peered":"2024-03-23T17:48:56.229000+0000","last_unstale":"2024-03-23T17:49:28.781514+0000","last_undegraded":"2024-03-23T17:49:28.781514+0000","last_fullsized":"2024-03-23T17:49:28.781514+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:23:28.057256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.781035+0000","last_change":"2024-03-23T17:48:56.224508+0000","last_active":"2024-03-23T17:49:28.781035+0000","last_peered":"2024-03-23T17:49:28.781035+0000","last_clean":"2024-03-23T17:49:28.781035+0000","last_became_active":"2024-03-23T17:48:56.224172+0000","last_became_peered":"2024-03-23T17:48:56.224172+0000","last_unstale":"2024-03-23T17:49:28.781035+0000","last_undegraded":"2024-03-23T17:49:28.781035+0000","last_fullsized":"2024-03-23T17:49:28.781035+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:03:57.063472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.784254+0000","last_change":"2024-03-23T17:48:56.234978+0000","last_active":"2024-03-23T17:49:28.784254+0000","last_peered":"2024-03-23T17:49:28.784254+0000","last_clean":"2024-03-23T17:49:28.784254+0000","last_became_active":"2024-03-23T17:48:56.234641+0000","last_became_peered":"2024-03-23T17:48:56.234641+0000","last_unstale":"2024-03-23T17:49:28.784254+0000","last_undegraded":"2024-03-23T17:49:28.784254+0000","last_fullsized":"2024-03-23T17:49:28.784254+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:34:30.745991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.777608+0000","last_change":"2024-03-23T17:48:56.218534+0000","last_active":"2024-03-23T17:49:28.777608+0000","last_peered":"2024-03-23T17:49:28.777608+0000","last_clean":"2024-03-23T17:49:28.777608+0000","last_became_active":"2024-03-23T17:48:56.217424+0000","last_became_peered":"2024-03-23T17:48:56.217424+0000","last_unstale":"2024-03-23T17:49:28.777608+0000","last_undegraded":"2024-03-23T17:49:28.777608+0000","last_fullsized":"2024-03-23T17:49:28.777608+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:32:05.618057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.783444+0000","last_change":"2024-03-23T17:48:56.235525+0000","last_active":"2024-03-23T17:49:28.783444+0000","last_peered":"2024-03-23T17:49:28.783444+0000","last_clean":"2024-03-23T17:49:28.783444+0000","last_became_active":"2024-03-23T17:48:56.234591+0000","last_became_peered":"2024-03-23T17:48:56.234591+0000","last_unstale":"2024-03-23T17:49:28.783444+0000","last_undegraded":"2024-03-23T17:49:28.783444+0000","last_fullsized":"2024-03-23T17:49:28.783444+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:19:38.381878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.783446+0000","last_change":"2024-03-23T17:48:56.243137+0000","last_active":"2024-03-23T17:49:28.783446+0000","last_peered":"2024-03-23T17:49:28.783446+0000","last_clean":"2024-03-23T17:49:28.783446+0000","last_became_active":"2024-03-23T17:48:56.242516+0000","last_became_peered":"2024-03-23T17:48:56.242516+0000","last_unstale":"2024-03-23T17:49:28.783446+0000","last_undegraded":"2024-03-23T17:49:28.783446+0000","last_fullsized":"2024-03-23T17:49:28.783446+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:54:59.849467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.922041+0000","last_change":"2024-03-23T17:48:56.236026+0000","last_active":"2024-03-23T17:49:28.922041+0000","last_peered":"2024-03-23T17:49:28.922041+0000","last_clean":"2024-03-23T17:49:28.922041+0000","last_became_active":"2024-03-23T17:48:56.235645+0000","last_became_peered":"2024-03-23T17:48:56.235645+0000","last_unstale":"2024-03-23T17:49:28.922041+0000","last_undegraded":"2024-03-23T17:49:28.922041+0000","last_fullsized":"2024-03-23T17:49:28.922041+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:47:01.076737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.782865+0000","last_change":"2024-03-23T17:48:56.239689+0000","last_active":"2024-03-23T17:49:28.782865+0000","last_peered":"2024-03-23T17:49:28.782865+0000","last_clean":"2024-03-23T17:49:28.782865+0000","last_became_active":"2024-03-23T17:48:56.239331+0000","last_became_peered":"2024-03-23T17:48:56.239331+0000","last_unstale":"2024-03-23T17:49:28.782865+0000","last_undegraded":"2024-03-23T17:49:28.782865+0000","last_fullsized":"2024-03-23T17:49:28.782865+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:21:15.287627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.777987+0000","last_change":"2024-03-23T17:48:56.229934+0000","last_active":"2024-03-23T17:49:28.777987+0000","last_peered":"2024-03-23T17:49:28.777987+0000","last_clean":"2024-03-23T17:49:28.777987+0000","last_became_active":"2024-03-23T17:48:56.229244+0000","last_became_peered":"2024-03-23T17:48:56.229244+0000","last_unstale":"2024-03-23T17:49:28.777987+0000","last_undegraded":"2024-03-23T17:49:28.777987+0000","last_fullsized":"2024-03-23T17:49:28.777987+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:02:40.254799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.784936+0000","last_change":"2024-03-23T17:48:56.233786+0000","last_active":"2024-03-23T17:49:28.784936+0000","last_peered":"2024-03-23T17:49:28.784936+0000","last_clean":"2024-03-23T17:49:28.784936+0000","last_became_active":"2024-03-23T17:48:56.233137+0000","last_became_peered":"2024-03-23T17:48:56.233137+0000","last_unstale":"2024-03-23T17:49:28.784936+0000","last_undegraded":"2024-03-23T17:49:28.784936+0000","last_fullsized":"2024-03-23T17:49:28.784936+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:33:35.202788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"82'14","reported_seq":108,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-23T17:50:20.814129+0000","last_change":"2024-03-23T17:48:55.712184+0000","last_active":"2024-03-23T17:50:20.814129+0000","last_peered":"2024-03-23T17:50:20.814129+0000","last_clean":"2024-03-23T17:50:20.814129+0000","last_became_active":"2024-03-23T17:48:55.711860+0000","last_became_peered":"2024-03-23T17:48:55.711860+0000","last_unstale":"2024-03-23T17:50:20.814129+0000","last_undegraded":"2024-03-23T17:50:20.814129+0000","last_fullsized":"2024-03-23T17:50:20.814129+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:59:01.133652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":80,"num_read_kb":80,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-23T17:49:29.789705+0000","last_change":"2024-03-23T17:49:29.789057+0000","last_active":"2024-03-23T17:49:29.789705+0000","last_peered":"2024-03-23T17:49:29.789705+0000","last_clean":"2024-03-23T17:49:29.789705+0000","last_became_active":"2024-03-23T17:49:29.788796+0000","last_became_peered":"2024-03-23T17:49:29.788796+0000","last_unstale":"2024-03-23T17:49:29.789705+0000","last_undegraded":"2024-03-23T17:49:29.789705+0000","last_fullsized":"2024-03-23T17:49:29.789705+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:30:53.062926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.780493+0000","last_change":"2024-03-23T17:48:55.725351+0000","last_active":"2024-03-23T17:49:28.780493+0000","last_peered":"2024-03-23T17:49:28.780493+0000","last_clean":"2024-03-23T17:49:28.780493+0000","last_became_active":"2024-03-23T17:48:55.724566+0000","last_became_peered":"2024-03-23T17:48:55.724566+0000","last_unstale":"2024-03-23T17:49:28.780493+0000","last_undegraded":"2024-03-23T17:49:28.780493+0000","last_fullsized":"2024-03-23T17:49:28.780493+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:58:21.088166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.784151+0000","last_change":"2024-03-23T17:48:56.243526+0000","last_active":"2024-03-23T17:49:28.784151+0000","last_peered":"2024-03-23T17:49:28.784151+0000","last_clean":"2024-03-23T17:49:28.784151+0000","last_became_active":"2024-03-23T17:48:56.242468+0000","last_became_peered":"2024-03-23T17:48:56.242468+0000","last_unstale":"2024-03-23T17:49:28.784151+0000","last_undegraded":"2024-03-23T17:49:28.784151+0000","last_fullsized":"2024-03-23T17:49:28.784151+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:16:50.392770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"82'13","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-23T17:49:29.781747+0000","last_change":"2024-03-23T17:48:55.720237+0000","last_active":"2024-03-23T17:49:29.781747+0000","last_peered":"2024-03-23T17:49:29.781747+0000","last_clean":"2024-03-23T17:49:29.781747+0000","last_became_active":"2024-03-23T17:48:55.719843+0000","last_became_peered":"2024-03-23T17:48:55.719843+0000","last_unstale":"2024-03-23T17:49:29.781747+0000","last_undegraded":"2024-03-23T17:49:29.781747+0000","last_fullsized":"2024-03-23T17:49:29.781747+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:07:26.973511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.922177+0000","last_change":"2024-03-23T17:48:56.235642+0000","last_active":"2024-03-23T17:49:28.922177+0000","last_peered":"2024-03-23T17:49:28.922177+0000","last_clean":"2024-03-23T17:49:28.922177+0000","last_became_active":"2024-03-23T17:48:56.235326+0000","last_became_peered":"2024-03-23T17:48:56.235326+0000","last_unstale":"2024-03-23T17:49:28.922177+0000","last_undegraded":"2024-03-23T17:49:28.922177+0000","last_fullsized":"2024-03-23T17:49:28.922177+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:21:59.934620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.781362+0000","last_change":"2024-03-23T17:48:55.718579+0000","last_active":"2024-03-23T17:49:28.781362+0000","last_peered":"2024-03-23T17:49:28.781362+0000","last_clean":"2024-03-23T17:49:28.781362+0000","last_became_active":"2024-03-23T17:48:55.718220+0000","last_became_peered":"2024-03-23T17:48:55.718220+0000","last_unstale":"2024-03-23T17:49:28.781362+0000","last_undegraded":"2024-03-23T17:49:28.781362+0000","last_fullsized":"2024-03-23T17:49:28.781362+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:37:56.417367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.785628+0000","last_change":"2024-03-23T17:48:56.224842+0000","last_active":"2024-03-23T17:49:28.785628+0000","last_peered":"2024-03-23T17:49:28.785628+0000","last_clean":"2024-03-23T17:49:28.785628+0000","last_became_active":"2024-03-23T17:48:56.224427+0000","last_became_peered":"2024-03-23T17:48:56.224427+0000","last_unstale":"2024-03-23T17:49:28.785628+0000","last_undegraded":"2024-03-23T17:49:28.785628+0000","last_fullsized":"2024-03-23T17:49:28.785628+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:16:22.684369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"80'5","reported_seq":103,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-23T17:50:20.781361+0000","last_change":"2024-03-23T17:48:55.713996+0000","last_active":"2024-03-23T17:50:20.781361+0000","last_peered":"2024-03-23T17:50:20.781361+0000","last_clean":"2024-03-23T17:50:20.781361+0000","last_became_active":"2024-03-23T17:48:55.713647+0000","last_became_peered":"2024-03-23T17:48:55.713647+0000","last_unstale":"2024-03-23T17:50:20.781361+0000","last_undegraded":"2024-03-23T17:50:20.781361+0000","last_fullsized":"2024-03-23T17:50:20.781361+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:12:22.251533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":84,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.781148+0000","last_change":"2024-03-23T17:48:56.219281+0000","last_active":"2024-03-23T17:49:28.781148+0000","last_peered":"2024-03-23T17:49:28.781148+0000","last_clean":"2024-03-23T17:49:28.781148+0000","last_became_active":"2024-03-23T17:48:56.218924+0000","last_became_peered":"2024-03-23T17:48:56.218924+0000","last_unstale":"2024-03-23T17:49:28.781148+0000","last_undegraded":"2024-03-23T17:49:28.781148+0000","last_fullsized":"2024-03-23T17:49:28.781148+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:49:04.793088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.781333+0000","last_change":"2024-03-23T17:48:55.718131+0000","last_active":"2024-03-23T17:49:28.781333+0000","last_peered":"2024-03-23T17:49:28.781333+0000","last_clean":"2024-03-23T17:49:28.781333+0000","last_became_active":"2024-03-23T17:48:55.717760+0000","last_became_peered":"2024-03-23T17:48:55.717760+0000","last_unstale":"2024-03-23T17:49:28.781333+0000","last_undegraded":"2024-03-23T17:49:28.781333+0000","last_fullsized":"2024-03-23T17:49:28.781333+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:16:54.355844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.784309+0000","last_change":"2024-03-23T17:48:56.225965+0000","last_active":"2024-03-23T17:49:28.784309+0000","last_peered":"2024-03-23T17:49:28.784309+0000","last_clean":"2024-03-23T17:49:28.784309+0000","last_became_active":"2024-03-23T17:48:56.225101+0000","last_became_peered":"2024-03-23T17:48:56.225101+0000","last_unstale":"2024-03-23T17:49:28.784309+0000","last_undegraded":"2024-03-23T17:49:28.784309+0000","last_fullsized":"2024-03-23T17:49:28.784309+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:52:14.302306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.776847+0000","last_change":"2024-03-23T17:48:55.722289+0000","last_active":"2024-03-23T17:49:28.776847+0000","last_peered":"2024-03-23T17:49:28.776847+0000","last_clean":"2024-03-23T17:49:28.776847+0000","last_became_active":"2024-03-23T17:48:55.721927+0000","last_became_peered":"2024-03-23T17:48:55.721927+0000","last_unstale":"2024-03-23T17:49:28.776847+0000","last_undegraded":"2024-03-23T17:49:28.776847+0000","last_fullsized":"2024-03-23T17:49:28.776847+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:20:08.354897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.779465+0000","last_change":"2024-03-23T17:48:56.232231+0000","last_active":"2024-03-23T17:49:28.779465+0000","last_peered":"2024-03-23T17:49:28.779465+0000","last_clean":"2024-03-23T17:49:28.779465+0000","last_became_active":"2024-03-23T17:48:56.227335+0000","last_became_peered":"2024-03-23T17:48:56.227335+0000","last_unstale":"2024-03-23T17:49:28.779465+0000","last_undegraded":"2024-03-23T17:49:28.779465+0000","last_fullsized":"2024-03-23T17:49:28.779465+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:49:01.460775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.785071+0000","last_change":"2024-03-23T17:48:55.717615+0000","last_active":"2024-03-23T17:49:28.785071+0000","last_peered":"2024-03-23T17:49:28.785071+0000","last_clean":"2024-03-23T17:49:28.785071+0000","last_became_active":"2024-03-23T17:48:55.717266+0000","last_became_peered":"2024-03-23T17:48:55.717266+0000","last_unstale":"2024-03-23T17:49:28.785071+0000","last_undegraded":"2024-03-23T17:49:28.785071+0000","last_fullsized":"2024-03-23T17:49:28.785071+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:40:07.988182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.776405+0000","last_change":"2024-03-23T17:48:56.229787+0000","last_active":"2024-03-23T17:49:28.776405+0000","last_peered":"2024-03-23T17:49:28.776405+0000","last_clean":"2024-03-23T17:49:28.776405+0000","last_became_active":"2024-03-23T17:48:56.229471+0000","last_became_peered":"2024-03-23T17:48:56.229471+0000","last_unstale":"2024-03-23T17:49:28.776405+0000","last_undegraded":"2024-03-23T17:49:28.776405+0000","last_fullsized":"2024-03-23T17:49:28.776405+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:44:09.358809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'4","reported_seq":173,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-23T17:50:18.777100+0000","last_change":"2024-03-23T17:48:55.717130+0000","last_active":"2024-03-23T17:50:18.777100+0000","last_peered":"2024-03-23T17:50:18.777100+0000","last_clean":"2024-03-23T17:50:18.777100+0000","last_became_active":"2024-03-23T17:48:55.716733+0000","last_became_peered":"2024-03-23T17:48:55.716733+0000","last_unstale":"2024-03-23T17:50:18.777100+0000","last_undegraded":"2024-03-23T17:50:18.777100+0000","last_fullsized":"2024-03-23T17:50:18.777100+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:22:43.229694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":155,"num_read_kb":155,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.922446+0000","last_change":"2024-03-23T17:48:56.235565+0000","last_active":"2024-03-23T17:49:28.922446+0000","last_peered":"2024-03-23T17:49:28.922446+0000","last_clean":"2024-03-23T17:49:28.922446+0000","last_became_active":"2024-03-23T17:48:56.235267+0000","last_became_peered":"2024-03-23T17:48:56.235267+0000","last_unstale":"2024-03-23T17:49:28.922446+0000","last_undegraded":"2024-03-23T17:49:28.922446+0000","last_fullsized":"2024-03-23T17:49:28.922446+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:38:45.906545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.780900+0000","last_change":"2024-03-23T17:48:55.719096+0000","last_active":"2024-03-23T17:49:28.780900+0000","last_peered":"2024-03-23T17:49:28.780900+0000","last_clean":"2024-03-23T17:49:28.780900+0000","last_became_active":"2024-03-23T17:48:55.718721+0000","last_became_peered":"2024-03-23T17:48:55.718721+0000","last_unstale":"2024-03-23T17:49:28.780900+0000","last_undegraded":"2024-03-23T17:49:28.780900+0000","last_fullsized":"2024-03-23T17:49:28.780900+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:43:05.532263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.784108+0000","last_change":"2024-03-23T17:48:56.235549+0000","last_active":"2024-03-23T17:49:28.784108+0000","last_peered":"2024-03-23T17:49:28.784108+0000","last_clean":"2024-03-23T17:49:28.784108+0000","last_became_active":"2024-03-23T17:48:56.234572+0000","last_became_peered":"2024-03-23T17:48:56.234572+0000","last_unstale":"2024-03-23T17:49:28.784108+0000","last_undegraded":"2024-03-23T17:49:28.784108+0000","last_fullsized":"2024-03-23T17:49:28.784108+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:19:58.639181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.785732+0000","last_change":"2024-03-23T17:48:55.724688+0000","last_active":"2024-03-23T17:49:28.785732+0000","last_peered":"2024-03-23T17:49:28.785732+0000","last_clean":"2024-03-23T17:49:28.785732+0000","last_became_active":"2024-03-23T17:48:55.724385+0000","last_became_peered":"2024-03-23T17:48:55.724385+0000","last_unstale":"2024-03-23T17:49:28.785732+0000","last_undegraded":"2024-03-23T17:49:28.785732+0000","last_fullsized":"2024-03-23T17:49:28.785732+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:17:54.338059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.922287+0000","last_change":"2024-03-23T17:48:56.242983+0000","last_active":"2024-03-23T17:49:28.922287+0000","last_peered":"2024-03-23T17:49:28.922287+0000","last_clean":"2024-03-23T17:49:28.922287+0000","last_became_active":"2024-03-23T17:48:56.242596+0000","last_became_peered":"2024-03-23T17:48:56.242596+0000","last_unstale":"2024-03-23T17:49:28.922287+0000","last_undegraded":"2024-03-23T17:49:28.922287+0000","last_fullsized":"2024-03-23T17:49:28.922287+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:55:51.971834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'32","reported_seq":211,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-23T17:50:19.225390+0000","last_change":"2024-03-23T17:48:55.717631+0000","last_active":"2024-03-23T17:50:19.225390+0000","last_peered":"2024-03-23T17:50:19.225390+0000","last_clean":"2024-03-23T17:50:19.225390+0000","last_became_active":"2024-03-23T17:48:55.717082+0000","last_became_peered":"2024-03-23T17:48:55.717082+0000","last_unstale":"2024-03-23T17:50:19.225390+0000","last_undegraded":"2024-03-23T17:50:19.225390+0000","last_fullsized":"2024-03-23T17:50:19.225390+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:36:25.323269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64038,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":165,"num_read_kb":140,"num_write":34,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.780792+0000","last_change":"2024-03-23T17:48:56.226174+0000","last_active":"2024-03-23T17:49:28.780792+0000","last_peered":"2024-03-23T17:49:28.780792+0000","last_clean":"2024-03-23T17:49:28.780792+0000","last_became_active":"2024-03-23T17:48:56.225834+0000","last_became_peered":"2024-03-23T17:48:56.225834+0000","last_unstale":"2024-03-23T17:49:28.780792+0000","last_undegraded":"2024-03-23T17:49:28.780792+0000","last_fullsized":"2024-03-23T17:49:28.780792+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:50:16.793071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-23T17:49:29.804494+0000","last_change":"2024-03-23T17:49:29.804099+0000","last_active":"2024-03-23T17:49:29.804494+0000","last_peered":"2024-03-23T17:49:29.804494+0000","last_clean":"2024-03-23T17:49:29.804494+0000","last_became_active":"2024-03-23T17:49:29.787483+0000","last_became_peered":"2024-03-23T17:49:29.787483+0000","last_unstale":"2024-03-23T17:49:29.804494+0000","last_undegraded":"2024-03-23T17:49:29.804494+0000","last_fullsized":"2024-03-23T17:49:29.804494+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:13:16.463738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.776667+0000","last_change":"2024-03-23T17:48:56.228459+0000","last_active":"2024-03-23T17:49:28.776667+0000","last_peered":"2024-03-23T17:49:28.776667+0000","last_clean":"2024-03-23T17:49:28.776667+0000","last_became_active":"2024-03-23T17:48:56.228179+0000","last_became_peered":"2024-03-23T17:48:56.228179+0000","last_unstale":"2024-03-23T17:49:28.776667+0000","last_undegraded":"2024-03-23T17:49:28.776667+0000","last_fullsized":"2024-03-23T17:49:28.776667+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:28:41.613432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.785318+0000","last_change":"2024-03-23T17:48:55.721533+0000","last_active":"2024-03-23T17:49:28.785318+0000","last_peered":"2024-03-23T17:49:28.785318+0000","last_clean":"2024-03-23T17:49:28.785318+0000","last_became_active":"2024-03-23T17:48:55.721100+0000","last_became_peered":"2024-03-23T17:48:55.721100+0000","last_unstale":"2024-03-23T17:49:28.785318+0000","last_undegraded":"2024-03-23T17:49:28.785318+0000","last_fullsized":"2024-03-23T17:49:28.785318+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:56:58.112649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-23T17:49:30.003965+0000","last_change":"2024-03-23T17:49:30.003965+0000","last_active":"2024-03-23T17:49:30.003965+0000","last_peered":"2024-03-23T17:49:30.003965+0000","last_clean":"2024-03-23T17:49:30.003965+0000","last_became_active":"2024-03-23T17:49:30.003643+0000","last_became_peered":"2024-03-23T17:49:30.003643+0000","last_unstale":"2024-03-23T17:49:30.003965+0000","last_undegraded":"2024-03-23T17:49:30.003965+0000","last_fullsized":"2024-03-23T17:49:30.003965+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:47:39.892782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'5","reported_seq":97,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-23T17:50:17.824512+0000","last_change":"2024-03-23T17:48:55.715205+0000","last_active":"2024-03-23T17:50:17.824512+0000","last_peered":"2024-03-23T17:50:17.824512+0000","last_clean":"2024-03-23T17:50:17.824512+0000","last_became_active":"2024-03-23T17:48:55.714810+0000","last_became_peered":"2024-03-23T17:48:55.714810+0000","last_unstale":"2024-03-23T17:50:17.824512+0000","last_undegraded":"2024-03-23T17:50:17.824512+0000","last_fullsized":"2024-03-23T17:50:17.824512+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:40:08.768998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":79,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-23T17:49:29.783025+0000","last_change":"2024-03-23T17:49:29.783025+0000","last_active":"2024-03-23T17:49:29.783025+0000","last_peered":"2024-03-23T17:49:29.783025+0000","last_clean":"2024-03-23T17:49:29.783025+0000","last_became_active":"2024-03-23T17:49:29.782541+0000","last_became_peered":"2024-03-23T17:49:29.782541+0000","last_unstale":"2024-03-23T17:49:29.783025+0000","last_undegraded":"2024-03-23T17:49:29.783025+0000","last_fullsized":"2024-03-23T17:49:29.783025+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:50:29.754074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":59,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-23T17:49:30.616960+0000","last_change":"2024-03-23T17:49:30.616960+0000","last_active":"2024-03-23T17:49:30.616960+0000","last_peered":"2024-03-23T17:49:30.616960+0000","last_clean":"2024-03-23T17:49:30.616960+0000","last_became_active":"2024-03-23T17:49:30.604464+0000","last_became_peered":"2024-03-23T17:49:30.604464+0000","last_unstale":"2024-03-23T17:49:30.616960+0000","last_undegraded":"2024-03-23T17:49:30.616960+0000","last_fullsized":"2024-03-23T17:49:30.616960+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:24:13.862841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.785079+0000","last_change":"2024-03-23T17:48:56.225624+0000","last_active":"2024-03-23T17:49:28.785079+0000","last_peered":"2024-03-23T17:49:28.785079+0000","last_clean":"2024-03-23T17:49:28.785079+0000","last_became_active":"2024-03-23T17:48:56.223798+0000","last_became_peered":"2024-03-23T17:48:56.223798+0000","last_unstale":"2024-03-23T17:49:28.785079+0000","last_undegraded":"2024-03-23T17:49:28.785079+0000","last_fullsized":"2024-03-23T17:49:28.785079+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:00:20.547553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.781323+0000","last_change":"2024-03-23T17:48:55.717792+0000","last_active":"2024-03-23T17:49:28.781323+0000","last_peered":"2024-03-23T17:49:28.781323+0000","last_clean":"2024-03-23T17:49:28.781323+0000","last_became_active":"2024-03-23T17:48:55.717479+0000","last_became_peered":"2024-03-23T17:48:55.717479+0000","last_unstale":"2024-03-23T17:49:28.781323+0000","last_undegraded":"2024-03-23T17:49:28.781323+0000","last_fullsized":"2024-03-23T17:49:28.781323+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:49:43.483785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.782572+0000","last_change":"2024-03-23T17:48:56.219135+0000","last_active":"2024-03-23T17:49:28.782572+0000","last_peered":"2024-03-23T17:49:28.782572+0000","last_clean":"2024-03-23T17:49:28.782572+0000","last_became_active":"2024-03-23T17:48:56.218824+0000","last_became_peered":"2024-03-23T17:48:56.218824+0000","last_unstale":"2024-03-23T17:49:28.782572+0000","last_undegraded":"2024-03-23T17:49:28.782572+0000","last_fullsized":"2024-03-23T17:49:28.782572+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:20:35.190781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.776774+0000","last_change":"2024-03-23T17:48:55.717530+0000","last_active":"2024-03-23T17:49:28.776774+0000","last_peered":"2024-03-23T17:49:28.776774+0000","last_clean":"2024-03-23T17:49:28.776774+0000","last_became_active":"2024-03-23T17:48:55.717213+0000","last_became_peered":"2024-03-23T17:48:55.717213+0000","last_unstale":"2024-03-23T17:49:28.776774+0000","last_undegraded":"2024-03-23T17:49:28.776774+0000","last_fullsized":"2024-03-23T17:49:28.776774+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:40:09.803021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'1","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.781617+0000","last_change":"2024-03-23T17:48:56.222237+0000","last_active":"2024-03-23T17:49:28.781617+0000","last_peered":"2024-03-23T17:49:28.781617+0000","last_clean":"2024-03-23T17:49:28.781617+0000","last_became_active":"2024-03-23T17:48:56.221916+0000","last_became_peered":"2024-03-23T17:48:56.221916+0000","last_unstale":"2024-03-23T17:49:28.781617+0000","last_undegraded":"2024-03-23T17:49:28.781617+0000","last_fullsized":"2024-03-23T17:49:28.781617+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:35:59.213482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"82'6","reported_seq":100,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-23T17:50:19.813475+0000","last_change":"2024-03-23T17:48:55.718143+0000","last_active":"2024-03-23T17:50:19.813475+0000","last_peered":"2024-03-23T17:50:19.813475+0000","last_clean":"2024-03-23T17:50:19.813475+0000","last_became_active":"2024-03-23T17:48:55.717834+0000","last_became_peered":"2024-03-23T17:48:55.717834+0000","last_unstale":"2024-03-23T17:50:19.813475+0000","last_undegraded":"2024-03-23T17:50:19.813475+0000","last_fullsized":"2024-03-23T17:50:19.813475+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:29:08.879636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":846,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":80,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.781385+0000","last_change":"2024-03-23T17:48:56.237136+0000","last_active":"2024-03-23T17:49:28.781385+0000","last_peered":"2024-03-23T17:49:28.781385+0000","last_clean":"2024-03-23T17:49:28.781385+0000","last_became_active":"2024-03-23T17:48:56.236795+0000","last_became_peered":"2024-03-23T17:48:56.236795+0000","last_unstale":"2024-03-23T17:49:28.781385+0000","last_undegraded":"2024-03-23T17:49:28.781385+0000","last_fullsized":"2024-03-23T17:49:28.781385+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:15:31.356382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-23T17:49:29.789433+0000","last_change":"2024-03-23T17:49:29.789433+0000","last_active":"2024-03-23T17:49:29.789433+0000","last_peered":"2024-03-23T17:49:29.789433+0000","last_clean":"2024-03-23T17:49:29.789433+0000","last_became_active":"2024-03-23T17:49:29.788956+0000","last_became_peered":"2024-03-23T17:49:29.788956+0000","last_unstale":"2024-03-23T17:49:29.789433+0000","last_undegraded":"2024-03-23T17:49:29.789433+0000","last_fullsized":"2024-03-23T17:49:29.789433+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:43:30.929796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"80'5","reported_seq":96,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-23T17:50:17.823614+0000","last_change":"2024-03-23T17:48:55.722968+0000","last_active":"2024-03-23T17:50:17.823614+0000","last_peered":"2024-03-23T17:50:17.823614+0000","last_clean":"2024-03-23T17:50:17.823614+0000","last_became_active":"2024-03-23T17:48:55.722578+0000","last_became_peered":"2024-03-23T17:48:55.722578+0000","last_unstale":"2024-03-23T17:50:17.823614+0000","last_undegraded":"2024-03-23T17:50:17.823614+0000","last_fullsized":"2024-03-23T17:50:17.823614+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:45:42.344283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":77,"num_read_kb":77,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.779906+0000","last_change":"2024-03-23T17:48:56.228000+0000","last_active":"2024-03-23T17:49:28.779906+0000","last_peered":"2024-03-23T17:49:28.779906+0000","last_clean":"2024-03-23T17:49:28.779906+0000","last_became_active":"2024-03-23T17:48:56.225645+0000","last_became_peered":"2024-03-23T17:48:56.225645+0000","last_unstale":"2024-03-23T17:49:28.779906+0000","last_undegraded":"2024-03-23T17:49:28.779906+0000","last_fullsized":"2024-03-23T17:49:28.779906+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:44:25.197761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.784417+0000","last_change":"2024-03-23T17:48:55.718307+0000","last_active":"2024-03-23T17:49:28.784417+0000","last_peered":"2024-03-23T17:49:28.784417+0000","last_clean":"2024-03-23T17:49:28.784417+0000","last_became_active":"2024-03-23T17:48:55.717839+0000","last_became_peered":"2024-03-23T17:48:55.717839+0000","last_unstale":"2024-03-23T17:49:28.784417+0000","last_undegraded":"2024-03-23T17:49:28.784417+0000","last_fullsized":"2024-03-23T17:49:28.784417+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:11:56.053940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":335,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.780763+0000","last_change":"2024-03-23T17:48:01.618019+0000","last_active":"2024-03-23T17:49:28.780763+0000","last_peered":"2024-03-23T17:49:28.780763+0000","last_clean":"2024-03-23T17:49:28.780763+0000","last_became_active":"2024-03-23T17:48:01.617553+0000","last_became_peered":"2024-03-23T17:48:01.617553+0000","last_unstale":"2024-03-23T17:49:28.780763+0000","last_undegraded":"2024-03-23T17:49:28.780763+0000","last_fullsized":"2024-03-23T17:49:28.780763+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:45:26.849029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:45:26.849029+0000","last_clean_scrub_stamp":"2024-03-23T17:45:26.849029+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:05:44.823364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.776693+0000","last_change":"2024-03-23T17:48:56.224285+0000","last_active":"2024-03-23T17:49:28.776693+0000","last_peered":"2024-03-23T17:49:28.776693+0000","last_clean":"2024-03-23T17:49:28.776693+0000","last_became_active":"2024-03-23T17:48:56.220943+0000","last_became_peered":"2024-03-23T17:48:56.220943+0000","last_unstale":"2024-03-23T17:49:28.776693+0000","last_undegraded":"2024-03-23T17:49:28.776693+0000","last_fullsized":"2024-03-23T17:49:28.776693+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:36:05.458243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.780994+0000","last_change":"2024-03-23T17:48:55.715877+0000","last_active":"2024-03-23T17:49:28.780994+0000","last_peered":"2024-03-23T17:49:28.780994+0000","last_clean":"2024-03-23T17:49:28.780994+0000","last_became_active":"2024-03-23T17:48:55.715550+0000","last_became_peered":"2024-03-23T17:48:55.715550+0000","last_unstale":"2024-03-23T17:49:28.780994+0000","last_undegraded":"2024-03-23T17:49:28.780994+0000","last_fullsized":"2024-03-23T17:49:28.780994+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:31:57.668495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.780911+0000","last_change":"2024-03-23T17:48:55.724618+0000","last_active":"2024-03-23T17:49:28.780911+0000","last_peered":"2024-03-23T17:49:28.780911+0000","last_clean":"2024-03-23T17:49:28.780911+0000","last_became_active":"2024-03-23T17:48:55.724278+0000","last_became_peered":"2024-03-23T17:48:55.724278+0000","last_unstale":"2024-03-23T17:49:28.780911+0000","last_undegraded":"2024-03-23T17:49:28.780911+0000","last_fullsized":"2024-03-23T17:49:28.780911+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:27:05.065115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.776532+0000","last_change":"2024-03-23T17:48:56.216141+0000","last_active":"2024-03-23T17:49:28.776532+0000","last_peered":"2024-03-23T17:49:28.776532+0000","last_clean":"2024-03-23T17:49:28.776532+0000","last_became_active":"2024-03-23T17:48:56.215653+0000","last_became_peered":"2024-03-23T17:48:56.215653+0000","last_unstale":"2024-03-23T17:49:28.776532+0000","last_undegraded":"2024-03-23T17:49:28.776532+0000","last_fullsized":"2024-03-23T17:49:28.776532+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:02:19.611056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'9","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-23T17:49:29.794454+0000","last_change":"2024-03-23T17:49:29.794454+0000","last_active":"2024-03-23T17:49:29.794454+0000","last_peered":"2024-03-23T17:49:29.794454+0000","last_clean":"2024-03-23T17:49:29.794454+0000","last_became_active":"2024-03-23T17:49:29.787238+0000","last_became_peered":"2024-03-23T17:49:29.787238+0000","last_unstale":"2024-03-23T17:49:29.794454+0000","last_undegraded":"2024-03-23T17:49:29.794454+0000","last_fullsized":"2024-03-23T17:49:29.794454+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:33:34.917283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.780480+0000","last_change":"2024-03-23T17:48:56.228439+0000","last_active":"2024-03-23T17:49:28.780480+0000","last_peered":"2024-03-23T17:49:28.780480+0000","last_clean":"2024-03-23T17:49:28.780480+0000","last_became_active":"2024-03-23T17:48:56.226677+0000","last_became_peered":"2024-03-23T17:48:56.226677+0000","last_unstale":"2024-03-23T17:49:28.780480+0000","last_undegraded":"2024-03-23T17:49:28.780480+0000","last_fullsized":"2024-03-23T17:49:28.780480+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:46:00.063266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.784329+0000","last_change":"2024-03-23T17:48:55.724872+0000","last_active":"2024-03-23T17:49:28.784329+0000","last_peered":"2024-03-23T17:49:28.784329+0000","last_clean":"2024-03-23T17:49:28.784329+0000","last_became_active":"2024-03-23T17:48:55.724468+0000","last_became_peered":"2024-03-23T17:48:55.724468+0000","last_unstale":"2024-03-23T17:49:28.784329+0000","last_undegraded":"2024-03-23T17:49:28.784329+0000","last_fullsized":"2024-03-23T17:49:28.784329+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:28:34.918056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.783623+0000","last_change":"2024-03-23T17:48:56.233730+0000","last_active":"2024-03-23T17:49:28.783623+0000","last_peered":"2024-03-23T17:49:28.783623+0000","last_clean":"2024-03-23T17:49:28.783623+0000","last_became_active":"2024-03-23T17:48:56.233189+0000","last_became_peered":"2024-03-23T17:48:56.233189+0000","last_unstale":"2024-03-23T17:49:28.783623+0000","last_undegraded":"2024-03-23T17:49:28.783623+0000","last_fullsized":"2024-03-23T17:49:28.783623+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:20:30.764344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.922070+0000","last_change":"2024-03-23T17:48:55.724633+0000","last_active":"2024-03-23T17:49:28.922070+0000","last_peered":"2024-03-23T17:49:28.922070+0000","last_clean":"2024-03-23T17:49:28.922070+0000","last_became_active":"2024-03-23T17:48:55.724301+0000","last_became_peered":"2024-03-23T17:48:55.724301+0000","last_unstale":"2024-03-23T17:49:28.922070+0000","last_undegraded":"2024-03-23T17:49:28.922070+0000","last_fullsized":"2024-03-23T17:49:28.922070+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:45:08.392668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.780692+0000","last_change":"2024-03-23T17:48:56.230981+0000","last_active":"2024-03-23T17:49:28.780692+0000","last_peered":"2024-03-23T17:49:28.780692+0000","last_clean":"2024-03-23T17:49:28.780692+0000","last_became_active":"2024-03-23T17:48:56.230644+0000","last_became_peered":"2024-03-23T17:48:56.230644+0000","last_unstale":"2024-03-23T17:49:28.780692+0000","last_undegraded":"2024-03-23T17:49:28.780692+0000","last_fullsized":"2024-03-23T17:49:28.780692+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:37:51.310938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.779873+0000","last_change":"2024-03-23T17:48:55.719112+0000","last_active":"2024-03-23T17:49:28.779873+0000","last_peered":"2024-03-23T17:49:28.779873+0000","last_clean":"2024-03-23T17:49:28.779873+0000","last_became_active":"2024-03-23T17:48:55.718756+0000","last_became_peered":"2024-03-23T17:48:55.718756+0000","last_unstale":"2024-03-23T17:49:28.779873+0000","last_undegraded":"2024-03-23T17:49:28.779873+0000","last_fullsized":"2024-03-23T17:49:28.779873+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:41:22.164706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.781982+0000","last_change":"2024-03-23T17:48:56.240289+0000","last_active":"2024-03-23T17:49:28.781982+0000","last_peered":"2024-03-23T17:49:28.781982+0000","last_clean":"2024-03-23T17:49:28.781982+0000","last_became_active":"2024-03-23T17:48:56.239896+0000","last_became_peered":"2024-03-23T17:48:56.239896+0000","last_unstale":"2024-03-23T17:49:28.781982+0000","last_undegraded":"2024-03-23T17:49:28.781982+0000","last_fullsized":"2024-03-23T17:49:28.781982+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:49:10.505576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'3","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.785515+0000","last_change":"2024-03-23T17:48:55.723330+0000","last_active":"2024-03-23T17:49:28.785515+0000","last_peered":"2024-03-23T17:49:28.785515+0000","last_clean":"2024-03-23T17:49:28.785515+0000","last_became_active":"2024-03-23T17:48:55.722879+0000","last_became_peered":"2024-03-23T17:48:55.722879+0000","last_unstale":"2024-03-23T17:49:28.785515+0000","last_undegraded":"2024-03-23T17:49:28.785515+0000","last_fullsized":"2024-03-23T17:49:28.785515+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:34:47.173411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.779841+0000","last_change":"2024-03-23T17:48:56.237042+0000","last_active":"2024-03-23T17:49:28.779841+0000","last_peered":"2024-03-23T17:49:28.779841+0000","last_clean":"2024-03-23T17:49:28.779841+0000","last_became_active":"2024-03-23T17:48:56.236722+0000","last_became_peered":"2024-03-23T17:48:56.236722+0000","last_unstale":"2024-03-23T17:49:28.779841+0000","last_undegraded":"2024-03-23T17:49:28.779841+0000","last_fullsized":"2024-03-23T17:49:28.779841+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:27:29.656120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.780900+0000","last_change":"2024-03-23T17:48:55.719223+0000","last_active":"2024-03-23T17:49:28.780900+0000","last_peered":"2024-03-23T17:49:28.780900+0000","last_clean":"2024-03-23T17:49:28.780900+0000","last_became_active":"2024-03-23T17:48:55.718864+0000","last_became_peered":"2024-03-23T17:48:55.718864+0000","last_unstale":"2024-03-23T17:49:28.780900+0000","last_undegraded":"2024-03-23T17:49:28.780900+0000","last_fullsized":"2024-03-23T17:49:28.780900+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:52:18.169557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.781086+0000","last_change":"2024-03-23T17:48:56.224885+0000","last_active":"2024-03-23T17:49:28.781086+0000","last_peered":"2024-03-23T17:49:28.781086+0000","last_clean":"2024-03-23T17:49:28.781086+0000","last_became_active":"2024-03-23T17:48:56.224160+0000","last_became_peered":"2024-03-23T17:48:56.224160+0000","last_unstale":"2024-03-23T17:49:28.781086+0000","last_undegraded":"2024-03-23T17:49:28.781086+0000","last_fullsized":"2024-03-23T17:49:28.781086+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:35:34.538313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'5","reported_seq":98,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-23T17:50:18.811876+0000","last_change":"2024-03-23T17:48:55.720518+0000","last_active":"2024-03-23T17:50:18.811876+0000","last_peered":"2024-03-23T17:50:18.811876+0000","last_clean":"2024-03-23T17:50:18.811876+0000","last_became_active":"2024-03-23T17:48:55.720045+0000","last_became_peered":"2024-03-23T17:48:55.720045+0000","last_unstale":"2024-03-23T17:50:18.811876+0000","last_undegraded":"2024-03-23T17:50:18.811876+0000","last_fullsized":"2024-03-23T17:50:18.811876+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:20:54.819406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":79,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.781093+0000","last_change":"2024-03-23T17:48:56.224171+0000","last_active":"2024-03-23T17:49:28.781093+0000","last_peered":"2024-03-23T17:49:28.781093+0000","last_clean":"2024-03-23T17:49:28.781093+0000","last_became_active":"2024-03-23T17:48:56.223737+0000","last_became_peered":"2024-03-23T17:48:56.223737+0000","last_unstale":"2024-03-23T17:49:28.781093+0000","last_undegraded":"2024-03-23T17:49:28.781093+0000","last_fullsized":"2024-03-23T17:49:28.781093+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:40:04.353945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'3","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.785531+0000","last_change":"2024-03-23T17:48:55.721734+0000","last_active":"2024-03-23T17:49:28.785531+0000","last_peered":"2024-03-23T17:49:28.785531+0000","last_clean":"2024-03-23T17:49:28.785531+0000","last_became_active":"2024-03-23T17:48:55.721417+0000","last_became_peered":"2024-03-23T17:48:55.721417+0000","last_unstale":"2024-03-23T17:49:28.785531+0000","last_undegraded":"2024-03-23T17:49:28.785531+0000","last_fullsized":"2024-03-23T17:49:28.785531+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:36:36.842740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'1","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.780817+0000","last_change":"2024-03-23T17:48:56.227211+0000","last_active":"2024-03-23T17:49:28.780817+0000","last_peered":"2024-03-23T17:49:28.780817+0000","last_clean":"2024-03-23T17:49:28.780817+0000","last_became_active":"2024-03-23T17:48:56.226881+0000","last_became_peered":"2024-03-23T17:48:56.226881+0000","last_unstale":"2024-03-23T17:49:28.780817+0000","last_undegraded":"2024-03-23T17:49:28.780817+0000","last_fullsized":"2024-03-23T17:49:28.780817+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:15:02.937666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.776701+0000","last_change":"2024-03-23T17:48:55.728679+0000","last_active":"2024-03-23T17:49:28.776701+0000","last_peered":"2024-03-23T17:49:28.776701+0000","last_clean":"2024-03-23T17:49:28.776701+0000","last_became_active":"2024-03-23T17:48:55.727754+0000","last_became_peered":"2024-03-23T17:48:55.727754+0000","last_unstale":"2024-03-23T17:49:28.776701+0000","last_undegraded":"2024-03-23T17:49:28.776701+0000","last_fullsized":"2024-03-23T17:49:28.776701+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:34:39.720488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.781328+0000","last_change":"2024-03-23T17:48:56.225214+0000","last_active":"2024-03-23T17:49:28.781328+0000","last_peered":"2024-03-23T17:49:28.781328+0000","last_clean":"2024-03-23T17:49:28.781328+0000","last_became_active":"2024-03-23T17:48:56.224635+0000","last_became_peered":"2024-03-23T17:48:56.224635+0000","last_unstale":"2024-03-23T17:49:28.781328+0000","last_undegraded":"2024-03-23T17:49:28.781328+0000","last_fullsized":"2024-03-23T17:49:28.781328+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:57:14.935434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.782109+0000","last_change":"2024-03-23T17:48:56.229835+0000","last_active":"2024-03-23T17:49:28.782109+0000","last_peered":"2024-03-23T17:49:28.782109+0000","last_clean":"2024-03-23T17:49:28.782109+0000","last_became_active":"2024-03-23T17:48:56.229437+0000","last_became_peered":"2024-03-23T17:48:56.229437+0000","last_unstale":"2024-03-23T17:49:28.782109+0000","last_undegraded":"2024-03-23T17:49:28.782109+0000","last_fullsized":"2024-03-23T17:49:28.782109+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:43:51.544867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.784779+0000","last_change":"2024-03-23T17:48:56.233552+0000","last_active":"2024-03-23T17:49:28.784779+0000","last_peered":"2024-03-23T17:49:28.784779+0000","last_clean":"2024-03-23T17:49:28.784779+0000","last_became_active":"2024-03-23T17:48:56.232941+0000","last_became_peered":"2024-03-23T17:48:56.232941+0000","last_unstale":"2024-03-23T17:49:28.784779+0000","last_undegraded":"2024-03-23T17:49:28.784779+0000","last_fullsized":"2024-03-23T17:49:28.784779+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:03:32.325496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.785402+0000","last_change":"2024-03-23T17:48:56.238238+0000","last_active":"2024-03-23T17:49:28.785402+0000","last_peered":"2024-03-23T17:49:28.785402+0000","last_clean":"2024-03-23T17:49:28.785402+0000","last_became_active":"2024-03-23T17:48:56.237882+0000","last_became_peered":"2024-03-23T17:48:56.237882+0000","last_unstale":"2024-03-23T17:49:28.785402+0000","last_undegraded":"2024-03-23T17:49:28.785402+0000","last_fullsized":"2024-03-23T17:49:28.785402+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:27:13.896205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.783881+0000","last_change":"2024-03-23T17:48:56.242895+0000","last_active":"2024-03-23T17:49:28.783881+0000","last_peered":"2024-03-23T17:49:28.783881+0000","last_clean":"2024-03-23T17:49:28.783881+0000","last_became_active":"2024-03-23T17:48:56.242515+0000","last_became_peered":"2024-03-23T17:48:56.242515+0000","last_unstale":"2024-03-23T17:49:28.783881+0000","last_undegraded":"2024-03-23T17:49:28.783881+0000","last_fullsized":"2024-03-23T17:49:28.783881+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:45:01.978843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.776603+0000","last_change":"2024-03-23T17:48:56.237029+0000","last_active":"2024-03-23T17:49:28.776603+0000","last_peered":"2024-03-23T17:49:28.776603+0000","last_clean":"2024-03-23T17:49:28.776603+0000","last_became_active":"2024-03-23T17:48:56.236692+0000","last_became_peered":"2024-03-23T17:48:56.236692+0000","last_unstale":"2024-03-23T17:49:28.776603+0000","last_undegraded":"2024-03-23T17:49:28.776603+0000","last_fullsized":"2024-03-23T17:49:28.776603+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:42:31.855469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.782393+0000","last_change":"2024-03-23T17:48:56.223236+0000","last_active":"2024-03-23T17:49:28.782393+0000","last_peered":"2024-03-23T17:49:28.782393+0000","last_clean":"2024-03-23T17:49:28.782393+0000","last_became_active":"2024-03-23T17:48:56.222824+0000","last_became_peered":"2024-03-23T17:48:56.222824+0000","last_unstale":"2024-03-23T17:49:28.782393+0000","last_undegraded":"2024-03-23T17:49:28.782393+0000","last_fullsized":"2024-03-23T17:49:28.782393+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:54:45.096969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.785731+0000","last_change":"2024-03-23T17:48:56.224471+0000","last_active":"2024-03-23T17:49:28.785731+0000","last_peered":"2024-03-23T17:49:28.785731+0000","last_clean":"2024-03-23T17:49:28.785731+0000","last_became_active":"2024-03-23T17:48:56.223140+0000","last_became_peered":"2024-03-23T17:48:56.223140+0000","last_unstale":"2024-03-23T17:49:28.785731+0000","last_undegraded":"2024-03-23T17:49:28.785731+0000","last_fullsized":"2024-03-23T17:49:28.785731+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:38:53.906820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.780461+0000","last_change":"2024-03-23T17:48:56.223564+0000","last_active":"2024-03-23T17:49:28.780461+0000","last_peered":"2024-03-23T17:49:28.780461+0000","last_clean":"2024-03-23T17:49:28.780461+0000","last_became_active":"2024-03-23T17:48:56.223217+0000","last_became_peered":"2024-03-23T17:48:56.223217+0000","last_unstale":"2024-03-23T17:49:28.780461+0000","last_undegraded":"2024-03-23T17:49:28.780461+0000","last_fullsized":"2024-03-23T17:49:28.780461+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:43:53.250082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.785081+0000","last_change":"2024-03-23T17:48:56.226283+0000","last_active":"2024-03-23T17:49:28.785081+0000","last_peered":"2024-03-23T17:49:28.785081+0000","last_clean":"2024-03-23T17:49:28.785081+0000","last_became_active":"2024-03-23T17:48:56.224211+0000","last_became_peered":"2024-03-23T17:48:56.224211+0000","last_unstale":"2024-03-23T17:49:28.785081+0000","last_undegraded":"2024-03-23T17:49:28.785081+0000","last_fullsized":"2024-03-23T17:49:28.785081+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:15:20.518950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.779571+0000","last_change":"2024-03-23T17:48:56.223557+0000","last_active":"2024-03-23T17:49:28.779571+0000","last_peered":"2024-03-23T17:49:28.779571+0000","last_clean":"2024-03-23T17:49:28.779571+0000","last_became_active":"2024-03-23T17:48:56.222792+0000","last_became_peered":"2024-03-23T17:48:56.222792+0000","last_unstale":"2024-03-23T17:49:28.779571+0000","last_undegraded":"2024-03-23T17:49:28.779571+0000","last_fullsized":"2024-03-23T17:49:28.779571+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:49:17.587817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.922230+0000","last_change":"2024-03-23T17:48:56.236041+0000","last_active":"2024-03-23T17:49:28.922230+0000","last_peered":"2024-03-23T17:49:28.922230+0000","last_clean":"2024-03-23T17:49:28.922230+0000","last_became_active":"2024-03-23T17:48:56.234259+0000","last_became_peered":"2024-03-23T17:48:56.234259+0000","last_unstale":"2024-03-23T17:49:28.922230+0000","last_undegraded":"2024-03-23T17:49:28.922230+0000","last_fullsized":"2024-03-23T17:49:28.922230+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:32:43.786783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.777528+0000","last_change":"2024-03-23T17:48:56.239283+0000","last_active":"2024-03-23T17:49:28.777528+0000","last_peered":"2024-03-23T17:49:28.777528+0000","last_clean":"2024-03-23T17:49:28.777528+0000","last_became_active":"2024-03-23T17:48:56.238952+0000","last_became_peered":"2024-03-23T17:48:56.238952+0000","last_unstale":"2024-03-23T17:49:28.777528+0000","last_undegraded":"2024-03-23T17:49:28.777528+0000","last_fullsized":"2024-03-23T17:49:28.777528+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:31:50.383284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.782143+0000","last_change":"2024-03-23T17:48:56.226684+0000","last_active":"2024-03-23T17:49:28.782143+0000","last_peered":"2024-03-23T17:49:28.782143+0000","last_clean":"2024-03-23T17:49:28.782143+0000","last_became_active":"2024-03-23T17:48:56.226367+0000","last_became_peered":"2024-03-23T17:48:56.226367+0000","last_unstale":"2024-03-23T17:49:28.782143+0000","last_undegraded":"2024-03-23T17:49:28.782143+0000","last_fullsized":"2024-03-23T17:49:28.782143+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:12:29.053330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70574,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":819,"num_read_kb":481,"num_write":235,"num_write_kb":138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":180,"num_keys_recovered":17,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":211722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":206,"ondisk_log_size":206,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547229,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31992,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713160,"statfs":{"total":95995035648,"available":95962275840,"internally_reserved":0,"allocated":897024,"data_stored":428738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31848901},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 17:50:19 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.53200000000000003,"15min":0.53900000000000003},"min":{"1min":0.38700000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.64300000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.60299999999999998,"15min":0.60899999999999999},"min":{"1min":0.435,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.73599999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.48399999999999999}]},{"osd":1,"last update":"Sat Mar 23 17:50:19 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.51600000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.749,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.63700000000000001,"15min":0.63600000000000001},"min":{"1min":0.503,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.85399999999999998,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.74099999999999999}]},{"osd":2,"last update":"Sat Mar 23 17:50:19 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.503,"15min":0.501},"min":{"1min":0.35399999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.47899999999999998,"15min":0.47299999999999998},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.51400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":4,"last update":"Sat Mar 23 17:50:19 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.64300000000000002,"15min":0.64500000000000002},"min":{"1min":0.44800000000000001,"5min":0.438,"15min":0.438},"max":{"1min":0.75700000000000001,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.66300000000000003,"15min":0.66100000000000003},"min":{"1min":0.505,"5min":0.441,"15min":0.441},"max":{"1min":0.83599999999999997,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.79000000000000004}]},{"osd":5,"last update":"Sat Mar 23 17:50:19 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.59599999999999997,"15min":0.58899999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.621,"5min":0.58899999999999997,"15min":0.58399999999999996},"min":{"1min":0.436,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.81299999999999994,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.61799999999999999}]},{"osd":6,"last update":"Sat Mar 23 17:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.84499999999999997}]},{"osd":7,"last update":"Sat Mar 23 17:50:19 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.71999999999999997,"15min":0.71899999999999997},"min":{"1min":0.54800000000000004,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.86399999999999999,"5min":1.002,"15min":1.002},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.753,"15min":0.75600000000000001},"min":{"1min":0.59299999999999997,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.89500000000000002,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.69299999999999995}]},{"osd":8,"last update":"Sat Mar 23 17:49:56 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.82999999999999996}]},{"osd":9,"last update":"Sat Mar 23 17:50:19 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.63200000000000001,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.93899999999999995,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.76300000000000001,"15min":0.76600000000000001},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.90200000000000002,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.80000000000000004}]},{"osd":10,"last update":"Sat Mar 23 17:50:19 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.78500000000000003,"15min":0.78600000000000003},"min":{"1min":0.66100000000000003,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.95299999999999996,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.79500000000000004,"15min":0.79400000000000004},"min":{"1min":0.69299999999999995,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.97299999999999998,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.86799999999999999}]}]},{"osd":10,"up_from":68,"seq":292057776159,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28568,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716584,"statfs":{"total":95995035648,"available":95965782016,"internally_reserved":0,"allocated":1585152,"data_stored":1078786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 17:49:56 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.438}]},{"osd":1,"last update":"Sat Mar 23 17:50:02 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.47099999999999997,"15min":0.47899999999999998},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.59599999999999997,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.44,"15min":0.44500000000000001},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.57199999999999995,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.41599999999999998}]},{"osd":2,"last update":"Sat Mar 23 17:50:02 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.58899999999999997,"15min":0.59399999999999997},"min":{"1min":0.41299999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.746,"5min":0.755,"15min":0.755},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.629,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.80700000000000005,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.623}]},{"osd":3,"last update":"Sat Mar 23 17:50:02 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.60899999999999999,"15min":0.61699999999999999},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.70999999999999996,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.626,"15min":0.63600000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.79000000000000004,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.58499999999999996}]},{"osd":4,"last update":"Sat Mar 23 17:50:02 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.56499999999999995,"15min":0.56899999999999995},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.72399999999999998,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.53900000000000003,"15min":0.55000000000000004},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.64900000000000002,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.55000000000000004}]},{"osd":5,"last update":"Sat Mar 23 17:50:02 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.64100000000000001,"15min":0.65100000000000002},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.77900000000000003,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.69899999999999995,"15min":0.70699999999999996},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.80500000000000005,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.72599999999999998}]},{"osd":6,"last update":"Sat Mar 23 17:50:02 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.68999999999999995,"15min":0.69599999999999995},"min":{"1min":0.47099999999999997,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.85099999999999998,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.71399999999999997,"15min":0.72599999999999998},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.82099999999999995,"5min":1.071,"15min":1.071},"last":0.70499999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":8,"last update":"Sat Mar 23 17:50:02 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.73899999999999999,"15min":0.748},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.86399999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.748,"15min":0.76000000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.82899999999999996,"5min":0.996,"15min":0.996},"last":0.67200000000000004}]},{"osd":9,"last update":"Sat Mar 23 17:50:02 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.76000000000000001,"15min":0.77000000000000002},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.92600000000000005,"5min":1.194,"15min":1.194},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.755,"15min":0.76900000000000002},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.879,"5min":1.042,"15min":1.042},"last":0.71499999999999997}]},{"osd":11,"last update":"Sat Mar 23 17:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.77500000000000002}]}]},{"osd":9,"up_from":63,"seq":270582939685,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36108,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709044,"statfs":{"total":95995035648,"available":95958061056,"internally_reserved":0,"allocated":913408,"data_stored":428648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":36043201},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 17:49:44 2024","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.54200000000000004,"15min":0.56200000000000006},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.64400000000000002,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.58499999999999996,"15min":0.60699999999999998},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.59999999999999998,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.51900000000000002}]},{"osd":1,"last update":"Sat Mar 23 17:49:44 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.78500000000000003,"15min":0.80500000000000005},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.70699999999999996,"15min":0.72499999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.78600000000000003,"5min":1.091,"15min":1.091},"last":0.45500000000000002}]},{"osd":2,"last update":"Sat Mar 23 17:49:44 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.78600000000000003,"15min":0.80100000000000005},"min":{"1min":0.47299999999999998,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.871,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.73599999999999999,"15min":0.752},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.97399999999999998,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.77700000000000002}]},{"osd":3,"last update":"Sat Mar 23 17:49:44 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.61499999999999999,"15min":0.63300000000000001},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.77900000000000003,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.71899999999999997,"15min":0.73699999999999999},"min":{"1min":0.44900000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.88900000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.79900000000000004}]},{"osd":4,"last update":"Sat Mar 23 17:48:42 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.61099999999999999}]},{"osd":5,"last update":"Sat Mar 23 17:49:44 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.73199999999999998,"15min":0.74199999999999999},"min":{"1min":0.54700000000000004,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.1180000000000001,"5min":1.167,"15min":1.167},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.85099999999999998,"15min":0.87},"min":{"1min":0.505,"5min":0.437,"15min":0.437},"max":{"1min":0.93100000000000005,"5min":1.48,"15min":1.48},"last":0.85799999999999998}]},{"osd":6,"last update":"Sat Mar 23 17:49:44 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.83099999999999996,"15min":0.85499999999999998},"min":{"1min":0.47099999999999997,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.877,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.78600000000000003,"15min":0.81200000000000006},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.85599999999999998,"5min":1.153,"15min":1.153},"last":0.69699999999999995}]},{"osd":7,"last update":"Sat Mar 23 17:49:44 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.81599999999999995,"15min":0.84399999999999997},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.78000000000000003,"5min":1.216,"15min":1.216},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.92200000000000004,"15min":0.94499999999999995},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.2969999999999999,"5min":1.373,"15min":1.373},"last":0.90400000000000003}]},{"osd":8,"last update":"Sat Mar 23 17:49:44 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.85999999999999999,"15min":0.878},"min":{"1min":0.57499999999999996,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.88200000000000001,"15min":0.90800000000000003},"min":{"1min":0.59099999999999997,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.95499999999999996,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.59699999999999998}]},{"osd":10,"last update":"Sat Mar 23 17:50:00 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.79500000000000004,"15min":0.81699999999999995},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.80300000000000005,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.89400000000000002,"15min":0.91300000000000003},"min":{"1min":0.67200000000000004,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":0.94299999999999995,"5min":1.339,"15min":1.339},"last":0.88200000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]}]},{"osd":8,"up_from":56,"seq":240518168615,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":429026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 17:50:23 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.57099999999999995,"15min":0.57899999999999996},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.79600000000000004,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55100000000000005,"15min":0.55200000000000005},"min":{"1min":0.38300000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.55600000000000005}]},{"osd":1,"last update":"Sat Mar 23 17:50:23 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.60799999999999998,"15min":0.59199999999999997},"min":{"1min":0.51800000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.60599999999999998,"15min":0.59899999999999998},"min":{"1min":0.44600000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.69399999999999995}]},{"osd":2,"last update":"Sat Mar 23 17:50:23 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.46899999999999997,"15min":0.45900000000000002},"min":{"1min":0.38300000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.71399999999999997,"5min":0.874,"15min":0.874},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.47499999999999998,"15min":0.46500000000000002},"min":{"1min":0.34399999999999997,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.747,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.65400000000000003}]},{"osd":3,"last update":"Sat Mar 23 17:50:23 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.58699999999999997,"15min":0.59099999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64800000000000002,"15min":0.66000000000000003},"min":{"1min":0.434,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.86299999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.60199999999999998}]},{"osd":4,"last update":"Sat Mar 23 17:50:23 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.61599999999999999,"15min":0.59499999999999997},"min":{"1min":0.55700000000000005,"5min":0.308,"15min":0.308},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.65400000000000003,"15min":0.64000000000000001},"min":{"1min":0.51300000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.82799999999999996}]},{"osd":5,"last update":"Sat Mar 23 17:50:23 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.55000000000000004,"15min":0.54900000000000004},"min":{"1min":0.45700000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.84599999999999997,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.56999999999999995,"15min":0.56000000000000005},"min":{"1min":0.39300000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.91800000000000004,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.61699999999999999}]},{"osd":6,"last update":"Sat Mar 23 17:50:23 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70299999999999996,"15min":0.70999999999999996},"min":{"1min":0.48199999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.94199999999999995,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.68700000000000006,"15min":0.69099999999999995},"min":{"1min":0.55300000000000005,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.89400000000000002,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.58699999999999997}]},{"osd":7,"last update":"Sat Mar 23 17:50:23 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.69299999999999995,"15min":0.68300000000000005},"min":{"1min":0.55300000000000005,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.71699999999999997,"15min":0.70399999999999996},"min":{"1min":0.60099999999999998,"5min":0.438,"15min":0.438},"max":{"1min":0.99199999999999999,"5min":1.04,"15min":1.04},"last":0.88800000000000001}]},{"osd":9,"last update":"Sat Mar 23 17:49:46 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.72899999999999998,"15min":0.73299999999999998},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.875,"5min":1.026,"15min":1.026},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.745,"5min":0.76100000000000001,"15min":0.76400000000000001},"min":{"1min":0.55500000000000005,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.94599999999999995,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.875}]},{"osd":10,"last update":"Sat Mar 23 17:50:02 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.72599999999999998,"15min":0.71599999999999997},"min":{"1min":0.61899999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.74299999999999999,"15min":0.73099999999999998},"min":{"1min":0.63600000000000001,"5min":0.496,"15min":0.496},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.93400000000000005}]},{"osd":11,"last update":"Sat Mar 23 17:49:57 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.91400000000000003}]}]},{"osd":7,"up_from":49,"seq":210453397547,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28488,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716664,"statfs":{"total":95995035648,"available":95965863936,"internally_reserved":0,"allocated":1507328,"data_stored":1021315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 17:48:02 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.54800000000000004}]},{"osd":1,"last update":"Sat Mar 23 17:50:08 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.39700000000000002},"min":{"1min":0.31,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.58499999999999996,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.434,"5min":0.40500000000000003,"15min":0.38200000000000001},"min":{"1min":0.314,"5min":0.248,"15min":0.248},"max":{"1min":0.64000000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.621}]},{"osd":2,"last update":"Sat Mar 23 17:50:08 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.57399999999999995,"15min":0.55800000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.57299999999999995,"15min":0.55900000000000005},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1,"5min":1,"15min":1},"last":0.64500000000000002}]},{"osd":3,"last update":"Sat Mar 23 17:50:08 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.55800000000000005,"15min":0.54700000000000004},"min":{"1min":0.41899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.59899999999999998,"15min":0.59099999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.77500000000000002,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.82899999999999996}]},{"osd":4,"last update":"Sat Mar 23 17:50:08 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.495,"15min":0.47899999999999998},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.72899999999999998,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.47699999999999998,"15min":0.47099999999999997},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.63700000000000001,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.69499999999999995}]},{"osd":5,"last update":"Sat Mar 23 17:50:08 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.60699999999999998,"15min":0.58999999999999997},"min":{"1min":0.46899999999999997,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.80400000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.58999999999999997,"15min":0.57799999999999996},"min":{"1min":0.41499999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.78100000000000003}]},{"osd":6,"last update":"Sat Mar 23 17:50:08 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.67300000000000004,"15min":0.66700000000000004},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.83899999999999997,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.879},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65200000000000002,"15min":0.65100000000000002},"min":{"1min":0.45000000000000001,"5min":0.433,"15min":0.433},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.80600000000000005}]},{"osd":8,"last update":"Sat Mar 23 17:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.64900000000000002,"15min":0.63300000000000001},"min":{"1min":0.47799999999999998,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.66600000000000004,"15min":0.64700000000000002},"min":{"1min":0.52700000000000002,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.92300000000000004}]},{"osd":9,"last update":"Sat Mar 23 17:49:43 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.71299999999999997,"15min":0.71699999999999997},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.88200000000000001,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.69299999999999995,"15min":0.69599999999999995},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.84499999999999997,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.84799999999999998}]},{"osd":10,"last update":"Sat Mar 23 17:50:02 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.73199999999999998,"15min":0.73999999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.92000000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.70499999999999996,"15min":0.70999999999999996},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.85999999999999999}]},{"osd":11,"last update":"Sat Mar 23 17:49:58 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.93500000000000005}]}]},{"osd":6,"up_from":45,"seq":193273528367,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36088,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93709064,"statfs":{"total":95995035648,"available":95958081536,"internally_reserved":0,"allocated":892928,"data_stored":428044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 17:49:47 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47499999999999998,"15min":0.47599999999999998},"min":{"1min":0.32200000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.51700000000000002,"15min":0.52100000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.48899999999999999}]},{"osd":1,"last update":"Sat Mar 23 17:49:47 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.59799999999999998,"15min":0.59399999999999997},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.83899999999999997,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.46999999999999997,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.95699999999999996,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.71799999999999997}]},{"osd":2,"last update":"Sat Mar 23 17:49:47 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.44900000000000001,"5min":0.374,"15min":0.374},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67200000000000004,"15min":0.67700000000000005},"min":{"1min":0.53000000000000003,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.88200000000000001,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.66400000000000003}]},{"osd":3,"last update":"Sat Mar 23 17:49:47 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.59799999999999998,"15min":0.61399999999999999},"min":{"1min":0.35399999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.878,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.65700000000000003,"15min":0.66300000000000003},"min":{"1min":0.376,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.1140000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.64600000000000002}]},{"osd":4,"last update":"Sat Mar 23 17:49:47 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.65200000000000002,"15min":0.65600000000000003},"min":{"1min":0.42999999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.71799999999999997,"15min":0.73299999999999998},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.90500000000000003,"5min":0.998,"15min":0.998},"last":0.69999999999999996}]},{"osd":5,"last update":"Sat Mar 23 17:49:47 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.77400000000000002,"15min":0.78100000000000003},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.77000000000000002,"15min":0.78200000000000003},"min":{"1min":0.49299999999999999,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.504,"5min":1.504,"15min":1.504},"last":0.69199999999999995}]},{"osd":7,"last update":"Sat Mar 23 17:50:08 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.745,"15min":0.74199999999999999},"min":{"1min":0.54900000000000004,"5min":0.5,"15min":0.5},"max":{"1min":1.448,"5min":1.448,"15min":1.448},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.749,"15min":0.77900000000000003},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.4339999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.621}]},{"osd":8,"last update":"Sat Mar 23 17:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.85599999999999998,"15min":0.876},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.92400000000000004,"5min":1.085,"15min":1.085},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.81499999999999995,"15min":0.82399999999999995},"min":{"1min":0.63200000000000001,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":0.91200000000000003,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.77600000000000002}]},{"osd":9,"last update":"Sat Mar 23 17:49:47 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.80500000000000005,"15min":0.80800000000000005},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.78400000000000003,"15min":0.79400000000000004},"min":{"1min":0.48399999999999999,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.93899999999999995,"5min":1.099,"15min":1.099},"last":0.81399999999999995}]},{"osd":10,"last update":"Sat Mar 23 17:50:02 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.80800000000000005,"15min":0.80200000000000005},"min":{"1min":0.67900000000000005,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.80300000000000005,"15min":0.80700000000000005},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.573,"5min":1.573,"15min":1.573},"last":0.83899999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":493574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 17:50:21 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53100000000000003,"15min":0.51500000000000001},"min":{"1min":0.41499999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.82299999999999995,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.63700000000000001,"15min":0.53500000000000003},"min":{"1min":0.39800000000000002,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.68200000000000005,"5min":5.8399999999999999,"15min":5.8399999999999999},"last":0.47699999999999998}]},{"osd":2,"last update":"Sat Mar 23 17:49:29 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.52700000000000002,"15min":0.48099999999999998},"min":{"1min":0.46600000000000003,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.81399999999999995,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.52200000000000002,"15min":0.49299999999999999},"min":{"1min":0.45900000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.77400000000000002,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.56899999999999995}]},{"osd":3,"last update":"Sat Mar 23 17:49:52 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55600000000000005,"15min":0.53400000000000003},"min":{"1min":0.38100000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.73799999999999999,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.55300000000000005,"15min":0.53900000000000003},"min":{"1min":0.38400000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.69199999999999995,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.503}]},{"osd":4,"last update":"Sat Mar 23 17:50:10 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.44400000000000001,"15min":0.40600000000000003},"min":{"1min":0.34300000000000003,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.59699999999999998,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.442,"15min":0.40300000000000002},"min":{"1min":0.36399999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.69799999999999995,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.39000000000000001}]},{"osd":5,"last update":"Sat Mar 23 17:49:29 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.59499999999999997,"15min":0.56299999999999994},"min":{"1min":0.55500000000000005,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.86599999999999999,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.61199999999999999,"15min":0.57799999999999996},"min":{"1min":0.55600000000000005,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.85399999999999998,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.71199999999999997}]},{"osd":6,"last update":"Sat Mar 23 17:49:45 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.59999999999999998,"15min":0.57299999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.80400000000000005,"5min":0.998,"15min":0.998},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.63600000000000001,"15min":0.621},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.78800000000000003,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.64100000000000001}]},{"osd":7,"last update":"Sat Mar 23 17:50:06 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53200000000000003,"15min":0.51300000000000001},"min":{"1min":0.38200000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.66000000000000003,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.372},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.57299999999999995,"15min":0.54100000000000004},"min":{"1min":0.39400000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.73399999999999999,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.65900000000000003}]},{"osd":8,"last update":"Sat Mar 23 17:50:21 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.69499999999999995,"15min":0.67300000000000004},"min":{"1min":0.57499999999999996,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.88,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.64800000000000002,"15min":0.63400000000000001},"min":{"1min":0.505,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.80600000000000005,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.59799999999999998}]},{"osd":9,"last update":"Sat Mar 23 17:49:45 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.75600000000000001,"15min":0.76200000000000001},"min":{"1min":0.48699999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.85399999999999998,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.63},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.78900000000000003,"15min":0.79900000000000004},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.90300000000000002,"5min":1.103,"15min":1.103},"last":0.76200000000000001}]},{"osd":10,"last update":"Sat Mar 23 17:50:00 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.70699999999999996,"15min":0.71399999999999997},"min":{"1min":0.56599999999999995,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.84199999999999997,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.76300000000000001,"15min":0.76800000000000002},"min":{"1min":0.46100000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.89100000000000001,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.747}]},{"osd":11,"last update":"Sat Mar 23 17:50:00 2024","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.78600000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738438,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":913408,"data_stored":429383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 23 17:50:17 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.61399999999999999,"15min":0.71399999999999997},"min":{"1min":0.38400000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.89800000000000002,"5min":6.2110000000000003,"15min":6.2110000000000003},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.628,"5min":0.56999999999999995,"15min":0.51600000000000001},"min":{"1min":0.41399999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.58099999999999996}]},{"osd":2,"last update":"Sat Mar 23 17:49:31 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.57799999999999996,"15min":0.55100000000000005},"min":{"1min":0.38100000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.82999999999999996,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.59999999999999998,"15min":0.56000000000000005},"min":{"1min":0.41299999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.751}]},{"osd":3,"last update":"Sat Mar 23 17:49:58 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.45800000000000002}]},{"osd":4,"last update":"Sat Mar 23 17:50:17 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.64700000000000002,"15min":0.625},"min":{"1min":0.45500000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.621},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.61099999999999999,"15min":0.59199999999999997},"min":{"1min":0.46100000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.873,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.79800000000000004}]},{"osd":5,"last update":"Sat Mar 23 17:49:28 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65700000000000003,"15min":0.63700000000000001},"min":{"1min":0.44700000000000001,"5min":0.44,"15min":0.44},"max":{"1min":1.018,"5min":1.167,"15min":1.167},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.64500000000000002,"15min":0.61599999999999999},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.65500000000000003}]},{"osd":6,"last update":"Sat Mar 23 17:49:48 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60099999999999998},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.68000000000000005,"15min":0.66800000000000004},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.0509999999999999,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.63800000000000001}]},{"osd":7,"last update":"Sat Mar 23 17:48:02 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.495},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.58599999999999997}]},{"osd":8,"last update":"Sat Mar 23 17:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.83299999999999996,"15min":0.84899999999999998},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.091,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.81299999999999994,"15min":0.82599999999999996},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.68100000000000005}]},{"osd":9,"last update":"Sat Mar 23 17:49:43 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.76900000000000002,"15min":0.77300000000000002},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.82899999999999996,"15min":0.83699999999999997},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.119,"5min":1.286,"15min":1.286},"last":0.78700000000000003}]},{"osd":10,"last update":"Sat Mar 23 17:50:04 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.81499999999999995,"15min":0.81100000000000005},"min":{"1min":0.59599999999999997,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.421,"5min":1.421,"15min":1.421},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.90000000000000002}]},{"osd":11,"last update":"Sat Mar 23 17:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.93400000000000005}]}]},{"osd":2,"up_from":18,"seq":77309411390,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36148,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709004,"statfs":{"total":95995035648,"available":95958020096,"internally_reserved":0,"allocated":962560,"data_stored":492987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 17:49:30 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.54000000000000004,"15min":0.51400000000000001},"min":{"1min":0.42199999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.5},{"interface":"front","average":{"1min":0.63,"5min":0.57899999999999996,"15min":0.55300000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.88700000000000001,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.35699999999999998}]},{"osd":1,"last update":"Sat Mar 23 17:49:30 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.54500000000000004,"15min":0.5},"min":{"1min":0.42199999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.80900000000000005,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.55800000000000005,"15min":0.51800000000000002},"min":{"1min":0.41999999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.79300000000000004,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.68600000000000005}]},{"osd":3,"last update":"Sat Mar 23 17:49:56 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.55900000000000005,"15min":0.53300000000000003},"min":{"1min":0.38200000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.438},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58899999999999997,"15min":0.56699999999999995},"min":{"1min":0.40300000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.82199999999999995,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.57299999999999995}]},{"osd":4,"last update":"Sat Mar 23 17:49:56 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.63300000000000001}]},{"osd":5,"last update":"Sat Mar 23 17:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.48299999999999998,"15min":0.45900000000000002},"min":{"1min":0.32600000000000001,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.495,"15min":0.46999999999999997},"min":{"1min":0.34100000000000003,"5min":0.308,"15min":0.308},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.61199999999999999}]},{"osd":6,"last update":"Sat Mar 23 17:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.63700000000000001,"15min":0.622},"min":{"1min":0.505,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.80800000000000005,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.65700000000000003,"15min":0.64200000000000002},"min":{"1min":0.51100000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.85099999999999998,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.52100000000000002}]},{"osd":7,"last update":"Sat Mar 23 17:48:01 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.745}]},{"osd":8,"last update":"Sat Mar 23 17:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.51000000000000001,"15min":0.50600000000000001},"min":{"1min":0.35599999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.59399999999999997,"15min":0.58599999999999997},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.78100000000000003,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.56000000000000005}]},{"osd":9,"last update":"Sat Mar 23 17:49:45 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.70399999999999996,"15min":0.69999999999999996},"min":{"1min":0.49399999999999999,"5min":0.433,"15min":0.433},"max":{"1min":0.84099999999999997,"5min":0.879,"15min":0.879},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.77300000000000002,"15min":0.77400000000000002},"min":{"1min":0.68400000000000005,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.94899999999999995,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.73599999999999999}]},{"osd":10,"last update":"Sat Mar 23 17:50:06 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68200000000000005,"15min":0.68000000000000005},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.79300000000000004,"15min":0.79100000000000004},"min":{"1min":0.68000000000000005,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.80800000000000005}]},{"osd":11,"last update":"Sat Mar 23 17:49:56 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.65900000000000003}]}]},{"osd":3,"up_from":26,"seq":111669149754,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32008,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713144,"statfs":{"total":95995035648,"available":95962259456,"internally_reserved":0,"allocated":909312,"data_stored":427917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 17:50:02 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.44,"15min":0.40200000000000002},"min":{"1min":0.32000000000000001,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.44500000000000001,"15min":0.40699999999999997},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.85899999999999999,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.39600000000000002}]},{"osd":1,"last update":"Sat Mar 23 17:50:02 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.58999999999999997,"15min":0.53100000000000003},"min":{"1min":0.38700000000000001,"5min":0.371,"15min":0.371},"max":{"1min":2.0649999999999999,"5min":2.0649999999999999,"15min":2.0649999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.57599999999999996,"15min":0.51900000000000002},"min":{"1min":0.48299999999999998,"5min":0.371,"15min":0.371},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.751}]},{"osd":2,"last update":"Sat Mar 23 17:50:02 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.629,"15min":0.58899999999999997},"min":{"1min":0.434,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.96699999999999997,"5min":1.02,"15min":1.02},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.64900000000000002,"15min":0.61899999999999999},"min":{"1min":0.47299999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":2.141,"5min":2.141,"15min":2.141},"last":0.61299999999999999}]},{"osd":4,"last update":"Sat Mar 23 17:50:12 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.67700000000000005,"15min":0.624},"min":{"1min":0.55300000000000005,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":2.2080000000000002,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.65900000000000003,"15min":0.621},"min":{"1min":0.45200000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.73099999999999998}]},{"osd":5,"last update":"Sat Mar 23 17:49:31 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.67800000000000005,"15min":0.65000000000000002},"min":{"1min":0.53500000000000003,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.01,"5min":1.036,"15min":1.036},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.70099999999999996,"15min":0.67300000000000004},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.56599999999999995}]},{"osd":6,"last update":"Sat Mar 23 17:49:45 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.66700000000000004,"15min":0.65100000000000002},"min":{"1min":0.41099999999999998,"5min":0.38,"15min":0.38},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.68000000000000005,"15min":0.66800000000000004},"min":{"1min":0.46300000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.64500000000000002}]},{"osd":7,"last update":"Sat Mar 23 17:50:08 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.75600000000000001,"15min":0.76500000000000001},"min":{"1min":0.54500000000000004,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.747,"15min":0.71999999999999997},"min":{"1min":0.55200000000000005,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":2.1720000000000002,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":0.89900000000000002}]},{"osd":8,"last update":"Sat Mar 23 17:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.88},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.85099999999999998}]},{"osd":9,"last update":"Sat Mar 23 17:49:45 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.78500000000000003,"15min":0.77300000000000002},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":2.2370000000000001,"5min":2.2370000000000001,"15min":2.2370000000000001},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.78400000000000003,"15min":0.78000000000000003},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.625}]},{"osd":10,"last update":"Sat Mar 23 17:50:03 2024","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.79600000000000004,"15min":0.78700000000000003},"min":{"1min":0.65400000000000003,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.46,"5min":1.46,"15min":1.46},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.85499999999999998,"15min":0.84099999999999997},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":2.6349999999999998,"5min":2.6349999999999998,"15min":2.6349999999999998},"last":0.82999999999999996}]},{"osd":11,"last update":"Sat Mar 23 17:49:57 2024","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":2.6819999999999999,"5min":2.6819999999999999,"15min":2.6819999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.51,"5min":1.51,"15min":1.51},"last":0.68100000000000005}]}]},{"osd":4,"up_from":33,"seq":141733920823,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":901120,"data_stored":428017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 17:48:06 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.53700000000000003,"15min":0.53400000000000003},"min":{"1min":0.41799999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.77600000000000002,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.53700000000000003,"15min":0.52500000000000002},"min":{"1min":0.45300000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.52000000000000002}]},{"osd":1,"last update":"Sat Mar 23 17:50:07 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.36799999999999999,"15min":0.33100000000000002},"min":{"1min":0.27500000000000002,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.62,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.38400000000000001,"15min":0.34899999999999998},"min":{"1min":0.28999999999999998,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.57399999999999995,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.52000000000000002}]},{"osd":2,"last update":"Sat Mar 23 17:50:07 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.57999999999999996,"15min":0.55900000000000005},"min":{"1min":0.379,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.83499999999999996,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.58599999999999997,"15min":0.55200000000000005},"min":{"1min":0.441,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.81100000000000005,"5min":0.88,"15min":0.88},"last":0.78200000000000003}]},{"osd":3,"last update":"Sat Mar 23 17:50:07 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57599999999999996,"15min":0.56100000000000005},"min":{"1min":0.46600000000000003,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.73999999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.56499999999999995,"15min":0.56200000000000006},"min":{"1min":0.39400000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.754,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.504}]},{"osd":5,"last update":"Sat Mar 23 17:49:29 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.57999999999999996,"15min":0.56599999999999995},"min":{"1min":0.41299999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.80600000000000005,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.629,"5min":0.59499999999999997,"15min":0.57799999999999996},"min":{"1min":0.44600000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.77700000000000002,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.82799999999999996}]},{"osd":6,"last update":"Sat Mar 23 17:49:53 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.73699999999999999,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59199999999999997,"15min":0.58999999999999997},"min":{"1min":0.41799999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.79700000000000004,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.54900000000000004}]},{"osd":7,"last update":"Sat Mar 23 17:50:03 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.55300000000000005,"15min":0.54300000000000004},"min":{"1min":0.39300000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.55300000000000005,"15min":0.53900000000000003},"min":{"1min":0.36599999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.80900000000000005}]},{"osd":8,"last update":"Sat Mar 23 17:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.65800000000000003,"15min":0.65400000000000003},"min":{"1min":0.501,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.75},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.71699999999999997,"15min":0.71799999999999997},"min":{"1min":0.48599999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.86599999999999999,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.90700000000000003}]},{"osd":9,"last update":"Sat Mar 23 17:49:43 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.71799999999999997,"15min":0.71599999999999997},"min":{"1min":0.60599999999999998,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.54500000000000004,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.65200000000000002}]},{"osd":10,"last update":"Sat Mar 23 17:50:03 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.69899999999999995,"15min":0.69499999999999995},"min":{"1min":0.52600000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.86599999999999999,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.87},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.68700000000000006,"15min":0.67900000000000005},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.89300000000000002}]},{"osd":11,"last update":"Sat Mar 23 17:49:59 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.92900000000000005}]}]},{"osd":5,"up_from":38,"seq":163208757299,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32600,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712552,"statfs":{"total":95995035648,"available":95961653248,"internally_reserved":0,"allocated":1515520,"data_stored":1020452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 17:49:28 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.54900000000000004,"15min":0.54100000000000004},"min":{"1min":0.435,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.54400000000000004,"15min":0.54800000000000004},"min":{"1min":0.36299999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.83499999999999996,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.435}]},{"osd":1,"last update":"Sat Mar 23 17:49:28 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.55100000000000005,"15min":0.52700000000000002},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.58799999999999997,"15min":0.56499999999999995},"min":{"1min":0.495,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.67200000000000004}]},{"osd":2,"last update":"Sat Mar 23 17:49:28 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.40400000000000003,"15min":0.39400000000000002},"min":{"1min":0.311,"5min":0.251,"15min":0.251},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.503,"5min":0.41799999999999998,"15min":0.40100000000000002},"min":{"1min":0.30399999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.374}]},{"osd":3,"last update":"Sat Mar 23 17:49:28 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58299999999999996,"15min":0.58499999999999996},"min":{"1min":0.38100000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.80100000000000005,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.57699999999999996,"15min":0.58499999999999996},"min":{"1min":0.40899999999999997,"5min":0.373,"15min":0.373},"max":{"1min":0.71899999999999997,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.41999999999999998}]},{"osd":4,"last update":"Sat Mar 23 17:49:28 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.60599999999999998,"15min":0.58199999999999996},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.89500000000000002,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.58199999999999996,"15min":0.56000000000000005},"min":{"1min":0.44400000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.877,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.64700000000000002}]},{"osd":6,"last update":"Sat Mar 23 17:49:50 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.64000000000000001,"15min":0.629},"min":{"1min":0.55100000000000005,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.61499999999999999,"15min":0.61199999999999999},"min":{"1min":0.436,"5min":0.434,"15min":0.434},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.47399999999999998}]},{"osd":7,"last update":"Sat Mar 23 17:50:06 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.67800000000000005,"15min":0.65200000000000002},"min":{"1min":0.623,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.66100000000000003,"15min":0.63900000000000001},"min":{"1min":0.58399999999999996,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.66100000000000003}]},{"osd":8,"last update":"Sat Mar 23 17:50:21 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.624,"15min":0.59799999999999998},"min":{"1min":0.45000000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.57699999999999996,"15min":0.56000000000000005},"min":{"1min":0.38800000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.59999999999999998}]},{"osd":9,"last update":"Sat Mar 23 17:49:44 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.91600000000000004,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.70599999999999996,"15min":0.70099999999999996},"min":{"1min":0.53700000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.61499999999999999}]},{"osd":10,"last update":"Sat Mar 23 17:50:01 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.748,"15min":0.73399999999999999},"min":{"1min":0.71399999999999997,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.71799999999999997,"15min":0.70699999999999996},"min":{"1min":0.65000000000000002,"5min":0.503,"15min":0.503},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.76900000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":66048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":65734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":65726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T17:50:25.761 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-03-23T17:50:25.761 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T17:50:25.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.988+0000 7fd9cca00700 1 -- 172.21.15.86:0/356086863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9c8100c80 msgr2=0x7fd9c81010a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:25.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.988+0000 7fd9cca00700 1 --2- 172.21.15.86:0/356086863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9c8100c80 0x7fd9c81010a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd9b80099c0 tx=0x7fd9b8009cd0 comp rx=0 tx=0).stop 2024-03-23T17:50:25.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.988+0000 7fd9cca00700 1 -- 172.21.15.86:0/356086863 shutdown_connections 2024-03-23T17:50:25.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.988+0000 7fd9cca00700 1 --2- 172.21.15.86:0/356086863 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd9c8102810 0x7fd9c81090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:25.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.988+0000 7fd9cca00700 1 --2- 172.21.15.86:0/356086863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9c8100c80 0x7fd9c81010a0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:25.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.988+0000 7fd9cca00700 1 --2- 172.21.15.86:0/356086863 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9c8101e70 0x7fd9c81022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:25.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.988+0000 7fd9cca00700 1 -- 172.21.15.86:0/356086863 >> 172.21.15.86:0/356086863 conn(0x7fd9c80fc1f0 msgr2=0x7fd9c80fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:25.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.988+0000 7fd9cca00700 1 -- 172.21.15.86:0/356086863 shutdown_connections 2024-03-23T17:50:25.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.988+0000 7fd9cca00700 1 -- 172.21.15.86:0/356086863 wait complete. 2024-03-23T17:50:25.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.989+0000 7fd9cca00700 1 Processor -- start 2024-03-23T17:50:25.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.989+0000 7fd9cca00700 1 -- start start 2024-03-23T17:50:25.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.989+0000 7fd9cca00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd9c8100c80 0x7fd9c8196c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:25.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.989+0000 7fd9cca00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9c8101e70 0x7fd9c8197160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:25.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.989+0000 7fd9cca00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9c8102810 0x7fd9c819e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:25.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.989+0000 7fd9cca00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c819e750 con 0x7fd9c8102810 2024-03-23T17:50:25.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.989+0000 7fd9cca00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c819e890 con 0x7fd9c8101e70 2024-03-23T17:50:25.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.989+0000 7fd9cca00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c819ea20 con 0x7fd9c8100c80 2024-03-23T17:50:25.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.990+0000 7fd9c6800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9c8102810 0x7fd9c819e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:25.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.990+0000 7fd9c6800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9c8102810 0x7fd9c819e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43060/0 (socket says 172.21.15.86:43060) 2024-03-23T17:50:25.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.990+0000 7fd9c6800700 1 -- 172.21.15.86:0/2723136281 learned_addr learned my addr 172.21.15.86:0/2723136281 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:25.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.990+0000 7fd9c5e00700 1 --2- 172.21.15.86:0/2723136281 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd9c8100c80 0x7fd9c8196c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:25.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.990+0000 7fd9c5400700 1 --2- 172.21.15.86:0/2723136281 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9c8101e70 0x7fd9c8197160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:25.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.990+0000 7fd9c6800700 1 -- 172.21.15.86:0/2723136281 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd9c8100c80 msgr2=0x7fd9c8196c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:25.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.990+0000 7fd9c6800700 1 --2- 172.21.15.86:0/2723136281 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd9c8100c80 0x7fd9c8196c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:25.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.990+0000 7fd9c6800700 1 -- 172.21.15.86:0/2723136281 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9c8101e70 msgr2=0x7fd9c8197160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:25.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.990+0000 7fd9c6800700 1 --2- 172.21.15.86:0/2723136281 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9c8101e70 0x7fd9c8197160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:25.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.990+0000 7fd9c6800700 1 -- 172.21.15.86:0/2723136281 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9b80095f0 con 0x7fd9c8102810 2024-03-23T17:50:25.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.990+0000 7fd9c6800700 1 --2- 172.21.15.86:0/2723136281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9c8102810 0x7fd9c819e180 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd9c000ef30 tx=0x7fd9c000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:25.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.990+0000 7fd9bea00700 1 -- 172.21.15.86:0/2723136281 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c0018070 con 0x7fd9c8102810 2024-03-23T17:50:25.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.991+0000 7fd9bea00700 1 -- 172.21.15.86:0/2723136281 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd9c000f040 con 0x7fd9c8102810 2024-03-23T17:50:25.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.991+0000 7fd9cca00700 1 -- 172.21.15.86:0/2723136281 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9c819eca0 con 0x7fd9c8102810 2024-03-23T17:50:25.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.991+0000 7fd9bea00700 1 -- 172.21.15.86:0/2723136281 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c000cd00 con 0x7fd9c8102810 2024-03-23T17:50:25.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.991+0000 7fd9cca00700 1 -- 172.21.15.86:0/2723136281 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9c819f1b0 con 0x7fd9c8102810 2024-03-23T17:50:25.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.992+0000 7fd9cca00700 1 -- 172.21.15.86:0/2723136281 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9c8108100 con 0x7fd9c8102810 2024-03-23T17:50:25.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.993+0000 7fd9bea00700 1 -- 172.21.15.86:0/2723136281 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fd9c0003a00 con 0x7fd9c8102810 2024-03-23T17:50:25.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.995+0000 7fd9bea00700 1 --2- 172.21.15.86:0/2723136281 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fd9ac0a1230 0x7fd9ac0a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:25.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.995+0000 7fd9bea00700 1 -- 172.21.15.86:0/2723136281 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fd9c00cc510 con 0x7fd9c8102810 2024-03-23T17:50:25.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.995+0000 7fd9c5e00700 1 --2- 172.21.15.86:0/2723136281 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fd9ac0a1230 0x7fd9ac0a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:25.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.996+0000 7fd9c5e00700 1 --2- 172.21.15.86:0/2723136281 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fd9ac0a1230 0x7fd9ac0a36f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd9b8009990 tx=0x7fd9b8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:25.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:25.999+0000 7fd9bea00700 1 -- 172.21.15.86:0/2723136281 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fd9c0095ba0 con 0x7fd9c8102810 2024-03-23T17:50:26.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:25 smithi086 ceph-mon[38221]: pgmap v280: 97 pgs: 97 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 9 op/s 2024-03-23T17:50:26.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:25 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/1885316573' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T17:50:26.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:25 smithi092 ceph-mon[79089]: pgmap v280: 97 pgs: 97 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 9 op/s 2024-03-23T17:50:26.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:25 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/1885316573' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T17:50:26.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:25 smithi152 ceph-mon[47949]: pgmap v280: 97 pgs: 97 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 9 op/s 2024-03-23T17:50:26.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:25 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/1885316573' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T17:50:26.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.205+0000 7fd9cca00700 1 -- 172.21.15.86:0/2723136281 --> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd9c802ce10 con 0x7fd9ac0a1230 2024-03-23T17:50:26.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.216+0000 7fd9bea00700 1 -- 172.21.15.86:0/2723136281 <== mgr.14152 v2:172.21.15.86:6800/2734784244 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+344441 (secure 0 0 0) 0x7fd9c802ce10 con 0x7fd9ac0a1230 2024-03-23T17:50:26.221 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:50:26.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.220+0000 7fd9cca00700 1 -- 172.21.15.86:0/2723136281 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fd9ac0a1230 msgr2=0x7fd9ac0a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:26.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.220+0000 7fd9cca00700 1 --2- 172.21.15.86:0/2723136281 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fd9ac0a1230 0x7fd9ac0a36f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd9b8009990 tx=0x7fd9b8000bc0 comp rx=0 tx=0).stop 2024-03-23T17:50:26.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.220+0000 7fd9cca00700 1 -- 172.21.15.86:0/2723136281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9c8102810 msgr2=0x7fd9c819e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:26.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.220+0000 7fd9cca00700 1 --2- 172.21.15.86:0/2723136281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9c8102810 0x7fd9c819e180 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd9c000ef30 tx=0x7fd9c000c3a0 comp rx=0 tx=0).stop 2024-03-23T17:50:26.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.221+0000 7fd9cca00700 1 -- 172.21.15.86:0/2723136281 shutdown_connections 2024-03-23T17:50:26.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.221+0000 7fd9cca00700 1 --2- 172.21.15.86:0/2723136281 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fd9ac0a1230 0x7fd9ac0a36f0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:26.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.221+0000 7fd9cca00700 1 --2- 172.21.15.86:0/2723136281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9c8102810 0x7fd9c819e180 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:26.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.221+0000 7fd9cca00700 1 --2- 172.21.15.86:0/2723136281 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd9c8100c80 0x7fd9c8196c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:26.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.221+0000 7fd9cca00700 1 --2- 172.21.15.86:0/2723136281 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9c8101e70 0x7fd9c8197160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:26.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.221+0000 7fd9cca00700 1 -- 172.21.15.86:0/2723136281 >> 172.21.15.86:0/2723136281 conn(0x7fd9c80fc1f0 msgr2=0x7fd9c8105240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:26.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.221+0000 7fd9cca00700 1 -- 172.21.15.86:0/2723136281 shutdown_connections 2024-03-23T17:50:26.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.221+0000 7fd9cca00700 1 -- 172.21.15.86:0/2723136281 wait complete. 2024-03-23T17:50:26.224 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-03-23T17:50:26.239 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":280,"stamp":"2024-03-23T17:50:24.329151+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661176,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":955,"num_read_kb":600,"num_write":418,"num_write_kb":3078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":1180916,"num_keys_recovered":17,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":345,"ondisk_log_size":345,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":373672,"kb_used_data":12568,"kb_used_omap":18,"kb_used_meta":356845,"kb_avail":1124568152,"statfs":{"total":1151940427776,"available":1151557787648,"internally_reserved":0,"allocated":12869632,"data_stored":7106887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19130,"internal_metadata":365409606},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4926,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":106,"num_read_kb":51,"num_write":7,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002979"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.782853+0000","last_change":"2024-03-23T17:48:56.239724+0000","last_active":"2024-03-23T17:49:28.782853+0000","last_peered":"2024-03-23T17:49:28.782853+0000","last_clean":"2024-03-23T17:49:28.782853+0000","last_became_active":"2024-03-23T17:48:56.239451+0000","last_became_peered":"2024-03-23T17:48:56.239451+0000","last_unstale":"2024-03-23T17:49:28.782853+0000","last_undegraded":"2024-03-23T17:49:28.782853+0000","last_fullsized":"2024-03-23T17:49:28.782853+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:51:38.687863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.783538+0000","last_change":"2024-03-23T17:48:56.232415+0000","last_active":"2024-03-23T17:49:28.783538+0000","last_peered":"2024-03-23T17:49:28.783538+0000","last_clean":"2024-03-23T17:49:28.783538+0000","last_became_active":"2024-03-23T17:48:56.231777+0000","last_became_peered":"2024-03-23T17:48:56.231777+0000","last_unstale":"2024-03-23T17:49:28.783538+0000","last_undegraded":"2024-03-23T17:49:28.783538+0000","last_fullsized":"2024-03-23T17:49:28.783538+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:16:22.631654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.784174+0000","last_change":"2024-03-23T17:48:56.243413+0000","last_active":"2024-03-23T17:49:28.784174+0000","last_peered":"2024-03-23T17:49:28.784174+0000","last_clean":"2024-03-23T17:49:28.784174+0000","last_became_active":"2024-03-23T17:48:56.241281+0000","last_became_peered":"2024-03-23T17:48:56.241281+0000","last_unstale":"2024-03-23T17:49:28.784174+0000","last_undegraded":"2024-03-23T17:49:28.784174+0000","last_fullsized":"2024-03-23T17:49:28.784174+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:52:48.521689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.784355+0000","last_change":"2024-03-23T17:48:56.226010+0000","last_active":"2024-03-23T17:49:28.784355+0000","last_peered":"2024-03-23T17:49:28.784355+0000","last_clean":"2024-03-23T17:49:28.784355+0000","last_became_active":"2024-03-23T17:48:56.225562+0000","last_became_peered":"2024-03-23T17:48:56.225562+0000","last_unstale":"2024-03-23T17:49:28.784355+0000","last_undegraded":"2024-03-23T17:49:28.784355+0000","last_fullsized":"2024-03-23T17:49:28.784355+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:17:47.597335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.780757+0000","last_change":"2024-03-23T17:48:56.223249+0000","last_active":"2024-03-23T17:49:28.780757+0000","last_peered":"2024-03-23T17:49:28.780757+0000","last_clean":"2024-03-23T17:49:28.780757+0000","last_became_active":"2024-03-23T17:48:56.222902+0000","last_became_peered":"2024-03-23T17:48:56.222902+0000","last_unstale":"2024-03-23T17:49:28.780757+0000","last_undegraded":"2024-03-23T17:49:28.780757+0000","last_fullsized":"2024-03-23T17:49:28.780757+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:51:49.029628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.781946+0000","last_change":"2024-03-23T17:48:56.229229+0000","last_active":"2024-03-23T17:49:28.781946+0000","last_peered":"2024-03-23T17:49:28.781946+0000","last_clean":"2024-03-23T17:49:28.781946+0000","last_became_active":"2024-03-23T17:48:56.228725+0000","last_became_peered":"2024-03-23T17:48:56.228725+0000","last_unstale":"2024-03-23T17:49:28.781946+0000","last_undegraded":"2024-03-23T17:49:28.781946+0000","last_fullsized":"2024-03-23T17:49:28.781946+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:53:44.118762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.780711+0000","last_change":"2024-03-23T17:48:56.229212+0000","last_active":"2024-03-23T17:49:28.780711+0000","last_peered":"2024-03-23T17:49:28.780711+0000","last_clean":"2024-03-23T17:49:28.780711+0000","last_became_active":"2024-03-23T17:48:56.228888+0000","last_became_peered":"2024-03-23T17:48:56.228888+0000","last_unstale":"2024-03-23T17:49:28.780711+0000","last_undegraded":"2024-03-23T17:49:28.780711+0000","last_fullsized":"2024-03-23T17:49:28.780711+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:47:21.509554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.779498+0000","last_change":"2024-03-23T17:48:56.222925+0000","last_active":"2024-03-23T17:49:28.779498+0000","last_peered":"2024-03-23T17:49:28.779498+0000","last_clean":"2024-03-23T17:49:28.779498+0000","last_became_active":"2024-03-23T17:48:56.221504+0000","last_became_peered":"2024-03-23T17:48:56.221504+0000","last_unstale":"2024-03-23T17:49:28.779498+0000","last_undegraded":"2024-03-23T17:49:28.779498+0000","last_fullsized":"2024-03-23T17:49:28.779498+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:14:34.327705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.922478+0000","last_change":"2024-03-23T17:48:56.230530+0000","last_active":"2024-03-23T17:49:28.922478+0000","last_peered":"2024-03-23T17:49:28.922478+0000","last_clean":"2024-03-23T17:49:28.922478+0000","last_became_active":"2024-03-23T17:48:56.230224+0000","last_became_peered":"2024-03-23T17:48:56.230224+0000","last_unstale":"2024-03-23T17:49:28.922478+0000","last_undegraded":"2024-03-23T17:49:28.922478+0000","last_fullsized":"2024-03-23T17:49:28.922478+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:10:28.031956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.781514+0000","last_change":"2024-03-23T17:48:56.229673+0000","last_active":"2024-03-23T17:49:28.781514+0000","last_peered":"2024-03-23T17:49:28.781514+0000","last_clean":"2024-03-23T17:49:28.781514+0000","last_became_active":"2024-03-23T17:48:56.229000+0000","last_became_peered":"2024-03-23T17:48:56.229000+0000","last_unstale":"2024-03-23T17:49:28.781514+0000","last_undegraded":"2024-03-23T17:49:28.781514+0000","last_fullsized":"2024-03-23T17:49:28.781514+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:23:28.057256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.781035+0000","last_change":"2024-03-23T17:48:56.224508+0000","last_active":"2024-03-23T17:49:28.781035+0000","last_peered":"2024-03-23T17:49:28.781035+0000","last_clean":"2024-03-23T17:49:28.781035+0000","last_became_active":"2024-03-23T17:48:56.224172+0000","last_became_peered":"2024-03-23T17:48:56.224172+0000","last_unstale":"2024-03-23T17:49:28.781035+0000","last_undegraded":"2024-03-23T17:49:28.781035+0000","last_fullsized":"2024-03-23T17:49:28.781035+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:03:57.063472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.784254+0000","last_change":"2024-03-23T17:48:56.234978+0000","last_active":"2024-03-23T17:49:28.784254+0000","last_peered":"2024-03-23T17:49:28.784254+0000","last_clean":"2024-03-23T17:49:28.784254+0000","last_became_active":"2024-03-23T17:48:56.234641+0000","last_became_peered":"2024-03-23T17:48:56.234641+0000","last_unstale":"2024-03-23T17:49:28.784254+0000","last_undegraded":"2024-03-23T17:49:28.784254+0000","last_fullsized":"2024-03-23T17:49:28.784254+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:34:30.745991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.777608+0000","last_change":"2024-03-23T17:48:56.218534+0000","last_active":"2024-03-23T17:49:28.777608+0000","last_peered":"2024-03-23T17:49:28.777608+0000","last_clean":"2024-03-23T17:49:28.777608+0000","last_became_active":"2024-03-23T17:48:56.217424+0000","last_became_peered":"2024-03-23T17:48:56.217424+0000","last_unstale":"2024-03-23T17:49:28.777608+0000","last_undegraded":"2024-03-23T17:49:28.777608+0000","last_fullsized":"2024-03-23T17:49:28.777608+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:32:05.618057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.783444+0000","last_change":"2024-03-23T17:48:56.235525+0000","last_active":"2024-03-23T17:49:28.783444+0000","last_peered":"2024-03-23T17:49:28.783444+0000","last_clean":"2024-03-23T17:49:28.783444+0000","last_became_active":"2024-03-23T17:48:56.234591+0000","last_became_peered":"2024-03-23T17:48:56.234591+0000","last_unstale":"2024-03-23T17:49:28.783444+0000","last_undegraded":"2024-03-23T17:49:28.783444+0000","last_fullsized":"2024-03-23T17:49:28.783444+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:19:38.381878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.783446+0000","last_change":"2024-03-23T17:48:56.243137+0000","last_active":"2024-03-23T17:49:28.783446+0000","last_peered":"2024-03-23T17:49:28.783446+0000","last_clean":"2024-03-23T17:49:28.783446+0000","last_became_active":"2024-03-23T17:48:56.242516+0000","last_became_peered":"2024-03-23T17:48:56.242516+0000","last_unstale":"2024-03-23T17:49:28.783446+0000","last_undegraded":"2024-03-23T17:49:28.783446+0000","last_fullsized":"2024-03-23T17:49:28.783446+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:54:59.849467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.922041+0000","last_change":"2024-03-23T17:48:56.236026+0000","last_active":"2024-03-23T17:49:28.922041+0000","last_peered":"2024-03-23T17:49:28.922041+0000","last_clean":"2024-03-23T17:49:28.922041+0000","last_became_active":"2024-03-23T17:48:56.235645+0000","last_became_peered":"2024-03-23T17:48:56.235645+0000","last_unstale":"2024-03-23T17:49:28.922041+0000","last_undegraded":"2024-03-23T17:49:28.922041+0000","last_fullsized":"2024-03-23T17:49:28.922041+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:47:01.076737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.782865+0000","last_change":"2024-03-23T17:48:56.239689+0000","last_active":"2024-03-23T17:49:28.782865+0000","last_peered":"2024-03-23T17:49:28.782865+0000","last_clean":"2024-03-23T17:49:28.782865+0000","last_became_active":"2024-03-23T17:48:56.239331+0000","last_became_peered":"2024-03-23T17:48:56.239331+0000","last_unstale":"2024-03-23T17:49:28.782865+0000","last_undegraded":"2024-03-23T17:49:28.782865+0000","last_fullsized":"2024-03-23T17:49:28.782865+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:21:15.287627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.777987+0000","last_change":"2024-03-23T17:48:56.229934+0000","last_active":"2024-03-23T17:49:28.777987+0000","last_peered":"2024-03-23T17:49:28.777987+0000","last_clean":"2024-03-23T17:49:28.777987+0000","last_became_active":"2024-03-23T17:48:56.229244+0000","last_became_peered":"2024-03-23T17:48:56.229244+0000","last_unstale":"2024-03-23T17:49:28.777987+0000","last_undegraded":"2024-03-23T17:49:28.777987+0000","last_fullsized":"2024-03-23T17:49:28.777987+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:02:40.254799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.784936+0000","last_change":"2024-03-23T17:48:56.233786+0000","last_active":"2024-03-23T17:49:28.784936+0000","last_peered":"2024-03-23T17:49:28.784936+0000","last_clean":"2024-03-23T17:49:28.784936+0000","last_became_active":"2024-03-23T17:48:56.233137+0000","last_became_peered":"2024-03-23T17:48:56.233137+0000","last_unstale":"2024-03-23T17:49:28.784936+0000","last_undegraded":"2024-03-23T17:49:28.784936+0000","last_fullsized":"2024-03-23T17:49:28.784936+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:33:35.202788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"82'14","reported_seq":108,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-23T17:50:20.814129+0000","last_change":"2024-03-23T17:48:55.712184+0000","last_active":"2024-03-23T17:50:20.814129+0000","last_peered":"2024-03-23T17:50:20.814129+0000","last_clean":"2024-03-23T17:50:20.814129+0000","last_became_active":"2024-03-23T17:48:55.711860+0000","last_became_peered":"2024-03-23T17:48:55.711860+0000","last_unstale":"2024-03-23T17:50:20.814129+0000","last_undegraded":"2024-03-23T17:50:20.814129+0000","last_fullsized":"2024-03-23T17:50:20.814129+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:59:01.133652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":80,"num_read_kb":80,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-23T17:49:29.789705+0000","last_change":"2024-03-23T17:49:29.789057+0000","last_active":"2024-03-23T17:49:29.789705+0000","last_peered":"2024-03-23T17:49:29.789705+0000","last_clean":"2024-03-23T17:49:29.789705+0000","last_became_active":"2024-03-23T17:49:29.788796+0000","last_became_peered":"2024-03-23T17:49:29.788796+0000","last_unstale":"2024-03-23T17:49:29.789705+0000","last_undegraded":"2024-03-23T17:49:29.789705+0000","last_fullsized":"2024-03-23T17:49:29.789705+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:30:53.062926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.780493+0000","last_change":"2024-03-23T17:48:55.725351+0000","last_active":"2024-03-23T17:49:28.780493+0000","last_peered":"2024-03-23T17:49:28.780493+0000","last_clean":"2024-03-23T17:49:28.780493+0000","last_became_active":"2024-03-23T17:48:55.724566+0000","last_became_peered":"2024-03-23T17:48:55.724566+0000","last_unstale":"2024-03-23T17:49:28.780493+0000","last_undegraded":"2024-03-23T17:49:28.780493+0000","last_fullsized":"2024-03-23T17:49:28.780493+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:58:21.088166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.784151+0000","last_change":"2024-03-23T17:48:56.243526+0000","last_active":"2024-03-23T17:49:28.784151+0000","last_peered":"2024-03-23T17:49:28.784151+0000","last_clean":"2024-03-23T17:49:28.784151+0000","last_became_active":"2024-03-23T17:48:56.242468+0000","last_became_peered":"2024-03-23T17:48:56.242468+0000","last_unstale":"2024-03-23T17:49:28.784151+0000","last_undegraded":"2024-03-23T17:49:28.784151+0000","last_fullsized":"2024-03-23T17:49:28.784151+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:16:50.392770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"82'13","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-23T17:49:29.781747+0000","last_change":"2024-03-23T17:48:55.720237+0000","last_active":"2024-03-23T17:49:29.781747+0000","last_peered":"2024-03-23T17:49:29.781747+0000","last_clean":"2024-03-23T17:49:29.781747+0000","last_became_active":"2024-03-23T17:48:55.719843+0000","last_became_peered":"2024-03-23T17:48:55.719843+0000","last_unstale":"2024-03-23T17:49:29.781747+0000","last_undegraded":"2024-03-23T17:49:29.781747+0000","last_fullsized":"2024-03-23T17:49:29.781747+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:07:26.973511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.922177+0000","last_change":"2024-03-23T17:48:56.235642+0000","last_active":"2024-03-23T17:49:28.922177+0000","last_peered":"2024-03-23T17:49:28.922177+0000","last_clean":"2024-03-23T17:49:28.922177+0000","last_became_active":"2024-03-23T17:48:56.235326+0000","last_became_peered":"2024-03-23T17:48:56.235326+0000","last_unstale":"2024-03-23T17:49:28.922177+0000","last_undegraded":"2024-03-23T17:49:28.922177+0000","last_fullsized":"2024-03-23T17:49:28.922177+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:21:59.934620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.781362+0000","last_change":"2024-03-23T17:48:55.718579+0000","last_active":"2024-03-23T17:49:28.781362+0000","last_peered":"2024-03-23T17:49:28.781362+0000","last_clean":"2024-03-23T17:49:28.781362+0000","last_became_active":"2024-03-23T17:48:55.718220+0000","last_became_peered":"2024-03-23T17:48:55.718220+0000","last_unstale":"2024-03-23T17:49:28.781362+0000","last_undegraded":"2024-03-23T17:49:28.781362+0000","last_fullsized":"2024-03-23T17:49:28.781362+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:37:56.417367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.785628+0000","last_change":"2024-03-23T17:48:56.224842+0000","last_active":"2024-03-23T17:49:28.785628+0000","last_peered":"2024-03-23T17:49:28.785628+0000","last_clean":"2024-03-23T17:49:28.785628+0000","last_became_active":"2024-03-23T17:48:56.224427+0000","last_became_peered":"2024-03-23T17:48:56.224427+0000","last_unstale":"2024-03-23T17:49:28.785628+0000","last_undegraded":"2024-03-23T17:49:28.785628+0000","last_fullsized":"2024-03-23T17:49:28.785628+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:16:22.684369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"80'5","reported_seq":103,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-23T17:50:20.781361+0000","last_change":"2024-03-23T17:48:55.713996+0000","last_active":"2024-03-23T17:50:20.781361+0000","last_peered":"2024-03-23T17:50:20.781361+0000","last_clean":"2024-03-23T17:50:20.781361+0000","last_became_active":"2024-03-23T17:48:55.713647+0000","last_became_peered":"2024-03-23T17:48:55.713647+0000","last_unstale":"2024-03-23T17:50:20.781361+0000","last_undegraded":"2024-03-23T17:50:20.781361+0000","last_fullsized":"2024-03-23T17:50:20.781361+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:12:22.251533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":84,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.781148+0000","last_change":"2024-03-23T17:48:56.219281+0000","last_active":"2024-03-23T17:49:28.781148+0000","last_peered":"2024-03-23T17:49:28.781148+0000","last_clean":"2024-03-23T17:49:28.781148+0000","last_became_active":"2024-03-23T17:48:56.218924+0000","last_became_peered":"2024-03-23T17:48:56.218924+0000","last_unstale":"2024-03-23T17:49:28.781148+0000","last_undegraded":"2024-03-23T17:49:28.781148+0000","last_fullsized":"2024-03-23T17:49:28.781148+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:49:04.793088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.781333+0000","last_change":"2024-03-23T17:48:55.718131+0000","last_active":"2024-03-23T17:49:28.781333+0000","last_peered":"2024-03-23T17:49:28.781333+0000","last_clean":"2024-03-23T17:49:28.781333+0000","last_became_active":"2024-03-23T17:48:55.717760+0000","last_became_peered":"2024-03-23T17:48:55.717760+0000","last_unstale":"2024-03-23T17:49:28.781333+0000","last_undegraded":"2024-03-23T17:49:28.781333+0000","last_fullsized":"2024-03-23T17:49:28.781333+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:16:54.355844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.784309+0000","last_change":"2024-03-23T17:48:56.225965+0000","last_active":"2024-03-23T17:49:28.784309+0000","last_peered":"2024-03-23T17:49:28.784309+0000","last_clean":"2024-03-23T17:49:28.784309+0000","last_became_active":"2024-03-23T17:48:56.225101+0000","last_became_peered":"2024-03-23T17:48:56.225101+0000","last_unstale":"2024-03-23T17:49:28.784309+0000","last_undegraded":"2024-03-23T17:49:28.784309+0000","last_fullsized":"2024-03-23T17:49:28.784309+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:52:14.302306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.776847+0000","last_change":"2024-03-23T17:48:55.722289+0000","last_active":"2024-03-23T17:49:28.776847+0000","last_peered":"2024-03-23T17:49:28.776847+0000","last_clean":"2024-03-23T17:49:28.776847+0000","last_became_active":"2024-03-23T17:48:55.721927+0000","last_became_peered":"2024-03-23T17:48:55.721927+0000","last_unstale":"2024-03-23T17:49:28.776847+0000","last_undegraded":"2024-03-23T17:49:28.776847+0000","last_fullsized":"2024-03-23T17:49:28.776847+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:20:08.354897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.779465+0000","last_change":"2024-03-23T17:48:56.232231+0000","last_active":"2024-03-23T17:49:28.779465+0000","last_peered":"2024-03-23T17:49:28.779465+0000","last_clean":"2024-03-23T17:49:28.779465+0000","last_became_active":"2024-03-23T17:48:56.227335+0000","last_became_peered":"2024-03-23T17:48:56.227335+0000","last_unstale":"2024-03-23T17:49:28.779465+0000","last_undegraded":"2024-03-23T17:49:28.779465+0000","last_fullsized":"2024-03-23T17:49:28.779465+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:49:01.460775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.785071+0000","last_change":"2024-03-23T17:48:55.717615+0000","last_active":"2024-03-23T17:49:28.785071+0000","last_peered":"2024-03-23T17:49:28.785071+0000","last_clean":"2024-03-23T17:49:28.785071+0000","last_became_active":"2024-03-23T17:48:55.717266+0000","last_became_peered":"2024-03-23T17:48:55.717266+0000","last_unstale":"2024-03-23T17:49:28.785071+0000","last_undegraded":"2024-03-23T17:49:28.785071+0000","last_fullsized":"2024-03-23T17:49:28.785071+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:40:07.988182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.776405+0000","last_change":"2024-03-23T17:48:56.229787+0000","last_active":"2024-03-23T17:49:28.776405+0000","last_peered":"2024-03-23T17:49:28.776405+0000","last_clean":"2024-03-23T17:49:28.776405+0000","last_became_active":"2024-03-23T17:48:56.229471+0000","last_became_peered":"2024-03-23T17:48:56.229471+0000","last_unstale":"2024-03-23T17:49:28.776405+0000","last_undegraded":"2024-03-23T17:49:28.776405+0000","last_fullsized":"2024-03-23T17:49:28.776405+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:44:09.358809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'4","reported_seq":173,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-23T17:50:18.777100+0000","last_change":"2024-03-23T17:48:55.717130+0000","last_active":"2024-03-23T17:50:18.777100+0000","last_peered":"2024-03-23T17:50:18.777100+0000","last_clean":"2024-03-23T17:50:18.777100+0000","last_became_active":"2024-03-23T17:48:55.716733+0000","last_became_peered":"2024-03-23T17:48:55.716733+0000","last_unstale":"2024-03-23T17:50:18.777100+0000","last_undegraded":"2024-03-23T17:50:18.777100+0000","last_fullsized":"2024-03-23T17:50:18.777100+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:22:43.229694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":155,"num_read_kb":155,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.922446+0000","last_change":"2024-03-23T17:48:56.235565+0000","last_active":"2024-03-23T17:49:28.922446+0000","last_peered":"2024-03-23T17:49:28.922446+0000","last_clean":"2024-03-23T17:49:28.922446+0000","last_became_active":"2024-03-23T17:48:56.235267+0000","last_became_peered":"2024-03-23T17:48:56.235267+0000","last_unstale":"2024-03-23T17:49:28.922446+0000","last_undegraded":"2024-03-23T17:49:28.922446+0000","last_fullsized":"2024-03-23T17:49:28.922446+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:38:45.906545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.780900+0000","last_change":"2024-03-23T17:48:55.719096+0000","last_active":"2024-03-23T17:49:28.780900+0000","last_peered":"2024-03-23T17:49:28.780900+0000","last_clean":"2024-03-23T17:49:28.780900+0000","last_became_active":"2024-03-23T17:48:55.718721+0000","last_became_peered":"2024-03-23T17:48:55.718721+0000","last_unstale":"2024-03-23T17:49:28.780900+0000","last_undegraded":"2024-03-23T17:49:28.780900+0000","last_fullsized":"2024-03-23T17:49:28.780900+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:43:05.532263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.784108+0000","last_change":"2024-03-23T17:48:56.235549+0000","last_active":"2024-03-23T17:49:28.784108+0000","last_peered":"2024-03-23T17:49:28.784108+0000","last_clean":"2024-03-23T17:49:28.784108+0000","last_became_active":"2024-03-23T17:48:56.234572+0000","last_became_peered":"2024-03-23T17:48:56.234572+0000","last_unstale":"2024-03-23T17:49:28.784108+0000","last_undegraded":"2024-03-23T17:49:28.784108+0000","last_fullsized":"2024-03-23T17:49:28.784108+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:19:58.639181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.785732+0000","last_change":"2024-03-23T17:48:55.724688+0000","last_active":"2024-03-23T17:49:28.785732+0000","last_peered":"2024-03-23T17:49:28.785732+0000","last_clean":"2024-03-23T17:49:28.785732+0000","last_became_active":"2024-03-23T17:48:55.724385+0000","last_became_peered":"2024-03-23T17:48:55.724385+0000","last_unstale":"2024-03-23T17:49:28.785732+0000","last_undegraded":"2024-03-23T17:49:28.785732+0000","last_fullsized":"2024-03-23T17:49:28.785732+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:17:54.338059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.922287+0000","last_change":"2024-03-23T17:48:56.242983+0000","last_active":"2024-03-23T17:49:28.922287+0000","last_peered":"2024-03-23T17:49:28.922287+0000","last_clean":"2024-03-23T17:49:28.922287+0000","last_became_active":"2024-03-23T17:48:56.242596+0000","last_became_peered":"2024-03-23T17:48:56.242596+0000","last_unstale":"2024-03-23T17:49:28.922287+0000","last_undegraded":"2024-03-23T17:49:28.922287+0000","last_fullsized":"2024-03-23T17:49:28.922287+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:55:51.971834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'32","reported_seq":211,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-23T17:50:19.225390+0000","last_change":"2024-03-23T17:48:55.717631+0000","last_active":"2024-03-23T17:50:19.225390+0000","last_peered":"2024-03-23T17:50:19.225390+0000","last_clean":"2024-03-23T17:50:19.225390+0000","last_became_active":"2024-03-23T17:48:55.717082+0000","last_became_peered":"2024-03-23T17:48:55.717082+0000","last_unstale":"2024-03-23T17:50:19.225390+0000","last_undegraded":"2024-03-23T17:50:19.225390+0000","last_fullsized":"2024-03-23T17:50:19.225390+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:36:25.323269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64038,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":165,"num_read_kb":140,"num_write":34,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.780792+0000","last_change":"2024-03-23T17:48:56.226174+0000","last_active":"2024-03-23T17:49:28.780792+0000","last_peered":"2024-03-23T17:49:28.780792+0000","last_clean":"2024-03-23T17:49:28.780792+0000","last_became_active":"2024-03-23T17:48:56.225834+0000","last_became_peered":"2024-03-23T17:48:56.225834+0000","last_unstale":"2024-03-23T17:49:28.780792+0000","last_undegraded":"2024-03-23T17:49:28.780792+0000","last_fullsized":"2024-03-23T17:49:28.780792+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:50:16.793071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-23T17:49:29.804494+0000","last_change":"2024-03-23T17:49:29.804099+0000","last_active":"2024-03-23T17:49:29.804494+0000","last_peered":"2024-03-23T17:49:29.804494+0000","last_clean":"2024-03-23T17:49:29.804494+0000","last_became_active":"2024-03-23T17:49:29.787483+0000","last_became_peered":"2024-03-23T17:49:29.787483+0000","last_unstale":"2024-03-23T17:49:29.804494+0000","last_undegraded":"2024-03-23T17:49:29.804494+0000","last_fullsized":"2024-03-23T17:49:29.804494+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:13:16.463738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.776667+0000","last_change":"2024-03-23T17:48:56.228459+0000","last_active":"2024-03-23T17:49:28.776667+0000","last_peered":"2024-03-23T17:49:28.776667+0000","last_clean":"2024-03-23T17:49:28.776667+0000","last_became_active":"2024-03-23T17:48:56.228179+0000","last_became_peered":"2024-03-23T17:48:56.228179+0000","last_unstale":"2024-03-23T17:49:28.776667+0000","last_undegraded":"2024-03-23T17:49:28.776667+0000","last_fullsized":"2024-03-23T17:49:28.776667+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:28:41.613432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.785318+0000","last_change":"2024-03-23T17:48:55.721533+0000","last_active":"2024-03-23T17:49:28.785318+0000","last_peered":"2024-03-23T17:49:28.785318+0000","last_clean":"2024-03-23T17:49:28.785318+0000","last_became_active":"2024-03-23T17:48:55.721100+0000","last_became_peered":"2024-03-23T17:48:55.721100+0000","last_unstale":"2024-03-23T17:49:28.785318+0000","last_undegraded":"2024-03-23T17:49:28.785318+0000","last_fullsized":"2024-03-23T17:49:28.785318+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:56:58.112649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-23T17:49:30.003965+0000","last_change":"2024-03-23T17:49:30.003965+0000","last_active":"2024-03-23T17:49:30.003965+0000","last_peered":"2024-03-23T17:49:30.003965+0000","last_clean":"2024-03-23T17:49:30.003965+0000","last_became_active":"2024-03-23T17:49:30.003643+0000","last_became_peered":"2024-03-23T17:49:30.003643+0000","last_unstale":"2024-03-23T17:49:30.003965+0000","last_undegraded":"2024-03-23T17:49:30.003965+0000","last_fullsized":"2024-03-23T17:49:30.003965+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:47:39.892782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'5","reported_seq":97,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-23T17:50:17.824512+0000","last_change":"2024-03-23T17:48:55.715205+0000","last_active":"2024-03-23T17:50:17.824512+0000","last_peered":"2024-03-23T17:50:17.824512+0000","last_clean":"2024-03-23T17:50:17.824512+0000","last_became_active":"2024-03-23T17:48:55.714810+0000","last_became_peered":"2024-03-23T17:48:55.714810+0000","last_unstale":"2024-03-23T17:50:17.824512+0000","last_undegraded":"2024-03-23T17:50:17.824512+0000","last_fullsized":"2024-03-23T17:50:17.824512+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:40:08.768998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":79,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-23T17:49:29.783025+0000","last_change":"2024-03-23T17:49:29.783025+0000","last_active":"2024-03-23T17:49:29.783025+0000","last_peered":"2024-03-23T17:49:29.783025+0000","last_clean":"2024-03-23T17:49:29.783025+0000","last_became_active":"2024-03-23T17:49:29.782541+0000","last_became_peered":"2024-03-23T17:49:29.782541+0000","last_unstale":"2024-03-23T17:49:29.783025+0000","last_undegraded":"2024-03-23T17:49:29.783025+0000","last_fullsized":"2024-03-23T17:49:29.783025+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:50:29.754074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":59,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-23T17:49:30.616960+0000","last_change":"2024-03-23T17:49:30.616960+0000","last_active":"2024-03-23T17:49:30.616960+0000","last_peered":"2024-03-23T17:49:30.616960+0000","last_clean":"2024-03-23T17:49:30.616960+0000","last_became_active":"2024-03-23T17:49:30.604464+0000","last_became_peered":"2024-03-23T17:49:30.604464+0000","last_unstale":"2024-03-23T17:49:30.616960+0000","last_undegraded":"2024-03-23T17:49:30.616960+0000","last_fullsized":"2024-03-23T17:49:30.616960+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:24:13.862841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.785079+0000","last_change":"2024-03-23T17:48:56.225624+0000","last_active":"2024-03-23T17:49:28.785079+0000","last_peered":"2024-03-23T17:49:28.785079+0000","last_clean":"2024-03-23T17:49:28.785079+0000","last_became_active":"2024-03-23T17:48:56.223798+0000","last_became_peered":"2024-03-23T17:48:56.223798+0000","last_unstale":"2024-03-23T17:49:28.785079+0000","last_undegraded":"2024-03-23T17:49:28.785079+0000","last_fullsized":"2024-03-23T17:49:28.785079+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:00:20.547553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.781323+0000","last_change":"2024-03-23T17:48:55.717792+0000","last_active":"2024-03-23T17:49:28.781323+0000","last_peered":"2024-03-23T17:49:28.781323+0000","last_clean":"2024-03-23T17:49:28.781323+0000","last_became_active":"2024-03-23T17:48:55.717479+0000","last_became_peered":"2024-03-23T17:48:55.717479+0000","last_unstale":"2024-03-23T17:49:28.781323+0000","last_undegraded":"2024-03-23T17:49:28.781323+0000","last_fullsized":"2024-03-23T17:49:28.781323+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:49:43.483785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.782572+0000","last_change":"2024-03-23T17:48:56.219135+0000","last_active":"2024-03-23T17:49:28.782572+0000","last_peered":"2024-03-23T17:49:28.782572+0000","last_clean":"2024-03-23T17:49:28.782572+0000","last_became_active":"2024-03-23T17:48:56.218824+0000","last_became_peered":"2024-03-23T17:48:56.218824+0000","last_unstale":"2024-03-23T17:49:28.782572+0000","last_undegraded":"2024-03-23T17:49:28.782572+0000","last_fullsized":"2024-03-23T17:49:28.782572+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:20:35.190781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.776774+0000","last_change":"2024-03-23T17:48:55.717530+0000","last_active":"2024-03-23T17:49:28.776774+0000","last_peered":"2024-03-23T17:49:28.776774+0000","last_clean":"2024-03-23T17:49:28.776774+0000","last_became_active":"2024-03-23T17:48:55.717213+0000","last_became_peered":"2024-03-23T17:48:55.717213+0000","last_unstale":"2024-03-23T17:49:28.776774+0000","last_undegraded":"2024-03-23T17:49:28.776774+0000","last_fullsized":"2024-03-23T17:49:28.776774+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:40:09.803021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'1","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.781617+0000","last_change":"2024-03-23T17:48:56.222237+0000","last_active":"2024-03-23T17:49:28.781617+0000","last_peered":"2024-03-23T17:49:28.781617+0000","last_clean":"2024-03-23T17:49:28.781617+0000","last_became_active":"2024-03-23T17:48:56.221916+0000","last_became_peered":"2024-03-23T17:48:56.221916+0000","last_unstale":"2024-03-23T17:49:28.781617+0000","last_undegraded":"2024-03-23T17:49:28.781617+0000","last_fullsized":"2024-03-23T17:49:28.781617+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:35:59.213482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"82'6","reported_seq":100,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-23T17:50:19.813475+0000","last_change":"2024-03-23T17:48:55.718143+0000","last_active":"2024-03-23T17:50:19.813475+0000","last_peered":"2024-03-23T17:50:19.813475+0000","last_clean":"2024-03-23T17:50:19.813475+0000","last_became_active":"2024-03-23T17:48:55.717834+0000","last_became_peered":"2024-03-23T17:48:55.717834+0000","last_unstale":"2024-03-23T17:50:19.813475+0000","last_undegraded":"2024-03-23T17:50:19.813475+0000","last_fullsized":"2024-03-23T17:50:19.813475+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:29:08.879636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":846,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":80,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.781385+0000","last_change":"2024-03-23T17:48:56.237136+0000","last_active":"2024-03-23T17:49:28.781385+0000","last_peered":"2024-03-23T17:49:28.781385+0000","last_clean":"2024-03-23T17:49:28.781385+0000","last_became_active":"2024-03-23T17:48:56.236795+0000","last_became_peered":"2024-03-23T17:48:56.236795+0000","last_unstale":"2024-03-23T17:49:28.781385+0000","last_undegraded":"2024-03-23T17:49:28.781385+0000","last_fullsized":"2024-03-23T17:49:28.781385+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:15:31.356382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-23T17:49:29.789433+0000","last_change":"2024-03-23T17:49:29.789433+0000","last_active":"2024-03-23T17:49:29.789433+0000","last_peered":"2024-03-23T17:49:29.789433+0000","last_clean":"2024-03-23T17:49:29.789433+0000","last_became_active":"2024-03-23T17:49:29.788956+0000","last_became_peered":"2024-03-23T17:49:29.788956+0000","last_unstale":"2024-03-23T17:49:29.789433+0000","last_undegraded":"2024-03-23T17:49:29.789433+0000","last_fullsized":"2024-03-23T17:49:29.789433+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:43:30.929796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"80'5","reported_seq":96,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-23T17:50:17.823614+0000","last_change":"2024-03-23T17:48:55.722968+0000","last_active":"2024-03-23T17:50:17.823614+0000","last_peered":"2024-03-23T17:50:17.823614+0000","last_clean":"2024-03-23T17:50:17.823614+0000","last_became_active":"2024-03-23T17:48:55.722578+0000","last_became_peered":"2024-03-23T17:48:55.722578+0000","last_unstale":"2024-03-23T17:50:17.823614+0000","last_undegraded":"2024-03-23T17:50:17.823614+0000","last_fullsized":"2024-03-23T17:50:17.823614+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:45:42.344283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":77,"num_read_kb":77,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.779906+0000","last_change":"2024-03-23T17:48:56.228000+0000","last_active":"2024-03-23T17:49:28.779906+0000","last_peered":"2024-03-23T17:49:28.779906+0000","last_clean":"2024-03-23T17:49:28.779906+0000","last_became_active":"2024-03-23T17:48:56.225645+0000","last_became_peered":"2024-03-23T17:48:56.225645+0000","last_unstale":"2024-03-23T17:49:28.779906+0000","last_undegraded":"2024-03-23T17:49:28.779906+0000","last_fullsized":"2024-03-23T17:49:28.779906+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:44:25.197761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.784417+0000","last_change":"2024-03-23T17:48:55.718307+0000","last_active":"2024-03-23T17:49:28.784417+0000","last_peered":"2024-03-23T17:49:28.784417+0000","last_clean":"2024-03-23T17:49:28.784417+0000","last_became_active":"2024-03-23T17:48:55.717839+0000","last_became_peered":"2024-03-23T17:48:55.717839+0000","last_unstale":"2024-03-23T17:49:28.784417+0000","last_undegraded":"2024-03-23T17:49:28.784417+0000","last_fullsized":"2024-03-23T17:49:28.784417+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:11:56.053940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":335,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.780763+0000","last_change":"2024-03-23T17:48:01.618019+0000","last_active":"2024-03-23T17:49:28.780763+0000","last_peered":"2024-03-23T17:49:28.780763+0000","last_clean":"2024-03-23T17:49:28.780763+0000","last_became_active":"2024-03-23T17:48:01.617553+0000","last_became_peered":"2024-03-23T17:48:01.617553+0000","last_unstale":"2024-03-23T17:49:28.780763+0000","last_undegraded":"2024-03-23T17:49:28.780763+0000","last_fullsized":"2024-03-23T17:49:28.780763+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:45:26.849029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:45:26.849029+0000","last_clean_scrub_stamp":"2024-03-23T17:45:26.849029+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:05:44.823364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.776693+0000","last_change":"2024-03-23T17:48:56.224285+0000","last_active":"2024-03-23T17:49:28.776693+0000","last_peered":"2024-03-23T17:49:28.776693+0000","last_clean":"2024-03-23T17:49:28.776693+0000","last_became_active":"2024-03-23T17:48:56.220943+0000","last_became_peered":"2024-03-23T17:48:56.220943+0000","last_unstale":"2024-03-23T17:49:28.776693+0000","last_undegraded":"2024-03-23T17:49:28.776693+0000","last_fullsized":"2024-03-23T17:49:28.776693+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:36:05.458243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.780994+0000","last_change":"2024-03-23T17:48:55.715877+0000","last_active":"2024-03-23T17:49:28.780994+0000","last_peered":"2024-03-23T17:49:28.780994+0000","last_clean":"2024-03-23T17:49:28.780994+0000","last_became_active":"2024-03-23T17:48:55.715550+0000","last_became_peered":"2024-03-23T17:48:55.715550+0000","last_unstale":"2024-03-23T17:49:28.780994+0000","last_undegraded":"2024-03-23T17:49:28.780994+0000","last_fullsized":"2024-03-23T17:49:28.780994+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:31:57.668495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.780911+0000","last_change":"2024-03-23T17:48:55.724618+0000","last_active":"2024-03-23T17:49:28.780911+0000","last_peered":"2024-03-23T17:49:28.780911+0000","last_clean":"2024-03-23T17:49:28.780911+0000","last_became_active":"2024-03-23T17:48:55.724278+0000","last_became_peered":"2024-03-23T17:48:55.724278+0000","last_unstale":"2024-03-23T17:49:28.780911+0000","last_undegraded":"2024-03-23T17:49:28.780911+0000","last_fullsized":"2024-03-23T17:49:28.780911+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:27:05.065115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.776532+0000","last_change":"2024-03-23T17:48:56.216141+0000","last_active":"2024-03-23T17:49:28.776532+0000","last_peered":"2024-03-23T17:49:28.776532+0000","last_clean":"2024-03-23T17:49:28.776532+0000","last_became_active":"2024-03-23T17:48:56.215653+0000","last_became_peered":"2024-03-23T17:48:56.215653+0000","last_unstale":"2024-03-23T17:49:28.776532+0000","last_undegraded":"2024-03-23T17:49:28.776532+0000","last_fullsized":"2024-03-23T17:49:28.776532+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:02:19.611056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'9","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-23T17:49:29.794454+0000","last_change":"2024-03-23T17:49:29.794454+0000","last_active":"2024-03-23T17:49:29.794454+0000","last_peered":"2024-03-23T17:49:29.794454+0000","last_clean":"2024-03-23T17:49:29.794454+0000","last_became_active":"2024-03-23T17:49:29.787238+0000","last_became_peered":"2024-03-23T17:49:29.787238+0000","last_unstale":"2024-03-23T17:49:29.794454+0000","last_undegraded":"2024-03-23T17:49:29.794454+0000","last_fullsized":"2024-03-23T17:49:29.794454+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:33:34.917283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.780480+0000","last_change":"2024-03-23T17:48:56.228439+0000","last_active":"2024-03-23T17:49:28.780480+0000","last_peered":"2024-03-23T17:49:28.780480+0000","last_clean":"2024-03-23T17:49:28.780480+0000","last_became_active":"2024-03-23T17:48:56.226677+0000","last_became_peered":"2024-03-23T17:48:56.226677+0000","last_unstale":"2024-03-23T17:49:28.780480+0000","last_undegraded":"2024-03-23T17:49:28.780480+0000","last_fullsized":"2024-03-23T17:49:28.780480+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:46:00.063266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.784329+0000","last_change":"2024-03-23T17:48:55.724872+0000","last_active":"2024-03-23T17:49:28.784329+0000","last_peered":"2024-03-23T17:49:28.784329+0000","last_clean":"2024-03-23T17:49:28.784329+0000","last_became_active":"2024-03-23T17:48:55.724468+0000","last_became_peered":"2024-03-23T17:48:55.724468+0000","last_unstale":"2024-03-23T17:49:28.784329+0000","last_undegraded":"2024-03-23T17:49:28.784329+0000","last_fullsized":"2024-03-23T17:49:28.784329+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:28:34.918056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.783623+0000","last_change":"2024-03-23T17:48:56.233730+0000","last_active":"2024-03-23T17:49:28.783623+0000","last_peered":"2024-03-23T17:49:28.783623+0000","last_clean":"2024-03-23T17:49:28.783623+0000","last_became_active":"2024-03-23T17:48:56.233189+0000","last_became_peered":"2024-03-23T17:48:56.233189+0000","last_unstale":"2024-03-23T17:49:28.783623+0000","last_undegraded":"2024-03-23T17:49:28.783623+0000","last_fullsized":"2024-03-23T17:49:28.783623+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:20:30.764344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.922070+0000","last_change":"2024-03-23T17:48:55.724633+0000","last_active":"2024-03-23T17:49:28.922070+0000","last_peered":"2024-03-23T17:49:28.922070+0000","last_clean":"2024-03-23T17:49:28.922070+0000","last_became_active":"2024-03-23T17:48:55.724301+0000","last_became_peered":"2024-03-23T17:48:55.724301+0000","last_unstale":"2024-03-23T17:49:28.922070+0000","last_undegraded":"2024-03-23T17:49:28.922070+0000","last_fullsized":"2024-03-23T17:49:28.922070+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:45:08.392668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.780692+0000","last_change":"2024-03-23T17:48:56.230981+0000","last_active":"2024-03-23T17:49:28.780692+0000","last_peered":"2024-03-23T17:49:28.780692+0000","last_clean":"2024-03-23T17:49:28.780692+0000","last_became_active":"2024-03-23T17:48:56.230644+0000","last_became_peered":"2024-03-23T17:48:56.230644+0000","last_unstale":"2024-03-23T17:49:28.780692+0000","last_undegraded":"2024-03-23T17:49:28.780692+0000","last_fullsized":"2024-03-23T17:49:28.780692+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:37:51.310938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.779873+0000","last_change":"2024-03-23T17:48:55.719112+0000","last_active":"2024-03-23T17:49:28.779873+0000","last_peered":"2024-03-23T17:49:28.779873+0000","last_clean":"2024-03-23T17:49:28.779873+0000","last_became_active":"2024-03-23T17:48:55.718756+0000","last_became_peered":"2024-03-23T17:48:55.718756+0000","last_unstale":"2024-03-23T17:49:28.779873+0000","last_undegraded":"2024-03-23T17:49:28.779873+0000","last_fullsized":"2024-03-23T17:49:28.779873+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:41:22.164706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.781982+0000","last_change":"2024-03-23T17:48:56.240289+0000","last_active":"2024-03-23T17:49:28.781982+0000","last_peered":"2024-03-23T17:49:28.781982+0000","last_clean":"2024-03-23T17:49:28.781982+0000","last_became_active":"2024-03-23T17:48:56.239896+0000","last_became_peered":"2024-03-23T17:48:56.239896+0000","last_unstale":"2024-03-23T17:49:28.781982+0000","last_undegraded":"2024-03-23T17:49:28.781982+0000","last_fullsized":"2024-03-23T17:49:28.781982+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:49:10.505576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'3","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.785515+0000","last_change":"2024-03-23T17:48:55.723330+0000","last_active":"2024-03-23T17:49:28.785515+0000","last_peered":"2024-03-23T17:49:28.785515+0000","last_clean":"2024-03-23T17:49:28.785515+0000","last_became_active":"2024-03-23T17:48:55.722879+0000","last_became_peered":"2024-03-23T17:48:55.722879+0000","last_unstale":"2024-03-23T17:49:28.785515+0000","last_undegraded":"2024-03-23T17:49:28.785515+0000","last_fullsized":"2024-03-23T17:49:28.785515+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:34:47.173411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.779841+0000","last_change":"2024-03-23T17:48:56.237042+0000","last_active":"2024-03-23T17:49:28.779841+0000","last_peered":"2024-03-23T17:49:28.779841+0000","last_clean":"2024-03-23T17:49:28.779841+0000","last_became_active":"2024-03-23T17:48:56.236722+0000","last_became_peered":"2024-03-23T17:48:56.236722+0000","last_unstale":"2024-03-23T17:49:28.779841+0000","last_undegraded":"2024-03-23T17:49:28.779841+0000","last_fullsized":"2024-03-23T17:49:28.779841+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:27:29.656120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.780900+0000","last_change":"2024-03-23T17:48:55.719223+0000","last_active":"2024-03-23T17:49:28.780900+0000","last_peered":"2024-03-23T17:49:28.780900+0000","last_clean":"2024-03-23T17:49:28.780900+0000","last_became_active":"2024-03-23T17:48:55.718864+0000","last_became_peered":"2024-03-23T17:48:55.718864+0000","last_unstale":"2024-03-23T17:49:28.780900+0000","last_undegraded":"2024-03-23T17:49:28.780900+0000","last_fullsized":"2024-03-23T17:49:28.780900+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:52:18.169557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.781086+0000","last_change":"2024-03-23T17:48:56.224885+0000","last_active":"2024-03-23T17:49:28.781086+0000","last_peered":"2024-03-23T17:49:28.781086+0000","last_clean":"2024-03-23T17:49:28.781086+0000","last_became_active":"2024-03-23T17:48:56.224160+0000","last_became_peered":"2024-03-23T17:48:56.224160+0000","last_unstale":"2024-03-23T17:49:28.781086+0000","last_undegraded":"2024-03-23T17:49:28.781086+0000","last_fullsized":"2024-03-23T17:49:28.781086+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:35:34.538313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'5","reported_seq":98,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-23T17:50:18.811876+0000","last_change":"2024-03-23T17:48:55.720518+0000","last_active":"2024-03-23T17:50:18.811876+0000","last_peered":"2024-03-23T17:50:18.811876+0000","last_clean":"2024-03-23T17:50:18.811876+0000","last_became_active":"2024-03-23T17:48:55.720045+0000","last_became_peered":"2024-03-23T17:48:55.720045+0000","last_unstale":"2024-03-23T17:50:18.811876+0000","last_undegraded":"2024-03-23T17:50:18.811876+0000","last_fullsized":"2024-03-23T17:50:18.811876+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:20:54.819406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":79,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.781093+0000","last_change":"2024-03-23T17:48:56.224171+0000","last_active":"2024-03-23T17:49:28.781093+0000","last_peered":"2024-03-23T17:49:28.781093+0000","last_clean":"2024-03-23T17:49:28.781093+0000","last_became_active":"2024-03-23T17:48:56.223737+0000","last_became_peered":"2024-03-23T17:48:56.223737+0000","last_unstale":"2024-03-23T17:49:28.781093+0000","last_undegraded":"2024-03-23T17:49:28.781093+0000","last_fullsized":"2024-03-23T17:49:28.781093+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:40:04.353945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'3","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.785531+0000","last_change":"2024-03-23T17:48:55.721734+0000","last_active":"2024-03-23T17:49:28.785531+0000","last_peered":"2024-03-23T17:49:28.785531+0000","last_clean":"2024-03-23T17:49:28.785531+0000","last_became_active":"2024-03-23T17:48:55.721417+0000","last_became_peered":"2024-03-23T17:48:55.721417+0000","last_unstale":"2024-03-23T17:49:28.785531+0000","last_undegraded":"2024-03-23T17:49:28.785531+0000","last_fullsized":"2024-03-23T17:49:28.785531+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:36:36.842740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'1","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.780817+0000","last_change":"2024-03-23T17:48:56.227211+0000","last_active":"2024-03-23T17:49:28.780817+0000","last_peered":"2024-03-23T17:49:28.780817+0000","last_clean":"2024-03-23T17:49:28.780817+0000","last_became_active":"2024-03-23T17:48:56.226881+0000","last_became_peered":"2024-03-23T17:48:56.226881+0000","last_unstale":"2024-03-23T17:49:28.780817+0000","last_undegraded":"2024-03-23T17:49:28.780817+0000","last_fullsized":"2024-03-23T17:49:28.780817+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:15:02.937666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.776701+0000","last_change":"2024-03-23T17:48:55.728679+0000","last_active":"2024-03-23T17:49:28.776701+0000","last_peered":"2024-03-23T17:49:28.776701+0000","last_clean":"2024-03-23T17:49:28.776701+0000","last_became_active":"2024-03-23T17:48:55.727754+0000","last_became_peered":"2024-03-23T17:48:55.727754+0000","last_unstale":"2024-03-23T17:49:28.776701+0000","last_undegraded":"2024-03-23T17:49:28.776701+0000","last_fullsized":"2024-03-23T17:49:28.776701+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:54.691736+0000","last_clean_scrub_stamp":"2024-03-23T17:48:54.691736+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:34:39.720488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.781328+0000","last_change":"2024-03-23T17:48:56.225214+0000","last_active":"2024-03-23T17:49:28.781328+0000","last_peered":"2024-03-23T17:49:28.781328+0000","last_clean":"2024-03-23T17:49:28.781328+0000","last_became_active":"2024-03-23T17:48:56.224635+0000","last_became_peered":"2024-03-23T17:48:56.224635+0000","last_unstale":"2024-03-23T17:49:28.781328+0000","last_undegraded":"2024-03-23T17:49:28.781328+0000","last_fullsized":"2024-03-23T17:49:28.781328+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:57:14.935434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.782109+0000","last_change":"2024-03-23T17:48:56.229835+0000","last_active":"2024-03-23T17:49:28.782109+0000","last_peered":"2024-03-23T17:49:28.782109+0000","last_clean":"2024-03-23T17:49:28.782109+0000","last_became_active":"2024-03-23T17:48:56.229437+0000","last_became_peered":"2024-03-23T17:48:56.229437+0000","last_unstale":"2024-03-23T17:49:28.782109+0000","last_undegraded":"2024-03-23T17:49:28.782109+0000","last_fullsized":"2024-03-23T17:49:28.782109+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:43:51.544867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.784779+0000","last_change":"2024-03-23T17:48:56.233552+0000","last_active":"2024-03-23T17:49:28.784779+0000","last_peered":"2024-03-23T17:49:28.784779+0000","last_clean":"2024-03-23T17:49:28.784779+0000","last_became_active":"2024-03-23T17:48:56.232941+0000","last_became_peered":"2024-03-23T17:48:56.232941+0000","last_unstale":"2024-03-23T17:49:28.784779+0000","last_undegraded":"2024-03-23T17:49:28.784779+0000","last_fullsized":"2024-03-23T17:49:28.784779+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:03:32.325496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.785402+0000","last_change":"2024-03-23T17:48:56.238238+0000","last_active":"2024-03-23T17:49:28.785402+0000","last_peered":"2024-03-23T17:49:28.785402+0000","last_clean":"2024-03-23T17:49:28.785402+0000","last_became_active":"2024-03-23T17:48:56.237882+0000","last_became_peered":"2024-03-23T17:48:56.237882+0000","last_unstale":"2024-03-23T17:49:28.785402+0000","last_undegraded":"2024-03-23T17:49:28.785402+0000","last_fullsized":"2024-03-23T17:49:28.785402+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:27:13.896205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.783881+0000","last_change":"2024-03-23T17:48:56.242895+0000","last_active":"2024-03-23T17:49:28.783881+0000","last_peered":"2024-03-23T17:49:28.783881+0000","last_clean":"2024-03-23T17:49:28.783881+0000","last_became_active":"2024-03-23T17:48:56.242515+0000","last_became_peered":"2024-03-23T17:48:56.242515+0000","last_unstale":"2024-03-23T17:49:28.783881+0000","last_undegraded":"2024-03-23T17:49:28.783881+0000","last_fullsized":"2024-03-23T17:49:28.783881+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:45:01.978843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.776603+0000","last_change":"2024-03-23T17:48:56.237029+0000","last_active":"2024-03-23T17:49:28.776603+0000","last_peered":"2024-03-23T17:49:28.776603+0000","last_clean":"2024-03-23T17:49:28.776603+0000","last_became_active":"2024-03-23T17:48:56.236692+0000","last_became_peered":"2024-03-23T17:48:56.236692+0000","last_unstale":"2024-03-23T17:49:28.776603+0000","last_undegraded":"2024-03-23T17:49:28.776603+0000","last_fullsized":"2024-03-23T17:49:28.776603+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:42:31.855469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.782393+0000","last_change":"2024-03-23T17:48:56.223236+0000","last_active":"2024-03-23T17:49:28.782393+0000","last_peered":"2024-03-23T17:49:28.782393+0000","last_clean":"2024-03-23T17:49:28.782393+0000","last_became_active":"2024-03-23T17:48:56.222824+0000","last_became_peered":"2024-03-23T17:48:56.222824+0000","last_unstale":"2024-03-23T17:49:28.782393+0000","last_undegraded":"2024-03-23T17:49:28.782393+0000","last_fullsized":"2024-03-23T17:49:28.782393+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:54:45.096969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.785731+0000","last_change":"2024-03-23T17:48:56.224471+0000","last_active":"2024-03-23T17:49:28.785731+0000","last_peered":"2024-03-23T17:49:28.785731+0000","last_clean":"2024-03-23T17:49:28.785731+0000","last_became_active":"2024-03-23T17:48:56.223140+0000","last_became_peered":"2024-03-23T17:48:56.223140+0000","last_unstale":"2024-03-23T17:49:28.785731+0000","last_undegraded":"2024-03-23T17:49:28.785731+0000","last_fullsized":"2024-03-23T17:49:28.785731+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:38:53.906820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.780461+0000","last_change":"2024-03-23T17:48:56.223564+0000","last_active":"2024-03-23T17:49:28.780461+0000","last_peered":"2024-03-23T17:49:28.780461+0000","last_clean":"2024-03-23T17:49:28.780461+0000","last_became_active":"2024-03-23T17:48:56.223217+0000","last_became_peered":"2024-03-23T17:48:56.223217+0000","last_unstale":"2024-03-23T17:49:28.780461+0000","last_undegraded":"2024-03-23T17:49:28.780461+0000","last_fullsized":"2024-03-23T17:49:28.780461+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:43:53.250082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.785081+0000","last_change":"2024-03-23T17:48:56.226283+0000","last_active":"2024-03-23T17:49:28.785081+0000","last_peered":"2024-03-23T17:49:28.785081+0000","last_clean":"2024-03-23T17:49:28.785081+0000","last_became_active":"2024-03-23T17:48:56.224211+0000","last_became_peered":"2024-03-23T17:48:56.224211+0000","last_unstale":"2024-03-23T17:49:28.785081+0000","last_undegraded":"2024-03-23T17:49:28.785081+0000","last_fullsized":"2024-03-23T17:49:28.785081+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:15:20.518950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.779571+0000","last_change":"2024-03-23T17:48:56.223557+0000","last_active":"2024-03-23T17:49:28.779571+0000","last_peered":"2024-03-23T17:49:28.779571+0000","last_clean":"2024-03-23T17:49:28.779571+0000","last_became_active":"2024-03-23T17:48:56.222792+0000","last_became_peered":"2024-03-23T17:48:56.222792+0000","last_unstale":"2024-03-23T17:49:28.779571+0000","last_undegraded":"2024-03-23T17:49:28.779571+0000","last_fullsized":"2024-03-23T17:49:28.779571+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:49:17.587817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.922230+0000","last_change":"2024-03-23T17:48:56.236041+0000","last_active":"2024-03-23T17:49:28.922230+0000","last_peered":"2024-03-23T17:49:28.922230+0000","last_clean":"2024-03-23T17:49:28.922230+0000","last_became_active":"2024-03-23T17:48:56.234259+0000","last_became_peered":"2024-03-23T17:48:56.234259+0000","last_unstale":"2024-03-23T17:49:28.922230+0000","last_undegraded":"2024-03-23T17:49:28.922230+0000","last_fullsized":"2024-03-23T17:49:28.922230+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:32:43.786783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.777528+0000","last_change":"2024-03-23T17:48:56.239283+0000","last_active":"2024-03-23T17:49:28.777528+0000","last_peered":"2024-03-23T17:49:28.777528+0000","last_clean":"2024-03-23T17:49:28.777528+0000","last_became_active":"2024-03-23T17:48:56.238952+0000","last_became_peered":"2024-03-23T17:48:56.238952+0000","last_unstale":"2024-03-23T17:49:28.777528+0000","last_undegraded":"2024-03-23T17:49:28.777528+0000","last_fullsized":"2024-03-23T17:49:28.777528+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:31:50.383284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-03-23T17:49:28.782143+0000","last_change":"2024-03-23T17:48:56.226684+0000","last_active":"2024-03-23T17:49:28.782143+0000","last_peered":"2024-03-23T17:49:28.782143+0000","last_clean":"2024-03-23T17:49:28.782143+0000","last_became_active":"2024-03-23T17:48:56.226367+0000","last_became_peered":"2024-03-23T17:48:56.226367+0000","last_unstale":"2024-03-23T17:49:28.782143+0000","last_undegraded":"2024-03-23T17:49:28.782143+0000","last_fullsized":"2024-03-23T17:49:28.782143+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T17:48:55.696607+0000","last_clean_scrub_stamp":"2024-03-23T17:48:55.696607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:12:29.053330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70574,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":819,"num_read_kb":481,"num_write":235,"num_write_kb":138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":180,"num_keys_recovered":17,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":211722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":206,"ondisk_log_size":206,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547229,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31992,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713160,"statfs":{"total":95995035648,"available":95962275840,"internally_reserved":0,"allocated":897024,"data_stored":428738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31848901},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 17:50:19 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.53200000000000003,"15min":0.53900000000000003},"min":{"1min":0.38700000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.64300000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.60299999999999998,"15min":0.60899999999999999},"min":{"1min":0.435,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.73599999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.48399999999999999}]},{"osd":1,"last update":"Sat Mar 23 17:50:19 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.51600000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.749,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.63700000000000001,"15min":0.63600000000000001},"min":{"1min":0.503,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.85399999999999998,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.74099999999999999}]},{"osd":2,"last update":"Sat Mar 23 17:50:19 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.503,"15min":0.501},"min":{"1min":0.35399999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.47899999999999998,"15min":0.47299999999999998},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.51400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":4,"last update":"Sat Mar 23 17:50:19 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.64300000000000002,"15min":0.64500000000000002},"min":{"1min":0.44800000000000001,"5min":0.438,"15min":0.438},"max":{"1min":0.75700000000000001,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.66300000000000003,"15min":0.66100000000000003},"min":{"1min":0.505,"5min":0.441,"15min":0.441},"max":{"1min":0.83599999999999997,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.79000000000000004}]},{"osd":5,"last update":"Sat Mar 23 17:50:19 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.59599999999999997,"15min":0.58899999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.621,"5min":0.58899999999999997,"15min":0.58399999999999996},"min":{"1min":0.436,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.81299999999999994,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.61799999999999999}]},{"osd":6,"last update":"Sat Mar 23 17:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.84499999999999997}]},{"osd":7,"last update":"Sat Mar 23 17:50:19 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.71999999999999997,"15min":0.71899999999999997},"min":{"1min":0.54800000000000004,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.86399999999999999,"5min":1.002,"15min":1.002},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.753,"15min":0.75600000000000001},"min":{"1min":0.59299999999999997,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.89500000000000002,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.69299999999999995}]},{"osd":8,"last update":"Sat Mar 23 17:49:56 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.82999999999999996}]},{"osd":9,"last update":"Sat Mar 23 17:50:19 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.63200000000000001,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.93899999999999995,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.76300000000000001,"15min":0.76600000000000001},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.90200000000000002,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.80000000000000004}]},{"osd":10,"last update":"Sat Mar 23 17:50:19 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.78500000000000003,"15min":0.78600000000000003},"min":{"1min":0.66100000000000003,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.95299999999999996,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.79500000000000004,"15min":0.79400000000000004},"min":{"1min":0.69299999999999995,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.97299999999999998,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.86799999999999999}]}]},{"osd":10,"up_from":68,"seq":292057776159,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28568,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716584,"statfs":{"total":95995035648,"available":95965782016,"internally_reserved":0,"allocated":1585152,"data_stored":1078786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 17:49:56 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.438}]},{"osd":1,"last update":"Sat Mar 23 17:50:02 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.47099999999999997,"15min":0.47899999999999998},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.59599999999999997,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.44,"15min":0.44500000000000001},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.57199999999999995,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.41599999999999998}]},{"osd":2,"last update":"Sat Mar 23 17:50:02 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.58899999999999997,"15min":0.59399999999999997},"min":{"1min":0.41299999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.746,"5min":0.755,"15min":0.755},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.629,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.80700000000000005,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.623}]},{"osd":3,"last update":"Sat Mar 23 17:50:02 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.60899999999999999,"15min":0.61699999999999999},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.70999999999999996,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.626,"15min":0.63600000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.79000000000000004,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.58499999999999996}]},{"osd":4,"last update":"Sat Mar 23 17:50:02 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.56499999999999995,"15min":0.56899999999999995},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.72399999999999998,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.53900000000000003,"15min":0.55000000000000004},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.64900000000000002,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.55000000000000004}]},{"osd":5,"last update":"Sat Mar 23 17:50:02 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.64100000000000001,"15min":0.65100000000000002},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.77900000000000003,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.69899999999999995,"15min":0.70699999999999996},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.80500000000000005,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.72599999999999998}]},{"osd":6,"last update":"Sat Mar 23 17:50:02 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.68999999999999995,"15min":0.69599999999999995},"min":{"1min":0.47099999999999997,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.85099999999999998,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.71399999999999997,"15min":0.72599999999999998},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.82099999999999995,"5min":1.071,"15min":1.071},"last":0.70499999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":8,"last update":"Sat Mar 23 17:50:02 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.73899999999999999,"15min":0.748},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.86399999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.748,"15min":0.76000000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.82899999999999996,"5min":0.996,"15min":0.996},"last":0.67200000000000004}]},{"osd":9,"last update":"Sat Mar 23 17:50:02 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.76000000000000001,"15min":0.77000000000000002},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.92600000000000005,"5min":1.194,"15min":1.194},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.755,"15min":0.76900000000000002},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.879,"5min":1.042,"15min":1.042},"last":0.71499999999999997}]},{"osd":11,"last update":"Sat Mar 23 17:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.77500000000000002}]}]},{"osd":9,"up_from":63,"seq":270582939685,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36108,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709044,"statfs":{"total":95995035648,"available":95958061056,"internally_reserved":0,"allocated":913408,"data_stored":428648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":36043201},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 17:49:44 2024","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.54200000000000004,"15min":0.56200000000000006},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.64400000000000002,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.58499999999999996,"15min":0.60699999999999998},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.59999999999999998,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.51900000000000002}]},{"osd":1,"last update":"Sat Mar 23 17:49:44 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.78500000000000003,"15min":0.80500000000000005},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.70699999999999996,"15min":0.72499999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.78600000000000003,"5min":1.091,"15min":1.091},"last":0.45500000000000002}]},{"osd":2,"last update":"Sat Mar 23 17:49:44 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.78600000000000003,"15min":0.80100000000000005},"min":{"1min":0.47299999999999998,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.871,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.73599999999999999,"15min":0.752},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.97399999999999998,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.77700000000000002}]},{"osd":3,"last update":"Sat Mar 23 17:49:44 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.61499999999999999,"15min":0.63300000000000001},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.77900000000000003,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.71899999999999997,"15min":0.73699999999999999},"min":{"1min":0.44900000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.88900000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.79900000000000004}]},{"osd":4,"last update":"Sat Mar 23 17:48:42 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.61099999999999999}]},{"osd":5,"last update":"Sat Mar 23 17:49:44 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.73199999999999998,"15min":0.74199999999999999},"min":{"1min":0.54700000000000004,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.1180000000000001,"5min":1.167,"15min":1.167},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.85099999999999998,"15min":0.87},"min":{"1min":0.505,"5min":0.437,"15min":0.437},"max":{"1min":0.93100000000000005,"5min":1.48,"15min":1.48},"last":0.85799999999999998}]},{"osd":6,"last update":"Sat Mar 23 17:49:44 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.83099999999999996,"15min":0.85499999999999998},"min":{"1min":0.47099999999999997,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.877,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.78600000000000003,"15min":0.81200000000000006},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.85599999999999998,"5min":1.153,"15min":1.153},"last":0.69699999999999995}]},{"osd":7,"last update":"Sat Mar 23 17:49:44 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.81599999999999995,"15min":0.84399999999999997},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.78000000000000003,"5min":1.216,"15min":1.216},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.92200000000000004,"15min":0.94499999999999995},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.2969999999999999,"5min":1.373,"15min":1.373},"last":0.90400000000000003}]},{"osd":8,"last update":"Sat Mar 23 17:49:44 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.85999999999999999,"15min":0.878},"min":{"1min":0.57499999999999996,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.88200000000000001,"15min":0.90800000000000003},"min":{"1min":0.59099999999999997,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.95499999999999996,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.59699999999999998}]},{"osd":10,"last update":"Sat Mar 23 17:50:00 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.79500000000000004,"15min":0.81699999999999995},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.80300000000000005,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.89400000000000002,"15min":0.91300000000000003},"min":{"1min":0.67200000000000004,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":0.94299999999999995,"5min":1.339,"15min":1.339},"last":0.88200000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]}]},{"osd":8,"up_from":56,"seq":240518168615,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":429026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 17:50:23 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.57099999999999995,"15min":0.57899999999999996},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.79600000000000004,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55100000000000005,"15min":0.55200000000000005},"min":{"1min":0.38300000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.55600000000000005}]},{"osd":1,"last update":"Sat Mar 23 17:50:23 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.60799999999999998,"15min":0.59199999999999997},"min":{"1min":0.51800000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.60599999999999998,"15min":0.59899999999999998},"min":{"1min":0.44600000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.69399999999999995}]},{"osd":2,"last update":"Sat Mar 23 17:50:23 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.46899999999999997,"15min":0.45900000000000002},"min":{"1min":0.38300000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.71399999999999997,"5min":0.874,"15min":0.874},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.47499999999999998,"15min":0.46500000000000002},"min":{"1min":0.34399999999999997,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.747,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.65400000000000003}]},{"osd":3,"last update":"Sat Mar 23 17:50:23 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.58699999999999997,"15min":0.59099999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64800000000000002,"15min":0.66000000000000003},"min":{"1min":0.434,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.86299999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.60199999999999998}]},{"osd":4,"last update":"Sat Mar 23 17:50:23 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.61599999999999999,"15min":0.59499999999999997},"min":{"1min":0.55700000000000005,"5min":0.308,"15min":0.308},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.65400000000000003,"15min":0.64000000000000001},"min":{"1min":0.51300000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.82799999999999996}]},{"osd":5,"last update":"Sat Mar 23 17:50:23 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.55000000000000004,"15min":0.54900000000000004},"min":{"1min":0.45700000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.84599999999999997,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.56999999999999995,"15min":0.56000000000000005},"min":{"1min":0.39300000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.91800000000000004,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.61699999999999999}]},{"osd":6,"last update":"Sat Mar 23 17:50:23 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70299999999999996,"15min":0.70999999999999996},"min":{"1min":0.48199999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.94199999999999995,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.68700000000000006,"15min":0.69099999999999995},"min":{"1min":0.55300000000000005,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.89400000000000002,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.58699999999999997}]},{"osd":7,"last update":"Sat Mar 23 17:50:23 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.69299999999999995,"15min":0.68300000000000005},"min":{"1min":0.55300000000000005,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.71699999999999997,"15min":0.70399999999999996},"min":{"1min":0.60099999999999998,"5min":0.438,"15min":0.438},"max":{"1min":0.99199999999999999,"5min":1.04,"15min":1.04},"last":0.88800000000000001}]},{"osd":9,"last update":"Sat Mar 23 17:49:46 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.72899999999999998,"15min":0.73299999999999998},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.875,"5min":1.026,"15min":1.026},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.745,"5min":0.76100000000000001,"15min":0.76400000000000001},"min":{"1min":0.55500000000000005,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.94599999999999995,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.875}]},{"osd":10,"last update":"Sat Mar 23 17:50:02 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.72599999999999998,"15min":0.71599999999999997},"min":{"1min":0.61899999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.74299999999999999,"15min":0.73099999999999998},"min":{"1min":0.63600000000000001,"5min":0.496,"15min":0.496},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.93400000000000005}]},{"osd":11,"last update":"Sat Mar 23 17:49:57 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.91400000000000003}]}]},{"osd":7,"up_from":49,"seq":210453397547,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28488,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716664,"statfs":{"total":95995035648,"available":95965863936,"internally_reserved":0,"allocated":1507328,"data_stored":1021315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 17:48:02 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.54800000000000004}]},{"osd":1,"last update":"Sat Mar 23 17:50:08 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.39700000000000002},"min":{"1min":0.31,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.58499999999999996,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.434,"5min":0.40500000000000003,"15min":0.38200000000000001},"min":{"1min":0.314,"5min":0.248,"15min":0.248},"max":{"1min":0.64000000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.621}]},{"osd":2,"last update":"Sat Mar 23 17:50:08 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.57399999999999995,"15min":0.55800000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.57299999999999995,"15min":0.55900000000000005},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1,"5min":1,"15min":1},"last":0.64500000000000002}]},{"osd":3,"last update":"Sat Mar 23 17:50:08 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.55800000000000005,"15min":0.54700000000000004},"min":{"1min":0.41899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.59899999999999998,"15min":0.59099999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.77500000000000002,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.82899999999999996}]},{"osd":4,"last update":"Sat Mar 23 17:50:08 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.495,"15min":0.47899999999999998},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.72899999999999998,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.47699999999999998,"15min":0.47099999999999997},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.63700000000000001,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.69499999999999995}]},{"osd":5,"last update":"Sat Mar 23 17:50:08 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.60699999999999998,"15min":0.58999999999999997},"min":{"1min":0.46899999999999997,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.80400000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.58999999999999997,"15min":0.57799999999999996},"min":{"1min":0.41499999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.78100000000000003}]},{"osd":6,"last update":"Sat Mar 23 17:50:08 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.67300000000000004,"15min":0.66700000000000004},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.83899999999999997,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.879},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65200000000000002,"15min":0.65100000000000002},"min":{"1min":0.45000000000000001,"5min":0.433,"15min":0.433},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.80600000000000005}]},{"osd":8,"last update":"Sat Mar 23 17:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.64900000000000002,"15min":0.63300000000000001},"min":{"1min":0.47799999999999998,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.66600000000000004,"15min":0.64700000000000002},"min":{"1min":0.52700000000000002,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.92300000000000004}]},{"osd":9,"last update":"Sat Mar 23 17:49:43 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.71299999999999997,"15min":0.71699999999999997},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.88200000000000001,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.69299999999999995,"15min":0.69599999999999995},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.84499999999999997,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.84799999999999998}]},{"osd":10,"last update":"Sat Mar 23 17:50:02 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.73199999999999998,"15min":0.73999999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.92000000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.70499999999999996,"15min":0.70999999999999996},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.85999999999999999}]},{"osd":11,"last update":"Sat Mar 23 17:49:58 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.93500000000000005}]}]},{"osd":6,"up_from":45,"seq":193273528367,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36088,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93709064,"statfs":{"total":95995035648,"available":95958081536,"internally_reserved":0,"allocated":892928,"data_stored":428044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 17:49:47 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47499999999999998,"15min":0.47599999999999998},"min":{"1min":0.32200000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.51700000000000002,"15min":0.52100000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.48899999999999999}]},{"osd":1,"last update":"Sat Mar 23 17:49:47 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.59799999999999998,"15min":0.59399999999999997},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.83899999999999997,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.46999999999999997,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.95699999999999996,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.71799999999999997}]},{"osd":2,"last update":"Sat Mar 23 17:49:47 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.44900000000000001,"5min":0.374,"15min":0.374},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67200000000000004,"15min":0.67700000000000005},"min":{"1min":0.53000000000000003,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.88200000000000001,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.66400000000000003}]},{"osd":3,"last update":"Sat Mar 23 17:49:47 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.59799999999999998,"15min":0.61399999999999999},"min":{"1min":0.35399999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.878,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.65700000000000003,"15min":0.66300000000000003},"min":{"1min":0.376,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.1140000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.64600000000000002}]},{"osd":4,"last update":"Sat Mar 23 17:49:47 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.65200000000000002,"15min":0.65600000000000003},"min":{"1min":0.42999999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.71799999999999997,"15min":0.73299999999999998},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.90500000000000003,"5min":0.998,"15min":0.998},"last":0.69999999999999996}]},{"osd":5,"last update":"Sat Mar 23 17:49:47 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.77400000000000002,"15min":0.78100000000000003},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.77000000000000002,"15min":0.78200000000000003},"min":{"1min":0.49299999999999999,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.504,"5min":1.504,"15min":1.504},"last":0.69199999999999995}]},{"osd":7,"last update":"Sat Mar 23 17:50:08 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.745,"15min":0.74199999999999999},"min":{"1min":0.54900000000000004,"5min":0.5,"15min":0.5},"max":{"1min":1.448,"5min":1.448,"15min":1.448},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.749,"15min":0.77900000000000003},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.4339999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.621}]},{"osd":8,"last update":"Sat Mar 23 17:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.85599999999999998,"15min":0.876},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.92400000000000004,"5min":1.085,"15min":1.085},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.81499999999999995,"15min":0.82399999999999995},"min":{"1min":0.63200000000000001,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":0.91200000000000003,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.77600000000000002}]},{"osd":9,"last update":"Sat Mar 23 17:49:47 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.80500000000000005,"15min":0.80800000000000005},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.78400000000000003,"15min":0.79400000000000004},"min":{"1min":0.48399999999999999,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.93899999999999995,"5min":1.099,"15min":1.099},"last":0.81399999999999995}]},{"osd":10,"last update":"Sat Mar 23 17:50:02 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.80800000000000005,"15min":0.80200000000000005},"min":{"1min":0.67900000000000005,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.80300000000000005,"15min":0.80700000000000005},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.573,"5min":1.573,"15min":1.573},"last":0.83899999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":493574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 17:50:21 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53100000000000003,"15min":0.51500000000000001},"min":{"1min":0.41499999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.82299999999999995,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.63700000000000001,"15min":0.53500000000000003},"min":{"1min":0.39800000000000002,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.68200000000000005,"5min":5.8399999999999999,"15min":5.8399999999999999},"last":0.47699999999999998}]},{"osd":2,"last update":"Sat Mar 23 17:49:29 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.52700000000000002,"15min":0.48099999999999998},"min":{"1min":0.46600000000000003,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.81399999999999995,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.52200000000000002,"15min":0.49299999999999999},"min":{"1min":0.45900000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.77400000000000002,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.56899999999999995}]},{"osd":3,"last update":"Sat Mar 23 17:49:52 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55600000000000005,"15min":0.53400000000000003},"min":{"1min":0.38100000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.73799999999999999,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.55300000000000005,"15min":0.53900000000000003},"min":{"1min":0.38400000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.69199999999999995,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.503}]},{"osd":4,"last update":"Sat Mar 23 17:50:10 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.44400000000000001,"15min":0.40600000000000003},"min":{"1min":0.34300000000000003,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.59699999999999998,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.442,"15min":0.40300000000000002},"min":{"1min":0.36399999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.69799999999999995,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.39000000000000001}]},{"osd":5,"last update":"Sat Mar 23 17:49:29 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.59499999999999997,"15min":0.56299999999999994},"min":{"1min":0.55500000000000005,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.86599999999999999,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.61199999999999999,"15min":0.57799999999999996},"min":{"1min":0.55600000000000005,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.85399999999999998,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.71199999999999997}]},{"osd":6,"last update":"Sat Mar 23 17:49:45 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.59999999999999998,"15min":0.57299999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.80400000000000005,"5min":0.998,"15min":0.998},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.63600000000000001,"15min":0.621},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.78800000000000003,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.64100000000000001}]},{"osd":7,"last update":"Sat Mar 23 17:50:06 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53200000000000003,"15min":0.51300000000000001},"min":{"1min":0.38200000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.66000000000000003,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.372},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.57299999999999995,"15min":0.54100000000000004},"min":{"1min":0.39400000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.73399999999999999,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.65900000000000003}]},{"osd":8,"last update":"Sat Mar 23 17:50:21 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.69499999999999995,"15min":0.67300000000000004},"min":{"1min":0.57499999999999996,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.88,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.64800000000000002,"15min":0.63400000000000001},"min":{"1min":0.505,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.80600000000000005,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.59799999999999998}]},{"osd":9,"last update":"Sat Mar 23 17:49:45 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.75600000000000001,"15min":0.76200000000000001},"min":{"1min":0.48699999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.85399999999999998,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.63},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.78900000000000003,"15min":0.79900000000000004},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.90300000000000002,"5min":1.103,"15min":1.103},"last":0.76200000000000001}]},{"osd":10,"last update":"Sat Mar 23 17:50:00 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.70699999999999996,"15min":0.71399999999999997},"min":{"1min":0.56599999999999995,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.84199999999999997,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.76300000000000001,"15min":0.76800000000000002},"min":{"1min":0.46100000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.89100000000000001,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.747}]},{"osd":11,"last update":"Sat Mar 23 17:50:00 2024","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.78600000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738438,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":913408,"data_stored":429383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 23 17:50:17 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.61399999999999999,"15min":0.71399999999999997},"min":{"1min":0.38400000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.89800000000000002,"5min":6.2110000000000003,"15min":6.2110000000000003},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.628,"5min":0.56999999999999995,"15min":0.51600000000000001},"min":{"1min":0.41399999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.58099999999999996}]},{"osd":2,"last update":"Sat Mar 23 17:49:31 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.57799999999999996,"15min":0.55100000000000005},"min":{"1min":0.38100000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.82999999999999996,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.59999999999999998,"15min":0.56000000000000005},"min":{"1min":0.41299999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.751}]},{"osd":3,"last update":"Sat Mar 23 17:49:58 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.45800000000000002}]},{"osd":4,"last update":"Sat Mar 23 17:50:17 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.64700000000000002,"15min":0.625},"min":{"1min":0.45500000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.621},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.61099999999999999,"15min":0.59199999999999997},"min":{"1min":0.46100000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.873,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.79800000000000004}]},{"osd":5,"last update":"Sat Mar 23 17:49:28 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65700000000000003,"15min":0.63700000000000001},"min":{"1min":0.44700000000000001,"5min":0.44,"15min":0.44},"max":{"1min":1.018,"5min":1.167,"15min":1.167},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.64500000000000002,"15min":0.61599999999999999},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.65500000000000003}]},{"osd":6,"last update":"Sat Mar 23 17:49:48 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60099999999999998},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.68000000000000005,"15min":0.66800000000000004},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.0509999999999999,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.63800000000000001}]},{"osd":7,"last update":"Sat Mar 23 17:48:02 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.495},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.58599999999999997}]},{"osd":8,"last update":"Sat Mar 23 17:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.83299999999999996,"15min":0.84899999999999998},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.091,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.81299999999999994,"15min":0.82599999999999996},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.68100000000000005}]},{"osd":9,"last update":"Sat Mar 23 17:49:43 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.76900000000000002,"15min":0.77300000000000002},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.82899999999999996,"15min":0.83699999999999997},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.119,"5min":1.286,"15min":1.286},"last":0.78700000000000003}]},{"osd":10,"last update":"Sat Mar 23 17:50:04 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.81499999999999995,"15min":0.81100000000000005},"min":{"1min":0.59599999999999997,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.421,"5min":1.421,"15min":1.421},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.90000000000000002}]},{"osd":11,"last update":"Sat Mar 23 17:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.93400000000000005}]}]},{"osd":2,"up_from":18,"seq":77309411390,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36148,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709004,"statfs":{"total":95995035648,"available":95958020096,"internally_reserved":0,"allocated":962560,"data_stored":492987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 17:49:30 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.54000000000000004,"15min":0.51400000000000001},"min":{"1min":0.42199999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.5},{"interface":"front","average":{"1min":0.63,"5min":0.57899999999999996,"15min":0.55300000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.88700000000000001,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.35699999999999998}]},{"osd":1,"last update":"Sat Mar 23 17:49:30 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.54500000000000004,"15min":0.5},"min":{"1min":0.42199999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.80900000000000005,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.55800000000000005,"15min":0.51800000000000002},"min":{"1min":0.41999999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.79300000000000004,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.68600000000000005}]},{"osd":3,"last update":"Sat Mar 23 17:49:56 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.55900000000000005,"15min":0.53300000000000003},"min":{"1min":0.38200000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.438},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58899999999999997,"15min":0.56699999999999995},"min":{"1min":0.40300000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.82199999999999995,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.57299999999999995}]},{"osd":4,"last update":"Sat Mar 23 17:49:56 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.63300000000000001}]},{"osd":5,"last update":"Sat Mar 23 17:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.48299999999999998,"15min":0.45900000000000002},"min":{"1min":0.32600000000000001,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.495,"15min":0.46999999999999997},"min":{"1min":0.34100000000000003,"5min":0.308,"15min":0.308},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.61199999999999999}]},{"osd":6,"last update":"Sat Mar 23 17:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.63700000000000001,"15min":0.622},"min":{"1min":0.505,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.80800000000000005,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.65700000000000003,"15min":0.64200000000000002},"min":{"1min":0.51100000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.85099999999999998,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.52100000000000002}]},{"osd":7,"last update":"Sat Mar 23 17:48:01 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.745}]},{"osd":8,"last update":"Sat Mar 23 17:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.51000000000000001,"15min":0.50600000000000001},"min":{"1min":0.35599999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.59399999999999997,"15min":0.58599999999999997},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.78100000000000003,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.56000000000000005}]},{"osd":9,"last update":"Sat Mar 23 17:49:45 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.70399999999999996,"15min":0.69999999999999996},"min":{"1min":0.49399999999999999,"5min":0.433,"15min":0.433},"max":{"1min":0.84099999999999997,"5min":0.879,"15min":0.879},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.77300000000000002,"15min":0.77400000000000002},"min":{"1min":0.68400000000000005,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.94899999999999995,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.73599999999999999}]},{"osd":10,"last update":"Sat Mar 23 17:50:06 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68200000000000005,"15min":0.68000000000000005},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.79300000000000004,"15min":0.79100000000000004},"min":{"1min":0.68000000000000005,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.80800000000000005}]},{"osd":11,"last update":"Sat Mar 23 17:49:56 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.65900000000000003}]}]},{"osd":3,"up_from":26,"seq":111669149754,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32008,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713144,"statfs":{"total":95995035648,"available":95962259456,"internally_reserved":0,"allocated":909312,"data_stored":427917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 17:50:02 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.44,"15min":0.40200000000000002},"min":{"1min":0.32000000000000001,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.44500000000000001,"15min":0.40699999999999997},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.85899999999999999,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.39600000000000002}]},{"osd":1,"last update":"Sat Mar 23 17:50:02 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.58999999999999997,"15min":0.53100000000000003},"min":{"1min":0.38700000000000001,"5min":0.371,"15min":0.371},"max":{"1min":2.0649999999999999,"5min":2.0649999999999999,"15min":2.0649999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.57599999999999996,"15min":0.51900000000000002},"min":{"1min":0.48299999999999998,"5min":0.371,"15min":0.371},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.751}]},{"osd":2,"last update":"Sat Mar 23 17:50:02 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.629,"15min":0.58899999999999997},"min":{"1min":0.434,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.96699999999999997,"5min":1.02,"15min":1.02},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.64900000000000002,"15min":0.61899999999999999},"min":{"1min":0.47299999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":2.141,"5min":2.141,"15min":2.141},"last":0.61299999999999999}]},{"osd":4,"last update":"Sat Mar 23 17:50:12 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.67700000000000005,"15min":0.624},"min":{"1min":0.55300000000000005,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":2.2080000000000002,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.65900000000000003,"15min":0.621},"min":{"1min":0.45200000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.73099999999999998}]},{"osd":5,"last update":"Sat Mar 23 17:49:31 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.67800000000000005,"15min":0.65000000000000002},"min":{"1min":0.53500000000000003,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.01,"5min":1.036,"15min":1.036},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.70099999999999996,"15min":0.67300000000000004},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.56599999999999995}]},{"osd":6,"last update":"Sat Mar 23 17:49:45 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.66700000000000004,"15min":0.65100000000000002},"min":{"1min":0.41099999999999998,"5min":0.38,"15min":0.38},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.68000000000000005,"15min":0.66800000000000004},"min":{"1min":0.46300000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.64500000000000002}]},{"osd":7,"last update":"Sat Mar 23 17:50:08 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.75600000000000001,"15min":0.76500000000000001},"min":{"1min":0.54500000000000004,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.747,"15min":0.71999999999999997},"min":{"1min":0.55200000000000005,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":2.1720000000000002,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":0.89900000000000002}]},{"osd":8,"last update":"Sat Mar 23 17:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.88},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.85099999999999998}]},{"osd":9,"last update":"Sat Mar 23 17:49:45 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.78500000000000003,"15min":0.77300000000000002},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":2.2370000000000001,"5min":2.2370000000000001,"15min":2.2370000000000001},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.78400000000000003,"15min":0.78000000000000003},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.625}]},{"osd":10,"last update":"Sat Mar 23 17:50:03 2024","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.79600000000000004,"15min":0.78700000000000003},"min":{"1min":0.65400000000000003,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.46,"5min":1.46,"15min":1.46},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.85499999999999998,"15min":0.84099999999999997},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":2.6349999999999998,"5min":2.6349999999999998,"15min":2.6349999999999998},"last":0.82999999999999996}]},{"osd":11,"last update":"Sat Mar 23 17:49:57 2024","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":2.6819999999999999,"5min":2.6819999999999999,"15min":2.6819999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.51,"5min":1.51,"15min":1.51},"last":0.68100000000000005}]}]},{"osd":4,"up_from":33,"seq":141733920823,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":901120,"data_stored":428017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 17:48:06 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.53700000000000003,"15min":0.53400000000000003},"min":{"1min":0.41799999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.77600000000000002,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.53700000000000003,"15min":0.52500000000000002},"min":{"1min":0.45300000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.52000000000000002}]},{"osd":1,"last update":"Sat Mar 23 17:50:07 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.36799999999999999,"15min":0.33100000000000002},"min":{"1min":0.27500000000000002,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.62,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.38400000000000001,"15min":0.34899999999999998},"min":{"1min":0.28999999999999998,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.57399999999999995,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.52000000000000002}]},{"osd":2,"last update":"Sat Mar 23 17:50:07 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.57999999999999996,"15min":0.55900000000000005},"min":{"1min":0.379,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.83499999999999996,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.58599999999999997,"15min":0.55200000000000005},"min":{"1min":0.441,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.81100000000000005,"5min":0.88,"15min":0.88},"last":0.78200000000000003}]},{"osd":3,"last update":"Sat Mar 23 17:50:07 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57599999999999996,"15min":0.56100000000000005},"min":{"1min":0.46600000000000003,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.73999999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.56499999999999995,"15min":0.56200000000000006},"min":{"1min":0.39400000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.754,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.504}]},{"osd":5,"last update":"Sat Mar 23 17:49:29 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.57999999999999996,"15min":0.56599999999999995},"min":{"1min":0.41299999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.80600000000000005,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.629,"5min":0.59499999999999997,"15min":0.57799999999999996},"min":{"1min":0.44600000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.77700000000000002,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.82799999999999996}]},{"osd":6,"last update":"Sat Mar 23 17:49:53 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.73699999999999999,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59199999999999997,"15min":0.58999999999999997},"min":{"1min":0.41799999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.79700000000000004,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.54900000000000004}]},{"osd":7,"last update":"Sat Mar 23 17:50:03 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.55300000000000005,"15min":0.54300000000000004},"min":{"1min":0.39300000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.55300000000000005,"15min":0.53900000000000003},"min":{"1min":0.36599999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.80900000000000005}]},{"osd":8,"last update":"Sat Mar 23 17:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.65800000000000003,"15min":0.65400000000000003},"min":{"1min":0.501,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.75},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.71699999999999997,"15min":0.71799999999999997},"min":{"1min":0.48599999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.86599999999999999,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.90700000000000003}]},{"osd":9,"last update":"Sat Mar 23 17:49:43 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.71799999999999997,"15min":0.71599999999999997},"min":{"1min":0.60599999999999998,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.54500000000000004,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.65200000000000002}]},{"osd":10,"last update":"Sat Mar 23 17:50:03 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.69899999999999995,"15min":0.69499999999999995},"min":{"1min":0.52600000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.86599999999999999,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.87},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.68700000000000006,"15min":0.67900000000000005},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.89300000000000002}]},{"osd":11,"last update":"Sat Mar 23 17:49:59 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.92900000000000005}]}]},{"osd":5,"up_from":38,"seq":163208757299,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32600,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712552,"statfs":{"total":95995035648,"available":95961653248,"internally_reserved":0,"allocated":1515520,"data_stored":1020452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 17:49:28 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.54900000000000004,"15min":0.54100000000000004},"min":{"1min":0.435,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.54400000000000004,"15min":0.54800000000000004},"min":{"1min":0.36299999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.83499999999999996,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.435}]},{"osd":1,"last update":"Sat Mar 23 17:49:28 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.55100000000000005,"15min":0.52700000000000002},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.58799999999999997,"15min":0.56499999999999995},"min":{"1min":0.495,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.67200000000000004}]},{"osd":2,"last update":"Sat Mar 23 17:49:28 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.40400000000000003,"15min":0.39400000000000002},"min":{"1min":0.311,"5min":0.251,"15min":0.251},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.503,"5min":0.41799999999999998,"15min":0.40100000000000002},"min":{"1min":0.30399999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.374}]},{"osd":3,"last update":"Sat Mar 23 17:49:28 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58299999999999996,"15min":0.58499999999999996},"min":{"1min":0.38100000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.80100000000000005,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.57699999999999996,"15min":0.58499999999999996},"min":{"1min":0.40899999999999997,"5min":0.373,"15min":0.373},"max":{"1min":0.71899999999999997,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.41999999999999998}]},{"osd":4,"last update":"Sat Mar 23 17:49:28 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.60599999999999998,"15min":0.58199999999999996},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.89500000000000002,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.58199999999999996,"15min":0.56000000000000005},"min":{"1min":0.44400000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.877,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.64700000000000002}]},{"osd":6,"last update":"Sat Mar 23 17:49:50 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.64000000000000001,"15min":0.629},"min":{"1min":0.55100000000000005,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.61499999999999999,"15min":0.61199999999999999},"min":{"1min":0.436,"5min":0.434,"15min":0.434},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.47399999999999998}]},{"osd":7,"last update":"Sat Mar 23 17:50:06 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.67800000000000005,"15min":0.65200000000000002},"min":{"1min":0.623,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.66100000000000003,"15min":0.63900000000000001},"min":{"1min":0.58399999999999996,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.66100000000000003}]},{"osd":8,"last update":"Sat Mar 23 17:50:21 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.624,"15min":0.59799999999999998},"min":{"1min":0.45000000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.57699999999999996,"15min":0.56000000000000005},"min":{"1min":0.38800000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.59999999999999998}]},{"osd":9,"last update":"Sat Mar 23 17:49:44 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.91600000000000004,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.70599999999999996,"15min":0.70099999999999996},"min":{"1min":0.53700000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.61499999999999999}]},{"osd":10,"last update":"Sat Mar 23 17:50:01 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.748,"15min":0.73399999999999999},"min":{"1min":0.71399999999999997,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.71799999999999997,"15min":0.70699999999999996},"min":{"1min":0.65000000000000002,"5min":0.503,"15min":0.503},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.76900000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":66048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":65734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":65726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T17:50:26.245 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T17:50:26.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.473+0000 7f3127000700 1 -- 172.21.15.86:0/1787614777 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3120102810 msgr2=0x7f31201090c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:26.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.473+0000 7f3127000700 1 --2- 172.21.15.86:0/1787614777 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3120102810 0x7f31201090c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f311800b0a0 tx=0x7f311800b3b0 comp rx=0 tx=0).stop 2024-03-23T17:50:26.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.473+0000 7f3127000700 1 -- 172.21.15.86:0/1787614777 shutdown_connections 2024-03-23T17:50:26.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.473+0000 7f3127000700 1 --2- 172.21.15.86:0/1787614777 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3120102810 0x7f31201090c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:26.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.473+0000 7f3127000700 1 --2- 172.21.15.86:0/1787614777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3120100c80 0x7f31201010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:26.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.473+0000 7f3127000700 1 --2- 172.21.15.86:0/1787614777 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3120101e70 0x7f31201022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:26.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.473+0000 7f3127000700 1 -- 172.21.15.86:0/1787614777 >> 172.21.15.86:0/1787614777 conn(0x7f31200fc1f0 msgr2=0x7f31200fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:26.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.473+0000 7f3127000700 1 -- 172.21.15.86:0/1787614777 shutdown_connections 2024-03-23T17:50:26.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.473+0000 7f3127000700 1 -- 172.21.15.86:0/1787614777 wait complete. 2024-03-23T17:50:26.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.474+0000 7f3127000700 1 Processor -- start 2024-03-23T17:50:26.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.474+0000 7f3127000700 1 -- start start 2024-03-23T17:50:26.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.475+0000 7f3127000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3120100c80 0x7f3120074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:26.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.475+0000 7f3127000700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3120101e70 0x7f31200731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:26.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.475+0000 7f3127000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3120102810 0x7f31200736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:26.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.475+0000 7f3127000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3120073e80 con 0x7f3120100c80 2024-03-23T17:50:26.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.475+0000 7f3127000700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3120073ff0 con 0x7f3120102810 2024-03-23T17:50:26.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.475+0000 7f3127000700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31201a0db0 con 0x7f3120101e70 2024-03-23T17:50:26.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.475+0000 7f311fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3120100c80 0x7f3120074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:26.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.475+0000 7f311fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3120100c80 0x7f3120074b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43086/0 (socket says 172.21.15.86:43086) 2024-03-23T17:50:26.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.475+0000 7f311fe00700 1 -- 172.21.15.86:0/3822697321 learned_addr learned my addr 172.21.15.86:0/3822697321 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:26.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.475+0000 7f3125000700 1 --2- 172.21.15.86:0/3822697321 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3120102810 0x7f31200736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:26.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.475+0000 7f311f400700 1 --2- 172.21.15.86:0/3822697321 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3120101e70 0x7f31200731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:26.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.475+0000 7f311fe00700 1 -- 172.21.15.86:0/3822697321 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3120101e70 msgr2=0x7f31200731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:26.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.475+0000 7f311fe00700 1 --2- 172.21.15.86:0/3822697321 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3120101e70 0x7f31200731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:26.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.475+0000 7f311fe00700 1 -- 172.21.15.86:0/3822697321 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3120102810 msgr2=0x7f31200736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:26.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.475+0000 7f311fe00700 1 --2- 172.21.15.86:0/3822697321 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3120102810 0x7f31200736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:26.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.475+0000 7f311fe00700 1 -- 172.21.15.86:0/3822697321 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3110009520 con 0x7f3120100c80 2024-03-23T17:50:26.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.475+0000 7f311f400700 1 --2- 172.21.15.86:0/3822697321 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3120101e70 0x7f31200731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:50:26.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.475+0000 7f311fe00700 1 --2- 172.21.15.86:0/3822697321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3120100c80 0x7f3120074b10 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f311000ea50 tx=0x7f311000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:26.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.476+0000 7f311cc00700 1 -- 172.21.15.86:0/3822697321 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311000cb70 con 0x7f3120100c80 2024-03-23T17:50:26.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.476+0000 7f311cc00700 1 -- 172.21.15.86:0/3822697321 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f311000ccd0 con 0x7f3120100c80 2024-03-23T17:50:26.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.476+0000 7f311cc00700 1 -- 172.21.15.86:0/3822697321 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3110018680 con 0x7f3120100c80 2024-03-23T17:50:26.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.476+0000 7f3127000700 1 -- 172.21.15.86:0/3822697321 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3118009d00 con 0x7f3120100c80 2024-03-23T17:50:26.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.476+0000 7f3127000700 1 -- 172.21.15.86:0/3822697321 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31201a1280 con 0x7f3120100c80 2024-03-23T17:50:26.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.477+0000 7f3127000700 1 -- 172.21.15.86:0/3822697321 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3120108080 con 0x7f3120100c80 2024-03-23T17:50:26.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.479+0000 7f311cc00700 1 -- 172.21.15.86:0/3822697321 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f3110010430 con 0x7f3120100c80 2024-03-23T17:50:26.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.480+0000 7f311cc00700 1 --2- 172.21.15.86:0/3822697321 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f31040a1230 0x7f31040a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:26.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.480+0000 7f311cc00700 1 -- 172.21.15.86:0/3822697321 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f3110014070 con 0x7f3120100c80 2024-03-23T17:50:26.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.480+0000 7f311f400700 1 --2- 172.21.15.86:0/3822697321 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f31040a1230 0x7f31040a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:26.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.481+0000 7f311f400700 1 --2- 172.21.15.86:0/3822697321 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f31040a1230 0x7f31040a36f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f310c009cb0 tx=0x7f310c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:26.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.482+0000 7f311cc00700 1 -- 172.21.15.86:0/3822697321 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f3110096fa0 con 0x7f3120100c80 2024-03-23T17:50:26.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.722+0000 7f3127000700 1 -- 172.21.15.86:0/3822697321 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f312010c590 con 0x7f3120100c80 2024-03-23T17:50:26.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.724+0000 7f311cc00700 1 -- 172.21.15.86:0/3822697321 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22414 (secure 0 0 0) 0x7f311009a5c0 con 0x7f3120100c80 2024-03-23T17:50:26.724 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:50:26.724 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":82,"fsid":"9d76c9e8-e93c-11ee-95cd-87774f69a715","created":"2024-03-23T17:42:30.097233+0000","modified":"2024-03-23T17:49:29.773954+0000","last_up_change":"2024-03-23T17:48:18.217601+0000","last_in_change":"2024-03-23T17:48:00.250579+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T17:45:26.848397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T17:48:54.331894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T17:48:55.640693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"656655c6-1c7f-4640-92c4-2f42de820143","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6803","nonce":2423505491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6805","nonce":2423505491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6809","nonce":2423505491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6807","nonce":2423505491}]},"public_addr":"172.21.15.86:6803/2423505491","cluster_addr":"172.21.15.86:6805/2423505491","heartbeat_back_addr":"172.21.15.86:6809/2423505491","heartbeat_front_addr":"172.21.15.86:6807/2423505491","state":["exists","up"]},{"osd":1,"uuid":"3e86dc65-20a6-4de8-b9ce-fcf01f68862b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6801","nonce":3776252597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6803","nonce":3776252597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6807","nonce":3776252597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6805","nonce":3776252597}]},"public_addr":"172.21.15.92:6801/3776252597","cluster_addr":"172.21.15.92:6803/3776252597","heartbeat_back_addr":"172.21.15.92:6807/3776252597","heartbeat_front_addr":"172.21.15.92:6805/3776252597","state":["exists","up"]},{"osd":2,"uuid":"09808786-3bd3-4d11-8c7b-1736119dfbb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6801","nonce":1601265891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6803","nonce":1601265891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6807","nonce":1601265891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6805","nonce":1601265891}]},"public_addr":"172.21.15.152:6801/1601265891","cluster_addr":"172.21.15.152:6803/1601265891","heartbeat_back_addr":"172.21.15.152:6807/1601265891","heartbeat_front_addr":"172.21.15.152:6805/1601265891","state":["exists","up"]},{"osd":3,"uuid":"79c61910-4b53-424c-b150-6f5385a2cd68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6811","nonce":694846466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6813","nonce":694846466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6817","nonce":694846466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6815","nonce":694846466}]},"public_addr":"172.21.15.86:6811/694846466","cluster_addr":"172.21.15.86:6813/694846466","heartbeat_back_addr":"172.21.15.86:6817/694846466","heartbeat_front_addr":"172.21.15.86:6815/694846466","state":["exists","up"]},{"osd":4,"uuid":"a2878035-0179-469f-bbd4-8ce2d3f6bddc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6809","nonce":1422821471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6811","nonce":1422821471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6815","nonce":1422821471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6813","nonce":1422821471}]},"public_addr":"172.21.15.92:6809/1422821471","cluster_addr":"172.21.15.92:6811/1422821471","heartbeat_back_addr":"172.21.15.92:6815/1422821471","heartbeat_front_addr":"172.21.15.92:6813/1422821471","state":["exists","up"]},{"osd":5,"uuid":"837366aa-ce25-42ca-b157-5bbb50159d0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6809","nonce":1039375239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6811","nonce":1039375239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6815","nonce":1039375239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6813","nonce":1039375239}]},"public_addr":"172.21.15.152:6809/1039375239","cluster_addr":"172.21.15.152:6811/1039375239","heartbeat_back_addr":"172.21.15.152:6815/1039375239","heartbeat_front_addr":"172.21.15.152:6813/1039375239","state":["exists","up"]},{"osd":6,"uuid":"61024437-7b91-432f-a1af-6443d0ef5213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6819","nonce":4080452707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6821","nonce":4080452707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6825","nonce":4080452707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6823","nonce":4080452707}]},"public_addr":"172.21.15.86:6819/4080452707","cluster_addr":"172.21.15.86:6821/4080452707","heartbeat_back_addr":"172.21.15.86:6825/4080452707","heartbeat_front_addr":"172.21.15.86:6823/4080452707","state":["exists","up"]},{"osd":7,"uuid":"2684bdae-6b30-4afc-af68-4ce41b3a75b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6817","nonce":4187707622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6819","nonce":4187707622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6823","nonce":4187707622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6821","nonce":4187707622}]},"public_addr":"172.21.15.92:6817/4187707622","cluster_addr":"172.21.15.92:6819/4187707622","heartbeat_back_addr":"172.21.15.92:6823/4187707622","heartbeat_front_addr":"172.21.15.92:6821/4187707622","state":["exists","up"]},{"osd":8,"uuid":"f03bbf0d-1281-4bcf-9773-9854388e65ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6817","nonce":3237612748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6819","nonce":3237612748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6823","nonce":3237612748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6821","nonce":3237612748}]},"public_addr":"172.21.15.152:6817/3237612748","cluster_addr":"172.21.15.152:6819/3237612748","heartbeat_back_addr":"172.21.15.152:6823/3237612748","heartbeat_front_addr":"172.21.15.152:6821/3237612748","state":["exists","up"]},{"osd":9,"uuid":"e7e8b775-99ff-48a4-a23a-91e7ab6622bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6827","nonce":2835218294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6829","nonce":2835218294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6833","nonce":2835218294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6831","nonce":2835218294}]},"public_addr":"172.21.15.86:6827/2835218294","cluster_addr":"172.21.15.86:6829/2835218294","heartbeat_back_addr":"172.21.15.86:6833/2835218294","heartbeat_front_addr":"172.21.15.86:6831/2835218294","state":["exists","up"]},{"osd":10,"uuid":"d9d616cd-a229-44de-8768-02a4415df115","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6825","nonce":4259465100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6827","nonce":4259465100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6831","nonce":4259465100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6829","nonce":4259465100}]},"public_addr":"172.21.15.92:6825/4259465100","cluster_addr":"172.21.15.92:6827/4259465100","heartbeat_back_addr":"172.21.15.92:6831/4259465100","heartbeat_front_addr":"172.21.15.92:6829/4259465100","state":["exists","up"]},{"osd":11,"uuid":"e7c0b865-ccf5-4233-8d22-0d9b6dbe5c97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6825","nonce":194662760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6827","nonce":194662760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6831","nonce":194662760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6829","nonce":194662760}]},"public_addr":"172.21.15.152:6825/194662760","cluster_addr":"172.21.15.152:6827/194662760","heartbeat_back_addr":"172.21.15.152:6831/194662760","heartbeat_front_addr":"172.21.15.152:6829/194662760","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:44:47.269167+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:45:05.363655+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:45:23.192677+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:45:44.156228+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:02.532971+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:20.153295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:41.713888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:59.676516+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:47:17.207087+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:47:38.970316+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:47:57.282525+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:48:15.437606+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/3553500912":"2024-03-24T17:43:28.242492+0000","172.21.15.86:6801/3600278228":"2024-03-24T17:43:28.242492+0000","172.21.15.86:0/887262329":"2024-03-24T17:43:28.242492+0000","172.21.15.86:6800/2313723492":"2024-03-24T17:43:05.043910+0000","172.21.15.86:6800/3600278228":"2024-03-24T17:43:28.242492+0000","172.21.15.86:0/3815705642":"2024-03-24T17:43:05.043910+0000","172.21.15.86:0/1614943062":"2024-03-24T17:43:28.242492+0000","172.21.15.86:6801/2313723492":"2024-03-24T17:43:05.043910+0000","172.21.15.86:0/2416321716":"2024-03-24T17:43:05.043910+0000","172.21.15.86:0/433588188":"2024-03-24T17:43:05.043910+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T17:50:26.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.726+0000 7f3127000700 1 -- 172.21.15.86:0/3822697321 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f31040a1230 msgr2=0x7f31040a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:26.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.726+0000 7f3127000700 1 --2- 172.21.15.86:0/3822697321 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f31040a1230 0x7f31040a36f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f310c009cb0 tx=0x7f310c009260 comp rx=0 tx=0).stop 2024-03-23T17:50:26.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.726+0000 7f3127000700 1 -- 172.21.15.86:0/3822697321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3120100c80 msgr2=0x7f3120074b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:26.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.726+0000 7f3127000700 1 --2- 172.21.15.86:0/3822697321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3120100c80 0x7f3120074b10 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f311000ea50 tx=0x7f311000ed60 comp rx=0 tx=0).stop 2024-03-23T17:50:26.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.726+0000 7f3127000700 1 -- 172.21.15.86:0/3822697321 shutdown_connections 2024-03-23T17:50:26.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.726+0000 7f3127000700 1 --2- 172.21.15.86:0/3822697321 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3120102810 0x7f31200736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:26.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.726+0000 7f3127000700 1 --2- 172.21.15.86:0/3822697321 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f31040a1230 0x7f31040a36f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:26.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.726+0000 7f3127000700 1 --2- 172.21.15.86:0/3822697321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3120100c80 0x7f3120074b10 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:26.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.726+0000 7f3127000700 1 --2- 172.21.15.86:0/3822697321 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3120101e70 0x7f31200731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:26.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.726+0000 7f3127000700 1 -- 172.21.15.86:0/3822697321 >> 172.21.15.86:0/3822697321 conn(0x7f31200fc1f0 msgr2=0x7f31201051c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:26.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.726+0000 7f3127000700 1 -- 172.21.15.86:0/3822697321 shutdown_connections 2024-03-23T17:50:26.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.727+0000 7f3127000700 1 -- 172.21.15.86:0/3822697321 wait complete. 2024-03-23T17:50:26.747 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T17:45:26.848397+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T17:48:54.331894+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T17:48:55.640693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T17:50:26.747 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T17:50:26.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.942+0000 7f2180a00700 1 -- 172.21.15.86:0/1003468185 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f217c102910 msgr2=0x7f217c1091c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:26.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.942+0000 7f2180a00700 1 --2- 172.21.15.86:0/1003468185 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f217c102910 0x7f217c1091c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f217400b0d0 tx=0x7f217400b3e0 comp rx=0 tx=0).stop 2024-03-23T17:50:26.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.942+0000 7f2180a00700 1 -- 172.21.15.86:0/1003468185 shutdown_connections 2024-03-23T17:50:26.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.942+0000 7f2180a00700 1 --2- 172.21.15.86:0/1003468185 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f217c102910 0x7f217c1091c0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:26.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.942+0000 7f2180a00700 1 --2- 172.21.15.86:0/1003468185 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f217c100d80 0x7f217c1011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:26.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.942+0000 7f2180a00700 1 --2- 172.21.15.86:0/1003468185 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f217c101f70 0x7f217c1023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:26.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.942+0000 7f2180a00700 1 -- 172.21.15.86:0/1003468185 >> 172.21.15.86:0/1003468185 conn(0x7f217c0fc2f0 msgr2=0x7f217c0fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:26.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.942+0000 7f2180a00700 1 -- 172.21.15.86:0/1003468185 shutdown_connections 2024-03-23T17:50:26.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.942+0000 7f2180a00700 1 -- 172.21.15.86:0/1003468185 wait complete. 2024-03-23T17:50:26.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.943+0000 7f2180a00700 1 Processor -- start 2024-03-23T17:50:26.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.943+0000 7f2180a00700 1 -- start start 2024-03-23T17:50:26.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.944+0000 7f2180a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f217c100d80 0x7f217c074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:26.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.944+0000 7f2180a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f217c101f70 0x7f217c0731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:26.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.944+0000 7f2180a00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f217c102910 0x7f217c0736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:26.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.944+0000 7f2180a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f217c073e80 con 0x7f217c101f70 2024-03-23T17:50:26.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.944+0000 7f2180a00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f217c073ff0 con 0x7f217c100d80 2024-03-23T17:50:26.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.944+0000 7f2180a00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f217c1a0e60 con 0x7f217c102910 2024-03-23T17:50:26.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.944+0000 7f2179400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f217c101f70 0x7f217c0731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:26.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.944+0000 7f2179e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f217c100d80 0x7f217c074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:26.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.944+0000 7f2179e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f217c100d80 0x7f217c074b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:59002/0 (socket says 172.21.15.86:59002) 2024-03-23T17:50:26.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.944+0000 7f2179400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f217c101f70 0x7f217c0731a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43096/0 (socket says 172.21.15.86:43096) 2024-03-23T17:50:26.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.944+0000 7f2179e00700 1 -- 172.21.15.86:0/2040437944 learned_addr learned my addr 172.21.15.86:0/2040437944 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:26.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.944+0000 7f217a800700 1 --2- 172.21.15.86:0/2040437944 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f217c102910 0x7f217c0736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:26.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.944+0000 7f2179e00700 1 -- 172.21.15.86:0/2040437944 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f217c102910 msgr2=0x7f217c0736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:26.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.944+0000 7f2179e00700 1 --2- 172.21.15.86:0/2040437944 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f217c102910 0x7f217c0736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:26.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.944+0000 7f2179e00700 1 -- 172.21.15.86:0/2040437944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f217c101f70 msgr2=0x7f217c0731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:26.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.944+0000 7f2179e00700 1 --2- 172.21.15.86:0/2040437944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f217c101f70 0x7f217c0731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:26.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.944+0000 7f2179e00700 1 -- 172.21.15.86:0/2040437944 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2174009d00 con 0x7f217c100d80 2024-03-23T17:50:26.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.945+0000 7f2179400700 1 --2- 172.21.15.86:0/2040437944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f217c101f70 0x7f217c0731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:50:26.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.945+0000 7f217a800700 1 --2- 172.21.15.86:0/2040437944 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f217c102910 0x7f217c0736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:50:26.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.945+0000 7f2179e00700 1 --2- 172.21.15.86:0/2040437944 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f217c100d80 0x7f217c074b10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f216c00e970 tx=0x7f216c00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:26.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.945+0000 7f2172a00700 1 -- 172.21.15.86:0/2040437944 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216c009790 con 0x7f217c100d80 2024-03-23T17:50:26.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.945+0000 7f2180a00700 1 -- 172.21.15.86:0/2040437944 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f217c1a10f0 con 0x7f217c100d80 2024-03-23T17:50:26.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.945+0000 7f2172a00700 1 -- 172.21.15.86:0/2040437944 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f216c00c780 con 0x7f217c100d80 2024-03-23T17:50:26.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.945+0000 7f2172a00700 1 -- 172.21.15.86:0/2040437944 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216c003be0 con 0x7f217c100d80 2024-03-23T17:50:26.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.945+0000 7f2180a00700 1 -- 172.21.15.86:0/2040437944 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f217c1a1650 con 0x7f217c100d80 2024-03-23T17:50:26.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.946+0000 7f2180a00700 1 -- 172.21.15.86:0/2040437944 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f217c108130 con 0x7f217c100d80 2024-03-23T17:50:26.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.948+0000 7f2172a00700 1 -- 172.21.15.86:0/2040437944 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f216c00cbf0 con 0x7f217c100d80 2024-03-23T17:50:26.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.949+0000 7f2172a00700 1 --2- 172.21.15.86:0/2040437944 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f21600a11b0 0x7f21600a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:26.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.949+0000 7f2172a00700 1 -- 172.21.15.86:0/2040437944 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f216c014070 con 0x7f217c100d80 2024-03-23T17:50:26.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.949+0000 7f2179400700 1 --2- 172.21.15.86:0/2040437944 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f21600a11b0 0x7f21600a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:26.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.950+0000 7f2179400700 1 --2- 172.21.15.86:0/2040437944 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f21600a11b0 0x7f21600a3670 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2168011490 tx=0x7f2168009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:26.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:26.954+0000 7f2172a00700 1 -- 172.21.15.86:0/2040437944 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f216c096260 con 0x7f217c100d80 2024-03-23T17:50:27.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:26 smithi086 ceph-mon[38221]: from='client.25321 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T17:50:27.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:26 smithi086 ceph-mon[38221]: from='client.25327 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T17:50:27.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:26 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/3822697321' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T17:50:27.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:26 smithi092 ceph-mon[79089]: from='client.25321 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T17:50:27.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:26 smithi092 ceph-mon[79089]: from='client.25327 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T17:50:27.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:26 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/3822697321' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T17:50:27.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:26 smithi152 ceph-mon[47949]: from='client.25321 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T17:50:27.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:26 smithi152 ceph-mon[47949]: from='client.25327 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T17:50:27.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:26 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/3822697321' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T17:50:27.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.158+0000 7f2180a00700 1 -- 172.21.15.86:0/2040437944 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f217c10c7a0 con 0x7f217c100d80 2024-03-23T17:50:27.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.159+0000 7f2172a00700 1 -- 172.21.15.86:0/2040437944 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f216c099880 con 0x7f217c100d80 2024-03-23T17:50:27.159 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-03-23T17:50:27.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.162+0000 7f2180a00700 1 -- 172.21.15.86:0/2040437944 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f21600a11b0 msgr2=0x7f21600a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:27.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.162+0000 7f2180a00700 1 --2- 172.21.15.86:0/2040437944 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f21600a11b0 0x7f21600a3670 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2168011490 tx=0x7f2168009260 comp rx=0 tx=0).stop 2024-03-23T17:50:27.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.162+0000 7f2180a00700 1 -- 172.21.15.86:0/2040437944 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f217c100d80 msgr2=0x7f217c074b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:27.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.162+0000 7f2180a00700 1 --2- 172.21.15.86:0/2040437944 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f217c100d80 0x7f217c074b10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f216c00e970 tx=0x7f216c00ec80 comp rx=0 tx=0).stop 2024-03-23T17:50:27.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.162+0000 7f2180a00700 1 -- 172.21.15.86:0/2040437944 shutdown_connections 2024-03-23T17:50:27.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.162+0000 7f2180a00700 1 --2- 172.21.15.86:0/2040437944 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f217c102910 0x7f217c0736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:27.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.162+0000 7f2180a00700 1 --2- 172.21.15.86:0/2040437944 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f217c100d80 0x7f217c074b10 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:27.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.162+0000 7f2180a00700 1 --2- 172.21.15.86:0/2040437944 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f21600a11b0 0x7f21600a3670 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:27.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.162+0000 7f2180a00700 1 --2- 172.21.15.86:0/2040437944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f217c101f70 0x7f217c0731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:27.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.162+0000 7f2180a00700 1 -- 172.21.15.86:0/2040437944 >> 172.21.15.86:0/2040437944 conn(0x7f217c0fc2f0 msgr2=0x7f217c105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:27.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.162+0000 7f2180a00700 1 -- 172.21.15.86:0/2040437944 shutdown_connections 2024-03-23T17:50:27.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.162+0000 7f2180a00700 1 -- 172.21.15.86:0/2040437944 wait complete. 2024-03-23T17:50:27.180 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T17:50:27.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.375+0000 7f23dd800700 1 -- 172.21.15.86:0/726962296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23d8108c40 msgr2=0x7f23d810b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:27.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.375+0000 7f23dd800700 1 --2- 172.21.15.86:0/726962296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23d8108c40 0x7f23d810b0d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f23d000b0a0 tx=0x7f23d000b3b0 comp rx=0 tx=0).stop 2024-03-23T17:50:27.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.375+0000 7f23dd800700 1 -- 172.21.15.86:0/726962296 shutdown_connections 2024-03-23T17:50:27.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.375+0000 7f23dd800700 1 --2- 172.21.15.86:0/726962296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23d8108c40 0x7f23d810b0d0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:27.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.375+0000 7f23dd800700 1 --2- 172.21.15.86:0/726962296 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f23d8074df0 0x7f23d8073240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:27.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.375+0000 7f23dd800700 1 --2- 172.21.15.86:0/726962296 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23d8073810 0x7f23d8073c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:27.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.375+0000 7f23dd800700 1 -- 172.21.15.86:0/726962296 >> 172.21.15.86:0/726962296 conn(0x7f23d80fc2f0 msgr2=0x7f23d80fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:27.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.375+0000 7f23dd800700 1 -- 172.21.15.86:0/726962296 shutdown_connections 2024-03-23T17:50:27.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.375+0000 7f23dd800700 1 -- 172.21.15.86:0/726962296 wait complete. 2024-03-23T17:50:27.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.376+0000 7f23dd800700 1 Processor -- start 2024-03-23T17:50:27.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.376+0000 7f23dd800700 1 -- start start 2024-03-23T17:50:27.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.377+0000 7f23dd800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23d8073810 0x7f23d8199000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:27.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.377+0000 7f23dd800700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f23d8074df0 0x7f23d8199540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:27.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.377+0000 7f23dd800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23d8108c40 0x7f23d81a0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:27.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.377+0000 7f23dd800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d81a0b30 con 0x7f23d8108c40 2024-03-23T17:50:27.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.377+0000 7f23dd800700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d81a0c70 con 0x7f23d8073810 2024-03-23T17:50:27.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.377+0000 7f23dd800700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d81a0e00 con 0x7f23d8074df0 2024-03-23T17:50:27.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.377+0000 7f23d6000700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f23d8074df0 0x7f23d8199540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:27.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.377+0000 7f23d7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23d8108c40 0x7f23d81a0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:27.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.377+0000 7f23d6000700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f23d8074df0 0x7f23d8199540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:33322/0 (socket says 172.21.15.86:33322) 2024-03-23T17:50:27.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.377+0000 7f23d6000700 1 -- 172.21.15.86:0/2297489518 learned_addr learned my addr 172.21.15.86:0/2297489518 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:27.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.377+0000 7f23d6a00700 1 --2- 172.21.15.86:0/2297489518 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23d8073810 0x7f23d8199000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:27.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.377+0000 7f23d6000700 1 -- 172.21.15.86:0/2297489518 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23d8073810 msgr2=0x7f23d8199000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:27.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.377+0000 7f23d6000700 1 --2- 172.21.15.86:0/2297489518 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23d8073810 0x7f23d8199000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:27.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.377+0000 7f23d6000700 1 -- 172.21.15.86:0/2297489518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23d8108c40 msgr2=0x7f23d81a0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:27.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.377+0000 7f23d6000700 1 --2- 172.21.15.86:0/2297489518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23d8108c40 0x7f23d81a0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:27.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.377+0000 7f23d6000700 1 -- 172.21.15.86:0/2297489518 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23d0009d00 con 0x7f23d8074df0 2024-03-23T17:50:27.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.378+0000 7f23d6000700 1 --2- 172.21.15.86:0/2297489518 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f23d8074df0 0x7f23d8199540 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f23c400e970 tx=0x7f23c400ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:27.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.378+0000 7f23cf400700 1 -- 172.21.15.86:0/2297489518 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c4009790 con 0x7f23d8074df0 2024-03-23T17:50:27.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.378+0000 7f23dd800700 1 -- 172.21.15.86:0/2297489518 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23d81a10e0 con 0x7f23d8074df0 2024-03-23T17:50:27.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.378+0000 7f23cf400700 1 -- 172.21.15.86:0/2297489518 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23c400c7e0 con 0x7f23d8074df0 2024-03-23T17:50:27.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.378+0000 7f23cf400700 1 -- 172.21.15.86:0/2297489518 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c4003c70 con 0x7f23d8074df0 2024-03-23T17:50:27.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.378+0000 7f23dd800700 1 -- 172.21.15.86:0/2297489518 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23d81a15f0 con 0x7f23d8074df0 2024-03-23T17:50:27.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.379+0000 7f23dd800700 1 -- 172.21.15.86:0/2297489518 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23d8107a00 con 0x7f23d8074df0 2024-03-23T17:50:27.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.381+0000 7f23cf400700 1 -- 172.21.15.86:0/2297489518 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f23c400cc50 con 0x7f23d8074df0 2024-03-23T17:50:27.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.382+0000 7f23cf400700 1 --2- 172.21.15.86:0/2297489518 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f23bc0a11b0 0x7f23bc0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:27.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.383+0000 7f23cf400700 1 -- 172.21.15.86:0/2297489518 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f23c4014070 con 0x7f23d8074df0 2024-03-23T17:50:27.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.383+0000 7f23d6a00700 1 --2- 172.21.15.86:0/2297489518 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f23bc0a11b0 0x7f23bc0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:27.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.383+0000 7f23d6a00700 1 --2- 172.21.15.86:0/2297489518 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f23bc0a11b0 0x7f23bc0a3670 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f23c8009c60 tx=0x7f23c8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:27.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.385+0000 7f23cf400700 1 -- 172.21.15.86:0/2297489518 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f23c40963d0 con 0x7f23d8074df0 2024-03-23T17:50:27.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.588+0000 7f23dd800700 1 -- 172.21.15.86:0/2297489518 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f23d810e7c0 con 0x7f23d8074df0 2024-03-23T17:50:27.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.589+0000 7f23cf400700 1 -- 172.21.15.86:0/2297489518 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v82) v1 ==== 104+0+11 (secure 0 0 0) 0x7f23c4003dd0 con 0x7f23d8074df0 2024-03-23T17:50:27.589 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-03-23T17:50:27.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.592+0000 7f23dd800700 1 -- 172.21.15.86:0/2297489518 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f23bc0a11b0 msgr2=0x7f23bc0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:27.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.592+0000 7f23dd800700 1 --2- 172.21.15.86:0/2297489518 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f23bc0a11b0 0x7f23bc0a3670 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f23c8009c60 tx=0x7f23c8009260 comp rx=0 tx=0).stop 2024-03-23T17:50:27.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.592+0000 7f23dd800700 1 -- 172.21.15.86:0/2297489518 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f23d8074df0 msgr2=0x7f23d8199540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:27.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.592+0000 7f23dd800700 1 --2- 172.21.15.86:0/2297489518 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f23d8074df0 0x7f23d8199540 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f23c400e970 tx=0x7f23c400ec80 comp rx=0 tx=0).stop 2024-03-23T17:50:27.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.592+0000 7f23dd800700 1 -- 172.21.15.86:0/2297489518 shutdown_connections 2024-03-23T17:50:27.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.592+0000 7f23dd800700 1 --2- 172.21.15.86:0/2297489518 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f23bc0a11b0 0x7f23bc0a3670 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:27.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.592+0000 7f23dd800700 1 --2- 172.21.15.86:0/2297489518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23d8108c40 0x7f23d81a0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:27.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.592+0000 7f23dd800700 1 --2- 172.21.15.86:0/2297489518 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23d8073810 0x7f23d8199000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:27.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.592+0000 7f23dd800700 1 --2- 172.21.15.86:0/2297489518 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f23d8074df0 0x7f23d8199540 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:27.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.592+0000 7f23dd800700 1 -- 172.21.15.86:0/2297489518 >> 172.21.15.86:0/2297489518 conn(0x7f23d80fc2f0 msgr2=0x7f23d8104b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:27.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.592+0000 7f23dd800700 1 -- 172.21.15.86:0/2297489518 shutdown_connections 2024-03-23T17:50:27.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.592+0000 7f23dd800700 1 -- 172.21.15.86:0/2297489518 wait complete. 2024-03-23T17:50:27.611 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T17:50:27.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.807+0000 7f5b5cc00700 1 -- 172.21.15.86:0/1008206978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b58101810 msgr2=0x7f5b58103c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:27.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.807+0000 7f5b5cc00700 1 --2- 172.21.15.86:0/1008206978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b58101810 0x7f5b58103c00 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5b48009990 tx=0x7f5b48009ca0 comp rx=0 tx=0).stop 2024-03-23T17:50:27.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.807+0000 7f5b5cc00700 1 -- 172.21.15.86:0/1008206978 shutdown_connections 2024-03-23T17:50:27.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.807+0000 7f5b5cc00700 1 --2- 172.21.15.86:0/1008206978 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5b58106a70 0x7f5b58108ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:27.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.807+0000 7f5b5cc00700 1 --2- 172.21.15.86:0/1008206978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b58101810 0x7f5b58103c00 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:27.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.807+0000 7f5b5cc00700 1 --2- 172.21.15.86:0/1008206978 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b58104140 0x7f5b58106530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:27.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.807+0000 7f5b5cc00700 1 -- 172.21.15.86:0/1008206978 >> 172.21.15.86:0/1008206978 conn(0x7f5b580fb200 msgr2=0x7f5b580fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:27.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.807+0000 7f5b5cc00700 1 -- 172.21.15.86:0/1008206978 shutdown_connections 2024-03-23T17:50:27.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.807+0000 7f5b5cc00700 1 -- 172.21.15.86:0/1008206978 wait complete. 2024-03-23T17:50:27.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.808+0000 7f5b5cc00700 1 Processor -- start 2024-03-23T17:50:27.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.808+0000 7f5b5cc00700 1 -- start start 2024-03-23T17:50:27.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.809+0000 7f5b5cc00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b58101810 0x7f5b58198f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:27.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.809+0000 7f5b5cc00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5b58104140 0x7f5b58199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:27.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.809+0000 7f5b5cc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b58106a70 0x7f5b581a0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:27.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.809+0000 7f5b5cc00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b581a0a60 con 0x7f5b58106a70 2024-03-23T17:50:27.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.809+0000 7f5b5cc00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b581a0ba0 con 0x7f5b58101810 2024-03-23T17:50:27.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.809+0000 7f5b5cc00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b581a0d30 con 0x7f5b58104140 2024-03-23T17:50:27.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.809+0000 7f5b55e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b58101810 0x7f5b58198f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:27.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.809+0000 7f5b56800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b58106a70 0x7f5b581a0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:27.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.809+0000 7f5b55e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b58101810 0x7f5b58198f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:59040/0 (socket says 172.21.15.86:59040) 2024-03-23T17:50:27.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.809+0000 7f5b55e00700 1 -- 172.21.15.86:0/4069304465 learned_addr learned my addr 172.21.15.86:0/4069304465 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:27.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.809+0000 7f5b55400700 1 --2- 172.21.15.86:0/4069304465 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5b58104140 0x7f5b58199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:27.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.809+0000 7f5b55e00700 1 -- 172.21.15.86:0/4069304465 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5b58104140 msgr2=0x7f5b58199470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:27.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.809+0000 7f5b55e00700 1 --2- 172.21.15.86:0/4069304465 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5b58104140 0x7f5b58199470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:27.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.809+0000 7f5b55e00700 1 -- 172.21.15.86:0/4069304465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b58106a70 msgr2=0x7f5b581a0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:27.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.809+0000 7f5b55e00700 1 --2- 172.21.15.86:0/4069304465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b58106a70 0x7f5b581a0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:27.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.809+0000 7f5b55e00700 1 -- 172.21.15.86:0/4069304465 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b480095f0 con 0x7f5b58101810 2024-03-23T17:50:27.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.809+0000 7f5b56800700 1 --2- 172.21.15.86:0/4069304465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b58106a70 0x7f5b581a0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:50:27.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.810+0000 7f5b55e00700 1 --2- 172.21.15.86:0/4069304465 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b58101810 0x7f5b58198f30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5b4800b680 tx=0x7f5b48009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:27.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.810+0000 7f5b4ea00700 1 -- 172.21.15.86:0/4069304465 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4801d070 con 0x7f5b58101810 2024-03-23T17:50:27.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.810+0000 7f5b4ea00700 1 -- 172.21.15.86:0/4069304465 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b48022470 con 0x7f5b58101810 2024-03-23T17:50:27.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.810+0000 7f5b5cc00700 1 -- 172.21.15.86:0/4069304465 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b581a0fb0 con 0x7f5b58101810 2024-03-23T17:50:27.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.810+0000 7f5b4ea00700 1 -- 172.21.15.86:0/4069304465 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b48017650 con 0x7f5b58101810 2024-03-23T17:50:27.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.810+0000 7f5b5cc00700 1 -- 172.21.15.86:0/4069304465 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b581a1320 con 0x7f5b58101810 2024-03-23T17:50:27.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.811+0000 7f5b5cc00700 1 -- 172.21.15.86:0/4069304465 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b58108640 con 0x7f5b58101810 2024-03-23T17:50:27.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.813+0000 7f5b4ea00700 1 -- 172.21.15.86:0/4069304465 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f5b480225e0 con 0x7f5b58101810 2024-03-23T17:50:27.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.814+0000 7f5b4ea00700 1 --2- 172.21.15.86:0/4069304465 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f5b3c0a11b0 0x7f5b3c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:27.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.814+0000 7f5b4ea00700 1 -- 172.21.15.86:0/4069304465 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f5b480cde90 con 0x7f5b58101810 2024-03-23T17:50:27.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.815+0000 7f5b55400700 1 --2- 172.21.15.86:0/4069304465 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f5b3c0a11b0 0x7f5b3c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:27.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.815+0000 7f5b55400700 1 --2- 172.21.15.86:0/4069304465 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f5b3c0a11b0 0x7f5b3c0a3670 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5b44009c60 tx=0x7f5b44009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:27.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:27.817+0000 7f5b4ea00700 1 -- 172.21.15.86:0/4069304465 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f5b48097520 con 0x7f5b58101810 2024-03-23T17:50:28.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:27 smithi086 ceph-mon[38221]: from='client.15282 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T17:50:28.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:27 smithi086 ceph-mon[38221]: pgmap v281: 97 pgs: 97 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 853 B/s wr, 11 op/s 2024-03-23T17:50:28.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:27 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/2040437944' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T17:50:28.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:27 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/2297489518' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T17:50:28.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.023+0000 7f5b5cc00700 1 -- 172.21.15.86:0/4069304465 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5b5810c6d0 con 0x7f5b58101810 2024-03-23T17:50:28.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.024+0000 7f5b4ea00700 1 -- 172.21.15.86:0/4069304465 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v82) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5b4809ab40 con 0x7f5b58101810 2024-03-23T17:50:28.024 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-03-23T17:50:28.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.026+0000 7f5b5cc00700 1 -- 172.21.15.86:0/4069304465 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f5b3c0a11b0 msgr2=0x7f5b3c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:28.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.026+0000 7f5b5cc00700 1 --2- 172.21.15.86:0/4069304465 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f5b3c0a11b0 0x7f5b3c0a3670 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5b44009c60 tx=0x7f5b44009260 comp rx=0 tx=0).stop 2024-03-23T17:50:28.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.026+0000 7f5b5cc00700 1 -- 172.21.15.86:0/4069304465 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b58101810 msgr2=0x7f5b58198f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:28.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.026+0000 7f5b5cc00700 1 --2- 172.21.15.86:0/4069304465 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b58101810 0x7f5b58198f30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5b4800b680 tx=0x7f5b48009580 comp rx=0 tx=0).stop 2024-03-23T17:50:28.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.026+0000 7f5b5cc00700 1 -- 172.21.15.86:0/4069304465 shutdown_connections 2024-03-23T17:50:28.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.026+0000 7f5b5cc00700 1 --2- 172.21.15.86:0/4069304465 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f5b3c0a11b0 0x7f5b3c0a3670 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:28.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.026+0000 7f5b5cc00700 1 --2- 172.21.15.86:0/4069304465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b58106a70 0x7f5b581a0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:28.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.026+0000 7f5b5cc00700 1 --2- 172.21.15.86:0/4069304465 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b58101810 0x7f5b58198f30 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:28.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.026+0000 7f5b5cc00700 1 --2- 172.21.15.86:0/4069304465 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5b58104140 0x7f5b58199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:28.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.026+0000 7f5b5cc00700 1 -- 172.21.15.86:0/4069304465 >> 172.21.15.86:0/4069304465 conn(0x7f5b580fb200 msgr2=0x7f5b58101100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:28.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.026+0000 7f5b5cc00700 1 -- 172.21.15.86:0/4069304465 shutdown_connections 2024-03-23T17:50:28.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.026+0000 7f5b5cc00700 1 -- 172.21.15.86:0/4069304465 wait complete. 2024-03-23T17:50:28.045 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T17:50:28.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:27 smithi092 ceph-mon[79089]: from='client.15282 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T17:50:28.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:27 smithi092 ceph-mon[79089]: pgmap v281: 97 pgs: 97 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 853 B/s wr, 11 op/s 2024-03-23T17:50:28.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:27 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/2040437944' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T17:50:28.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:27 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/2297489518' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T17:50:28.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:27 smithi152 ceph-mon[47949]: from='client.15282 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T17:50:28.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:27 smithi152 ceph-mon[47949]: pgmap v281: 97 pgs: 97 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 853 B/s wr, 11 op/s 2024-03-23T17:50:28.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:27 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/2040437944' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T17:50:28.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:27 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/2297489518' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T17:50:28.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.238+0000 7f6a3aa00700 1 -- 172.21.15.86:0/3770634294 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a34100c80 msgr2=0x7f6a341010a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:28.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.238+0000 7f6a3aa00700 1 --2- 172.21.15.86:0/3770634294 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a34100c80 0x7f6a341010a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6a240099c0 tx=0x7f6a24009cd0 comp rx=0 tx=0).stop 2024-03-23T17:50:28.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.238+0000 7f6a3aa00700 1 -- 172.21.15.86:0/3770634294 shutdown_connections 2024-03-23T17:50:28.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.238+0000 7f6a3aa00700 1 --2- 172.21.15.86:0/3770634294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a34102810 0x7f6a341090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:28.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.238+0000 7f6a3aa00700 1 --2- 172.21.15.86:0/3770634294 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a34100c80 0x7f6a341010a0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:28.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.238+0000 7f6a3aa00700 1 --2- 172.21.15.86:0/3770634294 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6a34101e70 0x7f6a341022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:28.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.238+0000 7f6a3aa00700 1 -- 172.21.15.86:0/3770634294 >> 172.21.15.86:0/3770634294 conn(0x7f6a340fc1f0 msgr2=0x7f6a340fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:28.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.238+0000 7f6a3aa00700 1 -- 172.21.15.86:0/3770634294 shutdown_connections 2024-03-23T17:50:28.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.238+0000 7f6a3aa00700 1 -- 172.21.15.86:0/3770634294 wait complete. 2024-03-23T17:50:28.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.239+0000 7f6a3aa00700 1 Processor -- start 2024-03-23T17:50:28.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.239+0000 7f6a3aa00700 1 -- start start 2024-03-23T17:50:28.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.239+0000 7f6a3aa00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6a34101e70 0x7f6a34196ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:28.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.239+0000 7f6a3aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a34102810 0x7f6a34197410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:28.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.239+0000 7f6a3aa00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a3419c3d0 0x7f6a3419e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:28.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.239+0000 7f6a3aa00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3419ed90 con 0x7f6a34102810 2024-03-23T17:50:28.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.239+0000 7f6a3aa00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3419eed0 con 0x7f6a3419c3d0 2024-03-23T17:50:28.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.239+0000 7f6a3aa00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3419f060 con 0x7f6a34101e70 2024-03-23T17:50:28.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.240+0000 7f6a33e00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6a34101e70 0x7f6a34196ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:28.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.240+0000 7f6a33e00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6a34101e70 0x7f6a34196ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:33364/0 (socket says 172.21.15.86:33364) 2024-03-23T17:50:28.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.240+0000 7f6a33e00700 1 -- 172.21.15.86:0/3238032736 learned_addr learned my addr 172.21.15.86:0/3238032736 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:28.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.240+0000 7f6a33400700 1 --2- 172.21.15.86:0/3238032736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a34102810 0x7f6a34197410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:28.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.240+0000 7f6a38a00700 1 --2- 172.21.15.86:0/3238032736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a3419c3d0 0x7f6a3419e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:28.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.240+0000 7f6a33e00700 1 -- 172.21.15.86:0/3238032736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a3419c3d0 msgr2=0x7f6a3419e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:28.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.240+0000 7f6a33e00700 1 --2- 172.21.15.86:0/3238032736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a3419c3d0 0x7f6a3419e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:28.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.240+0000 7f6a33e00700 1 -- 172.21.15.86:0/3238032736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a34102810 msgr2=0x7f6a34197410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:28.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.240+0000 7f6a33e00700 1 --2- 172.21.15.86:0/3238032736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a34102810 0x7f6a34197410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:28.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.240+0000 7f6a33e00700 1 -- 172.21.15.86:0/3238032736 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a240095f0 con 0x7f6a34101e70 2024-03-23T17:50:28.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.240+0000 7f6a33e00700 1 --2- 172.21.15.86:0/3238032736 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6a34101e70 0x7f6a34196ed0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6a240098e0 tx=0x7f6a2400f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:28.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.241+0000 7f6a30c00700 1 -- 172.21.15.86:0/3238032736 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2401d070 con 0x7f6a34101e70 2024-03-23T17:50:28.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.241+0000 7f6a30c00700 1 -- 172.21.15.86:0/3238032736 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a24005020 con 0x7f6a34101e70 2024-03-23T17:50:28.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.241+0000 7f6a3aa00700 1 -- 172.21.15.86:0/3238032736 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a3419f2e0 con 0x7f6a34101e70 2024-03-23T17:50:28.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.241+0000 7f6a30c00700 1 -- 172.21.15.86:0/3238032736 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a24004a50 con 0x7f6a34101e70 2024-03-23T17:50:28.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.241+0000 7f6a3aa00700 1 -- 172.21.15.86:0/3238032736 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a3419f6b0 con 0x7f6a34101e70 2024-03-23T17:50:28.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.242+0000 7f6a3aa00700 1 -- 172.21.15.86:0/3238032736 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a34108240 con 0x7f6a34101e70 2024-03-23T17:50:28.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.244+0000 7f6a30c00700 1 -- 172.21.15.86:0/3238032736 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f6a240054b0 con 0x7f6a34101e70 2024-03-23T17:50:28.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.245+0000 7f6a30c00700 1 --2- 172.21.15.86:0/3238032736 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f6a180a11b0 0x7f6a180a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:28.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.245+0000 7f6a30c00700 1 -- 172.21.15.86:0/3238032736 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f6a240cd950 con 0x7f6a34101e70 2024-03-23T17:50:28.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.245+0000 7f6a33400700 1 --2- 172.21.15.86:0/3238032736 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f6a180a11b0 0x7f6a180a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:28.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.246+0000 7f6a33400700 1 --2- 172.21.15.86:0/3238032736 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f6a180a11b0 0x7f6a180a3670 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6a20009c60 tx=0x7f6a20009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:28.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.248+0000 7f6a30c00700 1 -- 172.21.15.86:0/3238032736 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f6a240970d0 con 0x7f6a34101e70 2024-03-23T17:50:28.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.529+0000 7f6a3aa00700 1 -- 172.21.15.86:0/3238032736 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6a34067a10 con 0x7f6a34101e70 2024-03-23T17:50:28.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.531+0000 7f6a30c00700 1 -- 172.21.15.86:0/3238032736 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+10913 (secure 0 0 0) 0x7f6a2409a6f0 con 0x7f6a34101e70 2024-03-23T17:50:28.531 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:50:28.531 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14445,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1433140219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1433140219},{"type":"v1","addr":"172.21.15.86:6835","nonce":1433140219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6839/1879457730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1879457730},{"type":"v1","addr":"172.21.15.86:6839","nonce":1879457730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-23T17:48:56.202323+0000","modified":"2024-03-23T17:50:20.718165+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24553,"mds_2":24518,"mds_3":24505,"mds_4":24500},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14514":{"gid":14514,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/3026943866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3026943866},{"type":"v1","addr":"172.21.15.86:6841","nonce":3026943866}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24464":{"gid":24464,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6833/1467091186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1467091186},{"type":"v1","addr":"172.21.15.92:6833","nonce":1467091186}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6833/3075554623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3075554623},{"type":"v1","addr":"172.21.15.152:6833","nonce":3075554623}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.92:6835/908900027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":908900027},{"type":"v1","addr":"172.21.15.92:6835","nonce":908900027}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"d","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.86:6837/2933881328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2933881328},{"type":"v1","addr":"172.21.15.86:6837","nonce":2933881328}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6837/3967837948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":3967837948},{"type":"v1","addr":"172.21.15.92:6837","nonce":3967837948}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6835/3137085021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3137085021},{"type":"v1","addr":"172.21.15.152:6835","nonce":3137085021}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24538":{"gid":24538,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6837/1151208965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":1151208965},{"type":"v1","addr":"172.21.15.152:6837","nonce":1151208965}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.152:6839/1100723992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":1100723992},{"type":"v1","addr":"172.21.15.152:6839","nonce":1100723992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6839/1394139887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1394139887},{"type":"v1","addr":"172.21.15.92:6839","nonce":1394139887}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-23T17:50:28.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.533+0000 7f6a3aa00700 1 -- 172.21.15.86:0/3238032736 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f6a180a11b0 msgr2=0x7f6a180a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:28.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.533+0000 7f6a3aa00700 1 --2- 172.21.15.86:0/3238032736 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f6a180a11b0 0x7f6a180a3670 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6a20009c60 tx=0x7f6a20009260 comp rx=0 tx=0).stop 2024-03-23T17:50:28.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.533+0000 7f6a3aa00700 1 -- 172.21.15.86:0/3238032736 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6a34101e70 msgr2=0x7f6a34196ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:28.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.533+0000 7f6a3aa00700 1 --2- 172.21.15.86:0/3238032736 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6a34101e70 0x7f6a34196ed0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6a240098e0 tx=0x7f6a2400f690 comp rx=0 tx=0).stop 2024-03-23T17:50:28.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.533+0000 7f6a3aa00700 1 -- 172.21.15.86:0/3238032736 shutdown_connections 2024-03-23T17:50:28.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.533+0000 7f6a3aa00700 1 --2- 172.21.15.86:0/3238032736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a3419c3d0 0x7f6a3419e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:28.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.533+0000 7f6a3aa00700 1 --2- 172.21.15.86:0/3238032736 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6a34101e70 0x7f6a34196ed0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:28.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.533+0000 7f6a3aa00700 1 --2- 172.21.15.86:0/3238032736 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f6a180a11b0 0x7f6a180a3670 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:28.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.533+0000 7f6a3aa00700 1 --2- 172.21.15.86:0/3238032736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a34102810 0x7f6a34197410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:28.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.533+0000 7f6a3aa00700 1 -- 172.21.15.86:0/3238032736 >> 172.21.15.86:0/3238032736 conn(0x7f6a340fc1f0 msgr2=0x7f6a34105380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:28.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.534+0000 7f6a3aa00700 1 -- 172.21.15.86:0/3238032736 shutdown_connections 2024-03-23T17:50:28.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.534+0000 7f6a3aa00700 1 -- 172.21.15.86:0/3238032736 wait complete. 2024-03-23T17:50:28.535 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 28 2024-03-23T17:50:28.552 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T17:50:28.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.745+0000 7f11f4400700 1 -- 172.21.15.86:0/2139893534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11ec0737b0 msgr2=0x7f11ec073c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:28.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.745+0000 7f11f4400700 1 --2- 172.21.15.86:0/2139893534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11ec0737b0 0x7f11ec073c30 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f11d8009990 tx=0x7f11d8009ca0 comp rx=0 tx=0).stop 2024-03-23T17:50:28.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.745+0000 7f11f4400700 1 -- 172.21.15.86:0/2139893534 shutdown_connections 2024-03-23T17:50:28.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.745+0000 7f11f4400700 1 --2- 172.21.15.86:0/2139893534 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11ec108b50 0x7f11ec10afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:28.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.745+0000 7f11f4400700 1 --2- 172.21.15.86:0/2139893534 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f11ec074d90 0x7f11ec0731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:28.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.745+0000 7f11f4400700 1 --2- 172.21.15.86:0/2139893534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11ec0737b0 0x7f11ec073c30 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:28.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.745+0000 7f11f4400700 1 -- 172.21.15.86:0/2139893534 >> 172.21.15.86:0/2139893534 conn(0x7f11ec0fc1f0 msgr2=0x7f11ec0fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:28.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.745+0000 7f11f4400700 1 -- 172.21.15.86:0/2139893534 shutdown_connections 2024-03-23T17:50:28.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.745+0000 7f11f4400700 1 -- 172.21.15.86:0/2139893534 wait complete. 2024-03-23T17:50:28.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.746+0000 7f11f4400700 1 Processor -- start 2024-03-23T17:50:28.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.746+0000 7f11f4400700 1 -- start start 2024-03-23T17:50:28.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.746+0000 7f11f4400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11ec0737b0 0x7f11ec198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:28.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.746+0000 7f11f4400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f11ec074d90 0x7f11ec199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:28.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.746+0000 7f11f4400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11ec108b50 0x7f11ec1a03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:28.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.746+0000 7f11f4400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11ec1a0970 con 0x7f11ec108b50 2024-03-23T17:50:28.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.746+0000 7f11f4400700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11ec1a0ab0 con 0x7f11ec0737b0 2024-03-23T17:50:28.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.746+0000 7f11f4400700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11ec1a0c40 con 0x7f11ec074d90 2024-03-23T17:50:28.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.747+0000 7f11f2400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11ec108b50 0x7f11ec1a03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:28.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.747+0000 7f11f2400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11ec108b50 0x7f11ec1a03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43168/0 (socket says 172.21.15.86:43168) 2024-03-23T17:50:28.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.747+0000 7f11f2400700 1 -- 172.21.15.86:0/4100576536 learned_addr learned my addr 172.21.15.86:0/4100576536 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:28.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.747+0000 7f11f1a00700 1 --2- 172.21.15.86:0/4100576536 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11ec0737b0 0x7f11ec198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:28.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.747+0000 7f11f1000700 1 --2- 172.21.15.86:0/4100576536 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f11ec074d90 0x7f11ec199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:28.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.747+0000 7f11f2400700 1 -- 172.21.15.86:0/4100576536 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f11ec074d90 msgr2=0x7f11ec199380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:28.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.747+0000 7f11f2400700 1 --2- 172.21.15.86:0/4100576536 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f11ec074d90 0x7f11ec199380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:28.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.747+0000 7f11f2400700 1 -- 172.21.15.86:0/4100576536 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11ec0737b0 msgr2=0x7f11ec198e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:28.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.747+0000 7f11f2400700 1 --2- 172.21.15.86:0/4100576536 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11ec0737b0 0x7f11ec198e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:28.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.747+0000 7f11f2400700 1 -- 172.21.15.86:0/4100576536 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11e0009520 con 0x7f11ec108b50 2024-03-23T17:50:28.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.747+0000 7f11f1a00700 1 --2- 172.21.15.86:0/4100576536 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11ec0737b0 0x7f11ec198e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:50:28.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.747+0000 7f11f2400700 1 --2- 172.21.15.86:0/4100576536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11ec108b50 0x7f11ec1a03a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f11e400ef30 tx=0x7f11e400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:28.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.748+0000 7f11ea000700 1 -- 172.21.15.86:0/4100576536 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e4018070 con 0x7f11ec108b50 2024-03-23T17:50:28.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.748+0000 7f11ea000700 1 -- 172.21.15.86:0/4100576536 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f11e400f040 con 0x7f11ec108b50 2024-03-23T17:50:28.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.748+0000 7f11ea000700 1 -- 172.21.15.86:0/4100576536 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e400cd00 con 0x7f11ec108b50 2024-03-23T17:50:28.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.748+0000 7f11f4400700 1 -- 172.21.15.86:0/4100576536 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11d80095f0 con 0x7f11ec108b50 2024-03-23T17:50:28.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.748+0000 7f11f4400700 1 -- 172.21.15.86:0/4100576536 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11ec1a1190 con 0x7f11ec108b50 2024-03-23T17:50:28.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.749+0000 7f11f4400700 1 -- 172.21.15.86:0/4100576536 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11ec107960 con 0x7f11ec108b50 2024-03-23T17:50:28.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.750+0000 7f11ea000700 1 -- 172.21.15.86:0/4100576536 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f11e4003a00 con 0x7f11ec108b50 2024-03-23T17:50:28.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.752+0000 7f11ea000700 1 --2- 172.21.15.86:0/4100576536 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f11d40a11b0 0x7f11d40a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:28.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.752+0000 7f11ea000700 1 -- 172.21.15.86:0/4100576536 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f11e40cc530 con 0x7f11ec108b50 2024-03-23T17:50:28.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.752+0000 7f11f1a00700 1 --2- 172.21.15.86:0/4100576536 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f11d40a11b0 0x7f11d40a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:28.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.753+0000 7f11f1a00700 1 --2- 172.21.15.86:0/4100576536 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f11d40a11b0 0x7f11d40a3670 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f11e0009fd0 tx=0x7f11e0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:28.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.759+0000 7f11ea000700 1 -- 172.21.15.86:0/4100576536 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f11e4095c20 con 0x7f11ec108b50 2024-03-23T17:50:28.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:28.769+0000 7f11ea000700 1 -- 172.21.15.86:0/4100576536 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f11e4003cb0 con 0x7f11ec108b50 2024-03-23T17:50:29.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.002+0000 7f11f4400700 1 -- 172.21.15.86:0/4100576536 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f11ec10e6c0 con 0x7f11ec108b50 2024-03-23T17:50:29.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.004+0000 7f11ea000700 1 -- 172.21.15.86:0/4100576536 <== mon.0 v2:172.21.15.86:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22414 (secure 0 0 0) 0x7f11e40996b0 con 0x7f11ec108b50 2024-03-23T17:50:29.004 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:50:29.004 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":82,"fsid":"9d76c9e8-e93c-11ee-95cd-87774f69a715","created":"2024-03-23T17:42:30.097233+0000","modified":"2024-03-23T17:49:29.773954+0000","last_up_change":"2024-03-23T17:48:18.217601+0000","last_in_change":"2024-03-23T17:48:00.250579+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T17:45:26.848397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T17:48:54.331894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T17:48:55.640693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"656655c6-1c7f-4640-92c4-2f42de820143","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6803","nonce":2423505491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6805","nonce":2423505491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6809","nonce":2423505491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6807","nonce":2423505491}]},"public_addr":"172.21.15.86:6803/2423505491","cluster_addr":"172.21.15.86:6805/2423505491","heartbeat_back_addr":"172.21.15.86:6809/2423505491","heartbeat_front_addr":"172.21.15.86:6807/2423505491","state":["exists","up"]},{"osd":1,"uuid":"3e86dc65-20a6-4de8-b9ce-fcf01f68862b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6801","nonce":3776252597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6803","nonce":3776252597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6807","nonce":3776252597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6805","nonce":3776252597}]},"public_addr":"172.21.15.92:6801/3776252597","cluster_addr":"172.21.15.92:6803/3776252597","heartbeat_back_addr":"172.21.15.92:6807/3776252597","heartbeat_front_addr":"172.21.15.92:6805/3776252597","state":["exists","up"]},{"osd":2,"uuid":"09808786-3bd3-4d11-8c7b-1736119dfbb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6801","nonce":1601265891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6803","nonce":1601265891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6807","nonce":1601265891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6805","nonce":1601265891}]},"public_addr":"172.21.15.152:6801/1601265891","cluster_addr":"172.21.15.152:6803/1601265891","heartbeat_back_addr":"172.21.15.152:6807/1601265891","heartbeat_front_addr":"172.21.15.152:6805/1601265891","state":["exists","up"]},{"osd":3,"uuid":"79c61910-4b53-424c-b150-6f5385a2cd68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6811","nonce":694846466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6813","nonce":694846466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6817","nonce":694846466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6815","nonce":694846466}]},"public_addr":"172.21.15.86:6811/694846466","cluster_addr":"172.21.15.86:6813/694846466","heartbeat_back_addr":"172.21.15.86:6817/694846466","heartbeat_front_addr":"172.21.15.86:6815/694846466","state":["exists","up"]},{"osd":4,"uuid":"a2878035-0179-469f-bbd4-8ce2d3f6bddc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6809","nonce":1422821471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6811","nonce":1422821471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6815","nonce":1422821471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6813","nonce":1422821471}]},"public_addr":"172.21.15.92:6809/1422821471","cluster_addr":"172.21.15.92:6811/1422821471","heartbeat_back_addr":"172.21.15.92:6815/1422821471","heartbeat_front_addr":"172.21.15.92:6813/1422821471","state":["exists","up"]},{"osd":5,"uuid":"837366aa-ce25-42ca-b157-5bbb50159d0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6809","nonce":1039375239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6811","nonce":1039375239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6815","nonce":1039375239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6813","nonce":1039375239}]},"public_addr":"172.21.15.152:6809/1039375239","cluster_addr":"172.21.15.152:6811/1039375239","heartbeat_back_addr":"172.21.15.152:6815/1039375239","heartbeat_front_addr":"172.21.15.152:6813/1039375239","state":["exists","up"]},{"osd":6,"uuid":"61024437-7b91-432f-a1af-6443d0ef5213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6819","nonce":4080452707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6821","nonce":4080452707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6825","nonce":4080452707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6823","nonce":4080452707}]},"public_addr":"172.21.15.86:6819/4080452707","cluster_addr":"172.21.15.86:6821/4080452707","heartbeat_back_addr":"172.21.15.86:6825/4080452707","heartbeat_front_addr":"172.21.15.86:6823/4080452707","state":["exists","up"]},{"osd":7,"uuid":"2684bdae-6b30-4afc-af68-4ce41b3a75b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6817","nonce":4187707622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6819","nonce":4187707622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6823","nonce":4187707622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6821","nonce":4187707622}]},"public_addr":"172.21.15.92:6817/4187707622","cluster_addr":"172.21.15.92:6819/4187707622","heartbeat_back_addr":"172.21.15.92:6823/4187707622","heartbeat_front_addr":"172.21.15.92:6821/4187707622","state":["exists","up"]},{"osd":8,"uuid":"f03bbf0d-1281-4bcf-9773-9854388e65ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6817","nonce":3237612748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6819","nonce":3237612748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6823","nonce":3237612748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6821","nonce":3237612748}]},"public_addr":"172.21.15.152:6817/3237612748","cluster_addr":"172.21.15.152:6819/3237612748","heartbeat_back_addr":"172.21.15.152:6823/3237612748","heartbeat_front_addr":"172.21.15.152:6821/3237612748","state":["exists","up"]},{"osd":9,"uuid":"e7e8b775-99ff-48a4-a23a-91e7ab6622bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6827","nonce":2835218294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6829","nonce":2835218294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6833","nonce":2835218294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6831","nonce":2835218294}]},"public_addr":"172.21.15.86:6827/2835218294","cluster_addr":"172.21.15.86:6829/2835218294","heartbeat_back_addr":"172.21.15.86:6833/2835218294","heartbeat_front_addr":"172.21.15.86:6831/2835218294","state":["exists","up"]},{"osd":10,"uuid":"d9d616cd-a229-44de-8768-02a4415df115","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6825","nonce":4259465100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6827","nonce":4259465100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6831","nonce":4259465100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6829","nonce":4259465100}]},"public_addr":"172.21.15.92:6825/4259465100","cluster_addr":"172.21.15.92:6827/4259465100","heartbeat_back_addr":"172.21.15.92:6831/4259465100","heartbeat_front_addr":"172.21.15.92:6829/4259465100","state":["exists","up"]},{"osd":11,"uuid":"e7c0b865-ccf5-4233-8d22-0d9b6dbe5c97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6825","nonce":194662760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6827","nonce":194662760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6831","nonce":194662760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6829","nonce":194662760}]},"public_addr":"172.21.15.152:6825/194662760","cluster_addr":"172.21.15.152:6827/194662760","heartbeat_back_addr":"172.21.15.152:6831/194662760","heartbeat_front_addr":"172.21.15.152:6829/194662760","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:44:47.269167+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:45:05.363655+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:45:23.192677+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:45:44.156228+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:02.532971+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:20.153295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:41.713888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:59.676516+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:47:17.207087+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:47:38.970316+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:47:57.282525+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:48:15.437606+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/3553500912":"2024-03-24T17:43:28.242492+0000","172.21.15.86:6801/3600278228":"2024-03-24T17:43:28.242492+0000","172.21.15.86:0/887262329":"2024-03-24T17:43:28.242492+0000","172.21.15.86:6800/2313723492":"2024-03-24T17:43:05.043910+0000","172.21.15.86:6800/3600278228":"2024-03-24T17:43:28.242492+0000","172.21.15.86:0/3815705642":"2024-03-24T17:43:05.043910+0000","172.21.15.86:0/1614943062":"2024-03-24T17:43:28.242492+0000","172.21.15.86:6801/2313723492":"2024-03-24T17:43:05.043910+0000","172.21.15.86:0/2416321716":"2024-03-24T17:43:05.043910+0000","172.21.15.86:0/433588188":"2024-03-24T17:43:05.043910+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T17:50:29.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.006+0000 7f11f4400700 1 -- 172.21.15.86:0/4100576536 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f11d40a11b0 msgr2=0x7f11d40a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:29.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.006+0000 7f11f4400700 1 --2- 172.21.15.86:0/4100576536 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f11d40a11b0 0x7f11d40a3670 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f11e0009fd0 tx=0x7f11e0009260 comp rx=0 tx=0).stop 2024-03-23T17:50:29.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.006+0000 7f11f4400700 1 -- 172.21.15.86:0/4100576536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11ec108b50 msgr2=0x7f11ec1a03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:29.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.006+0000 7f11f4400700 1 --2- 172.21.15.86:0/4100576536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11ec108b50 0x7f11ec1a03a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f11e400ef30 tx=0x7f11e400c3a0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.006+0000 7f11f4400700 1 -- 172.21.15.86:0/4100576536 shutdown_connections 2024-03-23T17:50:29.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.006+0000 7f11f4400700 1 --2- 172.21.15.86:0/4100576536 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f11d40a11b0 0x7f11d40a3670 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.006+0000 7f11f4400700 1 --2- 172.21.15.86:0/4100576536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11ec108b50 0x7f11ec1a03a0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.006+0000 7f11f4400700 1 --2- 172.21.15.86:0/4100576536 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11ec0737b0 0x7f11ec198e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.006+0000 7f11f4400700 1 --2- 172.21.15.86:0/4100576536 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f11ec074d90 0x7f11ec199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.006+0000 7f11f4400700 1 -- 172.21.15.86:0/4100576536 >> 172.21.15.86:0/4100576536 conn(0x7f11ec0fc1f0 msgr2=0x7f11ec104aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:29.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.007+0000 7f11f4400700 1 -- 172.21.15.86:0/4100576536 shutdown_connections 2024-03-23T17:50:29.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.007+0000 7f11f4400700 1 -- 172.21.15.86:0/4100576536 wait complete. 2024-03-23T17:50:29.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:28 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/4069304465' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T17:50:29.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:28 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/3238032736' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T17:50:29.026 DEBUG:tasks.fwd_scrub:Yielding 2024-03-23T17:50:29.027 INFO:teuthology.run_tasks:Running task exec... 2024-03-23T17:50:29.040 INFO:teuthology.task.exec:Executing custom commands... 2024-03-23T17:50:29.040 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi086.front.sepia.ceph.com 2024-03-23T17:50:29.040 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-03-23T17:50:29.041 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-03-23T17:50:29.041 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-23T17:50:29.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T17:50:29.041 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-23T17:50:29.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:28 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/4069304465' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T17:50:29.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:28 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/3238032736' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T17:50:29.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:28 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/4069304465' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T17:50:29.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:28 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/3238032736' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T17:50:29.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.233+0000 7f1455600700 1 -- 172.21.15.86:0/1586801583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1450073810 msgr2=0x7f1450073c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:29.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.233+0000 7f1455600700 1 --2- 172.21.15.86:0/1586801583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1450073810 0x7f1450073c90 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f143c009990 tx=0x7f143c009ca0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.233+0000 7f1455600700 1 -- 172.21.15.86:0/1586801583 shutdown_connections 2024-03-23T17:50:29.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.233+0000 7f1455600700 1 --2- 172.21.15.86:0/1586801583 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1450108ae0 0x7f145010af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.233+0000 7f1455600700 1 --2- 172.21.15.86:0/1586801583 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1450074df0 0x7f1450073240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.233+0000 7f1455600700 1 --2- 172.21.15.86:0/1586801583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1450073810 0x7f1450073c90 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.233+0000 7f1455600700 1 -- 172.21.15.86:0/1586801583 >> 172.21.15.86:0/1586801583 conn(0x7f14500fc190 msgr2=0x7f14500fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:29.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.233+0000 7f1455600700 1 -- 172.21.15.86:0/1586801583 shutdown_connections 2024-03-23T17:50:29.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.233+0000 7f1455600700 1 -- 172.21.15.86:0/1586801583 wait complete. 2024-03-23T17:50:29.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.235+0000 7f1455600700 1 Processor -- start 2024-03-23T17:50:29.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.235+0000 7f1455600700 1 -- start start 2024-03-23T17:50:29.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.235+0000 7f1455600700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1450073810 0x7f1450198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:29.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.235+0000 7f1455600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1450074df0 0x7f1450199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:29.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.235+0000 7f1455600700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1450108ae0 0x7f14501a0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:29.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.235+0000 7f1455600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14501a0950 con 0x7f1450074df0 2024-03-23T17:50:29.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.235+0000 7f1455600700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14501a0a90 con 0x7f1450108ae0 2024-03-23T17:50:29.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.235+0000 7f1455600700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14501a0c20 con 0x7f1450073810 2024-03-23T17:50:29.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.236+0000 7f144e000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1450074df0 0x7f1450199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:29.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.236+0000 7f144e000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1450074df0 0x7f1450199360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43188/0 (socket says 172.21.15.86:43188) 2024-03-23T17:50:29.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.236+0000 7f144e000700 1 -- 172.21.15.86:0/2876855440 learned_addr learned my addr 172.21.15.86:0/2876855440 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:29.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.236+0000 7f144ea00700 1 --2- 172.21.15.86:0/2876855440 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1450073810 0x7f1450198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:29.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.236+0000 7f144f400700 1 --2- 172.21.15.86:0/2876855440 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1450108ae0 0x7f14501a0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:29.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.236+0000 7f144e000700 1 -- 172.21.15.86:0/2876855440 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1450073810 msgr2=0x7f1450198e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:29.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.236+0000 7f144e000700 1 --2- 172.21.15.86:0/2876855440 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1450073810 0x7f1450198e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.236+0000 7f144e000700 1 -- 172.21.15.86:0/2876855440 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1450108ae0 msgr2=0x7f14501a0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:29.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.236+0000 7f144e000700 1 --2- 172.21.15.86:0/2876855440 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1450108ae0 0x7f14501a0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.236+0000 7f144e000700 1 -- 172.21.15.86:0/2876855440 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f143c0095f0 con 0x7f1450074df0 2024-03-23T17:50:29.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.236+0000 7f144e000700 1 --2- 172.21.15.86:0/2876855440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1450074df0 0x7f1450199360 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f143c00b680 tx=0x7f143c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:29.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.237+0000 7f1447400700 1 -- 172.21.15.86:0/2876855440 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143c01d070 con 0x7f1450074df0 2024-03-23T17:50:29.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.237+0000 7f1447400700 1 -- 172.21.15.86:0/2876855440 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f143c021410 con 0x7f1450074df0 2024-03-23T17:50:29.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.237+0000 7f1455600700 1 -- 172.21.15.86:0/2876855440 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14501a0ea0 con 0x7f1450074df0 2024-03-23T17:50:29.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.237+0000 7f1447400700 1 -- 172.21.15.86:0/2876855440 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143c017650 con 0x7f1450074df0 2024-03-23T17:50:29.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.237+0000 7f1455600700 1 -- 172.21.15.86:0/2876855440 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14501a1210 con 0x7f1450074df0 2024-03-23T17:50:29.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.238+0000 7f1455600700 1 -- 172.21.15.86:0/2876855440 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f145004eb60 con 0x7f1450074df0 2024-03-23T17:50:29.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.240+0000 7f1447400700 1 -- 172.21.15.86:0/2876855440 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f143c021580 con 0x7f1450074df0 2024-03-23T17:50:29.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.241+0000 7f1447400700 1 --2- 172.21.15.86:0/2876855440 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f14340a1110 0x7f14340a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:29.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.241+0000 7f1447400700 1 -- 172.21.15.86:0/2876855440 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f143c0ce140 con 0x7f1450074df0 2024-03-23T17:50:29.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.242+0000 7f144ea00700 1 --2- 172.21.15.86:0/2876855440 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f14340a1110 0x7f14340a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:29.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.242+0000 7f144ea00700 1 --2- 172.21.15.86:0/2876855440 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f14340a1110 0x7f14340a35d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1440009c60 tx=0x7f1440009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:29.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.245+0000 7f1447400700 1 -- 172.21.15.86:0/2876855440 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f143c097990 con 0x7f1450074df0 2024-03-23T17:50:29.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.246+0000 7f8119200700 1 -- 172.21.15.86:0/2751870356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8114100d80 msgr2=0x7f81141011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:29.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.246+0000 7f8119200700 1 --2- 172.21.15.86:0/2751870356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8114100d80 0x7f81141011a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f81040099c0 tx=0x7f8104009cd0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.246+0000 7f8119200700 1 -- 172.21.15.86:0/2751870356 shutdown_connections 2024-03-23T17:50:29.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.246+0000 7f8119200700 1 --2- 172.21.15.86:0/2751870356 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8114102910 0x7f81141091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.246+0000 7f8119200700 1 --2- 172.21.15.86:0/2751870356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8114100d80 0x7f81141011a0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.246+0000 7f8119200700 1 --2- 172.21.15.86:0/2751870356 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8114101f70 0x7f81141023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.246+0000 7f8119200700 1 -- 172.21.15.86:0/2751870356 >> 172.21.15.86:0/2751870356 conn(0x7f81140fc2f0 msgr2=0x7f81140fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:29.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.246+0000 7f8119200700 1 -- 172.21.15.86:0/2751870356 shutdown_connections 2024-03-23T17:50:29.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.246+0000 7f8119200700 1 -- 172.21.15.86:0/2751870356 wait complete. 2024-03-23T17:50:29.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.247+0000 7f8119200700 1 Processor -- start 2024-03-23T17:50:29.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.247+0000 7f8119200700 1 -- start start 2024-03-23T17:50:29.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.248+0000 7f8119200700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8114100d80 0x7f8114196d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:29.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.248+0000 7f8119200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8114101f70 0x7f81141972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:29.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.248+0000 7f8119200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8114102910 0x7f811419e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:29.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.248+0000 7f8119200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f811419e870 con 0x7f8114101f70 2024-03-23T17:50:29.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.248+0000 7f8119200700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f811419e9e0 con 0x7f8114102910 2024-03-23T17:50:29.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.248+0000 7f8119200700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f811419eba0 con 0x7f8114100d80 2024-03-23T17:50:29.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.248+0000 7f8111400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8114101f70 0x7f81141972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:29.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.248+0000 7f8111e00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8114100d80 0x7f8114196d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:29.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.248+0000 7f8111400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8114101f70 0x7f81141972c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43216/0 (socket says 172.21.15.86:43216) 2024-03-23T17:50:29.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.248+0000 7f8111400700 1 -- 172.21.15.86:0/3688356479 learned_addr learned my addr 172.21.15.86:0/3688356479 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:29.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.248+0000 7f8112800700 1 --2- 172.21.15.86:0/3688356479 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8114102910 0x7f811419e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:29.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.248+0000 7f8111400700 1 -- 172.21.15.86:0/3688356479 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8114100d80 msgr2=0x7f8114196d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:29.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.248+0000 7f8111400700 1 --2- 172.21.15.86:0/3688356479 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8114100d80 0x7f8114196d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.248+0000 7f8111400700 1 -- 172.21.15.86:0/3688356479 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8114102910 msgr2=0x7f811419e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:29.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.248+0000 7f8111400700 1 --2- 172.21.15.86:0/3688356479 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8114102910 0x7f811419e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.248+0000 7f8111400700 1 -- 172.21.15.86:0/3688356479 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81040095f0 con 0x7f8114101f70 2024-03-23T17:50:29.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.248+0000 7f8111e00700 1 --2- 172.21.15.86:0/3688356479 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8114100d80 0x7f8114196d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:50:29.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.249+0000 7f8111400700 1 --2- 172.21.15.86:0/3688356479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8114101f70 0x7f81141972c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f810000e9a0 tx=0x7f810000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:29.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.249+0000 7f810aa00700 1 -- 172.21.15.86:0/3688356479 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8100009790 con 0x7f8114101f70 2024-03-23T17:50:29.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.249+0000 7f810aa00700 1 -- 172.21.15.86:0/3688356479 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8100010430 con 0x7f8114101f70 2024-03-23T17:50:29.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.249+0000 7f8119200700 1 -- 172.21.15.86:0/3688356479 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f811419ee80 con 0x7f8114101f70 2024-03-23T17:50:29.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.249+0000 7f810aa00700 1 -- 172.21.15.86:0/3688356479 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8100018680 con 0x7f8114101f70 2024-03-23T17:50:29.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.249+0000 7f8119200700 1 -- 172.21.15.86:0/3688356479 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f811419f2a0 con 0x7f8114101f70 2024-03-23T17:50:29.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.252+0000 7f810aa00700 1 -- 172.21.15.86:0/3688356479 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f81000105a0 con 0x7f8114101f70 2024-03-23T17:50:29.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.253+0000 7f810aa00700 1 --2- 172.21.15.86:0/3688356479 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f80f80a1120 0x7f80f80a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:29.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.253+0000 7f810aa00700 1 -- 172.21.15.86:0/3688356479 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f8100014070 con 0x7f8114101f70 2024-03-23T17:50:29.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.253+0000 7f812cd67b80 1 Processor -- start 2024-03-23T17:50:29.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.253+0000 7f8111e00700 1 --2- 172.21.15.86:0/3688356479 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f80f80a1120 0x7f80f80a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:29.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.254+0000 7f812cd67b80 1 -- start start 2024-03-23T17:50:29.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.254+0000 7f812cd67b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627f2115850 0x5627f2115c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:29.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.254+0000 7f812cd67b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5627f21161b0 0x5627f2116a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:29.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.254+0000 7f812cd67b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627f2117050 0x5627f21174d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:29.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.254+0000 7f812cd67b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627f206da10 con 0x5627f2117050 2024-03-23T17:50:29.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.254+0000 7f812cd67b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627f206b3d0 con 0x5627f2115850 2024-03-23T17:50:29.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.254+0000 7f812cd67b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627f2066cc0 con 0x5627f21161b0 2024-03-23T17:50:29.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.254+0000 7f8111e00700 1 --2- 172.21.15.86:0/3688356479 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f80f80a1120 0x7f80f80a35e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8104009990 tx=0x7f810401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:29.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.254+0000 7f8112800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627f2117050 0x5627f21174d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:29.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.254+0000 7f8111e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627f2115850 0x5627f2115c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:29.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.255+0000 7f8111e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627f2115850 0x5627f2115c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:59114/0 (socket says 172.21.15.86:59114) 2024-03-23T17:50:29.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.255+0000 7f8111e00700 1 -- 172.21.15.86:0/3432547001 learned_addr learned my addr 172.21.15.86:0/3432547001 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:29.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.255+0000 7f8111400700 1 --2- 172.21.15.86:0/3432547001 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5627f21161b0 0x5627f2116a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:29.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.255+0000 7f8111e00700 1 -- 172.21.15.86:0/3432547001 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5627f21161b0 msgr2=0x5627f2116a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:50:29.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.255+0000 7f8111e00700 1 --2- 172.21.15.86:0/3432547001 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5627f21161b0 0x5627f2116a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.255+0000 7f8111e00700 1 -- 172.21.15.86:0/3432547001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627f2117050 msgr2=0x5627f21174d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:50:29.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.255+0000 7f8111e00700 1 --2- 172.21.15.86:0/3432547001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627f2117050 0x5627f21174d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.255+0000 7f8111e00700 1 -- 172.21.15.86:0/3432547001 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627f1ee8430 con 0x5627f2115850 2024-03-23T17:50:29.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.255+0000 7f8111e00700 1 --2- 172.21.15.86:0/3432547001 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627f2115850 0x5627f2115c70 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8114101dd0 tx=0x7f810401f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ad1a49367d2607b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:29.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.255+0000 7f80ff400700 1 -- 172.21.15.86:0/3432547001 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8104005240 con 0x5627f2115850 2024-03-23T17:50:29.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.256+0000 7f80ff400700 1 -- 172.21.15.86:0/3432547001 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f810401e040 con 0x5627f2115850 2024-03-23T17:50:29.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.256+0000 7f80ff400700 1 -- 172.21.15.86:0/3432547001 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f810400fe10 con 0x5627f2115850 2024-03-23T17:50:29.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.256+0000 7f812cd67b80 1 -- 172.21.15.86:0/3432547001 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627f2115850 msgr2=0x5627f2115c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:29.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.256+0000 7f812cd67b80 1 --2- 172.21.15.86:0/3432547001 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627f2115850 0x5627f2115c70 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8114101dd0 tx=0x7f810401f000 comp rx=0 tx=0).stop 2024-03-23T17:50:29.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.256+0000 7f812cd67b80 1 -- 172.21.15.86:0/3432547001 shutdown_connections 2024-03-23T17:50:29.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.256+0000 7f812cd67b80 1 --2- 172.21.15.86:0/3432547001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627f2117050 0x5627f21174d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.256+0000 7f812cd67b80 1 --2- 172.21.15.86:0/3432547001 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627f2115850 0x5627f2115c70 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.256+0000 7f812cd67b80 1 --2- 172.21.15.86:0/3432547001 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5627f21161b0 0x5627f2116a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.256+0000 7f812cd67b80 1 -- 172.21.15.86:0/3432547001 >> 172.21.15.86:0/3432547001 conn(0x5627f2074720 msgr2=0x5627f2110a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:29.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.256+0000 7f812cd67b80 1 -- 172.21.15.86:0/3432547001 shutdown_connections 2024-03-23T17:50:29.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.256+0000 7f812cd67b80 1 -- 172.21.15.86:0/3432547001 wait complete. 2024-03-23T17:50:29.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.257+0000 7f812cd67b80 1 -- start start 2024-03-23T17:50:29.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.257+0000 7f812cd67b80 1 Processor -- start 2024-03-23T17:50:29.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.257+0000 7f812cd67b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627f2115850 0x5627f211f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:29.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.257+0000 7f812cd67b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627f21161b0 0x5627f211fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:29.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.258+0000 7f812cd67b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5627f2117050 0x5627f2126be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:29.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.258+0000 7f812cd67b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627f2002e00 con 0x5627f21161b0 2024-03-23T17:50:29.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.258+0000 7f812cd67b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627f2002aa0 con 0x5627f2115850 2024-03-23T17:50:29.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.258+0000 7f812cd67b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627f1eea720 con 0x5627f2117050 2024-03-23T17:50:29.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.258+0000 7f812cd67b80 10 client.0 _ll_register_callbacks cb 0x5627f200c4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:50:29.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.258+0000 7f8111e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627f2115850 0x5627f211f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:29.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.258+0000 7f8111400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627f21161b0 0x5627f211fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:29.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.258+0000 7f8111e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627f2115850 0x5627f211f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:59122/0 (socket says 172.21.15.86:59122) 2024-03-23T17:50:29.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.258+0000 7f8111e00700 1 -- 172.21.15.86:0/3190990322 learned_addr learned my addr 172.21.15.86:0/3190990322 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:29.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.258+0000 7f8112800700 1 --2- 172.21.15.86:0/3190990322 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5627f2117050 0x5627f2126be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:29.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.258+0000 7f8111e00700 1 -- 172.21.15.86:0/3190990322 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5627f2117050 msgr2=0x5627f2126be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:50:29.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.258+0000 7f8111e00700 1 --2- 172.21.15.86:0/3190990322 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5627f2117050 0x5627f2126be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.258+0000 7f8111e00700 1 -- 172.21.15.86:0/3190990322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627f21161b0 msgr2=0x5627f211fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:50:29.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.258+0000 7f8111e00700 1 --2- 172.21.15.86:0/3190990322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627f21161b0 0x5627f211fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.258+0000 7f8111e00700 1 -- 172.21.15.86:0/3190990322 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8114109b10 con 0x5627f2115850 2024-03-23T17:50:29.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.259+0000 7f8111e00700 1 --2- 172.21.15.86:0/3190990322 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627f2115850 0x5627f211f660 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8104005f50 tx=0x7f810400fe80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8050c94d6e5ee66b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:29.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.259+0000 7f80f3e00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T17:50:29.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.259+0000 7f80f3e00700 1 -- 172.21.15.86:0/3190990322 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8104005240 con 0x5627f2115850 2024-03-23T17:50:29.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.259+0000 7f812cd67b80 1 -- 172.21.15.86:0/3190990322 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5627f2027400 con 0x5627f2115850 2024-03-23T17:50:29.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.259+0000 7f80f3e00700 1 -- 172.21.15.86:0/3190990322 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f81040053a0 con 0x5627f2115850 2024-03-23T17:50:29.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.259+0000 7f80f3e00700 1 -- 172.21.15.86:0/3190990322 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f810400f7f0 con 0x5627f2115850 2024-03-23T17:50:29.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.259+0000 7f812cd67b80 10 client.25376 fetch_fsmap learned FSMap version 28 2024-03-23T17:50:29.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.259+0000 7f812cd67b80 1 -- 172.21.15.86:0/3190990322 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5627f1e94be0 con 0x5627f2115850 2024-03-23T17:50:29.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.260+0000 7f80f3e00700 1 -- 172.21.15.86:0/3190990322 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5627f2027400 con 0x5627f2115850 2024-03-23T17:50:29.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.260+0000 7f80f3e00700 1 -- 172.21.15.86:0/3190990322 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 28) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8104031430 con 0x5627f2115850 2024-03-23T17:50:29.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.260+0000 7f812cd67b80 10 client.25376 fetch_fsmap finished waiting for FSMap version 28 2024-03-23T17:50:29.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.260+0000 7f812cd67b80 10 client.25376 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T17:50:29.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.260+0000 7f812cd67b80 20 client.25376 populate_metadata read hostname 'smithi086' 2024-03-23T17:50:29.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.260+0000 7f812cd67b80 1 --2- 172.21.15.86:0/3190990322 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5627f212aef0 0x5627f212d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:29.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.260+0000 7f812cd67b80 4 client.25376 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:50:29.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.260+0000 7f812cd67b80 1 -- 172.21.15.86:0/3190990322 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5627f205f780 con 0x5627f212aef0 2024-03-23T17:50:29.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.260+0000 7f8111400700 1 --2- 172.21.15.86:0/3190990322 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5627f212aef0 0x5627f212d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:29.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.261+0000 7f8111400700 1 --2- 172.21.15.86:0/3190990322 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5627f212aef0 0x5627f212d310 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=860a993f2b7dde3f server_cookie=35b164c9cbdb3ac4 in_seq=0 out_seq=0 2024-03-23T17:50:29.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.261+0000 7f80f3e00700 10 client.25376 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:50:29.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.261+0000 7f80f3e00700 1 -- 172.21.15.86:0/3190990322 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5627f205f780 con 0x5627f212aef0 2024-03-23T17:50:29.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.261+0000 7f80f3e00700 10 client.25376 handle_command_reply: tid=0 2024-03-23T17:50:29.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.261+0000 7f812cd67b80 1 client.25376 shutdown 2024-03-23T17:50:29.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.261+0000 7f812cd67b80 1 -- 172.21.15.86:0/3190990322 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627f2115850 msgr2=0x5627f211f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:29.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.262+0000 7f812cd67b80 1 --2- 172.21.15.86:0/3190990322 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627f2115850 0x5627f211f660 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8104005f50 tx=0x7f810400fe80 comp rx=0 tx=0).stop 2024-03-23T17:50:29.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.262+0000 7f812cd67b80 1 -- 172.21.15.86:0/3190990322 shutdown_connections 2024-03-23T17:50:29.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.262+0000 7f812cd67b80 1 --2- 172.21.15.86:0/3190990322 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5627f212aef0 0x5627f212d310 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.262+0000 7f812cd67b80 1 --2- 172.21.15.86:0/3190990322 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5627f2117050 0x5627f2126be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.262+0000 7f812cd67b80 1 --2- 172.21.15.86:0/3190990322 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627f2115850 0x5627f211f660 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.262+0000 7f812cd67b80 1 --2- 172.21.15.86:0/3190990322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627f21161b0 0x5627f211fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.262+0000 7f812cd67b80 1 -- 172.21.15.86:0/3190990322 >> 172.21.15.86:0/3190990322 conn(0x5627f2074720 msgr2=0x5627f21141f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:29.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.262+0000 7f80f3e00700 0 client.25376 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:50:29.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.262+0000 7f812cd67b80 1 -- 172.21.15.86:0/3190990322 shutdown_connections 2024-03-23T17:50:29.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.262+0000 7f812cd67b80 1 -- 172.21.15.86:0/3190990322 wait complete. 2024-03-23T17:50:29.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.262+0000 7f812cd67b80 20 client.25376 trim_cache size 0 max 16384 2024-03-23T17:50:29.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.279+0000 7f812cd67b80 1 Processor -- start 2024-03-23T17:50:29.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.279+0000 7f812cd67b80 1 -- start start 2024-03-23T17:50:29.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.279+0000 7f812cd67b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627f2115850 0x5627f2124b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:29.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.279+0000 7f812cd67b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627f21161b0 0x5627f211a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:29.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.279+0000 7f812cd67b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5627f2117050 0x5627f2110610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:29.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.279+0000 7f812cd67b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627f2127d60 con 0x5627f2115850 2024-03-23T17:50:29.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.279+0000 7f812cd67b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627f2120690 con 0x5627f21161b0 2024-03-23T17:50:29.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.279+0000 7f812cd67b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627f2116590 con 0x5627f2117050 2024-03-23T17:50:29.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.279+0000 7f8111e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627f2115850 0x5627f2124b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:29.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.279+0000 7f8111e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627f2115850 0x5627f2124b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43246/0 (socket says 172.21.15.86:43246) 2024-03-23T17:50:29.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.279+0000 7f8111e00700 1 -- 172.21.15.86:0/1852433572 learned_addr learned my addr 172.21.15.86:0/1852433572 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:29.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.279+0000 7f8111400700 1 --2- 172.21.15.86:0/1852433572 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627f21161b0 0x5627f211a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:29.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.280+0000 7f8112800700 1 --2- 172.21.15.86:0/1852433572 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5627f2117050 0x5627f2110610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:29.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.280+0000 7f8111e00700 1 -- 172.21.15.86:0/1852433572 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5627f2117050 msgr2=0x5627f2110610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:50:29.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.280+0000 7f8111e00700 1 --2- 172.21.15.86:0/1852433572 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5627f2117050 0x5627f2110610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.280+0000 7f8111e00700 1 -- 172.21.15.86:0/1852433572 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627f21161b0 msgr2=0x5627f211a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:50:29.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.280+0000 7f8111e00700 1 --2- 172.21.15.86:0/1852433572 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627f21161b0 0x5627f211a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.280+0000 7f8111e00700 1 -- 172.21.15.86:0/1852433572 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627f202cf20 con 0x5627f2115850 2024-03-23T17:50:29.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.280+0000 7f8111400700 1 --2- 172.21.15.86:0/1852433572 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627f21161b0 0x5627f211a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:50:29.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.280+0000 7f8111e00700 1 --2- 172.21.15.86:0/1852433572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627f2115850 0x5627f2124b90 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f8104009d10 tx=0x7f8104031960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de59f4274df43e7c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:29.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.280+0000 7f80ffe00700 1 -- 172.21.15.86:0/1852433572 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f810400fa40 con 0x5627f2115850 2024-03-23T17:50:29.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.280+0000 7f80ffe00700 1 -- 172.21.15.86:0/1852433572 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f810400f7f0 con 0x5627f2115850 2024-03-23T17:50:29.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.280+0000 7f80ffe00700 1 -- 172.21.15.86:0/1852433572 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81040416e0 con 0x5627f2115850 2024-03-23T17:50:29.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.281+0000 7f812cd67b80 1 -- 172.21.15.86:0/1852433572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627f2115850 msgr2=0x5627f2124b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:29.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.281+0000 7f812cd67b80 1 --2- 172.21.15.86:0/1852433572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627f2115850 0x5627f2124b90 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f8104009d10 tx=0x7f8104031960 comp rx=0 tx=0).stop 2024-03-23T17:50:29.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.281+0000 7f812cd67b80 1 -- 172.21.15.86:0/1852433572 shutdown_connections 2024-03-23T17:50:29.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.281+0000 7f812cd67b80 1 --2- 172.21.15.86:0/1852433572 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5627f2117050 0x5627f2110610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.281+0000 7f812cd67b80 1 --2- 172.21.15.86:0/1852433572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627f2115850 0x5627f2124b90 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.281+0000 7f812cd67b80 1 --2- 172.21.15.86:0/1852433572 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627f21161b0 0x5627f211a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.281+0000 7f812cd67b80 1 -- 172.21.15.86:0/1852433572 >> 172.21.15.86:0/1852433572 conn(0x5627f2074720 msgr2=0x5627f2109620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:29.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.281+0000 7f812cd67b80 1 -- 172.21.15.86:0/1852433572 shutdown_connections 2024-03-23T17:50:29.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.281+0000 7f812cd67b80 1 -- 172.21.15.86:0/1852433572 wait complete. 2024-03-23T17:50:29.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.282+0000 7f812cd67b80 1 -- start start 2024-03-23T17:50:29.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.282+0000 7f812cd67b80 1 Processor -- start 2024-03-23T17:50:29.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.282+0000 7f812cd67b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627f2115850 0x5627f21755b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:29.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.282+0000 7f812cd67b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627f21161b0 0x5627f2175af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:29.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.282+0000 7f812cd67b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5627f2117050 0x5627f217ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:29.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.282+0000 7f812cd67b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627f217cf80 con 0x5627f21161b0 2024-03-23T17:50:29.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.282+0000 7f812cd67b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627f217d0c0 con 0x5627f2115850 2024-03-23T17:50:29.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.282+0000 7f812cd67b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627f217d250 con 0x5627f2117050 2024-03-23T17:50:29.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.282+0000 7f812cd67b80 10 client.0 _ll_register_callbacks cb 0x5627f200c4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:50:29.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.283+0000 7f8111400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627f21161b0 0x5627f2175af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:29.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.283+0000 7f8111400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627f21161b0 0x5627f2175af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43258/0 (socket says 172.21.15.86:43258) 2024-03-23T17:50:29.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.283+0000 7f8111400700 1 -- 172.21.15.86:0/939484960 learned_addr learned my addr 172.21.15.86:0/939484960 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:29.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.283+0000 7f8112800700 1 --2- 172.21.15.86:0/939484960 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5627f2117050 0x5627f217ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:29.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.283+0000 7f8111e00700 1 --2- 172.21.15.86:0/939484960 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627f2115850 0x5627f21755b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:29.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.283+0000 7f8111400700 1 -- 172.21.15.86:0/939484960 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5627f2117050 msgr2=0x5627f217ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:50:29.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.283+0000 7f8111400700 1 --2- 172.21.15.86:0/939484960 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5627f2117050 0x5627f217ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.283+0000 7f8111400700 1 -- 172.21.15.86:0/939484960 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627f2115850 msgr2=0x5627f21755b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:50:29.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.283+0000 7f8111400700 1 --2- 172.21.15.86:0/939484960 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627f2115850 0x5627f21755b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.283+0000 7f8111400700 1 -- 172.21.15.86:0/939484960 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f811419f2a0 con 0x5627f21161b0 2024-03-23T17:50:29.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.283+0000 7f8111400700 1 --2- 172.21.15.86:0/939484960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627f21161b0 0x5627f2175af0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f8100096960 tx=0x7f81000ca470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3169a765597153d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:29.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.283+0000 7f80f3e00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-23T17:50:29.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.283+0000 7f80f3e00700 1 -- 172.21.15.86:0/939484960 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81000b8300 con 0x5627f21161b0 2024-03-23T17:50:29.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.284+0000 7f80f3e00700 1 -- 172.21.15.86:0/939484960 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f81000a2370 con 0x5627f21161b0 2024-03-23T17:50:29.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.284+0000 7f812cd67b80 1 -- 172.21.15.86:0/939484960 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f810401e040 con 0x5627f21161b0 2024-03-23T17:50:29.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.284+0000 7f80f3e00700 1 -- 172.21.15.86:0/939484960 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8100010880 con 0x5627f21161b0 2024-03-23T17:50:29.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.284+0000 7f812cd67b80 10 client.15342 fetch_fsmap learned FSMap version 28 2024-03-23T17:50:29.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.284+0000 7f812cd67b80 1 -- 172.21.15.86:0/939484960 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5627f1e94be0 con 0x5627f21161b0 2024-03-23T17:50:29.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.284+0000 7f80f3e00700 1 -- 172.21.15.86:0/939484960 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f810401e040 con 0x5627f21161b0 2024-03-23T17:50:29.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.284+0000 7f80f3e00700 1 -- 172.21.15.86:0/939484960 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f81000a1ee0 con 0x5627f21161b0 2024-03-23T17:50:29.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.284+0000 7f812cd67b80 10 client.15342 fetch_fsmap finished waiting for FSMap version 28 2024-03-23T17:50:29.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.284+0000 7f812cd67b80 10 client.15342 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T17:50:29.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.284+0000 7f812cd67b80 20 client.15342 populate_metadata read hostname 'smithi086' 2024-03-23T17:50:29.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.284+0000 7f812cd67b80 1 --2- 172.21.15.86:0/939484960 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5627f217eca0 0x5627f2181090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:29.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.284+0000 7f812cd67b80 4 client.15342 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-23T17:50:29.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.284+0000 7f812cd67b80 1 -- 172.21.15.86:0/939484960 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5627f1eb5d70 con 0x5627f217eca0 2024-03-23T17:50:29.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.284+0000 7f8111e00700 1 --2- 172.21.15.86:0/939484960 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5627f217eca0 0x5627f2181090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:29.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.285+0000 7f8111e00700 1 --2- 172.21.15.86:0/939484960 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5627f217eca0 0x5627f2181090 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc96a2fa98f45c31 server_cookie=b0134ad544e1a404 in_seq=0 out_seq=0 2024-03-23T17:50:29.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.285+0000 7f80f3e00700 10 client.15342 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:50:29.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.286+0000 7f80f3e00700 1 -- 172.21.15.86:0/939484960 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5627f1eb5d70 con 0x5627f217eca0 2024-03-23T17:50:29.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.286+0000 7f80f3e00700 10 client.15342 handle_command_reply: tid=0 2024-03-23T17:50:29.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.286+0000 7f812cd67b80 1 client.15342 shutdown 2024-03-23T17:50:29.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.286+0000 7f812cd67b80 1 -- 172.21.15.86:0/939484960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627f21161b0 msgr2=0x5627f2175af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:29.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.286+0000 7f812cd67b80 1 --2- 172.21.15.86:0/939484960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627f21161b0 0x5627f2175af0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f8100096960 tx=0x7f81000ca470 comp rx=0 tx=0).stop 2024-03-23T17:50:29.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.286+0000 7f812cd67b80 1 -- 172.21.15.86:0/939484960 shutdown_connections 2024-03-23T17:50:29.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.286+0000 7f812cd67b80 1 --2- 172.21.15.86:0/939484960 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5627f217eca0 0x5627f2181090 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.286+0000 7f812cd67b80 1 --2- 172.21.15.86:0/939484960 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5627f2117050 0x5627f217ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.286+0000 7f812cd67b80 1 --2- 172.21.15.86:0/939484960 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5627f2115850 0x5627f21755b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.286+0000 7f80f3e00700 0 client.15342 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:50:29.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.286+0000 7f812cd67b80 1 --2- 172.21.15.86:0/939484960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627f21161b0 0x5627f2175af0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.286+0000 7f812cd67b80 1 -- 172.21.15.86:0/939484960 >> 172.21.15.86:0/939484960 conn(0x5627f2074720 msgr2=0x5627f211a260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:29.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.287+0000 7f812cd67b80 1 -- 172.21.15.86:0/939484960 shutdown_connections 2024-03-23T17:50:29.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.287+0000 7f812cd67b80 1 -- 172.21.15.86:0/939484960 wait complete. 2024-03-23T17:50:29.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.287+0000 7f812cd67b80 20 client.15342 trim_cache size 0 max 16384 2024-03-23T17:50:29.305 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-23T17:50:29.305 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-03-23T17:50:29.305 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "a3652b6e-9e12-4522-886d-7c43e45a2e9c", 2024-03-23T17:50:29.305 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-03-23T17:50:29.305 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-23T17:50:29.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.287+0000 7f8119200700 1 -- 172.21.15.86:0/3688356479 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f80f80a1120 msgr2=0x7f80f80a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:29.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.287+0000 7f8119200700 1 --2- 172.21.15.86:0/3688356479 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f80f80a1120 0x7f80f80a35e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8104009990 tx=0x7f810401a040 comp rx=0 tx=0).stop 2024-03-23T17:50:29.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.287+0000 7f8119200700 1 -- 172.21.15.86:0/3688356479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8114101f70 msgr2=0x7f81141972c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:29.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.287+0000 7f8119200700 1 --2- 172.21.15.86:0/3688356479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8114101f70 0x7f81141972c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f810000e9a0 tx=0x7f810000ecb0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.288+0000 7f8119200700 1 -- 172.21.15.86:0/3688356479 shutdown_connections 2024-03-23T17:50:29.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.288+0000 7f8119200700 1 --2- 172.21.15.86:0/3688356479 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8114102910 0x7f811419e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.288+0000 7f8119200700 1 --2- 172.21.15.86:0/3688356479 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8114100d80 0x7f8114196d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.288+0000 7f8119200700 1 --2- 172.21.15.86:0/3688356479 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f80f80a1120 0x7f80f80a35e0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.288+0000 7f8119200700 1 --2- 172.21.15.86:0/3688356479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8114101f70 0x7f81141972c0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.288+0000 7f8119200700 1 -- 172.21.15.86:0/3688356479 >> 172.21.15.86:0/3688356479 conn(0x7f81140fc2f0 msgr2=0x7f8114105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:29.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.288+0000 7f8119200700 1 -- 172.21.15.86:0/3688356479 shutdown_connections 2024-03-23T17:50:29.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.288+0000 7f8119200700 1 -- 172.21.15.86:0/3688356479 wait complete. 2024-03-23T17:50:29.309 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-23T17:50:29.310 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-23T17:50:29.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.505+0000 7f6fbfe00700 1 -- 172.21.15.86:0/3606062245 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fb8073810 msgr2=0x7f6fb8073c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:29.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.505+0000 7f6fbfe00700 1 --2- 172.21.15.86:0/3606062245 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fb8073810 0x7f6fb8073c90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6fa4009990 tx=0x7f6fa4009ca0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.505+0000 7f6fbfe00700 1 -- 172.21.15.86:0/3606062245 shutdown_connections 2024-03-23T17:50:29.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.505+0000 7f6fbfe00700 1 --2- 172.21.15.86:0/3606062245 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6fb810d350 0x7f6fb810f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.505+0000 7f6fbfe00700 1 --2- 172.21.15.86:0/3606062245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fb8074df0 0x7f6fb8073240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.505+0000 7f6fbfe00700 1 --2- 172.21.15.86:0/3606062245 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fb8073810 0x7f6fb8073c90 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.505+0000 7f6fbfe00700 1 -- 172.21.15.86:0/3606062245 >> 172.21.15.86:0/3606062245 conn(0x7f6fb80fc2f0 msgr2=0x7f6fb80fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:29.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.505+0000 7f6fbfe00700 1 -- 172.21.15.86:0/3606062245 shutdown_connections 2024-03-23T17:50:29.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.505+0000 7f6fbfe00700 1 -- 172.21.15.86:0/3606062245 wait complete. 2024-03-23T17:50:29.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.506+0000 7f6fbfe00700 1 Processor -- start 2024-03-23T17:50:29.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.506+0000 7f6fbfe00700 1 -- start start 2024-03-23T17:50:29.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.506+0000 7f6fbfe00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6fb8073810 0x7f6fb819d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:29.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.506+0000 7f6fbfe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fb8074df0 0x7f6fb819d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:29.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.506+0000 7f6fbfe00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fb810d350 0x7f6fb81a48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:29.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.507+0000 7f6fbfe00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb81a4e70 con 0x7f6fb8074df0 2024-03-23T17:50:29.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.507+0000 7f6fbfe00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb81a4fb0 con 0x7f6fb810d350 2024-03-23T17:50:29.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.507+0000 7f6fbfe00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb81a5140 con 0x7f6fb8073810 2024-03-23T17:50:29.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.507+0000 7f6fbca00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fb8074df0 0x7f6fb819d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:29.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.507+0000 7f6fbde00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fb810d350 0x7f6fb81a48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:29.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.507+0000 7f6fbca00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fb8074df0 0x7f6fb819d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43282/0 (socket says 172.21.15.86:43282) 2024-03-23T17:50:29.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.507+0000 7f6fbca00700 1 -- 172.21.15.86:0/3148530065 learned_addr learned my addr 172.21.15.86:0/3148530065 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:29.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.507+0000 7f6fbd400700 1 --2- 172.21.15.86:0/3148530065 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6fb8073810 0x7f6fb819d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:29.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.507+0000 7f6fbca00700 1 -- 172.21.15.86:0/3148530065 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6fb8073810 msgr2=0x7f6fb819d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:29.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.507+0000 7f6fbca00700 1 --2- 172.21.15.86:0/3148530065 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6fb8073810 0x7f6fb819d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.507+0000 7f6fbca00700 1 -- 172.21.15.86:0/3148530065 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fb810d350 msgr2=0x7f6fb81a48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:29.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.507+0000 7f6fbca00700 1 --2- 172.21.15.86:0/3148530065 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fb810d350 0x7f6fb81a48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.507+0000 7f6fbca00700 1 -- 172.21.15.86:0/3148530065 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fac009520 con 0x7f6fb8074df0 2024-03-23T17:50:29.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.507+0000 7f6fbde00700 1 --2- 172.21.15.86:0/3148530065 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fb810d350 0x7f6fb81a48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:50:29.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.508+0000 7f6fbca00700 1 --2- 172.21.15.86:0/3148530065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fb8074df0 0x7f6fb819d880 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f6fa4000c00 tx=0x7f6fa4009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:29.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.508+0000 7f6fb6000700 1 -- 172.21.15.86:0/3148530065 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa401d070 con 0x7f6fb8074df0 2024-03-23T17:50:29.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.508+0000 7f6fb6000700 1 -- 172.21.15.86:0/3148530065 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6fa4021410 con 0x7f6fb8074df0 2024-03-23T17:50:29.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.508+0000 7f6fbfe00700 1 -- 172.21.15.86:0/3148530065 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fa40095f0 con 0x7f6fb8074df0 2024-03-23T17:50:29.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.508+0000 7f6fb6000700 1 -- 172.21.15.86:0/3148530065 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa4017650 con 0x7f6fb8074df0 2024-03-23T17:50:29.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.508+0000 7f6fbfe00700 1 -- 172.21.15.86:0/3148530065 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fb81a5650 con 0x7f6fb8074df0 2024-03-23T17:50:29.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.511+0000 7f6fb6000700 1 -- 172.21.15.86:0/3148530065 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f6fa4021580 con 0x7f6fb8074df0 2024-03-23T17:50:29.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.512+0000 7f6fb6000700 1 --2- 172.21.15.86:0/3148530065 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f6fa00a1120 0x7f6fa00a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:29.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.512+0000 7f6fd3a12b80 1 Processor -- start 2024-03-23T17:50:29.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.512+0000 7f6fb6000700 1 -- 172.21.15.86:0/3148530065 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f6fa40cdaf0 con 0x7f6fb8074df0 2024-03-23T17:50:29.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.512+0000 7f6fd3a12b80 1 -- start start 2024-03-23T17:50:29.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.513+0000 7f6fbd400700 1 --2- 172.21.15.86:0/3148530065 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f6fa00a1120 0x7f6fa00a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:29.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.513+0000 7f6fd3a12b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2e20fa7d0 0x55b2e20fabf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:29.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.513+0000 7f6fd3a12b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b2e20fb130 0x55b2e20fb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:29.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.513+0000 7f6fd3a12b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b2e20fbfd0 0x55b2e20fc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:29.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.513+0000 7f6fd3a12b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2e2052990 con 0x55b2e20fa7d0 2024-03-23T17:50:29.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.513+0000 7f6fd3a12b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2e2050350 con 0x55b2e20fb130 2024-03-23T17:50:29.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.513+0000 7f6fd3a12b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2e204bc40 con 0x55b2e20fbfd0 2024-03-23T17:50:29.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.513+0000 7f6fbd400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2e20fa7d0 0x55b2e20fabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:29.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.513+0000 7f6fbd400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2e20fa7d0 0x55b2e20fabf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43298/0 (socket says 172.21.15.86:43298) 2024-03-23T17:50:29.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.513+0000 7f6fbd400700 1 -- 172.21.15.86:0/792603182 learned_addr learned my addr 172.21.15.86:0/792603182 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:29.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.513+0000 7f6fbca00700 1 --2- 172.21.15.86:0/792603182 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b2e20fb130 0x55b2e20fb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:29.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.513+0000 7f6fbde00700 1 --2- 172.21.15.86:0/792603182 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b2e20fbfd0 0x55b2e20fc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:29.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.514+0000 7f6fbca00700 1 -- 172.21.15.86:0/792603182 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b2e20fbfd0 msgr2=0x55b2e20fc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:50:29.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.514+0000 7f6fbca00700 1 --2- 172.21.15.86:0/792603182 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b2e20fbfd0 0x55b2e20fc450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.514+0000 7f6fbca00700 1 -- 172.21.15.86:0/792603182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2e20fa7d0 msgr2=0x55b2e20fabf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:50:29.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.514+0000 7f6fbca00700 1 --2- 172.21.15.86:0/792603182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2e20fa7d0 0x55b2e20fabf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.514+0000 7f6fbca00700 1 -- 172.21.15.86:0/792603182 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b2e1ecd3b0 con 0x55b2e20fb130 2024-03-23T17:50:29.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.514+0000 7f6fbd400700 1 --2- 172.21.15.86:0/3148530065 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f6fa00a1120 0x7f6fa00a35e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6fac009fd0 tx=0x7f6fac013ae0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:29.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.514+0000 7f6fbde00700 1 --2- 172.21.15.86:0/792603182 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b2e20fbfd0 0x55b2e20fc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:50:29.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.514+0000 7f6fbca00700 1 --2- 172.21.15.86:0/792603182 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b2e20fb130 0x55b2e20fb9f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6fa400b680 tx=0x7f6fa409a960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a60656e73c640be server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:29.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.514+0000 7f6faa600700 1 -- 172.21.15.86:0/792603182 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa409b000 con 0x55b2e20fb130 2024-03-23T17:50:29.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.514+0000 7f6faa600700 1 -- 172.21.15.86:0/792603182 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6fa409ee30 con 0x55b2e20fb130 2024-03-23T17:50:29.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.514+0000 7f6faa600700 1 -- 172.21.15.86:0/792603182 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa409b460 con 0x55b2e20fb130 2024-03-23T17:50:29.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.515+0000 7f6fd3a12b80 1 -- 172.21.15.86:0/792603182 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b2e20fb130 msgr2=0x55b2e20fb9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:29.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.515+0000 7f6fd3a12b80 1 --2- 172.21.15.86:0/792603182 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b2e20fb130 0x55b2e20fb9f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6fa400b680 tx=0x7f6fa409a960 comp rx=0 tx=0).stop 2024-03-23T17:50:29.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.515+0000 7f6fd3a12b80 1 -- 172.21.15.86:0/792603182 shutdown_connections 2024-03-23T17:50:29.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.515+0000 7f6fd3a12b80 1 --2- 172.21.15.86:0/792603182 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b2e20fbfd0 0x55b2e20fc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.515+0000 7f6fd3a12b80 1 --2- 172.21.15.86:0/792603182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2e20fa7d0 0x55b2e20fabf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.515+0000 7f6fd3a12b80 1 --2- 172.21.15.86:0/792603182 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b2e20fb130 0x55b2e20fb9f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.515+0000 7f6fd3a12b80 1 -- 172.21.15.86:0/792603182 >> 172.21.15.86:0/792603182 conn(0x55b2e20596a0 msgr2=0x55b2e20f59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:29.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.515+0000 7f6fd3a12b80 1 -- 172.21.15.86:0/792603182 shutdown_connections 2024-03-23T17:50:29.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.515+0000 7f6fd3a12b80 1 -- 172.21.15.86:0/792603182 wait complete. 2024-03-23T17:50:29.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.515+0000 7f6fd3a12b80 1 -- start start 2024-03-23T17:50:29.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.516+0000 7f6fd3a12b80 1 Processor -- start 2024-03-23T17:50:29.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.516+0000 7f6fd3a12b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2e20fa7d0 0x55b2e21042f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:29.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.516+0000 7f6fd3a12b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b2e20fb130 0x55b2e2104830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:29.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.516+0000 7f6fd3a12b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b2e20fbfd0 0x55b2e210b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:29.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.516+0000 7f6fd3a12b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2e1fe7d80 con 0x55b2e20fa7d0 2024-03-23T17:50:29.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.516+0000 7f6fd3a12b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2e1fe7a20 con 0x55b2e20fbfd0 2024-03-23T17:50:29.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.516+0000 7f6fd3a12b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2e1ecf6a0 con 0x55b2e20fb130 2024-03-23T17:50:29.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.516+0000 7f6fd3a12b80 10 client.0 _ll_register_callbacks cb 0x55b2e1ff1470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:50:29.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.516+0000 7f6fbd400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2e20fa7d0 0x55b2e21042f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:29.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.517+0000 7f6fbd400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2e20fa7d0 0x55b2e21042f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43310/0 (socket says 172.21.15.86:43310) 2024-03-23T17:50:29.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.517+0000 7f6fbd400700 1 -- 172.21.15.86:0/1900629064 learned_addr learned my addr 172.21.15.86:0/1900629064 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:29.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.517+0000 7f6fbde00700 1 --2- 172.21.15.86:0/1900629064 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b2e20fbfd0 0x55b2e210b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:29.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.517+0000 7f6fbca00700 1 --2- 172.21.15.86:0/1900629064 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b2e20fb130 0x55b2e2104830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:29.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.517+0000 7f6fbd400700 1 -- 172.21.15.86:0/1900629064 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b2e20fb130 msgr2=0x55b2e2104830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:50:29.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.517+0000 7f6fbd400700 1 --2- 172.21.15.86:0/1900629064 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b2e20fb130 0x55b2e2104830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.517+0000 7f6fbd400700 1 -- 172.21.15.86:0/1900629064 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b2e20fbfd0 msgr2=0x55b2e210b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:50:29.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.517+0000 7f6fbd400700 1 --2- 172.21.15.86:0/1900629064 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b2e20fbfd0 0x55b2e210b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.517+0000 7f6fbd400700 1 -- 172.21.15.86:0/1900629064 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fb0003a60 con 0x55b2e20fa7d0 2024-03-23T17:50:29.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.517+0000 7f6fbd400700 1 --2- 172.21.15.86:0/1900629064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2e20fa7d0 0x55b2e21042f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f6fac0099b0 tx=0x7f6fac0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a354616d644642f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:29.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.517+0000 7f6f9ea00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-23T17:50:29.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.517+0000 7f6f9ea00700 1 -- 172.21.15.86:0/1900629064 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fac018040 con 0x55b2e20fa7d0 2024-03-23T17:50:29.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.518+0000 7f6f9ea00700 1 -- 172.21.15.86:0/1900629064 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f6fac00f7c0 con 0x55b2e20fa7d0 2024-03-23T17:50:29.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.517+0000 7f6fd3a12b80 1 -- 172.21.15.86:0/1900629064 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b2e200c380 con 0x55b2e20fa7d0 2024-03-23T17:50:29.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.518+0000 7f6f9ea00700 1 -- 172.21.15.86:0/1900629064 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6fac003ea0 con 0x55b2e20fa7d0 2024-03-23T17:50:29.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.518+0000 7f6fd3a12b80 10 client.15354 fetch_fsmap learned FSMap version 28 2024-03-23T17:50:29.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.518+0000 7f6fd3a12b80 1 -- 172.21.15.86:0/1900629064 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f6fb81a5650 con 0x55b2e20fa7d0 2024-03-23T17:50:29.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.518+0000 7f6f9ea00700 1 -- 172.21.15.86:0/1900629064 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b2e200c380 con 0x55b2e20fa7d0 2024-03-23T17:50:29.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.518+0000 7f6f9ea00700 1 -- 172.21.15.86:0/1900629064 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6fac0299c0 con 0x55b2e20fa7d0 2024-03-23T17:50:29.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.518+0000 7f6fd3a12b80 10 client.15354 fetch_fsmap finished waiting for FSMap version 28 2024-03-23T17:50:29.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.518+0000 7f6fd3a12b80 10 client.15354 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T17:50:29.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.518+0000 7f6fd3a12b80 20 client.15354 populate_metadata read hostname 'smithi086' 2024-03-23T17:50:29.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.518+0000 7f6fd3a12b80 1 --2- 172.21.15.86:0/1900629064 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55b2e21100b0 0x55b2e21124d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:29.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.518+0000 7f6fd3a12b80 4 client.15354 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:50:29.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.518+0000 7f6fd3a12b80 1 -- 172.21.15.86:0/1900629064 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b2e2044700 con 0x55b2e21100b0 2024-03-23T17:50:29.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.519+0000 7f6fbca00700 1 --2- 172.21.15.86:0/1900629064 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55b2e21100b0 0x55b2e21124d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:29.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.519+0000 7f6fbca00700 1 --2- 172.21.15.86:0/1900629064 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55b2e21100b0 0x55b2e21124d0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=341dfa1167a1f651 server_cookie=8d2a35d667c2afd9 in_seq=0 out_seq=0 2024-03-23T17:50:29.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.519+0000 7f6f9ea00700 10 client.15354 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:50:29.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.520+0000 7f6f9ea00700 1 -- 172.21.15.86:0/1900629064 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55b2e2044700 con 0x55b2e21100b0 2024-03-23T17:50:29.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.520+0000 7f6f9ea00700 10 client.15354 handle_command_reply: tid=0 2024-03-23T17:50:29.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.520+0000 7f6fd3a12b80 1 client.15354 shutdown 2024-03-23T17:50:29.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.520+0000 7f6fd3a12b80 1 -- 172.21.15.86:0/1900629064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2e20fa7d0 msgr2=0x55b2e21042f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:29.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.520+0000 7f6fd3a12b80 1 --2- 172.21.15.86:0/1900629064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2e20fa7d0 0x55b2e21042f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f6fac0099b0 tx=0x7f6fac0047c0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.520+0000 7f6fd3a12b80 1 -- 172.21.15.86:0/1900629064 shutdown_connections 2024-03-23T17:50:29.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.520+0000 7f6fd3a12b80 1 --2- 172.21.15.86:0/1900629064 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55b2e21100b0 0x55b2e21124d0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.520+0000 7f6fd3a12b80 1 --2- 172.21.15.86:0/1900629064 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b2e20fbfd0 0x55b2e210b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.520+0000 7f6fd3a12b80 1 --2- 172.21.15.86:0/1900629064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2e20fa7d0 0x55b2e21042f0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.520+0000 7f6fd3a12b80 1 --2- 172.21.15.86:0/1900629064 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b2e20fb130 0x55b2e2104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.520+0000 7f6fd3a12b80 1 -- 172.21.15.86:0/1900629064 >> 172.21.15.86:0/1900629064 conn(0x55b2e20596a0 msgr2=0x55b2e20f9170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:29.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.520+0000 7f6f9ea00700 0 client.15354 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:50:29.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.520+0000 7f6fd3a12b80 1 -- 172.21.15.86:0/1900629064 shutdown_connections 2024-03-23T17:50:29.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.520+0000 7f6fd3a12b80 1 -- 172.21.15.86:0/1900629064 wait complete. 2024-03-23T17:50:29.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.521+0000 7f6fd3a12b80 20 client.15354 trim_cache size 0 max 16384 2024-03-23T17:50:29.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.533+0000 7f1455600700 1 -- 172.21.15.86:0/2876855440 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f145004eb60 con 0x7f1450074df0 2024-03-23T17:50:29.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.537+0000 7f6fd3a12b80 1 Processor -- start 2024-03-23T17:50:29.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.537+0000 7f6fd3a12b80 1 -- start start 2024-03-23T17:50:29.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.537+0000 7f6fd3a12b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b2e20fa7d0 0x55b2e2109820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:29.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.538+0000 7f6fd3a12b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2e20fb130 0x55b2e20f9170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:29.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.538+0000 7f6fd3a12b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b2e20fbfd0 0x55b2e20f96b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:29.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.538+0000 7f6fd3a12b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2e20f5f20 con 0x55b2e20fb130 2024-03-23T17:50:29.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.538+0000 7f6fd3a12b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2e20fa370 con 0x55b2e20fbfd0 2024-03-23T17:50:29.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.538+0000 7f6fd3a12b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2e20fb510 con 0x55b2e20fa7d0 2024-03-23T17:50:29.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.538+0000 7f6fbca00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2e20fb130 0x55b2e20f9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:29.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.538+0000 7f6fbde00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b2e20fbfd0 0x55b2e20f96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:29.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.538+0000 7f6fbca00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2e20fb130 0x55b2e20f9170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43312/0 (socket says 172.21.15.86:43312) 2024-03-23T17:50:29.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.538+0000 7f6fbca00700 1 -- 172.21.15.86:0/4220462251 learned_addr learned my addr 172.21.15.86:0/4220462251 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:29.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.538+0000 7f6fbd400700 1 --2- 172.21.15.86:0/4220462251 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b2e20fa7d0 0x55b2e2109820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:29.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.538+0000 7f6fbde00700 1 -- 172.21.15.86:0/4220462251 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b2e20fa7d0 msgr2=0x55b2e2109820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:50:29.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.538+0000 7f6fbde00700 1 --2- 172.21.15.86:0/4220462251 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b2e20fa7d0 0x55b2e2109820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.538+0000 7f6fbde00700 1 -- 172.21.15.86:0/4220462251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2e20fb130 msgr2=0x55b2e20f9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:50:29.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.538+0000 7f6fbde00700 1 --2- 172.21.15.86:0/4220462251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2e20fb130 0x55b2e20f9170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.538+0000 7f6fbde00700 1 -- 172.21.15.86:0/4220462251 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b2e1e79b60 con 0x55b2e20fbfd0 2024-03-23T17:50:29.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.538+0000 7f6fbd400700 1 --2- 172.21.15.86:0/4220462251 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b2e20fa7d0 0x55b2e2109820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:50:29.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.538+0000 7f6fbde00700 1 --2- 172.21.15.86:0/4220462251 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b2e20fbfd0 0x55b2e20f96b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x55b2e20fb7b0 tx=0x7f6fb0009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7f30e346c8dcc9 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:29.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.539+0000 7f6fab000700 1 -- 172.21.15.86:0/4220462251 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb0004db0 con 0x55b2e20fbfd0 2024-03-23T17:50:29.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.539+0000 7f6fab000700 1 -- 172.21.15.86:0/4220462251 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6fb000f040 con 0x55b2e20fbfd0 2024-03-23T17:50:29.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.539+0000 7f6fab000700 1 -- 172.21.15.86:0/4220462251 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb0007920 con 0x55b2e20fbfd0 2024-03-23T17:50:29.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.539+0000 7f6fd3a12b80 1 -- 172.21.15.86:0/4220462251 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b2e20fbfd0 msgr2=0x55b2e20f96b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:29.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.539+0000 7f6fd3a12b80 1 --2- 172.21.15.86:0/4220462251 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b2e20fbfd0 0x55b2e20f96b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x55b2e20fb7b0 tx=0x7f6fb0009920 comp rx=0 tx=0).stop 2024-03-23T17:50:29.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.539+0000 7f6fd3a12b80 1 -- 172.21.15.86:0/4220462251 shutdown_connections 2024-03-23T17:50:29.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.539+0000 7f6fd3a12b80 1 --2- 172.21.15.86:0/4220462251 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b2e20fbfd0 0x55b2e20f96b0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.539+0000 7f6fd3a12b80 1 --2- 172.21.15.86:0/4220462251 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b2e20fa7d0 0x55b2e2109820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.539+0000 7f6fd3a12b80 1 --2- 172.21.15.86:0/4220462251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2e20fb130 0x55b2e20f9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.539+0000 7f6fd3a12b80 1 -- 172.21.15.86:0/4220462251 >> 172.21.15.86:0/4220462251 conn(0x55b2e20596a0 msgr2=0x55b2e21024c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:29.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.539+0000 7f6fd3a12b80 1 -- 172.21.15.86:0/4220462251 shutdown_connections 2024-03-23T17:50:29.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.540+0000 7f6fd3a12b80 1 -- 172.21.15.86:0/4220462251 wait complete. 2024-03-23T17:50:29.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.540+0000 7f6fd3a12b80 1 -- start start 2024-03-23T17:50:29.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.540+0000 7f6fd3a12b80 1 Processor -- start 2024-03-23T17:50:29.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.540+0000 7f6fd3a12b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2e20fa7d0 0x55b2e2159eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:29.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.540+0000 7f6fd3a12b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b2e20fb130 0x55b2e215a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:29.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.541+0000 7f6fd3a12b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b2e20fbfd0 0x55b2e21613e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:29.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.541+0000 7f6fd3a12b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2e2161920 con 0x55b2e20fa7d0 2024-03-23T17:50:29.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.541+0000 7f6fd3a12b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2e2161a60 con 0x55b2e20fbfd0 2024-03-23T17:50:29.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.541+0000 7f6fd3a12b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2e2161bf0 con 0x55b2e20fb130 2024-03-23T17:50:29.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.541+0000 7f6fd3a12b80 10 client.0 _ll_register_callbacks cb 0x55b2e1ff1470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:50:29.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.541+0000 7f6fbd400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2e20fa7d0 0x55b2e2159eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:29.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.541+0000 7f6fbd400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2e20fa7d0 0x55b2e2159eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43326/0 (socket says 172.21.15.86:43326) 2024-03-23T17:50:29.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.541+0000 7f6fbd400700 1 -- 172.21.15.86:0/4201356145 learned_addr learned my addr 172.21.15.86:0/4201356145 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:29.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.541+0000 7f6fbca00700 1 --2- 172.21.15.86:0/4201356145 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b2e20fb130 0x55b2e215a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:29.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.541+0000 7f6fbde00700 1 --2- 172.21.15.86:0/4201356145 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b2e20fbfd0 0x55b2e21613e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:29.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.541+0000 7f6fbd400700 1 -- 172.21.15.86:0/4201356145 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b2e20fb130 msgr2=0x55b2e215a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:50:29.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.541+0000 7f6fbd400700 1 --2- 172.21.15.86:0/4201356145 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b2e20fb130 0x55b2e215a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.541+0000 7f6fbd400700 1 -- 172.21.15.86:0/4201356145 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b2e20fbfd0 msgr2=0x55b2e21613e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:50:29.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.541+0000 7f6fbd400700 1 --2- 172.21.15.86:0/4201356145 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b2e20fbfd0 0x55b2e21613e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.541+0000 7f6fbd400700 1 -- 172.21.15.86:0/4201356145 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fb000e680 con 0x55b2e20fa7d0 2024-03-23T17:50:29.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.541+0000 7f6fbd400700 1 --2- 172.21.15.86:0/4201356145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2e20fa7d0 0x55b2e2159eb0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f6fac0099b0 tx=0x7f6fac029720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9398d3f75ea74efc server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:29.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.541+0000 7f6f9ea00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-23T17:50:29.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.541+0000 7f6f9ea00700 1 -- 172.21.15.86:0/4201356145 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fac018050 con 0x55b2e20fa7d0 2024-03-23T17:50:29.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.542+0000 7f6f9ea00700 1 -- 172.21.15.86:0/4201356145 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f6fac013460 con 0x55b2e20fa7d0 2024-03-23T17:50:29.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.542+0000 7f6fd3a12b80 1 -- 172.21.15.86:0/4201356145 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6fa409ee30 con 0x55b2e20fa7d0 2024-03-23T17:50:29.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.542+0000 7f6f9ea00700 1 -- 172.21.15.86:0/4201356145 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6fac0122e0 con 0x55b2e20fa7d0 2024-03-23T17:50:29.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.542+0000 7f6fd3a12b80 10 client.15360 fetch_fsmap learned FSMap version 28 2024-03-23T17:50:29.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.542+0000 7f6fd3a12b80 1 -- 172.21.15.86:0/4201356145 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55b2e2011ea0 con 0x55b2e20fa7d0 2024-03-23T17:50:29.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.542+0000 7f6f9ea00700 1 -- 172.21.15.86:0/4201356145 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6fa409ee30 con 0x55b2e20fa7d0 2024-03-23T17:50:29.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.542+0000 7f6f9ea00700 1 -- 172.21.15.86:0/4201356145 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6fac03b020 con 0x55b2e20fa7d0 2024-03-23T17:50:29.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.542+0000 7f6fd3a12b80 10 client.15360 fetch_fsmap finished waiting for FSMap version 28 2024-03-23T17:50:29.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.542+0000 7f6fd3a12b80 10 client.15360 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T17:50:29.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.542+0000 7f6fd3a12b80 20 client.15360 populate_metadata read hostname 'smithi086' 2024-03-23T17:50:29.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.542+0000 7f6fd3a12b80 1 --2- 172.21.15.86:0/4201356145 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55b2e2163530 0x55b2e2165950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:29.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.542+0000 7f6fd3a12b80 4 client.15360 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-23T17:50:29.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.542+0000 7f6fd3a12b80 1 -- 172.21.15.86:0/4201356145 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b2e1e9acf0 con 0x55b2e2163530 2024-03-23T17:50:29.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.542+0000 7f6fbca00700 1 --2- 172.21.15.86:0/4201356145 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55b2e2163530 0x55b2e2165950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:29.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.543+0000 7f6fbca00700 1 --2- 172.21.15.86:0/4201356145 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55b2e2163530 0x55b2e2165950 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5eb880804b184e67 server_cookie=ee76571ffcce55fd in_seq=0 out_seq=0 2024-03-23T17:50:29.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.543+0000 7f6f9ea00700 10 client.15360 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:50:29.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.543+0000 7f6f9ea00700 1 -- 172.21.15.86:0/4201356145 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55b2e1e9acf0 con 0x55b2e2163530 2024-03-23T17:50:29.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.543+0000 7f6f9ea00700 10 client.15360 handle_command_reply: tid=0 2024-03-23T17:50:29.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.543+0000 7f6fd3a12b80 1 client.15360 shutdown 2024-03-23T17:50:29.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.543+0000 7f6fd3a12b80 1 -- 172.21.15.86:0/4201356145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2e20fa7d0 msgr2=0x55b2e2159eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:29.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.543+0000 7f6fd3a12b80 1 --2- 172.21.15.86:0/4201356145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2e20fa7d0 0x55b2e2159eb0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f6fac0099b0 tx=0x7f6fac029720 comp rx=0 tx=0).stop 2024-03-23T17:50:29.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.543+0000 7f6fd3a12b80 1 -- 172.21.15.86:0/4201356145 shutdown_connections 2024-03-23T17:50:29.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.543+0000 7f6fd3a12b80 1 --2- 172.21.15.86:0/4201356145 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55b2e2163530 0x55b2e2165950 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.543+0000 7f6fd3a12b80 1 --2- 172.21.15.86:0/4201356145 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b2e20fbfd0 0x55b2e21613e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.543+0000 7f6f9ea00700 0 client.15360 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:50:29.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.543+0000 7f6fd3a12b80 1 --2- 172.21.15.86:0/4201356145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2e20fa7d0 0x55b2e2159eb0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.543+0000 7f6fd3a12b80 1 --2- 172.21.15.86:0/4201356145 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b2e20fb130 0x55b2e215a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.543+0000 7f6fd3a12b80 1 -- 172.21.15.86:0/4201356145 >> 172.21.15.86:0/4201356145 conn(0x55b2e20596a0 msgr2=0x55b2e20f9170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:29.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.544+0000 7f6fd3a12b80 1 -- 172.21.15.86:0/4201356145 shutdown_connections 2024-03-23T17:50:29.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.544+0000 7f6fd3a12b80 1 -- 172.21.15.86:0/4201356145 wait complete. 2024-03-23T17:50:29.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.544+0000 7f6fd3a12b80 20 client.15360 trim_cache size 0 max 16384 2024-03-23T17:50:29.550 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-23T17:50:29.550 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-23T17:50:29.550 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-03-23T17:50:29.550 INFO:teuthology.orchestra.run.smithi086.stdout: "a3652b6e-9e12-4522-886d-7c43e45a2e9c": { 2024-03-23T17:50:29.550 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-03-23T17:50:29.550 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "a3652b6e-9e12-4522-886d-7c43e45a2e9c", 2024-03-23T17:50:29.550 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-03-23T17:50:29.550 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T17:50:29.550 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T17:50:29.550 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-23T17:50:29.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.544+0000 7f6fbfe00700 1 -- 172.21.15.86:0/3148530065 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f6fa00a1120 msgr2=0x7f6fa00a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:29.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.544+0000 7f6fbfe00700 1 --2- 172.21.15.86:0/3148530065 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f6fa00a1120 0x7f6fa00a35e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6fac009fd0 tx=0x7f6fac013ae0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.544+0000 7f6fbfe00700 1 -- 172.21.15.86:0/3148530065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fb8074df0 msgr2=0x7f6fb819d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:29.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.544+0000 7f6fbfe00700 1 --2- 172.21.15.86:0/3148530065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fb8074df0 0x7f6fb819d880 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f6fa4000c00 tx=0x7f6fa4009580 comp rx=0 tx=0).stop 2024-03-23T17:50:29.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.544+0000 7f6fbfe00700 1 -- 172.21.15.86:0/3148530065 shutdown_connections 2024-03-23T17:50:29.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.544+0000 7f6fbfe00700 1 --2- 172.21.15.86:0/3148530065 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fb810d350 0x7f6fb81a48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.544+0000 7f6fbfe00700 1 --2- 172.21.15.86:0/3148530065 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6fb8073810 0x7f6fb819d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.544+0000 7f6fbfe00700 1 --2- 172.21.15.86:0/3148530065 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f6fa00a1120 0x7f6fa00a35e0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.544+0000 7f6fbfe00700 1 --2- 172.21.15.86:0/3148530065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fb8074df0 0x7f6fb819d880 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.544+0000 7f6fbfe00700 1 -- 172.21.15.86:0/3148530065 >> 172.21.15.86:0/3148530065 conn(0x7f6fb80fc2f0 msgr2=0x7f6fb81006f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:29.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.545+0000 7f6fbfe00700 1 -- 172.21.15.86:0/3148530065 shutdown_connections 2024-03-23T17:50:29.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.545+0000 7f6fbfe00700 1 -- 172.21.15.86:0/3148530065 wait complete. 2024-03-23T17:50:29.562 INFO:tasks.cephfs.filesystem:scrub status for tag:a3652b6e-9e12-4522-886d-7c43e45a2e9c - {'path': '/', 'tag': 'a3652b6e-9e12-4522-886d-7c43e45a2e9c', 'options': 'recursive,force'} 2024-03-23T17:50:29.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.777+0000 7f1447400700 1 -- 172.21.15.86:0/2876855440 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mgrmap(e 19) v1 ==== 135882+0+0 (secure 0 0 0) 0x7f143c097d60 con 0x7f1450074df0 2024-03-23T17:50:29.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.778+0000 7f1447400700 1 -- 172.21.15.86:0/2876855440 <== mon.0 v2:172.21.15.86:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) v1 ==== 92+0+0 (secure 0 0 0) 0x7f143c0cfea0 con 0x7f1450074df0 2024-03-23T17:50:29.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.780+0000 7f1455600700 1 -- 172.21.15.86:0/2876855440 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f14340a1110 msgr2=0x7f14340a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:29.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.780+0000 7f1455600700 1 --2- 172.21.15.86:0/2876855440 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f14340a1110 0x7f14340a35d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1440009c60 tx=0x7f1440009260 comp rx=0 tx=0).stop 2024-03-23T17:50:29.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.780+0000 7f1455600700 1 -- 172.21.15.86:0/2876855440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1450074df0 msgr2=0x7f1450199360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:29.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.780+0000 7f1455600700 1 --2- 172.21.15.86:0/2876855440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1450074df0 0x7f1450199360 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f143c00b680 tx=0x7f143c004730 comp rx=0 tx=0).stop 2024-03-23T17:50:29.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.780+0000 7f1455600700 1 -- 172.21.15.86:0/2876855440 shutdown_connections 2024-03-23T17:50:29.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.780+0000 7f1455600700 1 --2- 172.21.15.86:0/2876855440 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1450108ae0 0x7f14501a0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.780+0000 7f1455600700 1 --2- 172.21.15.86:0/2876855440 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1450073810 0x7f1450198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.780+0000 7f1455600700 1 --2- 172.21.15.86:0/2876855440 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f14340a1110 0x7f14340a35d0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.780+0000 7f1455600700 1 --2- 172.21.15.86:0/2876855440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1450074df0 0x7f1450199360 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:29.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.780+0000 7f1455600700 1 -- 172.21.15.86:0/2876855440 >> 172.21.15.86:0/2876855440 conn(0x7f14500fc190 msgr2=0x7f14501049e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:29.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.781+0000 7f1455600700 1 -- 172.21.15.86:0/2876855440 shutdown_connections 2024-03-23T17:50:29.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:29.781+0000 7f1455600700 1 -- 172.21.15.86:0/2876855440 wait complete. 2024-03-23T17:50:29.803 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-03-23T17:50:29.859 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:29 smithi086 ceph-mon[38221]: pgmap v282: 97 pgs: 97 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 853 B/s wr, 10 op/s 2024-03-23T17:50:29.859 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:29 smithi086 ceph-mon[38221]: mgrmap e18: x(active, since 7m), standbys: z, y 2024-03-23T17:50:29.859 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:29 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/4100576536' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T17:50:29.859 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:29 smithi086 ceph-mon[38221]: scrub queued for path: / 2024-03-23T17:50:29.859 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:29 smithi086 ceph-mon[38221]: scrub summary: idle+waiting paths [/] 2024-03-23T17:50:29.859 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:29 smithi086 ceph-mon[38221]: scrub summary: active paths [/] 2024-03-23T17:50:29.859 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:29 smithi086 ceph-mon[38221]: scrub summary: idle+waiting paths [/] 2024-03-23T17:50:29.859 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:29 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/2876855440' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-23T17:50:30.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.007+0000 7fd39a800700 1 -- 172.21.15.86:0/4043512044 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd394101f00 msgr2=0x7fd394102360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:30.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.007+0000 7fd39a800700 1 --2- 172.21.15.86:0/4043512044 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd394101f00 0x7fd394102360 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd380009990 tx=0x7fd380009ca0 comp rx=0 tx=0).stop 2024-03-23T17:50:30.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.007+0000 7fd39a800700 1 -- 172.21.15.86:0/4043512044 shutdown_connections 2024-03-23T17:50:30.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.007+0000 7fd39a800700 1 --2- 172.21.15.86:0/4043512044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3941028a0 0x7fd394109150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:30.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.007+0000 7fd39a800700 1 --2- 172.21.15.86:0/4043512044 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd394100d10 0x7fd394101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:30.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.007+0000 7fd39a800700 1 --2- 172.21.15.86:0/4043512044 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd394101f00 0x7fd394102360 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:30.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.007+0000 7fd39a800700 1 -- 172.21.15.86:0/4043512044 >> 172.21.15.86:0/4043512044 conn(0x7fd3940fc2a0 msgr2=0x7fd3940fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:30.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.007+0000 7fd39a800700 1 -- 172.21.15.86:0/4043512044 shutdown_connections 2024-03-23T17:50:30.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.007+0000 7fd39a800700 1 -- 172.21.15.86:0/4043512044 wait complete. 2024-03-23T17:50:30.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.008+0000 7fd39a800700 1 Processor -- start 2024-03-23T17:50:30.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.008+0000 7fd39a800700 1 -- start start 2024-03-23T17:50:30.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.008+0000 7fd39a800700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd394100d10 0x7fd394198f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:30.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.008+0000 7fd39a800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd394101f00 0x7fd394199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:30.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.008+0000 7fd39a800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3941028a0 0x7fd3941a0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:30.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.008+0000 7fd39a800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3941a0a60 con 0x7fd3941028a0 2024-03-23T17:50:30.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.008+0000 7fd39a800700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3941a0ba0 con 0x7fd394101f00 2024-03-23T17:50:30.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.008+0000 7fd39a800700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3941a0d30 con 0x7fd394100d10 2024-03-23T17:50:30.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.009+0000 7fd393e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3941028a0 0x7fd3941a0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:30.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.009+0000 7fd392a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd394101f00 0x7fd394199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:30.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.009+0000 7fd393400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd394100d10 0x7fd394198f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:30.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.009+0000 7fd392a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd394101f00 0x7fd394199470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:59194/0 (socket says 172.21.15.86:59194) 2024-03-23T17:50:30.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.009+0000 7fd392a00700 1 -- 172.21.15.86:0/1586675833 learned_addr learned my addr 172.21.15.86:0/1586675833 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:30.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.009+0000 7fd392a00700 1 -- 172.21.15.86:0/1586675833 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd394100d10 msgr2=0x7fd394198f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:30.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.009+0000 7fd392a00700 1 --2- 172.21.15.86:0/1586675833 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd394100d10 0x7fd394198f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:30.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.009+0000 7fd392a00700 1 -- 172.21.15.86:0/1586675833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3941028a0 msgr2=0x7fd3941a0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:30.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.009+0000 7fd392a00700 1 --2- 172.21.15.86:0/1586675833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3941028a0 0x7fd3941a0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:30.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.009+0000 7fd392a00700 1 -- 172.21.15.86:0/1586675833 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd38c009c40 con 0x7fd394101f00 2024-03-23T17:50:30.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.009+0000 7fd392a00700 1 --2- 172.21.15.86:0/1586675833 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd394101f00 0x7fd394199470 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd380005910 tx=0x7fd380009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:30.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.009+0000 7fd38be00700 1 -- 172.21.15.86:0/1586675833 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd38001d070 con 0x7fd394101f00 2024-03-23T17:50:30.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.010+0000 7fd38be00700 1 -- 172.21.15.86:0/1586675833 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd380021410 con 0x7fd394101f00 2024-03-23T17:50:30.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.010+0000 7fd39a800700 1 -- 172.21.15.86:0/1586675833 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3800095f0 con 0x7fd394101f00 2024-03-23T17:50:30.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.010+0000 7fd38be00700 1 -- 172.21.15.86:0/1586675833 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd380017650 con 0x7fd394101f00 2024-03-23T17:50:30.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.010+0000 7fd39a800700 1 -- 172.21.15.86:0/1586675833 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3941a1240 con 0x7fd394101f00 2024-03-23T17:50:30.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.011+0000 7fd39a800700 1 -- 172.21.15.86:0/1586675833 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3940673a0 con 0x7fd394101f00 2024-03-23T17:50:30.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.013+0000 7fd38be00700 1 -- 172.21.15.86:0/1586675833 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) v1 ==== 135882+0+0 (secure 0 0 0) 0x7fd380021580 con 0x7fd394101f00 2024-03-23T17:50:30.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.014+0000 7fd38be00700 1 --2- 172.21.15.86:0/1586675833 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fd3780a11f0 0x7fd3780a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:30.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.014+0000 7fd393400700 1 -- 172.21.15.86:0/1586675833 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fd3780a11f0 msgr2=0x7fd3780a36b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/2734784244 2024-03-23T17:50:30.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.014+0000 7fd38be00700 1 -- 172.21.15.86:0/1586675833 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fd3800cdcb0 con 0x7fd394101f00 2024-03-23T17:50:30.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.014+0000 7fd393400700 1 --2- 172.21.15.86:0/1586675833 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fd3780a11f0 0x7fd3780a36b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-23T17:50:30.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.016+0000 7fd38be00700 1 -- 172.21.15.86:0/1586675833 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fd3800974a0 con 0x7fd394101f00 2024-03-23T17:50:30.057 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Mar 23 17:50:29 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-y[80565]: ignoring --setuser ceph since I am not root 2024-03-23T17:50:30.057 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Mar 23 17:50:29 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-y[80565]: ignoring --setgroup ceph since I am not root 2024-03-23T17:50:30.057 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Mar 23 17:50:29 smithi092 ceph-mgr[80588]: -- 172.21.15.92:0/576758300 <== mon.2 v2:172.21.15.92:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5603d0b321a0 con 0x5603d0b08c00 2024-03-23T17:50:30.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:29 smithi092 ceph-mon[79089]: pgmap v282: 97 pgs: 97 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 853 B/s wr, 10 op/s 2024-03-23T17:50:30.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:29 smithi092 ceph-mon[79089]: mgrmap e18: x(active, since 7m), standbys: z, y 2024-03-23T17:50:30.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:29 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/4100576536' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T17:50:30.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:29 smithi092 ceph-mon[79089]: scrub queued for path: / 2024-03-23T17:50:30.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:29 smithi092 ceph-mon[79089]: scrub summary: idle+waiting paths [/] 2024-03-23T17:50:30.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:29 smithi092 ceph-mon[79089]: scrub summary: active paths [/] 2024-03-23T17:50:30.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:29 smithi092 ceph-mon[79089]: scrub summary: idle+waiting paths [/] 2024-03-23T17:50:30.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:29 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/2876855440' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-23T17:50:30.086 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Mar 23 17:50:29 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-z[49806]: ignoring --setuser ceph since I am not root 2024-03-23T17:50:30.086 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Mar 23 17:50:29 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-z[49806]: ignoring --setgroup ceph since I am not root 2024-03-23T17:50:30.086 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Mar 23 17:50:29 smithi152 ceph-mgr[49829]: -- 172.21.15.152:0/2405283926 <== mon.1 v2:172.21.15.152:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55fcee83a1a0 con 0x55fcee810800 2024-03-23T17:50:30.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:29 smithi152 ceph-mon[47949]: pgmap v282: 97 pgs: 97 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 853 B/s wr, 10 op/s 2024-03-23T17:50:30.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:29 smithi152 ceph-mon[47949]: mgrmap e18: x(active, since 7m), standbys: z, y 2024-03-23T17:50:30.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:29 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/4100576536' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T17:50:30.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:29 smithi152 ceph-mon[47949]: scrub queued for path: / 2024-03-23T17:50:30.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:29 smithi152 ceph-mon[47949]: scrub summary: idle+waiting paths [/] 2024-03-23T17:50:30.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:29 smithi152 ceph-mon[47949]: scrub summary: active paths [/] 2024-03-23T17:50:30.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:29 smithi152 ceph-mon[47949]: scrub summary: idle+waiting paths [/] 2024-03-23T17:50:30.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:29 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/2876855440' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-23T17:50:30.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.215+0000 7fd393400700 1 -- 172.21.15.86:0/1586675833 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fd3780a11f0 msgr2=0x7fd3780a36b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/2734784244 2024-03-23T17:50:30.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.215+0000 7fd393400700 1 --2- 172.21.15.86:0/1586675833 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fd3780a11f0 0x7fd3780a36b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-23T17:50:30.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.261+0000 7fd39a800700 1 -- 172.21.15.86:0/1586675833 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7fd3941a1600 con 0x7fd394101f00 2024-03-23T17:50:30.270 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:50:29 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: ignoring --setuser ceph since I am not root 2024-03-23T17:50:30.270 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:50:29 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: ignoring --setgroup ceph since I am not root 2024-03-23T17:50:30.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.267+0000 7fd38be00700 1 -- 172.21.15.86:0/1586675833 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v29)=0 v29) v1 ==== 138+0+0 (secure 0 0 0) 0x7fd38009aac0 con 0x7fd394101f00 2024-03-23T17:50:30.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.270+0000 7fd39a800700 1 -- 172.21.15.86:0/1586675833 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fd3780a11f0 msgr2=0x7fd3780a36b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-23T17:50:30.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.270+0000 7fd39a800700 1 --2- 172.21.15.86:0/1586675833 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fd3780a11f0 0x7fd3780a36b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:30.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.270+0000 7fd39a800700 1 -- 172.21.15.86:0/1586675833 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd394101f00 msgr2=0x7fd394199470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:30.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.270+0000 7fd39a800700 1 --2- 172.21.15.86:0/1586675833 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd394101f00 0x7fd394199470 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd380005910 tx=0x7fd380009580 comp rx=0 tx=0).stop 2024-03-23T17:50:30.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.270+0000 7fd39a800700 1 -- 172.21.15.86:0/1586675833 shutdown_connections 2024-03-23T17:50:30.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.270+0000 7fd39a800700 1 --2- 172.21.15.86:0/1586675833 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7fd3780a11f0 0x7fd3780a36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:30.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.270+0000 7fd39a800700 1 --2- 172.21.15.86:0/1586675833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3941028a0 0x7fd3941a0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:30.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.270+0000 7fd39a800700 1 --2- 172.21.15.86:0/1586675833 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd394100d10 0x7fd394198f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:30.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.270+0000 7fd39a800700 1 --2- 172.21.15.86:0/1586675833 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd394101f00 0x7fd394199470 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:30.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.270+0000 7fd39a800700 1 -- 172.21.15.86:0/1586675833 >> 172.21.15.86:0/1586675833 conn(0x7fd3940fc2a0 msgr2=0x7fd394072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:30.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.270+0000 7fd39a800700 1 -- 172.21.15.86:0/1586675833 shutdown_connections 2024-03-23T17:50:30.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.270+0000 7fd39a800700 1 -- 172.21.15.86:0/1586675833 wait complete. 2024-03-23T17:50:30.290 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-03-23T17:50:30.478 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Mar 23 17:50:30 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-y[80565]: 2024-03-23T17:50:30.213+0000 7f3a49840200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-23T17:50:30.478 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Mar 23 17:50:30 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-y[80565]: 2024-03-23T17:50:30.306+0000 7f3a49840200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-23T17:50:30.478 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Mar 23 17:50:30 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-y[80565]: 2024-03-23T17:50:30.393+0000 7f3a49840200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-23T17:50:30.478 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Mar 23 17:50:30 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-y[80565]: 2024-03-23T17:50:30.477+0000 7f3a49840200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-23T17:50:30.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.484+0000 7f07e6a00700 1 -- 172.21.15.86:0/219935633 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f07e0101f10 msgr2=0x7f07e0102370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:30.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.484+0000 7f07e6a00700 1 --2- 172.21.15.86:0/219935633 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f07e0101f10 0x7f07e0102370 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f07cc009990 tx=0x7f07cc009ca0 comp rx=0 tx=0).stop 2024-03-23T17:50:30.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.484+0000 7f07e6a00700 1 -- 172.21.15.86:0/219935633 shutdown_connections 2024-03-23T17:50:30.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.484+0000 7f07e6a00700 1 --2- 172.21.15.86:0/219935633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07e01028b0 0x7f07e0109160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:30.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.484+0000 7f07e6a00700 1 --2- 172.21.15.86:0/219935633 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07e0100d20 0x7f07e0101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:30.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.484+0000 7f07e6a00700 1 --2- 172.21.15.86:0/219935633 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f07e0101f10 0x7f07e0102370 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:30.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.484+0000 7f07e6a00700 1 -- 172.21.15.86:0/219935633 >> 172.21.15.86:0/219935633 conn(0x7f07e00fc2b0 msgr2=0x7f07e00fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:30.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.484+0000 7f07e6a00700 1 -- 172.21.15.86:0/219935633 shutdown_connections 2024-03-23T17:50:30.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.484+0000 7f07e6a00700 1 -- 172.21.15.86:0/219935633 wait complete. 2024-03-23T17:50:30.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.485+0000 7f07e6a00700 1 Processor -- start 2024-03-23T17:50:30.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.485+0000 7f07e6a00700 1 -- start start 2024-03-23T17:50:30.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.486+0000 7f07e6a00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f07e0100d20 0x7f07e0074ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:30.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.486+0000 7f07e6a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07e0101f10 0x7f07e00731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:30.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.486+0000 7f07e6a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07e01028b0 0x7f07e00736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:30.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.486+0000 7f07e6a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e0073de0 con 0x7f07e01028b0 2024-03-23T17:50:30.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.486+0000 7f07e6a00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e0073f50 con 0x7f07e0101f10 2024-03-23T17:50:30.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.486+0000 7f07e6a00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e01a0d00 con 0x7f07e0100d20 2024-03-23T17:50:30.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.486+0000 7f07dfe00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f07e0100d20 0x7f07e0074ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:30.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.486+0000 7f07dfe00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f07e0100d20 0x7f07e0074ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:33532/0 (socket says 172.21.15.86:33532) 2024-03-23T17:50:30.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.486+0000 7f07dfe00700 1 -- 172.21.15.86:0/49335281 learned_addr learned my addr 172.21.15.86:0/49335281 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:30.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.486+0000 7f07df400700 1 --2- 172.21.15.86:0/49335281 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07e0101f10 0x7f07e00731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:30.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.486+0000 7f07e4a00700 1 --2- 172.21.15.86:0/49335281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07e01028b0 0x7f07e00736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:30.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.486+0000 7f07dfe00700 1 -- 172.21.15.86:0/49335281 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07e0101f10 msgr2=0x7f07e00731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:30.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.486+0000 7f07dfe00700 1 --2- 172.21.15.86:0/49335281 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07e0101f10 0x7f07e00731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:30.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.486+0000 7f07dfe00700 1 -- 172.21.15.86:0/49335281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07e01028b0 msgr2=0x7f07e00736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:30.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.486+0000 7f07dfe00700 1 --2- 172.21.15.86:0/49335281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07e01028b0 0x7f07e00736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:30.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.486+0000 7f07dfe00700 1 -- 172.21.15.86:0/49335281 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07d0009520 con 0x7f07e0100d20 2024-03-23T17:50:30.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.486+0000 7f07df400700 1 --2- 172.21.15.86:0/49335281 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07e0101f10 0x7f07e00731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:50:30.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.487+0000 7f07dfe00700 1 --2- 172.21.15.86:0/49335281 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f07e0100d20 0x7f07e0074ac0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f07d000ea50 tx=0x7f07d000ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:30.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.487+0000 7f07dcc00700 1 -- 172.21.15.86:0/49335281 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d000cbd0 con 0x7f07e0100d20 2024-03-23T17:50:30.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.487+0000 7f07dcc00700 1 -- 172.21.15.86:0/49335281 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07d000cd30 con 0x7f07e0100d20 2024-03-23T17:50:30.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.487+0000 7f07e6a00700 1 -- 172.21.15.86:0/49335281 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07cc0095f0 con 0x7f07e0100d20 2024-03-23T17:50:30.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.487+0000 7f07dcc00700 1 -- 172.21.15.86:0/49335281 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d0018680 con 0x7f07e0100d20 2024-03-23T17:50:30.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.487+0000 7f07e6a00700 1 -- 172.21.15.86:0/49335281 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07e01a1250 con 0x7f07e0100d20 2024-03-23T17:50:30.490 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Mar 23 17:50:30 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-z[49806]: 2024-03-23T17:50:30.220+0000 7f19463b4200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-23T17:50:30.490 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Mar 23 17:50:30 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-z[49806]: 2024-03-23T17:50:30.315+0000 7f19463b4200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-23T17:50:30.490 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Mar 23 17:50:30 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-z[49806]: 2024-03-23T17:50:30.402+0000 7f19463b4200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-23T17:50:30.490 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Mar 23 17:50:30 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-z[49806]: 2024-03-23T17:50:30.487+0000 7f19463b4200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-23T17:50:30.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.488+0000 7f07e6a00700 1 -- 172.21.15.86:0/49335281 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07e0108240 con 0x7f07e0100d20 2024-03-23T17:50:30.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.490+0000 7f07dcc00700 1 -- 172.21.15.86:0/49335281 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 19) v1 ==== 135882+0+0 (secure 0 0 0) 0x7f07d0010430 con 0x7f07e0100d20 2024-03-23T17:50:30.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.491+0000 7f07dcc00700 1 --2- 172.21.15.86:0/49335281 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f07c40a11f0 0x7f07c40a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:30.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.491+0000 7f07df400700 1 -- 172.21.15.86:0/49335281 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f07c40a11f0 msgr2=0x7f07c40a36b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/2734784244 2024-03-23T17:50:30.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.491+0000 7f07dcc00700 1 -- 172.21.15.86:0/49335281 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f07d0014070 con 0x7f07e0100d20 2024-03-23T17:50:30.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.491+0000 7f07df400700 1 --2- 172.21.15.86:0/49335281 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f07c40a11f0 0x7f07c40a36b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-23T17:50:30.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.499+0000 7f07dcc00700 1 -- 172.21.15.86:0/49335281 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f07d0096660 con 0x7f07e0100d20 2024-03-23T17:50:30.633 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:50:30 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:50:30.391+0000 7feaebda9200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-23T17:50:30.633 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:50:30 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:50:30.519+0000 7feaebda9200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-23T17:50:30.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.692+0000 7f07df400700 1 -- 172.21.15.86:0/49335281 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f07c40a11f0 msgr2=0x7f07c40a36b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/2734784244 2024-03-23T17:50:30.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.692+0000 7f07df400700 1 --2- 172.21.15.86:0/49335281 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f07c40a11f0 0x7f07c40a36b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-23T17:50:30.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.748+0000 7f07e6a00700 1 -- 172.21.15.86:0/49335281 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f07e0067c30 con 0x7f07e0100d20 2024-03-23T17:50:30.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.758+0000 7f07dcc00700 1 -- 172.21.15.86:0/49335281 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v30)=0 v30) v1 ==== 133+0+0 (secure 0 0 0) 0x7f07d0099c80 con 0x7f07e0100d20 2024-03-23T17:50:30.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.760+0000 7f07e6a00700 1 -- 172.21.15.86:0/49335281 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f07c40a11f0 msgr2=0x7f07c40a36b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-23T17:50:30.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.760+0000 7f07e6a00700 1 --2- 172.21.15.86:0/49335281 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f07c40a11f0 0x7f07c40a36b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:30.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.760+0000 7f07e6a00700 1 -- 172.21.15.86:0/49335281 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f07e0100d20 msgr2=0x7f07e0074ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:30.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.760+0000 7f07e6a00700 1 --2- 172.21.15.86:0/49335281 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f07e0100d20 0x7f07e0074ac0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f07d000ea50 tx=0x7f07d000ed60 comp rx=0 tx=0).stop 2024-03-23T17:50:30.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.760+0000 7f07e6a00700 1 -- 172.21.15.86:0/49335281 shutdown_connections 2024-03-23T17:50:30.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.760+0000 7f07e6a00700 1 --2- 172.21.15.86:0/49335281 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f07c40a11f0 0x7f07c40a36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:30.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.760+0000 7f07e6a00700 1 --2- 172.21.15.86:0/49335281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07e01028b0 0x7f07e00736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:30.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.760+0000 7f07e6a00700 1 --2- 172.21.15.86:0/49335281 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f07e0100d20 0x7f07e0074ac0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:30.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.760+0000 7f07e6a00700 1 --2- 172.21.15.86:0/49335281 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07e0101f10 0x7f07e00731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:30.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.760+0000 7f07e6a00700 1 -- 172.21.15.86:0/49335281 >> 172.21.15.86:0/49335281 conn(0x7f07e00fc2b0 msgr2=0x7f07e0105380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:30.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.760+0000 7f07e6a00700 1 -- 172.21.15.86:0/49335281 shutdown_connections 2024-03-23T17:50:30.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.761+0000 7f07e6a00700 1 -- 172.21.15.86:0/49335281 wait complete. 2024-03-23T17:50:30.781 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-03-23T17:50:30.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.975+0000 7f2aaa000700 1 -- 172.21.15.86:0/2642269833 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2aa4074df0 msgr2=0x7f2aa4073240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:30.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.975+0000 7f2aaa000700 1 --2- 172.21.15.86:0/2642269833 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2aa4074df0 0x7f2aa4073240 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2a94009990 tx=0x7f2a94009ca0 comp rx=0 tx=0).stop 2024-03-23T17:50:30.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.975+0000 7f2aaa000700 1 -- 172.21.15.86:0/2642269833 shutdown_connections 2024-03-23T17:50:30.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.975+0000 7f2aaa000700 1 --2- 172.21.15.86:0/2642269833 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2aa410d2f0 0x7f2aa410f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:30.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.975+0000 7f2aaa000700 1 --2- 172.21.15.86:0/2642269833 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2aa4074df0 0x7f2aa4073240 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:30.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.975+0000 7f2aaa000700 1 --2- 172.21.15.86:0/2642269833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2aa4073810 0x7f2aa4073c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:30.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.975+0000 7f2aaa000700 1 -- 172.21.15.86:0/2642269833 >> 172.21.15.86:0/2642269833 conn(0x7f2aa40fc2b0 msgr2=0x7f2aa40fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:30.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.975+0000 7f2aaa000700 1 -- 172.21.15.86:0/2642269833 shutdown_connections 2024-03-23T17:50:30.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.975+0000 7f2aaa000700 1 -- 172.21.15.86:0/2642269833 wait complete. 2024-03-23T17:50:30.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.976+0000 7f2aaa000700 1 Processor -- start 2024-03-23T17:50:30.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.977+0000 7f2aaa000700 1 -- start start 2024-03-23T17:50:30.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.977+0000 7f2aaa000700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2aa4073810 0x7f2aa419d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:30.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.977+0000 7f2aaa000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2aa4074df0 0x7f2aa419d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:30.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.977+0000 7f2aaa000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2aa410d2f0 0x7f2aa41a47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:30.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.977+0000 7f2aaa000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa41a4dc0 con 0x7f2aa410d2f0 2024-03-23T17:50:30.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.977+0000 7f2aaa000700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa41a4f00 con 0x7f2aa4074df0 2024-03-23T17:50:30.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.977+0000 7f2aaa000700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa41a5090 con 0x7f2aa4073810 2024-03-23T17:50:30.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.977+0000 7f2aa3400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2aa4073810 0x7f2aa419d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:30.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.977+0000 7f2aa2a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2aa4074df0 0x7f2aa419d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:30.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.977+0000 7f2aa3400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2aa4073810 0x7f2aa419d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:33538/0 (socket says 172.21.15.86:33538) 2024-03-23T17:50:30.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.977+0000 7f2aa3400700 1 -- 172.21.15.86:0/3103842014 learned_addr learned my addr 172.21.15.86:0/3103842014 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:30.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.977+0000 7f2aa3e00700 1 --2- 172.21.15.86:0/3103842014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2aa410d2f0 0x7f2aa41a47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:30.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.978+0000 7f2aa3400700 1 -- 172.21.15.86:0/3103842014 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2aa4074df0 msgr2=0x7f2aa419d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:30.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.978+0000 7f2aa3400700 1 --2- 172.21.15.86:0/3103842014 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2aa4074df0 0x7f2aa419d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:30.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.978+0000 7f2aa3400700 1 -- 172.21.15.86:0/3103842014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2aa410d2f0 msgr2=0x7f2aa41a47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:30.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.978+0000 7f2aa3400700 1 --2- 172.21.15.86:0/3103842014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2aa410d2f0 0x7f2aa41a47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:30.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.978+0000 7f2aa3400700 1 -- 172.21.15.86:0/3103842014 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a940095f0 con 0x7f2aa4073810 2024-03-23T17:50:30.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.978+0000 7f2aa2a00700 1 --2- 172.21.15.86:0/3103842014 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2aa4074df0 0x7f2aa419d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:50:30.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.978+0000 7f2aa3400700 1 --2- 172.21.15.86:0/3103842014 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2aa4073810 0x7f2aa419d2e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2a94005910 tx=0x7f2a94009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:30.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.978+0000 7f2a9be00700 1 -- 172.21.15.86:0/3103842014 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a9401d070 con 0x7f2aa4073810 2024-03-23T17:50:30.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.978+0000 7f2aaa000700 1 -- 172.21.15.86:0/3103842014 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2aa41a5310 con 0x7f2aa4073810 2024-03-23T17:50:30.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.978+0000 7f2a9be00700 1 -- 172.21.15.86:0/3103842014 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a94021410 con 0x7f2aa4073810 2024-03-23T17:50:30.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.978+0000 7f2a9be00700 1 -- 172.21.15.86:0/3103842014 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a94005720 con 0x7f2aa4073810 2024-03-23T17:50:30.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.979+0000 7f2aaa000700 1 -- 172.21.15.86:0/3103842014 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2aa41a5790 con 0x7f2aa4073810 2024-03-23T17:50:30.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.980+0000 7f2aaa000700 1 -- 172.21.15.86:0/3103842014 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2aa4103550 con 0x7f2aa4073810 2024-03-23T17:50:30.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.983+0000 7f2a9be00700 1 -- 172.21.15.86:0/3103842014 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 19) v1 ==== 135882+0+0 (secure 0 0 0) 0x7f2a94021580 con 0x7f2aa4073810 2024-03-23T17:50:30.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.984+0000 7f2a9be00700 1 --2- 172.21.15.86:0/3103842014 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f2a880a1170 0x7f2a880a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:30.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.984+0000 7f2aa2a00700 1 -- 172.21.15.86:0/3103842014 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f2a880a1170 msgr2=0x7f2a880a3630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/2734784244 2024-03-23T17:50:30.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.984+0000 7f2a9be00700 1 -- 172.21.15.86:0/3103842014 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f2a940ce2e0 con 0x7f2aa4073810 2024-03-23T17:50:30.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.984+0000 7f2aa2a00700 1 --2- 172.21.15.86:0/3103842014 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f2a880a1170 0x7f2a880a3630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-23T17:50:30.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:30.989+0000 7f2a9be00700 1 -- 172.21.15.86:0/3103842014 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f2a94097b30 con 0x7f2aa4073810 2024-03-23T17:50:31.016 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:50:30 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:50:30.632+0000 7feaebda9200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-23T17:50:31.016 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:50:30 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:50:30.747+0000 7feaebda9200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-23T17:50:31.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:30 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/2876855440' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-23T17:50:31.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:30 smithi086 ceph-mon[38221]: mgrmap e19: x(active, since 7m), standbys: z, y 2024-03-23T17:50:31.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:30 smithi086 ceph-mon[38221]: from='client.? ' entity='client.admin' 2024-03-23T17:50:31.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:30 smithi086 ceph-mon[38221]: from='client.? ' entity='client.admin' 2024-03-23T17:50:31.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:30 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/2876855440' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-23T17:50:31.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:30 smithi092 ceph-mon[79089]: mgrmap e19: x(active, since 7m), standbys: z, y 2024-03-23T17:50:31.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:30 smithi092 ceph-mon[79089]: from='client.? ' entity='client.admin' 2024-03-23T17:50:31.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:30 smithi092 ceph-mon[79089]: from='client.? ' entity='client.admin' 2024-03-23T17:50:31.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:30 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/2876855440' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-23T17:50:31.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:30 smithi152 ceph-mon[47949]: mgrmap e19: x(active, since 7m), standbys: z, y 2024-03-23T17:50:31.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:30 smithi152 ceph-mon[47949]: from='client.? ' entity='client.admin' 2024-03-23T17:50:31.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:30 smithi152 ceph-mon[47949]: from='client.? ' entity='client.admin' 2024-03-23T17:50:31.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:31.185+0000 7f2aa2a00700 1 -- 172.21.15.86:0/3103842014 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f2a880a1170 msgr2=0x7f2a880a3630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/2734784244 2024-03-23T17:50:31.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:31.185+0000 7f2aa2a00700 1 --2- 172.21.15.86:0/3103842014 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f2a880a1170 0x7f2a880a3630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-23T17:50:31.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:31.229+0000 7f2aaa000700 1 -- 172.21.15.86:0/3103842014 --> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f2aa40611f0 con 0x7f2a880a1170 2024-03-23T17:50:31.356 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Mar 23 17:50:31 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-y[80565]: 2024-03-23T17:50:31.105+0000 7f3a49840200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-23T17:50:31.379 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Mar 23 17:50:31 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-z[49806]: 2024-03-23T17:50:31.126+0000 7f19463b4200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-23T17:50:31.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:31.586+0000 7f2aa2a00700 1 -- 172.21.15.86:0/3103842014 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f2a880a1170 msgr2=0x7f2a880a3630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/2734784244 2024-03-23T17:50:31.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:31.586+0000 7f2aa2a00700 1 --2- 172.21.15.86:0/3103842014 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f2a880a1170 0x7f2a880a3630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-23T17:50:31.708 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Mar 23 17:50:31 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-y[80565]: 2024-03-23T17:50:31.356+0000 7f3a49840200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-23T17:50:31.708 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Mar 23 17:50:31 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-y[80565]: 2024-03-23T17:50:31.441+0000 7f3a49840200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-23T17:50:31.708 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Mar 23 17:50:31 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-y[80565]: 2024-03-23T17:50:31.550+0000 7f3a49840200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-23T17:50:31.708 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Mar 23 17:50:31 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-y[80565]: 2024-03-23T17:50:31.708+0000 7f3a49840200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-23T17:50:31.735 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Mar 23 17:50:31 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-z[49806]: 2024-03-23T17:50:31.379+0000 7f19463b4200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-23T17:50:31.735 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Mar 23 17:50:31 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-z[49806]: 2024-03-23T17:50:31.466+0000 7f19463b4200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-23T17:50:31.735 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Mar 23 17:50:31 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-z[49806]: 2024-03-23T17:50:31.574+0000 7f19463b4200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-23T17:50:31.735 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Mar 23 17:50:31 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-z[49806]: 2024-03-23T17:50:31.734+0000 7f19463b4200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-23T17:50:31.908 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:50:31 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:50:31.594+0000 7feaebda9200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-23T17:50:32.057 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Mar 23 17:50:31 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-y[80565]: 2024-03-23T17:50:31.837+0000 7f3a49840200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-23T17:50:32.057 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Mar 23 17:50:31 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-y[80565]: 2024-03-23T17:50:31.922+0000 7f3a49840200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-23T17:50:32.086 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Mar 23 17:50:31 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-z[49806]: 2024-03-23T17:50:31.863+0000 7f19463b4200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-23T17:50:32.086 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Mar 23 17:50:31 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-z[49806]: 2024-03-23T17:50:31.949+0000 7f19463b4200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-23T17:50:32.266 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:50:31 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:50:31.907+0000 7feaebda9200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-23T17:50:32.266 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:50:32 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:50:32.016+0000 7feaebda9200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-23T17:50:32.266 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:50:32 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:50:32.154+0000 7feaebda9200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-23T17:50:32.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:32.386+0000 7f2aa2a00700 1 -- 172.21.15.86:0/3103842014 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f2a880a1170 msgr2=0x7f2a880a3630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/2734784244 2024-03-23T17:50:32.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:32.386+0000 7f2aa2a00700 1 --2- 172.21.15.86:0/3103842014 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f2a880a1170 0x7f2a880a3630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-23T17:50:32.557 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Mar 23 17:50:32 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-y[80565]: 2024-03-23T17:50:32.294+0000 7f3a49840200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-23T17:50:32.585 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Mar 23 17:50:32 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-z[49806]: 2024-03-23T17:50:32.322+0000 7f19463b4200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-23T17:50:32.630 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:50:32 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:50:32.353+0000 7feaebda9200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-23T17:50:32.630 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:50:32 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:50:32.518+0000 7feaebda9200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-23T17:50:32.967 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Mar 23 17:50:32 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-y[80565]: 2024-03-23T17:50:32.679+0000 7f3a49840200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-23T17:50:32.967 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Mar 23 17:50:32 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-y[80565]: 2024-03-23T17:50:32.873+0000 7f3a49840200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-23T17:50:32.968 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Mar 23 17:50:32 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-y[80565]: 2024-03-23T17:50:32.967+0000 7f3a49840200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-23T17:50:32.998 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Mar 23 17:50:32 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-z[49806]: 2024-03-23T17:50:32.710+0000 7f19463b4200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-23T17:50:32.999 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Mar 23 17:50:32 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-z[49806]: 2024-03-23T17:50:32.905+0000 7f19463b4200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-23T17:50:32.999 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Mar 23 17:50:32 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-z[49806]: 2024-03-23T17:50:32.998+0000 7f19463b4200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-23T17:50:33.016 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:50:32 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:50:32.629+0000 7feaebda9200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-23T17:50:33.517 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:50:33 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:50:33.091+0000 7feaebda9200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-23T17:50:33.944 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:50:33 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:50:33.573+0000 7feaebda9200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-23T17:50:33.944 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:50:33 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:50:33.819+0000 7feaebda9200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-23T17:50:33.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:33.988+0000 7f2aa2a00700 1 -- 172.21.15.86:0/3103842014 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f2a880a1170 msgr2=0x7f2a880a3630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/2734784244 2024-03-23T17:50:33.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:33.988+0000 7f2aa2a00700 1 --2- 172.21.15.86:0/3103842014 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f2a880a1170 0x7f2a880a3630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-23T17:50:34.057 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Mar 23 17:50:33 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-y[80565]: 2024-03-23T17:50:33.725+0000 7f3a49840200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-23T17:50:34.057 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Mar 23 17:50:33 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-y[80565]: 2024-03-23T17:50:33.823+0000 7f3a49840200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-23T17:50:34.057 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Mar 23 17:50:33 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-y[80565]: 2024-03-23T17:50:33.971+0000 7f3a49840200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-23T17:50:34.086 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Mar 23 17:50:33 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-z[49806]: 2024-03-23T17:50:33.758+0000 7f19463b4200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-23T17:50:34.086 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Mar 23 17:50:33 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-z[49806]: 2024-03-23T17:50:33.855+0000 7f19463b4200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-23T17:50:34.086 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Mar 23 17:50:34 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-z[49806]: 2024-03-23T17:50:34.002+0000 7f19463b4200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-23T17:50:34.266 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:50:33 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:50:33.942+0000 7feaebda9200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-23T17:50:34.964 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Mar 23 17:50:34 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-y[80565]: 2024-03-23T17:50:34.590+0000 7f3a49840200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-23T17:50:34.965 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Mar 23 17:50:34 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-y[80565]: 2024-03-23T17:50:34.771+0000 7f3a49840200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-23T17:50:35.001 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Mar 23 17:50:34 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-z[49806]: 2024-03-23T17:50:34.626+0000 7f19463b4200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-23T17:50:35.001 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Mar 23 17:50:34 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-z[49806]: 2024-03-23T17:50:34.809+0000 7f19463b4200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-23T17:50:35.001 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Mar 23 17:50:35 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-z[49806]: 2024-03-23T17:50:35.001+0000 7f19463b4200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-23T17:50:35.202 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:50:34 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:50:34.886+0000 7feaebda9200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-23T17:50:35.202 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:50:35 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:50:35.010+0000 7feaebda9200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-23T17:50:35.307 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Mar 23 17:50:34 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-y[80565]: 2024-03-23T17:50:34.964+0000 7f3a49840200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-23T17:50:35.516 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:50:35 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:50:35.200+0000 7feaebda9200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-23T17:50:36.170 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Mar 23 17:50:35 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-y[80565]: 2024-03-23T17:50:35.887+0000 7f3a49840200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-23T17:50:36.212 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Mar 23 17:50:35 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-z[49806]: 2024-03-23T17:50:35.928+0000 7f19463b4200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-23T17:50:36.212 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Mar 23 17:50:36 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-z[49806]: 2024-03-23T17:50:36.211+0000 7f19463b4200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-23T17:50:36.266 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:50:35 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:50:35.997+0000 7feaebda9200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-23T17:50:36.266 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:50:36 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:50:36.238+0000 7feaebda9200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-23T17:50:36.447 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Mar 23 17:50:36 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-y[80565]: 2024-03-23T17:50:36.170+0000 7f3a49840200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-23T17:50:36.447 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Mar 23 17:50:36 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-y[80565]: 2024-03-23T17:50:36.258+0000 7f3a49840200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-23T17:50:36.490 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Mar 23 17:50:36 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-z[49806]: 2024-03-23T17:50:36.300+0000 7f19463b4200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-23T17:50:36.490 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Mar 23 17:50:36 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-z[49806]: 2024-03-23T17:50:36.489+0000 7f19463b4200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-23T17:50:36.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:36.687+0000 7f2a9be00700 1 -- 172.21.15.86:0/3103842014 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mgrmap(e 20) v1 ==== 135882+0+0 (secure 0 0 0) 0x7f2a94097f00 con 0x7f2aa4073810 2024-03-23T17:50:36.766 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:50:36 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:50:36.494+0000 7feaebda9200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-23T17:50:36.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:36 smithi086 ceph-mon[38221]: Standby manager daemon y restarted 2024-03-23T17:50:36.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:36 smithi086 ceph-mon[38221]: Standby manager daemon y started 2024-03-23T17:50:36.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:36 smithi086 ceph-mon[38221]: from='mgr.? 172.21.15.92:0/1645143844' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-23T17:50:36.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:36 smithi086 ceph-mon[38221]: from='mgr.? 172.21.15.92:0/1645143844' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-23T17:50:36.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:36 smithi086 ceph-mon[38221]: from='mgr.? 172.21.15.92:0/1645143844' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-23T17:50:36.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:36 smithi086 ceph-mon[38221]: from='mgr.? 172.21.15.92:0/1645143844' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-23T17:50:36.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:36 smithi086 ceph-mon[38221]: Standby manager daemon z restarted 2024-03-23T17:50:36.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:36 smithi086 ceph-mon[38221]: Standby manager daemon z started 2024-03-23T17:50:36.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:36 smithi086 ceph-mon[38221]: from='mgr.? 172.21.15.152:0/1259557590' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-23T17:50:36.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:36 smithi086 ceph-mon[38221]: from='mgr.? 172.21.15.152:0/1259557590' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-23T17:50:36.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:36 smithi086 ceph-mon[38221]: from='mgr.? 172.21.15.152:0/1259557590' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-23T17:50:36.768 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:36 smithi086 ceph-mon[38221]: from='mgr.? 172.21.15.152:0/1259557590' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-23T17:50:36.807 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Mar 23 17:50:36 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-y[80565]: 2024-03-23T17:50:36.447+0000 7f3a49840200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-23T17:50:36.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:36 smithi092 ceph-mon[79089]: Standby manager daemon y restarted 2024-03-23T17:50:36.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:36 smithi092 ceph-mon[79089]: Standby manager daemon y started 2024-03-23T17:50:36.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:36 smithi092 ceph-mon[79089]: from='mgr.? 172.21.15.92:0/1645143844' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-23T17:50:36.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:36 smithi092 ceph-mon[79089]: from='mgr.? 172.21.15.92:0/1645143844' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-23T17:50:36.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:36 smithi092 ceph-mon[79089]: from='mgr.? 172.21.15.92:0/1645143844' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-23T17:50:36.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:36 smithi092 ceph-mon[79089]: from='mgr.? 172.21.15.92:0/1645143844' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-23T17:50:36.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:36 smithi092 ceph-mon[79089]: Standby manager daemon z restarted 2024-03-23T17:50:36.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:36 smithi092 ceph-mon[79089]: Standby manager daemon z started 2024-03-23T17:50:36.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:36 smithi092 ceph-mon[79089]: from='mgr.? 172.21.15.152:0/1259557590' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-23T17:50:36.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:36 smithi092 ceph-mon[79089]: from='mgr.? 172.21.15.152:0/1259557590' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-23T17:50:36.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:36 smithi092 ceph-mon[79089]: from='mgr.? 172.21.15.152:0/1259557590' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-23T17:50:36.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:36 smithi092 ceph-mon[79089]: from='mgr.? 172.21.15.152:0/1259557590' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-23T17:50:36.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:36 smithi152 ceph-mon[47949]: Standby manager daemon y restarted 2024-03-23T17:50:36.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:36 smithi152 ceph-mon[47949]: Standby manager daemon y started 2024-03-23T17:50:36.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:36 smithi152 ceph-mon[47949]: from='mgr.? 172.21.15.92:0/1645143844' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-23T17:50:36.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:36 smithi152 ceph-mon[47949]: from='mgr.? 172.21.15.92:0/1645143844' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-23T17:50:36.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:36 smithi152 ceph-mon[47949]: from='mgr.? 172.21.15.92:0/1645143844' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-23T17:50:36.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:36 smithi152 ceph-mon[47949]: from='mgr.? 172.21.15.92:0/1645143844' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-23T17:50:36.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:36 smithi152 ceph-mon[47949]: Standby manager daemon z restarted 2024-03-23T17:50:36.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:36 smithi152 ceph-mon[47949]: Standby manager daemon z started 2024-03-23T17:50:36.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:36 smithi152 ceph-mon[47949]: from='mgr.? 172.21.15.152:0/1259557590' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-23T17:50:36.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:36 smithi152 ceph-mon[47949]: from='mgr.? 172.21.15.152:0/1259557590' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-23T17:50:36.837 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:36 smithi152 ceph-mon[47949]: from='mgr.? 172.21.15.152:0/1259557590' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-23T17:50:36.837 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:36 smithi152 ceph-mon[47949]: from='mgr.? 172.21.15.152:0/1259557590' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-23T17:50:37.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:37.191+0000 7f2aa2a00700 1 -- 172.21.15.86:0/3103842014 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f2a880a1170 msgr2=0x7f2a880a3630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/2734784244 2024-03-23T17:50:37.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:37.191+0000 7f2aa2a00700 1 --2- 172.21.15.86:0/3103842014 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f2a880a1170 0x7f2a880a3630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-03-23T17:50:38.002 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:37 smithi086 ceph-mon[38221]: mgrmap e20: x(active, since 7m), standbys: y, z 2024-03-23T17:50:38.002 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:50:37 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:50:37.625+0000 7feaebda9200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-23T17:50:38.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:37 smithi092 ceph-mon[79089]: mgrmap e20: x(active, since 7m), standbys: y, z 2024-03-23T17:50:38.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:37 smithi152 ceph-mon[47949]: mgrmap e20: x(active, since 7m), standbys: y, z 2024-03-23T17:50:38.267 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:50:38 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:50:38.000+0000 7feaebda9200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-23T17:50:38.267 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:50:38 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:50:38.119+0000 7feaebda9200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-23T17:50:38.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:38.391+0000 7f2a9be00700 1 -- 172.21.15.86:0/3103842014 <== mon.1 v2:172.21.15.152:3300/0 8 ==== mgrmap(e 21) v1 ==== 135550+0+0 (secure 0 0 0) 0x7f2a940cbe60 con 0x7f2aa4073810 2024-03-23T17:50:38.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:38.391+0000 7f2a9be00700 1 -- 172.21.15.86:0/3103842014 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f2a880a1170 msgr2=0x7f2a880a3630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-23T17:50:38.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:38.391+0000 7f2a9be00700 1 --2- 172.21.15.86:0/3103842014 >> [v2:172.21.15.86:6800/2734784244,v1:172.21.15.86:6801/2734784244] conn(0x7f2a880a1170 0x7f2a880a3630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:38.635 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:50:38 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:50:38.373+0000 7feaebda9200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-23T17:50:38.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:38 smithi092 ceph-mon[79089]: Active manager daemon x restarted 2024-03-23T17:50:38.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:38 smithi092 ceph-mon[79089]: Activating manager daemon x 2024-03-23T17:50:38.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:38 smithi092 ceph-mon[79089]: osdmap e83: 12 total, 12 up, 12 in 2024-03-23T17:50:38.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:38 smithi092 ceph-mon[79089]: mgrmap e21: x(active, starting, since 0.0120566s), standbys: y, z 2024-03-23T17:50:38.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:38 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-23T17:50:38.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:38 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T17:50:38.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:38 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T17:50:38.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:38 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-23T17:50:38.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:38 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-23T17:50:38.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:38 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-23T17:50:38.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:38 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-23T17:50:38.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:38 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-23T17:50:38.809 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:38 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-23T17:50:38.809 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:38 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-23T17:50:38.809 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:38 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-23T17:50:38.809 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:38 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-23T17:50:38.809 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:38 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-23T17:50:38.809 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:38 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-23T17:50:38.809 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:38 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-23T17:50:38.809 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:38 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-23T17:50:38.809 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:38 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-23T17:50:38.809 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:38 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-23T17:50:38.809 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:38 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T17:50:38.809 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:38 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T17:50:38.809 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:38 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T17:50:38.809 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:38 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T17:50:38.810 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:38 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T17:50:38.810 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:38 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T17:50:38.810 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:38 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-23T17:50:38.810 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:38 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-23T17:50:38.810 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:38 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-23T17:50:38.810 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:38 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-23T17:50:38.810 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:38 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-23T17:50:38.810 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:38 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T17:50:38.810 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:38 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-23T17:50:38.810 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:38 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-23T17:50:38.810 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:38 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-23T17:50:38.810 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:38 smithi092 ceph-mon[79089]: Manager daemon x is now available 2024-03-23T17:50:38.810 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:38 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:50:38.810 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:38 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:50:38.810 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:38 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-23T17:50:38.811 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:38 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-23T17:50:38.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:38 smithi152 ceph-mon[47949]: Active manager daemon x restarted 2024-03-23T17:50:38.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:38 smithi152 ceph-mon[47949]: Activating manager daemon x 2024-03-23T17:50:38.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:38 smithi152 ceph-mon[47949]: osdmap e83: 12 total, 12 up, 12 in 2024-03-23T17:50:38.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:38 smithi152 ceph-mon[47949]: mgrmap e21: x(active, starting, since 0.0120566s), standbys: y, z 2024-03-23T17:50:38.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:38 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-23T17:50:38.837 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:38 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T17:50:38.837 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:38 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T17:50:38.837 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:38 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-23T17:50:38.837 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:38 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-23T17:50:38.837 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:38 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-23T17:50:38.837 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:38 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-23T17:50:38.837 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:38 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-23T17:50:38.837 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:38 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-23T17:50:38.837 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:38 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-23T17:50:38.837 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:38 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-23T17:50:38.837 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:38 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-23T17:50:38.837 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:38 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-23T17:50:38.837 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:38 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-23T17:50:38.837 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:38 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-23T17:50:38.837 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:38 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-23T17:50:38.838 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:38 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-23T17:50:38.838 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:38 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-23T17:50:38.838 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:38 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T17:50:38.838 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:38 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T17:50:38.838 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:38 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T17:50:38.838 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:38 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T17:50:38.838 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:38 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T17:50:38.838 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:38 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T17:50:38.838 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:38 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-23T17:50:38.838 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:38 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-23T17:50:38.838 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:38 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-23T17:50:38.838 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:38 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-23T17:50:38.838 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:38 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-23T17:50:38.838 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:38 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T17:50:38.839 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:38 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-23T17:50:38.839 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:38 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-23T17:50:38.839 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:38 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-23T17:50:38.839 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:38 smithi152 ceph-mon[47949]: Manager daemon x is now available 2024-03-23T17:50:38.839 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:38 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:50:38.839 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:38 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:50:38.839 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:38 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-23T17:50:38.839 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:38 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-23T17:50:39.016 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:50:38 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:50:38.651+0000 7fea40800700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-23T17:50:39.017 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:50:38 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:50:38.651+0000 7fea40800700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-23T17:50:39.017 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:50:38 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:50:38.651+0000 7fea40800700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-23T17:50:39.017 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:50:38 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:50:38.651+0000 7fea40800700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-23T17:50:39.017 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:50:38 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:50:38.651+0000 7fea40800700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-23T17:50:39.017 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:50:38 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:50:38.652+0000 7fea3d600700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-23T17:50:39.017 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:50:38 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:50:38.652+0000 7fea3d600700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-23T17:50:39.017 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:50:38 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:50:38.652+0000 7fea3d600700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-23T17:50:39.017 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:50:38 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:50:38.652+0000 7fea3d600700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-23T17:50:39.017 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:50:38 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:50:38.652+0000 7fea3d600700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-23T17:50:39.018 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:38 smithi086 ceph-mon[38221]: Active manager daemon x restarted 2024-03-23T17:50:39.018 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:38 smithi086 ceph-mon[38221]: Activating manager daemon x 2024-03-23T17:50:39.018 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:38 smithi086 ceph-mon[38221]: osdmap e83: 12 total, 12 up, 12 in 2024-03-23T17:50:39.018 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:38 smithi086 ceph-mon[38221]: mgrmap e21: x(active, starting, since 0.0120566s), standbys: y, z 2024-03-23T17:50:39.018 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:38 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-23T17:50:39.018 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:38 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T17:50:39.018 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:38 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T17:50:39.018 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:38 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-23T17:50:39.018 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:38 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-23T17:50:39.019 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:38 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-23T17:50:39.019 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:38 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-23T17:50:39.019 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:38 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-23T17:50:39.019 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:38 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-23T17:50:39.019 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:38 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-23T17:50:39.019 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:38 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-23T17:50:39.019 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:38 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-23T17:50:39.019 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:38 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-23T17:50:39.019 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:38 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-23T17:50:39.019 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:38 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-23T17:50:39.019 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:38 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-23T17:50:39.019 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:38 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-23T17:50:39.020 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:38 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-23T17:50:39.020 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:38 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T17:50:39.020 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:38 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T17:50:39.020 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:38 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T17:50:39.020 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:38 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T17:50:39.020 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:38 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T17:50:39.020 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:38 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T17:50:39.020 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:38 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-23T17:50:39.020 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:38 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-23T17:50:39.020 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:38 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-23T17:50:39.020 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:38 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-23T17:50:39.020 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:38 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-23T17:50:39.021 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:38 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T17:50:39.021 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:38 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-23T17:50:39.021 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:38 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-23T17:50:39.021 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:38 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-23T17:50:39.021 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:38 smithi086 ceph-mon[38221]: Manager daemon x is now available 2024-03-23T17:50:39.021 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:38 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:50:39.021 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:38 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:50:39.021 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:38 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-23T17:50:39.021 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:38 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-23T17:50:39.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:39.399+0000 7f2a9be00700 1 -- 172.21.15.86:0/3103842014 <== mon.1 v2:172.21.15.152:3300/0 9 ==== mgrmap(e 22) v1 ==== 135677+0+0 (secure 0 0 0) 0x7f2a94007b40 con 0x7f2aa4073810 2024-03-23T17:50:39.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:39.399+0000 7f2a9be00700 1 --2- 172.21.15.86:0/3103842014 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f2a8801b960 0x7f2a8801bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:39.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:39.399+0000 7f2a9be00700 1 -- 172.21.15.86:0/3103842014 --> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f2aa40611f0 con 0x7f2a8801b960 2024-03-23T17:50:39.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:39.402+0000 7f2aa2a00700 1 --2- 172.21.15.86:0/3103842014 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f2a8801b960 0x7f2a8801bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:39.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:39.409+0000 7f2aa2a00700 1 --2- 172.21.15.86:0/3103842014 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f2a8801b960 0x7f2a8801bd40 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f2aa4074b10 tx=0x7f2a90009260 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:39.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:39.490+0000 7f2a9be00700 1 -- 172.21.15.86:0/3103842014 <== mgr.15375 v2:172.21.15.86:6800/4022683855 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f2aa40611f0 con 0x7f2a8801b960 2024-03-23T17:50:39.491 INFO:teuthology.orchestra.run.smithi086.stdout:Schedule set for path / 2024-03-23T17:50:39.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:39.492+0000 7f2aaa000700 1 -- 172.21.15.86:0/3103842014 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f2a8801b960 msgr2=0x7f2a8801bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:39.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:39.492+0000 7f2aaa000700 1 --2- 172.21.15.86:0/3103842014 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f2a8801b960 0x7f2a8801bd40 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f2aa4074b10 tx=0x7f2a90009260 comp rx=0 tx=0).stop 2024-03-23T17:50:39.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:39.492+0000 7f2aaa000700 1 -- 172.21.15.86:0/3103842014 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2aa4073810 msgr2=0x7f2aa419d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:39.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:39.492+0000 7f2aaa000700 1 --2- 172.21.15.86:0/3103842014 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2aa4073810 0x7f2aa419d2e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2a94005910 tx=0x7f2a94009580 comp rx=0 tx=0).stop 2024-03-23T17:50:39.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:39.492+0000 7f2aa3e00700 1 -- 172.21.15.86:0/3103842014 reap_dead start 2024-03-23T17:50:39.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:39.492+0000 7f2aaa000700 1 -- 172.21.15.86:0/3103842014 shutdown_connections 2024-03-23T17:50:39.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:39.492+0000 7f2aaa000700 1 -- 172.21.15.86:0/3103842014 >> 172.21.15.86:0/3103842014 conn(0x7f2aa40fc2b0 msgr2=0x7f2aa4100690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:39.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:39.494+0000 7f2aaa000700 1 -- 172.21.15.86:0/3103842014 shutdown_connections 2024-03-23T17:50:39.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:39.494+0000 7f2aaa000700 1 -- 172.21.15.86:0/3103842014 wait complete. 2024-03-23T17:50:39.513 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-03-23T17:50:39.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:39.716+0000 7fd5df200700 1 -- 172.21.15.86:0/4242162692 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5d8074df0 msgr2=0x7fd5d8073240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:39.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:39.716+0000 7fd5df200700 1 --2- 172.21.15.86:0/4242162692 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5d8074df0 0x7fd5d8073240 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd5c8009990 tx=0x7fd5c8009ca0 comp rx=0 tx=0).stop 2024-03-23T17:50:39.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:39.716+0000 7fd5df200700 1 -- 172.21.15.86:0/4242162692 shutdown_connections 2024-03-23T17:50:39.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:39.716+0000 7fd5df200700 1 --2- 172.21.15.86:0/4242162692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5d810d2f0 0x7fd5d810f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:39.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:39.716+0000 7fd5df200700 1 --2- 172.21.15.86:0/4242162692 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5d8074df0 0x7fd5d8073240 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:39.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:39.716+0000 7fd5df200700 1 --2- 172.21.15.86:0/4242162692 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd5d8073810 0x7fd5d8073c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:39.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:39.716+0000 7fd5df200700 1 -- 172.21.15.86:0/4242162692 >> 172.21.15.86:0/4242162692 conn(0x7fd5d80fc2b0 msgr2=0x7fd5d80fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:39.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:39.717+0000 7fd5df200700 1 -- 172.21.15.86:0/4242162692 shutdown_connections 2024-03-23T17:50:39.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:39.717+0000 7fd5df200700 1 -- 172.21.15.86:0/4242162692 wait complete. 2024-03-23T17:50:39.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:39.718+0000 7fd5df200700 1 Processor -- start 2024-03-23T17:50:39.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:39.718+0000 7fd5df200700 1 -- start start 2024-03-23T17:50:39.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:39.719+0000 7fd5df200700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd5d8073810 0x7fd5d819d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:39.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:39.719+0000 7fd5df200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5d8074df0 0x7fd5d819d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:39.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:39.719+0000 7fd5df200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5d810d2f0 0x7fd5d81a47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:39.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:39.719+0000 7fd5df200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d81a4dc0 con 0x7fd5d8074df0 2024-03-23T17:50:39.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:39.719+0000 7fd5df200700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d81a4f00 con 0x7fd5d810d2f0 2024-03-23T17:50:39.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:39.719+0000 7fd5df200700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d81a5090 con 0x7fd5d8073810 2024-03-23T17:50:39.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:39.719+0000 7fd5dd200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5d810d2f0 0x7fd5d81a47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:39.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:39.719+0000 7fd5d7e00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd5d8073810 0x7fd5d819d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:39.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:39.719+0000 7fd5dd200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5d810d2f0 0x7fd5d81a47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:56804/0 (socket says 172.21.15.86:56804) 2024-03-23T17:50:39.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:39.719+0000 7fd5dd200700 1 -- 172.21.15.86:0/865805818 learned_addr learned my addr 172.21.15.86:0/865805818 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:39.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:39.719+0000 7fd5d7400700 1 --2- 172.21.15.86:0/865805818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5d8074df0 0x7fd5d819d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:39.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:39.720+0000 7fd5dd200700 1 -- 172.21.15.86:0/865805818 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd5d8073810 msgr2=0x7fd5d819d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:39.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:39.720+0000 7fd5dd200700 1 --2- 172.21.15.86:0/865805818 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd5d8073810 0x7fd5d819d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:39.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:39.720+0000 7fd5dd200700 1 -- 172.21.15.86:0/865805818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5d8074df0 msgr2=0x7fd5d819d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:39.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:39.720+0000 7fd5dd200700 1 --2- 172.21.15.86:0/865805818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5d8074df0 0x7fd5d819d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:39.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:39.720+0000 7fd5dd200700 1 -- 172.21.15.86:0/865805818 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5c80095f0 con 0x7fd5d810d2f0 2024-03-23T17:50:39.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:39.720+0000 7fd5dd200700 1 --2- 172.21.15.86:0/865805818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5d810d2f0 0x7fd5d81a47f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd5d000ef30 tx=0x7fd5d000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:39.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:39.720+0000 7fd5d4c00700 1 -- 172.21.15.86:0/865805818 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d0018070 con 0x7fd5d810d2f0 2024-03-23T17:50:39.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:39.720+0000 7fd5d4c00700 1 -- 172.21.15.86:0/865805818 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd5d000f040 con 0x7fd5d810d2f0 2024-03-23T17:50:39.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:39.720+0000 7fd5df200700 1 -- 172.21.15.86:0/865805818 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5d81a5310 con 0x7fd5d810d2f0 2024-03-23T17:50:39.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:39.720+0000 7fd5d4c00700 1 -- 172.21.15.86:0/865805818 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d000cdb0 con 0x7fd5d810d2f0 2024-03-23T17:50:39.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:39.720+0000 7fd5df200700 1 -- 172.21.15.86:0/865805818 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5d81a5820 con 0x7fd5d810d2f0 2024-03-23T17:50:39.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:39.721+0000 7fd5df200700 1 -- 172.21.15.86:0/865805818 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5d8103550 con 0x7fd5d810d2f0 2024-03-23T17:50:39.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:39.725+0000 7fd5d4c00700 1 -- 172.21.15.86:0/865805818 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 22) v1 ==== 135677+0+0 (secure 0 0 0) 0x7fd5d0003a00 con 0x7fd5d810d2f0 2024-03-23T17:50:39.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:39.726+0000 7fd5d4c00700 1 --2- 172.21.15.86:0/865805818 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fd5bc0a0f20 0x7fd5bc0a33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:39.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:39.727+0000 7fd5d4c00700 1 -- 172.21.15.86:0/865805818 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fd5d00cc1b0 con 0x7fd5d810d2f0 2024-03-23T17:50:39.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:39.727+0000 7fd5d7e00700 1 --2- 172.21.15.86:0/865805818 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fd5bc0a0f20 0x7fd5bc0a33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:39.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:39.728+0000 7fd5d7e00700 1 --2- 172.21.15.86:0/865805818 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fd5bc0a0f20 0x7fd5bc0a33e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd5c800b2e0 tx=0x7fd5c8005fd0 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:39.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:39.730+0000 7fd5d4c00700 1 -- 172.21.15.86:0/865805818 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fd5d0095bf0 con 0x7fd5d810d2f0 2024-03-23T17:50:39.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:39.989+0000 7fd5df200700 1 -- 172.21.15.86:0/865805818 --> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) v1 -- 0x7fd5d802cdc0 con 0x7fd5bc0a0f20 2024-03-23T17:50:40.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.055+0000 7fd5d4c00700 1 -- 172.21.15.86:0/865805818 <== mgr.15375 v2:172.21.15.86:6800/4022683855 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7fd5d802cdc0 con 0x7fd5bc0a0f20 2024-03-23T17:50:40.056 INFO:teuthology.orchestra.run.smithi086.stdout:Retention added to path / 2024-03-23T17:50:40.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.058+0000 7fd5df200700 1 -- 172.21.15.86:0/865805818 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fd5bc0a0f20 msgr2=0x7fd5bc0a33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:40.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.058+0000 7fd5df200700 1 --2- 172.21.15.86:0/865805818 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fd5bc0a0f20 0x7fd5bc0a33e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd5c800b2e0 tx=0x7fd5c8005fd0 comp rx=0 tx=0).stop 2024-03-23T17:50:40.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.058+0000 7fd5df200700 1 -- 172.21.15.86:0/865805818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5d810d2f0 msgr2=0x7fd5d81a47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:40.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.058+0000 7fd5df200700 1 --2- 172.21.15.86:0/865805818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5d810d2f0 0x7fd5d81a47f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd5d000ef30 tx=0x7fd5d000c3a0 comp rx=0 tx=0).stop 2024-03-23T17:50:40.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.058+0000 7fd5df200700 1 -- 172.21.15.86:0/865805818 shutdown_connections 2024-03-23T17:50:40.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.058+0000 7fd5df200700 1 --2- 172.21.15.86:0/865805818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5d810d2f0 0x7fd5d81a47f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:40.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.058+0000 7fd5df200700 1 --2- 172.21.15.86:0/865805818 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fd5bc0a0f20 0x7fd5bc0a33e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:40.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.058+0000 7fd5df200700 1 --2- 172.21.15.86:0/865805818 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd5d8073810 0x7fd5d819d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:40.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.058+0000 7fd5df200700 1 --2- 172.21.15.86:0/865805818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5d8074df0 0x7fd5d819d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:40.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.058+0000 7fd5df200700 1 -- 172.21.15.86:0/865805818 >> 172.21.15.86:0/865805818 conn(0x7fd5d80fc2b0 msgr2=0x7fd5d8100690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:40.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.058+0000 7fd5df200700 1 -- 172.21.15.86:0/865805818 shutdown_connections 2024-03-23T17:50:40.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.058+0000 7fd5df200700 1 -- 172.21.15.86:0/865805818 wait complete. 2024-03-23T17:50:40.084 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-03-23T17:50:40.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.287+0000 7f666fe00700 1 -- 172.21.15.86:0/308544917 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6668100d10 msgr2=0x7f6668101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:40.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.287+0000 7f666fe00700 1 --2- 172.21.15.86:0/308544917 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6668100d10 0x7f6668101130 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f665c0099c0 tx=0x7f665c009cd0 comp rx=0 tx=0).stop 2024-03-23T17:50:40.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.287+0000 7f666fe00700 1 -- 172.21.15.86:0/308544917 shutdown_connections 2024-03-23T17:50:40.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.287+0000 7f666fe00700 1 --2- 172.21.15.86:0/308544917 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f66681028a0 0x7f6668109150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:40.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.287+0000 7f666fe00700 1 --2- 172.21.15.86:0/308544917 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6668100d10 0x7f6668101130 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:40.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.287+0000 7f666fe00700 1 --2- 172.21.15.86:0/308544917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6668101f00 0x7f6668102360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:40.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.287+0000 7f666fe00700 1 -- 172.21.15.86:0/308544917 >> 172.21.15.86:0/308544917 conn(0x7f66680fc2a0 msgr2=0x7f66680fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:40.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.287+0000 7f666fe00700 1 -- 172.21.15.86:0/308544917 shutdown_connections 2024-03-23T17:50:40.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.287+0000 7f666fe00700 1 -- 172.21.15.86:0/308544917 wait complete. 2024-03-23T17:50:40.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.289+0000 7f666fe00700 1 Processor -- start 2024-03-23T17:50:40.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.289+0000 7f666fe00700 1 -- start start 2024-03-23T17:50:40.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.289+0000 7f666fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6668100d10 0x7f6668198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:40.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.289+0000 7f666fe00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6668101f00 0x7f6668199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:40.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.290+0000 7f666fe00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66681028a0 0x7f66681a0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:40.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.290+0000 7f666fe00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66681a09f0 con 0x7f6668100d10 2024-03-23T17:50:40.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.290+0000 7f666fe00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66681a0b30 con 0x7f66681028a0 2024-03-23T17:50:40.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.290+0000 7f666fe00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66681a0cc0 con 0x7f6668101f00 2024-03-23T17:50:40.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.290+0000 7f666d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6668100d10 0x7f6668198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:40.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.290+0000 7f666d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6668100d10 0x7f6668198ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48054/0 (socket says 172.21.15.86:48054) 2024-03-23T17:50:40.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.290+0000 7f666de00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66681028a0 0x7f66681a0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:40.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.290+0000 7f666d400700 1 -- 172.21.15.86:0/3481021367 learned_addr learned my addr 172.21.15.86:0/3481021367 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:40.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.290+0000 7f666ca00700 1 --2- 172.21.15.86:0/3481021367 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6668101f00 0x7f6668199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:40.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.290+0000 7f666d400700 1 -- 172.21.15.86:0/3481021367 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6668101f00 msgr2=0x7f6668199400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:40.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.290+0000 7f666d400700 1 --2- 172.21.15.86:0/3481021367 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6668101f00 0x7f6668199400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:40.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.290+0000 7f666d400700 1 -- 172.21.15.86:0/3481021367 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66681028a0 msgr2=0x7f66681a0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:40.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.290+0000 7f666d400700 1 --2- 172.21.15.86:0/3481021367 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66681028a0 0x7f66681a0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:40.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.290+0000 7f666d400700 1 -- 172.21.15.86:0/3481021367 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f665c0095f0 con 0x7f6668100d10 2024-03-23T17:50:40.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.290+0000 7f666ca00700 1 --2- 172.21.15.86:0/3481021367 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6668101f00 0x7f6668199400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:50:40.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.290+0000 7f666de00700 1 --2- 172.21.15.86:0/3481021367 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66681028a0 0x7f66681a0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:50:40.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.290+0000 7f666d400700 1 --2- 172.21.15.86:0/3481021367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6668100d10 0x7f6668198ec0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f665c006010 tx=0x7f665c005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:40.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.291+0000 7f6666000700 1 -- 172.21.15.86:0/3481021367 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f665c01d070 con 0x7f6668100d10 2024-03-23T17:50:40.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.291+0000 7f6666000700 1 -- 172.21.15.86:0/3481021367 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f665c004050 con 0x7f6668100d10 2024-03-23T17:50:40.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.291+0000 7f666fe00700 1 -- 172.21.15.86:0/3481021367 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66681a0f40 con 0x7f6668100d10 2024-03-23T17:50:40.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.291+0000 7f6666000700 1 -- 172.21.15.86:0/3481021367 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f665c022e00 con 0x7f6668100d10 2024-03-23T17:50:40.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.291+0000 7f666fe00700 1 -- 172.21.15.86:0/3481021367 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66681a12b0 con 0x7f6668100d10 2024-03-23T17:50:40.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.292+0000 7f666fe00700 1 -- 172.21.15.86:0/3481021367 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66680673a0 con 0x7f6668100d10 2024-03-23T17:50:40.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.294+0000 7f6666000700 1 -- 172.21.15.86:0/3481021367 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 22) v1 ==== 135677+0+0 (secure 0 0 0) 0x7f665c022500 con 0x7f6668100d10 2024-03-23T17:50:40.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.295+0000 7f6666000700 1 --2- 172.21.15.86:0/3481021367 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f66500a0f20 0x7f66500a33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:40.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.296+0000 7f6666000700 1 -- 172.21.15.86:0/3481021367 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f665c0cda00 con 0x7f6668100d10 2024-03-23T17:50:40.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.296+0000 7f666ca00700 1 --2- 172.21.15.86:0/3481021367 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f66500a0f20 0x7f66500a33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:40.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.296+0000 7f666ca00700 1 --2- 172.21.15.86:0/3481021367 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f66500a0f20 0x7f66500a33e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6654005fd0 tx=0x7f6654005f60 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:40.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.300+0000 7f6666000700 1 -- 172.21.15.86:0/3481021367 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f665c097400 con 0x7f6668100d10 2024-03-23T17:50:40.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.574+0000 7f666fe00700 1 -- 172.21.15.86:0/3481021367 --> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f6668108e10 con 0x7f66500a0f20 2024-03-23T17:50:40.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.583+0000 7f6666000700 1 -- 172.21.15.86:0/3481021367 <== mgr.15375 v2:172.21.15.86:6800/4022683855 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+298 (secure 0 0 0) 0x7f6668108e10 con 0x7f66500a0f20 2024-03-23T17:50:40.583 INFO:teuthology.orchestra.run.smithi086.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-03-23T00:00:00", "created": "2024-03-23T17:50:39", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-03-23T17:50:40.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.585+0000 7f666fe00700 1 -- 172.21.15.86:0/3481021367 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f66500a0f20 msgr2=0x7f66500a33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:40.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.585+0000 7f666fe00700 1 --2- 172.21.15.86:0/3481021367 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f66500a0f20 0x7f66500a33e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6654005fd0 tx=0x7f6654005f60 comp rx=0 tx=0).stop 2024-03-23T17:50:40.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.585+0000 7f666fe00700 1 -- 172.21.15.86:0/3481021367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6668100d10 msgr2=0x7f6668198ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:40.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.585+0000 7f666fe00700 1 --2- 172.21.15.86:0/3481021367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6668100d10 0x7f6668198ec0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f665c006010 tx=0x7f665c005740 comp rx=0 tx=0).stop 2024-03-23T17:50:40.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.585+0000 7f666fe00700 1 -- 172.21.15.86:0/3481021367 shutdown_connections 2024-03-23T17:50:40.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.585+0000 7f666fe00700 1 --2- 172.21.15.86:0/3481021367 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66681028a0 0x7f66681a0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:40.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.585+0000 7f666fe00700 1 --2- 172.21.15.86:0/3481021367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6668100d10 0x7f6668198ec0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:40.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.585+0000 7f666fe00700 1 --2- 172.21.15.86:0/3481021367 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f66500a0f20 0x7f66500a33e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:40.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.585+0000 7f666fe00700 1 --2- 172.21.15.86:0/3481021367 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6668101f00 0x7f6668199400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:40.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.585+0000 7f666fe00700 1 -- 172.21.15.86:0/3481021367 >> 172.21.15.86:0/3481021367 conn(0x7f66680fc2a0 msgr2=0x7f66681076f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:40.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.586+0000 7f666fe00700 1 -- 172.21.15.86:0/3481021367 shutdown_connections 2024-03-23T17:50:40.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.586+0000 7f666fe00700 1 -- 172.21.15.86:0/3481021367 wait complete. 2024-03-23T17:50:40.609 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-03-23T17:50:40.673 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:40 smithi086 ceph-mon[38221]: mgrmap e22: x(active, since 1.01864s), standbys: y, z 2024-03-23T17:50:40.673 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:40 smithi086 ceph-mon[38221]: [23/Mar/2024:17:50:39] ENGINE Bus STARTING 2024-03-23T17:50:40.673 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:40 smithi086 ceph-mon[38221]: from='client.25444 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:50:40.673 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:40 smithi086 ceph-mon[38221]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T17:50:40.673 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:40 smithi086 ceph-mon[38221]: [23/Mar/2024:17:50:39] ENGINE Serving on https://172.21.15.86:7150 2024-03-23T17:50:40.673 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:40 smithi086 ceph-mon[38221]: scrub summary: idle 2024-03-23T17:50:40.673 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:40 smithi086 ceph-mon[38221]: [23/Mar/2024:17:50:39] ENGINE Serving on http://172.21.15.86:8765 2024-03-23T17:50:40.673 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:40 smithi086 ceph-mon[38221]: [23/Mar/2024:17:50:39] ENGINE Bus STARTED 2024-03-23T17:50:40.673 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:40 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:50:40.674 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:40 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:50:40.674 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:40 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:50:40.674 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:40 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:50:40.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:40 smithi092 ceph-mon[79089]: mgrmap e22: x(active, since 1.01864s), standbys: y, z 2024-03-23T17:50:40.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:40 smithi092 ceph-mon[79089]: [23/Mar/2024:17:50:39] ENGINE Bus STARTING 2024-03-23T17:50:40.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:40 smithi092 ceph-mon[79089]: from='client.25444 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:50:40.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:40 smithi092 ceph-mon[79089]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T17:50:40.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:40 smithi092 ceph-mon[79089]: [23/Mar/2024:17:50:39] ENGINE Serving on https://172.21.15.86:7150 2024-03-23T17:50:40.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:40 smithi092 ceph-mon[79089]: scrub summary: idle 2024-03-23T17:50:40.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:40 smithi092 ceph-mon[79089]: [23/Mar/2024:17:50:39] ENGINE Serving on http://172.21.15.86:8765 2024-03-23T17:50:40.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:40 smithi092 ceph-mon[79089]: [23/Mar/2024:17:50:39] ENGINE Bus STARTED 2024-03-23T17:50:40.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:40 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:50:40.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:40 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:50:40.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:40 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:50:40.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:40 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:50:40.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.820+0000 7f2fcae00700 1 -- 172.21.15.86:0/504087019 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2fc4073810 msgr2=0x7f2fc4073c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:40.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.820+0000 7f2fcae00700 1 --2- 172.21.15.86:0/504087019 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2fc4073810 0x7f2fc4073c90 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2fb0009990 tx=0x7f2fb0009ca0 comp rx=0 tx=0).stop 2024-03-23T17:50:40.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.821+0000 7f2fcae00700 1 -- 172.21.15.86:0/504087019 shutdown_connections 2024-03-23T17:50:40.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.821+0000 7f2fcae00700 1 --2- 172.21.15.86:0/504087019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fc4108bd0 0x7f2fc410b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:40.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.821+0000 7f2fcae00700 1 --2- 172.21.15.86:0/504087019 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2fc4074df0 0x7f2fc4073240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:40.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.821+0000 7f2fcae00700 1 --2- 172.21.15.86:0/504087019 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2fc4073810 0x7f2fc4073c90 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:40.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.821+0000 7f2fcae00700 1 -- 172.21.15.86:0/504087019 >> 172.21.15.86:0/504087019 conn(0x7f2fc40fc280 msgr2=0x7f2fc40fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:40.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.821+0000 7f2fcae00700 1 -- 172.21.15.86:0/504087019 shutdown_connections 2024-03-23T17:50:40.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.821+0000 7f2fcae00700 1 -- 172.21.15.86:0/504087019 wait complete. 2024-03-23T17:50:40.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.822+0000 7f2fcae00700 1 Processor -- start 2024-03-23T17:50:40.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.822+0000 7f2fcae00700 1 -- start start 2024-03-23T17:50:40.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.822+0000 7f2fcae00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fc4073810 0x7f2fc4198f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:40.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.822+0000 7f2fcae00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2fc4074df0 0x7f2fc4199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:40.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.822+0000 7f2fcae00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2fc4108bd0 0x7f2fc41a0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:40.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.822+0000 7f2fcae00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc41a0a20 con 0x7f2fc4073810 2024-03-23T17:50:40.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.822+0000 7f2fcae00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc41a0b60 con 0x7f2fc4074df0 2024-03-23T17:50:40.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.822+0000 7f2fcae00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc41a0cf0 con 0x7f2fc4108bd0 2024-03-23T17:50:40.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.822+0000 7f2fc3400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2fc4074df0 0x7f2fc4199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:40.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.822+0000 7f2fc3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fc4073810 0x7f2fc4198f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:40.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.822+0000 7f2fc8e00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2fc4108bd0 0x7f2fc41a0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:40.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.822+0000 7f2fc3400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2fc4074df0 0x7f2fc4199480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:56888/0 (socket says 172.21.15.86:56888) 2024-03-23T17:50:40.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.822+0000 7f2fc3400700 1 -- 172.21.15.86:0/1348195013 learned_addr learned my addr 172.21.15.86:0/1348195013 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:40.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.823+0000 7f2fc3400700 1 -- 172.21.15.86:0/1348195013 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2fc4108bd0 msgr2=0x7f2fc41a0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:40.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.823+0000 7f2fc3400700 1 --2- 172.21.15.86:0/1348195013 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2fc4108bd0 0x7f2fc41a0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:40.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.823+0000 7f2fc3400700 1 -- 172.21.15.86:0/1348195013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fc4073810 msgr2=0x7f2fc4198f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:40.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.823+0000 7f2fc3400700 1 --2- 172.21.15.86:0/1348195013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fc4073810 0x7f2fc4198f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:40.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.823+0000 7f2fc3400700 1 -- 172.21.15.86:0/1348195013 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fb00095f0 con 0x7f2fc4074df0 2024-03-23T17:50:40.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.823+0000 7f2fc3400700 1 --2- 172.21.15.86:0/1348195013 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2fc4074df0 0x7f2fc4199480 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2fb00098e0 tx=0x7f2fb000f670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:40.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.823+0000 7f2fc0c00700 1 -- 172.21.15.86:0/1348195013 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb001d070 con 0x7f2fc4074df0 2024-03-23T17:50:40.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.823+0000 7f2fcae00700 1 -- 172.21.15.86:0/1348195013 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fc41a0f70 con 0x7f2fc4074df0 2024-03-23T17:50:40.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.823+0000 7f2fc0c00700 1 -- 172.21.15.86:0/1348195013 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2fb0005050 con 0x7f2fc4074df0 2024-03-23T17:50:40.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.823+0000 7f2fc0c00700 1 -- 172.21.15.86:0/1348195013 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb0004b00 con 0x7f2fc4074df0 2024-03-23T17:50:40.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.824+0000 7f2fcae00700 1 -- 172.21.15.86:0/1348195013 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fc41a12e0 con 0x7f2fc4074df0 2024-03-23T17:50:40.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.824+0000 7f2fcae00700 1 -- 172.21.15.86:0/1348195013 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fc4107990 con 0x7f2fc4074df0 2024-03-23T17:50:40.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.826+0000 7f2fc0c00700 1 -- 172.21.15.86:0/1348195013 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 22) v1 ==== 135677+0+0 (secure 0 0 0) 0x7f2fb00051c0 con 0x7f2fc4074df0 2024-03-23T17:50:40.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.827+0000 7f2fc0c00700 1 --2- 172.21.15.86:0/1348195013 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f2fa80a0f20 0x7f2fa80a33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:40.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.828+0000 7f2fc0c00700 1 -- 172.21.15.86:0/1348195013 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f2fb00cd930 con 0x7f2fc4074df0 2024-03-23T17:50:40.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.828+0000 7f2fc3e00700 1 --2- 172.21.15.86:0/1348195013 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f2fa80a0f20 0x7f2fa80a33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:40.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.828+0000 7f2fc3e00700 1 --2- 172.21.15.86:0/1348195013 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f2fa80a0f20 0x7f2fa80a33e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2fb4007c60 tx=0x7f2fb40073d0 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:40.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.830+0000 7f2fc0c00700 1 -- 172.21.15.86:0/1348195013 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f2fb0097330 con 0x7f2fc4074df0 2024-03-23T17:50:40.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:40 smithi152 ceph-mon[47949]: mgrmap e22: x(active, since 1.01864s), standbys: y, z 2024-03-23T17:50:40.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:40 smithi152 ceph-mon[47949]: [23/Mar/2024:17:50:39] ENGINE Bus STARTING 2024-03-23T17:50:40.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:40 smithi152 ceph-mon[47949]: from='client.25444 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:50:40.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:40 smithi152 ceph-mon[47949]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T17:50:40.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:40 smithi152 ceph-mon[47949]: [23/Mar/2024:17:50:39] ENGINE Serving on https://172.21.15.86:7150 2024-03-23T17:50:40.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:40 smithi152 ceph-mon[47949]: scrub summary: idle 2024-03-23T17:50:40.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:40 smithi152 ceph-mon[47949]: [23/Mar/2024:17:50:39] ENGINE Serving on http://172.21.15.86:8765 2024-03-23T17:50:40.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:40 smithi152 ceph-mon[47949]: [23/Mar/2024:17:50:39] ENGINE Bus STARTED 2024-03-23T17:50:40.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:40 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:50:40.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:40 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:50:40.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:40 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:50:40.837 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:40 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:50:40.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:40.964+0000 7f2fc0c00700 1 -- 172.21.15.86:0/1348195013 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mgrmap(e 23) v1 ==== 135985+0+0 (secure 0 0 0) 0x7f2fb0097700 con 0x7f2fc4074df0 2024-03-23T17:50:41.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:41.081+0000 7f2fcae00700 1 -- 172.21.15.86:0/1348195013 --> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f2fc402cd20 con 0x7f2fa80a0f20 2024-03-23T17:50:41.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:41.088+0000 7f2fc0c00700 1 -- 172.21.15.86:0/1348195013 <== mgr.15375 v2:172.21.15.86:6800/4022683855 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f2fc402cd20 con 0x7f2fa80a0f20 2024-03-23T17:50:41.088 INFO:teuthology.orchestra.run.smithi086.stdout:/ 1m 6m3h 2024-03-23T17:50:41.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:41.090+0000 7f2fcae00700 1 -- 172.21.15.86:0/1348195013 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f2fa80a0f20 msgr2=0x7f2fa80a33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:41.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:41.090+0000 7f2fcae00700 1 --2- 172.21.15.86:0/1348195013 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f2fa80a0f20 0x7f2fa80a33e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2fb4007c60 tx=0x7f2fb40073d0 comp rx=0 tx=0).stop 2024-03-23T17:50:41.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:41.090+0000 7f2fcae00700 1 -- 172.21.15.86:0/1348195013 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2fc4074df0 msgr2=0x7f2fc4199480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:41.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:41.090+0000 7f2fcae00700 1 --2- 172.21.15.86:0/1348195013 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2fc4074df0 0x7f2fc4199480 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2fb00098e0 tx=0x7f2fb000f670 comp rx=0 tx=0).stop 2024-03-23T17:50:41.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:41.090+0000 7f2fcae00700 1 -- 172.21.15.86:0/1348195013 shutdown_connections 2024-03-23T17:50:41.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:41.090+0000 7f2fcae00700 1 --2- 172.21.15.86:0/1348195013 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f2fa80a0f20 0x7f2fa80a33e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:41.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:41.090+0000 7f2fcae00700 1 --2- 172.21.15.86:0/1348195013 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2fc4108bd0 0x7f2fc41a0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:41.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:41.090+0000 7f2fcae00700 1 --2- 172.21.15.86:0/1348195013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fc4073810 0x7f2fc4198f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:41.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:41.090+0000 7f2fcae00700 1 --2- 172.21.15.86:0/1348195013 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2fc4074df0 0x7f2fc4199480 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:41.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:41.090+0000 7f2fcae00700 1 -- 172.21.15.86:0/1348195013 >> 172.21.15.86:0/1348195013 conn(0x7f2fc40fc280 msgr2=0x7f2fc4104ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:41.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:41.090+0000 7f2fcae00700 1 -- 172.21.15.86:0/1348195013 shutdown_connections 2024-03-23T17:50:41.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:41.090+0000 7f2fcae00700 1 -- 172.21.15.86:0/1348195013 wait complete. 2024-03-23T17:50:41.110 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-03-23T17:50:41.187 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-03-23T17:50:41.210 INFO:teuthology.run_tasks:Running task background_exec... 2024-03-23T17:50:41.264 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi086.front.sepia.ceph.com 2024-03-23T17:50:41.264 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-03-23T17:50:41.270 INFO:teuthology.run_tasks:Running task workunit... 2024-03-23T17:50:41.285 INFO:tasks.workunit:Pulling workunits from ref 82316b04821a6c2c1d458a881920454bbadb8837 2024-03-23T17:50:41.286 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-23T17:50:41.286 INFO:tasks.workunit:timeout=3h 2024-03-23T17:50:41.286 INFO:tasks.workunit:cleanup=True 2024-03-23T17:50:41.286 DEBUG:teuthology.orchestra.run.smithi086:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-23T17:50:41.330 INFO:teuthology.orchestra.run.smithi086.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-03-23T17:50:41.330 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-03-23T17:50:41.330 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 7ah/122d Inode: 1099511628283 Links: 2 2024-03-23T17:50:41.330 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T17:50:41.330 INFO:teuthology.orchestra.run.smithi086.stdout:Context: system_u:object_r:cephfs_t:s0 2024-03-23T17:50:41.331 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-03-23 17:49:03.627434851 +0000 2024-03-23T17:50:41.331 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-03-23 17:49:03.627434851 +0000 2024-03-23T17:50:41.331 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-03-23 17:50:19.220262961 +0000 2024-03-23T17:50:41.331 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: 2024-03-23 17:49:03.627434851 +0000 2024-03-23T17:50:41.331 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-03-23T17:50:41.332 DEBUG:teuthology.orchestra.run.smithi086:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-03-23T17:50:41.453 DEBUG:teuthology.orchestra.run.smithi086:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 82316b04821a6c2c1d458a881920454bbadb8837 2024-03-23T17:50:41.501 INFO:tasks.workunit.client.0.smithi086.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-23T17:50:41.766 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:41 smithi086 ceph-mon[38221]: from='client.25475 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:50:41.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:41 smithi086 ceph-mon[38221]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T17:50:41.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:41 smithi086 ceph-mon[38221]: from='client.15435 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:50:41.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:41 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:50:41.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:41 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:50:41.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:41 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:50:41.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:41 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:50:41.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:41 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:50:41.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:41 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:50:41.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:41 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:50:41.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:41 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:50:41.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:41 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:50:41.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:41 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:50:41.768 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:41 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:50:41.768 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:41 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:50:41.768 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:41 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:50:41.768 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:41 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:50:41.768 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:41 smithi086 ceph-mon[38221]: Adjusting osd_memory_target on smithi152 to 182.5M 2024-03-23T17:50:41.768 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:41 smithi086 ceph-mon[38221]: Adjusting osd_memory_target on smithi092 to 182.5M 2024-03-23T17:50:41.768 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:41 smithi086 ceph-mon[38221]: Unable to set osd_memory_target on smithi152 to 191374848: error parsing value: Value '191374848' is below minimum 939524096 2024-03-23T17:50:41.768 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:41 smithi086 ceph-mon[38221]: Unable to set osd_memory_target on smithi092 to 191371980: error parsing value: Value '191371980' is below minimum 939524096 2024-03-23T17:50:41.768 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:41 smithi086 ceph-mon[38221]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-03-23T17:50:42.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:41 smithi092 ceph-mon[79089]: from='client.25475 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:50:42.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:41 smithi092 ceph-mon[79089]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T17:50:42.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:41 smithi092 ceph-mon[79089]: from='client.15435 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:50:42.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:41 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:50:42.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:41 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:50:42.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:41 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:50:42.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:41 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:50:42.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:41 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:50:42.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:41 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:50:42.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:41 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:50:42.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:41 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:50:42.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:41 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:50:42.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:41 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:50:42.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:41 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:50:42.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:41 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:50:42.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:41 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:50:42.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:41 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:50:42.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:41 smithi092 ceph-mon[79089]: Adjusting osd_memory_target on smithi152 to 182.5M 2024-03-23T17:50:42.059 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:41 smithi092 ceph-mon[79089]: Adjusting osd_memory_target on smithi092 to 182.5M 2024-03-23T17:50:42.059 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:41 smithi092 ceph-mon[79089]: Unable to set osd_memory_target on smithi152 to 191374848: error parsing value: Value '191374848' is below minimum 939524096 2024-03-23T17:50:42.059 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:41 smithi092 ceph-mon[79089]: Unable to set osd_memory_target on smithi092 to 191371980: error parsing value: Value '191371980' is below minimum 939524096 2024-03-23T17:50:42.059 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:41 smithi092 ceph-mon[79089]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-03-23T17:50:42.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:41 smithi152 ceph-mon[47949]: from='client.25475 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:50:42.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:41 smithi152 ceph-mon[47949]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T17:50:42.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:41 smithi152 ceph-mon[47949]: from='client.15435 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:50:42.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:41 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:50:42.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:41 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:50:42.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:41 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:50:42.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:41 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:50:42.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:41 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:50:42.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:41 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:50:42.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:41 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:50:42.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:41 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:50:42.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:41 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:50:42.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:41 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:50:42.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:41 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:50:42.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:41 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:50:42.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:41 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:50:42.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:41 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:50:42.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:41 smithi152 ceph-mon[47949]: Adjusting osd_memory_target on smithi152 to 182.5M 2024-03-23T17:50:42.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:41 smithi152 ceph-mon[47949]: Adjusting osd_memory_target on smithi092 to 182.5M 2024-03-23T17:50:42.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:41 smithi152 ceph-mon[47949]: Unable to set osd_memory_target on smithi152 to 191374848: error parsing value: Value '191374848' is below minimum 939524096 2024-03-23T17:50:42.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:41 smithi152 ceph-mon[47949]: Unable to set osd_memory_target on smithi092 to 191371980: error parsing value: Value '191371980' is below minimum 939524096 2024-03-23T17:50:42.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:41 smithi152 ceph-mon[47949]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-03-23T17:50:43.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:42 smithi086 ceph-mon[38221]: from='client.25493 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:50:43.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:42 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:50:43.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:42 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:50:43.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:42 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:50:43.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:42 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:50:43.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:42 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:50:43.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:42 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:50:43.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:42 smithi086 ceph-mon[38221]: Adjusting osd_memory_target on smithi086 to 221.5M 2024-03-23T17:50:43.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:42 smithi086 ceph-mon[38221]: Unable to set osd_memory_target on smithi086 to 232352870: error parsing value: Value '232352870' is below minimum 939524096 2024-03-23T17:50:43.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:42 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:50:43.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:42 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:50:43.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:42 smithi086 ceph-mon[38221]: Updating smithi086:/etc/ceph/ceph.conf 2024-03-23T17:50:43.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:42 smithi086 ceph-mon[38221]: Updating smithi092:/etc/ceph/ceph.conf 2024-03-23T17:50:43.268 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:42 smithi086 ceph-mon[38221]: Updating smithi152:/etc/ceph/ceph.conf 2024-03-23T17:50:43.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:42 smithi092 ceph-mon[79089]: from='client.25493 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:50:43.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:42 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:50:43.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:42 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:50:43.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:42 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:50:43.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:42 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:50:43.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:42 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:50:43.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:42 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:50:43.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:42 smithi092 ceph-mon[79089]: Adjusting osd_memory_target on smithi086 to 221.5M 2024-03-23T17:50:43.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:42 smithi092 ceph-mon[79089]: Unable to set osd_memory_target on smithi086 to 232352870: error parsing value: Value '232352870' is below minimum 939524096 2024-03-23T17:50:43.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:42 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:50:43.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:42 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:50:43.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:42 smithi092 ceph-mon[79089]: Updating smithi086:/etc/ceph/ceph.conf 2024-03-23T17:50:43.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:42 smithi092 ceph-mon[79089]: Updating smithi092:/etc/ceph/ceph.conf 2024-03-23T17:50:43.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:42 smithi092 ceph-mon[79089]: Updating smithi152:/etc/ceph/ceph.conf 2024-03-23T17:50:43.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:42 smithi152 ceph-mon[47949]: from='client.25493 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T17:50:43.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:42 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:50:43.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:42 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:50:43.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:42 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:50:43.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:42 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:50:43.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:42 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:50:43.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:42 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-23T17:50:43.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:42 smithi152 ceph-mon[47949]: Adjusting osd_memory_target on smithi086 to 221.5M 2024-03-23T17:50:43.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:42 smithi152 ceph-mon[47949]: Unable to set osd_memory_target on smithi086 to 232352870: error parsing value: Value '232352870' is below minimum 939524096 2024-03-23T17:50:43.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:42 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:50:43.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:42 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:50:43.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:42 smithi152 ceph-mon[47949]: Updating smithi086:/etc/ceph/ceph.conf 2024-03-23T17:50:43.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:42 smithi152 ceph-mon[47949]: Updating smithi092:/etc/ceph/ceph.conf 2024-03-23T17:50:43.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:42 smithi152 ceph-mon[47949]: Updating smithi152:/etc/ceph/ceph.conf 2024-03-23T17:50:44.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:43 smithi086 ceph-mon[38221]: Updating smithi152:/var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T17:50:44.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:43 smithi086 ceph-mon[38221]: Updating smithi092:/var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T17:50:44.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:43 smithi086 ceph-mon[38221]: Updating smithi086:/var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T17:50:44.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:43 smithi086 ceph-mon[38221]: Updating smithi152:/etc/ceph/ceph.client.admin.keyring 2024-03-23T17:50:44.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:43 smithi086 ceph-mon[38221]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T17:50:44.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:43 smithi086 ceph-mon[38221]: Updating smithi092:/etc/ceph/ceph.client.admin.keyring 2024-03-23T17:50:44.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:43 smithi086 ceph-mon[38221]: Updating smithi152:/var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-23T17:50:44.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:43 smithi086 ceph-mon[38221]: Updating smithi092:/var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-23T17:50:44.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:43 smithi086 ceph-mon[38221]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-03-23T17:50:44.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:43 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:50:44.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:43 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:50:44.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:43 smithi086 ceph-mon[38221]: Updating smithi086:/etc/ceph/ceph.client.admin.keyring 2024-03-23T17:50:44.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:43 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:50:44.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:43 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:50:44.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:43 smithi092 ceph-mon[79089]: Updating smithi152:/var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T17:50:44.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:43 smithi092 ceph-mon[79089]: Updating smithi092:/var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T17:50:44.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:43 smithi092 ceph-mon[79089]: Updating smithi086:/var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T17:50:44.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:43 smithi092 ceph-mon[79089]: Updating smithi152:/etc/ceph/ceph.client.admin.keyring 2024-03-23T17:50:44.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:43 smithi092 ceph-mon[79089]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T17:50:44.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:43 smithi092 ceph-mon[79089]: Updating smithi092:/etc/ceph/ceph.client.admin.keyring 2024-03-23T17:50:44.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:43 smithi092 ceph-mon[79089]: Updating smithi152:/var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-23T17:50:44.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:43 smithi092 ceph-mon[79089]: Updating smithi092:/var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-23T17:50:44.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:43 smithi092 ceph-mon[79089]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-03-23T17:50:44.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:43 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:50:44.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:43 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:50:44.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:43 smithi092 ceph-mon[79089]: Updating smithi086:/etc/ceph/ceph.client.admin.keyring 2024-03-23T17:50:44.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:43 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:50:44.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:43 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:50:44.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:43 smithi152 ceph-mon[47949]: Updating smithi152:/var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T17:50:44.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:43 smithi152 ceph-mon[47949]: Updating smithi092:/var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T17:50:44.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:43 smithi152 ceph-mon[47949]: Updating smithi086:/var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T17:50:44.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:43 smithi152 ceph-mon[47949]: Updating smithi152:/etc/ceph/ceph.client.admin.keyring 2024-03-23T17:50:44.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:43 smithi152 ceph-mon[47949]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T17:50:44.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:43 smithi152 ceph-mon[47949]: Updating smithi092:/etc/ceph/ceph.client.admin.keyring 2024-03-23T17:50:44.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:43 smithi152 ceph-mon[47949]: Updating smithi152:/var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-23T17:50:44.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:43 smithi152 ceph-mon[47949]: Updating smithi092:/var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-23T17:50:44.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:43 smithi152 ceph-mon[47949]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-03-23T17:50:44.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:43 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:50:44.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:43 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:50:44.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:43 smithi152 ceph-mon[47949]: Updating smithi086:/etc/ceph/ceph.client.admin.keyring 2024-03-23T17:50:44.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:43 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:50:44.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:43 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:50:45.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:44 smithi086 ceph-mon[38221]: Updating smithi086:/var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-23T17:50:45.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:44 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:50:45.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:44 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:50:45.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:44 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:50:45.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:44 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:50:45.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:44 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:50:45.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:44 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:50:45.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:44 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:50:45.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:44 smithi092 ceph-mon[79089]: Updating smithi086:/var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-23T17:50:45.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:44 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:50:45.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:44 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:50:45.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:44 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:50:45.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:44 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:50:45.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:44 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:50:45.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:44 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:50:45.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:44 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:50:45.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:44 smithi152 ceph-mon[47949]: Updating smithi086:/var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-23T17:50:45.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:44 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:50:45.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:44 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:50:45.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:44 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:50:45.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:44 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:50:45.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:44 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:50:45.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:44 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:50:45.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:44 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:50:46.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:45 smithi086 ceph-mon[38221]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T17:50:46.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:45 smithi092 ceph-mon[79089]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T17:50:46.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:45 smithi152 ceph-mon[47949]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T17:50:48.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:47 smithi086 ceph-mon[38221]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 132 KiB/s wr, 79 op/s 2024-03-23T17:50:48.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:47 smithi092 ceph-mon[79089]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 132 KiB/s wr, 79 op/s 2024-03-23T17:50:48.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:47 smithi152 ceph-mon[47949]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 132 KiB/s wr, 79 op/s 2024-03-23T17:50:50.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:49 smithi086 ceph-mon[38221]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 103 KiB/s wr, 61 op/s 2024-03-23T17:50:50.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:49 smithi092 ceph-mon[79089]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 103 KiB/s wr, 61 op/s 2024-03-23T17:50:50.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:49 smithi152 ceph-mon[47949]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 103 KiB/s wr, 61 op/s 2024-03-23T17:50:52.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:51 smithi086 ceph-mon[38221]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 84 KiB/s wr, 55 op/s 2024-03-23T17:50:52.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:51 smithi092 ceph-mon[79089]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 84 KiB/s wr, 55 op/s 2024-03-23T17:50:52.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:51 smithi152 ceph-mon[47949]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 84 KiB/s wr, 55 op/s 2024-03-23T17:50:54.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:53 smithi086 ceph-mon[38221]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-03-23T17:50:54.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:53 smithi092 ceph-mon[79089]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-03-23T17:50:54.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:53 smithi152 ceph-mon[47949]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-03-23T17:50:54.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.513+0000 7fa37ac00700 1 -- 172.21.15.86:0/287248235 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa374106b90 msgr2=0x7fa374106f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:54.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.513+0000 7fa37ac00700 1 --2- 172.21.15.86:0/287248235 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa374106b90 0x7fa374106f70 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa3680099c0 tx=0x7fa368009cd0 comp rx=0 tx=0).stop 2024-03-23T17:50:54.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.513+0000 7fa37ac00700 1 -- 172.21.15.86:0/287248235 shutdown_connections 2024-03-23T17:50:54.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.513+0000 7fa37ac00700 1 --2- 172.21.15.86:0/287248235 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa374068a40 0x7fa37410d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:54.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.513+0000 7fa37ac00700 1 --2- 172.21.15.86:0/287248235 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa374106b90 0x7fa374106f70 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:54.514 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.513+0000 7fa37ac00700 1 --2- 172.21.15.86:0/287248235 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa374068080 0x7fa374068500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:54.514 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.513+0000 7fa37ac00700 1 -- 172.21.15.86:0/287248235 >> 172.21.15.86:0/287248235 conn(0x7fa374075610 msgr2=0x7fa374075a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:54.514 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.513+0000 7fa37ac00700 1 -- 172.21.15.86:0/287248235 shutdown_connections 2024-03-23T17:50:54.514 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.513+0000 7fa37ac00700 1 -- 172.21.15.86:0/287248235 wait complete. 2024-03-23T17:50:54.516 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.514+0000 7fa37ac00700 1 Processor -- start 2024-03-23T17:50:54.516 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.514+0000 7fa37ac00700 1 -- start start 2024-03-23T17:50:54.516 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.514+0000 7fa37ac00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa374068080 0x7fa374197020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:54.516 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.514+0000 7fa37ac00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa374068a40 0x7fa374197560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:54.516 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.515+0000 7fa37ac00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa374106b90 0x7fa37419b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:54.516 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.515+0000 7fa37ac00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa37419bef0 con 0x7fa374106b90 2024-03-23T17:50:54.517 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.515+0000 7fa37ac00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa37419c060 con 0x7fa374068080 2024-03-23T17:50:54.517 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.515+0000 7fa37ac00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa37419c220 con 0x7fa374068a40 2024-03-23T17:50:54.517 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.515+0000 7fa379800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa374068080 0x7fa374197020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:54.517 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.515+0000 7fa379800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa374068080 0x7fa374197020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:60068/0 (socket says 172.21.15.86:60068) 2024-03-23T17:50:54.517 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.515+0000 7fa379800700 1 -- 172.21.15.86:0/1242652386 learned_addr learned my addr 172.21.15.86:0/1242652386 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:54.517 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.515+0000 7fa378e00700 1 --2- 172.21.15.86:0/1242652386 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa374068a40 0x7fa374197560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:54.517 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.515+0000 7fa37a200700 1 --2- 172.21.15.86:0/1242652386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa374106b90 0x7fa37419b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:54.517 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.515+0000 7fa379800700 1 -- 172.21.15.86:0/1242652386 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa374068a40 msgr2=0x7fa374197560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:54.517 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.515+0000 7fa379800700 1 --2- 172.21.15.86:0/1242652386 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa374068a40 0x7fa374197560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:54.517 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.515+0000 7fa379800700 1 -- 172.21.15.86:0/1242652386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa374106b90 msgr2=0x7fa37419b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:54.517 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.515+0000 7fa379800700 1 --2- 172.21.15.86:0/1242652386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa374106b90 0x7fa37419b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:54.517 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.515+0000 7fa379800700 1 -- 172.21.15.86:0/1242652386 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3680095f0 con 0x7fa374068080 2024-03-23T17:50:54.517 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.516+0000 7fa379800700 1 --2- 172.21.15.86:0/1242652386 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa374068080 0x7fa374197020 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa36800b580 tx=0x7fa36800bfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:54.518 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.516+0000 7fa372000700 1 -- 172.21.15.86:0/1242652386 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa36801c070 con 0x7fa374068080 2024-03-23T17:50:54.518 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.516+0000 7fa372000700 1 -- 172.21.15.86:0/1242652386 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa36800fdf0 con 0x7fa374068080 2024-03-23T17:50:54.518 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.516+0000 7fa37ac00700 1 -- 172.21.15.86:0/1242652386 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa37419c4a0 con 0x7fa374068080 2024-03-23T17:50:54.518 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.516+0000 7fa372000700 1 -- 172.21.15.86:0/1242652386 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa368022c40 con 0x7fa374068080 2024-03-23T17:50:54.518 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.516+0000 7fa37ac00700 1 -- 172.21.15.86:0/1242652386 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3741a3d40 con 0x7fa374068080 2024-03-23T17:50:54.519 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.520+0000 7fa372000700 1 -- 172.21.15.86:0/1242652386 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 24) v1 ==== 136041+0+0 (secure 0 0 0) 0x7fa368020070 con 0x7fa374068080 2024-03-23T17:50:54.521 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.521+0000 7fa372000700 1 --2- 172.21.15.86:0/1242652386 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fa35c0a1320 0x7fa35c0a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:54.521 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.521+0000 7fa372000700 1 -- 172.21.15.86:0/1242652386 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fa3680cdb40 con 0x7fa374068080 2024-03-23T17:50:54.521 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.521+0000 7fa378e00700 1 --2- 172.21.15.86:0/1242652386 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fa35c0a1320 0x7fa35c0a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:54.522 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.522+0000 7fa378e00700 1 --2- 172.21.15.86:0/1242652386 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fa35c0a1320 0x7fa35c0a37e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa374198520 tx=0x7fa360009260 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:54.522 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.522+0000 7fa38fcddb80 1 Processor -- start 2024-03-23T17:50:54.522 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.523+0000 7fa38fcddb80 1 -- start start 2024-03-23T17:50:54.524 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.523+0000 7fa38fcddb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee1f6ca2d0 0x55ee1f6ca6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:54.524 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.523+0000 7fa38fcddb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ee1f6cabf0 0x55ee1f6d5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:54.524 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.523+0000 7fa38fcddb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ee1f6d5ab0 0x55ee1f6d7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:54.524 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.523+0000 7fa38fcddb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee1f623930 con 0x55ee1f6ca2d0 2024-03-23T17:50:54.524 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.523+0000 7fa38fcddb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee1f6212f0 con 0x55ee1f6cabf0 2024-03-23T17:50:54.525 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.523+0000 7fa38fcddb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee1f61cbe0 con 0x55ee1f6d5ab0 2024-03-23T17:50:54.525 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.523+0000 7fa379800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee1f6ca2d0 0x55ee1f6ca6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:54.525 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.523+0000 7fa379800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee1f6ca2d0 0x55ee1f6ca6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53088/0 (socket says 172.21.15.86:53088) 2024-03-23T17:50:54.525 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.523+0000 7fa379800700 1 -- 172.21.15.86:0/2068486442 learned_addr learned my addr 172.21.15.86:0/2068486442 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:54.525 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.524+0000 7fa378e00700 1 --2- 172.21.15.86:0/2068486442 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ee1f6cabf0 0x55ee1f6d5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:54.525 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.524+0000 7fa37a200700 1 --2- 172.21.15.86:0/2068486442 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ee1f6d5ab0 0x55ee1f6d7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:54.525 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.524+0000 7fa379800700 1 -- 172.21.15.86:0/2068486442 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ee1f6d5ab0 msgr2=0x55ee1f6d7ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:50:54.525 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.524+0000 7fa379800700 1 --2- 172.21.15.86:0/2068486442 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ee1f6d5ab0 0x55ee1f6d7ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:54.525 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.524+0000 7fa379800700 1 -- 172.21.15.86:0/2068486442 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ee1f6cabf0 msgr2=0x55ee1f6d5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:50:54.525 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.524+0000 7fa379800700 1 --2- 172.21.15.86:0/2068486442 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ee1f6cabf0 0x55ee1f6d5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:54.525 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.524+0000 7fa379800700 1 -- 172.21.15.86:0/2068486442 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee1f49e350 con 0x55ee1f6ca2d0 2024-03-23T17:50:54.525 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.524+0000 7fa379800700 1 --2- 172.21.15.86:0/2068486442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee1f6ca2d0 0x55ee1f6ca6b0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa368009fd0 tx=0x7fa36809aa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35ff40bcb9ea2852 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:54.527 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.525+0000 7fa366a00700 1 -- 172.21.15.86:0/2068486442 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa36809b2c0 con 0x55ee1f6ca2d0 2024-03-23T17:50:54.527 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.525+0000 7fa366a00700 1 -- 172.21.15.86:0/2068486442 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa36809abc0 con 0x55ee1f6ca2d0 2024-03-23T17:50:54.527 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.525+0000 7fa366a00700 1 -- 172.21.15.86:0/2068486442 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa36809b080 con 0x55ee1f6ca2d0 2024-03-23T17:50:54.527 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.525+0000 7fa38fcddb80 1 -- 172.21.15.86:0/2068486442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee1f6ca2d0 msgr2=0x55ee1f6ca6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:54.527 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.525+0000 7fa38fcddb80 1 --2- 172.21.15.86:0/2068486442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee1f6ca2d0 0x55ee1f6ca6b0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa368009fd0 tx=0x7fa36809aa80 comp rx=0 tx=0).stop 2024-03-23T17:50:54.527 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.525+0000 7fa38fcddb80 1 -- 172.21.15.86:0/2068486442 shutdown_connections 2024-03-23T17:50:54.527 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.525+0000 7fa38fcddb80 1 --2- 172.21.15.86:0/2068486442 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ee1f6d5ab0 0x55ee1f6d7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:54.527 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.525+0000 7fa38fcddb80 1 --2- 172.21.15.86:0/2068486442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee1f6ca2d0 0x55ee1f6ca6b0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:54.527 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.525+0000 7fa38fcddb80 1 --2- 172.21.15.86:0/2068486442 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ee1f6cabf0 0x55ee1f6d5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:54.527 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.525+0000 7fa38fcddb80 1 -- 172.21.15.86:0/2068486442 >> 172.21.15.86:0/2068486442 conn(0x55ee1f62a640 msgr2=0x55ee1f6c6750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:54.527 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.526+0000 7fa38fcddb80 1 -- 172.21.15.86:0/2068486442 shutdown_connections 2024-03-23T17:50:54.527 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.526+0000 7fa38fcddb80 1 -- 172.21.15.86:0/2068486442 wait complete. 2024-03-23T17:50:54.527 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.526+0000 7fa38fcddb80 1 -- start start 2024-03-23T17:50:54.528 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.527+0000 7fa38fcddb80 1 Processor -- start 2024-03-23T17:50:54.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.527+0000 7fa38fcddb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ee1f6ca2d0 0x55ee1f6d5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:54.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.527+0000 7fa38fcddb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ee1f6cabf0 0x55ee1f6d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:54.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.527+0000 7fa38fcddb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee1f6d5ab0 0x55ee1f6d2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:54.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.527+0000 7fa38fcddb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee1f5b8d20 con 0x55ee1f6d5ab0 2024-03-23T17:50:54.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.527+0000 7fa38fcddb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee1f5b89c0 con 0x55ee1f6cabf0 2024-03-23T17:50:54.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.527+0000 7fa38fcddb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee1f4a0640 con 0x55ee1f6ca2d0 2024-03-23T17:50:54.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.528+0000 7fa38fcddb80 10 client.0 _ll_register_callbacks cb 0x55ee1f5c2410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:50:54.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.528+0000 7fa37a200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee1f6d5ab0 0x55ee1f6d2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:54.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.528+0000 7fa378e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ee1f6cabf0 0x55ee1f6d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:54.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.528+0000 7fa37a200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee1f6d5ab0 0x55ee1f6d2cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53094/0 (socket says 172.21.15.86:53094) 2024-03-23T17:50:54.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.528+0000 7fa37a200700 1 -- 172.21.15.86:0/3153424244 learned_addr learned my addr 172.21.15.86:0/3153424244 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:54.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.528+0000 7fa378e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ee1f6cabf0 0x55ee1f6d2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:60082/0 (socket says 172.21.15.86:60082) 2024-03-23T17:50:54.533 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.528+0000 7fa379800700 1 --2- 172.21.15.86:0/3153424244 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ee1f6ca2d0 0x55ee1f6d5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:54.533 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.528+0000 7fa378e00700 1 -- 172.21.15.86:0/3153424244 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ee1f6ca2d0 msgr2=0x55ee1f6d5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:50:54.533 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.528+0000 7fa378e00700 1 --2- 172.21.15.86:0/3153424244 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ee1f6ca2d0 0x55ee1f6d5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:54.533 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.528+0000 7fa378e00700 1 -- 172.21.15.86:0/3153424244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee1f6d5ab0 msgr2=0x55ee1f6d2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:50:54.533 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.528+0000 7fa378e00700 1 --2- 172.21.15.86:0/3153424244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee1f6d5ab0 0x55ee1f6d2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:54.533 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.528+0000 7fa378e00700 1 -- 172.21.15.86:0/3153424244 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3741a3d40 con 0x55ee1f6cabf0 2024-03-23T17:50:54.533 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.528+0000 7fa379800700 1 --2- 172.21.15.86:0/3153424244 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ee1f6ca2d0 0x55ee1f6d5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:50:54.533 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.528+0000 7fa37a200700 1 --2- 172.21.15.86:0/3153424244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee1f6d5ab0 0x55ee1f6d2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:50:54.533 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.528+0000 7fa378e00700 1 --2- 172.21.15.86:0/3153424244 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ee1f6cabf0 0x55ee1f6d2730 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x55ee1f6c7600 tx=0x7fa3600047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9127a053bc81b4b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:54.533 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.528+0000 7fa35b400700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T17:50:54.533 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.529+0000 7fa35b400700 1 -- 172.21.15.86:0/3153424244 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa360022070 con 0x55ee1f6cabf0 2024-03-23T17:50:54.533 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.529+0000 7fa38fcddb80 1 -- 172.21.15.86:0/3153424244 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ee1f5dd320 con 0x55ee1f6cabf0 2024-03-23T17:50:54.533 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.529+0000 7fa35b400700 1 -- 172.21.15.86:0/3153424244 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fa360003d40 con 0x55ee1f6cabf0 2024-03-23T17:50:54.534 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.529+0000 7fa35b400700 1 -- 172.21.15.86:0/3153424244 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa360027420 con 0x55ee1f6cabf0 2024-03-23T17:50:54.534 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.529+0000 7fa38fcddb80 10 client.25505 fetch_fsmap learned FSMap version 28 2024-03-23T17:50:54.534 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.529+0000 7fa38fcddb80 1 -- 172.21.15.86:0/3153424244 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55ee1f44ab00 con 0x55ee1f6cabf0 2024-03-23T17:50:54.534 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.529+0000 7fa35b400700 1 -- 172.21.15.86:0/3153424244 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ee1f5dd320 con 0x55ee1f6cabf0 2024-03-23T17:50:54.534 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.529+0000 7fa35b400700 1 -- 172.21.15.86:0/3153424244 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 28) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa36001d810 con 0x55ee1f6cabf0 2024-03-23T17:50:54.534 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.529+0000 7fa38fcddb80 10 client.25505 fetch_fsmap finished waiting for FSMap version 28 2024-03-23T17:50:54.534 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.529+0000 7fa38fcddb80 10 client.25505 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:50:54.534 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.530+0000 7fa38fcddb80 20 client.25505 populate_metadata read hostname 'smithi086' 2024-03-23T17:50:54.534 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.530+0000 7fa38fcddb80 1 --2- 172.21.15.86:0/3153424244 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55ee1f6e5400 0x55ee1f6e7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:54.534 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.530+0000 7fa38fcddb80 4 client.25505 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:50:54.534 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.530+0000 7fa38fcddb80 1 -- 172.21.15.86:0/3153424244 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ee1f6156a0 con 0x55ee1f6e5400 2024-03-23T17:50:54.534 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.530+0000 7fa379800700 1 --2- 172.21.15.86:0/3153424244 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55ee1f6e5400 0x55ee1f6e7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:54.534 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.530+0000 7fa379800700 1 --2- 172.21.15.86:0/3153424244 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55ee1f6e5400 0x55ee1f6e7820 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4d31ad6abc43bc3 server_cookie=90929fa152142906 in_seq=0 out_seq=0 2024-03-23T17:50:54.535 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.530+0000 7fa35b400700 10 client.25505 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:50:54.535 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.531+0000 7fa35b400700 1 -- 172.21.15.86:0/3153424244 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55ee1f6156a0 con 0x55ee1f6e5400 2024-03-23T17:50:54.535 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.531+0000 7fa35b400700 10 client.25505 handle_command_reply: tid=0 2024-03-23T17:50:54.535 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.531+0000 7fa38fcddb80 1 client.25505 shutdown 2024-03-23T17:50:54.535 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.531+0000 7fa38fcddb80 1 -- 172.21.15.86:0/3153424244 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ee1f6cabf0 msgr2=0x55ee1f6d2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:54.535 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.531+0000 7fa38fcddb80 1 --2- 172.21.15.86:0/3153424244 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ee1f6cabf0 0x55ee1f6d2730 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x55ee1f6c7600 tx=0x7fa3600047c0 comp rx=0 tx=0).stop 2024-03-23T17:50:54.535 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.531+0000 7fa38fcddb80 1 -- 172.21.15.86:0/3153424244 shutdown_connections 2024-03-23T17:50:54.535 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.531+0000 7fa38fcddb80 1 --2- 172.21.15.86:0/3153424244 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55ee1f6e5400 0x55ee1f6e7820 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:54.535 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.531+0000 7fa38fcddb80 1 --2- 172.21.15.86:0/3153424244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee1f6d5ab0 0x55ee1f6d2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:54.535 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.531+0000 7fa38fcddb80 1 --2- 172.21.15.86:0/3153424244 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ee1f6ca2d0 0x55ee1f6d5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:54.535 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.531+0000 7fa38fcddb80 1 --2- 172.21.15.86:0/3153424244 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ee1f6cabf0 0x55ee1f6d2730 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:54.535 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.531+0000 7fa35b400700 0 client.25505 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:50:54.535 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.531+0000 7fa38fcddb80 1 -- 172.21.15.86:0/3153424244 >> 172.21.15.86:0/3153424244 conn(0x55ee1f62a640 msgr2=0x55ee1f6cc390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:54.535 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.532+0000 7fa38fcddb80 1 -- 172.21.15.86:0/3153424244 shutdown_connections 2024-03-23T17:50:54.536 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.532+0000 7fa38fcddb80 1 -- 172.21.15.86:0/3153424244 wait complete. 2024-03-23T17:50:54.536 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.532+0000 7fa38fcddb80 20 client.25505 trim_cache size 0 max 16384 2024-03-23T17:50:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.547+0000 7fa38fcddb80 1 Processor -- start 2024-03-23T17:50:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.547+0000 7fa38fcddb80 1 -- start start 2024-03-23T17:50:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.547+0000 7fa38fcddb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ee1f6ca2d0 0x55ee1f6dc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.547+0000 7fa38fcddb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ee1f6cabf0 0x55ee1f6d4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.547+0000 7fa38fcddb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee1f6d5ab0 0x55ee1f6d26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.547+0000 7fa38fcddb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee1f6cb560 con 0x55ee1f6d5ab0 2024-03-23T17:50:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.547+0000 7fa38fcddb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee1f6d2520 con 0x55ee1f6ca2d0 2024-03-23T17:50:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.547+0000 7fa38fcddb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee1f6ca110 con 0x55ee1f6cabf0 2024-03-23T17:50:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.548+0000 7fa379800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ee1f6ca2d0 0x55ee1f6dc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.548+0000 7fa37a200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee1f6d5ab0 0x55ee1f6d26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.548+0000 7fa379800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ee1f6ca2d0 0x55ee1f6dc4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:60086/0 (socket says 172.21.15.86:60086) 2024-03-23T17:50:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.548+0000 7fa379800700 1 -- 172.21.15.86:0/3018711145 learned_addr learned my addr 172.21.15.86:0/3018711145 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.548+0000 7fa378e00700 1 --2- 172.21.15.86:0/3018711145 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ee1f6cabf0 0x55ee1f6d4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.548+0000 7fa379800700 1 -- 172.21.15.86:0/3018711145 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ee1f6cabf0 msgr2=0x55ee1f6d4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:50:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.548+0000 7fa379800700 1 --2- 172.21.15.86:0/3018711145 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ee1f6cabf0 0x55ee1f6d4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.548+0000 7fa379800700 1 -- 172.21.15.86:0/3018711145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee1f6d5ab0 msgr2=0x55ee1f6d26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:50:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.548+0000 7fa379800700 1 --2- 172.21.15.86:0/3018711145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee1f6d5ab0 0x55ee1f6d26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.548+0000 7fa379800700 1 -- 172.21.15.86:0/3018711145 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee1f5e2e40 con 0x55ee1f6ca2d0 2024-03-23T17:50:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.548+0000 7fa379800700 1 --2- 172.21.15.86:0/3018711145 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ee1f6ca2d0 0x55ee1f6dc4a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa3680cb300 tx=0x7fa3680226e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5010d2cdc0e9cdd server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.549+0000 7fa367400700 1 -- 172.21.15.86:0/3018711145 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa36809a990 con 0x55ee1f6ca2d0 2024-03-23T17:50:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.549+0000 7fa367400700 1 -- 172.21.15.86:0/3018711145 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa36809ea70 con 0x55ee1f6ca2d0 2024-03-23T17:50:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.549+0000 7fa367400700 1 -- 172.21.15.86:0/3018711145 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa36809bbd0 con 0x55ee1f6ca2d0 2024-03-23T17:50:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.550+0000 7fa38fcddb80 1 -- 172.21.15.86:0/3018711145 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ee1f6ca2d0 msgr2=0x55ee1f6dc4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.550+0000 7fa38fcddb80 1 --2- 172.21.15.86:0/3018711145 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ee1f6ca2d0 0x55ee1f6dc4a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa3680cb300 tx=0x7fa3680226e0 comp rx=0 tx=0).stop 2024-03-23T17:50:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.550+0000 7fa38fcddb80 1 -- 172.21.15.86:0/3018711145 shutdown_connections 2024-03-23T17:50:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.550+0000 7fa38fcddb80 1 --2- 172.21.15.86:0/3018711145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee1f6d5ab0 0x55ee1f6d26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.550+0000 7fa38fcddb80 1 --2- 172.21.15.86:0/3018711145 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ee1f6ca2d0 0x55ee1f6dc4a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.550+0000 7fa38fcddb80 1 --2- 172.21.15.86:0/3018711145 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ee1f6cabf0 0x55ee1f6d4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.550+0000 7fa38fcddb80 1 -- 172.21.15.86:0/3018711145 >> 172.21.15.86:0/3018711145 conn(0x55ee1f62a640 msgr2=0x55ee1f6bfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.550+0000 7fa38fcddb80 1 -- 172.21.15.86:0/3018711145 shutdown_connections 2024-03-23T17:50:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.550+0000 7fa38fcddb80 1 -- 172.21.15.86:0/3018711145 wait complete. 2024-03-23T17:50:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.550+0000 7fa38fcddb80 1 -- start start 2024-03-23T17:50:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.551+0000 7fa38fcddb80 1 Processor -- start 2024-03-23T17:50:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.551+0000 7fa38fcddb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ee1f6ca2d0 0x55ee1f72eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.551+0000 7fa38fcddb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee1f6cabf0 0x55ee1f72f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.551+0000 7fa38fcddb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ee1f6d5ab0 0x55ee1f72f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.551+0000 7fa38fcddb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee1f6db4e0 con 0x55ee1f6cabf0 2024-03-23T17:50:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.551+0000 7fa38fcddb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee1f6db650 con 0x55ee1f6d5ab0 2024-03-23T17:50:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.551+0000 7fa38fcddb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee1f6db810 con 0x55ee1f6ca2d0 2024-03-23T17:50:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.551+0000 7fa38fcddb80 10 client.0 _ll_register_callbacks cb 0x55ee1f5c2410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:50:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.552+0000 7fa378e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee1f6cabf0 0x55ee1f72f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.552+0000 7fa378e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee1f6cabf0 0x55ee1f72f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53106/0 (socket says 172.21.15.86:53106) 2024-03-23T17:50:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.552+0000 7fa379800700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ee1f6ca2d0 0x55ee1f72eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.552+0000 7fa378e00700 1 -- 172.21.15.86:0/138827736 learned_addr learned my addr 172.21.15.86:0/138827736 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.552+0000 7fa37a200700 1 --2- 172.21.15.86:0/138827736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ee1f6d5ab0 0x55ee1f72f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:54.555 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.552+0000 7fa379800700 1 -- 172.21.15.86:0/138827736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ee1f6d5ab0 msgr2=0x55ee1f72f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:50:54.555 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.552+0000 7fa379800700 1 --2- 172.21.15.86:0/138827736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ee1f6d5ab0 0x55ee1f72f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:54.555 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.552+0000 7fa379800700 1 -- 172.21.15.86:0/138827736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee1f6cabf0 msgr2=0x55ee1f72f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:50:54.555 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.552+0000 7fa379800700 1 --2- 172.21.15.86:0/138827736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee1f6cabf0 0x55ee1f72f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.552+0000 7fa379800700 1 -- 172.21.15.86:0/138827736 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ee1f49e350 con 0x55ee1f6ca2d0 2024-03-23T17:50:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.552+0000 7fa378e00700 1 --2- 172.21.15.86:0/138827736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee1f6cabf0 0x55ee1f72f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:50:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.552+0000 7fa379800700 1 --2- 172.21.15.86:0/138827736 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ee1f6ca2d0 0x55ee1f72eed0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa368009d70 tx=0x7fa36809a990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60d5f4957a029da4 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.552+0000 7fa35b400700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-03-23T17:50:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.552+0000 7fa35b400700 1 -- 172.21.15.86:0/138827736 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3680a2fb0 con 0x55ee1f6ca2d0 2024-03-23T17:50:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.553+0000 7fa38fcddb80 1 -- 172.21.15.86:0/138827736 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa36809abc0 con 0x55ee1f6ca2d0 2024-03-23T17:50:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.553+0000 7fa35b400700 1 -- 172.21.15.86:0/138827736 <== mon.1 v2:172.21.15.152:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fa36809b2c0 con 0x55ee1f6ca2d0 2024-03-23T17:50:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.553+0000 7fa35b400700 1 -- 172.21.15.86:0/138827736 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa36809ae20 con 0x55ee1f6ca2d0 2024-03-23T17:50:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.553+0000 7fa38fcddb80 10 client.25510 fetch_fsmap learned FSMap version 28 2024-03-23T17:50:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.553+0000 7fa38fcddb80 1 -- 172.21.15.86:0/138827736 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55ee1f4a2c70 con 0x55ee1f6ca2d0 2024-03-23T17:50:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.554+0000 7fa35b400700 1 -- 172.21.15.86:0/138827736 <== mon.1 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa36807d080 con 0x55ee1f6ca2d0 2024-03-23T17:50:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.554+0000 7fa35b400700 1 -- 172.21.15.86:0/138827736 <== mon.1 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa3680662b0 con 0x55ee1f6ca2d0 2024-03-23T17:50:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.554+0000 7fa38fcddb80 10 client.25510 fetch_fsmap finished waiting for FSMap version 28 2024-03-23T17:50:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.554+0000 7fa38fcddb80 10 client.25510 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:50:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.554+0000 7fa38fcddb80 20 client.25510 populate_metadata read hostname 'smithi086' 2024-03-23T17:50:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.554+0000 7fa38fcddb80 1 --2- 172.21.15.86:0/138827736 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55ee1f740190 0x55ee1f742580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.554+0000 7fa38fcddb80 4 client.25510 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T17:50:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.554+0000 7fa38fcddb80 1 -- 172.21.15.86:0/138827736 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ee1f46bc90 con 0x55ee1f740190 2024-03-23T17:50:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.554+0000 7fa378e00700 1 --2- 172.21.15.86:0/138827736 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55ee1f740190 0x55ee1f742580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.555+0000 7fa378e00700 1 --2- 172.21.15.86:0/138827736 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55ee1f740190 0x55ee1f742580 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e59b3752cc36140 server_cookie=fbbc9b6c48114738 in_seq=0 out_seq=0 2024-03-23T17:50:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.555+0000 7fa35b400700 10 client.25510 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:50:54.566 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.566+0000 7fa35b400700 1 -- 172.21.15.86:0/138827736 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ee1f46bc90 con 0x55ee1f740190 2024-03-23T17:50:54.566 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.566+0000 7fa35b400700 10 client.25510 handle_command_reply: tid=0 2024-03-23T17:50:54.566 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.566+0000 7fa38fcddb80 1 client.25510 shutdown 2024-03-23T17:50:54.567 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.566+0000 7fa38fcddb80 1 -- 172.21.15.86:0/138827736 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ee1f6ca2d0 msgr2=0x55ee1f72eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:54.567 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.566+0000 7fa38fcddb80 1 --2- 172.21.15.86:0/138827736 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ee1f6ca2d0 0x55ee1f72eed0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa368009d70 tx=0x7fa36809a990 comp rx=0 tx=0).stop 2024-03-23T17:50:54.567 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.566+0000 7fa38fcddb80 1 -- 172.21.15.86:0/138827736 shutdown_connections 2024-03-23T17:50:54.567 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.566+0000 7fa38fcddb80 1 --2- 172.21.15.86:0/138827736 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55ee1f740190 0x55ee1f742580 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:54.567 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.566+0000 7fa38fcddb80 1 --2- 172.21.15.86:0/138827736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ee1f6d5ab0 0x55ee1f72f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:54.567 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.566+0000 7fa38fcddb80 1 --2- 172.21.15.86:0/138827736 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ee1f6ca2d0 0x55ee1f72eed0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:54.567 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.566+0000 7fa35b400700 0 client.25510 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:50:54.567 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.566+0000 7fa38fcddb80 1 --2- 172.21.15.86:0/138827736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee1f6cabf0 0x55ee1f72f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:54.567 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.566+0000 7fa38fcddb80 1 -- 172.21.15.86:0/138827736 >> 172.21.15.86:0/138827736 conn(0x55ee1f62a640 msgr2=0x55ee1f6d26b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:54.567 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.566+0000 7fa38fcddb80 1 -- 172.21.15.86:0/138827736 shutdown_connections 2024-03-23T17:50:54.567 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.566+0000 7fa38fcddb80 1 -- 172.21.15.86:0/138827736 wait complete. 2024-03-23T17:50:54.567 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.566+0000 7fa38fcddb80 20 client.25510 trim_cache size 0 max 16384 2024-03-23T17:50:54.568 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-03-23T17:50:54.568 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-03-23T17:50:54.568 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-03-23T17:50:54.568 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-03-23T17:50:54.569 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.567+0000 7fa37ac00700 1 -- 172.21.15.86:0/1242652386 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fa35c0a1320 msgr2=0x7fa35c0a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:54.569 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.567+0000 7fa37ac00700 1 --2- 172.21.15.86:0/1242652386 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fa35c0a1320 0x7fa35c0a37e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa374198520 tx=0x7fa360009260 comp rx=0 tx=0).stop 2024-03-23T17:50:54.569 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.567+0000 7fa37ac00700 1 -- 172.21.15.86:0/1242652386 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa374068080 msgr2=0x7fa374197020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:54.569 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.567+0000 7fa37ac00700 1 --2- 172.21.15.86:0/1242652386 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa374068080 0x7fa374197020 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa36800b580 tx=0x7fa36800bfd0 comp rx=0 tx=0).stop 2024-03-23T17:50:54.569 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.567+0000 7fa37ac00700 1 -- 172.21.15.86:0/1242652386 shutdown_connections 2024-03-23T17:50:54.569 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.567+0000 7fa37ac00700 1 --2- 172.21.15.86:0/1242652386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa374106b90 0x7fa37419b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:54.569 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.567+0000 7fa37ac00700 1 --2- 172.21.15.86:0/1242652386 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa374068080 0x7fa374197020 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:54.569 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.567+0000 7fa37ac00700 1 --2- 172.21.15.86:0/1242652386 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fa35c0a1320 0x7fa35c0a37e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:54.569 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.567+0000 7fa37ac00700 1 --2- 172.21.15.86:0/1242652386 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa374068a40 0x7fa374197560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:54.570 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.567+0000 7fa37ac00700 1 -- 172.21.15.86:0/1242652386 >> 172.21.15.86:0/1242652386 conn(0x7fa374075610 msgr2=0x7fa3740feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:54.570 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.567+0000 7fa37ac00700 1 -- 172.21.15.86:0/1242652386 shutdown_connections 2024-03-23T17:50:54.570 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:50:54.567+0000 7fa37ac00700 1 -- 172.21.15.86:0/1242652386 wait complete. 2024-03-23T17:50:56.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:55 smithi086 ceph-mon[38221]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-03-23T17:50:56.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:55 smithi092 ceph-mon[79089]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-03-23T17:50:56.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:55 smithi152 ceph-mon[47949]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-03-23T17:50:58.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:57 smithi086 ceph-mon[38221]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 KiB/s wr, 54 op/s 2024-03-23T17:50:58.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:57 smithi092 ceph-mon[79089]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 KiB/s wr, 54 op/s 2024-03-23T17:50:58.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:57 smithi152 ceph-mon[47949]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 KiB/s wr, 54 op/s 2024-03-23T17:50:59.563 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-23T17:50:59.563 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-23T17:50:59.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.773+0000 7f3a17000700 1 -- 172.21.15.86:0/1907106106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a10106090 msgr2=0x7f3a1010dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:59.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.773+0000 7f3a17000700 1 --2- 172.21.15.86:0/1907106106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a10106090 0x7f3a1010dc10 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f3a0800b0a0 tx=0x7f3a0800b3b0 comp rx=0 tx=0).stop 2024-03-23T17:50:59.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.773+0000 7f3a17000700 1 -- 172.21.15.86:0/1907106106 shutdown_connections 2024-03-23T17:50:59.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.773+0000 7f3a17000700 1 --2- 172.21.15.86:0/1907106106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a10106090 0x7f3a1010dc10 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:59.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.773+0000 7f3a17000700 1 --2- 172.21.15.86:0/1907106106 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a10100f70 0x7f3a10101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:59.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.773+0000 7f3a17000700 1 --2- 172.21.15.86:0/1907106106 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a10101920 0x7f3a10105960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:59.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.773+0000 7f3a17000700 1 -- 172.21.15.86:0/1907106106 >> 172.21.15.86:0/1907106106 conn(0x7f3a100fc7f0 msgr2=0x7f3a100fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:59.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.773+0000 7f3a17000700 1 -- 172.21.15.86:0/1907106106 shutdown_connections 2024-03-23T17:50:59.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.773+0000 7f3a17000700 1 -- 172.21.15.86:0/1907106106 wait complete. 2024-03-23T17:50:59.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.774+0000 7f3a17000700 1 Processor -- start 2024-03-23T17:50:59.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.774+0000 7f3a17000700 1 -- start start 2024-03-23T17:50:59.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.775+0000 7f3a17000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a10100f70 0x7f3a10199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:59.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.775+0000 7f3a17000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a10106090 0x7f3a10199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:59.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.775+0000 7f3a17000700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a1019de00 0x7f3a1019e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:59.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.775+0000 7f3a17000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1019e880 con 0x7f3a10100f70 2024-03-23T17:50:59.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.775+0000 7f3a17000700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1019e9f0 con 0x7f3a10106090 2024-03-23T17:50:59.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.775+0000 7f3a17000700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1019ebb0 con 0x7f3a1019de00 2024-03-23T17:50:59.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.775+0000 7f3a0fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a10100f70 0x7f3a10199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:59.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.775+0000 7f3a0fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a10100f70 0x7f3a10199570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42998/0 (socket says 172.21.15.86:42998) 2024-03-23T17:50:59.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.775+0000 7f3a0fe00700 1 -- 172.21.15.86:0/2519072516 learned_addr learned my addr 172.21.15.86:0/2519072516 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:59.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.775+0000 7f3a0f400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a10106090 0x7f3a10199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:59.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.775+0000 7f3a15000700 1 --2- 172.21.15.86:0/2519072516 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a1019de00 0x7f3a1019e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:59.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.775+0000 7f3a0fe00700 1 -- 172.21.15.86:0/2519072516 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a1019de00 msgr2=0x7f3a1019e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:59.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.775+0000 7f3a0fe00700 1 --2- 172.21.15.86:0/2519072516 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a1019de00 0x7f3a1019e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:59.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.775+0000 7f3a0fe00700 1 -- 172.21.15.86:0/2519072516 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a10106090 msgr2=0x7f3a10199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:59.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.775+0000 7f3a0fe00700 1 --2- 172.21.15.86:0/2519072516 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a10106090 0x7f3a10199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:59.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.775+0000 7f3a0fe00700 1 -- 172.21.15.86:0/2519072516 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a08009d00 con 0x7f3a10100f70 2024-03-23T17:50:59.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.775+0000 7f3a0f400700 1 --2- 172.21.15.86:0/2519072516 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a10106090 0x7f3a10199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:50:59.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.776+0000 7f3a0fe00700 1 --2- 172.21.15.86:0/2519072516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a10100f70 0x7f3a10199570 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3a0000e9a0 tx=0x7f3a0000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:59.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.776+0000 7f3a0cc00700 1 -- 172.21.15.86:0/2519072516 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a00009790 con 0x7f3a10100f70 2024-03-23T17:50:59.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.776+0000 7f3a0cc00700 1 -- 172.21.15.86:0/2519072516 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a0000cbd0 con 0x7f3a10100f70 2024-03-23T17:50:59.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.776+0000 7f3a17000700 1 -- 172.21.15.86:0/2519072516 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a101a6110 con 0x7f3a10100f70 2024-03-23T17:50:59.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.776+0000 7f3a0cc00700 1 -- 172.21.15.86:0/2519072516 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a00018680 con 0x7f3a10100f70 2024-03-23T17:50:59.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.776+0000 7f3a17000700 1 -- 172.21.15.86:0/2519072516 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a101a65f0 con 0x7f3a10100f70 2024-03-23T17:50:59.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.779+0000 7f3a0cc00700 1 -- 172.21.15.86:0/2519072516 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) v1 ==== 136041+0+0 (secure 0 0 0) 0x7f3a0000cd40 con 0x7f3a10100f70 2024-03-23T17:50:59.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.780+0000 7f3a0cc00700 1 --2- 172.21.15.86:0/2519072516 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f39f40a1320 0x7f39f40a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:59.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.781+0000 7f3a0cc00700 1 -- 172.21.15.86:0/2519072516 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f3a00014070 con 0x7f3a10100f70 2024-03-23T17:50:59.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.781+0000 7f3a2ab9cb80 1 Processor -- start 2024-03-23T17:50:59.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.781+0000 7f3a0f400700 1 --2- 172.21.15.86:0/2519072516 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f39f40a1320 0x7f39f40a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:59.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.781+0000 7f3a2ab9cb80 1 -- start start 2024-03-23T17:50:59.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.782+0000 7f3a0f400700 1 --2- 172.21.15.86:0/2519072516 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f39f40a1320 0x7f39f40a37e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3a1019aa70 tx=0x7f39fc009260 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:59.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.782+0000 7f3a2ab9cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561083a405d0 0x561083a409b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:59.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.782+0000 7f3a2ab9cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561083a40ef0 0x561083a4b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:59.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.782+0000 7f3a2ab9cb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561083a4bdf0 0x561083a4e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:59.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.782+0000 7f3a2ab9cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x561083999990 con 0x561083a405d0 2024-03-23T17:50:59.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.782+0000 7f3a2ab9cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x561083997350 con 0x561083a40ef0 2024-03-23T17:50:59.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.782+0000 7f3a2ab9cb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x561083992c40 con 0x561083a4bdf0 2024-03-23T17:50:59.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.782+0000 7f3a0fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561083a405d0 0x561083a409b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:59.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.782+0000 7f3a0fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561083a405d0 0x561083a409b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43014/0 (socket says 172.21.15.86:43014) 2024-03-23T17:50:59.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.782+0000 7f3a0fe00700 1 -- 172.21.15.86:0/955349422 learned_addr learned my addr 172.21.15.86:0/955349422 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:59.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.782+0000 7f3a0f400700 1 --2- 172.21.15.86:0/955349422 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561083a40ef0 0x561083a4b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:59.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.782+0000 7f3a15000700 1 --2- 172.21.15.86:0/955349422 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561083a4bdf0 0x561083a4e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:59.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.783+0000 7f3a0f400700 1 -- 172.21.15.86:0/955349422 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561083a4bdf0 msgr2=0x561083a4e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:50:59.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.783+0000 7f3a0f400700 1 --2- 172.21.15.86:0/955349422 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561083a4bdf0 0x561083a4e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:59.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.783+0000 7f3a0f400700 1 -- 172.21.15.86:0/955349422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561083a405d0 msgr2=0x561083a409b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:50:59.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.783+0000 7f3a0f400700 1 --2- 172.21.15.86:0/955349422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561083a405d0 0x561083a409b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:59.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.783+0000 7f3a0f400700 1 -- 172.21.15.86:0/955349422 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5610838143b0 con 0x561083a40ef0 2024-03-23T17:50:59.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.783+0000 7f3a0fe00700 1 --2- 172.21.15.86:0/955349422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561083a405d0 0x561083a409b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:50:59.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.783+0000 7f3a15000700 1 --2- 172.21.15.86:0/955349422 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561083a4bdf0 0x561083a4e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:50:59.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.783+0000 7f3a0f400700 1 --2- 172.21.15.86:0/955349422 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561083a40ef0 0x561083a4b8b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f39fc00e630 tx=0x7f39fc014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d17b0040feb2bd27 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:59.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.783+0000 7f3a05600700 1 -- 172.21.15.86:0/955349422 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39fc004230 con 0x561083a40ef0 2024-03-23T17:50:59.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.783+0000 7f3a05600700 1 -- 172.21.15.86:0/955349422 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39fc00ee90 con 0x561083a40ef0 2024-03-23T17:50:59.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.783+0000 7f3a05600700 1 -- 172.21.15.86:0/955349422 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39fc01e650 con 0x561083a40ef0 2024-03-23T17:50:59.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.784+0000 7f3a2ab9cb80 1 -- 172.21.15.86:0/955349422 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561083a40ef0 msgr2=0x561083a4b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:59.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.784+0000 7f3a2ab9cb80 1 --2- 172.21.15.86:0/955349422 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561083a40ef0 0x561083a4b8b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f39fc00e630 tx=0x7f39fc014000 comp rx=0 tx=0).stop 2024-03-23T17:50:59.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.784+0000 7f3a2ab9cb80 1 -- 172.21.15.86:0/955349422 shutdown_connections 2024-03-23T17:50:59.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.784+0000 7f3a2ab9cb80 1 --2- 172.21.15.86:0/955349422 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561083a4bdf0 0x561083a4e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:59.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.784+0000 7f3a2ab9cb80 1 --2- 172.21.15.86:0/955349422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561083a405d0 0x561083a409b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:59.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.784+0000 7f3a2ab9cb80 1 --2- 172.21.15.86:0/955349422 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561083a40ef0 0x561083a4b8b0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:59.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.784+0000 7f3a2ab9cb80 1 -- 172.21.15.86:0/955349422 >> 172.21.15.86:0/955349422 conn(0x5610839a06a0 msgr2=0x561083a3ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:59.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.784+0000 7f3a2ab9cb80 1 -- 172.21.15.86:0/955349422 shutdown_connections 2024-03-23T17:50:59.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.784+0000 7f3a2ab9cb80 1 -- 172.21.15.86:0/955349422 wait complete. 2024-03-23T17:50:59.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.785+0000 7f3a2ab9cb80 1 -- start start 2024-03-23T17:50:59.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.785+0000 7f3a2ab9cb80 1 Processor -- start 2024-03-23T17:50:59.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.786+0000 7f3a2ab9cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561083a405d0 0x561083a4b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:59.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.786+0000 7f3a2ab9cb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561083a40ef0 0x561083a48b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:59.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.786+0000 7f3a2ab9cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561083a4bdf0 0x561083a492d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:59.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.786+0000 7f3a2ab9cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56108392ed80 con 0x561083a405d0 2024-03-23T17:50:59.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.786+0000 7f3a2ab9cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x56108392ea20 con 0x561083a4bdf0 2024-03-23T17:50:59.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.786+0000 7f3a2ab9cb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610838166a0 con 0x561083a40ef0 2024-03-23T17:50:59.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.786+0000 7f3a2ab9cb80 10 client.0 _ll_register_callbacks cb 0x561083938470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:50:59.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.786+0000 7f3a0fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561083a405d0 0x561083a4b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:59.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.786+0000 7f3a15000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561083a4bdf0 0x561083a492d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:59.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.786+0000 7f3a0fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561083a405d0 0x561083a4b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43018/0 (socket says 172.21.15.86:43018) 2024-03-23T17:50:59.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.786+0000 7f3a0fe00700 1 -- 172.21.15.86:0/2235792815 learned_addr learned my addr 172.21.15.86:0/2235792815 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:59.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.786+0000 7f3a0f400700 1 --2- 172.21.15.86:0/2235792815 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561083a40ef0 0x561083a48b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:59.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.786+0000 7f3a0fe00700 1 -- 172.21.15.86:0/2235792815 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561083a40ef0 msgr2=0x561083a48b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:50:59.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.786+0000 7f3a0fe00700 1 --2- 172.21.15.86:0/2235792815 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561083a40ef0 0x561083a48b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:59.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.786+0000 7f3a0fe00700 1 -- 172.21.15.86:0/2235792815 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561083a4bdf0 msgr2=0x561083a492d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:50:59.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.786+0000 7f3a0fe00700 1 --2- 172.21.15.86:0/2235792815 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561083a4bdf0 0x561083a492d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:59.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.787+0000 7f3a0fe00700 1 -- 172.21.15.86:0/2235792815 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a1010e480 con 0x561083a405d0 2024-03-23T17:50:59.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.787+0000 7f3a15000700 1 --2- 172.21.15.86:0/2235792815 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561083a4bdf0 0x561083a492d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:50:59.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.787+0000 7f3a0fe00700 1 --2- 172.21.15.86:0/2235792815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561083a405d0 0x561083a4b850 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3a00095e00 tx=0x7f3a000cad00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a415bd108e2e72f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:59.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.787+0000 7f39fa000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-23T17:50:59.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.787+0000 7f39fa000700 1 -- 172.21.15.86:0/2235792815 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a0009a780 con 0x561083a405d0 2024-03-23T17:50:59.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.787+0000 7f39fa000700 1 -- 172.21.15.86:0/2235792815 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f3a0009a080 con 0x561083a405d0 2024-03-23T17:50:59.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.787+0000 7f3a2ab9cb80 1 -- 172.21.15.86:0/2235792815 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561083953380 con 0x561083a405d0 2024-03-23T17:50:59.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.787+0000 7f39fa000700 1 -- 172.21.15.86:0/2235792815 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a0009deb0 con 0x561083a405d0 2024-03-23T17:50:59.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.787+0000 7f3a2ab9cb80 10 client.15477 fetch_fsmap learned FSMap version 28 2024-03-23T17:50:59.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.787+0000 7f3a2ab9cb80 1 -- 172.21.15.86:0/2235792815 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f39fc017070 con 0x561083a405d0 2024-03-23T17:50:59.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.787+0000 7f39fa000700 1 -- 172.21.15.86:0/2235792815 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a00063070 con 0x561083a405d0 2024-03-23T17:50:59.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.788+0000 7f39fa000700 1 -- 172.21.15.86:0/2235792815 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3a00098fe0 con 0x561083a405d0 2024-03-23T17:50:59.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.788+0000 7f3a2ab9cb80 10 client.15477 fetch_fsmap finished waiting for FSMap version 28 2024-03-23T17:50:59.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.788+0000 7f3a2ab9cb80 10 client.15477 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T17:50:59.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.788+0000 7f3a2ab9cb80 20 client.15477 populate_metadata read hostname 'smithi086' 2024-03-23T17:50:59.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.788+0000 7f3a2ab9cb80 1 --2- 172.21.15.86:0/2235792815 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x561083a5ba40 0x561083a5de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:59.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.788+0000 7f3a2ab9cb80 4 client.15477 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:50:59.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.788+0000 7f3a2ab9cb80 1 -- 172.21.15.86:0/2235792815 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56108398b700 con 0x561083a5ba40 2024-03-23T17:50:59.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.788+0000 7f3a0f400700 1 --2- 172.21.15.86:0/2235792815 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x561083a5ba40 0x561083a5de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:59.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.789+0000 7f39fa000700 10 client.15477 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:50:59.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.789+0000 7f3a0f400700 1 --2- 172.21.15.86:0/2235792815 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x561083a5ba40 0x561083a5de60 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdd5f28adaf80d18 server_cookie=f8050bc7c680905b in_seq=0 out_seq=0 2024-03-23T17:50:59.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.789+0000 7f39fa000700 1 -- 172.21.15.86:0/2235792815 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x56108398b700 con 0x561083a5ba40 2024-03-23T17:50:59.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.789+0000 7f39fa000700 10 client.15477 handle_command_reply: tid=0 2024-03-23T17:50:59.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.789+0000 7f3a2ab9cb80 1 client.15477 shutdown 2024-03-23T17:50:59.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.789+0000 7f3a2ab9cb80 1 -- 172.21.15.86:0/2235792815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561083a405d0 msgr2=0x561083a4b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:59.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.789+0000 7f3a2ab9cb80 1 --2- 172.21.15.86:0/2235792815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561083a405d0 0x561083a4b850 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3a00095e00 tx=0x7f3a000cad00 comp rx=0 tx=0).stop 2024-03-23T17:50:59.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.790+0000 7f3a2ab9cb80 1 -- 172.21.15.86:0/2235792815 shutdown_connections 2024-03-23T17:50:59.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.790+0000 7f3a2ab9cb80 1 --2- 172.21.15.86:0/2235792815 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x561083a5ba40 0x561083a5de60 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:59.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.790+0000 7f3a2ab9cb80 1 --2- 172.21.15.86:0/2235792815 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561083a4bdf0 0x561083a492d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:59.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.790+0000 7f3a2ab9cb80 1 --2- 172.21.15.86:0/2235792815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561083a405d0 0x561083a4b850 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:59.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.790+0000 7f3a2ab9cb80 1 --2- 172.21.15.86:0/2235792815 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561083a40ef0 0x561083a48b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:59.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.790+0000 7f39fa000700 0 client.15477 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:50:59.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.790+0000 7f3a2ab9cb80 1 -- 172.21.15.86:0/2235792815 >> 172.21.15.86:0/2235792815 conn(0x5610839a06a0 msgr2=0x561083a42530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:59.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.790+0000 7f3a2ab9cb80 1 -- 172.21.15.86:0/2235792815 shutdown_connections 2024-03-23T17:50:59.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.790+0000 7f3a2ab9cb80 1 -- 172.21.15.86:0/2235792815 wait complete. 2024-03-23T17:50:59.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.790+0000 7f3a2ab9cb80 20 client.15477 trim_cache size 0 max 16384 2024-03-23T17:50:59.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.807+0000 7f3a2ab9cb80 1 Processor -- start 2024-03-23T17:50:59.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.807+0000 7f3a2ab9cb80 1 -- start start 2024-03-23T17:50:59.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.807+0000 7f3a2ab9cb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561083a405d0 0x561083a49270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:59.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.807+0000 7f3a2ab9cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561083a40ef0 0x561083a52a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:59.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.807+0000 7f3a2ab9cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561083a4bdf0 0x561083a48420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:59.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.807+0000 7f3a2ab9cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x561083a578f0 con 0x561083a40ef0 2024-03-23T17:50:59.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.807+0000 7f3a2ab9cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x561083a57e10 con 0x561083a4bdf0 2024-03-23T17:50:59.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.807+0000 7f3a2ab9cb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x561083a583f0 con 0x561083a405d0 2024-03-23T17:50:59.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.808+0000 7f3a0f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561083a40ef0 0x561083a52a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:59.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.808+0000 7f3a0f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561083a40ef0 0x561083a52a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43028/0 (socket says 172.21.15.86:43028) 2024-03-23T17:50:59.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.808+0000 7f3a0f400700 1 -- 172.21.15.86:0/11457881 learned_addr learned my addr 172.21.15.86:0/11457881 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:59.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.808+0000 7f3a0fe00700 1 --2- 172.21.15.86:0/11457881 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561083a405d0 0x561083a49270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:59.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.808+0000 7f3a15000700 1 --2- 172.21.15.86:0/11457881 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561083a4bdf0 0x561083a48420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:59.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.808+0000 7f3a0f400700 1 -- 172.21.15.86:0/11457881 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561083a405d0 msgr2=0x561083a49270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:50:59.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.808+0000 7f3a0f400700 1 --2- 172.21.15.86:0/11457881 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561083a405d0 0x561083a49270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:59.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.808+0000 7f3a0f400700 1 -- 172.21.15.86:0/11457881 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561083a4bdf0 msgr2=0x561083a48420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:50:59.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.808+0000 7f3a0f400700 1 --2- 172.21.15.86:0/11457881 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561083a4bdf0 0x561083a48420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:59.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.808+0000 7f3a0f400700 1 -- 172.21.15.86:0/11457881 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5610837c0b60 con 0x561083a40ef0 2024-03-23T17:50:59.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.808+0000 7f3a0f400700 1 --2- 172.21.15.86:0/11457881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561083a40ef0 0x561083a52a70 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f39fc0097c0 tx=0x7f39fc013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7444996eb659e698 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:59.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.808+0000 7f3a06000700 1 -- 172.21.15.86:0/11457881 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39fc003ac0 con 0x561083a40ef0 2024-03-23T17:50:59.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.809+0000 7f3a06000700 1 -- 172.21.15.86:0/11457881 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39fc02e040 con 0x561083a40ef0 2024-03-23T17:50:59.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.809+0000 7f3a06000700 1 -- 172.21.15.86:0/11457881 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39fc033af0 con 0x561083a40ef0 2024-03-23T17:50:59.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.809+0000 7f3a2ab9cb80 1 -- 172.21.15.86:0/11457881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561083a40ef0 msgr2=0x561083a52a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:59.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.809+0000 7f3a2ab9cb80 1 --2- 172.21.15.86:0/11457881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561083a40ef0 0x561083a52a70 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f39fc0097c0 tx=0x7f39fc013040 comp rx=0 tx=0).stop 2024-03-23T17:50:59.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.809+0000 7f3a2ab9cb80 1 -- 172.21.15.86:0/11457881 shutdown_connections 2024-03-23T17:50:59.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.809+0000 7f3a2ab9cb80 1 --2- 172.21.15.86:0/11457881 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561083a4bdf0 0x561083a48420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:59.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.809+0000 7f3a2ab9cb80 1 --2- 172.21.15.86:0/11457881 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561083a405d0 0x561083a49270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:59.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.809+0000 7f3a2ab9cb80 1 --2- 172.21.15.86:0/11457881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561083a40ef0 0x561083a52a70 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:59.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.809+0000 7f3a2ab9cb80 1 -- 172.21.15.86:0/11457881 >> 172.21.15.86:0/11457881 conn(0x5610839a06a0 msgr2=0x561083a43ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:59.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.809+0000 7f3a2ab9cb80 1 -- 172.21.15.86:0/11457881 shutdown_connections 2024-03-23T17:50:59.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.810+0000 7f3a2ab9cb80 1 -- 172.21.15.86:0/11457881 wait complete. 2024-03-23T17:50:59.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.810+0000 7f3a2ab9cb80 1 -- start start 2024-03-23T17:50:59.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.810+0000 7f3a2ab9cb80 1 Processor -- start 2024-03-23T17:50:59.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.811+0000 7f3a2ab9cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561083a405d0 0x561083a512c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:59.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.811+0000 7f3a2ab9cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561083a40ef0 0x561083a51800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:59.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.811+0000 7f3a2ab9cb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561083a4bdf0 0x561083a51d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:59.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.811+0000 7f3a2ab9cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x561083aad6a0 con 0x561083a40ef0 2024-03-23T17:50:59.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.811+0000 7f3a2ab9cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x561083aad810 con 0x561083a405d0 2024-03-23T17:50:59.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.811+0000 7f3a2ab9cb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x561083aad9d0 con 0x561083a4bdf0 2024-03-23T17:50:59.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.811+0000 7f3a2ab9cb80 10 client.0 _ll_register_callbacks cb 0x561083938470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:50:59.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.811+0000 7f3a0f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561083a40ef0 0x561083a51800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:59.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.811+0000 7f3a0f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561083a40ef0 0x561083a51800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43032/0 (socket says 172.21.15.86:43032) 2024-03-23T17:50:59.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.811+0000 7f3a0f400700 1 -- 172.21.15.86:0/614615923 learned_addr learned my addr 172.21.15.86:0/614615923 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:50:59.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.811+0000 7f3a15000700 1 --2- 172.21.15.86:0/614615923 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561083a4bdf0 0x561083a51d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:59.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.811+0000 7f3a0fe00700 1 --2- 172.21.15.86:0/614615923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561083a405d0 0x561083a512c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:59.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.812+0000 7f3a0f400700 1 -- 172.21.15.86:0/614615923 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561083a4bdf0 msgr2=0x561083a51d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:50:59.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.812+0000 7f3a0f400700 1 --2- 172.21.15.86:0/614615923 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561083a4bdf0 0x561083a51d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:59.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.812+0000 7f3a0f400700 1 -- 172.21.15.86:0/614615923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561083a405d0 msgr2=0x561083a512c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:50:59.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.812+0000 7f3a0f400700 1 --2- 172.21.15.86:0/614615923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561083a405d0 0x561083a512c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:59.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.812+0000 7f3a0f400700 1 -- 172.21.15.86:0/614615923 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5610838143b0 con 0x561083a40ef0 2024-03-23T17:50:59.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.812+0000 7f3a0f400700 1 --2- 172.21.15.86:0/614615923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561083a40ef0 0x561083a51800 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f39fc02b040 tx=0x7f39fc033500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c60c32bb7d95413 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:50:59.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.812+0000 7f39fa000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-23T17:50:59.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.812+0000 7f39fa000700 1 -- 172.21.15.86:0/614615923 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39fc003840 con 0x561083a40ef0 2024-03-23T17:50:59.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.812+0000 7f3a2ab9cb80 1 -- 172.21.15.86:0/614615923 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f39fc00ee90 con 0x561083a40ef0 2024-03-23T17:50:59.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.812+0000 7f39fa000700 1 -- 172.21.15.86:0/614615923 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f39fc003cc0 con 0x561083a40ef0 2024-03-23T17:50:59.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.813+0000 7f39fa000700 1 -- 172.21.15.86:0/614615923 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39fc0050d0 con 0x561083a40ef0 2024-03-23T17:50:59.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.813+0000 7f3a2ab9cb80 10 client.15483 fetch_fsmap learned FSMap version 28 2024-03-23T17:50:59.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.813+0000 7f3a2ab9cb80 1 -- 172.21.15.86:0/614615923 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x561083958ea0 con 0x561083a40ef0 2024-03-23T17:50:59.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.813+0000 7f39fa000700 1 -- 172.21.15.86:0/614615923 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39fc00ee90 con 0x561083a40ef0 2024-03-23T17:50:59.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.813+0000 7f39fa000700 1 -- 172.21.15.86:0/614615923 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f39fc03a070 con 0x561083a40ef0 2024-03-23T17:50:59.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.813+0000 7f3a2ab9cb80 10 client.15483 fetch_fsmap finished waiting for FSMap version 28 2024-03-23T17:50:59.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.813+0000 7f3a2ab9cb80 10 client.15483 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T17:50:59.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.813+0000 7f3a2ab9cb80 20 client.15483 populate_metadata read hostname 'smithi086' 2024-03-23T17:50:59.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.813+0000 7f3a2ab9cb80 1 --2- 172.21.15.86:0/614615923 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x561083aaed50 0x561083ab1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:50:59.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.813+0000 7f3a2ab9cb80 4 client.15483 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-23T17:50:59.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.813+0000 7f3a2ab9cb80 1 -- 172.21.15.86:0/614615923 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5610837e1cf0 con 0x561083aaed50 2024-03-23T17:50:59.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.814+0000 7f3a0fe00700 1 --2- 172.21.15.86:0/614615923 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x561083aaed50 0x561083ab1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:50:59.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.814+0000 7f39fa000700 10 client.15483 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:50:59.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.814+0000 7f3a0fe00700 1 --2- 172.21.15.86:0/614615923 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x561083aaed50 0x561083ab1140 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=157586d957fa9f09 server_cookie=2a7868a16459920c in_seq=0 out_seq=0 2024-03-23T17:50:59.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.814+0000 7f39fa000700 1 -- 172.21.15.86:0/614615923 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5610837e1cf0 con 0x561083aaed50 2024-03-23T17:50:59.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.814+0000 7f39fa000700 10 client.15483 handle_command_reply: tid=0 2024-03-23T17:50:59.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.814+0000 7f3a2ab9cb80 1 client.15483 shutdown 2024-03-23T17:50:59.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.814+0000 7f3a2ab9cb80 1 -- 172.21.15.86:0/614615923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561083a40ef0 msgr2=0x561083a51800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:59.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.814+0000 7f3a2ab9cb80 1 --2- 172.21.15.86:0/614615923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561083a40ef0 0x561083a51800 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f39fc02b040 tx=0x7f39fc033500 comp rx=0 tx=0).stop 2024-03-23T17:50:59.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.815+0000 7f3a2ab9cb80 1 -- 172.21.15.86:0/614615923 shutdown_connections 2024-03-23T17:50:59.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.815+0000 7f3a2ab9cb80 1 --2- 172.21.15.86:0/614615923 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x561083aaed50 0x561083ab1140 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:59.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.815+0000 7f3a2ab9cb80 1 --2- 172.21.15.86:0/614615923 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561083a4bdf0 0x561083a51d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:59.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.815+0000 7f3a2ab9cb80 1 --2- 172.21.15.86:0/614615923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561083a405d0 0x561083a512c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:59.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.815+0000 7f3a2ab9cb80 1 --2- 172.21.15.86:0/614615923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561083a40ef0 0x561083a51800 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:59.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.815+0000 7f3a2ab9cb80 1 -- 172.21.15.86:0/614615923 >> 172.21.15.86:0/614615923 conn(0x5610839a06a0 msgr2=0x561083a3bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:59.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.815+0000 7f39fa000700 0 client.15483 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:50:59.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.815+0000 7f3a2ab9cb80 1 -- 172.21.15.86:0/614615923 shutdown_connections 2024-03-23T17:50:59.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.815+0000 7f3a2ab9cb80 1 -- 172.21.15.86:0/614615923 wait complete. 2024-03-23T17:50:59.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.815+0000 7f3a2ab9cb80 20 client.15483 trim_cache size 0 max 16384 2024-03-23T17:50:59.819 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-23T17:50:59.819 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-03-23T17:50:59.820 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-03-23T17:50:59.820 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-23T17:50:59.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.815+0000 7f3a17000700 1 -- 172.21.15.86:0/2519072516 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f39f40a1320 msgr2=0x7f39f40a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:59.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.815+0000 7f3a17000700 1 --2- 172.21.15.86:0/2519072516 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f39f40a1320 0x7f39f40a37e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3a1019aa70 tx=0x7f39fc009260 comp rx=0 tx=0).stop 2024-03-23T17:50:59.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.815+0000 7f3a17000700 1 -- 172.21.15.86:0/2519072516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a10100f70 msgr2=0x7f3a10199570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:50:59.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.815+0000 7f3a17000700 1 --2- 172.21.15.86:0/2519072516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a10100f70 0x7f3a10199570 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3a0000e9a0 tx=0x7f3a0000ecb0 comp rx=0 tx=0).stop 2024-03-23T17:50:59.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.816+0000 7f3a17000700 1 -- 172.21.15.86:0/2519072516 shutdown_connections 2024-03-23T17:50:59.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.816+0000 7f3a17000700 1 --2- 172.21.15.86:0/2519072516 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f39f40a1320 0x7f39f40a37e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:59.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.816+0000 7f3a17000700 1 --2- 172.21.15.86:0/2519072516 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a1019de00 0x7f3a1019e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:59.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.816+0000 7f3a17000700 1 --2- 172.21.15.86:0/2519072516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a10100f70 0x7f3a10199570 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:59.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.816+0000 7f3a17000700 1 --2- 172.21.15.86:0/2519072516 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a10106090 0x7f3a10199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:50:59.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.816+0000 7f3a17000700 1 -- 172.21.15.86:0/2519072516 >> 172.21.15.86:0/2519072516 conn(0x7f3a100fc7f0 msgr2=0x7f3a100feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:50:59.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.816+0000 7f3a17000700 1 -- 172.21.15.86:0/2519072516 shutdown_connections 2024-03-23T17:50:59.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:50:59.816+0000 7f3a17000700 1 -- 172.21.15.86:0/2519072516 wait complete. 2024-03-23T17:50:59.834 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-23T17:50:59.834 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T17:51:00.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.045+0000 7f7084600700 1 -- 172.21.15.86:0/544439776 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f707c100e60 msgr2=0x7f707c101240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:00.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.045+0000 7f7084600700 1 --2- 172.21.15.86:0/544439776 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f707c100e60 0x7f707c101240 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f7070009990 tx=0x7f7070009ca0 comp rx=0 tx=0).stop 2024-03-23T17:51:00.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.045+0000 7f7084600700 1 -- 172.21.15.86:0/544439776 shutdown_connections 2024-03-23T17:51:00.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.045+0000 7f7084600700 1 --2- 172.21.15.86:0/544439776 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f707c105f80 0x7f707c10db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:00.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.045+0000 7f7084600700 1 --2- 172.21.15.86:0/544439776 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f707c100e60 0x7f707c101240 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:00.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.045+0000 7f7084600700 1 --2- 172.21.15.86:0/544439776 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f707c101810 0x7f707c105850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:00.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.045+0000 7f7084600700 1 -- 172.21.15.86:0/544439776 >> 172.21.15.86:0/544439776 conn(0x7f707c0fc700 msgr2=0x7f707c0feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:00.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.045+0000 7f7084600700 1 -- 172.21.15.86:0/544439776 shutdown_connections 2024-03-23T17:51:00.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.045+0000 7f7084600700 1 -- 172.21.15.86:0/544439776 wait complete. 2024-03-23T17:51:00.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.046+0000 7f7084600700 1 Processor -- start 2024-03-23T17:51:00.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.046+0000 7f7084600700 1 -- start start 2024-03-23T17:51:00.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.046+0000 7f7084600700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f707c100e60 0x7f707c199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:00.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.046+0000 7f7084600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f707c105f80 0x7f707c199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:00.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.046+0000 7f7084600700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f707c19dd00 0x7f707c19e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:00.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.047+0000 7f7084600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f707c19e780 con 0x7f707c105f80 2024-03-23T17:51:00.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.047+0000 7f7084600700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f707c19e8f0 con 0x7f707c19dd00 2024-03-23T17:51:00.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.047+0000 7f7084600700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f707c19eab0 con 0x7f707c100e60 2024-03-23T17:51:00.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.047+0000 7f7081200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f707c105f80 0x7f707c199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:00.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.047+0000 7f7081200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f707c105f80 0x7f707c199960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43064/0 (socket says 172.21.15.86:43064) 2024-03-23T17:51:00.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.047+0000 7f7081200700 1 -- 172.21.15.86:0/3922679078 learned_addr learned my addr 172.21.15.86:0/3922679078 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:00.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.047+0000 7f7081c00700 1 --2- 172.21.15.86:0/3922679078 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f707c100e60 0x7f707c199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:00.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.047+0000 7f7082600700 1 --2- 172.21.15.86:0/3922679078 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f707c19dd00 0x7f707c19e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:00.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.047+0000 7f7081200700 1 -- 172.21.15.86:0/3922679078 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f707c100e60 msgr2=0x7f707c199420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:00.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.047+0000 7f7081200700 1 --2- 172.21.15.86:0/3922679078 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f707c100e60 0x7f707c199420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:00.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.047+0000 7f7081200700 1 -- 172.21.15.86:0/3922679078 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f707c19dd00 msgr2=0x7f707c19e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:00.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.047+0000 7f7081200700 1 --2- 172.21.15.86:0/3922679078 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f707c19dd00 0x7f707c19e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:00.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.047+0000 7f7081200700 1 -- 172.21.15.86:0/3922679078 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70700095f0 con 0x7f707c105f80 2024-03-23T17:51:00.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.048+0000 7f7081200700 1 --2- 172.21.15.86:0/3922679078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f707c105f80 0x7f707c199960 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f706800e970 tx=0x7f706800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:00.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.048+0000 7f707a000700 1 -- 172.21.15.86:0/3922679078 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7068009790 con 0x7f707c105f80 2024-03-23T17:51:00.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.048+0000 7f707a000700 1 -- 172.21.15.86:0/3922679078 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f706800c7e0 con 0x7f707c105f80 2024-03-23T17:51:00.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.048+0000 7f7084600700 1 -- 172.21.15.86:0/3922679078 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f707c1a5fb0 con 0x7f707c105f80 2024-03-23T17:51:00.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.048+0000 7f707a000700 1 -- 172.21.15.86:0/3922679078 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70680054e0 con 0x7f707c105f80 2024-03-23T17:51:00.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.048+0000 7f7084600700 1 -- 172.21.15.86:0/3922679078 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f707c1a6490 con 0x7f707c105f80 2024-03-23T17:51:00.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.049+0000 7f7084600700 1 -- 172.21.15.86:0/3922679078 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f707c102330 con 0x7f707c105f80 2024-03-23T17:51:00.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.051+0000 7f707a000700 1 -- 172.21.15.86:0/3922679078 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) v1 ==== 136041+0+0 (secure 0 0 0) 0x7f706800c950 con 0x7f707c105f80 2024-03-23T17:51:00.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.052+0000 7f707a000700 1 --2- 172.21.15.86:0/3922679078 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f70640a1320 0x7f70640a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:00.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.053+0000 7f707a000700 1 -- 172.21.15.86:0/3922679078 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f7068014070 con 0x7f707c105f80 2024-03-23T17:51:00.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.053+0000 7f7081c00700 1 --2- 172.21.15.86:0/3922679078 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f70640a1320 0x7f70640a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:00.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.053+0000 7f7081c00700 1 --2- 172.21.15.86:0/3922679078 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f70640a1320 0x7f70640a37e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7070006010 tx=0x7f707000b260 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:00.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.060+0000 7f707a000700 1 -- 172.21.15.86:0/3922679078 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f7068096540 con 0x7f707c105f80 2024-03-23T17:51:00.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:50:59 smithi086 ceph-mon[38221]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-23T17:51:00.267 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:51:00 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:51:00.068+0000 7fea17e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-23T17:51:00.267 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:51:00 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:51:00.068+0000 7fea17e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-23T17:51:00.267 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:51:00 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:51:00.068+0000 7fea17e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-23T17:51:00.267 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:51:00 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:51:00.068+0000 7fea17e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-23T17:51:00.267 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 17:51:00 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T17:51:00.068+0000 7fea17e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-23T17:51:00.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:50:59 smithi092 ceph-mon[79089]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-23T17:51:00.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:50:59 smithi152 ceph-mon[47949]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-23T17:51:00.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.357+0000 7f7084600700 1 -- 172.21.15.86:0/3922679078 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f707c067a10 con 0x7f707c105f80 2024-03-23T17:51:00.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.358+0000 7f707a000700 1 -- 172.21.15.86:0/3922679078 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+10913 (secure 0 0 0) 0x7f7068099b60 con 0x7f707c105f80 2024-03-23T17:51:00.358 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:51:00.358 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14445,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1433140219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1433140219},{"type":"v1","addr":"172.21.15.86:6835","nonce":1433140219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6839/1879457730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1879457730},{"type":"v1","addr":"172.21.15.86:6839","nonce":1879457730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-23T17:48:56.202323+0000","modified":"2024-03-23T17:50:20.718165+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24553,"mds_2":24518,"mds_3":24505,"mds_4":24500},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14514":{"gid":14514,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/3026943866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3026943866},{"type":"v1","addr":"172.21.15.86:6841","nonce":3026943866}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24464":{"gid":24464,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6833/1467091186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1467091186},{"type":"v1","addr":"172.21.15.92:6833","nonce":1467091186}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6833/3075554623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3075554623},{"type":"v1","addr":"172.21.15.152:6833","nonce":3075554623}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.92:6835/908900027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":908900027},{"type":"v1","addr":"172.21.15.92:6835","nonce":908900027}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"d","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.86:6837/2933881328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2933881328},{"type":"v1","addr":"172.21.15.86:6837","nonce":2933881328}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6837/3967837948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":3967837948},{"type":"v1","addr":"172.21.15.92:6837","nonce":3967837948}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6835/3137085021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3137085021},{"type":"v1","addr":"172.21.15.152:6835","nonce":3137085021}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24538":{"gid":24538,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6837/1151208965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":1151208965},{"type":"v1","addr":"172.21.15.152:6837","nonce":1151208965}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.152:6839/1100723992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":1100723992},{"type":"v1","addr":"172.21.15.152:6839","nonce":1100723992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6839/1394139887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1394139887},{"type":"v1","addr":"172.21.15.92:6839","nonce":1394139887}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-23T17:51:00.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.360+0000 7f7084600700 1 -- 172.21.15.86:0/3922679078 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f70640a1320 msgr2=0x7f70640a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:00.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.360+0000 7f7084600700 1 --2- 172.21.15.86:0/3922679078 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f70640a1320 0x7f70640a37e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7070006010 tx=0x7f707000b260 comp rx=0 tx=0).stop 2024-03-23T17:51:00.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.360+0000 7f7084600700 1 -- 172.21.15.86:0/3922679078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f707c105f80 msgr2=0x7f707c199960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:00.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.361+0000 7f7084600700 1 --2- 172.21.15.86:0/3922679078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f707c105f80 0x7f707c199960 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f706800e970 tx=0x7f706800ec80 comp rx=0 tx=0).stop 2024-03-23T17:51:00.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.361+0000 7f7084600700 1 -- 172.21.15.86:0/3922679078 shutdown_connections 2024-03-23T17:51:00.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.361+0000 7f7084600700 1 --2- 172.21.15.86:0/3922679078 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f707c19dd00 0x7f707c19e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:00.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.361+0000 7f7084600700 1 --2- 172.21.15.86:0/3922679078 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f70640a1320 0x7f70640a37e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:00.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.361+0000 7f7084600700 1 --2- 172.21.15.86:0/3922679078 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f707c100e60 0x7f707c199420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:00.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.361+0000 7f7084600700 1 --2- 172.21.15.86:0/3922679078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f707c105f80 0x7f707c199960 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:00.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.361+0000 7f7084600700 1 -- 172.21.15.86:0/3922679078 >> 172.21.15.86:0/3922679078 conn(0x7f707c0fc700 msgr2=0x7f707c0fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:00.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.361+0000 7f7084600700 1 -- 172.21.15.86:0/3922679078 shutdown_connections 2024-03-23T17:51:00.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.361+0000 7f7084600700 1 -- 172.21.15.86:0/3922679078 wait complete. 2024-03-23T17:51:00.362 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 28 2024-03-23T17:51:00.382 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-23T17:51:00.382 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-23T17:51:00.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.594+0000 7fcb61200700 1 -- 172.21.15.86:0/1022054341 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb5c100f70 msgr2=0x7fcb5c101350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:00.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.594+0000 7fcb61200700 1 --2- 172.21.15.86:0/1022054341 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb5c100f70 0x7fcb5c101350 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c009990 tx=0x7fcb4c009ca0 comp rx=0 tx=0).stop 2024-03-23T17:51:00.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.594+0000 7fcb61200700 1 -- 172.21.15.86:0/1022054341 shutdown_connections 2024-03-23T17:51:00.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.594+0000 7fcb61200700 1 --2- 172.21.15.86:0/1022054341 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb5c106090 0x7fcb5c10dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:00.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.594+0000 7fcb61200700 1 --2- 172.21.15.86:0/1022054341 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb5c100f70 0x7fcb5c101350 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:00.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.594+0000 7fcb61200700 1 --2- 172.21.15.86:0/1022054341 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcb5c101920 0x7fcb5c105960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:00.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.594+0000 7fcb61200700 1 -- 172.21.15.86:0/1022054341 >> 172.21.15.86:0/1022054341 conn(0x7fcb5c0fc7f0 msgr2=0x7fcb5c0fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:00.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.595+0000 7fcb61200700 1 -- 172.21.15.86:0/1022054341 shutdown_connections 2024-03-23T17:51:00.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.595+0000 7fcb61200700 1 -- 172.21.15.86:0/1022054341 wait complete. 2024-03-23T17:51:00.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.596+0000 7fcb61200700 1 Processor -- start 2024-03-23T17:51:00.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.596+0000 7fcb61200700 1 -- start start 2024-03-23T17:51:00.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.596+0000 7fcb61200700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcb5c100f70 0x7fcb5c0730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:00.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.596+0000 7fcb61200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb5c106090 0x7fcb5c0735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:00.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.596+0000 7fcb61200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb5c073b30 0x7fcb5c1a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:00.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.596+0000 7fcb61200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5c1a59a0 con 0x7fcb5c106090 2024-03-23T17:51:00.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.596+0000 7fcb61200700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5c1a5b10 con 0x7fcb5c073b30 2024-03-23T17:51:00.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.596+0000 7fcb61200700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5c1a5cd0 con 0x7fcb5c100f70 2024-03-23T17:51:00.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.596+0000 7fcb59400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb5c106090 0x7fcb5c0735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:00.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.596+0000 7fcb59400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb5c106090 0x7fcb5c0735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43112/0 (socket says 172.21.15.86:43112) 2024-03-23T17:51:00.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.596+0000 7fcb59400700 1 -- 172.21.15.86:0/3933523167 learned_addr learned my addr 172.21.15.86:0/3933523167 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:00.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.596+0000 7fcb59e00700 1 --2- 172.21.15.86:0/3933523167 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcb5c100f70 0x7fcb5c0730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:00.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.596+0000 7fcb5a800700 1 --2- 172.21.15.86:0/3933523167 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb5c073b30 0x7fcb5c1a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:00.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.597+0000 7fcb59400700 1 -- 172.21.15.86:0/3933523167 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcb5c100f70 msgr2=0x7fcb5c0730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:00.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.597+0000 7fcb59400700 1 --2- 172.21.15.86:0/3933523167 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcb5c100f70 0x7fcb5c0730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:00.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.597+0000 7fcb59400700 1 -- 172.21.15.86:0/3933523167 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb5c073b30 msgr2=0x7fcb5c1a53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:00.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.597+0000 7fcb59400700 1 --2- 172.21.15.86:0/3933523167 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb5c073b30 0x7fcb5c1a53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:00.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.597+0000 7fcb59400700 1 -- 172.21.15.86:0/3933523167 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb4c0095f0 con 0x7fcb5c106090 2024-03-23T17:51:00.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.597+0000 7fcb59400700 1 --2- 172.21.15.86:0/3933523167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb5c106090 0x7fcb5c0735f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fcb4800e970 tx=0x7fcb4800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:00.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.597+0000 7fcb52a00700 1 -- 172.21.15.86:0/3933523167 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb48009790 con 0x7fcb5c106090 2024-03-23T17:51:00.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.597+0000 7fcb52a00700 1 -- 172.21.15.86:0/3933523167 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb4800c7e0 con 0x7fcb5c106090 2024-03-23T17:51:00.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.597+0000 7fcb61200700 1 -- 172.21.15.86:0/3933523167 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb5c1a5fb0 con 0x7fcb5c106090 2024-03-23T17:51:00.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.597+0000 7fcb52a00700 1 -- 172.21.15.86:0/3933523167 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb480054e0 con 0x7fcb5c106090 2024-03-23T17:51:00.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.598+0000 7fcb61200700 1 -- 172.21.15.86:0/3933523167 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb5c1a6480 con 0x7fcb5c106090 2024-03-23T17:51:00.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.602+0000 7fcb52a00700 1 -- 172.21.15.86:0/3933523167 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) v1 ==== 136041+0+0 (secure 0 0 0) 0x7fcb4800c950 con 0x7fcb5c106090 2024-03-23T17:51:00.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.603+0000 7fcb74d56b80 1 Processor -- start 2024-03-23T17:51:00.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.603+0000 7fcb74d56b80 1 -- start start 2024-03-23T17:51:00.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.603+0000 7fcb52a00700 1 --2- 172.21.15.86:0/3933523167 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fcb400a13a0 0x7fcb400a3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:00.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.604+0000 7fcb74d56b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561db84955d0 0x561db84959b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:00.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.604+0000 7fcb74d56b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561db8495ef0 0x561db84a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:00.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.604+0000 7fcb74d56b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561db84a0df0 0x561db84a31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:00.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.604+0000 7fcb74d56b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x561db83ee990 con 0x561db8495ef0 2024-03-23T17:51:00.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.604+0000 7fcb74d56b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x561db83ec350 con 0x561db84a0df0 2024-03-23T17:51:00.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.604+0000 7fcb74d56b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x561db83e7c40 con 0x561db84955d0 2024-03-23T17:51:00.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.604+0000 7fcb52a00700 1 -- 172.21.15.86:0/3933523167 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fcb48014070 con 0x7fcb5c106090 2024-03-23T17:51:00.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.604+0000 7fcb5a800700 1 --2- 172.21.15.86:0/3933523167 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fcb400a13a0 0x7fcb400a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:00.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.604+0000 7fcb59e00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561db84955d0 0x561db84959b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:00.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.604+0000 7fcb5a800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561db84a0df0 0x561db84a31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:00.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.604+0000 7fcb59e00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561db84955d0 0x561db84959b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:53916/0 (socket says 172.21.15.86:53916) 2024-03-23T17:51:00.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.604+0000 7fcb59e00700 1 -- 172.21.15.86:0/292550346 learned_addr learned my addr 172.21.15.86:0/292550346 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:00.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.604+0000 7fcb59400700 1 --2- 172.21.15.86:0/292550346 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561db8495ef0 0x561db84a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:00.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.604+0000 7fcb59e00700 1 -- 172.21.15.86:0/292550346 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561db84a0df0 msgr2=0x561db84a31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:00.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.604+0000 7fcb59e00700 1 --2- 172.21.15.86:0/292550346 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561db84a0df0 0x561db84a31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:00.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.604+0000 7fcb59e00700 1 -- 172.21.15.86:0/292550346 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561db8495ef0 msgr2=0x561db84a08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:00.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.605+0000 7fcb59e00700 1 --2- 172.21.15.86:0/292550346 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561db8495ef0 0x561db84a08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:00.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.605+0000 7fcb59e00700 1 -- 172.21.15.86:0/292550346 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561db82693b0 con 0x561db84955d0 2024-03-23T17:51:00.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.605+0000 7fcb5a800700 1 --2- 172.21.15.86:0/3933523167 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fcb400a13a0 0x7fcb400a3860 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fcb54009ec0 tx=0x7fcb54016450 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:00.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.605+0000 7fcb59e00700 1 --2- 172.21.15.86:0/292550346 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561db84955d0 0x561db84959b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c005ec0 tx=0x7fcb4c005ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=682ed06d9231fb43 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:00.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.605+0000 7fcb47400700 1 -- 172.21.15.86:0/292550346 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c01d070 con 0x561db84955d0 2024-03-23T17:51:00.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.605+0000 7fcb47400700 1 -- 172.21.15.86:0/292550346 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb4c01e040 con 0x561db84955d0 2024-03-23T17:51:00.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.605+0000 7fcb47400700 1 -- 172.21.15.86:0/292550346 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c004790 con 0x561db84955d0 2024-03-23T17:51:00.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.606+0000 7fcb74d56b80 1 -- 172.21.15.86:0/292550346 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561db84955d0 msgr2=0x561db84959b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:00.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.606+0000 7fcb74d56b80 1 --2- 172.21.15.86:0/292550346 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561db84955d0 0x561db84959b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c005ec0 tx=0x7fcb4c005ef0 comp rx=0 tx=0).stop 2024-03-23T17:51:00.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.606+0000 7fcb74d56b80 1 -- 172.21.15.86:0/292550346 shutdown_connections 2024-03-23T17:51:00.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.606+0000 7fcb74d56b80 1 --2- 172.21.15.86:0/292550346 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561db84a0df0 0x561db84a31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:00.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.606+0000 7fcb74d56b80 1 --2- 172.21.15.86:0/292550346 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561db84955d0 0x561db84959b0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:00.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.606+0000 7fcb74d56b80 1 --2- 172.21.15.86:0/292550346 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561db8495ef0 0x561db84a08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:00.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.606+0000 7fcb74d56b80 1 -- 172.21.15.86:0/292550346 >> 172.21.15.86:0/292550346 conn(0x561db83f56a0 msgr2=0x561db8491a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:00.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.606+0000 7fcb74d56b80 1 -- 172.21.15.86:0/292550346 shutdown_connections 2024-03-23T17:51:00.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.606+0000 7fcb74d56b80 1 -- 172.21.15.86:0/292550346 wait complete. 2024-03-23T17:51:00.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.607+0000 7fcb74d56b80 1 -- start start 2024-03-23T17:51:00.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.607+0000 7fcb74d56b80 1 Processor -- start 2024-03-23T17:51:00.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.607+0000 7fcb74d56b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561db84955d0 0x561db84a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:00.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.607+0000 7fcb74d56b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561db8495ef0 0x561db849da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:00.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.607+0000 7fcb74d56b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561db84a0df0 0x561db849df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:00.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.607+0000 7fcb74d56b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x561db8383d80 con 0x561db84955d0 2024-03-23T17:51:00.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.607+0000 7fcb74d56b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x561db8383a20 con 0x561db84a0df0 2024-03-23T17:51:00.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.607+0000 7fcb74d56b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x561db826b6a0 con 0x561db8495ef0 2024-03-23T17:51:00.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.608+0000 7fcb74d56b80 10 client.0 _ll_register_callbacks cb 0x561db838d470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:51:00.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.608+0000 7fcb59e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561db84955d0 0x561db84a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:00.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.608+0000 7fcb59e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561db84955d0 0x561db84a06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43130/0 (socket says 172.21.15.86:43130) 2024-03-23T17:51:00.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.608+0000 7fcb59e00700 1 -- 172.21.15.86:0/1962728150 learned_addr learned my addr 172.21.15.86:0/1962728150 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:00.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.608+0000 7fcb5a800700 1 --2- 172.21.15.86:0/1962728150 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561db84a0df0 0x561db849df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:00.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.608+0000 7fcb59400700 1 --2- 172.21.15.86:0/1962728150 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561db8495ef0 0x561db849da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:00.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.608+0000 7fcb59e00700 1 -- 172.21.15.86:0/1962728150 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561db8495ef0 msgr2=0x561db849da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:00.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.608+0000 7fcb59e00700 1 --2- 172.21.15.86:0/1962728150 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561db8495ef0 0x561db849da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:00.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.608+0000 7fcb59e00700 1 -- 172.21.15.86:0/1962728150 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561db84a0df0 msgr2=0x561db849df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:00.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.608+0000 7fcb59e00700 1 --2- 172.21.15.86:0/1962728150 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561db84a0df0 0x561db849df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:00.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.608+0000 7fcb59e00700 1 -- 172.21.15.86:0/1962728150 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb5c10e480 con 0x561db84955d0 2024-03-23T17:51:00.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.609+0000 7fcb59e00700 1 --2- 172.21.15.86:0/1962728150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561db84955d0 0x561db84a06d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c005920 tx=0x7fcb4c00b9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcc2b71873fde984 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:00.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.609+0000 7fcb3be00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-23T17:51:00.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.609+0000 7fcb3be00700 1 -- 172.21.15.86:0/1962728150 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c01d030 con 0x561db84955d0 2024-03-23T17:51:00.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.609+0000 7fcb74d56b80 1 -- 172.21.15.86:0/1962728150 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561db83a8380 con 0x561db84955d0 2024-03-23T17:51:00.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.609+0000 7fcb3be00700 1 -- 172.21.15.86:0/1962728150 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fcb4c004700 con 0x561db84955d0 2024-03-23T17:51:00.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.609+0000 7fcb3be00700 1 -- 172.21.15.86:0/1962728150 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb4c005bc0 con 0x561db84955d0 2024-03-23T17:51:00.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.609+0000 7fcb74d56b80 10 client.15507 fetch_fsmap learned FSMap version 28 2024-03-23T17:51:00.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.609+0000 7fcb74d56b80 1 -- 172.21.15.86:0/1962728150 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x561db8215b60 con 0x561db84955d0 2024-03-23T17:51:00.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.609+0000 7fcb3be00700 1 -- 172.21.15.86:0/1962728150 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561db83a8380 con 0x561db84955d0 2024-03-23T17:51:00.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.610+0000 7fcb3be00700 1 -- 172.21.15.86:0/1962728150 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fcb4c00f9c0 con 0x561db84955d0 2024-03-23T17:51:00.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.610+0000 7fcb74d56b80 10 client.15507 fetch_fsmap finished waiting for FSMap version 28 2024-03-23T17:51:00.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.610+0000 7fcb74d56b80 10 client.15507 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-03-23T17:51:00.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.610+0000 7fcb74d56b80 20 client.15507 populate_metadata read hostname 'smithi086' 2024-03-23T17:51:00.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.610+0000 7fcb74d56b80 1 --2- 172.21.15.86:0/1962728150 >> [v2:172.21.15.92:6834/908900027,v1:172.21.15.92:6835/908900027] conn(0x561db84b03a0 0x561db84b2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:00.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.610+0000 7fcb74d56b80 4 client.15507 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:51:00.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.610+0000 7fcb74d56b80 1 -- 172.21.15.86:0/1962728150 --> [v2:172.21.15.92:6834/908900027,v1:172.21.15.92:6835/908900027] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561db83e0700 con 0x561db84b03a0 2024-03-23T17:51:00.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.610+0000 7fcb59400700 1 --2- 172.21.15.86:0/1962728150 >> [v2:172.21.15.92:6834/908900027,v1:172.21.15.92:6835/908900027] conn(0x561db84b03a0 0x561db84b2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:00.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.610+0000 7fcb59400700 1 --2- 172.21.15.86:0/1962728150 >> [v2:172.21.15.92:6834/908900027,v1:172.21.15.92:6835/908900027] conn(0x561db84b03a0 0x561db84b2790 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d0f7a2a41fd2563e server_cookie=d4cdfbafeb593ba6 in_seq=0 out_seq=0 2024-03-23T17:51:00.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.610+0000 7fcb3be00700 10 client.15507 ms_handle_connect on v2:172.21.15.92:6834/908900027 2024-03-23T17:51:00.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.611+0000 7fcb3be00700 1 -- 172.21.15.86:0/1962728150 <== mds.4 v2:172.21.15.92:6834/908900027 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x561db83e0700 con 0x561db84b03a0 2024-03-23T17:51:00.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.611+0000 7fcb3be00700 10 client.15507 handle_command_reply: tid=0 2024-03-23T17:51:00.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.611+0000 7fcb74d56b80 1 client.15507 shutdown 2024-03-23T17:51:00.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.611+0000 7fcb74d56b80 1 -- 172.21.15.86:0/1962728150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561db84955d0 msgr2=0x561db84a06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:00.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.611+0000 7fcb74d56b80 1 --2- 172.21.15.86:0/1962728150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561db84955d0 0x561db84a06d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c005920 tx=0x7fcb4c00b9e0 comp rx=0 tx=0).stop 2024-03-23T17:51:00.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.611+0000 7fcb74d56b80 1 -- 172.21.15.86:0/1962728150 shutdown_connections 2024-03-23T17:51:00.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.611+0000 7fcb74d56b80 1 --2- 172.21.15.86:0/1962728150 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561db84a0df0 0x561db849df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:00.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.611+0000 7fcb74d56b80 1 --2- 172.21.15.86:0/1962728150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561db84955d0 0x561db84a06d0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:00.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.611+0000 7fcb74d56b80 1 --2- 172.21.15.86:0/1962728150 >> [v2:172.21.15.92:6834/908900027,v1:172.21.15.92:6835/908900027] conn(0x561db84b03a0 0x561db84b2790 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:00.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.611+0000 7fcb74d56b80 1 --2- 172.21.15.86:0/1962728150 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561db8495ef0 0x561db849da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:00.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.611+0000 7fcb3be00700 0 client.15507 ms_handle_reset on v2:172.21.15.92:6834/908900027 2024-03-23T17:51:00.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.611+0000 7fcb74d56b80 1 -- 172.21.15.86:0/1962728150 >> 172.21.15.86:0/1962728150 conn(0x561db83f56a0 msgr2=0x561db8497670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:00.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.612+0000 7fcb74d56b80 1 -- 172.21.15.86:0/1962728150 shutdown_connections 2024-03-23T17:51:00.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.612+0000 7fcb74d56b80 1 -- 172.21.15.86:0/1962728150 wait complete. 2024-03-23T17:51:00.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.612+0000 7fcb74d56b80 20 client.15507 trim_cache size 0 max 16384 2024-03-23T17:51:00.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.625+0000 7fcb74d56b80 1 Processor -- start 2024-03-23T17:51:00.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.625+0000 7fcb74d56b80 1 -- start start 2024-03-23T17:51:00.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.625+0000 7fcb74d56b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561db84955d0 0x561db84a7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:00.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.625+0000 7fcb74d56b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561db8495ef0 0x561db8497670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:00.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.625+0000 7fcb74d56b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561db84a0df0 0x561db8497bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:00.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.625+0000 7fcb74d56b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x561db8496890 con 0x561db84a0df0 2024-03-23T17:51:00.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.625+0000 7fcb74d56b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x561db849e2f0 con 0x561db84955d0 2024-03-23T17:51:00.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.625+0000 7fcb74d56b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x561db8498840 con 0x561db8495ef0 2024-03-23T17:51:00.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.626+0000 7fcb5a800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561db84a0df0 0x561db8497bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:00.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.626+0000 7fcb5a800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561db84a0df0 0x561db8497bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43138/0 (socket says 172.21.15.86:43138) 2024-03-23T17:51:00.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.626+0000 7fcb5a800700 1 -- 172.21.15.86:0/872750634 learned_addr learned my addr 172.21.15.86:0/872750634 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:00.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.626+0000 7fcb59400700 1 --2- 172.21.15.86:0/872750634 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561db8495ef0 0x561db8497670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:00.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.626+0000 7fcb59e00700 1 --2- 172.21.15.86:0/872750634 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561db84955d0 0x561db84a7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:00.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.626+0000 7fcb5a800700 1 -- 172.21.15.86:0/872750634 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561db8495ef0 msgr2=0x561db8497670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:00.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.626+0000 7fcb5a800700 1 --2- 172.21.15.86:0/872750634 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561db8495ef0 0x561db8497670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:00.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.626+0000 7fcb5a800700 1 -- 172.21.15.86:0/872750634 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561db84955d0 msgr2=0x561db84a7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:00.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.626+0000 7fcb5a800700 1 --2- 172.21.15.86:0/872750634 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561db84955d0 0x561db84a7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:00.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.626+0000 7fcb5a800700 1 -- 172.21.15.86:0/872750634 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561db83adea0 con 0x561db84a0df0 2024-03-23T17:51:00.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.626+0000 7fcb5a800700 1 --2- 172.21.15.86:0/872750634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561db84a0df0 0x561db8497bb0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fcb5401ae70 tx=0x7fcb5401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcda3dbd44b1a6c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:00.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.627+0000 7fcb47e00700 1 -- 172.21.15.86:0/872750634 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5400e040 con 0x561db84a0df0 2024-03-23T17:51:00.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.627+0000 7fcb47e00700 1 -- 172.21.15.86:0/872750634 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb5400f040 con 0x561db84a0df0 2024-03-23T17:51:00.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.627+0000 7fcb47e00700 1 -- 172.21.15.86:0/872750634 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb54028b90 con 0x561db84a0df0 2024-03-23T17:51:00.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.627+0000 7fcb74d56b80 1 -- 172.21.15.86:0/872750634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561db84a0df0 msgr2=0x561db8497bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:00.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.627+0000 7fcb74d56b80 1 --2- 172.21.15.86:0/872750634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561db84a0df0 0x561db8497bb0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fcb5401ae70 tx=0x7fcb5401d000 comp rx=0 tx=0).stop 2024-03-23T17:51:00.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.628+0000 7fcb74d56b80 1 -- 172.21.15.86:0/872750634 shutdown_connections 2024-03-23T17:51:00.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.628+0000 7fcb74d56b80 1 --2- 172.21.15.86:0/872750634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561db84a0df0 0x561db8497bb0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:00.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.628+0000 7fcb74d56b80 1 --2- 172.21.15.86:0/872750634 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561db84955d0 0x561db84a7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:00.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.628+0000 7fcb74d56b80 1 --2- 172.21.15.86:0/872750634 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561db8495ef0 0x561db8497670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:00.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.628+0000 7fcb74d56b80 1 -- 172.21.15.86:0/872750634 >> 172.21.15.86:0/872750634 conn(0x561db83f56a0 msgr2=0x561db8498d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:00.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.628+0000 7fcb74d56b80 1 -- 172.21.15.86:0/872750634 shutdown_connections 2024-03-23T17:51:00.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.628+0000 7fcb74d56b80 1 -- 172.21.15.86:0/872750634 wait complete. 2024-03-23T17:51:00.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.628+0000 7fcb74d56b80 1 -- start start 2024-03-23T17:51:00.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.628+0000 7fcb74d56b80 1 Processor -- start 2024-03-23T17:51:00.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.629+0000 7fcb74d56b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561db84955d0 0x561db84fb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:00.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.629+0000 7fcb74d56b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561db8495ef0 0x561db84fbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:00.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.629+0000 7fcb74d56b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561db84a0df0 0x561db84f5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:00.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.629+0000 7fcb74d56b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x561db84f6490 con 0x561db84a0df0 2024-03-23T17:51:00.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.629+0000 7fcb74d56b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x561db84f6600 con 0x561db84955d0 2024-03-23T17:51:00.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.629+0000 7fcb74d56b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x561db84f67c0 con 0x561db8495ef0 2024-03-23T17:51:00.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.629+0000 7fcb74d56b80 10 client.0 _ll_register_callbacks cb 0x561db838d470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:51:00.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.629+0000 7fcb5a800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561db84a0df0 0x561db84f5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:00.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.630+0000 7fcb59400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561db8495ef0 0x561db84fbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:00.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.630+0000 7fcb59e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561db84955d0 0x561db84fb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:00.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.630+0000 7fcb59400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561db8495ef0 0x561db84fbbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:53942/0 (socket says 172.21.15.86:53942) 2024-03-23T17:51:00.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.630+0000 7fcb59400700 1 -- 172.21.15.86:0/3196054555 learned_addr learned my addr 172.21.15.86:0/3196054555 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:00.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.630+0000 7fcb59400700 1 -- 172.21.15.86:0/3196054555 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561db84955d0 msgr2=0x561db84fb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:00.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.630+0000 7fcb59400700 1 --2- 172.21.15.86:0/3196054555 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561db84955d0 0x561db84fb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:00.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.630+0000 7fcb59400700 1 -- 172.21.15.86:0/3196054555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561db84a0df0 msgr2=0x561db84f5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:00.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.630+0000 7fcb59400700 1 --2- 172.21.15.86:0/3196054555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561db84a0df0 0x561db84f5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:00.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.630+0000 7fcb59400700 1 -- 172.21.15.86:0/3196054555 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb5401abe0 con 0x561db8495ef0 2024-03-23T17:51:00.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.630+0000 7fcb59400700 1 --2- 172.21.15.86:0/3196054555 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561db8495ef0 0x561db84fbbd0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fcb480964a0 tx=0x7fcb48005b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8cf5a6f7f37587d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:00.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.630+0000 7fcb3be00700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-03-23T17:51:00.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.630+0000 7fcb3be00700 1 -- 172.21.15.86:0/3196054555 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb480caea0 con 0x561db8495ef0 2024-03-23T17:51:00.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.630+0000 7fcb3be00700 1 -- 172.21.15.86:0/3196054555 <== mon.1 v2:172.21.15.152:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fcb4809ddd0 con 0x561db8495ef0 2024-03-23T17:51:00.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.630+0000 7fcb74d56b80 1 -- 172.21.15.86:0/3196054555 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcb4c01e040 con 0x561db8495ef0 2024-03-23T17:51:00.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.631+0000 7fcb3be00700 1 -- 172.21.15.86:0/3196054555 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb480a2420 con 0x561db8495ef0 2024-03-23T17:51:00.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.631+0000 7fcb74d56b80 10 client.25558 fetch_fsmap learned FSMap version 28 2024-03-23T17:51:00.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.631+0000 7fcb74d56b80 1 -- 172.21.15.86:0/3196054555 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x561db826dcd0 con 0x561db8495ef0 2024-03-23T17:51:00.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.631+0000 7fcb3be00700 1 -- 172.21.15.86:0/3196054555 <== mon.1 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb4c01e040 con 0x561db8495ef0 2024-03-23T17:51:00.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.631+0000 7fcb3be00700 1 -- 172.21.15.86:0/3196054555 <== mon.1 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fcb480ca940 con 0x561db8495ef0 2024-03-23T17:51:00.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.631+0000 7fcb74d56b80 10 client.25558 fetch_fsmap finished waiting for FSMap version 28 2024-03-23T17:51:00.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.631+0000 7fcb74d56b80 10 client.25558 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-03-23T17:51:00.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.631+0000 7fcb74d56b80 20 client.25558 populate_metadata read hostname 'smithi086' 2024-03-23T17:51:00.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.631+0000 7fcb74d56b80 1 --2- 172.21.15.86:0/3196054555 >> [v2:172.21.15.92:6834/908900027,v1:172.21.15.92:6835/908900027] conn(0x561db8503320 0x561db8505740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:00.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.631+0000 7fcb74d56b80 4 client.25558 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-23T17:51:00.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.631+0000 7fcb74d56b80 1 -- 172.21.15.86:0/3196054555 --> [v2:172.21.15.92:6834/908900027,v1:172.21.15.92:6835/908900027] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561db8236cf0 con 0x561db8503320 2024-03-23T17:51:00.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.632+0000 7fcb59e00700 1 --2- 172.21.15.86:0/3196054555 >> [v2:172.21.15.92:6834/908900027,v1:172.21.15.92:6835/908900027] conn(0x561db8503320 0x561db8505740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:00.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.632+0000 7fcb59e00700 1 --2- 172.21.15.86:0/3196054555 >> [v2:172.21.15.92:6834/908900027,v1:172.21.15.92:6835/908900027] conn(0x561db8503320 0x561db8505740 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2708c041c2e9c2c5 server_cookie=76e6d30dfcbaee41 in_seq=0 out_seq=0 2024-03-23T17:51:00.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.632+0000 7fcb3be00700 10 client.25558 ms_handle_connect on v2:172.21.15.92:6834/908900027 2024-03-23T17:51:00.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.632+0000 7fcb3be00700 1 -- 172.21.15.86:0/3196054555 <== mds.4 v2:172.21.15.92:6834/908900027 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561db8236cf0 con 0x561db8503320 2024-03-23T17:51:00.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.632+0000 7fcb3be00700 10 client.25558 handle_command_reply: tid=0 2024-03-23T17:51:00.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.632+0000 7fcb74d56b80 1 client.25558 shutdown 2024-03-23T17:51:00.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.632+0000 7fcb74d56b80 1 -- 172.21.15.86:0/3196054555 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561db8495ef0 msgr2=0x561db84fbbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:00.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.632+0000 7fcb74d56b80 1 --2- 172.21.15.86:0/3196054555 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561db8495ef0 0x561db84fbbd0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fcb480964a0 tx=0x7fcb48005b40 comp rx=0 tx=0).stop 2024-03-23T17:51:00.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.632+0000 7fcb74d56b80 1 -- 172.21.15.86:0/3196054555 shutdown_connections 2024-03-23T17:51:00.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.632+0000 7fcb74d56b80 1 --2- 172.21.15.86:0/3196054555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561db84a0df0 0x561db84f5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:00.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.632+0000 7fcb74d56b80 1 --2- 172.21.15.86:0/3196054555 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561db84955d0 0x561db84fb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:00.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.632+0000 7fcb74d56b80 1 --2- 172.21.15.86:0/3196054555 >> [v2:172.21.15.92:6834/908900027,v1:172.21.15.92:6835/908900027] conn(0x561db8503320 0x561db8505740 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:00.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.632+0000 7fcb74d56b80 1 --2- 172.21.15.86:0/3196054555 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561db8495ef0 0x561db84fbbd0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:00.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.632+0000 7fcb74d56b80 1 -- 172.21.15.86:0/3196054555 >> 172.21.15.86:0/3196054555 conn(0x561db83f56a0 msgr2=0x561db8497670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:00.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.632+0000 7fcb3be00700 0 client.25558 ms_handle_reset on v2:172.21.15.92:6834/908900027 2024-03-23T17:51:00.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.633+0000 7fcb74d56b80 1 -- 172.21.15.86:0/3196054555 shutdown_connections 2024-03-23T17:51:00.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.633+0000 7fcb74d56b80 1 -- 172.21.15.86:0/3196054555 wait complete. 2024-03-23T17:51:00.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.633+0000 7fcb74d56b80 20 client.25558 trim_cache size 0 max 16384 2024-03-23T17:51:00.637 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-03-23T17:51:00.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.633+0000 7fcb61200700 1 -- 172.21.15.86:0/3933523167 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fcb400a13a0 msgr2=0x7fcb400a3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:00.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.633+0000 7fcb61200700 1 --2- 172.21.15.86:0/3933523167 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fcb400a13a0 0x7fcb400a3860 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fcb54009ec0 tx=0x7fcb54016450 comp rx=0 tx=0).stop 2024-03-23T17:51:00.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.633+0000 7fcb61200700 1 -- 172.21.15.86:0/3933523167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb5c106090 msgr2=0x7fcb5c0735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:00.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.633+0000 7fcb61200700 1 --2- 172.21.15.86:0/3933523167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb5c106090 0x7fcb5c0735f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fcb4800e970 tx=0x7fcb4800ec80 comp rx=0 tx=0).stop 2024-03-23T17:51:00.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.633+0000 7fcb61200700 1 -- 172.21.15.86:0/3933523167 shutdown_connections 2024-03-23T17:51:00.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.633+0000 7fcb61200700 1 --2- 172.21.15.86:0/3933523167 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb5c073b30 0x7fcb5c1a53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:00.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.633+0000 7fcb61200700 1 --2- 172.21.15.86:0/3933523167 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fcb400a13a0 0x7fcb400a3860 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:00.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.633+0000 7fcb61200700 1 --2- 172.21.15.86:0/3933523167 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcb5c100f70 0x7fcb5c0730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:00.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.633+0000 7fcb61200700 1 --2- 172.21.15.86:0/3933523167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb5c106090 0x7fcb5c0735f0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:00.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.633+0000 7fcb61200700 1 -- 172.21.15.86:0/3933523167 >> 172.21.15.86:0/3933523167 conn(0x7fcb5c0fc7f0 msgr2=0x7fcb5c0fdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:00.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.634+0000 7fcb61200700 1 -- 172.21.15.86:0/3933523167 shutdown_connections 2024-03-23T17:51:00.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.634+0000 7fcb61200700 1 -- 172.21.15.86:0/3933523167 wait complete. 2024-03-23T17:51:00.653 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-23T17:51:00.653 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-23T17:51:00.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.852+0000 7f6ccf400700 1 -- 172.21.15.86:0/2018461756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6cc8103170 msgr2=0x7f6cc8103550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:00.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.852+0000 7f6ccf400700 1 --2- 172.21.15.86:0/2018461756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6cc8103170 0x7f6cc8103550 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc009990 tx=0x7f6cbc009ca0 comp rx=0 tx=0).stop 2024-03-23T17:51:00.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.852+0000 7f6ccf400700 1 -- 172.21.15.86:0/2018461756 shutdown_connections 2024-03-23T17:51:00.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.852+0000 7f6ccf400700 1 --2- 172.21.15.86:0/2018461756 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6cc8108290 0x7f6cc810fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:00.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.852+0000 7f6ccf400700 1 --2- 172.21.15.86:0/2018461756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6cc8103170 0x7f6cc8103550 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:00.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.852+0000 7f6ccf400700 1 --2- 172.21.15.86:0/2018461756 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6cc8103b20 0x7f6cc8107b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:00.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.852+0000 7f6ccf400700 1 -- 172.21.15.86:0/2018461756 >> 172.21.15.86:0/2018461756 conn(0x7f6cc80fe990 msgr2=0x7f6cc8100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:00.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.852+0000 7f6ccf400700 1 -- 172.21.15.86:0/2018461756 shutdown_connections 2024-03-23T17:51:00.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.852+0000 7f6ccf400700 1 -- 172.21.15.86:0/2018461756 wait complete. 2024-03-23T17:51:00.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.853+0000 7f6ccf400700 1 Processor -- start 2024-03-23T17:51:00.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.853+0000 7f6ccf400700 1 -- start start 2024-03-23T17:51:00.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.853+0000 7f6ccf400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6cc8103170 0x7f6cc8199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:00.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.853+0000 7f6ccf400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6cc8108290 0x7f6cc81999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:00.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.853+0000 7f6ccf400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6cc819dcf0 0x7f6cc819e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:00.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.853+0000 7f6ccf400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc819e770 con 0x7f6cc8103170 2024-03-23T17:51:00.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.853+0000 7f6ccf400700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc819e8e0 con 0x7f6cc819dcf0 2024-03-23T17:51:00.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.853+0000 7f6ccf400700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc819eaa0 con 0x7f6cc8108290 2024-03-23T17:51:00.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.854+0000 7f6ccca00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6cc8103170 0x7f6cc8199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:00.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.854+0000 7f6ccca00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6cc8103170 0x7f6cc8199460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43162/0 (socket says 172.21.15.86:43162) 2024-03-23T17:51:00.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.854+0000 7f6ccca00700 1 -- 172.21.15.86:0/1985733722 learned_addr learned my addr 172.21.15.86:0/1985733722 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:00.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.854+0000 7f6cc7e00700 1 --2- 172.21.15.86:0/1985733722 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6cc8108290 0x7f6cc81999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:00.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.854+0000 7f6ccd400700 1 --2- 172.21.15.86:0/1985733722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6cc819dcf0 0x7f6cc819e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:00.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.854+0000 7f6ccca00700 1 -- 172.21.15.86:0/1985733722 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6cc8108290 msgr2=0x7f6cc81999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:00.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.854+0000 7f6ccca00700 1 --2- 172.21.15.86:0/1985733722 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6cc8108290 0x7f6cc81999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:00.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.854+0000 7f6ccca00700 1 -- 172.21.15.86:0/1985733722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6cc819dcf0 msgr2=0x7f6cc819e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:00.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.854+0000 7f6ccca00700 1 --2- 172.21.15.86:0/1985733722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6cc819dcf0 0x7f6cc819e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:00.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.854+0000 7f6ccca00700 1 -- 172.21.15.86:0/1985733722 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cbc0095f0 con 0x7f6cc8103170 2024-03-23T17:51:00.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.854+0000 7f6cc7e00700 1 --2- 172.21.15.86:0/1985733722 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6cc8108290 0x7f6cc81999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:51:00.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.855+0000 7f6ccca00700 1 --2- 172.21.15.86:0/1985733722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6cc8103170 0x7f6cc8199460 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc000c00 tx=0x7f6cbc005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:00.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.855+0000 7f6cc5600700 1 -- 172.21.15.86:0/1985733722 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cbc01d070 con 0x7f6cc8103170 2024-03-23T17:51:00.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.855+0000 7f6cc5600700 1 -- 172.21.15.86:0/1985733722 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6cbc022470 con 0x7f6cc8103170 2024-03-23T17:51:00.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.855+0000 7f6ccf400700 1 -- 172.21.15.86:0/1985733722 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cc81a5fa0 con 0x7f6cc8103170 2024-03-23T17:51:00.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.855+0000 7f6cc5600700 1 -- 172.21.15.86:0/1985733722 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cbc017650 con 0x7f6cc8103170 2024-03-23T17:51:00.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.855+0000 7f6ccf400700 1 -- 172.21.15.86:0/1985733722 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cc81a6360 con 0x7f6cc8103170 2024-03-23T17:51:00.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.858+0000 7f6cc5600700 1 -- 172.21.15.86:0/1985733722 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) v1 ==== 136041+0+0 (secure 0 0 0) 0x7f6cbc0225e0 con 0x7f6cc8103170 2024-03-23T17:51:00.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.859+0000 7f6cc5600700 1 --2- 172.21.15.86:0/1985733722 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f6cb00a1320 0x7f6cb00a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:00.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.860+0000 7f6ce2e32b80 1 Processor -- start 2024-03-23T17:51:00.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.860+0000 7f6cc5600700 1 -- 172.21.15.86:0/1985733722 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f6cbc0ce070 con 0x7f6cc8103170 2024-03-23T17:51:00.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.860+0000 7f6ce2e32b80 1 -- start start 2024-03-23T17:51:00.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.860+0000 7f6cc7e00700 1 --2- 172.21.15.86:0/1985733722 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f6cb00a1320 0x7f6cb00a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:00.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.860+0000 7f6ce2e32b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557315b4e5d0 0x557315b4e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:00.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.860+0000 7f6ce2e32b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557315b4eef0 0x557315b620c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:00.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.860+0000 7f6ce2e32b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557315b62600 0x557315b649f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:00.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.860+0000 7f6ce2e32b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x557315aa7990 con 0x557315b62600 2024-03-23T17:51:00.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.860+0000 7f6ce2e32b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x557315aa5350 con 0x557315b4eef0 2024-03-23T17:51:00.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.860+0000 7f6ce2e32b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x557315aa0c40 con 0x557315b4e5d0 2024-03-23T17:51:00.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.861+0000 7f6ccca00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557315b4e5d0 0x557315b4e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:00.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.861+0000 7f6ccca00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557315b4e5d0 0x557315b4e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:53966/0 (socket says 172.21.15.86:53966) 2024-03-23T17:51:00.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.861+0000 7f6ccca00700 1 -- 172.21.15.86:0/97489665 learned_addr learned my addr 172.21.15.86:0/97489665 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:00.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.861+0000 7f6cc7e00700 1 --2- 172.21.15.86:0/97489665 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557315b4eef0 0x557315b620c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:00.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.861+0000 7f6ccd400700 1 --2- 172.21.15.86:0/97489665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557315b62600 0x557315b649f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:00.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.861+0000 7f6ccca00700 1 -- 172.21.15.86:0/97489665 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557315b4eef0 msgr2=0x557315b620c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:00.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.861+0000 7f6ccca00700 1 --2- 172.21.15.86:0/97489665 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557315b4eef0 0x557315b620c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:00.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.861+0000 7f6ccca00700 1 -- 172.21.15.86:0/97489665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557315b62600 msgr2=0x557315b649f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:00.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.861+0000 7f6ccca00700 1 --2- 172.21.15.86:0/97489665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557315b62600 0x557315b649f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:00.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.861+0000 7f6ccca00700 1 -- 172.21.15.86:0/97489665 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5573159223b0 con 0x557315b4e5d0 2024-03-23T17:51:00.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.861+0000 7f6ccd400700 1 --2- 172.21.15.86:0/97489665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557315b62600 0x557315b649f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:51:00.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.861+0000 7f6cc7e00700 1 --2- 172.21.15.86:0/1985733722 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f6cb00a1320 0x7f6cb00a37e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6cc819a960 tx=0x7f6cb4009260 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:00.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.861+0000 7f6ccca00700 1 --2- 172.21.15.86:0/97489665 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557315b4e5d0 0x557315b4e9b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc09aeb0 tx=0x7f6cbc09af80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ff5591449819fdc server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:00.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.861+0000 7f6cb9c00700 1 -- 172.21.15.86:0/97489665 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cbc09ab10 con 0x557315b4e5d0 2024-03-23T17:51:00.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.861+0000 7f6cb9c00700 1 -- 172.21.15.86:0/97489665 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6cbc0b3070 con 0x557315b4e5d0 2024-03-23T17:51:00.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.861+0000 7f6cb9c00700 1 -- 172.21.15.86:0/97489665 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cbc0a10c0 con 0x557315b4e5d0 2024-03-23T17:51:00.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.862+0000 7f6ce2e32b80 1 -- 172.21.15.86:0/97489665 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557315b4e5d0 msgr2=0x557315b4e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:00.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.862+0000 7f6ce2e32b80 1 --2- 172.21.15.86:0/97489665 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557315b4e5d0 0x557315b4e9b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc09aeb0 tx=0x7f6cbc09af80 comp rx=0 tx=0).stop 2024-03-23T17:51:00.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.862+0000 7f6ce2e32b80 1 -- 172.21.15.86:0/97489665 shutdown_connections 2024-03-23T17:51:00.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.862+0000 7f6ce2e32b80 1 --2- 172.21.15.86:0/97489665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557315b62600 0x557315b649f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:00.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.862+0000 7f6ce2e32b80 1 --2- 172.21.15.86:0/97489665 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557315b4e5d0 0x557315b4e9b0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:00.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.862+0000 7f6ce2e32b80 1 --2- 172.21.15.86:0/97489665 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557315b4eef0 0x557315b620c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:00.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.862+0000 7f6ce2e32b80 1 -- 172.21.15.86:0/97489665 >> 172.21.15.86:0/97489665 conn(0x557315aae6a0 msgr2=0x557315b4aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:00.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.862+0000 7f6ce2e32b80 1 -- 172.21.15.86:0/97489665 shutdown_connections 2024-03-23T17:51:00.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.862+0000 7f6ce2e32b80 1 -- 172.21.15.86:0/97489665 wait complete. 2024-03-23T17:51:00.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.863+0000 7f6ce2e32b80 1 -- start start 2024-03-23T17:51:00.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.863+0000 7f6ce2e32b80 1 Processor -- start 2024-03-23T17:51:00.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.863+0000 7f6ce2e32b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557315b4e5d0 0x557315b5b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:00.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.863+0000 7f6ce2e32b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557315b4eef0 0x557315b5bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:00.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.863+0000 7f6ce2e32b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557315b62600 0x557315b56aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:00.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.863+0000 7f6ce2e32b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x557315a3cd80 con 0x557315b4e5d0 2024-03-23T17:51:00.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.863+0000 7f6ce2e32b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x557315a3ca20 con 0x557315b4eef0 2024-03-23T17:51:00.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.863+0000 7f6ce2e32b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573159246a0 con 0x557315b62600 2024-03-23T17:51:00.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.864+0000 7f6ce2e32b80 10 client.0 _ll_register_callbacks cb 0x557315a46470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:51:00.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.864+0000 7f6ccca00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557315b4e5d0 0x557315b5b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:00.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.864+0000 7f6cc7e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557315b4eef0 0x557315b5bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:00.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.864+0000 7f6ccca00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557315b4e5d0 0x557315b5b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43172/0 (socket says 172.21.15.86:43172) 2024-03-23T17:51:00.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.864+0000 7f6ccca00700 1 -- 172.21.15.86:0/2002663696 learned_addr learned my addr 172.21.15.86:0/2002663696 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:00.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.864+0000 7f6ccd400700 1 --2- 172.21.15.86:0/2002663696 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557315b62600 0x557315b56aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:00.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.864+0000 7f6cc7e00700 1 -- 172.21.15.86:0/2002663696 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557315b62600 msgr2=0x557315b56aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:00.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.864+0000 7f6cc7e00700 1 --2- 172.21.15.86:0/2002663696 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557315b62600 0x557315b56aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:00.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.864+0000 7f6cc7e00700 1 -- 172.21.15.86:0/2002663696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557315b4e5d0 msgr2=0x557315b5b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:00.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.864+0000 7f6cc7e00700 1 --2- 172.21.15.86:0/2002663696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557315b4e5d0 0x557315b5b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:00.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.864+0000 7f6cc7e00700 1 -- 172.21.15.86:0/2002663696 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cc81a6360 con 0x557315b4eef0 2024-03-23T17:51:00.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.864+0000 7f6ccd400700 1 --2- 172.21.15.86:0/2002663696 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557315b62600 0x557315b56aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:51:00.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.864+0000 7f6cc7e00700 1 --2- 172.21.15.86:0/2002663696 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557315b4eef0 0x557315b5bca0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6cc806a3d0 tx=0x7f6cb40047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84a0595f6417c60b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:00.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.865+0000 7f6cae000700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T17:51:00.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.865+0000 7f6cae000700 1 -- 172.21.15.86:0/2002663696 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb401c610 con 0x557315b4eef0 2024-03-23T17:51:00.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.865+0000 7f6cae000700 1 -- 172.21.15.86:0/2002663696 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f6cb401c770 con 0x557315b4eef0 2024-03-23T17:51:00.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.865+0000 7f6ce2e32b80 1 -- 172.21.15.86:0/2002663696 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557315a61380 con 0x557315b4eef0 2024-03-23T17:51:00.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.865+0000 7f6cae000700 1 -- 172.21.15.86:0/2002663696 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6cb401c930 con 0x557315b4eef0 2024-03-23T17:51:00.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.865+0000 7f6ce2e32b80 10 client.25571 fetch_fsmap learned FSMap version 28 2024-03-23T17:51:00.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.865+0000 7f6ce2e32b80 1 -- 172.21.15.86:0/2002663696 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5573158ceb60 con 0x557315b4eef0 2024-03-23T17:51:00.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.866+0000 7f6cae000700 1 -- 172.21.15.86:0/2002663696 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557315a61380 con 0x557315b4eef0 2024-03-23T17:51:00.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.866+0000 7f6cae000700 1 -- 172.21.15.86:0/2002663696 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 28) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6cb4025ab0 con 0x557315b4eef0 2024-03-23T17:51:00.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.866+0000 7f6ce2e32b80 10 client.25571 fetch_fsmap finished waiting for FSMap version 28 2024-03-23T17:51:00.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.866+0000 7f6ce2e32b80 10 client.25571 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.d 2024-03-23T17:51:00.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.866+0000 7f6ce2e32b80 20 client.25571 populate_metadata read hostname 'smithi086' 2024-03-23T17:51:00.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.866+0000 7f6ce2e32b80 1 --2- 172.21.15.86:0/2002663696 >> [v2:172.21.15.86:6836/2933881328,v1:172.21.15.86:6837/2933881328] conn(0x557315b7a540 0x557315b7c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:00.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.866+0000 7f6ce2e32b80 4 client.25571 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:51:00.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.866+0000 7f6ce2e32b80 1 -- 172.21.15.86:0/2002663696 --> [v2:172.21.15.86:6836/2933881328,v1:172.21.15.86:6837/2933881328] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557315a99700 con 0x557315b7a540 2024-03-23T17:51:00.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.866+0000 7f6ccca00700 1 --2- 172.21.15.86:0/2002663696 >> [v2:172.21.15.86:6836/2933881328,v1:172.21.15.86:6837/2933881328] conn(0x557315b7a540 0x557315b7c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:00.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.867+0000 7f6ccca00700 1 --2- 172.21.15.86:0/2002663696 >> [v2:172.21.15.86:6836/2933881328,v1:172.21.15.86:6837/2933881328] conn(0x557315b7a540 0x557315b7c960 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=290b271f123ed95c server_cookie=82698ef3b6ac3e33 in_seq=0 out_seq=0 2024-03-23T17:51:00.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.867+0000 7f6cae000700 10 client.25571 ms_handle_connect on v2:172.21.15.86:6836/2933881328 2024-03-23T17:51:00.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.867+0000 7f6cae000700 1 -- 172.21.15.86:0/2002663696 <== mds.3 v2:172.21.15.86:6836/2933881328 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x557315a99700 con 0x557315b7a540 2024-03-23T17:51:00.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.867+0000 7f6cae000700 10 client.25571 handle_command_reply: tid=0 2024-03-23T17:51:00.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.867+0000 7f6ce2e32b80 1 client.25571 shutdown 2024-03-23T17:51:00.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.867+0000 7f6ce2e32b80 1 -- 172.21.15.86:0/2002663696 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557315b4eef0 msgr2=0x557315b5bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:00.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.867+0000 7f6ce2e32b80 1 --2- 172.21.15.86:0/2002663696 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557315b4eef0 0x557315b5bca0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6cc806a3d0 tx=0x7f6cb40047c0 comp rx=0 tx=0).stop 2024-03-23T17:51:00.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.868+0000 7f6ce2e32b80 1 -- 172.21.15.86:0/2002663696 shutdown_connections 2024-03-23T17:51:00.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.868+0000 7f6ce2e32b80 1 --2- 172.21.15.86:0/2002663696 >> [v2:172.21.15.86:6836/2933881328,v1:172.21.15.86:6837/2933881328] conn(0x557315b7a540 0x557315b7c960 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:00.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.868+0000 7f6ce2e32b80 1 --2- 172.21.15.86:0/2002663696 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557315b62600 0x557315b56aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:00.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.868+0000 7f6ce2e32b80 1 --2- 172.21.15.86:0/2002663696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557315b4e5d0 0x557315b5b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:00.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.868+0000 7f6ce2e32b80 1 --2- 172.21.15.86:0/2002663696 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557315b4eef0 0x557315b5bca0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:00.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.868+0000 7f6cae000700 0 client.25571 ms_handle_reset on v2:172.21.15.86:6836/2933881328 2024-03-23T17:51:00.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.868+0000 7f6ce2e32b80 1 -- 172.21.15.86:0/2002663696 >> 172.21.15.86:0/2002663696 conn(0x557315aae6a0 msgr2=0x557315b506c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:00.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.868+0000 7f6ce2e32b80 1 -- 172.21.15.86:0/2002663696 shutdown_connections 2024-03-23T17:51:00.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.868+0000 7f6ce2e32b80 1 -- 172.21.15.86:0/2002663696 wait complete. 2024-03-23T17:51:00.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.868+0000 7f6ce2e32b80 20 client.25571 trim_cache size 0 max 16384 2024-03-23T17:51:00.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.881+0000 7f6ce2e32b80 1 Processor -- start 2024-03-23T17:51:00.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.881+0000 7f6ce2e32b80 1 -- start start 2024-03-23T17:51:00.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.881+0000 7f6ce2e32b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557315b4e5d0 0x557315b812c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:00.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.881+0000 7f6ce2e32b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557315b4eef0 0x557315b55370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:00.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.881+0000 7f6ce2e32b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557315b62600 0x557315b558b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:00.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.881+0000 7f6ce2e32b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x557315b580c0 con 0x557315b4eef0 2024-03-23T17:51:00.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.881+0000 7f6ce2e32b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x557315b4c370 con 0x557315b4e5d0 2024-03-23T17:51:00.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.881+0000 7f6ce2e32b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x557315b4f850 con 0x557315b62600 2024-03-23T17:51:00.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.882+0000 7f6cc7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557315b4eef0 0x557315b55370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:00.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.882+0000 7f6cc7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557315b4eef0 0x557315b55370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43184/0 (socket says 172.21.15.86:43184) 2024-03-23T17:51:00.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.882+0000 7f6cc7e00700 1 -- 172.21.15.86:0/3764894108 learned_addr learned my addr 172.21.15.86:0/3764894108 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:00.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.882+0000 7f6ccca00700 1 --2- 172.21.15.86:0/3764894108 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557315b4e5d0 0x557315b812c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:00.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.882+0000 7f6ccd400700 1 --2- 172.21.15.86:0/3764894108 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557315b62600 0x557315b558b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:00.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.882+0000 7f6cc7e00700 1 -- 172.21.15.86:0/3764894108 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557315b62600 msgr2=0x557315b558b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:00.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.882+0000 7f6cc7e00700 1 --2- 172.21.15.86:0/3764894108 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557315b62600 0x557315b558b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:00.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.882+0000 7f6cc7e00700 1 -- 172.21.15.86:0/3764894108 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557315b4e5d0 msgr2=0x557315b812c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:00.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.882+0000 7f6cc7e00700 1 --2- 172.21.15.86:0/3764894108 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557315b4e5d0 0x557315b812c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:00.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.882+0000 7f6cc7e00700 1 -- 172.21.15.86:0/3764894108 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557315a66ea0 con 0x557315b4eef0 2024-03-23T17:51:00.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.882+0000 7f6ccd400700 1 --2- 172.21.15.86:0/3764894108 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557315b62600 0x557315b558b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:51:00.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.882+0000 7f6cc7e00700 1 --2- 172.21.15.86:0/3764894108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557315b4eef0 0x557315b55370 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f6cb400f3e0 tx=0x7f6cb40092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24bce3b547201a4d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:00.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.883+0000 7f6cba600700 1 -- 172.21.15.86:0/3764894108 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb4003cf0 con 0x557315b4eef0 2024-03-23T17:51:00.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.883+0000 7f6cba600700 1 -- 172.21.15.86:0/3764894108 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6cb400f460 con 0x557315b4eef0 2024-03-23T17:51:00.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.883+0000 7f6cba600700 1 -- 172.21.15.86:0/3764894108 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb4025a70 con 0x557315b4eef0 2024-03-23T17:51:00.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.883+0000 7f6ce2e32b80 1 -- 172.21.15.86:0/3764894108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557315b4eef0 msgr2=0x557315b55370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:00.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.883+0000 7f6ce2e32b80 1 --2- 172.21.15.86:0/3764894108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557315b4eef0 0x557315b55370 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f6cb400f3e0 tx=0x7f6cb40092d0 comp rx=0 tx=0).stop 2024-03-23T17:51:00.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.883+0000 7f6ce2e32b80 1 -- 172.21.15.86:0/3764894108 shutdown_connections 2024-03-23T17:51:00.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.883+0000 7f6ce2e32b80 1 --2- 172.21.15.86:0/3764894108 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557315b62600 0x557315b558b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:00.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.883+0000 7f6ce2e32b80 1 --2- 172.21.15.86:0/3764894108 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557315b4e5d0 0x557315b812c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:00.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.883+0000 7f6ce2e32b80 1 --2- 172.21.15.86:0/3764894108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557315b4eef0 0x557315b55370 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:00.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.883+0000 7f6ce2e32b80 1 -- 172.21.15.86:0/3764894108 >> 172.21.15.86:0/3764894108 conn(0x557315aae6a0 msgr2=0x557315b43f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:00.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.884+0000 7f6ce2e32b80 1 -- 172.21.15.86:0/3764894108 shutdown_connections 2024-03-23T17:51:00.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.884+0000 7f6ce2e32b80 1 -- 172.21.15.86:0/3764894108 wait complete. 2024-03-23T17:51:00.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.884+0000 7f6ce2e32b80 1 -- start start 2024-03-23T17:51:00.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.884+0000 7f6ce2e32b80 1 Processor -- start 2024-03-23T17:51:00.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.885+0000 7f6ce2e32b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557315b4e5d0 0x557315b61200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:00.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.885+0000 7f6ce2e32b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557315b4eef0 0x557315b5f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:00.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.885+0000 7f6ce2e32b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557315b62600 0x557315b5fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:00.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.885+0000 7f6ce2e32b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x557315b603d0 con 0x557315b62600 2024-03-23T17:51:00.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.885+0000 7f6ce2e32b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x557315b60540 con 0x557315b4eef0 2024-03-23T17:51:00.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.885+0000 7f6ce2e32b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x557315bcc070 con 0x557315b4e5d0 2024-03-23T17:51:00.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.885+0000 7f6ce2e32b80 10 client.0 _ll_register_callbacks cb 0x557315a46470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:51:00.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.885+0000 7f6cc7e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557315b4eef0 0x557315b5f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:00.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.885+0000 7f6cc7e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557315b4eef0 0x557315b5f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:58228/0 (socket says 172.21.15.86:58228) 2024-03-23T17:51:00.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.885+0000 7f6ccd400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557315b62600 0x557315b5fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:00.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.885+0000 7f6cc7e00700 1 -- 172.21.15.86:0/1990819810 learned_addr learned my addr 172.21.15.86:0/1990819810 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:00.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.885+0000 7f6ccca00700 1 --2- 172.21.15.86:0/1990819810 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557315b4e5d0 0x557315b61200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:00.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.885+0000 7f6cc7e00700 1 -- 172.21.15.86:0/1990819810 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557315b4e5d0 msgr2=0x557315b61200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:00.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.885+0000 7f6cc7e00700 1 --2- 172.21.15.86:0/1990819810 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557315b4e5d0 0x557315b61200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:00.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.885+0000 7f6cc7e00700 1 -- 172.21.15.86:0/1990819810 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557315b62600 msgr2=0x557315b5fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:00.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.885+0000 7f6cc7e00700 1 --2- 172.21.15.86:0/1990819810 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557315b62600 0x557315b5fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:00.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.885+0000 7f6cc7e00700 1 -- 172.21.15.86:0/1990819810 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5573158ceb60 con 0x557315b4eef0 2024-03-23T17:51:00.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.886+0000 7f6cc7e00700 1 --2- 172.21.15.86:0/1990819810 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557315b4eef0 0x557315b5f740 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6cb400f430 tx=0x7f6cb4025d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3ca2b9f2cc662b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:00.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.886+0000 7f6cae000700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T17:51:00.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.886+0000 7f6cae000700 1 -- 172.21.15.86:0/1990819810 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb4003cf0 con 0x557315b4eef0 2024-03-23T17:51:00.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.886+0000 7f6cae000700 1 -- 172.21.15.86:0/1990819810 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f6cb4005240 con 0x557315b4eef0 2024-03-23T17:51:00.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.886+0000 7f6ce2e32b80 1 -- 172.21.15.86:0/1990819810 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6cbc0b3070 con 0x557315b4eef0 2024-03-23T17:51:00.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.886+0000 7f6cae000700 1 -- 172.21.15.86:0/1990819810 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6cb4005400 con 0x557315b4eef0 2024-03-23T17:51:00.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.886+0000 7f6ce2e32b80 10 client.25577 fetch_fsmap learned FSMap version 28 2024-03-23T17:51:00.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.886+0000 7f6ce2e32b80 1 -- 172.21.15.86:0/1990819810 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x557315926cd0 con 0x557315b4eef0 2024-03-23T17:51:00.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.886+0000 7f6cae000700 1 -- 172.21.15.86:0/1990819810 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6cbc0b3070 con 0x557315b4eef0 2024-03-23T17:51:00.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.886+0000 7f6cae000700 1 -- 172.21.15.86:0/1990819810 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 28) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6cb400f830 con 0x557315b4eef0 2024-03-23T17:51:00.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.886+0000 7f6ce2e32b80 10 client.25577 fetch_fsmap finished waiting for FSMap version 28 2024-03-23T17:51:00.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.887+0000 7f6ce2e32b80 10 client.25577 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.d 2024-03-23T17:51:00.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.887+0000 7f6ce2e32b80 20 client.25577 populate_metadata read hostname 'smithi086' 2024-03-23T17:51:00.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.887+0000 7f6ce2e32b80 1 --2- 172.21.15.86:0/1990819810 >> [v2:172.21.15.86:6836/2933881328,v1:172.21.15.86:6837/2933881328] conn(0x557315bcd3e0 0x557315bcf800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:00.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.887+0000 7f6ce2e32b80 4 client.25577 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-23T17:51:00.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.887+0000 7f6ce2e32b80 1 -- 172.21.15.86:0/1990819810 --> [v2:172.21.15.86:6836/2933881328,v1:172.21.15.86:6837/2933881328] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5573158efcf0 con 0x557315bcd3e0 2024-03-23T17:51:00.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.887+0000 7f6ccca00700 1 --2- 172.21.15.86:0/1990819810 >> [v2:172.21.15.86:6836/2933881328,v1:172.21.15.86:6837/2933881328] conn(0x557315bcd3e0 0x557315bcf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:00.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.887+0000 7f6cae000700 10 client.25577 ms_handle_connect on v2:172.21.15.86:6836/2933881328 2024-03-23T17:51:00.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.887+0000 7f6ccca00700 1 --2- 172.21.15.86:0/1990819810 >> [v2:172.21.15.86:6836/2933881328,v1:172.21.15.86:6837/2933881328] conn(0x557315bcd3e0 0x557315bcf800 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8a57bece64032ca4 server_cookie=ad8ee5bc4ae09d35 in_seq=0 out_seq=0 2024-03-23T17:51:00.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.888+0000 7f6cae000700 1 -- 172.21.15.86:0/1990819810 <== mds.3 v2:172.21.15.86:6836/2933881328 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5573158efcf0 con 0x557315bcd3e0 2024-03-23T17:51:00.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.888+0000 7f6cae000700 10 client.25577 handle_command_reply: tid=0 2024-03-23T17:51:00.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.888+0000 7f6ce2e32b80 1 client.25577 shutdown 2024-03-23T17:51:00.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.888+0000 7f6ce2e32b80 1 -- 172.21.15.86:0/1990819810 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557315b4eef0 msgr2=0x557315b5f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:00.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.888+0000 7f6ce2e32b80 1 --2- 172.21.15.86:0/1990819810 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557315b4eef0 0x557315b5f740 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6cb400f430 tx=0x7f6cb4025d10 comp rx=0 tx=0).stop 2024-03-23T17:51:00.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.888+0000 7f6ce2e32b80 1 -- 172.21.15.86:0/1990819810 shutdown_connections 2024-03-23T17:51:00.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.888+0000 7f6ce2e32b80 1 --2- 172.21.15.86:0/1990819810 >> [v2:172.21.15.86:6836/2933881328,v1:172.21.15.86:6837/2933881328] conn(0x557315bcd3e0 0x557315bcf800 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:00.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.888+0000 7f6ce2e32b80 1 --2- 172.21.15.86:0/1990819810 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557315b62600 0x557315b5fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:00.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.888+0000 7f6ce2e32b80 1 --2- 172.21.15.86:0/1990819810 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557315b4e5d0 0x557315b61200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:00.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.888+0000 7f6ce2e32b80 1 --2- 172.21.15.86:0/1990819810 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557315b4eef0 0x557315b5f740 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:00.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.888+0000 7f6ce2e32b80 1 -- 172.21.15.86:0/1990819810 >> 172.21.15.86:0/1990819810 conn(0x557315aae6a0 msgr2=0x557315b4a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:00.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.888+0000 7f6cae000700 0 client.25577 ms_handle_reset on v2:172.21.15.86:6836/2933881328 2024-03-23T17:51:00.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.888+0000 7f6ce2e32b80 1 -- 172.21.15.86:0/1990819810 shutdown_connections 2024-03-23T17:51:00.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.888+0000 7f6ce2e32b80 1 -- 172.21.15.86:0/1990819810 wait complete. 2024-03-23T17:51:00.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.888+0000 7f6ce2e32b80 20 client.25577 trim_cache size 0 max 16384 2024-03-23T17:51:00.892 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-03-23T17:51:00.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.889+0000 7f6ccf400700 1 -- 172.21.15.86:0/1985733722 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f6cb00a1320 msgr2=0x7f6cb00a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:00.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.889+0000 7f6ccf400700 1 --2- 172.21.15.86:0/1985733722 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f6cb00a1320 0x7f6cb00a37e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6cc819a960 tx=0x7f6cb4009260 comp rx=0 tx=0).stop 2024-03-23T17:51:00.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.889+0000 7f6ccf400700 1 -- 172.21.15.86:0/1985733722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6cc8103170 msgr2=0x7f6cc8199460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:00.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.889+0000 7f6ccf400700 1 --2- 172.21.15.86:0/1985733722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6cc8103170 0x7f6cc8199460 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc000c00 tx=0x7f6cbc005740 comp rx=0 tx=0).stop 2024-03-23T17:51:00.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.889+0000 7f6ccf400700 1 -- 172.21.15.86:0/1985733722 shutdown_connections 2024-03-23T17:51:00.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.889+0000 7f6ccf400700 1 --2- 172.21.15.86:0/1985733722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6cc819dcf0 0x7f6cc819e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:00.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.889+0000 7f6ccf400700 1 --2- 172.21.15.86:0/1985733722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6cc8103170 0x7f6cc8199460 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:00.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.889+0000 7f6ccf400700 1 --2- 172.21.15.86:0/1985733722 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f6cb00a1320 0x7f6cb00a37e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:00.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.889+0000 7f6ccf400700 1 --2- 172.21.15.86:0/1985733722 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6cc8108290 0x7f6cc81999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:00.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.889+0000 7f6ccf400700 1 -- 172.21.15.86:0/1985733722 >> 172.21.15.86:0/1985733722 conn(0x7f6cc80fe990 msgr2=0x7f6cc80ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:00.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.889+0000 7f6ccf400700 1 -- 172.21.15.86:0/1985733722 shutdown_connections 2024-03-23T17:51:00.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:00.889+0000 7f6ccf400700 1 -- 172.21.15.86:0/1985733722 wait complete. 2024-03-23T17:51:00.910 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:51:00 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/3922679078' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T17:51:00.912 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-23T17:51:00.912 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-23T17:51:01.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.113+0000 7f7692c00700 1 -- 172.21.15.86:0/206441615 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f768c073180 msgr2=0x7f768c073560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:01.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.113+0000 7f7692c00700 1 --2- 172.21.15.86:0/206441615 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f768c073180 0x7f768c073560 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f767c009990 tx=0x7f767c009ca0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.113+0000 7f7692c00700 1 -- 172.21.15.86:0/206441615 shutdown_connections 2024-03-23T17:51:01.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.113+0000 7f7692c00700 1 --2- 172.21.15.86:0/206441615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f768c10d580 0x7f768c10f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.113+0000 7f7692c00700 1 --2- 172.21.15.86:0/206441615 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f768c073180 0x7f768c073560 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.113+0000 7f7692c00700 1 --2- 172.21.15.86:0/206441615 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f768c073aa0 0x7f768c10d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.113+0000 7f7692c00700 1 -- 172.21.15.86:0/206441615 >> 172.21.15.86:0/206441615 conn(0x7f768c0fc7f0 msgr2=0x7f768c0fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:01.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.113+0000 7f7692c00700 1 -- 172.21.15.86:0/206441615 shutdown_connections 2024-03-23T17:51:01.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.113+0000 7f7692c00700 1 -- 172.21.15.86:0/206441615 wait complete. 2024-03-23T17:51:01.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.115+0000 7f7692c00700 1 Processor -- start 2024-03-23T17:51:01.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.115+0000 7f7692c00700 1 -- start start 2024-03-23T17:51:01.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.115+0000 7f7692c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f768c073180 0x7f768c199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:01.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.115+0000 7f7692c00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f768c073aa0 0x7f768c199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:01.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.115+0000 7f7692c00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f768c10d580 0x7f768c19dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:01.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.115+0000 7f7692c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f768c19e340 con 0x7f768c073180 2024-03-23T17:51:01.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.115+0000 7f7692c00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f768c19e4b0 con 0x7f768c073aa0 2024-03-23T17:51:01.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.115+0000 7f7692c00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f768c19e670 con 0x7f768c10d580 2024-03-23T17:51:01.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.116+0000 7f768be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f768c073180 0x7f768c199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:01.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.116+0000 7f768be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f768c073180 0x7f768c199420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43218/0 (socket says 172.21.15.86:43218) 2024-03-23T17:51:01.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.116+0000 7f768be00700 1 -- 172.21.15.86:0/1532988102 learned_addr learned my addr 172.21.15.86:0/1532988102 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:01.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.116+0000 7f768b400700 1 --2- 172.21.15.86:0/1532988102 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f768c073aa0 0x7f768c199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:01.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.116+0000 7f7690c00700 1 --2- 172.21.15.86:0/1532988102 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f768c10d580 0x7f768c19dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:01.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.116+0000 7f768be00700 1 -- 172.21.15.86:0/1532988102 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f768c10d580 msgr2=0x7f768c19dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:01.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.116+0000 7f768be00700 1 --2- 172.21.15.86:0/1532988102 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f768c10d580 0x7f768c19dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.116+0000 7f768be00700 1 -- 172.21.15.86:0/1532988102 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f768c073aa0 msgr2=0x7f768c199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:01.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.116+0000 7f768be00700 1 --2- 172.21.15.86:0/1532988102 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f768c073aa0 0x7f768c199960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.116+0000 7f768be00700 1 -- 172.21.15.86:0/1532988102 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7678009520 con 0x7f768c073180 2024-03-23T17:51:01.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.116+0000 7f768b400700 1 --2- 172.21.15.86:0/1532988102 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f768c073aa0 0x7f768c199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:51:01.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.117+0000 7f768be00700 1 --2- 172.21.15.86:0/1532988102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f768c073180 0x7f768c199420 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f767c005910 tx=0x7f767c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:01.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.117+0000 7f7688c00700 1 -- 172.21.15.86:0/1532988102 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f767c01d070 con 0x7f768c073180 2024-03-23T17:51:01.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.117+0000 7f7688c00700 1 -- 172.21.15.86:0/1532988102 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f767c022470 con 0x7f768c073180 2024-03-23T17:51:01.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.117+0000 7f7692c00700 1 -- 172.21.15.86:0/1532988102 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f767c0095f0 con 0x7f768c073180 2024-03-23T17:51:01.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.117+0000 7f7688c00700 1 -- 172.21.15.86:0/1532988102 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f767c017650 con 0x7f768c073180 2024-03-23T17:51:01.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.117+0000 7f7692c00700 1 -- 172.21.15.86:0/1532988102 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f768c1a5fb0 con 0x7f768c073180 2024-03-23T17:51:01.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.121+0000 7f7688c00700 1 -- 172.21.15.86:0/1532988102 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) v1 ==== 136098+0+0 (secure 0 0 0) 0x7f767c0225e0 con 0x7f768c073180 2024-03-23T17:51:01.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.123+0000 7f7688c00700 1 --2- 172.21.15.86:0/1532988102 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f76700a13b0 0x7f76700a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:01.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.123+0000 7f7688c00700 1 -- 172.21.15.86:0/1532988102 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f767c0ce1d0 con 0x7f768c073180 2024-03-23T17:51:01.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.123+0000 7f76a664eb80 1 Processor -- start 2024-03-23T17:51:01.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.123+0000 7f768b400700 1 --2- 172.21.15.86:0/1532988102 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f76700a13b0 0x7f76700a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:01.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.123+0000 7f76a664eb80 1 -- start start 2024-03-23T17:51:01.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.124+0000 7f76a664eb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56301443b5d0 0x56301443b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:01.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.124+0000 7f76a664eb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56301443bef0 0x5630144468b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:01.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.124+0000 7f76a664eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563014446df0 0x5630144491e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:01.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.124+0000 7f76a664eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x563014394990 con 0x563014446df0 2024-03-23T17:51:01.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.124+0000 7f76a664eb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x563014392350 con 0x56301443bef0 2024-03-23T17:51:01.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.124+0000 7f76a664eb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x56301438dc40 con 0x56301443b5d0 2024-03-23T17:51:01.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.124+0000 7f768be00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56301443b5d0 0x56301443b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:01.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.124+0000 7f768be00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56301443b5d0 0x56301443b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:54014/0 (socket says 172.21.15.86:54014) 2024-03-23T17:51:01.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.124+0000 7f768be00700 1 -- 172.21.15.86:0/3254771656 learned_addr learned my addr 172.21.15.86:0/3254771656 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:01.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.124+0000 7f7690c00700 1 --2- 172.21.15.86:0/3254771656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563014446df0 0x5630144491e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:01.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.124+0000 7f768b400700 1 --2- 172.21.15.86:0/3254771656 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56301443bef0 0x5630144468b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:01.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.125+0000 7f768b400700 1 --2- 172.21.15.86:0/1532988102 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f76700a13b0 0x7f76700a3870 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7678009fd0 tx=0x7f7678009260 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:01.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.125+0000 7f768be00700 1 -- 172.21.15.86:0/3254771656 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56301443bef0 msgr2=0x5630144468b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:01.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.125+0000 7f768be00700 1 --2- 172.21.15.86:0/3254771656 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56301443bef0 0x5630144468b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.125+0000 7f768be00700 1 -- 172.21.15.86:0/3254771656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563014446df0 msgr2=0x5630144491e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:01.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.125+0000 7f768be00700 1 --2- 172.21.15.86:0/3254771656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563014446df0 0x5630144491e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.125+0000 7f768be00700 1 -- 172.21.15.86:0/3254771656 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56301420f3b0 con 0x56301443b5d0 2024-03-23T17:51:01.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.125+0000 7f768be00700 1 --2- 172.21.15.86:0/3254771656 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56301443b5d0 0x56301443b9b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f767c006010 tx=0x7f767c022c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a32aae4d018a7dd1 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:01.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.125+0000 7f7681200700 1 -- 172.21.15.86:0/3254771656 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f767c09fea0 con 0x56301443b5d0 2024-03-23T17:51:01.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.125+0000 7f7681200700 1 -- 172.21.15.86:0/3254771656 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f767c09aab0 con 0x56301443b5d0 2024-03-23T17:51:01.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.125+0000 7f7681200700 1 -- 172.21.15.86:0/3254771656 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f767c0a3540 con 0x56301443b5d0 2024-03-23T17:51:01.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.126+0000 7f76a664eb80 1 -- 172.21.15.86:0/3254771656 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56301443b5d0 msgr2=0x56301443b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:01.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.126+0000 7f76a664eb80 1 --2- 172.21.15.86:0/3254771656 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56301443b5d0 0x56301443b9b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f767c006010 tx=0x7f767c022c90 comp rx=0 tx=0).stop 2024-03-23T17:51:01.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.126+0000 7f76a664eb80 1 -- 172.21.15.86:0/3254771656 shutdown_connections 2024-03-23T17:51:01.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.126+0000 7f76a664eb80 1 --2- 172.21.15.86:0/3254771656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563014446df0 0x5630144491e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.126+0000 7f76a664eb80 1 --2- 172.21.15.86:0/3254771656 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56301443b5d0 0x56301443b9b0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.126+0000 7f76a664eb80 1 --2- 172.21.15.86:0/3254771656 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56301443bef0 0x5630144468b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.126+0000 7f76a664eb80 1 -- 172.21.15.86:0/3254771656 >> 172.21.15.86:0/3254771656 conn(0x56301439b6a0 msgr2=0x563014437a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:01.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.126+0000 7f76a664eb80 1 -- 172.21.15.86:0/3254771656 shutdown_connections 2024-03-23T17:51:01.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.127+0000 7f76a664eb80 1 -- 172.21.15.86:0/3254771656 wait complete. 2024-03-23T17:51:01.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.127+0000 7f76a664eb80 1 -- start start 2024-03-23T17:51:01.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.127+0000 7f76a664eb80 1 Processor -- start 2024-03-23T17:51:01.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.128+0000 7f76a664eb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56301443b5d0 0x563014446740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:01.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.128+0000 7f76a664eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56301443bef0 0x563014443a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:01.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.128+0000 7f76a664eb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563014446df0 0x563014444060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:01.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.128+0000 7f76a664eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x563014329d80 con 0x56301443bef0 2024-03-23T17:51:01.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.128+0000 7f76a664eb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x563014329a20 con 0x563014446df0 2024-03-23T17:51:01.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.128+0000 7f76a664eb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630142116a0 con 0x56301443b5d0 2024-03-23T17:51:01.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.128+0000 7f76a664eb80 10 client.0 _ll_register_callbacks cb 0x563014333470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:51:01.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.128+0000 7f768b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56301443bef0 0x563014443a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:01.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.129+0000 7f768be00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56301443b5d0 0x563014446740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:01.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.129+0000 7f768be00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56301443b5d0 0x563014446740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:54022/0 (socket says 172.21.15.86:54022) 2024-03-23T17:51:01.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.129+0000 7f768be00700 1 -- 172.21.15.86:0/3775918950 learned_addr learned my addr 172.21.15.86:0/3775918950 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:01.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.129+0000 7f7690c00700 1 --2- 172.21.15.86:0/3775918950 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563014446df0 0x563014444060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:01.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.129+0000 7f768be00700 1 -- 172.21.15.86:0/3775918950 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563014446df0 msgr2=0x563014444060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:01.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.129+0000 7f768be00700 1 --2- 172.21.15.86:0/3775918950 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563014446df0 0x563014444060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.129+0000 7f768be00700 1 -- 172.21.15.86:0/3775918950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56301443bef0 msgr2=0x563014443a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:01.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.129+0000 7f768be00700 1 --2- 172.21.15.86:0/3775918950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56301443bef0 0x563014443a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.129+0000 7f768be00700 1 -- 172.21.15.86:0/3775918950 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f768c1a5fb0 con 0x56301443b5d0 2024-03-23T17:51:01.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.129+0000 7f768b400700 1 --2- 172.21.15.86:0/3775918950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56301443bef0 0x563014443a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:51:01.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.129+0000 7f768be00700 1 --2- 172.21.15.86:0/3775918950 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56301443b5d0 0x563014446740 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f767c097bc0 tx=0x7f767c022af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67693c4abe473f5a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:01.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.129+0000 7f7675600700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-03-23T17:51:01.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.129+0000 7f7675600700 1 -- 172.21.15.86:0/3775918950 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f767c09fea0 con 0x56301443b5d0 2024-03-23T17:51:01.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.130+0000 7f76a664eb80 1 -- 172.21.15.86:0/3775918950 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56301434e380 con 0x56301443b5d0 2024-03-23T17:51:01.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.130+0000 7f7675600700 1 -- 172.21.15.86:0/3775918950 <== mon.1 v2:172.21.15.152:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f767c09b850 con 0x56301443b5d0 2024-03-23T17:51:01.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.130+0000 7f7675600700 1 -- 172.21.15.86:0/3775918950 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f767c09ef80 con 0x56301443b5d0 2024-03-23T17:51:01.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.130+0000 7f76a664eb80 10 client.25588 fetch_fsmap learned FSMap version 28 2024-03-23T17:51:01.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.130+0000 7f76a664eb80 1 -- 172.21.15.86:0/3775918950 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5630141bbb60 con 0x56301443b5d0 2024-03-23T17:51:01.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.130+0000 7f7675600700 1 -- 172.21.15.86:0/3775918950 <== mon.1 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56301434e380 con 0x56301443b5d0 2024-03-23T17:51:01.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.130+0000 7f7675600700 1 -- 172.21.15.86:0/3775918950 <== mon.1 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f767c0a30b0 con 0x56301443b5d0 2024-03-23T17:51:01.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.130+0000 7f76a664eb80 10 client.25588 fetch_fsmap finished waiting for FSMap version 28 2024-03-23T17:51:01.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.130+0000 7f76a664eb80 10 client.25588 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-23T17:51:01.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.130+0000 7f76a664eb80 20 client.25588 populate_metadata read hostname 'smithi086' 2024-03-23T17:51:01.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.130+0000 7f76a664eb80 1 --2- 172.21.15.86:0/3775918950 >> [v2:172.21.15.92:6836/3967837948,v1:172.21.15.92:6837/3967837948] conn(0x563014456170 0x563014458560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:01.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.130+0000 7f76a664eb80 4 client.25588 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:51:01.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.130+0000 7f76a664eb80 1 -- 172.21.15.86:0/3775918950 --> [v2:172.21.15.92:6836/3967837948,v1:172.21.15.92:6837/3967837948] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563014386700 con 0x563014456170 2024-03-23T17:51:01.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.131+0000 7f768b400700 1 --2- 172.21.15.86:0/3775918950 >> [v2:172.21.15.92:6836/3967837948,v1:172.21.15.92:6837/3967837948] conn(0x563014456170 0x563014458560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:01.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.131+0000 7f768b400700 1 --2- 172.21.15.86:0/3775918950 >> [v2:172.21.15.92:6836/3967837948,v1:172.21.15.92:6837/3967837948] conn(0x563014456170 0x563014458560 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d27167ebdcf93692 server_cookie=95452f5a8d3d6451 in_seq=0 out_seq=0 2024-03-23T17:51:01.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.131+0000 7f7675600700 10 client.25588 ms_handle_connect on v2:172.21.15.92:6836/3967837948 2024-03-23T17:51:01.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.132+0000 7f7675600700 1 -- 172.21.15.86:0/3775918950 <== mds.2 v2:172.21.15.92:6836/3967837948 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x563014386700 con 0x563014456170 2024-03-23T17:51:01.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.132+0000 7f7675600700 10 client.25588 handle_command_reply: tid=0 2024-03-23T17:51:01.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.132+0000 7f76a664eb80 1 client.25588 shutdown 2024-03-23T17:51:01.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.132+0000 7f76a664eb80 1 -- 172.21.15.86:0/3775918950 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56301443b5d0 msgr2=0x563014446740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:01.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.132+0000 7f76a664eb80 1 --2- 172.21.15.86:0/3775918950 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56301443b5d0 0x563014446740 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f767c097bc0 tx=0x7f767c022af0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.132+0000 7f76a664eb80 1 -- 172.21.15.86:0/3775918950 shutdown_connections 2024-03-23T17:51:01.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.132+0000 7f76a664eb80 1 --2- 172.21.15.86:0/3775918950 >> [v2:172.21.15.92:6836/3967837948,v1:172.21.15.92:6837/3967837948] conn(0x563014456170 0x563014458560 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.132+0000 7f76a664eb80 1 --2- 172.21.15.86:0/3775918950 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563014446df0 0x563014444060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.132+0000 7f7675600700 0 client.25588 ms_handle_reset on v2:172.21.15.92:6836/3967837948 2024-03-23T17:51:01.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.132+0000 7f76a664eb80 1 --2- 172.21.15.86:0/3775918950 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56301443b5d0 0x563014446740 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.132+0000 7f76a664eb80 1 --2- 172.21.15.86:0/3775918950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56301443bef0 0x563014443a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.132+0000 7f76a664eb80 1 -- 172.21.15.86:0/3775918950 >> 172.21.15.86:0/3775918950 conn(0x56301439b6a0 msgr2=0x56301443d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:01.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.133+0000 7f76a664eb80 1 -- 172.21.15.86:0/3775918950 shutdown_connections 2024-03-23T17:51:01.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.133+0000 7f76a664eb80 1 -- 172.21.15.86:0/3775918950 wait complete. 2024-03-23T17:51:01.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.133+0000 7f76a664eb80 20 client.25588 trim_cache size 0 max 16384 2024-03-23T17:51:01.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.146+0000 7f76a664eb80 1 Processor -- start 2024-03-23T17:51:01.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.146+0000 7f76a664eb80 1 -- start start 2024-03-23T17:51:01.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.146+0000 7f76a664eb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56301443b5d0 0x563014458560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:01.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.146+0000 7f76a664eb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56301443bef0 0x56301445cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:01.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.146+0000 7f76a664eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563014446df0 0x56301444d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:01.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.146+0000 7f76a664eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56301443cf80 con 0x563014446df0 2024-03-23T17:51:01.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.146+0000 7f76a664eb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x56301443c300 con 0x56301443b5d0 2024-03-23T17:51:01.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.146+0000 7f76a664eb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x563014441fd0 con 0x56301443bef0 2024-03-23T17:51:01.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.147+0000 7f768be00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56301443b5d0 0x563014458560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:01.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.147+0000 7f768be00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56301443b5d0 0x563014458560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:58276/0 (socket says 172.21.15.86:58276) 2024-03-23T17:51:01.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.147+0000 7f768be00700 1 -- 172.21.15.86:0/2771439054 learned_addr learned my addr 172.21.15.86:0/2771439054 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:01.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.147+0000 7f7690c00700 1 --2- 172.21.15.86:0/2771439054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563014446df0 0x56301444d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:01.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.147+0000 7f768b400700 1 --2- 172.21.15.86:0/2771439054 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56301443bef0 0x56301445cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:01.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.147+0000 7f768be00700 1 -- 172.21.15.86:0/2771439054 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56301443bef0 msgr2=0x56301445cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:01.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.147+0000 7f768be00700 1 --2- 172.21.15.86:0/2771439054 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56301443bef0 0x56301445cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.147+0000 7f768be00700 1 -- 172.21.15.86:0/2771439054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563014446df0 msgr2=0x56301444d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:01.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.147+0000 7f768be00700 1 --2- 172.21.15.86:0/2771439054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563014446df0 0x56301444d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.147+0000 7f768be00700 1 -- 172.21.15.86:0/2771439054 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563014353ea0 con 0x56301443b5d0 2024-03-23T17:51:01.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.147+0000 7f7690c00700 1 --2- 172.21.15.86:0/2771439054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563014446df0 0x56301444d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:51:01.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.147+0000 7f768b400700 1 --2- 172.21.15.86:0/2771439054 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56301443bef0 0x56301445cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:51:01.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.147+0000 7f768be00700 1 --2- 172.21.15.86:0/2771439054 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56301443b5d0 0x563014458560 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f767c006010 tx=0x7f767c022f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a29d483d7d710e45 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:01.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.147+0000 7f7681c00700 1 -- 172.21.15.86:0/2771439054 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f767c09fea0 con 0x56301443b5d0 2024-03-23T17:51:01.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.148+0000 7f7681c00700 1 -- 172.21.15.86:0/2771439054 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f767c09ef80 con 0x56301443b5d0 2024-03-23T17:51:01.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.148+0000 7f7681c00700 1 -- 172.21.15.86:0/2771439054 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f767c09b150 con 0x56301443b5d0 2024-03-23T17:51:01.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.148+0000 7f76a664eb80 1 -- 172.21.15.86:0/2771439054 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56301443b5d0 msgr2=0x563014458560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:01.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.148+0000 7f76a664eb80 1 --2- 172.21.15.86:0/2771439054 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56301443b5d0 0x563014458560 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f767c006010 tx=0x7f767c022f90 comp rx=0 tx=0).stop 2024-03-23T17:51:01.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.148+0000 7f76a664eb80 1 -- 172.21.15.86:0/2771439054 shutdown_connections 2024-03-23T17:51:01.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.148+0000 7f76a664eb80 1 --2- 172.21.15.86:0/2771439054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563014446df0 0x56301444d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.148+0000 7f76a664eb80 1 --2- 172.21.15.86:0/2771439054 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56301443b5d0 0x563014458560 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.148+0000 7f76a664eb80 1 --2- 172.21.15.86:0/2771439054 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56301443bef0 0x56301445cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.148+0000 7f76a664eb80 1 -- 172.21.15.86:0/2771439054 >> 172.21.15.86:0/2771439054 conn(0x56301439b6a0 msgr2=0x563014442fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:01.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.149+0000 7f76a664eb80 1 -- 172.21.15.86:0/2771439054 shutdown_connections 2024-03-23T17:51:01.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.149+0000 7f76a664eb80 1 -- 172.21.15.86:0/2771439054 wait complete. 2024-03-23T17:51:01.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.149+0000 7f76a664eb80 1 -- start start 2024-03-23T17:51:01.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.149+0000 7f76a664eb80 1 Processor -- start 2024-03-23T17:51:01.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.150+0000 7f76a664eb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56301443b5d0 0x56301444b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:01.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.150+0000 7f76a664eb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56301443bef0 0x56301444bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:01.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.150+0000 7f76a664eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563014446df0 0x5630144a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:01.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.150+0000 7f76a664eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630144a1270 con 0x563014446df0 2024-03-23T17:51:01.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.150+0000 7f76a664eb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630144a13e0 con 0x56301443bef0 2024-03-23T17:51:01.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.150+0000 7f76a664eb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630144a15a0 con 0x56301443b5d0 2024-03-23T17:51:01.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.150+0000 7f76a664eb80 10 client.0 _ll_register_callbacks cb 0x563014333470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:51:01.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.150+0000 7f768be00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56301443b5d0 0x56301444b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:01.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.150+0000 7f768be00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56301443b5d0 0x56301444b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:54036/0 (socket says 172.21.15.86:54036) 2024-03-23T17:51:01.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.150+0000 7f768be00700 1 -- 172.21.15.86:0/1638730991 learned_addr learned my addr 172.21.15.86:0/1638730991 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:01.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.151+0000 7f768b400700 1 --2- 172.21.15.86:0/1638730991 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56301443bef0 0x56301444bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:01.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.151+0000 7f768be00700 1 -- 172.21.15.86:0/1638730991 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56301443bef0 msgr2=0x56301444bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:01.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.151+0000 7f768be00700 1 --2- 172.21.15.86:0/1638730991 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56301443bef0 0x56301444bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.151+0000 7f768be00700 1 -- 172.21.15.86:0/1638730991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563014446df0 msgr2=0x5630144a0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:01.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.151+0000 7f768be00700 1 --2- 172.21.15.86:0/1638730991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563014446df0 0x5630144a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.151+0000 7f768be00700 1 -- 172.21.15.86:0/1638730991 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5630141bbb60 con 0x56301443b5d0 2024-03-23T17:51:01.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.151+0000 7f768b400700 1 --2- 172.21.15.86:0/1638730991 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56301443bef0 0x56301444bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T17:51:01.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.151+0000 7f768be00700 1 --2- 172.21.15.86:0/1638730991 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56301443b5d0 0x56301444b780 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f767c097bc0 tx=0x7f767c09b6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=798c5748733d1961 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:01.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.151+0000 7f7675600700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-03-23T17:51:01.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.151+0000 7f7675600700 1 -- 172.21.15.86:0/1638730991 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f767c09b850 con 0x56301443b5d0 2024-03-23T17:51:01.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.151+0000 7f7675600700 1 -- 172.21.15.86:0/1638730991 <== mon.1 v2:172.21.15.152:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f767c0a3540 con 0x56301443b5d0 2024-03-23T17:51:01.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.151+0000 7f76a664eb80 1 -- 172.21.15.86:0/1638730991 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f767c09aab0 con 0x56301443b5d0 2024-03-23T17:51:01.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.151+0000 7f7675600700 1 -- 172.21.15.86:0/1638730991 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f767c09b150 con 0x56301443b5d0 2024-03-23T17:51:01.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.151+0000 7f76a664eb80 10 client.25594 fetch_fsmap learned FSMap version 28 2024-03-23T17:51:01.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.151+0000 7f76a664eb80 1 -- 172.21.15.86:0/1638730991 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x563014213cd0 con 0x56301443b5d0 2024-03-23T17:51:01.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.152+0000 7f7675600700 1 -- 172.21.15.86:0/1638730991 <== mon.1 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f767c09aab0 con 0x56301443b5d0 2024-03-23T17:51:01.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.152+0000 7f7675600700 1 -- 172.21.15.86:0/1638730991 <== mon.1 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f767c00fac0 con 0x56301443b5d0 2024-03-23T17:51:01.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.152+0000 7f76a664eb80 10 client.25594 fetch_fsmap finished waiting for FSMap version 28 2024-03-23T17:51:01.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.152+0000 7f76a664eb80 10 client.25594 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-23T17:51:01.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.152+0000 7f76a664eb80 20 client.25594 populate_metadata read hostname 'smithi086' 2024-03-23T17:51:01.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.152+0000 7f76a664eb80 1 --2- 172.21.15.86:0/1638730991 >> [v2:172.21.15.92:6836/3967837948,v1:172.21.15.92:6837/3967837948] conn(0x5630144a8910 0x5630144aad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:01.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.152+0000 7f76a664eb80 4 client.25594 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-23T17:51:01.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.152+0000 7f76a664eb80 1 -- 172.21.15.86:0/1638730991 --> [v2:172.21.15.92:6836/3967837948,v1:172.21.15.92:6837/3967837948] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5630141dccf0 con 0x5630144a8910 2024-03-23T17:51:01.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.152+0000 7f768b400700 1 --2- 172.21.15.86:0/1638730991 >> [v2:172.21.15.92:6836/3967837948,v1:172.21.15.92:6837/3967837948] conn(0x5630144a8910 0x5630144aad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:01.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.153+0000 7f7675600700 10 client.25594 ms_handle_connect on v2:172.21.15.92:6836/3967837948 2024-03-23T17:51:01.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.153+0000 7f768b400700 1 --2- 172.21.15.86:0/1638730991 >> [v2:172.21.15.92:6836/3967837948,v1:172.21.15.92:6837/3967837948] conn(0x5630144a8910 0x5630144aad00 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a0410f899dc48bd1 server_cookie=1a82fb6230806ed2 in_seq=0 out_seq=0 2024-03-23T17:51:01.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.153+0000 7f7675600700 1 -- 172.21.15.86:0/1638730991 <== mds.2 v2:172.21.15.92:6836/3967837948 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5630141dccf0 con 0x5630144a8910 2024-03-23T17:51:01.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.153+0000 7f7675600700 10 client.25594 handle_command_reply: tid=0 2024-03-23T17:51:01.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.153+0000 7f76a664eb80 1 client.25594 shutdown 2024-03-23T17:51:01.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.153+0000 7f76a664eb80 1 -- 172.21.15.86:0/1638730991 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56301443b5d0 msgr2=0x56301444b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:01.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.153+0000 7f76a664eb80 1 --2- 172.21.15.86:0/1638730991 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56301443b5d0 0x56301444b780 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f767c097bc0 tx=0x7f767c09b6c0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.153+0000 7f76a664eb80 1 -- 172.21.15.86:0/1638730991 shutdown_connections 2024-03-23T17:51:01.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.153+0000 7f76a664eb80 1 --2- 172.21.15.86:0/1638730991 >> [v2:172.21.15.92:6836/3967837948,v1:172.21.15.92:6837/3967837948] conn(0x5630144a8910 0x5630144aad00 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.153+0000 7f76a664eb80 1 --2- 172.21.15.86:0/1638730991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563014446df0 0x5630144a0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.153+0000 7f76a664eb80 1 --2- 172.21.15.86:0/1638730991 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56301443b5d0 0x56301444b780 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.153+0000 7f76a664eb80 1 --2- 172.21.15.86:0/1638730991 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56301443bef0 0x56301444bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.153+0000 7f76a664eb80 1 -- 172.21.15.86:0/1638730991 >> 172.21.15.86:0/1638730991 conn(0x56301439b6a0 msgr2=0x563014442290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:01.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.153+0000 7f7675600700 0 client.25594 ms_handle_reset on v2:172.21.15.92:6836/3967837948 2024-03-23T17:51:01.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.153+0000 7f76a664eb80 1 -- 172.21.15.86:0/1638730991 shutdown_connections 2024-03-23T17:51:01.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.153+0000 7f76a664eb80 1 -- 172.21.15.86:0/1638730991 wait complete. 2024-03-23T17:51:01.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.154+0000 7f76a664eb80 20 client.25594 trim_cache size 0 max 16384 2024-03-23T17:51:01.157 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-03-23T17:51:01.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.154+0000 7f7692c00700 1 -- 172.21.15.86:0/1532988102 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f76700a13b0 msgr2=0x7f76700a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:01.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.154+0000 7f7692c00700 1 --2- 172.21.15.86:0/1532988102 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f76700a13b0 0x7f76700a3870 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7678009fd0 tx=0x7f7678009260 comp rx=0 tx=0).stop 2024-03-23T17:51:01.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.154+0000 7f7692c00700 1 -- 172.21.15.86:0/1532988102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f768c073180 msgr2=0x7f768c199420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:01.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.154+0000 7f7692c00700 1 --2- 172.21.15.86:0/1532988102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f768c073180 0x7f768c199420 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f767c005910 tx=0x7f767c004730 comp rx=0 tx=0).stop 2024-03-23T17:51:01.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.154+0000 7f7692c00700 1 -- 172.21.15.86:0/1532988102 shutdown_connections 2024-03-23T17:51:01.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.154+0000 7f7692c00700 1 --2- 172.21.15.86:0/1532988102 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f76700a13b0 0x7f76700a3870 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.154+0000 7f7692c00700 1 --2- 172.21.15.86:0/1532988102 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f768c10d580 0x7f768c19dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.154+0000 7f7692c00700 1 --2- 172.21.15.86:0/1532988102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f768c073180 0x7f768c199420 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.154+0000 7f7692c00700 1 --2- 172.21.15.86:0/1532988102 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f768c073aa0 0x7f768c199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.154+0000 7f7692c00700 1 -- 172.21.15.86:0/1532988102 >> 172.21.15.86:0/1532988102 conn(0x7f768c0fc7f0 msgr2=0x7f768c0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:01.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.155+0000 7f7692c00700 1 -- 172.21.15.86:0/1532988102 shutdown_connections 2024-03-23T17:51:01.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.155+0000 7f7692c00700 1 -- 172.21.15.86:0/1532988102 wait complete. 2024-03-23T17:51:01.178 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-23T17:51:01.179 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-23T17:51:01.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:51:00 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/3922679078' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T17:51:01.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:51:00 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/3922679078' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T17:51:01.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.399+0000 7f5553e00700 1 -- 172.21.15.86:0/1399468507 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f555410b160 msgr2=0x7f555410d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:01.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.399+0000 7f5553e00700 1 --2- 172.21.15.86:0/1399468507 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f555410b160 0x7f555410d600 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f554c00b0a0 tx=0x7f554c00b3b0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.399+0000 7f5553e00700 1 -- 172.21.15.86:0/1399468507 shutdown_connections 2024-03-23T17:51:01.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.399+0000 7f5553e00700 1 --2- 172.21.15.86:0/1399468507 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f555410b160 0x7f555410d600 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.399+0000 7f5553e00700 1 --2- 172.21.15.86:0/1399468507 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5554101d90 0x7f5554102170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.399+0000 7f5553e00700 1 --2- 172.21.15.86:0/1399468507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55541026b0 0x7f555410ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.399+0000 7f5553e00700 1 -- 172.21.15.86:0/1399468507 >> 172.21.15.86:0/1399468507 conn(0x7f55540fb570 msgr2=0x7f55540fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:01.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.399+0000 7f5553e00700 1 -- 172.21.15.86:0/1399468507 shutdown_connections 2024-03-23T17:51:01.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.399+0000 7f5553e00700 1 -- 172.21.15.86:0/1399468507 wait complete. 2024-03-23T17:51:01.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.400+0000 7f5553e00700 1 Processor -- start 2024-03-23T17:51:01.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.401+0000 7f5553e00700 1 -- start start 2024-03-23T17:51:01.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.401+0000 7f5553e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5554101d90 0x7f5554197130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:01.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.401+0000 7f5553e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55541026b0 0x7f5554197670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:01.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.401+0000 7f5553e00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f555410b160 0x7f555419ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:01.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.401+0000 7f5553e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f555419c050 con 0x7f5554101d90 2024-03-23T17:51:01.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.401+0000 7f5553e00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f555419c1c0 con 0x7f55541026b0 2024-03-23T17:51:01.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.401+0000 7f5553e00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f555419c380 con 0x7f555410b160 2024-03-23T17:51:01.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.402+0000 7f5552a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5554101d90 0x7f5554197130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:01.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.402+0000 7f5553400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f555410b160 0x7f555419ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:01.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.402+0000 7f5552000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55541026b0 0x7f5554197670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:01.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.402+0000 7f5552a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5554101d90 0x7f5554197130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43274/0 (socket says 172.21.15.86:43274) 2024-03-23T17:51:01.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.402+0000 7f5552a00700 1 -- 172.21.15.86:0/2910936647 learned_addr learned my addr 172.21.15.86:0/2910936647 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:01.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.402+0000 7f5552a00700 1 -- 172.21.15.86:0/2910936647 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f555410b160 msgr2=0x7f555419ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:01.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.402+0000 7f5552a00700 1 --2- 172.21.15.86:0/2910936647 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f555410b160 0x7f555419ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.402+0000 7f5552a00700 1 -- 172.21.15.86:0/2910936647 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55541026b0 msgr2=0x7f5554197670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:01.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.402+0000 7f5552a00700 1 --2- 172.21.15.86:0/2910936647 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55541026b0 0x7f5554197670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.402+0000 7f5552a00700 1 -- 172.21.15.86:0/2910936647 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f554c009d00 con 0x7f5554101d90 2024-03-23T17:51:01.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.402+0000 7f5553400700 1 --2- 172.21.15.86:0/2910936647 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f555410b160 0x7f555419ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:51:01.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.402+0000 7f5552000700 1 --2- 172.21.15.86:0/2910936647 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55541026b0 0x7f5554197670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:51:01.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.402+0000 7f5552a00700 1 --2- 172.21.15.86:0/2910936647 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5554101d90 0x7f5554197130 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f5544009d30 tx=0x7f554400d5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:01.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.403+0000 7f554b400700 1 -- 172.21.15.86:0/2910936647 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5544010870 con 0x7f5554101d90 2024-03-23T17:51:01.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.403+0000 7f5553e00700 1 -- 172.21.15.86:0/2910936647 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f555419c660 con 0x7f5554101d90 2024-03-23T17:51:01.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.403+0000 7f554b400700 1 -- 172.21.15.86:0/2910936647 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5544009e80 con 0x7f5554101d90 2024-03-23T17:51:01.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.403+0000 7f554b400700 1 -- 172.21.15.86:0/2910936647 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f554400f620 con 0x7f5554101d90 2024-03-23T17:51:01.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.403+0000 7f5553e00700 1 -- 172.21.15.86:0/2910936647 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55540ffaf0 con 0x7f5554101d90 2024-03-23T17:51:01.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.406+0000 7f554b400700 1 -- 172.21.15.86:0/2910936647 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) v1 ==== 136098+0+0 (secure 0 0 0) 0x7f55440183f0 con 0x7f5554101d90 2024-03-23T17:51:01.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.408+0000 7f554b400700 1 --2- 172.21.15.86:0/2910936647 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f55380a13b0 0x7f55380a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:01.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.408+0000 7f554b400700 1 -- 172.21.15.86:0/2910936647 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f55440cd0f0 con 0x7f5554101d90 2024-03-23T17:51:01.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.408+0000 7f556d6f1b80 1 Processor -- start 2024-03-23T17:51:01.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.409+0000 7f5552000700 1 --2- 172.21.15.86:0/2910936647 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f55380a13b0 0x7f55380a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:01.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.409+0000 7f556d6f1b80 1 -- start start 2024-03-23T17:51:01.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.409+0000 7f556d6f1b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b9e43467a0 0x55b9e4346b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:01.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.409+0000 7f556d6f1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9e43470c0 0x55b9e4351a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:01.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.409+0000 7f556d6f1b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9e4351fc0 0x55b9e43543b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:01.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.409+0000 7f556d6f1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9e429f990 con 0x55b9e43470c0 2024-03-23T17:51:01.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.409+0000 7f556d6f1b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9e429d350 con 0x55b9e4351fc0 2024-03-23T17:51:01.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.409+0000 7f556d6f1b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9e4298c40 con 0x55b9e43467a0 2024-03-23T17:51:01.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.410+0000 7f5553400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9e4351fc0 0x55b9e43543b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:01.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.410+0000 7f5552000700 1 --2- 172.21.15.86:0/2910936647 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f55380a13b0 0x7f55380a3870 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5554198680 tx=0x7f5540016040 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:01.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.410+0000 7f5553400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9e4351fc0 0x55b9e43543b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:58318/0 (socket says 172.21.15.86:58318) 2024-03-23T17:51:01.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.410+0000 7f5553400700 1 -- 172.21.15.86:0/1954268327 learned_addr learned my addr 172.21.15.86:0/1954268327 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:01.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.410+0000 7f5552a00700 1 --2- 172.21.15.86:0/1954268327 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b9e43467a0 0x55b9e4346b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:01.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.410+0000 7f5552000700 1 --2- 172.21.15.86:0/1954268327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9e43470c0 0x55b9e4351a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:01.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.410+0000 7f5553400700 1 -- 172.21.15.86:0/1954268327 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b9e43467a0 msgr2=0x55b9e4346b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:01.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.410+0000 7f5553400700 1 --2- 172.21.15.86:0/1954268327 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b9e43467a0 0x55b9e4346b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.410+0000 7f5553400700 1 -- 172.21.15.86:0/1954268327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9e43470c0 msgr2=0x55b9e4351a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:01.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.410+0000 7f5553400700 1 --2- 172.21.15.86:0/1954268327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9e43470c0 0x55b9e4351a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.410+0000 7f5553400700 1 -- 172.21.15.86:0/1954268327 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b9e411a3b0 con 0x55b9e4351fc0 2024-03-23T17:51:01.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.410+0000 7f5553400700 1 --2- 172.21.15.86:0/1954268327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9e4351fc0 0x55b9e43543b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f554c015000 tx=0x7f554c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c0bc0f27f253f7 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:01.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.411+0000 7f553fe00700 1 -- 172.21.15.86:0/1954268327 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f554c00e040 con 0x55b9e4351fc0 2024-03-23T17:51:01.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.411+0000 7f553fe00700 1 -- 172.21.15.86:0/1954268327 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f554c004a30 con 0x55b9e4351fc0 2024-03-23T17:51:01.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.411+0000 7f553fe00700 1 -- 172.21.15.86:0/1954268327 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f554c0052a0 con 0x55b9e4351fc0 2024-03-23T17:51:01.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.411+0000 7f556d6f1b80 1 -- 172.21.15.86:0/1954268327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9e4351fc0 msgr2=0x55b9e43543b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:01.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.411+0000 7f556d6f1b80 1 --2- 172.21.15.86:0/1954268327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9e4351fc0 0x55b9e43543b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f554c015000 tx=0x7f554c009b80 comp rx=0 tx=0).stop 2024-03-23T17:51:01.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.412+0000 7f556d6f1b80 1 -- 172.21.15.86:0/1954268327 shutdown_connections 2024-03-23T17:51:01.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.412+0000 7f556d6f1b80 1 --2- 172.21.15.86:0/1954268327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9e4351fc0 0x55b9e43543b0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.412+0000 7f556d6f1b80 1 --2- 172.21.15.86:0/1954268327 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b9e43467a0 0x55b9e4346b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.412+0000 7f556d6f1b80 1 --2- 172.21.15.86:0/1954268327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9e43470c0 0x55b9e4351a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.412+0000 7f556d6f1b80 1 -- 172.21.15.86:0/1954268327 >> 172.21.15.86:0/1954268327 conn(0x55b9e42a66a0 msgr2=0x55b9e4342c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:01.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.412+0000 7f556d6f1b80 1 -- 172.21.15.86:0/1954268327 shutdown_connections 2024-03-23T17:51:01.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.412+0000 7f556d6f1b80 1 -- 172.21.15.86:0/1954268327 wait complete. 2024-03-23T17:51:01.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.412+0000 7f556d6f1b80 1 -- start start 2024-03-23T17:51:01.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.413+0000 7f556d6f1b80 1 Processor -- start 2024-03-23T17:51:01.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.413+0000 7f556d6f1b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9e43467a0 0x55b9e4351a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:01.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.413+0000 7f556d6f1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9e43470c0 0x55b9e434ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:01.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.413+0000 7f556d6f1b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b9e4351fc0 0x55b9e434f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:01.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.413+0000 7f556d6f1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9e4234d80 con 0x55b9e43470c0 2024-03-23T17:51:01.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.413+0000 7f556d6f1b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9e4234a20 con 0x55b9e43467a0 2024-03-23T17:51:01.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.413+0000 7f556d6f1b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9e411c6a0 con 0x55b9e4351fc0 2024-03-23T17:51:01.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.414+0000 7f556d6f1b80 10 client.0 _ll_register_callbacks cb 0x55b9e423e470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:51:01.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.414+0000 7f5552a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9e43467a0 0x55b9e4351a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:01.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.414+0000 7f5552a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9e43467a0 0x55b9e4351a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:58330/0 (socket says 172.21.15.86:58330) 2024-03-23T17:51:01.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.414+0000 7f5552a00700 1 -- 172.21.15.86:0/2887832115 learned_addr learned my addr 172.21.15.86:0/2887832115 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:01.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.414+0000 7f5552000700 1 --2- 172.21.15.86:0/2887832115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9e43470c0 0x55b9e434ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:01.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.414+0000 7f5553400700 1 --2- 172.21.15.86:0/2887832115 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b9e4351fc0 0x55b9e434f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:01.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.414+0000 7f5552a00700 1 -- 172.21.15.86:0/2887832115 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b9e4351fc0 msgr2=0x55b9e434f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:01.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.414+0000 7f5552a00700 1 --2- 172.21.15.86:0/2887832115 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b9e4351fc0 0x55b9e434f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.414+0000 7f5552a00700 1 -- 172.21.15.86:0/2887832115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9e43470c0 msgr2=0x55b9e434ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:01.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.414+0000 7f5552a00700 1 --2- 172.21.15.86:0/2887832115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9e43470c0 0x55b9e434ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.414+0000 7f5552a00700 1 -- 172.21.15.86:0/2887832115 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f555410e2c0 con 0x55b9e43467a0 2024-03-23T17:51:01.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.414+0000 7f5552000700 1 --2- 172.21.15.86:0/2887832115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9e43470c0 0x55b9e434ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T17:51:01.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.414+0000 7f5553400700 1 --2- 172.21.15.86:0/2887832115 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b9e4351fc0 0x55b9e434f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T17:51:01.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.414+0000 7f5552a00700 1 --2- 172.21.15.86:0/2887832115 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9e43467a0 0x55b9e4351a50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f554400d910 tx=0x7f554409a120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d23b9767c3b76433 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:01.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.414+0000 7f553cc00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T17:51:01.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.415+0000 7f553cc00700 1 -- 172.21.15.86:0/2887832115 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55440b1070 con 0x55b9e43467a0 2024-03-23T17:51:01.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.415+0000 7f553cc00700 1 -- 172.21.15.86:0/2887832115 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f55440a2420 con 0x55b9e43467a0 2024-03-23T17:51:01.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.415+0000 7f556d6f1b80 1 -- 172.21.15.86:0/2887832115 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b9e4259380 con 0x55b9e43467a0 2024-03-23T17:51:01.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.415+0000 7f553cc00700 1 -- 172.21.15.86:0/2887832115 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5544099490 con 0x55b9e43467a0 2024-03-23T17:51:01.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.415+0000 7f556d6f1b80 10 client.25604 fetch_fsmap learned FSMap version 28 2024-03-23T17:51:01.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.415+0000 7f556d6f1b80 1 -- 172.21.15.86:0/2887832115 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55b9e40c6b60 con 0x55b9e43467a0 2024-03-23T17:51:01.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.415+0000 7f553cc00700 1 -- 172.21.15.86:0/2887832115 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b9e4259380 con 0x55b9e43467a0 2024-03-23T17:51:01.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.416+0000 7f553cc00700 1 -- 172.21.15.86:0/2887832115 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 28) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f55440a2020 con 0x55b9e43467a0 2024-03-23T17:51:01.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.416+0000 7f556d6f1b80 10 client.25604 fetch_fsmap finished waiting for FSMap version 28 2024-03-23T17:51:01.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.416+0000 7f556d6f1b80 10 client.25604 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-23T17:51:01.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.416+0000 7f556d6f1b80 20 client.25604 populate_metadata read hostname 'smithi086' 2024-03-23T17:51:01.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.416+0000 7f556d6f1b80 1 --2- 172.21.15.86:0/2887832115 >> [v2:172.21.15.152:6838/1100723992,v1:172.21.15.152:6839/1100723992] conn(0x55b9e4361c40 0x55b9e4364060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:01.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.416+0000 7f556d6f1b80 4 client.25604 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:51:01.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.416+0000 7f556d6f1b80 1 -- 172.21.15.86:0/2887832115 --> [v2:172.21.15.152:6838/1100723992,v1:172.21.15.152:6839/1100723992] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b9e4291700 con 0x55b9e4361c40 2024-03-23T17:51:01.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.416+0000 7f5552000700 1 --2- 172.21.15.86:0/2887832115 >> [v2:172.21.15.152:6838/1100723992,v1:172.21.15.152:6839/1100723992] conn(0x55b9e4361c40 0x55b9e4364060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:01.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.417+0000 7f553cc00700 10 client.25604 ms_handle_connect on v2:172.21.15.152:6838/1100723992 2024-03-23T17:51:01.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.417+0000 7f5552000700 1 --2- 172.21.15.86:0/2887832115 >> [v2:172.21.15.152:6838/1100723992,v1:172.21.15.152:6839/1100723992] conn(0x55b9e4361c40 0x55b9e4364060 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2073202ccabeb6f2 server_cookie=508d1e3bff44ca91 in_seq=0 out_seq=0 2024-03-23T17:51:01.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.417+0000 7f553cc00700 1 -- 172.21.15.86:0/2887832115 <== mds.1 v2:172.21.15.152:6838/1100723992 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55b9e4291700 con 0x55b9e4361c40 2024-03-23T17:51:01.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.417+0000 7f553cc00700 10 client.25604 handle_command_reply: tid=0 2024-03-23T17:51:01.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.418+0000 7f556d6f1b80 1 client.25604 shutdown 2024-03-23T17:51:01.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.418+0000 7f556d6f1b80 1 -- 172.21.15.86:0/2887832115 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9e43467a0 msgr2=0x55b9e4351a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:01.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.418+0000 7f556d6f1b80 1 --2- 172.21.15.86:0/2887832115 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9e43467a0 0x55b9e4351a50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f554400d910 tx=0x7f554409a120 comp rx=0 tx=0).stop 2024-03-23T17:51:01.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.418+0000 7f556d6f1b80 1 -- 172.21.15.86:0/2887832115 shutdown_connections 2024-03-23T17:51:01.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.418+0000 7f556d6f1b80 1 --2- 172.21.15.86:0/2887832115 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b9e4351fc0 0x55b9e434f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.418+0000 7f556d6f1b80 1 --2- 172.21.15.86:0/2887832115 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9e43467a0 0x55b9e4351a50 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.418+0000 7f556d6f1b80 1 --2- 172.21.15.86:0/2887832115 >> [v2:172.21.15.152:6838/1100723992,v1:172.21.15.152:6839/1100723992] conn(0x55b9e4361c40 0x55b9e4364060 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.418+0000 7f556d6f1b80 1 --2- 172.21.15.86:0/2887832115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9e43470c0 0x55b9e434ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.418+0000 7f556d6f1b80 1 -- 172.21.15.86:0/2887832115 >> 172.21.15.86:0/2887832115 conn(0x55b9e42a66a0 msgr2=0x55b9e4348890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:01.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.418+0000 7f553cc00700 0 client.25604 ms_handle_reset on v2:172.21.15.152:6838/1100723992 2024-03-23T17:51:01.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.418+0000 7f556d6f1b80 1 -- 172.21.15.86:0/2887832115 shutdown_connections 2024-03-23T17:51:01.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.418+0000 7f556d6f1b80 1 -- 172.21.15.86:0/2887832115 wait complete. 2024-03-23T17:51:01.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.419+0000 7f556d6f1b80 20 client.25604 trim_cache size 0 max 16384 2024-03-23T17:51:01.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.434+0000 7f556d6f1b80 1 Processor -- start 2024-03-23T17:51:01.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.434+0000 7f556d6f1b80 1 -- start start 2024-03-23T17:51:01.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.434+0000 7f556d6f1b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b9e43467a0 0x55b9e434ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:01.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.434+0000 7f556d6f1b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9e43470c0 0x55b9e43689c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:01.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.434+0000 7f556d6f1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9e4351fc0 0x55b9e4358be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:01.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.435+0000 7f556d6f1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9e4364460 con 0x55b9e4351fc0 2024-03-23T17:51:01.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.435+0000 7f556d6f1b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9e435e5c0 con 0x55b9e43470c0 2024-03-23T17:51:01.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.435+0000 7f556d6f1b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9e4346b80 con 0x55b9e43467a0 2024-03-23T17:51:01.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.435+0000 7f5552000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9e43470c0 0x55b9e43689c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:01.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.435+0000 7f5552000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9e43470c0 0x55b9e43689c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:58344/0 (socket says 172.21.15.86:58344) 2024-03-23T17:51:01.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.435+0000 7f5552000700 1 -- 172.21.15.86:0/929695680 learned_addr learned my addr 172.21.15.86:0/929695680 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:01.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.435+0000 7f5553400700 1 --2- 172.21.15.86:0/929695680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9e4351fc0 0x55b9e4358be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:01.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.435+0000 7f5552a00700 1 --2- 172.21.15.86:0/929695680 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b9e43467a0 0x55b9e434ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:01.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.435+0000 7f5552a00700 1 -- 172.21.15.86:0/929695680 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9e43470c0 msgr2=0x55b9e43689c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:01.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.435+0000 7f5552a00700 1 --2- 172.21.15.86:0/929695680 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9e43470c0 0x55b9e43689c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.435+0000 7f5552a00700 1 -- 172.21.15.86:0/929695680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9e4351fc0 msgr2=0x55b9e4358be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:01.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.435+0000 7f5552a00700 1 --2- 172.21.15.86:0/929695680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9e4351fc0 0x55b9e4358be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.435+0000 7f5552a00700 1 -- 172.21.15.86:0/929695680 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b9e425eea0 con 0x55b9e43467a0 2024-03-23T17:51:01.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.436+0000 7f5553400700 1 --2- 172.21.15.86:0/929695680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9e4351fc0 0x55b9e4358be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:51:01.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.436+0000 7f5552000700 1 --2- 172.21.15.86:0/929695680 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9e43470c0 0x55b9e43689c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:51:01.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.436+0000 7f5552a00700 1 --2- 172.21.15.86:0/929695680 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b9e43467a0 0x55b9e434ed00 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5544009d00 tx=0x7f5544098550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b9f09f75e14f337 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:01.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.436+0000 7f5548c00700 1 -- 172.21.15.86:0/929695680 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55440b1070 con 0x55b9e43467a0 2024-03-23T17:51:01.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.436+0000 7f5548c00700 1 -- 172.21.15.86:0/929695680 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55440ca5f0 con 0x55b9e43467a0 2024-03-23T17:51:01.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.436+0000 7f5548c00700 1 -- 172.21.15.86:0/929695680 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5544099490 con 0x55b9e43467a0 2024-03-23T17:51:01.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.437+0000 7f556d6f1b80 1 -- 172.21.15.86:0/929695680 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b9e43467a0 msgr2=0x55b9e434ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:01.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.437+0000 7f556d6f1b80 1 --2- 172.21.15.86:0/929695680 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b9e43467a0 0x55b9e434ed00 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5544009d00 tx=0x7f5544098550 comp rx=0 tx=0).stop 2024-03-23T17:51:01.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.437+0000 7f556d6f1b80 1 -- 172.21.15.86:0/929695680 shutdown_connections 2024-03-23T17:51:01.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.437+0000 7f556d6f1b80 1 --2- 172.21.15.86:0/929695680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9e4351fc0 0x55b9e4358be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.437+0000 7f556d6f1b80 1 --2- 172.21.15.86:0/929695680 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b9e43467a0 0x55b9e434ed00 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.437+0000 7f556d6f1b80 1 --2- 172.21.15.86:0/929695680 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9e43470c0 0x55b9e43689c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.437+0000 7f556d6f1b80 1 -- 172.21.15.86:0/929695680 >> 172.21.15.86:0/929695680 conn(0x55b9e42a66a0 msgr2=0x55b9e433b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:01.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.437+0000 7f556d6f1b80 1 -- 172.21.15.86:0/929695680 shutdown_connections 2024-03-23T17:51:01.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.437+0000 7f556d6f1b80 1 -- 172.21.15.86:0/929695680 wait complete. 2024-03-23T17:51:01.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.438+0000 7f556d6f1b80 1 -- start start 2024-03-23T17:51:01.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.438+0000 7f556d6f1b80 1 Processor -- start 2024-03-23T17:51:01.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.438+0000 7f556d6f1b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9e43467a0 0x55b9e4356d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:01.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.438+0000 7f556d6f1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9e43470c0 0x55b9e43572d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:01.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.438+0000 7f556d6f1b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b9e4351fc0 0x55b9e4358140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:01.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.438+0000 7f556d6f1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9e4358710 con 0x55b9e43470c0 2024-03-23T17:51:01.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.438+0000 7f556d6f1b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9e4358880 con 0x55b9e43467a0 2024-03-23T17:51:01.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.438+0000 7f556d6f1b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9e4358a40 con 0x55b9e4351fc0 2024-03-23T17:51:01.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.438+0000 7f556d6f1b80 10 client.0 _ll_register_callbacks cb 0x55b9e423e470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:51:01.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.439+0000 7f5552a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9e43467a0 0x55b9e4356d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:01.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.439+0000 7f5553400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b9e4351fc0 0x55b9e4358140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:01.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.439+0000 7f5552a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9e43467a0 0x55b9e4356d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:58356/0 (socket says 172.21.15.86:58356) 2024-03-23T17:51:01.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.439+0000 7f5552000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9e43470c0 0x55b9e43572d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:01.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.439+0000 7f5552a00700 1 -- 172.21.15.86:0/1064401736 learned_addr learned my addr 172.21.15.86:0/1064401736 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:01.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.439+0000 7f5553400700 1 -- 172.21.15.86:0/1064401736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9e43467a0 msgr2=0x55b9e4356d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:01.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.439+0000 7f5553400700 1 --2- 172.21.15.86:0/1064401736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9e43467a0 0x55b9e4356d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.439+0000 7f5553400700 1 -- 172.21.15.86:0/1064401736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9e43470c0 msgr2=0x55b9e43572d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:01.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.439+0000 7f5553400700 1 --2- 172.21.15.86:0/1064401736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9e43470c0 0x55b9e43572d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.439+0000 7f5553400700 1 -- 172.21.15.86:0/1064401736 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5540025040 con 0x55b9e4351fc0 2024-03-23T17:51:01.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.439+0000 7f5552000700 1 --2- 172.21.15.86:0/1064401736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9e43470c0 0x55b9e43572d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:51:01.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.439+0000 7f5553400700 1 --2- 172.21.15.86:0/1064401736 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b9e4351fc0 0x55b9e4358140 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f554c000f80 tx=0x7f554c00bf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eaa0b0020c87a962 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:01.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.439+0000 7f553cc00700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-03-23T17:51:01.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.440+0000 7f553cc00700 1 -- 172.21.15.86:0/1064401736 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f554c00e040 con 0x55b9e4351fc0 2024-03-23T17:51:01.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.440+0000 7f556d6f1b80 1 -- 172.21.15.86:0/1064401736 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f554c004a30 con 0x55b9e4351fc0 2024-03-23T17:51:01.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.440+0000 7f553cc00700 1 -- 172.21.15.86:0/1064401736 <== mon.1 v2:172.21.15.152:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f554c0078b0 con 0x55b9e4351fc0 2024-03-23T17:51:01.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.440+0000 7f553cc00700 1 -- 172.21.15.86:0/1064401736 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f554c004ba0 con 0x55b9e4351fc0 2024-03-23T17:51:01.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.440+0000 7f556d6f1b80 10 client.25609 fetch_fsmap learned FSMap version 28 2024-03-23T17:51:01.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.440+0000 7f556d6f1b80 1 -- 172.21.15.86:0/1064401736 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55b9e411a3b0 con 0x55b9e4351fc0 2024-03-23T17:51:01.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.440+0000 7f553cc00700 1 -- 172.21.15.86:0/1064401736 <== mon.1 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f554c004a30 con 0x55b9e4351fc0 2024-03-23T17:51:01.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.441+0000 7f553cc00700 1 -- 172.21.15.86:0/1064401736 <== mon.1 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f554c024030 con 0x55b9e4351fc0 2024-03-23T17:51:01.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.441+0000 7f556d6f1b80 10 client.25609 fetch_fsmap finished waiting for FSMap version 28 2024-03-23T17:51:01.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.441+0000 7f556d6f1b80 10 client.25609 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-23T17:51:01.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.441+0000 7f556d6f1b80 20 client.25609 populate_metadata read hostname 'smithi086' 2024-03-23T17:51:01.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.441+0000 7f556d6f1b80 1 --2- 172.21.15.86:0/1064401736 >> [v2:172.21.15.152:6838/1100723992,v1:172.21.15.152:6839/1100723992] conn(0x55b9e43c5610 0x55b9e43c7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:01.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.441+0000 7f556d6f1b80 4 client.25609 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-23T17:51:01.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.441+0000 7f556d6f1b80 1 -- 172.21.15.86:0/1064401736 --> [v2:172.21.15.152:6838/1100723992,v1:172.21.15.152:6839/1100723992] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b9e40e7cf0 con 0x55b9e43c5610 2024-03-23T17:51:01.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.441+0000 7f5552a00700 1 --2- 172.21.15.86:0/1064401736 >> [v2:172.21.15.152:6838/1100723992,v1:172.21.15.152:6839/1100723992] conn(0x55b9e43c5610 0x55b9e43c7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:01.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.442+0000 7f553cc00700 10 client.25609 ms_handle_connect on v2:172.21.15.152:6838/1100723992 2024-03-23T17:51:01.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.442+0000 7f5552a00700 1 --2- 172.21.15.86:0/1064401736 >> [v2:172.21.15.152:6838/1100723992,v1:172.21.15.152:6839/1100723992] conn(0x55b9e43c5610 0x55b9e43c7a30 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=94b4a9b4ef131166 server_cookie=331fb7c302349f1b in_seq=0 out_seq=0 2024-03-23T17:51:01.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.442+0000 7f553cc00700 1 -- 172.21.15.86:0/1064401736 <== mds.1 v2:172.21.15.152:6838/1100723992 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b9e40e7cf0 con 0x55b9e43c5610 2024-03-23T17:51:01.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.442+0000 7f553cc00700 10 client.25609 handle_command_reply: tid=0 2024-03-23T17:51:01.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.442+0000 7f556d6f1b80 1 client.25609 shutdown 2024-03-23T17:51:01.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.442+0000 7f556d6f1b80 1 -- 172.21.15.86:0/1064401736 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b9e4351fc0 msgr2=0x55b9e4358140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:01.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.442+0000 7f556d6f1b80 1 --2- 172.21.15.86:0/1064401736 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b9e4351fc0 0x55b9e4358140 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f554c000f80 tx=0x7f554c00bf90 comp rx=0 tx=0).stop 2024-03-23T17:51:01.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.442+0000 7f556d6f1b80 1 -- 172.21.15.86:0/1064401736 shutdown_connections 2024-03-23T17:51:01.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.442+0000 7f556d6f1b80 1 --2- 172.21.15.86:0/1064401736 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b9e4351fc0 0x55b9e4358140 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.442+0000 7f556d6f1b80 1 --2- 172.21.15.86:0/1064401736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b9e43467a0 0x55b9e4356d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.442+0000 7f556d6f1b80 1 --2- 172.21.15.86:0/1064401736 >> [v2:172.21.15.152:6838/1100723992,v1:172.21.15.152:6839/1100723992] conn(0x55b9e43c5610 0x55b9e43c7a30 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.442+0000 7f556d6f1b80 1 --2- 172.21.15.86:0/1064401736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9e43470c0 0x55b9e43572d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.442+0000 7f556d6f1b80 1 -- 172.21.15.86:0/1064401736 >> 172.21.15.86:0/1064401736 conn(0x55b9e42a66a0 msgr2=0x55b9e434a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:01.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.442+0000 7f553cc00700 0 client.25609 ms_handle_reset on v2:172.21.15.152:6838/1100723992 2024-03-23T17:51:01.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.443+0000 7f556d6f1b80 1 -- 172.21.15.86:0/1064401736 shutdown_connections 2024-03-23T17:51:01.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.443+0000 7f556d6f1b80 1 -- 172.21.15.86:0/1064401736 wait complete. 2024-03-23T17:51:01.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.443+0000 7f556d6f1b80 20 client.25609 trim_cache size 0 max 16384 2024-03-23T17:51:01.446 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-03-23T17:51:01.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.443+0000 7f5553e00700 1 -- 172.21.15.86:0/2910936647 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f55380a13b0 msgr2=0x7f55380a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:01.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.443+0000 7f5553e00700 1 --2- 172.21.15.86:0/2910936647 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f55380a13b0 0x7f55380a3870 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5554198680 tx=0x7f5540016040 comp rx=0 tx=0).stop 2024-03-23T17:51:01.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.443+0000 7f5553e00700 1 -- 172.21.15.86:0/2910936647 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5554101d90 msgr2=0x7f5554197130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:01.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.443+0000 7f5553e00700 1 --2- 172.21.15.86:0/2910936647 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5554101d90 0x7f5554197130 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f5544009d30 tx=0x7f554400d5c0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.444+0000 7f5553e00700 1 -- 172.21.15.86:0/2910936647 shutdown_connections 2024-03-23T17:51:01.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.444+0000 7f5553e00700 1 --2- 172.21.15.86:0/2910936647 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f55380a13b0 0x7f55380a3870 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.444+0000 7f5553e00700 1 --2- 172.21.15.86:0/2910936647 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f555410b160 0x7f555419ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.444+0000 7f5553e00700 1 --2- 172.21.15.86:0/2910936647 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5554101d90 0x7f5554197130 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.444+0000 7f5553e00700 1 --2- 172.21.15.86:0/2910936647 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55541026b0 0x7f5554197670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.444+0000 7f5553e00700 1 -- 172.21.15.86:0/2910936647 >> 172.21.15.86:0/2910936647 conn(0x7f55540fb570 msgr2=0x7f55540fce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:01.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.444+0000 7f5553e00700 1 -- 172.21.15.86:0/2910936647 shutdown_connections 2024-03-23T17:51:01.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.444+0000 7f5553e00700 1 -- 172.21.15.86:0/2910936647 wait complete. 2024-03-23T17:51:01.467 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-23T17:51:01.467 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-23T17:51:01.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.724+0000 7fee97800700 1 -- 172.21.15.86:0/1422766509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee90100f60 msgr2=0x7fee90101340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:01.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.724+0000 7fee97800700 1 --2- 172.21.15.86:0/1422766509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee90100f60 0x7fee90101340 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fee84009990 tx=0x7fee84009ca0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.724+0000 7fee97800700 1 -- 172.21.15.86:0/1422766509 shutdown_connections 2024-03-23T17:51:01.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.724+0000 7fee97800700 1 --2- 172.21.15.86:0/1422766509 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee90106080 0x7fee9010dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.724+0000 7fee97800700 1 --2- 172.21.15.86:0/1422766509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee90100f60 0x7fee90101340 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.724+0000 7fee97800700 1 --2- 172.21.15.86:0/1422766509 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee90101910 0x7fee90105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.724+0000 7fee97800700 1 -- 172.21.15.86:0/1422766509 >> 172.21.15.86:0/1422766509 conn(0x7fee900fc7c0 msgr2=0x7fee900febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:01.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.724+0000 7fee97800700 1 -- 172.21.15.86:0/1422766509 shutdown_connections 2024-03-23T17:51:01.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.724+0000 7fee97800700 1 -- 172.21.15.86:0/1422766509 wait complete. 2024-03-23T17:51:01.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.725+0000 7fee97800700 1 Processor -- start 2024-03-23T17:51:01.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.725+0000 7fee97800700 1 -- start start 2024-03-23T17:51:01.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.726+0000 7fee97800700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee90100f60 0x7fee90197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:01.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.726+0000 7fee97800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee90106080 0x7fee90197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:01.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.726+0000 7fee97800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee9019bba0 0x7fee9019c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:01.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.726+0000 7fee97800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee9019c620 con 0x7fee90106080 2024-03-23T17:51:01.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.726+0000 7fee97800700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee9019c790 con 0x7fee9019bba0 2024-03-23T17:51:01.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.726+0000 7fee97800700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee9019c950 con 0x7fee90100f60 2024-03-23T17:51:01.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.726+0000 7fee94e00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee90100f60 0x7fee90197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:01.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.726+0000 7fee94e00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee90100f60 0x7fee90197310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:54112/0 (socket says 172.21.15.86:54112) 2024-03-23T17:51:01.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.726+0000 7fee94e00700 1 -- 172.21.15.86:0/4000274841 learned_addr learned my addr 172.21.15.86:0/4000274841 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:01.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.726+0000 7fee8fe00700 1 --2- 172.21.15.86:0/4000274841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee90106080 0x7fee90197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:01.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.726+0000 7fee95800700 1 --2- 172.21.15.86:0/4000274841 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee9019bba0 0x7fee9019c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:01.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.727+0000 7fee94e00700 1 -- 172.21.15.86:0/4000274841 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee9019bba0 msgr2=0x7fee9019c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:01.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.727+0000 7fee94e00700 1 --2- 172.21.15.86:0/4000274841 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee9019bba0 0x7fee9019c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.727+0000 7fee94e00700 1 -- 172.21.15.86:0/4000274841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee90106080 msgr2=0x7fee90197850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:01.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.727+0000 7fee94e00700 1 --2- 172.21.15.86:0/4000274841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee90106080 0x7fee90197850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.727+0000 7fee94e00700 1 -- 172.21.15.86:0/4000274841 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee88009c40 con 0x7fee90100f60 2024-03-23T17:51:01.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.727+0000 7fee95800700 1 --2- 172.21.15.86:0/4000274841 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee9019bba0 0x7fee9019c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:51:01.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.727+0000 7fee94e00700 1 --2- 172.21.15.86:0/4000274841 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee90100f60 0x7fee90197310 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fee84000c00 tx=0x7fee84005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:01.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.727+0000 7fee8d600700 1 -- 172.21.15.86:0/4000274841 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee8401d070 con 0x7fee90100f60 2024-03-23T17:51:01.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.727+0000 7fee97800700 1 -- 172.21.15.86:0/4000274841 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee840095f0 con 0x7fee90100f60 2024-03-23T17:51:01.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.727+0000 7fee8d600700 1 -- 172.21.15.86:0/4000274841 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee84022470 con 0x7fee90100f60 2024-03-23T17:51:01.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.727+0000 7fee8d600700 1 -- 172.21.15.86:0/4000274841 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee84017650 con 0x7fee90100f60 2024-03-23T17:51:01.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.728+0000 7fee97800700 1 -- 172.21.15.86:0/4000274841 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee90073270 con 0x7fee90100f60 2024-03-23T17:51:01.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.731+0000 7fee8d600700 1 -- 172.21.15.86:0/4000274841 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) v1 ==== 136098+0+0 (secure 0 0 0) 0x7fee840225e0 con 0x7fee90100f60 2024-03-23T17:51:01.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.732+0000 7feeab301b80 1 Processor -- start 2024-03-23T17:51:01.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.732+0000 7feeab301b80 1 -- start start 2024-03-23T17:51:01.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.733+0000 7feeab301b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558864a745d0 0x558864a749b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:01.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.733+0000 7feeab301b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558864a74ef0 0x558864a7f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:01.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.733+0000 7feeab301b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558864a7fdf0 0x558864a821e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:01.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.733+0000 7feeab301b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588649cd990 con 0x558864a74ef0 2024-03-23T17:51:01.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.733+0000 7feeab301b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588649cb350 con 0x558864a7fdf0 2024-03-23T17:51:01.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.733+0000 7feeab301b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588649c6c40 con 0x558864a745d0 2024-03-23T17:51:01.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.733+0000 7fee94e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558864a74ef0 0x558864a7f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:01.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.733+0000 7fee8d600700 1 --2- 172.21.15.86:0/4000274841 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fee780a1430 0x7fee780a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:01.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.733+0000 7fee94e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558864a74ef0 0x558864a7f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43344/0 (socket says 172.21.15.86:43344) 2024-03-23T17:51:01.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.733+0000 7fee94e00700 1 -- 172.21.15.86:0/395131114 learned_addr learned my addr 172.21.15.86:0/395131114 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:01.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.733+0000 7fee95800700 1 --2- 172.21.15.86:0/395131114 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558864a745d0 0x558864a749b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:01.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.734+0000 7fee8fe00700 1 --2- 172.21.15.86:0/395131114 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558864a7fdf0 0x558864a821e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:01.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.734+0000 7fee8d600700 1 -- 172.21.15.86:0/4000274841 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fee840ce0c0 con 0x7fee90100f60 2024-03-23T17:51:01.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.734+0000 7fee94e00700 1 -- 172.21.15.86:0/395131114 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558864a745d0 msgr2=0x558864a749b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:01.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.734+0000 7fee94e00700 1 --2- 172.21.15.86:0/395131114 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558864a745d0 0x558864a749b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.734+0000 7fee94e00700 1 -- 172.21.15.86:0/395131114 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558864a7fdf0 msgr2=0x558864a821e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:01.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.734+0000 7fee94e00700 1 --2- 172.21.15.86:0/395131114 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558864a7fdf0 0x558864a821e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.734+0000 7fee94e00700 1 -- 172.21.15.86:0/395131114 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588648483b0 con 0x558864a74ef0 2024-03-23T17:51:01.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.734+0000 7fee95800700 1 --2- 172.21.15.86:0/4000274841 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fee780a1430 0x7fee780a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:01.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.734+0000 7fee94e00700 1 --2- 172.21.15.86:0/395131114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558864a74ef0 0x558864a7f8b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fee84097a70 tx=0x7fee84022cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f941c52f34ee200b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:01.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.735+0000 7fee82000700 1 -- 172.21.15.86:0/395131114 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee8409ab40 con 0x558864a74ef0 2024-03-23T17:51:01.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.735+0000 7fee82000700 1 -- 172.21.15.86:0/395131114 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee8409a960 con 0x558864a74ef0 2024-03-23T17:51:01.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.735+0000 7fee82000700 1 -- 172.21.15.86:0/395131114 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee840d00c0 con 0x558864a74ef0 2024-03-23T17:51:01.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.735+0000 7fee95800700 1 --2- 172.21.15.86:0/4000274841 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fee780a1430 0x7fee780a38f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fee88007a20 tx=0x7fee88004b40 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:01.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.736+0000 7feeab301b80 1 -- 172.21.15.86:0/395131114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558864a74ef0 msgr2=0x558864a7f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:01.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.736+0000 7feeab301b80 1 --2- 172.21.15.86:0/395131114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558864a74ef0 0x558864a7f8b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fee84097a70 tx=0x7fee84022cb0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.736+0000 7feeab301b80 1 -- 172.21.15.86:0/395131114 shutdown_connections 2024-03-23T17:51:01.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.736+0000 7feeab301b80 1 --2- 172.21.15.86:0/395131114 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558864a7fdf0 0x558864a821e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.736+0000 7feeab301b80 1 --2- 172.21.15.86:0/395131114 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558864a745d0 0x558864a749b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.736+0000 7feeab301b80 1 --2- 172.21.15.86:0/395131114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558864a74ef0 0x558864a7f8b0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.736+0000 7feeab301b80 1 -- 172.21.15.86:0/395131114 >> 172.21.15.86:0/395131114 conn(0x5588649d46a0 msgr2=0x558864a70a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:01.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.736+0000 7feeab301b80 1 -- 172.21.15.86:0/395131114 shutdown_connections 2024-03-23T17:51:01.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.736+0000 7feeab301b80 1 -- 172.21.15.86:0/395131114 wait complete. 2024-03-23T17:51:01.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.737+0000 7feeab301b80 1 -- start start 2024-03-23T17:51:01.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.737+0000 7feeab301b80 1 Processor -- start 2024-03-23T17:51:01.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.738+0000 7feeab301b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558864a745d0 0x558864a7f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:01.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.738+0000 7feeab301b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558864a74ef0 0x558864a7cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:01.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.738+0000 7feeab301b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558864a7fdf0 0x558864a7d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:01.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.738+0000 7feeab301b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558864962d80 con 0x558864a74ef0 2024-03-23T17:51:01.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.738+0000 7feeab301b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x558864962a20 con 0x558864a745d0 2024-03-23T17:51:01.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.738+0000 7feeab301b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55886484a6a0 con 0x558864a7fdf0 2024-03-23T17:51:01.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.738+0000 7feeab301b80 10 client.0 _ll_register_callbacks cb 0x55886496c470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:51:01.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.738+0000 7fee94e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558864a745d0 0x558864a7f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:01.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.738+0000 7fee8fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558864a74ef0 0x558864a7cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:01.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.738+0000 7fee8fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558864a74ef0 0x558864a7cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43354/0 (socket says 172.21.15.86:43354) 2024-03-23T17:51:01.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.738+0000 7fee94e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558864a745d0 0x558864a7f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:58402/0 (socket says 172.21.15.86:58402) 2024-03-23T17:51:01.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.738+0000 7fee8fe00700 1 -- 172.21.15.86:0/412406544 learned_addr learned my addr 172.21.15.86:0/412406544 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:01.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.738+0000 7fee95800700 1 --2- 172.21.15.86:0/412406544 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558864a7fdf0 0x558864a7d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:01.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.739+0000 7fee94e00700 1 -- 172.21.15.86:0/412406544 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558864a7fdf0 msgr2=0x558864a7d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:01.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.739+0000 7fee94e00700 1 --2- 172.21.15.86:0/412406544 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558864a7fdf0 0x558864a7d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.739+0000 7fee94e00700 1 -- 172.21.15.86:0/412406544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558864a74ef0 msgr2=0x558864a7cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:01.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.739+0000 7fee94e00700 1 --2- 172.21.15.86:0/412406544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558864a74ef0 0x558864a7cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.739+0000 7fee94e00700 1 -- 172.21.15.86:0/412406544 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee90073270 con 0x558864a745d0 2024-03-23T17:51:01.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.739+0000 7fee8fe00700 1 --2- 172.21.15.86:0/412406544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558864a74ef0 0x558864a7cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:51:01.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.739+0000 7fee94e00700 1 --2- 172.21.15.86:0/412406544 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558864a745d0 0x558864a7f810 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fee8409aad0 tx=0x7fee84022b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df22efa3e0bda6c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:01.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.739+0000 7fee76a00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T17:51:01.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.739+0000 7fee76a00700 1 -- 172.21.15.86:0/412406544 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee8409b000 con 0x558864a745d0 2024-03-23T17:51:01.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.739+0000 7fee76a00700 1 -- 172.21.15.86:0/412406544 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fee840aee40 con 0x558864a745d0 2024-03-23T17:51:01.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.739+0000 7feeab301b80 1 -- 172.21.15.86:0/412406544 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558864987380 con 0x558864a745d0 2024-03-23T17:51:01.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.739+0000 7fee76a00700 1 -- 172.21.15.86:0/412406544 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fee8409b700 con 0x558864a745d0 2024-03-23T17:51:01.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.740+0000 7feeab301b80 10 client.25622 fetch_fsmap learned FSMap version 28 2024-03-23T17:51:01.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.740+0000 7feeab301b80 1 -- 172.21.15.86:0/412406544 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5588647f4b60 con 0x558864a745d0 2024-03-23T17:51:01.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.740+0000 7fee76a00700 1 -- 172.21.15.86:0/412406544 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558864987380 con 0x558864a745d0 2024-03-23T17:51:01.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.740+0000 7fee76a00700 1 -- 172.21.15.86:0/412406544 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 28) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fee840a2f60 con 0x558864a745d0 2024-03-23T17:51:01.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.740+0000 7feeab301b80 10 client.25622 fetch_fsmap finished waiting for FSMap version 28 2024-03-23T17:51:01.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.740+0000 7feeab301b80 10 client.25622 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T17:51:01.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.740+0000 7feeab301b80 20 client.25622 populate_metadata read hostname 'smithi086' 2024-03-23T17:51:01.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.740+0000 7feeab301b80 1 --2- 172.21.15.86:0/412406544 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x558864a97ab0 0x558864a99ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:01.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.740+0000 7feeab301b80 4 client.25622 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:51:01.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.740+0000 7feeab301b80 1 -- 172.21.15.86:0/412406544 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5588649bf700 con 0x558864a97ab0 2024-03-23T17:51:01.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.740+0000 7fee8fe00700 1 --2- 172.21.15.86:0/412406544 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x558864a97ab0 0x558864a99ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:01.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.741+0000 7fee8fe00700 1 --2- 172.21.15.86:0/412406544 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x558864a97ab0 0x558864a99ed0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ccc8d55df7638d4 server_cookie=6bc1ac7bc140c4f9 in_seq=0 out_seq=0 2024-03-23T17:51:01.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.741+0000 7fee76a00700 10 client.25622 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:51:01.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.742+0000 7fee76a00700 1 -- 172.21.15.86:0/412406544 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5588649bf700 con 0x558864a97ab0 2024-03-23T17:51:01.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.742+0000 7fee76a00700 10 client.25622 handle_command_reply: tid=0 2024-03-23T17:51:01.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.742+0000 7feeab301b80 1 client.25622 shutdown 2024-03-23T17:51:01.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.742+0000 7feeab301b80 1 -- 172.21.15.86:0/412406544 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558864a745d0 msgr2=0x558864a7f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:01.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.742+0000 7feeab301b80 1 --2- 172.21.15.86:0/412406544 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558864a745d0 0x558864a7f810 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fee8409aad0 tx=0x7fee84022b10 comp rx=0 tx=0).stop 2024-03-23T17:51:01.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.742+0000 7feeab301b80 1 -- 172.21.15.86:0/412406544 shutdown_connections 2024-03-23T17:51:01.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.742+0000 7feeab301b80 1 --2- 172.21.15.86:0/412406544 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x558864a97ab0 0x558864a99ed0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.742+0000 7feeab301b80 1 --2- 172.21.15.86:0/412406544 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558864a7fdf0 0x558864a7d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.742+0000 7feeab301b80 1 --2- 172.21.15.86:0/412406544 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558864a745d0 0x558864a7f810 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.742+0000 7feeab301b80 1 --2- 172.21.15.86:0/412406544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558864a74ef0 0x558864a7cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.742+0000 7feeab301b80 1 -- 172.21.15.86:0/412406544 >> 172.21.15.86:0/412406544 conn(0x5588649d46a0 msgr2=0x558864a766c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:01.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.742+0000 7fee76a00700 0 client.25622 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:51:01.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.742+0000 7feeab301b80 1 -- 172.21.15.86:0/412406544 shutdown_connections 2024-03-23T17:51:01.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.742+0000 7feeab301b80 1 -- 172.21.15.86:0/412406544 wait complete. 2024-03-23T17:51:01.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.743+0000 7feeab301b80 20 client.25622 trim_cache size 0 max 16384 2024-03-23T17:51:01.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.760+0000 7feeab301b80 1 Processor -- start 2024-03-23T17:51:01.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.761+0000 7feeab301b80 1 -- start start 2024-03-23T17:51:01.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.761+0000 7feeab301b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558864a745d0 0x558864a9e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:01.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.761+0000 7feeab301b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558864a74ef0 0x558864a8eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:01.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.761+0000 7feeab301b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558864a7fdf0 0x558864a7f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:01.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.761+0000 7feeab301b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558864a7da60 con 0x558864a74ef0 2024-03-23T17:51:01.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.761+0000 7feeab301b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x558864a7d780 con 0x558864a745d0 2024-03-23T17:51:01.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.761+0000 7feeab301b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x558864a743c0 con 0x558864a7fdf0 2024-03-23T17:51:01.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.761+0000 7fee94e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558864a745d0 0x558864a9e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:01.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.761+0000 7fee8fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558864a74ef0 0x558864a8eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:01.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.761+0000 7fee94e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558864a745d0 0x558864a9e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:58410/0 (socket says 172.21.15.86:58410) 2024-03-23T17:51:01.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.761+0000 7fee94e00700 1 -- 172.21.15.86:0/1667378323 learned_addr learned my addr 172.21.15.86:0/1667378323 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:01.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.761+0000 7fee95800700 1 --2- 172.21.15.86:0/1667378323 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558864a7fdf0 0x558864a7f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:01.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.762+0000 7fee94e00700 1 -- 172.21.15.86:0/1667378323 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558864a7fdf0 msgr2=0x558864a7f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:01.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.762+0000 7fee94e00700 1 --2- 172.21.15.86:0/1667378323 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558864a7fdf0 0x558864a7f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.762+0000 7fee94e00700 1 -- 172.21.15.86:0/1667378323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558864a74ef0 msgr2=0x558864a8eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:01.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.762+0000 7fee94e00700 1 --2- 172.21.15.86:0/1667378323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558864a74ef0 0x558864a8eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.762+0000 7fee94e00700 1 -- 172.21.15.86:0/1667378323 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55886498cea0 con 0x558864a745d0 2024-03-23T17:51:01.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.762+0000 7fee8fe00700 1 --2- 172.21.15.86:0/1667378323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558864a74ef0 0x558864a8eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:51:01.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.762+0000 7fee94e00700 1 --2- 172.21.15.86:0/1667378323 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558864a745d0 0x558864a9e830 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fee84099350 tx=0x7fee84097340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ebe799ddb4c6974 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:01.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.762+0000 7fee82a00700 1 -- 172.21.15.86:0/1667378323 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee8400fb00 con 0x558864a745d0 2024-03-23T17:51:01.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.762+0000 7fee82a00700 1 -- 172.21.15.86:0/1667378323 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee840cb5c0 con 0x558864a745d0 2024-03-23T17:51:01.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.762+0000 7fee82a00700 1 -- 172.21.15.86:0/1667378323 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee8409ee30 con 0x558864a745d0 2024-03-23T17:51:01.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.763+0000 7feeab301b80 1 -- 172.21.15.86:0/1667378323 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558864a745d0 msgr2=0x558864a9e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:01.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.763+0000 7feeab301b80 1 --2- 172.21.15.86:0/1667378323 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558864a745d0 0x558864a9e830 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fee84099350 tx=0x7fee84097340 comp rx=0 tx=0).stop 2024-03-23T17:51:01.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.763+0000 7feeab301b80 1 -- 172.21.15.86:0/1667378323 shutdown_connections 2024-03-23T17:51:01.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.763+0000 7feeab301b80 1 --2- 172.21.15.86:0/1667378323 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558864a7fdf0 0x558864a7f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.763+0000 7feeab301b80 1 --2- 172.21.15.86:0/1667378323 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558864a745d0 0x558864a9e830 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.763+0000 7feeab301b80 1 --2- 172.21.15.86:0/1667378323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558864a74ef0 0x558864a8eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.763+0000 7feeab301b80 1 -- 172.21.15.86:0/1667378323 >> 172.21.15.86:0/1667378323 conn(0x5588649d46a0 msgr2=0x558864a77e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:01.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.763+0000 7feeab301b80 1 -- 172.21.15.86:0/1667378323 shutdown_connections 2024-03-23T17:51:01.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.764+0000 7feeab301b80 1 -- 172.21.15.86:0/1667378323 wait complete. 2024-03-23T17:51:01.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.764+0000 7feeab301b80 1 -- start start 2024-03-23T17:51:01.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.764+0000 7feeab301b80 1 Processor -- start 2024-03-23T17:51:01.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.765+0000 7feeab301b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558864a745d0 0x558864a8eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:01.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.765+0000 7feeab301b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558864a74ef0 0x558864a8c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:01.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.765+0000 7feeab301b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558864a7fdf0 0x558864a8c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:01.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.765+0000 7feeab301b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558864a8cef0 con 0x558864a74ef0 2024-03-23T17:51:01.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.765+0000 7feeab301b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x558864a8d060 con 0x558864a7fdf0 2024-03-23T17:51:01.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.765+0000 7feeab301b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x558864ae85d0 con 0x558864a745d0 2024-03-23T17:51:01.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.765+0000 7feeab301b80 10 client.0 _ll_register_callbacks cb 0x55886496c470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:51:01.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.765+0000 7fee94e00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558864a745d0 0x558864a8eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:01.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.765+0000 7fee94e00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558864a745d0 0x558864a8eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:54164/0 (socket says 172.21.15.86:54164) 2024-03-23T17:51:01.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.765+0000 7fee94e00700 1 -- 172.21.15.86:0/2610961863 learned_addr learned my addr 172.21.15.86:0/2610961863 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:01.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.765+0000 7fee8fe00700 1 --2- 172.21.15.86:0/2610961863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558864a74ef0 0x558864a8c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:01.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.765+0000 7fee95800700 1 --2- 172.21.15.86:0/2610961863 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558864a7fdf0 0x558864a8c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:01.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.765+0000 7fee94e00700 1 -- 172.21.15.86:0/2610961863 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558864a7fdf0 msgr2=0x558864a8c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:01.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.765+0000 7fee94e00700 1 --2- 172.21.15.86:0/2610961863 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558864a7fdf0 0x558864a8c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.765+0000 7fee94e00700 1 -- 172.21.15.86:0/2610961863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558864a74ef0 msgr2=0x558864a8c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:01.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.766+0000 7fee94e00700 1 --2- 172.21.15.86:0/2610961863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558864a74ef0 0x558864a8c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.766+0000 7fee94e00700 1 -- 172.21.15.86:0/2610961863 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee7c005ce0 con 0x558864a745d0 2024-03-23T17:51:01.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.766+0000 7fee8fe00700 1 --2- 172.21.15.86:0/2610961863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558864a74ef0 0x558864a8c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:51:01.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.766+0000 7fee94e00700 1 --2- 172.21.15.86:0/2610961863 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558864a745d0 0x558864a8eea0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fee8409ad00 tx=0x7fee840973b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74760e01934c90a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:01.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.766+0000 7fee76a00700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-03-23T17:51:01.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.766+0000 7fee76a00700 1 -- 172.21.15.86:0/2610961863 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee8400fb00 con 0x558864a745d0 2024-03-23T17:51:01.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.766+0000 7fee76a00700 1 -- 172.21.15.86:0/2610961863 <== mon.1 v2:172.21.15.152:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fee840a9030 con 0x558864a745d0 2024-03-23T17:51:01.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.766+0000 7feeab301b80 1 -- 172.21.15.86:0/2610961863 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fee8409a960 con 0x558864a745d0 2024-03-23T17:51:01.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.766+0000 7fee76a00700 1 -- 172.21.15.86:0/2610961863 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fee8409b700 con 0x558864a745d0 2024-03-23T17:51:01.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.766+0000 7feeab301b80 10 client.25627 fetch_fsmap learned FSMap version 28 2024-03-23T17:51:01.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.766+0000 7feeab301b80 1 -- 172.21.15.86:0/2610961863 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5588647f4b60 con 0x558864a745d0 2024-03-23T17:51:01.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.767+0000 7fee76a00700 1 -- 172.21.15.86:0/2610961863 <== mon.1 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee8409a960 con 0x558864a745d0 2024-03-23T17:51:01.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.767+0000 7fee76a00700 1 -- 172.21.15.86:0/2610961863 <== mon.1 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fee840a2f60 con 0x558864a745d0 2024-03-23T17:51:01.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.767+0000 7feeab301b80 10 client.25627 fetch_fsmap finished waiting for FSMap version 28 2024-03-23T17:51:01.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.767+0000 7feeab301b80 10 client.25627 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T17:51:01.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.767+0000 7feeab301b80 20 client.25627 populate_metadata read hostname 'smithi086' 2024-03-23T17:51:01.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.767+0000 7feeab301b80 1 --2- 172.21.15.86:0/2610961863 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x558864ae9810 0x558864aebc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:01.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.767+0000 7feeab301b80 4 client.25627 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-23T17:51:01.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.767+0000 7feeab301b80 1 -- 172.21.15.86:0/2610961863 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558864815cf0 con 0x558864ae9810 2024-03-23T17:51:01.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.768+0000 7fee8fe00700 1 --2- 172.21.15.86:0/2610961863 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x558864ae9810 0x558864aebc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:01.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.768+0000 7fee8fe00700 1 --2- 172.21.15.86:0/2610961863 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x558864ae9810 0x558864aebc00 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85914433f1e6a795 server_cookie=d68dc08e0e85bf9d in_seq=0 out_seq=0 2024-03-23T17:51:01.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.768+0000 7fee76a00700 10 client.25627 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:51:01.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.768+0000 7fee76a00700 1 -- 172.21.15.86:0/2610961863 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558864815cf0 con 0x558864ae9810 2024-03-23T17:51:01.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.768+0000 7fee76a00700 10 client.25627 handle_command_reply: tid=0 2024-03-23T17:51:01.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.768+0000 7feeab301b80 1 client.25627 shutdown 2024-03-23T17:51:01.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.768+0000 7feeab301b80 1 -- 172.21.15.86:0/2610961863 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558864a745d0 msgr2=0x558864a8eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:01.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.768+0000 7feeab301b80 1 --2- 172.21.15.86:0/2610961863 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558864a745d0 0x558864a8eea0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fee8409ad00 tx=0x7fee840973b0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.769+0000 7feeab301b80 1 -- 172.21.15.86:0/2610961863 shutdown_connections 2024-03-23T17:51:01.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.769+0000 7feeab301b80 1 --2- 172.21.15.86:0/2610961863 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x558864ae9810 0x558864aebc00 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.769+0000 7feeab301b80 1 --2- 172.21.15.86:0/2610961863 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558864a7fdf0 0x558864a8c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.769+0000 7feeab301b80 1 --2- 172.21.15.86:0/2610961863 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558864a745d0 0x558864a8eea0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.769+0000 7fee76a00700 0 client.25627 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:51:01.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.769+0000 7feeab301b80 1 --2- 172.21.15.86:0/2610961863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558864a74ef0 0x558864a8c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.769+0000 7feeab301b80 1 -- 172.21.15.86:0/2610961863 >> 172.21.15.86:0/2610961863 conn(0x5588649d46a0 msgr2=0x558864a7c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:01.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.769+0000 7feeab301b80 1 -- 172.21.15.86:0/2610961863 shutdown_connections 2024-03-23T17:51:01.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.769+0000 7feeab301b80 1 -- 172.21.15.86:0/2610961863 wait complete. 2024-03-23T17:51:01.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.769+0000 7feeab301b80 20 client.25627 trim_cache size 0 max 16384 2024-03-23T17:51:01.772 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-03-23T17:51:01.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.770+0000 7fee97800700 1 -- 172.21.15.86:0/4000274841 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fee780a1430 msgr2=0x7fee780a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:01.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.770+0000 7fee97800700 1 --2- 172.21.15.86:0/4000274841 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fee780a1430 0x7fee780a38f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fee88007a20 tx=0x7fee88004b40 comp rx=0 tx=0).stop 2024-03-23T17:51:01.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.770+0000 7fee97800700 1 -- 172.21.15.86:0/4000274841 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee90100f60 msgr2=0x7fee90197310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:01.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.770+0000 7fee97800700 1 --2- 172.21.15.86:0/4000274841 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee90100f60 0x7fee90197310 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fee84000c00 tx=0x7fee84005740 comp rx=0 tx=0).stop 2024-03-23T17:51:01.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.770+0000 7fee97800700 1 -- 172.21.15.86:0/4000274841 shutdown_connections 2024-03-23T17:51:01.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.770+0000 7fee97800700 1 --2- 172.21.15.86:0/4000274841 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee9019bba0 0x7fee9019c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.770+0000 7fee97800700 1 --2- 172.21.15.86:0/4000274841 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fee780a1430 0x7fee780a38f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.770+0000 7fee97800700 1 --2- 172.21.15.86:0/4000274841 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee90100f60 0x7fee90197310 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.770+0000 7fee97800700 1 --2- 172.21.15.86:0/4000274841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee90106080 0x7fee90197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:01.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.770+0000 7fee97800700 1 -- 172.21.15.86:0/4000274841 >> 172.21.15.86:0/4000274841 conn(0x7fee900fc7c0 msgr2=0x7fee900fdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:01.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.770+0000 7fee97800700 1 -- 172.21.15.86:0/4000274841 shutdown_connections 2024-03-23T17:51:01.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:01.770+0000 7fee97800700 1 -- 172.21.15.86:0/4000274841 wait complete. 2024-03-23T17:51:02.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:51:01 smithi086 ceph-mon[38221]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.7 KiB/s wr, 17 op/s 2024-03-23T17:51:02.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:51:01 smithi086 ceph-mon[38221]: mgrmap e25: x(active, since 22s), standbys: y, z 2024-03-23T17:51:02.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:51:01 smithi092 ceph-mon[79089]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.7 KiB/s wr, 17 op/s 2024-03-23T17:51:02.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:51:01 smithi092 ceph-mon[79089]: mgrmap e25: x(active, since 22s), standbys: y, z 2024-03-23T17:51:02.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:51:01 smithi152 ceph-mon[47949]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.7 KiB/s wr, 17 op/s 2024-03-23T17:51:02.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:51:01 smithi152 ceph-mon[47949]: mgrmap e25: x(active, since 22s), standbys: y, z 2024-03-23T17:51:02.790 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-23T17:51:02.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T17:51:02.791 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-23T17:51:03.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.018+0000 7f7c0be00700 1 -- 172.21.15.86:0/1845478494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c0c102610 msgr2=0x7f7c0c10aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:03.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.018+0000 7f7c0be00700 1 --2- 172.21.15.86:0/1845478494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c0c102610 0x7f7c0c10aaf0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f7bf8009990 tx=0x7f7bf8009ca0 comp rx=0 tx=0).stop 2024-03-23T17:51:03.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.018+0000 7f7c0be00700 1 -- 172.21.15.86:0/1845478494 shutdown_connections 2024-03-23T17:51:03.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.018+0000 7f7c0be00700 1 --2- 172.21.15.86:0/1845478494 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c0c10b110 0x7f7c0c10d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:03.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.018+0000 7f7c0be00700 1 --2- 172.21.15.86:0/1845478494 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c0c101cf0 0x7f7c0c1020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:03.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.018+0000 7f7c0be00700 1 --2- 172.21.15.86:0/1845478494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c0c102610 0x7f7c0c10aaf0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:03.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.018+0000 7f7c0be00700 1 -- 172.21.15.86:0/1845478494 >> 172.21.15.86:0/1845478494 conn(0x7f7c0c0fb510 msgr2=0x7f7c0c0fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:03.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.018+0000 7f7c0be00700 1 -- 172.21.15.86:0/1845478494 shutdown_connections 2024-03-23T17:51:03.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.018+0000 7f7c0be00700 1 -- 172.21.15.86:0/1845478494 wait complete. 2024-03-23T17:51:03.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.019+0000 7f7c0be00700 1 Processor -- start 2024-03-23T17:51:03.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.020+0000 7f7c0be00700 1 -- start start 2024-03-23T17:51:03.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.020+0000 7f7c0be00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c0c101cf0 0x7f7c0c199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:03.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.020+0000 7f7c0be00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c0c102610 0x7f7c0c1997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:03.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.020+0000 7f7c0be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c0c10b110 0x7f7c0c19daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:03.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.020+0000 7f7c0be00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0c19e130 con 0x7f7c0c10b110 2024-03-23T17:51:03.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.020+0000 7f7c0be00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0c19e2a0 con 0x7f7c0c101cf0 2024-03-23T17:51:03.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.020+0000 7f7c0be00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0c19e460 con 0x7f7c0c102610 2024-03-23T17:51:03.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.021+0000 7f7c0b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c0c10b110 0x7f7c0c19daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:03.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.021+0000 7f7c0aa00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c0c101cf0 0x7f7c0c199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:03.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.021+0000 7f7c0a000700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c0c102610 0x7f7c0c1997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:03.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.021+0000 7f7c0b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c0c10b110 0x7f7c0c19daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43392/0 (socket says 172.21.15.86:43392) 2024-03-23T17:51:03.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.021+0000 7f7c0aa00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c0c101cf0 0x7f7c0c199260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:58430/0 (socket says 172.21.15.86:58430) 2024-03-23T17:51:03.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.021+0000 7f7c0b400700 1 -- 172.21.15.86:0/3284023833 learned_addr learned my addr 172.21.15.86:0/3284023833 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:03.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.021+0000 7f7c0a000700 1 -- 172.21.15.86:0/3284023833 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c0c101cf0 msgr2=0x7f7c0c199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:03.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.021+0000 7f7c0a000700 1 --2- 172.21.15.86:0/3284023833 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c0c101cf0 0x7f7c0c199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:03.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.021+0000 7f7c0a000700 1 -- 172.21.15.86:0/3284023833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c0c10b110 msgr2=0x7f7c0c19daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:03.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.021+0000 7f7c0a000700 1 --2- 172.21.15.86:0/3284023833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c0c10b110 0x7f7c0c19daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:03.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.021+0000 7f7c0a000700 1 -- 172.21.15.86:0/3284023833 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bfc009520 con 0x7f7c0c102610 2024-03-23T17:51:03.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.021+0000 7f7c0a000700 1 --2- 172.21.15.86:0/3284023833 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c0c102610 0x7f7c0c1997a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7bf8000c00 tx=0x7f7bf8004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:03.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.022+0000 7f7c03400700 1 -- 172.21.15.86:0/3284023833 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf801d070 con 0x7f7c0c102610 2024-03-23T17:51:03.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.022+0000 7f7c0be00700 1 -- 172.21.15.86:0/3284023833 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bf80095f0 con 0x7f7c0c102610 2024-03-23T17:51:03.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.022+0000 7f7c0be00700 1 -- 172.21.15.86:0/3284023833 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c0c1a5da0 con 0x7f7c0c102610 2024-03-23T17:51:03.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.022+0000 7f7c03400700 1 -- 172.21.15.86:0/3284023833 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7bf8005030 con 0x7f7c0c102610 2024-03-23T17:51:03.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.022+0000 7f7c03400700 1 -- 172.21.15.86:0/3284023833 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf80049a0 con 0x7f7c0c102610 2024-03-23T17:51:03.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.025+0000 7f7c03400700 1 -- 172.21.15.86:0/3284023833 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) v1 ==== 136098+0+0 (secure 0 0 0) 0x7f7bf8004b00 con 0x7f7c0c102610 2024-03-23T17:51:03.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.027+0000 7f7c03400700 1 --2- 172.21.15.86:0/3284023833 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f7bf00a13b0 0x7f7bf00a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:03.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.027+0000 7f7c252a8b80 1 Processor -- start 2024-03-23T17:51:03.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.028+0000 7f7c252a8b80 1 -- start start 2024-03-23T17:51:03.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.028+0000 7f7c03400700 1 -- 172.21.15.86:0/3284023833 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f7bf80cdd60 con 0x7f7c0c102610 2024-03-23T17:51:03.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.028+0000 7f7c0aa00700 1 --2- 172.21.15.86:0/3284023833 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f7bf00a13b0 0x7f7bf00a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:03.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.028+0000 7f7c252a8b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e35fcd5820 0x55e35fcd5c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:03.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.028+0000 7f7c252a8b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e35fcd6140 0x55e35fce0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:03.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.028+0000 7f7c252a8b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e35fce1040 0x55e35fce3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:03.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.028+0000 7f7c252a8b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e35fc2ea10 con 0x55e35fcd5820 2024-03-23T17:51:03.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.028+0000 7f7c252a8b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e35fc2c3d0 con 0x55e35fcd6140 2024-03-23T17:51:03.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.028+0000 7f7c252a8b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e35fc27cc0 con 0x55e35fce1040 2024-03-23T17:51:03.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.028+0000 7f7c0aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e35fcd5820 0x55e35fcd5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:03.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.029+0000 7f7c0a000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e35fcd6140 0x55e35fce0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:03.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.029+0000 7f7c0aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e35fcd5820 0x55e35fcd5c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43408/0 (socket says 172.21.15.86:43408) 2024-03-23T17:51:03.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.029+0000 7f7c0aa00700 1 -- 172.21.15.86:0/2641464180 learned_addr learned my addr 172.21.15.86:0/2641464180 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:03.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.029+0000 7f7c0b400700 1 --2- 172.21.15.86:0/2641464180 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e35fce1040 0x55e35fce3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:03.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.029+0000 7f7c0a000700 1 -- 172.21.15.86:0/2641464180 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e35fce1040 msgr2=0x55e35fce3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:03.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.029+0000 7f7c0a000700 1 --2- 172.21.15.86:0/2641464180 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e35fce1040 0x55e35fce3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:03.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.029+0000 7f7c0a000700 1 -- 172.21.15.86:0/2641464180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e35fcd5820 msgr2=0x55e35fcd5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:03.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.029+0000 7f7c0a000700 1 --2- 172.21.15.86:0/2641464180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e35fcd5820 0x55e35fcd5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:03.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.029+0000 7f7c0a000700 1 -- 172.21.15.86:0/2641464180 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e35faa9430 con 0x55e35fcd6140 2024-03-23T17:51:03.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.029+0000 7f7c0aa00700 1 --2- 172.21.15.86:0/2641464180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e35fcd5820 0x55e35fcd5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:51:03.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.029+0000 7f7c0a000700 1 --2- 172.21.15.86:0/2641464180 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e35fcd6140 0x55e35fce0b00 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7bf80cb380 tx=0x7f7bf809a940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c685edc9eb687684 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:03.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.029+0000 7f7bf7e00700 1 -- 172.21.15.86:0/2641464180 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf809eb30 con 0x55e35fcd6140 2024-03-23T17:51:03.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.029+0000 7f7c0aa00700 1 --2- 172.21.15.86:0/3284023833 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f7bf00a13b0 0x7f7bf00a3870 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc009f30 tx=0x7f7bfc0137f0 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:03.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.029+0000 7f7bf7e00700 1 -- 172.21.15.86:0/2641464180 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7bf80a30f0 con 0x55e35fcd6140 2024-03-23T17:51:03.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.029+0000 7f7bf7e00700 1 -- 172.21.15.86:0/2641464180 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf80a2ec0 con 0x55e35fcd6140 2024-03-23T17:51:03.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.030+0000 7f7c252a8b80 1 -- 172.21.15.86:0/2641464180 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e35fcd6140 msgr2=0x55e35fce0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:03.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.030+0000 7f7c252a8b80 1 --2- 172.21.15.86:0/2641464180 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e35fcd6140 0x55e35fce0b00 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7bf80cb380 tx=0x7f7bf809a940 comp rx=0 tx=0).stop 2024-03-23T17:51:03.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.030+0000 7f7c252a8b80 1 -- 172.21.15.86:0/2641464180 shutdown_connections 2024-03-23T17:51:03.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.030+0000 7f7c252a8b80 1 --2- 172.21.15.86:0/2641464180 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e35fce1040 0x55e35fce3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:03.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.030+0000 7f7c252a8b80 1 --2- 172.21.15.86:0/2641464180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e35fcd5820 0x55e35fcd5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:03.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.030+0000 7f7c252a8b80 1 --2- 172.21.15.86:0/2641464180 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e35fcd6140 0x55e35fce0b00 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:03.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.030+0000 7f7c252a8b80 1 -- 172.21.15.86:0/2641464180 >> 172.21.15.86:0/2641464180 conn(0x55e35fc35720 msgr2=0x55e35fcd1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:03.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.031+0000 7f7c252a8b80 1 -- 172.21.15.86:0/2641464180 shutdown_connections 2024-03-23T17:51:03.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.031+0000 7f7c252a8b80 1 -- 172.21.15.86:0/2641464180 wait complete. 2024-03-23T17:51:03.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.031+0000 7f7c252a8b80 1 -- start start 2024-03-23T17:51:03.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.031+0000 7f7c252a8b80 1 Processor -- start 2024-03-23T17:51:03.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.032+0000 7f7c252a8b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e35fcd5820 0x55e35fce09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:03.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.032+0000 7f7c252a8b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e35fcd6140 0x55e35fcddc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:03.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.032+0000 7f7c252a8b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e35fce1040 0x55e35fcde270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:03.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.032+0000 7f7c252a8b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e35fbc3e00 con 0x55e35fcd6140 2024-03-23T17:51:03.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.032+0000 7f7c252a8b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e35fbc3aa0 con 0x55e35fce1040 2024-03-23T17:51:03.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.032+0000 7f7c252a8b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e35faab720 con 0x55e35fcd5820 2024-03-23T17:51:03.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.032+0000 7f7c252a8b80 10 client.0 _ll_register_callbacks cb 0x55e35fbcd4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:51:03.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.032+0000 7f7c0a000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e35fcd6140 0x55e35fcddc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:03.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.032+0000 7f7c0a000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e35fcd6140 0x55e35fcddc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43414/0 (socket says 172.21.15.86:43414) 2024-03-23T17:51:03.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.032+0000 7f7c0a000700 1 -- 172.21.15.86:0/3282908246 learned_addr learned my addr 172.21.15.86:0/3282908246 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:03.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.032+0000 7f7c0aa00700 1 --2- 172.21.15.86:0/3282908246 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e35fcd5820 0x55e35fce09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:03.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.032+0000 7f7c0b400700 1 --2- 172.21.15.86:0/3282908246 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e35fce1040 0x55e35fcde270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:03.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.033+0000 7f7c0a000700 1 -- 172.21.15.86:0/3282908246 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e35fcd5820 msgr2=0x55e35fce09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:03.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.033+0000 7f7c0a000700 1 --2- 172.21.15.86:0/3282908246 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e35fcd5820 0x55e35fce09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:03.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.033+0000 7f7c0a000700 1 -- 172.21.15.86:0/3282908246 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e35fce1040 msgr2=0x55e35fcde270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:03.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.033+0000 7f7c0a000700 1 --2- 172.21.15.86:0/3282908246 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e35fce1040 0x55e35fcde270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:03.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.033+0000 7f7c0a000700 1 -- 172.21.15.86:0/3282908246 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c0c1a5da0 con 0x55e35fcd6140 2024-03-23T17:51:03.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.033+0000 7f7c0aa00700 1 --2- 172.21.15.86:0/3282908246 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e35fcd5820 0x55e35fce09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:51:03.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.033+0000 7f7c0b400700 1 --2- 172.21.15.86:0/3282908246 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e35fce1040 0x55e35fcde270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:51:03.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.033+0000 7f7c0a000700 1 --2- 172.21.15.86:0/3282908246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e35fcd6140 0x55e35fcddc90 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f7bf809ac90 tx=0x7f7bf80970b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=792e71414b6305fb server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:03.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.033+0000 7f7bf4c00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-23T17:51:03.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.033+0000 7f7bf4c00700 1 -- 172.21.15.86:0/3282908246 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf80a2e70 con 0x55e35fcd6140 2024-03-23T17:51:03.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.033+0000 7f7bf4c00700 1 -- 172.21.15.86:0/3282908246 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f7bf80cb660 con 0x55e35fcd6140 2024-03-23T17:51:03.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.033+0000 7f7c252a8b80 1 -- 172.21.15.86:0/3282908246 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e35fbe8400 con 0x55e35fcd6140 2024-03-23T17:51:03.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.034+0000 7f7bf4c00700 1 -- 172.21.15.86:0/3282908246 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7bf80cb820 con 0x55e35fcd6140 2024-03-23T17:51:03.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.034+0000 7f7c252a8b80 10 client.15591 fetch_fsmap learned FSMap version 28 2024-03-23T17:51:03.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.034+0000 7f7c252a8b80 1 -- 172.21.15.86:0/3282908246 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55e35fa55be0 con 0x55e35fcd6140 2024-03-23T17:51:03.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.034+0000 7f7bf4c00700 1 -- 172.21.15.86:0/3282908246 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7bf80b7070 con 0x55e35fcd6140 2024-03-23T17:51:03.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.034+0000 7f7bf4c00700 1 -- 172.21.15.86:0/3282908246 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7bf8096af0 con 0x55e35fcd6140 2024-03-23T17:51:03.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.034+0000 7f7c252a8b80 10 client.15591 fetch_fsmap finished waiting for FSMap version 28 2024-03-23T17:51:03.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.034+0000 7f7c252a8b80 10 client.15591 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T17:51:03.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.034+0000 7f7c252a8b80 20 client.15591 populate_metadata read hostname 'smithi086' 2024-03-23T17:51:03.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.034+0000 7f7c252a8b80 1 --2- 172.21.15.86:0/3282908246 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55e35fcf0460 0x55e35fcf2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:03.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.034+0000 7f7c252a8b80 4 client.15591 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:51:03.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.034+0000 7f7c252a8b80 1 -- 172.21.15.86:0/3282908246 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e35fc20780 con 0x55e35fcf0460 2024-03-23T17:51:03.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.035+0000 7f7c0aa00700 1 --2- 172.21.15.86:0/3282908246 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55e35fcf0460 0x55e35fcf2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:03.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.035+0000 7f7bf4c00700 10 client.15591 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:51:03.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.035+0000 7f7c0aa00700 1 --2- 172.21.15.86:0/3282908246 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55e35fcf0460 0x55e35fcf2850 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42bc8ac2c582eec6 server_cookie=86be3169c6459ac0 in_seq=0 out_seq=0 2024-03-23T17:51:03.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.036+0000 7f7bf4c00700 1 -- 172.21.15.86:0/3282908246 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55e35fc20780 con 0x55e35fcf0460 2024-03-23T17:51:03.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.036+0000 7f7bf4c00700 10 client.15591 handle_command_reply: tid=0 2024-03-23T17:51:03.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.036+0000 7f7c252a8b80 1 client.15591 shutdown 2024-03-23T17:51:03.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.037+0000 7f7c252a8b80 1 -- 172.21.15.86:0/3282908246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e35fcd6140 msgr2=0x55e35fcddc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:03.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.037+0000 7f7c252a8b80 1 --2- 172.21.15.86:0/3282908246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e35fcd6140 0x55e35fcddc90 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f7bf809ac90 tx=0x7f7bf80970b0 comp rx=0 tx=0).stop 2024-03-23T17:51:03.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.037+0000 7f7c252a8b80 1 -- 172.21.15.86:0/3282908246 shutdown_connections 2024-03-23T17:51:03.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.037+0000 7f7c252a8b80 1 --2- 172.21.15.86:0/3282908246 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55e35fcf0460 0x55e35fcf2850 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:03.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.037+0000 7f7c252a8b80 1 --2- 172.21.15.86:0/3282908246 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e35fce1040 0x55e35fcde270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:03.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.037+0000 7f7c252a8b80 1 --2- 172.21.15.86:0/3282908246 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e35fcd5820 0x55e35fce09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:03.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.037+0000 7f7c252a8b80 1 --2- 172.21.15.86:0/3282908246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e35fcd6140 0x55e35fcddc90 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:03.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.037+0000 7f7c252a8b80 1 -- 172.21.15.86:0/3282908246 >> 172.21.15.86:0/3282908246 conn(0x55e35fc35720 msgr2=0x55e35fcd7910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:03.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.037+0000 7f7bf4c00700 0 client.15591 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:51:03.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.037+0000 7f7c252a8b80 1 -- 172.21.15.86:0/3282908246 shutdown_connections 2024-03-23T17:51:03.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.037+0000 7f7c252a8b80 1 -- 172.21.15.86:0/3282908246 wait complete. 2024-03-23T17:51:03.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.037+0000 7f7c252a8b80 20 client.15591 trim_cache size 0 max 16384 2024-03-23T17:51:03.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.054+0000 7f7c252a8b80 1 Processor -- start 2024-03-23T17:51:03.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.054+0000 7f7c252a8b80 1 -- start start 2024-03-23T17:51:03.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.055+0000 7f7c252a8b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e35fcd5820 0x55e35fce7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:03.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.055+0000 7f7c252a8b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e35fcd6140 0x55e35fcd1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:03.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.055+0000 7f7c252a8b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e35fce1040 0x55e35fcec890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:03.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.055+0000 7f7c252a8b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e35fcd11e0 con 0x55e35fce1040 2024-03-23T17:51:03.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.055+0000 7f7c252a8b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e35fcdefe0 con 0x55e35fcd6140 2024-03-23T17:51:03.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.055+0000 7f7c252a8b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e35fcf2e80 con 0x55e35fcd5820 2024-03-23T17:51:03.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.055+0000 7f7c0b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e35fce1040 0x55e35fcec890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:03.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.055+0000 7f7c0aa00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e35fcd5820 0x55e35fce7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:03.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.055+0000 7f7c0aa00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e35fcd5820 0x55e35fce7a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:54200/0 (socket says 172.21.15.86:54200) 2024-03-23T17:51:03.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.055+0000 7f7c0aa00700 1 -- 172.21.15.86:0/3515242050 learned_addr learned my addr 172.21.15.86:0/3515242050 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:03.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.055+0000 7f7c0a000700 1 --2- 172.21.15.86:0/3515242050 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e35fcd6140 0x55e35fcd1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:03.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.055+0000 7f7c0aa00700 1 -- 172.21.15.86:0/3515242050 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e35fcd6140 msgr2=0x55e35fcd1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:03.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.055+0000 7f7c0aa00700 1 --2- 172.21.15.86:0/3515242050 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e35fcd6140 0x55e35fcd1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:03.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.055+0000 7f7c0aa00700 1 -- 172.21.15.86:0/3515242050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e35fce1040 msgr2=0x55e35fcec890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:03.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.056+0000 7f7c0aa00700 1 --2- 172.21.15.86:0/3515242050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e35fce1040 0x55e35fcec890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:03.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.056+0000 7f7c0aa00700 1 -- 172.21.15.86:0/3515242050 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e35fbedf20 con 0x55e35fcd5820 2024-03-23T17:51:03.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.056+0000 7f7c0aa00700 1 --2- 172.21.15.86:0/3515242050 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e35fcd5820 0x55e35fce7a30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc013bf0 tx=0x7f7bfc01f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7eaf8fc375526334 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:03.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.056+0000 7f7c00c00700 1 -- 172.21.15.86:0/3515242050 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bfc026040 con 0x55e35fcd5820 2024-03-23T17:51:03.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.056+0000 7f7c00c00700 1 -- 172.21.15.86:0/3515242050 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7bfc029340 con 0x55e35fcd5820 2024-03-23T17:51:03.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.056+0000 7f7c00c00700 1 -- 172.21.15.86:0/3515242050 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bfc018070 con 0x55e35fcd5820 2024-03-23T17:51:03.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.057+0000 7f7c252a8b80 1 -- 172.21.15.86:0/3515242050 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e35fcd5820 msgr2=0x55e35fce7a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:03.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.057+0000 7f7c252a8b80 1 --2- 172.21.15.86:0/3515242050 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e35fcd5820 0x55e35fce7a30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc013bf0 tx=0x7f7bfc01f000 comp rx=0 tx=0).stop 2024-03-23T17:51:03.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.057+0000 7f7c252a8b80 1 -- 172.21.15.86:0/3515242050 shutdown_connections 2024-03-23T17:51:03.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.057+0000 7f7c252a8b80 1 --2- 172.21.15.86:0/3515242050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e35fce1040 0x55e35fcec890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:03.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.057+0000 7f7c252a8b80 1 --2- 172.21.15.86:0/3515242050 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e35fcd5820 0x55e35fce7a30 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:03.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.057+0000 7f7c252a8b80 1 --2- 172.21.15.86:0/3515242050 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e35fcd6140 0x55e35fcd1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:03.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.057+0000 7f7c252a8b80 1 -- 172.21.15.86:0/3515242050 >> 172.21.15.86:0/3515242050 conn(0x55e35fc35720 msgr2=0x55e35fcd7720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:03.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.057+0000 7f7c252a8b80 1 -- 172.21.15.86:0/3515242050 shutdown_connections 2024-03-23T17:51:03.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.057+0000 7f7c252a8b80 1 -- 172.21.15.86:0/3515242050 wait complete. 2024-03-23T17:51:03.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.058+0000 7f7c252a8b80 1 -- start start 2024-03-23T17:51:03.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.058+0000 7f7c252a8b80 1 Processor -- start 2024-03-23T17:51:03.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.058+0000 7f7c252a8b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e35fcd5820 0x55e35fce7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:03.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.058+0000 7f7c252a8b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e35fcd6140 0x55e35fce7570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:03.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.058+0000 7f7c252a8b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e35fce1040 0x55e35fd3b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:03.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.058+0000 7f7c252a8b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e35fd3b990 con 0x55e35fce1040 2024-03-23T17:51:03.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.058+0000 7f7c252a8b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e35fd3bb00 con 0x55e35fcd6140 2024-03-23T17:51:03.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.058+0000 7f7c252a8b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e35fd3bcc0 con 0x55e35fcd5820 2024-03-23T17:51:03.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.058+0000 7f7c252a8b80 10 client.0 _ll_register_callbacks cb 0x55e35fbcd4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:51:03.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.059+0000 7f7c0a000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e35fcd6140 0x55e35fce7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:03.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.059+0000 7f7c0b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e35fce1040 0x55e35fd3b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:03.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.059+0000 7f7c0a000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e35fcd6140 0x55e35fce7570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:58476/0 (socket says 172.21.15.86:58476) 2024-03-23T17:51:03.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.059+0000 7f7c0a000700 1 -- 172.21.15.86:0/3209504189 learned_addr learned my addr 172.21.15.86:0/3209504189 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:03.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.059+0000 7f7c0aa00700 1 --2- 172.21.15.86:0/3209504189 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e35fcd5820 0x55e35fce7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:03.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.059+0000 7f7c0a000700 1 -- 172.21.15.86:0/3209504189 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e35fcd5820 msgr2=0x55e35fce7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:03.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.059+0000 7f7c0a000700 1 --2- 172.21.15.86:0/3209504189 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e35fcd5820 0x55e35fce7030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:03.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.059+0000 7f7c0a000700 1 -- 172.21.15.86:0/3209504189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e35fce1040 msgr2=0x55e35fd3b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:03.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.059+0000 7f7c0a000700 1 --2- 172.21.15.86:0/3209504189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e35fce1040 0x55e35fd3b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:03.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.059+0000 7f7c0a000700 1 -- 172.21.15.86:0/3209504189 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bfc0121f0 con 0x55e35fcd6140 2024-03-23T17:51:03.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.060+0000 7f7c0a000700 1 --2- 172.21.15.86:0/3209504189 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e35fcd6140 0x55e35fce7570 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7bf80098e0 tx=0x7f7bf800b490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3524cc2d311f0777 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:03.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.060+0000 7f7bf4c00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T17:51:03.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.060+0000 7f7bf4c00700 1 -- 172.21.15.86:0/3209504189 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf800fe10 con 0x55e35fcd6140 2024-03-23T17:51:03.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.060+0000 7f7bf4c00700 1 -- 172.21.15.86:0/3209504189 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f7bf80a2e70 con 0x55e35fcd6140 2024-03-23T17:51:03.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.060+0000 7f7c252a8b80 1 -- 172.21.15.86:0/3209504189 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7bf80a30f0 con 0x55e35fcd6140 2024-03-23T17:51:03.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.060+0000 7f7bf4c00700 1 -- 172.21.15.86:0/3209504189 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7bf8099fb0 con 0x55e35fcd6140 2024-03-23T17:51:03.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.060+0000 7f7c252a8b80 10 client.25646 fetch_fsmap learned FSMap version 28 2024-03-23T17:51:03.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.060+0000 7f7c252a8b80 1 -- 172.21.15.86:0/3209504189 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55e35faadd50 con 0x55e35fcd6140 2024-03-23T17:51:03.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.061+0000 7f7bf4c00700 1 -- 172.21.15.86:0/3209504189 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7bf80a30f0 con 0x55e35fcd6140 2024-03-23T17:51:03.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.061+0000 7f7bf4c00700 1 -- 172.21.15.86:0/3209504189 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 28) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7bf80cb660 con 0x55e35fcd6140 2024-03-23T17:51:03.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.061+0000 7f7c252a8b80 10 client.25646 fetch_fsmap finished waiting for FSMap version 28 2024-03-23T17:51:03.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.061+0000 7f7c252a8b80 10 client.25646 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T17:51:03.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.061+0000 7f7c252a8b80 20 client.25646 populate_metadata read hostname 'smithi086' 2024-03-23T17:51:03.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.061+0000 7f7c252a8b80 1 --2- 172.21.15.86:0/3209504189 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55e35fd44270 0x55e35fd46660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:03.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.061+0000 7f7c252a8b80 4 client.25646 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-23T17:51:03.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.061+0000 7f7c252a8b80 1 -- 172.21.15.86:0/3209504189 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55e35fa76d70 con 0x55e35fd44270 2024-03-23T17:51:03.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.061+0000 7f7c0aa00700 1 --2- 172.21.15.86:0/3209504189 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55e35fd44270 0x55e35fd46660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:03.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.062+0000 7f7bf4c00700 10 client.25646 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:51:03.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.062+0000 7f7c0aa00700 1 --2- 172.21.15.86:0/3209504189 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55e35fd44270 0x55e35fd46660 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7b2a0a2e1d997eb server_cookie=9b282051622576c1 in_seq=0 out_seq=0 2024-03-23T17:51:03.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.063+0000 7f7bf4c00700 1 -- 172.21.15.86:0/3209504189 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55e35fa76d70 con 0x55e35fd44270 2024-03-23T17:51:03.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.063+0000 7f7bf4c00700 10 client.25646 handle_command_reply: tid=0 2024-03-23T17:51:03.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.063+0000 7f7c252a8b80 1 client.25646 shutdown 2024-03-23T17:51:03.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.063+0000 7f7c252a8b80 1 -- 172.21.15.86:0/3209504189 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e35fcd6140 msgr2=0x55e35fce7570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:03.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.063+0000 7f7c252a8b80 1 --2- 172.21.15.86:0/3209504189 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e35fcd6140 0x55e35fce7570 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7bf80098e0 tx=0x7f7bf800b490 comp rx=0 tx=0).stop 2024-03-23T17:51:03.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.063+0000 7f7c252a8b80 1 -- 172.21.15.86:0/3209504189 shutdown_connections 2024-03-23T17:51:03.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.064+0000 7f7c252a8b80 1 --2- 172.21.15.86:0/3209504189 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55e35fd44270 0x55e35fd46660 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:03.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.064+0000 7f7c252a8b80 1 --2- 172.21.15.86:0/3209504189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e35fce1040 0x55e35fd3b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:03.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.064+0000 7f7c252a8b80 1 --2- 172.21.15.86:0/3209504189 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e35fcd5820 0x55e35fce7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:03.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.064+0000 7f7c252a8b80 1 --2- 172.21.15.86:0/3209504189 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e35fcd6140 0x55e35fce7570 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:03.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.064+0000 7f7bf4c00700 0 client.25646 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:51:03.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.064+0000 7f7c252a8b80 1 -- 172.21.15.86:0/3209504189 >> 172.21.15.86:0/3209504189 conn(0x55e35fc35720 msgr2=0x55e35fcec700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:03.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.064+0000 7f7c252a8b80 1 -- 172.21.15.86:0/3209504189 shutdown_connections 2024-03-23T17:51:03.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.064+0000 7f7c252a8b80 1 -- 172.21.15.86:0/3209504189 wait complete. 2024-03-23T17:51:03.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.064+0000 7f7c252a8b80 20 client.25646 trim_cache size 0 max 16384 2024-03-23T17:51:03.066 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-23T17:51:03.066 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-03-23T17:51:03.066 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "69575a08-bb4f-4558-b2f2-487c0ea7a5a9", 2024-03-23T17:51:03.066 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-03-23T17:51:03.066 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-23T17:51:03.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.065+0000 7f7c0be00700 1 -- 172.21.15.86:0/3284023833 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f7bf00a13b0 msgr2=0x7f7bf00a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:03.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.065+0000 7f7c0be00700 1 --2- 172.21.15.86:0/3284023833 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f7bf00a13b0 0x7f7bf00a3870 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc009f30 tx=0x7f7bfc0137f0 comp rx=0 tx=0).stop 2024-03-23T17:51:03.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.065+0000 7f7c0be00700 1 -- 172.21.15.86:0/3284023833 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c0c102610 msgr2=0x7f7c0c1997a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:03.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.065+0000 7f7c0be00700 1 --2- 172.21.15.86:0/3284023833 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c0c102610 0x7f7c0c1997a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7bf8000c00 tx=0x7f7bf8004730 comp rx=0 tx=0).stop 2024-03-23T17:51:03.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.065+0000 7f7c0be00700 1 -- 172.21.15.86:0/3284023833 shutdown_connections 2024-03-23T17:51:03.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.065+0000 7f7c0be00700 1 --2- 172.21.15.86:0/3284023833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c0c10b110 0x7f7c0c19daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:03.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.065+0000 7f7c0be00700 1 --2- 172.21.15.86:0/3284023833 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c0c101cf0 0x7f7c0c199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:03.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.065+0000 7f7c0be00700 1 --2- 172.21.15.86:0/3284023833 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f7bf00a13b0 0x7f7bf00a3870 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:03.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.065+0000 7f7c0be00700 1 --2- 172.21.15.86:0/3284023833 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7c0c102610 0x7f7c0c1997a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:03.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.065+0000 7f7c0be00700 1 -- 172.21.15.86:0/3284023833 >> 172.21.15.86:0/3284023833 conn(0x7f7c0c0fb510 msgr2=0x7f7c0c100a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:03.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.065+0000 7f7c0be00700 1 -- 172.21.15.86:0/3284023833 shutdown_connections 2024-03-23T17:51:03.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.065+0000 7f7c0be00700 1 -- 172.21.15.86:0/3284023833 wait complete. 2024-03-23T17:51:03.084 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-23T17:51:03.084 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-23T17:51:03.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.286+0000 7f09dba00700 1 -- 172.21.15.86:0/2404744433 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f09d4100f60 msgr2=0x7f09d4101340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:03.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.286+0000 7f09dba00700 1 --2- 172.21.15.86:0/2404744433 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f09d4100f60 0x7f09d4101340 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f09c8009990 tx=0x7f09c8009ca0 comp rx=0 tx=0).stop 2024-03-23T17:51:03.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.286+0000 7f09dba00700 1 -- 172.21.15.86:0/2404744433 shutdown_connections 2024-03-23T17:51:03.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.286+0000 7f09dba00700 1 --2- 172.21.15.86:0/2404744433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09d4106080 0x7f09d410dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:03.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.286+0000 7f09dba00700 1 --2- 172.21.15.86:0/2404744433 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f09d4100f60 0x7f09d4101340 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:03.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.286+0000 7f09dba00700 1 --2- 172.21.15.86:0/2404744433 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09d4101910 0x7f09d4105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:03.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.286+0000 7f09dba00700 1 -- 172.21.15.86:0/2404744433 >> 172.21.15.86:0/2404744433 conn(0x7f09d40fc7c0 msgr2=0x7f09d40febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:03.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.286+0000 7f09dba00700 1 -- 172.21.15.86:0/2404744433 shutdown_connections 2024-03-23T17:51:03.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.286+0000 7f09dba00700 1 -- 172.21.15.86:0/2404744433 wait complete. 2024-03-23T17:51:03.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.287+0000 7f09dba00700 1 Processor -- start 2024-03-23T17:51:03.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.287+0000 7f09dba00700 1 -- start start 2024-03-23T17:51:03.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.288+0000 7f09dba00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09d4100f60 0x7f09d4199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:03.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.288+0000 7f09dba00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f09d4106080 0x7f09d4199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:03.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.288+0000 7f09dba00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09d419dd30 0x7f09d419e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:03.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.288+0000 7f09dba00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d419e7b0 con 0x7f09d4100f60 2024-03-23T17:51:03.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.288+0000 7f09dba00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d419e920 con 0x7f09d419dd30 2024-03-23T17:51:03.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.288+0000 7f09dba00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d419eae0 con 0x7f09d4106080 2024-03-23T17:51:03.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.288+0000 7f09d9000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09d4100f60 0x7f09d4199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:03.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.288+0000 7f09d3e00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f09d4106080 0x7f09d4199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:03.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.288+0000 7f09d9000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09d4100f60 0x7f09d4199450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43444/0 (socket says 172.21.15.86:43444) 2024-03-23T17:51:03.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.288+0000 7f09d9000700 1 -- 172.21.15.86:0/1403169144 learned_addr learned my addr 172.21.15.86:0/1403169144 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:03.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.288+0000 7f09d9a00700 1 --2- 172.21.15.86:0/1403169144 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09d419dd30 0x7f09d419e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:03.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.289+0000 7f09d3e00700 1 -- 172.21.15.86:0/1403169144 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09d419dd30 msgr2=0x7f09d419e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:03.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.289+0000 7f09d3e00700 1 --2- 172.21.15.86:0/1403169144 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09d419dd30 0x7f09d419e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:03.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.289+0000 7f09d3e00700 1 -- 172.21.15.86:0/1403169144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09d4100f60 msgr2=0x7f09d4199450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:03.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.289+0000 7f09d3e00700 1 --2- 172.21.15.86:0/1403169144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09d4100f60 0x7f09d4199450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:03.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.289+0000 7f09d3e00700 1 -- 172.21.15.86:0/1403169144 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09c80095f0 con 0x7f09d4106080 2024-03-23T17:51:03.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.289+0000 7f09d3e00700 1 --2- 172.21.15.86:0/1403169144 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f09d4106080 0x7f09d4199990 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f09c000e9c0 tx=0x7f09c000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:03.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.289+0000 7f09d1600700 1 -- 172.21.15.86:0/1403169144 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c000c9c0 con 0x7f09d4106080 2024-03-23T17:51:03.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.289+0000 7f09d1600700 1 -- 172.21.15.86:0/1403169144 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09c0010430 con 0x7f09d4106080 2024-03-23T17:51:03.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.289+0000 7f09d1600700 1 -- 172.21.15.86:0/1403169144 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c0018680 con 0x7f09d4106080 2024-03-23T17:51:03.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.289+0000 7f09dba00700 1 -- 172.21.15.86:0/1403169144 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09d41a5fe0 con 0x7f09d4106080 2024-03-23T17:51:03.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.290+0000 7f09dba00700 1 -- 172.21.15.86:0/1403169144 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09d41a6470 con 0x7f09d4106080 2024-03-23T17:51:03.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.295+0000 7f09d1600700 1 -- 172.21.15.86:0/1403169144 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) v1 ==== 136098+0+0 (secure 0 0 0) 0x7f09c000cb20 con 0x7f09d4106080 2024-03-23T17:51:03.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.296+0000 7f09ef535b80 1 Processor -- start 2024-03-23T17:51:03.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.296+0000 7f09d1600700 1 --2- 172.21.15.86:0/1403169144 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f09bc0a13b0 0x7f09bc0a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:03.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.296+0000 7f09d1600700 1 -- 172.21.15.86:0/1403169144 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f09c0014070 con 0x7f09d4106080 2024-03-23T17:51:03.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.296+0000 7f09ef535b80 1 -- start start 2024-03-23T17:51:03.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.297+0000 7f09ef535b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555b655f75d0 0x555b655f79b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:03.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.297+0000 7f09d9a00700 1 --2- 172.21.15.86:0/1403169144 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f09bc0a13b0 0x7f09bc0a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:03.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.297+0000 7f09ef535b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b655f7ef0 0x555b6560b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:03.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.297+0000 7f09ef535b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b6560b600 0x555b6560d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:03.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.297+0000 7f09ef535b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b65550990 con 0x555b6560b600 2024-03-23T17:51:03.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.297+0000 7f09ef535b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b6554e350 con 0x555b655f75d0 2024-03-23T17:51:03.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.297+0000 7f09ef535b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b65549c40 con 0x555b655f7ef0 2024-03-23T17:51:03.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.297+0000 7f09d9000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555b655f75d0 0x555b655f79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:03.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.297+0000 7f09d3e00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b655f7ef0 0x555b6560b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:03.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.298+0000 7f09d3e00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b655f7ef0 0x555b6560b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:54238/0 (socket says 172.21.15.86:54238) 2024-03-23T17:51:03.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.298+0000 7f09d9000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555b655f75d0 0x555b655f79b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:58500/0 (socket says 172.21.15.86:58500) 2024-03-23T17:51:03.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.298+0000 7f09d3e00700 1 -- 172.21.15.86:0/2643621966 learned_addr learned my addr 172.21.15.86:0/2643621966 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:03.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.298+0000 7f09d9a00700 1 --2- 172.21.15.86:0/1403169144 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f09bc0a13b0 0x7f09bc0a3870 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f09cc009550 tx=0x7f09cc009130 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:03.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.298+0000 7f09d3e00700 1 -- 172.21.15.86:0/2643621966 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555b655f75d0 msgr2=0x555b655f79b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:03.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.298+0000 7f09d3e00700 1 --2- 172.21.15.86:0/2643621966 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555b655f75d0 0x555b655f79b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:03.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.298+0000 7f09d3e00700 1 -- 172.21.15.86:0/2643621966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b6560b600 msgr2=0x555b6560d9f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-23T17:51:03.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.298+0000 7f09d9a00700 1 --2- 172.21.15.86:0/2643621966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b6560b600 0x555b6560d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:03.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.298+0000 7f09d3e00700 1 --2- 172.21.15.86:0/2643621966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b6560b600 0x555b6560d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:03.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.298+0000 7f09d3e00700 1 -- 172.21.15.86:0/2643621966 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555b653cb3b0 con 0x555b655f7ef0 2024-03-23T17:51:03.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.298+0000 7f09d9000700 1 --2- 172.21.15.86:0/2643621966 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555b655f75d0 0x555b655f79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:51:03.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.299+0000 7f09d3e00700 1 --2- 172.21.15.86:0/2643621966 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b655f7ef0 0x555b6560b0c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f09c0099f00 tx=0x7f09c0099b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db0496fc289a4ba3 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:03.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.299+0000 7f09c6000700 1 -- 172.21.15.86:0/2643621966 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c0099f70 con 0x555b655f7ef0 2024-03-23T17:51:03.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.299+0000 7f09c6000700 1 -- 172.21.15.86:0/2643621966 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09c009dda0 con 0x555b655f7ef0 2024-03-23T17:51:03.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.299+0000 7f09c6000700 1 -- 172.21.15.86:0/2643621966 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c0010680 con 0x555b655f7ef0 2024-03-23T17:51:03.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.299+0000 7f09ef535b80 1 -- 172.21.15.86:0/2643621966 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b655f7ef0 msgr2=0x555b6560b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:03.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.299+0000 7f09ef535b80 1 --2- 172.21.15.86:0/2643621966 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b655f7ef0 0x555b6560b0c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f09c0099f00 tx=0x7f09c0099b30 comp rx=0 tx=0).stop 2024-03-23T17:51:03.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.300+0000 7f09ef535b80 1 -- 172.21.15.86:0/2643621966 shutdown_connections 2024-03-23T17:51:03.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.300+0000 7f09ef535b80 1 --2- 172.21.15.86:0/2643621966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b6560b600 0x555b6560d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:03.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.300+0000 7f09ef535b80 1 --2- 172.21.15.86:0/2643621966 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555b655f75d0 0x555b655f79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:03.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.300+0000 7f09ef535b80 1 --2- 172.21.15.86:0/2643621966 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b655f7ef0 0x555b6560b0c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:03.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.300+0000 7f09ef535b80 1 -- 172.21.15.86:0/2643621966 >> 172.21.15.86:0/2643621966 conn(0x555b655576a0 msgr2=0x555b655f3a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:03.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.300+0000 7f09ef535b80 1 -- 172.21.15.86:0/2643621966 shutdown_connections 2024-03-23T17:51:03.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.300+0000 7f09ef535b80 1 -- 172.21.15.86:0/2643621966 wait complete. 2024-03-23T17:51:03.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.301+0000 7f09ef535b80 1 -- start start 2024-03-23T17:51:03.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.301+0000 7f09ef535b80 1 Processor -- start 2024-03-23T17:51:03.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.302+0000 7f09ef535b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555b655f75d0 0x555b6560b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:03.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.302+0000 7f09ef535b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b655f7ef0 0x555b656082e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:03.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.302+0000 7f09ef535b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b6560b600 0x555b65608a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:03.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.302+0000 7f09ef535b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b654e5d80 con 0x555b6560b600 2024-03-23T17:51:03.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.302+0000 7f09ef535b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b654e5a20 con 0x555b655f75d0 2024-03-23T17:51:03.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.302+0000 7f09ef535b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b653cd6a0 con 0x555b655f7ef0 2024-03-23T17:51:03.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.302+0000 7f09ef535b80 10 client.0 _ll_register_callbacks cb 0x555b654ef470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:51:03.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.302+0000 7f09d9a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b6560b600 0x555b65608a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:03.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.302+0000 7f09d9000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555b655f75d0 0x555b6560b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:03.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.302+0000 7f09d9a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b6560b600 0x555b65608a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43460/0 (socket says 172.21.15.86:43460) 2024-03-23T17:51:03.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.303+0000 7f09d9a00700 1 -- 172.21.15.86:0/197697179 learned_addr learned my addr 172.21.15.86:0/197697179 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:03.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.303+0000 7f09d3e00700 1 --2- 172.21.15.86:0/197697179 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b655f7ef0 0x555b656082e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:03.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.303+0000 7f09d9a00700 1 -- 172.21.15.86:0/197697179 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b655f7ef0 msgr2=0x555b656082e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:03.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.303+0000 7f09d9a00700 1 --2- 172.21.15.86:0/197697179 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b655f7ef0 0x555b656082e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:03.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.303+0000 7f09d9a00700 1 -- 172.21.15.86:0/197697179 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555b655f75d0 msgr2=0x555b6560b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:03.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.303+0000 7f09d9a00700 1 --2- 172.21.15.86:0/197697179 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555b655f75d0 0x555b6560b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:03.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.303+0000 7f09d9a00700 1 -- 172.21.15.86:0/197697179 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09d41a6470 con 0x555b6560b600 2024-03-23T17:51:03.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.303+0000 7f09d9000700 1 --2- 172.21.15.86:0/197697179 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555b655f75d0 0x555b6560b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:51:03.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.303+0000 7f09d3e00700 1 --2- 172.21.15.86:0/197697179 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b655f7ef0 0x555b656082e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:51:03.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.303+0000 7f09d9a00700 1 --2- 172.21.15.86:0/197697179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b6560b600 0x555b65608a20 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f09cc004a60 tx=0x7f09cc004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc4a495c71075f29 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:03.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.303+0000 7f09baa00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-23T17:51:03.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.304+0000 7f09baa00700 1 -- 172.21.15.86:0/197697179 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09cc021610 con 0x555b6560b600 2024-03-23T17:51:03.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.304+0000 7f09baa00700 1 -- 172.21.15.86:0/197697179 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f09cc021770 con 0x555b6560b600 2024-03-23T17:51:03.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.304+0000 7f09ef535b80 1 -- 172.21.15.86:0/197697179 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555b6550a380 con 0x555b6560b600 2024-03-23T17:51:03.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.304+0000 7f09baa00700 1 -- 172.21.15.86:0/197697179 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f09cc021930 con 0x555b6560b600 2024-03-23T17:51:03.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.304+0000 7f09ef535b80 10 client.15606 fetch_fsmap learned FSMap version 28 2024-03-23T17:51:03.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.304+0000 7f09ef535b80 1 -- 172.21.15.86:0/197697179 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x555b65377b60 con 0x555b6560b600 2024-03-23T17:51:03.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.304+0000 7f09baa00700 1 -- 172.21.15.86:0/197697179 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555b6550a380 con 0x555b6560b600 2024-03-23T17:51:03.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.304+0000 7f09baa00700 1 -- 172.21.15.86:0/197697179 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f09cc016c20 con 0x555b6560b600 2024-03-23T17:51:03.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.305+0000 7f09ef535b80 10 client.15606 fetch_fsmap finished waiting for FSMap version 28 2024-03-23T17:51:03.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.305+0000 7f09ef535b80 10 client.15606 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T17:51:03.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.305+0000 7f09ef535b80 20 client.15606 populate_metadata read hostname 'smithi086' 2024-03-23T17:51:03.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.305+0000 7f09ef535b80 1 --2- 172.21.15.86:0/197697179 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x555b6561b300 0x555b6561d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:03.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.305+0000 7f09ef535b80 4 client.15606 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:51:03.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.305+0000 7f09ef535b80 1 -- 172.21.15.86:0/197697179 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555b65542700 con 0x555b6561b300 2024-03-23T17:51:03.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.306+0000 7f09d9000700 1 --2- 172.21.15.86:0/197697179 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x555b6561b300 0x555b6561d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:03.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.306+0000 7f09d9000700 1 --2- 172.21.15.86:0/197697179 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x555b6561b300 0x555b6561d720 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17a2fd553cc0551d server_cookie=6cbc412dc3c6d76e in_seq=0 out_seq=0 2024-03-23T17:51:03.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.306+0000 7f09baa00700 10 client.15606 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:51:03.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.307+0000 7f09baa00700 1 -- 172.21.15.86:0/197697179 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x555b65542700 con 0x555b6561b300 2024-03-23T17:51:03.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.307+0000 7f09baa00700 10 client.15606 handle_command_reply: tid=0 2024-03-23T17:51:03.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.307+0000 7f09ef535b80 1 client.15606 shutdown 2024-03-23T17:51:03.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.307+0000 7f09ef535b80 1 -- 172.21.15.86:0/197697179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b6560b600 msgr2=0x555b65608a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:03.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.307+0000 7f09ef535b80 1 --2- 172.21.15.86:0/197697179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b6560b600 0x555b65608a20 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f09cc004a60 tx=0x7f09cc004b40 comp rx=0 tx=0).stop 2024-03-23T17:51:03.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.307+0000 7f09ef535b80 1 -- 172.21.15.86:0/197697179 shutdown_connections 2024-03-23T17:51:03.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.307+0000 7f09ef535b80 1 --2- 172.21.15.86:0/197697179 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x555b6561b300 0x555b6561d720 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:03.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.307+0000 7f09ef535b80 1 --2- 172.21.15.86:0/197697179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b6560b600 0x555b65608a20 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:03.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.307+0000 7f09ef535b80 1 --2- 172.21.15.86:0/197697179 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555b655f75d0 0x555b6560b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:03.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.307+0000 7f09ef535b80 1 --2- 172.21.15.86:0/197697179 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b655f7ef0 0x555b656082e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:03.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.307+0000 7f09baa00700 0 client.15606 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:51:03.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.307+0000 7f09ef535b80 1 -- 172.21.15.86:0/197697179 >> 172.21.15.86:0/197697179 conn(0x555b655576a0 msgr2=0x555b655f96c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:03.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.308+0000 7f09ef535b80 1 -- 172.21.15.86:0/197697179 shutdown_connections 2024-03-23T17:51:03.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.308+0000 7f09ef535b80 1 -- 172.21.15.86:0/197697179 wait complete. 2024-03-23T17:51:03.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.308+0000 7f09ef535b80 20 client.15606 trim_cache size 0 max 16384 2024-03-23T17:51:03.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.325+0000 7f09ef535b80 1 Processor -- start 2024-03-23T17:51:03.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.325+0000 7f09ef535b80 1 -- start start 2024-03-23T17:51:03.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.325+0000 7f09ef535b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b655f75d0 0x555b65622040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:03.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.325+0000 7f09ef535b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b655f7ef0 0x555b65612280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:03.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.325+0000 7f09ef535b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555b6560b600 0x555b65607bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:03.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.325+0000 7f09ef535b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b65609020 con 0x555b655f7ef0 2024-03-23T17:51:03.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.325+0000 7f09ef535b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b6561dd80 con 0x555b6560b600 2024-03-23T17:51:03.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.325+0000 7f09ef535b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b65606830 con 0x555b655f75d0 2024-03-23T17:51:03.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.326+0000 7f09d9a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555b6560b600 0x555b65607bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:03.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.326+0000 7f09d9000700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b655f75d0 0x555b65622040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:03.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.326+0000 7f09d9a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555b6560b600 0x555b65607bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:58512/0 (socket says 172.21.15.86:58512) 2024-03-23T17:51:03.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.326+0000 7f09d9a00700 1 -- 172.21.15.86:0/218425343 learned_addr learned my addr 172.21.15.86:0/218425343 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:03.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.326+0000 7f09d3e00700 1 --2- 172.21.15.86:0/218425343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b655f7ef0 0x555b65612280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:03.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.326+0000 7f09d9a00700 1 -- 172.21.15.86:0/218425343 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b655f75d0 msgr2=0x555b65622040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:03.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.326+0000 7f09d9a00700 1 --2- 172.21.15.86:0/218425343 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b655f75d0 0x555b65622040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:03.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.326+0000 7f09d9a00700 1 -- 172.21.15.86:0/218425343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b655f7ef0 msgr2=0x555b65612280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:03.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.326+0000 7f09d9a00700 1 --2- 172.21.15.86:0/218425343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b655f7ef0 0x555b65612280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:03.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.326+0000 7f09d9a00700 1 -- 172.21.15.86:0/218425343 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555b6550fea0 con 0x555b6560b600 2024-03-23T17:51:03.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.326+0000 7f09d9000700 1 --2- 172.21.15.86:0/218425343 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b655f75d0 0x555b65622040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:51:03.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.327+0000 7f09d9a00700 1 --2- 172.21.15.86:0/218425343 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555b6560b600 0x555b65607bc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f09cc004b10 tx=0x7f09cc003c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44108b4fa4133fe5 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:03.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.327+0000 7f09c6a00700 1 -- 172.21.15.86:0/218425343 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09cc0077d0 con 0x555b6560b600 2024-03-23T17:51:03.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.327+0000 7f09c6a00700 1 -- 172.21.15.86:0/218425343 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09cc00f040 con 0x555b6560b600 2024-03-23T17:51:03.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.327+0000 7f09c6a00700 1 -- 172.21.15.86:0/218425343 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09cc016c20 con 0x555b6560b600 2024-03-23T17:51:03.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.328+0000 7f09ef535b80 1 -- 172.21.15.86:0/218425343 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555b6560b600 msgr2=0x555b65607bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:03.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.328+0000 7f09ef535b80 1 --2- 172.21.15.86:0/218425343 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555b6560b600 0x555b65607bc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f09cc004b10 tx=0x7f09cc003c80 comp rx=0 tx=0).stop 2024-03-23T17:51:03.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.328+0000 7f09ef535b80 1 -- 172.21.15.86:0/218425343 shutdown_connections 2024-03-23T17:51:03.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.328+0000 7f09ef535b80 1 --2- 172.21.15.86:0/218425343 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555b6560b600 0x555b65607bc0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:03.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.328+0000 7f09ef535b80 1 --2- 172.21.15.86:0/218425343 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b655f75d0 0x555b65622040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:03.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.328+0000 7f09ef535b80 1 --2- 172.21.15.86:0/218425343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b655f7ef0 0x555b65612280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:03.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.328+0000 7f09ef535b80 1 -- 172.21.15.86:0/218425343 >> 172.21.15.86:0/218425343 conn(0x555b655576a0 msgr2=0x555b655ec5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:03.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.328+0000 7f09ef535b80 1 -- 172.21.15.86:0/218425343 shutdown_connections 2024-03-23T17:51:03.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.328+0000 7f09ef535b80 1 -- 172.21.15.86:0/218425343 wait complete. 2024-03-23T17:51:03.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.329+0000 7f09ef535b80 1 -- start start 2024-03-23T17:51:03.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.329+0000 7f09ef535b80 1 Processor -- start 2024-03-23T17:51:03.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.330+0000 7f09ef535b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b655f75d0 0x555b65610930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:03.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.330+0000 7f09ef535b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b655f7ef0 0x555b65610e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:03.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.330+0000 7f09ef535b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555b6560b600 0x555b656113b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:03.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.330+0000 7f09ef535b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b6566cdf0 con 0x555b655f75d0 2024-03-23T17:51:03.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.330+0000 7f09ef535b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b6566cf30 con 0x555b6560b600 2024-03-23T17:51:03.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.330+0000 7f09ef535b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b6566d0f0 con 0x555b655f7ef0 2024-03-23T17:51:03.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.330+0000 7f09ef535b80 10 client.0 _ll_register_callbacks cb 0x555b654ef470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:51:03.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.331+0000 7f09d3e00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b655f7ef0 0x555b65610e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:03.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.331+0000 7f09d3e00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b655f7ef0 0x555b65610e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:54276/0 (socket says 172.21.15.86:54276) 2024-03-23T17:51:03.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.331+0000 7f09d3e00700 1 -- 172.21.15.86:0/299453107 learned_addr learned my addr 172.21.15.86:0/299453107 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:03.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.331+0000 7f09d9a00700 1 --2- 172.21.15.86:0/299453107 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555b6560b600 0x555b656113b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:03.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.331+0000 7f09d9000700 1 --2- 172.21.15.86:0/299453107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b655f75d0 0x555b65610930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:03.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.331+0000 7f09d9a00700 1 -- 172.21.15.86:0/299453107 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b655f7ef0 msgr2=0x555b65610e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:03.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.331+0000 7f09d9a00700 1 --2- 172.21.15.86:0/299453107 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b655f7ef0 0x555b65610e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:03.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.331+0000 7f09d9a00700 1 -- 172.21.15.86:0/299453107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b655f75d0 msgr2=0x555b65610930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:03.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.331+0000 7f09d9a00700 1 --2- 172.21.15.86:0/299453107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b655f75d0 0x555b65610930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:03.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.331+0000 7f09d9a00700 1 -- 172.21.15.86:0/299453107 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09d410e470 con 0x555b6560b600 2024-03-23T17:51:03.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.331+0000 7f09d9000700 1 --2- 172.21.15.86:0/299453107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b655f75d0 0x555b65610930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:51:03.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.331+0000 7f09d3e00700 1 --2- 172.21.15.86:0/299453107 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b655f7ef0 0x555b65610e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:51:03.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.331+0000 7f09d9a00700 1 --2- 172.21.15.86:0/299453107 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555b6560b600 0x555b656113b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f09cc016fd0 tx=0x7f09cc016620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=406082407e8df0c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:03.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.331+0000 7f09baa00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T17:51:03.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.332+0000 7f09baa00700 1 -- 172.21.15.86:0/299453107 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09cc016d90 con 0x555b6560b600 2024-03-23T17:51:03.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.332+0000 7f09baa00700 1 -- 172.21.15.86:0/299453107 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f09cc007580 con 0x555b6560b600 2024-03-23T17:51:03.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.332+0000 7f09ef535b80 1 -- 172.21.15.86:0/299453107 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f09c009dda0 con 0x555b6560b600 2024-03-23T17:51:03.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.332+0000 7f09baa00700 1 -- 172.21.15.86:0/299453107 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f09cc007740 con 0x555b6560b600 2024-03-23T17:51:03.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.332+0000 7f09ef535b80 10 client.25664 fetch_fsmap learned FSMap version 28 2024-03-23T17:51:03.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.332+0000 7f09ef535b80 1 -- 172.21.15.86:0/299453107 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x555b65377b60 con 0x555b6560b600 2024-03-23T17:51:03.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.332+0000 7f09baa00700 1 -- 172.21.15.86:0/299453107 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09cc04a080 con 0x555b6560b600 2024-03-23T17:51:03.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.332+0000 7f09baa00700 1 -- 172.21.15.86:0/299453107 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 28) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f09cc034850 con 0x555b6560b600 2024-03-23T17:51:03.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.332+0000 7f09ef535b80 10 client.25664 fetch_fsmap finished waiting for FSMap version 28 2024-03-23T17:51:03.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.332+0000 7f09ef535b80 10 client.25664 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T17:51:03.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.332+0000 7f09ef535b80 20 client.25664 populate_metadata read hostname 'smithi086' 2024-03-23T17:51:03.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.332+0000 7f09ef535b80 1 --2- 172.21.15.86:0/299453107 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x555b6566e560 0x555b65670950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:03.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.332+0000 7f09ef535b80 4 client.25664 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-23T17:51:03.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.332+0000 7f09ef535b80 1 -- 172.21.15.86:0/299453107 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555b65398cf0 con 0x555b6566e560 2024-03-23T17:51:03.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.333+0000 7f09d9000700 1 --2- 172.21.15.86:0/299453107 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x555b6566e560 0x555b65670950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:03.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.333+0000 7f09d9000700 1 --2- 172.21.15.86:0/299453107 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x555b6566e560 0x555b65670950 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a04212a022a3f3d3 server_cookie=52d8ba8c5a1d2db1 in_seq=0 out_seq=0 2024-03-23T17:51:03.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.333+0000 7f09baa00700 10 client.25664 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:51:03.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.334+0000 7f09baa00700 1 -- 172.21.15.86:0/299453107 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x555b65398cf0 con 0x555b6566e560 2024-03-23T17:51:03.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.334+0000 7f09baa00700 10 client.25664 handle_command_reply: tid=0 2024-03-23T17:51:03.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.334+0000 7f09ef535b80 1 client.25664 shutdown 2024-03-23T17:51:03.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.334+0000 7f09ef535b80 1 -- 172.21.15.86:0/299453107 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555b6560b600 msgr2=0x555b656113b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:03.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.334+0000 7f09ef535b80 1 --2- 172.21.15.86:0/299453107 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555b6560b600 0x555b656113b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f09cc016fd0 tx=0x7f09cc016620 comp rx=0 tx=0).stop 2024-03-23T17:51:03.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.334+0000 7f09ef535b80 1 -- 172.21.15.86:0/299453107 shutdown_connections 2024-03-23T17:51:03.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.334+0000 7f09ef535b80 1 --2- 172.21.15.86:0/299453107 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x555b6566e560 0x555b65670950 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:03.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.334+0000 7f09ef535b80 1 --2- 172.21.15.86:0/299453107 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555b6560b600 0x555b656113b0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:03.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.334+0000 7f09ef535b80 1 --2- 172.21.15.86:0/299453107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b655f75d0 0x555b65610930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:03.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.334+0000 7f09ef535b80 1 --2- 172.21.15.86:0/299453107 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b655f7ef0 0x555b65610e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:03.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.334+0000 7f09ef535b80 1 -- 172.21.15.86:0/299453107 >> 172.21.15.86:0/299453107 conn(0x555b655576a0 msgr2=0x555b656035c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:03.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.334+0000 7f09baa00700 0 client.25664 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:51:03.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.334+0000 7f09ef535b80 1 -- 172.21.15.86:0/299453107 shutdown_connections 2024-03-23T17:51:03.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.334+0000 7f09ef535b80 1 -- 172.21.15.86:0/299453107 wait complete. 2024-03-23T17:51:03.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.335+0000 7f09ef535b80 20 client.25664 trim_cache size 0 max 16384 2024-03-23T17:51:03.338 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-23T17:51:03.338 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-23T17:51:03.338 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-03-23T17:51:03.338 INFO:teuthology.orchestra.run.smithi086.stdout: "69575a08-bb4f-4558-b2f2-487c0ea7a5a9": { 2024-03-23T17:51:03.338 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-03-23T17:51:03.338 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "69575a08-bb4f-4558-b2f2-487c0ea7a5a9", 2024-03-23T17:51:03.338 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-03-23T17:51:03.338 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T17:51:03.338 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T17:51:03.339 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-23T17:51:03.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.335+0000 7f09dba00700 1 -- 172.21.15.86:0/1403169144 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f09bc0a13b0 msgr2=0x7f09bc0a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:03.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.335+0000 7f09dba00700 1 --2- 172.21.15.86:0/1403169144 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f09bc0a13b0 0x7f09bc0a3870 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f09cc009550 tx=0x7f09cc009130 comp rx=0 tx=0).stop 2024-03-23T17:51:03.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.335+0000 7f09dba00700 1 -- 172.21.15.86:0/1403169144 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f09d4106080 msgr2=0x7f09d4199990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:03.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.335+0000 7f09dba00700 1 --2- 172.21.15.86:0/1403169144 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f09d4106080 0x7f09d4199990 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f09c000e9c0 tx=0x7f09c000ecd0 comp rx=0 tx=0).stop 2024-03-23T17:51:03.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.335+0000 7f09dba00700 1 -- 172.21.15.86:0/1403169144 shutdown_connections 2024-03-23T17:51:03.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.335+0000 7f09dba00700 1 --2- 172.21.15.86:0/1403169144 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09d419dd30 0x7f09d419e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:03.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.335+0000 7f09dba00700 1 --2- 172.21.15.86:0/1403169144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09d4100f60 0x7f09d4199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:03.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.335+0000 7f09dba00700 1 --2- 172.21.15.86:0/1403169144 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f09bc0a13b0 0x7f09bc0a3870 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:03.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.335+0000 7f09dba00700 1 --2- 172.21.15.86:0/1403169144 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f09d4106080 0x7f09d4199990 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:03.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.335+0000 7f09dba00700 1 -- 172.21.15.86:0/1403169144 >> 172.21.15.86:0/1403169144 conn(0x7f09d40fc7c0 msgr2=0x7f09d40fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:03.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.336+0000 7f09dba00700 1 -- 172.21.15.86:0/1403169144 shutdown_connections 2024-03-23T17:51:03.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:03.336+0000 7f09dba00700 1 -- 172.21.15.86:0/1403169144 wait complete. 2024-03-23T17:51:03.360 INFO:tasks.cephfs.filesystem:scrub status for tag:69575a08-bb4f-4558-b2f2-487c0ea7a5a9 - {'path': '/', 'tag': '69575a08-bb4f-4558-b2f2-487c0ea7a5a9', 'options': 'recursive,force'} 2024-03-23T17:51:04.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:51:03 smithi086 ceph-mon[38221]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-03-23T17:51:04.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:51:03 smithi086 ceph-mon[38221]: scrub queued for path: / 2024-03-23T17:51:04.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:51:03 smithi086 ceph-mon[38221]: scrub summary: idle+waiting paths [/] 2024-03-23T17:51:04.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:51:03 smithi086 ceph-mon[38221]: scrub summary: active paths [/] 2024-03-23T17:51:04.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:51:03 smithi086 ceph-mon[38221]: scrub summary: idle+waiting paths [/] 2024-03-23T17:51:04.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:51:03 smithi092 ceph-mon[79089]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-03-23T17:51:04.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:51:03 smithi092 ceph-mon[79089]: scrub queued for path: / 2024-03-23T17:51:04.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:51:03 smithi092 ceph-mon[79089]: scrub summary: idle+waiting paths [/] 2024-03-23T17:51:04.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:51:03 smithi092 ceph-mon[79089]: scrub summary: active paths [/] 2024-03-23T17:51:04.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:51:03 smithi092 ceph-mon[79089]: scrub summary: idle+waiting paths [/] 2024-03-23T17:51:04.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:51:03 smithi152 ceph-mon[47949]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-03-23T17:51:04.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:51:03 smithi152 ceph-mon[47949]: scrub queued for path: / 2024-03-23T17:51:04.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:51:03 smithi152 ceph-mon[47949]: scrub summary: idle+waiting paths [/] 2024-03-23T17:51:04.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:51:03 smithi152 ceph-mon[47949]: scrub summary: active paths [/] 2024-03-23T17:51:04.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:51:03 smithi152 ceph-mon[47949]: scrub summary: idle+waiting paths [/] 2024-03-23T17:51:05.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:51:04 smithi086 ceph-mon[38221]: scrub summary: idle 2024-03-23T17:51:05.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:51:04 smithi092 ceph-mon[79089]: scrub summary: idle 2024-03-23T17:51:05.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:51:04 smithi152 ceph-mon[47949]: scrub summary: idle 2024-03-23T17:51:06.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:51:05 smithi086 ceph-mon[38221]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-03-23T17:51:06.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:51:05 smithi092 ceph-mon[79089]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-03-23T17:51:06.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:51:05 smithi152 ceph-mon[47949]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-03-23T17:51:07.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:51:06 smithi086 ceph-mon[38221]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 24 KiB/s wr, 33 op/s 2024-03-23T17:51:07.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:51:06 smithi092 ceph-mon[79089]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 24 KiB/s wr, 33 op/s 2024-03-23T17:51:07.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:51:06 smithi152 ceph-mon[47949]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 24 KiB/s wr, 33 op/s 2024-03-23T17:51:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.800+0000 7f8f55000700 1 -- 172.21.15.86:0/87743862 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f50100ef0 msgr2=0x7f8f501012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.800+0000 7f8f55000700 1 --2- 172.21.15.86:0/87743862 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f50100ef0 0x7f8f501012d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8f400098f0 tx=0x7f8f40009c00 comp rx=0 tx=0).stop 2024-03-23T17:51:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.800+0000 7f8f55000700 1 -- 172.21.15.86:0/87743862 shutdown_connections 2024-03-23T17:51:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.800+0000 7f8f55000700 1 --2- 172.21.15.86:0/87743862 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f50106010 0x7f8f5010db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.800+0000 7f8f55000700 1 --2- 172.21.15.86:0/87743862 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f50100ef0 0x7f8f501012d0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.800+0000 7f8f55000700 1 --2- 172.21.15.86:0/87743862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f501018a0 0x7f8f501058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.800+0000 7f8f55000700 1 -- 172.21.15.86:0/87743862 >> 172.21.15.86:0/87743862 conn(0x7f8f500fc750 msgr2=0x7f8f500feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.800+0000 7f8f55000700 1 -- 172.21.15.86:0/87743862 shutdown_connections 2024-03-23T17:51:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.800+0000 7f8f55000700 1 -- 172.21.15.86:0/87743862 wait complete. 2024-03-23T17:51:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.802+0000 7f8f55000700 1 Processor -- start 2024-03-23T17:51:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.802+0000 7f8f55000700 1 -- start start 2024-03-23T17:51:07.803 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.803+0000 7f8f55000700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f50100ef0 0x7f8f500730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:07.803 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.803+0000 7f8f55000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f50106010 0x7f8f500735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:07.803 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.803+0000 7f8f55000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f50073b30 0x7f8f501a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:07.803 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.803+0000 7f8f55000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f501a59a0 con 0x7f8f50106010 2024-03-23T17:51:07.804 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.803+0000 7f8f55000700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f501a5b10 con 0x7f8f50073b30 2024-03-23T17:51:07.804 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.803+0000 7f8f55000700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f501a5cd0 con 0x7f8f50100ef0 2024-03-23T17:51:07.804 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.803+0000 7f8f4d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f50106010 0x7f8f500735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.803+0000 7f8f4de00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f50100ef0 0x7f8f500730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.803+0000 7f8f4d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f50106010 0x7f8f500735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37448/0 (socket says 172.21.15.86:37448) 2024-03-23T17:51:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.803+0000 7f8f4d400700 1 -- 172.21.15.86:0/3448721061 learned_addr learned my addr 172.21.15.86:0/3448721061 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.803+0000 7f8f4e800700 1 --2- 172.21.15.86:0/3448721061 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f50073b30 0x7f8f501a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.804+0000 7f8f4d400700 1 -- 172.21.15.86:0/3448721061 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f50100ef0 msgr2=0x7f8f500730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.804+0000 7f8f4d400700 1 --2- 172.21.15.86:0/3448721061 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f50100ef0 0x7f8f500730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.804+0000 7f8f4d400700 1 -- 172.21.15.86:0/3448721061 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f50073b30 msgr2=0x7f8f501a53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.804+0000 7f8f4d400700 1 --2- 172.21.15.86:0/3448721061 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f50073b30 0x7f8f501a53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.804+0000 7f8f4d400700 1 -- 172.21.15.86:0/3448721061 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f3c0095f0 con 0x7f8f50106010 2024-03-23T17:51:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.804+0000 7f8f4e800700 1 --2- 172.21.15.86:0/3448721061 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f50073b30 0x7f8f501a53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:51:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.804+0000 7f8f4d400700 1 --2- 172.21.15.86:0/3448721061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f50106010 0x7f8f500735f0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c00ec70 tx=0x7f8f3c00ef80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.805+0000 7f8f46a00700 1 -- 172.21.15.86:0/3448721061 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c0099d0 con 0x7f8f50106010 2024-03-23T17:51:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.805+0000 7f8f46a00700 1 -- 172.21.15.86:0/3448721061 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f3c0044e0 con 0x7f8f50106010 2024-03-23T17:51:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.805+0000 7f8f46a00700 1 -- 172.21.15.86:0/3448721061 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c010430 con 0x7f8f50106010 2024-03-23T17:51:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.805+0000 7f8f55000700 1 -- 172.21.15.86:0/3448721061 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f40009520 con 0x7f8f50106010 2024-03-23T17:51:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.805+0000 7f8f55000700 1 -- 172.21.15.86:0/3448721061 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f501a6250 con 0x7f8f50106010 2024-03-23T17:51:07.811 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.811+0000 7f8f46a00700 1 -- 172.21.15.86:0/3448721061 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) v1 ==== 136098+0+0 (secure 0 0 0) 0x7f8f3c003ef0 con 0x7f8f50106010 2024-03-23T17:51:07.813 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.813+0000 7f8f46a00700 1 --2- 172.21.15.86:0/3448721061 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f8f340a1500 0x7f8f340a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:07.814 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.813+0000 7f8f46a00700 1 -- 172.21.15.86:0/3448721061 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f8f3c014070 con 0x7f8f50106010 2024-03-23T17:51:07.814 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.813+0000 7f8f68be5b80 1 Processor -- start 2024-03-23T17:51:07.814 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.814+0000 7f8f4de00700 1 --2- 172.21.15.86:0/3448721061 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f8f340a1500 0x7f8f340a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:07.814 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.814+0000 7f8f68be5b80 1 -- start start 2024-03-23T17:51:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.814+0000 7f8f68be5b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db24373910 0x55db24373cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.814+0000 7f8f68be5b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55db24374230 0x55db2437eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.814+0000 7f8f68be5b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55db2437f0e0 0x55db243814d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.814+0000 7f8f68be5b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db242c4930 con 0x55db24373910 2024-03-23T17:51:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.814+0000 7f8f68be5b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db242c22f0 con 0x55db2437f0e0 2024-03-23T17:51:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.814+0000 7f8f68be5b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db242bdbe0 con 0x55db24374230 2024-03-23T17:51:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.815+0000 7f8f4d400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55db24374230 0x55db2437eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.815+0000 7f8f4de00700 1 --2- 172.21.15.86:0/3448721061 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f8f340a1500 0x7f8f340a39c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8f400098c0 tx=0x7f8f40019040 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.815+0000 7f8f4e800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55db2437f0e0 0x55db243814d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.815+0000 7f8f4d400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55db24374230 0x55db2437eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:48232/0 (socket says 172.21.15.86:48232) 2024-03-23T17:51:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.815+0000 7f8f4d400700 1 -- 172.21.15.86:0/699758931 learned_addr learned my addr 172.21.15.86:0/699758931 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.815+0000 7f8f4d400700 1 -- 172.21.15.86:0/699758931 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55db2437f0e0 msgr2=0x55db243814d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:07.817 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.815+0000 7f8f4d400700 1 --2- 172.21.15.86:0/699758931 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55db2437f0e0 0x55db243814d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:07.817 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.815+0000 7f8f4d400700 1 -- 172.21.15.86:0/699758931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db24373910 msgr2=0x55db24373cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:07.817 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.815+0000 7f8f4d400700 1 --2- 172.21.15.86:0/699758931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db24373910 0x55db24373cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:07.817 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.815+0000 7f8f4d400700 1 -- 172.21.15.86:0/699758931 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55db2413f350 con 0x55db24374230 2024-03-23T17:51:07.817 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.815+0000 7f8f4d400700 1 --2- 172.21.15.86:0/699758931 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55db24374230 0x55db2437eba0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c00c5b0 tx=0x7f8f3c099b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4392733e794f977a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:07.817 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.816+0000 7f8f3b400700 1 -- 172.21.15.86:0/699758931 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c09a900 con 0x55db24374230 2024-03-23T17:51:07.817 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.816+0000 7f8f3b400700 1 -- 172.21.15.86:0/699758931 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f3c09a200 con 0x55db24374230 2024-03-23T17:51:07.817 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.816+0000 7f8f3b400700 1 -- 172.21.15.86:0/699758931 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c0986a0 con 0x55db24374230 2024-03-23T17:51:07.818 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.817+0000 7f8f68be5b80 1 -- 172.21.15.86:0/699758931 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55db24374230 msgr2=0x55db2437eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:07.818 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.817+0000 7f8f68be5b80 1 --2- 172.21.15.86:0/699758931 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55db24374230 0x55db2437eba0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c00c5b0 tx=0x7f8f3c099b60 comp rx=0 tx=0).stop 2024-03-23T17:51:07.818 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.817+0000 7f8f68be5b80 1 -- 172.21.15.86:0/699758931 shutdown_connections 2024-03-23T17:51:07.818 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.817+0000 7f8f68be5b80 1 --2- 172.21.15.86:0/699758931 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55db2437f0e0 0x55db243814d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:07.818 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.817+0000 7f8f68be5b80 1 --2- 172.21.15.86:0/699758931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db24373910 0x55db24373cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:07.818 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.817+0000 7f8f68be5b80 1 --2- 172.21.15.86:0/699758931 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55db24374230 0x55db2437eba0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:07.818 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.817+0000 7f8f68be5b80 1 -- 172.21.15.86:0/699758931 >> 172.21.15.86:0/699758931 conn(0x55db242cb640 msgr2=0x55db24367520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:07.819 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.817+0000 7f8f68be5b80 1 -- 172.21.15.86:0/699758931 shutdown_connections 2024-03-23T17:51:07.819 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.817+0000 7f8f68be5b80 1 -- 172.21.15.86:0/699758931 wait complete. 2024-03-23T17:51:07.819 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.818+0000 7f8f68be5b80 1 -- start start 2024-03-23T17:51:07.819 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.818+0000 7f8f68be5b80 1 Processor -- start 2024-03-23T17:51:07.820 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.819+0000 7f8f68be5b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55db24373910 0x55db2437eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:07.821 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.819+0000 7f8f68be5b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55db24374230 0x55db2437be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:07.821 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.819+0000 7f8f68be5b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db2437f0e0 0x55db2437c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:07.821 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.819+0000 7f8f68be5b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db24259d20 con 0x55db2437f0e0 2024-03-23T17:51:07.821 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.819+0000 7f8f68be5b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db242599c0 con 0x55db24374230 2024-03-23T17:51:07.821 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.819+0000 7f8f68be5b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db24141640 con 0x55db24373910 2024-03-23T17:51:07.821 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.819+0000 7f8f68be5b80 10 client.0 _ll_register_callbacks cb 0x55db24263410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:51:07.821 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.819+0000 7f8f4de00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55db24373910 0x55db2437eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:07.821 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.819+0000 7f8f4de00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55db24373910 0x55db2437eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:48236/0 (socket says 172.21.15.86:48236) 2024-03-23T17:51:07.821 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.819+0000 7f8f4de00700 1 -- 172.21.15.86:0/4259480331 learned_addr learned my addr 172.21.15.86:0/4259480331 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:07.821 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.819+0000 7f8f4d400700 1 --2- 172.21.15.86:0/4259480331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55db24374230 0x55db2437be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:07.821 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.820+0000 7f8f4e800700 1 --2- 172.21.15.86:0/4259480331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db2437f0e0 0x55db2437c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:07.821 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.820+0000 7f8f4d400700 1 -- 172.21.15.86:0/4259480331 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55db24373910 msgr2=0x55db2437eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:07.821 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.820+0000 7f8f4d400700 1 --2- 172.21.15.86:0/4259480331 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55db24373910 0x55db2437eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:07.822 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.820+0000 7f8f4d400700 1 -- 172.21.15.86:0/4259480331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db2437f0e0 msgr2=0x55db2437c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:07.822 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.820+0000 7f8f4d400700 1 --2- 172.21.15.86:0/4259480331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db2437f0e0 0x55db2437c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:07.822 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.820+0000 7f8f4d400700 1 -- 172.21.15.86:0/4259480331 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f501a6250 con 0x55db24374230 2024-03-23T17:51:07.822 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.820+0000 7f8f4d400700 1 --2- 172.21.15.86:0/4259480331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55db24374230 0x55db2437be40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c00c5b0 tx=0x7f8f3c0ca8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22ed6d361784dabc server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:07.822 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.820+0000 7f8f2fe00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T17:51:07.822 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.820+0000 7f8f2fe00700 1 -- 172.21.15.86:0/4259480331 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c0a2680 con 0x55db24374230 2024-03-23T17:51:07.822 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.820+0000 7f8f2fe00700 1 -- 172.21.15.86:0/4259480331 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f8f3c09a460 con 0x55db24374230 2024-03-23T17:51:07.822 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.820+0000 7f8f68be5b80 1 -- 172.21.15.86:0/4259480331 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55db2427e320 con 0x55db24374230 2024-03-23T17:51:07.822 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.821+0000 7f8f2fe00700 1 -- 172.21.15.86:0/4259480331 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f3c09a660 con 0x55db24374230 2024-03-23T17:51:07.822 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.821+0000 7f8f68be5b80 10 client.25676 fetch_fsmap learned FSMap version 28 2024-03-23T17:51:07.822 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.821+0000 7f8f68be5b80 1 -- 172.21.15.86:0/4259480331 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55db240ebb00 con 0x55db24374230 2024-03-23T17:51:07.824 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.821+0000 7f8f2fe00700 1 -- 172.21.15.86:0/4259480331 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55db2427e320 con 0x55db24374230 2024-03-23T17:51:07.824 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.821+0000 7f8f2fe00700 1 -- 172.21.15.86:0/4259480331 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 28) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8f3c0aa030 con 0x55db24374230 2024-03-23T17:51:07.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.821+0000 7f8f68be5b80 10 client.25676 fetch_fsmap finished waiting for FSMap version 28 2024-03-23T17:51:07.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.821+0000 7f8f68be5b80 10 client.25676 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:51:07.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.821+0000 7f8f68be5b80 20 client.25676 populate_metadata read hostname 'smithi086' 2024-03-23T17:51:07.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.821+0000 7f8f68be5b80 1 --2- 172.21.15.86:0/4259480331 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55db2438e910 0x55db24390d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:07.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.821+0000 7f8f68be5b80 4 client.25676 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:51:07.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.821+0000 7f8f68be5b80 1 -- 172.21.15.86:0/4259480331 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55db242b66a0 con 0x55db2438e910 2024-03-23T17:51:07.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.822+0000 7f8f4de00700 1 --2- 172.21.15.86:0/4259480331 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55db2438e910 0x55db24390d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:07.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.822+0000 7f8f4de00700 1 --2- 172.21.15.86:0/4259480331 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55db2438e910 0x55db24390d00 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=caa5c359e6e4e65f server_cookie=7ad46ab2b40236fb in_seq=0 out_seq=0 2024-03-23T17:51:07.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.822+0000 7f8f2fe00700 10 client.25676 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:51:07.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.823+0000 7f8f2fe00700 1 -- 172.21.15.86:0/4259480331 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55db242b66a0 con 0x55db2438e910 2024-03-23T17:51:07.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.823+0000 7f8f2fe00700 10 client.25676 handle_command_reply: tid=0 2024-03-23T17:51:07.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.823+0000 7f8f68be5b80 1 client.25676 shutdown 2024-03-23T17:51:07.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.823+0000 7f8f68be5b80 1 -- 172.21.15.86:0/4259480331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55db24374230 msgr2=0x55db2437be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:07.826 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.823+0000 7f8f68be5b80 1 --2- 172.21.15.86:0/4259480331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55db24374230 0x55db2437be40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c00c5b0 tx=0x7f8f3c0ca8b0 comp rx=0 tx=0).stop 2024-03-23T17:51:07.826 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.823+0000 7f8f68be5b80 1 -- 172.21.15.86:0/4259480331 shutdown_connections 2024-03-23T17:51:07.826 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.823+0000 7f8f68be5b80 1 --2- 172.21.15.86:0/4259480331 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55db2438e910 0x55db24390d00 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:07.826 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.823+0000 7f8f68be5b80 1 --2- 172.21.15.86:0/4259480331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db2437f0e0 0x55db2437c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:07.826 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.823+0000 7f8f68be5b80 1 --2- 172.21.15.86:0/4259480331 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55db24373910 0x55db2437eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:07.826 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.823+0000 7f8f2fe00700 0 client.25676 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:51:07.826 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.823+0000 7f8f68be5b80 1 --2- 172.21.15.86:0/4259480331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55db24374230 0x55db2437be40 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:07.826 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.823+0000 7f8f68be5b80 1 -- 172.21.15.86:0/4259480331 >> 172.21.15.86:0/4259480331 conn(0x55db242cb640 msgr2=0x55db24375870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:07.826 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.824+0000 7f8f68be5b80 1 -- 172.21.15.86:0/4259480331 shutdown_connections 2024-03-23T17:51:07.826 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.824+0000 7f8f68be5b80 1 -- 172.21.15.86:0/4259480331 wait complete. 2024-03-23T17:51:07.826 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.824+0000 7f8f68be5b80 20 client.25676 trim_cache size 0 max 16384 2024-03-23T17:51:07.846 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.845+0000 7f8f68be5b80 1 Processor -- start 2024-03-23T17:51:07.846 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.846+0000 7f8f68be5b80 1 -- start start 2024-03-23T17:51:07.846 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.846+0000 7f8f68be5b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55db24373910 0x55db24385df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:07.846 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.846+0000 7f8f68be5b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55db24374230 0x55db24375f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:07.846 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.846+0000 7f8f68be5b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db2437f0e0 0x55db24361500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:07.847 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.846+0000 7f8f68be5b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db2437cda0 con 0x55db2437f0e0 2024-03-23T17:51:07.847 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.846+0000 7f8f68be5b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db2438b8a0 con 0x55db24374230 2024-03-23T17:51:07.847 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.846+0000 7f8f68be5b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db24368ac0 con 0x55db24373910 2024-03-23T17:51:07.848 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.846+0000 7f8f4de00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55db24373910 0x55db24385df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:07.848 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.846+0000 7f8f4d400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55db24374230 0x55db24375f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:07.849 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.846+0000 7f8f4de00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55db24373910 0x55db24385df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:48252/0 (socket says 172.21.15.86:48252) 2024-03-23T17:51:07.849 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.846+0000 7f8f4de00700 1 -- 172.21.15.86:0/1812087946 learned_addr learned my addr 172.21.15.86:0/1812087946 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:07.849 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.847+0000 7f8f4de00700 1 -- 172.21.15.86:0/1812087946 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55db24374230 msgr2=0x55db24375f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:07.849 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.847+0000 7f8f4de00700 1 --2- 172.21.15.86:0/1812087946 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55db24374230 0x55db24375f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:07.849 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.847+0000 7f8f4de00700 1 -- 172.21.15.86:0/1812087946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db2437f0e0 msgr2=0x55db24361500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:07.849 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.847+0000 7f8f4de00700 1 --2- 172.21.15.86:0/1812087946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db2437f0e0 0x55db24361500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:07.849 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.847+0000 7f8f4de00700 1 -- 172.21.15.86:0/1812087946 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55db24283e40 con 0x55db24373910 2024-03-23T17:51:07.849 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.847+0000 7f8f4de00700 1 --2- 172.21.15.86:0/1812087946 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55db24373910 0x55db24385df0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8f4000c010 tx=0x7f8f40028040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1be32ae74e9f3bad server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:07.849 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.847+0000 7f8f3be00700 1 -- 172.21.15.86:0/1812087946 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f4002b040 con 0x55db24373910 2024-03-23T17:51:07.849 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.847+0000 7f8f3be00700 1 -- 172.21.15.86:0/1812087946 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f4001d040 con 0x55db24373910 2024-03-23T17:51:07.849 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.847+0000 7f8f3be00700 1 -- 172.21.15.86:0/1812087946 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f4000b900 con 0x55db24373910 2024-03-23T17:51:07.849 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.848+0000 7f8f68be5b80 1 -- 172.21.15.86:0/1812087946 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55db24373910 msgr2=0x55db24385df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:07.850 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.848+0000 7f8f68be5b80 1 --2- 172.21.15.86:0/1812087946 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55db24373910 0x55db24385df0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8f4000c010 tx=0x7f8f40028040 comp rx=0 tx=0).stop 2024-03-23T17:51:07.850 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.848+0000 7f8f68be5b80 1 -- 172.21.15.86:0/1812087946 shutdown_connections 2024-03-23T17:51:07.850 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.848+0000 7f8f68be5b80 1 --2- 172.21.15.86:0/1812087946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db2437f0e0 0x55db24361500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:07.850 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.848+0000 7f8f68be5b80 1 --2- 172.21.15.86:0/1812087946 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55db24373910 0x55db24385df0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:07.850 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.848+0000 7f8f68be5b80 1 --2- 172.21.15.86:0/1812087946 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55db24374230 0x55db24375f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:07.850 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.848+0000 7f8f68be5b80 1 -- 172.21.15.86:0/1812087946 >> 172.21.15.86:0/1812087946 conn(0x55db242cb640 msgr2=0x55db24375850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:07.850 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.848+0000 7f8f68be5b80 1 -- 172.21.15.86:0/1812087946 shutdown_connections 2024-03-23T17:51:07.850 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.848+0000 7f8f68be5b80 1 -- 172.21.15.86:0/1812087946 wait complete. 2024-03-23T17:51:07.850 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.849+0000 7f8f68be5b80 1 -- start start 2024-03-23T17:51:07.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.849+0000 7f8f68be5b80 1 Processor -- start 2024-03-23T17:51:07.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.849+0000 7f8f68be5b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55db24373910 0x55db24385d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:07.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.849+0000 7f8f68be5b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55db24374230 0x55db243843f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:07.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.849+0000 7f8f68be5b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db2437f0e0 0x55db24384b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:07.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.849+0000 7f8f68be5b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db24385270 con 0x55db2437f0e0 2024-03-23T17:51:07.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.849+0000 7f8f68be5b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db243e0410 con 0x55db24374230 2024-03-23T17:51:07.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.849+0000 7f8f68be5b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db243e0550 con 0x55db24373910 2024-03-23T17:51:07.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.849+0000 7f8f68be5b80 10 client.0 _ll_register_callbacks cb 0x55db24263410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:51:07.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.850+0000 7f8f4de00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55db24373910 0x55db24385d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:07.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.850+0000 7f8f4de00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55db24373910 0x55db24385d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:48258/0 (socket says 172.21.15.86:48258) 2024-03-23T17:51:07.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.850+0000 7f8f4de00700 1 -- 172.21.15.86:0/3177354007 learned_addr learned my addr 172.21.15.86:0/3177354007 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:07.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.850+0000 7f8f4e800700 1 --2- 172.21.15.86:0/3177354007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db2437f0e0 0x55db24384b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:07.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.850+0000 7f8f4de00700 1 -- 172.21.15.86:0/3177354007 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55db24374230 msgr2=0x55db243843f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-23T17:51:07.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.850+0000 7f8f4de00700 1 --2- 172.21.15.86:0/3177354007 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55db24374230 0x55db243843f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:07.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.850+0000 7f8f4de00700 1 -- 172.21.15.86:0/3177354007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db2437f0e0 msgr2=0x55db24384b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:07.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.850+0000 7f8f4de00700 1 --2- 172.21.15.86:0/3177354007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db2437f0e0 0x55db24384b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:07.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.850+0000 7f8f4de00700 1 -- 172.21.15.86:0/3177354007 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f5010e400 con 0x55db24373910 2024-03-23T17:51:07.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.850+0000 7f8f4de00700 1 --2- 172.21.15.86:0/3177354007 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55db24373910 0x55db24385d80 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8f40009810 tx=0x7f8f40000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e38e44ec14686bc5 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:07.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.850+0000 7f8f2fe00700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-03-23T17:51:07.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.851+0000 7f8f2fe00700 1 -- 172.21.15.86:0/3177354007 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f4002b030 con 0x55db24373910 2024-03-23T17:51:07.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.851+0000 7f8f68be5b80 1 -- 172.21.15.86:0/3177354007 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8f3c09a200 con 0x55db24373910 2024-03-23T17:51:07.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.851+0000 7f8f2fe00700 1 -- 172.21.15.86:0/3177354007 <== mon.1 v2:172.21.15.152:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f8f4000b5a0 con 0x55db24373910 2024-03-23T17:51:07.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.851+0000 7f8f2fe00700 1 -- 172.21.15.86:0/3177354007 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f40035070 con 0x55db24373910 2024-03-23T17:51:07.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.851+0000 7f8f68be5b80 10 client.25681 fetch_fsmap learned FSMap version 28 2024-03-23T17:51:07.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.851+0000 7f8f68be5b80 1 -- 172.21.15.86:0/3177354007 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55db24143c70 con 0x55db24373910 2024-03-23T17:51:07.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.851+0000 7f8f2fe00700 1 -- 172.21.15.86:0/3177354007 <== mon.1 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f3c09a200 con 0x55db24373910 2024-03-23T17:51:07.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.852+0000 7f8f2fe00700 1 -- 172.21.15.86:0/3177354007 <== mon.1 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8f40030780 con 0x55db24373910 2024-03-23T17:51:07.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.852+0000 7f8f68be5b80 10 client.25681 fetch_fsmap finished waiting for FSMap version 28 2024-03-23T17:51:07.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.852+0000 7f8f68be5b80 10 client.25681 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:51:07.855 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.852+0000 7f8f68be5b80 20 client.25681 populate_metadata read hostname 'smithi086' 2024-03-23T17:51:07.855 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.852+0000 7f8f68be5b80 1 --2- 172.21.15.86:0/3177354007 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55db243e1860 0x55db243e3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:07.855 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.852+0000 7f8f68be5b80 4 client.25681 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T17:51:07.855 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.852+0000 7f8f68be5b80 1 -- 172.21.15.86:0/3177354007 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55db2410cc90 con 0x55db243e1860 2024-03-23T17:51:07.855 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.852+0000 7f8f4d400700 1 --2- 172.21.15.86:0/3177354007 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55db243e1860 0x55db243e3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:07.856 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.853+0000 7f8f4d400700 1 --2- 172.21.15.86:0/3177354007 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55db243e1860 0x55db243e3c50 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=136df7af6388bcd0 server_cookie=8a4e776c21cd22a1 in_seq=0 out_seq=0 2024-03-23T17:51:07.856 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.853+0000 7f8f2fe00700 10 client.25681 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:51:07.862 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.862+0000 7f8f2fe00700 1 -- 172.21.15.86:0/3177354007 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55db2410cc90 con 0x55db243e1860 2024-03-23T17:51:07.862 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.862+0000 7f8f2fe00700 10 client.25681 handle_command_reply: tid=0 2024-03-23T17:51:07.862 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.862+0000 7f8f68be5b80 1 client.25681 shutdown 2024-03-23T17:51:07.862 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.862+0000 7f8f68be5b80 1 -- 172.21.15.86:0/3177354007 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55db24373910 msgr2=0x55db24385d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:07.862 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.862+0000 7f8f68be5b80 1 --2- 172.21.15.86:0/3177354007 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55db24373910 0x55db24385d80 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8f40009810 tx=0x7f8f40000bc0 comp rx=0 tx=0).stop 2024-03-23T17:51:07.862 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.862+0000 7f8f68be5b80 1 -- 172.21.15.86:0/3177354007 shutdown_connections 2024-03-23T17:51:07.862 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.862+0000 7f8f68be5b80 1 --2- 172.21.15.86:0/3177354007 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55db243e1860 0x55db243e3c50 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:07.863 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.862+0000 7f8f68be5b80 1 --2- 172.21.15.86:0/3177354007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db2437f0e0 0x55db24384b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:07.863 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.862+0000 7f8f68be5b80 1 --2- 172.21.15.86:0/3177354007 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55db24373910 0x55db24385d80 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:07.863 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.862+0000 7f8f68be5b80 1 --2- 172.21.15.86:0/3177354007 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55db24374230 0x55db243843f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:07.863 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.862+0000 7f8f68be5b80 1 -- 172.21.15.86:0/3177354007 >> 172.21.15.86:0/3177354007 conn(0x55db242cb640 msgr2=0x55db243603e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:07.863 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.862+0000 7f8f2fe00700 0 client.25681 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:51:07.863 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.862+0000 7f8f68be5b80 1 -- 172.21.15.86:0/3177354007 shutdown_connections 2024-03-23T17:51:07.864 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.862+0000 7f8f68be5b80 1 -- 172.21.15.86:0/3177354007 wait complete. 2024-03-23T17:51:07.864 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.863+0000 7f8f68be5b80 20 client.25681 trim_cache size 0 max 16384 2024-03-23T17:51:07.864 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-03-23T17:51:07.864 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-03-23T17:51:07.864 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-03-23T17:51:07.864 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-03-23T17:51:07.865 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.863+0000 7f8f55000700 1 -- 172.21.15.86:0/3448721061 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f8f340a1500 msgr2=0x7f8f340a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:07.865 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.863+0000 7f8f55000700 1 --2- 172.21.15.86:0/3448721061 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f8f340a1500 0x7f8f340a39c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8f400098c0 tx=0x7f8f40019040 comp rx=0 tx=0).stop 2024-03-23T17:51:07.865 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.864+0000 7f8f55000700 1 -- 172.21.15.86:0/3448721061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f50106010 msgr2=0x7f8f500735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:07.865 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.864+0000 7f8f55000700 1 --2- 172.21.15.86:0/3448721061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f50106010 0x7f8f500735f0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c00ec70 tx=0x7f8f3c00ef80 comp rx=0 tx=0).stop 2024-03-23T17:51:07.865 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.864+0000 7f8f55000700 1 -- 172.21.15.86:0/3448721061 shutdown_connections 2024-03-23T17:51:07.865 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.864+0000 7f8f55000700 1 --2- 172.21.15.86:0/3448721061 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f50073b30 0x7f8f501a53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:07.865 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.864+0000 7f8f55000700 1 --2- 172.21.15.86:0/3448721061 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f8f340a1500 0x7f8f340a39c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:07.865 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.864+0000 7f8f55000700 1 --2- 172.21.15.86:0/3448721061 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f50100ef0 0x7f8f500730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:07.866 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.864+0000 7f8f55000700 1 --2- 172.21.15.86:0/3448721061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f50106010 0x7f8f500735f0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:07.866 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.864+0000 7f8f55000700 1 -- 172.21.15.86:0/3448721061 >> 172.21.15.86:0/3448721061 conn(0x7f8f500fc750 msgr2=0x7f8f500fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:07.866 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.864+0000 7f8f55000700 1 -- 172.21.15.86:0/3448721061 shutdown_connections 2024-03-23T17:51:07.866 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:07.864+0000 7f8f55000700 1 -- 172.21.15.86:0/3448721061 wait complete. 2024-03-23T17:51:09.766 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:51:09 smithi086 ceph-mon[38221]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 29 op/s 2024-03-23T17:51:09.808 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:51:09 smithi092 ceph-mon[79089]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 29 op/s 2024-03-23T17:51:09.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:51:09 smithi152 ceph-mon[47949]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 29 op/s 2024-03-23T17:51:12.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:51:11 smithi086 ceph-mon[38221]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 26 KiB/s wr, 35 op/s 2024-03-23T17:51:12.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:51:11 smithi092 ceph-mon[79089]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 26 KiB/s wr, 35 op/s 2024-03-23T17:51:12.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:51:11 smithi152 ceph-mon[47949]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 26 KiB/s wr, 35 op/s 2024-03-23T17:51:14.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:51:13 smithi086 ceph-mon[38221]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 26 op/s 2024-03-23T17:51:14.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:51:13 smithi092 ceph-mon[79089]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 26 op/s 2024-03-23T17:51:14.085 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:51:13 smithi152 ceph-mon[47949]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 26 op/s 2024-03-23T17:51:16.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:51:15 smithi086 ceph-mon[38221]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 26 op/s 2024-03-23T17:51:16.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:51:15 smithi092 ceph-mon[79089]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 26 op/s 2024-03-23T17:51:16.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:51:15 smithi152 ceph-mon[47949]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 26 op/s 2024-03-23T17:51:18.018 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:51:17 smithi086 ceph-mon[38221]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 24 KiB/s wr, 31 op/s 2024-03-23T17:51:18.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:51:17 smithi092 ceph-mon[79089]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 24 KiB/s wr, 31 op/s 2024-03-23T17:51:18.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:51:17 smithi152 ceph-mon[47949]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 24 KiB/s wr, 31 op/s 2024-03-23T17:51:20.018 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:51:19 smithi086 ceph-mon[38221]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-03-23T17:51:20.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:51:19 smithi092 ceph-mon[79089]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-03-23T17:51:20.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:51:19 smithi152 ceph-mon[47949]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-03-23T17:51:21.051 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.049+0000 7f759a000700 1 -- 172.21.15.86:0/51816264 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7594073a80 msgr2=0x7f759410d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:21.051 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.049+0000 7f759a000700 1 --2- 172.21.15.86:0/51816264 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7594073a80 0x7f759410d020 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7580009990 tx=0x7f7580009ca0 comp rx=0 tx=0).stop 2024-03-23T17:51:21.051 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.049+0000 7f759a000700 1 -- 172.21.15.86:0/51816264 shutdown_connections 2024-03-23T17:51:21.051 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.049+0000 7f759a000700 1 --2- 172.21.15.86:0/51816264 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f759410d560 0x7f759410f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:21.051 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.049+0000 7f759a000700 1 --2- 172.21.15.86:0/51816264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7594073160 0x7f7594073540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:21.051 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.049+0000 7f759a000700 1 --2- 172.21.15.86:0/51816264 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7594073a80 0x7f759410d020 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:21.051 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.049+0000 7f759a000700 1 -- 172.21.15.86:0/51816264 >> 172.21.15.86:0/51816264 conn(0x7f75940fc770 msgr2=0x7f75940feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:21.051 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.049+0000 7f759a000700 1 -- 172.21.15.86:0/51816264 shutdown_connections 2024-03-23T17:51:21.051 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.050+0000 7f759a000700 1 -- 172.21.15.86:0/51816264 wait complete. 2024-03-23T17:51:21.052 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.051+0000 7f759a000700 1 Processor -- start 2024-03-23T17:51:21.052 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.051+0000 7f759a000700 1 -- start start 2024-03-23T17:51:21.053 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.052+0000 7f759a000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7594073160 0x7f75941993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:21.053 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.052+0000 7f759a000700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7594073a80 0x7f7594199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:21.053 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.052+0000 7f759a000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f759410d560 0x7f759419dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:21.053 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.052+0000 7f759a000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f759419e310 con 0x7f759410d560 2024-03-23T17:51:21.053 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.052+0000 7f759a000700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f759419e480 con 0x7f7594073160 2024-03-23T17:51:21.053 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.052+0000 7f759a000700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f759419e640 con 0x7f7594073a80 2024-03-23T17:51:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.052+0000 7f7593400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7594073160 0x7f75941993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.052+0000 7f7593400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7594073160 0x7f75941993f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:54886/0 (socket says 172.21.15.86:54886) 2024-03-23T17:51:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.052+0000 7f7593400700 1 -- 172.21.15.86:0/2215760127 learned_addr learned my addr 172.21.15.86:0/2215760127 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.052+0000 7f7592a00700 1 --2- 172.21.15.86:0/2215760127 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7594073a80 0x7f7594199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.053+0000 7f7593e00700 1 --2- 172.21.15.86:0/2215760127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f759410d560 0x7f759419dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.053+0000 7f7593400700 1 -- 172.21.15.86:0/2215760127 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7594073a80 msgr2=0x7f7594199930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.053+0000 7f7593400700 1 --2- 172.21.15.86:0/2215760127 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7594073a80 0x7f7594199930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:21.055 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.053+0000 7f7593400700 1 -- 172.21.15.86:0/2215760127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f759410d560 msgr2=0x7f759419dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:21.055 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.053+0000 7f7593400700 1 --2- 172.21.15.86:0/2215760127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f759410d560 0x7f759419dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:21.055 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.053+0000 7f7593400700 1 -- 172.21.15.86:0/2215760127 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75800095f0 con 0x7f7594073160 2024-03-23T17:51:21.055 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.053+0000 7f7592a00700 1 --2- 172.21.15.86:0/2215760127 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7594073a80 0x7f7594199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:51:21.055 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.053+0000 7f7593e00700 1 --2- 172.21.15.86:0/2215760127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f759410d560 0x7f759419dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T17:51:21.055 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.053+0000 7f7593400700 1 --2- 172.21.15.86:0/2215760127 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7594073160 0x7f75941993f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f758400e9a0 tx=0x7f758400ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:21.055 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.053+0000 7f758be00700 1 -- 172.21.15.86:0/2215760127 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758400cbd0 con 0x7f7594073160 2024-03-23T17:51:21.056 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.053+0000 7f758be00700 1 -- 172.21.15.86:0/2215760127 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f758400cd30 con 0x7f7594073160 2024-03-23T17:51:21.056 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.053+0000 7f758be00700 1 -- 172.21.15.86:0/2215760127 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7584018680 con 0x7f7594073160 2024-03-23T17:51:21.056 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.054+0000 7f759a000700 1 -- 172.21.15.86:0/2215760127 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f759419e920 con 0x7f7594073160 2024-03-23T17:51:21.056 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.054+0000 7f759a000700 1 -- 172.21.15.86:0/2215760127 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75941a6160 con 0x7f7594073160 2024-03-23T17:51:21.061 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.060+0000 7f758be00700 1 -- 172.21.15.86:0/2215760127 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 25) v1 ==== 136098+0+0 (secure 0 0 0) 0x7f75840187e0 con 0x7f7594073160 2024-03-23T17:51:21.061 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.060+0000 7f75ada62b80 1 Processor -- start 2024-03-23T17:51:21.062 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.061+0000 7f75ada62b80 1 -- start start 2024-03-23T17:51:21.062 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.061+0000 7f75ada62b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564e6c0e9100 0x564e6c0e94e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:21.062 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.061+0000 7f75ada62b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564e6c0e9a20 0x564e6c0f4390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:21.063 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.061+0000 7f75ada62b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e6c0f48d0 0x564e6c0f6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:21.063 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.061+0000 7f75ada62b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e6c042930 con 0x564e6c0f48d0 2024-03-23T17:51:21.063 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.061+0000 7f75ada62b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e6c0402f0 con 0x564e6c0e9100 2024-03-23T17:51:21.063 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.061+0000 7f75ada62b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e6c03bbe0 con 0x564e6c0e9a20 2024-03-23T17:51:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.061+0000 7f7593400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564e6c0e9a20 0x564e6c0f4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.062+0000 7f7593400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564e6c0e9a20 0x564e6c0f4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:44480/0 (socket says 172.21.15.86:44480) 2024-03-23T17:51:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.062+0000 7f7593400700 1 -- 172.21.15.86:0/1269025662 learned_addr learned my addr 172.21.15.86:0/1269025662 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.062+0000 7f7593e00700 1 --2- 172.21.15.86:0/1269025662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564e6c0e9100 0x564e6c0e94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.062+0000 7f7592a00700 1 --2- 172.21.15.86:0/1269025662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e6c0f48d0 0x564e6c0f6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.062+0000 7f7593400700 1 -- 172.21.15.86:0/1269025662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564e6c0e9100 msgr2=0x564e6c0e94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.062+0000 7f7593400700 1 --2- 172.21.15.86:0/1269025662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564e6c0e9100 0x564e6c0e94e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.062+0000 7f7593400700 1 -- 172.21.15.86:0/1269025662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e6c0f48d0 msgr2=0x564e6c0f6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.062+0000 7f7593400700 1 --2- 172.21.15.86:0/1269025662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e6c0f48d0 0x564e6c0f6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.062+0000 7f7593400700 1 -- 172.21.15.86:0/1269025662 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e6bebd350 con 0x564e6c0e9a20 2024-03-23T17:51:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.062+0000 7f7592a00700 1 --2- 172.21.15.86:0/1269025662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e6c0f48d0 0x564e6c0f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:51:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.062+0000 7f758be00700 1 --2- 172.21.15.86:0/2215760127 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f75780a1430 0x7f75780a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.063+0000 7f758be00700 1 -- 172.21.15.86:0/2215760127 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f7584014070 con 0x7f7594073160 2024-03-23T17:51:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.063+0000 7f7593400700 1 --2- 172.21.15.86:0/1269025662 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564e6c0e9a20 0x564e6c0f4390 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f75840ca900 tx=0x7f75840d5540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1348a9cc41e5100c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.063+0000 7f7593e00700 1 --2- 172.21.15.86:0/2215760127 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f75780a1430 0x7f75780a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:21.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.063+0000 7f757fe00700 1 -- 172.21.15.86:0/1269025662 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758409a020 con 0x564e6c0e9a20 2024-03-23T17:51:21.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.063+0000 7f757fe00700 1 -- 172.21.15.86:0/1269025662 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f75840d0d60 con 0x564e6c0e9a20 2024-03-23T17:51:21.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.063+0000 7f757fe00700 1 -- 172.21.15.86:0/1269025662 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758409a250 con 0x564e6c0e9a20 2024-03-23T17:51:21.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.064+0000 7f75ada62b80 1 -- 172.21.15.86:0/1269025662 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564e6c0e9a20 msgr2=0x564e6c0f4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:21.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.064+0000 7f7593e00700 1 --2- 172.21.15.86:0/2215760127 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f75780a1430 0x7f75780a38f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f758c009d50 tx=0x7f758c00b040 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:21.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.064+0000 7f75ada62b80 1 --2- 172.21.15.86:0/1269025662 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564e6c0e9a20 0x564e6c0f4390 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f75840ca900 tx=0x7f75840d5540 comp rx=0 tx=0).stop 2024-03-23T17:51:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.065+0000 7f75ada62b80 1 -- 172.21.15.86:0/1269025662 shutdown_connections 2024-03-23T17:51:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.065+0000 7f75ada62b80 1 --2- 172.21.15.86:0/1269025662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e6c0f48d0 0x564e6c0f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.065+0000 7f75ada62b80 1 --2- 172.21.15.86:0/1269025662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564e6c0e9100 0x564e6c0e94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.065+0000 7f75ada62b80 1 --2- 172.21.15.86:0/1269025662 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564e6c0e9a20 0x564e6c0f4390 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.065+0000 7f75ada62b80 1 -- 172.21.15.86:0/1269025662 >> 172.21.15.86:0/1269025662 conn(0x564e6c049640 msgr2=0x564e6c0e5520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.065+0000 7f75ada62b80 1 -- 172.21.15.86:0/1269025662 shutdown_connections 2024-03-23T17:51:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.065+0000 7f75ada62b80 1 -- 172.21.15.86:0/1269025662 wait complete. 2024-03-23T17:51:21.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.065+0000 7f75ada62b80 1 -- start start 2024-03-23T17:51:21.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.066+0000 7f75ada62b80 1 Processor -- start 2024-03-23T17:51:21.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.067+0000 7f75ada62b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564e6c0e9100 0x564e6c0f4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:21.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.067+0000 7f75ada62b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e6c0e9a20 0x564e6c0f1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:21.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.067+0000 7f75ada62b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564e6c0f48d0 0x564e6c0f1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:21.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.067+0000 7f75ada62b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e6bfd7d20 con 0x564e6c0e9a20 2024-03-23T17:51:21.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.067+0000 7f75ada62b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e6bfd79c0 con 0x564e6c0f48d0 2024-03-23T17:51:21.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.067+0000 7f75ada62b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e6bebf640 con 0x564e6c0e9100 2024-03-23T17:51:21.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.067+0000 7f75ada62b80 10 client.0 _ll_register_callbacks cb 0x564e6bfe1410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:51:21.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.067+0000 7f7593400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564e6c0e9100 0x564e6c0f4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:21.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.067+0000 7f7592a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e6c0e9a20 0x564e6c0f1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:21.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.067+0000 7f7593400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564e6c0e9100 0x564e6c0f4300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:44496/0 (socket says 172.21.15.86:44496) 2024-03-23T17:51:21.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.067+0000 7f7593400700 1 -- 172.21.15.86:0/1450933161 learned_addr learned my addr 172.21.15.86:0/1450933161 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:21.070 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.068+0000 7f7593e00700 1 --2- 172.21.15.86:0/1450933161 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564e6c0f48d0 0x564e6c0f1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:21.071 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.068+0000 7f7593400700 1 -- 172.21.15.86:0/1450933161 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564e6c0f48d0 msgr2=0x564e6c0f1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.068+0000 7f7593400700 1 --2- 172.21.15.86:0/1450933161 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564e6c0f48d0 0x564e6c0f1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.068+0000 7f7593400700 1 -- 172.21.15.86:0/1450933161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e6c0e9a20 msgr2=0x564e6c0f1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.068+0000 7f7593400700 1 --2- 172.21.15.86:0/1450933161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e6c0e9a20 0x564e6c0f1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.068+0000 7f7593400700 1 -- 172.21.15.86:0/1450933161 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75941a6160 con 0x564e6c0e9100 2024-03-23T17:51:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.068+0000 7f7593400700 1 --2- 172.21.15.86:0/1450933161 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564e6c0e9100 0x564e6c0f4300 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f75840a21a0 tx=0x7f7584096c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68ef1b78b030dec server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.068+0000 7f757cc00700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-03-23T17:51:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.068+0000 7f757cc00700 1 -- 172.21.15.86:0/1450933161 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75840d0760 con 0x564e6c0e9100 2024-03-23T17:51:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.069+0000 7f757cc00700 1 -- 172.21.15.86:0/1450933161 <== mon.1 v2:172.21.15.152:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f758409a950 con 0x564e6c0e9100 2024-03-23T17:51:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.069+0000 7f75ada62b80 1 -- 172.21.15.86:0/1450933161 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564e6c0346a0 con 0x564e6c0e9100 2024-03-23T17:51:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.069+0000 7f757cc00700 1 -- 172.21.15.86:0/1450933161 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f758409a020 con 0x564e6c0e9100 2024-03-23T17:51:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.069+0000 7f75ada62b80 10 client.25693 fetch_fsmap learned FSMap version 28 2024-03-23T17:51:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.069+0000 7f75ada62b80 1 -- 172.21.15.86:0/1450933161 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x564e6be69b00 con 0x564e6c0e9100 2024-03-23T17:51:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.069+0000 7f757cc00700 1 -- 172.21.15.86:0/1450933161 <== mon.1 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f758407f080 con 0x564e6c0e9100 2024-03-23T17:51:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.069+0000 7f757cc00700 1 -- 172.21.15.86:0/1450933161 <== mon.1 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7584099740 con 0x564e6c0e9100 2024-03-23T17:51:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.069+0000 7f75ada62b80 10 client.25693 fetch_fsmap finished waiting for FSMap version 28 2024-03-23T17:51:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.069+0000 7f75ada62b80 10 client.25693 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:51:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.070+0000 7f75ada62b80 20 client.25693 populate_metadata read hostname 'smithi086' 2024-03-23T17:51:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.070+0000 7f75ada62b80 1 --2- 172.21.15.86:0/1450933161 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x564e6c1151f0 0x564e6c1175e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.070+0000 7f75ada62b80 4 client.25693 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:51:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.070+0000 7f75ada62b80 1 -- 172.21.15.86:0/1450933161 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564e6be8ac90 con 0x564e6c1151f0 2024-03-23T17:51:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.070+0000 7f7592a00700 1 --2- 172.21.15.86:0/1450933161 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x564e6c1151f0 0x564e6c1175e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.070+0000 7f757cc00700 10 client.25693 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:51:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.070+0000 7f7592a00700 1 --2- 172.21.15.86:0/1450933161 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x564e6c1151f0 0x564e6c1175e0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3dc98e19fffb2431 server_cookie=8df7f805ed9b20e7 in_seq=0 out_seq=0 2024-03-23T17:51:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.071+0000 7f757cc00700 1 -- 172.21.15.86:0/1450933161 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x564e6be8ac90 con 0x564e6c1151f0 2024-03-23T17:51:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.071+0000 7f757cc00700 10 client.25693 handle_command_reply: tid=0 2024-03-23T17:51:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.071+0000 7f75ada62b80 1 client.25693 shutdown 2024-03-23T17:51:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.072+0000 7f75ada62b80 1 -- 172.21.15.86:0/1450933161 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564e6c0e9100 msgr2=0x564e6c0f4300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.072+0000 7f75ada62b80 1 --2- 172.21.15.86:0/1450933161 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564e6c0e9100 0x564e6c0f4300 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f75840a21a0 tx=0x7f7584096c10 comp rx=0 tx=0).stop 2024-03-23T17:51:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.072+0000 7f75ada62b80 1 -- 172.21.15.86:0/1450933161 shutdown_connections 2024-03-23T17:51:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.072+0000 7f75ada62b80 1 --2- 172.21.15.86:0/1450933161 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x564e6c1151f0 0x564e6c1175e0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.072+0000 7f75ada62b80 1 --2- 172.21.15.86:0/1450933161 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564e6c0f48d0 0x564e6c0f1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.072+0000 7f75ada62b80 1 --2- 172.21.15.86:0/1450933161 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564e6c0e9100 0x564e6c0f4300 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.072+0000 7f75ada62b80 1 --2- 172.21.15.86:0/1450933161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e6c0e9a20 0x564e6c0f1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.072+0000 7f75ada62b80 1 -- 172.21.15.86:0/1450933161 >> 172.21.15.86:0/1450933161 conn(0x564e6c049640 msgr2=0x564e6c0eb0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.072+0000 7f757cc00700 0 client.25693 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:51:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.072+0000 7f75ada62b80 1 -- 172.21.15.86:0/1450933161 shutdown_connections 2024-03-23T17:51:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.072+0000 7f75ada62b80 1 -- 172.21.15.86:0/1450933161 wait complete. 2024-03-23T17:51:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.072+0000 7f75ada62b80 20 client.25693 trim_cache size 0 max 16384 2024-03-23T17:51:21.102 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.100+0000 7f75ada62b80 1 Processor -- start 2024-03-23T17:51:21.102 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.101+0000 7f75ada62b80 1 -- start start 2024-03-23T17:51:21.102 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.101+0000 7f75ada62b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564e6c0e9100 0x564e6c0ec050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:21.102 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.101+0000 7f75ada62b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564e6c0e9a20 0x564e6c10c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:21.102 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.101+0000 7f75ada62b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e6c0f48d0 0x564e6c0df500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:21.102 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.101+0000 7f75ada62b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e6c0ea160 con 0x564e6c0f48d0 2024-03-23T17:51:21.102 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.101+0000 7f75ada62b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e6c11c300 con 0x564e6c0e9a20 2024-03-23T17:51:21.102 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.101+0000 7f75ada62b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e6c0f6f10 con 0x564e6c0e9100 2024-03-23T17:51:21.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.101+0000 7f7593400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564e6c0e9100 0x564e6c0ec050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:21.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.102+0000 7f7593400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564e6c0e9100 0x564e6c0ec050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:44508/0 (socket says 172.21.15.86:44508) 2024-03-23T17:51:21.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.102+0000 7f7592a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564e6c0e9a20 0x564e6c10c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:21.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.102+0000 7f7593400700 1 -- 172.21.15.86:0/2291861315 learned_addr learned my addr 172.21.15.86:0/2291861315 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:21.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.102+0000 7f7593e00700 1 --2- 172.21.15.86:0/2291861315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e6c0f48d0 0x564e6c0df500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:21.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.102+0000 7f7592a00700 1 -- 172.21.15.86:0/2291861315 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564e6c0e9100 msgr2=0x564e6c0ec050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:21.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.102+0000 7f7592a00700 1 --2- 172.21.15.86:0/2291861315 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564e6c0e9100 0x564e6c0ec050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:21.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.102+0000 7f7592a00700 1 -- 172.21.15.86:0/2291861315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e6c0f48d0 msgr2=0x564e6c0df500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:21.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.102+0000 7f7592a00700 1 --2- 172.21.15.86:0/2291861315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e6c0f48d0 0x564e6c0df500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:21.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.102+0000 7f7592a00700 1 -- 172.21.15.86:0/2291861315 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e6c001e40 con 0x564e6c0e9a20 2024-03-23T17:51:21.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.102+0000 7f7593e00700 1 --2- 172.21.15.86:0/2291861315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e6c0f48d0 0x564e6c0df500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:51:21.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.102+0000 7f7592a00700 1 --2- 172.21.15.86:0/2291861315 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564e6c0e9a20 0x564e6c10c5f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f759419a940 tx=0x7f758000b6b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be12eca9a096374d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:21.105 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.103+0000 7f7589200700 1 -- 172.21.15.86:0/2291861315 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75800282e0 con 0x564e6c0e9a20 2024-03-23T17:51:21.105 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.103+0000 7f7589200700 1 -- 172.21.15.86:0/2291861315 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f758001e040 con 0x564e6c0e9a20 2024-03-23T17:51:21.105 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.103+0000 7f7589200700 1 -- 172.21.15.86:0/2291861315 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758000b9b0 con 0x564e6c0e9a20 2024-03-23T17:51:21.105 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.104+0000 7f75ada62b80 1 -- 172.21.15.86:0/2291861315 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564e6c0e9a20 msgr2=0x564e6c10c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:21.105 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.104+0000 7f75ada62b80 1 --2- 172.21.15.86:0/2291861315 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564e6c0e9a20 0x564e6c10c5f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f759419a940 tx=0x7f758000b6b0 comp rx=0 tx=0).stop 2024-03-23T17:51:21.105 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.104+0000 7f75ada62b80 1 -- 172.21.15.86:0/2291861315 shutdown_connections 2024-03-23T17:51:21.106 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.104+0000 7f75ada62b80 1 --2- 172.21.15.86:0/2291861315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e6c0f48d0 0x564e6c0df500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:21.106 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.104+0000 7f75ada62b80 1 --2- 172.21.15.86:0/2291861315 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564e6c0e9100 0x564e6c0ec050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:21.106 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.104+0000 7f75ada62b80 1 --2- 172.21.15.86:0/2291861315 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564e6c0e9a20 0x564e6c10c5f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:21.106 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.104+0000 7f75ada62b80 1 -- 172.21.15.86:0/2291861315 >> 172.21.15.86:0/2291861315 conn(0x564e6c049640 msgr2=0x564e6c0e4d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:21.106 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.104+0000 7f75ada62b80 1 -- 172.21.15.86:0/2291861315 shutdown_connections 2024-03-23T17:51:21.106 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.104+0000 7f75ada62b80 1 -- 172.21.15.86:0/2291861315 wait complete. 2024-03-23T17:51:21.106 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.104+0000 7f75ada62b80 1 -- start start 2024-03-23T17:51:21.106 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.104+0000 7f75ada62b80 1 Processor -- start 2024-03-23T17:51:21.107 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.105+0000 7f75ada62b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564e6c0e9100 0x564e6c10c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:21.107 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.105+0000 7f75ada62b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e6c0e9a20 0x564e6c10c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:21.107 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.105+0000 7f75ada62b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564e6c0f48d0 0x564e6c10a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:21.107 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.105+0000 7f75ada62b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e6c10ae90 con 0x564e6c0e9a20 2024-03-23T17:51:21.107 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.105+0000 7f75ada62b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e6c10b000 con 0x564e6c0e9100 2024-03-23T17:51:21.107 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.105+0000 7f75ada62b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e6c10b1c0 con 0x564e6c0f48d0 2024-03-23T17:51:21.107 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.105+0000 7f75ada62b80 10 client.0 _ll_register_callbacks cb 0x564e6bfe1410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:51:21.107 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.105+0000 7f7593400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564e6c0e9100 0x564e6c10c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:21.107 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.105+0000 7f7593400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564e6c0e9100 0x564e6c10c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:54918/0 (socket says 172.21.15.86:54918) 2024-03-23T17:51:21.108 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.105+0000 7f7593400700 1 -- 172.21.15.86:0/75696644 learned_addr learned my addr 172.21.15.86:0/75696644 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:21.108 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.106+0000 7f7593400700 1 -- 172.21.15.86:0/75696644 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564e6c0f48d0 msgr2=0x564e6c10a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:21.108 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.106+0000 7f7593400700 1 --2- 172.21.15.86:0/75696644 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564e6c0f48d0 0x564e6c10a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:21.108 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.106+0000 7f7593400700 1 -- 172.21.15.86:0/75696644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e6c0e9a20 msgr2=0x564e6c10c5f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-23T17:51:21.108 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.106+0000 7f7593400700 1 --2- 172.21.15.86:0/75696644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e6c0e9a20 0x564e6c10c5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:21.108 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.106+0000 7f7593400700 1 -- 172.21.15.86:0/75696644 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7594110610 con 0x564e6c0e9100 2024-03-23T17:51:21.109 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.106+0000 7f7593400700 1 --2- 172.21.15.86:0/75696644 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564e6c0e9100 0x564e6c10c0b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7584005bf0 tx=0x7f7584009850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7be68b56c9f3937c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:21.109 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.106+0000 7f757cc00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T17:51:21.109 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.106+0000 7f757cc00700 1 -- 172.21.15.86:0/75696644 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75840d0760 con 0x564e6c0e9100 2024-03-23T17:51:21.109 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.106+0000 7f757cc00700 1 -- 172.21.15.86:0/75696644 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f758409a710 con 0x564e6c0e9100 2024-03-23T17:51:21.109 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.106+0000 7f75ada62b80 1 -- 172.21.15.86:0/75696644 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564e6bffc320 con 0x564e6c0e9100 2024-03-23T17:51:21.109 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.106+0000 7f757cc00700 1 -- 172.21.15.86:0/75696644 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f758409a020 con 0x564e6c0e9100 2024-03-23T17:51:21.109 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.106+0000 7f75ada62b80 10 client.25697 fetch_fsmap learned FSMap version 28 2024-03-23T17:51:21.110 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.106+0000 7f75ada62b80 1 -- 172.21.15.86:0/75696644 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x564e6bec1c70 con 0x564e6c0e9100 2024-03-23T17:51:21.110 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.107+0000 7f757cc00700 1 -- 172.21.15.86:0/75696644 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564e6bffc320 con 0x564e6c0e9100 2024-03-23T17:51:21.110 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.107+0000 7f757cc00700 1 -- 172.21.15.86:0/75696644 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 28) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7584020bc0 con 0x564e6c0e9100 2024-03-23T17:51:21.110 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.107+0000 7f75ada62b80 10 client.25697 fetch_fsmap finished waiting for FSMap version 28 2024-03-23T17:51:21.110 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.107+0000 7f75ada62b80 10 client.25697 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:51:21.110 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.107+0000 7f75ada62b80 20 client.25697 populate_metadata read hostname 'smithi086' 2024-03-23T17:51:21.110 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.107+0000 7f75ada62b80 1 --2- 172.21.15.86:0/75696644 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x564e6c167b50 0x564e6c169f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:21.110 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.107+0000 7f75ada62b80 4 client.25697 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T17:51:21.110 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.107+0000 7f75ada62b80 1 -- 172.21.15.86:0/75696644 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564e6bebfc80 con 0x564e6c167b50 2024-03-23T17:51:21.110 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.107+0000 7f7592a00700 1 --2- 172.21.15.86:0/75696644 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x564e6c167b50 0x564e6c169f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:21.110 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.108+0000 7f7592a00700 1 --2- 172.21.15.86:0/75696644 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x564e6c167b50 0x564e6c169f40 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=151078bb048aee50 server_cookie=7be7d44c72a84af4 in_seq=0 out_seq=0 2024-03-23T17:51:21.110 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.108+0000 7f757cc00700 10 client.25697 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:51:21.117 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.116+0000 7f757cc00700 1 -- 172.21.15.86:0/75696644 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564e6bebfc80 con 0x564e6c167b50 2024-03-23T17:51:21.117 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.116+0000 7f757cc00700 10 client.25697 handle_command_reply: tid=0 2024-03-23T17:51:21.117 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.116+0000 7f75ada62b80 1 client.25697 shutdown 2024-03-23T17:51:21.118 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.117+0000 7f75ada62b80 1 -- 172.21.15.86:0/75696644 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564e6c0e9100 msgr2=0x564e6c10c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:21.118 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.117+0000 7f75ada62b80 1 --2- 172.21.15.86:0/75696644 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564e6c0e9100 0x564e6c10c0b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7584005bf0 tx=0x7f7584009850 comp rx=0 tx=0).stop 2024-03-23T17:51:21.118 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.117+0000 7f75ada62b80 1 -- 172.21.15.86:0/75696644 shutdown_connections 2024-03-23T17:51:21.118 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.117+0000 7f75ada62b80 1 --2- 172.21.15.86:0/75696644 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x564e6c167b50 0x564e6c169f40 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:21.119 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.117+0000 7f75ada62b80 1 --2- 172.21.15.86:0/75696644 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564e6c0f48d0 0x564e6c10a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:21.119 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.117+0000 7f75ada62b80 1 --2- 172.21.15.86:0/75696644 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564e6c0e9100 0x564e6c10c0b0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:21.119 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.117+0000 7f75ada62b80 1 --2- 172.21.15.86:0/75696644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e6c0e9a20 0x564e6c10c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:21.119 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.117+0000 7f75ada62b80 1 -- 172.21.15.86:0/75696644 >> 172.21.15.86:0/75696644 conn(0x564e6c049640 msgr2=0x564e6c0e5450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:21.119 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.117+0000 7f757cc00700 0 client.25697 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:51:21.119 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.117+0000 7f75ada62b80 1 -- 172.21.15.86:0/75696644 shutdown_connections 2024-03-23T17:51:21.119 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.117+0000 7f75ada62b80 1 -- 172.21.15.86:0/75696644 wait complete. 2024-03-23T17:51:21.119 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.117+0000 7f75ada62b80 20 client.25697 trim_cache size 0 max 16384 2024-03-23T17:51:21.119 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-03-23T17:51:21.119 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-03-23T17:51:21.119 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-03-23T17:51:21.120 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-03-23T17:51:21.120 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.118+0000 7f759a000700 1 -- 172.21.15.86:0/2215760127 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f75780a1430 msgr2=0x7f75780a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:21.120 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.118+0000 7f759a000700 1 --2- 172.21.15.86:0/2215760127 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f75780a1430 0x7f75780a38f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f758c009d50 tx=0x7f758c00b040 comp rx=0 tx=0).stop 2024-03-23T17:51:21.120 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.118+0000 7f759a000700 1 -- 172.21.15.86:0/2215760127 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7594073160 msgr2=0x7f75941993f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:21.120 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.118+0000 7f759a000700 1 --2- 172.21.15.86:0/2215760127 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7594073160 0x7f75941993f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f758400e9a0 tx=0x7f758400ed60 comp rx=0 tx=0).stop 2024-03-23T17:51:21.120 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.118+0000 7f759a000700 1 -- 172.21.15.86:0/2215760127 shutdown_connections 2024-03-23T17:51:21.121 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.118+0000 7f759a000700 1 --2- 172.21.15.86:0/2215760127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f759410d560 0x7f759419dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:21.121 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.118+0000 7f759a000700 1 --2- 172.21.15.86:0/2215760127 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7594073160 0x7f75941993f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:21.121 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.118+0000 7f759a000700 1 --2- 172.21.15.86:0/2215760127 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f75780a1430 0x7f75780a38f0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:21.121 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.118+0000 7f759a000700 1 --2- 172.21.15.86:0/2215760127 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7594073a80 0x7f7594199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:21.121 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.118+0000 7f759a000700 1 -- 172.21.15.86:0/2215760127 >> 172.21.15.86:0/2215760127 conn(0x7f75940fc770 msgr2=0x7f75940fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:21.121 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.118+0000 7f759a000700 1 -- 172.21.15.86:0/2215760127 shutdown_connections 2024-03-23T17:51:21.121 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:21.119+0000 7f759a000700 1 -- 172.21.15.86:0/2215760127 wait complete. 2024-03-23T17:51:22.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:51:21 smithi086 ceph-mon[38221]: pgmap v24: 97 pgs: 97 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.5 KiB/s wr, 14 op/s 2024-03-23T17:51:22.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:51:21 smithi092 ceph-mon[79089]: pgmap v24: 97 pgs: 97 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.5 KiB/s wr, 14 op/s 2024-03-23T17:51:22.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:51:21 smithi152 ceph-mon[47949]: pgmap v24: 97 pgs: 97 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.5 KiB/s wr, 14 op/s 2024-03-23T17:51:24.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:51:23 smithi086 ceph-mon[38221]: pgmap v25: 97 pgs: 97 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-03-23T17:51:24.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:51:23 smithi092 ceph-mon[79089]: pgmap v25: 97 pgs: 97 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-03-23T17:51:24.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:51:23 smithi152 ceph-mon[47949]: pgmap v25: 97 pgs: 97 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-03-23T17:51:26.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:51:25 smithi086 ceph-mon[38221]: pgmap v26: 97 pgs: 97 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-03-23T17:51:26.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:51:25 smithi092 ceph-mon[79089]: pgmap v26: 97 pgs: 97 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-03-23T17:51:26.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:51:25 smithi152 ceph-mon[47949]: pgmap v26: 97 pgs: 97 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-03-23T17:51:28.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:51:27 smithi086 ceph-mon[38221]: pgmap v27: 97 pgs: 97 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.5 KiB/s wr, 14 op/s 2024-03-23T17:51:28.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:51:27 smithi092 ceph-mon[79089]: pgmap v27: 97 pgs: 97 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.5 KiB/s wr, 14 op/s 2024-03-23T17:51:28.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:51:27 smithi152 ceph-mon[47949]: pgmap v27: 97 pgs: 97 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.5 KiB/s wr, 14 op/s 2024-03-23T17:51:30.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:51:29 smithi086 ceph-mon[38221]: pgmap v28: 97 pgs: 97 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-03-23T17:51:30.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:51:29 smithi092 ceph-mon[79089]: pgmap v28: 97 pgs: 97 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-03-23T17:51:30.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:51:29 smithi152 ceph-mon[47949]: pgmap v28: 97 pgs: 97 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-03-23T17:51:32.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:51:31 smithi086 ceph-mon[38221]: pgmap v29: 97 pgs: 97 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.5 KiB/s wr, 14 op/s 2024-03-23T17:51:32.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:51:31 smithi092 ceph-mon[79089]: pgmap v29: 97 pgs: 97 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.5 KiB/s wr, 14 op/s 2024-03-23T17:51:32.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:51:31 smithi152 ceph-mon[47949]: pgmap v29: 97 pgs: 97 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.5 KiB/s wr, 14 op/s 2024-03-23T17:51:33.361 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-23T17:51:33.361 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-23T17:51:33.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.560+0000 7fc5c4e00700 1 -- 172.21.15.86:0/616508797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5c0100f70 msgr2=0x7fc5c0101350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:33.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.560+0000 7fc5c4e00700 1 --2- 172.21.15.86:0/616508797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5c0100f70 0x7fc5c0101350 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fc5b0009990 tx=0x7fc5b0009ca0 comp rx=0 tx=0).stop 2024-03-23T17:51:33.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.560+0000 7fc5c4e00700 1 -- 172.21.15.86:0/616508797 shutdown_connections 2024-03-23T17:51:33.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.560+0000 7fc5c4e00700 1 --2- 172.21.15.86:0/616508797 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5c0106090 0x7fc5c010dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:33.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.560+0000 7fc5c4e00700 1 --2- 172.21.15.86:0/616508797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5c0100f70 0x7fc5c0101350 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:33.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.560+0000 7fc5c4e00700 1 --2- 172.21.15.86:0/616508797 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc5c0101920 0x7fc5c0105960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:33.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.560+0000 7fc5c4e00700 1 -- 172.21.15.86:0/616508797 >> 172.21.15.86:0/616508797 conn(0x7fc5c00fc7f0 msgr2=0x7fc5c00fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:33.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.561+0000 7fc5c4e00700 1 -- 172.21.15.86:0/616508797 shutdown_connections 2024-03-23T17:51:33.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.561+0000 7fc5c4e00700 1 -- 172.21.15.86:0/616508797 wait complete. 2024-03-23T17:51:33.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.562+0000 7fc5c4e00700 1 Processor -- start 2024-03-23T17:51:33.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.562+0000 7fc5c4e00700 1 -- start start 2024-03-23T17:51:33.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.562+0000 7fc5c4e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5c0100f70 0x7fc5c0199490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:33.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.562+0000 7fc5c4e00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc5c0106090 0x7fc5c01999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:33.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.562+0000 7fc5c4e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5c019dd70 0x7fc5c019e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:33.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.562+0000 7fc5c4e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5c019e7f0 con 0x7fc5c019dd70 2024-03-23T17:51:33.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.562+0000 7fc5c4e00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5c019e960 con 0x7fc5c0100f70 2024-03-23T17:51:33.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.562+0000 7fc5c4e00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5c019eb20 con 0x7fc5c0106090 2024-03-23T17:51:33.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.562+0000 7fc5be800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5c019dd70 0x7fc5c019e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:33.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.562+0000 7fc5be800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5c019dd70 0x7fc5c019e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32818/0 (socket says 172.21.15.86:32818) 2024-03-23T17:51:33.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.562+0000 7fc5be800700 1 -- 172.21.15.86:0/1893218987 learned_addr learned my addr 172.21.15.86:0/1893218987 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:33.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.562+0000 7fc5bd400700 1 --2- 172.21.15.86:0/1893218987 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc5c0106090 0x7fc5c01999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:33.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.562+0000 7fc5bde00700 1 --2- 172.21.15.86:0/1893218987 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5c0100f70 0x7fc5c0199490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:33.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.563+0000 7fc5be800700 1 -- 172.21.15.86:0/1893218987 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc5c0106090 msgr2=0x7fc5c01999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:33.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.563+0000 7fc5be800700 1 --2- 172.21.15.86:0/1893218987 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc5c0106090 0x7fc5c01999d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:33.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.563+0000 7fc5be800700 1 -- 172.21.15.86:0/1893218987 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5c0100f70 msgr2=0x7fc5c0199490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:33.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.563+0000 7fc5be800700 1 --2- 172.21.15.86:0/1893218987 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5c0100f70 0x7fc5c0199490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:33.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.563+0000 7fc5be800700 1 -- 172.21.15.86:0/1893218987 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5b00095f0 con 0x7fc5c019dd70 2024-03-23T17:51:33.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.563+0000 7fc5be800700 1 --2- 172.21.15.86:0/1893218987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5c019dd70 0x7fc5c019e1f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fc5b800ef30 tx=0x7fc5b800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:33.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.563+0000 7fc5b6a00700 1 -- 172.21.15.86:0/1893218987 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b8018070 con 0x7fc5c019dd70 2024-03-23T17:51:33.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.563+0000 7fc5b6a00700 1 -- 172.21.15.86:0/1893218987 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc5b800f040 con 0x7fc5c019dd70 2024-03-23T17:51:33.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.563+0000 7fc5c4e00700 1 -- 172.21.15.86:0/1893218987 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5c01a6020 con 0x7fc5c019dd70 2024-03-23T17:51:33.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.563+0000 7fc5b6a00700 1 -- 172.21.15.86:0/1893218987 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b800cd00 con 0x7fc5c019dd70 2024-03-23T17:51:33.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.564+0000 7fc5c4e00700 1 -- 172.21.15.86:0/1893218987 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5c01a6500 con 0x7fc5c019dd70 2024-03-23T17:51:33.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.566+0000 7fc5b6a00700 1 -- 172.21.15.86:0/1893218987 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) v1 ==== 136098+0+0 (secure 0 0 0) 0x7fc5b8003a00 con 0x7fc5c019dd70 2024-03-23T17:51:33.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.568+0000 7fc5d898db80 1 Processor -- start 2024-03-23T17:51:33.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.568+0000 7fc5d898db80 1 -- start start 2024-03-23T17:51:33.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.568+0000 7fc5d898db80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d0069495d0 0x55d0069499b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:33.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.568+0000 7fc5d898db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d006949ef0 0x55d0069548b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:33.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.568+0000 7fc5d898db80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d006954df0 0x55d0069571e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:33.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.568+0000 7fc5d898db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0068a2990 con 0x55d006949ef0 2024-03-23T17:51:33.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.568+0000 7fc5d898db80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0068a0350 con 0x55d006954df0 2024-03-23T17:51:33.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.568+0000 7fc5d898db80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d00689bc40 con 0x55d0069495d0 2024-03-23T17:51:33.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.569+0000 7fc5bde00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d006949ef0 0x55d0069548b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:33.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.569+0000 7fc5bde00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d006949ef0 0x55d0069548b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32828/0 (socket says 172.21.15.86:32828) 2024-03-23T17:51:33.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.569+0000 7fc5bde00700 1 -- 172.21.15.86:0/2833315416 learned_addr learned my addr 172.21.15.86:0/2833315416 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:33.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.569+0000 7fc5bd400700 1 --2- 172.21.15.86:0/2833315416 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d0069495d0 0x55d0069499b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:33.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.569+0000 7fc5b6a00700 1 --2- 172.21.15.86:0/1893218987 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fc5a40a1430 0x7fc5a40a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:33.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.569+0000 7fc5bd400700 1 --2- 172.21.15.86:0/2833315416 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d006954df0 0x55d0069571e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:33.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.569+0000 7fc5b6a00700 1 -- 172.21.15.86:0/1893218987 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fc5b80cca30 con 0x7fc5c019dd70 2024-03-23T17:51:33.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.569+0000 7fc5bd400700 1 -- 172.21.15.86:0/2833315416 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d006954df0 msgr2=0x55d0069571e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:33.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.569+0000 7fc5bd400700 1 --2- 172.21.15.86:0/2833315416 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d006954df0 0x55d0069571e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:33.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.569+0000 7fc5bd400700 1 -- 172.21.15.86:0/2833315416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d006949ef0 msgr2=0x55d0069548b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:33.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.569+0000 7fc5bd400700 1 --2- 172.21.15.86:0/2833315416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d006949ef0 0x55d0069548b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:33.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.569+0000 7fc5be800700 1 --2- 172.21.15.86:0/1893218987 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fc5a40a1430 0x7fc5a40a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:33.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.569+0000 7fc5bd400700 1 -- 172.21.15.86:0/2833315416 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d00671d3b0 con 0x55d0069495d0 2024-03-23T17:51:33.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.569+0000 7fc5bd400700 1 --2- 172.21.15.86:0/2833315416 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d0069495d0 0x55d0069499b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc5c019a9e0 tx=0x7fc5ac005d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3c9e2f7e7839b5c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:33.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.569+0000 7fc5bde00700 1 --2- 172.21.15.86:0/2833315416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d006949ef0 0x55d0069548b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:51:33.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.570+0000 7fc5ab400700 1 -- 172.21.15.86:0/2833315416 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5ac012e40 con 0x55d0069495d0 2024-03-23T17:51:33.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.570+0000 7fc5ab400700 1 -- 172.21.15.86:0/2833315416 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc5ac0044f0 con 0x55d0069495d0 2024-03-23T17:51:33.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.570+0000 7fc5ab400700 1 -- 172.21.15.86:0/2833315416 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5ac00f3f0 con 0x55d0069495d0 2024-03-23T17:51:33.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.570+0000 7fc5be800700 1 --2- 172.21.15.86:0/1893218987 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fc5a40a1430 0x7fc5a40a38f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc5b80ca050 tx=0x7fc5b80092e0 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:33.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.570+0000 7fc5d898db80 1 -- 172.21.15.86:0/2833315416 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d0069495d0 msgr2=0x55d0069499b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:33.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.570+0000 7fc5d898db80 1 --2- 172.21.15.86:0/2833315416 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d0069495d0 0x55d0069499b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc5c019a9e0 tx=0x7fc5ac005d20 comp rx=0 tx=0).stop 2024-03-23T17:51:33.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.571+0000 7fc5d898db80 1 -- 172.21.15.86:0/2833315416 shutdown_connections 2024-03-23T17:51:33.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.571+0000 7fc5d898db80 1 --2- 172.21.15.86:0/2833315416 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d006954df0 0x55d0069571e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:33.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.571+0000 7fc5d898db80 1 --2- 172.21.15.86:0/2833315416 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d0069495d0 0x55d0069499b0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:33.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.571+0000 7fc5d898db80 1 --2- 172.21.15.86:0/2833315416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d006949ef0 0x55d0069548b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:33.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.571+0000 7fc5d898db80 1 -- 172.21.15.86:0/2833315416 >> 172.21.15.86:0/2833315416 conn(0x55d0068a96a0 msgr2=0x55d006945a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:33.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.571+0000 7fc5d898db80 1 -- 172.21.15.86:0/2833315416 shutdown_connections 2024-03-23T17:51:33.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.571+0000 7fc5d898db80 1 -- 172.21.15.86:0/2833315416 wait complete. 2024-03-23T17:51:33.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.571+0000 7fc5d898db80 1 -- start start 2024-03-23T17:51:33.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.571+0000 7fc5d898db80 1 Processor -- start 2024-03-23T17:51:33.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.572+0000 7fc5d898db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0069495d0 0x55d006954820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:33.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.572+0000 7fc5d898db80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d006949ef0 0x55d006951b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:33.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.572+0000 7fc5d898db80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d006954df0 0x55d006952250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:33.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.572+0000 7fc5d898db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d006837d80 con 0x55d0069495d0 2024-03-23T17:51:33.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.572+0000 7fc5d898db80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d006837a20 con 0x55d006949ef0 2024-03-23T17:51:33.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.572+0000 7fc5d898db80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d00671f6a0 con 0x55d006954df0 2024-03-23T17:51:33.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.572+0000 7fc5d898db80 10 client.0 _ll_register_callbacks cb 0x55d006841470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:51:33.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.572+0000 7fc5bd400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0069495d0 0x55d006954820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:33.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.572+0000 7fc5bde00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d006949ef0 0x55d006951b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:33.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.572+0000 7fc5bd400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0069495d0 0x55d006954820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32834/0 (socket says 172.21.15.86:32834) 2024-03-23T17:51:33.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.572+0000 7fc5bd400700 1 -- 172.21.15.86:0/1068582164 learned_addr learned my addr 172.21.15.86:0/1068582164 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:33.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.572+0000 7fc5bd400700 1 --2- 172.21.15.86:0/1068582164 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d006954df0 0x55d006952250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:33.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.573+0000 7fc5bde00700 1 -- 172.21.15.86:0/1068582164 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d006954df0 msgr2=0x55d006952250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:33.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.573+0000 7fc5bde00700 1 --2- 172.21.15.86:0/1068582164 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d006954df0 0x55d006952250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:33.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.573+0000 7fc5bde00700 1 -- 172.21.15.86:0/1068582164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0069495d0 msgr2=0x55d006954820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:33.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.573+0000 7fc5bde00700 1 --2- 172.21.15.86:0/1068582164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0069495d0 0x55d006954820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:33.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.573+0000 7fc5bde00700 1 -- 172.21.15.86:0/1068582164 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5c010e480 con 0x55d006949ef0 2024-03-23T17:51:33.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.573+0000 7fc5bd400700 1 --2- 172.21.15.86:0/1068582164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0069495d0 0x55d006954820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:51:33.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.573+0000 7fc5bde00700 1 --2- 172.21.15.86:0/1068582164 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d006949ef0 0x55d006951b10 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc5b0015000 tx=0x7fc5b0005a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0be876ed66459b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:33.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.573+0000 7fc59fe00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T17:51:33.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.573+0000 7fc59fe00700 1 -- 172.21.15.86:0/1068582164 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b001d070 con 0x55d006949ef0 2024-03-23T17:51:33.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.573+0000 7fc5d898db80 1 -- 172.21.15.86:0/1068582164 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d00685c380 con 0x55d006949ef0 2024-03-23T17:51:33.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.573+0000 7fc59fe00700 1 -- 172.21.15.86:0/1068582164 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fc5b0004500 con 0x55d006949ef0 2024-03-23T17:51:33.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.573+0000 7fc59fe00700 1 -- 172.21.15.86:0/1068582164 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc5b00046c0 con 0x55d006949ef0 2024-03-23T17:51:33.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.573+0000 7fc5d898db80 10 client.25709 fetch_fsmap learned FSMap version 28 2024-03-23T17:51:33.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.573+0000 7fc5d898db80 1 -- 172.21.15.86:0/1068582164 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7fc5ac01a040 con 0x55d006949ef0 2024-03-23T17:51:33.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.574+0000 7fc59fe00700 1 -- 172.21.15.86:0/1068582164 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d00685c380 con 0x55d006949ef0 2024-03-23T17:51:33.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.574+0000 7fc59fe00700 1 -- 172.21.15.86:0/1068582164 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 28) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc5b000f870 con 0x55d006949ef0 2024-03-23T17:51:33.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.574+0000 7fc5d898db80 10 client.25709 fetch_fsmap finished waiting for FSMap version 28 2024-03-23T17:51:33.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.574+0000 7fc5d898db80 10 client.25709 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T17:51:33.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.574+0000 7fc5d898db80 20 client.25709 populate_metadata read hostname 'smithi086' 2024-03-23T17:51:33.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.574+0000 7fc5d898db80 1 --2- 172.21.15.86:0/1068582164 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55d006964960 0x55d006966d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:33.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.574+0000 7fc5d898db80 4 client.25709 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:51:33.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.574+0000 7fc5d898db80 1 -- 172.21.15.86:0/1068582164 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d006894700 con 0x55d006964960 2024-03-23T17:51:33.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.574+0000 7fc5bd400700 1 --2- 172.21.15.86:0/1068582164 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55d006964960 0x55d006966d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:33.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.575+0000 7fc59fe00700 10 client.25709 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:51:33.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.575+0000 7fc5bd400700 1 --2- 172.21.15.86:0/1068582164 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55d006964960 0x55d006966d80 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbce000033901612 server_cookie=8c40b6f1a854d1b7 in_seq=0 out_seq=0 2024-03-23T17:51:33.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.575+0000 7fc59fe00700 1 -- 172.21.15.86:0/1068582164 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55d006894700 con 0x55d006964960 2024-03-23T17:51:33.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.575+0000 7fc59fe00700 10 client.25709 handle_command_reply: tid=0 2024-03-23T17:51:33.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.575+0000 7fc5d898db80 1 client.25709 shutdown 2024-03-23T17:51:33.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.576+0000 7fc5d898db80 1 -- 172.21.15.86:0/1068582164 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d006949ef0 msgr2=0x55d006951b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:33.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.576+0000 7fc5d898db80 1 --2- 172.21.15.86:0/1068582164 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d006949ef0 0x55d006951b10 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc5b0015000 tx=0x7fc5b0005a90 comp rx=0 tx=0).stop 2024-03-23T17:51:33.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.576+0000 7fc5d898db80 1 -- 172.21.15.86:0/1068582164 shutdown_connections 2024-03-23T17:51:33.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.576+0000 7fc5d898db80 1 --2- 172.21.15.86:0/1068582164 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55d006964960 0x55d006966d80 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:33.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.576+0000 7fc5d898db80 1 --2- 172.21.15.86:0/1068582164 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d006954df0 0x55d006952250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:33.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.576+0000 7fc5d898db80 1 --2- 172.21.15.86:0/1068582164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0069495d0 0x55d006954820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:33.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.576+0000 7fc59fe00700 0 client.25709 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:51:33.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.576+0000 7fc5d898db80 1 --2- 172.21.15.86:0/1068582164 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d006949ef0 0x55d006951b10 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:33.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.576+0000 7fc5d898db80 1 -- 172.21.15.86:0/1068582164 >> 172.21.15.86:0/1068582164 conn(0x55d0068a96a0 msgr2=0x55d00694b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:33.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.576+0000 7fc5d898db80 1 -- 172.21.15.86:0/1068582164 shutdown_connections 2024-03-23T17:51:33.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.576+0000 7fc5d898db80 1 -- 172.21.15.86:0/1068582164 wait complete. 2024-03-23T17:51:33.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.576+0000 7fc5d898db80 20 client.25709 trim_cache size 0 max 16384 2024-03-23T17:51:33.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.593+0000 7fc5d898db80 1 Processor -- start 2024-03-23T17:51:33.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.593+0000 7fc5d898db80 1 -- start start 2024-03-23T17:51:33.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.593+0000 7fc5d898db80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d0069495d0 0x55d006951ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:33.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.593+0000 7fc5d898db80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d006949ef0 0x55d00696b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:33.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.593+0000 7fc5d898db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d006954df0 0x55d00695ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:33.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.593+0000 7fc5d898db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d00694af60 con 0x55d006954df0 2024-03-23T17:51:33.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.593+0000 7fc5d898db80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d006952610 con 0x55d006949ef0 2024-03-23T17:51:33.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.593+0000 7fc5d898db80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d006953110 con 0x55d0069495d0 2024-03-23T17:51:33.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.594+0000 7fc5bd400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d0069495d0 0x55d006951ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:33.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.594+0000 7fc5bde00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d006949ef0 0x55d00696b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:33.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.594+0000 7fc5bd400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d0069495d0 0x55d006951ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:47646/0 (socket says 172.21.15.86:47646) 2024-03-23T17:51:33.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.594+0000 7fc5bde00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d006949ef0 0x55d00696b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:51328/0 (socket says 172.21.15.86:51328) 2024-03-23T17:51:33.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.594+0000 7fc5bd400700 1 -- 172.21.15.86:0/1995936403 learned_addr learned my addr 172.21.15.86:0/1995936403 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:33.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.594+0000 7fc5bd400700 1 --2- 172.21.15.86:0/1995936403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d006954df0 0x55d00695ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:33.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.594+0000 7fc5bde00700 1 -- 172.21.15.86:0/1995936403 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d0069495d0 msgr2=0x55d006951ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:33.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.594+0000 7fc5bde00700 1 --2- 172.21.15.86:0/1995936403 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d0069495d0 0x55d006951ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:33.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.594+0000 7fc5bde00700 1 -- 172.21.15.86:0/1995936403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d006954df0 msgr2=0x55d00695ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:33.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.594+0000 7fc5bde00700 1 --2- 172.21.15.86:0/1995936403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d006954df0 0x55d00695ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:33.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.594+0000 7fc5bde00700 1 -- 172.21.15.86:0/1995936403 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d0066c9b60 con 0x55d006949ef0 2024-03-23T17:51:33.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.594+0000 7fc5bde00700 1 --2- 172.21.15.86:0/1995936403 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d006949ef0 0x55d00696b6e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc5b000fc00 tx=0x7fc5b0009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e9f75f5f377695c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:33.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.594+0000 7fc5abe00700 1 -- 172.21.15.86:0/1995936403 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b001d070 con 0x55d006949ef0 2024-03-23T17:51:33.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.595+0000 7fc5abe00700 1 -- 172.21.15.86:0/1995936403 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc5b001e040 con 0x55d006949ef0 2024-03-23T17:51:33.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.595+0000 7fc5abe00700 1 -- 172.21.15.86:0/1995936403 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b00249b0 con 0x55d006949ef0 2024-03-23T17:51:33.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.595+0000 7fc5d898db80 1 -- 172.21.15.86:0/1995936403 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d006949ef0 msgr2=0x55d00696b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:33.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.595+0000 7fc5d898db80 1 --2- 172.21.15.86:0/1995936403 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d006949ef0 0x55d00696b6e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc5b000fc00 tx=0x7fc5b0009310 comp rx=0 tx=0).stop 2024-03-23T17:51:33.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.595+0000 7fc5d898db80 1 -- 172.21.15.86:0/1995936403 shutdown_connections 2024-03-23T17:51:33.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.595+0000 7fc5d898db80 1 --2- 172.21.15.86:0/1995936403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d006954df0 0x55d00695ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:33.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.595+0000 7fc5d898db80 1 --2- 172.21.15.86:0/1995936403 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d0069495d0 0x55d006951ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:33.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.595+0000 7fc5d898db80 1 --2- 172.21.15.86:0/1995936403 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d006949ef0 0x55d00696b6e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:33.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.595+0000 7fc5d898db80 1 -- 172.21.15.86:0/1995936403 >> 172.21.15.86:0/1995936403 conn(0x55d0068a96a0 msgr2=0x55d00693e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:33.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.595+0000 7fc5d898db80 1 -- 172.21.15.86:0/1995936403 shutdown_connections 2024-03-23T17:51:33.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.595+0000 7fc5d898db80 1 -- 172.21.15.86:0/1995936403 wait complete. 2024-03-23T17:51:33.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.596+0000 7fc5d898db80 1 -- start start 2024-03-23T17:51:33.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.596+0000 7fc5d898db80 1 Processor -- start 2024-03-23T17:51:33.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.596+0000 7fc5d898db80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d0069495d0 0x55d006959860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:33.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.596+0000 7fc5d898db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d006949ef0 0x55d006959da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:33.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.596+0000 7fc5d898db80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d006954df0 0x55d0069b5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:33.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.596+0000 7fc5d898db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0069b59f0 con 0x55d006949ef0 2024-03-23T17:51:33.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.596+0000 7fc5d898db80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0069b5b60 con 0x55d006954df0 2024-03-23T17:51:33.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.596+0000 7fc5d898db80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0069b5d20 con 0x55d0069495d0 2024-03-23T17:51:33.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.597+0000 7fc5d898db80 10 client.0 _ll_register_callbacks cb 0x55d006841470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:51:33.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.597+0000 7fc5bde00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d006949ef0 0x55d006959da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:33.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.597+0000 7fc5bd400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d0069495d0 0x55d006959860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:33.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.597+0000 7fc5bd400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d0069495d0 0x55d006959860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:47654/0 (socket says 172.21.15.86:47654) 2024-03-23T17:51:33.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.597+0000 7fc5bd400700 1 -- 172.21.15.86:0/3297527394 learned_addr learned my addr 172.21.15.86:0/3297527394 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:33.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.597+0000 7fc5bde00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d006949ef0 0x55d006959da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32852/0 (socket says 172.21.15.86:32852) 2024-03-23T17:51:33.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.597+0000 7fc5bd400700 1 --2- 172.21.15.86:0/3297527394 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d006954df0 0x55d0069b5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:33.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.597+0000 7fc5bd400700 1 -- 172.21.15.86:0/3297527394 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d006954df0 msgr2=0x55d0069b5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:33.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.597+0000 7fc5bd400700 1 --2- 172.21.15.86:0/3297527394 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d006954df0 0x55d0069b5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:33.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.597+0000 7fc5bd400700 1 -- 172.21.15.86:0/3297527394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d006949ef0 msgr2=0x55d006959da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:33.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.597+0000 7fc5bd400700 1 --2- 172.21.15.86:0/3297527394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d006949ef0 0x55d006959da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:33.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.597+0000 7fc5bd400700 1 -- 172.21.15.86:0/3297527394 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5ac01a040 con 0x55d0069495d0 2024-03-23T17:51:33.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.597+0000 7fc5bde00700 1 --2- 172.21.15.86:0/3297527394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d006949ef0 0x55d006959da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:51:33.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.597+0000 7fc5bd400700 1 --2- 172.21.15.86:0/3297527394 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d0069495d0 0x55d006959860 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac005850 tx=0x7fc5ac005d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=735252ea763bb5d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:33.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.597+0000 7fc59fe00700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-03-23T17:51:33.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.598+0000 7fc59fe00700 1 -- 172.21.15.86:0/3297527394 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5ac02a8c0 con 0x55d0069495d0 2024-03-23T17:51:33.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.598+0000 7fc59fe00700 1 -- 172.21.15.86:0/3297527394 <== mon.1 v2:172.21.15.152:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fc5ac012e40 con 0x55d0069495d0 2024-03-23T17:51:33.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.598+0000 7fc5d898db80 1 -- 172.21.15.86:0/3297527394 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc5ac0044f0 con 0x55d0069495d0 2024-03-23T17:51:33.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.598+0000 7fc59fe00700 1 -- 172.21.15.86:0/3297527394 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc5ac02aa20 con 0x55d0069495d0 2024-03-23T17:51:33.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.598+0000 7fc5d898db80 10 client.25714 fetch_fsmap learned FSMap version 28 2024-03-23T17:51:33.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.598+0000 7fc5d898db80 1 -- 172.21.15.86:0/3297527394 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55d006861ea0 con 0x55d0069495d0 2024-03-23T17:51:33.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.598+0000 7fc59fe00700 1 -- 172.21.15.86:0/3297527394 <== mon.1 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc5ac03b020 con 0x55d0069495d0 2024-03-23T17:51:33.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.598+0000 7fc59fe00700 1 -- 172.21.15.86:0/3297527394 <== mon.1 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc5ac035070 con 0x55d0069495d0 2024-03-23T17:51:33.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.598+0000 7fc5d898db80 10 client.25714 fetch_fsmap finished waiting for FSMap version 28 2024-03-23T17:51:33.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.598+0000 7fc5d898db80 10 client.25714 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T17:51:33.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.598+0000 7fc5d898db80 20 client.25714 populate_metadata read hostname 'smithi086' 2024-03-23T17:51:33.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.598+0000 7fc5d898db80 1 --2- 172.21.15.86:0/3297527394 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55d0069b7450 0x55d0069b9870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:33.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.598+0000 7fc5d898db80 4 client.25714 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-23T17:51:33.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.598+0000 7fc5d898db80 1 -- 172.21.15.86:0/3297527394 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d0066eacf0 con 0x55d0069b7450 2024-03-23T17:51:33.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.599+0000 7fc5bde00700 1 --2- 172.21.15.86:0/3297527394 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55d0069b7450 0x55d0069b9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:33.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.599+0000 7fc5bde00700 1 --2- 172.21.15.86:0/3297527394 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55d0069b7450 0x55d0069b9870 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1667e35da360ea29 server_cookie=30e47cce8dea74a4 in_seq=0 out_seq=0 2024-03-23T17:51:33.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.599+0000 7fc59fe00700 10 client.25714 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:51:33.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.600+0000 7fc59fe00700 1 -- 172.21.15.86:0/3297527394 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55d0066eacf0 con 0x55d0069b7450 2024-03-23T17:51:33.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.600+0000 7fc59fe00700 10 client.25714 handle_command_reply: tid=0 2024-03-23T17:51:33.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.600+0000 7fc5d898db80 1 client.25714 shutdown 2024-03-23T17:51:33.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.600+0000 7fc5d898db80 1 -- 172.21.15.86:0/3297527394 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d0069495d0 msgr2=0x55d006959860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:33.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.600+0000 7fc5d898db80 1 --2- 172.21.15.86:0/3297527394 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d0069495d0 0x55d006959860 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac005850 tx=0x7fc5ac005d20 comp rx=0 tx=0).stop 2024-03-23T17:51:33.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.600+0000 7fc5d898db80 1 -- 172.21.15.86:0/3297527394 shutdown_connections 2024-03-23T17:51:33.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.600+0000 7fc5d898db80 1 --2- 172.21.15.86:0/3297527394 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55d0069b7450 0x55d0069b9870 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:33.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.600+0000 7fc5d898db80 1 --2- 172.21.15.86:0/3297527394 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d006954df0 0x55d0069b5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:33.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.600+0000 7fc5d898db80 1 --2- 172.21.15.86:0/3297527394 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d0069495d0 0x55d006959860 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:33.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.600+0000 7fc59fe00700 0 client.25714 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:51:33.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.600+0000 7fc5d898db80 1 --2- 172.21.15.86:0/3297527394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d006949ef0 0x55d006959da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:33.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.600+0000 7fc5d898db80 1 -- 172.21.15.86:0/3297527394 >> 172.21.15.86:0/3297527394 conn(0x55d0068a96a0 msgr2=0x55d00693ef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:33.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.600+0000 7fc5d898db80 1 -- 172.21.15.86:0/3297527394 shutdown_connections 2024-03-23T17:51:33.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.600+0000 7fc5d898db80 1 -- 172.21.15.86:0/3297527394 wait complete. 2024-03-23T17:51:33.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.600+0000 7fc5d898db80 20 client.25714 trim_cache size 0 max 16384 2024-03-23T17:51:33.605 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-23T17:51:33.605 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-03-23T17:51:33.605 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-03-23T17:51:33.605 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-23T17:51:33.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.601+0000 7fc5c4e00700 1 -- 172.21.15.86:0/1893218987 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fc5a40a1430 msgr2=0x7fc5a40a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:33.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.601+0000 7fc5c4e00700 1 --2- 172.21.15.86:0/1893218987 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fc5a40a1430 0x7fc5a40a38f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc5b80ca050 tx=0x7fc5b80092e0 comp rx=0 tx=0).stop 2024-03-23T17:51:33.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.601+0000 7fc5c4e00700 1 -- 172.21.15.86:0/1893218987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5c019dd70 msgr2=0x7fc5c019e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:33.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.601+0000 7fc5c4e00700 1 --2- 172.21.15.86:0/1893218987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5c019dd70 0x7fc5c019e1f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fc5b800ef30 tx=0x7fc5b800c3a0 comp rx=0 tx=0).stop 2024-03-23T17:51:33.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.601+0000 7fc5c4e00700 1 -- 172.21.15.86:0/1893218987 shutdown_connections 2024-03-23T17:51:33.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.601+0000 7fc5c4e00700 1 --2- 172.21.15.86:0/1893218987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5c019dd70 0x7fc5c019e1f0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:33.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.601+0000 7fc5c4e00700 1 --2- 172.21.15.86:0/1893218987 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5c0100f70 0x7fc5c0199490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:33.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.601+0000 7fc5c4e00700 1 --2- 172.21.15.86:0/1893218987 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fc5a40a1430 0x7fc5a40a38f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:33.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.601+0000 7fc5c4e00700 1 --2- 172.21.15.86:0/1893218987 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc5c0106090 0x7fc5c01999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:33.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.601+0000 7fc5c4e00700 1 -- 172.21.15.86:0/1893218987 >> 172.21.15.86:0/1893218987 conn(0x7fc5c00fc7f0 msgr2=0x7fc5c00fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:33.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.601+0000 7fc5c4e00700 1 -- 172.21.15.86:0/1893218987 shutdown_connections 2024-03-23T17:51:33.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.601+0000 7fc5c4e00700 1 -- 172.21.15.86:0/1893218987 wait complete. 2024-03-23T17:51:33.621 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-23T17:51:33.621 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T17:51:33.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.857+0000 7fbffda00700 1 -- 172.21.15.86:0/2596903128 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbff8101810 msgr2=0x7fbff8105850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:33.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.857+0000 7fbffda00700 1 --2- 172.21.15.86:0/2596903128 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbff8101810 0x7fbff8105850 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fbfe40098f0 tx=0x7fbfe4009c00 comp rx=0 tx=0).stop 2024-03-23T17:51:33.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.857+0000 7fbffda00700 1 -- 172.21.15.86:0/2596903128 shutdown_connections 2024-03-23T17:51:33.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.857+0000 7fbffda00700 1 --2- 172.21.15.86:0/2596903128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbff8105f80 0x7fbff810db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:33.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.857+0000 7fbffda00700 1 --2- 172.21.15.86:0/2596903128 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbff8100e60 0x7fbff8101240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:33.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.857+0000 7fbffda00700 1 --2- 172.21.15.86:0/2596903128 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbff8101810 0x7fbff8105850 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:33.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.857+0000 7fbffda00700 1 -- 172.21.15.86:0/2596903128 >> 172.21.15.86:0/2596903128 conn(0x7fbff80fc760 msgr2=0x7fbff80feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:33.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.857+0000 7fbffda00700 1 -- 172.21.15.86:0/2596903128 shutdown_connections 2024-03-23T17:51:33.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.857+0000 7fbffda00700 1 -- 172.21.15.86:0/2596903128 wait complete. 2024-03-23T17:51:33.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.859+0000 7fbffda00700 1 Processor -- start 2024-03-23T17:51:33.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.859+0000 7fbffda00700 1 -- start start 2024-03-23T17:51:33.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.859+0000 7fbffda00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbff8100e60 0x7fbff8199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:33.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.859+0000 7fbffda00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbff8105f80 0x7fbff8199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:33.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.859+0000 7fbffda00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbff819dc00 0x7fbff819e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:33.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.859+0000 7fbffda00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff819e680 con 0x7fbff8100e60 2024-03-23T17:51:33.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.859+0000 7fbffda00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff819e7f0 con 0x7fbff8105f80 2024-03-23T17:51:33.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.859+0000 7fbffda00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff819e9b0 con 0x7fbff819dc00 2024-03-23T17:51:33.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.860+0000 7fbff6a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbff8100e60 0x7fbff8199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:33.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.860+0000 7fbff6a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbff8100e60 0x7fbff8199320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32872/0 (socket says 172.21.15.86:32872) 2024-03-23T17:51:33.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.860+0000 7fbff6a00700 1 -- 172.21.15.86:0/3692186522 learned_addr learned my addr 172.21.15.86:0/3692186522 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:33.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.860+0000 7fbff6000700 1 --2- 172.21.15.86:0/3692186522 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbff8105f80 0x7fbff8199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:33.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.860+0000 7fbff7400700 1 --2- 172.21.15.86:0/3692186522 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbff819dc00 0x7fbff819e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:33.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.860+0000 7fbff6a00700 1 -- 172.21.15.86:0/3692186522 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbff819dc00 msgr2=0x7fbff819e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:33.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.860+0000 7fbff6a00700 1 --2- 172.21.15.86:0/3692186522 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbff819dc00 0x7fbff819e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:33.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.860+0000 7fbff6a00700 1 -- 172.21.15.86:0/3692186522 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbff8105f80 msgr2=0x7fbff8199860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:33.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.860+0000 7fbff6a00700 1 --2- 172.21.15.86:0/3692186522 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbff8105f80 0x7fbff8199860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:33.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.860+0000 7fbff6a00700 1 -- 172.21.15.86:0/3692186522 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe4009520 con 0x7fbff8100e60 2024-03-23T17:51:33.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.860+0000 7fbff6000700 1 --2- 172.21.15.86:0/3692186522 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbff8105f80 0x7fbff8199860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:51:33.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.860+0000 7fbff6a00700 1 --2- 172.21.15.86:0/3692186522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbff8100e60 0x7fbff8199320 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fbfe800e9c0 tx=0x7fbfe800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:33.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.861+0000 7fbfef400700 1 -- 172.21.15.86:0/3692186522 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe8004a30 con 0x7fbff8100e60 2024-03-23T17:51:33.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.861+0000 7fbfef400700 1 -- 172.21.15.86:0/3692186522 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbfe800cc80 con 0x7fbff8100e60 2024-03-23T17:51:33.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.861+0000 7fbffda00700 1 -- 172.21.15.86:0/3692186522 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbff81a5f10 con 0x7fbff8100e60 2024-03-23T17:51:33.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.861+0000 7fbfef400700 1 -- 172.21.15.86:0/3692186522 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe80054e0 con 0x7fbff8100e60 2024-03-23T17:51:33.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.861+0000 7fbffda00700 1 -- 172.21.15.86:0/3692186522 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbff81a6340 con 0x7fbff8100e60 2024-03-23T17:51:33.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.862+0000 7fbffda00700 1 -- 172.21.15.86:0/3692186522 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbff8102330 con 0x7fbff8100e60 2024-03-23T17:51:33.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.864+0000 7fbfef400700 1 -- 172.21.15.86:0/3692186522 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) v1 ==== 136098+0+0 (secure 0 0 0) 0x7fbfe8004b90 con 0x7fbff8100e60 2024-03-23T17:51:33.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.865+0000 7fbfef400700 1 --2- 172.21.15.86:0/3692186522 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fbfdc0a1430 0x7fbfdc0a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:33.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.865+0000 7fbfef400700 1 -- 172.21.15.86:0/3692186522 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fbfe8014070 con 0x7fbff8100e60 2024-03-23T17:51:33.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.865+0000 7fbff6000700 1 --2- 172.21.15.86:0/3692186522 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fbfdc0a1430 0x7fbfdc0a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:33.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.866+0000 7fbff6000700 1 --2- 172.21.15.86:0/3692186522 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fbfdc0a1430 0x7fbfdc0a38f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbfe4005e90 tx=0x7fbfe4019040 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:33.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:33.869+0000 7fbfef400700 1 -- 172.21.15.86:0/3692186522 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fbfe80961a0 con 0x7fbff8100e60 2024-03-23T17:51:34.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:51:33 smithi086 ceph-mon[38221]: pgmap v30: 97 pgs: 97 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-03-23T17:51:34.030 INFO:tasks.workunit.client.0.smithi086.stderr:Updating files: 67% (8106/12088) Updating files: 68% (8220/12088) Updating files: 69% (8341/12088) Updating files: 70% (8462/12088) Updating files: 71% (8583/12088) Updating files: 72% (8704/12088) Updating files: 73% (8825/12088) Updating files: 74% (8946/12088) Updating files: 75% (9066/12088) Updating files: 76% (9187/12088) Updating files: 77% (9308/12088) Updating files: 78% (9429/12088) Updating files: 79% (9550/12088) Updating files: 80% (9671/12088) Updating files: 81% (9792/12088) Updating files: 82% (9913/12088) Updating files: 83% (10034/12088) Updating files: 84% (10154/12088) Updating files: 85% (10275/12088) Updating files: 86% (10396/12088) Updating files: 87% (10517/12088) Updating files: 88% (10638/12088) Updating files: 89% (10759/12088) Updating files: 90% (10880/12088) Updating files: 91% (11001/12088) Updating files: 92% (11121/12088) Updating files: 93% (11242/12088) Updating files: 94% (11363/12088) Updating files: 95% (11484/12088) Updating files: 96% (11605/12088) Updating files: 97% (11726/12088) Updating files: 98% (11847/12088) Updating files: 99% (11968/12088) Updating files: 100% (12088/12088) Updating files: 100% (12088/12088), done. 2024-03-23T17:51:34.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:51:33 smithi092 ceph-mon[79089]: pgmap v30: 97 pgs: 97 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-03-23T17:51:34.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:51:33 smithi152 ceph-mon[47949]: pgmap v30: 97 pgs: 97 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-03-23T17:51:34.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.134+0000 7fbffda00700 1 -- 172.21.15.86:0/3692186522 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbff8067a10 con 0x7fbff8100e60 2024-03-23T17:51:34.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.134+0000 7fbfef400700 1 -- 172.21.15.86:0/3692186522 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+10913 (secure 0 0 0) 0x7fbfe80997c0 con 0x7fbff8100e60 2024-03-23T17:51:34.135 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:51:34.135 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14445,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1433140219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1433140219},{"type":"v1","addr":"172.21.15.86:6835","nonce":1433140219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6839/1879457730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1879457730},{"type":"v1","addr":"172.21.15.86:6839","nonce":1879457730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-23T17:48:56.202323+0000","modified":"2024-03-23T17:50:20.718165+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24553,"mds_2":24518,"mds_3":24505,"mds_4":24500},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14514":{"gid":14514,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/3026943866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3026943866},{"type":"v1","addr":"172.21.15.86:6841","nonce":3026943866}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24464":{"gid":24464,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6833/1467091186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1467091186},{"type":"v1","addr":"172.21.15.92:6833","nonce":1467091186}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6833/3075554623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3075554623},{"type":"v1","addr":"172.21.15.152:6833","nonce":3075554623}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.92:6835/908900027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":908900027},{"type":"v1","addr":"172.21.15.92:6835","nonce":908900027}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"d","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.86:6837/2933881328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2933881328},{"type":"v1","addr":"172.21.15.86:6837","nonce":2933881328}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6837/3967837948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":3967837948},{"type":"v1","addr":"172.21.15.92:6837","nonce":3967837948}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6835/3137085021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3137085021},{"type":"v1","addr":"172.21.15.152:6835","nonce":3137085021}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24538":{"gid":24538,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6837/1151208965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":1151208965},{"type":"v1","addr":"172.21.15.152:6837","nonce":1151208965}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.152:6839/1100723992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":1100723992},{"type":"v1","addr":"172.21.15.152:6839","nonce":1100723992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6839/1394139887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1394139887},{"type":"v1","addr":"172.21.15.92:6839","nonce":1394139887}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-23T17:51:34.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.137+0000 7fbffda00700 1 -- 172.21.15.86:0/3692186522 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fbfdc0a1430 msgr2=0x7fbfdc0a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:34.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.137+0000 7fbffda00700 1 --2- 172.21.15.86:0/3692186522 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fbfdc0a1430 0x7fbfdc0a38f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbfe4005e90 tx=0x7fbfe4019040 comp rx=0 tx=0).stop 2024-03-23T17:51:34.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.137+0000 7fbffda00700 1 -- 172.21.15.86:0/3692186522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbff8100e60 msgr2=0x7fbff8199320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:34.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.137+0000 7fbffda00700 1 --2- 172.21.15.86:0/3692186522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbff8100e60 0x7fbff8199320 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fbfe800e9c0 tx=0x7fbfe800ecd0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.137+0000 7fbffda00700 1 -- 172.21.15.86:0/3692186522 shutdown_connections 2024-03-23T17:51:34.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.137+0000 7fbffda00700 1 --2- 172.21.15.86:0/3692186522 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fbfdc0a1430 0x7fbfdc0a38f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.137+0000 7fbffda00700 1 --2- 172.21.15.86:0/3692186522 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbff819dc00 0x7fbff819e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.137+0000 7fbffda00700 1 --2- 172.21.15.86:0/3692186522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbff8100e60 0x7fbff8199320 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.137+0000 7fbffda00700 1 --2- 172.21.15.86:0/3692186522 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbff8105f80 0x7fbff8199860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.137+0000 7fbffda00700 1 -- 172.21.15.86:0/3692186522 >> 172.21.15.86:0/3692186522 conn(0x7fbff80fc760 msgr2=0x7fbff80fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:34.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.138+0000 7fbffda00700 1 -- 172.21.15.86:0/3692186522 shutdown_connections 2024-03-23T17:51:34.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.138+0000 7fbffda00700 1 -- 172.21.15.86:0/3692186522 wait complete. 2024-03-23T17:51:34.139 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 28 2024-03-23T17:51:34.158 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-23T17:51:34.158 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-23T17:51:34.272 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.271+0000 7f27ddc00700 1 -- 172.21.15.86:0/2531108054 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27d8101ee0 msgr2=0x7f27d81022c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:34.272 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.271+0000 7f27ddc00700 1 --2- 172.21.15.86:0/2531108054 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27d8101ee0 0x7f27d81022c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f27c8009990 tx=0x7f27c8009ca0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.272 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.272+0000 7f27ddc00700 1 -- 172.21.15.86:0/2531108054 shutdown_connections 2024-03-23T17:51:34.272 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.272+0000 7f27ddc00700 1 --2- 172.21.15.86:0/2531108054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27d810b2b0 0x7f27d810d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.272 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.272+0000 7f27ddc00700 1 --2- 172.21.15.86:0/2531108054 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27d8101ee0 0x7f27d81022c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.273 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.272+0000 7f27ddc00700 1 --2- 172.21.15.86:0/2531108054 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f27d8102800 0x7f27d810ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.273 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.272+0000 7f27ddc00700 1 -- 172.21.15.86:0/2531108054 >> 172.21.15.86:0/2531108054 conn(0x7f27d80fb680 msgr2=0x7f27d80fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:34.273 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.272+0000 7f27ddc00700 1 -- 172.21.15.86:0/2531108054 shutdown_connections 2024-03-23T17:51:34.273 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.272+0000 7f27ddc00700 1 -- 172.21.15.86:0/2531108054 wait complete. 2024-03-23T17:51:34.274 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.273+0000 7f27ddc00700 1 Processor -- start 2024-03-23T17:51:34.274 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.273+0000 7f27ddc00700 1 -- start start 2024-03-23T17:51:34.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.274+0000 7f27ddc00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27d8101ee0 0x7f27d81993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:34.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.274+0000 7f27ddc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27d8102800 0x7f27d81998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:34.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.274+0000 7f27ddc00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f27d810b2b0 0x7f27d819dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:34.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.274+0000 7f27ddc00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d819e2c0 con 0x7f27d8102800 2024-03-23T17:51:34.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.274+0000 7f27ddc00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d819e430 con 0x7f27d8101ee0 2024-03-23T17:51:34.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.274+0000 7f27ddc00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d819e5f0 con 0x7f27d810b2b0 2024-03-23T17:51:34.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.274+0000 7f27d6000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27d8102800 0x7f27d81998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:34.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.274+0000 7f27d6a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27d8101ee0 0x7f27d81993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:34.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.274+0000 7f27d6000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27d8102800 0x7f27d81998e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32888/0 (socket says 172.21.15.86:32888) 2024-03-23T17:51:34.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.274+0000 7f27d6000700 1 -- 172.21.15.86:0/3684130881 learned_addr learned my addr 172.21.15.86:0/3684130881 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:34.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.274+0000 7f27d7400700 1 --2- 172.21.15.86:0/3684130881 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f27d810b2b0 0x7f27d819dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:34.277 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.275+0000 7f27d6a00700 1 -- 172.21.15.86:0/3684130881 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f27d810b2b0 msgr2=0x7f27d819dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:34.277 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.275+0000 7f27d6a00700 1 --2- 172.21.15.86:0/3684130881 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f27d810b2b0 0x7f27d819dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.277 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.275+0000 7f27d6a00700 1 -- 172.21.15.86:0/3684130881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27d8102800 msgr2=0x7f27d81998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:34.277 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.275+0000 7f27d6a00700 1 --2- 172.21.15.86:0/3684130881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27d8102800 0x7f27d81998e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.277 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.275+0000 7f27d6a00700 1 -- 172.21.15.86:0/3684130881 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27c80095f0 con 0x7f27d8101ee0 2024-03-23T17:51:34.277 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.275+0000 7f27d6000700 1 --2- 172.21.15.86:0/3684130881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27d8102800 0x7f27d81998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:51:34.278 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.275+0000 7f27d6a00700 1 --2- 172.21.15.86:0/3684130881 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27d8101ee0 0x7f27d81993a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f27c80059d0 tx=0x7f27c8004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:34.278 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.275+0000 7f27cf400700 1 -- 172.21.15.86:0/3684130881 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c801d070 con 0x7f27d8101ee0 2024-03-23T17:51:34.278 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.275+0000 7f27cf400700 1 -- 172.21.15.86:0/3684130881 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27c8022470 con 0x7f27d8101ee0 2024-03-23T17:51:34.278 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.275+0000 7f27ddc00700 1 -- 172.21.15.86:0/3684130881 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27d819e870 con 0x7f27d8101ee0 2024-03-23T17:51:34.278 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.275+0000 7f27cf400700 1 -- 172.21.15.86:0/3684130881 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c8017650 con 0x7f27d8101ee0 2024-03-23T17:51:34.278 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.276+0000 7f27ddc00700 1 -- 172.21.15.86:0/3684130881 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27d81a6110 con 0x7f27d8101ee0 2024-03-23T17:51:34.280 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.280+0000 7f27cf400700 1 -- 172.21.15.86:0/3684130881 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 25) v1 ==== 136098+0+0 (secure 0 0 0) 0x7f27c80177b0 con 0x7f27d8101ee0 2024-03-23T17:51:34.281 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.281+0000 7f27f173eb80 1 Processor -- start 2024-03-23T17:51:34.281 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.281+0000 7f27f173eb80 1 -- start start 2024-03-23T17:51:34.281 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.282+0000 7f27cf400700 1 --2- 172.21.15.86:0/3684130881 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f27bc0a13b0 0x7f27bc0a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:34.283 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.282+0000 7f27f173eb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56228a3e0100 0x56228a3e04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:34.283 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.282+0000 7f27f173eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56228a3e0a20 0x56228a3eb390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:34.284 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.282+0000 7f27cf400700 1 -- 172.21.15.86:0/3684130881 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f27c80cdff0 con 0x7f27d8101ee0 2024-03-23T17:51:34.284 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.282+0000 7f27f173eb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56228a3eb8d0 0x56228a3edcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:34.284 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.282+0000 7f27f173eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56228a339930 con 0x56228a3e0a20 2024-03-23T17:51:34.284 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.282+0000 7f27f173eb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x56228a3372f0 con 0x56228a3e0100 2024-03-23T17:51:34.284 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.282+0000 7f27f173eb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x56228a332be0 con 0x56228a3eb8d0 2024-03-23T17:51:34.284 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.282+0000 7f27d7400700 1 --2- 172.21.15.86:0/3684130881 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f27bc0a13b0 0x7f27bc0a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:34.284 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.282+0000 7f27d7400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56228a3eb8d0 0x56228a3edcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:34.284 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.282+0000 7f27d7400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56228a3eb8d0 0x56228a3edcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:47696/0 (socket says 172.21.15.86:47696) 2024-03-23T17:51:34.284 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.282+0000 7f27d7400700 1 -- 172.21.15.86:0/1418006602 learned_addr learned my addr 172.21.15.86:0/1418006602 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:34.284 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.282+0000 7f27d6000700 1 --2- 172.21.15.86:0/1418006602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56228a3e0a20 0x56228a3eb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.283+0000 7f27d7400700 1 -- 172.21.15.86:0/1418006602 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56228a3e0100 msgr2=0x56228a3e04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.283+0000 7f27d6a00700 1 --2- 172.21.15.86:0/1418006602 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56228a3e0100 0x56228a3e04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.283+0000 7f27d7400700 1 --2- 172.21.15.86:0/1418006602 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56228a3e0100 0x56228a3e04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.283+0000 7f27d7400700 1 -- 172.21.15.86:0/1418006602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56228a3e0a20 msgr2=0x56228a3eb390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.283+0000 7f27d7400700 1 --2- 172.21.15.86:0/1418006602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56228a3e0a20 0x56228a3eb390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.283+0000 7f27d7400700 1 -- 172.21.15.86:0/1418006602 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56228a1b4350 con 0x56228a3eb8d0 2024-03-23T17:51:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.283+0000 7f27d6a00700 1 --2- 172.21.15.86:0/1418006602 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56228a3e0100 0x56228a3e04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T17:51:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.283+0000 7f27d7400700 1 --2- 172.21.15.86:0/3684130881 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f27bc0a13b0 0x7f27bc0a3870 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f27d0009fd0 tx=0x7f27d0010000 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.283+0000 7f27d7400700 1 --2- 172.21.15.86:0/1418006602 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56228a3eb8d0 0x56228a3edcc0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f27d0008f80 tx=0x7f27d001a6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93f4e14f1416f125 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.283+0000 7f27c3e00700 1 -- 172.21.15.86:0/1418006602 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d0027070 con 0x56228a3eb8d0 2024-03-23T17:51:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.284+0000 7f27c3e00700 1 -- 172.21.15.86:0/1418006602 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27d001f070 con 0x56228a3eb8d0 2024-03-23T17:51:34.286 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.284+0000 7f27c3e00700 1 -- 172.21.15.86:0/1418006602 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d000e040 con 0x56228a3eb8d0 2024-03-23T17:51:34.292 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.284+0000 7f27f173eb80 1 -- 172.21.15.86:0/1418006602 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56228a3eb8d0 msgr2=0x56228a3edcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:34.292 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.284+0000 7f27f173eb80 1 --2- 172.21.15.86:0/1418006602 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56228a3eb8d0 0x56228a3edcc0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f27d0008f80 tx=0x7f27d001a6c0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.292 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.284+0000 7f27f173eb80 1 -- 172.21.15.86:0/1418006602 shutdown_connections 2024-03-23T17:51:34.292 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.284+0000 7f27f173eb80 1 --2- 172.21.15.86:0/1418006602 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56228a3eb8d0 0x56228a3edcc0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.292 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.284+0000 7f27f173eb80 1 --2- 172.21.15.86:0/1418006602 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56228a3e0100 0x56228a3e04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.292 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.284+0000 7f27f173eb80 1 --2- 172.21.15.86:0/1418006602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56228a3e0a20 0x56228a3eb390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.292 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.284+0000 7f27f173eb80 1 -- 172.21.15.86:0/1418006602 >> 172.21.15.86:0/1418006602 conn(0x56228a340640 msgr2=0x56228a3dc520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:34.292 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.284+0000 7f27f173eb80 1 -- 172.21.15.86:0/1418006602 shutdown_connections 2024-03-23T17:51:34.292 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.285+0000 7f27f173eb80 1 -- 172.21.15.86:0/1418006602 wait complete. 2024-03-23T17:51:34.292 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.285+0000 7f27f173eb80 1 -- start start 2024-03-23T17:51:34.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.286+0000 7f27f173eb80 1 Processor -- start 2024-03-23T17:51:34.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.286+0000 7f27f173eb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56228a3e0100 0x56228a3eb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:34.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.286+0000 7f27f173eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56228a3e0a20 0x56228a3e8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:34.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.286+0000 7f27f173eb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56228a3eb8d0 0x56228a3e8c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:34.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.286+0000 7f27f173eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56228a2ced20 con 0x56228a3e0a20 2024-03-23T17:51:34.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.286+0000 7f27f173eb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x56228a2ce9c0 con 0x56228a3eb8d0 2024-03-23T17:51:34.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.286+0000 7f27f173eb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x56228a1b6640 con 0x56228a3e0100 2024-03-23T17:51:34.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.286+0000 7f27f173eb80 10 client.0 _ll_register_callbacks cb 0x56228a2d8410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:51:34.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.287+0000 7f27d6000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56228a3e0a20 0x56228a3e8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:34.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.287+0000 7f27d6a00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56228a3e0100 0x56228a3eb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:34.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.287+0000 7f27d6a00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56228a3e0100 0x56228a3eb2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:47700/0 (socket says 172.21.15.86:47700) 2024-03-23T17:51:34.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.287+0000 7f27d6a00700 1 -- 172.21.15.86:0/3002661759 learned_addr learned my addr 172.21.15.86:0/3002661759 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:34.294 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.287+0000 7f27d6000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56228a3e0a20 0x56228a3e8610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32904/0 (socket says 172.21.15.86:32904) 2024-03-23T17:51:34.294 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.287+0000 7f27d7400700 1 --2- 172.21.15.86:0/3002661759 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56228a3eb8d0 0x56228a3e8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:34.294 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.287+0000 7f27d6a00700 1 -- 172.21.15.86:0/3002661759 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56228a3eb8d0 msgr2=0x56228a3e8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:34.294 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.287+0000 7f27d6a00700 1 --2- 172.21.15.86:0/3002661759 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56228a3eb8d0 0x56228a3e8c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.294 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.287+0000 7f27d6a00700 1 -- 172.21.15.86:0/3002661759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56228a3e0a20 msgr2=0x56228a3e8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:34.294 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.287+0000 7f27d6a00700 1 --2- 172.21.15.86:0/3002661759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56228a3e0a20 0x56228a3e8610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.294 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.287+0000 7f27d6a00700 1 -- 172.21.15.86:0/3002661759 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27d0008b30 con 0x56228a3e0100 2024-03-23T17:51:34.294 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.287+0000 7f27d7400700 1 --2- 172.21.15.86:0/3002661759 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56228a3eb8d0 0x56228a3e8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:51:34.294 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.287+0000 7f27d6a00700 1 --2- 172.21.15.86:0/3002661759 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56228a3e0100 0x56228a3eb2d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f27c8097490 tx=0x7f27c809ab50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b1c6636de861b8c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:34.294 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.287+0000 7f27c0c00700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-03-23T17:51:34.294 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.288+0000 7f27c0c00700 1 -- 172.21.15.86:0/3002661759 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c80a10c0 con 0x56228a3e0100 2024-03-23T17:51:34.295 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.288+0000 7f27c0c00700 1 -- 172.21.15.86:0/3002661759 <== mon.1 v2:172.21.15.152:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f27c809af90 con 0x56228a3e0100 2024-03-23T17:51:34.295 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.288+0000 7f27f173eb80 1 -- 172.21.15.86:0/3002661759 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56228a2f3320 con 0x56228a3e0100 2024-03-23T17:51:34.295 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.288+0000 7f27c0c00700 1 -- 172.21.15.86:0/3002661759 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27c809edc0 con 0x56228a3e0100 2024-03-23T17:51:34.295 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.288+0000 7f27f173eb80 10 client.25732 fetch_fsmap learned FSMap version 28 2024-03-23T17:51:34.295 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.288+0000 7f27f173eb80 1 -- 172.21.15.86:0/3002661759 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56228a160b00 con 0x56228a3e0100 2024-03-23T17:51:34.295 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.288+0000 7f27c0c00700 1 -- 172.21.15.86:0/3002661759 <== mon.1 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27c80b7020 con 0x56228a3e0100 2024-03-23T17:51:34.295 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.288+0000 7f27c0c00700 1 -- 172.21.15.86:0/3002661759 <== mon.1 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f27c80a3010 con 0x56228a3e0100 2024-03-23T17:51:34.295 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.289+0000 7f27f173eb80 10 client.25732 fetch_fsmap finished waiting for FSMap version 28 2024-03-23T17:51:34.295 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.289+0000 7f27f173eb80 10 client.25732 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:51:34.295 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.289+0000 7f27f173eb80 20 client.25732 populate_metadata read hostname 'smithi086' 2024-03-23T17:51:34.295 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.289+0000 7f27f173eb80 1 --2- 172.21.15.86:0/3002661759 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x56228a3fb090 0x56228a3fd480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:34.295 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.289+0000 7f27f173eb80 4 client.25732 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:51:34.296 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.289+0000 7f27f173eb80 1 -- 172.21.15.86:0/3002661759 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56228a32b6a0 con 0x56228a3fb090 2024-03-23T17:51:34.296 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.289+0000 7f27d6000700 1 --2- 172.21.15.86:0/3002661759 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x56228a3fb090 0x56228a3fd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:34.296 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.289+0000 7f27d6000700 1 --2- 172.21.15.86:0/3002661759 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x56228a3fb090 0x56228a3fd480 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c655e9ce4b7d5b7a server_cookie=a2cf3c2a9bb3d7fa in_seq=0 out_seq=0 2024-03-23T17:51:34.296 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.289+0000 7f27c0c00700 10 client.25732 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:51:34.296 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.290+0000 7f27c0c00700 1 -- 172.21.15.86:0/3002661759 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x56228a32b6a0 con 0x56228a3fb090 2024-03-23T17:51:34.296 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.290+0000 7f27c0c00700 10 client.25732 handle_command_reply: tid=0 2024-03-23T17:51:34.296 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.290+0000 7f27f173eb80 1 client.25732 shutdown 2024-03-23T17:51:34.296 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.290+0000 7f27f173eb80 1 -- 172.21.15.86:0/3002661759 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56228a3e0100 msgr2=0x56228a3eb2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:34.296 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.290+0000 7f27f173eb80 1 --2- 172.21.15.86:0/3002661759 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56228a3e0100 0x56228a3eb2d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f27c8097490 tx=0x7f27c809ab50 comp rx=0 tx=0).stop 2024-03-23T17:51:34.296 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.290+0000 7f27f173eb80 1 -- 172.21.15.86:0/3002661759 shutdown_connections 2024-03-23T17:51:34.296 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.290+0000 7f27f173eb80 1 --2- 172.21.15.86:0/3002661759 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x56228a3fb090 0x56228a3fd480 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.296 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.291+0000 7f27f173eb80 1 --2- 172.21.15.86:0/3002661759 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56228a3eb8d0 0x56228a3e8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.291+0000 7f27f173eb80 1 --2- 172.21.15.86:0/3002661759 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56228a3e0100 0x56228a3eb2d0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.291+0000 7f27c0c00700 0 client.25732 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:51:34.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.291+0000 7f27f173eb80 1 --2- 172.21.15.86:0/3002661759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56228a3e0a20 0x56228a3e8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.291+0000 7f27f173eb80 1 -- 172.21.15.86:0/3002661759 >> 172.21.15.86:0/3002661759 conn(0x56228a340640 msgr2=0x56228a3e20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:34.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.291+0000 7f27f173eb80 1 -- 172.21.15.86:0/3002661759 shutdown_connections 2024-03-23T17:51:34.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.291+0000 7f27f173eb80 1 -- 172.21.15.86:0/3002661759 wait complete. 2024-03-23T17:51:34.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.291+0000 7f27f173eb80 20 client.25732 trim_cache size 0 max 16384 2024-03-23T17:51:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.305+0000 7f27f173eb80 1 Processor -- start 2024-03-23T17:51:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.306+0000 7f27f173eb80 1 -- start start 2024-03-23T17:51:34.308 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.306+0000 7f27f173eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56228a3e0100 0x56228a3e85b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:34.308 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.306+0000 7f27f173eb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56228a3e0a20 0x56228a3f25b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:34.308 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.306+0000 7f27f173eb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56228a3eb8d0 0x56228a3d6500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:34.308 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.306+0000 7f27f173eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56228a3e13e0 con 0x56228a3e0100 2024-03-23T17:51:34.308 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.306+0000 7f27f173eb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x56228a3ddac0 con 0x56228a3e0a20 2024-03-23T17:51:34.308 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.306+0000 7f27f173eb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x56228a3f8040 con 0x56228a3eb8d0 2024-03-23T17:51:34.308 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.306+0000 7f27d6a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56228a3e0100 0x56228a3e85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:34.308 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.307+0000 7f27d6000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56228a3e0a20 0x56228a3f25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:34.308 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.307+0000 7f27d6000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56228a3e0a20 0x56228a3f25b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:51402/0 (socket says 172.21.15.86:51402) 2024-03-23T17:51:34.309 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.307+0000 7f27d6000700 1 -- 172.21.15.86:0/1256458860 learned_addr learned my addr 172.21.15.86:0/1256458860 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:34.309 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.307+0000 7f27d7400700 1 --2- 172.21.15.86:0/1256458860 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56228a3eb8d0 0x56228a3d6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:34.309 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.307+0000 7f27d6000700 1 -- 172.21.15.86:0/1256458860 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56228a3eb8d0 msgr2=0x56228a3d6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:34.309 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.307+0000 7f27d6000700 1 --2- 172.21.15.86:0/1256458860 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56228a3eb8d0 0x56228a3d6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.309 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.307+0000 7f27d6000700 1 -- 172.21.15.86:0/1256458860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56228a3e0100 msgr2=0x56228a3e85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:34.309 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.307+0000 7f27d6000700 1 --2- 172.21.15.86:0/1256458860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56228a3e0100 0x56228a3e85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.309 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.307+0000 7f27d6000700 1 -- 172.21.15.86:0/1256458860 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56228a2f8e40 con 0x56228a3e0a20 2024-03-23T17:51:34.309 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.307+0000 7f27d6a00700 1 --2- 172.21.15.86:0/1256458860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56228a3e0100 0x56228a3e85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:51:34.309 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.307+0000 7f27d6000700 1 --2- 172.21.15.86:0/1256458860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56228a3e0a20 0x56228a3f25b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f27c40096b0 tx=0x7f27c4004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=569bc502bed8db4e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:34.309 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.307+0000 7f27ccc00700 1 -- 172.21.15.86:0/1256458860 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c4016050 con 0x56228a3e0a20 2024-03-23T17:51:34.310 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.308+0000 7f27ccc00700 1 -- 172.21.15.86:0/1256458860 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27c4017040 con 0x56228a3e0a20 2024-03-23T17:51:34.310 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.308+0000 7f27ccc00700 1 -- 172.21.15.86:0/1256458860 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c40044f0 con 0x56228a3e0a20 2024-03-23T17:51:34.313 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.308+0000 7f27f173eb80 1 -- 172.21.15.86:0/1256458860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56228a3e0a20 msgr2=0x56228a3f25b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:34.313 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.308+0000 7f27f173eb80 1 --2- 172.21.15.86:0/1256458860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56228a3e0a20 0x56228a3f25b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f27c40096b0 tx=0x7f27c4004750 comp rx=0 tx=0).stop 2024-03-23T17:51:34.313 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.308+0000 7f27f173eb80 1 -- 172.21.15.86:0/1256458860 shutdown_connections 2024-03-23T17:51:34.313 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.308+0000 7f27f173eb80 1 --2- 172.21.15.86:0/1256458860 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56228a3eb8d0 0x56228a3d6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.313 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.308+0000 7f27f173eb80 1 --2- 172.21.15.86:0/1256458860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56228a3e0100 0x56228a3e85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.313 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.308+0000 7f27f173eb80 1 --2- 172.21.15.86:0/1256458860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56228a3e0a20 0x56228a3f25b0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.313 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.308+0000 7f27f173eb80 1 -- 172.21.15.86:0/1256458860 >> 172.21.15.86:0/1256458860 conn(0x56228a340640 msgr2=0x56228a3e7ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:34.313 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.308+0000 7f27f173eb80 1 -- 172.21.15.86:0/1256458860 shutdown_connections 2024-03-23T17:51:34.314 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.308+0000 7f27f173eb80 1 -- 172.21.15.86:0/1256458860 wait complete. 2024-03-23T17:51:34.314 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.309+0000 7f27f173eb80 1 -- start start 2024-03-23T17:51:34.314 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.309+0000 7f27f173eb80 1 Processor -- start 2024-03-23T17:51:34.314 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.310+0000 7f27f173eb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56228a3e0100 0x56228a3f21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:34.314 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.310+0000 7f27f173eb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56228a3e0a20 0x56228a3f0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:34.314 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.310+0000 7f27f173eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56228a3eb8d0 0x56228a3f0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:34.314 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.310+0000 7f27f173eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56228a3f10d0 con 0x56228a3eb8d0 2024-03-23T17:51:34.314 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.310+0000 7f27f173eb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x56228a3f1240 con 0x56228a3e0100 2024-03-23T17:51:34.314 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.310+0000 7f27f173eb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x56228a3f1400 con 0x56228a3e0a20 2024-03-23T17:51:34.314 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.310+0000 7f27f173eb80 10 client.0 _ll_register_callbacks cb 0x56228a2d8410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:51:34.314 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.310+0000 7f27d6a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56228a3e0100 0x56228a3f21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:34.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.310+0000 7f27d6a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56228a3e0100 0x56228a3f21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:51418/0 (socket says 172.21.15.86:51418) 2024-03-23T17:51:34.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.310+0000 7f27d6a00700 1 -- 172.21.15.86:0/4106493308 learned_addr learned my addr 172.21.15.86:0/4106493308 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:34.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.310+0000 7f27d7400700 1 --2- 172.21.15.86:0/4106493308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56228a3eb8d0 0x56228a3f0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:34.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.310+0000 7f27d6000700 1 --2- 172.21.15.86:0/4106493308 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56228a3e0a20 0x56228a3f0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:34.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.310+0000 7f27d6a00700 1 -- 172.21.15.86:0/4106493308 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56228a3e0a20 msgr2=0x56228a3f0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:34.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.310+0000 7f27d6a00700 1 --2- 172.21.15.86:0/4106493308 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56228a3e0a20 0x56228a3f0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.310+0000 7f27d6a00700 1 -- 172.21.15.86:0/4106493308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56228a3eb8d0 msgr2=0x56228a3f0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:34.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.310+0000 7f27d6a00700 1 --2- 172.21.15.86:0/4106493308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56228a3eb8d0 0x56228a3f0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.310+0000 7f27d6a00700 1 -- 172.21.15.86:0/4106493308 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56228a160b00 con 0x56228a3e0100 2024-03-23T17:51:34.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.310+0000 7f27d6000700 1 --2- 172.21.15.86:0/4106493308 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56228a3e0a20 0x56228a3f0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T17:51:34.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.310+0000 7f27d6a00700 1 --2- 172.21.15.86:0/4106493308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56228a3e0100 0x56228a3f21b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f27c80979b0 tx=0x7f27c8005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c92afa6f24af2e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:34.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.310+0000 7f27c0c00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T17:51:34.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.311+0000 7f27c0c00700 1 -- 172.21.15.86:0/4106493308 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c80a30f0 con 0x56228a3e0100 2024-03-23T17:51:34.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.311+0000 7f27c0c00700 1 -- 172.21.15.86:0/4106493308 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f27c80b8080 con 0x56228a3e0100 2024-03-23T17:51:34.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.311+0000 7f27f173eb80 1 -- 172.21.15.86:0/4106493308 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f27d001f070 con 0x56228a3e0100 2024-03-23T17:51:34.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.311+0000 7f27c0c00700 1 -- 172.21.15.86:0/4106493308 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27c809a8f0 con 0x56228a3e0100 2024-03-23T17:51:34.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.311+0000 7f27f173eb80 10 client.25736 fetch_fsmap learned FSMap version 28 2024-03-23T17:51:34.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.311+0000 7f27f173eb80 1 -- 172.21.15.86:0/4106493308 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f27c4010850 con 0x56228a3e0100 2024-03-23T17:51:34.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.311+0000 7f27c0c00700 1 -- 172.21.15.86:0/4106493308 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27d001f070 con 0x56228a3e0100 2024-03-23T17:51:34.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.311+0000 7f27c0c00700 1 -- 172.21.15.86:0/4106493308 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 28) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f27c8095ba0 con 0x56228a3e0100 2024-03-23T17:51:34.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.311+0000 7f27f173eb80 10 client.25736 fetch_fsmap finished waiting for FSMap version 28 2024-03-23T17:51:34.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.311+0000 7f27f173eb80 10 client.25736 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:51:34.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.311+0000 7f27f173eb80 20 client.25736 populate_metadata read hostname 'smithi086' 2024-03-23T17:51:34.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.311+0000 7f27f173eb80 1 --2- 172.21.15.86:0/4106493308 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x56228a44dce0 0x56228a450100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:34.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.311+0000 7f27f173eb80 4 client.25736 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T17:51:34.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.312+0000 7f27f173eb80 1 -- 172.21.15.86:0/4106493308 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56228a181c90 con 0x56228a44dce0 2024-03-23T17:51:34.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.312+0000 7f27d6000700 1 --2- 172.21.15.86:0/4106493308 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x56228a44dce0 0x56228a450100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:34.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.312+0000 7f27c0c00700 10 client.25736 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:51:34.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.312+0000 7f27d6000700 1 --2- 172.21.15.86:0/4106493308 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x56228a44dce0 0x56228a450100 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a6d78a488636696 server_cookie=2e5bceaf614e4015 in_seq=0 out_seq=0 2024-03-23T17:51:34.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.316+0000 7f27c0c00700 1 -- 172.21.15.86:0/4106493308 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56228a181c90 con 0x56228a44dce0 2024-03-23T17:51:34.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.316+0000 7f27c0c00700 10 client.25736 handle_command_reply: tid=0 2024-03-23T17:51:34.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.316+0000 7f27f173eb80 1 client.25736 shutdown 2024-03-23T17:51:34.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.316+0000 7f27f173eb80 1 -- 172.21.15.86:0/4106493308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56228a3e0100 msgr2=0x56228a3f21b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:34.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.316+0000 7f27f173eb80 1 --2- 172.21.15.86:0/4106493308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56228a3e0100 0x56228a3f21b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f27c80979b0 tx=0x7f27c8005c00 comp rx=0 tx=0).stop 2024-03-23T17:51:34.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.316+0000 7f27f173eb80 1 -- 172.21.15.86:0/4106493308 shutdown_connections 2024-03-23T17:51:34.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.316+0000 7f27f173eb80 1 --2- 172.21.15.86:0/4106493308 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x56228a44dce0 0x56228a450100 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.316+0000 7f27f173eb80 1 --2- 172.21.15.86:0/4106493308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56228a3eb8d0 0x56228a3f0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.316+0000 7f27c0c00700 0 client.25736 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:51:34.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.316+0000 7f27f173eb80 1 --2- 172.21.15.86:0/4106493308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56228a3e0100 0x56228a3f21b0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.316+0000 7f27f173eb80 1 --2- 172.21.15.86:0/4106493308 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56228a3e0a20 0x56228a3f0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.316+0000 7f27f173eb80 1 -- 172.21.15.86:0/4106493308 >> 172.21.15.86:0/4106493308 conn(0x56228a340640 msgr2=0x56228a3e2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:34.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.317+0000 7f27f173eb80 1 -- 172.21.15.86:0/4106493308 shutdown_connections 2024-03-23T17:51:34.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.317+0000 7f27f173eb80 1 -- 172.21.15.86:0/4106493308 wait complete. 2024-03-23T17:51:34.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.317+0000 7f27f173eb80 20 client.25736 trim_cache size 0 max 16384 2024-03-23T17:51:34.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-03-23T17:51:34.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-03-23T17:51:34.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-03-23T17:51:34.321 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-03-23T17:51:34.321 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.317+0000 7f27ddc00700 1 -- 172.21.15.86:0/3684130881 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f27bc0a13b0 msgr2=0x7f27bc0a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:34.321 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.317+0000 7f27ddc00700 1 --2- 172.21.15.86:0/3684130881 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f27bc0a13b0 0x7f27bc0a3870 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f27d0009fd0 tx=0x7f27d0010000 comp rx=0 tx=0).stop 2024-03-23T17:51:34.321 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.317+0000 7f27ddc00700 1 -- 172.21.15.86:0/3684130881 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27d8101ee0 msgr2=0x7f27d81993a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:34.322 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.317+0000 7f27ddc00700 1 --2- 172.21.15.86:0/3684130881 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27d8101ee0 0x7f27d81993a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f27c80059d0 tx=0x7f27c8004730 comp rx=0 tx=0).stop 2024-03-23T17:51:34.322 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.317+0000 7f27ddc00700 1 -- 172.21.15.86:0/3684130881 shutdown_connections 2024-03-23T17:51:34.322 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.317+0000 7f27ddc00700 1 --2- 172.21.15.86:0/3684130881 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f27bc0a13b0 0x7f27bc0a3870 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.322 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.317+0000 7f27ddc00700 1 --2- 172.21.15.86:0/3684130881 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f27d810b2b0 0x7f27d819dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.322 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.317+0000 7f27ddc00700 1 --2- 172.21.15.86:0/3684130881 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27d8101ee0 0x7f27d81993a0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.322 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.317+0000 7f27ddc00700 1 --2- 172.21.15.86:0/3684130881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27d8102800 0x7f27d81998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.322 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.317+0000 7f27ddc00700 1 -- 172.21.15.86:0/3684130881 >> 172.21.15.86:0/3684130881 conn(0x7f27d80fb680 msgr2=0x7f27d80fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:34.322 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.318+0000 7f27ddc00700 1 -- 172.21.15.86:0/3684130881 shutdown_connections 2024-03-23T17:51:34.322 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:34.318+0000 7f27ddc00700 1 -- 172.21.15.86:0/3684130881 wait complete. 2024-03-23T17:51:34.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.404+0000 7f93a1800700 1 -- 172.21.15.86:0/2686592689 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f939c0731e0 msgr2=0x7f939c0735c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:34.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.404+0000 7f93a1800700 1 --2- 172.21.15.86:0/2686592689 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f939c0731e0 0x7f939c0735c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f938c0098f0 tx=0x7f938c009c00 comp rx=0 tx=0).stop 2024-03-23T17:51:34.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.404+0000 7f93a1800700 1 -- 172.21.15.86:0/2686592689 shutdown_connections 2024-03-23T17:51:34.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.404+0000 7f93a1800700 1 --2- 172.21.15.86:0/2686592689 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f939c10d540 0x7f939c10f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.404+0000 7f93a1800700 1 --2- 172.21.15.86:0/2686592689 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f939c0731e0 0x7f939c0735c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.404+0000 7f93a1800700 1 --2- 172.21.15.86:0/2686592689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f939c073b00 0x7f939c10d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.404+0000 7f93a1800700 1 -- 172.21.15.86:0/2686592689 >> 172.21.15.86:0/2686592689 conn(0x7f939c0fc7f0 msgr2=0x7f939c0fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:34.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.404+0000 7f93a1800700 1 -- 172.21.15.86:0/2686592689 shutdown_connections 2024-03-23T17:51:34.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.404+0000 7f93a1800700 1 -- 172.21.15.86:0/2686592689 wait complete. 2024-03-23T17:51:34.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.405+0000 7f93a1800700 1 Processor -- start 2024-03-23T17:51:34.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.405+0000 7f93a1800700 1 -- start start 2024-03-23T17:51:34.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.406+0000 7f93a1800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f939c0731e0 0x7f939c199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:34.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.406+0000 7f93a1800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f939c073b00 0x7f939c1999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:34.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.406+0000 7f93a1800700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f939c10d540 0x7f939c19dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:34.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.406+0000 7f93a1800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f939c19e380 con 0x7f939c0731e0 2024-03-23T17:51:34.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.406+0000 7f93a1800700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f939c19e4f0 con 0x7f939c073b00 2024-03-23T17:51:34.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.406+0000 7f93a1800700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f939c19e6b0 con 0x7f939c10d540 2024-03-23T17:51:34.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.406+0000 7f939a000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f939c073b00 0x7f939c1999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:34.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.406+0000 7f939a000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f939c073b00 0x7f939c1999a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:51430/0 (socket says 172.21.15.86:51430) 2024-03-23T17:51:34.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.406+0000 7f939a000700 1 -- 172.21.15.86:0/143896774 learned_addr learned my addr 172.21.15.86:0/143896774 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:34.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.406+0000 7f939b400700 1 --2- 172.21.15.86:0/143896774 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f939c10d540 0x7f939c19dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:34.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.406+0000 7f939aa00700 1 --2- 172.21.15.86:0/143896774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f939c0731e0 0x7f939c199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:34.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.406+0000 7f939a000700 1 -- 172.21.15.86:0/143896774 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f939c10d540 msgr2=0x7f939c19dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:34.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.406+0000 7f939a000700 1 --2- 172.21.15.86:0/143896774 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f939c10d540 0x7f939c19dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.406+0000 7f939a000700 1 -- 172.21.15.86:0/143896774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f939c0731e0 msgr2=0x7f939c199460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:34.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.406+0000 7f939a000700 1 --2- 172.21.15.86:0/143896774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f939c0731e0 0x7f939c199460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.406+0000 7f939a000700 1 -- 172.21.15.86:0/143896774 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93880095f0 con 0x7f939c073b00 2024-03-23T17:51:34.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.407+0000 7f939a000700 1 --2- 172.21.15.86:0/143896774 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f939c073b00 0x7f939c1999a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f938800ec10 tx=0x7f938800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:34.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.407+0000 7f9393400700 1 -- 172.21.15.86:0/143896774 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f938800cc90 con 0x7f939c073b00 2024-03-23T17:51:34.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.407+0000 7f9393400700 1 -- 172.21.15.86:0/143896774 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93880044e0 con 0x7f939c073b00 2024-03-23T17:51:34.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.407+0000 7f93a1800700 1 -- 172.21.15.86:0/143896774 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f938c009520 con 0x7f939c073b00 2024-03-23T17:51:34.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.407+0000 7f9393400700 1 -- 172.21.15.86:0/143896774 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9388010430 con 0x7f939c073b00 2024-03-23T17:51:34.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.407+0000 7f93a1800700 1 -- 172.21.15.86:0/143896774 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f939c19eb80 con 0x7f939c073b00 2024-03-23T17:51:34.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.410+0000 7f9393400700 1 -- 172.21.15.86:0/143896774 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 25) v1 ==== 136098+0+0 (secure 0 0 0) 0x7f9388003fb0 con 0x7f939c073b00 2024-03-23T17:51:34.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.412+0000 7f93b541fb80 1 Processor -- start 2024-03-23T17:51:34.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.412+0000 7f9393400700 1 --2- 172.21.15.86:0/143896774 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f93800a1430 0x7f93800a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:34.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.412+0000 7f9393400700 1 -- 172.21.15.86:0/143896774 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f9388014070 con 0x7f939c073b00 2024-03-23T17:51:34.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.412+0000 7f93b541fb80 1 -- start start 2024-03-23T17:51:34.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.412+0000 7f939b400700 1 --2- 172.21.15.86:0/143896774 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f93800a1430 0x7f93800a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:34.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.413+0000 7f93b541fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c839afb5d0 0x55c839afb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:34.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.413+0000 7f93b541fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c839afbef0 0x55c839b068b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:34.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.413+0000 7f93b541fb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c839b06df0 0x55c839b091e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:34.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.413+0000 7f93b541fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c839a54990 con 0x55c839afb5d0 2024-03-23T17:51:34.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.413+0000 7f93b541fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c839a52350 con 0x55c839afbef0 2024-03-23T17:51:34.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.413+0000 7f93b541fb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c839a4dc40 con 0x55c839b06df0 2024-03-23T17:51:34.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.413+0000 7f939a000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c839afbef0 0x55c839b068b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:34.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.413+0000 7f939aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c839afb5d0 0x55c839afb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:34.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.413+0000 7f939a000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c839afbef0 0x55c839b068b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:51446/0 (socket says 172.21.15.86:51446) 2024-03-23T17:51:34.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.413+0000 7f939a000700 1 -- 172.21.15.86:0/1808359573 learned_addr learned my addr 172.21.15.86:0/1808359573 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:34.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.413+0000 7f939b400700 1 --2- 172.21.15.86:0/143896774 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f93800a1430 0x7f93800a38f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f939400f540 tx=0x7f939400b040 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:34.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.413+0000 7f939b400700 1 --2- 172.21.15.86:0/1808359573 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c839b06df0 0x55c839b091e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:34.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.413+0000 7f939a000700 1 -- 172.21.15.86:0/1808359573 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c839b06df0 msgr2=0x55c839b091e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:34.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.413+0000 7f939a000700 1 --2- 172.21.15.86:0/1808359573 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c839b06df0 0x55c839b091e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.413+0000 7f939a000700 1 -- 172.21.15.86:0/1808359573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c839afb5d0 msgr2=0x55c839afb9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:34.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.413+0000 7f939a000700 1 --2- 172.21.15.86:0/1808359573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c839afb5d0 0x55c839afb9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.413+0000 7f939a000700 1 -- 172.21.15.86:0/1808359573 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8398cf3b0 con 0x55c839afbef0 2024-03-23T17:51:34.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.414+0000 7f939a000700 1 --2- 172.21.15.86:0/1808359573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c839afbef0 0x55c839b068b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f93880a2290 tx=0x7f938809a190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=433ab635fa37992b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:34.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.414+0000 7f9387e00700 1 -- 172.21.15.86:0/1808359573 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f938809aa40 con 0x55c839afbef0 2024-03-23T17:51:34.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.414+0000 7f9387e00700 1 -- 172.21.15.86:0/1808359573 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f938809a340 con 0x55c839afbef0 2024-03-23T17:51:34.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.414+0000 7f9387e00700 1 -- 172.21.15.86:0/1808359573 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f938809a5a0 con 0x55c839afbef0 2024-03-23T17:51:34.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.415+0000 7f93b541fb80 1 -- 172.21.15.86:0/1808359573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c839afbef0 msgr2=0x55c839b068b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:34.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.415+0000 7f93b541fb80 1 --2- 172.21.15.86:0/1808359573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c839afbef0 0x55c839b068b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f93880a2290 tx=0x7f938809a190 comp rx=0 tx=0).stop 2024-03-23T17:51:34.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.415+0000 7f93b541fb80 1 -- 172.21.15.86:0/1808359573 shutdown_connections 2024-03-23T17:51:34.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.415+0000 7f93b541fb80 1 --2- 172.21.15.86:0/1808359573 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c839b06df0 0x55c839b091e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.415+0000 7f93b541fb80 1 --2- 172.21.15.86:0/1808359573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c839afb5d0 0x55c839afb9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.415+0000 7f93b541fb80 1 --2- 172.21.15.86:0/1808359573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c839afbef0 0x55c839b068b0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.415+0000 7f93b541fb80 1 -- 172.21.15.86:0/1808359573 >> 172.21.15.86:0/1808359573 conn(0x55c839a5b6a0 msgr2=0x55c839af7a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:34.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.415+0000 7f93b541fb80 1 -- 172.21.15.86:0/1808359573 shutdown_connections 2024-03-23T17:51:34.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.415+0000 7f93b541fb80 1 -- 172.21.15.86:0/1808359573 wait complete. 2024-03-23T17:51:34.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.416+0000 7f93b541fb80 1 -- start start 2024-03-23T17:51:34.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.416+0000 7f93b541fb80 1 Processor -- start 2024-03-23T17:51:34.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.416+0000 7f93b541fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c839afb5d0 0x55c839b06800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:34.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.416+0000 7f93b541fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c839afbef0 0x55c839b03af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:34.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.416+0000 7f93b541fb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c839b06df0 0x55c839b04120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:34.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.416+0000 7f93b541fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8399e9d80 con 0x55c839afb5d0 2024-03-23T17:51:34.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.416+0000 7f93b541fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8399e9a20 con 0x55c839afbef0 2024-03-23T17:51:34.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.416+0000 7f93b541fb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8398d16a0 con 0x55c839b06df0 2024-03-23T17:51:34.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.417+0000 7f93b541fb80 10 client.0 _ll_register_callbacks cb 0x55c8399f3470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:51:34.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.417+0000 7f939aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c839afb5d0 0x55c839b06800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:34.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.417+0000 7f939a000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c839afbef0 0x55c839b03af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:34.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.417+0000 7f939b400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c839b06df0 0x55c839b04120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:34.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.417+0000 7f939b400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c839b06df0 0x55c839b04120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:47754/0 (socket says 172.21.15.86:47754) 2024-03-23T17:51:34.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.417+0000 7f939b400700 1 -- 172.21.15.86:0/2770097288 learned_addr learned my addr 172.21.15.86:0/2770097288 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:34.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.417+0000 7f939a000700 1 -- 172.21.15.86:0/2770097288 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c839b06df0 msgr2=0x55c839b04120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:34.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.417+0000 7f939a000700 1 --2- 172.21.15.86:0/2770097288 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c839b06df0 0x55c839b04120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.417+0000 7f939a000700 1 -- 172.21.15.86:0/2770097288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c839afb5d0 msgr2=0x55c839b06800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:34.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.417+0000 7f939a000700 1 --2- 172.21.15.86:0/2770097288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c839afb5d0 0x55c839b06800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.417+0000 7f939a000700 1 -- 172.21.15.86:0/2770097288 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f939c19eb80 con 0x55c839afbef0 2024-03-23T17:51:34.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.417+0000 7f939a000700 1 --2- 172.21.15.86:0/2770097288 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c839afbef0 0x55c839b03af0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f938809a2d0 tx=0x7f93880163f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c26482411fb841ad server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:34.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.418+0000 7f9384c00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T17:51:34.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.418+0000 7f9384c00700 1 -- 172.21.15.86:0/2770097288 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f938809e170 con 0x55c839afbef0 2024-03-23T17:51:34.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.418+0000 7f9384c00700 1 -- 172.21.15.86:0/2770097288 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f93880a27c0 con 0x55c839afbef0 2024-03-23T17:51:34.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.418+0000 7f93b541fb80 1 -- 172.21.15.86:0/2770097288 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c839a0e380 con 0x55c839afbef0 2024-03-23T17:51:34.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.418+0000 7f9384c00700 1 -- 172.21.15.86:0/2770097288 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f938809a5a0 con 0x55c839afbef0 2024-03-23T17:51:34.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.418+0000 7f93b541fb80 10 client.25748 fetch_fsmap learned FSMap version 28 2024-03-23T17:51:34.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.418+0000 7f93b541fb80 1 -- 172.21.15.86:0/2770097288 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c83987bb60 con 0x55c839afbef0 2024-03-23T17:51:34.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.418+0000 7f9384c00700 1 -- 172.21.15.86:0/2770097288 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c839a0e380 con 0x55c839afbef0 2024-03-23T17:51:34.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.418+0000 7f9384c00700 1 -- 172.21.15.86:0/2770097288 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 28) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f93880b0070 con 0x55c839afbef0 2024-03-23T17:51:34.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.418+0000 7f93b541fb80 10 client.25748 fetch_fsmap finished waiting for FSMap version 28 2024-03-23T17:51:34.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.418+0000 7f93b541fb80 10 client.25748 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-03-23T17:51:34.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.418+0000 7f93b541fb80 20 client.25748 populate_metadata read hostname 'smithi086' 2024-03-23T17:51:34.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.419+0000 7f93b541fb80 1 --2- 172.21.15.86:0/2770097288 >> [v2:172.21.15.92:6834/908900027,v1:172.21.15.92:6835/908900027] conn(0x55c839b165b0 0x55c839b189d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:34.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.419+0000 7f93b541fb80 4 client.25748 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:51:34.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.419+0000 7f93b541fb80 1 -- 172.21.15.86:0/2770097288 --> [v2:172.21.15.92:6834/908900027,v1:172.21.15.92:6835/908900027] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c839a46700 con 0x55c839b165b0 2024-03-23T17:51:34.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.419+0000 7f939aa00700 1 --2- 172.21.15.86:0/2770097288 >> [v2:172.21.15.92:6834/908900027,v1:172.21.15.92:6835/908900027] conn(0x55c839b165b0 0x55c839b189d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:34.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.419+0000 7f939aa00700 1 --2- 172.21.15.86:0/2770097288 >> [v2:172.21.15.92:6834/908900027,v1:172.21.15.92:6835/908900027] conn(0x55c839b165b0 0x55c839b189d0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e95e9bb75f0bab0f server_cookie=a1b3bda5b8068ae in_seq=0 out_seq=0 2024-03-23T17:51:34.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.419+0000 7f9384c00700 10 client.25748 ms_handle_connect on v2:172.21.15.92:6834/908900027 2024-03-23T17:51:34.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.420+0000 7f9384c00700 1 -- 172.21.15.86:0/2770097288 <== mds.4 v2:172.21.15.92:6834/908900027 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55c839a46700 con 0x55c839b165b0 2024-03-23T17:51:34.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.420+0000 7f9384c00700 10 client.25748 handle_command_reply: tid=0 2024-03-23T17:51:34.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.420+0000 7f93b541fb80 1 client.25748 shutdown 2024-03-23T17:51:34.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.420+0000 7f93b541fb80 1 -- 172.21.15.86:0/2770097288 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c839afbef0 msgr2=0x55c839b03af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:34.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.420+0000 7f93b541fb80 1 --2- 172.21.15.86:0/2770097288 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c839afbef0 0x55c839b03af0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f938809a2d0 tx=0x7f93880163f0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.420+0000 7f93b541fb80 1 -- 172.21.15.86:0/2770097288 shutdown_connections 2024-03-23T17:51:34.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.420+0000 7f93b541fb80 1 --2- 172.21.15.86:0/2770097288 >> [v2:172.21.15.92:6834/908900027,v1:172.21.15.92:6835/908900027] conn(0x55c839b165b0 0x55c839b189d0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.420+0000 7f93b541fb80 1 --2- 172.21.15.86:0/2770097288 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c839b06df0 0x55c839b04120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.420+0000 7f93b541fb80 1 --2- 172.21.15.86:0/2770097288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c839afb5d0 0x55c839b06800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.420+0000 7f9384c00700 0 client.25748 ms_handle_reset on v2:172.21.15.92:6834/908900027 2024-03-23T17:51:34.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.420+0000 7f93b541fb80 1 --2- 172.21.15.86:0/2770097288 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c839afbef0 0x55c839b03af0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.420+0000 7f93b541fb80 1 -- 172.21.15.86:0/2770097288 >> 172.21.15.86:0/2770097288 conn(0x55c839a5b6a0 msgr2=0x55c839afd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:34.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.420+0000 7f93b541fb80 1 -- 172.21.15.86:0/2770097288 shutdown_connections 2024-03-23T17:51:34.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.420+0000 7f93b541fb80 1 -- 172.21.15.86:0/2770097288 wait complete. 2024-03-23T17:51:34.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.421+0000 7f93b541fb80 20 client.25748 trim_cache size 0 max 16384 2024-03-23T17:51:34.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.434+0000 7f93b541fb80 1 Processor -- start 2024-03-23T17:51:34.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.434+0000 7f93b541fb80 1 -- start start 2024-03-23T17:51:34.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.434+0000 7f93b541fb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c839afb5d0 0x55c839b189a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:34.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.434+0000 7f93b541fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c839afbef0 0x55c839b1d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:34.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.434+0000 7f93b541fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c839b06df0 0x55c839b0da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:34.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.434+0000 7f93b541fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c839afc890 con 0x55c839afbef0 2024-03-23T17:51:34.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.434+0000 7f93b541fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c839b04720 con 0x55c839b06df0 2024-03-23T17:51:34.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.434+0000 7f93b541fb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c839af92b0 con 0x55c839afb5d0 2024-03-23T17:51:34.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.434+0000 7f939a000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c839afbef0 0x55c839b1d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:34.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.435+0000 7f939a000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c839afbef0 0x55c839b1d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32968/0 (socket says 172.21.15.86:32968) 2024-03-23T17:51:34.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.435+0000 7f939a000700 1 -- 172.21.15.86:0/2694607783 learned_addr learned my addr 172.21.15.86:0/2694607783 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:34.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.435+0000 7f939b400700 1 --2- 172.21.15.86:0/2694607783 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c839b06df0 0x55c839b0da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:34.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.435+0000 7f939aa00700 1 --2- 172.21.15.86:0/2694607783 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c839afb5d0 0x55c839b189a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:34.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.435+0000 7f939a000700 1 -- 172.21.15.86:0/2694607783 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c839afb5d0 msgr2=0x55c839b189a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:34.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.435+0000 7f939a000700 1 --2- 172.21.15.86:0/2694607783 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c839afb5d0 0x55c839b189a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.435+0000 7f939a000700 1 -- 172.21.15.86:0/2694607783 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c839b06df0 msgr2=0x55c839b0da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:34.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.435+0000 7f939a000700 1 --2- 172.21.15.86:0/2694607783 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c839b06df0 0x55c839b0da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.435+0000 7f939a000700 1 -- 172.21.15.86:0/2694607783 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c839a13ea0 con 0x55c839afbef0 2024-03-23T17:51:34.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.435+0000 7f939aa00700 1 --2- 172.21.15.86:0/2694607783 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c839afb5d0 0x55c839b189a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:51:34.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.435+0000 7f939a000700 1 --2- 172.21.15.86:0/2694607783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c839afbef0 0x55c839b1d350 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f938809a040 tx=0x7f9388004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ececf79fbf42afd server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:34.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.435+0000 7f9390c00700 1 -- 172.21.15.86:0/2694607783 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f938809aa40 con 0x55c839afbef0 2024-03-23T17:51:34.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.435+0000 7f9390c00700 1 -- 172.21.15.86:0/2694607783 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9388099e80 con 0x55c839afbef0 2024-03-23T17:51:34.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.435+0000 7f9390c00700 1 -- 172.21.15.86:0/2694607783 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f938809a5a0 con 0x55c839afbef0 2024-03-23T17:51:34.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.436+0000 7f93b541fb80 1 -- 172.21.15.86:0/2694607783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c839afbef0 msgr2=0x55c839b1d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:34.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.436+0000 7f93b541fb80 1 --2- 172.21.15.86:0/2694607783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c839afbef0 0x55c839b1d350 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f938809a040 tx=0x7f9388004470 comp rx=0 tx=0).stop 2024-03-23T17:51:34.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.436+0000 7f93b541fb80 1 -- 172.21.15.86:0/2694607783 shutdown_connections 2024-03-23T17:51:34.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.436+0000 7f93b541fb80 1 --2- 172.21.15.86:0/2694607783 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c839b06df0 0x55c839b0da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.436+0000 7f93b541fb80 1 --2- 172.21.15.86:0/2694607783 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c839afb5d0 0x55c839b189a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.436+0000 7f93b541fb80 1 --2- 172.21.15.86:0/2694607783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c839afbef0 0x55c839b1d350 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.436+0000 7f93b541fb80 1 -- 172.21.15.86:0/2694607783 >> 172.21.15.86:0/2694607783 conn(0x55c839a5b6a0 msgr2=0x55c839af05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:34.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.436+0000 7f93b541fb80 1 -- 172.21.15.86:0/2694607783 shutdown_connections 2024-03-23T17:51:34.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.436+0000 7f93b541fb80 1 -- 172.21.15.86:0/2694607783 wait complete. 2024-03-23T17:51:34.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.437+0000 7f93b541fb80 1 -- start start 2024-03-23T17:51:34.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.437+0000 7f93b541fb80 1 Processor -- start 2024-03-23T17:51:34.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.437+0000 7f93b541fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c839afb5d0 0x55c839b0bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:34.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.437+0000 7f93b541fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c839afbef0 0x55c839b0c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:34.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.437+0000 7f93b541fb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c839b06df0 0x55c839b670f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:34.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.437+0000 7f93b541fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c839b0c860 con 0x55c839afbef0 2024-03-23T17:51:34.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.437+0000 7f93b541fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c839b67630 con 0x55c839afb5d0 2024-03-23T17:51:34.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.437+0000 7f93b541fb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c839b67770 con 0x55c839b06df0 2024-03-23T17:51:34.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.438+0000 7f93b541fb80 10 client.0 _ll_register_callbacks cb 0x55c8399f3470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:51:34.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.438+0000 7f939aa00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c839afb5d0 0x55c839b0bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:34.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.438+0000 7f939aa00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c839afb5d0 0x55c839b0bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:51470/0 (socket says 172.21.15.86:51470) 2024-03-23T17:51:34.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.438+0000 7f939aa00700 1 -- 172.21.15.86:0/87054460 learned_addr learned my addr 172.21.15.86:0/87054460 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:34.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.438+0000 7f939a000700 1 --2- 172.21.15.86:0/87054460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c839afbef0 0x55c839b0c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:34.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.438+0000 7f939b400700 1 --2- 172.21.15.86:0/87054460 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c839b06df0 0x55c839b670f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:34.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.438+0000 7f939aa00700 1 -- 172.21.15.86:0/87054460 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c839b06df0 msgr2=0x55c839b670f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:34.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.438+0000 7f939aa00700 1 --2- 172.21.15.86:0/87054460 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c839b06df0 0x55c839b670f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.438+0000 7f939aa00700 1 -- 172.21.15.86:0/87054460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c839afbef0 msgr2=0x55c839b0c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:34.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.438+0000 7f939aa00700 1 --2- 172.21.15.86:0/87054460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c839afbef0 0x55c839b0c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.438+0000 7f939aa00700 1 -- 172.21.15.86:0/87054460 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c83987bb60 con 0x55c839afb5d0 2024-03-23T17:51:34.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.438+0000 7f939aa00700 1 --2- 172.21.15.86:0/87054460 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c839afb5d0 0x55c839b0bc40 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f938c0098f0 tx=0x7f938c00f670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a80fcf05ab8461ab server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:34.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.438+0000 7f9384c00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T17:51:34.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.439+0000 7f9384c00700 1 -- 172.21.15.86:0/87054460 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f938c025a80 con 0x55c839afb5d0 2024-03-23T17:51:34.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.439+0000 7f93b541fb80 1 -- 172.21.15.86:0/87054460 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f938809a340 con 0x55c839afb5d0 2024-03-23T17:51:34.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.439+0000 7f9384c00700 1 -- 172.21.15.86:0/87054460 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f938c025be0 con 0x55c839afb5d0 2024-03-23T17:51:34.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.439+0000 7f9384c00700 1 -- 172.21.15.86:0/87054460 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f938c025410 con 0x55c839afb5d0 2024-03-23T17:51:34.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.439+0000 7f93b541fb80 10 client.25754 fetch_fsmap learned FSMap version 28 2024-03-23T17:51:34.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.439+0000 7f93b541fb80 1 -- 172.21.15.86:0/87054460 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c8398d3cd0 con 0x55c839afb5d0 2024-03-23T17:51:34.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.439+0000 7f9384c00700 1 -- 172.21.15.86:0/87054460 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f938809a340 con 0x55c839afb5d0 2024-03-23T17:51:34.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.440+0000 7f9384c00700 1 -- 172.21.15.86:0/87054460 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 28) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f938c009920 con 0x55c839afb5d0 2024-03-23T17:51:34.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.440+0000 7f93b541fb80 10 client.25754 fetch_fsmap finished waiting for FSMap version 28 2024-03-23T17:51:34.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.440+0000 7f93b541fb80 10 client.25754 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-03-23T17:51:34.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.440+0000 7f93b541fb80 20 client.25754 populate_metadata read hostname 'smithi086' 2024-03-23T17:51:34.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.440+0000 7f93b541fb80 1 --2- 172.21.15.86:0/87054460 >> [v2:172.21.15.92:6834/908900027,v1:172.21.15.92:6835/908900027] conn(0x55c839b717b0 0x55c839b73bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:34.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.440+0000 7f93b541fb80 4 client.25754 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-23T17:51:34.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.440+0000 7f93b541fb80 1 -- 172.21.15.86:0/87054460 --> [v2:172.21.15.92:6834/908900027,v1:172.21.15.92:6835/908900027] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c83989ccf0 con 0x55c839b717b0 2024-03-23T17:51:34.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.440+0000 7f939a000700 1 --2- 172.21.15.86:0/87054460 >> [v2:172.21.15.92:6834/908900027,v1:172.21.15.92:6835/908900027] conn(0x55c839b717b0 0x55c839b73bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:34.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.441+0000 7f9384c00700 10 client.25754 ms_handle_connect on v2:172.21.15.92:6834/908900027 2024-03-23T17:51:34.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.441+0000 7f939a000700 1 --2- 172.21.15.86:0/87054460 >> [v2:172.21.15.92:6834/908900027,v1:172.21.15.92:6835/908900027] conn(0x55c839b717b0 0x55c839b73bd0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9116bf58147fb7ce server_cookie=ce7cce160d8c38a7 in_seq=0 out_seq=0 2024-03-23T17:51:34.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.441+0000 7f9384c00700 1 -- 172.21.15.86:0/87054460 <== mds.4 v2:172.21.15.92:6834/908900027 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c83989ccf0 con 0x55c839b717b0 2024-03-23T17:51:34.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.441+0000 7f9384c00700 10 client.25754 handle_command_reply: tid=0 2024-03-23T17:51:34.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.441+0000 7f93b541fb80 1 client.25754 shutdown 2024-03-23T17:51:34.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.441+0000 7f93b541fb80 1 -- 172.21.15.86:0/87054460 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c839afb5d0 msgr2=0x55c839b0bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:34.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.441+0000 7f93b541fb80 1 --2- 172.21.15.86:0/87054460 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c839afb5d0 0x55c839b0bc40 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f938c0098f0 tx=0x7f938c00f670 comp rx=0 tx=0).stop 2024-03-23T17:51:34.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.441+0000 7f93b541fb80 1 -- 172.21.15.86:0/87054460 shutdown_connections 2024-03-23T17:51:34.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.441+0000 7f93b541fb80 1 --2- 172.21.15.86:0/87054460 >> [v2:172.21.15.92:6834/908900027,v1:172.21.15.92:6835/908900027] conn(0x55c839b717b0 0x55c839b73bd0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.441+0000 7f93b541fb80 1 --2- 172.21.15.86:0/87054460 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c839b06df0 0x55c839b670f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.441+0000 7f93b541fb80 1 --2- 172.21.15.86:0/87054460 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c839afb5d0 0x55c839b0bc40 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.441+0000 7f9384c00700 0 client.25754 ms_handle_reset on v2:172.21.15.92:6834/908900027 2024-03-23T17:51:34.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.441+0000 7f93b541fb80 1 --2- 172.21.15.86:0/87054460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c839afbef0 0x55c839b0c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.441+0000 7f93b541fb80 1 -- 172.21.15.86:0/87054460 >> 172.21.15.86:0/87054460 conn(0x55c839a5b6a0 msgr2=0x55c839b03380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:34.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.441+0000 7f93b541fb80 1 -- 172.21.15.86:0/87054460 shutdown_connections 2024-03-23T17:51:34.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.441+0000 7f93b541fb80 1 -- 172.21.15.86:0/87054460 wait complete. 2024-03-23T17:51:34.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.441+0000 7f93b541fb80 20 client.25754 trim_cache size 0 max 16384 2024-03-23T17:51:34.447 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-03-23T17:51:34.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.442+0000 7f93a1800700 1 -- 172.21.15.86:0/143896774 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f93800a1430 msgr2=0x7f93800a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:34.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.442+0000 7f93a1800700 1 --2- 172.21.15.86:0/143896774 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f93800a1430 0x7f93800a38f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f939400f540 tx=0x7f939400b040 comp rx=0 tx=0).stop 2024-03-23T17:51:34.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.442+0000 7f93a1800700 1 -- 172.21.15.86:0/143896774 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f939c073b00 msgr2=0x7f939c1999a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:34.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.442+0000 7f93a1800700 1 --2- 172.21.15.86:0/143896774 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f939c073b00 0x7f939c1999a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f938800ec10 tx=0x7f938800ef20 comp rx=0 tx=0).stop 2024-03-23T17:51:34.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.442+0000 7f93a1800700 1 -- 172.21.15.86:0/143896774 shutdown_connections 2024-03-23T17:51:34.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.442+0000 7f93a1800700 1 --2- 172.21.15.86:0/143896774 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f93800a1430 0x7f93800a38f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.442+0000 7f93a1800700 1 --2- 172.21.15.86:0/143896774 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f939c10d540 0x7f939c19dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.442+0000 7f93a1800700 1 --2- 172.21.15.86:0/143896774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f939c0731e0 0x7f939c199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.442+0000 7f93a1800700 1 --2- 172.21.15.86:0/143896774 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f939c073b00 0x7f939c1999a0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.442+0000 7f93a1800700 1 -- 172.21.15.86:0/143896774 >> 172.21.15.86:0/143896774 conn(0x7f939c0fc7f0 msgr2=0x7f939c0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:34.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.442+0000 7f93a1800700 1 -- 172.21.15.86:0/143896774 shutdown_connections 2024-03-23T17:51:34.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.442+0000 7f93a1800700 1 -- 172.21.15.86:0/143896774 wait complete. 2024-03-23T17:51:34.463 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-23T17:51:34.463 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-23T17:51:34.626 INFO:tasks.workunit.client.0.smithi086.stderr:Note: switching to '82316b04821a6c2c1d458a881920454bbadb8837'. 2024-03-23T17:51:34.626 INFO:tasks.workunit.client.0.smithi086.stderr: 2024-03-23T17:51:34.626 INFO:tasks.workunit.client.0.smithi086.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-23T17:51:34.626 INFO:tasks.workunit.client.0.smithi086.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-23T17:51:34.626 INFO:tasks.workunit.client.0.smithi086.stderr:state without impacting any branches by switching back to a branch. 2024-03-23T17:51:34.626 INFO:tasks.workunit.client.0.smithi086.stderr: 2024-03-23T17:51:34.626 INFO:tasks.workunit.client.0.smithi086.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-23T17:51:34.626 INFO:tasks.workunit.client.0.smithi086.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-23T17:51:34.626 INFO:tasks.workunit.client.0.smithi086.stderr: 2024-03-23T17:51:34.626 INFO:tasks.workunit.client.0.smithi086.stderr: git switch -c 2024-03-23T17:51:34.626 INFO:tasks.workunit.client.0.smithi086.stderr: 2024-03-23T17:51:34.627 INFO:tasks.workunit.client.0.smithi086.stderr:Or undo this operation with: 2024-03-23T17:51:34.627 INFO:tasks.workunit.client.0.smithi086.stderr: 2024-03-23T17:51:34.627 INFO:tasks.workunit.client.0.smithi086.stderr: git switch - 2024-03-23T17:51:34.627 INFO:tasks.workunit.client.0.smithi086.stderr: 2024-03-23T17:51:34.627 INFO:tasks.workunit.client.0.smithi086.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-23T17:51:34.627 INFO:tasks.workunit.client.0.smithi086.stderr: 2024-03-23T17:51:34.627 INFO:tasks.workunit.client.0.smithi086.stderr:HEAD is now at 82316b04821 Merge branch 'wip-64881-reef' of https://github.com/sseshasa/ceph into wip-yuri11-testing-2024-03-21-1324-reef 2024-03-23T17:51:34.637 DEBUG:teuthology.orchestra.run.smithi086:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-23T17:51:34.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.663+0000 7f0101600700 1 -- 172.21.15.86:0/2492618935 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f00fc101920 msgr2=0x7f00fc105960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:34.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.663+0000 7f0101600700 1 --2- 172.21.15.86:0/2492618935 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f00fc101920 0x7f00fc105960 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f00e800faf0 tx=0x7f00e800fe00 comp rx=0 tx=0).stop 2024-03-23T17:51:34.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.663+0000 7f0101600700 1 -- 172.21.15.86:0/2492618935 shutdown_connections 2024-03-23T17:51:34.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.663+0000 7f0101600700 1 --2- 172.21.15.86:0/2492618935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00fc106090 0x7f00fc10dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.663+0000 7f0101600700 1 --2- 172.21.15.86:0/2492618935 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00fc100f70 0x7f00fc101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.663+0000 7f0101600700 1 --2- 172.21.15.86:0/2492618935 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f00fc101920 0x7f00fc105960 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.663+0000 7f0101600700 1 -- 172.21.15.86:0/2492618935 >> 172.21.15.86:0/2492618935 conn(0x7f00fc0fc7f0 msgr2=0x7f00fc0fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:34.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.663+0000 7f0101600700 1 -- 172.21.15.86:0/2492618935 shutdown_connections 2024-03-23T17:51:34.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.663+0000 7f0101600700 1 -- 172.21.15.86:0/2492618935 wait complete. 2024-03-23T17:51:34.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.664+0000 7f0101600700 1 Processor -- start 2024-03-23T17:51:34.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.664+0000 7f0101600700 1 -- start start 2024-03-23T17:51:34.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.665+0000 7f0101600700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f00fc100f70 0x7f00fc1950e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:34.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.665+0000 7f0101600700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00fc106090 0x7f00fc195620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:34.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.665+0000 7f0101600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00fc1999c0 0x7f00fc199e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:34.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.665+0000 7f0101600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00fc19a440 con 0x7f00fc1999c0 2024-03-23T17:51:34.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.665+0000 7f0101600700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00fc19a5b0 con 0x7f00fc106090 2024-03-23T17:51:34.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.665+0000 7f0101600700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00fc19a770 con 0x7f00fc100f70 2024-03-23T17:51:34.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.665+0000 7f00fa000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00fc106090 0x7f00fc195620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:34.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.665+0000 7f00faa00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f00fc100f70 0x7f00fc1950e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:34.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.665+0000 7f00fa000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00fc106090 0x7f00fc195620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:51484/0 (socket says 172.21.15.86:51484) 2024-03-23T17:51:34.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.665+0000 7f00fa000700 1 -- 172.21.15.86:0/2102182354 learned_addr learned my addr 172.21.15.86:0/2102182354 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:34.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.665+0000 7f00fb400700 1 --2- 172.21.15.86:0/2102182354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00fc1999c0 0x7f00fc199e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:34.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.666+0000 7f00fa000700 1 -- 172.21.15.86:0/2102182354 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f00fc100f70 msgr2=0x7f00fc1950e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:34.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.666+0000 7f00fa000700 1 --2- 172.21.15.86:0/2102182354 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f00fc100f70 0x7f00fc1950e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.666+0000 7f00fa000700 1 -- 172.21.15.86:0/2102182354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00fc1999c0 msgr2=0x7f00fc199e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:34.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.666+0000 7f00fa000700 1 --2- 172.21.15.86:0/2102182354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00fc1999c0 0x7f00fc199e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.666+0000 7f00fa000700 1 -- 172.21.15.86:0/2102182354 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00e800f750 con 0x7f00fc106090 2024-03-23T17:51:34.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.666+0000 7f00fa000700 1 --2- 172.21.15.86:0/2102182354 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00fc106090 0x7f00fc195620 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f00e8005f50 tx=0x7f00e800f300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:34.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.666+0000 7f00f3400700 1 -- 172.21.15.86:0/2102182354 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00e8003ba0 con 0x7f00fc106090 2024-03-23T17:51:34.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.666+0000 7f00f3400700 1 -- 172.21.15.86:0/2102182354 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f00e80041c0 con 0x7f00fc106090 2024-03-23T17:51:34.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.666+0000 7f00f3400700 1 -- 172.21.15.86:0/2102182354 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00e800ed40 con 0x7f00fc106090 2024-03-23T17:51:34.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.666+0000 7f0101600700 1 -- 172.21.15.86:0/2102182354 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00fc1a1c70 con 0x7f00fc106090 2024-03-23T17:51:34.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.667+0000 7f0101600700 1 -- 172.21.15.86:0/2102182354 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00fc1a20f0 con 0x7f00fc106090 2024-03-23T17:51:34.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.669+0000 7f00f3400700 1 -- 172.21.15.86:0/2102182354 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 25) v1 ==== 136098+0+0 (secure 0 0 0) 0x7f00e8003d00 con 0x7f00fc106090 2024-03-23T17:51:34.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.671+0000 7f00f3400700 1 --2- 172.21.15.86:0/2102182354 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f00e00a1430 0x7f00e00a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:34.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.671+0000 7f01151cfb80 1 Processor -- start 2024-03-23T17:51:34.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.671+0000 7f00f3400700 1 -- 172.21.15.86:0/2102182354 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f00e80d2490 con 0x7f00fc106090 2024-03-23T17:51:34.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.671+0000 7f01151cfb80 1 -- start start 2024-03-23T17:51:34.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.671+0000 7f00faa00700 1 --2- 172.21.15.86:0/2102182354 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f00e00a1430 0x7f00e00a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:34.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.672+0000 7f01151cfb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ec991685d0 0x55ec991689b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:34.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.672+0000 7f01151cfb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec99168ef0 0x55ec991738b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:34.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.672+0000 7f01151cfb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec99173df0 0x55ec991761e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:34.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.672+0000 7f01151cfb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec990c1990 con 0x55ec99173df0 2024-03-23T17:51:34.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.672+0000 7f01151cfb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec990bf350 con 0x55ec99168ef0 2024-03-23T17:51:34.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.672+0000 7f01151cfb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec990bac40 con 0x55ec991685d0 2024-03-23T17:51:34.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.672+0000 7f00fa000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec99168ef0 0x55ec991738b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:34.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.672+0000 7f00fa000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec99168ef0 0x55ec991738b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:51490/0 (socket says 172.21.15.86:51490) 2024-03-23T17:51:34.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.672+0000 7f00fa000700 1 -- 172.21.15.86:0/3309903990 learned_addr learned my addr 172.21.15.86:0/3309903990 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:34.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.672+0000 7f00faa00700 1 --2- 172.21.15.86:0/3309903990 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ec991685d0 0x55ec991689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:34.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.672+0000 7f00fb400700 1 --2- 172.21.15.86:0/3309903990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec99173df0 0x55ec991761e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:34.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.672+0000 7f00faa00700 1 --2- 172.21.15.86:0/2102182354 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f00e00a1430 0x7f00e00a38f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f00ec009cb0 tx=0x7f00ec009260 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:34.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.672+0000 7f00fa000700 1 -- 172.21.15.86:0/3309903990 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ec991685d0 msgr2=0x55ec991689b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:34.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.672+0000 7f00fa000700 1 --2- 172.21.15.86:0/3309903990 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ec991685d0 0x55ec991689b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.672+0000 7f00fa000700 1 -- 172.21.15.86:0/3309903990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec99173df0 msgr2=0x55ec991761e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:34.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.672+0000 7f00fa000700 1 --2- 172.21.15.86:0/3309903990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec99173df0 0x55ec991761e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.672+0000 7f00fa000700 1 -- 172.21.15.86:0/3309903990 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec98f3c3b0 con 0x55ec99168ef0 2024-03-23T17:51:34.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.673+0000 7f00fa000700 1 --2- 172.21.15.86:0/3309903990 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec99168ef0 0x55ec991738b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f00e8006010 tx=0x7f00e800f6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41c7cbfdc124f9d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:34.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.673+0000 7f00e7400700 1 -- 172.21.15.86:0/3309903990 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00e800e660 con 0x55ec99168ef0 2024-03-23T17:51:34.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.673+0000 7f00e7400700 1 -- 172.21.15.86:0/3309903990 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f00e809fad0 con 0x55ec99168ef0 2024-03-23T17:51:34.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.673+0000 7f00e7400700 1 -- 172.21.15.86:0/3309903990 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00e809f890 con 0x55ec99168ef0 2024-03-23T17:51:34.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.674+0000 7f01151cfb80 1 -- 172.21.15.86:0/3309903990 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec99168ef0 msgr2=0x55ec991738b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:34.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.674+0000 7f01151cfb80 1 --2- 172.21.15.86:0/3309903990 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec99168ef0 0x55ec991738b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f00e8006010 tx=0x7f00e800f6a0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.674+0000 7f01151cfb80 1 -- 172.21.15.86:0/3309903990 shutdown_connections 2024-03-23T17:51:34.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.674+0000 7f01151cfb80 1 --2- 172.21.15.86:0/3309903990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec99173df0 0x55ec991761e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.674+0000 7f01151cfb80 1 --2- 172.21.15.86:0/3309903990 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ec991685d0 0x55ec991689b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.674+0000 7f01151cfb80 1 --2- 172.21.15.86:0/3309903990 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec99168ef0 0x55ec991738b0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.674+0000 7f01151cfb80 1 -- 172.21.15.86:0/3309903990 >> 172.21.15.86:0/3309903990 conn(0x55ec990c86a0 msgr2=0x55ec99164a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:34.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.674+0000 7f01151cfb80 1 -- 172.21.15.86:0/3309903990 shutdown_connections 2024-03-23T17:51:34.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.674+0000 7f01151cfb80 1 -- 172.21.15.86:0/3309903990 wait complete. 2024-03-23T17:51:34.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.674+0000 7f01151cfb80 1 -- start start 2024-03-23T17:51:34.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.675+0000 7f01151cfb80 1 Processor -- start 2024-03-23T17:51:34.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.675+0000 7f01151cfb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec991685d0 0x55ec991737e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:34.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.675+0000 7f01151cfb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ec99168ef0 0x55ec99170b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:34.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.675+0000 7f01151cfb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec99173df0 0x55ec99171130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:34.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.675+0000 7f01151cfb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec99056d80 con 0x55ec99173df0 2024-03-23T17:51:34.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.675+0000 7f01151cfb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec99056a20 con 0x55ec991685d0 2024-03-23T17:51:34.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.675+0000 7f01151cfb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec98f3e6a0 con 0x55ec99168ef0 2024-03-23T17:51:34.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.675+0000 7f01151cfb80 10 client.0 _ll_register_callbacks cb 0x55ec99060470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:51:34.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.675+0000 7f00fb400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec99173df0 0x55ec99171130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:34.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.675+0000 7f00fa000700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ec99168ef0 0x55ec99170b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:34.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.675+0000 7f00fa000700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ec99168ef0 0x55ec99170b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:47822/0 (socket says 172.21.15.86:47822) 2024-03-23T17:51:34.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.675+0000 7f00fa000700 1 -- 172.21.15.86:0/1522331582 learned_addr learned my addr 172.21.15.86:0/1522331582 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:34.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.675+0000 7f00faa00700 1 --2- 172.21.15.86:0/1522331582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec991685d0 0x55ec991737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:34.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.676+0000 7f00fa000700 1 -- 172.21.15.86:0/1522331582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec991685d0 msgr2=0x55ec991737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:34.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.676+0000 7f00fa000700 1 --2- 172.21.15.86:0/1522331582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec991685d0 0x55ec991737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.676+0000 7f00fa000700 1 -- 172.21.15.86:0/1522331582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec99173df0 msgr2=0x55ec99171130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:34.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.676+0000 7f00fa000700 1 --2- 172.21.15.86:0/1522331582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec99173df0 0x55ec99171130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.676+0000 7f00fa000700 1 -- 172.21.15.86:0/1522331582 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00fc1a20f0 con 0x55ec99168ef0 2024-03-23T17:51:34.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.676+0000 7f00fb400700 1 --2- 172.21.15.86:0/1522331582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec99173df0 0x55ec99171130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:51:34.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.676+0000 7f00fa000700 1 --2- 172.21.15.86:0/1522331582 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ec99168ef0 0x55ec99170b20 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f00e809f0d0 tx=0x7f00e8005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=479dd4a3f770b377 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:34.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.676+0000 7f00dbe00700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-03-23T17:51:34.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.676+0000 7f00dbe00700 1 -- 172.21.15.86:0/1522331582 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00e809f830 con 0x55ec99168ef0 2024-03-23T17:51:34.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.676+0000 7f01151cfb80 1 -- 172.21.15.86:0/1522331582 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ec9907b380 con 0x55ec99168ef0 2024-03-23T17:51:34.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.676+0000 7f00dbe00700 1 -- 172.21.15.86:0/1522331582 <== mon.1 v2:172.21.15.152:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f00e809f3d0 con 0x55ec99168ef0 2024-03-23T17:51:34.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.676+0000 7f00dbe00700 1 -- 172.21.15.86:0/1522331582 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00e80a77c0 con 0x55ec99168ef0 2024-03-23T17:51:34.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.677+0000 7f01151cfb80 10 client.25765 fetch_fsmap learned FSMap version 28 2024-03-23T17:51:34.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.677+0000 7f01151cfb80 1 -- 172.21.15.86:0/1522331582 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55ec98ee8b60 con 0x55ec99168ef0 2024-03-23T17:51:34.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.677+0000 7f00dbe00700 1 -- 172.21.15.86:0/1522331582 <== mon.1 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ec9907b380 con 0x55ec99168ef0 2024-03-23T17:51:34.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.677+0000 7f00dbe00700 1 -- 172.21.15.86:0/1522331582 <== mon.1 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f00e80b5070 con 0x55ec99168ef0 2024-03-23T17:51:34.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.677+0000 7f01151cfb80 10 client.25765 fetch_fsmap finished waiting for FSMap version 28 2024-03-23T17:51:34.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.677+0000 7f01151cfb80 10 client.25765 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.d 2024-03-23T17:51:34.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.677+0000 7f01151cfb80 20 client.25765 populate_metadata read hostname 'smithi086' 2024-03-23T17:51:34.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.677+0000 7f01151cfb80 1 --2- 172.21.15.86:0/1522331582 >> [v2:172.21.15.86:6836/2933881328,v1:172.21.15.86:6837/2933881328] conn(0x55ec99183190 0x55ec991855b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:34.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.677+0000 7f01151cfb80 4 client.25765 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:51:34.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.677+0000 7f01151cfb80 1 -- 172.21.15.86:0/1522331582 --> [v2:172.21.15.86:6836/2933881328,v1:172.21.15.86:6837/2933881328] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ec990b3700 con 0x55ec99183190 2024-03-23T17:51:34.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.677+0000 7f00faa00700 1 --2- 172.21.15.86:0/1522331582 >> [v2:172.21.15.86:6836/2933881328,v1:172.21.15.86:6837/2933881328] conn(0x55ec99183190 0x55ec991855b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:34.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.678+0000 7f00faa00700 1 --2- 172.21.15.86:0/1522331582 >> [v2:172.21.15.86:6836/2933881328,v1:172.21.15.86:6837/2933881328] conn(0x55ec99183190 0x55ec991855b0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=40ee9b73de865316 server_cookie=d03dcf21d2dfabec in_seq=0 out_seq=0 2024-03-23T17:51:34.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.678+0000 7f00dbe00700 10 client.25765 ms_handle_connect on v2:172.21.15.86:6836/2933881328 2024-03-23T17:51:34.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.678+0000 7f00dbe00700 1 -- 172.21.15.86:0/1522331582 <== mds.3 v2:172.21.15.86:6836/2933881328 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55ec990b3700 con 0x55ec99183190 2024-03-23T17:51:34.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.678+0000 7f00dbe00700 10 client.25765 handle_command_reply: tid=0 2024-03-23T17:51:34.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.678+0000 7f01151cfb80 1 client.25765 shutdown 2024-03-23T17:51:34.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.679+0000 7f01151cfb80 1 -- 172.21.15.86:0/1522331582 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ec99168ef0 msgr2=0x55ec99170b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:34.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.679+0000 7f01151cfb80 1 --2- 172.21.15.86:0/1522331582 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ec99168ef0 0x55ec99170b20 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f00e809f0d0 tx=0x7f00e8005c00 comp rx=0 tx=0).stop 2024-03-23T17:51:34.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.679+0000 7f01151cfb80 1 -- 172.21.15.86:0/1522331582 shutdown_connections 2024-03-23T17:51:34.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.679+0000 7f01151cfb80 1 --2- 172.21.15.86:0/1522331582 >> [v2:172.21.15.86:6836/2933881328,v1:172.21.15.86:6837/2933881328] conn(0x55ec99183190 0x55ec991855b0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.679+0000 7f01151cfb80 1 --2- 172.21.15.86:0/1522331582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec99173df0 0x55ec99171130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.679+0000 7f01151cfb80 1 --2- 172.21.15.86:0/1522331582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec991685d0 0x55ec991737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.679+0000 7f00dbe00700 0 client.25765 ms_handle_reset on v2:172.21.15.86:6836/2933881328 2024-03-23T17:51:34.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.679+0000 7f01151cfb80 1 --2- 172.21.15.86:0/1522331582 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ec99168ef0 0x55ec99170b20 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.679+0000 7f01151cfb80 1 -- 172.21.15.86:0/1522331582 >> 172.21.15.86:0/1522331582 conn(0x55ec990c86a0 msgr2=0x55ec9916a6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:34.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.679+0000 7f01151cfb80 1 -- 172.21.15.86:0/1522331582 shutdown_connections 2024-03-23T17:51:34.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.679+0000 7f01151cfb80 1 -- 172.21.15.86:0/1522331582 wait complete. 2024-03-23T17:51:34.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.679+0000 7f01151cfb80 20 client.25765 trim_cache size 0 max 16384 2024-03-23T17:51:34.687 INFO:tasks.workunit.client.0.smithi086.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-23T17:51:34.692 INFO:tasks.workunit.client.0.smithi086.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-23T17:51:34.692 INFO:tasks.workunit.client.0.smithi086.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-23T17:51:34.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.693+0000 7f01151cfb80 1 Processor -- start 2024-03-23T17:51:34.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.693+0000 7f01151cfb80 1 -- start start 2024-03-23T17:51:34.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.693+0000 7f01151cfb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec991685d0 0x55ec99189ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:34.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.693+0000 7f01151cfb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ec99168ef0 0x55ec9917a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:34.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.693+0000 7f01151cfb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec99173df0 0x55ec99170ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:34.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.693+0000 7f01151cfb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec99180200 con 0x55ec991685d0 2024-03-23T17:51:34.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.693+0000 7f01151cfb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec99169670 con 0x55ec99173df0 2024-03-23T17:51:34.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.693+0000 7f01151cfb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec991692d0 con 0x55ec99168ef0 2024-03-23T17:51:34.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.693+0000 7f00faa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec991685d0 0x55ec99189ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:34.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.694+0000 7f00fb400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec99173df0 0x55ec99170ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:34.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.694+0000 7f00faa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec991685d0 0x55ec99189ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33016/0 (socket says 172.21.15.86:33016) 2024-03-23T17:51:34.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.694+0000 7f00faa00700 1 -- 172.21.15.86:0/4233018223 learned_addr learned my addr 172.21.15.86:0/4233018223 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:34.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.694+0000 7f00fa000700 1 --2- 172.21.15.86:0/4233018223 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ec99168ef0 0x55ec9917a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:34.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.694+0000 7f00fb400700 1 -- 172.21.15.86:0/4233018223 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ec99168ef0 msgr2=0x55ec9917a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:34.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.694+0000 7f00fb400700 1 --2- 172.21.15.86:0/4233018223 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ec99168ef0 0x55ec9917a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.694+0000 7f00fb400700 1 -- 172.21.15.86:0/4233018223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec991685d0 msgr2=0x55ec99189ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:34.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.694+0000 7f00fb400700 1 --2- 172.21.15.86:0/4233018223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec991685d0 0x55ec99189ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.694+0000 7f00fb400700 1 -- 172.21.15.86:0/4233018223 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec99080ea0 con 0x55ec99173df0 2024-03-23T17:51:34.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.694+0000 7f00faa00700 1 --2- 172.21.15.86:0/4233018223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec991685d0 0x55ec99189ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:51:34.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.694+0000 7f00fb400700 1 --2- 172.21.15.86:0/4233018223 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec99173df0 0x55ec99170ac0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f00f4012a60 tx=0x7f00f4009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eac7c6bd98762e3d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:34.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.694+0000 7f00e7e00700 1 -- 172.21.15.86:0/4233018223 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f400e040 con 0x55ec99173df0 2024-03-23T17:51:34.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.694+0000 7f00e7e00700 1 -- 172.21.15.86:0/4233018223 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f00f400f040 con 0x55ec99173df0 2024-03-23T17:51:34.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.694+0000 7f00e7e00700 1 -- 172.21.15.86:0/4233018223 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f4017e20 con 0x55ec99173df0 2024-03-23T17:51:34.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.695+0000 7f01151cfb80 1 -- 172.21.15.86:0/4233018223 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec99173df0 msgr2=0x55ec99170ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:34.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.695+0000 7f01151cfb80 1 --2- 172.21.15.86:0/4233018223 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec99173df0 0x55ec99170ac0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f00f4012a60 tx=0x7f00f4009920 comp rx=0 tx=0).stop 2024-03-23T17:51:34.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.695+0000 7f01151cfb80 1 -- 172.21.15.86:0/4233018223 shutdown_connections 2024-03-23T17:51:34.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.695+0000 7f01151cfb80 1 --2- 172.21.15.86:0/4233018223 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec99173df0 0x55ec99170ac0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.695+0000 7f01151cfb80 1 --2- 172.21.15.86:0/4233018223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec991685d0 0x55ec99189ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.695+0000 7f01151cfb80 1 --2- 172.21.15.86:0/4233018223 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ec99168ef0 0x55ec9917a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.695+0000 7f01151cfb80 1 -- 172.21.15.86:0/4233018223 >> 172.21.15.86:0/4233018223 conn(0x55ec990c86a0 msgr2=0x55ec991703b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:34.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.695+0000 7f01151cfb80 1 -- 172.21.15.86:0/4233018223 shutdown_connections 2024-03-23T17:51:34.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.695+0000 7f01151cfb80 1 -- 172.21.15.86:0/4233018223 wait complete. 2024-03-23T17:51:34.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.696+0000 7f01151cfb80 1 -- start start 2024-03-23T17:51:34.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.696+0000 7f01151cfb80 1 Processor -- start 2024-03-23T17:51:34.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.696+0000 7f01151cfb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec991685d0 0x55ec9917a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:34.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.696+0000 7f01151cfb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ec99168ef0 0x55ec99178810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:34.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.696+0000 7f01151cfb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec99173df0 0x55ec99178d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:34.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.696+0000 7f01151cfb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec9917ac00 con 0x55ec991685d0 2024-03-23T17:51:34.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.696+0000 7f01151cfb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec99179290 con 0x55ec99173df0 2024-03-23T17:51:34.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.696+0000 7f01151cfb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec99179450 con 0x55ec99168ef0 2024-03-23T17:51:34.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.696+0000 7f01151cfb80 10 client.0 _ll_register_callbacks cb 0x55ec99060470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:51:34.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.697+0000 7f00faa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec991685d0 0x55ec9917a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:34.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.697+0000 7f00fb400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec99173df0 0x55ec99178d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:34.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.697+0000 7f00faa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec991685d0 0x55ec9917a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33022/0 (socket says 172.21.15.86:33022) 2024-03-23T17:51:34.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.697+0000 7f00faa00700 1 -- 172.21.15.86:0/2582390833 learned_addr learned my addr 172.21.15.86:0/2582390833 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:34.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.697+0000 7f00fa000700 1 --2- 172.21.15.86:0/2582390833 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ec99168ef0 0x55ec99178810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:34.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.697+0000 7f00fb400700 1 -- 172.21.15.86:0/2582390833 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ec99168ef0 msgr2=0x55ec99178810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:34.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.697+0000 7f00fb400700 1 --2- 172.21.15.86:0/2582390833 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ec99168ef0 0x55ec99178810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.697+0000 7f00fb400700 1 -- 172.21.15.86:0/2582390833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec991685d0 msgr2=0x55ec9917a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:34.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.697+0000 7f00fb400700 1 --2- 172.21.15.86:0/2582390833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec991685d0 0x55ec9917a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.697+0000 7f00fb400700 1 -- 172.21.15.86:0/2582390833 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00ec017050 con 0x55ec99173df0 2024-03-23T17:51:34.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.697+0000 7f00fb400700 1 --2- 172.21.15.86:0/2582390833 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec99173df0 0x55ec99178d50 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f00f40176f0 tx=0x7f00f4009510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e60565bcc0a29c11 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:34.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.697+0000 7f00dbe00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T17:51:34.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.697+0000 7f00dbe00700 1 -- 172.21.15.86:0/2582390833 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f400e050 con 0x55ec99173df0 2024-03-23T17:51:34.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.697+0000 7f01151cfb80 1 -- 172.21.15.86:0/2582390833 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f00e809fad0 con 0x55ec99173df0 2024-03-23T17:51:34.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.697+0000 7f00dbe00700 1 -- 172.21.15.86:0/2582390833 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f00f4003a00 con 0x55ec99173df0 2024-03-23T17:51:34.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.698+0000 7f00dbe00700 1 -- 172.21.15.86:0/2582390833 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00f4007500 con 0x55ec99173df0 2024-03-23T17:51:34.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.698+0000 7f01151cfb80 10 client.25772 fetch_fsmap learned FSMap version 28 2024-03-23T17:51:34.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.698+0000 7f01151cfb80 1 -- 172.21.15.86:0/2582390833 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f00f4012710 con 0x55ec99173df0 2024-03-23T17:51:34.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.698+0000 7f00dbe00700 1 -- 172.21.15.86:0/2582390833 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f00f4035020 con 0x55ec99173df0 2024-03-23T17:51:34.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.698+0000 7f00dbe00700 1 -- 172.21.15.86:0/2582390833 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 28) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f00f40316a0 con 0x55ec99173df0 2024-03-23T17:51:34.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.698+0000 7f01151cfb80 10 client.25772 fetch_fsmap finished waiting for FSMap version 28 2024-03-23T17:51:34.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.698+0000 7f01151cfb80 10 client.25772 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.d 2024-03-23T17:51:34.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.698+0000 7f01151cfb80 20 client.25772 populate_metadata read hostname 'smithi086' 2024-03-23T17:51:34.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.698+0000 7f01151cfb80 1 --2- 172.21.15.86:0/2582390833 >> [v2:172.21.15.86:6836/2933881328,v1:172.21.15.86:6837/2933881328] conn(0x55ec991d5f90 0x55ec991d83b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:34.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.698+0000 7f01151cfb80 4 client.25772 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-23T17:51:34.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.698+0000 7f01151cfb80 1 -- 172.21.15.86:0/2582390833 --> [v2:172.21.15.86:6836/2933881328,v1:172.21.15.86:6837/2933881328] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ec98f09cf0 con 0x55ec991d5f90 2024-03-23T17:51:34.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.698+0000 7f00faa00700 1 --2- 172.21.15.86:0/2582390833 >> [v2:172.21.15.86:6836/2933881328,v1:172.21.15.86:6837/2933881328] conn(0x55ec991d5f90 0x55ec991d83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:34.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.699+0000 7f00dbe00700 10 client.25772 ms_handle_connect on v2:172.21.15.86:6836/2933881328 2024-03-23T17:51:34.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.699+0000 7f00faa00700 1 --2- 172.21.15.86:0/2582390833 >> [v2:172.21.15.86:6836/2933881328,v1:172.21.15.86:6837/2933881328] conn(0x55ec991d5f90 0x55ec991d83b0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2a16fa85ecd82e9 server_cookie=1dccf8ff00aad6ee in_seq=0 out_seq=0 2024-03-23T17:51:34.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.699+0000 7f00dbe00700 1 -- 172.21.15.86:0/2582390833 <== mds.3 v2:172.21.15.86:6836/2933881328 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ec98f09cf0 con 0x55ec991d5f90 2024-03-23T17:51:34.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.699+0000 7f00dbe00700 10 client.25772 handle_command_reply: tid=0 2024-03-23T17:51:34.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.699+0000 7f01151cfb80 1 client.25772 shutdown 2024-03-23T17:51:34.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.699+0000 7f01151cfb80 1 -- 172.21.15.86:0/2582390833 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec99173df0 msgr2=0x55ec99178d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:34.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.699+0000 7f01151cfb80 1 --2- 172.21.15.86:0/2582390833 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec99173df0 0x55ec99178d50 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f00f40176f0 tx=0x7f00f4009510 comp rx=0 tx=0).stop 2024-03-23T17:51:34.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.699+0000 7f01151cfb80 1 -- 172.21.15.86:0/2582390833 shutdown_connections 2024-03-23T17:51:34.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.699+0000 7f01151cfb80 1 --2- 172.21.15.86:0/2582390833 >> [v2:172.21.15.86:6836/2933881328,v1:172.21.15.86:6837/2933881328] conn(0x55ec991d5f90 0x55ec991d83b0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.699+0000 7f01151cfb80 1 --2- 172.21.15.86:0/2582390833 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec99173df0 0x55ec99178d50 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.699+0000 7f01151cfb80 1 --2- 172.21.15.86:0/2582390833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec991685d0 0x55ec9917a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.699+0000 7f00dbe00700 0 client.25772 ms_handle_reset on v2:172.21.15.86:6836/2933881328 2024-03-23T17:51:34.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.699+0000 7f01151cfb80 1 --2- 172.21.15.86:0/2582390833 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ec99168ef0 0x55ec99178810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.699+0000 7f01151cfb80 1 -- 172.21.15.86:0/2582390833 >> 172.21.15.86:0/2582390833 conn(0x55ec990c86a0 msgr2=0x55ec9917f640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:34.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.699+0000 7f01151cfb80 1 -- 172.21.15.86:0/2582390833 shutdown_connections 2024-03-23T17:51:34.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.700+0000 7f01151cfb80 1 -- 172.21.15.86:0/2582390833 wait complete. 2024-03-23T17:51:34.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.700+0000 7f01151cfb80 20 client.25772 trim_cache size 0 max 16384 2024-03-23T17:51:34.704 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-03-23T17:51:34.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.700+0000 7f0101600700 1 -- 172.21.15.86:0/2102182354 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f00e00a1430 msgr2=0x7f00e00a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:34.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.700+0000 7f0101600700 1 --2- 172.21.15.86:0/2102182354 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f00e00a1430 0x7f00e00a38f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f00ec009cb0 tx=0x7f00ec009260 comp rx=0 tx=0).stop 2024-03-23T17:51:34.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.700+0000 7f0101600700 1 -- 172.21.15.86:0/2102182354 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00fc106090 msgr2=0x7f00fc195620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:34.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.700+0000 7f0101600700 1 --2- 172.21.15.86:0/2102182354 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00fc106090 0x7f00fc195620 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f00e8005f50 tx=0x7f00e800f300 comp rx=0 tx=0).stop 2024-03-23T17:51:34.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.700+0000 7f0101600700 1 -- 172.21.15.86:0/2102182354 shutdown_connections 2024-03-23T17:51:34.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.700+0000 7f0101600700 1 --2- 172.21.15.86:0/2102182354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00fc1999c0 0x7f00fc199e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.700+0000 7f0101600700 1 --2- 172.21.15.86:0/2102182354 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f00e00a1430 0x7f00e00a38f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.700+0000 7f0101600700 1 --2- 172.21.15.86:0/2102182354 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f00fc100f70 0x7f00fc1950e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.700+0000 7f0101600700 1 --2- 172.21.15.86:0/2102182354 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00fc106090 0x7f00fc195620 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.700+0000 7f0101600700 1 -- 172.21.15.86:0/2102182354 >> 172.21.15.86:0/2102182354 conn(0x7f00fc0fc7f0 msgr2=0x7f00fc0fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:34.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.700+0000 7f0101600700 1 -- 172.21.15.86:0/2102182354 shutdown_connections 2024-03-23T17:51:34.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.700+0000 7f0101600700 1 -- 172.21.15.86:0/2102182354 wait complete. 2024-03-23T17:51:34.718 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-23T17:51:34.718 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-23T17:51:34.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.961+0000 7f9c83600700 1 -- 172.21.15.86:0/896360197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c7c069bf0 msgr2=0x7f9c7c105980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:34.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.961+0000 7f9c83600700 1 --2- 172.21.15.86:0/896360197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c7c069bf0 0x7f9c7c105980 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f9c680099c0 tx=0x7f9c68009cd0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.961+0000 7f9c83600700 1 -- 172.21.15.86:0/896360197 shutdown_connections 2024-03-23T17:51:34.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.961+0000 7f9c83600700 1 --2- 172.21.15.86:0/896360197 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c7c1060b0 0x7f9c7c10dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.961+0000 7f9c83600700 1 --2- 172.21.15.86:0/896360197 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c7c069240 0x7f9c7c069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.961+0000 7f9c83600700 1 --2- 172.21.15.86:0/896360197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c7c069bf0 0x7f9c7c105980 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.961+0000 7f9c83600700 1 -- 172.21.15.86:0/896360197 >> 172.21.15.86:0/896360197 conn(0x7f9c7c076d80 msgr2=0x7f9c7c077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:34.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.961+0000 7f9c83600700 1 -- 172.21.15.86:0/896360197 shutdown_connections 2024-03-23T17:51:34.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.961+0000 7f9c83600700 1 -- 172.21.15.86:0/896360197 wait complete. 2024-03-23T17:51:34.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.962+0000 7f9c83600700 1 Processor -- start 2024-03-23T17:51:34.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.962+0000 7f9c83600700 1 -- start start 2024-03-23T17:51:34.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.963+0000 7f9c83600700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c7c069240 0x7f9c7c1993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:34.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.963+0000 7f9c83600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c7c1060b0 0x7f9c7c199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:34.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.963+0000 7f9c83600700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c7c19dcb0 0x7f9c7c19e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:34.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.963+0000 7f9c83600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c7c19e730 con 0x7f9c7c1060b0 2024-03-23T17:51:34.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.963+0000 7f9c83600700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c7c19e8a0 con 0x7f9c7c069240 2024-03-23T17:51:34.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.963+0000 7f9c83600700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c7c19ea60 con 0x7f9c7c19dcb0 2024-03-23T17:51:34.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.963+0000 7f9c7be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c7c1060b0 0x7f9c7c199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:34.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.963+0000 7f9c7be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c7c1060b0 0x7f9c7c199910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38162/0 (socket says 172.21.15.86:38162) 2024-03-23T17:51:34.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.963+0000 7f9c7be00700 1 -- 172.21.15.86:0/3964740507 learned_addr learned my addr 172.21.15.86:0/3964740507 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:34.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.963+0000 7f9c81600700 1 --2- 172.21.15.86:0/3964740507 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c7c19dcb0 0x7f9c7c19e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:34.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.963+0000 7f9c80c00700 1 --2- 172.21.15.86:0/3964740507 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c7c069240 0x7f9c7c1993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:34.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.963+0000 7f9c7be00700 1 -- 172.21.15.86:0/3964740507 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c7c19dcb0 msgr2=0x7f9c7c19e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:34.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.963+0000 7f9c7be00700 1 --2- 172.21.15.86:0/3964740507 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c7c19dcb0 0x7f9c7c19e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.963+0000 7f9c7be00700 1 -- 172.21.15.86:0/3964740507 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c7c069240 msgr2=0x7f9c7c1993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:34.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.963+0000 7f9c7be00700 1 --2- 172.21.15.86:0/3964740507 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c7c069240 0x7f9c7c1993d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.963+0000 7f9c7be00700 1 -- 172.21.15.86:0/3964740507 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c680095f0 con 0x7f9c7c1060b0 2024-03-23T17:51:34.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.964+0000 7f9c7be00700 1 --2- 172.21.15.86:0/3964740507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c7c1060b0 0x7f9c7c199910 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9c680098e0 tx=0x7f9c6800b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:34.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.964+0000 7f9c79600700 1 -- 172.21.15.86:0/3964740507 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c6801d070 con 0x7f9c7c1060b0 2024-03-23T17:51:34.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.964+0000 7f9c79600700 1 -- 172.21.15.86:0/3964740507 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c6800bde0 con 0x7f9c7c1060b0 2024-03-23T17:51:34.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.964+0000 7f9c79600700 1 -- 172.21.15.86:0/3964740507 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c6800fa40 con 0x7f9c7c1060b0 2024-03-23T17:51:34.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.964+0000 7f9c83600700 1 -- 172.21.15.86:0/3964740507 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c7c1a5f60 con 0x7f9c7c1060b0 2024-03-23T17:51:34.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.964+0000 7f9c83600700 1 -- 172.21.15.86:0/3964740507 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c7c1a62d0 con 0x7f9c7c1060b0 2024-03-23T17:51:34.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.967+0000 7f9c79600700 1 -- 172.21.15.86:0/3964740507 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) v1 ==== 136098+0+0 (secure 0 0 0) 0x7f9c68004cd0 con 0x7f9c7c1060b0 2024-03-23T17:51:34.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.968+0000 7f9c9719cb80 1 Processor -- start 2024-03-23T17:51:34.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.968+0000 7f9c79600700 1 --2- 172.21.15.86:0/3964740507 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f9c640a13b0 0x7f9c640a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:34.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.968+0000 7f9c9719cb80 1 -- start start 2024-03-23T17:51:34.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.968+0000 7f9c79600700 1 -- 172.21.15.86:0/3964740507 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f9c680cdcd0 con 0x7f9c7c1060b0 2024-03-23T17:51:34.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.969+0000 7f9c81600700 1 --2- 172.21.15.86:0/3964740507 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f9c640a13b0 0x7f9c640a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:34.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.969+0000 7f9c9719cb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557502b095d0 0x557502b099b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:34.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.969+0000 7f9c9719cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557502b09ef0 0x557502b148b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:34.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.969+0000 7f9c9719cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557502b14df0 0x557502b171e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:34.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.969+0000 7f9c9719cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x557502a62990 con 0x557502b09ef0 2024-03-23T17:51:34.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.969+0000 7f9c9719cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x557502a60350 con 0x557502b14df0 2024-03-23T17:51:34.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.969+0000 7f9c9719cb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x557502a5bc40 con 0x557502b095d0 2024-03-23T17:51:34.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.969+0000 7f9c7be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557502b09ef0 0x557502b148b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:34.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.969+0000 7f9c7be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557502b09ef0 0x557502b148b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38168/0 (socket says 172.21.15.86:38168) 2024-03-23T17:51:34.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.969+0000 7f9c7be00700 1 -- 172.21.15.86:0/2957771332 learned_addr learned my addr 172.21.15.86:0/2957771332 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:34.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.969+0000 7f9c80c00700 1 --2- 172.21.15.86:0/2957771332 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557502b095d0 0x557502b099b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:34.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.969+0000 7f9c81600700 1 --2- 172.21.15.86:0/2957771332 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557502b14df0 0x557502b171e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:34.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.969+0000 7f9c81600700 1 --2- 172.21.15.86:0/3964740507 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f9c640a13b0 0x7f9c640a3870 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9c74003ee0 tx=0x7f9c74009100 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:34.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.969+0000 7f9c7be00700 1 -- 172.21.15.86:0/2957771332 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557502b095d0 msgr2=0x557502b099b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:34.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.969+0000 7f9c7be00700 1 --2- 172.21.15.86:0/2957771332 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557502b095d0 0x557502b099b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.969+0000 7f9c7be00700 1 -- 172.21.15.86:0/2957771332 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557502b14df0 msgr2=0x557502b171e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:34.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.969+0000 7f9c7be00700 1 --2- 172.21.15.86:0/2957771332 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557502b14df0 0x557502b171e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.970+0000 7f9c7be00700 1 -- 172.21.15.86:0/2957771332 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5575028dd3b0 con 0x557502b09ef0 2024-03-23T17:51:34.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.970+0000 7f9c81600700 1 --2- 172.21.15.86:0/2957771332 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557502b14df0 0x557502b171e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:51:34.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.970+0000 7f9c7be00700 1 --2- 172.21.15.86:0/2957771332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557502b09ef0 0x557502b148b0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9c6809aba0 tx=0x7f9c6809a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c22cd2d710df62a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:34.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.970+0000 7f9c6dc00700 1 -- 172.21.15.86:0/2957771332 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c6809ac10 con 0x557502b09ef0 2024-03-23T17:51:34.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.970+0000 7f9c6dc00700 1 -- 172.21.15.86:0/2957771332 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c6809ea40 con 0x557502b09ef0 2024-03-23T17:51:34.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.970+0000 7f9c6dc00700 1 -- 172.21.15.86:0/2957771332 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c680a2db0 con 0x557502b09ef0 2024-03-23T17:51:34.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.971+0000 7f9c9719cb80 1 -- 172.21.15.86:0/2957771332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557502b09ef0 msgr2=0x557502b148b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:34.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.971+0000 7f9c9719cb80 1 --2- 172.21.15.86:0/2957771332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557502b09ef0 0x557502b148b0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9c6809aba0 tx=0x7f9c6809a640 comp rx=0 tx=0).stop 2024-03-23T17:51:34.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.971+0000 7f9c9719cb80 1 -- 172.21.15.86:0/2957771332 shutdown_connections 2024-03-23T17:51:34.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.971+0000 7f9c9719cb80 1 --2- 172.21.15.86:0/2957771332 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557502b14df0 0x557502b171e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.971+0000 7f9c9719cb80 1 --2- 172.21.15.86:0/2957771332 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557502b095d0 0x557502b099b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.971+0000 7f9c9719cb80 1 --2- 172.21.15.86:0/2957771332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557502b09ef0 0x557502b148b0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.971+0000 7f9c9719cb80 1 -- 172.21.15.86:0/2957771332 >> 172.21.15.86:0/2957771332 conn(0x557502a696a0 msgr2=0x557502b05a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:34.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.971+0000 7f9c9719cb80 1 -- 172.21.15.86:0/2957771332 shutdown_connections 2024-03-23T17:51:34.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.971+0000 7f9c9719cb80 1 -- 172.21.15.86:0/2957771332 wait complete. 2024-03-23T17:51:34.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.971+0000 7f9c9719cb80 1 -- start start 2024-03-23T17:51:34.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.972+0000 7f9c9719cb80 1 Processor -- start 2024-03-23T17:51:34.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.972+0000 7f9c9719cb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557502b095d0 0x557502b151d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:34.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.972+0000 7f9c9719cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557502b09ef0 0x557502b15710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:34.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.972+0000 7f9c9719cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557502b14df0 0x557502b20250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:34.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.972+0000 7f9c9719cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575029f7d80 con 0x557502b14df0 2024-03-23T17:51:34.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.972+0000 7f9c9719cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575029f7a20 con 0x557502b09ef0 2024-03-23T17:51:34.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.972+0000 7f9c9719cb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575028df6a0 con 0x557502b095d0 2024-03-23T17:51:34.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.972+0000 7f9c9719cb80 10 client.0 _ll_register_callbacks cb 0x557502a01470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:51:34.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.972+0000 7f9c81600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557502b14df0 0x557502b20250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:34.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.972+0000 7f9c81600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557502b14df0 0x557502b20250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38178/0 (socket says 172.21.15.86:38178) 2024-03-23T17:51:34.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.972+0000 7f9c7be00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557502b09ef0 0x557502b15710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:34.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.972+0000 7f9c81600700 1 -- 172.21.15.86:0/3233089890 learned_addr learned my addr 172.21.15.86:0/3233089890 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:34.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.972+0000 7f9c80c00700 1 --2- 172.21.15.86:0/3233089890 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557502b095d0 0x557502b151d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:34.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.973+0000 7f9c80c00700 1 -- 172.21.15.86:0/3233089890 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557502b09ef0 msgr2=0x557502b15710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:34.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.973+0000 7f9c80c00700 1 --2- 172.21.15.86:0/3233089890 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557502b09ef0 0x557502b15710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.973+0000 7f9c80c00700 1 -- 172.21.15.86:0/3233089890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557502b14df0 msgr2=0x557502b20250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:34.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.973+0000 7f9c80c00700 1 --2- 172.21.15.86:0/3233089890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557502b14df0 0x557502b20250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.973+0000 7f9c80c00700 1 -- 172.21.15.86:0/3233089890 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c7c1a62d0 con 0x557502b095d0 2024-03-23T17:51:34.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.973+0000 7f9c7be00700 1 --2- 172.21.15.86:0/3233089890 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557502b09ef0 0x557502b15710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:51:34.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.973+0000 7f9c81600700 1 --2- 172.21.15.86:0/3233089890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557502b14df0 0x557502b20250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:51:34.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.973+0000 7f9c80c00700 1 --2- 172.21.15.86:0/3233089890 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557502b095d0 0x557502b151d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9c700126b0 tx=0x7f9c700129c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=451bf95dcab23b03 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:34.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.973+0000 7f9c62000700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-03-23T17:51:34.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.973+0000 7f9c62000700 1 -- 172.21.15.86:0/3233089890 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c70017070 con 0x557502b095d0 2024-03-23T17:51:34.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.973+0000 7f9c62000700 1 -- 172.21.15.86:0/3233089890 <== mon.1 v2:172.21.15.152:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f9c70004830 con 0x557502b095d0 2024-03-23T17:51:34.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.973+0000 7f9c9719cb80 1 -- 172.21.15.86:0/3233089890 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557502a1c380 con 0x557502b095d0 2024-03-23T17:51:34.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.974+0000 7f9c62000700 1 -- 172.21.15.86:0/3233089890 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c700049f0 con 0x557502b095d0 2024-03-23T17:51:34.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.974+0000 7f9c9719cb80 10 client.25783 fetch_fsmap learned FSMap version 28 2024-03-23T17:51:34.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.974+0000 7f9c9719cb80 1 -- 172.21.15.86:0/3233089890 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x557502889b60 con 0x557502b095d0 2024-03-23T17:51:34.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.974+0000 7f9c62000700 1 -- 172.21.15.86:0/3233089890 <== mon.1 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557502a1c380 con 0x557502b095d0 2024-03-23T17:51:34.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.974+0000 7f9c62000700 1 -- 172.21.15.86:0/3233089890 <== mon.1 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9c7001c630 con 0x557502b095d0 2024-03-23T17:51:34.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.974+0000 7f9c9719cb80 10 client.25783 fetch_fsmap finished waiting for FSMap version 28 2024-03-23T17:51:34.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.974+0000 7f9c9719cb80 10 client.25783 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-23T17:51:34.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.974+0000 7f9c9719cb80 20 client.25783 populate_metadata read hostname 'smithi086' 2024-03-23T17:51:34.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.974+0000 7f9c9719cb80 1 --2- 172.21.15.86:0/3233089890 >> [v2:172.21.15.92:6836/3967837948,v1:172.21.15.92:6837/3967837948] conn(0x557502b24570 0x557502b26990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:34.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.974+0000 7f9c9719cb80 4 client.25783 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:51:34.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.974+0000 7f9c9719cb80 1 -- 172.21.15.86:0/3233089890 --> [v2:172.21.15.92:6836/3967837948,v1:172.21.15.92:6837/3967837948] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557502a54700 con 0x557502b24570 2024-03-23T17:51:34.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.974+0000 7f9c7be00700 1 --2- 172.21.15.86:0/3233089890 >> [v2:172.21.15.92:6836/3967837948,v1:172.21.15.92:6837/3967837948] conn(0x557502b24570 0x557502b26990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:34.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.975+0000 7f9c62000700 10 client.25783 ms_handle_connect on v2:172.21.15.92:6836/3967837948 2024-03-23T17:51:34.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.975+0000 7f9c7be00700 1 --2- 172.21.15.86:0/3233089890 >> [v2:172.21.15.92:6836/3967837948,v1:172.21.15.92:6837/3967837948] conn(0x557502b24570 0x557502b26990 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b4fdebeec12446ee server_cookie=af6a573d9f647166 in_seq=0 out_seq=0 2024-03-23T17:51:34.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.975+0000 7f9c62000700 1 -- 172.21.15.86:0/3233089890 <== mds.2 v2:172.21.15.92:6836/3967837948 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x557502a54700 con 0x557502b24570 2024-03-23T17:51:34.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.975+0000 7f9c62000700 10 client.25783 handle_command_reply: tid=0 2024-03-23T17:51:34.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.976+0000 7f9c9719cb80 1 client.25783 shutdown 2024-03-23T17:51:34.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.976+0000 7f9c9719cb80 1 -- 172.21.15.86:0/3233089890 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557502b095d0 msgr2=0x557502b151d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:34.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.976+0000 7f9c9719cb80 1 --2- 172.21.15.86:0/3233089890 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557502b095d0 0x557502b151d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9c700126b0 tx=0x7f9c700129c0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.976+0000 7f9c9719cb80 1 -- 172.21.15.86:0/3233089890 shutdown_connections 2024-03-23T17:51:34.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.976+0000 7f9c9719cb80 1 --2- 172.21.15.86:0/3233089890 >> [v2:172.21.15.92:6836/3967837948,v1:172.21.15.92:6837/3967837948] conn(0x557502b24570 0x557502b26990 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.976+0000 7f9c9719cb80 1 --2- 172.21.15.86:0/3233089890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557502b14df0 0x557502b20250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.976+0000 7f9c9719cb80 1 --2- 172.21.15.86:0/3233089890 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557502b095d0 0x557502b151d0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.976+0000 7f9c9719cb80 1 --2- 172.21.15.86:0/3233089890 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557502b09ef0 0x557502b15710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.976+0000 7f9c9719cb80 1 -- 172.21.15.86:0/3233089890 >> 172.21.15.86:0/3233089890 conn(0x557502a696a0 msgr2=0x557502b0b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:34.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.976+0000 7f9c62000700 0 client.25783 ms_handle_reset on v2:172.21.15.92:6836/3967837948 2024-03-23T17:51:34.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.976+0000 7f9c9719cb80 1 -- 172.21.15.86:0/3233089890 shutdown_connections 2024-03-23T17:51:34.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.976+0000 7f9c9719cb80 1 -- 172.21.15.86:0/3233089890 wait complete. 2024-03-23T17:51:34.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.976+0000 7f9c9719cb80 20 client.25783 trim_cache size 0 max 16384 2024-03-23T17:51:34.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.989+0000 7f9c9719cb80 1 Processor -- start 2024-03-23T17:51:34.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.989+0000 7f9c9719cb80 1 -- start start 2024-03-23T17:51:34.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.989+0000 7f9c9719cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557502b095d0 0x557502b1a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:34.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.989+0000 7f9c9719cb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557502b09ef0 0x557502b151d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:34.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.989+0000 7f9c9719cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557502b14df0 0x557502b15710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:34.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.989+0000 7f9c9719cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x557502b0a850 con 0x557502b095d0 2024-03-23T17:51:34.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.989+0000 7f9c9719cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x557502b07160 con 0x557502b14df0 2024-03-23T17:51:34.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.989+0000 7f9c9719cb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x557502b2b670 con 0x557502b09ef0 2024-03-23T17:51:34.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.990+0000 7f9c80c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557502b095d0 0x557502b1a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:34.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.990+0000 7f9c80c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557502b095d0 0x557502b1a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38194/0 (socket says 172.21.15.86:38194) 2024-03-23T17:51:34.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.990+0000 7f9c80c00700 1 -- 172.21.15.86:0/1615205286 learned_addr learned my addr 172.21.15.86:0/1615205286 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:34.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.990+0000 7f9c7be00700 1 --2- 172.21.15.86:0/1615205286 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557502b09ef0 0x557502b151d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:34.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.990+0000 7f9c81600700 1 --2- 172.21.15.86:0/1615205286 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557502b14df0 0x557502b15710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:34.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.990+0000 7f9c80c00700 1 -- 172.21.15.86:0/1615205286 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557502b09ef0 msgr2=0x557502b151d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:34.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.990+0000 7f9c80c00700 1 --2- 172.21.15.86:0/1615205286 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557502b09ef0 0x557502b151d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.990+0000 7f9c80c00700 1 -- 172.21.15.86:0/1615205286 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557502b14df0 msgr2=0x557502b15710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:34.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.990+0000 7f9c80c00700 1 --2- 172.21.15.86:0/1615205286 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557502b14df0 0x557502b15710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.990+0000 7f9c80c00700 1 -- 172.21.15.86:0/1615205286 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557502a21ea0 con 0x557502b095d0 2024-03-23T17:51:34.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.990+0000 7f9c80c00700 1 --2- 172.21.15.86:0/1615205286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557502b095d0 0x557502b1a0c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f9c70001eb0 tx=0x7f9c70009480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6994f7953fc2849d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:34.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.990+0000 7f9c6e600700 1 -- 172.21.15.86:0/1615205286 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c70017030 con 0x557502b095d0 2024-03-23T17:51:34.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.991+0000 7f9c6e600700 1 -- 172.21.15.86:0/1615205286 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c70018040 con 0x557502b095d0 2024-03-23T17:51:34.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.991+0000 7f9c6e600700 1 -- 172.21.15.86:0/1615205286 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c700266c0 con 0x557502b095d0 2024-03-23T17:51:34.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.991+0000 7f9c9719cb80 1 -- 172.21.15.86:0/1615205286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557502b095d0 msgr2=0x557502b1a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:34.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.991+0000 7f9c9719cb80 1 --2- 172.21.15.86:0/1615205286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557502b095d0 0x557502b1a0c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f9c70001eb0 tx=0x7f9c70009480 comp rx=0 tx=0).stop 2024-03-23T17:51:34.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.991+0000 7f9c9719cb80 1 -- 172.21.15.86:0/1615205286 shutdown_connections 2024-03-23T17:51:34.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.991+0000 7f9c9719cb80 1 --2- 172.21.15.86:0/1615205286 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557502b14df0 0x557502b15710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.991+0000 7f9c9719cb80 1 --2- 172.21.15.86:0/1615205286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557502b095d0 0x557502b1a0c0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.991+0000 7f9c9719cb80 1 --2- 172.21.15.86:0/1615205286 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557502b09ef0 0x557502b151d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.991+0000 7f9c9719cb80 1 -- 172.21.15.86:0/1615205286 >> 172.21.15.86:0/1615205286 conn(0x557502a696a0 msgr2=0x557502b0f3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:34.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.991+0000 7f9c9719cb80 1 -- 172.21.15.86:0/1615205286 shutdown_connections 2024-03-23T17:51:34.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.991+0000 7f9c9719cb80 1 -- 172.21.15.86:0/1615205286 wait complete. 2024-03-23T17:51:34.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.992+0000 7f9c9719cb80 1 -- start start 2024-03-23T17:51:34.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.992+0000 7f9c9719cb80 1 Processor -- start 2024-03-23T17:51:34.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.992+0000 7f9c9719cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557502b095d0 0x557502b6e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:34.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.992+0000 7f9c9719cb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557502b09ef0 0x557502b6e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:34.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.992+0000 7f9c9719cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557502b14df0 0x557502b69fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:34.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.992+0000 7f9c9719cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x557502b6a6f0 con 0x557502b14df0 2024-03-23T17:51:34.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.992+0000 7f9c9719cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x557502b6a860 con 0x557502b095d0 2024-03-23T17:51:34.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.992+0000 7f9c9719cb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x557502b6aa20 con 0x557502b09ef0 2024-03-23T17:51:34.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.992+0000 7f9c9719cb80 10 client.0 _ll_register_callbacks cb 0x557502a01470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:51:34.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.992+0000 7f9c80c00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557502b095d0 0x557502b6e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:34.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.992+0000 7f9c81600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557502b14df0 0x557502b69fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:34.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.993+0000 7f9c80c00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557502b095d0 0x557502b6e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:48224/0 (socket says 172.21.15.86:48224) 2024-03-23T17:51:34.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.993+0000 7f9c80c00700 1 -- 172.21.15.86:0/4258669617 learned_addr learned my addr 172.21.15.86:0/4258669617 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:34.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.993+0000 7f9c7be00700 1 --2- 172.21.15.86:0/4258669617 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557502b09ef0 0x557502b6e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:34.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.993+0000 7f9c80c00700 1 -- 172.21.15.86:0/4258669617 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557502b09ef0 msgr2=0x557502b6e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:34.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.993+0000 7f9c80c00700 1 --2- 172.21.15.86:0/4258669617 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557502b09ef0 0x557502b6e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.993+0000 7f9c80c00700 1 -- 172.21.15.86:0/4258669617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557502b14df0 msgr2=0x557502b69fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:34.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.993+0000 7f9c80c00700 1 --2- 172.21.15.86:0/4258669617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557502b14df0 0x557502b69fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.993+0000 7f9c80c00700 1 -- 172.21.15.86:0/4258669617 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557502889b60 con 0x557502b095d0 2024-03-23T17:51:34.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.993+0000 7f9c80c00700 1 --2- 172.21.15.86:0/4258669617 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557502b095d0 0x557502b6e3d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9c700095f0 tx=0x7f9c7001cc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5b47ee7431f9aef server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:34.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.993+0000 7f9c62000700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T17:51:34.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.993+0000 7f9c62000700 1 -- 172.21.15.86:0/4258669617 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c70017070 con 0x557502b095d0 2024-03-23T17:51:34.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.993+0000 7f9c9719cb80 1 -- 172.21.15.86:0/4258669617 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9c6809ea40 con 0x557502b095d0 2024-03-23T17:51:34.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.994+0000 7f9c62000700 1 -- 172.21.15.86:0/4258669617 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f9c7002b020 con 0x557502b095d0 2024-03-23T17:51:34.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.994+0000 7f9c62000700 1 -- 172.21.15.86:0/4258669617 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c70003680 con 0x557502b095d0 2024-03-23T17:51:34.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.994+0000 7f9c9719cb80 10 client.25790 fetch_fsmap learned FSMap version 28 2024-03-23T17:51:34.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.994+0000 7f9c9719cb80 1 -- 172.21.15.86:0/4258669617 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5575028e1cd0 con 0x557502b095d0 2024-03-23T17:51:34.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.994+0000 7f9c62000700 1 -- 172.21.15.86:0/4258669617 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c6809ea40 con 0x557502b095d0 2024-03-23T17:51:34.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.994+0000 7f9c62000700 1 -- 172.21.15.86:0/4258669617 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 28) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9c70026480 con 0x557502b095d0 2024-03-23T17:51:34.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.994+0000 7f9c9719cb80 10 client.25790 fetch_fsmap finished waiting for FSMap version 28 2024-03-23T17:51:34.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.994+0000 7f9c9719cb80 10 client.25790 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-23T17:51:34.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.994+0000 7f9c9719cb80 20 client.25790 populate_metadata read hostname 'smithi086' 2024-03-23T17:51:34.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.994+0000 7f9c9719cb80 1 --2- 172.21.15.86:0/4258669617 >> [v2:172.21.15.92:6836/3967837948,v1:172.21.15.92:6837/3967837948] conn(0x557502b76ea0 0x557502b79290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:34.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.994+0000 7f9c9719cb80 4 client.25790 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-23T17:51:34.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.994+0000 7f9c9719cb80 1 -- 172.21.15.86:0/4258669617 --> [v2:172.21.15.92:6836/3967837948,v1:172.21.15.92:6837/3967837948] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5575028aacf0 con 0x557502b76ea0 2024-03-23T17:51:34.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.995+0000 7f9c7be00700 1 --2- 172.21.15.86:0/4258669617 >> [v2:172.21.15.92:6836/3967837948,v1:172.21.15.92:6837/3967837948] conn(0x557502b76ea0 0x557502b79290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:34.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.995+0000 7f9c7be00700 1 --2- 172.21.15.86:0/4258669617 >> [v2:172.21.15.92:6836/3967837948,v1:172.21.15.92:6837/3967837948] conn(0x557502b76ea0 0x557502b79290 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d161d4fc96482dc0 server_cookie=9821f0dbd8129b30 in_seq=0 out_seq=0 2024-03-23T17:51:34.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.995+0000 7f9c62000700 10 client.25790 ms_handle_connect on v2:172.21.15.92:6836/3967837948 2024-03-23T17:51:34.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.995+0000 7f9c62000700 1 -- 172.21.15.86:0/4258669617 <== mds.2 v2:172.21.15.92:6836/3967837948 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5575028aacf0 con 0x557502b76ea0 2024-03-23T17:51:34.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.995+0000 7f9c62000700 10 client.25790 handle_command_reply: tid=0 2024-03-23T17:51:34.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.995+0000 7f9c9719cb80 1 client.25790 shutdown 2024-03-23T17:51:34.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.995+0000 7f9c9719cb80 1 -- 172.21.15.86:0/4258669617 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557502b095d0 msgr2=0x557502b6e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:34.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.995+0000 7f9c9719cb80 1 --2- 172.21.15.86:0/4258669617 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557502b095d0 0x557502b6e3d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9c700095f0 tx=0x7f9c7001cc60 comp rx=0 tx=0).stop 2024-03-23T17:51:34.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.995+0000 7f9c9719cb80 1 -- 172.21.15.86:0/4258669617 shutdown_connections 2024-03-23T17:51:34.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.995+0000 7f9c9719cb80 1 --2- 172.21.15.86:0/4258669617 >> [v2:172.21.15.92:6836/3967837948,v1:172.21.15.92:6837/3967837948] conn(0x557502b76ea0 0x557502b79290 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:34.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.995+0000 7f9c9719cb80 1 --2- 172.21.15.86:0/4258669617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557502b14df0 0x557502b69fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:35.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.995+0000 7f9c9719cb80 1 --2- 172.21.15.86:0/4258669617 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557502b095d0 0x557502b6e3d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:35.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.995+0000 7f9c9719cb80 1 --2- 172.21.15.86:0/4258669617 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557502b09ef0 0x557502b6e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:35.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.995+0000 7f9c9719cb80 1 -- 172.21.15.86:0/4258669617 >> 172.21.15.86:0/4258669617 conn(0x557502a696a0 msgr2=0x557502b151d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:35.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.995+0000 7f9c62000700 0 client.25790 ms_handle_reset on v2:172.21.15.92:6836/3967837948 2024-03-23T17:51:35.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.996+0000 7f9c9719cb80 1 -- 172.21.15.86:0/4258669617 shutdown_connections 2024-03-23T17:51:35.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.996+0000 7f9c9719cb80 1 -- 172.21.15.86:0/4258669617 wait complete. 2024-03-23T17:51:35.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.996+0000 7f9c9719cb80 20 client.25790 trim_cache size 0 max 16384 2024-03-23T17:51:35.000 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-03-23T17:51:35.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.996+0000 7f9c83600700 1 -- 172.21.15.86:0/3964740507 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f9c640a13b0 msgr2=0x7f9c640a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:35.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.996+0000 7f9c83600700 1 --2- 172.21.15.86:0/3964740507 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f9c640a13b0 0x7f9c640a3870 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9c74003ee0 tx=0x7f9c74009100 comp rx=0 tx=0).stop 2024-03-23T17:51:35.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.996+0000 7f9c83600700 1 -- 172.21.15.86:0/3964740507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c7c1060b0 msgr2=0x7f9c7c199910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:35.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.996+0000 7f9c83600700 1 --2- 172.21.15.86:0/3964740507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c7c1060b0 0x7f9c7c199910 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9c680098e0 tx=0x7f9c6800b9a0 comp rx=0 tx=0).stop 2024-03-23T17:51:35.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.996+0000 7f9c83600700 1 -- 172.21.15.86:0/3964740507 shutdown_connections 2024-03-23T17:51:35.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.996+0000 7f9c83600700 1 --2- 172.21.15.86:0/3964740507 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f9c640a13b0 0x7f9c640a3870 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:35.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.996+0000 7f9c83600700 1 --2- 172.21.15.86:0/3964740507 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c7c19dcb0 0x7f9c7c19e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:35.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.996+0000 7f9c83600700 1 --2- 172.21.15.86:0/3964740507 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c7c069240 0x7f9c7c1993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:35.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.996+0000 7f9c83600700 1 --2- 172.21.15.86:0/3964740507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c7c1060b0 0x7f9c7c199910 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:35.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.996+0000 7f9c83600700 1 -- 172.21.15.86:0/3964740507 >> 172.21.15.86:0/3964740507 conn(0x7f9c7c076d80 msgr2=0x7f9c7c10c120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:35.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.996+0000 7f9c83600700 1 -- 172.21.15.86:0/3964740507 shutdown_connections 2024-03-23T17:51:35.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:34.997+0000 7f9c83600700 1 -- 172.21.15.86:0/3964740507 wait complete. 2024-03-23T17:51:35.011 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:51:34 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/3692186522' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T17:51:35.014 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-23T17:51:35.014 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-23T17:51:35.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:51:34 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/3692186522' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T17:51:35.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:51:34 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/3692186522' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T17:51:35.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.213+0000 7f0a81a00700 1 -- 172.21.15.86:0/1050738573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a7c100f70 msgr2=0x7f0a7c101350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:35.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.213+0000 7f0a81a00700 1 --2- 172.21.15.86:0/1050738573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a7c100f70 0x7f0a7c101350 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c009990 tx=0x7f0a6c009ca0 comp rx=0 tx=0).stop 2024-03-23T17:51:35.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.214+0000 7f0a81a00700 1 -- 172.21.15.86:0/1050738573 shutdown_connections 2024-03-23T17:51:35.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.214+0000 7f0a81a00700 1 --2- 172.21.15.86:0/1050738573 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0a7c106090 0x7f0a7c10dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:35.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.214+0000 7f0a81a00700 1 --2- 172.21.15.86:0/1050738573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a7c100f70 0x7f0a7c101350 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:35.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.214+0000 7f0a81a00700 1 --2- 172.21.15.86:0/1050738573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a7c101920 0x7f0a7c105960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:35.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.214+0000 7f0a81a00700 1 -- 172.21.15.86:0/1050738573 >> 172.21.15.86:0/1050738573 conn(0x7f0a7c0fc7f0 msgr2=0x7f0a7c0fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:35.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.214+0000 7f0a81a00700 1 -- 172.21.15.86:0/1050738573 shutdown_connections 2024-03-23T17:51:35.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.214+0000 7f0a81a00700 1 -- 172.21.15.86:0/1050738573 wait complete. 2024-03-23T17:51:35.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.215+0000 7f0a81a00700 1 Processor -- start 2024-03-23T17:51:35.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.215+0000 7f0a81a00700 1 -- start start 2024-03-23T17:51:35.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.215+0000 7f0a81a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a7c100f70 0x7f0a7c0730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:35.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.215+0000 7f0a81a00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0a7c106090 0x7f0a7c0735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:35.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.215+0000 7f0a81a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a7c073b30 0x7f0a7c1a53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:35.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.215+0000 7f0a81a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7c1a5980 con 0x7f0a7c100f70 2024-03-23T17:51:35.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.215+0000 7f0a81a00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7c1a5af0 con 0x7f0a7c073b30 2024-03-23T17:51:35.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.215+0000 7f0a81a00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7c1a5cb0 con 0x7f0a7c106090 2024-03-23T17:51:35.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.215+0000 7f0a7aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a7c100f70 0x7f0a7c0730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:35.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.215+0000 7f0a7aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a7c100f70 0x7f0a7c0730b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38222/0 (socket says 172.21.15.86:38222) 2024-03-23T17:51:35.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.215+0000 7f0a7aa00700 1 -- 172.21.15.86:0/3185155909 learned_addr learned my addr 172.21.15.86:0/3185155909 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:35.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.216+0000 7f0a7a000700 1 --2- 172.21.15.86:0/3185155909 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0a7c106090 0x7f0a7c0735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:35.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.216+0000 7f0a7b400700 1 --2- 172.21.15.86:0/3185155909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a7c073b30 0x7f0a7c1a53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:35.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.216+0000 7f0a7aa00700 1 -- 172.21.15.86:0/3185155909 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0a7c106090 msgr2=0x7f0a7c0735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:35.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.216+0000 7f0a7aa00700 1 --2- 172.21.15.86:0/3185155909 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0a7c106090 0x7f0a7c0735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:35.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.216+0000 7f0a7aa00700 1 -- 172.21.15.86:0/3185155909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a7c073b30 msgr2=0x7f0a7c1a53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:35.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.216+0000 7f0a7aa00700 1 --2- 172.21.15.86:0/3185155909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a7c073b30 0x7f0a7c1a53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:35.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.216+0000 7f0a7aa00700 1 -- 172.21.15.86:0/3185155909 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a6c0095f0 con 0x7f0a7c100f70 2024-03-23T17:51:35.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.216+0000 7f0a7b400700 1 --2- 172.21.15.86:0/3185155909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a7c073b30 0x7f0a7c1a53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:51:35.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.216+0000 7f0a7aa00700 1 --2- 172.21.15.86:0/3185155909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a7c100f70 0x7f0a7c0730b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c005910 tx=0x7f0a6c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:35.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.216+0000 7f0a73400700 1 -- 172.21.15.86:0/3185155909 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c01d070 con 0x7f0a7c100f70 2024-03-23T17:51:35.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.216+0000 7f0a73400700 1 -- 172.21.15.86:0/3185155909 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a6c022470 con 0x7f0a7c100f70 2024-03-23T17:51:35.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.216+0000 7f0a81a00700 1 -- 172.21.15.86:0/3185155909 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a7c1a5f30 con 0x7f0a7c100f70 2024-03-23T17:51:35.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.216+0000 7f0a73400700 1 -- 172.21.15.86:0/3185155909 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c017650 con 0x7f0a7c100f70 2024-03-23T17:51:35.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.217+0000 7f0a81a00700 1 -- 172.21.15.86:0/3185155909 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a7c1a6350 con 0x7f0a7c100f70 2024-03-23T17:51:35.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.219+0000 7f0a73400700 1 -- 172.21.15.86:0/3185155909 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) v1 ==== 136098+0+0 (secure 0 0 0) 0x7f0a6c0225e0 con 0x7f0a7c100f70 2024-03-23T17:51:35.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.221+0000 7f0a73400700 1 --2- 172.21.15.86:0/3185155909 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f0a600a13b0 0x7f0a600a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:35.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.221+0000 7f0a955d0b80 1 Processor -- start 2024-03-23T17:51:35.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.221+0000 7f0a73400700 1 -- 172.21.15.86:0/3185155909 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f0a6c0ce2f0 con 0x7f0a7c100f70 2024-03-23T17:51:35.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.221+0000 7f0a7a000700 1 --2- 172.21.15.86:0/3185155909 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f0a600a13b0 0x7f0a600a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:35.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.221+0000 7f0a955d0b80 1 -- start start 2024-03-23T17:51:35.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.221+0000 7f0a955d0b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b65532c5d0 0x55b65532c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:35.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.221+0000 7f0a955d0b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b65532cef0 0x55b6553378b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:35.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.221+0000 7f0a955d0b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b655337df0 0x55b65533a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:35.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.221+0000 7f0a955d0b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b655285990 con 0x55b65532cef0 2024-03-23T17:51:35.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.221+0000 7f0a955d0b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b655283350 con 0x55b65532c5d0 2024-03-23T17:51:35.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.221+0000 7f0a955d0b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b65527ec40 con 0x55b655337df0 2024-03-23T17:51:35.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.222+0000 7f0a7a000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b65532cef0 0x55b6553378b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:35.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.222+0000 7f0a7aa00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b65532c5d0 0x55b65532c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:35.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.222+0000 7f0a7a000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b65532cef0 0x55b6553378b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38228/0 (socket says 172.21.15.86:38228) 2024-03-23T17:51:35.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.222+0000 7f0a7a000700 1 -- 172.21.15.86:0/745674504 learned_addr learned my addr 172.21.15.86:0/745674504 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:35.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.222+0000 7f0a7b400700 1 --2- 172.21.15.86:0/745674504 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b655337df0 0x55b65533a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:35.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.222+0000 7f0a7a000700 1 --2- 172.21.15.86:0/3185155909 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f0a600a13b0 0x7f0a600a3870 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c0747e0 tx=0x7f0a68011720 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:35.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.222+0000 7f0a7aa00700 1 -- 172.21.15.86:0/745674504 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b655337df0 msgr2=0x55b65533a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:35.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.222+0000 7f0a7aa00700 1 --2- 172.21.15.86:0/745674504 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b655337df0 0x55b65533a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:35.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.222+0000 7f0a7aa00700 1 -- 172.21.15.86:0/745674504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b65532cef0 msgr2=0x55b6553378b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:35.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.222+0000 7f0a7aa00700 1 --2- 172.21.15.86:0/745674504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b65532cef0 0x55b6553378b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:35.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.222+0000 7f0a7aa00700 1 -- 172.21.15.86:0/745674504 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b6551003b0 con 0x55b65532c5d0 2024-03-23T17:51:35.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.222+0000 7f0a7b400700 1 --2- 172.21.15.86:0/745674504 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b655337df0 0x55b65533a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:51:35.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.222+0000 7f0a7aa00700 1 --2- 172.21.15.86:0/745674504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b65532c5d0 0x55b65532c9b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c0cb8b0 tx=0x7f0a6c009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43503aa70335b4eb server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:35.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.223+0000 7f0a67e00700 1 -- 172.21.15.86:0/745674504 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c09acb0 con 0x55b65532c5d0 2024-03-23T17:51:35.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.223+0000 7f0a67e00700 1 -- 172.21.15.86:0/745674504 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a6c09ba50 con 0x55b65532c5d0 2024-03-23T17:51:35.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.223+0000 7f0a67e00700 1 -- 172.21.15.86:0/745674504 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c09aaa0 con 0x55b65532c5d0 2024-03-23T17:51:35.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.223+0000 7f0a955d0b80 1 -- 172.21.15.86:0/745674504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b65532c5d0 msgr2=0x55b65532c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:35.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.223+0000 7f0a955d0b80 1 --2- 172.21.15.86:0/745674504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b65532c5d0 0x55b65532c9b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c0cb8b0 tx=0x7f0a6c009310 comp rx=0 tx=0).stop 2024-03-23T17:51:35.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.223+0000 7f0a955d0b80 1 -- 172.21.15.86:0/745674504 shutdown_connections 2024-03-23T17:51:35.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.223+0000 7f0a955d0b80 1 --2- 172.21.15.86:0/745674504 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b655337df0 0x55b65533a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:35.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.223+0000 7f0a955d0b80 1 --2- 172.21.15.86:0/745674504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b65532c5d0 0x55b65532c9b0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:35.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.223+0000 7f0a955d0b80 1 --2- 172.21.15.86:0/745674504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b65532cef0 0x55b6553378b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:35.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.223+0000 7f0a955d0b80 1 -- 172.21.15.86:0/745674504 >> 172.21.15.86:0/745674504 conn(0x55b65528c6a0 msgr2=0x55b655328a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:35.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.223+0000 7f0a955d0b80 1 -- 172.21.15.86:0/745674504 shutdown_connections 2024-03-23T17:51:35.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.224+0000 7f0a955d0b80 1 -- 172.21.15.86:0/745674504 wait complete. 2024-03-23T17:51:35.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.224+0000 7f0a955d0b80 1 -- start start 2024-03-23T17:51:35.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.224+0000 7f0a955d0b80 1 Processor -- start 2024-03-23T17:51:35.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.225+0000 7f0a955d0b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b65532c5d0 0x55b6553376f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:35.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.225+0000 7f0a955d0b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b65532cef0 0x55b655334a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:35.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.225+0000 7f0a955d0b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b655337df0 0x55b655334fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:35.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.225+0000 7f0a955d0b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b65521ad80 con 0x55b65532cef0 2024-03-23T17:51:35.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.225+0000 7f0a955d0b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b65521aa20 con 0x55b65532c5d0 2024-03-23T17:51:35.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.225+0000 7f0a955d0b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6551026a0 con 0x55b655337df0 2024-03-23T17:51:35.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.225+0000 7f0a955d0b80 10 client.0 _ll_register_callbacks cb 0x55b655224470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:51:35.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.225+0000 7f0a7aa00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b65532c5d0 0x55b6553376f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:35.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.225+0000 7f0a7a000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b65532cef0 0x55b655334a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:35.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.225+0000 7f0a7aa00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b65532c5d0 0x55b6553376f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:48252/0 (socket says 172.21.15.86:48252) 2024-03-23T17:51:35.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.225+0000 7f0a7a000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b65532cef0 0x55b655334a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38244/0 (socket says 172.21.15.86:38244) 2024-03-23T17:51:35.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.225+0000 7f0a7aa00700 1 -- 172.21.15.86:0/1247098789 learned_addr learned my addr 172.21.15.86:0/1247098789 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:35.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.225+0000 7f0a7b400700 1 --2- 172.21.15.86:0/1247098789 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b655337df0 0x55b655334fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:35.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.225+0000 7f0a7aa00700 1 -- 172.21.15.86:0/1247098789 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b655337df0 msgr2=0x55b655334fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:35.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.225+0000 7f0a7aa00700 1 --2- 172.21.15.86:0/1247098789 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b655337df0 0x55b655334fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:35.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.225+0000 7f0a7aa00700 1 -- 172.21.15.86:0/1247098789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b65532cef0 msgr2=0x55b655334a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:35.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.225+0000 7f0a7aa00700 1 --2- 172.21.15.86:0/1247098789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b65532cef0 0x55b655334a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:35.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.225+0000 7f0a7aa00700 1 -- 172.21.15.86:0/1247098789 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a7c1a6350 con 0x55b65532c5d0 2024-03-23T17:51:35.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.226+0000 7f0a7aa00700 1 --2- 172.21.15.86:0/1247098789 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b65532c5d0 0x55b6553376f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c00f740 tx=0x7f0a6c09afc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1354e5ad0812407d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:35.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.226+0000 7f0a64c00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T17:51:35.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.226+0000 7f0a64c00700 1 -- 172.21.15.86:0/1247098789 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c09b350 con 0x55b65532c5d0 2024-03-23T17:51:35.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.226+0000 7f0a64c00700 1 -- 172.21.15.86:0/1247098789 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f0a6c0b0030 con 0x55b65532c5d0 2024-03-23T17:51:35.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.226+0000 7f0a955d0b80 1 -- 172.21.15.86:0/1247098789 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b65523f380 con 0x55b65532c5d0 2024-03-23T17:51:35.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.226+0000 7f0a64c00700 1 -- 172.21.15.86:0/1247098789 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a6c09f180 con 0x55b65532c5d0 2024-03-23T17:51:35.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.226+0000 7f0a955d0b80 10 client.25802 fetch_fsmap learned FSMap version 28 2024-03-23T17:51:35.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.226+0000 7f0a955d0b80 1 -- 172.21.15.86:0/1247098789 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55b6550acb60 con 0x55b65532c5d0 2024-03-23T17:51:35.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.226+0000 7f0a64c00700 1 -- 172.21.15.86:0/1247098789 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b65523f380 con 0x55b65532c5d0 2024-03-23T17:51:35.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.226+0000 7f0a64c00700 1 -- 172.21.15.86:0/1247098789 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 28) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0a6c097140 con 0x55b65532c5d0 2024-03-23T17:51:35.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.226+0000 7f0a955d0b80 10 client.25802 fetch_fsmap finished waiting for FSMap version 28 2024-03-23T17:51:35.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.226+0000 7f0a955d0b80 10 client.25802 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-23T17:51:35.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.226+0000 7f0a955d0b80 20 client.25802 populate_metadata read hostname 'smithi086' 2024-03-23T17:51:35.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.226+0000 7f0a955d0b80 1 --2- 172.21.15.86:0/1247098789 >> [v2:172.21.15.152:6838/1100723992,v1:172.21.15.152:6839/1100723992] conn(0x55b655347280 0x55b6553496a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:35.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.226+0000 7f0a955d0b80 4 client.25802 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:51:35.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.226+0000 7f0a955d0b80 1 -- 172.21.15.86:0/1247098789 --> [v2:172.21.15.152:6838/1100723992,v1:172.21.15.152:6839/1100723992] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b655277700 con 0x55b655347280 2024-03-23T17:51:35.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.227+0000 7f0a7a000700 1 --2- 172.21.15.86:0/1247098789 >> [v2:172.21.15.152:6838/1100723992,v1:172.21.15.152:6839/1100723992] conn(0x55b655347280 0x55b6553496a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:35.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.227+0000 7f0a64c00700 10 client.25802 ms_handle_connect on v2:172.21.15.152:6838/1100723992 2024-03-23T17:51:35.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.227+0000 7f0a7a000700 1 --2- 172.21.15.86:0/1247098789 >> [v2:172.21.15.152:6838/1100723992,v1:172.21.15.152:6839/1100723992] conn(0x55b655347280 0x55b6553496a0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ece3d1680e068530 server_cookie=7a3922949137c20a in_seq=0 out_seq=0 2024-03-23T17:51:35.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.228+0000 7f0a64c00700 1 -- 172.21.15.86:0/1247098789 <== mds.1 v2:172.21.15.152:6838/1100723992 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55b655277700 con 0x55b655347280 2024-03-23T17:51:35.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.228+0000 7f0a64c00700 10 client.25802 handle_command_reply: tid=0 2024-03-23T17:51:35.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.228+0000 7f0a955d0b80 1 client.25802 shutdown 2024-03-23T17:51:35.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.228+0000 7f0a955d0b80 1 -- 172.21.15.86:0/1247098789 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b65532c5d0 msgr2=0x55b6553376f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:35.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.228+0000 7f0a955d0b80 1 --2- 172.21.15.86:0/1247098789 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b65532c5d0 0x55b6553376f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c00f740 tx=0x7f0a6c09afc0 comp rx=0 tx=0).stop 2024-03-23T17:51:35.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.228+0000 7f0a955d0b80 1 -- 172.21.15.86:0/1247098789 shutdown_connections 2024-03-23T17:51:35.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.228+0000 7f0a955d0b80 1 --2- 172.21.15.86:0/1247098789 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b655337df0 0x55b655334fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:35.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.228+0000 7f0a955d0b80 1 --2- 172.21.15.86:0/1247098789 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b65532c5d0 0x55b6553376f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:35.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.228+0000 7f0a955d0b80 1 --2- 172.21.15.86:0/1247098789 >> [v2:172.21.15.152:6838/1100723992,v1:172.21.15.152:6839/1100723992] conn(0x55b655347280 0x55b6553496a0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:35.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.228+0000 7f0a955d0b80 1 --2- 172.21.15.86:0/1247098789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b65532cef0 0x55b655334a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:35.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.228+0000 7f0a955d0b80 1 -- 172.21.15.86:0/1247098789 >> 172.21.15.86:0/1247098789 conn(0x55b65528c6a0 msgr2=0x55b65532e6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:35.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.228+0000 7f0a64c00700 0 client.25802 ms_handle_reset on v2:172.21.15.152:6838/1100723992 2024-03-23T17:51:35.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.228+0000 7f0a955d0b80 1 -- 172.21.15.86:0/1247098789 shutdown_connections 2024-03-23T17:51:35.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.228+0000 7f0a955d0b80 1 -- 172.21.15.86:0/1247098789 wait complete. 2024-03-23T17:51:35.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.229+0000 7f0a955d0b80 20 client.25802 trim_cache size 0 max 16384 2024-03-23T17:51:35.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.242+0000 7f0a955d0b80 1 Processor -- start 2024-03-23T17:51:35.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.242+0000 7f0a955d0b80 1 -- start start 2024-03-23T17:51:35.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.242+0000 7f0a955d0b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b65532c5d0 0x55b655324480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:35.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.242+0000 7f0a955d0b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b65532cef0 0x55b655349670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:35.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.242+0000 7f0a955d0b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b655337df0 0x55b65534e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:35.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.242+0000 7f0a955d0b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b65532a3c0 con 0x55b65532c5d0 2024-03-23T17:51:35.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.242+0000 7f0a955d0b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b655343dd0 con 0x55b65532cef0 2024-03-23T17:51:35.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.242+0000 7f0a955d0b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b655332f40 con 0x55b655337df0 2024-03-23T17:51:35.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.242+0000 7f0a7aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b65532c5d0 0x55b655324480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:35.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.242+0000 7f0a7a000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b65532cef0 0x55b655349670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:35.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.242+0000 7f0a7aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b65532c5d0 0x55b655324480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38258/0 (socket says 172.21.15.86:38258) 2024-03-23T17:51:35.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.242+0000 7f0a7a000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b65532cef0 0x55b655349670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:48260/0 (socket says 172.21.15.86:48260) 2024-03-23T17:51:35.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.242+0000 7f0a7aa00700 1 -- 172.21.15.86:0/2157286468 learned_addr learned my addr 172.21.15.86:0/2157286468 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:35.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.242+0000 7f0a7b400700 1 --2- 172.21.15.86:0/2157286468 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b655337df0 0x55b65534e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:35.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.243+0000 7f0a7a000700 1 -- 172.21.15.86:0/2157286468 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b655337df0 msgr2=0x55b65534e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:35.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.243+0000 7f0a7a000700 1 --2- 172.21.15.86:0/2157286468 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b655337df0 0x55b65534e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:35.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.243+0000 7f0a7a000700 1 -- 172.21.15.86:0/2157286468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b65532c5d0 msgr2=0x55b655324480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:35.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.243+0000 7f0a7a000700 1 --2- 172.21.15.86:0/2157286468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b65532c5d0 0x55b655324480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:35.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.243+0000 7f0a7a000700 1 -- 172.21.15.86:0/2157286468 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b655244ea0 con 0x55b65532cef0 2024-03-23T17:51:35.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.243+0000 7f0a7aa00700 1 --2- 172.21.15.86:0/2157286468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b65532c5d0 0x55b655324480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:51:35.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.243+0000 7f0a7a000700 1 --2- 172.21.15.86:0/2157286468 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b65532cef0 0x55b655349670 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x55b655335f20 tx=0x7f0a6801c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68c261ed404e4935 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:35.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.243+0000 7f0a70c00700 1 -- 172.21.15.86:0/2157286468 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a6802d070 con 0x55b65532cef0 2024-03-23T17:51:35.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.243+0000 7f0a70c00700 1 -- 172.21.15.86:0/2157286468 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a68028340 con 0x55b65532cef0 2024-03-23T17:51:35.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.243+0000 7f0a70c00700 1 -- 172.21.15.86:0/2157286468 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a68023040 con 0x55b65532cef0 2024-03-23T17:51:35.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.244+0000 7f0a955d0b80 1 -- 172.21.15.86:0/2157286468 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b65532cef0 msgr2=0x55b655349670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:35.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.244+0000 7f0a955d0b80 1 --2- 172.21.15.86:0/2157286468 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b65532cef0 0x55b655349670 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x55b655335f20 tx=0x7f0a6801c000 comp rx=0 tx=0).stop 2024-03-23T17:51:35.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.244+0000 7f0a955d0b80 1 -- 172.21.15.86:0/2157286468 shutdown_connections 2024-03-23T17:51:35.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.244+0000 7f0a955d0b80 1 --2- 172.21.15.86:0/2157286468 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b655337df0 0x55b65534e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:35.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.244+0000 7f0a955d0b80 1 --2- 172.21.15.86:0/2157286468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b65532c5d0 0x55b655324480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:35.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.244+0000 7f0a955d0b80 1 --2- 172.21.15.86:0/2157286468 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b65532cef0 0x55b655349670 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:35.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.244+0000 7f0a955d0b80 1 -- 172.21.15.86:0/2157286468 >> 172.21.15.86:0/2157286468 conn(0x55b65528c6a0 msgr2=0x55b6553215a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:35.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.244+0000 7f0a955d0b80 1 -- 172.21.15.86:0/2157286468 shutdown_connections 2024-03-23T17:51:35.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.244+0000 7f0a955d0b80 1 -- 172.21.15.86:0/2157286468 wait complete. 2024-03-23T17:51:35.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.244+0000 7f0a955d0b80 1 -- start start 2024-03-23T17:51:35.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.244+0000 7f0a955d0b80 1 Processor -- start 2024-03-23T17:51:35.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.245+0000 7f0a955d0b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b65532c5d0 0x55b65533e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:35.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.245+0000 7f0a955d0b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b65532cef0 0x55b65533c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:35.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.245+0000 7f0a955d0b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b655337df0 0x55b65533cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:35.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.245+0000 7f0a955d0b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b65533d3e0 con 0x55b65532c5d0 2024-03-23T17:51:35.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.245+0000 7f0a955d0b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b65533d520 con 0x55b65532cef0 2024-03-23T17:51:35.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.245+0000 7f0a955d0b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b655398db0 con 0x55b655337df0 2024-03-23T17:51:35.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.245+0000 7f0a955d0b80 10 client.0 _ll_register_callbacks cb 0x55b655224470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:51:35.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.245+0000 7f0a7aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b65532c5d0 0x55b65533e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:35.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.245+0000 7f0a7b400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b655337df0 0x55b65533cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:35.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.245+0000 7f0a7a000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b65532cef0 0x55b65533c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:35.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.245+0000 7f0a7b400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b655337df0 0x55b65533cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:56900/0 (socket says 172.21.15.86:56900) 2024-03-23T17:51:35.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.245+0000 7f0a7aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b65532c5d0 0x55b65533e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38270/0 (socket says 172.21.15.86:38270) 2024-03-23T17:51:35.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.245+0000 7f0a7b400700 1 -- 172.21.15.86:0/973474261 learned_addr learned my addr 172.21.15.86:0/973474261 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:35.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.245+0000 7f0a7a000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b65532cef0 0x55b65533c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:48262/0 (socket says 172.21.15.86:48262) 2024-03-23T17:51:35.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.245+0000 7f0a7a000700 1 -- 172.21.15.86:0/973474261 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b655337df0 msgr2=0x55b65533cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:35.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.246+0000 7f0a7a000700 1 --2- 172.21.15.86:0/973474261 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b655337df0 0x55b65533cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:35.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.246+0000 7f0a7a000700 1 -- 172.21.15.86:0/973474261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b65532c5d0 msgr2=0x55b65533e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:35.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.246+0000 7f0a7a000700 1 --2- 172.21.15.86:0/973474261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b65532c5d0 0x55b65533e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:35.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.246+0000 7f0a7a000700 1 -- 172.21.15.86:0/973474261 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b6550acb60 con 0x55b65532cef0 2024-03-23T17:51:35.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.246+0000 7f0a7a000700 1 --2- 172.21.15.86:0/973474261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b65532cef0 0x55b65533c7c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x55b65532e260 tx=0x7f0a68028530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1daa24de240ce3e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:35.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.246+0000 7f0a64c00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T17:51:35.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.246+0000 7f0a64c00700 1 -- 172.21.15.86:0/973474261 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a68023050 con 0x55b65532cef0 2024-03-23T17:51:35.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.246+0000 7f0a64c00700 1 -- 172.21.15.86:0/973474261 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f0a6800fe20 con 0x55b65532cef0 2024-03-23T17:51:35.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.246+0000 7f0a955d0b80 1 -- 172.21.15.86:0/973474261 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0a6c09ba50 con 0x55b65532cef0 2024-03-23T17:51:35.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.246+0000 7f0a64c00700 1 -- 172.21.15.86:0/973474261 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a6802d040 con 0x55b65532cef0 2024-03-23T17:51:35.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.246+0000 7f0a955d0b80 10 client.25808 fetch_fsmap learned FSMap version 28 2024-03-23T17:51:35.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.246+0000 7f0a955d0b80 1 -- 172.21.15.86:0/973474261 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f0a6800f840 con 0x55b65532cef0 2024-03-23T17:51:35.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.247+0000 7f0a64c00700 1 -- 172.21.15.86:0/973474261 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a6c09ba50 con 0x55b65532cef0 2024-03-23T17:51:35.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.247+0000 7f0a64c00700 1 -- 172.21.15.86:0/973474261 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 28) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0a68026070 con 0x55b65532cef0 2024-03-23T17:51:35.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.247+0000 7f0a955d0b80 10 client.25808 fetch_fsmap finished waiting for FSMap version 28 2024-03-23T17:51:35.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.247+0000 7f0a955d0b80 10 client.25808 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-23T17:51:35.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.247+0000 7f0a955d0b80 20 client.25808 populate_metadata read hostname 'smithi086' 2024-03-23T17:51:35.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.247+0000 7f0a955d0b80 1 --2- 172.21.15.86:0/973474261 >> [v2:172.21.15.152:6838/1100723992,v1:172.21.15.152:6839/1100723992] conn(0x55b65539a3a0 0x55b65539c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:35.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.247+0000 7f0a955d0b80 4 client.25808 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-23T17:51:35.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.247+0000 7f0a955d0b80 1 -- 172.21.15.86:0/973474261 --> [v2:172.21.15.152:6838/1100723992,v1:172.21.15.152:6839/1100723992] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b6550cdcf0 con 0x55b65539a3a0 2024-03-23T17:51:35.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.247+0000 7f0a7aa00700 1 --2- 172.21.15.86:0/973474261 >> [v2:172.21.15.152:6838/1100723992,v1:172.21.15.152:6839/1100723992] conn(0x55b65539a3a0 0x55b65539c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:35.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.248+0000 7f0a64c00700 10 client.25808 ms_handle_connect on v2:172.21.15.152:6838/1100723992 2024-03-23T17:51:35.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.248+0000 7f0a7aa00700 1 --2- 172.21.15.86:0/973474261 >> [v2:172.21.15.152:6838/1100723992,v1:172.21.15.152:6839/1100723992] conn(0x55b65539a3a0 0x55b65539c7c0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=61abaa8e93e7e51f server_cookie=f3f702a4ebc4ff14 in_seq=0 out_seq=0 2024-03-23T17:51:35.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.248+0000 7f0a64c00700 1 -- 172.21.15.86:0/973474261 <== mds.1 v2:172.21.15.152:6838/1100723992 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b6550cdcf0 con 0x55b65539a3a0 2024-03-23T17:51:35.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.248+0000 7f0a64c00700 10 client.25808 handle_command_reply: tid=0 2024-03-23T17:51:35.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.248+0000 7f0a955d0b80 1 client.25808 shutdown 2024-03-23T17:51:35.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.248+0000 7f0a955d0b80 1 -- 172.21.15.86:0/973474261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b65532cef0 msgr2=0x55b65533c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:35.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.248+0000 7f0a955d0b80 1 --2- 172.21.15.86:0/973474261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b65532cef0 0x55b65533c7c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x55b65532e260 tx=0x7f0a68028530 comp rx=0 tx=0).stop 2024-03-23T17:51:35.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.248+0000 7f0a955d0b80 1 -- 172.21.15.86:0/973474261 shutdown_connections 2024-03-23T17:51:35.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.248+0000 7f0a955d0b80 1 --2- 172.21.15.86:0/973474261 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b655337df0 0x55b65533cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:35.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.248+0000 7f0a955d0b80 1 --2- 172.21.15.86:0/973474261 >> [v2:172.21.15.152:6838/1100723992,v1:172.21.15.152:6839/1100723992] conn(0x55b65539a3a0 0x55b65539c7c0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:35.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.248+0000 7f0a955d0b80 1 --2- 172.21.15.86:0/973474261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b65532c5d0 0x55b65533e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:35.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.248+0000 7f0a955d0b80 1 --2- 172.21.15.86:0/973474261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b65532cef0 0x55b65533c7c0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:35.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.248+0000 7f0a64c00700 0 client.25808 ms_handle_reset on v2:172.21.15.152:6838/1100723992 2024-03-23T17:51:35.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.248+0000 7f0a955d0b80 1 -- 172.21.15.86:0/973474261 >> 172.21.15.86:0/973474261 conn(0x55b65528c6a0 msgr2=0x55b6553332c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:35.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.248+0000 7f0a955d0b80 1 -- 172.21.15.86:0/973474261 shutdown_connections 2024-03-23T17:51:35.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.248+0000 7f0a955d0b80 1 -- 172.21.15.86:0/973474261 wait complete. 2024-03-23T17:51:35.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.249+0000 7f0a955d0b80 20 client.25808 trim_cache size 0 max 16384 2024-03-23T17:51:35.253 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-03-23T17:51:35.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.249+0000 7f0a81a00700 1 -- 172.21.15.86:0/3185155909 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f0a600a13b0 msgr2=0x7f0a600a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:35.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.249+0000 7f0a81a00700 1 --2- 172.21.15.86:0/3185155909 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f0a600a13b0 0x7f0a600a3870 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c0747e0 tx=0x7f0a68011720 comp rx=0 tx=0).stop 2024-03-23T17:51:35.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.249+0000 7f0a81a00700 1 -- 172.21.15.86:0/3185155909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a7c100f70 msgr2=0x7f0a7c0730b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:35.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.249+0000 7f0a81a00700 1 --2- 172.21.15.86:0/3185155909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a7c100f70 0x7f0a7c0730b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c005910 tx=0x7f0a6c004730 comp rx=0 tx=0).stop 2024-03-23T17:51:35.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.249+0000 7f0a81a00700 1 -- 172.21.15.86:0/3185155909 shutdown_connections 2024-03-23T17:51:35.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.249+0000 7f0a81a00700 1 --2- 172.21.15.86:0/3185155909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a7c073b30 0x7f0a7c1a53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:35.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.249+0000 7f0a81a00700 1 --2- 172.21.15.86:0/3185155909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a7c100f70 0x7f0a7c0730b0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:35.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.249+0000 7f0a81a00700 1 --2- 172.21.15.86:0/3185155909 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f0a600a13b0 0x7f0a600a3870 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:35.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.249+0000 7f0a81a00700 1 --2- 172.21.15.86:0/3185155909 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0a7c106090 0x7f0a7c0735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:35.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.249+0000 7f0a81a00700 1 -- 172.21.15.86:0/3185155909 >> 172.21.15.86:0/3185155909 conn(0x7f0a7c0fc7f0 msgr2=0x7f0a7c0fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:35.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.249+0000 7f0a81a00700 1 -- 172.21.15.86:0/3185155909 shutdown_connections 2024-03-23T17:51:35.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.249+0000 7f0a81a00700 1 -- 172.21.15.86:0/3185155909 wait complete. 2024-03-23T17:51:35.267 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-23T17:51:35.268 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-23T17:51:35.313 INFO:tasks.workunit.client.0.smithi086.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-23T17:51:35.425 INFO:tasks.workunit.client.0.smithi086.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-23T17:51:35.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.467+0000 7f661a000700 1 -- 172.21.15.86:0/2331995383 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6614073a70 msgr2=0x7f66141117c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:35.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.467+0000 7f661a000700 1 --2- 172.21.15.86:0/2331995383 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6614073a70 0x7f66141117c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6600009990 tx=0x7f6600009ca0 comp rx=0 tx=0).stop 2024-03-23T17:51:35.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.467+0000 7f661a000700 1 -- 172.21.15.86:0/2331995383 shutdown_connections 2024-03-23T17:51:35.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.467+0000 7f661a000700 1 --2- 172.21.15.86:0/2331995383 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6614111d00 0x7f66141140f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:35.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.467+0000 7f661a000700 1 --2- 172.21.15.86:0/2331995383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6614073150 0x7f6614073530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:35.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.467+0000 7f661a000700 1 --2- 172.21.15.86:0/2331995383 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6614073a70 0x7f66141117c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:35.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.467+0000 7f661a000700 1 -- 172.21.15.86:0/2331995383 >> 172.21.15.86:0/2331995383 conn(0x7f66140fc7c0 msgr2=0x7f66140febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:35.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.467+0000 7f661a000700 1 -- 172.21.15.86:0/2331995383 shutdown_connections 2024-03-23T17:51:35.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.467+0000 7f661a000700 1 -- 172.21.15.86:0/2331995383 wait complete. 2024-03-23T17:51:35.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.468+0000 7f661a000700 1 Processor -- start 2024-03-23T17:51:35.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.468+0000 7f661a000700 1 -- start start 2024-03-23T17:51:35.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.469+0000 7f661a000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6614073150 0x7f66141a3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:35.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.469+0000 7f661a000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6614073a70 0x7f66141a3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:35.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.469+0000 7f661a000700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6614111d00 0x7f661419d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:35.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.469+0000 7f661a000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f661419dcd0 con 0x7f6614073a70 2024-03-23T17:51:35.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.469+0000 7f661a000700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f661419de40 con 0x7f6614073150 2024-03-23T17:51:35.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.469+0000 7f661a000700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f661419e000 con 0x7f6614111d00 2024-03-23T17:51:35.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.469+0000 7f6612a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6614073a70 0x7f66141a3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:35.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.469+0000 7f6613400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6614073150 0x7f66141a3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:35.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.469+0000 7f6613400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6614073150 0x7f66141a3540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:48282/0 (socket says 172.21.15.86:48282) 2024-03-23T17:51:35.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.469+0000 7f6613e00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6614111d00 0x7f661419d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:35.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.469+0000 7f6613400700 1 -- 172.21.15.86:0/467795540 learned_addr learned my addr 172.21.15.86:0/467795540 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:35.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.469+0000 7f6613e00700 1 -- 172.21.15.86:0/467795540 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6614073150 msgr2=0x7f66141a3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:35.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.469+0000 7f6613e00700 1 --2- 172.21.15.86:0/467795540 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6614073150 0x7f66141a3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:35.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.469+0000 7f6613e00700 1 -- 172.21.15.86:0/467795540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6614073a70 msgr2=0x7f66141a3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:35.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.469+0000 7f6613e00700 1 --2- 172.21.15.86:0/467795540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6614073a70 0x7f66141a3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:35.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.469+0000 7f6613e00700 1 -- 172.21.15.86:0/467795540 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66000095f0 con 0x7f6614111d00 2024-03-23T17:51:35.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.470+0000 7f6613e00700 1 --2- 172.21.15.86:0/467795540 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6614111d00 0x7f661419d670 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f660c00ef90 tx=0x7f660c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:35.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.470+0000 7f660be00700 1 -- 172.21.15.86:0/467795540 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660c018070 con 0x7f6614111d00 2024-03-23T17:51:35.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.470+0000 7f660be00700 1 -- 172.21.15.86:0/467795540 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f660c00f040 con 0x7f6614111d00 2024-03-23T17:51:35.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.470+0000 7f661a000700 1 -- 172.21.15.86:0/467795540 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f661419e280 con 0x7f6614111d00 2024-03-23T17:51:35.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.470+0000 7f660be00700 1 -- 172.21.15.86:0/467795540 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660c00cc30 con 0x7f6614111d00 2024-03-23T17:51:35.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.470+0000 7f661a000700 1 -- 172.21.15.86:0/467795540 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66141aa530 con 0x7f6614111d00 2024-03-23T17:51:35.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.473+0000 7f660be00700 1 -- 172.21.15.86:0/467795540 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) v1 ==== 136098+0+0 (secure 0 0 0) 0x7f660c003a00 con 0x7f6614111d00 2024-03-23T17:51:35.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.475+0000 7f662dbaeb80 1 Processor -- start 2024-03-23T17:51:35.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.475+0000 7f662dbaeb80 1 -- start start 2024-03-23T17:51:35.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.475+0000 7f662dbaeb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e9f2e65d0 0x563e9f2e69b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:35.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.475+0000 7f662dbaeb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e9f2e6ef0 0x563e9f2f18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:35.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.475+0000 7f662dbaeb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563e9f2f1df0 0x563e9f2f41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:35.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.475+0000 7f662dbaeb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e9f23f990 con 0x563e9f2e65d0 2024-03-23T17:51:35.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.475+0000 7f662dbaeb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e9f23d350 con 0x563e9f2e6ef0 2024-03-23T17:51:35.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.475+0000 7f662dbaeb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e9f238c40 con 0x563e9f2f1df0 2024-03-23T17:51:35.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.476+0000 7f660be00700 1 --2- 172.21.15.86:0/467795540 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f65f80a13b0 0x7f65f80a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:35.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.476+0000 7f6612a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e9f2e65d0 0x563e9f2e69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:35.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.476+0000 7f660be00700 1 -- 172.21.15.86:0/467795540 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f660c0ccad0 con 0x7f6614111d00 2024-03-23T17:51:35.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.476+0000 7f6612a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e9f2e65d0 0x563e9f2e69b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38310/0 (socket says 172.21.15.86:38310) 2024-03-23T17:51:35.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.476+0000 7f6612a00700 1 -- 172.21.15.86:0/3885675751 learned_addr learned my addr 172.21.15.86:0/3885675751 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:35.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.476+0000 7f6613400700 1 --2- 172.21.15.86:0/3885675751 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e9f2e6ef0 0x563e9f2f18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:35.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.476+0000 7f6613e00700 1 --2- 172.21.15.86:0/467795540 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f65f80a13b0 0x7f65f80a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:35.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.476+0000 7f6612a00700 1 --2- 172.21.15.86:0/3885675751 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563e9f2f1df0 0x563e9f2f41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:35.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.476+0000 7f6613400700 1 -- 172.21.15.86:0/3885675751 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563e9f2f1df0 msgr2=0x563e9f2f41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:35.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.476+0000 7f6613400700 1 --2- 172.21.15.86:0/3885675751 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563e9f2f1df0 0x563e9f2f41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:35.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.476+0000 7f6613400700 1 -- 172.21.15.86:0/3885675751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e9f2e65d0 msgr2=0x563e9f2e69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:35.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.476+0000 7f6613400700 1 --2- 172.21.15.86:0/3885675751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e9f2e65d0 0x563e9f2e69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:35.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.476+0000 7f6613400700 1 -- 172.21.15.86:0/3885675751 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e9f0ba3b0 con 0x563e9f2e6ef0 2024-03-23T17:51:35.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.477+0000 7f6613e00700 1 --2- 172.21.15.86:0/467795540 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f65f80a13b0 0x7f65f80a3870 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f660c0ca0f0 tx=0x7f660c004c50 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:35.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.477+0000 7f6613400700 1 --2- 172.21.15.86:0/3885675751 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e9f2e6ef0 0x563e9f2f18b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6604009d10 tx=0x7f6604011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a1cab77f80c906 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:35.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.477+0000 7f65ffe00700 1 -- 172.21.15.86:0/3885675751 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6604004250 con 0x563e9f2e6ef0 2024-03-23T17:51:35.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.477+0000 7f65ffe00700 1 -- 172.21.15.86:0/3885675751 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6604004cf0 con 0x563e9f2e6ef0 2024-03-23T17:51:35.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.477+0000 7f65ffe00700 1 -- 172.21.15.86:0/3885675751 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6604005580 con 0x563e9f2e6ef0 2024-03-23T17:51:35.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.478+0000 7f662dbaeb80 1 -- 172.21.15.86:0/3885675751 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e9f2e6ef0 msgr2=0x563e9f2f18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:35.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.478+0000 7f662dbaeb80 1 --2- 172.21.15.86:0/3885675751 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e9f2e6ef0 0x563e9f2f18b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6604009d10 tx=0x7f6604011720 comp rx=0 tx=0).stop 2024-03-23T17:51:35.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.478+0000 7f662dbaeb80 1 -- 172.21.15.86:0/3885675751 shutdown_connections 2024-03-23T17:51:35.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.478+0000 7f662dbaeb80 1 --2- 172.21.15.86:0/3885675751 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563e9f2f1df0 0x563e9f2f41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:35.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.478+0000 7f662dbaeb80 1 --2- 172.21.15.86:0/3885675751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e9f2e65d0 0x563e9f2e69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:35.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.478+0000 7f662dbaeb80 1 --2- 172.21.15.86:0/3885675751 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e9f2e6ef0 0x563e9f2f18b0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:35.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.478+0000 7f662dbaeb80 1 -- 172.21.15.86:0/3885675751 >> 172.21.15.86:0/3885675751 conn(0x563e9f2466a0 msgr2=0x563e9f2e2a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:35.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.478+0000 7f662dbaeb80 1 -- 172.21.15.86:0/3885675751 shutdown_connections 2024-03-23T17:51:35.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.478+0000 7f662dbaeb80 1 -- 172.21.15.86:0/3885675751 wait complete. 2024-03-23T17:51:35.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.478+0000 7f662dbaeb80 1 -- start start 2024-03-23T17:51:35.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.479+0000 7f662dbaeb80 1 Processor -- start 2024-03-23T17:51:35.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.479+0000 7f662dbaeb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e9f2e65d0 0x563e9f2f1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:35.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.479+0000 7f662dbaeb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e9f2e6ef0 0x563e9f2eeb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:35.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.479+0000 7f662dbaeb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563e9f2f1df0 0x563e9f2ef260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:35.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.479+0000 7f662dbaeb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e9f1d4d80 con 0x563e9f2e65d0 2024-03-23T17:51:35.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.479+0000 7f662dbaeb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e9f1d4a20 con 0x563e9f2e6ef0 2024-03-23T17:51:35.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.479+0000 7f662dbaeb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e9f0bc6a0 con 0x563e9f2f1df0 2024-03-23T17:51:35.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.479+0000 7f662dbaeb80 10 client.0 _ll_register_callbacks cb 0x563e9f1de470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:51:35.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.479+0000 7f6612a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e9f2e65d0 0x563e9f2f1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:35.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.479+0000 7f6612a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e9f2e65d0 0x563e9f2f1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38318/0 (socket says 172.21.15.86:38318) 2024-03-23T17:51:35.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.479+0000 7f6612a00700 1 -- 172.21.15.86:0/166619042 learned_addr learned my addr 172.21.15.86:0/166619042 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:35.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.479+0000 7f6613400700 1 --2- 172.21.15.86:0/166619042 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e9f2e6ef0 0x563e9f2eeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:35.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.480+0000 7f6612a00700 1 --2- 172.21.15.86:0/166619042 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563e9f2f1df0 0x563e9f2ef260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:35.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.480+0000 7f6613400700 1 -- 172.21.15.86:0/166619042 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563e9f2f1df0 msgr2=0x563e9f2ef260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:35.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.480+0000 7f6613400700 1 --2- 172.21.15.86:0/166619042 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563e9f2f1df0 0x563e9f2ef260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:35.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.480+0000 7f6613400700 1 -- 172.21.15.86:0/166619042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e9f2e65d0 msgr2=0x563e9f2f1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:35.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.480+0000 7f6613400700 1 --2- 172.21.15.86:0/166619042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e9f2e65d0 0x563e9f2f1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:35.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.480+0000 7f6613400700 1 -- 172.21.15.86:0/166619042 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6604009970 con 0x563e9f2e6ef0 2024-03-23T17:51:35.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.480+0000 7f6612a00700 1 --2- 172.21.15.86:0/166619042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e9f2e65d0 0x563e9f2f1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:51:35.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.480+0000 7f6613400700 1 --2- 172.21.15.86:0/166619042 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e9f2e6ef0 0x563e9f2eeb20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6604000c00 tx=0x7f6604004650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb23ea5e99537bad server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:35.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.480+0000 7f65fcc00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T17:51:35.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.480+0000 7f65fcc00700 1 -- 172.21.15.86:0/166619042 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6604004ee0 con 0x563e9f2e6ef0 2024-03-23T17:51:35.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.480+0000 7f662dbaeb80 1 -- 172.21.15.86:0/166619042 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563e9f1f9380 con 0x563e9f2e6ef0 2024-03-23T17:51:35.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.480+0000 7f65fcc00700 1 -- 172.21.15.86:0/166619042 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f6604004960 con 0x563e9f2e6ef0 2024-03-23T17:51:35.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.481+0000 7f65fcc00700 1 -- 172.21.15.86:0/166619042 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6604004b20 con 0x563e9f2e6ef0 2024-03-23T17:51:35.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.481+0000 7f662dbaeb80 10 client.25820 fetch_fsmap learned FSMap version 28 2024-03-23T17:51:35.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.481+0000 7f662dbaeb80 1 -- 172.21.15.86:0/166619042 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x563e9f066b60 con 0x563e9f2e6ef0 2024-03-23T17:51:35.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.481+0000 7f65fcc00700 1 -- 172.21.15.86:0/166619042 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563e9f1f9380 con 0x563e9f2e6ef0 2024-03-23T17:51:35.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.481+0000 7f65fcc00700 1 -- 172.21.15.86:0/166619042 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 28) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6604022400 con 0x563e9f2e6ef0 2024-03-23T17:51:35.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.481+0000 7f662dbaeb80 10 client.25820 fetch_fsmap finished waiting for FSMap version 28 2024-03-23T17:51:35.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.481+0000 7f662dbaeb80 10 client.25820 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T17:51:35.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.481+0000 7f662dbaeb80 20 client.25820 populate_metadata read hostname 'smithi086' 2024-03-23T17:51:35.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.481+0000 7f662dbaeb80 1 --2- 172.21.15.86:0/166619042 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x563e9f301800 0x563e9f303c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:35.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.481+0000 7f662dbaeb80 4 client.25820 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:51:35.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.481+0000 7f662dbaeb80 1 -- 172.21.15.86:0/166619042 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563e9f231700 con 0x563e9f301800 2024-03-23T17:51:35.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.481+0000 7f6612a00700 1 --2- 172.21.15.86:0/166619042 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x563e9f301800 0x563e9f303c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:35.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.482+0000 7f6612a00700 1 --2- 172.21.15.86:0/166619042 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x563e9f301800 0x563e9f303c20 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8d32707fa052b16 server_cookie=de3aa7c040d0bbe in_seq=0 out_seq=0 2024-03-23T17:51:35.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.482+0000 7f65fcc00700 10 client.25820 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:51:35.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.483+0000 7f65fcc00700 1 -- 172.21.15.86:0/166619042 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x563e9f231700 con 0x563e9f301800 2024-03-23T17:51:35.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.483+0000 7f65fcc00700 10 client.25820 handle_command_reply: tid=0 2024-03-23T17:51:35.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.483+0000 7f662dbaeb80 1 client.25820 shutdown 2024-03-23T17:51:35.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.483+0000 7f662dbaeb80 1 -- 172.21.15.86:0/166619042 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e9f2e6ef0 msgr2=0x563e9f2eeb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:35.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.483+0000 7f662dbaeb80 1 --2- 172.21.15.86:0/166619042 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e9f2e6ef0 0x563e9f2eeb20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6604000c00 tx=0x7f6604004650 comp rx=0 tx=0).stop 2024-03-23T17:51:35.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.483+0000 7f662dbaeb80 1 -- 172.21.15.86:0/166619042 shutdown_connections 2024-03-23T17:51:35.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.483+0000 7f662dbaeb80 1 --2- 172.21.15.86:0/166619042 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x563e9f301800 0x563e9f303c20 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:35.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.483+0000 7f662dbaeb80 1 --2- 172.21.15.86:0/166619042 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563e9f2f1df0 0x563e9f2ef260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:35.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.483+0000 7f662dbaeb80 1 --2- 172.21.15.86:0/166619042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e9f2e65d0 0x563e9f2f1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:35.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.483+0000 7f662dbaeb80 1 --2- 172.21.15.86:0/166619042 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e9f2e6ef0 0x563e9f2eeb20 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:35.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.483+0000 7f65fcc00700 0 client.25820 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:51:35.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.483+0000 7f662dbaeb80 1 -- 172.21.15.86:0/166619042 >> 172.21.15.86:0/166619042 conn(0x563e9f2466a0 msgr2=0x563e9f2e8510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:35.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.483+0000 7f662dbaeb80 1 -- 172.21.15.86:0/166619042 shutdown_connections 2024-03-23T17:51:35.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.483+0000 7f662dbaeb80 1 -- 172.21.15.86:0/166619042 wait complete. 2024-03-23T17:51:35.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.483+0000 7f662dbaeb80 20 client.25820 trim_cache size 0 max 16384 2024-03-23T17:51:35.491 INFO:tasks.workunit.client.0.smithi086.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-23T17:51:35.495 INFO:tasks.workunit.client.0.smithi086.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-23T17:51:35.495 INFO:tasks.workunit.client.0.smithi086.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-23T17:51:35.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.497+0000 7f662dbaeb80 1 Processor -- start 2024-03-23T17:51:35.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.497+0000 7f662dbaeb80 1 -- start start 2024-03-23T17:51:35.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.497+0000 7f662dbaeb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e9f2e65d0 0x563e9f2eeae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:35.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.497+0000 7f662dbaeb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e9f2e6ef0 0x563e9f3085c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:35.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.497+0000 7f662dbaeb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563e9f2f1df0 0x563e9f2f8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:35.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.497+0000 7f662dbaeb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e9f2e7880 con 0x563e9f2e6ef0 2024-03-23T17:51:35.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.497+0000 7f662dbaeb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e9f2ef840 con 0x563e9f2e65d0 2024-03-23T17:51:35.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.497+0000 7f662dbaeb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e9f2fde10 con 0x563e9f2f1df0 2024-03-23T17:51:35.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.497+0000 7f6613400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e9f2e6ef0 0x563e9f3085c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:35.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.497+0000 7f6613400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e9f2e6ef0 0x563e9f3085c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38324/0 (socket says 172.21.15.86:38324) 2024-03-23T17:51:35.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.497+0000 7f6612a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e9f2e65d0 0x563e9f2eeae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:35.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.497+0000 7f6613400700 1 -- 172.21.15.86:0/244361899 learned_addr learned my addr 172.21.15.86:0/244361899 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:35.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.498+0000 7f6612a00700 1 --2- 172.21.15.86:0/244361899 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563e9f2f1df0 0x563e9f2f8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:35.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.498+0000 7f6613400700 1 -- 172.21.15.86:0/244361899 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563e9f2f1df0 msgr2=0x563e9f2f8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:35.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.498+0000 7f6613400700 1 --2- 172.21.15.86:0/244361899 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563e9f2f1df0 0x563e9f2f8a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:35.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.498+0000 7f6613400700 1 -- 172.21.15.86:0/244361899 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e9f2e65d0 msgr2=0x563e9f2eeae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:35.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.498+0000 7f6613400700 1 --2- 172.21.15.86:0/244361899 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e9f2e65d0 0x563e9f2eeae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:35.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.498+0000 7f6613400700 1 -- 172.21.15.86:0/244361899 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e9f1feea0 con 0x563e9f2e6ef0 2024-03-23T17:51:35.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.498+0000 7f6612a00700 1 --2- 172.21.15.86:0/244361899 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e9f2e65d0 0x563e9f2eeae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:51:35.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.498+0000 7f6613400700 1 --2- 172.21.15.86:0/244361899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e9f2e6ef0 0x563e9f3085c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f6604009d10 tx=0x7f6604005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=167794e6afc39879 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:35.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.498+0000 7f6609200700 1 -- 172.21.15.86:0/244361899 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6604022b50 con 0x563e9f2e6ef0 2024-03-23T17:51:35.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.499+0000 7f6609200700 1 -- 172.21.15.86:0/244361899 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6604004460 con 0x563e9f2e6ef0 2024-03-23T17:51:35.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.499+0000 7f6609200700 1 -- 172.21.15.86:0/244361899 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6604004960 con 0x563e9f2e6ef0 2024-03-23T17:51:35.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.499+0000 7f662dbaeb80 1 -- 172.21.15.86:0/244361899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e9f2e6ef0 msgr2=0x563e9f3085c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:35.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.500+0000 7f662dbaeb80 1 --2- 172.21.15.86:0/244361899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e9f2e6ef0 0x563e9f3085c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f6604009d10 tx=0x7f6604005e50 comp rx=0 tx=0).stop 2024-03-23T17:51:35.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.500+0000 7f662dbaeb80 1 -- 172.21.15.86:0/244361899 shutdown_connections 2024-03-23T17:51:35.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.500+0000 7f662dbaeb80 1 --2- 172.21.15.86:0/244361899 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563e9f2f1df0 0x563e9f2f8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:35.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.500+0000 7f662dbaeb80 1 --2- 172.21.15.86:0/244361899 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e9f2e65d0 0x563e9f2eeae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:35.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.500+0000 7f662dbaeb80 1 --2- 172.21.15.86:0/244361899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e9f2e6ef0 0x563e9f3085c0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:35.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.500+0000 7f662dbaeb80 1 -- 172.21.15.86:0/244361899 >> 172.21.15.86:0/244361899 conn(0x563e9f2466a0 msgr2=0x563e9f2ee3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:35.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.500+0000 7f662dbaeb80 1 -- 172.21.15.86:0/244361899 shutdown_connections 2024-03-23T17:51:35.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.500+0000 7f662dbaeb80 1 -- 172.21.15.86:0/244361899 wait complete. 2024-03-23T17:51:35.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.500+0000 7f662dbaeb80 1 -- start start 2024-03-23T17:51:35.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.501+0000 7f662dbaeb80 1 Processor -- start 2024-03-23T17:51:35.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.501+0000 7f662dbaeb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e9f2e65d0 0x563e9f34c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:35.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.501+0000 7f662dbaeb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e9f2e6ef0 0x563e9f34cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:35.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.501+0000 7f662dbaeb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563e9f2f1df0 0x563e9f2f6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:35.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.501+0000 7f662dbaeb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e9f2f6d90 con 0x563e9f2e65d0 2024-03-23T17:51:35.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.501+0000 7f662dbaeb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e9f2f6f00 con 0x563e9f2e6ef0 2024-03-23T17:51:35.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.501+0000 7f662dbaeb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e9f2f70c0 con 0x563e9f2f1df0 2024-03-23T17:51:35.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.501+0000 7f662dbaeb80 10 client.0 _ll_register_callbacks cb 0x563e9f1de470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:51:35.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.502+0000 7f6612a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e9f2e65d0 0x563e9f34c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:35.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.502+0000 7f6613400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e9f2e6ef0 0x563e9f34cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:35.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.502+0000 7f6613400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e9f2e6ef0 0x563e9f34cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:48314/0 (socket says 172.21.15.86:48314) 2024-03-23T17:51:35.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.502+0000 7f6613400700 1 -- 172.21.15.86:0/1916528733 learned_addr learned my addr 172.21.15.86:0/1916528733 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:35.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.502+0000 7f6613400700 1 -- 172.21.15.86:0/1916528733 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563e9f2f1df0 msgr2=0x563e9f2f6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:35.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.502+0000 7f6613400700 1 --2- 172.21.15.86:0/1916528733 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563e9f2f1df0 0x563e9f2f6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:35.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.502+0000 7f6613400700 1 -- 172.21.15.86:0/1916528733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e9f2e65d0 msgr2=0x563e9f34c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:35.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.502+0000 7f6613400700 1 --2- 172.21.15.86:0/1916528733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e9f2e65d0 0x563e9f34c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:35.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.502+0000 7f6613400700 1 -- 172.21.15.86:0/1916528733 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6600032040 con 0x563e9f2e6ef0 2024-03-23T17:51:35.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.502+0000 7f6613400700 1 --2- 172.21.15.86:0/1916528733 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e9f2e6ef0 0x563e9f34cba0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6604014040 tx=0x7f66040092e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f691d468e5dcf068 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:35.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.502+0000 7f65fcc00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T17:51:35.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.502+0000 7f65fcc00700 1 -- 172.21.15.86:0/1916528733 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6604004b70 con 0x563e9f2e6ef0 2024-03-23T17:51:35.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.502+0000 7f65fcc00700 1 -- 172.21.15.86:0/1916528733 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f6604000e40 con 0x563e9f2e6ef0 2024-03-23T17:51:35.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.502+0000 7f662dbaeb80 1 -- 172.21.15.86:0/1916528733 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6604004cf0 con 0x563e9f2e6ef0 2024-03-23T17:51:35.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.503+0000 7f65fcc00700 1 -- 172.21.15.86:0/1916528733 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6604022580 con 0x563e9f2e6ef0 2024-03-23T17:51:35.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.503+0000 7f662dbaeb80 10 client.25826 fetch_fsmap learned FSMap version 28 2024-03-23T17:51:35.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.503+0000 7f662dbaeb80 1 -- 172.21.15.86:0/1916528733 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x563e9f066b60 con 0x563e9f2e6ef0 2024-03-23T17:51:35.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.503+0000 7f65fcc00700 1 -- 172.21.15.86:0/1916528733 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6604004cf0 con 0x563e9f2e6ef0 2024-03-23T17:51:35.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.503+0000 7f65fcc00700 1 -- 172.21.15.86:0/1916528733 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 28) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6604003c50 con 0x563e9f2e6ef0 2024-03-23T17:51:35.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.503+0000 7f662dbaeb80 10 client.25826 fetch_fsmap finished waiting for FSMap version 28 2024-03-23T17:51:35.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.503+0000 7f662dbaeb80 10 client.25826 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T17:51:35.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.503+0000 7f662dbaeb80 20 client.25826 populate_metadata read hostname 'smithi086' 2024-03-23T17:51:35.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.503+0000 7f662dbaeb80 1 --2- 172.21.15.86:0/1916528733 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x563e9f353fb0 0x563e9f3563a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:35.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.503+0000 7f662dbaeb80 4 client.25826 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-23T17:51:35.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.503+0000 7f662dbaeb80 1 -- 172.21.15.86:0/1916528733 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563e9f087cf0 con 0x563e9f353fb0 2024-03-23T17:51:35.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.504+0000 7f6612a00700 1 --2- 172.21.15.86:0/1916528733 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x563e9f353fb0 0x563e9f3563a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:35.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.504+0000 7f65fcc00700 10 client.25826 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:51:35.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.504+0000 7f6612a00700 1 --2- 172.21.15.86:0/1916528733 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x563e9f353fb0 0x563e9f3563a0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d130ac8e606bb0c server_cookie=40beae6c4a60f83a in_seq=0 out_seq=0 2024-03-23T17:51:35.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.504+0000 7f65fcc00700 1 -- 172.21.15.86:0/1916528733 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563e9f087cf0 con 0x563e9f353fb0 2024-03-23T17:51:35.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.504+0000 7f65fcc00700 10 client.25826 handle_command_reply: tid=0 2024-03-23T17:51:35.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.505+0000 7f662dbaeb80 1 client.25826 shutdown 2024-03-23T17:51:35.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.505+0000 7f662dbaeb80 1 -- 172.21.15.86:0/1916528733 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e9f2e6ef0 msgr2=0x563e9f34cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:35.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.505+0000 7f662dbaeb80 1 --2- 172.21.15.86:0/1916528733 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e9f2e6ef0 0x563e9f34cba0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6604014040 tx=0x7f66040092e0 comp rx=0 tx=0).stop 2024-03-23T17:51:35.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.505+0000 7f662dbaeb80 1 -- 172.21.15.86:0/1916528733 shutdown_connections 2024-03-23T17:51:35.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.505+0000 7f662dbaeb80 1 --2- 172.21.15.86:0/1916528733 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x563e9f353fb0 0x563e9f3563a0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:35.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.505+0000 7f662dbaeb80 1 --2- 172.21.15.86:0/1916528733 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563e9f2f1df0 0x563e9f2f6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:35.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.505+0000 7f662dbaeb80 1 --2- 172.21.15.86:0/1916528733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e9f2e65d0 0x563e9f34c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:35.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.505+0000 7f662dbaeb80 1 --2- 172.21.15.86:0/1916528733 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563e9f2e6ef0 0x563e9f34cba0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:35.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.505+0000 7f65fcc00700 0 client.25826 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:51:35.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.505+0000 7f662dbaeb80 1 -- 172.21.15.86:0/1916528733 >> 172.21.15.86:0/1916528733 conn(0x563e9f2466a0 msgr2=0x563e9f2e9df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:35.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.505+0000 7f662dbaeb80 1 -- 172.21.15.86:0/1916528733 shutdown_connections 2024-03-23T17:51:35.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.505+0000 7f662dbaeb80 1 -- 172.21.15.86:0/1916528733 wait complete. 2024-03-23T17:51:35.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.505+0000 7f662dbaeb80 20 client.25826 trim_cache size 0 max 16384 2024-03-23T17:51:35.509 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-03-23T17:51:35.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.506+0000 7f661a000700 1 -- 172.21.15.86:0/467795540 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f65f80a13b0 msgr2=0x7f65f80a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:35.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.506+0000 7f661a000700 1 --2- 172.21.15.86:0/467795540 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f65f80a13b0 0x7f65f80a3870 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f660c0ca0f0 tx=0x7f660c004c50 comp rx=0 tx=0).stop 2024-03-23T17:51:35.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.506+0000 7f661a000700 1 -- 172.21.15.86:0/467795540 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6614111d00 msgr2=0x7f661419d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:35.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.506+0000 7f661a000700 1 --2- 172.21.15.86:0/467795540 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6614111d00 0x7f661419d670 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f660c00ef90 tx=0x7f660c00c3a0 comp rx=0 tx=0).stop 2024-03-23T17:51:35.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.506+0000 7f661a000700 1 -- 172.21.15.86:0/467795540 shutdown_connections 2024-03-23T17:51:35.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.506+0000 7f661a000700 1 --2- 172.21.15.86:0/467795540 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f65f80a13b0 0x7f65f80a3870 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:35.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.506+0000 7f661a000700 1 --2- 172.21.15.86:0/467795540 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6614111d00 0x7f661419d670 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:35.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.506+0000 7f661a000700 1 --2- 172.21.15.86:0/467795540 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6614073150 0x7f66141a3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:35.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.506+0000 7f661a000700 1 --2- 172.21.15.86:0/467795540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6614073a70 0x7f66141a3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:35.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.506+0000 7f661a000700 1 -- 172.21.15.86:0/467795540 >> 172.21.15.86:0/467795540 conn(0x7f66140fc7c0 msgr2=0x7f66140fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:35.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.506+0000 7f661a000700 1 -- 172.21.15.86:0/467795540 shutdown_connections 2024-03-23T17:51:35.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:35.506+0000 7f661a000700 1 -- 172.21.15.86:0/467795540 wait complete. 2024-03-23T17:51:35.582 INFO:tasks.workunit.client.0.smithi086.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-23T17:51:35.590 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-23T17:51:35.590 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-23T17:51:35.632 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2024-03-23T17:51:35.633 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-03-23T17:51:35.633 DEBUG:teuthology.orchestra.run.smithi086:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=82316b04821a6c2c1d458a881920454bbadb8837 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2024-03-23T17:51:35.767 INFO:tasks.workunit.client.0.smithi086.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-03-23T17:51:35.767 INFO:tasks.workunit.client.0.smithi086.stdout: 2024-03-23T17:51:35.767 INFO:tasks.workunit.client.0.smithi086.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-03-23T17:51:35.777 INFO:tasks.workunit.client.0.smithi086.stdout:0 of 1 processes prepared for launch 0 sec 2024-03-23T17:51:35.877 INFO:tasks.workunit.client.0.smithi086.stdout:1 of 1 processes prepared for launch 0 sec 2024-03-23T17:51:35.877 INFO:tasks.workunit.client.0.smithi086.stdout:releasing clients 2024-03-23T17:51:36.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:51:35 smithi086 ceph-mon[38221]: pgmap v31: 97 pgs: 97 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-03-23T17:51:36.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:51:35 smithi092 ceph-mon[79089]: pgmap v31: 97 pgs: 97 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-03-23T17:51:36.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:51:35 smithi152 ceph-mon[47949]: pgmap v31: 97 pgs: 97 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-03-23T17:51:36.527 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-23T17:51:36.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T17:51:36.527 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-23T17:51:36.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.739+0000 7fd395800700 1 -- 172.21.15.86:0/2186695642 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd390102650 msgr2=0x7fd39010ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:36.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.739+0000 7fd395800700 1 --2- 172.21.15.86:0/2186695642 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd390102650 0x7fd39010ab30 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd37c009990 tx=0x7fd37c009ca0 comp rx=0 tx=0).stop 2024-03-23T17:51:36.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.739+0000 7fd395800700 1 -- 172.21.15.86:0/2186695642 shutdown_connections 2024-03-23T17:51:36.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.739+0000 7fd395800700 1 --2- 172.21.15.86:0/2186695642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd39010b150 0x7fd39010d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:36.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.739+0000 7fd395800700 1 --2- 172.21.15.86:0/2186695642 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd390101d30 0x7fd390102110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:36.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.739+0000 7fd395800700 1 --2- 172.21.15.86:0/2186695642 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd390102650 0x7fd39010ab30 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:36.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.739+0000 7fd395800700 1 -- 172.21.15.86:0/2186695642 >> 172.21.15.86:0/2186695642 conn(0x7fd3900fb570 msgr2=0x7fd3900fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:36.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.739+0000 7fd395800700 1 -- 172.21.15.86:0/2186695642 shutdown_connections 2024-03-23T17:51:36.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.739+0000 7fd395800700 1 -- 172.21.15.86:0/2186695642 wait complete. 2024-03-23T17:51:36.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.741+0000 7fd395800700 1 Processor -- start 2024-03-23T17:51:36.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.741+0000 7fd395800700 1 -- start start 2024-03-23T17:51:36.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.741+0000 7fd395800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd390101d30 0x7fd3901992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:36.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.741+0000 7fd395800700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd390102650 0x7fd3901997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:36.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.741+0000 7fd395800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd39010b150 0x7fd39019db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:36.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.742+0000 7fd395800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd39019e170 con 0x7fd390101d30 2024-03-23T17:51:36.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.742+0000 7fd395800700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd39019e2e0 con 0x7fd39010b150 2024-03-23T17:51:36.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.742+0000 7fd395800700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd39019e4a0 con 0x7fd390102650 2024-03-23T17:51:36.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.742+0000 7fd38fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd390101d30 0x7fd3901992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:36.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.742+0000 7fd38f400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd390102650 0x7fd3901997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:36.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.742+0000 7fd394e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd39010b150 0x7fd39019db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:36.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.742+0000 7fd38fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd390101d30 0x7fd3901992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38354/0 (socket says 172.21.15.86:38354) 2024-03-23T17:51:36.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.742+0000 7fd394e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd39010b150 0x7fd39019db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:48340/0 (socket says 172.21.15.86:48340) 2024-03-23T17:51:36.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.742+0000 7fd38f400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd390102650 0x7fd3901997e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:56966/0 (socket says 172.21.15.86:56966) 2024-03-23T17:51:36.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.742+0000 7fd38fe00700 1 -- 172.21.15.86:0/3922696237 learned_addr learned my addr 172.21.15.86:0/3922696237 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:36.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.742+0000 7fd38f400700 1 -- 172.21.15.86:0/3922696237 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd39010b150 msgr2=0x7fd39019db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:36.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.742+0000 7fd38f400700 1 --2- 172.21.15.86:0/3922696237 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd39010b150 0x7fd39019db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:36.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.742+0000 7fd38f400700 1 -- 172.21.15.86:0/3922696237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd390101d30 msgr2=0x7fd3901992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:36.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.742+0000 7fd38f400700 1 --2- 172.21.15.86:0/3922696237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd390101d30 0x7fd3901992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:36.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.742+0000 7fd38f400700 1 -- 172.21.15.86:0/3922696237 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd37c0095f0 con 0x7fd390102650 2024-03-23T17:51:36.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.742+0000 7fd394e00700 1 --2- 172.21.15.86:0/3922696237 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd39010b150 0x7fd39019db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:51:36.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.742+0000 7fd38f400700 1 --2- 172.21.15.86:0/3922696237 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd390102650 0x7fd3901997e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd37c000c00 tx=0x7fd37c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:36.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.743+0000 7fd38cc00700 1 -- 172.21.15.86:0/3922696237 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37c01d070 con 0x7fd390102650 2024-03-23T17:51:36.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.743+0000 7fd38cc00700 1 -- 172.21.15.86:0/3922696237 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd37c021410 con 0x7fd390102650 2024-03-23T17:51:36.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.743+0000 7fd395800700 1 -- 172.21.15.86:0/3922696237 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd39019e720 con 0x7fd390102650 2024-03-23T17:51:36.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.743+0000 7fd38cc00700 1 -- 172.21.15.86:0/3922696237 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37c017650 con 0x7fd390102650 2024-03-23T17:51:36.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.743+0000 7fd395800700 1 -- 172.21.15.86:0/3922696237 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3901a5fc0 con 0x7fd390102650 2024-03-23T17:51:36.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.747+0000 7fd38cc00700 1 -- 172.21.15.86:0/3922696237 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) v1 ==== 136098+0+0 (secure 0 0 0) 0x7fd37c021580 con 0x7fd390102650 2024-03-23T17:51:36.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.748+0000 7fd38cc00700 1 --2- 172.21.15.86:0/3922696237 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fd3740a1430 0x7fd3740a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:36.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.748+0000 7fd3aa98cb80 1 Processor -- start 2024-03-23T17:51:36.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.748+0000 7fd3aa98cb80 1 -- start start 2024-03-23T17:51:36.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.748+0000 7fd38cc00700 1 -- 172.21.15.86:0/3922696237 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fd37c0cdfd0 con 0x7fd390102650 2024-03-23T17:51:36.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.749+0000 7fd38fe00700 1 --2- 172.21.15.86:0/3922696237 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fd3740a1430 0x7fd3740a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:36.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.749+0000 7fd3aa98cb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55855e69b820 0x55855e69bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:36.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.749+0000 7fd3aa98cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55855e69c140 0x55855e6a6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:36.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.749+0000 7fd3aa98cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55855e6a7040 0x55855e6a9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:36.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.749+0000 7fd3aa98cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55855e5f4a10 con 0x55855e6a7040 2024-03-23T17:51:36.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.749+0000 7fd3aa98cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55855e5f23d0 con 0x55855e69c140 2024-03-23T17:51:36.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.749+0000 7fd3aa98cb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55855e5edcc0 con 0x55855e69b820 2024-03-23T17:51:36.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.749+0000 7fd38f400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55855e69c140 0x55855e6a6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:36.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.749+0000 7fd38f400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55855e69c140 0x55855e6a6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:48354/0 (socket says 172.21.15.86:48354) 2024-03-23T17:51:36.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.749+0000 7fd38f400700 1 -- 172.21.15.86:0/484569832 learned_addr learned my addr 172.21.15.86:0/484569832 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:36.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.749+0000 7fd394e00700 1 --2- 172.21.15.86:0/484569832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55855e6a7040 0x55855e6a9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:36.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.749+0000 7fd38fe00700 1 --2- 172.21.15.86:0/484569832 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55855e69b820 0x55855e69bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:36.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.749+0000 7fd38fe00700 1 --2- 172.21.15.86:0/3922696237 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fd3740a1430 0x7fd3740a38f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd380009d70 tx=0x7fd380009260 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:36.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.749+0000 7fd38f400700 1 -- 172.21.15.86:0/484569832 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55855e69b820 msgr2=0x55855e69bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:36.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.749+0000 7fd38f400700 1 --2- 172.21.15.86:0/484569832 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55855e69b820 0x55855e69bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:36.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.750+0000 7fd38f400700 1 -- 172.21.15.86:0/484569832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55855e6a7040 msgr2=0x55855e6a9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:36.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.750+0000 7fd38f400700 1 --2- 172.21.15.86:0/484569832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55855e6a7040 0x55855e6a9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:36.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.750+0000 7fd38f400700 1 -- 172.21.15.86:0/484569832 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55855e46f430 con 0x55855e69c140 2024-03-23T17:51:36.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.750+0000 7fd38f400700 1 --2- 172.21.15.86:0/484569832 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55855e69c140 0x55855e6a6b00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd37c00b2e0 tx=0x7fd37c096cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80fe9eeb94eec26f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:36.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.750+0000 7fd385600700 1 -- 172.21.15.86:0/484569832 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37c0a10c0 con 0x55855e69c140 2024-03-23T17:51:36.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.750+0000 7fd385600700 1 -- 172.21.15.86:0/484569832 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd37c09aab0 con 0x55855e69c140 2024-03-23T17:51:36.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.750+0000 7fd385600700 1 -- 172.21.15.86:0/484569832 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37c09b3d0 con 0x55855e69c140 2024-03-23T17:51:36.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.751+0000 7fd3aa98cb80 1 -- 172.21.15.86:0/484569832 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55855e69c140 msgr2=0x55855e6a6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:36.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.751+0000 7fd3aa98cb80 1 --2- 172.21.15.86:0/484569832 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55855e69c140 0x55855e6a6b00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd37c00b2e0 tx=0x7fd37c096cb0 comp rx=0 tx=0).stop 2024-03-23T17:51:36.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.751+0000 7fd3aa98cb80 1 -- 172.21.15.86:0/484569832 shutdown_connections 2024-03-23T17:51:36.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.751+0000 7fd3aa98cb80 1 --2- 172.21.15.86:0/484569832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55855e6a7040 0x55855e6a9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:36.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.751+0000 7fd3aa98cb80 1 --2- 172.21.15.86:0/484569832 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55855e69b820 0x55855e69bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:36.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.751+0000 7fd3aa98cb80 1 --2- 172.21.15.86:0/484569832 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55855e69c140 0x55855e6a6b00 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:36.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.751+0000 7fd3aa98cb80 1 -- 172.21.15.86:0/484569832 >> 172.21.15.86:0/484569832 conn(0x55855e5fb720 msgr2=0x55855e697ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:36.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.751+0000 7fd3aa98cb80 1 -- 172.21.15.86:0/484569832 shutdown_connections 2024-03-23T17:51:36.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.751+0000 7fd3aa98cb80 1 -- 172.21.15.86:0/484569832 wait complete. 2024-03-23T17:51:36.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.751+0000 7fd3aa98cb80 1 -- start start 2024-03-23T17:51:36.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.752+0000 7fd3aa98cb80 1 Processor -- start 2024-03-23T17:51:36.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.752+0000 7fd3aa98cb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55855e69b820 0x55855e6a6970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:36.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.752+0000 7fd3aa98cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55855e69c140 0x55855e6a3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:36.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.752+0000 7fd3aa98cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55855e6a7040 0x55855e6a4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:36.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.752+0000 7fd3aa98cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55855e589e00 con 0x55855e69c140 2024-03-23T17:51:36.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.752+0000 7fd3aa98cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55855e589aa0 con 0x55855e6a7040 2024-03-23T17:51:36.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.752+0000 7fd3aa98cb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55855e471720 con 0x55855e69b820 2024-03-23T17:51:36.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.753+0000 7fd3aa98cb80 10 client.0 _ll_register_callbacks cb 0x55855e5934f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:51:36.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.753+0000 7fd38f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55855e69c140 0x55855e6a3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:36.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.753+0000 7fd38f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55855e69c140 0x55855e6a3c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38370/0 (socket says 172.21.15.86:38370) 2024-03-23T17:51:36.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.753+0000 7fd38f400700 1 -- 172.21.15.86:0/2601641943 learned_addr learned my addr 172.21.15.86:0/2601641943 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:36.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.753+0000 7fd38fe00700 1 --2- 172.21.15.86:0/2601641943 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55855e69b820 0x55855e6a6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:36.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.753+0000 7fd394e00700 1 --2- 172.21.15.86:0/2601641943 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55855e6a7040 0x55855e6a4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:36.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.753+0000 7fd38f400700 1 -- 172.21.15.86:0/2601641943 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55855e69b820 msgr2=0x55855e6a6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:36.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.753+0000 7fd38f400700 1 --2- 172.21.15.86:0/2601641943 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55855e69b820 0x55855e6a6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:36.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.753+0000 7fd38f400700 1 -- 172.21.15.86:0/2601641943 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55855e6a7040 msgr2=0x55855e6a4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:36.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.753+0000 7fd38f400700 1 --2- 172.21.15.86:0/2601641943 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55855e6a7040 0x55855e6a4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:36.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.753+0000 7fd38f400700 1 -- 172.21.15.86:0/2601641943 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3901a5fc0 con 0x55855e69c140 2024-03-23T17:51:36.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.753+0000 7fd394e00700 1 --2- 172.21.15.86:0/2601641943 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55855e6a7040 0x55855e6a4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:51:36.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.754+0000 7fd38f400700 1 --2- 172.21.15.86:0/2601641943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55855e69c140 0x55855e6a3c60 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fd37c09af00 tx=0x7fd37c09b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d94fc818a494ca4 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:36.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.754+0000 7fd37a000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-23T17:51:36.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.754+0000 7fd37a000700 1 -- 172.21.15.86:0/2601641943 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37c09a8d0 con 0x55855e69c140 2024-03-23T17:51:36.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.754+0000 7fd37a000700 1 -- 172.21.15.86:0/2601641943 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fd37c0a1050 con 0x55855e69c140 2024-03-23T17:51:36.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.754+0000 7fd3aa98cb80 1 -- 172.21.15.86:0/2601641943 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55855e5ae400 con 0x55855e69c140 2024-03-23T17:51:36.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.754+0000 7fd37a000700 1 -- 172.21.15.86:0/2601641943 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd37c0b92b0 con 0x55855e69c140 2024-03-23T17:51:36.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.754+0000 7fd3aa98cb80 10 client.15777 fetch_fsmap learned FSMap version 28 2024-03-23T17:51:36.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.754+0000 7fd3aa98cb80 1 -- 172.21.15.86:0/2601641943 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55855e41bbe0 con 0x55855e69c140 2024-03-23T17:51:36.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.754+0000 7fd37a000700 1 -- 172.21.15.86:0/2601641943 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55855e5ae400 con 0x55855e69c140 2024-03-23T17:51:36.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.754+0000 7fd37a000700 1 -- 172.21.15.86:0/2601641943 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd37c0cb8d0 con 0x55855e69c140 2024-03-23T17:51:36.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.754+0000 7fd3aa98cb80 10 client.15777 fetch_fsmap finished waiting for FSMap version 28 2024-03-23T17:51:36.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.754+0000 7fd3aa98cb80 10 client.15777 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T17:51:36.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.754+0000 7fd3aa98cb80 20 client.15777 populate_metadata read hostname 'smithi086' 2024-03-23T17:51:36.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.754+0000 7fd3aa98cb80 1 --2- 172.21.15.86:0/2601641943 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55855e6b63f0 0x55855e6b8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:36.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.754+0000 7fd3aa98cb80 4 client.15777 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:51:36.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.754+0000 7fd3aa98cb80 1 -- 172.21.15.86:0/2601641943 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55855e5e6780 con 0x55855e6b63f0 2024-03-23T17:51:36.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.755+0000 7fd38fe00700 1 --2- 172.21.15.86:0/2601641943 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55855e6b63f0 0x55855e6b8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:36.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.755+0000 7fd37a000700 10 client.15777 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:51:36.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.755+0000 7fd38fe00700 1 --2- 172.21.15.86:0/2601641943 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55855e6b63f0 0x55855e6b8810 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e189dfee3f753e47 server_cookie=a061ae175854c959 in_seq=0 out_seq=0 2024-03-23T17:51:36.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.756+0000 7fd37a000700 1 -- 172.21.15.86:0/2601641943 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55855e5e6780 con 0x55855e6b63f0 2024-03-23T17:51:36.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.756+0000 7fd37a000700 10 client.15777 handle_command_reply: tid=0 2024-03-23T17:51:36.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.757+0000 7fd3aa98cb80 1 client.15777 shutdown 2024-03-23T17:51:36.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.757+0000 7fd3aa98cb80 1 -- 172.21.15.86:0/2601641943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55855e69c140 msgr2=0x55855e6a3c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:36.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.757+0000 7fd3aa98cb80 1 --2- 172.21.15.86:0/2601641943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55855e69c140 0x55855e6a3c60 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fd37c09af00 tx=0x7fd37c09b460 comp rx=0 tx=0).stop 2024-03-23T17:51:36.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.757+0000 7fd3aa98cb80 1 -- 172.21.15.86:0/2601641943 shutdown_connections 2024-03-23T17:51:36.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.757+0000 7fd3aa98cb80 1 --2- 172.21.15.86:0/2601641943 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55855e6b63f0 0x55855e6b8810 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:36.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.757+0000 7fd3aa98cb80 1 --2- 172.21.15.86:0/2601641943 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55855e6a7040 0x55855e6a4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:36.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.757+0000 7fd3aa98cb80 1 --2- 172.21.15.86:0/2601641943 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55855e69b820 0x55855e6a6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:36.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.757+0000 7fd37a000700 0 client.15777 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:51:36.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.757+0000 7fd3aa98cb80 1 --2- 172.21.15.86:0/2601641943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55855e69c140 0x55855e6a3c60 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:36.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.757+0000 7fd3aa98cb80 1 -- 172.21.15.86:0/2601641943 >> 172.21.15.86:0/2601641943 conn(0x55855e5fb720 msgr2=0x55855e69d910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:36.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.757+0000 7fd3aa98cb80 1 -- 172.21.15.86:0/2601641943 shutdown_connections 2024-03-23T17:51:36.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.757+0000 7fd3aa98cb80 1 -- 172.21.15.86:0/2601641943 wait complete. 2024-03-23T17:51:36.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.757+0000 7fd3aa98cb80 20 client.15777 trim_cache size 0 max 16384 2024-03-23T17:51:36.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.774+0000 7fd3aa98cb80 1 Processor -- start 2024-03-23T17:51:36.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.775+0000 7fd3aa98cb80 1 -- start start 2024-03-23T17:51:36.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.775+0000 7fd3aa98cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55855e69b820 0x55855e6ada30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:36.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.775+0000 7fd3aa98cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55855e69c140 0x55855e6b2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:36.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.775+0000 7fd3aa98cb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55855e6a7040 0x55855e697d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:36.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.775+0000 7fd3aa98cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55855e698260 con 0x55855e69b820 2024-03-23T17:51:36.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.775+0000 7fd3aa98cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55855e6a45c0 con 0x55855e69c140 2024-03-23T17:51:36.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.775+0000 7fd3aa98cb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55855e6971e0 con 0x55855e6a7040 2024-03-23T17:51:36.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.775+0000 7fd38fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55855e69b820 0x55855e6ada30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:36.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.775+0000 7fd38f400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55855e69c140 0x55855e6b2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:36.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.775+0000 7fd38f400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55855e69c140 0x55855e6b2890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:48364/0 (socket says 172.21.15.86:48364) 2024-03-23T17:51:36.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.775+0000 7fd38f400700 1 -- 172.21.15.86:0/617165363 learned_addr learned my addr 172.21.15.86:0/617165363 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:36.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.776+0000 7fd394e00700 1 --2- 172.21.15.86:0/617165363 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55855e6a7040 0x55855e697d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:36.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.776+0000 7fd38f400700 1 -- 172.21.15.86:0/617165363 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55855e6a7040 msgr2=0x55855e697d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:36.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.776+0000 7fd38f400700 1 --2- 172.21.15.86:0/617165363 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55855e6a7040 0x55855e697d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:36.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.776+0000 7fd38f400700 1 -- 172.21.15.86:0/617165363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55855e69b820 msgr2=0x55855e6ada30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:36.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.776+0000 7fd38f400700 1 --2- 172.21.15.86:0/617165363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55855e69b820 0x55855e6ada30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:36.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.776+0000 7fd38f400700 1 -- 172.21.15.86:0/617165363 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55855e5b3f20 con 0x55855e69c140 2024-03-23T17:51:36.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.776+0000 7fd38f400700 1 --2- 172.21.15.86:0/617165363 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55855e69c140 0x55855e6b2890 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd37c09ac20 tx=0x7fd37c021c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40086d60d26947ac server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:36.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.776+0000 7fd386000700 1 -- 172.21.15.86:0/617165363 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37c0a3180 con 0x55855e69c140 2024-03-23T17:51:36.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.776+0000 7fd386000700 1 -- 172.21.15.86:0/617165363 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd37c07d050 con 0x55855e69c140 2024-03-23T17:51:36.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.776+0000 7fd386000700 1 -- 172.21.15.86:0/617165363 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37c09b670 con 0x55855e69c140 2024-03-23T17:51:36.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.777+0000 7fd3aa98cb80 1 -- 172.21.15.86:0/617165363 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55855e69c140 msgr2=0x55855e6b2890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:36.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.777+0000 7fd3aa98cb80 1 --2- 172.21.15.86:0/617165363 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55855e69c140 0x55855e6b2890 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd37c09ac20 tx=0x7fd37c021c00 comp rx=0 tx=0).stop 2024-03-23T17:51:36.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.777+0000 7fd3aa98cb80 1 -- 172.21.15.86:0/617165363 shutdown_connections 2024-03-23T17:51:36.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.777+0000 7fd3aa98cb80 1 --2- 172.21.15.86:0/617165363 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55855e6a7040 0x55855e697d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:36.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.777+0000 7fd3aa98cb80 1 --2- 172.21.15.86:0/617165363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55855e69b820 0x55855e6ada30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:36.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.777+0000 7fd3aa98cb80 1 --2- 172.21.15.86:0/617165363 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55855e69c140 0x55855e6b2890 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:36.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.777+0000 7fd3aa98cb80 1 -- 172.21.15.86:0/617165363 >> 172.21.15.86:0/617165363 conn(0x55855e5fb720 msgr2=0x55855e69d720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:36.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.777+0000 7fd3aa98cb80 1 -- 172.21.15.86:0/617165363 shutdown_connections 2024-03-23T17:51:36.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.777+0000 7fd3aa98cb80 1 -- 172.21.15.86:0/617165363 wait complete. 2024-03-23T17:51:36.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.777+0000 7fd3aa98cb80 1 -- start start 2024-03-23T17:51:36.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.778+0000 7fd3aa98cb80 1 Processor -- start 2024-03-23T17:51:36.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.778+0000 7fd3aa98cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55855e69b820 0x55855e6fde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:36.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.778+0000 7fd3aa98cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55855e69c140 0x55855e6fe3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:36.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.778+0000 7fd3aa98cb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55855e6a7040 0x55855e6fe8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:36.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.778+0000 7fd3aa98cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55855e708ef0 con 0x55855e69b820 2024-03-23T17:51:36.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.778+0000 7fd3aa98cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55855e709030 con 0x55855e69c140 2024-03-23T17:51:36.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.778+0000 7fd3aa98cb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55855e7091f0 con 0x55855e6a7040 2024-03-23T17:51:36.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.778+0000 7fd3aa98cb80 10 client.0 _ll_register_callbacks cb 0x55855e5934f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:51:36.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.778+0000 7fd38fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55855e69b820 0x55855e6fde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:36.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.778+0000 7fd38fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55855e69b820 0x55855e6fde70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38384/0 (socket says 172.21.15.86:38384) 2024-03-23T17:51:36.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.778+0000 7fd38f400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55855e69c140 0x55855e6fe3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:36.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.778+0000 7fd38fe00700 1 -- 172.21.15.86:0/909669146 learned_addr learned my addr 172.21.15.86:0/909669146 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:36.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.778+0000 7fd394e00700 1 --2- 172.21.15.86:0/909669146 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55855e6a7040 0x55855e6fe8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:36.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.779+0000 7fd38f400700 1 -- 172.21.15.86:0/909669146 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55855e6a7040 msgr2=0x55855e6fe8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:36.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.779+0000 7fd38f400700 1 --2- 172.21.15.86:0/909669146 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55855e6a7040 0x55855e6fe8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:36.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.779+0000 7fd38f400700 1 -- 172.21.15.86:0/909669146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55855e69b820 msgr2=0x55855e6fde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:36.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.779+0000 7fd38f400700 1 --2- 172.21.15.86:0/909669146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55855e69b820 0x55855e6fde70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:36.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.779+0000 7fd38f400700 1 -- 172.21.15.86:0/909669146 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55855e41bbe0 con 0x55855e69c140 2024-03-23T17:51:36.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.779+0000 7fd38fe00700 1 --2- 172.21.15.86:0/909669146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55855e69b820 0x55855e6fde70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:51:36.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.779+0000 7fd38f400700 1 --2- 172.21.15.86:0/909669146 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55855e69c140 0x55855e6fe3b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x55855e5e68c0 tx=0x7fd37c0047a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=180f9bb87132c72a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:36.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.779+0000 7fd37a000700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T17:51:36.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.779+0000 7fd37a000700 1 -- 172.21.15.86:0/909669146 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37c0a3180 con 0x55855e69c140 2024-03-23T17:51:36.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.779+0000 7fd37a000700 1 -- 172.21.15.86:0/909669146 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fd37c09b670 con 0x55855e69c140 2024-03-23T17:51:36.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.779+0000 7fd3aa98cb80 1 -- 172.21.15.86:0/909669146 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd37c09aab0 con 0x55855e69c140 2024-03-23T17:51:36.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.779+0000 7fd37a000700 1 -- 172.21.15.86:0/909669146 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd37c0a10b0 con 0x55855e69c140 2024-03-23T17:51:36.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.779+0000 7fd3aa98cb80 10 client.25844 fetch_fsmap learned FSMap version 28 2024-03-23T17:51:36.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.779+0000 7fd3aa98cb80 1 -- 172.21.15.86:0/909669146 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55855e473d50 con 0x55855e69c140 2024-03-23T17:51:36.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.779+0000 7fd37a000700 1 -- 172.21.15.86:0/909669146 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd37c09aab0 con 0x55855e69c140 2024-03-23T17:51:36.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.780+0000 7fd37a000700 1 -- 172.21.15.86:0/909669146 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 28) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd37c099ed0 con 0x55855e69c140 2024-03-23T17:51:36.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.780+0000 7fd3aa98cb80 10 client.25844 fetch_fsmap finished waiting for FSMap version 28 2024-03-23T17:51:36.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.780+0000 7fd3aa98cb80 10 client.25844 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T17:51:36.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.780+0000 7fd3aa98cb80 20 client.25844 populate_metadata read hostname 'smithi086' 2024-03-23T17:51:36.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.780+0000 7fd3aa98cb80 1 --2- 172.21.15.86:0/909669146 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55855e70a520 0x55855e70c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:36.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.780+0000 7fd3aa98cb80 4 client.25844 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-23T17:51:36.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.780+0000 7fd3aa98cb80 1 -- 172.21.15.86:0/909669146 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55855e43cd70 con 0x55855e70a520 2024-03-23T17:51:36.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.780+0000 7fd38fe00700 1 --2- 172.21.15.86:0/909669146 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55855e70a520 0x55855e70c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:36.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.780+0000 7fd38fe00700 1 --2- 172.21.15.86:0/909669146 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55855e70a520 0x55855e70c910 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=918af2b4ac5ee682 server_cookie=e2431fab409f62a7 in_seq=0 out_seq=0 2024-03-23T17:51:36.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.780+0000 7fd37a000700 10 client.25844 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:51:36.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.782+0000 7fd37a000700 1 -- 172.21.15.86:0/909669146 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55855e43cd70 con 0x55855e70a520 2024-03-23T17:51:36.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.782+0000 7fd37a000700 10 client.25844 handle_command_reply: tid=0 2024-03-23T17:51:36.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.782+0000 7fd3aa98cb80 1 client.25844 shutdown 2024-03-23T17:51:36.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.782+0000 7fd3aa98cb80 1 -- 172.21.15.86:0/909669146 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55855e69c140 msgr2=0x55855e6fe3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:36.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.782+0000 7fd3aa98cb80 1 --2- 172.21.15.86:0/909669146 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55855e69c140 0x55855e6fe3b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x55855e5e68c0 tx=0x7fd37c0047a0 comp rx=0 tx=0).stop 2024-03-23T17:51:36.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.782+0000 7fd3aa98cb80 1 -- 172.21.15.86:0/909669146 shutdown_connections 2024-03-23T17:51:36.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.782+0000 7fd3aa98cb80 1 --2- 172.21.15.86:0/909669146 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55855e70a520 0x55855e70c910 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:36.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.782+0000 7fd3aa98cb80 1 --2- 172.21.15.86:0/909669146 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55855e6a7040 0x55855e6fe8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:36.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.782+0000 7fd3aa98cb80 1 --2- 172.21.15.86:0/909669146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55855e69b820 0x55855e6fde70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:36.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.782+0000 7fd37a000700 0 client.25844 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:51:36.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.782+0000 7fd3aa98cb80 1 --2- 172.21.15.86:0/909669146 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55855e69c140 0x55855e6fe3b0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:36.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.782+0000 7fd3aa98cb80 1 -- 172.21.15.86:0/909669146 >> 172.21.15.86:0/909669146 conn(0x55855e5fb720 msgr2=0x55855e69dfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:36.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.782+0000 7fd3aa98cb80 1 -- 172.21.15.86:0/909669146 shutdown_connections 2024-03-23T17:51:36.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.782+0000 7fd3aa98cb80 1 -- 172.21.15.86:0/909669146 wait complete. 2024-03-23T17:51:36.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.783+0000 7fd3aa98cb80 20 client.25844 trim_cache size 0 max 16384 2024-03-23T17:51:36.786 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-23T17:51:36.787 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-03-23T17:51:36.787 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "5a561f59-fafa-4893-8029-1ba8b6f23a3b", 2024-03-23T17:51:36.787 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-03-23T17:51:36.787 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-23T17:51:36.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.783+0000 7fd395800700 1 -- 172.21.15.86:0/3922696237 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fd3740a1430 msgr2=0x7fd3740a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:36.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.783+0000 7fd395800700 1 --2- 172.21.15.86:0/3922696237 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fd3740a1430 0x7fd3740a38f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd380009d70 tx=0x7fd380009260 comp rx=0 tx=0).stop 2024-03-23T17:51:36.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.783+0000 7fd395800700 1 -- 172.21.15.86:0/3922696237 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd390102650 msgr2=0x7fd3901997e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:36.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.783+0000 7fd395800700 1 --2- 172.21.15.86:0/3922696237 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd390102650 0x7fd3901997e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd37c000c00 tx=0x7fd37c004730 comp rx=0 tx=0).stop 2024-03-23T17:51:36.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.783+0000 7fd395800700 1 -- 172.21.15.86:0/3922696237 shutdown_connections 2024-03-23T17:51:36.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.783+0000 7fd395800700 1 --2- 172.21.15.86:0/3922696237 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd39010b150 0x7fd39019db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:36.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.783+0000 7fd395800700 1 --2- 172.21.15.86:0/3922696237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd390101d30 0x7fd3901992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:36.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.783+0000 7fd395800700 1 --2- 172.21.15.86:0/3922696237 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fd3740a1430 0x7fd3740a38f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:36.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.783+0000 7fd395800700 1 --2- 172.21.15.86:0/3922696237 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd390102650 0x7fd3901997e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:36.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.783+0000 7fd395800700 1 -- 172.21.15.86:0/3922696237 >> 172.21.15.86:0/3922696237 conn(0x7fd3900fb570 msgr2=0x7fd390100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:36.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.783+0000 7fd395800700 1 -- 172.21.15.86:0/3922696237 shutdown_connections 2024-03-23T17:51:36.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:36.784+0000 7fd395800700 1 -- 172.21.15.86:0/3922696237 wait complete. 2024-03-23T17:51:36.803 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-23T17:51:36.803 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-23T17:51:36.877 INFO:tasks.workunit.client.0.smithi086.stdout: 1 3339 35.37 MB/sec warmup 1 sec latency 7.891 ms 2024-03-23T17:51:37.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.011+0000 7f1286800700 1 -- 172.21.15.86:0/2718323844 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1280073160 msgr2=0x7f1280073540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:37.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.011+0000 7f1286800700 1 --2- 172.21.15.86:0/2718323844 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1280073160 0x7f1280073540 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f12700098f0 tx=0x7f1270009c00 comp rx=0 tx=0).stop 2024-03-23T17:51:37.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.011+0000 7f1286800700 1 -- 172.21.15.86:0/2718323844 shutdown_connections 2024-03-23T17:51:37.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.011+0000 7f1286800700 1 --2- 172.21.15.86:0/2718323844 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f128010d5c0 0x7f128010f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:37.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.011+0000 7f1286800700 1 --2- 172.21.15.86:0/2718323844 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1280073160 0x7f1280073540 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:37.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.011+0000 7f1286800700 1 --2- 172.21.15.86:0/2718323844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1280073a80 0x7f128010d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:37.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.011+0000 7f1286800700 1 -- 172.21.15.86:0/2718323844 >> 172.21.15.86:0/2718323844 conn(0x7f12800fc7f0 msgr2=0x7f12800fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:37.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.011+0000 7f1286800700 1 -- 172.21.15.86:0/2718323844 shutdown_connections 2024-03-23T17:51:37.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.011+0000 7f1286800700 1 -- 172.21.15.86:0/2718323844 wait complete. 2024-03-23T17:51:37.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.012+0000 7f1286800700 1 Processor -- start 2024-03-23T17:51:37.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.012+0000 7f1286800700 1 -- start start 2024-03-23T17:51:37.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.012+0000 7f1286800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1280073160 0x7f1280199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:37.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.012+0000 7f1286800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1280073a80 0x7f12801999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:37.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.013+0000 7f1286800700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f128010d5c0 0x7f128019dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:37.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.013+0000 7f1286800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f128019e380 con 0x7f1280073a80 2024-03-23T17:51:37.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.013+0000 7f1286800700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f128019e4f0 con 0x7f1280073160 2024-03-23T17:51:37.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.013+0000 7f1286800700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f128019e6b0 con 0x7f128010d5c0 2024-03-23T17:51:37.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.013+0000 7f127f400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1280073160 0x7f1280199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:37.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.013+0000 7f127f400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1280073160 0x7f1280199460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:48394/0 (socket says 172.21.15.86:48394) 2024-03-23T17:51:37.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.013+0000 7f127f400700 1 -- 172.21.15.86:0/3622426937 learned_addr learned my addr 172.21.15.86:0/3622426937 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:37.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.013+0000 7f127fe00700 1 --2- 172.21.15.86:0/3622426937 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f128010d5c0 0x7f128019dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:37.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.013+0000 7f127ea00700 1 --2- 172.21.15.86:0/3622426937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1280073a80 0x7f12801999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:37.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.013+0000 7f127f400700 1 -- 172.21.15.86:0/3622426937 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f128010d5c0 msgr2=0x7f128019dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:37.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.013+0000 7f127f400700 1 --2- 172.21.15.86:0/3622426937 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f128010d5c0 0x7f128019dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:37.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.013+0000 7f127f400700 1 -- 172.21.15.86:0/3622426937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1280073a80 msgr2=0x7f12801999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:37.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.013+0000 7f127f400700 1 --2- 172.21.15.86:0/3622426937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1280073a80 0x7f12801999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:37.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.013+0000 7f127f400700 1 -- 172.21.15.86:0/3622426937 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1278009d00 con 0x7f1280073160 2024-03-23T17:51:37.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.013+0000 7f127f400700 1 --2- 172.21.15.86:0/3622426937 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1280073160 0x7f1280199460 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1270005950 tx=0x7f127000fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:37.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.014+0000 7f1277e00700 1 -- 172.21.15.86:0/3622426937 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f127001c070 con 0x7f1280073160 2024-03-23T17:51:37.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.014+0000 7f1277e00700 1 -- 172.21.15.86:0/3622426937 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f127000bb20 con 0x7f1280073160 2024-03-23T17:51:37.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.014+0000 7f1286800700 1 -- 172.21.15.86:0/3622426937 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1270009520 con 0x7f1280073160 2024-03-23T17:51:37.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.014+0000 7f1277e00700 1 -- 172.21.15.86:0/3622426937 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12700038e0 con 0x7f1280073160 2024-03-23T17:51:37.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.014+0000 7f1286800700 1 -- 172.21.15.86:0/3622426937 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12801a5ff0 con 0x7f1280073160 2024-03-23T17:51:37.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.017+0000 7f1277e00700 1 -- 172.21.15.86:0/3622426937 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 25) v1 ==== 136098+0+0 (secure 0 0 0) 0x7f1270020030 con 0x7f1280073160 2024-03-23T17:51:37.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.018+0000 7f129a284b80 1 Processor -- start 2024-03-23T17:51:37.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.018+0000 7f129a284b80 1 -- start start 2024-03-23T17:51:37.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.018+0000 7f129a284b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558fcbae45d0 0x558fcbae49b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:37.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.018+0000 7f129a284b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fcbae4ef0 0x558fcbaef8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:37.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.018+0000 7f129a284b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558fcbaefdf0 0x558fcbaf21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:37.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.018+0000 7f129a284b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fcba3d990 con 0x558fcbae4ef0 2024-03-23T17:51:37.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.018+0000 7f129a284b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fcba3b350 con 0x558fcbae45d0 2024-03-23T17:51:37.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.018+0000 7f129a284b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fcba36c40 con 0x558fcbaefdf0 2024-03-23T17:51:37.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.019+0000 7f127fe00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558fcbae45d0 0x558fcbae49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:37.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.019+0000 7f127fe00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558fcbae45d0 0x558fcbae49b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:48402/0 (socket says 172.21.15.86:48402) 2024-03-23T17:51:37.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.019+0000 7f127fe00700 1 -- 172.21.15.86:0/3252152461 learned_addr learned my addr 172.21.15.86:0/3252152461 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:37.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.019+0000 7f127f400700 1 --2- 172.21.15.86:0/3252152461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fcbae4ef0 0x558fcbaef8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:37.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.019+0000 7f127ea00700 1 --2- 172.21.15.86:0/3252152461 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558fcbaefdf0 0x558fcbaf21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:37.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.019+0000 7f1277e00700 1 --2- 172.21.15.86:0/3622426937 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f12640a13b0 0x7f12640a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:37.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.019+0000 7f127fe00700 1 -- 172.21.15.86:0/3252152461 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558fcbaefdf0 msgr2=0x558fcbaf21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:37.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.019+0000 7f127fe00700 1 --2- 172.21.15.86:0/3252152461 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558fcbaefdf0 0x558fcbaf21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:37.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.019+0000 7f127fe00700 1 -- 172.21.15.86:0/3252152461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fcbae4ef0 msgr2=0x558fcbaef8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:37.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.019+0000 7f127fe00700 1 --2- 172.21.15.86:0/3252152461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fcbae4ef0 0x558fcbaef8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:37.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.019+0000 7f127fe00700 1 -- 172.21.15.86:0/3252152461 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558fcb8b83b0 con 0x558fcbae45d0 2024-03-23T17:51:37.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.019+0000 7f127f400700 1 --2- 172.21.15.86:0/3252152461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fcbae4ef0 0x558fcbaef8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:51:37.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.019+0000 7f1277e00700 1 -- 172.21.15.86:0/3622426937 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f12700cddf0 con 0x7f1280073160 2024-03-23T17:51:37.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.019+0000 7f127fe00700 1 --2- 172.21.15.86:0/3252152461 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558fcbae45d0 0x558fcbae49b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1278009530 tx=0x7f12780099e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=438a3695bfe39397 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:37.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.020+0000 7f127fe00700 1 --2- 172.21.15.86:0/3622426937 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f12640a13b0 0x7f12640a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:37.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.020+0000 7f1274c00700 1 -- 172.21.15.86:0/3252152461 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f127800e030 con 0x558fcbae45d0 2024-03-23T17:51:37.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.020+0000 7f1274c00700 1 -- 172.21.15.86:0/3252152461 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f127800f040 con 0x558fcbae45d0 2024-03-23T17:51:37.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.020+0000 7f1274c00700 1 -- 172.21.15.86:0/3252152461 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1278015de0 con 0x558fcbae45d0 2024-03-23T17:51:37.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.020+0000 7f129a284b80 1 -- 172.21.15.86:0/3252152461 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558fcbae45d0 msgr2=0x558fcbae49b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:37.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.020+0000 7f127fe00700 1 --2- 172.21.15.86:0/3622426937 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f12640a13b0 0x7f12640a3870 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f127800e1d0 tx=0x7f12780203f0 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:37.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.020+0000 7f129a284b80 1 --2- 172.21.15.86:0/3252152461 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558fcbae45d0 0x558fcbae49b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1278009530 tx=0x7f12780099e0 comp rx=0 tx=0).stop 2024-03-23T17:51:37.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.021+0000 7f129a284b80 1 -- 172.21.15.86:0/3252152461 shutdown_connections 2024-03-23T17:51:37.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.021+0000 7f129a284b80 1 --2- 172.21.15.86:0/3252152461 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558fcbaefdf0 0x558fcbaf21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:37.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.021+0000 7f129a284b80 1 --2- 172.21.15.86:0/3252152461 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558fcbae45d0 0x558fcbae49b0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:37.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.021+0000 7f129a284b80 1 --2- 172.21.15.86:0/3252152461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fcbae4ef0 0x558fcbaef8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:37.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.021+0000 7f129a284b80 1 -- 172.21.15.86:0/3252152461 >> 172.21.15.86:0/3252152461 conn(0x558fcba446a0 msgr2=0x558fcbae0a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:37.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.021+0000 7f129a284b80 1 -- 172.21.15.86:0/3252152461 shutdown_connections 2024-03-23T17:51:37.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.021+0000 7f129a284b80 1 -- 172.21.15.86:0/3252152461 wait complete. 2024-03-23T17:51:37.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.021+0000 7f129a284b80 1 -- start start 2024-03-23T17:51:37.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.021+0000 7f129a284b80 1 Processor -- start 2024-03-23T17:51:37.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.022+0000 7f129a284b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558fcbae45d0 0x558fcbaef770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:37.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.022+0000 7f129a284b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fcbae4ef0 0x558fcbaecb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:37.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.022+0000 7f129a284b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558fcbaefdf0 0x558fcbaed0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:37.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.022+0000 7f129a284b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fcb9d2d80 con 0x558fcbae4ef0 2024-03-23T17:51:37.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.022+0000 7f129a284b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fcb9d2a20 con 0x558fcbae45d0 2024-03-23T17:51:37.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.022+0000 7f129a284b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fcb8ba6a0 con 0x558fcbaefdf0 2024-03-23T17:51:37.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.022+0000 7f129a284b80 10 client.0 _ll_register_callbacks cb 0x558fcb9dc470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:51:37.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.022+0000 7f127f400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558fcbae45d0 0x558fcbaef770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:37.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.022+0000 7f127f400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558fcbae45d0 0x558fcbaef770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:48410/0 (socket says 172.21.15.86:48410) 2024-03-23T17:51:37.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.022+0000 7f127f400700 1 -- 172.21.15.86:0/1310103727 learned_addr learned my addr 172.21.15.86:0/1310103727 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:37.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.022+0000 7f127ea00700 1 --2- 172.21.15.86:0/1310103727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fcbae4ef0 0x558fcbaecb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:37.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.022+0000 7f127fe00700 1 --2- 172.21.15.86:0/1310103727 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558fcbaefdf0 0x558fcbaed0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:37.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.022+0000 7f127f400700 1 -- 172.21.15.86:0/1310103727 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558fcbaefdf0 msgr2=0x558fcbaed0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:37.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.022+0000 7f127f400700 1 --2- 172.21.15.86:0/1310103727 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558fcbaefdf0 0x558fcbaed0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:37.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.022+0000 7f127f400700 1 -- 172.21.15.86:0/1310103727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fcbae4ef0 msgr2=0x558fcbaecb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:37.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.022+0000 7f127f400700 1 --2- 172.21.15.86:0/1310103727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fcbae4ef0 0x558fcbaecb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:37.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.022+0000 7f127f400700 1 -- 172.21.15.86:0/1310103727 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12780091e0 con 0x558fcbae45d0 2024-03-23T17:51:37.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.023+0000 7f127f400700 1 --2- 172.21.15.86:0/1310103727 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558fcbae45d0 0x558fcbaef770 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1270009fd0 tx=0x7f1270005570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e76c8b8370de6bd server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:37.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.023+0000 7f1269600700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T17:51:37.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.023+0000 7f1269600700 1 -- 172.21.15.86:0/1310103727 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12700053d0 con 0x558fcbae45d0 2024-03-23T17:51:37.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.023+0000 7f1269600700 1 -- 172.21.15.86:0/1310103727 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f127009b490 con 0x558fcbae45d0 2024-03-23T17:51:37.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.023+0000 7f129a284b80 1 -- 172.21.15.86:0/1310103727 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558fcb9f7380 con 0x558fcbae45d0 2024-03-23T17:51:37.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.023+0000 7f1269600700 1 -- 172.21.15.86:0/1310103727 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f127009ad90 con 0x558fcbae45d0 2024-03-23T17:51:37.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.023+0000 7f129a284b80 10 client.25856 fetch_fsmap learned FSMap version 28 2024-03-23T17:51:37.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.023+0000 7f129a284b80 1 -- 172.21.15.86:0/1310103727 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x558fcb864b60 con 0x558fcbae45d0 2024-03-23T17:51:37.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.023+0000 7f1269600700 1 -- 172.21.15.86:0/1310103727 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558fcb9f7380 con 0x558fcbae45d0 2024-03-23T17:51:37.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.023+0000 7f1269600700 1 -- 172.21.15.86:0/1310103727 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 28) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f127009aff0 con 0x558fcbae45d0 2024-03-23T17:51:37.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.023+0000 7f129a284b80 10 client.25856 fetch_fsmap finished waiting for FSMap version 28 2024-03-23T17:51:37.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.023+0000 7f129a284b80 10 client.25856 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T17:51:37.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.024+0000 7f129a284b80 20 client.25856 populate_metadata read hostname 'smithi086' 2024-03-23T17:51:37.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.024+0000 7f129a284b80 1 --2- 172.21.15.86:0/1310103727 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x558fcbaff540 0x558fcbb01960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:37.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.024+0000 7f129a284b80 4 client.25856 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:51:37.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.024+0000 7f129a284b80 1 -- 172.21.15.86:0/1310103727 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558fcba2f700 con 0x558fcbaff540 2024-03-23T17:51:37.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.024+0000 7f127ea00700 1 --2- 172.21.15.86:0/1310103727 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x558fcbaff540 0x558fcbb01960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:37.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.024+0000 7f127ea00700 1 --2- 172.21.15.86:0/1310103727 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x558fcbaff540 0x558fcbb01960 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80cb925166ba0e11 server_cookie=a652f7de08d4f7f8 in_seq=0 out_seq=0 2024-03-23T17:51:37.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.024+0000 7f1269600700 10 client.25856 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:51:37.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.025+0000 7f1269600700 1 -- 172.21.15.86:0/1310103727 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x558fcba2f700 con 0x558fcbaff540 2024-03-23T17:51:37.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.025+0000 7f1269600700 10 client.25856 handle_command_reply: tid=0 2024-03-23T17:51:37.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.025+0000 7f129a284b80 1 client.25856 shutdown 2024-03-23T17:51:37.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.025+0000 7f129a284b80 1 -- 172.21.15.86:0/1310103727 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558fcbae45d0 msgr2=0x558fcbaef770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:37.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.025+0000 7f129a284b80 1 --2- 172.21.15.86:0/1310103727 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558fcbae45d0 0x558fcbaef770 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1270009fd0 tx=0x7f1270005570 comp rx=0 tx=0).stop 2024-03-23T17:51:37.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.025+0000 7f129a284b80 1 -- 172.21.15.86:0/1310103727 shutdown_connections 2024-03-23T17:51:37.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.025+0000 7f129a284b80 1 --2- 172.21.15.86:0/1310103727 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x558fcbaff540 0x558fcbb01960 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:37.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.025+0000 7f129a284b80 1 --2- 172.21.15.86:0/1310103727 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558fcbaefdf0 0x558fcbaed0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:37.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.025+0000 7f1269600700 0 client.25856 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:51:37.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.025+0000 7f129a284b80 1 --2- 172.21.15.86:0/1310103727 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558fcbae45d0 0x558fcbaef770 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:37.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.025+0000 7f129a284b80 1 --2- 172.21.15.86:0/1310103727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fcbae4ef0 0x558fcbaecb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:37.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.025+0000 7f129a284b80 1 -- 172.21.15.86:0/1310103727 >> 172.21.15.86:0/1310103727 conn(0x558fcba446a0 msgr2=0x558fcbae66c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:37.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.025+0000 7f129a284b80 1 -- 172.21.15.86:0/1310103727 shutdown_connections 2024-03-23T17:51:37.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.025+0000 7f129a284b80 1 -- 172.21.15.86:0/1310103727 wait complete. 2024-03-23T17:51:37.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.026+0000 7f129a284b80 20 client.25856 trim_cache size 0 max 16384 2024-03-23T17:51:37.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.043+0000 7f129a284b80 1 Processor -- start 2024-03-23T17:51:37.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.043+0000 7f129a284b80 1 -- start start 2024-03-23T17:51:37.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.043+0000 7f129a284b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558fcbae45d0 0x558fcbb062c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:37.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.043+0000 7f129a284b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fcbae4ef0 0x558fcbaf6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:37.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.043+0000 7f129a284b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558fcbaefdf0 0x558fcbaef2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:37.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.043+0000 7f129a284b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fcbaee0a0 con 0x558fcbae4ef0 2024-03-23T17:51:37.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.043+0000 7f129a284b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fcbae5880 con 0x558fcbaefdf0 2024-03-23T17:51:37.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.043+0000 7f129a284b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fcbae21e0 con 0x558fcbae45d0 2024-03-23T17:51:37.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.043+0000 7f127fe00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558fcbaefdf0 0x558fcbaef2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:37.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.043+0000 7f127ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fcbae4ef0 0x558fcbaf6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:37.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.043+0000 7f127fe00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558fcbaefdf0 0x558fcbaef2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:48420/0 (socket says 172.21.15.86:48420) 2024-03-23T17:51:37.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.043+0000 7f127fe00700 1 -- 172.21.15.86:0/2657593070 learned_addr learned my addr 172.21.15.86:0/2657593070 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:37.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.043+0000 7f127f400700 1 --2- 172.21.15.86:0/2657593070 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558fcbae45d0 0x558fcbb062c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:37.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.044+0000 7f127fe00700 1 -- 172.21.15.86:0/2657593070 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558fcbae45d0 msgr2=0x558fcbb062c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:37.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.044+0000 7f127fe00700 1 --2- 172.21.15.86:0/2657593070 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558fcbae45d0 0x558fcbb062c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:37.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.044+0000 7f127fe00700 1 -- 172.21.15.86:0/2657593070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fcbae4ef0 msgr2=0x558fcbaf6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:37.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.044+0000 7f127fe00700 1 --2- 172.21.15.86:0/2657593070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fcbae4ef0 0x558fcbaf6aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:37.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.044+0000 7f127fe00700 1 -- 172.21.15.86:0/2657593070 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558fcb9fcea0 con 0x558fcbaefdf0 2024-03-23T17:51:37.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.044+0000 7f127fe00700 1 --2- 172.21.15.86:0/2657593070 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558fcbaefdf0 0x558fcbaef2a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1278007b80 tx=0x7f1278025000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dece833194d7f77d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:37.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.045+0000 7f1275600700 1 -- 172.21.15.86:0/2657593070 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1278020600 con 0x558fcbaefdf0 2024-03-23T17:51:37.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.045+0000 7f1275600700 1 -- 172.21.15.86:0/2657593070 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1278015410 con 0x558fcbaefdf0 2024-03-23T17:51:37.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.045+0000 7f1275600700 1 -- 172.21.15.86:0/2657593070 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12780177c0 con 0x558fcbaefdf0 2024-03-23T17:51:37.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.045+0000 7f129a284b80 1 -- 172.21.15.86:0/2657593070 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558fcbaefdf0 msgr2=0x558fcbaef2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:37.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.045+0000 7f129a284b80 1 --2- 172.21.15.86:0/2657593070 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558fcbaefdf0 0x558fcbaef2a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1278007b80 tx=0x7f1278025000 comp rx=0 tx=0).stop 2024-03-23T17:51:37.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.045+0000 7f129a284b80 1 -- 172.21.15.86:0/2657593070 shutdown_connections 2024-03-23T17:51:37.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.045+0000 7f129a284b80 1 --2- 172.21.15.86:0/2657593070 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558fcbaefdf0 0x558fcbaef2a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:37.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.045+0000 7f129a284b80 1 --2- 172.21.15.86:0/2657593070 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558fcbae45d0 0x558fcbb062c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:37.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.045+0000 7f129a284b80 1 --2- 172.21.15.86:0/2657593070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fcbae4ef0 0x558fcbaf6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:37.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.045+0000 7f129a284b80 1 -- 172.21.15.86:0/2657593070 >> 172.21.15.86:0/2657593070 conn(0x558fcba446a0 msgr2=0x558fcbaeb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:37.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.045+0000 7f129a284b80 1 -- 172.21.15.86:0/2657593070 shutdown_connections 2024-03-23T17:51:37.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.045+0000 7f129a284b80 1 -- 172.21.15.86:0/2657593070 wait complete. 2024-03-23T17:51:37.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.046+0000 7f129a284b80 1 -- start start 2024-03-23T17:51:37.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.046+0000 7f129a284b80 1 Processor -- start 2024-03-23T17:51:37.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.047+0000 7f129a284b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558fcbae45d0 0x558fcbaf50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:37.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.047+0000 7f129a284b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fcbae4ef0 0x558fcbaf5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:37.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.047+0000 7f129a284b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558fcbaefdf0 0x558fcbaf5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:37.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.047+0000 7f129a284b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fcbb51070 con 0x558fcbae4ef0 2024-03-23T17:51:37.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.047+0000 7f129a284b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fcbb511b0 con 0x558fcbae45d0 2024-03-23T17:51:37.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.047+0000 7f129a284b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fcbb51370 con 0x558fcbaefdf0 2024-03-23T17:51:37.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.047+0000 7f129a284b80 10 client.0 _ll_register_callbacks cb 0x558fcb9dc470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:51:37.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.047+0000 7f127f400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558fcbae45d0 0x558fcbaf50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:37.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.047+0000 7f127f400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558fcbae45d0 0x558fcbaf50e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:48432/0 (socket says 172.21.15.86:48432) 2024-03-23T17:51:37.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.047+0000 7f127f400700 1 -- 172.21.15.86:0/1355921687 learned_addr learned my addr 172.21.15.86:0/1355921687 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:37.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.047+0000 7f127ea00700 1 --2- 172.21.15.86:0/1355921687 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fcbae4ef0 0x558fcbaf5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:37.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.047+0000 7f127fe00700 1 --2- 172.21.15.86:0/1355921687 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558fcbaefdf0 0x558fcbaf5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:37.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.047+0000 7f127f400700 1 -- 172.21.15.86:0/1355921687 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558fcbaefdf0 msgr2=0x558fcbaf5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:37.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.047+0000 7f127f400700 1 --2- 172.21.15.86:0/1355921687 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558fcbaefdf0 0x558fcbaf5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:37.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.047+0000 7f127f400700 1 -- 172.21.15.86:0/1355921687 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fcbae4ef0 msgr2=0x558fcbaf5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:37.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.047+0000 7f127f400700 1 --2- 172.21.15.86:0/1355921687 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fcbae4ef0 0x558fcbaf5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:37.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.047+0000 7f127f400700 1 -- 172.21.15.86:0/1355921687 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1278028050 con 0x558fcbae45d0 2024-03-23T17:51:37.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.047+0000 7f127ea00700 1 --2- 172.21.15.86:0/1355921687 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fcbae4ef0 0x558fcbaf5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:51:37.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.047+0000 7f127fe00700 1 --2- 172.21.15.86:0/1355921687 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558fcbaefdf0 0x558fcbaf5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:51:37.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.048+0000 7f127f400700 1 --2- 172.21.15.86:0/1355921687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558fcbae45d0 0x558fcbaf50e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1270097800 tx=0x7f1270009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3301f3763e26bb60 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:37.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.048+0000 7f1269600700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T17:51:37.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.048+0000 7f1269600700 1 -- 172.21.15.86:0/1355921687 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12700217c0 con 0x558fcbae45d0 2024-03-23T17:51:37.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.048+0000 7f1269600700 1 -- 172.21.15.86:0/1355921687 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f127009b490 con 0x558fcbae45d0 2024-03-23T17:51:37.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.048+0000 7f129a284b80 1 -- 172.21.15.86:0/1355921687 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f127800f040 con 0x558fcbae45d0 2024-03-23T17:51:37.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.048+0000 7f1269600700 1 -- 172.21.15.86:0/1355921687 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f12700053d0 con 0x558fcbae45d0 2024-03-23T17:51:37.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.048+0000 7f129a284b80 10 client.25862 fetch_fsmap learned FSMap version 28 2024-03-23T17:51:37.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.048+0000 7f129a284b80 1 -- 172.21.15.86:0/1355921687 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x558fcb8bccd0 con 0x558fcbae45d0 2024-03-23T17:51:37.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.048+0000 7f1269600700 1 -- 172.21.15.86:0/1355921687 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1270066080 con 0x558fcbae45d0 2024-03-23T17:51:37.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.048+0000 7f1269600700 1 -- 172.21.15.86:0/1355921687 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 28) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f12700aebd0 con 0x558fcbae45d0 2024-03-23T17:51:37.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.048+0000 7f129a284b80 10 client.25862 fetch_fsmap finished waiting for FSMap version 28 2024-03-23T17:51:37.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.048+0000 7f129a284b80 10 client.25862 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T17:51:37.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.048+0000 7f129a284b80 20 client.25862 populate_metadata read hostname 'smithi086' 2024-03-23T17:51:37.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.048+0000 7f129a284b80 1 --2- 172.21.15.86:0/1355921687 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x558fcbb52670 0x558fcbb54a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:37.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.049+0000 7f129a284b80 4 client.25862 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-23T17:51:37.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.049+0000 7f129a284b80 1 -- 172.21.15.86:0/1355921687 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558fcb885cf0 con 0x558fcbb52670 2024-03-23T17:51:37.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.049+0000 7f127ea00700 1 --2- 172.21.15.86:0/1355921687 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x558fcbb52670 0x558fcbb54a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:37.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.049+0000 7f127ea00700 1 --2- 172.21.15.86:0/1355921687 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x558fcbb52670 0x558fcbb54a60 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81d4b4b15ff51ee1 server_cookie=6b1adbb13430ca45 in_seq=0 out_seq=0 2024-03-23T17:51:37.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.049+0000 7f1269600700 10 client.25862 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:51:37.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.050+0000 7f1269600700 1 -- 172.21.15.86:0/1355921687 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x558fcb885cf0 con 0x558fcbb52670 2024-03-23T17:51:37.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.050+0000 7f1269600700 10 client.25862 handle_command_reply: tid=0 2024-03-23T17:51:37.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.050+0000 7f129a284b80 1 client.25862 shutdown 2024-03-23T17:51:37.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.050+0000 7f129a284b80 1 -- 172.21.15.86:0/1355921687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558fcbae45d0 msgr2=0x558fcbaf50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:37.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.050+0000 7f129a284b80 1 --2- 172.21.15.86:0/1355921687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558fcbae45d0 0x558fcbaf50e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1270097800 tx=0x7f1270009c00 comp rx=0 tx=0).stop 2024-03-23T17:51:37.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.050+0000 7f129a284b80 1 -- 172.21.15.86:0/1355921687 shutdown_connections 2024-03-23T17:51:37.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.050+0000 7f129a284b80 1 --2- 172.21.15.86:0/1355921687 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x558fcbb52670 0x558fcbb54a60 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:37.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.050+0000 7f129a284b80 1 --2- 172.21.15.86:0/1355921687 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558fcbaefdf0 0x558fcbaf5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:37.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.050+0000 7f129a284b80 1 --2- 172.21.15.86:0/1355921687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558fcbae45d0 0x558fcbaf50e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:37.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.050+0000 7f129a284b80 1 --2- 172.21.15.86:0/1355921687 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fcbae4ef0 0x558fcbaf5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:37.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.050+0000 7f1269600700 0 client.25862 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:51:37.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.050+0000 7f129a284b80 1 -- 172.21.15.86:0/1355921687 >> 172.21.15.86:0/1355921687 conn(0x558fcba446a0 msgr2=0x558fcbaec390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:37.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.050+0000 7f129a284b80 1 -- 172.21.15.86:0/1355921687 shutdown_connections 2024-03-23T17:51:37.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.050+0000 7f129a284b80 1 -- 172.21.15.86:0/1355921687 wait complete. 2024-03-23T17:51:37.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.051+0000 7f129a284b80 20 client.25862 trim_cache size 0 max 16384 2024-03-23T17:51:37.055 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-23T17:51:37.055 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-23T17:51:37.055 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-03-23T17:51:37.055 INFO:teuthology.orchestra.run.smithi086.stdout: "5a561f59-fafa-4893-8029-1ba8b6f23a3b": { 2024-03-23T17:51:37.055 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-03-23T17:51:37.055 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "5a561f59-fafa-4893-8029-1ba8b6f23a3b", 2024-03-23T17:51:37.055 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-03-23T17:51:37.055 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T17:51:37.055 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T17:51:37.055 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-23T17:51:37.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.051+0000 7f1286800700 1 -- 172.21.15.86:0/3622426937 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f12640a13b0 msgr2=0x7f12640a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:37.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.051+0000 7f1286800700 1 --2- 172.21.15.86:0/3622426937 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f12640a13b0 0x7f12640a3870 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f127800e1d0 tx=0x7f12780203f0 comp rx=0 tx=0).stop 2024-03-23T17:51:37.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.051+0000 7f1286800700 1 -- 172.21.15.86:0/3622426937 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1280073160 msgr2=0x7f1280199460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:37.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.051+0000 7f1286800700 1 --2- 172.21.15.86:0/3622426937 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1280073160 0x7f1280199460 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1270005950 tx=0x7f127000fa00 comp rx=0 tx=0).stop 2024-03-23T17:51:37.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.051+0000 7f1286800700 1 -- 172.21.15.86:0/3622426937 shutdown_connections 2024-03-23T17:51:37.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.051+0000 7f1286800700 1 --2- 172.21.15.86:0/3622426937 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f12640a13b0 0x7f12640a3870 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:37.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.051+0000 7f1286800700 1 --2- 172.21.15.86:0/3622426937 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f128010d5c0 0x7f128019dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:37.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.051+0000 7f1286800700 1 --2- 172.21.15.86:0/3622426937 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1280073160 0x7f1280199460 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:37.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.051+0000 7f1286800700 1 --2- 172.21.15.86:0/3622426937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1280073a80 0x7f12801999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:37.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.051+0000 7f1286800700 1 -- 172.21.15.86:0/3622426937 >> 172.21.15.86:0/3622426937 conn(0x7f12800fc7f0 msgr2=0x7f12800fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:37.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.051+0000 7f1286800700 1 -- 172.21.15.86:0/3622426937 shutdown_connections 2024-03-23T17:51:37.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:51:37.051+0000 7f1286800700 1 -- 172.21.15.86:0/3622426937 wait complete. 2024-03-23T17:51:37.069 INFO:tasks.cephfs.filesystem:scrub status for tag:5a561f59-fafa-4893-8029-1ba8b6f23a3b - {'path': '/', 'tag': '5a561f59-fafa-4893-8029-1ba8b6f23a3b', 'options': 'recursive,force'} 2024-03-23T17:51:37.877 INFO:tasks.workunit.client.0.smithi086.stdout: 1 6324 25.54 MB/sec warmup 2 sec latency 6.032 ms 2024-03-23T17:51:38.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:51:37 smithi086 ceph-mon[38221]: pgmap v32: 97 pgs: 97 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.5 KiB/s wr, 14 op/s 2024-03-23T17:51:38.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:51:37 smithi086 ceph-mon[38221]: scrub queued for path: / 2024-03-23T17:51:38.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:51:37 smithi086 ceph-mon[38221]: scrub summary: idle+waiting paths [/] 2024-03-23T17:51:38.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:51:37 smithi086 ceph-mon[38221]: scrub summary: active paths [/] 2024-03-23T17:51:38.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:51:37 smithi086 ceph-mon[38221]: scrub summary: idle+waiting paths [/] 2024-03-23T17:51:38.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:51:37 smithi092 ceph-mon[79089]: pgmap v32: 97 pgs: 97 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.5 KiB/s wr, 14 op/s 2024-03-23T17:51:38.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:51:37 smithi092 ceph-mon[79089]: scrub queued for path: / 2024-03-23T17:51:38.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:51:37 smithi092 ceph-mon[79089]: scrub summary: idle+waiting paths [/] 2024-03-23T17:51:38.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:51:37 smithi092 ceph-mon[79089]: scrub summary: active paths [/] 2024-03-23T17:51:38.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:51:37 smithi092 ceph-mon[79089]: scrub summary: idle+waiting paths [/] 2024-03-23T17:51:38.085 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:51:37 smithi152 ceph-mon[47949]: pgmap v32: 97 pgs: 97 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.5 KiB/s wr, 14 op/s 2024-03-23T17:51:38.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:51:37 smithi152 ceph-mon[47949]: scrub queued for path: / 2024-03-23T17:51:38.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:51:37 smithi152 ceph-mon[47949]: scrub summary: idle+waiting paths [/] 2024-03-23T17:51:38.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:51:37 smithi152 ceph-mon[47949]: scrub summary: active paths [/] 2024-03-23T17:51:38.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:51:37 smithi152 ceph-mon[47949]: scrub summary: idle+waiting paths [/] 2024-03-23T17:51:38.877 INFO:tasks.workunit.client.0.smithi086.stdout: 1 9405 22.00 MB/sec warmup 3 sec latency 11.863 ms 2024-03-23T17:51:39.877 INFO:tasks.workunit.client.0.smithi086.stdout: 1 12085 20.84 MB/sec warmup 4 sec latency 7.097 ms 2024-03-23T17:51:40.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:51:39 smithi086 ceph-mon[38221]: pgmap v33: 97 pgs: 97 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-23T17:51:40.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:51:39 smithi086 ceph-mon[38221]: scrub summary: idle 2024-03-23T17:51:40.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:51:39 smithi092 ceph-mon[79089]: pgmap v33: 97 pgs: 97 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-23T17:51:40.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:51:39 smithi092 ceph-mon[79089]: scrub summary: idle 2024-03-23T17:51:40.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:51:39 smithi152 ceph-mon[47949]: pgmap v33: 97 pgs: 97 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-23T17:51:40.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:51:39 smithi152 ceph-mon[47949]: scrub summary: idle 2024-03-23T17:51:40.878 INFO:tasks.workunit.client.0.smithi086.stdout: 1 14789 19.49 MB/sec warmup 5 sec latency 19.901 ms 2024-03-23T17:51:41.878 INFO:tasks.workunit.client.0.smithi086.stdout: 1 18054 19.37 MB/sec warmup 6 sec latency 6.932 ms 2024-03-23T17:51:42.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:51:41 smithi086 ceph-mon[38221]: pgmap v34: 97 pgs: 97 active+clean; 6.7 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 533 KiB/s wr, 77 op/s 2024-03-23T17:51:42.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:51:41 smithi092 ceph-mon[79089]: pgmap v34: 97 pgs: 97 active+clean; 6.7 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 533 KiB/s wr, 77 op/s 2024-03-23T17:51:42.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:51:41 smithi152 ceph-mon[47949]: pgmap v34: 97 pgs: 97 active+clean; 6.7 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 533 KiB/s wr, 77 op/s 2024-03-23T17:51:42.878 INFO:tasks.workunit.client.0.smithi086.stdout: 1 21144 18.73 MB/sec warmup 7 sec latency 6.035 ms 2024-03-23T17:51:43.878 INFO:tasks.workunit.client.0.smithi086.stdout: 1 23649 17.82 MB/sec warmup 8 sec latency 11.394 ms 2024-03-23T17:51:44.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:51:43 smithi086 ceph-mon[38221]: pgmap v35: 97 pgs: 97 active+clean; 6.7 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 533 KiB/s wr, 72 op/s 2024-03-23T17:51:44.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:51:43 smithi092 ceph-mon[79089]: pgmap v35: 97 pgs: 97 active+clean; 6.7 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 533 KiB/s wr, 72 op/s 2024-03-23T17:51:44.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:51:43 smithi152 ceph-mon[47949]: pgmap v35: 97 pgs: 97 active+clean; 6.7 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 533 KiB/s wr, 72 op/s 2024-03-23T17:51:44.878 INFO:tasks.workunit.client.0.smithi086.stdout: 1 25742 17.27 MB/sec warmup 9 sec latency 11.093 ms 2024-03-23T17:51:45.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:51:44 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:51:45.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:51:44 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:51:45.085 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:51:44 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:51:45.878 INFO:tasks.workunit.client.0.smithi086.stdout: 1 28424 17.23 MB/sec warmup 10 sec latency 211.392 ms 2024-03-23T17:51:46.019 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:51:45 smithi086 ceph-mon[38221]: pgmap v36: 97 pgs: 97 active+clean; 10 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 863 KiB/s wr, 126 op/s 2024-03-23T17:51:46.019 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:51:45 smithi086 ceph-mon[38221]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T17:51:46.019 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:51:45 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:51:46.019 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:51:45 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:51:46.019 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:51:45 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:51:46.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:51:45 smithi092 ceph-mon[79089]: pgmap v36: 97 pgs: 97 active+clean; 10 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 863 KiB/s wr, 126 op/s 2024-03-23T17:51:46.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:51:45 smithi092 ceph-mon[79089]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T17:51:46.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:51:45 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:51:46.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:51:45 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:51:46.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:51:45 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:51:46.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:51:45 smithi152 ceph-mon[47949]: pgmap v36: 97 pgs: 97 active+clean; 10 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 863 KiB/s wr, 126 op/s 2024-03-23T17:51:46.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:51:45 smithi152 ceph-mon[47949]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T17:51:46.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:51:45 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:51:46.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:51:45 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:51:46.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:51:45 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:51:46.878 INFO:tasks.workunit.client.0.smithi086.stdout: 1 28424 15.66 MB/sec warmup 11 sec latency 1211.462 ms 2024-03-23T17:51:47.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.472+0000 7f6daa800700 1 -- 172.21.15.86:0/2826840725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6da4073ac0 msgr2=0x7f6da410cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:47.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.472+0000 7f6daa800700 1 --2- 172.21.15.86:0/2826840725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6da4073ac0 0x7f6da410cfe0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c009990 tx=0x7f6d8c009ca0 comp rx=0 tx=0).stop 2024-03-23T17:51:47.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.472+0000 7f6daa800700 1 -- 172.21.15.86:0/2826840725 shutdown_connections 2024-03-23T17:51:47.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.472+0000 7f6daa800700 1 --2- 172.21.15.86:0/2826840725 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6da410d520 0x7f6da410f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:47.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.472+0000 7f6daa800700 1 --2- 172.21.15.86:0/2826840725 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6da40731a0 0x7f6da4073580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:47.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.472+0000 7f6daa800700 1 --2- 172.21.15.86:0/2826840725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6da4073ac0 0x7f6da410cfe0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:47.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.472+0000 7f6daa800700 1 -- 172.21.15.86:0/2826840725 >> 172.21.15.86:0/2826840725 conn(0x7f6da40fc7b0 msgr2=0x7f6da40febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:47.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.472+0000 7f6daa800700 1 -- 172.21.15.86:0/2826840725 shutdown_connections 2024-03-23T17:51:47.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.472+0000 7f6daa800700 1 -- 172.21.15.86:0/2826840725 wait complete. 2024-03-23T17:51:47.475 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.473+0000 7f6daa800700 1 Processor -- start 2024-03-23T17:51:47.475 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.473+0000 7f6daa800700 1 -- start start 2024-03-23T17:51:47.475 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.474+0000 7f6daa800700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6da40731a0 0x7f6da4199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:47.475 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.474+0000 7f6daa800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6da4073ac0 0x7f6da4199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:47.475 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.474+0000 7f6daa800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6da410d520 0x7f6da419dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:47.475 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.474+0000 7f6daa800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da419e2d0 con 0x7f6da4073ac0 2024-03-23T17:51:47.475 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.474+0000 7f6daa800700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da419e440 con 0x7f6da410d520 2024-03-23T17:51:47.476 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.474+0000 7f6daa800700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da419e600 con 0x7f6da40731a0 2024-03-23T17:51:47.476 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.474+0000 7f6da3e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6da410d520 0x7f6da419dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:47.476 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.474+0000 7f6d9be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6da4073ac0 0x7f6da4199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:47.476 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.474+0000 7f6da3e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6da410d520 0x7f6da419dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:34412/0 (socket says 172.21.15.86:34412) 2024-03-23T17:51:47.476 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.474+0000 7f6da3e00700 1 -- 172.21.15.86:0/3288644896 learned_addr learned my addr 172.21.15.86:0/3288644896 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:47.476 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.474+0000 7f6da3400700 1 --2- 172.21.15.86:0/3288644896 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6da40731a0 0x7f6da4199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:47.476 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.474+0000 7f6da3e00700 1 -- 172.21.15.86:0/3288644896 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6da40731a0 msgr2=0x7f6da4199400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:47.476 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.474+0000 7f6da3e00700 1 --2- 172.21.15.86:0/3288644896 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6da40731a0 0x7f6da4199400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:47.476 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.474+0000 7f6da3e00700 1 -- 172.21.15.86:0/3288644896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6da4073ac0 msgr2=0x7f6da4199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:47.476 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.474+0000 7f6da3e00700 1 --2- 172.21.15.86:0/3288644896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6da4073ac0 0x7f6da4199940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:47.476 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.474+0000 7f6da3e00700 1 -- 172.21.15.86:0/3288644896 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d8c0095f0 con 0x7f6da410d520 2024-03-23T17:51:47.476 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.474+0000 7f6d9be00700 1 --2- 172.21.15.86:0/3288644896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6da4073ac0 0x7f6da4199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:51:47.476 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.475+0000 7f6da3e00700 1 --2- 172.21.15.86:0/3288644896 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6da410d520 0x7f6da419dc90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c00ef30 tx=0x7f6d9c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:47.477 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.475+0000 7f6da0c00700 1 -- 172.21.15.86:0/3288644896 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9c018070 con 0x7f6da410d520 2024-03-23T17:51:47.477 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.475+0000 7f6daa800700 1 -- 172.21.15.86:0/3288644896 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6da419e880 con 0x7f6da410d520 2024-03-23T17:51:47.477 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.475+0000 7f6da0c00700 1 -- 172.21.15.86:0/3288644896 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d9c00f040 con 0x7f6da410d520 2024-03-23T17:51:47.477 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.475+0000 7f6da0c00700 1 -- 172.21.15.86:0/3288644896 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9c00cd00 con 0x7f6da410d520 2024-03-23T17:51:47.477 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.475+0000 7f6daa800700 1 -- 172.21.15.86:0/3288644896 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6da41a6120 con 0x7f6da410d520 2024-03-23T17:51:47.478 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.478+0000 7f6da0c00700 1 -- 172.21.15.86:0/3288644896 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 25) v1 ==== 136098+0+0 (secure 0 0 0) 0x7f6d9c003a00 con 0x7f6da410d520 2024-03-23T17:51:47.480 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.480+0000 7f6dbe255b80 1 Processor -- start 2024-03-23T17:51:47.480 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.480+0000 7f6dbe255b80 1 -- start start 2024-03-23T17:51:47.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.480+0000 7f6dbe255b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bf1e66100 0x561bf1e664e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:47.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.480+0000 7f6da0c00700 1 --2- 172.21.15.86:0/3288644896 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f6d880a13b0 0x7f6d880a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:47.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.480+0000 7f6dbe255b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561bf1e66a20 0x561bf1e71390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:47.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.480+0000 7f6dbe255b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561bf1e718d0 0x561bf1e73cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:47.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.480+0000 7f6dbe255b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bf1dbf930 con 0x561bf1e66100 2024-03-23T17:51:47.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.480+0000 7f6dbe255b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bf1dbd2f0 con 0x561bf1e66a20 2024-03-23T17:51:47.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.480+0000 7f6dbe255b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bf1db8be0 con 0x561bf1e718d0 2024-03-23T17:51:47.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.481+0000 7f6da0c00700 1 -- 172.21.15.86:0/3288644896 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f6d9c0cca60 con 0x7f6da410d520 2024-03-23T17:51:47.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.481+0000 7f6da3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bf1e66100 0x561bf1e664e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:47.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.481+0000 7f6d9be00700 1 --2- 172.21.15.86:0/3288644896 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f6d880a13b0 0x7f6d880a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:47.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.481+0000 7f6da3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bf1e66100 0x561bf1e664e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36952/0 (socket says 172.21.15.86:36952) 2024-03-23T17:51:47.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.481+0000 7f6da3400700 1 -- 172.21.15.86:0/1619942282 learned_addr learned my addr 172.21.15.86:0/1619942282 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:47.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.481+0000 7f6da3e00700 1 --2- 172.21.15.86:0/1619942282 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561bf1e718d0 0x561bf1e73cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:47.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.481+0000 7f6d9be00700 1 --2- 172.21.15.86:0/1619942282 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561bf1e66a20 0x561bf1e71390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:47.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.481+0000 7f6da3400700 1 -- 172.21.15.86:0/1619942282 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561bf1e718d0 msgr2=0x561bf1e73cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:47.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.481+0000 7f6da3400700 1 --2- 172.21.15.86:0/1619942282 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561bf1e718d0 0x561bf1e73cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:47.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.481+0000 7f6da3400700 1 -- 172.21.15.86:0/1619942282 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561bf1e66a20 msgr2=0x561bf1e71390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:47.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.481+0000 7f6da3400700 1 --2- 172.21.15.86:0/1619942282 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561bf1e66a20 0x561bf1e71390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:47.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.481+0000 7f6da3400700 1 -- 172.21.15.86:0/1619942282 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561bf1c3a350 con 0x561bf1e66100 2024-03-23T17:51:47.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.481+0000 7f6da3e00700 1 --2- 172.21.15.86:0/1619942282 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561bf1e718d0 0x561bf1e73cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:51:47.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.481+0000 7f6d9be00700 1 --2- 172.21.15.86:0/3288644896 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f6d880a13b0 0x7f6d880a3870 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c005cd0 tx=0x7f6d8c005c60 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:47.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.482+0000 7f6da3400700 1 --2- 172.21.15.86:0/1619942282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bf1e66100 0x561bf1e664e0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f6d94009d80 tx=0x7f6d94011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef541b90b86eb0e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:47.484 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.482+0000 7f6d98c00700 1 -- 172.21.15.86:0/1619942282 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d94004230 con 0x561bf1e66100 2024-03-23T17:51:47.484 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.482+0000 7f6d98c00700 1 -- 172.21.15.86:0/1619942282 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d94004390 con 0x561bf1e66100 2024-03-23T17:51:47.484 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.482+0000 7f6d98c00700 1 -- 172.21.15.86:0/1619942282 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d940198c0 con 0x561bf1e66100 2024-03-23T17:51:47.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.482+0000 7f6dbe255b80 1 -- 172.21.15.86:0/1619942282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bf1e66100 msgr2=0x561bf1e664e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:47.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.482+0000 7f6dbe255b80 1 --2- 172.21.15.86:0/1619942282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bf1e66100 0x561bf1e664e0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f6d94009d80 tx=0x7f6d94011720 comp rx=0 tx=0).stop 2024-03-23T17:51:47.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.482+0000 7f6dbe255b80 1 -- 172.21.15.86:0/1619942282 shutdown_connections 2024-03-23T17:51:47.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.482+0000 7f6dbe255b80 1 --2- 172.21.15.86:0/1619942282 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561bf1e718d0 0x561bf1e73cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:47.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.482+0000 7f6dbe255b80 1 --2- 172.21.15.86:0/1619942282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bf1e66100 0x561bf1e664e0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:47.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.482+0000 7f6dbe255b80 1 --2- 172.21.15.86:0/1619942282 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561bf1e66a20 0x561bf1e71390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:47.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.483+0000 7f6dbe255b80 1 -- 172.21.15.86:0/1619942282 >> 172.21.15.86:0/1619942282 conn(0x561bf1dc6640 msgr2=0x561bf1e62520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:47.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.483+0000 7f6dbe255b80 1 -- 172.21.15.86:0/1619942282 shutdown_connections 2024-03-23T17:51:47.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.483+0000 7f6dbe255b80 1 -- 172.21.15.86:0/1619942282 wait complete. 2024-03-23T17:51:47.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.483+0000 7f6dbe255b80 1 -- start start 2024-03-23T17:51:47.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.484+0000 7f6dbe255b80 1 Processor -- start 2024-03-23T17:51:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.484+0000 7f6dbe255b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561bf1e66100 0x561bf1e711d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.484+0000 7f6dbe255b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561bf1e66a20 0x561bf1e6e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.484+0000 7f6dbe255b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bf1e718d0 0x561bf1e6eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.484+0000 7f6dbe255b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bf1d54d20 con 0x561bf1e718d0 2024-03-23T17:51:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.484+0000 7f6dbe255b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bf1d549c0 con 0x561bf1e66a20 2024-03-23T17:51:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.484+0000 7f6dbe255b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bf1c3c640 con 0x561bf1e66100 2024-03-23T17:51:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.484+0000 7f6dbe255b80 10 client.0 _ll_register_callbacks cb 0x561bf1d5e410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:51:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.484+0000 7f6da3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bf1e718d0 0x561bf1e6eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.485+0000 7f6da3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bf1e718d0 0x561bf1e6eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36966/0 (socket says 172.21.15.86:36966) 2024-03-23T17:51:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.485+0000 7f6da3e00700 1 -- 172.21.15.86:0/3422847444 learned_addr learned my addr 172.21.15.86:0/3422847444 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.485+0000 7f6da3400700 1 --2- 172.21.15.86:0/3422847444 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561bf1e66100 0x561bf1e711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.485+0000 7f6d9be00700 1 --2- 172.21.15.86:0/3422847444 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561bf1e66a20 0x561bf1e6e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.485+0000 7f6da3e00700 1 -- 172.21.15.86:0/3422847444 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561bf1e66100 msgr2=0x561bf1e711d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.485+0000 7f6da3e00700 1 --2- 172.21.15.86:0/3422847444 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561bf1e66100 0x561bf1e711d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.485+0000 7f6da3e00700 1 -- 172.21.15.86:0/3422847444 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561bf1e66a20 msgr2=0x561bf1e6e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.485+0000 7f6da3e00700 1 --2- 172.21.15.86:0/3422847444 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561bf1e66a20 0x561bf1e6e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.485+0000 7f6da3e00700 1 -- 172.21.15.86:0/3422847444 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6da41a6120 con 0x561bf1e718d0 2024-03-23T17:51:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.485+0000 7f6d9be00700 1 --2- 172.21.15.86:0/3422847444 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561bf1e66a20 0x561bf1e6e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:51:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.485+0000 7f6da3e00700 1 --2- 172.21.15.86:0/3422847444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bf1e718d0 0x561bf1e6eb00 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c099930 tx=0x7f6d9c0993f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=751a9eda78fd7b06 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.485+0000 7f6d91600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-23T17:51:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.485+0000 7f6d91600700 1 -- 172.21.15.86:0/3422847444 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9c099090 con 0x561bf1e718d0 2024-03-23T17:51:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.485+0000 7f6d91600700 1 -- 172.21.15.86:0/3422847444 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f6d9c004ad0 con 0x561bf1e718d0 2024-03-23T17:51:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.485+0000 7f6dbe255b80 1 -- 172.21.15.86:0/3422847444 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561bf1d79320 con 0x561bf1e718d0 2024-03-23T17:51:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.486+0000 7f6d91600700 1 -- 172.21.15.86:0/3422847444 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d9c01b320 con 0x561bf1e718d0 2024-03-23T17:51:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.486+0000 7f6dbe255b80 10 client.15813 fetch_fsmap learned FSMap version 29 2024-03-23T17:51:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.486+0000 7f6dbe255b80 1 -- 172.21.15.86:0/3422847444 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f6d940099e0 con 0x561bf1e718d0 2024-03-23T17:51:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.486+0000 7f6d91600700 1 -- 172.21.15.86:0/3422847444 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d9c075080 con 0x561bf1e718d0 2024-03-23T17:51:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.486+0000 7f6d91600700 1 -- 172.21.15.86:0/3422847444 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f6d9c0a1900 con 0x561bf1e718d0 2024-03-23T17:51:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.486+0000 7f6dbe255b80 10 client.15813 fetch_fsmap finished waiting for FSMap version 29 2024-03-23T17:51:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.486+0000 7f6dbe255b80 10 client.15813 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:51:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.486+0000 7f6dbe255b80 20 client.15813 populate_metadata read hostname 'smithi086' 2024-03-23T17:51:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.486+0000 7f6dbe255b80 1 --2- 172.21.15.86:0/3422847444 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x561bf1e814e0 0x561bf1e83900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.486+0000 7f6dbe255b80 4 client.15813 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:51:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.486+0000 7f6dbe255b80 1 -- 172.21.15.86:0/3422847444 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561bf1db16a0 con 0x561bf1e814e0 2024-03-23T17:51:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.487+0000 7f6da3400700 1 --2- 172.21.15.86:0/3422847444 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x561bf1e814e0 0x561bf1e83900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.487+0000 7f6da3400700 1 --2- 172.21.15.86:0/3422847444 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x561bf1e814e0 0x561bf1e83900 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b1e803af26f9f86 server_cookie=439f2af3fefe927e in_seq=0 out_seq=0 2024-03-23T17:51:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.487+0000 7f6d91600700 10 client.15813 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:51:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.488+0000 7f6d91600700 1 -- 172.21.15.86:0/3422847444 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x561bf1db16a0 con 0x561bf1e814e0 2024-03-23T17:51:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.488+0000 7f6d91600700 10 client.15813 handle_command_reply: tid=0 2024-03-23T17:51:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.488+0000 7f6dbe255b80 1 client.15813 shutdown 2024-03-23T17:51:47.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.488+0000 7f6dbe255b80 1 -- 172.21.15.86:0/3422847444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bf1e718d0 msgr2=0x561bf1e6eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:47.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.488+0000 7f6dbe255b80 1 --2- 172.21.15.86:0/3422847444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bf1e718d0 0x561bf1e6eb00 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c099930 tx=0x7f6d9c0993f0 comp rx=0 tx=0).stop 2024-03-23T17:51:47.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.488+0000 7f6dbe255b80 1 -- 172.21.15.86:0/3422847444 shutdown_connections 2024-03-23T17:51:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.488+0000 7f6dbe255b80 1 --2- 172.21.15.86:0/3422847444 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x561bf1e814e0 0x561bf1e83900 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.488+0000 7f6dbe255b80 1 --2- 172.21.15.86:0/3422847444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bf1e718d0 0x561bf1e6eb00 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.488+0000 7f6dbe255b80 1 --2- 172.21.15.86:0/3422847444 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561bf1e66100 0x561bf1e711d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.488+0000 7f6dbe255b80 1 --2- 172.21.15.86:0/3422847444 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561bf1e66a20 0x561bf1e6e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.488+0000 7f6d91600700 0 client.15813 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:51:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.488+0000 7f6dbe255b80 1 -- 172.21.15.86:0/3422847444 >> 172.21.15.86:0/3422847444 conn(0x561bf1dc6640 msgr2=0x561bf1e680a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.489+0000 7f6dbe255b80 1 -- 172.21.15.86:0/3422847444 shutdown_connections 2024-03-23T17:51:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.489+0000 7f6dbe255b80 1 -- 172.21.15.86:0/3422847444 wait complete. 2024-03-23T17:51:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.489+0000 7f6dbe255b80 20 client.15813 trim_cache size 0 max 16384 2024-03-23T17:51:47.504 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.503+0000 7f6dbe255b80 1 Processor -- start 2024-03-23T17:51:47.504 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.503+0000 7f6dbe255b80 1 -- start start 2024-03-23T17:51:47.504 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.504+0000 7f6dbe255b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561bf1e66100 0x561bf1e785e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:47.504 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.504+0000 7f6dbe255b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bf1e66a20 0x561bf1e5c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:47.504 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.504+0000 7f6dbe255b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561bf1e718d0 0x561bf1e68040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:47.504 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.504+0000 7f6dbe255b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bf1e88620 con 0x561bf1e66a20 2024-03-23T17:51:47.504 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.504+0000 7f6dbe255b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bf1e6a1c0 con 0x561bf1e66100 2024-03-23T17:51:47.504 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.504+0000 7f6dbe255b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bf1e69430 con 0x561bf1e718d0 2024-03-23T17:51:47.505 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.504+0000 7f6d9be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bf1e66a20 0x561bf1e5c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:47.505 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.504+0000 7f6d9be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bf1e66a20 0x561bf1e5c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36974/0 (socket says 172.21.15.86:36974) 2024-03-23T17:51:47.505 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.504+0000 7f6da3400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561bf1e66100 0x561bf1e785e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:47.505 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.504+0000 7f6d9be00700 1 -- 172.21.15.86:0/1462011359 learned_addr learned my addr 172.21.15.86:0/1462011359 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:47.505 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.504+0000 7f6da3e00700 1 --2- 172.21.15.86:0/1462011359 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561bf1e718d0 0x561bf1e68040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:47.509 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.504+0000 7f6d9be00700 1 -- 172.21.15.86:0/1462011359 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561bf1e718d0 msgr2=0x561bf1e68040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:47.509 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.504+0000 7f6d9be00700 1 --2- 172.21.15.86:0/1462011359 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561bf1e718d0 0x561bf1e68040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:47.509 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.504+0000 7f6d9be00700 1 -- 172.21.15.86:0/1462011359 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561bf1e66100 msgr2=0x561bf1e785e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:47.509 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.504+0000 7f6d9be00700 1 --2- 172.21.15.86:0/1462011359 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561bf1e66100 0x561bf1e785e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:47.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.504+0000 7f6d9be00700 1 -- 172.21.15.86:0/1462011359 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561bf1be6b00 con 0x561bf1e66a20 2024-03-23T17:51:47.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.504+0000 7f6da3400700 1 --2- 172.21.15.86:0/1462011359 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561bf1e66100 0x561bf1e785e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:51:47.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.505+0000 7f6d9be00700 1 --2- 172.21.15.86:0/1462011359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bf1e66a20 0x561bf1e5c500 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c000c00 tx=0x7f6d8c02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f4e35b19215fd38 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:47.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.505+0000 7f6d99600700 1 -- 172.21.15.86:0/1462011359 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8c01d070 con 0x561bf1e66a20 2024-03-23T17:51:47.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.505+0000 7f6d99600700 1 -- 172.21.15.86:0/1462011359 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d8c01e040 con 0x561bf1e66a20 2024-03-23T17:51:47.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.505+0000 7f6d99600700 1 -- 172.21.15.86:0/1462011359 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8c029bc0 con 0x561bf1e66a20 2024-03-23T17:51:47.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.505+0000 7f6dbe255b80 1 -- 172.21.15.86:0/1462011359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bf1e66a20 msgr2=0x561bf1e5c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:47.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.505+0000 7f6dbe255b80 1 --2- 172.21.15.86:0/1462011359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bf1e66a20 0x561bf1e5c500 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c000c00 tx=0x7f6d8c02c000 comp rx=0 tx=0).stop 2024-03-23T17:51:47.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.506+0000 7f6dbe255b80 1 -- 172.21.15.86:0/1462011359 shutdown_connections 2024-03-23T17:51:47.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.506+0000 7f6dbe255b80 1 --2- 172.21.15.86:0/1462011359 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561bf1e718d0 0x561bf1e68040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:47.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.506+0000 7f6dbe255b80 1 --2- 172.21.15.86:0/1462011359 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561bf1e66100 0x561bf1e785e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:47.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.506+0000 7f6dbe255b80 1 --2- 172.21.15.86:0/1462011359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bf1e66a20 0x561bf1e5c500 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:47.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.506+0000 7f6dbe255b80 1 -- 172.21.15.86:0/1462011359 >> 172.21.15.86:0/1462011359 conn(0x561bf1dc6640 msgr2=0x561bf1e6ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:47.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.506+0000 7f6dbe255b80 1 -- 172.21.15.86:0/1462011359 shutdown_connections 2024-03-23T17:51:47.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.506+0000 7f6dbe255b80 1 -- 172.21.15.86:0/1462011359 wait complete. 2024-03-23T17:51:47.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.506+0000 7f6dbe255b80 1 -- start start 2024-03-23T17:51:47.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.506+0000 7f6dbe255b80 1 Processor -- start 2024-03-23T17:51:47.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.507+0000 7f6dbe255b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bf1e66100 0x561bf1ecc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:47.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.507+0000 7f6dbe255b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561bf1e66a20 0x561bf1ecc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:47.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.507+0000 7f6dbe255b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561bf1e718d0 0x561bf1e765c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:47.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.507+0000 7f6dbe255b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bf1e76b80 con 0x561bf1e66100 2024-03-23T17:51:47.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.507+0000 7f6dbe255b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bf1e76cf0 con 0x561bf1e66a20 2024-03-23T17:51:47.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.507+0000 7f6dbe255b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bf1e76eb0 con 0x561bf1e718d0 2024-03-23T17:51:47.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.507+0000 7f6dbe255b80 10 client.0 _ll_register_callbacks cb 0x561bf1d5e410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:51:47.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.507+0000 7f6da3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bf1e66100 0x561bf1ecc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:47.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.507+0000 7f6da3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bf1e66100 0x561bf1ecc300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36978/0 (socket says 172.21.15.86:36978) 2024-03-23T17:51:47.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.507+0000 7f6da3400700 1 -- 172.21.15.86:0/1735451558 learned_addr learned my addr 172.21.15.86:0/1735451558 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:51:47.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.507+0000 7f6d9be00700 1 --2- 172.21.15.86:0/1735451558 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561bf1e66a20 0x561bf1ecc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:47.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.507+0000 7f6da3e00700 1 --2- 172.21.15.86:0/1735451558 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561bf1e718d0 0x561bf1e765c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:47.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.507+0000 7f6da3400700 1 -- 172.21.15.86:0/1735451558 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561bf1e718d0 msgr2=0x561bf1e765c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.507+0000 7f6da3400700 1 --2- 172.21.15.86:0/1735451558 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561bf1e718d0 0x561bf1e765c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.507+0000 7f6da3400700 1 -- 172.21.15.86:0/1735451558 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561bf1e66a20 msgr2=0x561bf1ecc840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:51:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.507+0000 7f6da3400700 1 --2- 172.21.15.86:0/1735451558 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561bf1e66a20 0x561bf1ecc840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.507+0000 7f6da3400700 1 -- 172.21.15.86:0/1735451558 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561bf1c3a350 con 0x561bf1e66100 2024-03-23T17:51:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.508+0000 7f6da3400700 1 --2- 172.21.15.86:0/1735451558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bf1e66100 0x561bf1ecc300 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f6d94005950 tx=0x7f6d94004100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50f097e6834835c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:51:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.508+0000 7f6d91600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-23T17:51:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.508+0000 7f6d91600700 1 -- 172.21.15.86:0/1735451558 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d94009ed0 con 0x561bf1e66100 2024-03-23T17:51:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.508+0000 7f6dbe255b80 1 -- 172.21.15.86:0/1735451558 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6d94004390 con 0x561bf1e66100 2024-03-23T17:51:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.508+0000 7f6d91600700 1 -- 172.21.15.86:0/1735451558 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f6d94003680 con 0x561bf1e66100 2024-03-23T17:51:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.508+0000 7f6d91600700 1 -- 172.21.15.86:0/1735451558 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d94005b00 con 0x561bf1e66100 2024-03-23T17:51:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.508+0000 7f6dbe255b80 10 client.15819 fetch_fsmap learned FSMap version 29 2024-03-23T17:51:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.508+0000 7f6dbe255b80 1 -- 172.21.15.86:0/1735451558 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f6d8c020040 con 0x561bf1e66100 2024-03-23T17:51:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.508+0000 7f6d91600700 1 -- 172.21.15.86:0/1735451558 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d94004390 con 0x561bf1e66100 2024-03-23T17:51:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.508+0000 7f6d91600700 1 -- 172.21.15.86:0/1735451558 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f6d94011440 con 0x561bf1e66100 2024-03-23T17:51:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.508+0000 7f6dbe255b80 10 client.15819 fetch_fsmap finished waiting for FSMap version 29 2024-03-23T17:51:47.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.508+0000 7f6dbe255b80 10 client.15819 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:51:47.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.508+0000 7f6dbe255b80 20 client.15819 populate_metadata read hostname 'smithi086' 2024-03-23T17:51:47.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.508+0000 7f6dbe255b80 1 --2- 172.21.15.86:0/1735451558 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x561bf1ed40c0 0x561bf1ed64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:51:47.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.508+0000 7f6dbe255b80 4 client.15819 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T17:51:47.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.508+0000 7f6dbe255b80 1 -- 172.21.15.86:0/1735451558 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561bf1c07c90 con 0x561bf1ed40c0 2024-03-23T17:51:47.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.509+0000 7f6d9be00700 1 --2- 172.21.15.86:0/1735451558 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x561bf1ed40c0 0x561bf1ed64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:51:47.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.509+0000 7f6d9be00700 1 --2- 172.21.15.86:0/1735451558 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x561bf1ed40c0 0x561bf1ed64e0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5dddbd5c9a00bac4 server_cookie=2bbb1b9d6338442f in_seq=0 out_seq=0 2024-03-23T17:51:47.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.509+0000 7f6d91600700 10 client.15819 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:51:47.607 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.606+0000 7f6d91600700 1 -- 172.21.15.86:0/1735451558 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561bf1c07c90 con 0x561bf1ed40c0 2024-03-23T17:51:47.607 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.606+0000 7f6d91600700 10 client.15819 handle_command_reply: tid=0 2024-03-23T17:51:47.607 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.606+0000 7f6dbe255b80 1 client.15819 shutdown 2024-03-23T17:51:47.607 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.606+0000 7f6dbe255b80 1 -- 172.21.15.86:0/1735451558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bf1e66100 msgr2=0x561bf1ecc300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:47.607 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.606+0000 7f6dbe255b80 1 --2- 172.21.15.86:0/1735451558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bf1e66100 0x561bf1ecc300 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f6d94005950 tx=0x7f6d94004100 comp rx=0 tx=0).stop 2024-03-23T17:51:47.607 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.606+0000 7f6dbe255b80 1 -- 172.21.15.86:0/1735451558 shutdown_connections 2024-03-23T17:51:47.607 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.606+0000 7f6dbe255b80 1 --2- 172.21.15.86:0/1735451558 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x561bf1ed40c0 0x561bf1ed64e0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:47.608 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.606+0000 7f6dbe255b80 1 --2- 172.21.15.86:0/1735451558 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561bf1e718d0 0x561bf1e765c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:47.608 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.606+0000 7f6dbe255b80 1 --2- 172.21.15.86:0/1735451558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bf1e66100 0x561bf1ecc300 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:47.608 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.606+0000 7f6dbe255b80 1 --2- 172.21.15.86:0/1735451558 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561bf1e66a20 0x561bf1ecc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:47.608 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.606+0000 7f6d91600700 0 client.15819 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:51:47.608 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.606+0000 7f6dbe255b80 1 -- 172.21.15.86:0/1735451558 >> 172.21.15.86:0/1735451558 conn(0x561bf1dc6640 msgr2=0x561bf1e68040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:47.608 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.607+0000 7f6dbe255b80 1 -- 172.21.15.86:0/1735451558 shutdown_connections 2024-03-23T17:51:47.608 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.607+0000 7f6dbe255b80 1 -- 172.21.15.86:0/1735451558 wait complete. 2024-03-23T17:51:47.608 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.607+0000 7f6dbe255b80 20 client.15819 trim_cache size 0 max 16384 2024-03-23T17:51:47.608 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-03-23T17:51:47.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-03-23T17:51:47.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-03-23T17:51:47.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-03-23T17:51:47.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.607+0000 7f6daa800700 1 -- 172.21.15.86:0/3288644896 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f6d880a13b0 msgr2=0x7f6d880a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:47.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.607+0000 7f6daa800700 1 --2- 172.21.15.86:0/3288644896 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f6d880a13b0 0x7f6d880a3870 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c005cd0 tx=0x7f6d8c005c60 comp rx=0 tx=0).stop 2024-03-23T17:51:47.610 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.607+0000 7f6daa800700 1 -- 172.21.15.86:0/3288644896 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6da410d520 msgr2=0x7f6da419dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:51:47.610 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.607+0000 7f6daa800700 1 --2- 172.21.15.86:0/3288644896 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6da410d520 0x7f6da419dc90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c00ef30 tx=0x7f6d9c00c3a0 comp rx=0 tx=0).stop 2024-03-23T17:51:47.610 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.607+0000 7f6daa800700 1 -- 172.21.15.86:0/3288644896 shutdown_connections 2024-03-23T17:51:47.610 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.607+0000 7f6daa800700 1 --2- 172.21.15.86:0/3288644896 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6da410d520 0x7f6da419dc90 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:47.610 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.607+0000 7f6daa800700 1 --2- 172.21.15.86:0/3288644896 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f6d880a13b0 0x7f6d880a3870 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:47.610 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.607+0000 7f6daa800700 1 --2- 172.21.15.86:0/3288644896 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6da40731a0 0x7f6da4199400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:47.610 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.607+0000 7f6daa800700 1 --2- 172.21.15.86:0/3288644896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6da4073ac0 0x7f6da4199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:51:47.610 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.607+0000 7f6daa800700 1 -- 172.21.15.86:0/3288644896 >> 172.21.15.86:0/3288644896 conn(0x7f6da40fc7b0 msgr2=0x7f6da40fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:51:47.610 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.608+0000 7f6daa800700 1 -- 172.21.15.86:0/3288644896 shutdown_connections 2024-03-23T17:51:47.610 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:51:47.608+0000 7f6daa800700 1 -- 172.21.15.86:0/3288644896 wait complete. 2024-03-23T17:51:47.878 INFO:tasks.workunit.client.0.smithi086.stdout: 1 28505 14.38 MB/sec warmup 12 sec latency 2069.663 ms 2024-03-23T17:51:48.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:51:47 smithi086 ceph-mon[38221]: pgmap v37: 97 pgs: 97 active+clean; 13 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 1.2 MiB/s wr, 161 op/s 2024-03-23T17:51:48.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:51:47 smithi092 ceph-mon[79089]: pgmap v37: 97 pgs: 97 active+clean; 13 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 1.2 MiB/s wr, 161 op/s 2024-03-23T17:51:48.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:51:47 smithi152 ceph-mon[47949]: pgmap v37: 97 pgs: 97 active+clean; 13 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 1.2 MiB/s wr, 161 op/s 2024-03-23T17:51:48.878 INFO:tasks.workunit.client.0.smithi086.stdout: 1 28505 13.27 MB/sec warmup 13 sec latency 1105.342 ms 2024-03-23T17:51:49.878 INFO:tasks.workunit.client.0.smithi086.stdout: 1 28505 12.32 MB/sec warmup 14 sec latency 2105.501 ms 2024-03-23T17:51:50.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:51:49 smithi086 ceph-mon[38221]: pgmap v38: 97 pgs: 97 active+clean; 13 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 1.2 MiB/s wr, 157 op/s 2024-03-23T17:51:50.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:51:49 smithi092 ceph-mon[79089]: pgmap v38: 97 pgs: 97 active+clean; 13 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 1.2 MiB/s wr, 157 op/s 2024-03-23T17:51:50.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:51:49 smithi152 ceph-mon[47949]: pgmap v38: 97 pgs: 97 active+clean; 13 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 1.2 MiB/s wr, 157 op/s 2024-03-23T17:51:50.879 INFO:tasks.workunit.client.0.smithi086.stdout: 1 28505 11.50 MB/sec warmup 15 sec latency 3105.637 ms 2024-03-23T17:51:51.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:51:50 smithi086 ceph-mon[38221]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T17:51:51.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:51:50 smithi092 ceph-mon[79089]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T17:51:51.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:51:50 smithi152 ceph-mon[47949]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T17:51:51.879 INFO:tasks.workunit.client.0.smithi086.stdout: 1 28505 10.78 MB/sec warmup 16 sec latency 4105.758 ms 2024-03-23T17:51:52.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:51:51 smithi092 ceph-mon[79089]: pgmap v39: 97 pgs: 97 active+clean; 6.7 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 1.4 MiB/s wr, 276 op/s 2024-03-23T17:51:52.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:51:51 smithi152 ceph-mon[47949]: pgmap v39: 97 pgs: 97 active+clean; 6.7 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 1.4 MiB/s wr, 276 op/s 2024-03-23T17:51:52.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:51:51 smithi086 ceph-mon[38221]: pgmap v39: 97 pgs: 97 active+clean; 6.7 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 1.4 MiB/s wr, 276 op/s 2024-03-23T17:51:52.879 INFO:tasks.workunit.client.0.smithi086.stdout: 1 28575 10.16 MB/sec warmup 17 sec latency 4968.542 ms 2024-03-23T17:51:53.879 INFO:tasks.workunit.client.0.smithi086.stdout: 1 28575 9.59 MB/sec warmup 18 sec latency 1062.231 ms 2024-03-23T17:51:54.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:51:53 smithi092 ceph-mon[79089]: pgmap v40: 97 pgs: 97 active+clean; 6.7 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 924 KiB/s wr, 208 op/s 2024-03-23T17:51:54.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:51:53 smithi152 ceph-mon[47949]: pgmap v40: 97 pgs: 97 active+clean; 6.7 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 924 KiB/s wr, 208 op/s 2024-03-23T17:51:54.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:51:53 smithi086 ceph-mon[38221]: pgmap v40: 97 pgs: 97 active+clean; 6.7 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 924 KiB/s wr, 208 op/s 2024-03-23T17:51:54.879 INFO:tasks.workunit.client.0.smithi086.stdout: 1 28575 9.09 MB/sec warmup 19 sec latency 2062.351 ms 2024-03-23T17:51:55.879 INFO:tasks.workunit.client.0.smithi086.stdout: 1 28575 8.63 MB/sec warmup 20 sec latency 3062.491 ms 2024-03-23T17:51:56.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:51:55 smithi092 ceph-mon[79089]: pgmap v41: 97 pgs: 97 active+clean; 6.7 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 926 KiB/s wr, 209 op/s 2024-03-23T17:51:56.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:51:55 smithi152 ceph-mon[47949]: pgmap v41: 97 pgs: 97 active+clean; 6.7 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 926 KiB/s wr, 209 op/s 2024-03-23T17:51:56.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:51:55 smithi086 ceph-mon[38221]: pgmap v41: 97 pgs: 97 active+clean; 6.7 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 926 KiB/s wr, 209 op/s 2024-03-23T17:51:56.879 INFO:tasks.workunit.client.0.smithi086.stdout: 1 28575 8.22 MB/sec warmup 21 sec latency 4062.567 ms 2024-03-23T17:51:57.879 INFO:tasks.workunit.client.0.smithi086.stdout: 1 28660 7.89 MB/sec warmup 22 sec latency 4967.193 ms 2024-03-23T17:51:58.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:51:57 smithi092 ceph-mon[79089]: pgmap v42: 97 pgs: 97 active+clean; 6.9 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 614 KiB/s wr, 162 op/s 2024-03-23T17:51:58.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:51:57 smithi152 ceph-mon[47949]: pgmap v42: 97 pgs: 97 active+clean; 6.9 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 614 KiB/s wr, 162 op/s 2024-03-23T17:51:58.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:51:57 smithi086 ceph-mon[38221]: pgmap v42: 97 pgs: 97 active+clean; 6.9 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 614 KiB/s wr, 162 op/s 2024-03-23T17:51:58.880 INFO:tasks.workunit.client.0.smithi086.stdout: 1 28660 7.55 MB/sec warmup 23 sec latency 1003.211 ms 2024-03-23T17:51:59.880 INFO:tasks.workunit.client.0.smithi086.stdout: 1 28660 7.24 MB/sec warmup 24 sec latency 2003.285 ms 2024-03-23T17:52:00.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:51:59 smithi092 ceph-mon[79089]: pgmap v43: 97 pgs: 97 active+clean; 6.9 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 237 KiB/s wr, 127 op/s 2024-03-23T17:52:00.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:51:59 smithi152 ceph-mon[47949]: pgmap v43: 97 pgs: 97 active+clean; 6.9 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 237 KiB/s wr, 127 op/s 2024-03-23T17:52:00.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:51:59 smithi086 ceph-mon[38221]: pgmap v43: 97 pgs: 97 active+clean; 6.9 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 237 KiB/s wr, 127 op/s 2024-03-23T17:52:00.763 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.762+0000 7f20a8400700 1 -- 172.21.15.86:0/2930849212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20a0101f20 msgr2=0x7f20a0102300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:00.763 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.762+0000 7f20a8400700 1 --2- 172.21.15.86:0/2930849212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20a0101f20 0x7f20a0102300 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f2094009990 tx=0x7f2094009ca0 comp rx=0 tx=0).stop 2024-03-23T17:52:00.763 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.762+0000 7f20a8400700 1 -- 172.21.15.86:0/2930849212 shutdown_connections 2024-03-23T17:52:00.763 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.762+0000 7f20a8400700 1 --2- 172.21.15.86:0/2930849212 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20a010b2f0 0x7f20a010d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:00.763 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.762+0000 7f20a8400700 1 --2- 172.21.15.86:0/2930849212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20a0101f20 0x7f20a0102300 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:00.763 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.762+0000 7f20a8400700 1 --2- 172.21.15.86:0/2930849212 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f20a0102840 0x7f20a010ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:00.763 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.762+0000 7f20a8400700 1 -- 172.21.15.86:0/2930849212 >> 172.21.15.86:0/2930849212 conn(0x7f20a00fb680 msgr2=0x7f20a00fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:00.763 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.762+0000 7f20a8400700 1 -- 172.21.15.86:0/2930849212 shutdown_connections 2024-03-23T17:52:00.763 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.762+0000 7f20a8400700 1 -- 172.21.15.86:0/2930849212 wait complete. 2024-03-23T17:52:00.766 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.763+0000 7f20a8400700 1 Processor -- start 2024-03-23T17:52:00.767 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.763+0000 7f20a8400700 1 -- start start 2024-03-23T17:52:00.767 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.764+0000 7f20a8400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20a0101f20 0x7f20a0199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:00.767 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.764+0000 7f20a8400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f20a0102840 0x7f20a0199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:00.767 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.764+0000 7f20a8400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20a010b2f0 0x7f20a019dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:00.767 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.764+0000 7f20a8400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a019e250 con 0x7f20a0101f20 2024-03-23T17:52:00.767 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.764+0000 7f20a8400700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a019e3c0 con 0x7f20a010b2f0 2024-03-23T17:52:00.767 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.764+0000 7f20a8400700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a019e580 con 0x7f20a0102840 2024-03-23T17:52:00.767 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.764+0000 7f20a5a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20a0101f20 0x7f20a0199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:00.767 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.764+0000 7f20a5a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20a0101f20 0x7f20a0199330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59508/0 (socket says 172.21.15.86:59508) 2024-03-23T17:52:00.767 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.764+0000 7f20a5a00700 1 -- 172.21.15.86:0/3922723993 learned_addr learned my addr 172.21.15.86:0/3922723993 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:00.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.764+0000 7f20a6400700 1 --2- 172.21.15.86:0/3922723993 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20a010b2f0 0x7f20a019dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:00.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.764+0000 7f20a5000700 1 --2- 172.21.15.86:0/3922723993 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f20a0102840 0x7f20a0199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:00.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.764+0000 7f20a5a00700 1 -- 172.21.15.86:0/3922723993 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f20a0102840 msgr2=0x7f20a0199870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:00.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.764+0000 7f20a5a00700 1 --2- 172.21.15.86:0/3922723993 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f20a0102840 0x7f20a0199870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:00.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.764+0000 7f20a5a00700 1 -- 172.21.15.86:0/3922723993 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20a010b2f0 msgr2=0x7f20a019dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:00.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.764+0000 7f20a5a00700 1 --2- 172.21.15.86:0/3922723993 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20a010b2f0 0x7f20a019dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:00.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.764+0000 7f20a5a00700 1 -- 172.21.15.86:0/3922723993 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20940095f0 con 0x7f20a0101f20 2024-03-23T17:52:00.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.765+0000 7f20a5a00700 1 --2- 172.21.15.86:0/3922723993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20a0101f20 0x7f20a0199330 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f20940095c0 tx=0x7f209400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:00.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.765+0000 7f209e000700 1 -- 172.21.15.86:0/3922723993 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f209401d070 con 0x7f20a0101f20 2024-03-23T17:52:00.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.765+0000 7f209e000700 1 -- 172.21.15.86:0/3922723993 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2094005050 con 0x7f20a0101f20 2024-03-23T17:52:00.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.765+0000 7f209e000700 1 -- 172.21.15.86:0/3922723993 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2094004a70 con 0x7f20a0101f20 2024-03-23T17:52:00.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.765+0000 7f20a8400700 1 -- 172.21.15.86:0/3922723993 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20a019e800 con 0x7f20a0101f20 2024-03-23T17:52:00.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.765+0000 7f20a8400700 1 -- 172.21.15.86:0/3922723993 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20a019ea80 con 0x7f20a0101f20 2024-03-23T17:52:00.769 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.768+0000 7f209e000700 1 -- 172.21.15.86:0/3922723993 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) v1 ==== 136098+0+0 (secure 0 0 0) 0x7f20940051c0 con 0x7f20a0101f20 2024-03-23T17:52:00.769 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.769+0000 7f209e000700 1 --2- 172.21.15.86:0/3922723993 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f20880a1430 0x7f20880a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:00.770 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.769+0000 7f20bbeecb80 1 Processor -- start 2024-03-23T17:52:00.770 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.770+0000 7f209e000700 1 -- 172.21.15.86:0/3922723993 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f20940cdaf0 con 0x7f20a0101f20 2024-03-23T17:52:00.770 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.770+0000 7f20a5000700 1 --2- 172.21.15.86:0/3922723993 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f20880a1430 0x7f20880a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:00.770 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.770+0000 7f20bbeecb80 1 -- start start 2024-03-23T17:52:00.771 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.770+0000 7f20bbeecb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d3e2078100 0x55d3e20784e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:00.772 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.770+0000 7f20bbeecb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3e2078a20 0x55d3e2083390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:00.772 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.770+0000 7f20bbeecb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d3e20838d0 0x55d3e2085cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:00.772 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.770+0000 7f20bbeecb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3e1fd1930 con 0x55d3e2078a20 2024-03-23T17:52:00.772 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.770+0000 7f20bbeecb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3e1fcf2f0 con 0x55d3e2078100 2024-03-23T17:52:00.772 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.770+0000 7f20bbeecb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3e1fcabe0 con 0x55d3e20838d0 2024-03-23T17:52:00.772 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.770+0000 7f20a5000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3e2078a20 0x55d3e2083390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:00.772 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.770+0000 7f20a5a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d3e2078100 0x55d3e20784e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:00.772 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.771+0000 7f20a5a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d3e2078100 0x55d3e20784e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:35220/0 (socket says 172.21.15.86:35220) 2024-03-23T17:52:00.772 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.771+0000 7f20a5a00700 1 -- 172.21.15.86:0/4188235777 learned_addr learned my addr 172.21.15.86:0/4188235777 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:00.772 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.771+0000 7f20a6400700 1 --2- 172.21.15.86:0/4188235777 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d3e20838d0 0x55d3e2085cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:00.772 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.771+0000 7f20a5000700 1 --2- 172.21.15.86:0/3922723993 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f20880a1430 0x7f20880a38f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f20a019a880 tx=0x7f208c0073d0 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:00.772 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.771+0000 7f20a5a00700 1 -- 172.21.15.86:0/4188235777 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d3e20838d0 msgr2=0x55d3e2085cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:00.772 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.771+0000 7f20a5a00700 1 --2- 172.21.15.86:0/4188235777 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d3e20838d0 0x55d3e2085cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:00.773 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.771+0000 7f20a5a00700 1 -- 172.21.15.86:0/4188235777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3e2078a20 msgr2=0x55d3e2083390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:00.773 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.771+0000 7f20a5a00700 1 --2- 172.21.15.86:0/4188235777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3e2078a20 0x55d3e2083390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:00.773 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.771+0000 7f20a5a00700 1 -- 172.21.15.86:0/4188235777 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3e1e4c350 con 0x55d3e2078100 2024-03-23T17:52:00.773 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.771+0000 7f20a5000700 1 --2- 172.21.15.86:0/4188235777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3e2078a20 0x55d3e2083390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:52:00.773 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.771+0000 7f20a5a00700 1 --2- 172.21.15.86:0/4188235777 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d3e2078100 0x55d3e20784e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f209409a9c0 tx=0x7f2094003680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=573bfd846a1375e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:00.773 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.771+0000 7f2092a00700 1 -- 172.21.15.86:0/4188235777 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f209409e860 con 0x55d3e2078100 2024-03-23T17:52:00.773 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.771+0000 7f2092a00700 1 -- 172.21.15.86:0/4188235777 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f20940a2e20 con 0x55d3e2078100 2024-03-23T17:52:00.773 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.771+0000 7f2092a00700 1 -- 172.21.15.86:0/4188235777 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f209409a180 con 0x55d3e2078100 2024-03-23T17:52:00.777 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.772+0000 7f20bbeecb80 1 -- 172.21.15.86:0/4188235777 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d3e2078100 msgr2=0x55d3e20784e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:00.777 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.772+0000 7f20bbeecb80 1 --2- 172.21.15.86:0/4188235777 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d3e2078100 0x55d3e20784e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f209409a9c0 tx=0x7f2094003680 comp rx=0 tx=0).stop 2024-03-23T17:52:00.778 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.772+0000 7f20bbeecb80 1 -- 172.21.15.86:0/4188235777 shutdown_connections 2024-03-23T17:52:00.778 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.772+0000 7f20bbeecb80 1 --2- 172.21.15.86:0/4188235777 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d3e20838d0 0x55d3e2085cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:00.778 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.772+0000 7f20bbeecb80 1 --2- 172.21.15.86:0/4188235777 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d3e2078100 0x55d3e20784e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:00.778 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.772+0000 7f20bbeecb80 1 --2- 172.21.15.86:0/4188235777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3e2078a20 0x55d3e2083390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:00.778 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.772+0000 7f20bbeecb80 1 -- 172.21.15.86:0/4188235777 >> 172.21.15.86:0/4188235777 conn(0x55d3e1fd8640 msgr2=0x55d3e2074520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:00.778 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.772+0000 7f20bbeecb80 1 -- 172.21.15.86:0/4188235777 shutdown_connections 2024-03-23T17:52:00.778 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.772+0000 7f20bbeecb80 1 -- 172.21.15.86:0/4188235777 wait complete. 2024-03-23T17:52:00.778 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.773+0000 7f20bbeecb80 1 -- start start 2024-03-23T17:52:00.778 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.773+0000 7f20bbeecb80 1 Processor -- start 2024-03-23T17:52:00.778 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.773+0000 7f20bbeecb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d3e2078100 0x55d3e20832d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:00.778 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.773+0000 7f20bbeecb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3e2078a20 0x55d3e20805c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:00.778 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.773+0000 7f20bbeecb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d3e20838d0 0x55d3e2080bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:00.779 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.773+0000 7f20bbeecb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3e1f66d20 con 0x55d3e2078a20 2024-03-23T17:52:00.779 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.773+0000 7f20bbeecb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3e1f669c0 con 0x55d3e2078100 2024-03-23T17:52:00.779 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.773+0000 7f20bbeecb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3e1e4e640 con 0x55d3e20838d0 2024-03-23T17:52:00.779 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.773+0000 7f20bbeecb80 10 client.0 _ll_register_callbacks cb 0x55d3e1f70410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:52:00.779 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.774+0000 7f20a5000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3e2078a20 0x55d3e20805c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:00.779 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.774+0000 7f20a5000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3e2078a20 0x55d3e20805c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59522/0 (socket says 172.21.15.86:59522) 2024-03-23T17:52:00.779 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.774+0000 7f20a5000700 1 -- 172.21.15.86:0/901112234 learned_addr learned my addr 172.21.15.86:0/901112234 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:00.779 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.774+0000 7f20a5a00700 1 --2- 172.21.15.86:0/901112234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d3e2078100 0x55d3e20832d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:00.779 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.774+0000 7f20a6400700 1 --2- 172.21.15.86:0/901112234 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d3e20838d0 0x55d3e2080bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:00.779 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.774+0000 7f20a5000700 1 -- 172.21.15.86:0/901112234 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d3e20838d0 msgr2=0x55d3e2080bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:00.779 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.774+0000 7f20a5000700 1 --2- 172.21.15.86:0/901112234 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d3e20838d0 0x55d3e2080bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:00.779 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.774+0000 7f20a5000700 1 -- 172.21.15.86:0/901112234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d3e2078100 msgr2=0x55d3e20832d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:00.779 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.774+0000 7f20a5000700 1 --2- 172.21.15.86:0/901112234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d3e2078100 0x55d3e20832d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:00.779 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.774+0000 7f20a5000700 1 -- 172.21.15.86:0/901112234 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f208c010a30 con 0x55d3e2078a20 2024-03-23T17:52:00.780 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.774+0000 7f20a5000700 1 --2- 172.21.15.86:0/901112234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3e2078a20 0x55d3e20805c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f208c011710 tx=0x7f208c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8e6d61a924c3e03 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:00.780 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.774+0000 7f2087400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-23T17:52:00.780 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.775+0000 7f2087400700 1 -- 172.21.15.86:0/901112234 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f208c004220 con 0x55d3e2078a20 2024-03-23T17:52:00.780 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.775+0000 7f20bbeecb80 1 -- 172.21.15.86:0/901112234 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d3e1f8b320 con 0x55d3e2078a20 2024-03-23T17:52:00.780 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.775+0000 7f2087400700 1 -- 172.21.15.86:0/901112234 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f208c017070 con 0x55d3e2078a20 2024-03-23T17:52:00.780 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.775+0000 7f2087400700 1 -- 172.21.15.86:0/901112234 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f208c004380 con 0x55d3e2078a20 2024-03-23T17:52:00.780 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.775+0000 7f20bbeecb80 10 client.15831 fetch_fsmap learned FSMap version 30 2024-03-23T17:52:00.780 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.775+0000 7f20bbeecb80 1 -- 172.21.15.86:0/901112234 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f20a019e800 con 0x55d3e2078a20 2024-03-23T17:52:00.780 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.775+0000 7f2087400700 1 -- 172.21.15.86:0/901112234 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d3e1f8b320 con 0x55d3e2078a20 2024-03-23T17:52:00.780 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.775+0000 7f2087400700 1 -- 172.21.15.86:0/901112234 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f208c01e810 con 0x55d3e2078a20 2024-03-23T17:52:00.780 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.775+0000 7f20bbeecb80 10 client.15831 fetch_fsmap finished waiting for FSMap version 30 2024-03-23T17:52:00.780 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.775+0000 7f20bbeecb80 10 client.15831 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:52:00.780 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.775+0000 7f20bbeecb80 20 client.15831 populate_metadata read hostname 'smithi086' 2024-03-23T17:52:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.775+0000 7f20bbeecb80 1 --2- 172.21.15.86:0/901112234 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55d3e20932b0 0x55d3e20956d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.775+0000 7f20bbeecb80 4 client.15831 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:52:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.775+0000 7f20bbeecb80 1 -- 172.21.15.86:0/901112234 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d3e1fc36a0 con 0x55d3e20932b0 2024-03-23T17:52:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.776+0000 7f20a5a00700 1 --2- 172.21.15.86:0/901112234 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55d3e20932b0 0x55d3e20956d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.776+0000 7f2087400700 10 client.15831 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:52:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.776+0000 7f20a5a00700 1 --2- 172.21.15.86:0/901112234 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55d3e20932b0 0x55d3e20956d0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b703656ea3e5a65b server_cookie=57817c9e0e344007 in_seq=0 out_seq=0 2024-03-23T17:52:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.777+0000 7f2087400700 1 -- 172.21.15.86:0/901112234 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55d3e1fc36a0 con 0x55d3e20932b0 2024-03-23T17:52:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.777+0000 7f2087400700 10 client.15831 handle_command_reply: tid=0 2024-03-23T17:52:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.777+0000 7f20bbeecb80 1 client.15831 shutdown 2024-03-23T17:52:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.777+0000 7f20bbeecb80 1 -- 172.21.15.86:0/901112234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3e2078a20 msgr2=0x55d3e20805c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.777+0000 7f20bbeecb80 1 --2- 172.21.15.86:0/901112234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3e2078a20 0x55d3e20805c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f208c011710 tx=0x7f208c0047c0 comp rx=0 tx=0).stop 2024-03-23T17:52:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.777+0000 7f20bbeecb80 1 -- 172.21.15.86:0/901112234 shutdown_connections 2024-03-23T17:52:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.777+0000 7f20bbeecb80 1 --2- 172.21.15.86:0/901112234 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55d3e20932b0 0x55d3e20956d0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.777+0000 7f20bbeecb80 1 --2- 172.21.15.86:0/901112234 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d3e20838d0 0x55d3e2080bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.777+0000 7f20bbeecb80 1 --2- 172.21.15.86:0/901112234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d3e2078100 0x55d3e20832d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.777+0000 7f20bbeecb80 1 --2- 172.21.15.86:0/901112234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3e2078a20 0x55d3e20805c0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.777+0000 7f2087400700 0 client.15831 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:52:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.777+0000 7f20bbeecb80 1 -- 172.21.15.86:0/901112234 >> 172.21.15.86:0/901112234 conn(0x55d3e1fd8640 msgr2=0x55d3e207a080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.777+0000 7f20bbeecb80 1 -- 172.21.15.86:0/901112234 shutdown_connections 2024-03-23T17:52:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.777+0000 7f20bbeecb80 1 -- 172.21.15.86:0/901112234 wait complete. 2024-03-23T17:52:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.778+0000 7f20bbeecb80 20 client.15831 trim_cache size 0 max 16384 2024-03-23T17:52:00.794 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.792+0000 7f20bbeecb80 1 Processor -- start 2024-03-23T17:52:00.794 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.792+0000 7f20bbeecb80 1 -- start start 2024-03-23T17:52:00.794 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.793+0000 7f20bbeecb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d3e2078100 0x55d3e208a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:00.794 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.793+0000 7f20bbeecb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3e2078a20 0x55d3e207f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:00.794 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.793+0000 7f20bbeecb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d3e20838d0 0x55d3e2080b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:00.794 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.793+0000 7f20bbeecb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3e2081bc0 con 0x55d3e2078a20 2024-03-23T17:52:00.794 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.793+0000 7f20bbeecb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3e20752d0 con 0x55d3e2078100 2024-03-23T17:52:00.795 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.793+0000 7f20bbeecb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3e209a3f0 con 0x55d3e20838d0 2024-03-23T17:52:00.795 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.793+0000 7f20a5000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3e2078a20 0x55d3e207f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:00.795 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.793+0000 7f20a5a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d3e2078100 0x55d3e208a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:00.795 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.793+0000 7f20a5a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d3e2078100 0x55d3e208a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:35228/0 (socket says 172.21.15.86:35228) 2024-03-23T17:52:00.795 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.793+0000 7f20a5000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3e2078a20 0x55d3e207f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59538/0 (socket says 172.21.15.86:59538) 2024-03-23T17:52:00.795 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.793+0000 7f20a5a00700 1 -- 172.21.15.86:0/2897820786 learned_addr learned my addr 172.21.15.86:0/2897820786 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:00.795 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.793+0000 7f20a6400700 1 --2- 172.21.15.86:0/2897820786 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d3e20838d0 0x55d3e2080b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:00.795 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.793+0000 7f20a5a00700 1 -- 172.21.15.86:0/2897820786 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d3e20838d0 msgr2=0x55d3e2080b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:00.795 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.793+0000 7f20a5a00700 1 --2- 172.21.15.86:0/2897820786 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d3e20838d0 0x55d3e2080b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:00.795 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.793+0000 7f20a5a00700 1 -- 172.21.15.86:0/2897820786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3e2078a20 msgr2=0x55d3e207f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:00.795 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.793+0000 7f20a5a00700 1 --2- 172.21.15.86:0/2897820786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3e2078a20 0x55d3e207f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:00.795 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.793+0000 7f20a5a00700 1 -- 172.21.15.86:0/2897820786 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3e1df8b00 con 0x55d3e2078100 2024-03-23T17:52:00.795 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.793+0000 7f20a5000700 1 --2- 172.21.15.86:0/2897820786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3e2078a20 0x55d3e207f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:52:00.795 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.794+0000 7f20a5a00700 1 --2- 172.21.15.86:0/2897820786 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d3e2078100 0x55d3e208a2c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f20940974a0 tx=0x7f20940ba040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6dcc4d9ac8c63707 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:00.796 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.794+0000 7f2093400700 1 -- 172.21.15.86:0/2897820786 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20940caff0 con 0x55d3e2078100 2024-03-23T17:52:00.796 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.794+0000 7f2093400700 1 -- 172.21.15.86:0/2897820786 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f20940b3120 con 0x55d3e2078100 2024-03-23T17:52:00.796 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.794+0000 7f2093400700 1 -- 172.21.15.86:0/2897820786 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20940a2990 con 0x55d3e2078100 2024-03-23T17:52:00.799 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.795+0000 7f20bbeecb80 1 -- 172.21.15.86:0/2897820786 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d3e2078100 msgr2=0x55d3e208a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:00.800 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.795+0000 7f20bbeecb80 1 --2- 172.21.15.86:0/2897820786 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d3e2078100 0x55d3e208a2c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f20940974a0 tx=0x7f20940ba040 comp rx=0 tx=0).stop 2024-03-23T17:52:00.800 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.795+0000 7f20bbeecb80 1 -- 172.21.15.86:0/2897820786 shutdown_connections 2024-03-23T17:52:00.800 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.795+0000 7f20bbeecb80 1 --2- 172.21.15.86:0/2897820786 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d3e20838d0 0x55d3e2080b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:00.800 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.795+0000 7f20bbeecb80 1 --2- 172.21.15.86:0/2897820786 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d3e2078100 0x55d3e208a2c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:00.800 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.795+0000 7f20bbeecb80 1 --2- 172.21.15.86:0/2897820786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3e2078a20 0x55d3e207f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:00.800 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.795+0000 7f20bbeecb80 1 -- 172.21.15.86:0/2897820786 >> 172.21.15.86:0/2897820786 conn(0x55d3e1fd8640 msgr2=0x55d3e2073d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:00.800 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.795+0000 7f20bbeecb80 1 -- 172.21.15.86:0/2897820786 shutdown_connections 2024-03-23T17:52:00.800 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.795+0000 7f20bbeecb80 1 -- 172.21.15.86:0/2897820786 wait complete. 2024-03-23T17:52:00.800 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.795+0000 7f20bbeecb80 1 -- start start 2024-03-23T17:52:00.800 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.795+0000 7f20bbeecb80 1 Processor -- start 2024-03-23T17:52:00.800 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.796+0000 7f20bbeecb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d3e2078100 0x55d3e2087b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:00.801 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.796+0000 7f20bbeecb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3e2078a20 0x55d3e2088070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:00.801 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.796+0000 7f20bbeecb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d3e20838d0 0x55d3e20e3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:00.801 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.796+0000 7f20bbeecb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3e2088850 con 0x55d3e2078a20 2024-03-23T17:52:00.801 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.796+0000 7f20bbeecb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3e20889c0 con 0x55d3e20838d0 2024-03-23T17:52:00.801 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.796+0000 7f20bbeecb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3e20e4360 con 0x55d3e2078100 2024-03-23T17:52:00.801 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.796+0000 7f20bbeecb80 10 client.0 _ll_register_callbacks cb 0x55d3e1f70410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:52:00.801 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.796+0000 7f20a5a00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d3e2078100 0x55d3e2087b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:00.801 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.796+0000 7f20a5000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3e2078a20 0x55d3e2088070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:00.801 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.796+0000 7f20a6400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d3e20838d0 0x55d3e20e3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:00.801 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.796+0000 7f20a5a00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d3e2078100 0x55d3e2087b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:36786/0 (socket says 172.21.15.86:36786) 2024-03-23T17:52:00.802 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.796+0000 7f20a5a00700 1 -- 172.21.15.86:0/767372981 learned_addr learned my addr 172.21.15.86:0/767372981 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:00.802 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.796+0000 7f20a5000700 1 -- 172.21.15.86:0/767372981 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d3e2078100 msgr2=0x55d3e2087b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:00.802 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.796+0000 7f20a5000700 1 --2- 172.21.15.86:0/767372981 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d3e2078100 0x55d3e2087b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:00.802 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.796+0000 7f20a5000700 1 -- 172.21.15.86:0/767372981 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d3e20838d0 msgr2=0x55d3e20e3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:00.802 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.796+0000 7f20a5000700 1 --2- 172.21.15.86:0/767372981 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d3e20838d0 0x55d3e20e3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:00.802 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.797+0000 7f20a5000700 1 -- 172.21.15.86:0/767372981 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20a019e800 con 0x55d3e2078a20 2024-03-23T17:52:00.802 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.797+0000 7f20a5a00700 1 --2- 172.21.15.86:0/767372981 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d3e2078100 0x55d3e2087b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:52:00.802 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.797+0000 7f20a6400700 1 --2- 172.21.15.86:0/767372981 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d3e20838d0 0x55d3e20e3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:52:00.802 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.797+0000 7f20a5000700 1 --2- 172.21.15.86:0/767372981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3e2078a20 0x55d3e2088070 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f208c0116e0 tx=0x7f208c005f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=414785d6f82abd0f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:00.802 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.797+0000 7f2087400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-23T17:52:00.803 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.797+0000 7f2087400700 1 -- 172.21.15.86:0/767372981 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f208c0318b0 con 0x55d3e2078a20 2024-03-23T17:52:00.803 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.797+0000 7f2087400700 1 -- 172.21.15.86:0/767372981 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f208c017040 con 0x55d3e2078a20 2024-03-23T17:52:00.803 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.797+0000 7f20bbeecb80 1 -- 172.21.15.86:0/767372981 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f20940a2e20 con 0x55d3e2078a20 2024-03-23T17:52:00.803 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.797+0000 7f2087400700 1 -- 172.21.15.86:0/767372981 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f208c007440 con 0x55d3e2078a20 2024-03-23T17:52:00.803 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.797+0000 7f20bbeecb80 10 client.15837 fetch_fsmap learned FSMap version 30 2024-03-23T17:52:00.803 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.797+0000 7f20bbeecb80 1 -- 172.21.15.86:0/767372981 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55d3e1e4c350 con 0x55d3e2078a20 2024-03-23T17:52:00.803 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.797+0000 7f2087400700 1 -- 172.21.15.86:0/767372981 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f20940a2e20 con 0x55d3e2078a20 2024-03-23T17:52:00.803 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.798+0000 7f2087400700 1 -- 172.21.15.86:0/767372981 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f208c039b60 con 0x55d3e2078a20 2024-03-23T17:52:00.803 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.798+0000 7f20bbeecb80 10 client.15837 fetch_fsmap finished waiting for FSMap version 30 2024-03-23T17:52:00.803 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.798+0000 7f20bbeecb80 10 client.15837 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:52:00.804 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.798+0000 7f20bbeecb80 20 client.15837 populate_metadata read hostname 'smithi086' 2024-03-23T17:52:00.804 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.798+0000 7f20bbeecb80 1 --2- 172.21.15.86:0/767372981 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55d3e20e5bf0 0x55d3e20e8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:00.804 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.798+0000 7f20bbeecb80 4 client.15837 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T17:52:00.804 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.798+0000 7f20bbeecb80 1 -- 172.21.15.86:0/767372981 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d3e1e19c90 con 0x55d3e20e5bf0 2024-03-23T17:52:00.804 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.798+0000 7f20a5a00700 1 --2- 172.21.15.86:0/767372981 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55d3e20e5bf0 0x55d3e20e8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:00.804 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.798+0000 7f20a5a00700 1 --2- 172.21.15.86:0/767372981 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55d3e20e5bf0 0x55d3e20e8010 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=576ce6deffa039bc server_cookie=c4fcd25cd0bd7ec9 in_seq=0 out_seq=0 2024-03-23T17:52:00.804 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.798+0000 7f2087400700 10 client.15837 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:52:00.822 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.821+0000 7f2087400700 1 -- 172.21.15.86:0/767372981 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d3e1e19c90 con 0x55d3e20e5bf0 2024-03-23T17:52:00.822 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.821+0000 7f2087400700 10 client.15837 handle_command_reply: tid=0 2024-03-23T17:52:00.822 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.821+0000 7f20bbeecb80 1 client.15837 shutdown 2024-03-23T17:52:00.822 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.821+0000 7f20bbeecb80 1 -- 172.21.15.86:0/767372981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3e2078a20 msgr2=0x55d3e2088070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:00.822 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.821+0000 7f20bbeecb80 1 --2- 172.21.15.86:0/767372981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3e2078a20 0x55d3e2088070 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f208c0116e0 tx=0x7f208c005f60 comp rx=0 tx=0).stop 2024-03-23T17:52:00.822 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.821+0000 7f20bbeecb80 1 -- 172.21.15.86:0/767372981 shutdown_connections 2024-03-23T17:52:00.822 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.821+0000 7f20bbeecb80 1 --2- 172.21.15.86:0/767372981 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55d3e20e5bf0 0x55d3e20e8010 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:00.822 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.821+0000 7f20bbeecb80 1 --2- 172.21.15.86:0/767372981 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d3e20838d0 0x55d3e20e3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:00.822 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.821+0000 7f20bbeecb80 1 --2- 172.21.15.86:0/767372981 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d3e2078100 0x55d3e2087b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:00.823 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.821+0000 7f2087400700 0 client.15837 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:52:00.823 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.821+0000 7f20bbeecb80 1 --2- 172.21.15.86:0/767372981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3e2078a20 0x55d3e2088070 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:00.823 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.821+0000 7f20bbeecb80 1 -- 172.21.15.86:0/767372981 >> 172.21.15.86:0/767372981 conn(0x55d3e1fd8640 msgr2=0x55d3e206d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:00.823 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.822+0000 7f20bbeecb80 1 -- 172.21.15.86:0/767372981 shutdown_connections 2024-03-23T17:52:00.823 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.822+0000 7f20bbeecb80 1 -- 172.21.15.86:0/767372981 wait complete. 2024-03-23T17:52:00.823 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.822+0000 7f20bbeecb80 20 client.15837 trim_cache size 0 max 16384 2024-03-23T17:52:00.823 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-03-23T17:52:00.823 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-03-23T17:52:00.823 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-03-23T17:52:00.823 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-03-23T17:52:00.824 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.822+0000 7f20a8400700 1 -- 172.21.15.86:0/3922723993 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f20880a1430 msgr2=0x7f20880a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:00.824 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.822+0000 7f20a8400700 1 --2- 172.21.15.86:0/3922723993 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f20880a1430 0x7f20880a38f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f20a019a880 tx=0x7f208c0073d0 comp rx=0 tx=0).stop 2024-03-23T17:52:00.824 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.822+0000 7f20a8400700 1 -- 172.21.15.86:0/3922723993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20a0101f20 msgr2=0x7f20a0199330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:00.824 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.822+0000 7f20a8400700 1 --2- 172.21.15.86:0/3922723993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20a0101f20 0x7f20a0199330 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f20940095c0 tx=0x7f209400f690 comp rx=0 tx=0).stop 2024-03-23T17:52:00.824 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.822+0000 7f20a8400700 1 -- 172.21.15.86:0/3922723993 shutdown_connections 2024-03-23T17:52:00.824 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.822+0000 7f20a8400700 1 --2- 172.21.15.86:0/3922723993 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20a010b2f0 0x7f20a019dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:00.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.822+0000 7f20a8400700 1 --2- 172.21.15.86:0/3922723993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20a0101f20 0x7f20a0199330 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:00.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.822+0000 7f20a8400700 1 --2- 172.21.15.86:0/3922723993 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f20880a1430 0x7f20880a38f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:00.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.822+0000 7f20a8400700 1 --2- 172.21.15.86:0/3922723993 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f20a0102840 0x7f20a0199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:00.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.822+0000 7f20a8400700 1 -- 172.21.15.86:0/3922723993 >> 172.21.15.86:0/3922723993 conn(0x7f20a00fb680 msgr2=0x7f20a00fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:00.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.823+0000 7f20a8400700 1 -- 172.21.15.86:0/3922723993 shutdown_connections 2024-03-23T17:52:00.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:00.823+0000 7f20a8400700 1 -- 172.21.15.86:0/3922723993 wait complete. 2024-03-23T17:52:00.879 INFO:tasks.workunit.client.0.smithi086.stdout: 1 28743 6.95 MB/sec warmup 25 sec latency 2300.674 ms 2024-03-23T17:52:01.880 INFO:tasks.workunit.client.0.smithi086.stdout: 1 28743 6.68 MB/sec warmup 26 sec latency 1589.418 ms 2024-03-23T17:52:02.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:01 smithi092 ceph-mon[79089]: pgmap v44: 97 pgs: 97 active+clean; 25 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 3.0 MiB/s wr, 180 op/s 2024-03-23T17:52:02.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:01 smithi152 ceph-mon[47949]: pgmap v44: 97 pgs: 97 active+clean; 25 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 3.0 MiB/s wr, 180 op/s 2024-03-23T17:52:02.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:01 smithi086 ceph-mon[38221]: pgmap v44: 97 pgs: 97 active+clean; 25 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 3.0 MiB/s wr, 180 op/s 2024-03-23T17:52:02.880 INFO:tasks.workunit.client.0.smithi086.stdout: 1 28791 6.44 MB/sec warmup 27 sec latency 2453.580 ms 2024-03-23T17:52:03.880 INFO:tasks.workunit.client.0.smithi086.stdout: 1 28791 6.21 MB/sec warmup 28 sec latency 1085.612 ms 2024-03-23T17:52:04.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:03 smithi092 ceph-mon[79089]: pgmap v45: 97 pgs: 97 active+clean; 25 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 2.8 MiB/s wr, 61 op/s 2024-03-23T17:52:04.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:03 smithi152 ceph-mon[47949]: pgmap v45: 97 pgs: 97 active+clean; 25 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 2.8 MiB/s wr, 61 op/s 2024-03-23T17:52:04.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:03 smithi086 ceph-mon[38221]: pgmap v45: 97 pgs: 97 active+clean; 25 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 2.8 MiB/s wr, 61 op/s 2024-03-23T17:52:04.880 INFO:tasks.workunit.client.0.smithi086.stdout: 1 28791 6.00 MB/sec warmup 29 sec latency 2085.748 ms 2024-03-23T17:52:05.880 INFO:tasks.workunit.client.0.smithi086.stdout: 1 28791 5.80 MB/sec warmup 30 sec latency 3085.823 ms 2024-03-23T17:52:06.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:05 smithi092 ceph-mon[79089]: pgmap v46: 97 pgs: 97 active+clean; 25 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 3.1 MiB/s wr, 81 op/s 2024-03-23T17:52:06.085 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:05 smithi152 ceph-mon[47949]: pgmap v46: 97 pgs: 97 active+clean; 25 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 3.1 MiB/s wr, 81 op/s 2024-03-23T17:52:06.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:05 smithi086 ceph-mon[38221]: pgmap v46: 97 pgs: 97 active+clean; 25 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 3.1 MiB/s wr, 81 op/s 2024-03-23T17:52:06.880 INFO:tasks.workunit.client.0.smithi086.stdout: 1 28791 5.61 MB/sec warmup 31 sec latency 4085.962 ms 2024-03-23T17:52:07.071 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-23T17:52:07.071 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-23T17:52:07.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.276+0000 7f96b8e00700 1 -- 172.21.15.86:0/936725756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96b4105ce0 msgr2=0x7f96b41060c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:07.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.276+0000 7f96b8e00700 1 --2- 172.21.15.86:0/936725756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96b4105ce0 0x7f96b41060c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f96a4009990 tx=0x7f96a4009ca0 comp rx=0 tx=0).stop 2024-03-23T17:52:07.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.276+0000 7f96b8e00700 1 -- 172.21.15.86:0/936725756 shutdown_connections 2024-03-23T17:52:07.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.276+0000 7f96b8e00700 1 --2- 172.21.15.86:0/936725756 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f96b40687f0 0x7f96b410db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:07.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.276+0000 7f96b8e00700 1 --2- 172.21.15.86:0/936725756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96b4105ce0 0x7f96b41060c0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:07.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.276+0000 7f96b8e00700 1 --2- 172.21.15.86:0/936725756 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96b4067e30 0x7f96b40682b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:07.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.276+0000 7f96b8e00700 1 -- 172.21.15.86:0/936725756 >> 172.21.15.86:0/936725756 conn(0x7f96b4076d40 msgr2=0x7f96b4077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:07.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.276+0000 7f96b8e00700 1 -- 172.21.15.86:0/936725756 shutdown_connections 2024-03-23T17:52:07.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.276+0000 7f96b8e00700 1 -- 172.21.15.86:0/936725756 wait complete. 2024-03-23T17:52:07.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.277+0000 7f96b8e00700 1 Processor -- start 2024-03-23T17:52:07.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.277+0000 7f96b8e00700 1 -- start start 2024-03-23T17:52:07.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.278+0000 7f96b8e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96b4067e30 0x7f96b4199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:07.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.278+0000 7f96b8e00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f96b40687f0 0x7f96b41999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:07.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.278+0000 7f96b8e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96b4105ce0 0x7f96b419dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:07.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.278+0000 7f96b8e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b419e380 con 0x7f96b4105ce0 2024-03-23T17:52:07.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.278+0000 7f96b8e00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b419e4f0 con 0x7f96b4067e30 2024-03-23T17:52:07.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.278+0000 7f96b8e00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b419e6b0 con 0x7f96b40687f0 2024-03-23T17:52:07.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.278+0000 7f96b1e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96b4067e30 0x7f96b4199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:07.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.278+0000 7f96b1e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96b4067e30 0x7f96b4199460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:41140/0 (socket says 172.21.15.86:41140) 2024-03-23T17:52:07.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.278+0000 7f96b1e00700 1 -- 172.21.15.86:0/3329262187 learned_addr learned my addr 172.21.15.86:0/3329262187 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:07.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.278+0000 7f96b2800700 1 --2- 172.21.15.86:0/3329262187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96b4105ce0 0x7f96b419dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:07.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.278+0000 7f96abe00700 1 --2- 172.21.15.86:0/3329262187 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f96b40687f0 0x7f96b41999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:07.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.278+0000 7f96b2800700 1 -- 172.21.15.86:0/3329262187 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f96b40687f0 msgr2=0x7f96b41999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:07.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.278+0000 7f96b2800700 1 --2- 172.21.15.86:0/3329262187 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f96b40687f0 0x7f96b41999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:07.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.278+0000 7f96b2800700 1 -- 172.21.15.86:0/3329262187 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96b4067e30 msgr2=0x7f96b4199460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:07.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.278+0000 7f96b2800700 1 --2- 172.21.15.86:0/3329262187 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96b4067e30 0x7f96b4199460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:07.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.278+0000 7f96b2800700 1 -- 172.21.15.86:0/3329262187 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96a40095f0 con 0x7f96b4105ce0 2024-03-23T17:52:07.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.278+0000 7f96b1e00700 1 --2- 172.21.15.86:0/3329262187 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96b4067e30 0x7f96b4199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:52:07.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.279+0000 7f96b2800700 1 --2- 172.21.15.86:0/3329262187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96b4105ce0 0x7f96b419dd40 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f96ac00efc0 tx=0x7f96ac00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:07.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.279+0000 7f96aaa00700 1 -- 172.21.15.86:0/3329262187 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96ac018070 con 0x7f96b4105ce0 2024-03-23T17:52:07.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.279+0000 7f96aaa00700 1 -- 172.21.15.86:0/3329262187 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f96ac00f040 con 0x7f96b4105ce0 2024-03-23T17:52:07.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.279+0000 7f96b8e00700 1 -- 172.21.15.86:0/3329262187 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96b419e930 con 0x7f96b4105ce0 2024-03-23T17:52:07.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.279+0000 7f96aaa00700 1 -- 172.21.15.86:0/3329262187 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96ac00cd40 con 0x7f96b4105ce0 2024-03-23T17:52:07.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.279+0000 7f96b8e00700 1 -- 172.21.15.86:0/3329262187 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96b41a61d0 con 0x7f96b4105ce0 2024-03-23T17:52:07.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.282+0000 7f96aaa00700 1 -- 172.21.15.86:0/3329262187 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) v1 ==== 136098+0+0 (secure 0 0 0) 0x7f96ac003a00 con 0x7f96b4105ce0 2024-03-23T17:52:07.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.283+0000 7f96cc910b80 1 Processor -- start 2024-03-23T17:52:07.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.283+0000 7f96aaa00700 1 --2- 172.21.15.86:0/3329262187 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f96980a13b0 0x7f96980a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:07.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.283+0000 7f96cc910b80 1 -- start start 2024-03-23T17:52:07.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.283+0000 7f96aaa00700 1 -- 172.21.15.86:0/3329262187 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f96ac0ccab0 con 0x7f96b4105ce0 2024-03-23T17:52:07.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.283+0000 7f96abe00700 1 --2- 172.21.15.86:0/3329262187 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f96980a13b0 0x7f96980a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:07.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.284+0000 7f96cc910b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b71007a5d0 0x55b71007a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:07.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.284+0000 7f96cc910b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b71007aef0 0x55b7100858b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:07.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.284+0000 7f96cc910b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b710085df0 0x55b7100881e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:07.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.284+0000 7f96cc910b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b70ffd3990 con 0x55b71007a5d0 2024-03-23T17:52:07.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.284+0000 7f96cc910b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b70ffd1350 con 0x55b71007aef0 2024-03-23T17:52:07.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.284+0000 7f96cc910b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b70ffccc40 con 0x55b710085df0 2024-03-23T17:52:07.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.284+0000 7f96b1e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b71007a5d0 0x55b71007a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:07.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.284+0000 7f96b2800700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b710085df0 0x55b7100881e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:07.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.284+0000 7f96abe00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b71007aef0 0x55b7100858b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:07.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.284+0000 7f96b2800700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b710085df0 0x55b7100881e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:50878/0 (socket says 172.21.15.86:50878) 2024-03-23T17:52:07.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.284+0000 7f96b1e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b71007a5d0 0x55b71007a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58970/0 (socket says 172.21.15.86:58970) 2024-03-23T17:52:07.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.284+0000 7f96b2800700 1 -- 172.21.15.86:0/996734605 learned_addr learned my addr 172.21.15.86:0/996734605 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:07.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.284+0000 7f96abe00700 1 --2- 172.21.15.86:0/3329262187 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f96980a13b0 0x7f96980a3870 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f96b419a9b0 tx=0x7f969c0106b0 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:07.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.284+0000 7f96abe00700 1 -- 172.21.15.86:0/996734605 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b710085df0 msgr2=0x55b7100881e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:07.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.284+0000 7f96abe00700 1 --2- 172.21.15.86:0/996734605 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b710085df0 0x55b7100881e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:07.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.284+0000 7f96abe00700 1 -- 172.21.15.86:0/996734605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b71007a5d0 msgr2=0x55b71007a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:07.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.284+0000 7f96abe00700 1 --2- 172.21.15.86:0/996734605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b71007a5d0 0x55b71007a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:07.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.284+0000 7f96abe00700 1 -- 172.21.15.86:0/996734605 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b70fe4e3b0 con 0x55b71007aef0 2024-03-23T17:52:07.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.284+0000 7f96b2800700 1 --2- 172.21.15.86:0/996734605 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b710085df0 0x55b7100881e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:52:07.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.285+0000 7f96abe00700 1 --2- 172.21.15.86:0/996734605 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b71007aef0 0x55b7100858b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f969c010f80 tx=0x7f969c01b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a991775f74251b3b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:07.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.285+0000 7f96a3400700 1 -- 172.21.15.86:0/996734605 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969c0132d0 con 0x55b71007aef0 2024-03-23T17:52:07.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.285+0000 7f96a3400700 1 -- 172.21.15.86:0/996734605 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f969c013bd0 con 0x55b71007aef0 2024-03-23T17:52:07.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.285+0000 7f96a3400700 1 -- 172.21.15.86:0/996734605 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969c01e420 con 0x55b71007aef0 2024-03-23T17:52:07.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.286+0000 7f96cc910b80 1 -- 172.21.15.86:0/996734605 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b71007aef0 msgr2=0x55b7100858b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:07.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.286+0000 7f96cc910b80 1 --2- 172.21.15.86:0/996734605 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b71007aef0 0x55b7100858b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f969c010f80 tx=0x7f969c01b000 comp rx=0 tx=0).stop 2024-03-23T17:52:07.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.286+0000 7f96cc910b80 1 -- 172.21.15.86:0/996734605 shutdown_connections 2024-03-23T17:52:07.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.286+0000 7f96cc910b80 1 --2- 172.21.15.86:0/996734605 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b710085df0 0x55b7100881e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:07.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.286+0000 7f96cc910b80 1 --2- 172.21.15.86:0/996734605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b71007a5d0 0x55b71007a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:07.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.286+0000 7f96cc910b80 1 --2- 172.21.15.86:0/996734605 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b71007aef0 0x55b7100858b0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:07.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.286+0000 7f96cc910b80 1 -- 172.21.15.86:0/996734605 >> 172.21.15.86:0/996734605 conn(0x55b70ffda6a0 msgr2=0x55b710076a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:07.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.286+0000 7f96cc910b80 1 -- 172.21.15.86:0/996734605 shutdown_connections 2024-03-23T17:52:07.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.286+0000 7f96cc910b80 1 -- 172.21.15.86:0/996734605 wait complete. 2024-03-23T17:52:07.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.286+0000 7f96cc910b80 1 -- start start 2024-03-23T17:52:07.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.287+0000 7f96cc910b80 1 Processor -- start 2024-03-23T17:52:07.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.287+0000 7f96cc910b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b71007a5d0 0x55b7100858a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:07.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.287+0000 7f96cc910b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b71007aef0 0x55b710082b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:07.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.287+0000 7f96cc910b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b710085df0 0x55b7100832d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:07.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.287+0000 7f96cc910b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b70ff68d80 con 0x55b710085df0 2024-03-23T17:52:07.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.287+0000 7f96cc910b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b70ff68a20 con 0x55b71007aef0 2024-03-23T17:52:07.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.287+0000 7f96cc910b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b70fe506a0 con 0x55b71007a5d0 2024-03-23T17:52:07.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.287+0000 7f96cc910b80 10 client.0 _ll_register_callbacks cb 0x55b70ff72470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:52:07.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.287+0000 7f96b2800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b710085df0 0x55b7100832d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:07.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.287+0000 7f96b2800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b710085df0 0x55b7100832d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58976/0 (socket says 172.21.15.86:58976) 2024-03-23T17:52:07.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.287+0000 7f96b2800700 1 -- 172.21.15.86:0/1761650084 learned_addr learned my addr 172.21.15.86:0/1761650084 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:07.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.287+0000 7f96b1e00700 1 --2- 172.21.15.86:0/1761650084 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b71007a5d0 0x55b7100858a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:07.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.287+0000 7f96abe00700 1 --2- 172.21.15.86:0/1761650084 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b71007aef0 0x55b710082b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:07.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.288+0000 7f96b2800700 1 -- 172.21.15.86:0/1761650084 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b71007a5d0 msgr2=0x55b7100858a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:07.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.288+0000 7f96b2800700 1 --2- 172.21.15.86:0/1761650084 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b71007a5d0 0x55b7100858a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:07.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.288+0000 7f96b2800700 1 -- 172.21.15.86:0/1761650084 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b71007aef0 msgr2=0x55b710082b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:07.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.288+0000 7f96b2800700 1 --2- 172.21.15.86:0/1761650084 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b71007aef0 0x55b710082b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:07.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.288+0000 7f96b2800700 1 -- 172.21.15.86:0/1761650084 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96b41a61d0 con 0x55b710085df0 2024-03-23T17:52:07.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.288+0000 7f96b2800700 1 --2- 172.21.15.86:0/1761650084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b710085df0 0x55b7100832d0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f96ac0090a0 tx=0x7f96ac099620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf240de2538a4ff0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:07.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.288+0000 7f9697e00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-23T17:52:07.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.288+0000 7f9697e00700 1 -- 172.21.15.86:0/1761650084 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96ac01b320 con 0x55b710085df0 2024-03-23T17:52:07.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.288+0000 7f9697e00700 1 -- 172.21.15.86:0/1761650084 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f96ac004ad0 con 0x55b710085df0 2024-03-23T17:52:07.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.288+0000 7f96cc910b80 1 -- 172.21.15.86:0/1761650084 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b70ff8d380 con 0x55b710085df0 2024-03-23T17:52:07.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.288+0000 7f9697e00700 1 -- 172.21.15.86:0/1761650084 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f96ac09a0f0 con 0x55b710085df0 2024-03-23T17:52:07.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.288+0000 7f96cc910b80 10 client.15849 fetch_fsmap learned FSMap version 30 2024-03-23T17:52:07.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.288+0000 7f96cc910b80 1 -- 172.21.15.86:0/1761650084 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f969c010c60 con 0x55b710085df0 2024-03-23T17:52:07.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.289+0000 7f9697e00700 1 -- 172.21.15.86:0/1761650084 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f96ac0b6070 con 0x55b710085df0 2024-03-23T17:52:07.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.289+0000 7f9697e00700 1 -- 172.21.15.86:0/1761650084 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f96ac0a1950 con 0x55b710085df0 2024-03-23T17:52:07.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.289+0000 7f96cc910b80 10 client.15849 fetch_fsmap finished waiting for FSMap version 30 2024-03-23T17:52:07.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.289+0000 7f96cc910b80 10 client.15849 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T17:52:07.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.289+0000 7f96cc910b80 20 client.15849 populate_metadata read hostname 'smithi086' 2024-03-23T17:52:07.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.289+0000 7f96cc910b80 1 --2- 172.21.15.86:0/1761650084 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55b710095ad0 0x55b710097ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:07.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.289+0000 7f96cc910b80 4 client.15849 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:52:07.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.289+0000 7f96cc910b80 1 -- 172.21.15.86:0/1761650084 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b70ffc5700 con 0x55b710095ad0 2024-03-23T17:52:07.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.289+0000 7f96b1e00700 1 --2- 172.21.15.86:0/1761650084 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55b710095ad0 0x55b710097ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:07.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.290+0000 7f96b1e00700 1 --2- 172.21.15.86:0/1761650084 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55b710095ad0 0x55b710097ef0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c69029e2ab1e5f23 server_cookie=20329fe536750b85 in_seq=0 out_seq=0 2024-03-23T17:52:07.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.290+0000 7f9697e00700 10 client.15849 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:52:07.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.290+0000 7f9697e00700 1 -- 172.21.15.86:0/1761650084 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55b70ffc5700 con 0x55b710095ad0 2024-03-23T17:52:07.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.290+0000 7f9697e00700 10 client.15849 handle_command_reply: tid=0 2024-03-23T17:52:07.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.290+0000 7f96cc910b80 1 client.15849 shutdown 2024-03-23T17:52:07.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.290+0000 7f96cc910b80 1 -- 172.21.15.86:0/1761650084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b710085df0 msgr2=0x55b7100832d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:07.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.291+0000 7f96cc910b80 1 --2- 172.21.15.86:0/1761650084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b710085df0 0x55b7100832d0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f96ac0090a0 tx=0x7f96ac099620 comp rx=0 tx=0).stop 2024-03-23T17:52:07.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.291+0000 7f96cc910b80 1 -- 172.21.15.86:0/1761650084 shutdown_connections 2024-03-23T17:52:07.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.291+0000 7f96cc910b80 1 --2- 172.21.15.86:0/1761650084 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55b710095ad0 0x55b710097ef0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:07.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.291+0000 7f96cc910b80 1 --2- 172.21.15.86:0/1761650084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b710085df0 0x55b7100832d0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:07.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.291+0000 7f96cc910b80 1 --2- 172.21.15.86:0/1761650084 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b71007a5d0 0x55b7100858a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:07.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.291+0000 7f96cc910b80 1 --2- 172.21.15.86:0/1761650084 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b71007aef0 0x55b710082b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:07.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.291+0000 7f9697e00700 0 client.15849 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:52:07.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.291+0000 7f96cc910b80 1 -- 172.21.15.86:0/1761650084 >> 172.21.15.86:0/1761650084 conn(0x55b70ffda6a0 msgr2=0x55b71007c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:07.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.291+0000 7f96cc910b80 1 -- 172.21.15.86:0/1761650084 shutdown_connections 2024-03-23T17:52:07.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.291+0000 7f96cc910b80 1 -- 172.21.15.86:0/1761650084 wait complete. 2024-03-23T17:52:07.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.291+0000 7f96cc910b80 20 client.15849 trim_cache size 0 max 16384 2024-03-23T17:52:07.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.307+0000 7f96cc910b80 1 Processor -- start 2024-03-23T17:52:07.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.308+0000 7f96cc910b80 1 -- start start 2024-03-23T17:52:07.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.308+0000 7f96cc910b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b71007a5d0 0x55b71009c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:07.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.308+0000 7f96cc910b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b71007aef0 0x55b71008ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:07.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.308+0000 7f96cc910b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b710085df0 0x55b71007de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:07.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.308+0000 7f96cc910b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b710081120 con 0x55b71007a5d0 2024-03-23T17:52:07.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.308+0000 7f96cc910b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b710076f20 con 0x55b710085df0 2024-03-23T17:52:07.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.308+0000 7f96cc910b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7100918f0 con 0x55b71007aef0 2024-03-23T17:52:07.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.308+0000 7f96b1e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b71007a5d0 0x55b71009c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:07.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.308+0000 7f96b1e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b71007a5d0 0x55b71009c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58982/0 (socket says 172.21.15.86:58982) 2024-03-23T17:52:07.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.308+0000 7f96b1e00700 1 -- 172.21.15.86:0/736570630 learned_addr learned my addr 172.21.15.86:0/736570630 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:07.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.308+0000 7f96abe00700 1 --2- 172.21.15.86:0/736570630 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b71007aef0 0x55b71008ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:07.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.308+0000 7f96b2800700 1 --2- 172.21.15.86:0/736570630 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b710085df0 0x55b71007de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:07.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.309+0000 7f96b1e00700 1 -- 172.21.15.86:0/736570630 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b71007aef0 msgr2=0x55b71008ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:07.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.309+0000 7f96b1e00700 1 --2- 172.21.15.86:0/736570630 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b71007aef0 0x55b71008ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:07.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.309+0000 7f96b1e00700 1 -- 172.21.15.86:0/736570630 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b710085df0 msgr2=0x55b71007de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:07.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.309+0000 7f96b1e00700 1 --2- 172.21.15.86:0/736570630 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b710085df0 0x55b71007de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:07.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.309+0000 7f96b1e00700 1 -- 172.21.15.86:0/736570630 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b70fdfab60 con 0x55b71007a5d0 2024-03-23T17:52:07.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.309+0000 7f96abe00700 1 --2- 172.21.15.86:0/736570630 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b71007aef0 0x55b71008ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:52:07.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.309+0000 7f96b2800700 1 --2- 172.21.15.86:0/736570630 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b710085df0 0x55b71007de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:52:07.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.309+0000 7f96b1e00700 1 --2- 172.21.15.86:0/736570630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b71007a5d0 0x55b71009c850 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f96a40098e0 tx=0x7f96a400b280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d370b299ca11e5b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:07.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.309+0000 7f96a3e00700 1 -- 172.21.15.86:0/736570630 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a401d040 con 0x55b71007a5d0 2024-03-23T17:52:07.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.309+0000 7f96a3e00700 1 -- 172.21.15.86:0/736570630 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f96a401e040 con 0x55b71007a5d0 2024-03-23T17:52:07.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.309+0000 7f96a3e00700 1 -- 172.21.15.86:0/736570630 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a4005680 con 0x55b71007a5d0 2024-03-23T17:52:07.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.310+0000 7f96cc910b80 1 -- 172.21.15.86:0/736570630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b71007a5d0 msgr2=0x55b71009c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:07.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.310+0000 7f96cc910b80 1 --2- 172.21.15.86:0/736570630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b71007a5d0 0x55b71009c850 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f96a40098e0 tx=0x7f96a400b280 comp rx=0 tx=0).stop 2024-03-23T17:52:07.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.310+0000 7f96cc910b80 1 -- 172.21.15.86:0/736570630 shutdown_connections 2024-03-23T17:52:07.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.310+0000 7f96cc910b80 1 --2- 172.21.15.86:0/736570630 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b710085df0 0x55b71007de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:07.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.310+0000 7f96cc910b80 1 --2- 172.21.15.86:0/736570630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b71007a5d0 0x55b71009c850 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:07.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.310+0000 7f96cc910b80 1 --2- 172.21.15.86:0/736570630 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b71007aef0 0x55b71008ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:07.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.310+0000 7f96cc910b80 1 -- 172.21.15.86:0/736570630 >> 172.21.15.86:0/736570630 conn(0x55b70ffda6a0 msgr2=0x55b71006f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:07.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.310+0000 7f96cc910b80 1 -- 172.21.15.86:0/736570630 shutdown_connections 2024-03-23T17:52:07.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.310+0000 7f96cc910b80 1 -- 172.21.15.86:0/736570630 wait complete. 2024-03-23T17:52:07.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.310+0000 7f96cc910b80 1 -- start start 2024-03-23T17:52:07.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.310+0000 7f96cc910b80 1 Processor -- start 2024-03-23T17:52:07.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.311+0000 7f96cc910b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b71007a5d0 0x55b71008af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:07.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.311+0000 7f96cc910b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b71007aef0 0x55b71008b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:07.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.311+0000 7f96cc910b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b710085df0 0x55b71008ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:07.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.311+0000 7f96cc910b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b71008ce40 con 0x55b71007a5d0 2024-03-23T17:52:07.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.311+0000 7f96cc910b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7100e7600 con 0x55b71007aef0 2024-03-23T17:52:07.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.311+0000 7f96cc910b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7100e7770 con 0x55b710085df0 2024-03-23T17:52:07.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.311+0000 7f96cc910b80 10 client.0 _ll_register_callbacks cb 0x55b70ff72470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:52:07.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.311+0000 7f96b1e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b71007a5d0 0x55b71008af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:07.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.311+0000 7f96b1e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b71007a5d0 0x55b71008af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58994/0 (socket says 172.21.15.86:58994) 2024-03-23T17:52:07.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.311+0000 7f96b1e00700 1 -- 172.21.15.86:0/3054842726 learned_addr learned my addr 172.21.15.86:0/3054842726 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:07.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.311+0000 7f96abe00700 1 --2- 172.21.15.86:0/3054842726 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b71007aef0 0x55b71008b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:07.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.311+0000 7f96b2800700 1 --2- 172.21.15.86:0/3054842726 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b710085df0 0x55b71008ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:07.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.311+0000 7f96b1e00700 1 -- 172.21.15.86:0/3054842726 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b710085df0 msgr2=0x55b71008ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:07.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.311+0000 7f96b1e00700 1 --2- 172.21.15.86:0/3054842726 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b710085df0 0x55b71008ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:07.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.311+0000 7f96b1e00700 1 -- 172.21.15.86:0/3054842726 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b71007aef0 msgr2=0x55b71008b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:07.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.311+0000 7f96b1e00700 1 --2- 172.21.15.86:0/3054842726 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b71007aef0 0x55b71008b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:07.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.311+0000 7f96b1e00700 1 -- 172.21.15.86:0/3054842726 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96b410e550 con 0x55b71007a5d0 2024-03-23T17:52:07.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.312+0000 7f96b1e00700 1 --2- 172.21.15.86:0/3054842726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b71007a5d0 0x55b71008af90 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f96a4005200 tx=0x7f96a400bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1a568e37e808644 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:07.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.312+0000 7f9697e00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-23T17:52:07.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.312+0000 7f9697e00700 1 -- 172.21.15.86:0/3054842726 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a401d040 con 0x55b71007a5d0 2024-03-23T17:52:07.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.312+0000 7f9697e00700 1 -- 172.21.15.86:0/3054842726 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f96a4029970 con 0x55b71007a5d0 2024-03-23T17:52:07.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.312+0000 7f96cc910b80 1 -- 172.21.15.86:0/3054842726 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f969c013bd0 con 0x55b71007a5d0 2024-03-23T17:52:07.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.312+0000 7f9697e00700 1 -- 172.21.15.86:0/3054842726 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f96a4005710 con 0x55b71007a5d0 2024-03-23T17:52:07.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.312+0000 7f96cc910b80 10 client.15855 fetch_fsmap learned FSMap version 30 2024-03-23T17:52:07.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.312+0000 7f96cc910b80 1 -- 172.21.15.86:0/3054842726 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55b70ff92ea0 con 0x55b71007a5d0 2024-03-23T17:52:07.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.312+0000 7f9697e00700 1 -- 172.21.15.86:0/3054842726 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f969c013bd0 con 0x55b71007a5d0 2024-03-23T17:52:07.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.312+0000 7f9697e00700 1 -- 172.21.15.86:0/3054842726 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f96a4038020 con 0x55b71007a5d0 2024-03-23T17:52:07.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.313+0000 7f96cc910b80 10 client.15855 fetch_fsmap finished waiting for FSMap version 30 2024-03-23T17:52:07.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.313+0000 7f96cc910b80 10 client.15855 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T17:52:07.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.313+0000 7f96cc910b80 20 client.15855 populate_metadata read hostname 'smithi086' 2024-03-23T17:52:07.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.313+0000 7f96cc910b80 1 --2- 172.21.15.86:0/3054842726 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55b7100e8ce0 0x55b7100eb100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:07.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.313+0000 7f96cc910b80 4 client.15855 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-23T17:52:07.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.313+0000 7f96cc910b80 1 -- 172.21.15.86:0/3054842726 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b70fe1bcf0 con 0x55b7100e8ce0 2024-03-23T17:52:07.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.313+0000 7f96abe00700 1 --2- 172.21.15.86:0/3054842726 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55b7100e8ce0 0x55b7100eb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:07.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.313+0000 7f9697e00700 10 client.15855 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:52:07.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.313+0000 7f96abe00700 1 --2- 172.21.15.86:0/3054842726 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55b7100e8ce0 0x55b7100eb100 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a361e446523827e server_cookie=74163dba700606bb in_seq=0 out_seq=0 2024-03-23T17:52:07.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.314+0000 7f9697e00700 1 -- 172.21.15.86:0/3054842726 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55b70fe1bcf0 con 0x55b7100e8ce0 2024-03-23T17:52:07.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.314+0000 7f9697e00700 10 client.15855 handle_command_reply: tid=0 2024-03-23T17:52:07.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.314+0000 7f96cc910b80 1 client.15855 shutdown 2024-03-23T17:52:07.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.314+0000 7f96cc910b80 1 -- 172.21.15.86:0/3054842726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b71007a5d0 msgr2=0x55b71008af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:07.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.314+0000 7f96cc910b80 1 --2- 172.21.15.86:0/3054842726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b71007a5d0 0x55b71008af90 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f96a4005200 tx=0x7f96a400bcd0 comp rx=0 tx=0).stop 2024-03-23T17:52:07.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.314+0000 7f96cc910b80 1 -- 172.21.15.86:0/3054842726 shutdown_connections 2024-03-23T17:52:07.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.314+0000 7f96cc910b80 1 --2- 172.21.15.86:0/3054842726 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55b7100e8ce0 0x55b7100eb100 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:07.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.314+0000 7f96cc910b80 1 --2- 172.21.15.86:0/3054842726 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b710085df0 0x55b71008ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:07.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.314+0000 7f96cc910b80 1 --2- 172.21.15.86:0/3054842726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b71007a5d0 0x55b71008af90 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:07.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.314+0000 7f9697e00700 0 client.15855 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:52:07.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.314+0000 7f96cc910b80 1 --2- 172.21.15.86:0/3054842726 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b71007aef0 0x55b71008b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:07.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.314+0000 7f96cc910b80 1 -- 172.21.15.86:0/3054842726 >> 172.21.15.86:0/3054842726 conn(0x55b70ffda6a0 msgr2=0x55b71007de90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:07.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.314+0000 7f96cc910b80 1 -- 172.21.15.86:0/3054842726 shutdown_connections 2024-03-23T17:52:07.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.314+0000 7f96cc910b80 1 -- 172.21.15.86:0/3054842726 wait complete. 2024-03-23T17:52:07.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.314+0000 7f96cc910b80 20 client.15855 trim_cache size 0 max 16384 2024-03-23T17:52:07.320 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-23T17:52:07.320 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-03-23T17:52:07.320 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-03-23T17:52:07.320 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-23T17:52:07.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.315+0000 7f96b8e00700 1 -- 172.21.15.86:0/3329262187 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f96980a13b0 msgr2=0x7f96980a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:07.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.315+0000 7f96b8e00700 1 --2- 172.21.15.86:0/3329262187 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f96980a13b0 0x7f96980a3870 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f96b419a9b0 tx=0x7f969c0106b0 comp rx=0 tx=0).stop 2024-03-23T17:52:07.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.315+0000 7f96b8e00700 1 -- 172.21.15.86:0/3329262187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96b4105ce0 msgr2=0x7f96b419dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:07.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.315+0000 7f96b8e00700 1 --2- 172.21.15.86:0/3329262187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96b4105ce0 0x7f96b419dd40 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f96ac00efc0 tx=0x7f96ac00c3a0 comp rx=0 tx=0).stop 2024-03-23T17:52:07.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.315+0000 7f96b8e00700 1 -- 172.21.15.86:0/3329262187 shutdown_connections 2024-03-23T17:52:07.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.315+0000 7f96b8e00700 1 --2- 172.21.15.86:0/3329262187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96b4105ce0 0x7f96b419dd40 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:07.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.315+0000 7f96b8e00700 1 --2- 172.21.15.86:0/3329262187 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96b4067e30 0x7f96b4199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:07.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.315+0000 7f96b8e00700 1 --2- 172.21.15.86:0/3329262187 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f96980a13b0 0x7f96980a3870 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:07.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.315+0000 7f96b8e00700 1 --2- 172.21.15.86:0/3329262187 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f96b40687f0 0x7f96b41999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:07.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.315+0000 7f96b8e00700 1 -- 172.21.15.86:0/3329262187 >> 172.21.15.86:0/3329262187 conn(0x7f96b4076d40 msgr2=0x7f96b410c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:07.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.315+0000 7f96b8e00700 1 -- 172.21.15.86:0/3329262187 shutdown_connections 2024-03-23T17:52:07.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.315+0000 7f96b8e00700 1 -- 172.21.15.86:0/3329262187 wait complete. 2024-03-23T17:52:07.334 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-23T17:52:07.334 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T17:52:07.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.527+0000 7f319b000700 1 -- 172.21.15.86:0/3442741730 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3194100e60 msgr2=0x7f3194101240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:07.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.527+0000 7f319b000700 1 --2- 172.21.15.86:0/3442741730 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3194100e60 0x7f3194101240 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f31840098f0 tx=0x7f3184009c00 comp rx=0 tx=0).stop 2024-03-23T17:52:07.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.527+0000 7f319b000700 1 -- 172.21.15.86:0/3442741730 shutdown_connections 2024-03-23T17:52:07.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.527+0000 7f319b000700 1 --2- 172.21.15.86:0/3442741730 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3194105f80 0x7f319410db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:07.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.527+0000 7f319b000700 1 --2- 172.21.15.86:0/3442741730 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3194100e60 0x7f3194101240 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:07.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.527+0000 7f319b000700 1 --2- 172.21.15.86:0/3442741730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3194101810 0x7f3194105850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:07.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.527+0000 7f319b000700 1 -- 172.21.15.86:0/3442741730 >> 172.21.15.86:0/3442741730 conn(0x7f31940fc760 msgr2=0x7f31940feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:07.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.527+0000 7f319b000700 1 -- 172.21.15.86:0/3442741730 shutdown_connections 2024-03-23T17:52:07.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.527+0000 7f319b000700 1 -- 172.21.15.86:0/3442741730 wait complete. 2024-03-23T17:52:07.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.528+0000 7f319b000700 1 Processor -- start 2024-03-23T17:52:07.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.528+0000 7f319b000700 1 -- start start 2024-03-23T17:52:07.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.528+0000 7f319b000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3194100e60 0x7f31941993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:07.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.528+0000 7f319b000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3194105f80 0x7f3194199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:07.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.528+0000 7f319b000700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f319419dcc0 0x7f319419e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:07.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.528+0000 7f319b000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f319419e740 con 0x7f3194100e60 2024-03-23T17:52:07.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.528+0000 7f319b000700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f319419e8b0 con 0x7f3194105f80 2024-03-23T17:52:07.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.528+0000 7f319b000700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f319419ea70 con 0x7f319419dcc0 2024-03-23T17:52:07.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.529+0000 7f3193e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3194100e60 0x7f31941993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:07.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.529+0000 7f3193e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3194100e60 0x7f31941993e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59002/0 (socket says 172.21.15.86:59002) 2024-03-23T17:52:07.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.529+0000 7f3193e00700 1 -- 172.21.15.86:0/2766001484 learned_addr learned my addr 172.21.15.86:0/2766001484 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:07.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.529+0000 7f318be00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3194105f80 0x7f3194199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:07.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.529+0000 7f3199000700 1 --2- 172.21.15.86:0/2766001484 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f319419dcc0 0x7f319419e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:07.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.529+0000 7f3193e00700 1 -- 172.21.15.86:0/2766001484 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f319419dcc0 msgr2=0x7f319419e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:07.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.529+0000 7f3193e00700 1 --2- 172.21.15.86:0/2766001484 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f319419dcc0 0x7f319419e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:07.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.529+0000 7f3193e00700 1 -- 172.21.15.86:0/2766001484 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3194105f80 msgr2=0x7f3194199920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:07.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.529+0000 7f3193e00700 1 --2- 172.21.15.86:0/2766001484 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3194105f80 0x7f3194199920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:07.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.529+0000 7f3193e00700 1 -- 172.21.15.86:0/2766001484 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f317c0095f0 con 0x7f3194100e60 2024-03-23T17:52:07.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.529+0000 7f3193e00700 1 --2- 172.21.15.86:0/2766001484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3194100e60 0x7f31941993e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f3184005950 tx=0x7f318400fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:07.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.530+0000 7f3191600700 1 -- 172.21.15.86:0/2766001484 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f318401c070 con 0x7f3194100e60 2024-03-23T17:52:07.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.530+0000 7f319b000700 1 -- 172.21.15.86:0/2766001484 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3184009520 con 0x7f3194100e60 2024-03-23T17:52:07.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.530+0000 7f3191600700 1 -- 172.21.15.86:0/2766001484 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f318400b9c0 con 0x7f3194100e60 2024-03-23T17:52:07.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.530+0000 7f3191600700 1 -- 172.21.15.86:0/2766001484 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31840038e0 con 0x7f3194100e60 2024-03-23T17:52:07.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.530+0000 7f319b000700 1 -- 172.21.15.86:0/2766001484 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31941a61b0 con 0x7f3194100e60 2024-03-23T17:52:07.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.531+0000 7f319b000700 1 -- 172.21.15.86:0/2766001484 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3194102330 con 0x7f3194100e60 2024-03-23T17:52:07.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.532+0000 7f3191600700 1 -- 172.21.15.86:0/2766001484 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) v1 ==== 136098+0+0 (secure 0 0 0) 0x7f3184020030 con 0x7f3194100e60 2024-03-23T17:52:07.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.534+0000 7f3191600700 1 --2- 172.21.15.86:0/2766001484 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f31780a13b0 0x7f31780a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:07.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.534+0000 7f3191600700 1 -- 172.21.15.86:0/2766001484 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f31840cdc70 con 0x7f3194100e60 2024-03-23T17:52:07.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.534+0000 7f318be00700 1 --2- 172.21.15.86:0/2766001484 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f31780a13b0 0x7f31780a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:07.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.535+0000 7f318be00700 1 --2- 172.21.15.86:0/2766001484 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f31780a13b0 0x7f31780a3870 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f319419a930 tx=0x7f317c009310 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:07.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.538+0000 7f3191600700 1 -- 172.21.15.86:0/2766001484 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f3184096f50 con 0x7f3194100e60 2024-03-23T17:52:07.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.793+0000 7f319b000700 1 -- 172.21.15.86:0/2766001484 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3194101810 con 0x7f3194100e60 2024-03-23T17:52:07.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.793+0000 7f3191600700 1 -- 172.21.15.86:0/2766001484 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+10920 (secure 0 0 0) 0x7f31840ba070 con 0x7f3194100e60 2024-03-23T17:52:07.794 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:52:07.795 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14445,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1433140219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1433140219},{"type":"v1","addr":"172.21.15.86:6835","nonce":1433140219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6839/1879457730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1879457730},{"type":"v1","addr":"172.21.15.86:6839","nonce":1879457730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-23T17:48:56.202323+0000","modified":"2024-03-23T17:51:49.749912+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24553,"mds_2":24518,"mds_3":24505,"mds_4":24500},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14514":{"gid":14514,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/3026943866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3026943866},{"type":"v1","addr":"172.21.15.86:6841","nonce":3026943866}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24464":{"gid":24464,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6833/1467091186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1467091186},{"type":"v1","addr":"172.21.15.92:6833","nonce":1467091186}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6833/3075554623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3075554623},{"type":"v1","addr":"172.21.15.152:6833","nonce":3075554623}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.92:6835/908900027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":908900027},{"type":"v1","addr":"172.21.15.92:6835","nonce":908900027}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"d","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.86:6837/2933881328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2933881328},{"type":"v1","addr":"172.21.15.86:6837","nonce":2933881328}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6837/3967837948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":3967837948},{"type":"v1","addr":"172.21.15.92:6837","nonce":3967837948}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6835/3137085021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3137085021},{"type":"v1","addr":"172.21.15.152:6835","nonce":3137085021}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24538":{"gid":24538,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6837/1151208965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":1151208965},{"type":"v1","addr":"172.21.15.152:6837","nonce":1151208965}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.152:6839/1100723992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":1100723992},{"type":"v1","addr":"172.21.15.152:6839","nonce":1100723992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6839/1394139887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1394139887},{"type":"v1","addr":"172.21.15.92:6839","nonce":1394139887}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-23T17:52:07.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.796+0000 7f319b000700 1 -- 172.21.15.86:0/2766001484 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f31780a13b0 msgr2=0x7f31780a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:07.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.796+0000 7f319b000700 1 --2- 172.21.15.86:0/2766001484 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f31780a13b0 0x7f31780a3870 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f319419a930 tx=0x7f317c009310 comp rx=0 tx=0).stop 2024-03-23T17:52:07.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.796+0000 7f319b000700 1 -- 172.21.15.86:0/2766001484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3194100e60 msgr2=0x7f31941993e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:07.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.796+0000 7f319b000700 1 --2- 172.21.15.86:0/2766001484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3194100e60 0x7f31941993e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f3184005950 tx=0x7f318400fa00 comp rx=0 tx=0).stop 2024-03-23T17:52:07.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.796+0000 7f319b000700 1 -- 172.21.15.86:0/2766001484 shutdown_connections 2024-03-23T17:52:07.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.796+0000 7f319b000700 1 --2- 172.21.15.86:0/2766001484 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f31780a13b0 0x7f31780a3870 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:07.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.796+0000 7f319b000700 1 --2- 172.21.15.86:0/2766001484 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f319419dcc0 0x7f319419e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:07.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.796+0000 7f319b000700 1 --2- 172.21.15.86:0/2766001484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3194100e60 0x7f31941993e0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:07.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.796+0000 7f319b000700 1 --2- 172.21.15.86:0/2766001484 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3194105f80 0x7f3194199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:07.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.796+0000 7f319b000700 1 -- 172.21.15.86:0/2766001484 >> 172.21.15.86:0/2766001484 conn(0x7f31940fc760 msgr2=0x7f31940fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:07.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.796+0000 7f319b000700 1 -- 172.21.15.86:0/2766001484 shutdown_connections 2024-03-23T17:52:07.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:07.796+0000 7f319b000700 1 -- 172.21.15.86:0/2766001484 wait complete. 2024-03-23T17:52:07.799 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 30 2024-03-23T17:52:07.817 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-23T17:52:07.817 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-23T17:52:07.880 INFO:tasks.workunit.client.0.smithi086.stdout: 1 28803 5.44 MB/sec warmup 32 sec latency 4946.076 ms 2024-03-23T17:52:08.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.011+0000 7f5d83400700 1 -- 172.21.15.86:0/2401744263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d7c100f60 msgr2=0x7f5d7c101340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:08.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.011+0000 7f5d83400700 1 --2- 172.21.15.86:0/2401744263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d7c100f60 0x7f5d7c101340 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f5d70009990 tx=0x7f5d70009ca0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.011+0000 7f5d83400700 1 -- 172.21.15.86:0/2401744263 shutdown_connections 2024-03-23T17:52:08.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.011+0000 7f5d83400700 1 --2- 172.21.15.86:0/2401744263 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d7c106080 0x7f5d7c10dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.011+0000 7f5d83400700 1 --2- 172.21.15.86:0/2401744263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d7c100f60 0x7f5d7c101340 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.011+0000 7f5d83400700 1 --2- 172.21.15.86:0/2401744263 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5d7c101910 0x7f5d7c105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.011+0000 7f5d83400700 1 -- 172.21.15.86:0/2401744263 >> 172.21.15.86:0/2401744263 conn(0x7f5d7c0fc7c0 msgr2=0x7f5d7c0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:08.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.011+0000 7f5d83400700 1 -- 172.21.15.86:0/2401744263 shutdown_connections 2024-03-23T17:52:08.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.011+0000 7f5d83400700 1 -- 172.21.15.86:0/2401744263 wait complete. 2024-03-23T17:52:08.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.012+0000 7f5d83400700 1 Processor -- start 2024-03-23T17:52:08.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.012+0000 7f5d83400700 1 -- start start 2024-03-23T17:52:08.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.013+0000 7f5d83400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5d7c100f60 0x7f5d7c199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:08.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.013+0000 7f5d83400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d7c106080 0x7f5d7c199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:08.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.013+0000 7f5d83400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d7c19dd00 0x7f5d7c19e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:08.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.013+0000 7f5d83400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7c19e780 con 0x7f5d7c19dd00 2024-03-23T17:52:08.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.013+0000 7f5d83400700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7c19e8f0 con 0x7f5d7c106080 2024-03-23T17:52:08.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.013+0000 7f5d83400700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7c19eab0 con 0x7f5d7c100f60 2024-03-23T17:52:08.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.013+0000 7f5d81400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d7c19dd00 0x7f5d7c19e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:08.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.013+0000 7f5d7be00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d7c106080 0x7f5d7c199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:08.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.013+0000 7f5d7be00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d7c106080 0x7f5d7c199960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:41202/0 (socket says 172.21.15.86:41202) 2024-03-23T17:52:08.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.013+0000 7f5d7be00700 1 -- 172.21.15.86:0/2401175342 learned_addr learned my addr 172.21.15.86:0/2401175342 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:08.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.013+0000 7f5d80a00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5d7c100f60 0x7f5d7c199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:08.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.013+0000 7f5d7be00700 1 -- 172.21.15.86:0/2401175342 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5d7c100f60 msgr2=0x7f5d7c199420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:08.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.013+0000 7f5d7be00700 1 --2- 172.21.15.86:0/2401175342 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5d7c100f60 0x7f5d7c199420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.013+0000 7f5d7be00700 1 -- 172.21.15.86:0/2401175342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d7c19dd00 msgr2=0x7f5d7c19e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:08.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.013+0000 7f5d7be00700 1 --2- 172.21.15.86:0/2401175342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d7c19dd00 0x7f5d7c19e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.013+0000 7f5d7be00700 1 -- 172.21.15.86:0/2401175342 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d700095f0 con 0x7f5d7c106080 2024-03-23T17:52:08.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.014+0000 7f5d7be00700 1 --2- 172.21.15.86:0/2401175342 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d7c106080 0x7f5d7c199960 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5d6800e970 tx=0x7f5d6800ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:08.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.014+0000 7f5d79600700 1 -- 172.21.15.86:0/2401175342 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d68009790 con 0x7f5d7c106080 2024-03-23T17:52:08.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.014+0000 7f5d79600700 1 -- 172.21.15.86:0/2401175342 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d6800c7e0 con 0x7f5d7c106080 2024-03-23T17:52:08.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.014+0000 7f5d83400700 1 -- 172.21.15.86:0/2401175342 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d7c1a5fb0 con 0x7f5d7c106080 2024-03-23T17:52:08.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.014+0000 7f5d79600700 1 -- 172.21.15.86:0/2401175342 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d680054e0 con 0x7f5d7c106080 2024-03-23T17:52:08.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.014+0000 7f5d83400700 1 -- 172.21.15.86:0/2401175342 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d7c1a6490 con 0x7f5d7c106080 2024-03-23T17:52:08.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.018+0000 7f5d79600700 1 -- 172.21.15.86:0/2401175342 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 25) v1 ==== 136098+0+0 (secure 0 0 0) 0x7f5d6800c950 con 0x7f5d7c106080 2024-03-23T17:52:08.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.019+0000 7f5d96e68b80 1 Processor -- start 2024-03-23T17:52:08.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.019+0000 7f5d96e68b80 1 -- start start 2024-03-23T17:52:08.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.020+0000 7f5d79600700 1 --2- 172.21.15.86:0/2401175342 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f5d640a13b0 0x7f5d640a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:08.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.020+0000 7f5d96e68b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56494ab7f5d0 0x56494ab7f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:08.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.020+0000 7f5d96e68b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56494ab7fef0 0x56494ab8a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:08.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.020+0000 7f5d96e68b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56494ab8adf0 0x56494ab8d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:08.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.020+0000 7f5d96e68b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56494aad8990 con 0x56494ab7fef0 2024-03-23T17:52:08.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.020+0000 7f5d96e68b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x56494aad6350 con 0x56494ab8adf0 2024-03-23T17:52:08.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.020+0000 7f5d96e68b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x56494aad1c40 con 0x56494ab7f5d0 2024-03-23T17:52:08.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.020+0000 7f5d79600700 1 -- 172.21.15.86:0/2401175342 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f5d68014070 con 0x7f5d7c106080 2024-03-23T17:52:08.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.020+0000 7f5d7be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56494ab7fef0 0x56494ab8a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:08.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.020+0000 7f5d81400700 1 --2- 172.21.15.86:0/2401175342 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f5d640a13b0 0x7f5d640a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:08.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.020+0000 7f5d7be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56494ab7fef0 0x56494ab8a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59012/0 (socket says 172.21.15.86:59012) 2024-03-23T17:52:08.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.020+0000 7f5d7be00700 1 -- 172.21.15.86:0/2576502879 learned_addr learned my addr 172.21.15.86:0/2576502879 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:08.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.020+0000 7f5d80a00700 1 --2- 172.21.15.86:0/2576502879 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56494ab7f5d0 0x56494ab7f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:08.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.020+0000 7f5d81400700 1 --2- 172.21.15.86:0/2576502879 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56494ab8adf0 0x56494ab8d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:08.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.020+0000 7f5d7be00700 1 -- 172.21.15.86:0/2576502879 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56494ab7f5d0 msgr2=0x56494ab7f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:08.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.020+0000 7f5d7be00700 1 --2- 172.21.15.86:0/2576502879 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56494ab7f5d0 0x56494ab7f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.020+0000 7f5d7be00700 1 -- 172.21.15.86:0/2576502879 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56494ab8adf0 msgr2=0x56494ab8d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:08.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.020+0000 7f5d7be00700 1 --2- 172.21.15.86:0/2576502879 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56494ab8adf0 0x56494ab8d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.020+0000 7f5d7be00700 1 -- 172.21.15.86:0/2576502879 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56494a9533b0 con 0x56494ab7fef0 2024-03-23T17:52:08.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.020+0000 7f5d81400700 1 --2- 172.21.15.86:0/2576502879 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56494ab8adf0 0x56494ab8d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T17:52:08.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.021+0000 7f5d7be00700 1 --2- 172.21.15.86:0/2576502879 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56494ab7fef0 0x56494ab8a8b0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f5d68099d90 tx=0x7f5d68099cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63bab1c0dcee91af server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:08.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.021+0000 7f5d81400700 1 --2- 172.21.15.86:0/2401175342 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f5d640a13b0 0x7f5d640a3870 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5d74009ec0 tx=0x7f5d74009920 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:08.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.021+0000 7f5d6dc00700 1 -- 172.21.15.86:0/2576502879 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d6809a790 con 0x56494ab7fef0 2024-03-23T17:52:08.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.021+0000 7f5d6dc00700 1 -- 172.21.15.86:0/2576502879 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d6809d020 con 0x56494ab7fef0 2024-03-23T17:52:08.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.021+0000 7f5d6dc00700 1 -- 172.21.15.86:0/2576502879 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d68005660 con 0x56494ab7fef0 2024-03-23T17:52:08.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.022+0000 7f5d96e68b80 1 -- 172.21.15.86:0/2576502879 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56494ab7fef0 msgr2=0x56494ab8a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:08.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.022+0000 7f5d96e68b80 1 --2- 172.21.15.86:0/2576502879 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56494ab7fef0 0x56494ab8a8b0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f5d68099d90 tx=0x7f5d68099cd0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.022+0000 7f5d96e68b80 1 -- 172.21.15.86:0/2576502879 shutdown_connections 2024-03-23T17:52:08.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.022+0000 7f5d96e68b80 1 --2- 172.21.15.86:0/2576502879 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56494ab8adf0 0x56494ab8d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.022+0000 7f5d96e68b80 1 --2- 172.21.15.86:0/2576502879 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56494ab7f5d0 0x56494ab7f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.022+0000 7f5d96e68b80 1 --2- 172.21.15.86:0/2576502879 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56494ab7fef0 0x56494ab8a8b0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.022+0000 7f5d96e68b80 1 -- 172.21.15.86:0/2576502879 >> 172.21.15.86:0/2576502879 conn(0x56494aadf6a0 msgr2=0x56494ab7ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:08.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.022+0000 7f5d96e68b80 1 -- 172.21.15.86:0/2576502879 shutdown_connections 2024-03-23T17:52:08.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.022+0000 7f5d96e68b80 1 -- 172.21.15.86:0/2576502879 wait complete. 2024-03-23T17:52:08.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.022+0000 7f5d96e68b80 1 -- start start 2024-03-23T17:52:08.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.022+0000 7f5d96e68b80 1 Processor -- start 2024-03-23T17:52:08.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.023+0000 7f5d96e68b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56494ab7f5d0 0x56494ab8a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:08.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.023+0000 7f5d96e68b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56494ab7fef0 0x56494ab87ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:08.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.023+0000 7f5d96e68b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56494ab8adf0 0x56494ab88060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:08.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.023+0000 7f5d96e68b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56494aa6dd80 con 0x56494ab7f5d0 2024-03-23T17:52:08.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.023+0000 7f5d96e68b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x56494aa6da20 con 0x56494ab8adf0 2024-03-23T17:52:08.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.023+0000 7f5d96e68b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x56494a9556a0 con 0x56494ab7fef0 2024-03-23T17:52:08.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.023+0000 7f5d96e68b80 10 client.0 _ll_register_callbacks cb 0x56494aa77470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:52:08.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.023+0000 7f5d80a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56494ab7f5d0 0x56494ab8a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:08.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.023+0000 7f5d7be00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56494ab7fef0 0x56494ab87ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:08.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.023+0000 7f5d80a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56494ab7f5d0 0x56494ab8a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59014/0 (socket says 172.21.15.86:59014) 2024-03-23T17:52:08.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.023+0000 7f5d80a00700 1 -- 172.21.15.86:0/1084596732 learned_addr learned my addr 172.21.15.86:0/1084596732 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:08.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.023+0000 7f5d7be00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56494ab7fef0 0x56494ab87ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:50958/0 (socket says 172.21.15.86:50958) 2024-03-23T17:52:08.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.023+0000 7f5d81400700 1 --2- 172.21.15.86:0/1084596732 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56494ab8adf0 0x56494ab88060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:08.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.024+0000 7f5d7be00700 1 -- 172.21.15.86:0/1084596732 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56494ab8adf0 msgr2=0x56494ab88060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:08.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.024+0000 7f5d7be00700 1 --2- 172.21.15.86:0/1084596732 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56494ab8adf0 0x56494ab88060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.024+0000 7f5d7be00700 1 -- 172.21.15.86:0/1084596732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56494ab7f5d0 msgr2=0x56494ab8a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:08.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.024+0000 7f5d7be00700 1 --2- 172.21.15.86:0/1084596732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56494ab7f5d0 0x56494ab8a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.024+0000 7f5d7be00700 1 -- 172.21.15.86:0/1084596732 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d7c1a6490 con 0x56494ab7fef0 2024-03-23T17:52:08.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.024+0000 7f5d81400700 1 --2- 172.21.15.86:0/1084596732 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56494ab8adf0 0x56494ab88060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:52:08.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.024+0000 7f5d7be00700 1 --2- 172.21.15.86:0/1084596732 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56494ab7fef0 0x56494ab87ac0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5d6809a250 tx=0x7f5d680ca710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9a31c6b20b4052f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:08.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.024+0000 7f5d62000700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-03-23T17:52:08.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.024+0000 7f5d62000700 1 -- 172.21.15.86:0/1084596732 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d6809a790 con 0x56494ab7fef0 2024-03-23T17:52:08.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.024+0000 7f5d62000700 1 -- 172.21.15.86:0/1084596732 <== mon.1 v2:172.21.15.152:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f5d6809a090 con 0x56494ab7fef0 2024-03-23T17:52:08.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.024+0000 7f5d96e68b80 1 -- 172.21.15.86:0/1084596732 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56494aa92380 con 0x56494ab7fef0 2024-03-23T17:52:08.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.024+0000 7f5d62000700 1 -- 172.21.15.86:0/1084596732 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d680a2510 con 0x56494ab7fef0 2024-03-23T17:52:08.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.024+0000 7f5d96e68b80 10 client.25927 fetch_fsmap learned FSMap version 30 2024-03-23T17:52:08.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.024+0000 7f5d96e68b80 1 -- 172.21.15.86:0/1084596732 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56494a8ffb60 con 0x56494ab7fef0 2024-03-23T17:52:08.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.024+0000 7f5d62000700 1 -- 172.21.15.86:0/1084596732 <== mon.1 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56494aa92380 con 0x56494ab7fef0 2024-03-23T17:52:08.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.025+0000 7f5d62000700 1 -- 172.21.15.86:0/1084596732 <== mon.1 v2:172.21.15.152:3300/0 5 ==== fsmap(e 30) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f5d680ca200 con 0x56494ab7fef0 2024-03-23T17:52:08.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.025+0000 7f5d96e68b80 10 client.25927 fetch_fsmap finished waiting for FSMap version 30 2024-03-23T17:52:08.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.025+0000 7f5d96e68b80 10 client.25927 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-03-23T17:52:08.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.025+0000 7f5d96e68b80 20 client.25927 populate_metadata read hostname 'smithi086' 2024-03-23T17:52:08.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.025+0000 7f5d96e68b80 1 --2- 172.21.15.86:0/1084596732 >> [v2:172.21.15.92:6834/908900027,v1:172.21.15.92:6835/908900027] conn(0x56494ab9a4f0 0x56494ab9c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:08.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.025+0000 7f5d96e68b80 4 client.25927 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:52:08.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.025+0000 7f5d96e68b80 1 -- 172.21.15.86:0/1084596732 --> [v2:172.21.15.92:6834/908900027,v1:172.21.15.92:6835/908900027] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56494aaca700 con 0x56494ab9a4f0 2024-03-23T17:52:08.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.025+0000 7f5d80a00700 1 --2- 172.21.15.86:0/1084596732 >> [v2:172.21.15.92:6834/908900027,v1:172.21.15.92:6835/908900027] conn(0x56494ab9a4f0 0x56494ab9c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:08.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.026+0000 7f5d80a00700 1 --2- 172.21.15.86:0/1084596732 >> [v2:172.21.15.92:6834/908900027,v1:172.21.15.92:6835/908900027] conn(0x56494ab9a4f0 0x56494ab9c8e0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=774a0c2c5fa40cf8 server_cookie=dfe377fd1e2addec in_seq=0 out_seq=0 2024-03-23T17:52:08.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.026+0000 7f5d62000700 10 client.25927 ms_handle_connect on v2:172.21.15.92:6834/908900027 2024-03-23T17:52:08.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.026+0000 7f5d62000700 1 -- 172.21.15.86:0/1084596732 <== mds.4 v2:172.21.15.92:6834/908900027 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x56494aaca700 con 0x56494ab9a4f0 2024-03-23T17:52:08.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.026+0000 7f5d62000700 10 client.25927 handle_command_reply: tid=0 2024-03-23T17:52:08.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.026+0000 7f5d96e68b80 1 client.25927 shutdown 2024-03-23T17:52:08.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.027+0000 7f5d96e68b80 1 -- 172.21.15.86:0/1084596732 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56494ab7fef0 msgr2=0x56494ab87ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:08.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.027+0000 7f5d96e68b80 1 --2- 172.21.15.86:0/1084596732 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56494ab7fef0 0x56494ab87ac0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5d6809a250 tx=0x7f5d680ca710 comp rx=0 tx=0).stop 2024-03-23T17:52:08.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.027+0000 7f5d96e68b80 1 -- 172.21.15.86:0/1084596732 shutdown_connections 2024-03-23T17:52:08.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.027+0000 7f5d96e68b80 1 --2- 172.21.15.86:0/1084596732 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56494ab8adf0 0x56494ab88060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.027+0000 7f5d96e68b80 1 --2- 172.21.15.86:0/1084596732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56494ab7f5d0 0x56494ab8a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.027+0000 7f5d96e68b80 1 --2- 172.21.15.86:0/1084596732 >> [v2:172.21.15.92:6834/908900027,v1:172.21.15.92:6835/908900027] conn(0x56494ab9a4f0 0x56494ab9c8e0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.027+0000 7f5d96e68b80 1 --2- 172.21.15.86:0/1084596732 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56494ab7fef0 0x56494ab87ac0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.027+0000 7f5d96e68b80 1 -- 172.21.15.86:0/1084596732 >> 172.21.15.86:0/1084596732 conn(0x56494aadf6a0 msgr2=0x56494ab816c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:08.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.027+0000 7f5d62000700 0 client.25927 ms_handle_reset on v2:172.21.15.92:6834/908900027 2024-03-23T17:52:08.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.027+0000 7f5d96e68b80 1 -- 172.21.15.86:0/1084596732 shutdown_connections 2024-03-23T17:52:08.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.027+0000 7f5d96e68b80 1 -- 172.21.15.86:0/1084596732 wait complete. 2024-03-23T17:52:08.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.027+0000 7f5d96e68b80 20 client.25927 trim_cache size 0 max 16384 2024-03-23T17:52:08.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.040+0000 7f5d96e68b80 1 Processor -- start 2024-03-23T17:52:08.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.040+0000 7f5d96e68b80 1 -- start start 2024-03-23T17:52:08.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.040+0000 7f5d96e68b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56494ab7f5d0 0x56494ab82570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:08.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.040+0000 7f5d96e68b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56494ab7fef0 0x56494aba1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:08.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.040+0000 7f5d96e68b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56494ab8adf0 0x56494ab91a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:08.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.040+0000 7f5d96e68b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56494ab9cca0 con 0x56494ab7fef0 2024-03-23T17:52:08.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.040+0000 7f5d96e68b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x56494ab88660 con 0x56494ab7f5d0 2024-03-23T17:52:08.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.040+0000 7f5d96e68b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x56494ab80670 con 0x56494ab8adf0 2024-03-23T17:52:08.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.041+0000 7f5d7be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56494ab7fef0 0x56494aba1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:08.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.041+0000 7f5d7be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56494ab7fef0 0x56494aba1200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59016/0 (socket says 172.21.15.86:59016) 2024-03-23T17:52:08.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.041+0000 7f5d81400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56494ab8adf0 0x56494ab91a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:08.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.041+0000 7f5d7be00700 1 -- 172.21.15.86:0/3437924130 learned_addr learned my addr 172.21.15.86:0/3437924130 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:08.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.041+0000 7f5d80a00700 1 --2- 172.21.15.86:0/3437924130 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56494ab7f5d0 0x56494ab82570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:08.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.041+0000 7f5d7be00700 1 -- 172.21.15.86:0/3437924130 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56494ab8adf0 msgr2=0x56494ab91a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:08.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.041+0000 7f5d7be00700 1 --2- 172.21.15.86:0/3437924130 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56494ab8adf0 0x56494ab91a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.041+0000 7f5d7be00700 1 -- 172.21.15.86:0/3437924130 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56494ab7f5d0 msgr2=0x56494ab82570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:08.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.041+0000 7f5d7be00700 1 --2- 172.21.15.86:0/3437924130 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56494ab7f5d0 0x56494ab82570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.041+0000 7f5d7be00700 1 -- 172.21.15.86:0/3437924130 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56494aa97ea0 con 0x56494ab7fef0 2024-03-23T17:52:08.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.041+0000 7f5d7be00700 1 --2- 172.21.15.86:0/3437924130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56494ab7fef0 0x56494aba1200 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f5d68005950 tx=0x7f5d68005870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72e7c94252fef2d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:08.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.041+0000 7f5d6e600700 1 -- 172.21.15.86:0/3437924130 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d680a2510 con 0x56494ab7fef0 2024-03-23T17:52:08.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.042+0000 7f5d6e600700 1 -- 172.21.15.86:0/3437924130 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d680b81c0 con 0x56494ab7fef0 2024-03-23T17:52:08.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.042+0000 7f5d6e600700 1 -- 172.21.15.86:0/3437924130 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d68095e80 con 0x56494ab7fef0 2024-03-23T17:52:08.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.042+0000 7f5d96e68b80 1 -- 172.21.15.86:0/3437924130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56494ab7fef0 msgr2=0x56494aba1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:08.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.042+0000 7f5d96e68b80 1 --2- 172.21.15.86:0/3437924130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56494ab7fef0 0x56494aba1200 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f5d68005950 tx=0x7f5d68005870 comp rx=0 tx=0).stop 2024-03-23T17:52:08.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.042+0000 7f5d96e68b80 1 -- 172.21.15.86:0/3437924130 shutdown_connections 2024-03-23T17:52:08.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.042+0000 7f5d96e68b80 1 --2- 172.21.15.86:0/3437924130 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56494ab8adf0 0x56494ab91a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.042+0000 7f5d96e68b80 1 --2- 172.21.15.86:0/3437924130 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56494ab7f5d0 0x56494ab82570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.042+0000 7f5d96e68b80 1 --2- 172.21.15.86:0/3437924130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56494ab7fef0 0x56494aba1200 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.042+0000 7f5d96e68b80 1 -- 172.21.15.86:0/3437924130 >> 172.21.15.86:0/3437924130 conn(0x56494aadf6a0 msgr2=0x56494ab86390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:08.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.042+0000 7f5d96e68b80 1 -- 172.21.15.86:0/3437924130 shutdown_connections 2024-03-23T17:52:08.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.042+0000 7f5d96e68b80 1 -- 172.21.15.86:0/3437924130 wait complete. 2024-03-23T17:52:08.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.043+0000 7f5d96e68b80 1 -- start start 2024-03-23T17:52:08.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.043+0000 7f5d96e68b80 1 Processor -- start 2024-03-23T17:52:08.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.043+0000 7f5d96e68b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56494ab7f5d0 0x56494ab8fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:08.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.043+0000 7f5d96e68b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56494ab7fef0 0x56494ab901a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:08.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.043+0000 7f5d96e68b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56494ab8adf0 0x56494abe4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:08.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.043+0000 7f5d96e68b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56494abe54b0 con 0x56494ab7f5d0 2024-03-23T17:52:08.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.043+0000 7f5d96e68b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x56494abe5620 con 0x56494ab7fef0 2024-03-23T17:52:08.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.043+0000 7f5d96e68b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x56494abe5790 con 0x56494ab8adf0 2024-03-23T17:52:08.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.043+0000 7f5d96e68b80 10 client.0 _ll_register_callbacks cb 0x56494aa77470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:52:08.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.043+0000 7f5d7be00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56494ab7fef0 0x56494ab901a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:08.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.043+0000 7f5d80a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56494ab7f5d0 0x56494ab8fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:08.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.043+0000 7f5d7be00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56494ab7fef0 0x56494ab901a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:41234/0 (socket says 172.21.15.86:41234) 2024-03-23T17:52:08.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.044+0000 7f5d7be00700 1 -- 172.21.15.86:0/1329444327 learned_addr learned my addr 172.21.15.86:0/1329444327 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:08.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.044+0000 7f5d81400700 1 --2- 172.21.15.86:0/1329444327 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56494ab8adf0 0x56494abe4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:08.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.044+0000 7f5d7be00700 1 -- 172.21.15.86:0/1329444327 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56494ab8adf0 msgr2=0x56494abe4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:08.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.044+0000 7f5d7be00700 1 --2- 172.21.15.86:0/1329444327 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56494ab8adf0 0x56494abe4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.044+0000 7f5d7be00700 1 -- 172.21.15.86:0/1329444327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56494ab7f5d0 msgr2=0x56494ab8fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:08.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.044+0000 7f5d7be00700 1 --2- 172.21.15.86:0/1329444327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56494ab7f5d0 0x56494ab8fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.044+0000 7f5d7be00700 1 -- 172.21.15.86:0/1329444327 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56494a8ffb60 con 0x56494ab7fef0 2024-03-23T17:52:08.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.044+0000 7f5d80a00700 1 --2- 172.21.15.86:0/1329444327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56494ab7f5d0 0x56494ab8fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:52:08.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.044+0000 7f5d7be00700 1 --2- 172.21.15.86:0/1329444327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56494ab7fef0 0x56494ab901a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5d680a1fe0 tx=0x7f5d68099b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8340db2ea0ea791e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:08.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.044+0000 7f5d62000700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T17:52:08.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.044+0000 7f5d62000700 1 -- 172.21.15.86:0/1329444327 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d680a2510 con 0x56494ab7fef0 2024-03-23T17:52:08.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.044+0000 7f5d62000700 1 -- 172.21.15.86:0/1329444327 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f5d680961d0 con 0x56494ab7fef0 2024-03-23T17:52:08.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.044+0000 7f5d96e68b80 1 -- 172.21.15.86:0/1329444327 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5d6809d020 con 0x56494ab7fef0 2024-03-23T17:52:08.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.044+0000 7f5d62000700 1 -- 172.21.15.86:0/1329444327 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d6809a790 con 0x56494ab7fef0 2024-03-23T17:52:08.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.045+0000 7f5d96e68b80 10 client.25937 fetch_fsmap learned FSMap version 30 2024-03-23T17:52:08.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.045+0000 7f5d96e68b80 1 -- 172.21.15.86:0/1329444327 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56494a957cd0 con 0x56494ab7fef0 2024-03-23T17:52:08.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.045+0000 7f5d62000700 1 -- 172.21.15.86:0/1329444327 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d6809d020 con 0x56494ab7fef0 2024-03-23T17:52:08.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.045+0000 7f5d62000700 1 -- 172.21.15.86:0/1329444327 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 30) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f5d68004a30 con 0x56494ab7fef0 2024-03-23T17:52:08.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.045+0000 7f5d96e68b80 10 client.25937 fetch_fsmap finished waiting for FSMap version 30 2024-03-23T17:52:08.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.045+0000 7f5d96e68b80 10 client.25937 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-03-23T17:52:08.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.045+0000 7f5d96e68b80 20 client.25937 populate_metadata read hostname 'smithi086' 2024-03-23T17:52:08.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.045+0000 7f5d96e68b80 1 --2- 172.21.15.86:0/1329444327 >> [v2:172.21.15.92:6834/908900027,v1:172.21.15.92:6835/908900027] conn(0x56494abecb20 0x56494abeef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:08.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.045+0000 7f5d96e68b80 4 client.25937 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-23T17:52:08.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.045+0000 7f5d96e68b80 1 -- 172.21.15.86:0/1329444327 --> [v2:172.21.15.92:6834/908900027,v1:172.21.15.92:6835/908900027] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56494a920cf0 con 0x56494abecb20 2024-03-23T17:52:08.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.045+0000 7f5d80a00700 1 --2- 172.21.15.86:0/1329444327 >> [v2:172.21.15.92:6834/908900027,v1:172.21.15.92:6835/908900027] conn(0x56494abecb20 0x56494abeef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:08.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.046+0000 7f5d62000700 10 client.25937 ms_handle_connect on v2:172.21.15.92:6834/908900027 2024-03-23T17:52:08.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.046+0000 7f5d80a00700 1 --2- 172.21.15.86:0/1329444327 >> [v2:172.21.15.92:6834/908900027,v1:172.21.15.92:6835/908900027] conn(0x56494abecb20 0x56494abeef10 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=83fc8b7de263ddb4 server_cookie=bf67c7a6ef2cfccc in_seq=0 out_seq=0 2024-03-23T17:52:08.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.046+0000 7f5d62000700 1 -- 172.21.15.86:0/1329444327 <== mds.4 v2:172.21.15.92:6834/908900027 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56494a920cf0 con 0x56494abecb20 2024-03-23T17:52:08.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.046+0000 7f5d62000700 10 client.25937 handle_command_reply: tid=0 2024-03-23T17:52:08.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.046+0000 7f5d96e68b80 1 client.25937 shutdown 2024-03-23T17:52:08.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.046+0000 7f5d96e68b80 1 -- 172.21.15.86:0/1329444327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56494ab7fef0 msgr2=0x56494ab901a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:08.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.046+0000 7f5d96e68b80 1 --2- 172.21.15.86:0/1329444327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56494ab7fef0 0x56494ab901a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5d680a1fe0 tx=0x7f5d68099b20 comp rx=0 tx=0).stop 2024-03-23T17:52:08.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.046+0000 7f5d96e68b80 1 -- 172.21.15.86:0/1329444327 shutdown_connections 2024-03-23T17:52:08.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.046+0000 7f5d96e68b80 1 --2- 172.21.15.86:0/1329444327 >> [v2:172.21.15.92:6834/908900027,v1:172.21.15.92:6835/908900027] conn(0x56494abecb20 0x56494abeef10 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.046+0000 7f5d96e68b80 1 --2- 172.21.15.86:0/1329444327 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56494ab8adf0 0x56494abe4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.046+0000 7f5d62000700 0 client.25937 ms_handle_reset on v2:172.21.15.92:6834/908900027 2024-03-23T17:52:08.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.046+0000 7f5d96e68b80 1 --2- 172.21.15.86:0/1329444327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56494ab7f5d0 0x56494ab8fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.046+0000 7f5d96e68b80 1 --2- 172.21.15.86:0/1329444327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56494ab7fef0 0x56494ab901a0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.046+0000 7f5d96e68b80 1 -- 172.21.15.86:0/1329444327 >> 172.21.15.86:0/1329444327 conn(0x56494aadf6a0 msgr2=0x56494ab87320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:08.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.046+0000 7f5d96e68b80 1 -- 172.21.15.86:0/1329444327 shutdown_connections 2024-03-23T17:52:08.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.046+0000 7f5d96e68b80 1 -- 172.21.15.86:0/1329444327 wait complete. 2024-03-23T17:52:08.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.046+0000 7f5d96e68b80 20 client.25937 trim_cache size 0 max 16384 2024-03-23T17:52:08.054 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-03-23T17:52:08.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.047+0000 7f5d83400700 1 -- 172.21.15.86:0/2401175342 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f5d640a13b0 msgr2=0x7f5d640a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:08.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.047+0000 7f5d83400700 1 --2- 172.21.15.86:0/2401175342 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f5d640a13b0 0x7f5d640a3870 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5d74009ec0 tx=0x7f5d74009920 comp rx=0 tx=0).stop 2024-03-23T17:52:08.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.047+0000 7f5d83400700 1 -- 172.21.15.86:0/2401175342 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d7c106080 msgr2=0x7f5d7c199960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:08.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.047+0000 7f5d83400700 1 --2- 172.21.15.86:0/2401175342 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d7c106080 0x7f5d7c199960 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5d6800e970 tx=0x7f5d6800ec80 comp rx=0 tx=0).stop 2024-03-23T17:52:08.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.047+0000 7f5d83400700 1 -- 172.21.15.86:0/2401175342 shutdown_connections 2024-03-23T17:52:08.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.047+0000 7f5d83400700 1 --2- 172.21.15.86:0/2401175342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d7c19dd00 0x7f5d7c19e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.047+0000 7f5d83400700 1 --2- 172.21.15.86:0/2401175342 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f5d640a13b0 0x7f5d640a3870 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.047+0000 7f5d83400700 1 --2- 172.21.15.86:0/2401175342 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5d7c100f60 0x7f5d7c199420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.047+0000 7f5d83400700 1 --2- 172.21.15.86:0/2401175342 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d7c106080 0x7f5d7c199960 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.047+0000 7f5d83400700 1 -- 172.21.15.86:0/2401175342 >> 172.21.15.86:0/2401175342 conn(0x7f5d7c0fc7c0 msgr2=0x7f5d7c0fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:08.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.047+0000 7f5d83400700 1 -- 172.21.15.86:0/2401175342 shutdown_connections 2024-03-23T17:52:08.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.047+0000 7f5d83400700 1 -- 172.21.15.86:0/2401175342 wait complete. 2024-03-23T17:52:08.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:07 smithi092 ceph-mon[79089]: pgmap v47: 97 pgs: 97 active+clean; 26 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 3.6 MiB/s wr, 115 op/s 2024-03-23T17:52:08.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:07 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/2766001484' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T17:52:08.064 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:07 smithi086 ceph-mon[38221]: pgmap v47: 97 pgs: 97 active+clean; 26 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 3.6 MiB/s wr, 115 op/s 2024-03-23T17:52:08.064 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:07 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/2766001484' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T17:52:08.066 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-23T17:52:08.067 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-23T17:52:08.085 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:07 smithi152 ceph-mon[47949]: pgmap v47: 97 pgs: 97 active+clean; 26 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 3.6 MiB/s wr, 115 op/s 2024-03-23T17:52:08.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:07 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/2766001484' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T17:52:08.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.263+0000 7f73c2800700 1 -- 172.21.15.86:0/2395641129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73bc100f60 msgr2=0x7f73bc101340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:08.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.263+0000 7f73c2800700 1 --2- 172.21.15.86:0/2395641129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73bc100f60 0x7f73bc101340 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f73ac009990 tx=0x7f73ac009ca0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.263+0000 7f73c2800700 1 -- 172.21.15.86:0/2395641129 shutdown_connections 2024-03-23T17:52:08.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.263+0000 7f73c2800700 1 --2- 172.21.15.86:0/2395641129 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f73bc106080 0x7f73bc10dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.263+0000 7f73c2800700 1 --2- 172.21.15.86:0/2395641129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73bc100f60 0x7f73bc101340 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.263+0000 7f73c2800700 1 --2- 172.21.15.86:0/2395641129 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73bc101910 0x7f73bc105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.263+0000 7f73c2800700 1 -- 172.21.15.86:0/2395641129 >> 172.21.15.86:0/2395641129 conn(0x7f73bc0fc7c0 msgr2=0x7f73bc0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:08.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.263+0000 7f73c2800700 1 -- 172.21.15.86:0/2395641129 shutdown_connections 2024-03-23T17:52:08.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.263+0000 7f73c2800700 1 -- 172.21.15.86:0/2395641129 wait complete. 2024-03-23T17:52:08.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.265+0000 7f73c2800700 1 Processor -- start 2024-03-23T17:52:08.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.265+0000 7f73c2800700 1 -- start start 2024-03-23T17:52:08.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.266+0000 7f73c2800700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f73bc100f60 0x7f73bc0730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:08.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.266+0000 7f73c2800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73bc106080 0x7f73bc0735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:08.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.266+0000 7f73c2800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73bc073b30 0x7f73bc1a5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:08.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.266+0000 7f73c2800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73bc1a59d0 con 0x7f73bc073b30 2024-03-23T17:52:08.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.266+0000 7f73c2800700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73bc1a5b40 con 0x7f73bc106080 2024-03-23T17:52:08.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.266+0000 7f73c2800700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73bc1a5d00 con 0x7f73bc100f60 2024-03-23T17:52:08.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.266+0000 7f73baa00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73bc106080 0x7f73bc0735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:08.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.266+0000 7f73bbe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73bc073b30 0x7f73bc1a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:08.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.266+0000 7f73baa00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73bc106080 0x7f73bc0735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:41244/0 (socket says 172.21.15.86:41244) 2024-03-23T17:52:08.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.266+0000 7f73baa00700 1 -- 172.21.15.86:0/3453338888 learned_addr learned my addr 172.21.15.86:0/3453338888 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:08.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.266+0000 7f73bb400700 1 --2- 172.21.15.86:0/3453338888 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f73bc100f60 0x7f73bc0730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:08.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.266+0000 7f73baa00700 1 -- 172.21.15.86:0/3453338888 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f73bc100f60 msgr2=0x7f73bc0730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:08.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.266+0000 7f73baa00700 1 --2- 172.21.15.86:0/3453338888 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f73bc100f60 0x7f73bc0730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.266+0000 7f73baa00700 1 -- 172.21.15.86:0/3453338888 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73bc073b30 msgr2=0x7f73bc1a5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:08.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.266+0000 7f73baa00700 1 --2- 172.21.15.86:0/3453338888 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73bc073b30 0x7f73bc1a5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.266+0000 7f73baa00700 1 -- 172.21.15.86:0/3453338888 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73ac0095f0 con 0x7f73bc106080 2024-03-23T17:52:08.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.266+0000 7f73bbe00700 1 --2- 172.21.15.86:0/3453338888 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73bc073b30 0x7f73bc1a5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:52:08.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.267+0000 7f73baa00700 1 --2- 172.21.15.86:0/3453338888 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73bc106080 0x7f73bc0735f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f73a800e970 tx=0x7f73a800ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:08.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.267+0000 7f73b3e00700 1 -- 172.21.15.86:0/3453338888 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a8009790 con 0x7f73bc106080 2024-03-23T17:52:08.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.267+0000 7f73c2800700 1 -- 172.21.15.86:0/3453338888 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73bc1a5fe0 con 0x7f73bc106080 2024-03-23T17:52:08.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.267+0000 7f73b3e00700 1 -- 172.21.15.86:0/3453338888 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73a800eec0 con 0x7f73bc106080 2024-03-23T17:52:08.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.267+0000 7f73b3e00700 1 -- 172.21.15.86:0/3453338888 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a8005590 con 0x7f73bc106080 2024-03-23T17:52:08.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.267+0000 7f73c2800700 1 -- 172.21.15.86:0/3453338888 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73bc1a6460 con 0x7f73bc106080 2024-03-23T17:52:08.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.270+0000 7f73b3e00700 1 -- 172.21.15.86:0/3453338888 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 25) v1 ==== 136098+0+0 (secure 0 0 0) 0x7f73a800cbd0 con 0x7f73bc106080 2024-03-23T17:52:08.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.271+0000 7f73d62feb80 1 Processor -- start 2024-03-23T17:52:08.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.271+0000 7f73d62feb80 1 -- start start 2024-03-23T17:52:08.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.271+0000 7f73b3e00700 1 --2- 172.21.15.86:0/3453338888 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f73a00a13b0 0x7f73a00a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:08.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.272+0000 7f73b3e00700 1 -- 172.21.15.86:0/3453338888 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f73a8014070 con 0x7f73bc106080 2024-03-23T17:52:08.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.272+0000 7f73d62feb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56466e5b95d0 0x56466e5b99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:08.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.272+0000 7f73d62feb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56466e5b9ef0 0x56466e5c48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:08.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.272+0000 7f73d62feb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56466e5c4df0 0x56466e5c71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:08.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.272+0000 7f73d62feb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56466e512990 con 0x56466e5b95d0 2024-03-23T17:52:08.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.272+0000 7f73d62feb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x56466e510350 con 0x56466e5c4df0 2024-03-23T17:52:08.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.272+0000 7f73d62feb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x56466e50bc40 con 0x56466e5b9ef0 2024-03-23T17:52:08.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.272+0000 7f73bbe00700 1 --2- 172.21.15.86:0/3453338888 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f73a00a13b0 0x7f73a00a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:08.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.272+0000 7f73baa00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56466e5b9ef0 0x56466e5c48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:08.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.272+0000 7f73bb400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56466e5b95d0 0x56466e5b99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:08.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.272+0000 7f73baa00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56466e5b9ef0 0x56466e5c48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:51016/0 (socket says 172.21.15.86:51016) 2024-03-23T17:52:08.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.272+0000 7f73baa00700 1 -- 172.21.15.86:0/3688585129 learned_addr learned my addr 172.21.15.86:0/3688585129 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:08.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.272+0000 7f73bbe00700 1 --2- 172.21.15.86:0/3688585129 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56466e5c4df0 0x56466e5c71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:08.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.272+0000 7f73baa00700 1 -- 172.21.15.86:0/3688585129 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56466e5c4df0 msgr2=0x56466e5c71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:08.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.272+0000 7f73baa00700 1 --2- 172.21.15.86:0/3688585129 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56466e5c4df0 0x56466e5c71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.272+0000 7f73baa00700 1 -- 172.21.15.86:0/3688585129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56466e5b95d0 msgr2=0x56466e5b99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:08.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.272+0000 7f73baa00700 1 --2- 172.21.15.86:0/3688585129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56466e5b95d0 0x56466e5b99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.273+0000 7f73baa00700 1 -- 172.21.15.86:0/3688585129 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56466e38d3b0 con 0x56466e5b9ef0 2024-03-23T17:52:08.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.273+0000 7f73bbe00700 1 --2- 172.21.15.86:0/3688585129 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56466e5c4df0 0x56466e5c71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T17:52:08.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.273+0000 7f73bb400700 1 --2- 172.21.15.86:0/3688585129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56466e5b95d0 0x56466e5b99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:52:08.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.273+0000 7f73baa00700 1 --2- 172.21.15.86:0/3688585129 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56466e5b9ef0 0x56466e5c48b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f73a80965e0 tx=0x7f73a809a860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=509c86f9e6524b8d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:08.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.273+0000 7f73bbe00700 1 --2- 172.21.15.86:0/3453338888 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f73a00a13b0 0x7f73a00a3870 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f73b4009550 tx=0x7f73b40079a0 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:08.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.273+0000 7f73b0c00700 1 -- 172.21.15.86:0/3688585129 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a80cd070 con 0x56466e5b9ef0 2024-03-23T17:52:08.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.273+0000 7f73b0c00700 1 -- 172.21.15.86:0/3688585129 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73a809df10 con 0x56466e5b9ef0 2024-03-23T17:52:08.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.273+0000 7f73b0c00700 1 -- 172.21.15.86:0/3688585129 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a80995d0 con 0x56466e5b9ef0 2024-03-23T17:52:08.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.274+0000 7f73d62feb80 1 -- 172.21.15.86:0/3688585129 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56466e5b9ef0 msgr2=0x56466e5c48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:08.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.274+0000 7f73d62feb80 1 --2- 172.21.15.86:0/3688585129 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56466e5b9ef0 0x56466e5c48b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f73a80965e0 tx=0x7f73a809a860 comp rx=0 tx=0).stop 2024-03-23T17:52:08.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.274+0000 7f73d62feb80 1 -- 172.21.15.86:0/3688585129 shutdown_connections 2024-03-23T17:52:08.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.274+0000 7f73d62feb80 1 --2- 172.21.15.86:0/3688585129 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56466e5c4df0 0x56466e5c71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.274+0000 7f73d62feb80 1 --2- 172.21.15.86:0/3688585129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56466e5b95d0 0x56466e5b99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.274+0000 7f73d62feb80 1 --2- 172.21.15.86:0/3688585129 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56466e5b9ef0 0x56466e5c48b0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.274+0000 7f73d62feb80 1 -- 172.21.15.86:0/3688585129 >> 172.21.15.86:0/3688585129 conn(0x56466e5196a0 msgr2=0x56466e5b5a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:08.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.274+0000 7f73d62feb80 1 -- 172.21.15.86:0/3688585129 shutdown_connections 2024-03-23T17:52:08.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.274+0000 7f73d62feb80 1 -- 172.21.15.86:0/3688585129 wait complete. 2024-03-23T17:52:08.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.274+0000 7f73d62feb80 1 -- start start 2024-03-23T17:52:08.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.275+0000 7f73d62feb80 1 Processor -- start 2024-03-23T17:52:08.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.275+0000 7f73d62feb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56466e5b95d0 0x56466e5c4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:08.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.276+0000 7f73d62feb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56466e5b9ef0 0x56466e5c1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:08.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.276+0000 7f73d62feb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56466e5c4df0 0x56466e5c22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:08.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.276+0000 7f73d62feb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56466e4a7d80 con 0x56466e5c4df0 2024-03-23T17:52:08.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.276+0000 7f73d62feb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x56466e4a7a20 con 0x56466e5b95d0 2024-03-23T17:52:08.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.276+0000 7f73d62feb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x56466e38f6a0 con 0x56466e5b9ef0 2024-03-23T17:52:08.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.276+0000 7f73d62feb80 10 client.0 _ll_register_callbacks cb 0x56466e4b1470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:52:08.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.276+0000 7f73bbe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56466e5c4df0 0x56466e5c22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:08.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.276+0000 7f73baa00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56466e5b9ef0 0x56466e5c1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:08.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.276+0000 7f73bb400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56466e5b95d0 0x56466e5c4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:08.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.276+0000 7f73bbe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56466e5c4df0 0x56466e5c22c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59064/0 (socket says 172.21.15.86:59064) 2024-03-23T17:52:08.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.276+0000 7f73baa00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56466e5b9ef0 0x56466e5c1b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:51022/0 (socket says 172.21.15.86:51022) 2024-03-23T17:52:08.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.276+0000 7f73bbe00700 1 -- 172.21.15.86:0/1090435015 learned_addr learned my addr 172.21.15.86:0/1090435015 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:08.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.276+0000 7f73bbe00700 1 -- 172.21.15.86:0/1090435015 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56466e5b9ef0 msgr2=0x56466e5c1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:08.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.276+0000 7f73bbe00700 1 --2- 172.21.15.86:0/1090435015 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56466e5b9ef0 0x56466e5c1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.276+0000 7f73bbe00700 1 -- 172.21.15.86:0/1090435015 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56466e5b95d0 msgr2=0x56466e5c4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:08.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.276+0000 7f73bbe00700 1 --2- 172.21.15.86:0/1090435015 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56466e5b95d0 0x56466e5c4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.276+0000 7f73bbe00700 1 -- 172.21.15.86:0/1090435015 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73bc10e470 con 0x56466e5c4df0 2024-03-23T17:52:08.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.276+0000 7f73baa00700 1 --2- 172.21.15.86:0/1090435015 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56466e5b9ef0 0x56466e5c1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:52:08.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.276+0000 7f73bbe00700 1 --2- 172.21.15.86:0/1090435015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56466e5c4df0 0x56466e5c22c0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f73b4004500 tx=0x7f73b4004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cadac152142c3845 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:08.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.277+0000 7f73a5600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-23T17:52:08.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.277+0000 7f73a5600700 1 -- 172.21.15.86:0/1090435015 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b4018b10 con 0x56466e5c4df0 2024-03-23T17:52:08.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.277+0000 7f73a5600700 1 -- 172.21.15.86:0/1090435015 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f73b4018c70 con 0x56466e5c4df0 2024-03-23T17:52:08.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.277+0000 7f73d62feb80 1 -- 172.21.15.86:0/1090435015 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56466e4cc380 con 0x56466e5c4df0 2024-03-23T17:52:08.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.277+0000 7f73a5600700 1 -- 172.21.15.86:0/1090435015 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f73b4018e30 con 0x56466e5c4df0 2024-03-23T17:52:08.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.277+0000 7f73d62feb80 10 client.15891 fetch_fsmap learned FSMap version 30 2024-03-23T17:52:08.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.277+0000 7f73d62feb80 1 -- 172.21.15.86:0/1090435015 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f73bc1a6460 con 0x56466e5c4df0 2024-03-23T17:52:08.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.277+0000 7f73a5600700 1 -- 172.21.15.86:0/1090435015 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73b4031020 con 0x56466e5c4df0 2024-03-23T17:52:08.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.277+0000 7f73a5600700 1 -- 172.21.15.86:0/1090435015 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f73b40235c0 con 0x56466e5c4df0 2024-03-23T17:52:08.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.277+0000 7f73d62feb80 10 client.15891 fetch_fsmap finished waiting for FSMap version 30 2024-03-23T17:52:08.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.277+0000 7f73d62feb80 10 client.15891 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.d 2024-03-23T17:52:08.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.278+0000 7f73d62feb80 20 client.15891 populate_metadata read hostname 'smithi086' 2024-03-23T17:52:08.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.278+0000 7f73d62feb80 1 --2- 172.21.15.86:0/1090435015 >> [v2:172.21.15.86:6836/2933881328,v1:172.21.15.86:6837/2933881328] conn(0x56466e5d49c0 0x56466e5d6de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:08.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.278+0000 7f73d62feb80 4 client.15891 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:52:08.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.278+0000 7f73d62feb80 1 -- 172.21.15.86:0/1090435015 --> [v2:172.21.15.86:6836/2933881328,v1:172.21.15.86:6837/2933881328] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56466e504700 con 0x56466e5d49c0 2024-03-23T17:52:08.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.278+0000 7f73bb400700 1 --2- 172.21.15.86:0/1090435015 >> [v2:172.21.15.86:6836/2933881328,v1:172.21.15.86:6837/2933881328] conn(0x56466e5d49c0 0x56466e5d6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:08.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.278+0000 7f73bb400700 1 --2- 172.21.15.86:0/1090435015 >> [v2:172.21.15.86:6836/2933881328,v1:172.21.15.86:6837/2933881328] conn(0x56466e5d49c0 0x56466e5d6de0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9a278200c6f480f5 server_cookie=9f33b8987c6bd46d in_seq=0 out_seq=0 2024-03-23T17:52:08.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.278+0000 7f73a5600700 10 client.15891 ms_handle_connect on v2:172.21.15.86:6836/2933881328 2024-03-23T17:52:08.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.279+0000 7f73a5600700 1 -- 172.21.15.86:0/1090435015 <== mds.3 v2:172.21.15.86:6836/2933881328 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x56466e504700 con 0x56466e5d49c0 2024-03-23T17:52:08.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.279+0000 7f73a5600700 10 client.15891 handle_command_reply: tid=0 2024-03-23T17:52:08.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.279+0000 7f73d62feb80 1 client.15891 shutdown 2024-03-23T17:52:08.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.279+0000 7f73d62feb80 1 -- 172.21.15.86:0/1090435015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56466e5c4df0 msgr2=0x56466e5c22c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:08.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.279+0000 7f73d62feb80 1 --2- 172.21.15.86:0/1090435015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56466e5c4df0 0x56466e5c22c0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f73b4004500 tx=0x7f73b4004b40 comp rx=0 tx=0).stop 2024-03-23T17:52:08.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.279+0000 7f73d62feb80 1 -- 172.21.15.86:0/1090435015 shutdown_connections 2024-03-23T17:52:08.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.279+0000 7f73d62feb80 1 --2- 172.21.15.86:0/1090435015 >> [v2:172.21.15.86:6836/2933881328,v1:172.21.15.86:6837/2933881328] conn(0x56466e5d49c0 0x56466e5d6de0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.279+0000 7f73d62feb80 1 --2- 172.21.15.86:0/1090435015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56466e5c4df0 0x56466e5c22c0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.279+0000 7f73d62feb80 1 --2- 172.21.15.86:0/1090435015 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56466e5b95d0 0x56466e5c4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.279+0000 7f73d62feb80 1 --2- 172.21.15.86:0/1090435015 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56466e5b9ef0 0x56466e5c1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.279+0000 7f73d62feb80 1 -- 172.21.15.86:0/1090435015 >> 172.21.15.86:0/1090435015 conn(0x56466e5196a0 msgr2=0x56466e5bb6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:08.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.279+0000 7f73a5600700 0 client.15891 ms_handle_reset on v2:172.21.15.86:6836/2933881328 2024-03-23T17:52:08.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.279+0000 7f73d62feb80 1 -- 172.21.15.86:0/1090435015 shutdown_connections 2024-03-23T17:52:08.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.279+0000 7f73d62feb80 1 -- 172.21.15.86:0/1090435015 wait complete. 2024-03-23T17:52:08.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.280+0000 7f73d62feb80 20 client.15891 trim_cache size 0 max 16384 2024-03-23T17:52:08.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.292+0000 7f73d62feb80 1 Processor -- start 2024-03-23T17:52:08.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.292+0000 7f73d62feb80 1 -- start start 2024-03-23T17:52:08.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.293+0000 7f73d62feb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56466e5b95d0 0x56466e5cba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:08.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.293+0000 7f73d62feb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56466e5b9ef0 0x56466e5ae5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:08.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.293+0000 7f73d62feb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56466e5c4df0 0x56466e5bce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:08.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.293+0000 7f73d62feb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56466e5b5fa0 con 0x56466e5b9ef0 2024-03-23T17:52:08.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.293+0000 7f73d62feb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x56466e5b6660 con 0x56466e5c4df0 2024-03-23T17:52:08.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.293+0000 7f73d62feb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x56466e5c00f0 con 0x56466e5b95d0 2024-03-23T17:52:08.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.293+0000 7f73baa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56466e5b9ef0 0x56466e5ae5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:08.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.293+0000 7f73baa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56466e5b9ef0 0x56466e5ae5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59074/0 (socket says 172.21.15.86:59074) 2024-03-23T17:52:08.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.293+0000 7f73baa00700 1 -- 172.21.15.86:0/2713015700 learned_addr learned my addr 172.21.15.86:0/2713015700 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:08.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.293+0000 7f73bbe00700 1 --2- 172.21.15.86:0/2713015700 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56466e5c4df0 0x56466e5bce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:08.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.293+0000 7f73bb400700 1 --2- 172.21.15.86:0/2713015700 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56466e5b95d0 0x56466e5cba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:08.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.293+0000 7f73baa00700 1 -- 172.21.15.86:0/2713015700 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56466e5b95d0 msgr2=0x56466e5cba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:08.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.293+0000 7f73baa00700 1 --2- 172.21.15.86:0/2713015700 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56466e5b95d0 0x56466e5cba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.293+0000 7f73baa00700 1 -- 172.21.15.86:0/2713015700 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56466e5c4df0 msgr2=0x56466e5bce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:08.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.293+0000 7f73baa00700 1 --2- 172.21.15.86:0/2713015700 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56466e5c4df0 0x56466e5bce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.293+0000 7f73baa00700 1 -- 172.21.15.86:0/2713015700 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56466e339b60 con 0x56466e5b9ef0 2024-03-23T17:52:08.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.294+0000 7f73baa00700 1 --2- 172.21.15.86:0/2713015700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56466e5b9ef0 0x56466e5ae5a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f73a8099bb0 tx=0x7f73a8016330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce986dea3851e80b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:08.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.294+0000 7f73b1600700 1 -- 172.21.15.86:0/2713015700 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a80cd040 con 0x56466e5b9ef0 2024-03-23T17:52:08.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.294+0000 7f73b1600700 1 -- 172.21.15.86:0/2713015700 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73a80a2560 con 0x56466e5b9ef0 2024-03-23T17:52:08.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.294+0000 7f73b1600700 1 -- 172.21.15.86:0/2713015700 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a8099830 con 0x56466e5b9ef0 2024-03-23T17:52:08.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.294+0000 7f73d62feb80 1 -- 172.21.15.86:0/2713015700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56466e5b9ef0 msgr2=0x56466e5ae5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:08.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.294+0000 7f73d62feb80 1 --2- 172.21.15.86:0/2713015700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56466e5b9ef0 0x56466e5ae5a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f73a8099bb0 tx=0x7f73a8016330 comp rx=0 tx=0).stop 2024-03-23T17:52:08.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.295+0000 7f73d62feb80 1 -- 172.21.15.86:0/2713015700 shutdown_connections 2024-03-23T17:52:08.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.295+0000 7f73d62feb80 1 --2- 172.21.15.86:0/2713015700 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56466e5c4df0 0x56466e5bce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.295+0000 7f73d62feb80 1 --2- 172.21.15.86:0/2713015700 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56466e5b95d0 0x56466e5cba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.295+0000 7f73d62feb80 1 --2- 172.21.15.86:0/2713015700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56466e5b9ef0 0x56466e5ae5a0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.295+0000 7f73d62feb80 1 -- 172.21.15.86:0/2713015700 >> 172.21.15.86:0/2713015700 conn(0x56466e5196a0 msgr2=0x56466e5aef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:08.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.295+0000 7f73d62feb80 1 -- 172.21.15.86:0/2713015700 shutdown_connections 2024-03-23T17:52:08.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.295+0000 7f73d62feb80 1 -- 172.21.15.86:0/2713015700 wait complete. 2024-03-23T17:52:08.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.295+0000 7f73d62feb80 1 -- start start 2024-03-23T17:52:08.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.295+0000 7f73d62feb80 1 Processor -- start 2024-03-23T17:52:08.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.296+0000 7f73d62feb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56466e5b95d0 0x56466e5c9a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:08.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.296+0000 7f73d62feb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56466e5b9ef0 0x56466e5c9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:08.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.296+0000 7f73d62feb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56466e5c4df0 0x56466e5ca4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:08.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.296+0000 7f73d62feb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56466e625650 con 0x56466e5c4df0 2024-03-23T17:52:08.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.296+0000 7f73d62feb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x56466e6257c0 con 0x56466e5b9ef0 2024-03-23T17:52:08.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.296+0000 7f73d62feb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x56466e625980 con 0x56466e5b95d0 2024-03-23T17:52:08.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.296+0000 7f73d62feb80 10 client.0 _ll_register_callbacks cb 0x56466e4b1470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:52:08.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.296+0000 7f73baa00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56466e5b9ef0 0x56466e5c9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:08.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.296+0000 7f73baa00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56466e5b9ef0 0x56466e5c9f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:41284/0 (socket says 172.21.15.86:41284) 2024-03-23T17:52:08.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.296+0000 7f73baa00700 1 -- 172.21.15.86:0/1208239566 learned_addr learned my addr 172.21.15.86:0/1208239566 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:08.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.296+0000 7f73bb400700 1 --2- 172.21.15.86:0/1208239566 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56466e5b95d0 0x56466e5c9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:08.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.296+0000 7f73baa00700 1 -- 172.21.15.86:0/1208239566 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56466e5b95d0 msgr2=0x56466e5c9a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:08.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.296+0000 7f73baa00700 1 --2- 172.21.15.86:0/1208239566 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56466e5b95d0 0x56466e5c9a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.296+0000 7f73bbe00700 1 --2- 172.21.15.86:0/1208239566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56466e5c4df0 0x56466e5ca4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:08.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.296+0000 7f73baa00700 1 -- 172.21.15.86:0/1208239566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56466e5c4df0 msgr2=0x56466e5ca4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:08.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.296+0000 7f73baa00700 1 --2- 172.21.15.86:0/1208239566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56466e5c4df0 0x56466e5ca4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.296+0000 7f73baa00700 1 -- 172.21.15.86:0/1208239566 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73a80b6040 con 0x56466e5b9ef0 2024-03-23T17:52:08.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.296+0000 7f73bbe00700 1 --2- 172.21.15.86:0/1208239566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56466e5c4df0 0x56466e5ca4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T17:52:08.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.296+0000 7f73baa00700 1 --2- 172.21.15.86:0/1208239566 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56466e5b9ef0 0x56466e5c9f90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f73a809a070 tx=0x7f73a80b82b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c42590ceebf8e0e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:08.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.296+0000 7f73a5600700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T17:52:08.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.297+0000 7f73a5600700 1 -- 172.21.15.86:0/1208239566 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a809a550 con 0x56466e5b9ef0 2024-03-23T17:52:08.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.297+0000 7f73a5600700 1 -- 172.21.15.86:0/1208239566 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f73a8063070 con 0x56466e5b9ef0 2024-03-23T17:52:08.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.297+0000 7f73d62feb80 1 -- 172.21.15.86:0/1208239566 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f73a809df10 con 0x56466e5b9ef0 2024-03-23T17:52:08.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.297+0000 7f73a5600700 1 -- 172.21.15.86:0/1208239566 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f73a80995d0 con 0x56466e5b9ef0 2024-03-23T17:52:08.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.297+0000 7f73d62feb80 10 client.25952 fetch_fsmap learned FSMap version 30 2024-03-23T17:52:08.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.297+0000 7f73d62feb80 1 -- 172.21.15.86:0/1208239566 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56466e4d1ea0 con 0x56466e5b9ef0 2024-03-23T17:52:08.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.297+0000 7f73a5600700 1 -- 172.21.15.86:0/1208239566 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73a809df10 con 0x56466e5b9ef0 2024-03-23T17:52:08.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.297+0000 7f73a5600700 1 -- 172.21.15.86:0/1208239566 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 30) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f73a807c070 con 0x56466e5b9ef0 2024-03-23T17:52:08.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.297+0000 7f73d62feb80 10 client.25952 fetch_fsmap finished waiting for FSMap version 30 2024-03-23T17:52:08.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.297+0000 7f73d62feb80 10 client.25952 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.d 2024-03-23T17:52:08.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.297+0000 7f73d62feb80 20 client.25952 populate_metadata read hostname 'smithi086' 2024-03-23T17:52:08.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.297+0000 7f73d62feb80 1 --2- 172.21.15.86:0/1208239566 >> [v2:172.21.15.86:6836/2933881328,v1:172.21.15.86:6837/2933881328] conn(0x56466e626c70 0x56466e629060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:08.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.297+0000 7f73d62feb80 4 client.25952 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-23T17:52:08.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.297+0000 7f73d62feb80 1 -- 172.21.15.86:0/1208239566 --> [v2:172.21.15.86:6836/2933881328,v1:172.21.15.86:6837/2933881328] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56466e35acf0 con 0x56466e626c70 2024-03-23T17:52:08.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.298+0000 7f73bb400700 1 --2- 172.21.15.86:0/1208239566 >> [v2:172.21.15.86:6836/2933881328,v1:172.21.15.86:6837/2933881328] conn(0x56466e626c70 0x56466e629060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:08.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.298+0000 7f73a5600700 10 client.25952 ms_handle_connect on v2:172.21.15.86:6836/2933881328 2024-03-23T17:52:08.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.298+0000 7f73bb400700 1 --2- 172.21.15.86:0/1208239566 >> [v2:172.21.15.86:6836/2933881328,v1:172.21.15.86:6837/2933881328] conn(0x56466e626c70 0x56466e629060 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ff714e43d1d143d0 server_cookie=ebe8934eb49f844f in_seq=0 out_seq=0 2024-03-23T17:52:08.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.298+0000 7f73a5600700 1 -- 172.21.15.86:0/1208239566 <== mds.3 v2:172.21.15.86:6836/2933881328 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56466e35acf0 con 0x56466e626c70 2024-03-23T17:52:08.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.298+0000 7f73a5600700 10 client.25952 handle_command_reply: tid=0 2024-03-23T17:52:08.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.298+0000 7f73d62feb80 1 client.25952 shutdown 2024-03-23T17:52:08.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.298+0000 7f73d62feb80 1 -- 172.21.15.86:0/1208239566 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56466e5b9ef0 msgr2=0x56466e5c9f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:08.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.298+0000 7f73d62feb80 1 --2- 172.21.15.86:0/1208239566 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56466e5b9ef0 0x56466e5c9f90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f73a809a070 tx=0x7f73a80b82b0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.298+0000 7f73d62feb80 1 -- 172.21.15.86:0/1208239566 shutdown_connections 2024-03-23T17:52:08.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.298+0000 7f73d62feb80 1 --2- 172.21.15.86:0/1208239566 >> [v2:172.21.15.86:6836/2933881328,v1:172.21.15.86:6837/2933881328] conn(0x56466e626c70 0x56466e629060 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.299+0000 7f73d62feb80 1 --2- 172.21.15.86:0/1208239566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56466e5c4df0 0x56466e5ca4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.299+0000 7f73d62feb80 1 --2- 172.21.15.86:0/1208239566 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56466e5b95d0 0x56466e5c9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.299+0000 7f73a5600700 0 client.25952 ms_handle_reset on v2:172.21.15.86:6836/2933881328 2024-03-23T17:52:08.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.299+0000 7f73d62feb80 1 --2- 172.21.15.86:0/1208239566 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56466e5b9ef0 0x56466e5c9f90 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.299+0000 7f73d62feb80 1 -- 172.21.15.86:0/1208239566 >> 172.21.15.86:0/1208239566 conn(0x56466e5196a0 msgr2=0x56466e5d0040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:08.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.299+0000 7f73d62feb80 1 -- 172.21.15.86:0/1208239566 shutdown_connections 2024-03-23T17:52:08.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.299+0000 7f73d62feb80 1 -- 172.21.15.86:0/1208239566 wait complete. 2024-03-23T17:52:08.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.299+0000 7f73d62feb80 20 client.25952 trim_cache size 0 max 16384 2024-03-23T17:52:08.306 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-03-23T17:52:08.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.299+0000 7f73c2800700 1 -- 172.21.15.86:0/3453338888 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f73a00a13b0 msgr2=0x7f73a00a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:08.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.299+0000 7f73c2800700 1 --2- 172.21.15.86:0/3453338888 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f73a00a13b0 0x7f73a00a3870 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f73b4009550 tx=0x7f73b40079a0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.299+0000 7f73c2800700 1 -- 172.21.15.86:0/3453338888 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73bc106080 msgr2=0x7f73bc0735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:08.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.299+0000 7f73c2800700 1 --2- 172.21.15.86:0/3453338888 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73bc106080 0x7f73bc0735f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f73a800e970 tx=0x7f73a800ec80 comp rx=0 tx=0).stop 2024-03-23T17:52:08.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.299+0000 7f73c2800700 1 -- 172.21.15.86:0/3453338888 shutdown_connections 2024-03-23T17:52:08.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.299+0000 7f73c2800700 1 --2- 172.21.15.86:0/3453338888 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73bc073b30 0x7f73bc1a5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.299+0000 7f73c2800700 1 --2- 172.21.15.86:0/3453338888 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f73a00a13b0 0x7f73a00a3870 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.299+0000 7f73c2800700 1 --2- 172.21.15.86:0/3453338888 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f73bc100f60 0x7f73bc0730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.299+0000 7f73c2800700 1 --2- 172.21.15.86:0/3453338888 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73bc106080 0x7f73bc0735f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.299+0000 7f73c2800700 1 -- 172.21.15.86:0/3453338888 >> 172.21.15.86:0/3453338888 conn(0x7f73bc0fc7c0 msgr2=0x7f73bc0fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:08.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.300+0000 7f73c2800700 1 -- 172.21.15.86:0/3453338888 shutdown_connections 2024-03-23T17:52:08.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.300+0000 7f73c2800700 1 -- 172.21.15.86:0/3453338888 wait complete. 2024-03-23T17:52:08.319 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-23T17:52:08.319 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-23T17:52:08.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.513+0000 7f45a4000700 1 -- 172.21.15.86:0/3763011327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f459c105ce0 msgr2=0x7f459c1060c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:08.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.513+0000 7f45a4000700 1 --2- 172.21.15.86:0/3763011327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f459c105ce0 0x7f459c1060c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f4590009990 tx=0x7f4590009ca0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.513+0000 7f45a4000700 1 -- 172.21.15.86:0/3763011327 shutdown_connections 2024-03-23T17:52:08.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.513+0000 7f45a4000700 1 --2- 172.21.15.86:0/3763011327 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f459c0687f0 0x7f459c10db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.513+0000 7f45a4000700 1 --2- 172.21.15.86:0/3763011327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f459c105ce0 0x7f459c1060c0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.513+0000 7f45a4000700 1 --2- 172.21.15.86:0/3763011327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f459c067e30 0x7f459c0682b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.513+0000 7f45a4000700 1 -- 172.21.15.86:0/3763011327 >> 172.21.15.86:0/3763011327 conn(0x7f459c076d40 msgr2=0x7f459c077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:08.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.514+0000 7f45a4000700 1 -- 172.21.15.86:0/3763011327 shutdown_connections 2024-03-23T17:52:08.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.514+0000 7f45a4000700 1 -- 172.21.15.86:0/3763011327 wait complete. 2024-03-23T17:52:08.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.515+0000 7f45a4000700 1 Processor -- start 2024-03-23T17:52:08.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.515+0000 7f45a4000700 1 -- start start 2024-03-23T17:52:08.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.516+0000 7f45a4000700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f459c067e30 0x7f459c197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:08.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.516+0000 7f45a4000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f459c0687f0 0x7f459c197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:08.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.516+0000 7f45a4000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f459c105ce0 0x7f459c19bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:08.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.516+0000 7f45a4000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f459c19c160 con 0x7f459c105ce0 2024-03-23T17:52:08.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.516+0000 7f45a4000700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f459c19c2d0 con 0x7f459c0687f0 2024-03-23T17:52:08.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.516+0000 7f45a4000700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f459c19c490 con 0x7f459c067e30 2024-03-23T17:52:08.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.516+0000 7f45a2000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f459c105ce0 0x7f459c19bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:08.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.516+0000 7f45a1600700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f459c067e30 0x7f459c197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:08.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.516+0000 7f45a1600700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f459c067e30 0x7f459c197240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:51056/0 (socket says 172.21.15.86:51056) 2024-03-23T17:52:08.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.516+0000 7f45a1600700 1 -- 172.21.15.86:0/202144085 learned_addr learned my addr 172.21.15.86:0/202144085 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:08.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.516+0000 7f45a0c00700 1 --2- 172.21.15.86:0/202144085 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f459c0687f0 0x7f459c197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:08.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.516+0000 7f45a0c00700 1 -- 172.21.15.86:0/202144085 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f459c067e30 msgr2=0x7f459c197240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:08.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.516+0000 7f45a0c00700 1 --2- 172.21.15.86:0/202144085 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f459c067e30 0x7f459c197240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.516+0000 7f45a0c00700 1 -- 172.21.15.86:0/202144085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f459c105ce0 msgr2=0x7f459c19bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:08.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.516+0000 7f45a0c00700 1 --2- 172.21.15.86:0/202144085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f459c105ce0 0x7f459c19bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.516+0000 7f45a0c00700 1 -- 172.21.15.86:0/202144085 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45900095f0 con 0x7f459c0687f0 2024-03-23T17:52:08.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.516+0000 7f45a1600700 1 --2- 172.21.15.86:0/202144085 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f459c067e30 0x7f459c197240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:52:08.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.516+0000 7f45a2000700 1 --2- 172.21.15.86:0/202144085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f459c105ce0 0x7f459c19bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:52:08.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.517+0000 7f45a0c00700 1 --2- 172.21.15.86:0/202144085 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f459c0687f0 0x7f459c197780 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f458800e9f0 tx=0x7f458800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:08.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.517+0000 7f459a000700 1 -- 172.21.15.86:0/202144085 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f458800c9c0 con 0x7f459c0687f0 2024-03-23T17:52:08.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.517+0000 7f459a000700 1 -- 172.21.15.86:0/202144085 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4588010430 con 0x7f459c0687f0 2024-03-23T17:52:08.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.517+0000 7f45a4000700 1 -- 172.21.15.86:0/202144085 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f459c19c770 con 0x7f459c0687f0 2024-03-23T17:52:08.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.517+0000 7f459a000700 1 -- 172.21.15.86:0/202144085 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4588018680 con 0x7f459c0687f0 2024-03-23T17:52:08.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.517+0000 7f45a4000700 1 -- 172.21.15.86:0/202144085 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f459c0ffd80 con 0x7f459c0687f0 2024-03-23T17:52:08.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.521+0000 7f45b7befb80 1 Processor -- start 2024-03-23T17:52:08.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.521+0000 7f45b7befb80 1 -- start start 2024-03-23T17:52:08.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.522+0000 7f45b7befb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557569395d0 0x5557569399b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:08.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.522+0000 7f45b7befb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555756939ef0 0x5557569448b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:08.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.522+0000 7f45b7befb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555756944df0 0x5557569471e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:08.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.522+0000 7f45b7befb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x555756892990 con 0x5557569395d0 2024-03-23T17:52:08.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.522+0000 7f45b7befb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x555756890350 con 0x555756939ef0 2024-03-23T17:52:08.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.522+0000 7f45b7befb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55575688bc40 con 0x555756944df0 2024-03-23T17:52:08.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.522+0000 7f459a000700 1 -- 172.21.15.86:0/202144085 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 25) v1 ==== 136098+0+0 (secure 0 0 0) 0x7f4588010860 con 0x7f459c0687f0 2024-03-23T17:52:08.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.522+0000 7f45a2000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557569395d0 0x5557569399b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:08.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.522+0000 7f45a1600700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555756939ef0 0x5557569448b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:08.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.522+0000 7f45a1600700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555756939ef0 0x5557569448b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:41298/0 (socket says 172.21.15.86:41298) 2024-03-23T17:52:08.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.522+0000 7f45a1600700 1 -- 172.21.15.86:0/4162681211 learned_addr learned my addr 172.21.15.86:0/4162681211 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:08.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.522+0000 7f45a2000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557569395d0 0x5557569399b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59118/0 (socket says 172.21.15.86:59118) 2024-03-23T17:52:08.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.522+0000 7f45a0c00700 1 --2- 172.21.15.86:0/4162681211 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555756944df0 0x5557569471e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:08.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.522+0000 7f45a1600700 1 -- 172.21.15.86:0/4162681211 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555756944df0 msgr2=0x5557569471e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:08.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.522+0000 7f45a1600700 1 --2- 172.21.15.86:0/4162681211 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555756944df0 0x5557569471e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.522+0000 7f45a1600700 1 -- 172.21.15.86:0/4162681211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557569395d0 msgr2=0x5557569399b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:08.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.522+0000 7f45a1600700 1 --2- 172.21.15.86:0/4162681211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557569395d0 0x5557569399b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.522+0000 7f45a1600700 1 -- 172.21.15.86:0/4162681211 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55575670d3b0 con 0x555756939ef0 2024-03-23T17:52:08.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.522+0000 7f45a0c00700 1 --2- 172.21.15.86:0/4162681211 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555756944df0 0x5557569471e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T17:52:08.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.522+0000 7f45a2000700 1 --2- 172.21.15.86:0/4162681211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557569395d0 0x5557569399b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:52:08.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.523+0000 7f45a1600700 1 --2- 172.21.15.86:0/4162681211 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555756939ef0 0x5557569448b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f459000b660 tx=0x7f459000b690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9535a10d92ec57c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:08.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.523+0000 7f458e600700 1 -- 172.21.15.86:0/4162681211 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f459001d070 con 0x555756939ef0 2024-03-23T17:52:08.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.523+0000 7f458e600700 1 -- 172.21.15.86:0/4162681211 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f459001e040 con 0x555756939ef0 2024-03-23T17:52:08.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.523+0000 7f458e600700 1 -- 172.21.15.86:0/4162681211 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4590004ca0 con 0x555756939ef0 2024-03-23T17:52:08.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.524+0000 7f459a000700 1 --2- 172.21.15.86:0/202144085 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f45840a1430 0x7f45840a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:08.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.524+0000 7f459a000700 1 -- 172.21.15.86:0/202144085 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f4588014070 con 0x7f459c0687f0 2024-03-23T17:52:08.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.524+0000 7f45a2000700 1 --2- 172.21.15.86:0/202144085 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f45840a1430 0x7f45840a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:08.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.524+0000 7f45b7befb80 1 -- 172.21.15.86:0/4162681211 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555756939ef0 msgr2=0x5557569448b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:08.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.524+0000 7f45b7befb80 1 --2- 172.21.15.86:0/4162681211 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555756939ef0 0x5557569448b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f459000b660 tx=0x7f459000b690 comp rx=0 tx=0).stop 2024-03-23T17:52:08.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.524+0000 7f45b7befb80 1 -- 172.21.15.86:0/4162681211 shutdown_connections 2024-03-23T17:52:08.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.524+0000 7f45b7befb80 1 --2- 172.21.15.86:0/4162681211 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555756944df0 0x5557569471e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.524+0000 7f45b7befb80 1 --2- 172.21.15.86:0/4162681211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557569395d0 0x5557569399b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.524+0000 7f45b7befb80 1 --2- 172.21.15.86:0/4162681211 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555756939ef0 0x5557569448b0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.524+0000 7f45b7befb80 1 -- 172.21.15.86:0/4162681211 >> 172.21.15.86:0/4162681211 conn(0x5557568996a0 msgr2=0x555756935a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:08.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.524+0000 7f45b7befb80 1 -- 172.21.15.86:0/4162681211 shutdown_connections 2024-03-23T17:52:08.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.524+0000 7f45b7befb80 1 -- 172.21.15.86:0/4162681211 wait complete. 2024-03-23T17:52:08.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.525+0000 7f45a2000700 1 --2- 172.21.15.86:0/202144085 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f45840a1430 0x7f45840a38f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4594010a80 tx=0x7f4594012040 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:08.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.525+0000 7f45b7befb80 1 -- start start 2024-03-23T17:52:08.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.525+0000 7f45b7befb80 1 Processor -- start 2024-03-23T17:52:08.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.525+0000 7f45b7befb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557569395d0 0x5557569447b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:08.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.526+0000 7f45b7befb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555756939ef0 0x555756941aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:08.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.526+0000 7f45b7befb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555756944df0 0x5557569420b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:08.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.526+0000 7f45b7befb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x555756827d80 con 0x5557569395d0 2024-03-23T17:52:08.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.526+0000 7f45b7befb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x555756827a20 con 0x555756939ef0 2024-03-23T17:52:08.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.526+0000 7f45b7befb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55575670f6a0 con 0x555756944df0 2024-03-23T17:52:08.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.526+0000 7f45b7befb80 10 client.0 _ll_register_callbacks cb 0x555756831470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:52:08.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.526+0000 7f45a0c00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555756939ef0 0x555756941aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:08.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.526+0000 7f45a1600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557569395d0 0x5557569447b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:08.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.526+0000 7f45a0c00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555756939ef0 0x555756941aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:41308/0 (socket says 172.21.15.86:41308) 2024-03-23T17:52:08.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.526+0000 7f45a0c00700 1 -- 172.21.15.86:0/2789122263 learned_addr learned my addr 172.21.15.86:0/2789122263 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:08.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.526+0000 7f45a2000700 1 --2- 172.21.15.86:0/2789122263 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555756944df0 0x5557569420b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:08.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.526+0000 7f45a0c00700 1 -- 172.21.15.86:0/2789122263 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555756944df0 msgr2=0x5557569420b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:08.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.526+0000 7f45a0c00700 1 --2- 172.21.15.86:0/2789122263 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555756944df0 0x5557569420b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.526+0000 7f45a0c00700 1 -- 172.21.15.86:0/2789122263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557569395d0 msgr2=0x5557569447b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:08.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.526+0000 7f45a0c00700 1 --2- 172.21.15.86:0/2789122263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557569395d0 0x5557569447b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.526+0000 7f45a0c00700 1 -- 172.21.15.86:0/2789122263 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4594010880 con 0x555756939ef0 2024-03-23T17:52:08.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.526+0000 7f45a2000700 1 --2- 172.21.15.86:0/2789122263 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555756944df0 0x5557569420b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:52:08.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.527+0000 7f45a0c00700 1 --2- 172.21.15.86:0/2789122263 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555756939ef0 0x555756941aa0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4588099b70 tx=0x7f458809a3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58ff042e38dc62d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:08.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.527+0000 7f4582a00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T17:52:08.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.527+0000 7f4582a00700 1 -- 172.21.15.86:0/2789122263 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45880a2060 con 0x555756939ef0 2024-03-23T17:52:08.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.527+0000 7f4582a00700 1 -- 172.21.15.86:0/2789122263 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f4588099e70 con 0x555756939ef0 2024-03-23T17:52:08.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.527+0000 7f45b7befb80 1 -- 172.21.15.86:0/2789122263 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55575684c380 con 0x555756939ef0 2024-03-23T17:52:08.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.527+0000 7f4582a00700 1 -- 172.21.15.86:0/2789122263 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f458809dca0 con 0x555756939ef0 2024-03-23T17:52:08.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.527+0000 7f45b7befb80 10 client.25964 fetch_fsmap learned FSMap version 30 2024-03-23T17:52:08.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.527+0000 7f45b7befb80 1 -- 172.21.15.86:0/2789122263 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f459c10e550 con 0x555756939ef0 2024-03-23T17:52:08.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.527+0000 7f4582a00700 1 -- 172.21.15.86:0/2789122263 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55575684c380 con 0x555756939ef0 2024-03-23T17:52:08.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.527+0000 7f4582a00700 1 -- 172.21.15.86:0/2789122263 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 30) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f4588095c60 con 0x555756939ef0 2024-03-23T17:52:08.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.527+0000 7f45b7befb80 10 client.25964 fetch_fsmap finished waiting for FSMap version 30 2024-03-23T17:52:08.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.527+0000 7f45b7befb80 10 client.25964 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-23T17:52:08.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.528+0000 7f45b7befb80 20 client.25964 populate_metadata read hostname 'smithi086' 2024-03-23T17:52:08.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.528+0000 7f45b7befb80 1 --2- 172.21.15.86:0/2789122263 >> [v2:172.21.15.92:6836/3967837948,v1:172.21.15.92:6837/3967837948] conn(0x5557569548a0 0x555756956cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:08.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.528+0000 7f45b7befb80 4 client.25964 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:52:08.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.528+0000 7f45b7befb80 1 -- 172.21.15.86:0/2789122263 --> [v2:172.21.15.92:6836/3967837948,v1:172.21.15.92:6837/3967837948] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555756884700 con 0x5557569548a0 2024-03-23T17:52:08.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.528+0000 7f45a1600700 1 --2- 172.21.15.86:0/2789122263 >> [v2:172.21.15.92:6836/3967837948,v1:172.21.15.92:6837/3967837948] conn(0x5557569548a0 0x555756956cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:08.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.529+0000 7f4582a00700 10 client.25964 ms_handle_connect on v2:172.21.15.92:6836/3967837948 2024-03-23T17:52:08.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.529+0000 7f45a1600700 1 --2- 172.21.15.86:0/2789122263 >> [v2:172.21.15.92:6836/3967837948,v1:172.21.15.92:6837/3967837948] conn(0x5557569548a0 0x555756956cc0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7e4192d08d172d2b server_cookie=ff0f941133499953 in_seq=0 out_seq=0 2024-03-23T17:52:08.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.529+0000 7f4582a00700 1 -- 172.21.15.86:0/2789122263 <== mds.2 v2:172.21.15.92:6836/3967837948 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x555756884700 con 0x5557569548a0 2024-03-23T17:52:08.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.529+0000 7f4582a00700 10 client.25964 handle_command_reply: tid=0 2024-03-23T17:52:08.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.529+0000 7f45b7befb80 1 client.25964 shutdown 2024-03-23T17:52:08.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.530+0000 7f45b7befb80 1 -- 172.21.15.86:0/2789122263 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555756939ef0 msgr2=0x555756941aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:08.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.530+0000 7f45b7befb80 1 --2- 172.21.15.86:0/2789122263 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555756939ef0 0x555756941aa0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4588099b70 tx=0x7f458809a3b0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.530+0000 7f45b7befb80 1 -- 172.21.15.86:0/2789122263 shutdown_connections 2024-03-23T17:52:08.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.530+0000 7f45b7befb80 1 --2- 172.21.15.86:0/2789122263 >> [v2:172.21.15.92:6836/3967837948,v1:172.21.15.92:6837/3967837948] conn(0x5557569548a0 0x555756956cc0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.530+0000 7f45b7befb80 1 --2- 172.21.15.86:0/2789122263 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555756944df0 0x5557569420b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.530+0000 7f45b7befb80 1 --2- 172.21.15.86:0/2789122263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557569395d0 0x5557569447b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.530+0000 7f4582a00700 0 client.25964 ms_handle_reset on v2:172.21.15.92:6836/3967837948 2024-03-23T17:52:08.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.530+0000 7f45b7befb80 1 --2- 172.21.15.86:0/2789122263 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555756939ef0 0x555756941aa0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.530+0000 7f45b7befb80 1 -- 172.21.15.86:0/2789122263 >> 172.21.15.86:0/2789122263 conn(0x5557568996a0 msgr2=0x55575693b530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:08.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.530+0000 7f45b7befb80 1 -- 172.21.15.86:0/2789122263 shutdown_connections 2024-03-23T17:52:08.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.530+0000 7f45b7befb80 1 -- 172.21.15.86:0/2789122263 wait complete. 2024-03-23T17:52:08.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.530+0000 7f45b7befb80 20 client.25964 trim_cache size 0 max 16384 2024-03-23T17:52:08.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.543+0000 7f45b7befb80 1 Processor -- start 2024-03-23T17:52:08.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.543+0000 7f45b7befb80 1 -- start start 2024-03-23T17:52:08.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.543+0000 7f45b7befb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5557569395d0 0x55575695b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:08.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.543+0000 7f45b7befb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555756939ef0 0x55575694ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:08.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.543+0000 7f45b7befb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555756944df0 0x55575693c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:08.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.543+0000 7f45b7befb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x555756957050 con 0x555756939ef0 2024-03-23T17:52:08.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.543+0000 7f45b7befb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x555756935f20 con 0x555756944df0 2024-03-23T17:52:08.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.543+0000 7f45b7befb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x555756942950 con 0x5557569395d0 2024-03-23T17:52:08.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.544+0000 7f45a0c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555756939ef0 0x55575694ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:08.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.544+0000 7f45a2000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555756944df0 0x55575693c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:08.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.544+0000 7f45a0c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555756939ef0 0x55575694ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59136/0 (socket says 172.21.15.86:59136) 2024-03-23T17:52:08.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.544+0000 7f45a0c00700 1 -- 172.21.15.86:0/3477639273 learned_addr learned my addr 172.21.15.86:0/3477639273 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:08.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.544+0000 7f45a1600700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5557569395d0 0x55575695b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:08.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.544+0000 7f45a0c00700 1 -- 172.21.15.86:0/3477639273 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5557569395d0 msgr2=0x55575695b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:08.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.544+0000 7f45a0c00700 1 --2- 172.21.15.86:0/3477639273 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5557569395d0 0x55575695b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.544+0000 7f45a0c00700 1 -- 172.21.15.86:0/3477639273 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555756944df0 msgr2=0x55575693c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:08.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.544+0000 7f45a0c00700 1 --2- 172.21.15.86:0/3477639273 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555756944df0 0x55575693c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.544+0000 7f45a0c00700 1 -- 172.21.15.86:0/3477639273 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5557566b9b60 con 0x555756939ef0 2024-03-23T17:52:08.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.544+0000 7f45a2000700 1 --2- 172.21.15.86:0/3477639273 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555756944df0 0x55575693c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:52:08.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.545+0000 7f45a0c00700 1 --2- 172.21.15.86:0/3477639273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555756939ef0 0x55575694ba40 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f458800c5b0 tx=0x7f4588010bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab6c1fdae1910355 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:08.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.545+0000 7f458f000700 1 -- 172.21.15.86:0/3477639273 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45880997d0 con 0x555756939ef0 2024-03-23T17:52:08.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.545+0000 7f458f000700 1 -- 172.21.15.86:0/3477639273 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f45880ca4c0 con 0x555756939ef0 2024-03-23T17:52:08.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.545+0000 7f458f000700 1 -- 172.21.15.86:0/3477639273 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45880a1e60 con 0x555756939ef0 2024-03-23T17:52:08.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.546+0000 7f45b7befb80 1 -- 172.21.15.86:0/3477639273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555756939ef0 msgr2=0x55575694ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:08.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.546+0000 7f45b7befb80 1 --2- 172.21.15.86:0/3477639273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555756939ef0 0x55575694ba40 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f458800c5b0 tx=0x7f4588010bd0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.546+0000 7f45b7befb80 1 -- 172.21.15.86:0/3477639273 shutdown_connections 2024-03-23T17:52:08.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.546+0000 7f45b7befb80 1 --2- 172.21.15.86:0/3477639273 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555756944df0 0x55575693c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.546+0000 7f45b7befb80 1 --2- 172.21.15.86:0/3477639273 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5557569395d0 0x55575695b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.546+0000 7f45b7befb80 1 --2- 172.21.15.86:0/3477639273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555756939ef0 0x55575694ba40 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.546+0000 7f45b7befb80 1 -- 172.21.15.86:0/3477639273 >> 172.21.15.86:0/3477639273 conn(0x5557568996a0 msgr2=0x55575692e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:08.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.546+0000 7f45b7befb80 1 -- 172.21.15.86:0/3477639273 shutdown_connections 2024-03-23T17:52:08.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.546+0000 7f45b7befb80 1 -- 172.21.15.86:0/3477639273 wait complete. 2024-03-23T17:52:08.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.546+0000 7f45b7befb80 1 -- start start 2024-03-23T17:52:08.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.546+0000 7f45b7befb80 1 Processor -- start 2024-03-23T17:52:08.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.547+0000 7f45b7befb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5557569395d0 0x55575694b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:08.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.547+0000 7f45b7befb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555756939ef0 0x555756949e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:08.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.547+0000 7f45b7befb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555756944df0 0x55575694a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:08.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.547+0000 7f45b7befb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55575694aa70 con 0x555756944df0 2024-03-23T17:52:08.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.547+0000 7f45b7befb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55575694abe0 con 0x5557569395d0 2024-03-23T17:52:08.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.547+0000 7f45b7befb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557569a63d0 con 0x555756939ef0 2024-03-23T17:52:08.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.547+0000 7f45b7befb80 10 client.0 _ll_register_callbacks cb 0x555756831470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:52:08.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.548+0000 7f45a1600700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5557569395d0 0x55575694b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:08.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.548+0000 7f45a1600700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5557569395d0 0x55575694b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:41320/0 (socket says 172.21.15.86:41320) 2024-03-23T17:52:08.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.548+0000 7f45a2000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555756944df0 0x55575694a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:08.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.548+0000 7f45a1600700 1 -- 172.21.15.86:0/185650296 learned_addr learned my addr 172.21.15.86:0/185650296 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:08.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.548+0000 7f45a0c00700 1 --2- 172.21.15.86:0/185650296 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555756939ef0 0x555756949e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:08.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.548+0000 7f45a2000700 1 -- 172.21.15.86:0/185650296 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555756939ef0 msgr2=0x555756949e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:08.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.548+0000 7f45a2000700 1 --2- 172.21.15.86:0/185650296 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555756939ef0 0x555756949e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.548+0000 7f45a2000700 1 -- 172.21.15.86:0/185650296 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5557569395d0 msgr2=0x55575694b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:08.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.548+0000 7f45a2000700 1 --2- 172.21.15.86:0/185650296 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5557569395d0 0x55575694b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.548+0000 7f45a2000700 1 -- 172.21.15.86:0/185650296 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4594008aa0 con 0x555756944df0 2024-03-23T17:52:08.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.548+0000 7f45a1600700 1 --2- 172.21.15.86:0/185650296 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5557569395d0 0x55575694b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:52:08.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.549+0000 7f45a2000700 1 --2- 172.21.15.86:0/185650296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555756944df0 0x55575694a390 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f4594009c10 tx=0x7f4594004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ed7625be1994cf2 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:08.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.549+0000 7f4582a00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-23T17:52:08.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.549+0000 7f4582a00700 1 -- 172.21.15.86:0/185650296 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f459401caa0 con 0x555756944df0 2024-03-23T17:52:08.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.549+0000 7f4582a00700 1 -- 172.21.15.86:0/185650296 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f459401cc00 con 0x555756944df0 2024-03-23T17:52:08.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.549+0000 7f45b7befb80 1 -- 172.21.15.86:0/185650296 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f459001e040 con 0x555756944df0 2024-03-23T17:52:08.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.549+0000 7f4582a00700 1 -- 172.21.15.86:0/185650296 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f459401cdc0 con 0x555756944df0 2024-03-23T17:52:08.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.549+0000 7f45b7befb80 10 client.15912 fetch_fsmap learned FSMap version 30 2024-03-23T17:52:08.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.549+0000 7f45b7befb80 1 -- 172.21.15.86:0/185650296 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f459c10e550 con 0x555756944df0 2024-03-23T17:52:08.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.549+0000 7f4582a00700 1 -- 172.21.15.86:0/185650296 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f459001e040 con 0x555756944df0 2024-03-23T17:52:08.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.550+0000 7f4582a00700 1 -- 172.21.15.86:0/185650296 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f45940046b0 con 0x555756944df0 2024-03-23T17:52:08.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.550+0000 7f45b7befb80 10 client.15912 fetch_fsmap finished waiting for FSMap version 30 2024-03-23T17:52:08.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.550+0000 7f45b7befb80 10 client.15912 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-23T17:52:08.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.550+0000 7f45b7befb80 20 client.15912 populate_metadata read hostname 'smithi086' 2024-03-23T17:52:08.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.550+0000 7f45b7befb80 1 --2- 172.21.15.86:0/185650296 >> [v2:172.21.15.92:6836/3967837948,v1:172.21.15.92:6837/3967837948] conn(0x5557569b0320 0x5557569b2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:08.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.550+0000 7f45b7befb80 4 client.15912 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-23T17:52:08.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.550+0000 7f45b7befb80 1 -- 172.21.15.86:0/185650296 --> [v2:172.21.15.92:6836/3967837948,v1:172.21.15.92:6837/3967837948] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5557566dacf0 con 0x5557569b0320 2024-03-23T17:52:08.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.550+0000 7f45a1600700 1 --2- 172.21.15.86:0/185650296 >> [v2:172.21.15.92:6836/3967837948,v1:172.21.15.92:6837/3967837948] conn(0x5557569b0320 0x5557569b2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:08.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.551+0000 7f4582a00700 10 client.15912 ms_handle_connect on v2:172.21.15.92:6836/3967837948 2024-03-23T17:52:08.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.551+0000 7f45a1600700 1 --2- 172.21.15.86:0/185650296 >> [v2:172.21.15.92:6836/3967837948,v1:172.21.15.92:6837/3967837948] conn(0x5557569b0320 0x5557569b2710 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=faeec62a3870db51 server_cookie=28deab43c201b93b in_seq=0 out_seq=0 2024-03-23T17:52:08.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.551+0000 7f4582a00700 1 -- 172.21.15.86:0/185650296 <== mds.2 v2:172.21.15.92:6836/3967837948 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5557566dacf0 con 0x5557569b0320 2024-03-23T17:52:08.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.551+0000 7f4582a00700 10 client.15912 handle_command_reply: tid=0 2024-03-23T17:52:08.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.551+0000 7f45b7befb80 1 client.15912 shutdown 2024-03-23T17:52:08.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.551+0000 7f45b7befb80 1 -- 172.21.15.86:0/185650296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555756944df0 msgr2=0x55575694a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:08.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.551+0000 7f45b7befb80 1 --2- 172.21.15.86:0/185650296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555756944df0 0x55575694a390 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f4594009c10 tx=0x7f4594004b40 comp rx=0 tx=0).stop 2024-03-23T17:52:08.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.551+0000 7f45b7befb80 1 -- 172.21.15.86:0/185650296 shutdown_connections 2024-03-23T17:52:08.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.551+0000 7f45b7befb80 1 --2- 172.21.15.86:0/185650296 >> [v2:172.21.15.92:6836/3967837948,v1:172.21.15.92:6837/3967837948] conn(0x5557569b0320 0x5557569b2710 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.551+0000 7f45b7befb80 1 --2- 172.21.15.86:0/185650296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555756944df0 0x55575694a390 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.551+0000 7f45b7befb80 1 --2- 172.21.15.86:0/185650296 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5557569395d0 0x55575694b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.551+0000 7f45b7befb80 1 --2- 172.21.15.86:0/185650296 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555756939ef0 0x555756949e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.551+0000 7f4582a00700 0 client.15912 ms_handle_reset on v2:172.21.15.92:6836/3967837948 2024-03-23T17:52:08.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.551+0000 7f45b7befb80 1 -- 172.21.15.86:0/185650296 >> 172.21.15.86:0/185650296 conn(0x5557568996a0 msgr2=0x555756940200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:08.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.551+0000 7f45b7befb80 1 -- 172.21.15.86:0/185650296 shutdown_connections 2024-03-23T17:52:08.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.551+0000 7f45b7befb80 1 -- 172.21.15.86:0/185650296 wait complete. 2024-03-23T17:52:08.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.551+0000 7f45b7befb80 20 client.15912 trim_cache size 0 max 16384 2024-03-23T17:52:08.557 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-03-23T17:52:08.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.552+0000 7f45a4000700 1 -- 172.21.15.86:0/202144085 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f45840a1430 msgr2=0x7f45840a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:08.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.552+0000 7f45a4000700 1 --2- 172.21.15.86:0/202144085 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f45840a1430 0x7f45840a38f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4594010a80 tx=0x7f4594012040 comp rx=0 tx=0).stop 2024-03-23T17:52:08.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.552+0000 7f45a4000700 1 -- 172.21.15.86:0/202144085 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f459c0687f0 msgr2=0x7f459c197780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:08.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.552+0000 7f45a4000700 1 --2- 172.21.15.86:0/202144085 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f459c0687f0 0x7f459c197780 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f458800e9f0 tx=0x7f458800ed00 comp rx=0 tx=0).stop 2024-03-23T17:52:08.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.552+0000 7f45a4000700 1 -- 172.21.15.86:0/202144085 shutdown_connections 2024-03-23T17:52:08.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.552+0000 7f45a4000700 1 --2- 172.21.15.86:0/202144085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f459c105ce0 0x7f459c19bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.552+0000 7f45a4000700 1 --2- 172.21.15.86:0/202144085 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f45840a1430 0x7f45840a38f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.552+0000 7f45a4000700 1 --2- 172.21.15.86:0/202144085 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f459c067e30 0x7f459c197240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.552+0000 7f45a4000700 1 --2- 172.21.15.86:0/202144085 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f459c0687f0 0x7f459c197780 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.552+0000 7f45a4000700 1 -- 172.21.15.86:0/202144085 >> 172.21.15.86:0/202144085 conn(0x7f459c076d40 msgr2=0x7f459c0fd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:08.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.552+0000 7f45a4000700 1 -- 172.21.15.86:0/202144085 shutdown_connections 2024-03-23T17:52:08.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.552+0000 7f45a4000700 1 -- 172.21.15.86:0/202144085 wait complete. 2024-03-23T17:52:08.571 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-23T17:52:08.572 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-23T17:52:08.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.776+0000 7f41e3200700 1 -- 172.21.15.86:0/2522097373 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41e4106c10 msgr2=0x7f41e4106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:08.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.776+0000 7f41e3200700 1 --2- 172.21.15.86:0/2522097373 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41e4106c10 0x7f41e4106ff0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f41d4009990 tx=0x7f41d4009ca0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.776+0000 7f41e3200700 1 -- 172.21.15.86:0/2522097373 shutdown_connections 2024-03-23T17:52:08.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.776+0000 7f41e3200700 1 --2- 172.21.15.86:0/2522097373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41e4101510 0x7f41e410d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.776+0000 7f41e3200700 1 --2- 172.21.15.86:0/2522097373 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41e4106c10 0x7f41e4106ff0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.776+0000 7f41e3200700 1 --2- 172.21.15.86:0/2522097373 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41e4100b50 0x7f41e4100fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.776+0000 7f41e3200700 1 -- 172.21.15.86:0/2522097373 >> 172.21.15.86:0/2522097373 conn(0x7f41e40fc620 msgr2=0x7f41e40fea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:08.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.776+0000 7f41e3200700 1 -- 172.21.15.86:0/2522097373 shutdown_connections 2024-03-23T17:52:08.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.776+0000 7f41e3200700 1 -- 172.21.15.86:0/2522097373 wait complete. 2024-03-23T17:52:08.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.777+0000 7f41e3200700 1 Processor -- start 2024-03-23T17:52:08.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.777+0000 7f41e3200700 1 -- start start 2024-03-23T17:52:08.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.778+0000 7f41e3200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41e4100b50 0x7f41e4196f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:08.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.778+0000 7f41e3200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41e4101510 0x7f41e41974d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:08.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.778+0000 7f41e3200700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41e4106c10 0x7f41e419b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:08.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.778+0000 7f41e3200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e419beb0 con 0x7f41e4101510 2024-03-23T17:52:08.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.778+0000 7f41e3200700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e419c020 con 0x7f41e4100b50 2024-03-23T17:52:08.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.778+0000 7f41e3200700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e419c1e0 con 0x7f41e4106c10 2024-03-23T17:52:08.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.778+0000 7f41e1400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41e4101510 0x7f41e41974d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:08.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.778+0000 7f41e2800700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41e4106c10 0x7f41e419b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:08.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.778+0000 7f41e2800700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41e4106c10 0x7f41e419b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:51094/0 (socket says 172.21.15.86:51094) 2024-03-23T17:52:08.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.778+0000 7f41e2800700 1 -- 172.21.15.86:0/3434427582 learned_addr learned my addr 172.21.15.86:0/3434427582 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:08.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.778+0000 7f41e1400700 1 --2- 172.21.15.86:0/3434427582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41e4101510 0x7f41e41974d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59170/0 (socket says 172.21.15.86:59170) 2024-03-23T17:52:08.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.778+0000 7f41e1e00700 1 --2- 172.21.15.86:0/3434427582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41e4100b50 0x7f41e4196f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:08.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.778+0000 7f41e2800700 1 -- 172.21.15.86:0/3434427582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41e4100b50 msgr2=0x7f41e4196f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:08.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.778+0000 7f41e2800700 1 --2- 172.21.15.86:0/3434427582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41e4100b50 0x7f41e4196f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.778+0000 7f41e2800700 1 -- 172.21.15.86:0/3434427582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41e4101510 msgr2=0x7f41e41974d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:08.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.778+0000 7f41e2800700 1 --2- 172.21.15.86:0/3434427582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41e4101510 0x7f41e41974d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.778+0000 7f41e2800700 1 -- 172.21.15.86:0/3434427582 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41d40095f0 con 0x7f41e4106c10 2024-03-23T17:52:08.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.778+0000 7f41e1400700 1 --2- 172.21.15.86:0/3434427582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41e4101510 0x7f41e41974d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:52:08.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.779+0000 7f41e2800700 1 --2- 172.21.15.86:0/3434427582 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41e4106c10 0x7f41e419b870 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f41dc00ef30 tx=0x7f41dc00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:08.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.779+0000 7f41daa00700 1 -- 172.21.15.86:0/3434427582 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41dc018070 con 0x7f41e4106c10 2024-03-23T17:52:08.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.779+0000 7f41daa00700 1 -- 172.21.15.86:0/3434427582 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f41dc00f040 con 0x7f41e4106c10 2024-03-23T17:52:08.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.779+0000 7f41e3200700 1 -- 172.21.15.86:0/3434427582 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41e419c460 con 0x7f41e4106c10 2024-03-23T17:52:08.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.779+0000 7f41daa00700 1 -- 172.21.15.86:0/3434427582 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41dc00cd00 con 0x7f41e4106c10 2024-03-23T17:52:08.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.779+0000 7f41e3200700 1 -- 172.21.15.86:0/3434427582 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41e41a3d00 con 0x7f41e4106c10 2024-03-23T17:52:08.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.782+0000 7f41daa00700 1 -- 172.21.15.86:0/3434427582 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) v1 ==== 136098+0+0 (secure 0 0 0) 0x7f41dc003a00 con 0x7f41e4106c10 2024-03-23T17:52:08.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.784+0000 7f41fc7a5b80 1 Processor -- start 2024-03-23T17:52:08.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.784+0000 7f41fc7a5b80 1 -- start start 2024-03-23T17:52:08.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.784+0000 7f41daa00700 1 --2- 172.21.15.86:0/3434427582 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f41c80a1430 0x7f41c80a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:08.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.784+0000 7f41daa00700 1 -- 172.21.15.86:0/3434427582 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f41dc0cca60 con 0x7f41e4106c10 2024-03-23T17:52:08.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.784+0000 7f41fc7a5b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f46a667a0 0x563f46a66b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:08.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.784+0000 7f41e1400700 1 --2- 172.21.15.86:0/3434427582 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f41c80a1430 0x7f41c80a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:08.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.784+0000 7f41fc7a5b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563f46a670c0 0x563f46a71a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:08.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.784+0000 7f41fc7a5b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563f46a71fc0 0x563f46a743b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:08.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.784+0000 7f41fc7a5b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f469bf990 con 0x563f46a667a0 2024-03-23T17:52:08.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.785+0000 7f41fc7a5b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f469bd350 con 0x563f46a71fc0 2024-03-23T17:52:08.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.785+0000 7f41fc7a5b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f469b8c40 con 0x563f46a670c0 2024-03-23T17:52:08.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.785+0000 7f41e1400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563f46a670c0 0x563f46a71a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:08.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.785+0000 7f41e1400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563f46a670c0 0x563f46a71a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:51098/0 (socket says 172.21.15.86:51098) 2024-03-23T17:52:08.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.785+0000 7f41e2800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563f46a71fc0 0x563f46a743b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:08.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.785+0000 7f41e1400700 1 -- 172.21.15.86:0/3762561362 learned_addr learned my addr 172.21.15.86:0/3762561362 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:08.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.785+0000 7f41e1e00700 1 --2- 172.21.15.86:0/3762561362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f46a667a0 0x563f46a66b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:08.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.785+0000 7f41e1400700 1 -- 172.21.15.86:0/3762561362 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563f46a71fc0 msgr2=0x563f46a743b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:08.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.785+0000 7f41e1400700 1 --2- 172.21.15.86:0/3762561362 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563f46a71fc0 0x563f46a743b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.785+0000 7f41e1400700 1 -- 172.21.15.86:0/3762561362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f46a667a0 msgr2=0x563f46a66b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:08.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.785+0000 7f41e1400700 1 --2- 172.21.15.86:0/3762561362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f46a667a0 0x563f46a66b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.785+0000 7f41e1400700 1 -- 172.21.15.86:0/3762561362 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f4683a3b0 con 0x563f46a670c0 2024-03-23T17:52:08.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.785+0000 7f41e1400700 1 --2- 172.21.15.86:0/3434427582 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f41c80a1430 0x7f41c80a38f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f41d0013e50 tx=0x7f41d00134e0 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:08.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.785+0000 7f41e1400700 1 --2- 172.21.15.86:0/3762561362 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563f46a670c0 0x563f46a71a80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f41d0009c90 tx=0x7f41d001d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e3fd9c7eae6ac7e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:08.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.786+0000 7f41cf000700 1 -- 172.21.15.86:0/3762561362 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d000fea0 con 0x563f46a670c0 2024-03-23T17:52:08.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.786+0000 7f41cf000700 1 -- 172.21.15.86:0/3762561362 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f41d0013170 con 0x563f46a670c0 2024-03-23T17:52:08.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.786+0000 7f41cf000700 1 -- 172.21.15.86:0/3762561362 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d0010670 con 0x563f46a670c0 2024-03-23T17:52:08.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.786+0000 7f41fc7a5b80 1 -- 172.21.15.86:0/3762561362 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563f46a670c0 msgr2=0x563f46a71a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:08.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.786+0000 7f41fc7a5b80 1 --2- 172.21.15.86:0/3762561362 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563f46a670c0 0x563f46a71a80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f41d0009c90 tx=0x7f41d001d000 comp rx=0 tx=0).stop 2024-03-23T17:52:08.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.787+0000 7f41fc7a5b80 1 -- 172.21.15.86:0/3762561362 shutdown_connections 2024-03-23T17:52:08.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.787+0000 7f41fc7a5b80 1 --2- 172.21.15.86:0/3762561362 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563f46a71fc0 0x563f46a743b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.787+0000 7f41fc7a5b80 1 --2- 172.21.15.86:0/3762561362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f46a667a0 0x563f46a66b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.787+0000 7f41fc7a5b80 1 --2- 172.21.15.86:0/3762561362 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563f46a670c0 0x563f46a71a80 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.787+0000 7f41fc7a5b80 1 -- 172.21.15.86:0/3762561362 >> 172.21.15.86:0/3762561362 conn(0x563f469c66a0 msgr2=0x563f46a62c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:08.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.787+0000 7f41fc7a5b80 1 -- 172.21.15.86:0/3762561362 shutdown_connections 2024-03-23T17:52:08.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.787+0000 7f41fc7a5b80 1 -- 172.21.15.86:0/3762561362 wait complete. 2024-03-23T17:52:08.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.787+0000 7f41fc7a5b80 1 -- start start 2024-03-23T17:52:08.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.787+0000 7f41fc7a5b80 1 Processor -- start 2024-03-23T17:52:08.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.788+0000 7f41fc7a5b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563f46a667a0 0x563f46a719e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:08.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.788+0000 7f41fc7a5b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f46a670c0 0x563f46a6ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:08.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.788+0000 7f41fc7a5b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563f46a71fc0 0x563f46a6f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:08.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.788+0000 7f41fc7a5b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f46954d80 con 0x563f46a670c0 2024-03-23T17:52:08.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.788+0000 7f41fc7a5b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f46954a20 con 0x563f46a667a0 2024-03-23T17:52:08.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.788+0000 7f41fc7a5b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f4683c6a0 con 0x563f46a71fc0 2024-03-23T17:52:08.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.788+0000 7f41fc7a5b80 10 client.0 _ll_register_callbacks cb 0x563f4695e470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:52:08.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.788+0000 7f41e1e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563f46a667a0 0x563f46a719e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:08.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.788+0000 7f41e1400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f46a670c0 0x563f46a6ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:08.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.788+0000 7f41e1e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563f46a667a0 0x563f46a719e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:41356/0 (socket says 172.21.15.86:41356) 2024-03-23T17:52:08.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.788+0000 7f41e1e00700 1 -- 172.21.15.86:0/1934742126 learned_addr learned my addr 172.21.15.86:0/1934742126 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:08.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.788+0000 7f41e2800700 1 --2- 172.21.15.86:0/1934742126 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563f46a71fc0 0x563f46a6f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:08.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.789+0000 7f41e1e00700 1 -- 172.21.15.86:0/1934742126 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563f46a71fc0 msgr2=0x563f46a6f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:08.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.789+0000 7f41e1e00700 1 --2- 172.21.15.86:0/1934742126 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563f46a71fc0 0x563f46a6f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.789+0000 7f41e1e00700 1 -- 172.21.15.86:0/1934742126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f46a670c0 msgr2=0x563f46a6ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:08.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.789+0000 7f41e1e00700 1 --2- 172.21.15.86:0/1934742126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f46a670c0 0x563f46a6ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.789+0000 7f41e1e00700 1 -- 172.21.15.86:0/1934742126 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41d001b040 con 0x563f46a667a0 2024-03-23T17:52:08.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.789+0000 7f41e1400700 1 --2- 172.21.15.86:0/1934742126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f46a670c0 0x563f46a6ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:52:08.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.789+0000 7f41e1e00700 1 --2- 172.21.15.86:0/1934742126 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563f46a667a0 0x563f46a719e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f41d4005800 tx=0x7f41d400b610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ab0e8f41e5a8c56 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:08.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.789+0000 7f41c3400700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T17:52:08.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.789+0000 7f41c3400700 1 -- 172.21.15.86:0/1934742126 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d401d070 con 0x563f46a667a0 2024-03-23T17:52:08.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.789+0000 7f41fc7a5b80 1 -- 172.21.15.86:0/1934742126 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563f46979380 con 0x563f46a667a0 2024-03-23T17:52:08.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.789+0000 7f41c3400700 1 -- 172.21.15.86:0/1934742126 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f41d4004500 con 0x563f46a667a0 2024-03-23T17:52:08.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.790+0000 7f41c3400700 1 -- 172.21.15.86:0/1934742126 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41d4004700 con 0x563f46a667a0 2024-03-23T17:52:08.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.790+0000 7f41fc7a5b80 10 client.25982 fetch_fsmap learned FSMap version 30 2024-03-23T17:52:08.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.790+0000 7f41fc7a5b80 1 -- 172.21.15.86:0/1934742126 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x563f467e6b60 con 0x563f46a667a0 2024-03-23T17:52:08.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.790+0000 7f41c3400700 1 -- 172.21.15.86:0/1934742126 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563f46979380 con 0x563f46a667a0 2024-03-23T17:52:08.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.790+0000 7f41c3400700 1 -- 172.21.15.86:0/1934742126 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 30) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f41d400f870 con 0x563f46a667a0 2024-03-23T17:52:08.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.790+0000 7f41fc7a5b80 10 client.25982 fetch_fsmap finished waiting for FSMap version 30 2024-03-23T17:52:08.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.790+0000 7f41fc7a5b80 10 client.25982 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-23T17:52:08.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.790+0000 7f41fc7a5b80 20 client.25982 populate_metadata read hostname 'smithi086' 2024-03-23T17:52:08.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.790+0000 7f41fc7a5b80 1 --2- 172.21.15.86:0/1934742126 >> [v2:172.21.15.152:6838/1100723992,v1:172.21.15.152:6839/1100723992] conn(0x563f46a85f90 0x563f46a883b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:08.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.790+0000 7f41fc7a5b80 4 client.25982 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:52:08.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.790+0000 7f41fc7a5b80 1 -- 172.21.15.86:0/1934742126 --> [v2:172.21.15.152:6838/1100723992,v1:172.21.15.152:6839/1100723992] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563f469b1700 con 0x563f46a85f90 2024-03-23T17:52:08.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.791+0000 7f41e1400700 1 --2- 172.21.15.86:0/1934742126 >> [v2:172.21.15.152:6838/1100723992,v1:172.21.15.152:6839/1100723992] conn(0x563f46a85f90 0x563f46a883b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:08.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.791+0000 7f41e1400700 1 --2- 172.21.15.86:0/1934742126 >> [v2:172.21.15.152:6838/1100723992,v1:172.21.15.152:6839/1100723992] conn(0x563f46a85f90 0x563f46a883b0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=10a977f98961ad5d server_cookie=e75e6de5338bb2a8 in_seq=0 out_seq=0 2024-03-23T17:52:08.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.791+0000 7f41c3400700 10 client.25982 ms_handle_connect on v2:172.21.15.152:6838/1100723992 2024-03-23T17:52:08.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.792+0000 7f41c3400700 1 -- 172.21.15.86:0/1934742126 <== mds.1 v2:172.21.15.152:6838/1100723992 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x563f469b1700 con 0x563f46a85f90 2024-03-23T17:52:08.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.792+0000 7f41c3400700 10 client.25982 handle_command_reply: tid=0 2024-03-23T17:52:08.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.792+0000 7f41fc7a5b80 1 client.25982 shutdown 2024-03-23T17:52:08.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.792+0000 7f41fc7a5b80 1 -- 172.21.15.86:0/1934742126 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563f46a667a0 msgr2=0x563f46a719e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:08.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.792+0000 7f41fc7a5b80 1 --2- 172.21.15.86:0/1934742126 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563f46a667a0 0x563f46a719e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f41d4005800 tx=0x7f41d400b610 comp rx=0 tx=0).stop 2024-03-23T17:52:08.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.792+0000 7f41fc7a5b80 1 -- 172.21.15.86:0/1934742126 shutdown_connections 2024-03-23T17:52:08.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.792+0000 7f41fc7a5b80 1 --2- 172.21.15.86:0/1934742126 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563f46a71fc0 0x563f46a6f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.792+0000 7f41fc7a5b80 1 --2- 172.21.15.86:0/1934742126 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563f46a667a0 0x563f46a719e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.792+0000 7f41fc7a5b80 1 --2- 172.21.15.86:0/1934742126 >> [v2:172.21.15.152:6838/1100723992,v1:172.21.15.152:6839/1100723992] conn(0x563f46a85f90 0x563f46a883b0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.792+0000 7f41fc7a5b80 1 --2- 172.21.15.86:0/1934742126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f46a670c0 0x563f46a6ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.792+0000 7f41c3400700 0 client.25982 ms_handle_reset on v2:172.21.15.152:6838/1100723992 2024-03-23T17:52:08.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.792+0000 7f41fc7a5b80 1 -- 172.21.15.86:0/1934742126 >> 172.21.15.86:0/1934742126 conn(0x563f469c66a0 msgr2=0x563f46a68700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:08.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.792+0000 7f41fc7a5b80 1 -- 172.21.15.86:0/1934742126 shutdown_connections 2024-03-23T17:52:08.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.792+0000 7f41fc7a5b80 1 -- 172.21.15.86:0/1934742126 wait complete. 2024-03-23T17:52:08.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.792+0000 7f41fc7a5b80 20 client.25982 trim_cache size 0 max 16384 2024-03-23T17:52:08.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.807+0000 7f41fc7a5b80 1 Processor -- start 2024-03-23T17:52:08.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.807+0000 7f41fc7a5b80 1 -- start start 2024-03-23T17:52:08.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.807+0000 7f41fc7a5b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563f46a667a0 0x563f46a88380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:08.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.807+0000 7f41fc7a5b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f46a670c0 0x563f46a8cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:08.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.807+0000 7f41fc7a5b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563f46a71fc0 0x563f46a7cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:08.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.807+0000 7f41fc7a5b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f46a700f0 con 0x563f46a670c0 2024-03-23T17:52:08.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.807+0000 7f41fc7a5b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f46a6f890 con 0x563f46a71fc0 2024-03-23T17:52:08.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.807+0000 7f41fc7a5b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f46a743b0 con 0x563f46a667a0 2024-03-23T17:52:08.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.807+0000 7f41e1400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f46a670c0 0x563f46a8cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:08.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.808+0000 7f41e1400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f46a670c0 0x563f46a8cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59194/0 (socket says 172.21.15.86:59194) 2024-03-23T17:52:08.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.808+0000 7f41e1400700 1 -- 172.21.15.86:0/4132790086 learned_addr learned my addr 172.21.15.86:0/4132790086 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:08.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.808+0000 7f41e2800700 1 --2- 172.21.15.86:0/4132790086 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563f46a71fc0 0x563f46a7cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:08.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.808+0000 7f41e1400700 1 -- 172.21.15.86:0/4132790086 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563f46a667a0 msgr2=0x563f46a88380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:08.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.808+0000 7f41e1400700 1 --2- 172.21.15.86:0/4132790086 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563f46a667a0 0x563f46a88380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.808+0000 7f41e1400700 1 -- 172.21.15.86:0/4132790086 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563f46a71fc0 msgr2=0x563f46a7cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:08.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.808+0000 7f41e1400700 1 --2- 172.21.15.86:0/4132790086 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563f46a71fc0 0x563f46a7cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.808+0000 7f41e1400700 1 -- 172.21.15.86:0/4132790086 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f4697eea0 con 0x563f46a670c0 2024-03-23T17:52:08.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.808+0000 7f41e1400700 1 --2- 172.21.15.86:0/4132790086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f46a670c0 0x563f46a8cd10 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f41d0016040 tx=0x7f41d0013b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1295fa881ecba3 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:08.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.809+0000 7f41cfa00700 1 -- 172.21.15.86:0/4132790086 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d0013830 con 0x563f46a670c0 2024-03-23T17:52:08.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.809+0000 7f41cfa00700 1 -- 172.21.15.86:0/4132790086 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f41d0009cc0 con 0x563f46a670c0 2024-03-23T17:52:08.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.809+0000 7f41cfa00700 1 -- 172.21.15.86:0/4132790086 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d0004280 con 0x563f46a670c0 2024-03-23T17:52:08.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.809+0000 7f41fc7a5b80 1 -- 172.21.15.86:0/4132790086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f46a670c0 msgr2=0x563f46a8cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:08.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.809+0000 7f41fc7a5b80 1 --2- 172.21.15.86:0/4132790086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f46a670c0 0x563f46a8cd10 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f41d0016040 tx=0x7f41d0013b60 comp rx=0 tx=0).stop 2024-03-23T17:52:08.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.809+0000 7f41fc7a5b80 1 -- 172.21.15.86:0/4132790086 shutdown_connections 2024-03-23T17:52:08.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.809+0000 7f41fc7a5b80 1 --2- 172.21.15.86:0/4132790086 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563f46a71fc0 0x563f46a7cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.809+0000 7f41fc7a5b80 1 --2- 172.21.15.86:0/4132790086 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563f46a667a0 0x563f46a88380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.809+0000 7f41fc7a5b80 1 --2- 172.21.15.86:0/4132790086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f46a670c0 0x563f46a8cd10 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.809+0000 7f41fc7a5b80 1 -- 172.21.15.86:0/4132790086 >> 172.21.15.86:0/4132790086 conn(0x563f469c66a0 msgr2=0x563f46a5b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:08.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.810+0000 7f41fc7a5b80 1 -- 172.21.15.86:0/4132790086 shutdown_connections 2024-03-23T17:52:08.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.810+0000 7f41fc7a5b80 1 -- 172.21.15.86:0/4132790086 wait complete. 2024-03-23T17:52:08.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.810+0000 7f41fc7a5b80 1 -- start start 2024-03-23T17:52:08.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.810+0000 7f41fc7a5b80 1 Processor -- start 2024-03-23T17:52:08.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.811+0000 7f41fc7a5b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563f46a667a0 0x563f46a7c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:08.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.811+0000 7f41fc7a5b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f46a670c0 0x563f46a7cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:08.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.811+0000 7f41fc7a5b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563f46a71fc0 0x563f46a7aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:08.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.811+0000 7f41fc7a5b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f46a7b470 con 0x563f46a670c0 2024-03-23T17:52:08.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.811+0000 7f41fc7a5b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f46a7b5e0 con 0x563f46a667a0 2024-03-23T17:52:08.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.811+0000 7f41fc7a5b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f46a7b7a0 con 0x563f46a71fc0 2024-03-23T17:52:08.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.811+0000 7f41fc7a5b80 10 client.0 _ll_register_callbacks cb 0x563f4695e470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:52:08.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.811+0000 7f41e1e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563f46a667a0 0x563f46a7c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:08.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.811+0000 7f41e1400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f46a670c0 0x563f46a7cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:08.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.811+0000 7f41e1e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563f46a667a0 0x563f46a7c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:41372/0 (socket says 172.21.15.86:41372) 2024-03-23T17:52:08.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.811+0000 7f41e1e00700 1 -- 172.21.15.86:0/1018009881 learned_addr learned my addr 172.21.15.86:0/1018009881 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:08.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.811+0000 7f41e2800700 1 --2- 172.21.15.86:0/1018009881 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563f46a71fc0 0x563f46a7aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:08.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.811+0000 7f41e1e00700 1 -- 172.21.15.86:0/1018009881 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563f46a71fc0 msgr2=0x563f46a7aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:08.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.811+0000 7f41e1e00700 1 --2- 172.21.15.86:0/1018009881 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563f46a71fc0 0x563f46a7aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.811+0000 7f41e1e00700 1 -- 172.21.15.86:0/1018009881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f46a670c0 msgr2=0x563f46a7cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:08.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.811+0000 7f41e1e00700 1 --2- 172.21.15.86:0/1018009881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f46a670c0 0x563f46a7cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.811+0000 7f41e1e00700 1 -- 172.21.15.86:0/1018009881 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563f4683a3b0 con 0x563f46a667a0 2024-03-23T17:52:08.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.811+0000 7f41e2800700 1 --2- 172.21.15.86:0/1018009881 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563f46a71fc0 0x563f46a7aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:52:08.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.812+0000 7f41e1e00700 1 --2- 172.21.15.86:0/1018009881 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563f46a667a0 0x563f46a7c810 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f41d40055a0 tx=0x7f41d4005f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26c3e4a9f2a3c234 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:08.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.812+0000 7f41c3400700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T17:52:08.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.812+0000 7f41c3400700 1 -- 172.21.15.86:0/1018009881 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d403c070 con 0x563f46a667a0 2024-03-23T17:52:08.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.812+0000 7f41c3400700 1 -- 172.21.15.86:0/1018009881 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f41d4000e40 con 0x563f46a667a0 2024-03-23T17:52:08.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.812+0000 7f41fc7a5b80 1 -- 172.21.15.86:0/1018009881 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f41d0013170 con 0x563f46a667a0 2024-03-23T17:52:08.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.812+0000 7f41c3400700 1 -- 172.21.15.86:0/1018009881 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41d401d030 con 0x563f46a667a0 2024-03-23T17:52:08.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.812+0000 7f41fc7a5b80 10 client.25988 fetch_fsmap learned FSMap version 30 2024-03-23T17:52:08.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.812+0000 7f41fc7a5b80 1 -- 172.21.15.86:0/1018009881 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x563f4683ecd0 con 0x563f46a667a0 2024-03-23T17:52:08.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.812+0000 7f41c3400700 1 -- 172.21.15.86:0/1018009881 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f41d0013170 con 0x563f46a667a0 2024-03-23T17:52:08.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.813+0000 7f41c3400700 1 -- 172.21.15.86:0/1018009881 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 30) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f41d403f030 con 0x563f46a667a0 2024-03-23T17:52:08.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.813+0000 7f41fc7a5b80 10 client.25988 fetch_fsmap finished waiting for FSMap version 30 2024-03-23T17:52:08.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.813+0000 7f41fc7a5b80 10 client.25988 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-23T17:52:08.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.813+0000 7f41fc7a5b80 20 client.25988 populate_metadata read hostname 'smithi086' 2024-03-23T17:52:08.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.813+0000 7f41fc7a5b80 1 --2- 172.21.15.86:0/1018009881 >> [v2:172.21.15.152:6838/1100723992,v1:172.21.15.152:6839/1100723992] conn(0x563f46ad8a10 0x563f46adae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:08.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.813+0000 7f41fc7a5b80 4 client.25988 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-23T17:52:08.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.813+0000 7f41fc7a5b80 1 -- 172.21.15.86:0/1018009881 --> [v2:172.21.15.152:6838/1100723992,v1:172.21.15.152:6839/1100723992] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563f46807cf0 con 0x563f46ad8a10 2024-03-23T17:52:08.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.813+0000 7f41e1400700 1 --2- 172.21.15.86:0/1018009881 >> [v2:172.21.15.152:6838/1100723992,v1:172.21.15.152:6839/1100723992] conn(0x563f46ad8a10 0x563f46adae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:08.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.814+0000 7f41c3400700 10 client.25988 ms_handle_connect on v2:172.21.15.152:6838/1100723992 2024-03-23T17:52:08.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.814+0000 7f41e1400700 1 --2- 172.21.15.86:0/1018009881 >> [v2:172.21.15.152:6838/1100723992,v1:172.21.15.152:6839/1100723992] conn(0x563f46ad8a10 0x563f46adae30 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6300d0e789248737 server_cookie=82590da3bfbaed02 in_seq=0 out_seq=0 2024-03-23T17:52:08.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.814+0000 7f41c3400700 1 -- 172.21.15.86:0/1018009881 <== mds.1 v2:172.21.15.152:6838/1100723992 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563f46807cf0 con 0x563f46ad8a10 2024-03-23T17:52:08.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.814+0000 7f41c3400700 10 client.25988 handle_command_reply: tid=0 2024-03-23T17:52:08.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.814+0000 7f41fc7a5b80 1 client.25988 shutdown 2024-03-23T17:52:08.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.814+0000 7f41fc7a5b80 1 -- 172.21.15.86:0/1018009881 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563f46a667a0 msgr2=0x563f46a7c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:08.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.814+0000 7f41fc7a5b80 1 --2- 172.21.15.86:0/1018009881 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563f46a667a0 0x563f46a7c810 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f41d40055a0 tx=0x7f41d4005f00 comp rx=0 tx=0).stop 2024-03-23T17:52:08.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.814+0000 7f41fc7a5b80 1 -- 172.21.15.86:0/1018009881 shutdown_connections 2024-03-23T17:52:08.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.814+0000 7f41fc7a5b80 1 --2- 172.21.15.86:0/1018009881 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563f46a71fc0 0x563f46a7aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.814+0000 7f41fc7a5b80 1 --2- 172.21.15.86:0/1018009881 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563f46a667a0 0x563f46a7c810 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.814+0000 7f41fc7a5b80 1 --2- 172.21.15.86:0/1018009881 >> [v2:172.21.15.152:6838/1100723992,v1:172.21.15.152:6839/1100723992] conn(0x563f46ad8a10 0x563f46adae30 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.814+0000 7f41fc7a5b80 1 --2- 172.21.15.86:0/1018009881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f46a670c0 0x563f46a7cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.814+0000 7f41c3400700 0 client.25988 ms_handle_reset on v2:172.21.15.152:6838/1100723992 2024-03-23T17:52:08.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.814+0000 7f41fc7a5b80 1 -- 172.21.15.86:0/1018009881 >> 172.21.15.86:0/1018009881 conn(0x563f469c66a0 msgr2=0x563f46a5b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:08.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.814+0000 7f41fc7a5b80 1 -- 172.21.15.86:0/1018009881 shutdown_connections 2024-03-23T17:52:08.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.814+0000 7f41fc7a5b80 1 -- 172.21.15.86:0/1018009881 wait complete. 2024-03-23T17:52:08.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.815+0000 7f41fc7a5b80 20 client.25988 trim_cache size 0 max 16384 2024-03-23T17:52:08.820 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-03-23T17:52:08.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.815+0000 7f41e3200700 1 -- 172.21.15.86:0/3434427582 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f41c80a1430 msgr2=0x7f41c80a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:08.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.815+0000 7f41e3200700 1 --2- 172.21.15.86:0/3434427582 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f41c80a1430 0x7f41c80a38f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f41d0013e50 tx=0x7f41d00134e0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.815+0000 7f41e3200700 1 -- 172.21.15.86:0/3434427582 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41e4106c10 msgr2=0x7f41e419b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:08.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.815+0000 7f41e3200700 1 --2- 172.21.15.86:0/3434427582 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41e4106c10 0x7f41e419b870 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f41dc00ef30 tx=0x7f41dc00c3a0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.815+0000 7f41e3200700 1 -- 172.21.15.86:0/3434427582 shutdown_connections 2024-03-23T17:52:08.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.815+0000 7f41e3200700 1 --2- 172.21.15.86:0/3434427582 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f41c80a1430 0x7f41c80a38f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.815+0000 7f41e3200700 1 --2- 172.21.15.86:0/3434427582 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41e4106c10 0x7f41e419b870 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.815+0000 7f41e3200700 1 --2- 172.21.15.86:0/3434427582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41e4100b50 0x7f41e4196f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.815+0000 7f41e3200700 1 --2- 172.21.15.86:0/3434427582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41e4101510 0x7f41e41974d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:08.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.815+0000 7f41e3200700 1 -- 172.21.15.86:0/3434427582 >> 172.21.15.86:0/3434427582 conn(0x7f41e40fc620 msgr2=0x7f41e40fdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:08.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.815+0000 7f41e3200700 1 -- 172.21.15.86:0/3434427582 shutdown_connections 2024-03-23T17:52:08.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:08.815+0000 7f41e3200700 1 -- 172.21.15.86:0/3434427582 wait complete. 2024-03-23T17:52:08.838 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-23T17:52:08.838 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-23T17:52:08.880 INFO:tasks.workunit.client.0.smithi086.stdout: 1 29009 5.27 MB/sec warmup 33 sec latency 885.376 ms 2024-03-23T17:52:09.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.038+0000 7f78e4c00700 1 -- 172.21.15.86:0/1998135403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78e0105ce0 msgr2=0x7f78e01060c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:09.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.038+0000 7f78e4c00700 1 --2- 172.21.15.86:0/1998135403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78e0105ce0 0x7f78e01060c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f78d0009990 tx=0x7f78d0009ca0 comp rx=0 tx=0).stop 2024-03-23T17:52:09.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.039+0000 7f78e4c00700 1 -- 172.21.15.86:0/1998135403 shutdown_connections 2024-03-23T17:52:09.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.039+0000 7f78e4c00700 1 --2- 172.21.15.86:0/1998135403 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78e00687f0 0x7f78e010db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:09.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.039+0000 7f78e4c00700 1 --2- 172.21.15.86:0/1998135403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78e0105ce0 0x7f78e01060c0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:09.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.039+0000 7f78e4c00700 1 --2- 172.21.15.86:0/1998135403 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78e0067e30 0x7f78e00682b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:09.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.039+0000 7f78e4c00700 1 -- 172.21.15.86:0/1998135403 >> 172.21.15.86:0/1998135403 conn(0x7f78e0076d40 msgr2=0x7f78e0077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:09.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.039+0000 7f78e4c00700 1 -- 172.21.15.86:0/1998135403 shutdown_connections 2024-03-23T17:52:09.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.039+0000 7f78e4c00700 1 -- 172.21.15.86:0/1998135403 wait complete. 2024-03-23T17:52:09.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.040+0000 7f78e4c00700 1 Processor -- start 2024-03-23T17:52:09.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.040+0000 7f78e4c00700 1 -- start start 2024-03-23T17:52:09.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.040+0000 7f78e4c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78e0067e30 0x7f78e0197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:09.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.040+0000 7f78e4c00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78e00687f0 0x7f78e0197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:09.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.040+0000 7f78e4c00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78e0105ce0 0x7f78e019bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:09.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.040+0000 7f78e4c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78e019c120 con 0x7f78e0067e30 2024-03-23T17:52:09.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.040+0000 7f78e4c00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78e019c290 con 0x7f78e0105ce0 2024-03-23T17:52:09.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.040+0000 7f78e4c00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78e019c450 con 0x7f78e00687f0 2024-03-23T17:52:09.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.040+0000 7f78dde00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78e0067e30 0x7f78e0197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:09.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.040+0000 7f78dde00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78e0067e30 0x7f78e0197200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59220/0 (socket says 172.21.15.86:59220) 2024-03-23T17:52:09.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.040+0000 7f78dde00700 1 -- 172.21.15.86:0/3668474187 learned_addr learned my addr 172.21.15.86:0/3668474187 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:09.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.040+0000 7f78de800700 1 --2- 172.21.15.86:0/3668474187 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78e0105ce0 0x7f78e019bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:09.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.040+0000 7f78dd400700 1 --2- 172.21.15.86:0/3668474187 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78e00687f0 0x7f78e0197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:09.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.041+0000 7f78dde00700 1 -- 172.21.15.86:0/3668474187 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78e00687f0 msgr2=0x7f78e0197740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:09.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.041+0000 7f78dde00700 1 --2- 172.21.15.86:0/3668474187 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78e00687f0 0x7f78e0197740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:09.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.041+0000 7f78dde00700 1 -- 172.21.15.86:0/3668474187 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78e0105ce0 msgr2=0x7f78e019bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:09.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.041+0000 7f78dde00700 1 --2- 172.21.15.86:0/3668474187 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78e0105ce0 0x7f78e019bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:09.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.041+0000 7f78dde00700 1 -- 172.21.15.86:0/3668474187 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78d00095f0 con 0x7f78e0067e30 2024-03-23T17:52:09.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.041+0000 7f78de800700 1 --2- 172.21.15.86:0/3668474187 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78e0105ce0 0x7f78e019bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:52:09.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.041+0000 7f78dde00700 1 --2- 172.21.15.86:0/3668474187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78e0067e30 0x7f78e0197200 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f78d0005910 tx=0x7f78d0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:09.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.041+0000 7f78d6a00700 1 -- 172.21.15.86:0/3668474187 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d001d070 con 0x7f78e0067e30 2024-03-23T17:52:09.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.041+0000 7f78d6a00700 1 -- 172.21.15.86:0/3668474187 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f78d0022470 con 0x7f78e0067e30 2024-03-23T17:52:09.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.041+0000 7f78e4c00700 1 -- 172.21.15.86:0/3668474187 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78e019c6d0 con 0x7f78e0067e30 2024-03-23T17:52:09.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.041+0000 7f78d6a00700 1 -- 172.21.15.86:0/3668474187 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d0017650 con 0x7f78e0067e30 2024-03-23T17:52:09.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.041+0000 7f78e4c00700 1 -- 172.21.15.86:0/3668474187 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78e01a3f70 con 0x7f78e0067e30 2024-03-23T17:52:09.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.046+0000 7f78d6a00700 1 -- 172.21.15.86:0/3668474187 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) v1 ==== 136098+0+0 (secure 0 0 0) 0x7f78d00225e0 con 0x7f78e0067e30 2024-03-23T17:52:09.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.047+0000 7f78f879cb80 1 Processor -- start 2024-03-23T17:52:09.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.047+0000 7f78d6a00700 1 --2- 172.21.15.86:0/3668474187 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f78c40a13b0 0x7f78c40a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:09.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.047+0000 7f78f879cb80 1 -- start start 2024-03-23T17:52:09.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.047+0000 7f78d6a00700 1 -- 172.21.15.86:0/3668474187 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f78d00ce1b0 con 0x7f78e0067e30 2024-03-23T17:52:09.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.047+0000 7f78de800700 1 --2- 172.21.15.86:0/3668474187 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f78c40a13b0 0x7f78c40a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:09.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.047+0000 7f78f879cb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c6224781c0 0x55c6224765d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:09.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.047+0000 7f78f879cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c622476b10 0x55c622476f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:09.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.047+0000 7f78f879cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c62248a490 0x55c62248c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:09.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.047+0000 7f78f879cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6223cf990 con 0x55c622476b10 2024-03-23T17:52:09.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.047+0000 7f78f879cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6223cd350 con 0x55c62248a490 2024-03-23T17:52:09.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.047+0000 7f78f879cb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6223c8c40 con 0x55c6224781c0 2024-03-23T17:52:09.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.048+0000 7f78dde00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c6224781c0 0x55c6224765d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:09.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.048+0000 7f78dd400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c622476b10 0x55c622476f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:09.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.048+0000 7f78dde00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c6224781c0 0x55c6224765d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:51162/0 (socket says 172.21.15.86:51162) 2024-03-23T17:52:09.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.048+0000 7f78dde00700 1 -- 172.21.15.86:0/3390063789 learned_addr learned my addr 172.21.15.86:0/3390063789 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:09.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.048+0000 7f78de800700 1 --2- 172.21.15.86:0/3390063789 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c62248a490 0x55c62248c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:09.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.048+0000 7f78dde00700 1 -- 172.21.15.86:0/3390063789 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c62248a490 msgr2=0x55c62248c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:09.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.048+0000 7f78dde00700 1 --2- 172.21.15.86:0/3390063789 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c62248a490 0x55c62248c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:09.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.048+0000 7f78dde00700 1 -- 172.21.15.86:0/3390063789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c622476b10 msgr2=0x55c622476f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:09.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.048+0000 7f78dde00700 1 --2- 172.21.15.86:0/3390063789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c622476b10 0x55c622476f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:09.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.048+0000 7f78de800700 1 --2- 172.21.15.86:0/3668474187 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f78c40a13b0 0x7f78c40a3870 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f78d8003e80 tx=0x7f78d8009920 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:09.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.048+0000 7f78dde00700 1 -- 172.21.15.86:0/3390063789 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c62224a3b0 con 0x55c6224781c0 2024-03-23T17:52:09.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.048+0000 7f78dd400700 1 --2- 172.21.15.86:0/3390063789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c622476b10 0x55c622476f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:52:09.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.048+0000 7f78dde00700 1 --2- 172.21.15.86:0/3390063789 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c6224781c0 0x55c6224765d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f78d000f740 tx=0x7f78d0022c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3588c90ff86e08d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:09.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.049+0000 7f78cb000700 1 -- 172.21.15.86:0/3390063789 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d009fea0 con 0x55c6224781c0 2024-03-23T17:52:09.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.049+0000 7f78cb000700 1 -- 172.21.15.86:0/3390063789 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f78d009aab0 con 0x55c6224781c0 2024-03-23T17:52:09.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.049+0000 7f78cb000700 1 -- 172.21.15.86:0/3390063789 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d00a3540 con 0x55c6224781c0 2024-03-23T17:52:09.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.049+0000 7f78f879cb80 1 -- 172.21.15.86:0/3390063789 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c6224781c0 msgr2=0x55c6224765d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:09.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.049+0000 7f78f879cb80 1 --2- 172.21.15.86:0/3390063789 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c6224781c0 0x55c6224765d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f78d000f740 tx=0x7f78d0022c90 comp rx=0 tx=0).stop 2024-03-23T17:52:09.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.049+0000 7f78f879cb80 1 -- 172.21.15.86:0/3390063789 shutdown_connections 2024-03-23T17:52:09.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.049+0000 7f78f879cb80 1 --2- 172.21.15.86:0/3390063789 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c62248a490 0x55c62248c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:09.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.049+0000 7f78f879cb80 1 --2- 172.21.15.86:0/3390063789 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c6224781c0 0x55c6224765d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:09.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.049+0000 7f78f879cb80 1 --2- 172.21.15.86:0/3390063789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c622476b10 0x55c622476f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:09.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.049+0000 7f78f879cb80 1 -- 172.21.15.86:0/3390063789 >> 172.21.15.86:0/3390063789 conn(0x55c6223d66a0 msgr2=0x55c622472a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:09.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.050+0000 7f78f879cb80 1 -- 172.21.15.86:0/3390063789 shutdown_connections 2024-03-23T17:52:09.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.050+0000 7f78f879cb80 1 -- 172.21.15.86:0/3390063789 wait complete. 2024-03-23T17:52:09.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.050+0000 7f78f879cb80 1 -- start start 2024-03-23T17:52:09.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.050+0000 7f78f879cb80 1 Processor -- start 2024-03-23T17:52:09.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.051+0000 7f78f879cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c622476b10 0x55c62247d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:09.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.051+0000 7f78f879cb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c6224781c0 0x55c62247a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:09.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.051+0000 7f78f879cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c62248a490 0x55c62247ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:09.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.051+0000 7f78f879cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c622364d80 con 0x55c62248a490 2024-03-23T17:52:09.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.051+0000 7f78f879cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c622364a20 con 0x55c622476b10 2024-03-23T17:52:09.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.051+0000 7f78f879cb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c62224c6a0 con 0x55c6224781c0 2024-03-23T17:52:09.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.051+0000 7f78f879cb80 10 client.0 _ll_register_callbacks cb 0x55c62236e470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:52:09.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.051+0000 7f78dde00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c622476b10 0x55c62247d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:09.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.051+0000 7f78de800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c62248a490 0x55c62247ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:09.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.051+0000 7f78dde00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c622476b10 0x55c62247d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:41408/0 (socket says 172.21.15.86:41408) 2024-03-23T17:52:09.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.051+0000 7f78dd400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c6224781c0 0x55c62247a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:09.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.051+0000 7f78dde00700 1 -- 172.21.15.86:0/557965404 learned_addr learned my addr 172.21.15.86:0/557965404 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:09.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.051+0000 7f78dde00700 1 -- 172.21.15.86:0/557965404 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c6224781c0 msgr2=0x55c62247a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:09.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.051+0000 7f78dde00700 1 --2- 172.21.15.86:0/557965404 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c6224781c0 0x55c62247a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:09.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.051+0000 7f78dde00700 1 -- 172.21.15.86:0/557965404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c62248a490 msgr2=0x55c62247ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:09.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.051+0000 7f78dde00700 1 --2- 172.21.15.86:0/557965404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c62248a490 0x55c62247ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:09.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.051+0000 7f78dde00700 1 -- 172.21.15.86:0/557965404 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78cc0098e0 con 0x55c622476b10 2024-03-23T17:52:09.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.051+0000 7f78de800700 1 --2- 172.21.15.86:0/557965404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c62248a490 0x55c62247ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:52:09.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.052+0000 7f78dde00700 1 --2- 172.21.15.86:0/557965404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c622476b10 0x55c62247d310 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f78d009b0e0 tx=0x7f78d0022af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c34592a90c43d05 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:09.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.052+0000 7f78bf400700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T17:52:09.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.052+0000 7f78bf400700 1 -- 172.21.15.86:0/557965404 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d009fea0 con 0x55c622476b10 2024-03-23T17:52:09.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.052+0000 7f78bf400700 1 -- 172.21.15.86:0/557965404 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f78d009b850 con 0x55c622476b10 2024-03-23T17:52:09.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.052+0000 7f78f879cb80 1 -- 172.21.15.86:0/557965404 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c622389380 con 0x55c622476b10 2024-03-23T17:52:09.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.052+0000 7f78bf400700 1 -- 172.21.15.86:0/557965404 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f78d00a3540 con 0x55c622476b10 2024-03-23T17:52:09.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.052+0000 7f78f879cb80 10 client.26000 fetch_fsmap learned FSMap version 30 2024-03-23T17:52:09.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.052+0000 7f78f879cb80 1 -- 172.21.15.86:0/557965404 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f78e01a3f70 con 0x55c622476b10 2024-03-23T17:52:09.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.052+0000 7f78bf400700 1 -- 172.21.15.86:0/557965404 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c622389380 con 0x55c622476b10 2024-03-23T17:52:09.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.053+0000 7f78bf400700 1 -- 172.21.15.86:0/557965404 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 30) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f78d0065030 con 0x55c622476b10 2024-03-23T17:52:09.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.053+0000 7f78f879cb80 10 client.26000 fetch_fsmap finished waiting for FSMap version 30 2024-03-23T17:52:09.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.053+0000 7f78f879cb80 10 client.26000 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T17:52:09.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.053+0000 7f78f879cb80 20 client.26000 populate_metadata read hostname 'smithi086' 2024-03-23T17:52:09.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.053+0000 7f78f879cb80 1 --2- 172.21.15.86:0/557965404 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55c622499a80 0x55c62249bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:09.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.053+0000 7f78f879cb80 4 client.26000 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:52:09.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.053+0000 7f78f879cb80 1 -- 172.21.15.86:0/557965404 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c6223c1700 con 0x55c622499a80 2024-03-23T17:52:09.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.053+0000 7f78dd400700 1 --2- 172.21.15.86:0/557965404 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55c622499a80 0x55c62249bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:09.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.054+0000 7f78bf400700 10 client.26000 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:52:09.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.054+0000 7f78dd400700 1 --2- 172.21.15.86:0/557965404 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55c622499a80 0x55c62249bea0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc192f39027543bb server_cookie=7227940ec03636ad in_seq=0 out_seq=0 2024-03-23T17:52:09.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.054+0000 7f78bf400700 1 -- 172.21.15.86:0/557965404 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55c6223c1700 con 0x55c622499a80 2024-03-23T17:52:09.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.054+0000 7f78bf400700 10 client.26000 handle_command_reply: tid=0 2024-03-23T17:52:09.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.054+0000 7f78f879cb80 1 client.26000 shutdown 2024-03-23T17:52:09.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.054+0000 7f78f879cb80 1 -- 172.21.15.86:0/557965404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c622476b10 msgr2=0x55c62247d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:09.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.054+0000 7f78f879cb80 1 --2- 172.21.15.86:0/557965404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c622476b10 0x55c62247d310 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f78d009b0e0 tx=0x7f78d0022af0 comp rx=0 tx=0).stop 2024-03-23T17:52:09.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.054+0000 7f78f879cb80 1 -- 172.21.15.86:0/557965404 shutdown_connections 2024-03-23T17:52:09.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.054+0000 7f78f879cb80 1 --2- 172.21.15.86:0/557965404 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55c622499a80 0x55c62249bea0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:09.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.054+0000 7f78f879cb80 1 --2- 172.21.15.86:0/557965404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c62248a490 0x55c62247ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:09.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.054+0000 7f78f879cb80 1 --2- 172.21.15.86:0/557965404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c622476b10 0x55c62247d310 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:09.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.054+0000 7f78f879cb80 1 --2- 172.21.15.86:0/557965404 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c6224781c0 0x55c62247a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:09.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.054+0000 7f78bf400700 0 client.26000 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:52:09.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.054+0000 7f78f879cb80 1 -- 172.21.15.86:0/557965404 >> 172.21.15.86:0/557965404 conn(0x55c6223d66a0 msgr2=0x55c622478640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:09.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.055+0000 7f78f879cb80 1 -- 172.21.15.86:0/557965404 shutdown_connections 2024-03-23T17:52:09.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.055+0000 7f78f879cb80 1 -- 172.21.15.86:0/557965404 wait complete. 2024-03-23T17:52:09.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.055+0000 7f78f879cb80 20 client.26000 trim_cache size 0 max 16384 2024-03-23T17:52:09.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.068+0000 7f78f879cb80 1 Processor -- start 2024-03-23T17:52:09.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.068+0000 7f78f879cb80 1 -- start start 2024-03-23T17:52:09.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.068+0000 7f78f879cb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c622476b10 0x55c622490ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:09.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.068+0000 7f78f879cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c6224781c0 0x55c62246bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:09.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.068+0000 7f78f879cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c62248a490 0x55c62247cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:09.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.068+0000 7f78f879cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c622477e20 con 0x55c62248a490 2024-03-23T17:52:09.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.068+0000 7f78f879cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c62247afe0 con 0x55c6224781c0 2024-03-23T17:52:09.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.068+0000 7f78f879cb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c622489790 con 0x55c622476b10 2024-03-23T17:52:09.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.069+0000 7f78de800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c62248a490 0x55c62247cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:09.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.069+0000 7f78dde00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c622476b10 0x55c622490ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:09.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.069+0000 7f78de800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c62248a490 0x55c62247cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59256/0 (socket says 172.21.15.86:59256) 2024-03-23T17:52:09.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.069+0000 7f78de800700 1 -- 172.21.15.86:0/1144549816 learned_addr learned my addr 172.21.15.86:0/1144549816 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:09.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.069+0000 7f78dd400700 1 --2- 172.21.15.86:0/1144549816 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c6224781c0 0x55c62246bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:09.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.069+0000 7f78dde00700 1 -- 172.21.15.86:0/1144549816 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c6224781c0 msgr2=0x55c62246bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:09.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.069+0000 7f78dde00700 1 --2- 172.21.15.86:0/1144549816 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c6224781c0 0x55c62246bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:09.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.069+0000 7f78dde00700 1 -- 172.21.15.86:0/1144549816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c62248a490 msgr2=0x55c62247cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:09.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.069+0000 7f78dde00700 1 --2- 172.21.15.86:0/1144549816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c62248a490 0x55c62247cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:09.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.069+0000 7f78dde00700 1 -- 172.21.15.86:0/1144549816 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6221f6b60 con 0x55c622476b10 2024-03-23T17:52:09.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.069+0000 7f78de800700 1 --2- 172.21.15.86:0/1144549816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c62248a490 0x55c62247cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:52:09.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.069+0000 7f78dde00700 1 --2- 172.21.15.86:0/1144549816 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c622476b10 0x55c622490ee0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f78d0097650 tx=0x7f78d0022f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7953c0aeedfc5ae server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:09.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.069+0000 7f78cba00700 1 -- 172.21.15.86:0/1144549816 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d009fea0 con 0x55c622476b10 2024-03-23T17:52:09.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.069+0000 7f78cba00700 1 -- 172.21.15.86:0/1144549816 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f78d009af20 con 0x55c622476b10 2024-03-23T17:52:09.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.069+0000 7f78cba00700 1 -- 172.21.15.86:0/1144549816 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d00a3540 con 0x55c622476b10 2024-03-23T17:52:09.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.070+0000 7f78f879cb80 1 -- 172.21.15.86:0/1144549816 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c622476b10 msgr2=0x55c622490ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:09.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.070+0000 7f78f879cb80 1 --2- 172.21.15.86:0/1144549816 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c622476b10 0x55c622490ee0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f78d0097650 tx=0x7f78d0022f90 comp rx=0 tx=0).stop 2024-03-23T17:52:09.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.070+0000 7f78f879cb80 1 -- 172.21.15.86:0/1144549816 shutdown_connections 2024-03-23T17:52:09.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.070+0000 7f78f879cb80 1 --2- 172.21.15.86:0/1144549816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c62248a490 0x55c62247cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:09.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.070+0000 7f78f879cb80 1 --2- 172.21.15.86:0/1144549816 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c622476b10 0x55c622490ee0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:09.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.070+0000 7f78f879cb80 1 --2- 172.21.15.86:0/1144549816 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c6224781c0 0x55c62246bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:09.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.070+0000 7f78f879cb80 1 -- 172.21.15.86:0/1144549816 >> 172.21.15.86:0/1144549816 conn(0x55c6223d66a0 msgr2=0x55c6224763b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:09.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.070+0000 7f78f879cb80 1 -- 172.21.15.86:0/1144549816 shutdown_connections 2024-03-23T17:52:09.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.070+0000 7f78f879cb80 1 -- 172.21.15.86:0/1144549816 wait complete. 2024-03-23T17:52:09.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.071+0000 7f78f879cb80 1 -- start start 2024-03-23T17:52:09.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.071+0000 7f78f879cb80 1 Processor -- start 2024-03-23T17:52:09.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.071+0000 7f78f879cb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c622476b10 0x55c62248ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:09.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.071+0000 7f78f879cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6224781c0 0x55c62248ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:09.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.071+0000 7f78f879cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c62248a490 0x55c6224ea560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:09.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.071+0000 7f78f879cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c62248f6b0 con 0x55c6224781c0 2024-03-23T17:52:09.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.071+0000 7f78f879cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6224eaaa0 con 0x55c62248a490 2024-03-23T17:52:09.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.071+0000 7f78f879cb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6224eac30 con 0x55c622476b10 2024-03-23T17:52:09.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.071+0000 7f78f879cb80 10 client.0 _ll_register_callbacks cb 0x55c62236e470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:52:09.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.072+0000 7f78dd400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6224781c0 0x55c62248ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:09.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.072+0000 7f78dd400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6224781c0 0x55c62248ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59270/0 (socket says 172.21.15.86:59270) 2024-03-23T17:52:09.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.072+0000 7f78dd400700 1 -- 172.21.15.86:0/472725649 learned_addr learned my addr 172.21.15.86:0/472725649 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:09.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.072+0000 7f78dde00700 1 --2- 172.21.15.86:0/472725649 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c622476b10 0x55c62248ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:09.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.072+0000 7f78de800700 1 --2- 172.21.15.86:0/472725649 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c62248a490 0x55c6224ea560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:09.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.072+0000 7f78dd400700 1 -- 172.21.15.86:0/472725649 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c622476b10 msgr2=0x55c62248ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:09.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.072+0000 7f78dd400700 1 --2- 172.21.15.86:0/472725649 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c622476b10 0x55c62248ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:09.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.072+0000 7f78dd400700 1 -- 172.21.15.86:0/472725649 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c62248a490 msgr2=0x55c6224ea560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:09.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.072+0000 7f78dd400700 1 --2- 172.21.15.86:0/472725649 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c62248a490 0x55c6224ea560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:09.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.072+0000 7f78dd400700 1 -- 172.21.15.86:0/472725649 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78d800e8b0 con 0x55c6224781c0 2024-03-23T17:52:09.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.072+0000 7f78dd400700 1 --2- 172.21.15.86:0/472725649 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6224781c0 0x55c62248ef80 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x55c62247bb40 tx=0x7f78cc004320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f831737d0c71239 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:09.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.072+0000 7f78bf400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-23T17:52:09.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.072+0000 7f78bf400700 1 -- 172.21.15.86:0/472725649 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78cc017070 con 0x55c6224781c0 2024-03-23T17:52:09.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.072+0000 7f78bf400700 1 -- 172.21.15.86:0/472725649 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f78cc021a80 con 0x55c6224781c0 2024-03-23T17:52:09.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.072+0000 7f78f879cb80 1 -- 172.21.15.86:0/472725649 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f78d009aab0 con 0x55c6224781c0 2024-03-23T17:52:09.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.073+0000 7f78bf400700 1 -- 172.21.15.86:0/472725649 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f78cc0122e0 con 0x55c6224781c0 2024-03-23T17:52:09.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.073+0000 7f78f879cb80 10 client.15948 fetch_fsmap learned FSMap version 30 2024-03-23T17:52:09.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.073+0000 7f78f879cb80 1 -- 172.21.15.86:0/472725649 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f78e01a3f70 con 0x55c6224781c0 2024-03-23T17:52:09.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.073+0000 7f78bf400700 1 -- 172.21.15.86:0/472725649 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f78d009aab0 con 0x55c6224781c0 2024-03-23T17:52:09.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.073+0000 7f78bf400700 1 -- 172.21.15.86:0/472725649 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f78cc0125c0 con 0x55c6224781c0 2024-03-23T17:52:09.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.073+0000 7f78f879cb80 10 client.15948 fetch_fsmap finished waiting for FSMap version 30 2024-03-23T17:52:09.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.073+0000 7f78f879cb80 10 client.15948 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T17:52:09.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.073+0000 7f78f879cb80 20 client.15948 populate_metadata read hostname 'smithi086' 2024-03-23T17:52:09.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.073+0000 7f78f879cb80 1 --2- 172.21.15.86:0/472725649 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55c6224ec5a0 0x55c6224ee9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:09.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.073+0000 7f78f879cb80 4 client.15948 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-23T17:52:09.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.073+0000 7f78f879cb80 1 -- 172.21.15.86:0/472725649 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c622217cf0 con 0x55c6224ec5a0 2024-03-23T17:52:09.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.073+0000 7f78dde00700 1 --2- 172.21.15.86:0/472725649 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55c6224ec5a0 0x55c6224ee9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:09.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.074+0000 7f78dde00700 1 --2- 172.21.15.86:0/472725649 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55c6224ec5a0 0x55c6224ee9c0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36482ba21f313dce server_cookie=99cc429afb22fb1b in_seq=0 out_seq=0 2024-03-23T17:52:09.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.074+0000 7f78bf400700 10 client.15948 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:52:09.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.074+0000 7f78bf400700 1 -- 172.21.15.86:0/472725649 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c622217cf0 con 0x55c6224ec5a0 2024-03-23T17:52:09.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.074+0000 7f78bf400700 10 client.15948 handle_command_reply: tid=0 2024-03-23T17:52:09.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.074+0000 7f78f879cb80 1 client.15948 shutdown 2024-03-23T17:52:09.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.074+0000 7f78f879cb80 1 -- 172.21.15.86:0/472725649 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6224781c0 msgr2=0x55c62248ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:09.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.074+0000 7f78f879cb80 1 --2- 172.21.15.86:0/472725649 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6224781c0 0x55c62248ef80 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x55c62247bb40 tx=0x7f78cc004320 comp rx=0 tx=0).stop 2024-03-23T17:52:09.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.074+0000 7f78f879cb80 1 -- 172.21.15.86:0/472725649 shutdown_connections 2024-03-23T17:52:09.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.074+0000 7f78f879cb80 1 --2- 172.21.15.86:0/472725649 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55c6224ec5a0 0x55c6224ee9c0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:09.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.074+0000 7f78f879cb80 1 --2- 172.21.15.86:0/472725649 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c62248a490 0x55c6224ea560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:09.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.074+0000 7f78f879cb80 1 --2- 172.21.15.86:0/472725649 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c622476b10 0x55c62248ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:09.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.074+0000 7f78f879cb80 1 --2- 172.21.15.86:0/472725649 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6224781c0 0x55c62248ef80 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:09.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.074+0000 7f78f879cb80 1 -- 172.21.15.86:0/472725649 >> 172.21.15.86:0/472725649 conn(0x55c6223d66a0 msgr2=0x55c62246bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:09.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.074+0000 7f78bf400700 0 client.15948 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:52:09.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.074+0000 7f78f879cb80 1 -- 172.21.15.86:0/472725649 shutdown_connections 2024-03-23T17:52:09.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.075+0000 7f78f879cb80 1 -- 172.21.15.86:0/472725649 wait complete. 2024-03-23T17:52:09.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.075+0000 7f78f879cb80 20 client.15948 trim_cache size 0 max 16384 2024-03-23T17:52:09.083 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-03-23T17:52:09.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.075+0000 7f78e4c00700 1 -- 172.21.15.86:0/3668474187 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f78c40a13b0 msgr2=0x7f78c40a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:09.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.075+0000 7f78e4c00700 1 --2- 172.21.15.86:0/3668474187 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f78c40a13b0 0x7f78c40a3870 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f78d8003e80 tx=0x7f78d8009920 comp rx=0 tx=0).stop 2024-03-23T17:52:09.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.075+0000 7f78e4c00700 1 -- 172.21.15.86:0/3668474187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78e0067e30 msgr2=0x7f78e0197200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:09.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.075+0000 7f78e4c00700 1 --2- 172.21.15.86:0/3668474187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78e0067e30 0x7f78e0197200 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f78d0005910 tx=0x7f78d0004730 comp rx=0 tx=0).stop 2024-03-23T17:52:09.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.075+0000 7f78e4c00700 1 -- 172.21.15.86:0/3668474187 shutdown_connections 2024-03-23T17:52:09.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.075+0000 7f78e4c00700 1 --2- 172.21.15.86:0/3668474187 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78e0105ce0 0x7f78e019bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:09.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.075+0000 7f78e4c00700 1 --2- 172.21.15.86:0/3668474187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78e0067e30 0x7f78e0197200 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:09.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.075+0000 7f78e4c00700 1 --2- 172.21.15.86:0/3668474187 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f78c40a13b0 0x7f78c40a3870 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:09.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.075+0000 7f78e4c00700 1 --2- 172.21.15.86:0/3668474187 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78e00687f0 0x7f78e0197740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:09.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.075+0000 7f78e4c00700 1 -- 172.21.15.86:0/3668474187 >> 172.21.15.86:0/3668474187 conn(0x7f78e0076d40 msgr2=0x7f78e00fd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:09.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.075+0000 7f78e4c00700 1 -- 172.21.15.86:0/3668474187 shutdown_connections 2024-03-23T17:52:09.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:09.075+0000 7f78e4c00700 1 -- 172.21.15.86:0/3668474187 wait complete. 2024-03-23T17:52:09.881 INFO:tasks.workunit.client.0.smithi086.stdout: 1 29083 5.12 MB/sec warmup 34 sec latency 945.373 ms 2024-03-23T17:52:10.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:09 smithi092 ceph-mon[79089]: pgmap v48: 97 pgs: 97 active+clean; 26 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 3.6 MiB/s wr, 108 op/s 2024-03-23T17:52:10.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:09 smithi152 ceph-mon[47949]: pgmap v48: 97 pgs: 97 active+clean; 26 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 3.6 MiB/s wr, 108 op/s 2024-03-23T17:52:10.097 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-23T17:52:10.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T17:52:10.097 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-23T17:52:10.161 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:09 smithi086 ceph-mon[38221]: pgmap v48: 97 pgs: 97 active+clean; 26 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 3.6 MiB/s wr, 108 op/s 2024-03-23T17:52:10.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.302+0000 7f99abc00700 1 -- 172.21.15.86:0/1482542077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99a4100f50 msgr2=0x7f99a4101330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:10.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.302+0000 7f99abc00700 1 --2- 172.21.15.86:0/1482542077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99a4100f50 0x7f99a4101330 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f9998009990 tx=0x7f9998009ca0 comp rx=0 tx=0).stop 2024-03-23T17:52:10.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.302+0000 7f99abc00700 1 -- 172.21.15.86:0/1482542077 shutdown_connections 2024-03-23T17:52:10.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.302+0000 7f99abc00700 1 --2- 172.21.15.86:0/1482542077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99a4106070 0x7f99a410dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:10.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.302+0000 7f99abc00700 1 --2- 172.21.15.86:0/1482542077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99a4100f50 0x7f99a4101330 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:10.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.302+0000 7f99abc00700 1 --2- 172.21.15.86:0/1482542077 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f99a4101900 0x7f99a4105940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:10.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.302+0000 7f99abc00700 1 -- 172.21.15.86:0/1482542077 >> 172.21.15.86:0/1482542077 conn(0x7f99a40fc7f0 msgr2=0x7f99a40fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:10.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.302+0000 7f99abc00700 1 -- 172.21.15.86:0/1482542077 shutdown_connections 2024-03-23T17:52:10.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.302+0000 7f99abc00700 1 -- 172.21.15.86:0/1482542077 wait complete. 2024-03-23T17:52:10.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.303+0000 7f99abc00700 1 Processor -- start 2024-03-23T17:52:10.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.303+0000 7f99abc00700 1 -- start start 2024-03-23T17:52:10.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.304+0000 7f99abc00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99a4100f50 0x7f99a4199520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:10.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.304+0000 7f99abc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99a4106070 0x7f99a4199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:10.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.304+0000 7f99abc00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f99a419dd40 0x7f99a419e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:10.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.304+0000 7f99abc00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a419e730 con 0x7f99a4106070 2024-03-23T17:52:10.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.304+0000 7f99abc00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a419e8a0 con 0x7f99a4100f50 2024-03-23T17:52:10.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.304+0000 7f99abc00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a419ea60 con 0x7f99a419dd40 2024-03-23T17:52:10.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.304+0000 7f99a3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99a4106070 0x7f99a4199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:10.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.304+0000 7f99a3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99a4106070 0x7f99a4199a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59286/0 (socket says 172.21.15.86:59286) 2024-03-23T17:52:10.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.304+0000 7f99a3e00700 1 -- 172.21.15.86:0/484572880 learned_addr learned my addr 172.21.15.86:0/484572880 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:10.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.304+0000 7f99a9200700 1 --2- 172.21.15.86:0/484572880 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99a4100f50 0x7f99a4199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:10.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.304+0000 7f99a9c00700 1 --2- 172.21.15.86:0/484572880 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f99a419dd40 0x7f99a419e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:10.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.304+0000 7f99a3e00700 1 -- 172.21.15.86:0/484572880 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f99a419dd40 msgr2=0x7f99a419e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:10.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.304+0000 7f99a3e00700 1 --2- 172.21.15.86:0/484572880 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f99a419dd40 0x7f99a419e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:10.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.304+0000 7f99a3e00700 1 -- 172.21.15.86:0/484572880 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99a4100f50 msgr2=0x7f99a4199520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:10.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.304+0000 7f99a3e00700 1 --2- 172.21.15.86:0/484572880 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99a4100f50 0x7f99a4199520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:10.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.304+0000 7f99a3e00700 1 -- 172.21.15.86:0/484572880 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99980095f0 con 0x7f99a4106070 2024-03-23T17:52:10.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.305+0000 7f99a3e00700 1 --2- 172.21.15.86:0/484572880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99a4106070 0x7f99a4199a60 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f999000e970 tx=0x7f999000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:10.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.305+0000 7f99a1600700 1 -- 172.21.15.86:0/484572880 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9990009790 con 0x7f99a4106070 2024-03-23T17:52:10.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.305+0000 7f99a1600700 1 -- 172.21.15.86:0/484572880 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f999000c7e0 con 0x7f99a4106070 2024-03-23T17:52:10.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.305+0000 7f99abc00700 1 -- 172.21.15.86:0/484572880 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99a41a6010 con 0x7f99a4106070 2024-03-23T17:52:10.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.305+0000 7f99a1600700 1 -- 172.21.15.86:0/484572880 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99900054e0 con 0x7f99a4106070 2024-03-23T17:52:10.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.305+0000 7f99abc00700 1 -- 172.21.15.86:0/484572880 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99a41a6520 con 0x7f99a4106070 2024-03-23T17:52:10.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.308+0000 7f99a1600700 1 -- 172.21.15.86:0/484572880 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) v1 ==== 136098+0+0 (secure 0 0 0) 0x7f999000c950 con 0x7f99a4106070 2024-03-23T17:52:10.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.309+0000 7f99bf747b80 1 Processor -- start 2024-03-23T17:52:10.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.309+0000 7f99bf747b80 1 -- start start 2024-03-23T17:52:10.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.309+0000 7f99a1600700 1 --2- 172.21.15.86:0/484572880 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f998c0a13e0 0x7f998c0a38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:10.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.309+0000 7f99bf747b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a76750a650 0x55a76750aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:10.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.309+0000 7f99bf747b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a76750af70 0x55a767515930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:10.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.309+0000 7f99a1600700 1 -- 172.21.15.86:0/484572880 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f9990014070 con 0x7f99a4106070 2024-03-23T17:52:10.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.309+0000 7f99bf747b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a767515e70 0x55a767518260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:10.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.310+0000 7f99bf747b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a767463a10 con 0x55a767515e70 2024-03-23T17:52:10.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.310+0000 7f99bf747b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7674613d0 con 0x55a76750af70 2024-03-23T17:52:10.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.310+0000 7f99bf747b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a76745ccc0 con 0x55a76750a650 2024-03-23T17:52:10.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.310+0000 7f99a9c00700 1 --2- 172.21.15.86:0/484572880 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f998c0a13e0 0x7f998c0a38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:10.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.310+0000 7f99a3e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a76750af70 0x55a767515930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:10.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.310+0000 7f99a9200700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a76750a650 0x55a76750aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:10.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.310+0000 7f99a3e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a76750af70 0x55a767515930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:41468/0 (socket says 172.21.15.86:41468) 2024-03-23T17:52:10.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.310+0000 7f99a3e00700 1 -- 172.21.15.86:0/898290135 learned_addr learned my addr 172.21.15.86:0/898290135 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:10.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.310+0000 7f99a9c00700 1 --2- 172.21.15.86:0/898290135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a767515e70 0x55a767518260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:10.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.310+0000 7f99a3e00700 1 -- 172.21.15.86:0/898290135 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a76750a650 msgr2=0x55a76750aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:10.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.310+0000 7f99a3e00700 1 --2- 172.21.15.86:0/898290135 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a76750a650 0x55a76750aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:10.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.310+0000 7f99a3e00700 1 -- 172.21.15.86:0/898290135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a767515e70 msgr2=0x55a767518260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:10.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.310+0000 7f99a3e00700 1 --2- 172.21.15.86:0/898290135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a767515e70 0x55a767518260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:10.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.310+0000 7f99a3e00700 1 -- 172.21.15.86:0/898290135 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7672de430 con 0x55a76750af70 2024-03-23T17:52:10.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.310+0000 7f99a9c00700 1 --2- 172.21.15.86:0/898290135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a767515e70 0x55a767518260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:52:10.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.310+0000 7f99a3e00700 1 --2- 172.21.15.86:0/898290135 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a76750af70 0x55a767515930 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9990099d90 tx=0x7f999000cc50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=921fc1f87d729cc3 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:10.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.311+0000 7f99a9c00700 1 --2- 172.21.15.86:0/484572880 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f998c0a13e0 0x7f998c0a38a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f999c009ec0 tx=0x7f999c009920 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:10.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.311+0000 7f9996000700 1 -- 172.21.15.86:0/898290135 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999009dec0 con 0x55a76750af70 2024-03-23T17:52:10.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.311+0000 7f9996000700 1 -- 172.21.15.86:0/898290135 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99900a2510 con 0x55a76750af70 2024-03-23T17:52:10.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.311+0000 7f9996000700 1 -- 172.21.15.86:0/898290135 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99900997e0 con 0x55a76750af70 2024-03-23T17:52:10.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.311+0000 7f99bf747b80 1 -- 172.21.15.86:0/898290135 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a76750af70 msgr2=0x55a767515930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:10.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.311+0000 7f99bf747b80 1 --2- 172.21.15.86:0/898290135 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a76750af70 0x55a767515930 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9990099d90 tx=0x7f999000cc50 comp rx=0 tx=0).stop 2024-03-23T17:52:10.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.311+0000 7f99bf747b80 1 -- 172.21.15.86:0/898290135 shutdown_connections 2024-03-23T17:52:10.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.311+0000 7f99bf747b80 1 --2- 172.21.15.86:0/898290135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a767515e70 0x55a767518260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:10.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.311+0000 7f99bf747b80 1 --2- 172.21.15.86:0/898290135 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a76750a650 0x55a76750aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:10.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.311+0000 7f99bf747b80 1 --2- 172.21.15.86:0/898290135 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a76750af70 0x55a767515930 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:10.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.311+0000 7f99bf747b80 1 -- 172.21.15.86:0/898290135 >> 172.21.15.86:0/898290135 conn(0x55a76746a720 msgr2=0x55a767506b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:10.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.311+0000 7f99bf747b80 1 -- 172.21.15.86:0/898290135 shutdown_connections 2024-03-23T17:52:10.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.312+0000 7f99bf747b80 1 -- 172.21.15.86:0/898290135 wait complete. 2024-03-23T17:52:10.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.312+0000 7f99bf747b80 1 -- start start 2024-03-23T17:52:10.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.312+0000 7f99bf747b80 1 Processor -- start 2024-03-23T17:52:10.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.313+0000 7f99bf747b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a76750a650 0x55a767515820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:10.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.313+0000 7f99bf747b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a76750af70 0x55a767512b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:10.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.313+0000 7f99bf747b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a767515e70 0x55a767513120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:10.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.313+0000 7f99bf747b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7673f8e00 con 0x55a76750a650 2024-03-23T17:52:10.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.313+0000 7f99bf747b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7673f8aa0 con 0x55a767515e70 2024-03-23T17:52:10.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.313+0000 7f99bf747b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7672e0720 con 0x55a76750af70 2024-03-23T17:52:10.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.313+0000 7f99bf747b80 10 client.0 _ll_register_callbacks cb 0x55a7674024f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:52:10.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.313+0000 7f99a9200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a76750a650 0x55a767515820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:10.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.313+0000 7f99a9c00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a767515e70 0x55a767513120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:10.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.313+0000 7f99a3e00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a76750af70 0x55a767512b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:10.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.313+0000 7f99a9c00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a767515e70 0x55a767513120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:41480/0 (socket says 172.21.15.86:41480) 2024-03-23T17:52:10.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.313+0000 7f99a3e00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a76750af70 0x55a767512b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:51220/0 (socket says 172.21.15.86:51220) 2024-03-23T17:52:10.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.313+0000 7f99a9c00700 1 -- 172.21.15.86:0/3916257274 learned_addr learned my addr 172.21.15.86:0/3916257274 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:10.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.313+0000 7f99a9c00700 1 -- 172.21.15.86:0/3916257274 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a76750af70 msgr2=0x55a767512b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:10.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.313+0000 7f99a9c00700 1 --2- 172.21.15.86:0/3916257274 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a76750af70 0x55a767512b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:10.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.313+0000 7f99a9c00700 1 -- 172.21.15.86:0/3916257274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a76750a650 msgr2=0x55a767515820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:10.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.313+0000 7f99a9c00700 1 --2- 172.21.15.86:0/3916257274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a76750a650 0x55a767515820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:10.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.313+0000 7f99a9c00700 1 -- 172.21.15.86:0/3916257274 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99a41a6520 con 0x55a767515e70 2024-03-23T17:52:10.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.314+0000 7f99a9c00700 1 --2- 172.21.15.86:0/3916257274 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a767515e70 0x55a767513120 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f999c018a70 tx=0x7f999c004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9301b8ba6046c36 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:10.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.314+0000 7f998aa00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T17:52:10.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.314+0000 7f998aa00700 1 -- 172.21.15.86:0/3916257274 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999c020070 con 0x55a767515e70 2024-03-23T17:52:10.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.314+0000 7f99bf747b80 1 -- 172.21.15.86:0/3916257274 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a76741d400 con 0x55a767515e70 2024-03-23T17:52:10.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.314+0000 7f998aa00700 1 -- 172.21.15.86:0/3916257274 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f999c01b890 con 0x55a767515e70 2024-03-23T17:52:10.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.314+0000 7f998aa00700 1 -- 172.21.15.86:0/3916257274 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f999c00e070 con 0x55a767515e70 2024-03-23T17:52:10.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.314+0000 7f99bf747b80 10 client.26018 fetch_fsmap learned FSMap version 30 2024-03-23T17:52:10.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.314+0000 7f99bf747b80 1 -- 172.21.15.86:0/3916257274 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55a76728abe0 con 0x55a767515e70 2024-03-23T17:52:10.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.315+0000 7f998aa00700 1 -- 172.21.15.86:0/3916257274 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f999c033020 con 0x55a767515e70 2024-03-23T17:52:10.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.315+0000 7f998aa00700 1 -- 172.21.15.86:0/3916257274 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 30) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f999c025b80 con 0x55a767515e70 2024-03-23T17:52:10.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.315+0000 7f99bf747b80 10 client.26018 fetch_fsmap finished waiting for FSMap version 30 2024-03-23T17:52:10.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.315+0000 7f99bf747b80 10 client.26018 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T17:52:10.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.315+0000 7f99bf747b80 20 client.26018 populate_metadata read hostname 'smithi086' 2024-03-23T17:52:10.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.315+0000 7f99bf747b80 1 --2- 172.21.15.86:0/3916257274 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55a767525a10 0x55a767527e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:10.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.315+0000 7f99bf747b80 4 client.26018 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:52:10.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.315+0000 7f99bf747b80 1 -- 172.21.15.86:0/3916257274 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a767455780 con 0x55a767525a10 2024-03-23T17:52:10.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.315+0000 7f99a9200700 1 --2- 172.21.15.86:0/3916257274 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55a767525a10 0x55a767527e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:10.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.316+0000 7f998aa00700 10 client.26018 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:52:10.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.316+0000 7f99a9200700 1 --2- 172.21.15.86:0/3916257274 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55a767525a10 0x55a767527e30 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3731f732861a517 server_cookie=499fe20f18e65774 in_seq=0 out_seq=0 2024-03-23T17:52:10.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.317+0000 7f998aa00700 1 -- 172.21.15.86:0/3916257274 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55a767455780 con 0x55a767525a10 2024-03-23T17:52:10.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.317+0000 7f998aa00700 10 client.26018 handle_command_reply: tid=0 2024-03-23T17:52:10.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.317+0000 7f99bf747b80 1 client.26018 shutdown 2024-03-23T17:52:10.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.317+0000 7f99bf747b80 1 -- 172.21.15.86:0/3916257274 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a767515e70 msgr2=0x55a767513120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:10.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.317+0000 7f99bf747b80 1 --2- 172.21.15.86:0/3916257274 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a767515e70 0x55a767513120 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f999c018a70 tx=0x7f999c004b40 comp rx=0 tx=0).stop 2024-03-23T17:52:10.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.317+0000 7f99bf747b80 1 -- 172.21.15.86:0/3916257274 shutdown_connections 2024-03-23T17:52:10.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.317+0000 7f99bf747b80 1 --2- 172.21.15.86:0/3916257274 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55a767525a10 0x55a767527e30 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:10.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.317+0000 7f99bf747b80 1 --2- 172.21.15.86:0/3916257274 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a767515e70 0x55a767513120 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:10.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.317+0000 7f99bf747b80 1 --2- 172.21.15.86:0/3916257274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a76750a650 0x55a767515820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:10.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.317+0000 7f99bf747b80 1 --2- 172.21.15.86:0/3916257274 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a76750af70 0x55a767512b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:10.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.317+0000 7f998aa00700 0 client.26018 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:52:10.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.317+0000 7f99bf747b80 1 -- 172.21.15.86:0/3916257274 >> 172.21.15.86:0/3916257274 conn(0x55a76746a720 msgr2=0x55a76750c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:10.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.317+0000 7f99bf747b80 1 -- 172.21.15.86:0/3916257274 shutdown_connections 2024-03-23T17:52:10.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.317+0000 7f99bf747b80 1 -- 172.21.15.86:0/3916257274 wait complete. 2024-03-23T17:52:10.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.317+0000 7f99bf747b80 20 client.26018 trim_cache size 0 max 16384 2024-03-23T17:52:10.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.336+0000 7f99bf747b80 1 Processor -- start 2024-03-23T17:52:10.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.337+0000 7f99bf747b80 1 -- start start 2024-03-23T17:52:10.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.337+0000 7f99bf747b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a76750a650 0x55a76751caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:10.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.337+0000 7f99bf747b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a76750af70 0x55a767511ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:10.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.337+0000 7f99bf747b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a767515e70 0x55a767521950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:10.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.337+0000 7f99bf747b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7675110f0 con 0x55a76750a650 2024-03-23T17:52:10.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.337+0000 7f99bf747b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a767506fa0 con 0x55a767515e70 2024-03-23T17:52:10.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.337+0000 7f99bf747b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7675230b0 con 0x55a76750af70 2024-03-23T17:52:10.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.337+0000 7f99a9200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a76750a650 0x55a76751caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:10.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.337+0000 7f99a9c00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a767515e70 0x55a767521950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:10.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.337+0000 7f99a3e00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a76750af70 0x55a767511ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:10.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.337+0000 7f99a9200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a76750a650 0x55a76751caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59318/0 (socket says 172.21.15.86:59318) 2024-03-23T17:52:10.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.337+0000 7f99a9200700 1 -- 172.21.15.86:0/112950246 learned_addr learned my addr 172.21.15.86:0/112950246 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:10.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.338+0000 7f99a3e00700 1 -- 172.21.15.86:0/112950246 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a767515e70 msgr2=0x55a767521950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:10.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.338+0000 7f99a3e00700 1 --2- 172.21.15.86:0/112950246 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a767515e70 0x55a767521950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:10.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.338+0000 7f99a3e00700 1 -- 172.21.15.86:0/112950246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a76750a650 msgr2=0x55a76751caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:10.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.338+0000 7f99a3e00700 1 --2- 172.21.15.86:0/112950246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a76750a650 0x55a76751caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:10.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.338+0000 7f99a3e00700 1 -- 172.21.15.86:0/112950246 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a767422f20 con 0x55a76750af70 2024-03-23T17:52:10.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.338+0000 7f99a9c00700 1 --2- 172.21.15.86:0/112950246 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a767515e70 0x55a767521950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:52:10.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.338+0000 7f99a3e00700 1 --2- 172.21.15.86:0/112950246 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a76750af70 0x55a767511ad0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x55a76750b3f0 tx=0x7f9990005910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eefa5aadb0c8c5eb server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:10.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.338+0000 7f9996a00700 1 -- 172.21.15.86:0/112950246 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999009dec0 con 0x55a76750af70 2024-03-23T17:52:10.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.338+0000 7f9996a00700 1 -- 172.21.15.86:0/112950246 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9990098430 con 0x55a76750af70 2024-03-23T17:52:10.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.338+0000 7f9996a00700 1 -- 172.21.15.86:0/112950246 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99900997e0 con 0x55a76750af70 2024-03-23T17:52:10.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.339+0000 7f99bf747b80 1 -- 172.21.15.86:0/112950246 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a76750af70 msgr2=0x55a767511ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:10.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.339+0000 7f99bf747b80 1 --2- 172.21.15.86:0/112950246 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a76750af70 0x55a767511ad0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x55a76750b3f0 tx=0x7f9990005910 comp rx=0 tx=0).stop 2024-03-23T17:52:10.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.339+0000 7f99bf747b80 1 -- 172.21.15.86:0/112950246 shutdown_connections 2024-03-23T17:52:10.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.339+0000 7f99bf747b80 1 --2- 172.21.15.86:0/112950246 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a767515e70 0x55a767521950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:10.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.339+0000 7f99bf747b80 1 --2- 172.21.15.86:0/112950246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a76750a650 0x55a76751caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:10.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.339+0000 7f99bf747b80 1 --2- 172.21.15.86:0/112950246 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a76750af70 0x55a767511ad0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:10.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.339+0000 7f99bf747b80 1 -- 172.21.15.86:0/112950246 >> 172.21.15.86:0/112950246 conn(0x55a76746a720 msgr2=0x55a76750c650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:10.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.339+0000 7f99bf747b80 1 -- 172.21.15.86:0/112950246 shutdown_connections 2024-03-23T17:52:10.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.339+0000 7f99bf747b80 1 -- 172.21.15.86:0/112950246 wait complete. 2024-03-23T17:52:10.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.339+0000 7f99bf747b80 1 -- start start 2024-03-23T17:52:10.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.340+0000 7f99bf747b80 1 Processor -- start 2024-03-23T17:52:10.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.340+0000 7f99bf747b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a76750a650 0x55a76751bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:10.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.340+0000 7f99bf747b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a76750af70 0x55a76751c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:10.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.340+0000 7f99bf747b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a767515e70 0x55a7675707c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:10.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.340+0000 7f99bf747b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a767570d70 con 0x55a76750af70 2024-03-23T17:52:10.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.340+0000 7f99bf747b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a767570ee0 con 0x55a76750a650 2024-03-23T17:52:10.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.340+0000 7f99bf747b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7675710a0 con 0x55a767515e70 2024-03-23T17:52:10.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.340+0000 7f99bf747b80 10 client.0 _ll_register_callbacks cb 0x55a7674024f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:52:10.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.340+0000 7f99a3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a76750af70 0x55a76751c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:10.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.340+0000 7f99a3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a76750af70 0x55a76751c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59322/0 (socket says 172.21.15.86:59322) 2024-03-23T17:52:10.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.340+0000 7f99a3e00700 1 -- 172.21.15.86:0/1094416984 learned_addr learned my addr 172.21.15.86:0/1094416984 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:10.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.341+0000 7f99a9c00700 1 --2- 172.21.15.86:0/1094416984 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a767515e70 0x55a7675707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:10.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.341+0000 7f99a9200700 1 --2- 172.21.15.86:0/1094416984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a76750a650 0x55a76751bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:10.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.341+0000 7f99a3e00700 1 -- 172.21.15.86:0/1094416984 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a767515e70 msgr2=0x55a7675707c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:10.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.341+0000 7f99a3e00700 1 --2- 172.21.15.86:0/1094416984 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a767515e70 0x55a7675707c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:10.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.341+0000 7f99a3e00700 1 -- 172.21.15.86:0/1094416984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a76750a650 msgr2=0x55a76751bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:10.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.341+0000 7f99a3e00700 1 --2- 172.21.15.86:0/1094416984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a76750a650 0x55a76751bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:10.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.341+0000 7f99a3e00700 1 -- 172.21.15.86:0/1094416984 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a76728abe0 con 0x55a76750af70 2024-03-23T17:52:10.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.341+0000 7f99a9200700 1 --2- 172.21.15.86:0/1094416984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a76750a650 0x55a76751bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T17:52:10.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.341+0000 7f99a3e00700 1 --2- 172.21.15.86:0/1094416984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a76750af70 0x55a76751c410 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f9990018000 tx=0x7f9990005870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff11c18fd133c3b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:10.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.341+0000 7f998aa00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-23T17:52:10.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.341+0000 7f998aa00700 1 -- 172.21.15.86:0/1094416984 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999009dec0 con 0x55a76750af70 2024-03-23T17:52:10.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.341+0000 7f998aa00700 1 -- 172.21.15.86:0/1094416984 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f999009a500 con 0x55a76750af70 2024-03-23T17:52:10.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.341+0000 7f99bf747b80 1 -- 172.21.15.86:0/1094416984 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f99900a2510 con 0x55a76750af70 2024-03-23T17:52:10.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.341+0000 7f998aa00700 1 -- 172.21.15.86:0/1094416984 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9990099580 con 0x55a76750af70 2024-03-23T17:52:10.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.341+0000 7f99bf747b80 10 client.15966 fetch_fsmap learned FSMap version 30 2024-03-23T17:52:10.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.341+0000 7f99bf747b80 1 -- 172.21.15.86:0/1094416984 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f99900b6040 con 0x55a76750af70 2024-03-23T17:52:10.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.342+0000 7f998aa00700 1 -- 172.21.15.86:0/1094416984 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99900a2510 con 0x55a76750af70 2024-03-23T17:52:10.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.342+0000 7f998aa00700 1 -- 172.21.15.86:0/1094416984 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f9990095e80 con 0x55a76750af70 2024-03-23T17:52:10.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.342+0000 7f99bf747b80 10 client.15966 fetch_fsmap finished waiting for FSMap version 30 2024-03-23T17:52:10.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.342+0000 7f99bf747b80 10 client.15966 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T17:52:10.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.342+0000 7f99bf747b80 20 client.15966 populate_metadata read hostname 'smithi086' 2024-03-23T17:52:10.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.342+0000 7f99bf747b80 1 --2- 172.21.15.86:0/1094416984 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55a767579770 0x55a76757bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:10.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.342+0000 7f99bf747b80 4 client.15966 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-23T17:52:10.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.342+0000 7f99bf747b80 1 -- 172.21.15.86:0/1094416984 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55a7672abd70 con 0x55a767579770 2024-03-23T17:52:10.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.342+0000 7f99a9200700 1 --2- 172.21.15.86:0/1094416984 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55a767579770 0x55a76757bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:10.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.343+0000 7f998aa00700 10 client.15966 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:52:10.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.343+0000 7f99a9200700 1 --2- 172.21.15.86:0/1094416984 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55a767579770 0x55a76757bb90 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f4fda9e69f4af16 server_cookie=86329bf2b07080cb in_seq=0 out_seq=0 2024-03-23T17:52:10.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.343+0000 7f998aa00700 1 -- 172.21.15.86:0/1094416984 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55a7672abd70 con 0x55a767579770 2024-03-23T17:52:10.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.343+0000 7f998aa00700 10 client.15966 handle_command_reply: tid=0 2024-03-23T17:52:10.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.344+0000 7f99bf747b80 1 client.15966 shutdown 2024-03-23T17:52:10.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.344+0000 7f99bf747b80 1 -- 172.21.15.86:0/1094416984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a76750af70 msgr2=0x55a76751c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:10.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.344+0000 7f99bf747b80 1 --2- 172.21.15.86:0/1094416984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a76750af70 0x55a76751c410 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f9990018000 tx=0x7f9990005870 comp rx=0 tx=0).stop 2024-03-23T17:52:10.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.344+0000 7f99bf747b80 1 -- 172.21.15.86:0/1094416984 shutdown_connections 2024-03-23T17:52:10.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.344+0000 7f99bf747b80 1 --2- 172.21.15.86:0/1094416984 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55a767579770 0x55a76757bb90 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:10.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.344+0000 7f99bf747b80 1 --2- 172.21.15.86:0/1094416984 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a767515e70 0x55a7675707c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:10.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.344+0000 7f99bf747b80 1 --2- 172.21.15.86:0/1094416984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a76750a650 0x55a76751bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:10.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.344+0000 7f998aa00700 0 client.15966 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:52:10.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.344+0000 7f99bf747b80 1 --2- 172.21.15.86:0/1094416984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a76750af70 0x55a76751c410 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:10.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.344+0000 7f99bf747b80 1 -- 172.21.15.86:0/1094416984 >> 172.21.15.86:0/1094416984 conn(0x55a76746a720 msgr2=0x55a76750cec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:10.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.344+0000 7f99bf747b80 1 -- 172.21.15.86:0/1094416984 shutdown_connections 2024-03-23T17:52:10.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.344+0000 7f99bf747b80 1 -- 172.21.15.86:0/1094416984 wait complete. 2024-03-23T17:52:10.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.344+0000 7f99bf747b80 20 client.15966 trim_cache size 0 max 16384 2024-03-23T17:52:10.350 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-23T17:52:10.350 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-03-23T17:52:10.350 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "5ee6a087-503e-4da8-8a76-c398aee6e05e", 2024-03-23T17:52:10.350 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-03-23T17:52:10.350 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-23T17:52:10.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.345+0000 7f99abc00700 1 -- 172.21.15.86:0/484572880 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f998c0a13e0 msgr2=0x7f998c0a38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:10.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.345+0000 7f99abc00700 1 --2- 172.21.15.86:0/484572880 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f998c0a13e0 0x7f998c0a38a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f999c009ec0 tx=0x7f999c009920 comp rx=0 tx=0).stop 2024-03-23T17:52:10.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.345+0000 7f99abc00700 1 -- 172.21.15.86:0/484572880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99a4106070 msgr2=0x7f99a4199a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:10.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.345+0000 7f99abc00700 1 --2- 172.21.15.86:0/484572880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99a4106070 0x7f99a4199a60 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f999000e970 tx=0x7f999000ec80 comp rx=0 tx=0).stop 2024-03-23T17:52:10.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.345+0000 7f99abc00700 1 -- 172.21.15.86:0/484572880 shutdown_connections 2024-03-23T17:52:10.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.345+0000 7f99abc00700 1 --2- 172.21.15.86:0/484572880 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f998c0a13e0 0x7f998c0a38a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:10.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.345+0000 7f99abc00700 1 --2- 172.21.15.86:0/484572880 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f99a419dd40 0x7f99a419e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:10.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.345+0000 7f99abc00700 1 --2- 172.21.15.86:0/484572880 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99a4100f50 0x7f99a4199520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:10.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.345+0000 7f99abc00700 1 --2- 172.21.15.86:0/484572880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99a4106070 0x7f99a4199a60 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:10.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.345+0000 7f99abc00700 1 -- 172.21.15.86:0/484572880 >> 172.21.15.86:0/484572880 conn(0x7f99a40fc7f0 msgr2=0x7f99a40febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:10.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.345+0000 7f99abc00700 1 -- 172.21.15.86:0/484572880 shutdown_connections 2024-03-23T17:52:10.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.345+0000 7f99abc00700 1 -- 172.21.15.86:0/484572880 wait complete. 2024-03-23T17:52:10.366 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-23T17:52:10.367 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-23T17:52:10.426 INFO:journalctl@ceph.mds.d.smithi086.stdout:Mar 23 17:52:10 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-d[89510]: 2024-03-23T17:52:10.351+0000 7f4b06000700 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020d [3,head] /volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/EXCEL/PCMAGCD.XLS auth v764 ap=1 s=260096 n(v0 rc2024-03-23T17:51:45.276433+0000 b260096 1=1+0) (iversion lock) caps={25258=pAsLsXsFscr/-@2},l=25258(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x557c46620b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000020d:[<0x1000000020c/PCMAGCD.XLS v2>,<0x10000000009/EXCEL v5280>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v37>,<0x10000000006/clients v25>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v28>,<0x100000001fa/aa7fd127-71c3-47dd-b16f-38c26b2c15fc v44>,<0x10000000001/sv_1 v65>,<0x10000000000/qa v54>,<0x1/volumes v50>]//[]","memoryvalue":"(3)0x1000000020d:[<0x1000000020c/PCMAGCD.XLS v764>,<0x10000000009/EXCEL v5574>,<0x10000000008/~dmtmp v206>,<0x10000000007/client0 v79>,<0x10000000006/clients v57>,<0x10000000005/tmp v57>,<0x100000001fb/client.0 v60>,<0x100000001fa/aa7fd127-71c3-47dd-b16f-38c26b2c15fc v74>,<0x10000000001/sv_1 v81>,<0x10000000000/qa v42>,<0x1/volumes v74>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-23T17:52:10.426 INFO:journalctl@ceph.mds.d.smithi086.stdout:Mar 23 17:52:10 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-d[89510]: 2024-03-23T17:52:10.353+0000 7f4b06000700 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000209 [3,head] /volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/WORD/CHAP10.DOC auth v486 ap=1 s=260096 n(v0 rc2024-03-23T17:51:35.932632+0000 b260096 1=1+0) (iversion lock) caps={25258=pAsLsXsFscr/-@2},l=25258(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x557c4663cb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000209:[<0x10000000208/CHAP10.DOC v2>,<0x10000000009/WORD v5120>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v37>,<0x10000000006/clients v25>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v28>,<0x100000001fa/aa7fd127-71c3-47dd-b16f-38c26b2c15fc v44>,<0x10000000001/sv_1 v65>,<0x10000000000/qa v54>,<0x1/volumes v50>]//[]","memoryvalue":"(3)0x10000000209:[<0x10000000208/CHAP10.DOC v486>,<0x10000000009/WORD v5572>,<0x10000000008/~dmtmp v206>,<0x10000000007/client0 v79>,<0x10000000006/clients v57>,<0x10000000005/tmp v57>,<0x100000001fb/client.0 v60>,<0x100000001fa/aa7fd127-71c3-47dd-b16f-38c26b2c15fc v74>,<0x10000000001/sv_1 v81>,<0x10000000000/qa v42>,<0x1/volumes v74>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-23T17:52:10.426 INFO:journalctl@ceph.mds.d.smithi086.stdout:Mar 23 17:52:10 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-d[89510]: 2024-03-23T17:52:10.353+0000 7f4b06000700 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020b [3,head] /volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/WORD/FACTS.DOC auth v488 ap=1 s=260096 n(v0 rc2024-03-23T17:51:35.943632+0000 b260096 1=1+0) (iversion lock) caps={25258=pAsLsXsFscr/-@2},l=25258(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x557c4663c000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000020b:[<0x10000000208/FACTS.DOC v6>,<0x10000000009/WORD v5120>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v37>,<0x10000000006/clients v25>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v28>,<0x100000001fa/aa7fd127-71c3-47dd-b16f-38c26b2c15fc v44>,<0x10000000001/sv_1 v65>,<0x10000000000/qa v54>,<0x1/volumes v50>]//[]","memoryvalue":"(3)0x1000000020b:[<0x10000000208/FACTS.DOC v488>,<0x10000000009/WORD v5572>,<0x10000000008/~dmtmp v206>,<0x10000000007/client0 v79>,<0x10000000006/clients v57>,<0x10000000005/tmp v57>,<0x100000001fb/client.0 v60>,<0x100000001fa/aa7fd127-71c3-47dd-b16f-38c26b2c15fc v74>,<0x10000000001/sv_1 v81>,<0x10000000000/qa v42>,<0x1/volumes v74>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-23T17:52:10.426 INFO:journalctl@ceph.mds.d.smithi086.stdout:Mar 23 17:52:10 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-d[89510]: 2024-03-23T17:52:10.353+0000 7f4b06000700 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020e [3,head] /volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/EXCEL/SALES.XLS auth v766 ap=1 s=260096 n(v0 rc2024-03-23T17:51:45.382430+0000 b260096 1=1+0) (iversion lock) caps={25258=pAsLsXsFscr/-@2},l=25258(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x557c46621600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000020e:[<0x1000000020c/SALES.XLS v4>,<0x10000000009/EXCEL v5280>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v37>,<0x10000000006/clients v25>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v28>,<0x100000001fa/aa7fd127-71c3-47dd-b16f-38c26b2c15fc v44>,<0x10000000001/sv_1 v65>,<0x10000000000/qa v54>,<0x1/volumes v50>]//[]","memoryvalue":"(3)0x1000000020e:[<0x1000000020c/SALES.XLS v766>,<0x10000000009/EXCEL v5574>,<0x10000000008/~dmtmp v206>,<0x10000000007/client0 v79>,<0x10000000006/clients v57>,<0x10000000005/tmp v57>,<0x100000001fb/client.0 v60>,<0x100000001fa/aa7fd127-71c3-47dd-b16f-38c26b2c15fc v74>,<0x10000000001/sv_1 v81>,<0x10000000000/qa v42>,<0x1/volumes v74>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-23T17:52:10.426 INFO:journalctl@ceph.mds.d.smithi086.stdout:Mar 23 17:52:10 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-d[89510]: 2024-03-23T17:52:10.353+0000 7f4b06000700 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020a [3,head] /volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/WORD/BASEMACH.DOC auth v490 ap=1 s=260096 n(v0 rc2024-03-23T17:51:35.938632+0000 b260096 1=1+0) (iversion lock) caps={25258=pAsLsXsFscr/-@2},l=25258(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x557c46623700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000020a:[<0x10000000208/BASEMACH.DOC v4>,<0x10000000009/WORD v5120>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v37>,<0x10000000006/clients v25>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v28>,<0x100000001fa/aa7fd127-71c3-47dd-b16f-38c26b2c15fc v44>,<0x10000000001/sv_1 v65>,<0x10000000000/qa v54>,<0x1/volumes v50>]//[]","memoryvalue":"(3)0x1000000020a:[<0x10000000208/BASEMACH.DOC v490>,<0x10000000009/WORD v5572>,<0x10000000008/~dmtmp v206>,<0x10000000007/client0 v79>,<0x10000000006/clients v57>,<0x10000000005/tmp v57>,<0x100000001fb/client.0 v60>,<0x100000001fa/aa7fd127-71c3-47dd-b16f-38c26b2c15fc v74>,<0x10000000001/sv_1 v81>,<0x10000000000/qa v42>,<0x1/volumes v74>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-23T17:52:10.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.562+0000 7ff165000700 1 -- 172.21.15.86:0/3272658807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff160073b00 msgr2=0x7ff16010cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:10.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.562+0000 7ff165000700 1 --2- 172.21.15.86:0/3272658807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff160073b00 0x7ff16010cff0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7ff14c009990 tx=0x7ff14c009ca0 comp rx=0 tx=0).stop 2024-03-23T17:52:10.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.562+0000 7ff165000700 1 -- 172.21.15.86:0/3272658807 shutdown_connections 2024-03-23T17:52:10.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.562+0000 7ff165000700 1 --2- 172.21.15.86:0/3272658807 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff16010d530 0x7ff16010f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:10.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.562+0000 7ff165000700 1 --2- 172.21.15.86:0/3272658807 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff1600731e0 0x7ff1600735c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:10.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.562+0000 7ff165000700 1 --2- 172.21.15.86:0/3272658807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff160073b00 0x7ff16010cff0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:10.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.562+0000 7ff165000700 1 -- 172.21.15.86:0/3272658807 >> 172.21.15.86:0/3272658807 conn(0x7ff1600fc7c0 msgr2=0x7ff1600febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:10.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.562+0000 7ff165000700 1 -- 172.21.15.86:0/3272658807 shutdown_connections 2024-03-23T17:52:10.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.562+0000 7ff165000700 1 -- 172.21.15.86:0/3272658807 wait complete. 2024-03-23T17:52:10.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.563+0000 7ff165000700 1 Processor -- start 2024-03-23T17:52:10.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.563+0000 7ff165000700 1 -- start start 2024-03-23T17:52:10.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.564+0000 7ff165000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1600731e0 0x7ff160199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:10.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.564+0000 7ff165000700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff160073b00 0x7ff160199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:10.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.564+0000 7ff165000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff16010d530 0x7ff16019dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:10.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.564+0000 7ff165000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff16019e360 con 0x7ff16010d530 2024-03-23T17:52:10.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.564+0000 7ff165000700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff16019e4d0 con 0x7ff1600731e0 2024-03-23T17:52:10.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.564+0000 7ff165000700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff16019e690 con 0x7ff160073b00 2024-03-23T17:52:10.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.564+0000 7ff15de00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1600731e0 0x7ff160199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:10.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.564+0000 7ff15e800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff16010d530 0x7ff16019dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:10.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.564+0000 7ff15d400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff160073b00 0x7ff160199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:10.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.564+0000 7ff15de00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1600731e0 0x7ff160199440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:41518/0 (socket says 172.21.15.86:41518) 2024-03-23T17:52:10.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.564+0000 7ff15de00700 1 -- 172.21.15.86:0/591045006 learned_addr learned my addr 172.21.15.86:0/591045006 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:10.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.564+0000 7ff15de00700 1 -- 172.21.15.86:0/591045006 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff160073b00 msgr2=0x7ff160199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:10.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.564+0000 7ff15de00700 1 --2- 172.21.15.86:0/591045006 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff160073b00 0x7ff160199980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:10.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.564+0000 7ff15de00700 1 -- 172.21.15.86:0/591045006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff16010d530 msgr2=0x7ff16019dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:10.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.564+0000 7ff15de00700 1 --2- 172.21.15.86:0/591045006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff16010d530 0x7ff16019dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:10.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.564+0000 7ff15de00700 1 -- 172.21.15.86:0/591045006 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff14c0095f0 con 0x7ff1600731e0 2024-03-23T17:52:10.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.564+0000 7ff15e800700 1 --2- 172.21.15.86:0/591045006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff16010d530 0x7ff16019dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:52:10.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.565+0000 7ff15de00700 1 --2- 172.21.15.86:0/591045006 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1600731e0 0x7ff160199440 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff15000e9a0 tx=0x7ff15000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:10.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.565+0000 7ff156a00700 1 -- 172.21.15.86:0/591045006 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff150009790 con 0x7ff1600731e0 2024-03-23T17:52:10.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.565+0000 7ff165000700 1 -- 172.21.15.86:0/591045006 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff16019e970 con 0x7ff1600731e0 2024-03-23T17:52:10.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.565+0000 7ff156a00700 1 -- 172.21.15.86:0/591045006 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff150010430 con 0x7ff1600731e0 2024-03-23T17:52:10.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.565+0000 7ff156a00700 1 -- 172.21.15.86:0/591045006 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff150018680 con 0x7ff1600731e0 2024-03-23T17:52:10.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.565+0000 7ff165000700 1 -- 172.21.15.86:0/591045006 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1601a61b0 con 0x7ff1600731e0 2024-03-23T17:52:10.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.568+0000 7ff156a00700 1 -- 172.21.15.86:0/591045006 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 25) v1 ==== 136098+0+0 (secure 0 0 0) 0x7ff1500105a0 con 0x7ff1600731e0 2024-03-23T17:52:10.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.570+0000 7ff178b7db80 1 Processor -- start 2024-03-23T17:52:10.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.570+0000 7ff178b7db80 1 -- start start 2024-03-23T17:52:10.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.570+0000 7ff156a00700 1 --2- 172.21.15.86:0/591045006 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7ff1440a13b0 0x7ff1440a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:10.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.570+0000 7ff178b7db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2402c75d0 0x55f2402c79b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:10.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.570+0000 7ff178b7db80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f2402c7ef0 0x55f2402d28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:10.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.570+0000 7ff178b7db80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f2402d2df0 0x55f2402d51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:10.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.570+0000 7ff178b7db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f240220990 con 0x55f2402c75d0 2024-03-23T17:52:10.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.570+0000 7ff178b7db80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f24021e350 con 0x55f2402c7ef0 2024-03-23T17:52:10.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.570+0000 7ff178b7db80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f240219c40 con 0x55f2402d2df0 2024-03-23T17:52:10.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.571+0000 7ff156a00700 1 -- 172.21.15.86:0/591045006 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7ff150014070 con 0x7ff1600731e0 2024-03-23T17:52:10.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.571+0000 7ff15de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2402c75d0 0x55f2402c79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:10.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.571+0000 7ff15de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2402c75d0 0x55f2402c79b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59346/0 (socket says 172.21.15.86:59346) 2024-03-23T17:52:10.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.571+0000 7ff15de00700 1 -- 172.21.15.86:0/2720714026 learned_addr learned my addr 172.21.15.86:0/2720714026 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:10.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.571+0000 7ff15d400700 1 --2- 172.21.15.86:0/2720714026 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f2402c7ef0 0x55f2402d28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:10.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.571+0000 7ff15e800700 1 --2- 172.21.15.86:0/591045006 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7ff1440a13b0 0x7ff1440a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:10.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.571+0000 7ff15de00700 1 -- 172.21.15.86:0/2720714026 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f2402d2df0 msgr2=0x55f2402d51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:10.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.571+0000 7ff15e800700 1 --2- 172.21.15.86:0/2720714026 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f2402d2df0 0x55f2402d51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:10.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.571+0000 7ff15de00700 1 --2- 172.21.15.86:0/2720714026 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f2402d2df0 0x55f2402d51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:10.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.571+0000 7ff15de00700 1 -- 172.21.15.86:0/2720714026 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f2402c7ef0 msgr2=0x55f2402d28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:10.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.571+0000 7ff15de00700 1 --2- 172.21.15.86:0/2720714026 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f2402c7ef0 0x55f2402d28b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:10.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.571+0000 7ff15de00700 1 -- 172.21.15.86:0/2720714026 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f24009b3b0 con 0x55f2402c75d0 2024-03-23T17:52:10.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.571+0000 7ff15e800700 1 --2- 172.21.15.86:0/2720714026 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f2402d2df0 0x55f2402d51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T17:52:10.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.571+0000 7ff15de00700 1 --2- 172.21.15.86:0/2720714026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2402c75d0 0x55f2402c79b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ff15000e970 tx=0x7ff150099a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ba474c8454dd078 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:10.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.571+0000 7ff15e800700 1 --2- 172.21.15.86:0/591045006 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7ff1440a13b0 0x7ff1440a3870 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff158007900 tx=0x7ff158009c70 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:10.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.571+0000 7ff14b400700 1 -- 172.21.15.86:0/2720714026 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff15009a730 con 0x55f2402c75d0 2024-03-23T17:52:10.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.572+0000 7ff14b400700 1 -- 172.21.15.86:0/2720714026 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff15009a030 con 0x55f2402c75d0 2024-03-23T17:52:10.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.572+0000 7ff14b400700 1 -- 172.21.15.86:0/2720714026 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1500a2280 con 0x55f2402c75d0 2024-03-23T17:52:10.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.572+0000 7ff178b7db80 1 -- 172.21.15.86:0/2720714026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2402c75d0 msgr2=0x55f2402c79b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:10.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.572+0000 7ff178b7db80 1 --2- 172.21.15.86:0/2720714026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2402c75d0 0x55f2402c79b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ff15000e970 tx=0x7ff150099a30 comp rx=0 tx=0).stop 2024-03-23T17:52:10.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.572+0000 7ff178b7db80 1 -- 172.21.15.86:0/2720714026 shutdown_connections 2024-03-23T17:52:10.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.572+0000 7ff178b7db80 1 --2- 172.21.15.86:0/2720714026 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f2402d2df0 0x55f2402d51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:10.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.572+0000 7ff178b7db80 1 --2- 172.21.15.86:0/2720714026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2402c75d0 0x55f2402c79b0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:10.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.572+0000 7ff178b7db80 1 --2- 172.21.15.86:0/2720714026 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f2402c7ef0 0x55f2402d28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:10.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.572+0000 7ff178b7db80 1 -- 172.21.15.86:0/2720714026 >> 172.21.15.86:0/2720714026 conn(0x55f2402276a0 msgr2=0x55f2402c3a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:10.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.572+0000 7ff178b7db80 1 -- 172.21.15.86:0/2720714026 shutdown_connections 2024-03-23T17:52:10.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.572+0000 7ff178b7db80 1 -- 172.21.15.86:0/2720714026 wait complete. 2024-03-23T17:52:10.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.573+0000 7ff178b7db80 1 -- start start 2024-03-23T17:52:10.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.573+0000 7ff178b7db80 1 Processor -- start 2024-03-23T17:52:10.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.573+0000 7ff178b7db80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f2402c75d0 0x55f2402d2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:10.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.573+0000 7ff178b7db80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f2402c7ef0 0x55f2402cfac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:10.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.573+0000 7ff178b7db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2402d2df0 0x55f2402d0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:10.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.574+0000 7ff178b7db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2401b5d80 con 0x55f2402d2df0 2024-03-23T17:52:10.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.574+0000 7ff178b7db80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2401b5a20 con 0x55f2402c7ef0 2024-03-23T17:52:10.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.574+0000 7ff178b7db80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f24009d6a0 con 0x55f2402c75d0 2024-03-23T17:52:10.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.574+0000 7ff178b7db80 10 client.0 _ll_register_callbacks cb 0x55f2401bf470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:52:10.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.574+0000 7ff15d400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f2402c7ef0 0x55f2402cfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:10.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.574+0000 7ff15e800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2402d2df0 0x55f2402d0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:10.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.574+0000 7ff15d400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f2402c7ef0 0x55f2402cfac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:41522/0 (socket says 172.21.15.86:41522) 2024-03-23T17:52:10.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.574+0000 7ff15d400700 1 -- 172.21.15.86:0/320001149 learned_addr learned my addr 172.21.15.86:0/320001149 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:10.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.574+0000 7ff15de00700 1 --2- 172.21.15.86:0/320001149 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f2402c75d0 0x55f2402d2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:10.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.574+0000 7ff15d400700 1 -- 172.21.15.86:0/320001149 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f2402c75d0 msgr2=0x55f2402d2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:10.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.574+0000 7ff15d400700 1 --2- 172.21.15.86:0/320001149 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f2402c75d0 0x55f2402d2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:10.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.574+0000 7ff15d400700 1 -- 172.21.15.86:0/320001149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2402d2df0 msgr2=0x55f2402d0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:10.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.574+0000 7ff15d400700 1 --2- 172.21.15.86:0/320001149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2402d2df0 0x55f2402d0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:10.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.574+0000 7ff15d400700 1 -- 172.21.15.86:0/320001149 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1601a61b0 con 0x55f2402c7ef0 2024-03-23T17:52:10.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.574+0000 7ff15d400700 1 --2- 172.21.15.86:0/320001149 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f2402c7ef0 0x55f2402cfac0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff14c015000 tx=0x7ff14c0057e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3d91dadb6f5b608 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:10.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.574+0000 7ff13fe00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T17:52:10.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.575+0000 7ff13fe00700 1 -- 172.21.15.86:0/320001149 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff14c01d070 con 0x55f2402c7ef0 2024-03-23T17:52:10.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.575+0000 7ff13fe00700 1 -- 172.21.15.86:0/320001149 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7ff14c004500 con 0x55f2402c7ef0 2024-03-23T17:52:10.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.575+0000 7ff178b7db80 1 -- 172.21.15.86:0/320001149 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f2401da380 con 0x55f2402c7ef0 2024-03-23T17:52:10.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.575+0000 7ff13fe00700 1 -- 172.21.15.86:0/320001149 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff14c004700 con 0x55f2402c7ef0 2024-03-23T17:52:10.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.575+0000 7ff178b7db80 10 client.26033 fetch_fsmap learned FSMap version 30 2024-03-23T17:52:10.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.575+0000 7ff178b7db80 1 -- 172.21.15.86:0/320001149 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55f240047b60 con 0x55f2402c7ef0 2024-03-23T17:52:10.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.575+0000 7ff13fe00700 1 -- 172.21.15.86:0/320001149 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f2401da380 con 0x55f2402c7ef0 2024-03-23T17:52:10.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.575+0000 7ff13fe00700 1 -- 172.21.15.86:0/320001149 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 30) v1 ==== 7478+0+0 (secure 0 0 0) 0x7ff14c00f870 con 0x55f2402c7ef0 2024-03-23T17:52:10.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.576+0000 7ff178b7db80 10 client.26033 fetch_fsmap finished waiting for FSMap version 30 2024-03-23T17:52:10.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.576+0000 7ff178b7db80 10 client.26033 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T17:52:10.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.576+0000 7ff178b7db80 20 client.26033 populate_metadata read hostname 'smithi086' 2024-03-23T17:52:10.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.576+0000 7ff178b7db80 1 --2- 172.21.15.86:0/320001149 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55f2402e2960 0x55f2402e4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:10.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.576+0000 7ff178b7db80 4 client.26033 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:52:10.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.576+0000 7ff178b7db80 1 -- 172.21.15.86:0/320001149 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f240212700 con 0x55f2402e2960 2024-03-23T17:52:10.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.576+0000 7ff15de00700 1 --2- 172.21.15.86:0/320001149 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55f2402e2960 0x55f2402e4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:10.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.576+0000 7ff15de00700 1 --2- 172.21.15.86:0/320001149 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55f2402e2960 0x55f2402e4d80 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e406c6f361dce02 server_cookie=a3f0020cb46fb276 in_seq=0 out_seq=0 2024-03-23T17:52:10.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.576+0000 7ff13fe00700 10 client.26033 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:52:10.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.577+0000 7ff13fe00700 1 -- 172.21.15.86:0/320001149 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55f240212700 con 0x55f2402e2960 2024-03-23T17:52:10.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.577+0000 7ff13fe00700 10 client.26033 handle_command_reply: tid=0 2024-03-23T17:52:10.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.577+0000 7ff178b7db80 1 client.26033 shutdown 2024-03-23T17:52:10.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.577+0000 7ff178b7db80 1 -- 172.21.15.86:0/320001149 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f2402c7ef0 msgr2=0x55f2402cfac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:10.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.577+0000 7ff178b7db80 1 --2- 172.21.15.86:0/320001149 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f2402c7ef0 0x55f2402cfac0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff14c015000 tx=0x7ff14c0057e0 comp rx=0 tx=0).stop 2024-03-23T17:52:10.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.577+0000 7ff178b7db80 1 -- 172.21.15.86:0/320001149 shutdown_connections 2024-03-23T17:52:10.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.577+0000 7ff178b7db80 1 --2- 172.21.15.86:0/320001149 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55f2402e2960 0x55f2402e4d80 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:10.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.577+0000 7ff178b7db80 1 --2- 172.21.15.86:0/320001149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2402d2df0 0x55f2402d0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:10.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.577+0000 7ff178b7db80 1 --2- 172.21.15.86:0/320001149 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f2402c75d0 0x55f2402d2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:10.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.577+0000 7ff178b7db80 1 --2- 172.21.15.86:0/320001149 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f2402c7ef0 0x55f2402cfac0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:10.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.577+0000 7ff13fe00700 0 client.26033 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:52:10.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.577+0000 7ff178b7db80 1 -- 172.21.15.86:0/320001149 >> 172.21.15.86:0/320001149 conn(0x55f2402276a0 msgr2=0x55f2402c96c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:10.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.577+0000 7ff178b7db80 1 -- 172.21.15.86:0/320001149 shutdown_connections 2024-03-23T17:52:10.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.577+0000 7ff178b7db80 1 -- 172.21.15.86:0/320001149 wait complete. 2024-03-23T17:52:10.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.578+0000 7ff178b7db80 20 client.26033 trim_cache size 0 max 16384 2024-03-23T17:52:10.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.594+0000 7ff178b7db80 1 Processor -- start 2024-03-23T17:52:10.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.594+0000 7ff178b7db80 1 -- start start 2024-03-23T17:52:10.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.594+0000 7ff178b7db80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f2402c75d0 0x55f2402d9a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:10.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.594+0000 7ff178b7db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2402c7ef0 0x55f2402d22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:10.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.594+0000 7ff178b7db80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f2402d2df0 0x55f2402ccd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:10.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.594+0000 7ff178b7db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2402dffa0 con 0x55f2402c7ef0 2024-03-23T17:52:10.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.594+0000 7ff178b7db80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2402df470 con 0x55f2402c75d0 2024-03-23T17:52:10.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.594+0000 7ff178b7db80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2402e9ad0 con 0x55f2402d2df0 2024-03-23T17:52:10.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.595+0000 7ff15d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2402c7ef0 0x55f2402d22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:10.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.595+0000 7ff15d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2402c7ef0 0x55f2402d22b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59376/0 (socket says 172.21.15.86:59376) 2024-03-23T17:52:10.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.595+0000 7ff15d400700 1 -- 172.21.15.86:0/1704624351 learned_addr learned my addr 172.21.15.86:0/1704624351 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:10.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.595+0000 7ff15de00700 1 --2- 172.21.15.86:0/1704624351 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f2402c75d0 0x55f2402d9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:10.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.595+0000 7ff15e800700 1 --2- 172.21.15.86:0/1704624351 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f2402d2df0 0x55f2402ccd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:10.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.595+0000 7ff15d400700 1 -- 172.21.15.86:0/1704624351 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f2402d2df0 msgr2=0x55f2402ccd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:10.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.595+0000 7ff15d400700 1 --2- 172.21.15.86:0/1704624351 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f2402d2df0 0x55f2402ccd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:10.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.595+0000 7ff15d400700 1 -- 172.21.15.86:0/1704624351 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f2402c75d0 msgr2=0x55f2402d9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:10.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.595+0000 7ff15d400700 1 --2- 172.21.15.86:0/1704624351 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f2402c75d0 0x55f2402d9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:10.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.595+0000 7ff15d400700 1 -- 172.21.15.86:0/1704624351 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f2401dfea0 con 0x55f2402c7ef0 2024-03-23T17:52:10.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.595+0000 7ff15d400700 1 --2- 172.21.15.86:0/1704624351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2402c7ef0 0x55f2402d22b0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7ff14c0098e0 tx=0x7ff14c009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7fa2fda54d1c0a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:10.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.595+0000 7ff14be00700 1 -- 172.21.15.86:0/1704624351 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff14c01d030 con 0x55f2402c7ef0 2024-03-23T17:52:10.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.595+0000 7ff14be00700 1 -- 172.21.15.86:0/1704624351 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff14c01e040 con 0x55f2402c7ef0 2024-03-23T17:52:10.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.595+0000 7ff14be00700 1 -- 172.21.15.86:0/1704624351 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff14c016890 con 0x55f2402c7ef0 2024-03-23T17:52:10.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.596+0000 7ff178b7db80 1 -- 172.21.15.86:0/1704624351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2402c7ef0 msgr2=0x55f2402d22b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:10.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.596+0000 7ff178b7db80 1 --2- 172.21.15.86:0/1704624351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2402c7ef0 0x55f2402d22b0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7ff14c0098e0 tx=0x7ff14c009510 comp rx=0 tx=0).stop 2024-03-23T17:52:10.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.596+0000 7ff178b7db80 1 -- 172.21.15.86:0/1704624351 shutdown_connections 2024-03-23T17:52:10.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.596+0000 7ff178b7db80 1 --2- 172.21.15.86:0/1704624351 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f2402d2df0 0x55f2402ccd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:10.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.596+0000 7ff178b7db80 1 --2- 172.21.15.86:0/1704624351 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f2402c75d0 0x55f2402d9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:10.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.596+0000 7ff178b7db80 1 --2- 172.21.15.86:0/1704624351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2402c7ef0 0x55f2402d22b0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:10.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.596+0000 7ff178b7db80 1 -- 172.21.15.86:0/1704624351 >> 172.21.15.86:0/1704624351 conn(0x55f2402276a0 msgr2=0x55f2402ce390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:10.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.596+0000 7ff178b7db80 1 -- 172.21.15.86:0/1704624351 shutdown_connections 2024-03-23T17:52:10.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.596+0000 7ff178b7db80 1 -- 172.21.15.86:0/1704624351 wait complete. 2024-03-23T17:52:10.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.596+0000 7ff178b7db80 1 -- start start 2024-03-23T17:52:10.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.597+0000 7ff178b7db80 1 Processor -- start 2024-03-23T17:52:10.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.597+0000 7ff178b7db80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f2402c75d0 0x55f2402d95d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:10.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.597+0000 7ff178b7db80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f2402c7ef0 0x55f2402d7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:10.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.597+0000 7ff178b7db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2402d2df0 0x55f2402d7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:10.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.597+0000 7ff178b7db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2402d84f0 con 0x55f2402d2df0 2024-03-23T17:52:10.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.597+0000 7ff178b7db80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2402d8660 con 0x55f2402c75d0 2024-03-23T17:52:10.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.597+0000 7ff178b7db80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2402d8820 con 0x55f2402c7ef0 2024-03-23T17:52:10.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.597+0000 7ff178b7db80 10 client.0 _ll_register_callbacks cb 0x55f2401bf470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:52:10.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.597+0000 7ff15e800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2402d2df0 0x55f2402d7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:10.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.598+0000 7ff15de00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f2402c75d0 0x55f2402d95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:10.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.598+0000 7ff15e800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2402d2df0 0x55f2402d7fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59380/0 (socket says 172.21.15.86:59380) 2024-03-23T17:52:10.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.598+0000 7ff15e800700 1 -- 172.21.15.86:0/1341693199 learned_addr learned my addr 172.21.15.86:0/1341693199 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:10.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.598+0000 7ff15d400700 1 --2- 172.21.15.86:0/1341693199 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f2402c7ef0 0x55f2402d7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:10.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.598+0000 7ff15e800700 1 -- 172.21.15.86:0/1341693199 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f2402c7ef0 msgr2=0x55f2402d7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:10.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.598+0000 7ff15e800700 1 --2- 172.21.15.86:0/1341693199 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f2402c7ef0 0x55f2402d7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:10.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.598+0000 7ff15e800700 1 -- 172.21.15.86:0/1341693199 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f2402c75d0 msgr2=0x55f2402d95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:10.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.598+0000 7ff15e800700 1 --2- 172.21.15.86:0/1341693199 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f2402c75d0 0x55f2402d95d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:10.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.598+0000 7ff15e800700 1 -- 172.21.15.86:0/1341693199 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f240047b60 con 0x55f2402d2df0 2024-03-23T17:52:10.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.598+0000 7ff15d400700 1 --2- 172.21.15.86:0/1341693199 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f2402c7ef0 0x55f2402d7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:52:10.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.598+0000 7ff15de00700 1 --2- 172.21.15.86:0/1341693199 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f2402c75d0 0x55f2402d95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:52:10.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.598+0000 7ff15e800700 1 --2- 172.21.15.86:0/1341693199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2402d2df0 0x55f2402d7fb0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7ff1580124c0 tx=0x7ff158004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57eaf42a24ddb491 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:10.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.598+0000 7ff13fe00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-23T17:52:10.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.598+0000 7ff13fe00700 1 -- 172.21.15.86:0/1341693199 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff158021030 con 0x55f2402d2df0 2024-03-23T17:52:10.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.598+0000 7ff13fe00700 1 -- 172.21.15.86:0/1341693199 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7ff1580125f0 con 0x55f2402d2df0 2024-03-23T17:52:10.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.599+0000 7ff178b7db80 1 -- 172.21.15.86:0/1341693199 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff15009a030 con 0x55f2402d2df0 2024-03-23T17:52:10.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.599+0000 7ff13fe00700 1 -- 172.21.15.86:0/1341693199 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff158004900 con 0x55f2402d2df0 2024-03-23T17:52:10.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.599+0000 7ff178b7db80 10 client.15984 fetch_fsmap learned FSMap version 30 2024-03-23T17:52:10.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.599+0000 7ff178b7db80 1 -- 172.21.15.86:0/1341693199 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55f24009fcd0 con 0x55f2402d2df0 2024-03-23T17:52:10.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.599+0000 7ff13fe00700 1 -- 172.21.15.86:0/1341693199 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff15009a030 con 0x55f2402d2df0 2024-03-23T17:52:10.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.599+0000 7ff13fe00700 1 -- 172.21.15.86:0/1341693199 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) v1 ==== 7478+0+0 (secure 0 0 0) 0x7ff158028830 con 0x55f2402d2df0 2024-03-23T17:52:10.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.599+0000 7ff178b7db80 10 client.15984 fetch_fsmap finished waiting for FSMap version 30 2024-03-23T17:52:10.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.599+0000 7ff178b7db80 10 client.15984 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T17:52:10.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.599+0000 7ff178b7db80 20 client.15984 populate_metadata read hostname 'smithi086' 2024-03-23T17:52:10.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.599+0000 7ff178b7db80 1 --2- 172.21.15.86:0/1341693199 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55f240335a30 0x55f240337e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:10.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.599+0000 7ff178b7db80 4 client.15984 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-23T17:52:10.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.599+0000 7ff178b7db80 1 -- 172.21.15.86:0/1341693199 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f240068cf0 con 0x55f240335a30 2024-03-23T17:52:10.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.600+0000 7ff15de00700 1 --2- 172.21.15.86:0/1341693199 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55f240335a30 0x55f240337e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:10.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.600+0000 7ff15de00700 1 --2- 172.21.15.86:0/1341693199 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55f240335a30 0x55f240337e50 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf8812259ea1d4b4 server_cookie=1d7f960c50aa2506 in_seq=0 out_seq=0 2024-03-23T17:52:10.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.600+0000 7ff13fe00700 10 client.15984 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:52:10.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.600+0000 7ff13fe00700 1 -- 172.21.15.86:0/1341693199 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55f240068cf0 con 0x55f240335a30 2024-03-23T17:52:10.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.600+0000 7ff13fe00700 10 client.15984 handle_command_reply: tid=0 2024-03-23T17:52:10.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.600+0000 7ff178b7db80 1 client.15984 shutdown 2024-03-23T17:52:10.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.600+0000 7ff178b7db80 1 -- 172.21.15.86:0/1341693199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2402d2df0 msgr2=0x55f2402d7fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:10.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.600+0000 7ff178b7db80 1 --2- 172.21.15.86:0/1341693199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2402d2df0 0x55f2402d7fb0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7ff1580124c0 tx=0x7ff158004ad0 comp rx=0 tx=0).stop 2024-03-23T17:52:10.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.601+0000 7ff178b7db80 1 -- 172.21.15.86:0/1341693199 shutdown_connections 2024-03-23T17:52:10.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.601+0000 7ff178b7db80 1 --2- 172.21.15.86:0/1341693199 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55f240335a30 0x55f240337e50 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:10.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.601+0000 7ff178b7db80 1 --2- 172.21.15.86:0/1341693199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2402d2df0 0x55f2402d7fb0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:10.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.601+0000 7ff178b7db80 1 --2- 172.21.15.86:0/1341693199 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f2402c75d0 0x55f2402d95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:10.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.601+0000 7ff13fe00700 0 client.15984 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:52:10.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.601+0000 7ff178b7db80 1 --2- 172.21.15.86:0/1341693199 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f2402c7ef0 0x55f2402d7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:10.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.601+0000 7ff178b7db80 1 -- 172.21.15.86:0/1341693199 >> 172.21.15.86:0/1341693199 conn(0x55f2402276a0 msgr2=0x55f2402c9540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:10.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.601+0000 7ff178b7db80 1 -- 172.21.15.86:0/1341693199 shutdown_connections 2024-03-23T17:52:10.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.601+0000 7ff178b7db80 1 -- 172.21.15.86:0/1341693199 wait complete. 2024-03-23T17:52:10.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.601+0000 7ff178b7db80 20 client.15984 trim_cache size 0 max 16384 2024-03-23T17:52:10.607 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-23T17:52:10.607 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-23T17:52:10.607 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-03-23T17:52:10.607 INFO:teuthology.orchestra.run.smithi086.stdout: "5ee6a087-503e-4da8-8a76-c398aee6e05e": { 2024-03-23T17:52:10.607 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-03-23T17:52:10.607 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "5ee6a087-503e-4da8-8a76-c398aee6e05e", 2024-03-23T17:52:10.607 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-03-23T17:52:10.608 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T17:52:10.608 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T17:52:10.608 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-23T17:52:10.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.601+0000 7ff165000700 1 -- 172.21.15.86:0/591045006 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7ff1440a13b0 msgr2=0x7ff1440a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:10.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.601+0000 7ff165000700 1 --2- 172.21.15.86:0/591045006 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7ff1440a13b0 0x7ff1440a3870 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff158007900 tx=0x7ff158009c70 comp rx=0 tx=0).stop 2024-03-23T17:52:10.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.601+0000 7ff165000700 1 -- 172.21.15.86:0/591045006 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1600731e0 msgr2=0x7ff160199440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:10.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.601+0000 7ff165000700 1 --2- 172.21.15.86:0/591045006 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1600731e0 0x7ff160199440 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff15000e9a0 tx=0x7ff15000ecb0 comp rx=0 tx=0).stop 2024-03-23T17:52:10.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.602+0000 7ff165000700 1 -- 172.21.15.86:0/591045006 shutdown_connections 2024-03-23T17:52:10.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.602+0000 7ff165000700 1 --2- 172.21.15.86:0/591045006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff16010d530 0x7ff16019dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:10.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.602+0000 7ff165000700 1 --2- 172.21.15.86:0/591045006 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1600731e0 0x7ff160199440 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:10.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.602+0000 7ff165000700 1 --2- 172.21.15.86:0/591045006 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7ff1440a13b0 0x7ff1440a3870 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:10.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.602+0000 7ff165000700 1 --2- 172.21.15.86:0/591045006 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff160073b00 0x7ff160199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:10.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.602+0000 7ff165000700 1 -- 172.21.15.86:0/591045006 >> 172.21.15.86:0/591045006 conn(0x7ff1600fc7c0 msgr2=0x7ff1600fe0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:10.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.602+0000 7ff165000700 1 -- 172.21.15.86:0/591045006 shutdown_connections 2024-03-23T17:52:10.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:10.602+0000 7ff165000700 1 -- 172.21.15.86:0/591045006 wait complete. 2024-03-23T17:52:10.621 INFO:tasks.cephfs.filesystem:scrub status for tag:5ee6a087-503e-4da8-8a76-c398aee6e05e - {'path': '/', 'tag': '5ee6a087-503e-4da8-8a76-c398aee6e05e', 'options': 'recursive,force'} 2024-03-23T17:52:10.807 INFO:journalctl@ceph.mds.h.smithi092.stdout:Mar 23 17:52:10 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-h[114780]: 2024-03-23T17:52:10.352+0000 7fe1dcffe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000222 [3,head] /volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/WORDPRO/BENCHS.LWP auth v729 ap=1 s=260096 n(v0 rc2024-03-23T17:51:36.039630+0000 b260096 1=1+0) (iversion lock) caps={25258=pAsLsXsFscr/-@2},l=25258(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5605647d8100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000222:[<0x10000000221/BENCHS.LWP v2>,<0x10000000009/WORDPRO v5358>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v37>,<0x10000000006/clients v25>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v28>,<0x100000001fa/aa7fd127-71c3-47dd-b16f-38c26b2c15fc v44>,<0x10000000001/sv_1 v65>,<0x10000000000/qa v54>,<0x1/volumes v50>]//[]","memoryvalue":"(3)0x10000000222:[<0x10000000221/BENCHS.LWP v729>,<0x10000000009/WORDPRO v5536>,<0x10000000008/~dmtmp v206>,<0x10000000007/client0 v79>,<0x10000000006/clients v57>,<0x10000000005/tmp v57>,<0x100000001fb/client.0 v60>,<0x100000001fa/aa7fd127-71c3-47dd-b16f-38c26b2c15fc v74>,<0x10000000001/sv_1 v81>,<0x10000000000/qa v42>,<0x1/volumes v74>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-23T17:52:10.807 INFO:journalctl@ceph.mds.h.smithi092.stdout:Mar 23 17:52:10 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-h[114780]: 2024-03-23T17:52:10.352+0000 7fe1dcffe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000224 [3,head] /volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/WORDPRO/NEWS1_1.LWP auth v725 ap=1 s=260096 n(v0 rc2024-03-23T17:51:36.049630+0000 b260096 1=1+0) (iversion lock) caps={25258=pAsLsXsFscr/-@2},l=25258(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5605647ec000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000224:[<0x10000000221/NEWS1_1.LWP v6>,<0x10000000009/WORDPRO v5358>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v37>,<0x10000000006/clients v25>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v28>,<0x100000001fa/aa7fd127-71c3-47dd-b16f-38c26b2c15fc v44>,<0x10000000001/sv_1 v65>,<0x10000000000/qa v54>,<0x1/volumes v50>]//[]","memoryvalue":"(3)0x10000000224:[<0x10000000221/NEWS1_1.LWP v725>,<0x10000000009/WORDPRO v5536>,<0x10000000008/~dmtmp v206>,<0x10000000007/client0 v79>,<0x10000000006/clients v57>,<0x10000000005/tmp v57>,<0x100000001fb/client.0 v60>,<0x100000001fa/aa7fd127-71c3-47dd-b16f-38c26b2c15fc v74>,<0x10000000001/sv_1 v81>,<0x10000000000/qa v42>,<0x1/volumes v74>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-23T17:52:10.808 INFO:journalctl@ceph.mds.h.smithi092.stdout:Mar 23 17:52:10 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-h[114780]: 2024-03-23T17:52:10.353+0000 7fe1dcffe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000223 [3,head] /volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/WORDPRO/RESULTS.XLS auth v727 ap=1 s=260096 n(v0 rc2024-03-23T17:51:36.044630+0000 b260096 1=1+0) (iversion lock) caps={25258=pAsLsXsFscr/-@2},l=25258(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5605647d8680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000223:[<0x10000000221/RESULTS.XLS v4>,<0x10000000009/WORDPRO v5358>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v37>,<0x10000000006/clients v25>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v28>,<0x100000001fa/aa7fd127-71c3-47dd-b16f-38c26b2c15fc v44>,<0x10000000001/sv_1 v65>,<0x10000000000/qa v54>,<0x1/volumes v50>]//[]","memoryvalue":"(3)0x10000000223:[<0x10000000221/RESULTS.XLS v727>,<0x10000000009/WORDPRO v5536>,<0x10000000008/~dmtmp v206>,<0x10000000007/client0 v79>,<0x10000000006/clients v57>,<0x10000000005/tmp v57>,<0x100000001fb/client.0 v60>,<0x100000001fa/aa7fd127-71c3-47dd-b16f-38c26b2c15fc v74>,<0x10000000001/sv_1 v81>,<0x10000000000/qa v42>,<0x1/volumes v74>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-23T17:52:10.808 INFO:journalctl@ceph.mds.h.smithi092.stdout:Mar 23 17:52:10 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-h[114780]: 2024-03-23T17:52:10.365+0000 7fe1dcffe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000210 [4,head] /volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/PWRPNT/PCBENCHM.PPT auth v948 ap=1 s=260096 n(v0 rc2024-03-23T17:52:08.921927+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25258=0-4194304@2} caps={25258=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@8},l=25258 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x560564774c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000210:[<0x1000000020f/PCBENCHM.PPT v2>,<0x10000000009/PWRPNT v5064>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v37>,<0x10000000006/clients v25>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v28>,<0x100000001fa/aa7fd127-71c3-47dd-b16f-38c26b2c15fc v44>,<0x10000000001/sv_1 v65>,<0x10000000000/qa v54>,<0x1/volumes v50>]//[]","memoryvalue":"(3)0x10000000210:[<0x1000000020f/PCBENCHM.PPT v948>,<0x10000000009/PWRPNT v5584>,<0x10000000008/~dmtmp v206>,<0x10000000007/client0 v79>,<0x10000000006/clients v57>,<0x10000000005/tmp v57>,<0x100000001fb/client.0 v60>,<0x100000001fa/aa7fd127-71c3-47dd-b16f-38c26b2c15fc v74>,<0x10000000001/sv_1 v81>,<0x10000000000/qa v42>,<0x1/volumes v74>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-23T17:52:10.808 INFO:journalctl@ceph.mds.h.smithi092.stdout:Mar 23 17:52:10 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-h[114780]: 2024-03-23T17:52:10.366+0000 7fe1dcffe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000212 [3,head] /volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/PWRPNT/PPTOOLS1.PPA auth{0=1} v930 ap=1 s=260096 n(v0 rc2024-03-23T17:51:35.967632+0000 b260096 1=1+0) (iversion lock) caps={25258=pAsLsXsFscr/-@3},l=25258(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5605647d6b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000212:[<0x1000000020f/PPTOOLS1.PPA v6>,<0x10000000009/PWRPNT v5064>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v37>,<0x10000000006/clients v25>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v28>,<0x100000001fa/aa7fd127-71c3-47dd-b16f-38c26b2c15fc v44>,<0x10000000001/sv_1 v65>,<0x10000000000/qa v54>,<0x1/volumes v50>]//[]","memoryvalue":"(3)0x10000000212:[<0x1000000020f/PPTOOLS1.PPA v930>,<0x10000000009/PWRPNT v5584>,<0x10000000008/~dmtmp v206>,<0x10000000007/client0 v79>,<0x10000000006/clients v57>,<0x10000000005/tmp v57>,<0x100000001fb/client.0 v60>,<0x100000001fa/aa7fd127-71c3-47dd-b16f-38c26b2c15fc v74>,<0x10000000001/sv_1 v81>,<0x10000000000/qa v42>,<0x1/volumes v74>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-23T17:52:10.808 INFO:journalctl@ceph.mds.h.smithi092.stdout:Mar 23 17:52:10 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-h[114780]: 2024-03-23T17:52:10.366+0000 7fe1dcffe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000211 [3,head] /volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/PWRPNT/ZD16.BMP auth v932 ap=1 s=260096 n(v0 rc2024-03-23T17:51:35.962632+0000 b260096 1=1+0) (iversion lock) caps={25258=pAsLsXsFscr/-@3},l=25258(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x560564775180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000211:[<0x1000000020f/ZD16.BMP v4>,<0x10000000009/PWRPNT v5064>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v37>,<0x10000000006/clients v25>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v28>,<0x100000001fa/aa7fd127-71c3-47dd-b16f-38c26b2c15fc v44>,<0x10000000001/sv_1 v65>,<0x10000000000/qa v54>,<0x1/volumes v50>]//[]","memoryvalue":"(3)0x10000000211:[<0x1000000020f/ZD16.BMP v932>,<0x10000000009/PWRPNT v5584>,<0x10000000008/~dmtmp v206>,<0x10000000007/client0 v79>,<0x10000000006/clients v57>,<0x10000000005/tmp v57>,<0x100000001fb/client.0 v60>,<0x100000001fa/aa7fd127-71c3-47dd-b16f-38c26b2c15fc v74>,<0x10000000001/sv_1 v81>,<0x10000000000/qa v42>,<0x1/volumes v74>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-23T17:52:10.808 INFO:journalctl@ceph.mds.h.smithi092.stdout:Mar 23 17:52:10 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-h[114780]: 2024-03-23T17:52:10.366+0000 7fe1dcffe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000213 [3,head] /volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/PWRPNT/TIPS.PPT auth v928 ap=1 s=260096 n(v0 rc2024-03-23T17:51:44.876441+0000 b260096 1=1+0) (iversion lock) caps={25258=pAsLsXsFscr/-@3},l=25258(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5605647d6580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000213:[<0x1000000020f/TIPS.PPT v8>,<0x10000000009/PWRPNT v5064>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v37>,<0x10000000006/clients v25>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v28>,<0x100000001fa/aa7fd127-71c3-47dd-b16f-38c26b2c15fc v44>,<0x10000000001/sv_1 v65>,<0x10000000000/qa v54>,<0x1/volumes v50>]//[]","memoryvalue":"(3)0x10000000213:[<0x1000000020f/TIPS.PPT v928>,<0x10000000009/PWRPNT v5584>,<0x10000000008/~dmtmp v206>,<0x10000000007/client0 v79>,<0x10000000006/clients v57>,<0x10000000005/tmp v57>,<0x100000001fb/client.0 v60>,<0x100000001fa/aa7fd127-71c3-47dd-b16f-38c26b2c15fc v74>,<0x10000000001/sv_1 v81>,<0x10000000000/qa v42>,<0x1/volumes v74>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-23T17:52:10.808 INFO:journalctl@ceph.mds.h.smithi092.stdout:Mar 23 17:52:10 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-h[114780]: 2024-03-23T17:52:10.366+0000 7fe1dcffe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000214 [3,head] /volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/PWRPNT/TRIDOTS.POT auth v926 ap=1 s=260096 n(v0 rc2024-03-23T17:51:35.978631+0000 b260096 1=1+0) (iversion lock) caps={25258=pAsLsXsFscr/-@3},l=25258(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5605647d6000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000214:[<0x1000000020f/TRIDOTS.POT v10>,<0x10000000009/PWRPNT v5064>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v37>,<0x10000000006/clients v25>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v28>,<0x100000001fa/aa7fd127-71c3-47dd-b16f-38c26b2c15fc v44>,<0x10000000001/sv_1 v65>,<0x10000000000/qa v54>,<0x1/volumes v50>]//[]","memoryvalue":"(3)0x10000000214:[<0x1000000020f/TRIDOTS.POT v926>,<0x10000000009/PWRPNT v5584>,<0x10000000008/~dmtmp v206>,<0x10000000007/client0 v79>,<0x10000000006/clients v57>,<0x10000000005/tmp v57>,<0x100000001fb/client.0 v60>,<0x100000001fa/aa7fd127-71c3-47dd-b16f-38c26b2c15fc v74>,<0x10000000001/sv_1 v81>,<0x10000000000/qa v42>,<0x1/volumes v74>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-23T17:52:10.809 INFO:journalctl@ceph.mds.e.smithi092.stdout:Mar 23 17:52:10 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-e[114438]: 2024-03-23T17:52:10.352+0000 7fa521878700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000241 [3,head] /volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/SEED/SMALL.FIL auth v14 ap=1 s=327680 n(v0 rc2024-03-23T17:51:45.568426+0000 b327680 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25258=0-4194304@2} caps={25258=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@6},l=25258 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55dee7506580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000241:[<0x10000000240/SMALL.FIL v2>,<0x10000000009/SEED v120>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v37>,<0x10000000006/clients v25>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v28>,<0x100000001fa/aa7fd127-71c3-47dd-b16f-38c26b2c15fc v44>,<0x10000000001/sv_1 v65>,<0x10000000000/qa v54>,<0x1/volumes v50>]//[]","memoryvalue":"(3)0x10000000241:[<0x10000000240/SMALL.FIL v14>,<0x10000000009/SEED v5530>,<0x10000000008/~dmtmp v206>,<0x10000000007/client0 v79>,<0x10000000006/clients v57>,<0x10000000005/tmp v57>,<0x100000001fb/client.0 v60>,<0x100000001fa/aa7fd127-71c3-47dd-b16f-38c26b2c15fc v74>,<0x10000000001/sv_1 v81>,<0x10000000000/qa v27>,<0x1/volumes v74>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-23T17:52:10.809 INFO:journalctl@ceph.mds.e.smithi092.stdout:Mar 23 17:52:10 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-e[114438]: 2024-03-23T17:52:10.353+0000 7fa521878700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000242 [3,head] /volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/SEED/MEDIUM.FIL auth v16 ap=1 s=1048576 n(v0 rc2024-03-23T17:51:45.568426+0000 b1048576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25258=0-4194304@2} caps={25258=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@6},l=25258 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55dee7509180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000242:[<0x10000000240/MEDIUM.FIL v4>,<0x10000000009/SEED v120>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v37>,<0x10000000006/clients v25>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v28>,<0x100000001fa/aa7fd127-71c3-47dd-b16f-38c26b2c15fc v44>,<0x10000000001/sv_1 v65>,<0x10000000000/qa v54>,<0x1/volumes v50>]//[]","memoryvalue":"(3)0x10000000242:[<0x10000000240/MEDIUM.FIL v16>,<0x10000000009/SEED v5530>,<0x10000000008/~dmtmp v206>,<0x10000000007/client0 v79>,<0x10000000006/clients v57>,<0x10000000005/tmp v57>,<0x100000001fb/client.0 v60>,<0x100000001fa/aa7fd127-71c3-47dd-b16f-38c26b2c15fc v74>,<0x10000000001/sv_1 v81>,<0x10000000000/qa v27>,<0x1/volumes v74>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-23T17:52:10.809 INFO:journalctl@ceph.mds.e.smithi092.stdout:Mar 23 17:52:10 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-e[114438]: 2024-03-23T17:52:10.353+0000 7fa521878700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000243 [3,head] /volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/SEED/LARGE.FIL auth v18 ap=1 s=1867776 n(v0 rc2024-03-23T17:51:45.568426+0000 b1867776 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25258=0-4194304@2} caps={25258=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@6},l=25258 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55dee7509700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000243:[<0x10000000240/LARGE.FIL v6>,<0x10000000009/SEED v120>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v37>,<0x10000000006/clients v25>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v28>,<0x100000001fa/aa7fd127-71c3-47dd-b16f-38c26b2c15fc v44>,<0x10000000001/sv_1 v65>,<0x10000000000/qa v54>,<0x1/volumes v50>]//[]","memoryvalue":"(3)0x10000000243:[<0x10000000240/LARGE.FIL v18>,<0x10000000009/SEED v5530>,<0x10000000008/~dmtmp v206>,<0x10000000007/client0 v79>,<0x10000000006/clients v57>,<0x10000000005/tmp v57>,<0x100000001fb/client.0 v60>,<0x100000001fa/aa7fd127-71c3-47dd-b16f-38c26b2c15fc v74>,<0x10000000001/sv_1 v81>,<0x10000000000/qa v27>,<0x1/volumes v74>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-23T17:52:10.809 INFO:journalctl@ceph.mds.e.smithi092.stdout:Mar 23 17:52:10 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-e[114438]: 2024-03-23T17:52:10.367+0000 7fa521878700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005ef [4,head] ~mds4/stray1/100000005ef auth v292 pv293 ap=4 s=62591 n(v0 rc2024-03-23T17:51:45.592426+0000 b62591 1=1+0)->n(v0 rc2024-03-23T17:52:08.894928+0000 b62591 1=1+0) (ilink xlockdone x=1) (iversion lock w=1 last_client=25258) caps={25258=pFc/Fc@2} | ptrwaiter=0 request=1 lock=3 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55dee7506b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000005ef:[<0x1000000023e/INV.PRN v272>,<0x10000000009/ACCESS v5388>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v37>,<0x10000000006/clients v25>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v28>,<0x100000001fa/aa7fd127-71c3-47dd-b16f-38c26b2c15fc v44>,<0x10000000001/sv_1 v65>,<0x10000000000/qa v54>,<0x1/volumes v50>]//[]","memoryvalue":"(3)0x100000005ef:[<0x1000000023e/INV.PRN v292>,<0x10000000009/ACCESS v5532>,<0x10000000008/~dmtmp v206>,<0x10000000007/client0 v79>,<0x10000000006/clients v57>,<0x10000000005/tmp v57>,<0x100000001fb/client.0 v60>,<0x100000001fa/aa7fd127-71c3-47dd-b16f-38c26b2c15fc v74>,<0x10000000001/sv_1 v81>,<0x10000000000/qa v27>,<0x1/volumes v74>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-23T17:52:10.809 INFO:journalctl@ceph.mds.e.smithi092.stdout:Mar 23 17:52:10 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-e[114438]: 2024-03-23T17:52:10.367+0000 7fa521878700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000023f [3,head] /volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/ACCESS/FASTENER.MDB auth v288 ap=1 s=260096 n(v0 rc2024-03-23T17:51:45.614425+0000 b260096 1=1+0) (iversion lock) caps={25258=pAsLsXsFscr/-@2},l=25258(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55dee741db80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000023f:[<0x1000000023e/FASTENER.MDB v266>,<0x10000000009/ACCESS v5388>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v37>,<0x10000000006/clients v25>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v28>,<0x100000001fa/aa7fd127-71c3-47dd-b16f-38c26b2c15fc v44>,<0x10000000001/sv_1 v65>,<0x10000000000/qa v54>,<0x1/volumes v50>]//[]","memoryvalue":"(3)0x1000000023f:[<0x1000000023e/FASTENER.MDB v288>,<0x10000000009/ACCESS v5532>,<0x10000000008/~dmtmp v206>,<0x10000000007/client0 v79>,<0x10000000006/clients v57>,<0x10000000005/tmp v57>,<0x100000001fb/client.0 v60>,<0x100000001fa/aa7fd127-71c3-47dd-b16f-38c26b2c15fc v74>,<0x10000000001/sv_1 v81>,<0x10000000000/qa v27>,<0x1/volumes v74>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-23T17:52:10.809 INFO:journalctl@ceph.mds.e.smithi092.stdout:Mar 23 17:52:10 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-e[114438]: 2024-03-23T17:52:10.367+0000 7fa521878700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005f1 [4,head] ~mds4/stray1/100000005f1 auth v296 pv297 ap=4 s=10845 n(v0 rc2024-03-23T17:51:45.603426+0000 b10845 1=1+0)->n(v0 rc2024-03-23T17:52:08.897928+0000 b10845 1=1+0) (ilink xlockdone x=1) (iversion lock w=1 last_client=25258) caps={25258=pFc/Fc@3} | ptrwaiter=0 request=1 lock=3 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55dee741f180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000005f1:[<0x1000000023e/SALES.PRN v278>,<0x10000000009/ACCESS v5388>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v37>,<0x10000000006/clients v25>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v28>,<0x100000001fa/aa7fd127-71c3-47dd-b16f-38c26b2c15fc v44>,<0x10000000001/sv_1 v65>,<0x10000000000/qa v54>,<0x1/volumes v50>]//[]","memoryvalue":"(3)0x100000005f1:[<0x1000000023e/SALES.PRN v296>,<0x10000000009/ACCESS v5532>,<0x10000000008/~dmtmp v206>,<0x10000000007/client0 v79>,<0x10000000006/clients v57>,<0x10000000005/tmp v57>,<0x100000001fb/client.0 v60>,<0x100000001fa/aa7fd127-71c3-47dd-b16f-38c26b2c15fc v74>,<0x10000000001/sv_1 v81>,<0x10000000000/qa v27>,<0x1/volumes v74>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-23T17:52:10.809 INFO:journalctl@ceph.mds.e.smithi092.stdout:Mar 23 17:52:10 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-e[114438]: 2024-03-23T17:52:10.368+0000 7fa521878700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005f3 [4,head] ~mds4/stray1/100000005f3 auth v300 pv301 ap=4 s=30118 n(v0 rc2024-03-23T17:51:45.614425+0000 b30118 1=1+0)->n(v0 rc2024-03-23T17:52:08.900928+0000 b30118 1=1+0) (ilink xlockdone x=1) (iversion lock w=1 last_client=25258) caps={25258=pFc/Fc@3} | ptrwaiter=0 request=1 lock=3 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55dee741f700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000005f3:[<0x1000000023e/LABELS.PRN v284>,<0x10000000009/ACCESS v5388>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v37>,<0x10000000006/clients v25>,<0x10000000005/tmp v25>,<0x100000001fb/client.0 v28>,<0x100000001fa/aa7fd127-71c3-47dd-b16f-38c26b2c15fc v44>,<0x10000000001/sv_1 v65>,<0x10000000000/qa v54>,<0x1/volumes v50>]//[]","memoryvalue":"(3)0x100000005f3:[<0x1000000023e/LABELS.PRN v300>,<0x10000000009/ACCESS v5532>,<0x10000000008/~dmtmp v206>,<0x10000000007/client0 v79>,<0x10000000006/clients v57>,<0x10000000005/tmp v57>,<0x100000001fb/client.0 v60>,<0x100000001fa/aa7fd127-71c3-47dd-b16f-38c26b2c15fc v74>,<0x10000000001/sv_1 v81>,<0x10000000000/qa v27>,<0x1/volumes v74>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-23T17:52:10.881 INFO:tasks.workunit.client.0.smithi086.stdout: 1 29138 4.98 MB/sec warmup 35 sec latency 1441.316 ms 2024-03-23T17:52:11.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:10 smithi152 ceph-mon[47949]: scrub queued for path: / 2024-03-23T17:52:11.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:10 smithi152 ceph-mon[47949]: scrub summary: idle+waiting paths [/] 2024-03-23T17:52:11.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:10 smithi152 ceph-mon[47949]: scrub summary: active paths [/] 2024-03-23T17:52:11.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:10 smithi152 ceph-mon[47949]: scrub summary: active paths [/] 2024-03-23T17:52:11.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:10 smithi152 ceph-mon[47949]: scrub summary: active paths [/] 2024-03-23T17:52:11.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:10 smithi152 ceph-mon[47949]: Scrub error on inode 0x10000000241 (/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/SEED/SMALL.FIL) see mds.e log and `damage ls` output for details 2024-03-23T17:52:11.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:10 smithi152 ceph-mon[47949]: Scrub error on inode 0x10000000242 (/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/SEED/MEDIUM.FIL) see mds.e log and `damage ls` output for details 2024-03-23T17:52:11.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:10 smithi152 ceph-mon[47949]: Scrub error on inode 0x10000000243 (/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/SEED/LARGE.FIL) see mds.e log and `damage ls` output for details 2024-03-23T17:52:11.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:10 smithi152 ceph-mon[47949]: scrub summary: idle+waiting paths [/] 2024-03-23T17:52:11.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:10 smithi152 ceph-mon[47949]: scrub summary: idle+waiting paths [/] 2024-03-23T17:52:11.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:10 smithi152 ceph-mon[47949]: scrub summary: active paths [/] 2024-03-23T17:52:11.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:10 smithi152 ceph-mon[47949]: scrub summary: idle+waiting paths [/] 2024-03-23T17:52:11.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:10 smithi152 ceph-mon[47949]: scrub summary: active paths [/] 2024-03-23T17:52:11.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:10 smithi152 ceph-mon[47949]: Scrub error on inode 0x100000005ef (~mds4/stray1/100000005ef) see mds.e log and `damage ls` output for details 2024-03-23T17:52:11.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:10 smithi152 ceph-mon[47949]: Scrub error on inode 0x1000000023f (/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/ACCESS/FASTENER.MDB) see mds.e log and `damage ls` output for details 2024-03-23T17:52:11.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:10 smithi152 ceph-mon[47949]: Scrub error on inode 0x100000005f1 (~mds4/stray1/100000005f1) see mds.e log and `damage ls` output for details 2024-03-23T17:52:11.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:10 smithi152 ceph-mon[47949]: Scrub error on inode 0x100000005f3 (~mds4/stray1/100000005f3) see mds.e log and `damage ls` output for details 2024-03-23T17:52:11.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:10 smithi152 ceph-mon[47949]: scrub summary: idle+waiting paths [/] 2024-03-23T17:52:11.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:10 smithi152 ceph-mon[47949]: scrub summary: idle+waiting paths [/] 2024-03-23T17:52:11.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:10 smithi086 ceph-mon[38221]: scrub queued for path: / 2024-03-23T17:52:11.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:10 smithi086 ceph-mon[38221]: scrub summary: idle+waiting paths [/] 2024-03-23T17:52:11.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:10 smithi086 ceph-mon[38221]: scrub summary: active paths [/] 2024-03-23T17:52:11.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:10 smithi086 ceph-mon[38221]: scrub summary: active paths [/] 2024-03-23T17:52:11.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:10 smithi086 ceph-mon[38221]: scrub summary: active paths [/] 2024-03-23T17:52:11.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:10 smithi086 ceph-mon[38221]: Scrub error on inode 0x10000000241 (/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/SEED/SMALL.FIL) see mds.e log and `damage ls` output for details 2024-03-23T17:52:11.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:10 smithi086 ceph-mon[38221]: Scrub error on inode 0x10000000242 (/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/SEED/MEDIUM.FIL) see mds.e log and `damage ls` output for details 2024-03-23T17:52:11.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:10 smithi086 ceph-mon[38221]: Scrub error on inode 0x10000000243 (/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/SEED/LARGE.FIL) see mds.e log and `damage ls` output for details 2024-03-23T17:52:11.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:10 smithi086 ceph-mon[38221]: scrub summary: idle+waiting paths [/] 2024-03-23T17:52:11.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:10 smithi086 ceph-mon[38221]: scrub summary: idle+waiting paths [/] 2024-03-23T17:52:11.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:10 smithi086 ceph-mon[38221]: scrub summary: active paths [/] 2024-03-23T17:52:11.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:10 smithi086 ceph-mon[38221]: scrub summary: idle+waiting paths [/] 2024-03-23T17:52:11.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:10 smithi086 ceph-mon[38221]: scrub summary: active paths [/] 2024-03-23T17:52:11.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:10 smithi086 ceph-mon[38221]: Scrub error on inode 0x100000005ef (~mds4/stray1/100000005ef) see mds.e log and `damage ls` output for details 2024-03-23T17:52:11.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:10 smithi086 ceph-mon[38221]: Scrub error on inode 0x1000000023f (/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/ACCESS/FASTENER.MDB) see mds.e log and `damage ls` output for details 2024-03-23T17:52:11.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:10 smithi086 ceph-mon[38221]: Scrub error on inode 0x100000005f1 (~mds4/stray1/100000005f1) see mds.e log and `damage ls` output for details 2024-03-23T17:52:11.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:10 smithi086 ceph-mon[38221]: Scrub error on inode 0x100000005f3 (~mds4/stray1/100000005f3) see mds.e log and `damage ls` output for details 2024-03-23T17:52:11.268 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:10 smithi086 ceph-mon[38221]: scrub summary: idle+waiting paths [/] 2024-03-23T17:52:11.268 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:10 smithi086 ceph-mon[38221]: scrub summary: idle+waiting paths [/] 2024-03-23T17:52:11.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:10 smithi092 ceph-mon[79089]: scrub queued for path: / 2024-03-23T17:52:11.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:10 smithi092 ceph-mon[79089]: scrub summary: idle+waiting paths [/] 2024-03-23T17:52:11.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:10 smithi092 ceph-mon[79089]: scrub summary: active paths [/] 2024-03-23T17:52:11.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:10 smithi092 ceph-mon[79089]: scrub summary: active paths [/] 2024-03-23T17:52:11.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:10 smithi092 ceph-mon[79089]: scrub summary: active paths [/] 2024-03-23T17:52:11.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:10 smithi092 ceph-mon[79089]: Scrub error on inode 0x10000000241 (/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/SEED/SMALL.FIL) see mds.e log and `damage ls` output for details 2024-03-23T17:52:11.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:10 smithi092 ceph-mon[79089]: Scrub error on inode 0x10000000242 (/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/SEED/MEDIUM.FIL) see mds.e log and `damage ls` output for details 2024-03-23T17:52:11.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:10 smithi092 ceph-mon[79089]: Scrub error on inode 0x10000000243 (/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/SEED/LARGE.FIL) see mds.e log and `damage ls` output for details 2024-03-23T17:52:11.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:10 smithi092 ceph-mon[79089]: scrub summary: idle+waiting paths [/] 2024-03-23T17:52:11.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:10 smithi092 ceph-mon[79089]: scrub summary: idle+waiting paths [/] 2024-03-23T17:52:11.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:10 smithi092 ceph-mon[79089]: scrub summary: active paths [/] 2024-03-23T17:52:11.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:10 smithi092 ceph-mon[79089]: scrub summary: idle+waiting paths [/] 2024-03-23T17:52:11.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:10 smithi092 ceph-mon[79089]: scrub summary: active paths [/] 2024-03-23T17:52:11.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:10 smithi092 ceph-mon[79089]: Scrub error on inode 0x100000005ef (~mds4/stray1/100000005ef) see mds.e log and `damage ls` output for details 2024-03-23T17:52:11.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:10 smithi092 ceph-mon[79089]: Scrub error on inode 0x1000000023f (/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/ACCESS/FASTENER.MDB) see mds.e log and `damage ls` output for details 2024-03-23T17:52:11.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:10 smithi092 ceph-mon[79089]: Scrub error on inode 0x100000005f1 (~mds4/stray1/100000005f1) see mds.e log and `damage ls` output for details 2024-03-23T17:52:11.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:10 smithi092 ceph-mon[79089]: Scrub error on inode 0x100000005f3 (~mds4/stray1/100000005f3) see mds.e log and `damage ls` output for details 2024-03-23T17:52:11.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:10 smithi092 ceph-mon[79089]: scrub summary: idle+waiting paths [/] 2024-03-23T17:52:11.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:10 smithi092 ceph-mon[79089]: scrub summary: idle+waiting paths [/] 2024-03-23T17:52:11.881 INFO:tasks.workunit.client.0.smithi086.stdout: 1 29139 4.84 MB/sec warmup 36 sec latency 977.346 ms 2024-03-23T17:52:12.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:11 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mon-a[38198]: 2024-03-23T17:52:11.699+0000 7f70a6200700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-03-23T17:52:12.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:11 smithi086 ceph-mon[38221]: scrub summary: active paths [/] 2024-03-23T17:52:12.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:11 smithi086 ceph-mon[38221]: scrub summary: active paths [/] 2024-03-23T17:52:12.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:11 smithi086 ceph-mon[38221]: Scrub error on inode 0x10000000222 (/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/WORDPRO/BENCHS.LWP) see mds.h log and `damage ls` output for details 2024-03-23T17:52:12.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:11 smithi086 ceph-mon[38221]: Scrub error on inode 0x1000000020d (/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/EXCEL/PCMAGCD.XLS) see mds.d log and `damage ls` output for details 2024-03-23T17:52:12.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:11 smithi086 ceph-mon[38221]: Scrub error on inode 0x10000000224 (/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/WORDPRO/NEWS1_1.LWP) see mds.h log and `damage ls` output for details 2024-03-23T17:52:12.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:11 smithi086 ceph-mon[38221]: Scrub error on inode 0x10000000223 (/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/WORDPRO/RESULTS.XLS) see mds.h log and `damage ls` output for details 2024-03-23T17:52:12.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:11 smithi086 ceph-mon[38221]: scrub summary: idle+waiting paths [/] 2024-03-23T17:52:12.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:11 smithi086 ceph-mon[38221]: Scrub error on inode 0x10000000209 (/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/WORD/CHAP10.DOC) see mds.d log and `damage ls` output for details 2024-03-23T17:52:12.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:11 smithi086 ceph-mon[38221]: Scrub error on inode 0x1000000020b (/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/WORD/FACTS.DOC) see mds.d log and `damage ls` output for details 2024-03-23T17:52:12.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:11 smithi086 ceph-mon[38221]: Scrub error on inode 0x1000000020e (/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/EXCEL/SALES.XLS) see mds.d log and `damage ls` output for details 2024-03-23T17:52:12.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:11 smithi086 ceph-mon[38221]: Scrub error on inode 0x1000000020a (/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/WORD/BASEMACH.DOC) see mds.d log and `damage ls` output for details 2024-03-23T17:52:12.018 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:11 smithi086 ceph-mon[38221]: scrub summary: idle+waiting paths [/] 2024-03-23T17:52:12.018 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:11 smithi086 ceph-mon[38221]: scrub summary: active paths [/] 2024-03-23T17:52:12.018 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:11 smithi086 ceph-mon[38221]: Scrub error on inode 0x10000000210 (/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/PWRPNT/PCBENCHM.PPT) see mds.h log and `damage ls` output for details 2024-03-23T17:52:12.018 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:11 smithi086 ceph-mon[38221]: Scrub error on inode 0x10000000212 (/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/PWRPNT/PPTOOLS1.PPA) see mds.h log and `damage ls` output for details 2024-03-23T17:52:12.018 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:11 smithi086 ceph-mon[38221]: Scrub error on inode 0x10000000211 (/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/PWRPNT/ZD16.BMP) see mds.h log and `damage ls` output for details 2024-03-23T17:52:12.018 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:11 smithi086 ceph-mon[38221]: Scrub error on inode 0x10000000213 (/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/PWRPNT/TIPS.PPT) see mds.h log and `damage ls` output for details 2024-03-23T17:52:12.018 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:11 smithi086 ceph-mon[38221]: Scrub error on inode 0x10000000214 (/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/PWRPNT/TRIDOTS.POT) see mds.h log and `damage ls` output for details 2024-03-23T17:52:12.018 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:11 smithi086 ceph-mon[38221]: scrub summary: idle+waiting paths [/] 2024-03-23T17:52:12.018 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:11 smithi086 ceph-mon[38221]: pgmap v49: 97 pgs: 97 active+clean; 26 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 3.6 MiB/s wr, 119 op/s 2024-03-23T17:52:12.018 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:11 smithi086 ceph-mon[38221]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-03-23T17:52:12.018 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:11 smithi086 ceph-mon[38221]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-03-23T17:52:12.018 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:11 smithi086 ceph-mon[38221]: mds.? [v2:172.21.15.92:6834/908900027,v1:172.21.15.92:6835/908900027] up:active 2024-03-23T17:52:12.018 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:11 smithi086 ceph-mon[38221]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T17:52:12.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:11 smithi152 ceph-mon[47949]: scrub summary: active paths [/] 2024-03-23T17:52:12.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:11 smithi152 ceph-mon[47949]: scrub summary: active paths [/] 2024-03-23T17:52:12.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:11 smithi152 ceph-mon[47949]: Scrub error on inode 0x10000000222 (/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/WORDPRO/BENCHS.LWP) see mds.h log and `damage ls` output for details 2024-03-23T17:52:12.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:11 smithi152 ceph-mon[47949]: Scrub error on inode 0x1000000020d (/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/EXCEL/PCMAGCD.XLS) see mds.d log and `damage ls` output for details 2024-03-23T17:52:12.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:11 smithi152 ceph-mon[47949]: Scrub error on inode 0x10000000224 (/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/WORDPRO/NEWS1_1.LWP) see mds.h log and `damage ls` output for details 2024-03-23T17:52:12.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:11 smithi152 ceph-mon[47949]: Scrub error on inode 0x10000000223 (/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/WORDPRO/RESULTS.XLS) see mds.h log and `damage ls` output for details 2024-03-23T17:52:12.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:11 smithi152 ceph-mon[47949]: scrub summary: idle+waiting paths [/] 2024-03-23T17:52:12.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:11 smithi152 ceph-mon[47949]: Scrub error on inode 0x10000000209 (/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/WORD/CHAP10.DOC) see mds.d log and `damage ls` output for details 2024-03-23T17:52:12.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:11 smithi152 ceph-mon[47949]: Scrub error on inode 0x1000000020b (/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/WORD/FACTS.DOC) see mds.d log and `damage ls` output for details 2024-03-23T17:52:12.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:11 smithi152 ceph-mon[47949]: Scrub error on inode 0x1000000020e (/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/EXCEL/SALES.XLS) see mds.d log and `damage ls` output for details 2024-03-23T17:52:12.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:11 smithi152 ceph-mon[47949]: Scrub error on inode 0x1000000020a (/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/WORD/BASEMACH.DOC) see mds.d log and `damage ls` output for details 2024-03-23T17:52:12.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:11 smithi152 ceph-mon[47949]: scrub summary: idle+waiting paths [/] 2024-03-23T17:52:12.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:11 smithi152 ceph-mon[47949]: scrub summary: active paths [/] 2024-03-23T17:52:12.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:11 smithi152 ceph-mon[47949]: Scrub error on inode 0x10000000210 (/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/PWRPNT/PCBENCHM.PPT) see mds.h log and `damage ls` output for details 2024-03-23T17:52:12.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:11 smithi152 ceph-mon[47949]: Scrub error on inode 0x10000000212 (/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/PWRPNT/PPTOOLS1.PPA) see mds.h log and `damage ls` output for details 2024-03-23T17:52:12.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:11 smithi152 ceph-mon[47949]: Scrub error on inode 0x10000000211 (/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/PWRPNT/ZD16.BMP) see mds.h log and `damage ls` output for details 2024-03-23T17:52:12.087 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:11 smithi152 ceph-mon[47949]: Scrub error on inode 0x10000000213 (/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/PWRPNT/TIPS.PPT) see mds.h log and `damage ls` output for details 2024-03-23T17:52:12.088 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:11 smithi152 ceph-mon[47949]: Scrub error on inode 0x10000000214 (/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/PWRPNT/TRIDOTS.POT) see mds.h log and `damage ls` output for details 2024-03-23T17:52:12.088 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:11 smithi152 ceph-mon[47949]: scrub summary: idle+waiting paths [/] 2024-03-23T17:52:12.088 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:11 smithi152 ceph-mon[47949]: pgmap v49: 97 pgs: 97 active+clean; 26 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 3.6 MiB/s wr, 119 op/s 2024-03-23T17:52:12.088 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:11 smithi152 ceph-mon[47949]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-03-23T17:52:12.088 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:11 smithi152 ceph-mon[47949]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-03-23T17:52:12.088 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:11 smithi152 ceph-mon[47949]: mds.? [v2:172.21.15.92:6834/908900027,v1:172.21.15.92:6835/908900027] up:active 2024-03-23T17:52:12.088 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:11 smithi152 ceph-mon[47949]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T17:52:12.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:11 smithi092 ceph-mon[79089]: scrub summary: active paths [/] 2024-03-23T17:52:12.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:11 smithi092 ceph-mon[79089]: scrub summary: active paths [/] 2024-03-23T17:52:12.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:11 smithi092 ceph-mon[79089]: Scrub error on inode 0x10000000222 (/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/WORDPRO/BENCHS.LWP) see mds.h log and `damage ls` output for details 2024-03-23T17:52:12.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:11 smithi092 ceph-mon[79089]: Scrub error on inode 0x1000000020d (/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/EXCEL/PCMAGCD.XLS) see mds.d log and `damage ls` output for details 2024-03-23T17:52:12.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:11 smithi092 ceph-mon[79089]: Scrub error on inode 0x10000000224 (/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/WORDPRO/NEWS1_1.LWP) see mds.h log and `damage ls` output for details 2024-03-23T17:52:12.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:11 smithi092 ceph-mon[79089]: Scrub error on inode 0x10000000223 (/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/WORDPRO/RESULTS.XLS) see mds.h log and `damage ls` output for details 2024-03-23T17:52:12.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:11 smithi092 ceph-mon[79089]: scrub summary: idle+waiting paths [/] 2024-03-23T17:52:12.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:11 smithi092 ceph-mon[79089]: Scrub error on inode 0x10000000209 (/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/WORD/CHAP10.DOC) see mds.d log and `damage ls` output for details 2024-03-23T17:52:12.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:11 smithi092 ceph-mon[79089]: Scrub error on inode 0x1000000020b (/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/WORD/FACTS.DOC) see mds.d log and `damage ls` output for details 2024-03-23T17:52:12.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:11 smithi092 ceph-mon[79089]: Scrub error on inode 0x1000000020e (/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/EXCEL/SALES.XLS) see mds.d log and `damage ls` output for details 2024-03-23T17:52:12.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:11 smithi092 ceph-mon[79089]: Scrub error on inode 0x1000000020a (/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/WORD/BASEMACH.DOC) see mds.d log and `damage ls` output for details 2024-03-23T17:52:12.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:11 smithi092 ceph-mon[79089]: scrub summary: idle+waiting paths [/] 2024-03-23T17:52:12.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:11 smithi092 ceph-mon[79089]: scrub summary: active paths [/] 2024-03-23T17:52:12.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:11 smithi092 ceph-mon[79089]: Scrub error on inode 0x10000000210 (/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/PWRPNT/PCBENCHM.PPT) see mds.h log and `damage ls` output for details 2024-03-23T17:52:12.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:11 smithi092 ceph-mon[79089]: Scrub error on inode 0x10000000212 (/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/PWRPNT/PPTOOLS1.PPA) see mds.h log and `damage ls` output for details 2024-03-23T17:52:12.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:11 smithi092 ceph-mon[79089]: Scrub error on inode 0x10000000211 (/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/PWRPNT/ZD16.BMP) see mds.h log and `damage ls` output for details 2024-03-23T17:52:12.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:11 smithi092 ceph-mon[79089]: Scrub error on inode 0x10000000213 (/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/PWRPNT/TIPS.PPT) see mds.h log and `damage ls` output for details 2024-03-23T17:52:12.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:11 smithi092 ceph-mon[79089]: Scrub error on inode 0x10000000214 (/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/PWRPNT/TRIDOTS.POT) see mds.h log and `damage ls` output for details 2024-03-23T17:52:12.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:11 smithi092 ceph-mon[79089]: scrub summary: idle+waiting paths [/] 2024-03-23T17:52:12.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:11 smithi092 ceph-mon[79089]: pgmap v49: 97 pgs: 97 active+clean; 26 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 3.6 MiB/s wr, 119 op/s 2024-03-23T17:52:12.309 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:11 smithi092 ceph-mon[79089]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-03-23T17:52:12.309 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:11 smithi092 ceph-mon[79089]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-03-23T17:52:12.309 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:11 smithi092 ceph-mon[79089]: mds.? [v2:172.21.15.92:6834/908900027,v1:172.21.15.92:6835/908900027] up:active 2024-03-23T17:52:12.309 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:11 smithi092 ceph-mon[79089]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T17:52:12.881 INFO:tasks.workunit.client.0.smithi086.stdout: 1 29139 4.71 MB/sec warmup 37 sec latency 1977.440 ms 2024-03-23T17:52:13.881 INFO:tasks.workunit.client.0.smithi086.stdout: 1 29139 4.58 MB/sec warmup 38 sec latency 2977.538 ms 2024-03-23T17:52:13.975 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.975+0000 7f09e6200700 1 -- 172.21.15.86:0/1197281450 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09e010b2f0 msgr2=0x7f09e010d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.975+0000 7f09e6200700 1 --2- 172.21.15.86:0/1197281450 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09e010b2f0 0x7f09e010d760 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f09d800b0a0 tx=0x7f09d800b3b0 comp rx=0 tx=0).stop 2024-03-23T17:52:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.975+0000 7f09e6200700 1 -- 172.21.15.86:0/1197281450 shutdown_connections 2024-03-23T17:52:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.975+0000 7f09e6200700 1 --2- 172.21.15.86:0/1197281450 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09e010b2f0 0x7f09e010d760 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.975+0000 7f09e6200700 1 --2- 172.21.15.86:0/1197281450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09e0101f20 0x7f09e0102300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.975+0000 7f09e6200700 1 --2- 172.21.15.86:0/1197281450 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f09e0102840 0x7f09e010ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.975+0000 7f09e6200700 1 -- 172.21.15.86:0/1197281450 >> 172.21.15.86:0/1197281450 conn(0x7f09e00fb680 msgr2=0x7f09e00fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.975+0000 7f09e6200700 1 -- 172.21.15.86:0/1197281450 shutdown_connections 2024-03-23T17:52:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.975+0000 7f09e6200700 1 -- 172.21.15.86:0/1197281450 wait complete. 2024-03-23T17:52:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.976+0000 7f09e6200700 1 Processor -- start 2024-03-23T17:52:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.976+0000 7f09e6200700 1 -- start start 2024-03-23T17:52:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.977+0000 7f09e6200700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f09e0101f20 0x7f09e00ffb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.977+0000 7f09e6200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09e0102840 0x7f09e01000b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.977+0000 7f09e6200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09e010b2f0 0x7f09e01005f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.977+0000 7f09e6200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09e01a5650 con 0x7f09e010b2f0 2024-03-23T17:52:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.977+0000 7f09e6200700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09e01a57c0 con 0x7f09e0102840 2024-03-23T17:52:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.977+0000 7f09e6200700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09e01a5980 con 0x7f09e0101f20 2024-03-23T17:52:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.977+0000 7f09dfe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09e010b2f0 0x7f09e01005f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.977+0000 7f09dea00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09e0102840 0x7f09e01000b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.977+0000 7f09dea00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09e0102840 0x7f09e01000b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:41560/0 (socket says 172.21.15.86:41560) 2024-03-23T17:52:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.977+0000 7f09dea00700 1 -- 172.21.15.86:0/2639973266 learned_addr learned my addr 172.21.15.86:0/2639973266 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.977+0000 7f09dfe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09e010b2f0 0x7f09e01005f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59402/0 (socket says 172.21.15.86:59402) 2024-03-23T17:52:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.977+0000 7f09df400700 1 --2- 172.21.15.86:0/2639973266 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f09e0101f20 0x7f09e00ffb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.977+0000 7f09dfe00700 1 -- 172.21.15.86:0/2639973266 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f09e0101f20 msgr2=0x7f09e00ffb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.977+0000 7f09dfe00700 1 --2- 172.21.15.86:0/2639973266 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f09e0101f20 0x7f09e00ffb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.977+0000 7f09dfe00700 1 -- 172.21.15.86:0/2639973266 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09e0102840 msgr2=0x7f09e01000b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.977+0000 7f09dfe00700 1 --2- 172.21.15.86:0/2639973266 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09e0102840 0x7f09e01000b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.977+0000 7f09dfe00700 1 -- 172.21.15.86:0/2639973266 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09d8009d00 con 0x7f09e010b2f0 2024-03-23T17:52:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.977+0000 7f09df400700 1 --2- 172.21.15.86:0/2639973266 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f09e0101f20 0x7f09e00ffb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:52:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.978+0000 7f09dfe00700 1 --2- 172.21.15.86:0/2639973266 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09e010b2f0 0x7f09e01005f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f09d800b070 tx=0x7f09d8007860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.978+0000 7f09d7e00700 1 -- 172.21.15.86:0/2639973266 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d800e040 con 0x7f09e010b2f0 2024-03-23T17:52:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.978+0000 7f09d7e00700 1 -- 172.21.15.86:0/2639973266 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09d801d470 con 0x7f09e010b2f0 2024-03-23T17:52:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.978+0000 7f09e6200700 1 -- 172.21.15.86:0/2639973266 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09e01a5c00 con 0x7f09e010b2f0 2024-03-23T17:52:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.978+0000 7f09d7e00700 1 -- 172.21.15.86:0/2639973266 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d8005280 con 0x7f09e010b2f0 2024-03-23T17:52:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.978+0000 7f09e6200700 1 -- 172.21.15.86:0/2639973266 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09e01a6110 con 0x7f09e010b2f0 2024-03-23T17:52:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.981+0000 7f09d7e00700 1 -- 172.21.15.86:0/2639973266 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f09d801d5e0 con 0x7f09e010b2f0 2024-03-23T17:52:13.984 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.982+0000 7f09f9cdab80 1 Processor -- start 2024-03-23T17:52:13.984 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.982+0000 7f09f9cdab80 1 -- start start 2024-03-23T17:52:13.984 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.982+0000 7f09d7e00700 1 --2- 172.21.15.86:0/2639973266 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f09c40a1310 0x7f09c40a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:13.984 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.983+0000 7f09d7e00700 1 -- 172.21.15.86:0/2639973266 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f09d80cdfd0 con 0x7f09e010b2f0 2024-03-23T17:52:13.984 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.983+0000 7f09f9cdab80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563059c83100 0x563059c834e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:13.984 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.983+0000 7f09dea00700 1 --2- 172.21.15.86:0/2639973266 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f09c40a1310 0x7f09c40a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:13.985 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.983+0000 7f09f9cdab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563059c83a20 0x563059c8e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:13.985 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.983+0000 7f09f9cdab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563059c8e8d0 0x563059c90cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:13.985 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.983+0000 7f09f9cdab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x563059bdc930 con 0x563059c8e8d0 2024-03-23T17:52:13.985 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.983+0000 7f09f9cdab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x563059bda2f0 con 0x563059c83a20 2024-03-23T17:52:13.985 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.983+0000 7f09f9cdab80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x563059bd5be0 con 0x563059c83100 2024-03-23T17:52:13.985 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.983+0000 7f09df400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563059c83100 0x563059c834e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:13.985 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.983+0000 7f09df400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563059c83100 0x563059c834e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:51322/0 (socket says 172.21.15.86:51322) 2024-03-23T17:52:13.985 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.983+0000 7f09df400700 1 -- 172.21.15.86:0/2487275566 learned_addr learned my addr 172.21.15.86:0/2487275566 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:13.985 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.983+0000 7f09dea00700 1 --2- 172.21.15.86:0/2487275566 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563059c83a20 0x563059c8e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:13.985 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.983+0000 7f09dfe00700 1 --2- 172.21.15.86:0/2487275566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563059c8e8d0 0x563059c90cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:13.985 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.984+0000 7f09df400700 1 -- 172.21.15.86:0/2487275566 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563059c83a20 msgr2=0x563059c8e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:13.985 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.984+0000 7f09df400700 1 --2- 172.21.15.86:0/2487275566 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563059c83a20 0x563059c8e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:13.985 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.984+0000 7f09df400700 1 -- 172.21.15.86:0/2487275566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563059c8e8d0 msgr2=0x563059c90cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:13.986 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.984+0000 7f09df400700 1 --2- 172.21.15.86:0/2487275566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563059c8e8d0 0x563059c90cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:13.986 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.984+0000 7f09df400700 1 -- 172.21.15.86:0/2487275566 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563059a57350 con 0x563059c83100 2024-03-23T17:52:13.986 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.984+0000 7f09dea00700 1 --2- 172.21.15.86:0/2639973266 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f09c40a1310 0x7f09c40a37d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f09cc0046e0 tx=0x7f09cc013900 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:13.986 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.984+0000 7f09df400700 1 --2- 172.21.15.86:0/2487275566 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563059c83100 0x563059c834e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f09d0009de0 tx=0x7f09d00116c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7b192d1d7785e42 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:13.986 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.984+0000 7f09cbe00700 1 -- 172.21.15.86:0/2487275566 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d0011da0 con 0x563059c83100 2024-03-23T17:52:13.986 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.984+0000 7f09cbe00700 1 -- 172.21.15.86:0/2487275566 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09d0009e70 con 0x563059c83100 2024-03-23T17:52:13.986 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.984+0000 7f09cbe00700 1 -- 172.21.15.86:0/2487275566 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d000f770 con 0x563059c83100 2024-03-23T17:52:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.985+0000 7f09f9cdab80 1 -- 172.21.15.86:0/2487275566 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563059c83100 msgr2=0x563059c834e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.985+0000 7f09f9cdab80 1 --2- 172.21.15.86:0/2487275566 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563059c83100 0x563059c834e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f09d0009de0 tx=0x7f09d00116c0 comp rx=0 tx=0).stop 2024-03-23T17:52:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.985+0000 7f09f9cdab80 1 -- 172.21.15.86:0/2487275566 shutdown_connections 2024-03-23T17:52:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.985+0000 7f09f9cdab80 1 --2- 172.21.15.86:0/2487275566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563059c8e8d0 0x563059c90cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.985+0000 7f09f9cdab80 1 --2- 172.21.15.86:0/2487275566 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563059c83100 0x563059c834e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.985+0000 7f09f9cdab80 1 --2- 172.21.15.86:0/2487275566 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563059c83a20 0x563059c8e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.985+0000 7f09f9cdab80 1 -- 172.21.15.86:0/2487275566 >> 172.21.15.86:0/2487275566 conn(0x563059be3640 msgr2=0x563059c7f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.985+0000 7f09f9cdab80 1 -- 172.21.15.86:0/2487275566 shutdown_connections 2024-03-23T17:52:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.985+0000 7f09f9cdab80 1 -- 172.21.15.86:0/2487275566 wait complete. 2024-03-23T17:52:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.985+0000 7f09f9cdab80 1 -- start start 2024-03-23T17:52:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.986+0000 7f09f9cdab80 1 Processor -- start 2024-03-23T17:52:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.986+0000 7f09f9cdab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563059c83100 0x563059c8e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.986+0000 7f09f9cdab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563059c83a20 0x563059c8b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.986+0000 7f09f9cdab80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563059c8e8d0 0x563059c8bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.986+0000 7f09f9cdab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x563059b71d20 con 0x563059c83a20 2024-03-23T17:52:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.986+0000 7f09f9cdab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x563059b719c0 con 0x563059c83100 2024-03-23T17:52:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.986+0000 7f09f9cdab80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x563059a59640 con 0x563059c8e8d0 2024-03-23T17:52:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.986+0000 7f09f9cdab80 10 client.0 _ll_register_callbacks cb 0x563059b7b410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:52:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.986+0000 7f09dea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563059c83a20 0x563059c8b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.986+0000 7f09dea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563059c83a20 0x563059c8b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59424/0 (socket says 172.21.15.86:59424) 2024-03-23T17:52:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.986+0000 7f09dea00700 1 -- 172.21.15.86:0/4035573642 learned_addr learned my addr 172.21.15.86:0/4035573642 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.987+0000 7f09dfe00700 1 --2- 172.21.15.86:0/4035573642 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563059c8e8d0 0x563059c8bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.987+0000 7f09df400700 1 --2- 172.21.15.86:0/4035573642 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563059c83100 0x563059c8e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.987+0000 7f09dea00700 1 -- 172.21.15.86:0/4035573642 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563059c8e8d0 msgr2=0x563059c8bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.987+0000 7f09dea00700 1 --2- 172.21.15.86:0/4035573642 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563059c8e8d0 0x563059c8bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.987+0000 7f09dea00700 1 -- 172.21.15.86:0/4035573642 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563059c83100 msgr2=0x563059c8e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.987+0000 7f09dea00700 1 --2- 172.21.15.86:0/4035573642 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563059c83100 0x563059c8e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.987+0000 7f09dea00700 1 -- 172.21.15.86:0/4035573642 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09d0009a10 con 0x563059c83a20 2024-03-23T17:52:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.987+0000 7f09dea00700 1 --2- 172.21.15.86:0/4035573642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563059c83a20 0x563059c8b6e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f09cc009ad0 tx=0x7f09cc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=514929d0ef8d6364 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.987+0000 7f09c8c00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-23T17:52:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.987+0000 7f09c8c00700 1 -- 172.21.15.86:0/4035573642 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09cc018040 con 0x563059c83a20 2024-03-23T17:52:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.987+0000 7f09f9cdab80 1 -- 172.21.15.86:0/4035573642 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563059b96320 con 0x563059c83a20 2024-03-23T17:52:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.987+0000 7f09c8c00700 1 -- 172.21.15.86:0/4035573642 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f09cc00f850 con 0x563059c83a20 2024-03-23T17:52:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.988+0000 7f09c8c00700 1 -- 172.21.15.86:0/4035573642 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f09cc00fa10 con 0x563059c83a20 2024-03-23T17:52:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.988+0000 7f09f9cdab80 10 client.15996 fetch_fsmap learned FSMap version 32 2024-03-23T17:52:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.988+0000 7f09f9cdab80 1 -- 172.21.15.86:0/4035573642 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x563059a03b00 con 0x563059c83a20 2024-03-23T17:52:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.988+0000 7f09c8c00700 1 -- 172.21.15.86:0/4035573642 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563059b96320 con 0x563059c83a20 2024-03-23T17:52:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.988+0000 7f09c8c00700 1 -- 172.21.15.86:0/4035573642 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f09cc026070 con 0x563059c83a20 2024-03-23T17:52:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.988+0000 7f09f9cdab80 10 client.15996 fetch_fsmap finished waiting for FSMap version 32 2024-03-23T17:52:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.988+0000 7f09f9cdab80 10 client.15996 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:52:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.988+0000 7f09f9cdab80 20 client.15996 populate_metadata read hostname 'smithi086' 2024-03-23T17:52:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.988+0000 7f09f9cdab80 1 --2- 172.21.15.86:0/4035573642 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x563059c9e640 0x563059ca0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.988+0000 7f09f9cdab80 4 client.15996 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:52:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.988+0000 7f09f9cdab80 1 -- 172.21.15.86:0/4035573642 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563059bce6a0 con 0x563059c9e640 2024-03-23T17:52:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.989+0000 7f09df400700 1 --2- 172.21.15.86:0/4035573642 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x563059c9e640 0x563059ca0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.989+0000 7f09df400700 1 --2- 172.21.15.86:0/4035573642 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x563059c9e640 0x563059ca0a60 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=297d58dd22b574bd server_cookie=f38bf1c5832fbf8b in_seq=0 out_seq=0 2024-03-23T17:52:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.989+0000 7f09c8c00700 10 client.15996 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:52:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.990+0000 7f09c8c00700 1 -- 172.21.15.86:0/4035573642 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x563059bce6a0 con 0x563059c9e640 2024-03-23T17:52:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.990+0000 7f09c8c00700 10 client.15996 handle_command_reply: tid=0 2024-03-23T17:52:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.990+0000 7f09f9cdab80 1 client.15996 shutdown 2024-03-23T17:52:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.990+0000 7f09f9cdab80 1 -- 172.21.15.86:0/4035573642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563059c83a20 msgr2=0x563059c8b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.990+0000 7f09f9cdab80 1 --2- 172.21.15.86:0/4035573642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563059c83a20 0x563059c8b6e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f09cc009ad0 tx=0x7f09cc0047c0 comp rx=0 tx=0).stop 2024-03-23T17:52:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.990+0000 7f09f9cdab80 1 -- 172.21.15.86:0/4035573642 shutdown_connections 2024-03-23T17:52:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.990+0000 7f09f9cdab80 1 --2- 172.21.15.86:0/4035573642 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x563059c9e640 0x563059ca0a60 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.990+0000 7f09f9cdab80 1 --2- 172.21.15.86:0/4035573642 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563059c8e8d0 0x563059c8bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.990+0000 7f09f9cdab80 1 --2- 172.21.15.86:0/4035573642 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563059c83100 0x563059c8e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.990+0000 7f09c8c00700 0 client.15996 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:52:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.990+0000 7f09f9cdab80 1 --2- 172.21.15.86:0/4035573642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563059c83a20 0x563059c8b6e0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.990+0000 7f09f9cdab80 1 -- 172.21.15.86:0/4035573642 >> 172.21.15.86:0/4035573642 conn(0x563059be3640 msgr2=0x563059c850a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.990+0000 7f09f9cdab80 1 -- 172.21.15.86:0/4035573642 shutdown_connections 2024-03-23T17:52:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.990+0000 7f09f9cdab80 1 -- 172.21.15.86:0/4035573642 wait complete. 2024-03-23T17:52:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:13.990+0000 7f09f9cdab80 20 client.15996 trim_cache size 0 max 16384 2024-03-23T17:52:14.006 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.005+0000 7f09f9cdab80 1 Processor -- start 2024-03-23T17:52:14.006 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.005+0000 7f09f9cdab80 1 -- start start 2024-03-23T17:52:14.006 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.005+0000 7f09f9cdab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563059c83100 0x563059c955a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:14.006 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.005+0000 7f09f9cdab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563059c83a20 0x563059c857b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:14.007 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.005+0000 7f09f9cdab80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563059c8e8d0 0x563059c79500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:14.007 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.005+0000 7f09f9cdab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x563059c808b0 con 0x563059c83a20 2024-03-23T17:52:14.007 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.005+0000 7f09f9cdab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x563059c8ca60 con 0x563059c83100 2024-03-23T17:52:14.007 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.005+0000 7f09f9cdab80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x563059c9b040 con 0x563059c8e8d0 2024-03-23T17:52:14.007 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.006+0000 7f09dea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563059c83a20 0x563059c857b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:14.007 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.006+0000 7f09dea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563059c83a20 0x563059c857b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59430/0 (socket says 172.21.15.86:59430) 2024-03-23T17:52:14.007 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.006+0000 7f09dea00700 1 -- 172.21.15.86:0/1966039586 learned_addr learned my addr 172.21.15.86:0/1966039586 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:14.007 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.006+0000 7f09df400700 1 --2- 172.21.15.86:0/1966039586 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563059c83100 0x563059c955a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:14.007 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.006+0000 7f09dfe00700 1 --2- 172.21.15.86:0/1966039586 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563059c8e8d0 0x563059c79500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:14.007 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.006+0000 7f09dea00700 1 -- 172.21.15.86:0/1966039586 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563059c8e8d0 msgr2=0x563059c79500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:14.007 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.006+0000 7f09dea00700 1 --2- 172.21.15.86:0/1966039586 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563059c8e8d0 0x563059c79500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:14.007 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.006+0000 7f09dea00700 1 -- 172.21.15.86:0/1966039586 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563059c83100 msgr2=0x563059c955a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:14.007 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.006+0000 7f09dea00700 1 --2- 172.21.15.86:0/1966039586 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563059c83100 0x563059c955a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:14.008 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.006+0000 7f09dea00700 1 -- 172.21.15.86:0/1966039586 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563059b9be40 con 0x563059c83a20 2024-03-23T17:52:14.008 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.006+0000 7f09dea00700 1 --2- 172.21.15.86:0/1966039586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563059c83a20 0x563059c857b0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f09cc009830 tx=0x7f09cc0123e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f84b65aa5adb9b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:14.008 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.006+0000 7f09d5200700 1 -- 172.21.15.86:0/1966039586 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09cc018070 con 0x563059c83a20 2024-03-23T17:52:14.008 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.006+0000 7f09d5200700 1 -- 172.21.15.86:0/1966039586 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09cc009b00 con 0x563059c83a20 2024-03-23T17:52:14.008 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.007+0000 7f09d5200700 1 -- 172.21.15.86:0/1966039586 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09cc032740 con 0x563059c83a20 2024-03-23T17:52:14.011 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.007+0000 7f09f9cdab80 1 -- 172.21.15.86:0/1966039586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563059c83a20 msgr2=0x563059c857b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:14.012 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.007+0000 7f09f9cdab80 1 --2- 172.21.15.86:0/1966039586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563059c83a20 0x563059c857b0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f09cc009830 tx=0x7f09cc0123e0 comp rx=0 tx=0).stop 2024-03-23T17:52:14.012 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.007+0000 7f09f9cdab80 1 -- 172.21.15.86:0/1966039586 shutdown_connections 2024-03-23T17:52:14.012 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.007+0000 7f09f9cdab80 1 --2- 172.21.15.86:0/1966039586 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563059c8e8d0 0x563059c79500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:14.012 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.007+0000 7f09f9cdab80 1 --2- 172.21.15.86:0/1966039586 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563059c83100 0x563059c955a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:14.012 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.007+0000 7f09f9cdab80 1 --2- 172.21.15.86:0/1966039586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563059c83a20 0x563059c857b0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:14.012 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.007+0000 7f09f9cdab80 1 -- 172.21.15.86:0/1966039586 >> 172.21.15.86:0/1966039586 conn(0x563059be3640 msgr2=0x563059c850a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:14.012 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.007+0000 7f09f9cdab80 1 -- 172.21.15.86:0/1966039586 shutdown_connections 2024-03-23T17:52:14.012 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.007+0000 7f09f9cdab80 1 -- 172.21.15.86:0/1966039586 wait complete. 2024-03-23T17:52:14.012 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.008+0000 7f09f9cdab80 1 -- start start 2024-03-23T17:52:14.012 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.008+0000 7f09f9cdab80 1 Processor -- start 2024-03-23T17:52:14.012 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.008+0000 7f09f9cdab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563059c83100 0x563059c94c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:14.012 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.008+0000 7f09f9cdab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563059c83a20 0x563059c951d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:14.013 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.008+0000 7f09f9cdab80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563059c8e8d0 0x563059c93580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:14.013 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.008+0000 7f09f9cdab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x563059c93c90 con 0x563059c83100 2024-03-23T17:52:14.013 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.008+0000 7f09f9cdab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x563059c93e00 con 0x563059c83a20 2024-03-23T17:52:14.013 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.008+0000 7f09f9cdab80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x563059c93fc0 con 0x563059c8e8d0 2024-03-23T17:52:14.013 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.008+0000 7f09f9cdab80 10 client.0 _ll_register_callbacks cb 0x563059b7b410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:52:14.013 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.008+0000 7f09df400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563059c83100 0x563059c94c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:14.013 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.008+0000 7f09df400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563059c83100 0x563059c94c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59434/0 (socket says 172.21.15.86:59434) 2024-03-23T17:52:14.013 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.009+0000 7f09df400700 1 -- 172.21.15.86:0/3255716622 learned_addr learned my addr 172.21.15.86:0/3255716622 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:14.013 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.009+0000 7f09dea00700 1 --2- 172.21.15.86:0/3255716622 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563059c83a20 0x563059c951d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:14.013 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.009+0000 7f09dfe00700 1 --2- 172.21.15.86:0/3255716622 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563059c8e8d0 0x563059c93580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:14.013 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.009+0000 7f09df400700 1 -- 172.21.15.86:0/3255716622 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563059c8e8d0 msgr2=0x563059c93580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:14.013 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.009+0000 7f09df400700 1 --2- 172.21.15.86:0/3255716622 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563059c8e8d0 0x563059c93580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:14.013 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.009+0000 7f09df400700 1 -- 172.21.15.86:0/3255716622 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563059c83a20 msgr2=0x563059c951d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:14.014 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.009+0000 7f09df400700 1 --2- 172.21.15.86:0/3255716622 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563059c83a20 0x563059c951d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:14.014 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.009+0000 7f09df400700 1 -- 172.21.15.86:0/3255716622 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563059a03b00 con 0x563059c83100 2024-03-23T17:52:14.014 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.009+0000 7f09dfe00700 1 --2- 172.21.15.86:0/3255716622 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563059c8e8d0 0x563059c93580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T17:52:14.014 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.009+0000 7f09dea00700 1 --2- 172.21.15.86:0/3255716622 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563059c83a20 0x563059c951d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:52:14.014 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.009+0000 7f09df400700 1 --2- 172.21.15.86:0/3255716622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563059c83100 0x563059c94c90 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f09d0009db0 tx=0x7f09d0011e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17ec2e14c5da6b43 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:14.014 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.009+0000 7f09c8c00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-23T17:52:14.014 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.009+0000 7f09c8c00700 1 -- 172.21.15.86:0/3255716622 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d0029180 con 0x563059c83100 2024-03-23T17:52:14.014 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.009+0000 7f09c8c00700 1 -- 172.21.15.86:0/3255716622 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f09d00292e0 con 0x563059c83100 2024-03-23T17:52:14.014 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.009+0000 7f09f9cdab80 1 -- 172.21.15.86:0/3255716622 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f09d0009e70 con 0x563059c83100 2024-03-23T17:52:14.014 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.010+0000 7f09c8c00700 1 -- 172.21.15.86:0/3255716622 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f09d0029980 con 0x563059c83100 2024-03-23T17:52:14.014 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.010+0000 7f09f9cdab80 10 client.16002 fetch_fsmap learned FSMap version 32 2024-03-23T17:52:14.014 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.010+0000 7f09f9cdab80 1 -- 172.21.15.86:0/3255716622 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x563059a5bc70 con 0x563059c83100 2024-03-23T17:52:14.014 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.010+0000 7f09c8c00700 1 -- 172.21.15.86:0/3255716622 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09d0009e70 con 0x563059c83100 2024-03-23T17:52:14.015 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.010+0000 7f09c8c00700 1 -- 172.21.15.86:0/3255716622 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f09d00113e0 con 0x563059c83100 2024-03-23T17:52:14.015 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.010+0000 7f09f9cdab80 10 client.16002 fetch_fsmap finished waiting for FSMap version 32 2024-03-23T17:52:14.015 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.010+0000 7f09f9cdab80 10 client.16002 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:52:14.015 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.010+0000 7f09f9cdab80 20 client.16002 populate_metadata read hostname 'smithi086' 2024-03-23T17:52:14.015 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.010+0000 7f09f9cdab80 1 --2- 172.21.15.86:0/3255716622 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x563059cf13a0 0x563059cf37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:14.015 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.010+0000 7f09f9cdab80 4 client.16002 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T17:52:14.015 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.010+0000 7f09f9cdab80 1 -- 172.21.15.86:0/3255716622 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563059a24c90 con 0x563059cf13a0 2024-03-23T17:52:14.015 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.010+0000 7f09dea00700 1 --2- 172.21.15.86:0/3255716622 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x563059cf13a0 0x563059cf37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:14.015 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.011+0000 7f09c8c00700 10 client.16002 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:52:14.015 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.011+0000 7f09dea00700 1 --2- 172.21.15.86:0/3255716622 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x563059cf13a0 0x563059cf37c0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd31d7ebd1e99130 server_cookie=e837c95a2c2d0259 in_seq=0 out_seq=0 2024-03-23T17:52:14.033 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.032+0000 7f09c8c00700 1 -- 172.21.15.86:0/3255716622 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563059a24c90 con 0x563059cf13a0 2024-03-23T17:52:14.033 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.032+0000 7f09c8c00700 10 client.16002 handle_command_reply: tid=0 2024-03-23T17:52:14.033 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.032+0000 7f09f9cdab80 1 client.16002 shutdown 2024-03-23T17:52:14.033 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.032+0000 7f09f9cdab80 1 -- 172.21.15.86:0/3255716622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563059c83100 msgr2=0x563059c94c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:14.033 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.032+0000 7f09f9cdab80 1 --2- 172.21.15.86:0/3255716622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563059c83100 0x563059c94c90 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f09d0009db0 tx=0x7f09d0011e80 comp rx=0 tx=0).stop 2024-03-23T17:52:14.033 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.032+0000 7f09f9cdab80 1 -- 172.21.15.86:0/3255716622 shutdown_connections 2024-03-23T17:52:14.033 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.032+0000 7f09f9cdab80 1 --2- 172.21.15.86:0/3255716622 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x563059cf13a0 0x563059cf37c0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:14.033 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.032+0000 7f09f9cdab80 1 --2- 172.21.15.86:0/3255716622 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563059c8e8d0 0x563059c93580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:14.033 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.032+0000 7f09f9cdab80 1 --2- 172.21.15.86:0/3255716622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563059c83100 0x563059c94c90 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:14.033 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.032+0000 7f09c8c00700 0 client.16002 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:52:14.033 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.032+0000 7f09f9cdab80 1 --2- 172.21.15.86:0/3255716622 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563059c83a20 0x563059c951d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:14.034 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.032+0000 7f09f9cdab80 1 -- 172.21.15.86:0/3255716622 >> 172.21.15.86:0/3255716622 conn(0x563059be3640 msgr2=0x563059c85740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:14.034 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.032+0000 7f09f9cdab80 1 -- 172.21.15.86:0/3255716622 shutdown_connections 2024-03-23T17:52:14.034 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.032+0000 7f09f9cdab80 1 -- 172.21.15.86:0/3255716622 wait complete. 2024-03-23T17:52:14.034 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.033+0000 7f09f9cdab80 20 client.16002 trim_cache size 0 max 16384 2024-03-23T17:52:14.034 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-03-23T17:52:14.034 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-03-23T17:52:14.034 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-03-23T17:52:14.034 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-03-23T17:52:14.035 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.033+0000 7f09e6200700 1 -- 172.21.15.86:0/2639973266 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f09c40a1310 msgr2=0x7f09c40a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:14.035 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.033+0000 7f09e6200700 1 --2- 172.21.15.86:0/2639973266 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f09c40a1310 0x7f09c40a37d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f09cc0046e0 tx=0x7f09cc013900 comp rx=0 tx=0).stop 2024-03-23T17:52:14.035 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.033+0000 7f09e6200700 1 -- 172.21.15.86:0/2639973266 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09e010b2f0 msgr2=0x7f09e01005f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:14.035 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.033+0000 7f09e6200700 1 --2- 172.21.15.86:0/2639973266 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09e010b2f0 0x7f09e01005f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f09d800b070 tx=0x7f09d8007860 comp rx=0 tx=0).stop 2024-03-23T17:52:14.035 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.033+0000 7f09e6200700 1 -- 172.21.15.86:0/2639973266 shutdown_connections 2024-03-23T17:52:14.035 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.033+0000 7f09e6200700 1 --2- 172.21.15.86:0/2639973266 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09e010b2f0 0x7f09e01005f0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:14.036 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.033+0000 7f09e6200700 1 --2- 172.21.15.86:0/2639973266 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f09c40a1310 0x7f09c40a37d0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:14.036 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.033+0000 7f09e6200700 1 --2- 172.21.15.86:0/2639973266 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f09e0101f20 0x7f09e00ffb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:14.036 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.033+0000 7f09e6200700 1 --2- 172.21.15.86:0/2639973266 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09e0102840 0x7f09e01000b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:14.036 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.033+0000 7f09e6200700 1 -- 172.21.15.86:0/2639973266 >> 172.21.15.86:0/2639973266 conn(0x7f09e00fb680 msgr2=0x7f09e00fc3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:14.036 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.033+0000 7f09e6200700 1 -- 172.21.15.86:0/2639973266 shutdown_connections 2024-03-23T17:52:14.036 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:14.033+0000 7f09e6200700 1 -- 172.21.15.86:0/2639973266 wait complete. 2024-03-23T17:52:14.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:13 smithi152 ceph-mon[47949]: pgmap v50: 97 pgs: 97 active+clean; 26 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 852 KiB/s wr, 66 op/s 2024-03-23T17:52:14.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:13 smithi152 ceph-mon[47949]: mds.? [v2:172.21.15.92:6836/3967837948,v1:172.21.15.92:6837/3967837948] up:active 2024-03-23T17:52:14.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:13 smithi152 ceph-mon[47949]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T17:52:14.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:13 smithi086 ceph-mon[38221]: pgmap v50: 97 pgs: 97 active+clean; 26 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 852 KiB/s wr, 66 op/s 2024-03-23T17:52:14.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:13 smithi086 ceph-mon[38221]: mds.? [v2:172.21.15.92:6836/3967837948,v1:172.21.15.92:6837/3967837948] up:active 2024-03-23T17:52:14.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:13 smithi086 ceph-mon[38221]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T17:52:14.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:13 smithi092 ceph-mon[79089]: pgmap v50: 97 pgs: 97 active+clean; 26 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 852 KiB/s wr, 66 op/s 2024-03-23T17:52:14.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:13 smithi092 ceph-mon[79089]: mds.? [v2:172.21.15.92:6836/3967837948,v1:172.21.15.92:6837/3967837948] up:active 2024-03-23T17:52:14.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:13 smithi092 ceph-mon[79089]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T17:52:14.881 INFO:tasks.workunit.client.0.smithi086.stdout: 1 29139 4.47 MB/sec warmup 39 sec latency 3977.623 ms 2024-03-23T17:52:15.881 INFO:tasks.workunit.client.0.smithi086.stdout: 1 29139 4.36 MB/sec warmup 40 sec latency 4977.712 ms 2024-03-23T17:52:16.085 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:15 smithi152 ceph-mon[47949]: pgmap v51: 97 pgs: 97 active+clean; 26 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 861 KiB/s wr, 78 op/s 2024-03-23T17:52:16.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:15 smithi086 ceph-mon[38221]: pgmap v51: 97 pgs: 97 active+clean; 26 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 861 KiB/s wr, 78 op/s 2024-03-23T17:52:16.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:15 smithi092 ceph-mon[79089]: pgmap v51: 97 pgs: 97 active+clean; 26 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 861 KiB/s wr, 78 op/s 2024-03-23T17:52:16.881 INFO:tasks.workunit.client.0.smithi086.stdout: 1 30859 4.42 MB/sec warmup 41 sec latency 4999.859 ms 2024-03-23T17:52:17.882 INFO:tasks.workunit.client.0.smithi086.stdout: 1 32472 4.60 MB/sec warmup 42 sec latency 11.567 ms 2024-03-23T17:52:18.085 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:17 smithi152 ceph-mon[47949]: pgmap v52: 97 pgs: 97 active+clean; 27 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 569 KiB/s wr, 73 op/s 2024-03-23T17:52:18.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:17 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mon-a[38198]: 2024-03-23T17:52:17.822+0000 7f70a6200700 -1 log_channel(cluster) log [ERR] : Health check update: 3 MDSs report damaged metadata (MDS_DAMAGE) 2024-03-23T17:52:18.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:17 smithi086 ceph-mon[38221]: pgmap v52: 97 pgs: 97 active+clean; 27 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 569 KiB/s wr, 73 op/s 2024-03-23T17:52:18.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:17 smithi092 ceph-mon[79089]: pgmap v52: 97 pgs: 97 active+clean; 27 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 569 KiB/s wr, 73 op/s 2024-03-23T17:52:18.881 INFO:tasks.workunit.client.0.smithi086.stdout: 1 34858 4.77 MB/sec warmup 43 sec latency 10.685 ms 2024-03-23T17:52:19.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:18 smithi152 ceph-mon[47949]: Health check update: 3 MDSs report damaged metadata (MDS_DAMAGE) 2024-03-23T17:52:19.257 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:18 smithi152 ceph-mon[47949]: mds.? [v2:172.21.15.86:6836/2933881328,v1:172.21.15.86:6837/2933881328] up:active 2024-03-23T17:52:19.257 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:18 smithi152 ceph-mon[47949]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T17:52:19.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:18 smithi086 ceph-mon[38221]: Health check update: 3 MDSs report damaged metadata (MDS_DAMAGE) 2024-03-23T17:52:19.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:18 smithi086 ceph-mon[38221]: mds.? [v2:172.21.15.86:6836/2933881328,v1:172.21.15.86:6837/2933881328] up:active 2024-03-23T17:52:19.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:18 smithi086 ceph-mon[38221]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T17:52:19.311 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:18 smithi092 ceph-mon[79089]: Health check update: 3 MDSs report damaged metadata (MDS_DAMAGE) 2024-03-23T17:52:19.311 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:18 smithi092 ceph-mon[79089]: mds.? [v2:172.21.15.86:6836/2933881328,v1:172.21.15.86:6837/2933881328] up:active 2024-03-23T17:52:19.311 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:18 smithi092 ceph-mon[79089]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T17:52:19.881 INFO:tasks.workunit.client.0.smithi086.stdout: 1 36166 4.83 MB/sec warmup 44 sec latency 12.002 ms 2024-03-23T17:52:20.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:19 smithi086 ceph-mon[38221]: pgmap v53: 97 pgs: 97 active+clean; 27 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 65 KiB/s wr, 37 op/s 2024-03-23T17:52:20.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:19 smithi086 ceph-mon[38221]: scrub summary: idle 2024-03-23T17:52:20.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:19 smithi086 ceph-mon[38221]: scrub summary: idle 2024-03-23T17:52:20.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:19 smithi086 ceph-mon[38221]: scrub summary: idle 2024-03-23T17:52:20.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:19 smithi092 ceph-mon[79089]: pgmap v53: 97 pgs: 97 active+clean; 27 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 65 KiB/s wr, 37 op/s 2024-03-23T17:52:20.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:19 smithi092 ceph-mon[79089]: scrub summary: idle 2024-03-23T17:52:20.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:19 smithi092 ceph-mon[79089]: scrub summary: idle 2024-03-23T17:52:20.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:19 smithi092 ceph-mon[79089]: scrub summary: idle 2024-03-23T17:52:20.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:19 smithi152 ceph-mon[47949]: pgmap v53: 97 pgs: 97 active+clean; 27 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 65 KiB/s wr, 37 op/s 2024-03-23T17:52:20.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:19 smithi152 ceph-mon[47949]: scrub summary: idle 2024-03-23T17:52:20.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:19 smithi152 ceph-mon[47949]: scrub summary: idle 2024-03-23T17:52:20.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:19 smithi152 ceph-mon[47949]: scrub summary: idle 2024-03-23T17:52:20.881 INFO:tasks.workunit.client.0.smithi086.stdout: 1 38517 5.00 MB/sec warmup 45 sec latency 13.253 ms 2024-03-23T17:52:21.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:20 smithi086 ceph-mon[38221]: scrub summary: idle 2024-03-23T17:52:21.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:20 smithi086 ceph-mon[38221]: scrub summary: idle 2024-03-23T17:52:21.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:20 smithi092 ceph-mon[79089]: scrub summary: idle 2024-03-23T17:52:21.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:20 smithi092 ceph-mon[79089]: scrub summary: idle 2024-03-23T17:52:21.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:20 smithi152 ceph-mon[47949]: scrub summary: idle 2024-03-23T17:52:21.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:20 smithi152 ceph-mon[47949]: scrub summary: idle 2024-03-23T17:52:21.882 INFO:tasks.workunit.client.0.smithi086.stdout: 1 39722 5.03 MB/sec warmup 46 sec latency 109.273 ms 2024-03-23T17:52:22.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:21 smithi086 ceph-mon[38221]: pgmap v54: 97 pgs: 97 active+clean; 30 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 516 KiB/s wr, 123 op/s 2024-03-23T17:52:22.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:21 smithi092 ceph-mon[79089]: pgmap v54: 97 pgs: 97 active+clean; 30 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 516 KiB/s wr, 123 op/s 2024-03-23T17:52:22.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:21 smithi152 ceph-mon[47949]: pgmap v54: 97 pgs: 97 active+clean; 30 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 516 KiB/s wr, 123 op/s 2024-03-23T17:52:22.882 INFO:tasks.workunit.client.0.smithi086.stdout: 1 39722 4.92 MB/sec warmup 47 sec latency 1109.376 ms 2024-03-23T17:52:23.882 INFO:tasks.workunit.client.0.smithi086.stdout: 1 39722 4.82 MB/sec warmup 48 sec latency 2109.482 ms 2024-03-23T17:52:24.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:23 smithi086 ceph-mon[38221]: pgmap v55: 97 pgs: 97 active+clean; 30 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 484 KiB/s wr, 111 op/s 2024-03-23T17:52:24.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:23 smithi092 ceph-mon[79089]: pgmap v55: 97 pgs: 97 active+clean; 30 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 484 KiB/s wr, 111 op/s 2024-03-23T17:52:24.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:23 smithi152 ceph-mon[47949]: pgmap v55: 97 pgs: 97 active+clean; 30 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 484 KiB/s wr, 111 op/s 2024-03-23T17:52:24.882 INFO:tasks.workunit.client.0.smithi086.stdout: 1 39722 4.72 MB/sec warmup 49 sec latency 3109.550 ms 2024-03-23T17:52:25.882 INFO:tasks.workunit.client.0.smithi086.stdout: 1 39722 4.63 MB/sec warmup 50 sec latency 4109.690 ms 2024-03-23T17:52:26.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:25 smithi086 ceph-mon[38221]: pgmap v56: 97 pgs: 97 active+clean; 32 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 715 KiB/s wr, 156 op/s 2024-03-23T17:52:26.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:25 smithi092 ceph-mon[79089]: pgmap v56: 97 pgs: 97 active+clean; 32 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 715 KiB/s wr, 156 op/s 2024-03-23T17:52:26.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:25 smithi152 ceph-mon[47949]: pgmap v56: 97 pgs: 97 active+clean; 32 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 715 KiB/s wr, 156 op/s 2024-03-23T17:52:26.882 INFO:tasks.workunit.client.0.smithi086.stdout: 1 42047 4.78 MB/sec warmup 51 sec latency 4130.483 ms 2024-03-23T17:52:27.187 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.186+0000 7fab23400700 1 -- 172.21.15.86:0/4235555142 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab1c1018c0 msgr2=0x7fab1c105900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:27.187 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.186+0000 7fab23400700 1 --2- 172.21.15.86:0/4235555142 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab1c1018c0 0x7fab1c105900 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fab08009990 tx=0x7fab08009ca0 comp rx=0 tx=0).stop 2024-03-23T17:52:27.187 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.186+0000 7fab23400700 1 -- 172.21.15.86:0/4235555142 shutdown_connections 2024-03-23T17:52:27.187 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.186+0000 7fab23400700 1 --2- 172.21.15.86:0/4235555142 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fab1c106030 0x7fab1c10dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:27.187 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.186+0000 7fab23400700 1 --2- 172.21.15.86:0/4235555142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab1c100f10 0x7fab1c1012f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:27.187 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.186+0000 7fab23400700 1 --2- 172.21.15.86:0/4235555142 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab1c1018c0 0x7fab1c105900 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:27.187 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.186+0000 7fab23400700 1 -- 172.21.15.86:0/4235555142 >> 172.21.15.86:0/4235555142 conn(0x7fab1c0fc7b0 msgr2=0x7fab1c0febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:27.187 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.186+0000 7fab23400700 1 -- 172.21.15.86:0/4235555142 shutdown_connections 2024-03-23T17:52:27.187 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.186+0000 7fab23400700 1 -- 172.21.15.86:0/4235555142 wait complete. 2024-03-23T17:52:27.189 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.187+0000 7fab23400700 1 Processor -- start 2024-03-23T17:52:27.189 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.188+0000 7fab23400700 1 -- start start 2024-03-23T17:52:27.189 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.188+0000 7fab23400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab1c100f10 0x7fab1c0730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:27.189 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.188+0000 7fab23400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab1c106030 0x7fab1c0735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:27.189 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.188+0000 7fab23400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fab1c073b30 0x7fab1c1a5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:27.189 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.188+0000 7fab23400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab1c1a59b0 con 0x7fab1c100f10 2024-03-23T17:52:27.189 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.188+0000 7fab23400700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab1c1a5b20 con 0x7fab1c106030 2024-03-23T17:52:27.189 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.188+0000 7fab23400700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab1c1a5ce0 con 0x7fab1c073b30 2024-03-23T17:52:27.190 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.188+0000 7fab1be00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab1c106030 0x7fab1c0735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:27.190 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.188+0000 7fab20a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab1c100f10 0x7fab1c0730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:27.190 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.188+0000 7fab1be00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab1c106030 0x7fab1c0735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:39006/0 (socket says 172.21.15.86:39006) 2024-03-23T17:52:27.190 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.188+0000 7fab1be00700 1 -- 172.21.15.86:0/106962961 learned_addr learned my addr 172.21.15.86:0/106962961 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:27.190 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.188+0000 7fab21400700 1 --2- 172.21.15.86:0/106962961 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fab1c073b30 0x7fab1c1a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:27.190 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.189+0000 7fab20a00700 1 -- 172.21.15.86:0/106962961 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fab1c073b30 msgr2=0x7fab1c1a5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:27.190 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.189+0000 7fab20a00700 1 --2- 172.21.15.86:0/106962961 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fab1c073b30 0x7fab1c1a5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:27.190 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.189+0000 7fab20a00700 1 -- 172.21.15.86:0/106962961 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab1c106030 msgr2=0x7fab1c0735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:27.190 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.189+0000 7fab20a00700 1 --2- 172.21.15.86:0/106962961 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab1c106030 0x7fab1c0735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:27.190 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.189+0000 7fab20a00700 1 -- 172.21.15.86:0/106962961 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab080095f0 con 0x7fab1c100f10 2024-03-23T17:52:27.190 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.189+0000 7fab1be00700 1 --2- 172.21.15.86:0/106962961 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab1c106030 0x7fab1c0735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:52:27.190 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.189+0000 7fab20a00700 1 --2- 172.21.15.86:0/106962961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab1c100f10 0x7fab1c0730b0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fab1000e9f0 tx=0x7fab1000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:27.191 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.189+0000 7fab19600700 1 -- 172.21.15.86:0/106962961 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1000c9c0 con 0x7fab1c100f10 2024-03-23T17:52:27.191 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.189+0000 7fab19600700 1 -- 172.21.15.86:0/106962961 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab10010430 con 0x7fab1c100f10 2024-03-23T17:52:27.191 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.190+0000 7fab23400700 1 -- 172.21.15.86:0/106962961 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab1c1a5fc0 con 0x7fab1c100f10 2024-03-23T17:52:27.191 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.190+0000 7fab19600700 1 -- 172.21.15.86:0/106962961 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab10018680 con 0x7fab1c100f10 2024-03-23T17:52:27.191 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.190+0000 7fab23400700 1 -- 172.21.15.86:0/106962961 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab1c1a6440 con 0x7fab1c100f10 2024-03-23T17:52:27.193 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.193+0000 7fab19600700 1 -- 172.21.15.86:0/106962961 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fab10010860 con 0x7fab1c100f10 2024-03-23T17:52:27.194 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.194+0000 7fab19600700 1 --2- 172.21.15.86:0/106962961 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fab040a1290 0x7fab040a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:27.194 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.194+0000 7fab36ed7b80 1 Processor -- start 2024-03-23T17:52:27.194 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.194+0000 7fab19600700 1 -- 172.21.15.86:0/106962961 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fab10014070 con 0x7fab1c100f10 2024-03-23T17:52:27.194 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.194+0000 7fab1be00700 1 --2- 172.21.15.86:0/106962961 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fab040a1290 0x7fab040a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:27.194 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.194+0000 7fab36ed7b80 1 -- start start 2024-03-23T17:52:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.195+0000 7fab36ed7b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558143f48100 0x558143f484e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.195+0000 7fab36ed7b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558143f48a20 0x558143f53390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.195+0000 7fab36ed7b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558143f538d0 0x558143f55cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.195+0000 7fab36ed7b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558143ea1930 con 0x558143f48a20 2024-03-23T17:52:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.195+0000 7fab36ed7b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x558143e9f2f0 con 0x558143f538d0 2024-03-23T17:52:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.195+0000 7fab36ed7b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x558143e9abe0 con 0x558143f48100 2024-03-23T17:52:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.195+0000 7fab21400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558143f538d0 0x558143f55cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.195+0000 7fab20a00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558143f48100 0x558143f484e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.195+0000 7fab21400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558143f538d0 0x558143f55cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:39008/0 (socket says 172.21.15.86:39008) 2024-03-23T17:52:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.195+0000 7fab1be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558143f48a20 0x558143f53390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.195+0000 7fab21400700 1 -- 172.21.15.86:0/3585658865 learned_addr learned my addr 172.21.15.86:0/3585658865 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.195+0000 7fab1be00700 1 --2- 172.21.15.86:0/106962961 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fab040a1290 0x7fab040a3750 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fab1c0747e0 tx=0x7fab0800b280 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:27.197 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.195+0000 7fab21400700 1 -- 172.21.15.86:0/3585658865 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558143f48100 msgr2=0x558143f484e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:27.197 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.195+0000 7fab21400700 1 --2- 172.21.15.86:0/3585658865 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558143f48100 0x558143f484e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:27.197 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.195+0000 7fab21400700 1 -- 172.21.15.86:0/3585658865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558143f48a20 msgr2=0x558143f53390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:27.197 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.195+0000 7fab21400700 1 --2- 172.21.15.86:0/3585658865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558143f48a20 0x558143f53390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:27.197 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.196+0000 7fab21400700 1 -- 172.21.15.86:0/3585658865 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558143d1c350 con 0x558143f538d0 2024-03-23T17:52:27.197 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.196+0000 7fab21400700 1 --2- 172.21.15.86:0/3585658865 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558143f538d0 0x558143f55cc0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fab14003fa0 tx=0x7fab14009ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffd7c93de05ebe7d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:27.198 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.196+0000 7fab0dc00700 1 -- 172.21.15.86:0/3585658865 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab14019070 con 0x558143f538d0 2024-03-23T17:52:27.198 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.196+0000 7fab0dc00700 1 -- 172.21.15.86:0/3585658865 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab1400f040 con 0x558143f538d0 2024-03-23T17:52:27.198 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.196+0000 7fab0dc00700 1 -- 172.21.15.86:0/3585658865 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab14007a60 con 0x558143f538d0 2024-03-23T17:52:27.198 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.197+0000 7fab36ed7b80 1 -- 172.21.15.86:0/3585658865 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558143f538d0 msgr2=0x558143f55cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:27.198 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.197+0000 7fab36ed7b80 1 --2- 172.21.15.86:0/3585658865 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558143f538d0 0x558143f55cc0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fab14003fa0 tx=0x7fab14009ba0 comp rx=0 tx=0).stop 2024-03-23T17:52:27.198 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.197+0000 7fab36ed7b80 1 -- 172.21.15.86:0/3585658865 shutdown_connections 2024-03-23T17:52:27.198 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.197+0000 7fab36ed7b80 1 --2- 172.21.15.86:0/3585658865 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558143f538d0 0x558143f55cc0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:27.198 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.197+0000 7fab36ed7b80 1 --2- 172.21.15.86:0/3585658865 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558143f48100 0x558143f484e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:27.198 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.197+0000 7fab36ed7b80 1 --2- 172.21.15.86:0/3585658865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558143f48a20 0x558143f53390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:27.199 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.197+0000 7fab36ed7b80 1 -- 172.21.15.86:0/3585658865 >> 172.21.15.86:0/3585658865 conn(0x558143ea8640 msgr2=0x558143f44520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:27.199 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.197+0000 7fab36ed7b80 1 -- 172.21.15.86:0/3585658865 shutdown_connections 2024-03-23T17:52:27.199 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.197+0000 7fab36ed7b80 1 -- 172.21.15.86:0/3585658865 wait complete. 2024-03-23T17:52:27.199 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.198+0000 7fab36ed7b80 1 -- start start 2024-03-23T17:52:27.199 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.198+0000 7fab36ed7b80 1 Processor -- start 2024-03-23T17:52:27.202 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.199+0000 7fab36ed7b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558143f48100 0x558143f532d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:27.202 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.199+0000 7fab36ed7b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558143f48a20 0x558143f505a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:27.202 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.199+0000 7fab36ed7b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558143f538d0 0x558143f50bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:27.202 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.199+0000 7fab36ed7b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558143e36d20 con 0x558143f538d0 2024-03-23T17:52:27.202 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.199+0000 7fab36ed7b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x558143e369c0 con 0x558143f48100 2024-03-23T17:52:27.202 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.199+0000 7fab36ed7b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x558143d1e640 con 0x558143f48a20 2024-03-23T17:52:27.202 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.199+0000 7fab36ed7b80 10 client.0 _ll_register_callbacks cb 0x558143e40410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:52:27.202 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.199+0000 7fab1be00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558143f48a20 0x558143f505a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:27.202 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.199+0000 7fab1be00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558143f48a20 0x558143f505a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:42278/0 (socket says 172.21.15.86:42278) 2024-03-23T17:52:27.202 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.199+0000 7fab20a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558143f48100 0x558143f532d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:27.202 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.199+0000 7fab1be00700 1 -- 172.21.15.86:0/1509651343 learned_addr learned my addr 172.21.15.86:0/1509651343 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:27.202 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.199+0000 7fab21400700 1 --2- 172.21.15.86:0/1509651343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558143f538d0 0x558143f50bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:27.202 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.199+0000 7fab20a00700 1 -- 172.21.15.86:0/1509651343 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558143f48a20 msgr2=0x558143f505a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:27.203 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.199+0000 7fab20a00700 1 --2- 172.21.15.86:0/1509651343 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558143f48a20 0x558143f505a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:27.203 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.199+0000 7fab20a00700 1 -- 172.21.15.86:0/1509651343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558143f538d0 msgr2=0x558143f50bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:27.203 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.199+0000 7fab20a00700 1 --2- 172.21.15.86:0/1509651343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558143f538d0 0x558143f50bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:27.203 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.199+0000 7fab20a00700 1 -- 172.21.15.86:0/1509651343 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab14003bd0 con 0x558143f48100 2024-03-23T17:52:27.203 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.199+0000 7fab1be00700 1 --2- 172.21.15.86:0/1509651343 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558143f48a20 0x558143f505a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:52:27.203 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.200+0000 7fab20a00700 1 --2- 172.21.15.86:0/1509651343 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558143f48100 0x558143f532d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fab100ca540 tx=0x7fab1009a690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d9d12866fe49b98 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:27.203 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.200+0000 7fab02000700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T17:52:27.203 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.200+0000 7fab02000700 1 -- 172.21.15.86:0/1509651343 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab100995f0 con 0x558143f48100 2024-03-23T17:52:27.203 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.200+0000 7fab02000700 1 -- 172.21.15.86:0/1509651343 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fab1009dd30 con 0x558143f48100 2024-03-23T17:52:27.203 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.200+0000 7fab36ed7b80 1 -- 172.21.15.86:0/1509651343 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558143e5b320 con 0x558143f48100 2024-03-23T17:52:27.203 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.200+0000 7fab02000700 1 -- 172.21.15.86:0/1509651343 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab100a2380 con 0x558143f48100 2024-03-23T17:52:27.203 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.200+0000 7fab36ed7b80 10 client.26069 fetch_fsmap learned FSMap version 33 2024-03-23T17:52:27.203 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.200+0000 7fab36ed7b80 1 -- 172.21.15.86:0/1509651343 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x558143cc8b00 con 0x558143f48100 2024-03-23T17:52:27.203 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.200+0000 7fab02000700 1 -- 172.21.15.86:0/1509651343 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558143e5b320 con 0x558143f48100 2024-03-23T17:52:27.203 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.200+0000 7fab02000700 1 -- 172.21.15.86:0/1509651343 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 33) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fab10094b10 con 0x558143f48100 2024-03-23T17:52:27.204 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.200+0000 7fab36ed7b80 10 client.26069 fetch_fsmap finished waiting for FSMap version 33 2024-03-23T17:52:27.204 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.200+0000 7fab36ed7b80 10 client.26069 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:52:27.204 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.201+0000 7fab36ed7b80 20 client.26069 populate_metadata read hostname 'smithi086' 2024-03-23T17:52:27.204 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.201+0000 7fab36ed7b80 1 --2- 172.21.15.86:0/1509651343 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x558143f63620 0x558143f65a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:27.204 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.201+0000 7fab36ed7b80 4 client.26069 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:52:27.204 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.201+0000 7fab36ed7b80 1 -- 172.21.15.86:0/1509651343 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558143e936a0 con 0x558143f63620 2024-03-23T17:52:27.204 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.201+0000 7fab1be00700 1 --2- 172.21.15.86:0/1509651343 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x558143f63620 0x558143f65a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:27.204 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.201+0000 7fab1be00700 1 --2- 172.21.15.86:0/1509651343 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x558143f63620 0x558143f65a40 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=943837d41cd402dc server_cookie=c5516c4cb3f7ce80 in_seq=0 out_seq=0 2024-03-23T17:52:27.204 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.201+0000 7fab02000700 10 client.26069 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:52:27.205 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.202+0000 7fab02000700 1 -- 172.21.15.86:0/1509651343 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x558143e936a0 con 0x558143f63620 2024-03-23T17:52:27.205 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.202+0000 7fab02000700 10 client.26069 handle_command_reply: tid=0 2024-03-23T17:52:27.205 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.202+0000 7fab36ed7b80 1 client.26069 shutdown 2024-03-23T17:52:27.205 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.202+0000 7fab36ed7b80 1 -- 172.21.15.86:0/1509651343 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558143f48100 msgr2=0x558143f532d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:27.205 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.202+0000 7fab36ed7b80 1 --2- 172.21.15.86:0/1509651343 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558143f48100 0x558143f532d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fab100ca540 tx=0x7fab1009a690 comp rx=0 tx=0).stop 2024-03-23T17:52:27.205 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.202+0000 7fab36ed7b80 1 -- 172.21.15.86:0/1509651343 shutdown_connections 2024-03-23T17:52:27.206 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.202+0000 7fab36ed7b80 1 --2- 172.21.15.86:0/1509651343 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x558143f63620 0x558143f65a40 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:27.206 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.202+0000 7fab36ed7b80 1 --2- 172.21.15.86:0/1509651343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558143f538d0 0x558143f50bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:27.206 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.202+0000 7fab36ed7b80 1 --2- 172.21.15.86:0/1509651343 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558143f48100 0x558143f532d0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:27.206 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.202+0000 7fab36ed7b80 1 --2- 172.21.15.86:0/1509651343 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558143f48a20 0x558143f505a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:27.206 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.202+0000 7fab02000700 0 client.26069 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:52:27.206 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.202+0000 7fab36ed7b80 1 -- 172.21.15.86:0/1509651343 >> 172.21.15.86:0/1509651343 conn(0x558143ea8640 msgr2=0x558143f4a0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:27.206 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.203+0000 7fab36ed7b80 1 -- 172.21.15.86:0/1509651343 shutdown_connections 2024-03-23T17:52:27.206 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.203+0000 7fab36ed7b80 1 -- 172.21.15.86:0/1509651343 wait complete. 2024-03-23T17:52:27.206 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.203+0000 7fab36ed7b80 20 client.26069 trim_cache size 0 max 16384 2024-03-23T17:52:27.220 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.219+0000 7fab36ed7b80 1 Processor -- start 2024-03-23T17:52:27.220 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.219+0000 7fab36ed7b80 1 -- start start 2024-03-23T17:52:27.220 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.219+0000 7fab36ed7b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558143f48100 0x558143f6a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:27.220 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.219+0000 7fab36ed7b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558143f48a20 0x558143f5a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:27.220 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.219+0000 7fab36ed7b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558143f538d0 0x558143f4f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:27.220 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.219+0000 7fab36ed7b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558143f65e10 con 0x558143f48a20 2024-03-23T17:52:27.220 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.219+0000 7fab36ed7b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x558143f49bc0 con 0x558143f48100 2024-03-23T17:52:27.220 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.219+0000 7fab36ed7b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x558143f51490 con 0x558143f538d0 2024-03-23T17:52:27.220 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.219+0000 7fab20a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558143f48100 0x558143f6a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:27.220 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.219+0000 7fab20a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558143f48100 0x558143f6a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:39034/0 (socket says 172.21.15.86:39034) 2024-03-23T17:52:27.221 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.220+0000 7fab20a00700 1 -- 172.21.15.86:0/1876098417 learned_addr learned my addr 172.21.15.86:0/1876098417 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:27.221 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.220+0000 7fab21400700 1 --2- 172.21.15.86:0/1876098417 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558143f538d0 0x558143f4f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:27.221 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.220+0000 7fab1be00700 1 --2- 172.21.15.86:0/1876098417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558143f48a20 0x558143f5a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:27.221 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.220+0000 7fab20a00700 1 -- 172.21.15.86:0/1876098417 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558143f538d0 msgr2=0x558143f4f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:27.221 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.220+0000 7fab20a00700 1 --2- 172.21.15.86:0/1876098417 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558143f538d0 0x558143f4f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:27.221 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.220+0000 7fab20a00700 1 -- 172.21.15.86:0/1876098417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558143f48a20 msgr2=0x558143f5a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:27.221 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.220+0000 7fab20a00700 1 --2- 172.21.15.86:0/1876098417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558143f48a20 0x558143f5a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:27.221 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.220+0000 7fab20a00700 1 -- 172.21.15.86:0/1876098417 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558143e60e40 con 0x558143f48100 2024-03-23T17:52:27.221 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.220+0000 7fab20a00700 1 --2- 172.21.15.86:0/1876098417 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558143f48100 0x558143f6a3a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fab1000e9c0 tx=0x7fab1009a080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e3ecc9cc0dd9830 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:27.224 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.220+0000 7fab0e600700 1 -- 172.21.15.86:0/1876098417 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1009dd30 con 0x558143f48100 2024-03-23T17:52:27.224 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.220+0000 7fab0e600700 1 -- 172.21.15.86:0/1876098417 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab100a2380 con 0x558143f48100 2024-03-23T17:52:27.224 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.220+0000 7fab0e600700 1 -- 172.21.15.86:0/1876098417 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab10099650 con 0x558143f48100 2024-03-23T17:52:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.221+0000 7fab36ed7b80 1 -- 172.21.15.86:0/1876098417 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558143f48100 msgr2=0x558143f6a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.221+0000 7fab36ed7b80 1 --2- 172.21.15.86:0/1876098417 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558143f48100 0x558143f6a3a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fab1000e9c0 tx=0x7fab1009a080 comp rx=0 tx=0).stop 2024-03-23T17:52:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.221+0000 7fab36ed7b80 1 -- 172.21.15.86:0/1876098417 shutdown_connections 2024-03-23T17:52:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.221+0000 7fab36ed7b80 1 --2- 172.21.15.86:0/1876098417 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558143f538d0 0x558143f4f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.221+0000 7fab36ed7b80 1 --2- 172.21.15.86:0/1876098417 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558143f48100 0x558143f6a3a0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.221+0000 7fab36ed7b80 1 --2- 172.21.15.86:0/1876098417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558143f48a20 0x558143f5a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.221+0000 7fab36ed7b80 1 -- 172.21.15.86:0/1876098417 >> 172.21.15.86:0/1876098417 conn(0x558143ea8640 msgr2=0x558143f43d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.221+0000 7fab36ed7b80 1 -- 172.21.15.86:0/1876098417 shutdown_connections 2024-03-23T17:52:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.221+0000 7fab36ed7b80 1 -- 172.21.15.86:0/1876098417 wait complete. 2024-03-23T17:52:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.222+0000 7fab36ed7b80 1 -- start start 2024-03-23T17:52:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.222+0000 7fab36ed7b80 1 Processor -- start 2024-03-23T17:52:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.222+0000 7fab36ed7b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558143f48100 0x558143f587b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.222+0000 7fab36ed7b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558143f48a20 0x558143f58cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.222+0000 7fab36ed7b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558143f538d0 0x558143fb4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.222+0000 7fab36ed7b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558143fb4680 con 0x558143f538d0 2024-03-23T17:52:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.222+0000 7fab36ed7b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x558143fb47c0 con 0x558143f48100 2024-03-23T17:52:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.222+0000 7fab36ed7b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x558143fb4980 con 0x558143f48a20 2024-03-23T17:52:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.222+0000 7fab36ed7b80 10 client.0 _ll_register_callbacks cb 0x558143e40410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:52:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.223+0000 7fab20a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558143f48100 0x558143f587b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.223+0000 7fab1be00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558143f48a20 0x558143f58cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.223+0000 7fab1be00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558143f48a20 0x558143f58cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:42282/0 (socket says 172.21.15.86:42282) 2024-03-23T17:52:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.223+0000 7fab20a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558143f48100 0x558143f587b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:39050/0 (socket says 172.21.15.86:39050) 2024-03-23T17:52:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.223+0000 7fab1be00700 1 -- 172.21.15.86:0/3248706769 learned_addr learned my addr 172.21.15.86:0/3248706769 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.223+0000 7fab21400700 1 --2- 172.21.15.86:0/3248706769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558143f538d0 0x558143fb4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.223+0000 7fab20a00700 1 -- 172.21.15.86:0/3248706769 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558143f48a20 msgr2=0x558143f58cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.223+0000 7fab20a00700 1 --2- 172.21.15.86:0/3248706769 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558143f48a20 0x558143f58cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.223+0000 7fab20a00700 1 -- 172.21.15.86:0/3248706769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558143f538d0 msgr2=0x558143fb4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.223+0000 7fab20a00700 1 --2- 172.21.15.86:0/3248706769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558143f538d0 0x558143fb4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.223+0000 7fab20a00700 1 -- 172.21.15.86:0/3248706769 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558143cc8b00 con 0x558143f48100 2024-03-23T17:52:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.223+0000 7fab20a00700 1 --2- 172.21.15.86:0/3248706769 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558143f48100 0x558143f587b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fab10099e90 tx=0x7fab10009f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=463577d3a7c72c40 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.223+0000 7fab02000700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T17:52:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.223+0000 7fab02000700 1 -- 172.21.15.86:0/3248706769 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1000cd20 con 0x558143f48100 2024-03-23T17:52:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.223+0000 7fab02000700 1 -- 172.21.15.86:0/3248706769 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fab10094b10 con 0x558143f48100 2024-03-23T17:52:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.223+0000 7fab36ed7b80 1 -- 172.21.15.86:0/3248706769 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fab1400f040 con 0x558143f48100 2024-03-23T17:52:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.223+0000 7fab02000700 1 -- 172.21.15.86:0/3248706769 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab1000ce80 con 0x558143f48100 2024-03-23T17:52:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.223+0000 7fab36ed7b80 10 client.26075 fetch_fsmap learned FSMap version 33 2024-03-23T17:52:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.223+0000 7fab36ed7b80 1 -- 172.21.15.86:0/3248706769 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x558143d20c70 con 0x558143f48100 2024-03-23T17:52:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.224+0000 7fab02000700 1 -- 172.21.15.86:0/3248706769 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab1007c080 con 0x558143f48100 2024-03-23T17:52:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.224+0000 7fab02000700 1 -- 172.21.15.86:0/3248706769 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 33) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fab10098e60 con 0x558143f48100 2024-03-23T17:52:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.224+0000 7fab36ed7b80 10 client.26075 fetch_fsmap finished waiting for FSMap version 33 2024-03-23T17:52:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.224+0000 7fab36ed7b80 10 client.26075 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:52:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.224+0000 7fab36ed7b80 20 client.26075 populate_metadata read hostname 'smithi086' 2024-03-23T17:52:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.224+0000 7fab36ed7b80 1 --2- 172.21.15.86:0/3248706769 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x558143fb5d20 0x558143fb8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.224+0000 7fab36ed7b80 4 client.26075 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T17:52:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.224+0000 7fab36ed7b80 1 -- 172.21.15.86:0/3248706769 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558143ce9c90 con 0x558143fb5d20 2024-03-23T17:52:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.224+0000 7fab1be00700 1 --2- 172.21.15.86:0/3248706769 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x558143fb5d20 0x558143fb8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.225+0000 7fab1be00700 1 --2- 172.21.15.86:0/3248706769 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x558143fb5d20 0x558143fb8110 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c72278f633e035dc server_cookie=ca26492a068503fc in_seq=0 out_seq=0 2024-03-23T17:52:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.225+0000 7fab02000700 10 client.26075 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:52:27.255 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.254+0000 7fab02000700 1 -- 172.21.15.86:0/3248706769 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558143ce9c90 con 0x558143fb5d20 2024-03-23T17:52:27.255 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.254+0000 7fab02000700 10 client.26075 handle_command_reply: tid=0 2024-03-23T17:52:27.255 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.254+0000 7fab36ed7b80 1 client.26075 shutdown 2024-03-23T17:52:27.255 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.255+0000 7fab36ed7b80 1 -- 172.21.15.86:0/3248706769 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558143f48100 msgr2=0x558143f587b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:27.255 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.255+0000 7fab36ed7b80 1 --2- 172.21.15.86:0/3248706769 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558143f48100 0x558143f587b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fab10099e90 tx=0x7fab10009f70 comp rx=0 tx=0).stop 2024-03-23T17:52:27.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.255+0000 7fab36ed7b80 1 -- 172.21.15.86:0/3248706769 shutdown_connections 2024-03-23T17:52:27.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.255+0000 7fab36ed7b80 1 --2- 172.21.15.86:0/3248706769 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x558143fb5d20 0x558143fb8110 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:27.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.255+0000 7fab36ed7b80 1 --2- 172.21.15.86:0/3248706769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558143f538d0 0x558143fb4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:27.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.255+0000 7fab02000700 0 client.26075 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:52:27.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.255+0000 7fab36ed7b80 1 --2- 172.21.15.86:0/3248706769 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558143f48100 0x558143f587b0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:27.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.255+0000 7fab36ed7b80 1 --2- 172.21.15.86:0/3248706769 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558143f48a20 0x558143f58cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:27.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.255+0000 7fab36ed7b80 1 -- 172.21.15.86:0/3248706769 >> 172.21.15.86:0/3248706769 conn(0x558143ea8640 msgr2=0x558143f3d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:27.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.255+0000 7fab36ed7b80 1 -- 172.21.15.86:0/3248706769 shutdown_connections 2024-03-23T17:52:27.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.255+0000 7fab36ed7b80 1 -- 172.21.15.86:0/3248706769 wait complete. 2024-03-23T17:52:27.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.255+0000 7fab36ed7b80 20 client.26075 trim_cache size 0 max 16384 2024-03-23T17:52:27.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-03-23T17:52:27.257 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-03-23T17:52:27.257 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-03-23T17:52:27.257 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-03-23T17:52:27.257 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.256+0000 7fab23400700 1 -- 172.21.15.86:0/106962961 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fab040a1290 msgr2=0x7fab040a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:27.257 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.256+0000 7fab23400700 1 --2- 172.21.15.86:0/106962961 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fab040a1290 0x7fab040a3750 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fab1c0747e0 tx=0x7fab0800b280 comp rx=0 tx=0).stop 2024-03-23T17:52:27.257 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.256+0000 7fab23400700 1 -- 172.21.15.86:0/106962961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab1c100f10 msgr2=0x7fab1c0730b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:27.258 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.256+0000 7fab23400700 1 --2- 172.21.15.86:0/106962961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab1c100f10 0x7fab1c0730b0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fab1000e9f0 tx=0x7fab1000ed00 comp rx=0 tx=0).stop 2024-03-23T17:52:27.258 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.256+0000 7fab23400700 1 -- 172.21.15.86:0/106962961 shutdown_connections 2024-03-23T17:52:27.258 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.256+0000 7fab23400700 1 --2- 172.21.15.86:0/106962961 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fab040a1290 0x7fab040a3750 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:27.258 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.256+0000 7fab23400700 1 --2- 172.21.15.86:0/106962961 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fab1c073b30 0x7fab1c1a5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:27.258 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.256+0000 7fab23400700 1 --2- 172.21.15.86:0/106962961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab1c100f10 0x7fab1c0730b0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:27.258 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.256+0000 7fab23400700 1 --2- 172.21.15.86:0/106962961 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab1c106030 0x7fab1c0735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:27.258 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.256+0000 7fab23400700 1 -- 172.21.15.86:0/106962961 >> 172.21.15.86:0/106962961 conn(0x7fab1c0fc7b0 msgr2=0x7fab1c0fdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:27.258 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.256+0000 7fab23400700 1 -- 172.21.15.86:0/106962961 shutdown_connections 2024-03-23T17:52:27.258 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:27.256+0000 7fab23400700 1 -- 172.21.15.86:0/106962961 wait complete. 2024-03-23T17:52:27.882 INFO:tasks.workunit.client.0.smithi086.stdout: 1 43320 4.82 MB/sec warmup 52 sec latency 58.338 ms 2024-03-23T17:52:28.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:27 smithi086 ceph-mon[38221]: pgmap v57: 97 pgs: 97 active+clean; 33 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 737 KiB/s wr, 147 op/s 2024-03-23T17:52:28.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:27 smithi092 ceph-mon[79089]: pgmap v57: 97 pgs: 97 active+clean; 33 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 737 KiB/s wr, 147 op/s 2024-03-23T17:52:28.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:27 smithi152 ceph-mon[47949]: pgmap v57: 97 pgs: 97 active+clean; 33 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 737 KiB/s wr, 147 op/s 2024-03-23T17:52:28.882 INFO:tasks.workunit.client.0.smithi086.stdout: 1 43320 4.73 MB/sec warmup 53 sec latency 1058.400 ms 2024-03-23T17:52:29.882 INFO:tasks.workunit.client.0.smithi086.stdout: 1 43320 4.64 MB/sec warmup 54 sec latency 2058.469 ms 2024-03-23T17:52:30.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:29 smithi086 ceph-mon[38221]: pgmap v58: 97 pgs: 97 active+clean; 33 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 714 KiB/s wr, 133 op/s 2024-03-23T17:52:30.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:29 smithi092 ceph-mon[79089]: pgmap v58: 97 pgs: 97 active+clean; 33 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 714 KiB/s wr, 133 op/s 2024-03-23T17:52:30.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:29 smithi152 ceph-mon[47949]: pgmap v58: 97 pgs: 97 active+clean; 33 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 714 KiB/s wr, 133 op/s 2024-03-23T17:52:30.882 INFO:tasks.workunit.client.0.smithi086.stdout: 1 43320 4.56 MB/sec warmup 55 sec latency 3058.616 ms 2024-03-23T17:52:31.882 INFO:tasks.workunit.client.0.smithi086.stdout: 1 43388 4.48 MB/sec warmup 56 sec latency 3078.667 ms 2024-03-23T17:52:32.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:31 smithi086 ceph-mon[38221]: pgmap v59: 97 pgs: 97 active+clean; 31 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 957 KiB/s wr, 193 op/s 2024-03-23T17:52:32.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:31 smithi092 ceph-mon[79089]: pgmap v59: 97 pgs: 97 active+clean; 31 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 957 KiB/s wr, 193 op/s 2024-03-23T17:52:32.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:31 smithi152 ceph-mon[47949]: pgmap v59: 97 pgs: 97 active+clean; 31 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 957 KiB/s wr, 193 op/s 2024-03-23T17:52:32.883 INFO:tasks.workunit.client.0.smithi086.stdout: 1 43388 4.40 MB/sec warmup 57 sec latency 1923.861 ms 2024-03-23T17:52:33.883 INFO:tasks.workunit.client.0.smithi086.stdout: 1 43388 4.32 MB/sec warmup 58 sec latency 2923.946 ms 2024-03-23T17:52:34.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:33 smithi086 ceph-mon[38221]: pgmap v60: 97 pgs: 97 active+clean; 31 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 506 KiB/s wr, 108 op/s 2024-03-23T17:52:34.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:33 smithi092 ceph-mon[79089]: pgmap v60: 97 pgs: 97 active+clean; 31 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 506 KiB/s wr, 108 op/s 2024-03-23T17:52:34.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:33 smithi152 ceph-mon[47949]: pgmap v60: 97 pgs: 97 active+clean; 31 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 506 KiB/s wr, 108 op/s 2024-03-23T17:52:34.883 INFO:tasks.workunit.client.0.smithi086.stdout: 1 43388 4.25 MB/sec warmup 59 sec latency 3924.033 ms 2024-03-23T17:52:35.883 INFO:tasks.workunit.client.0.smithi086.stdout: 1 43388 4.18 MB/sec warmup 60 sec latency 4924.176 ms 2024-03-23T17:52:36.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:35 smithi086 ceph-mon[38221]: pgmap v61: 97 pgs: 97 active+clean; 31 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 577 KiB/s wr, 115 op/s 2024-03-23T17:52:36.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:35 smithi092 ceph-mon[79089]: pgmap v61: 97 pgs: 97 active+clean; 31 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 577 KiB/s wr, 115 op/s 2024-03-23T17:52:36.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:35 smithi152 ceph-mon[47949]: pgmap v61: 97 pgs: 97 active+clean; 31 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 577 KiB/s wr, 115 op/s 2024-03-23T17:52:36.883 INFO:tasks.workunit.client.0.smithi086.stdout: 1 45807 4.32 MB/sec warmup 61 sec latency 4944.696 ms 2024-03-23T17:52:37.883 INFO:tasks.workunit.client.0.smithi086.stdout: 1 46870 4.35 MB/sec warmup 62 sec latency 144.716 ms 2024-03-23T17:52:38.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:37 smithi086 ceph-mon[38221]: pgmap v62: 97 pgs: 97 active+clean; 32 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 375 KiB/s wr, 74 op/s 2024-03-23T17:52:38.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:37 smithi092 ceph-mon[79089]: pgmap v62: 97 pgs: 97 active+clean; 32 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 375 KiB/s wr, 74 op/s 2024-03-23T17:52:38.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:37 smithi152 ceph-mon[47949]: pgmap v62: 97 pgs: 97 active+clean; 32 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 375 KiB/s wr, 74 op/s 2024-03-23T17:52:38.883 INFO:tasks.workunit.client.0.smithi086.stdout: 1 46870 4.28 MB/sec warmup 63 sec latency 1144.825 ms 2024-03-23T17:52:39.883 INFO:tasks.workunit.client.0.smithi086.stdout: 1 46870 4.22 MB/sec warmup 64 sec latency 2144.966 ms 2024-03-23T17:52:40.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:39 smithi086 ceph-mon[38221]: pgmap v63: 97 pgs: 97 active+clean; 32 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 343 KiB/s wr, 70 op/s 2024-03-23T17:52:40.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:39 smithi092 ceph-mon[79089]: pgmap v63: 97 pgs: 97 active+clean; 32 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 343 KiB/s wr, 70 op/s 2024-03-23T17:52:40.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:39 smithi152 ceph-mon[47949]: pgmap v63: 97 pgs: 97 active+clean; 32 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 343 KiB/s wr, 70 op/s 2024-03-23T17:52:40.408 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.408+0000 7f44edc00700 1 -- 172.21.15.86:0/2814898541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44e8106010 msgr2=0x7f44e810db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:40.409 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.408+0000 7f44edc00700 1 --2- 172.21.15.86:0/2814898541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44e8106010 0x7f44e810db90 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f44e000b0a0 tx=0x7f44e000b3b0 comp rx=0 tx=0).stop 2024-03-23T17:52:40.409 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.408+0000 7f44edc00700 1 -- 172.21.15.86:0/2814898541 shutdown_connections 2024-03-23T17:52:40.409 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.408+0000 7f44edc00700 1 --2- 172.21.15.86:0/2814898541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44e8106010 0x7f44e810db90 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:40.409 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.408+0000 7f44edc00700 1 --2- 172.21.15.86:0/2814898541 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f44e8100ef0 0x7f44e81012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:40.409 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.408+0000 7f44edc00700 1 --2- 172.21.15.86:0/2814898541 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44e81018a0 0x7f44e81058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:40.409 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.408+0000 7f44edc00700 1 -- 172.21.15.86:0/2814898541 >> 172.21.15.86:0/2814898541 conn(0x7f44e80fc750 msgr2=0x7f44e80feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:40.409 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.408+0000 7f44edc00700 1 -- 172.21.15.86:0/2814898541 shutdown_connections 2024-03-23T17:52:40.409 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.408+0000 7f44edc00700 1 -- 172.21.15.86:0/2814898541 wait complete. 2024-03-23T17:52:40.410 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.409+0000 7f44edc00700 1 Processor -- start 2024-03-23T17:52:40.410 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.409+0000 7f44edc00700 1 -- start start 2024-03-23T17:52:40.410 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.410+0000 7f44edc00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f44e8100ef0 0x7f44e80730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:40.411 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.410+0000 7f44edc00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44e8106010 0x7f44e80735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:40.411 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.410+0000 7f44edc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44e8073b30 0x7f44e81a5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:40.411 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.410+0000 7f44edc00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e81a5a10 con 0x7f44e8073b30 2024-03-23T17:52:40.411 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.410+0000 7f44edc00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e81a5b80 con 0x7f44e8106010 2024-03-23T17:52:40.411 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.410+0000 7f44edc00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e81a5d40 con 0x7f44e8100ef0 2024-03-23T17:52:40.411 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.410+0000 7f44e6a00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f44e8100ef0 0x7f44e80730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:40.411 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.410+0000 7f44e6a00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f44e8100ef0 0x7f44e80730b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:52262/0 (socket says 172.21.15.86:52262) 2024-03-23T17:52:40.411 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.410+0000 7f44e6a00700 1 -- 172.21.15.86:0/4201430170 learned_addr learned my addr 172.21.15.86:0/4201430170 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:40.411 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.410+0000 7f44e7400700 1 --2- 172.21.15.86:0/4201430170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44e8073b30 0x7f44e81a5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:40.411 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.410+0000 7f44dfe00700 1 --2- 172.21.15.86:0/4201430170 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44e8106010 0x7f44e80735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:40.411 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.410+0000 7f44e6a00700 1 -- 172.21.15.86:0/4201430170 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44e8106010 msgr2=0x7f44e80735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:40.411 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.410+0000 7f44e6a00700 1 --2- 172.21.15.86:0/4201430170 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44e8106010 0x7f44e80735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:40.411 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.410+0000 7f44e6a00700 1 -- 172.21.15.86:0/4201430170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44e8073b30 msgr2=0x7f44e81a5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:40.411 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.410+0000 7f44e6a00700 1 --2- 172.21.15.86:0/4201430170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44e8073b30 0x7f44e81a5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:40.412 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.411+0000 7f44e6a00700 1 -- 172.21.15.86:0/4201430170 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44e0009d00 con 0x7f44e8100ef0 2024-03-23T17:52:40.412 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.411+0000 7f44e6a00700 1 --2- 172.21.15.86:0/4201430170 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f44e8100ef0 0x7f44e80730b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f44d800e9a0 tx=0x7f44d800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:40.412 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.411+0000 7f44df400700 1 -- 172.21.15.86:0/4201430170 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d8009790 con 0x7f44e8100ef0 2024-03-23T17:52:40.412 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.411+0000 7f44edc00700 1 -- 172.21.15.86:0/4201430170 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44e81a5fc0 con 0x7f44e8100ef0 2024-03-23T17:52:40.413 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.411+0000 7f44df400700 1 -- 172.21.15.86:0/4201430170 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f44d8010430 con 0x7f44e8100ef0 2024-03-23T17:52:40.413 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.411+0000 7f44df400700 1 -- 172.21.15.86:0/4201430170 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d8018680 con 0x7f44e8100ef0 2024-03-23T17:52:40.413 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.411+0000 7f44edc00700 1 -- 172.21.15.86:0/4201430170 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44e81a64a0 con 0x7f44e8100ef0 2024-03-23T17:52:40.416 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.416+0000 7f44df400700 1 -- 172.21.15.86:0/4201430170 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f44d80105a0 con 0x7f44e8100ef0 2024-03-23T17:52:40.416 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.416+0000 7f4501704b80 1 Processor -- start 2024-03-23T17:52:40.416 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.417+0000 7f4501704b80 1 -- start start 2024-03-23T17:52:40.418 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.417+0000 7f4501704b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f55aa3100 0x561f55aa34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:40.418 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.417+0000 7f4501704b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561f55aa3a20 0x561f55aae390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:40.419 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.417+0000 7f4501704b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561f55aae8d0 0x561f55ab0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:40.419 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.417+0000 7f4501704b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f559fc930 con 0x561f55aa3100 2024-03-23T17:52:40.419 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.417+0000 7f4501704b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f559fa2f0 con 0x561f55aae8d0 2024-03-23T17:52:40.419 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.417+0000 7f4501704b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f559f5be0 con 0x561f55aa3a20 2024-03-23T17:52:40.419 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.417+0000 7f44e7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f55aa3100 0x561f55aa34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:40.419 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.417+0000 7f44e6a00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561f55aa3a20 0x561f55aae390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:40.419 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.417+0000 7f44e7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f55aa3100 0x561f55aa34e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34726/0 (socket says 172.21.15.86:34726) 2024-03-23T17:52:40.419 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.417+0000 7f44e7400700 1 -- 172.21.15.86:0/1293183840 learned_addr learned my addr 172.21.15.86:0/1293183840 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:40.419 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.417+0000 7f44dfe00700 1 --2- 172.21.15.86:0/1293183840 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561f55aae8d0 0x561f55ab0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:40.419 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.418+0000 7f44df400700 1 --2- 172.21.15.86:0/4201430170 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f44cc0a1290 0x7f44cc0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:40.419 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.418+0000 7f44df400700 1 -- 172.21.15.86:0/4201430170 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f44d8014070 con 0x7f44e8100ef0 2024-03-23T17:52:40.419 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.418+0000 7f44dfe00700 1 -- 172.21.15.86:0/1293183840 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561f55aa3a20 msgr2=0x561f55aae390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:40.419 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.418+0000 7f44dfe00700 1 --2- 172.21.15.86:0/1293183840 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561f55aa3a20 0x561f55aae390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:40.419 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.418+0000 7f44dfe00700 1 -- 172.21.15.86:0/1293183840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f55aa3100 msgr2=0x561f55aa34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:40.420 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.418+0000 7f44dfe00700 1 --2- 172.21.15.86:0/1293183840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f55aa3100 0x561f55aa34e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:40.420 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.418+0000 7f44e7400700 1 --2- 172.21.15.86:0/4201430170 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f44cc0a1290 0x7f44cc0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:40.420 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.418+0000 7f44dfe00700 1 -- 172.21.15.86:0/1293183840 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f55877350 con 0x561f55aae8d0 2024-03-23T17:52:40.420 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.418+0000 7f44e6a00700 1 --2- 172.21.15.86:0/1293183840 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561f55aa3a20 0x561f55aae390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:52:40.420 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.418+0000 7f44dfe00700 1 --2- 172.21.15.86:0/1293183840 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561f55aae8d0 0x561f55ab0cc0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f44e806a370 tx=0x7f44d0009b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d56160bf82ab6138 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:40.420 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.419+0000 7f44d7e00700 1 -- 172.21.15.86:0/1293183840 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d001c070 con 0x561f55aae8d0 2024-03-23T17:52:40.420 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.419+0000 7f44d7e00700 1 -- 172.21.15.86:0/1293183840 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f44d0004830 con 0x561f55aae8d0 2024-03-23T17:52:40.420 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.419+0000 7f44d7e00700 1 -- 172.21.15.86:0/1293183840 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d00053a0 con 0x561f55aae8d0 2024-03-23T17:52:40.423 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.419+0000 7f4501704b80 1 -- 172.21.15.86:0/1293183840 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561f55aae8d0 msgr2=0x561f55ab0cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:40.423 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.419+0000 7f44e7400700 1 --2- 172.21.15.86:0/4201430170 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f44cc0a1290 0x7f44cc0a3750 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f44e0009cd0 tx=0x7f44e000b3b0 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:40.423 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.419+0000 7f4501704b80 1 --2- 172.21.15.86:0/1293183840 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561f55aae8d0 0x561f55ab0cc0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f44e806a370 tx=0x7f44d0009b10 comp rx=0 tx=0).stop 2024-03-23T17:52:40.423 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.419+0000 7f4501704b80 1 -- 172.21.15.86:0/1293183840 shutdown_connections 2024-03-23T17:52:40.423 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.419+0000 7f4501704b80 1 --2- 172.21.15.86:0/1293183840 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561f55aae8d0 0x561f55ab0cc0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:40.424 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.419+0000 7f4501704b80 1 --2- 172.21.15.86:0/1293183840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f55aa3100 0x561f55aa34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:40.424 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.419+0000 7f4501704b80 1 --2- 172.21.15.86:0/1293183840 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561f55aa3a20 0x561f55aae390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:40.424 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.419+0000 7f4501704b80 1 -- 172.21.15.86:0/1293183840 >> 172.21.15.86:0/1293183840 conn(0x561f55a03640 msgr2=0x561f55a9f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:40.424 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.419+0000 7f4501704b80 1 -- 172.21.15.86:0/1293183840 shutdown_connections 2024-03-23T17:52:40.424 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.420+0000 7f4501704b80 1 -- 172.21.15.86:0/1293183840 wait complete. 2024-03-23T17:52:40.424 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.420+0000 7f4501704b80 1 -- start start 2024-03-23T17:52:40.424 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.420+0000 7f4501704b80 1 Processor -- start 2024-03-23T17:52:40.424 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.421+0000 7f4501704b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561f55aa3100 0x561f55aae2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:40.425 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.421+0000 7f4501704b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561f55aa3a20 0x561f55aab5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:40.425 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.421+0000 7f4501704b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f55aae8d0 0x561f55aabc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:40.425 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.421+0000 7f4501704b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f55991d20 con 0x561f55aae8d0 2024-03-23T17:52:40.425 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.421+0000 7f4501704b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f559919c0 con 0x561f55aa3100 2024-03-23T17:52:40.425 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.421+0000 7f4501704b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f55879640 con 0x561f55aa3a20 2024-03-23T17:52:40.425 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.421+0000 7f4501704b80 10 client.0 _ll_register_callbacks cb 0x561f5599b410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:52:40.425 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.421+0000 7f44e6a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561f55aa3100 0x561f55aae2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:40.425 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.421+0000 7f44e7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f55aae8d0 0x561f55aabc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:40.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.421+0000 7f44e6a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561f55aa3100 0x561f55aae2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:45726/0 (socket says 172.21.15.86:45726) 2024-03-23T17:52:40.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.421+0000 7f44e6a00700 1 -- 172.21.15.86:0/2749127821 learned_addr learned my addr 172.21.15.86:0/2749127821 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:40.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.421+0000 7f44dfe00700 1 --2- 172.21.15.86:0/2749127821 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561f55aa3a20 0x561f55aab5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:40.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.421+0000 7f44e7400700 1 -- 172.21.15.86:0/2749127821 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561f55aa3a20 msgr2=0x561f55aab5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:40.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.421+0000 7f44e7400700 1 --2- 172.21.15.86:0/2749127821 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561f55aa3a20 0x561f55aab5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:40.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.421+0000 7f44e7400700 1 -- 172.21.15.86:0/2749127821 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561f55aa3100 msgr2=0x561f55aae2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:40.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.421+0000 7f44e7400700 1 --2- 172.21.15.86:0/2749127821 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561f55aa3100 0x561f55aae2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:40.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.421+0000 7f44e7400700 1 -- 172.21.15.86:0/2749127821 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44e81a64a0 con 0x561f55aae8d0 2024-03-23T17:52:40.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.422+0000 7f44e7400700 1 --2- 172.21.15.86:0/2749127821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f55aae8d0 0x561f55aabc00 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f44e000d010 tx=0x7f44e00099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcfe37fd3825b6d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:40.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.422+0000 7f44d4c00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-23T17:52:40.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.422+0000 7f44d4c00700 1 -- 172.21.15.86:0/2749127821 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e000e040 con 0x561f55aae8d0 2024-03-23T17:52:40.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.422+0000 7f44d4c00700 1 -- 172.21.15.86:0/2749127821 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f44e00077c0 con 0x561f55aae8d0 2024-03-23T17:52:40.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.422+0000 7f4501704b80 1 -- 172.21.15.86:0/2749127821 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561f559b6320 con 0x561f55aae8d0 2024-03-23T17:52:40.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.422+0000 7f44d4c00700 1 -- 172.21.15.86:0/2749127821 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f44e0020480 con 0x561f55aae8d0 2024-03-23T17:52:40.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.422+0000 7f4501704b80 10 client.16032 fetch_fsmap learned FSMap version 33 2024-03-23T17:52:40.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.422+0000 7f4501704b80 1 -- 172.21.15.86:0/2749127821 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7f44d0017c80 con 0x561f55aae8d0 2024-03-23T17:52:40.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.422+0000 7f44d4c00700 1 -- 172.21.15.86:0/2749127821 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561f559b6320 con 0x561f55aae8d0 2024-03-23T17:52:40.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.423+0000 7f44d4c00700 1 -- 172.21.15.86:0/2749127821 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 33) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f44e0019070 con 0x561f55aae8d0 2024-03-23T17:52:40.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.423+0000 7f4501704b80 10 client.16032 fetch_fsmap finished waiting for FSMap version 33 2024-03-23T17:52:40.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.423+0000 7f4501704b80 10 client.16032 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:52:40.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.423+0000 7f4501704b80 20 client.16032 populate_metadata read hostname 'smithi086' 2024-03-23T17:52:40.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.423+0000 7f4501704b80 1 --2- 172.21.15.86:0/2749127821 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x561f55abe640 0x561f55ac0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:40.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.423+0000 7f4501704b80 4 client.16032 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:52:40.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.423+0000 7f4501704b80 1 -- 172.21.15.86:0/2749127821 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561f559ee6a0 con 0x561f55abe640 2024-03-23T17:52:40.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.423+0000 7f44e6a00700 1 --2- 172.21.15.86:0/2749127821 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x561f55abe640 0x561f55ac0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:40.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.423+0000 7f44d4c00700 10 client.16032 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:52:40.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.423+0000 7f44e6a00700 1 --2- 172.21.15.86:0/2749127821 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x561f55abe640 0x561f55ac0a60 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=352ddf95d8ada09 server_cookie=dcf7676b79735ab8 in_seq=0 out_seq=0 2024-03-23T17:52:40.429 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.424+0000 7f44d4c00700 1 -- 172.21.15.86:0/2749127821 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x561f559ee6a0 con 0x561f55abe640 2024-03-23T17:52:40.429 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.424+0000 7f44d4c00700 10 client.16032 handle_command_reply: tid=0 2024-03-23T17:52:40.429 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.424+0000 7f4501704b80 1 client.16032 shutdown 2024-03-23T17:52:40.429 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.424+0000 7f4501704b80 1 -- 172.21.15.86:0/2749127821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f55aae8d0 msgr2=0x561f55aabc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:40.429 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.424+0000 7f4501704b80 1 --2- 172.21.15.86:0/2749127821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f55aae8d0 0x561f55aabc00 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f44e000d010 tx=0x7f44e00099e0 comp rx=0 tx=0).stop 2024-03-23T17:52:40.429 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.424+0000 7f4501704b80 1 -- 172.21.15.86:0/2749127821 shutdown_connections 2024-03-23T17:52:40.429 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.424+0000 7f4501704b80 1 --2- 172.21.15.86:0/2749127821 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x561f55abe640 0x561f55ac0a60 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:40.429 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.424+0000 7f4501704b80 1 --2- 172.21.15.86:0/2749127821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f55aae8d0 0x561f55aabc00 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:40.429 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.424+0000 7f4501704b80 1 --2- 172.21.15.86:0/2749127821 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561f55aa3100 0x561f55aae2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:40.429 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.424+0000 7f44d4c00700 0 client.16032 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:52:40.429 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.424+0000 7f4501704b80 1 --2- 172.21.15.86:0/2749127821 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561f55aa3a20 0x561f55aab5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:40.429 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.424+0000 7f4501704b80 1 -- 172.21.15.86:0/2749127821 >> 172.21.15.86:0/2749127821 conn(0x561f55a03640 msgr2=0x561f55aa50a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:40.430 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.425+0000 7f4501704b80 1 -- 172.21.15.86:0/2749127821 shutdown_connections 2024-03-23T17:52:40.430 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.425+0000 7f4501704b80 1 -- 172.21.15.86:0/2749127821 wait complete. 2024-03-23T17:52:40.430 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.425+0000 7f4501704b80 20 client.16032 trim_cache size 0 max 16384 2024-03-23T17:52:40.440 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.439+0000 7f4501704b80 1 Processor -- start 2024-03-23T17:52:40.440 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.439+0000 7f4501704b80 1 -- start start 2024-03-23T17:52:40.440 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.439+0000 7f4501704b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561f55aa3100 0x561f55aa6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:40.440 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.439+0000 7f4501704b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f55aa3a20 0x561f55ac53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:40.440 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.439+0000 7f4501704b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561f55aae8d0 0x561f55aaa4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:40.440 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.440+0000 7f4501704b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f55aa7280 con 0x561f55aa3a20 2024-03-23T17:52:40.440 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.440+0000 7f4501704b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f55aa3e00 con 0x561f55aa3100 2024-03-23T17:52:40.440 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.440+0000 7f4501704b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f55abbc80 con 0x561f55aae8d0 2024-03-23T17:52:40.441 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.440+0000 7f44dfe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f55aa3a20 0x561f55ac53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:40.441 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.440+0000 7f44dfe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f55aa3a20 0x561f55ac53a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34738/0 (socket says 172.21.15.86:34738) 2024-03-23T17:52:40.441 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.440+0000 7f44dfe00700 1 -- 172.21.15.86:0/877038133 learned_addr learned my addr 172.21.15.86:0/877038133 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:40.441 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.440+0000 7f44e6a00700 1 --2- 172.21.15.86:0/877038133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561f55aa3100 0x561f55aa6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:40.441 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.440+0000 7f44e7400700 1 --2- 172.21.15.86:0/877038133 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561f55aae8d0 0x561f55aaa4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:40.441 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.440+0000 7f44dfe00700 1 -- 172.21.15.86:0/877038133 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561f55aae8d0 msgr2=0x561f55aaa4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:40.441 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.440+0000 7f44dfe00700 1 --2- 172.21.15.86:0/877038133 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561f55aae8d0 0x561f55aaa4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:40.441 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.440+0000 7f44dfe00700 1 -- 172.21.15.86:0/877038133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561f55aa3100 msgr2=0x561f55aa6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:40.441 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.440+0000 7f44dfe00700 1 --2- 172.21.15.86:0/877038133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561f55aa3100 0x561f55aa6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:40.441 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.440+0000 7f44dfe00700 1 -- 172.21.15.86:0/877038133 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f55823b00 con 0x561f55aa3a20 2024-03-23T17:52:40.441 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.440+0000 7f44e6a00700 1 --2- 172.21.15.86:0/877038133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561f55aa3100 0x561f55aa6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:52:40.445 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.440+0000 7f44dfe00700 1 --2- 172.21.15.86:0/877038133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f55aa3a20 0x561f55ac53a0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f44e806a370 tx=0x7f44d0005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=427e12d33c14b4cb server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:40.445 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.441+0000 7f44dcc00700 1 -- 172.21.15.86:0/877038133 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d001c050 con 0x561f55aa3a20 2024-03-23T17:52:40.445 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.441+0000 7f44dcc00700 1 -- 172.21.15.86:0/877038133 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f44d0027040 con 0x561f55aa3a20 2024-03-23T17:52:40.445 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.441+0000 7f44dcc00700 1 -- 172.21.15.86:0/877038133 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d00217a0 con 0x561f55aa3a20 2024-03-23T17:52:40.446 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.441+0000 7f4501704b80 1 -- 172.21.15.86:0/877038133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f55aa3a20 msgr2=0x561f55ac53a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:40.446 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.441+0000 7f4501704b80 1 --2- 172.21.15.86:0/877038133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f55aa3a20 0x561f55ac53a0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f44e806a370 tx=0x7f44d0005d90 comp rx=0 tx=0).stop 2024-03-23T17:52:40.446 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.441+0000 7f4501704b80 1 -- 172.21.15.86:0/877038133 shutdown_connections 2024-03-23T17:52:40.446 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.441+0000 7f4501704b80 1 --2- 172.21.15.86:0/877038133 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561f55aae8d0 0x561f55aaa4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:40.446 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.441+0000 7f4501704b80 1 --2- 172.21.15.86:0/877038133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561f55aa3100 0x561f55aa6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:40.446 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.441+0000 7f4501704b80 1 --2- 172.21.15.86:0/877038133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f55aa3a20 0x561f55ac53a0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:40.446 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.441+0000 7f4501704b80 1 -- 172.21.15.86:0/877038133 >> 172.21.15.86:0/877038133 conn(0x561f55a03640 msgr2=0x561f55a983e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:40.446 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.441+0000 7f4501704b80 1 -- 172.21.15.86:0/877038133 shutdown_connections 2024-03-23T17:52:40.446 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.441+0000 7f4501704b80 1 -- 172.21.15.86:0/877038133 wait complete. 2024-03-23T17:52:40.446 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.442+0000 7f4501704b80 1 -- start start 2024-03-23T17:52:40.446 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.442+0000 7f4501704b80 1 Processor -- start 2024-03-23T17:52:40.446 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.442+0000 7f4501704b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561f55aa3100 0x561f55ab37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:40.446 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.442+0000 7f4501704b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f55aa3a20 0x561f55ab3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:40.447 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.442+0000 7f4501704b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561f55aae8d0 0x561f55b09180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:40.447 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.442+0000 7f4501704b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f55b096c0 con 0x561f55aa3a20 2024-03-23T17:52:40.447 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.442+0000 7f4501704b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f55b09830 con 0x561f55aa3100 2024-03-23T17:52:40.447 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.442+0000 7f4501704b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f55b099f0 con 0x561f55aae8d0 2024-03-23T17:52:40.447 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.443+0000 7f4501704b80 10 client.0 _ll_register_callbacks cb 0x561f5599b410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:52:40.447 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.443+0000 7f44dfe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f55aa3a20 0x561f55ab3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:40.447 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.443+0000 7f44dfe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f55aa3a20 0x561f55ab3d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34752/0 (socket says 172.21.15.86:34752) 2024-03-23T17:52:40.447 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.443+0000 7f44dfe00700 1 -- 172.21.15.86:0/3187554851 learned_addr learned my addr 172.21.15.86:0/3187554851 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:40.447 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.443+0000 7f44e6a00700 1 --2- 172.21.15.86:0/3187554851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561f55aa3100 0x561f55ab37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:40.447 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.443+0000 7f44e7400700 1 --2- 172.21.15.86:0/3187554851 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561f55aae8d0 0x561f55b09180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:40.447 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.443+0000 7f44dfe00700 1 -- 172.21.15.86:0/3187554851 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561f55aae8d0 msgr2=0x561f55b09180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:40.447 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.443+0000 7f44dfe00700 1 --2- 172.21.15.86:0/3187554851 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561f55aae8d0 0x561f55b09180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:40.447 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.443+0000 7f44dfe00700 1 -- 172.21.15.86:0/3187554851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561f55aa3100 msgr2=0x561f55ab37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:40.447 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.443+0000 7f44dfe00700 1 --2- 172.21.15.86:0/3187554851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561f55aa3100 0x561f55ab37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:40.448 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.443+0000 7f44dfe00700 1 -- 172.21.15.86:0/3187554851 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44e81a5fc0 con 0x561f55aa3a20 2024-03-23T17:52:40.448 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.443+0000 7f44e6a00700 1 --2- 172.21.15.86:0/3187554851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561f55aa3100 0x561f55ab37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:52:40.448 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.443+0000 7f44dfe00700 1 --2- 172.21.15.86:0/3187554851 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f55aa3a20 0x561f55ab3d20 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f44d0000c00 tx=0x7f44d0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb6cdbb461e8b3b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:40.448 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.443+0000 7f44d4c00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-23T17:52:40.448 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.444+0000 7f44d4c00700 1 -- 172.21.15.86:0/3187554851 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d0021670 con 0x561f55aa3a20 2024-03-23T17:52:40.448 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.444+0000 7f4501704b80 1 -- 172.21.15.86:0/3187554851 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f44d0004830 con 0x561f55aa3a20 2024-03-23T17:52:40.448 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.444+0000 7f44d4c00700 1 -- 172.21.15.86:0/3187554851 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f44d00217d0 con 0x561f55aa3a20 2024-03-23T17:52:40.448 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.444+0000 7f44d4c00700 1 -- 172.21.15.86:0/3187554851 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f44d00046c0 con 0x561f55aa3a20 2024-03-23T17:52:40.448 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.444+0000 7f4501704b80 10 client.16038 fetch_fsmap learned FSMap version 33 2024-03-23T17:52:40.448 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.444+0000 7f4501704b80 1 -- 172.21.15.86:0/3187554851 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x561f55877350 con 0x561f55aa3a20 2024-03-23T17:52:40.448 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.444+0000 7f44d4c00700 1 -- 172.21.15.86:0/3187554851 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f44d0043020 con 0x561f55aa3a20 2024-03-23T17:52:40.448 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.444+0000 7f44d4c00700 1 -- 172.21.15.86:0/3187554851 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 33) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f44d003f980 con 0x561f55aa3a20 2024-03-23T17:52:40.448 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.444+0000 7f4501704b80 10 client.16038 fetch_fsmap finished waiting for FSMap version 33 2024-03-23T17:52:40.448 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.444+0000 7f4501704b80 10 client.16038 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:52:40.449 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.444+0000 7f4501704b80 20 client.16038 populate_metadata read hostname 'smithi086' 2024-03-23T17:52:40.449 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.445+0000 7f4501704b80 1 --2- 172.21.15.86:0/3187554851 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x561f55b10df0 0x561f55b13210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:40.449 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.445+0000 7f4501704b80 4 client.16038 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T17:52:40.449 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.445+0000 7f4501704b80 1 -- 172.21.15.86:0/3187554851 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561f55844c90 con 0x561f55b10df0 2024-03-23T17:52:40.449 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.445+0000 7f44e6a00700 1 --2- 172.21.15.86:0/3187554851 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x561f55b10df0 0x561f55b13210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:40.449 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.445+0000 7f44e6a00700 1 --2- 172.21.15.86:0/3187554851 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x561f55b10df0 0x561f55b13210 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c876479078e5b7c server_cookie=cf9866047e45303b in_seq=0 out_seq=0 2024-03-23T17:52:40.449 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.445+0000 7f44d4c00700 10 client.16038 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:52:40.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.470+0000 7f44d4c00700 1 -- 172.21.15.86:0/3187554851 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561f55844c90 con 0x561f55b10df0 2024-03-23T17:52:40.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.470+0000 7f44d4c00700 10 client.16038 handle_command_reply: tid=0 2024-03-23T17:52:40.471 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.470+0000 7f4501704b80 1 client.16038 shutdown 2024-03-23T17:52:40.471 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.470+0000 7f4501704b80 1 -- 172.21.15.86:0/3187554851 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f55aa3a20 msgr2=0x561f55ab3d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:40.471 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.470+0000 7f4501704b80 1 --2- 172.21.15.86:0/3187554851 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f55aa3a20 0x561f55ab3d20 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f44d0000c00 tx=0x7f44d0009780 comp rx=0 tx=0).stop 2024-03-23T17:52:40.471 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.470+0000 7f4501704b80 1 -- 172.21.15.86:0/3187554851 shutdown_connections 2024-03-23T17:52:40.471 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.470+0000 7f4501704b80 1 --2- 172.21.15.86:0/3187554851 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x561f55b10df0 0x561f55b13210 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:40.471 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.470+0000 7f4501704b80 1 --2- 172.21.15.86:0/3187554851 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561f55aae8d0 0x561f55b09180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:40.471 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.470+0000 7f4501704b80 1 --2- 172.21.15.86:0/3187554851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561f55aa3100 0x561f55ab37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:40.471 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.470+0000 7f4501704b80 1 --2- 172.21.15.86:0/3187554851 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f55aa3a20 0x561f55ab3d20 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:40.471 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.470+0000 7f44d4c00700 0 client.16038 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:52:40.471 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.470+0000 7f4501704b80 1 -- 172.21.15.86:0/3187554851 >> 172.21.15.86:0/3187554851 conn(0x561f55a03640 msgr2=0x561f55a9f450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:40.471 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.470+0000 7f4501704b80 1 -- 172.21.15.86:0/3187554851 shutdown_connections 2024-03-23T17:52:40.471 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.470+0000 7f4501704b80 1 -- 172.21.15.86:0/3187554851 wait complete. 2024-03-23T17:52:40.471 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.470+0000 7f4501704b80 20 client.16038 trim_cache size 0 max 16384 2024-03-23T17:52:40.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-03-23T17:52:40.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-03-23T17:52:40.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-03-23T17:52:40.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-03-23T17:52:40.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.471+0000 7f44edc00700 1 -- 172.21.15.86:0/4201430170 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f44cc0a1290 msgr2=0x7f44cc0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:40.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.471+0000 7f44edc00700 1 --2- 172.21.15.86:0/4201430170 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f44cc0a1290 0x7f44cc0a3750 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f44e0009cd0 tx=0x7f44e000b3b0 comp rx=0 tx=0).stop 2024-03-23T17:52:40.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.471+0000 7f44edc00700 1 -- 172.21.15.86:0/4201430170 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f44e8100ef0 msgr2=0x7f44e80730b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:40.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.471+0000 7f44edc00700 1 --2- 172.21.15.86:0/4201430170 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f44e8100ef0 0x7f44e80730b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f44d800e9a0 tx=0x7f44d800ecb0 comp rx=0 tx=0).stop 2024-03-23T17:52:40.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.471+0000 7f44edc00700 1 -- 172.21.15.86:0/4201430170 shutdown_connections 2024-03-23T17:52:40.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.471+0000 7f44edc00700 1 --2- 172.21.15.86:0/4201430170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44e8073b30 0x7f44e81a5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:40.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.471+0000 7f44edc00700 1 --2- 172.21.15.86:0/4201430170 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f44cc0a1290 0x7f44cc0a3750 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:40.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.471+0000 7f44edc00700 1 --2- 172.21.15.86:0/4201430170 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f44e8100ef0 0x7f44e80730b0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:40.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.471+0000 7f44edc00700 1 --2- 172.21.15.86:0/4201430170 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44e8106010 0x7f44e80735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:40.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.471+0000 7f44edc00700 1 -- 172.21.15.86:0/4201430170 >> 172.21.15.86:0/4201430170 conn(0x7f44e80fc750 msgr2=0x7f44e80fea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:40.474 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.471+0000 7f44edc00700 1 -- 172.21.15.86:0/4201430170 shutdown_connections 2024-03-23T17:52:40.474 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:40.471+0000 7f44edc00700 1 -- 172.21.15.86:0/4201430170 wait complete. 2024-03-23T17:52:40.622 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-23T17:52:40.622 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-23T17:52:40.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.837+0000 7feac7c00700 1 -- 172.21.15.86:0/4215848254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feac0106080 msgr2=0x7feac010dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:40.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.837+0000 7feac7c00700 1 --2- 172.21.15.86:0/4215848254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feac0106080 0x7feac010dc00 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7feab800b0a0 tx=0x7feab800b3b0 comp rx=0 tx=0).stop 2024-03-23T17:52:40.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.837+0000 7feac7c00700 1 -- 172.21.15.86:0/4215848254 shutdown_connections 2024-03-23T17:52:40.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.837+0000 7feac7c00700 1 --2- 172.21.15.86:0/4215848254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feac0106080 0x7feac010dc00 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:40.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.837+0000 7feac7c00700 1 --2- 172.21.15.86:0/4215848254 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feac0100f60 0x7feac0101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:40.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.837+0000 7feac7c00700 1 --2- 172.21.15.86:0/4215848254 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feac0101910 0x7feac0105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:40.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.837+0000 7feac7c00700 1 -- 172.21.15.86:0/4215848254 >> 172.21.15.86:0/4215848254 conn(0x7feac00fc7c0 msgr2=0x7feac00febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:40.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.837+0000 7feac7c00700 1 -- 172.21.15.86:0/4215848254 shutdown_connections 2024-03-23T17:52:40.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.837+0000 7feac7c00700 1 -- 172.21.15.86:0/4215848254 wait complete. 2024-03-23T17:52:40.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.838+0000 7feac7c00700 1 Processor -- start 2024-03-23T17:52:40.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.838+0000 7feac7c00700 1 -- start start 2024-03-23T17:52:40.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.838+0000 7feac7c00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feac0100f60 0x7feac00730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:40.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.839+0000 7feac7c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feac0106080 0x7feac00735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:40.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.839+0000 7feac7c00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feac0073b30 0x7feac01a5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:40.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.839+0000 7feac7c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac01a5b40 con 0x7feac0106080 2024-03-23T17:52:40.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.839+0000 7feac7c00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac01a5c80 con 0x7feac0073b30 2024-03-23T17:52:40.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.839+0000 7feac7c00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac01a5e40 con 0x7feac0100f60 2024-03-23T17:52:40.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.839+0000 7feabfe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feac0106080 0x7feac00735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:40.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.839+0000 7feabfe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feac0106080 0x7feac00735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34770/0 (socket says 172.21.15.86:34770) 2024-03-23T17:52:40.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.839+0000 7feabfe00700 1 -- 172.21.15.86:0/47722065 learned_addr learned my addr 172.21.15.86:0/47722065 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:40.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.839+0000 7feac5200700 1 --2- 172.21.15.86:0/47722065 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feac0100f60 0x7feac00730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:40.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.839+0000 7feac5c00700 1 --2- 172.21.15.86:0/47722065 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feac0073b30 0x7feac01a5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:40.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.839+0000 7feabfe00700 1 -- 172.21.15.86:0/47722065 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feac0100f60 msgr2=0x7feac00730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:40.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.839+0000 7feabfe00700 1 --2- 172.21.15.86:0/47722065 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feac0100f60 0x7feac00730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:40.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.839+0000 7feabfe00700 1 -- 172.21.15.86:0/47722065 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feac0073b30 msgr2=0x7feac01a5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:40.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.839+0000 7feabfe00700 1 --2- 172.21.15.86:0/47722065 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feac0073b30 0x7feac01a5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:40.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.839+0000 7feabfe00700 1 -- 172.21.15.86:0/47722065 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feab8009d00 con 0x7feac0106080 2024-03-23T17:52:40.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.839+0000 7feabfe00700 1 --2- 172.21.15.86:0/47722065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feac0106080 0x7feac00735f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7feaac00e970 tx=0x7feaac00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:40.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.840+0000 7feabd600700 1 -- 172.21.15.86:0/47722065 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaac009790 con 0x7feac0106080 2024-03-23T17:52:40.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.840+0000 7feabd600700 1 -- 172.21.15.86:0/47722065 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feaac00c7e0 con 0x7feac0106080 2024-03-23T17:52:40.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.840+0000 7feac7c00700 1 -- 172.21.15.86:0/47722065 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feac01a6120 con 0x7feac0106080 2024-03-23T17:52:40.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.840+0000 7feabd600700 1 -- 172.21.15.86:0/47722065 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaac0054e0 con 0x7feac0106080 2024-03-23T17:52:40.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.840+0000 7feac7c00700 1 -- 172.21.15.86:0/47722065 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feac01a6680 con 0x7feac0106080 2024-03-23T17:52:40.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.843+0000 7feabd600700 1 -- 172.21.15.86:0/47722065 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7feaac00c950 con 0x7feac0106080 2024-03-23T17:52:40.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.844+0000 7feadb805b80 1 Processor -- start 2024-03-23T17:52:40.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.844+0000 7feadb805b80 1 -- start start 2024-03-23T17:52:40.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.844+0000 7feabd600700 1 --2- 172.21.15.86:0/47722065 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7feaa80a1310 0x7feaa80a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:40.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.845+0000 7feadb805b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5608fc2065d0 0x5608fc2069b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:40.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.845+0000 7feadb805b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608fc206ef0 0x5608fc2118b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:40.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.845+0000 7feabd600700 1 -- 172.21.15.86:0/47722065 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7feaac014070 con 0x7feac0106080 2024-03-23T17:52:40.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.845+0000 7feadb805b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5608fc211df0 0x5608fc2141e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:40.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.845+0000 7feadb805b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608fc15f990 con 0x5608fc206ef0 2024-03-23T17:52:40.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.845+0000 7feadb805b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608fc15d350 con 0x5608fc211df0 2024-03-23T17:52:40.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.845+0000 7feadb805b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608fc158c40 con 0x5608fc2065d0 2024-03-23T17:52:40.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.845+0000 7feac5c00700 1 --2- 172.21.15.86:0/47722065 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7feaa80a1310 0x7feaa80a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:40.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.845+0000 7feabfe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608fc206ef0 0x5608fc2118b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:40.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.845+0000 7feac5200700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5608fc2065d0 0x5608fc2069b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:40.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.845+0000 7feac5200700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5608fc2065d0 0x5608fc2069b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:52344/0 (socket says 172.21.15.86:52344) 2024-03-23T17:52:40.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.845+0000 7feac5200700 1 -- 172.21.15.86:0/217340133 learned_addr learned my addr 172.21.15.86:0/217340133 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:40.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.845+0000 7feac5c00700 1 --2- 172.21.15.86:0/217340133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5608fc211df0 0x5608fc2141e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:40.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.845+0000 7feac5200700 1 -- 172.21.15.86:0/217340133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5608fc211df0 msgr2=0x5608fc2141e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:40.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.845+0000 7feac5200700 1 --2- 172.21.15.86:0/217340133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5608fc211df0 0x5608fc2141e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:40.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.845+0000 7feac5200700 1 -- 172.21.15.86:0/217340133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608fc206ef0 msgr2=0x5608fc2118b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:40.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.845+0000 7feac5200700 1 --2- 172.21.15.86:0/217340133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608fc206ef0 0x5608fc2118b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:40.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.845+0000 7feac5200700 1 -- 172.21.15.86:0/217340133 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5608fbfda3b0 con 0x5608fc2065d0 2024-03-23T17:52:40.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.845+0000 7feabfe00700 1 --2- 172.21.15.86:0/217340133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608fc206ef0 0x5608fc2118b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:52:40.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.846+0000 7feac5c00700 1 --2- 172.21.15.86:0/47722065 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7feaa80a1310 0x7feaa80a37d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7feab800b780 tx=0x7feab80095c0 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:40.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.846+0000 7feac5200700 1 --2- 172.21.15.86:0/217340133 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5608fc2065d0 0x5608fc2069b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7feab4009d80 tx=0x7feab4011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85d172df5c55a644 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:40.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.846+0000 7feab2000700 1 -- 172.21.15.86:0/217340133 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab4004230 con 0x5608fc2065d0 2024-03-23T17:52:40.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.846+0000 7feab2000700 1 -- 172.21.15.86:0/217340133 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feab4009e10 con 0x5608fc2065d0 2024-03-23T17:52:40.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.846+0000 7feab2000700 1 -- 172.21.15.86:0/217340133 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab40197e0 con 0x5608fc2065d0 2024-03-23T17:52:40.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.847+0000 7feadb805b80 1 -- 172.21.15.86:0/217340133 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5608fc2065d0 msgr2=0x5608fc2069b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:40.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.847+0000 7feadb805b80 1 --2- 172.21.15.86:0/217340133 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5608fc2065d0 0x5608fc2069b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7feab4009d80 tx=0x7feab4011720 comp rx=0 tx=0).stop 2024-03-23T17:52:40.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.847+0000 7feadb805b80 1 -- 172.21.15.86:0/217340133 shutdown_connections 2024-03-23T17:52:40.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.847+0000 7feadb805b80 1 --2- 172.21.15.86:0/217340133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5608fc211df0 0x5608fc2141e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:40.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.847+0000 7feadb805b80 1 --2- 172.21.15.86:0/217340133 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5608fc2065d0 0x5608fc2069b0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:40.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.847+0000 7feadb805b80 1 --2- 172.21.15.86:0/217340133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608fc206ef0 0x5608fc2118b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:40.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.847+0000 7feadb805b80 1 -- 172.21.15.86:0/217340133 >> 172.21.15.86:0/217340133 conn(0x5608fc1666a0 msgr2=0x5608fc202a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:40.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.847+0000 7feadb805b80 1 -- 172.21.15.86:0/217340133 shutdown_connections 2024-03-23T17:52:40.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.847+0000 7feadb805b80 1 -- 172.21.15.86:0/217340133 wait complete. 2024-03-23T17:52:40.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.847+0000 7feadb805b80 1 -- start start 2024-03-23T17:52:40.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.848+0000 7feadb805b80 1 Processor -- start 2024-03-23T17:52:40.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.849+0000 7feadb805b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608fc2065d0 0x5608fc211710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:40.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.849+0000 7feadb805b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5608fc206ef0 0x5608fc20ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:40.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.849+0000 7feadb805b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5608fc211df0 0x5608fc20eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:40.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.849+0000 7feadb805b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608fc0f4d80 con 0x5608fc2065d0 2024-03-23T17:52:40.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.849+0000 7feadb805b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608fc0f4a20 con 0x5608fc206ef0 2024-03-23T17:52:40.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.849+0000 7feadb805b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608fbfdc6a0 con 0x5608fc211df0 2024-03-23T17:52:40.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.849+0000 7feadb805b80 10 client.0 _ll_register_callbacks cb 0x5608fc0fe470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:52:40.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.849+0000 7feac5200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608fc2065d0 0x5608fc211710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:40.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.849+0000 7feac5200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608fc2065d0 0x5608fc211710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34800/0 (socket says 172.21.15.86:34800) 2024-03-23T17:52:40.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.849+0000 7feac5200700 1 -- 172.21.15.86:0/3624403590 learned_addr learned my addr 172.21.15.86:0/3624403590 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:40.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.849+0000 7feabfe00700 1 --2- 172.21.15.86:0/3624403590 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5608fc206ef0 0x5608fc20ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:40.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.849+0000 7feac5c00700 1 --2- 172.21.15.86:0/3624403590 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5608fc211df0 0x5608fc20eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:40.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.849+0000 7feac5200700 1 -- 172.21.15.86:0/3624403590 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5608fc211df0 msgr2=0x5608fc20eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:40.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.849+0000 7feac5200700 1 --2- 172.21.15.86:0/3624403590 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5608fc211df0 0x5608fc20eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:40.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.849+0000 7feac5200700 1 -- 172.21.15.86:0/3624403590 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5608fc206ef0 msgr2=0x5608fc20ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:40.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.849+0000 7feac5200700 1 --2- 172.21.15.86:0/3624403590 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5608fc206ef0 0x5608fc20ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:40.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.849+0000 7feac5200700 1 -- 172.21.15.86:0/3624403590 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feac01a6680 con 0x5608fc2065d0 2024-03-23T17:52:40.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.849+0000 7feac5200700 1 --2- 172.21.15.86:0/3624403590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608fc2065d0 0x5608fc211710 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7feab4000c00 tx=0x7feab4004280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc129d9b640aef8e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:40.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.849+0000 7feaa6a00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-23T17:52:40.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.850+0000 7feaa6a00700 1 -- 172.21.15.86:0/3624403590 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab4005450 con 0x5608fc2065d0 2024-03-23T17:52:40.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.850+0000 7feaa6a00700 1 -- 172.21.15.86:0/3624403590 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7feab4005ce0 con 0x5608fc2065d0 2024-03-23T17:52:40.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.850+0000 7feadb805b80 1 -- 172.21.15.86:0/3624403590 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5608fc119380 con 0x5608fc2065d0 2024-03-23T17:52:40.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.850+0000 7feaa6a00700 1 -- 172.21.15.86:0/3624403590 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7feab4003cf0 con 0x5608fc2065d0 2024-03-23T17:52:40.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.850+0000 7feadb805b80 10 client.16050 fetch_fsmap learned FSMap version 33 2024-03-23T17:52:40.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.850+0000 7feadb805b80 1 -- 172.21.15.86:0/3624403590 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7feab40099e0 con 0x5608fc2065d0 2024-03-23T17:52:40.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.850+0000 7feaa6a00700 1 -- 172.21.15.86:0/3624403590 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5608fc119380 con 0x5608fc2065d0 2024-03-23T17:52:40.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.850+0000 7feaa6a00700 1 -- 172.21.15.86:0/3624403590 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 33) v1 ==== 7478+0+0 (secure 0 0 0) 0x7feab4021040 con 0x5608fc2065d0 2024-03-23T17:52:40.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.850+0000 7feadb805b80 10 client.16050 fetch_fsmap finished waiting for FSMap version 33 2024-03-23T17:52:40.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.850+0000 7feadb805b80 10 client.16050 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T17:52:40.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.850+0000 7feadb805b80 20 client.16050 populate_metadata read hostname 'smithi086' 2024-03-23T17:52:40.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.850+0000 7feadb805b80 1 --2- 172.21.15.86:0/3624403590 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5608fc221910 0x5608fc223d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:40.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.850+0000 7feadb805b80 4 client.16050 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:52:40.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.850+0000 7feadb805b80 1 -- 172.21.15.86:0/3624403590 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5608fc151700 con 0x5608fc221910 2024-03-23T17:52:40.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.851+0000 7feabfe00700 1 --2- 172.21.15.86:0/3624403590 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5608fc221910 0x5608fc223d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:40.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.851+0000 7feabfe00700 1 --2- 172.21.15.86:0/3624403590 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5608fc221910 0x5608fc223d30 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da0ccde29511db20 server_cookie=98bfb379c043b77e in_seq=0 out_seq=0 2024-03-23T17:52:40.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.851+0000 7feaa6a00700 10 client.16050 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:52:40.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.852+0000 7feaa6a00700 1 -- 172.21.15.86:0/3624403590 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5608fc151700 con 0x5608fc221910 2024-03-23T17:52:40.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.852+0000 7feaa6a00700 10 client.16050 handle_command_reply: tid=0 2024-03-23T17:52:40.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.852+0000 7feadb805b80 1 client.16050 shutdown 2024-03-23T17:52:40.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.852+0000 7feadb805b80 1 -- 172.21.15.86:0/3624403590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608fc2065d0 msgr2=0x5608fc211710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:40.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.852+0000 7feadb805b80 1 --2- 172.21.15.86:0/3624403590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608fc2065d0 0x5608fc211710 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7feab4000c00 tx=0x7feab4004280 comp rx=0 tx=0).stop 2024-03-23T17:52:40.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.852+0000 7feadb805b80 1 -- 172.21.15.86:0/3624403590 shutdown_connections 2024-03-23T17:52:40.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.852+0000 7feadb805b80 1 --2- 172.21.15.86:0/3624403590 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5608fc221910 0x5608fc223d30 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:40.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.852+0000 7feadb805b80 1 --2- 172.21.15.86:0/3624403590 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5608fc211df0 0x5608fc20eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:40.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.852+0000 7feadb805b80 1 --2- 172.21.15.86:0/3624403590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608fc2065d0 0x5608fc211710 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:40.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.852+0000 7feadb805b80 1 --2- 172.21.15.86:0/3624403590 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5608fc206ef0 0x5608fc20ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:40.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.852+0000 7feaa6a00700 0 client.16050 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:52:40.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.852+0000 7feadb805b80 1 -- 172.21.15.86:0/3624403590 >> 172.21.15.86:0/3624403590 conn(0x5608fc1666a0 msgr2=0x5608fc208690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:40.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.852+0000 7feadb805b80 1 -- 172.21.15.86:0/3624403590 shutdown_connections 2024-03-23T17:52:40.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.852+0000 7feadb805b80 1 -- 172.21.15.86:0/3624403590 wait complete. 2024-03-23T17:52:40.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.853+0000 7feadb805b80 20 client.16050 trim_cache size 0 max 16384 2024-03-23T17:52:40.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.869+0000 7feadb805b80 1 Processor -- start 2024-03-23T17:52:40.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.869+0000 7feadb805b80 1 -- start start 2024-03-23T17:52:40.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.870+0000 7feadb805b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5608fc2065d0 0x5608fc218aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:40.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.870+0000 7feadb805b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608fc206ef0 0x5608fc20d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:40.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.870+0000 7feadb805b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5608fc211df0 0x5608fc20d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:40.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.870+0000 7feadb805b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608fc20dde0 con 0x5608fc206ef0 2024-03-23T17:52:40.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.870+0000 7feadb805b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608fc20fd50 con 0x5608fc2065d0 2024-03-23T17:52:40.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.870+0000 7feadb805b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608fc228a10 con 0x5608fc211df0 2024-03-23T17:52:40.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.870+0000 7feabfe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608fc206ef0 0x5608fc20d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:40.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.870+0000 7feabfe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608fc206ef0 0x5608fc20d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34806/0 (socket says 172.21.15.86:34806) 2024-03-23T17:52:40.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.870+0000 7feabfe00700 1 -- 172.21.15.86:0/1674993186 learned_addr learned my addr 172.21.15.86:0/1674993186 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:40.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.870+0000 7feac5200700 1 --2- 172.21.15.86:0/1674993186 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5608fc2065d0 0x5608fc218aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:40.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.870+0000 7feac5c00700 1 --2- 172.21.15.86:0/1674993186 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5608fc211df0 0x5608fc20d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:40.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.870+0000 7feabfe00700 1 -- 172.21.15.86:0/1674993186 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5608fc211df0 msgr2=0x5608fc20d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:40.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.870+0000 7feabfe00700 1 --2- 172.21.15.86:0/1674993186 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5608fc211df0 0x5608fc20d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:40.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.870+0000 7feabfe00700 1 -- 172.21.15.86:0/1674993186 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5608fc2065d0 msgr2=0x5608fc218aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:40.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.870+0000 7feabfe00700 1 --2- 172.21.15.86:0/1674993186 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5608fc2065d0 0x5608fc218aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:40.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.870+0000 7feabfe00700 1 -- 172.21.15.86:0/1674993186 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5608fbf86b60 con 0x5608fc206ef0 2024-03-23T17:52:40.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.871+0000 7feabfe00700 1 --2- 172.21.15.86:0/1674993186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608fc206ef0 0x5608fc20d360 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7feaac099f10 tx=0x7feaac09c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=550db57519d2b0a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:40.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.871+0000 7feab2a00700 1 -- 172.21.15.86:0/1674993186 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaac09ddb0 con 0x5608fc206ef0 2024-03-23T17:52:40.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.871+0000 7feab2a00700 1 -- 172.21.15.86:0/1674993186 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feaac0a2400 con 0x5608fc206ef0 2024-03-23T17:52:40.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.871+0000 7feab2a00700 1 -- 172.21.15.86:0/1674993186 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaac096690 con 0x5608fc206ef0 2024-03-23T17:52:40.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.871+0000 7feadb805b80 1 -- 172.21.15.86:0/1674993186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608fc206ef0 msgr2=0x5608fc20d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:40.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.871+0000 7feadb805b80 1 --2- 172.21.15.86:0/1674993186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608fc206ef0 0x5608fc20d360 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7feaac099f10 tx=0x7feaac09c040 comp rx=0 tx=0).stop 2024-03-23T17:52:40.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.871+0000 7feadb805b80 1 -- 172.21.15.86:0/1674993186 shutdown_connections 2024-03-23T17:52:40.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.871+0000 7feadb805b80 1 --2- 172.21.15.86:0/1674993186 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5608fc211df0 0x5608fc20d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:40.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.871+0000 7feadb805b80 1 --2- 172.21.15.86:0/1674993186 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5608fc2065d0 0x5608fc218aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:40.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.871+0000 7feadb805b80 1 --2- 172.21.15.86:0/1674993186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608fc206ef0 0x5608fc20d360 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:40.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.871+0000 7feadb805b80 1 -- 172.21.15.86:0/1674993186 >> 172.21.15.86:0/1674993186 conn(0x5608fc1666a0 msgr2=0x5608fc1fbf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:40.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.871+0000 7feadb805b80 1 -- 172.21.15.86:0/1674993186 shutdown_connections 2024-03-23T17:52:40.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.871+0000 7feadb805b80 1 -- 172.21.15.86:0/1674993186 wait complete. 2024-03-23T17:52:40.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.872+0000 7feadb805b80 1 -- start start 2024-03-23T17:52:40.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.872+0000 7feadb805b80 1 Processor -- start 2024-03-23T17:52:40.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.873+0000 7feadb805b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608fc2065d0 0x5608fc217890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:40.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.873+0000 7feadb805b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5608fc206ef0 0x5608fc217dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:40.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.873+0000 7feadb805b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5608fc211df0 0x5608fc273400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:40.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.873+0000 7feadb805b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608fc273940 con 0x5608fc2065d0 2024-03-23T17:52:40.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.873+0000 7feadb805b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608fc273a80 con 0x5608fc211df0 2024-03-23T17:52:40.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.873+0000 7feadb805b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608fc273bf0 con 0x5608fc206ef0 2024-03-23T17:52:40.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.873+0000 7feadb805b80 10 client.0 _ll_register_callbacks cb 0x5608fc0fe470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:52:40.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.873+0000 7feac5200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608fc2065d0 0x5608fc217890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:40.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.873+0000 7feac5200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608fc2065d0 0x5608fc217890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34818/0 (socket says 172.21.15.86:34818) 2024-03-23T17:52:40.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.873+0000 7feac5200700 1 -- 172.21.15.86:0/1181555724 learned_addr learned my addr 172.21.15.86:0/1181555724 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:40.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.873+0000 7feabfe00700 1 --2- 172.21.15.86:0/1181555724 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5608fc206ef0 0x5608fc217dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:40.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.873+0000 7feac5c00700 1 --2- 172.21.15.86:0/1181555724 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5608fc211df0 0x5608fc273400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:40.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.873+0000 7feac5200700 1 -- 172.21.15.86:0/1181555724 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5608fc206ef0 msgr2=0x5608fc217dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:40.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.873+0000 7feac5200700 1 --2- 172.21.15.86:0/1181555724 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5608fc206ef0 0x5608fc217dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:40.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.873+0000 7feac5200700 1 -- 172.21.15.86:0/1181555724 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5608fc211df0 msgr2=0x5608fc273400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:40.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.873+0000 7feac5200700 1 --2- 172.21.15.86:0/1181555724 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5608fc211df0 0x5608fc273400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:40.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.873+0000 7feac5200700 1 -- 172.21.15.86:0/1181555724 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feac01a6120 con 0x5608fc2065d0 2024-03-23T17:52:40.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.873+0000 7feabfe00700 1 --2- 172.21.15.86:0/1181555724 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5608fc206ef0 0x5608fc217dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:52:40.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.873+0000 7feac5c00700 1 --2- 172.21.15.86:0/1181555724 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5608fc211df0 0x5608fc273400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:52:40.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.874+0000 7feac5200700 1 --2- 172.21.15.86:0/1181555724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608fc2065d0 0x5608fc217890 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7feab4023530 tx=0x7feab4005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3f99e6d823d3e13 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:40.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.874+0000 7feaa6a00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-23T17:52:40.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.874+0000 7feaa6a00700 1 -- 172.21.15.86:0/1181555724 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab40239e0 con 0x5608fc2065d0 2024-03-23T17:52:40.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.874+0000 7feaa6a00700 1 -- 172.21.15.86:0/1181555724 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7feab4000e40 con 0x5608fc2065d0 2024-03-23T17:52:40.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.874+0000 7feadb805b80 1 -- 172.21.15.86:0/1181555724 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feab4009e10 con 0x5608fc2065d0 2024-03-23T17:52:40.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.874+0000 7feaa6a00700 1 -- 172.21.15.86:0/1181555724 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7feab4023b40 con 0x5608fc2065d0 2024-03-23T17:52:40.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.874+0000 7feadb805b80 10 client.16056 fetch_fsmap learned FSMap version 33 2024-03-23T17:52:40.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.874+0000 7feadb805b80 1 -- 172.21.15.86:0/1181555724 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5608fc11eea0 con 0x5608fc2065d0 2024-03-23T17:52:40.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.874+0000 7feaa6a00700 1 -- 172.21.15.86:0/1181555724 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feab4009e10 con 0x5608fc2065d0 2024-03-23T17:52:40.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.875+0000 7feaa6a00700 1 -- 172.21.15.86:0/1181555724 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 33) v1 ==== 7478+0+0 (secure 0 0 0) 0x7feab4021070 con 0x5608fc2065d0 2024-03-23T17:52:40.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.875+0000 7feadb805b80 10 client.16056 fetch_fsmap finished waiting for FSMap version 33 2024-03-23T17:52:40.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.875+0000 7feadb805b80 10 client.16056 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T17:52:40.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.875+0000 7feadb805b80 20 client.16056 populate_metadata read hostname 'smithi086' 2024-03-23T17:52:40.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.875+0000 7feadb805b80 1 --2- 172.21.15.86:0/1181555724 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5608fc275110 0x5608fc277530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:40.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.875+0000 7feadb805b80 4 client.16056 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-23T17:52:40.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.875+0000 7feadb805b80 1 -- 172.21.15.86:0/1181555724 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5608fbfa7cf0 con 0x5608fc275110 2024-03-23T17:52:40.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.875+0000 7feabfe00700 1 --2- 172.21.15.86:0/1181555724 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5608fc275110 0x5608fc277530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:40.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.875+0000 7feaa6a00700 10 client.16056 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:52:40.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.875+0000 7feabfe00700 1 --2- 172.21.15.86:0/1181555724 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5608fc275110 0x5608fc277530 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6e01a676a4d50c8 server_cookie=d938ee50ec6bc5b8 in_seq=0 out_seq=0 2024-03-23T17:52:40.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.876+0000 7feaa6a00700 1 -- 172.21.15.86:0/1181555724 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5608fbfa7cf0 con 0x5608fc275110 2024-03-23T17:52:40.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.876+0000 7feaa6a00700 10 client.16056 handle_command_reply: tid=0 2024-03-23T17:52:40.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.876+0000 7feadb805b80 1 client.16056 shutdown 2024-03-23T17:52:40.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.876+0000 7feadb805b80 1 -- 172.21.15.86:0/1181555724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608fc2065d0 msgr2=0x5608fc217890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:40.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.876+0000 7feadb805b80 1 --2- 172.21.15.86:0/1181555724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608fc2065d0 0x5608fc217890 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7feab4023530 tx=0x7feab4005e50 comp rx=0 tx=0).stop 2024-03-23T17:52:40.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.876+0000 7feadb805b80 1 -- 172.21.15.86:0/1181555724 shutdown_connections 2024-03-23T17:52:40.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.876+0000 7feadb805b80 1 --2- 172.21.15.86:0/1181555724 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5608fc275110 0x5608fc277530 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:40.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.876+0000 7feadb805b80 1 --2- 172.21.15.86:0/1181555724 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5608fc211df0 0x5608fc273400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:40.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.876+0000 7feadb805b80 1 --2- 172.21.15.86:0/1181555724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608fc2065d0 0x5608fc217890 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:40.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.876+0000 7feadb805b80 1 --2- 172.21.15.86:0/1181555724 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5608fc206ef0 0x5608fc217dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:40.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.876+0000 7feaa6a00700 0 client.16056 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:52:40.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.876+0000 7feadb805b80 1 -- 172.21.15.86:0/1181555724 >> 172.21.15.86:0/1181555724 conn(0x5608fc1666a0 msgr2=0x5608fc209db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:40.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.876+0000 7feadb805b80 1 -- 172.21.15.86:0/1181555724 shutdown_connections 2024-03-23T17:52:40.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.876+0000 7feadb805b80 1 -- 172.21.15.86:0/1181555724 wait complete. 2024-03-23T17:52:40.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.876+0000 7feadb805b80 20 client.16056 trim_cache size 0 max 16384 2024-03-23T17:52:40.881 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-23T17:52:40.881 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-03-23T17:52:40.881 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-03-23T17:52:40.881 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-23T17:52:40.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.877+0000 7feac7c00700 1 -- 172.21.15.86:0/47722065 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7feaa80a1310 msgr2=0x7feaa80a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:40.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.877+0000 7feac7c00700 1 --2- 172.21.15.86:0/47722065 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7feaa80a1310 0x7feaa80a37d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7feab800b780 tx=0x7feab80095c0 comp rx=0 tx=0).stop 2024-03-23T17:52:40.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.877+0000 7feac7c00700 1 -- 172.21.15.86:0/47722065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feac0106080 msgr2=0x7feac00735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:40.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.877+0000 7feac7c00700 1 --2- 172.21.15.86:0/47722065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feac0106080 0x7feac00735f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7feaac00e970 tx=0x7feaac00ec80 comp rx=0 tx=0).stop 2024-03-23T17:52:40.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.877+0000 7feac7c00700 1 -- 172.21.15.86:0/47722065 shutdown_connections 2024-03-23T17:52:40.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.877+0000 7feac7c00700 1 --2- 172.21.15.86:0/47722065 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feac0073b30 0x7feac01a5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:40.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.877+0000 7feac7c00700 1 --2- 172.21.15.86:0/47722065 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7feaa80a1310 0x7feaa80a37d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:40.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.877+0000 7feac7c00700 1 --2- 172.21.15.86:0/47722065 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feac0100f60 0x7feac00730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:40.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.877+0000 7feac7c00700 1 --2- 172.21.15.86:0/47722065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feac0106080 0x7feac00735f0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:40.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.877+0000 7feac7c00700 1 -- 172.21.15.86:0/47722065 >> 172.21.15.86:0/47722065 conn(0x7feac00fc7c0 msgr2=0x7feac00fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:40.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.877+0000 7feac7c00700 1 -- 172.21.15.86:0/47722065 shutdown_connections 2024-03-23T17:52:40.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:40.877+0000 7feac7c00700 1 -- 172.21.15.86:0/47722065 wait complete. 2024-03-23T17:52:40.883 INFO:tasks.workunit.client.0.smithi086.stdout: 1 46870 4.15 MB/sec warmup 65 sec latency 3145.032 ms 2024-03-23T17:52:40.895 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-23T17:52:40.895 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T17:52:41.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.102+0000 7f99b6c00700 1 -- 172.21.15.86:0/475044346 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99b0073a80 msgr2=0x7f99b010cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:41.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.102+0000 7f99b6c00700 1 --2- 172.21.15.86:0/475044346 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99b0073a80 0x7f99b010cf70 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f999c009990 tx=0x7f999c009ca0 comp rx=0 tx=0).stop 2024-03-23T17:52:41.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.102+0000 7f99b6c00700 1 -- 172.21.15.86:0/475044346 shutdown_connections 2024-03-23T17:52:41.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.102+0000 7f99b6c00700 1 --2- 172.21.15.86:0/475044346 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f99b010d4b0 0x7f99b010f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:41.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.102+0000 7f99b6c00700 1 --2- 172.21.15.86:0/475044346 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99b0073160 0x7f99b0073540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:41.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.102+0000 7f99b6c00700 1 --2- 172.21.15.86:0/475044346 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99b0073a80 0x7f99b010cf70 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:41.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.103+0000 7f99b6c00700 1 -- 172.21.15.86:0/475044346 >> 172.21.15.86:0/475044346 conn(0x7f99b00fc720 msgr2=0x7f99b00feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:41.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.103+0000 7f99b6c00700 1 -- 172.21.15.86:0/475044346 shutdown_connections 2024-03-23T17:52:41.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.103+0000 7f99b6c00700 1 -- 172.21.15.86:0/475044346 wait complete. 2024-03-23T17:52:41.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.104+0000 7f99b6c00700 1 Processor -- start 2024-03-23T17:52:41.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.104+0000 7f99b6c00700 1 -- start start 2024-03-23T17:52:41.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.104+0000 7f99b6c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99b0073160 0x7f99b0199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:41.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.104+0000 7f99b6c00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f99b0073a80 0x7f99b0199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:41.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.104+0000 7f99b6c00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99b010d4b0 0x7f99b019dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:41.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.104+0000 7f99b6c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99b019e240 con 0x7f99b0073160 2024-03-23T17:52:41.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.104+0000 7f99b6c00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99b019e3b0 con 0x7f99b010d4b0 2024-03-23T17:52:41.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.104+0000 7f99b6c00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99b019e570 con 0x7f99b0073a80 2024-03-23T17:52:41.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.105+0000 7f99afe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99b0073160 0x7f99b0199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:41.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.105+0000 7f99b4c00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99b010d4b0 0x7f99b019dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:41.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.105+0000 7f99b4c00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99b010d4b0 0x7f99b019dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:45812/0 (socket says 172.21.15.86:45812) 2024-03-23T17:52:41.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.105+0000 7f99b4c00700 1 -- 172.21.15.86:0/435626673 learned_addr learned my addr 172.21.15.86:0/435626673 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:41.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.105+0000 7f99af400700 1 --2- 172.21.15.86:0/435626673 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f99b0073a80 0x7f99b0199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:41.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.105+0000 7f99afe00700 1 -- 172.21.15.86:0/435626673 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f99b0073a80 msgr2=0x7f99b0199860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:41.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.105+0000 7f99afe00700 1 --2- 172.21.15.86:0/435626673 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f99b0073a80 0x7f99b0199860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:41.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.105+0000 7f99afe00700 1 -- 172.21.15.86:0/435626673 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99b010d4b0 msgr2=0x7f99b019dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:41.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.105+0000 7f99afe00700 1 --2- 172.21.15.86:0/435626673 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99b010d4b0 0x7f99b019dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:41.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.105+0000 7f99afe00700 1 -- 172.21.15.86:0/435626673 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f999c0095f0 con 0x7f99b0073160 2024-03-23T17:52:41.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.105+0000 7f99b4c00700 1 --2- 172.21.15.86:0/435626673 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99b010d4b0 0x7f99b019dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:52:41.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.106+0000 7f99afe00700 1 --2- 172.21.15.86:0/435626673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99b0073160 0x7f99b0199320 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f99a000d750 tx=0x7f99a000da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:41.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.106+0000 7f99acc00700 1 -- 172.21.15.86:0/435626673 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a0004020 con 0x7f99b0073160 2024-03-23T17:52:41.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.106+0000 7f99acc00700 1 -- 172.21.15.86:0/435626673 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99a0004830 con 0x7f99b0073160 2024-03-23T17:52:41.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.106+0000 7f99b6c00700 1 -- 172.21.15.86:0/435626673 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99b019e850 con 0x7f99b0073160 2024-03-23T17:52:41.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.106+0000 7f99acc00700 1 -- 172.21.15.86:0/435626673 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a00054c0 con 0x7f99b0073160 2024-03-23T17:52:41.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.106+0000 7f99b6c00700 1 -- 172.21.15.86:0/435626673 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99b01a6090 con 0x7f99b0073160 2024-03-23T17:52:41.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.107+0000 7f99b6c00700 1 -- 172.21.15.86:0/435626673 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99b01086c0 con 0x7f99b0073160 2024-03-23T17:52:41.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.109+0000 7f99acc00700 1 -- 172.21.15.86:0/435626673 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f99a00049a0 con 0x7f99b0073160 2024-03-23T17:52:41.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.111+0000 7f99acc00700 1 --2- 172.21.15.86:0/435626673 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f99940a1290 0x7f99940a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:41.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.112+0000 7f99acc00700 1 -- 172.21.15.86:0/435626673 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f99a00cc5a0 con 0x7f99b0073160 2024-03-23T17:52:41.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.112+0000 7f99af400700 1 --2- 172.21.15.86:0/435626673 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f99940a1290 0x7f99940a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:41.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.112+0000 7f99af400700 1 --2- 172.21.15.86:0/435626673 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f99940a1290 0x7f99940a3750 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f99b019a870 tx=0x7f999c00b260 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:41.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.116+0000 7f99acc00700 1 -- 172.21.15.86:0/435626673 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f99a00959e0 con 0x7f99b0073160 2024-03-23T17:52:41.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.412+0000 7f99b6c00700 1 -- 172.21.15.86:0/435626673 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f99b0074810 con 0x7f99b0073160 2024-03-23T17:52:41.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.413+0000 7f99acc00700 1 -- 172.21.15.86:0/435626673 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+10923 (secure 0 0 0) 0x7f99a0099000 con 0x7f99b0073160 2024-03-23T17:52:41.413 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T17:52:41.413 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14445,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1433140219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1433140219},{"type":"v1","addr":"172.21.15.86:6835","nonce":1433140219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6839/1879457730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1879457730},{"type":"v1","addr":"172.21.15.86:6839","nonce":1879457730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-23T17:48:56.202323+0000","modified":"2024-03-23T17:52:17.823189+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24553,"mds_2":24518,"mds_3":24505,"mds_4":24500},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14514":{"gid":14514,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/3026943866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3026943866},{"type":"v1","addr":"172.21.15.86:6841","nonce":3026943866}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24464":{"gid":24464,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6833/1467091186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1467091186},{"type":"v1","addr":"172.21.15.92:6833","nonce":1467091186}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6833/3075554623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3075554623},{"type":"v1","addr":"172.21.15.152:6833","nonce":3075554623}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":55,"addr":"172.21.15.92:6835/908900027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":908900027},{"type":"v1","addr":"172.21.15.92:6835","nonce":908900027}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"d","rank":3,"incarnation":23,"state":"up:active","state_seq":57,"addr":"172.21.15.86:6837/2933881328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2933881328},{"type":"v1","addr":"172.21.15.86:6837","nonce":2933881328}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":54,"addr":"172.21.15.92:6837/3967837948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":3967837948},{"type":"v1","addr":"172.21.15.92:6837","nonce":3967837948}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6835/3137085021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3137085021},{"type":"v1","addr":"172.21.15.152:6835","nonce":3137085021}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24538":{"gid":24538,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6837/1151208965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":1151208965},{"type":"v1","addr":"172.21.15.152:6837","nonce":1151208965}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.152:6839/1100723992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":1100723992},{"type":"v1","addr":"172.21.15.152:6839","nonce":1100723992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6839/1394139887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1394139887},{"type":"v1","addr":"172.21.15.92:6839","nonce":1394139887}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-23T17:52:41.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.416+0000 7f99b6c00700 1 -- 172.21.15.86:0/435626673 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f99940a1290 msgr2=0x7f99940a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:41.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.416+0000 7f99b6c00700 1 --2- 172.21.15.86:0/435626673 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f99940a1290 0x7f99940a3750 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f99b019a870 tx=0x7f999c00b260 comp rx=0 tx=0).stop 2024-03-23T17:52:41.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.416+0000 7f99b6c00700 1 -- 172.21.15.86:0/435626673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99b0073160 msgr2=0x7f99b0199320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:41.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.416+0000 7f99b6c00700 1 --2- 172.21.15.86:0/435626673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99b0073160 0x7f99b0199320 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f99a000d750 tx=0x7f99a000da60 comp rx=0 tx=0).stop 2024-03-23T17:52:41.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.416+0000 7f99b6c00700 1 -- 172.21.15.86:0/435626673 shutdown_connections 2024-03-23T17:52:41.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.416+0000 7f99b6c00700 1 --2- 172.21.15.86:0/435626673 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99b010d4b0 0x7f99b019dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:41.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.416+0000 7f99b6c00700 1 --2- 172.21.15.86:0/435626673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99b0073160 0x7f99b0199320 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:41.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.416+0000 7f99b6c00700 1 --2- 172.21.15.86:0/435626673 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f99940a1290 0x7f99940a3750 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:41.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.416+0000 7f99b6c00700 1 --2- 172.21.15.86:0/435626673 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f99b0073a80 0x7f99b0199860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:41.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.416+0000 7f99b6c00700 1 -- 172.21.15.86:0/435626673 >> 172.21.15.86:0/435626673 conn(0x7f99b00fc720 msgr2=0x7f99b00fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:41.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.416+0000 7f99b6c00700 1 -- 172.21.15.86:0/435626673 shutdown_connections 2024-03-23T17:52:41.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.416+0000 7f99b6c00700 1 -- 172.21.15.86:0/435626673 wait complete. 2024-03-23T17:52:41.417 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 33 2024-03-23T17:52:41.437 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-23T17:52:41.437 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-23T17:52:41.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.658+0000 7f13c7800700 1 -- 172.21.15.86:0/609921234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13c0067e30 msgr2=0x7f13c00682b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:41.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.658+0000 7f13c7800700 1 --2- 172.21.15.86:0/609921234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13c0067e30 0x7f13c00682b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f13ac0099c0 tx=0x7f13ac009cd0 comp rx=0 tx=0).stop 2024-03-23T17:52:41.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.659+0000 7f13c7800700 1 -- 172.21.15.86:0/609921234 shutdown_connections 2024-03-23T17:52:41.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.659+0000 7f13c7800700 1 --2- 172.21.15.86:0/609921234 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f13c00687f0 0x7f13c010db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:41.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.659+0000 7f13c7800700 1 --2- 172.21.15.86:0/609921234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13c0105ce0 0x7f13c01060c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:41.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.659+0000 7f13c7800700 1 --2- 172.21.15.86:0/609921234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13c0067e30 0x7f13c00682b0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:41.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.659+0000 7f13c7800700 1 -- 172.21.15.86:0/609921234 >> 172.21.15.86:0/609921234 conn(0x7f13c0076d40 msgr2=0x7f13c0077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:41.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.659+0000 7f13c7800700 1 -- 172.21.15.86:0/609921234 shutdown_connections 2024-03-23T17:52:41.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.659+0000 7f13c7800700 1 -- 172.21.15.86:0/609921234 wait complete. 2024-03-23T17:52:41.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.660+0000 7f13c7800700 1 Processor -- start 2024-03-23T17:52:41.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.660+0000 7f13c7800700 1 -- start start 2024-03-23T17:52:41.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.661+0000 7f13c7800700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f13c0067e30 0x7f13c0199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:41.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.661+0000 7f13c7800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13c00687f0 0x7f13c0199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:41.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.661+0000 7f13c7800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13c0105ce0 0x7f13c019dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:41.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.661+0000 7f13c7800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13c019e360 con 0x7f13c0105ce0 2024-03-23T17:52:41.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.661+0000 7f13c7800700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13c019e4d0 con 0x7f13c00687f0 2024-03-23T17:52:41.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.661+0000 7f13c7800700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13c019e690 con 0x7f13c0067e30 2024-03-23T17:52:41.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.661+0000 7f13bfe00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13c00687f0 0x7f13c0199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:41.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.661+0000 7f13c4e00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f13c0067e30 0x7f13c0199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:41.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.661+0000 7f13bfe00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13c00687f0 0x7f13c0199980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:45840/0 (socket says 172.21.15.86:45840) 2024-03-23T17:52:41.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.661+0000 7f13bfe00700 1 -- 172.21.15.86:0/4191217518 learned_addr learned my addr 172.21.15.86:0/4191217518 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:41.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.661+0000 7f13c5800700 1 --2- 172.21.15.86:0/4191217518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13c0105ce0 0x7f13c019dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:41.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.661+0000 7f13bfe00700 1 -- 172.21.15.86:0/4191217518 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f13c0067e30 msgr2=0x7f13c0199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:41.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.661+0000 7f13bfe00700 1 --2- 172.21.15.86:0/4191217518 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f13c0067e30 0x7f13c0199440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:41.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.661+0000 7f13bfe00700 1 -- 172.21.15.86:0/4191217518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13c0105ce0 msgr2=0x7f13c019dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:41.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.661+0000 7f13bfe00700 1 --2- 172.21.15.86:0/4191217518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13c0105ce0 0x7f13c019dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:41.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.661+0000 7f13bfe00700 1 -- 172.21.15.86:0/4191217518 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13ac0095f0 con 0x7f13c00687f0 2024-03-23T17:52:41.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.661+0000 7f13c4e00700 1 --2- 172.21.15.86:0/4191217518 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f13c0067e30 0x7f13c0199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:52:41.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.662+0000 7f13bfe00700 1 --2- 172.21.15.86:0/4191217518 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13c00687f0 0x7f13c0199980 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f13ac00b580 tx=0x7f13ac00bfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:41.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.662+0000 7f13bd600700 1 -- 172.21.15.86:0/4191217518 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13ac01c070 con 0x7f13c00687f0 2024-03-23T17:52:41.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.662+0000 7f13bd600700 1 -- 172.21.15.86:0/4191217518 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f13ac00fdf0 con 0x7f13c00687f0 2024-03-23T17:52:41.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.662+0000 7f13c7800700 1 -- 172.21.15.86:0/4191217518 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13c019e910 con 0x7f13c00687f0 2024-03-23T17:52:41.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.662+0000 7f13bd600700 1 -- 172.21.15.86:0/4191217518 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13ac022c40 con 0x7f13c00687f0 2024-03-23T17:52:41.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.662+0000 7f13c7800700 1 -- 172.21.15.86:0/4191217518 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13c01a61b0 con 0x7f13c00687f0 2024-03-23T17:52:41.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.665+0000 7f13bd600700 1 -- 172.21.15.86:0/4191217518 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f13ac020070 con 0x7f13c00687f0 2024-03-23T17:52:41.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.668+0000 7f13bd600700 1 --2- 172.21.15.86:0/4191217518 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f13a80a1290 0x7f13a80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:41.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.668+0000 7f13db2edb80 1 Processor -- start 2024-03-23T17:52:41.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.668+0000 7f13bd600700 1 -- 172.21.15.86:0/4191217518 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f13ac0cda00 con 0x7f13c00687f0 2024-03-23T17:52:41.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.668+0000 7f13c4e00700 1 --2- 172.21.15.86:0/4191217518 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f13a80a1290 0x7f13a80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:41.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.668+0000 7f13db2edb80 1 -- start start 2024-03-23T17:52:41.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.668+0000 7f13db2edb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f3ee6ce5d0 0x55f3ee6ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:41.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.668+0000 7f13db2edb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3ee6ceef0 0x55f3ee6e20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:41.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.668+0000 7f13db2edb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3ee6e2600 0x55f3ee6e49f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:41.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.668+0000 7f13db2edb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3ee627990 con 0x55f3ee6e2600 2024-03-23T17:52:41.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.668+0000 7f13db2edb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3ee625350 con 0x55f3ee6ceef0 2024-03-23T17:52:41.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.668+0000 7f13db2edb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3ee620c40 con 0x55f3ee6ce5d0 2024-03-23T17:52:41.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.669+0000 7f13bfe00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3ee6ceef0 0x55f3ee6e20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:41.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.669+0000 7f13bfe00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3ee6ceef0 0x55f3ee6e20c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:45842/0 (socket says 172.21.15.86:45842) 2024-03-23T17:52:41.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.669+0000 7f13bfe00700 1 -- 172.21.15.86:0/731447541 learned_addr learned my addr 172.21.15.86:0/731447541 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:41.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.669+0000 7f13c4e00700 1 --2- 172.21.15.86:0/4191217518 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f13a80a1290 0x7f13a80a3750 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f13b4011490 tx=0x7f13b4009260 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:41.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.669+0000 7f13c4e00700 1 --2- 172.21.15.86:0/731447541 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f3ee6ce5d0 0x55f3ee6ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:41.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.669+0000 7f13bfe00700 1 -- 172.21.15.86:0/731447541 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f3ee6ce5d0 msgr2=0x55f3ee6ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:41.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.669+0000 7f13bfe00700 1 --2- 172.21.15.86:0/731447541 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f3ee6ce5d0 0x55f3ee6ce9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:41.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.669+0000 7f13bfe00700 1 -- 172.21.15.86:0/731447541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3ee6e2600 msgr2=0x55f3ee6e49f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-23T17:52:41.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.670+0000 7f13c5800700 1 --2- 172.21.15.86:0/731447541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3ee6e2600 0x55f3ee6e49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:41.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.670+0000 7f13bfe00700 1 --2- 172.21.15.86:0/731447541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3ee6e2600 0x55f3ee6e49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:41.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.670+0000 7f13bfe00700 1 -- 172.21.15.86:0/731447541 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f3ee4a23b0 con 0x55f3ee6ceef0 2024-03-23T17:52:41.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.670+0000 7f13c4e00700 1 --2- 172.21.15.86:0/731447541 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f3ee6ce5d0 0x55f3ee6ce9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:52:41.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.670+0000 7f13c5800700 1 --2- 172.21.15.86:0/731447541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3ee6e2600 0x55f3ee6e49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T17:52:41.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.670+0000 7f13bfe00700 1 --2- 172.21.15.86:0/731447541 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3ee6ceef0 0x55f3ee6e20c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f13ac097090 tx=0x7f13ac09a720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f280a5306dbb5c67 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:41.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.670+0000 7f13b2000700 1 -- 172.21.15.86:0/731447541 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13ac09e9c0 con 0x55f3ee6ceef0 2024-03-23T17:52:41.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.670+0000 7f13b2000700 1 -- 172.21.15.86:0/731447541 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f13ac0a2f80 con 0x55f3ee6ceef0 2024-03-23T17:52:41.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.670+0000 7f13b2000700 1 -- 172.21.15.86:0/731447541 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13ac09a2e0 con 0x55f3ee6ceef0 2024-03-23T17:52:41.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.671+0000 7f13db2edb80 1 -- 172.21.15.86:0/731447541 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3ee6ceef0 msgr2=0x55f3ee6e20c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:41.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.671+0000 7f13db2edb80 1 --2- 172.21.15.86:0/731447541 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3ee6ceef0 0x55f3ee6e20c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f13ac097090 tx=0x7f13ac09a720 comp rx=0 tx=0).stop 2024-03-23T17:52:41.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.671+0000 7f13db2edb80 1 -- 172.21.15.86:0/731447541 shutdown_connections 2024-03-23T17:52:41.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.671+0000 7f13db2edb80 1 --2- 172.21.15.86:0/731447541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3ee6e2600 0x55f3ee6e49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:41.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.671+0000 7f13db2edb80 1 --2- 172.21.15.86:0/731447541 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f3ee6ce5d0 0x55f3ee6ce9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:41.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.671+0000 7f13db2edb80 1 --2- 172.21.15.86:0/731447541 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3ee6ceef0 0x55f3ee6e20c0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:41.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.671+0000 7f13db2edb80 1 -- 172.21.15.86:0/731447541 >> 172.21.15.86:0/731447541 conn(0x55f3ee62e6a0 msgr2=0x55f3ee6caa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:41.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.671+0000 7f13db2edb80 1 -- 172.21.15.86:0/731447541 shutdown_connections 2024-03-23T17:52:41.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.671+0000 7f13db2edb80 1 -- 172.21.15.86:0/731447541 wait complete. 2024-03-23T17:52:41.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.672+0000 7f13db2edb80 1 -- start start 2024-03-23T17:52:41.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.672+0000 7f13db2edb80 1 Processor -- start 2024-03-23T17:52:41.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.672+0000 7f13db2edb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f3ee6ce5d0 0x55f3ee6e29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:41.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.672+0000 7f13db2edb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3ee6ceef0 0x55f3ee6e2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:41.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.673+0000 7f13db2edb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3ee6e2600 0x55f3ee6eda60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:41.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.673+0000 7f13db2edb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3ee5bcd80 con 0x55f3ee6ceef0 2024-03-23T17:52:41.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.673+0000 7f13db2edb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3ee5bca20 con 0x55f3ee6e2600 2024-03-23T17:52:41.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.673+0000 7f13db2edb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3ee4a46a0 con 0x55f3ee6ce5d0 2024-03-23T17:52:41.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.673+0000 7f13db2edb80 10 client.0 _ll_register_callbacks cb 0x55f3ee5c6470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:52:41.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.673+0000 7f13c4e00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f3ee6ce5d0 0x55f3ee6e29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:41.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.673+0000 7f13c4e00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f3ee6ce5d0 0x55f3ee6e29e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:52410/0 (socket says 172.21.15.86:52410) 2024-03-23T17:52:41.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.673+0000 7f13c4e00700 1 -- 172.21.15.86:0/1329837444 learned_addr learned my addr 172.21.15.86:0/1329837444 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:41.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.673+0000 7f13c5800700 1 --2- 172.21.15.86:0/1329837444 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3ee6e2600 0x55f3ee6eda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:41.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.673+0000 7f13bfe00700 1 --2- 172.21.15.86:0/1329837444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3ee6ceef0 0x55f3ee6e2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:41.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.673+0000 7f13c4e00700 1 -- 172.21.15.86:0/1329837444 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3ee6e2600 msgr2=0x55f3ee6eda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:41.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.673+0000 7f13c4e00700 1 --2- 172.21.15.86:0/1329837444 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3ee6e2600 0x55f3ee6eda60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:41.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.673+0000 7f13c4e00700 1 -- 172.21.15.86:0/1329837444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3ee6ceef0 msgr2=0x55f3ee6e2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:41.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.673+0000 7f13c4e00700 1 --2- 172.21.15.86:0/1329837444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3ee6ceef0 0x55f3ee6e2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:41.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.673+0000 7f13c4e00700 1 -- 172.21.15.86:0/1329837444 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13c01a61b0 con 0x55f3ee6ce5d0 2024-03-23T17:52:41.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.673+0000 7f13bfe00700 1 --2- 172.21.15.86:0/1329837444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3ee6ceef0 0x55f3ee6e2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:52:41.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.673+0000 7f13c5800700 1 --2- 172.21.15.86:0/1329837444 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3ee6e2600 0x55f3ee6eda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:52:41.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.674+0000 7f13c4e00700 1 --2- 172.21.15.86:0/1329837444 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f3ee6ce5d0 0x55f3ee6e29e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f13b4003960 tx=0x7f13b40047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb4f69ce481d151c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:41.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.674+0000 7f13a6a00700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-03-23T17:52:41.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.674+0000 7f13a6a00700 1 -- 172.21.15.86:0/1329837444 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b400ea30 con 0x55f3ee6ce5d0 2024-03-23T17:52:41.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.674+0000 7f13a6a00700 1 -- 172.21.15.86:0/1329837444 <== mon.1 v2:172.21.15.152:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f13b400eb90 con 0x55f3ee6ce5d0 2024-03-23T17:52:41.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.674+0000 7f13db2edb80 1 -- 172.21.15.86:0/1329837444 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f3ee5e1380 con 0x55f3ee6ce5d0 2024-03-23T17:52:41.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.674+0000 7f13a6a00700 1 -- 172.21.15.86:0/1329837444 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13b400ed50 con 0x55f3ee6ce5d0 2024-03-23T17:52:41.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.674+0000 7f13db2edb80 10 client.26119 fetch_fsmap learned FSMap version 33 2024-03-23T17:52:41.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.674+0000 7f13db2edb80 1 -- 172.21.15.86:0/1329837444 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55f3ee44eb60 con 0x55f3ee6ce5d0 2024-03-23T17:52:41.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.675+0000 7f13a6a00700 1 -- 172.21.15.86:0/1329837444 <== mon.1 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f3ee5e1380 con 0x55f3ee6ce5d0 2024-03-23T17:52:41.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.675+0000 7f13a6a00700 1 -- 172.21.15.86:0/1329837444 <== mon.1 v2:172.21.15.152:3300/0 5 ==== fsmap(e 33) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f13b4028490 con 0x55f3ee6ce5d0 2024-03-23T17:52:41.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.675+0000 7f13db2edb80 10 client.26119 fetch_fsmap finished waiting for FSMap version 33 2024-03-23T17:52:41.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.675+0000 7f13db2edb80 10 client.26119 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-03-23T17:52:41.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.675+0000 7f13db2edb80 20 client.26119 populate_metadata read hostname 'smithi086' 2024-03-23T17:52:41.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.675+0000 7f13db2edb80 1 --2- 172.21.15.86:0/1329837444 >> [v2:172.21.15.92:6834/908900027,v1:172.21.15.92:6835/908900027] conn(0x55f3ee6f1af0 0x55f3ee6f3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:41.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.675+0000 7f13db2edb80 4 client.26119 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:52:41.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.675+0000 7f13db2edb80 1 -- 172.21.15.86:0/1329837444 --> [v2:172.21.15.92:6834/908900027,v1:172.21.15.92:6835/908900027] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f3ee619700 con 0x55f3ee6f1af0 2024-03-23T17:52:41.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.676+0000 7f13bfe00700 1 --2- 172.21.15.86:0/1329837444 >> [v2:172.21.15.92:6834/908900027,v1:172.21.15.92:6835/908900027] conn(0x55f3ee6f1af0 0x55f3ee6f3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:41.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.676+0000 7f13bfe00700 1 --2- 172.21.15.86:0/1329837444 >> [v2:172.21.15.92:6834/908900027,v1:172.21.15.92:6835/908900027] conn(0x55f3ee6f1af0 0x55f3ee6f3ee0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ee2ca135af6b50f server_cookie=ec4dcd961eb4f3e2 in_seq=0 out_seq=0 2024-03-23T17:52:41.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.676+0000 7f13a6a00700 10 client.26119 ms_handle_connect on v2:172.21.15.92:6834/908900027 2024-03-23T17:52:41.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.677+0000 7f13a6a00700 1 -- 172.21.15.86:0/1329837444 <== mds.4 v2:172.21.15.92:6834/908900027 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55f3ee619700 con 0x55f3ee6f1af0 2024-03-23T17:52:41.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.677+0000 7f13a6a00700 10 client.26119 handle_command_reply: tid=0 2024-03-23T17:52:41.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.677+0000 7f13db2edb80 1 client.26119 shutdown 2024-03-23T17:52:41.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.677+0000 7f13db2edb80 1 -- 172.21.15.86:0/1329837444 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f3ee6ce5d0 msgr2=0x55f3ee6e29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:41.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.677+0000 7f13db2edb80 1 --2- 172.21.15.86:0/1329837444 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f3ee6ce5d0 0x55f3ee6e29e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f13b4003960 tx=0x7f13b40047c0 comp rx=0 tx=0).stop 2024-03-23T17:52:41.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.677+0000 7f13db2edb80 1 -- 172.21.15.86:0/1329837444 shutdown_connections 2024-03-23T17:52:41.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.677+0000 7f13db2edb80 1 --2- 172.21.15.86:0/1329837444 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3ee6e2600 0x55f3ee6eda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:41.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.677+0000 7f13db2edb80 1 --2- 172.21.15.86:0/1329837444 >> [v2:172.21.15.92:6834/908900027,v1:172.21.15.92:6835/908900027] conn(0x55f3ee6f1af0 0x55f3ee6f3ee0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:41.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.677+0000 7f13db2edb80 1 --2- 172.21.15.86:0/1329837444 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f3ee6ce5d0 0x55f3ee6e29e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:41.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.677+0000 7f13db2edb80 1 --2- 172.21.15.86:0/1329837444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3ee6ceef0 0x55f3ee6e2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:41.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.677+0000 7f13a6a00700 0 client.26119 ms_handle_reset on v2:172.21.15.92:6834/908900027 2024-03-23T17:52:41.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.677+0000 7f13db2edb80 1 -- 172.21.15.86:0/1329837444 >> 172.21.15.86:0/1329837444 conn(0x55f3ee62e6a0 msgr2=0x55f3ee6d0530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:41.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.677+0000 7f13db2edb80 1 -- 172.21.15.86:0/1329837444 shutdown_connections 2024-03-23T17:52:41.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.677+0000 7f13db2edb80 1 -- 172.21.15.86:0/1329837444 wait complete. 2024-03-23T17:52:41.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.677+0000 7f13db2edb80 20 client.26119 trim_cache size 0 max 16384 2024-03-23T17:52:41.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.693+0000 7f13db2edb80 1 Processor -- start 2024-03-23T17:52:41.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.693+0000 7f13db2edb80 1 -- start start 2024-03-23T17:52:41.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.693+0000 7f13db2edb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3ee6ce5d0 0x55f3ee6e7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:41.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.693+0000 7f13db2edb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3ee6ceef0 0x55f3ee6edf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:41.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.693+0000 7f13db2edb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f3ee6e2600 0x55f3ee6e1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:41.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.693+0000 7f13db2edb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3ee6c6480 con 0x55f3ee6ce5d0 2024-03-23T17:52:41.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.693+0000 7f13db2edb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3ee6cb660 con 0x55f3ee6ceef0 2024-03-23T17:52:41.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.693+0000 7f13db2edb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3ee6e3610 con 0x55f3ee6e2600 2024-03-23T17:52:41.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.693+0000 7f13bfe00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3ee6ceef0 0x55f3ee6edf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:41.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.693+0000 7f13bfe00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3ee6ceef0 0x55f3ee6edf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:45870/0 (socket says 172.21.15.86:45870) 2024-03-23T17:52:41.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.693+0000 7f13bfe00700 1 -- 172.21.15.86:0/1636736759 learned_addr learned my addr 172.21.15.86:0/1636736759 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:41.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.693+0000 7f13c4e00700 1 --2- 172.21.15.86:0/1636736759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3ee6ce5d0 0x55f3ee6e7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:41.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.693+0000 7f13c5800700 1 --2- 172.21.15.86:0/1636736759 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f3ee6e2600 0x55f3ee6e1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:41.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.694+0000 7f13bfe00700 1 -- 172.21.15.86:0/1636736759 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f3ee6e2600 msgr2=0x55f3ee6e1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:41.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.694+0000 7f13bfe00700 1 --2- 172.21.15.86:0/1636736759 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f3ee6e2600 0x55f3ee6e1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:41.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.694+0000 7f13bfe00700 1 -- 172.21.15.86:0/1636736759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3ee6ce5d0 msgr2=0x55f3ee6e7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:41.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.694+0000 7f13bfe00700 1 --2- 172.21.15.86:0/1636736759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3ee6ce5d0 0x55f3ee6e7920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:41.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.694+0000 7f13bfe00700 1 -- 172.21.15.86:0/1636736759 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f3ee5e6ea0 con 0x55f3ee6ceef0 2024-03-23T17:52:41.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.694+0000 7f13c4e00700 1 --2- 172.21.15.86:0/1636736759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3ee6ce5d0 0x55f3ee6e7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:52:41.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.694+0000 7f13bfe00700 1 --2- 172.21.15.86:0/1636736759 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3ee6ceef0 0x55f3ee6edf00 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f13ac09ad50 tx=0x7f13ac009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41e368426a19408 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:41.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.694+0000 7f13b2a00700 1 -- 172.21.15.86:0/1636736759 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13ac09e9c0 con 0x55f3ee6ceef0 2024-03-23T17:52:41.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.694+0000 7f13b2a00700 1 -- 172.21.15.86:0/1636736759 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f13ac022420 con 0x55f3ee6ceef0 2024-03-23T17:52:41.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.694+0000 7f13b2a00700 1 -- 172.21.15.86:0/1636736759 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13ac09adf0 con 0x55f3ee6ceef0 2024-03-23T17:52:41.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.695+0000 7f13db2edb80 1 -- 172.21.15.86:0/1636736759 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3ee6ceef0 msgr2=0x55f3ee6edf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:41.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.695+0000 7f13db2edb80 1 --2- 172.21.15.86:0/1636736759 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3ee6ceef0 0x55f3ee6edf00 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f13ac09ad50 tx=0x7f13ac009310 comp rx=0 tx=0).stop 2024-03-23T17:52:41.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.695+0000 7f13db2edb80 1 -- 172.21.15.86:0/1636736759 shutdown_connections 2024-03-23T17:52:41.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.695+0000 7f13db2edb80 1 --2- 172.21.15.86:0/1636736759 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f3ee6e2600 0x55f3ee6e1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:41.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.695+0000 7f13db2edb80 1 --2- 172.21.15.86:0/1636736759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3ee6ce5d0 0x55f3ee6e7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:41.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.695+0000 7f13db2edb80 1 --2- 172.21.15.86:0/1636736759 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3ee6ceef0 0x55f3ee6edf00 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:41.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.695+0000 7f13db2edb80 1 -- 172.21.15.86:0/1636736759 >> 172.21.15.86:0/1636736759 conn(0x55f3ee62e6a0 msgr2=0x55f3ee6dc070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:41.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.695+0000 7f13db2edb80 1 -- 172.21.15.86:0/1636736759 shutdown_connections 2024-03-23T17:52:41.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.695+0000 7f13db2edb80 1 -- 172.21.15.86:0/1636736759 wait complete. 2024-03-23T17:52:41.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.695+0000 7f13db2edb80 1 -- start start 2024-03-23T17:52:41.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.696+0000 7f13db2edb80 1 Processor -- start 2024-03-23T17:52:41.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.696+0000 7f13db2edb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3ee6ce5d0 0x55f3ee737530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:41.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.696+0000 7f13db2edb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f3ee6ceef0 0x55f3ee737a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:41.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.696+0000 7f13db2edb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3ee6e2600 0x55f3ee737ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:41.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.696+0000 7f13db2edb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3ee742780 con 0x55f3ee6e2600 2024-03-23T17:52:41.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.696+0000 7f13db2edb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3ee7428f0 con 0x55f3ee6ce5d0 2024-03-23T17:52:41.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.696+0000 7f13db2edb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3ee742ab0 con 0x55f3ee6ceef0 2024-03-23T17:52:41.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.696+0000 7f13db2edb80 10 client.0 _ll_register_callbacks cb 0x55f3ee5c6470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:52:41.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.697+0000 7f13c4e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3ee6ce5d0 0x55f3ee737530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:41.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.697+0000 7f13c4e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3ee6ce5d0 0x55f3ee737530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:45872/0 (socket says 172.21.15.86:45872) 2024-03-23T17:52:41.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.697+0000 7f13c4e00700 1 -- 172.21.15.86:0/1334529676 learned_addr learned my addr 172.21.15.86:0/1334529676 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:41.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.697+0000 7f13c5800700 1 --2- 172.21.15.86:0/1334529676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3ee6e2600 0x55f3ee737ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:41.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.697+0000 7f13bfe00700 1 --2- 172.21.15.86:0/1334529676 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f3ee6ceef0 0x55f3ee737a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:41.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.697+0000 7f13c4e00700 1 -- 172.21.15.86:0/1334529676 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f3ee6ceef0 msgr2=0x55f3ee737a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:41.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.697+0000 7f13c4e00700 1 --2- 172.21.15.86:0/1334529676 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f3ee6ceef0 0x55f3ee737a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:41.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.697+0000 7f13c4e00700 1 -- 172.21.15.86:0/1334529676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3ee6e2600 msgr2=0x55f3ee737ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:41.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.697+0000 7f13c4e00700 1 --2- 172.21.15.86:0/1334529676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3ee6e2600 0x55f3ee737ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:41.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.697+0000 7f13c4e00700 1 -- 172.21.15.86:0/1334529676 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f3ee4a23b0 con 0x55f3ee6ce5d0 2024-03-23T17:52:41.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.697+0000 7f13c5800700 1 --2- 172.21.15.86:0/1334529676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3ee6e2600 0x55f3ee737ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:52:41.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.697+0000 7f13bfe00700 1 --2- 172.21.15.86:0/1334529676 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f3ee6ceef0 0x55f3ee737a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:52:41.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.697+0000 7f13c4e00700 1 --2- 172.21.15.86:0/1334529676 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3ee6ce5d0 0x55f3ee737530 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f13b4004150 tx=0x7f13b4028a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0c27f1e1f48e1c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:41.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.697+0000 7f13a6a00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T17:52:41.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.698+0000 7f13a6a00700 1 -- 172.21.15.86:0/1334529676 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b400ec40 con 0x55f3ee6ce5d0 2024-03-23T17:52:41.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.698+0000 7f13a6a00700 1 -- 172.21.15.86:0/1334529676 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f13b400eda0 con 0x55f3ee6ce5d0 2024-03-23T17:52:41.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.698+0000 7f13db2edb80 1 -- 172.21.15.86:0/1334529676 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f13ac0a2f80 con 0x55f3ee6ce5d0 2024-03-23T17:52:41.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.698+0000 7f13a6a00700 1 -- 172.21.15.86:0/1334529676 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13b4032480 con 0x55f3ee6ce5d0 2024-03-23T17:52:41.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.698+0000 7f13db2edb80 10 client.26135 fetch_fsmap learned FSMap version 33 2024-03-23T17:52:41.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.698+0000 7f13db2edb80 1 -- 172.21.15.86:0/1334529676 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55f3ee4a6cd0 con 0x55f3ee6ce5d0 2024-03-23T17:52:41.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.698+0000 7f13a6a00700 1 -- 172.21.15.86:0/1334529676 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f13ac0a2f80 con 0x55f3ee6ce5d0 2024-03-23T17:52:41.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.698+0000 7f13a6a00700 1 -- 172.21.15.86:0/1334529676 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 33) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f13b401f5a0 con 0x55f3ee6ce5d0 2024-03-23T17:52:41.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.698+0000 7f13db2edb80 10 client.26135 fetch_fsmap finished waiting for FSMap version 33 2024-03-23T17:52:41.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.698+0000 7f13db2edb80 10 client.26135 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-03-23T17:52:41.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.698+0000 7f13db2edb80 20 client.26135 populate_metadata read hostname 'smithi086' 2024-03-23T17:52:41.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.698+0000 7f13db2edb80 1 --2- 172.21.15.86:0/1334529676 >> [v2:172.21.15.92:6834/908900027,v1:172.21.15.92:6835/908900027] conn(0x55f3ee744270 0x55f3ee746690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:41.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.698+0000 7f13db2edb80 4 client.26135 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-23T17:52:41.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.698+0000 7f13db2edb80 1 -- 172.21.15.86:0/1334529676 --> [v2:172.21.15.92:6834/908900027,v1:172.21.15.92:6835/908900027] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f3ee46fcf0 con 0x55f3ee744270 2024-03-23T17:52:41.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.699+0000 7f13bfe00700 1 --2- 172.21.15.86:0/1334529676 >> [v2:172.21.15.92:6834/908900027,v1:172.21.15.92:6835/908900027] conn(0x55f3ee744270 0x55f3ee746690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:41.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.699+0000 7f13a6a00700 10 client.26135 ms_handle_connect on v2:172.21.15.92:6834/908900027 2024-03-23T17:52:41.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.699+0000 7f13bfe00700 1 --2- 172.21.15.86:0/1334529676 >> [v2:172.21.15.92:6834/908900027,v1:172.21.15.92:6835/908900027] conn(0x55f3ee744270 0x55f3ee746690 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=76114df1fbe51f4e server_cookie=50cdb3581e3d38eb in_seq=0 out_seq=0 2024-03-23T17:52:41.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.699+0000 7f13a6a00700 1 -- 172.21.15.86:0/1334529676 <== mds.4 v2:172.21.15.92:6834/908900027 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+1363 (crc 0 0 0) 0x55f3ee46fcf0 con 0x55f3ee744270 2024-03-23T17:52:41.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.699+0000 7f13a6a00700 10 client.26135 handle_command_reply: tid=0 2024-03-23T17:52:41.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.699+0000 7f13db2edb80 1 client.26135 shutdown 2024-03-23T17:52:41.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.700+0000 7f13db2edb80 1 -- 172.21.15.86:0/1334529676 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3ee6ce5d0 msgr2=0x55f3ee737530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:41.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.700+0000 7f13db2edb80 1 --2- 172.21.15.86:0/1334529676 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3ee6ce5d0 0x55f3ee737530 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f13b4004150 tx=0x7f13b4028a00 comp rx=0 tx=0).stop 2024-03-23T17:52:41.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.700+0000 7f13db2edb80 1 -- 172.21.15.86:0/1334529676 shutdown_connections 2024-03-23T17:52:41.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.700+0000 7f13db2edb80 1 --2- 172.21.15.86:0/1334529676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3ee6e2600 0x55f3ee737ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:41.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.700+0000 7f13db2edb80 1 --2- 172.21.15.86:0/1334529676 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3ee6ce5d0 0x55f3ee737530 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:41.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.700+0000 7f13db2edb80 1 --2- 172.21.15.86:0/1334529676 >> [v2:172.21.15.92:6834/908900027,v1:172.21.15.92:6835/908900027] conn(0x55f3ee744270 0x55f3ee746690 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:41.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.700+0000 7f13db2edb80 1 --2- 172.21.15.86:0/1334529676 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f3ee6ceef0 0x55f3ee737a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:41.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.700+0000 7f13a6a00700 0 client.26135 ms_handle_reset on v2:172.21.15.92:6834/908900027 2024-03-23T17:52:41.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.700+0000 7f13db2edb80 1 -- 172.21.15.86:0/1334529676 >> 172.21.15.86:0/1334529676 conn(0x55f3ee62e6a0 msgr2=0x55f3ee6e1120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:41.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.700+0000 7f13db2edb80 1 -- 172.21.15.86:0/1334529676 shutdown_connections 2024-03-23T17:52:41.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.700+0000 7f13db2edb80 1 -- 172.21.15.86:0/1334529676 wait complete. 2024-03-23T17:52:41.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.700+0000 7f13db2edb80 20 client.26135 trim_cache size 0 max 16384 2024-03-23T17:52:41.706 INFO:teuthology.orchestra.run.smithi086.stdout:[ 2024-03-23T17:52:41.706 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T17:52:41.706 INFO:teuthology.orchestra.run.smithi086.stdout: "damage_type": "backtrace", 2024-03-23T17:52:41.706 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 342231839, 2024-03-23T17:52:41.706 INFO:teuthology.orchestra.run.smithi086.stdout: "ino": 1099511629297, 2024-03-23T17:52:41.706 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "~mds4/stray1/100000005f1" 2024-03-23T17:52:41.706 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T17:52:41.706 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T17:52:41.706 INFO:teuthology.orchestra.run.smithi086.stdout: "damage_type": "backtrace", 2024-03-23T17:52:41.706 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 766548073, 2024-03-23T17:52:41.706 INFO:teuthology.orchestra.run.smithi086.stdout: "ino": 1099511628351, 2024-03-23T17:52:41.707 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/ACCESS/FASTENER.MDB" 2024-03-23T17:52:41.707 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T17:52:41.707 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T17:52:41.707 INFO:teuthology.orchestra.run.smithi086.stdout: "damage_type": "backtrace", 2024-03-23T17:52:41.707 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 1126266749, 2024-03-23T17:52:41.707 INFO:teuthology.orchestra.run.smithi086.stdout: "ino": 1099511628354, 2024-03-23T17:52:41.707 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/SEED/MEDIUM.FIL" 2024-03-23T17:52:41.707 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T17:52:41.707 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T17:52:41.707 INFO:teuthology.orchestra.run.smithi086.stdout: "damage_type": "backtrace", 2024-03-23T17:52:41.707 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 2478609256, 2024-03-23T17:52:41.707 INFO:teuthology.orchestra.run.smithi086.stdout: "ino": 1099511628355, 2024-03-23T17:52:41.707 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/SEED/LARGE.FIL" 2024-03-23T17:52:41.707 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T17:52:41.707 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T17:52:41.708 INFO:teuthology.orchestra.run.smithi086.stdout: "damage_type": "backtrace", 2024-03-23T17:52:41.708 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 2718211712, 2024-03-23T17:52:41.708 INFO:teuthology.orchestra.run.smithi086.stdout: "ino": 1099511629299, 2024-03-23T17:52:41.708 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "~mds4/stray1/100000005f3" 2024-03-23T17:52:41.708 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T17:52:41.708 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T17:52:41.708 INFO:teuthology.orchestra.run.smithi086.stdout: "damage_type": "backtrace", 2024-03-23T17:52:41.708 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 3153567874, 2024-03-23T17:52:41.708 INFO:teuthology.orchestra.run.smithi086.stdout: "ino": 1099511629295, 2024-03-23T17:52:41.708 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "~mds4/stray1/100000005ef" 2024-03-23T17:52:41.708 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T17:52:41.708 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T17:52:41.708 INFO:teuthology.orchestra.run.smithi086.stdout: "damage_type": "backtrace", 2024-03-23T17:52:41.708 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 3212837527, 2024-03-23T17:52:41.708 INFO:teuthology.orchestra.run.smithi086.stdout: "ino": 1099511628353, 2024-03-23T17:52:41.709 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/SEED/SMALL.FIL" 2024-03-23T17:52:41.709 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T17:52:41.709 INFO:teuthology.orchestra.run.smithi086.stdout:] 2024-03-23T17:52:41.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.701+0000 7f13c7800700 1 -- 172.21.15.86:0/4191217518 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f13a80a1290 msgr2=0x7f13a80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:41.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.701+0000 7f13c7800700 1 --2- 172.21.15.86:0/4191217518 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f13a80a1290 0x7f13a80a3750 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f13b4011490 tx=0x7f13b4009260 comp rx=0 tx=0).stop 2024-03-23T17:52:41.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.701+0000 7f13c7800700 1 -- 172.21.15.86:0/4191217518 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13c00687f0 msgr2=0x7f13c0199980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:41.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.701+0000 7f13c7800700 1 --2- 172.21.15.86:0/4191217518 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13c00687f0 0x7f13c0199980 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f13ac00b580 tx=0x7f13ac00bfd0 comp rx=0 tx=0).stop 2024-03-23T17:52:41.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.701+0000 7f13c7800700 1 -- 172.21.15.86:0/4191217518 shutdown_connections 2024-03-23T17:52:41.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.701+0000 7f13c7800700 1 --2- 172.21.15.86:0/4191217518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13c0105ce0 0x7f13c019dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:41.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.701+0000 7f13c7800700 1 --2- 172.21.15.86:0/4191217518 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f13a80a1290 0x7f13a80a3750 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:41.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.701+0000 7f13c7800700 1 --2- 172.21.15.86:0/4191217518 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f13c0067e30 0x7f13c0199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:41.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.701+0000 7f13c7800700 1 --2- 172.21.15.86:0/4191217518 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13c00687f0 0x7f13c0199980 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:41.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.701+0000 7f13c7800700 1 -- 172.21.15.86:0/4191217518 >> 172.21.15.86:0/4191217518 conn(0x7f13c0076d40 msgr2=0x7f13c00fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:41.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.701+0000 7f13c7800700 1 -- 172.21.15.86:0/4191217518 shutdown_connections 2024-03-23T17:52:41.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.701+0000 7f13c7800700 1 -- 172.21.15.86:0/4191217518 wait complete. 2024-03-23T17:52:41.720 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-23T17:52:41.721 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-23T17:52:41.883 INFO:tasks.workunit.client.0.smithi086.stdout: 1 49150 4.27 MB/sec warmup 66 sec latency 3163.586 ms 2024-03-23T17:52:41.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.928+0000 7f3a81a00700 1 -- 172.21.15.86:0/2654989573 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a7c073aa0 msgr2=0x7f3a7c10d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:41.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.928+0000 7f3a81a00700 1 --2- 172.21.15.86:0/2654989573 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a7c073aa0 0x7f3a7c10d040 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3a68009990 tx=0x7f3a68009ca0 comp rx=0 tx=0).stop 2024-03-23T17:52:41.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.928+0000 7f3a81a00700 1 -- 172.21.15.86:0/2654989573 shutdown_connections 2024-03-23T17:52:41.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.928+0000 7f3a81a00700 1 --2- 172.21.15.86:0/2654989573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a7c10d580 0x7f3a7c10f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:41.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.928+0000 7f3a81a00700 1 --2- 172.21.15.86:0/2654989573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a7c073180 0x7f3a7c073560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:41.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.928+0000 7f3a81a00700 1 --2- 172.21.15.86:0/2654989573 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a7c073aa0 0x7f3a7c10d040 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:41.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.928+0000 7f3a81a00700 1 -- 172.21.15.86:0/2654989573 >> 172.21.15.86:0/2654989573 conn(0x7f3a7c0fc7f0 msgr2=0x7f3a7c0fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:41.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.929+0000 7f3a81a00700 1 -- 172.21.15.86:0/2654989573 shutdown_connections 2024-03-23T17:52:41.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.929+0000 7f3a81a00700 1 -- 172.21.15.86:0/2654989573 wait complete. 2024-03-23T17:52:41.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.930+0000 7f3a81a00700 1 Processor -- start 2024-03-23T17:52:41.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.930+0000 7f3a81a00700 1 -- start start 2024-03-23T17:52:41.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.930+0000 7f3a81a00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a7c073180 0x7f3a7c199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:41.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.930+0000 7f3a81a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a7c073aa0 0x7f3a7c1999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:41.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.930+0000 7f3a81a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a7c10d580 0x7f3a7c19dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:41.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.930+0000 7f3a81a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7c19e390 con 0x7f3a7c073aa0 2024-03-23T17:52:41.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.930+0000 7f3a81a00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7c19e500 con 0x7f3a7c10d580 2024-03-23T17:52:41.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.930+0000 7f3a81a00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7c19e6c0 con 0x7f3a7c073180 2024-03-23T17:52:41.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.930+0000 7f3a7b400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a7c10d580 0x7f3a7c19dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:41.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.930+0000 7f3a7aa00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a7c073180 0x7f3a7c199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:41.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.930+0000 7f3a7a000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a7c073aa0 0x7f3a7c1999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:41.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.930+0000 7f3a7b400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a7c10d580 0x7f3a7c19dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:45888/0 (socket says 172.21.15.86:45888) 2024-03-23T17:52:41.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.930+0000 7f3a7b400700 1 -- 172.21.15.86:0/3048267459 learned_addr learned my addr 172.21.15.86:0/3048267459 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:41.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.931+0000 7f3a7aa00700 1 -- 172.21.15.86:0/3048267459 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a7c10d580 msgr2=0x7f3a7c19dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:41.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.931+0000 7f3a7aa00700 1 --2- 172.21.15.86:0/3048267459 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a7c10d580 0x7f3a7c19dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:41.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.931+0000 7f3a7aa00700 1 -- 172.21.15.86:0/3048267459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a7c073aa0 msgr2=0x7f3a7c1999b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:41.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.931+0000 7f3a7aa00700 1 --2- 172.21.15.86:0/3048267459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a7c073aa0 0x7f3a7c1999b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:41.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.931+0000 7f3a7aa00700 1 -- 172.21.15.86:0/3048267459 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a680095f0 con 0x7f3a7c073180 2024-03-23T17:52:41.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.931+0000 7f3a7b400700 1 --2- 172.21.15.86:0/3048267459 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a7c10d580 0x7f3a7c19dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:52:41.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.931+0000 7f3a7aa00700 1 --2- 172.21.15.86:0/3048267459 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a7c073180 0x7f3a7c199470 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c00e9f0 tx=0x7f3a6c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:41.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.931+0000 7f3a73400700 1 -- 172.21.15.86:0/3048267459 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a6c00c9c0 con 0x7f3a7c073180 2024-03-23T17:52:41.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.931+0000 7f3a73400700 1 -- 172.21.15.86:0/3048267459 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a6c010430 con 0x7f3a7c073180 2024-03-23T17:52:41.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.931+0000 7f3a81a00700 1 -- 172.21.15.86:0/3048267459 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a7c19e9a0 con 0x7f3a7c073180 2024-03-23T17:52:41.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.931+0000 7f3a73400700 1 -- 172.21.15.86:0/3048267459 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a6c018680 con 0x7f3a7c073180 2024-03-23T17:52:41.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.932+0000 7f3a81a00700 1 -- 172.21.15.86:0/3048267459 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a7c1a61e0 con 0x7f3a7c073180 2024-03-23T17:52:41.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.934+0000 7f3a73400700 1 -- 172.21.15.86:0/3048267459 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f3a6c010860 con 0x7f3a7c073180 2024-03-23T17:52:41.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.936+0000 7f3a73400700 1 --2- 172.21.15.86:0/3048267459 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f3a600a1310 0x7f3a600a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:41.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.936+0000 7f3a73400700 1 -- 172.21.15.86:0/3048267459 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f3a6c014070 con 0x7f3a7c073180 2024-03-23T17:52:41.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.936+0000 7f3a7a000700 1 --2- 172.21.15.86:0/3048267459 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f3a600a1310 0x7f3a600a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:41.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.936+0000 7f3a954f5b80 1 Processor -- start 2024-03-23T17:52:41.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.937+0000 7f3a954f5b80 1 -- start start 2024-03-23T17:52:41.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.937+0000 7f3a7a000700 1 --2- 172.21.15.86:0/3048267459 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f3a600a1310 0x7f3a600a37d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c19a9c0 tx=0x7f3a6800b260 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:41.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.937+0000 7f3a954f5b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56021c5f65d0 0x56021c5f69b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:41.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.937+0000 7f3a954f5b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56021c5f6ef0 0x56021c6018b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:41.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.937+0000 7f3a954f5b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56021c601df0 0x56021c6041e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:41.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.937+0000 7f3a954f5b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56021c54f990 con 0x56021c5f6ef0 2024-03-23T17:52:41.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.937+0000 7f3a954f5b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x56021c54d350 con 0x56021c601df0 2024-03-23T17:52:41.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.937+0000 7f3a954f5b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x56021c548c40 con 0x56021c5f65d0 2024-03-23T17:52:41.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.937+0000 7f3a7a000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56021c5f6ef0 0x56021c6018b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:41.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.937+0000 7f3a7aa00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56021c5f65d0 0x56021c5f69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:41.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.937+0000 7f3a7aa00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56021c5f65d0 0x56021c5f69b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:52460/0 (socket says 172.21.15.86:52460) 2024-03-23T17:52:41.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.937+0000 7f3a7aa00700 1 -- 172.21.15.86:0/2714402634 learned_addr learned my addr 172.21.15.86:0/2714402634 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:41.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.937+0000 7f3a7b400700 1 --2- 172.21.15.86:0/2714402634 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56021c601df0 0x56021c6041e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:41.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.938+0000 7f3a7aa00700 1 -- 172.21.15.86:0/2714402634 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56021c601df0 msgr2=0x56021c6041e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:41.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.938+0000 7f3a7aa00700 1 --2- 172.21.15.86:0/2714402634 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56021c601df0 0x56021c6041e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:41.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.938+0000 7f3a7aa00700 1 -- 172.21.15.86:0/2714402634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56021c5f6ef0 msgr2=0x56021c6018b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:41.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.938+0000 7f3a7aa00700 1 --2- 172.21.15.86:0/2714402634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56021c5f6ef0 0x56021c6018b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:41.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.938+0000 7f3a7aa00700 1 -- 172.21.15.86:0/2714402634 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56021c3ca3b0 con 0x56021c5f65d0 2024-03-23T17:52:41.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.938+0000 7f3a7aa00700 1 --2- 172.21.15.86:0/2714402634 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56021c5f65d0 0x56021c5f69b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c0968c0 tx=0x7f3a6c0998b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=252a48b0e16bfdf0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:41.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.938+0000 7f3a67e00700 1 -- 172.21.15.86:0/2714402634 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a6c09dc80 con 0x56021c5f65d0 2024-03-23T17:52:41.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.938+0000 7f3a67e00700 1 -- 172.21.15.86:0/2714402634 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a6c0a22d0 con 0x56021c5f65d0 2024-03-23T17:52:41.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.938+0000 7f3a67e00700 1 -- 172.21.15.86:0/2714402634 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a6c09a310 con 0x56021c5f65d0 2024-03-23T17:52:41.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.939+0000 7f3a954f5b80 1 -- 172.21.15.86:0/2714402634 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56021c5f65d0 msgr2=0x56021c5f69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:41.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.939+0000 7f3a954f5b80 1 --2- 172.21.15.86:0/2714402634 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56021c5f65d0 0x56021c5f69b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c0968c0 tx=0x7f3a6c0998b0 comp rx=0 tx=0).stop 2024-03-23T17:52:41.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.939+0000 7f3a954f5b80 1 -- 172.21.15.86:0/2714402634 shutdown_connections 2024-03-23T17:52:41.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.939+0000 7f3a954f5b80 1 --2- 172.21.15.86:0/2714402634 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56021c601df0 0x56021c6041e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:41.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.939+0000 7f3a954f5b80 1 --2- 172.21.15.86:0/2714402634 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56021c5f65d0 0x56021c5f69b0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:41.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.939+0000 7f3a954f5b80 1 --2- 172.21.15.86:0/2714402634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56021c5f6ef0 0x56021c6018b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:41.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.939+0000 7f3a954f5b80 1 -- 172.21.15.86:0/2714402634 >> 172.21.15.86:0/2714402634 conn(0x56021c5566a0 msgr2=0x56021c5f2a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:41.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.939+0000 7f3a954f5b80 1 -- 172.21.15.86:0/2714402634 shutdown_connections 2024-03-23T17:52:41.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.939+0000 7f3a954f5b80 1 -- 172.21.15.86:0/2714402634 wait complete. 2024-03-23T17:52:41.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.940+0000 7f3a954f5b80 1 -- start start 2024-03-23T17:52:41.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.940+0000 7f3a954f5b80 1 Processor -- start 2024-03-23T17:52:41.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.940+0000 7f3a954f5b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56021c5f65d0 0x56021c601810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:41.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.940+0000 7f3a954f5b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56021c5f6ef0 0x56021c5feb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:41.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.940+0000 7f3a954f5b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56021c601df0 0x56021c5ff130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:41.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.940+0000 7f3a954f5b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56021c4e4d80 con 0x56021c601df0 2024-03-23T17:52:41.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.940+0000 7f3a954f5b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x56021c4e4a20 con 0x56021c5f65d0 2024-03-23T17:52:41.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.940+0000 7f3a954f5b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x56021c3cc6a0 con 0x56021c5f6ef0 2024-03-23T17:52:41.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.941+0000 7f3a954f5b80 10 client.0 _ll_register_callbacks cb 0x56021c4ee470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:52:41.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.941+0000 7f3a7aa00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56021c5f65d0 0x56021c601810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:41.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.941+0000 7f3a7aa00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56021c5f65d0 0x56021c601810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:45894/0 (socket says 172.21.15.86:45894) 2024-03-23T17:52:41.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.941+0000 7f3a7aa00700 1 -- 172.21.15.86:0/3261938141 learned_addr learned my addr 172.21.15.86:0/3261938141 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:41.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.941+0000 7f3a7a000700 1 --2- 172.21.15.86:0/3261938141 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56021c5f6ef0 0x56021c5feb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:41.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.941+0000 7f3a7b400700 1 --2- 172.21.15.86:0/3261938141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56021c601df0 0x56021c5ff130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:41.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.941+0000 7f3a7aa00700 1 -- 172.21.15.86:0/3261938141 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56021c5f6ef0 msgr2=0x56021c5feb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:41.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.941+0000 7f3a7aa00700 1 --2- 172.21.15.86:0/3261938141 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56021c5f6ef0 0x56021c5feb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:41.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.941+0000 7f3a7aa00700 1 -- 172.21.15.86:0/3261938141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56021c601df0 msgr2=0x56021c5ff130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:41.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.941+0000 7f3a7aa00700 1 --2- 172.21.15.86:0/3261938141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56021c601df0 0x56021c5ff130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:41.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.941+0000 7f3a7aa00700 1 -- 172.21.15.86:0/3261938141 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a7c1a61e0 con 0x56021c5f65d0 2024-03-23T17:52:41.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.941+0000 7f3a7aa00700 1 --2- 172.21.15.86:0/3261938141 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56021c5f65d0 0x56021c601810 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c099b50 tx=0x7f3a6c099fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d84c6fc7a37279b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:41.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.941+0000 7f3a64c00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T17:52:41.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.942+0000 7f3a64c00700 1 -- 172.21.15.86:0/3261938141 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a6c09dc80 con 0x56021c5f65d0 2024-03-23T17:52:41.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.942+0000 7f3a64c00700 1 -- 172.21.15.86:0/3261938141 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f3a6c09a2c0 con 0x56021c5f65d0 2024-03-23T17:52:41.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.942+0000 7f3a954f5b80 1 -- 172.21.15.86:0/3261938141 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56021c509380 con 0x56021c5f65d0 2024-03-23T17:52:41.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.942+0000 7f3a64c00700 1 -- 172.21.15.86:0/3261938141 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a6c0a1e40 con 0x56021c5f65d0 2024-03-23T17:52:41.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.942+0000 7f3a954f5b80 10 client.26147 fetch_fsmap learned FSMap version 33 2024-03-23T17:52:41.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.942+0000 7f3a954f5b80 1 -- 172.21.15.86:0/3261938141 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x56021c376b60 con 0x56021c5f65d0 2024-03-23T17:52:41.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.942+0000 7f3a64c00700 1 -- 172.21.15.86:0/3261938141 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a6c07c020 con 0x56021c5f65d0 2024-03-23T17:52:41.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.942+0000 7f3a64c00700 1 -- 172.21.15.86:0/3261938141 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 33) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f3a6c0add20 con 0x56021c5f65d0 2024-03-23T17:52:41.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.942+0000 7f3a954f5b80 10 client.26147 fetch_fsmap finished waiting for FSMap version 33 2024-03-23T17:52:41.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.942+0000 7f3a954f5b80 10 client.26147 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.d 2024-03-23T17:52:41.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.942+0000 7f3a954f5b80 20 client.26147 populate_metadata read hostname 'smithi086' 2024-03-23T17:52:41.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.943+0000 7f3a954f5b80 1 --2- 172.21.15.86:0/3261938141 >> [v2:172.21.15.86:6836/2933881328,v1:172.21.15.86:6837/2933881328] conn(0x56021c611570 0x56021c613960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:41.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.943+0000 7f3a954f5b80 4 client.26147 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:52:41.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.943+0000 7f3a954f5b80 1 -- 172.21.15.86:0/3261938141 --> [v2:172.21.15.86:6836/2933881328,v1:172.21.15.86:6837/2933881328] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56021c541700 con 0x56021c611570 2024-03-23T17:52:41.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.943+0000 7f3a7a000700 1 --2- 172.21.15.86:0/3261938141 >> [v2:172.21.15.86:6836/2933881328,v1:172.21.15.86:6837/2933881328] conn(0x56021c611570 0x56021c613960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:41.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.943+0000 7f3a7a000700 1 --2- 172.21.15.86:0/3261938141 >> [v2:172.21.15.86:6836/2933881328,v1:172.21.15.86:6837/2933881328] conn(0x56021c611570 0x56021c613960 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=617d7a9ae4042c67 server_cookie=cea55293d50eff3d in_seq=0 out_seq=0 2024-03-23T17:52:41.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.943+0000 7f3a64c00700 10 client.26147 ms_handle_connect on v2:172.21.15.86:6836/2933881328 2024-03-23T17:52:41.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.944+0000 7f3a64c00700 1 -- 172.21.15.86:0/3261938141 <== mds.3 v2:172.21.15.86:6836/2933881328 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x56021c541700 con 0x56021c611570 2024-03-23T17:52:41.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.944+0000 7f3a64c00700 10 client.26147 handle_command_reply: tid=0 2024-03-23T17:52:41.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.944+0000 7f3a954f5b80 1 client.26147 shutdown 2024-03-23T17:52:41.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.944+0000 7f3a954f5b80 1 -- 172.21.15.86:0/3261938141 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56021c5f65d0 msgr2=0x56021c601810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:41.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.944+0000 7f3a954f5b80 1 --2- 172.21.15.86:0/3261938141 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56021c5f65d0 0x56021c601810 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c099b50 tx=0x7f3a6c099fd0 comp rx=0 tx=0).stop 2024-03-23T17:52:41.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.944+0000 7f3a954f5b80 1 -- 172.21.15.86:0/3261938141 shutdown_connections 2024-03-23T17:52:41.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.944+0000 7f3a954f5b80 1 --2- 172.21.15.86:0/3261938141 >> [v2:172.21.15.86:6836/2933881328,v1:172.21.15.86:6837/2933881328] conn(0x56021c611570 0x56021c613960 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:41.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.944+0000 7f3a954f5b80 1 --2- 172.21.15.86:0/3261938141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56021c601df0 0x56021c5ff130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:41.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.944+0000 7f3a954f5b80 1 --2- 172.21.15.86:0/3261938141 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56021c5f65d0 0x56021c601810 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:41.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.944+0000 7f3a954f5b80 1 --2- 172.21.15.86:0/3261938141 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56021c5f6ef0 0x56021c5feb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:41.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.944+0000 7f3a64c00700 0 client.26147 ms_handle_reset on v2:172.21.15.86:6836/2933881328 2024-03-23T17:52:41.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.944+0000 7f3a954f5b80 1 -- 172.21.15.86:0/3261938141 >> 172.21.15.86:0/3261938141 conn(0x56021c5566a0 msgr2=0x56021c5f86c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:41.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.944+0000 7f3a954f5b80 1 -- 172.21.15.86:0/3261938141 shutdown_connections 2024-03-23T17:52:41.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.944+0000 7f3a954f5b80 1 -- 172.21.15.86:0/3261938141 wait complete. 2024-03-23T17:52:41.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.945+0000 7f3a954f5b80 20 client.26147 trim_cache size 0 max 16384 2024-03-23T17:52:41.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.959+0000 7f3a954f5b80 1 Processor -- start 2024-03-23T17:52:41.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.959+0000 7f3a954f5b80 1 -- start start 2024-03-23T17:52:41.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.959+0000 7f3a954f5b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56021c5f65d0 0x56021c618290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:41.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.959+0000 7f3a954f5b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56021c5f6ef0 0x56021c5fd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:41.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.959+0000 7f3a954f5b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56021c601df0 0x56021c5fd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:41.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.959+0000 7f3a954f5b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56021c5ff710 con 0x56021c601df0 2024-03-23T17:52:41.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.959+0000 7f3a954f5b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x56021c5fde50 con 0x56021c5f6ef0 2024-03-23T17:52:41.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.959+0000 7f3a954f5b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x56021c5ff4b0 con 0x56021c5f65d0 2024-03-23T17:52:41.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.960+0000 7f3a7aa00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56021c5f65d0 0x56021c618290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:41.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.960+0000 7f3a7b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56021c601df0 0x56021c5fd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:41.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.960+0000 7f3a7a000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56021c5f6ef0 0x56021c5fd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:41.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.960+0000 7f3a7aa00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56021c5f65d0 0x56021c618290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:52490/0 (socket says 172.21.15.86:52490) 2024-03-23T17:52:41.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.960+0000 7f3a7a000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56021c5f6ef0 0x56021c5fd3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:45898/0 (socket says 172.21.15.86:45898) 2024-03-23T17:52:41.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.960+0000 7f3a7aa00700 1 -- 172.21.15.86:0/3648390427 learned_addr learned my addr 172.21.15.86:0/3648390427 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:41.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.960+0000 7f3a7aa00700 1 -- 172.21.15.86:0/3648390427 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56021c5f6ef0 msgr2=0x56021c5fd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:41.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.960+0000 7f3a7aa00700 1 --2- 172.21.15.86:0/3648390427 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56021c5f6ef0 0x56021c5fd3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:41.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.960+0000 7f3a7aa00700 1 -- 172.21.15.86:0/3648390427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56021c601df0 msgr2=0x56021c5fd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:41.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.960+0000 7f3a7aa00700 1 --2- 172.21.15.86:0/3648390427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56021c601df0 0x56021c5fd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:41.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.960+0000 7f3a7aa00700 1 -- 172.21.15.86:0/3648390427 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56021c50eea0 con 0x56021c5f65d0 2024-03-23T17:52:41.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.960+0000 7f3a7aa00700 1 --2- 172.21.15.86:0/3648390427 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56021c5f65d0 0x56021c618290 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c000c00 tx=0x7f3a6c00e940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6daef2fabaea24a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:41.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.960+0000 7f3a70c00700 1 -- 172.21.15.86:0/3648390427 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a6c0995f0 con 0x56021c5f65d0 2024-03-23T17:52:41.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.960+0000 7f3a70c00700 1 -- 172.21.15.86:0/3648390427 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a6c07c020 con 0x56021c5f65d0 2024-03-23T17:52:41.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.960+0000 7f3a70c00700 1 -- 172.21.15.86:0/3648390427 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a6c098310 con 0x56021c5f65d0 2024-03-23T17:52:41.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.961+0000 7f3a954f5b80 1 -- 172.21.15.86:0/3648390427 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56021c5f65d0 msgr2=0x56021c618290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:41.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.961+0000 7f3a954f5b80 1 --2- 172.21.15.86:0/3648390427 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56021c5f65d0 0x56021c618290 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c000c00 tx=0x7f3a6c00e940 comp rx=0 tx=0).stop 2024-03-23T17:52:41.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.961+0000 7f3a954f5b80 1 -- 172.21.15.86:0/3648390427 shutdown_connections 2024-03-23T17:52:41.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.961+0000 7f3a954f5b80 1 --2- 172.21.15.86:0/3648390427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56021c601df0 0x56021c5fd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:41.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.961+0000 7f3a954f5b80 1 --2- 172.21.15.86:0/3648390427 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56021c5f65d0 0x56021c618290 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:41.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.961+0000 7f3a954f5b80 1 --2- 172.21.15.86:0/3648390427 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56021c5f6ef0 0x56021c5fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:41.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.961+0000 7f3a954f5b80 1 -- 172.21.15.86:0/3648390427 >> 172.21.15.86:0/3648390427 conn(0x56021c5566a0 msgr2=0x56021c5f9e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:41.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.962+0000 7f3a954f5b80 1 -- 172.21.15.86:0/3648390427 shutdown_connections 2024-03-23T17:52:41.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.962+0000 7f3a954f5b80 1 -- 172.21.15.86:0/3648390427 wait complete. 2024-03-23T17:52:41.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.962+0000 7f3a954f5b80 1 -- start start 2024-03-23T17:52:41.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.962+0000 7f3a954f5b80 1 Processor -- start 2024-03-23T17:52:41.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.963+0000 7f3a954f5b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56021c5f65d0 0x56021c608700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:41.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.963+0000 7f3a954f5b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56021c5f6ef0 0x56021c606be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:41.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.963+0000 7f3a954f5b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56021c601df0 0x56021c607120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:41.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.963+0000 7f3a954f5b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56021c6076a0 con 0x56021c601df0 2024-03-23T17:52:41.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.963+0000 7f3a954f5b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x56021c6077e0 con 0x56021c5f65d0 2024-03-23T17:52:41.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.963+0000 7f3a954f5b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x56021c6079a0 con 0x56021c5f6ef0 2024-03-23T17:52:41.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.963+0000 7f3a954f5b80 10 client.0 _ll_register_callbacks cb 0x56021c4ee470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:52:41.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.963+0000 7f3a7aa00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56021c5f65d0 0x56021c608700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:41.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.963+0000 7f3a7a000700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56021c5f6ef0 0x56021c606be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:41.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.963+0000 7f3a7aa00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56021c5f65d0 0x56021c608700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:45904/0 (socket says 172.21.15.86:45904) 2024-03-23T17:52:41.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.963+0000 7f3a7aa00700 1 -- 172.21.15.86:0/2382649964 learned_addr learned my addr 172.21.15.86:0/2382649964 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:41.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.963+0000 7f3a7b400700 1 --2- 172.21.15.86:0/2382649964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56021c601df0 0x56021c607120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:41.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.963+0000 7f3a7aa00700 1 -- 172.21.15.86:0/2382649964 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56021c5f6ef0 msgr2=0x56021c606be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:41.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.963+0000 7f3a7aa00700 1 --2- 172.21.15.86:0/2382649964 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56021c5f6ef0 0x56021c606be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:41.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.963+0000 7f3a7aa00700 1 -- 172.21.15.86:0/2382649964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56021c601df0 msgr2=0x56021c607120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:41.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.963+0000 7f3a7aa00700 1 --2- 172.21.15.86:0/2382649964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56021c601df0 0x56021c607120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:41.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.964+0000 7f3a7aa00700 1 -- 172.21.15.86:0/2382649964 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56021c376b60 con 0x56021c5f65d0 2024-03-23T17:52:41.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.964+0000 7f3a7a000700 1 --2- 172.21.15.86:0/2382649964 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56021c5f6ef0 0x56021c606be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:52:41.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.964+0000 7f3a7aa00700 1 --2- 172.21.15.86:0/2382649964 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56021c5f65d0 0x56021c608700 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c0cc040 tx=0x7f3a6c009790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ee6cad60882c6db server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:41.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.964+0000 7f3a64c00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T17:52:41.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.964+0000 7f3a64c00700 1 -- 172.21.15.86:0/2382649964 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a6c0995f0 con 0x56021c5f65d0 2024-03-23T17:52:41.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.964+0000 7f3a64c00700 1 -- 172.21.15.86:0/2382649964 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f3a6c07d080 con 0x56021c5f65d0 2024-03-23T17:52:41.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.964+0000 7f3a954f5b80 1 -- 172.21.15.86:0/2382649964 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3a6c0a22d0 con 0x56021c5f65d0 2024-03-23T17:52:41.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.964+0000 7f3a64c00700 1 -- 172.21.15.86:0/2382649964 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a6c098bd0 con 0x56021c5f65d0 2024-03-23T17:52:41.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.964+0000 7f3a954f5b80 10 client.26153 fetch_fsmap learned FSMap version 33 2024-03-23T17:52:41.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.964+0000 7f3a954f5b80 1 -- 172.21.15.86:0/2382649964 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x56021c3cecd0 con 0x56021c5f65d0 2024-03-23T17:52:41.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.964+0000 7f3a64c00700 1 -- 172.21.15.86:0/2382649964 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a6c0a22d0 con 0x56021c5f65d0 2024-03-23T17:52:41.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.965+0000 7f3a64c00700 1 -- 172.21.15.86:0/2382649964 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 33) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f3a6c0add20 con 0x56021c5f65d0 2024-03-23T17:52:41.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.965+0000 7f3a954f5b80 10 client.26153 fetch_fsmap finished waiting for FSMap version 33 2024-03-23T17:52:41.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.965+0000 7f3a954f5b80 10 client.26153 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.d 2024-03-23T17:52:41.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.965+0000 7f3a954f5b80 20 client.26153 populate_metadata read hostname 'smithi086' 2024-03-23T17:52:41.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.965+0000 7f3a954f5b80 1 --2- 172.21.15.86:0/2382649964 >> [v2:172.21.15.86:6836/2933881328,v1:172.21.15.86:6837/2933881328] conn(0x56021c6640e0 0x56021c6664d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:41.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.965+0000 7f3a954f5b80 4 client.26153 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-23T17:52:41.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.965+0000 7f3a954f5b80 1 -- 172.21.15.86:0/2382649964 --> [v2:172.21.15.86:6836/2933881328,v1:172.21.15.86:6837/2933881328] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56021c397cf0 con 0x56021c6640e0 2024-03-23T17:52:41.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.965+0000 7f3a7a000700 1 --2- 172.21.15.86:0/2382649964 >> [v2:172.21.15.86:6836/2933881328,v1:172.21.15.86:6837/2933881328] conn(0x56021c6640e0 0x56021c6664d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:41.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.965+0000 7f3a7a000700 1 --2- 172.21.15.86:0/2382649964 >> [v2:172.21.15.86:6836/2933881328,v1:172.21.15.86:6837/2933881328] conn(0x56021c6640e0 0x56021c6664d0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=709a70ab944e0fd2 server_cookie=7f60681c272678a4 in_seq=0 out_seq=0 2024-03-23T17:52:41.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.965+0000 7f3a64c00700 10 client.26153 ms_handle_connect on v2:172.21.15.86:6836/2933881328 2024-03-23T17:52:41.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.966+0000 7f3a64c00700 1 -- 172.21.15.86:0/2382649964 <== mds.3 v2:172.21.15.86:6836/2933881328 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+1150 (crc 0 0 0) 0x56021c397cf0 con 0x56021c6640e0 2024-03-23T17:52:41.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.966+0000 7f3a64c00700 10 client.26153 handle_command_reply: tid=0 2024-03-23T17:52:41.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.966+0000 7f3a954f5b80 1 client.26153 shutdown 2024-03-23T17:52:41.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.966+0000 7f3a954f5b80 1 -- 172.21.15.86:0/2382649964 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56021c5f65d0 msgr2=0x56021c608700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:41.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.966+0000 7f3a954f5b80 1 --2- 172.21.15.86:0/2382649964 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56021c5f65d0 0x56021c608700 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c0cc040 tx=0x7f3a6c009790 comp rx=0 tx=0).stop 2024-03-23T17:52:41.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.966+0000 7f3a954f5b80 1 -- 172.21.15.86:0/2382649964 shutdown_connections 2024-03-23T17:52:41.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.966+0000 7f3a954f5b80 1 --2- 172.21.15.86:0/2382649964 >> [v2:172.21.15.86:6836/2933881328,v1:172.21.15.86:6837/2933881328] conn(0x56021c6640e0 0x56021c6664d0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:41.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.966+0000 7f3a954f5b80 1 --2- 172.21.15.86:0/2382649964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56021c601df0 0x56021c607120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:41.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.966+0000 7f3a954f5b80 1 --2- 172.21.15.86:0/2382649964 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56021c5f65d0 0x56021c608700 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:41.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.966+0000 7f3a954f5b80 1 --2- 172.21.15.86:0/2382649964 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56021c5f6ef0 0x56021c606be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:41.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.966+0000 7f3a64c00700 0 client.26153 ms_handle_reset on v2:172.21.15.86:6836/2933881328 2024-03-23T17:52:41.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.966+0000 7f3a954f5b80 1 -- 172.21.15.86:0/2382649964 >> 172.21.15.86:0/2382649964 conn(0x56021c5566a0 msgr2=0x56021c5fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:41.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.966+0000 7f3a954f5b80 1 -- 172.21.15.86:0/2382649964 shutdown_connections 2024-03-23T17:52:41.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.966+0000 7f3a954f5b80 1 -- 172.21.15.86:0/2382649964 wait complete. 2024-03-23T17:52:41.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.966+0000 7f3a954f5b80 20 client.26153 trim_cache size 0 max 16384 2024-03-23T17:52:41.971 INFO:teuthology.orchestra.run.smithi086.stdout:[ 2024-03-23T17:52:41.971 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T17:52:41.971 INFO:teuthology.orchestra.run.smithi086.stdout: "damage_type": "backtrace", 2024-03-23T17:52:41.972 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 178463209, 2024-03-23T17:52:41.972 INFO:teuthology.orchestra.run.smithi086.stdout: "ino": 1099511628299, 2024-03-23T17:52:41.972 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/WORD/FACTS.DOC" 2024-03-23T17:52:41.972 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T17:52:41.972 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T17:52:41.972 INFO:teuthology.orchestra.run.smithi086.stdout: "damage_type": "backtrace", 2024-03-23T17:52:41.972 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 1280717372, 2024-03-23T17:52:41.972 INFO:teuthology.orchestra.run.smithi086.stdout: "ino": 1099511628301, 2024-03-23T17:52:41.972 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/EXCEL/PCMAGCD.XLS" 2024-03-23T17:52:41.972 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T17:52:41.972 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T17:52:41.972 INFO:teuthology.orchestra.run.smithi086.stdout: "damage_type": "backtrace", 2024-03-23T17:52:41.972 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 1739423601, 2024-03-23T17:52:41.973 INFO:teuthology.orchestra.run.smithi086.stdout: "ino": 1099511628302, 2024-03-23T17:52:41.973 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/EXCEL/SALES.XLS" 2024-03-23T17:52:41.973 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T17:52:41.973 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T17:52:41.973 INFO:teuthology.orchestra.run.smithi086.stdout: "damage_type": "backtrace", 2024-03-23T17:52:41.973 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 2686714892, 2024-03-23T17:52:41.973 INFO:teuthology.orchestra.run.smithi086.stdout: "ino": 1099511628298, 2024-03-23T17:52:41.973 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/WORD/BASEMACH.DOC" 2024-03-23T17:52:41.973 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T17:52:41.973 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T17:52:41.973 INFO:teuthology.orchestra.run.smithi086.stdout: "damage_type": "backtrace", 2024-03-23T17:52:41.973 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 3180308212, 2024-03-23T17:52:41.973 INFO:teuthology.orchestra.run.smithi086.stdout: "ino": 1099511628297, 2024-03-23T17:52:41.973 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/WORD/CHAP10.DOC" 2024-03-23T17:52:41.973 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T17:52:41.974 INFO:teuthology.orchestra.run.smithi086.stdout:] 2024-03-23T17:52:41.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.967+0000 7f3a81a00700 1 -- 172.21.15.86:0/3048267459 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f3a600a1310 msgr2=0x7f3a600a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:41.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.967+0000 7f3a81a00700 1 --2- 172.21.15.86:0/3048267459 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f3a600a1310 0x7f3a600a37d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c19a9c0 tx=0x7f3a6800b260 comp rx=0 tx=0).stop 2024-03-23T17:52:41.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.967+0000 7f3a81a00700 1 -- 172.21.15.86:0/3048267459 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a7c073180 msgr2=0x7f3a7c199470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:41.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.967+0000 7f3a81a00700 1 --2- 172.21.15.86:0/3048267459 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a7c073180 0x7f3a7c199470 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c00e9f0 tx=0x7f3a6c00ed00 comp rx=0 tx=0).stop 2024-03-23T17:52:41.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.967+0000 7f3a81a00700 1 -- 172.21.15.86:0/3048267459 shutdown_connections 2024-03-23T17:52:41.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.967+0000 7f3a81a00700 1 --2- 172.21.15.86:0/3048267459 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a7c10d580 0x7f3a7c19dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:41.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.967+0000 7f3a81a00700 1 --2- 172.21.15.86:0/3048267459 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f3a600a1310 0x7f3a600a37d0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:41.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.967+0000 7f3a81a00700 1 --2- 172.21.15.86:0/3048267459 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a7c073180 0x7f3a7c199470 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:41.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.967+0000 7f3a81a00700 1 --2- 172.21.15.86:0/3048267459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a7c073aa0 0x7f3a7c1999b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:41.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.967+0000 7f3a81a00700 1 -- 172.21.15.86:0/3048267459 >> 172.21.15.86:0/3048267459 conn(0x7f3a7c0fc7f0 msgr2=0x7f3a7c0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:41.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.967+0000 7f3a81a00700 1 -- 172.21.15.86:0/3048267459 shutdown_connections 2024-03-23T17:52:41.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:41.967+0000 7f3a81a00700 1 -- 172.21.15.86:0/3048267459 wait complete. 2024-03-23T17:52:41.985 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:41 smithi086 ceph-mon[38221]: pgmap v64: 97 pgs: 97 active+clean; 34 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 569 KiB/s wr, 110 op/s 2024-03-23T17:52:41.985 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:41 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/435626673' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T17:52:41.987 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-23T17:52:41.988 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-23T17:52:42.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.198+0000 7f848c000700 1 -- 172.21.15.86:0/1860970216 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8484101920 msgr2=0x7f8484105960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:42.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.198+0000 7f848c000700 1 --2- 172.21.15.86:0/1860970216 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8484101920 0x7f8484105960 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8470009990 tx=0x7f8470009ca0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.198+0000 7f848c000700 1 -- 172.21.15.86:0/1860970216 shutdown_connections 2024-03-23T17:52:42.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.198+0000 7f848c000700 1 --2- 172.21.15.86:0/1860970216 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8484106090 0x7f848410dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.198+0000 7f848c000700 1 --2- 172.21.15.86:0/1860970216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8484100f70 0x7f8484101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.198+0000 7f848c000700 1 --2- 172.21.15.86:0/1860970216 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8484101920 0x7f8484105960 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.198+0000 7f848c000700 1 -- 172.21.15.86:0/1860970216 >> 172.21.15.86:0/1860970216 conn(0x7f84840fc7f0 msgr2=0x7f84840fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:42.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.198+0000 7f848c000700 1 -- 172.21.15.86:0/1860970216 shutdown_connections 2024-03-23T17:52:42.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.198+0000 7f848c000700 1 -- 172.21.15.86:0/1860970216 wait complete. 2024-03-23T17:52:42.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.200+0000 7f848c000700 1 Processor -- start 2024-03-23T17:52:42.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.200+0000 7f848c000700 1 -- start start 2024-03-23T17:52:42.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.200+0000 7f848c000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8484100f70 0x7f8484199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:42.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.200+0000 7f848c000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8484106090 0x7f84841999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:42.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.200+0000 7f848c000700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f848419dd10 0x7f848419e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:42.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.200+0000 7f848c000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f848419e790 con 0x7f8484106090 2024-03-23T17:52:42.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.200+0000 7f848c000700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f848419e900 con 0x7f8484100f70 2024-03-23T17:52:42.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.200+0000 7f848c000700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f848419eac0 con 0x7f848419dd10 2024-03-23T17:52:42.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.201+0000 7f8489600700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8484100f70 0x7f8484199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:42.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.201+0000 7f8489600700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8484100f70 0x7f8484199480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:45916/0 (socket says 172.21.15.86:45916) 2024-03-23T17:52:42.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.201+0000 7f8489600700 1 -- 172.21.15.86:0/2199470427 learned_addr learned my addr 172.21.15.86:0/2199470427 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:42.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.201+0000 7f848a000700 1 --2- 172.21.15.86:0/2199470427 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f848419dd10 0x7f848419e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:42.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.201+0000 7f8488c00700 1 --2- 172.21.15.86:0/2199470427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8484106090 0x7f84841999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:42.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.201+0000 7f8489600700 1 -- 172.21.15.86:0/2199470427 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f848419dd10 msgr2=0x7f848419e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:42.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.201+0000 7f8489600700 1 --2- 172.21.15.86:0/2199470427 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f848419dd10 0x7f848419e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.201+0000 7f8489600700 1 -- 172.21.15.86:0/2199470427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8484106090 msgr2=0x7f84841999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:42.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.201+0000 7f8489600700 1 --2- 172.21.15.86:0/2199470427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8484106090 0x7f84841999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.201+0000 7f8489600700 1 -- 172.21.15.86:0/2199470427 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f847c009c40 con 0x7f8484100f70 2024-03-23T17:52:42.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.201+0000 7f8489600700 1 --2- 172.21.15.86:0/2199470427 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8484100f70 0x7f8484199480 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f847800e9a0 tx=0x7f847800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:42.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.201+0000 7f8482000700 1 -- 172.21.15.86:0/2199470427 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8478009790 con 0x7f8484100f70 2024-03-23T17:52:42.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.201+0000 7f8482000700 1 -- 172.21.15.86:0/2199470427 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f847800ce80 con 0x7f8484100f70 2024-03-23T17:52:42.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.201+0000 7f848c000700 1 -- 172.21.15.86:0/2199470427 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84700095f0 con 0x7f8484100f70 2024-03-23T17:52:42.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.202+0000 7f8482000700 1 -- 172.21.15.86:0/2199470427 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8478018680 con 0x7f8484100f70 2024-03-23T17:52:42.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.202+0000 7f848c000700 1 -- 172.21.15.86:0/2199470427 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84841a6310 con 0x7f8484100f70 2024-03-23T17:52:42.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.205+0000 7f8482000700 1 -- 172.21.15.86:0/2199470427 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f8478010430 con 0x7f8484100f70 2024-03-23T17:52:42.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.207+0000 7f849fb64b80 1 Processor -- start 2024-03-23T17:52:42.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.207+0000 7f849fb64b80 1 -- start start 2024-03-23T17:52:42.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.207+0000 7f8482000700 1 --2- 172.21.15.86:0/2199470427 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f846c0a1290 0x7f846c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:42.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.207+0000 7f849fb64b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c986635d0 0x560c986639b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:42.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.207+0000 7f849fb64b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c98663ef0 0x560c9866e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:42.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.207+0000 7f849fb64b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c9866edf0 0x560c986711e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:42.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.207+0000 7f849fb64b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c985bc990 con 0x560c98663ef0 2024-03-23T17:52:42.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.207+0000 7f849fb64b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c985ba350 con 0x560c986635d0 2024-03-23T17:52:42.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.207+0000 7f849fb64b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c985b5c40 con 0x560c9866edf0 2024-03-23T17:52:42.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.207+0000 7f8482000700 1 -- 172.21.15.86:0/2199470427 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f8478014070 con 0x7f8484100f70 2024-03-23T17:52:42.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.207+0000 7f8489600700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c986635d0 0x560c986639b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:42.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.207+0000 7f848a000700 1 --2- 172.21.15.86:0/2199470427 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f846c0a1290 0x7f846c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:42.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.208+0000 7f8489600700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c986635d0 0x560c986639b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:45928/0 (socket says 172.21.15.86:45928) 2024-03-23T17:52:42.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.208+0000 7f8489600700 1 -- 172.21.15.86:0/2426639870 learned_addr learned my addr 172.21.15.86:0/2426639870 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:42.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.208+0000 7f848a000700 1 --2- 172.21.15.86:0/2426639870 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c9866edf0 0x560c986711e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:42.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.208+0000 7f8488c00700 1 --2- 172.21.15.86:0/2426639870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c98663ef0 0x560c9866e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:42.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.208+0000 7f8489600700 1 -- 172.21.15.86:0/2426639870 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c9866edf0 msgr2=0x560c986711e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:42.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.208+0000 7f8489600700 1 --2- 172.21.15.86:0/2426639870 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c9866edf0 0x560c986711e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.208+0000 7f8489600700 1 -- 172.21.15.86:0/2426639870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c98663ef0 msgr2=0x560c9866e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:42.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.208+0000 7f8489600700 1 --2- 172.21.15.86:0/2426639870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c98663ef0 0x560c9866e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.208+0000 7f8489600700 1 -- 172.21.15.86:0/2426639870 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c984373b0 con 0x560c986635d0 2024-03-23T17:52:42.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.208+0000 7f848a000700 1 --2- 172.21.15.86:0/2426639870 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c9866edf0 0x560c986711e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:52:42.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.208+0000 7f8488c00700 1 --2- 172.21.15.86:0/2426639870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c98663ef0 0x560c9866e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:52:42.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.208+0000 7f8489600700 1 --2- 172.21.15.86:0/2426639870 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c986635d0 0x560c986639b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8478096490 tx=0x7f8478099970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffcb50d37b257027 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:42.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.208+0000 7f848a000700 1 --2- 172.21.15.86:0/2199470427 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f846c0a1290 0x7f846c0a3750 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f847c009550 tx=0x7f847c009920 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:42.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.208+0000 7f8476600700 1 -- 172.21.15.86:0/2426639870 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f847809a690 con 0x560c986635d0 2024-03-23T17:52:42.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.208+0000 7f8476600700 1 -- 172.21.15.86:0/2426639870 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f847800c9c0 con 0x560c986635d0 2024-03-23T17:52:42.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.208+0000 7f8476600700 1 -- 172.21.15.86:0/2426639870 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f847800cc90 con 0x560c986635d0 2024-03-23T17:52:42.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.209+0000 7f849fb64b80 1 -- 172.21.15.86:0/2426639870 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c986635d0 msgr2=0x560c986639b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:42.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.209+0000 7f849fb64b80 1 --2- 172.21.15.86:0/2426639870 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c986635d0 0x560c986639b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8478096490 tx=0x7f8478099970 comp rx=0 tx=0).stop 2024-03-23T17:52:42.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.209+0000 7f849fb64b80 1 -- 172.21.15.86:0/2426639870 shutdown_connections 2024-03-23T17:52:42.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.209+0000 7f849fb64b80 1 --2- 172.21.15.86:0/2426639870 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c9866edf0 0x560c986711e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.209+0000 7f849fb64b80 1 --2- 172.21.15.86:0/2426639870 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c986635d0 0x560c986639b0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.209+0000 7f849fb64b80 1 --2- 172.21.15.86:0/2426639870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c98663ef0 0x560c9866e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.209+0000 7f849fb64b80 1 -- 172.21.15.86:0/2426639870 >> 172.21.15.86:0/2426639870 conn(0x560c985c36a0 msgr2=0x560c9865fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:42.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.209+0000 7f849fb64b80 1 -- 172.21.15.86:0/2426639870 shutdown_connections 2024-03-23T17:52:42.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.209+0000 7f849fb64b80 1 -- 172.21.15.86:0/2426639870 wait complete. 2024-03-23T17:52:42.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.210+0000 7f849fb64b80 1 -- start start 2024-03-23T17:52:42.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.210+0000 7f849fb64b80 1 Processor -- start 2024-03-23T17:52:42.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.211+0000 7f849fb64b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c986635d0 0x560c9866e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:42.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.211+0000 7f849fb64b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c98663ef0 0x560c9866bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:42.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.211+0000 7f849fb64b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c9866edf0 0x560c9866c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:42.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.211+0000 7f849fb64b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c98551d80 con 0x560c98663ef0 2024-03-23T17:52:42.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.211+0000 7f849fb64b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c98551a20 con 0x560c986635d0 2024-03-23T17:52:42.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.211+0000 7f849fb64b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c984396a0 con 0x560c9866edf0 2024-03-23T17:52:42.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.211+0000 7f849fb64b80 10 client.0 _ll_register_callbacks cb 0x560c9855b470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:52:42.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.211+0000 7f8489600700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c986635d0 0x560c9866e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:42.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.211+0000 7f8488c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c98663ef0 0x560c9866bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:42.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.211+0000 7f8489600700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c986635d0 0x560c9866e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:45932/0 (socket says 172.21.15.86:45932) 2024-03-23T17:52:42.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.211+0000 7f8489600700 1 -- 172.21.15.86:0/668840012 learned_addr learned my addr 172.21.15.86:0/668840012 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:42.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.211+0000 7f848a000700 1 --2- 172.21.15.86:0/668840012 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c9866edf0 0x560c9866c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:42.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.211+0000 7f8489600700 1 -- 172.21.15.86:0/668840012 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c9866edf0 msgr2=0x560c9866c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:42.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.211+0000 7f8489600700 1 --2- 172.21.15.86:0/668840012 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c9866edf0 0x560c9866c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.211+0000 7f8489600700 1 -- 172.21.15.86:0/668840012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c98663ef0 msgr2=0x560c9866bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:42.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.211+0000 7f8489600700 1 --2- 172.21.15.86:0/668840012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c98663ef0 0x560c9866bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.211+0000 7f8489600700 1 -- 172.21.15.86:0/668840012 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f847c008c00 con 0x560c986635d0 2024-03-23T17:52:42.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.211+0000 7f8488c00700 1 --2- 172.21.15.86:0/668840012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c98663ef0 0x560c9866bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:52:42.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.212+0000 7f8489600700 1 --2- 172.21.15.86:0/668840012 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c986635d0 0x560c9866e7e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8478009230 tx=0x7f8478010a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92c81ff0d633f0c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:42.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.212+0000 7f846aa00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T17:52:42.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.212+0000 7f846aa00700 1 -- 172.21.15.86:0/668840012 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f847809ddc0 con 0x560c986635d0 2024-03-23T17:52:42.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.212+0000 7f846aa00700 1 -- 172.21.15.86:0/668840012 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f84780a2410 con 0x560c986635d0 2024-03-23T17:52:42.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.212+0000 7f849fb64b80 1 -- 172.21.15.86:0/668840012 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560c98576380 con 0x560c986635d0 2024-03-23T17:52:42.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.212+0000 7f846aa00700 1 -- 172.21.15.86:0/668840012 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f847809a1f0 con 0x560c986635d0 2024-03-23T17:52:42.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.212+0000 7f849fb64b80 10 client.26165 fetch_fsmap learned FSMap version 33 2024-03-23T17:52:42.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.212+0000 7f849fb64b80 1 -- 172.21.15.86:0/668840012 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7f84841a6310 con 0x560c986635d0 2024-03-23T17:52:42.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.213+0000 7f846aa00700 1 -- 172.21.15.86:0/668840012 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560c98576380 con 0x560c986635d0 2024-03-23T17:52:42.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.213+0000 7f846aa00700 1 -- 172.21.15.86:0/668840012 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 33) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f84780cad20 con 0x560c986635d0 2024-03-23T17:52:42.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.213+0000 7f849fb64b80 10 client.26165 fetch_fsmap finished waiting for FSMap version 33 2024-03-23T17:52:42.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.213+0000 7f849fb64b80 10 client.26165 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-23T17:52:42.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.213+0000 7f849fb64b80 20 client.26165 populate_metadata read hostname 'smithi086' 2024-03-23T17:52:42.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.213+0000 7f849fb64b80 1 --2- 172.21.15.86:0/668840012 >> [v2:172.21.15.92:6836/3967837948,v1:172.21.15.92:6837/3967837948] conn(0x560c9867e5e0 0x560c986809d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:42.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.213+0000 7f849fb64b80 4 client.26165 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:52:42.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.213+0000 7f849fb64b80 1 -- 172.21.15.86:0/668840012 --> [v2:172.21.15.92:6836/3967837948,v1:172.21.15.92:6837/3967837948] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560c985ae700 con 0x560c9867e5e0 2024-03-23T17:52:42.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.213+0000 7f8488c00700 1 --2- 172.21.15.86:0/668840012 >> [v2:172.21.15.92:6836/3967837948,v1:172.21.15.92:6837/3967837948] conn(0x560c9867e5e0 0x560c986809d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:42.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.214+0000 7f8488c00700 1 --2- 172.21.15.86:0/668840012 >> [v2:172.21.15.92:6836/3967837948,v1:172.21.15.92:6837/3967837948] conn(0x560c9867e5e0 0x560c986809d0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=83f0b815d1ca8998 server_cookie=7a2f13e868cd8048 in_seq=0 out_seq=0 2024-03-23T17:52:42.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.214+0000 7f846aa00700 10 client.26165 ms_handle_connect on v2:172.21.15.92:6836/3967837948 2024-03-23T17:52:42.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.214+0000 7f846aa00700 1 -- 172.21.15.86:0/668840012 <== mds.2 v2:172.21.15.92:6836/3967837948 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x560c985ae700 con 0x560c9867e5e0 2024-03-23T17:52:42.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.214+0000 7f846aa00700 10 client.26165 handle_command_reply: tid=0 2024-03-23T17:52:42.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.214+0000 7f849fb64b80 1 client.26165 shutdown 2024-03-23T17:52:42.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.214+0000 7f849fb64b80 1 -- 172.21.15.86:0/668840012 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c986635d0 msgr2=0x560c9866e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:42.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.214+0000 7f849fb64b80 1 --2- 172.21.15.86:0/668840012 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c986635d0 0x560c9866e7e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8478009230 tx=0x7f8478010a10 comp rx=0 tx=0).stop 2024-03-23T17:52:42.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.215+0000 7f849fb64b80 1 -- 172.21.15.86:0/668840012 shutdown_connections 2024-03-23T17:52:42.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.215+0000 7f849fb64b80 1 --2- 172.21.15.86:0/668840012 >> [v2:172.21.15.92:6836/3967837948,v1:172.21.15.92:6837/3967837948] conn(0x560c9867e5e0 0x560c986809d0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.215+0000 7f849fb64b80 1 --2- 172.21.15.86:0/668840012 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c9866edf0 0x560c9866c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.215+0000 7f846aa00700 0 client.26165 ms_handle_reset on v2:172.21.15.92:6836/3967837948 2024-03-23T17:52:42.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.215+0000 7f849fb64b80 1 --2- 172.21.15.86:0/668840012 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c986635d0 0x560c9866e7e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.215+0000 7f849fb64b80 1 --2- 172.21.15.86:0/668840012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c98663ef0 0x560c9866bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.215+0000 7f849fb64b80 1 -- 172.21.15.86:0/668840012 >> 172.21.15.86:0/668840012 conn(0x560c985c36a0 msgr2=0x560c986656c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:42.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.215+0000 7f849fb64b80 1 -- 172.21.15.86:0/668840012 shutdown_connections 2024-03-23T17:52:42.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.215+0000 7f849fb64b80 1 -- 172.21.15.86:0/668840012 wait complete. 2024-03-23T17:52:42.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.215+0000 7f849fb64b80 20 client.26165 trim_cache size 0 max 16384 2024-03-23T17:52:42.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.228+0000 7f849fb64b80 1 Processor -- start 2024-03-23T17:52:42.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.228+0000 7f849fb64b80 1 -- start start 2024-03-23T17:52:42.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.229+0000 7f849fb64b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c986635d0 0x560c9866c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:42.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.229+0000 7f849fb64b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c98663ef0 0x560c986809d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:42.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.229+0000 7f849fb64b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c9866edf0 0x560c986852d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:42.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.229+0000 7f849fb64b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c9866a0d0 con 0x560c9866edf0 2024-03-23T17:52:42.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.229+0000 7f849fb64b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c9865ff20 con 0x560c986635d0 2024-03-23T17:52:42.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.229+0000 7f849fb64b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c9865b480 con 0x560c98663ef0 2024-03-23T17:52:42.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.229+0000 7f8489600700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c986635d0 0x560c9866c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:42.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.229+0000 7f8489600700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c986635d0 0x560c9866c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:45938/0 (socket says 172.21.15.86:45938) 2024-03-23T17:52:42.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.229+0000 7f8489600700 1 -- 172.21.15.86:0/2165918880 learned_addr learned my addr 172.21.15.86:0/2165918880 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:42.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.229+0000 7f848a000700 1 --2- 172.21.15.86:0/2165918880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c9866edf0 0x560c986852d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:42.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.229+0000 7f8488c00700 1 --2- 172.21.15.86:0/2165918880 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c98663ef0 0x560c986809d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:42.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.229+0000 7f8489600700 1 -- 172.21.15.86:0/2165918880 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c98663ef0 msgr2=0x560c986809d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:42.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.229+0000 7f8489600700 1 --2- 172.21.15.86:0/2165918880 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c98663ef0 0x560c986809d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.229+0000 7f8489600700 1 -- 172.21.15.86:0/2165918880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c9866edf0 msgr2=0x560c986852d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:42.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.229+0000 7f8489600700 1 --2- 172.21.15.86:0/2165918880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c9866edf0 0x560c986852d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.229+0000 7f8489600700 1 -- 172.21.15.86:0/2165918880 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c983e3b60 con 0x560c986635d0 2024-03-23T17:52:42.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.229+0000 7f8488c00700 1 --2- 172.21.15.86:0/2165918880 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c98663ef0 0x560c986809d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:52:42.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.230+0000 7f8489600700 1 --2- 172.21.15.86:0/2165918880 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c986635d0 0x560c9866c110 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8478096490 tx=0x7f8478010a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=300591973b1d0f34 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:42.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.230+0000 7f8477000700 1 -- 172.21.15.86:0/2165918880 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8478049530 con 0x560c986635d0 2024-03-23T17:52:42.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.230+0000 7f8477000700 1 -- 172.21.15.86:0/2165918880 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84780962d0 con 0x560c986635d0 2024-03-23T17:52:42.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.230+0000 7f8477000700 1 -- 172.21.15.86:0/2165918880 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84780a1fd0 con 0x560c986635d0 2024-03-23T17:52:42.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.231+0000 7f849fb64b80 1 -- 172.21.15.86:0/2165918880 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c986635d0 msgr2=0x560c9866c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:42.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.231+0000 7f849fb64b80 1 --2- 172.21.15.86:0/2165918880 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c986635d0 0x560c9866c110 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8478096490 tx=0x7f8478010a80 comp rx=0 tx=0).stop 2024-03-23T17:52:42.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.231+0000 7f849fb64b80 1 -- 172.21.15.86:0/2165918880 shutdown_connections 2024-03-23T17:52:42.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.231+0000 7f849fb64b80 1 --2- 172.21.15.86:0/2165918880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c9866edf0 0x560c986852d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.231+0000 7f849fb64b80 1 --2- 172.21.15.86:0/2165918880 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c986635d0 0x560c9866c110 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.231+0000 7f849fb64b80 1 --2- 172.21.15.86:0/2165918880 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c98663ef0 0x560c986809d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.231+0000 7f849fb64b80 1 -- 172.21.15.86:0/2165918880 >> 172.21.15.86:0/2165918880 conn(0x560c985c36a0 msgr2=0x560c986585a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:42.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.231+0000 7f849fb64b80 1 -- 172.21.15.86:0/2165918880 shutdown_connections 2024-03-23T17:52:42.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.231+0000 7f849fb64b80 1 -- 172.21.15.86:0/2165918880 wait complete. 2024-03-23T17:52:42.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.231+0000 7f849fb64b80 1 -- start start 2024-03-23T17:52:42.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.231+0000 7f849fb64b80 1 Processor -- start 2024-03-23T17:52:42.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.232+0000 7f849fb64b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c986635d0 0x560c986753f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:42.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.232+0000 7f849fb64b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c98663ef0 0x560c98675930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:42.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.232+0000 7f849fb64b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c9866edf0 0x560c986738d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:42.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.232+0000 7f849fb64b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c98673fb0 con 0x560c98663ef0 2024-03-23T17:52:42.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.232+0000 7f849fb64b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c98674120 con 0x560c9866edf0 2024-03-23T17:52:42.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.232+0000 7f849fb64b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c986742e0 con 0x560c986635d0 2024-03-23T17:52:42.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.232+0000 7f849fb64b80 10 client.0 _ll_register_callbacks cb 0x560c9855b470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:52:42.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.232+0000 7f848a000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c9866edf0 0x560c986738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:42.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.232+0000 7f8489600700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c986635d0 0x560c986753f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:42.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.232+0000 7f848a000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c9866edf0 0x560c986738d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:45950/0 (socket says 172.21.15.86:45950) 2024-03-23T17:52:42.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.232+0000 7f848a000700 1 -- 172.21.15.86:0/1498338716 learned_addr learned my addr 172.21.15.86:0/1498338716 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:42.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.232+0000 7f8488c00700 1 --2- 172.21.15.86:0/1498338716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c98663ef0 0x560c98675930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:42.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.233+0000 7f8489600700 1 -- 172.21.15.86:0/1498338716 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c9866edf0 msgr2=0x560c986738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:42.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.233+0000 7f8489600700 1 --2- 172.21.15.86:0/1498338716 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c9866edf0 0x560c986738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.233+0000 7f8489600700 1 -- 172.21.15.86:0/1498338716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c98663ef0 msgr2=0x560c98675930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:42.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.233+0000 7f8489600700 1 --2- 172.21.15.86:0/1498338716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c98663ef0 0x560c98675930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.233+0000 7f8489600700 1 -- 172.21.15.86:0/1498338716 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f848410e480 con 0x560c986635d0 2024-03-23T17:52:42.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.233+0000 7f8489600700 1 --2- 172.21.15.86:0/1498338716 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c986635d0 0x560c986753f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f847800cbd0 tx=0x7f84780ca5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53678586154186f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:42.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.233+0000 7f846aa00700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-03-23T17:52:42.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.233+0000 7f846aa00700 1 -- 172.21.15.86:0/1498338716 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8478049530 con 0x560c986635d0 2024-03-23T17:52:42.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.233+0000 7f846aa00700 1 -- 172.21.15.86:0/1498338716 <== mon.1 v2:172.21.15.152:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f847805fe40 con 0x560c986635d0 2024-03-23T17:52:42.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.233+0000 7f849fb64b80 1 -- 172.21.15.86:0/1498338716 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f847800c9c0 con 0x560c986635d0 2024-03-23T17:52:42.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.233+0000 7f846aa00700 1 -- 172.21.15.86:0/1498338716 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8478099480 con 0x560c986635d0 2024-03-23T17:52:42.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.233+0000 7f849fb64b80 10 client.26161 fetch_fsmap learned FSMap version 33 2024-03-23T17:52:42.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.233+0000 7f849fb64b80 1 -- 172.21.15.86:0/1498338716 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7f84841a6310 con 0x560c986635d0 2024-03-23T17:52:42.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.233+0000 7f846aa00700 1 -- 172.21.15.86:0/1498338716 <== mon.1 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f847800c9c0 con 0x560c986635d0 2024-03-23T17:52:42.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.234+0000 7f846aa00700 1 -- 172.21.15.86:0/1498338716 <== mon.1 v2:172.21.15.152:3300/0 5 ==== fsmap(e 33) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f8478049770 con 0x560c986635d0 2024-03-23T17:52:42.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.234+0000 7f849fb64b80 10 client.26161 fetch_fsmap finished waiting for FSMap version 33 2024-03-23T17:52:42.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.234+0000 7f849fb64b80 10 client.26161 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-23T17:52:42.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.234+0000 7f849fb64b80 20 client.26161 populate_metadata read hostname 'smithi086' 2024-03-23T17:52:42.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.234+0000 7f849fb64b80 1 --2- 172.21.15.86:0/1498338716 >> [v2:172.21.15.92:6836/3967837948,v1:172.21.15.92:6837/3967837948] conn(0x560c986d0e00 0x560c986d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:42.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.234+0000 7f849fb64b80 4 client.26161 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-23T17:52:42.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.234+0000 7f849fb64b80 1 -- 172.21.15.86:0/1498338716 --> [v2:172.21.15.92:6836/3967837948,v1:172.21.15.92:6837/3967837948] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560c98404cf0 con 0x560c986d0e00 2024-03-23T17:52:42.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.234+0000 7f8488c00700 1 --2- 172.21.15.86:0/1498338716 >> [v2:172.21.15.92:6836/3967837948,v1:172.21.15.92:6837/3967837948] conn(0x560c986d0e00 0x560c986d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:42.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.235+0000 7f8488c00700 1 --2- 172.21.15.86:0/1498338716 >> [v2:172.21.15.92:6836/3967837948,v1:172.21.15.92:6837/3967837948] conn(0x560c986d0e00 0x560c986d31f0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bd4eb82000fc689f server_cookie=6c99696d654d39cb in_seq=0 out_seq=0 2024-03-23T17:52:42.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.235+0000 7f846aa00700 10 client.26161 ms_handle_connect on v2:172.21.15.92:6836/3967837948 2024-03-23T17:52:42.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.235+0000 7f846aa00700 1 -- 172.21.15.86:0/1498338716 <== mds.2 v2:172.21.15.92:6836/3967837948 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+1854 (crc 0 0 0) 0x560c98404cf0 con 0x560c986d0e00 2024-03-23T17:52:42.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.235+0000 7f846aa00700 10 client.26161 handle_command_reply: tid=0 2024-03-23T17:52:42.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.235+0000 7f849fb64b80 1 client.26161 shutdown 2024-03-23T17:52:42.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.235+0000 7f849fb64b80 1 -- 172.21.15.86:0/1498338716 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c986635d0 msgr2=0x560c986753f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:42.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.235+0000 7f849fb64b80 1 --2- 172.21.15.86:0/1498338716 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c986635d0 0x560c986753f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f847800cbd0 tx=0x7f84780ca5f0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.235+0000 7f849fb64b80 1 -- 172.21.15.86:0/1498338716 shutdown_connections 2024-03-23T17:52:42.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.235+0000 7f849fb64b80 1 --2- 172.21.15.86:0/1498338716 >> [v2:172.21.15.92:6836/3967837948,v1:172.21.15.92:6837/3967837948] conn(0x560c986d0e00 0x560c986d31f0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.235+0000 7f846aa00700 0 client.26161 ms_handle_reset on v2:172.21.15.92:6836/3967837948 2024-03-23T17:52:42.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.235+0000 7f849fb64b80 1 --2- 172.21.15.86:0/1498338716 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c9866edf0 0x560c986738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.235+0000 7f849fb64b80 1 --2- 172.21.15.86:0/1498338716 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c986635d0 0x560c986753f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.235+0000 7f849fb64b80 1 --2- 172.21.15.86:0/1498338716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c98663ef0 0x560c98675930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.235+0000 7f849fb64b80 1 -- 172.21.15.86:0/1498338716 >> 172.21.15.86:0/1498338716 conn(0x560c985c36a0 msgr2=0x560c98658f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:42.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.235+0000 7f849fb64b80 1 -- 172.21.15.86:0/1498338716 shutdown_connections 2024-03-23T17:52:42.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.235+0000 7f849fb64b80 1 -- 172.21.15.86:0/1498338716 wait complete. 2024-03-23T17:52:42.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.235+0000 7f849fb64b80 20 client.26161 trim_cache size 0 max 16384 2024-03-23T17:52:42.241 INFO:teuthology.orchestra.run.smithi086.stdout:[ 2024-03-23T17:52:42.241 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T17:52:42.241 INFO:teuthology.orchestra.run.smithi086.stdout: "damage_type": "backtrace", 2024-03-23T17:52:42.241 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 41960410, 2024-03-23T17:52:42.241 INFO:teuthology.orchestra.run.smithi086.stdout: "ino": 1099511628324, 2024-03-23T17:52:42.241 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/WORDPRO/NEWS1_1.LWP" 2024-03-23T17:52:42.241 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T17:52:42.242 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T17:52:42.242 INFO:teuthology.orchestra.run.smithi086.stdout: "damage_type": "backtrace", 2024-03-23T17:52:42.242 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 967553592, 2024-03-23T17:52:42.242 INFO:teuthology.orchestra.run.smithi086.stdout: "ino": 1099511628304, 2024-03-23T17:52:42.242 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/PWRPNT/PCBENCHM.PPT" 2024-03-23T17:52:42.242 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T17:52:42.242 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T17:52:42.242 INFO:teuthology.orchestra.run.smithi086.stdout: "damage_type": "backtrace", 2024-03-23T17:52:42.242 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 1759052993, 2024-03-23T17:52:42.242 INFO:teuthology.orchestra.run.smithi086.stdout: "ino": 1099511628308, 2024-03-23T17:52:42.242 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/PWRPNT/TRIDOTS.POT" 2024-03-23T17:52:42.242 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T17:52:42.243 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T17:52:42.243 INFO:teuthology.orchestra.run.smithi086.stdout: "damage_type": "backtrace", 2024-03-23T17:52:42.243 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 2303084638, 2024-03-23T17:52:42.243 INFO:teuthology.orchestra.run.smithi086.stdout: "ino": 1099511628305, 2024-03-23T17:52:42.243 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/PWRPNT/ZD16.BMP" 2024-03-23T17:52:42.243 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T17:52:42.243 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T17:52:42.243 INFO:teuthology.orchestra.run.smithi086.stdout: "damage_type": "backtrace", 2024-03-23T17:52:42.243 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 3292507529, 2024-03-23T17:52:42.243 INFO:teuthology.orchestra.run.smithi086.stdout: "ino": 1099511628306, 2024-03-23T17:52:42.243 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/PWRPNT/PPTOOLS1.PPA" 2024-03-23T17:52:42.244 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T17:52:42.244 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T17:52:42.244 INFO:teuthology.orchestra.run.smithi086.stdout: "damage_type": "backtrace", 2024-03-23T17:52:42.244 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 3548755771, 2024-03-23T17:52:42.244 INFO:teuthology.orchestra.run.smithi086.stdout: "ino": 1099511628323, 2024-03-23T17:52:42.244 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/WORDPRO/RESULTS.XLS" 2024-03-23T17:52:42.244 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T17:52:42.244 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T17:52:42.244 INFO:teuthology.orchestra.run.smithi086.stdout: "damage_type": "backtrace", 2024-03-23T17:52:42.244 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 3644599087, 2024-03-23T17:52:42.244 INFO:teuthology.orchestra.run.smithi086.stdout: "ino": 1099511628322, 2024-03-23T17:52:42.245 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/WORDPRO/BENCHS.LWP" 2024-03-23T17:52:42.245 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T17:52:42.245 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T17:52:42.245 INFO:teuthology.orchestra.run.smithi086.stdout: "damage_type": "backtrace", 2024-03-23T17:52:42.245 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 4223820534, 2024-03-23T17:52:42.245 INFO:teuthology.orchestra.run.smithi086.stdout: "ino": 1099511628307, 2024-03-23T17:52:42.245 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/PWRPNT/TIPS.PPT" 2024-03-23T17:52:42.245 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T17:52:42.245 INFO:teuthology.orchestra.run.smithi086.stdout:] 2024-03-23T17:52:42.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.236+0000 7f848c000700 1 -- 172.21.15.86:0/2199470427 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f846c0a1290 msgr2=0x7f846c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:42.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.236+0000 7f848c000700 1 --2- 172.21.15.86:0/2199470427 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f846c0a1290 0x7f846c0a3750 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f847c009550 tx=0x7f847c009920 comp rx=0 tx=0).stop 2024-03-23T17:52:42.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.236+0000 7f848c000700 1 -- 172.21.15.86:0/2199470427 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8484100f70 msgr2=0x7f8484199480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:42.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.236+0000 7f848c000700 1 --2- 172.21.15.86:0/2199470427 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8484100f70 0x7f8484199480 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f847800e9a0 tx=0x7f847800ecb0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.236+0000 7f848c000700 1 -- 172.21.15.86:0/2199470427 shutdown_connections 2024-03-23T17:52:42.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.236+0000 7f848c000700 1 --2- 172.21.15.86:0/2199470427 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f846c0a1290 0x7f846c0a3750 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.236+0000 7f848c000700 1 --2- 172.21.15.86:0/2199470427 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f848419dd10 0x7f848419e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.236+0000 7f848c000700 1 --2- 172.21.15.86:0/2199470427 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8484100f70 0x7f8484199480 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.236+0000 7f848c000700 1 --2- 172.21.15.86:0/2199470427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8484106090 0x7f84841999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.236+0000 7f848c000700 1 -- 172.21.15.86:0/2199470427 >> 172.21.15.86:0/2199470427 conn(0x7f84840fc7f0 msgr2=0x7f84840feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:42.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.236+0000 7f848c000700 1 -- 172.21.15.86:0/2199470427 shutdown_connections 2024-03-23T17:52:42.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.236+0000 7f848c000700 1 -- 172.21.15.86:0/2199470427 wait complete. 2024-03-23T17:52:42.255 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-23T17:52:42.255 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-23T17:52:42.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:41 smithi092 ceph-mon[79089]: pgmap v64: 97 pgs: 97 active+clean; 34 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 569 KiB/s wr, 110 op/s 2024-03-23T17:52:42.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:41 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/435626673' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T17:52:42.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:41 smithi152 ceph-mon[47949]: pgmap v64: 97 pgs: 97 active+clean; 34 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 569 KiB/s wr, 110 op/s 2024-03-23T17:52:42.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:41 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/435626673' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T17:52:42.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.471+0000 7faf64a00700 1 -- 172.21.15.86:0/3387411484 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf60100f70 msgr2=0x7faf60101350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:42.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.471+0000 7faf64a00700 1 --2- 172.21.15.86:0/3387411484 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf60100f70 0x7faf60101350 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7faf50009990 tx=0x7faf50009ca0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.472+0000 7faf64a00700 1 -- 172.21.15.86:0/3387411484 shutdown_connections 2024-03-23T17:52:42.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.472+0000 7faf64a00700 1 --2- 172.21.15.86:0/3387411484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf60106090 0x7faf6010dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.472+0000 7faf64a00700 1 --2- 172.21.15.86:0/3387411484 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf60100f70 0x7faf60101350 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.472+0000 7faf64a00700 1 --2- 172.21.15.86:0/3387411484 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf60101920 0x7faf60105960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.472+0000 7faf64a00700 1 -- 172.21.15.86:0/3387411484 >> 172.21.15.86:0/3387411484 conn(0x7faf600fc7f0 msgr2=0x7faf600fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:42.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.472+0000 7faf64a00700 1 -- 172.21.15.86:0/3387411484 shutdown_connections 2024-03-23T17:52:42.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.472+0000 7faf64a00700 1 -- 172.21.15.86:0/3387411484 wait complete. 2024-03-23T17:52:42.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.473+0000 7faf64a00700 1 Processor -- start 2024-03-23T17:52:42.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.473+0000 7faf64a00700 1 -- start start 2024-03-23T17:52:42.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.473+0000 7faf64a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf60100f70 0x7faf601994c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:42.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.473+0000 7faf64a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf60106090 0x7faf60199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:42.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.473+0000 7faf64a00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf6019dd50 0x7faf6019e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:42.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.473+0000 7faf64a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf6019e7d0 con 0x7faf60100f70 2024-03-23T17:52:42.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.473+0000 7faf64a00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf6019e940 con 0x7faf60106090 2024-03-23T17:52:42.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.473+0000 7faf64a00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf6019eb00 con 0x7faf6019dd50 2024-03-23T17:52:42.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.473+0000 7faf5de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf60100f70 0x7faf601994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:42.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.473+0000 7faf5d400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf60106090 0x7faf60199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:42.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.473+0000 7faf5e800700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf6019dd50 0x7faf6019e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:42.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.473+0000 7faf5e800700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf6019dd50 0x7faf6019e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:52586/0 (socket says 172.21.15.86:52586) 2024-03-23T17:52:42.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.473+0000 7faf5de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf60100f70 0x7faf601994c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35008/0 (socket says 172.21.15.86:35008) 2024-03-23T17:52:42.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.473+0000 7faf5e800700 1 -- 172.21.15.86:0/470132327 learned_addr learned my addr 172.21.15.86:0/470132327 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:42.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.474+0000 7faf5d400700 1 -- 172.21.15.86:0/470132327 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf6019dd50 msgr2=0x7faf6019e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:42.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.474+0000 7faf5d400700 1 --2- 172.21.15.86:0/470132327 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf6019dd50 0x7faf6019e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.474+0000 7faf5d400700 1 -- 172.21.15.86:0/470132327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf60100f70 msgr2=0x7faf601994c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:42.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.474+0000 7faf5d400700 1 --2- 172.21.15.86:0/470132327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf60100f70 0x7faf601994c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.474+0000 7faf5d400700 1 -- 172.21.15.86:0/470132327 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf500095f0 con 0x7faf60106090 2024-03-23T17:52:42.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.474+0000 7faf5d400700 1 --2- 172.21.15.86:0/470132327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf60106090 0x7faf60199a00 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7faf4c00e9c0 tx=0x7faf4c00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:42.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.474+0000 7faf56a00700 1 -- 172.21.15.86:0/470132327 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf4c00c9c0 con 0x7faf60106090 2024-03-23T17:52:42.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.474+0000 7faf64a00700 1 -- 172.21.15.86:0/470132327 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf601a6060 con 0x7faf60106090 2024-03-23T17:52:42.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.474+0000 7faf56a00700 1 -- 172.21.15.86:0/470132327 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf4c010430 con 0x7faf60106090 2024-03-23T17:52:42.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.474+0000 7faf56a00700 1 -- 172.21.15.86:0/470132327 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf4c018680 con 0x7faf60106090 2024-03-23T17:52:42.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.474+0000 7faf64a00700 1 -- 172.21.15.86:0/470132327 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf601a6590 con 0x7faf60106090 2024-03-23T17:52:42.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.478+0000 7faf56a00700 1 -- 172.21.15.86:0/470132327 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7faf4c00cb20 con 0x7faf60106090 2024-03-23T17:52:42.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.479+0000 7faf78560b80 1 Processor -- start 2024-03-23T17:52:42.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.479+0000 7faf78560b80 1 -- start start 2024-03-23T17:52:42.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.479+0000 7faf56a00700 1 --2- 172.21.15.86:0/470132327 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7faf440a1290 0x7faf440a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:42.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.479+0000 7faf78560b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad841025d0 0x55ad841029b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:42.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.480+0000 7faf78560b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ad84102ef0 0x55ad8410d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:42.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.480+0000 7faf56a00700 1 -- 172.21.15.86:0/470132327 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7faf4c014070 con 0x7faf60106090 2024-03-23T17:52:42.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.480+0000 7faf78560b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ad8410ddf0 0x55ad841101e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:42.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.480+0000 7faf78560b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad8405b990 con 0x55ad841025d0 2024-03-23T17:52:42.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.480+0000 7faf78560b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad84059350 con 0x55ad8410ddf0 2024-03-23T17:52:42.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.480+0000 7faf78560b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad84054c40 con 0x55ad84102ef0 2024-03-23T17:52:42.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.480+0000 7faf5e800700 1 --2- 172.21.15.86:0/470132327 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7faf440a1290 0x7faf440a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:42.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.480+0000 7faf5d400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ad84102ef0 0x55ad8410d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:42.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.480+0000 7faf5de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad841025d0 0x55ad841029b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:42.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.480+0000 7faf5d400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ad84102ef0 0x55ad8410d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:52596/0 (socket says 172.21.15.86:52596) 2024-03-23T17:52:42.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.480+0000 7faf5d400700 1 -- 172.21.15.86:0/17531799 learned_addr learned my addr 172.21.15.86:0/17531799 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:42.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.480+0000 7faf5d400700 1 -- 172.21.15.86:0/17531799 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ad8410ddf0 msgr2=0x55ad841101e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-23T17:52:42.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.480+0000 7faf5d400700 1 --2- 172.21.15.86:0/17531799 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ad8410ddf0 0x55ad841101e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.480+0000 7faf5d400700 1 -- 172.21.15.86:0/17531799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad841025d0 msgr2=0x55ad841029b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:42.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.480+0000 7faf5d400700 1 --2- 172.21.15.86:0/17531799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad841025d0 0x55ad841029b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.480+0000 7faf5d400700 1 -- 172.21.15.86:0/17531799 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ad83ed63b0 con 0x55ad84102ef0 2024-03-23T17:52:42.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.480+0000 7faf5de00700 1 --2- 172.21.15.86:0/17531799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad841025d0 0x55ad841029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:52:42.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.481+0000 7faf5d400700 1 --2- 172.21.15.86:0/17531799 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ad84102ef0 0x55ad8410d8b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7faf4c0a1e50 tx=0x7faf4c099de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76a9d2a833e4e02c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:42.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.481+0000 7faf5e800700 1 --2- 172.21.15.86:0/470132327 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7faf440a1290 0x7faf440a3750 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7faf58009c10 tx=0x7faf5800b040 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:42.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.481+0000 7faf4b000700 1 -- 172.21.15.86:0/17531799 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf4c099f00 con 0x55ad84102ef0 2024-03-23T17:52:42.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.481+0000 7faf4b000700 1 -- 172.21.15.86:0/17531799 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf4c09dd30 con 0x55ad84102ef0 2024-03-23T17:52:42.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.481+0000 7faf4b000700 1 -- 172.21.15.86:0/17531799 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf4c09a160 con 0x55ad84102ef0 2024-03-23T17:52:42.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.481+0000 7faf78560b80 1 -- 172.21.15.86:0/17531799 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ad84102ef0 msgr2=0x55ad8410d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:42.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.481+0000 7faf78560b80 1 --2- 172.21.15.86:0/17531799 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ad84102ef0 0x55ad8410d8b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7faf4c0a1e50 tx=0x7faf4c099de0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.482+0000 7faf78560b80 1 -- 172.21.15.86:0/17531799 shutdown_connections 2024-03-23T17:52:42.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.482+0000 7faf78560b80 1 --2- 172.21.15.86:0/17531799 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ad8410ddf0 0x55ad841101e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.482+0000 7faf78560b80 1 --2- 172.21.15.86:0/17531799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad841025d0 0x55ad841029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.482+0000 7faf78560b80 1 --2- 172.21.15.86:0/17531799 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ad84102ef0 0x55ad8410d8b0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.482+0000 7faf78560b80 1 -- 172.21.15.86:0/17531799 >> 172.21.15.86:0/17531799 conn(0x55ad840626a0 msgr2=0x55ad840fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:42.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.482+0000 7faf78560b80 1 -- 172.21.15.86:0/17531799 shutdown_connections 2024-03-23T17:52:42.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.482+0000 7faf78560b80 1 -- 172.21.15.86:0/17531799 wait complete. 2024-03-23T17:52:42.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.482+0000 7faf78560b80 1 -- start start 2024-03-23T17:52:42.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.482+0000 7faf78560b80 1 Processor -- start 2024-03-23T17:52:42.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.483+0000 7faf78560b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ad841025d0 0x55ad8410ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:42.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.483+0000 7faf78560b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ad84102ef0 0x55ad8410b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:42.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.483+0000 7faf78560b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad8410ddf0 0x55ad8410b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:42.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.483+0000 7faf78560b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad83ff0d80 con 0x55ad8410ddf0 2024-03-23T17:52:42.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.483+0000 7faf78560b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad83ff0a20 con 0x55ad84102ef0 2024-03-23T17:52:42.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.483+0000 7faf78560b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad83ed86a0 con 0x55ad841025d0 2024-03-23T17:52:42.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.483+0000 7faf78560b80 10 client.0 _ll_register_callbacks cb 0x55ad83ffa470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:52:42.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.483+0000 7faf5de00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ad841025d0 0x55ad8410ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:42.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.483+0000 7faf5d400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ad84102ef0 0x55ad8410b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:42.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.483+0000 7faf5de00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ad841025d0 0x55ad8410ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:52600/0 (socket says 172.21.15.86:52600) 2024-03-23T17:52:42.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.483+0000 7faf5de00700 1 -- 172.21.15.86:0/2358842486 learned_addr learned my addr 172.21.15.86:0/2358842486 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:42.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.483+0000 7faf5d400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ad84102ef0 0x55ad8410b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:45990/0 (socket says 172.21.15.86:45990) 2024-03-23T17:52:42.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.483+0000 7faf5e800700 1 --2- 172.21.15.86:0/2358842486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad8410ddf0 0x55ad8410b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:42.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.483+0000 7faf5d400700 1 -- 172.21.15.86:0/2358842486 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ad841025d0 msgr2=0x55ad8410ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:42.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.483+0000 7faf5d400700 1 --2- 172.21.15.86:0/2358842486 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ad841025d0 0x55ad8410ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.483+0000 7faf5d400700 1 -- 172.21.15.86:0/2358842486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad8410ddf0 msgr2=0x55ad8410b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:42.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.483+0000 7faf5d400700 1 --2- 172.21.15.86:0/2358842486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad8410ddf0 0x55ad8410b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.483+0000 7faf5d400700 1 -- 172.21.15.86:0/2358842486 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf6010e480 con 0x55ad84102ef0 2024-03-23T17:52:42.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.483+0000 7faf5de00700 1 --2- 172.21.15.86:0/2358842486 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ad841025d0 0x55ad8410ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:52:42.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.484+0000 7faf5d400700 1 --2- 172.21.15.86:0/2358842486 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ad84102ef0 0x55ad8410b080 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7faf4c096970 tx=0x7faf4c0ca530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=900fb471ade29c9f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:42.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.484+0000 7faf3f400700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T17:52:42.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.484+0000 7faf3f400700 1 -- 172.21.15.86:0/2358842486 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf4c0b82c0 con 0x55ad84102ef0 2024-03-23T17:52:42.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.484+0000 7faf3f400700 1 -- 172.21.15.86:0/2358842486 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7faf4c0105a0 con 0x55ad84102ef0 2024-03-23T17:52:42.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.484+0000 7faf78560b80 1 -- 172.21.15.86:0/2358842486 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ad84015380 con 0x55ad84102ef0 2024-03-23T17:52:42.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.484+0000 7faf3f400700 1 -- 172.21.15.86:0/2358842486 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf4c09a160 con 0x55ad84102ef0 2024-03-23T17:52:42.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.484+0000 7faf78560b80 10 client.26180 fetch_fsmap learned FSMap version 33 2024-03-23T17:52:42.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.484+0000 7faf78560b80 1 -- 172.21.15.86:0/2358842486 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7faf601a6590 con 0x55ad84102ef0 2024-03-23T17:52:42.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.484+0000 7faf3f400700 1 -- 172.21.15.86:0/2358842486 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ad84015380 con 0x55ad84102ef0 2024-03-23T17:52:42.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.484+0000 7faf3f400700 1 -- 172.21.15.86:0/2358842486 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 33) v1 ==== 7478+0+0 (secure 0 0 0) 0x7faf4c095cf0 con 0x55ad84102ef0 2024-03-23T17:52:42.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.485+0000 7faf78560b80 10 client.26180 fetch_fsmap finished waiting for FSMap version 33 2024-03-23T17:52:42.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.485+0000 7faf78560b80 10 client.26180 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-23T17:52:42.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.485+0000 7faf78560b80 20 client.26180 populate_metadata read hostname 'smithi086' 2024-03-23T17:52:42.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.485+0000 7faf78560b80 1 --2- 172.21.15.86:0/2358842486 >> [v2:172.21.15.152:6838/1100723992,v1:172.21.15.152:6839/1100723992] conn(0x55ad8411d660 0x55ad8411fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:42.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.485+0000 7faf78560b80 4 client.26180 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:52:42.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.485+0000 7faf78560b80 1 -- 172.21.15.86:0/2358842486 --> [v2:172.21.15.152:6838/1100723992,v1:172.21.15.152:6839/1100723992] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ad8404d700 con 0x55ad8411d660 2024-03-23T17:52:42.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.485+0000 7faf5de00700 1 --2- 172.21.15.86:0/2358842486 >> [v2:172.21.15.152:6838/1100723992,v1:172.21.15.152:6839/1100723992] conn(0x55ad8411d660 0x55ad8411fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:42.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.485+0000 7faf3f400700 10 client.26180 ms_handle_connect on v2:172.21.15.152:6838/1100723992 2024-03-23T17:52:42.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.485+0000 7faf5de00700 1 --2- 172.21.15.86:0/2358842486 >> [v2:172.21.15.152:6838/1100723992,v1:172.21.15.152:6839/1100723992] conn(0x55ad8411d660 0x55ad8411fa80 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=aace96f296f875fe server_cookie=57879c2032192347 in_seq=0 out_seq=0 2024-03-23T17:52:42.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.486+0000 7faf3f400700 1 -- 172.21.15.86:0/2358842486 <== mds.1 v2:172.21.15.152:6838/1100723992 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55ad8404d700 con 0x55ad8411d660 2024-03-23T17:52:42.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.486+0000 7faf3f400700 10 client.26180 handle_command_reply: tid=0 2024-03-23T17:52:42.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.486+0000 7faf78560b80 1 client.26180 shutdown 2024-03-23T17:52:42.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.486+0000 7faf78560b80 1 -- 172.21.15.86:0/2358842486 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ad84102ef0 msgr2=0x55ad8410b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:42.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.486+0000 7faf78560b80 1 --2- 172.21.15.86:0/2358842486 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ad84102ef0 0x55ad8410b080 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7faf4c096970 tx=0x7faf4c0ca530 comp rx=0 tx=0).stop 2024-03-23T17:52:42.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.486+0000 7faf78560b80 1 -- 172.21.15.86:0/2358842486 shutdown_connections 2024-03-23T17:52:42.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.486+0000 7faf78560b80 1 --2- 172.21.15.86:0/2358842486 >> [v2:172.21.15.152:6838/1100723992,v1:172.21.15.152:6839/1100723992] conn(0x55ad8411d660 0x55ad8411fa80 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.486+0000 7faf78560b80 1 --2- 172.21.15.86:0/2358842486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad8410ddf0 0x55ad8410b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.486+0000 7faf78560b80 1 --2- 172.21.15.86:0/2358842486 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ad841025d0 0x55ad8410ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.486+0000 7faf78560b80 1 --2- 172.21.15.86:0/2358842486 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ad84102ef0 0x55ad8410b080 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.486+0000 7faf3f400700 0 client.26180 ms_handle_reset on v2:172.21.15.152:6838/1100723992 2024-03-23T17:52:42.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.486+0000 7faf78560b80 1 -- 172.21.15.86:0/2358842486 >> 172.21.15.86:0/2358842486 conn(0x55ad840626a0 msgr2=0x55ad841046c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:42.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.486+0000 7faf78560b80 1 -- 172.21.15.86:0/2358842486 shutdown_connections 2024-03-23T17:52:42.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.487+0000 7faf78560b80 1 -- 172.21.15.86:0/2358842486 wait complete. 2024-03-23T17:52:42.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.487+0000 7faf78560b80 20 client.26180 trim_cache size 0 max 16384 2024-03-23T17:52:42.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.500+0000 7faf78560b80 1 Processor -- start 2024-03-23T17:52:42.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.500+0000 7faf78560b80 1 -- start start 2024-03-23T17:52:42.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.501+0000 7faf78560b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ad841025d0 0x55ad84114a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:42.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.501+0000 7faf78560b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ad84102ef0 0x55ad841055b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:42.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.501+0000 7faf78560b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad8410ddf0 0x55ad8410b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:42.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.501+0000 7faf78560b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad84124760 con 0x55ad8410ddf0 2024-03-23T17:52:42.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.501+0000 7faf78560b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad841200e0 con 0x55ad841025d0 2024-03-23T17:52:42.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.501+0000 7faf78560b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad841090f0 con 0x55ad84102ef0 2024-03-23T17:52:42.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.501+0000 7faf5de00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ad841025d0 0x55ad84114a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:42.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.501+0000 7faf5de00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ad841025d0 0x55ad84114a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:45996/0 (socket says 172.21.15.86:45996) 2024-03-23T17:52:42.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.501+0000 7faf5de00700 1 -- 172.21.15.86:0/1484209664 learned_addr learned my addr 172.21.15.86:0/1484209664 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:42.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.501+0000 7faf5d400700 1 --2- 172.21.15.86:0/1484209664 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ad84102ef0 0x55ad841055b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:42.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.501+0000 7faf5e800700 1 --2- 172.21.15.86:0/1484209664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad8410ddf0 0x55ad8410b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:42.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.501+0000 7faf5de00700 1 -- 172.21.15.86:0/1484209664 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ad84102ef0 msgr2=0x55ad841055b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:42.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.501+0000 7faf5de00700 1 --2- 172.21.15.86:0/1484209664 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ad84102ef0 0x55ad841055b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.501+0000 7faf5de00700 1 -- 172.21.15.86:0/1484209664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad8410ddf0 msgr2=0x55ad8410b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:42.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.501+0000 7faf5de00700 1 --2- 172.21.15.86:0/1484209664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad8410ddf0 0x55ad8410b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.501+0000 7faf5de00700 1 -- 172.21.15.86:0/1484209664 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ad83e82b60 con 0x55ad841025d0 2024-03-23T17:52:42.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.502+0000 7faf5de00700 1 --2- 172.21.15.86:0/1484209664 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ad841025d0 0x55ad84114a10 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7faf500098e0 tx=0x7faf50009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83213037e346a897 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:42.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.502+0000 7faf4ba00700 1 -- 172.21.15.86:0/1484209664 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf5001d070 con 0x55ad841025d0 2024-03-23T17:52:42.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.502+0000 7faf4ba00700 1 -- 172.21.15.86:0/1484209664 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf5001e040 con 0x55ad841025d0 2024-03-23T17:52:42.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.502+0000 7faf4ba00700 1 -- 172.21.15.86:0/1484209664 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf5000f460 con 0x55ad841025d0 2024-03-23T17:52:42.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.503+0000 7faf78560b80 1 -- 172.21.15.86:0/1484209664 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ad841025d0 msgr2=0x55ad84114a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:42.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.503+0000 7faf78560b80 1 --2- 172.21.15.86:0/1484209664 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ad841025d0 0x55ad84114a10 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7faf500098e0 tx=0x7faf50009310 comp rx=0 tx=0).stop 2024-03-23T17:52:42.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.503+0000 7faf78560b80 1 -- 172.21.15.86:0/1484209664 shutdown_connections 2024-03-23T17:52:42.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.503+0000 7faf78560b80 1 --2- 172.21.15.86:0/1484209664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad8410ddf0 0x55ad8410b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.503+0000 7faf78560b80 1 --2- 172.21.15.86:0/1484209664 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ad841025d0 0x55ad84114a10 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.503+0000 7faf78560b80 1 --2- 172.21.15.86:0/1484209664 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ad84102ef0 0x55ad841055b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.503+0000 7faf78560b80 1 -- 172.21.15.86:0/1484209664 >> 172.21.15.86:0/1484209664 conn(0x55ad840626a0 msgr2=0x55ad840f75a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:42.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.503+0000 7faf78560b80 1 -- 172.21.15.86:0/1484209664 shutdown_connections 2024-03-23T17:52:42.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.503+0000 7faf78560b80 1 -- 172.21.15.86:0/1484209664 wait complete. 2024-03-23T17:52:42.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.503+0000 7faf78560b80 1 -- start start 2024-03-23T17:52:42.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.503+0000 7faf78560b80 1 Processor -- start 2024-03-23T17:52:42.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.504+0000 7faf78560b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ad841025d0 0x55ad84114530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:42.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.504+0000 7faf78560b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad84102ef0 0x55ad84112a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:42.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.504+0000 7faf78560b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ad8410ddf0 0x55ad84112f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:42.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.504+0000 7faf78560b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad84113490 con 0x55ad84102ef0 2024-03-23T17:52:42.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.504+0000 7faf78560b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad84113600 con 0x55ad841025d0 2024-03-23T17:52:42.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.504+0000 7faf78560b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad841137c0 con 0x55ad8410ddf0 2024-03-23T17:52:42.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.504+0000 7faf78560b80 10 client.0 _ll_register_callbacks cb 0x55ad83ffa470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:52:42.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.504+0000 7faf5de00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ad841025d0 0x55ad84114530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:42.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.504+0000 7faf5de00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ad841025d0 0x55ad84114530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:46000/0 (socket says 172.21.15.86:46000) 2024-03-23T17:52:42.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.504+0000 7faf5de00700 1 -- 172.21.15.86:0/2228276664 learned_addr learned my addr 172.21.15.86:0/2228276664 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:42.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.504+0000 7faf5e800700 1 --2- 172.21.15.86:0/2228276664 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ad8410ddf0 0x55ad84112f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:42.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.504+0000 7faf5d400700 1 --2- 172.21.15.86:0/2228276664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad84102ef0 0x55ad84112a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:42.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.504+0000 7faf5de00700 1 -- 172.21.15.86:0/2228276664 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ad8410ddf0 msgr2=0x55ad84112f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:42.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.504+0000 7faf5de00700 1 --2- 172.21.15.86:0/2228276664 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ad8410ddf0 0x55ad84112f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.504+0000 7faf5de00700 1 -- 172.21.15.86:0/2228276664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad84102ef0 msgr2=0x55ad84112a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:42.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.504+0000 7faf5de00700 1 --2- 172.21.15.86:0/2228276664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad84102ef0 0x55ad84112a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.504+0000 7faf5de00700 1 -- 172.21.15.86:0/2228276664 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf500055d0 con 0x55ad841025d0 2024-03-23T17:52:42.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.505+0000 7faf5de00700 1 --2- 172.21.15.86:0/2228276664 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ad841025d0 0x55ad84114530 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7faf50005850 tx=0x7faf50005350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=956bdd8a638e4e59 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:42.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.505+0000 7faf3f400700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T17:52:42.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.505+0000 7faf3f400700 1 -- 172.21.15.86:0/2228276664 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf5001d040 con 0x55ad841025d0 2024-03-23T17:52:42.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.505+0000 7faf78560b80 1 -- 172.21.15.86:0/2228276664 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faf4c09dd30 con 0x55ad841025d0 2024-03-23T17:52:42.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.505+0000 7faf3f400700 1 -- 172.21.15.86:0/2228276664 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7faf50004d70 con 0x55ad841025d0 2024-03-23T17:52:42.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.505+0000 7faf3f400700 1 -- 172.21.15.86:0/2228276664 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf50004500 con 0x55ad841025d0 2024-03-23T17:52:42.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.505+0000 7faf78560b80 10 client.26186 fetch_fsmap learned FSMap version 33 2024-03-23T17:52:42.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.505+0000 7faf78560b80 1 -- 172.21.15.86:0/2228276664 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55ad8401aea0 con 0x55ad841025d0 2024-03-23T17:52:42.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.505+0000 7faf3f400700 1 -- 172.21.15.86:0/2228276664 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf50041080 con 0x55ad841025d0 2024-03-23T17:52:42.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.506+0000 7faf3f400700 1 -- 172.21.15.86:0/2228276664 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 33) v1 ==== 7478+0+0 (secure 0 0 0) 0x7faf5000fc20 con 0x55ad841025d0 2024-03-23T17:52:42.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.506+0000 7faf78560b80 10 client.26186 fetch_fsmap finished waiting for FSMap version 33 2024-03-23T17:52:42.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.506+0000 7faf78560b80 10 client.26186 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-23T17:52:42.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.506+0000 7faf78560b80 20 client.26186 populate_metadata read hostname 'smithi086' 2024-03-23T17:52:42.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.506+0000 7faf78560b80 1 --2- 172.21.15.86:0/2228276664 >> [v2:172.21.15.152:6838/1100723992,v1:172.21.15.152:6839/1100723992] conn(0x55ad841700a0 0x55ad84172490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:42.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.506+0000 7faf78560b80 4 client.26186 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-23T17:52:42.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.506+0000 7faf78560b80 1 -- 172.21.15.86:0/2228276664 --> [v2:172.21.15.152:6838/1100723992,v1:172.21.15.152:6839/1100723992] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ad83ea3cf0 con 0x55ad841700a0 2024-03-23T17:52:42.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.506+0000 7faf5d400700 1 --2- 172.21.15.86:0/2228276664 >> [v2:172.21.15.152:6838/1100723992,v1:172.21.15.152:6839/1100723992] conn(0x55ad841700a0 0x55ad84172490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:42.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.506+0000 7faf5d400700 1 --2- 172.21.15.86:0/2228276664 >> [v2:172.21.15.152:6838/1100723992,v1:172.21.15.152:6839/1100723992] conn(0x55ad841700a0 0x55ad84172490 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5128f5dd6ba8c460 server_cookie=5da75669b3989f4e in_seq=0 out_seq=0 2024-03-23T17:52:42.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.506+0000 7faf3f400700 10 client.26186 ms_handle_connect on v2:172.21.15.152:6838/1100723992 2024-03-23T17:52:42.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.507+0000 7faf3f400700 1 -- 172.21.15.86:0/2228276664 <== mds.1 v2:172.21.15.152:6838/1100723992 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ad83ea3cf0 con 0x55ad841700a0 2024-03-23T17:52:42.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.507+0000 7faf3f400700 10 client.26186 handle_command_reply: tid=0 2024-03-23T17:52:42.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.507+0000 7faf78560b80 1 client.26186 shutdown 2024-03-23T17:52:42.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.507+0000 7faf78560b80 1 -- 172.21.15.86:0/2228276664 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ad841025d0 msgr2=0x55ad84114530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:42.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.507+0000 7faf78560b80 1 --2- 172.21.15.86:0/2228276664 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ad841025d0 0x55ad84114530 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7faf50005850 tx=0x7faf50005350 comp rx=0 tx=0).stop 2024-03-23T17:52:42.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.507+0000 7faf78560b80 1 -- 172.21.15.86:0/2228276664 shutdown_connections 2024-03-23T17:52:42.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.507+0000 7faf78560b80 1 --2- 172.21.15.86:0/2228276664 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ad8410ddf0 0x55ad84112f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.507+0000 7faf78560b80 1 --2- 172.21.15.86:0/2228276664 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ad841025d0 0x55ad84114530 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.507+0000 7faf78560b80 1 --2- 172.21.15.86:0/2228276664 >> [v2:172.21.15.152:6838/1100723992,v1:172.21.15.152:6839/1100723992] conn(0x55ad841700a0 0x55ad84172490 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.507+0000 7faf78560b80 1 --2- 172.21.15.86:0/2228276664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad84102ef0 0x55ad84112a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.507+0000 7faf3f400700 0 client.26186 ms_handle_reset on v2:172.21.15.152:6838/1100723992 2024-03-23T17:52:42.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.507+0000 7faf78560b80 1 -- 172.21.15.86:0/2228276664 >> 172.21.15.86:0/2228276664 conn(0x55ad840626a0 msgr2=0x55ad8410b480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:42.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.507+0000 7faf78560b80 1 -- 172.21.15.86:0/2228276664 shutdown_connections 2024-03-23T17:52:42.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.507+0000 7faf78560b80 1 -- 172.21.15.86:0/2228276664 wait complete. 2024-03-23T17:52:42.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.507+0000 7faf78560b80 20 client.26186 trim_cache size 0 max 16384 2024-03-23T17:52:42.512 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-03-23T17:52:42.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.508+0000 7faf64a00700 1 -- 172.21.15.86:0/470132327 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7faf440a1290 msgr2=0x7faf440a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:42.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.508+0000 7faf64a00700 1 --2- 172.21.15.86:0/470132327 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7faf440a1290 0x7faf440a3750 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7faf58009c10 tx=0x7faf5800b040 comp rx=0 tx=0).stop 2024-03-23T17:52:42.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.508+0000 7faf64a00700 1 -- 172.21.15.86:0/470132327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf60106090 msgr2=0x7faf60199a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:42.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.508+0000 7faf64a00700 1 --2- 172.21.15.86:0/470132327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf60106090 0x7faf60199a00 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7faf4c00e9c0 tx=0x7faf4c00ecd0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.508+0000 7faf64a00700 1 -- 172.21.15.86:0/470132327 shutdown_connections 2024-03-23T17:52:42.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.508+0000 7faf64a00700 1 --2- 172.21.15.86:0/470132327 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7faf440a1290 0x7faf440a3750 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.508+0000 7faf64a00700 1 --2- 172.21.15.86:0/470132327 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf6019dd50 0x7faf6019e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.508+0000 7faf64a00700 1 --2- 172.21.15.86:0/470132327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf60100f70 0x7faf601994c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.508+0000 7faf64a00700 1 --2- 172.21.15.86:0/470132327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf60106090 0x7faf60199a00 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.508+0000 7faf64a00700 1 -- 172.21.15.86:0/470132327 >> 172.21.15.86:0/470132327 conn(0x7faf600fc7f0 msgr2=0x7faf600feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:42.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.508+0000 7faf64a00700 1 -- 172.21.15.86:0/470132327 shutdown_connections 2024-03-23T17:52:42.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.508+0000 7faf64a00700 1 -- 172.21.15.86:0/470132327 wait complete. 2024-03-23T17:52:42.528 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-23T17:52:42.528 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-23T17:52:42.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.742+0000 7fc94f400700 1 -- 172.21.15.86:0/2140296511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc948101920 msgr2=0x7fc948105960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:42.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.742+0000 7fc94f400700 1 --2- 172.21.15.86:0/2140296511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc948101920 0x7fc948105960 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fc934009990 tx=0x7fc934009ca0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.742+0000 7fc94f400700 1 -- 172.21.15.86:0/2140296511 shutdown_connections 2024-03-23T17:52:42.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.742+0000 7fc94f400700 1 --2- 172.21.15.86:0/2140296511 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc948106090 0x7fc94810dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.742+0000 7fc94f400700 1 --2- 172.21.15.86:0/2140296511 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc948100f70 0x7fc948101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.742+0000 7fc94f400700 1 --2- 172.21.15.86:0/2140296511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc948101920 0x7fc948105960 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.742+0000 7fc94f400700 1 -- 172.21.15.86:0/2140296511 >> 172.21.15.86:0/2140296511 conn(0x7fc9480fc7f0 msgr2=0x7fc9480fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:42.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.742+0000 7fc94f400700 1 -- 172.21.15.86:0/2140296511 shutdown_connections 2024-03-23T17:52:42.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.742+0000 7fc94f400700 1 -- 172.21.15.86:0/2140296511 wait complete. 2024-03-23T17:52:42.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.743+0000 7fc94f400700 1 Processor -- start 2024-03-23T17:52:42.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.743+0000 7fc94f400700 1 -- start start 2024-03-23T17:52:42.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.744+0000 7fc94f400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc948100f70 0x7fc948199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:42.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.744+0000 7fc94f400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc948106090 0x7fc948199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:42.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.744+0000 7fc94f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc94819dd10 0x7fc94819e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:42.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.744+0000 7fc94f400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc94819e790 con 0x7fc94819dd10 2024-03-23T17:52:42.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.744+0000 7fc94f400700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc94819e900 con 0x7fc948106090 2024-03-23T17:52:42.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.744+0000 7fc94f400700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc94819eac0 con 0x7fc948100f70 2024-03-23T17:52:42.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.744+0000 7fc947e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc948106090 0x7fc948199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:42.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.744+0000 7fc94ca00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc948100f70 0x7fc948199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:42.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.744+0000 7fc947e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc948106090 0x7fc948199970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:46024/0 (socket says 172.21.15.86:46024) 2024-03-23T17:52:42.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.744+0000 7fc947e00700 1 -- 172.21.15.86:0/2691717305 learned_addr learned my addr 172.21.15.86:0/2691717305 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:42.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.744+0000 7fc94ca00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc948100f70 0x7fc948199430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:52632/0 (socket says 172.21.15.86:52632) 2024-03-23T17:52:42.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.744+0000 7fc94d400700 1 --2- 172.21.15.86:0/2691717305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc94819dd10 0x7fc94819e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:42.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.744+0000 7fc947e00700 1 -- 172.21.15.86:0/2691717305 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc948100f70 msgr2=0x7fc948199430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:42.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.744+0000 7fc947e00700 1 --2- 172.21.15.86:0/2691717305 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc948100f70 0x7fc948199430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.744+0000 7fc947e00700 1 -- 172.21.15.86:0/2691717305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc94819dd10 msgr2=0x7fc94819e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:42.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.744+0000 7fc947e00700 1 --2- 172.21.15.86:0/2691717305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc94819dd10 0x7fc94819e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.744+0000 7fc947e00700 1 -- 172.21.15.86:0/2691717305 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9340095f0 con 0x7fc948106090 2024-03-23T17:52:42.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.744+0000 7fc94ca00700 1 --2- 172.21.15.86:0/2691717305 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc948100f70 0x7fc948199430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:52:42.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.745+0000 7fc947e00700 1 --2- 172.21.15.86:0/2691717305 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc948106090 0x7fc948199970 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc934000c00 tx=0x7fc934004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:42.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.745+0000 7fc945600700 1 -- 172.21.15.86:0/2691717305 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc93401d070 con 0x7fc948106090 2024-03-23T17:52:42.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.745+0000 7fc94f400700 1 -- 172.21.15.86:0/2691717305 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9481a5fc0 con 0x7fc948106090 2024-03-23T17:52:42.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.745+0000 7fc945600700 1 -- 172.21.15.86:0/2691717305 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc934021410 con 0x7fc948106090 2024-03-23T17:52:42.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.745+0000 7fc945600700 1 -- 172.21.15.86:0/2691717305 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc934017650 con 0x7fc948106090 2024-03-23T17:52:42.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.745+0000 7fc94f400700 1 -- 172.21.15.86:0/2691717305 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9481a6390 con 0x7fc948106090 2024-03-23T17:52:42.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.749+0000 7fc945600700 1 -- 172.21.15.86:0/2691717305 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fc934021580 con 0x7fc948106090 2024-03-23T17:52:42.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.750+0000 7fc962e86b80 1 Processor -- start 2024-03-23T17:52:42.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.750+0000 7fc962e86b80 1 -- start start 2024-03-23T17:52:42.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.750+0000 7fc962e86b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575b7e955d0 0x5575b7e959b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:42.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.750+0000 7fc945600700 1 --2- 172.21.15.86:0/2691717305 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fc9300a1290 0x7fc9300a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:42.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.750+0000 7fc962e86b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5575b7e95ef0 0x5575b7ea08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:42.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.750+0000 7fc962e86b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5575b7ea0df0 0x5575b7ea31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:42.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.750+0000 7fc962e86b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575b7dee990 con 0x5575b7e955d0 2024-03-23T17:52:42.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.750+0000 7fc962e86b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575b7dec350 con 0x5575b7ea0df0 2024-03-23T17:52:42.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.750+0000 7fc962e86b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575b7de7c40 con 0x5575b7e95ef0 2024-03-23T17:52:42.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.751+0000 7fc945600700 1 -- 172.21.15.86:0/2691717305 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fc9340cde60 con 0x7fc948106090 2024-03-23T17:52:42.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.751+0000 7fc947e00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5575b7e95ef0 0x5575b7ea08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:42.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.751+0000 7fc94d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575b7e955d0 0x5575b7e959b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:42.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.751+0000 7fc947e00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5575b7e95ef0 0x5575b7ea08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:52642/0 (socket says 172.21.15.86:52642) 2024-03-23T17:52:42.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.751+0000 7fc947e00700 1 -- 172.21.15.86:0/1267229304 learned_addr learned my addr 172.21.15.86:0/1267229304 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:42.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.751+0000 7fc94ca00700 1 --2- 172.21.15.86:0/2691717305 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fc9300a1290 0x7fc9300a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:42.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.751+0000 7fc94d400700 1 --2- 172.21.15.86:0/1267229304 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5575b7ea0df0 0x5575b7ea31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:42.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.751+0000 7fc947e00700 1 -- 172.21.15.86:0/1267229304 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5575b7ea0df0 msgr2=0x5575b7ea31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:42.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.751+0000 7fc947e00700 1 --2- 172.21.15.86:0/1267229304 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5575b7ea0df0 0x5575b7ea31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.751+0000 7fc947e00700 1 -- 172.21.15.86:0/1267229304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575b7e955d0 msgr2=0x5575b7e959b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:42.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.751+0000 7fc947e00700 1 --2- 172.21.15.86:0/1267229304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575b7e955d0 0x5575b7e959b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.751+0000 7fc947e00700 1 -- 172.21.15.86:0/1267229304 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5575b7c693b0 con 0x5575b7e95ef0 2024-03-23T17:52:42.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.751+0000 7fc94d400700 1 --2- 172.21.15.86:0/1267229304 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5575b7ea0df0 0x5575b7ea31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:52:42.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.751+0000 7fc947e00700 1 --2- 172.21.15.86:0/1267229304 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5575b7e95ef0 0x5575b7ea08b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc934097a60 tx=0x7fc934096ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80f46afed96978e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:42.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.751+0000 7fc939c00700 1 -- 172.21.15.86:0/1267229304 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc93409ab30 con 0x5575b7e95ef0 2024-03-23T17:52:42.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.751+0000 7fc94ca00700 1 --2- 172.21.15.86:0/2691717305 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fc9300a1290 0x7fc9300a3750 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc93c011490 tx=0x7fc93c009260 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:42.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.752+0000 7fc939c00700 1 -- 172.21.15.86:0/1267229304 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc93409a950 con 0x5575b7e95ef0 2024-03-23T17:52:42.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.752+0000 7fc939c00700 1 -- 172.21.15.86:0/1267229304 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9340a3450 con 0x5575b7e95ef0 2024-03-23T17:52:42.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.752+0000 7fc962e86b80 1 -- 172.21.15.86:0/1267229304 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5575b7e95ef0 msgr2=0x5575b7ea08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:42.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.752+0000 7fc962e86b80 1 --2- 172.21.15.86:0/1267229304 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5575b7e95ef0 0x5575b7ea08b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc934097a60 tx=0x7fc934096ce0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.752+0000 7fc962e86b80 1 -- 172.21.15.86:0/1267229304 shutdown_connections 2024-03-23T17:52:42.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.752+0000 7fc962e86b80 1 --2- 172.21.15.86:0/1267229304 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5575b7ea0df0 0x5575b7ea31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.752+0000 7fc962e86b80 1 --2- 172.21.15.86:0/1267229304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575b7e955d0 0x5575b7e959b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.752+0000 7fc962e86b80 1 --2- 172.21.15.86:0/1267229304 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5575b7e95ef0 0x5575b7ea08b0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.752+0000 7fc962e86b80 1 -- 172.21.15.86:0/1267229304 >> 172.21.15.86:0/1267229304 conn(0x5575b7df56a0 msgr2=0x5575b7e91a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:42.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.752+0000 7fc962e86b80 1 -- 172.21.15.86:0/1267229304 shutdown_connections 2024-03-23T17:52:42.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.752+0000 7fc962e86b80 1 -- 172.21.15.86:0/1267229304 wait complete. 2024-03-23T17:52:42.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.753+0000 7fc962e86b80 1 -- start start 2024-03-23T17:52:42.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.753+0000 7fc962e86b80 1 Processor -- start 2024-03-23T17:52:42.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.754+0000 7fc962e86b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5575b7e955d0 0x5575b7ea07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:42.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.754+0000 7fc962e86b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5575b7e95ef0 0x5575b7e9db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:42.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.754+0000 7fc962e86b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575b7ea0df0 0x5575b7e9e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:42.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.754+0000 7fc962e86b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575b7d83d80 con 0x5575b7ea0df0 2024-03-23T17:52:42.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.754+0000 7fc962e86b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575b7d83a20 con 0x5575b7e955d0 2024-03-23T17:52:42.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.754+0000 7fc962e86b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575b7c6b6a0 con 0x5575b7e95ef0 2024-03-23T17:52:42.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.754+0000 7fc962e86b80 10 client.0 _ll_register_callbacks cb 0x5575b7d8d470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:52:42.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.754+0000 7fc94ca00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5575b7e955d0 0x5575b7ea07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:42.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.754+0000 7fc94ca00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5575b7e955d0 0x5575b7ea07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:46040/0 (socket says 172.21.15.86:46040) 2024-03-23T17:52:42.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.754+0000 7fc94ca00700 1 -- 172.21.15.86:0/70216425 learned_addr learned my addr 172.21.15.86:0/70216425 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:42.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.754+0000 7fc94d400700 1 --2- 172.21.15.86:0/70216425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575b7ea0df0 0x5575b7e9e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:42.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.754+0000 7fc947e00700 1 --2- 172.21.15.86:0/70216425 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5575b7e95ef0 0x5575b7e9db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:42.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.754+0000 7fc94ca00700 1 -- 172.21.15.86:0/70216425 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5575b7e95ef0 msgr2=0x5575b7e9db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:42.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.754+0000 7fc94ca00700 1 --2- 172.21.15.86:0/70216425 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5575b7e95ef0 0x5575b7e9db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.754+0000 7fc94ca00700 1 -- 172.21.15.86:0/70216425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575b7ea0df0 msgr2=0x5575b7e9e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:42.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.754+0000 7fc94ca00700 1 --2- 172.21.15.86:0/70216425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575b7ea0df0 0x5575b7e9e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.754+0000 7fc94ca00700 1 -- 172.21.15.86:0/70216425 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9481a6390 con 0x5575b7e955d0 2024-03-23T17:52:42.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.754+0000 7fc947e00700 1 --2- 172.21.15.86:0/70216425 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5575b7e95ef0 0x5575b7e9db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:52:42.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.755+0000 7fc94ca00700 1 --2- 172.21.15.86:0/70216425 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5575b7e955d0 0x5575b7ea07a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc93c0039f0 tx=0x7fc93c0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed49cc965b09ec8a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:42.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.755+0000 7fc92e000700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T17:52:42.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.755+0000 7fc92e000700 1 -- 172.21.15.86:0/70216425 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc93c00e9c0 con 0x5575b7e955d0 2024-03-23T17:52:42.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.755+0000 7fc92e000700 1 -- 172.21.15.86:0/70216425 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fc93c00eb20 con 0x5575b7e955d0 2024-03-23T17:52:42.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.755+0000 7fc962e86b80 1 -- 172.21.15.86:0/70216425 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5575b7da8380 con 0x5575b7e955d0 2024-03-23T17:52:42.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.755+0000 7fc92e000700 1 -- 172.21.15.86:0/70216425 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc93c00ece0 con 0x5575b7e955d0 2024-03-23T17:52:42.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.755+0000 7fc962e86b80 10 client.26198 fetch_fsmap learned FSMap version 33 2024-03-23T17:52:42.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.755+0000 7fc962e86b80 1 -- 172.21.15.86:0/70216425 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5575b7c15b60 con 0x5575b7e955d0 2024-03-23T17:52:42.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.756+0000 7fc92e000700 1 -- 172.21.15.86:0/70216425 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5575b7da8380 con 0x5575b7e955d0 2024-03-23T17:52:42.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.756+0000 7fc92e000700 1 -- 172.21.15.86:0/70216425 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 33) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fc93c024020 con 0x5575b7e955d0 2024-03-23T17:52:42.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.756+0000 7fc962e86b80 10 client.26198 fetch_fsmap finished waiting for FSMap version 33 2024-03-23T17:52:42.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.756+0000 7fc962e86b80 10 client.26198 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T17:52:42.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.756+0000 7fc962e86b80 20 client.26198 populate_metadata read hostname 'smithi086' 2024-03-23T17:52:42.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.756+0000 7fc962e86b80 1 --2- 172.21.15.86:0/70216425 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5575b7eb06e0 0x5575b7eb2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:42.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.756+0000 7fc962e86b80 4 client.26198 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:52:42.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.756+0000 7fc962e86b80 1 -- 172.21.15.86:0/70216425 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5575b7de0700 con 0x5575b7eb06e0 2024-03-23T17:52:42.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.756+0000 7fc947e00700 1 --2- 172.21.15.86:0/70216425 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5575b7eb06e0 0x5575b7eb2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:42.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.756+0000 7fc92e000700 10 client.26198 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:52:42.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.756+0000 7fc947e00700 1 --2- 172.21.15.86:0/70216425 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5575b7eb06e0 0x5575b7eb2b00 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6f268814d8ff751 server_cookie=a500365c4de38108 in_seq=0 out_seq=0 2024-03-23T17:52:42.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.757+0000 7fc92e000700 1 -- 172.21.15.86:0/70216425 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5575b7de0700 con 0x5575b7eb06e0 2024-03-23T17:52:42.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.757+0000 7fc92e000700 10 client.26198 handle_command_reply: tid=0 2024-03-23T17:52:42.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.757+0000 7fc962e86b80 1 client.26198 shutdown 2024-03-23T17:52:42.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.757+0000 7fc962e86b80 1 -- 172.21.15.86:0/70216425 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5575b7e955d0 msgr2=0x5575b7ea07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:42.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.757+0000 7fc962e86b80 1 --2- 172.21.15.86:0/70216425 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5575b7e955d0 0x5575b7ea07a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc93c0039f0 tx=0x7fc93c0047c0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.757+0000 7fc962e86b80 1 -- 172.21.15.86:0/70216425 shutdown_connections 2024-03-23T17:52:42.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.757+0000 7fc962e86b80 1 --2- 172.21.15.86:0/70216425 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5575b7eb06e0 0x5575b7eb2b00 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.757+0000 7fc962e86b80 1 --2- 172.21.15.86:0/70216425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575b7ea0df0 0x5575b7e9e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.757+0000 7fc962e86b80 1 --2- 172.21.15.86:0/70216425 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5575b7e955d0 0x5575b7ea07a0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.757+0000 7fc962e86b80 1 --2- 172.21.15.86:0/70216425 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5575b7e95ef0 0x5575b7e9db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.757+0000 7fc92e000700 0 client.26198 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:52:42.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.757+0000 7fc962e86b80 1 -- 172.21.15.86:0/70216425 >> 172.21.15.86:0/70216425 conn(0x5575b7df56a0 msgr2=0x5575b7e976c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:42.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.757+0000 7fc962e86b80 1 -- 172.21.15.86:0/70216425 shutdown_connections 2024-03-23T17:52:42.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.757+0000 7fc962e86b80 1 -- 172.21.15.86:0/70216425 wait complete. 2024-03-23T17:52:42.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.758+0000 7fc962e86b80 20 client.26198 trim_cache size 0 max 16384 2024-03-23T17:52:42.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.773+0000 7fc962e86b80 1 Processor -- start 2024-03-23T17:52:42.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.773+0000 7fc962e86b80 1 -- start start 2024-03-23T17:52:42.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.773+0000 7fc962e86b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575b7e955d0 0x5575b7eb7420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:42.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.773+0000 7fc962e86b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5575b7e95ef0 0x5575b7e985f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:42.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.773+0000 7fc962e86b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5575b7ea0df0 0x5575b7e9e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:42.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.773+0000 7fc962e86b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575b7e91fc0 con 0x5575b7e955d0 2024-03-23T17:52:42.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.773+0000 7fc962e86b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575b7eadd20 con 0x5575b7ea0df0 2024-03-23T17:52:42.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.773+0000 7fc962e86b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575b7e9e6d0 con 0x5575b7e95ef0 2024-03-23T17:52:42.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.773+0000 7fc947e00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5575b7e95ef0 0x5575b7e985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:42.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.773+0000 7fc947e00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5575b7e95ef0 0x5575b7e985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:52664/0 (socket says 172.21.15.86:52664) 2024-03-23T17:52:42.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.773+0000 7fc947e00700 1 -- 172.21.15.86:0/3987949886 learned_addr learned my addr 172.21.15.86:0/3987949886 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:42.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.773+0000 7fc94d400700 1 --2- 172.21.15.86:0/3987949886 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5575b7ea0df0 0x5575b7e9e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:42.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.773+0000 7fc94ca00700 1 --2- 172.21.15.86:0/3987949886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575b7e955d0 0x5575b7eb7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:42.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.773+0000 7fc947e00700 1 -- 172.21.15.86:0/3987949886 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5575b7ea0df0 msgr2=0x5575b7e9e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:42.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.773+0000 7fc947e00700 1 --2- 172.21.15.86:0/3987949886 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5575b7ea0df0 0x5575b7e9e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.773+0000 7fc947e00700 1 -- 172.21.15.86:0/3987949886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575b7e955d0 msgr2=0x5575b7eb7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:42.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.773+0000 7fc947e00700 1 --2- 172.21.15.86:0/3987949886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575b7e955d0 0x5575b7eb7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.773+0000 7fc947e00700 1 -- 172.21.15.86:0/3987949886 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5575b7dadea0 con 0x5575b7e95ef0 2024-03-23T17:52:42.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.773+0000 7fc94ca00700 1 --2- 172.21.15.86:0/3987949886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575b7e955d0 0x5575b7eb7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:52:42.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.774+0000 7fc947e00700 1 --2- 172.21.15.86:0/3987949886 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5575b7e95ef0 0x5575b7e985f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc934096d70 tx=0x7fc934021c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76490de1933a2942 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:42.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.774+0000 7fc93a600700 1 -- 172.21.15.86:0/3987949886 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc93409ab30 con 0x5575b7e95ef0 2024-03-23T17:52:42.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.774+0000 7fc93a600700 1 -- 172.21.15.86:0/3987949886 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc93409ee20 con 0x5575b7e95ef0 2024-03-23T17:52:42.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.774+0000 7fc93a600700 1 -- 172.21.15.86:0/3987949886 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc934099420 con 0x5575b7e95ef0 2024-03-23T17:52:42.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.775+0000 7fc962e86b80 1 -- 172.21.15.86:0/3987949886 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5575b7e95ef0 msgr2=0x5575b7e985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:42.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.775+0000 7fc962e86b80 1 --2- 172.21.15.86:0/3987949886 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5575b7e95ef0 0x5575b7e985f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc934096d70 tx=0x7fc934021c60 comp rx=0 tx=0).stop 2024-03-23T17:52:42.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.775+0000 7fc962e86b80 1 -- 172.21.15.86:0/3987949886 shutdown_connections 2024-03-23T17:52:42.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.775+0000 7fc962e86b80 1 --2- 172.21.15.86:0/3987949886 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5575b7ea0df0 0x5575b7e9e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.775+0000 7fc962e86b80 1 --2- 172.21.15.86:0/3987949886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575b7e955d0 0x5575b7eb7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.775+0000 7fc962e86b80 1 --2- 172.21.15.86:0/3987949886 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5575b7e95ef0 0x5575b7e985f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.775+0000 7fc962e86b80 1 -- 172.21.15.86:0/3987949886 >> 172.21.15.86:0/3987949886 conn(0x5575b7df56a0 msgr2=0x5575b7eabdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:42.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.775+0000 7fc962e86b80 1 -- 172.21.15.86:0/3987949886 shutdown_connections 2024-03-23T17:52:42.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.775+0000 7fc962e86b80 1 -- 172.21.15.86:0/3987949886 wait complete. 2024-03-23T17:52:42.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.775+0000 7fc962e86b80 1 -- start start 2024-03-23T17:52:42.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.775+0000 7fc962e86b80 1 Processor -- start 2024-03-23T17:52:42.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.776+0000 7fc962e86b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5575b7e955d0 0x5575b7ea59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:42.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.776+0000 7fc962e86b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5575b7e95ef0 0x5575b7ea5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:42.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.776+0000 7fc962e86b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575b7ea0df0 0x5575b7efb190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:42.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.776+0000 7fc962e86b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575b7efb710 con 0x5575b7ea0df0 2024-03-23T17:52:42.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.776+0000 7fc962e86b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575b7efb880 con 0x5575b7e955d0 2024-03-23T17:52:42.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.776+0000 7fc962e86b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575b7efba40 con 0x5575b7e95ef0 2024-03-23T17:52:42.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.776+0000 7fc962e86b80 10 client.0 _ll_register_callbacks cb 0x5575b7d8d470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:52:42.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.776+0000 7fc94ca00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5575b7e955d0 0x5575b7ea59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:42.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.776+0000 7fc94ca00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5575b7e955d0 0x5575b7ea59f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:46056/0 (socket says 172.21.15.86:46056) 2024-03-23T17:52:42.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.776+0000 7fc94ca00700 1 -- 172.21.15.86:0/471241186 learned_addr learned my addr 172.21.15.86:0/471241186 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:42.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.776+0000 7fc947e00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5575b7e95ef0 0x5575b7ea5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:42.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.776+0000 7fc94d400700 1 --2- 172.21.15.86:0/471241186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575b7ea0df0 0x5575b7efb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:42.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.776+0000 7fc94ca00700 1 -- 172.21.15.86:0/471241186 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5575b7e95ef0 msgr2=0x5575b7ea5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:42.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.776+0000 7fc94ca00700 1 --2- 172.21.15.86:0/471241186 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5575b7e95ef0 0x5575b7ea5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.776+0000 7fc94ca00700 1 -- 172.21.15.86:0/471241186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575b7ea0df0 msgr2=0x5575b7efb190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:42.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.776+0000 7fc94ca00700 1 --2- 172.21.15.86:0/471241186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575b7ea0df0 0x5575b7efb190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.776+0000 7fc94ca00700 1 -- 172.21.15.86:0/471241186 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5575b7c693b0 con 0x5575b7e955d0 2024-03-23T17:52:42.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.776+0000 7fc947e00700 1 --2- 172.21.15.86:0/471241186 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5575b7e95ef0 0x5575b7ea5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:52:42.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.777+0000 7fc94ca00700 1 --2- 172.21.15.86:0/471241186 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5575b7e955d0 0x5575b7ea59f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc93c0264a0 tx=0x7fc93c026ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49818baa09ebaf23 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:42.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.777+0000 7fc92e000700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T17:52:42.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.777+0000 7fc92e000700 1 -- 172.21.15.86:0/471241186 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc93c00ea60 con 0x5575b7e955d0 2024-03-23T17:52:42.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.777+0000 7fc92e000700 1 -- 172.21.15.86:0/471241186 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fc93c00ebc0 con 0x5575b7e955d0 2024-03-23T17:52:42.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.777+0000 7fc962e86b80 1 -- 172.21.15.86:0/471241186 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc93409a950 con 0x5575b7e955d0 2024-03-23T17:52:42.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.777+0000 7fc92e000700 1 -- 172.21.15.86:0/471241186 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc93c00ed80 con 0x5575b7e955d0 2024-03-23T17:52:42.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.777+0000 7fc962e86b80 10 client.26204 fetch_fsmap learned FSMap version 33 2024-03-23T17:52:42.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.777+0000 7fc962e86b80 1 -- 172.21.15.86:0/471241186 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5575b7c6dcd0 con 0x5575b7e955d0 2024-03-23T17:52:42.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.777+0000 7fc92e000700 1 -- 172.21.15.86:0/471241186 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc93409a950 con 0x5575b7e955d0 2024-03-23T17:52:42.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.778+0000 7fc92e000700 1 -- 172.21.15.86:0/471241186 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 33) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fc93c003a20 con 0x5575b7e955d0 2024-03-23T17:52:42.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.778+0000 7fc962e86b80 10 client.26204 fetch_fsmap finished waiting for FSMap version 33 2024-03-23T17:52:42.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.778+0000 7fc962e86b80 10 client.26204 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T17:52:42.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.778+0000 7fc962e86b80 20 client.26204 populate_metadata read hostname 'smithi086' 2024-03-23T17:52:42.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.778+0000 7fc962e86b80 1 --2- 172.21.15.86:0/471241186 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5575b7f03130 0x5575b7f05550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:42.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.778+0000 7fc962e86b80 4 client.26204 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-23T17:52:42.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.778+0000 7fc962e86b80 1 -- 172.21.15.86:0/471241186 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5575b7c36cf0 con 0x5575b7f03130 2024-03-23T17:52:42.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.778+0000 7fc947e00700 1 --2- 172.21.15.86:0/471241186 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5575b7f03130 0x5575b7f05550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:42.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.778+0000 7fc92e000700 10 client.26204 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:52:42.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.778+0000 7fc947e00700 1 --2- 172.21.15.86:0/471241186 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5575b7f03130 0x5575b7f05550 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c2552c27c2434b7 server_cookie=dd03853fb037b7b2 in_seq=0 out_seq=0 2024-03-23T17:52:42.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.779+0000 7fc92e000700 1 -- 172.21.15.86:0/471241186 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5575b7c36cf0 con 0x5575b7f03130 2024-03-23T17:52:42.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.779+0000 7fc92e000700 10 client.26204 handle_command_reply: tid=0 2024-03-23T17:52:42.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.779+0000 7fc962e86b80 1 client.26204 shutdown 2024-03-23T17:52:42.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.779+0000 7fc962e86b80 1 -- 172.21.15.86:0/471241186 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5575b7e955d0 msgr2=0x5575b7ea59f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:42.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.779+0000 7fc962e86b80 1 --2- 172.21.15.86:0/471241186 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5575b7e955d0 0x5575b7ea59f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc93c0264a0 tx=0x7fc93c026ce0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.779+0000 7fc962e86b80 1 -- 172.21.15.86:0/471241186 shutdown_connections 2024-03-23T17:52:42.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.779+0000 7fc962e86b80 1 --2- 172.21.15.86:0/471241186 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5575b7f03130 0x5575b7f05550 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.779+0000 7fc962e86b80 1 --2- 172.21.15.86:0/471241186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575b7ea0df0 0x5575b7efb190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.779+0000 7fc962e86b80 1 --2- 172.21.15.86:0/471241186 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5575b7e955d0 0x5575b7ea59f0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.779+0000 7fc962e86b80 1 --2- 172.21.15.86:0/471241186 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5575b7e95ef0 0x5575b7ea5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.779+0000 7fc92e000700 0 client.26204 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:52:42.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.779+0000 7fc962e86b80 1 -- 172.21.15.86:0/471241186 >> 172.21.15.86:0/471241186 conn(0x5575b7df56a0 msgr2=0x5575b7e917a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:42.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.779+0000 7fc962e86b80 1 -- 172.21.15.86:0/471241186 shutdown_connections 2024-03-23T17:52:42.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.779+0000 7fc962e86b80 1 -- 172.21.15.86:0/471241186 wait complete. 2024-03-23T17:52:42.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.779+0000 7fc962e86b80 20 client.26204 trim_cache size 0 max 16384 2024-03-23T17:52:42.784 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-03-23T17:52:42.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.779+0000 7fc94f400700 1 -- 172.21.15.86:0/2691717305 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fc9300a1290 msgr2=0x7fc9300a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:42.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.780+0000 7fc94f400700 1 --2- 172.21.15.86:0/2691717305 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fc9300a1290 0x7fc9300a3750 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc93c011490 tx=0x7fc93c009260 comp rx=0 tx=0).stop 2024-03-23T17:52:42.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.780+0000 7fc94f400700 1 -- 172.21.15.86:0/2691717305 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc948106090 msgr2=0x7fc948199970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:42.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.780+0000 7fc94f400700 1 --2- 172.21.15.86:0/2691717305 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc948106090 0x7fc948199970 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc934000c00 tx=0x7fc934004730 comp rx=0 tx=0).stop 2024-03-23T17:52:42.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.780+0000 7fc94f400700 1 -- 172.21.15.86:0/2691717305 shutdown_connections 2024-03-23T17:52:42.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.780+0000 7fc94f400700 1 --2- 172.21.15.86:0/2691717305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc94819dd10 0x7fc94819e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.780+0000 7fc94f400700 1 --2- 172.21.15.86:0/2691717305 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fc9300a1290 0x7fc9300a3750 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.780+0000 7fc94f400700 1 --2- 172.21.15.86:0/2691717305 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc948100f70 0x7fc948199430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.780+0000 7fc94f400700 1 --2- 172.21.15.86:0/2691717305 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc948106090 0x7fc948199970 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:42.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.780+0000 7fc94f400700 1 -- 172.21.15.86:0/2691717305 >> 172.21.15.86:0/2691717305 conn(0x7fc9480fc7f0 msgr2=0x7fc9480feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:42.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.780+0000 7fc94f400700 1 -- 172.21.15.86:0/2691717305 shutdown_connections 2024-03-23T17:52:42.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T17:52:42.780+0000 7fc94f400700 1 -- 172.21.15.86:0/2691717305 wait complete. 2024-03-23T17:52:42.798 ERROR:tasks.fwd_scrub:rank 4 damaged: [{'damage_type': 'backtrace', 'id': 342231839, 'ino': 1099511629297, 'path': '~mds4/stray1/100000005f1'}, {'damage_type': 'backtrace', 'id': 766548073, 'ino': 1099511628351, 'path': '/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/ACCESS/FASTENER.MDB'}, {'damage_type': 'backtrace', 'id': 1126266749, 'ino': 1099511628354, 'path': '/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/SEED/MEDIUM.FIL'}, {'damage_type': 'backtrace', 'id': 2478609256, 'ino': 1099511628355, 'path': '/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/SEED/LARGE.FIL'}, {'damage_type': 'backtrace', 'id': 2718211712, 'ino': 1099511629299, 'path': '~mds4/stray1/100000005f3'}, {'damage_type': 'backtrace', 'id': 3153567874, 'ino': 1099511629295, 'path': '~mds4/stray1/100000005ef'}, {'damage_type': 'backtrace', 'id': 3212837527, 'ino': 1099511628353, 'path': '/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/SEED/SMALL.FIL'}] 2024-03-23T17:52:42.798 ERROR:tasks.fwd_scrub:rank 3 damaged: [{'damage_type': 'backtrace', 'id': 178463209, 'ino': 1099511628299, 'path': '/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/WORD/FACTS.DOC'}, {'damage_type': 'backtrace', 'id': 1280717372, 'ino': 1099511628301, 'path': '/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/EXCEL/PCMAGCD.XLS'}, {'damage_type': 'backtrace', 'id': 1739423601, 'ino': 1099511628302, 'path': '/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/EXCEL/SALES.XLS'}, {'damage_type': 'backtrace', 'id': 2686714892, 'ino': 1099511628298, 'path': '/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/WORD/BASEMACH.DOC'}, {'damage_type': 'backtrace', 'id': 3180308212, 'ino': 1099511628297, 'path': '/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/WORD/CHAP10.DOC'}] 2024-03-23T17:52:42.798 ERROR:tasks.fwd_scrub:rank 2 damaged: [{'damage_type': 'backtrace', 'id': 41960410, 'ino': 1099511628324, 'path': '/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/WORDPRO/NEWS1_1.LWP'}, {'damage_type': 'backtrace', 'id': 967553592, 'ino': 1099511628304, 'path': '/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/PWRPNT/PCBENCHM.PPT'}, {'damage_type': 'backtrace', 'id': 1759052993, 'ino': 1099511628308, 'path': '/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/PWRPNT/TRIDOTS.POT'}, {'damage_type': 'backtrace', 'id': 2303084638, 'ino': 1099511628305, 'path': '/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/PWRPNT/ZD16.BMP'}, {'damage_type': 'backtrace', 'id': 3292507529, 'ino': 1099511628306, 'path': '/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/PWRPNT/PPTOOLS1.PPA'}, {'damage_type': 'backtrace', 'id': 3548755771, 'ino': 1099511628323, 'path': '/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/WORDPRO/RESULTS.XLS'}, {'damage_type': 'backtrace', 'id': 3644599087, 'ino': 1099511628322, 'path': '/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/WORDPRO/BENCHS.LWP'}, {'damage_type': 'backtrace', 'id': 4223820534, 'ino': 1099511628307, 'path': '/volumes/qa/sv_1/aa7fd127-71c3-47dd-b16f-38c26b2c15fc/client.0/tmp/clients/client0/~dmtmp/PWRPNT/TIPS.PPT'}] 2024-03-23T17:52:42.798 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph-c_82316b04821a6c2c1d458a881920454bbadb8837/qa/tasks/fwd_scrub.py", line 38, in _run self.do_scrub() File "/home/teuthworker/src/github.com_ceph_ceph-c_82316b04821a6c2c1d458a881920454bbadb8837/qa/tasks/fwd_scrub.py", line 55, in do_scrub self._scrub() File "/home/teuthworker/src/github.com_ceph_ceph-c_82316b04821a6c2c1d458a881920454bbadb8837/qa/tasks/fwd_scrub.py", line 79, in _scrub self._check_damage() File "/home/teuthworker/src/github.com_ceph_ceph-c_82316b04821a6c2c1d458a881920454bbadb8837/qa/tasks/fwd_scrub.py", line 90, in _check_damage raise RuntimeError(f"rank damage found: {types}") RuntimeError: rank damage found: {'backtrace'} 2024-03-23T17:52:42.884 INFO:tasks.workunit.client.0.smithi086.stdout: 1 50355 4.31 MB/sec warmup 67 sec latency 107.749 ms 2024-03-23T17:52:43.884 INFO:tasks.workunit.client.0.smithi086.stdout: 1 50355 4.25 MB/sec warmup 68 sec latency 1107.838 ms 2024-03-23T17:52:44.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:43 smithi086 ceph-mon[38221]: pgmap v65: 97 pgs: 97 active+clean; 34 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 326 KiB/s wr, 51 op/s 2024-03-23T17:52:44.348 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:43 smithi152 ceph-mon[47949]: pgmap v65: 97 pgs: 97 active+clean; 34 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 326 KiB/s wr, 51 op/s 2024-03-23T17:52:44.348 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:43 smithi092 ceph-mon[79089]: pgmap v65: 97 pgs: 97 active+clean; 34 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 326 KiB/s wr, 51 op/s 2024-03-23T17:52:44.884 INFO:tasks.workunit.client.0.smithi086.stdout: 1 50355 4.19 MB/sec warmup 69 sec latency 2107.886 ms 2024-03-23T17:52:45.884 INFO:tasks.workunit.client.0.smithi086.stdout: 1 50355 4.13 MB/sec warmup 70 sec latency 3107.931 ms 2024-03-23T17:52:46.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:45 smithi086 ceph-mon[38221]: pgmap v66: 97 pgs: 97 active+clean; 35 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 532 KiB/s wr, 98 op/s 2024-03-23T17:52:46.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:45 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:52:46.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:45 smithi092 ceph-mon[79089]: pgmap v66: 97 pgs: 97 active+clean; 35 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 532 KiB/s wr, 98 op/s 2024-03-23T17:52:46.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:45 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:52:46.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:45 smithi152 ceph-mon[47949]: pgmap v66: 97 pgs: 97 active+clean; 35 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 532 KiB/s wr, 98 op/s 2024-03-23T17:52:46.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:45 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:52:46.884 INFO:tasks.workunit.client.0.smithi086.stdout: 1 50479 4.07 MB/sec warmup 71 sec latency 3128.886 ms 2024-03-23T17:52:47.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:46 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:52:47.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:46 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:52:47.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:46 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:52:47.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:46 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:52:47.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:46 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:52:47.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:46 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:52:47.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:46 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:52:47.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:46 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:52:47.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:46 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:52:47.884 INFO:tasks.workunit.client.0.smithi086.stdout: 1 50479 4.01 MB/sec warmup 72 sec latency 1902.084 ms 2024-03-23T17:52:48.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:47 smithi086 ceph-mon[38221]: pgmap v67: 97 pgs: 97 active+clean; 35 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 469 KiB/s wr, 93 op/s 2024-03-23T17:52:48.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:47 smithi092 ceph-mon[79089]: pgmap v67: 97 pgs: 97 active+clean; 35 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 469 KiB/s wr, 93 op/s 2024-03-23T17:52:48.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:47 smithi152 ceph-mon[47949]: pgmap v67: 97 pgs: 97 active+clean; 35 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 469 KiB/s wr, 93 op/s 2024-03-23T17:52:48.884 INFO:tasks.workunit.client.0.smithi086.stdout: 1 52449 4.11 MB/sec warmup 73 sec latency 2079.801 ms 2024-03-23T17:52:49.885 INFO:tasks.workunit.client.0.smithi086.stdout: 1 53947 4.16 MB/sec warmup 74 sec latency 10.336 ms 2024-03-23T17:52:50.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:49 smithi086 ceph-mon[38221]: pgmap v68: 97 pgs: 97 active+clean; 35 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 439 KiB/s wr, 89 op/s 2024-03-23T17:52:50.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:49 smithi092 ceph-mon[79089]: pgmap v68: 97 pgs: 97 active+clean; 35 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 439 KiB/s wr, 89 op/s 2024-03-23T17:52:50.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:49 smithi152 ceph-mon[47949]: pgmap v68: 97 pgs: 97 active+clean; 35 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 439 KiB/s wr, 89 op/s 2024-03-23T17:52:50.885 INFO:tasks.workunit.client.0.smithi086.stdout: 1 53959 4.11 MB/sec warmup 75 sec latency 982.671 ms 2024-03-23T17:52:51.884 INFO:tasks.workunit.client.0.smithi086.stdout: 1 56235 4.22 MB/sec warmup 76 sec latency 1000.890 ms 2024-03-23T17:52:52.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:51 smithi086 ceph-mon[38221]: pgmap v69: 97 pgs: 97 active+clean; 35 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 2.1 MiB/s wr, 179 op/s 2024-03-23T17:52:52.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:51 smithi092 ceph-mon[79089]: pgmap v69: 97 pgs: 97 active+clean; 35 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 2.1 MiB/s wr, 179 op/s 2024-03-23T17:52:52.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:51 smithi152 ceph-mon[47949]: pgmap v69: 97 pgs: 97 active+clean; 35 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 2.1 MiB/s wr, 179 op/s 2024-03-23T17:52:52.884 INFO:tasks.workunit.client.0.smithi086.stdout: 1 57503 4.25 MB/sec warmup 77 sec latency 74.456 ms 2024-03-23T17:52:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.614+0000 7fbf42800700 1 -- 172.21.15.86:0/1949175891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf3c106030 msgr2=0x7fbf3c10dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.614+0000 7fbf42800700 1 --2- 172.21.15.86:0/1949175891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf3c106030 0x7fbf3c10dbb0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fbf3400b0a0 tx=0x7fbf3400b3b0 comp rx=0 tx=0).stop 2024-03-23T17:52:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.614+0000 7fbf42800700 1 -- 172.21.15.86:0/1949175891 shutdown_connections 2024-03-23T17:52:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.614+0000 7fbf42800700 1 --2- 172.21.15.86:0/1949175891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf3c106030 0x7fbf3c10dbb0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.614+0000 7fbf42800700 1 --2- 172.21.15.86:0/1949175891 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf3c100f10 0x7fbf3c1012f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.614+0000 7fbf42800700 1 --2- 172.21.15.86:0/1949175891 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbf3c1018c0 0x7fbf3c105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.614+0000 7fbf42800700 1 -- 172.21.15.86:0/1949175891 >> 172.21.15.86:0/1949175891 conn(0x7fbf3c0fc7b0 msgr2=0x7fbf3c0febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.614+0000 7fbf42800700 1 -- 172.21.15.86:0/1949175891 shutdown_connections 2024-03-23T17:52:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.614+0000 7fbf42800700 1 -- 172.21.15.86:0/1949175891 wait complete. 2024-03-23T17:52:53.619 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.615+0000 7fbf42800700 1 Processor -- start 2024-03-23T17:52:53.619 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.615+0000 7fbf42800700 1 -- start start 2024-03-23T17:52:53.619 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.616+0000 7fbf42800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf3c100f10 0x7fbf3c1994a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:53.619 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.616+0000 7fbf42800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf3c106030 0x7fbf3c1999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:53.619 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.616+0000 7fbf42800700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbf3c19dd80 0x7fbf3c19e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:53.619 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.616+0000 7fbf42800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3c19e800 con 0x7fbf3c106030 2024-03-23T17:52:53.619 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.616+0000 7fbf42800700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3c19e970 con 0x7fbf3c100f10 2024-03-23T17:52:53.619 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.616+0000 7fbf42800700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3c19eb30 con 0x7fbf3c19dd80 2024-03-23T17:52:53.619 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.616+0000 7fbf3aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf3c106030 0x7fbf3c1999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:53.619 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.616+0000 7fbf3aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf3c106030 0x7fbf3c1999e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40410/0 (socket says 172.21.15.86:40410) 2024-03-23T17:52:53.619 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.616+0000 7fbf3aa00700 1 -- 172.21.15.86:0/3622729572 learned_addr learned my addr 172.21.15.86:0/3622729572 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:53.620 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.616+0000 7fbf3be00700 1 --2- 172.21.15.86:0/3622729572 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbf3c19dd80 0x7fbf3c19e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:53.620 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.616+0000 7fbf3b400700 1 --2- 172.21.15.86:0/3622729572 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf3c100f10 0x7fbf3c1994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:53.620 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.617+0000 7fbf3aa00700 1 -- 172.21.15.86:0/3622729572 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbf3c19dd80 msgr2=0x7fbf3c19e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:53.620 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.617+0000 7fbf3aa00700 1 --2- 172.21.15.86:0/3622729572 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbf3c19dd80 0x7fbf3c19e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:53.620 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.617+0000 7fbf3aa00700 1 -- 172.21.15.86:0/3622729572 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf3c100f10 msgr2=0x7fbf3c1994a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:53.620 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.617+0000 7fbf3aa00700 1 --2- 172.21.15.86:0/3622729572 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf3c100f10 0x7fbf3c1994a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:53.620 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.617+0000 7fbf3aa00700 1 -- 172.21.15.86:0/3622729572 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf34009d00 con 0x7fbf3c106030 2024-03-23T17:52:53.620 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.617+0000 7fbf3aa00700 1 --2- 172.21.15.86:0/3622729572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf3c106030 0x7fbf3c1999e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fbf2800e970 tx=0x7fbf2800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:53.620 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.617+0000 7fbf33e00700 1 -- 172.21.15.86:0/3622729572 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf28009790 con 0x7fbf3c106030 2024-03-23T17:52:53.620 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.617+0000 7fbf33e00700 1 -- 172.21.15.86:0/3622729572 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf2800c7e0 con 0x7fbf3c106030 2024-03-23T17:52:53.620 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.617+0000 7fbf33e00700 1 -- 172.21.15.86:0/3622729572 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf280054e0 con 0x7fbf3c106030 2024-03-23T17:52:53.620 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.617+0000 7fbf42800700 1 -- 172.21.15.86:0/3622729572 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf3c1a6090 con 0x7fbf3c106030 2024-03-23T17:52:53.620 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.617+0000 7fbf42800700 1 -- 172.21.15.86:0/3622729572 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf3c1a6570 con 0x7fbf3c106030 2024-03-23T17:52:53.621 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.620+0000 7fbf33e00700 1 -- 172.21.15.86:0/3622729572 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fbf2800c950 con 0x7fbf3c106030 2024-03-23T17:52:53.624 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.622+0000 7fbf5626eb80 1 Processor -- start 2024-03-23T17:52:53.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.622+0000 7fbf33e00700 1 --2- 172.21.15.86:0/3622729572 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fbf200a1310 0x7fbf200a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:53.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.622+0000 7fbf5626eb80 1 -- start start 2024-03-23T17:52:53.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.622+0000 7fbf33e00700 1 -- 172.21.15.86:0/3622729572 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fbf28014070 con 0x7fbf3c106030 2024-03-23T17:52:53.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.622+0000 7fbf3be00700 1 --2- 172.21.15.86:0/3622729572 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fbf200a1310 0x7fbf200a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:53.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.622+0000 7fbf5626eb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555e31623100 0x555e316234e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:53.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.622+0000 7fbf5626eb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555e31623a20 0x555e3162e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:53.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.622+0000 7fbf5626eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e3162e8d0 0x555e31630cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:53.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.622+0000 7fbf5626eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e3157c930 con 0x555e3162e8d0 2024-03-23T17:52:53.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.622+0000 7fbf5626eb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e3157a2f0 con 0x555e31623100 2024-03-23T17:52:53.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.622+0000 7fbf5626eb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e31575be0 con 0x555e31623a20 2024-03-23T17:52:53.626 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.622+0000 7fbf3aa00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555e31623a20 0x555e3162e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:53.626 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.622+0000 7fbf3b400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555e31623100 0x555e316234e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:53.626 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.622+0000 7fbf3aa00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555e31623a20 0x555e3162e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:37834/0 (socket says 172.21.15.86:37834) 2024-03-23T17:52:53.626 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.623+0000 7fbf3aa00700 1 -- 172.21.15.86:0/2334037746 learned_addr learned my addr 172.21.15.86:0/2334037746 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:53.626 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.623+0000 7fbf3be00700 1 --2- 172.21.15.86:0/2334037746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e3162e8d0 0x555e31630cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:53.626 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.623+0000 7fbf3aa00700 1 -- 172.21.15.86:0/2334037746 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555e31623100 msgr2=0x555e316234e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:53.626 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.623+0000 7fbf3aa00700 1 --2- 172.21.15.86:0/2334037746 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555e31623100 0x555e316234e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:53.626 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.623+0000 7fbf3aa00700 1 -- 172.21.15.86:0/2334037746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e3162e8d0 msgr2=0x555e31630cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:53.626 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.623+0000 7fbf3aa00700 1 --2- 172.21.15.86:0/2334037746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e3162e8d0 0x555e31630cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:53.626 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.623+0000 7fbf3be00700 1 --2- 172.21.15.86:0/3622729572 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fbf200a1310 0x7fbf200a37d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbf3400b780 tx=0x7fbf340095c0 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:53.626 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.623+0000 7fbf3aa00700 1 -- 172.21.15.86:0/2334037746 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e313f7350 con 0x555e31623a20 2024-03-23T17:52:53.627 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.623+0000 7fbf3b400700 1 --2- 172.21.15.86:0/2334037746 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555e31623100 0x555e316234e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:52:53.627 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.623+0000 7fbf3aa00700 1 --2- 172.21.15.86:0/2334037746 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555e31623a20 0x555e3162e390 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbf28009230 tx=0x7fbf2800ccd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5b51dec14ae27fa server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:53.627 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.623+0000 7fbf30c00700 1 -- 172.21.15.86:0/2334037746 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf2809dd90 con 0x555e31623a20 2024-03-23T17:52:53.627 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.624+0000 7fbf30c00700 1 -- 172.21.15.86:0/2334037746 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf280a23e0 con 0x555e31623a20 2024-03-23T17:52:53.627 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.624+0000 7fbf30c00700 1 -- 172.21.15.86:0/2334037746 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf2809a1c0 con 0x555e31623a20 2024-03-23T17:52:53.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.624+0000 7fbf5626eb80 1 -- 172.21.15.86:0/2334037746 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555e31623a20 msgr2=0x555e3162e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:53.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.624+0000 7fbf5626eb80 1 --2- 172.21.15.86:0/2334037746 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555e31623a20 0x555e3162e390 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbf28009230 tx=0x7fbf2800ccd0 comp rx=0 tx=0).stop 2024-03-23T17:52:53.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.624+0000 7fbf5626eb80 1 -- 172.21.15.86:0/2334037746 shutdown_connections 2024-03-23T17:52:53.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.624+0000 7fbf5626eb80 1 --2- 172.21.15.86:0/2334037746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e3162e8d0 0x555e31630cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:53.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.624+0000 7fbf5626eb80 1 --2- 172.21.15.86:0/2334037746 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555e31623100 0x555e316234e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:53.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.624+0000 7fbf5626eb80 1 --2- 172.21.15.86:0/2334037746 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555e31623a20 0x555e3162e390 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:53.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.624+0000 7fbf5626eb80 1 -- 172.21.15.86:0/2334037746 >> 172.21.15.86:0/2334037746 conn(0x555e31583640 msgr2=0x555e3161f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:53.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.624+0000 7fbf5626eb80 1 -- 172.21.15.86:0/2334037746 shutdown_connections 2024-03-23T17:52:53.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.624+0000 7fbf5626eb80 1 -- 172.21.15.86:0/2334037746 wait complete. 2024-03-23T17:52:53.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.625+0000 7fbf5626eb80 1 -- start start 2024-03-23T17:52:53.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.625+0000 7fbf5626eb80 1 Processor -- start 2024-03-23T17:52:53.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.625+0000 7fbf5626eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e31623100 0x555e3162e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:53.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.626+0000 7fbf5626eb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555e31623a20 0x555e3162b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:53.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.626+0000 7fbf5626eb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555e3162e8d0 0x555e3162bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:53.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.626+0000 7fbf5626eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e31511d20 con 0x555e31623100 2024-03-23T17:52:53.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.626+0000 7fbf5626eb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e315119c0 con 0x555e31623a20 2024-03-23T17:52:53.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.626+0000 7fbf5626eb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e313f9640 con 0x555e3162e8d0 2024-03-23T17:52:53.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.626+0000 7fbf5626eb80 10 client.0 _ll_register_callbacks cb 0x555e3151b410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:52:53.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.626+0000 7fbf3b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e31623100 0x555e3162e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:53.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.626+0000 7fbf3b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e31623100 0x555e3162e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40428/0 (socket says 172.21.15.86:40428) 2024-03-23T17:52:53.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.626+0000 7fbf3b400700 1 -- 172.21.15.86:0/1824417083 learned_addr learned my addr 172.21.15.86:0/1824417083 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:53.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.626+0000 7fbf3be00700 1 --2- 172.21.15.86:0/1824417083 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555e3162e8d0 0x555e3162bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:53.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.626+0000 7fbf3aa00700 1 --2- 172.21.15.86:0/1824417083 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555e31623a20 0x555e3162b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:53.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.626+0000 7fbf3b400700 1 -- 172.21.15.86:0/1824417083 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555e3162e8d0 msgr2=0x555e3162bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:53.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.626+0000 7fbf3b400700 1 --2- 172.21.15.86:0/1824417083 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555e3162e8d0 0x555e3162bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:53.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.626+0000 7fbf3b400700 1 -- 172.21.15.86:0/1824417083 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555e31623a20 msgr2=0x555e3162b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:53.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.626+0000 7fbf3b400700 1 --2- 172.21.15.86:0/1824417083 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555e31623a20 0x555e3162b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:53.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.626+0000 7fbf3b400700 1 -- 172.21.15.86:0/1824417083 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf2c0098e0 con 0x555e31623100 2024-03-23T17:52:53.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.626+0000 7fbf3b400700 1 --2- 172.21.15.86:0/1824417083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e31623100 0x555e3162e390 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c012980 tx=0x7fbf2c012c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98f1328c2298ea8a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:53.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.626+0000 7fbf25600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-23T17:52:53.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.627+0000 7fbf25600700 1 -- 172.21.15.86:0/1824417083 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf2c017070 con 0x555e31623100 2024-03-23T17:52:53.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.627+0000 7fbf25600700 1 -- 172.21.15.86:0/1824417083 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fbf2c00f640 con 0x555e31623100 2024-03-23T17:52:53.634 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.627+0000 7fbf5626eb80 1 -- 172.21.15.86:0/1824417083 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555e31536320 con 0x555e31623100 2024-03-23T17:52:53.634 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.627+0000 7fbf25600700 1 -- 172.21.15.86:0/1824417083 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf2c0122e0 con 0x555e31623100 2024-03-23T17:52:53.634 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.627+0000 7fbf5626eb80 10 client.16161 fetch_fsmap learned FSMap version 33 2024-03-23T17:52:53.634 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.627+0000 7fbf5626eb80 1 -- 172.21.15.86:0/1824417083 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7fbf3c1a6570 con 0x555e31623100 2024-03-23T17:52:53.634 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.627+0000 7fbf25600700 1 -- 172.21.15.86:0/1824417083 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555e31536320 con 0x555e31623100 2024-03-23T17:52:53.634 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.627+0000 7fbf25600700 1 -- 172.21.15.86:0/1824417083 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 33) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fbf2c010a00 con 0x555e31623100 2024-03-23T17:52:53.634 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.627+0000 7fbf5626eb80 10 client.16161 fetch_fsmap finished waiting for FSMap version 33 2024-03-23T17:52:53.634 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.627+0000 7fbf5626eb80 10 client.16161 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:52:53.634 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.628+0000 7fbf5626eb80 20 client.16161 populate_metadata read hostname 'smithi086' 2024-03-23T17:52:53.634 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.628+0000 7fbf5626eb80 1 --2- 172.21.15.86:0/1824417083 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x555e3163e690 0x555e31640ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:53.634 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.628+0000 7fbf5626eb80 4 client.16161 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:52:53.634 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.628+0000 7fbf5626eb80 1 -- 172.21.15.86:0/1824417083 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555e3156e6a0 con 0x555e3163e690 2024-03-23T17:52:53.634 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.628+0000 7fbf3aa00700 1 --2- 172.21.15.86:0/1824417083 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x555e3163e690 0x555e31640ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:53.635 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.628+0000 7fbf3aa00700 1 --2- 172.21.15.86:0/1824417083 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x555e3163e690 0x555e31640ab0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4db96b2fd973e233 server_cookie=c0d91ce7d89027b8 in_seq=0 out_seq=0 2024-03-23T17:52:53.635 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.628+0000 7fbf25600700 10 client.16161 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:52:53.635 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.629+0000 7fbf25600700 1 -- 172.21.15.86:0/1824417083 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x555e3156e6a0 con 0x555e3163e690 2024-03-23T17:52:53.635 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.629+0000 7fbf25600700 10 client.16161 handle_command_reply: tid=0 2024-03-23T17:52:53.635 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.629+0000 7fbf5626eb80 1 client.16161 shutdown 2024-03-23T17:52:53.635 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.629+0000 7fbf5626eb80 1 -- 172.21.15.86:0/1824417083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e31623100 msgr2=0x555e3162e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:53.635 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.629+0000 7fbf5626eb80 1 --2- 172.21.15.86:0/1824417083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e31623100 0x555e3162e390 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c012980 tx=0x7fbf2c012c90 comp rx=0 tx=0).stop 2024-03-23T17:52:53.635 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.629+0000 7fbf5626eb80 1 -- 172.21.15.86:0/1824417083 shutdown_connections 2024-03-23T17:52:53.635 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.629+0000 7fbf5626eb80 1 --2- 172.21.15.86:0/1824417083 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x555e3163e690 0x555e31640ab0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:53.635 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.629+0000 7fbf5626eb80 1 --2- 172.21.15.86:0/1824417083 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555e3162e8d0 0x555e3162bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:53.635 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.629+0000 7fbf25600700 0 client.16161 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:52:53.635 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.629+0000 7fbf5626eb80 1 --2- 172.21.15.86:0/1824417083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e31623100 0x555e3162e390 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:53.635 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.629+0000 7fbf5626eb80 1 --2- 172.21.15.86:0/1824417083 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555e31623a20 0x555e3162b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:53.635 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.629+0000 7fbf5626eb80 1 -- 172.21.15.86:0/1824417083 >> 172.21.15.86:0/1824417083 conn(0x555e31583640 msgr2=0x555e316250a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.629+0000 7fbf5626eb80 1 -- 172.21.15.86:0/1824417083 shutdown_connections 2024-03-23T17:52:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.629+0000 7fbf5626eb80 1 -- 172.21.15.86:0/1824417083 wait complete. 2024-03-23T17:52:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.630+0000 7fbf5626eb80 20 client.16161 trim_cache size 0 max 16384 2024-03-23T17:52:53.648 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.644+0000 7fbf5626eb80 1 Processor -- start 2024-03-23T17:52:53.648 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.644+0000 7fbf5626eb80 1 -- start start 2024-03-23T17:52:53.648 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.644+0000 7fbf5626eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e31623100 0x555e31645410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:53.648 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.644+0000 7fbf5626eb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555e31623a20 0x555e31635540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:53.648 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.644+0000 7fbf5626eb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555e3162e8d0 0x555e316257b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:53.648 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.645+0000 7fbf5626eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e31640e40 con 0x555e31623100 2024-03-23T17:52:53.648 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.645+0000 7fbf5626eb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e316208b0 con 0x555e31623a20 2024-03-23T17:52:53.648 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.645+0000 7fbf5626eb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e31624bc0 con 0x555e3162e8d0 2024-03-23T17:52:53.648 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.645+0000 7fbf3b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e31623100 0x555e31645410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:53.648 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.645+0000 7fbf3b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e31623100 0x555e31645410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40444/0 (socket says 172.21.15.86:40444) 2024-03-23T17:52:53.648 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.645+0000 7fbf3b400700 1 -- 172.21.15.86:0/2662585882 learned_addr learned my addr 172.21.15.86:0/2662585882 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:53.649 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.645+0000 7fbf3aa00700 1 --2- 172.21.15.86:0/2662585882 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555e31623a20 0x555e31635540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:53.649 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.645+0000 7fbf3b400700 1 -- 172.21.15.86:0/2662585882 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555e3162e8d0 msgr2=0x555e316257b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:53.649 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.645+0000 7fbf3be00700 1 --2- 172.21.15.86:0/2662585882 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555e3162e8d0 0x555e316257b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:53.649 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.645+0000 7fbf3b400700 1 --2- 172.21.15.86:0/2662585882 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555e3162e8d0 0x555e316257b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:53.649 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.645+0000 7fbf3b400700 1 -- 172.21.15.86:0/2662585882 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555e31623a20 msgr2=0x555e31635540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:53.649 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.645+0000 7fbf3b400700 1 --2- 172.21.15.86:0/2662585882 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555e31623a20 0x555e31635540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:53.649 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.645+0000 7fbf3b400700 1 -- 172.21.15.86:0/2662585882 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e313a3b00 con 0x555e31623100 2024-03-23T17:52:53.649 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.645+0000 7fbf3aa00700 1 --2- 172.21.15.86:0/2662585882 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555e31623a20 0x555e31635540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T17:52:53.649 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.645+0000 7fbf3be00700 1 --2- 172.21.15.86:0/2662585882 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555e3162e8d0 0x555e316257b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T17:52:53.649 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.645+0000 7fbf3b400700 1 --2- 172.21.15.86:0/2662585882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e31623100 0x555e31645410 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c009340 tx=0x7fbf2c009730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=688aac7c2e5b0352 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:53.649 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.646+0000 7fbf31600700 1 -- 172.21.15.86:0/2662585882 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf2c017070 con 0x555e31623100 2024-03-23T17:52:53.649 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.646+0000 7fbf31600700 1 -- 172.21.15.86:0/2662585882 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf2c018040 con 0x555e31623100 2024-03-23T17:52:53.649 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.646+0000 7fbf31600700 1 -- 172.21.15.86:0/2662585882 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf2c00f640 con 0x555e31623100 2024-03-23T17:52:53.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.646+0000 7fbf5626eb80 1 -- 172.21.15.86:0/2662585882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e31623100 msgr2=0x555e31645410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:53.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.646+0000 7fbf5626eb80 1 --2- 172.21.15.86:0/2662585882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e31623100 0x555e31645410 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c009340 tx=0x7fbf2c009730 comp rx=0 tx=0).stop 2024-03-23T17:52:53.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.646+0000 7fbf5626eb80 1 -- 172.21.15.86:0/2662585882 shutdown_connections 2024-03-23T17:52:53.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.646+0000 7fbf5626eb80 1 --2- 172.21.15.86:0/2662585882 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555e3162e8d0 0x555e316257b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:53.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.646+0000 7fbf5626eb80 1 --2- 172.21.15.86:0/2662585882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e31623100 0x555e31645410 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:53.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.646+0000 7fbf5626eb80 1 --2- 172.21.15.86:0/2662585882 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555e31623a20 0x555e31635540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:53.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.646+0000 7fbf5626eb80 1 -- 172.21.15.86:0/2662585882 >> 172.21.15.86:0/2662585882 conn(0x555e31583640 msgr2=0x555e316250a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:53.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.646+0000 7fbf5626eb80 1 -- 172.21.15.86:0/2662585882 shutdown_connections 2024-03-23T17:52:53.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.646+0000 7fbf5626eb80 1 -- 172.21.15.86:0/2662585882 wait complete. 2024-03-23T17:52:53.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.647+0000 7fbf5626eb80 1 -- start start 2024-03-23T17:52:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.647+0000 7fbf5626eb80 1 Processor -- start 2024-03-23T17:52:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.647+0000 7fbf5626eb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555e31623100 0x555e31633710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.647+0000 7fbf5626eb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555e31623a20 0x555e31633c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.647+0000 7fbf5626eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e3162e8d0 0x555e3168f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.647+0000 7fbf5626eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e3168f6f0 con 0x555e3162e8d0 2024-03-23T17:52:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.647+0000 7fbf5626eb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e3168f860 con 0x555e31623a20 2024-03-23T17:52:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.647+0000 7fbf5626eb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e3168fa20 con 0x555e31623100 2024-03-23T17:52:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.647+0000 7fbf5626eb80 10 client.0 _ll_register_callbacks cb 0x555e3151b410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:52:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.648+0000 7fbf3be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e3162e8d0 0x555e3168f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:53.654 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.648+0000 7fbf3be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e3162e8d0 0x555e3168f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40450/0 (socket says 172.21.15.86:40450) 2024-03-23T17:52:53.654 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.648+0000 7fbf3be00700 1 -- 172.21.15.86:0/2042943002 learned_addr learned my addr 172.21.15.86:0/2042943002 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:52:53.654 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.648+0000 7fbf3aa00700 1 --2- 172.21.15.86:0/2042943002 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555e31623a20 0x555e31633c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:53.654 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.648+0000 7fbf3b400700 1 --2- 172.21.15.86:0/2042943002 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555e31623100 0x555e31633710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:53.654 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.648+0000 7fbf3be00700 1 -- 172.21.15.86:0/2042943002 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555e31623100 msgr2=0x555e31633710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:53.654 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.648+0000 7fbf3be00700 1 --2- 172.21.15.86:0/2042943002 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555e31623100 0x555e31633710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:53.654 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.648+0000 7fbf3be00700 1 -- 172.21.15.86:0/2042943002 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555e31623a20 msgr2=0x555e31633c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:52:53.654 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.648+0000 7fbf3be00700 1 --2- 172.21.15.86:0/2042943002 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555e31623a20 0x555e31633c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:53.654 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.648+0000 7fbf3be00700 1 -- 172.21.15.86:0/2042943002 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf3c1a6570 con 0x555e3162e8d0 2024-03-23T17:52:53.654 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.648+0000 7fbf3be00700 1 --2- 172.21.15.86:0/2042943002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e3162e8d0 0x555e3168f1b0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fbf3400b0a0 tx=0x7fbf3400b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d688b939fa58503 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:52:53.654 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.648+0000 7fbf25600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-23T17:52:53.654 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.649+0000 7fbf25600700 1 -- 172.21.15.86:0/2042943002 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf3400e040 con 0x555e3162e8d0 2024-03-23T17:52:53.654 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.649+0000 7fbf5626eb80 1 -- 172.21.15.86:0/2042943002 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbf280a23e0 con 0x555e3162e8d0 2024-03-23T17:52:53.655 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.649+0000 7fbf25600700 1 -- 172.21.15.86:0/2042943002 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fbf3401e660 con 0x555e3162e8d0 2024-03-23T17:52:53.655 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.649+0000 7fbf25600700 1 -- 172.21.15.86:0/2042943002 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf34003e60 con 0x555e3162e8d0 2024-03-23T17:52:53.655 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.649+0000 7fbf5626eb80 10 client.16167 fetch_fsmap learned FSMap version 33 2024-03-23T17:52:53.655 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.649+0000 7fbf5626eb80 1 -- 172.21.15.86:0/2042943002 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x555e3153be40 con 0x555e3162e8d0 2024-03-23T17:52:53.655 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.649+0000 7fbf25600700 1 -- 172.21.15.86:0/2042943002 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf280a23e0 con 0x555e3162e8d0 2024-03-23T17:52:53.655 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.649+0000 7fbf25600700 1 -- 172.21.15.86:0/2042943002 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 33) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fbf34005530 con 0x555e3162e8d0 2024-03-23T17:52:53.655 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.649+0000 7fbf5626eb80 10 client.16167 fetch_fsmap finished waiting for FSMap version 33 2024-03-23T17:52:53.655 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.649+0000 7fbf5626eb80 10 client.16167 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:52:53.655 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.649+0000 7fbf5626eb80 20 client.16167 populate_metadata read hostname 'smithi086' 2024-03-23T17:52:53.655 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.649+0000 7fbf5626eb80 1 --2- 172.21.15.86:0/2042943002 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x555e31691260 0x555e31693680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:52:53.655 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.649+0000 7fbf5626eb80 4 client.16167 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T17:52:53.655 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.649+0000 7fbf5626eb80 1 -- 172.21.15.86:0/2042943002 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555e313c4c90 con 0x555e31691260 2024-03-23T17:52:53.655 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.650+0000 7fbf3b400700 1 --2- 172.21.15.86:0/2042943002 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x555e31691260 0x555e31693680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:52:53.655 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.650+0000 7fbf25600700 10 client.16167 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:52:53.656 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.650+0000 7fbf3b400700 1 --2- 172.21.15.86:0/2042943002 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x555e31691260 0x555e31693680 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=970be48abf6597a9 server_cookie=63f7afcca490a8c in_seq=0 out_seq=0 2024-03-23T17:52:53.681 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.680+0000 7fbf25600700 1 -- 172.21.15.86:0/2042943002 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555e313c4c90 con 0x555e31691260 2024-03-23T17:52:53.681 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.680+0000 7fbf25600700 10 client.16167 handle_command_reply: tid=0 2024-03-23T17:52:53.682 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.680+0000 7fbf5626eb80 1 client.16167 shutdown 2024-03-23T17:52:53.682 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.680+0000 7fbf5626eb80 1 -- 172.21.15.86:0/2042943002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e3162e8d0 msgr2=0x555e3168f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:53.682 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.680+0000 7fbf5626eb80 1 --2- 172.21.15.86:0/2042943002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e3162e8d0 0x555e3168f1b0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fbf3400b0a0 tx=0x7fbf3400b3b0 comp rx=0 tx=0).stop 2024-03-23T17:52:53.682 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.680+0000 7fbf5626eb80 1 -- 172.21.15.86:0/2042943002 shutdown_connections 2024-03-23T17:52:53.682 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.680+0000 7fbf5626eb80 1 --2- 172.21.15.86:0/2042943002 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x555e31691260 0x555e31693680 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:53.682 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.680+0000 7fbf5626eb80 1 --2- 172.21.15.86:0/2042943002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e3162e8d0 0x555e3168f1b0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:53.682 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.680+0000 7fbf5626eb80 1 --2- 172.21.15.86:0/2042943002 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555e31623100 0x555e31633710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:53.682 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.680+0000 7fbf5626eb80 1 --2- 172.21.15.86:0/2042943002 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555e31623a20 0x555e31633c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:53.682 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.680+0000 7fbf25600700 0 client.16167 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:52:53.682 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.680+0000 7fbf5626eb80 1 -- 172.21.15.86:0/2042943002 >> 172.21.15.86:0/2042943002 conn(0x555e31583640 msgr2=0x555e3161ed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:53.682 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.680+0000 7fbf5626eb80 1 -- 172.21.15.86:0/2042943002 shutdown_connections 2024-03-23T17:52:53.682 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.680+0000 7fbf5626eb80 1 -- 172.21.15.86:0/2042943002 wait complete. 2024-03-23T17:52:53.682 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.680+0000 7fbf5626eb80 20 client.16167 trim_cache size 0 max 16384 2024-03-23T17:52:53.683 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-03-23T17:52:53.683 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-03-23T17:52:53.683 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-03-23T17:52:53.683 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-03-23T17:52:53.684 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.681+0000 7fbf42800700 1 -- 172.21.15.86:0/3622729572 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fbf200a1310 msgr2=0x7fbf200a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:53.684 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.681+0000 7fbf42800700 1 --2- 172.21.15.86:0/3622729572 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fbf200a1310 0x7fbf200a37d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbf3400b780 tx=0x7fbf340095c0 comp rx=0 tx=0).stop 2024-03-23T17:52:53.684 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.681+0000 7fbf42800700 1 -- 172.21.15.86:0/3622729572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf3c106030 msgr2=0x7fbf3c1999e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:52:53.684 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.681+0000 7fbf42800700 1 --2- 172.21.15.86:0/3622729572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf3c106030 0x7fbf3c1999e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fbf2800e970 tx=0x7fbf2800ec80 comp rx=0 tx=0).stop 2024-03-23T17:52:53.684 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.681+0000 7fbf42800700 1 -- 172.21.15.86:0/3622729572 shutdown_connections 2024-03-23T17:52:53.684 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.681+0000 7fbf42800700 1 --2- 172.21.15.86:0/3622729572 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fbf200a1310 0x7fbf200a37d0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:53.684 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.681+0000 7fbf42800700 1 --2- 172.21.15.86:0/3622729572 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbf3c19dd80 0x7fbf3c19e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:53.684 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.681+0000 7fbf42800700 1 --2- 172.21.15.86:0/3622729572 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf3c100f10 0x7fbf3c1994a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:53.684 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.681+0000 7fbf42800700 1 --2- 172.21.15.86:0/3622729572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf3c106030 0x7fbf3c1999e0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:52:53.684 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.681+0000 7fbf42800700 1 -- 172.21.15.86:0/3622729572 >> 172.21.15.86:0/3622729572 conn(0x7fbf3c0fc7b0 msgr2=0x7fbf3c10c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:52:53.684 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.681+0000 7fbf42800700 1 -- 172.21.15.86:0/3622729572 shutdown_connections 2024-03-23T17:52:53.685 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:52:53.681+0000 7fbf42800700 1 -- 172.21.15.86:0/3622729572 wait complete. 2024-03-23T17:52:53.884 INFO:tasks.workunit.client.0.smithi086.stdout: 1 57503 4.19 MB/sec warmup 78 sec latency 1074.514 ms 2024-03-23T17:52:54.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:53 smithi086 ceph-mon[38221]: pgmap v70: 97 pgs: 97 active+clean; 35 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 1.9 MiB/s wr, 138 op/s 2024-03-23T17:52:54.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:53 smithi092 ceph-mon[79089]: pgmap v70: 97 pgs: 97 active+clean; 35 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 1.9 MiB/s wr, 138 op/s 2024-03-23T17:52:54.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:53 smithi152 ceph-mon[47949]: pgmap v70: 97 pgs: 97 active+clean; 35 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 1.9 MiB/s wr, 138 op/s 2024-03-23T17:52:54.884 INFO:tasks.workunit.client.0.smithi086.stdout: 1 57503 4.14 MB/sec warmup 79 sec latency 2074.608 ms 2024-03-23T17:52:55.884 INFO:tasks.workunit.client.0.smithi086.stdout: 1 57503 4.09 MB/sec warmup 80 sec latency 3074.714 ms 2024-03-23T17:52:56.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:55 smithi086 ceph-mon[38221]: pgmap v71: 97 pgs: 97 active+clean; 37 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 2.2 MiB/s wr, 197 op/s 2024-03-23T17:52:56.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:55 smithi092 ceph-mon[79089]: pgmap v71: 97 pgs: 97 active+clean; 37 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 2.2 MiB/s wr, 197 op/s 2024-03-23T17:52:56.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:55 smithi152 ceph-mon[47949]: pgmap v71: 97 pgs: 97 active+clean; 37 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 2.2 MiB/s wr, 197 op/s 2024-03-23T17:52:56.884 INFO:tasks.workunit.client.0.smithi086.stdout: 1 59951 4.20 MB/sec warmup 81 sec latency 3098.162 ms 2024-03-23T17:52:57.885 INFO:tasks.workunit.client.0.smithi086.stdout: 1 61119 4.22 MB/sec warmup 82 sec latency 107.324 ms 2024-03-23T17:52:58.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:57 smithi086 ceph-mon[38221]: pgmap v72: 97 pgs: 97 active+clean; 37 MiB data, 578 MiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 2.0 MiB/s wr, 152 op/s 2024-03-23T17:52:58.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:57 smithi092 ceph-mon[79089]: pgmap v72: 97 pgs: 97 active+clean; 37 MiB data, 578 MiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 2.0 MiB/s wr, 152 op/s 2024-03-23T17:52:58.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:57 smithi152 ceph-mon[47949]: pgmap v72: 97 pgs: 97 active+clean; 37 MiB data, 578 MiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 2.0 MiB/s wr, 152 op/s 2024-03-23T17:52:58.885 INFO:tasks.workunit.client.0.smithi086.stdout: 1 61119 4.17 MB/sec warmup 83 sec latency 1107.394 ms 2024-03-23T17:52:59.885 INFO:tasks.workunit.client.0.smithi086.stdout: 1 61119 4.12 MB/sec warmup 84 sec latency 2107.475 ms 2024-03-23T17:53:00.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:52:59 smithi086 ceph-mon[38221]: pgmap v73: 97 pgs: 97 active+clean; 37 MiB data, 578 MiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 2.0 MiB/s wr, 149 op/s 2024-03-23T17:53:00.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:52:59 smithi092 ceph-mon[79089]: pgmap v73: 97 pgs: 97 active+clean; 37 MiB data, 578 MiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 2.0 MiB/s wr, 149 op/s 2024-03-23T17:53:00.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:52:59 smithi152 ceph-mon[47949]: pgmap v73: 97 pgs: 97 active+clean; 37 MiB data, 578 MiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 2.0 MiB/s wr, 149 op/s 2024-03-23T17:53:00.885 INFO:tasks.workunit.client.0.smithi086.stdout: 1 61781 4.13 MB/sec warmup 85 sec latency 2486.626 ms 2024-03-23T17:53:01.885 INFO:tasks.workunit.client.0.smithi086.stdout: 1 64158 4.24 MB/sec warmup 86 sec latency 11.865 ms 2024-03-23T17:53:02.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:53:01 smithi086 ceph-mon[38221]: pgmap v74: 97 pgs: 97 active+clean; 39 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 2.2 MiB/s wr, 193 op/s 2024-03-23T17:53:02.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:53:01 smithi092 ceph-mon[79089]: pgmap v74: 97 pgs: 97 active+clean; 39 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 2.2 MiB/s wr, 193 op/s 2024-03-23T17:53:02.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:53:01 smithi152 ceph-mon[47949]: pgmap v74: 97 pgs: 97 active+clean; 39 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 2.2 MiB/s wr, 193 op/s 2024-03-23T17:53:02.885 INFO:tasks.workunit.client.0.smithi086.stdout: 1 66163 4.28 MB/sec warmup 87 sec latency 11.663 ms 2024-03-23T17:53:03.885 INFO:tasks.workunit.client.0.smithi086.stdout: 1 67820 4.37 MB/sec warmup 88 sec latency 12.574 ms 2024-03-23T17:53:04.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:53:03 smithi086 ceph-mon[38221]: pgmap v75: 97 pgs: 97 active+clean; 39 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 501 KiB/s wr, 104 op/s 2024-03-23T17:53:04.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:53:03 smithi092 ceph-mon[79089]: pgmap v75: 97 pgs: 97 active+clean; 39 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 501 KiB/s wr, 104 op/s 2024-03-23T17:53:04.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:53:03 smithi152 ceph-mon[47949]: pgmap v75: 97 pgs: 97 active+clean; 39 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 501 KiB/s wr, 104 op/s 2024-03-23T17:53:04.885 INFO:tasks.workunit.client.0.smithi086.stdout: 1 68087 4.32 MB/sec warmup 89 sec latency 737.875 ms 2024-03-23T17:53:05.885 INFO:tasks.workunit.client.0.smithi086.stdout: 1 68087 4.27 MB/sec warmup 90 sec latency 1738.038 ms 2024-03-23T17:53:06.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:53:05 smithi086 ceph-mon[38221]: pgmap v76: 97 pgs: 97 active+clean; 43 MiB data, 621 MiB used, 1.0 TiB / 1.0 TiB avail; 891 KiB/s rd, 1.0 MiB/s wr, 208 op/s 2024-03-23T17:53:06.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:53:05 smithi092 ceph-mon[79089]: pgmap v76: 97 pgs: 97 active+clean; 43 MiB data, 621 MiB used, 1.0 TiB / 1.0 TiB avail; 891 KiB/s rd, 1.0 MiB/s wr, 208 op/s 2024-03-23T17:53:06.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:53:05 smithi152 ceph-mon[47949]: pgmap v76: 97 pgs: 97 active+clean; 43 MiB data, 621 MiB used, 1.0 TiB / 1.0 TiB avail; 891 KiB/s rd, 1.0 MiB/s wr, 208 op/s 2024-03-23T17:53:06.838 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.835+0000 7ff4d7000700 1 -- 172.21.15.86:0/916595324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4d0100ef0 msgr2=0x7ff4d01012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:53:06.838 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.835+0000 7ff4d7000700 1 --2- 172.21.15.86:0/916595324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4d0100ef0 0x7ff4d01012d0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7ff4c00098f0 tx=0x7ff4c0009c00 comp rx=0 tx=0).stop 2024-03-23T17:53:06.838 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.835+0000 7ff4d7000700 1 -- 172.21.15.86:0/916595324 shutdown_connections 2024-03-23T17:53:06.838 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.835+0000 7ff4d7000700 1 --2- 172.21.15.86:0/916595324 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4d0106010 0x7ff4d010db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:06.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.835+0000 7ff4d7000700 1 --2- 172.21.15.86:0/916595324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4d0100ef0 0x7ff4d01012d0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:06.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.835+0000 7ff4d7000700 1 --2- 172.21.15.86:0/916595324 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4d01018a0 0x7ff4d01058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:06.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.835+0000 7ff4d7000700 1 -- 172.21.15.86:0/916595324 >> 172.21.15.86:0/916595324 conn(0x7ff4d00fc750 msgr2=0x7ff4d00feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:53:06.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.835+0000 7ff4d7000700 1 -- 172.21.15.86:0/916595324 shutdown_connections 2024-03-23T17:53:06.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.835+0000 7ff4d7000700 1 -- 172.21.15.86:0/916595324 wait complete. 2024-03-23T17:53:06.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.836+0000 7ff4d7000700 1 Processor -- start 2024-03-23T17:53:06.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.836+0000 7ff4d7000700 1 -- start start 2024-03-23T17:53:06.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.837+0000 7ff4d7000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4d0100ef0 0x7ff4d01993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:06.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.837+0000 7ff4d7000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4d0106010 0x7ff4d0199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:06.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.837+0000 7ff4d7000700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4d019dcc0 0x7ff4d019e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:06.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.837+0000 7ff4d7000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d019e740 con 0x7ff4d0106010 2024-03-23T17:53:06.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.837+0000 7ff4d7000700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d019e8b0 con 0x7ff4d0100ef0 2024-03-23T17:53:06.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.837+0000 7ff4d7000700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d019ea70 con 0x7ff4d019dcc0 2024-03-23T17:53:06.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.837+0000 7ff4c7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4d0106010 0x7ff4d0199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:06.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.837+0000 7ff4cfe00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4d0100ef0 0x7ff4d01993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:06.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.837+0000 7ff4c7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4d0106010 0x7ff4d0199920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49888/0 (socket says 172.21.15.86:49888) 2024-03-23T17:53:06.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.837+0000 7ff4cfe00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4d0100ef0 0x7ff4d01993e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:54792/0 (socket says 172.21.15.86:54792) 2024-03-23T17:53:06.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.837+0000 7ff4c7e00700 1 -- 172.21.15.86:0/2819716843 learned_addr learned my addr 172.21.15.86:0/2819716843 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:53:06.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.837+0000 7ff4d5000700 1 --2- 172.21.15.86:0/2819716843 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4d019dcc0 0x7ff4d019e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:06.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.837+0000 7ff4c7e00700 1 -- 172.21.15.86:0/2819716843 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4d019dcc0 msgr2=0x7ff4d019e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:53:06.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.837+0000 7ff4c7e00700 1 --2- 172.21.15.86:0/2819716843 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4d019dcc0 0x7ff4d019e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:06.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.837+0000 7ff4c7e00700 1 -- 172.21.15.86:0/2819716843 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4d0100ef0 msgr2=0x7ff4d01993e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:53:06.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.837+0000 7ff4c7e00700 1 --2- 172.21.15.86:0/2819716843 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4d0100ef0 0x7ff4d01993e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:06.841 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.837+0000 7ff4c7e00700 1 -- 172.21.15.86:0/2819716843 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4c0009520 con 0x7ff4d0106010 2024-03-23T17:53:06.841 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.837+0000 7ff4cfe00700 1 --2- 172.21.15.86:0/2819716843 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4d0100ef0 0x7ff4d01993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:53:06.841 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.838+0000 7ff4c7e00700 1 --2- 172.21.15.86:0/2819716843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4d0106010 0x7ff4d0199920 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7ff4b800e970 tx=0x7ff4b800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:53:06.841 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.838+0000 7ff4cd600700 1 -- 172.21.15.86:0/2819716843 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b8009790 con 0x7ff4d0106010 2024-03-23T17:53:06.841 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.838+0000 7ff4cd600700 1 -- 172.21.15.86:0/2819716843 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff4b800c7e0 con 0x7ff4d0106010 2024-03-23T17:53:06.841 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.838+0000 7ff4d7000700 1 -- 172.21.15.86:0/2819716843 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4d01a5fd0 con 0x7ff4d0106010 2024-03-23T17:53:06.841 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.838+0000 7ff4cd600700 1 -- 172.21.15.86:0/2819716843 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b80054e0 con 0x7ff4d0106010 2024-03-23T17:53:06.841 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.838+0000 7ff4d7000700 1 -- 172.21.15.86:0/2819716843 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4d01a6450 con 0x7ff4d0106010 2024-03-23T17:53:06.842 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.842+0000 7ff4cd600700 1 -- 172.21.15.86:0/2819716843 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7ff4b800c950 con 0x7ff4d0106010 2024-03-23T17:53:06.844 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.843+0000 7ff4eab12b80 1 Processor -- start 2024-03-23T17:53:06.844 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.844+0000 7ff4eab12b80 1 -- start start 2024-03-23T17:53:06.844 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.844+0000 7ff4cd600700 1 --2- 172.21.15.86:0/2819716843 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7ff4b40a1290 0x7ff4b40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:06.844 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.844+0000 7ff4eab12b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583f1078100 0x5583f10784e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:06.845 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.844+0000 7ff4eab12b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5583f1078a20 0x5583f1083390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:06.845 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.844+0000 7ff4eab12b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5583f10838d0 0x5583f1085cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:06.845 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.844+0000 7ff4eab12b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583f0fd1930 con 0x5583f1078100 2024-03-23T17:53:06.845 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.844+0000 7ff4eab12b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583f0fcf2f0 con 0x5583f1078a20 2024-03-23T17:53:06.845 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.844+0000 7ff4eab12b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583f0fcabe0 con 0x5583f10838d0 2024-03-23T17:53:06.845 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.844+0000 7ff4cd600700 1 -- 172.21.15.86:0/2819716843 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7ff4b8014070 con 0x7ff4d0106010 2024-03-23T17:53:06.845 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.844+0000 7ff4d5000700 1 --2- 172.21.15.86:0/2819716843 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7ff4b40a1290 0x7ff4b40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:06.845 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.844+0000 7ff4cfe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583f1078100 0x5583f10784e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:06.845 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.844+0000 7ff4cfe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583f1078100 0x5583f10784e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49894/0 (socket says 172.21.15.86:49894) 2024-03-23T17:53:06.845 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.844+0000 7ff4cfe00700 1 -- 172.21.15.86:0/478810460 learned_addr learned my addr 172.21.15.86:0/478810460 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:53:06.845 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.844+0000 7ff4d5000700 1 --2- 172.21.15.86:0/478810460 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5583f10838d0 0x5583f1085cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:06.850 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.845+0000 7ff4cfe00700 1 -- 172.21.15.86:0/478810460 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5583f10838d0 msgr2=0x5583f1085cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:53:06.850 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.845+0000 7ff4cfe00700 1 --2- 172.21.15.86:0/478810460 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5583f10838d0 0x5583f1085cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:06.850 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.845+0000 7ff4cfe00700 1 -- 172.21.15.86:0/478810460 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5583f1078a20 msgr2=0x5583f1083390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:53:06.850 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.845+0000 7ff4c7e00700 1 --2- 172.21.15.86:0/478810460 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5583f1078a20 0x5583f1083390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:06.850 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.845+0000 7ff4cfe00700 1 --2- 172.21.15.86:0/478810460 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5583f1078a20 0x5583f1083390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:06.850 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.845+0000 7ff4cfe00700 1 -- 172.21.15.86:0/478810460 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5583f0e4c350 con 0x5583f1078100 2024-03-23T17:53:06.850 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.845+0000 7ff4d5000700 1 --2- 172.21.15.86:0/2819716843 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7ff4b40a1290 0x7ff4b40a3750 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8016db0 tx=0x7ff4c8016430 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:53:06.850 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.845+0000 7ff4cfe00700 1 --2- 172.21.15.86:0/478810460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583f1078100 0x5583f10784e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ff4c0009230 tx=0x7ff4c0005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5acaa2bff010ff9 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:53:06.851 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.845+0000 7ff4c5600700 1 -- 172.21.15.86:0/478810460 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c001c050 con 0x5583f1078100 2024-03-23T17:53:06.851 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.845+0000 7ff4c5600700 1 -- 172.21.15.86:0/478810460 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff4c001d040 con 0x5583f1078100 2024-03-23T17:53:06.851 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.845+0000 7ff4c5600700 1 -- 172.21.15.86:0/478810460 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c0005260 con 0x5583f1078100 2024-03-23T17:53:06.851 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.846+0000 7ff4eab12b80 1 -- 172.21.15.86:0/478810460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583f1078100 msgr2=0x5583f10784e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:53:06.851 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.846+0000 7ff4eab12b80 1 --2- 172.21.15.86:0/478810460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583f1078100 0x5583f10784e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ff4c0009230 tx=0x7ff4c0005dc0 comp rx=0 tx=0).stop 2024-03-23T17:53:06.851 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.846+0000 7ff4eab12b80 1 -- 172.21.15.86:0/478810460 shutdown_connections 2024-03-23T17:53:06.851 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.846+0000 7ff4eab12b80 1 --2- 172.21.15.86:0/478810460 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5583f10838d0 0x5583f1085cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:06.851 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.846+0000 7ff4eab12b80 1 --2- 172.21.15.86:0/478810460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583f1078100 0x5583f10784e0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:06.851 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.846+0000 7ff4eab12b80 1 --2- 172.21.15.86:0/478810460 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5583f1078a20 0x5583f1083390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:06.851 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.846+0000 7ff4eab12b80 1 -- 172.21.15.86:0/478810460 >> 172.21.15.86:0/478810460 conn(0x5583f0fd8640 msgr2=0x5583f1074520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:53:06.851 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.846+0000 7ff4eab12b80 1 -- 172.21.15.86:0/478810460 shutdown_connections 2024-03-23T17:53:06.851 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.846+0000 7ff4eab12b80 1 -- 172.21.15.86:0/478810460 wait complete. 2024-03-23T17:53:06.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.847+0000 7ff4eab12b80 1 -- start start 2024-03-23T17:53:06.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.847+0000 7ff4eab12b80 1 Processor -- start 2024-03-23T17:53:06.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.847+0000 7ff4eab12b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5583f1078100 0x5583f1083270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:06.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.847+0000 7ff4eab12b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583f1078a20 0x5583f1080540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:06.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.847+0000 7ff4eab12b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5583f10838d0 0x5583f1080b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:06.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.847+0000 7ff4eab12b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583f0f66d20 con 0x5583f1078a20 2024-03-23T17:53:06.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.847+0000 7ff4eab12b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583f0f669c0 con 0x5583f10838d0 2024-03-23T17:53:06.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.847+0000 7ff4eab12b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583f0e4e640 con 0x5583f1078100 2024-03-23T17:53:06.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.848+0000 7ff4eab12b80 10 client.0 _ll_register_callbacks cb 0x5583f0f70410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:53:06.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.848+0000 7ff4c7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583f1078a20 0x5583f1080540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:06.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.848+0000 7ff4cfe00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5583f1078100 0x5583f1083270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:06.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.848+0000 7ff4c7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583f1078a20 0x5583f1080540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49900/0 (socket says 172.21.15.86:49900) 2024-03-23T17:53:06.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.848+0000 7ff4c7e00700 1 -- 172.21.15.86:0/735725436 learned_addr learned my addr 172.21.15.86:0/735725436 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:53:06.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.848+0000 7ff4d5000700 1 --2- 172.21.15.86:0/735725436 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5583f10838d0 0x5583f1080b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:06.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.848+0000 7ff4c7e00700 1 -- 172.21.15.86:0/735725436 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5583f1078100 msgr2=0x5583f1083270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:53:06.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.848+0000 7ff4c7e00700 1 --2- 172.21.15.86:0/735725436 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5583f1078100 0x5583f1083270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:06.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.848+0000 7ff4c7e00700 1 -- 172.21.15.86:0/735725436 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5583f10838d0 msgr2=0x5583f1080b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:53:06.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.848+0000 7ff4c7e00700 1 --2- 172.21.15.86:0/735725436 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5583f10838d0 0x5583f1080b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:06.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.848+0000 7ff4c7e00700 1 -- 172.21.15.86:0/735725436 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4d010e400 con 0x5583f1078a20 2024-03-23T17:53:06.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.848+0000 7ff4c7e00700 1 --2- 172.21.15.86:0/735725436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583f1078a20 0x5583f1080540 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ff4b8099d20 tx=0x7ff4b800ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9c422f8909b17aa server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:53:06.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.848+0000 7ff4be000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-23T17:53:06.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.849+0000 7ff4be000700 1 -- 172.21.15.86:0/735725436 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b809a480 con 0x5583f1078a20 2024-03-23T17:53:06.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.849+0000 7ff4be000700 1 -- 172.21.15.86:0/735725436 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7ff4b809de50 con 0x5583f1078a20 2024-03-23T17:53:06.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.849+0000 7ff4eab12b80 1 -- 172.21.15.86:0/735725436 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5583f0f8b320 con 0x5583f1078a20 2024-03-23T17:53:06.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.849+0000 7ff4be000700 1 -- 172.21.15.86:0/735725436 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff4b80a24a0 con 0x5583f1078a20 2024-03-23T17:53:06.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.849+0000 7ff4eab12b80 10 client.16179 fetch_fsmap learned FSMap version 33 2024-03-23T17:53:06.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.849+0000 7ff4eab12b80 1 -- 172.21.15.86:0/735725436 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5583f0df8b00 con 0x5583f1078a20 2024-03-23T17:53:06.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.849+0000 7ff4be000700 1 -- 172.21.15.86:0/735725436 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5583f0f8b320 con 0x5583f1078a20 2024-03-23T17:53:06.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.849+0000 7ff4be000700 1 -- 172.21.15.86:0/735725436 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 33) v1 ==== 7478+0+0 (secure 0 0 0) 0x7ff4b80b2bd0 con 0x5583f1078a20 2024-03-23T17:53:06.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.849+0000 7ff4eab12b80 10 client.16179 fetch_fsmap finished waiting for FSMap version 33 2024-03-23T17:53:06.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.849+0000 7ff4eab12b80 10 client.16179 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:53:06.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.849+0000 7ff4eab12b80 20 client.16179 populate_metadata read hostname 'smithi086' 2024-03-23T17:53:06.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.849+0000 7ff4eab12b80 1 --2- 172.21.15.86:0/735725436 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5583f1093560 0x5583f1095980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:06.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.849+0000 7ff4eab12b80 4 client.16179 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:53:06.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.849+0000 7ff4eab12b80 1 -- 172.21.15.86:0/735725436 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5583f0fc36a0 con 0x5583f1093560 2024-03-23T17:53:06.855 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.850+0000 7ff4cfe00700 1 --2- 172.21.15.86:0/735725436 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5583f1093560 0x5583f1095980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:06.856 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.850+0000 7ff4be000700 10 client.16179 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:53:06.856 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.850+0000 7ff4cfe00700 1 --2- 172.21.15.86:0/735725436 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5583f1093560 0x5583f1095980 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c410b7633acbb74 server_cookie=8204cfe00bc607a9 in_seq=0 out_seq=0 2024-03-23T17:53:06.856 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.851+0000 7ff4be000700 1 -- 172.21.15.86:0/735725436 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5583f0fc36a0 con 0x5583f1093560 2024-03-23T17:53:06.856 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.851+0000 7ff4be000700 10 client.16179 handle_command_reply: tid=0 2024-03-23T17:53:06.856 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.851+0000 7ff4eab12b80 1 client.16179 shutdown 2024-03-23T17:53:06.856 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.851+0000 7ff4eab12b80 1 -- 172.21.15.86:0/735725436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583f1078a20 msgr2=0x5583f1080540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:53:06.856 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.851+0000 7ff4eab12b80 1 --2- 172.21.15.86:0/735725436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583f1078a20 0x5583f1080540 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ff4b8099d20 tx=0x7ff4b800ccd0 comp rx=0 tx=0).stop 2024-03-23T17:53:06.856 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.852+0000 7ff4eab12b80 1 -- 172.21.15.86:0/735725436 shutdown_connections 2024-03-23T17:53:06.856 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.852+0000 7ff4eab12b80 1 --2- 172.21.15.86:0/735725436 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5583f1093560 0x5583f1095980 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:06.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.852+0000 7ff4eab12b80 1 --2- 172.21.15.86:0/735725436 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5583f10838d0 0x5583f1080b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:06.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.852+0000 7ff4eab12b80 1 --2- 172.21.15.86:0/735725436 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5583f1078100 0x5583f1083270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:06.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.852+0000 7ff4be000700 0 client.16179 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:53:06.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.852+0000 7ff4eab12b80 1 --2- 172.21.15.86:0/735725436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583f1078a20 0x5583f1080540 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:06.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.852+0000 7ff4eab12b80 1 -- 172.21.15.86:0/735725436 >> 172.21.15.86:0/735725436 conn(0x5583f0fd8640 msgr2=0x5583f107a0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:53:06.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.852+0000 7ff4eab12b80 1 -- 172.21.15.86:0/735725436 shutdown_connections 2024-03-23T17:53:06.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.852+0000 7ff4eab12b80 1 -- 172.21.15.86:0/735725436 wait complete. 2024-03-23T17:53:06.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.852+0000 7ff4eab12b80 20 client.16179 trim_cache size 0 max 16384 2024-03-23T17:53:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.867+0000 7ff4eab12b80 1 Processor -- start 2024-03-23T17:53:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.867+0000 7ff4eab12b80 1 -- start start 2024-03-23T17:53:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.867+0000 7ff4eab12b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5583f1078100 0x5583f1080aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.867+0000 7ff4eab12b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5583f1078a20 0x5583f108a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.867+0000 7ff4eab12b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583f10838d0 0x5583f107f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.867+0000 7ff4eab12b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583f1070140 con 0x5583f10838d0 2024-03-23T17:53:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.867+0000 7ff4eab12b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583f107c3a0 con 0x5583f1078a20 2024-03-23T17:53:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.867+0000 7ff4eab12b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583f10751f0 con 0x5583f1078100 2024-03-23T17:53:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.867+0000 7ff4d5000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583f10838d0 0x5583f107f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.867+0000 7ff4d5000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583f10838d0 0x5583f107f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49904/0 (socket says 172.21.15.86:49904) 2024-03-23T17:53:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.867+0000 7ff4d5000700 1 -- 172.21.15.86:0/3301512999 learned_addr learned my addr 172.21.15.86:0/3301512999 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:53:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.867+0000 7ff4cfe00700 1 --2- 172.21.15.86:0/3301512999 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5583f1078100 0x5583f1080aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.867+0000 7ff4c7e00700 1 --2- 172.21.15.86:0/3301512999 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5583f1078a20 0x5583f108a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.868+0000 7ff4d5000700 1 -- 172.21.15.86:0/3301512999 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5583f1078100 msgr2=0x5583f1080aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:53:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.868+0000 7ff4d5000700 1 --2- 172.21.15.86:0/3301512999 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5583f1078100 0x5583f1080aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.868+0000 7ff4d5000700 1 -- 172.21.15.86:0/3301512999 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5583f1078a20 msgr2=0x5583f108a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:53:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.868+0000 7ff4d5000700 1 --2- 172.21.15.86:0/3301512999 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5583f1078a20 0x5583f108a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.868+0000 7ff4d5000700 1 -- 172.21.15.86:0/3301512999 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5583f0f90e40 con 0x5583f10838d0 2024-03-23T17:53:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.868+0000 7ff4cfe00700 1 --2- 172.21.15.86:0/3301512999 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5583f1078100 0x5583f1080aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:53:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.868+0000 7ff4d5000700 1 --2- 172.21.15.86:0/3301512999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583f10838d0 0x5583f107f490 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7ff4c801ae70 tx=0x7ff4c801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d63db3e1810c88e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:53:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.868+0000 7ff4c6000700 1 -- 172.21.15.86:0/3301512999 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c800e040 con 0x5583f10838d0 2024-03-23T17:53:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.868+0000 7ff4c6000700 1 -- 172.21.15.86:0/3301512999 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff4c800f040 con 0x5583f10838d0 2024-03-23T17:53:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.868+0000 7ff4c6000700 1 -- 172.21.15.86:0/3301512999 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c801bb60 con 0x5583f10838d0 2024-03-23T17:53:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.869+0000 7ff4eab12b80 1 -- 172.21.15.86:0/3301512999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583f10838d0 msgr2=0x5583f107f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:53:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.869+0000 7ff4eab12b80 1 --2- 172.21.15.86:0/3301512999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583f10838d0 0x5583f107f490 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7ff4c801ae70 tx=0x7ff4c801d000 comp rx=0 tx=0).stop 2024-03-23T17:53:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.869+0000 7ff4eab12b80 1 -- 172.21.15.86:0/3301512999 shutdown_connections 2024-03-23T17:53:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.869+0000 7ff4eab12b80 1 --2- 172.21.15.86:0/3301512999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583f10838d0 0x5583f107f490 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.869+0000 7ff4eab12b80 1 --2- 172.21.15.86:0/3301512999 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5583f1078100 0x5583f1080aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.869+0000 7ff4eab12b80 1 --2- 172.21.15.86:0/3301512999 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5583f1078a20 0x5583f108a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.869+0000 7ff4eab12b80 1 -- 172.21.15.86:0/3301512999 >> 172.21.15.86:0/3301512999 conn(0x5583f0fd8640 msgr2=0x5583f106d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:53:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.869+0000 7ff4eab12b80 1 -- 172.21.15.86:0/3301512999 shutdown_connections 2024-03-23T17:53:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.869+0000 7ff4eab12b80 1 -- 172.21.15.86:0/3301512999 wait complete. 2024-03-23T17:53:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.869+0000 7ff4eab12b80 1 -- start start 2024-03-23T17:53:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.869+0000 7ff4eab12b80 1 Processor -- start 2024-03-23T17:53:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.870+0000 7ff4eab12b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5583f1078100 0x5583f108a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.870+0000 7ff4eab12b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583f1078a20 0x5583f1088760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.870+0000 7ff4eab12b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5583f10838d0 0x5583f1088ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.870+0000 7ff4eab12b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583f10891e0 con 0x5583f1078a20 2024-03-23T17:53:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.870+0000 7ff4eab12b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583f1089350 con 0x5583f1078100 2024-03-23T17:53:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.870+0000 7ff4eab12b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583f1089510 con 0x5583f10838d0 2024-03-23T17:53:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.870+0000 7ff4eab12b80 10 client.0 _ll_register_callbacks cb 0x5583f0f70410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:53:06.874 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.870+0000 7ff4cfe00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5583f1078100 0x5583f108a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:06.874 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.870+0000 7ff4c7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583f1078a20 0x5583f1088760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:06.874 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.870+0000 7ff4cfe00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5583f1078100 0x5583f108a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:54832/0 (socket says 172.21.15.86:54832) 2024-03-23T17:53:06.874 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.870+0000 7ff4cfe00700 1 -- 172.21.15.86:0/1636405966 learned_addr learned my addr 172.21.15.86:0/1636405966 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:53:06.874 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.870+0000 7ff4d5000700 1 --2- 172.21.15.86:0/1636405966 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5583f10838d0 0x5583f1088ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:06.874 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.871+0000 7ff4cfe00700 1 -- 172.21.15.86:0/1636405966 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5583f10838d0 msgr2=0x5583f1088ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:53:06.874 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.871+0000 7ff4cfe00700 1 --2- 172.21.15.86:0/1636405966 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5583f10838d0 0x5583f1088ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:06.874 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.871+0000 7ff4cfe00700 1 -- 172.21.15.86:0/1636405966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583f1078a20 msgr2=0x5583f1088760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:53:06.874 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.871+0000 7ff4cfe00700 1 --2- 172.21.15.86:0/1636405966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583f1078a20 0x5583f1088760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:06.874 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.871+0000 7ff4cfe00700 1 -- 172.21.15.86:0/1636405966 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5583f0e4c350 con 0x5583f1078100 2024-03-23T17:53:06.874 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.871+0000 7ff4cfe00700 1 --2- 172.21.15.86:0/1636405966 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5583f1078100 0x5583f108a1f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff4c000b970 tx=0x7ff4c000b9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5463fcbb0f3c1c37 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:53:06.875 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.871+0000 7ff4be000700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T17:53:06.876 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.871+0000 7ff4be000700 1 -- 172.21.15.86:0/1636405966 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c0032070 con 0x5583f1078100 2024-03-23T17:53:06.876 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.871+0000 7ff4be000700 1 -- 172.21.15.86:0/1636405966 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7ff4c0034310 con 0x5583f1078100 2024-03-23T17:53:06.876 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.871+0000 7ff4eab12b80 1 -- 172.21.15.86:0/1636405966 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff4c001d040 con 0x5583f1078100 2024-03-23T17:53:06.876 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.871+0000 7ff4be000700 1 -- 172.21.15.86:0/1636405966 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff4c001c040 con 0x5583f1078100 2024-03-23T17:53:06.876 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.871+0000 7ff4eab12b80 10 client.26234 fetch_fsmap learned FSMap version 33 2024-03-23T17:53:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.871+0000 7ff4eab12b80 1 -- 172.21.15.86:0/1636405966 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7ff4c801abe0 con 0x5583f1078100 2024-03-23T17:53:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.872+0000 7ff4be000700 1 -- 172.21.15.86:0/1636405966 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff4c001d040 con 0x5583f1078100 2024-03-23T17:53:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.872+0000 7ff4be000700 1 -- 172.21.15.86:0/1636405966 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 33) v1 ==== 7478+0+0 (secure 0 0 0) 0x7ff4c0009c70 con 0x5583f1078100 2024-03-23T17:53:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.872+0000 7ff4eab12b80 10 client.26234 fetch_fsmap finished waiting for FSMap version 33 2024-03-23T17:53:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.872+0000 7ff4eab12b80 10 client.26234 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:53:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.872+0000 7ff4eab12b80 20 client.26234 populate_metadata read hostname 'smithi086' 2024-03-23T17:53:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.872+0000 7ff4eab12b80 1 --2- 172.21.15.86:0/1636405966 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5583f10e61f0 0x5583f10e8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.872+0000 7ff4eab12b80 4 client.26234 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T17:53:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.872+0000 7ff4eab12b80 1 -- 172.21.15.86:0/1636405966 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5583f0e19c90 con 0x5583f10e61f0 2024-03-23T17:53:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.872+0000 7ff4c7e00700 1 --2- 172.21.15.86:0/1636405966 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5583f10e61f0 0x5583f10e8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.873+0000 7ff4c7e00700 1 --2- 172.21.15.86:0/1636405966 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5583f10e61f0 0x5583f10e8610 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=538b08934b95516e server_cookie=7f9fc0332389b48b in_seq=0 out_seq=0 2024-03-23T17:53:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.873+0000 7ff4be000700 10 client.26234 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:53:06.885 INFO:tasks.workunit.client.0.smithi086.stdout: 1 68211 4.23 MB/sec warmup 91 sec latency 1756.854 ms 2024-03-23T17:53:06.907 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.905+0000 7ff4be000700 1 -- 172.21.15.86:0/1636405966 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5583f0e19c90 con 0x5583f10e61f0 2024-03-23T17:53:06.907 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.905+0000 7ff4be000700 10 client.26234 handle_command_reply: tid=0 2024-03-23T17:53:06.907 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.905+0000 7ff4eab12b80 1 client.26234 shutdown 2024-03-23T17:53:06.907 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.906+0000 7ff4eab12b80 1 -- 172.21.15.86:0/1636405966 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5583f1078100 msgr2=0x5583f108a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:53:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.906+0000 7ff4eab12b80 1 --2- 172.21.15.86:0/1636405966 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5583f1078100 0x5583f108a1f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff4c000b970 tx=0x7ff4c000b9a0 comp rx=0 tx=0).stop 2024-03-23T17:53:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.906+0000 7ff4eab12b80 1 -- 172.21.15.86:0/1636405966 shutdown_connections 2024-03-23T17:53:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.906+0000 7ff4eab12b80 1 --2- 172.21.15.86:0/1636405966 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5583f10e61f0 0x5583f10e8610 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.906+0000 7ff4eab12b80 1 --2- 172.21.15.86:0/1636405966 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5583f10838d0 0x5583f1088ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.906+0000 7ff4eab12b80 1 --2- 172.21.15.86:0/1636405966 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5583f1078100 0x5583f108a1f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.906+0000 7ff4be000700 0 client.26234 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:53:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.906+0000 7ff4eab12b80 1 --2- 172.21.15.86:0/1636405966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583f1078a20 0x5583f1088760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.906+0000 7ff4eab12b80 1 -- 172.21.15.86:0/1636405966 >> 172.21.15.86:0/1636405966 conn(0x5583f0fd8640 msgr2=0x5583f1074450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:53:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.906+0000 7ff4eab12b80 1 -- 172.21.15.86:0/1636405966 shutdown_connections 2024-03-23T17:53:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.906+0000 7ff4eab12b80 1 -- 172.21.15.86:0/1636405966 wait complete. 2024-03-23T17:53:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.906+0000 7ff4eab12b80 20 client.26234 trim_cache size 0 max 16384 2024-03-23T17:53:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-03-23T17:53:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-03-23T17:53:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-03-23T17:53:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-03-23T17:53:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.907+0000 7ff4d7000700 1 -- 172.21.15.86:0/2819716843 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7ff4b40a1290 msgr2=0x7ff4b40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:53:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.907+0000 7ff4d7000700 1 --2- 172.21.15.86:0/2819716843 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7ff4b40a1290 0x7ff4b40a3750 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8016db0 tx=0x7ff4c8016430 comp rx=0 tx=0).stop 2024-03-23T17:53:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.907+0000 7ff4d7000700 1 -- 172.21.15.86:0/2819716843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4d0106010 msgr2=0x7ff4d0199920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:53:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.907+0000 7ff4d7000700 1 --2- 172.21.15.86:0/2819716843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4d0106010 0x7ff4d0199920 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7ff4b800e970 tx=0x7ff4b800ec80 comp rx=0 tx=0).stop 2024-03-23T17:53:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.907+0000 7ff4d7000700 1 -- 172.21.15.86:0/2819716843 shutdown_connections 2024-03-23T17:53:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.907+0000 7ff4d7000700 1 --2- 172.21.15.86:0/2819716843 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7ff4b40a1290 0x7ff4b40a3750 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.907+0000 7ff4d7000700 1 --2- 172.21.15.86:0/2819716843 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4d019dcc0 0x7ff4d019e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.907+0000 7ff4d7000700 1 --2- 172.21.15.86:0/2819716843 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4d0100ef0 0x7ff4d01993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.907+0000 7ff4d7000700 1 --2- 172.21.15.86:0/2819716843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4d0106010 0x7ff4d0199920 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.907+0000 7ff4d7000700 1 -- 172.21.15.86:0/2819716843 >> 172.21.15.86:0/2819716843 conn(0x7ff4d00fc750 msgr2=0x7ff4d00fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:53:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.907+0000 7ff4d7000700 1 -- 172.21.15.86:0/2819716843 shutdown_connections 2024-03-23T17:53:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:06.907+0000 7ff4d7000700 1 -- 172.21.15.86:0/2819716843 wait complete. 2024-03-23T17:53:07.886 INFO:tasks.workunit.client.0.smithi086.stdout: 1 68211 4.18 MB/sec warmup 92 sec latency 1894.868 ms 2024-03-23T17:53:08.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:53:07 smithi086 ceph-mon[38221]: pgmap v77: 97 pgs: 97 active+clean; 43 MiB data, 622 MiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 790 KiB/s wr, 152 op/s 2024-03-23T17:53:08.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:53:07 smithi092 ceph-mon[79089]: pgmap v77: 97 pgs: 97 active+clean; 43 MiB data, 622 MiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 790 KiB/s wr, 152 op/s 2024-03-23T17:53:08.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:53:07 smithi152 ceph-mon[47949]: pgmap v77: 97 pgs: 97 active+clean; 43 MiB data, 622 MiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 790 KiB/s wr, 152 op/s 2024-03-23T17:53:08.886 INFO:tasks.workunit.client.0.smithi086.stdout: 1 68211 4.14 MB/sec warmup 93 sec latency 2895.012 ms 2024-03-23T17:53:09.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:53:08 smithi086 ceph-mon[38221]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T17:53:09.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:53:08 smithi092 ceph-mon[79089]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T17:53:09.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:53:08 smithi152 ceph-mon[47949]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T17:53:09.886 INFO:tasks.workunit.client.0.smithi086.stdout: 1 68211 4.09 MB/sec warmup 94 sec latency 3895.158 ms 2024-03-23T17:53:10.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:53:09 smithi086 ceph-mon[38221]: pgmap v78: 97 pgs: 97 active+clean; 43 MiB data, 622 MiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 771 KiB/s wr, 150 op/s 2024-03-23T17:53:10.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:53:09 smithi092 ceph-mon[79089]: pgmap v78: 97 pgs: 97 active+clean; 43 MiB data, 622 MiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 771 KiB/s wr, 150 op/s 2024-03-23T17:53:10.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:53:09 smithi152 ceph-mon[47949]: pgmap v78: 97 pgs: 97 active+clean; 43 MiB data, 622 MiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 771 KiB/s wr, 150 op/s 2024-03-23T17:53:10.886 INFO:tasks.workunit.client.0.smithi086.stdout: 1 68211 4.05 MB/sec warmup 95 sec latency 4895.317 ms 2024-03-23T17:53:11.886 INFO:tasks.workunit.client.0.smithi086.stdout: 1 70775 4.15 MB/sec warmup 96 sec latency 4910.828 ms 2024-03-23T17:53:12.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:53:11 smithi086 ceph-mon[38221]: pgmap v79: 97 pgs: 97 active+clean; 39 MiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 646 KiB/s rd, 831 KiB/s wr, 177 op/s 2024-03-23T17:53:12.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:53:11 smithi092 ceph-mon[79089]: pgmap v79: 97 pgs: 97 active+clean; 39 MiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 646 KiB/s rd, 831 KiB/s wr, 177 op/s 2024-03-23T17:53:12.088 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:53:11 smithi152 ceph-mon[47949]: pgmap v79: 97 pgs: 97 active+clean; 39 MiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 646 KiB/s rd, 831 KiB/s wr, 177 op/s 2024-03-23T17:53:12.887 INFO:tasks.workunit.client.0.smithi086.stdout: 1 71165 4.16 MB/sec warmup 97 sec latency 809.413 ms 2024-03-23T17:53:13.887 INFO:tasks.workunit.client.0.smithi086.stdout: 1 71629 4.12 MB/sec warmup 98 sec latency 460.581 ms 2024-03-23T17:53:14.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:53:13 smithi086 ceph-mon[38221]: pgmap v80: 97 pgs: 97 active+clean; 39 MiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 600 KiB/s wr, 133 op/s 2024-03-23T17:53:14.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:53:13 smithi092 ceph-mon[79089]: pgmap v80: 97 pgs: 97 active+clean; 39 MiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 600 KiB/s wr, 133 op/s 2024-03-23T17:53:14.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:53:13 smithi152 ceph-mon[47949]: pgmap v80: 97 pgs: 97 active+clean; 39 MiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 600 KiB/s wr, 133 op/s 2024-03-23T17:53:14.886 INFO:tasks.workunit.client.0.smithi086.stdout: 1 71629 4.08 MB/sec warmup 99 sec latency 1460.711 ms 2024-03-23T17:53:15.887 INFO:tasks.workunit.client.0.smithi086.stdout: 1 71629 4.03 MB/sec warmup 100 sec latency 2460.798 ms 2024-03-23T17:53:16.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:53:15 smithi086 ceph-mon[38221]: pgmap v81: 97 pgs: 97 active+clean; 41 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 642 KiB/s rd, 799 KiB/s wr, 168 op/s 2024-03-23T17:53:16.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:53:15 smithi092 ceph-mon[79089]: pgmap v81: 97 pgs: 97 active+clean; 41 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 642 KiB/s rd, 799 KiB/s wr, 168 op/s 2024-03-23T17:53:16.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:53:15 smithi152 ceph-mon[47949]: pgmap v81: 97 pgs: 97 active+clean; 41 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 642 KiB/s rd, 799 KiB/s wr, 168 op/s 2024-03-23T17:53:16.886 INFO:tasks.workunit.client.0.smithi086.stdout: 1 73986 4.12 MB/sec warmup 101 sec latency 2479.879 ms 2024-03-23T17:53:17.890 INFO:tasks.workunit.client.0.smithi086.stdout: 1 75266 4.15 MB/sec warmup 102 sec latency 89.362 ms 2024-03-23T17:53:18.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:53:17 smithi086 ceph-mon[38221]: pgmap v82: 97 pgs: 97 active+clean; 42 MiB data, 621 MiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 276 KiB/s wr, 65 op/s 2024-03-23T17:53:18.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:53:17 smithi092 ceph-mon[79089]: pgmap v82: 97 pgs: 97 active+clean; 42 MiB data, 621 MiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 276 KiB/s wr, 65 op/s 2024-03-23T17:53:18.085 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:53:17 smithi152 ceph-mon[47949]: pgmap v82: 97 pgs: 97 active+clean; 42 MiB data, 621 MiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 276 KiB/s wr, 65 op/s 2024-03-23T17:53:18.887 INFO:tasks.workunit.client.0.smithi086.stdout: 1 75266 4.11 MB/sec warmup 103 sec latency 1089.460 ms 2024-03-23T17:53:19.887 INFO:tasks.workunit.client.0.smithi086.stdout: 1 75266 4.07 MB/sec warmup 104 sec latency 2089.579 ms 2024-03-23T17:53:20.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:53:19 smithi086 ceph-mon[38221]: pgmap v83: 97 pgs: 97 active+clean; 42 MiB data, 621 MiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 266 KiB/s wr, 63 op/s 2024-03-23T17:53:20.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:53:19 smithi092 ceph-mon[79089]: pgmap v83: 97 pgs: 97 active+clean; 42 MiB data, 621 MiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 266 KiB/s wr, 63 op/s 2024-03-23T17:53:20.058 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.058+0000 7f0468400700 1 -- 172.21.15.86:0/838783289 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f046010b2a0 msgr2=0x7f046010d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:53:20.058 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.058+0000 7f0468400700 1 --2- 172.21.15.86:0/838783289 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f046010b2a0 0x7f046010d760 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0458009f90 tx=0x7f045800b330 comp rx=0 tx=0).stop 2024-03-23T17:53:20.058 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.058+0000 7f0468400700 1 -- 172.21.15.86:0/838783289 shutdown_connections 2024-03-23T17:53:20.059 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.058+0000 7f0468400700 1 --2- 172.21.15.86:0/838783289 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f046010b2a0 0x7f046010d760 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:20.059 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.058+0000 7f0468400700 1 --2- 172.21.15.86:0/838783289 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0460101e80 0x7f0460102260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:20.059 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.058+0000 7f0468400700 1 --2- 172.21.15.86:0/838783289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04601027a0 0x7f046010ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:20.059 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.058+0000 7f0468400700 1 -- 172.21.15.86:0/838783289 >> 172.21.15.86:0/838783289 conn(0x7f04600fb680 msgr2=0x7f04600fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:53:20.059 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.058+0000 7f0468400700 1 -- 172.21.15.86:0/838783289 shutdown_connections 2024-03-23T17:53:20.059 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.058+0000 7f0468400700 1 -- 172.21.15.86:0/838783289 wait complete. 2024-03-23T17:53:20.061 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.059+0000 7f0468400700 1 Processor -- start 2024-03-23T17:53:20.061 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.059+0000 7f0468400700 1 -- start start 2024-03-23T17:53:20.061 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.059+0000 7f0468400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0460101e80 0x7f04600ffad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:20.061 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.059+0000 7f0468400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04601027a0 0x7f0460100010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:20.061 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.059+0000 7f0468400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f046010b2a0 0x7f0460100550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:20.061 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.059+0000 7f0468400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04601a5700 con 0x7f046010b2a0 2024-03-23T17:53:20.061 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.059+0000 7f0468400700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04601a5870 con 0x7f04601027a0 2024-03-23T17:53:20.061 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.059+0000 7f0468400700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04601a5a30 con 0x7f0460101e80 2024-03-23T17:53:20.061 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.060+0000 7f0465000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04601027a0 0x7f0460100010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.060+0000 7f0466400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f046010b2a0 0x7f0460100550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.060+0000 7f0465000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04601027a0 0x7f0460100010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:33490/0 (socket says 172.21.15.86:33490) 2024-03-23T17:53:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.060+0000 7f0465000700 1 -- 172.21.15.86:0/1676328859 learned_addr learned my addr 172.21.15.86:0/1676328859 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:53:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.060+0000 7f0465a00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0460101e80 0x7f04600ffad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.060+0000 7f0465000700 1 -- 172.21.15.86:0/1676328859 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0460101e80 msgr2=0x7f04600ffad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:53:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.060+0000 7f0465000700 1 --2- 172.21.15.86:0/1676328859 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0460101e80 0x7f04600ffad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.060+0000 7f0465000700 1 -- 172.21.15.86:0/1676328859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f046010b2a0 msgr2=0x7f0460100550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:53:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.060+0000 7f0465000700 1 --2- 172.21.15.86:0/1676328859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f046010b2a0 0x7f0460100550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.060+0000 7f0465000700 1 -- 172.21.15.86:0/1676328859 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f044c0095f0 con 0x7f04601027a0 2024-03-23T17:53:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.060+0000 7f0466400700 1 --2- 172.21.15.86:0/1676328859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f046010b2a0 0x7f0460100550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:53:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.060+0000 7f0465000700 1 --2- 172.21.15.86:0/1676328859 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04601027a0 0x7f0460100010 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f044c009fd0 tx=0x7f044c00ee70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:53:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.061+0000 7f045e000700 1 -- 172.21.15.86:0/1676328859 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f044c009960 con 0x7f04601027a0 2024-03-23T17:53:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.061+0000 7f0468400700 1 -- 172.21.15.86:0/1676328859 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0458009c40 con 0x7f04601027a0 2024-03-23T17:53:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.061+0000 7f045e000700 1 -- 172.21.15.86:0/1676328859 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f044c0044e0 con 0x7f04601027a0 2024-03-23T17:53:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.061+0000 7f045e000700 1 -- 172.21.15.86:0/1676328859 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f044c003e90 con 0x7f04601027a0 2024-03-23T17:53:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.061+0000 7f0468400700 1 -- 172.21.15.86:0/1676328859 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04601a6090 con 0x7f04601027a0 2024-03-23T17:53:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.064+0000 7f045e000700 1 -- 172.21.15.86:0/1676328859 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f044c004000 con 0x7f04601027a0 2024-03-23T17:53:20.064 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.065+0000 7f047be91b80 1 Processor -- start 2024-03-23T17:53:20.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.065+0000 7f047be91b80 1 -- start start 2024-03-23T17:53:20.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.065+0000 7f045e000700 1 --2- 172.21.15.86:0/1676328859 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f04480a1290 0x7f04480a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:20.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.065+0000 7f045e000700 1 -- 172.21.15.86:0/1676328859 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f044c014070 con 0x7f04601027a0 2024-03-23T17:53:20.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.065+0000 7f047be91b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648a997b100 0x5648a997b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:20.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.065+0000 7f0466400700 1 --2- 172.21.15.86:0/1676328859 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f04480a1290 0x7f04480a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:20.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.065+0000 7f047be91b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5648a997ba20 0x5648a9986390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:20.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.065+0000 7f047be91b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648a99868d0 0x5648a9988cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:20.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.065+0000 7f047be91b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648a98d4930 con 0x5648a99868d0 2024-03-23T17:53:20.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.065+0000 7f047be91b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648a98d22f0 con 0x5648a997ba20 2024-03-23T17:53:20.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.065+0000 7f047be91b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648a98cdbe0 con 0x5648a997b100 2024-03-23T17:53:20.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.066+0000 7f0465000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5648a997ba20 0x5648a9986390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:20.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.066+0000 7f0465000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5648a997ba20 0x5648a9986390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:33494/0 (socket says 172.21.15.86:33494) 2024-03-23T17:53:20.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.066+0000 7f0465000700 1 -- 172.21.15.86:0/2902138538 learned_addr learned my addr 172.21.15.86:0/2902138538 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:53:20.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.066+0000 7f0466400700 1 --2- 172.21.15.86:0/2902138538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648a99868d0 0x5648a9988cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:20.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.066+0000 7f0465a00700 1 --2- 172.21.15.86:0/2902138538 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648a997b100 0x5648a997b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:20.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.066+0000 7f0465000700 1 -- 172.21.15.86:0/2902138538 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648a997b100 msgr2=0x5648a997b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:53:20.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.066+0000 7f0465000700 1 --2- 172.21.15.86:0/2902138538 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648a997b100 0x5648a997b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:20.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.066+0000 7f0465000700 1 -- 172.21.15.86:0/2902138538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648a99868d0 msgr2=0x5648a9988cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:53:20.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.066+0000 7f0465000700 1 --2- 172.21.15.86:0/2902138538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648a99868d0 0x5648a9988cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:20.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.066+0000 7f0465000700 1 -- 172.21.15.86:0/2902138538 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5648a974f350 con 0x5648a997ba20 2024-03-23T17:53:20.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.066+0000 7f0465000700 1 --2- 172.21.15.86:0/2902138538 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5648a997ba20 0x5648a9986390 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f044c096c20 tx=0x7f044c09a0b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5511fce0cb9e2a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:53:20.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.066+0000 7f0466400700 1 --2- 172.21.15.86:0/1676328859 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f04480a1290 0x7f04480a3750 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0458009790 tx=0x7f0458003be0 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:53:20.071 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.067+0000 7f0452a00700 1 -- 172.21.15.86:0/2902138538 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f044c016330 con 0x5648a997ba20 2024-03-23T17:53:20.071 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.067+0000 7f0452a00700 1 -- 172.21.15.86:0/2902138538 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f044c09a8b0 con 0x5648a997ba20 2024-03-23T17:53:20.071 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.067+0000 7f0452a00700 1 -- 172.21.15.86:0/2902138538 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f044c0968c0 con 0x5648a997ba20 2024-03-23T17:53:20.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.067+0000 7f047be91b80 1 -- 172.21.15.86:0/2902138538 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5648a997ba20 msgr2=0x5648a9986390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:53:20.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.067+0000 7f047be91b80 1 --2- 172.21.15.86:0/2902138538 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5648a997ba20 0x5648a9986390 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f044c096c20 tx=0x7f044c09a0b0 comp rx=0 tx=0).stop 2024-03-23T17:53:20.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.067+0000 7f047be91b80 1 -- 172.21.15.86:0/2902138538 shutdown_connections 2024-03-23T17:53:20.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.067+0000 7f047be91b80 1 --2- 172.21.15.86:0/2902138538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648a99868d0 0x5648a9988cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:20.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.067+0000 7f047be91b80 1 --2- 172.21.15.86:0/2902138538 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648a997b100 0x5648a997b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:20.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.067+0000 7f047be91b80 1 --2- 172.21.15.86:0/2902138538 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5648a997ba20 0x5648a9986390 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:20.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.067+0000 7f047be91b80 1 -- 172.21.15.86:0/2902138538 >> 172.21.15.86:0/2902138538 conn(0x5648a98db640 msgr2=0x5648a9977520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:53:20.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.067+0000 7f047be91b80 1 -- 172.21.15.86:0/2902138538 shutdown_connections 2024-03-23T17:53:20.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.067+0000 7f047be91b80 1 -- 172.21.15.86:0/2902138538 wait complete. 2024-03-23T17:53:20.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.068+0000 7f047be91b80 1 -- start start 2024-03-23T17:53:20.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.068+0000 7f047be91b80 1 Processor -- start 2024-03-23T17:53:20.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.068+0000 7f047be91b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5648a997b100 0x5648a9986380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:20.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.068+0000 7f047be91b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648a997ba20 0x5648a9983600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:20.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.068+0000 7f047be91b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648a99868d0 0x5648a9983d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:20.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.068+0000 7f047be91b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648a9869d20 con 0x5648a997ba20 2024-03-23T17:53:20.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.068+0000 7f047be91b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648a98699c0 con 0x5648a997b100 2024-03-23T17:53:20.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.068+0000 7f047be91b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648a9751640 con 0x5648a99868d0 2024-03-23T17:53:20.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.068+0000 7f047be91b80 10 client.0 _ll_register_callbacks cb 0x5648a9873410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:53:20.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.069+0000 7f0465000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648a997ba20 0x5648a9983600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:20.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.069+0000 7f0465000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648a997ba20 0x5648a9983600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33672/0 (socket says 172.21.15.86:33672) 2024-03-23T17:53:20.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.069+0000 7f0465000700 1 -- 172.21.15.86:0/2180275954 learned_addr learned my addr 172.21.15.86:0/2180275954 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:53:20.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.069+0000 7f0466400700 1 --2- 172.21.15.86:0/2180275954 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648a99868d0 0x5648a9983d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:20.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.069+0000 7f0465a00700 1 --2- 172.21.15.86:0/2180275954 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5648a997b100 0x5648a9986380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:20.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.069+0000 7f0465000700 1 -- 172.21.15.86:0/2180275954 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648a99868d0 msgr2=0x5648a9983d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:53:20.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.069+0000 7f0465000700 1 --2- 172.21.15.86:0/2180275954 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648a99868d0 0x5648a9983d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:20.074 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.069+0000 7f0465000700 1 -- 172.21.15.86:0/2180275954 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5648a997b100 msgr2=0x5648a9986380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:53:20.074 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.069+0000 7f0465000700 1 --2- 172.21.15.86:0/2180275954 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5648a997b100 0x5648a9986380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:20.074 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.069+0000 7f0465000700 1 -- 172.21.15.86:0/2180275954 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04601a6090 con 0x5648a997ba20 2024-03-23T17:53:20.074 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.069+0000 7f0465000700 1 --2- 172.21.15.86:0/2180275954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648a997ba20 0x5648a9983600 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f044c0a2100 tx=0x7f044c003930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9277ba289ae0eafd server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:53:20.074 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.069+0000 7f0447400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-23T17:53:20.074 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.069+0000 7f0447400700 1 -- 172.21.15.86:0/2180275954 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f044c09a1b0 con 0x5648a997ba20 2024-03-23T17:53:20.074 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.069+0000 7f0447400700 1 -- 172.21.15.86:0/2180275954 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f044c09dfe0 con 0x5648a997ba20 2024-03-23T17:53:20.074 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.070+0000 7f047be91b80 1 -- 172.21.15.86:0/2180275954 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5648a988e320 con 0x5648a997ba20 2024-03-23T17:53:20.074 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.070+0000 7f0447400700 1 -- 172.21.15.86:0/2180275954 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f044c09a410 con 0x5648a997ba20 2024-03-23T17:53:20.074 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.070+0000 7f047be91b80 10 client.16197 fetch_fsmap learned FSMap version 34 2024-03-23T17:53:20.074 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.070+0000 7f047be91b80 1 -- 172.21.15.86:0/2180275954 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5648a96fbb00 con 0x5648a997ba20 2024-03-23T17:53:20.074 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.070+0000 7f0447400700 1 -- 172.21.15.86:0/2180275954 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5648a988e320 con 0x5648a997ba20 2024-03-23T17:53:20.074 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.070+0000 7f0447400700 1 -- 172.21.15.86:0/2180275954 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 34) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f044c0a21a0 con 0x5648a997ba20 2024-03-23T17:53:20.075 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.070+0000 7f047be91b80 10 client.16197 fetch_fsmap finished waiting for FSMap version 34 2024-03-23T17:53:20.075 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.070+0000 7f047be91b80 10 client.16197 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:53:20.075 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.070+0000 7f047be91b80 20 client.16197 populate_metadata read hostname 'smithi086' 2024-03-23T17:53:20.075 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.070+0000 7f047be91b80 1 --2- 172.21.15.86:0/2180275954 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5648a99962b0 0x5648a99986d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:20.075 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.070+0000 7f047be91b80 4 client.16197 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:53:20.075 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.070+0000 7f047be91b80 1 -- 172.21.15.86:0/2180275954 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5648a98c66a0 con 0x5648a99962b0 2024-03-23T17:53:20.075 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.071+0000 7f0465a00700 1 --2- 172.21.15.86:0/2180275954 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5648a99962b0 0x5648a99986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:20.075 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.071+0000 7f0447400700 10 client.16197 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:53:20.075 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.071+0000 7f0465a00700 1 --2- 172.21.15.86:0/2180275954 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5648a99962b0 0x5648a99986d0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1af81654bf2450e3 server_cookie=27daa5a767f94519 in_seq=0 out_seq=0 2024-03-23T17:53:20.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.072+0000 7f0447400700 1 -- 172.21.15.86:0/2180275954 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5648a98c66a0 con 0x5648a99962b0 2024-03-23T17:53:20.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.072+0000 7f0447400700 10 client.16197 handle_command_reply: tid=0 2024-03-23T17:53:20.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.072+0000 7f047be91b80 1 client.16197 shutdown 2024-03-23T17:53:20.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.072+0000 7f047be91b80 1 -- 172.21.15.86:0/2180275954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648a997ba20 msgr2=0x5648a9983600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:53:20.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.072+0000 7f047be91b80 1 --2- 172.21.15.86:0/2180275954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648a997ba20 0x5648a9983600 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f044c0a2100 tx=0x7f044c003930 comp rx=0 tx=0).stop 2024-03-23T17:53:20.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.072+0000 7f047be91b80 1 -- 172.21.15.86:0/2180275954 shutdown_connections 2024-03-23T17:53:20.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.072+0000 7f047be91b80 1 --2- 172.21.15.86:0/2180275954 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5648a99962b0 0x5648a99986d0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:20.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.072+0000 7f047be91b80 1 --2- 172.21.15.86:0/2180275954 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648a99868d0 0x5648a9983d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:20.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.072+0000 7f047be91b80 1 --2- 172.21.15.86:0/2180275954 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5648a997b100 0x5648a9986380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:20.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.072+0000 7f0447400700 0 client.16197 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:53:20.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.072+0000 7f047be91b80 1 --2- 172.21.15.86:0/2180275954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648a997ba20 0x5648a9983600 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:20.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.072+0000 7f047be91b80 1 -- 172.21.15.86:0/2180275954 >> 172.21.15.86:0/2180275954 conn(0x5648a98db640 msgr2=0x5648a997d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:53:20.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.072+0000 7f047be91b80 1 -- 172.21.15.86:0/2180275954 shutdown_connections 2024-03-23T17:53:20.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.072+0000 7f047be91b80 1 -- 172.21.15.86:0/2180275954 wait complete. 2024-03-23T17:53:20.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.072+0000 7f047be91b80 20 client.16197 trim_cache size 0 max 16384 2024-03-23T17:53:20.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:53:19 smithi152 ceph-mon[47949]: pgmap v83: 97 pgs: 97 active+clean; 42 MiB data, 621 MiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 266 KiB/s wr, 63 op/s 2024-03-23T17:53:20.088 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.087+0000 7f047be91b80 1 Processor -- start 2024-03-23T17:53:20.088 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.087+0000 7f047be91b80 1 -- start start 2024-03-23T17:53:20.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.087+0000 7f047be91b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648a997b100 0x5648a997dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:20.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.087+0000 7f047be91b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5648a997ba20 0x5648a999cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:20.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.087+0000 7f047be91b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648a99868d0 0x5648a998d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:20.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.087+0000 7f047be91b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648a9984550 con 0x5648a99868d0 2024-03-23T17:53:20.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.087+0000 7f047be91b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648a9973400 con 0x5648a997ba20 2024-03-23T17:53:20.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.087+0000 7f047be91b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648a997cbc0 con 0x5648a997b100 2024-03-23T17:53:20.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.087+0000 7f0466400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648a99868d0 0x5648a998d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:20.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.088+0000 7f0466400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648a99868d0 0x5648a998d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33684/0 (socket says 172.21.15.86:33684) 2024-03-23T17:53:20.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.088+0000 7f0466400700 1 -- 172.21.15.86:0/3180615193 learned_addr learned my addr 172.21.15.86:0/3180615193 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:53:20.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.088+0000 7f0465a00700 1 --2- 172.21.15.86:0/3180615193 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648a997b100 0x5648a997dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:20.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.088+0000 7f0465000700 1 --2- 172.21.15.86:0/3180615193 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5648a997ba20 0x5648a999cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:20.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.088+0000 7f0466400700 1 -- 172.21.15.86:0/3180615193 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648a997b100 msgr2=0x5648a997dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:53:20.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.088+0000 7f0466400700 1 --2- 172.21.15.86:0/3180615193 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648a997b100 0x5648a997dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:20.090 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.088+0000 7f0466400700 1 -- 172.21.15.86:0/3180615193 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5648a997ba20 msgr2=0x5648a999cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:53:20.090 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.088+0000 7f0466400700 1 --2- 172.21.15.86:0/3180615193 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5648a997ba20 0x5648a999cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:20.090 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.088+0000 7f0466400700 1 -- 172.21.15.86:0/3180615193 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5648a9893e40 con 0x5648a99868d0 2024-03-23T17:53:20.090 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.088+0000 7f0466400700 1 --2- 172.21.15.86:0/3180615193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648a99868d0 0x5648a998d5a0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f0458003ed0 tx=0x7f0458023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b78ac1ae0f3acfc1 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:53:20.090 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.088+0000 7f0453400700 1 -- 172.21.15.86:0/3180615193 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f045800e040 con 0x5648a99868d0 2024-03-23T17:53:20.090 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.088+0000 7f0453400700 1 -- 172.21.15.86:0/3180615193 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f045800be80 con 0x5648a99868d0 2024-03-23T17:53:20.090 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.088+0000 7f0453400700 1 -- 172.21.15.86:0/3180615193 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f045801fd90 con 0x5648a99868d0 2024-03-23T17:53:20.094 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.089+0000 7f047be91b80 1 -- 172.21.15.86:0/3180615193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648a99868d0 msgr2=0x5648a998d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:53:20.094 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.089+0000 7f047be91b80 1 --2- 172.21.15.86:0/3180615193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648a99868d0 0x5648a998d5a0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f0458003ed0 tx=0x7f0458023000 comp rx=0 tx=0).stop 2024-03-23T17:53:20.094 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.089+0000 7f047be91b80 1 -- 172.21.15.86:0/3180615193 shutdown_connections 2024-03-23T17:53:20.094 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.089+0000 7f047be91b80 1 --2- 172.21.15.86:0/3180615193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648a99868d0 0x5648a998d5a0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:20.094 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.089+0000 7f047be91b80 1 --2- 172.21.15.86:0/3180615193 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648a997b100 0x5648a997dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:20.094 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.089+0000 7f047be91b80 1 --2- 172.21.15.86:0/3180615193 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5648a997ba20 0x5648a999cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:20.094 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.089+0000 7f047be91b80 1 -- 172.21.15.86:0/3180615193 >> 172.21.15.86:0/3180615193 conn(0x5648a98db640 msgr2=0x5648a997d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:53:20.095 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.089+0000 7f047be91b80 1 -- 172.21.15.86:0/3180615193 shutdown_connections 2024-03-23T17:53:20.095 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.089+0000 7f047be91b80 1 -- 172.21.15.86:0/3180615193 wait complete. 2024-03-23T17:53:20.095 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.089+0000 7f047be91b80 1 -- start start 2024-03-23T17:53:20.095 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.090+0000 7f047be91b80 1 Processor -- start 2024-03-23T17:53:20.095 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.090+0000 7f047be91b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648a997b100 0x5648a998d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:20.095 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.090+0000 7f047be91b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5648a997ba20 0x5648a998b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:20.095 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.090+0000 7f047be91b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648a99868d0 0x5648a998bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:20.095 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.090+0000 7f047be91b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648a998c000 con 0x5648a997b100 2024-03-23T17:53:20.095 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.090+0000 7f047be91b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648a998c170 con 0x5648a997ba20 2024-03-23T17:53:20.095 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.090+0000 7f047be91b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648a998c330 con 0x5648a99868d0 2024-03-23T17:53:20.095 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.090+0000 7f047be91b80 10 client.0 _ll_register_callbacks cb 0x5648a9873410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:53:20.095 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.090+0000 7f0465a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648a997b100 0x5648a998d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:20.095 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.090+0000 7f0465000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5648a997ba20 0x5648a998b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:20.096 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.090+0000 7f0465000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5648a997ba20 0x5648a998b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:33524/0 (socket says 172.21.15.86:33524) 2024-03-23T17:53:20.096 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.090+0000 7f0465000700 1 -- 172.21.15.86:0/3045919569 learned_addr learned my addr 172.21.15.86:0/3045919569 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:53:20.096 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.090+0000 7f0465a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648a997b100 0x5648a998d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33692/0 (socket says 172.21.15.86:33692) 2024-03-23T17:53:20.096 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.090+0000 7f0466400700 1 --2- 172.21.15.86:0/3045919569 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648a99868d0 0x5648a998bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:20.096 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.091+0000 7f0465000700 1 -- 172.21.15.86:0/3045919569 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648a99868d0 msgr2=0x5648a998bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:53:20.096 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.091+0000 7f0465000700 1 --2- 172.21.15.86:0/3045919569 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648a99868d0 0x5648a998bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:20.096 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.091+0000 7f0465000700 1 -- 172.21.15.86:0/3045919569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648a997b100 msgr2=0x5648a998d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:53:20.096 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.091+0000 7f0465000700 1 --2- 172.21.15.86:0/3045919569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648a997b100 0x5648a998d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:20.096 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.091+0000 7f0465000700 1 -- 172.21.15.86:0/3045919569 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0458026070 con 0x5648a997ba20 2024-03-23T17:53:20.096 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.091+0000 7f0465000700 1 --2- 172.21.15.86:0/3045919569 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5648a997ba20 0x5648a998b580 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f044c004f10 tx=0x7f044c00c6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1945a8dbf3ca0fc server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:53:20.096 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.091+0000 7f0447400700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T17:53:20.096 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.091+0000 7f0447400700 1 -- 172.21.15.86:0/3045919569 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f044c003680 con 0x5648a997ba20 2024-03-23T17:53:20.096 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.091+0000 7f0447400700 1 -- 172.21.15.86:0/3045919569 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f044c0a2630 con 0x5648a997ba20 2024-03-23T17:53:20.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.091+0000 7f047be91b80 1 -- 172.21.15.86:0/3045919569 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5648a971cc90 con 0x5648a997ba20 2024-03-23T17:53:20.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.091+0000 7f0447400700 1 -- 172.21.15.86:0/3045919569 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f044c016330 con 0x5648a997ba20 2024-03-23T17:53:20.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.091+0000 7f047be91b80 10 client.26252 fetch_fsmap learned FSMap version 34 2024-03-23T17:53:20.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.091+0000 7f047be91b80 1 -- 172.21.15.86:0/3045919569 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5648a9753c70 con 0x5648a997ba20 2024-03-23T17:53:20.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.092+0000 7f0447400700 1 -- 172.21.15.86:0/3045919569 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5648a971cc90 con 0x5648a997ba20 2024-03-23T17:53:20.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.092+0000 7f0447400700 1 -- 172.21.15.86:0/3045919569 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 34) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f044c099110 con 0x5648a997ba20 2024-03-23T17:53:20.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.092+0000 7f047be91b80 10 client.26252 fetch_fsmap finished waiting for FSMap version 34 2024-03-23T17:53:20.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.092+0000 7f047be91b80 10 client.26252 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:53:20.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.092+0000 7f047be91b80 20 client.26252 populate_metadata read hostname 'smithi086' 2024-03-23T17:53:20.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.092+0000 7f047be91b80 1 --2- 172.21.15.86:0/3045919569 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5648a99e8ca0 0x5648a99eb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:20.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.092+0000 7f047be91b80 4 client.26252 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T17:53:20.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.092+0000 7f047be91b80 1 -- 172.21.15.86:0/3045919569 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5648a9751c80 con 0x5648a99e8ca0 2024-03-23T17:53:20.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.092+0000 7f0465a00700 1 --2- 172.21.15.86:0/3045919569 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5648a99e8ca0 0x5648a99eb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:20.098 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.093+0000 7f0447400700 10 client.26252 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:53:20.098 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.093+0000 7f0465a00700 1 --2- 172.21.15.86:0/3045919569 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5648a99e8ca0 0x5648a99eb090 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1981f6d0e0dea73b server_cookie=69344e7a2d48d16c in_seq=0 out_seq=0 2024-03-23T17:53:20.119 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.118+0000 7f0447400700 1 -- 172.21.15.86:0/3045919569 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5648a9751c80 con 0x5648a99e8ca0 2024-03-23T17:53:20.119 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.118+0000 7f0447400700 10 client.26252 handle_command_reply: tid=0 2024-03-23T17:53:20.119 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.118+0000 7f047be91b80 1 client.26252 shutdown 2024-03-23T17:53:20.119 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.118+0000 7f047be91b80 1 -- 172.21.15.86:0/3045919569 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5648a997ba20 msgr2=0x5648a998b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:53:20.119 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.118+0000 7f047be91b80 1 --2- 172.21.15.86:0/3045919569 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5648a997ba20 0x5648a998b580 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f044c004f10 tx=0x7f044c00c6e0 comp rx=0 tx=0).stop 2024-03-23T17:53:20.119 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.119+0000 7f047be91b80 1 -- 172.21.15.86:0/3045919569 shutdown_connections 2024-03-23T17:53:20.119 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.119+0000 7f047be91b80 1 --2- 172.21.15.86:0/3045919569 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5648a99e8ca0 0x5648a99eb090 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:20.120 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.119+0000 7f047be91b80 1 --2- 172.21.15.86:0/3045919569 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648a99868d0 0x5648a998bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:20.120 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.119+0000 7f0447400700 0 client.26252 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:53:20.120 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.119+0000 7f047be91b80 1 --2- 172.21.15.86:0/3045919569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648a997b100 0x5648a998d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:20.120 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.119+0000 7f047be91b80 1 --2- 172.21.15.86:0/3045919569 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5648a997ba20 0x5648a998b580 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:20.120 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.119+0000 7f047be91b80 1 -- 172.21.15.86:0/3045919569 >> 172.21.15.86:0/3045919569 conn(0x5648a98db640 msgr2=0x5648a9977450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:53:20.120 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.119+0000 7f047be91b80 1 -- 172.21.15.86:0/3045919569 shutdown_connections 2024-03-23T17:53:20.120 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.119+0000 7f047be91b80 1 -- 172.21.15.86:0/3045919569 wait complete. 2024-03-23T17:53:20.120 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.119+0000 7f047be91b80 20 client.26252 trim_cache size 0 max 16384 2024-03-23T17:53:20.120 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-03-23T17:53:20.120 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-03-23T17:53:20.121 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-03-23T17:53:20.121 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-03-23T17:53:20.121 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.119+0000 7f0468400700 1 -- 172.21.15.86:0/1676328859 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f04480a1290 msgr2=0x7f04480a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:53:20.121 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.119+0000 7f0468400700 1 --2- 172.21.15.86:0/1676328859 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f04480a1290 0x7f04480a3750 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0458009790 tx=0x7f0458003be0 comp rx=0 tx=0).stop 2024-03-23T17:53:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.119+0000 7f0468400700 1 -- 172.21.15.86:0/1676328859 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04601027a0 msgr2=0x7f0460100010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:53:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.119+0000 7f0468400700 1 --2- 172.21.15.86:0/1676328859 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04601027a0 0x7f0460100010 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f044c009fd0 tx=0x7f044c00ee70 comp rx=0 tx=0).stop 2024-03-23T17:53:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.120+0000 7f0468400700 1 -- 172.21.15.86:0/1676328859 shutdown_connections 2024-03-23T17:53:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.120+0000 7f0468400700 1 --2- 172.21.15.86:0/1676328859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f046010b2a0 0x7f0460100550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.120+0000 7f0468400700 1 --2- 172.21.15.86:0/1676328859 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f04480a1290 0x7f04480a3750 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.120+0000 7f0468400700 1 --2- 172.21.15.86:0/1676328859 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0460101e80 0x7f04600ffad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.120+0000 7f0468400700 1 --2- 172.21.15.86:0/1676328859 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04601027a0 0x7f0460100010 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.120+0000 7f0468400700 1 -- 172.21.15.86:0/1676328859 >> 172.21.15.86:0/1676328859 conn(0x7f04600fb680 msgr2=0x7f04600fc390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:53:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.120+0000 7f0468400700 1 -- 172.21.15.86:0/1676328859 shutdown_connections 2024-03-23T17:53:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:20.120+0000 7f0468400700 1 -- 172.21.15.86:0/1676328859 wait complete. 2024-03-23T17:53:20.887 INFO:tasks.workunit.client.0.smithi086.stdout: 1 75266 4.03 MB/sec warmup 105 sec latency 3089.687 ms 2024-03-23T17:53:21.887 INFO:tasks.workunit.client.0.smithi086.stdout: 1 77566 4.11 MB/sec warmup 106 sec latency 3105.057 ms 2024-03-23T17:53:22.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:53:21 smithi086 ceph-mon[38221]: pgmap v84: 97 pgs: 97 active+clean; 47 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 807 KiB/s wr, 113 op/s 2024-03-23T17:53:22.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:53:21 smithi092 ceph-mon[79089]: pgmap v84: 97 pgs: 97 active+clean; 47 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 807 KiB/s wr, 113 op/s 2024-03-23T17:53:22.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:53:21 smithi152 ceph-mon[47949]: pgmap v84: 97 pgs: 97 active+clean; 47 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 807 KiB/s wr, 113 op/s 2024-03-23T17:53:22.887 INFO:tasks.workunit.client.0.smithi086.stdout: 1 78753 4.13 MB/sec warmup 107 sec latency 123.061 ms 2024-03-23T17:53:23.887 INFO:tasks.workunit.client.0.smithi086.stdout: 1 78753 4.09 MB/sec warmup 108 sec latency 1123.144 ms 2024-03-23T17:53:24.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:53:23 smithi086 ceph-mon[38221]: pgmap v85: 97 pgs: 97 active+clean; 47 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 747 KiB/s wr, 86 op/s 2024-03-23T17:53:24.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:53:23 smithi092 ceph-mon[79089]: pgmap v85: 97 pgs: 97 active+clean; 47 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 747 KiB/s wr, 86 op/s 2024-03-23T17:53:24.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:53:23 smithi152 ceph-mon[47949]: pgmap v85: 97 pgs: 97 active+clean; 47 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 747 KiB/s wr, 86 op/s 2024-03-23T17:53:24.887 INFO:tasks.workunit.client.0.smithi086.stdout: 1 78753 4.05 MB/sec warmup 109 sec latency 2123.280 ms 2024-03-23T17:53:25.888 INFO:tasks.workunit.client.0.smithi086.stdout: 1 78753 4.02 MB/sec warmup 110 sec latency 3123.408 ms 2024-03-23T17:53:26.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:53:25 smithi086 ceph-mon[38221]: pgmap v86: 97 pgs: 97 active+clean; 50 MiB data, 674 MiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 1.2 MiB/s wr, 149 op/s 2024-03-23T17:53:26.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:53:25 smithi092 ceph-mon[79089]: pgmap v86: 97 pgs: 97 active+clean; 50 MiB data, 674 MiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 1.2 MiB/s wr, 149 op/s 2024-03-23T17:53:26.085 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:53:25 smithi152 ceph-mon[47949]: pgmap v86: 97 pgs: 97 active+clean; 50 MiB data, 674 MiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 1.2 MiB/s wr, 149 op/s 2024-03-23T17:53:26.887 INFO:tasks.workunit.client.0.smithi086.stdout: 1 78877 3.98 MB/sec warmup 111 sec latency 3140.710 ms 2024-03-23T17:53:27.888 INFO:tasks.workunit.client.0.smithi086.stdout: 1 78877 3.95 MB/sec warmup 112 sec latency 1886.998 ms 2024-03-23T17:53:28.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:53:27 smithi086 ceph-mon[38221]: pgmap v87: 97 pgs: 97 active+clean; 50 MiB data, 674 MiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 1.0 MiB/s wr, 113 op/s 2024-03-23T17:53:28.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:53:27 smithi092 ceph-mon[79089]: pgmap v87: 97 pgs: 97 active+clean; 50 MiB data, 674 MiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 1.0 MiB/s wr, 113 op/s 2024-03-23T17:53:28.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:53:27 smithi152 ceph-mon[47949]: pgmap v87: 97 pgs: 97 active+clean; 50 MiB data, 674 MiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 1.0 MiB/s wr, 113 op/s 2024-03-23T17:53:28.888 INFO:tasks.workunit.client.0.smithi086.stdout: 1 78877 3.91 MB/sec warmup 113 sec latency 2887.109 ms 2024-03-23T17:53:29.888 INFO:tasks.workunit.client.0.smithi086.stdout: 1 78877 3.88 MB/sec warmup 114 sec latency 3887.219 ms 2024-03-23T17:53:30.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:53:29 smithi086 ceph-mon[38221]: pgmap v88: 97 pgs: 97 active+clean; 50 MiB data, 674 MiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 1.0 MiB/s wr, 112 op/s 2024-03-23T17:53:30.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:53:29 smithi092 ceph-mon[79089]: pgmap v88: 97 pgs: 97 active+clean; 50 MiB data, 674 MiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 1.0 MiB/s wr, 112 op/s 2024-03-23T17:53:30.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:53:29 smithi152 ceph-mon[47949]: pgmap v88: 97 pgs: 97 active+clean; 50 MiB data, 674 MiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 1.0 MiB/s wr, 112 op/s 2024-03-23T17:53:30.888 INFO:tasks.workunit.client.0.smithi086.stdout: 1 78877 3.84 MB/sec warmup 115 sec latency 4887.314 ms 2024-03-23T17:53:31.888 INFO:tasks.workunit.client.0.smithi086.stdout: 1 81252 3.92 MB/sec warmup 116 sec latency 4900.885 ms 2024-03-23T17:53:32.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:53:31 smithi086 ceph-mon[38221]: pgmap v89: 97 pgs: 97 active+clean; 47 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 1.2 MiB/s wr, 134 op/s 2024-03-23T17:53:32.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:53:31 smithi092 ceph-mon[79089]: pgmap v89: 97 pgs: 97 active+clean; 47 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 1.2 MiB/s wr, 134 op/s 2024-03-23T17:53:32.085 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:53:31 smithi152 ceph-mon[47949]: pgmap v89: 97 pgs: 97 active+clean; 47 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 1.2 MiB/s wr, 134 op/s 2024-03-23T17:53:32.888 INFO:tasks.workunit.client.0.smithi086.stdout: 1 82426 3.94 MB/sec warmup 117 sec latency 85.466 ms 2024-03-23T17:53:33.265 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.262+0000 7f140f800700 1 -- 172.21.15.86:0/3643768537 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1408100ef0 msgr2=0x7f14081012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:53:33.265 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.262+0000 7f140f800700 1 --2- 172.21.15.86:0/3643768537 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1408100ef0 0x7f14081012d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f13fc009990 tx=0x7f13fc009ca0 comp rx=0 tx=0).stop 2024-03-23T17:53:33.265 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.262+0000 7f140f800700 1 -- 172.21.15.86:0/3643768537 shutdown_connections 2024-03-23T17:53:33.265 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.262+0000 7f140f800700 1 --2- 172.21.15.86:0/3643768537 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1408106010 0x7f140810db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:33.265 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.262+0000 7f140f800700 1 --2- 172.21.15.86:0/3643768537 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1408100ef0 0x7f14081012d0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:33.266 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.262+0000 7f140f800700 1 --2- 172.21.15.86:0/3643768537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14081018a0 0x7f14081058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:33.266 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.262+0000 7f140f800700 1 -- 172.21.15.86:0/3643768537 >> 172.21.15.86:0/3643768537 conn(0x7f14080fc750 msgr2=0x7f14080feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:53:33.266 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.263+0000 7f140f800700 1 -- 172.21.15.86:0/3643768537 shutdown_connections 2024-03-23T17:53:33.266 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.263+0000 7f140f800700 1 -- 172.21.15.86:0/3643768537 wait complete. 2024-03-23T17:53:33.268 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.264+0000 7f140f800700 1 Processor -- start 2024-03-23T17:53:33.268 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.264+0000 7f140f800700 1 -- start start 2024-03-23T17:53:33.268 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.264+0000 7f140f800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1408100ef0 0x7f1408199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:33.268 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.264+0000 7f140f800700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1408106010 0x7f1408199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:33.268 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.264+0000 7f140f800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f140819dc10 0x7f140819e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:33.268 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.264+0000 7f140f800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f140819e690 con 0x7f1408100ef0 2024-03-23T17:53:33.269 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.264+0000 7f140f800700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f140819e800 con 0x7f140819dc10 2024-03-23T17:53:33.269 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.264+0000 7f140f800700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f140819e9c0 con 0x7f1408106010 2024-03-23T17:53:33.269 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.264+0000 7f140ce00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1408100ef0 0x7f1408199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:33.269 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.264+0000 7f140ce00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1408100ef0 0x7f1408199330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41964/0 (socket says 172.21.15.86:41964) 2024-03-23T17:53:33.269 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.264+0000 7f1407e00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1408106010 0x7f1408199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:33.269 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.264+0000 7f140ce00700 1 -- 172.21.15.86:0/2442105273 learned_addr learned my addr 172.21.15.86:0/2442105273 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:53:33.269 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.264+0000 7f140d800700 1 --2- 172.21.15.86:0/2442105273 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f140819dc10 0x7f140819e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:33.269 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.265+0000 7f140ce00700 1 -- 172.21.15.86:0/2442105273 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1408106010 msgr2=0x7f1408199870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:53:33.269 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.265+0000 7f140ce00700 1 --2- 172.21.15.86:0/2442105273 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1408106010 0x7f1408199870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:33.269 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.265+0000 7f140ce00700 1 -- 172.21.15.86:0/2442105273 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f140819dc10 msgr2=0x7f140819e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:53:33.269 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.265+0000 7f140ce00700 1 --2- 172.21.15.86:0/2442105273 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f140819dc10 0x7f140819e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:33.269 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.265+0000 7f140ce00700 1 -- 172.21.15.86:0/2442105273 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13fc0095f0 con 0x7f1408100ef0 2024-03-23T17:53:33.269 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.265+0000 7f140d800700 1 --2- 172.21.15.86:0/2442105273 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f140819dc10 0x7f140819e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:53:33.269 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.265+0000 7f140ce00700 1 --2- 172.21.15.86:0/2442105273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1408100ef0 0x7f1408199330 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f13fc005770 tx=0x7f13fc0057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:53:33.270 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.265+0000 7f1405600700 1 -- 172.21.15.86:0/2442105273 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13fc01d070 con 0x7f1408100ef0 2024-03-23T17:53:33.270 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.265+0000 7f140f800700 1 -- 172.21.15.86:0/2442105273 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14081a5ec0 con 0x7f1408100ef0 2024-03-23T17:53:33.270 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.265+0000 7f1405600700 1 -- 172.21.15.86:0/2442105273 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f13fc022470 con 0x7f1408100ef0 2024-03-23T17:53:33.270 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.265+0000 7f1405600700 1 -- 172.21.15.86:0/2442105273 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13fc017650 con 0x7f1408100ef0 2024-03-23T17:53:33.270 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.265+0000 7f140f800700 1 -- 172.21.15.86:0/2442105273 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14081a62f0 con 0x7f1408100ef0 2024-03-23T17:53:33.273 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.269+0000 7f1405600700 1 -- 172.21.15.86:0/2442105273 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f13fc00fb00 con 0x7f1408100ef0 2024-03-23T17:53:33.273 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.270+0000 7f14233deb80 1 Processor -- start 2024-03-23T17:53:33.273 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.270+0000 7f1405600700 1 --2- 172.21.15.86:0/2442105273 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f13f00a1290 0x7f13f00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:33.273 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.270+0000 7f14233deb80 1 -- start start 2024-03-23T17:53:33.273 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.270+0000 7f1405600700 1 -- 172.21.15.86:0/2442105273 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f13fc0cdf10 con 0x7f1408100ef0 2024-03-23T17:53:33.274 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.270+0000 7f140d800700 1 --2- 172.21.15.86:0/2442105273 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f13f00a1290 0x7f13f00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:33.274 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.270+0000 7f14233deb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612c6ebb100 0x5612c6ebb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:33.274 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.270+0000 7f14233deb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612c6ebba20 0x5612c6ec6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:33.274 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.270+0000 7f14233deb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5612c6ec68d0 0x5612c6ec8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:33.274 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.270+0000 7f14233deb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612c6e14930 con 0x5612c6ebba20 2024-03-23T17:53:33.274 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.270+0000 7f14233deb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612c6e122f0 con 0x5612c6ebb100 2024-03-23T17:53:33.274 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.270+0000 7f14233deb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612c6e0dbe0 con 0x5612c6ec68d0 2024-03-23T17:53:33.274 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.271+0000 7f1407e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612c6ebba20 0x5612c6ec6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:33.274 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.271+0000 7f1407e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612c6ebba20 0x5612c6ec6390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41972/0 (socket says 172.21.15.86:41972) 2024-03-23T17:53:33.274 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.271+0000 7f1407e00700 1 -- 172.21.15.86:0/1952189325 learned_addr learned my addr 172.21.15.86:0/1952189325 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:53:33.274 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.271+0000 7f140d800700 1 --2- 172.21.15.86:0/1952189325 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5612c6ec68d0 0x5612c6ec8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:33.274 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.271+0000 7f140ce00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612c6ebb100 0x5612c6ebb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:33.274 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.271+0000 7f140d800700 1 --2- 172.21.15.86:0/2442105273 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f13f00a1290 0x7f13f00a3750 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1400003e80 tx=0x7f140000b040 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:53:33.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.271+0000 7f1407e00700 1 -- 172.21.15.86:0/1952189325 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5612c6ec68d0 msgr2=0x5612c6ec8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:53:33.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.271+0000 7f1407e00700 1 --2- 172.21.15.86:0/1952189325 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5612c6ec68d0 0x5612c6ec8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:33.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.271+0000 7f1407e00700 1 -- 172.21.15.86:0/1952189325 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612c6ebb100 msgr2=0x5612c6ebb4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:53:33.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.271+0000 7f1407e00700 1 --2- 172.21.15.86:0/1952189325 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612c6ebb100 0x5612c6ebb4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:33.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.271+0000 7f1407e00700 1 -- 172.21.15.86:0/1952189325 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5612c6c8f350 con 0x5612c6ebba20 2024-03-23T17:53:33.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.271+0000 7f140ce00700 1 --2- 172.21.15.86:0/1952189325 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612c6ebb100 0x5612c6ebb4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:53:33.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.271+0000 7f1407e00700 1 --2- 172.21.15.86:0/1952189325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612c6ebba20 0x5612c6ec6390 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f13f4009cf0 tx=0x7f13f4011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbca92ed71c0cef8 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:53:33.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.272+0000 7f13fa000700 1 -- 172.21.15.86:0/1952189325 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f4011cc0 con 0x5612c6ebba20 2024-03-23T17:53:33.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.272+0000 7f13fa000700 1 -- 172.21.15.86:0/1952189325 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f13f4011e20 con 0x5612c6ebba20 2024-03-23T17:53:33.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.272+0000 7f13fa000700 1 -- 172.21.15.86:0/1952189325 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f4005280 con 0x5612c6ebba20 2024-03-23T17:53:33.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.272+0000 7f14233deb80 1 -- 172.21.15.86:0/1952189325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612c6ebba20 msgr2=0x5612c6ec6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:53:33.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.272+0000 7f14233deb80 1 --2- 172.21.15.86:0/1952189325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612c6ebba20 0x5612c6ec6390 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f13f4009cf0 tx=0x7f13f4011440 comp rx=0 tx=0).stop 2024-03-23T17:53:33.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.272+0000 7f14233deb80 1 -- 172.21.15.86:0/1952189325 shutdown_connections 2024-03-23T17:53:33.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.272+0000 7f14233deb80 1 --2- 172.21.15.86:0/1952189325 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5612c6ec68d0 0x5612c6ec8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:33.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.272+0000 7f14233deb80 1 --2- 172.21.15.86:0/1952189325 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612c6ebb100 0x5612c6ebb4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:33.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.272+0000 7f14233deb80 1 --2- 172.21.15.86:0/1952189325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612c6ebba20 0x5612c6ec6390 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:33.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.272+0000 7f14233deb80 1 -- 172.21.15.86:0/1952189325 >> 172.21.15.86:0/1952189325 conn(0x5612c6e1b640 msgr2=0x5612c6eb7520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:53:33.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.272+0000 7f14233deb80 1 -- 172.21.15.86:0/1952189325 shutdown_connections 2024-03-23T17:53:33.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.272+0000 7f14233deb80 1 -- 172.21.15.86:0/1952189325 wait complete. 2024-03-23T17:53:33.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.273+0000 7f14233deb80 1 -- start start 2024-03-23T17:53:33.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.273+0000 7f14233deb80 1 Processor -- start 2024-03-23T17:53:33.279 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.274+0000 7f14233deb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612c6ebb100 0x5612c6ec6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:33.280 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.274+0000 7f14233deb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612c6ebba20 0x5612c6ec3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:33.280 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.274+0000 7f14233deb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5612c6ec68d0 0x5612c6ec3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:33.280 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.274+0000 7f14233deb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612c6da9d20 con 0x5612c6ebb100 2024-03-23T17:53:33.280 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.274+0000 7f14233deb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612c6da99c0 con 0x5612c6ebba20 2024-03-23T17:53:33.280 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.274+0000 7f14233deb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612c6c91640 con 0x5612c6ec68d0 2024-03-23T17:53:33.280 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.274+0000 7f14233deb80 10 client.0 _ll_register_callbacks cb 0x5612c6db3410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:53:33.280 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.274+0000 7f140ce00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612c6ebb100 0x5612c6ec6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:33.280 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.274+0000 7f140ce00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612c6ebb100 0x5612c6ec6270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41978/0 (socket says 172.21.15.86:41978) 2024-03-23T17:53:33.280 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.274+0000 7f1407e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612c6ebba20 0x5612c6ec3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:33.280 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.274+0000 7f140ce00700 1 -- 172.21.15.86:0/3367790385 learned_addr learned my addr 172.21.15.86:0/3367790385 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:53:33.280 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.275+0000 7f140d800700 1 --2- 172.21.15.86:0/3367790385 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5612c6ec68d0 0x5612c6ec3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:33.280 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.275+0000 7f1407e00700 1 -- 172.21.15.86:0/3367790385 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5612c6ec68d0 msgr2=0x5612c6ec3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:53:33.280 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.275+0000 7f1407e00700 1 --2- 172.21.15.86:0/3367790385 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5612c6ec68d0 0x5612c6ec3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:33.281 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.275+0000 7f1407e00700 1 -- 172.21.15.86:0/3367790385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612c6ebb100 msgr2=0x5612c6ec6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:53:33.281 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.275+0000 7f1407e00700 1 --2- 172.21.15.86:0/3367790385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612c6ebb100 0x5612c6ec6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:33.281 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.275+0000 7f1407e00700 1 -- 172.21.15.86:0/3367790385 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14081a62f0 con 0x5612c6ebba20 2024-03-23T17:53:33.281 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.275+0000 7f1407e00700 1 --2- 172.21.15.86:0/3367790385 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612c6ebba20 0x5612c6ec3600 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f13f4009cc0 tx=0x7f13f40048b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84abeae7d2e8b10c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:53:33.281 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.275+0000 7f13eea00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T17:53:33.281 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.275+0000 7f13eea00700 1 -- 172.21.15.86:0/3367790385 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f4026070 con 0x5612c6ebba20 2024-03-23T17:53:33.281 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.275+0000 7f13eea00700 1 -- 172.21.15.86:0/3367790385 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f13f4004260 con 0x5612c6ebba20 2024-03-23T17:53:33.281 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.275+0000 7f14233deb80 1 -- 172.21.15.86:0/3367790385 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5612c6dce320 con 0x5612c6ebba20 2024-03-23T17:53:33.281 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.275+0000 7f13eea00700 1 -- 172.21.15.86:0/3367790385 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13f4004420 con 0x5612c6ebba20 2024-03-23T17:53:33.281 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.275+0000 7f14233deb80 10 client.26264 fetch_fsmap learned FSMap version 34 2024-03-23T17:53:33.281 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.275+0000 7f14233deb80 1 -- 172.21.15.86:0/3367790385 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x7f13f4009920 con 0x5612c6ebba20 2024-03-23T17:53:33.281 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.276+0000 7f13eea00700 1 -- 172.21.15.86:0/3367790385 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5612c6dce320 con 0x5612c6ebba20 2024-03-23T17:53:33.281 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.276+0000 7f13eea00700 1 -- 172.21.15.86:0/3367790385 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 34) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f13f4029030 con 0x5612c6ebba20 2024-03-23T17:53:33.281 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.276+0000 7f14233deb80 10 client.26264 fetch_fsmap finished waiting for FSMap version 34 2024-03-23T17:53:33.282 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.276+0000 7f14233deb80 10 client.26264 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:53:33.282 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.276+0000 7f14233deb80 20 client.26264 populate_metadata read hostname 'smithi086' 2024-03-23T17:53:33.282 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.276+0000 7f14233deb80 1 --2- 172.21.15.86:0/3367790385 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5612c6ed6390 0x5612c6ed87b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:33.282 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.276+0000 7f14233deb80 4 client.26264 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:53:33.282 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.276+0000 7f14233deb80 1 -- 172.21.15.86:0/3367790385 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5612c6e066a0 con 0x5612c6ed6390 2024-03-23T17:53:33.282 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.276+0000 7f140ce00700 1 --2- 172.21.15.86:0/3367790385 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5612c6ed6390 0x5612c6ed87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:33.282 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.277+0000 7f140ce00700 1 --2- 172.21.15.86:0/3367790385 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5612c6ed6390 0x5612c6ed87b0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b571f45d42ff42d6 server_cookie=6c8fdf3c00d3f2f2 in_seq=0 out_seq=0 2024-03-23T17:53:33.282 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.277+0000 7f13eea00700 10 client.26264 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:53:33.282 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.277+0000 7f13eea00700 1 -- 172.21.15.86:0/3367790385 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5612c6e066a0 con 0x5612c6ed6390 2024-03-23T17:53:33.282 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.277+0000 7f13eea00700 10 client.26264 handle_command_reply: tid=0 2024-03-23T17:53:33.282 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.277+0000 7f14233deb80 1 client.26264 shutdown 2024-03-23T17:53:33.282 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.277+0000 7f14233deb80 1 -- 172.21.15.86:0/3367790385 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612c6ebba20 msgr2=0x5612c6ec3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:53:33.282 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.277+0000 7f14233deb80 1 --2- 172.21.15.86:0/3367790385 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612c6ebba20 0x5612c6ec3600 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f13f4009cc0 tx=0x7f13f40048b0 comp rx=0 tx=0).stop 2024-03-23T17:53:33.283 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.277+0000 7f14233deb80 1 -- 172.21.15.86:0/3367790385 shutdown_connections 2024-03-23T17:53:33.283 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.277+0000 7f14233deb80 1 --2- 172.21.15.86:0/3367790385 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5612c6ed6390 0x5612c6ed87b0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:33.283 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.277+0000 7f14233deb80 1 --2- 172.21.15.86:0/3367790385 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5612c6ec68d0 0x5612c6ec3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:33.283 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.277+0000 7f14233deb80 1 --2- 172.21.15.86:0/3367790385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612c6ebb100 0x5612c6ec6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:33.283 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.277+0000 7f13eea00700 0 client.26264 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:53:33.283 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.277+0000 7f14233deb80 1 --2- 172.21.15.86:0/3367790385 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612c6ebba20 0x5612c6ec3600 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:33.283 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.277+0000 7f14233deb80 1 -- 172.21.15.86:0/3367790385 >> 172.21.15.86:0/3367790385 conn(0x5612c6e1b640 msgr2=0x5612c6ebd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:53:33.283 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.278+0000 7f14233deb80 1 -- 172.21.15.86:0/3367790385 shutdown_connections 2024-03-23T17:53:33.283 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.278+0000 7f14233deb80 1 -- 172.21.15.86:0/3367790385 wait complete. 2024-03-23T17:53:33.283 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.278+0000 7f14233deb80 20 client.26264 trim_cache size 0 max 16384 2024-03-23T17:53:33.292 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.292+0000 7f14233deb80 1 Processor -- start 2024-03-23T17:53:33.292 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.292+0000 7f14233deb80 1 -- start start 2024-03-23T17:53:33.294 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.292+0000 7f14233deb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612c6ebb100 0x5612c6ebe050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:33.295 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.292+0000 7f14233deb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5612c6ebba20 0x5612c6edd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:33.295 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.292+0000 7f14233deb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612c6ec68d0 0x5612c6ecd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:33.295 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.292+0000 7f14233deb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612c6eb3440 con 0x5612c6ec68d0 2024-03-23T17:53:33.295 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.293+0000 7f14233deb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612c6ec41c0 con 0x5612c6ebb100 2024-03-23T17:53:33.295 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.293+0000 7f14233deb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612c6ebf440 con 0x5612c6ebba20 2024-03-23T17:53:33.295 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.293+0000 7f140ce00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612c6ebb100 0x5612c6ebe050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:33.295 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.293+0000 7f140ce00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612c6ebb100 0x5612c6ebe050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:33876/0 (socket says 172.21.15.86:33876) 2024-03-23T17:53:33.295 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.293+0000 7f140ce00700 1 -- 172.21.15.86:0/443474570 learned_addr learned my addr 172.21.15.86:0/443474570 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:53:33.295 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.293+0000 7f140d800700 1 --2- 172.21.15.86:0/443474570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612c6ec68d0 0x5612c6ecd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:33.295 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.293+0000 7f1407e00700 1 --2- 172.21.15.86:0/443474570 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5612c6ebba20 0x5612c6edd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:33.295 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.293+0000 7f140ce00700 1 -- 172.21.15.86:0/443474570 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5612c6ebba20 msgr2=0x5612c6edd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:53:33.295 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.293+0000 7f140ce00700 1 --2- 172.21.15.86:0/443474570 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5612c6ebba20 0x5612c6edd0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:33.296 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.293+0000 7f140ce00700 1 -- 172.21.15.86:0/443474570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612c6ec68d0 msgr2=0x5612c6ecd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:53:33.296 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.293+0000 7f140ce00700 1 --2- 172.21.15.86:0/443474570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612c6ec68d0 0x5612c6ecd610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:33.296 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.293+0000 7f140ce00700 1 -- 172.21.15.86:0/443474570 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5612c6c3bb00 con 0x5612c6ebb100 2024-03-23T17:53:33.296 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.293+0000 7f140d800700 1 --2- 172.21.15.86:0/443474570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612c6ec68d0 0x5612c6ecd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:53:33.296 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.293+0000 7f1407e00700 1 --2- 172.21.15.86:0/443474570 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5612c6ebba20 0x5612c6edd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T17:53:33.296 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.293+0000 7f140ce00700 1 --2- 172.21.15.86:0/443474570 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612c6ebb100 0x5612c6ebe050 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f13fc09ada0 tx=0x7f13fc009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7259813edb39ef5 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:53:33.296 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.293+0000 7f13faa00700 1 -- 172.21.15.86:0/443474570 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13fc0b7070 con 0x5612c6ebb100 2024-03-23T17:53:33.296 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.293+0000 7f13faa00700 1 -- 172.21.15.86:0/443474570 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f13fc0b2070 con 0x5612c6ebb100 2024-03-23T17:53:33.296 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.293+0000 7f13faa00700 1 -- 172.21.15.86:0/443474570 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13fc099270 con 0x5612c6ebb100 2024-03-23T17:53:33.296 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.294+0000 7f14233deb80 1 -- 172.21.15.86:0/443474570 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612c6ebb100 msgr2=0x5612c6ebe050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:53:33.296 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.294+0000 7f14233deb80 1 --2- 172.21.15.86:0/443474570 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612c6ebb100 0x5612c6ebe050 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f13fc09ada0 tx=0x7f13fc009ca0 comp rx=0 tx=0).stop 2024-03-23T17:53:33.296 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.294+0000 7f14233deb80 1 -- 172.21.15.86:0/443474570 shutdown_connections 2024-03-23T17:53:33.296 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.294+0000 7f14233deb80 1 --2- 172.21.15.86:0/443474570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612c6ec68d0 0x5612c6ecd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:33.296 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.294+0000 7f14233deb80 1 --2- 172.21.15.86:0/443474570 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612c6ebb100 0x5612c6ebe050 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:33.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.294+0000 7f14233deb80 1 --2- 172.21.15.86:0/443474570 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5612c6ebba20 0x5612c6edd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:33.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.294+0000 7f14233deb80 1 -- 172.21.15.86:0/443474570 >> 172.21.15.86:0/443474570 conn(0x5612c6e1b640 msgr2=0x5612c6eb6d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:53:33.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.294+0000 7f14233deb80 1 -- 172.21.15.86:0/443474570 shutdown_connections 2024-03-23T17:53:33.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.294+0000 7f14233deb80 1 -- 172.21.15.86:0/443474570 wait complete. 2024-03-23T17:53:33.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.295+0000 7f14233deb80 1 -- start start 2024-03-23T17:53:33.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.295+0000 7f14233deb80 1 Processor -- start 2024-03-23T17:53:33.299 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.295+0000 7f14233deb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612c6ebb100 0x5612c6ecb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:33.300 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.295+0000 7f14233deb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5612c6ebba20 0x5612c6ecb900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:33.300 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.295+0000 7f14233deb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612c6ec68d0 0x5612c6f26e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:33.300 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.295+0000 7f14233deb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612c6f273b0 con 0x5612c6ebb100 2024-03-23T17:53:33.300 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.295+0000 7f14233deb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612c6f274f0 con 0x5612c6ec68d0 2024-03-23T17:53:33.300 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.295+0000 7f14233deb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612c6f27630 con 0x5612c6ebba20 2024-03-23T17:53:33.300 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.295+0000 7f14233deb80 10 client.0 _ll_register_callbacks cb 0x5612c6db3410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:53:33.300 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.295+0000 7f140ce00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612c6ebb100 0x5612c6ecb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:33.300 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.296+0000 7f140ce00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612c6ebb100 0x5612c6ecb3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41984/0 (socket says 172.21.15.86:41984) 2024-03-23T17:53:33.300 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.296+0000 7f140ce00700 1 -- 172.21.15.86:0/2230151618 learned_addr learned my addr 172.21.15.86:0/2230151618 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:53:33.300 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.296+0000 7f140d800700 1 --2- 172.21.15.86:0/2230151618 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612c6ec68d0 0x5612c6f26e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:33.301 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.296+0000 7f1407e00700 1 --2- 172.21.15.86:0/2230151618 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5612c6ebba20 0x5612c6ecb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:33.301 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.296+0000 7f140d800700 1 -- 172.21.15.86:0/2230151618 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5612c6ebba20 msgr2=0x5612c6ecb900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:53:33.301 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.296+0000 7f140d800700 1 --2- 172.21.15.86:0/2230151618 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5612c6ebba20 0x5612c6ecb900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:33.301 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.296+0000 7f140d800700 1 -- 172.21.15.86:0/2230151618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612c6ebb100 msgr2=0x5612c6ecb3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:53:33.301 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.296+0000 7f140d800700 1 --2- 172.21.15.86:0/2230151618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612c6ebb100 0x5612c6ecb3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:33.301 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.296+0000 7f140d800700 1 -- 172.21.15.86:0/2230151618 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14081a5ec0 con 0x5612c6ec68d0 2024-03-23T17:53:33.301 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.296+0000 7f140ce00700 1 --2- 172.21.15.86:0/2230151618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612c6ebb100 0x5612c6ecb3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:53:33.301 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.296+0000 7f140d800700 1 --2- 172.21.15.86:0/2230151618 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612c6ec68d0 0x5612c6f26e70 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x5612c6ebcb90 tx=0x7f1400004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a756f82e60260022 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:53:33.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.296+0000 7f13eea00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T17:53:33.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.296+0000 7f13eea00700 1 -- 172.21.15.86:0/2230151618 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14000048c0 con 0x5612c6ec68d0 2024-03-23T17:53:33.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.297+0000 7f14233deb80 1 -- 172.21.15.86:0/2230151618 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f13f4011e20 con 0x5612c6ec68d0 2024-03-23T17:53:33.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.297+0000 7f13eea00700 1 -- 172.21.15.86:0/2230151618 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f1400021440 con 0x5612c6ec68d0 2024-03-23T17:53:33.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.297+0000 7f13eea00700 1 -- 172.21.15.86:0/2230151618 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1400021600 con 0x5612c6ec68d0 2024-03-23T17:53:33.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.297+0000 7f14233deb80 10 client.26270 fetch_fsmap learned FSMap version 34 2024-03-23T17:53:33.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.297+0000 7f14233deb80 1 -- 172.21.15.86:0/2230151618 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5612c6dd3e40 con 0x5612c6ec68d0 2024-03-23T17:53:33.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.297+0000 7f13eea00700 1 -- 172.21.15.86:0/2230151618 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f13f4011e20 con 0x5612c6ec68d0 2024-03-23T17:53:33.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.297+0000 7f13eea00700 1 -- 172.21.15.86:0/2230151618 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 34) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f140002a990 con 0x5612c6ec68d0 2024-03-23T17:53:33.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.297+0000 7f14233deb80 10 client.26270 fetch_fsmap finished waiting for FSMap version 34 2024-03-23T17:53:33.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.297+0000 7f14233deb80 10 client.26270 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:53:33.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.297+0000 7f14233deb80 20 client.26270 populate_metadata read hostname 'smithi086' 2024-03-23T17:53:33.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.297+0000 7f14233deb80 1 --2- 172.21.15.86:0/2230151618 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5612c6f28ee0 0x5612c6f2b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:33.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.297+0000 7f14233deb80 4 client.26270 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T17:53:33.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.297+0000 7f14233deb80 1 -- 172.21.15.86:0/2230151618 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5612c6c5cc90 con 0x5612c6f28ee0 2024-03-23T17:53:33.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.297+0000 7f140ce00700 1 --2- 172.21.15.86:0/2230151618 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5612c6f28ee0 0x5612c6f2b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:33.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.298+0000 7f13eea00700 10 client.26270 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:53:33.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.298+0000 7f140ce00700 1 --2- 172.21.15.86:0/2230151618 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5612c6f28ee0 0x5612c6f2b300 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5495dc4aa30d2a65 server_cookie=87a769afe17bf391 in_seq=0 out_seq=0 2024-03-23T17:53:33.323 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.323+0000 7f13eea00700 1 -- 172.21.15.86:0/2230151618 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5612c6c5cc90 con 0x5612c6f28ee0 2024-03-23T17:53:33.323 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.323+0000 7f13eea00700 10 client.26270 handle_command_reply: tid=0 2024-03-23T17:53:33.323 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.323+0000 7f14233deb80 1 client.26270 shutdown 2024-03-23T17:53:33.324 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.323+0000 7f14233deb80 1 -- 172.21.15.86:0/2230151618 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612c6ec68d0 msgr2=0x5612c6f26e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:53:33.324 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.323+0000 7f14233deb80 1 --2- 172.21.15.86:0/2230151618 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612c6ec68d0 0x5612c6f26e70 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x5612c6ebcb90 tx=0x7f1400004ad0 comp rx=0 tx=0).stop 2024-03-23T17:53:33.324 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.323+0000 7f14233deb80 1 -- 172.21.15.86:0/2230151618 shutdown_connections 2024-03-23T17:53:33.324 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.323+0000 7f14233deb80 1 --2- 172.21.15.86:0/2230151618 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5612c6f28ee0 0x5612c6f2b300 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:33.324 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.323+0000 7f14233deb80 1 --2- 172.21.15.86:0/2230151618 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612c6ec68d0 0x5612c6f26e70 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:33.324 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.323+0000 7f14233deb80 1 --2- 172.21.15.86:0/2230151618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612c6ebb100 0x5612c6ecb3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:33.324 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.323+0000 7f14233deb80 1 --2- 172.21.15.86:0/2230151618 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5612c6ebba20 0x5612c6ecb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:33.324 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.323+0000 7f13eea00700 0 client.26270 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:53:33.324 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.323+0000 7f14233deb80 1 -- 172.21.15.86:0/2230151618 >> 172.21.15.86:0/2230151618 conn(0x5612c6e1b640 msgr2=0x5612c6eb03e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:53:33.324 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.323+0000 7f14233deb80 1 -- 172.21.15.86:0/2230151618 shutdown_connections 2024-03-23T17:53:33.324 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.323+0000 7f14233deb80 1 -- 172.21.15.86:0/2230151618 wait complete. 2024-03-23T17:53:33.324 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.323+0000 7f14233deb80 20 client.26270 trim_cache size 0 max 16384 2024-03-23T17:53:33.325 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-03-23T17:53:33.325 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-03-23T17:53:33.325 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-03-23T17:53:33.325 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-03-23T17:53:33.326 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.324+0000 7f140f800700 1 -- 172.21.15.86:0/2442105273 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f13f00a1290 msgr2=0x7f13f00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:53:33.326 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.324+0000 7f140f800700 1 --2- 172.21.15.86:0/2442105273 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f13f00a1290 0x7f13f00a3750 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1400003e80 tx=0x7f140000b040 comp rx=0 tx=0).stop 2024-03-23T17:53:33.326 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.324+0000 7f140f800700 1 -- 172.21.15.86:0/2442105273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1408100ef0 msgr2=0x7f1408199330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:53:33.326 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.324+0000 7f140f800700 1 --2- 172.21.15.86:0/2442105273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1408100ef0 0x7f1408199330 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f13fc005770 tx=0x7f13fc0057a0 comp rx=0 tx=0).stop 2024-03-23T17:53:33.326 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.324+0000 7f140f800700 1 -- 172.21.15.86:0/2442105273 shutdown_connections 2024-03-23T17:53:33.326 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.324+0000 7f140f800700 1 --2- 172.21.15.86:0/2442105273 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f140819dc10 0x7f140819e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:33.326 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.324+0000 7f140f800700 1 --2- 172.21.15.86:0/2442105273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1408100ef0 0x7f1408199330 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:33.326 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.324+0000 7f140f800700 1 --2- 172.21.15.86:0/2442105273 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f13f00a1290 0x7f13f00a3750 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:33.326 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.324+0000 7f140f800700 1 --2- 172.21.15.86:0/2442105273 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1408106010 0x7f1408199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:33.327 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.324+0000 7f140f800700 1 -- 172.21.15.86:0/2442105273 >> 172.21.15.86:0/2442105273 conn(0x7f14080fc750 msgr2=0x7f14080fdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:53:33.327 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.324+0000 7f140f800700 1 -- 172.21.15.86:0/2442105273 shutdown_connections 2024-03-23T17:53:33.327 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:33.324+0000 7f140f800700 1 -- 172.21.15.86:0/2442105273 wait complete. 2024-03-23T17:53:33.888 INFO:tasks.workunit.client.0.smithi086.stdout: 1 82426 3.91 MB/sec warmup 118 sec latency 1085.544 ms 2024-03-23T17:53:34.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:53:33 smithi086 ceph-mon[38221]: pgmap v90: 97 pgs: 97 active+clean; 47 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 650 KiB/s wr, 84 op/s 2024-03-23T17:53:34.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:53:33 smithi092 ceph-mon[79089]: pgmap v90: 97 pgs: 97 active+clean; 47 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 650 KiB/s wr, 84 op/s 2024-03-23T17:53:34.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:53:33 smithi152 ceph-mon[47949]: pgmap v90: 97 pgs: 97 active+clean; 47 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 650 KiB/s wr, 84 op/s 2024-03-23T17:53:34.888 INFO:tasks.workunit.client.0.smithi086.stdout: 1 82426 3.88 MB/sec warmup 119 sec latency 2085.662 ms 2024-03-23T17:53:36.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:53:35 smithi086 ceph-mon[38221]: pgmap v91: 97 pgs: 97 active+clean; 44 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 1.0 MiB/s wr, 147 op/s 2024-03-23T17:53:36.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:53:35 smithi092 ceph-mon[79089]: pgmap v91: 97 pgs: 97 active+clean; 44 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 1.0 MiB/s wr, 147 op/s 2024-03-23T17:53:36.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:53:35 smithi152 ceph-mon[47949]: pgmap v91: 97 pgs: 97 active+clean; 44 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 1.0 MiB/s wr, 147 op/s 2024-03-23T17:53:36.889 INFO:tasks.workunit.client.0.smithi086.stdout: 1 84822 12.89 MB/sec execute 1 sec latency 3106.425 ms 2024-03-23T17:53:37.890 INFO:tasks.workunit.client.0.smithi086.stdout: 1 85843 9.41 MB/sec execute 2 sec latency 220.775 ms 2024-03-23T17:53:38.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:53:37 smithi086 ceph-mon[38221]: pgmap v92: 97 pgs: 97 active+clean; 44 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 541 KiB/s wr, 84 op/s 2024-03-23T17:53:38.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:53:37 smithi092 ceph-mon[79089]: pgmap v92: 97 pgs: 97 active+clean; 44 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 541 KiB/s wr, 84 op/s 2024-03-23T17:53:38.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:53:37 smithi152 ceph-mon[47949]: pgmap v92: 97 pgs: 97 active+clean; 44 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 541 KiB/s wr, 84 op/s 2024-03-23T17:53:38.889 INFO:tasks.workunit.client.0.smithi086.stdout: 1 85843 6.27 MB/sec execute 3 sec latency 1220.865 ms 2024-03-23T17:53:39.889 INFO:tasks.workunit.client.0.smithi086.stdout: 1 85843 4.70 MB/sec execute 4 sec latency 2220.992 ms 2024-03-23T17:53:40.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:53:39 smithi086 ceph-mon[38221]: pgmap v93: 97 pgs: 97 active+clean; 44 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 541 KiB/s wr, 84 op/s 2024-03-23T17:53:40.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:53:39 smithi092 ceph-mon[79089]: pgmap v93: 97 pgs: 97 active+clean; 44 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 541 KiB/s wr, 84 op/s 2024-03-23T17:53:40.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:53:39 smithi152 ceph-mon[47949]: pgmap v93: 97 pgs: 97 active+clean; 44 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 541 KiB/s wr, 84 op/s 2024-03-23T17:53:40.889 INFO:tasks.workunit.client.0.smithi086.stdout: 1 85843 3.76 MB/sec execute 5 sec latency 3221.179 ms 2024-03-23T17:53:41.889 INFO:tasks.workunit.client.0.smithi086.stdout: 1 85899 3.16 MB/sec execute 6 sec latency 3236.560 ms 2024-03-23T17:53:42.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:53:41 smithi086 ceph-mon[38221]: pgmap v94: 97 pgs: 97 active+clean; 45 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 1.2 MiB/s wr, 141 op/s 2024-03-23T17:53:42.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:53:41 smithi092 ceph-mon[79089]: pgmap v94: 97 pgs: 97 active+clean; 45 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 1.2 MiB/s wr, 141 op/s 2024-03-23T17:53:42.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:53:41 smithi152 ceph-mon[47949]: pgmap v94: 97 pgs: 97 active+clean; 45 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 1.2 MiB/s wr, 141 op/s 2024-03-23T17:53:42.890 INFO:tasks.workunit.client.0.smithi086.stdout: 1 85899 2.71 MB/sec execute 7 sec latency 1941.234 ms 2024-03-23T17:53:43.890 INFO:tasks.workunit.client.0.smithi086.stdout: 1 85899 2.37 MB/sec execute 8 sec latency 2941.376 ms 2024-03-23T17:53:44.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:53:43 smithi092 ceph-mon[79089]: pgmap v95: 97 pgs: 97 active+clean; 45 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 1.1 MiB/s wr, 119 op/s 2024-03-23T17:53:44.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:53:43 smithi152 ceph-mon[47949]: pgmap v95: 97 pgs: 97 active+clean; 45 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 1.1 MiB/s wr, 119 op/s 2024-03-23T17:53:44.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:53:43 smithi086 ceph-mon[38221]: pgmap v95: 97 pgs: 97 active+clean; 45 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 1.1 MiB/s wr, 119 op/s 2024-03-23T17:53:44.890 INFO:tasks.workunit.client.0.smithi086.stdout: 1 85899 2.11 MB/sec execute 9 sec latency 3941.517 ms 2024-03-23T17:53:45.890 INFO:tasks.workunit.client.0.smithi086.stdout: 1 85899 1.90 MB/sec execute 10 sec latency 4941.613 ms 2024-03-23T17:53:46.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:53:45 smithi092 ceph-mon[79089]: pgmap v96: 97 pgs: 97 active+clean; 45 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 1.1 MiB/s wr, 129 op/s 2024-03-23T17:53:46.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:53:45 smithi152 ceph-mon[47949]: pgmap v96: 97 pgs: 97 active+clean; 45 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 1.1 MiB/s wr, 129 op/s 2024-03-23T17:53:46.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:53:45 smithi086 ceph-mon[38221]: pgmap v96: 97 pgs: 97 active+clean; 45 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 1.1 MiB/s wr, 129 op/s 2024-03-23T17:53:46.495 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.494+0000 7f97f6800700 1 -- 172.21.15.86:0/24502571 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97f0109750 msgr2=0x7f97f0109b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:53:46.495 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.494+0000 7f97f6800700 1 --2- 172.21.15.86:0/24502571 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97f0109750 0x7f97f0109b30 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f97e0009990 tx=0x7f97e0009ca0 comp rx=0 tx=0).stop 2024-03-23T17:53:46.495 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.494+0000 7f97f6800700 1 -- 172.21.15.86:0/24502571 shutdown_connections 2024-03-23T17:53:46.495 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.494+0000 7f97f6800700 1 --2- 172.21.15.86:0/24502571 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f97f0112d00 0x7f97f01150f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:46.496 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.494+0000 7f97f6800700 1 --2- 172.21.15.86:0/24502571 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97f0109750 0x7f97f0109b30 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:46.496 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.494+0000 7f97f6800700 1 --2- 172.21.15.86:0/24502571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97f010a070 0x7f97f01127c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:46.496 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.494+0000 7f97f6800700 1 -- 172.21.15.86:0/24502571 >> 172.21.15.86:0/24502571 conn(0x7f97f006bd80 msgr2=0x7f97f006c190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:53:46.496 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.494+0000 7f97f6800700 1 -- 172.21.15.86:0/24502571 shutdown_connections 2024-03-23T17:53:46.496 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.494+0000 7f97f6800700 1 -- 172.21.15.86:0/24502571 wait complete. 2024-03-23T17:53:46.496 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.495+0000 7f97f6800700 1 Processor -- start 2024-03-23T17:53:46.496 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.496+0000 7f97f6800700 1 -- start start 2024-03-23T17:53:46.499 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.496+0000 7f97f6800700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f97f0109750 0x7f97f006f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:46.499 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.496+0000 7f97f6800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97f010a070 0x7f97f006fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:46.499 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.496+0000 7f97f6800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97f0112d00 0x7f97f00702f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:46.499 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.496+0000 7f97f6800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f01adff0 con 0x7f97f0112d00 2024-03-23T17:53:46.499 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.496+0000 7f97f6800700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f01ae160 con 0x7f97f010a070 2024-03-23T17:53:46.499 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.496+0000 7f97f6800700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f01ae320 con 0x7f97f0109750 2024-03-23T17:53:46.499 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.496+0000 7f97eea00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97f010a070 0x7f97f006fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:46.499 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.496+0000 7f97ef400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f97f0109750 0x7f97f006f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:46.499 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.496+0000 7f97eea00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97f010a070 0x7f97f006fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:35168/0 (socket says 172.21.15.86:35168) 2024-03-23T17:53:46.499 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.496+0000 7f97ef400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f97f0109750 0x7f97f006f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:45064/0 (socket says 172.21.15.86:45064) 2024-03-23T17:53:46.499 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.496+0000 7f97eea00700 1 -- 172.21.15.86:0/2742411932 learned_addr learned my addr 172.21.15.86:0/2742411932 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:53:46.499 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.497+0000 7f97efe00700 1 --2- 172.21.15.86:0/2742411932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97f0112d00 0x7f97f00702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:46.500 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.497+0000 7f97eea00700 1 -- 172.21.15.86:0/2742411932 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f97f0109750 msgr2=0x7f97f006f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:53:46.500 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.497+0000 7f97eea00700 1 --2- 172.21.15.86:0/2742411932 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f97f0109750 0x7f97f006f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:46.500 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.497+0000 7f97eea00700 1 -- 172.21.15.86:0/2742411932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97f0112d00 msgr2=0x7f97f00702f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:53:46.500 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.497+0000 7f97eea00700 1 --2- 172.21.15.86:0/2742411932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97f0112d00 0x7f97f00702f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:46.500 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.497+0000 7f97eea00700 1 -- 172.21.15.86:0/2742411932 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97e8009c40 con 0x7f97f010a070 2024-03-23T17:53:46.500 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.497+0000 7f97efe00700 1 --2- 172.21.15.86:0/2742411932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97f0112d00 0x7f97f00702f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:53:46.500 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.497+0000 7f97eea00700 1 --2- 172.21.15.86:0/2742411932 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97f010a070 0x7f97f006fdb0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f97dc00e9a0 tx=0x7f97dc00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:53:46.500 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.497+0000 7f97e7e00700 1 -- 172.21.15.86:0/2742411932 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97dc009790 con 0x7f97f010a070 2024-03-23T17:53:46.500 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.497+0000 7f97e7e00700 1 -- 172.21.15.86:0/2742411932 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f97dc00cbd0 con 0x7f97f010a070 2024-03-23T17:53:46.500 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.497+0000 7f97f6800700 1 -- 172.21.15.86:0/2742411932 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97e00095f0 con 0x7f97f010a070 2024-03-23T17:53:46.500 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.497+0000 7f97e7e00700 1 -- 172.21.15.86:0/2742411932 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97dc018680 con 0x7f97f010a070 2024-03-23T17:53:46.500 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.498+0000 7f97f6800700 1 -- 172.21.15.86:0/2742411932 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97f01ae980 con 0x7f97f010a070 2024-03-23T17:53:46.502 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.501+0000 7f97e7e00700 1 -- 172.21.15.86:0/2742411932 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f97dc0187e0 con 0x7f97f010a070 2024-03-23T17:53:46.504 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.503+0000 7f980a281b80 1 Processor -- start 2024-03-23T17:53:46.504 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.503+0000 7f980a281b80 1 -- start start 2024-03-23T17:53:46.506 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.504+0000 7f97e7e00700 1 --2- 172.21.15.86:0/2742411932 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f97d40a1290 0x7f97d40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:46.506 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.504+0000 7f980a281b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559656a61100 0x559656a614e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:46.506 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.504+0000 7f980a281b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559656a61a20 0x559656a6c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:46.506 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.504+0000 7f980a281b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559656a6c8d0 0x559656a6ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:46.506 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.504+0000 7f980a281b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596569ba930 con 0x559656a61a20 2024-03-23T17:53:46.506 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.504+0000 7f980a281b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596569b82f0 con 0x559656a61100 2024-03-23T17:53:46.506 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.504+0000 7f980a281b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596569b3be0 con 0x559656a6c8d0 2024-03-23T17:53:46.506 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.504+0000 7f97e7e00700 1 -- 172.21.15.86:0/2742411932 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f97dc014070 con 0x7f97f010a070 2024-03-23T17:53:46.506 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.504+0000 7f97ef400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559656a61100 0x559656a614e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:46.506 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.504+0000 7f97efe00700 1 --2- 172.21.15.86:0/2742411932 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f97d40a1290 0x7f97d40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:46.507 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.504+0000 7f97ef400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559656a61100 0x559656a614e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:35182/0 (socket says 172.21.15.86:35182) 2024-03-23T17:53:46.507 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.504+0000 7f97ef400700 1 -- 172.21.15.86:0/871182648 learned_addr learned my addr 172.21.15.86:0/871182648 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:53:46.507 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.505+0000 7f97efe00700 1 --2- 172.21.15.86:0/871182648 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559656a6c8d0 0x559656a6ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:46.507 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.505+0000 7f97eea00700 1 --2- 172.21.15.86:0/871182648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559656a61a20 0x559656a6c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:46.507 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.505+0000 7f97ef400700 1 -- 172.21.15.86:0/871182648 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559656a6c8d0 msgr2=0x559656a6ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:53:46.507 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.505+0000 7f97ef400700 1 --2- 172.21.15.86:0/871182648 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559656a6c8d0 0x559656a6ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:46.507 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.505+0000 7f97ef400700 1 -- 172.21.15.86:0/871182648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559656a61a20 msgr2=0x559656a6c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:53:46.507 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.505+0000 7f97ef400700 1 --2- 172.21.15.86:0/871182648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559656a61a20 0x559656a6c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:46.507 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.505+0000 7f97ef400700 1 -- 172.21.15.86:0/871182648 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559656835350 con 0x559656a61100 2024-03-23T17:53:46.508 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.505+0000 7f97ef400700 1 --2- 172.21.15.86:0/871182648 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559656a61100 0x559656a614e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f97e0005ec0 tx=0x7f97e0005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3fa326eb62148726 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:53:46.508 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.505+0000 7f97efe00700 1 --2- 172.21.15.86:0/2742411932 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f97d40a1290 0x7f97d40a3750 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x559656a62b90 tx=0x7f97e8009920 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:53:46.509 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.505+0000 7f97e4c00700 1 -- 172.21.15.86:0/871182648 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e001d070 con 0x559656a61100 2024-03-23T17:53:46.509 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.506+0000 7f97e4c00700 1 -- 172.21.15.86:0/871182648 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f97e001e040 con 0x559656a61100 2024-03-23T17:53:46.509 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.506+0000 7f97e4c00700 1 -- 172.21.15.86:0/871182648 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e00047f0 con 0x559656a61100 2024-03-23T17:53:46.509 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.506+0000 7f980a281b80 1 -- 172.21.15.86:0/871182648 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559656a61100 msgr2=0x559656a614e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:53:46.509 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.506+0000 7f980a281b80 1 --2- 172.21.15.86:0/871182648 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559656a61100 0x559656a614e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f97e0005ec0 tx=0x7f97e0005ef0 comp rx=0 tx=0).stop 2024-03-23T17:53:46.509 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.506+0000 7f980a281b80 1 -- 172.21.15.86:0/871182648 shutdown_connections 2024-03-23T17:53:46.509 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.506+0000 7f980a281b80 1 --2- 172.21.15.86:0/871182648 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559656a6c8d0 0x559656a6ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:46.509 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.506+0000 7f980a281b80 1 --2- 172.21.15.86:0/871182648 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559656a61100 0x559656a614e0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:46.509 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.506+0000 7f980a281b80 1 --2- 172.21.15.86:0/871182648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559656a61a20 0x559656a6c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:46.509 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.506+0000 7f980a281b80 1 -- 172.21.15.86:0/871182648 >> 172.21.15.86:0/871182648 conn(0x5596569c1640 msgr2=0x559656a5d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:53:46.509 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.507+0000 7f980a281b80 1 -- 172.21.15.86:0/871182648 shutdown_connections 2024-03-23T17:53:46.509 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.507+0000 7f980a281b80 1 -- 172.21.15.86:0/871182648 wait complete. 2024-03-23T17:53:46.509 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.507+0000 7f980a281b80 1 -- start start 2024-03-23T17:53:46.509 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.508+0000 7f980a281b80 1 Processor -- start 2024-03-23T17:53:46.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.508+0000 7f980a281b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559656a61100 0x559656a6c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:46.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.508+0000 7f980a281b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559656a61a20 0x559656a69680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.508+0000 7f980a281b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559656a6c8d0 0x559656a69d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.508+0000 7f980a281b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55965694fd20 con 0x559656a6c8d0 2024-03-23T17:53:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.508+0000 7f980a281b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55965694f9c0 con 0x559656a61a20 2024-03-23T17:53:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.508+0000 7f980a281b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x559656837640 con 0x559656a61100 2024-03-23T17:53:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.508+0000 7f980a281b80 10 client.0 _ll_register_callbacks cb 0x559656959410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:53:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.508+0000 7f97eea00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559656a61a20 0x559656a69680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.508+0000 7f97eea00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559656a61a20 0x559656a69680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:35194/0 (socket says 172.21.15.86:35194) 2024-03-23T17:53:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.508+0000 7f97eea00700 1 -- 172.21.15.86:0/1930867592 learned_addr learned my addr 172.21.15.86:0/1930867592 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:53:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.509+0000 7f97ef400700 1 --2- 172.21.15.86:0/1930867592 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559656a61100 0x559656a6c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.509+0000 7f97efe00700 1 --2- 172.21.15.86:0/1930867592 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559656a6c8d0 0x559656a69d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.509+0000 7f97eea00700 1 -- 172.21.15.86:0/1930867592 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559656a61100 msgr2=0x559656a6c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:53:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.509+0000 7f97eea00700 1 --2- 172.21.15.86:0/1930867592 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559656a61100 0x559656a6c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.509+0000 7f97eea00700 1 -- 172.21.15.86:0/1930867592 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559656a6c8d0 msgr2=0x559656a69d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:53:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.509+0000 7f97eea00700 1 --2- 172.21.15.86:0/1930867592 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559656a6c8d0 0x559656a69d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.509+0000 7f97eea00700 1 -- 172.21.15.86:0/1930867592 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97f0115cf0 con 0x559656a61a20 2024-03-23T17:53:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.509+0000 7f97efe00700 1 --2- 172.21.15.86:0/1930867592 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559656a6c8d0 0x559656a69d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T17:53:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.509+0000 7f97eea00700 1 --2- 172.21.15.86:0/1930867592 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559656a61a20 0x559656a69680 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f97dc096530 tx=0x7f97dc09a7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7e958f851fe4f4b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:53:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.509+0000 7f97d9600700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T17:53:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.509+0000 7f97d9600700 1 -- 172.21.15.86:0/1930867592 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97dc0b2de0 con 0x559656a61a20 2024-03-23T17:53:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.509+0000 7f97d9600700 1 -- 172.21.15.86:0/1930867592 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f97dc0a24b0 con 0x559656a61a20 2024-03-23T17:53:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.509+0000 7f980a281b80 1 -- 172.21.15.86:0/1930867592 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559656974320 con 0x559656a61a20 2024-03-23T17:53:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.510+0000 7f97d9600700 1 -- 172.21.15.86:0/1930867592 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f97dc0b8270 con 0x559656a61a20 2024-03-23T17:53:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.510+0000 7f980a281b80 10 client.26282 fetch_fsmap learned FSMap version 34 2024-03-23T17:53:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.510+0000 7f980a281b80 1 -- 172.21.15.86:0/1930867592 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5596567e1b00 con 0x559656a61a20 2024-03-23T17:53:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.510+0000 7f97d9600700 1 -- 172.21.15.86:0/1930867592 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f97dc064070 con 0x559656a61a20 2024-03-23T17:53:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.510+0000 7f97d9600700 1 -- 172.21.15.86:0/1930867592 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 34) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f97dc0a2020 con 0x559656a61a20 2024-03-23T17:53:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.510+0000 7f980a281b80 10 client.26282 fetch_fsmap finished waiting for FSMap version 34 2024-03-23T17:53:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.510+0000 7f980a281b80 10 client.26282 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:53:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.510+0000 7f980a281b80 20 client.26282 populate_metadata read hostname 'smithi086' 2024-03-23T17:53:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.510+0000 7f980a281b80 1 --2- 172.21.15.86:0/1930867592 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x559656a7c5e0 0x559656a7ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.510+0000 7f980a281b80 4 client.26282 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:53:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.510+0000 7f980a281b80 1 -- 172.21.15.86:0/1930867592 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5596569ac6a0 con 0x559656a7c5e0 2024-03-23T17:53:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.511+0000 7f97ef400700 1 --2- 172.21.15.86:0/1930867592 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x559656a7c5e0 0x559656a7ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.511+0000 7f97ef400700 1 --2- 172.21.15.86:0/1930867592 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x559656a7c5e0 0x559656a7ea00 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1000ec29191d90eb server_cookie=3dbdeaf588224e18 in_seq=0 out_seq=0 2024-03-23T17:53:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.511+0000 7f97d9600700 10 client.26282 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:53:46.516 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.512+0000 7f97d9600700 1 -- 172.21.15.86:0/1930867592 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5596569ac6a0 con 0x559656a7c5e0 2024-03-23T17:53:46.516 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.512+0000 7f97d9600700 10 client.26282 handle_command_reply: tid=0 2024-03-23T17:53:46.517 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.513+0000 7f980a281b80 1 client.26282 shutdown 2024-03-23T17:53:46.517 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.513+0000 7f980a281b80 1 -- 172.21.15.86:0/1930867592 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559656a61a20 msgr2=0x559656a69680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:53:46.517 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.513+0000 7f980a281b80 1 --2- 172.21.15.86:0/1930867592 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559656a61a20 0x559656a69680 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f97dc096530 tx=0x7f97dc09a7b0 comp rx=0 tx=0).stop 2024-03-23T17:53:46.517 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.513+0000 7f980a281b80 1 -- 172.21.15.86:0/1930867592 shutdown_connections 2024-03-23T17:53:46.517 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.513+0000 7f980a281b80 1 --2- 172.21.15.86:0/1930867592 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x559656a7c5e0 0x559656a7ea00 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:46.517 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.513+0000 7f980a281b80 1 --2- 172.21.15.86:0/1930867592 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559656a6c8d0 0x559656a69d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:46.517 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.513+0000 7f980a281b80 1 --2- 172.21.15.86:0/1930867592 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559656a61100 0x559656a6c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:46.517 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.513+0000 7f980a281b80 1 --2- 172.21.15.86:0/1930867592 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559656a61a20 0x559656a69680 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:46.517 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.513+0000 7f980a281b80 1 -- 172.21.15.86:0/1930867592 >> 172.21.15.86:0/1930867592 conn(0x5596569c1640 msgr2=0x559656a630a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:53:46.517 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.513+0000 7f97d9600700 0 client.26282 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:53:46.517 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.513+0000 7f980a281b80 1 -- 172.21.15.86:0/1930867592 shutdown_connections 2024-03-23T17:53:46.517 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.513+0000 7f980a281b80 1 -- 172.21.15.86:0/1930867592 wait complete. 2024-03-23T17:53:46.517 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.513+0000 7f980a281b80 20 client.26282 trim_cache size 0 max 16384 2024-03-23T17:53:46.531 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.530+0000 7f980a281b80 1 Processor -- start 2024-03-23T17:53:46.531 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.530+0000 7f980a281b80 1 -- start start 2024-03-23T17:53:46.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.531+0000 7f980a281b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559656a61100 0x559656a73580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:46.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.531+0000 7f980a281b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559656a61a20 0x559656a57500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:46.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.531+0000 7f980a281b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559656a6c8d0 0x559656a63750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:46.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.531+0000 7f980a281b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x559656a63c90 con 0x559656a61a20 2024-03-23T17:53:46.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.531+0000 7f980a281b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x559656a6ac30 con 0x559656a6c8d0 2024-03-23T17:53:46.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.531+0000 7f980a281b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x559656a6a680 con 0x559656a61100 2024-03-23T17:53:46.533 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.531+0000 7f97ef400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559656a61100 0x559656a73580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:46.533 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.531+0000 7f97ef400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559656a61100 0x559656a73580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:45090/0 (socket says 172.21.15.86:45090) 2024-03-23T17:53:46.533 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.531+0000 7f97efe00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559656a6c8d0 0x559656a63750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:46.533 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.531+0000 7f97ef400700 1 -- 172.21.15.86:0/3082661303 learned_addr learned my addr 172.21.15.86:0/3082661303 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:53:46.533 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.531+0000 7f97eea00700 1 --2- 172.21.15.86:0/3082661303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559656a61a20 0x559656a57500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:46.534 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.531+0000 7f97ef400700 1 -- 172.21.15.86:0/3082661303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559656a6c8d0 msgr2=0x559656a63750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:53:46.534 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.531+0000 7f97ef400700 1 --2- 172.21.15.86:0/3082661303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559656a6c8d0 0x559656a63750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:46.534 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.531+0000 7f97ef400700 1 -- 172.21.15.86:0/3082661303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559656a61a20 msgr2=0x559656a57500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:53:46.534 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.531+0000 7f97ef400700 1 --2- 172.21.15.86:0/3082661303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559656a61a20 0x559656a57500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:46.534 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.531+0000 7f97ef400700 1 -- 172.21.15.86:0/3082661303 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559656979e40 con 0x559656a61100 2024-03-23T17:53:46.534 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.532+0000 7f97efe00700 1 --2- 172.21.15.86:0/3082661303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559656a6c8d0 0x559656a63750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:53:46.534 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.532+0000 7f97ef400700 1 --2- 172.21.15.86:0/3082661303 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559656a61100 0x559656a73580 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f97e0005ec0 tx=0x7f97e001a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=204de679b375700e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:53:46.534 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.532+0000 7f97e5600700 1 -- 172.21.15.86:0/3082661303 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e0032070 con 0x559656a61100 2024-03-23T17:53:46.535 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.532+0000 7f97e5600700 1 -- 172.21.15.86:0/3082661303 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f97e0038020 con 0x559656a61100 2024-03-23T17:53:46.535 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.532+0000 7f97e5600700 1 -- 172.21.15.86:0/3082661303 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e001d040 con 0x559656a61100 2024-03-23T17:53:46.535 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.533+0000 7f980a281b80 1 -- 172.21.15.86:0/3082661303 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559656a61100 msgr2=0x559656a73580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:53:46.535 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.533+0000 7f980a281b80 1 --2- 172.21.15.86:0/3082661303 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559656a61100 0x559656a73580 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f97e0005ec0 tx=0x7f97e001a040 comp rx=0 tx=0).stop 2024-03-23T17:53:46.535 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.533+0000 7f980a281b80 1 -- 172.21.15.86:0/3082661303 shutdown_connections 2024-03-23T17:53:46.535 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.533+0000 7f980a281b80 1 --2- 172.21.15.86:0/3082661303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559656a6c8d0 0x559656a63750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:46.535 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.533+0000 7f980a281b80 1 --2- 172.21.15.86:0/3082661303 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559656a61100 0x559656a73580 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:46.535 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.533+0000 7f980a281b80 1 --2- 172.21.15.86:0/3082661303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559656a61a20 0x559656a57500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:46.535 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.533+0000 7f980a281b80 1 -- 172.21.15.86:0/3082661303 >> 172.21.15.86:0/3082661303 conn(0x5596569c1640 msgr2=0x559656a63040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:53:46.535 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.533+0000 7f980a281b80 1 -- 172.21.15.86:0/3082661303 shutdown_connections 2024-03-23T17:53:46.535 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.533+0000 7f980a281b80 1 -- 172.21.15.86:0/3082661303 wait complete. 2024-03-23T17:53:46.537 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.534+0000 7f980a281b80 1 -- start start 2024-03-23T17:53:46.537 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.534+0000 7f980a281b80 1 Processor -- start 2024-03-23T17:53:46.537 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.535+0000 7f980a281b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559656a61100 0x559656a71750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:46.537 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.535+0000 7f980a281b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559656a61a20 0x559656a71c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:46.538 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.535+0000 7f980a281b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559656a6c8d0 0x559656acd100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:46.538 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.535+0000 7f980a281b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x559656a723d0 con 0x559656a61100 2024-03-23T17:53:46.538 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.535+0000 7f980a281b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x559656acd640 con 0x559656a61a20 2024-03-23T17:53:46.538 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.535+0000 7f980a281b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x559656acd7d0 con 0x559656a6c8d0 2024-03-23T17:53:46.538 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.535+0000 7f980a281b80 10 client.0 _ll_register_callbacks cb 0x559656959410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:53:46.538 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.535+0000 7f97ef400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559656a61100 0x559656a71750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:46.538 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.535+0000 7f97ef400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559656a61100 0x559656a71750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39944/0 (socket says 172.21.15.86:39944) 2024-03-23T17:53:46.538 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.535+0000 7f97ef400700 1 -- 172.21.15.86:0/1155011237 learned_addr learned my addr 172.21.15.86:0/1155011237 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:53:46.538 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.535+0000 7f97efe00700 1 --2- 172.21.15.86:0/1155011237 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559656a6c8d0 0x559656acd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:46.538 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.535+0000 7f97eea00700 1 --2- 172.21.15.86:0/1155011237 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559656a61a20 0x559656a71c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:46.538 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.536+0000 7f97ef400700 1 -- 172.21.15.86:0/1155011237 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559656a6c8d0 msgr2=0x559656acd100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:53:46.538 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.536+0000 7f97ef400700 1 --2- 172.21.15.86:0/1155011237 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559656a6c8d0 0x559656acd100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:46.538 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.536+0000 7f97ef400700 1 -- 172.21.15.86:0/1155011237 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559656a61a20 msgr2=0x559656a71c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:53:46.539 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.536+0000 7f97ef400700 1 --2- 172.21.15.86:0/1155011237 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559656a61a20 0x559656a71c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:46.539 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.536+0000 7f97ef400700 1 -- 172.21.15.86:0/1155011237 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97e8011040 con 0x559656a61100 2024-03-23T17:53:46.539 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.536+0000 7f97efe00700 1 --2- 172.21.15.86:0/1155011237 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559656a6c8d0 0x559656acd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:53:46.539 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.536+0000 7f97ef400700 1 --2- 172.21.15.86:0/1155011237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559656a61100 0x559656a71750 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f97e0005f50 tx=0x7f97e00053d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=671225f994af5897 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:53:46.539 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.536+0000 7f97d9600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-23T17:53:46.539 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.536+0000 7f97d9600700 1 -- 172.21.15.86:0/1155011237 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e001d030 con 0x559656a61100 2024-03-23T17:53:46.539 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.536+0000 7f97d9600700 1 -- 172.21.15.86:0/1155011237 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f97e0039030 con 0x559656a61100 2024-03-23T17:53:46.539 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.536+0000 7f980a281b80 1 -- 172.21.15.86:0/1155011237 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f97e001e040 con 0x559656a61100 2024-03-23T17:53:46.540 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.537+0000 7f97d9600700 1 -- 172.21.15.86:0/1155011237 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f97e0032070 con 0x559656a61100 2024-03-23T17:53:46.540 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.537+0000 7f980a281b80 10 client.16236 fetch_fsmap learned FSMap version 34 2024-03-23T17:53:46.540 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.537+0000 7f980a281b80 1 -- 172.21.15.86:0/1155011237 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x559656835350 con 0x559656a61100 2024-03-23T17:53:46.540 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.537+0000 7f97d9600700 1 -- 172.21.15.86:0/1155011237 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f97e001e040 con 0x559656a61100 2024-03-23T17:53:46.540 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.537+0000 7f97d9600700 1 -- 172.21.15.86:0/1155011237 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 34) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f97e0004cc0 con 0x559656a61100 2024-03-23T17:53:46.540 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.537+0000 7f980a281b80 10 client.16236 fetch_fsmap finished waiting for FSMap version 34 2024-03-23T17:53:46.541 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.537+0000 7f980a281b80 10 client.16236 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:53:46.541 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.537+0000 7f980a281b80 20 client.16236 populate_metadata read hostname 'smithi086' 2024-03-23T17:53:46.541 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.537+0000 7f980a281b80 1 --2- 172.21.15.86:0/1155011237 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x559656aceb50 0x559656ad0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:46.541 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.537+0000 7f980a281b80 4 client.16236 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T17:53:46.541 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.537+0000 7f980a281b80 1 -- 172.21.15.86:0/1155011237 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559656802c90 con 0x559656aceb50 2024-03-23T17:53:46.541 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.538+0000 7f97eea00700 1 --2- 172.21.15.86:0/1155011237 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x559656aceb50 0x559656ad0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:46.541 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.538+0000 7f97eea00700 1 --2- 172.21.15.86:0/1155011237 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x559656aceb50 0x559656ad0f40 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a603a271d93c519 server_cookie=4b6521128b2b266 in_seq=0 out_seq=0 2024-03-23T17:53:46.541 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.538+0000 7f97d9600700 10 client.16236 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:53:46.570 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.569+0000 7f97d9600700 1 -- 172.21.15.86:0/1155011237 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559656802c90 con 0x559656aceb50 2024-03-23T17:53:46.570 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.569+0000 7f97d9600700 10 client.16236 handle_command_reply: tid=0 2024-03-23T17:53:46.570 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.569+0000 7f980a281b80 1 client.16236 shutdown 2024-03-23T17:53:46.570 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.569+0000 7f980a281b80 1 -- 172.21.15.86:0/1155011237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559656a61100 msgr2=0x559656a71750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:53:46.570 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.569+0000 7f980a281b80 1 --2- 172.21.15.86:0/1155011237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559656a61100 0x559656a71750 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f97e0005f50 tx=0x7f97e00053d0 comp rx=0 tx=0).stop 2024-03-23T17:53:46.571 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.569+0000 7f980a281b80 1 -- 172.21.15.86:0/1155011237 shutdown_connections 2024-03-23T17:53:46.571 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.569+0000 7f980a281b80 1 --2- 172.21.15.86:0/1155011237 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x559656aceb50 0x559656ad0f40 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:46.571 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.569+0000 7f980a281b80 1 --2- 172.21.15.86:0/1155011237 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559656a6c8d0 0x559656acd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:46.571 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.569+0000 7f980a281b80 1 --2- 172.21.15.86:0/1155011237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559656a61100 0x559656a71750 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:46.571 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.569+0000 7f97d9600700 0 client.16236 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:53:46.572 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.569+0000 7f980a281b80 1 --2- 172.21.15.86:0/1155011237 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559656a61a20 0x559656a71c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:46.572 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.569+0000 7f980a281b80 1 -- 172.21.15.86:0/1155011237 >> 172.21.15.86:0/1155011237 conn(0x5596569c1640 msgr2=0x559656a5cd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:53:46.572 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.570+0000 7f980a281b80 1 -- 172.21.15.86:0/1155011237 shutdown_connections 2024-03-23T17:53:46.572 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.570+0000 7f980a281b80 1 -- 172.21.15.86:0/1155011237 wait complete. 2024-03-23T17:53:46.572 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.570+0000 7f980a281b80 20 client.16236 trim_cache size 0 max 16384 2024-03-23T17:53:46.572 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-03-23T17:53:46.572 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-03-23T17:53:46.572 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-03-23T17:53:46.572 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-03-23T17:53:46.573 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.570+0000 7f97f6800700 1 -- 172.21.15.86:0/2742411932 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f97d40a1290 msgr2=0x7f97d40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:53:46.573 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.570+0000 7f97f6800700 1 --2- 172.21.15.86:0/2742411932 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f97d40a1290 0x7f97d40a3750 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x559656a62b90 tx=0x7f97e8009920 comp rx=0 tx=0).stop 2024-03-23T17:53:46.573 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.570+0000 7f97f6800700 1 -- 172.21.15.86:0/2742411932 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97f010a070 msgr2=0x7f97f006fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:53:46.573 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.570+0000 7f97f6800700 1 --2- 172.21.15.86:0/2742411932 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97f010a070 0x7f97f006fdb0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f97dc00e9a0 tx=0x7f97dc00ecb0 comp rx=0 tx=0).stop 2024-03-23T17:53:46.573 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.571+0000 7f97f6800700 1 -- 172.21.15.86:0/2742411932 shutdown_connections 2024-03-23T17:53:46.574 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.571+0000 7f97f6800700 1 --2- 172.21.15.86:0/2742411932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97f0112d00 0x7f97f00702f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:46.574 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.571+0000 7f97f6800700 1 --2- 172.21.15.86:0/2742411932 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f97d40a1290 0x7f97d40a3750 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:46.574 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.571+0000 7f97f6800700 1 --2- 172.21.15.86:0/2742411932 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f97f0109750 0x7f97f006f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:46.574 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.571+0000 7f97f6800700 1 --2- 172.21.15.86:0/2742411932 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97f010a070 0x7f97f006fdb0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:46.574 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.571+0000 7f97f6800700 1 -- 172.21.15.86:0/2742411932 >> 172.21.15.86:0/2742411932 conn(0x7f97f006bd80 msgr2=0x7f97f0108b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:53:46.574 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.571+0000 7f97f6800700 1 -- 172.21.15.86:0/2742411932 shutdown_connections 2024-03-23T17:53:46.574 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:46.571+0000 7f97f6800700 1 -- 172.21.15.86:0/2742411932 wait complete. 2024-03-23T17:53:46.890 INFO:tasks.workunit.client.0.smithi086.stdout: 1 88077 2.78 MB/sec execute 11 sec latency 4957.233 ms 2024-03-23T17:53:47.059 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:53:46 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:53:47.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:53:46 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:53:47.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:53:46 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:53:47.890 INFO:tasks.workunit.client.0.smithi086.stdout: 1 89325 3.15 MB/sec execute 12 sec latency 13.273 ms 2024-03-23T17:53:48.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:53:47 smithi092 ceph-mon[79089]: pgmap v97: 97 pgs: 97 active+clean; 45 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 733 KiB/s wr, 67 op/s 2024-03-23T17:53:48.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:53:47 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:53:48.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:53:47 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:53:48.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:53:47 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:53:48.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:53:47 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:53:48.085 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:53:47 smithi152 ceph-mon[47949]: pgmap v97: 97 pgs: 97 active+clean; 45 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 733 KiB/s wr, 67 op/s 2024-03-23T17:53:48.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:53:47 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:53:48.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:53:47 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:53:48.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:53:47 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:53:48.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:53:47 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:53:48.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:53:47 smithi086 ceph-mon[38221]: pgmap v97: 97 pgs: 97 active+clean; 45 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 733 KiB/s wr, 67 op/s 2024-03-23T17:53:48.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:53:47 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:53:48.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:53:47 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:53:48.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:53:47 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:53:48.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:53:47 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:53:48.890 INFO:tasks.workunit.client.0.smithi086.stdout: 1 91692 3.88 MB/sec execute 13 sec latency 12.764 ms 2024-03-23T17:53:49.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:53:48 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:53:49.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:53:48 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:53:49.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:53:48 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:53:49.085 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:53:48 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:53:49.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:53:48 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:53:49.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:53:48 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:53:49.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:53:48 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:53:49.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:53:48 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:53:49.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:53:48 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:53:49.890 INFO:tasks.workunit.client.0.smithi086.stdout: 1 92936 4.08 MB/sec execute 14 sec latency 88.052 ms 2024-03-23T17:53:50.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:53:49 smithi086 ceph-mon[38221]: pgmap v98: 97 pgs: 97 active+clean; 45 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 733 KiB/s wr, 67 op/s 2024-03-23T17:53:50.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:53:49 smithi092 ceph-mon[79089]: pgmap v98: 97 pgs: 97 active+clean; 45 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 733 KiB/s wr, 67 op/s 2024-03-23T17:53:50.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:53:49 smithi152 ceph-mon[47949]: pgmap v98: 97 pgs: 97 active+clean; 45 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 733 KiB/s wr, 67 op/s 2024-03-23T17:53:50.890 INFO:tasks.workunit.client.0.smithi086.stdout: 1 92936 3.80 MB/sec execute 15 sec latency 1088.104 ms 2024-03-23T17:53:51.890 INFO:tasks.workunit.client.0.smithi086.stdout: 1 92992 3.58 MB/sec execute 16 sec latency 1106.500 ms 2024-03-23T17:53:52.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:53:51 smithi086 ceph-mon[38221]: pgmap v99: 97 pgs: 97 active+clean; 50 MiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 1007 KiB/s rd, 1.4 MiB/s wr, 157 op/s 2024-03-23T17:53:52.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:53:51 smithi092 ceph-mon[79089]: pgmap v99: 97 pgs: 97 active+clean; 50 MiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 1007 KiB/s rd, 1.4 MiB/s wr, 157 op/s 2024-03-23T17:53:52.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:53:51 smithi152 ceph-mon[47949]: pgmap v99: 97 pgs: 97 active+clean; 50 MiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 1007 KiB/s rd, 1.4 MiB/s wr, 157 op/s 2024-03-23T17:53:52.890 INFO:tasks.workunit.client.0.smithi086.stdout: 1 92992 3.36 MB/sec execute 17 sec latency 1945.383 ms 2024-03-23T17:53:53.891 INFO:tasks.workunit.client.0.smithi086.stdout: 1 92992 3.18 MB/sec execute 18 sec latency 2945.478 ms 2024-03-23T17:53:54.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:53:53 smithi086 ceph-mon[38221]: pgmap v100: 97 pgs: 97 active+clean; 50 MiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 703 KiB/s wr, 100 op/s 2024-03-23T17:53:54.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:53:53 smithi092 ceph-mon[79089]: pgmap v100: 97 pgs: 97 active+clean; 50 MiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 703 KiB/s wr, 100 op/s 2024-03-23T17:53:54.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:53:53 smithi152 ceph-mon[47949]: pgmap v100: 97 pgs: 97 active+clean; 50 MiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 703 KiB/s wr, 100 op/s 2024-03-23T17:53:54.890 INFO:tasks.workunit.client.0.smithi086.stdout: 1 92992 3.01 MB/sec execute 19 sec latency 3945.576 ms 2024-03-23T17:53:55.891 INFO:tasks.workunit.client.0.smithi086.stdout: 1 92992 2.86 MB/sec execute 20 sec latency 4945.646 ms 2024-03-23T17:53:56.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:53:55 smithi086 ceph-mon[38221]: pgmap v101: 97 pgs: 97 active+clean; 49 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 891 KiB/s rd, 1016 KiB/s wr, 128 op/s 2024-03-23T17:53:56.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:53:55 smithi092 ceph-mon[79089]: pgmap v101: 97 pgs: 97 active+clean; 49 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 891 KiB/s rd, 1016 KiB/s wr, 128 op/s 2024-03-23T17:53:56.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:53:55 smithi152 ceph-mon[47949]: pgmap v101: 97 pgs: 97 active+clean; 49 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 891 KiB/s rd, 1016 KiB/s wr, 128 op/s 2024-03-23T17:53:56.891 INFO:tasks.workunit.client.0.smithi086.stdout: 1 95436 3.34 MB/sec execute 21 sec latency 4961.446 ms 2024-03-23T17:53:57.891 INFO:tasks.workunit.client.0.smithi086.stdout: 1 96483 3.46 MB/sec execute 22 sec latency 188.244 ms 2024-03-23T17:53:58.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:53:57 smithi086 ceph-mon[38221]: pgmap v102: 97 pgs: 97 active+clean; 49 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 869 KiB/s rd, 984 KiB/s wr, 118 op/s 2024-03-23T17:53:58.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:53:57 smithi092 ceph-mon[79089]: pgmap v102: 97 pgs: 97 active+clean; 49 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 869 KiB/s rd, 984 KiB/s wr, 118 op/s 2024-03-23T17:53:58.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:53:57 smithi152 ceph-mon[47949]: pgmap v102: 97 pgs: 97 active+clean; 49 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 869 KiB/s rd, 984 KiB/s wr, 118 op/s 2024-03-23T17:53:58.891 INFO:tasks.workunit.client.0.smithi086.stdout: 1 96483 3.31 MB/sec execute 23 sec latency 1188.354 ms 2024-03-23T17:53:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.729+0000 7ff918000700 1 -- 172.21.15.86:0/3155610869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff910100f10 msgr2=0x7ff9101012f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:53:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.729+0000 7ff918000700 1 --2- 172.21.15.86:0/3155610869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff910100f10 0x7ff9101012f0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7ff9040098f0 tx=0x7ff904009c00 comp rx=0 tx=0).stop 2024-03-23T17:53:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.730+0000 7ff918000700 1 -- 172.21.15.86:0/3155610869 shutdown_connections 2024-03-23T17:53:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.730+0000 7ff918000700 1 --2- 172.21.15.86:0/3155610869 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff910106030 0x7ff91010dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.730+0000 7ff918000700 1 --2- 172.21.15.86:0/3155610869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff910100f10 0x7ff9101012f0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.730+0000 7ff918000700 1 --2- 172.21.15.86:0/3155610869 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9101018c0 0x7ff910105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.730+0000 7ff918000700 1 -- 172.21.15.86:0/3155610869 >> 172.21.15.86:0/3155610869 conn(0x7ff9100fc7b0 msgr2=0x7ff9100febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:53:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.730+0000 7ff918000700 1 -- 172.21.15.86:0/3155610869 shutdown_connections 2024-03-23T17:53:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.730+0000 7ff918000700 1 -- 172.21.15.86:0/3155610869 wait complete. 2024-03-23T17:53:59.732 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.731+0000 7ff918000700 1 Processor -- start 2024-03-23T17:53:59.733 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.731+0000 7ff918000700 1 -- start start 2024-03-23T17:53:59.733 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.731+0000 7ff918000700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff910100f10 0x7ff910199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:59.733 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.732+0000 7ff918000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff910106030 0x7ff910199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:59.733 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.732+0000 7ff918000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff91019dce0 0x7ff91019e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:59.733 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.732+0000 7ff918000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff91019e760 con 0x7ff910106030 2024-03-23T17:53:59.733 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.732+0000 7ff918000700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff91019e8d0 con 0x7ff91019dce0 2024-03-23T17:53:59.733 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.732+0000 7ff918000700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff91019ea90 con 0x7ff910100f10 2024-03-23T17:53:59.733 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.732+0000 7ff915600700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff910100f10 0x7ff910199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:59.733 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.732+0000 7ff914c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff910106030 0x7ff910199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:59.734 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.732+0000 7ff915600700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff910100f10 0x7ff910199400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:55016/0 (socket says 172.21.15.86:55016) 2024-03-23T17:53:59.734 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.732+0000 7ff915600700 1 -- 172.21.15.86:0/174451366 learned_addr learned my addr 172.21.15.86:0/174451366 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:53:59.734 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.732+0000 7ff916000700 1 --2- 172.21.15.86:0/174451366 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff91019dce0 0x7ff91019e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:59.734 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.732+0000 7ff915600700 1 -- 172.21.15.86:0/174451366 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff91019dce0 msgr2=0x7ff91019e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:53:59.734 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.732+0000 7ff915600700 1 --2- 172.21.15.86:0/174451366 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff91019dce0 0x7ff91019e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:59.734 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.732+0000 7ff915600700 1 -- 172.21.15.86:0/174451366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff910106030 msgr2=0x7ff910199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:53:59.734 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.732+0000 7ff915600700 1 --2- 172.21.15.86:0/174451366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff910106030 0x7ff910199940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:59.734 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.732+0000 7ff915600700 1 -- 172.21.15.86:0/174451366 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8fc0095f0 con 0x7ff910100f10 2024-03-23T17:53:59.734 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.732+0000 7ff916000700 1 --2- 172.21.15.86:0/174451366 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff91019dce0 0x7ff91019e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:53:59.734 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.732+0000 7ff914c00700 1 --2- 172.21.15.86:0/174451366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff910106030 0x7ff910199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:53:59.734 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.733+0000 7ff915600700 1 --2- 172.21.15.86:0/174451366 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff910100f10 0x7ff910199400 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff904005950 tx=0x7ff90400fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:53:59.735 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.733+0000 7ff90e000700 1 -- 172.21.15.86:0/174451366 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff90401c070 con 0x7ff910100f10 2024-03-23T17:53:59.735 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.733+0000 7ff918000700 1 -- 172.21.15.86:0/174451366 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff904009520 con 0x7ff910100f10 2024-03-23T17:53:59.735 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.733+0000 7ff90e000700 1 -- 172.21.15.86:0/174451366 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff90400b9c0 con 0x7ff910100f10 2024-03-23T17:53:59.735 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.733+0000 7ff90e000700 1 -- 172.21.15.86:0/174451366 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9040038e0 con 0x7ff910100f10 2024-03-23T17:53:59.735 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.733+0000 7ff918000700 1 -- 172.21.15.86:0/174451366 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9101a61d0 con 0x7ff910100f10 2024-03-23T17:53:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.736+0000 7ff90e000700 1 -- 172.21.15.86:0/174451366 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7ff904020030 con 0x7ff910100f10 2024-03-23T17:53:59.738 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.737+0000 7ff92badbb80 1 Processor -- start 2024-03-23T17:53:59.738 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.737+0000 7ff90e000700 1 --2- 172.21.15.86:0/174451366 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7ff8f80a1290 0x7ff8f80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:59.738 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.738+0000 7ff90e000700 1 -- 172.21.15.86:0/174451366 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7ff9040cda10 con 0x7ff910100f10 2024-03-23T17:53:59.738 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.738+0000 7ff92badbb80 1 -- start start 2024-03-23T17:53:59.738 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.738+0000 7ff916000700 1 --2- 172.21.15.86:0/174451366 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7ff8f80a1290 0x7ff8f80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:59.739 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.738+0000 7ff92badbb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620b9a1b100 0x5620b9a1b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:59.739 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.738+0000 7ff92badbb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5620b9a1ba20 0x5620b9a26390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:59.739 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.738+0000 7ff92badbb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620b9a268d0 0x5620b9a28cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:59.739 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.738+0000 7ff92badbb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620b9974930 con 0x5620b9a268d0 2024-03-23T17:53:59.740 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.738+0000 7ff92badbb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620b99722f0 con 0x5620b9a1b100 2024-03-23T17:53:59.740 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.738+0000 7ff92badbb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620b996dbe0 con 0x5620b9a1ba20 2024-03-23T17:53:59.740 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.739+0000 7ff914c00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5620b9a1ba20 0x5620b9a26390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:59.740 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.739+0000 7ff916000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620b9a268d0 0x5620b9a28cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:59.740 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.739+0000 7ff915600700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620b9a1b100 0x5620b9a1b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:59.740 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.739+0000 7ff914c00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5620b9a1ba20 0x5620b9a26390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:55020/0 (socket says 172.21.15.86:55020) 2024-03-23T17:53:59.740 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.739+0000 7ff914c00700 1 -- 172.21.15.86:0/3198245661 learned_addr learned my addr 172.21.15.86:0/3198245661 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:53:59.740 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.739+0000 7ff916000700 1 --2- 172.21.15.86:0/174451366 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7ff8f80a1290 0x7ff8f80a3750 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff908003e30 tx=0x7ff90800b040 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:53:59.740 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.739+0000 7ff915600700 1 -- 172.21.15.86:0/3198245661 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5620b9a1ba20 msgr2=0x5620b9a26390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:53:59.740 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.739+0000 7ff915600700 1 --2- 172.21.15.86:0/3198245661 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5620b9a1ba20 0x5620b9a26390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:59.740 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.739+0000 7ff915600700 1 -- 172.21.15.86:0/3198245661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620b9a268d0 msgr2=0x5620b9a28cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:53:59.740 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.739+0000 7ff915600700 1 --2- 172.21.15.86:0/3198245661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620b9a268d0 0x5620b9a28cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:59.740 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.739+0000 7ff915600700 1 -- 172.21.15.86:0/3198245661 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620b97ef350 con 0x5620b9a1b100 2024-03-23T17:53:59.740 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.739+0000 7ff915600700 1 --2- 172.21.15.86:0/3198245661 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620b9a1b100 0x5620b9a1b4e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff9040cb150 tx=0x7ff90409a580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c9371231585f8db server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:53:59.741 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.739+0000 7ff902600700 1 -- 172.21.15.86:0/3198245661 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff90409aba0 con 0x5620b9a1b100 2024-03-23T17:53:59.741 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.739+0000 7ff902600700 1 -- 172.21.15.86:0/3198245661 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff90409e9d0 con 0x5620b9a1b100 2024-03-23T17:53:59.741 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.739+0000 7ff902600700 1 -- 172.21.15.86:0/3198245661 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff904005140 con 0x5620b9a1b100 2024-03-23T17:53:59.744 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.740+0000 7ff92badbb80 1 -- 172.21.15.86:0/3198245661 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620b9a1b100 msgr2=0x5620b9a1b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:53:59.744 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.740+0000 7ff92badbb80 1 --2- 172.21.15.86:0/3198245661 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620b9a1b100 0x5620b9a1b4e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff9040cb150 tx=0x7ff90409a580 comp rx=0 tx=0).stop 2024-03-23T17:53:59.744 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.740+0000 7ff92badbb80 1 -- 172.21.15.86:0/3198245661 shutdown_connections 2024-03-23T17:53:59.744 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.740+0000 7ff92badbb80 1 --2- 172.21.15.86:0/3198245661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620b9a268d0 0x5620b9a28cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:59.744 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.740+0000 7ff92badbb80 1 --2- 172.21.15.86:0/3198245661 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620b9a1b100 0x5620b9a1b4e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:59.744 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.740+0000 7ff92badbb80 1 --2- 172.21.15.86:0/3198245661 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5620b9a1ba20 0x5620b9a26390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:59.744 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.740+0000 7ff92badbb80 1 -- 172.21.15.86:0/3198245661 >> 172.21.15.86:0/3198245661 conn(0x5620b997b640 msgr2=0x5620b9a17520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:53:59.744 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.740+0000 7ff92badbb80 1 -- 172.21.15.86:0/3198245661 shutdown_connections 2024-03-23T17:53:59.745 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.740+0000 7ff92badbb80 1 -- 172.21.15.86:0/3198245661 wait complete. 2024-03-23T17:53:59.745 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.741+0000 7ff92badbb80 1 -- start start 2024-03-23T17:53:59.745 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.741+0000 7ff92badbb80 1 Processor -- start 2024-03-23T17:53:59.745 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.741+0000 7ff92badbb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620b9a1b100 0x5620b9a26280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:59.745 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.741+0000 7ff92badbb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620b9a1ba20 0x5620b9a23570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:59.745 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.741+0000 7ff92badbb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5620b9a268d0 0x5620b9a23ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:59.745 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.741+0000 7ff92badbb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620b9909d20 con 0x5620b9a1ba20 2024-03-23T17:53:59.745 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.741+0000 7ff92badbb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620b99099c0 con 0x5620b9a1b100 2024-03-23T17:53:59.745 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.741+0000 7ff92badbb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620b97f1640 con 0x5620b9a268d0 2024-03-23T17:53:59.745 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.741+0000 7ff92badbb80 10 client.0 _ll_register_callbacks cb 0x5620b9913410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:53:59.745 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.742+0000 7ff914c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620b9a1ba20 0x5620b9a23570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:59.745 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.742+0000 7ff914c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620b9a1ba20 0x5620b9a23570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43772/0 (socket says 172.21.15.86:43772) 2024-03-23T17:53:59.745 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.742+0000 7ff914c00700 1 -- 172.21.15.86:0/4019995013 learned_addr learned my addr 172.21.15.86:0/4019995013 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:53:59.745 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.742+0000 7ff916000700 1 --2- 172.21.15.86:0/4019995013 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5620b9a268d0 0x5620b9a23ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:59.746 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.742+0000 7ff915600700 1 --2- 172.21.15.86:0/4019995013 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620b9a1b100 0x5620b9a26280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:59.746 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.742+0000 7ff914c00700 1 -- 172.21.15.86:0/4019995013 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5620b9a268d0 msgr2=0x5620b9a23ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:53:59.746 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.742+0000 7ff914c00700 1 --2- 172.21.15.86:0/4019995013 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5620b9a268d0 0x5620b9a23ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:59.746 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.742+0000 7ff914c00700 1 -- 172.21.15.86:0/4019995013 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620b9a1b100 msgr2=0x5620b9a26280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:53:59.746 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.742+0000 7ff914c00700 1 --2- 172.21.15.86:0/4019995013 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620b9a1b100 0x5620b9a26280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:59.746 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.742+0000 7ff914c00700 1 -- 172.21.15.86:0/4019995013 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9101a61d0 con 0x5620b9a1ba20 2024-03-23T17:53:59.746 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.742+0000 7ff914c00700 1 --2- 172.21.15.86:0/4019995013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620b9a1ba20 0x5620b9a23570 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc012cf0 tx=0x7ff8fc00f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c6ea063f410056 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:53:59.746 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.742+0000 7ff8f6a00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-23T17:53:59.746 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.742+0000 7ff8f6a00700 1 -- 172.21.15.86:0/4019995013 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8fc017070 con 0x5620b9a1ba20 2024-03-23T17:53:59.746 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.742+0000 7ff8f6a00700 1 -- 172.21.15.86:0/4019995013 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7ff8fc00f9c0 con 0x5620b9a1ba20 2024-03-23T17:53:59.746 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.743+0000 7ff92badbb80 1 -- 172.21.15.86:0/4019995013 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5620b992e320 con 0x5620b9a1ba20 2024-03-23T17:53:59.746 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.743+0000 7ff8f6a00700 1 -- 172.21.15.86:0/4019995013 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff8fc0122e0 con 0x5620b9a1ba20 2024-03-23T17:53:59.746 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.743+0000 7ff92badbb80 10 client.16248 fetch_fsmap learned FSMap version 34 2024-03-23T17:53:59.747 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.743+0000 7ff92badbb80 1 -- 172.21.15.86:0/4019995013 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5620b979bb00 con 0x5620b9a1ba20 2024-03-23T17:53:59.747 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.743+0000 7ff8f6a00700 1 -- 172.21.15.86:0/4019995013 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5620b992e320 con 0x5620b9a1ba20 2024-03-23T17:53:59.747 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.743+0000 7ff8f6a00700 1 -- 172.21.15.86:0/4019995013 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 34) v1 ==== 7482+0+0 (secure 0 0 0) 0x7ff8fc01c470 con 0x5620b9a1ba20 2024-03-23T17:53:59.747 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.743+0000 7ff92badbb80 10 client.16248 fetch_fsmap finished waiting for FSMap version 34 2024-03-23T17:53:59.747 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.743+0000 7ff92badbb80 10 client.16248 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:53:59.747 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.743+0000 7ff92badbb80 20 client.16248 populate_metadata read hostname 'smithi086' 2024-03-23T17:53:59.747 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.743+0000 7ff92badbb80 1 --2- 172.21.15.86:0/4019995013 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5620b9a36290 0x5620b9a386b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:59.747 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.743+0000 7ff92badbb80 4 client.16248 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:53:59.747 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.743+0000 7ff92badbb80 1 -- 172.21.15.86:0/4019995013 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5620b99666a0 con 0x5620b9a36290 2024-03-23T17:53:59.747 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.744+0000 7ff915600700 1 --2- 172.21.15.86:0/4019995013 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5620b9a36290 0x5620b9a386b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:59.747 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.744+0000 7ff8f6a00700 10 client.16248 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:53:59.748 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.744+0000 7ff915600700 1 --2- 172.21.15.86:0/4019995013 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5620b9a36290 0x5620b9a386b0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f23b979e83dbeeb server_cookie=fbc153613390d30f in_seq=0 out_seq=0 2024-03-23T17:53:59.749 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.745+0000 7ff8f6a00700 1 -- 172.21.15.86:0/4019995013 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5620b99666a0 con 0x5620b9a36290 2024-03-23T17:53:59.749 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.745+0000 7ff8f6a00700 10 client.16248 handle_command_reply: tid=0 2024-03-23T17:53:59.749 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.745+0000 7ff92badbb80 1 client.16248 shutdown 2024-03-23T17:53:59.749 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.745+0000 7ff92badbb80 1 -- 172.21.15.86:0/4019995013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620b9a1ba20 msgr2=0x5620b9a23570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:53:59.749 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.745+0000 7ff92badbb80 1 --2- 172.21.15.86:0/4019995013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620b9a1ba20 0x5620b9a23570 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc012cf0 tx=0x7ff8fc00f3c0 comp rx=0 tx=0).stop 2024-03-23T17:53:59.749 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.745+0000 7ff92badbb80 1 -- 172.21.15.86:0/4019995013 shutdown_connections 2024-03-23T17:53:59.749 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.745+0000 7ff92badbb80 1 --2- 172.21.15.86:0/4019995013 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5620b9a36290 0x5620b9a386b0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:59.749 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.745+0000 7ff92badbb80 1 --2- 172.21.15.86:0/4019995013 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5620b9a268d0 0x5620b9a23ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:59.749 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.745+0000 7ff92badbb80 1 --2- 172.21.15.86:0/4019995013 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620b9a1b100 0x5620b9a26280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:59.749 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.745+0000 7ff92badbb80 1 --2- 172.21.15.86:0/4019995013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620b9a1ba20 0x5620b9a23570 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:59.749 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.745+0000 7ff8f6a00700 0 client.16248 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:53:59.749 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.745+0000 7ff92badbb80 1 -- 172.21.15.86:0/4019995013 >> 172.21.15.86:0/4019995013 conn(0x5620b997b640 msgr2=0x5620b9a1d0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:53:59.749 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.746+0000 7ff92badbb80 1 -- 172.21.15.86:0/4019995013 shutdown_connections 2024-03-23T17:53:59.750 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.746+0000 7ff92badbb80 1 -- 172.21.15.86:0/4019995013 wait complete. 2024-03-23T17:53:59.750 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.746+0000 7ff92badbb80 20 client.16248 trim_cache size 0 max 16384 2024-03-23T17:53:59.760 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.760+0000 7ff92badbb80 1 Processor -- start 2024-03-23T17:53:59.761 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.760+0000 7ff92badbb80 1 -- start start 2024-03-23T17:53:59.761 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.761+0000 7ff92badbb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5620b9a1b100 0x5620b9a3d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:59.761 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.761+0000 7ff92badbb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620b9a1ba20 0x5620b9a2d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:59.761 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.761+0000 7ff92badbb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620b9a268d0 0x5620b9a17460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:59.761 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.761+0000 7ff92badbb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620b9a13400 con 0x5620b9a268d0 2024-03-23T17:53:59.761 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.761+0000 7ff92badbb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620b9a28f10 con 0x5620b9a1ba20 2024-03-23T17:53:59.761 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.761+0000 7ff92badbb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620b9a13140 con 0x5620b9a1b100 2024-03-23T17:53:59.762 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.761+0000 7ff916000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620b9a268d0 0x5620b9a17460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:59.763 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.761+0000 7ff916000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620b9a268d0 0x5620b9a17460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43774/0 (socket says 172.21.15.86:43774) 2024-03-23T17:53:59.763 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.761+0000 7ff916000700 1 -- 172.21.15.86:0/4194095533 learned_addr learned my addr 172.21.15.86:0/4194095533 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:53:59.763 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.761+0000 7ff914c00700 1 --2- 172.21.15.86:0/4194095533 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620b9a1ba20 0x5620b9a2d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:59.763 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.761+0000 7ff915600700 1 --2- 172.21.15.86:0/4194095533 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5620b9a1b100 0x5620b9a3d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:59.763 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.761+0000 7ff914c00700 1 -- 172.21.15.86:0/4194095533 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5620b9a1b100 msgr2=0x5620b9a3d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:53:59.763 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.761+0000 7ff914c00700 1 --2- 172.21.15.86:0/4194095533 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5620b9a1b100 0x5620b9a3d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:59.763 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.761+0000 7ff914c00700 1 -- 172.21.15.86:0/4194095533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620b9a268d0 msgr2=0x5620b9a17460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:53:59.763 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.761+0000 7ff914c00700 1 --2- 172.21.15.86:0/4194095533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620b9a268d0 0x5620b9a17460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:59.763 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.761+0000 7ff914c00700 1 -- 172.21.15.86:0/4194095533 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620b9933e40 con 0x5620b9a1ba20 2024-03-23T17:53:59.763 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.762+0000 7ff914c00700 1 --2- 172.21.15.86:0/4194095533 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620b9a1ba20 0x5620b9a2d2c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc01c880 tx=0x7ff8fc01ca20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=300b295399622cd server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:53:59.763 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.762+0000 7ff903000700 1 -- 172.21.15.86:0/4194095533 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8fc017070 con 0x5620b9a1ba20 2024-03-23T17:53:59.763 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.762+0000 7ff903000700 1 -- 172.21.15.86:0/4194095533 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff8fc018040 con 0x5620b9a1ba20 2024-03-23T17:53:59.763 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.762+0000 7ff903000700 1 -- 172.21.15.86:0/4194095533 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8fc004cd0 con 0x5620b9a1ba20 2024-03-23T17:53:59.764 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.762+0000 7ff92badbb80 1 -- 172.21.15.86:0/4194095533 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620b9a1ba20 msgr2=0x5620b9a2d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:53:59.764 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.762+0000 7ff92badbb80 1 --2- 172.21.15.86:0/4194095533 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620b9a1ba20 0x5620b9a2d2c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc01c880 tx=0x7ff8fc01ca20 comp rx=0 tx=0).stop 2024-03-23T17:53:59.764 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.763+0000 7ff92badbb80 1 -- 172.21.15.86:0/4194095533 shutdown_connections 2024-03-23T17:53:59.764 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.763+0000 7ff92badbb80 1 --2- 172.21.15.86:0/4194095533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620b9a268d0 0x5620b9a17460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:59.764 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.763+0000 7ff92badbb80 1 --2- 172.21.15.86:0/4194095533 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5620b9a1b100 0x5620b9a3d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:59.764 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.763+0000 7ff92badbb80 1 --2- 172.21.15.86:0/4194095533 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620b9a1ba20 0x5620b9a2d2c0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:59.764 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.763+0000 7ff92badbb80 1 -- 172.21.15.86:0/4194095533 >> 172.21.15.86:0/4194095533 conn(0x5620b997b640 msgr2=0x5620b9a16d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:53:59.764 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.763+0000 7ff92badbb80 1 -- 172.21.15.86:0/4194095533 shutdown_connections 2024-03-23T17:53:59.764 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.763+0000 7ff92badbb80 1 -- 172.21.15.86:0/4194095533 wait complete. 2024-03-23T17:53:59.767 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.763+0000 7ff92badbb80 1 -- start start 2024-03-23T17:53:59.767 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.763+0000 7ff92badbb80 1 Processor -- start 2024-03-23T17:53:59.767 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.764+0000 7ff92badbb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620b9a1b100 0x5620b9a2b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:59.767 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.764+0000 7ff92badbb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620b9a1ba20 0x5620b9a2b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:59.767 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.764+0000 7ff92badbb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5620b9a268d0 0x5620b9a80df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:59.767 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.764+0000 7ff92badbb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620b9a81330 con 0x5620b9a1ba20 2024-03-23T17:53:59.767 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.764+0000 7ff92badbb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620b9a81470 con 0x5620b9a1b100 2024-03-23T17:53:59.767 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.764+0000 7ff92badbb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620b9a81630 con 0x5620b9a268d0 2024-03-23T17:53:59.767 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.764+0000 7ff92badbb80 10 client.0 _ll_register_callbacks cb 0x5620b9913410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:53:59.767 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.764+0000 7ff915600700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620b9a1b100 0x5620b9a2b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:59.767 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.764+0000 7ff914c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620b9a1ba20 0x5620b9a2b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:59.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.764+0000 7ff915600700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620b9a1b100 0x5620b9a2b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:35844/0 (socket says 172.21.15.86:35844) 2024-03-23T17:53:59.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.764+0000 7ff915600700 1 -- 172.21.15.86:0/2503799631 learned_addr learned my addr 172.21.15.86:0/2503799631 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:53:59.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.764+0000 7ff916000700 1 --2- 172.21.15.86:0/2503799631 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5620b9a268d0 0x5620b9a80df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:59.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.764+0000 7ff915600700 1 -- 172.21.15.86:0/2503799631 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5620b9a268d0 msgr2=0x5620b9a80df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:53:59.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.764+0000 7ff915600700 1 --2- 172.21.15.86:0/2503799631 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5620b9a268d0 0x5620b9a80df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:59.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.764+0000 7ff915600700 1 -- 172.21.15.86:0/2503799631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620b9a1ba20 msgr2=0x5620b9a2b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:53:59.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.764+0000 7ff915600700 1 --2- 172.21.15.86:0/2503799631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620b9a1ba20 0x5620b9a2b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:59.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.764+0000 7ff915600700 1 -- 172.21.15.86:0/2503799631 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5620b979bb00 con 0x5620b9a1b100 2024-03-23T17:53:59.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.764+0000 7ff914c00700 1 --2- 172.21.15.86:0/2503799631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620b9a1ba20 0x5620b9a2b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:53:59.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.764+0000 7ff916000700 1 --2- 172.21.15.86:0/2503799631 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5620b9a268d0 0x5620b9a80df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T17:53:59.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.764+0000 7ff915600700 1 --2- 172.21.15.86:0/2503799631 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620b9a1b100 0x5620b9a2b490 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff904005e90 tx=0x7ff904005840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0fd285954d68b2c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:53:59.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.764+0000 7ff8f6a00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T17:53:59.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.765+0000 7ff8f6a00700 1 -- 172.21.15.86:0/2503799631 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff90409ac20 con 0x5620b9a1b100 2024-03-23T17:53:59.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.765+0000 7ff92badbb80 1 -- 172.21.15.86:0/2503799631 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff90409e9d0 con 0x5620b9a1b100 2024-03-23T17:53:59.769 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.765+0000 7ff8f6a00700 1 -- 172.21.15.86:0/2503799631 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7ff9040a2f90 con 0x5620b9a1b100 2024-03-23T17:53:59.769 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.765+0000 7ff8f6a00700 1 -- 172.21.15.86:0/2503799631 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff90409a500 con 0x5620b9a1b100 2024-03-23T17:53:59.769 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.765+0000 7ff92badbb80 10 client.26306 fetch_fsmap learned FSMap version 34 2024-03-23T17:53:59.769 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.765+0000 7ff92badbb80 1 -- 172.21.15.86:0/2503799631 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5620b97f3c70 con 0x5620b9a1b100 2024-03-23T17:53:59.769 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.765+0000 7ff8f6a00700 1 -- 172.21.15.86:0/2503799631 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff90409e9d0 con 0x5620b9a1b100 2024-03-23T17:53:59.769 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.765+0000 7ff8f6a00700 1 -- 172.21.15.86:0/2503799631 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 34) v1 ==== 7482+0+0 (secure 0 0 0) 0x7ff9040b3260 con 0x5620b9a1b100 2024-03-23T17:53:59.769 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.765+0000 7ff92badbb80 10 client.26306 fetch_fsmap finished waiting for FSMap version 34 2024-03-23T17:53:59.769 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.765+0000 7ff92badbb80 10 client.26306 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:53:59.769 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.765+0000 7ff92badbb80 20 client.26306 populate_metadata read hostname 'smithi086' 2024-03-23T17:53:59.769 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.765+0000 7ff92badbb80 1 --2- 172.21.15.86:0/2503799631 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5620b9a889c0 0x5620b9a8ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:53:59.769 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.765+0000 7ff92badbb80 4 client.26306 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T17:53:59.769 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.765+0000 7ff92badbb80 1 -- 172.21.15.86:0/2503799631 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5620b97bcc90 con 0x5620b9a889c0 2024-03-23T17:53:59.769 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.766+0000 7ff914c00700 1 --2- 172.21.15.86:0/2503799631 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5620b9a889c0 0x5620b9a8ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:53:59.769 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.766+0000 7ff914c00700 1 --2- 172.21.15.86:0/2503799631 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5620b9a889c0 0x5620b9a8ade0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fac823ff6849f9bb server_cookie=47c427371ec5cb1c in_seq=0 out_seq=0 2024-03-23T17:53:59.770 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.766+0000 7ff8f6a00700 10 client.26306 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:53:59.797 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.796+0000 7ff8f6a00700 1 -- 172.21.15.86:0/2503799631 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5620b97bcc90 con 0x5620b9a889c0 2024-03-23T17:53:59.797 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.796+0000 7ff8f6a00700 10 client.26306 handle_command_reply: tid=0 2024-03-23T17:53:59.797 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.796+0000 7ff92badbb80 1 client.26306 shutdown 2024-03-23T17:53:59.797 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.796+0000 7ff92badbb80 1 -- 172.21.15.86:0/2503799631 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620b9a1b100 msgr2=0x5620b9a2b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:53:59.797 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.796+0000 7ff92badbb80 1 --2- 172.21.15.86:0/2503799631 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620b9a1b100 0x5620b9a2b490 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff904005e90 tx=0x7ff904005840 comp rx=0 tx=0).stop 2024-03-23T17:53:59.797 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.796+0000 7ff92badbb80 1 -- 172.21.15.86:0/2503799631 shutdown_connections 2024-03-23T17:53:59.797 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.796+0000 7ff92badbb80 1 --2- 172.21.15.86:0/2503799631 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5620b9a889c0 0x5620b9a8ade0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:59.797 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.796+0000 7ff92badbb80 1 --2- 172.21.15.86:0/2503799631 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5620b9a268d0 0x5620b9a80df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:59.797 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.796+0000 7ff92badbb80 1 --2- 172.21.15.86:0/2503799631 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620b9a1b100 0x5620b9a2b490 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:59.797 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.796+0000 7ff92badbb80 1 --2- 172.21.15.86:0/2503799631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620b9a1ba20 0x5620b9a2b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:59.798 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.796+0000 7ff92badbb80 1 -- 172.21.15.86:0/2503799631 >> 172.21.15.86:0/2503799631 conn(0x5620b997b640 msgr2=0x5620b9a103e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:53:59.798 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.796+0000 7ff8f6a00700 0 client.26306 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:53:59.798 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.796+0000 7ff92badbb80 1 -- 172.21.15.86:0/2503799631 shutdown_connections 2024-03-23T17:53:59.798 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.797+0000 7ff92badbb80 1 -- 172.21.15.86:0/2503799631 wait complete. 2024-03-23T17:53:59.798 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.797+0000 7ff92badbb80 20 client.26306 trim_cache size 0 max 16384 2024-03-23T17:53:59.798 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-03-23T17:53:59.798 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-03-23T17:53:59.798 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-03-23T17:53:59.798 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-03-23T17:53:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.797+0000 7ff918000700 1 -- 172.21.15.86:0/174451366 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7ff8f80a1290 msgr2=0x7ff8f80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:53:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.797+0000 7ff918000700 1 --2- 172.21.15.86:0/174451366 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7ff8f80a1290 0x7ff8f80a3750 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff908003e30 tx=0x7ff90800b040 comp rx=0 tx=0).stop 2024-03-23T17:53:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.797+0000 7ff918000700 1 -- 172.21.15.86:0/174451366 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff910100f10 msgr2=0x7ff910199400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:53:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.797+0000 7ff918000700 1 --2- 172.21.15.86:0/174451366 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff910100f10 0x7ff910199400 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff904005950 tx=0x7ff90400fa00 comp rx=0 tx=0).stop 2024-03-23T17:53:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.798+0000 7ff918000700 1 -- 172.21.15.86:0/174451366 shutdown_connections 2024-03-23T17:53:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.798+0000 7ff918000700 1 --2- 172.21.15.86:0/174451366 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff91019dce0 0x7ff91019e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.798+0000 7ff918000700 1 --2- 172.21.15.86:0/174451366 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7ff8f80a1290 0x7ff8f80a3750 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.798+0000 7ff918000700 1 --2- 172.21.15.86:0/174451366 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff910100f10 0x7ff910199400 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:59.800 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.798+0000 7ff918000700 1 --2- 172.21.15.86:0/174451366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff910106030 0x7ff910199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:53:59.800 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.798+0000 7ff918000700 1 -- 172.21.15.86:0/174451366 >> 172.21.15.86:0/174451366 conn(0x7ff9100fc7b0 msgr2=0x7ff9100fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:53:59.800 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.798+0000 7ff918000700 1 -- 172.21.15.86:0/174451366 shutdown_connections 2024-03-23T17:53:59.800 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:53:59.798+0000 7ff918000700 1 -- 172.21.15.86:0/174451366 wait complete. 2024-03-23T17:53:59.891 INFO:tasks.workunit.client.0.smithi086.stdout: 1 96483 3.17 MB/sec execute 24 sec latency 2188.436 ms 2024-03-23T17:54:00.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:53:59 smithi086 ceph-mon[38221]: pgmap v103: 97 pgs: 97 active+clean; 49 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 869 KiB/s rd, 984 KiB/s wr, 118 op/s 2024-03-23T17:54:00.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:53:59 smithi092 ceph-mon[79089]: pgmap v103: 97 pgs: 97 active+clean; 49 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 869 KiB/s rd, 984 KiB/s wr, 118 op/s 2024-03-23T17:54:00.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:53:59 smithi152 ceph-mon[47949]: pgmap v103: 97 pgs: 97 active+clean; 49 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 869 KiB/s rd, 984 KiB/s wr, 118 op/s 2024-03-23T17:54:00.891 INFO:tasks.workunit.client.0.smithi086.stdout: 1 96539 3.05 MB/sec execute 25 sec latency 2649.365 ms 2024-03-23T17:54:01.891 INFO:tasks.workunit.client.0.smithi086.stdout: 1 96607 2.94 MB/sec execute 26 sec latency 937.467 ms 2024-03-23T17:54:02.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:54:01 smithi086 ceph-mon[38221]: pgmap v104: 97 pgs: 97 active+clean; 51 MiB data, 673 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 171 op/s 2024-03-23T17:54:02.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:54:01 smithi092 ceph-mon[79089]: pgmap v104: 97 pgs: 97 active+clean; 51 MiB data, 673 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 171 op/s 2024-03-23T17:54:02.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:54:01 smithi152 ceph-mon[47949]: pgmap v104: 97 pgs: 97 active+clean; 51 MiB data, 673 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 171 op/s 2024-03-23T17:54:02.891 INFO:tasks.workunit.client.0.smithi086.stdout: 1 96607 2.83 MB/sec execute 27 sec latency 1937.587 ms 2024-03-23T17:54:03.891 INFO:tasks.workunit.client.0.smithi086.stdout: 1 96607 2.73 MB/sec execute 28 sec latency 2937.654 ms 2024-03-23T17:54:04.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:54:03 smithi086 ceph-mon[38221]: pgmap v105: 97 pgs: 97 active+clean; 51 MiB data, 673 MiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 861 KiB/s wr, 81 op/s 2024-03-23T17:54:04.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:54:03 smithi092 ceph-mon[79089]: pgmap v105: 97 pgs: 97 active+clean; 51 MiB data, 673 MiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 861 KiB/s wr, 81 op/s 2024-03-23T17:54:04.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:54:03 smithi152 ceph-mon[47949]: pgmap v105: 97 pgs: 97 active+clean; 51 MiB data, 673 MiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 861 KiB/s wr, 81 op/s 2024-03-23T17:54:04.891 INFO:tasks.workunit.client.0.smithi086.stdout: 1 96607 2.64 MB/sec execute 29 sec latency 3937.746 ms 2024-03-23T17:54:05.891 INFO:tasks.workunit.client.0.smithi086.stdout: 1 96607 2.55 MB/sec execute 30 sec latency 4937.819 ms 2024-03-23T17:54:06.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:54:05 smithi086 ceph-mon[38221]: pgmap v106: 97 pgs: 97 active+clean; 47 MiB data, 669 MiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 940 KiB/s wr, 121 op/s 2024-03-23T17:54:06.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:54:05 smithi086 ceph-mon[38221]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T17:54:06.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:54:05 smithi092 ceph-mon[79089]: pgmap v106: 97 pgs: 97 active+clean; 47 MiB data, 669 MiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 940 KiB/s wr, 121 op/s 2024-03-23T17:54:06.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:54:05 smithi092 ceph-mon[79089]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T17:54:06.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:54:05 smithi152 ceph-mon[47949]: pgmap v106: 97 pgs: 97 active+clean; 47 MiB data, 669 MiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 940 KiB/s wr, 121 op/s 2024-03-23T17:54:06.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:54:05 smithi152 ceph-mon[47949]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T17:54:06.891 INFO:tasks.workunit.client.0.smithi086.stdout: 1 98314 2.70 MB/sec execute 31 sec latency 4951.969 ms 2024-03-23T17:54:07.892 INFO:tasks.workunit.client.0.smithi086.stdout: 1 99905 2.98 MB/sec execute 32 sec latency 12.096 ms 2024-03-23T17:54:08.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:54:07 smithi086 ceph-mon[38221]: pgmap v107: 97 pgs: 97 active+clean; 47 MiB data, 669 MiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 628 KiB/s wr, 93 op/s 2024-03-23T17:54:08.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:54:07 smithi092 ceph-mon[79089]: pgmap v107: 97 pgs: 97 active+clean; 47 MiB data, 669 MiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 628 KiB/s wr, 93 op/s 2024-03-23T17:54:08.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:54:07 smithi152 ceph-mon[47949]: pgmap v107: 97 pgs: 97 active+clean; 47 MiB data, 669 MiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 628 KiB/s wr, 93 op/s 2024-03-23T17:54:08.892 INFO:tasks.workunit.client.0.smithi086.stdout: 1 100028 2.89 MB/sec execute 33 sec latency 911.680 ms 2024-03-23T17:54:09.892 INFO:tasks.workunit.client.0.smithi086.stdout: 1 100028 2.80 MB/sec execute 34 sec latency 1911.761 ms 2024-03-23T17:54:10.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:54:09 smithi086 ceph-mon[38221]: pgmap v108: 97 pgs: 97 active+clean; 47 MiB data, 669 MiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 628 KiB/s wr, 93 op/s 2024-03-23T17:54:10.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:54:09 smithi092 ceph-mon[79089]: pgmap v108: 97 pgs: 97 active+clean; 47 MiB data, 669 MiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 628 KiB/s wr, 93 op/s 2024-03-23T17:54:10.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:54:09 smithi152 ceph-mon[47949]: pgmap v108: 97 pgs: 97 active+clean; 47 MiB data, 669 MiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 628 KiB/s wr, 93 op/s 2024-03-23T17:54:10.892 INFO:tasks.workunit.client.0.smithi086.stdout: 1 100028 2.72 MB/sec execute 35 sec latency 2911.814 ms 2024-03-23T17:54:11.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:54:10 smithi086 ceph-mon[38221]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T17:54:11.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:54:10 smithi092 ceph-mon[79089]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T17:54:11.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:54:10 smithi152 ceph-mon[47949]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T17:54:11.894 INFO:tasks.workunit.client.0.smithi086.stdout: 1 100084 2.65 MB/sec execute 36 sec latency 2932.589 ms 2024-03-23T17:54:12.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:54:11 smithi086 ceph-mon[38221]: pgmap v109: 97 pgs: 97 active+clean; 49 MiB data, 671 MiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 1.2 MiB/s wr, 160 op/s 2024-03-23T17:54:12.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:54:11 smithi092 ceph-mon[79089]: pgmap v109: 97 pgs: 97 active+clean; 49 MiB data, 671 MiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 1.2 MiB/s wr, 160 op/s 2024-03-23T17:54:12.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:54:11 smithi152 ceph-mon[47949]: pgmap v109: 97 pgs: 97 active+clean; 49 MiB data, 671 MiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 1.2 MiB/s wr, 160 op/s 2024-03-23T17:54:12.892 INFO:tasks.workunit.client.0.smithi086.stdout: 1 100084 2.58 MB/sec execute 37 sec latency 1941.009 ms 2024-03-23T17:54:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.957+0000 7fad0be00700 1 -- 172.21.15.86:0/2236897424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad041018a0 msgr2=0x7fad041058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:54:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.957+0000 7fad0be00700 1 --2- 172.21.15.86:0/2236897424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad041018a0 0x7fad041058e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7facf0009990 tx=0x7facf0009ca0 comp rx=0 tx=0).stop 2024-03-23T17:54:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.957+0000 7fad0be00700 1 -- 172.21.15.86:0/2236897424 shutdown_connections 2024-03-23T17:54:12.958 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.957+0000 7fad0be00700 1 --2- 172.21.15.86:0/2236897424 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad04106010 0x7fad0410db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:12.958 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.957+0000 7fad0be00700 1 --2- 172.21.15.86:0/2236897424 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad04100ef0 0x7fad041012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:12.958 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.957+0000 7fad0be00700 1 --2- 172.21.15.86:0/2236897424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad041018a0 0x7fad041058e0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:12.958 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.957+0000 7fad0be00700 1 -- 172.21.15.86:0/2236897424 >> 172.21.15.86:0/2236897424 conn(0x7fad040fc750 msgr2=0x7fad040feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:54:12.958 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.957+0000 7fad0be00700 1 -- 172.21.15.86:0/2236897424 shutdown_connections 2024-03-23T17:54:12.958 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.957+0000 7fad0be00700 1 -- 172.21.15.86:0/2236897424 wait complete. 2024-03-23T17:54:12.960 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.958+0000 7fad0be00700 1 Processor -- start 2024-03-23T17:54:12.960 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.958+0000 7fad0be00700 1 -- start start 2024-03-23T17:54:12.960 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.958+0000 7fad0be00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad04100ef0 0x7fad040730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:54:12.960 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.958+0000 7fad0be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad04106010 0x7fad040735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:54:12.960 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.958+0000 7fad0be00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad04073b30 0x7fad041a5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:54:12.960 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.958+0000 7fad0be00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad041a5960 con 0x7fad04106010 2024-03-23T17:54:12.961 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.958+0000 7fad0be00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad041a5ad0 con 0x7fad04100ef0 2024-03-23T17:54:12.961 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.958+0000 7fad0be00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad041a5c90 con 0x7fad04073b30 2024-03-23T17:54:12.961 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.959+0000 7fad08a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad04106010 0x7fad040735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:54:12.961 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.959+0000 7fad08a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad04106010 0x7fad040735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46172/0 (socket says 172.21.15.86:46172) 2024-03-23T17:54:12.961 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.959+0000 7fad09400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad04100ef0 0x7fad040730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:54:12.961 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.959+0000 7fad08a00700 1 -- 172.21.15.86:0/3102426729 learned_addr learned my addr 172.21.15.86:0/3102426729 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:54:12.961 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.959+0000 7fad09e00700 1 --2- 172.21.15.86:0/3102426729 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad04073b30 0x7fad041a5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:54:12.961 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.959+0000 7fad08a00700 1 -- 172.21.15.86:0/3102426729 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad04073b30 msgr2=0x7fad041a5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:54:12.961 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.959+0000 7fad08a00700 1 --2- 172.21.15.86:0/3102426729 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad04073b30 0x7fad041a5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:12.961 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.959+0000 7fad08a00700 1 -- 172.21.15.86:0/3102426729 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad04100ef0 msgr2=0x7fad040730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:54:12.961 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.959+0000 7fad08a00700 1 --2- 172.21.15.86:0/3102426729 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad04100ef0 0x7fad040730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.959+0000 7fad08a00700 1 -- 172.21.15.86:0/3102426729 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facf00095f0 con 0x7fad04106010 2024-03-23T17:54:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.959+0000 7fad09400700 1 --2- 172.21.15.86:0/3102426729 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad04100ef0 0x7fad040730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:54:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.959+0000 7fad08a00700 1 --2- 172.21.15.86:0/3102426729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad04106010 0x7fad040735f0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7facf0000c00 tx=0x7facf0005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:54:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.960+0000 7fad02000700 1 -- 172.21.15.86:0/3102426729 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf001d070 con 0x7fad04106010 2024-03-23T17:54:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.960+0000 7fad02000700 1 -- 172.21.15.86:0/3102426729 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7facf0022470 con 0x7fad04106010 2024-03-23T17:54:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.960+0000 7fad02000700 1 -- 172.21.15.86:0/3102426729 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf0017650 con 0x7fad04106010 2024-03-23T17:54:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.960+0000 7fad0be00700 1 -- 172.21.15.86:0/3102426729 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad041a5f10 con 0x7fad04106010 2024-03-23T17:54:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.960+0000 7fad0be00700 1 -- 172.21.15.86:0/3102426729 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad041a6280 con 0x7fad04106010 2024-03-23T17:54:12.963 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.963+0000 7fad02000700 1 -- 172.21.15.86:0/3102426729 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7facf000fad0 con 0x7fad04106010 2024-03-23T17:54:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.964+0000 7fad1f93cb80 1 Processor -- start 2024-03-23T17:54:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.964+0000 7fad02000700 1 --2- 172.21.15.86:0/3102426729 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7facec0a1290 0x7facec0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:54:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.964+0000 7fad1f93cb80 1 -- start start 2024-03-23T17:54:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.964+0000 7fad02000700 1 -- 172.21.15.86:0/3102426729 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7facf00ce040 con 0x7fad04106010 2024-03-23T17:54:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.965+0000 7fad09e00700 1 --2- 172.21.15.86:0/3102426729 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7facec0a1290 0x7facec0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:54:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.965+0000 7fad1f93cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55875d6bd100 0x55875d6bd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:54:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.965+0000 7fad1f93cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55875d6bda20 0x55875d6c8390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:54:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.965+0000 7fad1f93cb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55875d6c88d0 0x55875d6cacc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:54:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.965+0000 7fad1f93cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55875d616930 con 0x55875d6bda20 2024-03-23T17:54:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.965+0000 7fad1f93cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55875d6142f0 con 0x55875d6bd100 2024-03-23T17:54:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.965+0000 7fad1f93cb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55875d60fbe0 con 0x55875d6c88d0 2024-03-23T17:54:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.965+0000 7fad08a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55875d6bda20 0x55875d6c8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:54:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.965+0000 7fad08a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55875d6bda20 0x55875d6c8390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46182/0 (socket says 172.21.15.86:46182) 2024-03-23T17:54:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.965+0000 7fad09e00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55875d6c88d0 0x55875d6cacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:54:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.965+0000 7fad08a00700 1 -- 172.21.15.86:0/2250139329 learned_addr learned my addr 172.21.15.86:0/2250139329 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:54:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.965+0000 7fad09400700 1 --2- 172.21.15.86:0/2250139329 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55875d6bd100 0x55875d6bd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:54:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.965+0000 7fad09e00700 1 --2- 172.21.15.86:0/3102426729 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7facec0a1290 0x7facec0a3750 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7facfc003ee0 tx=0x7facfc009100 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:54:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.965+0000 7fad08a00700 1 -- 172.21.15.86:0/2250139329 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55875d6c88d0 msgr2=0x55875d6cacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:54:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.965+0000 7fad08a00700 1 --2- 172.21.15.86:0/2250139329 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55875d6c88d0 0x55875d6cacc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.965+0000 7fad08a00700 1 -- 172.21.15.86:0/2250139329 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55875d6bd100 msgr2=0x55875d6bd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:54:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.965+0000 7fad08a00700 1 --2- 172.21.15.86:0/2250139329 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55875d6bd100 0x55875d6bd4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.965+0000 7fad08a00700 1 -- 172.21.15.86:0/2250139329 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55875d491350 con 0x55875d6bda20 2024-03-23T17:54:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.965+0000 7fad09400700 1 --2- 172.21.15.86:0/2250139329 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55875d6bd100 0x55875d6bd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:54:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.966+0000 7fad08a00700 1 --2- 172.21.15.86:0/2250139329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55875d6bda20 0x55875d6c8390 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7facf0009350 tx=0x7facf000b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=581a23cb57e0dc4b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:54:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.966+0000 7facf6600700 1 -- 172.21.15.86:0/2250139329 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf00b0070 con 0x55875d6bda20 2024-03-23T17:54:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.966+0000 7facf6600700 1 -- 172.21.15.86:0/2250139329 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7facf009ac50 con 0x55875d6bda20 2024-03-23T17:54:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.966+0000 7facf6600700 1 -- 172.21.15.86:0/2250139329 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf0022850 con 0x55875d6bda20 2024-03-23T17:54:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.967+0000 7fad1f93cb80 1 -- 172.21.15.86:0/2250139329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55875d6bda20 msgr2=0x55875d6c8390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:54:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.967+0000 7fad1f93cb80 1 --2- 172.21.15.86:0/2250139329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55875d6bda20 0x55875d6c8390 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7facf0009350 tx=0x7facf000b650 comp rx=0 tx=0).stop 2024-03-23T17:54:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.967+0000 7fad1f93cb80 1 -- 172.21.15.86:0/2250139329 shutdown_connections 2024-03-23T17:54:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.967+0000 7fad1f93cb80 1 --2- 172.21.15.86:0/2250139329 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55875d6c88d0 0x55875d6cacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.967+0000 7fad1f93cb80 1 --2- 172.21.15.86:0/2250139329 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55875d6bd100 0x55875d6bd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.967+0000 7fad1f93cb80 1 --2- 172.21.15.86:0/2250139329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55875d6bda20 0x55875d6c8390 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.967+0000 7fad1f93cb80 1 -- 172.21.15.86:0/2250139329 >> 172.21.15.86:0/2250139329 conn(0x55875d61d640 msgr2=0x55875d6b9520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:54:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.967+0000 7fad1f93cb80 1 -- 172.21.15.86:0/2250139329 shutdown_connections 2024-03-23T17:54:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.967+0000 7fad1f93cb80 1 -- 172.21.15.86:0/2250139329 wait complete. 2024-03-23T17:54:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.967+0000 7fad1f93cb80 1 -- start start 2024-03-23T17:54:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.968+0000 7fad1f93cb80 1 Processor -- start 2024-03-23T17:54:12.973 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.968+0000 7fad1f93cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55875d6bd100 0x55875d6c8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:54:12.974 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.968+0000 7fad1f93cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55875d6bda20 0x55875d6c55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:54:12.974 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.968+0000 7fad1f93cb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55875d6c88d0 0x55875d6c5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:54:12.974 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.968+0000 7fad1f93cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55875d5abd20 con 0x55875d6bd100 2024-03-23T17:54:12.974 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.968+0000 7fad1f93cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55875d5ab9c0 con 0x55875d6bda20 2024-03-23T17:54:12.974 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.968+0000 7fad1f93cb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55875d493640 con 0x55875d6c88d0 2024-03-23T17:54:12.974 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.968+0000 7fad1f93cb80 10 client.0 _ll_register_callbacks cb 0x55875d5b5410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:54:12.974 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.968+0000 7fad09400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55875d6bd100 0x55875d6c8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:54:12.974 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.968+0000 7fad09400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55875d6bd100 0x55875d6c8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46186/0 (socket says 172.21.15.86:46186) 2024-03-23T17:54:12.974 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.968+0000 7fad09400700 1 -- 172.21.15.86:0/2971094350 learned_addr learned my addr 172.21.15.86:0/2971094350 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:54:12.974 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.968+0000 7fad09e00700 1 --2- 172.21.15.86:0/2971094350 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55875d6c88d0 0x55875d6c5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:54:12.975 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.969+0000 7fad08a00700 1 --2- 172.21.15.86:0/2971094350 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55875d6bda20 0x55875d6c55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:54:12.975 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.969+0000 7fad09400700 1 -- 172.21.15.86:0/2971094350 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55875d6c88d0 msgr2=0x55875d6c5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:54:12.975 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.969+0000 7fad09400700 1 --2- 172.21.15.86:0/2971094350 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55875d6c88d0 0x55875d6c5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:12.975 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.969+0000 7fad09400700 1 -- 172.21.15.86:0/2971094350 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55875d6bda20 msgr2=0x55875d6c55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:54:12.975 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.969+0000 7fad09400700 1 --2- 172.21.15.86:0/2971094350 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55875d6bda20 0x55875d6c55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:12.975 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.969+0000 7fad09400700 1 -- 172.21.15.86:0/2971094350 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad041a6280 con 0x55875d6bd100 2024-03-23T17:54:12.975 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.969+0000 7fad08a00700 1 --2- 172.21.15.86:0/2971094350 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55875d6bda20 0x55875d6c55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:54:12.975 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.969+0000 7fad09400700 1 --2- 172.21.15.86:0/2971094350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55875d6bd100 0x55875d6c8300 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7facf80129c0 tx=0x7facf8012cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b4a5dd491838ca1 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:54:12.975 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.969+0000 7faceaa00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-23T17:54:12.975 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.969+0000 7faceaa00700 1 -- 172.21.15.86:0/2971094350 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf8017070 con 0x55875d6bd100 2024-03-23T17:54:12.975 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.969+0000 7faceaa00700 1 -- 172.21.15.86:0/2971094350 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7facf800f8b0 con 0x55875d6bd100 2024-03-23T17:54:12.976 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.969+0000 7fad1f93cb80 1 -- 172.21.15.86:0/2971094350 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55875d5d0320 con 0x55875d6bd100 2024-03-23T17:54:12.976 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.970+0000 7faceaa00700 1 -- 172.21.15.86:0/2971094350 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7facf80122e0 con 0x55875d6bd100 2024-03-23T17:54:12.976 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.970+0000 7fad1f93cb80 10 client.16266 fetch_fsmap learned FSMap version 36 2024-03-23T17:54:12.976 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.970+0000 7fad1f93cb80 1 -- 172.21.15.86:0/2971094350 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55875d43db00 con 0x55875d6bd100 2024-03-23T17:54:12.976 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.970+0000 7faceaa00700 1 -- 172.21.15.86:0/2971094350 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55875d5d0320 con 0x55875d6bd100 2024-03-23T17:54:12.976 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.970+0000 7faceaa00700 1 -- 172.21.15.86:0/2971094350 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7facf8010c50 con 0x55875d6bd100 2024-03-23T17:54:12.976 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.970+0000 7fad1f93cb80 10 client.16266 fetch_fsmap finished waiting for FSMap version 36 2024-03-23T17:54:12.976 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.970+0000 7fad1f93cb80 10 client.16266 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:54:12.976 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.970+0000 7fad1f93cb80 20 client.16266 populate_metadata read hostname 'smithi086' 2024-03-23T17:54:12.976 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.970+0000 7fad1f93cb80 1 --2- 172.21.15.86:0/2971094350 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55875d6d82d0 0x55875d6da6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:54:12.976 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.970+0000 7fad1f93cb80 4 client.16266 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:54:12.976 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.970+0000 7fad1f93cb80 1 -- 172.21.15.86:0/2971094350 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55875d6086a0 con 0x55875d6d82d0 2024-03-23T17:54:12.976 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.971+0000 7fad08a00700 1 --2- 172.21.15.86:0/2971094350 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55875d6d82d0 0x55875d6da6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:54:12.976 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.971+0000 7fad08a00700 1 --2- 172.21.15.86:0/2971094350 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55875d6d82d0 0x55875d6da6f0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b04672f5013e211 server_cookie=bf98a40ffe166e58 in_seq=0 out_seq=0 2024-03-23T17:54:12.976 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.971+0000 7faceaa00700 10 client.16266 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:54:12.977 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.972+0000 7faceaa00700 1 -- 172.21.15.86:0/2971094350 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55875d6086a0 con 0x55875d6d82d0 2024-03-23T17:54:12.977 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.972+0000 7faceaa00700 10 client.16266 handle_command_reply: tid=0 2024-03-23T17:54:12.977 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.972+0000 7fad1f93cb80 1 client.16266 shutdown 2024-03-23T17:54:12.977 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.972+0000 7fad1f93cb80 1 -- 172.21.15.86:0/2971094350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55875d6bd100 msgr2=0x55875d6c8300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:54:12.977 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.972+0000 7fad1f93cb80 1 --2- 172.21.15.86:0/2971094350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55875d6bd100 0x55875d6c8300 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7facf80129c0 tx=0x7facf8012cd0 comp rx=0 tx=0).stop 2024-03-23T17:54:12.977 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.972+0000 7fad1f93cb80 1 -- 172.21.15.86:0/2971094350 shutdown_connections 2024-03-23T17:54:12.977 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.972+0000 7fad1f93cb80 1 --2- 172.21.15.86:0/2971094350 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55875d6d82d0 0x55875d6da6f0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:12.977 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.972+0000 7fad1f93cb80 1 --2- 172.21.15.86:0/2971094350 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55875d6c88d0 0x55875d6c5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:12.977 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.972+0000 7fad1f93cb80 1 --2- 172.21.15.86:0/2971094350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55875d6bd100 0x55875d6c8300 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:12.977 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.972+0000 7fad1f93cb80 1 --2- 172.21.15.86:0/2971094350 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55875d6bda20 0x55875d6c55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:12.977 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.972+0000 7fad1f93cb80 1 -- 172.21.15.86:0/2971094350 >> 172.21.15.86:0/2971094350 conn(0x55875d61d640 msgr2=0x55875d6bf0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:54:12.977 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.972+0000 7faceaa00700 0 client.16266 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:54:12.977 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.972+0000 7fad1f93cb80 1 -- 172.21.15.86:0/2971094350 shutdown_connections 2024-03-23T17:54:12.977 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.972+0000 7fad1f93cb80 1 -- 172.21.15.86:0/2971094350 wait complete. 2024-03-23T17:54:12.978 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.972+0000 7fad1f93cb80 20 client.16266 trim_cache size 0 max 16384 2024-03-23T17:54:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.987+0000 7fad1f93cb80 1 Processor -- start 2024-03-23T17:54:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.987+0000 7fad1f93cb80 1 -- start start 2024-03-23T17:54:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.987+0000 7fad1f93cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55875d6bd100 0x55875d6cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:54:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.987+0000 7fad1f93cb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55875d6bda20 0x55875d6b3500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:54:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.987+0000 7fad1f93cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55875d6c88d0 0x55875d6c08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:54:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.987+0000 7fad1f93cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55875d6d5930 con 0x55875d6c88d0 2024-03-23T17:54:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.987+0000 7fad1f93cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55875d6b5400 con 0x55875d6bd100 2024-03-23T17:54:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.987+0000 7fad1f93cb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55875d6df410 con 0x55875d6bda20 2024-03-23T17:54:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.987+0000 7fad09e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55875d6c88d0 0x55875d6c08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:54:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.988+0000 7fad09400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55875d6bd100 0x55875d6cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:54:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.988+0000 7fad09400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55875d6bd100 0x55875d6cf2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:56614/0 (socket says 172.21.15.86:56614) 2024-03-23T17:54:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.988+0000 7fad09e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55875d6c88d0 0x55875d6c08d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46188/0 (socket says 172.21.15.86:46188) 2024-03-23T17:54:12.989 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.988+0000 7fad09400700 1 -- 172.21.15.86:0/3744784499 learned_addr learned my addr 172.21.15.86:0/3744784499 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:54:12.989 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.988+0000 7fad08a00700 1 --2- 172.21.15.86:0/3744784499 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55875d6bda20 0x55875d6b3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:54:12.989 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.988+0000 7fad09400700 1 -- 172.21.15.86:0/3744784499 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55875d6bda20 msgr2=0x55875d6b3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:54:12.989 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.988+0000 7fad09400700 1 --2- 172.21.15.86:0/3744784499 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55875d6bda20 0x55875d6b3500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:12.989 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.988+0000 7fad09400700 1 -- 172.21.15.86:0/3744784499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55875d6c88d0 msgr2=0x55875d6c08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:54:12.989 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.988+0000 7fad09400700 1 --2- 172.21.15.86:0/3744784499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55875d6c88d0 0x55875d6c08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:12.989 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.988+0000 7fad09400700 1 -- 172.21.15.86:0/3744784499 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55875d5d5e40 con 0x55875d6bd100 2024-03-23T17:54:12.993 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.988+0000 7fad09400700 1 --2- 172.21.15.86:0/3744784499 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55875d6bd100 0x55875d6cf2c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7facf8009350 tx=0x7facf8009380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc7cb3c5cc0e1e5c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:54:12.993 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.988+0000 7facf7000700 1 -- 172.21.15.86:0/3744784499 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf8017070 con 0x55875d6bd100 2024-03-23T17:54:12.993 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.989+0000 7facf7000700 1 -- 172.21.15.86:0/3744784499 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7facf8018040 con 0x55875d6bd100 2024-03-23T17:54:12.993 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.989+0000 7facf7000700 1 -- 172.21.15.86:0/3744784499 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf8005280 con 0x55875d6bd100 2024-03-23T17:54:12.993 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.989+0000 7fad1f93cb80 1 -- 172.21.15.86:0/3744784499 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55875d6bd100 msgr2=0x55875d6cf2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:54:12.993 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.989+0000 7fad1f93cb80 1 --2- 172.21.15.86:0/3744784499 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55875d6bd100 0x55875d6cf2c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7facf8009350 tx=0x7facf8009380 comp rx=0 tx=0).stop 2024-03-23T17:54:12.993 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.989+0000 7fad1f93cb80 1 -- 172.21.15.86:0/3744784499 shutdown_connections 2024-03-23T17:54:12.993 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.989+0000 7fad1f93cb80 1 --2- 172.21.15.86:0/3744784499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55875d6c88d0 0x55875d6c08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:12.993 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.989+0000 7fad1f93cb80 1 --2- 172.21.15.86:0/3744784499 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55875d6bd100 0x55875d6cf2c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:12.994 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.989+0000 7fad1f93cb80 1 --2- 172.21.15.86:0/3744784499 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55875d6bda20 0x55875d6b3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:12.994 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.989+0000 7fad1f93cb80 1 -- 172.21.15.86:0/3744784499 >> 172.21.15.86:0/3744784499 conn(0x55875d61d640 msgr2=0x55875d6d3890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:54:12.994 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.989+0000 7fad1f93cb80 1 -- 172.21.15.86:0/3744784499 shutdown_connections 2024-03-23T17:54:12.994 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.989+0000 7fad1f93cb80 1 -- 172.21.15.86:0/3744784499 wait complete. 2024-03-23T17:54:12.994 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.990+0000 7fad1f93cb80 1 -- start start 2024-03-23T17:54:12.994 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.990+0000 7fad1f93cb80 1 Processor -- start 2024-03-23T17:54:12.994 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.990+0000 7fad1f93cb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55875d6bd100 0x55875d6c4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:54:12.994 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.991+0000 7fad1f93cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55875d6bda20 0x55875d6c5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:54:12.994 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.991+0000 7fad1f93cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55875d6c88d0 0x55875d6c55b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:54:12.994 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.991+0000 7fad1f93cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55875d6ce4b0 con 0x55875d6c88d0 2024-03-23T17:54:12.994 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.991+0000 7fad1f93cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55875d6ce620 con 0x55875d6bda20 2024-03-23T17:54:12.994 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.991+0000 7fad1f93cb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55875d6ce7e0 con 0x55875d6bd100 2024-03-23T17:54:12.994 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.991+0000 7fad1f93cb80 10 client.0 _ll_register_callbacks cb 0x55875d5b5410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:54:12.995 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.991+0000 7fad09e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55875d6c88d0 0x55875d6c55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:54:12.995 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.991+0000 7fad08a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55875d6bda20 0x55875d6c5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:54:12.995 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.991+0000 7fad09400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55875d6bd100 0x55875d6c4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:54:12.995 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.991+0000 7fad09e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55875d6c88d0 0x55875d6c55b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46204/0 (socket says 172.21.15.86:46204) 2024-03-23T17:54:12.995 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.991+0000 7fad09e00700 1 -- 172.21.15.86:0/3235771474 learned_addr learned my addr 172.21.15.86:0/3235771474 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:54:12.995 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.991+0000 7fad08a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55875d6bda20 0x55875d6c5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:56620/0 (socket says 172.21.15.86:56620) 2024-03-23T17:54:12.995 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.991+0000 7fad09400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55875d6bd100 0x55875d6c4b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:42470/0 (socket says 172.21.15.86:42470) 2024-03-23T17:54:12.995 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.991+0000 7fad09e00700 1 -- 172.21.15.86:0/3235771474 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55875d6bd100 msgr2=0x55875d6c4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:54:12.995 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.991+0000 7fad09e00700 1 --2- 172.21.15.86:0/3235771474 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55875d6bd100 0x55875d6c4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:12.995 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.991+0000 7fad09e00700 1 -- 172.21.15.86:0/3235771474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55875d6bda20 msgr2=0x55875d6c5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:54:12.995 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.991+0000 7fad09e00700 1 --2- 172.21.15.86:0/3235771474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55875d6bda20 0x55875d6c5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:12.995 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.991+0000 7fad09e00700 1 -- 172.21.15.86:0/3235771474 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55875d43db00 con 0x55875d6c88d0 2024-03-23T17:54:12.995 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.991+0000 7fad08a00700 1 --2- 172.21.15.86:0/3235771474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55875d6bda20 0x55875d6c5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:54:12.996 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.992+0000 7fad09e00700 1 --2- 172.21.15.86:0/3235771474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55875d6c88d0 0x55875d6c55b0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7facfc008890 tx=0x7facfc004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76dae8be606284ba server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:54:12.996 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.992+0000 7faceaa00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-23T17:54:12.996 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.992+0000 7faceaa00700 1 -- 172.21.15.86:0/3235771474 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facfc028070 con 0x55875d6c88d0 2024-03-23T17:54:12.996 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.992+0000 7faceaa00700 1 -- 172.21.15.86:0/3235771474 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7facfc004bb0 con 0x55875d6c88d0 2024-03-23T17:54:12.996 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.992+0000 7fad1f93cb80 1 -- 172.21.15.86:0/3235771474 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7facf009ac50 con 0x55875d6c88d0 2024-03-23T17:54:12.996 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.992+0000 7faceaa00700 1 -- 172.21.15.86:0/3235771474 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7facfc004d70 con 0x55875d6c88d0 2024-03-23T17:54:12.996 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.992+0000 7fad1f93cb80 10 client.16272 fetch_fsmap learned FSMap version 36 2024-03-23T17:54:12.996 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.992+0000 7fad1f93cb80 1 -- 172.21.15.86:0/3235771474 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55875d495c70 con 0x55875d6c88d0 2024-03-23T17:54:12.996 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.992+0000 7faceaa00700 1 -- 172.21.15.86:0/3235771474 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7facf009ac50 con 0x55875d6c88d0 2024-03-23T17:54:12.996 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.992+0000 7faceaa00700 1 -- 172.21.15.86:0/3235771474 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7facfc0377d0 con 0x55875d6c88d0 2024-03-23T17:54:12.996 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.992+0000 7fad1f93cb80 10 client.16272 fetch_fsmap finished waiting for FSMap version 36 2024-03-23T17:54:12.996 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.992+0000 7fad1f93cb80 10 client.16272 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:54:12.996 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.992+0000 7fad1f93cb80 20 client.16272 populate_metadata read hostname 'smithi086' 2024-03-23T17:54:12.996 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.992+0000 7fad1f93cb80 1 --2- 172.21.15.86:0/3235771474 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55875d72ab20 0x55875d72cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:54:12.997 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.992+0000 7fad1f93cb80 4 client.16272 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T17:54:12.997 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.992+0000 7fad1f93cb80 1 -- 172.21.15.86:0/3235771474 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55875d45ec90 con 0x55875d72ab20 2024-03-23T17:54:12.997 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.993+0000 7fad09400700 1 --2- 172.21.15.86:0/3235771474 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55875d72ab20 0x55875d72cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:54:12.997 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.993+0000 7fad09400700 1 --2- 172.21.15.86:0/3235771474 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55875d72ab20 0x55875d72cf40 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2958eb307438cb4 server_cookie=89350253c985beeb in_seq=0 out_seq=0 2024-03-23T17:54:12.997 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:12.993+0000 7faceaa00700 10 client.16272 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:54:13.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:13.020+0000 7faceaa00700 1 -- 172.21.15.86:0/3235771474 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55875d45ec90 con 0x55875d72ab20 2024-03-23T17:54:13.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:13.020+0000 7faceaa00700 10 client.16272 handle_command_reply: tid=0 2024-03-23T17:54:13.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:13.020+0000 7fad1f93cb80 1 client.16272 shutdown 2024-03-23T17:54:13.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:13.021+0000 7fad1f93cb80 1 -- 172.21.15.86:0/3235771474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55875d6c88d0 msgr2=0x55875d6c55b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:54:13.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:13.021+0000 7fad1f93cb80 1 --2- 172.21.15.86:0/3235771474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55875d6c88d0 0x55875d6c55b0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7facfc008890 tx=0x7facfc004b40 comp rx=0 tx=0).stop 2024-03-23T17:54:13.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:13.021+0000 7fad1f93cb80 1 -- 172.21.15.86:0/3235771474 shutdown_connections 2024-03-23T17:54:13.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:13.021+0000 7fad1f93cb80 1 --2- 172.21.15.86:0/3235771474 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55875d72ab20 0x55875d72cf40 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:13.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:13.021+0000 7fad1f93cb80 1 --2- 172.21.15.86:0/3235771474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55875d6c88d0 0x55875d6c55b0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:13.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:13.021+0000 7fad1f93cb80 1 --2- 172.21.15.86:0/3235771474 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55875d6bd100 0x55875d6c4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:13.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:13.021+0000 7faceaa00700 0 client.16272 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:54:13.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:13.021+0000 7fad1f93cb80 1 --2- 172.21.15.86:0/3235771474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55875d6bda20 0x55875d6c5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:13.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:13.021+0000 7fad1f93cb80 1 -- 172.21.15.86:0/3235771474 >> 172.21.15.86:0/3235771474 conn(0x55875d61d640 msgr2=0x55875d6c3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:54:13.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:13.021+0000 7fad1f93cb80 1 -- 172.21.15.86:0/3235771474 shutdown_connections 2024-03-23T17:54:13.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:13.021+0000 7fad1f93cb80 1 -- 172.21.15.86:0/3235771474 wait complete. 2024-03-23T17:54:13.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:13.021+0000 7fad1f93cb80 20 client.16272 trim_cache size 0 max 16384 2024-03-23T17:54:13.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-03-23T17:54:13.023 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-03-23T17:54:13.023 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-03-23T17:54:13.023 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-03-23T17:54:13.023 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:13.022+0000 7fad0be00700 1 -- 172.21.15.86:0/3102426729 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7facec0a1290 msgr2=0x7facec0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:54:13.023 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:13.022+0000 7fad0be00700 1 --2- 172.21.15.86:0/3102426729 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7facec0a1290 0x7facec0a3750 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7facfc003ee0 tx=0x7facfc009100 comp rx=0 tx=0).stop 2024-03-23T17:54:13.023 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:13.022+0000 7fad0be00700 1 -- 172.21.15.86:0/3102426729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad04106010 msgr2=0x7fad040735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:54:13.024 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:13.022+0000 7fad0be00700 1 --2- 172.21.15.86:0/3102426729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad04106010 0x7fad040735f0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7facf0000c00 tx=0x7facf0005740 comp rx=0 tx=0).stop 2024-03-23T17:54:13.024 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:13.022+0000 7fad0be00700 1 -- 172.21.15.86:0/3102426729 shutdown_connections 2024-03-23T17:54:13.024 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:13.022+0000 7fad0be00700 1 --2- 172.21.15.86:0/3102426729 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7facec0a1290 0x7facec0a3750 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:13.024 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:13.022+0000 7fad0be00700 1 --2- 172.21.15.86:0/3102426729 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad04073b30 0x7fad041a5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:13.024 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:13.022+0000 7fad0be00700 1 --2- 172.21.15.86:0/3102426729 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad04100ef0 0x7fad040730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:13.024 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:13.022+0000 7fad0be00700 1 --2- 172.21.15.86:0/3102426729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad04106010 0x7fad040735f0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:13.024 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:13.022+0000 7fad0be00700 1 -- 172.21.15.86:0/3102426729 >> 172.21.15.86:0/3102426729 conn(0x7fad040fc750 msgr2=0x7fad040fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:54:13.024 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:13.022+0000 7fad0be00700 1 -- 172.21.15.86:0/3102426729 shutdown_connections 2024-03-23T17:54:13.024 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:13.022+0000 7fad0be00700 1 -- 172.21.15.86:0/3102426729 wait complete. 2024-03-23T17:54:13.892 INFO:tasks.workunit.client.0.smithi086.stdout: 1 100084 2.51 MB/sec execute 38 sec latency 2941.115 ms 2024-03-23T17:54:14.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:54:13 smithi086 ceph-mon[38221]: pgmap v110: 97 pgs: 97 active+clean; 49 MiB data, 671 MiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 629 KiB/s wr, 106 op/s 2024-03-23T17:54:14.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:54:13 smithi092 ceph-mon[79089]: pgmap v110: 97 pgs: 97 active+clean; 49 MiB data, 671 MiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 629 KiB/s wr, 106 op/s 2024-03-23T17:54:14.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:54:13 smithi152 ceph-mon[47949]: pgmap v110: 97 pgs: 97 active+clean; 49 MiB data, 671 MiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 629 KiB/s wr, 106 op/s 2024-03-23T17:54:14.892 INFO:tasks.workunit.client.0.smithi086.stdout: 1 100084 2.45 MB/sec execute 39 sec latency 3941.201 ms 2024-03-23T17:54:15.893 INFO:tasks.workunit.client.0.smithi086.stdout: 1 100084 2.39 MB/sec execute 40 sec latency 4941.375 ms 2024-03-23T17:54:16.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:54:15 smithi086 ceph-mon[38221]: pgmap v111: 97 pgs: 97 active+clean; 49 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 925 KiB/s wr, 136 op/s 2024-03-23T17:54:16.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:54:15 smithi092 ceph-mon[79089]: pgmap v111: 97 pgs: 97 active+clean; 49 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 925 KiB/s wr, 136 op/s 2024-03-23T17:54:16.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:54:15 smithi152 ceph-mon[47949]: pgmap v111: 97 pgs: 97 active+clean; 49 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 925 KiB/s wr, 136 op/s 2024-03-23T17:54:16.892 INFO:tasks.workunit.client.0.smithi086.stdout: 1 100152 2.33 MB/sec execute 41 sec latency 4956.344 ms 2024-03-23T17:54:17.893 INFO:tasks.workunit.client.0.smithi086.stdout: 1 100152 2.28 MB/sec execute 42 sec latency 1931.063 ms 2024-03-23T17:54:18.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:54:17 smithi086 ceph-mon[38221]: pgmap v112: 97 pgs: 97 active+clean; 49 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 846 KiB/s wr, 97 op/s 2024-03-23T17:54:18.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:54:17 smithi092 ceph-mon[79089]: pgmap v112: 97 pgs: 97 active+clean; 49 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 846 KiB/s wr, 97 op/s 2024-03-23T17:54:18.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:54:17 smithi152 ceph-mon[47949]: pgmap v112: 97 pgs: 97 active+clean; 49 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 846 KiB/s wr, 97 op/s 2024-03-23T17:54:18.893 INFO:tasks.workunit.client.0.smithi086.stdout: 1 100152 2.22 MB/sec execute 43 sec latency 2931.144 ms 2024-03-23T17:54:19.893 INFO:tasks.workunit.client.0.smithi086.stdout: 1 100152 2.17 MB/sec execute 44 sec latency 3931.286 ms 2024-03-23T17:54:20.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:54:19 smithi086 ceph-mon[38221]: pgmap v113: 97 pgs: 97 active+clean; 49 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 846 KiB/s wr, 97 op/s 2024-03-23T17:54:20.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:54:19 smithi092 ceph-mon[79089]: pgmap v113: 97 pgs: 97 active+clean; 49 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 846 KiB/s wr, 97 op/s 2024-03-23T17:54:20.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:54:19 smithi152 ceph-mon[47949]: pgmap v113: 97 pgs: 97 active+clean; 49 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 846 KiB/s wr, 97 op/s 2024-03-23T17:54:20.893 INFO:tasks.workunit.client.0.smithi086.stdout: 1 100152 2.12 MB/sec execute 45 sec latency 4931.444 ms 2024-03-23T17:54:21.893 INFO:tasks.workunit.client.0.smithi086.stdout: 1 102698 2.38 MB/sec execute 46 sec latency 4948.670 ms 2024-03-23T17:54:22.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:54:21 smithi086 ceph-mon[38221]: pgmap v114: 97 pgs: 97 active+clean; 49 MiB data, 671 MiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 870 KiB/s wr, 105 op/s 2024-03-23T17:54:22.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:54:21 smithi092 ceph-mon[79089]: pgmap v114: 97 pgs: 97 active+clean; 49 MiB data, 671 MiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 870 KiB/s wr, 105 op/s 2024-03-23T17:54:22.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:54:21 smithi152 ceph-mon[47949]: pgmap v114: 97 pgs: 97 active+clean; 49 MiB data, 671 MiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 870 KiB/s wr, 105 op/s 2024-03-23T17:54:22.893 INFO:tasks.workunit.client.0.smithi086.stdout: 1 103632 2.44 MB/sec execute 47 sec latency 177.405 ms 2024-03-23T17:54:23.893 INFO:tasks.workunit.client.0.smithi086.stdout: 1 103632 2.39 MB/sec execute 48 sec latency 1177.545 ms 2024-03-23T17:54:24.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:54:23 smithi086 ceph-mon[38221]: pgmap v115: 97 pgs: 97 active+clean; 49 MiB data, 671 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 320 KiB/s wr, 38 op/s 2024-03-23T17:54:24.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:54:23 smithi092 ceph-mon[79089]: pgmap v115: 97 pgs: 97 active+clean; 49 MiB data, 671 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 320 KiB/s wr, 38 op/s 2024-03-23T17:54:24.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:54:23 smithi152 ceph-mon[47949]: pgmap v115: 97 pgs: 97 active+clean; 49 MiB data, 671 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 320 KiB/s wr, 38 op/s 2024-03-23T17:54:24.894 INFO:tasks.workunit.client.0.smithi086.stdout: 1 103632 2.34 MB/sec execute 49 sec latency 2177.690 ms 2024-03-23T17:54:25.894 INFO:tasks.workunit.client.0.smithi086.stdout: 1 103632 2.29 MB/sec execute 50 sec latency 3177.832 ms 2024-03-23T17:54:26.169 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.167+0000 7efd72400700 1 -- 172.21.15.86:0/2263775074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd6c073150 msgr2=0x7efd6c073530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:54:26.169 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.167+0000 7efd72400700 1 --2- 172.21.15.86:0/2263775074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd6c073150 0x7efd6c073530 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7efd5c009990 tx=0x7efd5c009ca0 comp rx=0 tx=0).stop 2024-03-23T17:54:26.169 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.167+0000 7efd72400700 1 -- 172.21.15.86:0/2263775074 shutdown_connections 2024-03-23T17:54:26.169 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.167+0000 7efd72400700 1 --2- 172.21.15.86:0/2263775074 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd6c111c90 0x7efd6c114080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:26.169 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.167+0000 7efd72400700 1 --2- 172.21.15.86:0/2263775074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd6c073150 0x7efd6c073530 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:26.169 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.167+0000 7efd72400700 1 --2- 172.21.15.86:0/2263775074 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efd6c073a70 0x7efd6c111750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:26.169 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.167+0000 7efd72400700 1 -- 172.21.15.86:0/2263775074 >> 172.21.15.86:0/2263775074 conn(0x7efd6c0fc750 msgr2=0x7efd6c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:54:26.169 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.167+0000 7efd72400700 1 -- 172.21.15.86:0/2263775074 shutdown_connections 2024-03-23T17:54:26.169 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.167+0000 7efd72400700 1 -- 172.21.15.86:0/2263775074 wait complete. 2024-03-23T17:54:26.169 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.169+0000 7efd72400700 1 Processor -- start 2024-03-23T17:54:26.170 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.169+0000 7efd72400700 1 -- start start 2024-03-23T17:54:26.171 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.169+0000 7efd72400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd6c073150 0x7efd6c1a33f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:54:26.171 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.169+0000 7efd72400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efd6c073a70 0x7efd6c1a3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:54:26.171 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.170+0000 7efd72400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd6c111c90 0x7efd6c19d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:54:26.171 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.170+0000 7efd72400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd6c19dc30 con 0x7efd6c111c90 2024-03-23T17:54:26.171 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.170+0000 7efd72400700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd6c19dda0 con 0x7efd6c073150 2024-03-23T17:54:26.171 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.170+0000 7efd72400700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd6c19df60 con 0x7efd6c073a70 2024-03-23T17:54:26.171 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.170+0000 7efd6be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd6c111c90 0x7efd6c19d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:54:26.171 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.170+0000 7efd6be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd6c111c90 0x7efd6c19d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53094/0 (socket says 172.21.15.86:53094) 2024-03-23T17:54:26.171 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.170+0000 7efd6b400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd6c073150 0x7efd6c1a33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:54:26.172 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.170+0000 7efd6be00700 1 -- 172.21.15.86:0/3411021537 learned_addr learned my addr 172.21.15.86:0/3411021537 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:54:26.172 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.170+0000 7efd6aa00700 1 --2- 172.21.15.86:0/3411021537 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efd6c073a70 0x7efd6c1a3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:54:26.172 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.170+0000 7efd6be00700 1 -- 172.21.15.86:0/3411021537 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efd6c073a70 msgr2=0x7efd6c1a3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:54:26.172 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.170+0000 7efd6be00700 1 --2- 172.21.15.86:0/3411021537 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efd6c073a70 0x7efd6c1a3930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:26.172 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.170+0000 7efd6be00700 1 -- 172.21.15.86:0/3411021537 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd6c073150 msgr2=0x7efd6c1a33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:54:26.172 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.170+0000 7efd6be00700 1 --2- 172.21.15.86:0/3411021537 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd6c073150 0x7efd6c1a33f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:26.172 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.170+0000 7efd6be00700 1 -- 172.21.15.86:0/3411021537 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd5c0095f0 con 0x7efd6c111c90 2024-03-23T17:54:26.173 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.170+0000 7efd6be00700 1 --2- 172.21.15.86:0/3411021537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd6c111c90 0x7efd6c19d5d0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7efd6400ef90 tx=0x7efd6400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:54:26.173 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.171+0000 7efd63e00700 1 -- 172.21.15.86:0/3411021537 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd64018070 con 0x7efd6c111c90 2024-03-23T17:54:26.173 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.171+0000 7efd63e00700 1 -- 172.21.15.86:0/3411021537 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd6400f040 con 0x7efd6c111c90 2024-03-23T17:54:26.173 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.171+0000 7efd72400700 1 -- 172.21.15.86:0/3411021537 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd6c19e1e0 con 0x7efd6c111c90 2024-03-23T17:54:26.173 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.171+0000 7efd63e00700 1 -- 172.21.15.86:0/3411021537 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd6400cda0 con 0x7efd6c111c90 2024-03-23T17:54:26.173 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.171+0000 7efd72400700 1 -- 172.21.15.86:0/3411021537 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd6c1aa490 con 0x7efd6c111c90 2024-03-23T17:54:26.175 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.174+0000 7efd63e00700 1 -- 172.21.15.86:0/3411021537 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7efd64003a00 con 0x7efd6c111c90 2024-03-23T17:54:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.175+0000 7efd85e94b80 1 Processor -- start 2024-03-23T17:54:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.175+0000 7efd85e94b80 1 -- start start 2024-03-23T17:54:26.178 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.175+0000 7efd63e00700 1 --2- 172.21.15.86:0/3411021537 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7efd500a1310 0x7efd500a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:54:26.178 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.176+0000 7efd85e94b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609f410e100 0x5609f410e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:54:26.178 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.176+0000 7efd85e94b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5609f410ea20 0x5609f4119390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:54:26.178 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.176+0000 7efd85e94b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609f41198d0 0x5609f411bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:54:26.178 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.176+0000 7efd85e94b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609f4067930 con 0x5609f410e100 2024-03-23T17:54:26.178 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.176+0000 7efd85e94b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609f40652f0 con 0x5609f410ea20 2024-03-23T17:54:26.178 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.176+0000 7efd85e94b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609f4060be0 con 0x5609f41198d0 2024-03-23T17:54:26.178 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.176+0000 7efd63e00700 1 -- 172.21.15.86:0/3411021537 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7efd640cc7b0 con 0x7efd6c111c90 2024-03-23T17:54:26.178 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.176+0000 7efd6aa00700 1 --2- 172.21.15.86:0/3411021537 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7efd500a1310 0x7efd500a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:54:26.178 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.176+0000 7efd6b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609f410e100 0x5609f410e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:54:26.178 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.176+0000 7efd6aa00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5609f410ea20 0x5609f4119390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:54:26.178 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.176+0000 7efd6b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609f410e100 0x5609f410e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53100/0 (socket says 172.21.15.86:53100) 2024-03-23T17:54:26.179 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.176+0000 7efd6b400700 1 -- 172.21.15.86:0/2513048376 learned_addr learned my addr 172.21.15.86:0/2513048376 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:54:26.179 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.176+0000 7efd6be00700 1 --2- 172.21.15.86:0/2513048376 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609f41198d0 0x5609f411bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:54:26.180 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.177+0000 7efd6b400700 1 -- 172.21.15.86:0/2513048376 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609f41198d0 msgr2=0x5609f411bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:54:26.180 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.177+0000 7efd6b400700 1 --2- 172.21.15.86:0/2513048376 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609f41198d0 0x5609f411bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.177+0000 7efd6b400700 1 -- 172.21.15.86:0/2513048376 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5609f410ea20 msgr2=0x5609f4119390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:54:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.177+0000 7efd6b400700 1 --2- 172.21.15.86:0/2513048376 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5609f410ea20 0x5609f4119390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.177+0000 7efd6b400700 1 -- 172.21.15.86:0/2513048376 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609f3ee2350 con 0x5609f410e100 2024-03-23T17:54:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.177+0000 7efd6aa00700 1 --2- 172.21.15.86:0/2513048376 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5609f410ea20 0x5609f4119390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:54:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.177+0000 7efd6aa00700 1 --2- 172.21.15.86:0/3411021537 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7efd500a1310 0x7efd500a37d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7efd6c19ecb0 tx=0x7efd58012780 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:54:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.177+0000 7efd6be00700 1 --2- 172.21.15.86:0/2513048376 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609f41198d0 0x5609f411bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T17:54:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.177+0000 7efd6b400700 1 --2- 172.21.15.86:0/2513048376 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609f410e100 0x5609f410e4e0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7efd5c00b610 tx=0x7efd5c00b640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8350a8be3bd7a9c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:54:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.177+0000 7efd60c00700 1 -- 172.21.15.86:0/2513048376 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd5c01d070 con 0x5609f410e100 2024-03-23T17:54:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.178+0000 7efd60c00700 1 -- 172.21.15.86:0/2513048376 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd5c01e040 con 0x5609f410e100 2024-03-23T17:54:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.178+0000 7efd60c00700 1 -- 172.21.15.86:0/2513048376 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd5c004820 con 0x5609f410e100 2024-03-23T17:54:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.178+0000 7efd85e94b80 1 -- 172.21.15.86:0/2513048376 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609f410e100 msgr2=0x5609f410e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:54:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.178+0000 7efd85e94b80 1 --2- 172.21.15.86:0/2513048376 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609f410e100 0x5609f410e4e0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7efd5c00b610 tx=0x7efd5c00b640 comp rx=0 tx=0).stop 2024-03-23T17:54:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.178+0000 7efd85e94b80 1 -- 172.21.15.86:0/2513048376 shutdown_connections 2024-03-23T17:54:26.182 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.178+0000 7efd85e94b80 1 --2- 172.21.15.86:0/2513048376 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609f41198d0 0x5609f411bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:26.182 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.178+0000 7efd85e94b80 1 --2- 172.21.15.86:0/2513048376 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609f410e100 0x5609f410e4e0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:26.182 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.178+0000 7efd85e94b80 1 --2- 172.21.15.86:0/2513048376 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5609f410ea20 0x5609f4119390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:26.182 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.178+0000 7efd85e94b80 1 -- 172.21.15.86:0/2513048376 >> 172.21.15.86:0/2513048376 conn(0x5609f406e640 msgr2=0x5609f410a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:54:26.182 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.178+0000 7efd85e94b80 1 -- 172.21.15.86:0/2513048376 shutdown_connections 2024-03-23T17:54:26.182 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.178+0000 7efd85e94b80 1 -- 172.21.15.86:0/2513048376 wait complete. 2024-03-23T17:54:26.182 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.179+0000 7efd85e94b80 1 -- start start 2024-03-23T17:54:26.182 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.179+0000 7efd85e94b80 1 Processor -- start 2024-03-23T17:54:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.180+0000 7efd85e94b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609f410e100 0x5609f4119340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:54:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.180+0000 7efd85e94b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609f410ea20 0x5609f4116680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:54:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.180+0000 7efd85e94b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5609f41198d0 0x5609f4116d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:54:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.180+0000 7efd85e94b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609f3ffcd20 con 0x5609f410ea20 2024-03-23T17:54:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.180+0000 7efd85e94b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609f3ffc9c0 con 0x5609f41198d0 2024-03-23T17:54:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.180+0000 7efd85e94b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609f3ee4640 con 0x5609f410e100 2024-03-23T17:54:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.180+0000 7efd85e94b80 10 client.0 _ll_register_callbacks cb 0x5609f4006410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:54:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.180+0000 7efd6aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609f410ea20 0x5609f4116680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:54:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.180+0000 7efd6aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609f410ea20 0x5609f4116680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53116/0 (socket says 172.21.15.86:53116) 2024-03-23T17:54:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.180+0000 7efd6aa00700 1 -- 172.21.15.86:0/559871541 learned_addr learned my addr 172.21.15.86:0/559871541 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:54:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.180+0000 7efd6b400700 1 --2- 172.21.15.86:0/559871541 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609f410e100 0x5609f4119340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:54:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.181+0000 7efd6be00700 1 --2- 172.21.15.86:0/559871541 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5609f41198d0 0x5609f4116d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:54:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.181+0000 7efd6aa00700 1 -- 172.21.15.86:0/559871541 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609f410e100 msgr2=0x5609f4119340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:54:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.181+0000 7efd6aa00700 1 --2- 172.21.15.86:0/559871541 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609f410e100 0x5609f4119340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.181+0000 7efd6aa00700 1 -- 172.21.15.86:0/559871541 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5609f41198d0 msgr2=0x5609f4116d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:54:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.181+0000 7efd6aa00700 1 --2- 172.21.15.86:0/559871541 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5609f41198d0 0x5609f4116d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.181+0000 7efd6aa00700 1 -- 172.21.15.86:0/559871541 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd58012df0 con 0x5609f410ea20 2024-03-23T17:54:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.181+0000 7efd6be00700 1 --2- 172.21.15.86:0/559871541 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5609f41198d0 0x5609f4116d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:54:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.181+0000 7efd6aa00700 1 --2- 172.21.15.86:0/559871541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609f410ea20 0x5609f4116680 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7efd58010920 tx=0x7efd5801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11ba9b451d6584c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:54:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.181+0000 7efd55600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-23T17:54:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.181+0000 7efd55600700 1 -- 172.21.15.86:0/559871541 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd5801f740 con 0x5609f410ea20 2024-03-23T17:54:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.181+0000 7efd85e94b80 1 -- 172.21.15.86:0/559871541 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5609f4021320 con 0x5609f410ea20 2024-03-23T17:54:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.181+0000 7efd55600700 1 -- 172.21.15.86:0/559871541 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7efd5801f8a0 con 0x5609f410ea20 2024-03-23T17:54:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.182+0000 7efd55600700 1 -- 172.21.15.86:0/559871541 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7efd5801fa60 con 0x5609f410ea20 2024-03-23T17:54:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.182+0000 7efd85e94b80 10 client.16284 fetch_fsmap learned FSMap version 37 2024-03-23T17:54:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.182+0000 7efd85e94b80 1 -- 172.21.15.86:0/559871541 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x7efd6c114a30 con 0x5609f410ea20 2024-03-23T17:54:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.182+0000 7efd55600700 1 -- 172.21.15.86:0/559871541 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5609f4021320 con 0x5609f410ea20 2024-03-23T17:54:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.182+0000 7efd55600700 1 -- 172.21.15.86:0/559871541 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 37) v1 ==== 7466+0+0 (secure 0 0 0) 0x7efd58028ce0 con 0x5609f410ea20 2024-03-23T17:54:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.182+0000 7efd85e94b80 10 client.16284 fetch_fsmap finished waiting for FSMap version 37 2024-03-23T17:54:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.182+0000 7efd85e94b80 10 client.16284 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:54:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.182+0000 7efd85e94b80 20 client.16284 populate_metadata read hostname 'smithi086' 2024-03-23T17:54:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.182+0000 7efd85e94b80 1 --2- 172.21.15.86:0/559871541 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5609f4129570 0x5609f412b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:54:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.182+0000 7efd85e94b80 4 client.16284 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:54:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.182+0000 7efd85e94b80 1 -- 172.21.15.86:0/559871541 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5609f40596a0 con 0x5609f4129570 2024-03-23T17:54:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.183+0000 7efd6b400700 1 --2- 172.21.15.86:0/559871541 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5609f4129570 0x5609f412b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:54:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.183+0000 7efd6b400700 1 --2- 172.21.15.86:0/559871541 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5609f4129570 0x5609f412b990 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc245820fb9d2ab7 server_cookie=9f5f148d3902c86c in_seq=0 out_seq=0 2024-03-23T17:54:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.183+0000 7efd55600700 10 client.16284 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:54:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.184+0000 7efd55600700 1 -- 172.21.15.86:0/559871541 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5609f40596a0 con 0x5609f4129570 2024-03-23T17:54:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.184+0000 7efd55600700 10 client.16284 handle_command_reply: tid=0 2024-03-23T17:54:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.184+0000 7efd85e94b80 1 client.16284 shutdown 2024-03-23T17:54:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.184+0000 7efd85e94b80 1 -- 172.21.15.86:0/559871541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609f410ea20 msgr2=0x5609f4116680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:54:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.184+0000 7efd85e94b80 1 --2- 172.21.15.86:0/559871541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609f410ea20 0x5609f4116680 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7efd58010920 tx=0x7efd5801a000 comp rx=0 tx=0).stop 2024-03-23T17:54:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.184+0000 7efd85e94b80 1 -- 172.21.15.86:0/559871541 shutdown_connections 2024-03-23T17:54:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.184+0000 7efd85e94b80 1 --2- 172.21.15.86:0/559871541 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5609f4129570 0x5609f412b990 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.184+0000 7efd85e94b80 1 --2- 172.21.15.86:0/559871541 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5609f41198d0 0x5609f4116d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.184+0000 7efd85e94b80 1 --2- 172.21.15.86:0/559871541 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609f410e100 0x5609f4119340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.184+0000 7efd85e94b80 1 --2- 172.21.15.86:0/559871541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609f410ea20 0x5609f4116680 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.184+0000 7efd55600700 0 client.16284 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:54:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.184+0000 7efd85e94b80 1 -- 172.21.15.86:0/559871541 >> 172.21.15.86:0/559871541 conn(0x5609f406e640 msgr2=0x5609f41100a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:54:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.184+0000 7efd85e94b80 1 -- 172.21.15.86:0/559871541 shutdown_connections 2024-03-23T17:54:26.190 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.184+0000 7efd85e94b80 1 -- 172.21.15.86:0/559871541 wait complete. 2024-03-23T17:54:26.190 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.185+0000 7efd85e94b80 20 client.16284 trim_cache size 0 max 16384 2024-03-23T17:54:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.199+0000 7efd85e94b80 1 Processor -- start 2024-03-23T17:54:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.199+0000 7efd85e94b80 1 -- start start 2024-03-23T17:54:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.200+0000 7efd85e94b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5609f410e100 0x5609f41302f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:54:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.200+0000 7efd85e94b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609f410ea20 0x5609f4120540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:54:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.200+0000 7efd85e94b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609f41198d0 0x5609f41107b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:54:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.200+0000 7efd85e94b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609f4117c30 con 0x5609f41198d0 2024-03-23T17:54:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.200+0000 7efd85e94b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609f4106430 con 0x5609f410e100 2024-03-23T17:54:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.200+0000 7efd85e94b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609f41124e0 con 0x5609f410ea20 2024-03-23T17:54:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.200+0000 7efd6be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609f41198d0 0x5609f41107b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:54:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.200+0000 7efd6b400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5609f410e100 0x5609f41302f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:54:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.200+0000 7efd6be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609f41198d0 0x5609f41107b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53132/0 (socket says 172.21.15.86:53132) 2024-03-23T17:54:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.200+0000 7efd6be00700 1 -- 172.21.15.86:0/1173943950 learned_addr learned my addr 172.21.15.86:0/1173943950 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:54:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.200+0000 7efd6b400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5609f410e100 0x5609f41302f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:58422/0 (socket says 172.21.15.86:58422) 2024-03-23T17:54:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.200+0000 7efd6aa00700 1 --2- 172.21.15.86:0/1173943950 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609f410ea20 0x5609f4120540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:54:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.200+0000 7efd6be00700 1 -- 172.21.15.86:0/1173943950 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609f410ea20 msgr2=0x5609f4120540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:54:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.200+0000 7efd6be00700 1 --2- 172.21.15.86:0/1173943950 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609f410ea20 0x5609f4120540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:26.203 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.200+0000 7efd6be00700 1 -- 172.21.15.86:0/1173943950 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5609f410e100 msgr2=0x5609f41302f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:54:26.203 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.200+0000 7efd6be00700 1 --2- 172.21.15.86:0/1173943950 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5609f410e100 0x5609f41302f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:26.203 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.200+0000 7efd6be00700 1 -- 172.21.15.86:0/1173943950 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609f3e8eb00 con 0x5609f41198d0 2024-03-23T17:54:26.203 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.200+0000 7efd6b400700 1 --2- 172.21.15.86:0/1173943950 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5609f410e100 0x5609f41302f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:54:26.206 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.200+0000 7efd6be00700 1 --2- 172.21.15.86:0/1173943950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609f41198d0 0x5609f41107b0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7efd6400c8b0 tx=0x7efd64099710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e35c26f1e0b5e80 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:54:26.206 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.201+0000 7efd61600700 1 -- 172.21.15.86:0/1173943950 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd640b7ea0 con 0x5609f41198d0 2024-03-23T17:54:26.206 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.201+0000 7efd61600700 1 -- 172.21.15.86:0/1173943950 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd64084040 con 0x5609f41198d0 2024-03-23T17:54:26.206 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.201+0000 7efd61600700 1 -- 172.21.15.86:0/1173943950 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd6409d770 con 0x5609f41198d0 2024-03-23T17:54:26.206 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.201+0000 7efd85e94b80 1 -- 172.21.15.86:0/1173943950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609f41198d0 msgr2=0x5609f41107b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:54:26.206 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.201+0000 7efd85e94b80 1 --2- 172.21.15.86:0/1173943950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609f41198d0 0x5609f41107b0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7efd6400c8b0 tx=0x7efd64099710 comp rx=0 tx=0).stop 2024-03-23T17:54:26.206 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.201+0000 7efd85e94b80 1 -- 172.21.15.86:0/1173943950 shutdown_connections 2024-03-23T17:54:26.207 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.201+0000 7efd85e94b80 1 --2- 172.21.15.86:0/1173943950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609f41198d0 0x5609f41107b0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:26.207 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.201+0000 7efd85e94b80 1 --2- 172.21.15.86:0/1173943950 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5609f410e100 0x5609f41302f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:26.207 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.201+0000 7efd85e94b80 1 --2- 172.21.15.86:0/1173943950 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609f410ea20 0x5609f4120540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:26.207 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.201+0000 7efd85e94b80 1 -- 172.21.15.86:0/1173943950 >> 172.21.15.86:0/1173943950 conn(0x5609f406e640 msgr2=0x5609f41100a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:54:26.207 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.201+0000 7efd85e94b80 1 -- 172.21.15.86:0/1173943950 shutdown_connections 2024-03-23T17:54:26.207 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.201+0000 7efd85e94b80 1 -- 172.21.15.86:0/1173943950 wait complete. 2024-03-23T17:54:26.207 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.202+0000 7efd85e94b80 1 -- start start 2024-03-23T17:54:26.207 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.202+0000 7efd85e94b80 1 Processor -- start 2024-03-23T17:54:26.207 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.202+0000 7efd85e94b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609f410e100 0x5609f4120000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:54:26.207 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.202+0000 7efd85e94b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5609f410ea20 0x5609f4120540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:54:26.207 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.202+0000 7efd85e94b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609f41198d0 0x5609f411e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:54:26.207 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.202+0000 7efd85e94b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609f411edb0 con 0x5609f410e100 2024-03-23T17:54:26.207 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.202+0000 7efd85e94b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609f411ef20 con 0x5609f410ea20 2024-03-23T17:54:26.208 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.202+0000 7efd85e94b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609f411f0e0 con 0x5609f41198d0 2024-03-23T17:54:26.208 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.202+0000 7efd85e94b80 10 client.0 _ll_register_callbacks cb 0x5609f4006410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:54:26.208 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.203+0000 7efd6b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609f410e100 0x5609f4120000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:54:26.208 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.203+0000 7efd6b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609f410e100 0x5609f4120000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53144/0 (socket says 172.21.15.86:53144) 2024-03-23T17:54:26.208 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.203+0000 7efd6b400700 1 -- 172.21.15.86:0/2147881947 learned_addr learned my addr 172.21.15.86:0/2147881947 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:54:26.208 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.203+0000 7efd6be00700 1 --2- 172.21.15.86:0/2147881947 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609f41198d0 0x5609f411e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:54:26.208 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.203+0000 7efd6aa00700 1 --2- 172.21.15.86:0/2147881947 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5609f410ea20 0x5609f4120540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:54:26.208 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.203+0000 7efd6b400700 1 -- 172.21.15.86:0/2147881947 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609f41198d0 msgr2=0x5609f411e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:54:26.208 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.203+0000 7efd6b400700 1 --2- 172.21.15.86:0/2147881947 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609f41198d0 0x5609f411e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:26.208 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.203+0000 7efd6b400700 1 -- 172.21.15.86:0/2147881947 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5609f410ea20 msgr2=0x5609f4120540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:54:26.208 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.203+0000 7efd6b400700 1 --2- 172.21.15.86:0/2147881947 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5609f410ea20 0x5609f4120540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:26.208 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.203+0000 7efd6b400700 1 -- 172.21.15.86:0/2147881947 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5609f3ee2350 con 0x5609f410e100 2024-03-23T17:54:26.208 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.203+0000 7efd6aa00700 1 --2- 172.21.15.86:0/2147881947 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5609f410ea20 0x5609f4120540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:54:26.208 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.203+0000 7efd6b400700 1 --2- 172.21.15.86:0/2147881947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609f410e100 0x5609f4120000 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7efd5c009ce0 tx=0x7efd5c01a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a3f6e3ecbc0da42 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:54:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.203+0000 7efd55600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-23T17:54:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.203+0000 7efd55600700 1 -- 172.21.15.86:0/2147881947 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd5c032040 con 0x5609f410e100 2024-03-23T17:54:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.203+0000 7efd55600700 1 -- 172.21.15.86:0/2147881947 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7efd5c005320 con 0x5609f410e100 2024-03-23T17:54:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.203+0000 7efd85e94b80 1 -- 172.21.15.86:0/2147881947 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efd5c01e040 con 0x5609f410e100 2024-03-23T17:54:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.204+0000 7efd55600700 1 -- 172.21.15.86:0/2147881947 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7efd5c01d040 con 0x5609f410e100 2024-03-23T17:54:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.204+0000 7efd85e94b80 10 client.16290 fetch_fsmap learned FSMap version 37 2024-03-23T17:54:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.204+0000 7efd85e94b80 1 -- 172.21.15.86:0/2147881947 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x7efd6c1aa490 con 0x5609f410e100 2024-03-23T17:54:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.204+0000 7efd55600700 1 -- 172.21.15.86:0/2147881947 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd5c01e040 con 0x5609f410e100 2024-03-23T17:54:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.204+0000 7efd55600700 1 -- 172.21.15.86:0/2147881947 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 37) v1 ==== 7466+0+0 (secure 0 0 0) 0x7efd5c034c30 con 0x5609f410e100 2024-03-23T17:54:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.204+0000 7efd85e94b80 10 client.16290 fetch_fsmap finished waiting for FSMap version 37 2024-03-23T17:54:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.204+0000 7efd85e94b80 10 client.16290 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:54:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.204+0000 7efd85e94b80 20 client.16290 populate_metadata read hostname 'smithi086' 2024-03-23T17:54:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.204+0000 7efd85e94b80 1 --2- 172.21.15.86:0/2147881947 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5609f417bff0 0x5609f417e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:54:26.210 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.204+0000 7efd85e94b80 4 client.16290 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T17:54:26.210 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.204+0000 7efd85e94b80 1 -- 172.21.15.86:0/2147881947 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5609f3eafc90 con 0x5609f417bff0 2024-03-23T17:54:26.210 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.204+0000 7efd6aa00700 1 --2- 172.21.15.86:0/2147881947 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5609f417bff0 0x5609f417e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:54:26.210 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.205+0000 7efd6aa00700 1 --2- 172.21.15.86:0/2147881947 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5609f417bff0 0x5609f417e3e0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8385424bb5741ec6 server_cookie=c57cdb757a67eb87 in_seq=0 out_seq=0 2024-03-23T17:54:26.210 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.205+0000 7efd55600700 10 client.16290 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:54:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.232+0000 7efd55600700 1 -- 172.21.15.86:0/2147881947 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5609f3eafc90 con 0x5609f417bff0 2024-03-23T17:54:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.232+0000 7efd55600700 10 client.16290 handle_command_reply: tid=0 2024-03-23T17:54:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.232+0000 7efd85e94b80 1 client.16290 shutdown 2024-03-23T17:54:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.232+0000 7efd85e94b80 1 -- 172.21.15.86:0/2147881947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609f410e100 msgr2=0x5609f4120000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:54:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.232+0000 7efd85e94b80 1 --2- 172.21.15.86:0/2147881947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609f410e100 0x5609f4120000 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7efd5c009ce0 tx=0x7efd5c01a040 comp rx=0 tx=0).stop 2024-03-23T17:54:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.232+0000 7efd85e94b80 1 -- 172.21.15.86:0/2147881947 shutdown_connections 2024-03-23T17:54:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.232+0000 7efd85e94b80 1 --2- 172.21.15.86:0/2147881947 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5609f417bff0 0x5609f417e3e0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.232+0000 7efd85e94b80 1 --2- 172.21.15.86:0/2147881947 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609f41198d0 0x5609f411e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.232+0000 7efd85e94b80 1 --2- 172.21.15.86:0/2147881947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609f410e100 0x5609f4120000 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.232+0000 7efd55600700 0 client.16290 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:54:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.232+0000 7efd85e94b80 1 --2- 172.21.15.86:0/2147881947 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5609f410ea20 0x5609f4120540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:26.235 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.232+0000 7efd85e94b80 1 -- 172.21.15.86:0/2147881947 >> 172.21.15.86:0/2147881947 conn(0x5609f406e640 msgr2=0x5609f4109d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:54:26.235 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.233+0000 7efd85e94b80 1 -- 172.21.15.86:0/2147881947 shutdown_connections 2024-03-23T17:54:26.235 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.233+0000 7efd85e94b80 1 -- 172.21.15.86:0/2147881947 wait complete. 2024-03-23T17:54:26.235 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.233+0000 7efd85e94b80 20 client.16290 trim_cache size 0 max 16384 2024-03-23T17:54:26.235 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-03-23T17:54:26.235 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-03-23T17:54:26.235 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-03-23T17:54:26.235 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-03-23T17:54:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.233+0000 7efd72400700 1 -- 172.21.15.86:0/3411021537 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7efd500a1310 msgr2=0x7efd500a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:54:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.233+0000 7efd72400700 1 --2- 172.21.15.86:0/3411021537 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7efd500a1310 0x7efd500a37d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7efd6c19ecb0 tx=0x7efd58012780 comp rx=0 tx=0).stop 2024-03-23T17:54:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.233+0000 7efd72400700 1 -- 172.21.15.86:0/3411021537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd6c111c90 msgr2=0x7efd6c19d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:54:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.233+0000 7efd72400700 1 --2- 172.21.15.86:0/3411021537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd6c111c90 0x7efd6c19d5d0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7efd6400ef90 tx=0x7efd6400c3a0 comp rx=0 tx=0).stop 2024-03-23T17:54:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.233+0000 7efd72400700 1 -- 172.21.15.86:0/3411021537 shutdown_connections 2024-03-23T17:54:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.233+0000 7efd72400700 1 --2- 172.21.15.86:0/3411021537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd6c111c90 0x7efd6c19d5d0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.233+0000 7efd72400700 1 --2- 172.21.15.86:0/3411021537 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd6c073150 0x7efd6c1a33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.233+0000 7efd72400700 1 --2- 172.21.15.86:0/3411021537 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7efd500a1310 0x7efd500a37d0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:26.237 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.233+0000 7efd72400700 1 --2- 172.21.15.86:0/3411021537 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efd6c073a70 0x7efd6c1a3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:26.237 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.233+0000 7efd72400700 1 -- 172.21.15.86:0/3411021537 >> 172.21.15.86:0/3411021537 conn(0x7efd6c0fc750 msgr2=0x7efd6c0fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:54:26.237 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.234+0000 7efd72400700 1 -- 172.21.15.86:0/3411021537 shutdown_connections 2024-03-23T17:54:26.237 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:26.234+0000 7efd72400700 1 -- 172.21.15.86:0/3411021537 wait complete. 2024-03-23T17:54:26.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:54:25 smithi086 ceph-mon[38221]: pgmap v116: 97 pgs: 97 active+clean; 51 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 1.0 MiB/s wr, 95 op/s 2024-03-23T17:54:26.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:54:25 smithi086 ceph-mon[38221]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T17:54:26.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:54:25 smithi092 ceph-mon[79089]: pgmap v116: 97 pgs: 97 active+clean; 51 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 1.0 MiB/s wr, 95 op/s 2024-03-23T17:54:26.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:54:25 smithi092 ceph-mon[79089]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T17:54:26.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:54:25 smithi152 ceph-mon[47949]: pgmap v116: 97 pgs: 97 active+clean; 51 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 1.0 MiB/s wr, 95 op/s 2024-03-23T17:54:26.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:54:25 smithi152 ceph-mon[47949]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T17:54:26.894 INFO:tasks.workunit.client.0.smithi086.stdout: 1 103700 2.25 MB/sec execute 51 sec latency 3192.117 ms 2024-03-23T17:54:27.894 INFO:tasks.workunit.client.0.smithi086.stdout: 1 103700 2.21 MB/sec execute 52 sec latency 1936.875 ms 2024-03-23T17:54:28.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:54:27 smithi086 ceph-mon[38221]: pgmap v117: 97 pgs: 97 active+clean; 51 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 753 KiB/s wr, 65 op/s 2024-03-23T17:54:28.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:54:27 smithi092 ceph-mon[79089]: pgmap v117: 97 pgs: 97 active+clean; 51 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 753 KiB/s wr, 65 op/s 2024-03-23T17:54:28.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:54:27 smithi152 ceph-mon[47949]: pgmap v117: 97 pgs: 97 active+clean; 51 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 753 KiB/s wr, 65 op/s 2024-03-23T17:54:28.894 INFO:tasks.workunit.client.0.smithi086.stdout: 1 103700 2.16 MB/sec execute 53 sec latency 2936.979 ms 2024-03-23T17:54:29.894 INFO:tasks.workunit.client.0.smithi086.stdout: 1 103700 2.12 MB/sec execute 54 sec latency 3937.133 ms 2024-03-23T17:54:30.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:54:29 smithi086 ceph-mon[38221]: pgmap v118: 97 pgs: 97 active+clean; 51 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 753 KiB/s wr, 65 op/s 2024-03-23T17:54:30.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:54:29 smithi092 ceph-mon[79089]: pgmap v118: 97 pgs: 97 active+clean; 51 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 753 KiB/s wr, 65 op/s 2024-03-23T17:54:30.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:54:29 smithi152 ceph-mon[47949]: pgmap v118: 97 pgs: 97 active+clean; 51 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 753 KiB/s wr, 65 op/s 2024-03-23T17:54:30.894 INFO:tasks.workunit.client.0.smithi086.stdout: 1 103700 2.09 MB/sec execute 55 sec latency 4937.188 ms 2024-03-23T17:54:31.894 INFO:tasks.workunit.client.0.smithi086.stdout: 1 106116 2.28 MB/sec execute 56 sec latency 4946.505 ms 2024-03-23T17:54:32.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:54:31 smithi086 ceph-mon[38221]: pgmap v119: 97 pgs: 97 active+clean; 47 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 796 KiB/s wr, 84 op/s 2024-03-23T17:54:32.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:54:31 smithi092 ceph-mon[79089]: pgmap v119: 97 pgs: 97 active+clean; 47 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 796 KiB/s wr, 84 op/s 2024-03-23T17:54:32.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:54:31 smithi152 ceph-mon[47949]: pgmap v119: 97 pgs: 97 active+clean; 47 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 796 KiB/s wr, 84 op/s 2024-03-23T17:54:32.894 INFO:tasks.workunit.client.0.smithi086.stdout: 1 107241 2.35 MB/sec execute 57 sec latency 54.341 ms 2024-03-23T17:54:33.894 INFO:tasks.workunit.client.0.smithi086.stdout: 1 107241 2.31 MB/sec execute 58 sec latency 1054.480 ms 2024-03-23T17:54:34.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:54:33 smithi086 ceph-mon[38221]: pgmap v120: 97 pgs: 97 active+clean; 47 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 772 KiB/s wr, 76 op/s 2024-03-23T17:54:34.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:54:33 smithi092 ceph-mon[79089]: pgmap v120: 97 pgs: 97 active+clean; 47 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 772 KiB/s wr, 76 op/s 2024-03-23T17:54:34.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:54:33 smithi152 ceph-mon[47949]: pgmap v120: 97 pgs: 97 active+clean; 47 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 772 KiB/s wr, 76 op/s 2024-03-23T17:54:34.895 INFO:tasks.workunit.client.0.smithi086.stdout: 1 107241 2.27 MB/sec execute 59 sec latency 2054.638 ms 2024-03-23T17:54:35.895 INFO:tasks.workunit.client.0.smithi086.stdout: 1 107241 2.23 MB/sec execute 60 sec latency 3054.723 ms 2024-03-23T17:54:36.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:54:35 smithi086 ceph-mon[38221]: pgmap v121: 97 pgs: 97 active+clean; 50 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 1.3 MiB/s wr, 127 op/s 2024-03-23T17:54:36.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:54:35 smithi092 ceph-mon[79089]: pgmap v121: 97 pgs: 97 active+clean; 50 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 1.3 MiB/s wr, 127 op/s 2024-03-23T17:54:36.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:54:35 smithi152 ceph-mon[47949]: pgmap v121: 97 pgs: 97 active+clean; 50 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 1.3 MiB/s wr, 127 op/s 2024-03-23T17:54:36.895 INFO:tasks.workunit.client.0.smithi086.stdout: 1 109475 2.39 MB/sec execute 61 sec latency 3068.376 ms 2024-03-23T17:54:37.895 INFO:tasks.workunit.client.0.smithi086.stdout: 1 110667 2.46 MB/sec execute 62 sec latency 140.197 ms 2024-03-23T17:54:38.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:54:37 smithi086 ceph-mon[38221]: pgmap v122: 97 pgs: 97 active+clean; 50 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 622 KiB/s wr, 70 op/s 2024-03-23T17:54:38.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:54:37 smithi092 ceph-mon[79089]: pgmap v122: 97 pgs: 97 active+clean; 50 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 622 KiB/s wr, 70 op/s 2024-03-23T17:54:38.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:54:37 smithi152 ceph-mon[47949]: pgmap v122: 97 pgs: 97 active+clean; 50 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 622 KiB/s wr, 70 op/s 2024-03-23T17:54:38.895 INFO:tasks.workunit.client.0.smithi086.stdout: 1 110667 2.42 MB/sec execute 63 sec latency 1140.331 ms 2024-03-23T17:54:39.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:54:38 smithi086 ceph-mon[38221]: pgmap v123: 97 pgs: 97 active+clean; 50 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 622 KiB/s wr, 70 op/s 2024-03-23T17:54:39.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:54:38 smithi092 ceph-mon[79089]: pgmap v123: 97 pgs: 97 active+clean; 50 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 622 KiB/s wr, 70 op/s 2024-03-23T17:54:39.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:54:38 smithi152 ceph-mon[47949]: pgmap v123: 97 pgs: 97 active+clean; 50 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 622 KiB/s wr, 70 op/s 2024-03-23T17:54:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.388+0000 7fe3c2c00700 1 -- 172.21.15.86:0/656821841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3bc1018a0 msgr2=0x7fe3bc1058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:54:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.388+0000 7fe3c2c00700 1 --2- 172.21.15.86:0/656821841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3bc1018a0 0x7fe3bc1058e0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fe3a8009990 tx=0x7fe3a8009ca0 comp rx=0 tx=0).stop 2024-03-23T17:54:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.388+0000 7fe3c2c00700 1 -- 172.21.15.86:0/656821841 shutdown_connections 2024-03-23T17:54:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.388+0000 7fe3c2c00700 1 --2- 172.21.15.86:0/656821841 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3bc106010 0x7fe3bc10db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.388+0000 7fe3c2c00700 1 --2- 172.21.15.86:0/656821841 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3bc100ef0 0x7fe3bc1012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.388+0000 7fe3c2c00700 1 --2- 172.21.15.86:0/656821841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3bc1018a0 0x7fe3bc1058e0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:39.390 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.388+0000 7fe3c2c00700 1 -- 172.21.15.86:0/656821841 >> 172.21.15.86:0/656821841 conn(0x7fe3bc0fc750 msgr2=0x7fe3bc0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:54:39.390 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.388+0000 7fe3c2c00700 1 -- 172.21.15.86:0/656821841 shutdown_connections 2024-03-23T17:54:39.390 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.389+0000 7fe3c2c00700 1 -- 172.21.15.86:0/656821841 wait complete. 2024-03-23T17:54:39.392 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.389+0000 7fe3c2c00700 1 Processor -- start 2024-03-23T17:54:39.392 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.390+0000 7fe3c2c00700 1 -- start start 2024-03-23T17:54:39.392 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.390+0000 7fe3c2c00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3bc100ef0 0x7fe3bc0730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:54:39.392 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.390+0000 7fe3c2c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3bc106010 0x7fe3bc0735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:54:39.392 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.390+0000 7fe3c2c00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3bc073b30 0x7fe3bc1a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:54:39.392 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.390+0000 7fe3c2c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3bc1a59c0 con 0x7fe3bc106010 2024-03-23T17:54:39.392 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.390+0000 7fe3c2c00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3bc1a5b30 con 0x7fe3bc073b30 2024-03-23T17:54:39.392 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.390+0000 7fe3c2c00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3bc1a5cf0 con 0x7fe3bc100ef0 2024-03-23T17:54:39.393 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.390+0000 7fe3bb400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3bc106010 0x7fe3bc0735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:54:39.393 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.390+0000 7fe3bb400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3bc106010 0x7fe3bc0735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59446/0 (socket says 172.21.15.86:59446) 2024-03-23T17:54:39.393 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.390+0000 7fe3bb400700 1 -- 172.21.15.86:0/509998736 learned_addr learned my addr 172.21.15.86:0/509998736 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:54:39.393 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.390+0000 7fe3bbe00700 1 --2- 172.21.15.86:0/509998736 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3bc100ef0 0x7fe3bc0730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:54:39.393 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.390+0000 7fe3c0c00700 1 --2- 172.21.15.86:0/509998736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3bc073b30 0x7fe3bc1a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:54:39.393 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.391+0000 7fe3bb400700 1 -- 172.21.15.86:0/509998736 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3bc100ef0 msgr2=0x7fe3bc0730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:54:39.393 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.391+0000 7fe3bb400700 1 --2- 172.21.15.86:0/509998736 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3bc100ef0 0x7fe3bc0730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:39.393 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.391+0000 7fe3bb400700 1 -- 172.21.15.86:0/509998736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3bc073b30 msgr2=0x7fe3bc1a5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:54:39.393 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.391+0000 7fe3bb400700 1 --2- 172.21.15.86:0/509998736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3bc073b30 0x7fe3bc1a5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:39.393 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.391+0000 7fe3bb400700 1 -- 172.21.15.86:0/509998736 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3a80095f0 con 0x7fe3bc106010 2024-03-23T17:54:39.393 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.391+0000 7fe3bb400700 1 --2- 172.21.15.86:0/509998736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3bc106010 0x7fe3bc0735f0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fe3a8000c00 tx=0x7fe3a8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:54:39.393 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.391+0000 7fe3b8c00700 1 -- 172.21.15.86:0/509998736 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a801d070 con 0x7fe3bc106010 2024-03-23T17:54:39.393 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.391+0000 7fe3b8c00700 1 -- 172.21.15.86:0/509998736 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3a8021410 con 0x7fe3bc106010 2024-03-23T17:54:39.393 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.391+0000 7fe3c2c00700 1 -- 172.21.15.86:0/509998736 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3bc1a5f70 con 0x7fe3bc106010 2024-03-23T17:54:39.394 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.391+0000 7fe3b8c00700 1 -- 172.21.15.86:0/509998736 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a8017650 con 0x7fe3bc106010 2024-03-23T17:54:39.394 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.391+0000 7fe3c2c00700 1 -- 172.21.15.86:0/509998736 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3bc1a6330 con 0x7fe3bc106010 2024-03-23T17:54:39.394 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.395+0000 7fe3b8c00700 1 -- 172.21.15.86:0/509998736 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fe3a8021580 con 0x7fe3bc106010 2024-03-23T17:54:39.396 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.396+0000 7fe3d67aab80 1 Processor -- start 2024-03-23T17:54:39.396 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.396+0000 7fe3b8c00700 1 --2- 172.21.15.86:0/509998736 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fe3a00a1290 0x7fe3a00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:54:39.396 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.396+0000 7fe3d67aab80 1 -- start start 2024-03-23T17:54:39.398 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.396+0000 7fe3b8c00700 1 -- 172.21.15.86:0/509998736 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fe3a80cde00 con 0x7fe3bc106010 2024-03-23T17:54:39.398 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.396+0000 7fe3d67aab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f7b4164100 0x55f7b41644e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:54:39.398 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.396+0000 7fe3c0c00700 1 --2- 172.21.15.86:0/509998736 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fe3a00a1290 0x7fe3a00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:54:39.398 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.397+0000 7fe3d67aab80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f7b4164a20 0x55f7b416f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:54:39.398 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.397+0000 7fe3d67aab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7b416f8d0 0x55f7b4171cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:54:39.398 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.397+0000 7fe3d67aab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7b40bd930 con 0x55f7b416f8d0 2024-03-23T17:54:39.398 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.397+0000 7fe3d67aab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7b40bb2f0 con 0x55f7b4164100 2024-03-23T17:54:39.398 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.397+0000 7fe3d67aab80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7b40b6be0 con 0x55f7b4164a20 2024-03-23T17:54:39.398 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.397+0000 7fe3bbe00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f7b4164100 0x55f7b41644e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:54:39.398 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.397+0000 7fe3bbe00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f7b4164100 0x55f7b41644e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:44974/0 (socket says 172.21.15.86:44974) 2024-03-23T17:54:39.399 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.397+0000 7fe3c0c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7b416f8d0 0x55f7b4171cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:54:39.399 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.397+0000 7fe3bbe00700 1 -- 172.21.15.86:0/3200735405 learned_addr learned my addr 172.21.15.86:0/3200735405 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:54:39.399 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.397+0000 7fe3bb400700 1 --2- 172.21.15.86:0/3200735405 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f7b4164a20 0x55f7b416f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:54:39.399 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.397+0000 7fe3bbe00700 1 -- 172.21.15.86:0/3200735405 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f7b4164a20 msgr2=0x55f7b416f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:54:39.399 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.397+0000 7fe3bbe00700 1 --2- 172.21.15.86:0/3200735405 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f7b4164a20 0x55f7b416f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:39.399 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.397+0000 7fe3bbe00700 1 -- 172.21.15.86:0/3200735405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7b416f8d0 msgr2=0x55f7b4171cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:54:39.399 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.397+0000 7fe3bbe00700 1 --2- 172.21.15.86:0/3200735405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7b416f8d0 0x55f7b4171cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:39.399 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.397+0000 7fe3bbe00700 1 -- 172.21.15.86:0/3200735405 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f7b3f38350 con 0x55f7b4164100 2024-03-23T17:54:39.399 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.397+0000 7fe3c0c00700 1 --2- 172.21.15.86:0/509998736 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fe3a00a1290 0x7fe3a00a3750 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe3b400f870 tx=0x7fe3b400fc30 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:54:39.399 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.398+0000 7fe3bbe00700 1 --2- 172.21.15.86:0/3200735405 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f7b4164100 0x55f7b41644e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac009d80 tx=0x7fe3ac011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3220b18738827a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:54:39.399 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.398+0000 7fe3b1200700 1 -- 172.21.15.86:0/3200735405 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac004230 con 0x55f7b4164100 2024-03-23T17:54:39.399 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.398+0000 7fe3b1200700 1 -- 172.21.15.86:0/3200735405 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3ac004390 con 0x55f7b4164100 2024-03-23T17:54:39.399 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.398+0000 7fe3b1200700 1 -- 172.21.15.86:0/3200735405 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac0197c0 con 0x55f7b4164100 2024-03-23T17:54:39.404 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.398+0000 7fe3d67aab80 1 -- 172.21.15.86:0/3200735405 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f7b4164100 msgr2=0x55f7b41644e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:54:39.404 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.398+0000 7fe3d67aab80 1 --2- 172.21.15.86:0/3200735405 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f7b4164100 0x55f7b41644e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac009d80 tx=0x7fe3ac011720 comp rx=0 tx=0).stop 2024-03-23T17:54:39.404 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.399+0000 7fe3d67aab80 1 -- 172.21.15.86:0/3200735405 shutdown_connections 2024-03-23T17:54:39.404 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.399+0000 7fe3d67aab80 1 --2- 172.21.15.86:0/3200735405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7b416f8d0 0x55f7b4171cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:39.404 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.399+0000 7fe3d67aab80 1 --2- 172.21.15.86:0/3200735405 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f7b4164100 0x55f7b41644e0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:39.404 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.399+0000 7fe3d67aab80 1 --2- 172.21.15.86:0/3200735405 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f7b4164a20 0x55f7b416f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:39.404 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.399+0000 7fe3d67aab80 1 -- 172.21.15.86:0/3200735405 >> 172.21.15.86:0/3200735405 conn(0x55f7b40c4640 msgr2=0x55f7b4160520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:54:39.404 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.399+0000 7fe3d67aab80 1 -- 172.21.15.86:0/3200735405 shutdown_connections 2024-03-23T17:54:39.404 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.399+0000 7fe3d67aab80 1 -- 172.21.15.86:0/3200735405 wait complete. 2024-03-23T17:54:39.405 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.399+0000 7fe3d67aab80 1 -- start start 2024-03-23T17:54:39.405 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.399+0000 7fe3d67aab80 1 Processor -- start 2024-03-23T17:54:39.405 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.400+0000 7fe3d67aab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7b4164100 0x55f7b416f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:54:39.405 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.400+0000 7fe3d67aab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f7b4164a20 0x55f7b416c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:54:39.405 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.400+0000 7fe3d67aab80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f7b416f8d0 0x55f7b416cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:54:39.405 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.400+0000 7fe3d67aab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7b4052d20 con 0x55f7b4164100 2024-03-23T17:54:39.405 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.400+0000 7fe3d67aab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7b40529c0 con 0x55f7b4164a20 2024-03-23T17:54:39.405 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.400+0000 7fe3d67aab80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7b3f3a640 con 0x55f7b416f8d0 2024-03-23T17:54:39.405 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.400+0000 7fe3d67aab80 10 client.0 _ll_register_callbacks cb 0x55f7b405c410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:54:39.405 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.400+0000 7fe3bbe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7b4164100 0x55f7b416f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:54:39.405 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.400+0000 7fe3c0c00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f7b416f8d0 0x55f7b416cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:54:39.405 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.400+0000 7fe3bbe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7b4164100 0x55f7b416f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59468/0 (socket says 172.21.15.86:59468) 2024-03-23T17:54:39.405 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.400+0000 7fe3bbe00700 1 -- 172.21.15.86:0/4047780778 learned_addr learned my addr 172.21.15.86:0/4047780778 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:54:39.405 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.400+0000 7fe3bb400700 1 --2- 172.21.15.86:0/4047780778 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f7b4164a20 0x55f7b416c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:54:39.406 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.400+0000 7fe3bbe00700 1 -- 172.21.15.86:0/4047780778 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f7b416f8d0 msgr2=0x55f7b416cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:54:39.406 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.400+0000 7fe3bbe00700 1 --2- 172.21.15.86:0/4047780778 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f7b416f8d0 0x55f7b416cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:39.406 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.400+0000 7fe3bbe00700 1 -- 172.21.15.86:0/4047780778 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f7b4164a20 msgr2=0x55f7b416c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:54:39.406 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.401+0000 7fe3bbe00700 1 --2- 172.21.15.86:0/4047780778 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f7b4164a20 0x55f7b416c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:39.406 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.401+0000 7fe3bbe00700 1 -- 172.21.15.86:0/4047780778 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3ac0099e0 con 0x55f7b4164100 2024-03-23T17:54:39.406 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.401+0000 7fe3bbe00700 1 --2- 172.21.15.86:0/4047780778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7b4164100 0x55f7b416f2a0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac005950 tx=0x7fe3ac0038b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2427feebeb02daf5 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:54:39.406 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.401+0000 7fe3a5600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-23T17:54:39.406 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.401+0000 7fe3a5600700 1 -- 172.21.15.86:0/4047780778 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac005450 con 0x55f7b4164100 2024-03-23T17:54:39.406 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.401+0000 7fe3a5600700 1 -- 172.21.15.86:0/4047780778 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fe3ac005ce0 con 0x55f7b4164100 2024-03-23T17:54:39.406 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.401+0000 7fe3d67aab80 1 -- 172.21.15.86:0/4047780778 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f7b4077320 con 0x55f7b4164100 2024-03-23T17:54:39.406 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.401+0000 7fe3a5600700 1 -- 172.21.15.86:0/4047780778 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3ac004500 con 0x55f7b4164100 2024-03-23T17:54:39.406 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.401+0000 7fe3d67aab80 10 client.16302 fetch_fsmap learned FSMap version 37 2024-03-23T17:54:39.407 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.401+0000 7fe3d67aab80 1 -- 172.21.15.86:0/4047780778 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55f7b3ee4b00 con 0x55f7b4164100 2024-03-23T17:54:39.407 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.402+0000 7fe3a5600700 1 -- 172.21.15.86:0/4047780778 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f7b4077320 con 0x55f7b4164100 2024-03-23T17:54:39.407 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.402+0000 7fe3a5600700 1 -- 172.21.15.86:0/4047780778 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 37) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fe3ac021040 con 0x55f7b4164100 2024-03-23T17:54:39.407 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.402+0000 7fe3d67aab80 10 client.16302 fetch_fsmap finished waiting for FSMap version 37 2024-03-23T17:54:39.407 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.402+0000 7fe3d67aab80 10 client.16302 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:54:39.407 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.402+0000 7fe3d67aab80 20 client.16302 populate_metadata read hostname 'smithi086' 2024-03-23T17:54:39.407 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.402+0000 7fe3d67aab80 1 --2- 172.21.15.86:0/4047780778 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55f7b417f420 0x55f7b4181810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:54:39.407 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.402+0000 7fe3d67aab80 4 client.16302 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:54:39.407 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.402+0000 7fe3d67aab80 1 -- 172.21.15.86:0/4047780778 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f7b40af6a0 con 0x55f7b417f420 2024-03-23T17:54:39.407 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.402+0000 7fe3bb400700 1 --2- 172.21.15.86:0/4047780778 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55f7b417f420 0x55f7b4181810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:54:39.407 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.403+0000 7fe3a5600700 10 client.16302 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:54:39.407 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.403+0000 7fe3bb400700 1 --2- 172.21.15.86:0/4047780778 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55f7b417f420 0x55f7b4181810 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba89ebecff9b460a server_cookie=f416e09c4390cf40 in_seq=0 out_seq=0 2024-03-23T17:54:39.407 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.403+0000 7fe3a5600700 1 -- 172.21.15.86:0/4047780778 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55f7b40af6a0 con 0x55f7b417f420 2024-03-23T17:54:39.408 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.403+0000 7fe3a5600700 10 client.16302 handle_command_reply: tid=0 2024-03-23T17:54:39.408 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.403+0000 7fe3d67aab80 1 client.16302 shutdown 2024-03-23T17:54:39.408 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.404+0000 7fe3d67aab80 1 -- 172.21.15.86:0/4047780778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7b4164100 msgr2=0x55f7b416f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:54:39.408 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.404+0000 7fe3d67aab80 1 --2- 172.21.15.86:0/4047780778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7b4164100 0x55f7b416f2a0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac005950 tx=0x7fe3ac0038b0 comp rx=0 tx=0).stop 2024-03-23T17:54:39.408 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.404+0000 7fe3d67aab80 1 -- 172.21.15.86:0/4047780778 shutdown_connections 2024-03-23T17:54:39.408 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.404+0000 7fe3d67aab80 1 --2- 172.21.15.86:0/4047780778 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55f7b417f420 0x55f7b4181810 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:39.408 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.404+0000 7fe3d67aab80 1 --2- 172.21.15.86:0/4047780778 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f7b416f8d0 0x55f7b416cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:39.408 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.404+0000 7fe3d67aab80 1 --2- 172.21.15.86:0/4047780778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7b4164100 0x55f7b416f2a0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:39.408 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.404+0000 7fe3a5600700 0 client.16302 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:54:39.408 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.404+0000 7fe3d67aab80 1 --2- 172.21.15.86:0/4047780778 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f7b4164a20 0x55f7b416c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:39.408 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.404+0000 7fe3d67aab80 1 -- 172.21.15.86:0/4047780778 >> 172.21.15.86:0/4047780778 conn(0x55f7b40c4640 msgr2=0x55f7b41660a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:54:39.408 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.404+0000 7fe3d67aab80 1 -- 172.21.15.86:0/4047780778 shutdown_connections 2024-03-23T17:54:39.408 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.404+0000 7fe3d67aab80 1 -- 172.21.15.86:0/4047780778 wait complete. 2024-03-23T17:54:39.408 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.404+0000 7fe3d67aab80 20 client.16302 trim_cache size 0 max 16384 2024-03-23T17:54:39.420 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.419+0000 7fe3d67aab80 1 Processor -- start 2024-03-23T17:54:39.420 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.419+0000 7fe3d67aab80 1 -- start start 2024-03-23T17:54:39.420 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.419+0000 7fe3d67aab80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f7b4164100 0x55f7b41765e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:54:39.420 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.419+0000 7fe3d67aab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f7b4164a20 0x55f7b416c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:54:39.420 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.419+0000 7fe3d67aab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7b416f8d0 0x55f7b415a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:54:39.420 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.419+0000 7fe3d67aab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7b4186530 con 0x55f7b416f8d0 2024-03-23T17:54:39.420 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.419+0000 7fe3d67aab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7b4164e00 con 0x55f7b4164a20 2024-03-23T17:54:39.420 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.419+0000 7fe3d67aab80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7b416aba0 con 0x55f7b4164100 2024-03-23T17:54:39.420 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.419+0000 7fe3bb400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f7b4164a20 0x55f7b416c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:54:39.420 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.419+0000 7fe3bb400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f7b4164a20 0x55f7b416c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:45002/0 (socket says 172.21.15.86:45002) 2024-03-23T17:54:39.420 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.419+0000 7fe3bb400700 1 -- 172.21.15.86:0/3487285223 learned_addr learned my addr 172.21.15.86:0/3487285223 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:54:39.421 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.419+0000 7fe3bbe00700 1 --2- 172.21.15.86:0/3487285223 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f7b4164100 0x55f7b41765e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:54:39.421 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.419+0000 7fe3c0c00700 1 --2- 172.21.15.86:0/3487285223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7b416f8d0 0x55f7b415a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:54:39.421 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.420+0000 7fe3bb400700 1 -- 172.21.15.86:0/3487285223 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f7b4164100 msgr2=0x55f7b41765e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:54:39.421 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.420+0000 7fe3bb400700 1 --2- 172.21.15.86:0/3487285223 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f7b4164100 0x55f7b41765e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:39.421 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.420+0000 7fe3bb400700 1 -- 172.21.15.86:0/3487285223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7b416f8d0 msgr2=0x55f7b415a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:54:39.421 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.420+0000 7fe3bb400700 1 --2- 172.21.15.86:0/3487285223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7b416f8d0 0x55f7b415a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:39.421 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.420+0000 7fe3bb400700 1 -- 172.21.15.86:0/3487285223 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f7b407ce40 con 0x55f7b4164a20 2024-03-23T17:54:39.421 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.420+0000 7fe3c0c00700 1 --2- 172.21.15.86:0/3487285223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7b416f8d0 0x55f7b415a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:54:39.425 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.420+0000 7fe3bb400700 1 --2- 172.21.15.86:0/3487285223 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f7b4164a20 0x55f7b416c5d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe3a809acf0 tx=0x7fe3a8021c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f15b05ed585c08dc server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:54:39.425 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.420+0000 7fe3b1c00700 1 -- 172.21.15.86:0/3487285223 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a809b6f0 con 0x55f7b4164a20 2024-03-23T17:54:39.425 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.420+0000 7fe3b1c00700 1 -- 172.21.15.86:0/3487285223 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3a809aff0 con 0x55f7b4164a20 2024-03-23T17:54:39.425 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.420+0000 7fe3b1c00700 1 -- 172.21.15.86:0/3487285223 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a8099460 con 0x55f7b4164a20 2024-03-23T17:54:39.425 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.421+0000 7fe3d67aab80 1 -- 172.21.15.86:0/3487285223 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f7b4164a20 msgr2=0x55f7b416c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:54:39.425 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.421+0000 7fe3d67aab80 1 --2- 172.21.15.86:0/3487285223 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f7b4164a20 0x55f7b416c5d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe3a809acf0 tx=0x7fe3a8021c60 comp rx=0 tx=0).stop 2024-03-23T17:54:39.425 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.421+0000 7fe3d67aab80 1 -- 172.21.15.86:0/3487285223 shutdown_connections 2024-03-23T17:54:39.425 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.421+0000 7fe3d67aab80 1 --2- 172.21.15.86:0/3487285223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7b416f8d0 0x55f7b415a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:39.425 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.421+0000 7fe3d67aab80 1 --2- 172.21.15.86:0/3487285223 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f7b4164100 0x55f7b41765e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:39.425 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.421+0000 7fe3d67aab80 1 --2- 172.21.15.86:0/3487285223 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f7b4164a20 0x55f7b416c5d0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:39.425 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.421+0000 7fe3d67aab80 1 -- 172.21.15.86:0/3487285223 >> 172.21.15.86:0/3487285223 conn(0x55f7b40c4640 msgr2=0x55f7b416bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:54:39.425 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.421+0000 7fe3d67aab80 1 -- 172.21.15.86:0/3487285223 shutdown_connections 2024-03-23T17:54:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.421+0000 7fe3d67aab80 1 -- 172.21.15.86:0/3487285223 wait complete. 2024-03-23T17:54:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.421+0000 7fe3d67aab80 1 -- start start 2024-03-23T17:54:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.422+0000 7fe3d67aab80 1 Processor -- start 2024-03-23T17:54:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.422+0000 7fe3d67aab80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f7b4164100 0x55f7b41747b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:54:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.422+0000 7fe3d67aab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7b4164a20 0x55f7b4174cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:54:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.422+0000 7fe3d67aab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f7b416f8d0 0x55f7b41c9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:54:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.422+0000 7fe3d67aab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7b41ca500 con 0x55f7b4164a20 2024-03-23T17:54:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.422+0000 7fe3d67aab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7b41ca670 con 0x55f7b416f8d0 2024-03-23T17:54:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.422+0000 7fe3d67aab80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7b41ca830 con 0x55f7b4164100 2024-03-23T17:54:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.422+0000 7fe3d67aab80 10 client.0 _ll_register_callbacks cb 0x55f7b405c410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:54:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.422+0000 7fe3bb400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7b4164a20 0x55f7b4174cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:54:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.422+0000 7fe3bb400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7b4164a20 0x55f7b4174cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59476/0 (socket says 172.21.15.86:59476) 2024-03-23T17:54:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.422+0000 7fe3bb400700 1 -- 172.21.15.86:0/379704024 learned_addr learned my addr 172.21.15.86:0/379704024 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:54:39.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.422+0000 7fe3c0c00700 1 --2- 172.21.15.86:0/379704024 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f7b416f8d0 0x55f7b41c9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:54:39.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.423+0000 7fe3bbe00700 1 --2- 172.21.15.86:0/379704024 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f7b4164100 0x55f7b41747b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:54:39.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.423+0000 7fe3bb400700 1 -- 172.21.15.86:0/379704024 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f7b4164100 msgr2=0x55f7b41747b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:54:39.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.423+0000 7fe3bb400700 1 --2- 172.21.15.86:0/379704024 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f7b4164100 0x55f7b41747b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:39.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.423+0000 7fe3bb400700 1 -- 172.21.15.86:0/379704024 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f7b416f8d0 msgr2=0x55f7b41c9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:54:39.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.423+0000 7fe3bb400700 1 --2- 172.21.15.86:0/379704024 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f7b416f8d0 0x55f7b41c9fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:39.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.423+0000 7fe3bb400700 1 -- 172.21.15.86:0/379704024 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3b4009040 con 0x55f7b4164a20 2024-03-23T17:54:39.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.423+0000 7fe3bb400700 1 --2- 172.21.15.86:0/379704024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7b4164a20 0x55f7b4174cf0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fe3a80974f0 tx=0x7fe3a8009330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eb869ee7f8574ab server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:54:39.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.423+0000 7fe3a5600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-23T17:54:39.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.423+0000 7fe3a5600700 1 -- 172.21.15.86:0/379704024 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a8099130 con 0x55f7b4164a20 2024-03-23T17:54:39.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.423+0000 7fe3d67aab80 1 -- 172.21.15.86:0/379704024 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe3ac004390 con 0x55f7b4164a20 2024-03-23T17:54:39.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.423+0000 7fe3a5600700 1 -- 172.21.15.86:0/379704024 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fe3a80b0e40 con 0x55f7b4164a20 2024-03-23T17:54:39.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.423+0000 7fe3a5600700 1 -- 172.21.15.86:0/379704024 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3a809b6f0 con 0x55f7b4164a20 2024-03-23T17:54:39.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.423+0000 7fe3d67aab80 10 client.16308 fetch_fsmap learned FSMap version 37 2024-03-23T17:54:39.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.423+0000 7fe3d67aab80 1 -- 172.21.15.86:0/379704024 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x7fe3bc1a6330 con 0x55f7b4164a20 2024-03-23T17:54:39.428 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.424+0000 7fe3a5600700 1 -- 172.21.15.86:0/379704024 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3ac004390 con 0x55f7b4164a20 2024-03-23T17:54:39.428 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.424+0000 7fe3a5600700 1 -- 172.21.15.86:0/379704024 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 37) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fe3a809a570 con 0x55f7b4164a20 2024-03-23T17:54:39.428 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.424+0000 7fe3d67aab80 10 client.16308 fetch_fsmap finished waiting for FSMap version 37 2024-03-23T17:54:39.428 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.424+0000 7fe3d67aab80 10 client.16308 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:54:39.428 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.424+0000 7fe3d67aab80 20 client.16308 populate_metadata read hostname 'smithi086' 2024-03-23T17:54:39.428 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.424+0000 7fe3d67aab80 1 --2- 172.21.15.86:0/379704024 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55f7b41d1c50 0x55f7b41d4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:54:39.428 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.424+0000 7fe3d67aab80 4 client.16308 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T17:54:39.428 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.424+0000 7fe3d67aab80 1 -- 172.21.15.86:0/379704024 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f7b3f05c90 con 0x55f7b41d1c50 2024-03-23T17:54:39.428 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.424+0000 7fe3bbe00700 1 --2- 172.21.15.86:0/379704024 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55f7b41d1c50 0x55f7b41d4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:54:39.428 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.425+0000 7fe3bbe00700 1 --2- 172.21.15.86:0/379704024 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55f7b41d1c50 0x55f7b41d4070 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcc1f3b6cfcaed4d server_cookie=573ed64adfd3b2c5 in_seq=0 out_seq=0 2024-03-23T17:54:39.428 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.425+0000 7fe3a5600700 10 client.16308 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:54:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.455+0000 7fe3a5600700 1 -- 172.21.15.86:0/379704024 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f7b3f05c90 con 0x55f7b41d1c50 2024-03-23T17:54:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.455+0000 7fe3a5600700 10 client.16308 handle_command_reply: tid=0 2024-03-23T17:54:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.455+0000 7fe3d67aab80 1 client.16308 shutdown 2024-03-23T17:54:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.455+0000 7fe3d67aab80 1 -- 172.21.15.86:0/379704024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7b4164a20 msgr2=0x55f7b4174cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:54:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.455+0000 7fe3d67aab80 1 --2- 172.21.15.86:0/379704024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7b4164a20 0x55f7b4174cf0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fe3a80974f0 tx=0x7fe3a8009330 comp rx=0 tx=0).stop 2024-03-23T17:54:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.455+0000 7fe3d67aab80 1 -- 172.21.15.86:0/379704024 shutdown_connections 2024-03-23T17:54:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.455+0000 7fe3d67aab80 1 --2- 172.21.15.86:0/379704024 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55f7b41d1c50 0x55f7b41d4070 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.455+0000 7fe3d67aab80 1 --2- 172.21.15.86:0/379704024 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f7b416f8d0 0x55f7b41c9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.455+0000 7fe3d67aab80 1 --2- 172.21.15.86:0/379704024 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f7b4164100 0x55f7b41747b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.455+0000 7fe3d67aab80 1 --2- 172.21.15.86:0/379704024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f7b4164a20 0x55f7b4174cf0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.455+0000 7fe3a5600700 0 client.16308 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:54:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.455+0000 7fe3d67aab80 1 -- 172.21.15.86:0/379704024 >> 172.21.15.86:0/379704024 conn(0x55f7b40c4640 msgr2=0x55f7b416bde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:54:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.455+0000 7fe3d67aab80 1 -- 172.21.15.86:0/379704024 shutdown_connections 2024-03-23T17:54:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.455+0000 7fe3d67aab80 1 -- 172.21.15.86:0/379704024 wait complete. 2024-03-23T17:54:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.456+0000 7fe3d67aab80 20 client.16308 trim_cache size 0 max 16384 2024-03-23T17:54:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-03-23T17:54:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-03-23T17:54:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-03-23T17:54:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-03-23T17:54:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.456+0000 7fe3c2c00700 1 -- 172.21.15.86:0/509998736 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fe3a00a1290 msgr2=0x7fe3a00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:54:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.456+0000 7fe3c2c00700 1 --2- 172.21.15.86:0/509998736 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fe3a00a1290 0x7fe3a00a3750 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe3b400f870 tx=0x7fe3b400fc30 comp rx=0 tx=0).stop 2024-03-23T17:54:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.456+0000 7fe3c2c00700 1 -- 172.21.15.86:0/509998736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3bc106010 msgr2=0x7fe3bc0735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:54:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.456+0000 7fe3c2c00700 1 --2- 172.21.15.86:0/509998736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3bc106010 0x7fe3bc0735f0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fe3a8000c00 tx=0x7fe3a8004730 comp rx=0 tx=0).stop 2024-03-23T17:54:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.456+0000 7fe3c2c00700 1 -- 172.21.15.86:0/509998736 shutdown_connections 2024-03-23T17:54:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.456+0000 7fe3c2c00700 1 --2- 172.21.15.86:0/509998736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3bc073b30 0x7fe3bc1a5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.456+0000 7fe3c2c00700 1 --2- 172.21.15.86:0/509998736 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fe3a00a1290 0x7fe3a00a3750 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.456+0000 7fe3c2c00700 1 --2- 172.21.15.86:0/509998736 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3bc100ef0 0x7fe3bc0730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.456+0000 7fe3c2c00700 1 --2- 172.21.15.86:0/509998736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3bc106010 0x7fe3bc0735f0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.456+0000 7fe3c2c00700 1 -- 172.21.15.86:0/509998736 >> 172.21.15.86:0/509998736 conn(0x7fe3bc0fc750 msgr2=0x7fe3bc0fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:54:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.456+0000 7fe3c2c00700 1 -- 172.21.15.86:0/509998736 shutdown_connections 2024-03-23T17:54:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:39.456+0000 7fe3c2c00700 1 -- 172.21.15.86:0/509998736 wait complete. 2024-03-23T17:54:39.895 INFO:tasks.workunit.client.0.smithi086.stdout: 1 110667 2.39 MB/sec execute 64 sec latency 2140.422 ms 2024-03-23T17:54:40.895 INFO:tasks.workunit.client.0.smithi086.stdout: 1 110667 2.35 MB/sec execute 65 sec latency 3140.521 ms 2024-03-23T17:54:41.895 INFO:tasks.workunit.client.0.smithi086.stdout: 1 112914 2.50 MB/sec execute 66 sec latency 3151.348 ms 2024-03-23T17:54:42.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:54:41 smithi086 ceph-mon[38221]: pgmap v124: 97 pgs: 97 active+clean; 52 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 867 KiB/s wr, 113 op/s 2024-03-23T17:54:42.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:54:41 smithi092 ceph-mon[79089]: pgmap v124: 97 pgs: 97 active+clean; 52 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 867 KiB/s wr, 113 op/s 2024-03-23T17:54:42.085 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:54:41 smithi152 ceph-mon[47949]: pgmap v124: 97 pgs: 97 active+clean; 52 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 867 KiB/s wr, 113 op/s 2024-03-23T17:54:42.895 INFO:tasks.workunit.client.0.smithi086.stdout: 1 114266 2.57 MB/sec execute 67 sec latency 11.014 ms 2024-03-23T17:54:43.895 INFO:tasks.workunit.client.0.smithi086.stdout: 1 114266 2.53 MB/sec execute 68 sec latency 1005.115 ms 2024-03-23T17:54:44.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:54:43 smithi086 ceph-mon[38221]: pgmap v125: 97 pgs: 97 active+clean; 52 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 824 KiB/s wr, 93 op/s 2024-03-23T17:54:44.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:54:43 smithi092 ceph-mon[79089]: pgmap v125: 97 pgs: 97 active+clean; 52 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 824 KiB/s wr, 93 op/s 2024-03-23T17:54:44.085 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:54:43 smithi152 ceph-mon[47949]: pgmap v125: 97 pgs: 97 active+clean; 52 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 824 KiB/s wr, 93 op/s 2024-03-23T17:54:44.896 INFO:tasks.workunit.client.0.smithi086.stdout: 1 114266 2.49 MB/sec execute 69 sec latency 2005.234 ms 2024-03-23T17:54:45.896 INFO:tasks.workunit.client.0.smithi086.stdout: 1 114266 2.46 MB/sec execute 70 sec latency 3005.365 ms 2024-03-23T17:54:46.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:54:45 smithi086 ceph-mon[38221]: pgmap v126: 97 pgs: 97 active+clean; 54 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 950 KiB/s rd, 1.3 MiB/s wr, 157 op/s 2024-03-23T17:54:46.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:54:45 smithi092 ceph-mon[79089]: pgmap v126: 97 pgs: 97 active+clean; 54 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 950 KiB/s rd, 1.3 MiB/s wr, 157 op/s 2024-03-23T17:54:46.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:54:45 smithi152 ceph-mon[47949]: pgmap v126: 97 pgs: 97 active+clean; 54 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 950 KiB/s rd, 1.3 MiB/s wr, 157 op/s 2024-03-23T17:54:46.896 INFO:tasks.workunit.client.0.smithi086.stdout: 1 114334 2.42 MB/sec execute 71 sec latency 3021.492 ms 2024-03-23T17:54:47.896 INFO:tasks.workunit.client.0.smithi086.stdout: 1 114334 2.39 MB/sec execute 72 sec latency 1935.879 ms 2024-03-23T17:54:48.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:54:47 smithi086 ceph-mon[38221]: pgmap v127: 97 pgs: 97 active+clean; 54 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 760 KiB/s wr, 106 op/s 2024-03-23T17:54:48.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:54:47 smithi092 ceph-mon[79089]: pgmap v127: 97 pgs: 97 active+clean; 54 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 760 KiB/s wr, 106 op/s 2024-03-23T17:54:48.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:54:47 smithi152 ceph-mon[47949]: pgmap v127: 97 pgs: 97 active+clean; 54 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 760 KiB/s wr, 106 op/s 2024-03-23T17:54:48.896 INFO:tasks.workunit.client.0.smithi086.stdout: 1 114334 2.36 MB/sec execute 73 sec latency 2935.924 ms 2024-03-23T17:54:49.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:54:48 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:54:49.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:54:48 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:54:49.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:54:48 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:54:49.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:54:48 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:54:49.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:54:48 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:54:49.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:54:48 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:54:49.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:54:48 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:54:49.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:54:48 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:54:49.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:54:48 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:54:49.896 INFO:tasks.workunit.client.0.smithi086.stdout: 1 114334 2.33 MB/sec execute 74 sec latency 3936.034 ms 2024-03-23T17:54:50.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:54:49 smithi086 ceph-mon[38221]: pgmap v128: 97 pgs: 97 active+clean; 54 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 760 KiB/s wr, 106 op/s 2024-03-23T17:54:50.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:54:49 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:54:50.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:54:49 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:54:50.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:54:49 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:54:50.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:54:49 smithi092 ceph-mon[79089]: pgmap v128: 97 pgs: 97 active+clean; 54 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 760 KiB/s wr, 106 op/s 2024-03-23T17:54:50.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:54:49 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:54:50.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:54:49 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:54:50.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:54:49 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:54:50.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:54:49 smithi152 ceph-mon[47949]: pgmap v128: 97 pgs: 97 active+clean; 54 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 760 KiB/s wr, 106 op/s 2024-03-23T17:54:50.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:54:49 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:54:50.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:54:49 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:54:50.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:54:49 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:54:50.896 INFO:tasks.workunit.client.0.smithi086.stdout: 1 114334 2.29 MB/sec execute 75 sec latency 4936.106 ms 2024-03-23T17:54:51.896 INFO:tasks.workunit.client.0.smithi086.stdout: 1 116709 2.43 MB/sec execute 76 sec latency 4944.645 ms 2024-03-23T17:54:52.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:54:51 smithi086 ceph-mon[38221]: pgmap v129: 97 pgs: 97 active+clean; 55 MiB data, 697 MiB used, 1.0 TiB / 1.0 TiB avail; 750 KiB/s rd, 799 KiB/s wr, 118 op/s 2024-03-23T17:54:52.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:54:51 smithi092 ceph-mon[79089]: pgmap v129: 97 pgs: 97 active+clean; 55 MiB data, 697 MiB used, 1.0 TiB / 1.0 TiB avail; 750 KiB/s rd, 799 KiB/s wr, 118 op/s 2024-03-23T17:54:52.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:54:51 smithi152 ceph-mon[47949]: pgmap v129: 97 pgs: 97 active+clean; 55 MiB data, 697 MiB used, 1.0 TiB / 1.0 TiB avail; 750 KiB/s rd, 799 KiB/s wr, 118 op/s 2024-03-23T17:54:52.605 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.605+0000 7f5b7f000700 1 -- 172.21.15.86:0/2619536332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b781018a0 msgr2=0x7f5b781058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:54:52.605 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.605+0000 7f5b7f000700 1 --2- 172.21.15.86:0/2619536332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b781018a0 0x7f5b781058e0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f5b640098f0 tx=0x7f5b64009c00 comp rx=0 tx=0).stop 2024-03-23T17:54:52.605 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.605+0000 7f5b7f000700 1 -- 172.21.15.86:0/2619536332 shutdown_connections 2024-03-23T17:54:52.605 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.605+0000 7f5b7f000700 1 --2- 172.21.15.86:0/2619536332 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5b78106010 0x7f5b7810db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:52.606 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.605+0000 7f5b7f000700 1 --2- 172.21.15.86:0/2619536332 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b78100ef0 0x7f5b781012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:52.606 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.605+0000 7f5b7f000700 1 --2- 172.21.15.86:0/2619536332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b781018a0 0x7f5b781058e0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:52.606 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.605+0000 7f5b7f000700 1 -- 172.21.15.86:0/2619536332 >> 172.21.15.86:0/2619536332 conn(0x7f5b780fc750 msgr2=0x7f5b780feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:54:52.606 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.605+0000 7f5b7f000700 1 -- 172.21.15.86:0/2619536332 shutdown_connections 2024-03-23T17:54:52.606 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.605+0000 7f5b7f000700 1 -- 172.21.15.86:0/2619536332 wait complete. 2024-03-23T17:54:52.608 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.606+0000 7f5b7f000700 1 Processor -- start 2024-03-23T17:54:52.608 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.606+0000 7f5b7f000700 1 -- start start 2024-03-23T17:54:52.608 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.606+0000 7f5b7f000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b78100ef0 0x7f5b781993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:54:52.608 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.606+0000 7f5b7f000700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5b78106010 0x7f5b78199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:54:52.608 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.606+0000 7f5b7f000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b7819dcd0 0x7f5b7819e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:54:52.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.606+0000 7f5b7f000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7819e750 con 0x7f5b7819dcd0 2024-03-23T17:54:52.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.606+0000 7f5b7f000700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7819e8c0 con 0x7f5b78100ef0 2024-03-23T17:54:52.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.607+0000 7f5b7f000700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7819ea80 con 0x7f5b78106010 2024-03-23T17:54:52.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.607+0000 7f5b77e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b78100ef0 0x7f5b781993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:54:52.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.607+0000 7f5b77e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b78100ef0 0x7f5b781993f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:51506/0 (socket says 172.21.15.86:51506) 2024-03-23T17:54:52.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.607+0000 7f5b77e00700 1 -- 172.21.15.86:0/1784122949 learned_addr learned my addr 172.21.15.86:0/1784122949 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:54:52.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.607+0000 7f5b77400700 1 --2- 172.21.15.86:0/1784122949 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5b78106010 0x7f5b78199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:54:52.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.607+0000 7f5b7d000700 1 --2- 172.21.15.86:0/1784122949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b7819dcd0 0x7f5b7819e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:54:52.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.607+0000 7f5b77e00700 1 -- 172.21.15.86:0/1784122949 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5b78106010 msgr2=0x7f5b78199930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:54:52.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.607+0000 7f5b77e00700 1 --2- 172.21.15.86:0/1784122949 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5b78106010 0x7f5b78199930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:52.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.607+0000 7f5b77e00700 1 -- 172.21.15.86:0/1784122949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b7819dcd0 msgr2=0x7f5b7819e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:54:52.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.607+0000 7f5b77e00700 1 --2- 172.21.15.86:0/1784122949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b7819dcd0 0x7f5b7819e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:52.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.607+0000 7f5b77e00700 1 -- 172.21.15.86:0/1784122949 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b64009520 con 0x7f5b78100ef0 2024-03-23T17:54:52.610 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.607+0000 7f5b77e00700 1 --2- 172.21.15.86:0/1784122949 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b78100ef0 0x7f5b781993f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5b6800e9c0 tx=0x7f5b6800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:54:52.610 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.607+0000 7f5b74c00700 1 -- 172.21.15.86:0/1784122949 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b68004a30 con 0x7f5b78100ef0 2024-03-23T17:54:52.610 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.608+0000 7f5b74c00700 1 -- 172.21.15.86:0/1784122949 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b6800cc80 con 0x7f5b78100ef0 2024-03-23T17:54:52.610 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.608+0000 7f5b7f000700 1 -- 172.21.15.86:0/1784122949 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b781a5fe0 con 0x7f5b78100ef0 2024-03-23T17:54:52.610 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.608+0000 7f5b74c00700 1 -- 172.21.15.86:0/1784122949 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b680054e0 con 0x7f5b78100ef0 2024-03-23T17:54:52.610 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.608+0000 7f5b7f000700 1 -- 172.21.15.86:0/1784122949 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b781a6410 con 0x7f5b78100ef0 2024-03-23T17:54:52.610 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.611+0000 7f5b74c00700 1 -- 172.21.15.86:0/1784122949 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f5b68004b90 con 0x7f5b78100ef0 2024-03-23T17:54:52.612 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.612+0000 7f5b92abcb80 1 Processor -- start 2024-03-23T17:54:52.612 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.612+0000 7f5b74c00700 1 --2- 172.21.15.86:0/1784122949 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f5b5c0a1310 0x7f5b5c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:54:52.612 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.612+0000 7f5b92abcb80 1 -- start start 2024-03-23T17:54:52.614 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.612+0000 7f5b74c00700 1 -- 172.21.15.86:0/1784122949 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f5b68014070 con 0x7f5b78100ef0 2024-03-23T17:54:52.614 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.612+0000 7f5b92abcb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562d49651910 0x562d49651cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:54:52.614 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.612+0000 7f5b92abcb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562d49652230 0x562d4965cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:54:52.614 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.612+0000 7f5b92abcb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562d4965d0e0 0x562d4965f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:54:52.614 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.612+0000 7f5b92abcb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d495a2930 con 0x562d49651910 2024-03-23T17:54:52.614 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.612+0000 7f5b92abcb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d495a02f0 con 0x562d4965d0e0 2024-03-23T17:54:52.614 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.612+0000 7f5b92abcb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d4959bbe0 con 0x562d49652230 2024-03-23T17:54:52.614 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.612+0000 7f5b7d000700 1 --2- 172.21.15.86:0/1784122949 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f5b5c0a1310 0x7f5b5c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:54:52.614 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.613+0000 7f5b77400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562d49652230 0x562d4965cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:54:52.614 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.613+0000 7f5b77400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562d49652230 0x562d4965cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:53216/0 (socket says 172.21.15.86:53216) 2024-03-23T17:54:52.614 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.613+0000 7f5b77400700 1 -- 172.21.15.86:0/1218994545 learned_addr learned my addr 172.21.15.86:0/1218994545 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:54:52.615 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.613+0000 7f5b77e00700 1 --2- 172.21.15.86:0/1218994545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562d49651910 0x562d49651cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:54:52.615 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.613+0000 7f5b77400700 1 -- 172.21.15.86:0/1218994545 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562d4965d0e0 msgr2=0x562d4965f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:54:52.615 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.613+0000 7f5b77400700 1 --2- 172.21.15.86:0/1218994545 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562d4965d0e0 0x562d4965f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:52.615 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.613+0000 7f5b77400700 1 -- 172.21.15.86:0/1218994545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562d49651910 msgr2=0x562d49651cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:54:52.615 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.613+0000 7f5b77400700 1 --2- 172.21.15.86:0/1218994545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562d49651910 0x562d49651cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:52.615 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.613+0000 7f5b77400700 1 -- 172.21.15.86:0/1218994545 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d4941d350 con 0x562d49652230 2024-03-23T17:54:52.615 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.613+0000 7f5b7d000700 1 --2- 172.21.15.86:0/1784122949 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f5b5c0a1310 0x7f5b5c0a37d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5b7000f540 tx=0x7f5b70009920 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:54:52.615 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.613+0000 7f5b77400700 1 --2- 172.21.15.86:0/1218994545 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562d49652230 0x562d4965cba0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5b7806a370 tx=0x7f5b64005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc8a16e9a9a72c3a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:54:52.615 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.613+0000 7f5b6d600700 1 -- 172.21.15.86:0/1218994545 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b6400fd70 con 0x562d49652230 2024-03-23T17:54:52.615 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.614+0000 7f5b6d600700 1 -- 172.21.15.86:0/1218994545 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b6401d040 con 0x562d49652230 2024-03-23T17:54:52.615 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.614+0000 7f5b6d600700 1 -- 172.21.15.86:0/1218994545 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b64005040 con 0x562d49652230 2024-03-23T17:54:52.620 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.614+0000 7f5b92abcb80 1 -- 172.21.15.86:0/1218994545 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562d49652230 msgr2=0x562d4965cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:54:52.620 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.614+0000 7f5b92abcb80 1 --2- 172.21.15.86:0/1218994545 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562d49652230 0x562d4965cba0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5b7806a370 tx=0x7f5b64005dc0 comp rx=0 tx=0).stop 2024-03-23T17:54:52.620 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.614+0000 7f5b92abcb80 1 -- 172.21.15.86:0/1218994545 shutdown_connections 2024-03-23T17:54:52.620 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.614+0000 7f5b92abcb80 1 --2- 172.21.15.86:0/1218994545 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562d4965d0e0 0x562d4965f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:52.620 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.614+0000 7f5b92abcb80 1 --2- 172.21.15.86:0/1218994545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562d49651910 0x562d49651cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:52.620 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.614+0000 7f5b92abcb80 1 --2- 172.21.15.86:0/1218994545 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562d49652230 0x562d4965cba0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:52.620 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.614+0000 7f5b92abcb80 1 -- 172.21.15.86:0/1218994545 >> 172.21.15.86:0/1218994545 conn(0x562d495a9640 msgr2=0x562d49645520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:54:52.620 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.614+0000 7f5b92abcb80 1 -- 172.21.15.86:0/1218994545 shutdown_connections 2024-03-23T17:54:52.620 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.614+0000 7f5b92abcb80 1 -- 172.21.15.86:0/1218994545 wait complete. 2024-03-23T17:54:52.621 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.615+0000 7f5b92abcb80 1 -- start start 2024-03-23T17:54:52.621 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.615+0000 7f5b92abcb80 1 Processor -- start 2024-03-23T17:54:52.621 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.615+0000 7f5b92abcb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562d49651910 0x562d4965ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:54:52.621 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.615+0000 7f5b92abcb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562d49652230 0x562d49659ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:54:52.621 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.615+0000 7f5b92abcb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562d4965d0e0 0x562d4965a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:54:52.621 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.616+0000 7f5b92abcb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d49537d20 con 0x562d4965d0e0 2024-03-23T17:54:52.621 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.616+0000 7f5b92abcb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d495379c0 con 0x562d49652230 2024-03-23T17:54:52.621 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.616+0000 7f5b92abcb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d4941f640 con 0x562d49651910 2024-03-23T17:54:52.621 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.616+0000 7f5b92abcb80 10 client.0 _ll_register_callbacks cb 0x562d49541410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:54:52.621 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.616+0000 7f5b77e00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562d49651910 0x562d4965ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:54:52.621 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.616+0000 7f5b7d000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562d4965d0e0 0x562d4965a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:54:52.621 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.616+0000 7f5b77400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562d49652230 0x562d49659ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:54:52.621 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.616+0000 7f5b77e00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562d49651910 0x562d4965ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:53224/0 (socket says 172.21.15.86:53224) 2024-03-23T17:54:52.622 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.616+0000 7f5b77e00700 1 -- 172.21.15.86:0/1506272676 learned_addr learned my addr 172.21.15.86:0/1506272676 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:54:52.622 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.616+0000 7f5b7d000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562d4965d0e0 0x562d4965a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43950/0 (socket says 172.21.15.86:43950) 2024-03-23T17:54:52.622 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.616+0000 7f5b7d000700 1 -- 172.21.15.86:0/1506272676 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562d49651910 msgr2=0x562d4965ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:54:52.622 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.616+0000 7f5b7d000700 1 --2- 172.21.15.86:0/1506272676 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562d49651910 0x562d4965ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:52.622 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.616+0000 7f5b7d000700 1 -- 172.21.15.86:0/1506272676 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562d49652230 msgr2=0x562d49659ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:54:52.622 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.616+0000 7f5b7d000700 1 --2- 172.21.15.86:0/1506272676 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562d49652230 0x562d49659ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:52.622 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.616+0000 7f5b7d000700 1 -- 172.21.15.86:0/1506272676 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b7810e400 con 0x562d4965d0e0 2024-03-23T17:54:52.622 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.617+0000 7f5b77e00700 1 --2- 172.21.15.86:0/1506272676 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562d49651910 0x562d4965ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:54:52.622 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.617+0000 7f5b77400700 1 --2- 172.21.15.86:0/1506272676 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562d49652230 0x562d49659ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:54:52.622 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.617+0000 7f5b7d000700 1 --2- 172.21.15.86:0/1506272676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562d4965d0e0 0x562d4965a240 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f5b7000fc90 tx=0x7f5b70004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd9e2be56032843e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:54:52.622 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.617+0000 7f5b62000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-23T17:54:52.622 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.617+0000 7f5b62000700 1 -- 172.21.15.86:0/1506272676 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b7000e040 con 0x562d4965d0e0 2024-03-23T17:54:52.622 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.617+0000 7f5b62000700 1 -- 172.21.15.86:0/1506272676 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f5b70004db0 con 0x562d4965d0e0 2024-03-23T17:54:52.622 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.617+0000 7f5b92abcb80 1 -- 172.21.15.86:0/1506272676 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562d4955c320 con 0x562d4965d0e0 2024-03-23T17:54:52.623 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.617+0000 7f5b62000700 1 -- 172.21.15.86:0/1506272676 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b70017e50 con 0x562d4965d0e0 2024-03-23T17:54:52.623 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.617+0000 7f5b92abcb80 10 client.16320 fetch_fsmap learned FSMap version 37 2024-03-23T17:54:52.623 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.618+0000 7f5b92abcb80 1 -- 172.21.15.86:0/1506272676 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x562d493c9b00 con 0x562d4965d0e0 2024-03-23T17:54:52.623 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.618+0000 7f5b62000700 1 -- 172.21.15.86:0/1506272676 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562d4955c320 con 0x562d4965d0e0 2024-03-23T17:54:52.623 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.618+0000 7f5b62000700 1 -- 172.21.15.86:0/1506272676 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 37) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f5b70005550 con 0x562d4965d0e0 2024-03-23T17:54:52.623 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.618+0000 7f5b92abcb80 10 client.16320 fetch_fsmap finished waiting for FSMap version 37 2024-03-23T17:54:52.623 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.618+0000 7f5b92abcb80 10 client.16320 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:54:52.623 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.618+0000 7f5b92abcb80 20 client.16320 populate_metadata read hostname 'smithi086' 2024-03-23T17:54:52.623 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.618+0000 7f5b92abcb80 1 --2- 172.21.15.86:0/1506272676 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x562d4966cd40 0x562d4966f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:54:52.623 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.618+0000 7f5b92abcb80 4 client.16320 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:54:52.623 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.618+0000 7f5b92abcb80 1 -- 172.21.15.86:0/1506272676 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562d495946a0 con 0x562d4966cd40 2024-03-23T17:54:52.623 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.618+0000 7f5b77e00700 1 --2- 172.21.15.86:0/1506272676 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x562d4966cd40 0x562d4966f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:54:52.623 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.619+0000 7f5b77e00700 1 --2- 172.21.15.86:0/1506272676 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x562d4966cd40 0x562d4966f160 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f11fa1cae319218 server_cookie=bf36b5a044c47339 in_seq=0 out_seq=0 2024-03-23T17:54:52.624 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.619+0000 7f5b62000700 10 client.16320 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:54:52.624 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.619+0000 7f5b62000700 1 -- 172.21.15.86:0/1506272676 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x562d495946a0 con 0x562d4966cd40 2024-03-23T17:54:52.624 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.619+0000 7f5b62000700 10 client.16320 handle_command_reply: tid=0 2024-03-23T17:54:52.624 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.619+0000 7f5b92abcb80 1 client.16320 shutdown 2024-03-23T17:54:52.624 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.620+0000 7f5b92abcb80 1 -- 172.21.15.86:0/1506272676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562d4965d0e0 msgr2=0x562d4965a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:54:52.624 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.620+0000 7f5b92abcb80 1 --2- 172.21.15.86:0/1506272676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562d4965d0e0 0x562d4965a240 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f5b7000fc90 tx=0x7f5b70004b40 comp rx=0 tx=0).stop 2024-03-23T17:54:52.624 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.620+0000 7f5b92abcb80 1 -- 172.21.15.86:0/1506272676 shutdown_connections 2024-03-23T17:54:52.624 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.620+0000 7f5b92abcb80 1 --2- 172.21.15.86:0/1506272676 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x562d4966cd40 0x562d4966f160 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:52.624 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.620+0000 7f5b92abcb80 1 --2- 172.21.15.86:0/1506272676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562d4965d0e0 0x562d4965a240 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:52.624 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.620+0000 7f5b92abcb80 1 --2- 172.21.15.86:0/1506272676 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562d49651910 0x562d4965ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:52.624 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.620+0000 7f5b62000700 0 client.16320 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:54:52.624 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.620+0000 7f5b92abcb80 1 --2- 172.21.15.86:0/1506272676 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562d49652230 0x562d49659ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:52.624 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.620+0000 7f5b92abcb80 1 -- 172.21.15.86:0/1506272676 >> 172.21.15.86:0/1506272676 conn(0x562d495a9640 msgr2=0x562d49653890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:54:52.624 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.620+0000 7f5b92abcb80 1 -- 172.21.15.86:0/1506272676 shutdown_connections 2024-03-23T17:54:52.624 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.620+0000 7f5b92abcb80 1 -- 172.21.15.86:0/1506272676 wait complete. 2024-03-23T17:54:52.624 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.620+0000 7f5b92abcb80 20 client.16320 trim_cache size 0 max 16384 2024-03-23T17:54:52.635 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.635+0000 7f5b92abcb80 1 Processor -- start 2024-03-23T17:54:52.635 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.635+0000 7f5b92abcb80 1 -- start start 2024-03-23T17:54:52.637 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.635+0000 7f5b92abcb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562d49651910 0x562d49663dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:54:52.637 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.635+0000 7f5b92abcb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562d49652230 0x562d4963f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:54:52.637 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.635+0000 7f5b92abcb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562d4965d0e0 0x562d49668aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:54:52.637 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.635+0000 7f5b92abcb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d49646ac0 con 0x562d49651910 2024-03-23T17:54:52.637 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.635+0000 7f5b92abcb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d49652bc0 con 0x562d49652230 2024-03-23T17:54:52.637 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.635+0000 7f5b92abcb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d49659ad0 con 0x562d4965d0e0 2024-03-23T17:54:52.637 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.636+0000 7f5b77e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562d49651910 0x562d49663dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:54:52.637 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.636+0000 7f5b77e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562d49651910 0x562d49663dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43964/0 (socket says 172.21.15.86:43964) 2024-03-23T17:54:52.637 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.636+0000 7f5b77e00700 1 -- 172.21.15.86:0/2472492948 learned_addr learned my addr 172.21.15.86:0/2472492948 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:54:52.637 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.636+0000 7f5b7d000700 1 --2- 172.21.15.86:0/2472492948 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562d4965d0e0 0x562d49668aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:54:52.637 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.636+0000 7f5b77400700 1 --2- 172.21.15.86:0/2472492948 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562d49652230 0x562d4963f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:54:52.637 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.636+0000 7f5b77e00700 1 -- 172.21.15.86:0/2472492948 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562d4965d0e0 msgr2=0x562d49668aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:54:52.637 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.636+0000 7f5b77e00700 1 --2- 172.21.15.86:0/2472492948 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562d4965d0e0 0x562d49668aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:52.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.636+0000 7f5b77e00700 1 -- 172.21.15.86:0/2472492948 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562d49652230 msgr2=0x562d4963f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:54:52.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.636+0000 7f5b77e00700 1 --2- 172.21.15.86:0/2472492948 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562d49652230 0x562d4963f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:52.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.636+0000 7f5b77e00700 1 -- 172.21.15.86:0/2472492948 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d49561e40 con 0x562d49651910 2024-03-23T17:54:52.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.636+0000 7f5b77400700 1 --2- 172.21.15.86:0/2472492948 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562d49652230 0x562d4963f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:54:52.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.636+0000 7f5b77e00700 1 --2- 172.21.15.86:0/2472492948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562d49651910 0x562d49663dc0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f5b680ca3d0 tx=0x7f5b6809c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bb0875db64e545b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:54:52.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.636+0000 7f5b6e000700 1 -- 172.21.15.86:0/2472492948 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b680a2270 con 0x562d49651910 2024-03-23T17:54:52.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.637+0000 7f5b6e000700 1 -- 172.21.15.86:0/2472492948 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b68097f30 con 0x562d49651910 2024-03-23T17:54:52.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.637+0000 7f5b6e000700 1 -- 172.21.15.86:0/2472492948 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b6809a050 con 0x562d49651910 2024-03-23T17:54:52.641 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.637+0000 7f5b92abcb80 1 -- 172.21.15.86:0/2472492948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562d49651910 msgr2=0x562d49663dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:54:52.641 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.637+0000 7f5b92abcb80 1 --2- 172.21.15.86:0/2472492948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562d49651910 0x562d49663dc0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f5b680ca3d0 tx=0x7f5b6809c040 comp rx=0 tx=0).stop 2024-03-23T17:54:52.641 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.637+0000 7f5b92abcb80 1 -- 172.21.15.86:0/2472492948 shutdown_connections 2024-03-23T17:54:52.642 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.637+0000 7f5b92abcb80 1 --2- 172.21.15.86:0/2472492948 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562d4965d0e0 0x562d49668aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:52.642 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.637+0000 7f5b92abcb80 1 --2- 172.21.15.86:0/2472492948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562d49651910 0x562d49663dc0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:52.642 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.637+0000 7f5b92abcb80 1 --2- 172.21.15.86:0/2472492948 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562d49652230 0x562d4963f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:52.642 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.637+0000 7f5b92abcb80 1 -- 172.21.15.86:0/2472492948 >> 172.21.15.86:0/2472492948 conn(0x562d495a9640 msgr2=0x562d49668390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:54:52.642 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.637+0000 7f5b92abcb80 1 -- 172.21.15.86:0/2472492948 shutdown_connections 2024-03-23T17:54:52.642 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.637+0000 7f5b92abcb80 1 -- 172.21.15.86:0/2472492948 wait complete. 2024-03-23T17:54:52.642 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.638+0000 7f5b92abcb80 1 -- start start 2024-03-23T17:54:52.642 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.638+0000 7f5b92abcb80 1 Processor -- start 2024-03-23T17:54:52.642 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.639+0000 7f5b92abcb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562d49651910 0x562d496634f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:54:52.642 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.639+0000 7f5b92abcb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562d49652230 0x562d49663a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:54:52.642 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.639+0000 7f5b92abcb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562d4965d0e0 0x562d49661ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:54:52.642 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.639+0000 7f5b92abcb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d49662170 con 0x562d49652230 2024-03-23T17:54:52.642 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.639+0000 7f5b92abcb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d496622e0 con 0x562d4965d0e0 2024-03-23T17:54:52.643 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.639+0000 7f5b92abcb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d496624a0 con 0x562d49651910 2024-03-23T17:54:52.643 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.639+0000 7f5b92abcb80 10 client.0 _ll_register_callbacks cb 0x562d49541410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:54:52.643 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.639+0000 7f5b77400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562d49652230 0x562d49663a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:54:52.643 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.639+0000 7f5b7d000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562d4965d0e0 0x562d49661ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:54:52.643 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.639+0000 7f5b7d000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562d4965d0e0 0x562d49661ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:51550/0 (socket says 172.21.15.86:51550) 2024-03-23T17:54:52.643 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.639+0000 7f5b7d000700 1 -- 172.21.15.86:0/187425696 learned_addr learned my addr 172.21.15.86:0/187425696 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:54:52.643 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.639+0000 7f5b77e00700 1 --2- 172.21.15.86:0/187425696 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562d49651910 0x562d496634f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:54:52.643 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.639+0000 7f5b7d000700 1 -- 172.21.15.86:0/187425696 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562d49651910 msgr2=0x562d496634f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:54:52.643 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.639+0000 7f5b7d000700 1 --2- 172.21.15.86:0/187425696 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562d49651910 0x562d496634f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:52.643 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.639+0000 7f5b7d000700 1 -- 172.21.15.86:0/187425696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562d49652230 msgr2=0x562d49663a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:54:52.643 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.639+0000 7f5b7d000700 1 --2- 172.21.15.86:0/187425696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562d49652230 0x562d49663a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:52.643 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.640+0000 7f5b7d000700 1 -- 172.21.15.86:0/187425696 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b781a6410 con 0x562d4965d0e0 2024-03-23T17:54:52.643 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.640+0000 7f5b7d000700 1 --2- 172.21.15.86:0/187425696 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562d4965d0e0 0x562d49661ad0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5b70004b10 tx=0x7f5b70019c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcb9f78e7772ed97 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:54:52.643 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.640+0000 7f5b62000700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T17:54:52.644 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.640+0000 7f5b62000700 1 -- 172.21.15.86:0/187425696 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b7000e040 con 0x562d4965d0e0 2024-03-23T17:54:52.644 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.640+0000 7f5b62000700 1 -- 172.21.15.86:0/187425696 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f5b70006000 con 0x562d4965d0e0 2024-03-23T17:54:52.644 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.640+0000 7f5b92abcb80 1 -- 172.21.15.86:0/187425696 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5b6401d040 con 0x562d4965d0e0 2024-03-23T17:54:52.644 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.640+0000 7f5b62000700 1 -- 172.21.15.86:0/187425696 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b70019e50 con 0x562d4965d0e0 2024-03-23T17:54:52.644 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.640+0000 7f5b92abcb80 10 client.26375 fetch_fsmap learned FSMap version 37 2024-03-23T17:54:52.644 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.640+0000 7f5b92abcb80 1 -- 172.21.15.86:0/187425696 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x562d49421c70 con 0x562d4965d0e0 2024-03-23T17:54:52.644 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.641+0000 7f5b62000700 1 -- 172.21.15.86:0/187425696 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b70043020 con 0x562d4965d0e0 2024-03-23T17:54:52.645 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.641+0000 7f5b62000700 1 -- 172.21.15.86:0/187425696 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 37) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f5b7003d070 con 0x562d4965d0e0 2024-03-23T17:54:52.645 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.641+0000 7f5b92abcb80 10 client.26375 fetch_fsmap finished waiting for FSMap version 37 2024-03-23T17:54:52.645 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.641+0000 7f5b92abcb80 10 client.26375 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:54:52.645 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.641+0000 7f5b92abcb80 20 client.26375 populate_metadata read hostname 'smithi086' 2024-03-23T17:54:52.645 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.641+0000 7f5b92abcb80 1 --2- 172.21.15.86:0/187425696 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x562d496bf9d0 0x562d496c1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:54:52.645 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.641+0000 7f5b92abcb80 4 client.26375 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T17:54:52.645 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.641+0000 7f5b92abcb80 1 -- 172.21.15.86:0/187425696 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562d493eac90 con 0x562d496bf9d0 2024-03-23T17:54:52.645 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.641+0000 7f5b77e00700 1 --2- 172.21.15.86:0/187425696 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x562d496bf9d0 0x562d496c1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:54:52.646 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.642+0000 7f5b77e00700 1 --2- 172.21.15.86:0/187425696 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x562d496bf9d0 0x562d496c1df0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c0e766c02c8e43e server_cookie=b07d69153108e14b in_seq=0 out_seq=0 2024-03-23T17:54:52.646 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.642+0000 7f5b62000700 10 client.26375 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:54:52.668 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.667+0000 7f5b62000700 1 -- 172.21.15.86:0/187425696 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562d493eac90 con 0x562d496bf9d0 2024-03-23T17:54:52.668 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.667+0000 7f5b62000700 10 client.26375 handle_command_reply: tid=0 2024-03-23T17:54:52.668 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.667+0000 7f5b92abcb80 1 client.26375 shutdown 2024-03-23T17:54:52.668 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.668+0000 7f5b92abcb80 1 -- 172.21.15.86:0/187425696 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562d4965d0e0 msgr2=0x562d49661ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:54:52.668 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.668+0000 7f5b92abcb80 1 --2- 172.21.15.86:0/187425696 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562d4965d0e0 0x562d49661ad0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5b70004b10 tx=0x7f5b70019c60 comp rx=0 tx=0).stop 2024-03-23T17:54:52.669 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.668+0000 7f5b92abcb80 1 -- 172.21.15.86:0/187425696 shutdown_connections 2024-03-23T17:54:52.669 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.668+0000 7f5b92abcb80 1 --2- 172.21.15.86:0/187425696 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x562d496bf9d0 0x562d496c1df0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:52.669 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.668+0000 7f5b62000700 0 client.26375 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:54:52.669 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.668+0000 7f5b92abcb80 1 --2- 172.21.15.86:0/187425696 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562d4965d0e0 0x562d49661ad0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:52.669 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.668+0000 7f5b92abcb80 1 --2- 172.21.15.86:0/187425696 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562d49651910 0x562d496634f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:52.669 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.668+0000 7f5b92abcb80 1 --2- 172.21.15.86:0/187425696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562d49652230 0x562d49663a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:52.669 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.668+0000 7f5b92abcb80 1 -- 172.21.15.86:0/187425696 >> 172.21.15.86:0/187425696 conn(0x562d495a9640 msgr2=0x562d49645450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:54:52.669 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.668+0000 7f5b92abcb80 1 -- 172.21.15.86:0/187425696 shutdown_connections 2024-03-23T17:54:52.669 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.668+0000 7f5b92abcb80 1 -- 172.21.15.86:0/187425696 wait complete. 2024-03-23T17:54:52.669 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.668+0000 7f5b92abcb80 20 client.26375 trim_cache size 0 max 16384 2024-03-23T17:54:52.669 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-03-23T17:54:52.669 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-03-23T17:54:52.669 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-03-23T17:54:52.670 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-03-23T17:54:52.670 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.669+0000 7f5b7f000700 1 -- 172.21.15.86:0/1784122949 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f5b5c0a1310 msgr2=0x7f5b5c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:54:52.670 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.669+0000 7f5b7f000700 1 --2- 172.21.15.86:0/1784122949 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f5b5c0a1310 0x7f5b5c0a37d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5b7000f540 tx=0x7f5b70009920 comp rx=0 tx=0).stop 2024-03-23T17:54:52.670 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.669+0000 7f5b7f000700 1 -- 172.21.15.86:0/1784122949 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b78100ef0 msgr2=0x7f5b781993f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:54:52.670 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.669+0000 7f5b7f000700 1 --2- 172.21.15.86:0/1784122949 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b78100ef0 0x7f5b781993f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5b6800e9c0 tx=0x7f5b6800ecd0 comp rx=0 tx=0).stop 2024-03-23T17:54:52.670 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.669+0000 7f5b7f000700 1 -- 172.21.15.86:0/1784122949 shutdown_connections 2024-03-23T17:54:52.671 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.669+0000 7f5b7f000700 1 --2- 172.21.15.86:0/1784122949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b7819dcd0 0x7f5b7819e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:52.671 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.669+0000 7f5b7f000700 1 --2- 172.21.15.86:0/1784122949 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b78100ef0 0x7f5b781993f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:52.671 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.669+0000 7f5b7f000700 1 --2- 172.21.15.86:0/1784122949 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f5b5c0a1310 0x7f5b5c0a37d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:52.671 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.669+0000 7f5b7f000700 1 --2- 172.21.15.86:0/1784122949 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5b78106010 0x7f5b78199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:54:52.671 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.669+0000 7f5b7f000700 1 -- 172.21.15.86:0/1784122949 >> 172.21.15.86:0/1784122949 conn(0x7f5b780fc750 msgr2=0x7f5b780fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:54:52.671 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.669+0000 7f5b7f000700 1 -- 172.21.15.86:0/1784122949 shutdown_connections 2024-03-23T17:54:52.671 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:54:52.669+0000 7f5b7f000700 1 -- 172.21.15.86:0/1784122949 wait complete. 2024-03-23T17:54:52.896 INFO:tasks.workunit.client.0.smithi086.stdout: 1 117880 2.48 MB/sec execute 77 sec latency 48.877 ms 2024-03-23T17:54:53.896 INFO:tasks.workunit.client.0.smithi086.stdout: 1 117880 2.45 MB/sec execute 78 sec latency 1049.013 ms 2024-03-23T17:54:54.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:54:53 smithi092 ceph-mon[79089]: pgmap v130: 97 pgs: 97 active+clean; 55 MiB data, 697 MiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 554 KiB/s wr, 75 op/s 2024-03-23T17:54:54.085 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:54:53 smithi152 ceph-mon[47949]: pgmap v130: 97 pgs: 97 active+clean; 55 MiB data, 697 MiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 554 KiB/s wr, 75 op/s 2024-03-23T17:54:54.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:54:53 smithi086 ceph-mon[38221]: pgmap v130: 97 pgs: 97 active+clean; 55 MiB data, 697 MiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 554 KiB/s wr, 75 op/s 2024-03-23T17:54:54.897 INFO:tasks.workunit.client.0.smithi086.stdout: 1 117880 2.42 MB/sec execute 79 sec latency 2049.158 ms 2024-03-23T17:54:55.897 INFO:tasks.workunit.client.0.smithi086.stdout: 1 117880 2.39 MB/sec execute 80 sec latency 3049.285 ms 2024-03-23T17:54:56.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:54:55 smithi092 ceph-mon[79089]: pgmap v131: 97 pgs: 97 active+clean; 53 MiB data, 749 MiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 834 KiB/s wr, 134 op/s 2024-03-23T17:54:56.085 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:54:55 smithi152 ceph-mon[47949]: pgmap v131: 97 pgs: 97 active+clean; 53 MiB data, 749 MiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 834 KiB/s wr, 134 op/s 2024-03-23T17:54:56.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:54:55 smithi086 ceph-mon[38221]: pgmap v131: 97 pgs: 97 active+clean; 53 MiB data, 749 MiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 834 KiB/s wr, 134 op/s 2024-03-23T17:54:56.897 INFO:tasks.workunit.client.0.smithi086.stdout: 1 120114 2.51 MB/sec execute 81 sec latency 3060.707 ms 2024-03-23T17:54:57.897 INFO:tasks.workunit.client.0.smithi086.stdout: 1 121460 2.57 MB/sec execute 82 sec latency 44.686 ms 2024-03-23T17:54:58.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:54:57 smithi092 ceph-mon[79089]: pgmap v132: 97 pgs: 97 active+clean; 53 MiB data, 749 MiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 318 KiB/s wr, 70 op/s 2024-03-23T17:54:58.085 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:54:57 smithi152 ceph-mon[47949]: pgmap v132: 97 pgs: 97 active+clean; 53 MiB data, 749 MiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 318 KiB/s wr, 70 op/s 2024-03-23T17:54:58.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:54:57 smithi086 ceph-mon[38221]: pgmap v132: 97 pgs: 97 active+clean; 53 MiB data, 749 MiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 318 KiB/s wr, 70 op/s 2024-03-23T17:54:58.897 INFO:tasks.workunit.client.0.smithi086.stdout: 1 123982 2.70 MB/sec execute 83 sec latency 24.021 ms 2024-03-23T17:54:59.897 INFO:tasks.workunit.client.0.smithi086.stdout: 1 124849 2.73 MB/sec execute 84 sec latency 213.385 ms 2024-03-23T17:55:00.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:54:59 smithi092 ceph-mon[79089]: pgmap v133: 97 pgs: 97 active+clean; 53 MiB data, 749 MiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 318 KiB/s wr, 70 op/s 2024-03-23T17:55:00.085 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:54:59 smithi152 ceph-mon[47949]: pgmap v133: 97 pgs: 97 active+clean; 53 MiB data, 749 MiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 318 KiB/s wr, 70 op/s 2024-03-23T17:55:00.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:54:59 smithi086 ceph-mon[38221]: pgmap v133: 97 pgs: 97 active+clean; 53 MiB data, 749 MiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 318 KiB/s wr, 70 op/s 2024-03-23T17:55:00.897 INFO:tasks.workunit.client.0.smithi086.stdout: 1 125214 2.71 MB/sec execute 85 sec latency 759.704 ms 2024-03-23T17:55:01.897 INFO:tasks.workunit.client.0.smithi086.stdout: 1 127525 2.83 MB/sec execute 86 sec latency 26.006 ms 2024-03-23T17:55:02.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:55:01 smithi092 ceph-mon[79089]: pgmap v134: 97 pgs: 97 active+clean; 57 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 987 KiB/s wr, 146 op/s 2024-03-23T17:55:02.085 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:55:01 smithi152 ceph-mon[47949]: pgmap v134: 97 pgs: 97 active+clean; 57 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 987 KiB/s wr, 146 op/s 2024-03-23T17:55:02.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:55:01 smithi086 ceph-mon[38221]: pgmap v134: 97 pgs: 97 active+clean; 57 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 987 KiB/s wr, 146 op/s 2024-03-23T17:55:02.897 INFO:tasks.workunit.client.0.smithi086.stdout: 1 128800 2.87 MB/sec execute 87 sec latency 12.023 ms 2024-03-23T17:55:03.897 INFO:tasks.workunit.client.0.smithi086.stdout: 1 131384 3.02 MB/sec execute 88 sec latency 12.359 ms 2024-03-23T17:55:04.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:55:03 smithi092 ceph-mon[79089]: pgmap v135: 97 pgs: 97 active+clean; 57 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 949 KiB/s wr, 134 op/s 2024-03-23T17:55:04.085 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:55:03 smithi152 ceph-mon[47949]: pgmap v135: 97 pgs: 97 active+clean; 57 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 949 KiB/s wr, 134 op/s 2024-03-23T17:55:04.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:55:03 smithi086 ceph-mon[38221]: pgmap v135: 97 pgs: 97 active+clean; 57 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 949 KiB/s wr, 134 op/s 2024-03-23T17:55:04.897 INFO:tasks.workunit.client.0.smithi086.stdout: 1 132529 3.03 MB/sec execute 89 sec latency 11.847 ms 2024-03-23T17:55:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.828+0000 7fcaaac00700 1 -- 172.21.15.86:0/1237181430 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcaa4073180 msgr2=0x7fcaa4073560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:55:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.828+0000 7fcaaac00700 1 --2- 172.21.15.86:0/1237181430 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcaa4073180 0x7fcaa4073560 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fca98009990 tx=0x7fca98009ca0 comp rx=0 tx=0).stop 2024-03-23T17:55:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.828+0000 7fcaaac00700 1 -- 172.21.15.86:0/1237181430 shutdown_connections 2024-03-23T17:55:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.828+0000 7fcaaac00700 1 --2- 172.21.15.86:0/1237181430 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcaa410d380 0x7fcaa410f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.828+0000 7fcaaac00700 1 --2- 172.21.15.86:0/1237181430 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcaa4073180 0x7fcaa4073560 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.828+0000 7fcaaac00700 1 --2- 172.21.15.86:0/1237181430 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcaa4073aa0 0x7fcaa410ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.828+0000 7fcaaac00700 1 -- 172.21.15.86:0/1237181430 >> 172.21.15.86:0/1237181430 conn(0x7fcaa40fc610 msgr2=0x7fcaa40fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:55:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.828+0000 7fcaaac00700 1 -- 172.21.15.86:0/1237181430 shutdown_connections 2024-03-23T17:55:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.828+0000 7fcaaac00700 1 -- 172.21.15.86:0/1237181430 wait complete. 2024-03-23T17:55:05.831 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.829+0000 7fcaaac00700 1 Processor -- start 2024-03-23T17:55:05.831 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.829+0000 7fcaaac00700 1 -- start start 2024-03-23T17:55:05.831 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.830+0000 7fcaaac00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcaa4073180 0x7fcaa4199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:05.831 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.830+0000 7fcaaac00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcaa4073aa0 0x7fcaa4199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:05.831 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.830+0000 7fcaaac00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcaa410d380 0x7fcaa419dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:05.831 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.830+0000 7fcaaac00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa419e110 con 0x7fcaa410d380 2024-03-23T17:55:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.830+0000 7fcaaac00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa419e280 con 0x7fcaa4073aa0 2024-03-23T17:55:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.830+0000 7fcaaac00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa419e440 con 0x7fcaa4073180 2024-03-23T17:55:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.830+0000 7fcaa9800700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcaa4073180 0x7fcaa4199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.830+0000 7fcaa9800700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcaa4073180 0x7fcaa4199240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:60918/0 (socket says 172.21.15.86:60918) 2024-03-23T17:55:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.830+0000 7fcaa9800700 1 -- 172.21.15.86:0/2637999089 learned_addr learned my addr 172.21.15.86:0/2637999089 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:55:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.830+0000 7fcaa8e00700 1 --2- 172.21.15.86:0/2637999089 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcaa4073aa0 0x7fcaa4199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.830+0000 7fcaaa200700 1 --2- 172.21.15.86:0/2637999089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcaa410d380 0x7fcaa419dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.830+0000 7fcaa9800700 1 -- 172.21.15.86:0/2637999089 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcaa4073aa0 msgr2=0x7fcaa4199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:55:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.830+0000 7fcaa9800700 1 --2- 172.21.15.86:0/2637999089 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcaa4073aa0 0x7fcaa4199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.830+0000 7fcaa9800700 1 -- 172.21.15.86:0/2637999089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcaa410d380 msgr2=0x7fcaa419dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:55:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.830+0000 7fcaa9800700 1 --2- 172.21.15.86:0/2637999089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcaa410d380 0x7fcaa419dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.830+0000 7fcaa9800700 1 -- 172.21.15.86:0/2637999089 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca980095f0 con 0x7fcaa4073180 2024-03-23T17:55:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.831+0000 7fcaa8e00700 1 --2- 172.21.15.86:0/2637999089 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcaa4073aa0 0x7fcaa4199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:55:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.831+0000 7fcaa9800700 1 --2- 172.21.15.86:0/2637999089 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcaa4073180 0x7fcaa4199240 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fca98005910 tx=0x7fca98004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:55:05.833 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.831+0000 7fcaa2000700 1 -- 172.21.15.86:0/2637999089 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9801d070 con 0x7fcaa4073180 2024-03-23T17:55:05.833 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.831+0000 7fcaa2000700 1 -- 172.21.15.86:0/2637999089 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca98022470 con 0x7fcaa4073180 2024-03-23T17:55:05.833 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.831+0000 7fcaaac00700 1 -- 172.21.15.86:0/2637999089 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaa419e6c0 con 0x7fcaa4073180 2024-03-23T17:55:05.833 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.831+0000 7fcaa2000700 1 -- 172.21.15.86:0/2637999089 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca98017650 con 0x7fcaa4073180 2024-03-23T17:55:05.833 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.831+0000 7fcaaac00700 1 -- 172.21.15.86:0/2637999089 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaa41a5f60 con 0x7fcaa4073180 2024-03-23T17:55:05.834 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.834+0000 7fcaa2000700 1 -- 172.21.15.86:0/2637999089 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fca980225e0 con 0x7fcaa4073180 2024-03-23T17:55:05.835 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.835+0000 7fcaa2000700 1 --2- 172.21.15.86:0/2637999089 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fca8c0a1310 0x7fca8c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:05.835 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.836+0000 7fcaa2000700 1 -- 172.21.15.86:0/2637999089 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fca980cdf40 con 0x7fcaa4073180 2024-03-23T17:55:05.836 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.836+0000 7fcaa8e00700 1 --2- 172.21.15.86:0/2637999089 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fca8c0a1310 0x7fca8c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:05.836 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.836+0000 7fcabfdeab80 1 Processor -- start 2024-03-23T17:55:05.836 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.836+0000 7fcabfdeab80 1 -- start start 2024-03-23T17:55:05.838 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.836+0000 7fcabfdeab80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557f70c9d2d0 0x557f70c9d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:05.838 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.836+0000 7fcaa8e00700 1 --2- 172.21.15.86:0/2637999089 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fca8c0a1310 0x7fca8c0a37d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fcaa419a740 tx=0x7fca90009260 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:55:05.838 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.836+0000 7fcabfdeab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f70c9dbf0 0x557f70ca8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:05.838 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.837+0000 7fcabfdeab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f70ca8ab0 0x557f70caaea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:05.838 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.837+0000 7fcabfdeab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f70bf6930 con 0x557f70ca8ab0 2024-03-23T17:55:05.838 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.837+0000 7fcabfdeab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f70bf42f0 con 0x557f70c9dbf0 2024-03-23T17:55:05.838 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.837+0000 7fcabfdeab80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f70befbe0 con 0x557f70c9d2d0 2024-03-23T17:55:05.838 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.837+0000 7fcaa8e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f70c9dbf0 0x557f70ca8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:05.838 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.837+0000 7fcaa9800700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557f70c9d2d0 0x557f70c9d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:05.838 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.837+0000 7fcaa9800700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557f70c9d2d0 0x557f70c9d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:60920/0 (socket says 172.21.15.86:60920) 2024-03-23T17:55:05.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.837+0000 7fcaa8e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f70c9dbf0 0x557f70ca8570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:46666/0 (socket says 172.21.15.86:46666) 2024-03-23T17:55:05.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.837+0000 7fcaa9800700 1 -- 172.21.15.86:0/2825196809 learned_addr learned my addr 172.21.15.86:0/2825196809 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:55:05.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.837+0000 7fcaaa200700 1 --2- 172.21.15.86:0/2825196809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f70ca8ab0 0x557f70caaea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:05.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.837+0000 7fcaa9800700 1 -- 172.21.15.86:0/2825196809 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f70c9dbf0 msgr2=0x557f70ca8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:55:05.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.837+0000 7fcaa9800700 1 --2- 172.21.15.86:0/2825196809 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f70c9dbf0 0x557f70ca8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:05.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.837+0000 7fcaa9800700 1 -- 172.21.15.86:0/2825196809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f70ca8ab0 msgr2=0x557f70caaea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:55:05.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.837+0000 7fcaa9800700 1 --2- 172.21.15.86:0/2825196809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f70ca8ab0 0x557f70caaea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:05.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.837+0000 7fcaa9800700 1 -- 172.21.15.86:0/2825196809 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f70a71350 con 0x557f70c9d2d0 2024-03-23T17:55:05.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.837+0000 7fcaa8e00700 1 --2- 172.21.15.86:0/2825196809 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f70c9dbf0 0x557f70ca8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:55:05.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.837+0000 7fcaaa200700 1 --2- 172.21.15.86:0/2825196809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f70ca8ab0 0x557f70caaea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:55:05.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.837+0000 7fcaa9800700 1 --2- 172.21.15.86:0/2825196809 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557f70c9d2d0 0x557f70c9d6b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fca980a30b0 tx=0x7fca98009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d3180de45428202 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:55:05.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.838+0000 7fca96a00700 1 -- 172.21.15.86:0/2825196809 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9809ab50 con 0x557f70c9d2d0 2024-03-23T17:55:05.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.838+0000 7fca96a00700 1 -- 172.21.15.86:0/2825196809 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca9809b8f0 con 0x557f70c9d2d0 2024-03-23T17:55:05.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.838+0000 7fca96a00700 1 -- 172.21.15.86:0/2825196809 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca980a1320 con 0x557f70c9d2d0 2024-03-23T17:55:05.844 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.838+0000 7fcabfdeab80 1 -- 172.21.15.86:0/2825196809 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557f70c9d2d0 msgr2=0x557f70c9d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:55:05.844 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.838+0000 7fcabfdeab80 1 --2- 172.21.15.86:0/2825196809 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557f70c9d2d0 0x557f70c9d6b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fca980a30b0 tx=0x7fca98009ca0 comp rx=0 tx=0).stop 2024-03-23T17:55:05.844 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.838+0000 7fcabfdeab80 1 -- 172.21.15.86:0/2825196809 shutdown_connections 2024-03-23T17:55:05.844 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.838+0000 7fcabfdeab80 1 --2- 172.21.15.86:0/2825196809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f70ca8ab0 0x557f70caaea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:05.844 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.838+0000 7fcabfdeab80 1 --2- 172.21.15.86:0/2825196809 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557f70c9d2d0 0x557f70c9d6b0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:05.844 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.838+0000 7fcabfdeab80 1 --2- 172.21.15.86:0/2825196809 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f70c9dbf0 0x557f70ca8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:05.844 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.838+0000 7fcabfdeab80 1 -- 172.21.15.86:0/2825196809 >> 172.21.15.86:0/2825196809 conn(0x557f70bfd640 msgr2=0x557f70c99750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:55:05.844 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.839+0000 7fcabfdeab80 1 -- 172.21.15.86:0/2825196809 shutdown_connections 2024-03-23T17:55:05.844 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.839+0000 7fcabfdeab80 1 -- 172.21.15.86:0/2825196809 wait complete. 2024-03-23T17:55:05.844 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.839+0000 7fcabfdeab80 1 -- start start 2024-03-23T17:55:05.844 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.839+0000 7fcabfdeab80 1 Processor -- start 2024-03-23T17:55:05.845 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.840+0000 7fcabfdeab80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557f70c9d2d0 0x557f70ca84f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:05.845 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.840+0000 7fcabfdeab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f70c9dbf0 0x557f70ca5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:05.845 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.840+0000 7fcabfdeab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f70ca8ab0 0x557f70ca5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:05.845 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.840+0000 7fcabfdeab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f70b8bd20 con 0x557f70c9dbf0 2024-03-23T17:55:05.845 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.840+0000 7fcabfdeab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f70b8b9c0 con 0x557f70ca8ab0 2024-03-23T17:55:05.845 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.840+0000 7fcabfdeab80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f70a73640 con 0x557f70c9d2d0 2024-03-23T17:55:05.845 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.840+0000 7fcabfdeab80 10 client.0 _ll_register_callbacks cb 0x557f70b95410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:55:05.845 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.840+0000 7fcaa9800700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557f70c9d2d0 0x557f70ca84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:05.845 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.840+0000 7fcaa8e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f70c9dbf0 0x557f70ca5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:05.845 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.840+0000 7fcaa9800700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557f70c9d2d0 0x557f70ca84f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:60930/0 (socket says 172.21.15.86:60930) 2024-03-23T17:55:05.845 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.840+0000 7fcaa9800700 1 -- 172.21.15.86:0/3950484071 learned_addr learned my addr 172.21.15.86:0/3950484071 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:55:05.845 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.840+0000 7fcaaa200700 1 --2- 172.21.15.86:0/3950484071 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f70ca8ab0 0x557f70ca5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:05.845 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.840+0000 7fcaa9800700 1 -- 172.21.15.86:0/3950484071 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f70ca8ab0 msgr2=0x557f70ca5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:55:05.845 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.840+0000 7fcaa9800700 1 --2- 172.21.15.86:0/3950484071 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f70ca8ab0 0x557f70ca5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.840+0000 7fcaa9800700 1 -- 172.21.15.86:0/3950484071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f70c9dbf0 msgr2=0x557f70ca5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:55:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.840+0000 7fcaa9800700 1 --2- 172.21.15.86:0/3950484071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f70c9dbf0 0x557f70ca5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.840+0000 7fcaa9800700 1 -- 172.21.15.86:0/3950484071 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca90017070 con 0x557f70c9d2d0 2024-03-23T17:55:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.841+0000 7fcaa9800700 1 --2- 172.21.15.86:0/3950484071 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557f70c9d2d0 0x557f70ca84f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fca98005f50 tx=0x7fca98022af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7bc47a8551a67ce server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:55:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.841+0000 7fca8b400700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-03-23T17:55:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.841+0000 7fca8b400700 1 -- 172.21.15.86:0/3950484071 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9809b1f0 con 0x557f70c9d2d0 2024-03-23T17:55:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.841+0000 7fca8b400700 1 -- 172.21.15.86:0/3950484071 <== mon.1 v2:172.21.15.152:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fca9809f020 con 0x557f70c9d2d0 2024-03-23T17:55:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.841+0000 7fcabfdeab80 1 -- 172.21.15.86:0/3950484071 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557f70bb0320 con 0x557f70c9d2d0 2024-03-23T17:55:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.841+0000 7fca8b400700 1 -- 172.21.15.86:0/3950484071 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca980a35e0 con 0x557f70c9d2d0 2024-03-23T17:55:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.841+0000 7fcabfdeab80 10 client.26377 fetch_fsmap learned FSMap version 37 2024-03-23T17:55:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.841+0000 7fcabfdeab80 1 -- 172.21.15.86:0/3950484071 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x7fcaa41a5f60 con 0x557f70c9d2d0 2024-03-23T17:55:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.841+0000 7fca8b400700 1 -- 172.21.15.86:0/3950484071 <== mon.1 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557f70bb0320 con 0x557f70c9d2d0 2024-03-23T17:55:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.841+0000 7fca8b400700 1 -- 172.21.15.86:0/3950484071 <== mon.1 v2:172.21.15.152:3300/0 5 ==== fsmap(e 37) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fca980a1150 con 0x557f70c9d2d0 2024-03-23T17:55:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.841+0000 7fcabfdeab80 10 client.26377 fetch_fsmap finished waiting for FSMap version 37 2024-03-23T17:55:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.841+0000 7fcabfdeab80 10 client.26377 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:55:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.841+0000 7fcabfdeab80 20 client.26377 populate_metadata read hostname 'smithi086' 2024-03-23T17:55:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.841+0000 7fcabfdeab80 1 --2- 172.21.15.86:0/3950484071 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x557f70cb7ea0 0x557f70cba2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.841+0000 7fcabfdeab80 4 client.26377 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:55:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.841+0000 7fcabfdeab80 1 -- 172.21.15.86:0/3950484071 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557f70be86a0 con 0x557f70cb7ea0 2024-03-23T17:55:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.842+0000 7fcaa8e00700 1 --2- 172.21.15.86:0/3950484071 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x557f70cb7ea0 0x557f70cba2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.842+0000 7fcaa8e00700 1 --2- 172.21.15.86:0/3950484071 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x557f70cb7ea0 0x557f70cba2c0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a9e487150d47945 server_cookie=632532a1b1aaf520 in_seq=0 out_seq=0 2024-03-23T17:55:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.842+0000 7fca8b400700 10 client.26377 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:55:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.843+0000 7fca8b400700 1 -- 172.21.15.86:0/3950484071 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x557f70be86a0 con 0x557f70cb7ea0 2024-03-23T17:55:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.843+0000 7fca8b400700 10 client.26377 handle_command_reply: tid=0 2024-03-23T17:55:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.843+0000 7fcabfdeab80 1 client.26377 shutdown 2024-03-23T17:55:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.843+0000 7fcabfdeab80 1 -- 172.21.15.86:0/3950484071 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557f70c9d2d0 msgr2=0x557f70ca84f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:55:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.843+0000 7fcabfdeab80 1 --2- 172.21.15.86:0/3950484071 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557f70c9d2d0 0x557f70ca84f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fca98005f50 tx=0x7fca98022af0 comp rx=0 tx=0).stop 2024-03-23T17:55:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.843+0000 7fcabfdeab80 1 -- 172.21.15.86:0/3950484071 shutdown_connections 2024-03-23T17:55:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.843+0000 7fcabfdeab80 1 --2- 172.21.15.86:0/3950484071 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x557f70cb7ea0 0x557f70cba2c0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.843+0000 7fcabfdeab80 1 --2- 172.21.15.86:0/3950484071 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f70ca8ab0 0x557f70ca5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.843+0000 7fcabfdeab80 1 --2- 172.21.15.86:0/3950484071 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557f70c9d2d0 0x557f70ca84f0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.843+0000 7fca8b400700 0 client.26377 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:55:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.843+0000 7fcabfdeab80 1 --2- 172.21.15.86:0/3950484071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f70c9dbf0 0x557f70ca5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.843+0000 7fcabfdeab80 1 -- 172.21.15.86:0/3950484071 >> 172.21.15.86:0/3950484071 conn(0x557f70bfd640 msgr2=0x557f70c9f250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:55:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.843+0000 7fcabfdeab80 1 -- 172.21.15.86:0/3950484071 shutdown_connections 2024-03-23T17:55:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.843+0000 7fcabfdeab80 1 -- 172.21.15.86:0/3950484071 wait complete. 2024-03-23T17:55:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.844+0000 7fcabfdeab80 20 client.26377 trim_cache size 0 max 16384 2024-03-23T17:55:05.859 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.859+0000 7fcabfdeab80 1 Processor -- start 2024-03-23T17:55:05.859 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.859+0000 7fcabfdeab80 1 -- start start 2024-03-23T17:55:05.859 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.859+0000 7fcabfdeab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f70c9d2d0 0x557f70ca01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:05.859 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.859+0000 7fcabfdeab80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557f70c9dbf0 0x557f70caf4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:05.859 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.859+0000 7fcabfdeab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f70ca8ab0 0x557f70ca5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:05.860 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.859+0000 7fcabfdeab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f70cb4320 con 0x557f70c9d2d0 2024-03-23T17:55:05.860 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.859+0000 7fcabfdeab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f70c955e0 con 0x557f70ca8ab0 2024-03-23T17:55:05.860 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.859+0000 7fcabfdeab80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f70c9ee00 con 0x557f70c9dbf0 2024-03-23T17:55:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.860+0000 7fcaaa200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f70ca8ab0 0x557f70ca5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.860+0000 7fcaaa200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f70ca8ab0 0x557f70ca5000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:46680/0 (socket says 172.21.15.86:46680) 2024-03-23T17:55:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.860+0000 7fcaaa200700 1 -- 172.21.15.86:0/4257693383 learned_addr learned my addr 172.21.15.86:0/4257693383 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:55:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.860+0000 7fcaa9800700 1 --2- 172.21.15.86:0/4257693383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f70c9d2d0 0x557f70ca01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.860+0000 7fcaa8e00700 1 --2- 172.21.15.86:0/4257693383 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557f70c9dbf0 0x557f70caf4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.860+0000 7fcaaa200700 1 -- 172.21.15.86:0/4257693383 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557f70c9dbf0 msgr2=0x557f70caf4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:55:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.860+0000 7fcaaa200700 1 --2- 172.21.15.86:0/4257693383 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557f70c9dbf0 0x557f70caf4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.860+0000 7fcaaa200700 1 -- 172.21.15.86:0/4257693383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f70c9d2d0 msgr2=0x557f70ca01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:55:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.860+0000 7fcaaa200700 1 --2- 172.21.15.86:0/4257693383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f70c9d2d0 0x557f70ca01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.860+0000 7fcaaa200700 1 -- 172.21.15.86:0/4257693383 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f70a1db00 con 0x557f70ca8ab0 2024-03-23T17:55:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.860+0000 7fcaa8e00700 1 --2- 172.21.15.86:0/4257693383 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557f70c9dbf0 0x557f70caf4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:55:05.862 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.860+0000 7fcaaa200700 1 --2- 172.21.15.86:0/4257693383 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f70ca8ab0 0x557f70ca5000 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fca9c009d70 tx=0x7fca9c009100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14c05cc4ef13f6e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:55:05.862 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.860+0000 7fca97400700 1 -- 172.21.15.86:0/4257693383 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9c009890 con 0x557f70ca8ab0 2024-03-23T17:55:05.862 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.861+0000 7fca97400700 1 -- 172.21.15.86:0/4257693383 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca9c00f040 con 0x557f70ca8ab0 2024-03-23T17:55:05.862 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.861+0000 7fca97400700 1 -- 172.21.15.86:0/4257693383 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9c017410 con 0x557f70ca8ab0 2024-03-23T17:55:05.865 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.861+0000 7fcabfdeab80 1 -- 172.21.15.86:0/4257693383 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f70ca8ab0 msgr2=0x557f70ca5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:55:05.865 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.861+0000 7fcabfdeab80 1 --2- 172.21.15.86:0/4257693383 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f70ca8ab0 0x557f70ca5000 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fca9c009d70 tx=0x7fca9c009100 comp rx=0 tx=0).stop 2024-03-23T17:55:05.865 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.861+0000 7fcabfdeab80 1 -- 172.21.15.86:0/4257693383 shutdown_connections 2024-03-23T17:55:05.865 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.861+0000 7fcabfdeab80 1 --2- 172.21.15.86:0/4257693383 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f70ca8ab0 0x557f70ca5000 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:05.865 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.861+0000 7fcabfdeab80 1 --2- 172.21.15.86:0/4257693383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f70c9d2d0 0x557f70ca01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:05.865 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.861+0000 7fcabfdeab80 1 --2- 172.21.15.86:0/4257693383 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557f70c9dbf0 0x557f70caf4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:05.865 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.861+0000 7fcabfdeab80 1 -- 172.21.15.86:0/4257693383 >> 172.21.15.86:0/4257693383 conn(0x557f70bfd640 msgr2=0x557f70c92c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:55:05.866 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.861+0000 7fcabfdeab80 1 -- 172.21.15.86:0/4257693383 shutdown_connections 2024-03-23T17:55:05.866 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.861+0000 7fcabfdeab80 1 -- 172.21.15.86:0/4257693383 wait complete. 2024-03-23T17:55:05.866 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.862+0000 7fcabfdeab80 1 -- start start 2024-03-23T17:55:05.866 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.862+0000 7fcabfdeab80 1 Processor -- start 2024-03-23T17:55:05.866 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.862+0000 7fcabfdeab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f70c9d2d0 0x557f70caf0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:05.866 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.862+0000 7fcabfdeab80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557f70c9dbf0 0x557f70cad5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:05.866 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.862+0000 7fcabfdeab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f70ca8ab0 0x557f70cadb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:05.866 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.862+0000 7fcabfdeab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f70cae060 con 0x557f70ca8ab0 2024-03-23T17:55:05.866 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.862+0000 7fcabfdeab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f70cae1d0 con 0x557f70c9d2d0 2024-03-23T17:55:05.866 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.862+0000 7fcabfdeab80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f70cae390 con 0x557f70c9dbf0 2024-03-23T17:55:05.866 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.862+0000 7fcabfdeab80 10 client.0 _ll_register_callbacks cb 0x557f70b95410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:55:05.866 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.863+0000 7fcaa9800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f70c9d2d0 0x557f70caf0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:05.866 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.863+0000 7fcaa8e00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557f70c9dbf0 0x557f70cad5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:05.867 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.863+0000 7fcaa9800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f70c9d2d0 0x557f70caf0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:46688/0 (socket says 172.21.15.86:46688) 2024-03-23T17:55:05.867 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.863+0000 7fcaa9800700 1 -- 172.21.15.86:0/815379909 learned_addr learned my addr 172.21.15.86:0/815379909 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:55:05.867 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.863+0000 7fcaaa200700 1 --2- 172.21.15.86:0/815379909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f70ca8ab0 0x557f70cadb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:05.867 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.863+0000 7fcaa9800700 1 -- 172.21.15.86:0/815379909 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557f70c9dbf0 msgr2=0x557f70cad5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:55:05.867 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.863+0000 7fcaa9800700 1 --2- 172.21.15.86:0/815379909 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557f70c9dbf0 0x557f70cad5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:05.867 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.863+0000 7fcaa9800700 1 -- 172.21.15.86:0/815379909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f70ca8ab0 msgr2=0x557f70cadb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:55:05.867 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.863+0000 7fcaa9800700 1 --2- 172.21.15.86:0/815379909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f70ca8ab0 0x557f70cadb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:05.867 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.863+0000 7fcaa9800700 1 -- 172.21.15.86:0/815379909 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca90004210 con 0x557f70c9d2d0 2024-03-23T17:55:05.867 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.863+0000 7fcaaa200700 1 --2- 172.21.15.86:0/815379909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f70ca8ab0 0x557f70cadb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:55:05.867 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.863+0000 7fcaa9800700 1 --2- 172.21.15.86:0/815379909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f70c9d2d0 0x557f70caf0f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fca980b3af0 tx=0x7fca98005d40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9a48385d8de6259 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:55:05.867 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.863+0000 7fca8b400700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T17:55:05.867 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.863+0000 7fca8b400700 1 -- 172.21.15.86:0/815379909 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca980a35e0 con 0x557f70c9d2d0 2024-03-23T17:55:05.867 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.863+0000 7fca8b400700 1 -- 172.21.15.86:0/815379909 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fca9809f020 con 0x557f70c9d2d0 2024-03-23T17:55:05.867 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.863+0000 7fcabfdeab80 1 -- 172.21.15.86:0/815379909 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fca9809b8f0 con 0x557f70c9d2d0 2024-03-23T17:55:05.867 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.864+0000 7fca8b400700 1 -- 172.21.15.86:0/815379909 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca980b3b20 con 0x557f70c9d2d0 2024-03-23T17:55:05.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.864+0000 7fcabfdeab80 10 client.26393 fetch_fsmap learned FSMap version 37 2024-03-23T17:55:05.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.864+0000 7fcabfdeab80 1 -- 172.21.15.86:0/815379909 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x7fca9c0099f0 con 0x557f70c9d2d0 2024-03-23T17:55:05.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.864+0000 7fca8b400700 1 -- 172.21.15.86:0/815379909 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca9809b8f0 con 0x557f70c9d2d0 2024-03-23T17:55:05.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.864+0000 7fca8b400700 1 -- 172.21.15.86:0/815379909 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 37) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fca980b91e0 con 0x557f70c9d2d0 2024-03-23T17:55:05.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.864+0000 7fcabfdeab80 10 client.26393 fetch_fsmap finished waiting for FSMap version 37 2024-03-23T17:55:05.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.864+0000 7fcabfdeab80 10 client.26393 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:55:05.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.864+0000 7fcabfdeab80 20 client.26393 populate_metadata read hostname 'smithi086' 2024-03-23T17:55:05.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.864+0000 7fcabfdeab80 1 --2- 172.21.15.86:0/815379909 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x557f70d0aa10 0x557f70d0ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:05.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.864+0000 7fcabfdeab80 4 client.26393 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T17:55:05.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.864+0000 7fcabfdeab80 1 -- 172.21.15.86:0/815379909 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557f70a3ec90 con 0x557f70d0aa10 2024-03-23T17:55:05.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.864+0000 7fcaa8e00700 1 --2- 172.21.15.86:0/815379909 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x557f70d0aa10 0x557f70d0ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:05.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.865+0000 7fca8b400700 10 client.26393 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:55:05.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.865+0000 7fcaa8e00700 1 --2- 172.21.15.86:0/815379909 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x557f70d0aa10 0x557f70d0ce00 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=750029f0e78681d server_cookie=86b4480bb0fd0b8a in_seq=0 out_seq=0 2024-03-23T17:55:05.897 INFO:tasks.workunit.client.0.smithi086.stdout: 1 135042 3.18 MB/sec execute 90 sec latency 25.487 ms 2024-03-23T17:55:05.902 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.901+0000 7fca8b400700 1 -- 172.21.15.86:0/815379909 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557f70a3ec90 con 0x557f70d0aa10 2024-03-23T17:55:05.902 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.901+0000 7fca8b400700 10 client.26393 handle_command_reply: tid=0 2024-03-23T17:55:05.902 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.901+0000 7fcabfdeab80 1 client.26393 shutdown 2024-03-23T17:55:05.902 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.902+0000 7fcabfdeab80 1 -- 172.21.15.86:0/815379909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f70c9d2d0 msgr2=0x557f70caf0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:55:05.902 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.902+0000 7fcabfdeab80 1 --2- 172.21.15.86:0/815379909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f70c9d2d0 0x557f70caf0f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fca980b3af0 tx=0x7fca98005d40 comp rx=0 tx=0).stop 2024-03-23T17:55:05.902 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.902+0000 7fcabfdeab80 1 -- 172.21.15.86:0/815379909 shutdown_connections 2024-03-23T17:55:05.903 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.902+0000 7fcabfdeab80 1 --2- 172.21.15.86:0/815379909 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x557f70d0aa10 0x557f70d0ce00 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:05.903 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.902+0000 7fcabfdeab80 1 --2- 172.21.15.86:0/815379909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f70ca8ab0 0x557f70cadb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:05.903 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.902+0000 7fcabfdeab80 1 --2- 172.21.15.86:0/815379909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f70c9d2d0 0x557f70caf0f0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:05.903 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.902+0000 7fca8b400700 0 client.26393 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:55:05.903 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.902+0000 7fcabfdeab80 1 --2- 172.21.15.86:0/815379909 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557f70c9dbf0 0x557f70cad5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:05.903 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.902+0000 7fcabfdeab80 1 -- 172.21.15.86:0/815379909 >> 172.21.15.86:0/815379909 conn(0x557f70bfd640 msgr2=0x557f70c98eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:55:05.903 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.902+0000 7fcabfdeab80 1 -- 172.21.15.86:0/815379909 shutdown_connections 2024-03-23T17:55:05.903 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.902+0000 7fcabfdeab80 1 -- 172.21.15.86:0/815379909 wait complete. 2024-03-23T17:55:05.903 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.902+0000 7fcabfdeab80 20 client.26393 trim_cache size 0 max 16384 2024-03-23T17:55:05.903 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-03-23T17:55:05.904 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-03-23T17:55:05.904 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-03-23T17:55:05.904 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-03-23T17:55:05.904 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.903+0000 7fcaaac00700 1 -- 172.21.15.86:0/2637999089 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fca8c0a1310 msgr2=0x7fca8c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:55:05.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.903+0000 7fcaaac00700 1 --2- 172.21.15.86:0/2637999089 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fca8c0a1310 0x7fca8c0a37d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fcaa419a740 tx=0x7fca90009260 comp rx=0 tx=0).stop 2024-03-23T17:55:05.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.903+0000 7fcaaac00700 1 -- 172.21.15.86:0/2637999089 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcaa4073180 msgr2=0x7fcaa4199240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:55:05.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.903+0000 7fcaaac00700 1 --2- 172.21.15.86:0/2637999089 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcaa4073180 0x7fcaa4199240 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fca98005910 tx=0x7fca98004730 comp rx=0 tx=0).stop 2024-03-23T17:55:05.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.903+0000 7fcaaac00700 1 -- 172.21.15.86:0/2637999089 shutdown_connections 2024-03-23T17:55:05.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.903+0000 7fcaaac00700 1 --2- 172.21.15.86:0/2637999089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcaa410d380 0x7fcaa419dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:05.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.903+0000 7fcaaac00700 1 --2- 172.21.15.86:0/2637999089 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fca8c0a1310 0x7fca8c0a37d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:05.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.903+0000 7fcaaac00700 1 --2- 172.21.15.86:0/2637999089 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcaa4073180 0x7fcaa4199240 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:05.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.903+0000 7fcaaac00700 1 --2- 172.21.15.86:0/2637999089 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcaa4073aa0 0x7fcaa4199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:05.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.903+0000 7fcaaac00700 1 -- 172.21.15.86:0/2637999089 >> 172.21.15.86:0/2637999089 conn(0x7fcaa40fc610 msgr2=0x7fcaa40fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:55:05.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.903+0000 7fcaaac00700 1 -- 172.21.15.86:0/2637999089 shutdown_connections 2024-03-23T17:55:05.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:05.903+0000 7fcaaac00700 1 -- 172.21.15.86:0/2637999089 wait complete. 2024-03-23T17:55:06.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:55:05 smithi092 ceph-mon[79089]: pgmap v136: 97 pgs: 97 active+clean; 63 MiB data, 814 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 253 op/s 2024-03-23T17:55:06.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:55:05 smithi152 ceph-mon[47949]: pgmap v136: 97 pgs: 97 active+clean; 63 MiB data, 814 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 253 op/s 2024-03-23T17:55:06.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:55:05 smithi086 ceph-mon[38221]: pgmap v136: 97 pgs: 97 active+clean; 63 MiB data, 814 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 253 op/s 2024-03-23T17:55:06.897 INFO:tasks.workunit.client.0.smithi086.stdout: 1 136316 3.20 MB/sec execute 91 sec latency 19.814 ms 2024-03-23T17:55:07.897 INFO:tasks.workunit.client.0.smithi086.stdout: 1 138641 3.32 MB/sec execute 92 sec latency 14.446 ms 2024-03-23T17:55:08.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:55:07 smithi092 ceph-mon[79089]: pgmap v137: 97 pgs: 97 active+clean; 63 MiB data, 814 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 194 op/s 2024-03-23T17:55:08.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:55:07 smithi152 ceph-mon[47949]: pgmap v137: 97 pgs: 97 active+clean; 63 MiB data, 814 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 194 op/s 2024-03-23T17:55:08.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:55:07 smithi086 ceph-mon[38221]: pgmap v137: 97 pgs: 97 active+clean; 63 MiB data, 814 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 194 op/s 2024-03-23T17:55:08.897 INFO:tasks.workunit.client.0.smithi086.stdout: 1 140346 3.34 MB/sec execute 93 sec latency 10.759 ms 2024-03-23T17:55:09.897 INFO:tasks.workunit.client.0.smithi086.stdout: 1 142238 3.45 MB/sec execute 94 sec latency 13.868 ms 2024-03-23T17:55:10.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:55:09 smithi092 ceph-mon[79089]: pgmap v138: 97 pgs: 97 active+clean; 63 MiB data, 814 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 194 op/s 2024-03-23T17:55:10.085 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:55:09 smithi152 ceph-mon[47949]: pgmap v138: 97 pgs: 97 active+clean; 63 MiB data, 814 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 194 op/s 2024-03-23T17:55:10.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:55:09 smithi086 ceph-mon[38221]: pgmap v138: 97 pgs: 97 active+clean; 63 MiB data, 814 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 194 op/s 2024-03-23T17:55:10.897 INFO:tasks.workunit.client.0.smithi086.stdout: 1 144295 3.49 MB/sec execute 95 sec latency 14.112 ms 2024-03-23T17:55:11.898 INFO:tasks.workunit.client.0.smithi086.stdout: 1 145846 3.58 MB/sec execute 96 sec latency 20.286 ms 2024-03-23T17:55:12.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:55:11 smithi092 ceph-mon[79089]: pgmap v139: 97 pgs: 97 active+clean; 65 MiB data, 827 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 323 op/s 2024-03-23T17:55:12.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:55:11 smithi152 ceph-mon[47949]: pgmap v139: 97 pgs: 97 active+clean; 65 MiB data, 827 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 323 op/s 2024-03-23T17:55:12.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:55:11 smithi086 ceph-mon[38221]: pgmap v139: 97 pgs: 97 active+clean; 65 MiB data, 827 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 323 op/s 2024-03-23T17:55:12.898 INFO:tasks.workunit.client.0.smithi086.stdout: 1 147915 3.62 MB/sec execute 97 sec latency 15.726 ms 2024-03-23T17:55:13.898 INFO:tasks.workunit.client.0.smithi086.stdout: 1 149479 3.70 MB/sec execute 98 sec latency 11.778 ms 2024-03-23T17:55:14.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:55:13 smithi092 ceph-mon[79089]: pgmap v140: 97 pgs: 97 active+clean; 65 MiB data, 827 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 247 op/s 2024-03-23T17:55:14.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:55:13 smithi092 ceph-mon[79089]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T17:55:14.085 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:55:13 smithi152 ceph-mon[47949]: pgmap v140: 97 pgs: 97 active+clean; 65 MiB data, 827 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 247 op/s 2024-03-23T17:55:14.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:55:13 smithi152 ceph-mon[47949]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T17:55:14.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:55:13 smithi086 ceph-mon[38221]: pgmap v140: 97 pgs: 97 active+clean; 65 MiB data, 827 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 247 op/s 2024-03-23T17:55:14.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:55:13 smithi086 ceph-mon[38221]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T17:55:14.898 INFO:tasks.workunit.client.0.smithi086.stdout: 1 151779 3.78 MB/sec execute 99 sec latency 9.796 ms 2024-03-23T17:55:15.898 INFO:tasks.workunit.client.0.smithi086.stdout: 1 153153 3.82 MB/sec execute 100 sec latency 12.078 ms 2024-03-23T17:55:16.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:55:15 smithi092 ceph-mon[79089]: pgmap v141: 97 pgs: 97 active+clean; 71 MiB data, 848 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.7 MiB/s wr, 354 op/s 2024-03-23T17:55:16.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:55:15 smithi152 ceph-mon[47949]: pgmap v141: 97 pgs: 97 active+clean; 71 MiB data, 848 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.7 MiB/s wr, 354 op/s 2024-03-23T17:55:16.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:55:15 smithi086 ceph-mon[38221]: pgmap v141: 97 pgs: 97 active+clean; 71 MiB data, 848 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.7 MiB/s wr, 354 op/s 2024-03-23T17:55:16.898 INFO:tasks.workunit.client.0.smithi086.stdout: 1 155558 3.91 MB/sec execute 101 sec latency 10.635 ms 2024-03-23T17:55:17.898 INFO:tasks.workunit.client.0.smithi086.stdout: 1 156902 3.94 MB/sec execute 102 sec latency 15.222 ms 2024-03-23T17:55:18.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:55:17 smithi092 ceph-mon[79089]: pgmap v142: 97 pgs: 97 active+clean; 71 MiB data, 848 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 236 op/s 2024-03-23T17:55:18.085 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:55:17 smithi152 ceph-mon[47949]: pgmap v142: 97 pgs: 97 active+clean; 71 MiB data, 848 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 236 op/s 2024-03-23T17:55:18.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:55:17 smithi086 ceph-mon[38221]: pgmap v142: 97 pgs: 97 active+clean; 71 MiB data, 848 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 236 op/s 2024-03-23T17:55:18.898 INFO:tasks.workunit.client.0.smithi086.stdout: 1 159468 4.04 MB/sec execute 103 sec latency 10.751 ms 2024-03-23T17:55:19.059 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.057+0000 7fdb4dc00700 1 -- 172.21.15.86:0/1962595147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb481007d0 msgr2=0x7fdb48104810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:55:19.059 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.057+0000 7fdb4dc00700 1 --2- 172.21.15.86:0/1962595147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb481007d0 0x7fdb48104810 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fdb30009990 tx=0x7fdb30009ca0 comp rx=0 tx=0).stop 2024-03-23T17:55:19.059 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.057+0000 7fdb4dc00700 1 -- 172.21.15.86:0/1962595147 shutdown_connections 2024-03-23T17:55:19.059 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.057+0000 7fdb4dc00700 1 --2- 172.21.15.86:0/1962595147 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdb48104f40 0x7fdb4810cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:19.059 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.057+0000 7fdb4dc00700 1 --2- 172.21.15.86:0/1962595147 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb480ffe20 0x7fdb48100200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:19.059 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.057+0000 7fdb4dc00700 1 --2- 172.21.15.86:0/1962595147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb481007d0 0x7fdb48104810 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:19.059 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.057+0000 7fdb4dc00700 1 -- 172.21.15.86:0/1962595147 >> 172.21.15.86:0/1962595147 conn(0x7fdb480fb680 msgr2=0x7fdb480fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:55:19.059 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.057+0000 7fdb4dc00700 1 -- 172.21.15.86:0/1962595147 shutdown_connections 2024-03-23T17:55:19.059 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.057+0000 7fdb4dc00700 1 -- 172.21.15.86:0/1962595147 wait complete. 2024-03-23T17:55:19.061 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.058+0000 7fdb4dc00700 1 Processor -- start 2024-03-23T17:55:19.061 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.058+0000 7fdb4dc00700 1 -- start start 2024-03-23T17:55:19.061 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.059+0000 7fdb4dc00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb480ffe20 0x7fdb48195df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:19.061 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.059+0000 7fdb4dc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb48104f40 0x7fdb48196330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.059+0000 7fdb4dc00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdb481968c0 0x7fdb4819ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.059+0000 7fdb4dc00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4819b580 con 0x7fdb48104f40 2024-03-23T17:55:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.059+0000 7fdb4dc00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4819b6f0 con 0x7fdb480ffe20 2024-03-23T17:55:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.059+0000 7fdb4dc00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4819b8b0 con 0x7fdb481968c0 2024-03-23T17:55:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.059+0000 7fdb47400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdb481968c0 0x7fdb4819ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.059+0000 7fdb46a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb480ffe20 0x7fdb48195df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.059+0000 7fdb3fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb48104f40 0x7fdb48196330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.059+0000 7fdb46a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb480ffe20 0x7fdb48195df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:60216/0 (socket says 172.21.15.86:60216) 2024-03-23T17:55:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.059+0000 7fdb47400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdb481968c0 0x7fdb4819ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:50786/0 (socket says 172.21.15.86:50786) 2024-03-23T17:55:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.059+0000 7fdb46a00700 1 -- 172.21.15.86:0/1924895876 learned_addr learned my addr 172.21.15.86:0/1924895876 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:55:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.059+0000 7fdb46a00700 1 -- 172.21.15.86:0/1924895876 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdb481968c0 msgr2=0x7fdb4819ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:55:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.059+0000 7fdb46a00700 1 --2- 172.21.15.86:0/1924895876 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdb481968c0 0x7fdb4819ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.059+0000 7fdb46a00700 1 -- 172.21.15.86:0/1924895876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb48104f40 msgr2=0x7fdb48196330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:55:19.063 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.059+0000 7fdb46a00700 1 --2- 172.21.15.86:0/1924895876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb48104f40 0x7fdb48196330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:19.063 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.059+0000 7fdb46a00700 1 -- 172.21.15.86:0/1924895876 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb300095f0 con 0x7fdb480ffe20 2024-03-23T17:55:19.063 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.060+0000 7fdb46a00700 1 --2- 172.21.15.86:0/1924895876 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb480ffe20 0x7fdb48195df0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fdb3800e9a0 tx=0x7fdb3800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:55:19.063 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.060+0000 7fdb3f400700 1 -- 172.21.15.86:0/1924895876 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb38009790 con 0x7fdb480ffe20 2024-03-23T17:55:19.063 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.060+0000 7fdb4dc00700 1 -- 172.21.15.86:0/1924895876 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb4819bb90 con 0x7fdb480ffe20 2024-03-23T17:55:19.063 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.060+0000 7fdb3f400700 1 -- 172.21.15.86:0/1924895876 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb38010430 con 0x7fdb480ffe20 2024-03-23T17:55:19.063 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.060+0000 7fdb3f400700 1 -- 172.21.15.86:0/1924895876 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb38018680 con 0x7fdb480ffe20 2024-03-23T17:55:19.063 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.060+0000 7fdb4dc00700 1 -- 172.21.15.86:0/1924895876 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb4819c0c0 con 0x7fdb480ffe20 2024-03-23T17:55:19.064 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.063+0000 7fdb3f400700 1 -- 172.21.15.86:0/1924895876 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fdb380105a0 con 0x7fdb480ffe20 2024-03-23T17:55:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.065+0000 7fdb3f400700 1 --2- 172.21.15.86:0/1924895876 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fdb2c0a1290 0x7fdb2c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.065+0000 7fdb3f400700 1 -- 172.21.15.86:0/1924895876 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fdb38014070 con 0x7fdb480ffe20 2024-03-23T17:55:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.065+0000 7fdb3fe00700 1 --2- 172.21.15.86:0/1924895876 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fdb2c0a1290 0x7fdb2c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.065+0000 7fdb617e9b80 1 Processor -- start 2024-03-23T17:55:19.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.065+0000 7fdb617e9b80 1 -- start start 2024-03-23T17:55:19.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.065+0000 7fdb3fe00700 1 --2- 172.21.15.86:0/1924895876 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fdb2c0a1290 0x7fdb2c0a3750 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fdb48069a90 tx=0x7fdb3000b260 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:55:19.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.066+0000 7fdb617e9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2c8105100 0x55c2c81054e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:19.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.066+0000 7fdb617e9b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c2c8105a20 0x55c2c8110390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:19.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.066+0000 7fdb617e9b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c2c81108d0 0x55c2c8112cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:19.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.066+0000 7fdb617e9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2c805e930 con 0x55c2c8105100 2024-03-23T17:55:19.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.066+0000 7fdb617e9b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2c805c2f0 con 0x55c2c81108d0 2024-03-23T17:55:19.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.066+0000 7fdb617e9b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2c8057be0 con 0x55c2c8105a20 2024-03-23T17:55:19.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.066+0000 7fdb46a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2c8105100 0x55c2c81054e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:19.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.066+0000 7fdb3fe00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c2c8105a20 0x55c2c8110390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:19.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.066+0000 7fdb3fe00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c2c8105a20 0x55c2c8110390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:50802/0 (socket says 172.21.15.86:50802) 2024-03-23T17:55:19.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.066+0000 7fdb46a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2c8105100 0x55c2c81054e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34918/0 (socket says 172.21.15.86:34918) 2024-03-23T17:55:19.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.066+0000 7fdb3fe00700 1 -- 172.21.15.86:0/688332999 learned_addr learned my addr 172.21.15.86:0/688332999 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:55:19.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.066+0000 7fdb47400700 1 --2- 172.21.15.86:0/688332999 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c2c81108d0 0x55c2c8112cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:19.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.066+0000 7fdb3fe00700 1 -- 172.21.15.86:0/688332999 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c2c81108d0 msgr2=0x55c2c8112cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:55:19.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.066+0000 7fdb3fe00700 1 --2- 172.21.15.86:0/688332999 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c2c81108d0 0x55c2c8112cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:19.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.066+0000 7fdb3fe00700 1 -- 172.21.15.86:0/688332999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2c8105100 msgr2=0x55c2c81054e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:55:19.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.066+0000 7fdb3fe00700 1 --2- 172.21.15.86:0/688332999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2c8105100 0x55c2c81054e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:19.070 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.066+0000 7fdb3fe00700 1 -- 172.21.15.86:0/688332999 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2c7ed9350 con 0x55c2c8105a20 2024-03-23T17:55:19.070 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.066+0000 7fdb46a00700 1 --2- 172.21.15.86:0/688332999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2c8105100 0x55c2c81054e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:55:19.070 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.067+0000 7fdb3fe00700 1 --2- 172.21.15.86:0/688332999 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c2c8105a20 0x55c2c8110390 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fdb30015000 tx=0x7fdb30020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a932ee067888dc20 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:55:19.070 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.067+0000 7fdb37e00700 1 -- 172.21.15.86:0/688332999 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb30005a00 con 0x55c2c8105a20 2024-03-23T17:55:19.070 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.067+0000 7fdb37e00700 1 -- 172.21.15.86:0/688332999 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb3001e040 con 0x55c2c8105a20 2024-03-23T17:55:19.070 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.067+0000 7fdb37e00700 1 -- 172.21.15.86:0/688332999 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb3000f730 con 0x55c2c8105a20 2024-03-23T17:55:19.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.068+0000 7fdb617e9b80 1 -- 172.21.15.86:0/688332999 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c2c8105a20 msgr2=0x55c2c8110390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:55:19.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.068+0000 7fdb617e9b80 1 --2- 172.21.15.86:0/688332999 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c2c8105a20 0x55c2c8110390 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fdb30015000 tx=0x7fdb30020000 comp rx=0 tx=0).stop 2024-03-23T17:55:19.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.068+0000 7fdb617e9b80 1 -- 172.21.15.86:0/688332999 shutdown_connections 2024-03-23T17:55:19.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.068+0000 7fdb617e9b80 1 --2- 172.21.15.86:0/688332999 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c2c81108d0 0x55c2c8112cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:19.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.068+0000 7fdb617e9b80 1 --2- 172.21.15.86:0/688332999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2c8105100 0x55c2c81054e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:19.078 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.068+0000 7fdb617e9b80 1 --2- 172.21.15.86:0/688332999 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c2c8105a20 0x55c2c8110390 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:19.078 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.068+0000 7fdb617e9b80 1 -- 172.21.15.86:0/688332999 >> 172.21.15.86:0/688332999 conn(0x55c2c8065640 msgr2=0x55c2c8101520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:55:19.078 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.068+0000 7fdb617e9b80 1 -- 172.21.15.86:0/688332999 shutdown_connections 2024-03-23T17:55:19.078 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.068+0000 7fdb617e9b80 1 -- 172.21.15.86:0/688332999 wait complete. 2024-03-23T17:55:19.078 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.069+0000 7fdb617e9b80 1 -- start start 2024-03-23T17:55:19.078 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.069+0000 7fdb617e9b80 1 Processor -- start 2024-03-23T17:55:19.078 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.069+0000 7fdb617e9b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c2c8105100 0x55c2c81102d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:19.078 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.069+0000 7fdb617e9b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c2c8105a20 0x55c2c810d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:19.078 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.069+0000 7fdb617e9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2c81108d0 0x55c2c810dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:19.078 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.069+0000 7fdb617e9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2c7ff3d20 con 0x55c2c81108d0 2024-03-23T17:55:19.078 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.069+0000 7fdb617e9b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2c7ff39c0 con 0x55c2c8105100 2024-03-23T17:55:19.078 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.069+0000 7fdb617e9b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2c7edb640 con 0x55c2c8105a20 2024-03-23T17:55:19.078 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.069+0000 7fdb617e9b80 10 client.0 _ll_register_callbacks cb 0x55c2c7ffd410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:55:19.079 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.070+0000 7fdb46a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c2c8105100 0x55c2c81102d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:19.079 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.070+0000 7fdb46a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c2c8105100 0x55c2c81102d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:60236/0 (socket says 172.21.15.86:60236) 2024-03-23T17:55:19.079 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.070+0000 7fdb46a00700 1 -- 172.21.15.86:0/2620042842 learned_addr learned my addr 172.21.15.86:0/2620042842 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:55:19.079 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.070+0000 7fdb3fe00700 1 --2- 172.21.15.86:0/2620042842 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c2c8105a20 0x55c2c810d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:19.079 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.070+0000 7fdb47400700 1 --2- 172.21.15.86:0/2620042842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2c81108d0 0x55c2c810dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:19.079 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.070+0000 7fdb46a00700 1 -- 172.21.15.86:0/2620042842 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c2c8105a20 msgr2=0x55c2c810d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:55:19.079 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.070+0000 7fdb46a00700 1 --2- 172.21.15.86:0/2620042842 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c2c8105a20 0x55c2c810d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:19.079 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.070+0000 7fdb46a00700 1 -- 172.21.15.86:0/2620042842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2c81108d0 msgr2=0x55c2c810dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:55:19.079 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.070+0000 7fdb46a00700 1 --2- 172.21.15.86:0/2620042842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2c81108d0 0x55c2c810dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:19.079 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.070+0000 7fdb46a00700 1 -- 172.21.15.86:0/2620042842 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb4819c0c0 con 0x55c2c8105100 2024-03-23T17:55:19.079 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.070+0000 7fdb3fe00700 1 --2- 172.21.15.86:0/2620042842 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c2c8105a20 0x55c2c810d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:55:19.079 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.070+0000 7fdb46a00700 1 --2- 172.21.15.86:0/2620042842 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c2c8105100 0x55c2c81102d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fdb38096a20 tx=0x7fdb3809a7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ca12b22bfabcd53 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:55:19.080 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.070+0000 7fdb34c00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T17:55:19.080 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.070+0000 7fdb34c00700 1 -- 172.21.15.86:0/2620042842 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb380996a0 con 0x55c2c8105100 2024-03-23T17:55:19.080 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.070+0000 7fdb34c00700 1 -- 172.21.15.86:0/2620042842 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fdb3809dde0 con 0x55c2c8105100 2024-03-23T17:55:19.080 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.070+0000 7fdb617e9b80 1 -- 172.21.15.86:0/2620042842 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c2c8018320 con 0x55c2c8105100 2024-03-23T17:55:19.080 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.071+0000 7fdb34c00700 1 -- 172.21.15.86:0/2620042842 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb380a2430 con 0x55c2c8105100 2024-03-23T17:55:19.080 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.071+0000 7fdb617e9b80 10 client.26405 fetch_fsmap learned FSMap version 38 2024-03-23T17:55:19.080 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.071+0000 7fdb617e9b80 1 -- 172.21.15.86:0/2620042842 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x7fdb4810d330 con 0x55c2c8105100 2024-03-23T17:55:19.080 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.071+0000 7fdb34c00700 1 -- 172.21.15.86:0/2620042842 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c2c8018320 con 0x55c2c8105100 2024-03-23T17:55:19.080 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.071+0000 7fdb34c00700 1 -- 172.21.15.86:0/2620042842 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 38) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fdb38094bc0 con 0x55c2c8105100 2024-03-23T17:55:19.080 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.071+0000 7fdb617e9b80 10 client.26405 fetch_fsmap finished waiting for FSMap version 38 2024-03-23T17:55:19.080 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.071+0000 7fdb617e9b80 10 client.26405 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:55:19.080 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.071+0000 7fdb617e9b80 20 client.26405 populate_metadata read hostname 'smithi086' 2024-03-23T17:55:19.080 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.071+0000 7fdb617e9b80 1 --2- 172.21.15.86:0/2620042842 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55c2c8120500 0x55c2c81228f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:19.081 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.071+0000 7fdb617e9b80 4 client.26405 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:55:19.081 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.071+0000 7fdb617e9b80 1 -- 172.21.15.86:0/2620042842 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c2c80506a0 con 0x55c2c8120500 2024-03-23T17:55:19.081 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.071+0000 7fdb3fe00700 1 --2- 172.21.15.86:0/2620042842 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55c2c8120500 0x55c2c81228f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:19.081 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.072+0000 7fdb3fe00700 1 --2- 172.21.15.86:0/2620042842 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55c2c8120500 0x55c2c81228f0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2dca1cdba9315c37 server_cookie=97bcaa63e7bcd418 in_seq=0 out_seq=0 2024-03-23T17:55:19.081 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.072+0000 7fdb34c00700 10 client.26405 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:55:19.081 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.072+0000 7fdb34c00700 1 -- 172.21.15.86:0/2620042842 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55c2c80506a0 con 0x55c2c8120500 2024-03-23T17:55:19.081 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.072+0000 7fdb34c00700 10 client.26405 handle_command_reply: tid=0 2024-03-23T17:55:19.081 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.072+0000 7fdb617e9b80 1 client.26405 shutdown 2024-03-23T17:55:19.081 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.073+0000 7fdb617e9b80 1 -- 172.21.15.86:0/2620042842 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c2c8105100 msgr2=0x55c2c81102d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:55:19.081 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.073+0000 7fdb617e9b80 1 --2- 172.21.15.86:0/2620042842 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c2c8105100 0x55c2c81102d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fdb38096a20 tx=0x7fdb3809a7a0 comp rx=0 tx=0).stop 2024-03-23T17:55:19.081 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.073+0000 7fdb617e9b80 1 -- 172.21.15.86:0/2620042842 shutdown_connections 2024-03-23T17:55:19.081 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.073+0000 7fdb617e9b80 1 --2- 172.21.15.86:0/2620042842 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55c2c8120500 0x55c2c81228f0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:19.081 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.073+0000 7fdb617e9b80 1 --2- 172.21.15.86:0/2620042842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2c81108d0 0x55c2c810dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:19.081 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.073+0000 7fdb617e9b80 1 --2- 172.21.15.86:0/2620042842 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c2c8105100 0x55c2c81102d0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:19.081 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.073+0000 7fdb617e9b80 1 --2- 172.21.15.86:0/2620042842 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c2c8105a20 0x55c2c810d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:19.082 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.073+0000 7fdb34c00700 0 client.26405 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:55:19.082 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.073+0000 7fdb617e9b80 1 -- 172.21.15.86:0/2620042842 >> 172.21.15.86:0/2620042842 conn(0x55c2c8065640 msgr2=0x55c2c8107080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:55:19.082 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.073+0000 7fdb617e9b80 1 -- 172.21.15.86:0/2620042842 shutdown_connections 2024-03-23T17:55:19.082 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.073+0000 7fdb617e9b80 1 -- 172.21.15.86:0/2620042842 wait complete. 2024-03-23T17:55:19.082 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.073+0000 7fdb617e9b80 20 client.26405 trim_cache size 0 max 16384 2024-03-23T17:55:19.090 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.088+0000 7fdb617e9b80 1 Processor -- start 2024-03-23T17:55:19.090 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.089+0000 7fdb617e9b80 1 -- start start 2024-03-23T17:55:19.090 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.089+0000 7fdb617e9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2c8105100 0x55c2c8127220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:19.090 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.089+0000 7fdb617e9b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c2c8105a20 0x55c2c8117540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:19.091 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.089+0000 7fdb617e9b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c2c81108d0 0x55c2c810d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:19.091 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.089+0000 7fdb617e9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2c81022b0 con 0x55c2c8105100 2024-03-23T17:55:19.091 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.089+0000 7fdb617e9b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2c81063b0 con 0x55c2c81108d0 2024-03-23T17:55:19.091 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.089+0000 7fdb617e9b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2c81028b0 con 0x55c2c8105a20 2024-03-23T17:55:19.091 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.089+0000 7fdb3fe00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c2c8105a20 0x55c2c8117540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.089+0000 7fdb3fe00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c2c8105a20 0x55c2c8117540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:50818/0 (socket says 172.21.15.86:50818) 2024-03-23T17:55:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.089+0000 7fdb3fe00700 1 -- 172.21.15.86:0/1561084622 learned_addr learned my addr 172.21.15.86:0/1561084622 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:55:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.089+0000 7fdb47400700 1 --2- 172.21.15.86:0/1561084622 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c2c81108d0 0x55c2c810d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.089+0000 7fdb46a00700 1 --2- 172.21.15.86:0/1561084622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2c8105100 0x55c2c8127220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.090+0000 7fdb3fe00700 1 -- 172.21.15.86:0/1561084622 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c2c81108d0 msgr2=0x55c2c810d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:55:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.090+0000 7fdb3fe00700 1 --2- 172.21.15.86:0/1561084622 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c2c81108d0 0x55c2c810d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.090+0000 7fdb3fe00700 1 -- 172.21.15.86:0/1561084622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2c8105100 msgr2=0x55c2c8127220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:55:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.090+0000 7fdb3fe00700 1 --2- 172.21.15.86:0/1561084622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2c8105100 0x55c2c8127220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.090+0000 7fdb3fe00700 1 -- 172.21.15.86:0/1561084622 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2c7e85b00 con 0x55c2c8105a20 2024-03-23T17:55:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.090+0000 7fdb3fe00700 1 --2- 172.21.15.86:0/1561084622 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c2c8105a20 0x55c2c8117540 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fdb30006010 tx=0x7fdb3001a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7abfb4a19515193 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:55:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.090+0000 7fdb3cc00700 1 -- 172.21.15.86:0/1561084622 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb30005a00 con 0x55c2c8105a20 2024-03-23T17:55:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.090+0000 7fdb3cc00700 1 -- 172.21.15.86:0/1561084622 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb3003d070 con 0x55c2c8105a20 2024-03-23T17:55:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.090+0000 7fdb3cc00700 1 -- 172.21.15.86:0/1561084622 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb30004760 con 0x55c2c8105a20 2024-03-23T17:55:19.095 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.091+0000 7fdb617e9b80 1 -- 172.21.15.86:0/1561084622 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c2c8105a20 msgr2=0x55c2c8117540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:55:19.095 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.091+0000 7fdb617e9b80 1 --2- 172.21.15.86:0/1561084622 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c2c8105a20 0x55c2c8117540 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fdb30006010 tx=0x7fdb3001a040 comp rx=0 tx=0).stop 2024-03-23T17:55:19.095 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.091+0000 7fdb617e9b80 1 -- 172.21.15.86:0/1561084622 shutdown_connections 2024-03-23T17:55:19.096 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.091+0000 7fdb617e9b80 1 --2- 172.21.15.86:0/1561084622 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c2c81108d0 0x55c2c810d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:19.096 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.091+0000 7fdb617e9b80 1 --2- 172.21.15.86:0/1561084622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2c8105100 0x55c2c8127220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:19.096 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.091+0000 7fdb617e9b80 1 --2- 172.21.15.86:0/1561084622 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c2c8105a20 0x55c2c8117540 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:19.096 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.091+0000 7fdb617e9b80 1 -- 172.21.15.86:0/1561084622 >> 172.21.15.86:0/1561084622 conn(0x55c2c8065640 msgr2=0x55c2c810cef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:55:19.096 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.091+0000 7fdb617e9b80 1 -- 172.21.15.86:0/1561084622 shutdown_connections 2024-03-23T17:55:19.096 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.091+0000 7fdb617e9b80 1 -- 172.21.15.86:0/1561084622 wait complete. 2024-03-23T17:55:19.096 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.091+0000 7fdb617e9b80 1 -- start start 2024-03-23T17:55:19.096 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.092+0000 7fdb617e9b80 1 Processor -- start 2024-03-23T17:55:19.096 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.092+0000 7fdb617e9b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c2c8105100 0x55c2c8114b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:19.096 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.092+0000 7fdb617e9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2c8105a20 0x55c2c8115080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:19.096 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.092+0000 7fdb617e9b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c2c81108d0 0x55c2c81155c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:19.096 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.092+0000 7fdb617e9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2c8171050 con 0x55c2c8105a20 2024-03-23T17:55:19.096 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.092+0000 7fdb617e9b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2c81711c0 con 0x55c2c8105100 2024-03-23T17:55:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.092+0000 7fdb617e9b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2c8171380 con 0x55c2c81108d0 2024-03-23T17:55:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.092+0000 7fdb617e9b80 10 client.0 _ll_register_callbacks cb 0x55c2c7ffd410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:55:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.092+0000 7fdb46a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c2c8105100 0x55c2c8114b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.092+0000 7fdb3fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2c8105a20 0x55c2c8115080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.092+0000 7fdb46a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c2c8105100 0x55c2c8114b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:60258/0 (socket says 172.21.15.86:60258) 2024-03-23T17:55:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.092+0000 7fdb46a00700 1 -- 172.21.15.86:0/2084818195 learned_addr learned my addr 172.21.15.86:0/2084818195 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:55:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.092+0000 7fdb47400700 1 --2- 172.21.15.86:0/2084818195 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c2c81108d0 0x55c2c81155c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.093+0000 7fdb46a00700 1 -- 172.21.15.86:0/2084818195 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c2c81108d0 msgr2=0x55c2c81155c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:55:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.093+0000 7fdb46a00700 1 --2- 172.21.15.86:0/2084818195 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c2c81108d0 0x55c2c81155c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.093+0000 7fdb46a00700 1 -- 172.21.15.86:0/2084818195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2c8105a20 msgr2=0x55c2c8115080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:55:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.093+0000 7fdb46a00700 1 --2- 172.21.15.86:0/2084818195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2c8105a20 0x55c2c8115080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.093+0000 7fdb46a00700 1 -- 172.21.15.86:0/2084818195 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c2c7ed9350 con 0x55c2c8105100 2024-03-23T17:55:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.093+0000 7fdb46a00700 1 --2- 172.21.15.86:0/2084818195 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c2c8105100 0x55c2c8114b40 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fdb38099cb0 tx=0x7fdb380b8410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=240bb95e6b3f5771 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:55:19.098 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.093+0000 7fdb34c00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T17:55:19.098 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.093+0000 7fdb34c00700 1 -- 172.21.15.86:0/2084818195 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb38098f60 con 0x55c2c8105100 2024-03-23T17:55:19.098 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.093+0000 7fdb34c00700 1 -- 172.21.15.86:0/2084818195 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fdb380b8020 con 0x55c2c8105100 2024-03-23T17:55:19.098 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.093+0000 7fdb617e9b80 1 -- 172.21.15.86:0/2084818195 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdb3001e040 con 0x55c2c8105100 2024-03-23T17:55:19.098 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.093+0000 7fdb34c00700 1 -- 172.21.15.86:0/2084818195 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb3809dde0 con 0x55c2c8105100 2024-03-23T17:55:19.098 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.093+0000 7fdb617e9b80 10 client.26411 fetch_fsmap learned FSMap version 38 2024-03-23T17:55:19.098 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.093+0000 7fdb617e9b80 1 -- 172.21.15.86:0/2084818195 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55c2c801de40 con 0x55c2c8105100 2024-03-23T17:55:19.098 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.094+0000 7fdb34c00700 1 -- 172.21.15.86:0/2084818195 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb3001e040 con 0x55c2c8105100 2024-03-23T17:55:19.098 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.094+0000 7fdb34c00700 1 -- 172.21.15.86:0/2084818195 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 38) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fdb380b2bf0 con 0x55c2c8105100 2024-03-23T17:55:19.098 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.094+0000 7fdb617e9b80 10 client.26411 fetch_fsmap finished waiting for FSMap version 38 2024-03-23T17:55:19.098 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.094+0000 7fdb617e9b80 10 client.26411 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:55:19.098 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.094+0000 7fdb617e9b80 20 client.26411 populate_metadata read hostname 'smithi086' 2024-03-23T17:55:19.098 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.094+0000 7fdb617e9b80 1 --2- 172.21.15.86:0/2084818195 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55c2c8172660 0x55c2c8174a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:19.098 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.094+0000 7fdb617e9b80 4 client.26411 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T17:55:19.099 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.094+0000 7fdb617e9b80 1 -- 172.21.15.86:0/2084818195 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c2c7ea6c90 con 0x55c2c8172660 2024-03-23T17:55:19.099 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.094+0000 7fdb3fe00700 1 --2- 172.21.15.86:0/2084818195 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55c2c8172660 0x55c2c8174a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:19.099 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.094+0000 7fdb3fe00700 1 --2- 172.21.15.86:0/2084818195 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55c2c8172660 0x55c2c8174a50 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe75eb71e5953c1a server_cookie=740db8291135f623 in_seq=0 out_seq=0 2024-03-23T17:55:19.099 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.094+0000 7fdb34c00700 10 client.26411 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:55:19.140 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.138+0000 7fdb34c00700 1 -- 172.21.15.86:0/2084818195 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c2c7ea6c90 con 0x55c2c8172660 2024-03-23T17:55:19.140 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.138+0000 7fdb34c00700 10 client.26411 handle_command_reply: tid=0 2024-03-23T17:55:19.141 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.139+0000 7fdb617e9b80 1 client.26411 shutdown 2024-03-23T17:55:19.141 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.139+0000 7fdb617e9b80 1 -- 172.21.15.86:0/2084818195 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c2c8105100 msgr2=0x55c2c8114b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:55:19.141 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.139+0000 7fdb617e9b80 1 --2- 172.21.15.86:0/2084818195 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c2c8105100 0x55c2c8114b40 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fdb38099cb0 tx=0x7fdb380b8410 comp rx=0 tx=0).stop 2024-03-23T17:55:19.141 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.139+0000 7fdb617e9b80 1 -- 172.21.15.86:0/2084818195 shutdown_connections 2024-03-23T17:55:19.141 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.139+0000 7fdb617e9b80 1 --2- 172.21.15.86:0/2084818195 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55c2c8172660 0x55c2c8174a50 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:19.141 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.139+0000 7fdb617e9b80 1 --2- 172.21.15.86:0/2084818195 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c2c81108d0 0x55c2c81155c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:19.141 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.139+0000 7fdb617e9b80 1 --2- 172.21.15.86:0/2084818195 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c2c8105100 0x55c2c8114b40 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:19.141 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.139+0000 7fdb34c00700 0 client.26411 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:55:19.141 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.139+0000 7fdb617e9b80 1 --2- 172.21.15.86:0/2084818195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2c8105a20 0x55c2c8115080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:19.141 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.139+0000 7fdb617e9b80 1 -- 172.21.15.86:0/2084818195 >> 172.21.15.86:0/2084818195 conn(0x55c2c8065640 msgr2=0x55c2c8107040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:55:19.141 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.139+0000 7fdb617e9b80 1 -- 172.21.15.86:0/2084818195 shutdown_connections 2024-03-23T17:55:19.141 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.139+0000 7fdb617e9b80 1 -- 172.21.15.86:0/2084818195 wait complete. 2024-03-23T17:55:19.141 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.139+0000 7fdb617e9b80 20 client.26411 trim_cache size 0 max 16384 2024-03-23T17:55:19.142 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-03-23T17:55:19.142 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-03-23T17:55:19.142 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-03-23T17:55:19.142 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-03-23T17:55:19.143 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.140+0000 7fdb4dc00700 1 -- 172.21.15.86:0/1924895876 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fdb2c0a1290 msgr2=0x7fdb2c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:55:19.143 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.140+0000 7fdb4dc00700 1 --2- 172.21.15.86:0/1924895876 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fdb2c0a1290 0x7fdb2c0a3750 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fdb48069a90 tx=0x7fdb3000b260 comp rx=0 tx=0).stop 2024-03-23T17:55:19.143 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.140+0000 7fdb4dc00700 1 -- 172.21.15.86:0/1924895876 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb480ffe20 msgr2=0x7fdb48195df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:55:19.143 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.140+0000 7fdb4dc00700 1 --2- 172.21.15.86:0/1924895876 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb480ffe20 0x7fdb48195df0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fdb3800e9a0 tx=0x7fdb3800ecb0 comp rx=0 tx=0).stop 2024-03-23T17:55:19.143 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.140+0000 7fdb4dc00700 1 -- 172.21.15.86:0/1924895876 shutdown_connections 2024-03-23T17:55:19.143 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.140+0000 7fdb4dc00700 1 --2- 172.21.15.86:0/1924895876 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fdb2c0a1290 0x7fdb2c0a3750 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:19.143 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.140+0000 7fdb4dc00700 1 --2- 172.21.15.86:0/1924895876 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdb481968c0 0x7fdb4819ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:19.143 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.140+0000 7fdb4dc00700 1 --2- 172.21.15.86:0/1924895876 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb480ffe20 0x7fdb48195df0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:19.143 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.140+0000 7fdb4dc00700 1 --2- 172.21.15.86:0/1924895876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb48104f40 0x7fdb48196330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:19.143 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.140+0000 7fdb4dc00700 1 -- 172.21.15.86:0/1924895876 >> 172.21.15.86:0/1924895876 conn(0x7fdb480fb680 msgr2=0x7fdb480fc9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:55:19.143 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.140+0000 7fdb4dc00700 1 -- 172.21.15.86:0/1924895876 shutdown_connections 2024-03-23T17:55:19.143 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:19.141+0000 7fdb4dc00700 1 -- 172.21.15.86:0/1924895876 wait complete. 2024-03-23T17:55:19.898 INFO:tasks.workunit.client.0.smithi086.stdout: 1 160619 4.05 MB/sec execute 104 sec latency 36.492 ms 2024-03-23T17:55:20.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:55:19 smithi092 ceph-mon[79089]: pgmap v143: 97 pgs: 97 active+clean; 71 MiB data, 848 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 236 op/s 2024-03-23T17:55:20.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:55:19 smithi152 ceph-mon[47949]: pgmap v143: 97 pgs: 97 active+clean; 71 MiB data, 848 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 236 op/s 2024-03-23T17:55:20.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:55:19 smithi086 ceph-mon[38221]: pgmap v143: 97 pgs: 97 active+clean; 71 MiB data, 848 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 236 op/s 2024-03-23T17:55:20.898 INFO:tasks.workunit.client.0.smithi086.stdout: 1 163247 4.17 MB/sec execute 105 sec latency 10.412 ms 2024-03-23T17:55:21.899 INFO:tasks.workunit.client.0.smithi086.stdout: 1 164309 4.16 MB/sec execute 106 sec latency 11.117 ms 2024-03-23T17:55:22.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:55:21 smithi092 ceph-mon[79089]: pgmap v144: 97 pgs: 97 active+clean; 70 MiB data, 883 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 364 op/s 2024-03-23T17:55:22.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:55:21 smithi152 ceph-mon[47949]: pgmap v144: 97 pgs: 97 active+clean; 70 MiB data, 883 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 364 op/s 2024-03-23T17:55:22.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:55:21 smithi086 ceph-mon[38221]: pgmap v144: 97 pgs: 97 active+clean; 70 MiB data, 883 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 364 op/s 2024-03-23T17:55:22.898 INFO:tasks.workunit.client.0.smithi086.stdout: 1 166825 4.27 MB/sec execute 107 sec latency 16.916 ms 2024-03-23T17:55:23.899 INFO:tasks.workunit.client.0.smithi086.stdout: 1 167990 4.26 MB/sec execute 108 sec latency 11.315 ms 2024-03-23T17:55:24.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:55:23 smithi152 ceph-mon[47949]: pgmap v145: 97 pgs: 97 active+clean; 70 MiB data, 883 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 236 op/s 2024-03-23T17:55:24.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:55:23 smithi086 ceph-mon[38221]: pgmap v145: 97 pgs: 97 active+clean; 70 MiB data, 883 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 236 op/s 2024-03-23T17:55:24.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:55:23 smithi092 ceph-mon[79089]: pgmap v145: 97 pgs: 97 active+clean; 70 MiB data, 883 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 236 op/s 2024-03-23T17:55:24.899 INFO:tasks.workunit.client.0.smithi086.stdout: 1 170502 4.38 MB/sec execute 109 sec latency 13.126 ms 2024-03-23T17:55:25.899 INFO:tasks.workunit.client.0.smithi086.stdout: 1 171779 4.39 MB/sec execute 110 sec latency 10.595 ms 2024-03-23T17:55:26.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:55:25 smithi152 ceph-mon[47949]: pgmap v146: 97 pgs: 97 active+clean; 76 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 339 op/s 2024-03-23T17:55:26.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:55:25 smithi086 ceph-mon[38221]: pgmap v146: 97 pgs: 97 active+clean; 76 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 339 op/s 2024-03-23T17:55:26.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:55:25 smithi092 ceph-mon[79089]: pgmap v146: 97 pgs: 97 active+clean; 76 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 339 op/s 2024-03-23T17:55:26.899 INFO:tasks.workunit.client.0.smithi086.stdout: 1 174074 4.47 MB/sec execute 111 sec latency 12.767 ms 2024-03-23T17:55:27.899 INFO:tasks.workunit.client.0.smithi086.stdout: 1 174812 4.44 MB/sec execute 112 sec latency 124.998 ms 2024-03-23T17:55:28.085 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:55:27 smithi152 ceph-mon[47949]: pgmap v147: 97 pgs: 97 active+clean; 76 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 232 op/s 2024-03-23T17:55:28.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:55:27 smithi086 ceph-mon[38221]: pgmap v147: 97 pgs: 97 active+clean; 76 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 232 op/s 2024-03-23T17:55:28.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:55:27 smithi092 ceph-mon[79089]: pgmap v147: 97 pgs: 97 active+clean; 76 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 232 op/s 2024-03-23T17:55:28.899 INFO:tasks.workunit.client.0.smithi086.stdout: 1 177170 4.52 MB/sec execute 113 sec latency 31.992 ms 2024-03-23T17:55:29.899 INFO:tasks.workunit.client.0.smithi086.stdout: 1 178438 4.54 MB/sec execute 114 sec latency 15.504 ms 2024-03-23T17:55:30.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:55:29 smithi152 ceph-mon[47949]: pgmap v148: 97 pgs: 97 active+clean; 76 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 232 op/s 2024-03-23T17:55:30.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:55:29 smithi086 ceph-mon[38221]: pgmap v148: 97 pgs: 97 active+clean; 76 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 232 op/s 2024-03-23T17:55:30.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:55:29 smithi092 ceph-mon[79089]: pgmap v148: 97 pgs: 97 active+clean; 76 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 232 op/s 2024-03-23T17:55:30.899 INFO:tasks.workunit.client.0.smithi086.stdout: 1 181050 4.64 MB/sec execute 115 sec latency 9.637 ms 2024-03-23T17:55:31.899 INFO:tasks.workunit.client.0.smithi086.stdout: 1 182347 4.66 MB/sec execute 116 sec latency 11.081 ms 2024-03-23T17:55:32.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:55:31 smithi152 ceph-mon[47949]: pgmap v149: 97 pgs: 97 active+clean; 75 MiB data, 987 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.0 MiB/s wr, 450 op/s 2024-03-23T17:55:32.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:55:31 smithi086 ceph-mon[38221]: pgmap v149: 97 pgs: 97 active+clean; 75 MiB data, 987 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.0 MiB/s wr, 450 op/s 2024-03-23T17:55:32.309 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:55:31 smithi092 ceph-mon[79089]: pgmap v149: 97 pgs: 97 active+clean; 75 MiB data, 987 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.0 MiB/s wr, 450 op/s 2024-03-23T17:55:32.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.318+0000 7fa10ae00700 1 -- 172.21.15.86:0/679369893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa10410f4d0 msgr2=0x7fa104107db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:55:32.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.318+0000 7fa10ae00700 1 --2- 172.21.15.86:0/679369893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa10410f4d0 0x7fa104107db0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc009920 tx=0x7fa0fc009c30 comp rx=0 tx=0).stop 2024-03-23T17:55:32.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.318+0000 7fa10ae00700 1 -- 172.21.15.86:0/679369893 shutdown_connections 2024-03-23T17:55:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.318+0000 7fa10ae00700 1 --2- 172.21.15.86:0/679369893 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa10411a590 0x7fa10411c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.318+0000 7fa10ae00700 1 --2- 172.21.15.86:0/679369893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa10410f4d0 0x7fa104107db0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.318+0000 7fa10ae00700 1 --2- 172.21.15.86:0/679369893 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa10410f940 0x7fa1041082f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.318+0000 7fa10ae00700 1 -- 172.21.15.86:0/679369893 >> 172.21.15.86:0/679369893 conn(0x7fa10406bd80 msgr2=0x7fa10406c190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:55:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.318+0000 7fa10ae00700 1 -- 172.21.15.86:0/679369893 shutdown_connections 2024-03-23T17:55:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.318+0000 7fa10ae00700 1 -- 172.21.15.86:0/679369893 wait complete. 2024-03-23T17:55:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.319+0000 7fa10ae00700 1 Processor -- start 2024-03-23T17:55:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.319+0000 7fa10ae00700 1 -- start start 2024-03-23T17:55:32.321 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.320+0000 7fa10ae00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa10410f4d0 0x7fa1041a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:32.322 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.320+0000 7fa10ae00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa10410f940 0x7fa1041a6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:32.322 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.320+0000 7fa10ae00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa10411a590 0x7fa1041aa9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:32.322 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.320+0000 7fa10ae00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1041ab000 con 0x7fa10410f4d0 2024-03-23T17:55:32.322 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.320+0000 7fa10ae00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1041ab170 con 0x7fa10410f940 2024-03-23T17:55:32.322 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.320+0000 7fa10ae00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1041ab330 con 0x7fa10411a590 2024-03-23T17:55:32.322 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.320+0000 7fa103400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa10410f940 0x7fa1041a6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:32.322 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.320+0000 7fa103400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa10410f940 0x7fa1041a6620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:51748/0 (socket says 172.21.15.86:51748) 2024-03-23T17:55:32.322 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.320+0000 7fa103400700 1 -- 172.21.15.86:0/765015967 learned_addr learned my addr 172.21.15.86:0/765015967 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:55:32.322 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.321+0000 7fa108e00700 1 --2- 172.21.15.86:0/765015967 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa10411a590 0x7fa1041aa9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:32.322 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.321+0000 7fa103e00700 1 --2- 172.21.15.86:0/765015967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa10410f4d0 0x7fa1041a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:32.322 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.321+0000 7fa103400700 1 -- 172.21.15.86:0/765015967 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa10411a590 msgr2=0x7fa1041aa9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:55:32.323 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.321+0000 7fa103400700 1 --2- 172.21.15.86:0/765015967 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa10411a590 0x7fa1041aa9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:32.323 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.321+0000 7fa103400700 1 -- 172.21.15.86:0/765015967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa10410f4d0 msgr2=0x7fa1041a60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:55:32.323 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.321+0000 7fa103400700 1 --2- 172.21.15.86:0/765015967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa10410f4d0 0x7fa1041a60e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:32.323 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.321+0000 7fa103400700 1 -- 172.21.15.86:0/765015967 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0ec009c40 con 0x7fa10410f940 2024-03-23T17:55:32.323 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.321+0000 7fa103400700 1 --2- 172.21.15.86:0/765015967 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa10410f940 0x7fa1041a6620 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa0f400e9f0 tx=0x7fa0f400edb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:55:32.323 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.321+0000 7fa100c00700 1 -- 172.21.15.86:0/765015967 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f400cbd0 con 0x7fa10410f940 2024-03-23T17:55:32.323 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.321+0000 7fa100c00700 1 -- 172.21.15.86:0/765015967 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0f4009e50 con 0x7fa10410f940 2024-03-23T17:55:32.323 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.321+0000 7fa100c00700 1 -- 172.21.15.86:0/765015967 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f4018420 con 0x7fa10410f940 2024-03-23T17:55:32.323 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.321+0000 7fa10ae00700 1 -- 172.21.15.86:0/765015967 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0fc009520 con 0x7fa10410f940 2024-03-23T17:55:32.323 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.322+0000 7fa10ae00700 1 -- 172.21.15.86:0/765015967 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1041b2c70 con 0x7fa10410f940 2024-03-23T17:55:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.325+0000 7fa100c00700 1 -- 172.21.15.86:0/765015967 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fa0f4010430 con 0x7fa10410f940 2024-03-23T17:55:32.327 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.327+0000 7fa11e90eb80 1 Processor -- start 2024-03-23T17:55:32.327 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.327+0000 7fa100c00700 1 --2- 172.21.15.86:0/765015967 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fa0e80a1310 0x7fa0e80a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:32.327 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.327+0000 7fa11e90eb80 1 -- start start 2024-03-23T17:55:32.327 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.327+0000 7fa100c00700 1 -- 172.21.15.86:0/765015967 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fa0f4014070 con 0x7fa10410f940 2024-03-23T17:55:32.329 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.327+0000 7fa11e90eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8a0e0c100 0x55a8a0e0c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:32.329 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.327+0000 7fa11e90eb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a8a0e0ca20 0x55a8a0e17390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:32.329 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.327+0000 7fa108e00700 1 --2- 172.21.15.86:0/765015967 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fa0e80a1310 0x7fa0e80a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:32.329 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.327+0000 7fa11e90eb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a8a0e178d0 0x55a8a0e19cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:32.329 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.327+0000 7fa11e90eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8a0d65930 con 0x55a8a0e0c100 2024-03-23T17:55:32.329 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.327+0000 7fa11e90eb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8a0d632f0 con 0x55a8a0e178d0 2024-03-23T17:55:32.329 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.327+0000 7fa11e90eb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8a0d5ebe0 con 0x55a8a0e0ca20 2024-03-23T17:55:32.329 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.328+0000 7fa103400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a8a0e0ca20 0x55a8a0e17390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:32.329 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.328+0000 7fa103400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a8a0e0ca20 0x55a8a0e17390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:45776/0 (socket says 172.21.15.86:45776) 2024-03-23T17:55:32.329 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.328+0000 7fa103400700 1 -- 172.21.15.86:0/106184198 learned_addr learned my addr 172.21.15.86:0/106184198 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:55:32.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.328+0000 7fa108e00700 1 --2- 172.21.15.86:0/106184198 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a8a0e178d0 0x55a8a0e19cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:32.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.328+0000 7fa103e00700 1 --2- 172.21.15.86:0/106184198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8a0e0c100 0x55a8a0e0c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:32.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.328+0000 7fa103400700 1 -- 172.21.15.86:0/106184198 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a8a0e178d0 msgr2=0x55a8a0e19cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:55:32.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.328+0000 7fa103400700 1 --2- 172.21.15.86:0/106184198 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a8a0e178d0 0x55a8a0e19cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:32.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.328+0000 7fa103400700 1 -- 172.21.15.86:0/106184198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8a0e0c100 msgr2=0x55a8a0e0c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:55:32.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.328+0000 7fa103400700 1 --2- 172.21.15.86:0/106184198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8a0e0c100 0x55a8a0e0c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:32.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.328+0000 7fa103400700 1 -- 172.21.15.86:0/106184198 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8a0be0350 con 0x55a8a0e0ca20 2024-03-23T17:55:32.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.328+0000 7fa108e00700 1 --2- 172.21.15.86:0/106184198 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a8a0e178d0 0x55a8a0e19cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:55:32.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.328+0000 7fa108e00700 1 --2- 172.21.15.86:0/765015967 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fa0e80a1310 0x7fa0e80a37d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec009fd0 tx=0x7fa0ec00fdd0 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:55:32.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.329+0000 7fa103400700 1 --2- 172.21.15.86:0/106184198 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a8a0e0ca20 0x55a8a0e17390 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa0f4096960 tx=0x7fa0f4099cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc11f833cf37cd42 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:55:32.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.329+0000 7fa0f9600700 1 -- 172.21.15.86:0/106184198 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f409a5f0 con 0x55a8a0e0ca20 2024-03-23T17:55:32.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.329+0000 7fa0f9600700 1 -- 172.21.15.86:0/106184198 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0f4099ef0 con 0x55a8a0e0ca20 2024-03-23T17:55:32.331 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.329+0000 7fa0f9600700 1 -- 172.21.15.86:0/106184198 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f409a390 con 0x55a8a0e0ca20 2024-03-23T17:55:32.333 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.329+0000 7fa11e90eb80 1 -- 172.21.15.86:0/106184198 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a8a0e0ca20 msgr2=0x55a8a0e17390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:55:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.329+0000 7fa11e90eb80 1 --2- 172.21.15.86:0/106184198 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a8a0e0ca20 0x55a8a0e17390 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa0f4096960 tx=0x7fa0f4099cc0 comp rx=0 tx=0).stop 2024-03-23T17:55:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.330+0000 7fa11e90eb80 1 -- 172.21.15.86:0/106184198 shutdown_connections 2024-03-23T17:55:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.330+0000 7fa11e90eb80 1 --2- 172.21.15.86:0/106184198 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a8a0e178d0 0x55a8a0e19cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.330+0000 7fa11e90eb80 1 --2- 172.21.15.86:0/106184198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8a0e0c100 0x55a8a0e0c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.330+0000 7fa11e90eb80 1 --2- 172.21.15.86:0/106184198 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a8a0e0ca20 0x55a8a0e17390 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.330+0000 7fa11e90eb80 1 -- 172.21.15.86:0/106184198 >> 172.21.15.86:0/106184198 conn(0x55a8a0d6c640 msgr2=0x55a8a0e08520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:55:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.330+0000 7fa11e90eb80 1 -- 172.21.15.86:0/106184198 shutdown_connections 2024-03-23T17:55:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.330+0000 7fa11e90eb80 1 -- 172.21.15.86:0/106184198 wait complete. 2024-03-23T17:55:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.330+0000 7fa11e90eb80 1 -- start start 2024-03-23T17:55:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.330+0000 7fa11e90eb80 1 Processor -- start 2024-03-23T17:55:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.331+0000 7fa11e90eb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a8a0e0c100 0x55a8a0e147c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:32.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.331+0000 7fa11e90eb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a8a0e0ca20 0x55a8a0e14d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:32.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.331+0000 7fa11e90eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8a0e178d0 0x55a8a0e15240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:32.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.331+0000 7fa11e90eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8a0cfad20 con 0x55a8a0e178d0 2024-03-23T17:55:32.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.331+0000 7fa11e90eb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8a0cfa9c0 con 0x55a8a0e0c100 2024-03-23T17:55:32.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.331+0000 7fa11e90eb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8a0be2640 con 0x55a8a0e0ca20 2024-03-23T17:55:32.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.331+0000 7fa11e90eb80 10 client.0 _ll_register_callbacks cb 0x55a8a0d04410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:55:32.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.332+0000 7fa103400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a8a0e0ca20 0x55a8a0e14d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:32.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.332+0000 7fa103400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a8a0e0ca20 0x55a8a0e14d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:45788/0 (socket says 172.21.15.86:45788) 2024-03-23T17:55:32.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.332+0000 7fa103400700 1 -- 172.21.15.86:0/3351546192 learned_addr learned my addr 172.21.15.86:0/3351546192 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:55:32.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.332+0000 7fa108e00700 1 --2- 172.21.15.86:0/3351546192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8a0e178d0 0x55a8a0e15240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:32.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.332+0000 7fa103e00700 1 --2- 172.21.15.86:0/3351546192 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a8a0e0c100 0x55a8a0e147c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:32.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.332+0000 7fa103400700 1 -- 172.21.15.86:0/3351546192 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a8a0e0c100 msgr2=0x55a8a0e147c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:55:32.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.332+0000 7fa103400700 1 --2- 172.21.15.86:0/3351546192 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a8a0e0c100 0x55a8a0e147c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.332+0000 7fa103400700 1 -- 172.21.15.86:0/3351546192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8a0e178d0 msgr2=0x55a8a0e15240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:55:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.332+0000 7fa103400700 1 --2- 172.21.15.86:0/3351546192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8a0e178d0 0x55a8a0e15240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.332+0000 7fa103400700 1 -- 172.21.15.86:0/3351546192 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1041b2c70 con 0x55a8a0e0ca20 2024-03-23T17:55:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.332+0000 7fa108e00700 1 --2- 172.21.15.86:0/3351546192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8a0e178d0 0x55a8a0e15240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:55:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.332+0000 7fa103400700 1 --2- 172.21.15.86:0/3351546192 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a8a0e0ca20 0x55a8a0e14d00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa0f400e9c0 tx=0x7fa0f4099670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81f125a575faa186 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:55:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.332+0000 7fa0f2000700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-03-23T17:55:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.333+0000 7fa0f2000700 1 -- 172.21.15.86:0/3351546192 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f409a5f0 con 0x55a8a0e0ca20 2024-03-23T17:55:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.333+0000 7fa11e90eb80 1 -- 172.21.15.86:0/3351546192 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a8a0d1f320 con 0x55a8a0e0ca20 2024-03-23T17:55:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.333+0000 7fa0f2000700 1 -- 172.21.15.86:0/3351546192 <== mon.1 v2:172.21.15.152:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fa0f409dd20 con 0x55a8a0e0ca20 2024-03-23T17:55:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.333+0000 7fa0f2000700 1 -- 172.21.15.86:0/3351546192 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa0f40a20f0 con 0x55a8a0e0ca20 2024-03-23T17:55:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.333+0000 7fa11e90eb80 10 client.26413 fetch_fsmap learned FSMap version 38 2024-03-23T17:55:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.333+0000 7fa11e90eb80 1 -- 172.21.15.86:0/3351546192 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55a8a0b8cb00 con 0x55a8a0e0ca20 2024-03-23T17:55:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.333+0000 7fa0f2000700 1 -- 172.21.15.86:0/3351546192 <== mon.1 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a8a0d1f320 con 0x55a8a0e0ca20 2024-03-23T17:55:32.337 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.333+0000 7fa0f2000700 1 -- 172.21.15.86:0/3351546192 <== mon.1 v2:172.21.15.152:3300/0 5 ==== fsmap(e 38) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa0f4098ee0 con 0x55a8a0e0ca20 2024-03-23T17:55:32.337 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.333+0000 7fa11e90eb80 10 client.26413 fetch_fsmap finished waiting for FSMap version 38 2024-03-23T17:55:32.337 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.333+0000 7fa11e90eb80 10 client.26413 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:55:32.337 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.333+0000 7fa11e90eb80 20 client.26413 populate_metadata read hostname 'smithi086' 2024-03-23T17:55:32.337 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.333+0000 7fa11e90eb80 1 --2- 172.21.15.86:0/3351546192 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55a8a0e27280 0x55a8a0e296a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:32.337 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.333+0000 7fa11e90eb80 4 client.26413 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:55:32.337 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.333+0000 7fa11e90eb80 1 -- 172.21.15.86:0/3351546192 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a8a0d576a0 con 0x55a8a0e27280 2024-03-23T17:55:32.337 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.334+0000 7fa103e00700 1 --2- 172.21.15.86:0/3351546192 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55a8a0e27280 0x55a8a0e296a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:32.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.334+0000 7fa0f2000700 10 client.26413 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:55:32.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.334+0000 7fa103e00700 1 --2- 172.21.15.86:0/3351546192 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55a8a0e27280 0x55a8a0e296a0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc98338ba662fd3c server_cookie=6627a952531d98bb in_seq=0 out_seq=0 2024-03-23T17:55:32.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.335+0000 7fa0f2000700 1 -- 172.21.15.86:0/3351546192 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55a8a0d576a0 con 0x55a8a0e27280 2024-03-23T17:55:32.339 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.335+0000 7fa0f2000700 10 client.26413 handle_command_reply: tid=0 2024-03-23T17:55:32.339 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.335+0000 7fa11e90eb80 1 client.26413 shutdown 2024-03-23T17:55:32.339 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.335+0000 7fa11e90eb80 1 -- 172.21.15.86:0/3351546192 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a8a0e0ca20 msgr2=0x55a8a0e14d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:55:32.339 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.335+0000 7fa11e90eb80 1 --2- 172.21.15.86:0/3351546192 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a8a0e0ca20 0x55a8a0e14d00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa0f400e9c0 tx=0x7fa0f4099670 comp rx=0 tx=0).stop 2024-03-23T17:55:32.339 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.335+0000 7fa11e90eb80 1 -- 172.21.15.86:0/3351546192 shutdown_connections 2024-03-23T17:55:32.339 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.335+0000 7fa11e90eb80 1 --2- 172.21.15.86:0/3351546192 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55a8a0e27280 0x55a8a0e296a0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:32.339 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.335+0000 7fa11e90eb80 1 --2- 172.21.15.86:0/3351546192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8a0e178d0 0x55a8a0e15240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:32.339 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.335+0000 7fa11e90eb80 1 --2- 172.21.15.86:0/3351546192 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a8a0e0c100 0x55a8a0e147c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:32.339 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.335+0000 7fa0f2000700 0 client.26413 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:55:32.339 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.335+0000 7fa11e90eb80 1 --2- 172.21.15.86:0/3351546192 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a8a0e0ca20 0x55a8a0e14d00 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:32.339 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.335+0000 7fa11e90eb80 1 -- 172.21.15.86:0/3351546192 >> 172.21.15.86:0/3351546192 conn(0x55a8a0d6c640 msgr2=0x55a8a0e0e0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:55:32.339 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.335+0000 7fa11e90eb80 1 -- 172.21.15.86:0/3351546192 shutdown_connections 2024-03-23T17:55:32.339 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.335+0000 7fa11e90eb80 1 -- 172.21.15.86:0/3351546192 wait complete. 2024-03-23T17:55:32.340 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.336+0000 7fa11e90eb80 20 client.26413 trim_cache size 0 max 16384 2024-03-23T17:55:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.353+0000 7fa11e90eb80 1 Processor -- start 2024-03-23T17:55:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.353+0000 7fa11e90eb80 1 -- start start 2024-03-23T17:55:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.353+0000 7fa11e90eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8a0e0c100 0x55a8a0e1e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:32.354 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.353+0000 7fa11e90eb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a8a0e0ca20 0x55a8a0e02500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:32.354 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.353+0000 7fa11e90eb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a8a0e178d0 0x55a8a0e0e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:32.354 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.353+0000 7fa11e90eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8a0e0ec90 con 0x55a8a0e0c100 2024-03-23T17:55:32.354 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.353+0000 7fa11e90eb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8a0e2e380 con 0x55a8a0e0ca20 2024-03-23T17:55:32.354 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.353+0000 7fa11e90eb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8a0e092b0 con 0x55a8a0e178d0 2024-03-23T17:55:32.354 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.353+0000 7fa103400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a8a0e0ca20 0x55a8a0e02500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:32.354 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.353+0000 7fa108e00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a8a0e178d0 0x55a8a0e0e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:32.354 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.354+0000 7fa103400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a8a0e0ca20 0x55a8a0e02500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:51778/0 (socket says 172.21.15.86:51778) 2024-03-23T17:55:32.354 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.354+0000 7fa103400700 1 -- 172.21.15.86:0/1983452107 learned_addr learned my addr 172.21.15.86:0/1983452107 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:55:32.354 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.354+0000 7fa103e00700 1 --2- 172.21.15.86:0/1983452107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8a0e0c100 0x55a8a0e1e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:32.356 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.354+0000 7fa108e00700 1 -- 172.21.15.86:0/1983452107 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a8a0e0ca20 msgr2=0x55a8a0e02500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:55:32.356 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.354+0000 7fa108e00700 1 --2- 172.21.15.86:0/1983452107 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a8a0e0ca20 0x55a8a0e02500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:32.356 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.354+0000 7fa108e00700 1 -- 172.21.15.86:0/1983452107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8a0e0c100 msgr2=0x55a8a0e1e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:55:32.356 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.354+0000 7fa108e00700 1 --2- 172.21.15.86:0/1983452107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8a0e0c100 0x55a8a0e1e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:32.356 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.354+0000 7fa108e00700 1 -- 172.21.15.86:0/1983452107 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8a0d24e40 con 0x55a8a0e178d0 2024-03-23T17:55:32.356 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.354+0000 7fa103400700 1 --2- 172.21.15.86:0/1983452107 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a8a0e0ca20 0x55a8a0e02500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:55:32.356 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.354+0000 7fa108e00700 1 --2- 172.21.15.86:0/1983452107 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a8a0e178d0 0x55a8a0e0e750 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x55a8a0e0db90 tx=0x7fa0ec01f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e841aecabf7d2ef server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:55:32.357 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.355+0000 7fa0fa000700 1 -- 172.21.15.86:0/1983452107 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0ec00e040 con 0x55a8a0e178d0 2024-03-23T17:55:32.357 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.355+0000 7fa0fa000700 1 -- 172.21.15.86:0/1983452107 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0ec00f040 con 0x55a8a0e178d0 2024-03-23T17:55:32.357 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.355+0000 7fa0fa000700 1 -- 172.21.15.86:0/1983452107 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0ec01a8a0 con 0x55a8a0e178d0 2024-03-23T17:55:32.357 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.356+0000 7fa11e90eb80 1 -- 172.21.15.86:0/1983452107 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a8a0e178d0 msgr2=0x55a8a0e0e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:55:32.357 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.356+0000 7fa11e90eb80 1 --2- 172.21.15.86:0/1983452107 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a8a0e178d0 0x55a8a0e0e750 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x55a8a0e0db90 tx=0x7fa0ec01f000 comp rx=0 tx=0).stop 2024-03-23T17:55:32.357 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.356+0000 7fa11e90eb80 1 -- 172.21.15.86:0/1983452107 shutdown_connections 2024-03-23T17:55:32.357 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.356+0000 7fa11e90eb80 1 --2- 172.21.15.86:0/1983452107 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a8a0e178d0 0x55a8a0e0e750 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:32.357 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.356+0000 7fa11e90eb80 1 --2- 172.21.15.86:0/1983452107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8a0e0c100 0x55a8a0e1e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:32.357 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.356+0000 7fa11e90eb80 1 --2- 172.21.15.86:0/1983452107 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a8a0e0ca20 0x55a8a0e02500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:32.357 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.356+0000 7fa11e90eb80 1 -- 172.21.15.86:0/1983452107 >> 172.21.15.86:0/1983452107 conn(0x55a8a0d6c640 msgr2=0x55a8a0e0e040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:55:32.357 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.356+0000 7fa11e90eb80 1 -- 172.21.15.86:0/1983452107 shutdown_connections 2024-03-23T17:55:32.357 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.356+0000 7fa11e90eb80 1 -- 172.21.15.86:0/1983452107 wait complete. 2024-03-23T17:55:32.357 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.356+0000 7fa11e90eb80 1 -- start start 2024-03-23T17:55:32.357 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.356+0000 7fa11e90eb80 1 Processor -- start 2024-03-23T17:55:32.360 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.357+0000 7fa11e90eb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a8a0e0c100 0x55a8a0e1e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:32.360 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.357+0000 7fa11e90eb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a8a0e0ca20 0x55a8a0e1c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:32.360 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.357+0000 7fa11e90eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8a0e178d0 0x55a8a0e1cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:32.360 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.357+0000 7fa11e90eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8a0e1d0e0 con 0x55a8a0e178d0 2024-03-23T17:55:32.360 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.357+0000 7fa11e90eb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8a0e1d250 con 0x55a8a0e0ca20 2024-03-23T17:55:32.360 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.357+0000 7fa11e90eb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8a0e1d410 con 0x55a8a0e0c100 2024-03-23T17:55:32.361 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.357+0000 7fa11e90eb80 10 client.0 _ll_register_callbacks cb 0x7fa0f40091e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:55:32.361 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.357+0000 7fa103400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a8a0e0ca20 0x55a8a0e1c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:32.361 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.357+0000 7fa103e00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a8a0e0c100 0x55a8a0e1e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:32.361 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.357+0000 7fa103400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a8a0e0ca20 0x55a8a0e1c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:51786/0 (socket says 172.21.15.86:51786) 2024-03-23T17:55:32.361 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.357+0000 7fa103400700 1 -- 172.21.15.86:0/1840268665 learned_addr learned my addr 172.21.15.86:0/1840268665 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:55:32.361 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.357+0000 7fa108e00700 1 --2- 172.21.15.86:0/1840268665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8a0e178d0 0x55a8a0e1cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:32.361 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.357+0000 7fa103400700 1 -- 172.21.15.86:0/1840268665 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a8a0e0c100 msgr2=0x55a8a0e1e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:55:32.361 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.357+0000 7fa103400700 1 --2- 172.21.15.86:0/1840268665 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a8a0e0c100 0x55a8a0e1e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:32.361 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.357+0000 7fa103400700 1 -- 172.21.15.86:0/1840268665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8a0e178d0 msgr2=0x55a8a0e1cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:55:32.361 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.357+0000 7fa103400700 1 --2- 172.21.15.86:0/1840268665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8a0e178d0 0x55a8a0e1cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:32.361 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.358+0000 7fa103400700 1 -- 172.21.15.86:0/1840268665 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a8a0b8cb00 con 0x55a8a0e0ca20 2024-03-23T17:55:32.361 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.358+0000 7fa103e00700 1 --2- 172.21.15.86:0/1840268665 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a8a0e0c100 0x55a8a0e1e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:55:32.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.358+0000 7fa103400700 1 --2- 172.21.15.86:0/1840268665 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a8a0e0ca20 0x55a8a0e1c660 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa0f400e9c0 tx=0x7fa0f4097890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a3d2fada5169156 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:55:32.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.358+0000 7fa0f2000700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T17:55:32.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.358+0000 7fa0f2000700 1 -- 172.21.15.86:0/1840268665 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f409a5f0 con 0x55a8a0e0ca20 2024-03-23T17:55:32.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.358+0000 7fa0f2000700 1 -- 172.21.15.86:0/1840268665 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fa0f40a2370 con 0x55a8a0e0ca20 2024-03-23T17:55:32.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.358+0000 7fa11e90eb80 1 -- 172.21.15.86:0/1840268665 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa0f4099ef0 con 0x55a8a0e0ca20 2024-03-23T17:55:32.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.358+0000 7fa0f2000700 1 -- 172.21.15.86:0/1840268665 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa0f4099cc0 con 0x55a8a0e0ca20 2024-03-23T17:55:32.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.358+0000 7fa11e90eb80 10 client.26429 fetch_fsmap learned FSMap version 38 2024-03-23T17:55:32.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.358+0000 7fa11e90eb80 1 -- 172.21.15.86:0/1840268665 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x7fa0ec009030 con 0x55a8a0e0ca20 2024-03-23T17:55:32.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.358+0000 7fa0f2000700 1 -- 172.21.15.86:0/1840268665 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0f4099ef0 con 0x55a8a0e0ca20 2024-03-23T17:55:32.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.359+0000 7fa0f2000700 1 -- 172.21.15.86:0/1840268665 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 38) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa0f4095ce0 con 0x55a8a0e0ca20 2024-03-23T17:55:32.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.359+0000 7fa11e90eb80 10 client.26429 fetch_fsmap finished waiting for FSMap version 38 2024-03-23T17:55:32.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.359+0000 7fa11e90eb80 10 client.26429 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:55:32.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.359+0000 7fa11e90eb80 20 client.26429 populate_metadata read hostname 'smithi086' 2024-03-23T17:55:32.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.359+0000 7fa11e90eb80 1 --2- 172.21.15.86:0/1840268665 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55a8a0e79da0 0x55a8a0e7c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:32.363 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.359+0000 7fa11e90eb80 4 client.26429 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T17:55:32.363 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.359+0000 7fa11e90eb80 1 -- 172.21.15.86:0/1840268665 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a8a0badc90 con 0x55a8a0e79da0 2024-03-23T17:55:32.363 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.359+0000 7fa103e00700 1 --2- 172.21.15.86:0/1840268665 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55a8a0e79da0 0x55a8a0e7c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:32.363 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.359+0000 7fa103e00700 1 --2- 172.21.15.86:0/1840268665 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55a8a0e79da0 0x55a8a0e7c190 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abc97af9c22df605 server_cookie=5fb2446be28d7aa in_seq=0 out_seq=0 2024-03-23T17:55:32.363 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.359+0000 7fa0f2000700 10 client.26429 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:55:32.395 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.394+0000 7fa0f2000700 1 -- 172.21.15.86:0/1840268665 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a8a0badc90 con 0x55a8a0e79da0 2024-03-23T17:55:32.395 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.395+0000 7fa0f2000700 10 client.26429 handle_command_reply: tid=0 2024-03-23T17:55:32.396 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.395+0000 7fa11e90eb80 1 client.26429 shutdown 2024-03-23T17:55:32.396 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.395+0000 7fa11e90eb80 1 -- 172.21.15.86:0/1840268665 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a8a0e0ca20 msgr2=0x55a8a0e1c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:55:32.396 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.395+0000 7fa11e90eb80 1 --2- 172.21.15.86:0/1840268665 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a8a0e0ca20 0x55a8a0e1c660 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa0f400e9c0 tx=0x7fa0f4097890 comp rx=0 tx=0).stop 2024-03-23T17:55:32.396 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.395+0000 7fa11e90eb80 1 -- 172.21.15.86:0/1840268665 shutdown_connections 2024-03-23T17:55:32.396 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.395+0000 7fa11e90eb80 1 --2- 172.21.15.86:0/1840268665 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55a8a0e79da0 0x55a8a0e7c190 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:32.396 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.395+0000 7fa11e90eb80 1 --2- 172.21.15.86:0/1840268665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8a0e178d0 0x55a8a0e1cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:32.396 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.395+0000 7fa11e90eb80 1 --2- 172.21.15.86:0/1840268665 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a8a0e0c100 0x55a8a0e1e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:32.396 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.395+0000 7fa0f2000700 0 client.26429 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:55:32.396 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.395+0000 7fa11e90eb80 1 --2- 172.21.15.86:0/1840268665 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a8a0e0ca20 0x55a8a0e1c660 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:32.396 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.395+0000 7fa11e90eb80 1 -- 172.21.15.86:0/1840268665 >> 172.21.15.86:0/1840268665 conn(0x55a8a0d6c640 msgr2=0x55a8a0e14f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:55:32.396 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.395+0000 7fa11e90eb80 1 -- 172.21.15.86:0/1840268665 shutdown_connections 2024-03-23T17:55:32.396 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.395+0000 7fa11e90eb80 1 -- 172.21.15.86:0/1840268665 wait complete. 2024-03-23T17:55:32.396 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.395+0000 7fa11e90eb80 20 client.26429 trim_cache size 0 max 16384 2024-03-23T17:55:32.397 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-03-23T17:55:32.397 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-03-23T17:55:32.397 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-03-23T17:55:32.397 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-03-23T17:55:32.398 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.396+0000 7fa10ae00700 1 -- 172.21.15.86:0/765015967 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fa0e80a1310 msgr2=0x7fa0e80a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:55:32.398 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.396+0000 7fa10ae00700 1 --2- 172.21.15.86:0/765015967 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fa0e80a1310 0x7fa0e80a37d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec009fd0 tx=0x7fa0ec00fdd0 comp rx=0 tx=0).stop 2024-03-23T17:55:32.398 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.396+0000 7fa10ae00700 1 -- 172.21.15.86:0/765015967 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa10410f940 msgr2=0x7fa1041a6620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:55:32.398 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.396+0000 7fa10ae00700 1 --2- 172.21.15.86:0/765015967 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa10410f940 0x7fa1041a6620 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa0f400e9f0 tx=0x7fa0f400edb0 comp rx=0 tx=0).stop 2024-03-23T17:55:32.398 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.396+0000 7fa10ae00700 1 -- 172.21.15.86:0/765015967 shutdown_connections 2024-03-23T17:55:32.398 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.396+0000 7fa10ae00700 1 --2- 172.21.15.86:0/765015967 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fa0e80a1310 0x7fa0e80a37d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:32.398 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.396+0000 7fa10ae00700 1 --2- 172.21.15.86:0/765015967 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa10411a590 0x7fa1041aa9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:32.398 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.396+0000 7fa10ae00700 1 --2- 172.21.15.86:0/765015967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa10410f4d0 0x7fa1041a60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:32.398 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.396+0000 7fa10ae00700 1 --2- 172.21.15.86:0/765015967 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa10410f940 0x7fa1041a6620 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:32.398 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.396+0000 7fa10ae00700 1 -- 172.21.15.86:0/765015967 >> 172.21.15.86:0/765015967 conn(0x7fa10406bd80 msgr2=0x7fa104118580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:55:32.398 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.396+0000 7fa10ae00700 1 -- 172.21.15.86:0/765015967 shutdown_connections 2024-03-23T17:55:32.399 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:32.396+0000 7fa10ae00700 1 -- 172.21.15.86:0/765015967 wait complete. 2024-03-23T17:55:32.899 INFO:tasks.workunit.client.0.smithi086.stdout: 1 184667 4.73 MB/sec execute 117 sec latency 28.028 ms 2024-03-23T17:55:33.899 INFO:tasks.workunit.client.0.smithi086.stdout: 1 185979 4.74 MB/sec execute 118 sec latency 29.591 ms 2024-03-23T17:55:34.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:55:33 smithi152 ceph-mon[47949]: pgmap v150: 97 pgs: 97 active+clean; 75 MiB data, 987 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.2 MiB/s wr, 322 op/s 2024-03-23T17:55:34.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:55:33 smithi086 ceph-mon[38221]: pgmap v150: 97 pgs: 97 active+clean; 75 MiB data, 987 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.2 MiB/s wr, 322 op/s 2024-03-23T17:55:34.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:55:33 smithi092 ceph-mon[79089]: pgmap v150: 97 pgs: 97 active+clean; 75 MiB data, 987 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.2 MiB/s wr, 322 op/s 2024-03-23T17:55:34.899 INFO:tasks.workunit.client.0.smithi086.stdout: 1 188281 4.81 MB/sec execute 119 sec latency 11.105 ms 2024-03-23T17:55:35.899 INFO:tasks.workunit.client.0.smithi086.stdout: 1 190303 4.83 MB/sec execute 120 sec latency 10.395 ms 2024-03-23T17:55:36.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:55:35 smithi152 ceph-mon[47949]: pgmap v151: 97 pgs: 97 active+clean; 74 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.1 MiB/s wr, 446 op/s 2024-03-23T17:55:36.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:55:35 smithi086 ceph-mon[38221]: pgmap v151: 97 pgs: 97 active+clean; 74 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.1 MiB/s wr, 446 op/s 2024-03-23T17:55:36.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:55:35 smithi092 ceph-mon[79089]: pgmap v151: 97 pgs: 97 active+clean; 74 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.1 MiB/s wr, 446 op/s 2024-03-23T17:55:36.899 INFO:tasks.workunit.client.0.smithi086.stdout: 1 191943 4.90 MB/sec execute 121 sec latency 10.463 ms 2024-03-23T17:55:37.899 INFO:tasks.workunit.client.0.smithi086.stdout: 1 193989 4.91 MB/sec execute 122 sec latency 14.479 ms 2024-03-23T17:55:38.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:55:37 smithi152 ceph-mon[47949]: pgmap v152: 97 pgs: 97 active+clean; 74 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.2 MiB/s wr, 342 op/s 2024-03-23T17:55:38.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:55:37 smithi086 ceph-mon[38221]: pgmap v152: 97 pgs: 97 active+clean; 74 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.2 MiB/s wr, 342 op/s 2024-03-23T17:55:38.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:55:37 smithi092 ceph-mon[79089]: pgmap v152: 97 pgs: 97 active+clean; 74 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.2 MiB/s wr, 342 op/s 2024-03-23T17:55:38.899 INFO:tasks.workunit.client.0.smithi086.stdout: 1 195532 4.97 MB/sec execute 123 sec latency 10.304 ms 2024-03-23T17:55:39.899 INFO:tasks.workunit.client.0.smithi086.stdout: 1 197595 4.99 MB/sec execute 124 sec latency 12.093 ms 2024-03-23T17:55:40.085 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:55:39 smithi152 ceph-mon[47949]: pgmap v153: 97 pgs: 97 active+clean; 74 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.2 MiB/s wr, 342 op/s 2024-03-23T17:55:40.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:55:39 smithi086 ceph-mon[38221]: pgmap v153: 97 pgs: 97 active+clean; 74 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.2 MiB/s wr, 342 op/s 2024-03-23T17:55:40.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:55:39 smithi092 ceph-mon[79089]: pgmap v153: 97 pgs: 97 active+clean; 74 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.2 MiB/s wr, 342 op/s 2024-03-23T17:55:40.900 INFO:tasks.workunit.client.0.smithi086.stdout: 1 199090 5.04 MB/sec execute 125 sec latency 12.869 ms 2024-03-23T17:55:41.900 INFO:tasks.workunit.client.0.smithi086.stdout: 1 201247 5.07 MB/sec execute 126 sec latency 10.766 ms 2024-03-23T17:55:42.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:55:41 smithi152 ceph-mon[47949]: pgmap v154: 97 pgs: 97 active+clean; 81 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.9 MiB/s wr, 439 op/s 2024-03-23T17:55:42.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:55:41 smithi086 ceph-mon[38221]: pgmap v154: 97 pgs: 97 active+clean; 81 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.9 MiB/s wr, 439 op/s 2024-03-23T17:55:42.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:55:41 smithi092 ceph-mon[79089]: pgmap v154: 97 pgs: 97 active+clean; 81 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.9 MiB/s wr, 439 op/s 2024-03-23T17:55:42.900 INFO:tasks.workunit.client.0.smithi086.stdout: 1 202686 5.12 MB/sec execute 127 sec latency 12.537 ms 2024-03-23T17:55:43.900 INFO:tasks.workunit.client.0.smithi086.stdout: 1 204795 5.14 MB/sec execute 128 sec latency 13.621 ms 2024-03-23T17:55:44.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:55:43 smithi086 ceph-mon[38221]: pgmap v155: 97 pgs: 97 active+clean; 81 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 220 op/s 2024-03-23T17:55:44.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:55:43 smithi092 ceph-mon[79089]: pgmap v155: 97 pgs: 97 active+clean; 81 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 220 op/s 2024-03-23T17:55:44.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:55:43 smithi152 ceph-mon[47949]: pgmap v155: 97 pgs: 97 active+clean; 81 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 220 op/s 2024-03-23T17:55:44.900 INFO:tasks.workunit.client.0.smithi086.stdout: 1 206207 5.18 MB/sec execute 129 sec latency 15.905 ms 2024-03-23T17:55:45.553 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.552+0000 7fd98fa00700 1 -- 172.21.15.86:0/1815303417 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd988100f10 msgr2=0x7fd9881012f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:55:45.553 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.552+0000 7fd98fa00700 1 --2- 172.21.15.86:0/1815303417 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd988100f10 0x7fd9881012f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd97c009990 tx=0x7fd97c009ca0 comp rx=0 tx=0).stop 2024-03-23T17:55:45.553 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.552+0000 7fd98fa00700 1 -- 172.21.15.86:0/1815303417 shutdown_connections 2024-03-23T17:55:45.553 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.552+0000 7fd98fa00700 1 --2- 172.21.15.86:0/1815303417 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd988106030 0x7fd98810dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:45.553 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.552+0000 7fd98fa00700 1 --2- 172.21.15.86:0/1815303417 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd988100f10 0x7fd9881012f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:45.553 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.552+0000 7fd98fa00700 1 --2- 172.21.15.86:0/1815303417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9881018c0 0x7fd988105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:45.553 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.552+0000 7fd98fa00700 1 -- 172.21.15.86:0/1815303417 >> 172.21.15.86:0/1815303417 conn(0x7fd9880fc7b0 msgr2=0x7fd9880febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:55:45.553 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.552+0000 7fd98fa00700 1 -- 172.21.15.86:0/1815303417 shutdown_connections 2024-03-23T17:55:45.553 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.552+0000 7fd98fa00700 1 -- 172.21.15.86:0/1815303417 wait complete. 2024-03-23T17:55:45.555 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.553+0000 7fd98fa00700 1 Processor -- start 2024-03-23T17:55:45.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.553+0000 7fd98fa00700 1 -- start start 2024-03-23T17:55:45.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.554+0000 7fd98fa00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd988100f10 0x7fd988197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:45.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.554+0000 7fd98fa00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd988106030 0x7fd9881977b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:45.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.554+0000 7fd98fa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd98819bb50 0x7fd98819bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:45.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.554+0000 7fd98fa00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd98819c5d0 con 0x7fd98819bb50 2024-03-23T17:55:45.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.554+0000 7fd98fa00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd98819c740 con 0x7fd988100f10 2024-03-23T17:55:45.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.554+0000 7fd98fa00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd98819c900 con 0x7fd988106030 2024-03-23T17:55:45.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.554+0000 7fd98d000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd988100f10 0x7fd988197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:45.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.554+0000 7fd98d000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd988100f10 0x7fd988197270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:46594/0 (socket says 172.21.15.86:46594) 2024-03-23T17:55:45.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.554+0000 7fd98d000700 1 -- 172.21.15.86:0/2884838483 learned_addr learned my addr 172.21.15.86:0/2884838483 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:55:45.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.554+0000 7fd98da00700 1 --2- 172.21.15.86:0/2884838483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd98819bb50 0x7fd98819bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:45.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.554+0000 7fd987e00700 1 --2- 172.21.15.86:0/2884838483 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd988106030 0x7fd9881977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:45.557 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.554+0000 7fd98d000700 1 -- 172.21.15.86:0/2884838483 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd988106030 msgr2=0x7fd9881977b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:55:45.557 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.554+0000 7fd98d000700 1 --2- 172.21.15.86:0/2884838483 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd988106030 0x7fd9881977b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:45.557 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.554+0000 7fd98d000700 1 -- 172.21.15.86:0/2884838483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd98819bb50 msgr2=0x7fd98819bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:55:45.557 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.554+0000 7fd98d000700 1 --2- 172.21.15.86:0/2884838483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd98819bb50 0x7fd98819bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:45.557 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.554+0000 7fd98d000700 1 -- 172.21.15.86:0/2884838483 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd980009c40 con 0x7fd988100f10 2024-03-23T17:55:45.557 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.555+0000 7fd98d000700 1 --2- 172.21.15.86:0/2884838483 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd988100f10 0x7fd988197270 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd97c000c00 tx=0x7fd97c005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:55:45.557 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.555+0000 7fd985600700 1 -- 172.21.15.86:0/2884838483 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97c01d070 con 0x7fd988100f10 2024-03-23T17:55:45.557 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.555+0000 7fd985600700 1 -- 172.21.15.86:0/2884838483 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd97c022470 con 0x7fd988100f10 2024-03-23T17:55:45.557 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.555+0000 7fd985600700 1 -- 172.21.15.86:0/2884838483 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97c017650 con 0x7fd988100f10 2024-03-23T17:55:45.557 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.555+0000 7fd98fa00700 1 -- 172.21.15.86:0/2884838483 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd97c0095f0 con 0x7fd988100f10 2024-03-23T17:55:45.557 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.555+0000 7fd98fa00700 1 -- 172.21.15.86:0/2884838483 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9881a40f0 con 0x7fd988100f10 2024-03-23T17:55:45.558 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.558+0000 7fd985600700 1 -- 172.21.15.86:0/2884838483 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fd97c0225e0 con 0x7fd988100f10 2024-03-23T17:55:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.559+0000 7fd985600700 1 --2- 172.21.15.86:0/2884838483 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fd9700a1310 0x7fd9700a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.560+0000 7fd9a35d6b80 1 Processor -- start 2024-03-23T17:55:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.560+0000 7fd985600700 1 -- 172.21.15.86:0/2884838483 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fd97c0cde60 con 0x7fd988100f10 2024-03-23T17:55:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.560+0000 7fd9a35d6b80 1 -- start start 2024-03-23T17:55:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.560+0000 7fd987e00700 1 --2- 172.21.15.86:0/2884838483 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fd9700a1310 0x7fd9700a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:45.562 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.560+0000 7fd9a35d6b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c4e5a3100 0x560c4e5a34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:45.562 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.560+0000 7fd9a35d6b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c4e5a3a20 0x560c4e5b27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:45.562 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.560+0000 7fd9a35d6b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c4e5b2ce0 0x560c4e5b50d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:45.562 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.560+0000 7fd9a35d6b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c4e4fc930 con 0x560c4e5b2ce0 2024-03-23T17:55:45.563 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.560+0000 7fd9a35d6b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c4e4fa2f0 con 0x560c4e5a3100 2024-03-23T17:55:45.563 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.560+0000 7fd9a35d6b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c4e4f5be0 con 0x560c4e5a3a20 2024-03-23T17:55:45.563 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.561+0000 7fd98d000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c4e5a3100 0x560c4e5a34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:45.563 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.561+0000 7fd987e00700 1 --2- 172.21.15.86:0/2884838483 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fd9700a1310 0x7fd9700a37d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd9881987c0 tx=0x7fd974009260 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:55:45.563 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.561+0000 7fd98d000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c4e5a3100 0x560c4e5a34e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:46608/0 (socket says 172.21.15.86:46608) 2024-03-23T17:55:45.563 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.561+0000 7fd98d000700 1 -- 172.21.15.86:0/4067364773 learned_addr learned my addr 172.21.15.86:0/4067364773 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:55:45.563 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.561+0000 7fd98da00700 1 --2- 172.21.15.86:0/4067364773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c4e5b2ce0 0x560c4e5b50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:45.563 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.561+0000 7fd987e00700 1 --2- 172.21.15.86:0/4067364773 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c4e5a3a20 0x560c4e5b27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:45.563 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.561+0000 7fd98d000700 1 -- 172.21.15.86:0/4067364773 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c4e5a3a20 msgr2=0x560c4e5b27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:55:45.563 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.561+0000 7fd98d000700 1 --2- 172.21.15.86:0/4067364773 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c4e5a3a20 0x560c4e5b27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:45.563 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.561+0000 7fd98d000700 1 -- 172.21.15.86:0/4067364773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c4e5b2ce0 msgr2=0x560c4e5b50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:55:45.564 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.561+0000 7fd98d000700 1 --2- 172.21.15.86:0/4067364773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c4e5b2ce0 0x560c4e5b50d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:45.564 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.561+0000 7fd98d000700 1 -- 172.21.15.86:0/4067364773 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c4e377350 con 0x560c4e5a3100 2024-03-23T17:55:45.564 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.561+0000 7fd98d000700 1 --2- 172.21.15.86:0/4067364773 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c4e5a3100 0x560c4e5a34e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fd97c097a60 tx=0x7fd97c09adc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f287ae618984eb server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:55:45.564 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.561+0000 7fd97a000700 1 -- 172.21.15.86:0/4067364773 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97c09b6f0 con 0x560c4e5a3100 2024-03-23T17:55:45.564 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.561+0000 7fd97a000700 1 -- 172.21.15.86:0/4067364773 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd97c09aff0 con 0x560c4e5a3100 2024-03-23T17:55:45.564 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.561+0000 7fd97a000700 1 -- 172.21.15.86:0/4067364773 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97c09b250 con 0x560c4e5a3100 2024-03-23T17:55:45.564 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.562+0000 7fd9a35d6b80 1 -- 172.21.15.86:0/4067364773 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c4e5a3100 msgr2=0x560c4e5a34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:55:45.564 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.562+0000 7fd9a35d6b80 1 --2- 172.21.15.86:0/4067364773 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c4e5a3100 0x560c4e5a34e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fd97c097a60 tx=0x7fd97c09adc0 comp rx=0 tx=0).stop 2024-03-23T17:55:45.564 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.562+0000 7fd9a35d6b80 1 -- 172.21.15.86:0/4067364773 shutdown_connections 2024-03-23T17:55:45.564 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.562+0000 7fd9a35d6b80 1 --2- 172.21.15.86:0/4067364773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c4e5b2ce0 0x560c4e5b50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:45.564 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.562+0000 7fd9a35d6b80 1 --2- 172.21.15.86:0/4067364773 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c4e5a3100 0x560c4e5a34e0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:45.564 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.562+0000 7fd9a35d6b80 1 --2- 172.21.15.86:0/4067364773 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c4e5a3a20 0x560c4e5b27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:45.564 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.562+0000 7fd9a35d6b80 1 -- 172.21.15.86:0/4067364773 >> 172.21.15.86:0/4067364773 conn(0x560c4e503640 msgr2=0x560c4e59f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:55:45.564 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.562+0000 7fd9a35d6b80 1 -- 172.21.15.86:0/4067364773 shutdown_connections 2024-03-23T17:55:45.565 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.562+0000 7fd9a35d6b80 1 -- 172.21.15.86:0/4067364773 wait complete. 2024-03-23T17:55:45.569 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.563+0000 7fd9a35d6b80 1 -- start start 2024-03-23T17:55:45.569 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.563+0000 7fd9a35d6b80 1 Processor -- start 2024-03-23T17:55:45.569 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.563+0000 7fd9a35d6b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c4e5a3100 0x560c4e5b2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:45.569 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.563+0000 7fd9a35d6b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c4e5a3a20 0x560c4e5af9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:45.569 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.563+0000 7fd9a35d6b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c4e5b2ce0 0x560c4e5b0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:45.569 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.563+0000 7fd9a35d6b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c4e491d20 con 0x560c4e5a3100 2024-03-23T17:55:45.569 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.563+0000 7fd9a35d6b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c4e4919c0 con 0x560c4e5a3a20 2024-03-23T17:55:45.569 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.563+0000 7fd9a35d6b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c4e379640 con 0x560c4e5b2ce0 2024-03-23T17:55:45.570 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.564+0000 7fd9a35d6b80 10 client.0 _ll_register_callbacks cb 0x560c4e49b410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:55:45.570 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.564+0000 7fd987e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c4e5a3a20 0x560c4e5af9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:45.570 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.564+0000 7fd987e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c4e5a3a20 0x560c4e5af9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:46612/0 (socket says 172.21.15.86:46612) 2024-03-23T17:55:45.570 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.564+0000 7fd987e00700 1 -- 172.21.15.86:0/3169116017 learned_addr learned my addr 172.21.15.86:0/3169116017 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:55:45.570 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.564+0000 7fd98d000700 1 --2- 172.21.15.86:0/3169116017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c4e5a3100 0x560c4e5b2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:45.570 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.564+0000 7fd98da00700 1 --2- 172.21.15.86:0/3169116017 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c4e5b2ce0 0x560c4e5b0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:45.570 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.564+0000 7fd987e00700 1 -- 172.21.15.86:0/3169116017 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c4e5b2ce0 msgr2=0x560c4e5b0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:55:45.570 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.564+0000 7fd987e00700 1 --2- 172.21.15.86:0/3169116017 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c4e5b2ce0 0x560c4e5b0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:45.570 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.564+0000 7fd987e00700 1 -- 172.21.15.86:0/3169116017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c4e5a3100 msgr2=0x560c4e5b2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:55:45.570 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.564+0000 7fd987e00700 1 --2- 172.21.15.86:0/3169116017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c4e5a3100 0x560c4e5b2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:45.570 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.564+0000 7fd987e00700 1 -- 172.21.15.86:0/3169116017 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9881a40f0 con 0x560c4e5a3a20 2024-03-23T17:55:45.570 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.564+0000 7fd987e00700 1 --2- 172.21.15.86:0/3169116017 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c4e5a3a20 0x560c4e5af9f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd97401e7a0 tx=0x7fd9740047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b9f48275df96cae server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:55:45.571 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.564+0000 7fd96ea00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T17:55:45.571 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.565+0000 7fd96ea00700 1 -- 172.21.15.86:0/3169116017 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97401c640 con 0x560c4e5a3a20 2024-03-23T17:55:45.571 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.565+0000 7fd96ea00700 1 -- 172.21.15.86:0/3169116017 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fd97401c7a0 con 0x560c4e5a3a20 2024-03-23T17:55:45.571 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.565+0000 7fd9a35d6b80 1 -- 172.21.15.86:0/3169116017 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560c4e4b6320 con 0x560c4e5a3a20 2024-03-23T17:55:45.571 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.565+0000 7fd96ea00700 1 -- 172.21.15.86:0/3169116017 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd97401c960 con 0x560c4e5a3a20 2024-03-23T17:55:45.571 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.565+0000 7fd9a35d6b80 10 client.26441 fetch_fsmap learned FSMap version 38 2024-03-23T17:55:45.571 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.565+0000 7fd9a35d6b80 1 -- 172.21.15.86:0/3169116017 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x560c4e323b00 con 0x560c4e5a3a20 2024-03-23T17:55:45.571 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.565+0000 7fd96ea00700 1 -- 172.21.15.86:0/3169116017 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560c4e4b6320 con 0x560c4e5a3a20 2024-03-23T17:55:45.571 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.566+0000 7fd96ea00700 1 -- 172.21.15.86:0/3169116017 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 38) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd9740050b0 con 0x560c4e5a3a20 2024-03-23T17:55:45.571 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.566+0000 7fd9a35d6b80 10 client.26441 fetch_fsmap finished waiting for FSMap version 38 2024-03-23T17:55:45.571 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.566+0000 7fd9a35d6b80 10 client.26441 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:55:45.571 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.566+0000 7fd9a35d6b80 20 client.26441 populate_metadata read hostname 'smithi086' 2024-03-23T17:55:45.571 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.566+0000 7fd9a35d6b80 1 --2- 172.21.15.86:0/3169116017 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x560c4e5c2660 0x560c4e5c4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:45.571 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.566+0000 7fd9a35d6b80 4 client.26441 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:55:45.572 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.566+0000 7fd9a35d6b80 1 -- 172.21.15.86:0/3169116017 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560c4e4ee6a0 con 0x560c4e5c2660 2024-03-23T17:55:45.572 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.566+0000 7fd98d000700 1 --2- 172.21.15.86:0/3169116017 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x560c4e5c2660 0x560c4e5c4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:45.572 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.566+0000 7fd96ea00700 10 client.26441 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:55:45.572 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.566+0000 7fd98d000700 1 --2- 172.21.15.86:0/3169116017 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x560c4e5c2660 0x560c4e5c4a80 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c950b3f87d6f8e52 server_cookie=21876484f993817d in_seq=0 out_seq=0 2024-03-23T17:55:45.572 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.567+0000 7fd96ea00700 1 -- 172.21.15.86:0/3169116017 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x560c4e4ee6a0 con 0x560c4e5c2660 2024-03-23T17:55:45.572 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.567+0000 7fd96ea00700 10 client.26441 handle_command_reply: tid=0 2024-03-23T17:55:45.572 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.567+0000 7fd9a35d6b80 1 client.26441 shutdown 2024-03-23T17:55:45.572 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.567+0000 7fd9a35d6b80 1 -- 172.21.15.86:0/3169116017 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c4e5a3a20 msgr2=0x560c4e5af9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:55:45.572 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.567+0000 7fd9a35d6b80 1 --2- 172.21.15.86:0/3169116017 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c4e5a3a20 0x560c4e5af9f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd97401e7a0 tx=0x7fd9740047c0 comp rx=0 tx=0).stop 2024-03-23T17:55:45.572 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.567+0000 7fd9a35d6b80 1 -- 172.21.15.86:0/3169116017 shutdown_connections 2024-03-23T17:55:45.572 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.567+0000 7fd9a35d6b80 1 --2- 172.21.15.86:0/3169116017 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x560c4e5c2660 0x560c4e5c4a80 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:45.572 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.567+0000 7fd9a35d6b80 1 --2- 172.21.15.86:0/3169116017 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c4e5b2ce0 0x560c4e5b0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:45.572 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.567+0000 7fd96ea00700 0 client.26441 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:55:45.572 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.567+0000 7fd9a35d6b80 1 --2- 172.21.15.86:0/3169116017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c4e5a3100 0x560c4e5b2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:45.573 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.567+0000 7fd9a35d6b80 1 --2- 172.21.15.86:0/3169116017 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c4e5a3a20 0x560c4e5af9f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:45.573 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.567+0000 7fd9a35d6b80 1 -- 172.21.15.86:0/3169116017 >> 172.21.15.86:0/3169116017 conn(0x560c4e503640 msgr2=0x560c4e5a5080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:55:45.573 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.567+0000 7fd9a35d6b80 1 -- 172.21.15.86:0/3169116017 shutdown_connections 2024-03-23T17:55:45.573 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.567+0000 7fd9a35d6b80 1 -- 172.21.15.86:0/3169116017 wait complete. 2024-03-23T17:55:45.573 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.568+0000 7fd9a35d6b80 20 client.26441 trim_cache size 0 max 16384 2024-03-23T17:55:45.583 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.583+0000 7fd9a35d6b80 1 Processor -- start 2024-03-23T17:55:45.583 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.583+0000 7fd9a35d6b80 1 -- start start 2024-03-23T17:55:45.586 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.584+0000 7fd9a35d6b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c4e5a3100 0x560c4e5a5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:45.586 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.584+0000 7fd9a35d6b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c4e5a3a20 0x560c4e5af9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:45.586 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.584+0000 7fd9a35d6b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c4e5b2ce0 0x560c4e5c93e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:45.586 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.584+0000 7fd9a35d6b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c4e5c4e40 con 0x560c4e5a3100 2024-03-23T17:55:45.586 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.584+0000 7fd9a35d6b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c4e5b0d40 con 0x560c4e5a3a20 2024-03-23T17:55:45.586 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.584+0000 7fd9a35d6b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c4e5a01f0 con 0x560c4e5b2ce0 2024-03-23T17:55:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.584+0000 7fd987e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c4e5a3a20 0x560c4e5af9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.584+0000 7fd987e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c4e5a3a20 0x560c4e5af9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:46628/0 (socket says 172.21.15.86:46628) 2024-03-23T17:55:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.584+0000 7fd987e00700 1 -- 172.21.15.86:0/1690213204 learned_addr learned my addr 172.21.15.86:0/1690213204 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:55:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.584+0000 7fd98d000700 1 --2- 172.21.15.86:0/1690213204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c4e5a3100 0x560c4e5a5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.584+0000 7fd98da00700 1 --2- 172.21.15.86:0/1690213204 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c4e5b2ce0 0x560c4e5c93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.584+0000 7fd987e00700 1 -- 172.21.15.86:0/1690213204 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c4e5b2ce0 msgr2=0x560c4e5c93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:55:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.584+0000 7fd987e00700 1 --2- 172.21.15.86:0/1690213204 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c4e5b2ce0 0x560c4e5c93e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.584+0000 7fd987e00700 1 -- 172.21.15.86:0/1690213204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c4e5a3100 msgr2=0x560c4e5a5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:55:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.584+0000 7fd987e00700 1 --2- 172.21.15.86:0/1690213204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c4e5a3100 0x560c4e5a5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.584+0000 7fd987e00700 1 -- 172.21.15.86:0/1690213204 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c4e4bbe40 con 0x560c4e5a3a20 2024-03-23T17:55:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.585+0000 7fd987e00700 1 --2- 172.21.15.86:0/1690213204 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c4e5a3a20 0x560c4e5af9b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd9740054e0 tx=0x7fd974004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b79517cc5c54570 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:55:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.585+0000 7fd97aa00700 1 -- 172.21.15.86:0/1690213204 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd974005600 con 0x560c4e5a3a20 2024-03-23T17:55:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.585+0000 7fd97aa00700 1 -- 172.21.15.86:0/1690213204 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd974005760 con 0x560c4e5a3a20 2024-03-23T17:55:45.588 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.585+0000 7fd97aa00700 1 -- 172.21.15.86:0/1690213204 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd974003d90 con 0x560c4e5a3a20 2024-03-23T17:55:45.588 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.586+0000 7fd9a35d6b80 1 -- 172.21.15.86:0/1690213204 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c4e5a3a20 msgr2=0x560c4e5af9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:55:45.588 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.586+0000 7fd9a35d6b80 1 --2- 172.21.15.86:0/1690213204 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c4e5a3a20 0x560c4e5af9b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd9740054e0 tx=0x7fd974004750 comp rx=0 tx=0).stop 2024-03-23T17:55:45.588 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.586+0000 7fd9a35d6b80 1 -- 172.21.15.86:0/1690213204 shutdown_connections 2024-03-23T17:55:45.588 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.586+0000 7fd9a35d6b80 1 --2- 172.21.15.86:0/1690213204 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c4e5b2ce0 0x560c4e5c93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:45.588 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.586+0000 7fd9a35d6b80 1 --2- 172.21.15.86:0/1690213204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c4e5a3100 0x560c4e5a5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:45.588 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.586+0000 7fd9a35d6b80 1 --2- 172.21.15.86:0/1690213204 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c4e5a3a20 0x560c4e5af9b0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:45.588 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.586+0000 7fd9a35d6b80 1 -- 172.21.15.86:0/1690213204 >> 172.21.15.86:0/1690213204 conn(0x560c4e503640 msgr2=0x560c4e5983e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:55:45.588 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.586+0000 7fd9a35d6b80 1 -- 172.21.15.86:0/1690213204 shutdown_connections 2024-03-23T17:55:45.588 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.586+0000 7fd9a35d6b80 1 -- 172.21.15.86:0/1690213204 wait complete. 2024-03-23T17:55:45.588 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.586+0000 7fd9a35d6b80 1 -- start start 2024-03-23T17:55:45.588 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.586+0000 7fd9a35d6b80 1 Processor -- start 2024-03-23T17:55:45.591 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.587+0000 7fd9a35d6b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c4e5a3100 0x560c4e5b8f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:45.591 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.587+0000 7fd9a35d6b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c4e5a3a20 0x560c4e5b94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:45.591 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.587+0000 7fd9a35d6b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c4e5b2ce0 0x560c4e5b76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:45.591 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.587+0000 7fd9a35d6b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c4e5b7d70 con 0x560c4e5a3a20 2024-03-23T17:55:45.591 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.587+0000 7fd9a35d6b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c4e5b7ee0 con 0x560c4e5a3100 2024-03-23T17:55:45.591 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.587+0000 7fd9a35d6b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c4e5b80a0 con 0x560c4e5b2ce0 2024-03-23T17:55:45.591 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.587+0000 7fd9a35d6b80 10 client.0 _ll_register_callbacks cb 0x560c4e49b410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:55:45.591 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.587+0000 7fd98d000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c4e5a3100 0x560c4e5b8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:45.592 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.587+0000 7fd98d000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c4e5a3100 0x560c4e5b8f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:46640/0 (socket says 172.21.15.86:46640) 2024-03-23T17:55:45.592 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.587+0000 7fd98d000700 1 -- 172.21.15.86:0/2261488171 learned_addr learned my addr 172.21.15.86:0/2261488171 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:55:45.592 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.587+0000 7fd987e00700 1 --2- 172.21.15.86:0/2261488171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c4e5a3a20 0x560c4e5b94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:45.592 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.587+0000 7fd98da00700 1 --2- 172.21.15.86:0/2261488171 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c4e5b2ce0 0x560c4e5b76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:45.592 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.587+0000 7fd98d000700 1 -- 172.21.15.86:0/2261488171 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c4e5b2ce0 msgr2=0x560c4e5b76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:55:45.592 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.587+0000 7fd98d000700 1 --2- 172.21.15.86:0/2261488171 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c4e5b2ce0 0x560c4e5b76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:45.592 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.587+0000 7fd98d000700 1 -- 172.21.15.86:0/2261488171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c4e5a3a20 msgr2=0x560c4e5b94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:55:45.592 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.587+0000 7fd98d000700 1 --2- 172.21.15.86:0/2261488171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c4e5a3a20 0x560c4e5b94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:45.592 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.587+0000 7fd98d000700 1 -- 172.21.15.86:0/2261488171 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560c4e323b00 con 0x560c4e5a3100 2024-03-23T17:55:45.592 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.588+0000 7fd98d000700 1 --2- 172.21.15.86:0/2261488171 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c4e5a3100 0x560c4e5b8f90 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd97c0974f0 tx=0x7fd97c022f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd46d141f3911903 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:55:45.592 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.588+0000 7fd96ea00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T17:55:45.592 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.588+0000 7fd96ea00700 1 -- 172.21.15.86:0/2261488171 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97c0a33e0 con 0x560c4e5a3100 2024-03-23T17:55:45.592 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.588+0000 7fd96ea00700 1 -- 172.21.15.86:0/2261488171 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fd97c09ee20 con 0x560c4e5a3100 2024-03-23T17:55:45.593 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.588+0000 7fd9a35d6b80 1 -- 172.21.15.86:0/2261488171 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560c4e344c90 con 0x560c4e5a3100 2024-03-23T17:55:45.593 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.588+0000 7fd96ea00700 1 -- 172.21.15.86:0/2261488171 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd97c0b9270 con 0x560c4e5a3100 2024-03-23T17:55:45.593 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.588+0000 7fd9a35d6b80 10 client.26447 fetch_fsmap learned FSMap version 38 2024-03-23T17:55:45.593 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.588+0000 7fd9a35d6b80 1 -- 172.21.15.86:0/2261488171 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x560c4e37bc70 con 0x560c4e5a3100 2024-03-23T17:55:45.593 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.588+0000 7fd96ea00700 1 -- 172.21.15.86:0/2261488171 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd97c083030 con 0x560c4e5a3100 2024-03-23T17:55:45.593 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.588+0000 7fd96ea00700 1 -- 172.21.15.86:0/2261488171 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 38) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd97c09b250 con 0x560c4e5a3100 2024-03-23T17:55:45.593 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.588+0000 7fd9a35d6b80 10 client.26447 fetch_fsmap finished waiting for FSMap version 38 2024-03-23T17:55:45.593 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.588+0000 7fd9a35d6b80 10 client.26447 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:55:45.593 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.588+0000 7fd9a35d6b80 20 client.26447 populate_metadata read hostname 'smithi086' 2024-03-23T17:55:45.593 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.588+0000 7fd9a35d6b80 1 --2- 172.21.15.86:0/2261488171 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x560c4e614f40 0x560c4e617360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:45.593 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.588+0000 7fd9a35d6b80 4 client.26447 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T17:55:45.593 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.588+0000 7fd9a35d6b80 1 -- 172.21.15.86:0/2261488171 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560c4e379c80 con 0x560c4e614f40 2024-03-23T17:55:45.593 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.589+0000 7fd987e00700 1 --2- 172.21.15.86:0/2261488171 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x560c4e614f40 0x560c4e617360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:45.594 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.589+0000 7fd987e00700 1 --2- 172.21.15.86:0/2261488171 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x560c4e614f40 0x560c4e617360 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8426af967f0b181a server_cookie=a8120f615319cea0 in_seq=0 out_seq=0 2024-03-23T17:55:45.594 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.589+0000 7fd96ea00700 10 client.26447 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:55:45.637 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.637+0000 7fd96ea00700 1 -- 172.21.15.86:0/2261488171 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560c4e379c80 con 0x560c4e614f40 2024-03-23T17:55:45.637 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.637+0000 7fd96ea00700 10 client.26447 handle_command_reply: tid=0 2024-03-23T17:55:45.637 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.637+0000 7fd9a35d6b80 1 client.26447 shutdown 2024-03-23T17:55:45.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.637+0000 7fd9a35d6b80 1 -- 172.21.15.86:0/2261488171 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c4e5a3100 msgr2=0x560c4e5b8f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:55:45.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.637+0000 7fd9a35d6b80 1 --2- 172.21.15.86:0/2261488171 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c4e5a3100 0x560c4e5b8f90 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd97c0974f0 tx=0x7fd97c022f90 comp rx=0 tx=0).stop 2024-03-23T17:55:45.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.637+0000 7fd9a35d6b80 1 -- 172.21.15.86:0/2261488171 shutdown_connections 2024-03-23T17:55:45.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.637+0000 7fd9a35d6b80 1 --2- 172.21.15.86:0/2261488171 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x560c4e614f40 0x560c4e617360 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:45.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.637+0000 7fd9a35d6b80 1 --2- 172.21.15.86:0/2261488171 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560c4e5b2ce0 0x560c4e5b76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:45.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.637+0000 7fd9a35d6b80 1 --2- 172.21.15.86:0/2261488171 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c4e5a3100 0x560c4e5b8f90 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:45.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.637+0000 7fd96ea00700 0 client.26447 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:55:45.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.637+0000 7fd9a35d6b80 1 --2- 172.21.15.86:0/2261488171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c4e5a3a20 0x560c4e5b94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:45.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.637+0000 7fd9a35d6b80 1 -- 172.21.15.86:0/2261488171 >> 172.21.15.86:0/2261488171 conn(0x560c4e503640 msgr2=0x560c4e59f450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:55:45.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.638+0000 7fd9a35d6b80 1 -- 172.21.15.86:0/2261488171 shutdown_connections 2024-03-23T17:55:45.639 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.638+0000 7fd9a35d6b80 1 -- 172.21.15.86:0/2261488171 wait complete. 2024-03-23T17:55:45.639 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.638+0000 7fd9a35d6b80 20 client.26447 trim_cache size 0 max 16384 2024-03-23T17:55:45.639 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-03-23T17:55:45.639 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-03-23T17:55:45.640 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-03-23T17:55:45.640 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-03-23T17:55:45.640 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.639+0000 7fd98fa00700 1 -- 172.21.15.86:0/2884838483 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fd9700a1310 msgr2=0x7fd9700a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:55:45.640 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.639+0000 7fd98fa00700 1 --2- 172.21.15.86:0/2884838483 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fd9700a1310 0x7fd9700a37d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd9881987c0 tx=0x7fd974009260 comp rx=0 tx=0).stop 2024-03-23T17:55:45.640 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.639+0000 7fd98fa00700 1 -- 172.21.15.86:0/2884838483 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd988100f10 msgr2=0x7fd988197270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:55:45.641 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.639+0000 7fd98fa00700 1 --2- 172.21.15.86:0/2884838483 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd988100f10 0x7fd988197270 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd97c000c00 tx=0x7fd97c005740 comp rx=0 tx=0).stop 2024-03-23T17:55:45.641 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.639+0000 7fd98fa00700 1 -- 172.21.15.86:0/2884838483 shutdown_connections 2024-03-23T17:55:45.641 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.639+0000 7fd98fa00700 1 --2- 172.21.15.86:0/2884838483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd98819bb50 0x7fd98819bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:45.641 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.639+0000 7fd98fa00700 1 --2- 172.21.15.86:0/2884838483 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd988100f10 0x7fd988197270 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:45.641 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.639+0000 7fd98fa00700 1 --2- 172.21.15.86:0/2884838483 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fd9700a1310 0x7fd9700a37d0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:45.641 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.639+0000 7fd98fa00700 1 --2- 172.21.15.86:0/2884838483 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd988106030 0x7fd9881977b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:45.641 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.639+0000 7fd98fa00700 1 -- 172.21.15.86:0/2884838483 >> 172.21.15.86:0/2884838483 conn(0x7fd9880fc7b0 msgr2=0x7fd9880fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:55:45.641 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.639+0000 7fd98fa00700 1 -- 172.21.15.86:0/2884838483 shutdown_connections 2024-03-23T17:55:45.641 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:45.639+0000 7fd98fa00700 1 -- 172.21.15.86:0/2884838483 wait complete. 2024-03-23T17:55:45.900 INFO:tasks.workunit.client.0.smithi086.stdout: 1 208310 5.21 MB/sec execute 130 sec latency 15.746 ms 2024-03-23T17:55:46.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:55:45 smithi086 ceph-mon[38221]: pgmap v156: 97 pgs: 97 active+clean; 87 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 313 op/s 2024-03-23T17:55:46.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:55:45 smithi092 ceph-mon[79089]: pgmap v156: 97 pgs: 97 active+clean; 87 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 313 op/s 2024-03-23T17:55:46.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:55:45 smithi152 ceph-mon[47949]: pgmap v156: 97 pgs: 97 active+clean; 87 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 313 op/s 2024-03-23T17:55:46.900 INFO:tasks.workunit.client.0.smithi086.stdout: 1 209785 5.25 MB/sec execute 131 sec latency 15.663 ms 2024-03-23T17:55:47.900 INFO:tasks.workunit.client.0.smithi086.stdout: 1 212155 5.30 MB/sec execute 132 sec latency 11.349 ms 2024-03-23T17:55:48.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:55:47 smithi086 ceph-mon[38221]: pgmap v157: 97 pgs: 97 active+clean; 87 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 189 op/s 2024-03-23T17:55:48.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:55:47 smithi092 ceph-mon[79089]: pgmap v157: 97 pgs: 97 active+clean; 87 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 189 op/s 2024-03-23T17:55:48.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:55:47 smithi152 ceph-mon[47949]: pgmap v157: 97 pgs: 97 active+clean; 87 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 189 op/s 2024-03-23T17:55:48.900 INFO:tasks.workunit.client.0.smithi086.stdout: 1 213421 5.32 MB/sec execute 133 sec latency 10.752 ms 2024-03-23T17:55:49.900 INFO:tasks.workunit.client.0.smithi086.stdout: 1 215687 5.36 MB/sec execute 134 sec latency 14.498 ms 2024-03-23T17:55:50.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:55:49 smithi086 ceph-mon[38221]: pgmap v158: 97 pgs: 97 active+clean; 87 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 189 op/s 2024-03-23T17:55:50.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:55:49 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:55:50.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:55:49 smithi092 ceph-mon[79089]: pgmap v158: 97 pgs: 97 active+clean; 87 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 189 op/s 2024-03-23T17:55:50.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:55:49 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:55:50.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:55:49 smithi152 ceph-mon[47949]: pgmap v158: 97 pgs: 97 active+clean; 87 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 189 op/s 2024-03-23T17:55:50.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:55:49 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:55:50.900 INFO:tasks.workunit.client.0.smithi086.stdout: 1 216957 5.38 MB/sec execute 135 sec latency 11.750 ms 2024-03-23T17:55:51.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:55:50 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:55:51.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:55:50 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:55:51.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:55:50 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:55:51.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:55:50 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:55:51.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:55:50 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:55:51.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:55:50 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:55:51.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:55:50 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:55:51.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:55:50 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:55:51.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:55:50 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:55:51.900 INFO:tasks.workunit.client.0.smithi086.stdout: 1 219004 5.40 MB/sec execute 136 sec latency 13.201 ms 2024-03-23T17:55:52.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:55:51 smithi086 ceph-mon[38221]: pgmap v159: 97 pgs: 97 active+clean; 85 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 308 op/s 2024-03-23T17:55:52.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:55:51 smithi092 ceph-mon[79089]: pgmap v159: 97 pgs: 97 active+clean; 85 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 308 op/s 2024-03-23T17:55:52.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:55:51 smithi152 ceph-mon[47949]: pgmap v159: 97 pgs: 97 active+clean; 85 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 308 op/s 2024-03-23T17:55:52.900 INFO:tasks.workunit.client.0.smithi086.stdout: 1 220390 5.44 MB/sec execute 137 sec latency 13.613 ms 2024-03-23T17:55:53.900 INFO:tasks.workunit.client.0.smithi086.stdout: 1 222542 5.46 MB/sec execute 138 sec latency 13.467 ms 2024-03-23T17:55:54.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:55:53 smithi086 ceph-mon[38221]: pgmap v160: 97 pgs: 97 active+clean; 85 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 212 op/s 2024-03-23T17:55:54.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:55:53 smithi092 ceph-mon[79089]: pgmap v160: 97 pgs: 97 active+clean; 85 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 212 op/s 2024-03-23T17:55:54.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:55:53 smithi152 ceph-mon[47949]: pgmap v160: 97 pgs: 97 active+clean; 85 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 212 op/s 2024-03-23T17:55:54.900 INFO:tasks.workunit.client.0.smithi086.stdout: 1 224020 5.50 MB/sec execute 139 sec latency 11.919 ms 2024-03-23T17:55:55.900 INFO:tasks.workunit.client.0.smithi086.stdout: 1 226391 5.54 MB/sec execute 140 sec latency 11.250 ms 2024-03-23T17:55:56.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:55:55 smithi086 ceph-mon[38221]: pgmap v161: 97 pgs: 97 active+clean; 91 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.9 MiB/s wr, 305 op/s 2024-03-23T17:55:56.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:55:55 smithi092 ceph-mon[79089]: pgmap v161: 97 pgs: 97 active+clean; 91 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.9 MiB/s wr, 305 op/s 2024-03-23T17:55:56.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:55:55 smithi152 ceph-mon[47949]: pgmap v161: 97 pgs: 97 active+clean; 91 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.9 MiB/s wr, 305 op/s 2024-03-23T17:55:56.901 INFO:tasks.workunit.client.0.smithi086.stdout: 1 227513 5.56 MB/sec execute 141 sec latency 201.748 ms 2024-03-23T17:55:57.900 INFO:tasks.workunit.client.0.smithi086.stdout: 1 229652 5.57 MB/sec execute 142 sec latency 14.316 ms 2024-03-23T17:55:58.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:55:57 smithi086 ceph-mon[38221]: pgmap v162: 97 pgs: 97 active+clean; 91 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 212 op/s 2024-03-23T17:55:58.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:55:57 smithi092 ceph-mon[79089]: pgmap v162: 97 pgs: 97 active+clean; 91 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 212 op/s 2024-03-23T17:55:58.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:55:57 smithi152 ceph-mon[47949]: pgmap v162: 97 pgs: 97 active+clean; 91 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 212 op/s 2024-03-23T17:55:58.791 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.790+0000 7f3e46a00700 1 -- 172.21.15.86:0/3532788550 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e40101ec0 msgr2=0x7f3e401022a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:55:58.791 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.790+0000 7f3e46a00700 1 --2- 172.21.15.86:0/3532788550 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e40101ec0 0x7f3e401022a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3e30009990 tx=0x7f3e30009ca0 comp rx=0 tx=0).stop 2024-03-23T17:55:58.791 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.790+0000 7f3e46a00700 1 -- 172.21.15.86:0/3532788550 shutdown_connections 2024-03-23T17:55:58.791 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.790+0000 7f3e46a00700 1 --2- 172.21.15.86:0/3532788550 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3e4010b2e0 0x7f3e4010d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:58.791 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.790+0000 7f3e46a00700 1 --2- 172.21.15.86:0/3532788550 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e40101ec0 0x7f3e401022a0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:58.791 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.790+0000 7f3e46a00700 1 --2- 172.21.15.86:0/3532788550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e401027e0 0x7f3e4010acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:58.791 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.790+0000 7f3e46a00700 1 -- 172.21.15.86:0/3532788550 >> 172.21.15.86:0/3532788550 conn(0x7f3e400fb680 msgr2=0x7f3e400fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:55:58.791 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.790+0000 7f3e46a00700 1 -- 172.21.15.86:0/3532788550 shutdown_connections 2024-03-23T17:55:58.791 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.790+0000 7f3e46a00700 1 -- 172.21.15.86:0/3532788550 wait complete. 2024-03-23T17:55:58.793 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.791+0000 7f3e46a00700 1 Processor -- start 2024-03-23T17:55:58.793 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.792+0000 7f3e46a00700 1 -- start start 2024-03-23T17:55:58.794 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.792+0000 7f3e46a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e40101ec0 0x7f3e40199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:58.794 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.792+0000 7f3e46a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e401027e0 0x7f3e401998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:58.794 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.792+0000 7f3e46a00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3e4010b2e0 0x7f3e4019dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:58.794 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.792+0000 7f3e46a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4019e290 con 0x7f3e40101ec0 2024-03-23T17:55:58.794 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.792+0000 7f3e46a00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4019e400 con 0x7f3e401027e0 2024-03-23T17:55:58.794 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.792+0000 7f3e46a00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4019e5c0 con 0x7f3e4010b2e0 2024-03-23T17:55:58.794 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.792+0000 7f3e3fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e40101ec0 0x7f3e40199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:58.794 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.792+0000 7f3e3f400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e401027e0 0x7f3e401998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:58.794 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.792+0000 7f3e3fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e40101ec0 0x7f3e40199370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40468/0 (socket says 172.21.15.86:40468) 2024-03-23T17:55:58.794 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.792+0000 7f3e3fe00700 1 -- 172.21.15.86:0/4276068897 learned_addr learned my addr 172.21.15.86:0/4276068897 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:55:58.794 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.792+0000 7f3e44a00700 1 --2- 172.21.15.86:0/4276068897 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3e4010b2e0 0x7f3e4019dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:58.794 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.793+0000 7f3e3fe00700 1 -- 172.21.15.86:0/4276068897 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3e4010b2e0 msgr2=0x7f3e4019dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:55:58.795 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.793+0000 7f3e3fe00700 1 --2- 172.21.15.86:0/4276068897 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3e4010b2e0 0x7f3e4019dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:58.795 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.793+0000 7f3e3fe00700 1 -- 172.21.15.86:0/4276068897 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e401027e0 msgr2=0x7f3e401998b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:55:58.795 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.793+0000 7f3e3fe00700 1 --2- 172.21.15.86:0/4276068897 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e401027e0 0x7f3e401998b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:58.795 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.793+0000 7f3e3fe00700 1 -- 172.21.15.86:0/4276068897 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e300095f0 con 0x7f3e40101ec0 2024-03-23T17:55:58.795 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.793+0000 7f3e3f400700 1 --2- 172.21.15.86:0/4276068897 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e401027e0 0x7f3e401998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:55:58.795 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.793+0000 7f3e3fe00700 1 --2- 172.21.15.86:0/4276068897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e40101ec0 0x7f3e40199370 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f3e30005910 tx=0x7f3e30004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:55:58.795 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.793+0000 7f3e3cc00700 1 -- 172.21.15.86:0/4276068897 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e3001d070 con 0x7f3e40101ec0 2024-03-23T17:55:58.795 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.793+0000 7f3e3cc00700 1 -- 172.21.15.86:0/4276068897 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e30022470 con 0x7f3e40101ec0 2024-03-23T17:55:58.795 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.793+0000 7f3e46a00700 1 -- 172.21.15.86:0/4276068897 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e4019e840 con 0x7f3e40101ec0 2024-03-23T17:55:58.795 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.793+0000 7f3e3cc00700 1 -- 172.21.15.86:0/4276068897 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e30017650 con 0x7f3e40101ec0 2024-03-23T17:55:58.795 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.793+0000 7f3e46a00700 1 -- 172.21.15.86:0/4276068897 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e401a60e0 con 0x7f3e40101ec0 2024-03-23T17:55:58.796 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.796+0000 7f3e3cc00700 1 -- 172.21.15.86:0/4276068897 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f3e300225e0 con 0x7f3e40101ec0 2024-03-23T17:55:58.797 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.797+0000 7f3e3cc00700 1 --2- 172.21.15.86:0/4276068897 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f3e240a1290 0x7f3e240a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:58.798 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.798+0000 7f3e3cc00700 1 -- 172.21.15.86:0/4276068897 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f3e300cdf50 con 0x7f3e40101ec0 2024-03-23T17:55:58.798 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.798+0000 7f3e3f400700 1 --2- 172.21.15.86:0/4276068897 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f3e240a1290 0x7f3e240a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:58.798 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.798+0000 7f3e5a509b80 1 Processor -- start 2024-03-23T17:55:58.798 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.798+0000 7f3e5a509b80 1 -- start start 2024-03-23T17:55:58.798 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.798+0000 7f3e3f400700 1 --2- 172.21.15.86:0/4276068897 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f3e240a1290 0x7f3e240a3750 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3e4019a8c0 tx=0x7f3e2c009260 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:55:58.800 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.799+0000 7f3e5a509b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55730b112100 0x55730b1124e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:58.800 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.799+0000 7f3e5a509b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55730b112a20 0x55730b11d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:58.800 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.799+0000 7f3e5a509b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55730b11d8d0 0x55730b11fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:58.800 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.799+0000 7f3e5a509b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55730b06b930 con 0x55730b112100 2024-03-23T17:55:58.800 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.799+0000 7f3e5a509b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55730b0692f0 con 0x55730b11d8d0 2024-03-23T17:55:58.800 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.799+0000 7f3e5a509b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55730b064be0 con 0x55730b112a20 2024-03-23T17:55:58.800 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.799+0000 7f3e3f400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55730b112a20 0x55730b11d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:58.800 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.799+0000 7f3e3f400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55730b112a20 0x55730b11d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:54922/0 (socket says 172.21.15.86:54922) 2024-03-23T17:55:58.800 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.799+0000 7f3e3f400700 1 -- 172.21.15.86:0/2696798866 learned_addr learned my addr 172.21.15.86:0/2696798866 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:55:58.801 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.799+0000 7f3e3fe00700 1 --2- 172.21.15.86:0/2696798866 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55730b112100 0x55730b1124e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:58.801 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.799+0000 7f3e44a00700 1 --2- 172.21.15.86:0/2696798866 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55730b11d8d0 0x55730b11fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:58.801 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.799+0000 7f3e3f400700 1 -- 172.21.15.86:0/2696798866 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55730b11d8d0 msgr2=0x55730b11fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:55:58.801 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.799+0000 7f3e3f400700 1 --2- 172.21.15.86:0/2696798866 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55730b11d8d0 0x55730b11fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:58.801 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.799+0000 7f3e3f400700 1 -- 172.21.15.86:0/2696798866 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55730b112100 msgr2=0x55730b1124e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:55:58.801 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.799+0000 7f3e3f400700 1 --2- 172.21.15.86:0/2696798866 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55730b112100 0x55730b1124e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:58.801 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.799+0000 7f3e3f400700 1 -- 172.21.15.86:0/2696798866 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55730aee6350 con 0x55730b112a20 2024-03-23T17:55:58.801 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.800+0000 7f3e3f400700 1 --2- 172.21.15.86:0/2696798866 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55730b112a20 0x55730b11d390 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c0043c0 tx=0x7f3e2c014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ebe062ae0168d16 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:55:58.801 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.800+0000 7f3e35200700 1 -- 172.21.15.86:0/2696798866 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e2c003d10 con 0x55730b112a20 2024-03-23T17:55:58.801 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.800+0000 7f3e35200700 1 -- 172.21.15.86:0/2696798866 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e2c003e70 con 0x55730b112a20 2024-03-23T17:55:58.801 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.800+0000 7f3e35200700 1 -- 172.21.15.86:0/2696798866 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e2c01e7e0 con 0x55730b112a20 2024-03-23T17:55:58.804 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.801+0000 7f3e5a509b80 1 -- 172.21.15.86:0/2696798866 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55730b112a20 msgr2=0x55730b11d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:55:58.804 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.801+0000 7f3e5a509b80 1 --2- 172.21.15.86:0/2696798866 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55730b112a20 0x55730b11d390 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c0043c0 tx=0x7f3e2c014000 comp rx=0 tx=0).stop 2024-03-23T17:55:58.805 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.801+0000 7f3e5a509b80 1 -- 172.21.15.86:0/2696798866 shutdown_connections 2024-03-23T17:55:58.805 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.801+0000 7f3e5a509b80 1 --2- 172.21.15.86:0/2696798866 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55730b11d8d0 0x55730b11fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:58.805 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.801+0000 7f3e5a509b80 1 --2- 172.21.15.86:0/2696798866 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55730b112100 0x55730b1124e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:58.805 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.801+0000 7f3e5a509b80 1 --2- 172.21.15.86:0/2696798866 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55730b112a20 0x55730b11d390 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:58.805 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.801+0000 7f3e5a509b80 1 -- 172.21.15.86:0/2696798866 >> 172.21.15.86:0/2696798866 conn(0x55730b072640 msgr2=0x55730b10e520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:55:58.805 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.801+0000 7f3e5a509b80 1 -- 172.21.15.86:0/2696798866 shutdown_connections 2024-03-23T17:55:58.805 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.801+0000 7f3e5a509b80 1 -- 172.21.15.86:0/2696798866 wait complete. 2024-03-23T17:55:58.805 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.801+0000 7f3e5a509b80 1 -- start start 2024-03-23T17:55:58.805 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.802+0000 7f3e5a509b80 1 Processor -- start 2024-03-23T17:55:58.805 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.802+0000 7f3e5a509b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55730b112100 0x55730b11d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:58.805 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.802+0000 7f3e5a509b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55730b112a20 0x55730b11a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:58.805 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.802+0000 7f3e5a509b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55730b11d8d0 0x55730b11ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:58.805 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.802+0000 7f3e5a509b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55730b000d20 con 0x55730b112100 2024-03-23T17:55:58.806 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.802+0000 7f3e5a509b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55730b0009c0 con 0x55730b11d8d0 2024-03-23T17:55:58.806 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.802+0000 7f3e5a509b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55730aee8640 con 0x55730b112a20 2024-03-23T17:55:58.806 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.802+0000 7f3e5a509b80 10 client.0 _ll_register_callbacks cb 0x55730b00a410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:55:58.806 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.802+0000 7f3e3fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55730b112100 0x55730b11d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:58.806 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.802+0000 7f3e3f400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55730b112a20 0x55730b11a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:58.806 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.802+0000 7f3e44a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55730b11d8d0 0x55730b11ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:58.806 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.802+0000 7f3e3f400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55730b112a20 0x55730b11a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:54930/0 (socket says 172.21.15.86:54930) 2024-03-23T17:55:58.806 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.803+0000 7f3e3f400700 1 -- 172.21.15.86:0/1514084226 learned_addr learned my addr 172.21.15.86:0/1514084226 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:55:58.806 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.803+0000 7f3e44a00700 1 -- 172.21.15.86:0/1514084226 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55730b112a20 msgr2=0x55730b11a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:55:58.806 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.803+0000 7f3e44a00700 1 --2- 172.21.15.86:0/1514084226 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55730b112a20 0x55730b11a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:58.806 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.803+0000 7f3e44a00700 1 -- 172.21.15.86:0/1514084226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55730b112100 msgr2=0x55730b11d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:55:58.806 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.803+0000 7f3e44a00700 1 --2- 172.21.15.86:0/1514084226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55730b112100 0x55730b11d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:58.806 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.803+0000 7f3e44a00700 1 -- 172.21.15.86:0/1514084226 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e2c017070 con 0x55730b11d8d0 2024-03-23T17:55:58.807 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.803+0000 7f3e3f400700 1 --2- 172.21.15.86:0/1514084226 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55730b112a20 0x55730b11a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:55:58.807 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.803+0000 7f3e44a00700 1 --2- 172.21.15.86:0/1514084226 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55730b11d8d0 0x55730b11ac60 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3e38011e60 tx=0x7f3e38014600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e3f238fad77418 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:55:58.807 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.803+0000 7f3e29600700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T17:55:58.807 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.803+0000 7f3e29600700 1 -- 172.21.15.86:0/1514084226 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e3800e040 con 0x55730b11d8d0 2024-03-23T17:55:58.807 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.803+0000 7f3e5a509b80 1 -- 172.21.15.86:0/1514084226 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55730b025320 con 0x55730b11d8d0 2024-03-23T17:55:58.807 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.803+0000 7f3e29600700 1 -- 172.21.15.86:0/1514084226 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f3e380093a0 con 0x55730b11d8d0 2024-03-23T17:55:58.807 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.804+0000 7f3e29600700 1 -- 172.21.15.86:0/1514084226 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3e38004db0 con 0x55730b11d8d0 2024-03-23T17:55:58.807 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.804+0000 7f3e5a509b80 10 client.26459 fetch_fsmap learned FSMap version 38 2024-03-23T17:55:58.807 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.804+0000 7f3e5a509b80 1 -- 172.21.15.86:0/1514084226 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55730ae92b00 con 0x55730b11d8d0 2024-03-23T17:55:58.807 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.804+0000 7f3e29600700 1 -- 172.21.15.86:0/1514084226 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55730b025320 con 0x55730b11d8d0 2024-03-23T17:55:58.807 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.804+0000 7f3e29600700 1 -- 172.21.15.86:0/1514084226 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 38) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3e3801c030 con 0x55730b11d8d0 2024-03-23T17:55:58.807 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.804+0000 7f3e5a509b80 10 client.26459 fetch_fsmap finished waiting for FSMap version 38 2024-03-23T17:55:58.807 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.804+0000 7f3e5a509b80 10 client.26459 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:55:58.807 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.804+0000 7f3e5a509b80 20 client.26459 populate_metadata read hostname 'smithi086' 2024-03-23T17:55:58.808 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.804+0000 7f3e5a509b80 1 --2- 172.21.15.86:0/1514084226 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55730b12d5f0 0x55730b12fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:58.808 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.804+0000 7f3e5a509b80 4 client.26459 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:55:58.808 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.804+0000 7f3e5a509b80 1 -- 172.21.15.86:0/1514084226 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55730b05d6a0 con 0x55730b12d5f0 2024-03-23T17:55:58.808 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.805+0000 7f3e3fe00700 1 --2- 172.21.15.86:0/1514084226 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55730b12d5f0 0x55730b12fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:58.809 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.805+0000 7f3e3fe00700 1 --2- 172.21.15.86:0/1514084226 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55730b12d5f0 0x55730b12fa10 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56fb4c6cbcc13e54 server_cookie=79d8a0ed92f9d261 in_seq=0 out_seq=0 2024-03-23T17:55:58.809 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.805+0000 7f3e29600700 10 client.26459 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:55:58.809 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.806+0000 7f3e29600700 1 -- 172.21.15.86:0/1514084226 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55730b05d6a0 con 0x55730b12d5f0 2024-03-23T17:55:58.809 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.806+0000 7f3e29600700 10 client.26459 handle_command_reply: tid=0 2024-03-23T17:55:58.809 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.806+0000 7f3e5a509b80 1 client.26459 shutdown 2024-03-23T17:55:58.809 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.806+0000 7f3e5a509b80 1 -- 172.21.15.86:0/1514084226 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55730b11d8d0 msgr2=0x55730b11ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:55:58.809 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.806+0000 7f3e5a509b80 1 --2- 172.21.15.86:0/1514084226 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55730b11d8d0 0x55730b11ac60 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3e38011e60 tx=0x7f3e38014600 comp rx=0 tx=0).stop 2024-03-23T17:55:58.810 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.806+0000 7f3e5a509b80 1 -- 172.21.15.86:0/1514084226 shutdown_connections 2024-03-23T17:55:58.810 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.806+0000 7f3e5a509b80 1 --2- 172.21.15.86:0/1514084226 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55730b12d5f0 0x55730b12fa10 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:58.810 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.806+0000 7f3e5a509b80 1 --2- 172.21.15.86:0/1514084226 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55730b11d8d0 0x55730b11ac60 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:58.810 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.806+0000 7f3e5a509b80 1 --2- 172.21.15.86:0/1514084226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55730b112100 0x55730b11d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:58.810 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.806+0000 7f3e29600700 0 client.26459 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:55:58.810 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.806+0000 7f3e5a509b80 1 --2- 172.21.15.86:0/1514084226 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55730b112a20 0x55730b11a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:58.810 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.806+0000 7f3e5a509b80 1 -- 172.21.15.86:0/1514084226 >> 172.21.15.86:0/1514084226 conn(0x55730b072640 msgr2=0x55730b114080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:55:58.810 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.806+0000 7f3e5a509b80 1 -- 172.21.15.86:0/1514084226 shutdown_connections 2024-03-23T17:55:58.810 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.806+0000 7f3e5a509b80 1 -- 172.21.15.86:0/1514084226 wait complete. 2024-03-23T17:55:58.810 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.807+0000 7f3e5a509b80 20 client.26459 trim_cache size 0 max 16384 2024-03-23T17:55:58.823 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.822+0000 7f3e5a509b80 1 Processor -- start 2024-03-23T17:55:58.823 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.822+0000 7f3e5a509b80 1 -- start start 2024-03-23T17:55:58.823 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.822+0000 7f3e5a509b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55730b112100 0x55730b1245e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:58.824 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.822+0000 7f3e5a509b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55730b112a20 0x55730b11a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:58.824 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.822+0000 7f3e5a509b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55730b11d8d0 0x55730b108500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:58.824 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.822+0000 7f3e5a509b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55730b10fac0 con 0x55730b11d8d0 2024-03-23T17:55:58.824 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.822+0000 7f3e5a509b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55730b10a400 con 0x55730b112a20 2024-03-23T17:55:58.824 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.822+0000 7f3e5a509b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55730b11b540 con 0x55730b112100 2024-03-23T17:55:58.824 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.823+0000 7f3e3fe00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55730b112100 0x55730b1245e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:58.824 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.823+0000 7f3e44a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55730b11d8d0 0x55730b108500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:58.824 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.823+0000 7f3e3fe00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55730b112100 0x55730b1245e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:54938/0 (socket says 172.21.15.86:54938) 2024-03-23T17:55:58.824 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.823+0000 7f3e3fe00700 1 -- 172.21.15.86:0/2374019471 learned_addr learned my addr 172.21.15.86:0/2374019471 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:55:58.824 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.823+0000 7f3e3f400700 1 --2- 172.21.15.86:0/2374019471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55730b112a20 0x55730b11a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:58.824 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.823+0000 7f3e3fe00700 1 -- 172.21.15.86:0/2374019471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55730b112a20 msgr2=0x55730b11a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:55:58.824 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.823+0000 7f3e3fe00700 1 --2- 172.21.15.86:0/2374019471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55730b112a20 0x55730b11a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:58.824 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.823+0000 7f3e3fe00700 1 -- 172.21.15.86:0/2374019471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55730b11d8d0 msgr2=0x55730b108500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:55:58.824 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.823+0000 7f3e3fe00700 1 --2- 172.21.15.86:0/2374019471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55730b11d8d0 0x55730b108500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:58.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.823+0000 7f3e3fe00700 1 -- 172.21.15.86:0/2374019471 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55730b02ae40 con 0x55730b112100 2024-03-23T17:55:58.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.823+0000 7f3e3fe00700 1 --2- 172.21.15.86:0/2374019471 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55730b112100 0x55730b1245e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3e3009af00 tx=0x7f3e300cd040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b50fddde7521c4bb server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:55:58.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.823+0000 7f3e35c00700 1 -- 172.21.15.86:0/2374019471 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e3009b7e0 con 0x55730b112100 2024-03-23T17:55:58.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.824+0000 7f3e35c00700 1 -- 172.21.15.86:0/2374019471 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e3009b0e0 con 0x55730b112100 2024-03-23T17:55:58.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.824+0000 7f3e35c00700 1 -- 172.21.15.86:0/2374019471 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e300b92c0 con 0x55730b112100 2024-03-23T17:55:58.828 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.824+0000 7f3e5a509b80 1 -- 172.21.15.86:0/2374019471 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55730b112100 msgr2=0x55730b1245e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:55:58.828 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.824+0000 7f3e5a509b80 1 --2- 172.21.15.86:0/2374019471 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55730b112100 0x55730b1245e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3e3009af00 tx=0x7f3e300cd040 comp rx=0 tx=0).stop 2024-03-23T17:55:58.828 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.824+0000 7f3e5a509b80 1 -- 172.21.15.86:0/2374019471 shutdown_connections 2024-03-23T17:55:58.828 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.824+0000 7f3e5a509b80 1 --2- 172.21.15.86:0/2374019471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55730b11d8d0 0x55730b108500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:58.828 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.824+0000 7f3e5a509b80 1 --2- 172.21.15.86:0/2374019471 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55730b112100 0x55730b1245e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:58.828 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.824+0000 7f3e5a509b80 1 --2- 172.21.15.86:0/2374019471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55730b112a20 0x55730b11a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:58.829 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.824+0000 7f3e5a509b80 1 -- 172.21.15.86:0/2374019471 >> 172.21.15.86:0/2374019471 conn(0x55730b072640 msgr2=0x55730b119ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:55:58.829 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.824+0000 7f3e5a509b80 1 -- 172.21.15.86:0/2374019471 shutdown_connections 2024-03-23T17:55:58.829 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.824+0000 7f3e5a509b80 1 -- 172.21.15.86:0/2374019471 wait complete. 2024-03-23T17:55:58.829 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.825+0000 7f3e5a509b80 1 -- start start 2024-03-23T17:55:58.829 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.825+0000 7f3e5a509b80 1 Processor -- start 2024-03-23T17:55:58.829 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.825+0000 7f3e5a509b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55730b112100 0x55730b1239f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:58.829 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.825+0000 7f3e5a509b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55730b112a20 0x55730b123f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:58.829 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.825+0000 7f3e5a509b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55730b11d8d0 0x55730b17e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:58.829 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.825+0000 7f3e5a509b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55730b17e610 con 0x55730b112100 2024-03-23T17:55:58.829 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.825+0000 7f3e5a509b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55730b17e750 con 0x55730b112a20 2024-03-23T17:55:58.829 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.825+0000 7f3e5a509b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55730b17e890 con 0x55730b11d8d0 2024-03-23T17:55:58.829 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.825+0000 7f3e5a509b80 10 client.0 _ll_register_callbacks cb 0x55730b00a410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:55:58.829 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.826+0000 7f3e3f400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55730b112a20 0x55730b123f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:58.829 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.826+0000 7f3e3fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55730b112100 0x55730b1239f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:58.830 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.826+0000 7f3e3f400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55730b112a20 0x55730b123f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:56684/0 (socket says 172.21.15.86:56684) 2024-03-23T17:55:58.830 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.826+0000 7f3e3f400700 1 -- 172.21.15.86:0/3755101529 learned_addr learned my addr 172.21.15.86:0/3755101529 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:55:58.830 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.826+0000 7f3e44a00700 1 --2- 172.21.15.86:0/3755101529 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55730b11d8d0 0x55730b17e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:58.830 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.826+0000 7f3e3f400700 1 -- 172.21.15.86:0/3755101529 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55730b11d8d0 msgr2=0x55730b17e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:55:58.830 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.826+0000 7f3e3f400700 1 --2- 172.21.15.86:0/3755101529 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55730b11d8d0 0x55730b17e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:58.830 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.826+0000 7f3e3f400700 1 -- 172.21.15.86:0/3755101529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55730b112100 msgr2=0x55730b1239f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:55:58.830 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.826+0000 7f3e3f400700 1 --2- 172.21.15.86:0/3755101529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55730b112100 0x55730b1239f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:58.830 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.826+0000 7f3e3f400700 1 -- 172.21.15.86:0/3755101529 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e3009e040 con 0x55730b112a20 2024-03-23T17:55:58.830 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.826+0000 7f3e3fe00700 1 --2- 172.21.15.86:0/3755101529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55730b112100 0x55730b1239f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:55:58.830 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.826+0000 7f3e3f400700 1 --2- 172.21.15.86:0/3755101529 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55730b112a20 0x55730b123f30 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c004360 tx=0x7f3e2c026c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94713d6edb2df2e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:55:58.830 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.826+0000 7f3e29600700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T17:55:58.830 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.827+0000 7f3e29600700 1 -- 172.21.15.86:0/3755101529 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e2c02a070 con 0x55730b112a20 2024-03-23T17:55:58.830 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.827+0000 7f3e5a509b80 1 -- 172.21.15.86:0/3755101529 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3e2c003e70 con 0x55730b112a20 2024-03-23T17:55:58.830 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.827+0000 7f3e29600700 1 -- 172.21.15.86:0/3755101529 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f3e2c005230 con 0x55730b112a20 2024-03-23T17:55:58.831 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.827+0000 7f3e29600700 1 -- 172.21.15.86:0/3755101529 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3e2c0053f0 con 0x55730b112a20 2024-03-23T17:55:58.831 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.827+0000 7f3e5a509b80 10 client.26465 fetch_fsmap learned FSMap version 38 2024-03-23T17:55:58.831 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.827+0000 7f3e5a509b80 1 -- 172.21.15.86:0/3755101529 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55730aeeac70 con 0x55730b112a20 2024-03-23T17:55:58.831 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.827+0000 7f3e29600700 1 -- 172.21.15.86:0/3755101529 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e2c003e70 con 0x55730b112a20 2024-03-23T17:55:58.831 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.827+0000 7f3e29600700 1 -- 172.21.15.86:0/3755101529 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 38) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3e2c024020 con 0x55730b112a20 2024-03-23T17:55:58.831 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.827+0000 7f3e5a509b80 10 client.26465 fetch_fsmap finished waiting for FSMap version 38 2024-03-23T17:55:58.831 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.827+0000 7f3e5a509b80 10 client.26465 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:55:58.831 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.827+0000 7f3e5a509b80 20 client.26465 populate_metadata read hostname 'smithi086' 2024-03-23T17:55:58.831 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.827+0000 7f3e5a509b80 1 --2- 172.21.15.86:0/3755101529 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55730b17ff80 0x55730b1823a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:55:58.831 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.827+0000 7f3e5a509b80 4 client.26465 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T17:55:58.831 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.827+0000 7f3e5a509b80 1 -- 172.21.15.86:0/3755101529 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55730aeb3c90 con 0x55730b17ff80 2024-03-23T17:55:58.831 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.828+0000 7f3e3fe00700 1 --2- 172.21.15.86:0/3755101529 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55730b17ff80 0x55730b1823a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:55:58.832 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.828+0000 7f3e3fe00700 1 --2- 172.21.15.86:0/3755101529 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55730b17ff80 0x55730b1823a0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=184c64b59d9e578a server_cookie=bc62fc5e9bc45ee4 in_seq=0 out_seq=0 2024-03-23T17:55:58.832 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.828+0000 7f3e29600700 10 client.26465 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:55:58.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.868+0000 7f3e29600700 1 -- 172.21.15.86:0/3755101529 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55730aeb3c90 con 0x55730b17ff80 2024-03-23T17:55:58.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.868+0000 7f3e29600700 10 client.26465 handle_command_reply: tid=0 2024-03-23T17:55:58.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.868+0000 7f3e5a509b80 1 client.26465 shutdown 2024-03-23T17:55:58.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.869+0000 7f3e5a509b80 1 -- 172.21.15.86:0/3755101529 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55730b112a20 msgr2=0x55730b123f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:55:58.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.869+0000 7f3e5a509b80 1 --2- 172.21.15.86:0/3755101529 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55730b112a20 0x55730b123f30 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c004360 tx=0x7f3e2c026c30 comp rx=0 tx=0).stop 2024-03-23T17:55:58.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.869+0000 7f3e5a509b80 1 -- 172.21.15.86:0/3755101529 shutdown_connections 2024-03-23T17:55:58.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.869+0000 7f3e5a509b80 1 --2- 172.21.15.86:0/3755101529 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55730b17ff80 0x55730b1823a0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:58.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.869+0000 7f3e5a509b80 1 --2- 172.21.15.86:0/3755101529 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55730b11d8d0 0x55730b17e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:58.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.869+0000 7f3e5a509b80 1 --2- 172.21.15.86:0/3755101529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55730b112100 0x55730b1239f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:58.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.869+0000 7f3e5a509b80 1 --2- 172.21.15.86:0/3755101529 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55730b112a20 0x55730b123f30 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:58.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.869+0000 7f3e5a509b80 1 -- 172.21.15.86:0/3755101529 >> 172.21.15.86:0/3755101529 conn(0x55730b072640 msgr2=0x55730b10dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:55:58.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.869+0000 7f3e29600700 0 client.26465 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:55:58.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.869+0000 7f3e5a509b80 1 -- 172.21.15.86:0/3755101529 shutdown_connections 2024-03-23T17:55:58.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.869+0000 7f3e5a509b80 1 -- 172.21.15.86:0/3755101529 wait complete. 2024-03-23T17:55:58.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.869+0000 7f3e5a509b80 20 client.26465 trim_cache size 0 max 16384 2024-03-23T17:55:58.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-03-23T17:55:58.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-03-23T17:55:58.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-03-23T17:55:58.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-03-23T17:55:58.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.870+0000 7f3e46a00700 1 -- 172.21.15.86:0/4276068897 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f3e240a1290 msgr2=0x7f3e240a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:55:58.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.870+0000 7f3e46a00700 1 --2- 172.21.15.86:0/4276068897 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f3e240a1290 0x7f3e240a3750 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3e4019a8c0 tx=0x7f3e2c009260 comp rx=0 tx=0).stop 2024-03-23T17:55:58.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.870+0000 7f3e46a00700 1 -- 172.21.15.86:0/4276068897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e40101ec0 msgr2=0x7f3e40199370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:55:58.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.870+0000 7f3e46a00700 1 --2- 172.21.15.86:0/4276068897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e40101ec0 0x7f3e40199370 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f3e30005910 tx=0x7f3e30004730 comp rx=0 tx=0).stop 2024-03-23T17:55:58.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.870+0000 7f3e46a00700 1 -- 172.21.15.86:0/4276068897 shutdown_connections 2024-03-23T17:55:58.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.870+0000 7f3e46a00700 1 --2- 172.21.15.86:0/4276068897 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f3e240a1290 0x7f3e240a3750 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:58.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.870+0000 7f3e46a00700 1 --2- 172.21.15.86:0/4276068897 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3e4010b2e0 0x7f3e4019dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:58.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.870+0000 7f3e46a00700 1 --2- 172.21.15.86:0/4276068897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e40101ec0 0x7f3e40199370 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:58.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.870+0000 7f3e46a00700 1 --2- 172.21.15.86:0/4276068897 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e401027e0 0x7f3e401998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:55:58.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.870+0000 7f3e46a00700 1 -- 172.21.15.86:0/4276068897 >> 172.21.15.86:0/4276068897 conn(0x7f3e400fb680 msgr2=0x7f3e400fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:55:58.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.871+0000 7f3e46a00700 1 -- 172.21.15.86:0/4276068897 shutdown_connections 2024-03-23T17:55:58.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:55:58.871+0000 7f3e46a00700 1 -- 172.21.15.86:0/4276068897 wait complete. 2024-03-23T17:55:58.900 INFO:tasks.workunit.client.0.smithi086.stdout: 1 231168 5.61 MB/sec execute 143 sec latency 34.058 ms 2024-03-23T17:55:59.901 INFO:tasks.workunit.client.0.smithi086.stdout: 1 233504 5.66 MB/sec execute 144 sec latency 12.554 ms 2024-03-23T17:56:00.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:55:59 smithi086 ceph-mon[38221]: pgmap v163: 97 pgs: 97 active+clean; 91 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 212 op/s 2024-03-23T17:56:00.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:55:59 smithi092 ceph-mon[79089]: pgmap v163: 97 pgs: 97 active+clean; 91 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 212 op/s 2024-03-23T17:56:00.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:55:59 smithi152 ceph-mon[47949]: pgmap v163: 97 pgs: 97 active+clean; 91 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 212 op/s 2024-03-23T17:56:00.901 INFO:tasks.workunit.client.0.smithi086.stdout: 1 234836 5.67 MB/sec execute 145 sec latency 12.271 ms 2024-03-23T17:56:01.901 INFO:tasks.workunit.client.0.smithi086.stdout: 1 236549 5.68 MB/sec execute 146 sec latency 25.594 ms 2024-03-23T17:56:02.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:56:01 smithi086 ceph-mon[38221]: pgmap v164: 97 pgs: 97 active+clean; 89 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 332 op/s 2024-03-23T17:56:02.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:56:01 smithi092 ceph-mon[79089]: pgmap v164: 97 pgs: 97 active+clean; 89 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 332 op/s 2024-03-23T17:56:02.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:56:01 smithi152 ceph-mon[47949]: pgmap v164: 97 pgs: 97 active+clean; 89 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 332 op/s 2024-03-23T17:56:02.901 INFO:tasks.workunit.client.0.smithi086.stdout: 1 238124 5.72 MB/sec execute 147 sec latency 10.167 ms 2024-03-23T17:56:03.901 INFO:tasks.workunit.client.0.smithi086.stdout: 1 240125 5.73 MB/sec execute 148 sec latency 23.289 ms 2024-03-23T17:56:04.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:56:03 smithi086 ceph-mon[38221]: pgmap v165: 97 pgs: 97 active+clean; 89 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 212 op/s 2024-03-23T17:56:04.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:56:03 smithi092 ceph-mon[79089]: pgmap v165: 97 pgs: 97 active+clean; 89 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 212 op/s 2024-03-23T17:56:04.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:56:03 smithi152 ceph-mon[47949]: pgmap v165: 97 pgs: 97 active+clean; 89 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 212 op/s 2024-03-23T17:56:04.901 INFO:tasks.workunit.client.0.smithi086.stdout: 1 241664 5.77 MB/sec execute 149 sec latency 14.207 ms 2024-03-23T17:56:05.901 INFO:tasks.workunit.client.0.smithi086.stdout: 1 243694 5.78 MB/sec execute 150 sec latency 11.105 ms 2024-03-23T17:56:06.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:56:05 smithi086 ceph-mon[38221]: pgmap v166: 97 pgs: 97 active+clean; 96 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 362 op/s 2024-03-23T17:56:06.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:56:05 smithi092 ceph-mon[79089]: pgmap v166: 97 pgs: 97 active+clean; 96 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 362 op/s 2024-03-23T17:56:06.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:56:05 smithi152 ceph-mon[47949]: pgmap v166: 97 pgs: 97 active+clean; 96 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 362 op/s 2024-03-23T17:56:06.901 INFO:tasks.workunit.client.0.smithi086.stdout: 1 245217 5.82 MB/sec execute 151 sec latency 11.415 ms 2024-03-23T17:56:07.901 INFO:tasks.workunit.client.0.smithi086.stdout: 1 247254 5.83 MB/sec execute 152 sec latency 15.241 ms 2024-03-23T17:56:08.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:56:07 smithi086 ceph-mon[38221]: pgmap v167: 97 pgs: 97 active+clean; 96 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 269 op/s 2024-03-23T17:56:08.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:56:07 smithi092 ceph-mon[79089]: pgmap v167: 97 pgs: 97 active+clean; 96 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 269 op/s 2024-03-23T17:56:08.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:56:07 smithi152 ceph-mon[47949]: pgmap v167: 97 pgs: 97 active+clean; 96 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 269 op/s 2024-03-23T17:56:08.901 INFO:tasks.workunit.client.0.smithi086.stdout: 1 248767 5.87 MB/sec execute 153 sec latency 45.720 ms 2024-03-23T17:56:09.901 INFO:tasks.workunit.client.0.smithi086.stdout: 1 250887 5.88 MB/sec execute 154 sec latency 12.767 ms 2024-03-23T17:56:10.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:56:09 smithi086 ceph-mon[38221]: pgmap v168: 97 pgs: 97 active+clean; 96 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 269 op/s 2024-03-23T17:56:10.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:56:09 smithi092 ceph-mon[79089]: pgmap v168: 97 pgs: 97 active+clean; 96 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 269 op/s 2024-03-23T17:56:10.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:56:09 smithi152 ceph-mon[47949]: pgmap v168: 97 pgs: 97 active+clean; 96 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 269 op/s 2024-03-23T17:56:10.901 INFO:tasks.workunit.client.0.smithi086.stdout: 1 252384 5.92 MB/sec execute 155 sec latency 12.800 ms 2024-03-23T17:56:11.901 INFO:tasks.workunit.client.0.smithi086.stdout: 1 252473 5.88 MB/sec execute 156 sec latency 925.462 ms 2024-03-23T17:56:12.024 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.023+0000 7f343ce00700 1 -- 172.21.15.86:0/3999204462 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34380731e0 msgr2=0x7f34380735c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:56:12.024 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.023+0000 7f343ce00700 1 --2- 172.21.15.86:0/3999204462 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34380731e0 0x7f34380735c0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f3428009990 tx=0x7f3428009ca0 comp rx=0 tx=0).stop 2024-03-23T17:56:12.024 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.023+0000 7f343ce00700 1 -- 172.21.15.86:0/3999204462 shutdown_connections 2024-03-23T17:56:12.024 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.023+0000 7f343ce00700 1 --2- 172.21.15.86:0/3999204462 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f343810d4c0 0x7f343810f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:12.024 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.023+0000 7f343ce00700 1 --2- 172.21.15.86:0/3999204462 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34380731e0 0x7f34380735c0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:12.024 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.023+0000 7f343ce00700 1 --2- 172.21.15.86:0/3999204462 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3438073b00 0x7f343810cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:12.024 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.023+0000 7f343ce00700 1 -- 172.21.15.86:0/3999204462 >> 172.21.15.86:0/3999204462 conn(0x7f34380fc750 msgr2=0x7f34380feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:56:12.025 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.023+0000 7f343ce00700 1 -- 172.21.15.86:0/3999204462 shutdown_connections 2024-03-23T17:56:12.025 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.024+0000 7f343ce00700 1 -- 172.21.15.86:0/3999204462 wait complete. 2024-03-23T17:56:12.027 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.024+0000 7f343ce00700 1 Processor -- start 2024-03-23T17:56:12.027 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.025+0000 7f343ce00700 1 -- start start 2024-03-23T17:56:12.027 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.025+0000 7f343ce00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34380731e0 0x7f3438199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:56:12.027 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.025+0000 7f343ce00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3438073b00 0x7f34381998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:56:12.027 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.025+0000 7f343ce00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f343810d4c0 0x7f343819dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:56:12.027 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.025+0000 7f343ce00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f343819e280 con 0x7f343810d4c0 2024-03-23T17:56:12.027 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.025+0000 7f343ce00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f343819e3f0 con 0x7f34380731e0 2024-03-23T17:56:12.028 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.025+0000 7f343ce00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f343819e5b0 con 0x7f3438073b00 2024-03-23T17:56:12.028 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.025+0000 7f3436800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f343810d4c0 0x7f343819dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:56:12.028 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.025+0000 7f3435e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34380731e0 0x7f3438199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:56:12.028 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.025+0000 7f3436800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f343810d4c0 0x7f343819dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50196/0 (socket says 172.21.15.86:50196) 2024-03-23T17:56:12.028 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.025+0000 7f3436800700 1 -- 172.21.15.86:0/3793548235 learned_addr learned my addr 172.21.15.86:0/3793548235 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:56:12.028 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.025+0000 7f3435400700 1 --2- 172.21.15.86:0/3793548235 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3438073b00 0x7f34381998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:56:12.028 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.026+0000 7f3436800700 1 -- 172.21.15.86:0/3793548235 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3438073b00 msgr2=0x7f34381998a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:56:12.028 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.026+0000 7f3436800700 1 --2- 172.21.15.86:0/3793548235 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3438073b00 0x7f34381998a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:12.028 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.026+0000 7f3436800700 1 -- 172.21.15.86:0/3793548235 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34380731e0 msgr2=0x7f3438199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:56:12.028 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.026+0000 7f3436800700 1 --2- 172.21.15.86:0/3793548235 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34380731e0 0x7f3438199360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:12.028 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.026+0000 7f3436800700 1 -- 172.21.15.86:0/3793548235 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34280095f0 con 0x7f343810d4c0 2024-03-23T17:56:12.028 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.026+0000 7f3436800700 1 --2- 172.21.15.86:0/3793548235 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f343810d4c0 0x7f343819dc40 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f343000ef30 tx=0x7f343000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:56:12.028 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.026+0000 7f342ea00700 1 -- 172.21.15.86:0/3793548235 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3430018070 con 0x7f343810d4c0 2024-03-23T17:56:12.029 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.026+0000 7f343ce00700 1 -- 172.21.15.86:0/3793548235 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f343819e830 con 0x7f343810d4c0 2024-03-23T17:56:12.029 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.026+0000 7f342ea00700 1 -- 172.21.15.86:0/3793548235 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f343000f040 con 0x7f343810d4c0 2024-03-23T17:56:12.029 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.026+0000 7f342ea00700 1 -- 172.21.15.86:0/3793548235 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f343000cd00 con 0x7f343810d4c0 2024-03-23T17:56:12.029 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.027+0000 7f343ce00700 1 -- 172.21.15.86:0/3793548235 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34381a60d0 con 0x7f343810d4c0 2024-03-23T17:56:12.030 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.029+0000 7f342ea00700 1 -- 172.21.15.86:0/3793548235 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f3430003a00 con 0x7f343810d4c0 2024-03-23T17:56:12.031 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.031+0000 7f342ea00700 1 --2- 172.21.15.86:0/3793548235 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f341c0a1290 0x7f341c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:56:12.031 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.031+0000 7f345089eb80 1 Processor -- start 2024-03-23T17:56:12.031 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.031+0000 7f342ea00700 1 -- 172.21.15.86:0/3793548235 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f34300cc7a0 con 0x7f343810d4c0 2024-03-23T17:56:12.031 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.031+0000 7f3435e00700 1 --2- 172.21.15.86:0/3793548235 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f341c0a1290 0x7f341c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:56:12.031 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.031+0000 7f345089eb80 1 -- start start 2024-03-23T17:56:12.034 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.032+0000 7f345089eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e4e500100 0x564e4e5004e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:56:12.034 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.032+0000 7f345089eb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564e4e500a20 0x564e4e50b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:56:12.034 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.032+0000 7f345089eb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564e4e50b8d0 0x564e4e50dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:56:12.034 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.032+0000 7f345089eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e4e459930 con 0x564e4e500100 2024-03-23T17:56:12.034 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.032+0000 7f345089eb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e4e4572f0 con 0x564e4e500a20 2024-03-23T17:56:12.034 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.032+0000 7f345089eb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e4e452be0 con 0x564e4e50b8d0 2024-03-23T17:56:12.034 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.032+0000 7f3435e00700 1 --2- 172.21.15.86:0/3793548235 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f341c0a1290 0x7f341c0a3750 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3428006010 tx=0x7f342800b260 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:56:12.034 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.032+0000 7f3435e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e4e500100 0x564e4e5004e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:56:12.034 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.032+0000 7f3435e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e4e500100 0x564e4e5004e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50208/0 (socket says 172.21.15.86:50208) 2024-03-23T17:56:12.035 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.032+0000 7f3435e00700 1 -- 172.21.15.86:0/2586226533 learned_addr learned my addr 172.21.15.86:0/2586226533 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:56:12.035 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.032+0000 7f3435400700 1 --2- 172.21.15.86:0/2586226533 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564e4e500a20 0x564e4e50b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:56:12.035 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.032+0000 7f3436800700 1 --2- 172.21.15.86:0/2586226533 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564e4e50b8d0 0x564e4e50dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:56:12.035 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.032+0000 7f3435e00700 1 -- 172.21.15.86:0/2586226533 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564e4e50b8d0 msgr2=0x564e4e50dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:56:12.035 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.032+0000 7f3435e00700 1 --2- 172.21.15.86:0/2586226533 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564e4e50b8d0 0x564e4e50dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:12.035 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.032+0000 7f3435e00700 1 -- 172.21.15.86:0/2586226533 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564e4e500a20 msgr2=0x564e4e50b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:56:12.035 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.032+0000 7f3435e00700 1 --2- 172.21.15.86:0/2586226533 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564e4e500a20 0x564e4e50b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:12.035 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.032+0000 7f3435e00700 1 -- 172.21.15.86:0/2586226533 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e4e2d4350 con 0x564e4e500100 2024-03-23T17:56:12.035 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.032+0000 7f3436800700 1 --2- 172.21.15.86:0/2586226533 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564e4e50b8d0 0x564e4e50dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:56:12.035 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.033+0000 7f3435e00700 1 --2- 172.21.15.86:0/2586226533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e4e500100 0x564e4e5004e0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f3428005640 tx=0x7f3428020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c60f5c951cd8782 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:56:12.035 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.033+0000 7f3423400700 1 -- 172.21.15.86:0/2586226533 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f342800bec0 con 0x564e4e500100 2024-03-23T17:56:12.035 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.033+0000 7f3423400700 1 -- 172.21.15.86:0/2586226533 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f342801e040 con 0x564e4e500100 2024-03-23T17:56:12.035 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.033+0000 7f3423400700 1 -- 172.21.15.86:0/2586226533 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f342800f8b0 con 0x564e4e500100 2024-03-23T17:56:12.036 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.033+0000 7f345089eb80 1 -- 172.21.15.86:0/2586226533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e4e500100 msgr2=0x564e4e5004e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:56:12.036 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.033+0000 7f345089eb80 1 --2- 172.21.15.86:0/2586226533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e4e500100 0x564e4e5004e0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f3428005640 tx=0x7f3428020000 comp rx=0 tx=0).stop 2024-03-23T17:56:12.036 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.034+0000 7f345089eb80 1 -- 172.21.15.86:0/2586226533 shutdown_connections 2024-03-23T17:56:12.036 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.034+0000 7f345089eb80 1 --2- 172.21.15.86:0/2586226533 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564e4e50b8d0 0x564e4e50dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:12.036 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.034+0000 7f345089eb80 1 --2- 172.21.15.86:0/2586226533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e4e500100 0x564e4e5004e0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:12.036 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.034+0000 7f345089eb80 1 --2- 172.21.15.86:0/2586226533 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564e4e500a20 0x564e4e50b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:12.036 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.034+0000 7f345089eb80 1 -- 172.21.15.86:0/2586226533 >> 172.21.15.86:0/2586226533 conn(0x564e4e460640 msgr2=0x564e4e4fc520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:56:12.036 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.034+0000 7f345089eb80 1 -- 172.21.15.86:0/2586226533 shutdown_connections 2024-03-23T17:56:12.036 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.034+0000 7f345089eb80 1 -- 172.21.15.86:0/2586226533 wait complete. 2024-03-23T17:56:12.040 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.034+0000 7f345089eb80 1 -- start start 2024-03-23T17:56:12.040 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.035+0000 7f345089eb80 1 Processor -- start 2024-03-23T17:56:12.040 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.035+0000 7f345089eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e4e500100 0x564e4e50b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:56:12.040 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.035+0000 7f345089eb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564e4e500a20 0x564e4e508550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:56:12.040 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.035+0000 7f345089eb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564e4e50b8d0 0x564e4e508b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:56:12.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.035+0000 7f345089eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e4e3eed20 con 0x564e4e500100 2024-03-23T17:56:12.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.035+0000 7f345089eb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e4e3ee9c0 con 0x564e4e50b8d0 2024-03-23T17:56:12.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.035+0000 7f345089eb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e4e2d6640 con 0x564e4e500a20 2024-03-23T17:56:12.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.035+0000 7f345089eb80 10 client.0 _ll_register_callbacks cb 0x564e4e3f8410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:56:12.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.036+0000 7f3435e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e4e500100 0x564e4e50b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:56:12.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.036+0000 7f3435e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e4e500100 0x564e4e50b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50222/0 (socket says 172.21.15.86:50222) 2024-03-23T17:56:12.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.036+0000 7f3435e00700 1 -- 172.21.15.86:0/3108636086 learned_addr learned my addr 172.21.15.86:0/3108636086 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:56:12.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.036+0000 7f3435400700 1 --2- 172.21.15.86:0/3108636086 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564e4e500a20 0x564e4e508550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:56:12.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.036+0000 7f3436800700 1 --2- 172.21.15.86:0/3108636086 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564e4e50b8d0 0x564e4e508b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:56:12.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.036+0000 7f3435e00700 1 -- 172.21.15.86:0/3108636086 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564e4e500a20 msgr2=0x564e4e508550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:56:12.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.036+0000 7f3435e00700 1 --2- 172.21.15.86:0/3108636086 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564e4e500a20 0x564e4e508550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:12.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.036+0000 7f3435e00700 1 -- 172.21.15.86:0/3108636086 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564e4e50b8d0 msgr2=0x564e4e508b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:56:12.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.036+0000 7f3435e00700 1 --2- 172.21.15.86:0/3108636086 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564e4e50b8d0 0x564e4e508b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:12.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.036+0000 7f3435e00700 1 -- 172.21.15.86:0/3108636086 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34381105f0 con 0x564e4e500100 2024-03-23T17:56:12.042 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.036+0000 7f3435e00700 1 --2- 172.21.15.86:0/3108636086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e4e500100 0x564e4e50b260 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f3428005a00 tx=0x7f342800b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dcde53df9d35e25 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:56:12.042 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.036+0000 7f3417e00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-23T17:56:12.042 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.037+0000 7f3417e00700 1 -- 172.21.15.86:0/3108636086 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f342800f770 con 0x564e4e500100 2024-03-23T17:56:12.042 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.037+0000 7f3417e00700 1 -- 172.21.15.86:0/3108636086 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f3428004e70 con 0x564e4e500100 2024-03-23T17:56:12.042 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.037+0000 7f345089eb80 1 -- 172.21.15.86:0/3108636086 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564e4e413320 con 0x564e4e500100 2024-03-23T17:56:12.042 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.037+0000 7f3417e00700 1 -- 172.21.15.86:0/3108636086 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f342800f8d0 con 0x564e4e500100 2024-03-23T17:56:12.042 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.037+0000 7f345089eb80 10 client.16428 fetch_fsmap learned FSMap version 38 2024-03-23T17:56:12.042 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.037+0000 7f345089eb80 1 -- 172.21.15.86:0/3108636086 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x564e4e280b00 con 0x564e4e500100 2024-03-23T17:56:12.042 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.037+0000 7f3417e00700 1 -- 172.21.15.86:0/3108636086 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564e4e413320 con 0x564e4e500100 2024-03-23T17:56:12.042 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.037+0000 7f3417e00700 1 -- 172.21.15.86:0/3108636086 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 38) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3428030950 con 0x564e4e500100 2024-03-23T17:56:12.042 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.037+0000 7f345089eb80 10 client.16428 fetch_fsmap finished waiting for FSMap version 38 2024-03-23T17:56:12.042 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.037+0000 7f345089eb80 10 client.16428 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:56:12.042 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.037+0000 7f345089eb80 20 client.16428 populate_metadata read hostname 'smithi086' 2024-03-23T17:56:12.043 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.037+0000 7f345089eb80 1 --2- 172.21.15.86:0/3108636086 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x564e4e51b110 0x564e4e51d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:56:12.043 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.037+0000 7f345089eb80 4 client.16428 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:56:12.043 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.037+0000 7f345089eb80 1 -- 172.21.15.86:0/3108636086 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564e4e44b6a0 con 0x564e4e51b110 2024-03-23T17:56:12.043 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.038+0000 7f3435400700 1 --2- 172.21.15.86:0/3108636086 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x564e4e51b110 0x564e4e51d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:56:12.043 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.038+0000 7f3417e00700 10 client.16428 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:56:12.043 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.038+0000 7f3435400700 1 --2- 172.21.15.86:0/3108636086 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x564e4e51b110 0x564e4e51d530 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1b61650ec115e6d server_cookie=a98051eda90a47b in_seq=0 out_seq=0 2024-03-23T17:56:12.043 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.039+0000 7f3417e00700 1 -- 172.21.15.86:0/3108636086 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x564e4e44b6a0 con 0x564e4e51b110 2024-03-23T17:56:12.043 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.039+0000 7f3417e00700 10 client.16428 handle_command_reply: tid=0 2024-03-23T17:56:12.043 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.039+0000 7f345089eb80 1 client.16428 shutdown 2024-03-23T17:56:12.043 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.039+0000 7f345089eb80 1 -- 172.21.15.86:0/3108636086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e4e500100 msgr2=0x564e4e50b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:56:12.043 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.039+0000 7f345089eb80 1 --2- 172.21.15.86:0/3108636086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e4e500100 0x564e4e50b260 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f3428005a00 tx=0x7f342800b3b0 comp rx=0 tx=0).stop 2024-03-23T17:56:12.043 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.039+0000 7f345089eb80 1 -- 172.21.15.86:0/3108636086 shutdown_connections 2024-03-23T17:56:12.043 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.039+0000 7f345089eb80 1 --2- 172.21.15.86:0/3108636086 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x564e4e51b110 0x564e4e51d530 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:12.043 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.039+0000 7f345089eb80 1 --2- 172.21.15.86:0/3108636086 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564e4e50b8d0 0x564e4e508b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:12.044 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.039+0000 7f345089eb80 1 --2- 172.21.15.86:0/3108636086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e4e500100 0x564e4e50b260 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:12.044 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.039+0000 7f3417e00700 0 client.16428 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:56:12.044 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.039+0000 7f345089eb80 1 --2- 172.21.15.86:0/3108636086 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564e4e500a20 0x564e4e508550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:12.044 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.039+0000 7f345089eb80 1 -- 172.21.15.86:0/3108636086 >> 172.21.15.86:0/3108636086 conn(0x564e4e460640 msgr2=0x564e4e5020a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:56:12.044 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.040+0000 7f345089eb80 1 -- 172.21.15.86:0/3108636086 shutdown_connections 2024-03-23T17:56:12.044 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.040+0000 7f345089eb80 1 -- 172.21.15.86:0/3108636086 wait complete. 2024-03-23T17:56:12.044 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.040+0000 7f345089eb80 20 client.16428 trim_cache size 0 max 16384 2024-03-23T17:56:12.055 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.054+0000 7f345089eb80 1 Processor -- start 2024-03-23T17:56:12.055 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.054+0000 7f345089eb80 1 -- start start 2024-03-23T17:56:12.055 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.055+0000 7f345089eb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564e4e500100 0x564e4e512540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:56:12.055 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.055+0000 7f345089eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e4e500a20 0x564e4e4f6500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:56:12.055 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.055+0000 7f345089eb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564e4e50b8d0 0x564e4e4fc460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:56:12.055 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.055+0000 7f345089eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e4e4fd8b0 con 0x564e4e500a20 2024-03-23T17:56:12.055 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.055+0000 7f345089eb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e4e4f8400 con 0x564e4e500100 2024-03-23T17:56:12.055 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.055+0000 7f345089eb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e4e508ec0 con 0x564e4e50b8d0 2024-03-23T17:56:12.055 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.055+0000 7f3435400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e4e500a20 0x564e4e4f6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:56:12.056 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.055+0000 7f3435e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564e4e500100 0x564e4e512540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:56:12.056 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.055+0000 7f3435400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e4e500a20 0x564e4e4f6500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50232/0 (socket says 172.21.15.86:50232) 2024-03-23T17:56:12.056 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.055+0000 7f3435400700 1 -- 172.21.15.86:0/1703134493 learned_addr learned my addr 172.21.15.86:0/1703134493 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:56:12.056 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.055+0000 7f3436800700 1 --2- 172.21.15.86:0/1703134493 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564e4e50b8d0 0x564e4e4fc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:56:12.060 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.055+0000 7f3435400700 1 -- 172.21.15.86:0/1703134493 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564e4e50b8d0 msgr2=0x564e4e4fc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:56:12.060 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.055+0000 7f3435400700 1 --2- 172.21.15.86:0/1703134493 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564e4e50b8d0 0x564e4e4fc460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:12.060 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.055+0000 7f3435400700 1 -- 172.21.15.86:0/1703134493 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564e4e500100 msgr2=0x564e4e512540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:56:12.060 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.055+0000 7f3435400700 1 --2- 172.21.15.86:0/1703134493 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564e4e500100 0x564e4e512540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:12.060 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.055+0000 7f3435400700 1 -- 172.21.15.86:0/1703134493 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e4e418e40 con 0x564e4e500a20 2024-03-23T17:56:12.060 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.055+0000 7f3435e00700 1 --2- 172.21.15.86:0/1703134493 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564e4e500100 0x564e4e512540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:56:12.060 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.056+0000 7f3435400700 1 --2- 172.21.15.86:0/1703134493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e4e500a20 0x564e4e4f6500 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x564e4e509a90 tx=0x7f3424009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84b69a62cdcb5ba5 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:56:12.060 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.056+0000 7f3423e00700 1 -- 172.21.15.86:0/1703134493 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3424017030 con 0x564e4e500a20 2024-03-23T17:56:12.060 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.056+0000 7f3423e00700 1 -- 172.21.15.86:0/1703134493 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3424018040 con 0x564e4e500a20 2024-03-23T17:56:12.060 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.056+0000 7f3423e00700 1 -- 172.21.15.86:0/1703134493 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f342401ee60 con 0x564e4e500a20 2024-03-23T17:56:12.061 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.056+0000 7f345089eb80 1 -- 172.21.15.86:0/1703134493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e4e500a20 msgr2=0x564e4e4f6500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:56:12.061 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.056+0000 7f345089eb80 1 --2- 172.21.15.86:0/1703134493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e4e500a20 0x564e4e4f6500 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x564e4e509a90 tx=0x7f3424009260 comp rx=0 tx=0).stop 2024-03-23T17:56:12.061 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.057+0000 7f345089eb80 1 -- 172.21.15.86:0/1703134493 shutdown_connections 2024-03-23T17:56:12.061 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.057+0000 7f345089eb80 1 --2- 172.21.15.86:0/1703134493 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564e4e50b8d0 0x564e4e4fc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:12.061 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.057+0000 7f345089eb80 1 --2- 172.21.15.86:0/1703134493 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564e4e500100 0x564e4e512540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:12.061 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.057+0000 7f345089eb80 1 --2- 172.21.15.86:0/1703134493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e4e500a20 0x564e4e4f6500 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:12.061 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.057+0000 7f345089eb80 1 -- 172.21.15.86:0/1703134493 >> 172.21.15.86:0/1703134493 conn(0x564e4e460640 msgr2=0x564e4e4fbd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:56:12.061 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.057+0000 7f345089eb80 1 -- 172.21.15.86:0/1703134493 shutdown_connections 2024-03-23T17:56:12.061 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.057+0000 7f345089eb80 1 -- 172.21.15.86:0/1703134493 wait complete. 2024-03-23T17:56:12.061 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.057+0000 7f345089eb80 1 -- start start 2024-03-23T17:56:12.061 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.057+0000 7f345089eb80 1 Processor -- start 2024-03-23T17:56:12.061 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.058+0000 7f345089eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e4e500100 0x564e4e510710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:56:12.061 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.058+0000 7f345089eb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564e4e500a20 0x564e4e510c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:56:12.061 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.058+0000 7f345089eb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564e4e50b8d0 0x564e4e565bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:56:12.062 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.058+0000 7f345089eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e4e566170 con 0x564e4e500100 2024-03-23T17:56:12.062 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.058+0000 7f345089eb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e4e5662e0 con 0x564e4e500a20 2024-03-23T17:56:12.062 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.058+0000 7f345089eb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e4e5664a0 con 0x564e4e50b8d0 2024-03-23T17:56:12.062 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.058+0000 7f345089eb80 10 client.0 _ll_register_callbacks cb 0x564e4e3f8410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:56:12.062 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.058+0000 7f3435e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e4e500100 0x564e4e510710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:56:12.062 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.058+0000 7f3435e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e4e500100 0x564e4e510710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50234/0 (socket says 172.21.15.86:50234) 2024-03-23T17:56:12.062 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.058+0000 7f3435e00700 1 -- 172.21.15.86:0/3846374352 learned_addr learned my addr 172.21.15.86:0/3846374352 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:56:12.062 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.058+0000 7f3435400700 1 --2- 172.21.15.86:0/3846374352 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564e4e500a20 0x564e4e510c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:56:12.062 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.058+0000 7f3436800700 1 --2- 172.21.15.86:0/3846374352 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564e4e50b8d0 0x564e4e565bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:56:12.062 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.058+0000 7f3435e00700 1 -- 172.21.15.86:0/3846374352 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564e4e50b8d0 msgr2=0x564e4e565bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:56:12.062 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.058+0000 7f3435e00700 1 --2- 172.21.15.86:0/3846374352 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564e4e50b8d0 0x564e4e565bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:12.062 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.058+0000 7f3435e00700 1 -- 172.21.15.86:0/3846374352 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564e4e500a20 msgr2=0x564e4e510c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:56:12.062 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.058+0000 7f3435e00700 1 --2- 172.21.15.86:0/3846374352 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564e4e500a20 0x564e4e510c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:12.062 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.058+0000 7f3435e00700 1 -- 172.21.15.86:0/3846374352 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564e4e280b00 con 0x564e4e500100 2024-03-23T17:56:12.063 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.058+0000 7f3436800700 1 --2- 172.21.15.86:0/3846374352 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564e4e50b8d0 0x564e4e565bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:56:12.063 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.058+0000 7f3435e00700 1 --2- 172.21.15.86:0/3846374352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e4e500100 0x564e4e510710 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f3428009990 tx=0x7f342800bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=883070eba7cb7dd0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:56:12.063 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.058+0000 7f3417e00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-23T17:56:12.063 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.059+0000 7f3417e00700 1 -- 172.21.15.86:0/3846374352 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3428030860 con 0x564e4e500100 2024-03-23T17:56:12.063 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.059+0000 7f345089eb80 1 -- 172.21.15.86:0/3846374352 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f342801e040 con 0x564e4e500100 2024-03-23T17:56:12.063 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.059+0000 7f3417e00700 1 -- 172.21.15.86:0/3846374352 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f34280309c0 con 0x564e4e500100 2024-03-23T17:56:12.063 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.059+0000 7f3417e00700 1 -- 172.21.15.86:0/3846374352 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3428039650 con 0x564e4e500100 2024-03-23T17:56:12.063 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.059+0000 7f345089eb80 10 client.16434 fetch_fsmap learned FSMap version 38 2024-03-23T17:56:12.063 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.059+0000 7f345089eb80 1 -- 172.21.15.86:0/3846374352 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x7f3424009b30 con 0x564e4e500100 2024-03-23T17:56:12.063 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.059+0000 7f3417e00700 1 -- 172.21.15.86:0/3846374352 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f342801e040 con 0x564e4e500100 2024-03-23T17:56:12.063 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.059+0000 7f3417e00700 1 -- 172.21.15.86:0/3846374352 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 38) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f342800f960 con 0x564e4e500100 2024-03-23T17:56:12.063 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.059+0000 7f345089eb80 10 client.16434 fetch_fsmap finished waiting for FSMap version 38 2024-03-23T17:56:12.063 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.059+0000 7f345089eb80 10 client.16434 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:56:12.063 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.059+0000 7f345089eb80 20 client.16434 populate_metadata read hostname 'smithi086' 2024-03-23T17:56:12.064 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.059+0000 7f345089eb80 1 --2- 172.21.15.86:0/3846374352 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x564e4e56d890 0x564e4e56fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:56:12.064 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.059+0000 7f345089eb80 4 client.16434 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T17:56:12.064 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.059+0000 7f345089eb80 1 -- 172.21.15.86:0/3846374352 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564e4e2a1c90 con 0x564e4e56d890 2024-03-23T17:56:12.064 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.060+0000 7f3435400700 1 --2- 172.21.15.86:0/3846374352 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x564e4e56d890 0x564e4e56fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:56:12.064 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.060+0000 7f3435400700 1 --2- 172.21.15.86:0/3846374352 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x564e4e56d890 0x564e4e56fcb0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=519688d99d8c35a8 server_cookie=dfd33929c9527a38 in_seq=0 out_seq=0 2024-03-23T17:56:12.064 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:12.060+0000 7f3417e00700 10 client.16434 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:56:12.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:56:11 smithi086 ceph-mon[38221]: pgmap v169: 97 pgs: 97 active+clean; 102 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.6 MiB/s wr, 390 op/s 2024-03-23T17:56:12.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:56:11 smithi092 ceph-mon[79089]: pgmap v169: 97 pgs: 97 active+clean; 102 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.6 MiB/s wr, 390 op/s 2024-03-23T17:56:12.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:56:11 smithi152 ceph-mon[47949]: pgmap v169: 97 pgs: 97 active+clean; 102 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.6 MiB/s wr, 390 op/s 2024-03-23T17:56:12.901 INFO:tasks.workunit.client.0.smithi086.stdout: 1 252473 5.84 MB/sec execute 157 sec latency 1925.563 ms 2024-03-23T17:56:13.901 INFO:tasks.workunit.client.0.smithi086.stdout: 1 252473 5.81 MB/sec execute 158 sec latency 2925.640 ms 2024-03-23T17:56:14.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:56:13 smithi086 ceph-mon[38221]: pgmap v170: 97 pgs: 97 active+clean; 102 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 270 op/s 2024-03-23T17:56:14.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:56:13 smithi092 ceph-mon[79089]: pgmap v170: 97 pgs: 97 active+clean; 102 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 270 op/s 2024-03-23T17:56:14.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:56:13 smithi152 ceph-mon[47949]: pgmap v170: 97 pgs: 97 active+clean; 102 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 270 op/s 2024-03-23T17:56:14.902 INFO:tasks.workunit.client.0.smithi086.stdout: 1 252473 5.77 MB/sec execute 159 sec latency 3925.803 ms 2024-03-23T17:56:15.643 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:15.643+0000 7f3417e00700 1 -- 172.21.15.86:0/3846374352 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564e4e2a1c90 con 0x564e4e56d890 2024-03-23T17:56:15.643 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:15.643+0000 7f3417e00700 10 client.16434 handle_command_reply: tid=0 2024-03-23T17:56:15.643 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:15.643+0000 7f345089eb80 1 client.16434 shutdown 2024-03-23T17:56:15.644 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:15.643+0000 7f345089eb80 1 -- 172.21.15.86:0/3846374352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e4e500100 msgr2=0x564e4e510710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:56:15.644 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:15.643+0000 7f345089eb80 1 --2- 172.21.15.86:0/3846374352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e4e500100 0x564e4e510710 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f3428009990 tx=0x7f342800bd90 comp rx=0 tx=0).stop 2024-03-23T17:56:15.644 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:15.643+0000 7f345089eb80 1 -- 172.21.15.86:0/3846374352 shutdown_connections 2024-03-23T17:56:15.644 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:15.643+0000 7f345089eb80 1 --2- 172.21.15.86:0/3846374352 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x564e4e56d890 0x564e4e56fcb0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:15.644 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:15.643+0000 7f345089eb80 1 --2- 172.21.15.86:0/3846374352 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564e4e50b8d0 0x564e4e565bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:15.644 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:15.643+0000 7f345089eb80 1 --2- 172.21.15.86:0/3846374352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e4e500100 0x564e4e510710 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:15.645 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:15.643+0000 7f345089eb80 1 --2- 172.21.15.86:0/3846374352 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564e4e500a20 0x564e4e510c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:15.645 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:15.643+0000 7f345089eb80 1 -- 172.21.15.86:0/3846374352 >> 172.21.15.86:0/3846374352 conn(0x564e4e460640 msgr2=0x564e4e4fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:56:15.645 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:15.643+0000 7f3417e00700 0 client.16434 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:56:15.645 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:15.644+0000 7f345089eb80 1 -- 172.21.15.86:0/3846374352 shutdown_connections 2024-03-23T17:56:15.645 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:15.644+0000 7f345089eb80 1 -- 172.21.15.86:0/3846374352 wait complete. 2024-03-23T17:56:15.645 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:15.644+0000 7f345089eb80 20 client.16434 trim_cache size 0 max 16384 2024-03-23T17:56:15.645 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-03-23T17:56:15.645 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-03-23T17:56:15.645 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-03-23T17:56:15.645 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-03-23T17:56:15.646 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:15.645+0000 7f343ce00700 1 -- 172.21.15.86:0/3793548235 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f341c0a1290 msgr2=0x7f341c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:56:15.646 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:15.645+0000 7f343ce00700 1 --2- 172.21.15.86:0/3793548235 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f341c0a1290 0x7f341c0a3750 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3428006010 tx=0x7f342800b260 comp rx=0 tx=0).stop 2024-03-23T17:56:15.646 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:15.645+0000 7f343ce00700 1 -- 172.21.15.86:0/3793548235 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f343810d4c0 msgr2=0x7f343819dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:56:15.646 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:15.645+0000 7f343ce00700 1 --2- 172.21.15.86:0/3793548235 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f343810d4c0 0x7f343819dc40 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f343000ef30 tx=0x7f343000c3a0 comp rx=0 tx=0).stop 2024-03-23T17:56:15.646 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:15.645+0000 7f343ce00700 1 -- 172.21.15.86:0/3793548235 shutdown_connections 2024-03-23T17:56:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:15.645+0000 7f343ce00700 1 --2- 172.21.15.86:0/3793548235 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f343810d4c0 0x7f343819dc40 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:15.645+0000 7f343ce00700 1 --2- 172.21.15.86:0/3793548235 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34380731e0 0x7f3438199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:15.645+0000 7f343ce00700 1 --2- 172.21.15.86:0/3793548235 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f341c0a1290 0x7f341c0a3750 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:15.645+0000 7f343ce00700 1 --2- 172.21.15.86:0/3793548235 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3438073b00 0x7f34381998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:15.645+0000 7f343ce00700 1 -- 172.21.15.86:0/3793548235 >> 172.21.15.86:0/3793548235 conn(0x7f34380fc750 msgr2=0x7f34380fd520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:56:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:15.646+0000 7f343ce00700 1 -- 172.21.15.86:0/3793548235 shutdown_connections 2024-03-23T17:56:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:15.646+0000 7f343ce00700 1 -- 172.21.15.86:0/3793548235 wait complete. 2024-03-23T17:56:15.902 INFO:tasks.workunit.client.0.smithi086.stdout: 1 252858 5.74 MB/sec execute 160 sec latency 4675.164 ms 2024-03-23T17:56:16.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:56:15 smithi086 ceph-mon[38221]: pgmap v171: 97 pgs: 97 active+clean; 111 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.9 MiB/s wr, 341 op/s 2024-03-23T17:56:16.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:56:15 smithi092 ceph-mon[79089]: pgmap v171: 97 pgs: 97 active+clean; 111 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.9 MiB/s wr, 341 op/s 2024-03-23T17:56:16.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:56:15 smithi152 ceph-mon[47949]: pgmap v171: 97 pgs: 97 active+clean; 111 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.9 MiB/s wr, 341 op/s 2024-03-23T17:56:16.902 INFO:tasks.workunit.client.0.smithi086.stdout: 1 254866 5.78 MB/sec execute 161 sec latency 15.101 ms 2024-03-23T17:56:17.902 INFO:tasks.workunit.client.0.smithi086.stdout: 1 256142 5.78 MB/sec execute 162 sec latency 19.744 ms 2024-03-23T17:56:18.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:56:17 smithi086 ceph-mon[38221]: pgmap v172: 97 pgs: 97 active+clean; 111 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 191 op/s 2024-03-23T17:56:18.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:56:17 smithi092 ceph-mon[79089]: pgmap v172: 97 pgs: 97 active+clean; 111 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 191 op/s 2024-03-23T17:56:18.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:56:17 smithi152 ceph-mon[47949]: pgmap v172: 97 pgs: 97 active+clean; 111 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 191 op/s 2024-03-23T17:56:18.902 INFO:tasks.workunit.client.0.smithi086.stdout: 1 257776 5.79 MB/sec execute 163 sec latency 67.460 ms 2024-03-23T17:56:19.902 INFO:tasks.workunit.client.0.smithi086.stdout: 1 259338 5.83 MB/sec execute 164 sec latency 14.837 ms 2024-03-23T17:56:20.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:56:19 smithi086 ceph-mon[38221]: pgmap v173: 97 pgs: 97 active+clean; 111 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 191 op/s 2024-03-23T17:56:20.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:56:19 smithi092 ceph-mon[79089]: pgmap v173: 97 pgs: 97 active+clean; 111 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 191 op/s 2024-03-23T17:56:20.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:56:19 smithi152 ceph-mon[47949]: pgmap v173: 97 pgs: 97 active+clean; 111 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 191 op/s 2024-03-23T17:56:20.902 INFO:tasks.workunit.client.0.smithi086.stdout: 1 260830 5.82 MB/sec execute 165 sec latency 13.232 ms 2024-03-23T17:56:21.902 INFO:tasks.workunit.client.0.smithi086.stdout: 1 262790 5.87 MB/sec execute 166 sec latency 14.842 ms 2024-03-23T17:56:22.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:56:21 smithi086 ceph-mon[38221]: pgmap v174: 97 pgs: 97 active+clean; 108 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.6 MiB/s wr, 294 op/s 2024-03-23T17:56:22.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:56:21 smithi092 ceph-mon[79089]: pgmap v174: 97 pgs: 97 active+clean; 108 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.6 MiB/s wr, 294 op/s 2024-03-23T17:56:22.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:56:21 smithi152 ceph-mon[47949]: pgmap v174: 97 pgs: 97 active+clean; 108 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.6 MiB/s wr, 294 op/s 2024-03-23T17:56:22.902 INFO:tasks.workunit.client.0.smithi086.stdout: 1 264007 5.87 MB/sec execute 167 sec latency 12.896 ms 2024-03-23T17:56:23.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:56:23 smithi086 ceph-mon[38221]: pgmap v175: 97 pgs: 97 active+clean; 108 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 980 KiB/s rd, 2.2 MiB/s wr, 173 op/s 2024-03-23T17:56:23.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:56:23 smithi092 ceph-mon[79089]: pgmap v175: 97 pgs: 97 active+clean; 108 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 980 KiB/s rd, 2.2 MiB/s wr, 173 op/s 2024-03-23T17:56:23.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:56:23 smithi152 ceph-mon[47949]: pgmap v175: 97 pgs: 97 active+clean; 108 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 980 KiB/s rd, 2.2 MiB/s wr, 173 op/s 2024-03-23T17:56:23.902 INFO:tasks.workunit.client.0.smithi086.stdout: 1 265491 5.88 MB/sec execute 168 sec latency 19.165 ms 2024-03-23T17:56:24.902 INFO:tasks.workunit.client.0.smithi086.stdout: 1 266800 5.88 MB/sec execute 169 sec latency 13.738 ms 2024-03-23T17:56:25.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:56:25 smithi092 ceph-mon[79089]: pgmap v176: 97 pgs: 97 active+clean; 108 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 276 op/s 2024-03-23T17:56:25.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:56:25 smithi152 ceph-mon[47949]: pgmap v176: 97 pgs: 97 active+clean; 108 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 276 op/s 2024-03-23T17:56:25.902 INFO:tasks.workunit.client.0.smithi086.stdout: 1 268934 5.91 MB/sec execute 170 sec latency 23.671 ms 2024-03-23T17:56:26.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:56:25 smithi086 ceph-mon[38221]: pgmap v176: 97 pgs: 97 active+clean; 108 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 276 op/s 2024-03-23T17:56:26.807 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:56:26 smithi092 ceph-mon[79089]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T17:56:26.835 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:56:26 smithi152 ceph-mon[47949]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T17:56:26.902 INFO:tasks.workunit.client.0.smithi086.stdout: 1 270195 5.92 MB/sec execute 171 sec latency 12.195 ms 2024-03-23T17:56:27.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:56:26 smithi086 ceph-mon[38221]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T17:56:27.902 INFO:tasks.workunit.client.0.smithi086.stdout: 1 271445 5.92 MB/sec execute 172 sec latency 16.984 ms 2024-03-23T17:56:28.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:56:27 smithi092 ceph-mon[79089]: pgmap v177: 97 pgs: 97 active+clean; 108 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 205 op/s 2024-03-23T17:56:28.085 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:56:27 smithi152 ceph-mon[47949]: pgmap v177: 97 pgs: 97 active+clean; 108 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 205 op/s 2024-03-23T17:56:28.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:56:27 smithi086 ceph-mon[38221]: pgmap v177: 97 pgs: 97 active+clean; 108 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 205 op/s 2024-03-23T17:56:28.811 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.810+0000 7fcb59c00700 1 -- 172.21.15.86:0/312149465 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb54100ef0 msgr2=0x7fcb541012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:56:28.811 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.810+0000 7fcb59c00700 1 --2- 172.21.15.86:0/312149465 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb54100ef0 0x7fcb541012d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fcb44009990 tx=0x7fcb44009ca0 comp rx=0 tx=0).stop 2024-03-23T17:56:28.811 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.810+0000 7fcb59c00700 1 -- 172.21.15.86:0/312149465 shutdown_connections 2024-03-23T17:56:28.811 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.810+0000 7fcb59c00700 1 --2- 172.21.15.86:0/312149465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb54106010 0x7fcb5410db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:28.811 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.810+0000 7fcb59c00700 1 --2- 172.21.15.86:0/312149465 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb54100ef0 0x7fcb541012d0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:28.811 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.810+0000 7fcb59c00700 1 --2- 172.21.15.86:0/312149465 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcb541018a0 0x7fcb541058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:28.811 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.810+0000 7fcb59c00700 1 -- 172.21.15.86:0/312149465 >> 172.21.15.86:0/312149465 conn(0x7fcb540fc750 msgr2=0x7fcb540feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:56:28.811 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.810+0000 7fcb59c00700 1 -- 172.21.15.86:0/312149465 shutdown_connections 2024-03-23T17:56:28.811 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.810+0000 7fcb59c00700 1 -- 172.21.15.86:0/312149465 wait complete. 2024-03-23T17:56:28.812 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.812+0000 7fcb59c00700 1 Processor -- start 2024-03-23T17:56:28.812 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.812+0000 7fcb59c00700 1 -- start start 2024-03-23T17:56:28.813 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.812+0000 7fcb59c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb54100ef0 0x7fcb54199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:56:28.813 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.812+0000 7fcb59c00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcb54106010 0x7fcb541998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:56:28.813 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.812+0000 7fcb59c00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb5419dc00 0x7fcb5419e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:56:28.814 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.812+0000 7fcb59c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5419e680 con 0x7fcb54100ef0 2024-03-23T17:56:28.814 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.813+0000 7fcb59c00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5419e7f0 con 0x7fcb5419dc00 2024-03-23T17:56:28.814 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.813+0000 7fcb59c00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5419e9b0 con 0x7fcb54106010 2024-03-23T17:56:28.814 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.813+0000 7fcb52000700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcb54106010 0x7fcb541998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:56:28.814 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.813+0000 7fcb52a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb54100ef0 0x7fcb54199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:56:28.814 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.813+0000 7fcb53400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb5419dc00 0x7fcb5419e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:56:28.814 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.813+0000 7fcb53400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb5419dc00 0x7fcb5419e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:60404/0 (socket says 172.21.15.86:60404) 2024-03-23T17:56:28.814 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.813+0000 7fcb53400700 1 -- 172.21.15.86:0/3308628738 learned_addr learned my addr 172.21.15.86:0/3308628738 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:56:28.814 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.813+0000 7fcb52000700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcb54106010 0x7fcb541998d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:40072/0 (socket says 172.21.15.86:40072) 2024-03-23T17:56:28.814 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.813+0000 7fcb52000700 1 -- 172.21.15.86:0/3308628738 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb5419dc00 msgr2=0x7fcb5419e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:56:28.814 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.813+0000 7fcb52000700 1 --2- 172.21.15.86:0/3308628738 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb5419dc00 0x7fcb5419e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:28.814 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.813+0000 7fcb52000700 1 -- 172.21.15.86:0/3308628738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb54100ef0 msgr2=0x7fcb54199390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:56:28.815 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.813+0000 7fcb52000700 1 --2- 172.21.15.86:0/3308628738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb54100ef0 0x7fcb54199390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:28.815 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.813+0000 7fcb52000700 1 -- 172.21.15.86:0/3308628738 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb440095f0 con 0x7fcb54106010 2024-03-23T17:56:28.815 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.813+0000 7fcb53400700 1 --2- 172.21.15.86:0/3308628738 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb5419dc00 0x7fcb5419e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:56:28.815 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.813+0000 7fcb52a00700 1 --2- 172.21.15.86:0/3308628738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb54100ef0 0x7fcb54199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:56:28.815 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.814+0000 7fcb52000700 1 --2- 172.21.15.86:0/3308628738 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcb54106010 0x7fcb541998d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fcb4000e9c0 tx=0x7fcb4000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:56:28.815 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.814+0000 7fcb4b400700 1 -- 172.21.15.86:0/3308628738 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4000c9c0 con 0x7fcb54106010 2024-03-23T17:56:28.816 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.814+0000 7fcb59c00700 1 -- 172.21.15.86:0/3308628738 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb541a5f10 con 0x7fcb54106010 2024-03-23T17:56:28.816 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.814+0000 7fcb4b400700 1 -- 172.21.15.86:0/3308628738 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb40010430 con 0x7fcb54106010 2024-03-23T17:56:28.816 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.814+0000 7fcb4b400700 1 -- 172.21.15.86:0/3308628738 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb40018680 con 0x7fcb54106010 2024-03-23T17:56:28.816 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.814+0000 7fcb59c00700 1 -- 172.21.15.86:0/3308628738 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb541a6390 con 0x7fcb54106010 2024-03-23T17:56:28.817 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.817+0000 7fcb4b400700 1 -- 172.21.15.86:0/3308628738 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fcb4000cb20 con 0x7fcb54106010 2024-03-23T17:56:28.819 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.818+0000 7fcb4b400700 1 --2- 172.21.15.86:0/3308628738 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fcb380a1290 0x7fcb380a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:56:28.819 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.819+0000 7fcb6d6c4b80 1 Processor -- start 2024-03-23T17:56:28.819 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.819+0000 7fcb4b400700 1 -- 172.21.15.86:0/3308628738 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fcb40014070 con 0x7fcb54106010 2024-03-23T17:56:28.819 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.819+0000 7fcb52a00700 1 --2- 172.21.15.86:0/3308628738 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fcb380a1290 0x7fcb380a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:56:28.819 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.819+0000 7fcb6d6c4b80 1 -- start start 2024-03-23T17:56:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.819+0000 7fcb6d6c4b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ec4e0f100 0x558ec4e0f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:56:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.819+0000 7fcb52a00700 1 --2- 172.21.15.86:0/3308628738 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fcb380a1290 0x7fcb380a3750 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fcb44006010 tx=0x7fcb4400b260 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:56:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.819+0000 7fcb6d6c4b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558ec4e0fa20 0x558ec4e1a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:56:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.819+0000 7fcb6d6c4b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ec4e1a8d0 0x558ec4e1ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:56:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.819+0000 7fcb6d6c4b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ec4d68930 con 0x558ec4e0f100 2024-03-23T17:56:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.819+0000 7fcb6d6c4b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ec4d662f0 con 0x558ec4e1a8d0 2024-03-23T17:56:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.819+0000 7fcb6d6c4b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ec4d61be0 con 0x558ec4e0fa20 2024-03-23T17:56:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.820+0000 7fcb52a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ec4e0f100 0x558ec4e0f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:56:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.820+0000 7fcb53400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ec4e1a8d0 0x558ec4e1ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:56:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.820+0000 7fcb53400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ec4e1a8d0 0x558ec4e1ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:60414/0 (socket says 172.21.15.86:60414) 2024-03-23T17:56:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.820+0000 7fcb53400700 1 -- 172.21.15.86:0/1399834820 learned_addr learned my addr 172.21.15.86:0/1399834820 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:56:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.820+0000 7fcb52000700 1 --2- 172.21.15.86:0/1399834820 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558ec4e0fa20 0x558ec4e1a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:56:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.820+0000 7fcb53400700 1 -- 172.21.15.86:0/1399834820 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558ec4e0fa20 msgr2=0x558ec4e1a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:56:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.820+0000 7fcb53400700 1 --2- 172.21.15.86:0/1399834820 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558ec4e0fa20 0x558ec4e1a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.820+0000 7fcb53400700 1 -- 172.21.15.86:0/1399834820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ec4e0f100 msgr2=0x558ec4e0f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:56:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.820+0000 7fcb53400700 1 --2- 172.21.15.86:0/1399834820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ec4e0f100 0x558ec4e0f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.820+0000 7fcb53400700 1 -- 172.21.15.86:0/1399834820 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558ec4be3350 con 0x558ec4e1a8d0 2024-03-23T17:56:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.820+0000 7fcb52000700 1 --2- 172.21.15.86:0/1399834820 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558ec4e0fa20 0x558ec4e1a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:56:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.820+0000 7fcb52a00700 1 --2- 172.21.15.86:0/1399834820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ec4e0f100 0x558ec4e0f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:56:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.820+0000 7fcb53400700 1 --2- 172.21.15.86:0/1399834820 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ec4e1a8d0 0x558ec4e1ccc0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c009500 tx=0x7fcb4c009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa7ac5571a7b4c35 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:56:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.821+0000 7fcb3fe00700 1 -- 172.21.15.86:0/1399834820 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c00e900 con 0x558ec4e1a8d0 2024-03-23T17:56:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.821+0000 7fcb3fe00700 1 -- 172.21.15.86:0/1399834820 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb4c00f040 con 0x558ec4e1a8d0 2024-03-23T17:56:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.821+0000 7fcb3fe00700 1 -- 172.21.15.86:0/1399834820 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c017d90 con 0x558ec4e1a8d0 2024-03-23T17:56:28.827 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.821+0000 7fcb6d6c4b80 1 -- 172.21.15.86:0/1399834820 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ec4e1a8d0 msgr2=0x558ec4e1ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:56:28.827 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.821+0000 7fcb6d6c4b80 1 --2- 172.21.15.86:0/1399834820 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ec4e1a8d0 0x558ec4e1ccc0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c009500 tx=0x7fcb4c009920 comp rx=0 tx=0).stop 2024-03-23T17:56:28.827 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.821+0000 7fcb6d6c4b80 1 -- 172.21.15.86:0/1399834820 shutdown_connections 2024-03-23T17:56:28.827 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.821+0000 7fcb6d6c4b80 1 --2- 172.21.15.86:0/1399834820 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ec4e1a8d0 0x558ec4e1ccc0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:28.827 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.821+0000 7fcb6d6c4b80 1 --2- 172.21.15.86:0/1399834820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ec4e0f100 0x558ec4e0f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:28.827 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.821+0000 7fcb6d6c4b80 1 --2- 172.21.15.86:0/1399834820 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558ec4e0fa20 0x558ec4e1a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:28.827 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.821+0000 7fcb6d6c4b80 1 -- 172.21.15.86:0/1399834820 >> 172.21.15.86:0/1399834820 conn(0x558ec4d6f640 msgr2=0x558ec4e0b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:56:28.828 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.821+0000 7fcb6d6c4b80 1 -- 172.21.15.86:0/1399834820 shutdown_connections 2024-03-23T17:56:28.828 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.821+0000 7fcb6d6c4b80 1 -- 172.21.15.86:0/1399834820 wait complete. 2024-03-23T17:56:28.828 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.822+0000 7fcb6d6c4b80 1 -- start start 2024-03-23T17:56:28.828 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.822+0000 7fcb6d6c4b80 1 Processor -- start 2024-03-23T17:56:28.828 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.823+0000 7fcb6d6c4b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ec4e0f100 0x558ec4e1a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:56:28.828 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.823+0000 7fcb6d6c4b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558ec4e0fa20 0x558ec4e176c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:56:28.828 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.823+0000 7fcb6d6c4b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ec4e1a8d0 0x558ec4e17cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:56:28.828 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.823+0000 7fcb6d6c4b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ec4cfdd20 con 0x558ec4e0f100 2024-03-23T17:56:28.829 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.823+0000 7fcb6d6c4b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ec4cfd9c0 con 0x558ec4e1a8d0 2024-03-23T17:56:28.829 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.823+0000 7fcb6d6c4b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ec4be5640 con 0x558ec4e0fa20 2024-03-23T17:56:28.829 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.823+0000 7fcb6d6c4b80 10 client.0 _ll_register_callbacks cb 0x558ec4d07410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:56:28.829 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.823+0000 7fcb52a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ec4e0f100 0x558ec4e1a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:56:28.829 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.823+0000 7fcb52000700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558ec4e0fa20 0x558ec4e176c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:56:28.829 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.823+0000 7fcb52a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ec4e0f100 0x558ec4e1a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52720/0 (socket says 172.21.15.86:52720) 2024-03-23T17:56:28.829 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.823+0000 7fcb52a00700 1 -- 172.21.15.86:0/3451976537 learned_addr learned my addr 172.21.15.86:0/3451976537 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:56:28.829 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.823+0000 7fcb53400700 1 --2- 172.21.15.86:0/3451976537 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ec4e1a8d0 0x558ec4e17cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:56:28.829 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.824+0000 7fcb52000700 1 -- 172.21.15.86:0/3451976537 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ec4e1a8d0 msgr2=0x558ec4e17cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:56:28.829 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.824+0000 7fcb52000700 1 --2- 172.21.15.86:0/3451976537 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ec4e1a8d0 0x558ec4e17cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:28.829 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.824+0000 7fcb52000700 1 -- 172.21.15.86:0/3451976537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ec4e0f100 msgr2=0x558ec4e1a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:56:28.829 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.824+0000 7fcb52000700 1 --2- 172.21.15.86:0/3451976537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ec4e0f100 0x558ec4e1a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:28.829 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.824+0000 7fcb52000700 1 -- 172.21.15.86:0/3451976537 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb4c0091e0 con 0x558ec4e0fa20 2024-03-23T17:56:28.830 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.824+0000 7fcb52000700 1 --2- 172.21.15.86:0/3451976537 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558ec4e0fa20 0x558ec4e176c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcb400ca510 tx=0x7fcb4009a650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b7137dd9e518072 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:56:28.830 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.824+0000 7fcb3cc00700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-03-23T17:56:28.830 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.824+0000 7fcb3cc00700 1 -- 172.21.15.86:0/3451976537 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb400a2350 con 0x558ec4e0fa20 2024-03-23T17:56:28.830 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.824+0000 7fcb3cc00700 1 -- 172.21.15.86:0/3451976537 <== mon.1 v2:172.21.15.152:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fcb40010860 con 0x558ec4e0fa20 2024-03-23T17:56:28.830 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.824+0000 7fcb6d6c4b80 1 -- 172.21.15.86:0/3451976537 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558ec4d22320 con 0x558ec4e0fa20 2024-03-23T17:56:28.830 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.824+0000 7fcb3cc00700 1 -- 172.21.15.86:0/3451976537 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb400b82c0 con 0x558ec4e0fa20 2024-03-23T17:56:28.830 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.824+0000 7fcb6d6c4b80 10 client.26485 fetch_fsmap learned FSMap version 39 2024-03-23T17:56:28.830 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.824+0000 7fcb6d6c4b80 1 -- 172.21.15.86:0/3451976537 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x558ec4b8fb00 con 0x558ec4e0fa20 2024-03-23T17:56:28.830 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.825+0000 7fcb3cc00700 1 -- 172.21.15.86:0/3451976537 <== mon.1 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb40063020 con 0x558ec4e0fa20 2024-03-23T17:56:28.830 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.825+0000 7fcb3cc00700 1 -- 172.21.15.86:0/3451976537 <== mon.1 v2:172.21.15.152:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fcb400a1ec0 con 0x558ec4e0fa20 2024-03-23T17:56:28.830 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.825+0000 7fcb6d6c4b80 10 client.26485 fetch_fsmap finished waiting for FSMap version 39 2024-03-23T17:56:28.830 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.825+0000 7fcb6d6c4b80 10 client.26485 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:56:28.830 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.825+0000 7fcb6d6c4b80 20 client.26485 populate_metadata read hostname 'smithi086' 2024-03-23T17:56:28.831 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.825+0000 7fcb6d6c4b80 1 --2- 172.21.15.86:0/3451976537 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x558ec4e2a4e0 0x558ec4e2c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:56:28.831 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.825+0000 7fcb6d6c4b80 4 client.26485 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:56:28.831 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.825+0000 7fcb6d6c4b80 1 -- 172.21.15.86:0/3451976537 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558ec4d5a6a0 con 0x558ec4e2a4e0 2024-03-23T17:56:28.831 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.825+0000 7fcb52a00700 1 --2- 172.21.15.86:0/3451976537 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x558ec4e2a4e0 0x558ec4e2c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:56:28.831 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.826+0000 7fcb3cc00700 10 client.26485 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:56:28.831 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.826+0000 7fcb52a00700 1 --2- 172.21.15.86:0/3451976537 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x558ec4e2a4e0 0x558ec4e2c900 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c78440985954892 server_cookie=ce2fa5effbb32b5c in_seq=0 out_seq=0 2024-03-23T17:56:28.831 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.826+0000 7fcb3cc00700 1 -- 172.21.15.86:0/3451976537 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x558ec4d5a6a0 con 0x558ec4e2a4e0 2024-03-23T17:56:28.831 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.826+0000 7fcb3cc00700 10 client.26485 handle_command_reply: tid=0 2024-03-23T17:56:28.831 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.826+0000 7fcb6d6c4b80 1 client.26485 shutdown 2024-03-23T17:56:28.831 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.826+0000 7fcb6d6c4b80 1 -- 172.21.15.86:0/3451976537 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558ec4e0fa20 msgr2=0x558ec4e176c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:56:28.831 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.826+0000 7fcb6d6c4b80 1 --2- 172.21.15.86:0/3451976537 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558ec4e0fa20 0x558ec4e176c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcb400ca510 tx=0x7fcb4009a650 comp rx=0 tx=0).stop 2024-03-23T17:56:28.831 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.827+0000 7fcb6d6c4b80 1 -- 172.21.15.86:0/3451976537 shutdown_connections 2024-03-23T17:56:28.831 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.827+0000 7fcb6d6c4b80 1 --2- 172.21.15.86:0/3451976537 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x558ec4e2a4e0 0x558ec4e2c900 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:28.832 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.827+0000 7fcb6d6c4b80 1 --2- 172.21.15.86:0/3451976537 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ec4e1a8d0 0x558ec4e17cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:28.832 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.827+0000 7fcb6d6c4b80 1 --2- 172.21.15.86:0/3451976537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ec4e0f100 0x558ec4e1a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:28.832 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.827+0000 7fcb3cc00700 0 client.26485 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:56:28.832 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.827+0000 7fcb6d6c4b80 1 --2- 172.21.15.86:0/3451976537 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558ec4e0fa20 0x558ec4e176c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:28.832 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.827+0000 7fcb6d6c4b80 1 -- 172.21.15.86:0/3451976537 >> 172.21.15.86:0/3451976537 conn(0x558ec4d6f640 msgr2=0x558ec4e110a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:56:28.832 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.827+0000 7fcb6d6c4b80 1 -- 172.21.15.86:0/3451976537 shutdown_connections 2024-03-23T17:56:28.832 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.827+0000 7fcb6d6c4b80 1 -- 172.21.15.86:0/3451976537 wait complete. 2024-03-23T17:56:28.832 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.827+0000 7fcb6d6c4b80 20 client.26485 trim_cache size 0 max 16384 2024-03-23T17:56:28.849 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.849+0000 7fcb6d6c4b80 1 Processor -- start 2024-03-23T17:56:28.849 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.849+0000 7fcb6d6c4b80 1 -- start start 2024-03-23T17:56:28.849 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.849+0000 7fcb6d6c4b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ec4e0f100 0x558ec4e215e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:56:28.849 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.849+0000 7fcb6d6c4b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ec4e0fa20 0x558ec4e05500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:56:28.850 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.849+0000 7fcb6d6c4b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558ec4e1a8d0 0x558ec4e11040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:56:28.850 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.849+0000 7fcb6d6c4b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ec4e07400 con 0x558ec4e0f100 2024-03-23T17:56:28.850 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.849+0000 7fcb6d6c4b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ec4e31620 con 0x558ec4e0fa20 2024-03-23T17:56:28.850 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.849+0000 7fcb6d6c4b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ec4e10bc0 con 0x558ec4e1a8d0 2024-03-23T17:56:28.851 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.850+0000 7fcb53400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558ec4e1a8d0 0x558ec4e11040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:56:28.851 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.850+0000 7fcb53400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558ec4e1a8d0 0x558ec4e11040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:40090/0 (socket says 172.21.15.86:40090) 2024-03-23T17:56:28.851 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.850+0000 7fcb53400700 1 -- 172.21.15.86:0/3562474849 learned_addr learned my addr 172.21.15.86:0/3562474849 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:56:28.851 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.850+0000 7fcb52a00700 1 --2- 172.21.15.86:0/3562474849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ec4e0f100 0x558ec4e215e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:56:28.851 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.850+0000 7fcb52000700 1 --2- 172.21.15.86:0/3562474849 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ec4e0fa20 0x558ec4e05500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:56:28.851 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.850+0000 7fcb53400700 1 -- 172.21.15.86:0/3562474849 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ec4e0fa20 msgr2=0x558ec4e05500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:56:28.851 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.850+0000 7fcb53400700 1 --2- 172.21.15.86:0/3562474849 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ec4e0fa20 0x558ec4e05500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:28.851 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.850+0000 7fcb53400700 1 -- 172.21.15.86:0/3562474849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ec4e0f100 msgr2=0x558ec4e215e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:56:28.851 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.850+0000 7fcb53400700 1 --2- 172.21.15.86:0/3562474849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ec4e0f100 0x558ec4e215e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:28.851 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.850+0000 7fcb53400700 1 -- 172.21.15.86:0/3562474849 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558ec4d27e40 con 0x558ec4e1a8d0 2024-03-23T17:56:28.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.851+0000 7fcb53400700 1 --2- 172.21.15.86:0/3562474849 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558ec4e1a8d0 0x558ec4e11040 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c009500 tx=0x7fcb4c009650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b56fa96395f7a88f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:56:28.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.851+0000 7fcb48c00700 1 -- 172.21.15.86:0/3562474849 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c003c80 con 0x558ec4e1a8d0 2024-03-23T17:56:28.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.851+0000 7fcb48c00700 1 -- 172.21.15.86:0/3562474849 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb4c003de0 con 0x558ec4e1a8d0 2024-03-23T17:56:28.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.851+0000 7fcb48c00700 1 -- 172.21.15.86:0/3562474849 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c0056d0 con 0x558ec4e1a8d0 2024-03-23T17:56:28.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.852+0000 7fcb6d6c4b80 1 -- 172.21.15.86:0/3562474849 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558ec4e1a8d0 msgr2=0x558ec4e11040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:56:28.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.852+0000 7fcb6d6c4b80 1 --2- 172.21.15.86:0/3562474849 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558ec4e1a8d0 0x558ec4e11040 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c009500 tx=0x7fcb4c009650 comp rx=0 tx=0).stop 2024-03-23T17:56:28.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.852+0000 7fcb6d6c4b80 1 -- 172.21.15.86:0/3562474849 shutdown_connections 2024-03-23T17:56:28.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.852+0000 7fcb6d6c4b80 1 --2- 172.21.15.86:0/3562474849 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558ec4e1a8d0 0x558ec4e11040 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:28.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.852+0000 7fcb6d6c4b80 1 --2- 172.21.15.86:0/3562474849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ec4e0f100 0x558ec4e215e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:28.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.852+0000 7fcb6d6c4b80 1 --2- 172.21.15.86:0/3562474849 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ec4e0fa20 0x558ec4e05500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:28.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.852+0000 7fcb6d6c4b80 1 -- 172.21.15.86:0/3562474849 >> 172.21.15.86:0/3562474849 conn(0x558ec4d6f640 msgr2=0x558ec4e15f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:56:28.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.852+0000 7fcb6d6c4b80 1 -- 172.21.15.86:0/3562474849 shutdown_connections 2024-03-23T17:56:28.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.852+0000 7fcb6d6c4b80 1 -- 172.21.15.86:0/3562474849 wait complete. 2024-03-23T17:56:28.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.853+0000 7fcb6d6c4b80 1 -- start start 2024-03-23T17:56:28.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.853+0000 7fcb6d6c4b80 1 Processor -- start 2024-03-23T17:56:28.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.853+0000 7fcb6d6c4b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ec4e0f100 0x558ec4e21030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:56:28.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.853+0000 7fcb6d6c4b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558ec4e0fa20 0x558ec4e21570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:56:28.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.853+0000 7fcb6d6c4b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ec4e1a8d0 0x558ec4e1f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:56:28.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.853+0000 7fcb6d6c4b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ec4e1fed0 con 0x558ec4e1a8d0 2024-03-23T17:56:28.855 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.853+0000 7fcb6d6c4b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ec4e20040 con 0x558ec4e0f100 2024-03-23T17:56:28.855 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.853+0000 7fcb6d6c4b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ec4e20200 con 0x558ec4e0fa20 2024-03-23T17:56:28.855 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.853+0000 7fcb6d6c4b80 10 client.0 _ll_register_callbacks cb 0x558ec4d07410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:56:28.855 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.854+0000 7fcb52a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ec4e0f100 0x558ec4e21030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:56:28.855 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.854+0000 7fcb52000700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558ec4e0fa20 0x558ec4e21570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:56:28.855 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.854+0000 7fcb52a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ec4e0f100 0x558ec4e21030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:60430/0 (socket says 172.21.15.86:60430) 2024-03-23T17:56:28.855 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.854+0000 7fcb52a00700 1 -- 172.21.15.86:0/145870103 learned_addr learned my addr 172.21.15.86:0/145870103 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:56:28.855 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.854+0000 7fcb53400700 1 --2- 172.21.15.86:0/145870103 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ec4e1a8d0 0x558ec4e1f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:56:28.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.854+0000 7fcb52a00700 1 -- 172.21.15.86:0/145870103 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558ec4e0fa20 msgr2=0x558ec4e21570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:56:28.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.854+0000 7fcb52a00700 1 --2- 172.21.15.86:0/145870103 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558ec4e0fa20 0x558ec4e21570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:28.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.854+0000 7fcb52a00700 1 -- 172.21.15.86:0/145870103 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ec4e1a8d0 msgr2=0x558ec4e1f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:56:28.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.854+0000 7fcb52a00700 1 --2- 172.21.15.86:0/145870103 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ec4e1a8d0 0x558ec4e1f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:28.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.854+0000 7fcb52a00700 1 -- 172.21.15.86:0/145870103 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558ec4be3350 con 0x558ec4e0f100 2024-03-23T17:56:28.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.854+0000 7fcb52000700 1 --2- 172.21.15.86:0/145870103 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558ec4e0fa20 0x558ec4e21570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:56:28.858 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.854+0000 7fcb52a00700 1 --2- 172.21.15.86:0/145870103 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ec4e0f100 0x558ec4e21030 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fcb4400bb70 tx=0x7fcb44009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=266590a11261f01 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:56:28.858 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.854+0000 7fcb3cc00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T17:56:28.858 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.854+0000 7fcb3cc00700 1 -- 172.21.15.86:0/145870103 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb44034280 con 0x558ec4e0f100 2024-03-23T17:56:28.858 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.855+0000 7fcb6d6c4b80 1 -- 172.21.15.86:0/145870103 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcb4c00f040 con 0x558ec4e0f100 2024-03-23T17:56:28.858 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.855+0000 7fcb3cc00700 1 -- 172.21.15.86:0/145870103 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fcb440343e0 con 0x558ec4e0f100 2024-03-23T17:56:28.858 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.855+0000 7fcb3cc00700 1 -- 172.21.15.86:0/145870103 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb440048e0 con 0x558ec4e0f100 2024-03-23T17:56:28.858 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.855+0000 7fcb6d6c4b80 10 client.26501 fetch_fsmap learned FSMap version 39 2024-03-23T17:56:28.858 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.855+0000 7fcb6d6c4b80 1 -- 172.21.15.86:0/145870103 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x558ec4be7c70 con 0x558ec4e0f100 2024-03-23T17:56:28.858 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.855+0000 7fcb3cc00700 1 -- 172.21.15.86:0/145870103 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb4c00f040 con 0x558ec4e0f100 2024-03-23T17:56:28.858 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.855+0000 7fcb3cc00700 1 -- 172.21.15.86:0/145870103 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fcb4400f460 con 0x558ec4e0f100 2024-03-23T17:56:28.858 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.855+0000 7fcb6d6c4b80 10 client.26501 fetch_fsmap finished waiting for FSMap version 39 2024-03-23T17:56:28.858 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.855+0000 7fcb6d6c4b80 10 client.26501 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:56:28.859 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.855+0000 7fcb6d6c4b80 20 client.26501 populate_metadata read hostname 'smithi086' 2024-03-23T17:56:28.859 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.855+0000 7fcb6d6c4b80 1 --2- 172.21.15.86:0/145870103 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x558ec4e7d3a0 0x558ec4e7f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:56:28.859 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.855+0000 7fcb6d6c4b80 4 client.26501 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T17:56:28.859 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.855+0000 7fcb6d6c4b80 1 -- 172.21.15.86:0/145870103 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558ec4bb0c90 con 0x558ec4e7d3a0 2024-03-23T17:56:28.859 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.856+0000 7fcb52000700 1 --2- 172.21.15.86:0/145870103 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x558ec4e7d3a0 0x558ec4e7f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:56:28.859 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.856+0000 7fcb52000700 1 --2- 172.21.15.86:0/145870103 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x558ec4e7d3a0 0x558ec4e7f7c0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32f190c856eb2b03 server_cookie=27d2e3b16d7a51f5 in_seq=0 out_seq=0 2024-03-23T17:56:28.859 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.856+0000 7fcb3cc00700 10 client.26501 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:56:28.897 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.896+0000 7fcb3cc00700 1 -- 172.21.15.86:0/145870103 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558ec4bb0c90 con 0x558ec4e7d3a0 2024-03-23T17:56:28.897 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.896+0000 7fcb3cc00700 10 client.26501 handle_command_reply: tid=0 2024-03-23T17:56:28.897 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.896+0000 7fcb6d6c4b80 1 client.26501 shutdown 2024-03-23T17:56:28.897 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.896+0000 7fcb6d6c4b80 1 -- 172.21.15.86:0/145870103 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ec4e0f100 msgr2=0x558ec4e21030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:56:28.897 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.896+0000 7fcb6d6c4b80 1 --2- 172.21.15.86:0/145870103 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ec4e0f100 0x558ec4e21030 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fcb4400bb70 tx=0x7fcb44009ca0 comp rx=0 tx=0).stop 2024-03-23T17:56:28.897 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.896+0000 7fcb6d6c4b80 1 -- 172.21.15.86:0/145870103 shutdown_connections 2024-03-23T17:56:28.897 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.896+0000 7fcb6d6c4b80 1 --2- 172.21.15.86:0/145870103 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x558ec4e7d3a0 0x558ec4e7f7c0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:28.897 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.897+0000 7fcb6d6c4b80 1 --2- 172.21.15.86:0/145870103 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ec4e1a8d0 0x558ec4e1f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:28.898 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.897+0000 7fcb3cc00700 0 client.26501 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:56:28.898 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.897+0000 7fcb6d6c4b80 1 --2- 172.21.15.86:0/145870103 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558ec4e0f100 0x558ec4e21030 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:28.898 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.897+0000 7fcb6d6c4b80 1 --2- 172.21.15.86:0/145870103 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558ec4e0fa20 0x558ec4e21570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:28.898 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.897+0000 7fcb6d6c4b80 1 -- 172.21.15.86:0/145870103 >> 172.21.15.86:0/145870103 conn(0x558ec4d6f640 msgr2=0x558ec4e11040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:56:28.898 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.897+0000 7fcb6d6c4b80 1 -- 172.21.15.86:0/145870103 shutdown_connections 2024-03-23T17:56:28.898 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.897+0000 7fcb6d6c4b80 1 -- 172.21.15.86:0/145870103 wait complete. 2024-03-23T17:56:28.898 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.897+0000 7fcb6d6c4b80 20 client.26501 trim_cache size 0 max 16384 2024-03-23T17:56:28.898 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-03-23T17:56:28.898 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-03-23T17:56:28.899 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-03-23T17:56:28.899 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-03-23T17:56:28.899 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.897+0000 7fcb59c00700 1 -- 172.21.15.86:0/3308628738 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fcb380a1290 msgr2=0x7fcb380a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:56:28.899 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.897+0000 7fcb59c00700 1 --2- 172.21.15.86:0/3308628738 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fcb380a1290 0x7fcb380a3750 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fcb44006010 tx=0x7fcb4400b260 comp rx=0 tx=0).stop 2024-03-23T17:56:28.899 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.897+0000 7fcb59c00700 1 -- 172.21.15.86:0/3308628738 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcb54106010 msgr2=0x7fcb541998d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:56:28.899 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.897+0000 7fcb59c00700 1 --2- 172.21.15.86:0/3308628738 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcb54106010 0x7fcb541998d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fcb4000e9c0 tx=0x7fcb4000ecd0 comp rx=0 tx=0).stop 2024-03-23T17:56:28.900 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.898+0000 7fcb59c00700 1 -- 172.21.15.86:0/3308628738 shutdown_connections 2024-03-23T17:56:28.900 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.898+0000 7fcb59c00700 1 --2- 172.21.15.86:0/3308628738 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb5419dc00 0x7fcb5419e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:28.900 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.898+0000 7fcb59c00700 1 --2- 172.21.15.86:0/3308628738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb54100ef0 0x7fcb54199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:28.900 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.898+0000 7fcb59c00700 1 --2- 172.21.15.86:0/3308628738 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fcb380a1290 0x7fcb380a3750 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:28.900 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.898+0000 7fcb59c00700 1 --2- 172.21.15.86:0/3308628738 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcb54106010 0x7fcb541998d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:28.900 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.898+0000 7fcb59c00700 1 -- 172.21.15.86:0/3308628738 >> 172.21.15.86:0/3308628738 conn(0x7fcb540fc750 msgr2=0x7fcb540fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:56:28.900 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.898+0000 7fcb59c00700 1 -- 172.21.15.86:0/3308628738 shutdown_connections 2024-03-23T17:56:28.900 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:28.898+0000 7fcb59c00700 1 -- 172.21.15.86:0/3308628738 wait complete. 2024-03-23T17:56:28.902 INFO:tasks.workunit.client.0.smithi086.stdout: 1 273130 5.96 MB/sec execute 173 sec latency 72.309 ms 2024-03-23T17:56:29.902 INFO:tasks.workunit.client.0.smithi086.stdout: 1 274154 5.94 MB/sec execute 174 sec latency 17.802 ms 2024-03-23T17:56:30.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:56:29 smithi092 ceph-mon[79089]: pgmap v178: 97 pgs: 97 active+clean; 108 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 205 op/s 2024-03-23T17:56:30.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:56:29 smithi152 ceph-mon[47949]: pgmap v178: 97 pgs: 97 active+clean; 108 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 205 op/s 2024-03-23T17:56:30.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:56:29 smithi086 ceph-mon[38221]: pgmap v178: 97 pgs: 97 active+clean; 108 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 205 op/s 2024-03-23T17:56:30.902 INFO:tasks.workunit.client.0.smithi086.stdout: 1 275747 5.94 MB/sec execute 175 sec latency 225.479 ms 2024-03-23T17:56:31.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:56:30 smithi092 ceph-mon[79089]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T17:56:31.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:56:30 smithi152 ceph-mon[47949]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T17:56:31.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:56:30 smithi086 ceph-mon[38221]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T17:56:31.903 INFO:tasks.workunit.client.0.smithi086.stdout: 1 277147 5.97 MB/sec execute 176 sec latency 11.442 ms 2024-03-23T17:56:32.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:56:31 smithi092 ceph-mon[79089]: pgmap v179: 97 pgs: 97 active+clean; 100 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.7 MiB/s wr, 341 op/s 2024-03-23T17:56:32.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:56:31 smithi152 ceph-mon[47949]: pgmap v179: 97 pgs: 97 active+clean; 100 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.7 MiB/s wr, 341 op/s 2024-03-23T17:56:32.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:56:31 smithi086 ceph-mon[38221]: pgmap v179: 97 pgs: 97 active+clean; 100 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.7 MiB/s wr, 341 op/s 2024-03-23T17:56:32.902 INFO:tasks.workunit.client.0.smithi086.stdout: 1 278385 5.97 MB/sec execute 177 sec latency 14.696 ms 2024-03-23T17:56:33.902 INFO:tasks.workunit.client.0.smithi086.stdout: 1 280246 6.00 MB/sec execute 178 sec latency 20.123 ms 2024-03-23T17:56:34.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:56:33 smithi092 ceph-mon[79089]: pgmap v180: 97 pgs: 97 active+clean; 100 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 238 op/s 2024-03-23T17:56:34.085 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:56:33 smithi152 ceph-mon[47949]: pgmap v180: 97 pgs: 97 active+clean; 100 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 238 op/s 2024-03-23T17:56:34.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:56:33 smithi086 ceph-mon[38221]: pgmap v180: 97 pgs: 97 active+clean; 100 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 238 op/s 2024-03-23T17:56:34.902 INFO:tasks.workunit.client.0.smithi086.stdout: 1 281398 5.99 MB/sec execute 179 sec latency 12.557 ms 2024-03-23T17:56:35.903 INFO:tasks.workunit.client.0.smithi086.stdout: 1 283568 6.03 MB/sec execute 180 sec latency 16.467 ms 2024-03-23T17:56:36.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:56:35 smithi092 ceph-mon[79089]: pgmap v181: 97 pgs: 97 active+clean; 107 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.4 MiB/s wr, 337 op/s 2024-03-23T17:56:36.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:56:35 smithi152 ceph-mon[47949]: pgmap v181: 97 pgs: 97 active+clean; 107 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.4 MiB/s wr, 337 op/s 2024-03-23T17:56:36.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:56:35 smithi086 ceph-mon[38221]: pgmap v181: 97 pgs: 97 active+clean; 107 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.4 MiB/s wr, 337 op/s 2024-03-23T17:56:36.903 INFO:tasks.workunit.client.0.smithi086.stdout: 1 284763 6.02 MB/sec execute 181 sec latency 15.429 ms 2024-03-23T17:56:37.903 INFO:tasks.workunit.client.0.smithi086.stdout: 1 286121 6.02 MB/sec execute 182 sec latency 18.678 ms 2024-03-23T17:56:38.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:56:37 smithi092 ceph-mon[79089]: pgmap v182: 97 pgs: 97 active+clean; 107 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 234 op/s 2024-03-23T17:56:38.085 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:56:37 smithi152 ceph-mon[47949]: pgmap v182: 97 pgs: 97 active+clean; 107 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 234 op/s 2024-03-23T17:56:38.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:56:37 smithi086 ceph-mon[38221]: pgmap v182: 97 pgs: 97 active+clean; 107 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 234 op/s 2024-03-23T17:56:38.903 INFO:tasks.workunit.client.0.smithi086.stdout: 1 287670 6.06 MB/sec execute 183 sec latency 32.381 ms 2024-03-23T17:56:39.903 INFO:tasks.workunit.client.0.smithi086.stdout: 1 288838 6.05 MB/sec execute 184 sec latency 23.081 ms 2024-03-23T17:56:40.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:56:39 smithi092 ceph-mon[79089]: pgmap v183: 97 pgs: 97 active+clean; 107 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 234 op/s 2024-03-23T17:56:40.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:56:39 smithi152 ceph-mon[47949]: pgmap v183: 97 pgs: 97 active+clean; 107 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 234 op/s 2024-03-23T17:56:40.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:56:39 smithi086 ceph-mon[38221]: pgmap v183: 97 pgs: 97 active+clean; 107 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 234 op/s 2024-03-23T17:56:40.903 INFO:tasks.workunit.client.0.smithi086.stdout: 1 290882 6.09 MB/sec execute 185 sec latency 15.790 ms 2024-03-23T17:56:41.903 INFO:tasks.workunit.client.0.smithi086.stdout: 1 291878 6.07 MB/sec execute 186 sec latency 64.596 ms 2024-03-23T17:56:42.056 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.056+0000 7f7e19400700 1 -- 172.21.15.86:0/228848491 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7e14106b70 msgr2=0x7f7e14106f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:56:42.057 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.056+0000 7f7e19400700 1 --2- 172.21.15.86:0/228848491 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7e14106b70 0x7f7e14106f50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7e04009990 tx=0x7f7e04009ca0 comp rx=0 tx=0).stop 2024-03-23T17:56:42.057 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.056+0000 7f7e19400700 1 -- 172.21.15.86:0/228848491 shutdown_connections 2024-03-23T17:56:42.057 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.056+0000 7f7e19400700 1 --2- 172.21.15.86:0/228848491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e14101490 0x7f7e1410d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:42.057 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.056+0000 7f7e19400700 1 --2- 172.21.15.86:0/228848491 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7e14106b70 0x7f7e14106f50 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:42.057 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.056+0000 7f7e19400700 1 --2- 172.21.15.86:0/228848491 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e14100ad0 0x7f7e14100f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:42.057 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.056+0000 7f7e19400700 1 -- 172.21.15.86:0/228848491 >> 172.21.15.86:0/228848491 conn(0x7f7e140fc5e0 msgr2=0x7f7e140fea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:56:42.057 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.056+0000 7f7e19400700 1 -- 172.21.15.86:0/228848491 shutdown_connections 2024-03-23T17:56:42.057 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.056+0000 7f7e19400700 1 -- 172.21.15.86:0/228848491 wait complete. 2024-03-23T17:56:42.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:56:41 smithi092 ceph-mon[79089]: pgmap v184: 97 pgs: 97 active+clean; 118 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.6 MiB/s wr, 341 op/s 2024-03-23T17:56:42.059 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.058+0000 7f7e19400700 1 Processor -- start 2024-03-23T17:56:42.059 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.058+0000 7f7e19400700 1 -- start start 2024-03-23T17:56:42.059 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.058+0000 7f7e19400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e14100ad0 0x7f7e14199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:56:42.059 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.058+0000 7f7e19400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7e14101490 0x7f7e141996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:56:42.059 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.058+0000 7f7e19400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e14106b70 0x7f7e1419da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:56:42.059 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.058+0000 7f7e19400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e1419e040 con 0x7f7e14100ad0 2024-03-23T17:56:42.059 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.058+0000 7f7e19400700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e1419e1b0 con 0x7f7e14106b70 2024-03-23T17:56:42.060 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.058+0000 7f7e19400700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e1419e370 con 0x7f7e14101490 2024-03-23T17:56:42.060 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.059+0000 7f7e13e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e14100ad0 0x7f7e14199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:56:42.060 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.059+0000 7f7e13400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7e14101490 0x7f7e141996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:56:42.060 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.059+0000 7f7e13e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e14100ad0 0x7f7e14199170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60176/0 (socket says 172.21.15.86:60176) 2024-03-23T17:56:42.060 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.059+0000 7f7e13e00700 1 -- 172.21.15.86:0/2046622740 learned_addr learned my addr 172.21.15.86:0/2046622740 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:56:42.060 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.059+0000 7f7e18a00700 1 --2- 172.21.15.86:0/2046622740 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e14106b70 0x7f7e1419da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:56:42.060 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.059+0000 7f7e18a00700 1 -- 172.21.15.86:0/2046622740 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7e14101490 msgr2=0x7f7e141996b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:56:42.060 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.059+0000 7f7e18a00700 1 --2- 172.21.15.86:0/2046622740 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7e14101490 0x7f7e141996b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:42.060 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.059+0000 7f7e18a00700 1 -- 172.21.15.86:0/2046622740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e14100ad0 msgr2=0x7f7e14199170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:56:42.060 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.059+0000 7f7e18a00700 1 --2- 172.21.15.86:0/2046622740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e14100ad0 0x7f7e14199170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:42.060 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.059+0000 7f7e18a00700 1 -- 172.21.15.86:0/2046622740 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e00009520 con 0x7f7e14106b70 2024-03-23T17:56:42.060 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.059+0000 7f7e13e00700 1 --2- 172.21.15.86:0/2046622740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e14100ad0 0x7f7e14199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:56:42.061 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.059+0000 7f7e18a00700 1 --2- 172.21.15.86:0/2046622740 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e14106b70 0x7f7e1419da00 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c00efc0 tx=0x7f7e0c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:56:42.061 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.060+0000 7f7e10c00700 1 -- 172.21.15.86:0/2046622740 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e0c018070 con 0x7f7e14106b70 2024-03-23T17:56:42.061 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.060+0000 7f7e19400700 1 -- 172.21.15.86:0/2046622740 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e040095f0 con 0x7f7e14106b70 2024-03-23T17:56:42.062 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.060+0000 7f7e10c00700 1 -- 172.21.15.86:0/2046622740 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e0c00f040 con 0x7f7e14106b70 2024-03-23T17:56:42.062 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.060+0000 7f7e10c00700 1 -- 172.21.15.86:0/2046622740 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e0c00cd40 con 0x7f7e14106b70 2024-03-23T17:56:42.062 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.060+0000 7f7e19400700 1 -- 172.21.15.86:0/2046622740 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e141a5cb0 con 0x7f7e14106b70 2024-03-23T17:56:42.063 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.063+0000 7f7e10c00700 1 -- 172.21.15.86:0/2046622740 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f7e0c003fc0 con 0x7f7e14106b70 2024-03-23T17:56:42.065 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.065+0000 7f7e2e4c3b80 1 Processor -- start 2024-03-23T17:56:42.065 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.065+0000 7f7e2e4c3b80 1 -- start start 2024-03-23T17:56:42.065 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.065+0000 7f7e10c00700 1 --2- 172.21.15.86:0/2046622740 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f7df80a1290 0x7f7df80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:56:42.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.065+0000 7f7e2e4c3b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cacd5cb2d0 0x55cacd5cb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:56:42.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.065+0000 7f7e10c00700 1 -- 172.21.15.86:0/2046622740 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f7e0c0cc820 con 0x7f7e14106b70 2024-03-23T17:56:42.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.065+0000 7f7e2e4c3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cacd5cbbf0 0x55cacd5d6570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:56:42.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.065+0000 7f7e2e4c3b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cacd5d6ab0 0x55cacd5d8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:56:42.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.065+0000 7f7e2e4c3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cacd524930 con 0x55cacd5cbbf0 2024-03-23T17:56:42.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.065+0000 7f7e2e4c3b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cacd5222f0 con 0x55cacd5d6ab0 2024-03-23T17:56:42.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.065+0000 7f7e2e4c3b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cacd51dbe0 con 0x55cacd5cb2d0 2024-03-23T17:56:42.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.065+0000 7f7e13400700 1 --2- 172.21.15.86:0/2046622740 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f7df80a1290 0x7f7df80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:56:42.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.066+0000 7f7e13e00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cacd5cb2d0 0x55cacd5cb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:56:42.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.066+0000 7f7e18a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cacd5d6ab0 0x55cacd5d8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:56:42.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.066+0000 7f7e13e00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cacd5cb2d0 0x55cacd5cb6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:40606/0 (socket says 172.21.15.86:40606) 2024-03-23T17:56:42.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.066+0000 7f7e13e00700 1 -- 172.21.15.86:0/1076082047 learned_addr learned my addr 172.21.15.86:0/1076082047 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:56:42.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.066+0000 7f7e13400700 1 --2- 172.21.15.86:0/1076082047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cacd5cbbf0 0x55cacd5d6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:56:42.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.066+0000 7f7e13e00700 1 -- 172.21.15.86:0/1076082047 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cacd5d6ab0 msgr2=0x55cacd5d8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:56:42.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.066+0000 7f7e13e00700 1 --2- 172.21.15.86:0/1076082047 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cacd5d6ab0 0x55cacd5d8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:42.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.066+0000 7f7e13e00700 1 -- 172.21.15.86:0/1076082047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cacd5cbbf0 msgr2=0x55cacd5d6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:56:42.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.066+0000 7f7e13e00700 1 --2- 172.21.15.86:0/1076082047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cacd5cbbf0 0x55cacd5d6570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:42.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.066+0000 7f7e13e00700 1 -- 172.21.15.86:0/1076082047 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cacd39f350 con 0x55cacd5cb2d0 2024-03-23T17:56:42.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.066+0000 7f7e13400700 1 --2- 172.21.15.86:0/1076082047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cacd5cbbf0 0x55cacd5d6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:56:42.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.066+0000 7f7e18a00700 1 --2- 172.21.15.86:0/1076082047 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cacd5d6ab0 0x55cacd5d8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:56:42.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.066+0000 7f7e13400700 1 --2- 172.21.15.86:0/2046622740 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f7df80a1290 0x7f7df80a3750 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7e1419a670 tx=0x7f7e00013a10 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:56:42.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.066+0000 7f7e13e00700 1 --2- 172.21.15.86:0/1076082047 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cacd5cb2d0 0x55cacd5cb6b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7e0400b660 tx=0x7f7e0400b690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af8b81aa21b03a65 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:56:42.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.067+0000 7f7e09200700 1 -- 172.21.15.86:0/1076082047 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e0401d070 con 0x55cacd5cb2d0 2024-03-23T17:56:42.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.067+0000 7f7e09200700 1 -- 172.21.15.86:0/1076082047 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e0401e040 con 0x55cacd5cb2d0 2024-03-23T17:56:42.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.067+0000 7f7e09200700 1 -- 172.21.15.86:0/1076082047 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e04004bf0 con 0x55cacd5cb2d0 2024-03-23T17:56:42.071 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.067+0000 7f7e2e4c3b80 1 -- 172.21.15.86:0/1076082047 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cacd5cb2d0 msgr2=0x55cacd5cb6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:56:42.071 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.067+0000 7f7e2e4c3b80 1 --2- 172.21.15.86:0/1076082047 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cacd5cb2d0 0x55cacd5cb6b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7e0400b660 tx=0x7f7e0400b690 comp rx=0 tx=0).stop 2024-03-23T17:56:42.071 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.068+0000 7f7e2e4c3b80 1 -- 172.21.15.86:0/1076082047 shutdown_connections 2024-03-23T17:56:42.071 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.068+0000 7f7e2e4c3b80 1 --2- 172.21.15.86:0/1076082047 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cacd5d6ab0 0x55cacd5d8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:42.071 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.068+0000 7f7e2e4c3b80 1 --2- 172.21.15.86:0/1076082047 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cacd5cb2d0 0x55cacd5cb6b0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:42.071 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.068+0000 7f7e2e4c3b80 1 --2- 172.21.15.86:0/1076082047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cacd5cbbf0 0x55cacd5d6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.068+0000 7f7e2e4c3b80 1 -- 172.21.15.86:0/1076082047 >> 172.21.15.86:0/1076082047 conn(0x55cacd52b640 msgr2=0x55cacd5c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:56:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.068+0000 7f7e2e4c3b80 1 -- 172.21.15.86:0/1076082047 shutdown_connections 2024-03-23T17:56:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.068+0000 7f7e2e4c3b80 1 -- 172.21.15.86:0/1076082047 wait complete. 2024-03-23T17:56:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.068+0000 7f7e2e4c3b80 1 -- start start 2024-03-23T17:56:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.069+0000 7f7e2e4c3b80 1 Processor -- start 2024-03-23T17:56:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.069+0000 7f7e2e4c3b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cacd5cb2d0 0x55cacd5d6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:56:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.069+0000 7f7e2e4c3b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cacd5cbbf0 0x55cacd5d3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:56:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.069+0000 7f7e2e4c3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cacd5d6ab0 0x55cacd5d3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:56:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.069+0000 7f7e2e4c3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cacd4b9d20 con 0x55cacd5d6ab0 2024-03-23T17:56:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.069+0000 7f7e2e4c3b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cacd4b99c0 con 0x55cacd5cbbf0 2024-03-23T17:56:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.069+0000 7f7e2e4c3b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cacd3a1640 con 0x55cacd5cb2d0 2024-03-23T17:56:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.069+0000 7f7e2e4c3b80 10 client.0 _ll_register_callbacks cb 0x55cacd4c3410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:56:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.069+0000 7f7e13e00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cacd5cb2d0 0x55cacd5d6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:56:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.070+0000 7f7e18a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cacd5d6ab0 0x55cacd5d3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:56:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.070+0000 7f7e13400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cacd5cbbf0 0x55cacd5d3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:56:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.070+0000 7f7e18a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cacd5d6ab0 0x55cacd5d3d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60200/0 (socket says 172.21.15.86:60200) 2024-03-23T17:56:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.070+0000 7f7e18a00700 1 -- 172.21.15.86:0/67168970 learned_addr learned my addr 172.21.15.86:0/67168970 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:56:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.070+0000 7f7e18a00700 1 -- 172.21.15.86:0/67168970 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cacd5cb2d0 msgr2=0x55cacd5d6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:56:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.070+0000 7f7e18a00700 1 --2- 172.21.15.86:0/67168970 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cacd5cb2d0 0x55cacd5d6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.070+0000 7f7e18a00700 1 -- 172.21.15.86:0/67168970 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cacd5cbbf0 msgr2=0x55cacd5d3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:56:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.070+0000 7f7e18a00700 1 --2- 172.21.15.86:0/67168970 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cacd5cbbf0 0x55cacd5d3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.070+0000 7f7e18a00700 1 -- 172.21.15.86:0/67168970 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e141a5cb0 con 0x55cacd5d6ab0 2024-03-23T17:56:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.070+0000 7f7e13400700 1 --2- 172.21.15.86:0/67168970 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cacd5cbbf0 0x55cacd5d3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:56:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.070+0000 7f7e18a00700 1 --2- 172.21.15.86:0/67168970 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cacd5d6ab0 0x55cacd5d3d70 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c0996b0 tx=0x7f7e0c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34229cacc82e89c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:56:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.070+0000 7f7dfd600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-23T17:56:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.071+0000 7f7dfd600700 1 -- 172.21.15.86:0/67168970 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e0c09a0b0 con 0x55cacd5d6ab0 2024-03-23T17:56:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.071+0000 7f7dfd600700 1 -- 172.21.15.86:0/67168970 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f7e0c0999b0 con 0x55cacd5d6ab0 2024-03-23T17:56:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.071+0000 7f7e2e4c3b80 1 -- 172.21.15.86:0/67168970 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cacd4de320 con 0x55cacd5d6ab0 2024-03-23T17:56:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.071+0000 7f7dfd600700 1 -- 172.21.15.86:0/67168970 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e0c09d7e0 con 0x55cacd5d6ab0 2024-03-23T17:56:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.071+0000 7f7e2e4c3b80 10 client.16464 fetch_fsmap learned FSMap version 40 2024-03-23T17:56:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.071+0000 7f7e2e4c3b80 1 -- 172.21.15.86:0/67168970 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x7f7e1410e2a0 con 0x55cacd5d6ab0 2024-03-23T17:56:42.075 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.071+0000 7f7dfd600700 1 -- 172.21.15.86:0/67168970 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e0c064020 con 0x55cacd5d6ab0 2024-03-23T17:56:42.075 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.072+0000 7f7dfd600700 1 -- 172.21.15.86:0/67168970 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 40) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f7e0c0a1980 con 0x55cacd5d6ab0 2024-03-23T17:56:42.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.072+0000 7f7e2e4c3b80 10 client.16464 fetch_fsmap finished waiting for FSMap version 40 2024-03-23T17:56:42.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.072+0000 7f7e2e4c3b80 10 client.16464 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:56:42.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.072+0000 7f7e2e4c3b80 20 client.16464 populate_metadata read hostname 'smithi086' 2024-03-23T17:56:42.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.072+0000 7f7e2e4c3b80 1 --2- 172.21.15.86:0/67168970 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55cacd5e6610 0x55cacd5e8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:56:42.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.072+0000 7f7e2e4c3b80 4 client.16464 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:56:42.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.072+0000 7f7e2e4c3b80 1 -- 172.21.15.86:0/67168970 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cacd5166a0 con 0x55cacd5e6610 2024-03-23T17:56:42.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.072+0000 7f7e13e00700 1 --2- 172.21.15.86:0/67168970 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55cacd5e6610 0x55cacd5e8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:56:42.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.073+0000 7f7e13e00700 1 --2- 172.21.15.86:0/67168970 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55cacd5e6610 0x55cacd5e8a30 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfcba10a2447be76 server_cookie=56fc05cc1bd3777 in_seq=0 out_seq=0 2024-03-23T17:56:42.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.073+0000 7f7dfd600700 10 client.16464 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:56:42.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.073+0000 7f7dfd600700 1 -- 172.21.15.86:0/67168970 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55cacd5166a0 con 0x55cacd5e6610 2024-03-23T17:56:42.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.073+0000 7f7dfd600700 10 client.16464 handle_command_reply: tid=0 2024-03-23T17:56:42.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.073+0000 7f7e2e4c3b80 1 client.16464 shutdown 2024-03-23T17:56:42.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.074+0000 7f7e2e4c3b80 1 -- 172.21.15.86:0/67168970 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cacd5d6ab0 msgr2=0x55cacd5d3d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:56:42.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.074+0000 7f7e2e4c3b80 1 --2- 172.21.15.86:0/67168970 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cacd5d6ab0 0x55cacd5d3d70 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c0996b0 tx=0x7f7e0c004ad0 comp rx=0 tx=0).stop 2024-03-23T17:56:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.074+0000 7f7e2e4c3b80 1 -- 172.21.15.86:0/67168970 shutdown_connections 2024-03-23T17:56:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.074+0000 7f7e2e4c3b80 1 --2- 172.21.15.86:0/67168970 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55cacd5e6610 0x55cacd5e8a30 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.074+0000 7f7e2e4c3b80 1 --2- 172.21.15.86:0/67168970 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cacd5d6ab0 0x55cacd5d3d70 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.074+0000 7f7e2e4c3b80 1 --2- 172.21.15.86:0/67168970 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cacd5cb2d0 0x55cacd5d6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.074+0000 7f7e2e4c3b80 1 --2- 172.21.15.86:0/67168970 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cacd5cbbf0 0x55cacd5d3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.074+0000 7f7e2e4c3b80 1 -- 172.21.15.86:0/67168970 >> 172.21.15.86:0/67168970 conn(0x55cacd52b640 msgr2=0x55cacd5cd390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:56:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.074+0000 7f7dfd600700 0 client.16464 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:56:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.074+0000 7f7e2e4c3b80 1 -- 172.21.15.86:0/67168970 shutdown_connections 2024-03-23T17:56:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.074+0000 7f7e2e4c3b80 1 -- 172.21.15.86:0/67168970 wait complete. 2024-03-23T17:56:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.074+0000 7f7e2e4c3b80 20 client.16464 trim_cache size 0 max 16384 2024-03-23T17:56:42.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:56:41 smithi152 ceph-mon[47949]: pgmap v184: 97 pgs: 97 active+clean; 118 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.6 MiB/s wr, 341 op/s 2024-03-23T17:56:42.091 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.090+0000 7f7e2e4c3b80 1 Processor -- start 2024-03-23T17:56:42.091 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.090+0000 7f7e2e4c3b80 1 -- start start 2024-03-23T17:56:42.091 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.090+0000 7f7e2e4c3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cacd5cb2d0 0x55cacd5dd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:56:42.091 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.090+0000 7f7e2e4c3b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cacd5cbbf0 0x55cacd5c1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:56:42.091 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.090+0000 7f7e2e4c3b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cacd5d6ab0 0x55cacd5cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:56:42.091 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.090+0000 7f7e2e4c3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cacd5cf5d0 con 0x55cacd5cb2d0 2024-03-23T17:56:42.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.090+0000 7f7e2e4c3b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cacd5cc2c0 con 0x55cacd5d6ab0 2024-03-23T17:56:42.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.090+0000 7f7e2e4c3b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cacd5e8a00 con 0x55cacd5cbbf0 2024-03-23T17:56:42.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.091+0000 7f7e13e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cacd5cb2d0 0x55cacd5dd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:56:42.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.091+0000 7f7e18a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cacd5d6ab0 0x55cacd5cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:56:42.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.091+0000 7f7e13e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cacd5cb2d0 0x55cacd5dd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60214/0 (socket says 172.21.15.86:60214) 2024-03-23T17:56:42.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.091+0000 7f7e13e00700 1 -- 172.21.15.86:0/3141238585 learned_addr learned my addr 172.21.15.86:0/3141238585 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:56:42.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.091+0000 7f7e18a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cacd5d6ab0 0x55cacd5cea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:48506/0 (socket says 172.21.15.86:48506) 2024-03-23T17:56:42.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.091+0000 7f7e13400700 1 --2- 172.21.15.86:0/3141238585 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cacd5cbbf0 0x55cacd5c1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:56:42.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.091+0000 7f7e13e00700 1 -- 172.21.15.86:0/3141238585 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cacd5cbbf0 msgr2=0x55cacd5c1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:56:42.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.091+0000 7f7e13e00700 1 --2- 172.21.15.86:0/3141238585 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cacd5cbbf0 0x55cacd5c1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:42.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.091+0000 7f7e13e00700 1 -- 172.21.15.86:0/3141238585 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cacd5d6ab0 msgr2=0x55cacd5cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:56:42.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.091+0000 7f7e13e00700 1 --2- 172.21.15.86:0/3141238585 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cacd5d6ab0 0x55cacd5cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:42.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.091+0000 7f7e13e00700 1 -- 172.21.15.86:0/3141238585 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cacd34bb00 con 0x55cacd5cb2d0 2024-03-23T17:56:42.093 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.091+0000 7f7e18a00700 1 --2- 172.21.15.86:0/3141238585 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cacd5d6ab0 0x55cacd5cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:56:42.093 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.091+0000 7f7e13400700 1 --2- 172.21.15.86:0/3141238585 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cacd5cbbf0 0x55cacd5c1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:56:42.093 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.091+0000 7f7e13e00700 1 --2- 172.21.15.86:0/3141238585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cacd5cb2d0 0x55cacd5dd700 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f7e04006010 tx=0x7f7e0401a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=491fced06a0a9d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:56:42.096 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.092+0000 7f7e09c00700 1 -- 172.21.15.86:0/3141238585 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e0401d040 con 0x55cacd5cb2d0 2024-03-23T17:56:42.096 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.092+0000 7f7e09c00700 1 -- 172.21.15.86:0/3141238585 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e0400be90 con 0x55cacd5cb2d0 2024-03-23T17:56:42.096 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.092+0000 7f7e09c00700 1 -- 172.21.15.86:0/3141238585 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e04004d00 con 0x55cacd5cb2d0 2024-03-23T17:56:42.096 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.092+0000 7f7e2e4c3b80 1 -- 172.21.15.86:0/3141238585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cacd5cb2d0 msgr2=0x55cacd5dd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:56:42.096 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.092+0000 7f7e2e4c3b80 1 --2- 172.21.15.86:0/3141238585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cacd5cb2d0 0x55cacd5dd700 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f7e04006010 tx=0x7f7e0401a040 comp rx=0 tx=0).stop 2024-03-23T17:56:42.096 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.092+0000 7f7e2e4c3b80 1 -- 172.21.15.86:0/3141238585 shutdown_connections 2024-03-23T17:56:42.096 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.092+0000 7f7e2e4c3b80 1 --2- 172.21.15.86:0/3141238585 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cacd5d6ab0 0x55cacd5cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:42.096 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.092+0000 7f7e2e4c3b80 1 --2- 172.21.15.86:0/3141238585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cacd5cb2d0 0x55cacd5dd700 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:42.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.092+0000 7f7e2e4c3b80 1 --2- 172.21.15.86:0/3141238585 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cacd5cbbf0 0x55cacd5c1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:42.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.092+0000 7f7e2e4c3b80 1 -- 172.21.15.86:0/3141238585 >> 172.21.15.86:0/3141238585 conn(0x55cacd52b640 msgr2=0x55cacd5d2c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:56:42.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.092+0000 7f7e2e4c3b80 1 -- 172.21.15.86:0/3141238585 shutdown_connections 2024-03-23T17:56:42.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.093+0000 7f7e2e4c3b80 1 -- 172.21.15.86:0/3141238585 wait complete. 2024-03-23T17:56:42.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.093+0000 7f7e2e4c3b80 1 -- start start 2024-03-23T17:56:42.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.093+0000 7f7e2e4c3b80 1 Processor -- start 2024-03-23T17:56:42.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.093+0000 7f7e2e4c3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cacd5cb2d0 0x55cacd5daec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:56:42.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.093+0000 7f7e2e4c3b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cacd5cbbf0 0x55cacd5db400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:56:42.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.094+0000 7f7e2e4c3b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cacd5d6ab0 0x55cacd5db940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:56:42.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.094+0000 7f7e2e4c3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cacd637170 con 0x55cacd5cb2d0 2024-03-23T17:56:42.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.094+0000 7f7e2e4c3b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cacd6372b0 con 0x55cacd5d6ab0 2024-03-23T17:56:42.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.094+0000 7f7e2e4c3b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cacd637470 con 0x55cacd5cbbf0 2024-03-23T17:56:42.098 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.094+0000 7f7e2e4c3b80 10 client.0 _ll_register_callbacks cb 0x55cacd4c3410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:56:42.098 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.094+0000 7f7e13e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cacd5cb2d0 0x55cacd5daec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:56:42.098 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.094+0000 7f7e18a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cacd5d6ab0 0x55cacd5db940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:56:42.098 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.094+0000 7f7e18a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cacd5d6ab0 0x55cacd5db940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:48510/0 (socket says 172.21.15.86:48510) 2024-03-23T17:56:42.098 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.094+0000 7f7e18a00700 1 -- 172.21.15.86:0/1715274056 learned_addr learned my addr 172.21.15.86:0/1715274056 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:56:42.098 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.094+0000 7f7e13400700 1 --2- 172.21.15.86:0/1715274056 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cacd5cbbf0 0x55cacd5db400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:56:42.098 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.094+0000 7f7e18a00700 1 -- 172.21.15.86:0/1715274056 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cacd5cbbf0 msgr2=0x55cacd5db400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:56:42.098 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.094+0000 7f7e18a00700 1 --2- 172.21.15.86:0/1715274056 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cacd5cbbf0 0x55cacd5db400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:42.098 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.094+0000 7f7e18a00700 1 -- 172.21.15.86:0/1715274056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cacd5cb2d0 msgr2=0x55cacd5daec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:56:42.098 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.094+0000 7f7e18a00700 1 --2- 172.21.15.86:0/1715274056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cacd5cb2d0 0x55cacd5daec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:42.098 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.094+0000 7f7e18a00700 1 -- 172.21.15.86:0/1715274056 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e1410e2a0 con 0x55cacd5d6ab0 2024-03-23T17:56:42.098 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.094+0000 7f7e13e00700 1 --2- 172.21.15.86:0/1715274056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cacd5cb2d0 0x55cacd5daec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:56:42.098 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.094+0000 7f7e18a00700 1 --2- 172.21.15.86:0/1715274056 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cacd5d6ab0 0x55cacd5db940 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c095e60 tx=0x7f7e0c00c810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d06741be1480a12e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:56:42.099 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.094+0000 7f7dfd600700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T17:56:42.099 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.095+0000 7f7dfd600700 1 -- 172.21.15.86:0/1715274056 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e0c09a0b0 con 0x55cacd5d6ab0 2024-03-23T17:56:42.099 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.095+0000 7f7dfd600700 1 -- 172.21.15.86:0/1715274056 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f7e0c065080 con 0x55cacd5d6ab0 2024-03-23T17:56:42.099 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.095+0000 7f7e2e4c3b80 1 -- 172.21.15.86:0/1715274056 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7e0401e040 con 0x55cacd5d6ab0 2024-03-23T17:56:42.099 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.095+0000 7f7dfd600700 1 -- 172.21.15.86:0/1715274056 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e0c0999b0 con 0x55cacd5d6ab0 2024-03-23T17:56:42.099 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.095+0000 7f7e2e4c3b80 10 client.26519 fetch_fsmap learned FSMap version 40 2024-03-23T17:56:42.099 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.095+0000 7f7e2e4c3b80 1 -- 172.21.15.86:0/1715274056 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55cacd39f350 con 0x55cacd5d6ab0 2024-03-23T17:56:42.099 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.095+0000 7f7dfd600700 1 -- 172.21.15.86:0/1715274056 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e0401e040 con 0x55cacd5d6ab0 2024-03-23T17:56:42.099 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.095+0000 7f7dfd600700 1 -- 172.21.15.86:0/1715274056 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 40) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f7e0c098910 con 0x55cacd5d6ab0 2024-03-23T17:56:42.099 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.095+0000 7f7e2e4c3b80 10 client.26519 fetch_fsmap finished waiting for FSMap version 40 2024-03-23T17:56:42.099 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.095+0000 7f7e2e4c3b80 10 client.26519 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:56:42.099 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.095+0000 7f7e2e4c3b80 20 client.26519 populate_metadata read hostname 'smithi086' 2024-03-23T17:56:42.100 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.095+0000 7f7e2e4c3b80 1 --2- 172.21.15.86:0/1715274056 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55cacd638910 0x55cacd63ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:56:42.100 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.095+0000 7f7e2e4c3b80 4 client.26519 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T17:56:42.100 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.095+0000 7f7e2e4c3b80 1 -- 172.21.15.86:0/1715274056 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55cacd36cc90 con 0x55cacd638910 2024-03-23T17:56:42.100 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.096+0000 7f7e13e00700 1 --2- 172.21.15.86:0/1715274056 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55cacd638910 0x55cacd63ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:56:42.100 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.096+0000 7f7dfd600700 10 client.26519 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:56:42.100 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.096+0000 7f7e13e00700 1 --2- 172.21.15.86:0/1715274056 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55cacd638910 0x55cacd63ad30 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc010680fea34609 server_cookie=578f283490f5dd16 in_seq=0 out_seq=0 2024-03-23T17:56:42.136 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.136+0000 7f7dfd600700 1 -- 172.21.15.86:0/1715274056 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55cacd36cc90 con 0x55cacd638910 2024-03-23T17:56:42.137 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.136+0000 7f7dfd600700 10 client.26519 handle_command_reply: tid=0 2024-03-23T17:56:42.137 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.136+0000 7f7e2e4c3b80 1 client.26519 shutdown 2024-03-23T17:56:42.137 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.136+0000 7f7e2e4c3b80 1 -- 172.21.15.86:0/1715274056 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cacd5d6ab0 msgr2=0x55cacd5db940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:56:42.137 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.136+0000 7f7e2e4c3b80 1 --2- 172.21.15.86:0/1715274056 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cacd5d6ab0 0x55cacd5db940 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c095e60 tx=0x7f7e0c00c810 comp rx=0 tx=0).stop 2024-03-23T17:56:42.137 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.136+0000 7f7e2e4c3b80 1 -- 172.21.15.86:0/1715274056 shutdown_connections 2024-03-23T17:56:42.137 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.136+0000 7f7e2e4c3b80 1 --2- 172.21.15.86:0/1715274056 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55cacd638910 0x55cacd63ad30 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:42.137 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.136+0000 7f7e2e4c3b80 1 --2- 172.21.15.86:0/1715274056 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cacd5d6ab0 0x55cacd5db940 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:42.137 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.136+0000 7f7e2e4c3b80 1 --2- 172.21.15.86:0/1715274056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cacd5cb2d0 0x55cacd5daec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:42.137 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.136+0000 7f7e2e4c3b80 1 --2- 172.21.15.86:0/1715274056 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cacd5cbbf0 0x55cacd5db400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:42.137 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.136+0000 7f7dfd600700 0 client.26519 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:56:42.137 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.136+0000 7f7e2e4c3b80 1 -- 172.21.15.86:0/1715274056 >> 172.21.15.86:0/1715274056 conn(0x55cacd52b640 msgr2=0x55cacd5d1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:56:42.137 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.137+0000 7f7e2e4c3b80 1 -- 172.21.15.86:0/1715274056 shutdown_connections 2024-03-23T17:56:42.137 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.137+0000 7f7e2e4c3b80 1 -- 172.21.15.86:0/1715274056 wait complete. 2024-03-23T17:56:42.138 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.137+0000 7f7e2e4c3b80 20 client.26519 trim_cache size 0 max 16384 2024-03-23T17:56:42.138 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-03-23T17:56:42.138 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-03-23T17:56:42.138 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-03-23T17:56:42.138 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-03-23T17:56:42.139 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.138+0000 7f7e19400700 1 -- 172.21.15.86:0/2046622740 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f7df80a1290 msgr2=0x7f7df80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:56:42.139 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.138+0000 7f7e19400700 1 --2- 172.21.15.86:0/2046622740 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f7df80a1290 0x7f7df80a3750 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7e1419a670 tx=0x7f7e00013a10 comp rx=0 tx=0).stop 2024-03-23T17:56:42.139 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.138+0000 7f7e19400700 1 -- 172.21.15.86:0/2046622740 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e14106b70 msgr2=0x7f7e1419da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:56:42.139 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.138+0000 7f7e19400700 1 --2- 172.21.15.86:0/2046622740 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e14106b70 0x7f7e1419da00 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c00efc0 tx=0x7f7e0c00c3a0 comp rx=0 tx=0).stop 2024-03-23T17:56:42.139 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.138+0000 7f7e19400700 1 -- 172.21.15.86:0/2046622740 shutdown_connections 2024-03-23T17:56:42.139 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.138+0000 7f7e19400700 1 --2- 172.21.15.86:0/2046622740 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e14106b70 0x7f7e1419da00 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:42.139 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.138+0000 7f7e19400700 1 --2- 172.21.15.86:0/2046622740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e14100ad0 0x7f7e14199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:42.140 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.138+0000 7f7e19400700 1 --2- 172.21.15.86:0/2046622740 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f7df80a1290 0x7f7df80a3750 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:42.140 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.138+0000 7f7e19400700 1 --2- 172.21.15.86:0/2046622740 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7e14101490 0x7f7e141996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:42.140 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.138+0000 7f7e19400700 1 -- 172.21.15.86:0/2046622740 >> 172.21.15.86:0/2046622740 conn(0x7f7e140fc5e0 msgr2=0x7f7e1410c170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:56:42.140 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.138+0000 7f7e19400700 1 -- 172.21.15.86:0/2046622740 shutdown_connections 2024-03-23T17:56:42.140 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:42.138+0000 7f7e19400700 1 -- 172.21.15.86:0/2046622740 wait complete. 2024-03-23T17:56:42.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:56:41 smithi086 ceph-mon[38221]: pgmap v184: 97 pgs: 97 active+clean; 118 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.6 MiB/s wr, 341 op/s 2024-03-23T17:56:42.903 INFO:tasks.workunit.client.0.smithi086.stdout: 1 293804 6.10 MB/sec execute 187 sec latency 50.776 ms 2024-03-23T17:56:43.903 INFO:tasks.workunit.client.0.smithi086.stdout: 1 295134 6.10 MB/sec execute 188 sec latency 15.861 ms 2024-03-23T17:56:44.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:56:43 smithi092 ceph-mon[79089]: pgmap v185: 97 pgs: 97 active+clean; 118 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 205 op/s 2024-03-23T17:56:44.085 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:56:43 smithi152 ceph-mon[47949]: pgmap v185: 97 pgs: 97 active+clean; 118 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 205 op/s 2024-03-23T17:56:44.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:56:43 smithi086 ceph-mon[38221]: pgmap v185: 97 pgs: 97 active+clean; 118 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 205 op/s 2024-03-23T17:56:44.903 INFO:tasks.workunit.client.0.smithi086.stdout: 1 297003 6.11 MB/sec execute 189 sec latency 17.702 ms 2024-03-23T17:56:45.903 INFO:tasks.workunit.client.0.smithi086.stdout: 1 298439 6.14 MB/sec execute 190 sec latency 12.699 ms 2024-03-23T17:56:46.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:56:45 smithi092 ceph-mon[79089]: pgmap v186: 97 pgs: 97 active+clean; 111 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.2 MiB/s wr, 329 op/s 2024-03-23T17:56:46.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:56:45 smithi152 ceph-mon[47949]: pgmap v186: 97 pgs: 97 active+clean; 111 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.2 MiB/s wr, 329 op/s 2024-03-23T17:56:46.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:56:45 smithi086 ceph-mon[38221]: pgmap v186: 97 pgs: 97 active+clean; 111 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.2 MiB/s wr, 329 op/s 2024-03-23T17:56:46.903 INFO:tasks.workunit.client.0.smithi086.stdout: 1 300364 6.14 MB/sec execute 191 sec latency 17.646 ms 2024-03-23T17:56:47.903 INFO:tasks.workunit.client.0.smithi086.stdout: 1 301911 6.17 MB/sec execute 192 sec latency 12.570 ms 2024-03-23T17:56:48.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:56:47 smithi092 ceph-mon[79089]: pgmap v187: 97 pgs: 97 active+clean; 111 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 231 op/s 2024-03-23T17:56:48.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:56:47 smithi152 ceph-mon[47949]: pgmap v187: 97 pgs: 97 active+clean; 111 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 231 op/s 2024-03-23T17:56:48.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:56:47 smithi086 ceph-mon[38221]: pgmap v187: 97 pgs: 97 active+clean; 111 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 231 op/s 2024-03-23T17:56:48.903 INFO:tasks.workunit.client.0.smithi086.stdout: 1 303084 6.17 MB/sec execute 193 sec latency 22.981 ms 2024-03-23T17:56:49.903 INFO:tasks.workunit.client.0.smithi086.stdout: 1 305011 6.20 MB/sec execute 194 sec latency 36.542 ms 2024-03-23T17:56:50.085 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:56:49 smithi152 ceph-mon[47949]: pgmap v188: 97 pgs: 97 active+clean; 111 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 231 op/s 2024-03-23T17:56:50.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:56:49 smithi086 ceph-mon[38221]: pgmap v188: 97 pgs: 97 active+clean; 111 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 231 op/s 2024-03-23T17:56:50.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:56:49 smithi092 ceph-mon[79089]: pgmap v188: 97 pgs: 97 active+clean; 111 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 231 op/s 2024-03-23T17:56:50.903 INFO:tasks.workunit.client.0.smithi086.stdout: 1 306124 6.19 MB/sec execute 195 sec latency 15.408 ms 2024-03-23T17:56:51.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:56:50 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:56:51.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:56:50 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:56:51.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:56:50 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:56:51.904 INFO:tasks.workunit.client.0.smithi086.stdout: 1 308074 6.21 MB/sec execute 196 sec latency 15.009 ms 2024-03-23T17:56:52.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:56:51 smithi152 ceph-mon[47949]: pgmap v189: 97 pgs: 97 active+clean; 119 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.2 MiB/s wr, 330 op/s 2024-03-23T17:56:52.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:56:51 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:56:52.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:56:51 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:56:52.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:56:51 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:56:52.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:56:51 smithi086 ceph-mon[38221]: pgmap v189: 97 pgs: 97 active+clean; 119 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.2 MiB/s wr, 330 op/s 2024-03-23T17:56:52.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:56:51 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:56:52.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:56:51 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:56:52.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:56:51 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:56:52.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:56:51 smithi092 ceph-mon[79089]: pgmap v189: 97 pgs: 97 active+clean; 119 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.2 MiB/s wr, 330 op/s 2024-03-23T17:56:52.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:56:51 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:56:52.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:56:51 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:56:52.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:56:51 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:56:52.904 INFO:tasks.workunit.client.0.smithi086.stdout: 1 309289 6.21 MB/sec execute 197 sec latency 18.067 ms 2024-03-23T17:56:53.904 INFO:tasks.workunit.client.0.smithi086.stdout: 1 310528 6.21 MB/sec execute 198 sec latency 16.915 ms 2024-03-23T17:56:54.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:56:53 smithi152 ceph-mon[47949]: pgmap v190: 97 pgs: 97 active+clean; 119 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.9 MiB/s wr, 223 op/s 2024-03-23T17:56:54.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:56:53 smithi086 ceph-mon[38221]: pgmap v190: 97 pgs: 97 active+clean; 119 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.9 MiB/s wr, 223 op/s 2024-03-23T17:56:54.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:56:53 smithi092 ceph-mon[79089]: pgmap v190: 97 pgs: 97 active+clean; 119 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.9 MiB/s wr, 223 op/s 2024-03-23T17:56:54.904 INFO:tasks.workunit.client.0.smithi086.stdout: 1 312379 6.24 MB/sec execute 199 sec latency 39.170 ms 2024-03-23T17:56:55.299 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.299+0000 7fd2bba00700 1 -- 172.21.15.86:0/2445315733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2b41018a0 msgr2=0x7fd2b41058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:56:55.300 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.299+0000 7fd2bba00700 1 --2- 172.21.15.86:0/2445315733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2b41018a0 0x7fd2b41058e0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0009990 tx=0x7fd2a0009ca0 comp rx=0 tx=0).stop 2024-03-23T17:56:55.300 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.299+0000 7fd2bba00700 1 -- 172.21.15.86:0/2445315733 shutdown_connections 2024-03-23T17:56:55.300 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.299+0000 7fd2bba00700 1 --2- 172.21.15.86:0/2445315733 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2b4106010 0x7fd2b410db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:55.300 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.299+0000 7fd2bba00700 1 --2- 172.21.15.86:0/2445315733 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2b4100ef0 0x7fd2b41012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:55.300 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.299+0000 7fd2bba00700 1 --2- 172.21.15.86:0/2445315733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2b41018a0 0x7fd2b41058e0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:55.300 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.299+0000 7fd2bba00700 1 -- 172.21.15.86:0/2445315733 >> 172.21.15.86:0/2445315733 conn(0x7fd2b40fc750 msgr2=0x7fd2b40feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:56:55.300 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.299+0000 7fd2bba00700 1 -- 172.21.15.86:0/2445315733 shutdown_connections 2024-03-23T17:56:55.300 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.299+0000 7fd2bba00700 1 -- 172.21.15.86:0/2445315733 wait complete. 2024-03-23T17:56:55.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.300+0000 7fd2bba00700 1 Processor -- start 2024-03-23T17:56:55.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.300+0000 7fd2bba00700 1 -- start start 2024-03-23T17:56:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.301+0000 7fd2bba00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2b4100ef0 0x7fd2b40730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:56:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.301+0000 7fd2bba00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2b4106010 0x7fd2b40735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:56:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.301+0000 7fd2bba00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2b4073b30 0x7fd2b41a5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:56:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.301+0000 7fd2bba00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b41a5960 con 0x7fd2b4100ef0 2024-03-23T17:56:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.301+0000 7fd2bba00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b41a5ad0 con 0x7fd2b4106010 2024-03-23T17:56:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.301+0000 7fd2bba00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b41a5c90 con 0x7fd2b4073b30 2024-03-23T17:56:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.301+0000 7fd2b3e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2b4106010 0x7fd2b40735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:56:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.301+0000 7fd2b3e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2b4106010 0x7fd2b40735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:44056/0 (socket says 172.21.15.86:44056) 2024-03-23T17:56:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.301+0000 7fd2b3e00700 1 -- 172.21.15.86:0/2603531201 learned_addr learned my addr 172.21.15.86:0/2603531201 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:56:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.301+0000 7fd2b9a00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2b4073b30 0x7fd2b41a5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:56:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.301+0000 7fd2b9000700 1 --2- 172.21.15.86:0/2603531201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2b4100ef0 0x7fd2b40730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:56:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.301+0000 7fd2b3e00700 1 -- 172.21.15.86:0/2603531201 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2b4073b30 msgr2=0x7fd2b41a5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:56:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.301+0000 7fd2b3e00700 1 --2- 172.21.15.86:0/2603531201 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2b4073b30 0x7fd2b41a5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:55.304 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.301+0000 7fd2b3e00700 1 -- 172.21.15.86:0/2603531201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2b4100ef0 msgr2=0x7fd2b40730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:56:55.304 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.301+0000 7fd2b3e00700 1 --2- 172.21.15.86:0/2603531201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2b4100ef0 0x7fd2b40730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:55.304 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.301+0000 7fd2b3e00700 1 -- 172.21.15.86:0/2603531201 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2a00095f0 con 0x7fd2b4106010 2024-03-23T17:56:55.304 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.302+0000 7fd2b9a00700 1 --2- 172.21.15.86:0/2603531201 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2b4073b30 0x7fd2b41a5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:56:55.304 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.302+0000 7fd2b3e00700 1 --2- 172.21.15.86:0/2603531201 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2b4106010 0x7fd2b40735f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0000c00 tx=0x7fd2a0005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:56:55.304 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.302+0000 7fd2b1600700 1 -- 172.21.15.86:0/2603531201 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a001d070 con 0x7fd2b4106010 2024-03-23T17:56:55.304 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.302+0000 7fd2b1600700 1 -- 172.21.15.86:0/2603531201 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2a0022470 con 0x7fd2b4106010 2024-03-23T17:56:55.304 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.302+0000 7fd2b1600700 1 -- 172.21.15.86:0/2603531201 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a0017650 con 0x7fd2b4106010 2024-03-23T17:56:55.304 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.302+0000 7fd2bba00700 1 -- 172.21.15.86:0/2603531201 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2b41a5f10 con 0x7fd2b4106010 2024-03-23T17:56:55.304 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.302+0000 7fd2bba00700 1 -- 172.21.15.86:0/2603531201 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2b41a6280 con 0x7fd2b4106010 2024-03-23T17:56:55.305 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.306+0000 7fd2b1600700 1 -- 172.21.15.86:0/2603531201 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fd2a000fad0 con 0x7fd2b4106010 2024-03-23T17:56:55.307 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.307+0000 7fd2cf440b80 1 Processor -- start 2024-03-23T17:56:55.307 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.307+0000 7fd2cf440b80 1 -- start start 2024-03-23T17:56:55.307 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.307+0000 7fd2b1600700 1 --2- 172.21.15.86:0/2603531201 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fd29c0a1290 0x7fd29c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:56:55.307 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.307+0000 7fd2b1600700 1 -- 172.21.15.86:0/2603531201 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fd2a00cdfa0 con 0x7fd2b4106010 2024-03-23T17:56:55.309 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.307+0000 7fd2b9a00700 1 --2- 172.21.15.86:0/2603531201 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fd29c0a1290 0x7fd29c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:56:55.309 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.307+0000 7fd2cf440b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598daf0b100 0x5598daf0b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:56:55.309 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.308+0000 7fd2cf440b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5598daf0ba20 0x5598daf16390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:56:55.309 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.308+0000 7fd2cf440b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5598daf168d0 0x5598daf18cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:56:55.309 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.308+0000 7fd2cf440b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598dae64930 con 0x5598daf0b100 2024-03-23T17:56:55.309 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.308+0000 7fd2cf440b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598dae622f0 con 0x5598daf0ba20 2024-03-23T17:56:55.309 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.308+0000 7fd2cf440b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598dae5dbe0 con 0x5598daf168d0 2024-03-23T17:56:55.310 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.308+0000 7fd2b9000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598daf0b100 0x5598daf0b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:56:55.310 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.308+0000 7fd2b9000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598daf0b100 0x5598daf0b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41346/0 (socket says 172.21.15.86:41346) 2024-03-23T17:56:55.310 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.308+0000 7fd2b9000700 1 -- 172.21.15.86:0/2640643522 learned_addr learned my addr 172.21.15.86:0/2640643522 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:56:55.310 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.308+0000 7fd2b3e00700 1 --2- 172.21.15.86:0/2640643522 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5598daf0ba20 0x5598daf16390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:56:55.310 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.308+0000 7fd2b9a00700 1 --2- 172.21.15.86:0/2640643522 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5598daf168d0 0x5598daf18cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:56:55.310 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.308+0000 7fd2b9a00700 1 --2- 172.21.15.86:0/2603531201 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fd29c0a1290 0x7fd29c0a3750 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac003e80 tx=0x7fd2ac00b040 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:56:55.310 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.308+0000 7fd2b9000700 1 -- 172.21.15.86:0/2640643522 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5598daf168d0 msgr2=0x5598daf18cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:56:55.310 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.308+0000 7fd2b9000700 1 --2- 172.21.15.86:0/2640643522 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5598daf168d0 0x5598daf18cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:55.310 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.308+0000 7fd2b9000700 1 -- 172.21.15.86:0/2640643522 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5598daf0ba20 msgr2=0x5598daf16390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:56:55.310 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.308+0000 7fd2b9000700 1 --2- 172.21.15.86:0/2640643522 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5598daf0ba20 0x5598daf16390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:55.310 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.308+0000 7fd2b9000700 1 -- 172.21.15.86:0/2640643522 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598dacdf350 con 0x5598daf0b100 2024-03-23T17:56:55.310 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.308+0000 7fd2b3e00700 1 --2- 172.21.15.86:0/2640643522 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5598daf0ba20 0x5598daf16390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:56:55.310 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.309+0000 7fd2b9000700 1 --2- 172.21.15.86:0/2640643522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598daf0b100 0x5598daf0b4e0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8009d80 tx=0x7fd2a8011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf63e43790abc3c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:56:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.309+0000 7fd2a6000700 1 -- 172.21.15.86:0/2640643522 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a8004230 con 0x5598daf0b100 2024-03-23T17:56:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.309+0000 7fd2a6000700 1 -- 172.21.15.86:0/2640643522 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2a8009e10 con 0x5598daf0b100 2024-03-23T17:56:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.309+0000 7fd2a6000700 1 -- 172.21.15.86:0/2640643522 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a80197e0 con 0x5598daf0b100 2024-03-23T17:56:55.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.310+0000 7fd2cf440b80 1 -- 172.21.15.86:0/2640643522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598daf0b100 msgr2=0x5598daf0b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:56:55.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.310+0000 7fd2cf440b80 1 --2- 172.21.15.86:0/2640643522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598daf0b100 0x5598daf0b4e0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8009d80 tx=0x7fd2a8011720 comp rx=0 tx=0).stop 2024-03-23T17:56:55.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.310+0000 7fd2cf440b80 1 -- 172.21.15.86:0/2640643522 shutdown_connections 2024-03-23T17:56:55.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.310+0000 7fd2cf440b80 1 --2- 172.21.15.86:0/2640643522 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5598daf168d0 0x5598daf18cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:55.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.310+0000 7fd2cf440b80 1 --2- 172.21.15.86:0/2640643522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598daf0b100 0x5598daf0b4e0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:55.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.310+0000 7fd2cf440b80 1 --2- 172.21.15.86:0/2640643522 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5598daf0ba20 0x5598daf16390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:55.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.310+0000 7fd2cf440b80 1 -- 172.21.15.86:0/2640643522 >> 172.21.15.86:0/2640643522 conn(0x5598dae6b640 msgr2=0x5598daf07520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:56:55.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.310+0000 7fd2cf440b80 1 -- 172.21.15.86:0/2640643522 shutdown_connections 2024-03-23T17:56:55.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.310+0000 7fd2cf440b80 1 -- 172.21.15.86:0/2640643522 wait complete. 2024-03-23T17:56:55.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.310+0000 7fd2cf440b80 1 -- start start 2024-03-23T17:56:55.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.311+0000 7fd2cf440b80 1 Processor -- start 2024-03-23T17:56:55.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.311+0000 7fd2cf440b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5598daf0b100 0x5598daf16270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:56:55.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.311+0000 7fd2cf440b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598daf0ba20 0x5598daf13560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:56:55.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.311+0000 7fd2cf440b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5598daf168d0 0x5598daf13b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:56:55.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.311+0000 7fd2cf440b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598dadf9d20 con 0x5598daf0ba20 2024-03-23T17:56:55.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.311+0000 7fd2cf440b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598dadf99c0 con 0x5598daf0b100 2024-03-23T17:56:55.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.311+0000 7fd2cf440b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598dace1640 con 0x5598daf168d0 2024-03-23T17:56:55.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.311+0000 7fd2cf440b80 10 client.0 _ll_register_callbacks cb 0x5598dae03410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:56:55.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.311+0000 7fd2b9000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5598daf0b100 0x5598daf16270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:56:55.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.311+0000 7fd2b9000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5598daf0b100 0x5598daf16270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:44068/0 (socket says 172.21.15.86:44068) 2024-03-23T17:56:55.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.311+0000 7fd2b9000700 1 -- 172.21.15.86:0/3670920113 learned_addr learned my addr 172.21.15.86:0/3670920113 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:56:55.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.311+0000 7fd2b3e00700 1 --2- 172.21.15.86:0/3670920113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598daf0ba20 0x5598daf13560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:56:55.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.311+0000 7fd2b9a00700 1 --2- 172.21.15.86:0/3670920113 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5598daf168d0 0x5598daf13b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:56:55.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.312+0000 7fd2b9000700 1 -- 172.21.15.86:0/3670920113 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5598daf168d0 msgr2=0x5598daf13b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:56:55.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.312+0000 7fd2b9000700 1 --2- 172.21.15.86:0/3670920113 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5598daf168d0 0x5598daf13b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:55.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.312+0000 7fd2b9000700 1 -- 172.21.15.86:0/3670920113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598daf0ba20 msgr2=0x5598daf13560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:56:55.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.312+0000 7fd2b9000700 1 --2- 172.21.15.86:0/3670920113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598daf0ba20 0x5598daf13560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:55.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.312+0000 7fd2b9000700 1 -- 172.21.15.86:0/3670920113 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2b41a6280 con 0x5598daf0b100 2024-03-23T17:56:55.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.312+0000 7fd2b9000700 1 --2- 172.21.15.86:0/3670920113 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5598daf0b100 0x5598daf16270 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8000c00 tx=0x7fd2a8004280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb39c952b0bd7ed3 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:56:55.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.312+0000 7fd29aa00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T17:56:55.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.312+0000 7fd29aa00700 1 -- 172.21.15.86:0/3670920113 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a8005240 con 0x5598daf0b100 2024-03-23T17:56:55.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.312+0000 7fd29aa00700 1 -- 172.21.15.86:0/3670920113 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fd2a8005ce0 con 0x5598daf0b100 2024-03-23T17:56:55.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.312+0000 7fd2cf440b80 1 -- 172.21.15.86:0/3670920113 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5598dae1e320 con 0x5598daf0b100 2024-03-23T17:56:55.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.312+0000 7fd29aa00700 1 -- 172.21.15.86:0/3670920113 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2a8003cf0 con 0x5598daf0b100 2024-03-23T17:56:55.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.312+0000 7fd2cf440b80 10 client.26531 fetch_fsmap learned FSMap version 40 2024-03-23T17:56:55.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.313+0000 7fd2cf440b80 1 -- 172.21.15.86:0/3670920113 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x7fd2a80099e0 con 0x5598daf0b100 2024-03-23T17:56:55.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.313+0000 7fd29aa00700 1 -- 172.21.15.86:0/3670920113 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5598dae1e320 con 0x5598daf0b100 2024-03-23T17:56:55.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.313+0000 7fd29aa00700 1 -- 172.21.15.86:0/3670920113 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 40) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fd2a8021040 con 0x5598daf0b100 2024-03-23T17:56:55.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.313+0000 7fd2cf440b80 10 client.26531 fetch_fsmap finished waiting for FSMap version 40 2024-03-23T17:56:55.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.313+0000 7fd2cf440b80 10 client.26531 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:56:55.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.313+0000 7fd2cf440b80 20 client.26531 populate_metadata read hostname 'smithi086' 2024-03-23T17:56:55.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.313+0000 7fd2cf440b80 1 --2- 172.21.15.86:0/3670920113 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5598daf264c0 0x5598daf288e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:56:55.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.313+0000 7fd2cf440b80 4 client.26531 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:56:55.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.313+0000 7fd2cf440b80 1 -- 172.21.15.86:0/3670920113 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5598dae566a0 con 0x5598daf264c0 2024-03-23T17:56:55.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.313+0000 7fd2b3e00700 1 --2- 172.21.15.86:0/3670920113 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5598daf264c0 0x5598daf288e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:56:55.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.314+0000 7fd2b3e00700 1 --2- 172.21.15.86:0/3670920113 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5598daf264c0 0x5598daf288e0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56436f2cf1638917 server_cookie=d87c373d242fded4 in_seq=0 out_seq=0 2024-03-23T17:56:55.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.314+0000 7fd29aa00700 10 client.26531 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:56:55.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.314+0000 7fd29aa00700 1 -- 172.21.15.86:0/3670920113 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5598dae566a0 con 0x5598daf264c0 2024-03-23T17:56:55.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.314+0000 7fd29aa00700 10 client.26531 handle_command_reply: tid=0 2024-03-23T17:56:55.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.314+0000 7fd2cf440b80 1 client.26531 shutdown 2024-03-23T17:56:55.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.315+0000 7fd2cf440b80 1 -- 172.21.15.86:0/3670920113 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5598daf0b100 msgr2=0x5598daf16270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:56:55.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.315+0000 7fd2cf440b80 1 --2- 172.21.15.86:0/3670920113 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5598daf0b100 0x5598daf16270 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8000c00 tx=0x7fd2a8004280 comp rx=0 tx=0).stop 2024-03-23T17:56:55.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.315+0000 7fd2cf440b80 1 -- 172.21.15.86:0/3670920113 shutdown_connections 2024-03-23T17:56:55.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.315+0000 7fd2cf440b80 1 --2- 172.21.15.86:0/3670920113 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5598daf264c0 0x5598daf288e0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:55.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.315+0000 7fd2cf440b80 1 --2- 172.21.15.86:0/3670920113 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5598daf168d0 0x5598daf13b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:55.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.315+0000 7fd2cf440b80 1 --2- 172.21.15.86:0/3670920113 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5598daf0b100 0x5598daf16270 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:55.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.315+0000 7fd2cf440b80 1 --2- 172.21.15.86:0/3670920113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598daf0ba20 0x5598daf13560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:55.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.315+0000 7fd2cf440b80 1 -- 172.21.15.86:0/3670920113 >> 172.21.15.86:0/3670920113 conn(0x5598dae6b640 msgr2=0x5598daf0d0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:56:55.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.315+0000 7fd29aa00700 0 client.26531 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:56:55.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.315+0000 7fd2cf440b80 1 -- 172.21.15.86:0/3670920113 shutdown_connections 2024-03-23T17:56:55.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.315+0000 7fd2cf440b80 1 -- 172.21.15.86:0/3670920113 wait complete. 2024-03-23T17:56:55.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.315+0000 7fd2cf440b80 20 client.26531 trim_cache size 0 max 16384 2024-03-23T17:56:55.332 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.332+0000 7fd2cf440b80 1 Processor -- start 2024-03-23T17:56:55.332 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.332+0000 7fd2cf440b80 1 -- start start 2024-03-23T17:56:55.334 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.333+0000 7fd2cf440b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598daf0b100 0x5598daf0e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:56:55.334 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.333+0000 7fd2cf440b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5598daf0ba20 0x5598daf1d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:56:55.334 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.333+0000 7fd2cf440b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5598daf168d0 0x5598daf13500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:56:55.334 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.333+0000 7fd2cf440b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598daf14ba0 con 0x5598daf0b100 2024-03-23T17:56:55.334 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.333+0000 7fd2cf440b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598daf2d5f0 con 0x5598daf168d0 2024-03-23T17:56:55.334 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.333+0000 7fd2cf440b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598daf0f410 con 0x5598daf0ba20 2024-03-23T17:56:55.334 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.333+0000 7fd2b3e00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5598daf0ba20 0x5598daf1d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:56:55.334 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.333+0000 7fd2b9000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598daf0b100 0x5598daf0e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:56:55.334 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.333+0000 7fd2b3e00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5598daf0ba20 0x5598daf1d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:60528/0 (socket says 172.21.15.86:60528) 2024-03-23T17:56:55.334 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.333+0000 7fd2b3e00700 1 -- 172.21.15.86:0/3534559717 learned_addr learned my addr 172.21.15.86:0/3534559717 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:56:55.334 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.333+0000 7fd2b9a00700 1 --2- 172.21.15.86:0/3534559717 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5598daf168d0 0x5598daf13500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:56:55.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.333+0000 7fd2b3e00700 1 -- 172.21.15.86:0/3534559717 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5598daf168d0 msgr2=0x5598daf13500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:56:55.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.333+0000 7fd2b3e00700 1 --2- 172.21.15.86:0/3534559717 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5598daf168d0 0x5598daf13500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:55.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.333+0000 7fd2b3e00700 1 -- 172.21.15.86:0/3534559717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598daf0b100 msgr2=0x5598daf0e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:56:55.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.333+0000 7fd2b3e00700 1 --2- 172.21.15.86:0/3534559717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598daf0b100 0x5598daf0e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:55.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.333+0000 7fd2b3e00700 1 -- 172.21.15.86:0/3534559717 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598dac8bb00 con 0x5598daf0ba20 2024-03-23T17:56:55.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.334+0000 7fd2b3e00700 1 --2- 172.21.15.86:0/3534559717 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5598daf0ba20 0x5598daf1d5e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0005910 tx=0x7fd2a00cd040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23c267ccd1504e97 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:56:55.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.334+0000 7fd2a6a00700 1 -- 172.21.15.86:0/3534559717 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a009abb0 con 0x5598daf0ba20 2024-03-23T17:56:55.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.334+0000 7fd2a6a00700 1 -- 172.21.15.86:0/3534559717 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2a009b950 con 0x5598daf0ba20 2024-03-23T17:56:55.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.334+0000 7fd2a6a00700 1 -- 172.21.15.86:0/3534559717 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a0099680 con 0x5598daf0ba20 2024-03-23T17:56:55.339 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.334+0000 7fd2cf440b80 1 -- 172.21.15.86:0/3534559717 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5598daf0ba20 msgr2=0x5598daf1d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:56:55.339 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.334+0000 7fd2cf440b80 1 --2- 172.21.15.86:0/3534559717 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5598daf0ba20 0x5598daf1d5e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0005910 tx=0x7fd2a00cd040 comp rx=0 tx=0).stop 2024-03-23T17:56:55.339 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.334+0000 7fd2cf440b80 1 -- 172.21.15.86:0/3534559717 shutdown_connections 2024-03-23T17:56:55.339 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.334+0000 7fd2cf440b80 1 --2- 172.21.15.86:0/3534559717 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5598daf168d0 0x5598daf13500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:55.339 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.334+0000 7fd2cf440b80 1 --2- 172.21.15.86:0/3534559717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598daf0b100 0x5598daf0e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:55.339 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.335+0000 7fd2cf440b80 1 --2- 172.21.15.86:0/3534559717 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5598daf0ba20 0x5598daf1d5e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:55.339 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.335+0000 7fd2cf440b80 1 -- 172.21.15.86:0/3534559717 >> 172.21.15.86:0/3534559717 conn(0x5598dae6b640 msgr2=0x5598daf12df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:56:55.339 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.335+0000 7fd2cf440b80 1 -- 172.21.15.86:0/3534559717 shutdown_connections 2024-03-23T17:56:55.339 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.335+0000 7fd2cf440b80 1 -- 172.21.15.86:0/3534559717 wait complete. 2024-03-23T17:56:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.335+0000 7fd2cf440b80 1 -- start start 2024-03-23T17:56:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.335+0000 7fd2cf440b80 1 Processor -- start 2024-03-23T17:56:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.336+0000 7fd2cf440b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5598daf0b100 0x5598daf1b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:56:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.336+0000 7fd2cf440b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598daf0ba20 0x5598daf1bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:56:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.336+0000 7fd2cf440b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5598daf168d0 0x5598daf1c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:56:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.336+0000 7fd2cf440b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598daf77040 con 0x5598daf0ba20 2024-03-23T17:56:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.336+0000 7fd2cf440b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598daf77180 con 0x5598daf0b100 2024-03-23T17:56:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.336+0000 7fd2cf440b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598daf77340 con 0x5598daf168d0 2024-03-23T17:56:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.336+0000 7fd2cf440b80 10 client.0 _ll_register_callbacks cb 0x5598dae03410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:56:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.336+0000 7fd2b9000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5598daf0b100 0x5598daf1b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:56:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.336+0000 7fd2b9000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5598daf0b100 0x5598daf1b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:44090/0 (socket says 172.21.15.86:44090) 2024-03-23T17:56:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.336+0000 7fd2b9000700 1 -- 172.21.15.86:0/2783411423 learned_addr learned my addr 172.21.15.86:0/2783411423 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:56:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.336+0000 7fd2b9a00700 1 --2- 172.21.15.86:0/2783411423 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5598daf168d0 0x5598daf1c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:56:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.336+0000 7fd2b3e00700 1 --2- 172.21.15.86:0/2783411423 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598daf0ba20 0x5598daf1bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:56:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.336+0000 7fd2b9000700 1 -- 172.21.15.86:0/2783411423 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5598daf168d0 msgr2=0x5598daf1c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:56:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.336+0000 7fd2b9000700 1 --2- 172.21.15.86:0/2783411423 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5598daf168d0 0x5598daf1c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.336+0000 7fd2b9000700 1 -- 172.21.15.86:0/2783411423 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598daf0ba20 msgr2=0x5598daf1bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:56:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.336+0000 7fd2b9000700 1 --2- 172.21.15.86:0/2783411423 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598daf0ba20 0x5598daf1bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.336+0000 7fd2b9000700 1 -- 172.21.15.86:0/2783411423 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2b41a5f10 con 0x5598daf0b100 2024-03-23T17:56:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.336+0000 7fd2b3e00700 1 --2- 172.21.15.86:0/2783411423 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598daf0ba20 0x5598daf1bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:56:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.337+0000 7fd2b9000700 1 --2- 172.21.15.86:0/2783411423 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5598daf0b100 0x5598daf1b5c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8009cd0 tx=0x7fd2a8005e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=184f910d3e3a30fe server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:56:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.337+0000 7fd29aa00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T17:56:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.337+0000 7fd29aa00700 1 -- 172.21.15.86:0/2783411423 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a800e3b0 con 0x5598daf0b100 2024-03-23T17:56:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.337+0000 7fd2cf440b80 1 -- 172.21.15.86:0/2783411423 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd2a8009e10 con 0x5598daf0b100 2024-03-23T17:56:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.337+0000 7fd29aa00700 1 -- 172.21.15.86:0/2783411423 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fd2a8023750 con 0x5598daf0b100 2024-03-23T17:56:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.337+0000 7fd29aa00700 1 -- 172.21.15.86:0/2783411423 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2a8023990 con 0x5598daf0b100 2024-03-23T17:56:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.337+0000 7fd2cf440b80 10 client.26537 fetch_fsmap learned FSMap version 40 2024-03-23T17:56:55.342 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.337+0000 7fd2cf440b80 1 -- 172.21.15.86:0/2783411423 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5598dae23e40 con 0x5598daf0b100 2024-03-23T17:56:55.342 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.338+0000 7fd29aa00700 1 -- 172.21.15.86:0/2783411423 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2a8009e10 con 0x5598daf0b100 2024-03-23T17:56:55.342 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.338+0000 7fd29aa00700 1 -- 172.21.15.86:0/2783411423 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 40) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fd2a8011790 con 0x5598daf0b100 2024-03-23T17:56:55.342 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.338+0000 7fd2cf440b80 10 client.26537 fetch_fsmap finished waiting for FSMap version 40 2024-03-23T17:56:55.342 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.338+0000 7fd2cf440b80 10 client.26537 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:56:55.342 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.338+0000 7fd2cf440b80 20 client.26537 populate_metadata read hostname 'smithi086' 2024-03-23T17:56:55.342 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.338+0000 7fd2cf440b80 1 --2- 172.21.15.86:0/2783411423 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5598daf786e0 0x5598daf7ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:56:55.342 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.338+0000 7fd2cf440b80 4 client.26537 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T17:56:55.342 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.338+0000 7fd2cf440b80 1 -- 172.21.15.86:0/2783411423 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5598dacacc90 con 0x5598daf786e0 2024-03-23T17:56:55.342 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.338+0000 7fd2b3e00700 1 --2- 172.21.15.86:0/2783411423 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5598daf786e0 0x5598daf7ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:56:55.342 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.338+0000 7fd29aa00700 10 client.26537 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:56:55.342 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.338+0000 7fd2b3e00700 1 --2- 172.21.15.86:0/2783411423 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5598daf786e0 0x5598daf7ab00 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c73ad43cb1af507 server_cookie=ca88d4b8106b056d in_seq=0 out_seq=0 2024-03-23T17:56:55.374 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.374+0000 7fd29aa00700 1 -- 172.21.15.86:0/2783411423 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5598dacacc90 con 0x5598daf786e0 2024-03-23T17:56:55.375 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.374+0000 7fd29aa00700 10 client.26537 handle_command_reply: tid=0 2024-03-23T17:56:55.375 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.374+0000 7fd2cf440b80 1 client.26537 shutdown 2024-03-23T17:56:55.375 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.374+0000 7fd2cf440b80 1 -- 172.21.15.86:0/2783411423 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5598daf0b100 msgr2=0x5598daf1b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:56:55.375 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.374+0000 7fd2cf440b80 1 --2- 172.21.15.86:0/2783411423 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5598daf0b100 0x5598daf1b5c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8009cd0 tx=0x7fd2a8005e50 comp rx=0 tx=0).stop 2024-03-23T17:56:55.375 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.374+0000 7fd2cf440b80 1 -- 172.21.15.86:0/2783411423 shutdown_connections 2024-03-23T17:56:55.375 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.374+0000 7fd2cf440b80 1 --2- 172.21.15.86:0/2783411423 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5598daf786e0 0x5598daf7ab00 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:55.375 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.374+0000 7fd2cf440b80 1 --2- 172.21.15.86:0/2783411423 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5598daf168d0 0x5598daf1c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:55.375 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.374+0000 7fd2cf440b80 1 --2- 172.21.15.86:0/2783411423 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5598daf0b100 0x5598daf1b5c0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:55.375 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.374+0000 7fd29aa00700 0 client.26537 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:56:55.375 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.374+0000 7fd2cf440b80 1 --2- 172.21.15.86:0/2783411423 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598daf0ba20 0x5598daf1bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:55.375 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.374+0000 7fd2cf440b80 1 -- 172.21.15.86:0/2783411423 >> 172.21.15.86:0/2783411423 conn(0x5598dae6b640 msgr2=0x5598daf13410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:56:55.375 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.374+0000 7fd2cf440b80 1 -- 172.21.15.86:0/2783411423 shutdown_connections 2024-03-23T17:56:55.376 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.374+0000 7fd2cf440b80 1 -- 172.21.15.86:0/2783411423 wait complete. 2024-03-23T17:56:55.376 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.375+0000 7fd2cf440b80 20 client.26537 trim_cache size 0 max 16384 2024-03-23T17:56:55.376 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-03-23T17:56:55.376 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-03-23T17:56:55.376 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-03-23T17:56:55.376 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-03-23T17:56:55.377 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.375+0000 7fd2bba00700 1 -- 172.21.15.86:0/2603531201 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fd29c0a1290 msgr2=0x7fd29c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:56:55.377 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.375+0000 7fd2bba00700 1 --2- 172.21.15.86:0/2603531201 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fd29c0a1290 0x7fd29c0a3750 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac003e80 tx=0x7fd2ac00b040 comp rx=0 tx=0).stop 2024-03-23T17:56:55.377 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.375+0000 7fd2bba00700 1 -- 172.21.15.86:0/2603531201 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2b4106010 msgr2=0x7fd2b40735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:56:55.377 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.375+0000 7fd2bba00700 1 --2- 172.21.15.86:0/2603531201 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2b4106010 0x7fd2b40735f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0000c00 tx=0x7fd2a0005740 comp rx=0 tx=0).stop 2024-03-23T17:56:55.377 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.375+0000 7fd2bba00700 1 -- 172.21.15.86:0/2603531201 shutdown_connections 2024-03-23T17:56:55.377 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.375+0000 7fd2bba00700 1 --2- 172.21.15.86:0/2603531201 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fd29c0a1290 0x7fd29c0a3750 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:55.377 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.375+0000 7fd2bba00700 1 --2- 172.21.15.86:0/2603531201 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2b4073b30 0x7fd2b41a5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:55.377 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.375+0000 7fd2bba00700 1 --2- 172.21.15.86:0/2603531201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2b4100ef0 0x7fd2b40730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:55.377 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.375+0000 7fd2bba00700 1 --2- 172.21.15.86:0/2603531201 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2b4106010 0x7fd2b40735f0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:56:55.377 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.375+0000 7fd2bba00700 1 -- 172.21.15.86:0/2603531201 >> 172.21.15.86:0/2603531201 conn(0x7fd2b40fc750 msgr2=0x7fd2b40fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:56:55.378 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.375+0000 7fd2bba00700 1 -- 172.21.15.86:0/2603531201 shutdown_connections 2024-03-23T17:56:55.378 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:56:55.376+0000 7fd2bba00700 1 -- 172.21.15.86:0/2603531201 wait complete. 2024-03-23T17:56:55.904 INFO:tasks.workunit.client.0.smithi086.stdout: 1 313540 6.24 MB/sec execute 200 sec latency 16.208 ms 2024-03-23T17:56:56.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:56:55 smithi152 ceph-mon[47949]: pgmap v191: 97 pgs: 97 active+clean; 127 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.1 MiB/s wr, 325 op/s 2024-03-23T17:56:56.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:56:55 smithi086 ceph-mon[38221]: pgmap v191: 97 pgs: 97 active+clean; 127 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.1 MiB/s wr, 325 op/s 2024-03-23T17:56:56.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:56:55 smithi092 ceph-mon[79089]: pgmap v191: 97 pgs: 97 active+clean; 127 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.1 MiB/s wr, 325 op/s 2024-03-23T17:56:56.904 INFO:tasks.workunit.client.0.smithi086.stdout: 1 315708 6.27 MB/sec execute 201 sec latency 14.904 ms 2024-03-23T17:56:57.904 INFO:tasks.workunit.client.0.smithi086.stdout: 1 316816 6.26 MB/sec execute 202 sec latency 16.790 ms 2024-03-23T17:56:58.085 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:56:57 smithi152 ceph-mon[47949]: pgmap v192: 97 pgs: 97 active+clean; 127 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 201 op/s 2024-03-23T17:56:58.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:56:57 smithi086 ceph-mon[38221]: pgmap v192: 97 pgs: 97 active+clean; 127 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 201 op/s 2024-03-23T17:56:58.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:56:57 smithi092 ceph-mon[79089]: pgmap v192: 97 pgs: 97 active+clean; 127 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 201 op/s 2024-03-23T17:56:58.904 INFO:tasks.workunit.client.0.smithi086.stdout: 1 318300 6.25 MB/sec execute 203 sec latency 18.874 ms 2024-03-23T17:56:59.904 INFO:tasks.workunit.client.0.smithi086.stdout: 1 319700 6.28 MB/sec execute 204 sec latency 38.116 ms 2024-03-23T17:57:00.085 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:56:59 smithi152 ceph-mon[47949]: pgmap v193: 97 pgs: 97 active+clean; 127 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 201 op/s 2024-03-23T17:57:00.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:56:59 smithi086 ceph-mon[38221]: pgmap v193: 97 pgs: 97 active+clean; 127 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 201 op/s 2024-03-23T17:57:00.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:56:59 smithi092 ceph-mon[79089]: pgmap v193: 97 pgs: 97 active+clean; 127 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 201 op/s 2024-03-23T17:57:00.904 INFO:tasks.workunit.client.0.smithi086.stdout: 1 321387 6.28 MB/sec execute 205 sec latency 23.667 ms 2024-03-23T17:57:01.904 INFO:tasks.workunit.client.0.smithi086.stdout: 1 322854 6.30 MB/sec execute 206 sec latency 28.030 ms 2024-03-23T17:57:02.085 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:57:01 smithi152 ceph-mon[47949]: pgmap v194: 97 pgs: 97 active+clean; 123 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 327 op/s 2024-03-23T17:57:02.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:57:01 smithi086 ceph-mon[38221]: pgmap v194: 97 pgs: 97 active+clean; 123 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 327 op/s 2024-03-23T17:57:02.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:57:01 smithi092 ceph-mon[79089]: pgmap v194: 97 pgs: 97 active+clean; 123 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 327 op/s 2024-03-23T17:57:02.904 INFO:tasks.workunit.client.0.smithi086.stdout: 1 323930 6.29 MB/sec execute 207 sec latency 11.989 ms 2024-03-23T17:57:03.904 INFO:tasks.workunit.client.0.smithi086.stdout: 1 326002 6.31 MB/sec execute 208 sec latency 19.037 ms 2024-03-23T17:57:04.085 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:57:03 smithi152 ceph-mon[47949]: pgmap v195: 97 pgs: 97 active+clean; 123 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 228 op/s 2024-03-23T17:57:04.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:57:03 smithi086 ceph-mon[38221]: pgmap v195: 97 pgs: 97 active+clean; 123 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 228 op/s 2024-03-23T17:57:04.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:57:03 smithi092 ceph-mon[79089]: pgmap v195: 97 pgs: 97 active+clean; 123 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 228 op/s 2024-03-23T17:57:04.905 INFO:tasks.workunit.client.0.smithi086.stdout: 1 327145 6.31 MB/sec execute 209 sec latency 22.330 ms 2024-03-23T17:57:05.904 INFO:tasks.workunit.client.0.smithi086.stdout: 1 328781 6.31 MB/sec execute 210 sec latency 73.063 ms 2024-03-23T17:57:06.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:57:05 smithi152 ceph-mon[47949]: pgmap v196: 97 pgs: 97 active+clean; 123 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.1 MiB/s wr, 363 op/s 2024-03-23T17:57:06.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:57:05 smithi152 ceph-mon[47949]: osdmap e84: 12 total, 12 up, 12 in 2024-03-23T17:57:06.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:57:05 smithi086 ceph-mon[38221]: pgmap v196: 97 pgs: 97 active+clean; 123 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.1 MiB/s wr, 363 op/s 2024-03-23T17:57:06.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:57:05 smithi086 ceph-mon[38221]: osdmap e84: 12 total, 12 up, 12 in 2024-03-23T17:57:06.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:57:05 smithi092 ceph-mon[79089]: pgmap v196: 97 pgs: 97 active+clean; 123 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.1 MiB/s wr, 363 op/s 2024-03-23T17:57:06.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:57:05 smithi092 ceph-mon[79089]: osdmap e84: 12 total, 12 up, 12 in 2024-03-23T17:57:06.904 INFO:tasks.workunit.client.0.smithi086.stdout: 1 330321 6.34 MB/sec execute 211 sec latency 11.457 ms 2024-03-23T17:57:07.904 INFO:tasks.workunit.client.0.smithi086.stdout: 1 331525 6.34 MB/sec execute 212 sec latency 17.358 ms 2024-03-23T17:57:08.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:57:07 smithi086 ceph-mon[38221]: pgmap v198: 97 pgs: 97 active+clean; 123 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 313 op/s 2024-03-23T17:57:08.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:57:07 smithi092 ceph-mon[79089]: pgmap v198: 97 pgs: 97 active+clean; 123 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 313 op/s 2024-03-23T17:57:08.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:57:07 smithi152 ceph-mon[47949]: pgmap v198: 97 pgs: 97 active+clean; 123 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 313 op/s 2024-03-23T17:57:08.529 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.529+0000 7f5f12600700 1 -- 172.21.15.86:0/3881343804 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f0c101890 msgr2=0x7f5f0c1058d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:57:08.529 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.529+0000 7f5f12600700 1 --2- 172.21.15.86:0/3881343804 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f0c101890 0x7f5f0c1058d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5ef80098f0 tx=0x7f5ef8009c00 comp rx=0 tx=0).stop 2024-03-23T17:57:08.529 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.529+0000 7f5f12600700 1 -- 172.21.15.86:0/3881343804 shutdown_connections 2024-03-23T17:57:08.529 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.529+0000 7f5f12600700 1 --2- 172.21.15.86:0/3881343804 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f0c106000 0x7f5f0c10db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:08.529 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.529+0000 7f5f12600700 1 --2- 172.21.15.86:0/3881343804 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f0c100ee0 0x7f5f0c1012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:08.529 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.529+0000 7f5f12600700 1 --2- 172.21.15.86:0/3881343804 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f0c101890 0x7f5f0c1058d0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:08.529 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.529+0000 7f5f12600700 1 -- 172.21.15.86:0/3881343804 >> 172.21.15.86:0/3881343804 conn(0x7f5f0c0fc780 msgr2=0x7f5f0c0feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:57:08.529 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.529+0000 7f5f12600700 1 -- 172.21.15.86:0/3881343804 shutdown_connections 2024-03-23T17:57:08.530 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.529+0000 7f5f12600700 1 -- 172.21.15.86:0/3881343804 wait complete. 2024-03-23T17:57:08.531 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.530+0000 7f5f12600700 1 Processor -- start 2024-03-23T17:57:08.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.530+0000 7f5f12600700 1 -- start start 2024-03-23T17:57:08.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.530+0000 7f5f12600700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f0c100ee0 0x7f5f0c0730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:57:08.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.530+0000 7f5f12600700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f0c106000 0x7f5f0c0735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:57:08.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.530+0000 7f5f12600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f0c073b30 0x7f5f0c1a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:57:08.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.530+0000 7f5f12600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0c1a5a00 con 0x7f5f0c073b30 2024-03-23T17:57:08.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.530+0000 7f5f12600700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0c1a5b70 con 0x7f5f0c106000 2024-03-23T17:57:08.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.530+0000 7f5f12600700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0c1a5d30 con 0x7f5f0c100ee0 2024-03-23T17:57:08.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.530+0000 7f5f0aa00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f0c106000 0x7f5f0c0735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:57:08.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.530+0000 7f5f0b400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f0c100ee0 0x7f5f0c0730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:57:08.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.530+0000 7f5f0aa00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f0c106000 0x7f5f0c0735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:46292/0 (socket says 172.21.15.86:46292) 2024-03-23T17:57:08.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.530+0000 7f5f0aa00700 1 -- 172.21.15.86:0/374841848 learned_addr learned my addr 172.21.15.86:0/374841848 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:57:08.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.530+0000 7f5f0be00700 1 --2- 172.21.15.86:0/374841848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f0c073b30 0x7f5f0c1a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:57:08.533 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.531+0000 7f5f0b400700 1 -- 172.21.15.86:0/374841848 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f0c106000 msgr2=0x7f5f0c0735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:57:08.533 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.531+0000 7f5f0b400700 1 --2- 172.21.15.86:0/374841848 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f0c106000 0x7f5f0c0735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:08.533 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.531+0000 7f5f0b400700 1 -- 172.21.15.86:0/374841848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f0c073b30 msgr2=0x7f5f0c1a53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:57:08.533 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.531+0000 7f5f0b400700 1 --2- 172.21.15.86:0/374841848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f0c073b30 0x7f5f0c1a53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:08.533 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.531+0000 7f5f0b400700 1 -- 172.21.15.86:0/374841848 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ef8009520 con 0x7f5f0c100ee0 2024-03-23T17:57:08.533 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.531+0000 7f5f0b400700 1 --2- 172.21.15.86:0/374841848 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f0c100ee0 0x7f5f0c0730b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5efc00e9f0 tx=0x7f5efc00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:57:08.533 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.531+0000 7f5f03e00700 1 -- 172.21.15.86:0/374841848 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5efc00c9c0 con 0x7f5f0c100ee0 2024-03-23T17:57:08.533 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.531+0000 7f5f03e00700 1 -- 172.21.15.86:0/374841848 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5efc010430 con 0x7f5f0c100ee0 2024-03-23T17:57:08.533 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.531+0000 7f5f03e00700 1 -- 172.21.15.86:0/374841848 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5efc018680 con 0x7f5f0c100ee0 2024-03-23T17:57:08.533 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.531+0000 7f5f12600700 1 -- 172.21.15.86:0/374841848 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f0c1a5fb0 con 0x7f5f0c100ee0 2024-03-23T17:57:08.534 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.532+0000 7f5f12600700 1 -- 172.21.15.86:0/374841848 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f0c1a6440 con 0x7f5f0c100ee0 2024-03-23T17:57:08.535 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.535+0000 7f5f03e00700 1 -- 172.21.15.86:0/374841848 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f5efc010860 con 0x7f5f0c100ee0 2024-03-23T17:57:08.536 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.536+0000 7f5f26209b80 1 Processor -- start 2024-03-23T17:57:08.536 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.536+0000 7f5f26209b80 1 -- start start 2024-03-23T17:57:08.536 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.536+0000 7f5f03e00700 1 --2- 172.21.15.86:0/374841848 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f5ef00a1390 0x7f5ef00a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:57:08.538 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.536+0000 7f5f03e00700 1 -- 172.21.15.86:0/374841848 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f5efc014070 con 0x7f5f0c100ee0 2024-03-23T17:57:08.538 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.536+0000 7f5f26209b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609cbcbe100 0x5609cbcbe4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:57:08.538 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.536+0000 7f5f26209b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609cbcbea20 0x5609cbcc9390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:57:08.538 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.536+0000 7f5f26209b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5609cbcc98d0 0x5609cbccbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:57:08.538 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.536+0000 7f5f26209b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609cbc17930 con 0x5609cbcbea20 2024-03-23T17:57:08.538 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.536+0000 7f5f26209b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609cbc152f0 con 0x5609cbcc98d0 2024-03-23T17:57:08.538 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.536+0000 7f5f26209b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609cbc10be0 con 0x5609cbcbe100 2024-03-23T17:57:08.538 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.537+0000 7f5f0be00700 1 --2- 172.21.15.86:0/374841848 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f5ef00a1390 0x7f5ef00a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:57:08.538 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.537+0000 7f5f0b400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609cbcbe100 0x5609cbcbe4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:57:08.538 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.537+0000 7f5f0be00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5609cbcc98d0 0x5609cbccbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:57:08.539 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.537+0000 7f5f0b400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609cbcbe100 0x5609cbcbe4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:38618/0 (socket says 172.21.15.86:38618) 2024-03-23T17:57:08.539 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.537+0000 7f5f0b400700 1 -- 172.21.15.86:0/2561344486 learned_addr learned my addr 172.21.15.86:0/2561344486 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:57:08.539 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.537+0000 7f5f0aa00700 1 --2- 172.21.15.86:0/2561344486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609cbcbea20 0x5609cbcc9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:57:08.539 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.537+0000 7f5f0b400700 1 -- 172.21.15.86:0/2561344486 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5609cbcc98d0 msgr2=0x5609cbccbcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:57:08.539 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.537+0000 7f5f0b400700 1 --2- 172.21.15.86:0/2561344486 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5609cbcc98d0 0x5609cbccbcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:08.539 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.537+0000 7f5f0b400700 1 -- 172.21.15.86:0/2561344486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609cbcbea20 msgr2=0x5609cbcc9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:57:08.539 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.537+0000 7f5f0b400700 1 --2- 172.21.15.86:0/2561344486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609cbcbea20 0x5609cbcc9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:08.539 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.537+0000 7f5f0b400700 1 -- 172.21.15.86:0/2561344486 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609cba92350 con 0x5609cbcbe100 2024-03-23T17:57:08.539 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.537+0000 7f5f0aa00700 1 --2- 172.21.15.86:0/2561344486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609cbcbea20 0x5609cbcc9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:57:08.539 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.537+0000 7f5f0be00700 1 --2- 172.21.15.86:0/374841848 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f5ef00a1390 0x7f5ef00a3850 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5f04008be0 tx=0x7f5f040092d0 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:57:08.539 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.537+0000 7f5f0b400700 1 --2- 172.21.15.86:0/2561344486 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609cbcbe100 0x5609cbcbe4e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5efc0968e0 tx=0x7f5efc0998a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc4dc2f8394253ad server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:57:08.539 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.538+0000 7f5f00c00700 1 -- 172.21.15.86:0/2561344486 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5efc099e70 con 0x5609cbcbe100 2024-03-23T17:57:08.539 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.538+0000 7f5f00c00700 1 -- 172.21.15.86:0/2561344486 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5efc09dca0 con 0x5609cbcbe100 2024-03-23T17:57:08.539 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.538+0000 7f5f00c00700 1 -- 172.21.15.86:0/2561344486 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5efc0995c0 con 0x5609cbcbe100 2024-03-23T17:57:08.543 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.538+0000 7f5f26209b80 1 -- 172.21.15.86:0/2561344486 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609cbcbe100 msgr2=0x5609cbcbe4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:57:08.543 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.538+0000 7f5f26209b80 1 --2- 172.21.15.86:0/2561344486 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609cbcbe100 0x5609cbcbe4e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5efc0968e0 tx=0x7f5efc0998a0 comp rx=0 tx=0).stop 2024-03-23T17:57:08.543 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.538+0000 7f5f26209b80 1 -- 172.21.15.86:0/2561344486 shutdown_connections 2024-03-23T17:57:08.543 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.538+0000 7f5f26209b80 1 --2- 172.21.15.86:0/2561344486 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5609cbcc98d0 0x5609cbccbcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:08.543 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.538+0000 7f5f26209b80 1 --2- 172.21.15.86:0/2561344486 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609cbcbe100 0x5609cbcbe4e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:08.543 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.538+0000 7f5f26209b80 1 --2- 172.21.15.86:0/2561344486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609cbcbea20 0x5609cbcc9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:08.543 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.538+0000 7f5f26209b80 1 -- 172.21.15.86:0/2561344486 >> 172.21.15.86:0/2561344486 conn(0x5609cbc1e640 msgr2=0x5609cbcba520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:57:08.543 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.539+0000 7f5f26209b80 1 -- 172.21.15.86:0/2561344486 shutdown_connections 2024-03-23T17:57:08.543 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.539+0000 7f5f26209b80 1 -- 172.21.15.86:0/2561344486 wait complete. 2024-03-23T17:57:08.543 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.539+0000 7f5f26209b80 1 -- start start 2024-03-23T17:57:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.539+0000 7f5f26209b80 1 Processor -- start 2024-03-23T17:57:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.540+0000 7f5f26209b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5609cbcbe100 0x5609cbcc9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:57:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.540+0000 7f5f26209b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609cbcbea20 0x5609cbcc6550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:57:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.540+0000 7f5f26209b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609cbcc98d0 0x5609cbcc6b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:57:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.540+0000 7f5f26209b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609cbbacd20 con 0x5609cbcbea20 2024-03-23T17:57:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.540+0000 7f5f26209b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609cbbac9c0 con 0x5609cbcbe100 2024-03-23T17:57:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.540+0000 7f5f26209b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609cba94640 con 0x5609cbcc98d0 2024-03-23T17:57:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.540+0000 7f5f26209b80 10 client.0 _ll_register_callbacks cb 0x5609cbbb6410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:57:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.540+0000 7f5f0b400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5609cbcbe100 0x5609cbcc9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:57:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.540+0000 7f5f0b400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5609cbcbe100 0x5609cbcc9260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:46304/0 (socket says 172.21.15.86:46304) 2024-03-23T17:57:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.540+0000 7f5f0be00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609cbcc98d0 0x5609cbcc6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:57:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.541+0000 7f5f0b400700 1 -- 172.21.15.86:0/707994889 learned_addr learned my addr 172.21.15.86:0/707994889 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:57:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.541+0000 7f5f0aa00700 1 --2- 172.21.15.86:0/707994889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609cbcbea20 0x5609cbcc6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:57:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.541+0000 7f5f0b400700 1 -- 172.21.15.86:0/707994889 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609cbcc98d0 msgr2=0x5609cbcc6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:57:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.541+0000 7f5f0b400700 1 --2- 172.21.15.86:0/707994889 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609cbcc98d0 0x5609cbcc6b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.541+0000 7f5f0b400700 1 -- 172.21.15.86:0/707994889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609cbcbea20 msgr2=0x5609cbcc6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:57:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.541+0000 7f5f0b400700 1 --2- 172.21.15.86:0/707994889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609cbcbea20 0x5609cbcc6550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.541+0000 7f5f0b400700 1 -- 172.21.15.86:0/707994889 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f0c1a6440 con 0x5609cbcbe100 2024-03-23T17:57:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.541+0000 7f5f0aa00700 1 --2- 172.21.15.86:0/707994889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609cbcbea20 0x5609cbcc6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:57:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.541+0000 7f5f0b400700 1 --2- 172.21.15.86:0/707994889 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5609cbcbe100 0x5609cbcc9260 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f5efc0a1dc0 tx=0x7f5efc0107d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb607db30a3a812 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:57:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.541+0000 7f5ef5600700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T17:57:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.541+0000 7f5ef5600700 1 -- 172.21.15.86:0/707994889 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5efc099e70 con 0x5609cbcbe100 2024-03-23T17:57:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.541+0000 7f5f26209b80 1 -- 172.21.15.86:0/707994889 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5609cbbd1320 con 0x5609cbcbe100 2024-03-23T17:57:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.541+0000 7f5ef5600700 1 -- 172.21.15.86:0/707994889 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f5efc09a2e0 con 0x5609cbcbe100 2024-03-23T17:57:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.542+0000 7f5ef5600700 1 -- 172.21.15.86:0/707994889 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5efc0a1e60 con 0x5609cbcbe100 2024-03-23T17:57:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.542+0000 7f5f26209b80 10 client.26549 fetch_fsmap learned FSMap version 40 2024-03-23T17:57:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.542+0000 7f5f26209b80 1 -- 172.21.15.86:0/707994889 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5609cba3eb00 con 0x5609cbcbe100 2024-03-23T17:57:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.542+0000 7f5ef5600700 1 -- 172.21.15.86:0/707994889 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5609cbbd1320 con 0x5609cbcbe100 2024-03-23T17:57:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.542+0000 7f5ef5600700 1 -- 172.21.15.86:0/707994889 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 40) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f5efc07e020 con 0x5609cbcbe100 2024-03-23T17:57:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.542+0000 7f5f26209b80 10 client.26549 fetch_fsmap finished waiting for FSMap version 40 2024-03-23T17:57:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.542+0000 7f5f26209b80 10 client.26549 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:57:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.542+0000 7f5f26209b80 20 client.26549 populate_metadata read hostname 'smithi086' 2024-03-23T17:57:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.542+0000 7f5f26209b80 1 --2- 172.21.15.86:0/707994889 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5609cbce1a30 0x5609cbce3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:57:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.542+0000 7f5f26209b80 4 client.26549 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:57:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.542+0000 7f5f26209b80 1 -- 172.21.15.86:0/707994889 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5609cbc096a0 con 0x5609cbce1a30 2024-03-23T17:57:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.542+0000 7f5f0aa00700 1 --2- 172.21.15.86:0/707994889 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5609cbce1a30 0x5609cbce3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:57:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.543+0000 7f5f0aa00700 1 --2- 172.21.15.86:0/707994889 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5609cbce1a30 0x5609cbce3e50 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b854e30c38bbd162 server_cookie=e90f90dc55a434ca in_seq=0 out_seq=0 2024-03-23T17:57:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.543+0000 7f5ef5600700 10 client.26549 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:57:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.543+0000 7f5ef5600700 1 -- 172.21.15.86:0/707994889 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5609cbc096a0 con 0x5609cbce1a30 2024-03-23T17:57:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.543+0000 7f5ef5600700 10 client.26549 handle_command_reply: tid=0 2024-03-23T17:57:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.544+0000 7f5f26209b80 1 client.26549 shutdown 2024-03-23T17:57:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.544+0000 7f5f26209b80 1 -- 172.21.15.86:0/707994889 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5609cbcbe100 msgr2=0x5609cbcc9260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:57:08.548 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.544+0000 7f5f26209b80 1 --2- 172.21.15.86:0/707994889 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5609cbcbe100 0x5609cbcc9260 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f5efc0a1dc0 tx=0x7f5efc0107d0 comp rx=0 tx=0).stop 2024-03-23T17:57:08.548 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.544+0000 7f5f26209b80 1 -- 172.21.15.86:0/707994889 shutdown_connections 2024-03-23T17:57:08.548 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.544+0000 7f5f26209b80 1 --2- 172.21.15.86:0/707994889 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5609cbce1a30 0x5609cbce3e50 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:08.548 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.544+0000 7f5f26209b80 1 --2- 172.21.15.86:0/707994889 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609cbcc98d0 0x5609cbcc6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:08.548 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.544+0000 7f5f26209b80 1 --2- 172.21.15.86:0/707994889 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5609cbcbe100 0x5609cbcc9260 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:08.548 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.544+0000 7f5ef5600700 0 client.26549 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:57:08.548 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.544+0000 7f5f26209b80 1 --2- 172.21.15.86:0/707994889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609cbcbea20 0x5609cbcc6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:08.548 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.544+0000 7f5f26209b80 1 -- 172.21.15.86:0/707994889 >> 172.21.15.86:0/707994889 conn(0x5609cbc1e640 msgr2=0x5609cbcc0040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:57:08.548 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.544+0000 7f5f26209b80 1 -- 172.21.15.86:0/707994889 shutdown_connections 2024-03-23T17:57:08.548 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.544+0000 7f5f26209b80 1 -- 172.21.15.86:0/707994889 wait complete. 2024-03-23T17:57:08.548 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.544+0000 7f5f26209b80 20 client.26549 trim_cache size 0 max 16384 2024-03-23T17:57:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.560+0000 7f5f26209b80 1 Processor -- start 2024-03-23T17:57:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.560+0000 7f5f26209b80 1 -- start start 2024-03-23T17:57:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.561+0000 7f5f26209b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609cbcbe100 0x5609cbcc0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:57:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.561+0000 7f5f26209b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5609cbcbea20 0x5609cbcd8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:57:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.561+0000 7f5f26209b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609cbcc98d0 0x5609cbcc0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:57:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.561+0000 7f5f26209b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609cbce8b70 con 0x5609cbcc98d0 2024-03-23T17:57:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.561+0000 7f5f26209b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609cbcc4aa0 con 0x5609cbcbea20 2024-03-23T17:57:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.561+0000 7f5f26209b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609cbcc23b0 con 0x5609cbcbe100 2024-03-23T17:57:08.562 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.561+0000 7f5f0b400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609cbcbe100 0x5609cbcc0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:57:08.562 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.561+0000 7f5f0aa00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5609cbcbea20 0x5609cbcd8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:57:08.562 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.561+0000 7f5f0b400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609cbcbe100 0x5609cbcc0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:38626/0 (socket says 172.21.15.86:38626) 2024-03-23T17:57:08.562 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.561+0000 7f5f0be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609cbcc98d0 0x5609cbcc0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:57:08.562 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.561+0000 7f5f0b400700 1 -- 172.21.15.86:0/4087999790 learned_addr learned my addr 172.21.15.86:0/4087999790 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:57:08.562 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.561+0000 7f5f0aa00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5609cbcbea20 0x5609cbcd8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:46310/0 (socket says 172.21.15.86:46310) 2024-03-23T17:57:08.562 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.561+0000 7f5f0b400700 1 -- 172.21.15.86:0/4087999790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5609cbcbea20 msgr2=0x5609cbcd8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:57:08.562 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.561+0000 7f5f0b400700 1 --2- 172.21.15.86:0/4087999790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5609cbcbea20 0x5609cbcd8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:08.562 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.561+0000 7f5f0b400700 1 -- 172.21.15.86:0/4087999790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609cbcc98d0 msgr2=0x5609cbcc0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:57:08.562 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.561+0000 7f5f0b400700 1 --2- 172.21.15.86:0/4087999790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609cbcc98d0 0x5609cbcc0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:08.562 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.561+0000 7f5f0b400700 1 -- 172.21.15.86:0/4087999790 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609cbbd6e40 con 0x5609cbcbe100 2024-03-23T17:57:08.562 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.561+0000 7f5f0aa00700 1 --2- 172.21.15.86:0/4087999790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5609cbcbea20 0x5609cbcd8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:57:08.564 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.562+0000 7f5f0b400700 1 --2- 172.21.15.86:0/4087999790 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609cbcbe100 0x5609cbcc0750 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5efc009800 tx=0x7f5efc0a22f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd7cb78f17ff5422 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:57:08.564 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.562+0000 7f5f01600700 1 -- 172.21.15.86:0/4087999790 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5efc095fe0 con 0x5609cbcbe100 2024-03-23T17:57:08.564 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.562+0000 7f5f01600700 1 -- 172.21.15.86:0/4087999790 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5efc099e70 con 0x5609cbcbe100 2024-03-23T17:57:08.564 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.562+0000 7f5f01600700 1 -- 172.21.15.86:0/4087999790 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5efc0a1e60 con 0x5609cbcbe100 2024-03-23T17:57:08.564 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.563+0000 7f5f26209b80 1 -- 172.21.15.86:0/4087999790 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609cbcbe100 msgr2=0x5609cbcc0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:57:08.564 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.563+0000 7f5f26209b80 1 --2- 172.21.15.86:0/4087999790 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609cbcbe100 0x5609cbcc0750 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5efc009800 tx=0x7f5efc0a22f0 comp rx=0 tx=0).stop 2024-03-23T17:57:08.564 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.563+0000 7f5f26209b80 1 -- 172.21.15.86:0/4087999790 shutdown_connections 2024-03-23T17:57:08.564 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.563+0000 7f5f26209b80 1 --2- 172.21.15.86:0/4087999790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609cbcc98d0 0x5609cbcc0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:08.564 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.563+0000 7f5f26209b80 1 --2- 172.21.15.86:0/4087999790 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609cbcbe100 0x5609cbcc0750 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:08.564 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.563+0000 7f5f26209b80 1 --2- 172.21.15.86:0/4087999790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5609cbcbea20 0x5609cbcd8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:08.564 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.563+0000 7f5f26209b80 1 -- 172.21.15.86:0/4087999790 >> 172.21.15.86:0/4087999790 conn(0x5609cbc1e640 msgr2=0x5609cbcc0040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:57:08.565 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.563+0000 7f5f26209b80 1 -- 172.21.15.86:0/4087999790 shutdown_connections 2024-03-23T17:57:08.565 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.563+0000 7f5f26209b80 1 -- 172.21.15.86:0/4087999790 wait complete. 2024-03-23T17:57:08.565 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.563+0000 7f5f26209b80 1 -- start start 2024-03-23T17:57:08.565 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.564+0000 7f5f26209b80 1 Processor -- start 2024-03-23T17:57:08.567 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.564+0000 7f5f26209b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5609cbcbe100 0x5609cbcc7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:57:08.567 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.564+0000 7f5f26209b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609cbcbea20 0x5609cbcc8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:57:08.567 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.564+0000 7f5f26209b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609cbcc98d0 0x5609cbd2c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:57:08.567 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.564+0000 7f5f26209b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609cbd2cbb0 con 0x5609cbcbea20 2024-03-23T17:57:08.567 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.564+0000 7f5f26209b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609cbd2cd20 con 0x5609cbcbe100 2024-03-23T17:57:08.567 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.564+0000 7f5f26209b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609cbd2cee0 con 0x5609cbcc98d0 2024-03-23T17:57:08.567 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.565+0000 7f5f26209b80 10 client.0 _ll_register_callbacks cb 0x5609cbbb6410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:57:08.568 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.565+0000 7f5f0b400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5609cbcbe100 0x5609cbcc7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:57:08.568 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.565+0000 7f5f0b400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5609cbcbe100 0x5609cbcc7fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:46318/0 (socket says 172.21.15.86:46318) 2024-03-23T17:57:08.568 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.565+0000 7f5f0b400700 1 -- 172.21.15.86:0/3030820115 learned_addr learned my addr 172.21.15.86:0/3030820115 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:57:08.568 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.565+0000 7f5f0aa00700 1 --2- 172.21.15.86:0/3030820115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609cbcbea20 0x5609cbcc8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:57:08.568 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.565+0000 7f5f0be00700 1 --2- 172.21.15.86:0/3030820115 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609cbcc98d0 0x5609cbd2c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:57:08.568 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.565+0000 7f5f0b400700 1 -- 172.21.15.86:0/3030820115 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609cbcc98d0 msgr2=0x5609cbd2c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:57:08.568 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.565+0000 7f5f0b400700 1 --2- 172.21.15.86:0/3030820115 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609cbcc98d0 0x5609cbd2c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:08.568 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.565+0000 7f5f0b400700 1 -- 172.21.15.86:0/3030820115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609cbcbea20 msgr2=0x5609cbcc8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:57:08.568 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.565+0000 7f5f0b400700 1 --2- 172.21.15.86:0/3030820115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609cbcbea20 0x5609cbcc8520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:08.568 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.565+0000 7f5f0b400700 1 -- 172.21.15.86:0/3030820115 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f0c10e3f0 con 0x5609cbcbe100 2024-03-23T17:57:08.568 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.565+0000 7f5f0b400700 1 --2- 172.21.15.86:0/3030820115 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5609cbcbe100 0x5609cbcc7fe0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5efc0968e0 tx=0x7f5efc099120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ec329a9a2be257a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:57:08.568 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.565+0000 7f5ef5600700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T17:57:08.568 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.566+0000 7f5ef5600700 1 -- 172.21.15.86:0/3030820115 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5efc09a2e0 con 0x5609cbcbe100 2024-03-23T17:57:08.568 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.566+0000 7f5ef5600700 1 -- 172.21.15.86:0/3030820115 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f5efc095e70 con 0x5609cbcbe100 2024-03-23T17:57:08.569 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.566+0000 7f5f26209b80 1 -- 172.21.15.86:0/3030820115 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5efc09dca0 con 0x5609cbcbe100 2024-03-23T17:57:08.569 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.566+0000 7f5ef5600700 1 -- 172.21.15.86:0/3030820115 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5efc00cb20 con 0x5609cbcbe100 2024-03-23T17:57:08.569 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.566+0000 7f5f26209b80 10 client.26555 fetch_fsmap learned FSMap version 40 2024-03-23T17:57:08.569 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.566+0000 7f5f26209b80 1 -- 172.21.15.86:0/3030820115 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5609cba3eb00 con 0x5609cbcbe100 2024-03-23T17:57:08.569 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.566+0000 7f5ef5600700 1 -- 172.21.15.86:0/3030820115 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5efc09dca0 con 0x5609cbcbe100 2024-03-23T17:57:08.569 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.566+0000 7f5ef5600700 1 -- 172.21.15.86:0/3030820115 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 40) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f5efc07e070 con 0x5609cbcbe100 2024-03-23T17:57:08.569 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.566+0000 7f5f26209b80 10 client.26555 fetch_fsmap finished waiting for FSMap version 40 2024-03-23T17:57:08.569 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.566+0000 7f5f26209b80 10 client.26555 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:57:08.569 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.566+0000 7f5f26209b80 20 client.26555 populate_metadata read hostname 'smithi086' 2024-03-23T17:57:08.569 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.566+0000 7f5f26209b80 1 --2- 172.21.15.86:0/3030820115 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5609cbd341b0 0x5609cbd365a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:57:08.569 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.566+0000 7f5f26209b80 4 client.26555 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T17:57:08.569 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.566+0000 7f5f26209b80 1 -- 172.21.15.86:0/3030820115 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5609cba5fc90 con 0x5609cbd341b0 2024-03-23T17:57:08.569 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.567+0000 7f5f0aa00700 1 --2- 172.21.15.86:0/3030820115 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5609cbd341b0 0x5609cbd365a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:57:08.569 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.567+0000 7f5f0aa00700 1 --2- 172.21.15.86:0/3030820115 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5609cbd341b0 0x5609cbd365a0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf1aba0b8af10ec0 server_cookie=14d5ecc87e01627b in_seq=0 out_seq=0 2024-03-23T17:57:08.569 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.567+0000 7f5ef5600700 10 client.26555 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:57:08.605 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.605+0000 7f5ef5600700 1 -- 172.21.15.86:0/3030820115 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5609cba5fc90 con 0x5609cbd341b0 2024-03-23T17:57:08.605 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.605+0000 7f5ef5600700 10 client.26555 handle_command_reply: tid=0 2024-03-23T17:57:08.605 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.605+0000 7f5f26209b80 1 client.26555 shutdown 2024-03-23T17:57:08.605 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.605+0000 7f5f26209b80 1 -- 172.21.15.86:0/3030820115 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5609cbcbe100 msgr2=0x5609cbcc7fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:57:08.606 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.605+0000 7f5f26209b80 1 --2- 172.21.15.86:0/3030820115 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5609cbcbe100 0x5609cbcc7fe0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5efc0968e0 tx=0x7f5efc099120 comp rx=0 tx=0).stop 2024-03-23T17:57:08.606 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.605+0000 7f5f26209b80 1 -- 172.21.15.86:0/3030820115 shutdown_connections 2024-03-23T17:57:08.606 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.605+0000 7f5f26209b80 1 --2- 172.21.15.86:0/3030820115 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5609cbd341b0 0x5609cbd365a0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:08.606 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.605+0000 7f5f26209b80 1 --2- 172.21.15.86:0/3030820115 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609cbcc98d0 0x5609cbd2c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:08.606 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.605+0000 7f5f26209b80 1 --2- 172.21.15.86:0/3030820115 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5609cbcbe100 0x5609cbcc7fe0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:08.606 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.605+0000 7f5f26209b80 1 --2- 172.21.15.86:0/3030820115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609cbcbea20 0x5609cbcc8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:08.606 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.605+0000 7f5f26209b80 1 -- 172.21.15.86:0/3030820115 >> 172.21.15.86:0/3030820115 conn(0x5609cbc1e640 msgr2=0x5609cbcb9d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:57:08.606 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.605+0000 7f5ef5600700 0 client.26555 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:57:08.607 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.606+0000 7f5f26209b80 1 -- 172.21.15.86:0/3030820115 shutdown_connections 2024-03-23T17:57:08.607 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.606+0000 7f5f26209b80 1 -- 172.21.15.86:0/3030820115 wait complete. 2024-03-23T17:57:08.607 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.606+0000 7f5f26209b80 20 client.26555 trim_cache size 0 max 16384 2024-03-23T17:57:08.607 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-03-23T17:57:08.607 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-03-23T17:57:08.607 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-03-23T17:57:08.607 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-03-23T17:57:08.608 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.607+0000 7f5f12600700 1 -- 172.21.15.86:0/374841848 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f5ef00a1390 msgr2=0x7f5ef00a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:57:08.608 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.607+0000 7f5f12600700 1 --2- 172.21.15.86:0/374841848 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f5ef00a1390 0x7f5ef00a3850 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5f04008be0 tx=0x7f5f040092d0 comp rx=0 tx=0).stop 2024-03-23T17:57:08.608 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.607+0000 7f5f12600700 1 -- 172.21.15.86:0/374841848 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f0c100ee0 msgr2=0x7f5f0c0730b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:57:08.608 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.607+0000 7f5f12600700 1 --2- 172.21.15.86:0/374841848 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f0c100ee0 0x7f5f0c0730b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5efc00e9f0 tx=0x7f5efc00ed00 comp rx=0 tx=0).stop 2024-03-23T17:57:08.608 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.607+0000 7f5f12600700 1 -- 172.21.15.86:0/374841848 shutdown_connections 2024-03-23T17:57:08.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.607+0000 7f5f12600700 1 --2- 172.21.15.86:0/374841848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f0c073b30 0x7f5f0c1a53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:08.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.607+0000 7f5f12600700 1 --2- 172.21.15.86:0/374841848 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f5ef00a1390 0x7f5ef00a3850 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:08.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.607+0000 7f5f12600700 1 --2- 172.21.15.86:0/374841848 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f0c100ee0 0x7f5f0c0730b0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:08.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.607+0000 7f5f12600700 1 --2- 172.21.15.86:0/374841848 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f0c106000 0x7f5f0c0735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:08.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.607+0000 7f5f12600700 1 -- 172.21.15.86:0/374841848 >> 172.21.15.86:0/374841848 conn(0x7f5f0c0fc780 msgr2=0x7f5f0c0fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:57:08.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.607+0000 7f5f12600700 1 -- 172.21.15.86:0/374841848 shutdown_connections 2024-03-23T17:57:08.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:08.607+0000 7f5f12600700 1 -- 172.21.15.86:0/374841848 wait complete. 2024-03-23T17:57:08.904 INFO:tasks.workunit.client.0.smithi086.stdout: 1 333551 6.37 MB/sec execute 213 sec latency 30.089 ms 2024-03-23T17:57:09.904 INFO:tasks.workunit.client.0.smithi086.stdout: 1 334524 6.35 MB/sec execute 214 sec latency 46.330 ms 2024-03-23T17:57:10.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:57:09 smithi086 ceph-mon[38221]: pgmap v199: 97 pgs: 97 active+clean; 123 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 313 op/s 2024-03-23T17:57:10.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:57:09 smithi092 ceph-mon[79089]: pgmap v199: 97 pgs: 97 active+clean; 123 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 313 op/s 2024-03-23T17:57:10.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:57:09 smithi152 ceph-mon[47949]: pgmap v199: 97 pgs: 97 active+clean; 123 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 313 op/s 2024-03-23T17:57:10.904 INFO:tasks.workunit.client.0.smithi086.stdout: 1 336446 6.37 MB/sec execute 215 sec latency 19.249 ms 2024-03-23T17:57:11.905 INFO:tasks.workunit.client.0.smithi086.stdout: 1 337709 6.37 MB/sec execute 216 sec latency 16.539 ms 2024-03-23T17:57:12.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:57:11 smithi086 ceph-mon[38221]: pgmap v200: 97 pgs: 97 active+clean; 122 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 305 op/s 2024-03-23T17:57:12.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:57:11 smithi092 ceph-mon[79089]: pgmap v200: 97 pgs: 97 active+clean; 122 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 305 op/s 2024-03-23T17:57:12.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:57:11 smithi152 ceph-mon[47949]: pgmap v200: 97 pgs: 97 active+clean; 122 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 305 op/s 2024-03-23T17:57:12.905 INFO:tasks.workunit.client.0.smithi086.stdout: 1 338893 6.37 MB/sec execute 217 sec latency 15.333 ms 2024-03-23T17:57:13.905 INFO:tasks.workunit.client.0.smithi086.stdout: 1 340782 6.40 MB/sec execute 218 sec latency 14.564 ms 2024-03-23T17:57:14.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:57:13 smithi086 ceph-mon[38221]: pgmap v201: 97 pgs: 97 active+clean; 122 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 305 op/s 2024-03-23T17:57:14.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:57:13 smithi092 ceph-mon[79089]: pgmap v201: 97 pgs: 97 active+clean; 122 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 305 op/s 2024-03-23T17:57:14.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:57:13 smithi152 ceph-mon[47949]: pgmap v201: 97 pgs: 97 active+clean; 122 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 305 op/s 2024-03-23T17:57:14.905 INFO:tasks.workunit.client.0.smithi086.stdout: 1 341953 6.39 MB/sec execute 219 sec latency 36.640 ms 2024-03-23T17:57:15.905 INFO:tasks.workunit.client.0.smithi086.stdout: 1 343792 6.41 MB/sec execute 220 sec latency 46.050 ms 2024-03-23T17:57:16.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:57:15 smithi086 ceph-mon[38221]: pgmap v202: 97 pgs: 97 active+clean; 128 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 267 op/s 2024-03-23T17:57:16.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:57:15 smithi086 ceph-mon[38221]: osdmap e85: 12 total, 12 up, 12 in 2024-03-23T17:57:16.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:57:15 smithi092 ceph-mon[79089]: pgmap v202: 97 pgs: 97 active+clean; 128 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 267 op/s 2024-03-23T17:57:16.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:57:15 smithi092 ceph-mon[79089]: osdmap e85: 12 total, 12 up, 12 in 2024-03-23T17:57:16.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:57:15 smithi152 ceph-mon[47949]: pgmap v202: 97 pgs: 97 active+clean; 128 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 267 op/s 2024-03-23T17:57:16.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:57:15 smithi152 ceph-mon[47949]: osdmap e85: 12 total, 12 up, 12 in 2024-03-23T17:57:16.905 INFO:tasks.workunit.client.0.smithi086.stdout: 1 344991 6.40 MB/sec execute 221 sec latency 12.846 ms 2024-03-23T17:57:17.905 INFO:tasks.workunit.client.0.smithi086.stdout: 1 346226 6.40 MB/sec execute 222 sec latency 16.834 ms 2024-03-23T17:57:18.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:57:17 smithi086 ceph-mon[38221]: pgmap v204: 97 pgs: 97 active+clean; 128 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 267 op/s 2024-03-23T17:57:18.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:57:17 smithi092 ceph-mon[79089]: pgmap v204: 97 pgs: 97 active+clean; 128 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 267 op/s 2024-03-23T17:57:18.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:57:17 smithi152 ceph-mon[47949]: pgmap v204: 97 pgs: 97 active+clean; 128 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 267 op/s 2024-03-23T17:57:18.905 INFO:tasks.workunit.client.0.smithi086.stdout: 1 347937 6.43 MB/sec execute 223 sec latency 14.429 ms 2024-03-23T17:57:19.905 INFO:tasks.workunit.client.0.smithi086.stdout: 1 349133 6.42 MB/sec execute 224 sec latency 36.297 ms 2024-03-23T17:57:20.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:57:19 smithi086 ceph-mon[38221]: pgmap v205: 97 pgs: 97 active+clean; 128 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 267 op/s 2024-03-23T17:57:20.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:57:19 smithi092 ceph-mon[79089]: pgmap v205: 97 pgs: 97 active+clean; 128 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 267 op/s 2024-03-23T17:57:20.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:57:19 smithi152 ceph-mon[47949]: pgmap v205: 97 pgs: 97 active+clean; 128 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 267 op/s 2024-03-23T17:57:20.905 INFO:tasks.workunit.client.0.smithi086.stdout: 1 351233 6.45 MB/sec execute 225 sec latency 20.007 ms 2024-03-23T17:57:21.759 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.758+0000 7fdec3200700 1 -- 172.21.15.86:0/2750055610 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdebc100ef0 msgr2=0x7fdebc1012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:57:21.759 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.758+0000 7fdec3200700 1 --2- 172.21.15.86:0/2750055610 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdebc100ef0 0x7fdebc1012d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fdeac009990 tx=0x7fdeac009ca0 comp rx=0 tx=0).stop 2024-03-23T17:57:21.759 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.758+0000 7fdec3200700 1 -- 172.21.15.86:0/2750055610 shutdown_connections 2024-03-23T17:57:21.759 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.758+0000 7fdec3200700 1 --2- 172.21.15.86:0/2750055610 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdebc106010 0x7fdebc10db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:21.759 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.758+0000 7fdec3200700 1 --2- 172.21.15.86:0/2750055610 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdebc100ef0 0x7fdebc1012d0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:21.759 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.758+0000 7fdec3200700 1 --2- 172.21.15.86:0/2750055610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdebc1018a0 0x7fdebc1058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:21.759 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.758+0000 7fdec3200700 1 -- 172.21.15.86:0/2750055610 >> 172.21.15.86:0/2750055610 conn(0x7fdebc0fc750 msgr2=0x7fdebc0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:57:21.759 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.758+0000 7fdec3200700 1 -- 172.21.15.86:0/2750055610 shutdown_connections 2024-03-23T17:57:21.759 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.758+0000 7fdec3200700 1 -- 172.21.15.86:0/2750055610 wait complete. 2024-03-23T17:57:21.760 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.760+0000 7fdec3200700 1 Processor -- start 2024-03-23T17:57:21.760 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.760+0000 7fdec3200700 1 -- start start 2024-03-23T17:57:21.763 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.760+0000 7fdec3200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdebc100ef0 0x7fdebc0730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:57:21.763 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.760+0000 7fdec3200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdebc106010 0x7fdebc0735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:57:21.763 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.761+0000 7fdec3200700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdebc073b30 0x7fdebc1a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:57:21.763 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.761+0000 7fdec3200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdebc1a5930 con 0x7fdebc106010 2024-03-23T17:57:21.763 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.761+0000 7fdec3200700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdebc1a5a70 con 0x7fdebc100ef0 2024-03-23T17:57:21.763 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.761+0000 7fdec3200700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdebc1a5c30 con 0x7fdebc073b30 2024-03-23T17:57:21.763 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.761+0000 7fdebbe00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdebc100ef0 0x7fdebc0730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:57:21.763 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.761+0000 7fdebbe00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdebc100ef0 0x7fdebc0730b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:52404/0 (socket says 172.21.15.86:52404) 2024-03-23T17:57:21.763 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.761+0000 7fdebbe00700 1 -- 172.21.15.86:0/3459159208 learned_addr learned my addr 172.21.15.86:0/3459159208 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:57:21.763 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.761+0000 7fdebb400700 1 --2- 172.21.15.86:0/3459159208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdebc106010 0x7fdebc0735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:57:21.763 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.761+0000 7fdec1200700 1 --2- 172.21.15.86:0/3459159208 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdebc073b30 0x7fdebc1a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:57:21.764 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.761+0000 7fdebbe00700 1 -- 172.21.15.86:0/3459159208 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdebc073b30 msgr2=0x7fdebc1a5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:57:21.764 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.761+0000 7fdebbe00700 1 --2- 172.21.15.86:0/3459159208 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdebc073b30 0x7fdebc1a5360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:21.764 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.761+0000 7fdebbe00700 1 -- 172.21.15.86:0/3459159208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdebc106010 msgr2=0x7fdebc0735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:57:21.764 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.761+0000 7fdebbe00700 1 --2- 172.21.15.86:0/3459159208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdebc106010 0x7fdebc0735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:21.764 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.761+0000 7fdebbe00700 1 -- 172.21.15.86:0/3459159208 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeac0095f0 con 0x7fdebc100ef0 2024-03-23T17:57:21.764 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.761+0000 7fdebbe00700 1 --2- 172.21.15.86:0/3459159208 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdebc100ef0 0x7fdebc0730b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fdeac005740 tx=0x7fdeac005770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:57:21.764 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.762+0000 7fdeb8c00700 1 -- 172.21.15.86:0/3459159208 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeac01d070 con 0x7fdebc100ef0 2024-03-23T17:57:21.764 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.762+0000 7fdeb8c00700 1 -- 172.21.15.86:0/3459159208 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdeac022470 con 0x7fdebc100ef0 2024-03-23T17:57:21.764 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.762+0000 7fdeb8c00700 1 -- 172.21.15.86:0/3459159208 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeac017650 con 0x7fdebc100ef0 2024-03-23T17:57:21.764 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.762+0000 7fdec3200700 1 -- 172.21.15.86:0/3459159208 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdebc1a5eb0 con 0x7fdebc100ef0 2024-03-23T17:57:21.764 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.762+0000 7fdec3200700 1 -- 172.21.15.86:0/3459159208 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdebc1a6280 con 0x7fdebc100ef0 2024-03-23T17:57:21.765 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.765+0000 7fdeb8c00700 1 -- 172.21.15.86:0/3459159208 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fdeac0225e0 con 0x7fdebc100ef0 2024-03-23T17:57:21.766 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.766+0000 7fdeb8c00700 1 --2- 172.21.15.86:0/3459159208 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fdea00a1310 0x7fdea00a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:57:21.766 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.766+0000 7fded6d13b80 1 Processor -- start 2024-03-23T17:57:21.766 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.766+0000 7fdeb8c00700 1 -- 172.21.15.86:0/3459159208 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fdeac0cde60 con 0x7fdebc100ef0 2024-03-23T17:57:21.767 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.766+0000 7fded6d13b80 1 -- start start 2024-03-23T17:57:21.767 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.766+0000 7fdebb400700 1 --2- 172.21.15.86:0/3459159208 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fdea00a1310 0x7fdea00a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:57:21.769 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.767+0000 7fded6d13b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601ccb66100 0x5601ccb664e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:57:21.769 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.767+0000 7fded6d13b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601ccb66a20 0x5601ccb71390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:57:21.769 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.767+0000 7fded6d13b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5601ccb718d0 0x5601ccb73cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:57:21.769 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.767+0000 7fded6d13b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601ccabf930 con 0x5601ccb66a20 2024-03-23T17:57:21.769 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.767+0000 7fded6d13b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601ccabd2f0 con 0x5601ccb66100 2024-03-23T17:57:21.769 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.767+0000 7fded6d13b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601ccab8be0 con 0x5601ccb718d0 2024-03-23T17:57:21.770 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.767+0000 7fdebbe00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601ccb66100 0x5601ccb664e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:57:21.770 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.767+0000 7fdebbe00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601ccb66100 0x5601ccb664e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:52420/0 (socket says 172.21.15.86:52420) 2024-03-23T17:57:21.770 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.767+0000 7fdebbe00700 1 -- 172.21.15.86:0/2790917376 learned_addr learned my addr 172.21.15.86:0/2790917376 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:57:21.770 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.767+0000 7fdec1200700 1 --2- 172.21.15.86:0/2790917376 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5601ccb718d0 0x5601ccb73cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:57:21.770 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.767+0000 7fdebb400700 1 --2- 172.21.15.86:0/2790917376 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601ccb66a20 0x5601ccb71390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:57:21.770 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.767+0000 7fdebb400700 1 --2- 172.21.15.86:0/3459159208 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fdea00a1310 0x7fdea00a37d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdebc1075b0 tx=0x7fdea8009260 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:57:21.770 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.767+0000 7fdebbe00700 1 -- 172.21.15.86:0/2790917376 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5601ccb718d0 msgr2=0x5601ccb73cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:57:21.770 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.767+0000 7fdebbe00700 1 --2- 172.21.15.86:0/2790917376 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5601ccb718d0 0x5601ccb73cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:21.770 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.767+0000 7fdebbe00700 1 -- 172.21.15.86:0/2790917376 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601ccb66a20 msgr2=0x5601ccb71390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:57:21.770 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.767+0000 7fdebbe00700 1 --2- 172.21.15.86:0/2790917376 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601ccb66a20 0x5601ccb71390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:21.770 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.768+0000 7fdebbe00700 1 -- 172.21.15.86:0/2790917376 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5601cc93a350 con 0x5601ccb66100 2024-03-23T17:57:21.770 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.768+0000 7fdebb400700 1 --2- 172.21.15.86:0/2790917376 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601ccb66a20 0x5601ccb71390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:57:21.771 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.768+0000 7fdebbe00700 1 --2- 172.21.15.86:0/2790917376 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601ccb66100 0x5601ccb664e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fdeac0974c0 tx=0x7fdeac09ad90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17743b8cc212b778 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:57:21.771 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.768+0000 7fdeb1600700 1 -- 172.21.15.86:0/2790917376 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeac09b6c0 con 0x5601ccb66100 2024-03-23T17:57:21.771 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.768+0000 7fdeb1600700 1 -- 172.21.15.86:0/2790917376 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdeac09afc0 con 0x5601ccb66100 2024-03-23T17:57:21.771 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.768+0000 7fdeb1600700 1 -- 172.21.15.86:0/2790917376 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeac0228c0 con 0x5601ccb66100 2024-03-23T17:57:21.771 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.769+0000 7fded6d13b80 1 -- 172.21.15.86:0/2790917376 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601ccb66100 msgr2=0x5601ccb664e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:57:21.771 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.769+0000 7fded6d13b80 1 --2- 172.21.15.86:0/2790917376 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601ccb66100 0x5601ccb664e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fdeac0974c0 tx=0x7fdeac09ad90 comp rx=0 tx=0).stop 2024-03-23T17:57:21.771 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.769+0000 7fded6d13b80 1 -- 172.21.15.86:0/2790917376 shutdown_connections 2024-03-23T17:57:21.771 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.769+0000 7fded6d13b80 1 --2- 172.21.15.86:0/2790917376 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5601ccb718d0 0x5601ccb73cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:21.771 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.769+0000 7fded6d13b80 1 --2- 172.21.15.86:0/2790917376 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601ccb66100 0x5601ccb664e0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:21.771 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.769+0000 7fded6d13b80 1 --2- 172.21.15.86:0/2790917376 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601ccb66a20 0x5601ccb71390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:21.771 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.769+0000 7fded6d13b80 1 -- 172.21.15.86:0/2790917376 >> 172.21.15.86:0/2790917376 conn(0x5601ccac6640 msgr2=0x5601ccb62520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:57:21.771 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.769+0000 7fded6d13b80 1 -- 172.21.15.86:0/2790917376 shutdown_connections 2024-03-23T17:57:21.771 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.769+0000 7fded6d13b80 1 -- 172.21.15.86:0/2790917376 wait complete. 2024-03-23T17:57:21.775 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.769+0000 7fded6d13b80 1 -- start start 2024-03-23T17:57:21.776 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.770+0000 7fded6d13b80 1 Processor -- start 2024-03-23T17:57:21.776 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.770+0000 7fded6d13b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601ccb66100 0x5601ccb71300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:57:21.776 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.770+0000 7fded6d13b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5601ccb66a20 0x5601ccb6e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:57:21.776 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.770+0000 7fded6d13b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601ccb718d0 0x5601ccb6ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:57:21.776 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.770+0000 7fded6d13b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601cca54d20 con 0x5601ccb718d0 2024-03-23T17:57:21.776 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.770+0000 7fded6d13b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601cca549c0 con 0x5601ccb66100 2024-03-23T17:57:21.776 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.770+0000 7fded6d13b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601cc93c640 con 0x5601ccb66a20 2024-03-23T17:57:21.776 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.770+0000 7fded6d13b80 10 client.0 _ll_register_callbacks cb 0x5601cca5e410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:57:21.776 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.771+0000 7fdebbe00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601ccb66100 0x5601ccb71300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:57:21.776 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.771+0000 7fdebbe00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601ccb66100 0x5601ccb71300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:52426/0 (socket says 172.21.15.86:52426) 2024-03-23T17:57:21.776 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.771+0000 7fdebbe00700 1 -- 172.21.15.86:0/1242458370 learned_addr learned my addr 172.21.15.86:0/1242458370 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:57:21.777 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.771+0000 7fdec1200700 1 --2- 172.21.15.86:0/1242458370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601ccb718d0 0x5601ccb6ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:57:21.777 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.771+0000 7fdebb400700 1 --2- 172.21.15.86:0/1242458370 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5601ccb66a20 0x5601ccb6e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:57:21.777 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.771+0000 7fdebbe00700 1 -- 172.21.15.86:0/1242458370 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5601ccb66a20 msgr2=0x5601ccb6e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:57:21.777 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.771+0000 7fdebbe00700 1 --2- 172.21.15.86:0/1242458370 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5601ccb66a20 0x5601ccb6e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:21.777 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.771+0000 7fdebbe00700 1 -- 172.21.15.86:0/1242458370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601ccb718d0 msgr2=0x5601ccb6ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:57:21.777 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.771+0000 7fdebbe00700 1 --2- 172.21.15.86:0/1242458370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601ccb718d0 0x5601ccb6ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:21.777 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.771+0000 7fdebbe00700 1 -- 172.21.15.86:0/1242458370 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdebc1a6280 con 0x5601ccb66100 2024-03-23T17:57:21.777 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.771+0000 7fdebbe00700 1 --2- 172.21.15.86:0/1242458370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601ccb66100 0x5601ccb71300 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fdeac000c00 tx=0x7fdeac09b5e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31f8b41b360e975c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:57:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.771+0000 7fdea6000700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T17:57:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.772+0000 7fdea6000700 1 -- 172.21.15.86:0/1242458370 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeac0a33b0 con 0x5601ccb66100 2024-03-23T17:57:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.772+0000 7fdea6000700 1 -- 172.21.15.86:0/1242458370 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fdeac099100 con 0x5601ccb66100 2024-03-23T17:57:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.772+0000 7fded6d13b80 1 -- 172.21.15.86:0/1242458370 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5601cca79320 con 0x5601ccb66100 2024-03-23T17:57:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.772+0000 7fdea6000700 1 -- 172.21.15.86:0/1242458370 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdeac0b92e0 con 0x5601ccb66100 2024-03-23T17:57:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.772+0000 7fded6d13b80 10 client.26567 fetch_fsmap learned FSMap version 40 2024-03-23T17:57:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.772+0000 7fded6d13b80 1 -- 172.21.15.86:0/1242458370 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5601cc8e6b00 con 0x5601ccb66100 2024-03-23T17:57:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.772+0000 7fdea6000700 1 -- 172.21.15.86:0/1242458370 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5601cca79320 con 0x5601ccb66100 2024-03-23T17:57:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.772+0000 7fdea6000700 1 -- 172.21.15.86:0/1242458370 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 40) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fdeac0a2f20 con 0x5601ccb66100 2024-03-23T17:57:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.772+0000 7fded6d13b80 10 client.26567 fetch_fsmap finished waiting for FSMap version 40 2024-03-23T17:57:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.772+0000 7fded6d13b80 10 client.26567 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:57:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.772+0000 7fded6d13b80 20 client.26567 populate_metadata read hostname 'smithi086' 2024-03-23T17:57:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.772+0000 7fded6d13b80 1 --2- 172.21.15.86:0/1242458370 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5601ccb80ea0 0x5601ccb832c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:57:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.772+0000 7fded6d13b80 4 client.26567 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:57:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.772+0000 7fded6d13b80 1 -- 172.21.15.86:0/1242458370 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5601ccab16a0 con 0x5601ccb80ea0 2024-03-23T17:57:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.773+0000 7fdebb400700 1 --2- 172.21.15.86:0/1242458370 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5601ccb80ea0 0x5601ccb832c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:57:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.773+0000 7fdebb400700 1 --2- 172.21.15.86:0/1242458370 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5601ccb80ea0 0x5601ccb832c0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b40bd8f69b75a0a0 server_cookie=57da19f08052f1bb in_seq=0 out_seq=0 2024-03-23T17:57:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.773+0000 7fdea6000700 10 client.26567 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:57:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.774+0000 7fdea6000700 1 -- 172.21.15.86:0/1242458370 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5601ccab16a0 con 0x5601ccb80ea0 2024-03-23T17:57:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.774+0000 7fdea6000700 10 client.26567 handle_command_reply: tid=0 2024-03-23T17:57:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.774+0000 7fded6d13b80 1 client.26567 shutdown 2024-03-23T17:57:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.774+0000 7fded6d13b80 1 -- 172.21.15.86:0/1242458370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601ccb66100 msgr2=0x5601ccb71300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:57:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.774+0000 7fded6d13b80 1 --2- 172.21.15.86:0/1242458370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601ccb66100 0x5601ccb71300 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fdeac000c00 tx=0x7fdeac09b5e0 comp rx=0 tx=0).stop 2024-03-23T17:57:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.774+0000 7fded6d13b80 1 -- 172.21.15.86:0/1242458370 shutdown_connections 2024-03-23T17:57:21.780 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.774+0000 7fded6d13b80 1 --2- 172.21.15.86:0/1242458370 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5601ccb80ea0 0x5601ccb832c0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:21.780 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.774+0000 7fded6d13b80 1 --2- 172.21.15.86:0/1242458370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601ccb718d0 0x5601ccb6ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:21.780 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.774+0000 7fded6d13b80 1 --2- 172.21.15.86:0/1242458370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601ccb66100 0x5601ccb71300 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:21.780 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.774+0000 7fded6d13b80 1 --2- 172.21.15.86:0/1242458370 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5601ccb66a20 0x5601ccb6e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:21.780 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.774+0000 7fdea6000700 0 client.26567 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:57:21.780 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.774+0000 7fded6d13b80 1 -- 172.21.15.86:0/1242458370 >> 172.21.15.86:0/1242458370 conn(0x5601ccac6640 msgr2=0x5601ccb68080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:57:21.780 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.774+0000 7fded6d13b80 1 -- 172.21.15.86:0/1242458370 shutdown_connections 2024-03-23T17:57:21.780 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.774+0000 7fded6d13b80 1 -- 172.21.15.86:0/1242458370 wait complete. 2024-03-23T17:57:21.780 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.774+0000 7fded6d13b80 20 client.26567 trim_cache size 0 max 16384 2024-03-23T17:57:21.790 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.790+0000 7fded6d13b80 1 Processor -- start 2024-03-23T17:57:21.790 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.790+0000 7fded6d13b80 1 -- start start 2024-03-23T17:57:21.792 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.791+0000 7fded6d13b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601ccb66100 0x5601ccb73cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:57:21.792 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.791+0000 7fded6d13b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5601ccb66a20 0x5601ccb6ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:57:21.792 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.791+0000 7fded6d13b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601ccb718d0 0x5601ccb87be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:57:21.793 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.791+0000 7fded6d13b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601ccb5ec80 con 0x5601ccb718d0 2024-03-23T17:57:21.793 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.791+0000 7fded6d13b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601ccb83680 con 0x5601ccb66100 2024-03-23T17:57:21.793 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.791+0000 7fded6d13b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601ccb5e400 con 0x5601ccb66a20 2024-03-23T17:57:21.793 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.791+0000 7fdebbe00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601ccb66100 0x5601ccb73cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:57:21.793 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.791+0000 7fdebbe00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601ccb66100 0x5601ccb73cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:52432/0 (socket says 172.21.15.86:52432) 2024-03-23T17:57:21.793 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.791+0000 7fdebb400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5601ccb66a20 0x5601ccb6ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:57:21.793 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.791+0000 7fdec1200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601ccb718d0 0x5601ccb87be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:57:21.793 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.791+0000 7fdebbe00700 1 -- 172.21.15.86:0/2551046008 learned_addr learned my addr 172.21.15.86:0/2551046008 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:57:21.793 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.792+0000 7fdebb400700 1 -- 172.21.15.86:0/2551046008 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601ccb66100 msgr2=0x5601ccb73cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:57:21.793 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.792+0000 7fdebb400700 1 --2- 172.21.15.86:0/2551046008 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601ccb66100 0x5601ccb73cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:21.793 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.792+0000 7fdebb400700 1 -- 172.21.15.86:0/2551046008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601ccb718d0 msgr2=0x5601ccb87be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:57:21.793 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.792+0000 7fdebb400700 1 --2- 172.21.15.86:0/2551046008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601ccb718d0 0x5601ccb87be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:21.794 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.792+0000 7fdebb400700 1 -- 172.21.15.86:0/2551046008 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5601cca7ee40 con 0x5601ccb66a20 2024-03-23T17:57:21.794 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.792+0000 7fdec1200700 1 --2- 172.21.15.86:0/2551046008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601ccb718d0 0x5601ccb87be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:57:21.794 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.792+0000 7fdebbe00700 1 --2- 172.21.15.86:0/2551046008 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601ccb66100 0x5601ccb73cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:57:21.794 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.792+0000 7fdebb400700 1 --2- 172.21.15.86:0/2551046008 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5601ccb66a20 0x5601ccb6ed80 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x5601ccb83b60 tx=0x7fdea8014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63521073b4258a02 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:57:21.794 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.792+0000 7fdeb2000700 1 -- 172.21.15.86:0/2551046008 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea8023040 con 0x5601ccb66a20 2024-03-23T17:57:21.794 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.792+0000 7fdeb2000700 1 -- 172.21.15.86:0/2551046008 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdea801f110 con 0x5601ccb66a20 2024-03-23T17:57:21.794 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.792+0000 7fdeb2000700 1 -- 172.21.15.86:0/2551046008 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea8004a90 con 0x5601ccb66a20 2024-03-23T17:57:21.797 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.793+0000 7fded6d13b80 1 -- 172.21.15.86:0/2551046008 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5601ccb66a20 msgr2=0x5601ccb6ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:57:21.798 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.793+0000 7fded6d13b80 1 --2- 172.21.15.86:0/2551046008 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5601ccb66a20 0x5601ccb6ed80 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x5601ccb83b60 tx=0x7fdea8014000 comp rx=0 tx=0).stop 2024-03-23T17:57:21.798 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.793+0000 7fded6d13b80 1 -- 172.21.15.86:0/2551046008 shutdown_connections 2024-03-23T17:57:21.798 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.793+0000 7fded6d13b80 1 --2- 172.21.15.86:0/2551046008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601ccb718d0 0x5601ccb87be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:21.798 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.793+0000 7fded6d13b80 1 --2- 172.21.15.86:0/2551046008 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601ccb66100 0x5601ccb73cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:21.798 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.793+0000 7fded6d13b80 1 --2- 172.21.15.86:0/2551046008 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5601ccb66a20 0x5601ccb6ed80 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:21.798 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.793+0000 7fded6d13b80 1 -- 172.21.15.86:0/2551046008 >> 172.21.15.86:0/2551046008 conn(0x5601ccac6640 msgr2=0x5601ccb6cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:57:21.798 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.793+0000 7fded6d13b80 1 -- 172.21.15.86:0/2551046008 shutdown_connections 2024-03-23T17:57:21.798 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.793+0000 7fded6d13b80 1 -- 172.21.15.86:0/2551046008 wait complete. 2024-03-23T17:57:21.798 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.794+0000 7fded6d13b80 1 -- start start 2024-03-23T17:57:21.798 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.794+0000 7fded6d13b80 1 Processor -- start 2024-03-23T17:57:21.798 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.794+0000 7fded6d13b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601ccb66100 0x5601ccb6cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:57:21.798 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.794+0000 7fded6d13b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5601ccb66a20 0x5601ccb6d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:57:21.799 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.794+0000 7fded6d13b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601ccb718d0 0x5601ccb6d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:57:21.799 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.794+0000 7fded6d13b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601ccbd1ab0 con 0x5601ccb66100 2024-03-23T17:57:21.799 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.794+0000 7fded6d13b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601ccbd1c20 con 0x5601ccb718d0 2024-03-23T17:57:21.799 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.794+0000 7fded6d13b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601ccbd1de0 con 0x5601ccb66a20 2024-03-23T17:57:21.799 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.794+0000 7fded6d13b80 10 client.0 _ll_register_callbacks cb 0x5601cca5e410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:57:21.799 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.795+0000 7fdebbe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601ccb66100 0x5601ccb6cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:57:21.799 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.795+0000 7fdebb400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5601ccb66a20 0x5601ccb6d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:57:21.799 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.795+0000 7fdebb400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5601ccb66a20 0x5601ccb6d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:45486/0 (socket says 172.21.15.86:45486) 2024-03-23T17:57:21.799 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.795+0000 7fdebb400700 1 -- 172.21.15.86:0/3215931431 learned_addr learned my addr 172.21.15.86:0/3215931431 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:57:21.799 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.795+0000 7fdec1200700 1 --2- 172.21.15.86:0/3215931431 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601ccb718d0 0x5601ccb6d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:57:21.799 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.795+0000 7fdebb400700 1 -- 172.21.15.86:0/3215931431 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601ccb718d0 msgr2=0x5601ccb6d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:57:21.799 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.795+0000 7fdebb400700 1 --2- 172.21.15.86:0/3215931431 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601ccb718d0 0x5601ccb6d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:21.799 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.795+0000 7fdebb400700 1 -- 172.21.15.86:0/3215931431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601ccb66100 msgr2=0x5601ccb6cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:57:21.800 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.795+0000 7fdebb400700 1 --2- 172.21.15.86:0/3215931431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601ccb66100 0x5601ccb6cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:21.800 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.795+0000 7fdebb400700 1 -- 172.21.15.86:0/3215931431 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5601cc8e6b00 con 0x5601ccb66a20 2024-03-23T17:57:21.800 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.795+0000 7fdebbe00700 1 --2- 172.21.15.86:0/3215931431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601ccb66100 0x5601ccb6cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:57:21.800 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.795+0000 7fdebb400700 1 --2- 172.21.15.86:0/3215931431 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5601ccb66a20 0x5601ccb6d080 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fdea8005e10 tx=0x7fdea8004430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5aa49d62f0208b31 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:57:21.800 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.795+0000 7fdea6000700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-03-23T17:57:21.800 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.796+0000 7fdea6000700 1 -- 172.21.15.86:0/3215931431 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea8023070 con 0x5601ccb66a20 2024-03-23T17:57:21.800 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.796+0000 7fded6d13b80 1 -- 172.21.15.86:0/3215931431 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdeac09afc0 con 0x5601ccb66a20 2024-03-23T17:57:21.800 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.796+0000 7fdea6000700 1 -- 172.21.15.86:0/3215931431 <== mon.1 v2:172.21.15.152:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fdea801faf0 con 0x5601ccb66a20 2024-03-23T17:57:21.800 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.796+0000 7fdea6000700 1 -- 172.21.15.86:0/3215931431 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdea801fcb0 con 0x5601ccb66a20 2024-03-23T17:57:21.800 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.796+0000 7fded6d13b80 10 client.26563 fetch_fsmap learned FSMap version 40 2024-03-23T17:57:21.800 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.796+0000 7fded6d13b80 1 -- 172.21.15.86:0/3215931431 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x7fdea8017070 con 0x5601ccb66a20 2024-03-23T17:57:21.800 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.796+0000 7fdea6000700 1 -- 172.21.15.86:0/3215931431 <== mon.1 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdea803e020 con 0x5601ccb66a20 2024-03-23T17:57:21.800 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.797+0000 7fdea6000700 1 -- 172.21.15.86:0/3215931431 <== mon.1 v2:172.21.15.152:3300/0 5 ==== fsmap(e 40) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fdea80048f0 con 0x5601ccb66a20 2024-03-23T17:57:21.800 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.797+0000 7fded6d13b80 10 client.26563 fetch_fsmap finished waiting for FSMap version 40 2024-03-23T17:57:21.801 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.797+0000 7fded6d13b80 10 client.26563 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:57:21.801 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.797+0000 7fded6d13b80 20 client.26563 populate_metadata read hostname 'smithi086' 2024-03-23T17:57:21.801 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.797+0000 7fded6d13b80 1 --2- 172.21.15.86:0/3215931431 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5601ccbd3680 0x5601ccbd5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:57:21.801 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.797+0000 7fded6d13b80 4 client.26563 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T17:57:21.801 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.797+0000 7fded6d13b80 1 -- 172.21.15.86:0/3215931431 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5601cc907c90 con 0x5601ccbd3680 2024-03-23T17:57:21.801 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.797+0000 7fdebbe00700 1 --2- 172.21.15.86:0/3215931431 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5601ccbd3680 0x5601ccbd5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:57:21.801 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.797+0000 7fdea6000700 10 client.26563 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:57:21.801 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.797+0000 7fdebbe00700 1 --2- 172.21.15.86:0/3215931431 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5601ccbd3680 0x5601ccbd5aa0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c05ec26198df6a7f server_cookie=a1077a9562abc in_seq=0 out_seq=0 2024-03-23T17:57:21.837 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.836+0000 7fdea6000700 1 -- 172.21.15.86:0/3215931431 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5601cc907c90 con 0x5601ccbd3680 2024-03-23T17:57:21.837 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.836+0000 7fdea6000700 10 client.26563 handle_command_reply: tid=0 2024-03-23T17:57:21.837 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.836+0000 7fded6d13b80 1 client.26563 shutdown 2024-03-23T17:57:21.837 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.836+0000 7fded6d13b80 1 -- 172.21.15.86:0/3215931431 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5601ccb66a20 msgr2=0x5601ccb6d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:57:21.837 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.836+0000 7fded6d13b80 1 --2- 172.21.15.86:0/3215931431 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5601ccb66a20 0x5601ccb6d080 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fdea8005e10 tx=0x7fdea8004430 comp rx=0 tx=0).stop 2024-03-23T17:57:21.837 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.836+0000 7fded6d13b80 1 -- 172.21.15.86:0/3215931431 shutdown_connections 2024-03-23T17:57:21.837 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.836+0000 7fded6d13b80 1 --2- 172.21.15.86:0/3215931431 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5601ccbd3680 0x5601ccbd5aa0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:21.837 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.836+0000 7fded6d13b80 1 --2- 172.21.15.86:0/3215931431 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601ccb718d0 0x5601ccb6d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:21.837 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.836+0000 7fded6d13b80 1 --2- 172.21.15.86:0/3215931431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601ccb66100 0x5601ccb6cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:21.838 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.836+0000 7fded6d13b80 1 --2- 172.21.15.86:0/3215931431 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5601ccb66a20 0x5601ccb6d080 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:21.838 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.836+0000 7fded6d13b80 1 -- 172.21.15.86:0/3215931431 >> 172.21.15.86:0/3215931431 conn(0x5601ccac6640 msgr2=0x5601ccb61d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:57:21.838 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.836+0000 7fdea6000700 0 client.26563 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:57:21.838 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.837+0000 7fded6d13b80 1 -- 172.21.15.86:0/3215931431 shutdown_connections 2024-03-23T17:57:21.838 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.837+0000 7fded6d13b80 1 -- 172.21.15.86:0/3215931431 wait complete. 2024-03-23T17:57:21.838 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.837+0000 7fded6d13b80 20 client.26563 trim_cache size 0 max 16384 2024-03-23T17:57:21.838 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-03-23T17:57:21.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-03-23T17:57:21.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-03-23T17:57:21.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-03-23T17:57:21.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.837+0000 7fdec3200700 1 -- 172.21.15.86:0/3459159208 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fdea00a1310 msgr2=0x7fdea00a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:57:21.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.838+0000 7fdec3200700 1 --2- 172.21.15.86:0/3459159208 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fdea00a1310 0x7fdea00a37d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdebc1075b0 tx=0x7fdea8009260 comp rx=0 tx=0).stop 2024-03-23T17:57:21.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.838+0000 7fdec3200700 1 -- 172.21.15.86:0/3459159208 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdebc100ef0 msgr2=0x7fdebc0730b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:57:21.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.838+0000 7fdec3200700 1 --2- 172.21.15.86:0/3459159208 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdebc100ef0 0x7fdebc0730b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fdeac005740 tx=0x7fdeac005770 comp rx=0 tx=0).stop 2024-03-23T17:57:21.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.839+0000 7fdec3200700 1 -- 172.21.15.86:0/3459159208 shutdown_connections 2024-03-23T17:57:21.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.839+0000 7fdec3200700 1 --2- 172.21.15.86:0/3459159208 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fdea00a1310 0x7fdea00a37d0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:21.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.839+0000 7fdec3200700 1 --2- 172.21.15.86:0/3459159208 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdebc073b30 0x7fdebc1a5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:21.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.839+0000 7fdec3200700 1 --2- 172.21.15.86:0/3459159208 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdebc100ef0 0x7fdebc0730b0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:21.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.839+0000 7fdec3200700 1 --2- 172.21.15.86:0/3459159208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdebc106010 0x7fdebc0735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:21.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.839+0000 7fdec3200700 1 -- 172.21.15.86:0/3459159208 >> 172.21.15.86:0/3459159208 conn(0x7fdebc0fc750 msgr2=0x7fdebc0fdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:57:21.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.839+0000 7fdec3200700 1 -- 172.21.15.86:0/3459159208 shutdown_connections 2024-03-23T17:57:21.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:21.839+0000 7fdec3200700 1 -- 172.21.15.86:0/3459159208 wait complete. 2024-03-23T17:57:21.905 INFO:tasks.workunit.client.0.smithi086.stdout: 1 352251 6.44 MB/sec execute 226 sec latency 35.536 ms 2024-03-23T17:57:22.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:57:21 smithi086 ceph-mon[38221]: pgmap v206: 97 pgs: 97 active+clean; 134 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.8 MiB/s wr, 252 op/s 2024-03-23T17:57:22.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:57:21 smithi092 ceph-mon[79089]: pgmap v206: 97 pgs: 97 active+clean; 134 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.8 MiB/s wr, 252 op/s 2024-03-23T17:57:22.340 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:57:21 smithi152 ceph-mon[47949]: pgmap v206: 97 pgs: 97 active+clean; 134 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.8 MiB/s wr, 252 op/s 2024-03-23T17:57:22.905 INFO:tasks.workunit.client.0.smithi086.stdout: 1 353457 6.43 MB/sec execute 227 sec latency 85.113 ms 2024-03-23T17:57:23.905 INFO:tasks.workunit.client.0.smithi086.stdout: 1 355064 6.46 MB/sec execute 228 sec latency 10.803 ms 2024-03-23T17:57:24.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:57:23 smithi086 ceph-mon[38221]: pgmap v207: 97 pgs: 97 active+clean; 134 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.8 MiB/s wr, 252 op/s 2024-03-23T17:57:24.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:57:23 smithi092 ceph-mon[79089]: pgmap v207: 97 pgs: 97 active+clean; 134 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.8 MiB/s wr, 252 op/s 2024-03-23T17:57:24.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:57:23 smithi152 ceph-mon[47949]: pgmap v207: 97 pgs: 97 active+clean; 134 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.8 MiB/s wr, 252 op/s 2024-03-23T17:57:24.905 INFO:tasks.workunit.client.0.smithi086.stdout: 1 356206 6.45 MB/sec execute 229 sec latency 14.903 ms 2024-03-23T17:57:25.905 INFO:tasks.workunit.client.0.smithi086.stdout: 1 358090 6.46 MB/sec execute 230 sec latency 48.947 ms 2024-03-23T17:57:26.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:57:25 smithi086 ceph-mon[38221]: pgmap v208: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.0 MiB/s wr, 272 op/s 2024-03-23T17:57:26.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:57:25 smithi092 ceph-mon[79089]: pgmap v208: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.0 MiB/s wr, 272 op/s 2024-03-23T17:57:26.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:57:25 smithi152 ceph-mon[47949]: pgmap v208: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.0 MiB/s wr, 272 op/s 2024-03-23T17:57:26.906 INFO:tasks.workunit.client.0.smithi086.stdout: 1 359162 6.46 MB/sec execute 231 sec latency 20.544 ms 2024-03-23T17:57:27.905 INFO:tasks.workunit.client.0.smithi086.stdout: 1 360407 6.45 MB/sec execute 232 sec latency 16.906 ms 2024-03-23T17:57:28.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:57:27 smithi086 ceph-mon[38221]: pgmap v209: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 254 op/s 2024-03-23T17:57:28.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:57:27 smithi092 ceph-mon[79089]: pgmap v209: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 254 op/s 2024-03-23T17:57:28.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:57:27 smithi152 ceph-mon[47949]: pgmap v209: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 254 op/s 2024-03-23T17:57:28.905 INFO:tasks.workunit.client.0.smithi086.stdout: 1 362091 6.48 MB/sec execute 233 sec latency 12.194 ms 2024-03-23T17:57:29.907 INFO:tasks.workunit.client.0.smithi086.stdout: 1 363285 6.48 MB/sec execute 234 sec latency 15.505 ms 2024-03-23T17:57:30.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:57:29 smithi086 ceph-mon[38221]: pgmap v210: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 226 op/s 2024-03-23T17:57:30.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:57:29 smithi092 ceph-mon[79089]: pgmap v210: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 226 op/s 2024-03-23T17:57:30.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:57:29 smithi152 ceph-mon[47949]: pgmap v210: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 226 op/s 2024-03-23T17:57:30.906 INFO:tasks.workunit.client.0.smithi086.stdout: 1 364998 6.48 MB/sec execute 235 sec latency 28.301 ms 2024-03-23T17:57:31.906 INFO:tasks.workunit.client.0.smithi086.stdout: 1 366213 6.48 MB/sec execute 236 sec latency 12.554 ms 2024-03-23T17:57:32.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:57:31 smithi086 ceph-mon[38221]: pgmap v211: 97 pgs: 97 active+clean; 137 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.7 MiB/s wr, 328 op/s 2024-03-23T17:57:32.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:57:31 smithi092 ceph-mon[79089]: pgmap v211: 97 pgs: 97 active+clean; 137 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.7 MiB/s wr, 328 op/s 2024-03-23T17:57:32.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:57:31 smithi152 ceph-mon[47949]: pgmap v211: 97 pgs: 97 active+clean; 137 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.7 MiB/s wr, 328 op/s 2024-03-23T17:57:32.906 INFO:tasks.workunit.client.0.smithi086.stdout: 1 367377 6.48 MB/sec execute 237 sec latency 17.083 ms 2024-03-23T17:57:33.906 INFO:tasks.workunit.client.0.smithi086.stdout: 1 369170 6.50 MB/sec execute 238 sec latency 13.637 ms 2024-03-23T17:57:34.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:57:33 smithi086 ceph-mon[38221]: pgmap v212: 97 pgs: 97 active+clean; 137 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 221 op/s 2024-03-23T17:57:34.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:57:33 smithi092 ceph-mon[79089]: pgmap v212: 97 pgs: 97 active+clean; 137 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 221 op/s 2024-03-23T17:57:34.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:57:33 smithi152 ceph-mon[47949]: pgmap v212: 97 pgs: 97 active+clean; 137 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 221 op/s 2024-03-23T17:57:34.906 INFO:tasks.workunit.client.0.smithi086.stdout: 1 370353 6.50 MB/sec execute 239 sec latency 17.515 ms 2024-03-23T17:57:35.001 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:34.999+0000 7f9896400700 1 -- 172.21.15.86:0/1751507544 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98901014a0 msgr2=0x7f989010d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:57:35.001 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:34.999+0000 7f9896400700 1 --2- 172.21.15.86:0/1751507544 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98901014a0 0x7f989010d9b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f988800b0a0 tx=0x7f988800b3b0 comp rx=0 tx=0).stop 2024-03-23T17:57:35.001 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:34.999+0000 7f9896400700 1 -- 172.21.15.86:0/1751507544 shutdown_connections 2024-03-23T17:57:35.001 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:34.999+0000 7f9896400700 1 --2- 172.21.15.86:0/1751507544 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98901014a0 0x7f989010d9b0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:35.001 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:34.999+0000 7f9896400700 1 --2- 172.21.15.86:0/1751507544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9890106b80 0x7f9890106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:35.001 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:34.999+0000 7f9896400700 1 --2- 172.21.15.86:0/1751507544 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9890100ae0 0x7f9890100f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:35.001 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:34.999+0000 7f9896400700 1 -- 172.21.15.86:0/1751507544 >> 172.21.15.86:0/1751507544 conn(0x7f98900fc610 msgr2=0x7f98900fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:57:35.002 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:34.999+0000 7f9896400700 1 -- 172.21.15.86:0/1751507544 shutdown_connections 2024-03-23T17:57:35.002 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:34.999+0000 7f9896400700 1 -- 172.21.15.86:0/1751507544 wait complete. 2024-03-23T17:57:35.004 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.000+0000 7f9896400700 1 Processor -- start 2024-03-23T17:57:35.004 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.001+0000 7f9896400700 1 -- start start 2024-03-23T17:57:35.004 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.001+0000 7f9896400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9890100ae0 0x7f9890199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:57:35.004 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.001+0000 7f9896400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f98901014a0 0x7f9890199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:57:35.004 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.001+0000 7f9896400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9890106b80 0x7f989019d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:57:35.004 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.001+0000 7f9896400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f989019e020 con 0x7f9890106b80 2024-03-23T17:57:35.004 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.001+0000 7f9896400700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f989019e190 con 0x7f9890100ae0 2024-03-23T17:57:35.004 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.001+0000 7f9896400700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f989019e350 con 0x7f98901014a0 2024-03-23T17:57:35.004 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.001+0000 7f9895000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9890100ae0 0x7f9890199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:57:35.004 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.001+0000 7f9895000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9890100ae0 0x7f9890199100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:36934/0 (socket says 172.21.15.86:36934) 2024-03-23T17:57:35.004 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.001+0000 7f9895000700 1 -- 172.21.15.86:0/715379916 learned_addr learned my addr 172.21.15.86:0/715379916 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:57:35.004 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.001+0000 7f988fe00700 1 --2- 172.21.15.86:0/715379916 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f98901014a0 0x7f9890199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:57:35.004 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.001+0000 7f9895a00700 1 --2- 172.21.15.86:0/715379916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9890106b80 0x7f989019d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:57:35.005 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.002+0000 7f9895000700 1 -- 172.21.15.86:0/715379916 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f98901014a0 msgr2=0x7f9890199640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:57:35.005 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.002+0000 7f9895000700 1 --2- 172.21.15.86:0/715379916 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f98901014a0 0x7f9890199640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:35.005 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.002+0000 7f9895000700 1 -- 172.21.15.86:0/715379916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9890106b80 msgr2=0x7f989019d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:57:35.005 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.002+0000 7f9895000700 1 --2- 172.21.15.86:0/715379916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9890106b80 0x7f989019d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:35.005 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.002+0000 7f9895000700 1 -- 172.21.15.86:0/715379916 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9888009d00 con 0x7f9890100ae0 2024-03-23T17:57:35.005 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.002+0000 7f988fe00700 1 --2- 172.21.15.86:0/715379916 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f98901014a0 0x7f9890199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:57:35.005 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.002+0000 7f9895000700 1 --2- 172.21.15.86:0/715379916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9890100ae0 0x7f9890199100 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f988400e9a0 tx=0x7f988400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:57:35.005 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.002+0000 7f988d600700 1 -- 172.21.15.86:0/715379916 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9884009790 con 0x7f9890100ae0 2024-03-23T17:57:35.005 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.002+0000 7f988d600700 1 -- 172.21.15.86:0/715379916 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9884010430 con 0x7f9890100ae0 2024-03-23T17:57:35.005 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.002+0000 7f988d600700 1 -- 172.21.15.86:0/715379916 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9884018680 con 0x7f9890100ae0 2024-03-23T17:57:35.005 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.002+0000 7f9896400700 1 -- 172.21.15.86:0/715379916 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f989019e630 con 0x7f9890100ae0 2024-03-23T17:57:35.005 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.002+0000 7f9896400700 1 -- 172.21.15.86:0/715379916 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98901a5e70 con 0x7f9890100ae0 2024-03-23T17:57:35.006 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.005+0000 7f988d600700 1 -- 172.21.15.86:0/715379916 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f98840105a0 con 0x7f9890100ae0 2024-03-23T17:57:35.008 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.007+0000 7f98ab57db80 1 Processor -- start 2024-03-23T17:57:35.008 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.007+0000 7f98ab57db80 1 -- start start 2024-03-23T17:57:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.008+0000 7f98ab57db80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4b3c1a2d0 0x55a4b3c1a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:57:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.008+0000 7f98ab57db80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a4b3c1abf0 0x55a4b3c25570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:57:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.008+0000 7f98ab57db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4b3c25ab0 0x55a4b3c27ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:57:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.008+0000 7f988d600700 1 --2- 172.21.15.86:0/715379916 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f98780a1290 0x7f98780a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:57:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.008+0000 7f98ab57db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4b3b73930 con 0x55a4b3c25ab0 2024-03-23T17:57:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.008+0000 7f98ab57db80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4b3b712f0 con 0x55a4b3c1a2d0 2024-03-23T17:57:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.008+0000 7f98ab57db80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4b3b6cbe0 con 0x55a4b3c1abf0 2024-03-23T17:57:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.008+0000 7f988d600700 1 -- 172.21.15.86:0/715379916 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f9884014070 con 0x7f9890100ae0 2024-03-23T17:57:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.008+0000 7f9895a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4b3c1a2d0 0x55a4b3c1a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:57:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.008+0000 7f9895a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4b3c1a2d0 0x55a4b3c1a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:36940/0 (socket says 172.21.15.86:36940) 2024-03-23T17:57:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.008+0000 7f9895a00700 1 -- 172.21.15.86:0/264513820 learned_addr learned my addr 172.21.15.86:0/264513820 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:57:35.011 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.008+0000 7f9895000700 1 --2- 172.21.15.86:0/264513820 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a4b3c1abf0 0x55a4b3c25570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:57:35.011 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.008+0000 7f988fe00700 1 --2- 172.21.15.86:0/264513820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4b3c25ab0 0x55a4b3c27ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:57:35.011 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.008+0000 7f9895a00700 1 --2- 172.21.15.86:0/715379916 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f98780a1290 0x7f98780a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:57:35.011 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.009+0000 7f9895a00700 1 -- 172.21.15.86:0/264513820 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a4b3c1abf0 msgr2=0x55a4b3c25570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:57:35.011 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.009+0000 7f9895a00700 1 --2- 172.21.15.86:0/264513820 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a4b3c1abf0 0x55a4b3c25570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:35.011 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.009+0000 7f9895a00700 1 -- 172.21.15.86:0/264513820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4b3c25ab0 msgr2=0x55a4b3c27ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:57:35.011 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.009+0000 7f9895a00700 1 --2- 172.21.15.86:0/264513820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4b3c25ab0 0x55a4b3c27ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:35.011 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.009+0000 7f9895a00700 1 -- 172.21.15.86:0/264513820 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a4b39ee350 con 0x55a4b3c1a2d0 2024-03-23T17:57:35.011 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.009+0000 7f988fe00700 1 --2- 172.21.15.86:0/264513820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4b3c25ab0 0x55a4b3c27ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:57:35.011 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.009+0000 7f9895000700 1 --2- 172.21.15.86:0/264513820 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a4b3c1abf0 0x55a4b3c25570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:57:35.011 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.009+0000 7f9895a00700 1 --2- 172.21.15.86:0/264513820 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4b3c1a2d0 0x55a4b3c1a6b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f988800b0a0 tx=0x7f9888008fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30b72aeccfea64c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:57:35.015 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.009+0000 7f9882000700 1 -- 172.21.15.86:0/264513820 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988800e040 con 0x55a4b3c1a2d0 2024-03-23T17:57:35.015 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.009+0000 7f9882000700 1 -- 172.21.15.86:0/264513820 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f98880079a0 con 0x55a4b3c1a2d0 2024-03-23T17:57:35.015 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.009+0000 7f9882000700 1 -- 172.21.15.86:0/264513820 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988801e9b0 con 0x55a4b3c1a2d0 2024-03-23T17:57:35.015 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.010+0000 7f98ab57db80 1 -- 172.21.15.86:0/264513820 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4b3c1a2d0 msgr2=0x55a4b3c1a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:57:35.015 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.010+0000 7f9895a00700 1 --2- 172.21.15.86:0/715379916 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f98780a1290 0x7f98780a3750 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f988802b000 tx=0x7f9888026000 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:57:35.015 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.010+0000 7f98ab57db80 1 --2- 172.21.15.86:0/264513820 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4b3c1a2d0 0x55a4b3c1a6b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f988800b0a0 tx=0x7f9888008fb0 comp rx=0 tx=0).stop 2024-03-23T17:57:35.016 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.010+0000 7f98ab57db80 1 -- 172.21.15.86:0/264513820 shutdown_connections 2024-03-23T17:57:35.016 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.010+0000 7f98ab57db80 1 --2- 172.21.15.86:0/264513820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4b3c25ab0 0x55a4b3c27ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:35.016 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.010+0000 7f98ab57db80 1 --2- 172.21.15.86:0/264513820 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4b3c1a2d0 0x55a4b3c1a6b0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:35.016 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.010+0000 7f98ab57db80 1 --2- 172.21.15.86:0/264513820 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a4b3c1abf0 0x55a4b3c25570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:35.016 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.010+0000 7f98ab57db80 1 -- 172.21.15.86:0/264513820 >> 172.21.15.86:0/264513820 conn(0x55a4b3b7a640 msgr2=0x55a4b3c16750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:57:35.016 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.010+0000 7f98ab57db80 1 -- 172.21.15.86:0/264513820 shutdown_connections 2024-03-23T17:57:35.016 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.010+0000 7f98ab57db80 1 -- 172.21.15.86:0/264513820 wait complete. 2024-03-23T17:57:35.016 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.011+0000 7f98ab57db80 1 -- start start 2024-03-23T17:57:35.016 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.011+0000 7f98ab57db80 1 Processor -- start 2024-03-23T17:57:35.016 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.011+0000 7f98ab57db80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4b3c1a2d0 0x55a4b3c25500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:57:35.016 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.011+0000 7f98ab57db80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a4b3c1abf0 0x55a4b3c22780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:57:35.016 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.012+0000 7f98ab57db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4b3c25ab0 0x55a4b3c22ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:57:35.016 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.012+0000 7f98ab57db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4b3b08d20 con 0x55a4b3c25ab0 2024-03-23T17:57:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.012+0000 7f98ab57db80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4b3b089c0 con 0x55a4b3c1a2d0 2024-03-23T17:57:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.012+0000 7f98ab57db80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4b39f0640 con 0x55a4b3c1abf0 2024-03-23T17:57:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.012+0000 7f9895000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4b3c1a2d0 0x55a4b3c25500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:57:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.012+0000 7f98ab57db80 10 client.0 _ll_register_callbacks cb 0x55a4b3b12410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:57:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.012+0000 7f9895000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4b3c1a2d0 0x55a4b3c25500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:36944/0 (socket says 172.21.15.86:36944) 2024-03-23T17:57:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.012+0000 7f9895000700 1 -- 172.21.15.86:0/3596780641 learned_addr learned my addr 172.21.15.86:0/3596780641 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:57:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.012+0000 7f988fe00700 1 --2- 172.21.15.86:0/3596780641 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a4b3c1abf0 0x55a4b3c22780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:57:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.012+0000 7f9895000700 1 -- 172.21.15.86:0/3596780641 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a4b3c1abf0 msgr2=0x55a4b3c22780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:57:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.012+0000 7f9895a00700 1 --2- 172.21.15.86:0/3596780641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4b3c25ab0 0x55a4b3c22ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:57:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.012+0000 7f9895000700 1 --2- 172.21.15.86:0/3596780641 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a4b3c1abf0 0x55a4b3c22780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.012+0000 7f9895000700 1 -- 172.21.15.86:0/3596780641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4b3c25ab0 msgr2=0x55a4b3c22ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:57:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.012+0000 7f9895000700 1 --2- 172.21.15.86:0/3596780641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4b3c25ab0 0x55a4b3c22ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.012+0000 7f9895000700 1 -- 172.21.15.86:0/3596780641 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98901a5e70 con 0x55a4b3c1a2d0 2024-03-23T17:57:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.012+0000 7f9895000700 1 --2- 172.21.15.86:0/3596780641 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4b3c1a2d0 0x55a4b3c25500 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f9884096450 tx=0x7f9884092970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aca42901e873a58 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:57:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.012+0000 7f9876a00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T17:57:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.013+0000 7f9876a00700 1 -- 172.21.15.86:0/3596780641 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988409a3b0 con 0x55a4b3c1a2d0 2024-03-23T17:57:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.013+0000 7f9876a00700 1 -- 172.21.15.86:0/3596780641 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f98840a23d0 con 0x55a4b3c1a2d0 2024-03-23T17:57:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.013+0000 7f98ab57db80 1 -- 172.21.15.86:0/3596780641 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a4b3b2d320 con 0x55a4b3c1a2d0 2024-03-23T17:57:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.013+0000 7f9876a00700 1 -- 172.21.15.86:0/3596780641 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9884099440 con 0x55a4b3c1a2d0 2024-03-23T17:57:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.013+0000 7f98ab57db80 10 client.26585 fetch_fsmap learned FSMap version 40 2024-03-23T17:57:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.013+0000 7f98ab57db80 1 -- 172.21.15.86:0/3596780641 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x7f989010e2b0 con 0x55a4b3c1a2d0 2024-03-23T17:57:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.013+0000 7f9876a00700 1 -- 172.21.15.86:0/3596780641 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a4b3b2d320 con 0x55a4b3c1a2d0 2024-03-23T17:57:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.013+0000 7f9876a00700 1 -- 172.21.15.86:0/3596780641 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 40) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f9884095d40 con 0x55a4b3c1a2d0 2024-03-23T17:57:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.013+0000 7f98ab57db80 10 client.26585 fetch_fsmap finished waiting for FSMap version 40 2024-03-23T17:57:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.013+0000 7f98ab57db80 10 client.26585 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:57:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.013+0000 7f98ab57db80 20 client.26585 populate_metadata read hostname 'smithi086' 2024-03-23T17:57:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.013+0000 7f98ab57db80 1 --2- 172.21.15.86:0/3596780641 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55a4b3c35760 0x55a4b3c37b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:57:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.013+0000 7f98ab57db80 4 client.26585 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:57:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.014+0000 7f98ab57db80 1 -- 172.21.15.86:0/3596780641 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a4b3b656a0 con 0x55a4b3c35760 2024-03-23T17:57:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.014+0000 7f988fe00700 1 --2- 172.21.15.86:0/3596780641 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55a4b3c35760 0x55a4b3c37b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:57:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.014+0000 7f988fe00700 1 --2- 172.21.15.86:0/3596780641 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55a4b3c35760 0x55a4b3c37b80 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4877cf7bc1cf7d21 server_cookie=5168308058f120bf in_seq=0 out_seq=0 2024-03-23T17:57:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.014+0000 7f9876a00700 10 client.26585 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:57:35.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.015+0000 7f9876a00700 1 -- 172.21.15.86:0/3596780641 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55a4b3b656a0 con 0x55a4b3c35760 2024-03-23T17:57:35.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.015+0000 7f9876a00700 10 client.26585 handle_command_reply: tid=0 2024-03-23T17:57:35.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.015+0000 7f98ab57db80 1 client.26585 shutdown 2024-03-23T17:57:35.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.015+0000 7f98ab57db80 1 -- 172.21.15.86:0/3596780641 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4b3c1a2d0 msgr2=0x55a4b3c25500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:57:35.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.015+0000 7f98ab57db80 1 --2- 172.21.15.86:0/3596780641 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4b3c1a2d0 0x55a4b3c25500 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f9884096450 tx=0x7f9884092970 comp rx=0 tx=0).stop 2024-03-23T17:57:35.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.015+0000 7f98ab57db80 1 -- 172.21.15.86:0/3596780641 shutdown_connections 2024-03-23T17:57:35.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.015+0000 7f98ab57db80 1 --2- 172.21.15.86:0/3596780641 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55a4b3c35760 0x55a4b3c37b80 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:35.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.015+0000 7f98ab57db80 1 --2- 172.21.15.86:0/3596780641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4b3c25ab0 0x55a4b3c22ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:35.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.015+0000 7f98ab57db80 1 --2- 172.21.15.86:0/3596780641 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4b3c1a2d0 0x55a4b3c25500 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:35.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.015+0000 7f98ab57db80 1 --2- 172.21.15.86:0/3596780641 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a4b3c1abf0 0x55a4b3c22780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:35.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.015+0000 7f9876a00700 0 client.26585 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:57:35.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.015+0000 7f98ab57db80 1 -- 172.21.15.86:0/3596780641 >> 172.21.15.86:0/3596780641 conn(0x55a4b3b7a640 msgr2=0x55a4b3c1c250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:57:35.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.015+0000 7f98ab57db80 1 -- 172.21.15.86:0/3596780641 shutdown_connections 2024-03-23T17:57:35.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.015+0000 7f98ab57db80 1 -- 172.21.15.86:0/3596780641 wait complete. 2024-03-23T17:57:35.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.016+0000 7f98ab57db80 20 client.26585 trim_cache size 0 max 16384 2024-03-23T17:57:35.032 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.030+0000 7f98ab57db80 1 Processor -- start 2024-03-23T17:57:35.032 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.031+0000 7f98ab57db80 1 -- start start 2024-03-23T17:57:35.032 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.031+0000 7f98ab57db80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a4b3c1a2d0 0x55a4b3c1d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:57:35.032 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.031+0000 7f98ab57db80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4b3c1abf0 0x55a4b3c3c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:57:35.032 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.031+0000 7f98ab57db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4b3c25ab0 0x55a4b3c2c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:57:35.032 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.031+0000 7f98ab57db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4b3c1b560 con 0x55a4b3c25ab0 2024-03-23T17:57:35.032 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.031+0000 7f98ab57db80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4b3c17a90 con 0x55a4b3c1abf0 2024-03-23T17:57:35.032 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.031+0000 7f98ab57db80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4b3c23700 con 0x55a4b3c1a2d0 2024-03-23T17:57:35.034 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.031+0000 7f988fe00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4b3c1abf0 0x55a4b3c3c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:57:35.034 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.031+0000 7f988fe00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4b3c1abf0 0x55a4b3c3c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:36960/0 (socket says 172.21.15.86:36960) 2024-03-23T17:57:35.035 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.031+0000 7f988fe00700 1 -- 172.21.15.86:0/1450324426 learned_addr learned my addr 172.21.15.86:0/1450324426 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:57:35.035 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.031+0000 7f9895a00700 1 --2- 172.21.15.86:0/1450324426 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4b3c25ab0 0x55a4b3c2c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:57:35.035 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.031+0000 7f9895000700 1 --2- 172.21.15.86:0/1450324426 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a4b3c1a2d0 0x55a4b3c1d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:57:35.035 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.032+0000 7f988fe00700 1 -- 172.21.15.86:0/1450324426 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a4b3c1a2d0 msgr2=0x55a4b3c1d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:57:35.035 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.032+0000 7f988fe00700 1 --2- 172.21.15.86:0/1450324426 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a4b3c1a2d0 0x55a4b3c1d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:35.035 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.032+0000 7f988fe00700 1 -- 172.21.15.86:0/1450324426 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4b3c25ab0 msgr2=0x55a4b3c2c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:57:35.035 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.032+0000 7f988fe00700 1 --2- 172.21.15.86:0/1450324426 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4b3c25ab0 0x55a4b3c2c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:35.035 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.032+0000 7f988fe00700 1 -- 172.21.15.86:0/1450324426 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a4b399ab00 con 0x55a4b3c1abf0 2024-03-23T17:57:35.035 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.032+0000 7f988fe00700 1 --2- 172.21.15.86:0/1450324426 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4b3c1abf0 0x55a4b3c3c4e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x55a4b3c38490 tx=0x7f987c009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b1a0fe60bada4b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:57:35.035 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.032+0000 7f9882a00700 1 -- 172.21.15.86:0/1450324426 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987c003c40 con 0x55a4b3c1abf0 2024-03-23T17:57:35.035 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.032+0000 7f9882a00700 1 -- 172.21.15.86:0/1450324426 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f987c017040 con 0x55a4b3c1abf0 2024-03-23T17:57:35.035 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.032+0000 7f9882a00700 1 -- 172.21.15.86:0/1450324426 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987c0109b0 con 0x55a4b3c1abf0 2024-03-23T17:57:35.035 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.033+0000 7f98ab57db80 1 -- 172.21.15.86:0/1450324426 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4b3c1abf0 msgr2=0x55a4b3c3c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:57:35.036 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.033+0000 7f98ab57db80 1 --2- 172.21.15.86:0/1450324426 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4b3c1abf0 0x55a4b3c3c4e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x55a4b3c38490 tx=0x7f987c009260 comp rx=0 tx=0).stop 2024-03-23T17:57:35.036 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.033+0000 7f98ab57db80 1 -- 172.21.15.86:0/1450324426 shutdown_connections 2024-03-23T17:57:35.036 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.033+0000 7f98ab57db80 1 --2- 172.21.15.86:0/1450324426 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4b3c25ab0 0x55a4b3c2c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:35.036 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.033+0000 7f98ab57db80 1 --2- 172.21.15.86:0/1450324426 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a4b3c1a2d0 0x55a4b3c1d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:35.036 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.033+0000 7f98ab57db80 1 --2- 172.21.15.86:0/1450324426 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4b3c1abf0 0x55a4b3c3c4e0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:35.036 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.033+0000 7f98ab57db80 1 -- 172.21.15.86:0/1450324426 >> 172.21.15.86:0/1450324426 conn(0x55a4b3b7a640 msgr2=0x55a4b3c0fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:57:35.036 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.033+0000 7f98ab57db80 1 -- 172.21.15.86:0/1450324426 shutdown_connections 2024-03-23T17:57:35.036 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.033+0000 7f98ab57db80 1 -- 172.21.15.86:0/1450324426 wait complete. 2024-03-23T17:57:35.036 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.033+0000 7f98ab57db80 1 -- start start 2024-03-23T17:57:35.036 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.034+0000 7f98ab57db80 1 Processor -- start 2024-03-23T17:57:35.038 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.034+0000 7f98ab57db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4b3c1a2d0 0x55a4b3c2c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:57:35.039 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.034+0000 7f98ab57db80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a4b3c1abf0 0x55a4b3c2a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:57:35.039 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.034+0000 7f98ab57db80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4b3c25ab0 0x55a4b3c2ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:57:35.039 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.034+0000 7f98ab57db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4b3c2b390 con 0x55a4b3c1a2d0 2024-03-23T17:57:35.039 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.034+0000 7f98ab57db80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4b3c2b4d0 con 0x55a4b3c25ab0 2024-03-23T17:57:35.039 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.034+0000 7f98ab57db80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4b3c2b690 con 0x55a4b3c1abf0 2024-03-23T17:57:35.039 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.035+0000 7f98ab57db80 10 client.0 _ll_register_callbacks cb 0x55a4b3b12410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:57:35.039 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.035+0000 7f9895000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4b3c1a2d0 0x55a4b3c2c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:57:35.039 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.035+0000 7f988fe00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a4b3c1abf0 0x55a4b3c2a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:57:35.039 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.035+0000 7f9895000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4b3c1a2d0 0x55a4b3c2c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46376/0 (socket says 172.21.15.86:46376) 2024-03-23T17:57:35.039 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.035+0000 7f988fe00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a4b3c1abf0 0x55a4b3c2a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:50170/0 (socket says 172.21.15.86:50170) 2024-03-23T17:57:35.039 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.035+0000 7f9895000700 1 -- 172.21.15.86:0/3424821153 learned_addr learned my addr 172.21.15.86:0/3424821153 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:57:35.039 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.035+0000 7f9895a00700 1 --2- 172.21.15.86:0/3424821153 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4b3c25ab0 0x55a4b3c2ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:57:35.040 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.035+0000 7f988fe00700 1 -- 172.21.15.86:0/3424821153 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4b3c25ab0 msgr2=0x55a4b3c2ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:57:35.040 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.035+0000 7f988fe00700 1 --2- 172.21.15.86:0/3424821153 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4b3c25ab0 0x55a4b3c2ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:35.040 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.035+0000 7f988fe00700 1 -- 172.21.15.86:0/3424821153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4b3c1a2d0 msgr2=0x55a4b3c2c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:57:35.040 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.035+0000 7f988fe00700 1 --2- 172.21.15.86:0/3424821153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4b3c1a2d0 0x55a4b3c2c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:35.040 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.035+0000 7f988fe00700 1 -- 172.21.15.86:0/3424821153 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f987c004040 con 0x55a4b3c1abf0 2024-03-23T17:57:35.040 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.035+0000 7f988fe00700 1 --2- 172.21.15.86:0/3424821153 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a4b3c1abf0 0x55a4b3c2a910 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x55a4b3c1d740 tx=0x7f987c0058e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5d76bdcac79ba8a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:57:35.040 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.035+0000 7f9876a00700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-03-23T17:57:35.040 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.036+0000 7f9876a00700 1 -- 172.21.15.86:0/3424821153 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987c01d800 con 0x55a4b3c1abf0 2024-03-23T17:57:35.040 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.036+0000 7f9876a00700 1 -- 172.21.15.86:0/3424821153 <== mon.1 v2:172.21.15.152:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f987c01d960 con 0x55a4b3c1abf0 2024-03-23T17:57:35.040 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.036+0000 7f98ab57db80 1 -- 172.21.15.86:0/3424821153 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f98880079a0 con 0x55a4b3c1abf0 2024-03-23T17:57:35.040 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.036+0000 7f9876a00700 1 -- 172.21.15.86:0/3424821153 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f987c01db20 con 0x55a4b3c1abf0 2024-03-23T17:57:35.040 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.036+0000 7f98ab57db80 10 client.26581 fetch_fsmap learned FSMap version 40 2024-03-23T17:57:35.040 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.036+0000 7f98ab57db80 1 -- 172.21.15.86:0/3424821153 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55a4b3b32e40 con 0x55a4b3c1abf0 2024-03-23T17:57:35.040 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.036+0000 7f9876a00700 1 -- 172.21.15.86:0/3424821153 <== mon.1 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f98880079a0 con 0x55a4b3c1abf0 2024-03-23T17:57:35.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.036+0000 7f9876a00700 1 -- 172.21.15.86:0/3424821153 <== mon.1 v2:172.21.15.152:3300/0 5 ==== fsmap(e 40) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f987c004b20 con 0x55a4b3c1abf0 2024-03-23T17:57:35.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.036+0000 7f98ab57db80 10 client.26581 fetch_fsmap finished waiting for FSMap version 40 2024-03-23T17:57:35.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.036+0000 7f98ab57db80 10 client.26581 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:57:35.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.036+0000 7f98ab57db80 20 client.26581 populate_metadata read hostname 'smithi086' 2024-03-23T17:57:35.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.036+0000 7f98ab57db80 1 --2- 172.21.15.86:0/3424821153 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55a4b3c88780 0x55a4b3c8aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:57:35.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.036+0000 7f98ab57db80 4 client.26581 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T17:57:35.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.036+0000 7f98ab57db80 1 -- 172.21.15.86:0/3424821153 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a4b39bbc90 con 0x55a4b3c88780 2024-03-23T17:57:35.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.037+0000 7f9895000700 1 --2- 172.21.15.86:0/3424821153 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55a4b3c88780 0x55a4b3c8aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:57:35.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.037+0000 7f9895000700 1 --2- 172.21.15.86:0/3424821153 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55a4b3c88780 0x55a4b3c8aba0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=957b61cd58b8d2c7 server_cookie=40323c4ac360d57b in_seq=0 out_seq=0 2024-03-23T17:57:35.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.037+0000 7f9876a00700 10 client.26581 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:57:35.081 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.080+0000 7f9876a00700 1 -- 172.21.15.86:0/3424821153 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a4b39bbc90 con 0x55a4b3c88780 2024-03-23T17:57:35.081 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.080+0000 7f9876a00700 10 client.26581 handle_command_reply: tid=0 2024-03-23T17:57:35.081 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.080+0000 7f98ab57db80 1 client.26581 shutdown 2024-03-23T17:57:35.081 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.080+0000 7f98ab57db80 1 -- 172.21.15.86:0/3424821153 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a4b3c1abf0 msgr2=0x55a4b3c2a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:57:35.081 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.080+0000 7f98ab57db80 1 --2- 172.21.15.86:0/3424821153 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a4b3c1abf0 0x55a4b3c2a910 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x55a4b3c1d740 tx=0x7f987c0058e0 comp rx=0 tx=0).stop 2024-03-23T17:57:35.082 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.080+0000 7f98ab57db80 1 -- 172.21.15.86:0/3424821153 shutdown_connections 2024-03-23T17:57:35.082 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.080+0000 7f98ab57db80 1 --2- 172.21.15.86:0/3424821153 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55a4b3c88780 0x55a4b3c8aba0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:35.082 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.080+0000 7f98ab57db80 1 --2- 172.21.15.86:0/3424821153 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a4b3c25ab0 0x55a4b3c2ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:35.082 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.080+0000 7f98ab57db80 1 --2- 172.21.15.86:0/3424821153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4b3c1a2d0 0x55a4b3c2c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:35.082 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.080+0000 7f9876a00700 0 client.26581 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:57:35.082 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.080+0000 7f98ab57db80 1 --2- 172.21.15.86:0/3424821153 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a4b3c1abf0 0x55a4b3c2a910 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:35.082 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.080+0000 7f98ab57db80 1 -- 172.21.15.86:0/3424821153 >> 172.21.15.86:0/3424821153 conn(0x55a4b3b7a640 msgr2=0x55a4b3c16100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:57:35.082 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.080+0000 7f98ab57db80 1 -- 172.21.15.86:0/3424821153 shutdown_connections 2024-03-23T17:57:35.082 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.080+0000 7f98ab57db80 1 -- 172.21.15.86:0/3424821153 wait complete. 2024-03-23T17:57:35.082 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.080+0000 7f98ab57db80 20 client.26581 trim_cache size 0 max 16384 2024-03-23T17:57:35.083 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-03-23T17:57:35.083 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-03-23T17:57:35.083 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-03-23T17:57:35.083 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-03-23T17:57:35.083 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.081+0000 7f9896400700 1 -- 172.21.15.86:0/715379916 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f98780a1290 msgr2=0x7f98780a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:57:35.083 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.081+0000 7f9896400700 1 --2- 172.21.15.86:0/715379916 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f98780a1290 0x7f98780a3750 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f988802b000 tx=0x7f9888026000 comp rx=0 tx=0).stop 2024-03-23T17:57:35.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.081+0000 7f9896400700 1 -- 172.21.15.86:0/715379916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9890100ae0 msgr2=0x7f9890199100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:57:35.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.081+0000 7f9896400700 1 --2- 172.21.15.86:0/715379916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9890100ae0 0x7f9890199100 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f988400e9a0 tx=0x7f988400ecb0 comp rx=0 tx=0).stop 2024-03-23T17:57:35.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.081+0000 7f9896400700 1 -- 172.21.15.86:0/715379916 shutdown_connections 2024-03-23T17:57:35.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.081+0000 7f9896400700 1 --2- 172.21.15.86:0/715379916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9890106b80 0x7f989019d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:35.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.081+0000 7f9896400700 1 --2- 172.21.15.86:0/715379916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9890100ae0 0x7f9890199100 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:35.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.081+0000 7f9896400700 1 --2- 172.21.15.86:0/715379916 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f98780a1290 0x7f98780a3750 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:35.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.081+0000 7f9896400700 1 --2- 172.21.15.86:0/715379916 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f98901014a0 0x7f9890199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:35.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.081+0000 7f9896400700 1 -- 172.21.15.86:0/715379916 >> 172.21.15.86:0/715379916 conn(0x7f98900fc610 msgr2=0x7f98900fcbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:57:35.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.081+0000 7f9896400700 1 -- 172.21.15.86:0/715379916 shutdown_connections 2024-03-23T17:57:35.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:35.081+0000 7f9896400700 1 -- 172.21.15.86:0/715379916 wait complete. 2024-03-23T17:57:35.906 INFO:tasks.workunit.client.0.smithi086.stdout: 1 372050 6.51 MB/sec execute 240 sec latency 45.381 ms 2024-03-23T17:57:36.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:57:35 smithi086 ceph-mon[38221]: pgmap v213: 97 pgs: 97 active+clean; 142 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.6 MiB/s wr, 321 op/s 2024-03-23T17:57:36.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:57:35 smithi092 ceph-mon[79089]: pgmap v213: 97 pgs: 97 active+clean; 142 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.6 MiB/s wr, 321 op/s 2024-03-23T17:57:36.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:57:35 smithi152 ceph-mon[47949]: pgmap v213: 97 pgs: 97 active+clean; 142 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.6 MiB/s wr, 321 op/s 2024-03-23T17:57:36.906 INFO:tasks.workunit.client.0.smithi086.stdout: 1 373212 6.51 MB/sec execute 241 sec latency 19.094 ms 2024-03-23T17:57:37.906 INFO:tasks.workunit.client.0.smithi086.stdout: 1 374410 6.50 MB/sec execute 242 sec latency 16.543 ms 2024-03-23T17:57:38.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:57:37 smithi086 ceph-mon[38221]: pgmap v214: 97 pgs: 97 active+clean; 142 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 201 op/s 2024-03-23T17:57:38.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:57:37 smithi092 ceph-mon[79089]: pgmap v214: 97 pgs: 97 active+clean; 142 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 201 op/s 2024-03-23T17:57:38.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:57:37 smithi152 ceph-mon[47949]: pgmap v214: 97 pgs: 97 active+clean; 142 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 201 op/s 2024-03-23T17:57:38.906 INFO:tasks.workunit.client.0.smithi086.stdout: 1 376223 6.53 MB/sec execute 243 sec latency 15.750 ms 2024-03-23T17:57:39.906 INFO:tasks.workunit.client.0.smithi086.stdout: 1 377384 6.52 MB/sec execute 244 sec latency 12.363 ms 2024-03-23T17:57:40.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:57:39 smithi086 ceph-mon[38221]: pgmap v215: 97 pgs: 97 active+clean; 142 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 201 op/s 2024-03-23T17:57:40.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:57:39 smithi092 ceph-mon[79089]: pgmap v215: 97 pgs: 97 active+clean; 142 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 201 op/s 2024-03-23T17:57:40.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:57:39 smithi152 ceph-mon[47949]: pgmap v215: 97 pgs: 97 active+clean; 142 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 201 op/s 2024-03-23T17:57:40.906 INFO:tasks.workunit.client.0.smithi086.stdout: 1 378986 6.53 MB/sec execute 245 sec latency 39.232 ms 2024-03-23T17:57:41.906 INFO:tasks.workunit.client.0.smithi086.stdout: 1 380171 6.53 MB/sec execute 246 sec latency 14.888 ms 2024-03-23T17:57:42.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:57:41 smithi086 ceph-mon[38221]: pgmap v216: 97 pgs: 97 active+clean; 139 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.4 MiB/s wr, 316 op/s 2024-03-23T17:57:42.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:57:41 smithi092 ceph-mon[79089]: pgmap v216: 97 pgs: 97 active+clean; 139 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.4 MiB/s wr, 316 op/s 2024-03-23T17:57:42.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:57:41 smithi152 ceph-mon[47949]: pgmap v216: 97 pgs: 97 active+clean; 139 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.4 MiB/s wr, 316 op/s 2024-03-23T17:57:42.906 INFO:tasks.workunit.client.0.smithi086.stdout: 1 382042 6.53 MB/sec execute 247 sec latency 17.661 ms 2024-03-23T17:57:43.906 INFO:tasks.workunit.client.0.smithi086.stdout: 1 383548 6.55 MB/sec execute 248 sec latency 10.817 ms 2024-03-23T17:57:44.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:57:43 smithi086 ceph-mon[38221]: pgmap v217: 97 pgs: 97 active+clean; 139 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 214 op/s 2024-03-23T17:57:44.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:57:43 smithi092 ceph-mon[79089]: pgmap v217: 97 pgs: 97 active+clean; 139 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 214 op/s 2024-03-23T17:57:44.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:57:43 smithi152 ceph-mon[47949]: pgmap v217: 97 pgs: 97 active+clean; 139 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 214 op/s 2024-03-23T17:57:44.907 INFO:tasks.workunit.client.0.smithi086.stdout: 1 384753 6.55 MB/sec execute 249 sec latency 19.590 ms 2024-03-23T17:57:45.907 INFO:tasks.workunit.client.0.smithi086.stdout: 1 386060 6.55 MB/sec execute 250 sec latency 57.302 ms 2024-03-23T17:57:46.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:57:45 smithi086 ceph-mon[38221]: pgmap v218: 97 pgs: 97 active+clean; 145 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.5 MiB/s wr, 324 op/s 2024-03-23T17:57:46.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:57:45 smithi092 ceph-mon[79089]: pgmap v218: 97 pgs: 97 active+clean; 145 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.5 MiB/s wr, 324 op/s 2024-03-23T17:57:46.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:57:45 smithi152 ceph-mon[47949]: pgmap v218: 97 pgs: 97 active+clean; 145 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.5 MiB/s wr, 324 op/s 2024-03-23T17:57:46.907 INFO:tasks.workunit.client.0.smithi086.stdout: 1 387252 6.55 MB/sec execute 251 sec latency 11.190 ms 2024-03-23T17:57:47.907 INFO:tasks.workunit.client.0.smithi086.stdout: 1 389105 6.55 MB/sec execute 252 sec latency 15.937 ms 2024-03-23T17:57:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.241+0000 7f5682c00700 1 -- 172.21.15.86:0/3239193448 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f567c101890 msgr2=0x7f567c1058d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:57:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.241+0000 7f5682c00700 1 --2- 172.21.15.86:0/3239193448 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f567c101890 0x7f567c1058d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5668009990 tx=0x7f5668009ca0 comp rx=0 tx=0).stop 2024-03-23T17:57:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.241+0000 7f5682c00700 1 -- 172.21.15.86:0/3239193448 shutdown_connections 2024-03-23T17:57:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.241+0000 7f5682c00700 1 --2- 172.21.15.86:0/3239193448 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f567c106000 0x7f567c10db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.241+0000 7f5682c00700 1 --2- 172.21.15.86:0/3239193448 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f567c100ee0 0x7f567c1012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.241+0000 7f5682c00700 1 --2- 172.21.15.86:0/3239193448 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f567c101890 0x7f567c1058d0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.241+0000 7f5682c00700 1 -- 172.21.15.86:0/3239193448 >> 172.21.15.86:0/3239193448 conn(0x7f567c0fc780 msgr2=0x7f567c0feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:57:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.241+0000 7f5682c00700 1 -- 172.21.15.86:0/3239193448 shutdown_connections 2024-03-23T17:57:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.241+0000 7f5682c00700 1 -- 172.21.15.86:0/3239193448 wait complete. 2024-03-23T17:57:48.244 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.243+0000 7f5682c00700 1 Processor -- start 2024-03-23T17:57:48.245 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.243+0000 7f5682c00700 1 -- start start 2024-03-23T17:57:48.245 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.243+0000 7f5682c00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f567c100ee0 0x7f567c199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:57:48.245 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.243+0000 7f5682c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f567c106000 0x7f567c199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:57:48.245 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.243+0000 7f5682c00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f567c19dc30 0x7f567c19e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:57:48.245 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.243+0000 7f5682c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f567c19e6b0 con 0x7f567c106000 2024-03-23T17:57:48.245 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.243+0000 7f5682c00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f567c19e820 con 0x7f567c100ee0 2024-03-23T17:57:48.245 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.243+0000 7f5682c00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f567c19e9e0 con 0x7f567c19dc30 2024-03-23T17:57:48.245 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.243+0000 7f567be00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f567c100ee0 0x7f567c199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:57:48.245 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.243+0000 7f567be00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f567c100ee0 0x7f567c199350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:58092/0 (socket says 172.21.15.86:58092) 2024-03-23T17:57:48.245 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.243+0000 7f567b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f567c106000 0x7f567c199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:57:48.245 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.243+0000 7f567be00700 1 -- 172.21.15.86:0/3471239958 learned_addr learned my addr 172.21.15.86:0/3471239958 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:57:48.245 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.244+0000 7f5680c00700 1 --2- 172.21.15.86:0/3471239958 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f567c19dc30 0x7f567c19e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:57:48.246 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.244+0000 7f567be00700 1 -- 172.21.15.86:0/3471239958 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f567c19dc30 msgr2=0x7f567c19e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:57:48.246 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.244+0000 7f567be00700 1 --2- 172.21.15.86:0/3471239958 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f567c19dc30 0x7f567c19e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:48.246 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.244+0000 7f567be00700 1 -- 172.21.15.86:0/3471239958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f567c106000 msgr2=0x7f567c199890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:57:48.246 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.244+0000 7f567be00700 1 --2- 172.21.15.86:0/3471239958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f567c106000 0x7f567c199890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:48.246 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.244+0000 7f567be00700 1 -- 172.21.15.86:0/3471239958 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56680095f0 con 0x7f567c100ee0 2024-03-23T17:57:48.246 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.244+0000 7f567b400700 1 --2- 172.21.15.86:0/3471239958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f567c106000 0x7f567c199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:57:48.246 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.244+0000 7f567be00700 1 --2- 172.21.15.86:0/3471239958 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f567c100ee0 0x7f567c199350 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f566c00e9a0 tx=0x7f566c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:57:48.246 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.244+0000 7f5678c00700 1 -- 172.21.15.86:0/3471239958 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f566c009790 con 0x7f567c100ee0 2024-03-23T17:57:48.246 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.244+0000 7f5678c00700 1 -- 172.21.15.86:0/3471239958 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f566c010430 con 0x7f567c100ee0 2024-03-23T17:57:48.246 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.244+0000 7f5678c00700 1 -- 172.21.15.86:0/3471239958 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f566c018680 con 0x7f567c100ee0 2024-03-23T17:57:48.246 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.244+0000 7f5682c00700 1 -- 172.21.15.86:0/3471239958 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f567c1a5f40 con 0x7f567c100ee0 2024-03-23T17:57:48.246 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.244+0000 7f5682c00700 1 -- 172.21.15.86:0/3471239958 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f567c1a6360 con 0x7f567c100ee0 2024-03-23T17:57:48.247 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.248+0000 7f5678c00700 1 -- 172.21.15.86:0/3471239958 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f566c0105a0 con 0x7f567c100ee0 2024-03-23T17:57:48.250 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.249+0000 7f5696734b80 1 Processor -- start 2024-03-23T17:57:48.250 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.249+0000 7f5678c00700 1 --2- 172.21.15.86:0/3471239958 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f56600a1310 0x7f56600a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:57:48.251 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.249+0000 7f5696734b80 1 -- start start 2024-03-23T17:57:48.251 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.249+0000 7f5678c00700 1 -- 172.21.15.86:0/3471239958 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f566c014070 con 0x7f567c100ee0 2024-03-23T17:57:48.251 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.249+0000 7f5696734b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e8a6c9100 0x560e8a6c94e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:57:48.251 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.249+0000 7f5680c00700 1 --2- 172.21.15.86:0/3471239958 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f56600a1310 0x7f56600a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:57:48.251 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.249+0000 7f5696734b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e8a6c9a20 0x560e8a6d4390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:57:48.251 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.249+0000 7f5696734b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e8a6d48d0 0x560e8a6d6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:57:48.251 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.249+0000 7f5696734b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e8a622930 con 0x560e8a6c9100 2024-03-23T17:57:48.251 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.249+0000 7f5696734b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e8a6202f0 con 0x560e8a6c9a20 2024-03-23T17:57:48.251 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.249+0000 7f5696734b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e8a61bbe0 con 0x560e8a6d48d0 2024-03-23T17:57:48.251 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.250+0000 7f567be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e8a6c9100 0x560e8a6c94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:57:48.251 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.250+0000 7f567b400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e8a6c9a20 0x560e8a6d4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:57:48.251 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.250+0000 7f567b400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e8a6c9a20 0x560e8a6d4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:58094/0 (socket says 172.21.15.86:58094) 2024-03-23T17:57:48.251 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.250+0000 7f567b400700 1 -- 172.21.15.86:0/1883433668 learned_addr learned my addr 172.21.15.86:0/1883433668 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:57:48.251 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.250+0000 7f5680c00700 1 --2- 172.21.15.86:0/1883433668 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e8a6d48d0 0x560e8a6d6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:57:48.252 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.250+0000 7f567b400700 1 -- 172.21.15.86:0/1883433668 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e8a6d48d0 msgr2=0x560e8a6d6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:57:48.252 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.250+0000 7f567b400700 1 --2- 172.21.15.86:0/1883433668 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e8a6d48d0 0x560e8a6d6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:48.252 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.250+0000 7f567b400700 1 -- 172.21.15.86:0/1883433668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e8a6c9100 msgr2=0x560e8a6c94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:57:48.252 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.250+0000 7f567b400700 1 --2- 172.21.15.86:0/1883433668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e8a6c9100 0x560e8a6c94e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:48.252 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.250+0000 7f567b400700 1 -- 172.21.15.86:0/1883433668 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e8a49d350 con 0x560e8a6c9a20 2024-03-23T17:57:48.252 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.250+0000 7f567be00700 1 --2- 172.21.15.86:0/1883433668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e8a6c9100 0x560e8a6c94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:57:48.252 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.250+0000 7f5680c00700 1 --2- 172.21.15.86:0/3471239958 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f56600a1310 0x7f56600a37d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f567400f540 tx=0x7f5674009920 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:57:48.252 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.250+0000 7f567b400700 1 --2- 172.21.15.86:0/1883433668 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e8a6c9a20 0x560e8a6d4390 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f567c06a4f0 tx=0x7f5668005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0ee28f463cf64cb server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:57:48.255 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.251+0000 7f5671200700 1 -- 172.21.15.86:0/1883433668 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f566801d070 con 0x560e8a6c9a20 2024-03-23T17:57:48.255 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.251+0000 7f5671200700 1 -- 172.21.15.86:0/1883433668 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f566801e040 con 0x560e8a6c9a20 2024-03-23T17:57:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.251+0000 7f5671200700 1 -- 172.21.15.86:0/1883433668 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5668004bb0 con 0x560e8a6c9a20 2024-03-23T17:57:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.252+0000 7f5696734b80 1 -- 172.21.15.86:0/1883433668 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e8a6c9a20 msgr2=0x560e8a6d4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:57:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.252+0000 7f5696734b80 1 --2- 172.21.15.86:0/1883433668 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e8a6c9a20 0x560e8a6d4390 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f567c06a4f0 tx=0x7f5668005dc0 comp rx=0 tx=0).stop 2024-03-23T17:57:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.252+0000 7f5696734b80 1 -- 172.21.15.86:0/1883433668 shutdown_connections 2024-03-23T17:57:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.252+0000 7f5696734b80 1 --2- 172.21.15.86:0/1883433668 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e8a6d48d0 0x560e8a6d6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.252+0000 7f5696734b80 1 --2- 172.21.15.86:0/1883433668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e8a6c9100 0x560e8a6c94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.252+0000 7f5696734b80 1 --2- 172.21.15.86:0/1883433668 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e8a6c9a20 0x560e8a6d4390 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.252+0000 7f5696734b80 1 -- 172.21.15.86:0/1883433668 >> 172.21.15.86:0/1883433668 conn(0x560e8a629640 msgr2=0x560e8a6c5520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:57:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.252+0000 7f5696734b80 1 -- 172.21.15.86:0/1883433668 shutdown_connections 2024-03-23T17:57:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.252+0000 7f5696734b80 1 -- 172.21.15.86:0/1883433668 wait complete. 2024-03-23T17:57:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.253+0000 7f5696734b80 1 -- start start 2024-03-23T17:57:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.253+0000 7f5696734b80 1 Processor -- start 2024-03-23T17:57:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.253+0000 7f5696734b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e8a6c9100 0x560e8a6d4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:57:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.253+0000 7f5696734b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e8a6c9a20 0x560e8a6d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:57:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.253+0000 7f5696734b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e8a6d48d0 0x560e8a6d1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:57:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.253+0000 7f5696734b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e8a5b7d20 con 0x560e8a6c9100 2024-03-23T17:57:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.253+0000 7f5696734b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e8a5b79c0 con 0x560e8a6c9a20 2024-03-23T17:57:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.253+0000 7f5696734b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e8a49f640 con 0x560e8a6d48d0 2024-03-23T17:57:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.253+0000 7f5696734b80 10 client.0 _ll_register_callbacks cb 0x560e8a5c1410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:57:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.254+0000 7f567b400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e8a6c9a20 0x560e8a6d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:57:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.254+0000 7f567b400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e8a6c9a20 0x560e8a6d1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:58102/0 (socket says 172.21.15.86:58102) 2024-03-23T17:57:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.254+0000 7f567b400700 1 -- 172.21.15.86:0/543047048 learned_addr learned my addr 172.21.15.86:0/543047048 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:57:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.254+0000 7f5680c00700 1 --2- 172.21.15.86:0/543047048 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e8a6d48d0 0x560e8a6d1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:57:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.254+0000 7f567be00700 1 --2- 172.21.15.86:0/543047048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e8a6c9100 0x560e8a6d4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:57:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.254+0000 7f567b400700 1 -- 172.21.15.86:0/543047048 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e8a6d48d0 msgr2=0x560e8a6d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:57:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.254+0000 7f567b400700 1 --2- 172.21.15.86:0/543047048 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e8a6d48d0 0x560e8a6d1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.254+0000 7f567b400700 1 -- 172.21.15.86:0/543047048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e8a6c9100 msgr2=0x560e8a6d4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:57:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.254+0000 7f567b400700 1 --2- 172.21.15.86:0/543047048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e8a6c9100 0x560e8a6d4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.254+0000 7f567b400700 1 -- 172.21.15.86:0/543047048 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f567c1a6360 con 0x560e8a6c9a20 2024-03-23T17:57:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.254+0000 7f567b400700 1 --2- 172.21.15.86:0/543047048 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e8a6c9a20 0x560e8a6d1580 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f56680098e0 tx=0x7f566800fb70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ffe8f1082cec5e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:57:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.254+0000 7f5665600700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T17:57:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.254+0000 7f5665600700 1 -- 172.21.15.86:0/543047048 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f566801d040 con 0x560e8a6c9a20 2024-03-23T17:57:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.254+0000 7f5696734b80 1 -- 172.21.15.86:0/543047048 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560e8a5dc320 con 0x560e8a6c9a20 2024-03-23T17:57:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.255+0000 7f5665600700 1 -- 172.21.15.86:0/543047048 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f5668022400 con 0x560e8a6c9a20 2024-03-23T17:57:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.255+0000 7f5665600700 1 -- 172.21.15.86:0/543047048 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5668005bc0 con 0x560e8a6c9a20 2024-03-23T17:57:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.255+0000 7f5696734b80 10 client.26603 fetch_fsmap learned FSMap version 40 2024-03-23T17:57:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.255+0000 7f5696734b80 1 -- 172.21.15.86:0/543047048 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x7f567c10e3f0 con 0x560e8a6c9a20 2024-03-23T17:57:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.255+0000 7f5665600700 1 -- 172.21.15.86:0/543047048 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560e8a5dc320 con 0x560e8a6c9a20 2024-03-23T17:57:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.255+0000 7f5665600700 1 -- 172.21.15.86:0/543047048 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 40) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f566803d860 con 0x560e8a6c9a20 2024-03-23T17:57:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.255+0000 7f5696734b80 10 client.26603 fetch_fsmap finished waiting for FSMap version 40 2024-03-23T17:57:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.255+0000 7f5696734b80 10 client.26603 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:57:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.255+0000 7f5696734b80 20 client.26603 populate_metadata read hostname 'smithi086' 2024-03-23T17:57:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.255+0000 7f5696734b80 1 --2- 172.21.15.86:0/543047048 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x560e8a6ec850 0x560e8a6eec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:57:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.255+0000 7f5696734b80 4 client.26603 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:57:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.255+0000 7f5696734b80 1 -- 172.21.15.86:0/543047048 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560e8a6146a0 con 0x560e8a6ec850 2024-03-23T17:57:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.255+0000 7f567be00700 1 --2- 172.21.15.86:0/543047048 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x560e8a6ec850 0x560e8a6eec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:57:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.256+0000 7f5665600700 10 client.26603 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:57:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.256+0000 7f567be00700 1 --2- 172.21.15.86:0/543047048 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x560e8a6ec850 0x560e8a6eec70 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d320053669f21d3 server_cookie=2649ce93d60bdbb5 in_seq=0 out_seq=0 2024-03-23T17:57:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.256+0000 7f5665600700 1 -- 172.21.15.86:0/543047048 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x560e8a6146a0 con 0x560e8a6ec850 2024-03-23T17:57:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.256+0000 7f5665600700 10 client.26603 handle_command_reply: tid=0 2024-03-23T17:57:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.256+0000 7f5696734b80 1 client.26603 shutdown 2024-03-23T17:57:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.257+0000 7f5696734b80 1 -- 172.21.15.86:0/543047048 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e8a6c9a20 msgr2=0x560e8a6d1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:57:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.257+0000 7f5696734b80 1 --2- 172.21.15.86:0/543047048 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e8a6c9a20 0x560e8a6d1580 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f56680098e0 tx=0x7f566800fb70 comp rx=0 tx=0).stop 2024-03-23T17:57:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.257+0000 7f5696734b80 1 -- 172.21.15.86:0/543047048 shutdown_connections 2024-03-23T17:57:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.257+0000 7f5696734b80 1 --2- 172.21.15.86:0/543047048 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x560e8a6ec850 0x560e8a6eec70 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.257+0000 7f5696734b80 1 --2- 172.21.15.86:0/543047048 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e8a6d48d0 0x560e8a6d1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.257+0000 7f5696734b80 1 --2- 172.21.15.86:0/543047048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e8a6c9100 0x560e8a6d4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.257+0000 7f5665600700 0 client.26603 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:57:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.257+0000 7f5696734b80 1 --2- 172.21.15.86:0/543047048 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e8a6c9a20 0x560e8a6d1580 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.257+0000 7f5696734b80 1 -- 172.21.15.86:0/543047048 >> 172.21.15.86:0/543047048 conn(0x560e8a629640 msgr2=0x560e8a6cb0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:57:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.257+0000 7f5696734b80 1 -- 172.21.15.86:0/543047048 shutdown_connections 2024-03-23T17:57:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.257+0000 7f5696734b80 1 -- 172.21.15.86:0/543047048 wait complete. 2024-03-23T17:57:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.257+0000 7f5696734b80 20 client.26603 trim_cache size 0 max 16384 2024-03-23T17:57:48.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:57:47 smithi086 ceph-mon[38221]: pgmap v219: 97 pgs: 97 active+clean; 145 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 225 op/s 2024-03-23T17:57:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.272+0000 7f5696734b80 1 Processor -- start 2024-03-23T17:57:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.272+0000 7f5696734b80 1 -- start start 2024-03-23T17:57:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.272+0000 7f5696734b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e8a6c9100 0x560e8a6cb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:57:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.272+0000 7f5696734b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e8a6c9a20 0x560e8a6e3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:57:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.273+0000 7f5696734b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e8a6d48d0 0x560e8a6bf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:57:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.273+0000 7f5696734b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e8a690780 con 0x560e8a6c9100 2024-03-23T17:57:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.273+0000 7f5696734b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e8a6c9e00 con 0x560e8a6d48d0 2024-03-23T17:57:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.273+0000 7f5696734b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e8a6e9820 con 0x560e8a6c9a20 2024-03-23T17:57:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.273+0000 7f5680c00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e8a6d48d0 0x560e8a6bf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:57:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.273+0000 7f567be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e8a6c9100 0x560e8a6cb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:57:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.273+0000 7f5680c00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e8a6d48d0 0x560e8a6bf500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:58114/0 (socket says 172.21.15.86:58114) 2024-03-23T17:57:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.273+0000 7f5680c00700 1 -- 172.21.15.86:0/1966013460 learned_addr learned my addr 172.21.15.86:0/1966013460 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:57:48.274 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.273+0000 7f567b400700 1 --2- 172.21.15.86:0/1966013460 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e8a6c9a20 0x560e8a6e3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:57:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.273+0000 7f5680c00700 1 -- 172.21.15.86:0/1966013460 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e8a6c9a20 msgr2=0x560e8a6e3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:57:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.273+0000 7f5680c00700 1 --2- 172.21.15.86:0/1966013460 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e8a6c9a20 0x560e8a6e3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.273+0000 7f5680c00700 1 -- 172.21.15.86:0/1966013460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e8a6c9100 msgr2=0x560e8a6cb7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:57:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.273+0000 7f5680c00700 1 --2- 172.21.15.86:0/1966013460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e8a6c9100 0x560e8a6cb7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.273+0000 7f5680c00700 1 -- 172.21.15.86:0/1966013460 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e8a449b00 con 0x560e8a6d48d0 2024-03-23T17:57:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.273+0000 7f567b400700 1 --2- 172.21.15.86:0/1966013460 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e8a6c9a20 0x560e8a6e3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:57:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.273+0000 7f567be00700 1 --2- 172.21.15.86:0/1966013460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e8a6c9100 0x560e8a6cb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:57:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.274+0000 7f5680c00700 1 --2- 172.21.15.86:0/1966013460 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e8a6d48d0 0x560e8a6bf500 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5674019890 tx=0x7f567401c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b9fee08530985f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:57:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.274+0000 7f5671c00700 1 -- 172.21.15.86:0/1966013460 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567400e040 con 0x560e8a6d48d0 2024-03-23T17:57:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.274+0000 7f5671c00700 1 -- 172.21.15.86:0/1966013460 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f567400f040 con 0x560e8a6d48d0 2024-03-23T17:57:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.274+0000 7f5671c00700 1 -- 172.21.15.86:0/1966013460 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5674023b20 con 0x560e8a6d48d0 2024-03-23T17:57:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.275+0000 7f5696734b80 1 -- 172.21.15.86:0/1966013460 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e8a6d48d0 msgr2=0x560e8a6bf500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:57:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.275+0000 7f5696734b80 1 --2- 172.21.15.86:0/1966013460 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e8a6d48d0 0x560e8a6bf500 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5674019890 tx=0x7f567401c000 comp rx=0 tx=0).stop 2024-03-23T17:57:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.275+0000 7f5696734b80 1 -- 172.21.15.86:0/1966013460 shutdown_connections 2024-03-23T17:57:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.275+0000 7f5696734b80 1 --2- 172.21.15.86:0/1966013460 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e8a6d48d0 0x560e8a6bf500 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.275+0000 7f5696734b80 1 --2- 172.21.15.86:0/1966013460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e8a6c9100 0x560e8a6cb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.275+0000 7f5696734b80 1 --2- 172.21.15.86:0/1966013460 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e8a6c9a20 0x560e8a6e3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.275+0000 7f5696734b80 1 -- 172.21.15.86:0/1966013460 >> 172.21.15.86:0/1966013460 conn(0x560e8a629640 msgr2=0x560e8a6cb0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:57:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.275+0000 7f5696734b80 1 -- 172.21.15.86:0/1966013460 shutdown_connections 2024-03-23T17:57:48.277 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.275+0000 7f5696734b80 1 -- 172.21.15.86:0/1966013460 wait complete. 2024-03-23T17:57:48.277 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.275+0000 7f5696734b80 1 -- start start 2024-03-23T17:57:48.277 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.275+0000 7f5696734b80 1 Processor -- start 2024-03-23T17:57:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.276+0000 7f5696734b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e8a6c9100 0x560e8a6e3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:57:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.276+0000 7f5696734b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e8a6c9a20 0x560e8a6e2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:57:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.276+0000 7f5696734b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e8a6d48d0 0x560e8a6e2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:57:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.276+0000 7f5696734b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e8a6e3130 con 0x560e8a6c9a20 2024-03-23T17:57:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.276+0000 7f5696734b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e8a6e32a0 con 0x560e8a6c9100 2024-03-23T17:57:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.276+0000 7f5696734b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e8a73e390 con 0x560e8a6d48d0 2024-03-23T17:57:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.276+0000 7f5696734b80 10 client.0 _ll_register_callbacks cb 0x560e8a5c1410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:57:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.276+0000 7f567be00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e8a6c9100 0x560e8a6e3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:57:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.276+0000 7f567be00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e8a6c9100 0x560e8a6e3db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:58116/0 (socket says 172.21.15.86:58116) 2024-03-23T17:57:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.276+0000 7f567b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e8a6c9a20 0x560e8a6e2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:57:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.276+0000 7f567be00700 1 -- 172.21.15.86:0/1734805356 learned_addr learned my addr 172.21.15.86:0/1734805356 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:57:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.276+0000 7f5680c00700 1 --2- 172.21.15.86:0/1734805356 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e8a6d48d0 0x560e8a6e2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:57:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.277+0000 7f567b400700 1 -- 172.21.15.86:0/1734805356 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e8a6d48d0 msgr2=0x560e8a6e2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:57:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.277+0000 7f567b400700 1 --2- 172.21.15.86:0/1734805356 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e8a6d48d0 0x560e8a6e2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.277+0000 7f567b400700 1 -- 172.21.15.86:0/1734805356 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e8a6c9100 msgr2=0x560e8a6e3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:57:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.277+0000 7f567b400700 1 --2- 172.21.15.86:0/1734805356 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e8a6c9100 0x560e8a6e3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.277+0000 7f567b400700 1 -- 172.21.15.86:0/1734805356 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f567c1a5f40 con 0x560e8a6c9a20 2024-03-23T17:57:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.277+0000 7f567b400700 1 --2- 172.21.15.86:0/1734805356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e8a6c9a20 0x560e8a6e2410 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f5668009990 tx=0x7f566800b2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=614704ebf60d9a99 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:57:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.277+0000 7f5665600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-23T17:57:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.277+0000 7f5665600700 1 -- 172.21.15.86:0/1734805356 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f566801d070 con 0x560e8a6c9a20 2024-03-23T17:57:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.277+0000 7f5696734b80 1 -- 172.21.15.86:0/1734805356 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f566801e040 con 0x560e8a6c9a20 2024-03-23T17:57:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.277+0000 7f5665600700 1 -- 172.21.15.86:0/1734805356 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f5668000e40 con 0x560e8a6c9a20 2024-03-23T17:57:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.278+0000 7f5665600700 1 -- 172.21.15.86:0/1734805356 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5668005bc0 con 0x560e8a6c9a20 2024-03-23T17:57:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.278+0000 7f5696734b80 10 client.16557 fetch_fsmap learned FSMap version 40 2024-03-23T17:57:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.278+0000 7f5696734b80 1 -- 172.21.15.86:0/1734805356 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x7f567c10e3f0 con 0x560e8a6c9a20 2024-03-23T17:57:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.278+0000 7f5665600700 1 -- 172.21.15.86:0/1734805356 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f566804a020 con 0x560e8a6c9a20 2024-03-23T17:57:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.278+0000 7f5665600700 1 -- 172.21.15.86:0/1734805356 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 40) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f5668022a50 con 0x560e8a6c9a20 2024-03-23T17:57:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.278+0000 7f5696734b80 10 client.16557 fetch_fsmap finished waiting for FSMap version 40 2024-03-23T17:57:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.278+0000 7f5696734b80 10 client.16557 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:57:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.278+0000 7f5696734b80 20 client.16557 populate_metadata read hostname 'smithi086' 2024-03-23T17:57:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.278+0000 7f5696734b80 1 --2- 172.21.15.86:0/1734805356 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x560e8a73f730 0x560e8a741b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:57:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.278+0000 7f5696734b80 4 client.16557 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T17:57:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.278+0000 7f5696734b80 1 -- 172.21.15.86:0/1734805356 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560e8a46ac90 con 0x560e8a73f730 2024-03-23T17:57:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.278+0000 7f567be00700 1 --2- 172.21.15.86:0/1734805356 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x560e8a73f730 0x560e8a741b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:57:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.279+0000 7f5665600700 10 client.16557 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:57:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.279+0000 7f567be00700 1 --2- 172.21.15.86:0/1734805356 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x560e8a73f730 0x560e8a741b50 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2d82bfad100ec32 server_cookie=471ae08f4eb2fd0 in_seq=0 out_seq=0 2024-03-23T17:57:48.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:57:47 smithi092 ceph-mon[79089]: pgmap v219: 97 pgs: 97 active+clean; 145 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 225 op/s 2024-03-23T17:57:48.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.315+0000 7f5665600700 1 -- 172.21.15.86:0/1734805356 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560e8a46ac90 con 0x560e8a73f730 2024-03-23T17:57:48.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.315+0000 7f5665600700 10 client.16557 handle_command_reply: tid=0 2024-03-23T17:57:48.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.315+0000 7f5696734b80 1 client.16557 shutdown 2024-03-23T17:57:48.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.315+0000 7f5696734b80 1 -- 172.21.15.86:0/1734805356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e8a6c9a20 msgr2=0x560e8a6e2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:57:48.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.315+0000 7f5696734b80 1 --2- 172.21.15.86:0/1734805356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e8a6c9a20 0x560e8a6e2410 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f5668009990 tx=0x7f566800b2c0 comp rx=0 tx=0).stop 2024-03-23T17:57:48.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.315+0000 7f5696734b80 1 -- 172.21.15.86:0/1734805356 shutdown_connections 2024-03-23T17:57:48.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.315+0000 7f5696734b80 1 --2- 172.21.15.86:0/1734805356 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x560e8a73f730 0x560e8a741b50 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:48.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.315+0000 7f5696734b80 1 --2- 172.21.15.86:0/1734805356 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e8a6d48d0 0x560e8a6e2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:48.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.315+0000 7f5696734b80 1 --2- 172.21.15.86:0/1734805356 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e8a6c9100 0x560e8a6e3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:48.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.315+0000 7f5696734b80 1 --2- 172.21.15.86:0/1734805356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e8a6c9a20 0x560e8a6e2410 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:48.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.315+0000 7f5696734b80 1 -- 172.21.15.86:0/1734805356 >> 172.21.15.86:0/1734805356 conn(0x560e8a629640 msgr2=0x560e8a6c4d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:57:48.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.315+0000 7f5665600700 0 client.16557 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:57:48.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.315+0000 7f5696734b80 1 -- 172.21.15.86:0/1734805356 shutdown_connections 2024-03-23T17:57:48.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.315+0000 7f5696734b80 1 -- 172.21.15.86:0/1734805356 wait complete. 2024-03-23T17:57:48.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.316+0000 7f5696734b80 20 client.16557 trim_cache size 0 max 16384 2024-03-23T17:57:48.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-03-23T17:57:48.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-03-23T17:57:48.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-03-23T17:57:48.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-03-23T17:57:48.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.316+0000 7f5682c00700 1 -- 172.21.15.86:0/3471239958 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f56600a1310 msgr2=0x7f56600a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:57:48.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.316+0000 7f5682c00700 1 --2- 172.21.15.86:0/3471239958 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f56600a1310 0x7f56600a37d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f567400f540 tx=0x7f5674009920 comp rx=0 tx=0).stop 2024-03-23T17:57:48.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.316+0000 7f5682c00700 1 -- 172.21.15.86:0/3471239958 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f567c100ee0 msgr2=0x7f567c199350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:57:48.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.316+0000 7f5682c00700 1 --2- 172.21.15.86:0/3471239958 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f567c100ee0 0x7f567c199350 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f566c00e9a0 tx=0x7f566c00ecb0 comp rx=0 tx=0).stop 2024-03-23T17:57:48.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.316+0000 7f5682c00700 1 -- 172.21.15.86:0/3471239958 shutdown_connections 2024-03-23T17:57:48.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.316+0000 7f5682c00700 1 --2- 172.21.15.86:0/3471239958 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f56600a1310 0x7f56600a37d0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:48.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.316+0000 7f5682c00700 1 --2- 172.21.15.86:0/3471239958 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f567c19dc30 0x7f567c19e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:48.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.316+0000 7f5682c00700 1 --2- 172.21.15.86:0/3471239958 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f567c100ee0 0x7f567c199350 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:48.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.316+0000 7f5682c00700 1 --2- 172.21.15.86:0/3471239958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f567c106000 0x7f567c199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:57:48.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.316+0000 7f5682c00700 1 -- 172.21.15.86:0/3471239958 >> 172.21.15.86:0/3471239958 conn(0x7f567c0fc780 msgr2=0x7f567c0feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:57:48.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.317+0000 7f5682c00700 1 -- 172.21.15.86:0/3471239958 shutdown_connections 2024-03-23T17:57:48.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:57:48.317+0000 7f5682c00700 1 -- 172.21.15.86:0/3471239958 wait complete. 2024-03-23T17:57:48.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:57:47 smithi152 ceph-mon[47949]: pgmap v219: 97 pgs: 97 active+clean; 145 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 225 op/s 2024-03-23T17:57:48.907 INFO:tasks.workunit.client.0.smithi086.stdout: 1 390645 6.57 MB/sec execute 253 sec latency 27.465 ms 2024-03-23T17:57:49.907 INFO:tasks.workunit.client.0.smithi086.stdout: 1 391859 6.57 MB/sec execute 254 sec latency 14.282 ms 2024-03-23T17:57:50.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:57:49 smithi086 ceph-mon[38221]: pgmap v220: 97 pgs: 97 active+clean; 145 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 225 op/s 2024-03-23T17:57:50.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:57:49 smithi092 ceph-mon[79089]: pgmap v220: 97 pgs: 97 active+clean; 145 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 225 op/s 2024-03-23T17:57:50.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:57:49 smithi152 ceph-mon[47949]: pgmap v220: 97 pgs: 97 active+clean; 145 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 225 op/s 2024-03-23T17:57:50.907 INFO:tasks.workunit.client.0.smithi086.stdout: 1 393588 6.58 MB/sec execute 255 sec latency 54.252 ms 2024-03-23T17:57:51.907 INFO:tasks.workunit.client.0.smithi086.stdout: 1 394700 6.57 MB/sec execute 256 sec latency 16.198 ms 2024-03-23T17:57:52.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:57:51 smithi086 ceph-mon[38221]: pgmap v221: 97 pgs: 97 active+clean; 143 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.5 MiB/s wr, 343 op/s 2024-03-23T17:57:52.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:57:51 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:57:52.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:57:51 smithi092 ceph-mon[79089]: pgmap v221: 97 pgs: 97 active+clean; 143 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.5 MiB/s wr, 343 op/s 2024-03-23T17:57:52.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:57:51 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:57:52.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:57:51 smithi152 ceph-mon[47949]: pgmap v221: 97 pgs: 97 active+clean; 143 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.5 MiB/s wr, 343 op/s 2024-03-23T17:57:52.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:57:51 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:57:52.907 INFO:tasks.workunit.client.0.smithi086.stdout: 1 396747 6.59 MB/sec execute 257 sec latency 14.242 ms 2024-03-23T17:57:53.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:57:52 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:57:53.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:57:52 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:57:53.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:57:52 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:57:53.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:57:52 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:57:53.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:57:52 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:57:53.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:57:52 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:57:53.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:57:52 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:57:53.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:57:52 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:57:53.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:57:52 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:57:53.907 INFO:tasks.workunit.client.0.smithi086.stdout: 1 398054 6.60 MB/sec execute 258 sec latency 12.167 ms 2024-03-23T17:57:54.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:57:53 smithi086 ceph-mon[38221]: pgmap v222: 97 pgs: 97 active+clean; 143 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 228 op/s 2024-03-23T17:57:54.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:57:53 smithi092 ceph-mon[79089]: pgmap v222: 97 pgs: 97 active+clean; 143 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 228 op/s 2024-03-23T17:57:54.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:57:53 smithi152 ceph-mon[47949]: pgmap v222: 97 pgs: 97 active+clean; 143 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 228 op/s 2024-03-23T17:57:54.907 INFO:tasks.workunit.client.0.smithi086.stdout: 1 399001 6.59 MB/sec execute 259 sec latency 243.555 ms 2024-03-23T17:57:55.907 INFO:tasks.workunit.client.0.smithi086.stdout: 1 399001 6.56 MB/sec execute 260 sec latency 1243.615 ms 2024-03-23T17:57:56.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:57:55 smithi086 ceph-mon[38221]: pgmap v223: 97 pgs: 97 active+clean; 148 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.5 MiB/s wr, 332 op/s 2024-03-23T17:57:56.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:57:55 smithi092 ceph-mon[79089]: pgmap v223: 97 pgs: 97 active+clean; 148 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.5 MiB/s wr, 332 op/s 2024-03-23T17:57:56.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:57:55 smithi152 ceph-mon[47949]: pgmap v223: 97 pgs: 97 active+clean; 148 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.5 MiB/s wr, 332 op/s 2024-03-23T17:57:56.907 INFO:tasks.workunit.client.0.smithi086.stdout: 1 401078 6.59 MB/sec execute 261 sec latency 1252.644 ms 2024-03-23T17:57:57.907 INFO:tasks.workunit.client.0.smithi086.stdout: 1 402354 6.59 MB/sec execute 262 sec latency 17.088 ms 2024-03-23T17:57:58.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:57:57 smithi086 ceph-mon[38221]: pgmap v224: 97 pgs: 97 active+clean; 148 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 222 op/s 2024-03-23T17:57:58.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:57:57 smithi092 ceph-mon[79089]: pgmap v224: 97 pgs: 97 active+clean; 148 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 222 op/s 2024-03-23T17:57:58.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:57:57 smithi152 ceph-mon[47949]: pgmap v224: 97 pgs: 97 active+clean; 148 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 222 op/s 2024-03-23T17:57:58.908 INFO:tasks.workunit.client.0.smithi086.stdout: 1 402685 6.56 MB/sec execute 263 sec latency 791.372 ms 2024-03-23T17:57:59.907 INFO:tasks.workunit.client.0.smithi086.stdout: 1 404680 6.59 MB/sec execute 264 sec latency 14.928 ms 2024-03-23T17:58:00.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:57:59 smithi086 ceph-mon[38221]: pgmap v225: 97 pgs: 97 active+clean; 148 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 222 op/s 2024-03-23T17:58:00.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:57:59 smithi092 ceph-mon[79089]: pgmap v225: 97 pgs: 97 active+clean; 148 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 222 op/s 2024-03-23T17:58:00.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:57:59 smithi152 ceph-mon[47949]: pgmap v225: 97 pgs: 97 active+clean; 148 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 222 op/s 2024-03-23T17:58:00.908 INFO:tasks.workunit.client.0.smithi086.stdout: 1 405853 6.58 MB/sec execute 265 sec latency 23.648 ms 2024-03-23T17:58:01.462 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.461+0000 7ff3cb200700 1 -- 172.21.15.86:0/46313914 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff3cc101cb0 msgr2=0x7ff3cc102090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:58:01.462 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.461+0000 7ff3cb200700 1 --2- 172.21.15.86:0/46313914 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff3cc101cb0 0x7ff3cc102090 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc009990 tx=0x7ff3bc009ca0 comp rx=0 tx=0).stop 2024-03-23T17:58:01.462 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.461+0000 7ff3cb200700 1 -- 172.21.15.86:0/46313914 shutdown_connections 2024-03-23T17:58:01.462 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.461+0000 7ff3cb200700 1 --2- 172.21.15.86:0/46313914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3cc10b0d0 0x7ff3cc10d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:01.462 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.461+0000 7ff3cb200700 1 --2- 172.21.15.86:0/46313914 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff3cc101cb0 0x7ff3cc102090 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:01.462 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.461+0000 7ff3cb200700 1 --2- 172.21.15.86:0/46313914 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3cc1025d0 0x7ff3cc10aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:01.462 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.461+0000 7ff3cb200700 1 -- 172.21.15.86:0/46313914 >> 172.21.15.86:0/46313914 conn(0x7ff3cc0fb4f0 msgr2=0x7ff3cc0fd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:58:01.462 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.461+0000 7ff3cb200700 1 -- 172.21.15.86:0/46313914 shutdown_connections 2024-03-23T17:58:01.462 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.461+0000 7ff3cb200700 1 -- 172.21.15.86:0/46313914 wait complete. 2024-03-23T17:58:01.465 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.462+0000 7ff3cb200700 1 Processor -- start 2024-03-23T17:58:01.465 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.463+0000 7ff3cb200700 1 -- start start 2024-03-23T17:58:01.465 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.463+0000 7ff3cb200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3cc101cb0 0x7ff3cc0ff900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:01.465 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.463+0000 7ff3cb200700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff3cc1025d0 0x7ff3cc0ffe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:01.465 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.463+0000 7ff3cb200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3cc10b0d0 0x7ff3cc100380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:01.465 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.463+0000 7ff3cb200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3cc1a5370 con 0x7ff3cc101cb0 2024-03-23T17:58:01.465 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.463+0000 7ff3cb200700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3cc1a54e0 con 0x7ff3cc10b0d0 2024-03-23T17:58:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.463+0000 7ff3cb200700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3cc1a56a0 con 0x7ff3cc1025d0 2024-03-23T17:58:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.463+0000 7ff3c9e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3cc101cb0 0x7ff3cc0ff900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.463+0000 7ff3c9e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3cc101cb0 0x7ff3cc0ff900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55354/0 (socket says 172.21.15.86:55354) 2024-03-23T17:58:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.463+0000 7ff3c9e00700 1 -- 172.21.15.86:0/760273303 learned_addr learned my addr 172.21.15.86:0/760273303 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:58:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.463+0000 7ff3c9400700 1 --2- 172.21.15.86:0/760273303 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff3cc1025d0 0x7ff3cc0ffe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.463+0000 7ff3ca800700 1 --2- 172.21.15.86:0/760273303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3cc10b0d0 0x7ff3cc100380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.464+0000 7ff3c9e00700 1 -- 172.21.15.86:0/760273303 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff3cc1025d0 msgr2=0x7ff3cc0ffe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:58:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.464+0000 7ff3c9e00700 1 --2- 172.21.15.86:0/760273303 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff3cc1025d0 0x7ff3cc0ffe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.464+0000 7ff3c9e00700 1 -- 172.21.15.86:0/760273303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3cc10b0d0 msgr2=0x7ff3cc100380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:58:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.464+0000 7ff3c9e00700 1 --2- 172.21.15.86:0/760273303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3cc10b0d0 0x7ff3cc100380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.464+0000 7ff3c9e00700 1 -- 172.21.15.86:0/760273303 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3bc0095f0 con 0x7ff3cc101cb0 2024-03-23T17:58:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.464+0000 7ff3c9400700 1 --2- 172.21.15.86:0/760273303 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff3cc1025d0 0x7ff3cc0ffe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:58:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.464+0000 7ff3c9e00700 1 --2- 172.21.15.86:0/760273303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3cc101cb0 0x7ff3cc0ff900 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc000c00 tx=0x7ff3bc005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:58:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.464+0000 7ff3c2a00700 1 -- 172.21.15.86:0/760273303 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc01d070 con 0x7ff3cc101cb0 2024-03-23T17:58:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.464+0000 7ff3cb200700 1 -- 172.21.15.86:0/760273303 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3cc1a5920 con 0x7ff3cc101cb0 2024-03-23T17:58:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.464+0000 7ff3c2a00700 1 -- 172.21.15.86:0/760273303 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff3bc022470 con 0x7ff3cc101cb0 2024-03-23T17:58:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.464+0000 7ff3c2a00700 1 -- 172.21.15.86:0/760273303 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc017650 con 0x7ff3cc101cb0 2024-03-23T17:58:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.464+0000 7ff3cb200700 1 -- 172.21.15.86:0/760273303 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3cc1a5dd0 con 0x7ff3cc101cb0 2024-03-23T17:58:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.467+0000 7ff3c2a00700 1 -- 172.21.15.86:0/760273303 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7ff3bc0225e0 con 0x7ff3cc101cb0 2024-03-23T17:58:01.471 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.468+0000 7ff3e4bd7b80 1 Processor -- start 2024-03-23T17:58:01.471 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.469+0000 7ff3e4bd7b80 1 -- start start 2024-03-23T17:58:01.471 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.469+0000 7ff3c2a00700 1 --2- 172.21.15.86:0/760273303 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7ff3b00a1290 0x7ff3b00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:01.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.469+0000 7ff3e4bd7b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560fecc212d0 0x560fecc216b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:01.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.469+0000 7ff3c2a00700 1 -- 172.21.15.86:0/760273303 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8484+0+0 (secure 0 0 0) 0x7ff3bc0ce110 con 0x7ff3cc101cb0 2024-03-23T17:58:01.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.469+0000 7ff3e4bd7b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560fecc21bf0 0x560fecc2c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:01.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.469+0000 7ff3ca800700 1 --2- 172.21.15.86:0/760273303 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7ff3b00a1290 0x7ff3b00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:01.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.469+0000 7ff3e4bd7b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560fecc2cab0 0x560fecc2eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:01.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.469+0000 7ff3e4bd7b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fecb7a930 con 0x560fecc212d0 2024-03-23T17:58:01.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.469+0000 7ff3e4bd7b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fecb782f0 con 0x560fecc21bf0 2024-03-23T17:58:01.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.469+0000 7ff3e4bd7b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fecb73be0 con 0x560fecc2cab0 2024-03-23T17:58:01.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.469+0000 7ff3c9e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560fecc212d0 0x560fecc216b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:01.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.469+0000 7ff3c9e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560fecc212d0 0x560fecc216b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55362/0 (socket says 172.21.15.86:55362) 2024-03-23T17:58:01.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.469+0000 7ff3c9e00700 1 -- 172.21.15.86:0/1454180517 learned_addr learned my addr 172.21.15.86:0/1454180517 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:58:01.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.469+0000 7ff3c9400700 1 --2- 172.21.15.86:0/1454180517 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560fecc21bf0 0x560fecc2c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:01.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.469+0000 7ff3ca800700 1 --2- 172.21.15.86:0/1454180517 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560fecc2cab0 0x560fecc2eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:01.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.470+0000 7ff3c9e00700 1 -- 172.21.15.86:0/1454180517 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560fecc2cab0 msgr2=0x560fecc2eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:58:01.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.470+0000 7ff3c9e00700 1 --2- 172.21.15.86:0/1454180517 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560fecc2cab0 0x560fecc2eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:01.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.470+0000 7ff3c9e00700 1 -- 172.21.15.86:0/1454180517 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560fecc21bf0 msgr2=0x560fecc2c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:58:01.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.470+0000 7ff3c9e00700 1 --2- 172.21.15.86:0/1454180517 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560fecc21bf0 0x560fecc2c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:01.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.470+0000 7ff3c9e00700 1 -- 172.21.15.86:0/1454180517 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560fec9f5350 con 0x560fecc212d0 2024-03-23T17:58:01.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.470+0000 7ff3c9400700 1 --2- 172.21.15.86:0/1454180517 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560fecc21bf0 0x560fecc2c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:58:01.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.470+0000 7ff3c9e00700 1 --2- 172.21.15.86:0/1454180517 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560fecc212d0 0x560fecc216b0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc097890 tx=0x7ff3bc097750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=151e818d371d32b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:58:01.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.470+0000 7ff3ca800700 1 --2- 172.21.15.86:0/760273303 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7ff3b00a1290 0x7ff3b00a3750 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff3c4003fd0 tx=0x7ff3c4003dc0 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:58:01.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.470+0000 7ff3b7400700 1 -- 172.21.15.86:0/1454180517 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc09acf0 con 0x560fecc212d0 2024-03-23T17:58:01.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.470+0000 7ff3b7400700 1 -- 172.21.15.86:0/1454180517 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff3bc09ba90 con 0x560fecc212d0 2024-03-23T17:58:01.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.470+0000 7ff3b7400700 1 -- 172.21.15.86:0/1454180517 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc0228c0 con 0x560fecc212d0 2024-03-23T17:58:01.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.471+0000 7ff3e4bd7b80 1 -- 172.21.15.86:0/1454180517 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560fecc212d0 msgr2=0x560fecc216b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:58:01.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.471+0000 7ff3e4bd7b80 1 --2- 172.21.15.86:0/1454180517 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560fecc212d0 0x560fecc216b0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc097890 tx=0x7ff3bc097750 comp rx=0 tx=0).stop 2024-03-23T17:58:01.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.471+0000 7ff3e4bd7b80 1 -- 172.21.15.86:0/1454180517 shutdown_connections 2024-03-23T17:58:01.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.471+0000 7ff3e4bd7b80 1 --2- 172.21.15.86:0/1454180517 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560fecc2cab0 0x560fecc2eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:01.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.471+0000 7ff3e4bd7b80 1 --2- 172.21.15.86:0/1454180517 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560fecc212d0 0x560fecc216b0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.471+0000 7ff3e4bd7b80 1 --2- 172.21.15.86:0/1454180517 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560fecc21bf0 0x560fecc2c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.471+0000 7ff3e4bd7b80 1 -- 172.21.15.86:0/1454180517 >> 172.21.15.86:0/1454180517 conn(0x560fecb81640 msgr2=0x560fecc1d750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:58:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.471+0000 7ff3e4bd7b80 1 -- 172.21.15.86:0/1454180517 shutdown_connections 2024-03-23T17:58:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.471+0000 7ff3e4bd7b80 1 -- 172.21.15.86:0/1454180517 wait complete. 2024-03-23T17:58:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.472+0000 7ff3e4bd7b80 1 -- start start 2024-03-23T17:58:01.478 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.472+0000 7ff3e4bd7b80 1 Processor -- start 2024-03-23T17:58:01.478 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.473+0000 7ff3e4bd7b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560fecc212d0 0x560fecc2c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:01.478 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.473+0000 7ff3e4bd7b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560fecc21bf0 0x560fecc297d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:01.479 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.473+0000 7ff3e4bd7b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560fecc2cab0 0x560fecc29e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:01.479 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.473+0000 7ff3e4bd7b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fecb0fd20 con 0x560fecc21bf0 2024-03-23T17:58:01.479 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.473+0000 7ff3e4bd7b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fecb0f9c0 con 0x560fecc2cab0 2024-03-23T17:58:01.479 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.473+0000 7ff3e4bd7b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fec9f7640 con 0x560fecc212d0 2024-03-23T17:58:01.479 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.473+0000 7ff3e4bd7b80 10 client.0 _ll_register_callbacks cb 0x560fecb19410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:58:01.479 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.473+0000 7ff3c9400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560fecc21bf0 0x560fecc297d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:01.479 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.473+0000 7ff3c9400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560fecc21bf0 0x560fecc297d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55366/0 (socket says 172.21.15.86:55366) 2024-03-23T17:58:01.479 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.473+0000 7ff3c9400700 1 -- 172.21.15.86:0/1168001453 learned_addr learned my addr 172.21.15.86:0/1168001453 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:58:01.479 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.473+0000 7ff3c9e00700 1 --2- 172.21.15.86:0/1168001453 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560fecc212d0 0x560fecc2c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:01.479 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.473+0000 7ff3ca800700 1 --2- 172.21.15.86:0/1168001453 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560fecc2cab0 0x560fecc29e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:01.479 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.473+0000 7ff3c9400700 1 -- 172.21.15.86:0/1168001453 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560fecc212d0 msgr2=0x560fecc2c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:58:01.479 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.473+0000 7ff3c9400700 1 --2- 172.21.15.86:0/1168001453 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560fecc212d0 0x560fecc2c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:01.479 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.473+0000 7ff3c9400700 1 -- 172.21.15.86:0/1168001453 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560fecc2cab0 msgr2=0x560fecc29e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:58:01.480 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.473+0000 7ff3c9400700 1 --2- 172.21.15.86:0/1168001453 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560fecc2cab0 0x560fecc29e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:01.480 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.473+0000 7ff3c9400700 1 -- 172.21.15.86:0/1168001453 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3b8009c80 con 0x560fecc21bf0 2024-03-23T17:58:01.480 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.474+0000 7ff3c9400700 1 --2- 172.21.15.86:0/1168001453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560fecc21bf0 0x560fecc297d0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8011cf0 tx=0x7ff3b800f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cb269c797b65817 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:58:01.480 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.474+0000 7ff3abe00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-23T17:58:01.480 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.474+0000 7ff3abe00700 1 -- 172.21.15.86:0/1168001453 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b8016070 con 0x560fecc21bf0 2024-03-23T17:58:01.480 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.474+0000 7ff3abe00700 1 -- 172.21.15.86:0/1168001453 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8484+0+0 (secure 0 0 0) 0x7ff3b800fc30 con 0x560fecc21bf0 2024-03-23T17:58:01.480 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.474+0000 7ff3e4bd7b80 1 -- 172.21.15.86:0/1168001453 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560fecb34320 con 0x560fecc21bf0 2024-03-23T17:58:01.480 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.474+0000 7ff3abe00700 1 -- 172.21.15.86:0/1168001453 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff3b800fdf0 con 0x560fecc21bf0 2024-03-23T17:58:01.480 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.474+0000 7ff3e4bd7b80 10 client.16569 fetch_fsmap learned FSMap version 40 2024-03-23T17:58:01.480 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.474+0000 7ff3e4bd7b80 1 -- 172.21.15.86:0/1168001453 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x7ff3cc1a5dd0 con 0x560fecc21bf0 2024-03-23T17:58:01.481 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.474+0000 7ff3abe00700 1 -- 172.21.15.86:0/1168001453 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff3b8027020 con 0x560fecc21bf0 2024-03-23T17:58:01.481 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.474+0000 7ff3abe00700 1 -- 172.21.15.86:0/1168001453 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 40) v1 ==== 7478+0+0 (secure 0 0 0) 0x7ff3b8021070 con 0x560fecc21bf0 2024-03-23T17:58:01.481 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.474+0000 7ff3e4bd7b80 10 client.16569 fetch_fsmap finished waiting for FSMap version 40 2024-03-23T17:58:01.481 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.474+0000 7ff3e4bd7b80 10 client.16569 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:58:01.481 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.475+0000 7ff3e4bd7b80 20 client.16569 populate_metadata read hostname 'smithi086' 2024-03-23T17:58:01.481 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.475+0000 7ff3e4bd7b80 1 --2- 172.21.15.86:0/1168001453 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x560fecc3c520 0x560fecc3e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:01.481 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.475+0000 7ff3e4bd7b80 4 client.16569 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:58:01.481 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.475+0000 7ff3e4bd7b80 1 -- 172.21.15.86:0/1168001453 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560fecb6c6a0 con 0x560fecc3c520 2024-03-23T17:58:01.481 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.475+0000 7ff3c9e00700 1 --2- 172.21.15.86:0/1168001453 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x560fecc3c520 0x560fecc3e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:01.481 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.475+0000 7ff3c9e00700 1 --2- 172.21.15.86:0/1168001453 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x560fecc3c520 0x560fecc3e940 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8e89beedb0ad8e3 server_cookie=21710b1a835c6ac3 in_seq=0 out_seq=0 2024-03-23T17:58:01.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.475+0000 7ff3abe00700 10 client.16569 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:58:01.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.476+0000 7ff3abe00700 1 -- 172.21.15.86:0/1168001453 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x560fecb6c6a0 con 0x560fecc3c520 2024-03-23T17:58:01.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.476+0000 7ff3abe00700 10 client.16569 handle_command_reply: tid=0 2024-03-23T17:58:01.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.476+0000 7ff3e4bd7b80 1 client.16569 shutdown 2024-03-23T17:58:01.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.476+0000 7ff3e4bd7b80 1 -- 172.21.15.86:0/1168001453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560fecc21bf0 msgr2=0x560fecc297d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:58:01.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.476+0000 7ff3e4bd7b80 1 --2- 172.21.15.86:0/1168001453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560fecc21bf0 0x560fecc297d0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8011cf0 tx=0x7ff3b800f3c0 comp rx=0 tx=0).stop 2024-03-23T17:58:01.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.476+0000 7ff3e4bd7b80 1 -- 172.21.15.86:0/1168001453 shutdown_connections 2024-03-23T17:58:01.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.476+0000 7ff3e4bd7b80 1 --2- 172.21.15.86:0/1168001453 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x560fecc3c520 0x560fecc3e940 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:01.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.476+0000 7ff3e4bd7b80 1 --2- 172.21.15.86:0/1168001453 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560fecc2cab0 0x560fecc29e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:01.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.476+0000 7ff3e4bd7b80 1 --2- 172.21.15.86:0/1168001453 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560fecc212d0 0x560fecc2c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:01.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.476+0000 7ff3abe00700 0 client.16569 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:58:01.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.476+0000 7ff3e4bd7b80 1 --2- 172.21.15.86:0/1168001453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560fecc21bf0 0x560fecc297d0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:01.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.476+0000 7ff3e4bd7b80 1 -- 172.21.15.86:0/1168001453 >> 172.21.15.86:0/1168001453 conn(0x560fecb81640 msgr2=0x560fecc233c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:58:01.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.476+0000 7ff3e4bd7b80 1 -- 172.21.15.86:0/1168001453 shutdown_connections 2024-03-23T17:58:01.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.476+0000 7ff3e4bd7b80 1 -- 172.21.15.86:0/1168001453 wait complete. 2024-03-23T17:58:01.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.476+0000 7ff3e4bd7b80 20 client.16569 trim_cache size 0 max 16384 2024-03-23T17:58:01.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.491+0000 7ff3e4bd7b80 1 Processor -- start 2024-03-23T17:58:01.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.491+0000 7ff3e4bd7b80 1 -- start start 2024-03-23T17:58:01.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.491+0000 7ff3e4bd7b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560fecc212d0 0x560fecc334a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:01.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.491+0000 7ff3e4bd7b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560fecc21bf0 0x560fecc17700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:01.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.491+0000 7ff3e4bd7b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560fecc2cab0 0x560fecc280c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:01.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.491+0000 7ff3e4bd7b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fecc23220 con 0x560fecc21bf0 2024-03-23T17:58:01.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.491+0000 7ff3e4bd7b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fecc1ea90 con 0x560fecc212d0 2024-03-23T17:58:01.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.491+0000 7ff3e4bd7b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fecc21110 con 0x560fecc2cab0 2024-03-23T17:58:01.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.491+0000 7ff3c9e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560fecc212d0 0x560fecc334a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:01.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.492+0000 7ff3c9e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560fecc212d0 0x560fecc334a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:46392/0 (socket says 172.21.15.86:46392) 2024-03-23T17:58:01.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.492+0000 7ff3ca800700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560fecc2cab0 0x560fecc280c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:01.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.492+0000 7ff3c9400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560fecc21bf0 0x560fecc17700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:01.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.492+0000 7ff3c9e00700 1 -- 172.21.15.86:0/2249065081 learned_addr learned my addr 172.21.15.86:0/2249065081 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:58:01.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.492+0000 7ff3c9e00700 1 -- 172.21.15.86:0/2249065081 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560fecc2cab0 msgr2=0x560fecc280c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:58:01.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.492+0000 7ff3c9e00700 1 --2- 172.21.15.86:0/2249065081 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560fecc2cab0 0x560fecc280c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:01.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.492+0000 7ff3c9e00700 1 -- 172.21.15.86:0/2249065081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560fecc21bf0 msgr2=0x560fecc17700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:58:01.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.492+0000 7ff3c9e00700 1 --2- 172.21.15.86:0/2249065081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560fecc21bf0 0x560fecc17700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:01.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.492+0000 7ff3c9e00700 1 -- 172.21.15.86:0/2249065081 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560fec9a1b00 con 0x560fecc212d0 2024-03-23T17:58:01.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.492+0000 7ff3c9e00700 1 --2- 172.21.15.86:0/2249065081 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560fecc212d0 0x560fecc334a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc009350 tx=0x7ff3bc01a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd314425a620985d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:58:01.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.492+0000 7ff3b7e00700 1 -- 172.21.15.86:0/2249065081 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc0b9040 con 0x560fecc212d0 2024-03-23T17:58:01.494 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.492+0000 7ff3b7e00700 1 -- 172.21.15.86:0/2249065081 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff3bc0a3780 con 0x560fecc212d0 2024-03-23T17:58:01.494 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.492+0000 7ff3b7e00700 1 -- 172.21.15.86:0/2249065081 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc09b5f0 con 0x560fecc212d0 2024-03-23T17:58:01.498 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.493+0000 7ff3e4bd7b80 1 -- 172.21.15.86:0/2249065081 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560fecc212d0 msgr2=0x560fecc334a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:58:01.498 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.493+0000 7ff3e4bd7b80 1 --2- 172.21.15.86:0/2249065081 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560fecc212d0 0x560fecc334a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc009350 tx=0x7ff3bc01a040 comp rx=0 tx=0).stop 2024-03-23T17:58:01.498 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.493+0000 7ff3e4bd7b80 1 -- 172.21.15.86:0/2249065081 shutdown_connections 2024-03-23T17:58:01.498 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.493+0000 7ff3e4bd7b80 1 --2- 172.21.15.86:0/2249065081 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560fecc2cab0 0x560fecc280c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:01.498 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.493+0000 7ff3e4bd7b80 1 --2- 172.21.15.86:0/2249065081 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560fecc212d0 0x560fecc334a0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:01.498 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.493+0000 7ff3e4bd7b80 1 --2- 172.21.15.86:0/2249065081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560fecc21bf0 0x560fecc17700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:01.498 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.493+0000 7ff3e4bd7b80 1 -- 172.21.15.86:0/2249065081 >> 172.21.15.86:0/2249065081 conn(0x560fecb81640 msgr2=0x560fecc16c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:58:01.498 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.493+0000 7ff3e4bd7b80 1 -- 172.21.15.86:0/2249065081 shutdown_connections 2024-03-23T17:58:01.499 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.493+0000 7ff3e4bd7b80 1 -- 172.21.15.86:0/2249065081 wait complete. 2024-03-23T17:58:01.499 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.493+0000 7ff3e4bd7b80 1 -- start start 2024-03-23T17:58:01.499 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.494+0000 7ff3e4bd7b80 1 Processor -- start 2024-03-23T17:58:01.499 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.494+0000 7ff3e4bd7b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560fecc212d0 0x560fecc31670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:01.499 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.494+0000 7ff3e4bd7b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560fecc21bf0 0x560fecc31bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:01.499 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.494+0000 7ff3e4bd7b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560fecc2cab0 0x560fecc87010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:01.499 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.494+0000 7ff3e4bd7b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fecc87550 con 0x560fecc21bf0 2024-03-23T17:58:01.499 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.494+0000 7ff3e4bd7b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fecc87690 con 0x560fecc2cab0 2024-03-23T17:58:01.499 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.494+0000 7ff3e4bd7b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fecc87850 con 0x560fecc212d0 2024-03-23T17:58:01.499 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.494+0000 7ff3e4bd7b80 10 client.0 _ll_register_callbacks cb 0x560fecb19410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:58:01.499 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.494+0000 7ff3c9400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560fecc21bf0 0x560fecc31bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:01.499 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.494+0000 7ff3c9400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560fecc21bf0 0x560fecc31bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55386/0 (socket says 172.21.15.86:55386) 2024-03-23T17:58:01.500 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.494+0000 7ff3c9400700 1 -- 172.21.15.86:0/1359296100 learned_addr learned my addr 172.21.15.86:0/1359296100 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:58:01.500 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.494+0000 7ff3ca800700 1 --2- 172.21.15.86:0/1359296100 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560fecc2cab0 0x560fecc87010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:01.500 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.494+0000 7ff3c9e00700 1 --2- 172.21.15.86:0/1359296100 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560fecc212d0 0x560fecc31670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:01.500 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.495+0000 7ff3c9400700 1 -- 172.21.15.86:0/1359296100 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560fecc212d0 msgr2=0x560fecc31670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:58:01.500 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.495+0000 7ff3c9400700 1 --2- 172.21.15.86:0/1359296100 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560fecc212d0 0x560fecc31670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:01.500 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.495+0000 7ff3c9400700 1 -- 172.21.15.86:0/1359296100 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560fecc2cab0 msgr2=0x560fecc87010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:58:01.500 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.495+0000 7ff3c9400700 1 --2- 172.21.15.86:0/1359296100 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560fecc2cab0 0x560fecc87010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:01.500 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.495+0000 7ff3c9400700 1 -- 172.21.15.86:0/1359296100 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560fec9f5350 con 0x560fecc21bf0 2024-03-23T17:58:01.500 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.495+0000 7ff3c9400700 1 --2- 172.21.15.86:0/1359296100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560fecc21bf0 0x560fecc31bb0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x560fecc173a0 tx=0x7ff3b8022d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfd49ac6beab647c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:58:01.500 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.495+0000 7ff3abe00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-23T17:58:01.501 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.495+0000 7ff3abe00700 1 -- 172.21.15.86:0/1359296100 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b8016040 con 0x560fecc21bf0 2024-03-23T17:58:01.501 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.495+0000 7ff3abe00700 1 -- 172.21.15.86:0/1359296100 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8484+0+0 (secure 0 0 0) 0x7ff3b8000e40 con 0x560fecc21bf0 2024-03-23T17:58:01.501 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.495+0000 7ff3e4bd7b80 1 -- 172.21.15.86:0/1359296100 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff3bc09ba90 con 0x560fecc21bf0 2024-03-23T17:58:01.501 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.495+0000 7ff3abe00700 1 -- 172.21.15.86:0/1359296100 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff3b80223e0 con 0x560fecc21bf0 2024-03-23T17:58:01.501 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.495+0000 7ff3e4bd7b80 10 client.16575 fetch_fsmap learned FSMap version 40 2024-03-23T17:58:01.501 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.495+0000 7ff3e4bd7b80 1 -- 172.21.15.86:0/1359296100 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x560fecb39e40 con 0x560fecc21bf0 2024-03-23T17:58:01.502 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.496+0000 7ff3abe00700 1 -- 172.21.15.86:0/1359296100 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff3b8039020 con 0x560fecc21bf0 2024-03-23T17:58:01.502 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.496+0000 7ff3abe00700 1 -- 172.21.15.86:0/1359296100 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 40) v1 ==== 7478+0+0 (secure 0 0 0) 0x7ff3b8011d20 con 0x560fecc21bf0 2024-03-23T17:58:01.502 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.496+0000 7ff3e4bd7b80 10 client.16575 fetch_fsmap finished waiting for FSMap version 40 2024-03-23T17:58:01.502 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.496+0000 7ff3e4bd7b80 10 client.16575 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:58:01.502 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.496+0000 7ff3e4bd7b80 20 client.16575 populate_metadata read hostname 'smithi086' 2024-03-23T17:58:01.502 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.496+0000 7ff3e4bd7b80 1 --2- 172.21.15.86:0/1359296100 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x560fecc8f1b0 0x560fecc915d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:01.502 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.496+0000 7ff3e4bd7b80 4 client.16575 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T17:58:01.502 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.496+0000 7ff3e4bd7b80 1 -- 172.21.15.86:0/1359296100 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560fec9c2c90 con 0x560fecc8f1b0 2024-03-23T17:58:01.502 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.496+0000 7ff3c9e00700 1 --2- 172.21.15.86:0/1359296100 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x560fecc8f1b0 0x560fecc915d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:01.502 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.496+0000 7ff3c9e00700 1 --2- 172.21.15.86:0/1359296100 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x560fecc8f1b0 0x560fecc915d0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74e70add0559fb0a server_cookie=a6362965c2d99f85 in_seq=0 out_seq=0 2024-03-23T17:58:01.502 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.496+0000 7ff3abe00700 10 client.16575 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:58:01.535 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.534+0000 7ff3abe00700 1 -- 172.21.15.86:0/1359296100 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560fec9c2c90 con 0x560fecc8f1b0 2024-03-23T17:58:01.535 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.534+0000 7ff3abe00700 10 client.16575 handle_command_reply: tid=0 2024-03-23T17:58:01.535 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.534+0000 7ff3e4bd7b80 1 client.16575 shutdown 2024-03-23T17:58:01.535 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.534+0000 7ff3e4bd7b80 1 -- 172.21.15.86:0/1359296100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560fecc21bf0 msgr2=0x560fecc31bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:58:01.535 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.534+0000 7ff3e4bd7b80 1 --2- 172.21.15.86:0/1359296100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560fecc21bf0 0x560fecc31bb0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x560fecc173a0 tx=0x7ff3b8022d50 comp rx=0 tx=0).stop 2024-03-23T17:58:01.535 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.535+0000 7ff3e4bd7b80 1 -- 172.21.15.86:0/1359296100 shutdown_connections 2024-03-23T17:58:01.535 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.535+0000 7ff3e4bd7b80 1 --2- 172.21.15.86:0/1359296100 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x560fecc8f1b0 0x560fecc915d0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:01.535 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.535+0000 7ff3e4bd7b80 1 --2- 172.21.15.86:0/1359296100 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560fecc2cab0 0x560fecc87010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:01.536 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.535+0000 7ff3e4bd7b80 1 --2- 172.21.15.86:0/1359296100 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560fecc212d0 0x560fecc31670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:01.536 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.535+0000 7ff3e4bd7b80 1 --2- 172.21.15.86:0/1359296100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560fecc21bf0 0x560fecc31bb0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:01.536 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.535+0000 7ff3e4bd7b80 1 -- 172.21.15.86:0/1359296100 >> 172.21.15.86:0/1359296100 conn(0x560fecb81640 msgr2=0x560fecc28000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:58:01.536 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.535+0000 7ff3abe00700 0 client.16575 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:58:01.536 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.535+0000 7ff3e4bd7b80 1 -- 172.21.15.86:0/1359296100 shutdown_connections 2024-03-23T17:58:01.536 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.535+0000 7ff3e4bd7b80 1 -- 172.21.15.86:0/1359296100 wait complete. 2024-03-23T17:58:01.536 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.535+0000 7ff3e4bd7b80 20 client.16575 trim_cache size 0 max 16384 2024-03-23T17:58:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-03-23T17:58:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-03-23T17:58:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-03-23T17:58:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-03-23T17:58:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.536+0000 7ff3cb200700 1 -- 172.21.15.86:0/760273303 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7ff3b00a1290 msgr2=0x7ff3b00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:58:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.536+0000 7ff3cb200700 1 --2- 172.21.15.86:0/760273303 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7ff3b00a1290 0x7ff3b00a3750 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff3c4003fd0 tx=0x7ff3c4003dc0 comp rx=0 tx=0).stop 2024-03-23T17:58:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.536+0000 7ff3cb200700 1 -- 172.21.15.86:0/760273303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3cc101cb0 msgr2=0x7ff3cc0ff900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:58:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.536+0000 7ff3cb200700 1 --2- 172.21.15.86:0/760273303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3cc101cb0 0x7ff3cc0ff900 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc000c00 tx=0x7ff3bc005740 comp rx=0 tx=0).stop 2024-03-23T17:58:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.536+0000 7ff3cb200700 1 -- 172.21.15.86:0/760273303 shutdown_connections 2024-03-23T17:58:01.539 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.536+0000 7ff3cb200700 1 --2- 172.21.15.86:0/760273303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3cc10b0d0 0x7ff3cc100380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:01.539 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.536+0000 7ff3cb200700 1 --2- 172.21.15.86:0/760273303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3cc101cb0 0x7ff3cc0ff900 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:01.539 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.536+0000 7ff3cb200700 1 --2- 172.21.15.86:0/760273303 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7ff3b00a1290 0x7ff3b00a3750 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:01.539 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.536+0000 7ff3cb200700 1 --2- 172.21.15.86:0/760273303 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff3cc1025d0 0x7ff3cc0ffe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:01.539 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.536+0000 7ff3cb200700 1 -- 172.21.15.86:0/760273303 >> 172.21.15.86:0/760273303 conn(0x7ff3cc0fb4f0 msgr2=0x7ff3cc0fc340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:58:01.539 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.537+0000 7ff3cb200700 1 -- 172.21.15.86:0/760273303 shutdown_connections 2024-03-23T17:58:01.539 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:01.537+0000 7ff3cb200700 1 -- 172.21.15.86:0/760273303 wait complete. 2024-03-23T17:58:01.908 INFO:tasks.workunit.client.0.smithi086.stdout: 1 406092 6.56 MB/sec execute 266 sec latency 795.996 ms 2024-03-23T17:58:02.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:58:01 smithi086 ceph-mon[38221]: pgmap v226: 97 pgs: 97 active+clean; 152 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 303 op/s 2024-03-23T17:58:02.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:58:01 smithi092 ceph-mon[79089]: pgmap v226: 97 pgs: 97 active+clean; 152 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 303 op/s 2024-03-23T17:58:02.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:58:01 smithi152 ceph-mon[47949]: pgmap v226: 97 pgs: 97 active+clean; 152 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 303 op/s 2024-03-23T17:58:02.908 INFO:tasks.workunit.client.0.smithi086.stdout: 1 406092 6.54 MB/sec execute 267 sec latency 1796.091 ms 2024-03-23T17:58:03.908 INFO:tasks.workunit.client.0.smithi086.stdout: 1 406092 6.51 MB/sec execute 268 sec latency 2796.210 ms 2024-03-23T17:58:04.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:58:03 smithi086 ceph-mon[38221]: pgmap v227: 97 pgs: 97 active+clean; 152 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 185 op/s 2024-03-23T17:58:04.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:58:03 smithi092 ceph-mon[79089]: pgmap v227: 97 pgs: 97 active+clean; 152 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 185 op/s 2024-03-23T17:58:04.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:58:03 smithi152 ceph-mon[47949]: pgmap v227: 97 pgs: 97 active+clean; 152 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 185 op/s 2024-03-23T17:58:04.908 INFO:tasks.workunit.client.0.smithi086.stdout: 1 406092 6.49 MB/sec execute 269 sec latency 3796.317 ms 2024-03-23T17:58:05.912 INFO:tasks.workunit.client.0.smithi086.stdout: 1 406395 6.46 MB/sec execute 270 sec latency 4506.914 ms 2024-03-23T17:58:06.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:58:05 smithi086 ceph-mon[38221]: pgmap v228: 97 pgs: 97 active+clean; 146 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.0 MiB/s wr, 272 op/s 2024-03-23T17:58:06.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:58:05 smithi086 ceph-mon[38221]: osdmap e86: 12 total, 12 up, 12 in 2024-03-23T17:58:06.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:58:05 smithi092 ceph-mon[79089]: pgmap v228: 97 pgs: 97 active+clean; 146 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.0 MiB/s wr, 272 op/s 2024-03-23T17:58:06.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:58:05 smithi092 ceph-mon[79089]: osdmap e86: 12 total, 12 up, 12 in 2024-03-23T17:58:06.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:58:05 smithi152 ceph-mon[47949]: pgmap v228: 97 pgs: 97 active+clean; 146 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.0 MiB/s wr, 272 op/s 2024-03-23T17:58:06.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:58:05 smithi152 ceph-mon[47949]: osdmap e86: 12 total, 12 up, 12 in 2024-03-23T17:58:06.908 INFO:tasks.workunit.client.0.smithi086.stdout: 1 408190 6.49 MB/sec execute 271 sec latency 20.725 ms 2024-03-23T17:58:07.910 INFO:tasks.workunit.client.0.smithi086.stdout: 1 409352 6.48 MB/sec execute 272 sec latency 12.286 ms 2024-03-23T17:58:08.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:58:07 smithi086 ceph-mon[38221]: pgmap v230: 97 pgs: 97 active+clean; 146 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 201 op/s 2024-03-23T17:58:08.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:58:07 smithi092 ceph-mon[79089]: pgmap v230: 97 pgs: 97 active+clean; 146 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 201 op/s 2024-03-23T17:58:08.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:58:07 smithi152 ceph-mon[47949]: pgmap v230: 97 pgs: 97 active+clean; 146 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 201 op/s 2024-03-23T17:58:08.908 INFO:tasks.workunit.client.0.smithi086.stdout: 1 409641 6.46 MB/sec execute 273 sec latency 608.549 ms 2024-03-23T17:58:09.909 INFO:tasks.workunit.client.0.smithi086.stdout: 1 409641 6.44 MB/sec execute 274 sec latency 1608.679 ms 2024-03-23T17:58:10.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:58:09 smithi086 ceph-mon[38221]: pgmap v231: 97 pgs: 97 active+clean; 146 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 201 op/s 2024-03-23T17:58:10.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:58:09 smithi092 ceph-mon[79089]: pgmap v231: 97 pgs: 97 active+clean; 146 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 201 op/s 2024-03-23T17:58:10.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:58:09 smithi152 ceph-mon[47949]: pgmap v231: 97 pgs: 97 active+clean; 146 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 201 op/s 2024-03-23T17:58:10.908 INFO:tasks.workunit.client.0.smithi086.stdout: 1 409641 6.41 MB/sec execute 275 sec latency 2608.751 ms 2024-03-23T17:58:11.909 INFO:tasks.workunit.client.0.smithi086.stdout: 1 411637 6.44 MB/sec execute 276 sec latency 2610.932 ms 2024-03-23T17:58:12.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:58:11 smithi086 ceph-mon[38221]: pgmap v232: 97 pgs: 97 active+clean; 145 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 185 op/s 2024-03-23T17:58:12.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:58:11 smithi092 ceph-mon[79089]: pgmap v232: 97 pgs: 97 active+clean; 145 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 185 op/s 2024-03-23T17:58:12.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:58:11 smithi152 ceph-mon[47949]: pgmap v232: 97 pgs: 97 active+clean; 145 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 185 op/s 2024-03-23T17:58:12.909 INFO:tasks.workunit.client.0.smithi086.stdout: 1 412712 6.43 MB/sec execute 277 sec latency 15.257 ms 2024-03-23T17:58:13.909 INFO:tasks.workunit.client.0.smithi086.stdout: 1 413183 6.41 MB/sec execute 278 sec latency 526.318 ms 2024-03-23T17:58:14.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:58:13 smithi086 ceph-mon[38221]: pgmap v233: 97 pgs: 97 active+clean; 145 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 185 op/s 2024-03-23T17:58:14.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:58:13 smithi092 ceph-mon[79089]: pgmap v233: 97 pgs: 97 active+clean; 145 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 185 op/s 2024-03-23T17:58:14.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:58:13 smithi152 ceph-mon[47949]: pgmap v233: 97 pgs: 97 active+clean; 145 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 185 op/s 2024-03-23T17:58:14.696 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.695+0000 7f1f90c00700 1 -- 172.21.15.86:0/593706703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f8c100f10 msgr2=0x7f1f8c1012f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:58:14.696 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.695+0000 7f1f90c00700 1 --2- 172.21.15.86:0/593706703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f8c100f10 0x7f1f8c1012f0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c009990 tx=0x7f1f7c009ca0 comp rx=0 tx=0).stop 2024-03-23T17:58:14.696 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.696+0000 7f1f90c00700 1 -- 172.21.15.86:0/593706703 shutdown_connections 2024-03-23T17:58:14.696 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.696+0000 7f1f90c00700 1 --2- 172.21.15.86:0/593706703 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1f8c106030 0x7f1f8c10dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:14.696 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.696+0000 7f1f90c00700 1 --2- 172.21.15.86:0/593706703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f8c100f10 0x7f1f8c1012f0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:14.696 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.696+0000 7f1f90c00700 1 --2- 172.21.15.86:0/593706703 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f8c1018c0 0x7f1f8c105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:14.696 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.696+0000 7f1f90c00700 1 -- 172.21.15.86:0/593706703 >> 172.21.15.86:0/593706703 conn(0x7f1f8c0fc7b0 msgr2=0x7f1f8c0febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:58:14.697 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.696+0000 7f1f90c00700 1 -- 172.21.15.86:0/593706703 shutdown_connections 2024-03-23T17:58:14.697 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.696+0000 7f1f90c00700 1 -- 172.21.15.86:0/593706703 wait complete. 2024-03-23T17:58:14.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.697+0000 7f1f90c00700 1 Processor -- start 2024-03-23T17:58:14.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.697+0000 7f1f90c00700 1 -- start start 2024-03-23T17:58:14.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.697+0000 7f1f90c00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f8c100f10 0x7f1f8c199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:14.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.697+0000 7f1f90c00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1f8c106030 0x7f1f8c199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:14.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.697+0000 7f1f90c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f8c19dce0 0x7f1f8c19e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:14.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.697+0000 7f1f90c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8c19e760 con 0x7f1f8c19dce0 2024-03-23T17:58:14.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.697+0000 7f1f90c00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8c19e8d0 con 0x7f1f8c100f10 2024-03-23T17:58:14.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.697+0000 7f1f90c00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8c19ea90 con 0x7f1f8c106030 2024-03-23T17:58:14.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.697+0000 7f1f8a800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f8c19dce0 0x7f1f8c19e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:14.700 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.697+0000 7f1f8a800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f8c19dce0 0x7f1f8c19e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34298/0 (socket says 172.21.15.86:34298) 2024-03-23T17:58:14.700 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.697+0000 7f1f8a800700 1 -- 172.21.15.86:0/66983013 learned_addr learned my addr 172.21.15.86:0/66983013 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:58:14.700 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.697+0000 7f1f89e00700 1 --2- 172.21.15.86:0/66983013 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f8c100f10 0x7f1f8c199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:14.700 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.698+0000 7f1f89400700 1 --2- 172.21.15.86:0/66983013 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1f8c106030 0x7f1f8c199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:14.700 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.698+0000 7f1f8a800700 1 -- 172.21.15.86:0/66983013 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1f8c106030 msgr2=0x7f1f8c199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:58:14.700 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.698+0000 7f1f8a800700 1 --2- 172.21.15.86:0/66983013 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1f8c106030 0x7f1f8c199940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:14.700 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.698+0000 7f1f8a800700 1 -- 172.21.15.86:0/66983013 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f8c100f10 msgr2=0x7f1f8c199400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:58:14.700 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.698+0000 7f1f8a800700 1 --2- 172.21.15.86:0/66983013 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f8c100f10 0x7f1f8c199400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:14.700 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.698+0000 7f1f8a800700 1 -- 172.21.15.86:0/66983013 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f7c0095f0 con 0x7f1f8c19dce0 2024-03-23T17:58:14.700 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.698+0000 7f1f89e00700 1 --2- 172.21.15.86:0/66983013 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f8c100f10 0x7f1f8c199400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:58:14.700 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.698+0000 7f1f8a800700 1 --2- 172.21.15.86:0/66983013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f8c19dce0 0x7f1f8c19e160 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f1f8400ef30 tx=0x7f1f8400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:58:14.700 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.698+0000 7f1f82a00700 1 -- 172.21.15.86:0/66983013 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f84018070 con 0x7f1f8c19dce0 2024-03-23T17:58:14.700 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.698+0000 7f1f82a00700 1 -- 172.21.15.86:0/66983013 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f8400f040 con 0x7f1f8c19dce0 2024-03-23T17:58:14.700 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.698+0000 7f1f90c00700 1 -- 172.21.15.86:0/66983013 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f8c1a5f90 con 0x7f1f8c19dce0 2024-03-23T17:58:14.701 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.698+0000 7f1f82a00700 1 -- 172.21.15.86:0/66983013 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f8400cd00 con 0x7f1f8c19dce0 2024-03-23T17:58:14.701 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.698+0000 7f1f90c00700 1 -- 172.21.15.86:0/66983013 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f8c1a64c0 con 0x7f1f8c19dce0 2024-03-23T17:58:14.701 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.701+0000 7f1f82a00700 1 -- 172.21.15.86:0/66983013 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f1f84003a00 con 0x7f1f8c19dce0 2024-03-23T17:58:14.703 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.703+0000 7f1fa468db80 1 Processor -- start 2024-03-23T17:58:14.703 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.703+0000 7f1f82a00700 1 --2- 172.21.15.86:0/66983013 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f1f700a1310 0x7f1f700a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:14.703 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.703+0000 7f1f82a00700 1 -- 172.21.15.86:0/66983013 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f1f840cc750 con 0x7f1f8c19dce0 2024-03-23T17:58:14.703 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.703+0000 7f1fa468db80 1 -- start start 2024-03-23T17:58:14.703 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.703+0000 7f1f89400700 1 --2- 172.21.15.86:0/66983013 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f1f700a1310 0x7f1f700a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:14.704 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.703+0000 7f1fa468db80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9f270d100 0x55f9f270d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:14.704 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.703+0000 7f1fa468db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9f270da20 0x55f9f2718390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:14.704 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.703+0000 7f1fa468db80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f9f27188d0 0x55f9f271acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:14.704 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.703+0000 7f1fa468db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9f2666930 con 0x55f9f270da20 2024-03-23T17:58:14.704 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.703+0000 7f1fa468db80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9f26642f0 con 0x55f9f270d100 2024-03-23T17:58:14.704 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.703+0000 7f1fa468db80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9f265fbe0 con 0x55f9f27188d0 2024-03-23T17:58:14.706 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.704+0000 7f1f89e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9f270d100 0x55f9f270d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:14.706 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.704+0000 7f1f89e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9f270d100 0x55f9f270d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:42510/0 (socket says 172.21.15.86:42510) 2024-03-23T17:58:14.706 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.704+0000 7f1f89e00700 1 -- 172.21.15.86:0/922194928 learned_addr learned my addr 172.21.15.86:0/922194928 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:58:14.706 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.704+0000 7f1f89400700 1 --2- 172.21.15.86:0/922194928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9f270da20 0x55f9f2718390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:14.706 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.704+0000 7f1f8a800700 1 --2- 172.21.15.86:0/922194928 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f9f27188d0 0x55f9f271acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:14.706 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.704+0000 7f1f89400700 1 --2- 172.21.15.86:0/66983013 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f1f700a1310 0x7f1f700a37d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c19a950 tx=0x7f1f78009260 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:58:14.706 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.704+0000 7f1f89e00700 1 -- 172.21.15.86:0/922194928 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f9f27188d0 msgr2=0x55f9f271acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:58:14.706 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.704+0000 7f1f89e00700 1 --2- 172.21.15.86:0/922194928 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f9f27188d0 0x55f9f271acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:14.706 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.704+0000 7f1f89e00700 1 -- 172.21.15.86:0/922194928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9f270da20 msgr2=0x55f9f2718390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:58:14.707 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.704+0000 7f1f89e00700 1 --2- 172.21.15.86:0/922194928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9f270da20 0x55f9f2718390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:14.707 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.704+0000 7f1f89e00700 1 -- 172.21.15.86:0/922194928 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f9f24e1350 con 0x55f9f270d100 2024-03-23T17:58:14.707 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.704+0000 7f1f89e00700 1 --2- 172.21.15.86:0/922194928 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9f270d100 0x55f9f270d4e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c005ec0 tx=0x7f1f7c005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be9954b88eaff92d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:58:14.707 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.705+0000 7f1f77000700 1 -- 172.21.15.86:0/922194928 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c027070 con 0x55f9f270d100 2024-03-23T17:58:14.707 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.705+0000 7f1f77000700 1 -- 172.21.15.86:0/922194928 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f7c01e040 con 0x55f9f270d100 2024-03-23T17:58:14.707 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.705+0000 7f1f77000700 1 -- 172.21.15.86:0/922194928 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c01d050 con 0x55f9f270d100 2024-03-23T17:58:14.707 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.705+0000 7f1fa468db80 1 -- 172.21.15.86:0/922194928 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9f270d100 msgr2=0x55f9f270d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:58:14.707 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.705+0000 7f1fa468db80 1 --2- 172.21.15.86:0/922194928 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9f270d100 0x55f9f270d4e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c005ec0 tx=0x7f1f7c005ef0 comp rx=0 tx=0).stop 2024-03-23T17:58:14.707 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.705+0000 7f1fa468db80 1 -- 172.21.15.86:0/922194928 shutdown_connections 2024-03-23T17:58:14.707 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.705+0000 7f1fa468db80 1 --2- 172.21.15.86:0/922194928 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f9f27188d0 0x55f9f271acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:14.707 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.705+0000 7f1fa468db80 1 --2- 172.21.15.86:0/922194928 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9f270d100 0x55f9f270d4e0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:14.707 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.705+0000 7f1fa468db80 1 --2- 172.21.15.86:0/922194928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9f270da20 0x55f9f2718390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:14.707 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.705+0000 7f1fa468db80 1 -- 172.21.15.86:0/922194928 >> 172.21.15.86:0/922194928 conn(0x55f9f266d640 msgr2=0x55f9f2709520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:58:14.707 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.705+0000 7f1fa468db80 1 -- 172.21.15.86:0/922194928 shutdown_connections 2024-03-23T17:58:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.705+0000 7f1fa468db80 1 -- 172.21.15.86:0/922194928 wait complete. 2024-03-23T17:58:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.706+0000 7f1fa468db80 1 -- start start 2024-03-23T17:58:14.711 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.706+0000 7f1fa468db80 1 Processor -- start 2024-03-23T17:58:14.711 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.707+0000 7f1fa468db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9f270d100 0x55f9f2718270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:14.711 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.707+0000 7f1fa468db80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f9f270da20 0x55f9f2715660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:14.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.707+0000 7f1fa468db80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9f27188d0 0x55f9f2715c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:14.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.707+0000 7f1fa468db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9f25fbd20 con 0x55f9f270d100 2024-03-23T17:58:14.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.707+0000 7f1fa468db80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9f25fb9c0 con 0x55f9f27188d0 2024-03-23T17:58:14.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.707+0000 7f1fa468db80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9f24e3640 con 0x55f9f270da20 2024-03-23T17:58:14.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.707+0000 7f1fa468db80 10 client.0 _ll_register_callbacks cb 0x55f9f2605410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:58:14.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.707+0000 7f1f89e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9f270d100 0x55f9f2718270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:14.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.707+0000 7f1f89e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9f270d100 0x55f9f2718270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34310/0 (socket says 172.21.15.86:34310) 2024-03-23T17:58:14.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.707+0000 7f1f89e00700 1 -- 172.21.15.86:0/1879469702 learned_addr learned my addr 172.21.15.86:0/1879469702 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:58:14.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.707+0000 7f1f89400700 1 --2- 172.21.15.86:0/1879469702 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f9f270da20 0x55f9f2715660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:14.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.707+0000 7f1f8a800700 1 --2- 172.21.15.86:0/1879469702 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9f27188d0 0x55f9f2715c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:14.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.708+0000 7f1f89e00700 1 -- 172.21.15.86:0/1879469702 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f9f270da20 msgr2=0x55f9f2715660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:58:14.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.708+0000 7f1f89e00700 1 --2- 172.21.15.86:0/1879469702 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f9f270da20 0x55f9f2715660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:14.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.708+0000 7f1f89e00700 1 -- 172.21.15.86:0/1879469702 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9f27188d0 msgr2=0x55f9f2715c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:58:14.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.708+0000 7f1f89e00700 1 --2- 172.21.15.86:0/1879469702 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9f27188d0 0x55f9f2715c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:14.713 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.708+0000 7f1f89e00700 1 -- 172.21.15.86:0/1879469702 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f8c10e420 con 0x55f9f270d100 2024-03-23T17:58:14.713 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.708+0000 7f1f8a800700 1 --2- 172.21.15.86:0/1879469702 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9f27188d0 0x55f9f2715c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:58:14.713 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.708+0000 7f1f89e00700 1 --2- 172.21.15.86:0/1879469702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9f270d100 0x55f9f2718270 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c000c00 tx=0x7f1f7c00bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e92d61431fe727af server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:58:14.713 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.708+0000 7f1f6b400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-23T17:58:14.713 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.708+0000 7f1f6b400700 1 -- 172.21.15.86:0/1879469702 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c027030 con 0x55f9f270d100 2024-03-23T17:58:14.713 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.708+0000 7f1f6b400700 1 -- 172.21.15.86:0/1879469702 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f1f7c005160 con 0x55f9f270d100 2024-03-23T17:58:14.713 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.708+0000 7f1fa468db80 1 -- 172.21.15.86:0/1879469702 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f9f2620320 con 0x55f9f270d100 2024-03-23T17:58:14.713 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.708+0000 7f1f6b400700 1 -- 172.21.15.86:0/1879469702 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f7c01d020 con 0x55f9f270d100 2024-03-23T17:58:14.713 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.708+0000 7f1fa468db80 10 client.16587 fetch_fsmap learned FSMap version 40 2024-03-23T17:58:14.713 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.708+0000 7f1fa468db80 1 -- 172.21.15.86:0/1879469702 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55f9f248db00 con 0x55f9f270d100 2024-03-23T17:58:14.713 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.709+0000 7f1f6b400700 1 -- 172.21.15.86:0/1879469702 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f9f2620320 con 0x55f9f270d100 2024-03-23T17:58:14.713 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.709+0000 7f1f6b400700 1 -- 172.21.15.86:0/1879469702 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 40) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f1f7c02cb50 con 0x55f9f270d100 2024-03-23T17:58:14.713 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.709+0000 7f1fa468db80 10 client.16587 fetch_fsmap finished waiting for FSMap version 40 2024-03-23T17:58:14.714 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.709+0000 7f1fa468db80 10 client.16587 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:58:14.714 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.709+0000 7f1fa468db80 20 client.16587 populate_metadata read hostname 'smithi086' 2024-03-23T17:58:14.714 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.709+0000 7f1fa468db80 1 --2- 172.21.15.86:0/1879469702 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55f9f27280a0 0x55f9f272a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:14.714 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.709+0000 7f1fa468db80 4 client.16587 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:58:14.714 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.709+0000 7f1fa468db80 1 -- 172.21.15.86:0/1879469702 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f9f26586a0 con 0x55f9f27280a0 2024-03-23T17:58:14.714 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.709+0000 7f1f89400700 1 --2- 172.21.15.86:0/1879469702 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55f9f27280a0 0x55f9f272a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:14.714 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.710+0000 7f1f89400700 1 --2- 172.21.15.86:0/1879469702 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55f9f27280a0 0x55f9f272a4c0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61385c6b719723e5 server_cookie=2fbd9d5dd4574b71 in_seq=0 out_seq=0 2024-03-23T17:58:14.714 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.710+0000 7f1f6b400700 10 client.16587 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:58:14.714 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.711+0000 7f1f6b400700 1 -- 172.21.15.86:0/1879469702 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55f9f26586a0 con 0x55f9f27280a0 2024-03-23T17:58:14.714 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.711+0000 7f1f6b400700 10 client.16587 handle_command_reply: tid=0 2024-03-23T17:58:14.714 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.711+0000 7f1fa468db80 1 client.16587 shutdown 2024-03-23T17:58:14.714 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.711+0000 7f1fa468db80 1 -- 172.21.15.86:0/1879469702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9f270d100 msgr2=0x55f9f2718270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:58:14.714 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.711+0000 7f1fa468db80 1 --2- 172.21.15.86:0/1879469702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9f270d100 0x55f9f2718270 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c000c00 tx=0x7f1f7c00bf90 comp rx=0 tx=0).stop 2024-03-23T17:58:14.714 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.711+0000 7f1fa468db80 1 -- 172.21.15.86:0/1879469702 shutdown_connections 2024-03-23T17:58:14.715 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.711+0000 7f1fa468db80 1 --2- 172.21.15.86:0/1879469702 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55f9f27280a0 0x55f9f272a4c0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:14.715 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.711+0000 7f1fa468db80 1 --2- 172.21.15.86:0/1879469702 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9f27188d0 0x55f9f2715c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:14.715 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.711+0000 7f1fa468db80 1 --2- 172.21.15.86:0/1879469702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9f270d100 0x55f9f2718270 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:14.715 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.711+0000 7f1fa468db80 1 --2- 172.21.15.86:0/1879469702 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f9f270da20 0x55f9f2715660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:14.715 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.711+0000 7f1f6b400700 0 client.16587 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:58:14.715 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.711+0000 7f1fa468db80 1 -- 172.21.15.86:0/1879469702 >> 172.21.15.86:0/1879469702 conn(0x55f9f266d640 msgr2=0x55f9f270f0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:58:14.715 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.711+0000 7f1fa468db80 1 -- 172.21.15.86:0/1879469702 shutdown_connections 2024-03-23T17:58:14.715 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.711+0000 7f1fa468db80 1 -- 172.21.15.86:0/1879469702 wait complete. 2024-03-23T17:58:14.715 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.711+0000 7f1fa468db80 20 client.16587 trim_cache size 0 max 16384 2024-03-23T17:58:14.727 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.726+0000 7f1fa468db80 1 Processor -- start 2024-03-23T17:58:14.727 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.726+0000 7f1fa468db80 1 -- start start 2024-03-23T17:58:14.727 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.726+0000 7f1fa468db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9f270d100 0x55f9f272ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:14.727 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.726+0000 7f1fa468db80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f9f270da20 0x55f9f271f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:14.727 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.726+0000 7f1fa468db80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9f27188d0 0x55f9f2703500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:14.727 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.726+0000 7f1fa468db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9f270e3b0 con 0x55f9f270d100 2024-03-23T17:58:14.727 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.726+0000 7f1fa468db80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9f2710430 con 0x55f9f27188d0 2024-03-23T17:58:14.727 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.726+0000 7f1fa468db80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9f2713b90 con 0x55f9f270da20 2024-03-23T17:58:14.727 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.726+0000 7f1f89e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9f270d100 0x55f9f272ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:14.727 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.727+0000 7f1f89e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9f270d100 0x55f9f272ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34318/0 (socket says 172.21.15.86:34318) 2024-03-23T17:58:14.728 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.727+0000 7f1f89e00700 1 -- 172.21.15.86:0/2629169477 learned_addr learned my addr 172.21.15.86:0/2629169477 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:58:14.728 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.727+0000 7f1f89400700 1 --2- 172.21.15.86:0/2629169477 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f9f270da20 0x55f9f271f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:14.728 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.727+0000 7f1f8a800700 1 --2- 172.21.15.86:0/2629169477 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9f27188d0 0x55f9f2703500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:14.728 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.727+0000 7f1f89e00700 1 -- 172.21.15.86:0/2629169477 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f9f270da20 msgr2=0x55f9f271f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:58:14.728 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.727+0000 7f1f89e00700 1 --2- 172.21.15.86:0/2629169477 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f9f270da20 0x55f9f271f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:14.728 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.727+0000 7f1f89e00700 1 -- 172.21.15.86:0/2629169477 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9f27188d0 msgr2=0x55f9f2703500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:58:14.728 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.727+0000 7f1f89e00700 1 --2- 172.21.15.86:0/2629169477 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9f27188d0 0x55f9f2703500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:14.728 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.727+0000 7f1f89e00700 1 -- 172.21.15.86:0/2629169477 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f9f2625e40 con 0x55f9f270d100 2024-03-23T17:58:14.728 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.727+0000 7f1f89e00700 1 --2- 172.21.15.86:0/2629169477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9f270d100 0x55f9f272ee20 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c00b2e0 tx=0x7f1f7c004cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72984d2d8f865571 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:58:14.732 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.727+0000 7f1f77a00700 1 -- 172.21.15.86:0/2629169477 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c01d030 con 0x55f9f270d100 2024-03-23T17:58:14.732 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.727+0000 7f1f77a00700 1 -- 172.21.15.86:0/2629169477 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f7c028040 con 0x55f9f270d100 2024-03-23T17:58:14.732 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.727+0000 7f1f77a00700 1 -- 172.21.15.86:0/2629169477 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c027040 con 0x55f9f270d100 2024-03-23T17:58:14.732 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.728+0000 7f1fa468db80 1 -- 172.21.15.86:0/2629169477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9f270d100 msgr2=0x55f9f272ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:58:14.732 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.728+0000 7f1fa468db80 1 --2- 172.21.15.86:0/2629169477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9f270d100 0x55f9f272ee20 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c00b2e0 tx=0x7f1f7c004cd0 comp rx=0 tx=0).stop 2024-03-23T17:58:14.732 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.728+0000 7f1fa468db80 1 -- 172.21.15.86:0/2629169477 shutdown_connections 2024-03-23T17:58:14.732 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.728+0000 7f1fa468db80 1 --2- 172.21.15.86:0/2629169477 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9f27188d0 0x55f9f2703500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:14.732 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.728+0000 7f1fa468db80 1 --2- 172.21.15.86:0/2629169477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9f270d100 0x55f9f272ee20 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:14.732 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.728+0000 7f1fa468db80 1 --2- 172.21.15.86:0/2629169477 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f9f270da20 0x55f9f271f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:14.732 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.728+0000 7f1fa468db80 1 -- 172.21.15.86:0/2629169477 >> 172.21.15.86:0/2629169477 conn(0x55f9f266d640 msgr2=0x55f9f2713f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:58:14.732 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.728+0000 7f1fa468db80 1 -- 172.21.15.86:0/2629169477 shutdown_connections 2024-03-23T17:58:14.733 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.728+0000 7f1fa468db80 1 -- 172.21.15.86:0/2629169477 wait complete. 2024-03-23T17:58:14.733 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.728+0000 7f1fa468db80 1 -- start start 2024-03-23T17:58:14.733 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.729+0000 7f1fa468db80 1 Processor -- start 2024-03-23T17:58:14.733 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.729+0000 7f1fa468db80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f9f270d100 0x55f9f271d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:14.733 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.729+0000 7f1fa468db80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9f270da20 0x55f9f271da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:14.733 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.729+0000 7f1fa468db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9f27188d0 0x55f9f2778bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:14.733 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.729+0000 7f1fa468db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9f271e320 con 0x55f9f27188d0 2024-03-23T17:58:14.733 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.729+0000 7f1fa468db80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9f2779100 con 0x55f9f270da20 2024-03-23T17:58:14.733 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.729+0000 7f1fa468db80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9f2779240 con 0x55f9f270d100 2024-03-23T17:58:14.733 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.729+0000 7f1fa468db80 10 client.0 _ll_register_callbacks cb 0x55f9f2605410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:58:14.733 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.729+0000 7f1f8a800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9f27188d0 0x55f9f2778bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:14.733 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.729+0000 7f1f8a800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9f27188d0 0x55f9f2778bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34332/0 (socket says 172.21.15.86:34332) 2024-03-23T17:58:14.733 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.729+0000 7f1f8a800700 1 -- 172.21.15.86:0/516040474 learned_addr learned my addr 172.21.15.86:0/516040474 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:58:14.733 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.729+0000 7f1f89400700 1 --2- 172.21.15.86:0/516040474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9f270da20 0x55f9f271da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:14.734 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.729+0000 7f1f89e00700 1 --2- 172.21.15.86:0/516040474 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f9f270d100 0x55f9f271d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:14.734 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.730+0000 7f1f8a800700 1 -- 172.21.15.86:0/516040474 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f9f270d100 msgr2=0x55f9f271d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:58:14.734 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.730+0000 7f1f8a800700 1 --2- 172.21.15.86:0/516040474 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f9f270d100 0x55f9f271d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:14.734 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.730+0000 7f1f8a800700 1 -- 172.21.15.86:0/516040474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9f270da20 msgr2=0x55f9f271da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:58:14.734 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.730+0000 7f1f8a800700 1 --2- 172.21.15.86:0/516040474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9f270da20 0x55f9f271da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:14.734 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.730+0000 7f1f8a800700 1 -- 172.21.15.86:0/516040474 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f9f248db00 con 0x55f9f27188d0 2024-03-23T17:58:14.734 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.730+0000 7f1f89400700 1 --2- 172.21.15.86:0/516040474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9f270da20 0x55f9f271da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:58:14.734 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.730+0000 7f1f8a800700 1 --2- 172.21.15.86:0/516040474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9f27188d0 0x55f9f2778bc0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f1f84096320 tx=0x7f1f84099500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f916a0c894db0cb server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:58:14.734 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.730+0000 7f1f6b400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-23T17:58:14.734 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.730+0000 7f1f6b400700 1 -- 172.21.15.86:0/516040474 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f84099d10 con 0x55f9f27188d0 2024-03-23T17:58:14.734 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.730+0000 7f1fa468db80 1 -- 172.21.15.86:0/516040474 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1f7c01e040 con 0x55f9f27188d0 2024-03-23T17:58:14.734 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.730+0000 7f1f6b400700 1 -- 172.21.15.86:0/516040474 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f1f8401b320 con 0x55f9f27188d0 2024-03-23T17:58:14.734 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.730+0000 7f1f6b400700 1 -- 172.21.15.86:0/516040474 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f840b92c0 con 0x55f9f27188d0 2024-03-23T17:58:14.734 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.730+0000 7f1fa468db80 10 client.16593 fetch_fsmap learned FSMap version 40 2024-03-23T17:58:14.735 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.730+0000 7f1fa468db80 1 -- 172.21.15.86:0/516040474 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55f9f24e5c70 con 0x55f9f27188d0 2024-03-23T17:58:14.735 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.731+0000 7f1f6b400700 1 -- 172.21.15.86:0/516040474 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f7c01e040 con 0x55f9f27188d0 2024-03-23T17:58:14.735 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.731+0000 7f1f6b400700 1 -- 172.21.15.86:0/516040474 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 40) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f1f84098da0 con 0x55f9f27188d0 2024-03-23T17:58:14.735 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.731+0000 7f1fa468db80 10 client.16593 fetch_fsmap finished waiting for FSMap version 40 2024-03-23T17:58:14.735 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.731+0000 7f1fa468db80 10 client.16593 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:58:14.735 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.731+0000 7f1fa468db80 20 client.16593 populate_metadata read hostname 'smithi086' 2024-03-23T17:58:14.735 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.731+0000 7f1fa468db80 1 --2- 172.21.15.86:0/516040474 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55f9f277aaa0 0x55f9f277cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:14.735 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.731+0000 7f1fa468db80 4 client.16593 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T17:58:14.735 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.731+0000 7f1fa468db80 1 -- 172.21.15.86:0/516040474 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f9f24aec90 con 0x55f9f277aaa0 2024-03-23T17:58:14.735 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.731+0000 7f1f89e00700 1 --2- 172.21.15.86:0/516040474 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55f9f277aaa0 0x55f9f277cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:14.735 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.732+0000 7f1f6b400700 10 client.16593 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:58:14.735 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.732+0000 7f1f89e00700 1 --2- 172.21.15.86:0/516040474 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55f9f277aaa0 0x55f9f277cec0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=478ea6e6aea58220 server_cookie=e071f1162f51aab8 in_seq=0 out_seq=0 2024-03-23T17:58:14.764 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.764+0000 7f1f6b400700 1 -- 172.21.15.86:0/516040474 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f9f24aec90 con 0x55f9f277aaa0 2024-03-23T17:58:14.765 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.764+0000 7f1f6b400700 10 client.16593 handle_command_reply: tid=0 2024-03-23T17:58:14.765 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.764+0000 7f1fa468db80 1 client.16593 shutdown 2024-03-23T17:58:14.765 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.764+0000 7f1fa468db80 1 -- 172.21.15.86:0/516040474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9f27188d0 msgr2=0x55f9f2778bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:58:14.765 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.764+0000 7f1fa468db80 1 --2- 172.21.15.86:0/516040474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9f27188d0 0x55f9f2778bc0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f1f84096320 tx=0x7f1f84099500 comp rx=0 tx=0).stop 2024-03-23T17:58:14.765 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.764+0000 7f1fa468db80 1 -- 172.21.15.86:0/516040474 shutdown_connections 2024-03-23T17:58:14.765 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.764+0000 7f1fa468db80 1 --2- 172.21.15.86:0/516040474 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55f9f277aaa0 0x55f9f277cec0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:14.765 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.764+0000 7f1fa468db80 1 --2- 172.21.15.86:0/516040474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9f27188d0 0x55f9f2778bc0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:14.765 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.764+0000 7f1fa468db80 1 --2- 172.21.15.86:0/516040474 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f9f270d100 0x55f9f271d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:14.765 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.764+0000 7f1f6b400700 0 client.16593 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:58:14.765 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.764+0000 7f1fa468db80 1 --2- 172.21.15.86:0/516040474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9f270da20 0x55f9f271da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:14.765 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.764+0000 7f1fa468db80 1 -- 172.21.15.86:0/516040474 >> 172.21.15.86:0/516040474 conn(0x55f9f266d640 msgr2=0x55f9f270f040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:58:14.765 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.764+0000 7f1fa468db80 1 -- 172.21.15.86:0/516040474 shutdown_connections 2024-03-23T17:58:14.765 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.764+0000 7f1fa468db80 1 -- 172.21.15.86:0/516040474 wait complete. 2024-03-23T17:58:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.765+0000 7f1fa468db80 20 client.16593 trim_cache size 0 max 16384 2024-03-23T17:58:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-03-23T17:58:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-03-23T17:58:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-03-23T17:58:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-03-23T17:58:14.767 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.765+0000 7f1f90c00700 1 -- 172.21.15.86:0/66983013 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f1f700a1310 msgr2=0x7f1f700a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:58:14.767 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.765+0000 7f1f90c00700 1 --2- 172.21.15.86:0/66983013 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f1f700a1310 0x7f1f700a37d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c19a950 tx=0x7f1f78009260 comp rx=0 tx=0).stop 2024-03-23T17:58:14.767 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.765+0000 7f1f90c00700 1 -- 172.21.15.86:0/66983013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f8c19dce0 msgr2=0x7f1f8c19e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:58:14.767 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.765+0000 7f1f90c00700 1 --2- 172.21.15.86:0/66983013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f8c19dce0 0x7f1f8c19e160 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f1f8400ef30 tx=0x7f1f8400c3a0 comp rx=0 tx=0).stop 2024-03-23T17:58:14.767 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.765+0000 7f1f90c00700 1 -- 172.21.15.86:0/66983013 shutdown_connections 2024-03-23T17:58:14.767 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.765+0000 7f1f90c00700 1 --2- 172.21.15.86:0/66983013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f8c19dce0 0x7f1f8c19e160 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:14.767 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.765+0000 7f1f90c00700 1 --2- 172.21.15.86:0/66983013 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f8c100f10 0x7f1f8c199400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:14.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.765+0000 7f1f90c00700 1 --2- 172.21.15.86:0/66983013 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f1f700a1310 0x7f1f700a37d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:14.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.765+0000 7f1f90c00700 1 --2- 172.21.15.86:0/66983013 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1f8c106030 0x7f1f8c199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:14.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.765+0000 7f1f90c00700 1 -- 172.21.15.86:0/66983013 >> 172.21.15.86:0/66983013 conn(0x7f1f8c0fc7b0 msgr2=0x7f1f8c0fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:58:14.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.765+0000 7f1f90c00700 1 -- 172.21.15.86:0/66983013 shutdown_connections 2024-03-23T17:58:14.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:14.765+0000 7f1f90c00700 1 -- 172.21.15.86:0/66983013 wait complete. 2024-03-23T17:58:14.909 INFO:tasks.workunit.client.0.smithi086.stdout: 1 413183 6.39 MB/sec execute 279 sec latency 1526.439 ms 2024-03-23T17:58:15.909 INFO:tasks.workunit.client.0.smithi086.stdout: 1 413183 6.37 MB/sec execute 280 sec latency 2526.605 ms 2024-03-23T17:58:16.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:58:15 smithi086 ceph-mon[38221]: pgmap v234: 97 pgs: 97 active+clean; 148 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.1 MiB/s wr, 158 op/s 2024-03-23T17:58:16.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:58:15 smithi086 ceph-mon[38221]: osdmap e87: 12 total, 12 up, 12 in 2024-03-23T17:58:16.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:58:15 smithi092 ceph-mon[79089]: pgmap v234: 97 pgs: 97 active+clean; 148 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.1 MiB/s wr, 158 op/s 2024-03-23T17:58:16.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:58:15 smithi092 ceph-mon[79089]: osdmap e87: 12 total, 12 up, 12 in 2024-03-23T17:58:16.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:58:15 smithi152 ceph-mon[47949]: pgmap v234: 97 pgs: 97 active+clean; 148 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.1 MiB/s wr, 158 op/s 2024-03-23T17:58:16.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:58:15 smithi152 ceph-mon[47949]: osdmap e87: 12 total, 12 up, 12 in 2024-03-23T17:58:16.909 INFO:tasks.workunit.client.0.smithi086.stdout: 1 415177 6.39 MB/sec execute 281 sec latency 2534.111 ms 2024-03-23T17:58:17.909 INFO:tasks.workunit.client.0.smithi086.stdout: 1 416272 6.38 MB/sec execute 282 sec latency 13.571 ms 2024-03-23T17:58:18.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:58:17 smithi086 ceph-mon[38221]: pgmap v236: 97 pgs: 97 active+clean; 148 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.1 MiB/s wr, 158 op/s 2024-03-23T17:58:18.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:58:17 smithi092 ceph-mon[79089]: pgmap v236: 97 pgs: 97 active+clean; 148 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.1 MiB/s wr, 158 op/s 2024-03-23T17:58:18.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:58:17 smithi152 ceph-mon[47949]: pgmap v236: 97 pgs: 97 active+clean; 148 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.1 MiB/s wr, 158 op/s 2024-03-23T17:58:18.909 INFO:tasks.workunit.client.0.smithi086.stdout: 1 416866 6.37 MB/sec execute 283 sec latency 682.704 ms 2024-03-23T17:58:19.909 INFO:tasks.workunit.client.0.smithi086.stdout: 1 418867 6.39 MB/sec execute 284 sec latency 15.743 ms 2024-03-23T17:58:20.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:58:19 smithi086 ceph-mon[38221]: pgmap v237: 97 pgs: 97 active+clean; 148 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.1 MiB/s wr, 158 op/s 2024-03-23T17:58:20.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:58:19 smithi092 ceph-mon[79089]: pgmap v237: 97 pgs: 97 active+clean; 148 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.1 MiB/s wr, 158 op/s 2024-03-23T17:58:20.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:58:19 smithi152 ceph-mon[47949]: pgmap v237: 97 pgs: 97 active+clean; 148 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.1 MiB/s wr, 158 op/s 2024-03-23T17:58:20.909 INFO:tasks.workunit.client.0.smithi086.stdout: 1 420177 6.39 MB/sec execute 285 sec latency 14.859 ms 2024-03-23T17:58:21.909 INFO:tasks.workunit.client.0.smithi086.stdout: 1 420275 6.37 MB/sec execute 286 sec latency 940.907 ms 2024-03-23T17:58:22.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:58:21 smithi086 ceph-mon[38221]: pgmap v238: 97 pgs: 97 active+clean; 147 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 179 op/s 2024-03-23T17:58:22.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:58:21 smithi092 ceph-mon[79089]: pgmap v238: 97 pgs: 97 active+clean; 147 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 179 op/s 2024-03-23T17:58:22.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:58:21 smithi152 ceph-mon[47949]: pgmap v238: 97 pgs: 97 active+clean; 147 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 179 op/s 2024-03-23T17:58:22.910 INFO:tasks.workunit.client.0.smithi086.stdout: 1 420275 6.35 MB/sec execute 287 sec latency 1940.988 ms 2024-03-23T17:58:23.910 INFO:tasks.workunit.client.0.smithi086.stdout: 1 420275 6.32 MB/sec execute 288 sec latency 2941.131 ms 2024-03-23T17:58:24.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:58:23 smithi086 ceph-mon[38221]: pgmap v239: 97 pgs: 97 active+clean; 147 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 179 op/s 2024-03-23T17:58:24.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:58:23 smithi092 ceph-mon[79089]: pgmap v239: 97 pgs: 97 active+clean; 147 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 179 op/s 2024-03-23T17:58:24.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:58:23 smithi152 ceph-mon[47949]: pgmap v239: 97 pgs: 97 active+clean; 147 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 179 op/s 2024-03-23T17:58:24.910 INFO:tasks.workunit.client.0.smithi086.stdout: 1 420275 6.30 MB/sec execute 289 sec latency 3941.298 ms 2024-03-23T17:58:25.910 INFO:tasks.workunit.client.0.smithi086.stdout: 1 420985 6.29 MB/sec execute 290 sec latency 4655.423 ms 2024-03-23T17:58:26.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:58:25 smithi086 ceph-mon[38221]: pgmap v240: 97 pgs: 97 active+clean; 152 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 163 op/s 2024-03-23T17:58:26.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:58:25 smithi092 ceph-mon[79089]: pgmap v240: 97 pgs: 97 active+clean; 152 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 163 op/s 2024-03-23T17:58:26.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:58:25 smithi152 ceph-mon[47949]: pgmap v240: 97 pgs: 97 active+clean; 152 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 163 op/s 2024-03-23T17:58:26.910 INFO:tasks.workunit.client.0.smithi086.stdout: 1 422519 6.31 MB/sec execute 291 sec latency 27.571 ms 2024-03-23T17:58:27.910 INFO:tasks.workunit.client.0.smithi086.stdout: 1 423824 6.30 MB/sec execute 292 sec latency 55.074 ms 2024-03-23T17:58:27.918 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.917+0000 7fe652600700 1 -- 172.21.15.86:0/2481543997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe64c100f10 msgr2=0x7fe64c1012f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:58:27.918 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.917+0000 7fe652600700 1 --2- 172.21.15.86:0/2481543997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe64c100f10 0x7fe64c1012f0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fe63c009990 tx=0x7fe63c009ca0 comp rx=0 tx=0).stop 2024-03-23T17:58:27.918 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.917+0000 7fe652600700 1 -- 172.21.15.86:0/2481543997 shutdown_connections 2024-03-23T17:58:27.918 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.917+0000 7fe652600700 1 --2- 172.21.15.86:0/2481543997 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe64c106030 0x7fe64c10dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:27.918 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.917+0000 7fe652600700 1 --2- 172.21.15.86:0/2481543997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe64c100f10 0x7fe64c1012f0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:27.918 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.917+0000 7fe652600700 1 --2- 172.21.15.86:0/2481543997 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe64c1018c0 0x7fe64c105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:27.919 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.917+0000 7fe652600700 1 -- 172.21.15.86:0/2481543997 >> 172.21.15.86:0/2481543997 conn(0x7fe64c0fc7b0 msgr2=0x7fe64c0febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:58:27.919 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.917+0000 7fe652600700 1 -- 172.21.15.86:0/2481543997 shutdown_connections 2024-03-23T17:58:27.919 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.917+0000 7fe652600700 1 -- 172.21.15.86:0/2481543997 wait complete. 2024-03-23T17:58:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.918+0000 7fe652600700 1 Processor -- start 2024-03-23T17:58:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.918+0000 7fe652600700 1 -- start start 2024-03-23T17:58:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.918+0000 7fe652600700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe64c100f10 0x7fe64c199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.918+0000 7fe652600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe64c106030 0x7fe64c199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.918+0000 7fe652600700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe64c19dd10 0x7fe64c19e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.918+0000 7fe652600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe64c19e790 con 0x7fe64c106030 2024-03-23T17:58:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.918+0000 7fe652600700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe64c19e900 con 0x7fe64c100f10 2024-03-23T17:58:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.918+0000 7fe652600700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe64c19eac0 con 0x7fe64c19dd10 2024-03-23T17:58:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.919+0000 7fe64aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe64c106030 0x7fe64c199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.919+0000 7fe64aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe64c106030 0x7fe64c199970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42294/0 (socket says 172.21.15.86:42294) 2024-03-23T17:58:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.919+0000 7fe64aa00700 1 -- 172.21.15.86:0/2373180561 learned_addr learned my addr 172.21.15.86:0/2373180561 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:58:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.919+0000 7fe64b400700 1 --2- 172.21.15.86:0/2373180561 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe64c100f10 0x7fe64c199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.919+0000 7fe64be00700 1 --2- 172.21.15.86:0/2373180561 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe64c19dd10 0x7fe64c19e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.919+0000 7fe64aa00700 1 -- 172.21.15.86:0/2373180561 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe64c19dd10 msgr2=0x7fe64c19e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:58:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.919+0000 7fe64aa00700 1 --2- 172.21.15.86:0/2373180561 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe64c19dd10 0x7fe64c19e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.919+0000 7fe64aa00700 1 -- 172.21.15.86:0/2373180561 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe64c100f10 msgr2=0x7fe64c199430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:58:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.919+0000 7fe64aa00700 1 --2- 172.21.15.86:0/2373180561 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe64c100f10 0x7fe64c199430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.919+0000 7fe64aa00700 1 -- 172.21.15.86:0/2373180561 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe63c0095f0 con 0x7fe64c106030 2024-03-23T17:58:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.919+0000 7fe64aa00700 1 --2- 172.21.15.86:0/2373180561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe64c106030 0x7fe64c199970 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fe63800d980 tx=0x7fe63800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:58:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.920+0000 7fe643e00700 1 -- 172.21.15.86:0/2373180561 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe638009930 con 0x7fe64c106030 2024-03-23T17:58:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.920+0000 7fe643e00700 1 -- 172.21.15.86:0/2373180561 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe638010460 con 0x7fe64c106030 2024-03-23T17:58:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.920+0000 7fe652600700 1 -- 172.21.15.86:0/2373180561 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe64c1a5fc0 con 0x7fe64c106030 2024-03-23T17:58:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.920+0000 7fe643e00700 1 -- 172.21.15.86:0/2373180561 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe63800f620 con 0x7fe64c106030 2024-03-23T17:58:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.920+0000 7fe652600700 1 -- 172.21.15.86:0/2373180561 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe64c1a64a0 con 0x7fe64c106030 2024-03-23T17:58:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.923+0000 7fe643e00700 1 -- 172.21.15.86:0/2373180561 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fe6380108d0 con 0x7fe64c106030 2024-03-23T17:58:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.924+0000 7fe666072b80 1 Processor -- start 2024-03-23T17:58:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.924+0000 7fe666072b80 1 -- start start 2024-03-23T17:58:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.924+0000 7fe643e00700 1 --2- 172.21.15.86:0/2373180561 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fe6300a1290 0x7fe6300a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.924+0000 7fe666072b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc2ac03100 0x55fc2ac034e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.924+0000 7fe666072b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fc2ac03a20 0x55fc2ac0e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.924+0000 7fe643e00700 1 -- 172.21.15.86:0/2373180561 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fe6380cc7a0 con 0x7fe64c106030 2024-03-23T17:58:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.924+0000 7fe666072b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc2ac0e8d0 0x55fc2ac10cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.924+0000 7fe666072b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc2ab5c930 con 0x55fc2ac0e8d0 2024-03-23T17:58:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.924+0000 7fe666072b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc2ab5a2f0 con 0x55fc2ac03100 2024-03-23T17:58:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.924+0000 7fe666072b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc2ab55be0 con 0x55fc2ac03a20 2024-03-23T17:58:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.925+0000 7fe64be00700 1 --2- 172.21.15.86:0/2373180561 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fe6300a1290 0x7fe6300a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:27.929 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.925+0000 7fe64b400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc2ac03100 0x55fc2ac034e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:27.929 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.925+0000 7fe64b400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc2ac03100 0x55fc2ac034e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:46000/0 (socket says 172.21.15.86:46000) 2024-03-23T17:58:27.929 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.925+0000 7fe64b400700 1 -- 172.21.15.86:0/3126905227 learned_addr learned my addr 172.21.15.86:0/3126905227 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:58:27.929 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.925+0000 7fe64be00700 1 --2- 172.21.15.86:0/3126905227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc2ac0e8d0 0x55fc2ac10cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:27.929 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.925+0000 7fe64aa00700 1 --2- 172.21.15.86:0/3126905227 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fc2ac03a20 0x55fc2ac0e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:27.929 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.925+0000 7fe64b400700 1 -- 172.21.15.86:0/3126905227 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fc2ac03a20 msgr2=0x55fc2ac0e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:58:27.929 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.925+0000 7fe64b400700 1 --2- 172.21.15.86:0/3126905227 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fc2ac03a20 0x55fc2ac0e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:27.929 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.925+0000 7fe64b400700 1 -- 172.21.15.86:0/3126905227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc2ac0e8d0 msgr2=0x55fc2ac10cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:58:27.929 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.925+0000 7fe64b400700 1 --2- 172.21.15.86:0/3126905227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc2ac0e8d0 0x55fc2ac10cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:27.929 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.925+0000 7fe64b400700 1 -- 172.21.15.86:0/3126905227 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc2a9d7350 con 0x55fc2ac03100 2024-03-23T17:58:27.929 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.925+0000 7fe64be00700 1 --2- 172.21.15.86:0/3126905227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc2ac0e8d0 0x55fc2ac10cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:58:27.929 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.925+0000 7fe64b400700 1 --2- 172.21.15.86:0/3126905227 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc2ac03100 0x55fc2ac034e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe63c009f60 tx=0x7fe63c009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c5343e2abcfdbdc server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:58:27.929 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.925+0000 7fe64be00700 1 --2- 172.21.15.86:0/2373180561 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fe6300a1290 0x7fe6300a3750 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x55fc2ac04b90 tx=0x7fe6440090b0 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:58:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.926+0000 7fe640c00700 1 -- 172.21.15.86:0/3126905227 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe63c01d070 con 0x55fc2ac03100 2024-03-23T17:58:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.926+0000 7fe640c00700 1 -- 172.21.15.86:0/3126905227 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe63c01e040 con 0x55fc2ac03100 2024-03-23T17:58:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.926+0000 7fe640c00700 1 -- 172.21.15.86:0/3126905227 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe63c004700 con 0x55fc2ac03100 2024-03-23T17:58:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.926+0000 7fe666072b80 1 -- 172.21.15.86:0/3126905227 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc2ac03100 msgr2=0x55fc2ac034e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:58:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.926+0000 7fe666072b80 1 --2- 172.21.15.86:0/3126905227 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc2ac03100 0x55fc2ac034e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe63c009f60 tx=0x7fe63c009f90 comp rx=0 tx=0).stop 2024-03-23T17:58:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.926+0000 7fe666072b80 1 -- 172.21.15.86:0/3126905227 shutdown_connections 2024-03-23T17:58:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.926+0000 7fe666072b80 1 --2- 172.21.15.86:0/3126905227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc2ac0e8d0 0x55fc2ac10cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.926+0000 7fe666072b80 1 --2- 172.21.15.86:0/3126905227 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc2ac03100 0x55fc2ac034e0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.926+0000 7fe666072b80 1 --2- 172.21.15.86:0/3126905227 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fc2ac03a20 0x55fc2ac0e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.926+0000 7fe666072b80 1 -- 172.21.15.86:0/3126905227 >> 172.21.15.86:0/3126905227 conn(0x55fc2ab63640 msgr2=0x55fc2abff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:58:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.926+0000 7fe666072b80 1 -- 172.21.15.86:0/3126905227 shutdown_connections 2024-03-23T17:58:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.926+0000 7fe666072b80 1 -- 172.21.15.86:0/3126905227 wait complete. 2024-03-23T17:58:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.927+0000 7fe666072b80 1 -- start start 2024-03-23T17:58:27.934 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.927+0000 7fe666072b80 1 Processor -- start 2024-03-23T17:58:27.934 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.928+0000 7fe666072b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc2ac03100 0x55fc2ac0e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:27.935 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.928+0000 7fe666072b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fc2ac03a20 0x55fc2ac0b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:27.935 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.928+0000 7fe666072b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc2ac0e8d0 0x55fc2ac0ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:27.935 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.928+0000 7fe666072b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc2aaf1d20 con 0x55fc2ac03100 2024-03-23T17:58:27.935 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.928+0000 7fe666072b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc2aaf19c0 con 0x55fc2ac0e8d0 2024-03-23T17:58:27.935 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.928+0000 7fe666072b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc2a9d9640 con 0x55fc2ac03a20 2024-03-23T17:58:27.935 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.928+0000 7fe666072b80 10 client.0 _ll_register_callbacks cb 0x55fc2aafb410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:58:27.935 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.928+0000 7fe64b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc2ac03100 0x55fc2ac0e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:27.935 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.928+0000 7fe64b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc2ac03100 0x55fc2ac0e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42316/0 (socket says 172.21.15.86:42316) 2024-03-23T17:58:27.935 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.928+0000 7fe64b400700 1 -- 172.21.15.86:0/2103933908 learned_addr learned my addr 172.21.15.86:0/2103933908 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:58:27.935 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.928+0000 7fe64aa00700 1 --2- 172.21.15.86:0/2103933908 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fc2ac03a20 0x55fc2ac0b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:27.935 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.929+0000 7fe64be00700 1 --2- 172.21.15.86:0/2103933908 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc2ac0e8d0 0x55fc2ac0ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:27.935 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.929+0000 7fe64b400700 1 -- 172.21.15.86:0/2103933908 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fc2ac03a20 msgr2=0x55fc2ac0b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:58:27.936 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.929+0000 7fe64b400700 1 --2- 172.21.15.86:0/2103933908 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fc2ac03a20 0x55fc2ac0b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:27.936 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.929+0000 7fe64b400700 1 -- 172.21.15.86:0/2103933908 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc2ac0e8d0 msgr2=0x55fc2ac0ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:58:27.936 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.929+0000 7fe64b400700 1 --2- 172.21.15.86:0/2103933908 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc2ac0e8d0 0x55fc2ac0ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:27.936 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.929+0000 7fe64b400700 1 -- 172.21.15.86:0/2103933908 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe64c10e420 con 0x55fc2ac03100 2024-03-23T17:58:27.936 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.929+0000 7fe64aa00700 1 --2- 172.21.15.86:0/2103933908 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fc2ac03a20 0x55fc2ac0b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:58:27.936 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.929+0000 7fe64b400700 1 --2- 172.21.15.86:0/2103933908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc2ac03100 0x55fc2ac0e1b0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fe63c009ce0 tx=0x7fe63c005470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeeeecb4d433cbf3 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:58:27.936 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.929+0000 7fe635600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-23T17:58:27.936 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.929+0000 7fe635600700 1 -- 172.21.15.86:0/2103933908 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe63c01d030 con 0x55fc2ac03100 2024-03-23T17:58:27.936 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.929+0000 7fe635600700 1 -- 172.21.15.86:0/2103933908 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fe63c004910 con 0x55fc2ac03100 2024-03-23T17:58:27.936 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.929+0000 7fe666072b80 1 -- 172.21.15.86:0/2103933908 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fc2ab16320 con 0x55fc2ac03100 2024-03-23T17:58:27.936 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.930+0000 7fe635600700 1 -- 172.21.15.86:0/2103933908 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe63c00bbe0 con 0x55fc2ac03100 2024-03-23T17:58:27.936 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.930+0000 7fe666072b80 10 client.16605 fetch_fsmap learned FSMap version 40 2024-03-23T17:58:27.936 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.930+0000 7fe666072b80 1 -- 172.21.15.86:0/2103933908 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55fc2a983b00 con 0x55fc2ac03100 2024-03-23T17:58:27.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.930+0000 7fe635600700 1 -- 172.21.15.86:0/2103933908 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fc2ab16320 con 0x55fc2ac03100 2024-03-23T17:58:27.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.930+0000 7fe635600700 1 -- 172.21.15.86:0/2103933908 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 40) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fe63c00f790 con 0x55fc2ac03100 2024-03-23T17:58:27.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.930+0000 7fe666072b80 10 client.16605 fetch_fsmap finished waiting for FSMap version 40 2024-03-23T17:58:27.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.930+0000 7fe666072b80 10 client.16605 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:58:27.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.930+0000 7fe666072b80 20 client.16605 populate_metadata read hostname 'smithi086' 2024-03-23T17:58:27.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.930+0000 7fe666072b80 1 --2- 172.21.15.86:0/2103933908 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55fc2ac1e0f0 0x55fc2ac20510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:27.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.930+0000 7fe666072b80 4 client.16605 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:58:27.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.930+0000 7fe666072b80 1 -- 172.21.15.86:0/2103933908 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fc2ab4e6a0 con 0x55fc2ac1e0f0 2024-03-23T17:58:27.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.930+0000 7fe64aa00700 1 --2- 172.21.15.86:0/2103933908 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55fc2ac1e0f0 0x55fc2ac20510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:27.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.931+0000 7fe635600700 10 client.16605 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:58:27.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.931+0000 7fe64aa00700 1 --2- 172.21.15.86:0/2103933908 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55fc2ac1e0f0 0x55fc2ac20510 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8973f588922c2b7e server_cookie=190e4df310063c98 in_seq=0 out_seq=0 2024-03-23T17:58:27.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.931+0000 7fe635600700 1 -- 172.21.15.86:0/2103933908 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55fc2ab4e6a0 con 0x55fc2ac1e0f0 2024-03-23T17:58:27.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.931+0000 7fe635600700 10 client.16605 handle_command_reply: tid=0 2024-03-23T17:58:27.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.931+0000 7fe666072b80 1 client.16605 shutdown 2024-03-23T17:58:27.938 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.931+0000 7fe666072b80 1 -- 172.21.15.86:0/2103933908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc2ac03100 msgr2=0x55fc2ac0e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:58:27.938 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.931+0000 7fe666072b80 1 --2- 172.21.15.86:0/2103933908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc2ac03100 0x55fc2ac0e1b0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fe63c009ce0 tx=0x7fe63c005470 comp rx=0 tx=0).stop 2024-03-23T17:58:27.938 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.931+0000 7fe666072b80 1 -- 172.21.15.86:0/2103933908 shutdown_connections 2024-03-23T17:58:27.938 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.931+0000 7fe666072b80 1 --2- 172.21.15.86:0/2103933908 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55fc2ac1e0f0 0x55fc2ac20510 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:27.938 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.931+0000 7fe666072b80 1 --2- 172.21.15.86:0/2103933908 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc2ac0e8d0 0x55fc2ac0ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:27.938 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.931+0000 7fe666072b80 1 --2- 172.21.15.86:0/2103933908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc2ac03100 0x55fc2ac0e1b0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:27.938 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.931+0000 7fe666072b80 1 --2- 172.21.15.86:0/2103933908 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fc2ac03a20 0x55fc2ac0b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:27.938 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.931+0000 7fe666072b80 1 -- 172.21.15.86:0/2103933908 >> 172.21.15.86:0/2103933908 conn(0x55fc2ab63640 msgr2=0x55fc2ac050a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:58:27.938 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.932+0000 7fe635600700 0 client.16605 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:58:27.938 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.932+0000 7fe666072b80 1 -- 172.21.15.86:0/2103933908 shutdown_connections 2024-03-23T17:58:27.938 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.932+0000 7fe666072b80 1 -- 172.21.15.86:0/2103933908 wait complete. 2024-03-23T17:58:27.938 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.932+0000 7fe666072b80 20 client.16605 trim_cache size 0 max 16384 2024-03-23T17:58:27.948 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.947+0000 7fe666072b80 1 Processor -- start 2024-03-23T17:58:27.949 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.947+0000 7fe666072b80 1 -- start start 2024-03-23T17:58:27.949 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.947+0000 7fe666072b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc2ac03100 0x55fc2ac155b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:27.949 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.947+0000 7fe666072b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc2ac03a20 0x55fc2abf9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:27.949 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.947+0000 7fe666072b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fc2ac0e8d0 0x55fc2abf8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:27.949 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.947+0000 7fe666072b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc2ac041a0 con 0x55fc2ac03a20 2024-03-23T17:58:27.949 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.947+0000 7fe666072b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc2ac0bd80 con 0x55fc2ac03100 2024-03-23T17:58:27.949 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.947+0000 7fe666072b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc2ac072e0 con 0x55fc2ac0e8d0 2024-03-23T17:58:27.949 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.947+0000 7fe64aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc2ac03a20 0x55fc2abf9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:27.949 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.947+0000 7fe64aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc2ac03a20 0x55fc2abf9500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42318/0 (socket says 172.21.15.86:42318) 2024-03-23T17:58:27.949 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.947+0000 7fe64aa00700 1 -- 172.21.15.86:0/208983590 learned_addr learned my addr 172.21.15.86:0/208983590 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:58:27.949 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.947+0000 7fe64be00700 1 --2- 172.21.15.86:0/208983590 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fc2ac0e8d0 0x55fc2abf8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:27.949 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.947+0000 7fe64b400700 1 --2- 172.21.15.86:0/208983590 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc2ac03100 0x55fc2ac155b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:27.949 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.948+0000 7fe64aa00700 1 -- 172.21.15.86:0/208983590 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fc2ac0e8d0 msgr2=0x55fc2abf8af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:58:27.949 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.948+0000 7fe64aa00700 1 --2- 172.21.15.86:0/208983590 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fc2ac0e8d0 0x55fc2abf8af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:27.950 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.948+0000 7fe64aa00700 1 -- 172.21.15.86:0/208983590 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc2ac03100 msgr2=0x55fc2ac155b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:58:27.950 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.948+0000 7fe64aa00700 1 --2- 172.21.15.86:0/208983590 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc2ac03100 0x55fc2ac155b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:27.950 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.948+0000 7fe64aa00700 1 -- 172.21.15.86:0/208983590 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc2ab1be40 con 0x55fc2ac03a20 2024-03-23T17:58:27.953 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.948+0000 7fe64aa00700 1 --2- 172.21.15.86:0/208983590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc2ac03a20 0x55fc2abf9500 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fe638009520 tx=0x7fe638005c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d111c77008355d51 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:58:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.948+0000 7fe641600700 1 -- 172.21.15.86:0/208983590 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6380b2210 con 0x55fc2ac03a20 2024-03-23T17:58:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.948+0000 7fe641600700 1 -- 172.21.15.86:0/208983590 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe6380a1cf0 con 0x55fc2ac03a20 2024-03-23T17:58:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.948+0000 7fe641600700 1 -- 172.21.15.86:0/208983590 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6380b9310 con 0x55fc2ac03a20 2024-03-23T17:58:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.949+0000 7fe666072b80 1 -- 172.21.15.86:0/208983590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc2ac03a20 msgr2=0x55fc2abf9500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:58:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.949+0000 7fe666072b80 1 --2- 172.21.15.86:0/208983590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc2ac03a20 0x55fc2abf9500 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fe638009520 tx=0x7fe638005c80 comp rx=0 tx=0).stop 2024-03-23T17:58:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.949+0000 7fe666072b80 1 -- 172.21.15.86:0/208983590 shutdown_connections 2024-03-23T17:58:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.949+0000 7fe666072b80 1 --2- 172.21.15.86:0/208983590 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fc2ac0e8d0 0x55fc2abf8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.949+0000 7fe666072b80 1 --2- 172.21.15.86:0/208983590 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc2ac03100 0x55fc2ac155b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.949+0000 7fe666072b80 1 --2- 172.21.15.86:0/208983590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc2ac03a20 0x55fc2abf9500 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.949+0000 7fe666072b80 1 -- 172.21.15.86:0/208983590 >> 172.21.15.86:0/208983590 conn(0x55fc2ab63640 msgr2=0x55fc2abf83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:58:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.949+0000 7fe666072b80 1 -- 172.21.15.86:0/208983590 shutdown_connections 2024-03-23T17:58:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.949+0000 7fe666072b80 1 -- 172.21.15.86:0/208983590 wait complete. 2024-03-23T17:58:27.955 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.949+0000 7fe666072b80 1 -- start start 2024-03-23T17:58:27.955 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.949+0000 7fe666072b80 1 Processor -- start 2024-03-23T17:58:27.955 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.950+0000 7fe666072b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc2ac03100 0x55fc2ac63d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:27.955 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.950+0000 7fe666072b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fc2ac03a20 0x55fc2ac64290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:27.955 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.950+0000 7fe666072b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc2ac0e8d0 0x55fc2ac68c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:27.955 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.950+0000 7fe666072b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc2ac69160 con 0x55fc2ac03100 2024-03-23T17:58:27.955 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.950+0000 7fe666072b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc2ac692d0 con 0x55fc2ac0e8d0 2024-03-23T17:58:27.955 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.950+0000 7fe666072b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc2ac69490 con 0x55fc2ac03a20 2024-03-23T17:58:27.955 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.950+0000 7fe666072b80 10 client.0 _ll_register_callbacks cb 0x55fc2aafb410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:58:27.955 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.950+0000 7fe64be00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc2ac0e8d0 0x55fc2ac68c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:27.955 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.950+0000 7fe64b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc2ac03100 0x55fc2ac63d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:27.955 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.950+0000 7fe64aa00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fc2ac03a20 0x55fc2ac64290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:27.955 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.950+0000 7fe64b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc2ac03100 0x55fc2ac63d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42334/0 (socket says 172.21.15.86:42334) 2024-03-23T17:58:27.956 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.950+0000 7fe64be00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc2ac0e8d0 0x55fc2ac68c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:46012/0 (socket says 172.21.15.86:46012) 2024-03-23T17:58:27.956 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.950+0000 7fe64b400700 1 -- 172.21.15.86:0/642895418 learned_addr learned my addr 172.21.15.86:0/642895418 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:58:27.956 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.951+0000 7fe64aa00700 1 -- 172.21.15.86:0/642895418 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc2ac0e8d0 msgr2=0x55fc2ac68c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:58:27.956 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.951+0000 7fe64aa00700 1 --2- 172.21.15.86:0/642895418 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc2ac0e8d0 0x55fc2ac68c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:27.956 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.951+0000 7fe64aa00700 1 -- 172.21.15.86:0/642895418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc2ac03100 msgr2=0x55fc2ac63d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:58:27.956 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.951+0000 7fe64aa00700 1 --2- 172.21.15.86:0/642895418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc2ac03100 0x55fc2ac63d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:27.956 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.951+0000 7fe64aa00700 1 -- 172.21.15.86:0/642895418 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe64c1a64a0 con 0x55fc2ac03a20 2024-03-23T17:58:27.956 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.951+0000 7fe64b400700 1 --2- 172.21.15.86:0/642895418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc2ac03100 0x55fc2ac63d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:58:27.956 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.951+0000 7fe64be00700 1 --2- 172.21.15.86:0/642895418 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc2ac0e8d0 0x55fc2ac68c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:58:27.956 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.951+0000 7fe64aa00700 1 --2- 172.21.15.86:0/642895418 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fc2ac03a20 0x55fc2ac64290 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x55fc2ab16460 tx=0x7fe638099970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60428b35607a074d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:58:27.956 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.951+0000 7fe635600700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-03-23T17:58:27.956 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.951+0000 7fe635600700 1 -- 172.21.15.86:0/642895418 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6380b2210 con 0x55fc2ac03a20 2024-03-23T17:58:27.956 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.951+0000 7fe635600700 1 -- 172.21.15.86:0/642895418 <== mon.1 v2:172.21.15.152:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fe6380b90c0 con 0x55fc2ac03a20 2024-03-23T17:58:27.956 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.951+0000 7fe666072b80 1 -- 172.21.15.86:0/642895418 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe63c01e040 con 0x55fc2ac03a20 2024-03-23T17:58:27.957 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.951+0000 7fe635600700 1 -- 172.21.15.86:0/642895418 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe6380b9280 con 0x55fc2ac03a20 2024-03-23T17:58:27.957 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.951+0000 7fe666072b80 10 client.26653 fetch_fsmap learned FSMap version 40 2024-03-23T17:58:27.957 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.951+0000 7fe666072b80 1 -- 172.21.15.86:0/642895418 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55fc2a9dbc70 con 0x55fc2ac03a20 2024-03-23T17:58:27.957 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.951+0000 7fe635600700 1 -- 172.21.15.86:0/642895418 <== mon.1 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe63c01e040 con 0x55fc2ac03a20 2024-03-23T17:58:27.957 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.952+0000 7fe635600700 1 -- 172.21.15.86:0/642895418 <== mon.1 v2:172.21.15.152:3300/0 5 ==== fsmap(e 40) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fe6380a1860 con 0x55fc2ac03a20 2024-03-23T17:58:27.957 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.952+0000 7fe666072b80 10 client.26653 fetch_fsmap finished waiting for FSMap version 40 2024-03-23T17:58:27.957 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.952+0000 7fe666072b80 10 client.26653 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:58:27.957 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.952+0000 7fe666072b80 20 client.26653 populate_metadata read hostname 'smithi086' 2024-03-23T17:58:27.957 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.952+0000 7fe666072b80 1 --2- 172.21.15.86:0/642895418 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55fc2ac70960 0x55fc2ac72d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:27.957 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.952+0000 7fe666072b80 4 client.26653 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T17:58:27.957 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.952+0000 7fe666072b80 1 -- 172.21.15.86:0/642895418 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fc2a9a4c90 con 0x55fc2ac70960 2024-03-23T17:58:27.957 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.952+0000 7fe64b400700 1 --2- 172.21.15.86:0/642895418 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55fc2ac70960 0x55fc2ac72d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:27.957 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.952+0000 7fe635600700 10 client.26653 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:58:27.957 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.952+0000 7fe64b400700 1 --2- 172.21.15.86:0/642895418 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55fc2ac70960 0x55fc2ac72d80 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=766dcca16b758ce8 server_cookie=99f17af191b987c2 in_seq=0 out_seq=0 2024-03-23T17:58:27.983 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.982+0000 7fe635600700 1 -- 172.21.15.86:0/642895418 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fc2a9a4c90 con 0x55fc2ac70960 2024-03-23T17:58:27.983 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.982+0000 7fe635600700 10 client.26653 handle_command_reply: tid=0 2024-03-23T17:58:27.983 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.982+0000 7fe666072b80 1 client.26653 shutdown 2024-03-23T17:58:27.983 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.982+0000 7fe666072b80 1 -- 172.21.15.86:0/642895418 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fc2ac03a20 msgr2=0x55fc2ac64290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:58:27.983 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.982+0000 7fe666072b80 1 --2- 172.21.15.86:0/642895418 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fc2ac03a20 0x55fc2ac64290 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x55fc2ab16460 tx=0x7fe638099970 comp rx=0 tx=0).stop 2024-03-23T17:58:27.983 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.982+0000 7fe666072b80 1 -- 172.21.15.86:0/642895418 shutdown_connections 2024-03-23T17:58:27.983 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.982+0000 7fe666072b80 1 --2- 172.21.15.86:0/642895418 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55fc2ac70960 0x55fc2ac72d80 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:27.984 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.982+0000 7fe666072b80 1 --2- 172.21.15.86:0/642895418 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc2ac0e8d0 0x55fc2ac68c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:27.984 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.982+0000 7fe666072b80 1 --2- 172.21.15.86:0/642895418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc2ac03100 0x55fc2ac63d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:27.984 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.982+0000 7fe635600700 0 client.26653 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:58:27.984 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.982+0000 7fe666072b80 1 --2- 172.21.15.86:0/642895418 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fc2ac03a20 0x55fc2ac64290 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:27.984 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.982+0000 7fe666072b80 1 -- 172.21.15.86:0/642895418 >> 172.21.15.86:0/642895418 conn(0x55fc2ab63640 msgr2=0x55fc2ac0cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:58:27.984 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.982+0000 7fe666072b80 1 -- 172.21.15.86:0/642895418 shutdown_connections 2024-03-23T17:58:27.984 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.982+0000 7fe666072b80 1 -- 172.21.15.86:0/642895418 wait complete. 2024-03-23T17:58:27.984 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.982+0000 7fe666072b80 20 client.26653 trim_cache size 0 max 16384 2024-03-23T17:58:27.984 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-03-23T17:58:27.984 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-03-23T17:58:27.985 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-03-23T17:58:27.985 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-03-23T17:58:27.985 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.983+0000 7fe652600700 1 -- 172.21.15.86:0/2373180561 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fe6300a1290 msgr2=0x7fe6300a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:58:27.985 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.983+0000 7fe652600700 1 --2- 172.21.15.86:0/2373180561 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fe6300a1290 0x7fe6300a3750 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x55fc2ac04b90 tx=0x7fe6440090b0 comp rx=0 tx=0).stop 2024-03-23T17:58:27.986 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.983+0000 7fe652600700 1 -- 172.21.15.86:0/2373180561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe64c106030 msgr2=0x7fe64c199970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:58:27.986 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.983+0000 7fe652600700 1 --2- 172.21.15.86:0/2373180561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe64c106030 0x7fe64c199970 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fe63800d980 tx=0x7fe63800dc90 comp rx=0 tx=0).stop 2024-03-23T17:58:27.986 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.983+0000 7fe652600700 1 -- 172.21.15.86:0/2373180561 shutdown_connections 2024-03-23T17:58:27.986 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.983+0000 7fe652600700 1 --2- 172.21.15.86:0/2373180561 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fe6300a1290 0x7fe6300a3750 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:27.986 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.983+0000 7fe652600700 1 --2- 172.21.15.86:0/2373180561 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe64c19dd10 0x7fe64c19e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:27.986 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.983+0000 7fe652600700 1 --2- 172.21.15.86:0/2373180561 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe64c100f10 0x7fe64c199430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:27.986 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.983+0000 7fe652600700 1 --2- 172.21.15.86:0/2373180561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe64c106030 0x7fe64c199970 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:27.986 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.983+0000 7fe652600700 1 -- 172.21.15.86:0/2373180561 >> 172.21.15.86:0/2373180561 conn(0x7fe64c0fc7b0 msgr2=0x7fe64c0fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:58:27.986 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.983+0000 7fe652600700 1 -- 172.21.15.86:0/2373180561 shutdown_connections 2024-03-23T17:58:27.986 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:27.983+0000 7fe652600700 1 -- 172.21.15.86:0/2373180561 wait complete. 2024-03-23T17:58:28.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:58:27 smithi086 ceph-mon[38221]: pgmap v241: 97 pgs: 97 active+clean; 152 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 152 op/s 2024-03-23T17:58:28.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:58:27 smithi092 ceph-mon[79089]: pgmap v241: 97 pgs: 97 active+clean; 152 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 152 op/s 2024-03-23T17:58:28.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:58:27 smithi152 ceph-mon[47949]: pgmap v241: 97 pgs: 97 active+clean; 152 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 152 op/s 2024-03-23T17:58:28.910 INFO:tasks.workunit.client.0.smithi086.stdout: 1 423824 6.28 MB/sec execute 293 sec latency 1055.200 ms 2024-03-23T17:58:29.910 INFO:tasks.workunit.client.0.smithi086.stdout: 1 423824 6.26 MB/sec execute 294 sec latency 2055.278 ms 2024-03-23T17:58:30.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:58:29 smithi086 ceph-mon[38221]: pgmap v242: 97 pgs: 97 active+clean; 152 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 136 op/s 2024-03-23T17:58:30.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:58:29 smithi092 ceph-mon[79089]: pgmap v242: 97 pgs: 97 active+clean; 152 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 136 op/s 2024-03-23T17:58:30.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:58:29 smithi152 ceph-mon[47949]: pgmap v242: 97 pgs: 97 active+clean; 152 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 136 op/s 2024-03-23T17:58:30.910 INFO:tasks.workunit.client.0.smithi086.stdout: 1 423824 6.24 MB/sec execute 295 sec latency 3055.336 ms 2024-03-23T17:58:31.910 INFO:tasks.workunit.client.0.smithi086.stdout: 1 425820 6.26 MB/sec execute 296 sec latency 3058.824 ms 2024-03-23T17:58:32.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:58:31 smithi086 ceph-mon[38221]: pgmap v243: 97 pgs: 97 active+clean; 148 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 215 op/s 2024-03-23T17:58:32.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:58:31 smithi092 ceph-mon[79089]: pgmap v243: 97 pgs: 97 active+clean; 148 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 215 op/s 2024-03-23T17:58:32.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:58:31 smithi152 ceph-mon[47949]: pgmap v243: 97 pgs: 97 active+clean; 148 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 215 op/s 2024-03-23T17:58:32.910 INFO:tasks.workunit.client.0.smithi086.stdout: 1 426857 6.25 MB/sec execute 297 sec latency 13.491 ms 2024-03-23T17:58:33.910 INFO:tasks.workunit.client.0.smithi086.stdout: 1 427365 6.24 MB/sec execute 298 sec latency 706.000 ms 2024-03-23T17:58:34.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:58:33 smithi086 ceph-mon[38221]: pgmap v244: 97 pgs: 97 active+clean; 148 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 130 op/s 2024-03-23T17:58:34.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:58:33 smithi092 ceph-mon[79089]: pgmap v244: 97 pgs: 97 active+clean; 148 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 130 op/s 2024-03-23T17:58:34.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:58:33 smithi152 ceph-mon[47949]: pgmap v244: 97 pgs: 97 active+clean; 148 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 130 op/s 2024-03-23T17:58:34.911 INFO:tasks.workunit.client.0.smithi086.stdout: 1 427365 6.22 MB/sec execute 299 sec latency 1706.163 ms 2024-03-23T17:58:35.911 INFO:tasks.workunit.client.0.smithi086.stdout: 1 427365 6.20 MB/sec execute 300 sec latency 2706.220 ms 2024-03-23T17:58:36.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:58:35 smithi086 ceph-mon[38221]: pgmap v245: 97 pgs: 97 active+clean; 150 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 191 op/s 2024-03-23T17:58:36.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:58:35 smithi092 ceph-mon[79089]: pgmap v245: 97 pgs: 97 active+clean; 150 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 191 op/s 2024-03-23T17:58:36.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:58:35 smithi152 ceph-mon[47949]: pgmap v245: 97 pgs: 97 active+clean; 150 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 191 op/s 2024-03-23T17:58:36.911 INFO:tasks.workunit.client.0.smithi086.stdout: 1 429434 6.22 MB/sec execute 301 sec latency 2709.474 ms 2024-03-23T17:58:37.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:58:36 smithi086 ceph-mon[38221]: pgmap v246: 97 pgs: 97 active+clean; 150 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 141 op/s 2024-03-23T17:58:37.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:58:36 smithi092 ceph-mon[79089]: pgmap v246: 97 pgs: 97 active+clean; 150 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 141 op/s 2024-03-23T17:58:37.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:58:36 smithi152 ceph-mon[47949]: pgmap v246: 97 pgs: 97 active+clean; 150 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 141 op/s 2024-03-23T17:58:37.911 INFO:tasks.workunit.client.0.smithi086.stdout: 1 430625 6.22 MB/sec execute 302 sec latency 11.461 ms 2024-03-23T17:58:38.911 INFO:tasks.workunit.client.0.smithi086.stdout: 1 431048 6.20 MB/sec execute 303 sec latency 602.974 ms 2024-03-23T17:58:39.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:58:39 smithi152 ceph-mon[47949]: pgmap v247: 97 pgs: 97 active+clean; 150 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 141 op/s 2024-03-23T17:58:39.911 INFO:tasks.workunit.client.0.smithi086.stdout: 1 433036 6.22 MB/sec execute 304 sec latency 14.977 ms 2024-03-23T17:58:40.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:58:39 smithi086 ceph-mon[38221]: pgmap v247: 97 pgs: 97 active+clean; 150 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 141 op/s 2024-03-23T17:58:40.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:58:39 smithi092 ceph-mon[79089]: pgmap v247: 97 pgs: 97 active+clean; 150 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 141 op/s 2024-03-23T17:58:40.911 INFO:tasks.workunit.client.0.smithi086.stdout: 1 434200 6.22 MB/sec execute 305 sec latency 15.820 ms 2024-03-23T17:58:41.144 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.144+0000 7f1f4dc00700 1 -- 172.21.15.86:0/2488468184 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f48073180 msgr2=0x7f1f48073560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:58:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.144+0000 7f1f4dc00700 1 --2- 172.21.15.86:0/2488468184 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f48073180 0x7f1f48073560 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1f38009990 tx=0x7f1f38009ca0 comp rx=0 tx=0).stop 2024-03-23T17:58:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.144+0000 7f1f4dc00700 1 -- 172.21.15.86:0/2488468184 shutdown_connections 2024-03-23T17:58:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.144+0000 7f1f4dc00700 1 --2- 172.21.15.86:0/2488468184 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1f4810d540 0x7f1f4810f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.144+0000 7f1f4dc00700 1 --2- 172.21.15.86:0/2488468184 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f48073180 0x7f1f48073560 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.144+0000 7f1f4dc00700 1 --2- 172.21.15.86:0/2488468184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f48073aa0 0x7f1f4810d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:41.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.144+0000 7f1f4dc00700 1 -- 172.21.15.86:0/2488468184 >> 172.21.15.86:0/2488468184 conn(0x7f1f480fc770 msgr2=0x7f1f480feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:58:41.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.144+0000 7f1f4dc00700 1 -- 172.21.15.86:0/2488468184 shutdown_connections 2024-03-23T17:58:41.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.144+0000 7f1f4dc00700 1 -- 172.21.15.86:0/2488468184 wait complete. 2024-03-23T17:58:41.148 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.145+0000 7f1f4dc00700 1 Processor -- start 2024-03-23T17:58:41.148 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.145+0000 7f1f4dc00700 1 -- start start 2024-03-23T17:58:41.148 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.146+0000 7f1f4dc00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f48073180 0x7f1f481993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:41.148 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.146+0000 7f1f4dc00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1f48073aa0 0x7f1f48199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:41.149 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.146+0000 7f1f4dc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f4810d540 0x7f1f4819dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:41.149 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.146+0000 7f1f4dc00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4819e300 con 0x7f1f4810d540 2024-03-23T17:58:41.149 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.146+0000 7f1f4dc00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4819e470 con 0x7f1f48073180 2024-03-23T17:58:41.149 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.146+0000 7f1f4dc00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4819e630 con 0x7f1f48073aa0 2024-03-23T17:58:41.149 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.146+0000 7f1f46a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f48073180 0x7f1f481993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:41.149 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.146+0000 7f1f46a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f48073180 0x7f1f481993e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:44276/0 (socket says 172.21.15.86:44276) 2024-03-23T17:58:41.149 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.146+0000 7f1f46a00700 1 -- 172.21.15.86:0/2532862234 learned_addr learned my addr 172.21.15.86:0/2532862234 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:58:41.149 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.146+0000 7f1f46000700 1 --2- 172.21.15.86:0/2532862234 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1f48073aa0 0x7f1f48199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:41.149 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.146+0000 7f1f47400700 1 --2- 172.21.15.86:0/2532862234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f4810d540 0x7f1f4819dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:41.149 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.146+0000 7f1f46a00700 1 -- 172.21.15.86:0/2532862234 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1f48073aa0 msgr2=0x7f1f48199920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:58:41.149 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.146+0000 7f1f46a00700 1 --2- 172.21.15.86:0/2532862234 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1f48073aa0 0x7f1f48199920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:41.149 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.146+0000 7f1f46a00700 1 -- 172.21.15.86:0/2532862234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f4810d540 msgr2=0x7f1f4819dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:58:41.149 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.146+0000 7f1f46a00700 1 --2- 172.21.15.86:0/2532862234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f4810d540 0x7f1f4819dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:41.150 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.146+0000 7f1f46a00700 1 -- 172.21.15.86:0/2532862234 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f34009520 con 0x7f1f48073180 2024-03-23T17:58:41.150 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.147+0000 7f1f46a00700 1 --2- 172.21.15.86:0/2532862234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f48073180 0x7f1f481993e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f1f38005910 tx=0x7f1f38004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:58:41.150 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.147+0000 7f1f3f400700 1 -- 172.21.15.86:0/2532862234 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3801d070 con 0x7f1f48073180 2024-03-23T17:58:41.150 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.147+0000 7f1f3f400700 1 -- 172.21.15.86:0/2532862234 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f38022470 con 0x7f1f48073180 2024-03-23T17:58:41.150 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.147+0000 7f1f3f400700 1 -- 172.21.15.86:0/2532862234 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f38017650 con 0x7f1f48073180 2024-03-23T17:58:41.150 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.147+0000 7f1f4dc00700 1 -- 172.21.15.86:0/2532862234 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f380095f0 con 0x7f1f48073180 2024-03-23T17:58:41.150 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.147+0000 7f1f4dc00700 1 -- 172.21.15.86:0/2532862234 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f481a5f70 con 0x7f1f48073180 2024-03-23T17:58:41.150 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.150+0000 7f1f3f400700 1 -- 172.21.15.86:0/2532862234 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f1f380225e0 con 0x7f1f48073180 2024-03-23T17:58:41.152 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.151+0000 7f1f3f400700 1 --2- 172.21.15.86:0/2532862234 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f1f2c0a1310 0x7f1f2c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:41.152 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.151+0000 7f1f61804b80 1 Processor -- start 2024-03-23T17:58:41.152 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.151+0000 7f1f3f400700 1 -- 172.21.15.86:0/2532862234 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f1f380cdef0 con 0x7f1f48073180 2024-03-23T17:58:41.152 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.152+0000 7f1f46000700 1 --2- 172.21.15.86:0/2532862234 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f1f2c0a1310 0x7f1f2c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:41.152 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.152+0000 7f1f61804b80 1 -- start start 2024-03-23T17:58:41.154 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.152+0000 7f1f61804b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558815c24100 0x558815c244e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:41.154 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.152+0000 7f1f61804b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558815c24a20 0x558815c2f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:41.154 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.152+0000 7f1f61804b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558815c2f8d0 0x558815c31cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:41.154 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.152+0000 7f1f61804b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558815b7d930 con 0x558815c24a20 2024-03-23T17:58:41.154 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.152+0000 7f1f61804b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x558815b7b2f0 con 0x558815c2f8d0 2024-03-23T17:58:41.154 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.152+0000 7f1f61804b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x558815b76be0 con 0x558815c24100 2024-03-23T17:58:41.154 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.152+0000 7f1f47400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558815c2f8d0 0x558815c31cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:41.154 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.152+0000 7f1f46000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558815c24a20 0x558815c2f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:41.154 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.152+0000 7f1f47400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558815c2f8d0 0x558815c31cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:44280/0 (socket says 172.21.15.86:44280) 2024-03-23T17:58:41.154 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.152+0000 7f1f47400700 1 -- 172.21.15.86:0/3697063826 learned_addr learned my addr 172.21.15.86:0/3697063826 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:58:41.154 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.153+0000 7f1f46000700 1 --2- 172.21.15.86:0/2532862234 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f1f2c0a1310 0x7f1f2c0a37d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1f4819a930 tx=0x7f1f34009260 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:58:41.155 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.153+0000 7f1f46a00700 1 --2- 172.21.15.86:0/3697063826 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558815c24100 0x558815c244e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:41.155 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.153+0000 7f1f47400700 1 -- 172.21.15.86:0/3697063826 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558815c24100 msgr2=0x558815c244e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:58:41.155 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.153+0000 7f1f47400700 1 --2- 172.21.15.86:0/3697063826 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558815c24100 0x558815c244e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:41.155 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.153+0000 7f1f47400700 1 -- 172.21.15.86:0/3697063826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558815c24a20 msgr2=0x558815c2f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:58:41.155 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.153+0000 7f1f47400700 1 --2- 172.21.15.86:0/3697063826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558815c24a20 0x558815c2f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:41.155 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.153+0000 7f1f47400700 1 -- 172.21.15.86:0/3697063826 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588159f8350 con 0x558815c2f8d0 2024-03-23T17:58:41.155 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.153+0000 7f1f46a00700 1 --2- 172.21.15.86:0/3697063826 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558815c24100 0x558815c244e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:58:41.155 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.153+0000 7f1f47400700 1 --2- 172.21.15.86:0/3697063826 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558815c2f8d0 0x558815c31cc0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f1f40011600 tx=0x7f1f40009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1561cdb93abcf9 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:58:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.153+0000 7f1f33e00700 1 -- 172.21.15.86:0/3697063826 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f4000e040 con 0x558815c2f8d0 2024-03-23T17:58:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.153+0000 7f1f33e00700 1 -- 172.21.15.86:0/3697063826 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f4000f040 con 0x558815c2f8d0 2024-03-23T17:58:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.153+0000 7f1f33e00700 1 -- 172.21.15.86:0/3697063826 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f40004800 con 0x558815c2f8d0 2024-03-23T17:58:41.160 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.154+0000 7f1f61804b80 1 -- 172.21.15.86:0/3697063826 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558815c2f8d0 msgr2=0x558815c31cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:58:41.160 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.154+0000 7f1f61804b80 1 --2- 172.21.15.86:0/3697063826 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558815c2f8d0 0x558815c31cc0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f1f40011600 tx=0x7f1f40009920 comp rx=0 tx=0).stop 2024-03-23T17:58:41.161 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.154+0000 7f1f61804b80 1 -- 172.21.15.86:0/3697063826 shutdown_connections 2024-03-23T17:58:41.161 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.154+0000 7f1f61804b80 1 --2- 172.21.15.86:0/3697063826 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558815c2f8d0 0x558815c31cc0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:41.161 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.154+0000 7f1f61804b80 1 --2- 172.21.15.86:0/3697063826 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558815c24100 0x558815c244e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:41.161 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.154+0000 7f1f61804b80 1 --2- 172.21.15.86:0/3697063826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558815c24a20 0x558815c2f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:41.161 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.154+0000 7f1f61804b80 1 -- 172.21.15.86:0/3697063826 >> 172.21.15.86:0/3697063826 conn(0x558815b84640 msgr2=0x558815c20520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:58:41.161 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.154+0000 7f1f61804b80 1 -- 172.21.15.86:0/3697063826 shutdown_connections 2024-03-23T17:58:41.161 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.154+0000 7f1f61804b80 1 -- 172.21.15.86:0/3697063826 wait complete. 2024-03-23T17:58:41.161 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.155+0000 7f1f61804b80 1 -- start start 2024-03-23T17:58:41.161 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.155+0000 7f1f61804b80 1 Processor -- start 2024-03-23T17:58:41.161 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.156+0000 7f1f61804b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558815c24100 0x558815c2f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:41.161 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.156+0000 7f1f61804b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558815c24a20 0x558815c2c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:41.162 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.156+0000 7f1f61804b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558815c2f8d0 0x558815c2cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:41.162 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.156+0000 7f1f61804b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558815b12d20 con 0x558815c2f8d0 2024-03-23T17:58:41.162 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.156+0000 7f1f61804b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x558815b129c0 con 0x558815c24a20 2024-03-23T17:58:41.162 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.156+0000 7f1f61804b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588159fa640 con 0x558815c24100 2024-03-23T17:58:41.162 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.156+0000 7f1f61804b80 10 client.0 _ll_register_callbacks cb 0x558815b1c410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:58:41.162 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.156+0000 7f1f46000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558815c24a20 0x558815c2c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:41.162 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.156+0000 7f1f46a00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558815c24100 0x558815c2f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:41.162 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.156+0000 7f1f46000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558815c24a20 0x558815c2c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:44296/0 (socket says 172.21.15.86:44296) 2024-03-23T17:58:41.162 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.156+0000 7f1f46a00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558815c24100 0x558815c2f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:52514/0 (socket says 172.21.15.86:52514) 2024-03-23T17:58:41.162 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.156+0000 7f1f46000700 1 -- 172.21.15.86:0/2332709329 learned_addr learned my addr 172.21.15.86:0/2332709329 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:58:41.162 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.156+0000 7f1f47400700 1 --2- 172.21.15.86:0/2332709329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558815c2f8d0 0x558815c2cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:41.162 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.156+0000 7f1f46a00700 1 -- 172.21.15.86:0/2332709329 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558815c24a20 msgr2=0x558815c2c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:58:41.162 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.156+0000 7f1f46a00700 1 --2- 172.21.15.86:0/2332709329 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558815c24a20 0x558815c2c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:41.162 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.156+0000 7f1f46a00700 1 -- 172.21.15.86:0/2332709329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558815c2f8d0 msgr2=0x558815c2cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:58:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.156+0000 7f1f46a00700 1 --2- 172.21.15.86:0/2332709329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558815c2f8d0 0x558815c2cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.156+0000 7f1f46a00700 1 -- 172.21.15.86:0/2332709329 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f400112b0 con 0x558815c24100 2024-03-23T17:58:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.156+0000 7f1f46000700 1 --2- 172.21.15.86:0/2332709329 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558815c24a20 0x558815c2c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:58:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.157+0000 7f1f46a00700 1 --2- 172.21.15.86:0/2332709329 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558815c24100 0x558815c2f2b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1f380cb600 tx=0x7f1f3800b6b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11a07053b1db865e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:58:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.157+0000 7f1f30c00700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-03-23T17:58:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.157+0000 7f1f30c00700 1 -- 172.21.15.86:0/2332709329 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3809b750 con 0x558815c24100 2024-03-23T17:58:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.157+0000 7f1f30c00700 1 -- 172.21.15.86:0/2332709329 <== mon.1 v2:172.21.15.152:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f1f3809b050 con 0x558815c24100 2024-03-23T17:58:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.157+0000 7f1f61804b80 1 -- 172.21.15.86:0/2332709329 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558815b37320 con 0x558815c24100 2024-03-23T17:58:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.157+0000 7f1f30c00700 1 -- 172.21.15.86:0/2332709329 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f3809ee80 con 0x558815c24100 2024-03-23T17:58:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.157+0000 7f1f61804b80 10 client.26665 fetch_fsmap learned FSMap version 40 2024-03-23T17:58:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.157+0000 7f1f61804b80 1 -- 172.21.15.86:0/2332709329 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5588159a4b00 con 0x558815c24100 2024-03-23T17:58:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.157+0000 7f1f30c00700 1 -- 172.21.15.86:0/2332709329 <== mon.1 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558815b37320 con 0x558815c24100 2024-03-23T17:58:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.158+0000 7f1f30c00700 1 -- 172.21.15.86:0/2332709329 <== mon.1 v2:172.21.15.152:3300/0 5 ==== fsmap(e 40) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f1f380a2fb0 con 0x558815c24100 2024-03-23T17:58:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.158+0000 7f1f61804b80 10 client.26665 fetch_fsmap finished waiting for FSMap version 40 2024-03-23T17:58:41.164 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.158+0000 7f1f61804b80 10 client.26665 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:58:41.164 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.158+0000 7f1f61804b80 20 client.26665 populate_metadata read hostname 'smithi086' 2024-03-23T17:58:41.164 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.158+0000 7f1f61804b80 1 --2- 172.21.15.86:0/2332709329 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x558815c3f210 0x558815c41630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:41.164 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.158+0000 7f1f61804b80 4 client.26665 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:58:41.164 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.158+0000 7f1f61804b80 1 -- 172.21.15.86:0/2332709329 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558815b6f6a0 con 0x558815c3f210 2024-03-23T17:58:41.164 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.158+0000 7f1f46000700 1 --2- 172.21.15.86:0/2332709329 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x558815c3f210 0x558815c41630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:41.164 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.158+0000 7f1f46000700 1 --2- 172.21.15.86:0/2332709329 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x558815c3f210 0x558815c41630 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9647e40e3aa7cbc server_cookie=75c2d9013f603ff9 in_seq=0 out_seq=0 2024-03-23T17:58:41.164 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.158+0000 7f1f30c00700 10 client.26665 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:58:41.164 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.159+0000 7f1f30c00700 1 -- 172.21.15.86:0/2332709329 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x558815b6f6a0 con 0x558815c3f210 2024-03-23T17:58:41.164 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.159+0000 7f1f30c00700 10 client.26665 handle_command_reply: tid=0 2024-03-23T17:58:41.164 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.159+0000 7f1f61804b80 1 client.26665 shutdown 2024-03-23T17:58:41.164 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.159+0000 7f1f61804b80 1 -- 172.21.15.86:0/2332709329 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558815c24100 msgr2=0x558815c2f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:58:41.164 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.159+0000 7f1f61804b80 1 --2- 172.21.15.86:0/2332709329 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558815c24100 0x558815c2f2b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1f380cb600 tx=0x7f1f3800b6b0 comp rx=0 tx=0).stop 2024-03-23T17:58:41.165 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.159+0000 7f1f61804b80 1 -- 172.21.15.86:0/2332709329 shutdown_connections 2024-03-23T17:58:41.165 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.159+0000 7f1f61804b80 1 --2- 172.21.15.86:0/2332709329 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x558815c3f210 0x558815c41630 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:41.165 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.159+0000 7f1f61804b80 1 --2- 172.21.15.86:0/2332709329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558815c2f8d0 0x558815c2cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:41.165 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.159+0000 7f1f30c00700 0 client.26665 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:58:41.165 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.159+0000 7f1f61804b80 1 --2- 172.21.15.86:0/2332709329 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558815c24100 0x558815c2f2b0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:41.165 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.159+0000 7f1f61804b80 1 --2- 172.21.15.86:0/2332709329 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558815c24a20 0x558815c2c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:41.165 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.159+0000 7f1f61804b80 1 -- 172.21.15.86:0/2332709329 >> 172.21.15.86:0/2332709329 conn(0x558815b84640 msgr2=0x558815c260a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:58:41.165 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.160+0000 7f1f61804b80 1 -- 172.21.15.86:0/2332709329 shutdown_connections 2024-03-23T17:58:41.165 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.160+0000 7f1f61804b80 1 -- 172.21.15.86:0/2332709329 wait complete. 2024-03-23T17:58:41.165 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.160+0000 7f1f61804b80 20 client.26665 trim_cache size 0 max 16384 2024-03-23T17:58:41.180 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.178+0000 7f1f61804b80 1 Processor -- start 2024-03-23T17:58:41.180 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.178+0000 7f1f61804b80 1 -- start start 2024-03-23T17:58:41.180 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.178+0000 7f1f61804b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558815c24100 0x558815c365e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:41.180 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.178+0000 7f1f61804b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558815c24a20 0x558815c2c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:41.180 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.179+0000 7f1f61804b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558815c2f8d0 0x558815c1a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:41.180 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.179+0000 7f1f61804b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558815c2d210 con 0x558815c24a20 2024-03-23T17:58:41.180 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.179+0000 7f1f61804b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x558815c1c400 con 0x558815c24100 2024-03-23T17:58:41.180 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.179+0000 7f1f61804b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x558815c46350 con 0x558815c2f8d0 2024-03-23T17:58:41.180 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.179+0000 7f1f46a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558815c24100 0x558815c365e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:41.180 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.179+0000 7f1f46000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558815c24a20 0x558815c2c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:41.180 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.179+0000 7f1f46a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558815c24100 0x558815c365e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:44304/0 (socket says 172.21.15.86:44304) 2024-03-23T17:58:41.180 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.179+0000 7f1f46a00700 1 -- 172.21.15.86:0/3993436541 learned_addr learned my addr 172.21.15.86:0/3993436541 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:58:41.181 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.179+0000 7f1f47400700 1 --2- 172.21.15.86:0/3993436541 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558815c2f8d0 0x558815c1a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:41.181 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.179+0000 7f1f46a00700 1 -- 172.21.15.86:0/3993436541 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558815c2f8d0 msgr2=0x558815c1a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:58:41.181 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.179+0000 7f1f46a00700 1 --2- 172.21.15.86:0/3993436541 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558815c2f8d0 0x558815c1a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:41.181 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.179+0000 7f1f46a00700 1 -- 172.21.15.86:0/3993436541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558815c24a20 msgr2=0x558815c2c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:58:41.181 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.179+0000 7f1f46a00700 1 --2- 172.21.15.86:0/3993436541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558815c24a20 0x558815c2c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:41.181 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.179+0000 7f1f46a00700 1 -- 172.21.15.86:0/3993436541 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558815b3ce40 con 0x558815c24100 2024-03-23T17:58:41.181 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.179+0000 7f1f47400700 1 --2- 172.21.15.86:0/3993436541 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558815c2f8d0 0x558815c1a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T17:58:41.181 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.179+0000 7f1f46a00700 1 --2- 172.21.15.86:0/3993436541 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558815c24100 0x558815c365e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f1f3809afe0 tx=0x7f1f38005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=203dbd38e9ad4033 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:58:41.181 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.180+0000 7f1f3cc00700 1 -- 172.21.15.86:0/3993436541 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3809a770 con 0x558815c24100 2024-03-23T17:58:41.181 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.180+0000 7f1f3cc00700 1 -- 172.21.15.86:0/3993436541 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f380aee90 con 0x558815c24100 2024-03-23T17:58:41.181 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.180+0000 7f1f3cc00700 1 -- 172.21.15.86:0/3993436541 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3809ee80 con 0x558815c24100 2024-03-23T17:58:41.184 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.181+0000 7f1f61804b80 1 -- 172.21.15.86:0/3993436541 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558815c24100 msgr2=0x558815c365e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:58:41.184 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.181+0000 7f1f61804b80 1 --2- 172.21.15.86:0/3993436541 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558815c24100 0x558815c365e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f1f3809afe0 tx=0x7f1f38005e80 comp rx=0 tx=0).stop 2024-03-23T17:58:41.184 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.181+0000 7f1f61804b80 1 -- 172.21.15.86:0/3993436541 shutdown_connections 2024-03-23T17:58:41.185 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.181+0000 7f1f61804b80 1 --2- 172.21.15.86:0/3993436541 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558815c2f8d0 0x558815c1a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:41.185 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.181+0000 7f1f61804b80 1 --2- 172.21.15.86:0/3993436541 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558815c24100 0x558815c365e0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:41.185 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.181+0000 7f1f61804b80 1 --2- 172.21.15.86:0/3993436541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558815c24a20 0x558815c2c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:41.185 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.181+0000 7f1f61804b80 1 -- 172.21.15.86:0/3993436541 >> 172.21.15.86:0/3993436541 conn(0x558815b84640 msgr2=0x558815c2bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:58:41.185 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.181+0000 7f1f61804b80 1 -- 172.21.15.86:0/3993436541 shutdown_connections 2024-03-23T17:58:41.185 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.181+0000 7f1f61804b80 1 -- 172.21.15.86:0/3993436541 wait complete. 2024-03-23T17:58:41.185 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.181+0000 7f1f61804b80 1 -- start start 2024-03-23T17:58:41.185 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.181+0000 7f1f61804b80 1 Processor -- start 2024-03-23T17:58:41.185 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.182+0000 7f1f61804b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558815c24100 0x558815c343a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:41.185 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.182+0000 7f1f61804b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558815c24a20 0x558815c348e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:41.186 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.182+0000 7f1f61804b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558815c2f8d0 0x558815c8fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:41.186 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.182+0000 7f1f61804b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558815c90270 con 0x558815c2f8d0 2024-03-23T17:58:41.186 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.182+0000 7f1f61804b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x558815c903b0 con 0x558815c24a20 2024-03-23T17:58:41.186 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.182+0000 7f1f61804b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x558815c90540 con 0x558815c24100 2024-03-23T17:58:41.186 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.182+0000 7f1f61804b80 10 client.0 _ll_register_callbacks cb 0x558815b1c410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:58:41.186 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.182+0000 7f1f46000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558815c24a20 0x558815c348e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:41.186 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.182+0000 7f1f46000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558815c24a20 0x558815c348e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:44320/0 (socket says 172.21.15.86:44320) 2024-03-23T17:58:41.186 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.182+0000 7f1f46000700 1 -- 172.21.15.86:0/2063534334 learned_addr learned my addr 172.21.15.86:0/2063534334 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:58:41.186 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.182+0000 7f1f46a00700 1 --2- 172.21.15.86:0/2063534334 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558815c24100 0x558815c343a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:41.187 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.182+0000 7f1f47400700 1 --2- 172.21.15.86:0/2063534334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558815c2f8d0 0x558815c8fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:41.187 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.182+0000 7f1f46000700 1 -- 172.21.15.86:0/2063534334 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558815c24100 msgr2=0x558815c343a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:58:41.187 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.182+0000 7f1f46000700 1 --2- 172.21.15.86:0/2063534334 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558815c24100 0x558815c343a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:41.187 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.182+0000 7f1f46000700 1 -- 172.21.15.86:0/2063534334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558815c2f8d0 msgr2=0x558815c8fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:58:41.187 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.182+0000 7f1f46000700 1 --2- 172.21.15.86:0/2063534334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558815c2f8d0 0x558815c8fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:41.187 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.182+0000 7f1f46000700 1 -- 172.21.15.86:0/2063534334 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5588159a4b00 con 0x558815c24a20 2024-03-23T17:58:41.187 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.182+0000 7f1f47400700 1 --2- 172.21.15.86:0/2063534334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558815c2f8d0 0x558815c8fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:58:41.187 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.183+0000 7f1f46000700 1 --2- 172.21.15.86:0/2063534334 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558815c24a20 0x558815c348e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f1f34011ab0 tx=0x7f1f340047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e885080749b6464 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:58:41.187 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.183+0000 7f1f30c00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T17:58:41.187 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.183+0000 7f1f30c00700 1 -- 172.21.15.86:0/2063534334 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f34022040 con 0x558815c24a20 2024-03-23T17:58:41.188 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.183+0000 7f1f30c00700 1 -- 172.21.15.86:0/2063534334 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f1f3402b3e0 con 0x558815c24a20 2024-03-23T17:58:41.188 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.183+0000 7f1f61804b80 1 -- 172.21.15.86:0/2063534334 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1f4000f040 con 0x558815c24a20 2024-03-23T17:58:41.188 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.183+0000 7f1f30c00700 1 -- 172.21.15.86:0/2063534334 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f34004030 con 0x558815c24a20 2024-03-23T17:58:41.188 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.183+0000 7f1f61804b80 10 client.26681 fetch_fsmap learned FSMap version 40 2024-03-23T17:58:41.188 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.183+0000 7f1f61804b80 1 -- 172.21.15.86:0/2063534334 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5588159fcc70 con 0x558815c24a20 2024-03-23T17:58:41.188 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.184+0000 7f1f30c00700 1 -- 172.21.15.86:0/2063534334 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f4000f040 con 0x558815c24a20 2024-03-23T17:58:41.188 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.184+0000 7f1f30c00700 1 -- 172.21.15.86:0/2063534334 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 40) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f1f340037c0 con 0x558815c24a20 2024-03-23T17:58:41.189 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.184+0000 7f1f61804b80 10 client.26681 fetch_fsmap finished waiting for FSMap version 40 2024-03-23T17:58:41.189 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.184+0000 7f1f61804b80 10 client.26681 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:58:41.189 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.184+0000 7f1f61804b80 20 client.26681 populate_metadata read hostname 'smithi086' 2024-03-23T17:58:41.189 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.184+0000 7f1f61804b80 1 --2- 172.21.15.86:0/2063534334 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x558815c91d60 0x558815c94180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:41.189 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.184+0000 7f1f61804b80 4 client.26681 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T17:58:41.189 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.184+0000 7f1f61804b80 1 -- 172.21.15.86:0/2063534334 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5588159c5c90 con 0x558815c91d60 2024-03-23T17:58:41.189 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.184+0000 7f1f46a00700 1 --2- 172.21.15.86:0/2063534334 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x558815c91d60 0x558815c94180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:41.189 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.184+0000 7f1f46a00700 1 --2- 172.21.15.86:0/2063534334 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x558815c91d60 0x558815c94180 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43c7074ef046944f server_cookie=2e9550bc5f7a7f12 in_seq=0 out_seq=0 2024-03-23T17:58:41.189 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.184+0000 7f1f30c00700 10 client.26681 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:58:41.215 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.214+0000 7f1f30c00700 1 -- 172.21.15.86:0/2063534334 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5588159c5c90 con 0x558815c91d60 2024-03-23T17:58:41.215 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.214+0000 7f1f30c00700 10 client.26681 handle_command_reply: tid=0 2024-03-23T17:58:41.215 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.214+0000 7f1f61804b80 1 client.26681 shutdown 2024-03-23T17:58:41.215 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.215+0000 7f1f61804b80 1 -- 172.21.15.86:0/2063534334 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558815c24a20 msgr2=0x558815c348e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:58:41.215 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.215+0000 7f1f61804b80 1 --2- 172.21.15.86:0/2063534334 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558815c24a20 0x558815c348e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f1f34011ab0 tx=0x7f1f340047c0 comp rx=0 tx=0).stop 2024-03-23T17:58:41.215 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.215+0000 7f1f61804b80 1 -- 172.21.15.86:0/2063534334 shutdown_connections 2024-03-23T17:58:41.216 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.215+0000 7f1f61804b80 1 --2- 172.21.15.86:0/2063534334 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x558815c91d60 0x558815c94180 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:41.216 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.215+0000 7f1f61804b80 1 --2- 172.21.15.86:0/2063534334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558815c2f8d0 0x558815c8fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:41.216 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.215+0000 7f1f61804b80 1 --2- 172.21.15.86:0/2063534334 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558815c24100 0x558815c343a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:41.216 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.215+0000 7f1f61804b80 1 --2- 172.21.15.86:0/2063534334 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558815c24a20 0x558815c348e0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:41.216 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.215+0000 7f1f61804b80 1 -- 172.21.15.86:0/2063534334 >> 172.21.15.86:0/2063534334 conn(0x558815b84640 msgr2=0x558815c26040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:58:41.216 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.215+0000 7f1f30c00700 0 client.26681 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:58:41.216 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.215+0000 7f1f61804b80 1 -- 172.21.15.86:0/2063534334 shutdown_connections 2024-03-23T17:58:41.216 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.215+0000 7f1f61804b80 1 -- 172.21.15.86:0/2063534334 wait complete. 2024-03-23T17:58:41.216 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.215+0000 7f1f61804b80 20 client.26681 trim_cache size 0 max 16384 2024-03-23T17:58:41.217 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-03-23T17:58:41.217 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-03-23T17:58:41.217 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-03-23T17:58:41.217 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-03-23T17:58:41.218 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.216+0000 7f1f4dc00700 1 -- 172.21.15.86:0/2532862234 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f1f2c0a1310 msgr2=0x7f1f2c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:58:41.218 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.216+0000 7f1f4dc00700 1 --2- 172.21.15.86:0/2532862234 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f1f2c0a1310 0x7f1f2c0a37d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1f4819a930 tx=0x7f1f34009260 comp rx=0 tx=0).stop 2024-03-23T17:58:41.218 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.216+0000 7f1f4dc00700 1 -- 172.21.15.86:0/2532862234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f48073180 msgr2=0x7f1f481993e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:58:41.218 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.216+0000 7f1f4dc00700 1 --2- 172.21.15.86:0/2532862234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f48073180 0x7f1f481993e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f1f38005910 tx=0x7f1f38004730 comp rx=0 tx=0).stop 2024-03-23T17:58:41.218 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.216+0000 7f1f4dc00700 1 -- 172.21.15.86:0/2532862234 shutdown_connections 2024-03-23T17:58:41.218 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.216+0000 7f1f4dc00700 1 --2- 172.21.15.86:0/2532862234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f4810d540 0x7f1f4819dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:41.218 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.216+0000 7f1f4dc00700 1 --2- 172.21.15.86:0/2532862234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f48073180 0x7f1f481993e0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:41.218 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.216+0000 7f1f4dc00700 1 --2- 172.21.15.86:0/2532862234 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f1f2c0a1310 0x7f1f2c0a37d0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:41.218 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.216+0000 7f1f4dc00700 1 --2- 172.21.15.86:0/2532862234 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1f48073aa0 0x7f1f48199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:41.219 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.216+0000 7f1f4dc00700 1 -- 172.21.15.86:0/2532862234 >> 172.21.15.86:0/2532862234 conn(0x7f1f480fc770 msgr2=0x7f1f480fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:58:41.219 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.216+0000 7f1f4dc00700 1 -- 172.21.15.86:0/2532862234 shutdown_connections 2024-03-23T17:58:41.219 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:41.216+0000 7f1f4dc00700 1 -- 172.21.15.86:0/2532862234 wait complete. 2024-03-23T17:58:41.911 INFO:tasks.workunit.client.0.smithi086.stdout: 1 434458 6.20 MB/sec execute 306 sec latency 634.760 ms 2024-03-23T17:58:42.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:58:41 smithi092 ceph-mon[79089]: pgmap v248: 97 pgs: 97 active+clean; 155 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 204 op/s 2024-03-23T17:58:42.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:58:41 smithi152 ceph-mon[47949]: pgmap v248: 97 pgs: 97 active+clean; 155 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 204 op/s 2024-03-23T17:58:42.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:58:41 smithi086 ceph-mon[38221]: pgmap v248: 97 pgs: 97 active+clean; 155 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 204 op/s 2024-03-23T17:58:42.911 INFO:tasks.workunit.client.0.smithi086.stdout: 1 434458 6.18 MB/sec execute 307 sec latency 1634.841 ms 2024-03-23T17:58:43.911 INFO:tasks.workunit.client.0.smithi086.stdout: 1 434458 6.16 MB/sec execute 308 sec latency 2634.913 ms 2024-03-23T17:58:44.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:58:43 smithi152 ceph-mon[47949]: pgmap v249: 97 pgs: 97 active+clean; 155 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 125 op/s 2024-03-23T17:58:44.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:58:43 smithi086 ceph-mon[38221]: pgmap v249: 97 pgs: 97 active+clean; 155 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 125 op/s 2024-03-23T17:58:44.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:58:43 smithi092 ceph-mon[79089]: pgmap v249: 97 pgs: 97 active+clean; 155 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 125 op/s 2024-03-23T17:58:44.911 INFO:tasks.workunit.client.0.smithi086.stdout: 1 434458 6.14 MB/sec execute 309 sec latency 3635.052 ms 2024-03-23T17:58:45.911 INFO:tasks.workunit.client.0.smithi086.stdout: 1 434745 6.12 MB/sec execute 310 sec latency 4347.912 ms 2024-03-23T17:58:46.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:58:45 smithi152 ceph-mon[47949]: pgmap v250: 97 pgs: 97 active+clean; 155 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 177 op/s 2024-03-23T17:58:46.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:58:45 smithi086 ceph-mon[38221]: pgmap v250: 97 pgs: 97 active+clean; 155 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 177 op/s 2024-03-23T17:58:46.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:58:45 smithi092 ceph-mon[79089]: pgmap v250: 97 pgs: 97 active+clean; 155 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 177 op/s 2024-03-23T17:58:46.912 INFO:tasks.workunit.client.0.smithi086.stdout: 1 436554 6.15 MB/sec execute 311 sec latency 35.452 ms 2024-03-23T17:58:47.911 INFO:tasks.workunit.client.0.smithi086.stdout: 1 437766 6.14 MB/sec execute 312 sec latency 15.302 ms 2024-03-23T17:58:48.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:58:47 smithi152 ceph-mon[47949]: pgmap v251: 97 pgs: 97 active+clean; 155 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 115 op/s 2024-03-23T17:58:48.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:58:47 smithi086 ceph-mon[38221]: pgmap v251: 97 pgs: 97 active+clean; 155 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 115 op/s 2024-03-23T17:58:48.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:58:47 smithi092 ceph-mon[79089]: pgmap v251: 97 pgs: 97 active+clean; 155 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 115 op/s 2024-03-23T17:58:48.912 INFO:tasks.workunit.client.0.smithi086.stdout: 1 438005 6.12 MB/sec execute 313 sec latency 824.988 ms 2024-03-23T17:58:49.912 INFO:tasks.workunit.client.0.smithi086.stdout: 1 438005 6.10 MB/sec execute 314 sec latency 1825.080 ms 2024-03-23T17:58:50.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:58:49 smithi152 ceph-mon[47949]: pgmap v252: 97 pgs: 97 active+clean; 155 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 115 op/s 2024-03-23T17:58:50.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:58:49 smithi086 ceph-mon[38221]: pgmap v252: 97 pgs: 97 active+clean; 155 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 115 op/s 2024-03-23T17:58:50.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:58:49 smithi092 ceph-mon[79089]: pgmap v252: 97 pgs: 97 active+clean; 155 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 115 op/s 2024-03-23T17:58:50.912 INFO:tasks.workunit.client.0.smithi086.stdout: 1 438008 6.09 MB/sec execute 315 sec latency 2822.674 ms 2024-03-23T17:58:51.912 INFO:tasks.workunit.client.0.smithi086.stdout: 1 440016 6.11 MB/sec execute 316 sec latency 19.452 ms 2024-03-23T17:58:52.085 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:58:51 smithi152 ceph-mon[47949]: pgmap v253: 97 pgs: 97 active+clean; 157 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 176 op/s 2024-03-23T17:58:52.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:58:51 smithi086 ceph-mon[38221]: pgmap v253: 97 pgs: 97 active+clean; 157 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 176 op/s 2024-03-23T17:58:52.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:58:51 smithi092 ceph-mon[79089]: pgmap v253: 97 pgs: 97 active+clean; 157 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 176 op/s 2024-03-23T17:58:52.912 INFO:tasks.workunit.client.0.smithi086.stdout: 1 441039 6.10 MB/sec execute 317 sec latency 32.052 ms 2024-03-23T17:58:53.085 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:58:52 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:58:53.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:58:52 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:58:53.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:58:52 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:58:53.912 INFO:tasks.workunit.client.0.smithi086.stdout: 1 441547 6.09 MB/sec execute 318 sec latency 677.403 ms 2024-03-23T17:58:54.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:58:53 smithi152 ceph-mon[47949]: pgmap v254: 97 pgs: 97 active+clean; 157 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1022 KiB/s rd, 1.1 MiB/s wr, 113 op/s 2024-03-23T17:58:54.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:58:53 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:58:54.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:58:53 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:58:54.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:58:53 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:58:54.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:58:53 smithi086 ceph-mon[38221]: pgmap v254: 97 pgs: 97 active+clean; 157 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1022 KiB/s rd, 1.1 MiB/s wr, 113 op/s 2024-03-23T17:58:54.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:58:53 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:58:54.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:58:53 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:58:54.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:58:53 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:58:54.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:58:53 smithi092 ceph-mon[79089]: pgmap v254: 97 pgs: 97 active+clean; 157 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1022 KiB/s rd, 1.1 MiB/s wr, 113 op/s 2024-03-23T17:58:54.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:58:53 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:58:54.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:58:53 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:58:54.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:58:53 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:58:54.365 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.364+0000 7f0010e00700 1 -- 172.21.15.86:0/3130538742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f000c1018c0 msgr2=0x7f000c105900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:58:54.365 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.364+0000 7f0010e00700 1 --2- 172.21.15.86:0/3130538742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f000c1018c0 0x7f000c105900 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7efff8009990 tx=0x7efff8009ca0 comp rx=0 tx=0).stop 2024-03-23T17:58:54.365 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.364+0000 7f0010e00700 1 -- 172.21.15.86:0/3130538742 shutdown_connections 2024-03-23T17:58:54.365 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.364+0000 7f0010e00700 1 --2- 172.21.15.86:0/3130538742 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f000c106030 0x7f000c10dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:54.366 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.364+0000 7f0010e00700 1 --2- 172.21.15.86:0/3130538742 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f000c100f10 0x7f000c1012f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:54.366 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.364+0000 7f0010e00700 1 --2- 172.21.15.86:0/3130538742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f000c1018c0 0x7f000c105900 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:54.366 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.364+0000 7f0010e00700 1 -- 172.21.15.86:0/3130538742 >> 172.21.15.86:0/3130538742 conn(0x7f000c0fc7b0 msgr2=0x7f000c0febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:58:54.366 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.364+0000 7f0010e00700 1 -- 172.21.15.86:0/3130538742 shutdown_connections 2024-03-23T17:58:54.366 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.364+0000 7f0010e00700 1 -- 172.21.15.86:0/3130538742 wait complete. 2024-03-23T17:58:54.366 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.365+0000 7f0010e00700 1 Processor -- start 2024-03-23T17:58:54.369 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.365+0000 7f0010e00700 1 -- start start 2024-03-23T17:58:54.369 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.366+0000 7f0010e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f000c100f10 0x7f000c197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:54.369 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.366+0000 7f0010e00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f000c106030 0x7f000c1977b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:54.369 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.366+0000 7f0010e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f000c19bb00 0x7f000c19bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:54.369 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.366+0000 7f0010e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f000c19c580 con 0x7f000c100f10 2024-03-23T17:58:54.369 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.366+0000 7f0010e00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f000c19c6f0 con 0x7f000c19bb00 2024-03-23T17:58:54.369 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.366+0000 7f0010e00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f000c19c8b0 con 0x7f000c106030 2024-03-23T17:58:54.369 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.366+0000 7f0009e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f000c100f10 0x7f000c197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:54.369 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.366+0000 7f0009e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f000c100f10 0x7f000c197270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49476/0 (socket says 172.21.15.86:49476) 2024-03-23T17:58:54.369 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.366+0000 7f0009400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f000c106030 0x7f000c1977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:54.369 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.366+0000 7f0009e00700 1 -- 172.21.15.86:0/1654684924 learned_addr learned my addr 172.21.15.86:0/1654684924 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:58:54.370 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.366+0000 7f000a800700 1 --2- 172.21.15.86:0/1654684924 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f000c19bb00 0x7f000c19bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:54.370 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.367+0000 7f0009e00700 1 -- 172.21.15.86:0/1654684924 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f000c106030 msgr2=0x7f000c1977b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:58:54.370 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.367+0000 7f0009e00700 1 --2- 172.21.15.86:0/1654684924 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f000c106030 0x7f000c1977b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:54.370 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.367+0000 7f0009e00700 1 -- 172.21.15.86:0/1654684924 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f000c19bb00 msgr2=0x7f000c19bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:58:54.370 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.367+0000 7f0009e00700 1 --2- 172.21.15.86:0/1654684924 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f000c19bb00 0x7f000c19bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:54.370 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.367+0000 7f0009e00700 1 -- 172.21.15.86:0/1654684924 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efff80095f0 con 0x7f000c100f10 2024-03-23T17:58:54.370 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.367+0000 7f0009400700 1 --2- 172.21.15.86:0/1654684924 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f000c106030 0x7f000c1977b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:58:54.370 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.367+0000 7f0009e00700 1 --2- 172.21.15.86:0/1654684924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f000c100f10 0x7f000c197270 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7efffc00e9a0 tx=0x7efffc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:58:54.370 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.367+0000 7f0002a00700 1 -- 172.21.15.86:0/1654684924 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efffc009790 con 0x7f000c100f10 2024-03-23T17:58:54.370 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.367+0000 7f0002a00700 1 -- 172.21.15.86:0/1654684924 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efffc010430 con 0x7f000c100f10 2024-03-23T17:58:54.370 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.367+0000 7f0002a00700 1 -- 172.21.15.86:0/1654684924 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efffc018680 con 0x7f000c100f10 2024-03-23T17:58:54.370 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.367+0000 7f0010e00700 1 -- 172.21.15.86:0/1654684924 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f000c1a3e10 con 0x7f000c100f10 2024-03-23T17:58:54.370 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.367+0000 7f0010e00700 1 -- 172.21.15.86:0/1654684924 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f000c1a4230 con 0x7f000c100f10 2024-03-23T17:58:54.373 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.370+0000 7f0002a00700 1 -- 172.21.15.86:0/1654684924 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7efffc0105a0 con 0x7f000c100f10 2024-03-23T17:58:54.373 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.371+0000 7f0002a00700 1 --2- 172.21.15.86:0/1654684924 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7efff00a1290 0x7efff00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:54.373 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.371+0000 7f0024910b80 1 Processor -- start 2024-03-23T17:58:54.373 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.372+0000 7f0002a00700 1 -- 172.21.15.86:0/1654684924 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8484+0+0 (secure 0 0 0) 0x7efffc014070 con 0x7f000c100f10 2024-03-23T17:58:54.374 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.372+0000 7f0024910b80 1 -- start start 2024-03-23T17:58:54.374 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.372+0000 7f0009400700 1 --2- 172.21.15.86:0/1654684924 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7efff00a1290 0x7efff00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:54.374 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.372+0000 7f0024910b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56353988f100 0x56353988f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:54.374 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.372+0000 7f0024910b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56353988fa20 0x56353989a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:54.374 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.372+0000 7f0024910b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56353989a8d0 0x56353989ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:54.374 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.372+0000 7f0024910b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635397e8930 con 0x56353988f100 2024-03-23T17:58:54.374 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.372+0000 7f0024910b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635397e62f0 con 0x56353989a8d0 2024-03-23T17:58:54.374 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.372+0000 7f0024910b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635397e1be0 con 0x56353988fa20 2024-03-23T17:58:54.374 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.372+0000 7f0009e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56353988f100 0x56353988f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:54.375 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.372+0000 7f0009e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56353988f100 0x56353988f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49482/0 (socket says 172.21.15.86:49482) 2024-03-23T17:58:54.375 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.372+0000 7f0009e00700 1 -- 172.21.15.86:0/3114704190 learned_addr learned my addr 172.21.15.86:0/3114704190 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:58:54.375 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.372+0000 7f0009400700 1 --2- 172.21.15.86:0/3114704190 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56353988fa20 0x56353989a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:54.375 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.372+0000 7f000a800700 1 --2- 172.21.15.86:0/3114704190 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56353989a8d0 0x56353989ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:54.375 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.373+0000 7f0009e00700 1 -- 172.21.15.86:0/3114704190 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56353988fa20 msgr2=0x56353989a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:58:54.375 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.373+0000 7f0009e00700 1 --2- 172.21.15.86:0/3114704190 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56353988fa20 0x56353989a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:54.375 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.373+0000 7f0009e00700 1 -- 172.21.15.86:0/3114704190 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56353989a8d0 msgr2=0x56353989ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:58:54.375 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.373+0000 7f0009e00700 1 --2- 172.21.15.86:0/3114704190 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56353989a8d0 0x56353989ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:54.375 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.373+0000 7f0009e00700 1 -- 172.21.15.86:0/3114704190 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563539663350 con 0x56353988f100 2024-03-23T17:58:54.375 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.373+0000 7f0009400700 1 --2- 172.21.15.86:0/1654684924 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7efff00a1290 0x7efff00a3750 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f000c198770 tx=0x7efff800be20 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:58:54.375 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.373+0000 7f0009e00700 1 --2- 172.21.15.86:0/3114704190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56353988f100 0x56353988f4e0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7efffc0969a0 tx=0x7efffc099ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e169e00c5659666 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:58:54.375 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.373+0000 7efff7400700 1 -- 172.21.15.86:0/3114704190 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efffc09dd60 con 0x56353988f100 2024-03-23T17:58:54.376 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.373+0000 7efff7400700 1 -- 172.21.15.86:0/3114704190 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efffc0a23b0 con 0x56353988f100 2024-03-23T17:58:54.376 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.373+0000 7efff7400700 1 -- 172.21.15.86:0/3114704190 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efffc099680 con 0x56353988f100 2024-03-23T17:58:54.382 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.374+0000 7f0024910b80 1 -- 172.21.15.86:0/3114704190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56353988f100 msgr2=0x56353988f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:58:54.382 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.374+0000 7f0024910b80 1 --2- 172.21.15.86:0/3114704190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56353988f100 0x56353988f4e0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7efffc0969a0 tx=0x7efffc099ba0 comp rx=0 tx=0).stop 2024-03-23T17:58:54.382 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.374+0000 7f0024910b80 1 -- 172.21.15.86:0/3114704190 shutdown_connections 2024-03-23T17:58:54.382 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.374+0000 7f0024910b80 1 --2- 172.21.15.86:0/3114704190 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56353989a8d0 0x56353989ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:54.382 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.374+0000 7f0024910b80 1 --2- 172.21.15.86:0/3114704190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56353988f100 0x56353988f4e0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:54.382 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.374+0000 7f0024910b80 1 --2- 172.21.15.86:0/3114704190 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56353988fa20 0x56353989a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:54.382 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.374+0000 7f0024910b80 1 -- 172.21.15.86:0/3114704190 >> 172.21.15.86:0/3114704190 conn(0x5635397ef640 msgr2=0x56353988b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:58:54.382 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.374+0000 7f0024910b80 1 -- 172.21.15.86:0/3114704190 shutdown_connections 2024-03-23T17:58:54.382 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.374+0000 7f0024910b80 1 -- 172.21.15.86:0/3114704190 wait complete. 2024-03-23T17:58:54.383 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.374+0000 7f0024910b80 1 -- start start 2024-03-23T17:58:54.383 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.375+0000 7f0024910b80 1 Processor -- start 2024-03-23T17:58:54.383 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.375+0000 7f0024910b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56353988f100 0x56353989a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:54.383 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.375+0000 7f0024910b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56353988fa20 0x563539897620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:54.383 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.375+0000 7f0024910b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56353989a8d0 0x563539897cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:54.383 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.375+0000 7f0024910b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56353977dd20 con 0x56353988fa20 2024-03-23T17:58:54.383 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.375+0000 7f0024910b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x56353977d9c0 con 0x56353988f100 2024-03-23T17:58:54.383 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.375+0000 7f0024910b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x563539665640 con 0x56353989a8d0 2024-03-23T17:58:54.383 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.375+0000 7f0024910b80 10 client.0 _ll_register_callbacks cb 0x563539787410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:58:54.383 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.375+0000 7f0009400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56353988fa20 0x563539897620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:54.383 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.375+0000 7f0009400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56353988fa20 0x563539897620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49498/0 (socket says 172.21.15.86:49498) 2024-03-23T17:58:54.383 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.375+0000 7f0009400700 1 -- 172.21.15.86:0/1584698482 learned_addr learned my addr 172.21.15.86:0/1584698482 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:58:54.383 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.376+0000 7f000a800700 1 --2- 172.21.15.86:0/1584698482 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56353989a8d0 0x563539897cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:54.384 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.376+0000 7f0009e00700 1 --2- 172.21.15.86:0/1584698482 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56353988f100 0x56353989a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:54.384 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.376+0000 7f0009400700 1 -- 172.21.15.86:0/1584698482 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56353989a8d0 msgr2=0x563539897cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:58:54.384 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.376+0000 7f0009400700 1 --2- 172.21.15.86:0/1584698482 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56353989a8d0 0x563539897cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:54.384 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.376+0000 7f0009400700 1 -- 172.21.15.86:0/1584698482 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56353988f100 msgr2=0x56353989a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:58:54.384 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.376+0000 7f0009400700 1 --2- 172.21.15.86:0/1584698482 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56353988f100 0x56353989a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:54.384 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.376+0000 7f0009400700 1 -- 172.21.15.86:0/1584698482 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f000c1a4230 con 0x56353988fa20 2024-03-23T17:58:54.384 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.376+0000 7f0009e00700 1 --2- 172.21.15.86:0/1584698482 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56353988f100 0x56353989a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:58:54.384 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.376+0000 7f0009400700 1 --2- 172.21.15.86:0/1584698482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56353988fa20 0x563539897620 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7efff800b660 tx=0x7efff8009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8ec050d9ad31f75 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:58:54.384 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.376+0000 7effebe00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-23T17:58:54.384 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.376+0000 7effebe00700 1 -- 172.21.15.86:0/1584698482 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff8004d70 con 0x56353988fa20 2024-03-23T17:58:54.384 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.376+0000 7effebe00700 1 -- 172.21.15.86:0/1584698482 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8484+0+0 (secure 0 0 0) 0x7efff800f460 con 0x56353988fa20 2024-03-23T17:58:54.384 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.376+0000 7f0024910b80 1 -- 172.21.15.86:0/1584698482 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5635397a2320 con 0x56353988fa20 2024-03-23T17:58:54.384 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.377+0000 7effebe00700 1 -- 172.21.15.86:0/1584698482 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7efff800f620 con 0x56353988fa20 2024-03-23T17:58:54.384 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.377+0000 7f0024910b80 10 client.16641 fetch_fsmap learned FSMap version 40 2024-03-23T17:58:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.377+0000 7f0024910b80 1 -- 172.21.15.86:0/1584698482 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x56353960fb00 con 0x56353988fa20 2024-03-23T17:58:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.377+0000 7effebe00700 1 -- 172.21.15.86:0/1584698482 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5635397a2320 con 0x56353988fa20 2024-03-23T17:58:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.377+0000 7effebe00700 1 -- 172.21.15.86:0/1584698482 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 40) v1 ==== 7478+0+0 (secure 0 0 0) 0x7efff802f070 con 0x56353988fa20 2024-03-23T17:58:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.377+0000 7f0024910b80 10 client.16641 fetch_fsmap finished waiting for FSMap version 40 2024-03-23T17:58:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.377+0000 7f0024910b80 10 client.16641 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:58:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.377+0000 7f0024910b80 20 client.16641 populate_metadata read hostname 'smithi086' 2024-03-23T17:58:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.377+0000 7f0024910b80 1 --2- 172.21.15.86:0/1584698482 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5635398aa5a0 0x5635398ac9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.377+0000 7f0024910b80 4 client.16641 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:58:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.377+0000 7f0024910b80 1 -- 172.21.15.86:0/1584698482 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5635397da6a0 con 0x5635398aa5a0 2024-03-23T17:58:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.378+0000 7f0009e00700 1 --2- 172.21.15.86:0/1584698482 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5635398aa5a0 0x5635398ac9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.378+0000 7f0009e00700 1 --2- 172.21.15.86:0/1584698482 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5635398aa5a0 0x5635398ac9c0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e92ee70e67d1044 server_cookie=397a178f9759fc3c in_seq=0 out_seq=0 2024-03-23T17:58:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.378+0000 7effebe00700 10 client.16641 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:58:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.379+0000 7effebe00700 1 -- 172.21.15.86:0/1584698482 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5635397da6a0 con 0x5635398aa5a0 2024-03-23T17:58:54.386 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.379+0000 7effebe00700 10 client.16641 handle_command_reply: tid=0 2024-03-23T17:58:54.386 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.379+0000 7f0024910b80 1 client.16641 shutdown 2024-03-23T17:58:54.386 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.379+0000 7f0024910b80 1 -- 172.21.15.86:0/1584698482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56353988fa20 msgr2=0x563539897620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:58:54.386 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.379+0000 7f0024910b80 1 --2- 172.21.15.86:0/1584698482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56353988fa20 0x563539897620 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7efff800b660 tx=0x7efff8009ca0 comp rx=0 tx=0).stop 2024-03-23T17:58:54.386 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.379+0000 7f0024910b80 1 -- 172.21.15.86:0/1584698482 shutdown_connections 2024-03-23T17:58:54.386 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.379+0000 7f0024910b80 1 --2- 172.21.15.86:0/1584698482 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5635398aa5a0 0x5635398ac9c0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:54.386 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.379+0000 7f0024910b80 1 --2- 172.21.15.86:0/1584698482 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56353989a8d0 0x563539897cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:54.386 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.379+0000 7f0024910b80 1 --2- 172.21.15.86:0/1584698482 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56353988f100 0x56353989a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:54.386 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.379+0000 7f0024910b80 1 --2- 172.21.15.86:0/1584698482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56353988fa20 0x563539897620 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:54.386 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.379+0000 7f0024910b80 1 -- 172.21.15.86:0/1584698482 >> 172.21.15.86:0/1584698482 conn(0x5635397ef640 msgr2=0x563539891080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:58:54.386 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.379+0000 7effebe00700 0 client.16641 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:58:54.386 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.379+0000 7f0024910b80 1 -- 172.21.15.86:0/1584698482 shutdown_connections 2024-03-23T17:58:54.387 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.379+0000 7f0024910b80 1 -- 172.21.15.86:0/1584698482 wait complete. 2024-03-23T17:58:54.387 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.380+0000 7f0024910b80 20 client.16641 trim_cache size 0 max 16384 2024-03-23T17:58:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.394+0000 7f0024910b80 1 Processor -- start 2024-03-23T17:58:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.394+0000 7f0024910b80 1 -- start start 2024-03-23T17:58:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.394+0000 7f0024910b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56353988f100 0x5635398a15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.394+0000 7f0024910b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56353988fa20 0x563539885500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.395+0000 7f0024910b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56353989a8d0 0x563539891040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.395+0000 7f0024910b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x563539887400 con 0x56353989a8d0 2024-03-23T17:58:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.395+0000 7f0024910b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635398b16c0 con 0x56353988f100 2024-03-23T17:58:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.395+0000 7f0024910b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635398934e0 con 0x56353988fa20 2024-03-23T17:58:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.395+0000 7f0009e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56353988f100 0x5635398a15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.395+0000 7f000a800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56353989a8d0 0x563539891040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.395+0000 7f0009e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56353988f100 0x5635398a15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:50540/0 (socket says 172.21.15.86:50540) 2024-03-23T17:58:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.395+0000 7f0009e00700 1 -- 172.21.15.86:0/689049544 learned_addr learned my addr 172.21.15.86:0/689049544 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:58:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.395+0000 7f0009400700 1 --2- 172.21.15.86:0/689049544 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56353988fa20 0x563539885500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.395+0000 7f0009e00700 1 -- 172.21.15.86:0/689049544 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56353988fa20 msgr2=0x563539885500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:58:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.395+0000 7f0009e00700 1 --2- 172.21.15.86:0/689049544 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56353988fa20 0x563539885500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.395+0000 7f0009e00700 1 -- 172.21.15.86:0/689049544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56353989a8d0 msgr2=0x563539891040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:58:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.395+0000 7f0009e00700 1 --2- 172.21.15.86:0/689049544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56353989a8d0 0x563539891040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.395+0000 7f0009e00700 1 -- 172.21.15.86:0/689049544 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5635397a7e40 con 0x56353988f100 2024-03-23T17:58:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.395+0000 7f0009400700 1 --2- 172.21.15.86:0/689049544 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56353988fa20 0x563539885500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:58:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.395+0000 7f000a800700 1 --2- 172.21.15.86:0/689049544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56353989a8d0 0x563539891040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:58:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.395+0000 7f0009e00700 1 --2- 172.21.15.86:0/689049544 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56353988f100 0x5635398a15e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7efffc098280 tx=0x7efffc010c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b0dba0a582e562 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:58:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.396+0000 7efff7e00700 1 -- 172.21.15.86:0/689049544 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efffc098350 con 0x56353988f100 2024-03-23T17:58:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.396+0000 7efff7e00700 1 -- 172.21.15.86:0/689049544 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efffc010880 con 0x56353988f100 2024-03-23T17:58:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.396+0000 7efff7e00700 1 -- 172.21.15.86:0/689049544 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efffc09a3f0 con 0x56353988f100 2024-03-23T17:58:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.396+0000 7f0024910b80 1 -- 172.21.15.86:0/689049544 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56353988f100 msgr2=0x5635398a15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:58:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.396+0000 7f0024910b80 1 --2- 172.21.15.86:0/689049544 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56353988f100 0x5635398a15e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7efffc098280 tx=0x7efffc010c80 comp rx=0 tx=0).stop 2024-03-23T17:58:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.396+0000 7f0024910b80 1 -- 172.21.15.86:0/689049544 shutdown_connections 2024-03-23T17:58:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.396+0000 7f0024910b80 1 --2- 172.21.15.86:0/689049544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56353989a8d0 0x563539891040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.396+0000 7f0024910b80 1 --2- 172.21.15.86:0/689049544 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56353988f100 0x5635398a15e0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.396+0000 7f0024910b80 1 --2- 172.21.15.86:0/689049544 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56353988fa20 0x563539885500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.396+0000 7f0024910b80 1 -- 172.21.15.86:0/689049544 >> 172.21.15.86:0/689049544 conn(0x5635397ef640 msgr2=0x563539895f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:58:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.396+0000 7f0024910b80 1 -- 172.21.15.86:0/689049544 shutdown_connections 2024-03-23T17:58:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.396+0000 7f0024910b80 1 -- 172.21.15.86:0/689049544 wait complete. 2024-03-23T17:58:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.397+0000 7f0024910b80 1 -- start start 2024-03-23T17:58:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.397+0000 7f0024910b80 1 Processor -- start 2024-03-23T17:58:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.398+0000 7f0024910b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56353988f100 0x5635398a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.398+0000 7f0024910b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56353988fa20 0x5635398a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.398+0000 7f0024910b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56353989a8d0 0x56353989f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.398+0000 7f0024910b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56353989fb90 con 0x56353988f100 2024-03-23T17:58:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.398+0000 7f0024910b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x56353989fd00 con 0x56353988fa20 2024-03-23T17:58:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.398+0000 7f0024910b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x56353989fec0 con 0x56353989a8d0 2024-03-23T17:58:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.398+0000 7f0024910b80 10 client.0 _ll_register_callbacks cb 0x563539787410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:58:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.398+0000 7f0009e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56353988f100 0x5635398a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.398+0000 7f0009e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56353988f100 0x5635398a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49506/0 (socket says 172.21.15.86:49506) 2024-03-23T17:58:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.398+0000 7f0009e00700 1 -- 172.21.15.86:0/4180840151 learned_addr learned my addr 172.21.15.86:0/4180840151 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:58:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.398+0000 7f000a800700 1 --2- 172.21.15.86:0/4180840151 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56353989a8d0 0x56353989f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:54.403 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.398+0000 7f0009400700 1 --2- 172.21.15.86:0/4180840151 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56353988fa20 0x5635398a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:54.403 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.398+0000 7f0009e00700 1 -- 172.21.15.86:0/4180840151 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56353989a8d0 msgr2=0x56353989f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:58:54.403 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.398+0000 7f0009e00700 1 --2- 172.21.15.86:0/4180840151 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56353989a8d0 0x56353989f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:54.403 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.398+0000 7f0009e00700 1 -- 172.21.15.86:0/4180840151 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56353988fa20 msgr2=0x5635398a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:58:54.403 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.398+0000 7f0009e00700 1 --2- 172.21.15.86:0/4180840151 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56353988fa20 0x5635398a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:54.403 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.398+0000 7f0009e00700 1 -- 172.21.15.86:0/4180840151 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0004009240 con 0x56353988f100 2024-03-23T17:58:54.403 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.399+0000 7f0009e00700 1 --2- 172.21.15.86:0/4180840151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56353988f100 0x5635398a0e00 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7efffc0a1e80 tx=0x7efffc00c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9037add32e0d9777 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:58:54.403 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.399+0000 7effebe00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-23T17:58:54.403 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.399+0000 7effebe00700 1 -- 172.21.15.86:0/4180840151 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efffc09a3a0 con 0x56353988f100 2024-03-23T17:58:54.403 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.399+0000 7effebe00700 1 -- 172.21.15.86:0/4180840151 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8484+0+0 (secure 0 0 0) 0x7efffc099420 con 0x56353988f100 2024-03-23T17:58:54.403 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.399+0000 7f0024910b80 1 -- 172.21.15.86:0/4180840151 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efffc0a23b0 con 0x56353988f100 2024-03-23T17:58:54.403 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.399+0000 7effebe00700 1 -- 172.21.15.86:0/4180840151 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7efffc099600 con 0x56353988f100 2024-03-23T17:58:54.403 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.399+0000 7f0024910b80 10 client.16647 fetch_fsmap learned FSMap version 40 2024-03-23T17:58:54.404 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.399+0000 7f0024910b80 1 -- 172.21.15.86:0/4180840151 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x563539663350 con 0x56353988f100 2024-03-23T17:58:54.404 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.399+0000 7effebe00700 1 -- 172.21.15.86:0/4180840151 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efffc0a23b0 con 0x56353988f100 2024-03-23T17:58:54.404 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.399+0000 7effebe00700 1 -- 172.21.15.86:0/4180840151 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 40) v1 ==== 7478+0+0 (secure 0 0 0) 0x7efffc0b2bc0 con 0x56353988f100 2024-03-23T17:58:54.404 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.399+0000 7f0024910b80 10 client.16647 fetch_fsmap finished waiting for FSMap version 40 2024-03-23T17:58:54.404 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.399+0000 7f0024910b80 10 client.16647 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:58:54.404 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.399+0000 7f0024910b80 20 client.16647 populate_metadata read hostname 'smithi086' 2024-03-23T17:58:54.404 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.399+0000 7f0024910b80 1 --2- 172.21.15.86:0/4180840151 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5635399012d0 0x5635399036c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:58:54.404 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.399+0000 7f0024910b80 4 client.16647 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T17:58:54.404 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.399+0000 7f0024910b80 1 -- 172.21.15.86:0/4180840151 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563539630c90 con 0x5635399012d0 2024-03-23T17:58:54.404 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.400+0000 7f0009400700 1 --2- 172.21.15.86:0/4180840151 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5635399012d0 0x5635399036c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:58:54.404 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.400+0000 7effebe00700 10 client.16647 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:58:54.404 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.400+0000 7f0009400700 1 --2- 172.21.15.86:0/4180840151 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5635399012d0 0x5635399036c0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=340d50e05c3cfb16 server_cookie=340c9b52ff2117ae in_seq=0 out_seq=0 2024-03-23T17:58:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.431+0000 7effebe00700 1 -- 172.21.15.86:0/4180840151 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563539630c90 con 0x5635399012d0 2024-03-23T17:58:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.431+0000 7effebe00700 10 client.16647 handle_command_reply: tid=0 2024-03-23T17:58:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.431+0000 7f0024910b80 1 client.16647 shutdown 2024-03-23T17:58:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.431+0000 7f0024910b80 1 -- 172.21.15.86:0/4180840151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56353988f100 msgr2=0x5635398a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:58:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.431+0000 7f0024910b80 1 --2- 172.21.15.86:0/4180840151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56353988f100 0x5635398a0e00 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7efffc0a1e80 tx=0x7efffc00c470 comp rx=0 tx=0).stop 2024-03-23T17:58:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.431+0000 7f0024910b80 1 -- 172.21.15.86:0/4180840151 shutdown_connections 2024-03-23T17:58:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.431+0000 7f0024910b80 1 --2- 172.21.15.86:0/4180840151 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5635399012d0 0x5635399036c0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.431+0000 7f0024910b80 1 --2- 172.21.15.86:0/4180840151 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56353989a8d0 0x56353989f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.431+0000 7f0024910b80 1 --2- 172.21.15.86:0/4180840151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56353988f100 0x5635398a0e00 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.431+0000 7f0024910b80 1 --2- 172.21.15.86:0/4180840151 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56353988fa20 0x5635398a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.431+0000 7effebe00700 0 client.16647 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:58:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.431+0000 7f0024910b80 1 -- 172.21.15.86:0/4180840151 >> 172.21.15.86:0/4180840151 conn(0x5635397ef640 msgr2=0x56353988ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:58:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.431+0000 7f0024910b80 1 -- 172.21.15.86:0/4180840151 shutdown_connections 2024-03-23T17:58:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.431+0000 7f0024910b80 1 -- 172.21.15.86:0/4180840151 wait complete. 2024-03-23T17:58:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.432+0000 7f0024910b80 20 client.16647 trim_cache size 0 max 16384 2024-03-23T17:58:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-03-23T17:58:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-03-23T17:58:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-03-23T17:58:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-03-23T17:58:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.432+0000 7f0010e00700 1 -- 172.21.15.86:0/1654684924 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7efff00a1290 msgr2=0x7efff00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:58:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.432+0000 7f0010e00700 1 --2- 172.21.15.86:0/1654684924 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7efff00a1290 0x7efff00a3750 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f000c198770 tx=0x7efff800be20 comp rx=0 tx=0).stop 2024-03-23T17:58:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.432+0000 7f0010e00700 1 -- 172.21.15.86:0/1654684924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f000c100f10 msgr2=0x7f000c197270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:58:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.432+0000 7f0010e00700 1 --2- 172.21.15.86:0/1654684924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f000c100f10 0x7f000c197270 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7efffc00e9a0 tx=0x7efffc00ecb0 comp rx=0 tx=0).stop 2024-03-23T17:58:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.432+0000 7f0010e00700 1 -- 172.21.15.86:0/1654684924 shutdown_connections 2024-03-23T17:58:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.432+0000 7f0010e00700 1 --2- 172.21.15.86:0/1654684924 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f000c19bb00 0x7f000c19bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.432+0000 7f0010e00700 1 --2- 172.21.15.86:0/1654684924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f000c100f10 0x7f000c197270 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.432+0000 7f0010e00700 1 --2- 172.21.15.86:0/1654684924 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7efff00a1290 0x7efff00a3750 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.432+0000 7f0010e00700 1 --2- 172.21.15.86:0/1654684924 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f000c106030 0x7f000c1977b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:58:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.432+0000 7f0010e00700 1 -- 172.21.15.86:0/1654684924 >> 172.21.15.86:0/1654684924 conn(0x7f000c0fc7b0 msgr2=0x7f000c0fdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:58:54.435 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.433+0000 7f0010e00700 1 -- 172.21.15.86:0/1654684924 shutdown_connections 2024-03-23T17:58:54.435 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:58:54.433+0000 7f0010e00700 1 -- 172.21.15.86:0/1654684924 wait complete. 2024-03-23T17:58:54.912 INFO:tasks.workunit.client.0.smithi086.stdout: 1 441547 6.07 MB/sec execute 319 sec latency 1677.501 ms 2024-03-23T17:58:55.912 INFO:tasks.workunit.client.0.smithi086.stdout: 1 441550 6.05 MB/sec execute 320 sec latency 2676.900 ms 2024-03-23T17:58:56.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:58:55 smithi152 ceph-mon[47949]: pgmap v255: 97 pgs: 97 active+clean; 155 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 182 op/s 2024-03-23T17:58:56.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:58:55 smithi086 ceph-mon[38221]: pgmap v255: 97 pgs: 97 active+clean; 155 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 182 op/s 2024-03-23T17:58:56.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:58:55 smithi092 ceph-mon[79089]: pgmap v255: 97 pgs: 97 active+clean; 155 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 182 op/s 2024-03-23T17:58:56.912 INFO:tasks.workunit.client.0.smithi086.stdout: 1 443620 6.07 MB/sec execute 321 sec latency 17.282 ms 2024-03-23T17:58:57.912 INFO:tasks.workunit.client.0.smithi086.stdout: 1 444656 6.06 MB/sec execute 322 sec latency 17.459 ms 2024-03-23T17:58:58.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:58:57 smithi086 ceph-mon[38221]: pgmap v256: 97 pgs: 97 active+clean; 155 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 130 op/s 2024-03-23T17:58:58.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:58:57 smithi092 ceph-mon[79089]: pgmap v256: 97 pgs: 97 active+clean; 155 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 130 op/s 2024-03-23T17:58:58.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:58:57 smithi152 ceph-mon[47949]: pgmap v256: 97 pgs: 97 active+clean; 155 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 130 op/s 2024-03-23T17:58:58.912 INFO:tasks.workunit.client.0.smithi086.stdout: 1 445228 6.05 MB/sec execute 323 sec latency 582.954 ms 2024-03-23T17:58:59.912 INFO:tasks.workunit.client.0.smithi086.stdout: 1 447169 6.08 MB/sec execute 324 sec latency 55.178 ms 2024-03-23T17:59:00.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:58:59 smithi086 ceph-mon[38221]: pgmap v257: 97 pgs: 97 active+clean; 155 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 130 op/s 2024-03-23T17:59:00.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:58:59 smithi092 ceph-mon[79089]: pgmap v257: 97 pgs: 97 active+clean; 155 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 130 op/s 2024-03-23T17:59:00.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:58:59 smithi152 ceph-mon[47949]: pgmap v257: 97 pgs: 97 active+clean; 155 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 130 op/s 2024-03-23T17:59:00.913 INFO:tasks.workunit.client.0.smithi086.stdout: 1 448205 6.07 MB/sec execute 325 sec latency 15.448 ms 2024-03-23T17:59:01.913 INFO:tasks.workunit.client.0.smithi086.stdout: 1 448638 6.06 MB/sec execute 326 sec latency 548.131 ms 2024-03-23T17:59:02.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:59:01 smithi086 ceph-mon[38221]: pgmap v258: 97 pgs: 97 active+clean; 160 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 193 op/s 2024-03-23T17:59:02.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:59:01 smithi092 ceph-mon[79089]: pgmap v258: 97 pgs: 97 active+clean; 160 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 193 op/s 2024-03-23T17:59:02.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:59:01 smithi152 ceph-mon[47949]: pgmap v258: 97 pgs: 97 active+clean; 160 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 193 op/s 2024-03-23T17:59:02.913 INFO:tasks.workunit.client.0.smithi086.stdout: 1 448638 6.04 MB/sec execute 327 sec latency 1548.220 ms 2024-03-23T17:59:03.913 INFO:tasks.workunit.client.0.smithi086.stdout: 1 448638 6.02 MB/sec execute 328 sec latency 2548.386 ms 2024-03-23T17:59:04.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:59:03 smithi086 ceph-mon[38221]: pgmap v259: 97 pgs: 97 active+clean; 160 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 131 op/s 2024-03-23T17:59:04.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:59:03 smithi092 ceph-mon[79089]: pgmap v259: 97 pgs: 97 active+clean; 160 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 131 op/s 2024-03-23T17:59:04.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:59:03 smithi152 ceph-mon[47949]: pgmap v259: 97 pgs: 97 active+clean; 160 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 131 op/s 2024-03-23T17:59:04.914 INFO:tasks.workunit.client.0.smithi086.stdout: 1 448638 6.00 MB/sec execute 329 sec latency 3548.944 ms 2024-03-23T17:59:05.913 INFO:tasks.workunit.client.0.smithi086.stdout: 1 448946 5.98 MB/sec execute 330 sec latency 4260.827 ms 2024-03-23T17:59:06.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:59:05 smithi086 ceph-mon[38221]: pgmap v260: 97 pgs: 97 active+clean; 163 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 191 op/s 2024-03-23T17:59:06.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:59:05 smithi086 ceph-mon[38221]: osdmap e88: 12 total, 12 up, 12 in 2024-03-23T17:59:06.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:59:05 smithi092 ceph-mon[79089]: pgmap v260: 97 pgs: 97 active+clean; 163 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 191 op/s 2024-03-23T17:59:06.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:59:05 smithi092 ceph-mon[79089]: osdmap e88: 12 total, 12 up, 12 in 2024-03-23T17:59:06.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:59:05 smithi152 ceph-mon[47949]: pgmap v260: 97 pgs: 97 active+clean; 163 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 191 op/s 2024-03-23T17:59:06.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:59:05 smithi152 ceph-mon[47949]: osdmap e88: 12 total, 12 up, 12 in 2024-03-23T17:59:06.913 INFO:tasks.workunit.client.0.smithi086.stdout: 1 450283 5.99 MB/sec execute 331 sec latency 29.556 ms 2024-03-23T17:59:07.590 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.590+0000 7f41aec00700 1 -- 172.21.15.86:0/4657533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41a80ff2a0 msgr2=0x7f41a80ff680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:59:07.591 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.590+0000 7f41aec00700 1 --2- 172.21.15.86:0/4657533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41a80ff2a0 0x7f41a80ff680 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f419c009990 tx=0x7f419c009ca0 comp rx=0 tx=0).stop 2024-03-23T17:59:07.591 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.591+0000 7f41aec00700 1 -- 172.21.15.86:0/4657533 shutdown_connections 2024-03-23T17:59:07.591 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.591+0000 7f41aec00700 1 --2- 172.21.15.86:0/4657533 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41a8105e60 0x7f41a810da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:07.591 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.591+0000 7f41aec00700 1 --2- 172.21.15.86:0/4657533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41a80ff2a0 0x7f41a80ff680 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:07.591 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.591+0000 7f41aec00700 1 --2- 172.21.15.86:0/4657533 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41a80ffc50 0x7f41a8105780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:07.592 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.591+0000 7f41aec00700 1 -- 172.21.15.86:0/4657533 >> 172.21.15.86:0/4657533 conn(0x7f41a8075ff0 msgr2=0x7f41a80fd340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:59:07.592 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.591+0000 7f41aec00700 1 -- 172.21.15.86:0/4657533 shutdown_connections 2024-03-23T17:59:07.592 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.591+0000 7f41aec00700 1 -- 172.21.15.86:0/4657533 wait complete. 2024-03-23T17:59:07.592 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.593+0000 7f41aec00700 1 Processor -- start 2024-03-23T17:59:07.593 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.593+0000 7f41aec00700 1 -- start start 2024-03-23T17:59:07.595 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.593+0000 7f41aec00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41a80ff2a0 0x7f41a8100210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:59:07.595 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.593+0000 7f41aec00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41a8105e60 0x7f41a8100750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:59:07.595 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.593+0000 7f41aec00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41a8100c90 0x7f41a81a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:59:07.595 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.593+0000 7f41aec00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a81a5710 con 0x7f41a8105e60 2024-03-23T17:59:07.595 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.593+0000 7f41aec00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a81a5880 con 0x7f41a8100c90 2024-03-23T17:59:07.595 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.593+0000 7f41aec00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a81a5a40 con 0x7f41a80ff2a0 2024-03-23T17:59:07.595 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.594+0000 7f41ace00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41a8105e60 0x7f41a8100750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:59:07.595 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.594+0000 7f41ad800700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41a80ff2a0 0x7f41a8100210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:59:07.595 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.594+0000 7f41ace00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41a8105e60 0x7f41a8100750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60464/0 (socket says 172.21.15.86:60464) 2024-03-23T17:59:07.596 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.594+0000 7f41ace00700 1 -- 172.21.15.86:0/1981378164 learned_addr learned my addr 172.21.15.86:0/1981378164 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:59:07.596 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.594+0000 7f41ae200700 1 --2- 172.21.15.86:0/1981378164 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41a8100c90 0x7f41a81a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:59:07.596 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.594+0000 7f41ae200700 1 -- 172.21.15.86:0/1981378164 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41a80ff2a0 msgr2=0x7f41a8100210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:59:07.596 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.594+0000 7f41ae200700 1 --2- 172.21.15.86:0/1981378164 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41a80ff2a0 0x7f41a8100210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:07.596 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.594+0000 7f41ae200700 1 -- 172.21.15.86:0/1981378164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41a8105e60 msgr2=0x7f41a8100750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:59:07.596 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.594+0000 7f41ae200700 1 --2- 172.21.15.86:0/1981378164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41a8105e60 0x7f41a8100750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:07.596 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.594+0000 7f41ae200700 1 -- 172.21.15.86:0/1981378164 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41a0009c40 con 0x7f41a8100c90 2024-03-23T17:59:07.596 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.594+0000 7f41ace00700 1 --2- 172.21.15.86:0/1981378164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41a8105e60 0x7f41a8100750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:59:07.596 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.594+0000 7f41ad800700 1 --2- 172.21.15.86:0/1981378164 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41a80ff2a0 0x7f41a8100210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:59:07.596 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.595+0000 7f41ae200700 1 --2- 172.21.15.86:0/1981378164 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41a8100c90 0x7f41a81a5140 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f41a000efb0 tx=0x7f41a000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:59:07.596 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.595+0000 7f41a6000700 1 -- 172.21.15.86:0/1981378164 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a0003a00 con 0x7f41a8100c90 2024-03-23T17:59:07.596 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.595+0000 7f41aec00700 1 -- 172.21.15.86:0/1981378164 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f419c0095f0 con 0x7f41a8100c90 2024-03-23T17:59:07.596 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.595+0000 7f41a6000700 1 -- 172.21.15.86:0/1981378164 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f41a000f040 con 0x7f41a8100c90 2024-03-23T17:59:07.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.595+0000 7f41a6000700 1 -- 172.21.15.86:0/1981378164 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a0004540 con 0x7f41a8100c90 2024-03-23T17:59:07.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.595+0000 7f41aec00700 1 -- 172.21.15.86:0/1981378164 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41a81a6050 con 0x7f41a8100c90 2024-03-23T17:59:07.598 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.598+0000 7f41a6000700 1 -- 172.21.15.86:0/1981378164 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f41a0003bf0 con 0x7f41a8100c90 2024-03-23T17:59:07.599 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.599+0000 7f41a6000700 1 --2- 172.21.15.86:0/1981378164 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f41900a1290 0x7f41900a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:59:07.599 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.599+0000 7f41c3dfab80 1 Processor -- start 2024-03-23T17:59:07.599 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.599+0000 7f41a6000700 1 -- 172.21.15.86:0/1981378164 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f41a00cc8c0 con 0x7f41a8100c90 2024-03-23T17:59:07.601 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.600+0000 7f41c3dfab80 1 -- start start 2024-03-23T17:59:07.601 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.600+0000 7f41ad800700 1 --2- 172.21.15.86:0/1981378164 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f41900a1290 0x7f41900a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:59:07.601 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.600+0000 7f41c3dfab80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558bfd0562d0 0x558bfd0566b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:59:07.601 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.600+0000 7f41c3dfab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558bfd056bf0 0x558bfd069d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:59:07.601 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.600+0000 7f41c3dfab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558bfd06a2c0 0x558bfd06c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:59:07.601 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.600+0000 7f41c3dfab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bfcfaf930 con 0x558bfd056bf0 2024-03-23T17:59:07.602 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.600+0000 7f41c3dfab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bfcfad2f0 con 0x558bfd06a2c0 2024-03-23T17:59:07.602 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.600+0000 7f41c3dfab80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bfcfa8be0 con 0x558bfd0562d0 2024-03-23T17:59:07.602 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.600+0000 7f41ad800700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558bfd0562d0 0x558bfd0566b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:59:07.602 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.600+0000 7f41ae200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558bfd06a2c0 0x558bfd06c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:59:07.602 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.600+0000 7f41ad800700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558bfd0562d0 0x558bfd0566b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:34216/0 (socket says 172.21.15.86:34216) 2024-03-23T17:59:07.602 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.600+0000 7f41ace00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558bfd056bf0 0x558bfd069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:59:07.602 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.600+0000 7f41ad800700 1 -- 172.21.15.86:0/3941778691 learned_addr learned my addr 172.21.15.86:0/3941778691 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:59:07.602 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.601+0000 7f41ae200700 1 -- 172.21.15.86:0/3941778691 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558bfd0562d0 msgr2=0x558bfd0566b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:59:07.602 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.601+0000 7f41ae200700 1 --2- 172.21.15.86:0/3941778691 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558bfd0562d0 0x558bfd0566b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:07.602 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.601+0000 7f41ae200700 1 -- 172.21.15.86:0/3941778691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558bfd056bf0 msgr2=0x558bfd069d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:59:07.602 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.601+0000 7f41ae200700 1 --2- 172.21.15.86:0/3941778691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558bfd056bf0 0x558bfd069d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:07.602 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.601+0000 7f41ae200700 1 -- 172.21.15.86:0/3941778691 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558bfce2a350 con 0x558bfd06a2c0 2024-03-23T17:59:07.602 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.601+0000 7f41ad800700 1 --2- 172.21.15.86:0/1981378164 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f41900a1290 0x7f41900a3750 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f419c015000 tx=0x7f419c00bbe0 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:59:07.603 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.601+0000 7f41ae200700 1 --2- 172.21.15.86:0/3941778691 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558bfd06a2c0 0x558bfd06c6b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f41a00c9fd0 tx=0x7f41a00995e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e281119c8555e736 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:59:07.603 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.601+0000 7f419aa00700 1 -- 172.21.15.86:0/3941778691 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a0099a20 con 0x558bfd06a2c0 2024-03-23T17:59:07.603 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.601+0000 7f419aa00700 1 -- 172.21.15.86:0/3941778691 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f41a009d850 con 0x558bfd06a2c0 2024-03-23T17:59:07.603 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.601+0000 7f419aa00700 1 -- 172.21.15.86:0/3941778691 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a0096310 con 0x558bfd06a2c0 2024-03-23T17:59:07.606 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.602+0000 7f41c3dfab80 1 -- 172.21.15.86:0/3941778691 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558bfd06a2c0 msgr2=0x558bfd06c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:59:07.606 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.602+0000 7f41c3dfab80 1 --2- 172.21.15.86:0/3941778691 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558bfd06a2c0 0x558bfd06c6b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f41a00c9fd0 tx=0x7f41a00995e0 comp rx=0 tx=0).stop 2024-03-23T17:59:07.606 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.602+0000 7f41c3dfab80 1 -- 172.21.15.86:0/3941778691 shutdown_connections 2024-03-23T17:59:07.606 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.602+0000 7f41c3dfab80 1 --2- 172.21.15.86:0/3941778691 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558bfd06a2c0 0x558bfd06c6b0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:07.606 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.602+0000 7f41c3dfab80 1 --2- 172.21.15.86:0/3941778691 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558bfd0562d0 0x558bfd0566b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:07.606 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.602+0000 7f41c3dfab80 1 --2- 172.21.15.86:0/3941778691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558bfd056bf0 0x558bfd069d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:07.606 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.602+0000 7f41c3dfab80 1 -- 172.21.15.86:0/3941778691 >> 172.21.15.86:0/3941778691 conn(0x558bfcfb6640 msgr2=0x558bfd052750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:59:07.607 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.602+0000 7f41c3dfab80 1 -- 172.21.15.86:0/3941778691 shutdown_connections 2024-03-23T17:59:07.607 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.602+0000 7f41c3dfab80 1 -- 172.21.15.86:0/3941778691 wait complete. 2024-03-23T17:59:07.607 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.603+0000 7f41c3dfab80 1 -- start start 2024-03-23T17:59:07.607 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.603+0000 7f41c3dfab80 1 Processor -- start 2024-03-23T17:59:07.607 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.603+0000 7f41c3dfab80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558bfd0562d0 0x558bfd069d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:59:07.607 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.603+0000 7f41c3dfab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558bfd056bf0 0x558bfd067070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:59:07.607 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.603+0000 7f41c3dfab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558bfd06a2c0 0x558bfd067760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:59:07.607 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.603+0000 7f41c3dfab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bfcf44d20 con 0x558bfd056bf0 2024-03-23T17:59:07.607 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.603+0000 7f41c3dfab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bfcf449c0 con 0x558bfd06a2c0 2024-03-23T17:59:07.607 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.603+0000 7f41c3dfab80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bfce2c640 con 0x558bfd0562d0 2024-03-23T17:59:07.607 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.603+0000 7f41c3dfab80 10 client.0 _ll_register_callbacks cb 0x558bfcf4e410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:59:07.607 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.604+0000 7f41ae200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558bfd06a2c0 0x558bfd067760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:59:07.607 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.604+0000 7f41ace00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558bfd056bf0 0x558bfd067070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:59:07.608 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.604+0000 7f41ad800700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558bfd0562d0 0x558bfd069d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:59:07.608 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.604+0000 7f41ae200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558bfd06a2c0 0x558bfd067760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:47760/0 (socket says 172.21.15.86:47760) 2024-03-23T17:59:07.608 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.604+0000 7f41ae200700 1 -- 172.21.15.86:0/3366728998 learned_addr learned my addr 172.21.15.86:0/3366728998 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:59:07.608 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.604+0000 7f41ae200700 1 -- 172.21.15.86:0/3366728998 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558bfd0562d0 msgr2=0x558bfd069d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:59:07.608 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.604+0000 7f41ae200700 1 --2- 172.21.15.86:0/3366728998 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558bfd0562d0 0x558bfd069d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:07.608 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.604+0000 7f41ae200700 1 -- 172.21.15.86:0/3366728998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558bfd056bf0 msgr2=0x558bfd067070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:59:07.608 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.604+0000 7f41ae200700 1 --2- 172.21.15.86:0/3366728998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558bfd056bf0 0x558bfd067070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:07.608 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.604+0000 7f41ae200700 1 -- 172.21.15.86:0/3366728998 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41a81a6050 con 0x558bfd06a2c0 2024-03-23T17:59:07.608 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.604+0000 7f41ae200700 1 --2- 172.21.15.86:0/3366728998 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558bfd06a2c0 0x558bfd067760 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f41a000c6f0 tx=0x7f41a00094e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=915c3529b3ddfb24 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:59:07.608 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.604+0000 7f418f400700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T17:59:07.608 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.604+0000 7f418f400700 1 -- 172.21.15.86:0/3366728998 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a00048a0 con 0x558bfd06a2c0 2024-03-23T17:59:07.608 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.604+0000 7f418f400700 1 -- 172.21.15.86:0/3366728998 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f41a0099c80 con 0x558bfd06a2c0 2024-03-23T17:59:07.608 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.604+0000 7f41c3dfab80 1 -- 172.21.15.86:0/3366728998 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558bfcf69320 con 0x558bfd06a2c0 2024-03-23T17:59:07.608 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.605+0000 7f418f400700 1 -- 172.21.15.86:0/3366728998 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41a00b22e0 con 0x558bfd06a2c0 2024-03-23T17:59:07.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.605+0000 7f41c3dfab80 10 client.26711 fetch_fsmap learned FSMap version 40 2024-03-23T17:59:07.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.605+0000 7f41c3dfab80 1 -- 172.21.15.86:0/3366728998 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x558bfcdd6b00 con 0x558bfd06a2c0 2024-03-23T17:59:07.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.605+0000 7f418f400700 1 -- 172.21.15.86:0/3366728998 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558bfcf69320 con 0x558bfd06a2c0 2024-03-23T17:59:07.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.605+0000 7f418f400700 1 -- 172.21.15.86:0/3366728998 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 40) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f41a0004bd0 con 0x558bfd06a2c0 2024-03-23T17:59:07.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.605+0000 7f41c3dfab80 10 client.26711 fetch_fsmap finished waiting for FSMap version 40 2024-03-23T17:59:07.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.605+0000 7f41c3dfab80 10 client.26711 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:59:07.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.605+0000 7f41c3dfab80 20 client.26711 populate_metadata read hostname 'smithi086' 2024-03-23T17:59:07.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.605+0000 7f41c3dfab80 1 --2- 172.21.15.86:0/3366728998 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x558bfd079d70 0x558bfd07c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:59:07.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.605+0000 7f41c3dfab80 4 client.26711 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:59:07.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.605+0000 7f41c3dfab80 1 -- 172.21.15.86:0/3366728998 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558bfcfa16a0 con 0x558bfd079d70 2024-03-23T17:59:07.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.605+0000 7f41ad800700 1 --2- 172.21.15.86:0/3366728998 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x558bfd079d70 0x558bfd07c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:59:07.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.606+0000 7f41ad800700 1 --2- 172.21.15.86:0/3366728998 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x558bfd079d70 0x558bfd07c190 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fca27ecc2dfce2 server_cookie=583ed69c29fd7fdb in_seq=0 out_seq=0 2024-03-23T17:59:07.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.606+0000 7f418f400700 10 client.26711 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:59:07.611 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.607+0000 7f418f400700 1 -- 172.21.15.86:0/3366728998 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x558bfcfa16a0 con 0x558bfd079d70 2024-03-23T17:59:07.611 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.607+0000 7f418f400700 10 client.26711 handle_command_reply: tid=0 2024-03-23T17:59:07.611 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.607+0000 7f41c3dfab80 1 client.26711 shutdown 2024-03-23T17:59:07.611 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.607+0000 7f41c3dfab80 1 -- 172.21.15.86:0/3366728998 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558bfd06a2c0 msgr2=0x558bfd067760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:59:07.611 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.607+0000 7f41c3dfab80 1 --2- 172.21.15.86:0/3366728998 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558bfd06a2c0 0x558bfd067760 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f41a000c6f0 tx=0x7f41a00094e0 comp rx=0 tx=0).stop 2024-03-23T17:59:07.611 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.607+0000 7f41c3dfab80 1 -- 172.21.15.86:0/3366728998 shutdown_connections 2024-03-23T17:59:07.611 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.607+0000 7f41c3dfab80 1 --2- 172.21.15.86:0/3366728998 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x558bfd079d70 0x558bfd07c190 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:07.612 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.607+0000 7f41c3dfab80 1 --2- 172.21.15.86:0/3366728998 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558bfd06a2c0 0x558bfd067760 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:07.612 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.607+0000 7f418f400700 0 client.26711 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:59:07.612 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.607+0000 7f41c3dfab80 1 --2- 172.21.15.86:0/3366728998 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558bfd0562d0 0x558bfd069d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:07.612 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.607+0000 7f41c3dfab80 1 --2- 172.21.15.86:0/3366728998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558bfd056bf0 0x558bfd067070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:07.612 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.607+0000 7f41c3dfab80 1 -- 172.21.15.86:0/3366728998 >> 172.21.15.86:0/3366728998 conn(0x558bfcfb6640 msgr2=0x558bfd058390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:59:07.612 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.607+0000 7f41c3dfab80 1 -- 172.21.15.86:0/3366728998 shutdown_connections 2024-03-23T17:59:07.612 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.607+0000 7f41c3dfab80 1 -- 172.21.15.86:0/3366728998 wait complete. 2024-03-23T17:59:07.612 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.607+0000 7f41c3dfab80 20 client.26711 trim_cache size 0 max 16384 2024-03-23T17:59:07.624 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.622+0000 7f41c3dfab80 1 Processor -- start 2024-03-23T17:59:07.624 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.622+0000 7f41c3dfab80 1 -- start start 2024-03-23T17:59:07.624 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.623+0000 7f41c3dfab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558bfd0562d0 0x558bfd070ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:59:07.624 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.623+0000 7f41c3dfab80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558bfd056bf0 0x558bfd067650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:59:07.624 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.623+0000 7f41c3dfab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558bfd06a2c0 0x558bfd0642f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:59:07.624 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.623+0000 7f41c3dfab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bfd057e30 con 0x558bfd06a2c0 2024-03-23T17:59:07.624 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.623+0000 7f41c3dfab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bfd067d60 con 0x558bfd0562d0 2024-03-23T17:59:07.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.623+0000 7f41c3dfab80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bfd062ed0 con 0x558bfd056bf0 2024-03-23T17:59:07.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.623+0000 7f41ad800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558bfd0562d0 0x558bfd070ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:59:07.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.623+0000 7f41ad800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558bfd0562d0 0x558bfd070ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:47770/0 (socket says 172.21.15.86:47770) 2024-03-23T17:59:07.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.623+0000 7f41ad800700 1 -- 172.21.15.86:0/2118908686 learned_addr learned my addr 172.21.15.86:0/2118908686 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:59:07.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.623+0000 7f41ace00700 1 --2- 172.21.15.86:0/2118908686 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558bfd056bf0 0x558bfd067650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:59:07.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.623+0000 7f41ae200700 1 --2- 172.21.15.86:0/2118908686 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558bfd06a2c0 0x558bfd0642f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:59:07.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.623+0000 7f41ad800700 1 -- 172.21.15.86:0/2118908686 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558bfd056bf0 msgr2=0x558bfd067650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:59:07.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.623+0000 7f41ad800700 1 --2- 172.21.15.86:0/2118908686 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558bfd056bf0 0x558bfd067650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:07.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.623+0000 7f41ad800700 1 -- 172.21.15.86:0/2118908686 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558bfd06a2c0 msgr2=0x558bfd0642f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:59:07.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.623+0000 7f41ad800700 1 --2- 172.21.15.86:0/2118908686 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558bfd06a2c0 0x558bfd0642f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:07.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.623+0000 7f41ad800700 1 -- 172.21.15.86:0/2118908686 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558bfcf6ee40 con 0x558bfd0562d0 2024-03-23T17:59:07.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.623+0000 7f41ae200700 1 --2- 172.21.15.86:0/2118908686 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558bfd06a2c0 0x558bfd0642f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:59:07.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.624+0000 7f41ad800700 1 --2- 172.21.15.86:0/2118908686 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558bfd0562d0 0x558bfd070ee0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f419c00b2e0 tx=0x7f419c027000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dab8eeaedb9bfc6e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:59:07.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.624+0000 7f419b400700 1 -- 172.21.15.86:0/2118908686 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f419c0343b0 con 0x558bfd0562d0 2024-03-23T17:59:07.626 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.624+0000 7f419b400700 1 -- 172.21.15.86:0/2118908686 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f419c01e040 con 0x558bfd0562d0 2024-03-23T17:59:07.626 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.624+0000 7f419b400700 1 -- 172.21.15.86:0/2118908686 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f419c017630 con 0x558bfd0562d0 2024-03-23T17:59:07.629 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.625+0000 7f41c3dfab80 1 -- 172.21.15.86:0/2118908686 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558bfd0562d0 msgr2=0x558bfd070ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:59:07.629 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.625+0000 7f41c3dfab80 1 --2- 172.21.15.86:0/2118908686 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558bfd0562d0 0x558bfd070ee0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f419c00b2e0 tx=0x7f419c027000 comp rx=0 tx=0).stop 2024-03-23T17:59:07.629 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.625+0000 7f41c3dfab80 1 -- 172.21.15.86:0/2118908686 shutdown_connections 2024-03-23T17:59:07.629 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.625+0000 7f41c3dfab80 1 --2- 172.21.15.86:0/2118908686 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558bfd06a2c0 0x558bfd0642f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:07.629 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.625+0000 7f41c3dfab80 1 --2- 172.21.15.86:0/2118908686 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558bfd0562d0 0x558bfd070ee0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:07.629 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.625+0000 7f41c3dfab80 1 --2- 172.21.15.86:0/2118908686 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558bfd056bf0 0x558bfd067650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:07.629 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.625+0000 7f41c3dfab80 1 -- 172.21.15.86:0/2118908686 >> 172.21.15.86:0/2118908686 conn(0x558bfcfb6640 msgr2=0x558bfd04bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:59:07.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.625+0000 7f41c3dfab80 1 -- 172.21.15.86:0/2118908686 shutdown_connections 2024-03-23T17:59:07.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.625+0000 7f41c3dfab80 1 -- 172.21.15.86:0/2118908686 wait complete. 2024-03-23T17:59:07.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.625+0000 7f41c3dfab80 1 -- start start 2024-03-23T17:59:07.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.625+0000 7f41c3dfab80 1 Processor -- start 2024-03-23T17:59:07.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.626+0000 7f41c3dfab80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558bfd0562d0 0x558bfd070d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:59:07.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.626+0000 7f41c3dfab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558bfd056bf0 0x558bfd06f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:59:07.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.626+0000 7f41c3dfab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558bfd06a2c0 0x558bfd06f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:59:07.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.626+0000 7f41c3dfab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bfd06fef0 con 0x558bfd06a2c0 2024-03-23T17:59:07.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.626+0000 7f41c3dfab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bfd070060 con 0x558bfd056bf0 2024-03-23T17:59:07.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.626+0000 7f41c3dfab80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bfd0cb860 con 0x558bfd0562d0 2024-03-23T17:59:07.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.626+0000 7f41c3dfab80 10 client.0 _ll_register_callbacks cb 0x558bfcf4e410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:59:07.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.626+0000 7f41ace00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558bfd056bf0 0x558bfd06f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:59:07.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.626+0000 7f41ad800700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558bfd0562d0 0x558bfd070d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:59:07.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.626+0000 7f41ace00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558bfd056bf0 0x558bfd06f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:47782/0 (socket says 172.21.15.86:47782) 2024-03-23T17:59:07.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.626+0000 7f41ace00700 1 -- 172.21.15.86:0/1832345922 learned_addr learned my addr 172.21.15.86:0/1832345922 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:59:07.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.626+0000 7f41ae200700 1 --2- 172.21.15.86:0/1832345922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558bfd06a2c0 0x558bfd06f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:59:07.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.626+0000 7f41ad800700 1 -- 172.21.15.86:0/1832345922 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558bfd056bf0 msgr2=0x558bfd06f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:59:07.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.626+0000 7f41ad800700 1 --2- 172.21.15.86:0/1832345922 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558bfd056bf0 0x558bfd06f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:07.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.626+0000 7f41ad800700 1 -- 172.21.15.86:0/1832345922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558bfd06a2c0 msgr2=0x558bfd06f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:59:07.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.626+0000 7f41ad800700 1 --2- 172.21.15.86:0/1832345922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558bfd06a2c0 0x558bfd06f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:07.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.626+0000 7f41ad800700 1 -- 172.21.15.86:0/1832345922 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558bfcdd6b00 con 0x558bfd0562d0 2024-03-23T17:59:07.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.626+0000 7f41ace00700 1 --2- 172.21.15.86:0/1832345922 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558bfd056bf0 0x558bfd06f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:59:07.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.627+0000 7f41ad800700 1 --2- 172.21.15.86:0/1832345922 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558bfd0562d0 0x558bfd070d10 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f419c000c00 tx=0x7f419c034430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=511ace59a958cc29 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:59:07.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.627+0000 7f418f400700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-03-23T17:59:07.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.627+0000 7f418f400700 1 -- 172.21.15.86:0/1832345922 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f419c00f4e0 con 0x558bfd0562d0 2024-03-23T17:59:07.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.627+0000 7f418f400700 1 -- 172.21.15.86:0/1832345922 <== mon.1 v2:172.21.15.152:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f419c00f640 con 0x558bfd0562d0 2024-03-23T17:59:07.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.627+0000 7f41c3dfab80 1 -- 172.21.15.86:0/1832345922 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f41a009d850 con 0x558bfd0562d0 2024-03-23T17:59:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.627+0000 7f418f400700 1 -- 172.21.15.86:0/1832345922 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f419c00f800 con 0x558bfd0562d0 2024-03-23T17:59:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.627+0000 7f41c3dfab80 10 client.26701 fetch_fsmap learned FSMap version 40 2024-03-23T17:59:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.627+0000 7f41c3dfab80 1 -- 172.21.15.86:0/1832345922 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x7f419c02e040 con 0x558bfd0562d0 2024-03-23T17:59:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.627+0000 7f418f400700 1 -- 172.21.15.86:0/1832345922 <== mon.1 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f419c052080 con 0x558bfd0562d0 2024-03-23T17:59:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.628+0000 7f418f400700 1 -- 172.21.15.86:0/1832345922 <== mon.1 v2:172.21.15.152:3300/0 5 ==== fsmap(e 40) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f419c04c030 con 0x558bfd0562d0 2024-03-23T17:59:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.628+0000 7f41c3dfab80 10 client.26701 fetch_fsmap finished waiting for FSMap version 40 2024-03-23T17:59:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.628+0000 7f41c3dfab80 10 client.26701 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:59:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.628+0000 7f41c3dfab80 20 client.26701 populate_metadata read hostname 'smithi086' 2024-03-23T17:59:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.628+0000 7f41c3dfab80 1 --2- 172.21.15.86:0/1832345922 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x558bfd0ccb50 0x558bfd0cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:59:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.628+0000 7f41c3dfab80 4 client.26701 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T17:59:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.628+0000 7f41c3dfab80 1 -- 172.21.15.86:0/1832345922 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558bfcdf7c90 con 0x558bfd0ccb50 2024-03-23T17:59:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.628+0000 7f41ace00700 1 --2- 172.21.15.86:0/1832345922 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x558bfd0ccb50 0x558bfd0cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:59:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.628+0000 7f41ace00700 1 --2- 172.21.15.86:0/1832345922 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x558bfd0ccb50 0x558bfd0cef70 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c989afc7b6dba20 server_cookie=e371ac4bea0a2bc2 in_seq=0 out_seq=0 2024-03-23T17:59:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.628+0000 7f418f400700 10 client.26701 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:59:07.658 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.658+0000 7f418f400700 1 -- 172.21.15.86:0/1832345922 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558bfcdf7c90 con 0x558bfd0ccb50 2024-03-23T17:59:07.658 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.658+0000 7f418f400700 10 client.26701 handle_command_reply: tid=0 2024-03-23T17:59:07.658 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.658+0000 7f41c3dfab80 1 client.26701 shutdown 2024-03-23T17:59:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.659+0000 7f41c3dfab80 1 -- 172.21.15.86:0/1832345922 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558bfd0562d0 msgr2=0x558bfd070d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:59:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.659+0000 7f41c3dfab80 1 --2- 172.21.15.86:0/1832345922 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558bfd0562d0 0x558bfd070d10 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f419c000c00 tx=0x7f419c034430 comp rx=0 tx=0).stop 2024-03-23T17:59:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.659+0000 7f41c3dfab80 1 -- 172.21.15.86:0/1832345922 shutdown_connections 2024-03-23T17:59:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.659+0000 7f41c3dfab80 1 --2- 172.21.15.86:0/1832345922 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x558bfd0ccb50 0x558bfd0cef70 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.659+0000 7f41c3dfab80 1 --2- 172.21.15.86:0/1832345922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558bfd06a2c0 0x558bfd06f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.659+0000 7f41c3dfab80 1 --2- 172.21.15.86:0/1832345922 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558bfd0562d0 0x558bfd070d10 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:07.660 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.659+0000 7f41c3dfab80 1 --2- 172.21.15.86:0/1832345922 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558bfd056bf0 0x558bfd06f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:07.660 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.659+0000 7f41c3dfab80 1 -- 172.21.15.86:0/1832345922 >> 172.21.15.86:0/1832345922 conn(0x558bfcfb6640 msgr2=0x558bfd052740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:59:07.660 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.659+0000 7f418f400700 0 client.26701 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:59:07.660 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.659+0000 7f41c3dfab80 1 -- 172.21.15.86:0/1832345922 shutdown_connections 2024-03-23T17:59:07.660 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.659+0000 7f41c3dfab80 1 -- 172.21.15.86:0/1832345922 wait complete. 2024-03-23T17:59:07.660 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.660+0000 7f41c3dfab80 20 client.26701 trim_cache size 0 max 16384 2024-03-23T17:59:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-03-23T17:59:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-03-23T17:59:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-03-23T17:59:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-03-23T17:59:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.660+0000 7f41aec00700 1 -- 172.21.15.86:0/1981378164 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f41900a1290 msgr2=0x7f41900a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:59:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.660+0000 7f41aec00700 1 --2- 172.21.15.86:0/1981378164 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f41900a1290 0x7f41900a3750 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f419c015000 tx=0x7f419c00bbe0 comp rx=0 tx=0).stop 2024-03-23T17:59:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.660+0000 7f41aec00700 1 -- 172.21.15.86:0/1981378164 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41a8100c90 msgr2=0x7f41a81a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:59:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.660+0000 7f41aec00700 1 --2- 172.21.15.86:0/1981378164 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41a8100c90 0x7f41a81a5140 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f41a000efb0 tx=0x7f41a000c3a0 comp rx=0 tx=0).stop 2024-03-23T17:59:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.661+0000 7f41aec00700 1 -- 172.21.15.86:0/1981378164 shutdown_connections 2024-03-23T17:59:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.661+0000 7f41aec00700 1 --2- 172.21.15.86:0/1981378164 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41a8100c90 0x7f41a81a5140 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.661+0000 7f41aec00700 1 --2- 172.21.15.86:0/1981378164 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f41900a1290 0x7f41900a3750 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.661+0000 7f41aec00700 1 --2- 172.21.15.86:0/1981378164 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41a80ff2a0 0x7f41a8100210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.661+0000 7f41aec00700 1 --2- 172.21.15.86:0/1981378164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41a8105e60 0x7f41a8100750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.661+0000 7f41aec00700 1 -- 172.21.15.86:0/1981378164 >> 172.21.15.86:0/1981378164 conn(0x7f41a8075ff0 msgr2=0x7f41a80fd340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:59:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.661+0000 7f41aec00700 1 -- 172.21.15.86:0/1981378164 shutdown_connections 2024-03-23T17:59:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:07.661+0000 7f41aec00700 1 -- 172.21.15.86:0/1981378164 wait complete. 2024-03-23T17:59:07.914 INFO:tasks.workunit.client.0.smithi086.stdout: 1 451188 5.99 MB/sec execute 332 sec latency 22.123 ms 2024-03-23T17:59:08.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:59:07 smithi086 ceph-mon[38221]: pgmap v262: 97 pgs: 97 active+clean; 163 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 146 op/s 2024-03-23T17:59:08.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:59:07 smithi092 ceph-mon[79089]: pgmap v262: 97 pgs: 97 active+clean; 163 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 146 op/s 2024-03-23T17:59:08.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:59:07 smithi152 ceph-mon[47949]: pgmap v262: 97 pgs: 97 active+clean; 163 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 146 op/s 2024-03-23T17:59:08.914 INFO:tasks.workunit.client.0.smithi086.stdout: 1 452187 5.99 MB/sec execute 333 sec latency 223.882 ms 2024-03-23T17:59:09.914 INFO:tasks.workunit.client.0.smithi086.stdout: 1 452187 5.97 MB/sec execute 334 sec latency 1223.988 ms 2024-03-23T17:59:10.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:59:09 smithi086 ceph-mon[38221]: pgmap v263: 97 pgs: 97 active+clean; 163 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 146 op/s 2024-03-23T17:59:10.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:59:09 smithi092 ceph-mon[79089]: pgmap v263: 97 pgs: 97 active+clean; 163 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 146 op/s 2024-03-23T17:59:10.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:59:09 smithi152 ceph-mon[47949]: pgmap v263: 97 pgs: 97 active+clean; 163 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 146 op/s 2024-03-23T17:59:10.914 INFO:tasks.workunit.client.0.smithi086.stdout: 1 452190 5.95 MB/sec execute 335 sec latency 2222.501 ms 2024-03-23T17:59:11.914 INFO:tasks.workunit.client.0.smithi086.stdout: 1 454260 5.97 MB/sec execute 336 sec latency 20.705 ms 2024-03-23T17:59:12.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:59:11 smithi086 ceph-mon[38221]: pgmap v264: 97 pgs: 97 active+clean; 157 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 178 op/s 2024-03-23T17:59:12.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:59:11 smithi092 ceph-mon[79089]: pgmap v264: 97 pgs: 97 active+clean; 157 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 178 op/s 2024-03-23T17:59:12.338 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:59:11 smithi152 ceph-mon[47949]: pgmap v264: 97 pgs: 97 active+clean; 157 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 178 op/s 2024-03-23T17:59:12.914 INFO:tasks.workunit.client.0.smithi086.stdout: 1 455439 5.97 MB/sec execute 337 sec latency 26.907 ms 2024-03-23T17:59:13.914 INFO:tasks.workunit.client.0.smithi086.stdout: 1 455730 5.95 MB/sec execute 338 sec latency 764.573 ms 2024-03-23T17:59:14.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:59:13 smithi086 ceph-mon[38221]: pgmap v265: 97 pgs: 97 active+clean; 157 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 178 op/s 2024-03-23T17:59:14.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:59:13 smithi092 ceph-mon[79089]: pgmap v265: 97 pgs: 97 active+clean; 157 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 178 op/s 2024-03-23T17:59:14.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:59:13 smithi152 ceph-mon[47949]: pgmap v265: 97 pgs: 97 active+clean; 157 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 178 op/s 2024-03-23T17:59:14.914 INFO:tasks.workunit.client.0.smithi086.stdout: 1 455730 5.94 MB/sec execute 339 sec latency 1764.716 ms 2024-03-23T17:59:15.915 INFO:tasks.workunit.client.0.smithi086.stdout: 1 455730 5.92 MB/sec execute 340 sec latency 2764.805 ms 2024-03-23T17:59:16.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:59:15 smithi086 ceph-mon[38221]: pgmap v266: 97 pgs: 97 active+clean; 160 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 177 op/s 2024-03-23T17:59:16.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:59:15 smithi086 ceph-mon[38221]: osdmap e89: 12 total, 12 up, 12 in 2024-03-23T17:59:16.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:59:15 smithi092 ceph-mon[79089]: pgmap v266: 97 pgs: 97 active+clean; 160 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 177 op/s 2024-03-23T17:59:16.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:59:15 smithi092 ceph-mon[79089]: osdmap e89: 12 total, 12 up, 12 in 2024-03-23T17:59:16.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:59:15 smithi152 ceph-mon[47949]: pgmap v266: 97 pgs: 97 active+clean; 160 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 177 op/s 2024-03-23T17:59:16.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:59:15 smithi152 ceph-mon[47949]: osdmap e89: 12 total, 12 up, 12 in 2024-03-23T17:59:16.914 INFO:tasks.workunit.client.0.smithi086.stdout: 1 457816 5.94 MB/sec execute 341 sec latency 2767.222 ms 2024-03-23T17:59:17.914 INFO:tasks.workunit.client.0.smithi086.stdout: 1 458567 5.95 MB/sec execute 342 sec latency 52.812 ms 2024-03-23T17:59:18.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:59:17 smithi086 ceph-mon[38221]: pgmap v268: 97 pgs: 97 active+clean; 160 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 177 op/s 2024-03-23T17:59:18.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:59:17 smithi092 ceph-mon[79089]: pgmap v268: 97 pgs: 97 active+clean; 160 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 177 op/s 2024-03-23T17:59:18.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:59:17 smithi152 ceph-mon[47949]: pgmap v268: 97 pgs: 97 active+clean; 160 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 177 op/s 2024-03-23T17:59:18.914 INFO:tasks.workunit.client.0.smithi086.stdout: 1 459987 5.98 MB/sec execute 343 sec latency 18.898 ms 2024-03-23T17:59:19.914 INFO:tasks.workunit.client.0.smithi086.stdout: 1 460030 5.97 MB/sec execute 344 sec latency 958.796 ms 2024-03-23T17:59:20.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:59:19 smithi086 ceph-mon[38221]: pgmap v269: 97 pgs: 97 active+clean; 160 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 177 op/s 2024-03-23T17:59:20.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:59:19 smithi092 ceph-mon[79089]: pgmap v269: 97 pgs: 97 active+clean; 160 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 177 op/s 2024-03-23T17:59:20.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:59:19 smithi152 ceph-mon[47949]: pgmap v269: 97 pgs: 97 active+clean; 160 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 177 op/s 2024-03-23T17:59:20.820 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.820+0000 7fda5cc00700 1 -- 172.21.15.86:0/2629331596 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda58106030 msgr2=0x7fda5810dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:59:20.821 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.820+0000 7fda5cc00700 1 --2- 172.21.15.86:0/2629331596 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda58106030 0x7fda5810dbb0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fda50009f90 tx=0x7fda5000b330 comp rx=0 tx=0).stop 2024-03-23T17:59:20.821 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.820+0000 7fda5cc00700 1 -- 172.21.15.86:0/2629331596 shutdown_connections 2024-03-23T17:59:20.821 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.820+0000 7fda5cc00700 1 --2- 172.21.15.86:0/2629331596 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda58106030 0x7fda5810dbb0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:20.821 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.820+0000 7fda5cc00700 1 --2- 172.21.15.86:0/2629331596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda58100f10 0x7fda581012f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:20.821 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.820+0000 7fda5cc00700 1 --2- 172.21.15.86:0/2629331596 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda581018c0 0x7fda58105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:20.821 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.820+0000 7fda5cc00700 1 -- 172.21.15.86:0/2629331596 >> 172.21.15.86:0/2629331596 conn(0x7fda580fc770 msgr2=0x7fda580feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:59:20.821 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.820+0000 7fda5cc00700 1 -- 172.21.15.86:0/2629331596 shutdown_connections 2024-03-23T17:59:20.821 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.820+0000 7fda5cc00700 1 -- 172.21.15.86:0/2629331596 wait complete. 2024-03-23T17:59:20.823 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.821+0000 7fda5cc00700 1 Processor -- start 2024-03-23T17:59:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.821+0000 7fda5cc00700 1 -- start start 2024-03-23T17:59:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.821+0000 7fda5cc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda58100f10 0x7fda5819deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:59:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.821+0000 7fda5cc00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda58106030 0x7fda5819e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:59:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.821+0000 7fda5cc00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda581991e0 0x7fda58199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:59:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.821+0000 7fda5cc00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda58199c90 con 0x7fda58100f10 2024-03-23T17:59:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.821+0000 7fda5cc00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda58199e00 con 0x7fda581991e0 2024-03-23T17:59:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.821+0000 7fda5cc00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda58199fc0 con 0x7fda58106030 2024-03-23T17:59:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.822+0000 7fda55e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda58100f10 0x7fda5819deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:59:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.822+0000 7fda55e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda58100f10 0x7fda5819deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44200/0 (socket says 172.21.15.86:44200) 2024-03-23T17:59:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.822+0000 7fda55e00700 1 -- 172.21.15.86:0/3047453471 learned_addr learned my addr 172.21.15.86:0/3047453471 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:59:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.822+0000 7fda55400700 1 --2- 172.21.15.86:0/3047453471 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda58106030 0x7fda5819e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:59:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.822+0000 7fda56800700 1 --2- 172.21.15.86:0/3047453471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda581991e0 0x7fda58199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:59:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.822+0000 7fda55e00700 1 -- 172.21.15.86:0/3047453471 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda58106030 msgr2=0x7fda5819e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:59:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.822+0000 7fda55e00700 1 --2- 172.21.15.86:0/3047453471 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda58106030 0x7fda5819e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.822+0000 7fda55e00700 1 -- 172.21.15.86:0/3047453471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda581991e0 msgr2=0x7fda58199690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:59:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.822+0000 7fda55e00700 1 --2- 172.21.15.86:0/3047453471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda581991e0 0x7fda58199690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.822+0000 7fda55e00700 1 -- 172.21.15.86:0/3047453471 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda50009c40 con 0x7fda58100f10 2024-03-23T17:59:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.822+0000 7fda55e00700 1 --2- 172.21.15.86:0/3047453471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda58100f10 0x7fda5819deb0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fda4800e9c0 tx=0x7fda4800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:59:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.823+0000 7fda4ea00700 1 -- 172.21.15.86:0/3047453471 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda48004a30 con 0x7fda58100f10 2024-03-23T17:59:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.823+0000 7fda4ea00700 1 -- 172.21.15.86:0/3047453471 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda4800cc80 con 0x7fda58100f10 2024-03-23T17:59:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.823+0000 7fda5cc00700 1 -- 172.21.15.86:0/3047453471 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda581a5e80 con 0x7fda58100f10 2024-03-23T17:59:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.823+0000 7fda4ea00700 1 -- 172.21.15.86:0/3047453471 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda480054e0 con 0x7fda58100f10 2024-03-23T17:59:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.823+0000 7fda5cc00700 1 -- 172.21.15.86:0/3047453471 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda581a6240 con 0x7fda58100f10 2024-03-23T17:59:20.826 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.826+0000 7fda4ea00700 1 -- 172.21.15.86:0/3047453471 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fda48004b90 con 0x7fda58100f10 2024-03-23T17:59:20.829 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.827+0000 7fda4ea00700 1 --2- 172.21.15.86:0/3047453471 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fda3c0a1290 0x7fda3c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:59:20.829 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.828+0000 7fda70698b80 1 Processor -- start 2024-03-23T17:59:20.829 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.828+0000 7fda4ea00700 1 -- 172.21.15.86:0/3047453471 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fda48014070 con 0x7fda58100f10 2024-03-23T17:59:20.830 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.828+0000 7fda55400700 1 --2- 172.21.15.86:0/3047453471 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fda3c0a1290 0x7fda3c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:59:20.830 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.828+0000 7fda70698b80 1 -- start start 2024-03-23T17:59:20.830 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.828+0000 7fda70698b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5612cb141100 0x5612cb1414e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:59:20.830 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.828+0000 7fda70698b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612cb141a20 0x5612cb14c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:59:20.830 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.828+0000 7fda70698b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612cb14c8d0 0x5612cb14ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:59:20.830 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.828+0000 7fda70698b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612cb09a930 con 0x5612cb141a20 2024-03-23T17:59:20.830 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.828+0000 7fda70698b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612cb0982f0 con 0x5612cb14c8d0 2024-03-23T17:59:20.830 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.828+0000 7fda70698b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612cb093be0 con 0x5612cb141100 2024-03-23T17:59:20.830 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.828+0000 7fda55400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612cb141a20 0x5612cb14c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:59:20.830 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.828+0000 7fda55e00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5612cb141100 0x5612cb1414e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:59:20.830 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.828+0000 7fda55400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612cb141a20 0x5612cb14c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44206/0 (socket says 172.21.15.86:44206) 2024-03-23T17:59:20.830 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.828+0000 7fda55400700 1 -- 172.21.15.86:0/723692477 learned_addr learned my addr 172.21.15.86:0/723692477 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:59:20.831 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.828+0000 7fda56800700 1 --2- 172.21.15.86:0/723692477 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612cb14c8d0 0x5612cb14ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:59:20.831 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.829+0000 7fda55400700 1 --2- 172.21.15.86:0/3047453471 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fda3c0a1290 0x7fda3c0a3750 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fda580ff270 tx=0x7fda44007f40 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:59:20.831 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.829+0000 7fda55e00700 1 -- 172.21.15.86:0/723692477 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612cb14c8d0 msgr2=0x5612cb14ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:59:20.831 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.829+0000 7fda55e00700 1 --2- 172.21.15.86:0/723692477 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612cb14c8d0 0x5612cb14ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:20.831 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.829+0000 7fda55e00700 1 -- 172.21.15.86:0/723692477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612cb141a20 msgr2=0x5612cb14c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:59:20.831 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.829+0000 7fda55e00700 1 --2- 172.21.15.86:0/723692477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612cb141a20 0x5612cb14c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:20.831 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.829+0000 7fda55e00700 1 -- 172.21.15.86:0/723692477 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5612caf15350 con 0x5612cb141100 2024-03-23T17:59:20.831 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.829+0000 7fda55400700 1 --2- 172.21.15.86:0/723692477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612cb141a20 0x5612cb14c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:59:20.831 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.829+0000 7fda55e00700 1 --2- 172.21.15.86:0/723692477 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5612cb141100 0x5612cb1414e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fda480ca4e0 tx=0x7fda48099880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35c281d012a7b091 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:59:20.831 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.829+0000 7fda43000700 1 -- 172.21.15.86:0/723692477 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda480b1070 con 0x5612cb141100 2024-03-23T17:59:20.831 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.829+0000 7fda43000700 1 -- 172.21.15.86:0/723692477 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda48099ea0 con 0x5612cb141100 2024-03-23T17:59:20.831 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.829+0000 7fda43000700 1 -- 172.21.15.86:0/723692477 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda4809a100 con 0x5612cb141100 2024-03-23T17:59:20.835 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.830+0000 7fda70698b80 1 -- 172.21.15.86:0/723692477 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5612cb141100 msgr2=0x5612cb1414e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:59:20.836 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.830+0000 7fda70698b80 1 --2- 172.21.15.86:0/723692477 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5612cb141100 0x5612cb1414e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fda480ca4e0 tx=0x7fda48099880 comp rx=0 tx=0).stop 2024-03-23T17:59:20.836 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.830+0000 7fda70698b80 1 -- 172.21.15.86:0/723692477 shutdown_connections 2024-03-23T17:59:20.836 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.830+0000 7fda70698b80 1 --2- 172.21.15.86:0/723692477 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612cb14c8d0 0x5612cb14ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:20.836 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.830+0000 7fda70698b80 1 --2- 172.21.15.86:0/723692477 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5612cb141100 0x5612cb1414e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:20.836 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.830+0000 7fda70698b80 1 --2- 172.21.15.86:0/723692477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612cb141a20 0x5612cb14c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:20.836 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.830+0000 7fda70698b80 1 -- 172.21.15.86:0/723692477 >> 172.21.15.86:0/723692477 conn(0x5612cb0a1640 msgr2=0x5612cb13d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:59:20.836 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.830+0000 7fda70698b80 1 -- 172.21.15.86:0/723692477 shutdown_connections 2024-03-23T17:59:20.836 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.830+0000 7fda70698b80 1 -- 172.21.15.86:0/723692477 wait complete. 2024-03-23T17:59:20.836 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.831+0000 7fda70698b80 1 -- start start 2024-03-23T17:59:20.836 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.831+0000 7fda70698b80 1 Processor -- start 2024-03-23T17:59:20.836 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.831+0000 7fda70698b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612cb141100 0x5612cb14c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:59:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.831+0000 7fda70698b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5612cb141a20 0x5612cb149600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:59:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.831+0000 7fda70698b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612cb14c8d0 0x5612cb149d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:59:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.831+0000 7fda70698b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612cb02fd20 con 0x5612cb141100 2024-03-23T17:59:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.831+0000 7fda70698b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612cb02f9c0 con 0x5612cb14c8d0 2024-03-23T17:59:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.831+0000 7fda70698b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612caf17640 con 0x5612cb141a20 2024-03-23T17:59:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.831+0000 7fda70698b80 10 client.0 _ll_register_callbacks cb 0x5612cb039410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:59:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.832+0000 7fda55e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612cb141100 0x5612cb14c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:59:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.832+0000 7fda55e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612cb141100 0x5612cb14c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44222/0 (socket says 172.21.15.86:44222) 2024-03-23T17:59:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.832+0000 7fda55e00700 1 -- 172.21.15.86:0/4218665954 learned_addr learned my addr 172.21.15.86:0/4218665954 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:59:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.832+0000 7fda56800700 1 --2- 172.21.15.86:0/4218665954 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612cb14c8d0 0x5612cb149d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:59:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.832+0000 7fda55400700 1 --2- 172.21.15.86:0/4218665954 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5612cb141a20 0x5612cb149600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:59:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.832+0000 7fda55e00700 1 -- 172.21.15.86:0/4218665954 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5612cb141a20 msgr2=0x5612cb149600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:59:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.832+0000 7fda55e00700 1 --2- 172.21.15.86:0/4218665954 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5612cb141a20 0x5612cb149600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.832+0000 7fda55e00700 1 -- 172.21.15.86:0/4218665954 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612cb14c8d0 msgr2=0x5612cb149d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:59:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.832+0000 7fda55e00700 1 --2- 172.21.15.86:0/4218665954 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612cb14c8d0 0x5612cb149d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.832+0000 7fda55e00700 1 -- 172.21.15.86:0/4218665954 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda44010ab0 con 0x5612cb141100 2024-03-23T17:59:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.832+0000 7fda55e00700 1 --2- 172.21.15.86:0/4218665954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612cb141100 0x5612cb14c310 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fda480ca4b0 tx=0x7fda48098450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=552cd2ac8ec48954 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:59:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.832+0000 7fda37400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-23T17:59:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.832+0000 7fda37400700 1 -- 172.21.15.86:0/4218665954 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda480b1070 con 0x5612cb141100 2024-03-23T17:59:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.832+0000 7fda37400700 1 -- 172.21.15.86:0/4218665954 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fda480a2320 con 0x5612cb141100 2024-03-23T17:59:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.832+0000 7fda70698b80 1 -- 172.21.15.86:0/4218665954 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5612cb054320 con 0x5612cb141100 2024-03-23T17:59:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.833+0000 7fda37400700 1 -- 172.21.15.86:0/4218665954 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda48099390 con 0x5612cb141100 2024-03-23T17:59:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.833+0000 7fda70698b80 10 client.16677 fetch_fsmap learned FSMap version 40 2024-03-23T17:59:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.833+0000 7fda70698b80 1 -- 172.21.15.86:0/4218665954 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x7fda581a6240 con 0x5612cb141100 2024-03-23T17:59:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.833+0000 7fda37400700 1 -- 172.21.15.86:0/4218665954 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda48063080 con 0x5612cb141100 2024-03-23T17:59:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.833+0000 7fda37400700 1 -- 172.21.15.86:0/4218665954 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 40) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fda480a1e90 con 0x5612cb141100 2024-03-23T17:59:20.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.833+0000 7fda70698b80 10 client.16677 fetch_fsmap finished waiting for FSMap version 40 2024-03-23T17:59:20.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.833+0000 7fda70698b80 10 client.16677 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:59:20.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.833+0000 7fda70698b80 20 client.16677 populate_metadata read hostname 'smithi086' 2024-03-23T17:59:20.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.833+0000 7fda70698b80 1 --2- 172.21.15.86:0/4218665954 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5612cb15c2a0 0x5612cb15e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:59:20.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.833+0000 7fda70698b80 4 client.16677 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:59:20.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.833+0000 7fda70698b80 1 -- 172.21.15.86:0/4218665954 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5612cb08c6a0 con 0x5612cb15c2a0 2024-03-23T17:59:20.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.833+0000 7fda55400700 1 --2- 172.21.15.86:0/4218665954 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5612cb15c2a0 0x5612cb15e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:59:20.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.834+0000 7fda55400700 1 --2- 172.21.15.86:0/4218665954 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5612cb15c2a0 0x5612cb15e6c0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae10465a24b723c0 server_cookie=64e41d8ae5101944 in_seq=0 out_seq=0 2024-03-23T17:59:20.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.834+0000 7fda37400700 10 client.16677 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:59:20.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.835+0000 7fda37400700 1 -- 172.21.15.86:0/4218665954 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5612cb08c6a0 con 0x5612cb15c2a0 2024-03-23T17:59:20.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.835+0000 7fda37400700 10 client.16677 handle_command_reply: tid=0 2024-03-23T17:59:20.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.835+0000 7fda70698b80 1 client.16677 shutdown 2024-03-23T17:59:20.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.835+0000 7fda70698b80 1 -- 172.21.15.86:0/4218665954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612cb141100 msgr2=0x5612cb14c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:59:20.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.835+0000 7fda70698b80 1 --2- 172.21.15.86:0/4218665954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612cb141100 0x5612cb14c310 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fda480ca4b0 tx=0x7fda48098450 comp rx=0 tx=0).stop 2024-03-23T17:59:20.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.835+0000 7fda70698b80 1 -- 172.21.15.86:0/4218665954 shutdown_connections 2024-03-23T17:59:20.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.835+0000 7fda70698b80 1 --2- 172.21.15.86:0/4218665954 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5612cb15c2a0 0x5612cb15e6c0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:20.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.835+0000 7fda70698b80 1 --2- 172.21.15.86:0/4218665954 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612cb14c8d0 0x5612cb149d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:20.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.835+0000 7fda70698b80 1 --2- 172.21.15.86:0/4218665954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612cb141100 0x5612cb14c310 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:20.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.835+0000 7fda70698b80 1 --2- 172.21.15.86:0/4218665954 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5612cb141a20 0x5612cb149600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:20.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.835+0000 7fda37400700 0 client.16677 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:59:20.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.835+0000 7fda70698b80 1 -- 172.21.15.86:0/4218665954 >> 172.21.15.86:0/4218665954 conn(0x5612cb0a1640 msgr2=0x5612cb143080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:59:20.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.835+0000 7fda70698b80 1 -- 172.21.15.86:0/4218665954 shutdown_connections 2024-03-23T17:59:20.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.835+0000 7fda70698b80 1 -- 172.21.15.86:0/4218665954 wait complete. 2024-03-23T17:59:20.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.835+0000 7fda70698b80 20 client.16677 trim_cache size 0 max 16384 2024-03-23T17:59:20.851 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.850+0000 7fda70698b80 1 Processor -- start 2024-03-23T17:59:20.851 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.850+0000 7fda70698b80 1 -- start start 2024-03-23T17:59:20.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.851+0000 7fda70698b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612cb141100 0x5612cb163020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:59:20.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.851+0000 7fda70698b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612cb141a20 0x5612cb1535a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:59:20.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.851+0000 7fda70698b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5612cb14c8d0 0x5612cb13cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:59:20.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.851+0000 7fda70698b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612cb147bf0 con 0x5612cb141100 2024-03-23T17:59:20.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.851+0000 7fda70698b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612cb145180 con 0x5612cb141a20 2024-03-23T17:59:20.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.851+0000 7fda70698b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612cb137500 con 0x5612cb14c8d0 2024-03-23T17:59:20.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.851+0000 7fda55e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612cb141100 0x5612cb163020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:59:20.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.851+0000 7fda55e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612cb141100 0x5612cb163020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44234/0 (socket says 172.21.15.86:44234) 2024-03-23T17:59:20.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.851+0000 7fda55e00700 1 -- 172.21.15.86:0/822413091 learned_addr learned my addr 172.21.15.86:0/822413091 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:59:20.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.851+0000 7fda56800700 1 --2- 172.21.15.86:0/822413091 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5612cb14c8d0 0x5612cb13cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:59:20.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.851+0000 7fda55400700 1 --2- 172.21.15.86:0/822413091 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612cb141a20 0x5612cb1535a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:59:20.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.851+0000 7fda55e00700 1 -- 172.21.15.86:0/822413091 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5612cb14c8d0 msgr2=0x5612cb13cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:59:20.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.851+0000 7fda55e00700 1 --2- 172.21.15.86:0/822413091 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5612cb14c8d0 0x5612cb13cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.851+0000 7fda55e00700 1 -- 172.21.15.86:0/822413091 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612cb141a20 msgr2=0x5612cb1535a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:59:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.851+0000 7fda55e00700 1 --2- 172.21.15.86:0/822413091 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612cb141a20 0x5612cb1535a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.851+0000 7fda55e00700 1 -- 172.21.15.86:0/822413091 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5612caec1b00 con 0x5612cb141100 2024-03-23T17:59:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.851+0000 7fda55e00700 1 --2- 172.21.15.86:0/822413091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612cb141100 0x5612cb163020 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fda480963a0 tx=0x7fda48004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f2a8f1529877977 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:59:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.852+0000 7fda43a00700 1 -- 172.21.15.86:0/822413091 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda480b1070 con 0x5612cb141100 2024-03-23T17:59:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.852+0000 7fda43a00700 1 -- 172.21.15.86:0/822413091 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5612cb054320 con 0x5612cb141100 2024-03-23T17:59:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.852+0000 7fda43a00700 1 -- 172.21.15.86:0/822413091 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda48099390 con 0x5612cb141100 2024-03-23T17:59:20.856 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.852+0000 7fda70698b80 1 -- 172.21.15.86:0/822413091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612cb141100 msgr2=0x5612cb163020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:59:20.856 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.852+0000 7fda70698b80 1 --2- 172.21.15.86:0/822413091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612cb141100 0x5612cb163020 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fda480963a0 tx=0x7fda48004750 comp rx=0 tx=0).stop 2024-03-23T17:59:20.856 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.852+0000 7fda70698b80 1 -- 172.21.15.86:0/822413091 shutdown_connections 2024-03-23T17:59:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.852+0000 7fda70698b80 1 --2- 172.21.15.86:0/822413091 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5612cb14c8d0 0x5612cb13cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.852+0000 7fda70698b80 1 --2- 172.21.15.86:0/822413091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612cb141100 0x5612cb163020 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.852+0000 7fda70698b80 1 --2- 172.21.15.86:0/822413091 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612cb141a20 0x5612cb1535a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.852+0000 7fda70698b80 1 -- 172.21.15.86:0/822413091 >> 172.21.15.86:0/822413091 conn(0x5612cb0a1640 msgr2=0x5612cb147ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:59:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.852+0000 7fda70698b80 1 -- 172.21.15.86:0/822413091 shutdown_connections 2024-03-23T17:59:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.852+0000 7fda70698b80 1 -- 172.21.15.86:0/822413091 wait complete. 2024-03-23T17:59:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.853+0000 7fda70698b80 1 -- start start 2024-03-23T17:59:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.853+0000 7fda70698b80 1 Processor -- start 2024-03-23T17:59:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.853+0000 7fda70698b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612cb141100 0x5612cb151ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:59:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.853+0000 7fda70698b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612cb141a20 0x5612cb1520e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:59:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.853+0000 7fda70698b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5612cb14c8d0 0x5612cb152620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:59:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.853+0000 7fda70698b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612cb1539e0 con 0x5612cb141a20 2024-03-23T17:59:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.853+0000 7fda70698b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612cb1addd0 con 0x5612cb141100 2024-03-23T17:59:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.853+0000 7fda70698b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612cb1adf90 con 0x5612cb14c8d0 2024-03-23T17:59:20.858 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.853+0000 7fda70698b80 10 client.0 _ll_register_callbacks cb 0x5612cb039410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:59:20.858 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.854+0000 7fda55400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612cb141a20 0x5612cb1520e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:59:20.858 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.854+0000 7fda55e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612cb141100 0x5612cb151ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:59:20.858 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.854+0000 7fda55e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612cb141100 0x5612cb151ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:55926/0 (socket says 172.21.15.86:55926) 2024-03-23T17:59:20.858 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.854+0000 7fda55e00700 1 -- 172.21.15.86:0/3892234029 learned_addr learned my addr 172.21.15.86:0/3892234029 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:59:20.858 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.854+0000 7fda56800700 1 --2- 172.21.15.86:0/3892234029 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5612cb14c8d0 0x5612cb152620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:59:20.858 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.854+0000 7fda55e00700 1 -- 172.21.15.86:0/3892234029 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5612cb14c8d0 msgr2=0x5612cb152620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:59:20.858 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.854+0000 7fda55e00700 1 --2- 172.21.15.86:0/3892234029 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5612cb14c8d0 0x5612cb152620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:20.858 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.854+0000 7fda55e00700 1 -- 172.21.15.86:0/3892234029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612cb141a20 msgr2=0x5612cb1520e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:59:20.858 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.854+0000 7fda55e00700 1 --2- 172.21.15.86:0/3892234029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612cb141a20 0x5612cb1520e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:20.858 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.854+0000 7fda55e00700 1 -- 172.21.15.86:0/3892234029 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda581a6240 con 0x5612cb141100 2024-03-23T17:59:20.858 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.854+0000 7fda56800700 1 --2- 172.21.15.86:0/3892234029 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5612cb14c8d0 0x5612cb152620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:59:20.858 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.854+0000 7fda55400700 1 --2- 172.21.15.86:0/3892234029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612cb141a20 0x5612cb1520e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:59:20.859 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.854+0000 7fda55e00700 1 --2- 172.21.15.86:0/3892234029 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612cb141100 0x5612cb151ba0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fda48020000 tx=0x7fda48009550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f06a2209ce38b15 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:59:20.859 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.854+0000 7fda37400700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T17:59:20.859 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.854+0000 7fda37400700 1 -- 172.21.15.86:0/3892234029 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda480b1070 con 0x5612cb141100 2024-03-23T17:59:20.859 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.854+0000 7fda37400700 1 -- 172.21.15.86:0/3892234029 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fda4800cdf0 con 0x5612cb141100 2024-03-23T17:59:20.859 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.854+0000 7fda70698b80 1 -- 172.21.15.86:0/3892234029 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fda48099ea0 con 0x5612cb141100 2024-03-23T17:59:20.859 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.855+0000 7fda37400700 1 -- 172.21.15.86:0/3892234029 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda480961e0 con 0x5612cb141100 2024-03-23T17:59:20.859 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.855+0000 7fda70698b80 10 client.26735 fetch_fsmap learned FSMap version 40 2024-03-23T17:59:20.859 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.855+0000 7fda70698b80 1 -- 172.21.15.86:0/3892234029 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5612cb059e40 con 0x5612cb141100 2024-03-23T17:59:20.859 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.855+0000 7fda37400700 1 -- 172.21.15.86:0/3892234029 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda48099ea0 con 0x5612cb141100 2024-03-23T17:59:20.859 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.855+0000 7fda37400700 1 -- 172.21.15.86:0/3892234029 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 40) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fda48098e00 con 0x5612cb141100 2024-03-23T17:59:20.859 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.855+0000 7fda70698b80 10 client.26735 fetch_fsmap finished waiting for FSMap version 40 2024-03-23T17:59:20.859 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.855+0000 7fda70698b80 10 client.26735 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:59:20.859 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.855+0000 7fda70698b80 20 client.26735 populate_metadata read hostname 'smithi086' 2024-03-23T17:59:20.859 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.855+0000 7fda70698b80 1 --2- 172.21.15.86:0/3892234029 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5612cb1af270 0x5612cb1b1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:59:20.859 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.855+0000 7fda70698b80 4 client.26735 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T17:59:20.860 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.855+0000 7fda70698b80 1 -- 172.21.15.86:0/3892234029 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5612caee2c90 con 0x5612cb1af270 2024-03-23T17:59:20.860 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.855+0000 7fda55400700 1 --2- 172.21.15.86:0/3892234029 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5612cb1af270 0x5612cb1b1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:59:20.860 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.856+0000 7fda37400700 10 client.26735 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:59:20.860 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.856+0000 7fda55400700 1 --2- 172.21.15.86:0/3892234029 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5612cb1af270 0x5612cb1b1660 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cf3e7184dc40e96 server_cookie=1eb42c4412bc3a73 in_seq=0 out_seq=0 2024-03-23T17:59:20.898 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.897+0000 7fda37400700 1 -- 172.21.15.86:0/3892234029 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5612caee2c90 con 0x5612cb1af270 2024-03-23T17:59:20.898 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.897+0000 7fda37400700 10 client.26735 handle_command_reply: tid=0 2024-03-23T17:59:20.898 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.897+0000 7fda70698b80 1 client.26735 shutdown 2024-03-23T17:59:20.898 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.897+0000 7fda70698b80 1 -- 172.21.15.86:0/3892234029 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612cb141100 msgr2=0x5612cb151ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:59:20.898 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.897+0000 7fda70698b80 1 --2- 172.21.15.86:0/3892234029 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612cb141100 0x5612cb151ba0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fda48020000 tx=0x7fda48009550 comp rx=0 tx=0).stop 2024-03-23T17:59:20.899 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.897+0000 7fda70698b80 1 -- 172.21.15.86:0/3892234029 shutdown_connections 2024-03-23T17:59:20.899 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.897+0000 7fda70698b80 1 --2- 172.21.15.86:0/3892234029 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5612cb1af270 0x5612cb1b1660 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:20.899 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.897+0000 7fda70698b80 1 --2- 172.21.15.86:0/3892234029 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5612cb14c8d0 0x5612cb152620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:20.899 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.897+0000 7fda70698b80 1 --2- 172.21.15.86:0/3892234029 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612cb141100 0x5612cb151ba0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:20.899 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.897+0000 7fda70698b80 1 --2- 172.21.15.86:0/3892234029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612cb141a20 0x5612cb1520e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:20.899 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.897+0000 7fda37400700 0 client.26735 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:59:20.899 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.897+0000 7fda70698b80 1 -- 172.21.15.86:0/3892234029 >> 172.21.15.86:0/3892234029 conn(0x5612cb0a1640 msgr2=0x5612cb13cd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:59:20.899 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.898+0000 7fda70698b80 1 -- 172.21.15.86:0/3892234029 shutdown_connections 2024-03-23T17:59:20.899 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.898+0000 7fda70698b80 1 -- 172.21.15.86:0/3892234029 wait complete. 2024-03-23T17:59:20.899 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.898+0000 7fda70698b80 20 client.26735 trim_cache size 0 max 16384 2024-03-23T17:59:20.899 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-03-23T17:59:20.900 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-03-23T17:59:20.900 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-03-23T17:59:20.900 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-03-23T17:59:20.900 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.899+0000 7fda5cc00700 1 -- 172.21.15.86:0/3047453471 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fda3c0a1290 msgr2=0x7fda3c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:59:20.900 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.899+0000 7fda5cc00700 1 --2- 172.21.15.86:0/3047453471 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fda3c0a1290 0x7fda3c0a3750 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fda580ff270 tx=0x7fda44007f40 comp rx=0 tx=0).stop 2024-03-23T17:59:20.901 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.899+0000 7fda5cc00700 1 -- 172.21.15.86:0/3047453471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda58100f10 msgr2=0x7fda5819deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:59:20.901 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.899+0000 7fda5cc00700 1 --2- 172.21.15.86:0/3047453471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda58100f10 0x7fda5819deb0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fda4800e9c0 tx=0x7fda4800ecd0 comp rx=0 tx=0).stop 2024-03-23T17:59:20.901 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.899+0000 7fda5cc00700 1 -- 172.21.15.86:0/3047453471 shutdown_connections 2024-03-23T17:59:20.901 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.899+0000 7fda5cc00700 1 --2- 172.21.15.86:0/3047453471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda581991e0 0x7fda58199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:20.901 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.899+0000 7fda5cc00700 1 --2- 172.21.15.86:0/3047453471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda58100f10 0x7fda5819deb0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:20.901 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.899+0000 7fda5cc00700 1 --2- 172.21.15.86:0/3047453471 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fda3c0a1290 0x7fda3c0a3750 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:20.901 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.899+0000 7fda5cc00700 1 --2- 172.21.15.86:0/3047453471 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda58106030 0x7fda5819e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:20.901 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.899+0000 7fda5cc00700 1 -- 172.21.15.86:0/3047453471 >> 172.21.15.86:0/3047453471 conn(0x7fda580fc770 msgr2=0x7fda580feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:59:20.901 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.899+0000 7fda5cc00700 1 -- 172.21.15.86:0/3047453471 shutdown_connections 2024-03-23T17:59:20.901 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:20.899+0000 7fda5cc00700 1 -- 172.21.15.86:0/3047453471 wait complete. 2024-03-23T17:59:20.914 INFO:tasks.workunit.client.0.smithi086.stdout: 1 460030 5.95 MB/sec execute 345 sec latency 1958.847 ms 2024-03-23T17:59:21.915 INFO:tasks.workunit.client.0.smithi086.stdout: 1 461734 5.96 MB/sec execute 346 sec latency 1961.754 ms 2024-03-23T17:59:22.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:59:21 smithi086 ceph-mon[38221]: pgmap v270: 97 pgs: 97 active+clean; 165 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 147 op/s 2024-03-23T17:59:22.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:59:21 smithi086 ceph-mon[38221]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T17:59:22.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:59:21 smithi092 ceph-mon[79089]: pgmap v270: 97 pgs: 97 active+clean; 165 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 147 op/s 2024-03-23T17:59:22.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:59:21 smithi092 ceph-mon[79089]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T17:59:22.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:59:21 smithi152 ceph-mon[47949]: pgmap v270: 97 pgs: 97 active+clean; 165 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 147 op/s 2024-03-23T17:59:22.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:59:21 smithi152 ceph-mon[47949]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T17:59:22.915 INFO:tasks.workunit.client.0.smithi086.stdout: 1 462501 5.96 MB/sec execute 347 sec latency 287.792 ms 2024-03-23T17:59:23.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:59:22 smithi086 ceph-mon[38221]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T17:59:23.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:59:22 smithi092 ceph-mon[79089]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T17:59:23.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:59:22 smithi152 ceph-mon[47949]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T17:59:23.915 INFO:tasks.workunit.client.0.smithi086.stdout: 1 462501 5.94 MB/sec execute 348 sec latency 1287.919 ms 2024-03-23T17:59:24.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:59:23 smithi086 ceph-mon[38221]: pgmap v271: 97 pgs: 97 active+clean; 165 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 147 op/s 2024-03-23T17:59:24.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:59:23 smithi092 ceph-mon[79089]: pgmap v271: 97 pgs: 97 active+clean; 165 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 147 op/s 2024-03-23T17:59:24.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:59:23 smithi152 ceph-mon[47949]: pgmap v271: 97 pgs: 97 active+clean; 165 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 147 op/s 2024-03-23T17:59:24.918 INFO:tasks.workunit.client.0.smithi086.stdout: 1 462501 5.92 MB/sec execute 349 sec latency 2288.001 ms 2024-03-23T17:59:25.915 INFO:tasks.workunit.client.0.smithi086.stdout: 1 462501 5.90 MB/sec execute 350 sec latency 3288.062 ms 2024-03-23T17:59:26.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:59:25 smithi086 ceph-mon[38221]: pgmap v272: 97 pgs: 97 active+clean; 164 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 1.5 MiB/s wr, 144 op/s 2024-03-23T17:59:26.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:59:25 smithi092 ceph-mon[79089]: pgmap v272: 97 pgs: 97 active+clean; 164 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 1.5 MiB/s wr, 144 op/s 2024-03-23T17:59:26.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:59:25 smithi152 ceph-mon[47949]: pgmap v272: 97 pgs: 97 active+clean; 164 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 1.5 MiB/s wr, 144 op/s 2024-03-23T17:59:26.915 INFO:tasks.workunit.client.0.smithi086.stdout: 1 462501 5.89 MB/sec execute 351 sec latency 4288.122 ms 2024-03-23T17:59:27.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:59:26 smithi086 ceph-mon[38221]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T17:59:27.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:59:26 smithi092 ceph-mon[79089]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T17:59:27.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:59:26 smithi152 ceph-mon[47949]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T17:59:27.915 INFO:tasks.workunit.client.0.smithi086.stdout: 1 462501 5.87 MB/sec execute 352 sec latency 5288.252 ms 2024-03-23T17:59:28.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:59:27 smithi086 ceph-mon[38221]: pgmap v273: 97 pgs: 97 active+clean; 164 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 1.4 MiB/s wr, 134 op/s 2024-03-23T17:59:28.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:59:27 smithi092 ceph-mon[79089]: pgmap v273: 97 pgs: 97 active+clean; 164 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 1.4 MiB/s wr, 134 op/s 2024-03-23T17:59:28.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:59:27 smithi152 ceph-mon[47949]: pgmap v273: 97 pgs: 97 active+clean; 164 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 1.4 MiB/s wr, 134 op/s 2024-03-23T17:59:28.915 INFO:tasks.workunit.client.0.smithi086.stdout: 1 462501 5.85 MB/sec execute 353 sec latency 6288.386 ms 2024-03-23T17:59:29.916 INFO:tasks.workunit.client.0.smithi086.stdout: 1 462501 5.84 MB/sec execute 354 sec latency 7288.544 ms 2024-03-23T17:59:30.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:59:29 smithi086 ceph-mon[38221]: pgmap v274: 97 pgs: 97 active+clean; 164 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 1.3 MiB/s wr, 120 op/s 2024-03-23T17:59:30.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:59:29 smithi092 ceph-mon[79089]: pgmap v274: 97 pgs: 97 active+clean; 164 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 1.3 MiB/s wr, 120 op/s 2024-03-23T17:59:30.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:59:29 smithi152 ceph-mon[47949]: pgmap v274: 97 pgs: 97 active+clean; 164 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 1.3 MiB/s wr, 120 op/s 2024-03-23T17:59:30.915 INFO:tasks.workunit.client.0.smithi086.stdout: 1 462963 5.83 MB/sec execute 355 sec latency 8008.540 ms 2024-03-23T17:59:31.916 INFO:tasks.workunit.client.0.smithi086.stdout: 1 465109 5.84 MB/sec execute 356 sec latency 14.981 ms 2024-03-23T17:59:32.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:59:31 smithi086 ceph-mon[38221]: pgmap v275: 97 pgs: 97 active+clean; 164 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 1.5 MiB/s wr, 139 op/s 2024-03-23T17:59:32.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:59:31 smithi092 ceph-mon[79089]: pgmap v275: 97 pgs: 97 active+clean; 164 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 1.5 MiB/s wr, 139 op/s 2024-03-23T17:59:32.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:59:31 smithi152 ceph-mon[47949]: pgmap v275: 97 pgs: 97 active+clean; 164 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 1.5 MiB/s wr, 139 op/s 2024-03-23T17:59:32.916 INFO:tasks.workunit.client.0.smithi086.stdout: 1 466505 5.85 MB/sec execute 357 sec latency 16.305 ms 2024-03-23T17:59:33.916 INFO:tasks.workunit.client.0.smithi086.stdout: 1 468618 5.86 MB/sec execute 358 sec latency 17.559 ms 2024-03-23T17:59:34.060 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.059+0000 7f7792c00700 1 -- 172.21.15.86:0/60906027 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f778c100ef0 msgr2=0x7f778c1012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:59:34.060 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.059+0000 7f7792c00700 1 --2- 172.21.15.86:0/60906027 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f778c100ef0 0x7f778c1012d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f777c009990 tx=0x7f777c009ca0 comp rx=0 tx=0).stop 2024-03-23T17:59:34.060 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.059+0000 7f7792c00700 1 -- 172.21.15.86:0/60906027 shutdown_connections 2024-03-23T17:59:34.060 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.059+0000 7f7792c00700 1 --2- 172.21.15.86:0/60906027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f778c106010 0x7f778c10db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:34.060 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.059+0000 7f7792c00700 1 --2- 172.21.15.86:0/60906027 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f778c100ef0 0x7f778c1012d0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:34.060 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.059+0000 7f7792c00700 1 --2- 172.21.15.86:0/60906027 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f778c1018a0 0x7f778c1058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:34.061 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.059+0000 7f7792c00700 1 -- 172.21.15.86:0/60906027 >> 172.21.15.86:0/60906027 conn(0x7f778c0fc750 msgr2=0x7f778c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:59:34.061 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.059+0000 7f7792c00700 1 -- 172.21.15.86:0/60906027 shutdown_connections 2024-03-23T17:59:34.061 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.059+0000 7f7792c00700 1 -- 172.21.15.86:0/60906027 wait complete. 2024-03-23T17:59:34.063 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.061+0000 7f7792c00700 1 Processor -- start 2024-03-23T17:59:34.063 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.061+0000 7f7792c00700 1 -- start start 2024-03-23T17:59:34.063 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.061+0000 7f7792c00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f778c100ef0 0x7f778c199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:59:34.063 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.061+0000 7f7792c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f778c106010 0x7f778c199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:59:34.064 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.061+0000 7f7792c00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f778c19dca0 0x7f778c19e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:59:34.064 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.062+0000 7f7792c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f778c19e720 con 0x7f778c106010 2024-03-23T17:59:34.064 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.062+0000 7f7792c00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f778c19e890 con 0x7f778c19dca0 2024-03-23T17:59:34.064 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.062+0000 7f7792c00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f778c19ea50 con 0x7f778c100ef0 2024-03-23T17:59:34.064 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.062+0000 7f778be00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f778c100ef0 0x7f778c199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:59:34.064 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.062+0000 7f778be00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f778c100ef0 0x7f778c199410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:49722/0 (socket says 172.21.15.86:49722) 2024-03-23T17:59:34.064 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.062+0000 7f778be00700 1 -- 172.21.15.86:0/713147551 learned_addr learned my addr 172.21.15.86:0/713147551 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:59:34.064 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.062+0000 7f778b400700 1 --2- 172.21.15.86:0/713147551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f778c106010 0x7f778c199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:59:34.064 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.062+0000 7f7790c00700 1 --2- 172.21.15.86:0/713147551 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f778c19dca0 0x7f778c19e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:59:34.064 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.062+0000 7f778be00700 1 -- 172.21.15.86:0/713147551 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f778c19dca0 msgr2=0x7f778c19e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:59:34.064 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.062+0000 7f778be00700 1 --2- 172.21.15.86:0/713147551 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f778c19dca0 0x7f778c19e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:34.064 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.062+0000 7f778be00700 1 -- 172.21.15.86:0/713147551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f778c106010 msgr2=0x7f778c199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:59:34.064 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.062+0000 7f778be00700 1 --2- 172.21.15.86:0/713147551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f778c106010 0x7f778c199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:34.064 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.062+0000 7f778be00700 1 -- 172.21.15.86:0/713147551 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f777c0095f0 con 0x7f778c100ef0 2024-03-23T17:59:34.065 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.062+0000 7f778b400700 1 --2- 172.21.15.86:0/713147551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f778c106010 0x7f778c199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T17:59:34.065 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.062+0000 7f778be00700 1 --2- 172.21.15.86:0/713147551 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f778c100ef0 0x7f778c199410 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f777c000c00 tx=0x7f777c005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:59:34.065 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.063+0000 7f7788c00700 1 -- 172.21.15.86:0/713147551 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f777c01d070 con 0x7f778c100ef0 2024-03-23T17:59:34.065 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.063+0000 7f7788c00700 1 -- 172.21.15.86:0/713147551 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f777c022470 con 0x7f778c100ef0 2024-03-23T17:59:34.065 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.063+0000 7f7792c00700 1 -- 172.21.15.86:0/713147551 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f778c1a5f50 con 0x7f778c100ef0 2024-03-23T17:59:34.065 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.063+0000 7f7788c00700 1 -- 172.21.15.86:0/713147551 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f777c017650 con 0x7f778c100ef0 2024-03-23T17:59:34.065 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.063+0000 7f7792c00700 1 -- 172.21.15.86:0/713147551 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f778c1a6370 con 0x7f778c100ef0 2024-03-23T17:59:34.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.066+0000 7f7788c00700 1 -- 172.21.15.86:0/713147551 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f777c0225e0 con 0x7f778c100ef0 2024-03-23T17:59:34.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.067+0000 7f7788c00700 1 --2- 172.21.15.86:0/713147551 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f77700a1310 0x7f77700a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:59:34.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.067+0000 7f7788c00700 1 -- 172.21.15.86:0/713147551 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f777c0cdf40 con 0x7f778c100ef0 2024-03-23T17:59:34.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.067+0000 7f77a6729b80 1 Processor -- start 2024-03-23T17:59:34.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.068+0000 7f778b400700 1 --2- 172.21.15.86:0/713147551 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f77700a1310 0x7f77700a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:59:34.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.068+0000 7f77a6729b80 1 -- start start 2024-03-23T17:59:34.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.068+0000 7f77a6729b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c40281c100 0x55c40281c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:59:34.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.068+0000 7f77a6729b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c40281ca20 0x55c402827390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:59:34.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.068+0000 7f77a6729b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4028278d0 0x55c402829cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:59:34.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.068+0000 7f77a6729b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c402775930 con 0x55c40281c100 2024-03-23T17:59:34.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.068+0000 7f77a6729b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4027732f0 con 0x55c4028278d0 2024-03-23T17:59:34.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.068+0000 7f77a6729b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c40276ebe0 con 0x55c40281ca20 2024-03-23T17:59:34.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.068+0000 7f778be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c40281c100 0x55c40281c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:59:34.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.069+0000 7f778b400700 1 --2- 172.21.15.86:0/713147551 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f77700a1310 0x7f77700a37d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f778c19a910 tx=0x7f7778009260 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:59:34.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.069+0000 7f778be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c40281c100 0x55c40281c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43368/0 (socket says 172.21.15.86:43368) 2024-03-23T17:59:34.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.069+0000 7f778be00700 1 -- 172.21.15.86:0/2676318183 learned_addr learned my addr 172.21.15.86:0/2676318183 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:59:34.070 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.069+0000 7f7790c00700 1 --2- 172.21.15.86:0/2676318183 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4028278d0 0x55c402829cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:59:34.070 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.069+0000 7f778b400700 1 --2- 172.21.15.86:0/2676318183 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c40281ca20 0x55c402827390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:59:34.071 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.069+0000 7f778be00700 1 -- 172.21.15.86:0/2676318183 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c40281ca20 msgr2=0x55c402827390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:59:34.071 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.069+0000 7f778be00700 1 --2- 172.21.15.86:0/2676318183 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c40281ca20 0x55c402827390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:34.071 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.069+0000 7f778be00700 1 -- 172.21.15.86:0/2676318183 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4028278d0 msgr2=0x55c402829cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:59:34.071 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.069+0000 7f778be00700 1 --2- 172.21.15.86:0/2676318183 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4028278d0 0x55c402829cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:34.071 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.069+0000 7f778be00700 1 -- 172.21.15.86:0/2676318183 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c4025f0350 con 0x55c40281c100 2024-03-23T17:59:34.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.069+0000 7f778be00700 1 --2- 172.21.15.86:0/2676318183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c40281c100 0x55c40281c4e0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f777c097c90 tx=0x7f777c005860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4a8162cbbf33d1a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:59:34.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.069+0000 7f7781200700 1 -- 172.21.15.86:0/2676318183 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f777c09ab80 con 0x55c40281c100 2024-03-23T17:59:34.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.070+0000 7f7781200700 1 -- 172.21.15.86:0/2676318183 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f777c09b920 con 0x55c40281c100 2024-03-23T17:59:34.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.070+0000 7f7781200700 1 -- 172.21.15.86:0/2676318183 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f777c0a10c0 con 0x55c40281c100 2024-03-23T17:59:34.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.070+0000 7f77a6729b80 1 -- 172.21.15.86:0/2676318183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c40281c100 msgr2=0x55c40281c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:59:34.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.070+0000 7f77a6729b80 1 --2- 172.21.15.86:0/2676318183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c40281c100 0x55c40281c4e0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f777c097c90 tx=0x7f777c005860 comp rx=0 tx=0).stop 2024-03-23T17:59:34.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.070+0000 7f77a6729b80 1 -- 172.21.15.86:0/2676318183 shutdown_connections 2024-03-23T17:59:34.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.070+0000 7f77a6729b80 1 --2- 172.21.15.86:0/2676318183 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4028278d0 0x55c402829cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:34.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.070+0000 7f77a6729b80 1 --2- 172.21.15.86:0/2676318183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c40281c100 0x55c40281c4e0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:34.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.070+0000 7f77a6729b80 1 --2- 172.21.15.86:0/2676318183 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c40281ca20 0x55c402827390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:34.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.070+0000 7f77a6729b80 1 -- 172.21.15.86:0/2676318183 >> 172.21.15.86:0/2676318183 conn(0x55c40277c640 msgr2=0x55c402818520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:59:34.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.070+0000 7f77a6729b80 1 -- 172.21.15.86:0/2676318183 shutdown_connections 2024-03-23T17:59:34.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.071+0000 7f77a6729b80 1 -- 172.21.15.86:0/2676318183 wait complete. 2024-03-23T17:59:34.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.071+0000 7f77a6729b80 1 -- start start 2024-03-23T17:59:34.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.071+0000 7f77a6729b80 1 Processor -- start 2024-03-23T17:59:34.075 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.072+0000 7f77a6729b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c40281c100 0x55c4028272b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:59:34.075 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.072+0000 7f77a6729b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c40281ca20 0x55c402824530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:59:34.075 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.072+0000 7f77a6729b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4028278d0 0x55c402824b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:59:34.075 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.072+0000 7f77a6729b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c40270ad20 con 0x55c40281c100 2024-03-23T17:59:34.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.072+0000 7f77a6729b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c40270a9c0 con 0x55c4028278d0 2024-03-23T17:59:34.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.072+0000 7f77a6729b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4025f2640 con 0x55c40281ca20 2024-03-23T17:59:34.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.072+0000 7f77a6729b80 10 client.0 _ll_register_callbacks cb 0x55c402714410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:59:34.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.072+0000 7f7790c00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4028278d0 0x55c402824b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:59:34.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.072+0000 7f778be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c40281c100 0x55c4028272b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:59:34.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.072+0000 7f7790c00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4028278d0 0x55c402824b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:36504/0 (socket says 172.21.15.86:36504) 2024-03-23T17:59:34.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.072+0000 7f7790c00700 1 -- 172.21.15.86:0/1348750882 learned_addr learned my addr 172.21.15.86:0/1348750882 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:59:34.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.072+0000 7f778b400700 1 --2- 172.21.15.86:0/1348750882 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c40281ca20 0x55c402824530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:59:34.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.072+0000 7f7790c00700 1 -- 172.21.15.86:0/1348750882 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c40281ca20 msgr2=0x55c402824530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:59:34.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.072+0000 7f7790c00700 1 --2- 172.21.15.86:0/1348750882 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c40281ca20 0x55c402824530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:34.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.072+0000 7f7790c00700 1 -- 172.21.15.86:0/1348750882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c40281c100 msgr2=0x55c4028272b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:59:34.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.072+0000 7f7790c00700 1 --2- 172.21.15.86:0/1348750882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c40281c100 0x55c4028272b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:34.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.073+0000 7f7790c00700 1 -- 172.21.15.86:0/1348750882 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f778c1a6370 con 0x55c4028278d0 2024-03-23T17:59:34.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.073+0000 7f7790c00700 1 --2- 172.21.15.86:0/1348750882 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4028278d0 0x55c402824b20 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7784011ef0 tx=0x7f7784014600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31f2b15a20e9088c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:59:34.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.073+0000 7f7775600700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T17:59:34.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.073+0000 7f7775600700 1 -- 172.21.15.86:0/1348750882 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f778400e040 con 0x55c4028278d0 2024-03-23T17:59:34.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.073+0000 7f7775600700 1 -- 172.21.15.86:0/1348750882 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f77840093a0 con 0x55c4028278d0 2024-03-23T17:59:34.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.073+0000 7f77a6729b80 1 -- 172.21.15.86:0/1348750882 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c40272f320 con 0x55c4028278d0 2024-03-23T17:59:34.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.074+0000 7f7775600700 1 -- 172.21.15.86:0/1348750882 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7784014d90 con 0x55c4028278d0 2024-03-23T17:59:34.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.074+0000 7f77a6729b80 10 client.26747 fetch_fsmap learned FSMap version 43 2024-03-23T17:59:34.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.074+0000 7f77a6729b80 1 -- 172.21.15.86:0/1348750882 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55c40259cb00 con 0x55c4028278d0 2024-03-23T17:59:34.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.074+0000 7f7775600700 1 -- 172.21.15.86:0/1348750882 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c40272f320 con 0x55c4028278d0 2024-03-23T17:59:34.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.074+0000 7f7775600700 1 -- 172.21.15.86:0/1348750882 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 43) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f778401c020 con 0x55c4028278d0 2024-03-23T17:59:34.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.074+0000 7f77a6729b80 10 client.26747 fetch_fsmap finished waiting for FSMap version 43 2024-03-23T17:59:34.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.074+0000 7f77a6729b80 10 client.26747 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:59:34.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.074+0000 7f77a6729b80 20 client.26747 populate_metadata read hostname 'smithi086' 2024-03-23T17:59:34.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.074+0000 7f77a6729b80 1 --2- 172.21.15.86:0/1348750882 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55c402837320 0x55c402839740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:59:34.078 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.074+0000 7f77a6729b80 4 client.26747 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:59:34.078 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.074+0000 7f77a6729b80 1 -- 172.21.15.86:0/1348750882 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c4027676a0 con 0x55c402837320 2024-03-23T17:59:34.078 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.075+0000 7f778be00700 1 --2- 172.21.15.86:0/1348750882 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55c402837320 0x55c402839740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:59:34.079 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.076+0000 7f778be00700 1 --2- 172.21.15.86:0/1348750882 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55c402837320 0x55c402839740 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fd1035427209ccf server_cookie=70fe12bb2d08d733 in_seq=0 out_seq=0 2024-03-23T17:59:34.079 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.076+0000 7f7775600700 10 client.26747 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:59:34.079 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.077+0000 7f7775600700 1 -- 172.21.15.86:0/1348750882 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55c4027676a0 con 0x55c402837320 2024-03-23T17:59:34.079 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.077+0000 7f7775600700 10 client.26747 handle_command_reply: tid=0 2024-03-23T17:59:34.079 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.077+0000 7f77a6729b80 1 client.26747 shutdown 2024-03-23T17:59:34.079 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.078+0000 7f77a6729b80 1 -- 172.21.15.86:0/1348750882 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4028278d0 msgr2=0x55c402824b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:59:34.079 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.078+0000 7f77a6729b80 1 --2- 172.21.15.86:0/1348750882 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4028278d0 0x55c402824b20 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7784011ef0 tx=0x7f7784014600 comp rx=0 tx=0).stop 2024-03-23T17:59:34.079 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.078+0000 7f77a6729b80 1 -- 172.21.15.86:0/1348750882 shutdown_connections 2024-03-23T17:59:34.079 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.078+0000 7f77a6729b80 1 --2- 172.21.15.86:0/1348750882 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55c402837320 0x55c402839740 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:34.080 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.078+0000 7f77a6729b80 1 --2- 172.21.15.86:0/1348750882 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c4028278d0 0x55c402824b20 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:34.080 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.078+0000 7f77a6729b80 1 --2- 172.21.15.86:0/1348750882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c40281c100 0x55c4028272b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:34.080 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.078+0000 7f77a6729b80 1 --2- 172.21.15.86:0/1348750882 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c40281ca20 0x55c402824530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:34.080 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.078+0000 7f77a6729b80 1 -- 172.21.15.86:0/1348750882 >> 172.21.15.86:0/1348750882 conn(0x55c40277c640 msgr2=0x55c40281e080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:59:34.080 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.078+0000 7f7775600700 0 client.26747 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:59:34.080 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.079+0000 7f77a6729b80 1 -- 172.21.15.86:0/1348750882 shutdown_connections 2024-03-23T17:59:34.080 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.079+0000 7f77a6729b80 1 -- 172.21.15.86:0/1348750882 wait complete. 2024-03-23T17:59:34.080 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.079+0000 7f77a6729b80 20 client.26747 trim_cache size 0 max 16384 2024-03-23T17:59:34.094 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.094+0000 7f77a6729b80 1 Processor -- start 2024-03-23T17:59:34.095 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.094+0000 7f77a6729b80 1 -- start start 2024-03-23T17:59:34.095 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.095+0000 7f77a6729b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c40277c640 0x55c402811c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:59:34.095 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.095+0000 7f77a6729b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c40281c100 0x55c40282e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:59:34.095 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.095+0000 7f77a6729b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c40281ca20 0x55c4028234c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:59:34.095 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.095+0000 7f77a6729b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c402825190 con 0x55c40281c100 2024-03-23T17:59:34.095 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.095+0000 7f77a6729b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c40281ce30 con 0x55c40277c640 2024-03-23T17:59:34.095 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.095+0000 7f77a6729b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c402822ac0 con 0x55c40281ca20 2024-03-23T17:59:34.096 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.095+0000 7f778b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c40281c100 0x55c40282e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:59:34.096 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.095+0000 7f778be00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c40277c640 0x55c402811c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:59:34.096 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.095+0000 7f7790c00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c40281ca20 0x55c4028234c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:59:34.096 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.095+0000 7f778b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c40281c100 0x55c40282e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43390/0 (socket says 172.21.15.86:43390) 2024-03-23T17:59:34.096 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.095+0000 7f778b400700 1 -- 172.21.15.86:0/3050211491 learned_addr learned my addr 172.21.15.86:0/3050211491 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:59:34.096 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.095+0000 7f7790c00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c40281ca20 0x55c4028234c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:49754/0 (socket says 172.21.15.86:49754) 2024-03-23T17:59:34.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.095+0000 7f778be00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c40277c640 0x55c402811c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:36510/0 (socket says 172.21.15.86:36510) 2024-03-23T17:59:34.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.095+0000 7f778be00700 1 -- 172.21.15.86:0/3050211491 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c40281ca20 msgr2=0x55c4028234c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:59:34.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.095+0000 7f778be00700 1 --2- 172.21.15.86:0/3050211491 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c40281ca20 0x55c4028234c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:34.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.095+0000 7f778be00700 1 -- 172.21.15.86:0/3050211491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c40281c100 msgr2=0x55c40282e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:59:34.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.095+0000 7f778be00700 1 --2- 172.21.15.86:0/3050211491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c40281c100 0x55c40282e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:34.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.095+0000 7f778be00700 1 -- 172.21.15.86:0/3050211491 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c402734e40 con 0x55c40277c640 2024-03-23T17:59:34.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.096+0000 7f778be00700 1 --2- 172.21.15.86:0/3050211491 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c40277c640 0x55c402811c70 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f777c009580 tx=0x7f777c0a1040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7241edb6c3e06ff server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:59:34.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.096+0000 7f7781c00700 1 -- 172.21.15.86:0/3050211491 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f777c09ab80 con 0x55c40277c640 2024-03-23T17:59:34.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.096+0000 7f7781c00700 1 -- 172.21.15.86:0/3050211491 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f777c09f050 con 0x55c40277c640 2024-03-23T17:59:34.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.096+0000 7f7781c00700 1 -- 172.21.15.86:0/3050211491 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f777c09b480 con 0x55c40277c640 2024-03-23T17:59:34.100 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.097+0000 7f77a6729b80 1 -- 172.21.15.86:0/3050211491 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c40277c640 msgr2=0x55c402811c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:59:34.100 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.097+0000 7f77a6729b80 1 --2- 172.21.15.86:0/3050211491 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c40277c640 0x55c402811c70 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f777c009580 tx=0x7f777c0a1040 comp rx=0 tx=0).stop 2024-03-23T17:59:34.100 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.097+0000 7f77a6729b80 1 -- 172.21.15.86:0/3050211491 shutdown_connections 2024-03-23T17:59:34.100 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.097+0000 7f77a6729b80 1 --2- 172.21.15.86:0/3050211491 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c40281ca20 0x55c4028234c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:34.100 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.097+0000 7f77a6729b80 1 --2- 172.21.15.86:0/3050211491 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c40277c640 0x55c402811c70 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:34.100 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.097+0000 7f77a6729b80 1 --2- 172.21.15.86:0/3050211491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c40281c100 0x55c40282e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:34.100 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.097+0000 7f77a6729b80 1 -- 172.21.15.86:0/3050211491 >> 172.21.15.86:0/3050211491 conn(0x55c40281ffb0 msgr2=0x55c402817d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:59:34.101 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.097+0000 7f77a6729b80 1 -- 172.21.15.86:0/3050211491 shutdown_connections 2024-03-23T17:59:34.101 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.097+0000 7f77a6729b80 1 -- 172.21.15.86:0/3050211491 wait complete. 2024-03-23T17:59:34.101 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.097+0000 7f77a6729b80 1 -- start start 2024-03-23T17:59:34.101 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.098+0000 7f77a6729b80 1 Processor -- start 2024-03-23T17:59:34.101 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.098+0000 7f77a6729b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c40277c640 0x55c40282c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:59:34.101 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.098+0000 7f77a6729b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c40281c100 0x55c40282c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:59:34.101 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.098+0000 7f77a6729b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c40281ca20 0x55c402881dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:59:34.101 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.098+0000 7f77a6729b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c402882380 con 0x55c40281c100 2024-03-23T17:59:34.101 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.098+0000 7f77a6729b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4028824f0 con 0x55c40277c640 2024-03-23T17:59:34.101 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.098+0000 7f77a6729b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4028826b0 con 0x55c40281ca20 2024-03-23T17:59:34.101 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.098+0000 7f77a6729b80 10 client.0 _ll_register_callbacks cb 0x55c402714410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:59:34.101 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.098+0000 7f778be00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c40277c640 0x55c40282c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:59:34.101 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.098+0000 7f778be00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c40277c640 0x55c40282c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:36514/0 (socket says 172.21.15.86:36514) 2024-03-23T17:59:34.102 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.098+0000 7f778b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c40281c100 0x55c40282c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:59:34.102 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.098+0000 7f778be00700 1 -- 172.21.15.86:0/3001379863 learned_addr learned my addr 172.21.15.86:0/3001379863 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:59:34.102 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.098+0000 7f7790c00700 1 --2- 172.21.15.86:0/3001379863 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c40281ca20 0x55c402881dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:59:34.102 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.099+0000 7f778be00700 1 -- 172.21.15.86:0/3001379863 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c40281ca20 msgr2=0x55c402881dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:59:34.102 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.099+0000 7f778be00700 1 --2- 172.21.15.86:0/3001379863 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c40281ca20 0x55c402881dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:34.102 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.099+0000 7f778be00700 1 -- 172.21.15.86:0/3001379863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c40281c100 msgr2=0x55c40282c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:59:34.102 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.099+0000 7f778be00700 1 --2- 172.21.15.86:0/3001379863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c40281c100 0x55c40282c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:34.102 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.099+0000 7f778be00700 1 -- 172.21.15.86:0/3001379863 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c4025f0350 con 0x55c40277c640 2024-03-23T17:59:34.102 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.099+0000 7f778be00700 1 --2- 172.21.15.86:0/3001379863 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c40277c640 0x55c40282c490 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f777c097720 tx=0x7f777c09ba90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea2df6e436a3cc40 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:59:34.102 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.099+0000 7f7775600700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T17:59:34.102 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.099+0000 7f7775600700 1 -- 172.21.15.86:0/3001379863 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f777c09ab80 con 0x55c40277c640 2024-03-23T17:59:34.102 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.099+0000 7f7775600700 1 -- 172.21.15.86:0/3001379863 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f777c0b8040 con 0x55c40277c640 2024-03-23T17:59:34.102 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.099+0000 7f77a6729b80 1 -- 172.21.15.86:0/3001379863 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f777c09b920 con 0x55c40277c640 2024-03-23T17:59:34.102 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.099+0000 7f7775600700 1 -- 172.21.15.86:0/3001379863 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f777c0a3610 con 0x55c40277c640 2024-03-23T17:59:34.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.099+0000 7f77a6729b80 10 client.26753 fetch_fsmap learned FSMap version 43 2024-03-23T17:59:34.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.099+0000 7f77a6729b80 1 -- 172.21.15.86:0/3001379863 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55c4025f4c70 con 0x55c40277c640 2024-03-23T17:59:34.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.100+0000 7f7775600700 1 -- 172.21.15.86:0/3001379863 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f777c09b920 con 0x55c40277c640 2024-03-23T17:59:34.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.100+0000 7f7775600700 1 -- 172.21.15.86:0/3001379863 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 43) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f777c0a10b0 con 0x55c40277c640 2024-03-23T17:59:34.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.100+0000 7f77a6729b80 10 client.26753 fetch_fsmap finished waiting for FSMap version 43 2024-03-23T17:59:34.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.100+0000 7f77a6729b80 10 client.26753 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:59:34.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.100+0000 7f77a6729b80 20 client.26753 populate_metadata read hostname 'smithi086' 2024-03-23T17:59:34.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.100+0000 7f77a6729b80 1 --2- 172.21.15.86:0/3001379863 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55c4028278d0 0x55c40288bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:59:34.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.100+0000 7f77a6729b80 4 client.26753 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T17:59:34.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.100+0000 7f77a6729b80 1 -- 172.21.15.86:0/3001379863 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c4025bdc90 con 0x55c4028278d0 2024-03-23T17:59:34.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.100+0000 7f778b400700 1 --2- 172.21.15.86:0/3001379863 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55c4028278d0 0x55c40288bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:59:34.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.100+0000 7f7775600700 10 client.26753 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:59:34.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.100+0000 7f778b400700 1 --2- 172.21.15.86:0/3001379863 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55c4028278d0 0x55c40288bbe0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45038fcbc59e00b6 server_cookie=6cc66838f8b3523 in_seq=0 out_seq=0 2024-03-23T17:59:34.143 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.143+0000 7f7775600700 1 -- 172.21.15.86:0/3001379863 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c4025bdc90 con 0x55c4028278d0 2024-03-23T17:59:34.143 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.143+0000 7f7775600700 10 client.26753 handle_command_reply: tid=0 2024-03-23T17:59:34.143 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.143+0000 7f77a6729b80 1 client.26753 shutdown 2024-03-23T17:59:34.144 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.144+0000 7f77a6729b80 1 -- 172.21.15.86:0/3001379863 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c40277c640 msgr2=0x55c40282c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:59:34.144 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.144+0000 7f77a6729b80 1 --2- 172.21.15.86:0/3001379863 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c40277c640 0x55c40282c490 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f777c097720 tx=0x7f777c09ba90 comp rx=0 tx=0).stop 2024-03-23T17:59:34.144 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.144+0000 7f77a6729b80 1 -- 172.21.15.86:0/3001379863 shutdown_connections 2024-03-23T17:59:34.144 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.144+0000 7f77a6729b80 1 --2- 172.21.15.86:0/3001379863 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55c4028278d0 0x55c40288bbe0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:34.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.144+0000 7f77a6729b80 1 --2- 172.21.15.86:0/3001379863 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c40281ca20 0x55c402881dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:34.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.144+0000 7f77a6729b80 1 --2- 172.21.15.86:0/3001379863 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c40277c640 0x55c40282c490 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:34.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.144+0000 7f77a6729b80 1 --2- 172.21.15.86:0/3001379863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c40281c100 0x55c40282c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:34.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.144+0000 7f77a6729b80 1 -- 172.21.15.86:0/3001379863 >> 172.21.15.86:0/3001379863 conn(0x55c40281ffb0 msgr2=0x55c402818450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:59:34.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.144+0000 7f7775600700 0 client.26753 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:59:34.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.144+0000 7f77a6729b80 1 -- 172.21.15.86:0/3001379863 shutdown_connections 2024-03-23T17:59:34.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.144+0000 7f77a6729b80 1 -- 172.21.15.86:0/3001379863 wait complete. 2024-03-23T17:59:34.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.144+0000 7f77a6729b80 20 client.26753 trim_cache size 0 max 16384 2024-03-23T17:59:34.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-03-23T17:59:34.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-03-23T17:59:34.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-03-23T17:59:34.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-03-23T17:59:34.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.145+0000 7f7792c00700 1 -- 172.21.15.86:0/713147551 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f77700a1310 msgr2=0x7f77700a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:59:34.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.145+0000 7f7792c00700 1 --2- 172.21.15.86:0/713147551 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f77700a1310 0x7f77700a37d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f778c19a910 tx=0x7f7778009260 comp rx=0 tx=0).stop 2024-03-23T17:59:34.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.145+0000 7f7792c00700 1 -- 172.21.15.86:0/713147551 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f778c100ef0 msgr2=0x7f778c199410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:59:34.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.145+0000 7f7792c00700 1 --2- 172.21.15.86:0/713147551 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f778c100ef0 0x7f778c199410 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f777c000c00 tx=0x7f777c005740 comp rx=0 tx=0).stop 2024-03-23T17:59:34.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.145+0000 7f7792c00700 1 -- 172.21.15.86:0/713147551 shutdown_connections 2024-03-23T17:59:34.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.145+0000 7f7792c00700 1 --2- 172.21.15.86:0/713147551 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f778c19dca0 0x7f778c19e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:34.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.145+0000 7f7792c00700 1 --2- 172.21.15.86:0/713147551 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f77700a1310 0x7f77700a37d0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:34.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.145+0000 7f7792c00700 1 --2- 172.21.15.86:0/713147551 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f778c100ef0 0x7f778c199410 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:34.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.145+0000 7f7792c00700 1 --2- 172.21.15.86:0/713147551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f778c106010 0x7f778c199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:34.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.145+0000 7f7792c00700 1 -- 172.21.15.86:0/713147551 >> 172.21.15.86:0/713147551 conn(0x7f778c0fc750 msgr2=0x7f778c0fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:59:34.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.145+0000 7f7792c00700 1 -- 172.21.15.86:0/713147551 shutdown_connections 2024-03-23T17:59:34.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:34.145+0000 7f7792c00700 1 -- 172.21.15.86:0/713147551 wait complete. 2024-03-23T17:59:34.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:59:33 smithi086 ceph-mon[38221]: pgmap v276: 97 pgs: 97 active+clean; 164 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 877 KiB/s wr, 75 op/s 2024-03-23T17:59:34.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:59:33 smithi092 ceph-mon[79089]: pgmap v276: 97 pgs: 97 active+clean; 164 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 877 KiB/s wr, 75 op/s 2024-03-23T17:59:34.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:59:33 smithi152 ceph-mon[47949]: pgmap v276: 97 pgs: 97 active+clean; 164 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 877 KiB/s wr, 75 op/s 2024-03-23T17:59:34.916 INFO:tasks.workunit.client.0.smithi086.stdout: 1 469805 5.86 MB/sec execute 359 sec latency 34.590 ms 2024-03-23T17:59:35.916 INFO:tasks.workunit.client.0.smithi086.stdout: 1 471925 5.88 MB/sec execute 360 sec latency 14.091 ms 2024-03-23T17:59:36.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:59:35 smithi086 ceph-mon[38221]: pgmap v277: 97 pgs: 97 active+clean; 164 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 651 KiB/s rd, 1.7 MiB/s wr, 179 op/s 2024-03-23T17:59:36.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:59:35 smithi092 ceph-mon[79089]: pgmap v277: 97 pgs: 97 active+clean; 164 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 651 KiB/s rd, 1.7 MiB/s wr, 179 op/s 2024-03-23T17:59:36.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:59:35 smithi152 ceph-mon[47949]: pgmap v277: 97 pgs: 97 active+clean; 164 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 651 KiB/s rd, 1.7 MiB/s wr, 179 op/s 2024-03-23T17:59:36.916 INFO:tasks.workunit.client.0.smithi086.stdout: 1 473320 5.89 MB/sec execute 361 sec latency 13.706 ms 2024-03-23T17:59:37.916 INFO:tasks.workunit.client.0.smithi086.stdout: 1 475298 5.90 MB/sec execute 362 sec latency 15.427 ms 2024-03-23T17:59:38.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:59:37 smithi086 ceph-mon[38221]: pgmap v278: 97 pgs: 97 active+clean; 164 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 1.1 MiB/s wr, 123 op/s 2024-03-23T17:59:38.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:59:37 smithi092 ceph-mon[79089]: pgmap v278: 97 pgs: 97 active+clean; 164 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 1.1 MiB/s wr, 123 op/s 2024-03-23T17:59:38.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:59:37 smithi152 ceph-mon[47949]: pgmap v278: 97 pgs: 97 active+clean; 164 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 1.1 MiB/s wr, 123 op/s 2024-03-23T17:59:38.917 INFO:tasks.workunit.client.0.smithi086.stdout: 1 476702 5.90 MB/sec execute 363 sec latency 20.270 ms 2024-03-23T17:59:39.916 INFO:tasks.workunit.client.0.smithi086.stdout: 1 478698 5.91 MB/sec execute 364 sec latency 13.279 ms 2024-03-23T17:59:40.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:59:39 smithi086 ceph-mon[38221]: pgmap v279: 97 pgs: 97 active+clean; 164 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 1.1 MiB/s wr, 123 op/s 2024-03-23T17:59:40.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:59:39 smithi092 ceph-mon[79089]: pgmap v279: 97 pgs: 97 active+clean; 164 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 1.1 MiB/s wr, 123 op/s 2024-03-23T17:59:40.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:59:39 smithi152 ceph-mon[47949]: pgmap v279: 97 pgs: 97 active+clean; 164 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 1.1 MiB/s wr, 123 op/s 2024-03-23T17:59:40.916 INFO:tasks.workunit.client.0.smithi086.stdout: 1 480236 5.92 MB/sec execute 365 sec latency 11.320 ms 2024-03-23T17:59:41.916 INFO:tasks.workunit.client.0.smithi086.stdout: 1 482159 5.93 MB/sec execute 366 sec latency 15.519 ms 2024-03-23T17:59:42.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:59:41 smithi086 ceph-mon[38221]: pgmap v280: 97 pgs: 97 active+clean; 171 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 992 KiB/s rd, 2.1 MiB/s wr, 219 op/s 2024-03-23T17:59:42.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:59:41 smithi092 ceph-mon[79089]: pgmap v280: 97 pgs: 97 active+clean; 171 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 992 KiB/s rd, 2.1 MiB/s wr, 219 op/s 2024-03-23T17:59:42.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:59:41 smithi152 ceph-mon[47949]: pgmap v280: 97 pgs: 97 active+clean; 171 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 992 KiB/s rd, 2.1 MiB/s wr, 219 op/s 2024-03-23T17:59:42.916 INFO:tasks.workunit.client.0.smithi086.stdout: 1 483693 5.94 MB/sec execute 367 sec latency 11.430 ms 2024-03-23T17:59:43.916 INFO:tasks.workunit.client.0.smithi086.stdout: 1 485573 5.95 MB/sec execute 368 sec latency 16.893 ms 2024-03-23T17:59:44.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:59:43 smithi086 ceph-mon[38221]: pgmap v281: 97 pgs: 97 active+clean; 171 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 946 KiB/s rd, 1.9 MiB/s wr, 200 op/s 2024-03-23T17:59:44.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:59:43 smithi092 ceph-mon[79089]: pgmap v281: 97 pgs: 97 active+clean; 171 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 946 KiB/s rd, 1.9 MiB/s wr, 200 op/s 2024-03-23T17:59:44.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:59:43 smithi152 ceph-mon[47949]: pgmap v281: 97 pgs: 97 active+clean; 171 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 946 KiB/s rd, 1.9 MiB/s wr, 200 op/s 2024-03-23T17:59:44.916 INFO:tasks.workunit.client.0.smithi086.stdout: 1 487130 5.96 MB/sec execute 369 sec latency 15.732 ms 2024-03-23T17:59:45.916 INFO:tasks.workunit.client.0.smithi086.stdout: 1 488537 5.96 MB/sec execute 370 sec latency 20.524 ms 2024-03-23T17:59:46.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:59:45 smithi086 ceph-mon[38221]: pgmap v282: 97 pgs: 97 active+clean; 178 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 298 op/s 2024-03-23T17:59:46.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:59:45 smithi086 ceph-mon[38221]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T17:59:46.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:59:45 smithi092 ceph-mon[79089]: pgmap v282: 97 pgs: 97 active+clean; 178 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 298 op/s 2024-03-23T17:59:46.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:59:45 smithi092 ceph-mon[79089]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T17:59:46.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:59:45 smithi152 ceph-mon[47949]: pgmap v282: 97 pgs: 97 active+clean; 178 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 298 op/s 2024-03-23T17:59:46.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:59:45 smithi152 ceph-mon[47949]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T17:59:46.916 INFO:tasks.workunit.client.0.smithi086.stdout: 1 490561 5.98 MB/sec execute 371 sec latency 15.450 ms 2024-03-23T17:59:47.298 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.297+0000 7f6265000700 1 -- 172.21.15.86:0/3098811416 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62601018a0 msgr2=0x7f62601058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:59:47.298 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.297+0000 7f6265000700 1 --2- 172.21.15.86:0/3098811416 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62601018a0 0x7f62601058e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f624c009990 tx=0x7f624c009ca0 comp rx=0 tx=0).stop 2024-03-23T17:59:47.298 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.298+0000 7f6265000700 1 -- 172.21.15.86:0/3098811416 shutdown_connections 2024-03-23T17:59:47.298 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.298+0000 7f6265000700 1 --2- 172.21.15.86:0/3098811416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6260106010 0x7f626010db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:47.299 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.298+0000 7f6265000700 1 --2- 172.21.15.86:0/3098811416 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6260100ef0 0x7f62601012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:47.299 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.298+0000 7f6265000700 1 --2- 172.21.15.86:0/3098811416 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62601018a0 0x7f62601058e0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:47.299 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.298+0000 7f6265000700 1 -- 172.21.15.86:0/3098811416 >> 172.21.15.86:0/3098811416 conn(0x7f62600fc750 msgr2=0x7f62600feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:59:47.299 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.298+0000 7f6265000700 1 -- 172.21.15.86:0/3098811416 shutdown_connections 2024-03-23T17:59:47.299 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.298+0000 7f6265000700 1 -- 172.21.15.86:0/3098811416 wait complete. 2024-03-23T17:59:47.301 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.299+0000 7f6265000700 1 Processor -- start 2024-03-23T17:59:47.301 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.299+0000 7f6265000700 1 -- start start 2024-03-23T17:59:47.301 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.299+0000 7f6265000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6260100ef0 0x7f6260199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:59:47.301 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.299+0000 7f6265000700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6260106010 0x7f62601998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:59:47.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.300+0000 7f6265000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f626019dc70 0x7f626019e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:59:47.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.300+0000 7f6265000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f626019e6f0 con 0x7f626019dc70 2024-03-23T17:59:47.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.300+0000 7f6265000700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f626019e860 con 0x7f6260100ef0 2024-03-23T17:59:47.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.300+0000 7f6265000700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f626019ea20 con 0x7f6260106010 2024-03-23T17:59:47.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.300+0000 7f625de00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6260100ef0 0x7f6260199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:59:47.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.300+0000 7f625de00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6260100ef0 0x7f6260199390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:51788/0 (socket says 172.21.15.86:51788) 2024-03-23T17:59:47.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.300+0000 7f625de00700 1 -- 172.21.15.86:0/780577165 learned_addr learned my addr 172.21.15.86:0/780577165 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:59:47.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.300+0000 7f625e800700 1 --2- 172.21.15.86:0/780577165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f626019dc70 0x7f626019e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:59:47.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.300+0000 7f625d400700 1 --2- 172.21.15.86:0/780577165 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6260106010 0x7f62601998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:59:47.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.300+0000 7f625de00700 1 -- 172.21.15.86:0/780577165 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6260106010 msgr2=0x7f62601998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:59:47.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.300+0000 7f625de00700 1 --2- 172.21.15.86:0/780577165 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6260106010 0x7f62601998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:47.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.300+0000 7f625de00700 1 -- 172.21.15.86:0/780577165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f626019dc70 msgr2=0x7f626019e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:59:47.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.300+0000 7f625de00700 1 --2- 172.21.15.86:0/780577165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f626019dc70 0x7f626019e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:47.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.300+0000 7f625de00700 1 -- 172.21.15.86:0/780577165 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f624c0095f0 con 0x7f6260100ef0 2024-03-23T17:59:47.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.300+0000 7f625e800700 1 --2- 172.21.15.86:0/780577165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f626019dc70 0x7f626019e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:59:47.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.300+0000 7f625de00700 1 --2- 172.21.15.86:0/780577165 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6260100ef0 0x7f6260199390 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f625000e9a0 tx=0x7f625000ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:59:47.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.301+0000 7f6256a00700 1 -- 172.21.15.86:0/780577165 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6250009790 con 0x7f6260100ef0 2024-03-23T17:59:47.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.301+0000 7f6256a00700 1 -- 172.21.15.86:0/780577165 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f625000cbd0 con 0x7f6260100ef0 2024-03-23T17:59:47.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.301+0000 7f6265000700 1 -- 172.21.15.86:0/780577165 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62601a5f80 con 0x7f6260100ef0 2024-03-23T17:59:47.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.301+0000 7f6256a00700 1 -- 172.21.15.86:0/780577165 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6250018680 con 0x7f6260100ef0 2024-03-23T17:59:47.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.301+0000 7f6265000700 1 -- 172.21.15.86:0/780577165 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62601a6450 con 0x7f6260100ef0 2024-03-23T17:59:47.304 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.304+0000 7f6256a00700 1 -- 172.21.15.86:0/780577165 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f625000cd40 con 0x7f6260100ef0 2024-03-23T17:59:47.305 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.305+0000 7f6256a00700 1 --2- 172.21.15.86:0/780577165 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f62440a1310 0x7f62440a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:59:47.306 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.305+0000 7f6278bb3b80 1 Processor -- start 2024-03-23T17:59:47.306 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.305+0000 7f6256a00700 1 -- 172.21.15.86:0/780577165 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f6250014070 con 0x7f6260100ef0 2024-03-23T17:59:47.306 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.306+0000 7f625d400700 1 --2- 172.21.15.86:0/780577165 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f62440a1310 0x7f62440a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:59:47.306 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.306+0000 7f6278bb3b80 1 -- start start 2024-03-23T17:59:47.308 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.306+0000 7f6278bb3b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d50e29100 0x557d50e294e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:59:47.308 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.306+0000 7f6278bb3b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557d50e29a20 0x557d50e34390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:59:47.308 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.306+0000 7f6278bb3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d50e348d0 0x557d50e36cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:59:47.309 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.306+0000 7f6278bb3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d50d82930 con 0x557d50e348d0 2024-03-23T17:59:47.309 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.306+0000 7f6278bb3b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d50d802f0 con 0x557d50e29100 2024-03-23T17:59:47.309 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.306+0000 7f6278bb3b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d50d7bbe0 con 0x557d50e29a20 2024-03-23T17:59:47.309 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.306+0000 7f625d400700 1 --2- 172.21.15.86:0/780577165 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f62440a1310 0x7f62440a37d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f624c00b2e0 tx=0x7f624c005fd0 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:59:47.309 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.306+0000 7f625de00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d50e29100 0x557d50e294e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:59:47.309 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.306+0000 7f625de00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d50e29100 0x557d50e294e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:51804/0 (socket says 172.21.15.86:51804) 2024-03-23T17:59:47.309 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.306+0000 7f625de00700 1 -- 172.21.15.86:0/1556102006 learned_addr learned my addr 172.21.15.86:0/1556102006 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:59:47.309 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.306+0000 7f625d400700 1 --2- 172.21.15.86:0/1556102006 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557d50e29a20 0x557d50e34390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:59:47.309 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.307+0000 7f625e800700 1 --2- 172.21.15.86:0/1556102006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d50e348d0 0x557d50e36cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:59:47.309 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.307+0000 7f625de00700 1 -- 172.21.15.86:0/1556102006 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557d50e29a20 msgr2=0x557d50e34390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:59:47.309 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.307+0000 7f625de00700 1 --2- 172.21.15.86:0/1556102006 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557d50e29a20 0x557d50e34390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:47.309 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.307+0000 7f625de00700 1 -- 172.21.15.86:0/1556102006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d50e348d0 msgr2=0x557d50e36cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:59:47.309 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.307+0000 7f625de00700 1 --2- 172.21.15.86:0/1556102006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d50e348d0 0x557d50e36cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:47.310 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.307+0000 7f625de00700 1 -- 172.21.15.86:0/1556102006 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d50bfd350 con 0x557d50e29100 2024-03-23T17:59:47.310 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.307+0000 7f625de00700 1 --2- 172.21.15.86:0/1556102006 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d50e29100 0x557d50e294e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f625000f010 tx=0x7f625009a860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce43c70b4e75cfe4 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:59:47.310 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.307+0000 7f624b400700 1 -- 172.21.15.86:0/1556102006 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62500a2510 con 0x557d50e29100 2024-03-23T17:59:47.310 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.307+0000 7f624b400700 1 -- 172.21.15.86:0/1556102006 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f625009a2f0 con 0x557d50e29100 2024-03-23T17:59:47.310 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.307+0000 7f624b400700 1 -- 172.21.15.86:0/1556102006 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62500a2280 con 0x557d50e29100 2024-03-23T17:59:47.310 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.308+0000 7f6278bb3b80 1 -- 172.21.15.86:0/1556102006 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d50e29100 msgr2=0x557d50e294e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:59:47.310 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.308+0000 7f6278bb3b80 1 --2- 172.21.15.86:0/1556102006 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d50e29100 0x557d50e294e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f625000f010 tx=0x7f625009a860 comp rx=0 tx=0).stop 2024-03-23T17:59:47.310 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.308+0000 7f6278bb3b80 1 -- 172.21.15.86:0/1556102006 shutdown_connections 2024-03-23T17:59:47.310 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.308+0000 7f6278bb3b80 1 --2- 172.21.15.86:0/1556102006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d50e348d0 0x557d50e36cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:47.310 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.308+0000 7f6278bb3b80 1 --2- 172.21.15.86:0/1556102006 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d50e29100 0x557d50e294e0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:47.310 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.308+0000 7f6278bb3b80 1 --2- 172.21.15.86:0/1556102006 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557d50e29a20 0x557d50e34390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:47.310 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.308+0000 7f6278bb3b80 1 -- 172.21.15.86:0/1556102006 >> 172.21.15.86:0/1556102006 conn(0x557d50d89640 msgr2=0x557d50e25520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:59:47.310 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.308+0000 7f6278bb3b80 1 -- 172.21.15.86:0/1556102006 shutdown_connections 2024-03-23T17:59:47.311 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.308+0000 7f6278bb3b80 1 -- 172.21.15.86:0/1556102006 wait complete. 2024-03-23T17:59:47.314 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.309+0000 7f6278bb3b80 1 -- start start 2024-03-23T17:59:47.314 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.309+0000 7f6278bb3b80 1 Processor -- start 2024-03-23T17:59:47.314 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.309+0000 7f6278bb3b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d50e29100 0x557d50e342c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:59:47.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.309+0000 7f6278bb3b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557d50e29a20 0x557d50e31600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:59:47.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.309+0000 7f6278bb3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d50e348d0 0x557d50e31c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:59:47.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.309+0000 7f6278bb3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d50d17d20 con 0x557d50e348d0 2024-03-23T17:59:47.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.309+0000 7f6278bb3b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d50d179c0 con 0x557d50e29100 2024-03-23T17:59:47.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.309+0000 7f6278bb3b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d50bff640 con 0x557d50e29a20 2024-03-23T17:59:47.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.309+0000 7f6278bb3b80 10 client.0 _ll_register_callbacks cb 0x557d50d21410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:59:47.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.310+0000 7f625de00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d50e29100 0x557d50e342c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:59:47.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.310+0000 7f625de00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d50e29100 0x557d50e342c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:51806/0 (socket says 172.21.15.86:51806) 2024-03-23T17:59:47.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.310+0000 7f625de00700 1 -- 172.21.15.86:0/3938358137 learned_addr learned my addr 172.21.15.86:0/3938358137 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:59:47.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.310+0000 7f625e800700 1 --2- 172.21.15.86:0/3938358137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d50e348d0 0x557d50e31c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:59:47.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.310+0000 7f625d400700 1 --2- 172.21.15.86:0/3938358137 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557d50e29a20 0x557d50e31600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:59:47.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.310+0000 7f625de00700 1 -- 172.21.15.86:0/3938358137 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557d50e29a20 msgr2=0x557d50e31600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:59:47.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.310+0000 7f625de00700 1 --2- 172.21.15.86:0/3938358137 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557d50e29a20 0x557d50e31600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:47.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.310+0000 7f625de00700 1 -- 172.21.15.86:0/3938358137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d50e348d0 msgr2=0x557d50e31c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:59:47.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.310+0000 7f625de00700 1 --2- 172.21.15.86:0/3938358137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d50e348d0 0x557d50e31c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:47.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.310+0000 7f625de00700 1 -- 172.21.15.86:0/3938358137 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62601a6450 con 0x557d50e29100 2024-03-23T17:59:47.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.310+0000 7f625e800700 1 --2- 172.21.15.86:0/3938358137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d50e348d0 0x557d50e31c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:59:47.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.310+0000 7f625d400700 1 --2- 172.21.15.86:0/3938358137 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557d50e29a20 0x557d50e31600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T17:59:47.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.310+0000 7f625de00700 1 --2- 172.21.15.86:0/3938358137 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d50e29100 0x557d50e342c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f625009a020 tx=0x7f6250010c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82047126db31b989 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:59:47.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.310+0000 7f623fe00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T17:59:47.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.310+0000 7f623fe00700 1 -- 172.21.15.86:0/3938358137 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62500a2290 con 0x557d50e29100 2024-03-23T17:59:47.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.310+0000 7f623fe00700 1 -- 172.21.15.86:0/3938358137 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f62500b82d0 con 0x557d50e29100 2024-03-23T17:59:47.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.310+0000 7f6278bb3b80 1 -- 172.21.15.86:0/3938358137 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557d50d3c320 con 0x557d50e29100 2024-03-23T17:59:47.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.311+0000 7f623fe00700 1 -- 172.21.15.86:0/3938358137 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f62500b8490 con 0x557d50e29100 2024-03-23T17:59:47.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.311+0000 7f6278bb3b80 10 client.26765 fetch_fsmap learned FSMap version 44 2024-03-23T17:59:47.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.311+0000 7f6278bb3b80 1 -- 172.21.15.86:0/3938358137 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x557d50ba9b00 con 0x557d50e29100 2024-03-23T17:59:47.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.311+0000 7f623fe00700 1 -- 172.21.15.86:0/3938358137 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557d50d3c320 con 0x557d50e29100 2024-03-23T17:59:47.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.311+0000 7f623fe00700 1 -- 172.21.15.86:0/3938358137 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 44) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f625009f050 con 0x557d50e29100 2024-03-23T17:59:47.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.311+0000 7f6278bb3b80 10 client.26765 fetch_fsmap finished waiting for FSMap version 44 2024-03-23T17:59:47.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.311+0000 7f6278bb3b80 10 client.26765 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:59:47.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.311+0000 7f6278bb3b80 20 client.26765 populate_metadata read hostname 'smithi086' 2024-03-23T17:59:47.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.311+0000 7f6278bb3b80 1 --2- 172.21.15.86:0/3938358137 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x557d50e441e0 0x557d50e46600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:59:47.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.311+0000 7f6278bb3b80 4 client.26765 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T17:59:47.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.311+0000 7f6278bb3b80 1 -- 172.21.15.86:0/3938358137 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557d50d746a0 con 0x557d50e441e0 2024-03-23T17:59:47.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.311+0000 7f625d400700 1 --2- 172.21.15.86:0/3938358137 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x557d50e441e0 0x557d50e46600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:59:47.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.312+0000 7f625d400700 1 --2- 172.21.15.86:0/3938358137 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x557d50e441e0 0x557d50e46600 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc35f6e604f3debb server_cookie=e5a2a4289b07eae4 in_seq=0 out_seq=0 2024-03-23T17:59:47.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.312+0000 7f623fe00700 10 client.26765 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:59:47.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.312+0000 7f623fe00700 1 -- 172.21.15.86:0/3938358137 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x557d50d746a0 con 0x557d50e441e0 2024-03-23T17:59:47.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.312+0000 7f623fe00700 10 client.26765 handle_command_reply: tid=0 2024-03-23T17:59:47.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.312+0000 7f6278bb3b80 1 client.26765 shutdown 2024-03-23T17:59:47.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.313+0000 7f6278bb3b80 1 -- 172.21.15.86:0/3938358137 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d50e29100 msgr2=0x557d50e342c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:59:47.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.313+0000 7f6278bb3b80 1 --2- 172.21.15.86:0/3938358137 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d50e29100 0x557d50e342c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f625009a020 tx=0x7f6250010c00 comp rx=0 tx=0).stop 2024-03-23T17:59:47.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.313+0000 7f6278bb3b80 1 -- 172.21.15.86:0/3938358137 shutdown_connections 2024-03-23T17:59:47.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.313+0000 7f6278bb3b80 1 --2- 172.21.15.86:0/3938358137 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x557d50e441e0 0x557d50e46600 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:47.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.313+0000 7f6278bb3b80 1 --2- 172.21.15.86:0/3938358137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d50e348d0 0x557d50e31c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:47.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.313+0000 7f623fe00700 0 client.26765 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:59:47.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.313+0000 7f6278bb3b80 1 --2- 172.21.15.86:0/3938358137 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d50e29100 0x557d50e342c0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:47.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.313+0000 7f6278bb3b80 1 --2- 172.21.15.86:0/3938358137 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557d50e29a20 0x557d50e31600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:47.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.313+0000 7f6278bb3b80 1 -- 172.21.15.86:0/3938358137 >> 172.21.15.86:0/3938358137 conn(0x557d50d89640 msgr2=0x557d50e2b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:59:47.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.313+0000 7f6278bb3b80 1 -- 172.21.15.86:0/3938358137 shutdown_connections 2024-03-23T17:59:47.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.313+0000 7f6278bb3b80 1 -- 172.21.15.86:0/3938358137 wait complete. 2024-03-23T17:59:47.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.313+0000 7f6278bb3b80 20 client.26765 trim_cache size 0 max 16384 2024-03-23T17:59:47.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.329+0000 7f6278bb3b80 1 Processor -- start 2024-03-23T17:59:47.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.329+0000 7f6278bb3b80 1 -- start start 2024-03-23T17:59:47.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.329+0000 7f6278bb3b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557d50e29100 0x557d50e3b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:59:47.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.329+0000 7f6278bb3b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d50e29a20 0x557d50e1f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:59:47.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.329+0000 7f6278bb3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d50e348d0 0x557d50e2b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:59:47.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.329+0000 7f6278bb3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d50e4b2e0 con 0x557d50e348d0 2024-03-23T17:59:47.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.329+0000 7f6278bb3b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d50e2c450 con 0x557d50e29a20 2024-03-23T17:59:47.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.329+0000 7f6278bb3b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d50e26a30 con 0x557d50e29100 2024-03-23T17:59:47.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.330+0000 7f625de00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557d50e29100 0x557d50e3b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:59:47.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.330+0000 7f625d400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d50e29a20 0x557d50e1f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:59:47.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.330+0000 7f625de00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557d50e29100 0x557d50e3b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:45852/0 (socket says 172.21.15.86:45852) 2024-03-23T17:59:47.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.330+0000 7f625e800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d50e348d0 0x557d50e2b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:59:47.331 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.330+0000 7f625de00700 1 -- 172.21.15.86:0/3637851577 learned_addr learned my addr 172.21.15.86:0/3637851577 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:59:47.332 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.330+0000 7f625de00700 1 -- 172.21.15.86:0/3637851577 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d50e29a20 msgr2=0x557d50e1f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:59:47.332 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.330+0000 7f625de00700 1 --2- 172.21.15.86:0/3637851577 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d50e29a20 0x557d50e1f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:47.332 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.330+0000 7f625de00700 1 -- 172.21.15.86:0/3637851577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d50e348d0 msgr2=0x557d50e2b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:59:47.332 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.330+0000 7f625de00700 1 --2- 172.21.15.86:0/3637851577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d50e348d0 0x557d50e2b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:47.332 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.330+0000 7f625de00700 1 -- 172.21.15.86:0/3637851577 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d50d41e40 con 0x557d50e29100 2024-03-23T17:59:47.333 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.330+0000 7f625de00700 1 --2- 172.21.15.86:0/3637851577 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557d50e29100 0x557d50e3b5a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f625000f010 tx=0x7f625000c6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=374a1bffe7c0d403 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:59:47.333 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.331+0000 7f624be00700 1 -- 172.21.15.86:0/3637851577 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6250083430 con 0x557d50e29100 2024-03-23T17:59:47.333 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.331+0000 7f624be00700 1 -- 172.21.15.86:0/3637851577 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6250083620 con 0x557d50e29100 2024-03-23T17:59:47.333 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.331+0000 7f624be00700 1 -- 172.21.15.86:0/3637851577 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6250098ff0 con 0x557d50e29100 2024-03-23T17:59:47.333 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.331+0000 7f6278bb3b80 1 -- 172.21.15.86:0/3637851577 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557d50e29100 msgr2=0x557d50e3b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:59:47.333 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.331+0000 7f6278bb3b80 1 --2- 172.21.15.86:0/3637851577 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557d50e29100 0x557d50e3b5a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f625000f010 tx=0x7f625000c6e0 comp rx=0 tx=0).stop 2024-03-23T17:59:47.333 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.331+0000 7f6278bb3b80 1 -- 172.21.15.86:0/3637851577 shutdown_connections 2024-03-23T17:59:47.333 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.331+0000 7f6278bb3b80 1 --2- 172.21.15.86:0/3637851577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d50e348d0 0x557d50e2b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:47.333 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.331+0000 7f6278bb3b80 1 --2- 172.21.15.86:0/3637851577 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557d50e29100 0x557d50e3b5a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:47.333 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.331+0000 7f6278bb3b80 1 --2- 172.21.15.86:0/3637851577 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d50e29a20 0x557d50e1f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:47.333 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.331+0000 7f6278bb3b80 1 -- 172.21.15.86:0/3637851577 >> 172.21.15.86:0/3637851577 conn(0x557d50d89640 msgr2=0x557d50e2fed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:59:47.333 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.332+0000 7f6278bb3b80 1 -- 172.21.15.86:0/3637851577 shutdown_connections 2024-03-23T17:59:47.334 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.332+0000 7f6278bb3b80 1 -- 172.21.15.86:0/3637851577 wait complete. 2024-03-23T17:59:47.334 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.332+0000 7f6278bb3b80 1 -- start start 2024-03-23T17:59:47.334 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.332+0000 7f6278bb3b80 1 Processor -- start 2024-03-23T17:59:47.336 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.333+0000 7f6278bb3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d50e29100 0x557d50e3acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:59:47.336 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.333+0000 7f6278bb3b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557d50e29a20 0x557d50e3b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:59:47.337 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.333+0000 7f6278bb3b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d50e348d0 0x557d50e30b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:59:47.337 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.333+0000 7f6278bb3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d50e31280 con 0x557d50e29100 2024-03-23T17:59:47.337 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.333+0000 7f6278bb3b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d50e313f0 con 0x557d50e348d0 2024-03-23T17:59:47.337 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.333+0000 7f6278bb3b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d50e315b0 con 0x557d50e29a20 2024-03-23T17:59:47.337 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.333+0000 7f6278bb3b80 10 client.0 _ll_register_callbacks cb 0x557d50d21410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T17:59:47.337 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.333+0000 7f625de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d50e29100 0x557d50e3acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:59:47.337 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.333+0000 7f625de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d50e29100 0x557d50e3acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40880/0 (socket says 172.21.15.86:40880) 2024-03-23T17:59:47.337 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.333+0000 7f625de00700 1 -- 172.21.15.86:0/1819153964 learned_addr learned my addr 172.21.15.86:0/1819153964 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T17:59:47.337 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.333+0000 7f625d400700 1 --2- 172.21.15.86:0/1819153964 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557d50e29a20 0x557d50e3b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:59:47.337 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.333+0000 7f625e800700 1 --2- 172.21.15.86:0/1819153964 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d50e348d0 0x557d50e30b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:59:47.337 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.333+0000 7f625d400700 1 -- 172.21.15.86:0/1819153964 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d50e348d0 msgr2=0x557d50e30b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:59:47.337 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.333+0000 7f625d400700 1 --2- 172.21.15.86:0/1819153964 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d50e348d0 0x557d50e30b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:47.337 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.333+0000 7f625d400700 1 -- 172.21.15.86:0/1819153964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d50e29100 msgr2=0x557d50e3acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T17:59:47.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.333+0000 7f625d400700 1 --2- 172.21.15.86:0/1819153964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d50e29100 0x557d50e3acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:47.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.333+0000 7f625d400700 1 -- 172.21.15.86:0/1819153964 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557d50ba9b00 con 0x557d50e29a20 2024-03-23T17:59:47.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.334+0000 7f625d400700 1 --2- 172.21.15.86:0/1819153964 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557d50e29a20 0x557d50e3b230 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f624c0098e0 tx=0x7f624c01a050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54ef6d19920af397 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T17:59:47.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.334+0000 7f623fe00700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-03-23T17:59:47.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.334+0000 7f623fe00700 1 -- 172.21.15.86:0/1819153964 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624c029070 con 0x557d50e29a20 2024-03-23T17:59:47.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.334+0000 7f6278bb3b80 1 -- 172.21.15.86:0/1819153964 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f625009a2f0 con 0x557d50e29a20 2024-03-23T17:59:47.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.334+0000 7f623fe00700 1 -- 172.21.15.86:0/1819153964 <== mon.1 v2:172.21.15.152:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f624c032a00 con 0x557d50e29a20 2024-03-23T17:59:47.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.334+0000 7f623fe00700 1 -- 172.21.15.86:0/1819153964 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f624c02b040 con 0x557d50e29a20 2024-03-23T17:59:47.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.334+0000 7f6278bb3b80 10 client.26755 fetch_fsmap learned FSMap version 44 2024-03-23T17:59:47.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.334+0000 7f6278bb3b80 1 -- 172.21.15.86:0/1819153964 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x557d50c01c70 con 0x557d50e29a20 2024-03-23T17:59:47.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.335+0000 7f623fe00700 1 -- 172.21.15.86:0/1819153964 <== mon.1 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f625009a2f0 con 0x557d50e29a20 2024-03-23T17:59:47.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.335+0000 7f623fe00700 1 -- 172.21.15.86:0/1819153964 <== mon.1 v2:172.21.15.152:3300/0 5 ==== fsmap(e 44) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f624c004d50 con 0x557d50e29a20 2024-03-23T17:59:47.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.335+0000 7f6278bb3b80 10 client.26755 fetch_fsmap finished waiting for FSMap version 44 2024-03-23T17:59:47.339 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.335+0000 7f6278bb3b80 10 client.26755 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T17:59:47.339 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.335+0000 7f6278bb3b80 20 client.26755 populate_metadata read hostname 'smithi086' 2024-03-23T17:59:47.339 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.335+0000 7f6278bb3b80 1 --2- 172.21.15.86:0/1819153964 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x557d50e96ef0 0x557d50e99310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T17:59:47.339 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.335+0000 7f6278bb3b80 4 client.26755 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T17:59:47.339 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.335+0000 7f6278bb3b80 1 -- 172.21.15.86:0/1819153964 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557d50bcac90 con 0x557d50e96ef0 2024-03-23T17:59:47.340 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.335+0000 7f625de00700 1 --2- 172.21.15.86:0/1819153964 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x557d50e96ef0 0x557d50e99310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T17:59:47.340 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.336+0000 7f623fe00700 10 client.26755 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T17:59:47.340 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.336+0000 7f625de00700 1 --2- 172.21.15.86:0/1819153964 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x557d50e96ef0 0x557d50e99310 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ddfe6e135de81fb server_cookie=379192bf0e2ce482 in_seq=0 out_seq=0 2024-03-23T17:59:47.388 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.388+0000 7f623fe00700 1 -- 172.21.15.86:0/1819153964 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557d50bcac90 con 0x557d50e96ef0 2024-03-23T17:59:47.389 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.388+0000 7f623fe00700 10 client.26755 handle_command_reply: tid=0 2024-03-23T17:59:47.389 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.388+0000 7f6278bb3b80 1 client.26755 shutdown 2024-03-23T17:59:47.390 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.389+0000 7f6278bb3b80 1 -- 172.21.15.86:0/1819153964 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557d50e29a20 msgr2=0x557d50e3b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:59:47.390 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.389+0000 7f6278bb3b80 1 --2- 172.21.15.86:0/1819153964 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557d50e29a20 0x557d50e3b230 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f624c0098e0 tx=0x7f624c01a050 comp rx=0 tx=0).stop 2024-03-23T17:59:47.390 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.389+0000 7f6278bb3b80 1 -- 172.21.15.86:0/1819153964 shutdown_connections 2024-03-23T17:59:47.390 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.389+0000 7f6278bb3b80 1 --2- 172.21.15.86:0/1819153964 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x557d50e96ef0 0x557d50e99310 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:47.390 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.389+0000 7f6278bb3b80 1 --2- 172.21.15.86:0/1819153964 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557d50e348d0 0x557d50e30b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:47.390 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.389+0000 7f6278bb3b80 1 --2- 172.21.15.86:0/1819153964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d50e29100 0x557d50e3acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:47.390 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.389+0000 7f6278bb3b80 1 --2- 172.21.15.86:0/1819153964 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557d50e29a20 0x557d50e3b230 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:47.390 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.389+0000 7f6278bb3b80 1 -- 172.21.15.86:0/1819153964 >> 172.21.15.86:0/1819153964 conn(0x557d50d89640 msgr2=0x557d50e2b040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:59:47.391 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.389+0000 7f623fe00700 0 client.26755 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T17:59:47.391 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.389+0000 7f6278bb3b80 1 -- 172.21.15.86:0/1819153964 shutdown_connections 2024-03-23T17:59:47.391 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.389+0000 7f6278bb3b80 1 -- 172.21.15.86:0/1819153964 wait complete. 2024-03-23T17:59:47.391 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.389+0000 7f6278bb3b80 20 client.26755 trim_cache size 0 max 16384 2024-03-23T17:59:47.391 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-03-23T17:59:47.391 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-03-23T17:59:47.391 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-03-23T17:59:47.391 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-03-23T17:59:47.392 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.391+0000 7f6265000700 1 -- 172.21.15.86:0/780577165 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f62440a1310 msgr2=0x7f62440a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:59:47.392 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.391+0000 7f6265000700 1 --2- 172.21.15.86:0/780577165 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f62440a1310 0x7f62440a37d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f624c00b2e0 tx=0x7f624c005fd0 comp rx=0 tx=0).stop 2024-03-23T17:59:47.392 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.391+0000 7f6265000700 1 -- 172.21.15.86:0/780577165 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6260100ef0 msgr2=0x7f6260199390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T17:59:47.392 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.391+0000 7f6265000700 1 --2- 172.21.15.86:0/780577165 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6260100ef0 0x7f6260199390 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f625000e9a0 tx=0x7f625000ed60 comp rx=0 tx=0).stop 2024-03-23T17:59:47.392 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.391+0000 7f6265000700 1 -- 172.21.15.86:0/780577165 shutdown_connections 2024-03-23T17:59:47.392 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.391+0000 7f6265000700 1 --2- 172.21.15.86:0/780577165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f626019dc70 0x7f626019e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:47.392 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.391+0000 7f6265000700 1 --2- 172.21.15.86:0/780577165 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6260100ef0 0x7f6260199390 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:47.392 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.391+0000 7f6265000700 1 --2- 172.21.15.86:0/780577165 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f62440a1310 0x7f62440a37d0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:47.393 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.391+0000 7f6265000700 1 --2- 172.21.15.86:0/780577165 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6260106010 0x7f62601998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T17:59:47.393 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.391+0000 7f6265000700 1 -- 172.21.15.86:0/780577165 >> 172.21.15.86:0/780577165 conn(0x7f62600fc750 msgr2=0x7f62600fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T17:59:47.393 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.391+0000 7f6265000700 1 -- 172.21.15.86:0/780577165 shutdown_connections 2024-03-23T17:59:47.393 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T17:59:47.391+0000 7f6265000700 1 -- 172.21.15.86:0/780577165 wait complete. 2024-03-23T17:59:47.916 INFO:tasks.workunit.client.0.smithi086.stdout: 1 491831 5.98 MB/sec execute 372 sec latency 20.312 ms 2024-03-23T17:59:48.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:59:47 smithi086 ceph-mon[38221]: pgmap v283: 97 pgs: 97 active+clean; 178 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 193 op/s 2024-03-23T17:59:48.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:59:47 smithi092 ceph-mon[79089]: pgmap v283: 97 pgs: 97 active+clean; 178 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 193 op/s 2024-03-23T17:59:48.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:59:47 smithi152 ceph-mon[47949]: pgmap v283: 97 pgs: 97 active+clean; 178 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 193 op/s 2024-03-23T17:59:48.916 INFO:tasks.workunit.client.0.smithi086.stdout: 1 493941 6.00 MB/sec execute 373 sec latency 52.469 ms 2024-03-23T17:59:49.917 INFO:tasks.workunit.client.0.smithi086.stdout: 1 495103 5.99 MB/sec execute 374 sec latency 15.228 ms 2024-03-23T17:59:50.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:59:49 smithi086 ceph-mon[38221]: pgmap v284: 97 pgs: 97 active+clean; 178 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 193 op/s 2024-03-23T17:59:50.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:59:49 smithi092 ceph-mon[79089]: pgmap v284: 97 pgs: 97 active+clean; 178 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 193 op/s 2024-03-23T17:59:50.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:59:49 smithi152 ceph-mon[47949]: pgmap v284: 97 pgs: 97 active+clean; 178 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 193 op/s 2024-03-23T17:59:50.916 INFO:tasks.workunit.client.0.smithi086.stdout: 1 497227 6.01 MB/sec execute 375 sec latency 16.083 ms 2024-03-23T17:59:51.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:59:50 smithi086 ceph-mon[38221]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T17:59:51.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:59:50 smithi092 ceph-mon[79089]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T17:59:51.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:59:50 smithi152 ceph-mon[47949]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T17:59:51.917 INFO:tasks.workunit.client.0.smithi086.stdout: 1 498583 6.01 MB/sec execute 376 sec latency 12.198 ms 2024-03-23T17:59:52.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:59:51 smithi086 ceph-mon[38221]: pgmap v285: 97 pgs: 97 active+clean; 194 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.4 MiB/s wr, 319 op/s 2024-03-23T17:59:52.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:59:51 smithi092 ceph-mon[79089]: pgmap v285: 97 pgs: 97 active+clean; 194 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.4 MiB/s wr, 319 op/s 2024-03-23T17:59:52.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:59:51 smithi152 ceph-mon[47949]: pgmap v285: 97 pgs: 97 active+clean; 194 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.4 MiB/s wr, 319 op/s 2024-03-23T17:59:52.917 INFO:tasks.workunit.client.0.smithi086.stdout: 1 500762 6.03 MB/sec execute 377 sec latency 14.243 ms 2024-03-23T17:59:53.917 INFO:tasks.workunit.client.0.smithi086.stdout: 1 501991 6.03 MB/sec execute 378 sec latency 19.443 ms 2024-03-23T17:59:54.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:59:53 smithi086 ceph-mon[38221]: pgmap v286: 97 pgs: 97 active+clean; 194 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.4 MiB/s wr, 223 op/s 2024-03-23T17:59:54.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:59:53 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:59:54.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:59:53 smithi092 ceph-mon[79089]: pgmap v286: 97 pgs: 97 active+clean; 194 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.4 MiB/s wr, 223 op/s 2024-03-23T17:59:54.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:59:53 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:59:54.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:59:53 smithi152 ceph-mon[47949]: pgmap v286: 97 pgs: 97 active+clean; 194 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.4 MiB/s wr, 223 op/s 2024-03-23T17:59:54.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:59:53 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T17:59:54.917 INFO:tasks.workunit.client.0.smithi086.stdout: 1 503856 6.04 MB/sec execute 379 sec latency 18.600 ms 2024-03-23T17:59:55.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:59:54 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:59:55.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:59:54 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:59:55.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:59:54 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:59:55.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:59:54 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:59:55.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:59:54 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:59:55.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:59:54 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:59:55.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:59:54 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T17:59:55.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:59:54 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T17:59:55.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:59:54 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T17:59:55.917 INFO:tasks.workunit.client.0.smithi086.stdout: 1 505208 6.04 MB/sec execute 380 sec latency 13.875 ms 2024-03-23T17:59:56.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:59:55 smithi086 ceph-mon[38221]: pgmap v287: 97 pgs: 97 active+clean; 204 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.6 MiB/s wr, 323 op/s 2024-03-23T17:59:56.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:59:55 smithi092 ceph-mon[79089]: pgmap v287: 97 pgs: 97 active+clean; 204 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.6 MiB/s wr, 323 op/s 2024-03-23T17:59:56.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:59:55 smithi152 ceph-mon[47949]: pgmap v287: 97 pgs: 97 active+clean; 204 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.6 MiB/s wr, 323 op/s 2024-03-23T17:59:56.917 INFO:tasks.workunit.client.0.smithi086.stdout: 1 507049 6.05 MB/sec execute 381 sec latency 15.564 ms 2024-03-23T17:59:57.917 INFO:tasks.workunit.client.0.smithi086.stdout: 1 508509 6.06 MB/sec execute 382 sec latency 14.178 ms 2024-03-23T17:59:58.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:59:57 smithi086 ceph-mon[38221]: pgmap v288: 97 pgs: 97 active+clean; 204 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.6 MiB/s wr, 225 op/s 2024-03-23T17:59:58.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:59:57 smithi092 ceph-mon[79089]: pgmap v288: 97 pgs: 97 active+clean; 204 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.6 MiB/s wr, 225 op/s 2024-03-23T17:59:58.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:59:57 smithi152 ceph-mon[47949]: pgmap v288: 97 pgs: 97 active+clean; 204 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.6 MiB/s wr, 225 op/s 2024-03-23T17:59:58.917 INFO:tasks.workunit.client.0.smithi086.stdout: 1 510403 6.06 MB/sec execute 383 sec latency 16.962 ms 2024-03-23T17:59:59.917 INFO:tasks.workunit.client.0.smithi086.stdout: 1 511966 6.08 MB/sec execute 384 sec latency 14.748 ms 2024-03-23T18:00:00.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 17:59:59 smithi086 ceph-mon[38221]: pgmap v289: 97 pgs: 97 active+clean; 204 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.6 MiB/s wr, 225 op/s 2024-03-23T18:00:00.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:00:00 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mon-a[38198]: 2024-03-23T17:59:59.998+0000 7f70a6200700 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR 3 MDSs report damaged metadata 2024-03-23T18:00:00.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:00:00 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mon-a[38198]: 2024-03-23T17:59:59.998+0000 7f70a6200700 -1 log_channel(cluster) log [ERR] : [ERR] MDS_DAMAGE: 3 MDSs report damaged metadata 2024-03-23T18:00:00.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:00:00 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mon-a[38198]: 2024-03-23T17:59:59.998+0000 7f70a6200700 -1 log_channel(cluster) log [ERR] : mds.e(mds.4): Metadata damage detected 2024-03-23T18:00:00.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:00:00 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mon-a[38198]: 2024-03-23T17:59:59.998+0000 7f70a6200700 -1 log_channel(cluster) log [ERR] : mds.d(mds.3): Metadata damage detected 2024-03-23T18:00:00.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:00:00 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mon-a[38198]: 2024-03-23T17:59:59.998+0000 7f70a6200700 -1 log_channel(cluster) log [ERR] : mds.h(mds.2): Metadata damage detected 2024-03-23T18:00:00.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 17:59:59 smithi092 ceph-mon[79089]: pgmap v289: 97 pgs: 97 active+clean; 204 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.6 MiB/s wr, 225 op/s 2024-03-23T18:00:00.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 17:59:59 smithi152 ceph-mon[47949]: pgmap v289: 97 pgs: 97 active+clean; 204 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.6 MiB/s wr, 225 op/s 2024-03-23T18:00:00.557 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.556+0000 7f2415800700 1 -- 172.21.15.86:0/1042451358 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2410100ad0 msgr2=0x7f2410100f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:00:00.558 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.556+0000 7f2415800700 1 --2- 172.21.15.86:0/1042451358 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2410100ad0 0x7f2410100f50 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f23fc009990 tx=0x7f23fc009ca0 comp rx=0 tx=0).stop 2024-03-23T18:00:00.558 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.556+0000 7f2415800700 1 -- 172.21.15.86:0/1042451358 shutdown_connections 2024-03-23T18:00:00.558 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.556+0000 7f2415800700 1 --2- 172.21.15.86:0/1042451358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2410101490 0x7f241010d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:00.558 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.556+0000 7f2415800700 1 --2- 172.21.15.86:0/1042451358 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2410106b70 0x7f2410106f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:00.558 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.556+0000 7f2415800700 1 --2- 172.21.15.86:0/1042451358 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2410100ad0 0x7f2410100f50 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:00.558 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.556+0000 7f2415800700 1 -- 172.21.15.86:0/1042451358 >> 172.21.15.86:0/1042451358 conn(0x7f24100fc5e0 msgr2=0x7f24100fea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:00:00.558 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.556+0000 7f2415800700 1 -- 172.21.15.86:0/1042451358 shutdown_connections 2024-03-23T18:00:00.558 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.556+0000 7f2415800700 1 -- 172.21.15.86:0/1042451358 wait complete. 2024-03-23T18:00:00.559 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.557+0000 7f2415800700 1 Processor -- start 2024-03-23T18:00:00.559 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.557+0000 7f2415800700 1 -- start start 2024-03-23T18:00:00.559 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.558+0000 7f2415800700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2410100ad0 0x7f24100730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.558+0000 7f2415800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2410101490 0x7f24100735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.558+0000 7f2415800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2410106b70 0x7f2410073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.558+0000 7f2415800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24101a52f0 con 0x7f2410101490 2024-03-23T18:00:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.558+0000 7f2415800700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24101a5460 con 0x7f2410106b70 2024-03-23T18:00:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.558+0000 7f2415800700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24101a5620 con 0x7f2410100ad0 2024-03-23T18:00:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.558+0000 7f240f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2410101490 0x7f24100735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.558+0000 7f240f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2410101490 0x7f24100735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59352/0 (socket says 172.21.15.86:59352) 2024-03-23T18:00:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.558+0000 7f240f400700 1 -- 172.21.15.86:0/1408245520 learned_addr learned my addr 172.21.15.86:0/1408245520 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:00:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.558+0000 7f2414e00700 1 --2- 172.21.15.86:0/1408245520 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2410106b70 0x7f2410073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:00.561 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.559+0000 7f240f400700 1 -- 172.21.15.86:0/1408245520 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2410100ad0 msgr2=0x7f24100730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:00:00.561 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.559+0000 7f240f400700 1 --2- 172.21.15.86:0/1408245520 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2410100ad0 0x7f24100730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:00.561 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.559+0000 7f240f400700 1 -- 172.21.15.86:0/1408245520 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2410106b70 msgr2=0x7f2410073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:00:00.561 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.559+0000 7f240f400700 1 --2- 172.21.15.86:0/1408245520 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2410106b70 0x7f2410073b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:00.561 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.559+0000 7f240f400700 1 -- 172.21.15.86:0/1408245520 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23fc0095f0 con 0x7f2410101490 2024-03-23T18:00:00.561 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.559+0000 7f2414e00700 1 --2- 172.21.15.86:0/1408245520 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2410106b70 0x7f2410073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T18:00:00.561 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.559+0000 7f240f400700 1 --2- 172.21.15.86:0/1408245520 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2410101490 0x7f24100735f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f23fc005910 tx=0x7f23fc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:00:00.562 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.559+0000 7f240cc00700 1 -- 172.21.15.86:0/1408245520 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23fc01d070 con 0x7f2410101490 2024-03-23T18:00:00.562 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.560+0000 7f240cc00700 1 -- 172.21.15.86:0/1408245520 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23fc022470 con 0x7f2410101490 2024-03-23T18:00:00.562 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.560+0000 7f2415800700 1 -- 172.21.15.86:0/1408245520 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24101a58a0 con 0x7f2410101490 2024-03-23T18:00:00.562 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.560+0000 7f240cc00700 1 -- 172.21.15.86:0/1408245520 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23fc017650 con 0x7f2410101490 2024-03-23T18:00:00.562 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.560+0000 7f2415800700 1 -- 172.21.15.86:0/1408245520 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24101a5dd0 con 0x7f2410101490 2024-03-23T18:00:00.564 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.563+0000 7f240cc00700 1 -- 172.21.15.86:0/1408245520 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f23fc0225e0 con 0x7f2410101490 2024-03-23T18:00:00.565 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.565+0000 7f242a87eb80 1 Processor -- start 2024-03-23T18:00:00.566 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.565+0000 7f242a87eb80 1 -- start start 2024-03-23T18:00:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.565+0000 7f240cc00700 1 --2- 172.21.15.86:0/1408245520 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f23f40a1310 0x7f23f40a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.565+0000 7f242a87eb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ff8f28c2d0 0x55ff8f28c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.565+0000 7f240cc00700 1 -- 172.21.15.86:0/1408245520 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f23fc0cdf90 con 0x7f2410101490 2024-03-23T18:00:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.565+0000 7f242a87eb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff8f28cbf0 0x55ff8f297570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.565+0000 7f242a87eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff8f297ab0 0x55ff8f299ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.565+0000 7f242a87eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff8f1e5930 con 0x55ff8f297ab0 2024-03-23T18:00:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.565+0000 7f242a87eb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff8f1e32f0 con 0x55ff8f28cbf0 2024-03-23T18:00:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.565+0000 7f242a87eb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff8f1debe0 con 0x55ff8f28c2d0 2024-03-23T18:00:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.566+0000 7f240f400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff8f28cbf0 0x55ff8f297570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.566+0000 7f240f400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff8f28cbf0 0x55ff8f297570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:46762/0 (socket says 172.21.15.86:46762) 2024-03-23T18:00:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.566+0000 7f2414e00700 1 --2- 172.21.15.86:0/1408245520 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f23f40a1310 0x7f23f40a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.566+0000 7f240f400700 1 -- 172.21.15.86:0/1779947509 learned_addr learned my addr 172.21.15.86:0/1779947509 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:00:00.569 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.566+0000 7f240fe00700 1 --2- 172.21.15.86:0/1779947509 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ff8f28c2d0 0x55ff8f28c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:00.569 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.566+0000 7f2414e00700 1 --2- 172.21.15.86:0/1779947509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff8f297ab0 0x55ff8f299ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:00.569 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.566+0000 7f240f400700 1 -- 172.21.15.86:0/1779947509 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ff8f28c2d0 msgr2=0x55ff8f28c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:00:00.569 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.566+0000 7f240f400700 1 --2- 172.21.15.86:0/1779947509 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ff8f28c2d0 0x55ff8f28c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:00.569 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.566+0000 7f240f400700 1 -- 172.21.15.86:0/1779947509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff8f297ab0 msgr2=0x55ff8f299ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:00:00.569 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.566+0000 7f240f400700 1 --2- 172.21.15.86:0/1779947509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff8f297ab0 0x55ff8f299ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:00.569 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.566+0000 7f240f400700 1 -- 172.21.15.86:0/1779947509 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff8f060350 con 0x55ff8f28cbf0 2024-03-23T18:00:00.569 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.566+0000 7f240f400700 1 --2- 172.21.15.86:0/1779947509 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff8f28cbf0 0x55ff8f297570 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f23fc0a2fb0 tx=0x7f23fc022c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0c93ad63349da19 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:00:00.569 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.566+0000 7f2414e00700 1 --2- 172.21.15.86:0/1408245520 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f23f40a1310 0x7f23f40a37d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2408009fd0 tx=0x7f2408009920 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:00:00.569 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.566+0000 7f2405600700 1 -- 172.21.15.86:0/1779947509 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23fc09b0f0 con 0x55ff8f28cbf0 2024-03-23T18:00:00.569 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.566+0000 7f2405600700 1 -- 172.21.15.86:0/1779947509 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23fc09ef20 con 0x55ff8f28cbf0 2024-03-23T18:00:00.569 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.566+0000 7f2405600700 1 -- 172.21.15.86:0/1779947509 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23fc09b350 con 0x55ff8f28cbf0 2024-03-23T18:00:00.575 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.567+0000 7f242a87eb80 1 -- 172.21.15.86:0/1779947509 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff8f28cbf0 msgr2=0x55ff8f297570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:00:00.575 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.567+0000 7f242a87eb80 1 --2- 172.21.15.86:0/1779947509 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff8f28cbf0 0x55ff8f297570 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f23fc0a2fb0 tx=0x7f23fc022c90 comp rx=0 tx=0).stop 2024-03-23T18:00:00.575 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.567+0000 7f242a87eb80 1 -- 172.21.15.86:0/1779947509 shutdown_connections 2024-03-23T18:00:00.576 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.567+0000 7f242a87eb80 1 --2- 172.21.15.86:0/1779947509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff8f297ab0 0x55ff8f299ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:00.576 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.567+0000 7f242a87eb80 1 --2- 172.21.15.86:0/1779947509 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ff8f28c2d0 0x55ff8f28c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:00.576 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.567+0000 7f242a87eb80 1 --2- 172.21.15.86:0/1779947509 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff8f28cbf0 0x55ff8f297570 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:00.576 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.567+0000 7f242a87eb80 1 -- 172.21.15.86:0/1779947509 >> 172.21.15.86:0/1779947509 conn(0x55ff8f1ec640 msgr2=0x55ff8f288750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:00:00.576 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.567+0000 7f242a87eb80 1 -- 172.21.15.86:0/1779947509 shutdown_connections 2024-03-23T18:00:00.576 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.567+0000 7f242a87eb80 1 -- 172.21.15.86:0/1779947509 wait complete. 2024-03-23T18:00:00.576 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.568+0000 7f242a87eb80 1 -- start start 2024-03-23T18:00:00.576 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.568+0000 7f242a87eb80 1 Processor -- start 2024-03-23T18:00:00.576 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.569+0000 7f242a87eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff8f28c2d0 0x55ff8f298080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:00.576 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.569+0000 7f242a87eb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ff8f28cbf0 0x55ff8f2985c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:00.576 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.569+0000 7f242a87eb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff8f297ab0 0x55ff8f2a2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:00.576 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.569+0000 7f242a87eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff8f17ad20 con 0x55ff8f28c2d0 2024-03-23T18:00:00.576 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.569+0000 7f242a87eb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff8f17a9c0 con 0x55ff8f297ab0 2024-03-23T18:00:00.576 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.569+0000 7f242a87eb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff8f062640 con 0x55ff8f28cbf0 2024-03-23T18:00:00.577 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.569+0000 7f242a87eb80 10 client.0 _ll_register_callbacks cb 0x55ff8f184410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T18:00:00.577 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.569+0000 7f240fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff8f28c2d0 0x55ff8f298080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:00.577 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.569+0000 7f240fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff8f28c2d0 0x55ff8f298080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59358/0 (socket says 172.21.15.86:59358) 2024-03-23T18:00:00.577 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.569+0000 7f240fe00700 1 -- 172.21.15.86:0/3113244185 learned_addr learned my addr 172.21.15.86:0/3113244185 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:00:00.577 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.569+0000 7f240f400700 1 --2- 172.21.15.86:0/3113244185 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ff8f28cbf0 0x55ff8f2985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:00.577 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.570+0000 7f2414e00700 1 --2- 172.21.15.86:0/3113244185 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff8f297ab0 0x55ff8f2a2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:00.577 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.570+0000 7f240f400700 1 -- 172.21.15.86:0/3113244185 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff8f297ab0 msgr2=0x55ff8f2a2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:00:00.577 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.570+0000 7f240f400700 1 --2- 172.21.15.86:0/3113244185 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff8f297ab0 0x55ff8f2a2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:00.577 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.570+0000 7f240f400700 1 -- 172.21.15.86:0/3113244185 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff8f28c2d0 msgr2=0x55ff8f298080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:00:00.577 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.570+0000 7f240f400700 1 --2- 172.21.15.86:0/3113244185 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff8f28c2d0 0x55ff8f298080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:00.577 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.570+0000 7f240f400700 1 -- 172.21.15.86:0/3113244185 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24101a5dd0 con 0x55ff8f28cbf0 2024-03-23T18:00:00.577 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.570+0000 7f240fe00700 1 --2- 172.21.15.86:0/3113244185 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff8f28c2d0 0x55ff8f298080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T18:00:00.577 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.570+0000 7f240f400700 1 --2- 172.21.15.86:0/3113244185 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ff8f28cbf0 0x55ff8f2985c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f23fc000c00 tx=0x7f23fc005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78acd0e750dc9d72 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:00:00.577 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.570+0000 7f23fa000700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-03-23T18:00:00.578 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.570+0000 7f23fa000700 1 -- 172.21.15.86:0/3113244185 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23fc0a34e0 con 0x55ff8f28cbf0 2024-03-23T18:00:00.578 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.570+0000 7f23fa000700 1 -- 172.21.15.86:0/3113244185 <== mon.1 v2:172.21.15.152:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f23fc09fe40 con 0x55ff8f28cbf0 2024-03-23T18:00:00.578 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.570+0000 7f242a87eb80 1 -- 172.21.15.86:0/3113244185 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ff8f19f320 con 0x55ff8f28cbf0 2024-03-23T18:00:00.578 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.571+0000 7f23fa000700 1 -- 172.21.15.86:0/3113244185 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23fc09b0f0 con 0x55ff8f28cbf0 2024-03-23T18:00:00.578 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.571+0000 7f242a87eb80 10 client.26764 fetch_fsmap learned FSMap version 45 2024-03-23T18:00:00.578 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.571+0000 7f242a87eb80 1 -- 172.21.15.86:0/3113244185 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x55ff8f00cb00 con 0x55ff8f28cbf0 2024-03-23T18:00:00.578 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.571+0000 7f23fa000700 1 -- 172.21.15.86:0/3113244185 <== mon.1 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ff8f19f320 con 0x55ff8f28cbf0 2024-03-23T18:00:00.578 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.571+0000 7f23fa000700 1 -- 172.21.15.86:0/3113244185 <== mon.1 v2:172.21.15.152:3300/0 5 ==== fsmap(e 45) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f23fc09b350 con 0x55ff8f28cbf0 2024-03-23T18:00:00.578 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.571+0000 7f242a87eb80 10 client.26764 fetch_fsmap finished waiting for FSMap version 45 2024-03-23T18:00:00.578 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.571+0000 7f242a87eb80 10 client.26764 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T18:00:00.578 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.571+0000 7f242a87eb80 20 client.26764 populate_metadata read hostname 'smithi086' 2024-03-23T18:00:00.578 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.571+0000 7f242a87eb80 1 --2- 172.21.15.86:0/3113244185 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55ff8f2a6f90 0x55ff8f2a93b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:00.578 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.571+0000 7f242a87eb80 4 client.26764 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T18:00:00.579 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.571+0000 7f242a87eb80 1 -- 172.21.15.86:0/3113244185 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ff8f1d76a0 con 0x55ff8f2a6f90 2024-03-23T18:00:00.579 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.571+0000 7f240fe00700 1 --2- 172.21.15.86:0/3113244185 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55ff8f2a6f90 0x55ff8f2a93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:00.579 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.572+0000 7f240fe00700 1 --2- 172.21.15.86:0/3113244185 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55ff8f2a6f90 0x55ff8f2a93b0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f47ba83df2e29f4 server_cookie=3c2eb0d949ff2c67 in_seq=0 out_seq=0 2024-03-23T18:00:00.579 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.572+0000 7f23fa000700 10 client.26764 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T18:00:00.579 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.572+0000 7f23fa000700 1 -- 172.21.15.86:0/3113244185 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55ff8f1d76a0 con 0x55ff8f2a6f90 2024-03-23T18:00:00.579 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.572+0000 7f23fa000700 10 client.26764 handle_command_reply: tid=0 2024-03-23T18:00:00.579 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.573+0000 7f242a87eb80 1 client.26764 shutdown 2024-03-23T18:00:00.579 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.573+0000 7f242a87eb80 1 -- 172.21.15.86:0/3113244185 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ff8f28cbf0 msgr2=0x55ff8f2985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:00:00.579 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.573+0000 7f242a87eb80 1 --2- 172.21.15.86:0/3113244185 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ff8f28cbf0 0x55ff8f2985c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f23fc000c00 tx=0x7f23fc005e80 comp rx=0 tx=0).stop 2024-03-23T18:00:00.579 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.573+0000 7f242a87eb80 1 -- 172.21.15.86:0/3113244185 shutdown_connections 2024-03-23T18:00:00.579 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.573+0000 7f242a87eb80 1 --2- 172.21.15.86:0/3113244185 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55ff8f2a6f90 0x55ff8f2a93b0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:00.579 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.573+0000 7f242a87eb80 1 --2- 172.21.15.86:0/3113244185 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff8f297ab0 0x55ff8f2a2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:00.579 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.573+0000 7f242a87eb80 1 --2- 172.21.15.86:0/3113244185 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff8f28c2d0 0x55ff8f298080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:00.579 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.573+0000 7f242a87eb80 1 --2- 172.21.15.86:0/3113244185 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ff8f28cbf0 0x55ff8f2985c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:00.580 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.573+0000 7f242a87eb80 1 -- 172.21.15.86:0/3113244185 >> 172.21.15.86:0/3113244185 conn(0x55ff8f1ec640 msgr2=0x55ff8f28e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:00:00.580 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.573+0000 7f23fa000700 0 client.26764 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T18:00:00.580 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.573+0000 7f242a87eb80 1 -- 172.21.15.86:0/3113244185 shutdown_connections 2024-03-23T18:00:00.580 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.573+0000 7f242a87eb80 1 -- 172.21.15.86:0/3113244185 wait complete. 2024-03-23T18:00:00.580 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.573+0000 7f242a87eb80 20 client.26764 trim_cache size 0 max 16384 2024-03-23T18:00:00.591 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.590+0000 7f242a87eb80 1 Processor -- start 2024-03-23T18:00:00.591 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.590+0000 7f242a87eb80 1 -- start start 2024-03-23T18:00:00.591 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.590+0000 7f242a87eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff8f28c2d0 0x55ff8f29ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:00.591 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.590+0000 7f242a87eb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ff8f28cbf0 0x55ff8f297e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:00.591 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.590+0000 7f242a87eb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff8f297ab0 0x55ff8f2983d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:00.591 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.590+0000 7f242a87eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff8f284de0 con 0x55ff8f28c2d0 2024-03-23T18:00:00.591 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.590+0000 7f242a87eb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff8f289c10 con 0x55ff8f297ab0 2024-03-23T18:00:00.592 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.590+0000 7f242a87eb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff8f28c110 con 0x55ff8f28cbf0 2024-03-23T18:00:00.593 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.591+0000 7f2414e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff8f297ab0 0x55ff8f2983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:00.593 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.591+0000 7f240fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff8f28c2d0 0x55ff8f29ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:00.593 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.591+0000 7f2414e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff8f297ab0 0x55ff8f2983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:46776/0 (socket says 172.21.15.86:46776) 2024-03-23T18:00:00.593 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.591+0000 7f2414e00700 1 -- 172.21.15.86:0/455100468 learned_addr learned my addr 172.21.15.86:0/455100468 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:00:00.593 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.591+0000 7f240f400700 1 --2- 172.21.15.86:0/455100468 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ff8f28cbf0 0x55ff8f297e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:00.593 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.591+0000 7f2414e00700 1 -- 172.21.15.86:0/455100468 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ff8f28cbf0 msgr2=0x55ff8f297e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:00:00.593 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.591+0000 7f2414e00700 1 --2- 172.21.15.86:0/455100468 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ff8f28cbf0 0x55ff8f297e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:00.594 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.591+0000 7f2414e00700 1 -- 172.21.15.86:0/455100468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff8f28c2d0 msgr2=0x55ff8f29ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:00:00.594 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.591+0000 7f2414e00700 1 --2- 172.21.15.86:0/455100468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff8f28c2d0 0x55ff8f29ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:00.594 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.591+0000 7f2414e00700 1 -- 172.21.15.86:0/455100468 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff8f1a4e40 con 0x55ff8f297ab0 2024-03-23T18:00:00.594 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.591+0000 7f240f400700 1 --2- 172.21.15.86:0/455100468 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ff8f28cbf0 0x55ff8f297e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T18:00:00.594 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.591+0000 7f2414e00700 1 --2- 172.21.15.86:0/455100468 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff8f297ab0 0x55ff8f2983d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f2408009000 tx=0x7f240800b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c26f35ce1cb14f7a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:00:00.594 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.591+0000 7f2406000700 1 -- 172.21.15.86:0/455100468 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2408025680 con 0x55ff8f297ab0 2024-03-23T18:00:00.594 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.591+0000 7f2406000700 1 -- 172.21.15.86:0/455100468 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f240800f040 con 0x55ff8f297ab0 2024-03-23T18:00:00.594 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.592+0000 7f2406000700 1 -- 172.21.15.86:0/455100468 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f240801abb0 con 0x55ff8f297ab0 2024-03-23T18:00:00.594 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.592+0000 7f242a87eb80 1 -- 172.21.15.86:0/455100468 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff8f297ab0 msgr2=0x55ff8f2983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:00:00.594 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.592+0000 7f242a87eb80 1 --2- 172.21.15.86:0/455100468 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff8f297ab0 0x55ff8f2983d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f2408009000 tx=0x7f240800b040 comp rx=0 tx=0).stop 2024-03-23T18:00:00.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.592+0000 7f242a87eb80 1 -- 172.21.15.86:0/455100468 shutdown_connections 2024-03-23T18:00:00.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.592+0000 7f242a87eb80 1 --2- 172.21.15.86:0/455100468 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff8f297ab0 0x55ff8f2983d0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:00.598 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.592+0000 7f242a87eb80 1 --2- 172.21.15.86:0/455100468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff8f28c2d0 0x55ff8f29ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:00.598 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.592+0000 7f242a87eb80 1 --2- 172.21.15.86:0/455100468 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ff8f28cbf0 0x55ff8f297e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:00.598 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.592+0000 7f242a87eb80 1 -- 172.21.15.86:0/455100468 >> 172.21.15.86:0/455100468 conn(0x55ff8f1ec640 msgr2=0x55ff8f2912b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:00:00.598 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.593+0000 7f242a87eb80 1 -- 172.21.15.86:0/455100468 shutdown_connections 2024-03-23T18:00:00.598 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.593+0000 7f242a87eb80 1 -- 172.21.15.86:0/455100468 wait complete. 2024-03-23T18:00:00.598 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.593+0000 7f242a87eb80 1 -- start start 2024-03-23T18:00:00.598 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.593+0000 7f242a87eb80 1 Processor -- start 2024-03-23T18:00:00.598 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.594+0000 7f242a87eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff8f28c2d0 0x55ff8f2eca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:00.598 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.594+0000 7f242a87eb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff8f28cbf0 0x55ff8f2ecf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:00.598 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.594+0000 7f242a87eb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ff8f297ab0 0x55ff8f2f7a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:00.598 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.594+0000 7f242a87eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff8f2ed6b0 con 0x55ff8f28c2d0 2024-03-23T18:00:00.598 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.594+0000 7f242a87eb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff8f2ed820 con 0x55ff8f28cbf0 2024-03-23T18:00:00.598 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.594+0000 7f242a87eb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff8f2f7fc0 con 0x55ff8f297ab0 2024-03-23T18:00:00.599 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.594+0000 7f242a87eb80 10 client.0 _ll_register_callbacks cb 0x55ff8f184410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T18:00:00.599 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.594+0000 7f240f400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff8f28cbf0 0x55ff8f2ecf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:00.599 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.594+0000 7f240f400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff8f28cbf0 0x55ff8f2ecf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:46784/0 (socket says 172.21.15.86:46784) 2024-03-23T18:00:00.599 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.594+0000 7f240f400700 1 -- 172.21.15.86:0/4147609598 learned_addr learned my addr 172.21.15.86:0/4147609598 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:00:00.599 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.594+0000 7f240fe00700 1 --2- 172.21.15.86:0/4147609598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff8f28c2d0 0x55ff8f2eca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:00.599 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.594+0000 7f2414e00700 1 --2- 172.21.15.86:0/4147609598 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ff8f297ab0 0x55ff8f2f7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:00.599 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.594+0000 7f240f400700 1 -- 172.21.15.86:0/4147609598 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ff8f297ab0 msgr2=0x55ff8f2f7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:00:00.599 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.594+0000 7f240f400700 1 --2- 172.21.15.86:0/4147609598 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ff8f297ab0 0x55ff8f2f7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:00.599 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.594+0000 7f240f400700 1 -- 172.21.15.86:0/4147609598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff8f28c2d0 msgr2=0x55ff8f2eca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:00:00.599 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.594+0000 7f240f400700 1 --2- 172.21.15.86:0/4147609598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff8f28c2d0 0x55ff8f2eca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:00.599 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.594+0000 7f240f400700 1 -- 172.21.15.86:0/4147609598 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2408008d70 con 0x55ff8f28cbf0 2024-03-23T18:00:00.599 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.594+0000 7f2414e00700 1 --2- 172.21.15.86:0/4147609598 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ff8f297ab0 0x55ff8f2f7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T18:00:00.599 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.595+0000 7f240f400700 1 --2- 172.21.15.86:0/4147609598 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff8f28cbf0 0x55ff8f2ecf40 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f23fc099440 tx=0x7f23fc022ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e0d8d221f23d3f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:00:00.599 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.595+0000 7f23fa000700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T18:00:00.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.595+0000 7f23fa000700 1 -- 172.21.15.86:0/4147609598 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23fc09a7e0 con 0x55ff8f28cbf0 2024-03-23T18:00:00.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.595+0000 7f23fa000700 1 -- 172.21.15.86:0/4147609598 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f23fc09fe40 con 0x55ff8f28cbf0 2024-03-23T18:00:00.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.595+0000 7f242a87eb80 1 -- 172.21.15.86:0/4147609598 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f23fc09ef20 con 0x55ff8f28cbf0 2024-03-23T18:00:00.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.595+0000 7f23fa000700 1 -- 172.21.15.86:0/4147609598 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23fc0a30f0 con 0x55ff8f28cbf0 2024-03-23T18:00:00.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.595+0000 7f242a87eb80 10 client.26789 fetch_fsmap learned FSMap version 45 2024-03-23T18:00:00.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.595+0000 7f242a87eb80 1 -- 172.21.15.86:0/4147609598 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x55ff8f064c70 con 0x55ff8f28cbf0 2024-03-23T18:00:00.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.595+0000 7f23fa000700 1 -- 172.21.15.86:0/4147609598 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23fc09ef20 con 0x55ff8f28cbf0 2024-03-23T18:00:00.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.596+0000 7f23fa000700 1 -- 172.21.15.86:0/4147609598 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 45) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f23fc00fb50 con 0x55ff8f28cbf0 2024-03-23T18:00:00.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.596+0000 7f242a87eb80 10 client.26789 fetch_fsmap finished waiting for FSMap version 45 2024-03-23T18:00:00.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.596+0000 7f242a87eb80 10 client.26789 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T18:00:00.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.596+0000 7f242a87eb80 20 client.26789 populate_metadata read hostname 'smithi086' 2024-03-23T18:00:00.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.596+0000 7f242a87eb80 1 --2- 172.21.15.86:0/4147609598 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55ff8f2f9290 0x55ff8f2fb680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:00.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.596+0000 7f242a87eb80 4 client.26789 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T18:00:00.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.596+0000 7f242a87eb80 1 -- 172.21.15.86:0/4147609598 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ff8f02dc90 con 0x55ff8f2f9290 2024-03-23T18:00:00.601 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.596+0000 7f240fe00700 1 --2- 172.21.15.86:0/4147609598 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55ff8f2f9290 0x55ff8f2fb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:00.601 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.596+0000 7f23fa000700 10 client.26789 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T18:00:00.601 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.596+0000 7f240fe00700 1 --2- 172.21.15.86:0/4147609598 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55ff8f2f9290 0x55ff8f2fb680 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=656140be923fe2cd server_cookie=2c1197df3dcdb3f6 in_seq=0 out_seq=0 2024-03-23T18:00:00.639 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.637+0000 7f23fa000700 1 -- 172.21.15.86:0/4147609598 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ff8f02dc90 con 0x55ff8f2f9290 2024-03-23T18:00:00.639 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.637+0000 7f23fa000700 10 client.26789 handle_command_reply: tid=0 2024-03-23T18:00:00.639 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.637+0000 7f242a87eb80 1 client.26789 shutdown 2024-03-23T18:00:00.639 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.637+0000 7f242a87eb80 1 -- 172.21.15.86:0/4147609598 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff8f28cbf0 msgr2=0x55ff8f2ecf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:00:00.639 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.637+0000 7f242a87eb80 1 --2- 172.21.15.86:0/4147609598 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff8f28cbf0 0x55ff8f2ecf40 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f23fc099440 tx=0x7f23fc022ad0 comp rx=0 tx=0).stop 2024-03-23T18:00:00.639 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.638+0000 7f242a87eb80 1 -- 172.21.15.86:0/4147609598 shutdown_connections 2024-03-23T18:00:00.640 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.638+0000 7f242a87eb80 1 --2- 172.21.15.86:0/4147609598 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55ff8f2f9290 0x55ff8f2fb680 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:00.640 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.638+0000 7f242a87eb80 1 --2- 172.21.15.86:0/4147609598 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ff8f297ab0 0x55ff8f2f7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:00.640 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.638+0000 7f242a87eb80 1 --2- 172.21.15.86:0/4147609598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff8f28c2d0 0x55ff8f2eca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:00.640 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.638+0000 7f23fa000700 0 client.26789 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T18:00:00.640 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.638+0000 7f242a87eb80 1 --2- 172.21.15.86:0/4147609598 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff8f28cbf0 0x55ff8f2ecf40 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:00.640 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.638+0000 7f242a87eb80 1 -- 172.21.15.86:0/4147609598 >> 172.21.15.86:0/4147609598 conn(0x55ff8f1ec640 msgr2=0x55ff8f297e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:00:00.640 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.638+0000 7f242a87eb80 1 -- 172.21.15.86:0/4147609598 shutdown_connections 2024-03-23T18:00:00.640 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.638+0000 7f242a87eb80 1 -- 172.21.15.86:0/4147609598 wait complete. 2024-03-23T18:00:00.640 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.638+0000 7f242a87eb80 20 client.26789 trim_cache size 0 max 16384 2024-03-23T18:00:00.640 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-03-23T18:00:00.641 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-03-23T18:00:00.641 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-03-23T18:00:00.641 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-03-23T18:00:00.641 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.639+0000 7f2415800700 1 -- 172.21.15.86:0/1408245520 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f23f40a1310 msgr2=0x7f23f40a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:00:00.641 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.639+0000 7f2415800700 1 --2- 172.21.15.86:0/1408245520 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f23f40a1310 0x7f23f40a37d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2408009fd0 tx=0x7f2408009920 comp rx=0 tx=0).stop 2024-03-23T18:00:00.641 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.639+0000 7f2415800700 1 -- 172.21.15.86:0/1408245520 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2410101490 msgr2=0x7f24100735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:00:00.642 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.639+0000 7f2415800700 1 --2- 172.21.15.86:0/1408245520 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2410101490 0x7f24100735f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f23fc005910 tx=0x7f23fc004730 comp rx=0 tx=0).stop 2024-03-23T18:00:00.642 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.639+0000 7f2415800700 1 -- 172.21.15.86:0/1408245520 shutdown_connections 2024-03-23T18:00:00.642 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.639+0000 7f2415800700 1 --2- 172.21.15.86:0/1408245520 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2410106b70 0x7f2410073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:00.642 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.639+0000 7f2415800700 1 --2- 172.21.15.86:0/1408245520 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f23f40a1310 0x7f23f40a37d0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:00.642 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.639+0000 7f2415800700 1 --2- 172.21.15.86:0/1408245520 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2410100ad0 0x7f24100730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:00.642 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.639+0000 7f2415800700 1 --2- 172.21.15.86:0/1408245520 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2410101490 0x7f24100735f0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:00.642 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.639+0000 7f2415800700 1 -- 172.21.15.86:0/1408245520 >> 172.21.15.86:0/1408245520 conn(0x7f24100fc5e0 msgr2=0x7f24100fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:00:00.642 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.639+0000 7f2415800700 1 -- 172.21.15.86:0/1408245520 shutdown_connections 2024-03-23T18:00:00.642 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:00.639+0000 7f2415800700 1 -- 172.21.15.86:0/1408245520 wait complete. 2024-03-23T18:00:00.917 INFO:tasks.workunit.client.0.smithi086.stdout: 1 513273 6.08 MB/sec execute 385 sec latency 34.432 ms 2024-03-23T18:00:01.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:00:00 smithi086 ceph-mon[38221]: Health detail: HEALTH_ERR 3 MDSs report damaged metadata 2024-03-23T18:00:01.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:00:00 smithi086 ceph-mon[38221]: [ERR] MDS_DAMAGE: 3 MDSs report damaged metadata 2024-03-23T18:00:01.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:00:00 smithi086 ceph-mon[38221]: mds.e(mds.4): Metadata damage detected 2024-03-23T18:00:01.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:00:00 smithi086 ceph-mon[38221]: mds.d(mds.3): Metadata damage detected 2024-03-23T18:00:01.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:00:00 smithi086 ceph-mon[38221]: mds.h(mds.2): Metadata damage detected 2024-03-23T18:00:01.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:00:00 smithi092 ceph-mon[79089]: Health detail: HEALTH_ERR 3 MDSs report damaged metadata 2024-03-23T18:00:01.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:00:00 smithi092 ceph-mon[79089]: [ERR] MDS_DAMAGE: 3 MDSs report damaged metadata 2024-03-23T18:00:01.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:00:00 smithi092 ceph-mon[79089]: mds.e(mds.4): Metadata damage detected 2024-03-23T18:00:01.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:00:00 smithi092 ceph-mon[79089]: mds.d(mds.3): Metadata damage detected 2024-03-23T18:00:01.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:00:00 smithi092 ceph-mon[79089]: mds.h(mds.2): Metadata damage detected 2024-03-23T18:00:01.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:00:00 smithi152 ceph-mon[47949]: Health detail: HEALTH_ERR 3 MDSs report damaged metadata 2024-03-23T18:00:01.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:00:00 smithi152 ceph-mon[47949]: [ERR] MDS_DAMAGE: 3 MDSs report damaged metadata 2024-03-23T18:00:01.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:00:00 smithi152 ceph-mon[47949]: mds.e(mds.4): Metadata damage detected 2024-03-23T18:00:01.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:00:00 smithi152 ceph-mon[47949]: mds.d(mds.3): Metadata damage detected 2024-03-23T18:00:01.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:00:00 smithi152 ceph-mon[47949]: mds.h(mds.2): Metadata damage detected 2024-03-23T18:00:01.917 INFO:tasks.workunit.client.0.smithi086.stdout: 1 514748 6.08 MB/sec execute 386 sec latency 27.679 ms 2024-03-23T18:00:02.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:00:01 smithi086 ceph-mon[38221]: pgmap v290: 97 pgs: 97 active+clean; 212 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.4 MiB/s wr, 312 op/s 2024-03-23T18:00:02.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:00:01 smithi092 ceph-mon[79089]: pgmap v290: 97 pgs: 97 active+clean; 212 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.4 MiB/s wr, 312 op/s 2024-03-23T18:00:02.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:00:01 smithi152 ceph-mon[47949]: pgmap v290: 97 pgs: 97 active+clean; 212 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.4 MiB/s wr, 312 op/s 2024-03-23T18:00:02.917 INFO:tasks.workunit.client.0.smithi086.stdout: 1 515835 6.08 MB/sec execute 387 sec latency 102.097 ms 2024-03-23T18:00:03.917 INFO:tasks.workunit.client.0.smithi086.stdout: 1 517602 6.09 MB/sec execute 388 sec latency 23.965 ms 2024-03-23T18:00:04.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:00:03 smithi086 ceph-mon[38221]: pgmap v291: 97 pgs: 97 active+clean; 212 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 186 op/s 2024-03-23T18:00:04.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:00:03 smithi092 ceph-mon[79089]: pgmap v291: 97 pgs: 97 active+clean; 212 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 186 op/s 2024-03-23T18:00:04.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:00:03 smithi152 ceph-mon[47949]: pgmap v291: 97 pgs: 97 active+clean; 212 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 186 op/s 2024-03-23T18:00:04.917 INFO:tasks.workunit.client.0.smithi086.stdout: 1 519146 6.10 MB/sec execute 389 sec latency 16.280 ms 2024-03-23T18:00:05.917 INFO:tasks.workunit.client.0.smithi086.stdout: 1 520652 6.10 MB/sec execute 390 sec latency 31.818 ms 2024-03-23T18:00:06.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:00:05 smithi086 ceph-mon[38221]: pgmap v292: 97 pgs: 97 active+clean; 207 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.1 MiB/s wr, 332 op/s 2024-03-23T18:00:06.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:00:05 smithi086 ceph-mon[38221]: osdmap e90: 12 total, 12 up, 12 in 2024-03-23T18:00:06.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:00:05 smithi092 ceph-mon[79089]: pgmap v292: 97 pgs: 97 active+clean; 207 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.1 MiB/s wr, 332 op/s 2024-03-23T18:00:06.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:00:05 smithi092 ceph-mon[79089]: osdmap e90: 12 total, 12 up, 12 in 2024-03-23T18:00:06.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:00:05 smithi152 ceph-mon[47949]: pgmap v292: 97 pgs: 97 active+clean; 207 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.1 MiB/s wr, 332 op/s 2024-03-23T18:00:06.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:00:05 smithi152 ceph-mon[47949]: osdmap e90: 12 total, 12 up, 12 in 2024-03-23T18:00:06.917 INFO:tasks.workunit.client.0.smithi086.stdout: 1 522542 6.12 MB/sec execute 391 sec latency 13.935 ms 2024-03-23T18:00:07.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:00:06 smithi086 ceph-mon[38221]: pgmap v294: 97 pgs: 97 active+clean; 207 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 278 op/s 2024-03-23T18:00:07.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:00:06 smithi092 ceph-mon[79089]: pgmap v294: 97 pgs: 97 active+clean; 207 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 278 op/s 2024-03-23T18:00:07.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:00:06 smithi152 ceph-mon[47949]: pgmap v294: 97 pgs: 97 active+clean; 207 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 278 op/s 2024-03-23T18:00:07.917 INFO:tasks.workunit.client.0.smithi086.stdout: 1 523991 6.12 MB/sec execute 392 sec latency 17.343 ms 2024-03-23T18:00:08.918 INFO:tasks.workunit.client.0.smithi086.stdout: 1 525997 6.13 MB/sec execute 393 sec latency 26.651 ms 2024-03-23T18:00:09.836 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:00:09 smithi152 ceph-mon[47949]: pgmap v295: 97 pgs: 97 active+clean; 207 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 278 op/s 2024-03-23T18:00:09.918 INFO:tasks.workunit.client.0.smithi086.stdout: 1 527233 6.13 MB/sec execute 394 sec latency 19.761 ms 2024-03-23T18:00:10.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:00:09 smithi086 ceph-mon[38221]: pgmap v295: 97 pgs: 97 active+clean; 207 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 278 op/s 2024-03-23T18:00:10.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:00:09 smithi092 ceph-mon[79089]: pgmap v295: 97 pgs: 97 active+clean; 207 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 278 op/s 2024-03-23T18:00:10.918 INFO:tasks.workunit.client.0.smithi086.stdout: 1 529289 6.15 MB/sec execute 395 sec latency 16.315 ms 2024-03-23T18:00:11.918 INFO:tasks.workunit.client.0.smithi086.stdout: 1 530472 6.14 MB/sec execute 396 sec latency 12.313 ms 2024-03-23T18:00:12.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:00:11 smithi152 ceph-mon[47949]: pgmap v296: 97 pgs: 97 active+clean; 218 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 298 op/s 2024-03-23T18:00:12.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:00:11 smithi152 ceph-mon[47949]: osdmap e91: 12 total, 12 up, 12 in 2024-03-23T18:00:12.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:00:11 smithi086 ceph-mon[38221]: pgmap v296: 97 pgs: 97 active+clean; 218 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 298 op/s 2024-03-23T18:00:12.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:00:11 smithi086 ceph-mon[38221]: osdmap e91: 12 total, 12 up, 12 in 2024-03-23T18:00:12.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:00:11 smithi092 ceph-mon[79089]: pgmap v296: 97 pgs: 97 active+clean; 218 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 298 op/s 2024-03-23T18:00:12.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:00:11 smithi092 ceph-mon[79089]: osdmap e91: 12 total, 12 up, 12 in 2024-03-23T18:00:12.918 INFO:tasks.workunit.client.0.smithi086.stdout: 1 532330 6.15 MB/sec execute 397 sec latency 42.201 ms 2024-03-23T18:00:13.804 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.804+0000 7f2099c00700 1 -- 172.21.15.86:0/2579018516 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20941018c0 msgr2=0x7f2094105900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:00:13.804 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.804+0000 7f2099c00700 1 --2- 172.21.15.86:0/2579018516 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20941018c0 0x7f2094105900 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2080009990 tx=0x7f2080009ca0 comp rx=0 tx=0).stop 2024-03-23T18:00:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.804+0000 7f2099c00700 1 -- 172.21.15.86:0/2579018516 shutdown_connections 2024-03-23T18:00:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.804+0000 7f2099c00700 1 --2- 172.21.15.86:0/2579018516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2094106030 0x7f209410dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.804+0000 7f2099c00700 1 --2- 172.21.15.86:0/2579018516 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2094100f10 0x7f20941012f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.804+0000 7f2099c00700 1 --2- 172.21.15.86:0/2579018516 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20941018c0 0x7f2094105900 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.804+0000 7f2099c00700 1 -- 172.21.15.86:0/2579018516 >> 172.21.15.86:0/2579018516 conn(0x7f20940fc770 msgr2=0x7f20940feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:00:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.804+0000 7f2099c00700 1 -- 172.21.15.86:0/2579018516 shutdown_connections 2024-03-23T18:00:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.804+0000 7f2099c00700 1 -- 172.21.15.86:0/2579018516 wait complete. 2024-03-23T18:00:13.807 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.805+0000 7f2099c00700 1 Processor -- start 2024-03-23T18:00:13.807 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.805+0000 7f2099c00700 1 -- start start 2024-03-23T18:00:13.807 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.806+0000 7f2099c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2094100f10 0x7f2094199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:13.807 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.806+0000 7f2099c00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2094106030 0x7f2094199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:13.807 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.806+0000 7f2099c00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f209419dd20 0x7f209419e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:13.807 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.806+0000 7f2099c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f209419e7a0 con 0x7f2094100f10 2024-03-23T18:00:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.806+0000 7f2099c00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f209419e910 con 0x7f209419dd20 2024-03-23T18:00:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.806+0000 7f2099c00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f209419ead0 con 0x7f2094106030 2024-03-23T18:00:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.806+0000 7f2092a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2094100f10 0x7f2094199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.806+0000 7f2092a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2094100f10 0x7f2094199440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58340/0 (socket says 172.21.15.86:58340) 2024-03-23T18:00:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.806+0000 7f2092a00700 1 -- 172.21.15.86:0/944676341 learned_addr learned my addr 172.21.15.86:0/944676341 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:00:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.806+0000 7f2092000700 1 --2- 172.21.15.86:0/944676341 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2094106030 0x7f2094199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.806+0000 7f2093400700 1 --2- 172.21.15.86:0/944676341 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f209419dd20 0x7f209419e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.806+0000 7f2092a00700 1 -- 172.21.15.86:0/944676341 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2094106030 msgr2=0x7f2094199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:00:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.806+0000 7f2092a00700 1 --2- 172.21.15.86:0/944676341 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2094106030 0x7f2094199980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.806+0000 7f2092a00700 1 -- 172.21.15.86:0/944676341 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f209419dd20 msgr2=0x7f209419e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:00:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.806+0000 7f2092a00700 1 --2- 172.21.15.86:0/944676341 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f209419dd20 0x7f209419e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.806+0000 7f2092a00700 1 -- 172.21.15.86:0/944676341 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20800095f0 con 0x7f2094100f10 2024-03-23T18:00:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.807+0000 7f2092a00700 1 --2- 172.21.15.86:0/944676341 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2094100f10 0x7f2094199440 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f208400d950 tx=0x7f208400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:00:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.807+0000 7f208b400700 1 -- 172.21.15.86:0/944676341 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f208400f890 con 0x7f2094100f10 2024-03-23T18:00:13.809 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.807+0000 7f208b400700 1 -- 172.21.15.86:0/944676341 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2084010460 con 0x7f2094100f10 2024-03-23T18:00:13.809 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.807+0000 7f2099c00700 1 -- 172.21.15.86:0/944676341 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20941a5fd0 con 0x7f2094100f10 2024-03-23T18:00:13.809 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.807+0000 7f208b400700 1 -- 172.21.15.86:0/944676341 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f208400e610 con 0x7f2094100f10 2024-03-23T18:00:13.809 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.807+0000 7f2099c00700 1 -- 172.21.15.86:0/944676341 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20941a64b0 con 0x7f2094100f10 2024-03-23T18:00:13.810 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.810+0000 7f208b400700 1 -- 172.21.15.86:0/944676341 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f20840108d0 con 0x7f2094100f10 2024-03-23T18:00:13.811 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.812+0000 7f208b400700 1 --2- 172.21.15.86:0/944676341 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f20780a1290 0x7f20780a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:13.812 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.812+0000 7f208b400700 1 -- 172.21.15.86:0/944676341 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f20840ccab0 con 0x7f2094100f10 2024-03-23T18:00:13.812 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.812+0000 7f20ad6cbb80 1 Processor -- start 2024-03-23T18:00:13.812 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.812+0000 7f2092000700 1 --2- 172.21.15.86:0/944676341 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f20780a1290 0x7f20780a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:13.812 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.812+0000 7f20ad6cbb80 1 -- start start 2024-03-23T18:00:13.814 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.813+0000 7f20ad6cbb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fec1292100 0x55fec12924e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:13.814 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.813+0000 7f20ad6cbb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fec1292a20 0x55fec129d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:13.814 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.813+0000 7f20ad6cbb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fec129d8d0 0x55fec129fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:13.814 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.813+0000 7f20ad6cbb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fec11eb930 con 0x55fec1292100 2024-03-23T18:00:13.814 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.813+0000 7f20ad6cbb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fec11e92f0 con 0x55fec129d8d0 2024-03-23T18:00:13.815 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.813+0000 7f20ad6cbb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fec11e4be0 con 0x55fec1292a20 2024-03-23T18:00:13.815 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.813+0000 7f2092000700 1 --2- 172.21.15.86:0/944676341 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f20780a1290 0x7f20780a3750 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f209419a990 tx=0x7f208000b260 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:00:13.815 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.813+0000 7f2092000700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fec1292a20 0x55fec129d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:13.815 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.813+0000 7f2092a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fec1292100 0x55fec12924e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:13.815 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.813+0000 7f2093400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fec129d8d0 0x55fec129fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:13.815 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.813+0000 7f2092000700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fec1292a20 0x55fec129d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:40466/0 (socket says 172.21.15.86:40466) 2024-03-23T18:00:13.815 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.813+0000 7f2092000700 1 -- 172.21.15.86:0/2696282662 learned_addr learned my addr 172.21.15.86:0/2696282662 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:00:13.815 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.813+0000 7f2093400700 1 -- 172.21.15.86:0/2696282662 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fec1292a20 msgr2=0x55fec129d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:00:13.815 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.813+0000 7f2093400700 1 --2- 172.21.15.86:0/2696282662 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fec1292a20 0x55fec129d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:13.815 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.813+0000 7f2093400700 1 -- 172.21.15.86:0/2696282662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fec1292100 msgr2=0x55fec12924e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:00:13.815 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.813+0000 7f2093400700 1 --2- 172.21.15.86:0/2696282662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fec1292100 0x55fec12924e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:13.815 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.813+0000 7f2093400700 1 -- 172.21.15.86:0/2696282662 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fec1066350 con 0x55fec129d8d0 2024-03-23T18:00:13.815 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.814+0000 7f2093400700 1 --2- 172.21.15.86:0/2696282662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fec129d8d0 0x55fec129fcc0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f208c011580 tx=0x7f208c009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f82710c5b8e7516 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:00:13.816 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.814+0000 7f207fe00700 1 -- 172.21.15.86:0/2696282662 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f208c00e040 con 0x55fec129d8d0 2024-03-23T18:00:13.816 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.814+0000 7f207fe00700 1 -- 172.21.15.86:0/2696282662 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f208c00f040 con 0x55fec129d8d0 2024-03-23T18:00:13.816 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.814+0000 7f207fe00700 1 -- 172.21.15.86:0/2696282662 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f208c0044f0 con 0x55fec129d8d0 2024-03-23T18:00:13.819 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.815+0000 7f20ad6cbb80 1 -- 172.21.15.86:0/2696282662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fec129d8d0 msgr2=0x55fec129fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:00:13.820 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.815+0000 7f20ad6cbb80 1 --2- 172.21.15.86:0/2696282662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fec129d8d0 0x55fec129fcc0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f208c011580 tx=0x7f208c009920 comp rx=0 tx=0).stop 2024-03-23T18:00:13.820 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.815+0000 7f20ad6cbb80 1 -- 172.21.15.86:0/2696282662 shutdown_connections 2024-03-23T18:00:13.820 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.815+0000 7f20ad6cbb80 1 --2- 172.21.15.86:0/2696282662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fec129d8d0 0x55fec129fcc0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:13.820 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.815+0000 7f20ad6cbb80 1 --2- 172.21.15.86:0/2696282662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fec1292100 0x55fec12924e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:13.820 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.815+0000 7f20ad6cbb80 1 --2- 172.21.15.86:0/2696282662 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fec1292a20 0x55fec129d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:13.820 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.815+0000 7f20ad6cbb80 1 -- 172.21.15.86:0/2696282662 >> 172.21.15.86:0/2696282662 conn(0x55fec11f2640 msgr2=0x55fec128e520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:00:13.820 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.815+0000 7f20ad6cbb80 1 -- 172.21.15.86:0/2696282662 shutdown_connections 2024-03-23T18:00:13.820 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.815+0000 7f20ad6cbb80 1 -- 172.21.15.86:0/2696282662 wait complete. 2024-03-23T18:00:13.820 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.816+0000 7f20ad6cbb80 1 -- start start 2024-03-23T18:00:13.820 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.816+0000 7f20ad6cbb80 1 Processor -- start 2024-03-23T18:00:13.820 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.816+0000 7f20ad6cbb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fec1292100 0x55fec129d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:13.821 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.816+0000 7f20ad6cbb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fec1292a20 0x55fec129be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:13.821 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.816+0000 7f20ad6cbb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fec129d8d0 0x55fec129c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:13.821 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.816+0000 7f20ad6cbb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fec1180d20 con 0x55fec1292100 2024-03-23T18:00:13.821 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.816+0000 7f20ad6cbb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fec11809c0 con 0x55fec1292a20 2024-03-23T18:00:13.821 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.816+0000 7f20ad6cbb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fec1068640 con 0x55fec129d8d0 2024-03-23T18:00:13.821 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.816+0000 7f20ad6cbb80 10 client.0 _ll_register_callbacks cb 0x55fec118a410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T18:00:13.821 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.817+0000 7f2092000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fec1292a20 0x55fec129be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:13.821 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.817+0000 7f2092a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fec1292100 0x55fec129d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:13.821 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.817+0000 7f2092000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fec1292a20 0x55fec129be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:43258/0 (socket says 172.21.15.86:43258) 2024-03-23T18:00:13.821 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.817+0000 7f2092000700 1 -- 172.21.15.86:0/2085029160 learned_addr learned my addr 172.21.15.86:0/2085029160 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:00:13.821 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.817+0000 7f2093400700 1 --2- 172.21.15.86:0/2085029160 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fec129d8d0 0x55fec129c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:13.821 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.817+0000 7f2092000700 1 -- 172.21.15.86:0/2085029160 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fec129d8d0 msgr2=0x55fec129c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:00:13.821 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.817+0000 7f2092000700 1 --2- 172.21.15.86:0/2085029160 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fec129d8d0 0x55fec129c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:13.822 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.817+0000 7f2092000700 1 -- 172.21.15.86:0/2085029160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fec1292100 msgr2=0x55fec129d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:00:13.822 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.817+0000 7f2092000700 1 --2- 172.21.15.86:0/2085029160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fec1292100 0x55fec129d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:13.822 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.817+0000 7f2092000700 1 -- 172.21.15.86:0/2085029160 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f208c011230 con 0x55fec1292a20 2024-03-23T18:00:13.822 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.817+0000 7f2092a00700 1 --2- 172.21.15.86:0/2085029160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fec1292100 0x55fec129d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T18:00:13.822 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.817+0000 7f2092000700 1 --2- 172.21.15.86:0/2085029160 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fec1292a20 0x55fec129be50 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2080006010 tx=0x7f208001a050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=500b3884612b7c81 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:00:13.822 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.817+0000 7f207cc00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T18:00:13.822 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.818+0000 7f207cc00700 1 -- 172.21.15.86:0/2085029160 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f208000f660 con 0x55fec1292a20 2024-03-23T18:00:13.822 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.818+0000 7f207cc00700 1 -- 172.21.15.86:0/2085029160 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f208000f7c0 con 0x55fec1292a20 2024-03-23T18:00:13.822 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.818+0000 7f20ad6cbb80 1 -- 172.21.15.86:0/2085029160 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fec11a5320 con 0x55fec1292a20 2024-03-23T18:00:13.822 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.818+0000 7f207cc00700 1 -- 172.21.15.86:0/2085029160 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f208000f980 con 0x55fec1292a20 2024-03-23T18:00:13.822 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.818+0000 7f20ad6cbb80 10 client.26801 fetch_fsmap learned FSMap version 45 2024-03-23T18:00:13.822 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.818+0000 7f20ad6cbb80 1 -- 172.21.15.86:0/2085029160 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x55fec1012b00 con 0x55fec1292a20 2024-03-23T18:00:13.822 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.818+0000 7f207cc00700 1 -- 172.21.15.86:0/2085029160 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f208003c020 con 0x55fec1292a20 2024-03-23T18:00:13.823 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.818+0000 7f207cc00700 1 -- 172.21.15.86:0/2085029160 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 45) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f208002f960 con 0x55fec1292a20 2024-03-23T18:00:13.823 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.818+0000 7f20ad6cbb80 10 client.26801 fetch_fsmap finished waiting for FSMap version 45 2024-03-23T18:00:13.823 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.818+0000 7f20ad6cbb80 10 client.26801 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T18:00:13.823 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.818+0000 7f20ad6cbb80 20 client.26801 populate_metadata read hostname 'smithi086' 2024-03-23T18:00:13.823 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.818+0000 7f20ad6cbb80 1 --2- 172.21.15.86:0/2085029160 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55fec12ad650 0x55fec12afa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:13.823 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.818+0000 7f20ad6cbb80 4 client.26801 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T18:00:13.823 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.818+0000 7f20ad6cbb80 1 -- 172.21.15.86:0/2085029160 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fec11dd6a0 con 0x55fec12ad650 2024-03-23T18:00:13.823 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.819+0000 7f2092a00700 1 --2- 172.21.15.86:0/2085029160 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55fec12ad650 0x55fec12afa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:13.823 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.819+0000 7f2092a00700 1 --2- 172.21.15.86:0/2085029160 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55fec12ad650 0x55fec12afa70 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8015a28b18706c1b server_cookie=1ddab6580c885f5c in_seq=0 out_seq=0 2024-03-23T18:00:13.823 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.819+0000 7f207cc00700 10 client.26801 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T18:00:13.823 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.820+0000 7f207cc00700 1 -- 172.21.15.86:0/2085029160 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55fec11dd6a0 con 0x55fec12ad650 2024-03-23T18:00:13.823 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.820+0000 7f207cc00700 10 client.26801 handle_command_reply: tid=0 2024-03-23T18:00:13.823 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.820+0000 7f20ad6cbb80 1 client.26801 shutdown 2024-03-23T18:00:13.824 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.820+0000 7f20ad6cbb80 1 -- 172.21.15.86:0/2085029160 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fec1292a20 msgr2=0x55fec129be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:00:13.824 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.820+0000 7f20ad6cbb80 1 --2- 172.21.15.86:0/2085029160 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fec1292a20 0x55fec129be50 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2080006010 tx=0x7f208001a050 comp rx=0 tx=0).stop 2024-03-23T18:00:13.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.820+0000 7f20ad6cbb80 1 -- 172.21.15.86:0/2085029160 shutdown_connections 2024-03-23T18:00:13.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.820+0000 7f20ad6cbb80 1 --2- 172.21.15.86:0/2085029160 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55fec12ad650 0x55fec12afa70 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:13.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.820+0000 7f20ad6cbb80 1 --2- 172.21.15.86:0/2085029160 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fec129d8d0 0x55fec129c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:13.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.820+0000 7f20ad6cbb80 1 --2- 172.21.15.86:0/2085029160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fec1292100 0x55fec129d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:13.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.820+0000 7f20ad6cbb80 1 --2- 172.21.15.86:0/2085029160 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fec1292a20 0x55fec129be50 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:13.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.820+0000 7f20ad6cbb80 1 -- 172.21.15.86:0/2085029160 >> 172.21.15.86:0/2085029160 conn(0x55fec11f2640 msgr2=0x55fec12940a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:00:13.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.820+0000 7f207cc00700 0 client.26801 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T18:00:13.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.820+0000 7f20ad6cbb80 1 -- 172.21.15.86:0/2085029160 shutdown_connections 2024-03-23T18:00:13.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.820+0000 7f20ad6cbb80 1 -- 172.21.15.86:0/2085029160 wait complete. 2024-03-23T18:00:13.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.821+0000 7f20ad6cbb80 20 client.26801 trim_cache size 0 max 16384 2024-03-23T18:00:13.843 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.842+0000 7f20ad6cbb80 1 Processor -- start 2024-03-23T18:00:13.843 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.842+0000 7f20ad6cbb80 1 -- start start 2024-03-23T18:00:13.843 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.843+0000 7f20ad6cbb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fec1292100 0x55fec12a45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:13.843 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.843+0000 7f20ad6cbb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fec1292a20 0x55fec1288500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:13.843 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.843+0000 7f20ad6cbb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fec129d8d0 0x55fec1294040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:13.844 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.843+0000 7f20ad6cbb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fec12933e0 con 0x55fec1292a20 2024-03-23T18:00:13.844 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.843+0000 7f20ad6cbb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fec128f8b0 con 0x55fec1292100 2024-03-23T18:00:13.844 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.843+0000 7f20ad6cbb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fec129cdc0 con 0x55fec129d8d0 2024-03-23T18:00:13.844 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.843+0000 7f2092000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fec1292a20 0x55fec1288500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:13.844 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.843+0000 7f2092a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fec1292100 0x55fec12a45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:13.844 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.843+0000 7f2092000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fec1292a20 0x55fec1288500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58380/0 (socket says 172.21.15.86:58380) 2024-03-23T18:00:13.844 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.843+0000 7f2092000700 1 -- 172.21.15.86:0/3686432377 learned_addr learned my addr 172.21.15.86:0/3686432377 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:00:13.844 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.843+0000 7f2093400700 1 --2- 172.21.15.86:0/3686432377 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fec129d8d0 0x55fec1294040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:13.845 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.844+0000 7f2092a00700 1 -- 172.21.15.86:0/3686432377 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fec129d8d0 msgr2=0x55fec1294040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:00:13.845 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.844+0000 7f2092a00700 1 --2- 172.21.15.86:0/3686432377 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fec129d8d0 0x55fec1294040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:13.846 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.844+0000 7f2092a00700 1 -- 172.21.15.86:0/3686432377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fec1292a20 msgr2=0x55fec1288500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:00:13.846 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.844+0000 7f2092a00700 1 --2- 172.21.15.86:0/3686432377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fec1292a20 0x55fec1288500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:13.846 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.844+0000 7f2092a00700 1 -- 172.21.15.86:0/3686432377 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fec11aae40 con 0x55fec1292100 2024-03-23T18:00:13.846 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.844+0000 7f2092000700 1 --2- 172.21.15.86:0/3686432377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fec1292a20 0x55fec1288500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T18:00:13.846 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.844+0000 7f2092a00700 1 --2- 172.21.15.86:0/3686432377 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fec1292100 0x55fec12a45e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f20840ca190 tx=0x7f20840cc040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b79a85ca2524158f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:00:13.846 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.844+0000 7f2088c00700 1 -- 172.21.15.86:0/3686432377 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2084014070 con 0x55fec1292100 2024-03-23T18:00:13.846 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.844+0000 7f2088c00700 1 -- 172.21.15.86:0/3686432377 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f20840a2000 con 0x55fec1292100 2024-03-23T18:00:13.846 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.844+0000 7f2088c00700 1 -- 172.21.15.86:0/3686432377 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2084099070 con 0x55fec1292100 2024-03-23T18:00:13.846 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.845+0000 7f20ad6cbb80 1 -- 172.21.15.86:0/3686432377 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fec1292100 msgr2=0x55fec12a45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:00:13.846 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.845+0000 7f20ad6cbb80 1 --2- 172.21.15.86:0/3686432377 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fec1292100 0x55fec12a45e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f20840ca190 tx=0x7f20840cc040 comp rx=0 tx=0).stop 2024-03-23T18:00:13.846 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.845+0000 7f20ad6cbb80 1 -- 172.21.15.86:0/3686432377 shutdown_connections 2024-03-23T18:00:13.846 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.845+0000 7f20ad6cbb80 1 --2- 172.21.15.86:0/3686432377 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fec129d8d0 0x55fec1294040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:13.847 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.845+0000 7f20ad6cbb80 1 --2- 172.21.15.86:0/3686432377 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fec1292100 0x55fec12a45e0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:13.847 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.845+0000 7f20ad6cbb80 1 --2- 172.21.15.86:0/3686432377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fec1292a20 0x55fec1288500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:13.847 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.845+0000 7f20ad6cbb80 1 -- 172.21.15.86:0/3686432377 >> 172.21.15.86:0/3686432377 conn(0x55fec11f2640 msgr2=0x55fec1299030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:00:13.847 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.845+0000 7f20ad6cbb80 1 -- 172.21.15.86:0/3686432377 shutdown_connections 2024-03-23T18:00:13.847 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.845+0000 7f20ad6cbb80 1 -- 172.21.15.86:0/3686432377 wait complete. 2024-03-23T18:00:13.847 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.846+0000 7f20ad6cbb80 1 -- start start 2024-03-23T18:00:13.848 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.846+0000 7f20ad6cbb80 1 Processor -- start 2024-03-23T18:00:13.848 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.847+0000 7f20ad6cbb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fec1292100 0x55fec12a1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:13.849 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.847+0000 7f20ad6cbb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fec1292a20 0x55fec12a2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:13.849 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.847+0000 7f2092a00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fec1292100 0x55fec12a1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:13.849 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.847+0000 7f2092a00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fec1292100 0x55fec12a1f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:40486/0 (socket says 172.21.15.86:40486) 2024-03-23T18:00:13.849 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.847+0000 7f2092a00700 1 -- 172.21.15.86:0/4093153938 learned_addr learned my addr 172.21.15.86:0/4093153938 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:00:13.849 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.847+0000 7f20ad6cbb80 1 --2- 172.21.15.86:0/4093153938 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fec129d8d0 0x55fec12fe170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:13.849 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.847+0000 7f20ad6cbb80 1 -- 172.21.15.86:0/4093153938 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fec12a2c00 con 0x55fec1292a20 2024-03-23T18:00:13.849 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.847+0000 7f20ad6cbb80 1 -- 172.21.15.86:0/4093153938 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fec12a2d70 con 0x55fec129d8d0 2024-03-23T18:00:13.849 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.848+0000 7f20ad6cbb80 1 -- 172.21.15.86:0/4093153938 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fec12fe6b0 con 0x55fec1292100 2024-03-23T18:00:13.849 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.848+0000 7f20ad6cbb80 10 client.0 _ll_register_callbacks cb 0x55fec118a410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T18:00:13.849 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.848+0000 7f2092000700 1 --2- 172.21.15.86:0/4093153938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fec1292a20 0x55fec12a2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:13.849 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.848+0000 7f2093400700 1 --2- 172.21.15.86:0/4093153938 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fec129d8d0 0x55fec12fe170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:13.849 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.848+0000 7f2092a00700 1 -- 172.21.15.86:0/4093153938 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fec129d8d0 msgr2=0x55fec12fe170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:00:13.849 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.848+0000 7f2092a00700 1 --2- 172.21.15.86:0/4093153938 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fec129d8d0 0x55fec12fe170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:13.849 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.848+0000 7f2092a00700 1 -- 172.21.15.86:0/4093153938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fec1292a20 msgr2=0x55fec12a2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:00:13.850 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.848+0000 7f2092a00700 1 --2- 172.21.15.86:0/4093153938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fec1292a20 0x55fec12a2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:13.850 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.848+0000 7f2092a00700 1 -- 172.21.15.86:0/4093153938 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fec1012b00 con 0x55fec1292100 2024-03-23T18:00:13.850 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.848+0000 7f2092000700 1 --2- 172.21.15.86:0/4093153938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fec1292a20 0x55fec12a2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T18:00:13.850 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.848+0000 7f2093400700 1 --2- 172.21.15.86:0/4093153938 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fec129d8d0 0x55fec12fe170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T18:00:13.850 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.848+0000 7f2092a00700 1 --2- 172.21.15.86:0/4093153938 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fec1292100 0x55fec12a1f30 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2084096080 tx=0x7f2084097fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a761e00df7f4a6ee server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:00:13.850 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.848+0000 7f207cc00700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-03-23T18:00:13.850 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.849+0000 7f207cc00700 1 -- 172.21.15.86:0/4093153938 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2084014070 con 0x55fec1292100 2024-03-23T18:00:13.851 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.849+0000 7f207cc00700 1 -- 172.21.15.86:0/4093153938 <== mon.1 v2:172.21.15.152:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f2084099270 con 0x55fec1292100 2024-03-23T18:00:13.851 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.849+0000 7f20ad6cbb80 1 -- 172.21.15.86:0/4093153938 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f208c00f040 con 0x55fec1292100 2024-03-23T18:00:13.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.849+0000 7f207cc00700 1 -- 172.21.15.86:0/4093153938 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20840652e0 con 0x55fec1292100 2024-03-23T18:00:13.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.849+0000 7f20ad6cbb80 10 client.26788 fetch_fsmap learned FSMap version 45 2024-03-23T18:00:13.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.849+0000 7f20ad6cbb80 1 -- 172.21.15.86:0/4093153938 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x7f20941a64b0 con 0x55fec1292100 2024-03-23T18:00:13.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.849+0000 7f207cc00700 1 -- 172.21.15.86:0/4093153938 <== mon.1 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f208c00f040 con 0x55fec1292100 2024-03-23T18:00:13.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.849+0000 7f207cc00700 1 -- 172.21.15.86:0/4093153938 <== mon.1 v2:172.21.15.152:3300/0 5 ==== fsmap(e 45) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f20840a1c70 con 0x55fec1292100 2024-03-23T18:00:13.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.849+0000 7f20ad6cbb80 10 client.26788 fetch_fsmap finished waiting for FSMap version 45 2024-03-23T18:00:13.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.849+0000 7f20ad6cbb80 10 client.26788 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T18:00:13.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.849+0000 7f20ad6cbb80 20 client.26788 populate_metadata read hostname 'smithi086' 2024-03-23T18:00:13.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.849+0000 7f20ad6cbb80 1 --2- 172.21.15.86:0/4093153938 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55fec12ffb00 0x55fec1301f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:13.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.849+0000 7f20ad6cbb80 4 client.26788 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T18:00:13.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.849+0000 7f20ad6cbb80 1 -- 172.21.15.86:0/4093153938 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fec1033c90 con 0x55fec12ffb00 2024-03-23T18:00:13.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.850+0000 7f2092000700 1 --2- 172.21.15.86:0/4093153938 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55fec12ffb00 0x55fec1301f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:13.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.850+0000 7f2092000700 1 --2- 172.21.15.86:0/4093153938 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55fec12ffb00 0x55fec1301f20 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1946fc2ca1bdab6 server_cookie=d1163d2e51bfe1e7 in_seq=0 out_seq=0 2024-03-23T18:00:13.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.850+0000 7f207cc00700 10 client.26788 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T18:00:13.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.911+0000 7f207cc00700 1 -- 172.21.15.86:0/4093153938 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fec1033c90 con 0x55fec12ffb00 2024-03-23T18:00:13.912 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.911+0000 7f207cc00700 10 client.26788 handle_command_reply: tid=0 2024-03-23T18:00:13.912 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.911+0000 7f20ad6cbb80 1 client.26788 shutdown 2024-03-23T18:00:13.912 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.912+0000 7f20ad6cbb80 1 -- 172.21.15.86:0/4093153938 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fec1292100 msgr2=0x55fec12a1f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:00:13.912 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.912+0000 7f20ad6cbb80 1 --2- 172.21.15.86:0/4093153938 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fec1292100 0x55fec12a1f30 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2084096080 tx=0x7f2084097fd0 comp rx=0 tx=0).stop 2024-03-23T18:00:13.913 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.913+0000 7f20ad6cbb80 1 -- 172.21.15.86:0/4093153938 shutdown_connections 2024-03-23T18:00:13.913 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.913+0000 7f20ad6cbb80 1 --2- 172.21.15.86:0/4093153938 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55fec12ffb00 0x55fec1301f20 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:13.913 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.913+0000 7f20ad6cbb80 1 --2- 172.21.15.86:0/4093153938 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fec129d8d0 0x55fec12fe170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:13.913 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.913+0000 7f20ad6cbb80 1 --2- 172.21.15.86:0/4093153938 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fec1292100 0x55fec12a1f30 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:13.913 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.913+0000 7f20ad6cbb80 1 --2- 172.21.15.86:0/4093153938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fec1292a20 0x55fec12a2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:13.913 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.913+0000 7f20ad6cbb80 1 -- 172.21.15.86:0/4093153938 >> 172.21.15.86:0/4093153938 conn(0x55fec11f2640 msgr2=0x55fec1294040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:00:13.914 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.913+0000 7f207cc00700 0 client.26788 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T18:00:13.914 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.914+0000 7f20ad6cbb80 1 -- 172.21.15.86:0/4093153938 shutdown_connections 2024-03-23T18:00:13.914 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.914+0000 7f20ad6cbb80 1 -- 172.21.15.86:0/4093153938 wait complete. 2024-03-23T18:00:13.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-03-23T18:00:13.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-03-23T18:00:13.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-03-23T18:00:13.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-03-23T18:00:13.917 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.915+0000 7f20ad6cbb80 20 client.26788 trim_cache size 0 max 16384 2024-03-23T18:00:13.917 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.916+0000 7f2099c00700 1 -- 172.21.15.86:0/944676341 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f20780a1290 msgr2=0x7f20780a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:00:13.917 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.916+0000 7f2099c00700 1 --2- 172.21.15.86:0/944676341 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f20780a1290 0x7f20780a3750 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f209419a990 tx=0x7f208000b260 comp rx=0 tx=0).stop 2024-03-23T18:00:13.917 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.916+0000 7f2099c00700 1 -- 172.21.15.86:0/944676341 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2094100f10 msgr2=0x7f2094199440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:00:13.917 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.916+0000 7f2099c00700 1 --2- 172.21.15.86:0/944676341 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2094100f10 0x7f2094199440 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f208400d950 tx=0x7f208400dc60 comp rx=0 tx=0).stop 2024-03-23T18:00:13.917 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.917+0000 7f2099c00700 1 -- 172.21.15.86:0/944676341 shutdown_connections 2024-03-23T18:00:13.917 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.917+0000 7f2099c00700 1 --2- 172.21.15.86:0/944676341 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f209419dd20 0x7f209419e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:13.917 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.917+0000 7f2099c00700 1 --2- 172.21.15.86:0/944676341 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2094100f10 0x7f2094199440 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:13.918 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.917+0000 7f2099c00700 1 --2- 172.21.15.86:0/944676341 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f20780a1290 0x7f20780a3750 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:13.918 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.917+0000 7f2099c00700 1 --2- 172.21.15.86:0/944676341 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2094106030 0x7f2094199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:13.918 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.917+0000 7f2099c00700 1 -- 172.21.15.86:0/944676341 >> 172.21.15.86:0/944676341 conn(0x7f20940fc770 msgr2=0x7f20940fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:00:13.918 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.917+0000 7f2099c00700 1 -- 172.21.15.86:0/944676341 shutdown_connections 2024-03-23T18:00:13.918 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:13.917+0000 7f2099c00700 1 -- 172.21.15.86:0/944676341 wait complete. 2024-03-23T18:00:13.918 INFO:tasks.workunit.client.0.smithi086.stdout: 1 533610 6.15 MB/sec execute 398 sec latency 37.503 ms 2024-03-23T18:00:14.085 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:00:13 smithi152 ceph-mon[47949]: pgmap v298: 97 pgs: 97 active+clean; 218 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 983 KiB/s rd, 2.1 MiB/s wr, 155 op/s 2024-03-23T18:00:14.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:00:13 smithi086 ceph-mon[38221]: pgmap v298: 97 pgs: 97 active+clean; 218 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 983 KiB/s rd, 2.1 MiB/s wr, 155 op/s 2024-03-23T18:00:14.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:00:13 smithi092 ceph-mon[79089]: pgmap v298: 97 pgs: 97 active+clean; 218 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 983 KiB/s rd, 2.1 MiB/s wr, 155 op/s 2024-03-23T18:00:14.918 INFO:tasks.workunit.client.0.smithi086.stdout: 1 535329 6.16 MB/sec execute 399 sec latency 37.403 ms 2024-03-23T18:00:15.918 INFO:tasks.workunit.client.0.smithi086.stdout: 1 536873 6.17 MB/sec execute 400 sec latency 12.625 ms 2024-03-23T18:00:16.085 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:00:15 smithi152 ceph-mon[47949]: pgmap v299: 97 pgs: 97 active+clean; 209 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 293 op/s 2024-03-23T18:00:16.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:00:15 smithi086 ceph-mon[38221]: pgmap v299: 97 pgs: 97 active+clean; 209 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 293 op/s 2024-03-23T18:00:16.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:00:15 smithi092 ceph-mon[79089]: pgmap v299: 97 pgs: 97 active+clean; 209 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 293 op/s 2024-03-23T18:00:16.918 INFO:tasks.workunit.client.0.smithi086.stdout: 1 538631 6.17 MB/sec execute 401 sec latency 16.183 ms 2024-03-23T18:00:17.918 INFO:tasks.workunit.client.0.smithi086.stdout: 1 540338 6.19 MB/sec execute 402 sec latency 12.721 ms 2024-03-23T18:00:18.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:00:17 smithi086 ceph-mon[38221]: pgmap v300: 97 pgs: 97 active+clean; 209 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 280 op/s 2024-03-23T18:00:18.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:00:17 smithi092 ceph-mon[79089]: pgmap v300: 97 pgs: 97 active+clean; 209 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 280 op/s 2024-03-23T18:00:18.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:00:17 smithi152 ceph-mon[47949]: pgmap v300: 97 pgs: 97 active+clean; 209 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 280 op/s 2024-03-23T18:00:18.918 INFO:tasks.workunit.client.0.smithi086.stdout: 1 541683 6.19 MB/sec execute 403 sec latency 17.197 ms 2024-03-23T18:00:19.918 INFO:tasks.workunit.client.0.smithi086.stdout: 1 543673 6.20 MB/sec execute 404 sec latency 32.589 ms 2024-03-23T18:00:20.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:00:19 smithi086 ceph-mon[38221]: pgmap v301: 97 pgs: 97 active+clean; 209 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 280 op/s 2024-03-23T18:00:20.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:00:19 smithi092 ceph-mon[79089]: pgmap v301: 97 pgs: 97 active+clean; 209 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 280 op/s 2024-03-23T18:00:20.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:00:19 smithi152 ceph-mon[47949]: pgmap v301: 97 pgs: 97 active+clean; 209 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 280 op/s 2024-03-23T18:00:20.918 INFO:tasks.workunit.client.0.smithi086.stdout: 1 544905 6.20 MB/sec execute 405 sec latency 17.003 ms 2024-03-23T18:00:21.918 INFO:tasks.workunit.client.0.smithi086.stdout: 1 546896 6.21 MB/sec execute 406 sec latency 18.445 ms 2024-03-23T18:00:22.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:00:21 smithi086 ceph-mon[38221]: pgmap v302: 97 pgs: 97 active+clean; 220 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.3 MiB/s wr, 285 op/s 2024-03-23T18:00:22.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:00:21 smithi092 ceph-mon[79089]: pgmap v302: 97 pgs: 97 active+clean; 220 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.3 MiB/s wr, 285 op/s 2024-03-23T18:00:22.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:00:21 smithi152 ceph-mon[47949]: pgmap v302: 97 pgs: 97 active+clean; 220 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.3 MiB/s wr, 285 op/s 2024-03-23T18:00:22.919 INFO:tasks.workunit.client.0.smithi086.stdout: 1 548129 6.21 MB/sec execute 407 sec latency 14.107 ms 2024-03-23T18:00:23.919 INFO:tasks.workunit.client.0.smithi086.stdout: 1 550042 6.22 MB/sec execute 408 sec latency 15.624 ms 2024-03-23T18:00:24.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:00:23 smithi086 ceph-mon[38221]: pgmap v303: 97 pgs: 97 active+clean; 220 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 244 op/s 2024-03-23T18:00:24.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:00:23 smithi092 ceph-mon[79089]: pgmap v303: 97 pgs: 97 active+clean; 220 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 244 op/s 2024-03-23T18:00:24.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:00:23 smithi152 ceph-mon[47949]: pgmap v303: 97 pgs: 97 active+clean; 220 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 244 op/s 2024-03-23T18:00:24.919 INFO:tasks.workunit.client.0.smithi086.stdout: 1 551337 6.22 MB/sec execute 409 sec latency 40.703 ms 2024-03-23T18:00:25.919 INFO:tasks.workunit.client.0.smithi086.stdout: 1 553105 6.23 MB/sec execute 410 sec latency 21.886 ms 2024-03-23T18:00:26.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:00:25 smithi086 ceph-mon[38221]: pgmap v304: 97 pgs: 97 active+clean; 228 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.0 MiB/s wr, 342 op/s 2024-03-23T18:00:26.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:00:25 smithi092 ceph-mon[79089]: pgmap v304: 97 pgs: 97 active+clean; 228 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.0 MiB/s wr, 342 op/s 2024-03-23T18:00:26.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:00:25 smithi152 ceph-mon[47949]: pgmap v304: 97 pgs: 97 active+clean; 228 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.0 MiB/s wr, 342 op/s 2024-03-23T18:00:26.919 INFO:tasks.workunit.client.0.smithi086.stdout: 1 554692 6.24 MB/sec execute 411 sec latency 10.893 ms 2024-03-23T18:00:27.091 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.091+0000 7f386ca00700 1 -- 172.21.15.86:0/3158360025 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3868106b70 msgr2=0x7f3868106f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:00:27.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.091+0000 7f386ca00700 1 --2- 172.21.15.86:0/3158360025 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3868106b70 0x7f3868106f50 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3858009990 tx=0x7f3858009ca0 comp rx=0 tx=0).stop 2024-03-23T18:00:27.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.091+0000 7f386ca00700 1 -- 172.21.15.86:0/3158360025 shutdown_connections 2024-03-23T18:00:27.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.091+0000 7f386ca00700 1 --2- 172.21.15.86:0/3158360025 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3868101490 0x7f386810d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:27.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.091+0000 7f386ca00700 1 --2- 172.21.15.86:0/3158360025 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3868106b70 0x7f3868106f50 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:27.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.091+0000 7f386ca00700 1 --2- 172.21.15.86:0/3158360025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3868100ad0 0x7f3868100f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:27.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.091+0000 7f386ca00700 1 -- 172.21.15.86:0/3158360025 >> 172.21.15.86:0/3158360025 conn(0x7f38680fc5e0 msgr2=0x7f38680fea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:00:27.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.091+0000 7f386ca00700 1 -- 172.21.15.86:0/3158360025 shutdown_connections 2024-03-23T18:00:27.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.092+0000 7f386ca00700 1 -- 172.21.15.86:0/3158360025 wait complete. 2024-03-23T18:00:27.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.093+0000 7f386ca00700 1 Processor -- start 2024-03-23T18:00:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.093+0000 7f386ca00700 1 -- start start 2024-03-23T18:00:27.094 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.093+0000 7f386ca00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3868100ad0 0x7f38680730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:27.094 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.093+0000 7f386ca00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3868101490 0x7f38680735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:27.094 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.094+0000 7f386ca00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3868106b70 0x7f3868073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:27.094 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.094+0000 7f386ca00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38681a52d0 con 0x7f3868100ad0 2024-03-23T18:00:27.094 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.094+0000 7f386ca00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38681a5440 con 0x7f3868106b70 2024-03-23T18:00:27.094 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.094+0000 7f386ca00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38681a5600 con 0x7f3868101490 2024-03-23T18:00:27.094 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.094+0000 7f3867400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3868100ad0 0x7f38680730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:27.094 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.094+0000 7f3866a00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3868101490 0x7f38680735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.094+0000 7f3866a00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3868101490 0x7f38680735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:55916/0 (socket says 172.21.15.86:55916) 2024-03-23T18:00:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.094+0000 7f3866a00700 1 -- 172.21.15.86:0/3313371974 learned_addr learned my addr 172.21.15.86:0/3313371974 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:00:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.094+0000 7f3867e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3868106b70 0x7f3868073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.094+0000 7f3866a00700 1 -- 172.21.15.86:0/3313371974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3868106b70 msgr2=0x7f3868073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:00:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.094+0000 7f3866a00700 1 --2- 172.21.15.86:0/3313371974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3868106b70 0x7f3868073b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.094+0000 7f3866a00700 1 -- 172.21.15.86:0/3313371974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3868100ad0 msgr2=0x7f38680730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:00:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.094+0000 7f3866a00700 1 --2- 172.21.15.86:0/3313371974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3868100ad0 0x7f38680730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.094+0000 7f3866a00700 1 -- 172.21.15.86:0/3313371974 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38580095f0 con 0x7f3868101490 2024-03-23T18:00:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.094+0000 7f3867400700 1 --2- 172.21.15.86:0/3313371974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3868100ad0 0x7f38680730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T18:00:27.096 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.095+0000 7f3866a00700 1 --2- 172.21.15.86:0/3313371974 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3868101490 0x7f38680735f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f385400e9a0 tx=0x7f385400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:00:27.096 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.095+0000 7f385fe00700 1 -- 172.21.15.86:0/3313371974 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3854009790 con 0x7f3868101490 2024-03-23T18:00:27.096 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.095+0000 7f385fe00700 1 -- 172.21.15.86:0/3313371974 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3854010430 con 0x7f3868101490 2024-03-23T18:00:27.096 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.095+0000 7f386ca00700 1 -- 172.21.15.86:0/3313371974 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38681a58e0 con 0x7f3868101490 2024-03-23T18:00:27.096 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.095+0000 7f385fe00700 1 -- 172.21.15.86:0/3313371974 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3854018680 con 0x7f3868101490 2024-03-23T18:00:27.096 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.095+0000 7f386ca00700 1 -- 172.21.15.86:0/3313371974 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38681a5e40 con 0x7f3868101490 2024-03-23T18:00:27.098 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.098+0000 7f385fe00700 1 -- 172.21.15.86:0/3313371974 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f38540105a0 con 0x7f3868101490 2024-03-23T18:00:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.099+0000 7f385fe00700 1 --2- 172.21.15.86:0/3313371974 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f384c0a1310 0x7f384c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.100+0000 7f385fe00700 1 -- 172.21.15.86:0/3313371974 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f3854014070 con 0x7f3868101490 2024-03-23T18:00:27.100 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.100+0000 7f3867400700 1 --2- 172.21.15.86:0/3313371974 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f384c0a1310 0x7f384c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:27.100 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.100+0000 7f3881b44b80 1 Processor -- start 2024-03-23T18:00:27.100 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.100+0000 7f3881b44b80 1 -- start start 2024-03-23T18:00:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.100+0000 7f3881b44b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56177d4baae0 0x56177d4baec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.100+0000 7f3867400700 1 --2- 172.21.15.86:0/3313371974 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f384c0a1310 0x7f384c0a37d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f385800b2e0 tx=0x7f3858005960 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:00:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.100+0000 7f3881b44b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56177d4bb400 0x56177d4c5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.100+0000 7f3881b44b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56177d4c62b0 0x56177d4c86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.101+0000 7f3881b44b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56177d40b930 con 0x56177d4c62b0 2024-03-23T18:00:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.101+0000 7f3881b44b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x56177d4092f0 con 0x56177d4baae0 2024-03-23T18:00:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.101+0000 7f3881b44b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x56177d404be0 con 0x56177d4bb400 2024-03-23T18:00:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.101+0000 7f3867400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56177d4baae0 0x56177d4baec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.101+0000 7f3867400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56177d4baae0 0x56177d4baec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:35166/0 (socket says 172.21.15.86:35166) 2024-03-23T18:00:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.101+0000 7f3867400700 1 -- 172.21.15.86:0/2364234087 learned_addr learned my addr 172.21.15.86:0/2364234087 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:00:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.101+0000 7f3866a00700 1 --2- 172.21.15.86:0/2364234087 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56177d4bb400 0x56177d4c5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.101+0000 7f3867e00700 1 --2- 172.21.15.86:0/2364234087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56177d4c62b0 0x56177d4c86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.101+0000 7f3867400700 1 -- 172.21.15.86:0/2364234087 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56177d4bb400 msgr2=0x56177d4c5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:00:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.101+0000 7f3867400700 1 --2- 172.21.15.86:0/2364234087 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56177d4bb400 0x56177d4c5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.101+0000 7f3867400700 1 -- 172.21.15.86:0/2364234087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56177d4c62b0 msgr2=0x56177d4c86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:00:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.101+0000 7f3867400700 1 --2- 172.21.15.86:0/2364234087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56177d4c62b0 0x56177d4c86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.101+0000 7f3867400700 1 -- 172.21.15.86:0/2364234087 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56177d286350 con 0x56177d4baae0 2024-03-23T18:00:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.101+0000 7f3867400700 1 --2- 172.21.15.86:0/2364234087 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56177d4baae0 0x56177d4baec0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f3858009990 tx=0x7f3858020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1bb749d8cdfb5a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:00:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.102+0000 7f3853e00700 1 -- 172.21.15.86:0/2364234087 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385802b070 con 0x56177d4baae0 2024-03-23T18:00:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.102+0000 7f3853e00700 1 -- 172.21.15.86:0/2364234087 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f385801e040 con 0x56177d4baae0 2024-03-23T18:00:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.102+0000 7f3853e00700 1 -- 172.21.15.86:0/2364234087 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3858004890 con 0x56177d4baae0 2024-03-23T18:00:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.102+0000 7f3881b44b80 1 -- 172.21.15.86:0/2364234087 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56177d4baae0 msgr2=0x56177d4baec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:00:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.102+0000 7f3881b44b80 1 --2- 172.21.15.86:0/2364234087 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56177d4baae0 0x56177d4baec0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f3858009990 tx=0x7f3858020000 comp rx=0 tx=0).stop 2024-03-23T18:00:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.103+0000 7f3881b44b80 1 -- 172.21.15.86:0/2364234087 shutdown_connections 2024-03-23T18:00:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.103+0000 7f3881b44b80 1 --2- 172.21.15.86:0/2364234087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56177d4c62b0 0x56177d4c86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.103+0000 7f3881b44b80 1 --2- 172.21.15.86:0/2364234087 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56177d4baae0 0x56177d4baec0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.103+0000 7f3881b44b80 1 --2- 172.21.15.86:0/2364234087 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56177d4bb400 0x56177d4c5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.103+0000 7f3881b44b80 1 -- 172.21.15.86:0/2364234087 >> 172.21.15.86:0/2364234087 conn(0x56177d412640 msgr2=0x56177d4ae710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:00:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.103+0000 7f3881b44b80 1 -- 172.21.15.86:0/2364234087 shutdown_connections 2024-03-23T18:00:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.103+0000 7f3881b44b80 1 -- 172.21.15.86:0/2364234087 wait complete. 2024-03-23T18:00:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.103+0000 7f3881b44b80 1 -- start start 2024-03-23T18:00:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.103+0000 7f3881b44b80 1 Processor -- start 2024-03-23T18:00:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.104+0000 7f3881b44b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56177d4baae0 0x56177d4c5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.104+0000 7f3881b44b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56177d4bb400 0x56177d4c2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.104+0000 7f3881b44b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56177d4c62b0 0x56177d4c3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.104+0000 7f3881b44b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56177d3a0d20 con 0x56177d4bb400 2024-03-23T18:00:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.104+0000 7f3881b44b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x56177d3a09c0 con 0x56177d4c62b0 2024-03-23T18:00:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.104+0000 7f3881b44b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x56177d288640 con 0x56177d4baae0 2024-03-23T18:00:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.104+0000 7f3881b44b80 10 client.0 _ll_register_callbacks cb 0x56177d3aa410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T18:00:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.104+0000 7f3866a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56177d4bb400 0x56177d4c2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.104+0000 7f3866a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56177d4bb400 0x56177d4c2fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41670/0 (socket says 172.21.15.86:41670) 2024-03-23T18:00:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.104+0000 7f3866a00700 1 -- 172.21.15.86:0/1686417302 learned_addr learned my addr 172.21.15.86:0/1686417302 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:00:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.104+0000 7f3867400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56177d4baae0 0x56177d4c5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.104+0000 7f3867e00700 1 --2- 172.21.15.86:0/1686417302 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56177d4c62b0 0x56177d4c3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.105+0000 7f3867400700 1 -- 172.21.15.86:0/1686417302 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56177d4c62b0 msgr2=0x56177d4c3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:00:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.105+0000 7f3867400700 1 --2- 172.21.15.86:0/1686417302 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56177d4c62b0 0x56177d4c3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.105+0000 7f3867400700 1 -- 172.21.15.86:0/1686417302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56177d4bb400 msgr2=0x56177d4c2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:00:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.105+0000 7f3867400700 1 --2- 172.21.15.86:0/1686417302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56177d4bb400 0x56177d4c2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.105+0000 7f3867400700 1 -- 172.21.15.86:0/1686417302 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f386810e2a0 con 0x56177d4baae0 2024-03-23T18:00:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.105+0000 7f3867400700 1 --2- 172.21.15.86:0/1686417302 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56177d4baae0 0x56177d4c5c60 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3858006010 tx=0x7f385800fdc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb80bcb4f3caef06 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:00:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.105+0000 7f3850c00700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-03-23T18:00:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.105+0000 7f3850c00700 1 -- 172.21.15.86:0/1686417302 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385802b040 con 0x56177d4baae0 2024-03-23T18:00:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.105+0000 7f3850c00700 1 -- 172.21.15.86:0/1686417302 <== mon.1 v2:172.21.15.152:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f38580049d0 con 0x56177d4baae0 2024-03-23T18:00:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.105+0000 7f3881b44b80 1 -- 172.21.15.86:0/1686417302 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56177d3c5320 con 0x56177d4baae0 2024-03-23T18:00:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.106+0000 7f3850c00700 1 -- 172.21.15.86:0/1686417302 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3858004bd0 con 0x56177d4baae0 2024-03-23T18:00:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.106+0000 7f3881b44b80 10 client.26800 fetch_fsmap learned FSMap version 45 2024-03-23T18:00:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.106+0000 7f3881b44b80 1 -- 172.21.15.86:0/1686417302 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x56177d232b00 con 0x56177d4baae0 2024-03-23T18:00:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.106+0000 7f3850c00700 1 -- 172.21.15.86:0/1686417302 <== mon.1 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3858044080 con 0x56177d4baae0 2024-03-23T18:00:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.106+0000 7f3850c00700 1 -- 172.21.15.86:0/1686417302 <== mon.1 v2:172.21.15.152:3300/0 5 ==== fsmap(e 45) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f3858030cf0 con 0x56177d4baae0 2024-03-23T18:00:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.106+0000 7f3881b44b80 10 client.26800 fetch_fsmap finished waiting for FSMap version 45 2024-03-23T18:00:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.106+0000 7f3881b44b80 10 client.26800 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T18:00:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.106+0000 7f3881b44b80 20 client.26800 populate_metadata read hostname 'smithi086' 2024-03-23T18:00:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.106+0000 7f3881b44b80 1 --2- 172.21.15.86:0/1686417302 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x56177d4d5b40 0x56177d4d7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.106+0000 7f3881b44b80 4 client.26800 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T18:00:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.106+0000 7f3881b44b80 1 -- 172.21.15.86:0/1686417302 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56177d3fd6a0 con 0x56177d4d5b40 2024-03-23T18:00:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.106+0000 7f3866a00700 1 --2- 172.21.15.86:0/1686417302 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x56177d4d5b40 0x56177d4d7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.107+0000 7f3850c00700 10 client.26800 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T18:00:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.107+0000 7f3866a00700 1 --2- 172.21.15.86:0/1686417302 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x56177d4d5b40 0x56177d4d7f60 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21c361df3d900d64 server_cookie=9ac5b0ae6690c83e in_seq=0 out_seq=0 2024-03-23T18:00:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.107+0000 7f3850c00700 1 -- 172.21.15.86:0/1686417302 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x56177d3fd6a0 con 0x56177d4d5b40 2024-03-23T18:00:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.107+0000 7f3850c00700 10 client.26800 handle_command_reply: tid=0 2024-03-23T18:00:27.111 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.107+0000 7f3881b44b80 1 client.26800 shutdown 2024-03-23T18:00:27.111 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.108+0000 7f3881b44b80 1 -- 172.21.15.86:0/1686417302 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56177d4baae0 msgr2=0x56177d4c5c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:00:27.111 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.108+0000 7f3881b44b80 1 --2- 172.21.15.86:0/1686417302 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56177d4baae0 0x56177d4c5c60 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3858006010 tx=0x7f385800fdc0 comp rx=0 tx=0).stop 2024-03-23T18:00:27.111 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.108+0000 7f3881b44b80 1 -- 172.21.15.86:0/1686417302 shutdown_connections 2024-03-23T18:00:27.111 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.108+0000 7f3881b44b80 1 --2- 172.21.15.86:0/1686417302 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x56177d4d5b40 0x56177d4d7f60 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:27.111 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.108+0000 7f3881b44b80 1 --2- 172.21.15.86:0/1686417302 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56177d4c62b0 0x56177d4c3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:27.111 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.108+0000 7f3881b44b80 1 --2- 172.21.15.86:0/1686417302 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56177d4baae0 0x56177d4c5c60 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:27.111 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.108+0000 7f3850c00700 0 client.26800 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T18:00:27.111 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.108+0000 7f3881b44b80 1 --2- 172.21.15.86:0/1686417302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56177d4bb400 0x56177d4c2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:27.111 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.108+0000 7f3881b44b80 1 -- 172.21.15.86:0/1686417302 >> 172.21.15.86:0/1686417302 conn(0x56177d412640 msgr2=0x56177d4bc9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:00:27.111 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.108+0000 7f3881b44b80 1 -- 172.21.15.86:0/1686417302 shutdown_connections 2024-03-23T18:00:27.111 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.108+0000 7f3881b44b80 1 -- 172.21.15.86:0/1686417302 wait complete. 2024-03-23T18:00:27.112 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.108+0000 7f3881b44b80 20 client.26800 trim_cache size 0 max 16384 2024-03-23T18:00:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.125+0000 7f3881b44b80 1 Processor -- start 2024-03-23T18:00:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.125+0000 7f3881b44b80 1 -- start start 2024-03-23T18:00:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.126+0000 7f3881b44b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56177d4baae0 0x56177d4ccf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.126+0000 7f3881b44b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56177d4bb400 0x56177d4a8700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.126+0000 7f3881b44b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56177d4c62b0 0x56177d4c2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.126+0000 7f3881b44b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56177d4af320 con 0x56177d4c62b0 2024-03-23T18:00:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.126+0000 7f3881b44b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x56177d4c33b0 con 0x56177d4bb400 2024-03-23T18:00:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.126+0000 7f3881b44b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x56177d4c3bb0 con 0x56177d4baae0 2024-03-23T18:00:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.126+0000 7f3867400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56177d4baae0 0x56177d4ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.126+0000 7f3866a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56177d4bb400 0x56177d4a8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.126+0000 7f3867400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56177d4baae0 0x56177d4ccf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:55940/0 (socket says 172.21.15.86:55940) 2024-03-23T18:00:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.126+0000 7f3867400700 1 -- 172.21.15.86:0/691013014 learned_addr learned my addr 172.21.15.86:0/691013014 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:00:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.126+0000 7f3867e00700 1 --2- 172.21.15.86:0/691013014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56177d4c62b0 0x56177d4c2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.126+0000 7f3866a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56177d4bb400 0x56177d4a8700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:35180/0 (socket says 172.21.15.86:35180) 2024-03-23T18:00:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.126+0000 7f3866a00700 1 -- 172.21.15.86:0/691013014 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56177d4baae0 msgr2=0x56177d4ccf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:00:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.126+0000 7f3866a00700 1 --2- 172.21.15.86:0/691013014 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56177d4baae0 0x56177d4ccf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.126+0000 7f3866a00700 1 -- 172.21.15.86:0/691013014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56177d4c62b0 msgr2=0x56177d4c2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:00:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.126+0000 7f3866a00700 1 --2- 172.21.15.86:0/691013014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56177d4c62b0 0x56177d4c2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.126+0000 7f3866a00700 1 -- 172.21.15.86:0/691013014 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56177d3cae40 con 0x56177d4bb400 2024-03-23T18:00:27.129 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.127+0000 7f3866a00700 1 --2- 172.21.15.86:0/691013014 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56177d4bb400 0x56177d4a8700 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f38540969a0 tx=0x7f38540cc040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94a1c3423dafdcba server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:00:27.129 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.127+0000 7f385d200700 1 -- 172.21.15.86:0/691013014 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38540a23b0 con 0x56177d4bb400 2024-03-23T18:00:27.129 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.127+0000 7f385d200700 1 -- 172.21.15.86:0/691013014 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3854098070 con 0x56177d4bb400 2024-03-23T18:00:27.129 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.127+0000 7f385d200700 1 -- 172.21.15.86:0/691013014 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3854099420 con 0x56177d4bb400 2024-03-23T18:00:27.129 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.128+0000 7f3881b44b80 1 -- 172.21.15.86:0/691013014 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56177d4bb400 msgr2=0x56177d4a8700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:00:27.129 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.128+0000 7f3881b44b80 1 --2- 172.21.15.86:0/691013014 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56177d4bb400 0x56177d4a8700 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f38540969a0 tx=0x7f38540cc040 comp rx=0 tx=0).stop 2024-03-23T18:00:27.129 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.128+0000 7f3881b44b80 1 -- 172.21.15.86:0/691013014 shutdown_connections 2024-03-23T18:00:27.129 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.128+0000 7f3881b44b80 1 --2- 172.21.15.86:0/691013014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56177d4c62b0 0x56177d4c2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:27.129 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.128+0000 7f3881b44b80 1 --2- 172.21.15.86:0/691013014 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56177d4baae0 0x56177d4ccf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:27.129 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.128+0000 7f3881b44b80 1 --2- 172.21.15.86:0/691013014 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56177d4bb400 0x56177d4a8700 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:27.129 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.128+0000 7f3881b44b80 1 -- 172.21.15.86:0/691013014 >> 172.21.15.86:0/691013014 conn(0x56177d412640 msgr2=0x56177d4c5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:00:27.129 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.128+0000 7f3881b44b80 1 -- 172.21.15.86:0/691013014 shutdown_connections 2024-03-23T18:00:27.129 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.128+0000 7f3881b44b80 1 -- 172.21.15.86:0/691013014 wait complete. 2024-03-23T18:00:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.128+0000 7f3881b44b80 1 -- start start 2024-03-23T18:00:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.128+0000 7f3881b44b80 1 Processor -- start 2024-03-23T18:00:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.129+0000 7f3881b44b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56177d4baae0 0x56177d4ccb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.129+0000 7f3881b44b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56177d4bb400 0x56177d4cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.129+0000 7f3881b44b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56177d4c62b0 0x56177d4cb520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.129+0000 7f3881b44b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56177d4cba60 con 0x56177d4c62b0 2024-03-23T18:00:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.129+0000 7f3881b44b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x56177d4cbbd0 con 0x56177d4bb400 2024-03-23T18:00:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.129+0000 7f3881b44b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x56177d4cbd40 con 0x56177d4baae0 2024-03-23T18:00:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.129+0000 7f3881b44b80 10 client.0 _ll_register_callbacks cb 0x56177d3aa410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T18:00:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.129+0000 7f3867400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56177d4baae0 0x56177d4ccb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.129+0000 7f3866a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56177d4bb400 0x56177d4cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.129+0000 7f3866a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56177d4bb400 0x56177d4cafe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:35194/0 (socket says 172.21.15.86:35194) 2024-03-23T18:00:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.129+0000 7f3866a00700 1 -- 172.21.15.86:0/3903298582 learned_addr learned my addr 172.21.15.86:0/3903298582 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:00:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.129+0000 7f3867400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56177d4baae0 0x56177d4ccb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:55954/0 (socket says 172.21.15.86:55954) 2024-03-23T18:00:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.130+0000 7f3866a00700 1 -- 172.21.15.86:0/3903298582 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56177d4baae0 msgr2=0x56177d4ccb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:00:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.130+0000 7f3866a00700 1 --2- 172.21.15.86:0/3903298582 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56177d4baae0 0x56177d4ccb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.130+0000 7f3866a00700 1 -- 172.21.15.86:0/3903298582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56177d4c62b0 msgr2=0x56177d4cb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:00:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.130+0000 7f3866a00700 1 --2- 172.21.15.86:0/3903298582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56177d4c62b0 0x56177d4cb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.130+0000 7f3866a00700 1 -- 172.21.15.86:0/3903298582 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38681a5e40 con 0x56177d4bb400 2024-03-23T18:00:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.130+0000 7f3866a00700 1 --2- 172.21.15.86:0/3903298582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56177d4bb400 0x56177d4cafe0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3854099c30 tx=0x7f3854010c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bad98302d0ed34b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:00:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.130+0000 7f3850c00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T18:00:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.130+0000 7f3850c00700 1 -- 172.21.15.86:0/3903298582 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3854096270 con 0x56177d4bb400 2024-03-23T18:00:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.130+0000 7f3850c00700 1 -- 172.21.15.86:0/3903298582 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f38540b82c0 con 0x56177d4bb400 2024-03-23T18:00:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.130+0000 7f3881b44b80 1 -- 172.21.15.86:0/3903298582 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f385801e040 con 0x56177d4bb400 2024-03-23T18:00:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.130+0000 7f3850c00700 1 -- 172.21.15.86:0/3903298582 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f38540a23b0 con 0x56177d4bb400 2024-03-23T18:00:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.130+0000 7f3881b44b80 10 client.26822 fetch_fsmap learned FSMap version 45 2024-03-23T18:00:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.130+0000 7f3881b44b80 1 -- 172.21.15.86:0/3903298582 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x56177d28ac70 con 0x56177d4bb400 2024-03-23T18:00:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.131+0000 7f3850c00700 1 -- 172.21.15.86:0/3903298582 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f385801e040 con 0x56177d4bb400 2024-03-23T18:00:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.131+0000 7f3850c00700 1 -- 172.21.15.86:0/3903298582 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 45) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f38540a1f20 con 0x56177d4bb400 2024-03-23T18:00:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.131+0000 7f3881b44b80 10 client.26822 fetch_fsmap finished waiting for FSMap version 45 2024-03-23T18:00:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.131+0000 7f3881b44b80 10 client.26822 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T18:00:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.131+0000 7f3881b44b80 20 client.26822 populate_metadata read hostname 'smithi086' 2024-03-23T18:00:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.131+0000 7f3881b44b80 1 --2- 172.21.15.86:0/3903298582 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x56177d528940 0x56177d52ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.131+0000 7f3881b44b80 4 client.26822 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T18:00:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.131+0000 7f3881b44b80 1 -- 172.21.15.86:0/3903298582 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56177d253c90 con 0x56177d528940 2024-03-23T18:00:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.131+0000 7f3867400700 1 --2- 172.21.15.86:0/3903298582 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x56177d528940 0x56177d52ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.132+0000 7f3850c00700 10 client.26822 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T18:00:27.135 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.132+0000 7f3867400700 1 --2- 172.21.15.86:0/3903298582 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x56177d528940 0x56177d52ad60 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31590e7756260af6 server_cookie=c87abb04c130baee in_seq=0 out_seq=0 2024-03-23T18:00:27.175 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.175+0000 7f3850c00700 1 -- 172.21.15.86:0/3903298582 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56177d253c90 con 0x56177d528940 2024-03-23T18:00:27.175 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.175+0000 7f3850c00700 10 client.26822 handle_command_reply: tid=0 2024-03-23T18:00:27.175 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.175+0000 7f3881b44b80 1 client.26822 shutdown 2024-03-23T18:00:27.175 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.175+0000 7f3881b44b80 1 -- 172.21.15.86:0/3903298582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56177d4bb400 msgr2=0x56177d4cafe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:00:27.175 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.175+0000 7f3881b44b80 1 --2- 172.21.15.86:0/3903298582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56177d4bb400 0x56177d4cafe0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3854099c30 tx=0x7f3854010c80 comp rx=0 tx=0).stop 2024-03-23T18:00:27.176 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.175+0000 7f3881b44b80 1 -- 172.21.15.86:0/3903298582 shutdown_connections 2024-03-23T18:00:27.176 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.175+0000 7f3881b44b80 1 --2- 172.21.15.86:0/3903298582 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x56177d528940 0x56177d52ad60 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:27.176 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.175+0000 7f3881b44b80 1 --2- 172.21.15.86:0/3903298582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56177d4c62b0 0x56177d4cb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:27.176 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.175+0000 7f3881b44b80 1 --2- 172.21.15.86:0/3903298582 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56177d4baae0 0x56177d4ccb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:27.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.176+0000 7f3881b44b80 1 --2- 172.21.15.86:0/3903298582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56177d4bb400 0x56177d4cafe0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:27.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.176+0000 7f3881b44b80 1 -- 172.21.15.86:0/3903298582 >> 172.21.15.86:0/3903298582 conn(0x56177d412640 msgr2=0x56177d4c17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:00:27.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.176+0000 7f3850c00700 0 client.26822 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T18:00:27.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.176+0000 7f3881b44b80 1 -- 172.21.15.86:0/3903298582 shutdown_connections 2024-03-23T18:00:27.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.176+0000 7f3881b44b80 1 -- 172.21.15.86:0/3903298582 wait complete. 2024-03-23T18:00:27.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.176+0000 7f3881b44b80 20 client.26822 trim_cache size 0 max 16384 2024-03-23T18:00:27.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-03-23T18:00:27.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-03-23T18:00:27.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-03-23T18:00:27.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-03-23T18:00:27.178 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.177+0000 7f386ca00700 1 -- 172.21.15.86:0/3313371974 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f384c0a1310 msgr2=0x7f384c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:00:27.178 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.177+0000 7f386ca00700 1 --2- 172.21.15.86:0/3313371974 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f384c0a1310 0x7f384c0a37d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f385800b2e0 tx=0x7f3858005960 comp rx=0 tx=0).stop 2024-03-23T18:00:27.178 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.177+0000 7f386ca00700 1 -- 172.21.15.86:0/3313371974 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3868101490 msgr2=0x7f38680735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:00:27.178 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.177+0000 7f386ca00700 1 --2- 172.21.15.86:0/3313371974 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3868101490 0x7f38680735f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f385400e9a0 tx=0x7f385400ecb0 comp rx=0 tx=0).stop 2024-03-23T18:00:27.179 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.177+0000 7f386ca00700 1 -- 172.21.15.86:0/3313371974 shutdown_connections 2024-03-23T18:00:27.181 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.177+0000 7f386ca00700 1 --2- 172.21.15.86:0/3313371974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3868106b70 0x7f3868073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:27.181 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.177+0000 7f386ca00700 1 --2- 172.21.15.86:0/3313371974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3868100ad0 0x7f38680730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:27.181 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.177+0000 7f386ca00700 1 --2- 172.21.15.86:0/3313371974 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f384c0a1310 0x7f384c0a37d0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:27.181 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.177+0000 7f386ca00700 1 --2- 172.21.15.86:0/3313371974 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3868101490 0x7f38680735f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:27.181 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.177+0000 7f386ca00700 1 -- 172.21.15.86:0/3313371974 >> 172.21.15.86:0/3313371974 conn(0x7f38680fc5e0 msgr2=0x7f38680fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:00:27.181 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.177+0000 7f386ca00700 1 -- 172.21.15.86:0/3313371974 shutdown_connections 2024-03-23T18:00:27.182 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:27.177+0000 7f386ca00700 1 -- 172.21.15.86:0/3313371974 wait complete. 2024-03-23T18:00:27.919 INFO:tasks.workunit.client.0.smithi086.stdout: 1 556496 6.24 MB/sec execute 412 sec latency 21.794 ms 2024-03-23T18:00:28.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:00:27 smithi086 ceph-mon[38221]: pgmap v305: 97 pgs: 97 active+clean; 228 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.7 MiB/s wr, 212 op/s 2024-03-23T18:00:28.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:00:27 smithi092 ceph-mon[79089]: pgmap v305: 97 pgs: 97 active+clean; 228 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.7 MiB/s wr, 212 op/s 2024-03-23T18:00:28.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:00:27 smithi152 ceph-mon[47949]: pgmap v305: 97 pgs: 97 active+clean; 228 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.7 MiB/s wr, 212 op/s 2024-03-23T18:00:28.919 INFO:tasks.workunit.client.0.smithi086.stdout: 1 558097 6.25 MB/sec execute 413 sec latency 12.849 ms 2024-03-23T18:00:29.919 INFO:tasks.workunit.client.0.smithi086.stdout: 1 559550 6.25 MB/sec execute 414 sec latency 29.024 ms 2024-03-23T18:00:30.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:00:29 smithi086 ceph-mon[38221]: pgmap v306: 97 pgs: 97 active+clean; 228 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.7 MiB/s wr, 212 op/s 2024-03-23T18:00:30.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:00:29 smithi092 ceph-mon[79089]: pgmap v306: 97 pgs: 97 active+clean; 228 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.7 MiB/s wr, 212 op/s 2024-03-23T18:00:30.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:00:29 smithi152 ceph-mon[47949]: pgmap v306: 97 pgs: 97 active+clean; 228 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.7 MiB/s wr, 212 op/s 2024-03-23T18:00:30.919 INFO:tasks.workunit.client.0.smithi086.stdout: 1 561497 6.27 MB/sec execute 415 sec latency 13.993 ms 2024-03-23T18:00:31.919 INFO:tasks.workunit.client.0.smithi086.stdout: 1 562894 6.27 MB/sec execute 416 sec latency 12.772 ms 2024-03-23T18:00:32.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:00:31 smithi086 ceph-mon[38221]: pgmap v307: 97 pgs: 97 active+clean; 230 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.2 MiB/s wr, 341 op/s 2024-03-23T18:00:32.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:00:31 smithi092 ceph-mon[79089]: pgmap v307: 97 pgs: 97 active+clean; 230 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.2 MiB/s wr, 341 op/s 2024-03-23T18:00:32.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:00:31 smithi152 ceph-mon[47949]: pgmap v307: 97 pgs: 97 active+clean; 230 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.2 MiB/s wr, 341 op/s 2024-03-23T18:00:32.919 INFO:tasks.workunit.client.0.smithi086.stdout: 1 565007 6.29 MB/sec execute 417 sec latency 14.976 ms 2024-03-23T18:00:33.919 INFO:tasks.workunit.client.0.smithi086.stdout: 1 566187 6.28 MB/sec execute 418 sec latency 38.066 ms 2024-03-23T18:00:34.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:00:33 smithi086 ceph-mon[38221]: pgmap v308: 97 pgs: 97 active+clean; 230 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 233 op/s 2024-03-23T18:00:34.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:00:33 smithi092 ceph-mon[79089]: pgmap v308: 97 pgs: 97 active+clean; 230 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 233 op/s 2024-03-23T18:00:34.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:00:33 smithi152 ceph-mon[47949]: pgmap v308: 97 pgs: 97 active+clean; 230 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 233 op/s 2024-03-23T18:00:34.919 INFO:tasks.workunit.client.0.smithi086.stdout: 1 567807 6.29 MB/sec execute 419 sec latency 62.482 ms 2024-03-23T18:00:35.919 INFO:tasks.workunit.client.0.smithi086.stdout: 1 569171 6.29 MB/sec execute 420 sec latency 15.181 ms 2024-03-23T18:00:36.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:00:35 smithi086 ceph-mon[38221]: pgmap v309: 97 pgs: 97 active+clean; 234 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.0 MiB/s wr, 340 op/s 2024-03-23T18:00:36.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:00:35 smithi092 ceph-mon[79089]: pgmap v309: 97 pgs: 97 active+clean; 234 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.0 MiB/s wr, 340 op/s 2024-03-23T18:00:36.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:00:35 smithi152 ceph-mon[47949]: pgmap v309: 97 pgs: 97 active+clean; 234 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.0 MiB/s wr, 340 op/s 2024-03-23T18:00:36.919 INFO:tasks.workunit.client.0.smithi086.stdout: 1 570951 6.29 MB/sec execute 421 sec latency 16.492 ms 2024-03-23T18:00:37.919 INFO:tasks.workunit.client.0.smithi086.stdout: 1 572443 6.30 MB/sec execute 422 sec latency 12.071 ms 2024-03-23T18:00:38.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:00:37 smithi086 ceph-mon[38221]: pgmap v310: 97 pgs: 97 active+clean; 234 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 236 op/s 2024-03-23T18:00:38.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:00:37 smithi092 ceph-mon[79089]: pgmap v310: 97 pgs: 97 active+clean; 234 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 236 op/s 2024-03-23T18:00:38.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:00:37 smithi152 ceph-mon[47949]: pgmap v310: 97 pgs: 97 active+clean; 234 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 236 op/s 2024-03-23T18:00:38.919 INFO:tasks.workunit.client.0.smithi086.stdout: 1 574265 6.30 MB/sec execute 423 sec latency 20.413 ms 2024-03-23T18:00:39.919 INFO:tasks.workunit.client.0.smithi086.stdout: 1 575806 6.32 MB/sec execute 424 sec latency 20.378 ms 2024-03-23T18:00:40.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:00:39 smithi086 ceph-mon[38221]: pgmap v311: 97 pgs: 97 active+clean; 234 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 236 op/s 2024-03-23T18:00:40.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:00:39 smithi092 ceph-mon[79089]: pgmap v311: 97 pgs: 97 active+clean; 234 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 236 op/s 2024-03-23T18:00:40.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:00:39 smithi152 ceph-mon[47949]: pgmap v311: 97 pgs: 97 active+clean; 234 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 236 op/s 2024-03-23T18:00:40.345 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.345+0000 7f8ec3200700 1 -- 172.21.15.86:0/1240362952 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ebc105c60 msgr2=0x7f8ebc106040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:00:40.345 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.345+0000 7f8ec3200700 1 --2- 172.21.15.86:0/1240362952 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ebc105c60 0x7f8ebc106040 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f8eac009990 tx=0x7f8eac009ca0 comp rx=0 tx=0).stop 2024-03-23T18:00:40.345 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.345+0000 7f8ec3200700 1 -- 172.21.15.86:0/1240362952 shutdown_connections 2024-03-23T18:00:40.345 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.345+0000 7f8ec3200700 1 --2- 172.21.15.86:0/1240362952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ebc100570 0x7f8ebc10ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:40.346 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.345+0000 7f8ec3200700 1 --2- 172.21.15.86:0/1240362952 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ebc105c60 0x7f8ebc106040 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:40.346 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.345+0000 7f8ec3200700 1 --2- 172.21.15.86:0/1240362952 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8ebc0ffbb0 0x7f8ebc100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:40.346 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.345+0000 7f8ec3200700 1 -- 172.21.15.86:0/1240362952 >> 172.21.15.86:0/1240362952 conn(0x7f8ebc076d50 msgr2=0x7f8ebc077160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:00:40.346 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.345+0000 7f8ec3200700 1 -- 172.21.15.86:0/1240362952 shutdown_connections 2024-03-23T18:00:40.346 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.345+0000 7f8ec3200700 1 -- 172.21.15.86:0/1240362952 wait complete. 2024-03-23T18:00:40.346 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.346+0000 7f8ec3200700 1 Processor -- start 2024-03-23T18:00:40.347 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.346+0000 7f8ec3200700 1 -- start start 2024-03-23T18:00:40.347 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.347+0000 7f8ec3200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ebc0ffbb0 0x7f8ebc199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:40.347 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.347+0000 7f8ec3200700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8ebc100570 0x7f8ebc1998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:40.349 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.347+0000 7f8ec3200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ebc105c60 0x7f8ebc19dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:40.349 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.347+0000 7f8ec3200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ebc19e230 con 0x7f8ebc0ffbb0 2024-03-23T18:00:40.349 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.347+0000 7f8ec3200700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ebc19e3a0 con 0x7f8ebc105c60 2024-03-23T18:00:40.349 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.347+0000 7f8ec3200700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ebc19e560 con 0x7f8ebc100570 2024-03-23T18:00:40.349 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.347+0000 7f8ebb400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8ebc100570 0x7f8ebc1998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:40.349 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.347+0000 7f8ec1200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ebc105c60 0x7f8ebc19dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:40.349 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.347+0000 7f8ebb400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8ebc100570 0x7f8ebc1998a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:44364/0 (socket says 172.21.15.86:44364) 2024-03-23T18:00:40.349 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.347+0000 7f8ebbe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ebc0ffbb0 0x7f8ebc199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:40.349 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.347+0000 7f8ebb400700 1 -- 172.21.15.86:0/1541735565 learned_addr learned my addr 172.21.15.86:0/1541735565 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:00:40.349 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.348+0000 7f8ec1200700 1 -- 172.21.15.86:0/1541735565 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8ebc100570 msgr2=0x7f8ebc1998a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:00:40.349 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.348+0000 7f8ec1200700 1 --2- 172.21.15.86:0/1541735565 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8ebc100570 0x7f8ebc1998a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:40.349 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.348+0000 7f8ec1200700 1 -- 172.21.15.86:0/1541735565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ebc0ffbb0 msgr2=0x7f8ebc199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:00:40.350 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.348+0000 7f8ec1200700 1 --2- 172.21.15.86:0/1541735565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ebc0ffbb0 0x7f8ebc199360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:40.350 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.348+0000 7f8ec1200700 1 -- 172.21.15.86:0/1541735565 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eac0095f0 con 0x7f8ebc105c60 2024-03-23T18:00:40.350 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.348+0000 7f8ebbe00700 1 --2- 172.21.15.86:0/1541735565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ebc0ffbb0 0x7f8ebc199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T18:00:40.350 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.348+0000 7f8ec1200700 1 --2- 172.21.15.86:0/1541735565 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ebc105c60 0x7f8ebc19dbf0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f8eb4009990 tx=0x7f8eb4009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:00:40.350 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.348+0000 7f8eb8c00700 1 -- 172.21.15.86:0/1541735565 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb4010070 con 0x7f8ebc105c60 2024-03-23T18:00:40.350 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.348+0000 7f8eb8c00700 1 -- 172.21.15.86:0/1541735565 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8eb4008d40 con 0x7f8ebc105c60 2024-03-23T18:00:40.350 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.348+0000 7f8ec3200700 1 -- 172.21.15.86:0/1541735565 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ebc19e840 con 0x7f8ebc105c60 2024-03-23T18:00:40.350 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.348+0000 7f8eb8c00700 1 -- 172.21.15.86:0/1541735565 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb4016df0 con 0x7f8ebc105c60 2024-03-23T18:00:40.350 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.348+0000 7f8ec3200700 1 -- 172.21.15.86:0/1541735565 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ebc106b20 con 0x7f8ebc105c60 2024-03-23T18:00:40.353 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.353+0000 7f8eb8c00700 1 -- 172.21.15.86:0/1541735565 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f8eb4014070 con 0x7f8ebc105c60 2024-03-23T18:00:40.354 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.354+0000 7f8ed6c23b80 1 Processor -- start 2024-03-23T18:00:40.356 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.354+0000 7f8ed6c23b80 1 -- start start 2024-03-23T18:00:40.356 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.355+0000 7f8eb8c00700 1 --2- 172.21.15.86:0/1541735565 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f8ea00a1310 0x7f8ea00a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:40.356 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.355+0000 7f8ed6c23b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5633ee5d5100 0x5633ee5d54e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:40.356 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.355+0000 7f8ed6c23b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5633ee5d5a20 0x5633ee5e0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:40.356 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.355+0000 7f8eb8c00700 1 -- 172.21.15.86:0/1541735565 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f8eb40cbdf0 con 0x7f8ebc105c60 2024-03-23T18:00:40.356 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.355+0000 7f8ed6c23b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633ee5e08d0 0x5633ee5e2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:40.356 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.355+0000 7f8ed6c23b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633ee52e930 con 0x5633ee5e08d0 2024-03-23T18:00:40.356 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.355+0000 7f8ed6c23b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633ee52c2f0 con 0x5633ee5d5100 2024-03-23T18:00:40.356 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.355+0000 7f8ed6c23b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633ee527be0 con 0x5633ee5d5a20 2024-03-23T18:00:40.356 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.355+0000 7f8ebbe00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5633ee5d5100 0x5633ee5d54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:40.356 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.355+0000 7f8ebbe00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5633ee5d5100 0x5633ee5d54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:51672/0 (socket says 172.21.15.86:51672) 2024-03-23T18:00:40.356 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.355+0000 7f8ebbe00700 1 -- 172.21.15.86:0/2756508658 learned_addr learned my addr 172.21.15.86:0/2756508658 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:00:40.357 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.355+0000 7f8ec1200700 1 --2- 172.21.15.86:0/2756508658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633ee5e08d0 0x5633ee5e2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:40.357 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.356+0000 7f8ebbe00700 1 -- 172.21.15.86:0/2756508658 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5633ee5d5a20 msgr2=0x5633ee5e0390 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-23T18:00:40.357 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.356+0000 7f8ebbe00700 1 --2- 172.21.15.86:0/2756508658 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5633ee5d5a20 0x5633ee5e0390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:40.357 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.356+0000 7f8ebbe00700 1 -- 172.21.15.86:0/2756508658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633ee5e08d0 msgr2=0x5633ee5e2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:00:40.357 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.356+0000 7f8ebbe00700 1 --2- 172.21.15.86:0/2756508658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633ee5e08d0 0x5633ee5e2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:40.357 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.356+0000 7f8ebbe00700 1 -- 172.21.15.86:0/2756508658 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633ee3a9350 con 0x5633ee5d5100 2024-03-23T18:00:40.357 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.356+0000 7f8ebb400700 1 --2- 172.21.15.86:0/1541735565 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f8ea00a1310 0x7f8ea00a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:40.357 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.356+0000 7f8ebbe00700 1 --2- 172.21.15.86:0/2756508658 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5633ee5d5100 0x5633ee5d54e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8eac006010 tx=0x7f8eac005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a2597d54b928de0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:00:40.358 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.356+0000 7f8eb1600700 1 -- 172.21.15.86:0/2756508658 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eac00b7d0 con 0x5633ee5d5100 2024-03-23T18:00:40.358 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.356+0000 7f8eb1600700 1 -- 172.21.15.86:0/2756508658 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8eac02e040 con 0x5633ee5d5100 2024-03-23T18:00:40.358 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.356+0000 7f8eb1600700 1 -- 172.21.15.86:0/2756508658 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eac003c30 con 0x5633ee5d5100 2024-03-23T18:00:40.358 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.357+0000 7f8ed6c23b80 1 -- 172.21.15.86:0/2756508658 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5633ee5d5100 msgr2=0x5633ee5d54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:00:40.358 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.357+0000 7f8ebb400700 1 --2- 172.21.15.86:0/1541735565 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f8ea00a1310 0x7f8ea00a37d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8015fd0 tx=0x7f8ea800b500 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:00:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.357+0000 7f8ed6c23b80 1 --2- 172.21.15.86:0/2756508658 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5633ee5d5100 0x5633ee5d54e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8eac006010 tx=0x7f8eac005e80 comp rx=0 tx=0).stop 2024-03-23T18:00:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.357+0000 7f8ed6c23b80 1 -- 172.21.15.86:0/2756508658 shutdown_connections 2024-03-23T18:00:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.357+0000 7f8ed6c23b80 1 --2- 172.21.15.86:0/2756508658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633ee5e08d0 0x5633ee5e2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.357+0000 7f8ed6c23b80 1 --2- 172.21.15.86:0/2756508658 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5633ee5d5100 0x5633ee5d54e0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.357+0000 7f8ed6c23b80 1 --2- 172.21.15.86:0/2756508658 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5633ee5d5a20 0x5633ee5e0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.357+0000 7f8ed6c23b80 1 -- 172.21.15.86:0/2756508658 >> 172.21.15.86:0/2756508658 conn(0x5633ee535640 msgr2=0x5633ee5d1520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:00:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.357+0000 7f8ed6c23b80 1 -- 172.21.15.86:0/2756508658 shutdown_connections 2024-03-23T18:00:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.357+0000 7f8ed6c23b80 1 -- 172.21.15.86:0/2756508658 wait complete. 2024-03-23T18:00:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.358+0000 7f8ed6c23b80 1 -- start start 2024-03-23T18:00:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.358+0000 7f8ed6c23b80 1 Processor -- start 2024-03-23T18:00:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.359+0000 7f8ed6c23b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5633ee5d5100 0x5633ee5e02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.359+0000 7f8ed6c23b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633ee5d5a20 0x5633ee5dd520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.359+0000 7f8ed6c23b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5633ee5e08d0 0x5633ee5ddb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:40.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.359+0000 7f8ed6c23b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633ee4c3d20 con 0x5633ee5d5a20 2024-03-23T18:00:40.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.359+0000 7f8ed6c23b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633ee4c39c0 con 0x5633ee5d5100 2024-03-23T18:00:40.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.359+0000 7f8ed6c23b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633ee3ab640 con 0x5633ee5e08d0 2024-03-23T18:00:40.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.359+0000 7f8ed6c23b80 10 client.0 _ll_register_callbacks cb 0x5633ee4cd410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T18:00:40.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.359+0000 7f8ebbe00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5633ee5d5100 0x5633ee5e02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:40.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.359+0000 7f8ebbe00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5633ee5d5100 0x5633ee5e02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:51688/0 (socket says 172.21.15.86:51688) 2024-03-23T18:00:40.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.359+0000 7f8ebbe00700 1 -- 172.21.15.86:0/274570234 learned_addr learned my addr 172.21.15.86:0/274570234 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:00:40.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.359+0000 7f8ec1200700 1 --2- 172.21.15.86:0/274570234 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5633ee5e08d0 0x5633ee5ddb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:40.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.359+0000 7f8ebb400700 1 --2- 172.21.15.86:0/274570234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633ee5d5a20 0x5633ee5dd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:40.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.359+0000 7f8ec1200700 1 -- 172.21.15.86:0/274570234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5633ee5d5100 msgr2=0x5633ee5e02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:00:40.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.359+0000 7f8ec1200700 1 --2- 172.21.15.86:0/274570234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5633ee5d5100 0x5633ee5e02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:40.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.359+0000 7f8ec1200700 1 -- 172.21.15.86:0/274570234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633ee5d5a20 msgr2=0x5633ee5dd520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:00:40.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.359+0000 7f8ec1200700 1 --2- 172.21.15.86:0/274570234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633ee5d5a20 0x5633ee5dd520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:40.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.359+0000 7f8ec1200700 1 -- 172.21.15.86:0/274570234 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ebc10d2f0 con 0x5633ee5e08d0 2024-03-23T18:00:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.359+0000 7f8ebb400700 1 --2- 172.21.15.86:0/274570234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633ee5d5a20 0x5633ee5dd520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T18:00:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.360+0000 7f8ec1200700 1 --2- 172.21.15.86:0/274570234 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5633ee5e08d0 0x5633ee5ddb30 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8eb40c9500 tx=0x7f8eb4098af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc87259ee754afa8 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:00:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.360+0000 7f8ea6000700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-03-23T18:00:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.360+0000 7f8ea6000700 1 -- 172.21.15.86:0/274570234 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb409ccf0 con 0x5633ee5e08d0 2024-03-23T18:00:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.360+0000 7f8ea6000700 1 -- 172.21.15.86:0/274570234 <== mon.1 v2:172.21.15.152:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f8eb40a1340 con 0x5633ee5e08d0 2024-03-23T18:00:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.360+0000 7f8ed6c23b80 1 -- 172.21.15.86:0/274570234 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5633ee4e8320 con 0x5633ee5e08d0 2024-03-23T18:00:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.360+0000 7f8ea6000700 1 -- 172.21.15.86:0/274570234 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8eb40983b0 con 0x5633ee5e08d0 2024-03-23T18:00:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.360+0000 7f8ed6c23b80 10 client.26815 fetch_fsmap learned FSMap version 45 2024-03-23T18:00:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.360+0000 7f8ed6c23b80 1 -- 172.21.15.86:0/274570234 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x5633ee355b00 con 0x5633ee5e08d0 2024-03-23T18:00:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.360+0000 7f8ea6000700 1 -- 172.21.15.86:0/274570234 <== mon.1 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5633ee4e8320 con 0x5633ee5e08d0 2024-03-23T18:00:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.361+0000 7f8ea6000700 1 -- 172.21.15.86:0/274570234 <== mon.1 v2:172.21.15.152:3300/0 5 ==== fsmap(e 45) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f8eb4094cb0 con 0x5633ee5e08d0 2024-03-23T18:00:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.361+0000 7f8ed6c23b80 10 client.26815 fetch_fsmap finished waiting for FSMap version 45 2024-03-23T18:00:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.361+0000 7f8ed6c23b80 10 client.26815 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T18:00:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.361+0000 7f8ed6c23b80 20 client.26815 populate_metadata read hostname 'smithi086' 2024-03-23T18:00:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.361+0000 7f8ed6c23b80 1 --2- 172.21.15.86:0/274570234 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5633ee5f05c0 0x5633ee5f29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.361+0000 7f8ed6c23b80 4 client.26815 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T18:00:40.364 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.361+0000 7f8ed6c23b80 1 -- 172.21.15.86:0/274570234 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5633ee5206a0 con 0x5633ee5f05c0 2024-03-23T18:00:40.364 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.361+0000 7f8ebbe00700 1 --2- 172.21.15.86:0/274570234 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5633ee5f05c0 0x5633ee5f29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:40.364 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.361+0000 7f8ea6000700 10 client.26815 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T18:00:40.364 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.361+0000 7f8ebbe00700 1 --2- 172.21.15.86:0/274570234 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5633ee5f05c0 0x5633ee5f29e0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9c688c32b63134c server_cookie=40f92783c73effc1 in_seq=0 out_seq=0 2024-03-23T18:00:40.365 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.362+0000 7f8ea6000700 1 -- 172.21.15.86:0/274570234 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5633ee5206a0 con 0x5633ee5f05c0 2024-03-23T18:00:40.365 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.362+0000 7f8ea6000700 10 client.26815 handle_command_reply: tid=0 2024-03-23T18:00:40.365 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.362+0000 7f8ed6c23b80 1 client.26815 shutdown 2024-03-23T18:00:40.365 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.363+0000 7f8ed6c23b80 1 -- 172.21.15.86:0/274570234 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5633ee5e08d0 msgr2=0x5633ee5ddb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:00:40.365 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.363+0000 7f8ed6c23b80 1 --2- 172.21.15.86:0/274570234 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5633ee5e08d0 0x5633ee5ddb30 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8eb40c9500 tx=0x7f8eb4098af0 comp rx=0 tx=0).stop 2024-03-23T18:00:40.365 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.363+0000 7f8ed6c23b80 1 -- 172.21.15.86:0/274570234 shutdown_connections 2024-03-23T18:00:40.365 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.363+0000 7f8ed6c23b80 1 --2- 172.21.15.86:0/274570234 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5633ee5f05c0 0x5633ee5f29e0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:40.365 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.363+0000 7f8ed6c23b80 1 --2- 172.21.15.86:0/274570234 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5633ee5e08d0 0x5633ee5ddb30 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:40.366 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.363+0000 7f8ed6c23b80 1 --2- 172.21.15.86:0/274570234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5633ee5d5100 0x5633ee5e02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:40.366 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.363+0000 7f8ed6c23b80 1 --2- 172.21.15.86:0/274570234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633ee5d5a20 0x5633ee5dd520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:40.366 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.363+0000 7f8ea6000700 0 client.26815 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T18:00:40.366 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.363+0000 7f8ed6c23b80 1 -- 172.21.15.86:0/274570234 >> 172.21.15.86:0/274570234 conn(0x5633ee535640 msgr2=0x5633ee5d70a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:00:40.366 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.363+0000 7f8ed6c23b80 1 -- 172.21.15.86:0/274570234 shutdown_connections 2024-03-23T18:00:40.366 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.363+0000 7f8ed6c23b80 1 -- 172.21.15.86:0/274570234 wait complete. 2024-03-23T18:00:40.366 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.363+0000 7f8ed6c23b80 20 client.26815 trim_cache size 0 max 16384 2024-03-23T18:00:40.380 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.380+0000 7f8ed6c23b80 1 Processor -- start 2024-03-23T18:00:40.380 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.380+0000 7f8ed6c23b80 1 -- start start 2024-03-23T18:00:40.382 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.381+0000 7f8ed6c23b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633ee5d5100 0x5633ee5e7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:40.382 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.381+0000 7f8ed6c23b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5633ee5d5a20 0x5633ee5dc490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:40.382 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.381+0000 7f8ed6c23b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5633ee5e08d0 0x5633ee5cb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:40.382 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.381+0000 7f8ed6c23b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633ee5edb40 con 0x5633ee5d5100 2024-03-23T18:00:40.382 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.381+0000 7f8ed6c23b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633ee5dde70 con 0x5633ee5d5a20 2024-03-23T18:00:40.382 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.381+0000 7f8ed6c23b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633ee5d93a0 con 0x5633ee5e08d0 2024-03-23T18:00:40.382 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.381+0000 7f8ec1200700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5633ee5e08d0 0x5633ee5cb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:40.382 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.381+0000 7f8ec1200700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5633ee5e08d0 0x5633ee5cb500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:44382/0 (socket says 172.21.15.86:44382) 2024-03-23T18:00:40.382 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.381+0000 7f8ec1200700 1 -- 172.21.15.86:0/1177381877 learned_addr learned my addr 172.21.15.86:0/1177381877 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:00:40.382 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.381+0000 7f8ebb400700 1 --2- 172.21.15.86:0/1177381877 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5633ee5d5a20 0x5633ee5dc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:40.382 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.381+0000 7f8ebbe00700 1 --2- 172.21.15.86:0/1177381877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633ee5d5100 0x5633ee5e7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:40.382 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.381+0000 7f8ec1200700 1 -- 172.21.15.86:0/1177381877 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5633ee5d5a20 msgr2=0x5633ee5dc490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:00:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.381+0000 7f8ec1200700 1 --2- 172.21.15.86:0/1177381877 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5633ee5d5a20 0x5633ee5dc490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.381+0000 7f8ec1200700 1 -- 172.21.15.86:0/1177381877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633ee5d5100 msgr2=0x5633ee5e7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:00:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.381+0000 7f8ec1200700 1 --2- 172.21.15.86:0/1177381877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633ee5d5100 0x5633ee5e7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.381+0000 7f8ec1200700 1 -- 172.21.15.86:0/1177381877 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633ee4ede40 con 0x5633ee5e08d0 2024-03-23T18:00:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.381+0000 7f8ebbe00700 1 --2- 172.21.15.86:0/1177381877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633ee5d5100 0x5633ee5e7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T18:00:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.382+0000 7f8ec1200700 1 --2- 172.21.15.86:0/1177381877 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5633ee5e08d0 0x5633ee5cb500 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8eb4098bc0 tx=0x7f8eb4099700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61c46f9dc7e7888e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:00:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.382+0000 7f8eb2000700 1 -- 172.21.15.86:0/1177381877 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb400de40 con 0x5633ee5e08d0 2024-03-23T18:00:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.382+0000 7f8eb2000700 1 -- 172.21.15.86:0/1177381877 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8eb40a1340 con 0x5633ee5e08d0 2024-03-23T18:00:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.382+0000 7f8eb2000700 1 -- 172.21.15.86:0/1177381877 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb40a1110 con 0x5633ee5e08d0 2024-03-23T18:00:40.386 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.382+0000 7f8ed6c23b80 1 -- 172.21.15.86:0/1177381877 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5633ee5e08d0 msgr2=0x5633ee5cb500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:00:40.386 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.382+0000 7f8ed6c23b80 1 --2- 172.21.15.86:0/1177381877 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5633ee5e08d0 0x5633ee5cb500 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8eb4098bc0 tx=0x7f8eb4099700 comp rx=0 tx=0).stop 2024-03-23T18:00:40.386 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.382+0000 7f8ed6c23b80 1 -- 172.21.15.86:0/1177381877 shutdown_connections 2024-03-23T18:00:40.386 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.382+0000 7f8ed6c23b80 1 --2- 172.21.15.86:0/1177381877 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5633ee5e08d0 0x5633ee5cb500 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:40.386 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.382+0000 7f8ed6c23b80 1 --2- 172.21.15.86:0/1177381877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633ee5d5100 0x5633ee5e7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:40.387 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.382+0000 7f8ed6c23b80 1 --2- 172.21.15.86:0/1177381877 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5633ee5d5a20 0x5633ee5dc490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:40.387 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.382+0000 7f8ed6c23b80 1 -- 172.21.15.86:0/1177381877 >> 172.21.15.86:0/1177381877 conn(0x5633ee535640 msgr2=0x5633ee5d0d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:00:40.387 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.383+0000 7f8ed6c23b80 1 -- 172.21.15.86:0/1177381877 shutdown_connections 2024-03-23T18:00:40.387 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.383+0000 7f8ed6c23b80 1 -- 172.21.15.86:0/1177381877 wait complete. 2024-03-23T18:00:40.387 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.383+0000 7f8ed6c23b80 1 -- start start 2024-03-23T18:00:40.387 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.383+0000 7f8ed6c23b80 1 Processor -- start 2024-03-23T18:00:40.387 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.383+0000 7f8ed6c23b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5633ee5d5100 0x5633ee5e5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:40.387 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.384+0000 7f8ed6c23b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5633ee5d5a20 0x5633ee5e5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:40.387 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.384+0000 7f8ed6c23b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633ee5e08d0 0x5633ee63b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:40.387 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.384+0000 7f8ed6c23b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633ee63b680 con 0x5633ee5e08d0 2024-03-23T18:00:40.387 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.384+0000 7f8ed6c23b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633ee63b7f0 con 0x5633ee5d5a20 2024-03-23T18:00:40.387 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.384+0000 7f8ed6c23b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633ee63b9b0 con 0x5633ee5d5100 2024-03-23T18:00:40.387 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.384+0000 7f8ed6c23b80 10 client.0 _ll_register_callbacks cb 0x5633ee4cd410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T18:00:40.387 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.384+0000 7f8ebb400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5633ee5d5a20 0x5633ee5e5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:40.388 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.384+0000 7f8ec1200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633ee5e08d0 0x5633ee63b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:40.388 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.384+0000 7f8ebb400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5633ee5d5a20 0x5633ee5e5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:51720/0 (socket says 172.21.15.86:51720) 2024-03-23T18:00:40.388 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.384+0000 7f8ec1200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633ee5e08d0 0x5633ee63b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39388/0 (socket says 172.21.15.86:39388) 2024-03-23T18:00:40.388 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.384+0000 7f8ebb400700 1 -- 172.21.15.86:0/4211483366 learned_addr learned my addr 172.21.15.86:0/4211483366 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:00:40.388 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.384+0000 7f8ebbe00700 1 --2- 172.21.15.86:0/4211483366 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5633ee5d5100 0x5633ee5e5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:40.388 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.384+0000 7f8ebbe00700 1 -- 172.21.15.86:0/4211483366 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5633ee5d5a20 msgr2=0x5633ee5e5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:00:40.388 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.384+0000 7f8ebbe00700 1 --2- 172.21.15.86:0/4211483366 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5633ee5d5a20 0x5633ee5e5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:40.388 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.384+0000 7f8ebbe00700 1 -- 172.21.15.86:0/4211483366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633ee5e08d0 msgr2=0x5633ee63b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:00:40.388 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.384+0000 7f8ebbe00700 1 --2- 172.21.15.86:0/4211483366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633ee5e08d0 0x5633ee63b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:40.388 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.384+0000 7f8ebbe00700 1 -- 172.21.15.86:0/4211483366 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5633ee355b00 con 0x5633ee5d5100 2024-03-23T18:00:40.388 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.384+0000 7f8ebb400700 1 --2- 172.21.15.86:0/4211483366 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5633ee5d5a20 0x5633ee5e5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T18:00:40.388 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.385+0000 7f8ebbe00700 1 --2- 172.21.15.86:0/4211483366 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5633ee5d5100 0x5633ee5e5730 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x5633ee5f3290 tx=0x7f8eac003830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49788c6f6cb6f8b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:00:40.388 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.385+0000 7f8ea6000700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-03-23T18:00:40.388 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.385+0000 7f8ea6000700 1 -- 172.21.15.86:0/4211483366 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eac0050c0 con 0x5633ee5d5100 2024-03-23T18:00:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.385+0000 7f8ea6000700 1 -- 172.21.15.86:0/4211483366 <== mon.1 v2:172.21.15.152:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f8eac005220 con 0x5633ee5d5100 2024-03-23T18:00:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.385+0000 7f8ed6c23b80 1 -- 172.21.15.86:0/4211483366 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8eac02e040 con 0x5633ee5d5100 2024-03-23T18:00:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.385+0000 7f8ea6000700 1 -- 172.21.15.86:0/4211483366 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8eac0053e0 con 0x5633ee5d5100 2024-03-23T18:00:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.385+0000 7f8ed6c23b80 10 client.26821 fetch_fsmap learned FSMap version 45 2024-03-23T18:00:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.385+0000 7f8ed6c23b80 1 -- 172.21.15.86:0/4211483366 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x5633ee3adc70 con 0x5633ee5d5100 2024-03-23T18:00:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.385+0000 7f8ea6000700 1 -- 172.21.15.86:0/4211483366 <== mon.1 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8eac02e040 con 0x5633ee5d5100 2024-03-23T18:00:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.386+0000 7f8ea6000700 1 -- 172.21.15.86:0/4211483366 <== mon.1 v2:172.21.15.152:3300/0 5 ==== fsmap(e 45) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f8eac00b260 con 0x5633ee5d5100 2024-03-23T18:00:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.386+0000 7f8ed6c23b80 10 client.26821 fetch_fsmap finished waiting for FSMap version 45 2024-03-23T18:00:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.386+0000 7f8ed6c23b80 10 client.26821 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T18:00:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.386+0000 7f8ed6c23b80 20 client.26821 populate_metadata read hostname 'smithi086' 2024-03-23T18:00:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.386+0000 7f8ed6c23b80 1 --2- 172.21.15.86:0/4211483366 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5633ee642f90 0x5633ee6453b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.386+0000 7f8ed6c23b80 4 client.26821 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T18:00:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.386+0000 7f8ed6c23b80 1 -- 172.21.15.86:0/4211483366 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5633ee376c90 con 0x5633ee642f90 2024-03-23T18:00:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.386+0000 7f8ebb400700 1 --2- 172.21.15.86:0/4211483366 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5633ee642f90 0x5633ee6453b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:40.390 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.387+0000 7f8ebb400700 1 --2- 172.21.15.86:0/4211483366 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5633ee642f90 0x5633ee6453b0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b45cc95a9dde49db server_cookie=3b46cbbfdcbae59a in_seq=0 out_seq=0 2024-03-23T18:00:40.390 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.387+0000 7f8ea6000700 10 client.26821 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T18:00:40.437 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.437+0000 7f8ea6000700 1 -- 172.21.15.86:0/4211483366 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5633ee376c90 con 0x5633ee642f90 2024-03-23T18:00:40.437 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.437+0000 7f8ea6000700 10 client.26821 handle_command_reply: tid=0 2024-03-23T18:00:40.437 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.437+0000 7f8ed6c23b80 1 client.26821 shutdown 2024-03-23T18:00:40.437 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.437+0000 7f8ed6c23b80 1 -- 172.21.15.86:0/4211483366 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5633ee5d5100 msgr2=0x5633ee5e5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:00:40.437 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.437+0000 7f8ed6c23b80 1 --2- 172.21.15.86:0/4211483366 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5633ee5d5100 0x5633ee5e5730 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x5633ee5f3290 tx=0x7f8eac003830 comp rx=0 tx=0).stop 2024-03-23T18:00:40.438 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.438+0000 7f8ed6c23b80 1 -- 172.21.15.86:0/4211483366 shutdown_connections 2024-03-23T18:00:40.438 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.438+0000 7f8ed6c23b80 1 --2- 172.21.15.86:0/4211483366 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5633ee642f90 0x5633ee6453b0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:40.439 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.438+0000 7f8ed6c23b80 1 --2- 172.21.15.86:0/4211483366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633ee5e08d0 0x5633ee63b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:40.439 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.438+0000 7f8ed6c23b80 1 --2- 172.21.15.86:0/4211483366 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5633ee5d5100 0x5633ee5e5730 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:40.439 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.438+0000 7f8ed6c23b80 1 --2- 172.21.15.86:0/4211483366 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5633ee5d5a20 0x5633ee5e5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:40.439 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.438+0000 7f8ed6c23b80 1 -- 172.21.15.86:0/4211483366 >> 172.21.15.86:0/4211483366 conn(0x5633ee535640 msgr2=0x5633ee5ca3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:00:40.439 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.438+0000 7f8ea6000700 0 client.26821 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T18:00:40.439 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.438+0000 7f8ed6c23b80 1 -- 172.21.15.86:0/4211483366 shutdown_connections 2024-03-23T18:00:40.439 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.438+0000 7f8ed6c23b80 1 -- 172.21.15.86:0/4211483366 wait complete. 2024-03-23T18:00:40.439 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.439+0000 7f8ed6c23b80 20 client.26821 trim_cache size 0 max 16384 2024-03-23T18:00:40.440 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-03-23T18:00:40.440 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-03-23T18:00:40.440 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-03-23T18:00:40.440 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-03-23T18:00:40.441 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.441+0000 7f8ec3200700 1 -- 172.21.15.86:0/1541735565 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f8ea00a1310 msgr2=0x7f8ea00a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:00:40.441 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.441+0000 7f8ec3200700 1 --2- 172.21.15.86:0/1541735565 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f8ea00a1310 0x7f8ea00a37d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8015fd0 tx=0x7f8ea800b500 comp rx=0 tx=0).stop 2024-03-23T18:00:40.441 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.441+0000 7f8ec3200700 1 -- 172.21.15.86:0/1541735565 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ebc105c60 msgr2=0x7f8ebc19dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:00:40.441 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.441+0000 7f8ec3200700 1 --2- 172.21.15.86:0/1541735565 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ebc105c60 0x7f8ebc19dbf0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f8eb4009990 tx=0x7f8eb4009ca0 comp rx=0 tx=0).stop 2024-03-23T18:00:40.442 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.442+0000 7f8ec3200700 1 -- 172.21.15.86:0/1541735565 shutdown_connections 2024-03-23T18:00:40.442 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.442+0000 7f8ec3200700 1 --2- 172.21.15.86:0/1541735565 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ebc105c60 0x7f8ebc19dbf0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:40.442 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.442+0000 7f8ec3200700 1 --2- 172.21.15.86:0/1541735565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ebc0ffbb0 0x7f8ebc199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:40.442 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.442+0000 7f8ec3200700 1 --2- 172.21.15.86:0/1541735565 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f8ea00a1310 0x7f8ea00a37d0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:40.442 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.442+0000 7f8ec3200700 1 --2- 172.21.15.86:0/1541735565 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8ebc100570 0x7f8ebc1998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:40.442 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.442+0000 7f8ec3200700 1 -- 172.21.15.86:0/1541735565 >> 172.21.15.86:0/1541735565 conn(0x7f8ebc076d50 msgr2=0x7f8ebc0fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:00:40.442 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.442+0000 7f8ec3200700 1 -- 172.21.15.86:0/1541735565 shutdown_connections 2024-03-23T18:00:40.442 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:40.442+0000 7f8ec3200700 1 -- 172.21.15.86:0/1541735565 wait complete. 2024-03-23T18:00:40.920 INFO:tasks.workunit.client.0.smithi086.stdout: 1 577126 6.32 MB/sec execute 425 sec latency 44.425 ms 2024-03-23T18:00:41.336 INFO:journalctl@ceph.mds.c.smithi152.stdout:Mar 23 18:00:40 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-c[84552]: ignoring --setuser ceph since I am not root 2024-03-23T18:00:41.336 INFO:journalctl@ceph.mds.c.smithi152.stdout:Mar 23 18:00:40 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-c[84552]: ignoring --setgroup ceph since I am not root 2024-03-23T18:00:41.336 INFO:journalctl@ceph.mds.c.smithi152.stdout:Mar 23 18:00:40 smithi152 ceph-mds[84575]: -- 172.21.15.152:0/2888549317 <== mon.1 v2:172.21.15.152:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a11c4041a0 con 0x55a11c3da800 2024-03-23T18:00:41.336 INFO:journalctl@ceph.mds.c.smithi152.stdout:Mar 23 18:00:40 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-c[84552]: starting mds.c at 2024-03-23T18:00:41.919 INFO:tasks.workunit.client.0.smithi086.stdout: 1 579197 6.33 MB/sec execute 426 sec latency 19.725 ms 2024-03-23T18:00:42.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:00:41 smithi086 ceph-mon[38221]: pgmap v312: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 334 op/s 2024-03-23T18:00:42.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:00:41 smithi086 ceph-mon[38221]: daemon mds.c restarted 2024-03-23T18:00:42.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:00:41 smithi086 ceph-mon[38221]: osdmap e92: 12 total, 12 up, 12 in 2024-03-23T18:00:42.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:00:41 smithi092 ceph-mon[79089]: pgmap v312: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 334 op/s 2024-03-23T18:00:42.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:00:41 smithi092 ceph-mon[79089]: daemon mds.c restarted 2024-03-23T18:00:42.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:00:41 smithi092 ceph-mon[79089]: osdmap e92: 12 total, 12 up, 12 in 2024-03-23T18:00:42.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:00:41 smithi152 ceph-mon[47949]: pgmap v312: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 334 op/s 2024-03-23T18:00:42.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:00:41 smithi152 ceph-mon[47949]: daemon mds.c restarted 2024-03-23T18:00:42.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:00:41 smithi152 ceph-mon[47949]: osdmap e92: 12 total, 12 up, 12 in 2024-03-23T18:00:42.920 INFO:tasks.workunit.client.0.smithi086.stdout: 1 580470 6.33 MB/sec execute 427 sec latency 12.785 ms 2024-03-23T18:00:43.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:00:42 smithi086 ceph-mon[38221]: mds.? [v2:172.21.15.152:6836/2750420399,v1:172.21.15.152:6837/2750420399] up:boot 2024-03-23T18:00:43.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:00:42 smithi086 ceph-mon[38221]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-23T18:00:43.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:00:42 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-23T18:00:43.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:00:42 smithi086 ceph-mon[38221]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:00:43.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:00:42 smithi092 ceph-mon[79089]: mds.? [v2:172.21.15.152:6836/2750420399,v1:172.21.15.152:6837/2750420399] up:boot 2024-03-23T18:00:43.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:00:42 smithi092 ceph-mon[79089]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-23T18:00:43.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:00:42 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-23T18:00:43.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:00:42 smithi092 ceph-mon[79089]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:00:43.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:00:42 smithi152 ceph-mon[47949]: mds.? [v2:172.21.15.152:6836/2750420399,v1:172.21.15.152:6837/2750420399] up:boot 2024-03-23T18:00:43.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:00:42 smithi152 ceph-mon[47949]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-23T18:00:43.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:00:42 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-23T18:00:43.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:00:42 smithi152 ceph-mon[47949]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:00:43.920 INFO:tasks.workunit.client.0.smithi086.stdout: 1 582143 6.33 MB/sec execute 428 sec latency 213.606 ms 2024-03-23T18:00:44.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:00:43 smithi086 ceph-mon[38221]: pgmap v314: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 247 op/s 2024-03-23T18:00:44.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:00:43 smithi092 ceph-mon[79089]: pgmap v314: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 247 op/s 2024-03-23T18:00:44.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:00:43 smithi152 ceph-mon[47949]: pgmap v314: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 247 op/s 2024-03-23T18:00:44.920 INFO:tasks.workunit.client.0.smithi086.stdout: 1 583398 6.33 MB/sec execute 429 sec latency 19.899 ms 2024-03-23T18:00:45.920 INFO:tasks.workunit.client.0.smithi086.stdout: 1 585134 6.34 MB/sec execute 430 sec latency 16.714 ms 2024-03-23T18:00:46.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:00:45 smithi086 ceph-mon[38221]: pgmap v315: 97 pgs: 97 active+clean; 230 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 264 op/s 2024-03-23T18:00:46.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:00:45 smithi092 ceph-mon[79089]: pgmap v315: 97 pgs: 97 active+clean; 230 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 264 op/s 2024-03-23T18:00:46.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:00:45 smithi152 ceph-mon[47949]: pgmap v315: 97 pgs: 97 active+clean; 230 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 264 op/s 2024-03-23T18:00:46.920 INFO:tasks.workunit.client.0.smithi086.stdout: 1 586664 6.35 MB/sec execute 431 sec latency 13.797 ms 2024-03-23T18:00:47.920 INFO:tasks.workunit.client.0.smithi086.stdout: 1 588473 6.35 MB/sec execute 432 sec latency 43.565 ms 2024-03-23T18:00:48.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:00:47 smithi086 ceph-mon[38221]: pgmap v316: 97 pgs: 97 active+clean; 230 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 264 op/s 2024-03-23T18:00:48.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:00:47 smithi092 ceph-mon[79089]: pgmap v316: 97 pgs: 97 active+clean; 230 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 264 op/s 2024-03-23T18:00:48.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:00:47 smithi152 ceph-mon[47949]: pgmap v316: 97 pgs: 97 active+clean; 230 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 264 op/s 2024-03-23T18:00:48.920 INFO:tasks.workunit.client.0.smithi086.stdout: 1 590013 6.36 MB/sec execute 433 sec latency 12.376 ms 2024-03-23T18:00:49.920 INFO:tasks.workunit.client.0.smithi086.stdout: 1 591396 6.36 MB/sec execute 434 sec latency 62.080 ms 2024-03-23T18:00:50.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:00:49 smithi086 ceph-mon[38221]: pgmap v317: 97 pgs: 97 active+clean; 230 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 264 op/s 2024-03-23T18:00:50.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:00:49 smithi092 ceph-mon[79089]: pgmap v317: 97 pgs: 97 active+clean; 230 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 264 op/s 2024-03-23T18:00:50.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:00:49 smithi152 ceph-mon[47949]: pgmap v317: 97 pgs: 97 active+clean; 230 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 264 op/s 2024-03-23T18:00:50.920 INFO:tasks.workunit.client.0.smithi086.stdout: 1 593366 6.38 MB/sec execute 435 sec latency 19.263 ms 2024-03-23T18:00:51.920 INFO:tasks.workunit.client.0.smithi086.stdout: 1 594574 6.37 MB/sec execute 436 sec latency 13.642 ms 2024-03-23T18:00:52.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:00:51 smithi086 ceph-mon[38221]: pgmap v318: 97 pgs: 97 active+clean; 237 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 276 op/s 2024-03-23T18:00:52.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:00:51 smithi092 ceph-mon[79089]: pgmap v318: 97 pgs: 97 active+clean; 237 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 276 op/s 2024-03-23T18:00:52.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:00:51 smithi152 ceph-mon[47949]: pgmap v318: 97 pgs: 97 active+clean; 237 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 276 op/s 2024-03-23T18:00:52.920 INFO:tasks.workunit.client.0.smithi086.stdout: 1 596540 6.38 MB/sec execute 437 sec latency 18.257 ms 2024-03-23T18:00:53.596 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.596+0000 7f56e1c00700 1 -- 172.21.15.86:0/649485747 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56dc10d4e0 msgr2=0x7f56dc10f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:00:53.596 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.596+0000 7f56e1c00700 1 --2- 172.21.15.86:0/649485747 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56dc10d4e0 0x7f56dc10f950 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f56d4009f90 tx=0x7f56d400b330 comp rx=0 tx=0).stop 2024-03-23T18:00:53.596 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.596+0000 7f56e1c00700 1 -- 172.21.15.86:0/649485747 shutdown_connections 2024-03-23T18:00:53.596 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.596+0000 7f56e1c00700 1 --2- 172.21.15.86:0/649485747 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56dc10d4e0 0x7f56dc10f950 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:53.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.596+0000 7f56e1c00700 1 --2- 172.21.15.86:0/649485747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56dc0731e0 0x7f56dc0735c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:53.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.596+0000 7f56e1c00700 1 --2- 172.21.15.86:0/649485747 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f56dc073b00 0x7f56dc10cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:53.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.596+0000 7f56e1c00700 1 -- 172.21.15.86:0/649485747 >> 172.21.15.86:0/649485747 conn(0x7f56dc0fc7b0 msgr2=0x7f56dc0febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:00:53.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.596+0000 7f56e1c00700 1 -- 172.21.15.86:0/649485747 shutdown_connections 2024-03-23T18:00:53.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.596+0000 7f56e1c00700 1 -- 172.21.15.86:0/649485747 wait complete. 2024-03-23T18:00:53.599 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.597+0000 7f56e1c00700 1 Processor -- start 2024-03-23T18:00:53.599 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.597+0000 7f56e1c00700 1 -- start start 2024-03-23T18:00:53.599 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.598+0000 7f56e1c00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f56dc0731e0 0x7f56dc199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:53.599 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.598+0000 7f56e1c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56dc073b00 0x7f56dc199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:53.599 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.598+0000 7f56e1c00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56dc10d4e0 0x7f56dc19dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:53.599 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.598+0000 7f56e1c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56dc19e360 con 0x7f56dc073b00 2024-03-23T18:00:53.599 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.598+0000 7f56e1c00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56dc19e4d0 con 0x7f56dc10d4e0 2024-03-23T18:00:53.599 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.598+0000 7f56e1c00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56dc19e690 con 0x7f56dc0731e0 2024-03-23T18:00:53.599 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.598+0000 7f56daa00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f56dc0731e0 0x7f56dc199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.598+0000 7f56daa00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f56dc0731e0 0x7f56dc199440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:42126/0 (socket says 172.21.15.86:42126) 2024-03-23T18:00:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.598+0000 7f56daa00700 1 -- 172.21.15.86:0/789010231 learned_addr learned my addr 172.21.15.86:0/789010231 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:00:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.598+0000 7f56da000700 1 --2- 172.21.15.86:0/789010231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56dc073b00 0x7f56dc199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.598+0000 7f56db400700 1 --2- 172.21.15.86:0/789010231 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56dc10d4e0 0x7f56dc19dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.598+0000 7f56daa00700 1 -- 172.21.15.86:0/789010231 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56dc10d4e0 msgr2=0x7f56dc19dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:00:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.598+0000 7f56daa00700 1 --2- 172.21.15.86:0/789010231 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56dc10d4e0 0x7f56dc19dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.598+0000 7f56daa00700 1 -- 172.21.15.86:0/789010231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56dc073b00 msgr2=0x7f56dc199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:00:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.598+0000 7f56daa00700 1 --2- 172.21.15.86:0/789010231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56dc073b00 0x7f56dc199980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.598+0000 7f56daa00700 1 -- 172.21.15.86:0/789010231 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56cc009520 con 0x7f56dc0731e0 2024-03-23T18:00:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.599+0000 7f56daa00700 1 --2- 172.21.15.86:0/789010231 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f56dc0731e0 0x7f56dc199440 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f56cc00eb30 tx=0x7f56cc00ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:00:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.599+0000 7f56d3400700 1 -- 172.21.15.86:0/789010231 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56cc009880 con 0x7f56dc0731e0 2024-03-23T18:00:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.599+0000 7f56d3400700 1 -- 172.21.15.86:0/789010231 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f56cc0044e0 con 0x7f56dc0731e0 2024-03-23T18:00:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.599+0000 7f56e1c00700 1 -- 172.21.15.86:0/789010231 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56c80095f0 con 0x7f56dc0731e0 2024-03-23T18:00:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.599+0000 7f56d3400700 1 -- 172.21.15.86:0/789010231 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56cc003d80 con 0x7f56dc0731e0 2024-03-23T18:00:53.601 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.599+0000 7f56e1c00700 1 -- 172.21.15.86:0/789010231 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56d4009c40 con 0x7f56dc0731e0 2024-03-23T18:00:53.602 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.602+0000 7f56d3400700 1 -- 172.21.15.86:0/789010231 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f56cc003f20 con 0x7f56dc0731e0 2024-03-23T18:00:53.604 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.603+0000 7f56d3400700 1 --2- 172.21.15.86:0/789010231 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f56c00a1290 0x7f56c00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:53.604 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.604+0000 7f56f573cb80 1 Processor -- start 2024-03-23T18:00:53.604 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.604+0000 7f56d3400700 1 -- 172.21.15.86:0/789010231 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f56cc014070 con 0x7f56dc0731e0 2024-03-23T18:00:53.604 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.604+0000 7f56f573cb80 1 -- start start 2024-03-23T18:00:53.604 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.604+0000 7f56da000700 1 --2- 172.21.15.86:0/789010231 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f56c00a1290 0x7f56c00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:53.606 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.604+0000 7f56f573cb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56395ba98100 0x56395ba984e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:53.606 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.604+0000 7f56f573cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56395ba98a20 0x56395baa3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:53.606 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.604+0000 7f56f573cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56395baa38d0 0x56395baa5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:53.606 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.604+0000 7f56f573cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56395b9f1930 con 0x56395baa38d0 2024-03-23T18:00:53.606 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.604+0000 7f56f573cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x56395b9ef2f0 con 0x56395ba98a20 2024-03-23T18:00:53.606 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.604+0000 7f56f573cb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x56395b9eabe0 con 0x56395ba98100 2024-03-23T18:00:53.606 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.605+0000 7f56daa00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56395ba98100 0x56395ba984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:53.606 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.605+0000 7f56daa00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56395ba98100 0x56395ba984e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:42138/0 (socket says 172.21.15.86:42138) 2024-03-23T18:00:53.606 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.605+0000 7f56daa00700 1 -- 172.21.15.86:0/1255424932 learned_addr learned my addr 172.21.15.86:0/1255424932 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:00:53.606 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.605+0000 7f56db400700 1 --2- 172.21.15.86:0/1255424932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56395baa38d0 0x56395baa5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:53.606 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.605+0000 7f56da000700 1 --2- 172.21.15.86:0/1255424932 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56395ba98a20 0x56395baa3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:53.607 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.605+0000 7f56da000700 1 --2- 172.21.15.86:0/789010231 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f56c00a1290 0x7f56c00a3750 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f56dc19a990 tx=0x7f56c8009310 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:00:53.607 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.605+0000 7f56daa00700 1 -- 172.21.15.86:0/1255424932 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56395ba98a20 msgr2=0x56395baa3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:00:53.607 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.605+0000 7f56daa00700 1 --2- 172.21.15.86:0/1255424932 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56395ba98a20 0x56395baa3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:53.607 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.605+0000 7f56daa00700 1 -- 172.21.15.86:0/1255424932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56395baa38d0 msgr2=0x56395baa5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:00:53.607 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.605+0000 7f56daa00700 1 --2- 172.21.15.86:0/1255424932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56395baa38d0 0x56395baa5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:53.607 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.605+0000 7f56daa00700 1 -- 172.21.15.86:0/1255424932 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56395b86c350 con 0x56395ba98100 2024-03-23T18:00:53.607 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.605+0000 7f56da000700 1 --2- 172.21.15.86:0/1255424932 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56395ba98a20 0x56395baa3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T18:00:53.607 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.605+0000 7f56db400700 1 --2- 172.21.15.86:0/1255424932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56395baa38d0 0x56395baa5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T18:00:53.607 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.605+0000 7f56daa00700 1 --2- 172.21.15.86:0/1255424932 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56395ba98100 0x56395ba984e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f56cc0a2280 tx=0x7f56cc099c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ece76451e33cbad8 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:00:53.607 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.605+0000 7f56c7e00700 1 -- 172.21.15.86:0/1255424932 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56cc09a330 con 0x56395ba98100 2024-03-23T18:00:53.607 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.605+0000 7f56c7e00700 1 -- 172.21.15.86:0/1255424932 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f56cc09e160 con 0x56395ba98100 2024-03-23T18:00:53.607 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.605+0000 7f56c7e00700 1 -- 172.21.15.86:0/1255424932 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56cc09a590 con 0x56395ba98100 2024-03-23T18:00:53.611 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.606+0000 7f56f573cb80 1 -- 172.21.15.86:0/1255424932 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56395ba98100 msgr2=0x56395ba984e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:00:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.606+0000 7f56f573cb80 1 --2- 172.21.15.86:0/1255424932 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56395ba98100 0x56395ba984e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f56cc0a2280 tx=0x7f56cc099c90 comp rx=0 tx=0).stop 2024-03-23T18:00:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.606+0000 7f56f573cb80 1 -- 172.21.15.86:0/1255424932 shutdown_connections 2024-03-23T18:00:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.606+0000 7f56f573cb80 1 --2- 172.21.15.86:0/1255424932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56395baa38d0 0x56395baa5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.606+0000 7f56f573cb80 1 --2- 172.21.15.86:0/1255424932 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56395ba98100 0x56395ba984e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.606+0000 7f56f573cb80 1 --2- 172.21.15.86:0/1255424932 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56395ba98a20 0x56395baa3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.606+0000 7f56f573cb80 1 -- 172.21.15.86:0/1255424932 >> 172.21.15.86:0/1255424932 conn(0x56395b9f8640 msgr2=0x56395ba94520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:00:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.606+0000 7f56f573cb80 1 -- 172.21.15.86:0/1255424932 shutdown_connections 2024-03-23T18:00:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.606+0000 7f56f573cb80 1 -- 172.21.15.86:0/1255424932 wait complete. 2024-03-23T18:00:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.607+0000 7f56f573cb80 1 -- start start 2024-03-23T18:00:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.607+0000 7f56f573cb80 1 Processor -- start 2024-03-23T18:00:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.608+0000 7f56f573cb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56395ba98100 0x56395baa3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.608+0000 7f56f573cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56395ba98a20 0x56395baa0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:53.613 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.608+0000 7f56f573cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56395baa38d0 0x56395baa0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:53.613 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.608+0000 7f56f573cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56395b986d20 con 0x56395ba98a20 2024-03-23T18:00:53.613 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.608+0000 7f56f573cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x56395b9869c0 con 0x56395baa38d0 2024-03-23T18:00:53.613 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.608+0000 7f56f573cb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x56395b86e640 con 0x56395ba98100 2024-03-23T18:00:53.613 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.608+0000 7f56f573cb80 10 client.0 _ll_register_callbacks cb 0x56395b990410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T18:00:53.613 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.608+0000 7f56daa00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56395ba98100 0x56395baa3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:53.613 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.608+0000 7f56db400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56395baa38d0 0x56395baa0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:53.613 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.608+0000 7f56daa00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56395ba98100 0x56395baa3310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:42144/0 (socket says 172.21.15.86:42144) 2024-03-23T18:00:53.613 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.608+0000 7f56daa00700 1 -- 172.21.15.86:0/3654814768 learned_addr learned my addr 172.21.15.86:0/3654814768 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:00:53.613 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.608+0000 7f56da000700 1 --2- 172.21.15.86:0/3654814768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56395ba98a20 0x56395baa0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:53.613 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.608+0000 7f56daa00700 1 -- 172.21.15.86:0/3654814768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56395baa38d0 msgr2=0x56395baa0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:00:53.613 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.608+0000 7f56daa00700 1 --2- 172.21.15.86:0/3654814768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56395baa38d0 0x56395baa0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:53.613 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.608+0000 7f56daa00700 1 -- 172.21.15.86:0/3654814768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56395ba98a20 msgr2=0x56395baa0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:00:53.614 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.608+0000 7f56daa00700 1 --2- 172.21.15.86:0/3654814768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56395ba98a20 0x56395baa0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:53.614 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.608+0000 7f56daa00700 1 -- 172.21.15.86:0/3654814768 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56d4009c40 con 0x56395ba98100 2024-03-23T18:00:53.614 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.608+0000 7f56db400700 1 --2- 172.21.15.86:0/3654814768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56395baa38d0 0x56395baa0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T18:00:53.614 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.609+0000 7f56daa00700 1 --2- 172.21.15.86:0/3654814768 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56395ba98100 0x56395baa3310 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f56cc09a2c0 tx=0x7f56cc00ce40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c025350e14ead5a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:00:53.614 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.609+0000 7f56c4c00700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-03-23T18:00:53.614 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.609+0000 7f56c4c00700 1 -- 172.21.15.86:0/3654814768 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56cc09a330 con 0x56395ba98100 2024-03-23T18:00:53.614 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.609+0000 7f56c4c00700 1 -- 172.21.15.86:0/3654814768 <== mon.1 v2:172.21.15.152:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f56cc096a40 con 0x56395ba98100 2024-03-23T18:00:53.614 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.609+0000 7f56f573cb80 1 -- 172.21.15.86:0/3654814768 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56395b9ab320 con 0x56395ba98100 2024-03-23T18:00:53.614 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.609+0000 7f56c4c00700 1 -- 172.21.15.86:0/3654814768 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f56cc096c00 con 0x56395ba98100 2024-03-23T18:00:53.614 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.609+0000 7f56f573cb80 10 client.26839 fetch_fsmap learned FSMap version 47 2024-03-23T18:00:53.614 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.609+0000 7f56f573cb80 1 -- 172.21.15.86:0/3654814768 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x56395b818b00 con 0x56395ba98100 2024-03-23T18:00:53.614 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.609+0000 7f56c4c00700 1 -- 172.21.15.86:0/3654814768 <== mon.1 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56395b9ab320 con 0x56395ba98100 2024-03-23T18:00:53.614 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.610+0000 7f56c4c00700 1 -- 172.21.15.86:0/3654814768 <== mon.1 v2:172.21.15.152:3300/0 5 ==== fsmap(e 47) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f56cc0a2390 con 0x56395ba98100 2024-03-23T18:00:53.614 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.610+0000 7f56f573cb80 10 client.26839 fetch_fsmap finished waiting for FSMap version 47 2024-03-23T18:00:53.615 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.610+0000 7f56f573cb80 10 client.26839 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T18:00:53.615 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.610+0000 7f56f573cb80 20 client.26839 populate_metadata read hostname 'smithi086' 2024-03-23T18:00:53.615 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.610+0000 7f56f573cb80 1 --2- 172.21.15.86:0/3654814768 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x56395bab3260 0x56395bab5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:53.615 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.610+0000 7f56f573cb80 4 client.26839 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T18:00:53.615 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.610+0000 7f56f573cb80 1 -- 172.21.15.86:0/3654814768 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56395b9e36a0 con 0x56395bab3260 2024-03-23T18:00:53.615 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.610+0000 7f56da000700 1 --2- 172.21.15.86:0/3654814768 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x56395bab3260 0x56395bab5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:53.615 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.610+0000 7f56c4c00700 10 client.26839 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T18:00:53.615 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.610+0000 7f56da000700 1 --2- 172.21.15.86:0/3654814768 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x56395bab3260 0x56395bab5680 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b73926ed35739249 server_cookie=3a98aacd0f8a0c5 in_seq=0 out_seq=0 2024-03-23T18:00:53.615 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.611+0000 7f56c4c00700 1 -- 172.21.15.86:0/3654814768 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x56395b9e36a0 con 0x56395bab3260 2024-03-23T18:00:53.615 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.611+0000 7f56c4c00700 10 client.26839 handle_command_reply: tid=0 2024-03-23T18:00:53.615 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.611+0000 7f56f573cb80 1 client.26839 shutdown 2024-03-23T18:00:53.615 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.611+0000 7f56f573cb80 1 -- 172.21.15.86:0/3654814768 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56395ba98100 msgr2=0x56395baa3310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:00:53.615 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.611+0000 7f56f573cb80 1 --2- 172.21.15.86:0/3654814768 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56395ba98100 0x56395baa3310 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f56cc09a2c0 tx=0x7f56cc00ce40 comp rx=0 tx=0).stop 2024-03-23T18:00:53.615 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.611+0000 7f56f573cb80 1 -- 172.21.15.86:0/3654814768 shutdown_connections 2024-03-23T18:00:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.611+0000 7f56f573cb80 1 --2- 172.21.15.86:0/3654814768 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x56395bab3260 0x56395bab5680 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.611+0000 7f56f573cb80 1 --2- 172.21.15.86:0/3654814768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56395baa38d0 0x56395baa0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.611+0000 7f56f573cb80 1 --2- 172.21.15.86:0/3654814768 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56395ba98100 0x56395baa3310 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.611+0000 7f56f573cb80 1 --2- 172.21.15.86:0/3654814768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56395ba98a20 0x56395baa0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.611+0000 7f56f573cb80 1 -- 172.21.15.86:0/3654814768 >> 172.21.15.86:0/3654814768 conn(0x56395b9f8640 msgr2=0x56395ba9a0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:00:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.611+0000 7f56c4c00700 0 client.26839 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T18:00:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.612+0000 7f56f573cb80 1 -- 172.21.15.86:0/3654814768 shutdown_connections 2024-03-23T18:00:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.612+0000 7f56f573cb80 1 -- 172.21.15.86:0/3654814768 wait complete. 2024-03-23T18:00:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.612+0000 7f56f573cb80 20 client.26839 trim_cache size 0 max 16384 2024-03-23T18:00:53.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.628+0000 7f56f573cb80 1 Processor -- start 2024-03-23T18:00:53.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.628+0000 7f56f573cb80 1 -- start start 2024-03-23T18:00:53.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.629+0000 7f56f573cb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56395ba98100 0x56395baaa5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:53.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.629+0000 7f56f573cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56395ba98a20 0x56395baa05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:53.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.629+0000 7f56f573cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56395baa38d0 0x56395ba8e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:53.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.629+0000 7f56f573cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56395ba993b0 con 0x56395ba98a20 2024-03-23T18:00:53.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.629+0000 7f56f573cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x56395baa1bd0 con 0x56395baa38d0 2024-03-23T18:00:53.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.629+0000 7f56f573cb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x56395ba950f0 con 0x56395ba98100 2024-03-23T18:00:53.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.629+0000 7f56daa00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56395ba98100 0x56395baaa5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:53.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.629+0000 7f56db400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56395baa38d0 0x56395ba8e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:53.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.629+0000 7f56daa00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56395ba98100 0x56395baaa5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:42158/0 (socket says 172.21.15.86:42158) 2024-03-23T18:00:53.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.629+0000 7f56daa00700 1 -- 172.21.15.86:0/2776616982 learned_addr learned my addr 172.21.15.86:0/2776616982 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:00:53.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.629+0000 7f56db400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56395baa38d0 0x56395ba8e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:55782/0 (socket says 172.21.15.86:55782) 2024-03-23T18:00:53.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.629+0000 7f56da000700 1 --2- 172.21.15.86:0/2776616982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56395ba98a20 0x56395baa05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:53.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.629+0000 7f56daa00700 1 -- 172.21.15.86:0/2776616982 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56395baa38d0 msgr2=0x56395ba8e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:00:53.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.629+0000 7f56daa00700 1 --2- 172.21.15.86:0/2776616982 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56395baa38d0 0x56395ba8e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:53.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.629+0000 7f56daa00700 1 -- 172.21.15.86:0/2776616982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56395ba98a20 msgr2=0x56395baa05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:00:53.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.629+0000 7f56daa00700 1 --2- 172.21.15.86:0/2776616982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56395ba98a20 0x56395baa05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:53.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.629+0000 7f56daa00700 1 -- 172.21.15.86:0/2776616982 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56395b9b0e40 con 0x56395ba98100 2024-03-23T18:00:53.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.629+0000 7f56daa00700 1 --2- 172.21.15.86:0/2776616982 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56395ba98100 0x56395baaa5e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f56cc0ca9d0 tx=0x7f56cc0caa20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12382513ff34481f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:00:53.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.630+0000 7f56d0c00700 1 -- 172.21.15.86:0/2776616982 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56cc00cbd0 con 0x56395ba98100 2024-03-23T18:00:53.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.630+0000 7f56d0c00700 1 -- 172.21.15.86:0/2776616982 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f56cc096460 con 0x56395ba98100 2024-03-23T18:00:53.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.630+0000 7f56d0c00700 1 -- 172.21.15.86:0/2776616982 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56cc01f340 con 0x56395ba98100 2024-03-23T18:00:53.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.630+0000 7f56f573cb80 1 -- 172.21.15.86:0/2776616982 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56395ba98100 msgr2=0x56395baaa5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:00:53.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.630+0000 7f56f573cb80 1 --2- 172.21.15.86:0/2776616982 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56395ba98100 0x56395baaa5e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f56cc0ca9d0 tx=0x7f56cc0caa20 comp rx=0 tx=0).stop 2024-03-23T18:00:53.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.630+0000 7f56f573cb80 1 -- 172.21.15.86:0/2776616982 shutdown_connections 2024-03-23T18:00:53.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.630+0000 7f56f573cb80 1 --2- 172.21.15.86:0/2776616982 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56395baa38d0 0x56395ba8e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:53.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.630+0000 7f56f573cb80 1 --2- 172.21.15.86:0/2776616982 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56395ba98100 0x56395baaa5e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:53.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.630+0000 7f56f573cb80 1 --2- 172.21.15.86:0/2776616982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56395ba98a20 0x56395baa05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:53.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.630+0000 7f56f573cb80 1 -- 172.21.15.86:0/2776616982 >> 172.21.15.86:0/2776616982 conn(0x56395b9f8640 msgr2=0x56395ba9fe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:00:53.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.631+0000 7f56f573cb80 1 -- 172.21.15.86:0/2776616982 shutdown_connections 2024-03-23T18:00:53.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.631+0000 7f56f573cb80 1 -- 172.21.15.86:0/2776616982 wait complete. 2024-03-23T18:00:53.634 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.631+0000 7f56f573cb80 1 -- start start 2024-03-23T18:00:53.634 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.631+0000 7f56f573cb80 1 Processor -- start 2024-03-23T18:00:53.634 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.632+0000 7f56f573cb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56395ba98100 0x56395baa85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:53.634 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.632+0000 7f56f573cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56395ba98a20 0x56395baa8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:53.634 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.632+0000 7f56f573cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56395baa38d0 0x56395baa9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:53.634 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.632+0000 7f56f573cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56395bb01d70 con 0x56395baa38d0 2024-03-23T18:00:53.634 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.632+0000 7f56f573cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x56395bb01ee0 con 0x56395ba98a20 2024-03-23T18:00:53.634 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.632+0000 7f56f573cb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x56395bb020a0 con 0x56395ba98100 2024-03-23T18:00:53.634 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.632+0000 7f56f573cb80 10 client.0 _ll_register_callbacks cb 0x56395b990410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T18:00:53.634 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.632+0000 7f56da000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56395ba98a20 0x56395baa8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:53.634 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.632+0000 7f56db400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56395baa38d0 0x56395baa9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:53.634 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.632+0000 7f56da000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56395ba98a20 0x56395baa8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:55792/0 (socket says 172.21.15.86:55792) 2024-03-23T18:00:53.634 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.632+0000 7f56da000700 1 -- 172.21.15.86:0/63339623 learned_addr learned my addr 172.21.15.86:0/63339623 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:00:53.635 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.632+0000 7f56daa00700 1 --2- 172.21.15.86:0/63339623 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56395ba98100 0x56395baa85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:53.635 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.633+0000 7f56da000700 1 -- 172.21.15.86:0/63339623 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56395ba98100 msgr2=0x56395baa85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:00:53.635 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.633+0000 7f56da000700 1 --2- 172.21.15.86:0/63339623 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56395ba98100 0x56395baa85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:53.635 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.633+0000 7f56da000700 1 -- 172.21.15.86:0/63339623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56395baa38d0 msgr2=0x56395baa9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:00:53.635 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.633+0000 7f56da000700 1 --2- 172.21.15.86:0/63339623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56395baa38d0 0x56395baa9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:53.635 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.633+0000 7f56da000700 1 -- 172.21.15.86:0/63339623 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56395b818b00 con 0x56395ba98a20 2024-03-23T18:00:53.635 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.633+0000 7f56da000700 1 --2- 172.21.15.86:0/63339623 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56395ba98a20 0x56395baa8b00 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f56c8009fd0 tx=0x7f56c8013040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b976405d1f45677 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:00:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.634+0000 7f56c4c00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T18:00:53.637 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.634+0000 7f56c4c00700 1 -- 172.21.15.86:0/63339623 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c8023040 con 0x56395ba98a20 2024-03-23T18:00:53.637 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.634+0000 7f56c4c00700 1 -- 172.21.15.86:0/63339623 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f56c80283d0 con 0x56395ba98a20 2024-03-23T18:00:53.637 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.634+0000 7f56f573cb80 1 -- 172.21.15.86:0/63339623 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f56cc09e160 con 0x56395ba98a20 2024-03-23T18:00:53.637 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.634+0000 7f56c4c00700 1 -- 172.21.15.86:0/63339623 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f56c8028780 con 0x56395ba98a20 2024-03-23T18:00:53.637 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.634+0000 7f56f573cb80 10 client.26864 fetch_fsmap learned FSMap version 47 2024-03-23T18:00:53.637 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.634+0000 7f56f573cb80 1 -- 172.21.15.86:0/63339623 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x56395b870c70 con 0x56395ba98a20 2024-03-23T18:00:53.637 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.635+0000 7f56c4c00700 1 -- 172.21.15.86:0/63339623 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f56cc09e160 con 0x56395ba98a20 2024-03-23T18:00:53.637 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.635+0000 7f56c4c00700 1 -- 172.21.15.86:0/63339623 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 47) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f56c80319d0 con 0x56395ba98a20 2024-03-23T18:00:53.637 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.635+0000 7f56f573cb80 10 client.26864 fetch_fsmap finished waiting for FSMap version 47 2024-03-23T18:00:53.637 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.635+0000 7f56f573cb80 10 client.26864 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T18:00:53.637 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.635+0000 7f56f573cb80 20 client.26864 populate_metadata read hostname 'smithi086' 2024-03-23T18:00:53.637 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.635+0000 7f56f573cb80 1 --2- 172.21.15.86:0/63339623 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x56395bb169a0 0x56395bb18dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:00:53.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.635+0000 7f56f573cb80 4 client.26864 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T18:00:53.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.635+0000 7f56f573cb80 1 -- 172.21.15.86:0/63339623 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56395b839c90 con 0x56395bb169a0 2024-03-23T18:00:53.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.635+0000 7f56daa00700 1 --2- 172.21.15.86:0/63339623 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x56395bb169a0 0x56395bb18dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:00:53.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.636+0000 7f56c4c00700 10 client.26864 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T18:00:53.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.636+0000 7f56daa00700 1 --2- 172.21.15.86:0/63339623 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x56395bb169a0 0x56395bb18dc0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3db9bfe5e4e4ed19 server_cookie=485231fa7756f2b1 in_seq=0 out_seq=0 2024-03-23T18:00:53.687 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.687+0000 7f56c4c00700 1 -- 172.21.15.86:0/63339623 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56395b839c90 con 0x56395bb169a0 2024-03-23T18:00:53.688 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.687+0000 7f56c4c00700 10 client.26864 handle_command_reply: tid=0 2024-03-23T18:00:53.688 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.687+0000 7f56f573cb80 1 client.26864 shutdown 2024-03-23T18:00:53.688 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.687+0000 7f56f573cb80 1 -- 172.21.15.86:0/63339623 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56395ba98a20 msgr2=0x56395baa8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:00:53.688 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.687+0000 7f56f573cb80 1 --2- 172.21.15.86:0/63339623 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56395ba98a20 0x56395baa8b00 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f56c8009fd0 tx=0x7f56c8013040 comp rx=0 tx=0).stop 2024-03-23T18:00:53.688 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.687+0000 7f56f573cb80 1 -- 172.21.15.86:0/63339623 shutdown_connections 2024-03-23T18:00:53.688 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.687+0000 7f56f573cb80 1 --2- 172.21.15.86:0/63339623 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x56395bb169a0 0x56395bb18dc0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:53.688 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.687+0000 7f56f573cb80 1 --2- 172.21.15.86:0/63339623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56395baa38d0 0x56395baa9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:53.688 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.687+0000 7f56f573cb80 1 --2- 172.21.15.86:0/63339623 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56395ba98100 0x56395baa85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:53.689 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.687+0000 7f56f573cb80 1 --2- 172.21.15.86:0/63339623 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56395ba98a20 0x56395baa8b00 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:53.689 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.687+0000 7f56f573cb80 1 -- 172.21.15.86:0/63339623 >> 172.21.15.86:0/63339623 conn(0x56395b9f8640 msgr2=0x56395ba93d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:00:53.689 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.687+0000 7f56c4c00700 0 client.26864 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T18:00:53.689 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.687+0000 7f56f573cb80 1 -- 172.21.15.86:0/63339623 shutdown_connections 2024-03-23T18:00:53.689 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.687+0000 7f56f573cb80 1 -- 172.21.15.86:0/63339623 wait complete. 2024-03-23T18:00:53.689 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.688+0000 7f56f573cb80 20 client.26864 trim_cache size 0 max 16384 2024-03-23T18:00:53.689 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-03-23T18:00:53.690 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-03-23T18:00:53.690 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-03-23T18:00:53.690 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-03-23T18:00:53.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.688+0000 7f56e1c00700 1 -- 172.21.15.86:0/789010231 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f56c00a1290 msgr2=0x7f56c00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:00:53.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.688+0000 7f56e1c00700 1 --2- 172.21.15.86:0/789010231 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f56c00a1290 0x7f56c00a3750 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f56dc19a990 tx=0x7f56c8009310 comp rx=0 tx=0).stop 2024-03-23T18:00:53.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.688+0000 7f56e1c00700 1 -- 172.21.15.86:0/789010231 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f56dc0731e0 msgr2=0x7f56dc199440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:00:53.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.688+0000 7f56e1c00700 1 --2- 172.21.15.86:0/789010231 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f56dc0731e0 0x7f56dc199440 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f56cc00eb30 tx=0x7f56cc00ee40 comp rx=0 tx=0).stop 2024-03-23T18:00:53.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.688+0000 7f56e1c00700 1 -- 172.21.15.86:0/789010231 shutdown_connections 2024-03-23T18:00:53.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.688+0000 7f56e1c00700 1 --2- 172.21.15.86:0/789010231 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56dc10d4e0 0x7f56dc19dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:53.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.688+0000 7f56e1c00700 1 --2- 172.21.15.86:0/789010231 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f56c00a1290 0x7f56c00a3750 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:53.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.688+0000 7f56e1c00700 1 --2- 172.21.15.86:0/789010231 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f56dc0731e0 0x7f56dc199440 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:53.692 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.688+0000 7f56e1c00700 1 --2- 172.21.15.86:0/789010231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56dc073b00 0x7f56dc199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:00:53.692 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.688+0000 7f56e1c00700 1 -- 172.21.15.86:0/789010231 >> 172.21.15.86:0/789010231 conn(0x7f56dc0fc7b0 msgr2=0x7f56dc0fd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:00:53.692 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.689+0000 7f56e1c00700 1 -- 172.21.15.86:0/789010231 shutdown_connections 2024-03-23T18:00:53.692 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:00:53.689+0000 7f56e1c00700 1 -- 172.21.15.86:0/789010231 wait complete. 2024-03-23T18:00:53.920 INFO:tasks.workunit.client.0.smithi086.stdout: 1 597598 6.38 MB/sec execute 438 sec latency 22.097 ms 2024-03-23T18:00:54.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:00:53 smithi086 ceph-mon[38221]: pgmap v319: 97 pgs: 97 active+clean; 237 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 237 op/s 2024-03-23T18:00:54.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:00:53 smithi092 ceph-mon[79089]: pgmap v319: 97 pgs: 97 active+clean; 237 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 237 op/s 2024-03-23T18:00:54.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:00:53 smithi152 ceph-mon[47949]: pgmap v319: 97 pgs: 97 active+clean; 237 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 237 op/s 2024-03-23T18:00:54.921 INFO:tasks.workunit.client.0.smithi086.stdout: 1 599282 6.38 MB/sec execute 439 sec latency 29.976 ms 2024-03-23T18:00:55.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:00:54 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T18:00:55.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:00:54 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T18:00:55.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:00:54 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T18:00:55.920 INFO:tasks.workunit.client.0.smithi086.stdout: 1 600625 6.39 MB/sec execute 440 sec latency 23.906 ms 2024-03-23T18:00:56.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:00:55 smithi086 ceph-mon[38221]: pgmap v320: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.9 MiB/s wr, 352 op/s 2024-03-23T18:00:56.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:00:55 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T18:00:56.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:00:55 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T18:00:56.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:00:55 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T18:00:56.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:00:55 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T18:00:56.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:00:55 smithi092 ceph-mon[79089]: pgmap v320: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.9 MiB/s wr, 352 op/s 2024-03-23T18:00:56.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:00:55 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T18:00:56.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:00:55 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T18:00:56.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:00:55 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T18:00:56.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:00:55 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T18:00:56.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:00:55 smithi152 ceph-mon[47949]: pgmap v320: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.9 MiB/s wr, 352 op/s 2024-03-23T18:00:56.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:00:55 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T18:00:56.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:00:55 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T18:00:56.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:00:55 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T18:00:56.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:00:55 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T18:00:56.920 INFO:tasks.workunit.client.0.smithi086.stdout: 1 601904 6.39 MB/sec execute 441 sec latency 17.765 ms 2024-03-23T18:00:57.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:00:56 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-23T18:00:57.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:00:56 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-23T18:00:57.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:00:56 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-23T18:00:57.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:00:56 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-23T18:00:57.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:00:56 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-23T18:00:57.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:00:56 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-23T18:00:57.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:00:56 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-23T18:00:57.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:00:56 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-23T18:00:57.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:00:56 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T18:00:57.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:00:56 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T18:00:57.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:00:56 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-23T18:00:57.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:00:56 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-23T18:00:57.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:00:56 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-23T18:00:57.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:00:56 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-23T18:00:57.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:00:56 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-23T18:00:57.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:00:56 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-23T18:00:57.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:00:56 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-23T18:00:57.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:00:56 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-23T18:00:57.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:00:56 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T18:00:57.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:00:56 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T18:00:57.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:00:56 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-23T18:00:57.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:00:56 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-23T18:00:57.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:00:56 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-23T18:00:57.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:00:56 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-23T18:00:57.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:00:56 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-23T18:00:57.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:00:56 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-23T18:00:57.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:00:56 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-23T18:00:57.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:00:56 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-23T18:00:57.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:00:56 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T18:00:57.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:00:56 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T18:00:57.921 INFO:tasks.workunit.client.0.smithi086.stdout: 1 603949 6.40 MB/sec execute 442 sec latency 19.217 ms 2024-03-23T18:00:58.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:00:57 smithi086 ceph-mon[38221]: Adjusting osd_memory_target on smithi092 to 182.5M 2024-03-23T18:00:58.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:00:57 smithi086 ceph-mon[38221]: Unable to set osd_memory_target on smithi092 to 191371980: error parsing value: Value '191371980' is below minimum 939524096 2024-03-23T18:00:58.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:00:57 smithi086 ceph-mon[38221]: Adjusting osd_memory_target on smithi152 to 182.5M 2024-03-23T18:00:58.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:00:57 smithi086 ceph-mon[38221]: Unable to set osd_memory_target on smithi152 to 191374848: error parsing value: Value '191374848' is below minimum 939524096 2024-03-23T18:00:58.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:00:57 smithi086 ceph-mon[38221]: pgmap v321: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 230 op/s 2024-03-23T18:00:58.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:00:57 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-23T18:00:58.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:00:57 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-23T18:00:58.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:00:57 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-23T18:00:58.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:00:57 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-23T18:00:58.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:00:57 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T18:00:58.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:00:57 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T18:00:58.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:00:57 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T18:00:58.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:00:57 smithi092 ceph-mon[79089]: Adjusting osd_memory_target on smithi092 to 182.5M 2024-03-23T18:00:58.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:00:57 smithi092 ceph-mon[79089]: Unable to set osd_memory_target on smithi092 to 191371980: error parsing value: Value '191371980' is below minimum 939524096 2024-03-23T18:00:58.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:00:57 smithi092 ceph-mon[79089]: Adjusting osd_memory_target on smithi152 to 182.5M 2024-03-23T18:00:58.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:00:57 smithi092 ceph-mon[79089]: Unable to set osd_memory_target on smithi152 to 191374848: error parsing value: Value '191374848' is below minimum 939524096 2024-03-23T18:00:58.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:00:57 smithi092 ceph-mon[79089]: pgmap v321: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 230 op/s 2024-03-23T18:00:58.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:00:57 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-23T18:00:58.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:00:57 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-23T18:00:58.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:00:57 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-23T18:00:58.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:00:57 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-23T18:00:58.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:00:57 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T18:00:58.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:00:57 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T18:00:58.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:00:57 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T18:00:58.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:00:57 smithi152 ceph-mon[47949]: Adjusting osd_memory_target on smithi092 to 182.5M 2024-03-23T18:00:58.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:00:57 smithi152 ceph-mon[47949]: Unable to set osd_memory_target on smithi092 to 191371980: error parsing value: Value '191371980' is below minimum 939524096 2024-03-23T18:00:58.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:00:57 smithi152 ceph-mon[47949]: Adjusting osd_memory_target on smithi152 to 182.5M 2024-03-23T18:00:58.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:00:57 smithi152 ceph-mon[47949]: Unable to set osd_memory_target on smithi152 to 191374848: error parsing value: Value '191374848' is below minimum 939524096 2024-03-23T18:00:58.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:00:57 smithi152 ceph-mon[47949]: pgmap v321: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 230 op/s 2024-03-23T18:00:58.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:00:57 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-23T18:00:58.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:00:57 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-23T18:00:58.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:00:57 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-23T18:00:58.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:00:57 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-23T18:00:58.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:00:57 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T18:00:58.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:00:57 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T18:00:58.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:00:57 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T18:00:58.920 INFO:tasks.workunit.client.0.smithi086.stdout: 1 605213 6.40 MB/sec execute 443 sec latency 13.767 ms 2024-03-23T18:00:59.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:00:58 smithi086 ceph-mon[38221]: Adjusting osd_memory_target on smithi086 to 221.5M 2024-03-23T18:00:59.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:00:58 smithi086 ceph-mon[38221]: Unable to set osd_memory_target on smithi086 to 232352870: error parsing value: Value '232352870' is below minimum 939524096 2024-03-23T18:00:59.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:00:58 smithi092 ceph-mon[79089]: Adjusting osd_memory_target on smithi086 to 221.5M 2024-03-23T18:00:59.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:00:58 smithi092 ceph-mon[79089]: Unable to set osd_memory_target on smithi086 to 232352870: error parsing value: Value '232352870' is below minimum 939524096 2024-03-23T18:00:59.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:00:58 smithi152 ceph-mon[47949]: Adjusting osd_memory_target on smithi086 to 221.5M 2024-03-23T18:00:59.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:00:58 smithi152 ceph-mon[47949]: Unable to set osd_memory_target on smithi086 to 232352870: error parsing value: Value '232352870' is below minimum 939524096 2024-03-23T18:00:59.920 INFO:tasks.workunit.client.0.smithi086.stdout: 1 607291 6.42 MB/sec execute 444 sec latency 17.573 ms 2024-03-23T18:01:00.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:00:59 smithi086 ceph-mon[38221]: pgmap v322: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 230 op/s 2024-03-23T18:01:00.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:00:59 smithi092 ceph-mon[79089]: pgmap v322: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 230 op/s 2024-03-23T18:01:00.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:00:59 smithi152 ceph-mon[47949]: pgmap v322: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 230 op/s 2024-03-23T18:01:00.920 INFO:tasks.workunit.client.0.smithi086.stdout: 1 608247 6.41 MB/sec execute 445 sec latency 39.082 ms 2024-03-23T18:01:01.921 INFO:tasks.workunit.client.0.smithi086.stdout: 1 609435 6.40 MB/sec execute 446 sec latency 27.560 ms 2024-03-23T18:01:02.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:01:01 smithi086 ceph-mon[38221]: pgmap v323: 97 pgs: 97 active+clean; 241 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.9 MiB/s wr, 330 op/s 2024-03-23T18:01:02.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:01:01 smithi092 ceph-mon[79089]: pgmap v323: 97 pgs: 97 active+clean; 241 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.9 MiB/s wr, 330 op/s 2024-03-23T18:01:02.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:01:01 smithi152 ceph-mon[47949]: pgmap v323: 97 pgs: 97 active+clean; 241 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.9 MiB/s wr, 330 op/s 2024-03-23T18:01:02.921 INFO:tasks.workunit.client.0.smithi086.stdout: 1 611186 6.42 MB/sec execute 447 sec latency 15.570 ms 2024-03-23T18:01:03.921 INFO:tasks.workunit.client.0.smithi086.stdout: 1 612484 6.42 MB/sec execute 448 sec latency 15.467 ms 2024-03-23T18:01:04.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:01:03 smithi086 ceph-mon[38221]: pgmap v324: 97 pgs: 97 active+clean; 241 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 221 op/s 2024-03-23T18:01:04.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:01:03 smithi092 ceph-mon[79089]: pgmap v324: 97 pgs: 97 active+clean; 241 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 221 op/s 2024-03-23T18:01:04.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:01:03 smithi152 ceph-mon[47949]: pgmap v324: 97 pgs: 97 active+clean; 241 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 221 op/s 2024-03-23T18:01:04.921 INFO:tasks.workunit.client.0.smithi086.stdout: 1 614543 6.43 MB/sec execute 449 sec latency 16.491 ms 2024-03-23T18:01:05.921 INFO:tasks.workunit.client.0.smithi086.stdout: 1 615179 6.42 MB/sec execute 450 sec latency 53.938 ms 2024-03-23T18:01:06.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:01:05 smithi086 ceph-mon[38221]: pgmap v325: 97 pgs: 97 active+clean; 248 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.0 MiB/s wr, 359 op/s 2024-03-23T18:01:06.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:01:05 smithi086 ceph-mon[38221]: osdmap e93: 12 total, 12 up, 12 in 2024-03-23T18:01:06.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:01:05 smithi092 ceph-mon[79089]: pgmap v325: 97 pgs: 97 active+clean; 248 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.0 MiB/s wr, 359 op/s 2024-03-23T18:01:06.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:01:05 smithi092 ceph-mon[79089]: osdmap e93: 12 total, 12 up, 12 in 2024-03-23T18:01:06.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:01:05 smithi152 ceph-mon[47949]: pgmap v325: 97 pgs: 97 active+clean; 248 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.0 MiB/s wr, 359 op/s 2024-03-23T18:01:06.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:01:05 smithi152 ceph-mon[47949]: osdmap e93: 12 total, 12 up, 12 in 2024-03-23T18:01:06.855 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.855+0000 7f6882600700 1 -- 172.21.15.86:0/1821706791 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f687c106030 msgr2=0x7f687c10dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:01:06.855 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.855+0000 7f6882600700 1 --2- 172.21.15.86:0/1821706791 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f687c106030 0x7f687c10dbb0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f687400b0a0 tx=0x7f687400b3b0 comp rx=0 tx=0).stop 2024-03-23T18:01:06.856 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.855+0000 7f6882600700 1 -- 172.21.15.86:0/1821706791 shutdown_connections 2024-03-23T18:01:06.856 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.855+0000 7f6882600700 1 --2- 172.21.15.86:0/1821706791 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f687c106030 0x7f687c10dbb0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:06.856 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.855+0000 7f6882600700 1 --2- 172.21.15.86:0/1821706791 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f687c100f10 0x7f687c1012f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:06.856 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.855+0000 7f6882600700 1 --2- 172.21.15.86:0/1821706791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f687c1018c0 0x7f687c105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:06.856 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.855+0000 7f6882600700 1 -- 172.21.15.86:0/1821706791 >> 172.21.15.86:0/1821706791 conn(0x7f687c0fc750 msgr2=0x7f687c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:01:06.856 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.855+0000 7f6882600700 1 -- 172.21.15.86:0/1821706791 shutdown_connections 2024-03-23T18:01:06.856 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.855+0000 7f6882600700 1 -- 172.21.15.86:0/1821706791 wait complete. 2024-03-23T18:01:06.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.856+0000 7f6882600700 1 Processor -- start 2024-03-23T18:01:06.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.856+0000 7f6882600700 1 -- start start 2024-03-23T18:01:06.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.857+0000 7f6882600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f687c100f10 0x7f687c199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:06.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.857+0000 7f6882600700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f687c106030 0x7f687c199880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:06.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.857+0000 7f6882600700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f687c19dc20 0x7f687c19e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:06.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.857+0000 7f6882600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f687c19e6a0 con 0x7f687c100f10 2024-03-23T18:01:06.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.857+0000 7f6882600700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f687c19e810 con 0x7f687c106030 2024-03-23T18:01:06.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.857+0000 7f6882600700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f687c19e9d0 con 0x7f687c19dc20 2024-03-23T18:01:06.859 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.857+0000 7f687b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f687c100f10 0x7f687c199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:06.859 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.857+0000 7f687aa00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f687c106030 0x7f687c199880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:06.859 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.857+0000 7f687b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f687c100f10 0x7f687c199340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42784/0 (socket says 172.21.15.86:42784) 2024-03-23T18:01:06.859 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.857+0000 7f687b400700 1 -- 172.21.15.86:0/2533945979 learned_addr learned my addr 172.21.15.86:0/2533945979 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:01:06.859 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.857+0000 7f687be00700 1 --2- 172.21.15.86:0/2533945979 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f687c19dc20 0x7f687c19e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:06.859 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.858+0000 7f687aa00700 1 -- 172.21.15.86:0/2533945979 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f687c19dc20 msgr2=0x7f687c19e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:01:06.859 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.858+0000 7f687aa00700 1 --2- 172.21.15.86:0/2533945979 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f687c19dc20 0x7f687c19e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:06.859 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.858+0000 7f687aa00700 1 -- 172.21.15.86:0/2533945979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f687c100f10 msgr2=0x7f687c199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:01:06.859 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.858+0000 7f687aa00700 1 --2- 172.21.15.86:0/2533945979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f687c100f10 0x7f687c199340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:06.859 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.858+0000 7f687aa00700 1 -- 172.21.15.86:0/2533945979 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6874009d00 con 0x7f687c106030 2024-03-23T18:01:06.859 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.858+0000 7f687aa00700 1 --2- 172.21.15.86:0/2533945979 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f687c106030 0x7f687c199880 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f686800e9c0 tx=0x7f686800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:01:06.860 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.858+0000 7f6873e00700 1 -- 172.21.15.86:0/2533945979 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f686800c9c0 con 0x7f687c106030 2024-03-23T18:01:06.860 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.858+0000 7f6873e00700 1 -- 172.21.15.86:0/2533945979 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6868010430 con 0x7f687c106030 2024-03-23T18:01:06.860 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.858+0000 7f6882600700 1 -- 172.21.15.86:0/2533945979 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f687c1a5f30 con 0x7f687c106030 2024-03-23T18:01:06.860 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.858+0000 7f6873e00700 1 -- 172.21.15.86:0/2533945979 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6868018680 con 0x7f687c106030 2024-03-23T18:01:06.860 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.858+0000 7f6882600700 1 -- 172.21.15.86:0/2533945979 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f687c1a6360 con 0x7f687c106030 2024-03-23T18:01:06.861 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.862+0000 7f6873e00700 1 -- 172.21.15.86:0/2533945979 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f686800cb20 con 0x7f687c106030 2024-03-23T18:01:06.863 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.863+0000 7f6873e00700 1 --2- 172.21.15.86:0/2533945979 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f68600a1290 0x7f68600a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:06.863 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.863+0000 7f6873e00700 1 -- 172.21.15.86:0/2533945979 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f6868014070 con 0x7f687c106030 2024-03-23T18:01:06.863 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.863+0000 7f687b400700 1 --2- 172.21.15.86:0/2533945979 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f68600a1290 0x7f68600a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:06.864 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.864+0000 7f68961aab80 1 Processor -- start 2024-03-23T18:01:06.864 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.864+0000 7f68961aab80 1 -- start start 2024-03-23T18:01:06.864 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.864+0000 7f687b400700 1 --2- 172.21.15.86:0/2533945979 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f68600a1290 0x7f68600a3750 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f686c009d20 tx=0x7f686c009260 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:01:06.865 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.864+0000 7f68961aab80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ccbf210100 0x55ccbf2104e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:06.866 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.864+0000 7f68961aab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccbf210a20 0x55ccbf21b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:06.866 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.864+0000 7f68961aab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ccbf21b8d0 0x55ccbf21dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:06.866 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.864+0000 7f68961aab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccbf169930 con 0x55ccbf210a20 2024-03-23T18:01:06.866 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.864+0000 7f68961aab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccbf1672f0 con 0x55ccbf21b8d0 2024-03-23T18:01:06.866 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.864+0000 7f68961aab80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccbf162be0 con 0x55ccbf210100 2024-03-23T18:01:06.866 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.865+0000 7f687b400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ccbf210100 0x55ccbf2104e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:06.866 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.865+0000 7f687be00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ccbf21b8d0 0x55ccbf21dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:06.866 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.865+0000 7f687be00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ccbf21b8d0 0x55ccbf21dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:59628/0 (socket says 172.21.15.86:59628) 2024-03-23T18:01:06.866 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.865+0000 7f687be00700 1 -- 172.21.15.86:0/623462434 learned_addr learned my addr 172.21.15.86:0/623462434 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:01:06.866 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.865+0000 7f687aa00700 1 --2- 172.21.15.86:0/623462434 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccbf210a20 0x55ccbf21b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:06.866 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.865+0000 7f687be00700 1 -- 172.21.15.86:0/623462434 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ccbf210100 msgr2=0x55ccbf2104e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:01:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.865+0000 7f687be00700 1 --2- 172.21.15.86:0/623462434 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ccbf210100 0x55ccbf2104e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.865+0000 7f687be00700 1 -- 172.21.15.86:0/623462434 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccbf210a20 msgr2=0x55ccbf21b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:01:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.865+0000 7f687be00700 1 --2- 172.21.15.86:0/623462434 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccbf210a20 0x55ccbf21b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.865+0000 7f687be00700 1 -- 172.21.15.86:0/623462434 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ccbefe4350 con 0x55ccbf21b8d0 2024-03-23T18:01:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.865+0000 7f687be00700 1 --2- 172.21.15.86:0/623462434 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ccbf21b8d0 0x55ccbf21dcc0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f6874015000 tx=0x7f6874009470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3b3111b7386b98e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:01:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.866+0000 7f6870c00700 1 -- 172.21.15.86:0/623462434 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f687400e040 con 0x55ccbf21b8d0 2024-03-23T18:01:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.866+0000 7f6870c00700 1 -- 172.21.15.86:0/623462434 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6874007e00 con 0x55ccbf21b8d0 2024-03-23T18:01:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.866+0000 7f6870c00700 1 -- 172.21.15.86:0/623462434 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6874007940 con 0x55ccbf21b8d0 2024-03-23T18:01:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.866+0000 7f68961aab80 1 -- 172.21.15.86:0/623462434 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ccbf21b8d0 msgr2=0x55ccbf21dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:01:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.866+0000 7f68961aab80 1 --2- 172.21.15.86:0/623462434 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ccbf21b8d0 0x55ccbf21dcc0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f6874015000 tx=0x7f6874009470 comp rx=0 tx=0).stop 2024-03-23T18:01:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.866+0000 7f68961aab80 1 -- 172.21.15.86:0/623462434 shutdown_connections 2024-03-23T18:01:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.866+0000 7f68961aab80 1 --2- 172.21.15.86:0/623462434 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ccbf21b8d0 0x55ccbf21dcc0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.866+0000 7f68961aab80 1 --2- 172.21.15.86:0/623462434 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ccbf210100 0x55ccbf2104e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.866+0000 7f68961aab80 1 --2- 172.21.15.86:0/623462434 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccbf210a20 0x55ccbf21b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.866+0000 7f68961aab80 1 -- 172.21.15.86:0/623462434 >> 172.21.15.86:0/623462434 conn(0x55ccbf170640 msgr2=0x55ccbf20c520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:01:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.867+0000 7f68961aab80 1 -- 172.21.15.86:0/623462434 shutdown_connections 2024-03-23T18:01:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.867+0000 7f68961aab80 1 -- 172.21.15.86:0/623462434 wait complete. 2024-03-23T18:01:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.867+0000 7f68961aab80 1 -- start start 2024-03-23T18:01:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.867+0000 7f68961aab80 1 Processor -- start 2024-03-23T18:01:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.868+0000 7f68961aab80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ccbf210100 0x55ccbf21b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.868+0000 7f68961aab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ccbf210a20 0x55ccbf218590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.868+0000 7f68961aab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccbf21b8d0 0x55ccbf218bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.868+0000 7f68961aab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccbf0fed20 con 0x55ccbf21b8d0 2024-03-23T18:01:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.868+0000 7f68961aab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccbf0fe9c0 con 0x55ccbf210a20 2024-03-23T18:01:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.868+0000 7f68961aab80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccbefe6640 con 0x55ccbf210100 2024-03-23T18:01:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.868+0000 7f68961aab80 10 client.0 _ll_register_callbacks cb 0x55ccbf108410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T18:01:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.868+0000 7f687be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccbf21b8d0 0x55ccbf218bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.868+0000 7f687aa00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ccbf210a20 0x55ccbf218590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.868+0000 7f687aa00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ccbf210a20 0x55ccbf218590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:59642/0 (socket says 172.21.15.86:59642) 2024-03-23T18:01:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.868+0000 7f687be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccbf21b8d0 0x55ccbf218bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42802/0 (socket says 172.21.15.86:42802) 2024-03-23T18:01:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.868+0000 7f687aa00700 1 -- 172.21.15.86:0/3261215353 learned_addr learned my addr 172.21.15.86:0/3261215353 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:01:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.868+0000 7f687b400700 1 --2- 172.21.15.86:0/3261215353 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ccbf210100 0x55ccbf21b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.868+0000 7f687aa00700 1 -- 172.21.15.86:0/3261215353 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ccbf210100 msgr2=0x55ccbf21b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:01:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.868+0000 7f687aa00700 1 --2- 172.21.15.86:0/3261215353 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ccbf210100 0x55ccbf21b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.868+0000 7f687aa00700 1 -- 172.21.15.86:0/3261215353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccbf21b8d0 msgr2=0x55ccbf218bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:01:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.868+0000 7f687aa00700 1 --2- 172.21.15.86:0/3261215353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccbf21b8d0 0x55ccbf218bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.869+0000 7f687aa00700 1 -- 172.21.15.86:0/3261215353 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f687c10e420 con 0x55ccbf210a20 2024-03-23T18:01:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.869+0000 7f687be00700 1 --2- 172.21.15.86:0/3261215353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccbf21b8d0 0x55ccbf218bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T18:01:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.869+0000 7f687aa00700 1 --2- 172.21.15.86:0/3261215353 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ccbf210a20 0x55ccbf218590 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f686800f010 tx=0x7f686809a590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e41551b4d705f32 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:01:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.869+0000 7f6865600700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T18:01:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.869+0000 7f6865600700 1 -- 172.21.15.86:0/3261215353 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f686809aea0 con 0x55ccbf210a20 2024-03-23T18:01:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.869+0000 7f6865600700 1 -- 172.21.15.86:0/3261215353 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f6868099e90 con 0x55ccbf210a20 2024-03-23T18:01:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.869+0000 7f68961aab80 1 -- 172.21.15.86:0/3261215353 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ccbf123320 con 0x55ccbf210a20 2024-03-23T18:01:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.869+0000 7f6865600700 1 -- 172.21.15.86:0/3261215353 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f686809dcc0 con 0x55ccbf210a20 2024-03-23T18:01:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.869+0000 7f68961aab80 10 client.26876 fetch_fsmap learned FSMap version 47 2024-03-23T18:01:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.869+0000 7f68961aab80 1 -- 172.21.15.86:0/3261215353 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x55ccbef90b00 con 0x55ccbf210a20 2024-03-23T18:01:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.869+0000 7f6865600700 1 -- 172.21.15.86:0/3261215353 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6868063070 con 0x55ccbf210a20 2024-03-23T18:01:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.870+0000 7f6865600700 1 -- 172.21.15.86:0/3261215353 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 47) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f68680a1e80 con 0x55ccbf210a20 2024-03-23T18:01:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.870+0000 7f68961aab80 10 client.26876 fetch_fsmap finished waiting for FSMap version 47 2024-03-23T18:01:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.870+0000 7f68961aab80 10 client.26876 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T18:01:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.870+0000 7f68961aab80 20 client.26876 populate_metadata read hostname 'smithi086' 2024-03-23T18:01:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.870+0000 7f68961aab80 1 --2- 172.21.15.86:0/3261215353 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55ccbf22b640 0x55ccbf22da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:06.874 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.870+0000 7f68961aab80 4 client.26876 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T18:01:06.874 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.870+0000 7f68961aab80 1 -- 172.21.15.86:0/3261215353 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ccbf15b6a0 con 0x55ccbf22b640 2024-03-23T18:01:06.874 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.870+0000 7f687b400700 1 --2- 172.21.15.86:0/3261215353 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55ccbf22b640 0x55ccbf22da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:06.874 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.870+0000 7f687b400700 1 --2- 172.21.15.86:0/3261215353 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55ccbf22b640 0x55ccbf22da60 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9f8f99789b46e6d server_cookie=6966f1afa5705f97 in_seq=0 out_seq=0 2024-03-23T18:01:06.874 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.870+0000 7f6865600700 10 client.26876 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T18:01:06.875 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.871+0000 7f6865600700 1 -- 172.21.15.86:0/3261215353 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55ccbf15b6a0 con 0x55ccbf22b640 2024-03-23T18:01:06.875 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.871+0000 7f6865600700 10 client.26876 handle_command_reply: tid=0 2024-03-23T18:01:06.875 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.871+0000 7f68961aab80 1 client.26876 shutdown 2024-03-23T18:01:06.875 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.871+0000 7f68961aab80 1 -- 172.21.15.86:0/3261215353 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ccbf210a20 msgr2=0x55ccbf218590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:01:06.875 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.871+0000 7f68961aab80 1 --2- 172.21.15.86:0/3261215353 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ccbf210a20 0x55ccbf218590 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f686800f010 tx=0x7f686809a590 comp rx=0 tx=0).stop 2024-03-23T18:01:06.875 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.872+0000 7f68961aab80 1 -- 172.21.15.86:0/3261215353 shutdown_connections 2024-03-23T18:01:06.876 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.872+0000 7f68961aab80 1 --2- 172.21.15.86:0/3261215353 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55ccbf22b640 0x55ccbf22da60 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:06.876 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.872+0000 7f68961aab80 1 --2- 172.21.15.86:0/3261215353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccbf21b8d0 0x55ccbf218bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:06.876 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.872+0000 7f68961aab80 1 --2- 172.21.15.86:0/3261215353 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ccbf210100 0x55ccbf21b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:06.876 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.872+0000 7f6865600700 0 client.26876 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T18:01:06.876 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.872+0000 7f68961aab80 1 --2- 172.21.15.86:0/3261215353 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ccbf210a20 0x55ccbf218590 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:06.876 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.872+0000 7f68961aab80 1 -- 172.21.15.86:0/3261215353 >> 172.21.15.86:0/3261215353 conn(0x55ccbf170640 msgr2=0x55ccbf2120a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:01:06.876 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.872+0000 7f68961aab80 1 -- 172.21.15.86:0/3261215353 shutdown_connections 2024-03-23T18:01:06.876 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.872+0000 7f68961aab80 1 -- 172.21.15.86:0/3261215353 wait complete. 2024-03-23T18:01:06.876 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.872+0000 7f68961aab80 20 client.26876 trim_cache size 0 max 16384 2024-03-23T18:01:06.889 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.889+0000 7f68961aab80 1 Processor -- start 2024-03-23T18:01:06.889 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.889+0000 7f68961aab80 1 -- start start 2024-03-23T18:01:06.889 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.889+0000 7f68961aab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccbf210100 0x55ccbf232380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:06.889 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.889+0000 7f68961aab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ccbf210a20 0x55ccbf222540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:06.889 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.889+0000 7f68961aab80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ccbf21b8d0 0x55ccbf2053e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:06.889 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.889+0000 7f68961aab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccbf2080d0 con 0x55ccbf210100 2024-03-23T18:01:06.889 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.889+0000 7f68961aab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccbf208c80 con 0x55ccbf210a20 2024-03-23T18:01:06.890 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.889+0000 7f68961aab80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccbf208400 con 0x55ccbf21b8d0 2024-03-23T18:01:06.891 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.889+0000 7f687aa00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ccbf210a20 0x55ccbf222540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:06.891 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.889+0000 7f687aa00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ccbf210a20 0x55ccbf222540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:59650/0 (socket says 172.21.15.86:59650) 2024-03-23T18:01:06.891 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.889+0000 7f687aa00700 1 -- 172.21.15.86:0/610181831 learned_addr learned my addr 172.21.15.86:0/610181831 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:01:06.891 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.889+0000 7f687be00700 1 --2- 172.21.15.86:0/610181831 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ccbf21b8d0 0x55ccbf2053e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:06.892 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.890+0000 7f687b400700 1 --2- 172.21.15.86:0/610181831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccbf210100 0x55ccbf232380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:06.892 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.890+0000 7f687aa00700 1 -- 172.21.15.86:0/610181831 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ccbf21b8d0 msgr2=0x55ccbf2053e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:01:06.892 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.890+0000 7f687aa00700 1 --2- 172.21.15.86:0/610181831 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ccbf21b8d0 0x55ccbf2053e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:06.892 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.890+0000 7f687aa00700 1 -- 172.21.15.86:0/610181831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccbf210100 msgr2=0x55ccbf232380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:01:06.892 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.890+0000 7f687aa00700 1 --2- 172.21.15.86:0/610181831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccbf210100 0x55ccbf232380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:06.892 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.890+0000 7f687aa00700 1 -- 172.21.15.86:0/610181831 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ccbf128e40 con 0x55ccbf210a20 2024-03-23T18:01:06.892 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.890+0000 7f687b400700 1 --2- 172.21.15.86:0/610181831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccbf210100 0x55ccbf232380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T18:01:06.892 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.890+0000 7f687aa00700 1 --2- 172.21.15.86:0/610181831 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ccbf210a20 0x55ccbf222540 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f68680a2280 tx=0x7f6868098440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac5087fc84e0426e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:01:06.892 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.890+0000 7f6871600700 1 -- 172.21.15.86:0/610181831 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f686809aea0 con 0x55ccbf210a20 2024-03-23T18:01:06.892 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.890+0000 7f6871600700 1 -- 172.21.15.86:0/610181831 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6868063070 con 0x55ccbf210a20 2024-03-23T18:01:06.892 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.890+0000 7f6871600700 1 -- 172.21.15.86:0/610181831 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68680a2380 con 0x55ccbf210a20 2024-03-23T18:01:06.892 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.891+0000 7f68961aab80 1 -- 172.21.15.86:0/610181831 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ccbf210a20 msgr2=0x55ccbf222540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:01:06.892 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.891+0000 7f68961aab80 1 --2- 172.21.15.86:0/610181831 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ccbf210a20 0x55ccbf222540 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f68680a2280 tx=0x7f6868098440 comp rx=0 tx=0).stop 2024-03-23T18:01:06.892 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.891+0000 7f68961aab80 1 -- 172.21.15.86:0/610181831 shutdown_connections 2024-03-23T18:01:06.893 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.891+0000 7f68961aab80 1 --2- 172.21.15.86:0/610181831 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ccbf21b8d0 0x55ccbf2053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:06.893 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.891+0000 7f68961aab80 1 --2- 172.21.15.86:0/610181831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccbf210100 0x55ccbf232380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:06.893 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.891+0000 7f68961aab80 1 --2- 172.21.15.86:0/610181831 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ccbf210a20 0x55ccbf222540 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:06.893 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.891+0000 7f68961aab80 1 -- 172.21.15.86:0/610181831 >> 172.21.15.86:0/610181831 conn(0x55ccbf170640 msgr2=0x55ccbf216e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:01:06.893 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.891+0000 7f68961aab80 1 -- 172.21.15.86:0/610181831 shutdown_connections 2024-03-23T18:01:06.893 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.891+0000 7f68961aab80 1 -- 172.21.15.86:0/610181831 wait complete. 2024-03-23T18:01:06.896 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.892+0000 7f68961aab80 1 -- start start 2024-03-23T18:01:06.896 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.892+0000 7f68961aab80 1 Processor -- start 2024-03-23T18:01:06.896 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.893+0000 7f68961aab80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ccbf210100 0x55ccbf220710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:06.896 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.893+0000 7f68961aab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccbf210a20 0x55ccbf220c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:06.896 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.893+0000 7f68961aab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ccbf21b8d0 0x55ccbf27c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:06.896 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.893+0000 7f68961aab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccbf221330 con 0x55ccbf210a20 2024-03-23T18:01:06.896 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.893+0000 7f68961aab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccbf27c660 con 0x55ccbf21b8d0 2024-03-23T18:01:06.896 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.893+0000 7f68961aab80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccbf27c7a0 con 0x55ccbf210100 2024-03-23T18:01:06.896 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.893+0000 7f68961aab80 10 client.0 _ll_register_callbacks cb 0x55ccbf108410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T18:01:06.896 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.893+0000 7f687b400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ccbf210100 0x55ccbf220710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.893+0000 7f687b400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ccbf210100 0x55ccbf220710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:43990/0 (socket says 172.21.15.86:43990) 2024-03-23T18:01:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.893+0000 7f687b400700 1 -- 172.21.15.86:0/1023659032 learned_addr learned my addr 172.21.15.86:0/1023659032 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:01:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.893+0000 7f687be00700 1 --2- 172.21.15.86:0/1023659032 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ccbf21b8d0 0x55ccbf27c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.893+0000 7f687aa00700 1 --2- 172.21.15.86:0/1023659032 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccbf210a20 0x55ccbf220c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.893+0000 7f687b400700 1 -- 172.21.15.86:0/1023659032 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ccbf21b8d0 msgr2=0x55ccbf27c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:01:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.893+0000 7f687b400700 1 --2- 172.21.15.86:0/1023659032 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ccbf21b8d0 0x55ccbf27c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.893+0000 7f687b400700 1 -- 172.21.15.86:0/1023659032 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccbf210a20 msgr2=0x55ccbf220c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:01:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.893+0000 7f687b400700 1 --2- 172.21.15.86:0/1023659032 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccbf210a20 0x55ccbf220c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.893+0000 7f687b400700 1 -- 172.21.15.86:0/1023659032 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ccbef90b00 con 0x55ccbf210100 2024-03-23T18:01:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.893+0000 7f687aa00700 1 --2- 172.21.15.86:0/1023659032 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccbf210a20 0x55ccbf220c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T18:01:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.894+0000 7f687b400700 1 --2- 172.21.15.86:0/1023659032 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ccbf210100 0x55ccbf220710 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f686c005bf0 tx=0x7f686c0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7fbb2d79b4642a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:01:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.894+0000 7f6865600700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-03-23T18:01:06.898 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.894+0000 7f6865600700 1 -- 172.21.15.86:0/1023659032 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f686c022070 con 0x55ccbf210100 2024-03-23T18:01:06.898 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.894+0000 7f68961aab80 1 -- 172.21.15.86:0/1023659032 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6874007e00 con 0x55ccbf210100 2024-03-23T18:01:06.898 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.894+0000 7f6865600700 1 -- 172.21.15.86:0/1023659032 <== mon.1 v2:172.21.15.152:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f686c02be20 con 0x55ccbf210100 2024-03-23T18:01:06.898 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.894+0000 7f6865600700 1 -- 172.21.15.86:0/1023659032 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f686c01de20 con 0x55ccbf210100 2024-03-23T18:01:06.898 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.894+0000 7f68961aab80 10 client.26863 fetch_fsmap learned FSMap version 47 2024-03-23T18:01:06.898 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.894+0000 7f68961aab80 1 -- 172.21.15.86:0/1023659032 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x55ccbefe8c70 con 0x55ccbf210100 2024-03-23T18:01:06.898 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.894+0000 7f6865600700 1 -- 172.21.15.86:0/1023659032 <== mon.1 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6874007e00 con 0x55ccbf210100 2024-03-23T18:01:06.898 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.894+0000 7f6865600700 1 -- 172.21.15.86:0/1023659032 <== mon.1 v2:172.21.15.152:3300/0 5 ==== fsmap(e 47) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f686c03dc10 con 0x55ccbf210100 2024-03-23T18:01:06.898 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.895+0000 7f68961aab80 10 client.26863 fetch_fsmap finished waiting for FSMap version 47 2024-03-23T18:01:06.898 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.895+0000 7f68961aab80 10 client.26863 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T18:01:06.898 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.895+0000 7f68961aab80 20 client.26863 populate_metadata read hostname 'smithi086' 2024-03-23T18:01:06.898 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.895+0000 7f68961aab80 1 --2- 172.21.15.86:0/1023659032 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55ccbf27dee0 0x55ccbf280300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:06.899 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.895+0000 7f68961aab80 4 client.26863 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T18:01:06.899 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.895+0000 7f68961aab80 1 -- 172.21.15.86:0/1023659032 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ccbefb1c90 con 0x55ccbf27dee0 2024-03-23T18:01:06.899 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.895+0000 7f687aa00700 1 --2- 172.21.15.86:0/1023659032 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55ccbf27dee0 0x55ccbf280300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:06.899 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.895+0000 7f687aa00700 1 --2- 172.21.15.86:0/1023659032 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55ccbf27dee0 0x55ccbf280300 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1632c83fe726faec server_cookie=1888eca9a8afc1f6 in_seq=0 out_seq=0 2024-03-23T18:01:06.899 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.895+0000 7f6865600700 10 client.26863 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T18:01:06.921 INFO:tasks.workunit.client.0.smithi086.stdout: 1 616886 6.42 MB/sec execute 451 sec latency 21.585 ms 2024-03-23T18:01:06.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.943+0000 7f6865600700 1 -- 172.21.15.86:0/1023659032 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ccbefb1c90 con 0x55ccbf27dee0 2024-03-23T18:01:06.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.943+0000 7f6865600700 10 client.26863 handle_command_reply: tid=0 2024-03-23T18:01:06.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.943+0000 7f68961aab80 1 client.26863 shutdown 2024-03-23T18:01:06.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.943+0000 7f68961aab80 1 -- 172.21.15.86:0/1023659032 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ccbf210100 msgr2=0x55ccbf220710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:01:06.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.943+0000 7f68961aab80 1 --2- 172.21.15.86:0/1023659032 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ccbf210100 0x55ccbf220710 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f686c005bf0 tx=0x7f686c0047c0 comp rx=0 tx=0).stop 2024-03-23T18:01:06.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.943+0000 7f68961aab80 1 -- 172.21.15.86:0/1023659032 shutdown_connections 2024-03-23T18:01:06.944 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.943+0000 7f68961aab80 1 --2- 172.21.15.86:0/1023659032 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55ccbf27dee0 0x55ccbf280300 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:06.944 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.943+0000 7f68961aab80 1 --2- 172.21.15.86:0/1023659032 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ccbf21b8d0 0x55ccbf27c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:06.944 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.943+0000 7f68961aab80 1 --2- 172.21.15.86:0/1023659032 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ccbf210100 0x55ccbf220710 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:06.944 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.943+0000 7f68961aab80 1 --2- 172.21.15.86:0/1023659032 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccbf210a20 0x55ccbf220c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:06.944 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.943+0000 7f68961aab80 1 -- 172.21.15.86:0/1023659032 >> 172.21.15.86:0/1023659032 conn(0x55ccbf170640 msgr2=0x55ccbf20bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:01:06.944 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.943+0000 7f6865600700 0 client.26863 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T18:01:06.945 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.944+0000 7f68961aab80 1 -- 172.21.15.86:0/1023659032 shutdown_connections 2024-03-23T18:01:06.945 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.944+0000 7f68961aab80 1 -- 172.21.15.86:0/1023659032 wait complete. 2024-03-23T18:01:06.945 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.944+0000 7f68961aab80 20 client.26863 trim_cache size 0 max 16384 2024-03-23T18:01:06.945 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-03-23T18:01:06.945 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-03-23T18:01:06.945 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-03-23T18:01:06.945 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-03-23T18:01:06.946 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.945+0000 7f6882600700 1 -- 172.21.15.86:0/2533945979 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f68600a1290 msgr2=0x7f68600a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:01:06.946 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.945+0000 7f6882600700 1 --2- 172.21.15.86:0/2533945979 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f68600a1290 0x7f68600a3750 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f686c009d20 tx=0x7f686c009260 comp rx=0 tx=0).stop 2024-03-23T18:01:06.946 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.945+0000 7f6882600700 1 -- 172.21.15.86:0/2533945979 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f687c106030 msgr2=0x7f687c199880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:01:06.946 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.945+0000 7f6882600700 1 --2- 172.21.15.86:0/2533945979 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f687c106030 0x7f687c199880 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f686800e9c0 tx=0x7f686800ecd0 comp rx=0 tx=0).stop 2024-03-23T18:01:06.946 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.945+0000 7f6882600700 1 -- 172.21.15.86:0/2533945979 shutdown_connections 2024-03-23T18:01:06.946 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.945+0000 7f6882600700 1 --2- 172.21.15.86:0/2533945979 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f68600a1290 0x7f68600a3750 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:06.946 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.945+0000 7f6882600700 1 --2- 172.21.15.86:0/2533945979 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f687c19dc20 0x7f687c19e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:06.947 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.945+0000 7f6882600700 1 --2- 172.21.15.86:0/2533945979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f687c100f10 0x7f687c199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:06.947 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.945+0000 7f6882600700 1 --2- 172.21.15.86:0/2533945979 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f687c106030 0x7f687c199880 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:06.947 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.945+0000 7f6882600700 1 -- 172.21.15.86:0/2533945979 >> 172.21.15.86:0/2533945979 conn(0x7f687c0fc750 msgr2=0x7f687c10d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:01:06.947 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.945+0000 7f6882600700 1 -- 172.21.15.86:0/2533945979 shutdown_connections 2024-03-23T18:01:06.947 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:06.945+0000 7f6882600700 1 -- 172.21.15.86:0/2533945979 wait complete. 2024-03-23T18:01:07.921 INFO:tasks.workunit.client.0.smithi086.stdout: 1 618384 6.43 MB/sec execute 452 sec latency 20.640 ms 2024-03-23T18:01:08.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:01:07 smithi086 ceph-mon[38221]: pgmap v327: 97 pgs: 97 active+clean; 248 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.1 MiB/s wr, 285 op/s 2024-03-23T18:01:08.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:01:07 smithi092 ceph-mon[79089]: pgmap v327: 97 pgs: 97 active+clean; 248 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.1 MiB/s wr, 285 op/s 2024-03-23T18:01:08.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:01:07 smithi152 ceph-mon[47949]: pgmap v327: 97 pgs: 97 active+clean; 248 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.1 MiB/s wr, 285 op/s 2024-03-23T18:01:08.921 INFO:tasks.workunit.client.0.smithi086.stdout: 1 619898 6.43 MB/sec execute 453 sec latency 15.797 ms 2024-03-23T18:01:09.921 INFO:tasks.workunit.client.0.smithi086.stdout: 1 621771 6.45 MB/sec execute 454 sec latency 14.367 ms 2024-03-23T18:01:10.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:01:09 smithi086 ceph-mon[38221]: pgmap v328: 97 pgs: 97 active+clean; 248 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.1 MiB/s wr, 285 op/s 2024-03-23T18:01:10.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:01:09 smithi092 ceph-mon[79089]: pgmap v328: 97 pgs: 97 active+clean; 248 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.1 MiB/s wr, 285 op/s 2024-03-23T18:01:10.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:01:09 smithi152 ceph-mon[47949]: pgmap v328: 97 pgs: 97 active+clean; 248 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.1 MiB/s wr, 285 op/s 2024-03-23T18:01:10.921 INFO:tasks.workunit.client.0.smithi086.stdout: 1 622778 6.44 MB/sec execute 455 sec latency 43.820 ms 2024-03-23T18:01:11.921 INFO:tasks.workunit.client.0.smithi086.stdout: 1 624732 6.45 MB/sec execute 456 sec latency 21.981 ms 2024-03-23T18:01:12.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:01:11 smithi086 ceph-mon[38221]: pgmap v329: 97 pgs: 97 active+clean; 246 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 310 op/s 2024-03-23T18:01:12.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:01:11 smithi086 ceph-mon[38221]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:01:12.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:01:11 smithi092 ceph-mon[79089]: pgmap v329: 97 pgs: 97 active+clean; 246 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 310 op/s 2024-03-23T18:01:12.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:01:11 smithi092 ceph-mon[79089]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:01:12.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:01:11 smithi152 ceph-mon[47949]: pgmap v329: 97 pgs: 97 active+clean; 246 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 310 op/s 2024-03-23T18:01:12.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:01:11 smithi152 ceph-mon[47949]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:01:12.921 INFO:tasks.workunit.client.0.smithi086.stdout: 1 625950 6.45 MB/sec execute 457 sec latency 15.379 ms 2024-03-23T18:01:13.922 INFO:tasks.workunit.client.0.smithi086.stdout: 1 627953 6.46 MB/sec execute 458 sec latency 15.260 ms 2024-03-23T18:01:14.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:01:13 smithi086 ceph-mon[38221]: pgmap v330: 97 pgs: 97 active+clean; 246 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 310 op/s 2024-03-23T18:01:14.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:01:13 smithi092 ceph-mon[79089]: pgmap v330: 97 pgs: 97 active+clean; 246 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 310 op/s 2024-03-23T18:01:14.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:01:13 smithi152 ceph-mon[47949]: pgmap v330: 97 pgs: 97 active+clean; 246 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 310 op/s 2024-03-23T18:01:14.922 INFO:tasks.workunit.client.0.smithi086.stdout: 1 629298 6.46 MB/sec execute 459 sec latency 24.382 ms 2024-03-23T18:01:15.922 INFO:tasks.workunit.client.0.smithi086.stdout: 1 631019 6.46 MB/sec execute 460 sec latency 56.074 ms 2024-03-23T18:01:16.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:01:15 smithi086 ceph-mon[38221]: pgmap v331: 97 pgs: 97 active+clean; 253 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 267 op/s 2024-03-23T18:01:16.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:01:15 smithi086 ceph-mon[38221]: osdmap e94: 12 total, 12 up, 12 in 2024-03-23T18:01:16.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:01:15 smithi092 ceph-mon[79089]: pgmap v331: 97 pgs: 97 active+clean; 253 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 267 op/s 2024-03-23T18:01:16.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:01:15 smithi092 ceph-mon[79089]: osdmap e94: 12 total, 12 up, 12 in 2024-03-23T18:01:16.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:01:15 smithi152 ceph-mon[47949]: pgmap v331: 97 pgs: 97 active+clean; 253 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 267 op/s 2024-03-23T18:01:16.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:01:15 smithi152 ceph-mon[47949]: osdmap e94: 12 total, 12 up, 12 in 2024-03-23T18:01:16.922 INFO:tasks.workunit.client.0.smithi086.stdout: 1 632546 6.47 MB/sec execute 461 sec latency 16.389 ms 2024-03-23T18:01:17.922 INFO:tasks.workunit.client.0.smithi086.stdout: 1 633992 6.47 MB/sec execute 462 sec latency 16.492 ms 2024-03-23T18:01:18.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:01:17 smithi086 ceph-mon[38221]: pgmap v333: 97 pgs: 97 active+clean; 253 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 267 op/s 2024-03-23T18:01:18.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:01:17 smithi086 ceph-mon[38221]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:01:18.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:01:17 smithi092 ceph-mon[79089]: pgmap v333: 97 pgs: 97 active+clean; 253 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 267 op/s 2024-03-23T18:01:18.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:01:17 smithi092 ceph-mon[79089]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:01:18.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:01:17 smithi152 ceph-mon[47949]: pgmap v333: 97 pgs: 97 active+clean; 253 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 267 op/s 2024-03-23T18:01:18.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:01:17 smithi152 ceph-mon[47949]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:01:18.922 INFO:tasks.workunit.client.0.smithi086.stdout: 1 635952 6.49 MB/sec execute 463 sec latency 14.338 ms 2024-03-23T18:01:19.922 INFO:tasks.workunit.client.0.smithi086.stdout: 1 637202 6.48 MB/sec execute 464 sec latency 14.296 ms 2024-03-23T18:01:20.105 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.105+0000 7f3bd1c00700 1 -- 172.21.15.86:0/341980699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bcc10b2a0 msgr2=0x7f3bcc10d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:01:20.106 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.105+0000 7f3bd1c00700 1 --2- 172.21.15.86:0/341980699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bcc10b2a0 0x7f3bcc10d760 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f3bc4009f90 tx=0x7f3bc400b330 comp rx=0 tx=0).stop 2024-03-23T18:01:20.106 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.105+0000 7f3bd1c00700 1 -- 172.21.15.86:0/341980699 shutdown_connections 2024-03-23T18:01:20.106 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.105+0000 7f3bd1c00700 1 --2- 172.21.15.86:0/341980699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bcc10b2a0 0x7f3bcc10d760 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:20.106 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.105+0000 7f3bd1c00700 1 --2- 172.21.15.86:0/341980699 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3bcc101e80 0x7f3bcc102260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:20.106 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.105+0000 7f3bd1c00700 1 --2- 172.21.15.86:0/341980699 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bcc1027a0 0x7f3bcc10ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:20.106 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.105+0000 7f3bd1c00700 1 -- 172.21.15.86:0/341980699 >> 172.21.15.86:0/341980699 conn(0x7f3bcc0fb680 msgr2=0x7f3bcc0fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:01:20.106 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.105+0000 7f3bd1c00700 1 -- 172.21.15.86:0/341980699 shutdown_connections 2024-03-23T18:01:20.106 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.105+0000 7f3bd1c00700 1 -- 172.21.15.86:0/341980699 wait complete. 2024-03-23T18:01:20.108 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.106+0000 7f3bd1c00700 1 Processor -- start 2024-03-23T18:01:20.108 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.106+0000 7f3bd1c00700 1 -- start start 2024-03-23T18:01:20.108 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.107+0000 7f3bd1c00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bcc101e80 0x7f3bcc0ffad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:20.109 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.107+0000 7f3bd1c00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3bcc1027a0 0x7f3bcc100010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:20.109 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.107+0000 7f3bd1c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bcc10b2a0 0x7f3bcc100570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:20.109 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.107+0000 7f3bd1c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bcc1a56f0 con 0x7f3bcc10b2a0 2024-03-23T18:01:20.109 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.107+0000 7f3bd1c00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bcc1a5860 con 0x7f3bcc101e80 2024-03-23T18:01:20.109 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.107+0000 7f3bd1c00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bcc1a5a20 con 0x7f3bcc1027a0 2024-03-23T18:01:20.109 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.107+0000 7f3bcaa00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bcc101e80 0x7f3bcc0ffad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:20.109 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.107+0000 7f3bcaa00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bcc101e80 0x7f3bcc0ffad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:36586/0 (socket says 172.21.15.86:36586) 2024-03-23T18:01:20.109 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.107+0000 7f3bcaa00700 1 -- 172.21.15.86:0/1495345383 learned_addr learned my addr 172.21.15.86:0/1495345383 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:01:20.109 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.107+0000 7f3bca000700 1 --2- 172.21.15.86:0/1495345383 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3bcc1027a0 0x7f3bcc100010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:20.109 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.107+0000 7f3bcb400700 1 --2- 172.21.15.86:0/1495345383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bcc10b2a0 0x7f3bcc100570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:20.109 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.107+0000 7f3bcaa00700 1 -- 172.21.15.86:0/1495345383 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3bcc1027a0 msgr2=0x7f3bcc100010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:01:20.109 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.107+0000 7f3bcaa00700 1 --2- 172.21.15.86:0/1495345383 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3bcc1027a0 0x7f3bcc100010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.107+0000 7f3bcaa00700 1 -- 172.21.15.86:0/1495345383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bcc10b2a0 msgr2=0x7f3bcc100570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:01:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.107+0000 7f3bcaa00700 1 --2- 172.21.15.86:0/1495345383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bcc10b2a0 0x7f3bcc100570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.107+0000 7f3bcaa00700 1 -- 172.21.15.86:0/1495345383 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bc4009c40 con 0x7f3bcc101e80 2024-03-23T18:01:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.108+0000 7f3bcaa00700 1 --2- 172.21.15.86:0/1495345383 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bcc101e80 0x7f3bcc0ffad0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc00e9a0 tx=0x7f3bbc00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:01:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.108+0000 7f3bc3400700 1 -- 172.21.15.86:0/1495345383 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc009790 con 0x7f3bcc101e80 2024-03-23T18:01:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.108+0000 7f3bc3400700 1 -- 172.21.15.86:0/1495345383 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3bbc010430 con 0x7f3bcc101e80 2024-03-23T18:01:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.108+0000 7f3bd1c00700 1 -- 172.21.15.86:0/1495345383 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bcc1a5d00 con 0x7f3bcc101e80 2024-03-23T18:01:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.108+0000 7f3bc3400700 1 -- 172.21.15.86:0/1495345383 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc018680 con 0x7f3bcc101e80 2024-03-23T18:01:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.108+0000 7f3bd1c00700 1 -- 172.21.15.86:0/1495345383 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bcc1a6200 con 0x7f3bcc101e80 2024-03-23T18:01:20.111 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.111+0000 7f3bc3400700 1 -- 172.21.15.86:0/1495345383 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f3bbc0105a0 con 0x7f3bcc101e80 2024-03-23T18:01:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.113+0000 7f3bc3400700 1 --2- 172.21.15.86:0/1495345383 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f3bb00a1310 0x7f3bb00a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.113+0000 7f3bc3400700 1 -- 172.21.15.86:0/1495345383 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f3bbc014070 con 0x7f3bcc101e80 2024-03-23T18:01:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.113+0000 7f3bca000700 1 --2- 172.21.15.86:0/1495345383 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f3bb00a1310 0x7f3bb00a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.113+0000 7f3be568fb80 1 Processor -- start 2024-03-23T18:01:20.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.114+0000 7f3be568fb80 1 -- start start 2024-03-23T18:01:20.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.114+0000 7f3bca000700 1 --2- 172.21.15.86:0/1495345383 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f3bb00a1310 0x7f3bb00a37d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc101200 tx=0x7f3bb8009310 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:01:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.114+0000 7f3be568fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ed25b43100 0x55ed25b434e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.114+0000 7f3be568fb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ed25b43a20 0x55ed25b4e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:20.117 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.114+0000 7f3be568fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed25b4e8d0 0x55ed25b50cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:20.117 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.114+0000 7f3be568fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed25a9c930 con 0x55ed25b4e8d0 2024-03-23T18:01:20.117 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.114+0000 7f3be568fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed25a9a2f0 con 0x55ed25b43100 2024-03-23T18:01:20.117 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.114+0000 7f3be568fb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed25a95be0 con 0x55ed25b43a20 2024-03-23T18:01:20.117 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.115+0000 7f3bcaa00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ed25b43100 0x55ed25b434e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:20.117 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.115+0000 7f3bcaa00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ed25b43100 0x55ed25b434e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:36588/0 (socket says 172.21.15.86:36588) 2024-03-23T18:01:20.117 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.115+0000 7f3bcaa00700 1 -- 172.21.15.86:0/1639816918 learned_addr learned my addr 172.21.15.86:0/1639816918 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:01:20.117 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.115+0000 7f3bcb400700 1 --2- 172.21.15.86:0/1639816918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed25b4e8d0 0x55ed25b50cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:20.117 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.115+0000 7f3bca000700 1 --2- 172.21.15.86:0/1639816918 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ed25b43a20 0x55ed25b4e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:20.117 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.115+0000 7f3bcaa00700 1 -- 172.21.15.86:0/1639816918 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ed25b43a20 msgr2=0x55ed25b4e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:01:20.117 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.115+0000 7f3bcaa00700 1 --2- 172.21.15.86:0/1639816918 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ed25b43a20 0x55ed25b4e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:20.117 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.115+0000 7f3bcaa00700 1 -- 172.21.15.86:0/1639816918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed25b4e8d0 msgr2=0x55ed25b50cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:01:20.117 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.115+0000 7f3bcaa00700 1 --2- 172.21.15.86:0/1639816918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed25b4e8d0 0x55ed25b50cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:20.118 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.115+0000 7f3bcaa00700 1 -- 172.21.15.86:0/1639816918 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed25917350 con 0x55ed25b43100 2024-03-23T18:01:20.118 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.115+0000 7f3bca000700 1 --2- 172.21.15.86:0/1639816918 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ed25b43a20 0x55ed25b4e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T18:01:20.118 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.115+0000 7f3bcaa00700 1 --2- 172.21.15.86:0/1639816918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ed25b43100 0x55ed25b434e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc096430 tx=0x7f3bbc09a630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1208ca96b635e727 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:01:20.118 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.115+0000 7f3bb7e00700 1 -- 172.21.15.86:0/1639816918 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc0a23b0 con 0x55ed25b43100 2024-03-23T18:01:20.118 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.115+0000 7f3bb7e00700 1 -- 172.21.15.86:0/1639816918 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3bbc010880 con 0x55ed25b43100 2024-03-23T18:01:20.118 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.115+0000 7f3bb7e00700 1 -- 172.21.15.86:0/1639816918 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc0996f0 con 0x55ed25b43100 2024-03-23T18:01:20.118 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.116+0000 7f3be568fb80 1 -- 172.21.15.86:0/1639816918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ed25b43100 msgr2=0x55ed25b434e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:01:20.118 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.116+0000 7f3be568fb80 1 --2- 172.21.15.86:0/1639816918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ed25b43100 0x55ed25b434e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc096430 tx=0x7f3bbc09a630 comp rx=0 tx=0).stop 2024-03-23T18:01:20.118 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.116+0000 7f3be568fb80 1 -- 172.21.15.86:0/1639816918 shutdown_connections 2024-03-23T18:01:20.118 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.116+0000 7f3be568fb80 1 --2- 172.21.15.86:0/1639816918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed25b4e8d0 0x55ed25b50cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:20.118 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.116+0000 7f3be568fb80 1 --2- 172.21.15.86:0/1639816918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ed25b43100 0x55ed25b434e0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:20.118 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.116+0000 7f3be568fb80 1 --2- 172.21.15.86:0/1639816918 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ed25b43a20 0x55ed25b4e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:20.118 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.116+0000 7f3be568fb80 1 -- 172.21.15.86:0/1639816918 >> 172.21.15.86:0/1639816918 conn(0x55ed25aa3640 msgr2=0x55ed25b3f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:01:20.118 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.116+0000 7f3be568fb80 1 -- 172.21.15.86:0/1639816918 shutdown_connections 2024-03-23T18:01:20.119 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.116+0000 7f3be568fb80 1 -- 172.21.15.86:0/1639816918 wait complete. 2024-03-23T18:01:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.117+0000 7f3be568fb80 1 -- start start 2024-03-23T18:01:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.117+0000 7f3be568fb80 1 Processor -- start 2024-03-23T18:01:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.118+0000 7f3be568fb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ed25b43100 0x55ed25b4e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.118+0000 7f3be568fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ed25b43a20 0x55ed25b4b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.118+0000 7f3be568fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed25b4e8d0 0x55ed25b4bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.118+0000 7f3be568fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed25a31d20 con 0x55ed25b4e8d0 2024-03-23T18:01:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.118+0000 7f3be568fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed25a319c0 con 0x55ed25b43a20 2024-03-23T18:01:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.118+0000 7f3be568fb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed25919640 con 0x55ed25b43100 2024-03-23T18:01:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.118+0000 7f3be568fb80 10 client.0 _ll_register_callbacks cb 0x55ed25a3b410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T18:01:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.118+0000 7f3bca000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ed25b43a20 0x55ed25b4b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.118+0000 7f3bcaa00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ed25b43100 0x55ed25b4e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.118+0000 7f3bcaa00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ed25b43100 0x55ed25b4e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:58358/0 (socket says 172.21.15.86:58358) 2024-03-23T18:01:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.118+0000 7f3bcaa00700 1 -- 172.21.15.86:0/857791760 learned_addr learned my addr 172.21.15.86:0/857791760 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:01:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.118+0000 7f3bca000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ed25b43a20 0x55ed25b4b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:36598/0 (socket says 172.21.15.86:36598) 2024-03-23T18:01:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.118+0000 7f3bcb400700 1 --2- 172.21.15.86:0/857791760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed25b4e8d0 0x55ed25b4bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:20.124 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.118+0000 7f3bcaa00700 1 -- 172.21.15.86:0/857791760 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ed25b43a20 msgr2=0x55ed25b4b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:01:20.124 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.118+0000 7f3bcaa00700 1 --2- 172.21.15.86:0/857791760 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ed25b43a20 0x55ed25b4b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:20.124 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.118+0000 7f3bcaa00700 1 -- 172.21.15.86:0/857791760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed25b4e8d0 msgr2=0x55ed25b4bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:01:20.124 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.118+0000 7f3bcaa00700 1 --2- 172.21.15.86:0/857791760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed25b4e8d0 0x55ed25b4bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:20.124 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.119+0000 7f3bcaa00700 1 -- 172.21.15.86:0/857791760 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bcc1a6200 con 0x55ed25b43100 2024-03-23T18:01:20.124 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.119+0000 7f3bca000700 1 --2- 172.21.15.86:0/857791760 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ed25b43a20 0x55ed25b4b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T18:01:20.124 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.119+0000 7f3bcaa00700 1 --2- 172.21.15.86:0/857791760 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ed25b43100 0x55ed25b4e350 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc00e970 tx=0x7f3bbc0996d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9adfbf80c3aec8b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:01:20.124 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.119+0000 7f3bb4c00700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-03-23T18:01:20.124 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.119+0000 7f3bb4c00700 1 -- 172.21.15.86:0/857791760 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc0a23b0 con 0x55ed25b43100 2024-03-23T18:01:20.124 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.119+0000 7f3be568fb80 1 -- 172.21.15.86:0/857791760 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ed25a56320 con 0x55ed25b43100 2024-03-23T18:01:20.124 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.119+0000 7f3bb4c00700 1 -- 172.21.15.86:0/857791760 <== mon.1 v2:172.21.15.152:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f3bbc0982d0 con 0x55ed25b43100 2024-03-23T18:01:20.124 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.119+0000 7f3bb4c00700 1 -- 172.21.15.86:0/857791760 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3bbc0a2130 con 0x55ed25b43100 2024-03-23T18:01:20.124 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.119+0000 7f3be568fb80 10 client.26875 fetch_fsmap learned FSMap version 49 2024-03-23T18:01:20.125 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.119+0000 7f3be568fb80 1 -- 172.21.15.86:0/857791760 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55ed258c3b00 con 0x55ed25b43100 2024-03-23T18:01:20.125 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.120+0000 7f3bb4c00700 1 -- 172.21.15.86:0/857791760 <== mon.1 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3bbc07d080 con 0x55ed25b43100 2024-03-23T18:01:20.125 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.120+0000 7f3bb4c00700 1 -- 172.21.15.86:0/857791760 <== mon.1 v2:172.21.15.152:3300/0 5 ==== fsmap(e 49) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f3bbc0b2be0 con 0x55ed25b43100 2024-03-23T18:01:20.125 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.120+0000 7f3be568fb80 10 client.26875 fetch_fsmap finished waiting for FSMap version 49 2024-03-23T18:01:20.125 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.120+0000 7f3be568fb80 10 client.26875 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T18:01:20.125 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.120+0000 7f3be568fb80 20 client.26875 populate_metadata read hostname 'smithi086' 2024-03-23T18:01:20.125 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.120+0000 7f3be568fb80 1 --2- 172.21.15.86:0/857791760 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55ed25b5e240 0x55ed25b60660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:20.125 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.120+0000 7f3be568fb80 4 client.26875 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T18:01:20.126 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.120+0000 7f3be568fb80 1 -- 172.21.15.86:0/857791760 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ed25a8e6a0 con 0x55ed25b5e240 2024-03-23T18:01:20.126 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.120+0000 7f3bca000700 1 --2- 172.21.15.86:0/857791760 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55ed25b5e240 0x55ed25b60660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:20.126 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.121+0000 7f3bca000700 1 --2- 172.21.15.86:0/857791760 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55ed25b5e240 0x55ed25b60660 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed7609258ad09b0d server_cookie=bb7e90e4f6bddeb5 in_seq=0 out_seq=0 2024-03-23T18:01:20.126 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.121+0000 7f3bb4c00700 10 client.26875 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T18:01:20.126 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.121+0000 7f3bb4c00700 1 -- 172.21.15.86:0/857791760 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55ed25a8e6a0 con 0x55ed25b5e240 2024-03-23T18:01:20.126 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.121+0000 7f3bb4c00700 10 client.26875 handle_command_reply: tid=0 2024-03-23T18:01:20.126 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.121+0000 7f3be568fb80 1 client.26875 shutdown 2024-03-23T18:01:20.126 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.121+0000 7f3be568fb80 1 -- 172.21.15.86:0/857791760 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ed25b43100 msgr2=0x55ed25b4e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:01:20.126 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.121+0000 7f3be568fb80 1 --2- 172.21.15.86:0/857791760 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ed25b43100 0x55ed25b4e350 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc00e970 tx=0x7f3bbc0996d0 comp rx=0 tx=0).stop 2024-03-23T18:01:20.126 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.122+0000 7f3be568fb80 1 -- 172.21.15.86:0/857791760 shutdown_connections 2024-03-23T18:01:20.126 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.122+0000 7f3be568fb80 1 --2- 172.21.15.86:0/857791760 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55ed25b5e240 0x55ed25b60660 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:20.126 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.122+0000 7f3be568fb80 1 --2- 172.21.15.86:0/857791760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed25b4e8d0 0x55ed25b4bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:20.126 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.122+0000 7f3be568fb80 1 --2- 172.21.15.86:0/857791760 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ed25b43100 0x55ed25b4e350 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:20.127 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.122+0000 7f3bb4c00700 0 client.26875 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T18:01:20.127 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.122+0000 7f3be568fb80 1 --2- 172.21.15.86:0/857791760 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ed25b43a20 0x55ed25b4b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:20.127 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.122+0000 7f3be568fb80 1 -- 172.21.15.86:0/857791760 >> 172.21.15.86:0/857791760 conn(0x55ed25aa3640 msgr2=0x55ed25b45080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:01:20.127 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.122+0000 7f3be568fb80 1 -- 172.21.15.86:0/857791760 shutdown_connections 2024-03-23T18:01:20.127 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.122+0000 7f3be568fb80 1 -- 172.21.15.86:0/857791760 wait complete. 2024-03-23T18:01:20.127 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.122+0000 7f3be568fb80 20 client.26875 trim_cache size 0 max 16384 2024-03-23T18:01:20.140 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.138+0000 7f3be568fb80 1 Processor -- start 2024-03-23T18:01:20.140 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.138+0000 7f3be568fb80 1 -- start start 2024-03-23T18:01:20.140 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.139+0000 7f3be568fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ed25b43100 0x55ed25b555a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:20.140 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.139+0000 7f3be568fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed25b43a20 0x55ed25b39500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:20.140 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.139+0000 7f3be568fb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ed25b4e8d0 0x55ed25b45750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:20.140 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.139+0000 7f3be568fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed25b653d0 con 0x55ed25b43a20 2024-03-23T18:01:20.140 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.139+0000 7f3be568fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed25b45c90 con 0x55ed25b43100 2024-03-23T18:01:20.140 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.139+0000 7f3be568fb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed25b49ca0 con 0x55ed25b4e8d0 2024-03-23T18:01:20.140 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.139+0000 7f3bcaa00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ed25b43100 0x55ed25b555a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:20.141 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.139+0000 7f3bcaa00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ed25b43100 0x55ed25b555a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:36606/0 (socket says 172.21.15.86:36606) 2024-03-23T18:01:20.141 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.139+0000 7f3bcaa00700 1 -- 172.21.15.86:0/3082582414 learned_addr learned my addr 172.21.15.86:0/3082582414 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:01:20.141 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.139+0000 7f3bcb400700 1 --2- 172.21.15.86:0/3082582414 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ed25b4e8d0 0x55ed25b45750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:20.141 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.139+0000 7f3bca000700 1 --2- 172.21.15.86:0/3082582414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed25b43a20 0x55ed25b39500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:20.141 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.139+0000 7f3bcaa00700 1 -- 172.21.15.86:0/3082582414 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ed25b4e8d0 msgr2=0x55ed25b45750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:01:20.141 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.139+0000 7f3bcaa00700 1 --2- 172.21.15.86:0/3082582414 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ed25b4e8d0 0x55ed25b45750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:20.141 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.139+0000 7f3bcaa00700 1 -- 172.21.15.86:0/3082582414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed25b43a20 msgr2=0x55ed25b39500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:01:20.141 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.139+0000 7f3bcaa00700 1 --2- 172.21.15.86:0/3082582414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed25b43a20 0x55ed25b39500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:20.141 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.139+0000 7f3bcaa00700 1 -- 172.21.15.86:0/3082582414 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed25a5be40 con 0x55ed25b43100 2024-03-23T18:01:20.141 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.139+0000 7f3bca000700 1 --2- 172.21.15.86:0/3082582414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed25b43a20 0x55ed25b39500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T18:01:20.141 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.140+0000 7f3bcaa00700 1 --2- 172.21.15.86:0/3082582414 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ed25b43100 0x55ed25b555a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc096430 tx=0x7f3bbc099ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24e8b19c3e26d440 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:01:20.141 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.140+0000 7f3bc0c00700 1 -- 172.21.15.86:0/3082582414 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc0109f0 con 0x55ed25b43100 2024-03-23T18:01:20.141 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.140+0000 7f3bc0c00700 1 -- 172.21.15.86:0/3082582414 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ed25a56320 con 0x55ed25b43100 2024-03-23T18:01:20.142 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.140+0000 7f3bc0c00700 1 -- 172.21.15.86:0/3082582414 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc0a23b0 con 0x55ed25b43100 2024-03-23T18:01:20.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.140+0000 7f3be568fb80 1 -- 172.21.15.86:0/3082582414 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ed25b43100 msgr2=0x55ed25b555a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:01:20.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.140+0000 7f3be568fb80 1 --2- 172.21.15.86:0/3082582414 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ed25b43100 0x55ed25b555a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc096430 tx=0x7f3bbc099ba0 comp rx=0 tx=0).stop 2024-03-23T18:01:20.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.141+0000 7f3be568fb80 1 -- 172.21.15.86:0/3082582414 shutdown_connections 2024-03-23T18:01:20.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.141+0000 7f3be568fb80 1 --2- 172.21.15.86:0/3082582414 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ed25b4e8d0 0x55ed25b45750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:20.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.141+0000 7f3be568fb80 1 --2- 172.21.15.86:0/3082582414 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ed25b43100 0x55ed25b555a0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:20.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.141+0000 7f3be568fb80 1 --2- 172.21.15.86:0/3082582414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed25b43a20 0x55ed25b39500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:20.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.141+0000 7f3be568fb80 1 -- 172.21.15.86:0/3082582414 >> 172.21.15.86:0/3082582414 conn(0x55ed25aa3640 msgr2=0x55ed25b45040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:01:20.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.141+0000 7f3be568fb80 1 -- 172.21.15.86:0/3082582414 shutdown_connections 2024-03-23T18:01:20.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.141+0000 7f3be568fb80 1 -- 172.21.15.86:0/3082582414 wait complete. 2024-03-23T18:01:20.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.141+0000 7f3be568fb80 1 -- start start 2024-03-23T18:01:20.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.141+0000 7f3be568fb80 1 Processor -- start 2024-03-23T18:01:20.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.142+0000 7f3be568fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed25b43100 0x55ed25b53580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:20.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.142+0000 7f3be568fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ed25b43a20 0x55ed25b53ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:20.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.142+0000 7f3be568fb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ed25b4e8d0 0x55ed25b54000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:20.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.142+0000 7f3be568fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed25baed60 con 0x55ed25b43100 2024-03-23T18:01:20.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.142+0000 7f3be568fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed25baeea0 con 0x55ed25b43a20 2024-03-23T18:01:20.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.142+0000 7f3be568fb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed25baf060 con 0x55ed25b4e8d0 2024-03-23T18:01:20.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.142+0000 7f3be568fb80 10 client.0 _ll_register_callbacks cb 0x55ed25a3b410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T18:01:20.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.142+0000 7f3bca000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ed25b43a20 0x55ed25b53ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:20.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.142+0000 7f3bcaa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed25b43100 0x55ed25b53580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:20.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.142+0000 7f3bca000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ed25b43a20 0x55ed25b53ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:36618/0 (socket says 172.21.15.86:36618) 2024-03-23T18:01:20.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.142+0000 7f3bca000700 1 -- 172.21.15.86:0/1266116359 learned_addr learned my addr 172.21.15.86:0/1266116359 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:01:20.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.142+0000 7f3bcb400700 1 --2- 172.21.15.86:0/1266116359 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ed25b4e8d0 0x55ed25b54000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:20.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.143+0000 7f3bca000700 1 -- 172.21.15.86:0/1266116359 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ed25b4e8d0 msgr2=0x55ed25b54000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:01:20.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.143+0000 7f3bca000700 1 --2- 172.21.15.86:0/1266116359 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ed25b4e8d0 0x55ed25b54000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:20.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.143+0000 7f3bca000700 1 -- 172.21.15.86:0/1266116359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed25b43100 msgr2=0x55ed25b53580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:01:20.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.143+0000 7f3bca000700 1 --2- 172.21.15.86:0/1266116359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed25b43100 0x55ed25b53580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:20.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.143+0000 7f3bca000700 1 -- 172.21.15.86:0/1266116359 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ed258c3b00 con 0x55ed25b43a20 2024-03-23T18:01:20.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.143+0000 7f3bca000700 1 --2- 172.21.15.86:0/1266116359 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ed25b43a20 0x55ed25b53ac0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f3bb8011920 tx=0x7f3bb80047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a70650fd034798a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:01:20.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.143+0000 7f3bb4c00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T18:01:20.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.143+0000 7f3bb4c00700 1 -- 172.21.15.86:0/1266116359 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb802f070 con 0x55ed25b43a20 2024-03-23T18:01:20.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.143+0000 7f3be568fb80 1 -- 172.21.15.86:0/1266116359 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3bbc010880 con 0x55ed25b43a20 2024-03-23T18:01:20.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.143+0000 7f3bb4c00700 1 -- 172.21.15.86:0/1266116359 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f3bb8004830 con 0x55ed25b43a20 2024-03-23T18:01:20.148 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.143+0000 7f3bb4c00700 1 -- 172.21.15.86:0/1266116359 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3bb8024040 con 0x55ed25b43a20 2024-03-23T18:01:20.148 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.143+0000 7f3be568fb80 10 client.26900 fetch_fsmap learned FSMap version 49 2024-03-23T18:01:20.148 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.143+0000 7f3be568fb80 1 -- 172.21.15.86:0/1266116359 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55ed2591bc70 con 0x55ed25b43a20 2024-03-23T18:01:20.148 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.144+0000 7f3bb4c00700 1 -- 172.21.15.86:0/1266116359 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3bbc010880 con 0x55ed25b43a20 2024-03-23T18:01:20.148 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.144+0000 7f3bb4c00700 1 -- 172.21.15.86:0/1266116359 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 49) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f3bb8032020 con 0x55ed25b43a20 2024-03-23T18:01:20.148 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.144+0000 7f3be568fb80 10 client.26900 fetch_fsmap finished waiting for FSMap version 49 2024-03-23T18:01:20.148 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.144+0000 7f3be568fb80 10 client.26900 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T18:01:20.148 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.144+0000 7f3be568fb80 20 client.26900 populate_metadata read hostname 'smithi086' 2024-03-23T18:01:20.148 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.144+0000 7f3be568fb80 1 --2- 172.21.15.86:0/1266116359 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55ed25bb0950 0x55ed25bb2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:20.148 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.144+0000 7f3be568fb80 4 client.26900 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T18:01:20.148 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.144+0000 7f3be568fb80 1 -- 172.21.15.86:0/1266116359 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ed258e4c90 con 0x55ed25bb0950 2024-03-23T18:01:20.148 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.144+0000 7f3bcaa00700 1 --2- 172.21.15.86:0/1266116359 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55ed25bb0950 0x55ed25bb2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:20.148 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.145+0000 7f3bb4c00700 10 client.26900 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T18:01:20.149 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.145+0000 7f3bcaa00700 1 --2- 172.21.15.86:0/1266116359 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55ed25bb0950 0x55ed25bb2d70 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9acd557bd21b942d server_cookie=3459358dc831de0b in_seq=0 out_seq=0 2024-03-23T18:01:20.193 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.193+0000 7f3bb4c00700 1 -- 172.21.15.86:0/1266116359 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ed258e4c90 con 0x55ed25bb0950 2024-03-23T18:01:20.193 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.193+0000 7f3bb4c00700 10 client.26900 handle_command_reply: tid=0 2024-03-23T18:01:20.194 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.193+0000 7f3be568fb80 1 client.26900 shutdown 2024-03-23T18:01:20.195 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.194+0000 7f3be568fb80 1 -- 172.21.15.86:0/1266116359 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ed25b43a20 msgr2=0x55ed25b53ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:01:20.195 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.194+0000 7f3be568fb80 1 --2- 172.21.15.86:0/1266116359 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ed25b43a20 0x55ed25b53ac0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f3bb8011920 tx=0x7f3bb80047c0 comp rx=0 tx=0).stop 2024-03-23T18:01:20.195 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.194+0000 7f3be568fb80 1 -- 172.21.15.86:0/1266116359 shutdown_connections 2024-03-23T18:01:20.195 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.194+0000 7f3be568fb80 1 --2- 172.21.15.86:0/1266116359 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55ed25bb0950 0x55ed25bb2d70 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:20.195 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.194+0000 7f3be568fb80 1 --2- 172.21.15.86:0/1266116359 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ed25b4e8d0 0x55ed25b54000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:20.195 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.194+0000 7f3be568fb80 1 --2- 172.21.15.86:0/1266116359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed25b43100 0x55ed25b53580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:20.195 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.194+0000 7f3be568fb80 1 --2- 172.21.15.86:0/1266116359 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ed25b43a20 0x55ed25b53ac0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:20.195 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.194+0000 7f3bb4c00700 0 client.26900 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T18:01:20.195 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.194+0000 7f3be568fb80 1 -- 172.21.15.86:0/1266116359 >> 172.21.15.86:0/1266116359 conn(0x55ed25aa3640 msgr2=0x55ed25b3ed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:01:20.196 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.194+0000 7f3be568fb80 1 -- 172.21.15.86:0/1266116359 shutdown_connections 2024-03-23T18:01:20.196 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.194+0000 7f3be568fb80 1 -- 172.21.15.86:0/1266116359 wait complete. 2024-03-23T18:01:20.196 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.195+0000 7f3be568fb80 20 client.26900 trim_cache size 0 max 16384 2024-03-23T18:01:20.196 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-03-23T18:01:20.196 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-03-23T18:01:20.196 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-03-23T18:01:20.196 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-03-23T18:01:20.196 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.196+0000 7f3bd1c00700 1 -- 172.21.15.86:0/1495345383 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f3bb00a1310 msgr2=0x7f3bb00a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:01:20.197 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.196+0000 7f3bd1c00700 1 --2- 172.21.15.86:0/1495345383 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f3bb00a1310 0x7f3bb00a37d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc101200 tx=0x7f3bb8009310 comp rx=0 tx=0).stop 2024-03-23T18:01:20.197 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.196+0000 7f3bd1c00700 1 -- 172.21.15.86:0/1495345383 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bcc101e80 msgr2=0x7f3bcc0ffad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:01:20.197 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.196+0000 7f3bd1c00700 1 --2- 172.21.15.86:0/1495345383 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bcc101e80 0x7f3bcc0ffad0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc00e9a0 tx=0x7f3bbc00ecb0 comp rx=0 tx=0).stop 2024-03-23T18:01:20.197 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.196+0000 7f3bd1c00700 1 -- 172.21.15.86:0/1495345383 shutdown_connections 2024-03-23T18:01:20.197 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.196+0000 7f3bd1c00700 1 --2- 172.21.15.86:0/1495345383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bcc10b2a0 0x7f3bcc100570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:20.197 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.196+0000 7f3bd1c00700 1 --2- 172.21.15.86:0/1495345383 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bcc101e80 0x7f3bcc0ffad0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:20.197 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.196+0000 7f3bd1c00700 1 --2- 172.21.15.86:0/1495345383 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f3bb00a1310 0x7f3bb00a37d0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:20.197 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.196+0000 7f3bd1c00700 1 --2- 172.21.15.86:0/1495345383 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3bcc1027a0 0x7f3bcc100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:20.197 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.196+0000 7f3bd1c00700 1 -- 172.21.15.86:0/1495345383 >> 172.21.15.86:0/1495345383 conn(0x7f3bcc0fb680 msgr2=0x7f3bcc0fc390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:01:20.197 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.196+0000 7f3bd1c00700 1 -- 172.21.15.86:0/1495345383 shutdown_connections 2024-03-23T18:01:20.198 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:20.196+0000 7f3bd1c00700 1 -- 172.21.15.86:0/1495345383 wait complete. 2024-03-23T18:01:20.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:01:19 smithi086 ceph-mon[38221]: pgmap v334: 97 pgs: 97 active+clean; 253 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 267 op/s 2024-03-23T18:01:20.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:01:19 smithi092 ceph-mon[79089]: pgmap v334: 97 pgs: 97 active+clean; 253 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 267 op/s 2024-03-23T18:01:20.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:01:19 smithi152 ceph-mon[47949]: pgmap v334: 97 pgs: 97 active+clean; 253 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 267 op/s 2024-03-23T18:01:20.922 INFO:tasks.workunit.client.0.smithi086.stdout: 1 638881 6.49 MB/sec execute 465 sec latency 48.908 ms 2024-03-23T18:01:21.922 INFO:tasks.workunit.client.0.smithi086.stdout: 1 639989 6.49 MB/sec execute 466 sec latency 21.763 ms 2024-03-23T18:01:22.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:01:21 smithi086 ceph-mon[38221]: pgmap v335: 97 pgs: 97 active+clean; 260 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 253 op/s 2024-03-23T18:01:22.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:01:21 smithi092 ceph-mon[79089]: pgmap v335: 97 pgs: 97 active+clean; 260 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 253 op/s 2024-03-23T18:01:22.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:01:21 smithi152 ceph-mon[47949]: pgmap v335: 97 pgs: 97 active+clean; 260 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 253 op/s 2024-03-23T18:01:22.922 INFO:tasks.workunit.client.0.smithi086.stdout: 1 641724 6.49 MB/sec execute 467 sec latency 17.872 ms 2024-03-23T18:01:23.922 INFO:tasks.workunit.client.0.smithi086.stdout: 1 643280 6.50 MB/sec execute 468 sec latency 11.906 ms 2024-03-23T18:01:24.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:01:23 smithi086 ceph-mon[38221]: pgmap v336: 97 pgs: 97 active+clean; 260 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 253 op/s 2024-03-23T18:01:24.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:01:23 smithi092 ceph-mon[79089]: pgmap v336: 97 pgs: 97 active+clean; 260 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 253 op/s 2024-03-23T18:01:24.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:01:23 smithi152 ceph-mon[47949]: pgmap v336: 97 pgs: 97 active+clean; 260 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 253 op/s 2024-03-23T18:01:24.922 INFO:tasks.workunit.client.0.smithi086.stdout: 1 645189 6.50 MB/sec execute 469 sec latency 14.999 ms 2024-03-23T18:01:25.922 INFO:tasks.workunit.client.0.smithi086.stdout: 1 646737 6.51 MB/sec execute 470 sec latency 33.671 ms 2024-03-23T18:01:26.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:01:25 smithi086 ceph-mon[38221]: pgmap v337: 97 pgs: 97 active+clean; 255 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.2 MiB/s wr, 280 op/s 2024-03-23T18:01:26.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:01:25 smithi092 ceph-mon[79089]: pgmap v337: 97 pgs: 97 active+clean; 255 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.2 MiB/s wr, 280 op/s 2024-03-23T18:01:26.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:01:25 smithi152 ceph-mon[47949]: pgmap v337: 97 pgs: 97 active+clean; 255 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.2 MiB/s wr, 280 op/s 2024-03-23T18:01:26.923 INFO:tasks.workunit.client.0.smithi086.stdout: 1 648240 6.51 MB/sec execute 471 sec latency 15.880 ms 2024-03-23T18:01:27.922 INFO:tasks.workunit.client.0.smithi086.stdout: 1 650187 6.53 MB/sec execute 472 sec latency 13.359 ms 2024-03-23T18:01:28.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:01:27 smithi086 ceph-mon[38221]: pgmap v338: 97 pgs: 97 active+clean; 255 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 261 op/s 2024-03-23T18:01:28.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:01:27 smithi092 ceph-mon[79089]: pgmap v338: 97 pgs: 97 active+clean; 255 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 261 op/s 2024-03-23T18:01:28.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:01:27 smithi152 ceph-mon[47949]: pgmap v338: 97 pgs: 97 active+clean; 255 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 261 op/s 2024-03-23T18:01:28.923 INFO:tasks.workunit.client.0.smithi086.stdout: 1 651540 6.52 MB/sec execute 473 sec latency 14.918 ms 2024-03-23T18:01:29.923 INFO:tasks.workunit.client.0.smithi086.stdout: 1 653636 6.54 MB/sec execute 474 sec latency 15.527 ms 2024-03-23T18:01:30.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:01:29 smithi086 ceph-mon[38221]: pgmap v339: 97 pgs: 97 active+clean; 255 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 233 op/s 2024-03-23T18:01:30.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:01:29 smithi092 ceph-mon[79089]: pgmap v339: 97 pgs: 97 active+clean; 255 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 233 op/s 2024-03-23T18:01:30.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:01:29 smithi152 ceph-mon[47949]: pgmap v339: 97 pgs: 97 active+clean; 255 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 233 op/s 2024-03-23T18:01:30.923 INFO:tasks.workunit.client.0.smithi086.stdout: 1 654692 6.53 MB/sec execute 475 sec latency 44.280 ms 2024-03-23T18:01:31.923 INFO:tasks.workunit.client.0.smithi086.stdout: 1 655132 6.52 MB/sec execute 476 sec latency 750.872 ms 2024-03-23T18:01:32.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:01:31 smithi086 ceph-mon[38221]: pgmap v340: 97 pgs: 97 active+clean; 263 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.1 MiB/s wr, 339 op/s 2024-03-23T18:01:32.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:01:31 smithi086 ceph-mon[38221]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:01:32.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:01:31 smithi092 ceph-mon[79089]: pgmap v340: 97 pgs: 97 active+clean; 263 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.1 MiB/s wr, 339 op/s 2024-03-23T18:01:32.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:01:31 smithi092 ceph-mon[79089]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:01:32.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:01:31 smithi152 ceph-mon[47949]: pgmap v340: 97 pgs: 97 active+clean; 263 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.1 MiB/s wr, 339 op/s 2024-03-23T18:01:32.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:01:31 smithi152 ceph-mon[47949]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:01:32.923 INFO:tasks.workunit.client.0.smithi086.stdout: 1 655132 6.51 MB/sec execute 477 sec latency 1750.992 ms 2024-03-23T18:01:33.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:01:32 smithi086 ceph-mon[38221]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:01:33.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:01:32 smithi092 ceph-mon[79089]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:01:33.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:01:32 smithi152 ceph-mon[47949]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:01:33.355 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.354+0000 7f7f56000700 1 -- 172.21.15.86:0/1355487867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f501018c0 msgr2=0x7f7f50105900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:01:33.355 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.354+0000 7f7f56000700 1 --2- 172.21.15.86:0/1355487867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f501018c0 0x7f7f50105900 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f7f38009990 tx=0x7f7f38009ca0 comp rx=0 tx=0).stop 2024-03-23T18:01:33.355 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.354+0000 7f7f56000700 1 -- 172.21.15.86:0/1355487867 shutdown_connections 2024-03-23T18:01:33.356 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.354+0000 7f7f56000700 1 --2- 172.21.15.86:0/1355487867 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f50106030 0x7f7f5010dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:33.356 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.354+0000 7f7f56000700 1 --2- 172.21.15.86:0/1355487867 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7f50100f10 0x7f7f501012f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:33.356 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.354+0000 7f7f56000700 1 --2- 172.21.15.86:0/1355487867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f501018c0 0x7f7f50105900 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:33.356 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.354+0000 7f7f56000700 1 -- 172.21.15.86:0/1355487867 >> 172.21.15.86:0/1355487867 conn(0x7f7f500fc7b0 msgr2=0x7f7f500febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:01:33.356 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.354+0000 7f7f56000700 1 -- 172.21.15.86:0/1355487867 shutdown_connections 2024-03-23T18:01:33.356 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.354+0000 7f7f56000700 1 -- 172.21.15.86:0/1355487867 wait complete. 2024-03-23T18:01:33.359 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.355+0000 7f7f56000700 1 Processor -- start 2024-03-23T18:01:33.359 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.355+0000 7f7f56000700 1 -- start start 2024-03-23T18:01:33.359 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.356+0000 7f7f56000700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7f50100f10 0x7f7f501971d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:33.359 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.356+0000 7f7f56000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f50106030 0x7f7f50197710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:33.359 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.356+0000 7f7f56000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f5019bab0 0x7f7f5019bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:33.359 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.356+0000 7f7f56000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5019c530 con 0x7f7f5019bab0 2024-03-23T18:01:33.359 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.356+0000 7f7f56000700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5019c6a0 con 0x7f7f50106030 2024-03-23T18:01:33.359 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.356+0000 7f7f56000700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5019c860 con 0x7f7f50100f10 2024-03-23T18:01:33.360 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.356+0000 7f7f4fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f5019bab0 0x7f7f5019bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:33.360 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.356+0000 7f7f4f400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7f50100f10 0x7f7f501971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:33.360 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.356+0000 7f7f4fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f5019bab0 0x7f7f5019bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38752/0 (socket says 172.21.15.86:38752) 2024-03-23T18:01:33.360 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.356+0000 7f7f4fe00700 1 -- 172.21.15.86:0/4074890678 learned_addr learned my addr 172.21.15.86:0/4074890678 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:01:33.360 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.356+0000 7f7f4f400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7f50100f10 0x7f7f501971d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:56226/0 (socket says 172.21.15.86:56226) 2024-03-23T18:01:33.360 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.356+0000 7f7f47e00700 1 --2- 172.21.15.86:0/4074890678 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f50106030 0x7f7f50197710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:33.360 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.356+0000 7f7f4f400700 1 -- 172.21.15.86:0/4074890678 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f50106030 msgr2=0x7f7f50197710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:01:33.360 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.356+0000 7f7f4f400700 1 --2- 172.21.15.86:0/4074890678 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f50106030 0x7f7f50197710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:33.360 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.356+0000 7f7f4f400700 1 -- 172.21.15.86:0/4074890678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f5019bab0 msgr2=0x7f7f5019bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:01:33.360 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.356+0000 7f7f4f400700 1 --2- 172.21.15.86:0/4074890678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f5019bab0 0x7f7f5019bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:33.360 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.356+0000 7f7f4f400700 1 -- 172.21.15.86:0/4074890678 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f380095f0 con 0x7f7f50100f10 2024-03-23T18:01:33.360 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.356+0000 7f7f4fe00700 1 --2- 172.21.15.86:0/4074890678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f5019bab0 0x7f7f5019bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T18:01:33.360 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.356+0000 7f7f47e00700 1 --2- 172.21.15.86:0/4074890678 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f50106030 0x7f7f50197710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T18:01:33.361 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.357+0000 7f7f4f400700 1 --2- 172.21.15.86:0/4074890678 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7f50100f10 0x7f7f501971d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7f4000e9f0 tx=0x7f7f4000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:01:33.361 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.357+0000 7f7f4cc00700 1 -- 172.21.15.86:0/4074890678 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f4000c9c0 con 0x7f7f50100f10 2024-03-23T18:01:33.361 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.357+0000 7f7f56000700 1 -- 172.21.15.86:0/4074890678 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f501a3dc0 con 0x7f7f50100f10 2024-03-23T18:01:33.361 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.357+0000 7f7f4cc00700 1 -- 172.21.15.86:0/4074890678 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f40010430 con 0x7f7f50100f10 2024-03-23T18:01:33.361 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.357+0000 7f7f4cc00700 1 -- 172.21.15.86:0/4074890678 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f40018680 con 0x7f7f50100f10 2024-03-23T18:01:33.361 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.357+0000 7f7f56000700 1 -- 172.21.15.86:0/4074890678 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f501a41f0 con 0x7f7f50100f10 2024-03-23T18:01:33.361 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.360+0000 7f7f4cc00700 1 -- 172.21.15.86:0/4074890678 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f7f40010860 con 0x7f7f50100f10 2024-03-23T18:01:33.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.361+0000 7f7f69ad7b80 1 Processor -- start 2024-03-23T18:01:33.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.362+0000 7f7f4cc00700 1 --2- 172.21.15.86:0/4074890678 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f7f340a1290 0x7f7f340a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:33.363 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.362+0000 7f7f69ad7b80 1 -- start start 2024-03-23T18:01:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.362+0000 7f7f4cc00700 1 -- 172.21.15.86:0/4074890678 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f7f40014070 con 0x7f7f50100f10 2024-03-23T18:01:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.362+0000 7f7f4fe00700 1 --2- 172.21.15.86:0/4074890678 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f7f340a1290 0x7f7f340a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.362+0000 7f7f69ad7b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626c7db7100 0x5626c7db74e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:33.365 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.362+0000 7f7f69ad7b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5626c7db7a20 0x5626c7dc2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:33.365 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.362+0000 7f7f69ad7b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626c7dc28d0 0x5626c7dc4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:33.365 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.362+0000 7f7f69ad7b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626c7d10930 con 0x5626c7dc28d0 2024-03-23T18:01:33.365 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.362+0000 7f7f69ad7b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626c7d0e2f0 con 0x5626c7db7100 2024-03-23T18:01:33.365 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.362+0000 7f7f69ad7b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626c7d09be0 con 0x5626c7db7a20 2024-03-23T18:01:33.365 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.363+0000 7f7f4f400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626c7db7100 0x5626c7db74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:33.365 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.363+0000 7f7f47e00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5626c7db7a20 0x5626c7dc2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:33.365 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.363+0000 7f7f4f400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626c7db7100 0x5626c7db74e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:52298/0 (socket says 172.21.15.86:52298) 2024-03-23T18:01:33.365 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.363+0000 7f7f4f400700 1 -- 172.21.15.86:0/1791130789 learned_addr learned my addr 172.21.15.86:0/1791130789 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:01:33.365 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.363+0000 7f7f4fe00700 1 --2- 172.21.15.86:0/1791130789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626c7dc28d0 0x5626c7dc4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:33.365 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.363+0000 7f7f47e00700 1 -- 172.21.15.86:0/1791130789 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626c7db7100 msgr2=0x5626c7db74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:01:33.365 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.363+0000 7f7f47e00700 1 --2- 172.21.15.86:0/1791130789 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626c7db7100 0x5626c7db74e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:33.365 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.363+0000 7f7f47e00700 1 -- 172.21.15.86:0/1791130789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626c7dc28d0 msgr2=0x5626c7dc4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:01:33.365 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.363+0000 7f7f47e00700 1 --2- 172.21.15.86:0/1791130789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626c7dc28d0 0x5626c7dc4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:33.366 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.363+0000 7f7f47e00700 1 -- 172.21.15.86:0/1791130789 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626c7b8b350 con 0x5626c7db7a20 2024-03-23T18:01:33.366 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.363+0000 7f7f4fe00700 1 --2- 172.21.15.86:0/4074890678 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f7f340a1290 0x7f7f340a3750 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7f48009c10 tx=0x7f7f48009920 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:01:33.366 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.363+0000 7f7f4f400700 1 --2- 172.21.15.86:0/1791130789 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626c7db7100 0x5626c7db74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T18:01:33.366 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.363+0000 7f7f47e00700 1 --2- 172.21.15.86:0/1791130789 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5626c7db7a20 0x5626c7dc2390 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7f380098e0 tx=0x7f7f38005f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4a58626cb82374c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:01:33.366 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.363+0000 7f7f3fe00700 1 -- 172.21.15.86:0/1791130789 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3801d070 con 0x5626c7db7a20 2024-03-23T18:01:33.366 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.364+0000 7f7f3fe00700 1 -- 172.21.15.86:0/1791130789 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f3801e040 con 0x5626c7db7a20 2024-03-23T18:01:33.366 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.364+0000 7f7f3fe00700 1 -- 172.21.15.86:0/1791130789 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f38004ae0 con 0x5626c7db7a20 2024-03-23T18:01:33.370 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.364+0000 7f7f69ad7b80 1 -- 172.21.15.86:0/1791130789 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5626c7db7a20 msgr2=0x5626c7dc2390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:01:33.371 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.364+0000 7f7f69ad7b80 1 --2- 172.21.15.86:0/1791130789 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5626c7db7a20 0x5626c7dc2390 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7f380098e0 tx=0x7f7f38005f00 comp rx=0 tx=0).stop 2024-03-23T18:01:33.371 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.364+0000 7f7f69ad7b80 1 -- 172.21.15.86:0/1791130789 shutdown_connections 2024-03-23T18:01:33.371 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.364+0000 7f7f69ad7b80 1 --2- 172.21.15.86:0/1791130789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626c7dc28d0 0x5626c7dc4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:33.371 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.364+0000 7f7f69ad7b80 1 --2- 172.21.15.86:0/1791130789 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626c7db7100 0x5626c7db74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:33.371 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.364+0000 7f7f69ad7b80 1 --2- 172.21.15.86:0/1791130789 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5626c7db7a20 0x5626c7dc2390 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:33.371 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.364+0000 7f7f69ad7b80 1 -- 172.21.15.86:0/1791130789 >> 172.21.15.86:0/1791130789 conn(0x5626c7d17640 msgr2=0x5626c7db3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:01:33.371 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.364+0000 7f7f69ad7b80 1 -- 172.21.15.86:0/1791130789 shutdown_connections 2024-03-23T18:01:33.371 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.364+0000 7f7f69ad7b80 1 -- 172.21.15.86:0/1791130789 wait complete. 2024-03-23T18:01:33.371 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.365+0000 7f7f69ad7b80 1 -- start start 2024-03-23T18:01:33.371 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.365+0000 7f7f69ad7b80 1 Processor -- start 2024-03-23T18:01:33.371 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.366+0000 7f7f69ad7b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626c7db7100 0x5626c7dc2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:33.371 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.366+0000 7f7f69ad7b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626c7db7a20 0x5626c7dbf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:33.372 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.366+0000 7f7f69ad7b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5626c7dc28d0 0x5626c7dbfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:33.372 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.366+0000 7f7f69ad7b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626c7ca5d20 con 0x5626c7db7a20 2024-03-23T18:01:33.372 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.366+0000 7f7f69ad7b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626c7ca59c0 con 0x5626c7db7100 2024-03-23T18:01:33.372 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.366+0000 7f7f69ad7b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626c7b8d640 con 0x5626c7dc28d0 2024-03-23T18:01:33.372 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.366+0000 7f7f69ad7b80 10 client.0 _ll_register_callbacks cb 0x5626c7caf410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T18:01:33.372 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.366+0000 7f7f47e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626c7db7a20 0x5626c7dbf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:33.372 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.366+0000 7f7f47e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626c7db7a20 0x5626c7dbf610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38778/0 (socket says 172.21.15.86:38778) 2024-03-23T18:01:33.372 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.366+0000 7f7f47e00700 1 -- 172.21.15.86:0/3444844088 learned_addr learned my addr 172.21.15.86:0/3444844088 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:01:33.372 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.366+0000 7f7f4f400700 1 --2- 172.21.15.86:0/3444844088 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626c7db7100 0x5626c7dc2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:33.372 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.366+0000 7f7f4fe00700 1 --2- 172.21.15.86:0/3444844088 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5626c7dc28d0 0x5626c7dbfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:33.372 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.366+0000 7f7f47e00700 1 -- 172.21.15.86:0/3444844088 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5626c7dc28d0 msgr2=0x5626c7dbfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:01:33.372 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.366+0000 7f7f47e00700 1 --2- 172.21.15.86:0/3444844088 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5626c7dc28d0 0x5626c7dbfc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:33.372 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.366+0000 7f7f47e00700 1 -- 172.21.15.86:0/3444844088 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626c7db7100 msgr2=0x5626c7dc2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:01:33.372 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.366+0000 7f7f47e00700 1 --2- 172.21.15.86:0/3444844088 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626c7db7100 0x5626c7dc2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:33.373 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.366+0000 7f7f47e00700 1 -- 172.21.15.86:0/3444844088 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f501a41f0 con 0x5626c7db7a20 2024-03-23T18:01:33.373 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.366+0000 7f7f4f400700 1 --2- 172.21.15.86:0/3444844088 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626c7db7100 0x5626c7dc2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T18:01:33.373 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.366+0000 7f7f4fe00700 1 --2- 172.21.15.86:0/3444844088 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5626c7dc28d0 0x5626c7dbfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T18:01:33.373 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.367+0000 7f7f47e00700 1 --2- 172.21.15.86:0/3444844088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626c7db7a20 0x5626c7dbf610 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f7f38009350 tx=0x7f7f3800fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3caabe49e8bba0eb server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:01:33.373 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.367+0000 7f7f3cc00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-23T18:01:33.373 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.367+0000 7f7f3cc00700 1 -- 172.21.15.86:0/3444844088 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3801d040 con 0x5626c7db7a20 2024-03-23T18:01:33.373 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.367+0000 7f7f3cc00700 1 -- 172.21.15.86:0/3444844088 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f7f38005050 con 0x5626c7db7a20 2024-03-23T18:01:33.373 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.367+0000 7f7f69ad7b80 1 -- 172.21.15.86:0/3444844088 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5626c7cca320 con 0x5626c7db7a20 2024-03-23T18:01:33.373 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.367+0000 7f7f3cc00700 1 -- 172.21.15.86:0/3444844088 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f38005940 con 0x5626c7db7a20 2024-03-23T18:01:33.373 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.367+0000 7f7f69ad7b80 10 client.16857 fetch_fsmap learned FSMap version 51 2024-03-23T18:01:33.373 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.367+0000 7f7f69ad7b80 1 -- 172.21.15.86:0/3444844088 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x7f7f5010e420 con 0x5626c7db7a20 2024-03-23T18:01:33.373 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.367+0000 7f7f3cc00700 1 -- 172.21.15.86:0/3444844088 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5626c7cca320 con 0x5626c7db7a20 2024-03-23T18:01:33.373 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.368+0000 7f7f3cc00700 1 -- 172.21.15.86:0/3444844088 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 51) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f7f38022400 con 0x5626c7db7a20 2024-03-23T18:01:33.374 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.368+0000 7f7f69ad7b80 10 client.16857 fetch_fsmap finished waiting for FSMap version 51 2024-03-23T18:01:33.374 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.368+0000 7f7f69ad7b80 10 client.16857 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T18:01:33.374 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.368+0000 7f7f69ad7b80 20 client.16857 populate_metadata read hostname 'smithi086' 2024-03-23T18:01:33.374 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.368+0000 7f7f69ad7b80 1 --2- 172.21.15.86:0/3444844088 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5626c7dd6650 0x5626c7dd8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:33.374 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.368+0000 7f7f69ad7b80 4 client.16857 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T18:01:33.374 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.368+0000 7f7f69ad7b80 1 -- 172.21.15.86:0/3444844088 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5626c7d026a0 con 0x5626c7dd6650 2024-03-23T18:01:33.374 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.368+0000 7f7f4f400700 1 --2- 172.21.15.86:0/3444844088 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5626c7dd6650 0x5626c7dd8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:33.374 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.369+0000 7f7f4f400700 1 --2- 172.21.15.86:0/3444844088 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5626c7dd6650 0x5626c7dd8a70 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8eeb12d39ac9c9a server_cookie=b2d9253fd0d74925 in_seq=0 out_seq=0 2024-03-23T18:01:33.374 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.369+0000 7f7f3cc00700 10 client.16857 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T18:01:33.374 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.369+0000 7f7f3cc00700 1 -- 172.21.15.86:0/3444844088 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5626c7d026a0 con 0x5626c7dd6650 2024-03-23T18:01:33.374 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.369+0000 7f7f3cc00700 10 client.16857 handle_command_reply: tid=0 2024-03-23T18:01:33.374 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.369+0000 7f7f69ad7b80 1 client.16857 shutdown 2024-03-23T18:01:33.374 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.369+0000 7f7f69ad7b80 1 -- 172.21.15.86:0/3444844088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626c7db7a20 msgr2=0x5626c7dbf610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:01:33.374 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.369+0000 7f7f69ad7b80 1 --2- 172.21.15.86:0/3444844088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626c7db7a20 0x5626c7dbf610 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f7f38009350 tx=0x7f7f3800fae0 comp rx=0 tx=0).stop 2024-03-23T18:01:33.375 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.369+0000 7f7f69ad7b80 1 -- 172.21.15.86:0/3444844088 shutdown_connections 2024-03-23T18:01:33.375 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.369+0000 7f7f69ad7b80 1 --2- 172.21.15.86:0/3444844088 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5626c7dd6650 0x5626c7dd8a70 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:33.375 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.369+0000 7f7f69ad7b80 1 --2- 172.21.15.86:0/3444844088 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5626c7dc28d0 0x5626c7dbfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:33.375 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.369+0000 7f7f69ad7b80 1 --2- 172.21.15.86:0/3444844088 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626c7db7100 0x5626c7dc2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:33.375 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.369+0000 7f7f3cc00700 0 client.16857 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T18:01:33.375 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.369+0000 7f7f69ad7b80 1 --2- 172.21.15.86:0/3444844088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626c7db7a20 0x5626c7dbf610 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:33.375 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.369+0000 7f7f69ad7b80 1 -- 172.21.15.86:0/3444844088 >> 172.21.15.86:0/3444844088 conn(0x5626c7d17640 msgr2=0x5626c7db90a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:01:33.375 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.370+0000 7f7f69ad7b80 1 -- 172.21.15.86:0/3444844088 shutdown_connections 2024-03-23T18:01:33.375 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.370+0000 7f7f69ad7b80 1 -- 172.21.15.86:0/3444844088 wait complete. 2024-03-23T18:01:33.375 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.370+0000 7f7f69ad7b80 20 client.16857 trim_cache size 0 max 16384 2024-03-23T18:01:33.385 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.384+0000 7f7f69ad7b80 1 Processor -- start 2024-03-23T18:01:33.385 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.384+0000 7f7f69ad7b80 1 -- start start 2024-03-23T18:01:33.388 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.385+0000 7f7f69ad7b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626c7db7100 0x5626c7dbfbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:33.388 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.385+0000 7f7f69ad7b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5626c7db7a20 0x5626c7dcda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:33.388 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.385+0000 7f7f69ad7b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626c7dc28d0 0x5626c7db3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:33.388 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.385+0000 7f7f69ad7b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626c7dd3430 con 0x5626c7dc28d0 2024-03-23T18:01:33.388 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.385+0000 7f7f69ad7b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626c7daf400 con 0x5626c7db7100 2024-03-23T18:01:33.388 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.385+0000 7f7f69ad7b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626c7db8bc0 con 0x5626c7db7a20 2024-03-23T18:01:33.388 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.385+0000 7f7f47e00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5626c7db7a20 0x5626c7dcda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:33.388 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.385+0000 7f7f4fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626c7dc28d0 0x5626c7db3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:33.388 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.385+0000 7f7f47e00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5626c7db7a20 0x5626c7dcda10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:56252/0 (socket says 172.21.15.86:56252) 2024-03-23T18:01:33.388 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.385+0000 7f7f47e00700 1 -- 172.21.15.86:0/1627963192 learned_addr learned my addr 172.21.15.86:0/1627963192 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:01:33.388 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.385+0000 7f7f4f400700 1 --2- 172.21.15.86:0/1627963192 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626c7db7100 0x5626c7dbfbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:33.388 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.385+0000 7f7f47e00700 1 -- 172.21.15.86:0/1627963192 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626c7db7100 msgr2=0x5626c7dbfbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:01:33.389 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.385+0000 7f7f47e00700 1 --2- 172.21.15.86:0/1627963192 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626c7db7100 0x5626c7dbfbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:33.389 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.385+0000 7f7f47e00700 1 -- 172.21.15.86:0/1627963192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626c7dc28d0 msgr2=0x5626c7db3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:01:33.389 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.385+0000 7f7f47e00700 1 --2- 172.21.15.86:0/1627963192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626c7dc28d0 0x5626c7db3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:33.389 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.385+0000 7f7f47e00700 1 -- 172.21.15.86:0/1627963192 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626c7b37b00 con 0x5626c7db7a20 2024-03-23T18:01:33.389 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.385+0000 7f7f4fe00700 1 --2- 172.21.15.86:0/1627963192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626c7dc28d0 0x5626c7db3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T18:01:33.389 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.386+0000 7f7f47e00700 1 --2- 172.21.15.86:0/1627963192 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5626c7db7a20 0x5626c7dcda10 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7f38022860 tx=0x7f7f38022980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=427b21e8b9f3117f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:01:33.389 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.386+0000 7f7f45200700 1 -- 172.21.15.86:0/1627963192 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f38005bc0 con 0x5626c7db7a20 2024-03-23T18:01:33.389 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.386+0000 7f7f45200700 1 -- 172.21.15.86:0/1627963192 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f380058e0 con 0x5626c7db7a20 2024-03-23T18:01:33.389 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.386+0000 7f7f45200700 1 -- 172.21.15.86:0/1627963192 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f38004b10 con 0x5626c7db7a20 2024-03-23T18:01:33.389 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.386+0000 7f7f69ad7b80 1 -- 172.21.15.86:0/1627963192 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5626c7db7a20 msgr2=0x5626c7dcda10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:01:33.389 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.386+0000 7f7f69ad7b80 1 --2- 172.21.15.86:0/1627963192 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5626c7db7a20 0x5626c7dcda10 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7f38022860 tx=0x7f7f38022980 comp rx=0 tx=0).stop 2024-03-23T18:01:33.389 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.386+0000 7f7f69ad7b80 1 -- 172.21.15.86:0/1627963192 shutdown_connections 2024-03-23T18:01:33.389 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.386+0000 7f7f69ad7b80 1 --2- 172.21.15.86:0/1627963192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626c7dc28d0 0x5626c7db3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:33.389 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.386+0000 7f7f69ad7b80 1 --2- 172.21.15.86:0/1627963192 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626c7db7100 0x5626c7dbfbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:33.390 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.386+0000 7f7f69ad7b80 1 --2- 172.21.15.86:0/1627963192 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5626c7db7a20 0x5626c7dcda10 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:33.390 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.387+0000 7f7f69ad7b80 1 -- 172.21.15.86:0/1627963192 >> 172.21.15.86:0/1627963192 conn(0x5626c7d17640 msgr2=0x5626c7db2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:01:33.390 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.387+0000 7f7f69ad7b80 1 -- 172.21.15.86:0/1627963192 shutdown_connections 2024-03-23T18:01:33.390 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.387+0000 7f7f69ad7b80 1 -- 172.21.15.86:0/1627963192 wait complete. 2024-03-23T18:01:33.390 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.387+0000 7f7f69ad7b80 1 -- start start 2024-03-23T18:01:33.393 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.387+0000 7f7f69ad7b80 1 Processor -- start 2024-03-23T18:01:33.393 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.388+0000 7f7f69ad7b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626c7db7100 0x5626c7dbcf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:33.393 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.388+0000 7f7f69ad7b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626c7db7a20 0x5626c7dcb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:33.393 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.388+0000 7f7f69ad7b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5626c7dc28d0 0x5626c7dcbf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:33.393 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.388+0000 7f7f69ad7b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626c7dcc470 con 0x5626c7db7a20 2024-03-23T18:01:33.393 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.388+0000 7f7f69ad7b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626c7dcc5e0 con 0x5626c7db7100 2024-03-23T18:01:33.393 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.388+0000 7f7f69ad7b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626c7dcc7a0 con 0x5626c7dc28d0 2024-03-23T18:01:33.394 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.388+0000 7f7f69ad7b80 10 client.0 _ll_register_callbacks cb 0x7f7f48009270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T18:01:33.394 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.388+0000 7f7f47e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626c7db7a20 0x5626c7dcb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:33.394 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.388+0000 7f7f4f400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626c7db7100 0x5626c7dbcf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:33.394 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.388+0000 7f7f47e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626c7db7a20 0x5626c7dcb9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38798/0 (socket says 172.21.15.86:38798) 2024-03-23T18:01:33.394 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.388+0000 7f7f4f400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626c7db7100 0x5626c7dbcf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:52326/0 (socket says 172.21.15.86:52326) 2024-03-23T18:01:33.394 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.388+0000 7f7f47e00700 1 -- 172.21.15.86:0/4034552495 learned_addr learned my addr 172.21.15.86:0/4034552495 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:01:33.394 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.388+0000 7f7f4fe00700 1 --2- 172.21.15.86:0/4034552495 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5626c7dc28d0 0x5626c7dcbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:33.394 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.389+0000 7f7f47e00700 1 -- 172.21.15.86:0/4034552495 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5626c7dc28d0 msgr2=0x5626c7dcbf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:01:33.394 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.389+0000 7f7f47e00700 1 --2- 172.21.15.86:0/4034552495 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5626c7dc28d0 0x5626c7dcbf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:33.394 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.389+0000 7f7f47e00700 1 -- 172.21.15.86:0/4034552495 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626c7db7100 msgr2=0x5626c7dbcf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:01:33.394 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.389+0000 7f7f47e00700 1 --2- 172.21.15.86:0/4034552495 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626c7db7100 0x5626c7dbcf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:33.394 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.389+0000 7f7f47e00700 1 -- 172.21.15.86:0/4034552495 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f4800e680 con 0x5626c7db7a20 2024-03-23T18:01:33.394 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.389+0000 7f7f4fe00700 1 --2- 172.21.15.86:0/4034552495 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5626c7dc28d0 0x5626c7dcbf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T18:01:33.395 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.389+0000 7f7f4f400700 1 --2- 172.21.15.86:0/4034552495 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626c7db7100 0x5626c7dbcf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T18:01:33.395 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.389+0000 7f7f47e00700 1 --2- 172.21.15.86:0/4034552495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626c7db7a20 0x5626c7dcb9f0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f7f3800f880 tx=0x7f7f38022c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb21aefff44cf6da server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:01:33.395 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.389+0000 7f7f3cc00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-23T18:01:33.395 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.389+0000 7f7f3cc00700 1 -- 172.21.15.86:0/4034552495 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3801d030 con 0x5626c7db7a20 2024-03-23T18:01:33.395 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.389+0000 7f7f3cc00700 1 -- 172.21.15.86:0/4034552495 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f7f38005440 con 0x5626c7db7a20 2024-03-23T18:01:33.395 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.389+0000 7f7f69ad7b80 1 -- 172.21.15.86:0/4034552495 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7f3801e040 con 0x5626c7db7a20 2024-03-23T18:01:33.395 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.389+0000 7f7f3cc00700 1 -- 172.21.15.86:0/4034552495 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f38005600 con 0x5626c7db7a20 2024-03-23T18:01:33.395 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.389+0000 7f7f69ad7b80 10 client.16863 fetch_fsmap learned FSMap version 51 2024-03-23T18:01:33.395 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.389+0000 7f7f69ad7b80 1 -- 172.21.15.86:0/4034552495 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x7f7f5010e420 con 0x5626c7db7a20 2024-03-23T18:01:33.395 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.390+0000 7f7f3cc00700 1 -- 172.21.15.86:0/4034552495 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f3801e040 con 0x5626c7db7a20 2024-03-23T18:01:33.395 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.390+0000 7f7f3cc00700 1 -- 172.21.15.86:0/4034552495 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 51) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f7f38041750 con 0x5626c7db7a20 2024-03-23T18:01:33.395 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.390+0000 7f7f69ad7b80 10 client.16863 fetch_fsmap finished waiting for FSMap version 51 2024-03-23T18:01:33.395 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.390+0000 7f7f69ad7b80 10 client.16863 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T18:01:33.395 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.390+0000 7f7f69ad7b80 20 client.16863 populate_metadata read hostname 'smithi086' 2024-03-23T18:01:33.396 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.390+0000 7f7f69ad7b80 1 --2- 172.21.15.86:0/4034552495 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5626c7e28110 0x5626c7e2a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:33.396 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.390+0000 7f7f69ad7b80 4 client.16863 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T18:01:33.396 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.390+0000 7f7f69ad7b80 1 -- 172.21.15.86:0/4034552495 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5626c7b58c90 con 0x5626c7e28110 2024-03-23T18:01:33.396 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.390+0000 7f7f4f400700 1 --2- 172.21.15.86:0/4034552495 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5626c7e28110 0x5626c7e2a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:33.396 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.391+0000 7f7f4f400700 1 --2- 172.21.15.86:0/4034552495 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5626c7e28110 0x5626c7e2a530 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ff4ac7690fc38d6 server_cookie=69d9c68b098e3a00 in_seq=0 out_seq=0 2024-03-23T18:01:33.396 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.391+0000 7f7f3cc00700 10 client.16863 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T18:01:33.444 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.442+0000 7f7f3cc00700 1 -- 172.21.15.86:0/4034552495 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5626c7b58c90 con 0x5626c7e28110 2024-03-23T18:01:33.444 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.443+0000 7f7f3cc00700 10 client.16863 handle_command_reply: tid=0 2024-03-23T18:01:33.444 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.443+0000 7f7f69ad7b80 1 client.16863 shutdown 2024-03-23T18:01:33.445 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.443+0000 7f7f69ad7b80 1 -- 172.21.15.86:0/4034552495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626c7db7a20 msgr2=0x5626c7dcb9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:01:33.445 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.443+0000 7f7f69ad7b80 1 --2- 172.21.15.86:0/4034552495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626c7db7a20 0x5626c7dcb9f0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f7f3800f880 tx=0x7f7f38022c20 comp rx=0 tx=0).stop 2024-03-23T18:01:33.445 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.443+0000 7f7f69ad7b80 1 -- 172.21.15.86:0/4034552495 shutdown_connections 2024-03-23T18:01:33.445 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.443+0000 7f7f69ad7b80 1 --2- 172.21.15.86:0/4034552495 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5626c7e28110 0x5626c7e2a530 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:33.445 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.443+0000 7f7f69ad7b80 1 --2- 172.21.15.86:0/4034552495 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5626c7dc28d0 0x5626c7dcbf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:33.445 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.443+0000 7f7f69ad7b80 1 --2- 172.21.15.86:0/4034552495 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5626c7db7100 0x5626c7dbcf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:33.445 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.443+0000 7f7f69ad7b80 1 --2- 172.21.15.86:0/4034552495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626c7db7a20 0x5626c7dcb9f0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:33.445 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.443+0000 7f7f3cc00700 0 client.16863 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T18:01:33.445 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.443+0000 7f7f69ad7b80 1 -- 172.21.15.86:0/4034552495 >> 172.21.15.86:0/4034552495 conn(0x5626c7d17640 msgr2=0x5626c7db3450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:01:33.445 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.443+0000 7f7f69ad7b80 1 -- 172.21.15.86:0/4034552495 shutdown_connections 2024-03-23T18:01:33.445 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.443+0000 7f7f69ad7b80 1 -- 172.21.15.86:0/4034552495 wait complete. 2024-03-23T18:01:33.445 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.443+0000 7f7f69ad7b80 20 client.16863 trim_cache size 0 max 16384 2024-03-23T18:01:33.446 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-03-23T18:01:33.446 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-03-23T18:01:33.446 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-03-23T18:01:33.446 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-03-23T18:01:33.447 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.444+0000 7f7f56000700 1 -- 172.21.15.86:0/4074890678 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f7f340a1290 msgr2=0x7f7f340a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:01:33.447 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.444+0000 7f7f56000700 1 --2- 172.21.15.86:0/4074890678 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f7f340a1290 0x7f7f340a3750 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7f48009c10 tx=0x7f7f48009920 comp rx=0 tx=0).stop 2024-03-23T18:01:33.447 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.444+0000 7f7f56000700 1 -- 172.21.15.86:0/4074890678 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7f50100f10 msgr2=0x7f7f501971d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:01:33.447 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.444+0000 7f7f56000700 1 --2- 172.21.15.86:0/4074890678 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7f50100f10 0x7f7f501971d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7f4000e9f0 tx=0x7f7f4000ed00 comp rx=0 tx=0).stop 2024-03-23T18:01:33.447 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.444+0000 7f7f56000700 1 -- 172.21.15.86:0/4074890678 shutdown_connections 2024-03-23T18:01:33.447 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.444+0000 7f7f56000700 1 --2- 172.21.15.86:0/4074890678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f5019bab0 0x7f7f5019bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:33.447 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.444+0000 7f7f56000700 1 --2- 172.21.15.86:0/4074890678 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f7f340a1290 0x7f7f340a3750 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:33.447 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.444+0000 7f7f56000700 1 --2- 172.21.15.86:0/4074890678 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7f50100f10 0x7f7f501971d0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:33.447 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.444+0000 7f7f56000700 1 --2- 172.21.15.86:0/4074890678 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f50106030 0x7f7f50197710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:33.447 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.444+0000 7f7f56000700 1 -- 172.21.15.86:0/4074890678 >> 172.21.15.86:0/4074890678 conn(0x7f7f500fc7b0 msgr2=0x7f7f500fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:01:33.447 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.445+0000 7f7f56000700 1 -- 172.21.15.86:0/4074890678 shutdown_connections 2024-03-23T18:01:33.447 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:33.445+0000 7f7f56000700 1 -- 172.21.15.86:0/4074890678 wait complete. 2024-03-23T18:01:33.923 INFO:tasks.workunit.client.0.smithi086.stdout: 1 656048 6.50 MB/sec execute 478 sec latency 2233.341 ms 2024-03-23T18:01:34.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:01:33 smithi086 ceph-mon[38221]: pgmap v341: 97 pgs: 97 active+clean; 263 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 230 op/s 2024-03-23T18:01:34.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:01:33 smithi092 ceph-mon[79089]: pgmap v341: 97 pgs: 97 active+clean; 263 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 230 op/s 2024-03-23T18:01:34.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:01:33 smithi152 ceph-mon[47949]: pgmap v341: 97 pgs: 97 active+clean; 263 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 230 op/s 2024-03-23T18:01:34.923 INFO:tasks.workunit.client.0.smithi086.stdout: 1 657537 6.51 MB/sec execute 479 sec latency 14.544 ms 2024-03-23T18:01:35.923 INFO:tasks.workunit.client.0.smithi086.stdout: 1 658678 6.51 MB/sec execute 480 sec latency 276.813 ms 2024-03-23T18:01:36.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:01:35 smithi086 ceph-mon[38221]: pgmap v342: 97 pgs: 97 active+clean; 258 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.8 MiB/s wr, 319 op/s 2024-03-23T18:01:36.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:01:35 smithi092 ceph-mon[79089]: pgmap v342: 97 pgs: 97 active+clean; 258 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.8 MiB/s wr, 319 op/s 2024-03-23T18:01:36.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:01:35 smithi152 ceph-mon[47949]: pgmap v342: 97 pgs: 97 active+clean; 258 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.8 MiB/s wr, 319 op/s 2024-03-23T18:01:36.923 INFO:tasks.workunit.client.0.smithi086.stdout: 1 660751 6.52 MB/sec execute 481 sec latency 16.716 ms 2024-03-23T18:01:37.923 INFO:tasks.workunit.client.0.smithi086.stdout: 1 662023 6.52 MB/sec execute 482 sec latency 15.595 ms 2024-03-23T18:01:38.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:01:37 smithi086 ceph-mon[38221]: pgmap v343: 97 pgs: 97 active+clean; 258 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1000 KiB/s rd, 2.4 MiB/s wr, 194 op/s 2024-03-23T18:01:38.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:01:37 smithi092 ceph-mon[79089]: pgmap v343: 97 pgs: 97 active+clean; 258 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1000 KiB/s rd, 2.4 MiB/s wr, 194 op/s 2024-03-23T18:01:38.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:01:37 smithi152 ceph-mon[47949]: pgmap v343: 97 pgs: 97 active+clean; 258 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1000 KiB/s rd, 2.4 MiB/s wr, 194 op/s 2024-03-23T18:01:38.923 INFO:tasks.workunit.client.0.smithi086.stdout: 1 662356 6.51 MB/sec execute 483 sec latency 788.014 ms 2024-03-23T18:01:39.923 INFO:tasks.workunit.client.0.smithi086.stdout: 1 664309 6.52 MB/sec execute 484 sec latency 15.544 ms 2024-03-23T18:01:40.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:01:39 smithi086 ceph-mon[38221]: pgmap v344: 97 pgs: 97 active+clean; 258 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1000 KiB/s rd, 2.4 MiB/s wr, 194 op/s 2024-03-23T18:01:40.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:01:39 smithi092 ceph-mon[79089]: pgmap v344: 97 pgs: 97 active+clean; 258 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1000 KiB/s rd, 2.4 MiB/s wr, 194 op/s 2024-03-23T18:01:40.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:01:39 smithi152 ceph-mon[47949]: pgmap v344: 97 pgs: 97 active+clean; 258 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1000 KiB/s rd, 2.4 MiB/s wr, 194 op/s 2024-03-23T18:01:40.923 INFO:tasks.workunit.client.0.smithi086.stdout: 1 665579 6.52 MB/sec execute 485 sec latency 13.623 ms 2024-03-23T18:01:41.923 INFO:tasks.workunit.client.0.smithi086.stdout: 1 665765 6.51 MB/sec execute 486 sec latency 826.484 ms 2024-03-23T18:01:42.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:01:41 smithi086 ceph-mon[38221]: pgmap v345: 97 pgs: 97 active+clean; 264 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.5 MiB/s wr, 284 op/s 2024-03-23T18:01:42.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:01:41 smithi092 ceph-mon[79089]: pgmap v345: 97 pgs: 97 active+clean; 264 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.5 MiB/s wr, 284 op/s 2024-03-23T18:01:42.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:01:41 smithi152 ceph-mon[47949]: pgmap v345: 97 pgs: 97 active+clean; 264 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.5 MiB/s wr, 284 op/s 2024-03-23T18:01:42.924 INFO:tasks.workunit.client.0.smithi086.stdout: 1 665765 6.49 MB/sec execute 487 sec latency 1826.598 ms 2024-03-23T18:01:43.924 INFO:tasks.workunit.client.0.smithi086.stdout: 1 665765 6.48 MB/sec execute 488 sec latency 2826.693 ms 2024-03-23T18:01:44.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:01:43 smithi086 ceph-mon[38221]: pgmap v346: 97 pgs: 97 active+clean; 264 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 864 KiB/s rd, 2.1 MiB/s wr, 178 op/s 2024-03-23T18:01:44.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:01:43 smithi092 ceph-mon[79089]: pgmap v346: 97 pgs: 97 active+clean; 264 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 864 KiB/s rd, 2.1 MiB/s wr, 178 op/s 2024-03-23T18:01:44.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:01:43 smithi152 ceph-mon[47949]: pgmap v346: 97 pgs: 97 active+clean; 264 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 864 KiB/s rd, 2.1 MiB/s wr, 178 op/s 2024-03-23T18:01:44.924 INFO:tasks.workunit.client.0.smithi086.stdout: 1 666457 6.47 MB/sec execute 489 sec latency 3528.337 ms 2024-03-23T18:01:45.924 INFO:tasks.workunit.client.0.smithi086.stdout: 1 667946 6.48 MB/sec execute 490 sec latency 16.654 ms 2024-03-23T18:01:46.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:01:45 smithi086 ceph-mon[38221]: pgmap v347: 97 pgs: 97 active+clean; 271 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 229 op/s 2024-03-23T18:01:46.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:01:45 smithi092 ceph-mon[79089]: pgmap v347: 97 pgs: 97 active+clean; 271 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 229 op/s 2024-03-23T18:01:46.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:01:45 smithi152 ceph-mon[47949]: pgmap v347: 97 pgs: 97 active+clean; 271 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 229 op/s 2024-03-23T18:01:46.592 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.592+0000 7f5a7aa00700 1 -- 172.21.15.86:0/2208830393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a74106030 msgr2=0x7f5a7410dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:01:46.593 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.592+0000 7f5a7aa00700 1 --2- 172.21.15.86:0/2208830393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a74106030 0x7f5a7410dbb0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c00b0a0 tx=0x7f5a6c00b3b0 comp rx=0 tx=0).stop 2024-03-23T18:01:46.593 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.592+0000 7f5a7aa00700 1 -- 172.21.15.86:0/2208830393 shutdown_connections 2024-03-23T18:01:46.593 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.592+0000 7f5a7aa00700 1 --2- 172.21.15.86:0/2208830393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a74106030 0x7f5a7410dbb0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:46.593 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.592+0000 7f5a7aa00700 1 --2- 172.21.15.86:0/2208830393 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5a74100f10 0x7f5a741012f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:46.593 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.592+0000 7f5a7aa00700 1 --2- 172.21.15.86:0/2208830393 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a741018c0 0x7f5a74105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:46.593 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.592+0000 7f5a7aa00700 1 -- 172.21.15.86:0/2208830393 >> 172.21.15.86:0/2208830393 conn(0x7f5a740fc7b0 msgr2=0x7f5a740febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:01:46.593 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.592+0000 7f5a7aa00700 1 -- 172.21.15.86:0/2208830393 shutdown_connections 2024-03-23T18:01:46.593 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.592+0000 7f5a7aa00700 1 -- 172.21.15.86:0/2208830393 wait complete. 2024-03-23T18:01:46.595 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.593+0000 7f5a7aa00700 1 Processor -- start 2024-03-23T18:01:46.595 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.594+0000 7f5a7aa00700 1 -- start start 2024-03-23T18:01:46.595 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.594+0000 7f5a7aa00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a74100f10 0x7f5a74199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:46.595 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.594+0000 7f5a7aa00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5a74106030 0x7f5a74199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:46.595 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.594+0000 7f5a7aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a7419dda0 0x7f5a7419e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:46.595 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.594+0000 7f5a7aa00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a7419e820 con 0x7f5a7419dda0 2024-03-23T18:01:46.595 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.594+0000 7f5a7aa00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a7419e990 con 0x7f5a74100f10 2024-03-23T18:01:46.595 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.594+0000 7f5a7aa00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a7419eb50 con 0x7f5a74106030 2024-03-23T18:01:46.595 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.594+0000 7f5a78a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a7419dda0 0x7f5a7419e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:46.595 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.594+0000 7f5a78a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a7419dda0 0x7f5a7419e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40832/0 (socket says 172.21.15.86:40832) 2024-03-23T18:01:46.595 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.594+0000 7f5a78a00700 1 -- 172.21.15.86:0/426216147 learned_addr learned my addr 172.21.15.86:0/426216147 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:01:46.596 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.594+0000 7f5a73e00700 1 --2- 172.21.15.86:0/426216147 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a74100f10 0x7f5a74199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:46.596 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.594+0000 7f5a73400700 1 --2- 172.21.15.86:0/426216147 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5a74106030 0x7f5a74199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:46.596 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.595+0000 7f5a78a00700 1 -- 172.21.15.86:0/426216147 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5a74106030 msgr2=0x7f5a74199a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:01:46.596 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.595+0000 7f5a78a00700 1 --2- 172.21.15.86:0/426216147 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5a74106030 0x7f5a74199a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:46.596 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.595+0000 7f5a78a00700 1 -- 172.21.15.86:0/426216147 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a74100f10 msgr2=0x7f5a74199510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:01:46.596 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.595+0000 7f5a78a00700 1 --2- 172.21.15.86:0/426216147 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a74100f10 0x7f5a74199510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:46.596 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.595+0000 7f5a78a00700 1 -- 172.21.15.86:0/426216147 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a6c009d00 con 0x7f5a7419dda0 2024-03-23T18:01:46.596 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.595+0000 7f5a78a00700 1 --2- 172.21.15.86:0/426216147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a7419dda0 0x7f5a7419e220 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c00b070 tx=0x7f5a6c00bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:01:46.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.595+0000 7f5a70c00700 1 -- 172.21.15.86:0/426216147 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6c00e040 con 0x7f5a7419dda0 2024-03-23T18:01:46.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.595+0000 7f5a70c00700 1 -- 172.21.15.86:0/426216147 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a6c01d480 con 0x7f5a7419dda0 2024-03-23T18:01:46.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.595+0000 7f5a7aa00700 1 -- 172.21.15.86:0/426216147 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a741a6050 con 0x7f5a7419dda0 2024-03-23T18:01:46.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.595+0000 7f5a70c00700 1 -- 172.21.15.86:0/426216147 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6c0052a0 con 0x7f5a7419dda0 2024-03-23T18:01:46.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.595+0000 7f5a7aa00700 1 -- 172.21.15.86:0/426216147 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a741a6420 con 0x7f5a7419dda0 2024-03-23T18:01:46.598 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.599+0000 7f5a70c00700 1 -- 172.21.15.86:0/426216147 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f5a6c01d5f0 con 0x7f5a7419dda0 2024-03-23T18:01:46.602 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.600+0000 7f5a70c00700 1 --2- 172.21.15.86:0/426216147 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f5a580a1290 0x7f5a580a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:46.602 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.600+0000 7f5a8e527b80 1 Processor -- start 2024-03-23T18:01:46.602 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.600+0000 7f5a70c00700 1 -- 172.21.15.86:0/426216147 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f5a6c0cdac0 con 0x7f5a7419dda0 2024-03-23T18:01:46.602 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.600+0000 7f5a8e527b80 1 -- start start 2024-03-23T18:01:46.602 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.600+0000 7f5a73e00700 1 --2- 172.21.15.86:0/426216147 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f5a580a1290 0x7f5a580a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:46.602 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.600+0000 7f5a8e527b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef19e41100 0x55ef19e414e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:46.602 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.600+0000 7f5a8e527b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef19e41a20 0x55ef19e4c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:46.602 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.600+0000 7f5a8e527b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ef19e4c8d0 0x55ef19e4ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:46.602 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.601+0000 7f5a8e527b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef19d9a930 con 0x55ef19e41100 2024-03-23T18:01:46.602 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.601+0000 7f5a8e527b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef19d982f0 con 0x55ef19e41a20 2024-03-23T18:01:46.602 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.601+0000 7f5a8e527b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef19d93be0 con 0x55ef19e4c8d0 2024-03-23T18:01:46.603 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.601+0000 7f5a73e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef19e41100 0x55ef19e414e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:46.603 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.601+0000 7f5a73400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef19e41a20 0x55ef19e4c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:46.603 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.601+0000 7f5a78a00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ef19e4c8d0 0x55ef19e4ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:46.603 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.601+0000 7f5a73400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef19e41a20 0x55ef19e4c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:37406/0 (socket says 172.21.15.86:37406) 2024-03-23T18:01:46.603 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.601+0000 7f5a73400700 1 -- 172.21.15.86:0/3270653302 learned_addr learned my addr 172.21.15.86:0/3270653302 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:01:46.603 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.601+0000 7f5a73400700 1 -- 172.21.15.86:0/3270653302 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ef19e4c8d0 msgr2=0x55ef19e4ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:01:46.603 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.601+0000 7f5a73400700 1 --2- 172.21.15.86:0/3270653302 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ef19e4c8d0 0x55ef19e4ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:46.603 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.601+0000 7f5a73400700 1 -- 172.21.15.86:0/3270653302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef19e41100 msgr2=0x55ef19e414e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:01:46.603 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.601+0000 7f5a73400700 1 --2- 172.21.15.86:0/3270653302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef19e41100 0x55ef19e414e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:46.603 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.601+0000 7f5a73400700 1 -- 172.21.15.86:0/3270653302 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ef19c15350 con 0x55ef19e41a20 2024-03-23T18:01:46.603 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.601+0000 7f5a73e00700 1 --2- 172.21.15.86:0/3270653302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef19e41100 0x55ef19e414e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T18:01:46.604 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.601+0000 7f5a73e00700 1 --2- 172.21.15.86:0/426216147 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f5a580a1290 0x7f5a580a3750 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5a64013570 tx=0x7f5a64013930 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:01:46.604 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.601+0000 7f5a73400700 1 --2- 172.21.15.86:0/3270653302 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef19e41a20 0x55ef19e4c390 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f5a60009cb0 tx=0x7f5a60011440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4e490d60032cab9 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:01:46.604 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.602+0000 7f5a69200700 1 -- 172.21.15.86:0/3270653302 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a60011d20 con 0x55ef19e41a20 2024-03-23T18:01:46.604 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.602+0000 7f5a69200700 1 -- 172.21.15.86:0/3270653302 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a60011e80 con 0x55ef19e41a20 2024-03-23T18:01:46.604 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.602+0000 7f5a69200700 1 -- 172.21.15.86:0/3270653302 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a60004ea0 con 0x55ef19e41a20 2024-03-23T18:01:46.608 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.602+0000 7f5a8e527b80 1 -- 172.21.15.86:0/3270653302 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef19e41a20 msgr2=0x55ef19e4c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:01:46.608 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.602+0000 7f5a8e527b80 1 --2- 172.21.15.86:0/3270653302 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef19e41a20 0x55ef19e4c390 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f5a60009cb0 tx=0x7f5a60011440 comp rx=0 tx=0).stop 2024-03-23T18:01:46.608 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.602+0000 7f5a8e527b80 1 -- 172.21.15.86:0/3270653302 shutdown_connections 2024-03-23T18:01:46.608 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.602+0000 7f5a8e527b80 1 --2- 172.21.15.86:0/3270653302 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ef19e4c8d0 0x55ef19e4ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:46.608 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.602+0000 7f5a8e527b80 1 --2- 172.21.15.86:0/3270653302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef19e41100 0x55ef19e414e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:46.608 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.602+0000 7f5a8e527b80 1 --2- 172.21.15.86:0/3270653302 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef19e41a20 0x55ef19e4c390 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:46.608 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.602+0000 7f5a8e527b80 1 -- 172.21.15.86:0/3270653302 >> 172.21.15.86:0/3270653302 conn(0x55ef19da1640 msgr2=0x55ef19e3d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:01:46.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.602+0000 7f5a8e527b80 1 -- 172.21.15.86:0/3270653302 shutdown_connections 2024-03-23T18:01:46.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.602+0000 7f5a8e527b80 1 -- 172.21.15.86:0/3270653302 wait complete. 2024-03-23T18:01:46.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.603+0000 7f5a8e527b80 1 -- start start 2024-03-23T18:01:46.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.603+0000 7f5a8e527b80 1 Processor -- start 2024-03-23T18:01:46.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.604+0000 7f5a8e527b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ef19e41100 0x55ef19e4c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:46.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.604+0000 7f5a8e527b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef19e41a20 0x55ef19e495b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:46.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.604+0000 7f5a8e527b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef19e4c8d0 0x55ef19e49b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:46.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.604+0000 7f5a8e527b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef19d2fd20 con 0x55ef19e41a20 2024-03-23T18:01:46.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.604+0000 7f5a8e527b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef19d2f9c0 con 0x55ef19e4c8d0 2024-03-23T18:01:46.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.604+0000 7f5a8e527b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef19c17640 con 0x55ef19e41100 2024-03-23T18:01:46.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.604+0000 7f5a8e527b80 10 client.0 _ll_register_callbacks cb 0x55ef19d39410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T18:01:46.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.604+0000 7f5a73400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef19e41a20 0x55ef19e495b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:46.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.604+0000 7f5a73e00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ef19e41100 0x55ef19e4c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:46.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.604+0000 7f5a73400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef19e41a20 0x55ef19e495b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40846/0 (socket says 172.21.15.86:40846) 2024-03-23T18:01:46.610 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.604+0000 7f5a73400700 1 -- 172.21.15.86:0/2198031179 learned_addr learned my addr 172.21.15.86:0/2198031179 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:01:46.610 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.604+0000 7f5a78a00700 1 --2- 172.21.15.86:0/2198031179 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef19e4c8d0 0x55ef19e49b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:46.610 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.604+0000 7f5a73e00700 1 -- 172.21.15.86:0/2198031179 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef19e4c8d0 msgr2=0x55ef19e49b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:01:46.610 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.604+0000 7f5a73e00700 1 --2- 172.21.15.86:0/2198031179 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef19e4c8d0 0x55ef19e49b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:46.610 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.604+0000 7f5a73e00700 1 -- 172.21.15.86:0/2198031179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef19e41a20 msgr2=0x55ef19e495b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:01:46.610 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.604+0000 7f5a73e00700 1 --2- 172.21.15.86:0/2198031179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef19e41a20 0x55ef19e495b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:46.610 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.604+0000 7f5a73e00700 1 -- 172.21.15.86:0/2198031179 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a600098e0 con 0x55ef19e41100 2024-03-23T18:01:46.610 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.605+0000 7f5a73e00700 1 --2- 172.21.15.86:0/2198031179 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ef19e41100 0x55ef19e4c220 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5a64013d30 tx=0x7f5a640047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c17d2566f85c108 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:01:46.610 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.605+0000 7f5a5d600700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-03-23T18:01:46.610 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.605+0000 7f5a5d600700 1 -- 172.21.15.86:0/2198031179 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a64018070 con 0x55ef19e41100 2024-03-23T18:01:46.610 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.605+0000 7f5a8e527b80 1 -- 172.21.15.86:0/2198031179 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ef19d8c6a0 con 0x55ef19e41100 2024-03-23T18:01:46.610 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.605+0000 7f5a5d600700 1 -- 172.21.15.86:0/2198031179 <== mon.1 v2:172.21.15.152:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f5a64004a30 con 0x55ef19e41100 2024-03-23T18:01:46.610 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.605+0000 7f5a5d600700 1 -- 172.21.15.86:0/2198031179 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a640122e0 con 0x55ef19e41100 2024-03-23T18:01:46.611 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.605+0000 7f5a8e527b80 10 client.26911 fetch_fsmap learned FSMap version 51 2024-03-23T18:01:46.611 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.605+0000 7f5a8e527b80 1 -- 172.21.15.86:0/2198031179 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x55ef19bc1b00 con 0x55ef19e41100 2024-03-23T18:01:46.611 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.605+0000 7f5a5d600700 1 -- 172.21.15.86:0/2198031179 <== mon.1 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ef19d8c6a0 con 0x55ef19e41100 2024-03-23T18:01:46.611 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.606+0000 7f5a5d600700 1 -- 172.21.15.86:0/2198031179 <== mon.1 v2:172.21.15.152:3300/0 5 ==== fsmap(e 51) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f5a64010430 con 0x55ef19e41100 2024-03-23T18:01:46.611 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.606+0000 7f5a8e527b80 10 client.26911 fetch_fsmap finished waiting for FSMap version 51 2024-03-23T18:01:46.611 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.606+0000 7f5a8e527b80 10 client.26911 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T18:01:46.611 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.606+0000 7f5a8e527b80 20 client.26911 populate_metadata read hostname 'smithi086' 2024-03-23T18:01:46.611 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.606+0000 7f5a8e527b80 1 --2- 172.21.15.86:0/2198031179 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55ef19e5c530 0x55ef19e5e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:46.611 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.606+0000 7f5a8e527b80 4 client.26911 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T18:01:46.611 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.606+0000 7f5a8e527b80 1 -- 172.21.15.86:0/2198031179 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ef19be2c90 con 0x55ef19e5c530 2024-03-23T18:01:46.611 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.606+0000 7f5a73400700 1 --2- 172.21.15.86:0/2198031179 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55ef19e5c530 0x55ef19e5e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:46.611 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.607+0000 7f5a5d600700 10 client.26911 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T18:01:46.611 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.607+0000 7f5a73400700 1 --2- 172.21.15.86:0/2198031179 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55ef19e5c530 0x55ef19e5e950 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7dab0f5d5c5112ad server_cookie=9a22192180b9d1d4 in_seq=0 out_seq=0 2024-03-23T18:01:46.611 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.607+0000 7f5a5d600700 1 -- 172.21.15.86:0/2198031179 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55ef19be2c90 con 0x55ef19e5c530 2024-03-23T18:01:46.612 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.607+0000 7f5a5d600700 10 client.26911 handle_command_reply: tid=0 2024-03-23T18:01:46.612 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.607+0000 7f5a8e527b80 1 client.26911 shutdown 2024-03-23T18:01:46.612 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.607+0000 7f5a8e527b80 1 -- 172.21.15.86:0/2198031179 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ef19e41100 msgr2=0x55ef19e4c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:01:46.612 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.607+0000 7f5a8e527b80 1 --2- 172.21.15.86:0/2198031179 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ef19e41100 0x55ef19e4c220 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5a64013d30 tx=0x7f5a640047c0 comp rx=0 tx=0).stop 2024-03-23T18:01:46.612 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.607+0000 7f5a8e527b80 1 -- 172.21.15.86:0/2198031179 shutdown_connections 2024-03-23T18:01:46.612 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.607+0000 7f5a8e527b80 1 --2- 172.21.15.86:0/2198031179 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55ef19e5c530 0x55ef19e5e950 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:46.612 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.607+0000 7f5a8e527b80 1 --2- 172.21.15.86:0/2198031179 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef19e4c8d0 0x55ef19e49b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:46.612 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.607+0000 7f5a8e527b80 1 --2- 172.21.15.86:0/2198031179 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ef19e41100 0x55ef19e4c220 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:46.612 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.607+0000 7f5a8e527b80 1 --2- 172.21.15.86:0/2198031179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef19e41a20 0x55ef19e495b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:46.612 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.607+0000 7f5a8e527b80 1 -- 172.21.15.86:0/2198031179 >> 172.21.15.86:0/2198031179 conn(0x55ef19da1640 msgr2=0x55ef19e430a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:01:46.612 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.607+0000 7f5a5d600700 0 client.26911 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T18:01:46.612 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.608+0000 7f5a8e527b80 1 -- 172.21.15.86:0/2198031179 shutdown_connections 2024-03-23T18:01:46.612 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.608+0000 7f5a8e527b80 1 -- 172.21.15.86:0/2198031179 wait complete. 2024-03-23T18:01:46.612 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.608+0000 7f5a8e527b80 20 client.26911 trim_cache size 0 max 16384 2024-03-23T18:01:46.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.624+0000 7f5a8e527b80 1 Processor -- start 2024-03-23T18:01:46.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.624+0000 7f5a8e527b80 1 -- start start 2024-03-23T18:01:46.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.624+0000 7f5a8e527b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef19e41100 0x55ef19e5e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:46.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.624+0000 7f5a8e527b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef19e41a20 0x55ef19e44050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:46.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.624+0000 7f5a8e527b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ef19e4c8d0 0x55ef19e632b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:46.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.624+0000 7f5a8e527b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef19e39cc0 con 0x55ef19e41100 2024-03-23T18:01:46.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.624+0000 7f5a8e527b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef19e418d0 con 0x55ef19e41a20 2024-03-23T18:01:46.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.624+0000 7f5a8e527b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef19e45490 con 0x55ef19e4c8d0 2024-03-23T18:01:46.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.625+0000 7f5a73e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef19e41100 0x55ef19e5e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:46.626 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.625+0000 7f5a73400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef19e41a20 0x55ef19e44050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:46.626 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.625+0000 7f5a73e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef19e41100 0x55ef19e5e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40862/0 (socket says 172.21.15.86:40862) 2024-03-23T18:01:46.626 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.625+0000 7f5a73e00700 1 -- 172.21.15.86:0/4061675870 learned_addr learned my addr 172.21.15.86:0/4061675870 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:01:46.626 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.625+0000 7f5a78a00700 1 --2- 172.21.15.86:0/4061675870 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ef19e4c8d0 0x55ef19e632b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:46.626 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.625+0000 7f5a73400700 1 -- 172.21.15.86:0/4061675870 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ef19e4c8d0 msgr2=0x55ef19e632b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:01:46.626 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.625+0000 7f5a73400700 1 --2- 172.21.15.86:0/4061675870 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ef19e4c8d0 0x55ef19e632b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:46.626 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.625+0000 7f5a73400700 1 -- 172.21.15.86:0/4061675870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef19e41100 msgr2=0x55ef19e5e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:01:46.626 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.625+0000 7f5a73400700 1 --2- 172.21.15.86:0/4061675870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef19e41100 0x55ef19e5e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:46.626 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.625+0000 7f5a73400700 1 -- 172.21.15.86:0/4061675870 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ef19d59e40 con 0x55ef19e41a20 2024-03-23T18:01:46.626 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.625+0000 7f5a73e00700 1 --2- 172.21.15.86:0/4061675870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef19e41100 0x55ef19e5e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T18:01:46.626 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.625+0000 7f5a78a00700 1 --2- 172.21.15.86:0/4061675870 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ef19e4c8d0 0x55ef19e632b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T18:01:46.626 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.625+0000 7f5a73400700 1 --2- 172.21.15.86:0/4061675870 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef19e41a20 0x55ef19e44050 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f5a60004c80 tx=0x7f5a60009230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=452cc7be69294479 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:01:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.625+0000 7f5a69c00700 1 -- 172.21.15.86:0/4061675870 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a60011c20 con 0x55ef19e41a20 2024-03-23T18:01:46.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.626+0000 7f5a69c00700 1 -- 172.21.15.86:0/4061675870 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a60004500 con 0x55ef19e41a20 2024-03-23T18:01:46.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.626+0000 7f5a69c00700 1 -- 172.21.15.86:0/4061675870 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a60003b80 con 0x55ef19e41a20 2024-03-23T18:01:46.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.626+0000 7f5a8e527b80 1 -- 172.21.15.86:0/4061675870 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef19e41a20 msgr2=0x55ef19e44050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:01:46.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.626+0000 7f5a8e527b80 1 --2- 172.21.15.86:0/4061675870 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef19e41a20 0x55ef19e44050 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f5a60004c80 tx=0x7f5a60009230 comp rx=0 tx=0).stop 2024-03-23T18:01:46.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.626+0000 7f5a8e527b80 1 -- 172.21.15.86:0/4061675870 shutdown_connections 2024-03-23T18:01:46.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.626+0000 7f5a8e527b80 1 --2- 172.21.15.86:0/4061675870 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ef19e4c8d0 0x55ef19e632b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:46.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.626+0000 7f5a8e527b80 1 --2- 172.21.15.86:0/4061675870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef19e41100 0x55ef19e5e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:46.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.626+0000 7f5a8e527b80 1 --2- 172.21.15.86:0/4061675870 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef19e41a20 0x55ef19e44050 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:46.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.626+0000 7f5a8e527b80 1 -- 172.21.15.86:0/4061675870 >> 172.21.15.86:0/4061675870 conn(0x55ef19da1640 msgr2=0x55ef19e3cd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:01:46.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.626+0000 7f5a8e527b80 1 -- 172.21.15.86:0/4061675870 shutdown_connections 2024-03-23T18:01:46.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.626+0000 7f5a8e527b80 1 -- 172.21.15.86:0/4061675870 wait complete. 2024-03-23T18:01:46.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.627+0000 7f5a8e527b80 1 -- start start 2024-03-23T18:01:46.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.627+0000 7f5a8e527b80 1 Processor -- start 2024-03-23T18:01:46.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.627+0000 7f5a8e527b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef19e41100 0x55ef19e52d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:46.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.627+0000 7f5a8e527b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef19e41a20 0x55ef19e53280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:46.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.627+0000 7f5a8e527b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ef19e4c8d0 0x55ef19e51580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:46.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.627+0000 7f5a8e527b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef19e51c20 con 0x55ef19e41a20 2024-03-23T18:01:46.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.627+0000 7f5a8e527b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef19e51d90 con 0x55ef19e41100 2024-03-23T18:01:46.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.627+0000 7f5a8e527b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef19e51f50 con 0x55ef19e4c8d0 2024-03-23T18:01:46.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.627+0000 7f5a8e527b80 10 client.0 _ll_register_callbacks cb 0x55ef19d39410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T18:01:46.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.628+0000 7f5a78a00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ef19e4c8d0 0x55ef19e51580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:46.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.628+0000 7f5a73e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef19e41100 0x55ef19e52d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:46.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.628+0000 7f5a73400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef19e41a20 0x55ef19e53280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:46.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.628+0000 7f5a78a00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ef19e4c8d0 0x55ef19e51580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:48440/0 (socket says 172.21.15.86:48440) 2024-03-23T18:01:46.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.628+0000 7f5a78a00700 1 -- 172.21.15.86:0/2093471749 learned_addr learned my addr 172.21.15.86:0/2093471749 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:01:46.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.628+0000 7f5a73e00700 1 -- 172.21.15.86:0/2093471749 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ef19e4c8d0 msgr2=0x55ef19e51580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:01:46.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.628+0000 7f5a73e00700 1 --2- 172.21.15.86:0/2093471749 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ef19e4c8d0 0x55ef19e51580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:46.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.628+0000 7f5a73e00700 1 -- 172.21.15.86:0/2093471749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef19e41a20 msgr2=0x55ef19e53280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:01:46.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.628+0000 7f5a73e00700 1 --2- 172.21.15.86:0/2093471749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef19e41a20 0x55ef19e53280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:46.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.628+0000 7f5a73e00700 1 -- 172.21.15.86:0/2093471749 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ef19bc1b00 con 0x55ef19e41100 2024-03-23T18:01:46.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.628+0000 7f5a78a00700 1 --2- 172.21.15.86:0/2093471749 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ef19e4c8d0 0x55ef19e51580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T18:01:46.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.628+0000 7f5a73400700 1 --2- 172.21.15.86:0/2093471749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef19e41a20 0x55ef19e53280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T18:01:46.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.628+0000 7f5a73e00700 1 --2- 172.21.15.86:0/2093471749 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef19e41100 0x55ef19e52d40 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f5a64013d30 tx=0x7f5a64010660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=133e78bc87d7b161 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:01:46.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.628+0000 7f5a5d600700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T18:01:46.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.628+0000 7f5a5d600700 1 -- 172.21.15.86:0/2093471749 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a64018070 con 0x55ef19e41100 2024-03-23T18:01:46.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.628+0000 7f5a8e527b80 1 -- 172.21.15.86:0/2093471749 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ef19d54320 con 0x55ef19e41100 2024-03-23T18:01:46.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.628+0000 7f5a5d600700 1 -- 172.21.15.86:0/2093471749 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f5a64009e70 con 0x55ef19e41100 2024-03-23T18:01:46.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.629+0000 7f5a5d600700 1 -- 172.21.15.86:0/2093471749 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a640122e0 con 0x55ef19e41100 2024-03-23T18:01:46.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.629+0000 7f5a8e527b80 10 client.26936 fetch_fsmap learned FSMap version 51 2024-03-23T18:01:46.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.629+0000 7f5a8e527b80 1 -- 172.21.15.86:0/2093471749 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x7f5a6001d040 con 0x55ef19e41100 2024-03-23T18:01:46.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.629+0000 7f5a5d600700 1 -- 172.21.15.86:0/2093471749 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a6403f080 con 0x55ef19e41100 2024-03-23T18:01:46.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.629+0000 7f5a5d600700 1 -- 172.21.15.86:0/2093471749 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 51) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f5a64005020 con 0x55ef19e41100 2024-03-23T18:01:46.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.629+0000 7f5a8e527b80 10 client.26936 fetch_fsmap finished waiting for FSMap version 51 2024-03-23T18:01:46.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.629+0000 7f5a8e527b80 10 client.26936 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T18:01:46.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.629+0000 7f5a8e527b80 20 client.26936 populate_metadata read hostname 'smithi086' 2024-03-23T18:01:46.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.629+0000 7f5a8e527b80 1 --2- 172.21.15.86:0/2093471749 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55ef19eaf220 0x55ef19eb1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:46.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.629+0000 7f5a8e527b80 4 client.26936 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T18:01:46.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.629+0000 7f5a8e527b80 1 -- 172.21.15.86:0/2093471749 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ef19c17c80 con 0x55ef19eaf220 2024-03-23T18:01:46.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.629+0000 7f5a73400700 1 --2- 172.21.15.86:0/2093471749 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55ef19eaf220 0x55ef19eb1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:46.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.630+0000 7f5a5d600700 10 client.26936 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T18:01:46.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.630+0000 7f5a73400700 1 --2- 172.21.15.86:0/2093471749 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55ef19eaf220 0x55ef19eb1640 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2abb0e0692401082 server_cookie=5b7d332a8360ee3a in_seq=0 out_seq=0 2024-03-23T18:01:46.749 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.748+0000 7f5a5d600700 1 -- 172.21.15.86:0/2093471749 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ef19c17c80 con 0x55ef19eaf220 2024-03-23T18:01:46.749 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.748+0000 7f5a5d600700 10 client.26936 handle_command_reply: tid=0 2024-03-23T18:01:46.750 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.748+0000 7f5a8e527b80 1 client.26936 shutdown 2024-03-23T18:01:46.750 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.749+0000 7f5a8e527b80 1 -- 172.21.15.86:0/2093471749 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef19e41100 msgr2=0x55ef19e52d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:01:46.750 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.749+0000 7f5a8e527b80 1 --2- 172.21.15.86:0/2093471749 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef19e41100 0x55ef19e52d40 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f5a64013d30 tx=0x7f5a64010660 comp rx=0 tx=0).stop 2024-03-23T18:01:46.750 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.749+0000 7f5a8e527b80 1 -- 172.21.15.86:0/2093471749 shutdown_connections 2024-03-23T18:01:46.750 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.749+0000 7f5a8e527b80 1 --2- 172.21.15.86:0/2093471749 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55ef19eaf220 0x55ef19eb1640 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:46.750 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.749+0000 7f5a8e527b80 1 --2- 172.21.15.86:0/2093471749 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ef19e4c8d0 0x55ef19e51580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:46.750 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.749+0000 7f5a8e527b80 1 --2- 172.21.15.86:0/2093471749 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ef19e41100 0x55ef19e52d40 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:46.750 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.749+0000 7f5a8e527b80 1 --2- 172.21.15.86:0/2093471749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef19e41a20 0x55ef19e53280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:46.750 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.749+0000 7f5a8e527b80 1 -- 172.21.15.86:0/2093471749 >> 172.21.15.86:0/2093471749 conn(0x55ef19da1640 msgr2=0x55ef19e363e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:01:46.750 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.749+0000 7f5a5d600700 0 client.26936 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T18:01:46.750 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.749+0000 7f5a8e527b80 1 -- 172.21.15.86:0/2093471749 shutdown_connections 2024-03-23T18:01:46.750 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.749+0000 7f5a8e527b80 1 -- 172.21.15.86:0/2093471749 wait complete. 2024-03-23T18:01:46.750 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.749+0000 7f5a8e527b80 20 client.26936 trim_cache size 0 max 16384 2024-03-23T18:01:46.751 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-03-23T18:01:46.751 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-03-23T18:01:46.751 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-03-23T18:01:46.751 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-03-23T18:01:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.750+0000 7f5a7aa00700 1 -- 172.21.15.86:0/426216147 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f5a580a1290 msgr2=0x7f5a580a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:01:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.750+0000 7f5a7aa00700 1 --2- 172.21.15.86:0/426216147 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f5a580a1290 0x7f5a580a3750 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5a64013570 tx=0x7f5a64013930 comp rx=0 tx=0).stop 2024-03-23T18:01:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.750+0000 7f5a7aa00700 1 -- 172.21.15.86:0/426216147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a7419dda0 msgr2=0x7f5a7419e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:01:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.750+0000 7f5a7aa00700 1 --2- 172.21.15.86:0/426216147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a7419dda0 0x7f5a7419e220 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c00b070 tx=0x7f5a6c00bda0 comp rx=0 tx=0).stop 2024-03-23T18:01:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.750+0000 7f5a7aa00700 1 -- 172.21.15.86:0/426216147 shutdown_connections 2024-03-23T18:01:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.750+0000 7f5a7aa00700 1 --2- 172.21.15.86:0/426216147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a7419dda0 0x7f5a7419e220 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.750+0000 7f5a7aa00700 1 --2- 172.21.15.86:0/426216147 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a74100f10 0x7f5a74199510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.750+0000 7f5a7aa00700 1 --2- 172.21.15.86:0/426216147 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f5a580a1290 0x7f5a580a3750 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.750+0000 7f5a7aa00700 1 --2- 172.21.15.86:0/426216147 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5a74106030 0x7f5a74199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:46.753 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.750+0000 7f5a7aa00700 1 -- 172.21.15.86:0/426216147 >> 172.21.15.86:0/426216147 conn(0x7f5a740fc7b0 msgr2=0x7f5a740fea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:01:46.753 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.750+0000 7f5a7aa00700 1 -- 172.21.15.86:0/426216147 shutdown_connections 2024-03-23T18:01:46.753 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:46.750+0000 7f5a7aa00700 1 -- 172.21.15.86:0/426216147 wait complete. 2024-03-23T18:01:46.924 INFO:tasks.workunit.client.0.smithi086.stdout: 1 668730 6.47 MB/sec execute 491 sec latency 252.019 ms 2024-03-23T18:01:47.924 INFO:tasks.workunit.client.0.smithi086.stdout: 1 669807 6.47 MB/sec execute 492 sec latency 454.008 ms 2024-03-23T18:01:48.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:01:47 smithi086 ceph-mon[38221]: pgmap v348: 97 pgs: 97 active+clean; 271 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 1.9 MiB/s wr, 140 op/s 2024-03-23T18:01:48.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:01:47 smithi092 ceph-mon[79089]: pgmap v348: 97 pgs: 97 active+clean; 271 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 1.9 MiB/s wr, 140 op/s 2024-03-23T18:01:48.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:01:47 smithi152 ceph-mon[47949]: pgmap v348: 97 pgs: 97 active+clean; 271 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 1.9 MiB/s wr, 140 op/s 2024-03-23T18:01:48.924 INFO:tasks.workunit.client.0.smithi086.stdout: 1 671459 6.48 MB/sec execute 493 sec latency 13.928 ms 2024-03-23T18:01:49.924 INFO:tasks.workunit.client.0.smithi086.stdout: 1 672694 6.48 MB/sec execute 494 sec latency 16.848 ms 2024-03-23T18:01:50.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:01:49 smithi086 ceph-mon[38221]: pgmap v349: 97 pgs: 97 active+clean; 271 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 1.9 MiB/s wr, 140 op/s 2024-03-23T18:01:50.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:01:49 smithi092 ceph-mon[79089]: pgmap v349: 97 pgs: 97 active+clean; 271 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 1.9 MiB/s wr, 140 op/s 2024-03-23T18:01:50.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:01:49 smithi152 ceph-mon[47949]: pgmap v349: 97 pgs: 97 active+clean; 271 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 1.9 MiB/s wr, 140 op/s 2024-03-23T18:01:50.924 INFO:tasks.workunit.client.0.smithi086.stdout: 1 673620 6.47 MB/sec execute 495 sec latency 601.407 ms 2024-03-23T18:01:51.924 INFO:tasks.workunit.client.0.smithi086.stdout: 1 675074 6.48 MB/sec execute 496 sec latency 31.690 ms 2024-03-23T18:01:52.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:01:51 smithi086 ceph-mon[38221]: pgmap v350: 97 pgs: 97 active+clean; 263 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.1 MiB/s wr, 252 op/s 2024-03-23T18:01:52.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:01:51 smithi086 ceph-mon[38221]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:01:52.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:01:51 smithi092 ceph-mon[79089]: pgmap v350: 97 pgs: 97 active+clean; 263 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.1 MiB/s wr, 252 op/s 2024-03-23T18:01:52.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:01:51 smithi092 ceph-mon[79089]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:01:52.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:01:51 smithi152 ceph-mon[47949]: pgmap v350: 97 pgs: 97 active+clean; 263 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.1 MiB/s wr, 252 op/s 2024-03-23T18:01:52.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:01:51 smithi152 ceph-mon[47949]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:01:52.924 INFO:tasks.workunit.client.0.smithi086.stdout: 1 676325 6.48 MB/sec execute 497 sec latency 15.525 ms 2024-03-23T18:01:53.924 INFO:tasks.workunit.client.0.smithi086.stdout: 1 676458 6.46 MB/sec execute 498 sec latency 912.587 ms 2024-03-23T18:01:54.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:01:53 smithi086 ceph-mon[38221]: pgmap v351: 97 pgs: 97 active+clean; 263 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 1.9 MiB/s wr, 162 op/s 2024-03-23T18:01:54.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:01:53 smithi092 ceph-mon[79089]: pgmap v351: 97 pgs: 97 active+clean; 263 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 1.9 MiB/s wr, 162 op/s 2024-03-23T18:01:54.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:01:53 smithi152 ceph-mon[47949]: pgmap v351: 97 pgs: 97 active+clean; 263 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 1.9 MiB/s wr, 162 op/s 2024-03-23T18:01:54.924 INFO:tasks.workunit.client.0.smithi086.stdout: 1 677157 6.46 MB/sec execute 499 sec latency 1630.661 ms 2024-03-23T18:01:55.924 INFO:tasks.workunit.client.0.smithi086.stdout: 1 678691 6.47 MB/sec execute 500 sec latency 12.810 ms 2024-03-23T18:01:56.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:01:55 smithi086 ceph-mon[38221]: pgmap v352: 97 pgs: 97 active+clean; 267 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 234 op/s 2024-03-23T18:01:56.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:01:55 smithi092 ceph-mon[79089]: pgmap v352: 97 pgs: 97 active+clean; 267 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 234 op/s 2024-03-23T18:01:56.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:01:55 smithi152 ceph-mon[47949]: pgmap v352: 97 pgs: 97 active+clean; 267 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 234 op/s 2024-03-23T18:01:56.924 INFO:tasks.workunit.client.0.smithi086.stdout: 1 679968 6.46 MB/sec execute 501 sec latency 23.840 ms 2024-03-23T18:01:57.925 INFO:tasks.workunit.client.0.smithi086.stdout: 1 680451 6.45 MB/sec execute 502 sec latency 817.420 ms 2024-03-23T18:01:58.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:01:57 smithi086 ceph-mon[38221]: pgmap v353: 97 pgs: 97 active+clean; 267 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 840 KiB/s rd, 2.2 MiB/s wr, 184 op/s 2024-03-23T18:01:58.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:01:57 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T18:01:58.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:01:57 smithi092 ceph-mon[79089]: pgmap v353: 97 pgs: 97 active+clean; 267 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 840 KiB/s rd, 2.2 MiB/s wr, 184 op/s 2024-03-23T18:01:58.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:01:57 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T18:01:58.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:01:57 smithi152 ceph-mon[47949]: pgmap v353: 97 pgs: 97 active+clean; 267 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 840 KiB/s rd, 2.2 MiB/s wr, 184 op/s 2024-03-23T18:01:58.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:01:57 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T18:01:58.925 INFO:tasks.workunit.client.0.smithi086.stdout: 1 682126 6.47 MB/sec execute 503 sec latency 16.531 ms 2024-03-23T18:01:59.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:01:58 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T18:01:59.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:01:58 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T18:01:59.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:01:58 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T18:01:59.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:01:58 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T18:01:59.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:01:58 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T18:01:59.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:01:58 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T18:01:59.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:01:58 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T18:01:59.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:01:58 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T18:01:59.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:01:58 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T18:01:59.912 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.912+0000 7f6a1d800700 1 -- 172.21.15.86:0/746691083 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6a18073160 msgr2=0x7f6a18073540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:01:59.912 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.912+0000 7f6a1d800700 1 --2- 172.21.15.86:0/746691083 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6a18073160 0x7f6a18073540 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6a08009990 tx=0x7f6a08009ca0 comp rx=0 tx=0).stop 2024-03-23T18:01:59.912 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.912+0000 7f6a1d800700 1 -- 172.21.15.86:0/746691083 shutdown_connections 2024-03-23T18:01:59.912 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.912+0000 7f6a1d800700 1 --2- 172.21.15.86:0/746691083 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a1810d560 0x7f6a1810f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:59.912 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.912+0000 7f6a1d800700 1 --2- 172.21.15.86:0/746691083 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6a18073160 0x7f6a18073540 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:59.912 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.912+0000 7f6a1d800700 1 --2- 172.21.15.86:0/746691083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a18073a80 0x7f6a1810d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:59.912 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.912+0000 7f6a1d800700 1 -- 172.21.15.86:0/746691083 >> 172.21.15.86:0/746691083 conn(0x7f6a180fc7b0 msgr2=0x7f6a180febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:01:59.913 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.912+0000 7f6a1d800700 1 -- 172.21.15.86:0/746691083 shutdown_connections 2024-03-23T18:01:59.913 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.912+0000 7f6a1d800700 1 -- 172.21.15.86:0/746691083 wait complete. 2024-03-23T18:01:59.913 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.913+0000 7f6a1d800700 1 Processor -- start 2024-03-23T18:01:59.913 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.913+0000 7f6a1d800700 1 -- start start 2024-03-23T18:01:59.915 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.914+0000 7f6a1d800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a18073160 0x7f6a181993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:59.915 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.914+0000 7f6a1d800700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6a18073a80 0x7f6a18199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:59.915 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.914+0000 7f6a1d800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a1810d560 0x7f6a1819dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:59.915 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.914+0000 7f6a1d800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1819e290 con 0x7f6a18073160 2024-03-23T18:01:59.915 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.914+0000 7f6a1d800700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1819e400 con 0x7f6a1810d560 2024-03-23T18:01:59.915 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.914+0000 7f6a1d800700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1819e5c0 con 0x7f6a18073a80 2024-03-23T18:01:59.915 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.914+0000 7f6a16000700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6a18073a80 0x7f6a18199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:59.915 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.914+0000 7f6a16a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a18073160 0x7f6a181993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:59.915 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.914+0000 7f6a16000700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6a18073a80 0x7f6a18199900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:39416/0 (socket says 172.21.15.86:39416) 2024-03-23T18:01:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.914+0000 7f6a16000700 1 -- 172.21.15.86:0/1277254511 learned_addr learned my addr 172.21.15.86:0/1277254511 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:01:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.914+0000 7f6a17400700 1 --2- 172.21.15.86:0/1277254511 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a1810d560 0x7f6a1819dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.914+0000 7f6a16000700 1 -- 172.21.15.86:0/1277254511 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a1810d560 msgr2=0x7f6a1819dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:01:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.914+0000 7f6a16000700 1 --2- 172.21.15.86:0/1277254511 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a1810d560 0x7f6a1819dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.914+0000 7f6a16000700 1 -- 172.21.15.86:0/1277254511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a18073160 msgr2=0x7f6a181993c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:01:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.914+0000 7f6a16000700 1 --2- 172.21.15.86:0/1277254511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a18073160 0x7f6a181993c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.914+0000 7f6a16000700 1 -- 172.21.15.86:0/1277254511 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a080095f0 con 0x7f6a18073a80 2024-03-23T18:01:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.915+0000 7f6a16000700 1 --2- 172.21.15.86:0/1277254511 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6a18073a80 0x7f6a18199900 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6a0400e9a0 tx=0x7f6a0400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:01:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.915+0000 7f6a0f400700 1 -- 172.21.15.86:0/1277254511 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a04009790 con 0x7f6a18073a80 2024-03-23T18:01:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.915+0000 7f6a0f400700 1 -- 172.21.15.86:0/1277254511 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a04010430 con 0x7f6a18073a80 2024-03-23T18:01:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.915+0000 7f6a1d800700 1 -- 172.21.15.86:0/1277254511 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a1819e8a0 con 0x7f6a18073a80 2024-03-23T18:01:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.915+0000 7f6a0f400700 1 -- 172.21.15.86:0/1277254511 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a04018680 con 0x7f6a18073a80 2024-03-23T18:01:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.915+0000 7f6a1d800700 1 -- 172.21.15.86:0/1277254511 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a181a60e0 con 0x7f6a18073a80 2024-03-23T18:01:59.918 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.918+0000 7f6a0f400700 1 -- 172.21.15.86:0/1277254511 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f6a040105a0 con 0x7f6a18073a80 2024-03-23T18:01:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.921+0000 7f6a0f400700 1 --2- 172.21.15.86:0/1277254511 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f69fc0a1290 0x7f69fc0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.921+0000 7f6a31243b80 1 Processor -- start 2024-03-23T18:01:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.921+0000 7f6a0f400700 1 -- 172.21.15.86:0/1277254511 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f6a04014070 con 0x7f6a18073a80 2024-03-23T18:01:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.921+0000 7f6a16a00700 1 --2- 172.21.15.86:0/1277254511 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f69fc0a1290 0x7f69fc0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.921+0000 7f6a31243b80 1 -- start start 2024-03-23T18:01:59.923 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.921+0000 7f6a31243b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d1e79b100 0x556d1e79b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:59.923 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.921+0000 7f6a31243b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556d1e79ba20 0x556d1e7a6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:59.923 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.921+0000 7f6a31243b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d1e7a68d0 0x556d1e7a8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:59.923 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.922+0000 7f6a31243b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d1e6f4930 con 0x556d1e79b100 2024-03-23T18:01:59.923 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.922+0000 7f6a31243b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d1e6f22f0 con 0x556d1e7a68d0 2024-03-23T18:01:59.923 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.922+0000 7f6a31243b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d1e6edbe0 con 0x556d1e79ba20 2024-03-23T18:01:59.923 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.922+0000 7f6a16000700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556d1e79ba20 0x556d1e7a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:59.923 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.922+0000 7f6a16000700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556d1e79ba20 0x556d1e7a6390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:39420/0 (socket says 172.21.15.86:39420) 2024-03-23T18:01:59.923 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.922+0000 7f6a17400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d1e7a68d0 0x556d1e7a8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:59.923 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.922+0000 7f6a16000700 1 -- 172.21.15.86:0/416578028 learned_addr learned my addr 172.21.15.86:0/416578028 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:01:59.923 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.922+0000 7f6a16a00700 1 --2- 172.21.15.86:0/416578028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d1e79b100 0x556d1e79b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:59.924 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.922+0000 7f6a17400700 1 -- 172.21.15.86:0/416578028 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556d1e79ba20 msgr2=0x556d1e7a6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:01:59.924 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.922+0000 7f6a17400700 1 --2- 172.21.15.86:0/416578028 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556d1e79ba20 0x556d1e7a6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:59.924 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.922+0000 7f6a17400700 1 -- 172.21.15.86:0/416578028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d1e79b100 msgr2=0x556d1e79b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:01:59.924 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.922+0000 7f6a17400700 1 --2- 172.21.15.86:0/416578028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d1e79b100 0x556d1e79b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:59.924 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.922+0000 7f6a16a00700 1 --2- 172.21.15.86:0/1277254511 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f69fc0a1290 0x7f69fc0a3750 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6a08006010 tx=0x7f6a0800b260 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:01:59.924 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.922+0000 7f6a17400700 1 -- 172.21.15.86:0/416578028 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d1e56f350 con 0x556d1e7a68d0 2024-03-23T18:01:59.924 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.922+0000 7f6a16000700 1 --2- 172.21.15.86:0/416578028 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556d1e79ba20 0x556d1e7a6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T18:01:59.924 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.922+0000 7f6a17400700 1 --2- 172.21.15.86:0/416578028 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d1e7a68d0 0x556d1e7a8cc0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6a10011580 tx=0x7f6a10009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7d8aed1e61bbea8 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:01:59.924 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.923+0000 7f6a03e00700 1 -- 172.21.15.86:0/416578028 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a1000e040 con 0x556d1e7a68d0 2024-03-23T18:01:59.924 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.923+0000 7f6a03e00700 1 -- 172.21.15.86:0/416578028 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a1000f040 con 0x556d1e7a68d0 2024-03-23T18:01:59.924 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.923+0000 7f6a03e00700 1 -- 172.21.15.86:0/416578028 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a100044f0 con 0x556d1e7a68d0 2024-03-23T18:01:59.925 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.923+0000 7f6a31243b80 1 -- 172.21.15.86:0/416578028 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d1e7a68d0 msgr2=0x556d1e7a8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:01:59.925 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.923+0000 7f6a31243b80 1 --2- 172.21.15.86:0/416578028 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d1e7a68d0 0x556d1e7a8cc0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6a10011580 tx=0x7f6a10009920 comp rx=0 tx=0).stop 2024-03-23T18:01:59.925 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.923+0000 7f6a31243b80 1 -- 172.21.15.86:0/416578028 shutdown_connections 2024-03-23T18:01:59.925 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.923+0000 7f6a31243b80 1 --2- 172.21.15.86:0/416578028 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d1e7a68d0 0x556d1e7a8cc0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:59.926 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.923+0000 7f6a31243b80 1 --2- 172.21.15.86:0/416578028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d1e79b100 0x556d1e79b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:59.926 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.923+0000 7f6a31243b80 1 --2- 172.21.15.86:0/416578028 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556d1e79ba20 0x556d1e7a6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:59.926 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.923+0000 7f6a31243b80 1 -- 172.21.15.86:0/416578028 >> 172.21.15.86:0/416578028 conn(0x556d1e6fb640 msgr2=0x556d1e797520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:01:59.926 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.923+0000 7f6a31243b80 1 -- 172.21.15.86:0/416578028 shutdown_connections 2024-03-23T18:01:59.926 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.924+0000 7f6a31243b80 1 -- 172.21.15.86:0/416578028 wait complete. 2024-03-23T18:01:59.926 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.924+0000 7f6a31243b80 1 -- start start 2024-03-23T18:01:59.926 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.924+0000 7f6a31243b80 1 Processor -- start 2024-03-23T18:01:59.926 INFO:tasks.workunit.client.0.smithi086.stdout: 1 683353 6.46 MB/sec execute 504 sec latency 17.342 ms 2024-03-23T18:01:59.929 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.925+0000 7f6a31243b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d1e79b100 0x556d1e7a6290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:59.930 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.925+0000 7f6a31243b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556d1e79ba20 0x556d1e7a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:59.930 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.925+0000 7f6a31243b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d1e7a68d0 0x556d1e7a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:59.930 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.925+0000 7f6a31243b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d1e689d20 con 0x556d1e7a68d0 2024-03-23T18:01:59.931 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.925+0000 7f6a31243b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d1e6899c0 con 0x556d1e79b100 2024-03-23T18:01:59.931 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.925+0000 7f6a31243b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d1e571640 con 0x556d1e79ba20 2024-03-23T18:01:59.931 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.925+0000 7f6a31243b80 10 client.0 _ll_register_callbacks cb 0x556d1e693410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T18:01:59.931 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.925+0000 7f6a16a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d1e79b100 0x556d1e7a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:59.931 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.925+0000 7f6a16a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d1e79b100 0x556d1e7a6290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:53730/0 (socket says 172.21.15.86:53730) 2024-03-23T18:01:59.931 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.925+0000 7f6a16a00700 1 -- 172.21.15.86:0/522788543 learned_addr learned my addr 172.21.15.86:0/522788543 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:01:59.931 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.925+0000 7f6a16000700 1 --2- 172.21.15.86:0/522788543 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556d1e79ba20 0x556d1e7a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:59.931 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.925+0000 7f6a17400700 1 --2- 172.21.15.86:0/522788543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d1e7a68d0 0x556d1e7a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:59.931 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.926+0000 7f6a16000700 1 -- 172.21.15.86:0/522788543 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d1e79b100 msgr2=0x556d1e7a6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:01:59.931 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.926+0000 7f6a16000700 1 --2- 172.21.15.86:0/522788543 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d1e79b100 0x556d1e7a6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:59.931 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.926+0000 7f6a16000700 1 -- 172.21.15.86:0/522788543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d1e7a68d0 msgr2=0x556d1e7a3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:01:59.931 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.926+0000 7f6a16000700 1 --2- 172.21.15.86:0/522788543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d1e7a68d0 0x556d1e7a3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:59.931 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.926+0000 7f6a16000700 1 -- 172.21.15.86:0/522788543 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a181a60e0 con 0x556d1e79ba20 2024-03-23T18:01:59.932 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.926+0000 7f6a16a00700 1 --2- 172.21.15.86:0/522788543 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d1e79b100 0x556d1e7a6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T18:01:59.932 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.926+0000 7f6a17400700 1 --2- 172.21.15.86:0/522788543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d1e7a68d0 0x556d1e7a3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T18:01:59.932 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.926+0000 7f6a16000700 1 --2- 172.21.15.86:0/522788543 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556d1e79ba20 0x556d1e7a35d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6a04096a20 tx=0x7f6a0409a730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd33dae3fa03654b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:01:59.932 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.926+0000 7f6a00c00700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-03-23T18:01:59.932 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.926+0000 7f6a00c00700 1 -- 172.21.15.86:0/522788543 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a040996a0 con 0x556d1e79ba20 2024-03-23T18:01:59.932 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.926+0000 7f6a00c00700 1 -- 172.21.15.86:0/522788543 <== mon.1 v2:172.21.15.152:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f6a0409dde0 con 0x556d1e79ba20 2024-03-23T18:01:59.932 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.926+0000 7f6a31243b80 1 -- 172.21.15.86:0/522788543 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556d1e6ae320 con 0x556d1e79ba20 2024-03-23T18:01:59.932 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.926+0000 7f6a00c00700 1 -- 172.21.15.86:0/522788543 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a040a2430 con 0x556d1e79ba20 2024-03-23T18:01:59.932 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.926+0000 7f6a31243b80 10 client.26929 fetch_fsmap learned FSMap version 52 2024-03-23T18:01:59.932 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.926+0000 7f6a31243b80 1 -- 172.21.15.86:0/522788543 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x7f6a10011230 con 0x556d1e79ba20 2024-03-23T18:01:59.932 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.926+0000 7f6a00c00700 1 -- 172.21.15.86:0/522788543 <== mon.1 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a04063020 con 0x556d1e79ba20 2024-03-23T18:01:59.932 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.927+0000 7f6a00c00700 1 -- 172.21.15.86:0/522788543 <== mon.1 v2:172.21.15.152:3300/0 5 ==== fsmap(e 52) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f6a04094ce0 con 0x556d1e79ba20 2024-03-23T18:01:59.932 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.927+0000 7f6a31243b80 10 client.26929 fetch_fsmap finished waiting for FSMap version 52 2024-03-23T18:01:59.933 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.927+0000 7f6a31243b80 10 client.26929 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T18:01:59.933 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.927+0000 7f6a31243b80 20 client.26929 populate_metadata read hostname 'smithi086' 2024-03-23T18:01:59.933 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.927+0000 7f6a31243b80 1 --2- 172.21.15.86:0/522788543 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x556d1e7b6690 0x556d1e7b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:59.933 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.927+0000 7f6a31243b80 4 client.26929 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T18:01:59.933 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.927+0000 7f6a31243b80 1 -- 172.21.15.86:0/522788543 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556d1e6e66a0 con 0x556d1e7b6690 2024-03-23T18:01:59.933 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.927+0000 7f6a16a00700 1 --2- 172.21.15.86:0/522788543 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x556d1e7b6690 0x556d1e7b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:59.933 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.927+0000 7f6a16a00700 1 --2- 172.21.15.86:0/522788543 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x556d1e7b6690 0x556d1e7b8ab0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4eb05986fccb4d56 server_cookie=b572f59f46ee65eb in_seq=0 out_seq=0 2024-03-23T18:01:59.933 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.927+0000 7f6a00c00700 10 client.26929 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T18:01:59.933 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.928+0000 7f6a00c00700 1 -- 172.21.15.86:0/522788543 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x556d1e6e66a0 con 0x556d1e7b6690 2024-03-23T18:01:59.933 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.928+0000 7f6a00c00700 10 client.26929 handle_command_reply: tid=0 2024-03-23T18:01:59.933 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.928+0000 7f6a31243b80 1 client.26929 shutdown 2024-03-23T18:01:59.933 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.928+0000 7f6a31243b80 1 -- 172.21.15.86:0/522788543 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556d1e79ba20 msgr2=0x556d1e7a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:01:59.933 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.928+0000 7f6a31243b80 1 --2- 172.21.15.86:0/522788543 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556d1e79ba20 0x556d1e7a35d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6a04096a20 tx=0x7f6a0409a730 comp rx=0 tx=0).stop 2024-03-23T18:01:59.933 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.928+0000 7f6a31243b80 1 -- 172.21.15.86:0/522788543 shutdown_connections 2024-03-23T18:01:59.934 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.928+0000 7f6a31243b80 1 --2- 172.21.15.86:0/522788543 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x556d1e7b6690 0x556d1e7b8ab0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:59.934 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.928+0000 7f6a31243b80 1 --2- 172.21.15.86:0/522788543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d1e7a68d0 0x556d1e7a3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:59.934 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.928+0000 7f6a00c00700 0 client.26929 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T18:01:59.934 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.928+0000 7f6a31243b80 1 --2- 172.21.15.86:0/522788543 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d1e79b100 0x556d1e7a6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:59.934 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.928+0000 7f6a31243b80 1 --2- 172.21.15.86:0/522788543 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556d1e79ba20 0x556d1e7a35d0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:59.934 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.928+0000 7f6a31243b80 1 -- 172.21.15.86:0/522788543 >> 172.21.15.86:0/522788543 conn(0x556d1e6fb640 msgr2=0x556d1e79d0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:01:59.934 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.928+0000 7f6a31243b80 1 -- 172.21.15.86:0/522788543 shutdown_connections 2024-03-23T18:01:59.934 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.929+0000 7f6a31243b80 1 -- 172.21.15.86:0/522788543 wait complete. 2024-03-23T18:01:59.934 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.929+0000 7f6a31243b80 20 client.26929 trim_cache size 0 max 16384 2024-03-23T18:01:59.946 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.944+0000 7f6a31243b80 1 Processor -- start 2024-03-23T18:01:59.946 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.945+0000 7f6a31243b80 1 -- start start 2024-03-23T18:01:59.946 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.945+0000 7f6a31243b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d1e6fb640 0x556d1e7ad5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:59.947 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.945+0000 7f6a31243b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556d1e79b100 0x556d1e7a2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:59.947 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.945+0000 7f6a31243b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d1e79ba20 0x556d1e790c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:59.947 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.945+0000 7f6a31243b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d1e7930d0 con 0x556d1e6fb640 2024-03-23T18:01:59.947 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.945+0000 7f6a31243b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d1e7b3080 con 0x556d1e79ba20 2024-03-23T18:01:59.947 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.945+0000 7f6a31243b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d1e7a4520 con 0x556d1e79b100 2024-03-23T18:01:59.947 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.945+0000 7f6a16a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d1e6fb640 0x556d1e7ad5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:59.947 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.945+0000 7f6a17400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d1e79ba20 0x556d1e790c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:59.947 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.945+0000 7f6a16a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d1e6fb640 0x556d1e7ad5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42114/0 (socket says 172.21.15.86:42114) 2024-03-23T18:01:59.947 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.945+0000 7f6a16a00700 1 -- 172.21.15.86:0/2591091541 learned_addr learned my addr 172.21.15.86:0/2591091541 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:01:59.948 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.945+0000 7f6a16000700 1 --2- 172.21.15.86:0/2591091541 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556d1e79b100 0x556d1e7a2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:59.948 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.945+0000 7f6a16a00700 1 -- 172.21.15.86:0/2591091541 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556d1e79b100 msgr2=0x556d1e7a2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:01:59.948 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.945+0000 7f6a16a00700 1 --2- 172.21.15.86:0/2591091541 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556d1e79b100 0x556d1e7a2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:59.948 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.946+0000 7f6a16a00700 1 -- 172.21.15.86:0/2591091541 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d1e79ba20 msgr2=0x556d1e790c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:01:59.948 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.946+0000 7f6a16a00700 1 --2- 172.21.15.86:0/2591091541 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d1e79ba20 0x556d1e790c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:59.948 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.946+0000 7f6a16a00700 1 -- 172.21.15.86:0/2591091541 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d1e51bb00 con 0x556d1e6fb640 2024-03-23T18:01:59.948 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.946+0000 7f6a17400700 1 --2- 172.21.15.86:0/2591091541 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d1e79ba20 0x556d1e790c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T18:01:59.948 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.946+0000 7f6a16a00700 1 --2- 172.21.15.86:0/2591091541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d1e6fb640 0x556d1e7ad5e0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f6a08009990 tx=0x7f6a08022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d916f0b47dd160d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:01:59.948 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.946+0000 7f6a0cc00700 1 -- 172.21.15.86:0/2591091541 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a080343b0 con 0x556d1e6fb640 2024-03-23T18:01:59.948 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.946+0000 7f6a0cc00700 1 -- 172.21.15.86:0/2591091541 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a0802a040 con 0x556d1e6fb640 2024-03-23T18:01:59.948 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.946+0000 7f6a0cc00700 1 -- 172.21.15.86:0/2591091541 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a08004b50 con 0x556d1e6fb640 2024-03-23T18:01:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.947+0000 7f6a31243b80 1 -- 172.21.15.86:0/2591091541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d1e6fb640 msgr2=0x556d1e7ad5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:01:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.947+0000 7f6a31243b80 1 --2- 172.21.15.86:0/2591091541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d1e6fb640 0x556d1e7ad5e0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f6a08009990 tx=0x7f6a08022000 comp rx=0 tx=0).stop 2024-03-23T18:01:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.947+0000 7f6a31243b80 1 -- 172.21.15.86:0/2591091541 shutdown_connections 2024-03-23T18:01:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.947+0000 7f6a31243b80 1 --2- 172.21.15.86:0/2591091541 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d1e79ba20 0x556d1e790c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:59.952 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.947+0000 7f6a31243b80 1 --2- 172.21.15.86:0/2591091541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d1e6fb640 0x556d1e7ad5e0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:59.952 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.947+0000 7f6a31243b80 1 --2- 172.21.15.86:0/2591091541 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556d1e79b100 0x556d1e7a2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:59.952 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.947+0000 7f6a31243b80 1 -- 172.21.15.86:0/2591091541 >> 172.21.15.86:0/2591091541 conn(0x556d1e79eff0 msgr2=0x556d1e796d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:01:59.952 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.947+0000 7f6a31243b80 1 -- 172.21.15.86:0/2591091541 shutdown_connections 2024-03-23T18:01:59.952 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.947+0000 7f6a31243b80 1 -- 172.21.15.86:0/2591091541 wait complete. 2024-03-23T18:01:59.952 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.947+0000 7f6a31243b80 1 -- start start 2024-03-23T18:01:59.952 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.948+0000 7f6a31243b80 1 Processor -- start 2024-03-23T18:01:59.952 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.948+0000 7f6a31243b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d1e6fb640 0x556d1e7ab7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:59.952 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.948+0000 7f6a31243b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d1e79b100 0x556d1e7abd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:59.952 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.948+0000 7f6a31243b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556d1e79ba20 0x556d1e801140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:59.952 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.948+0000 7f6a31243b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d1e801680 con 0x556d1e79b100 2024-03-23T18:01:59.952 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.948+0000 7f6a31243b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d1e8017f0 con 0x556d1e6fb640 2024-03-23T18:01:59.952 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.948+0000 7f6a31243b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d1e8019b0 con 0x556d1e79ba20 2024-03-23T18:01:59.953 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.948+0000 7f6a31243b80 10 client.0 _ll_register_callbacks cb 0x556d1e693410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T18:01:59.953 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.948+0000 7f6a16a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d1e6fb640 0x556d1e7ab7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:59.953 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.948+0000 7f6a16a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d1e6fb640 0x556d1e7ab7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:53750/0 (socket says 172.21.15.86:53750) 2024-03-23T18:01:59.953 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.948+0000 7f6a16a00700 1 -- 172.21.15.86:0/730252491 learned_addr learned my addr 172.21.15.86:0/730252491 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:01:59.953 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.948+0000 7f6a17400700 1 --2- 172.21.15.86:0/730252491 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556d1e79ba20 0x556d1e801140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:59.953 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.948+0000 7f6a16000700 1 --2- 172.21.15.86:0/730252491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d1e79b100 0x556d1e7abd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:59.953 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.949+0000 7f6a16a00700 1 -- 172.21.15.86:0/730252491 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556d1e79ba20 msgr2=0x556d1e801140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:01:59.953 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.949+0000 7f6a16a00700 1 --2- 172.21.15.86:0/730252491 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556d1e79ba20 0x556d1e801140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:59.953 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.949+0000 7f6a16a00700 1 -- 172.21.15.86:0/730252491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d1e79b100 msgr2=0x556d1e7abd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:01:59.953 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.949+0000 7f6a16a00700 1 --2- 172.21.15.86:0/730252491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d1e79b100 0x556d1e7abd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:01:59.953 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.949+0000 7f6a16a00700 1 -- 172.21.15.86:0/730252491 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556d1e56f350 con 0x556d1e6fb640 2024-03-23T18:01:59.953 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.949+0000 7f6a16a00700 1 --2- 172.21.15.86:0/730252491 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d1e6fb640 0x556d1e7ab7d0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f6a08009ce0 tx=0x7f6a080341b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0cbacb07441511f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:01:59.954 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.949+0000 7f6a00c00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T18:01:59.954 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.949+0000 7f6a00c00700 1 -- 172.21.15.86:0/730252491 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a08034900 con 0x556d1e6fb640 2024-03-23T18:01:59.954 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.949+0000 7f6a31243b80 1 -- 172.21.15.86:0/730252491 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6a1000f040 con 0x556d1e6fb640 2024-03-23T18:01:59.954 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.949+0000 7f6a00c00700 1 -- 172.21.15.86:0/730252491 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f6a08034a60 con 0x556d1e6fb640 2024-03-23T18:01:59.954 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.950+0000 7f6a00c00700 1 -- 172.21.15.86:0/730252491 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a08034c20 con 0x556d1e6fb640 2024-03-23T18:01:59.954 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.950+0000 7f6a31243b80 10 client.26954 fetch_fsmap learned FSMap version 52 2024-03-23T18:01:59.954 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.950+0000 7f6a31243b80 1 -- 172.21.15.86:0/730252491 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x7f6a18110610 con 0x556d1e6fb640 2024-03-23T18:01:59.954 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.950+0000 7f6a00c00700 1 -- 172.21.15.86:0/730252491 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a1000f040 con 0x556d1e6fb640 2024-03-23T18:01:59.954 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.950+0000 7f6a00c00700 1 -- 172.21.15.86:0/730252491 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 52) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f6a0804f620 con 0x556d1e6fb640 2024-03-23T18:01:59.954 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.950+0000 7f6a31243b80 10 client.26954 fetch_fsmap finished waiting for FSMap version 52 2024-03-23T18:01:59.954 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.950+0000 7f6a31243b80 10 client.26954 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T18:01:59.954 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.950+0000 7f6a31243b80 20 client.26954 populate_metadata read hostname 'smithi086' 2024-03-23T18:01:59.954 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.950+0000 7f6a31243b80 1 --2- 172.21.15.86:0/730252491 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x556d1e7a68d0 0x556d1e80af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:01:59.955 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.950+0000 7f6a31243b80 4 client.26954 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T18:01:59.955 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.950+0000 7f6a31243b80 1 -- 172.21.15.86:0/730252491 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556d1e53cc90 con 0x556d1e7a68d0 2024-03-23T18:01:59.955 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.950+0000 7f6a16000700 1 --2- 172.21.15.86:0/730252491 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x556d1e7a68d0 0x556d1e80af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:01:59.955 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.951+0000 7f6a16000700 1 --2- 172.21.15.86:0/730252491 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x556d1e7a68d0 0x556d1e80af80 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de9f38bcd412927a server_cookie=3c772932f487c310 in_seq=0 out_seq=0 2024-03-23T18:01:59.955 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:01:59.951+0000 7f6a00c00700 10 client.26954 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T18:02:00.001 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:00.000+0000 7f6a00c00700 1 -- 172.21.15.86:0/730252491 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556d1e53cc90 con 0x556d1e7a68d0 2024-03-23T18:02:00.001 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:00.000+0000 7f6a00c00700 10 client.26954 handle_command_reply: tid=0 2024-03-23T18:02:00.001 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:00.000+0000 7f6a31243b80 1 client.26954 shutdown 2024-03-23T18:02:00.001 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:00.000+0000 7f6a31243b80 1 -- 172.21.15.86:0/730252491 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d1e6fb640 msgr2=0x556d1e7ab7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:02:00.001 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:00.000+0000 7f6a31243b80 1 --2- 172.21.15.86:0/730252491 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d1e6fb640 0x556d1e7ab7d0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f6a08009ce0 tx=0x7f6a080341b0 comp rx=0 tx=0).stop 2024-03-23T18:02:00.001 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:00.001+0000 7f6a31243b80 1 -- 172.21.15.86:0/730252491 shutdown_connections 2024-03-23T18:02:00.001 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:00.001+0000 7f6a31243b80 1 --2- 172.21.15.86:0/730252491 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x556d1e7a68d0 0x556d1e80af80 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:00.002 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:00.001+0000 7f6a31243b80 1 --2- 172.21.15.86:0/730252491 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556d1e79ba20 0x556d1e801140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:00.002 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:00.001+0000 7f6a31243b80 1 --2- 172.21.15.86:0/730252491 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556d1e6fb640 0x556d1e7ab7d0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:00.002 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:00.001+0000 7f6a00c00700 0 client.26954 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T18:02:00.002 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:00.001+0000 7f6a31243b80 1 --2- 172.21.15.86:0/730252491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d1e79b100 0x556d1e7abd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:00.002 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:00.001+0000 7f6a31243b80 1 -- 172.21.15.86:0/730252491 >> 172.21.15.86:0/730252491 conn(0x556d1e79eff0 msgr2=0x556d1e797450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:02:00.002 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:00.001+0000 7f6a31243b80 1 -- 172.21.15.86:0/730252491 shutdown_connections 2024-03-23T18:02:00.002 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:00.001+0000 7f6a31243b80 1 -- 172.21.15.86:0/730252491 wait complete. 2024-03-23T18:02:00.002 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:00.001+0000 7f6a31243b80 20 client.26954 trim_cache size 0 max 16384 2024-03-23T18:02:00.002 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-03-23T18:02:00.003 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-03-23T18:02:00.003 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-03-23T18:02:00.003 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-03-23T18:02:00.003 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:00.002+0000 7f6a1d800700 1 -- 172.21.15.86:0/1277254511 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f69fc0a1290 msgr2=0x7f69fc0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:02:00.004 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:00.002+0000 7f6a1d800700 1 --2- 172.21.15.86:0/1277254511 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f69fc0a1290 0x7f69fc0a3750 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6a08006010 tx=0x7f6a0800b260 comp rx=0 tx=0).stop 2024-03-23T18:02:00.004 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:00.002+0000 7f6a1d800700 1 -- 172.21.15.86:0/1277254511 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6a18073a80 msgr2=0x7f6a18199900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:02:00.004 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:00.002+0000 7f6a1d800700 1 --2- 172.21.15.86:0/1277254511 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6a18073a80 0x7f6a18199900 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6a0400e9a0 tx=0x7f6a0400ecb0 comp rx=0 tx=0).stop 2024-03-23T18:02:00.004 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:00.002+0000 7f6a1d800700 1 -- 172.21.15.86:0/1277254511 shutdown_connections 2024-03-23T18:02:00.004 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:00.002+0000 7f6a1d800700 1 --2- 172.21.15.86:0/1277254511 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a1810d560 0x7f6a1819dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:00.004 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:00.002+0000 7f6a1d800700 1 --2- 172.21.15.86:0/1277254511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a18073160 0x7f6a181993c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:00.004 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:00.002+0000 7f6a1d800700 1 --2- 172.21.15.86:0/1277254511 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f69fc0a1290 0x7f69fc0a3750 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:00.004 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:00.002+0000 7f6a1d800700 1 --2- 172.21.15.86:0/1277254511 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6a18073a80 0x7f6a18199900 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:00.004 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:00.002+0000 7f6a1d800700 1 -- 172.21.15.86:0/1277254511 >> 172.21.15.86:0/1277254511 conn(0x7f6a180fc7b0 msgr2=0x7f6a180fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:02:00.004 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:00.002+0000 7f6a1d800700 1 -- 172.21.15.86:0/1277254511 shutdown_connections 2024-03-23T18:02:00.004 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:00.002+0000 7f6a1d800700 1 -- 172.21.15.86:0/1277254511 wait complete. 2024-03-23T18:02:00.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:01:59 smithi086 ceph-mon[38221]: pgmap v354: 97 pgs: 97 active+clean; 267 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 840 KiB/s rd, 2.2 MiB/s wr, 184 op/s 2024-03-23T18:02:00.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:01:59 smithi092 ceph-mon[79089]: pgmap v354: 97 pgs: 97 active+clean; 267 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 840 KiB/s rd, 2.2 MiB/s wr, 184 op/s 2024-03-23T18:02:00.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:01:59 smithi152 ceph-mon[47949]: pgmap v354: 97 pgs: 97 active+clean; 267 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 840 KiB/s rd, 2.2 MiB/s wr, 184 op/s 2024-03-23T18:02:00.925 INFO:tasks.workunit.client.0.smithi086.stdout: 1 684129 6.45 MB/sec execute 505 sec latency 499.737 ms 2024-03-23T18:02:01.925 INFO:tasks.workunit.client.0.smithi086.stdout: 1 685479 6.46 MB/sec execute 506 sec latency 53.416 ms 2024-03-23T18:02:02.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:02:01 smithi086 ceph-mon[38221]: pgmap v355: 97 pgs: 97 active+clean; 271 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 273 op/s 2024-03-23T18:02:02.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:02:01 smithi092 ceph-mon[79089]: pgmap v355: 97 pgs: 97 active+clean; 271 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 273 op/s 2024-03-23T18:02:02.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:02:01 smithi152 ceph-mon[47949]: pgmap v355: 97 pgs: 97 active+clean; 271 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 273 op/s 2024-03-23T18:02:02.925 INFO:tasks.workunit.client.0.smithi086.stdout: 1 686578 6.46 MB/sec execute 507 sec latency 13.093 ms 2024-03-23T18:02:03.925 INFO:tasks.workunit.client.0.smithi086.stdout: 1 687092 6.45 MB/sec execute 508 sec latency 686.256 ms 2024-03-23T18:02:04.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:02:03 smithi086 ceph-mon[38221]: pgmap v356: 97 pgs: 97 active+clean; 271 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 811 KiB/s rd, 2.2 MiB/s wr, 160 op/s 2024-03-23T18:02:04.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:02:03 smithi092 ceph-mon[79089]: pgmap v356: 97 pgs: 97 active+clean; 271 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 811 KiB/s rd, 2.2 MiB/s wr, 160 op/s 2024-03-23T18:02:04.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:02:03 smithi152 ceph-mon[47949]: pgmap v356: 97 pgs: 97 active+clean; 271 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 811 KiB/s rd, 2.2 MiB/s wr, 160 op/s 2024-03-23T18:02:04.925 INFO:tasks.workunit.client.0.smithi086.stdout: 1 687807 6.44 MB/sec execute 509 sec latency 1388.418 ms 2024-03-23T18:02:05.925 INFO:tasks.workunit.client.0.smithi086.stdout: 1 689232 6.45 MB/sec execute 510 sec latency 35.954 ms 2024-03-23T18:02:06.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:02:05 smithi086 ceph-mon[38221]: pgmap v357: 97 pgs: 97 active+clean; 272 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.3 MiB/s wr, 286 op/s 2024-03-23T18:02:06.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:02:05 smithi086 ceph-mon[38221]: osdmap e95: 12 total, 12 up, 12 in 2024-03-23T18:02:06.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:02:05 smithi092 ceph-mon[79089]: pgmap v357: 97 pgs: 97 active+clean; 272 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.3 MiB/s wr, 286 op/s 2024-03-23T18:02:06.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:02:05 smithi092 ceph-mon[79089]: osdmap e95: 12 total, 12 up, 12 in 2024-03-23T18:02:06.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:02:05 smithi152 ceph-mon[47949]: pgmap v357: 97 pgs: 97 active+clean; 272 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.3 MiB/s wr, 286 op/s 2024-03-23T18:02:06.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:02:05 smithi152 ceph-mon[47949]: osdmap e95: 12 total, 12 up, 12 in 2024-03-23T18:02:06.925 INFO:tasks.workunit.client.0.smithi086.stdout: 1 690467 6.45 MB/sec execute 511 sec latency 21.782 ms 2024-03-23T18:02:07.926 INFO:tasks.workunit.client.0.smithi086.stdout: 1 691133 6.44 MB/sec execute 512 sec latency 725.639 ms 2024-03-23T18:02:08.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:02:07 smithi086 ceph-mon[38221]: pgmap v359: 97 pgs: 97 active+clean; 272 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.7 MiB/s wr, 256 op/s 2024-03-23T18:02:08.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:02:07 smithi092 ceph-mon[79089]: pgmap v359: 97 pgs: 97 active+clean; 272 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.7 MiB/s wr, 256 op/s 2024-03-23T18:02:08.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:02:07 smithi152 ceph-mon[47949]: pgmap v359: 97 pgs: 97 active+clean; 272 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.7 MiB/s wr, 256 op/s 2024-03-23T18:02:08.925 INFO:tasks.workunit.client.0.smithi086.stdout: 1 692800 6.45 MB/sec execute 513 sec latency 15.845 ms 2024-03-23T18:02:09.925 INFO:tasks.workunit.client.0.smithi086.stdout: 1 694113 6.45 MB/sec execute 514 sec latency 18.601 ms 2024-03-23T18:02:10.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:02:09 smithi086 ceph-mon[38221]: pgmap v360: 97 pgs: 97 active+clean; 272 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.7 MiB/s wr, 256 op/s 2024-03-23T18:02:10.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:02:09 smithi092 ceph-mon[79089]: pgmap v360: 97 pgs: 97 active+clean; 272 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.7 MiB/s wr, 256 op/s 2024-03-23T18:02:10.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:02:09 smithi152 ceph-mon[47949]: pgmap v360: 97 pgs: 97 active+clean; 272 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.7 MiB/s wr, 256 op/s 2024-03-23T18:02:10.925 INFO:tasks.workunit.client.0.smithi086.stdout: 1 694881 6.44 MB/sec execute 515 sec latency 650.925 ms 2024-03-23T18:02:11.925 INFO:tasks.workunit.client.0.smithi086.stdout: 1 696349 6.45 MB/sec execute 516 sec latency 24.099 ms 2024-03-23T18:02:12.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:02:11 smithi086 ceph-mon[38221]: pgmap v361: 97 pgs: 97 active+clean; 274 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 252 op/s 2024-03-23T18:02:12.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:02:11 smithi092 ceph-mon[79089]: pgmap v361: 97 pgs: 97 active+clean; 274 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 252 op/s 2024-03-23T18:02:12.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:02:11 smithi152 ceph-mon[47949]: pgmap v361: 97 pgs: 97 active+clean; 274 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 252 op/s 2024-03-23T18:02:12.926 INFO:tasks.workunit.client.0.smithi086.stdout: 1 697598 6.45 MB/sec execute 517 sec latency 19.881 ms 2024-03-23T18:02:13.155 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.155+0000 7f6ba1600700 1 -- 172.21.15.86:0/332083885 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b9c1025d0 msgr2=0x7f6b9c10aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:02:13.155 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.155+0000 7f6ba1600700 1 --2- 172.21.15.86:0/332083885 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b9c1025d0 0x7f6b9c10aab0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6b88009990 tx=0x7f6b88009ca0 comp rx=0 tx=0).stop 2024-03-23T18:02:13.155 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.155+0000 7f6ba1600700 1 -- 172.21.15.86:0/332083885 shutdown_connections 2024-03-23T18:02:13.155 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.155+0000 7f6ba1600700 1 --2- 172.21.15.86:0/332083885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b9c10b0d0 0x7f6b9c10d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:13.155 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.155+0000 7f6ba1600700 1 --2- 172.21.15.86:0/332083885 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b9c101cb0 0x7f6b9c102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:13.155 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.155+0000 7f6ba1600700 1 --2- 172.21.15.86:0/332083885 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b9c1025d0 0x7f6b9c10aab0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:13.156 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.155+0000 7f6ba1600700 1 -- 172.21.15.86:0/332083885 >> 172.21.15.86:0/332083885 conn(0x7f6b9c0fb4f0 msgr2=0x7f6b9c0fd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:02:13.156 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.155+0000 7f6ba1600700 1 -- 172.21.15.86:0/332083885 shutdown_connections 2024-03-23T18:02:13.156 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.155+0000 7f6ba1600700 1 -- 172.21.15.86:0/332083885 wait complete. 2024-03-23T18:02:13.158 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.156+0000 7f6ba1600700 1 Processor -- start 2024-03-23T18:02:13.158 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.156+0000 7f6ba1600700 1 -- start start 2024-03-23T18:02:13.158 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.156+0000 7f6ba1600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b9c101cb0 0x7f6b9c196fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:02:13.158 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.156+0000 7f6ba1600700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b9c1025d0 0x7f6b9c1974f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:02:13.158 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.156+0000 7f6ba1600700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b9c10b0d0 0x7f6b9c19b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:02:13.158 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.156+0000 7f6ba1600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b9c19bed0 con 0x7f6b9c101cb0 2024-03-23T18:02:13.158 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.156+0000 7f6ba1600700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b9c19c040 con 0x7f6b9c10b0d0 2024-03-23T18:02:13.158 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.156+0000 7f6ba1600700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b9c19c200 con 0x7f6b9c1025d0 2024-03-23T18:02:13.158 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.157+0000 7f6b9be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b9c101cb0 0x7f6b9c196fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:02:13.158 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.157+0000 7f6ba0c00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b9c10b0d0 0x7f6b9c19b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:02:13.158 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.157+0000 7f6b9b400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b9c1025d0 0x7f6b9c1974f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:02:13.158 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.157+0000 7f6ba0c00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b9c10b0d0 0x7f6b9c19b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:39148/0 (socket says 172.21.15.86:39148) 2024-03-23T18:02:13.159 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.157+0000 7f6b9b400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b9c1025d0 0x7f6b9c1974f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:52418/0 (socket says 172.21.15.86:52418) 2024-03-23T18:02:13.159 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.157+0000 7f6ba0c00700 1 -- 172.21.15.86:0/2454953385 learned_addr learned my addr 172.21.15.86:0/2454953385 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:02:13.159 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.157+0000 7f6b9be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b9c101cb0 0x7f6b9c196fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52948/0 (socket says 172.21.15.86:52948) 2024-03-23T18:02:13.159 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.157+0000 7f6ba0c00700 1 -- 172.21.15.86:0/2454953385 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b9c1025d0 msgr2=0x7f6b9c1974f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:02:13.159 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.157+0000 7f6ba0c00700 1 --2- 172.21.15.86:0/2454953385 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b9c1025d0 0x7f6b9c1974f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:13.159 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.157+0000 7f6ba0c00700 1 -- 172.21.15.86:0/2454953385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b9c101cb0 msgr2=0x7f6b9c196fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:02:13.159 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.157+0000 7f6ba0c00700 1 --2- 172.21.15.86:0/2454953385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b9c101cb0 0x7f6b9c196fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:13.159 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.157+0000 7f6ba0c00700 1 -- 172.21.15.86:0/2454953385 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b880095f0 con 0x7f6b9c10b0d0 2024-03-23T18:02:13.159 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.157+0000 7f6ba0c00700 1 --2- 172.21.15.86:0/2454953385 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b9c10b0d0 0x7f6b9c19b890 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f6b9400ef30 tx=0x7f6b9400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:02:13.159 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.157+0000 7f6b98c00700 1 -- 172.21.15.86:0/2454953385 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b94018070 con 0x7f6b9c10b0d0 2024-03-23T18:02:13.159 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.157+0000 7f6ba1600700 1 -- 172.21.15.86:0/2454953385 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b9c19c480 con 0x7f6b9c10b0d0 2024-03-23T18:02:13.159 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.157+0000 7f6b98c00700 1 -- 172.21.15.86:0/2454953385 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b9400f040 con 0x7f6b9c10b0d0 2024-03-23T18:02:13.159 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.157+0000 7f6b98c00700 1 -- 172.21.15.86:0/2454953385 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b9400cd00 con 0x7f6b9c10b0d0 2024-03-23T18:02:13.159 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.158+0000 7f6ba1600700 1 -- 172.21.15.86:0/2454953385 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b9c1a3d20 con 0x7f6b9c10b0d0 2024-03-23T18:02:13.160 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.160+0000 7f6b98c00700 1 -- 172.21.15.86:0/2454953385 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f6b94003a00 con 0x7f6b9c10b0d0 2024-03-23T18:02:13.162 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.162+0000 7f6bb66cfb80 1 Processor -- start 2024-03-23T18:02:13.162 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.162+0000 7f6bb66cfb80 1 -- start start 2024-03-23T18:02:13.162 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.162+0000 7f6b98c00700 1 --2- 172.21.15.86:0/2454953385 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f6b800a1290 0x7f6b800a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:02:13.162 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.162+0000 7f6bb66cfb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648f28892d0 0x5648f28896b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:02:13.163 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.162+0000 7f6bb66cfb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5648f2889bf0 0x5648f2898980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:02:13.164 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.162+0000 7f6b98c00700 1 -- 172.21.15.86:0/2454953385 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f6b940cc7d0 con 0x7f6b9c10b0d0 2024-03-23T18:02:13.164 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.162+0000 7f6bb66cfb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648f2898ec0 0x5648f289b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:02:13.164 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.162+0000 7f6bb66cfb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648f27e2930 con 0x5648f2898ec0 2024-03-23T18:02:13.164 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.162+0000 7f6bb66cfb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648f27e02f0 con 0x5648f2889bf0 2024-03-23T18:02:13.164 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.162+0000 7f6bb66cfb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648f27dbbe0 con 0x5648f28892d0 2024-03-23T18:02:13.164 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.162+0000 7f6b9b400700 1 --2- 172.21.15.86:0/2454953385 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f6b800a1290 0x7f6b800a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:02:13.164 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.162+0000 7f6ba0c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648f2898ec0 0x5648f289b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:02:13.164 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.162+0000 7f6b9be00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648f28892d0 0x5648f28896b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:02:13.164 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.162+0000 7f6ba0c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648f2898ec0 0x5648f289b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52962/0 (socket says 172.21.15.86:52962) 2024-03-23T18:02:13.164 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.162+0000 7f6ba0c00700 1 -- 172.21.15.86:0/3793468753 learned_addr learned my addr 172.21.15.86:0/3793468753 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:02:13.164 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.163+0000 7f6b9b400700 1 --2- 172.21.15.86:0/3793468753 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5648f2889bf0 0x5648f2898980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:02:13.164 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.163+0000 7f6b9be00700 1 -- 172.21.15.86:0/3793468753 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5648f2889bf0 msgr2=0x5648f2898980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:02:13.164 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.163+0000 7f6b9be00700 1 --2- 172.21.15.86:0/3793468753 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5648f2889bf0 0x5648f2898980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:13.165 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.163+0000 7f6b9be00700 1 -- 172.21.15.86:0/3793468753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648f2898ec0 msgr2=0x5648f289b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:02:13.165 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.163+0000 7f6b9be00700 1 --2- 172.21.15.86:0/3793468753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648f2898ec0 0x5648f289b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:13.165 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.163+0000 7f6b9be00700 1 -- 172.21.15.86:0/3793468753 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5648f265d350 con 0x5648f28892d0 2024-03-23T18:02:13.165 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.163+0000 7f6b9b400700 1 --2- 172.21.15.86:0/2454953385 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f6b800a1290 0x7f6b800a3750 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c198500 tx=0x7f6b88005320 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:02:13.165 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.163+0000 7f6b9be00700 1 --2- 172.21.15.86:0/3793468753 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648f28892d0 0x5648f28896b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c009c30 tx=0x7f6b8c009f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df55c22a7129555c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:02:13.165 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.163+0000 7f6b91200700 1 -- 172.21.15.86:0/3793468753 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c011cb0 con 0x5648f28892d0 2024-03-23T18:02:13.165 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.163+0000 7f6b91200700 1 -- 172.21.15.86:0/3793468753 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b8c011e10 con 0x5648f28892d0 2024-03-23T18:02:13.165 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.163+0000 7f6b91200700 1 -- 172.21.15.86:0/3793468753 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c005280 con 0x5648f28892d0 2024-03-23T18:02:13.169 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.164+0000 7f6bb66cfb80 1 -- 172.21.15.86:0/3793468753 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648f28892d0 msgr2=0x5648f28896b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:02:13.169 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.164+0000 7f6bb66cfb80 1 --2- 172.21.15.86:0/3793468753 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648f28892d0 0x5648f28896b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c009c30 tx=0x7f6b8c009f40 comp rx=0 tx=0).stop 2024-03-23T18:02:13.169 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.164+0000 7f6bb66cfb80 1 -- 172.21.15.86:0/3793468753 shutdown_connections 2024-03-23T18:02:13.169 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.164+0000 7f6bb66cfb80 1 --2- 172.21.15.86:0/3793468753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648f2898ec0 0x5648f289b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:13.169 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.164+0000 7f6bb66cfb80 1 --2- 172.21.15.86:0/3793468753 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648f28892d0 0x5648f28896b0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:13.170 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.164+0000 7f6bb66cfb80 1 --2- 172.21.15.86:0/3793468753 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5648f2889bf0 0x5648f2898980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:13.170 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.164+0000 7f6bb66cfb80 1 -- 172.21.15.86:0/3793468753 >> 172.21.15.86:0/3793468753 conn(0x5648f27e9640 msgr2=0x5648f2885750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:02:13.170 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.164+0000 7f6bb66cfb80 1 -- 172.21.15.86:0/3793468753 shutdown_connections 2024-03-23T18:02:13.170 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.164+0000 7f6bb66cfb80 1 -- 172.21.15.86:0/3793468753 wait complete. 2024-03-23T18:02:13.170 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.165+0000 7f6bb66cfb80 1 -- start start 2024-03-23T18:02:13.170 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.165+0000 7f6bb66cfb80 1 Processor -- start 2024-03-23T18:02:13.170 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.165+0000 7f6bb66cfb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5648f28892d0 0x5648f28988f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:02:13.170 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.165+0000 7f6bb66cfb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648f2889bf0 0x5648f2895d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:02:13.170 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.165+0000 7f6bb66cfb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648f2898ec0 0x5648f2896460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:02:13.170 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.165+0000 7f6bb66cfb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648f2777d20 con 0x5648f2898ec0 2024-03-23T18:02:13.170 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.165+0000 7f6bb66cfb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648f27779c0 con 0x5648f28892d0 2024-03-23T18:02:13.170 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.165+0000 7f6bb66cfb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648f265f640 con 0x5648f2889bf0 2024-03-23T18:02:13.170 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.165+0000 7f6bb66cfb80 10 client.0 _ll_register_callbacks cb 0x5648f2781410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T18:02:13.171 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.166+0000 7f6b9b400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648f2889bf0 0x5648f2895d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:02:13.171 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.166+0000 7f6ba0c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648f2898ec0 0x5648f2896460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:02:13.171 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.166+0000 7f6b9be00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5648f28892d0 0x5648f28988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:02:13.171 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.166+0000 7f6b9b400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648f2889bf0 0x5648f2895d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:52446/0 (socket says 172.21.15.86:52446) 2024-03-23T18:02:13.171 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.166+0000 7f6b9be00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5648f28892d0 0x5648f28988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:39160/0 (socket says 172.21.15.86:39160) 2024-03-23T18:02:13.171 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.166+0000 7f6b9b400700 1 -- 172.21.15.86:0/3045887307 learned_addr learned my addr 172.21.15.86:0/3045887307 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:02:13.171 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.166+0000 7f6ba0c00700 1 -- 172.21.15.86:0/3045887307 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648f2889bf0 msgr2=0x5648f2895d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:02:13.171 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.166+0000 7f6ba0c00700 1 --2- 172.21.15.86:0/3045887307 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648f2889bf0 0x5648f2895d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:13.171 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.166+0000 7f6ba0c00700 1 -- 172.21.15.86:0/3045887307 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5648f28892d0 msgr2=0x5648f28988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:02:13.171 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.166+0000 7f6ba0c00700 1 --2- 172.21.15.86:0/3045887307 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5648f28892d0 0x5648f28988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:13.171 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.166+0000 7f6ba0c00700 1 -- 172.21.15.86:0/3045887307 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b8c0098e0 con 0x5648f2898ec0 2024-03-23T18:02:13.171 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.166+0000 7f6ba0c00700 1 --2- 172.21.15.86:0/3045887307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648f2898ec0 0x5648f2896460 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f6b940a17f0 tx=0x7f6b94099560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c662722738e25041 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:02:13.171 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.166+0000 7f6b85600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-23T18:02:13.172 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.166+0000 7f6b85600700 1 -- 172.21.15.86:0/3045887307 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b94084040 con 0x5648f2898ec0 2024-03-23T18:02:13.172 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.166+0000 7f6b85600700 1 -- 172.21.15.86:0/3045887307 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f6b9401b320 con 0x5648f2898ec0 2024-03-23T18:02:13.172 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.166+0000 7f6bb66cfb80 1 -- 172.21.15.86:0/3045887307 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5648f279c320 con 0x5648f2898ec0 2024-03-23T18:02:13.172 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.167+0000 7f6b85600700 1 -- 172.21.15.86:0/3045887307 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6b9409a030 con 0x5648f2898ec0 2024-03-23T18:02:13.172 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.167+0000 7f6bb66cfb80 10 client.16911 fetch_fsmap learned FSMap version 52 2024-03-23T18:02:13.172 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.167+0000 7f6bb66cfb80 1 -- 172.21.15.86:0/3045887307 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x5648f2609b00 con 0x5648f2898ec0 2024-03-23T18:02:13.172 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.167+0000 7f6b85600700 1 -- 172.21.15.86:0/3045887307 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5648f279c320 con 0x5648f2898ec0 2024-03-23T18:02:13.172 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.167+0000 7f6b85600700 1 -- 172.21.15.86:0/3045887307 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 52) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f6b94098e20 con 0x5648f2898ec0 2024-03-23T18:02:13.172 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.167+0000 7f6bb66cfb80 10 client.16911 fetch_fsmap finished waiting for FSMap version 52 2024-03-23T18:02:13.172 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.167+0000 7f6bb66cfb80 10 client.16911 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T18:02:13.172 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.167+0000 7f6bb66cfb80 20 client.16911 populate_metadata read hostname 'smithi086' 2024-03-23T18:02:13.173 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.167+0000 7f6bb66cfb80 1 --2- 172.21.15.86:0/3045887307 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5648f28a8aa0 0x5648f28aaec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:02:13.173 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.167+0000 7f6bb66cfb80 4 client.16911 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T18:02:13.173 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.167+0000 7f6bb66cfb80 1 -- 172.21.15.86:0/3045887307 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5648f27d46a0 con 0x5648f28a8aa0 2024-03-23T18:02:13.173 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.167+0000 7f6b9be00700 1 --2- 172.21.15.86:0/3045887307 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5648f28a8aa0 0x5648f28aaec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:02:13.173 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.168+0000 7f6b85600700 10 client.16911 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T18:02:13.173 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.168+0000 7f6b9be00700 1 --2- 172.21.15.86:0/3045887307 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5648f28a8aa0 0x5648f28aaec0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f820e3288dd4a40 server_cookie=b28cfff943c16ef6 in_seq=0 out_seq=0 2024-03-23T18:02:13.173 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.168+0000 7f6b85600700 1 -- 172.21.15.86:0/3045887307 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5648f27d46a0 con 0x5648f28a8aa0 2024-03-23T18:02:13.173 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.168+0000 7f6b85600700 10 client.16911 handle_command_reply: tid=0 2024-03-23T18:02:13.173 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.168+0000 7f6bb66cfb80 1 client.16911 shutdown 2024-03-23T18:02:13.173 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.169+0000 7f6bb66cfb80 1 -- 172.21.15.86:0/3045887307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648f2898ec0 msgr2=0x5648f2896460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:02:13.174 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.169+0000 7f6bb66cfb80 1 --2- 172.21.15.86:0/3045887307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648f2898ec0 0x5648f2896460 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f6b940a17f0 tx=0x7f6b94099560 comp rx=0 tx=0).stop 2024-03-23T18:02:13.174 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.169+0000 7f6bb66cfb80 1 -- 172.21.15.86:0/3045887307 shutdown_connections 2024-03-23T18:02:13.174 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.169+0000 7f6bb66cfb80 1 --2- 172.21.15.86:0/3045887307 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5648f28a8aa0 0x5648f28aaec0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:13.174 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.169+0000 7f6bb66cfb80 1 --2- 172.21.15.86:0/3045887307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648f2898ec0 0x5648f2896460 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:13.174 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.169+0000 7f6b85600700 0 client.16911 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T18:02:13.174 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.169+0000 7f6bb66cfb80 1 --2- 172.21.15.86:0/3045887307 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5648f28892d0 0x5648f28988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:13.174 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.169+0000 7f6bb66cfb80 1 --2- 172.21.15.86:0/3045887307 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648f2889bf0 0x5648f2895d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:13.174 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.169+0000 7f6bb66cfb80 1 -- 172.21.15.86:0/3045887307 >> 172.21.15.86:0/3045887307 conn(0x5648f27e9640 msgr2=0x5648f288b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:02:13.174 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.169+0000 7f6bb66cfb80 1 -- 172.21.15.86:0/3045887307 shutdown_connections 2024-03-23T18:02:13.174 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.169+0000 7f6bb66cfb80 1 -- 172.21.15.86:0/3045887307 wait complete. 2024-03-23T18:02:13.174 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.169+0000 7f6bb66cfb80 20 client.16911 trim_cache size 0 max 16384 2024-03-23T18:02:13.184 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.184+0000 7f6bb66cfb80 1 Processor -- start 2024-03-23T18:02:13.185 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.184+0000 7f6bb66cfb80 1 -- start start 2024-03-23T18:02:13.185 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.184+0000 7f6bb66cfb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648f28892d0 0x5648f289fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:02:13.185 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.184+0000 7f6bb66cfb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5648f2889bf0 0x5648f287f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:02:13.185 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.184+0000 7f6bb66cfb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648f2898ec0 0x5648f287e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:02:13.185 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.184+0000 7f6bb66cfb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648f2889fd0 con 0x5648f2898ec0 2024-03-23T18:02:13.185 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.184+0000 7f6bb66cfb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648f2881370 con 0x5648f2889bf0 2024-03-23T18:02:13.185 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.184+0000 7f6bb66cfb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648f2890b70 con 0x5648f28892d0 2024-03-23T18:02:13.185 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.184+0000 7f6ba0c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648f2898ec0 0x5648f287e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:02:13.185 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.184+0000 7f6b9be00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648f28892d0 0x5648f289fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:02:13.185 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.184+0000 7f6b9b400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5648f2889bf0 0x5648f287f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:02:13.185 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.184+0000 7f6b9be00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648f28892d0 0x5648f289fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:52458/0 (socket says 172.21.15.86:52458) 2024-03-23T18:02:13.185 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.184+0000 7f6ba0c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648f2898ec0 0x5648f287e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52984/0 (socket says 172.21.15.86:52984) 2024-03-23T18:02:13.185 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.184+0000 7f6b9be00700 1 -- 172.21.15.86:0/1425582414 learned_addr learned my addr 172.21.15.86:0/1425582414 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:02:13.185 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.185+0000 7f6b9b400700 1 -- 172.21.15.86:0/1425582414 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648f28892d0 msgr2=0x5648f289fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:02:13.186 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.185+0000 7f6b9b400700 1 --2- 172.21.15.86:0/1425582414 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648f28892d0 0x5648f289fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:13.186 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.185+0000 7f6b9b400700 1 -- 172.21.15.86:0/1425582414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648f2898ec0 msgr2=0x5648f287e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:02:13.186 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.185+0000 7f6b9b400700 1 --2- 172.21.15.86:0/1425582414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648f2898ec0 0x5648f287e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:13.186 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.185+0000 7f6b9b400700 1 -- 172.21.15.86:0/1425582414 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5648f27a1e40 con 0x5648f2889bf0 2024-03-23T18:02:13.189 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.185+0000 7f6b9b400700 1 --2- 172.21.15.86:0/1425582414 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5648f2889bf0 0x5648f287f700 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x5648f28971b0 tx=0x7f6b88027000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb0c24b9c8519f6a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:02:13.189 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.185+0000 7f6b91c00700 1 -- 172.21.15.86:0/1425582414 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b88037070 con 0x5648f2889bf0 2024-03-23T18:02:13.189 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.185+0000 7f6b91c00700 1 -- 172.21.15.86:0/1425582414 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b8800bed0 con 0x5648f2889bf0 2024-03-23T18:02:13.189 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.185+0000 7f6b91c00700 1 -- 172.21.15.86:0/1425582414 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b8801d070 con 0x5648f2889bf0 2024-03-23T18:02:13.190 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.186+0000 7f6bb66cfb80 1 -- 172.21.15.86:0/1425582414 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5648f2889bf0 msgr2=0x5648f287f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:02:13.190 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.186+0000 7f6bb66cfb80 1 --2- 172.21.15.86:0/1425582414 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5648f2889bf0 0x5648f287f700 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x5648f28971b0 tx=0x7f6b88027000 comp rx=0 tx=0).stop 2024-03-23T18:02:13.190 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.186+0000 7f6bb66cfb80 1 -- 172.21.15.86:0/1425582414 shutdown_connections 2024-03-23T18:02:13.190 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.186+0000 7f6bb66cfb80 1 --2- 172.21.15.86:0/1425582414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648f2898ec0 0x5648f287e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:13.190 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.186+0000 7f6bb66cfb80 1 --2- 172.21.15.86:0/1425582414 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648f28892d0 0x5648f289fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:13.190 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.186+0000 7f6bb66cfb80 1 --2- 172.21.15.86:0/1425582414 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5648f2889bf0 0x5648f287f700 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:13.190 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.186+0000 7f6bb66cfb80 1 -- 172.21.15.86:0/1425582414 >> 172.21.15.86:0/1425582414 conn(0x5648f27e9640 msgr2=0x5648f28955b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:02:13.190 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.186+0000 7f6bb66cfb80 1 -- 172.21.15.86:0/1425582414 shutdown_connections 2024-03-23T18:02:13.190 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.186+0000 7f6bb66cfb80 1 -- 172.21.15.86:0/1425582414 wait complete. 2024-03-23T18:02:13.190 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.186+0000 7f6bb66cfb80 1 -- start start 2024-03-23T18:02:13.190 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.187+0000 7f6bb66cfb80 1 Processor -- start 2024-03-23T18:02:13.190 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.187+0000 7f6bb66cfb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648f28892d0 0x5648f289f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:02:13.191 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.187+0000 7f6bb66cfb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5648f2889bf0 0x5648f289deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:02:13.191 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.187+0000 7f6bb66cfb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648f2898ec0 0x5648f289e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:02:13.191 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.187+0000 7f6bb66cfb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648f289ebf0 con 0x5648f28892d0 2024-03-23T18:02:13.191 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.187+0000 7f6bb66cfb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648f289ed60 con 0x5648f2889bf0 2024-03-23T18:02:13.191 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.187+0000 7f6bb66cfb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648f28fa650 con 0x5648f2898ec0 2024-03-23T18:02:13.191 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.187+0000 7f6bb66cfb80 10 client.0 _ll_register_callbacks cb 0x5648f2781410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T18:02:13.191 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.187+0000 7f6b9be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648f28892d0 0x5648f289f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:02:13.191 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.187+0000 7f6b9be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648f28892d0 0x5648f289f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52986/0 (socket says 172.21.15.86:52986) 2024-03-23T18:02:13.191 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.187+0000 7f6b9be00700 1 -- 172.21.15.86:0/2102874573 learned_addr learned my addr 172.21.15.86:0/2102874573 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:02:13.191 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.187+0000 7f6b9b400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5648f2889bf0 0x5648f289deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:02:13.191 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.187+0000 7f6ba0c00700 1 --2- 172.21.15.86:0/2102874573 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648f2898ec0 0x5648f289e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:02:13.191 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.188+0000 7f6b9be00700 1 -- 172.21.15.86:0/2102874573 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648f2898ec0 msgr2=0x5648f289e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:02:13.191 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.188+0000 7f6b9be00700 1 --2- 172.21.15.86:0/2102874573 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648f2898ec0 0x5648f289e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:13.191 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.188+0000 7f6b9be00700 1 -- 172.21.15.86:0/2102874573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5648f2889bf0 msgr2=0x5648f289deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:02:13.192 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.188+0000 7f6b9be00700 1 --2- 172.21.15.86:0/2102874573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5648f2889bf0 0x5648f289deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:13.192 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.188+0000 7f6b9be00700 1 -- 172.21.15.86:0/2102874573 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b88020040 con 0x5648f28892d0 2024-03-23T18:02:13.192 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.188+0000 7f6ba0c00700 1 --2- 172.21.15.86:0/2102874573 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648f2898ec0 0x5648f289e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T18:02:13.192 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.188+0000 7f6b9b400700 1 --2- 172.21.15.86:0/2102874573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5648f2889bf0 0x5648f289deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T18:02:13.192 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.188+0000 7f6b9be00700 1 --2- 172.21.15.86:0/2102874573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648f28892d0 0x5648f289f9d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c0118a0 tx=0x7f6b8c009710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47d4b0b699e986f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:02:13.192 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.188+0000 7f6b85600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-23T18:02:13.192 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.188+0000 7f6b85600700 1 -- 172.21.15.86:0/2102874573 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c02b040 con 0x5648f28892d0 2024-03-23T18:02:13.192 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.188+0000 7f6bb66cfb80 1 -- 172.21.15.86:0/2102874573 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6b8c011e10 con 0x5648f28892d0 2024-03-23T18:02:13.192 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.188+0000 7f6b85600700 1 -- 172.21.15.86:0/2102874573 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f6b8c011ab0 con 0x5648f28892d0 2024-03-23T18:02:13.192 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.188+0000 7f6b85600700 1 -- 172.21.15.86:0/2102874573 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6b8c004bb0 con 0x5648f28892d0 2024-03-23T18:02:13.192 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.188+0000 7f6bb66cfb80 10 client.16917 fetch_fsmap learned FSMap version 52 2024-03-23T18:02:13.192 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.189+0000 7f6bb66cfb80 1 -- 172.21.15.86:0/2102874573 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x5648f2661c70 con 0x5648f28892d0 2024-03-23T18:02:13.192 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.189+0000 7f6b85600700 1 -- 172.21.15.86:0/2102874573 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b8c03c080 con 0x5648f28892d0 2024-03-23T18:02:13.192 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.189+0000 7f6b85600700 1 -- 172.21.15.86:0/2102874573 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 52) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f6b8c036030 con 0x5648f28892d0 2024-03-23T18:02:13.193 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.189+0000 7f6bb66cfb80 10 client.16917 fetch_fsmap finished waiting for FSMap version 52 2024-03-23T18:02:13.193 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.189+0000 7f6bb66cfb80 10 client.16917 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T18:02:13.193 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.189+0000 7f6bb66cfb80 20 client.16917 populate_metadata read hostname 'smithi086' 2024-03-23T18:02:13.193 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.189+0000 7f6bb66cfb80 1 --2- 172.21.15.86:0/2102874573 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5648f28fbe30 0x5648f28fe250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:02:13.193 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.189+0000 7f6bb66cfb80 4 client.16917 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T18:02:13.193 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.189+0000 7f6bb66cfb80 1 -- 172.21.15.86:0/2102874573 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5648f262ac90 con 0x5648f28fbe30 2024-03-23T18:02:13.193 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.189+0000 7f6b9b400700 1 --2- 172.21.15.86:0/2102874573 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5648f28fbe30 0x5648f28fe250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:02:13.193 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.190+0000 7f6b9b400700 1 --2- 172.21.15.86:0/2102874573 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5648f28fbe30 0x5648f28fe250 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e3ce24d2568dca3 server_cookie=ab3df72a3566acfb in_seq=0 out_seq=0 2024-03-23T18:02:13.193 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.190+0000 7f6b85600700 10 client.16917 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T18:02:13.239 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.239+0000 7f6b85600700 1 -- 172.21.15.86:0/2102874573 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5648f262ac90 con 0x5648f28fbe30 2024-03-23T18:02:13.239 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.239+0000 7f6b85600700 10 client.16917 handle_command_reply: tid=0 2024-03-23T18:02:13.240 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.239+0000 7f6bb66cfb80 1 client.16917 shutdown 2024-03-23T18:02:13.240 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.240+0000 7f6bb66cfb80 1 -- 172.21.15.86:0/2102874573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648f28892d0 msgr2=0x5648f289f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:02:13.240 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.240+0000 7f6bb66cfb80 1 --2- 172.21.15.86:0/2102874573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648f28892d0 0x5648f289f9d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c0118a0 tx=0x7f6b8c009710 comp rx=0 tx=0).stop 2024-03-23T18:02:13.241 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.241+0000 7f6bb66cfb80 1 -- 172.21.15.86:0/2102874573 shutdown_connections 2024-03-23T18:02:13.241 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.241+0000 7f6bb66cfb80 1 --2- 172.21.15.86:0/2102874573 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5648f28fbe30 0x5648f28fe250 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:13.241 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.241+0000 7f6bb66cfb80 1 --2- 172.21.15.86:0/2102874573 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648f2898ec0 0x5648f289e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:13.241 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.241+0000 7f6bb66cfb80 1 --2- 172.21.15.86:0/2102874573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648f28892d0 0x5648f289f9d0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:13.242 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.241+0000 7f6bb66cfb80 1 --2- 172.21.15.86:0/2102874573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5648f2889bf0 0x5648f289deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:13.242 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.241+0000 7f6bb66cfb80 1 -- 172.21.15.86:0/2102874573 >> 172.21.15.86:0/2102874573 conn(0x5648f27e9640 msgr2=0x5648f288b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:02:13.242 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.242+0000 7f6b85600700 0 client.16917 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T18:02:13.242 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.242+0000 7f6bb66cfb80 1 -- 172.21.15.86:0/2102874573 shutdown_connections 2024-03-23T18:02:13.243 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.242+0000 7f6bb66cfb80 1 -- 172.21.15.86:0/2102874573 wait complete. 2024-03-23T18:02:13.243 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.242+0000 7f6bb66cfb80 20 client.16917 trim_cache size 0 max 16384 2024-03-23T18:02:13.243 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-03-23T18:02:13.243 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-03-23T18:02:13.243 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-03-23T18:02:13.243 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-03-23T18:02:13.244 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.243+0000 7f6ba1600700 1 -- 172.21.15.86:0/2454953385 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f6b800a1290 msgr2=0x7f6b800a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:02:13.244 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.243+0000 7f6ba1600700 1 --2- 172.21.15.86:0/2454953385 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f6b800a1290 0x7f6b800a3750 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c198500 tx=0x7f6b88005320 comp rx=0 tx=0).stop 2024-03-23T18:02:13.244 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.243+0000 7f6ba1600700 1 -- 172.21.15.86:0/2454953385 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b9c10b0d0 msgr2=0x7f6b9c19b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:02:13.244 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.243+0000 7f6ba1600700 1 --2- 172.21.15.86:0/2454953385 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b9c10b0d0 0x7f6b9c19b890 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f6b9400ef30 tx=0x7f6b9400c3a0 comp rx=0 tx=0).stop 2024-03-23T18:02:13.245 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.244+0000 7f6ba1600700 1 -- 172.21.15.86:0/2454953385 shutdown_connections 2024-03-23T18:02:13.245 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.244+0000 7f6ba1600700 1 --2- 172.21.15.86:0/2454953385 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b9c10b0d0 0x7f6b9c19b890 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:13.245 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.244+0000 7f6ba1600700 1 --2- 172.21.15.86:0/2454953385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b9c101cb0 0x7f6b9c196fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:13.245 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.244+0000 7f6ba1600700 1 --2- 172.21.15.86:0/2454953385 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f6b800a1290 0x7f6b800a3750 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:13.245 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.244+0000 7f6ba1600700 1 --2- 172.21.15.86:0/2454953385 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b9c1025d0 0x7f6b9c1974f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:13.245 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.244+0000 7f6ba1600700 1 -- 172.21.15.86:0/2454953385 >> 172.21.15.86:0/2454953385 conn(0x7f6b9c0fb4f0 msgr2=0x7f6b9c0fc2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:02:13.245 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.244+0000 7f6ba1600700 1 -- 172.21.15.86:0/2454953385 shutdown_connections 2024-03-23T18:02:13.245 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:13.244+0000 7f6ba1600700 1 -- 172.21.15.86:0/2454953385 wait complete. 2024-03-23T18:02:13.926 INFO:tasks.workunit.client.0.smithi086.stdout: 1 699206 6.45 MB/sec execute 518 sec latency 214.887 ms 2024-03-23T18:02:14.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:02:13 smithi086 ceph-mon[38221]: pgmap v362: 97 pgs: 97 active+clean; 274 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 252 op/s 2024-03-23T18:02:14.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:02:13 smithi092 ceph-mon[79089]: pgmap v362: 97 pgs: 97 active+clean; 274 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 252 op/s 2024-03-23T18:02:14.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:02:13 smithi152 ceph-mon[47949]: pgmap v362: 97 pgs: 97 active+clean; 274 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 252 op/s 2024-03-23T18:02:14.926 INFO:tasks.workunit.client.0.smithi086.stdout: 1 700485 6.45 MB/sec execute 519 sec latency 14.106 ms 2024-03-23T18:02:15.926 INFO:tasks.workunit.client.0.smithi086.stdout: 1 701985 6.45 MB/sec execute 520 sec latency 206.821 ms 2024-03-23T18:02:16.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:02:15 smithi086 ceph-mon[38221]: pgmap v363: 97 pgs: 97 active+clean; 272 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 222 op/s 2024-03-23T18:02:16.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:02:15 smithi086 ceph-mon[38221]: osdmap e96: 12 total, 12 up, 12 in 2024-03-23T18:02:16.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:02:15 smithi092 ceph-mon[79089]: pgmap v363: 97 pgs: 97 active+clean; 272 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 222 op/s 2024-03-23T18:02:16.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:02:15 smithi092 ceph-mon[79089]: osdmap e96: 12 total, 12 up, 12 in 2024-03-23T18:02:16.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:02:15 smithi152 ceph-mon[47949]: pgmap v363: 97 pgs: 97 active+clean; 272 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 222 op/s 2024-03-23T18:02:16.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:02:15 smithi152 ceph-mon[47949]: osdmap e96: 12 total, 12 up, 12 in 2024-03-23T18:02:16.926 INFO:tasks.workunit.client.0.smithi086.stdout: 1 703503 6.46 MB/sec execute 521 sec latency 15.780 ms 2024-03-23T18:02:17.926 INFO:tasks.workunit.client.0.smithi086.stdout: 1 704822 6.46 MB/sec execute 522 sec latency 17.130 ms 2024-03-23T18:02:18.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:02:17 smithi086 ceph-mon[38221]: pgmap v365: 97 pgs: 97 active+clean; 272 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 222 op/s 2024-03-23T18:02:18.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:02:17 smithi092 ceph-mon[79089]: pgmap v365: 97 pgs: 97 active+clean; 272 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 222 op/s 2024-03-23T18:02:18.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:02:17 smithi152 ceph-mon[47949]: pgmap v365: 97 pgs: 97 active+clean; 272 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 222 op/s 2024-03-23T18:02:18.926 INFO:tasks.workunit.client.0.smithi086.stdout: 1 704973 6.45 MB/sec execute 523 sec latency 932.964 ms 2024-03-23T18:02:19.926 INFO:tasks.workunit.client.0.smithi086.stdout: 1 706909 6.46 MB/sec execute 524 sec latency 19.342 ms 2024-03-23T18:02:20.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:02:19 smithi086 ceph-mon[38221]: pgmap v366: 97 pgs: 97 active+clean; 272 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 222 op/s 2024-03-23T18:02:20.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:02:19 smithi092 ceph-mon[79089]: pgmap v366: 97 pgs: 97 active+clean; 272 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 222 op/s 2024-03-23T18:02:20.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:02:19 smithi152 ceph-mon[47949]: pgmap v366: 97 pgs: 97 active+clean; 272 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 222 op/s 2024-03-23T18:02:20.926 INFO:tasks.workunit.client.0.smithi086.stdout: 1 708166 6.46 MB/sec execute 525 sec latency 13.497 ms 2024-03-23T18:02:21.926 INFO:tasks.workunit.client.0.smithi086.stdout: 1 708365 6.45 MB/sec execute 526 sec latency 864.575 ms 2024-03-23T18:02:22.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:02:21 smithi086 ceph-mon[38221]: pgmap v367: 97 pgs: 97 active+clean; 278 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 211 op/s 2024-03-23T18:02:22.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:02:21 smithi092 ceph-mon[79089]: pgmap v367: 97 pgs: 97 active+clean; 278 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 211 op/s 2024-03-23T18:02:22.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:02:21 smithi152 ceph-mon[47949]: pgmap v367: 97 pgs: 97 active+clean; 278 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 211 op/s 2024-03-23T18:02:22.926 INFO:tasks.workunit.client.0.smithi086.stdout: 1 708365 6.44 MB/sec execute 527 sec latency 1864.700 ms 2024-03-23T18:02:23.926 INFO:tasks.workunit.client.0.smithi086.stdout: 1 708365 6.42 MB/sec execute 528 sec latency 2864.876 ms 2024-03-23T18:02:24.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:02:23 smithi086 ceph-mon[38221]: pgmap v368: 97 pgs: 97 active+clean; 278 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 211 op/s 2024-03-23T18:02:24.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:02:23 smithi092 ceph-mon[79089]: pgmap v368: 97 pgs: 97 active+clean; 278 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 211 op/s 2024-03-23T18:02:24.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:02:23 smithi152 ceph-mon[47949]: pgmap v368: 97 pgs: 97 active+clean; 278 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 211 op/s 2024-03-23T18:02:24.927 INFO:tasks.workunit.client.0.smithi086.stdout: 1 709062 6.41 MB/sec execute 529 sec latency 3568.264 ms 2024-03-23T18:02:25.927 INFO:tasks.workunit.client.0.smithi086.stdout: 1 710607 6.43 MB/sec execute 530 sec latency 12.381 ms 2024-03-23T18:02:26.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:02:25 smithi086 ceph-mon[38221]: pgmap v369: 97 pgs: 97 active+clean; 285 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 795 KiB/s rd, 1.8 MiB/s wr, 147 op/s 2024-03-23T18:02:26.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:02:25 smithi092 ceph-mon[79089]: pgmap v369: 97 pgs: 97 active+clean; 285 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 795 KiB/s rd, 1.8 MiB/s wr, 147 op/s 2024-03-23T18:02:26.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:02:25 smithi152 ceph-mon[47949]: pgmap v369: 97 pgs: 97 active+clean; 285 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 795 KiB/s rd, 1.8 MiB/s wr, 147 op/s 2024-03-23T18:02:26.413 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.412+0000 7f9149c00700 1 -- 172.21.15.86:0/3675826230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f914410b0d0 msgr2=0x7f914410d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:02:26.413 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.412+0000 7f9149c00700 1 --2- 172.21.15.86:0/3675826230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f914410b0d0 0x7f914410d590 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f913c00b0a0 tx=0x7f913c00b3b0 comp rx=0 tx=0).stop 2024-03-23T18:02:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.412+0000 7f9149c00700 1 -- 172.21.15.86:0/3675826230 shutdown_connections 2024-03-23T18:02:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.412+0000 7f9149c00700 1 --2- 172.21.15.86:0/3675826230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f914410b0d0 0x7f914410d590 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.412+0000 7f9149c00700 1 --2- 172.21.15.86:0/3675826230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9144101cb0 0x7f9144102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.412+0000 7f9149c00700 1 --2- 172.21.15.86:0/3675826230 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f91441025d0 0x7f914410aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.412+0000 7f9149c00700 1 -- 172.21.15.86:0/3675826230 >> 172.21.15.86:0/3675826230 conn(0x7f91440fb4f0 msgr2=0x7f91440fd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:02:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.412+0000 7f9149c00700 1 -- 172.21.15.86:0/3675826230 shutdown_connections 2024-03-23T18:02:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.412+0000 7f9149c00700 1 -- 172.21.15.86:0/3675826230 wait complete. 2024-03-23T18:02:26.417 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.413+0000 7f9149c00700 1 Processor -- start 2024-03-23T18:02:26.417 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.414+0000 7f9149c00700 1 -- start start 2024-03-23T18:02:26.417 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.414+0000 7f9149c00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9144101cb0 0x7f91441992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:02:26.418 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.414+0000 7f9149c00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91441025d0 0x7f91441997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:02:26.418 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.414+0000 7f9149c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f914410b0d0 0x7f914419db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:02:26.418 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.414+0000 7f9149c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f914419e1d0 con 0x7f914410b0d0 2024-03-23T18:02:26.418 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.414+0000 7f9149c00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f914419e340 con 0x7f91441025d0 2024-03-23T18:02:26.418 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.414+0000 7f9149c00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f914419e500 con 0x7f9144101cb0 2024-03-23T18:02:26.418 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.414+0000 7f9143400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91441025d0 0x7f91441997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:02:26.418 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.414+0000 7f9143400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91441025d0 0x7f91441997f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:51214/0 (socket says 172.21.15.86:51214) 2024-03-23T18:02:26.418 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.414+0000 7f9143400700 1 -- 172.21.15.86:0/1256897166 learned_addr learned my addr 172.21.15.86:0/1256897166 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:02:26.418 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.415+0000 7f9143e00700 1 --2- 172.21.15.86:0/1256897166 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9144101cb0 0x7f91441992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:02:26.418 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.415+0000 7f9149200700 1 --2- 172.21.15.86:0/1256897166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f914410b0d0 0x7f914419db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:02:26.418 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.415+0000 7f9143400700 1 -- 172.21.15.86:0/1256897166 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9144101cb0 msgr2=0x7f91441992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:02:26.418 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.415+0000 7f9143400700 1 --2- 172.21.15.86:0/1256897166 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9144101cb0 0x7f91441992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:26.419 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.415+0000 7f9143400700 1 -- 172.21.15.86:0/1256897166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f914410b0d0 msgr2=0x7f914419db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:02:26.419 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.415+0000 7f9143400700 1 --2- 172.21.15.86:0/1256897166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f914410b0d0 0x7f914419db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:26.419 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.415+0000 7f9143400700 1 -- 172.21.15.86:0/1256897166 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f913c009d00 con 0x7f91441025d0 2024-03-23T18:02:26.419 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.415+0000 7f9149200700 1 --2- 172.21.15.86:0/1256897166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f914410b0d0 0x7f914419db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T18:02:26.419 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.415+0000 7f9143400700 1 --2- 172.21.15.86:0/1256897166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91441025d0 0x7f91441997f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f913000e9f0 tx=0x7f913000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:02:26.419 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.415+0000 7f9140c00700 1 -- 172.21.15.86:0/1256897166 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f913000c9c0 con 0x7f91441025d0 2024-03-23T18:02:26.419 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.415+0000 7f9140c00700 1 -- 172.21.15.86:0/1256897166 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9130010430 con 0x7f91441025d0 2024-03-23T18:02:26.419 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.415+0000 7f9149c00700 1 -- 172.21.15.86:0/1256897166 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f914419e7e0 con 0x7f91441025d0 2024-03-23T18:02:26.419 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.415+0000 7f9140c00700 1 -- 172.21.15.86:0/1256897166 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9130018680 con 0x7f91441025d0 2024-03-23T18:02:26.419 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.416+0000 7f9149c00700 1 -- 172.21.15.86:0/1256897166 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91441a6020 con 0x7f91441025d0 2024-03-23T18:02:26.420 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.419+0000 7f9140c00700 1 -- 172.21.15.86:0/1256897166 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f9130010860 con 0x7f91441025d0 2024-03-23T18:02:26.422 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.421+0000 7f9140c00700 1 --2- 172.21.15.86:0/1256897166 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f91280a1290 0x7f91280a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:02:26.422 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.421+0000 7f9140c00700 1 -- 172.21.15.86:0/1256897166 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f9130014070 con 0x7f91441025d0 2024-03-23T18:02:26.422 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.421+0000 7f9143e00700 1 --2- 172.21.15.86:0/1256897166 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f91280a1290 0x7f91280a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:02:26.422 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.421+0000 7f915edf7b80 1 Processor -- start 2024-03-23T18:02:26.424 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.422+0000 7f915edf7b80 1 -- start start 2024-03-23T18:02:26.424 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.422+0000 7f915edf7b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9e58012d0 0x55d9e58016b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:02:26.424 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.422+0000 7f915edf7b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d9e5801bf0 0x55d9e580c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:02:26.425 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.422+0000 7f9143e00700 1 --2- 172.21.15.86:0/1256897166 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f91280a1290 0x7f91280a3750 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9134009c60 tx=0x7f9134009260 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:02:26.425 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.422+0000 7f915edf7b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d9e580cab0 0x55d9e580eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:02:26.425 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.422+0000 7f915edf7b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9e575a930 con 0x55d9e58012d0 2024-03-23T18:02:26.425 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.422+0000 7f915edf7b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9e57582f0 con 0x55d9e5801bf0 2024-03-23T18:02:26.425 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.422+0000 7f915edf7b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9e5753be0 con 0x55d9e580cab0 2024-03-23T18:02:26.425 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.422+0000 7f9143400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d9e5801bf0 0x55d9e580c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:02:26.425 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.422+0000 7f9143400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d9e5801bf0 0x55d9e580c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:51216/0 (socket says 172.21.15.86:51216) 2024-03-23T18:02:26.425 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.422+0000 7f9143400700 1 -- 172.21.15.86:0/2252519931 learned_addr learned my addr 172.21.15.86:0/2252519931 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:02:26.425 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.422+0000 7f9143e00700 1 --2- 172.21.15.86:0/2252519931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9e58012d0 0x55d9e58016b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:02:26.425 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.423+0000 7f9149200700 1 --2- 172.21.15.86:0/2252519931 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d9e580cab0 0x55d9e580eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:02:26.425 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.423+0000 7f9143400700 1 -- 172.21.15.86:0/2252519931 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d9e580cab0 msgr2=0x55d9e580eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:02:26.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.423+0000 7f9143400700 1 --2- 172.21.15.86:0/2252519931 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d9e580cab0 0x55d9e580eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:26.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.423+0000 7f9143400700 1 -- 172.21.15.86:0/2252519931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9e58012d0 msgr2=0x55d9e58016b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:02:26.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.423+0000 7f9143400700 1 --2- 172.21.15.86:0/2252519931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9e58012d0 0x55d9e58016b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:26.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.423+0000 7f9143400700 1 -- 172.21.15.86:0/2252519931 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d9e55d5350 con 0x55d9e5801bf0 2024-03-23T18:02:26.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.423+0000 7f9143400700 1 --2- 172.21.15.86:0/2252519931 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d9e5801bf0 0x55d9e580c570 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f913000f010 tx=0x7f9130099dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56180549ada7c44d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:02:26.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.423+0000 7f9139600700 1 -- 172.21.15.86:0/2252519931 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f913009dfa0 con 0x55d9e5801bf0 2024-03-23T18:02:26.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.423+0000 7f9139600700 1 -- 172.21.15.86:0/2252519931 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f91300a25f0 con 0x55d9e5801bf0 2024-03-23T18:02:26.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.423+0000 7f9139600700 1 -- 172.21.15.86:0/2252519931 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9130099930 con 0x55d9e5801bf0 2024-03-23T18:02:26.430 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.424+0000 7f915edf7b80 1 -- 172.21.15.86:0/2252519931 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d9e5801bf0 msgr2=0x55d9e580c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:02:26.430 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.424+0000 7f915edf7b80 1 --2- 172.21.15.86:0/2252519931 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d9e5801bf0 0x55d9e580c570 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f913000f010 tx=0x7f9130099dc0 comp rx=0 tx=0).stop 2024-03-23T18:02:26.430 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.424+0000 7f915edf7b80 1 -- 172.21.15.86:0/2252519931 shutdown_connections 2024-03-23T18:02:26.430 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.424+0000 7f915edf7b80 1 --2- 172.21.15.86:0/2252519931 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d9e580cab0 0x55d9e580eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:26.430 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.424+0000 7f915edf7b80 1 --2- 172.21.15.86:0/2252519931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9e58012d0 0x55d9e58016b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:26.430 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.424+0000 7f915edf7b80 1 --2- 172.21.15.86:0/2252519931 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d9e5801bf0 0x55d9e580c570 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:26.430 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.424+0000 7f915edf7b80 1 -- 172.21.15.86:0/2252519931 >> 172.21.15.86:0/2252519931 conn(0x55d9e5761640 msgr2=0x55d9e57fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:02:26.430 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.425+0000 7f915edf7b80 1 -- 172.21.15.86:0/2252519931 shutdown_connections 2024-03-23T18:02:26.430 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.425+0000 7f915edf7b80 1 -- 172.21.15.86:0/2252519931 wait complete. 2024-03-23T18:02:26.430 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.425+0000 7f915edf7b80 1 -- start start 2024-03-23T18:02:26.430 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.426+0000 7f915edf7b80 1 Processor -- start 2024-03-23T18:02:26.431 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.426+0000 7f915edf7b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d9e58012d0 0x55d9e5809910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:02:26.431 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.426+0000 7f915edf7b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d9e5801bf0 0x55d9e5809e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:02:26.431 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.426+0000 7f915edf7b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9e580cab0 0x55d9e580a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:02:26.431 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.426+0000 7f915edf7b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9e56efd20 con 0x55d9e580cab0 2024-03-23T18:02:26.431 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.426+0000 7f915edf7b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9e56ef9c0 con 0x55d9e58012d0 2024-03-23T18:02:26.431 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.426+0000 7f915edf7b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9e55d7640 con 0x55d9e5801bf0 2024-03-23T18:02:26.431 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.426+0000 7f915edf7b80 10 client.0 _ll_register_callbacks cb 0x55d9e56f9410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T18:02:26.431 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.426+0000 7f9143e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d9e58012d0 0x55d9e5809910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:02:26.431 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.426+0000 7f9143e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d9e58012d0 0x55d9e5809910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:51232/0 (socket says 172.21.15.86:51232) 2024-03-23T18:02:26.431 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.427+0000 7f9143e00700 1 -- 172.21.15.86:0/1412906360 learned_addr learned my addr 172.21.15.86:0/1412906360 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:02:26.432 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.427+0000 7f9149200700 1 --2- 172.21.15.86:0/1412906360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9e580cab0 0x55d9e580a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:02:26.432 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.427+0000 7f9143400700 1 --2- 172.21.15.86:0/1412906360 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d9e5801bf0 0x55d9e5809e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:02:26.432 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.427+0000 7f9143e00700 1 -- 172.21.15.86:0/1412906360 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d9e5801bf0 msgr2=0x55d9e5809e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:02:26.432 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.427+0000 7f9143e00700 1 --2- 172.21.15.86:0/1412906360 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d9e5801bf0 0x55d9e5809e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:26.432 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.427+0000 7f9143e00700 1 -- 172.21.15.86:0/1412906360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9e580cab0 msgr2=0x55d9e580a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:02:26.432 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.427+0000 7f9143e00700 1 --2- 172.21.15.86:0/1412906360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9e580cab0 0x55d9e580a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:26.432 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.427+0000 7f9143e00700 1 -- 172.21.15.86:0/1412906360 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91441a6020 con 0x55d9e58012d0 2024-03-23T18:02:26.432 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.427+0000 7f9143e00700 1 --2- 172.21.15.86:0/1412906360 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d9e58012d0 0x55d9e5809910 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f9134003900 tx=0x7f91340047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90898995292b63df server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:02:26.432 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.427+0000 7f912e000700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T18:02:26.432 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.427+0000 7f912e000700 1 -- 172.21.15.86:0/1412906360 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9134022070 con 0x55d9e58012d0 2024-03-23T18:02:26.432 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.427+0000 7f912e000700 1 -- 172.21.15.86:0/1412906360 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f913401d620 con 0x55d9e58012d0 2024-03-23T18:02:26.432 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.427+0000 7f915edf7b80 1 -- 172.21.15.86:0/1412906360 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d9e5714320 con 0x55d9e58012d0 2024-03-23T18:02:26.432 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.428+0000 7f912e000700 1 -- 172.21.15.86:0/1412906360 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f913401d7e0 con 0x55d9e58012d0 2024-03-23T18:02:26.432 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.428+0000 7f915edf7b80 10 client.26984 fetch_fsmap learned FSMap version 52 2024-03-23T18:02:26.433 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.428+0000 7f915edf7b80 1 -- 172.21.15.86:0/1412906360 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x55d9e5581b00 con 0x55d9e58012d0 2024-03-23T18:02:26.433 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.428+0000 7f912e000700 1 -- 172.21.15.86:0/1412906360 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d9e5714320 con 0x55d9e58012d0 2024-03-23T18:02:26.433 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.428+0000 7f912e000700 1 -- 172.21.15.86:0/1412906360 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 52) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f91340279c0 con 0x55d9e58012d0 2024-03-23T18:02:26.433 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.428+0000 7f915edf7b80 10 client.26984 fetch_fsmap finished waiting for FSMap version 52 2024-03-23T18:02:26.433 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.428+0000 7f915edf7b80 10 client.26984 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T18:02:26.433 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.428+0000 7f915edf7b80 20 client.26984 populate_metadata read hostname 'smithi086' 2024-03-23T18:02:26.433 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.428+0000 7f915edf7b80 1 --2- 172.21.15.86:0/1412906360 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55d9e58250a0 0x55d9e58274c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:02:26.433 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.428+0000 7f915edf7b80 4 client.26984 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T18:02:26.433 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.428+0000 7f915edf7b80 1 -- 172.21.15.86:0/1412906360 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d9e574c6a0 con 0x55d9e58250a0 2024-03-23T18:02:26.433 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.428+0000 7f9143400700 1 --2- 172.21.15.86:0/1412906360 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55d9e58250a0 0x55d9e58274c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:02:26.433 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.429+0000 7f912e000700 10 client.26984 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T18:02:26.433 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.429+0000 7f9143400700 1 --2- 172.21.15.86:0/1412906360 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55d9e58250a0 0x55d9e58274c0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83731ea567f79d12 server_cookie=7cdfa13308237a13 in_seq=0 out_seq=0 2024-03-23T18:02:26.435 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.429+0000 7f912e000700 1 -- 172.21.15.86:0/1412906360 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55d9e574c6a0 con 0x55d9e58250a0 2024-03-23T18:02:26.435 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.429+0000 7f912e000700 10 client.26984 handle_command_reply: tid=0 2024-03-23T18:02:26.435 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.429+0000 7f915edf7b80 1 client.26984 shutdown 2024-03-23T18:02:26.435 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.430+0000 7f915edf7b80 1 -- 172.21.15.86:0/1412906360 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d9e58012d0 msgr2=0x55d9e5809910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:02:26.435 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.430+0000 7f915edf7b80 1 --2- 172.21.15.86:0/1412906360 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d9e58012d0 0x55d9e5809910 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f9134003900 tx=0x7f91340047c0 comp rx=0 tx=0).stop 2024-03-23T18:02:26.435 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.430+0000 7f915edf7b80 1 -- 172.21.15.86:0/1412906360 shutdown_connections 2024-03-23T18:02:26.436 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.430+0000 7f915edf7b80 1 --2- 172.21.15.86:0/1412906360 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55d9e58250a0 0x55d9e58274c0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:26.436 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.430+0000 7f915edf7b80 1 --2- 172.21.15.86:0/1412906360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9e580cab0 0x55d9e580a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:26.436 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.430+0000 7f915edf7b80 1 --2- 172.21.15.86:0/1412906360 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d9e58012d0 0x55d9e5809910 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:26.436 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.430+0000 7f912e000700 0 client.26984 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T18:02:26.436 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.430+0000 7f915edf7b80 1 --2- 172.21.15.86:0/1412906360 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d9e5801bf0 0x55d9e5809e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:26.436 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.430+0000 7f915edf7b80 1 -- 172.21.15.86:0/1412906360 >> 172.21.15.86:0/1412906360 conn(0x55d9e5761640 msgr2=0x55d9e5803390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:02:26.436 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.430+0000 7f915edf7b80 1 -- 172.21.15.86:0/1412906360 shutdown_connections 2024-03-23T18:02:26.437 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.430+0000 7f915edf7b80 1 -- 172.21.15.86:0/1412906360 wait complete. 2024-03-23T18:02:26.437 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.430+0000 7f915edf7b80 20 client.26984 trim_cache size 0 max 16384 2024-03-23T18:02:26.448 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.446+0000 7f915edf7b80 1 Processor -- start 2024-03-23T18:02:26.448 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.446+0000 7f915edf7b80 1 -- start start 2024-03-23T18:02:26.448 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.446+0000 7f915edf7b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9e58012d0 0x55d9e5813730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:02:26.448 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.446+0000 7f915edf7b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d9e5801bf0 0x55d9e57fd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:02:26.448 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.446+0000 7f915edf7b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d9e580cab0 0x55d9e57fd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:02:26.448 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.446+0000 7f915edf7b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9e57fdd00 con 0x55d9e58012d0 2024-03-23T18:02:26.448 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.446+0000 7f915edf7b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9e5805750 con 0x55d9e5801bf0 2024-03-23T18:02:26.448 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.446+0000 7f915edf7b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9e58022c0 con 0x55d9e580cab0 2024-03-23T18:02:26.448 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.446+0000 7f9143400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d9e5801bf0 0x55d9e57fd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:02:26.449 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.446+0000 7f9143400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d9e5801bf0 0x55d9e57fd280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:51240/0 (socket says 172.21.15.86:51240) 2024-03-23T18:02:26.449 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.446+0000 7f9143400700 1 -- 172.21.15.86:0/2340702099 learned_addr learned my addr 172.21.15.86:0/2340702099 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:02:26.449 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.446+0000 7f9143e00700 1 --2- 172.21.15.86:0/2340702099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9e58012d0 0x55d9e5813730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:02:26.449 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.446+0000 7f9149200700 1 --2- 172.21.15.86:0/2340702099 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d9e580cab0 0x55d9e57fd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:02:26.449 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.446+0000 7f9143400700 1 -- 172.21.15.86:0/2340702099 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d9e580cab0 msgr2=0x55d9e57fd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:02:26.449 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.446+0000 7f9143400700 1 --2- 172.21.15.86:0/2340702099 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d9e580cab0 0x55d9e57fd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:26.449 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.446+0000 7f9143400700 1 -- 172.21.15.86:0/2340702099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9e58012d0 msgr2=0x55d9e5813730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:02:26.449 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.446+0000 7f9143400700 1 --2- 172.21.15.86:0/2340702099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9e58012d0 0x55d9e5813730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:26.449 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.446+0000 7f9143400700 1 -- 172.21.15.86:0/2340702099 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d9e5719e40 con 0x55d9e5801bf0 2024-03-23T18:02:26.449 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.447+0000 7f9143400700 1 --2- 172.21.15.86:0/2340702099 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d9e5801bf0 0x55d9e57fd280 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f91300ca860 tx=0x7f913000efa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fcc6a9643a1f39f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:02:26.449 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.447+0000 7f913a000700 1 -- 172.21.15.86:0/2340702099 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9130098610 con 0x55d9e5801bf0 2024-03-23T18:02:26.449 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.447+0000 7f913a000700 1 -- 172.21.15.86:0/2340702099 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f913009a3d0 con 0x55d9e5801bf0 2024-03-23T18:02:26.449 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.447+0000 7f913a000700 1 -- 172.21.15.86:0/2340702099 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91300a23a0 con 0x55d9e5801bf0 2024-03-23T18:02:26.453 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.448+0000 7f915edf7b80 1 -- 172.21.15.86:0/2340702099 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d9e5801bf0 msgr2=0x55d9e57fd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:02:26.453 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.448+0000 7f915edf7b80 1 --2- 172.21.15.86:0/2340702099 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d9e5801bf0 0x55d9e57fd280 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f91300ca860 tx=0x7f913000efa0 comp rx=0 tx=0).stop 2024-03-23T18:02:26.453 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.448+0000 7f915edf7b80 1 -- 172.21.15.86:0/2340702099 shutdown_connections 2024-03-23T18:02:26.453 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.448+0000 7f915edf7b80 1 --2- 172.21.15.86:0/2340702099 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d9e580cab0 0x55d9e57fd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:26.453 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.448+0000 7f915edf7b80 1 --2- 172.21.15.86:0/2340702099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9e58012d0 0x55d9e5813730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:26.453 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.448+0000 7f915edf7b80 1 --2- 172.21.15.86:0/2340702099 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d9e5801bf0 0x55d9e57fd280 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:26.454 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.448+0000 7f915edf7b80 1 -- 172.21.15.86:0/2340702099 >> 172.21.15.86:0/2340702099 conn(0x55d9e5761640 msgr2=0x55d9e5804c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:02:26.454 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.448+0000 7f915edf7b80 1 -- 172.21.15.86:0/2340702099 shutdown_connections 2024-03-23T18:02:26.454 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.448+0000 7f915edf7b80 1 -- 172.21.15.86:0/2340702099 wait complete. 2024-03-23T18:02:26.454 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.449+0000 7f915edf7b80 1 -- start start 2024-03-23T18:02:26.454 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.449+0000 7f915edf7b80 1 Processor -- start 2024-03-23T18:02:26.454 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.449+0000 7f915edf7b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9e58012d0 0x55d9e5812f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:02:26.454 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.449+0000 7f915edf7b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d9e5801bf0 0x55d9e5813480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:02:26.454 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.449+0000 7f915edf7b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d9e580cab0 0x55d9e5811710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:02:26.454 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.449+0000 7f915edf7b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9e5811ce0 con 0x55d9e58012d0 2024-03-23T18:02:26.454 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.449+0000 7f915edf7b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9e5811e20 con 0x55d9e5801bf0 2024-03-23T18:02:26.454 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.449+0000 7f915edf7b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9e5811fe0 con 0x55d9e580cab0 2024-03-23T18:02:26.454 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.450+0000 7f915edf7b80 10 client.0 _ll_register_callbacks cb 0x55d9e56f9410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T18:02:26.455 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.450+0000 7f9143400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d9e5801bf0 0x55d9e5813480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:02:26.455 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.450+0000 7f9143400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d9e5801bf0 0x55d9e5813480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:51248/0 (socket says 172.21.15.86:51248) 2024-03-23T18:02:26.455 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.450+0000 7f9143400700 1 -- 172.21.15.86:0/3029389899 learned_addr learned my addr 172.21.15.86:0/3029389899 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:02:26.455 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.450+0000 7f9143e00700 1 --2- 172.21.15.86:0/3029389899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9e58012d0 0x55d9e5812f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:02:26.455 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.450+0000 7f9149200700 1 --2- 172.21.15.86:0/3029389899 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d9e580cab0 0x55d9e5811710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:02:26.455 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.450+0000 7f9143400700 1 -- 172.21.15.86:0/3029389899 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d9e580cab0 msgr2=0x55d9e5811710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:02:26.455 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.450+0000 7f9143400700 1 --2- 172.21.15.86:0/3029389899 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d9e580cab0 0x55d9e5811710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:26.455 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.450+0000 7f9143400700 1 -- 172.21.15.86:0/3029389899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9e58012d0 msgr2=0x55d9e5812f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:02:26.455 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.450+0000 7f9143400700 1 --2- 172.21.15.86:0/3029389899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9e58012d0 0x55d9e5812f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:26.455 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.450+0000 7f9143400700 1 -- 172.21.15.86:0/3029389899 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d9e55d5350 con 0x55d9e5801bf0 2024-03-23T18:02:26.455 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.450+0000 7f9143400700 1 --2- 172.21.15.86:0/3029389899 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d9e5801bf0 0x55d9e5813480 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f9130096670 tx=0x7f91300986e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c7c9a247ee6a184 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:02:26.455 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.450+0000 7f912e000700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T18:02:26.455 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.451+0000 7f912e000700 1 -- 172.21.15.86:0/3029389899 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f913009a540 con 0x55d9e5801bf0 2024-03-23T18:02:26.455 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.451+0000 7f912e000700 1 -- 172.21.15.86:0/3029389899 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f9130099660 con 0x55d9e5801bf0 2024-03-23T18:02:26.456 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.451+0000 7f915edf7b80 1 -- 172.21.15.86:0/3029389899 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f91300a25f0 con 0x55d9e5801bf0 2024-03-23T18:02:26.456 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.451+0000 7f912e000700 1 -- 172.21.15.86:0/3029389899 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f913009a6a0 con 0x55d9e5801bf0 2024-03-23T18:02:26.456 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.451+0000 7f915edf7b80 10 client.26990 fetch_fsmap learned FSMap version 52 2024-03-23T18:02:26.456 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.451+0000 7f915edf7b80 1 -- 172.21.15.86:0/3029389899 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x55d9e55d9c70 con 0x55d9e5801bf0 2024-03-23T18:02:26.456 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.451+0000 7f912e000700 1 -- 172.21.15.86:0/3029389899 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f91300a25f0 con 0x55d9e5801bf0 2024-03-23T18:02:26.456 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.451+0000 7f912e000700 1 -- 172.21.15.86:0/3029389899 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 52) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f91300b2d20 con 0x55d9e5801bf0 2024-03-23T18:02:26.456 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.451+0000 7f915edf7b80 10 client.26990 fetch_fsmap finished waiting for FSMap version 52 2024-03-23T18:02:26.456 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.451+0000 7f915edf7b80 10 client.26990 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T18:02:26.456 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.451+0000 7f915edf7b80 20 client.26990 populate_metadata read hostname 'smithi086' 2024-03-23T18:02:26.456 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.451+0000 7f915edf7b80 1 --2- 172.21.15.86:0/3029389899 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55d9e58779f0 0x55d9e5879de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:02:26.456 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.451+0000 7f915edf7b80 4 client.26990 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T18:02:26.456 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.451+0000 7f915edf7b80 1 -- 172.21.15.86:0/3029389899 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d9e55a2c90 con 0x55d9e58779f0 2024-03-23T18:02:26.456 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.451+0000 7f9143e00700 1 --2- 172.21.15.86:0/3029389899 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55d9e58779f0 0x55d9e5879de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:02:26.457 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.452+0000 7f912e000700 10 client.26990 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T18:02:26.457 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.452+0000 7f9143e00700 1 --2- 172.21.15.86:0/3029389899 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55d9e58779f0 0x55d9e5879de0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd32854faad59a73 server_cookie=8579019680cccd75 in_seq=0 out_seq=0 2024-03-23T18:02:26.491 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.490+0000 7f912e000700 1 -- 172.21.15.86:0/3029389899 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d9e55a2c90 con 0x55d9e58779f0 2024-03-23T18:02:26.491 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.490+0000 7f912e000700 10 client.26990 handle_command_reply: tid=0 2024-03-23T18:02:26.491 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.490+0000 7f915edf7b80 1 client.26990 shutdown 2024-03-23T18:02:26.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.490+0000 7f915edf7b80 1 -- 172.21.15.86:0/3029389899 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d9e5801bf0 msgr2=0x55d9e5813480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:02:26.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.490+0000 7f915edf7b80 1 --2- 172.21.15.86:0/3029389899 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d9e5801bf0 0x55d9e5813480 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f9130096670 tx=0x7f91300986e0 comp rx=0 tx=0).stop 2024-03-23T18:02:26.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.490+0000 7f915edf7b80 1 -- 172.21.15.86:0/3029389899 shutdown_connections 2024-03-23T18:02:26.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.490+0000 7f915edf7b80 1 --2- 172.21.15.86:0/3029389899 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55d9e58779f0 0x55d9e5879de0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:26.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.490+0000 7f915edf7b80 1 --2- 172.21.15.86:0/3029389899 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d9e580cab0 0x55d9e5811710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:26.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.491+0000 7f912e000700 0 client.26990 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T18:02:26.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.490+0000 7f915edf7b80 1 --2- 172.21.15.86:0/3029389899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9e58012d0 0x55d9e5812f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:26.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.491+0000 7f915edf7b80 1 --2- 172.21.15.86:0/3029389899 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d9e5801bf0 0x55d9e5813480 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:26.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.491+0000 7f915edf7b80 1 -- 172.21.15.86:0/3029389899 >> 172.21.15.86:0/3029389899 conn(0x55d9e5761640 msgr2=0x55d9e57fd280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:02:26.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.491+0000 7f915edf7b80 1 -- 172.21.15.86:0/3029389899 shutdown_connections 2024-03-23T18:02:26.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.491+0000 7f915edf7b80 1 -- 172.21.15.86:0/3029389899 wait complete. 2024-03-23T18:02:26.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.491+0000 7f915edf7b80 20 client.26990 trim_cache size 0 max 16384 2024-03-23T18:02:26.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-03-23T18:02:26.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-03-23T18:02:26.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-03-23T18:02:26.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-03-23T18:02:26.494 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.492+0000 7f9149c00700 1 -- 172.21.15.86:0/1256897166 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f91280a1290 msgr2=0x7f91280a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:02:26.494 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.492+0000 7f9149c00700 1 --2- 172.21.15.86:0/1256897166 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f91280a1290 0x7f91280a3750 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9134009c60 tx=0x7f9134009260 comp rx=0 tx=0).stop 2024-03-23T18:02:26.494 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.492+0000 7f9149c00700 1 -- 172.21.15.86:0/1256897166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91441025d0 msgr2=0x7f91441997f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:02:26.494 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.492+0000 7f9149c00700 1 --2- 172.21.15.86:0/1256897166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91441025d0 0x7f91441997f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f913000e9f0 tx=0x7f913000ed00 comp rx=0 tx=0).stop 2024-03-23T18:02:26.494 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.492+0000 7f9149c00700 1 -- 172.21.15.86:0/1256897166 shutdown_connections 2024-03-23T18:02:26.494 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.492+0000 7f9149c00700 1 --2- 172.21.15.86:0/1256897166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f914410b0d0 0x7f914419db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:26.494 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.492+0000 7f9149c00700 1 --2- 172.21.15.86:0/1256897166 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f91280a1290 0x7f91280a3750 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:26.494 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.492+0000 7f9149c00700 1 --2- 172.21.15.86:0/1256897166 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9144101cb0 0x7f91441992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:26.494 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.492+0000 7f9149c00700 1 --2- 172.21.15.86:0/1256897166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91441025d0 0x7f91441997f0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:26.495 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.492+0000 7f9149c00700 1 -- 172.21.15.86:0/1256897166 >> 172.21.15.86:0/1256897166 conn(0x7f91440fb4f0 msgr2=0x7f914410c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:02:26.495 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.493+0000 7f9149c00700 1 -- 172.21.15.86:0/1256897166 shutdown_connections 2024-03-23T18:02:26.495 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:26.493+0000 7f9149c00700 1 -- 172.21.15.86:0/1256897166 wait complete. 2024-03-23T18:02:26.927 INFO:tasks.workunit.client.0.smithi086.stdout: 1 711916 6.42 MB/sec execute 531 sec latency 24.077 ms 2024-03-23T18:02:27.927 INFO:tasks.workunit.client.0.smithi086.stdout: 1 712365 6.41 MB/sec execute 532 sec latency 854.847 ms 2024-03-23T18:02:28.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:02:27 smithi086 ceph-mon[38221]: pgmap v370: 97 pgs: 97 active+clean; 285 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 1.7 MiB/s wr, 137 op/s 2024-03-23T18:02:28.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:02:27 smithi092 ceph-mon[79089]: pgmap v370: 97 pgs: 97 active+clean; 285 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 1.7 MiB/s wr, 137 op/s 2024-03-23T18:02:28.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:02:27 smithi152 ceph-mon[47949]: pgmap v370: 97 pgs: 97 active+clean; 285 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 1.7 MiB/s wr, 137 op/s 2024-03-23T18:02:28.927 INFO:tasks.workunit.client.0.smithi086.stdout: 1 714060 6.42 MB/sec execute 533 sec latency 22.532 ms 2024-03-23T18:02:29.927 INFO:tasks.workunit.client.0.smithi086.stdout: 1 715414 6.42 MB/sec execute 534 sec latency 14.597 ms 2024-03-23T18:02:30.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:02:29 smithi086 ceph-mon[38221]: pgmap v371: 97 pgs: 97 active+clean; 285 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 1.5 MiB/s wr, 122 op/s 2024-03-23T18:02:30.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:02:29 smithi092 ceph-mon[79089]: pgmap v371: 97 pgs: 97 active+clean; 285 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 1.5 MiB/s wr, 122 op/s 2024-03-23T18:02:30.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:02:29 smithi152 ceph-mon[47949]: pgmap v371: 97 pgs: 97 active+clean; 285 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 1.5 MiB/s wr, 122 op/s 2024-03-23T18:02:30.927 INFO:tasks.workunit.client.0.smithi086.stdout: 1 716167 6.41 MB/sec execute 535 sec latency 656.936 ms 2024-03-23T18:02:31.927 INFO:tasks.workunit.client.0.smithi086.stdout: 1 717680 6.42 MB/sec execute 536 sec latency 13.604 ms 2024-03-23T18:02:32.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:02:31 smithi086 ceph-mon[38221]: pgmap v372: 97 pgs: 97 active+clean; 278 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 229 op/s 2024-03-23T18:02:32.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:02:31 smithi092 ceph-mon[79089]: pgmap v372: 97 pgs: 97 active+clean; 278 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 229 op/s 2024-03-23T18:02:32.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:02:31 smithi152 ceph-mon[47949]: pgmap v372: 97 pgs: 97 active+clean; 278 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 229 op/s 2024-03-23T18:02:32.927 INFO:tasks.workunit.client.0.smithi086.stdout: 1 718872 6.42 MB/sec execute 537 sec latency 21.321 ms 2024-03-23T18:02:33.930 INFO:tasks.workunit.client.0.smithi086.stdout: 1 719005 6.41 MB/sec execute 538 sec latency 908.434 ms 2024-03-23T18:02:34.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:02:33 smithi086 ceph-mon[38221]: pgmap v373: 97 pgs: 97 active+clean; 278 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 1.7 MiB/s wr, 152 op/s 2024-03-23T18:02:34.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:02:33 smithi092 ceph-mon[79089]: pgmap v373: 97 pgs: 97 active+clean; 278 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 1.7 MiB/s wr, 152 op/s 2024-03-23T18:02:34.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:02:33 smithi152 ceph-mon[47949]: pgmap v373: 97 pgs: 97 active+clean; 278 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 1.7 MiB/s wr, 152 op/s 2024-03-23T18:02:34.927 INFO:tasks.workunit.client.0.smithi086.stdout: 1 719682 6.40 MB/sec execute 539 sec latency 1628.226 ms 2024-03-23T18:02:35.927 INFO:tasks.workunit.client.0.smithi086.stdout: 1 721159 6.41 MB/sec execute 540 sec latency 13.466 ms 2024-03-23T18:02:36.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:02:35 smithi086 ceph-mon[38221]: pgmap v374: 97 pgs: 97 active+clean; 282 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.6 MiB/s wr, 218 op/s 2024-03-23T18:02:36.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:02:35 smithi092 ceph-mon[79089]: pgmap v374: 97 pgs: 97 active+clean; 282 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.6 MiB/s wr, 218 op/s 2024-03-23T18:02:36.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:02:35 smithi152 ceph-mon[47949]: pgmap v374: 97 pgs: 97 active+clean; 282 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.6 MiB/s wr, 218 op/s 2024-03-23T18:02:36.927 INFO:tasks.workunit.client.0.smithi086.stdout: 1 722386 6.41 MB/sec execute 541 sec latency 17.306 ms 2024-03-23T18:02:37.927 INFO:tasks.workunit.client.0.smithi086.stdout: 1 722998 6.40 MB/sec execute 542 sec latency 736.598 ms 2024-03-23T18:02:38.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:02:37 smithi086 ceph-mon[38221]: pgmap v375: 97 pgs: 97 active+clean; 282 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 838 KiB/s rd, 1.8 MiB/s wr, 172 op/s 2024-03-23T18:02:38.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:02:37 smithi092 ceph-mon[79089]: pgmap v375: 97 pgs: 97 active+clean; 282 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 838 KiB/s rd, 1.8 MiB/s wr, 172 op/s 2024-03-23T18:02:38.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:02:37 smithi152 ceph-mon[47949]: pgmap v375: 97 pgs: 97 active+clean; 282 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 838 KiB/s rd, 1.8 MiB/s wr, 172 op/s 2024-03-23T18:02:38.927 INFO:tasks.workunit.client.0.smithi086.stdout: 1 724656 6.41 MB/sec execute 543 sec latency 18.449 ms 2024-03-23T18:02:39.652 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.652+0000 7f58be800700 1 -- 172.21.15.86:0/3807263903 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58b8105c40 msgr2=0x7f58b81036c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:02:39.653 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.652+0000 7f58be800700 1 --2- 172.21.15.86:0/3807263903 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58b8105c40 0x7f58b81036c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f58a8009990 tx=0x7f58a8009ca0 comp rx=0 tx=0).stop 2024-03-23T18:02:39.653 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.652+0000 7f58be800700 1 -- 172.21.15.86:0/3807263903 shutdown_connections 2024-03-23T18:02:39.653 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.652+0000 7f58be800700 1 --2- 172.21.15.86:0/3807263903 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58b80680e0 0x7f58b8068560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:39.653 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.652+0000 7f58be800700 1 --2- 172.21.15.86:0/3807263903 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58b8105c40 0x7f58b81036c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:39.653 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.652+0000 7f58be800700 1 --2- 172.21.15.86:0/3807263903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58b8103c00 0x7f58b8104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:39.653 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.652+0000 7f58be800700 1 -- 172.21.15.86:0/3807263903 >> 172.21.15.86:0/3807263903 conn(0x7f58b8076d50 msgr2=0x7f58b8077160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:02:39.653 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.652+0000 7f58be800700 1 -- 172.21.15.86:0/3807263903 shutdown_connections 2024-03-23T18:02:39.653 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.652+0000 7f58be800700 1 -- 172.21.15.86:0/3807263903 wait complete. 2024-03-23T18:02:39.653 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.653+0000 7f58be800700 1 Processor -- start 2024-03-23T18:02:39.654 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.654+0000 7f58be800700 1 -- start start 2024-03-23T18:02:39.656 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.654+0000 7f58be800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58b80680e0 0x7f58b819d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:02:39.656 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.654+0000 7f58be800700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58b8103c00 0x7f58b819dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:02:39.656 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.654+0000 7f58be800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58b8105c40 0x7f58b81a2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:02:39.656 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.654+0000 7f58be800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b81a2640 con 0x7f58b8105c40 2024-03-23T18:02:39.656 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.654+0000 7f58be800700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b81a27b0 con 0x7f58b80680e0 2024-03-23T18:02:39.656 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.654+0000 7f58be800700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b81a2970 con 0x7f58b8103c00 2024-03-23T18:02:39.656 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.654+0000 7f58b7400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58b80680e0 0x7f58b819d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:02:39.656 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.654+0000 7f58b7400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58b80680e0 0x7f58b819d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:32862/0 (socket says 172.21.15.86:32862) 2024-03-23T18:02:39.656 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.655+0000 7f58b7400700 1 -- 172.21.15.86:0/2807139437 learned_addr learned my addr 172.21.15.86:0/2807139437 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:02:39.656 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.655+0000 7f58b6a00700 1 --2- 172.21.15.86:0/2807139437 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58b8103c00 0x7f58b819dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:02:39.656 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.655+0000 7f58b7e00700 1 --2- 172.21.15.86:0/2807139437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58b8105c40 0x7f58b81a2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:02:39.656 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.655+0000 7f58b7400700 1 -- 172.21.15.86:0/2807139437 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58b8103c00 msgr2=0x7f58b819dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:02:39.657 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.655+0000 7f58b7400700 1 --2- 172.21.15.86:0/2807139437 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58b8103c00 0x7f58b819dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:39.657 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.655+0000 7f58b7400700 1 -- 172.21.15.86:0/2807139437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58b8105c40 msgr2=0x7f58b81a2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:02:39.657 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.655+0000 7f58b7400700 1 --2- 172.21.15.86:0/2807139437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58b8105c40 0x7f58b81a2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:39.657 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.655+0000 7f58b7400700 1 -- 172.21.15.86:0/2807139437 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58a80095f0 con 0x7f58b80680e0 2024-03-23T18:02:39.657 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.655+0000 7f58b7e00700 1 --2- 172.21.15.86:0/2807139437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58b8105c40 0x7f58b81a2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T18:02:39.657 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.655+0000 7f58b7400700 1 --2- 172.21.15.86:0/2807139437 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58b80680e0 0x7f58b819d720 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f58a8000c00 tx=0x7f58a8005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:02:39.657 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.655+0000 7f58afe00700 1 -- 172.21.15.86:0/2807139437 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a801d070 con 0x7f58b80680e0 2024-03-23T18:02:39.658 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.655+0000 7f58be800700 1 -- 172.21.15.86:0/2807139437 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58b81a2bf0 con 0x7f58b80680e0 2024-03-23T18:02:39.658 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.655+0000 7f58afe00700 1 -- 172.21.15.86:0/2807139437 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58a8022470 con 0x7f58b80680e0 2024-03-23T18:02:39.658 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.655+0000 7f58afe00700 1 -- 172.21.15.86:0/2807139437 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a8017650 con 0x7f58b80680e0 2024-03-23T18:02:39.658 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.656+0000 7f58be800700 1 -- 172.21.15.86:0/2807139437 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58b81aa490 con 0x7f58b80680e0 2024-03-23T18:02:39.659 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.659+0000 7f58afe00700 1 -- 172.21.15.86:0/2807139437 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f58a80225e0 con 0x7f58b80680e0 2024-03-23T18:02:39.660 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.660+0000 7f58d223ab80 1 Processor -- start 2024-03-23T18:02:39.661 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.660+0000 7f58afe00700 1 --2- 172.21.15.86:0/2807139437 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f589c0a1290 0x7f589c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:02:39.661 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.660+0000 7f58afe00700 1 -- 172.21.15.86:0/2807139437 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f58a80ce110 con 0x7f58b80680e0 2024-03-23T18:02:39.661 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.661+0000 7f58d223ab80 1 -- start start 2024-03-23T18:02:39.661 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.661+0000 7f58b7e00700 1 --2- 172.21.15.86:0/2807139437 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f589c0a1290 0x7f589c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:02:39.669 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.661+0000 7f58d223ab80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564bf192100 0x5564bf1924e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:02:39.670 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.661+0000 7f58d223ab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5564bf192a20 0x5564bf19d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:02:39.670 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.661+0000 7f58d223ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564bf19d8d0 0x5564bf19fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:02:39.670 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.661+0000 7f58d223ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564bf0eb930 con 0x5564bf19d8d0 2024-03-23T18:02:39.670 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.661+0000 7f58d223ab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564bf0e92f0 con 0x5564bf192a20 2024-03-23T18:02:39.670 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.661+0000 7f58d223ab80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564bf0e4be0 con 0x5564bf192100 2024-03-23T18:02:39.670 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.661+0000 7f58b6a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5564bf192a20 0x5564bf19d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:02:39.670 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.661+0000 7f58b6a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5564bf192a20 0x5564bf19d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:32866/0 (socket says 172.21.15.86:32866) 2024-03-23T18:02:39.670 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.661+0000 7f58b6a00700 1 -- 172.21.15.86:0/3154852245 learned_addr learned my addr 172.21.15.86:0/3154852245 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:02:39.670 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.661+0000 7f58b7400700 1 --2- 172.21.15.86:0/3154852245 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564bf192100 0x5564bf1924e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:02:39.670 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.662+0000 7f58b7e00700 1 --2- 172.21.15.86:0/3154852245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564bf19d8d0 0x5564bf19fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:02:39.671 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.662+0000 7f58b6a00700 1 -- 172.21.15.86:0/3154852245 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564bf192100 msgr2=0x5564bf1924e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:02:39.671 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.662+0000 7f58b6a00700 1 --2- 172.21.15.86:0/3154852245 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564bf192100 0x5564bf1924e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:39.671 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.662+0000 7f58b6a00700 1 -- 172.21.15.86:0/3154852245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564bf19d8d0 msgr2=0x5564bf19fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:02:39.671 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.662+0000 7f58b6a00700 1 --2- 172.21.15.86:0/3154852245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564bf19d8d0 0x5564bf19fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:39.671 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.662+0000 7f58b6a00700 1 -- 172.21.15.86:0/3154852245 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5564bef66350 con 0x5564bf192a20 2024-03-23T18:02:39.671 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.662+0000 7f58b7e00700 1 --2- 172.21.15.86:0/3154852245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564bf19d8d0 0x5564bf19fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T18:02:39.671 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.662+0000 7f58b7e00700 1 --2- 172.21.15.86:0/2807139437 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f589c0a1290 0x7f589c0a3750 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f58b00093a0 tx=0x7f58b000b040 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:02:39.671 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.662+0000 7f58b6a00700 1 --2- 172.21.15.86:0/3154852245 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5564bf192a20 0x5564bf19d390 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f58a4009ca0 tx=0x7f58a4011440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a79f3510ec53269 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:02:39.671 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.662+0000 7f58acc00700 1 -- 172.21.15.86:0/3154852245 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a4011d70 con 0x5564bf192a20 2024-03-23T18:02:39.671 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.662+0000 7f58acc00700 1 -- 172.21.15.86:0/3154852245 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58a4004cf0 con 0x5564bf192a20 2024-03-23T18:02:39.671 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.662+0000 7f58acc00700 1 -- 172.21.15.86:0/3154852245 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a4019690 con 0x5564bf192a20 2024-03-23T18:02:39.671 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.663+0000 7f58d223ab80 1 -- 172.21.15.86:0/3154852245 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5564bf192a20 msgr2=0x5564bf19d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:02:39.671 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.663+0000 7f58d223ab80 1 --2- 172.21.15.86:0/3154852245 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5564bf192a20 0x5564bf19d390 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f58a4009ca0 tx=0x7f58a4011440 comp rx=0 tx=0).stop 2024-03-23T18:02:39.672 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.663+0000 7f58d223ab80 1 -- 172.21.15.86:0/3154852245 shutdown_connections 2024-03-23T18:02:39.672 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.663+0000 7f58d223ab80 1 --2- 172.21.15.86:0/3154852245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564bf19d8d0 0x5564bf19fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:39.672 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.663+0000 7f58d223ab80 1 --2- 172.21.15.86:0/3154852245 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564bf192100 0x5564bf1924e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:39.672 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.663+0000 7f58d223ab80 1 --2- 172.21.15.86:0/3154852245 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5564bf192a20 0x5564bf19d390 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:39.672 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.663+0000 7f58d223ab80 1 -- 172.21.15.86:0/3154852245 >> 172.21.15.86:0/3154852245 conn(0x5564bf0f2640 msgr2=0x5564bf18e520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:02:39.672 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.663+0000 7f58d223ab80 1 -- 172.21.15.86:0/3154852245 shutdown_connections 2024-03-23T18:02:39.672 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.663+0000 7f58d223ab80 1 -- 172.21.15.86:0/3154852245 wait complete. 2024-03-23T18:02:39.672 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.664+0000 7f58d223ab80 1 -- start start 2024-03-23T18:02:39.672 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.664+0000 7f58d223ab80 1 Processor -- start 2024-03-23T18:02:39.672 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.665+0000 7f58d223ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564bf192100 0x5564bf19d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:02:39.672 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.665+0000 7f58d223ab80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564bf192a20 0x5564bf19a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:02:39.672 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.665+0000 7f58d223ab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5564bf19d8d0 0x5564bf19ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:02:39.672 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.665+0000 7f58d223ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564bf080d20 con 0x5564bf192100 2024-03-23T18:02:39.673 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.665+0000 7f58d223ab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564bf0809c0 con 0x5564bf19d8d0 2024-03-23T18:02:39.673 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.665+0000 7f58d223ab80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564bef68640 con 0x5564bf192a20 2024-03-23T18:02:39.673 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.665+0000 7f58d223ab80 10 client.0 _ll_register_callbacks cb 0x5564bf08a410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T18:02:39.673 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.665+0000 7f58b7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564bf192100 0x5564bf19d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:02:39.673 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.665+0000 7f58b7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564bf192100 0x5564bf19d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60550/0 (socket says 172.21.15.86:60550) 2024-03-23T18:02:39.673 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.665+0000 7f58b7400700 1 -- 172.21.15.86:0/1705387504 learned_addr learned my addr 172.21.15.86:0/1705387504 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:02:39.673 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.665+0000 7f58b7e00700 1 --2- 172.21.15.86:0/1705387504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5564bf19d8d0 0x5564bf19ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:02:39.673 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.665+0000 7f58b6a00700 1 --2- 172.21.15.86:0/1705387504 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564bf192a20 0x5564bf19a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:02:39.673 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.665+0000 7f58b7400700 1 -- 172.21.15.86:0/1705387504 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564bf192a20 msgr2=0x5564bf19a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:02:39.673 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.665+0000 7f58b7400700 1 --2- 172.21.15.86:0/1705387504 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564bf192a20 0x5564bf19a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:39.673 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.665+0000 7f58b7400700 1 -- 172.21.15.86:0/1705387504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5564bf19d8d0 msgr2=0x5564bf19ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:02:39.673 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.665+0000 7f58b7400700 1 --2- 172.21.15.86:0/1705387504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5564bf19d8d0 0x5564bf19ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:39.673 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.665+0000 7f58b7400700 1 -- 172.21.15.86:0/1705387504 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58a4009900 con 0x5564bf192100 2024-03-23T18:02:39.674 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.665+0000 7f58b7e00700 1 --2- 172.21.15.86:0/1705387504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5564bf19d8d0 0x5564bf19ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T18:02:39.674 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.666+0000 7f58b7400700 1 --2- 172.21.15.86:0/1705387504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564bf192100 0x5564bf19d360 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f58a809b320 tx=0x7f58a809b160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bf0732e1e737bd9 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:02:39.674 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.666+0000 7f58a1600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-23T18:02:39.674 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.666+0000 7f58a1600700 1 -- 172.21.15.86:0/1705387504 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a809acf0 con 0x5564bf192100 2024-03-23T18:02:39.674 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.666+0000 7f58a1600700 1 -- 172.21.15.86:0/1705387504 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f58a809ba90 con 0x5564bf192100 2024-03-23T18:02:39.674 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.666+0000 7f58d223ab80 1 -- 172.21.15.86:0/1705387504 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5564bf0a5320 con 0x5564bf192100 2024-03-23T18:02:39.674 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.666+0000 7f58a1600700 1 -- 172.21.15.86:0/1705387504 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f58a809b390 con 0x5564bf192100 2024-03-23T18:02:39.674 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.666+0000 7f58d223ab80 10 client.16941 fetch_fsmap learned FSMap version 52 2024-03-23T18:02:39.674 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.666+0000 7f58d223ab80 1 -- 172.21.15.86:0/1705387504 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x5564bef12b00 con 0x5564bf192100 2024-03-23T18:02:39.674 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.666+0000 7f58a1600700 1 -- 172.21.15.86:0/1705387504 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5564bf0a5320 con 0x5564bf192100 2024-03-23T18:02:39.674 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.666+0000 7f58a1600700 1 -- 172.21.15.86:0/1705387504 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 52) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f58a80a32f0 con 0x5564bf192100 2024-03-23T18:02:39.674 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.667+0000 7f58d223ab80 10 client.16941 fetch_fsmap finished waiting for FSMap version 52 2024-03-23T18:02:39.674 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.667+0000 7f58d223ab80 10 client.16941 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T18:02:39.675 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.667+0000 7f58d223ab80 20 client.16941 populate_metadata read hostname 'smithi086' 2024-03-23T18:02:39.675 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.667+0000 7f58d223ab80 1 --2- 172.21.15.86:0/1705387504 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5564bf1ad260 0x5564bf1af680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:02:39.675 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.667+0000 7f58d223ab80 4 client.16941 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T18:02:39.675 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.667+0000 7f58d223ab80 1 -- 172.21.15.86:0/1705387504 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5564bf0dd6a0 con 0x5564bf1ad260 2024-03-23T18:02:39.675 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.667+0000 7f58b6a00700 1 --2- 172.21.15.86:0/1705387504 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5564bf1ad260 0x5564bf1af680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:02:39.675 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.667+0000 7f58a1600700 10 client.16941 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T18:02:39.675 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.667+0000 7f58b6a00700 1 --2- 172.21.15.86:0/1705387504 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5564bf1ad260 0x5564bf1af680 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5880d41201c3e495 server_cookie=7a961bf7bfb5c3ad in_seq=0 out_seq=0 2024-03-23T18:02:39.675 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.668+0000 7f58a1600700 1 -- 172.21.15.86:0/1705387504 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5564bf0dd6a0 con 0x5564bf1ad260 2024-03-23T18:02:39.675 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.668+0000 7f58a1600700 10 client.16941 handle_command_reply: tid=0 2024-03-23T18:02:39.675 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.668+0000 7f58d223ab80 1 client.16941 shutdown 2024-03-23T18:02:39.675 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.668+0000 7f58d223ab80 1 -- 172.21.15.86:0/1705387504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564bf192100 msgr2=0x5564bf19d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:02:39.675 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.668+0000 7f58d223ab80 1 --2- 172.21.15.86:0/1705387504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564bf192100 0x5564bf19d360 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f58a809b320 tx=0x7f58a809b160 comp rx=0 tx=0).stop 2024-03-23T18:02:39.675 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.668+0000 7f58d223ab80 1 -- 172.21.15.86:0/1705387504 shutdown_connections 2024-03-23T18:02:39.676 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.668+0000 7f58d223ab80 1 --2- 172.21.15.86:0/1705387504 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5564bf1ad260 0x5564bf1af680 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:39.676 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.668+0000 7f58d223ab80 1 --2- 172.21.15.86:0/1705387504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5564bf19d8d0 0x5564bf19ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:39.676 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.668+0000 7f58d223ab80 1 --2- 172.21.15.86:0/1705387504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564bf192100 0x5564bf19d360 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:39.676 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.668+0000 7f58a1600700 0 client.16941 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T18:02:39.676 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.668+0000 7f58d223ab80 1 --2- 172.21.15.86:0/1705387504 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564bf192a20 0x5564bf19a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:39.676 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.668+0000 7f58d223ab80 1 -- 172.21.15.86:0/1705387504 >> 172.21.15.86:0/1705387504 conn(0x5564bf0f2640 msgr2=0x5564bf1940a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:02:39.676 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.669+0000 7f58d223ab80 1 -- 172.21.15.86:0/1705387504 shutdown_connections 2024-03-23T18:02:39.676 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.669+0000 7f58d223ab80 1 -- 172.21.15.86:0/1705387504 wait complete. 2024-03-23T18:02:39.676 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.669+0000 7f58d223ab80 20 client.16941 trim_cache size 0 max 16384 2024-03-23T18:02:39.686 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.684+0000 7f58d223ab80 1 Processor -- start 2024-03-23T18:02:39.686 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.684+0000 7f58d223ab80 1 -- start start 2024-03-23T18:02:39.686 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.685+0000 7f58d223ab80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564bf192100 0x5564bf1a45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:02:39.686 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.685+0000 7f58d223ab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5564bf192a20 0x5564bf187c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:02:39.686 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.685+0000 7f58d223ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564bf19d8d0 0x5564bf188500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:02:39.686 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.685+0000 7f58d223ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564bf1ad070 con 0x5564bf19d8d0 2024-03-23T18:02:39.686 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.685+0000 7f58d223ab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564bf159780 con 0x5564bf192a20 2024-03-23T18:02:39.686 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.685+0000 7f58d223ab80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564bf18ac80 con 0x5564bf192100 2024-03-23T18:02:39.687 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.685+0000 7f58b7400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564bf192100 0x5564bf1a45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:02:39.687 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.685+0000 7f58b6a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5564bf192a20 0x5564bf187c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:02:39.687 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.685+0000 7f58b7400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564bf192100 0x5564bf1a45a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:37908/0 (socket says 172.21.15.86:37908) 2024-03-23T18:02:39.687 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.685+0000 7f58b7400700 1 -- 172.21.15.86:0/1919145114 learned_addr learned my addr 172.21.15.86:0/1919145114 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:02:39.687 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.685+0000 7f58b7e00700 1 --2- 172.21.15.86:0/1919145114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564bf19d8d0 0x5564bf188500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:02:39.687 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.685+0000 7f58b7400700 1 -- 172.21.15.86:0/1919145114 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5564bf192a20 msgr2=0x5564bf187c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:02:39.687 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.685+0000 7f58b7400700 1 --2- 172.21.15.86:0/1919145114 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5564bf192a20 0x5564bf187c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:39.687 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.685+0000 7f58b7400700 1 -- 172.21.15.86:0/1919145114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564bf19d8d0 msgr2=0x5564bf188500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:02:39.687 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.685+0000 7f58b7400700 1 --2- 172.21.15.86:0/1919145114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564bf19d8d0 0x5564bf188500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:39.687 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.685+0000 7f58b7400700 1 -- 172.21.15.86:0/1919145114 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5564bf0aae40 con 0x5564bf192100 2024-03-23T18:02:39.687 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.685+0000 7f58b6a00700 1 --2- 172.21.15.86:0/1919145114 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5564bf192a20 0x5564bf187c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T18:02:39.687 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.686+0000 7f58b7400700 1 --2- 172.21.15.86:0/1919145114 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564bf192100 0x5564bf1a45a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f58a800f740 tx=0x7f58a8005f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb35d07a074074ab server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:02:39.688 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.686+0000 7f58ad600700 1 -- 172.21.15.86:0/1919145114 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a809acf0 con 0x5564bf192100 2024-03-23T18:02:39.688 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.686+0000 7f58ad600700 1 -- 172.21.15.86:0/1919145114 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58a809ba90 con 0x5564bf192100 2024-03-23T18:02:39.688 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.686+0000 7f58ad600700 1 -- 172.21.15.86:0/1919145114 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a809a880 con 0x5564bf192100 2024-03-23T18:02:39.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.687+0000 7f58d223ab80 1 -- 172.21.15.86:0/1919145114 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564bf192100 msgr2=0x5564bf1a45a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:02:39.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.687+0000 7f58d223ab80 1 --2- 172.21.15.86:0/1919145114 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564bf192100 0x5564bf1a45a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f58a800f740 tx=0x7f58a8005f10 comp rx=0 tx=0).stop 2024-03-23T18:02:39.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.687+0000 7f58d223ab80 1 -- 172.21.15.86:0/1919145114 shutdown_connections 2024-03-23T18:02:39.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.687+0000 7f58d223ab80 1 --2- 172.21.15.86:0/1919145114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564bf19d8d0 0x5564bf188500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:39.692 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.687+0000 7f58d223ab80 1 --2- 172.21.15.86:0/1919145114 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564bf192100 0x5564bf1a45a0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:39.692 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.687+0000 7f58d223ab80 1 --2- 172.21.15.86:0/1919145114 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5564bf192a20 0x5564bf187c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:39.692 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.687+0000 7f58d223ab80 1 -- 172.21.15.86:0/1919145114 >> 172.21.15.86:0/1919145114 conn(0x5564bf0f2640 msgr2=0x5564bf18dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:02:39.692 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.687+0000 7f58d223ab80 1 -- 172.21.15.86:0/1919145114 shutdown_connections 2024-03-23T18:02:39.692 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.687+0000 7f58d223ab80 1 -- 172.21.15.86:0/1919145114 wait complete. 2024-03-23T18:02:39.692 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.687+0000 7f58d223ab80 1 -- start start 2024-03-23T18:02:39.692 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.687+0000 7f58d223ab80 1 Processor -- start 2024-03-23T18:02:39.692 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.688+0000 7f58d223ab80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564bf192100 0x5564bf1a2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:02:39.692 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.688+0000 7f58d223ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564bf192a20 0x5564bf1a2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:02:39.692 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.688+0000 7f58d223ab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5564bf19d8d0 0x5564bf1a3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:02:39.692 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.688+0000 7f58d223ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564bf1a49e0 con 0x5564bf192a20 2024-03-23T18:02:39.692 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.688+0000 7f58d223ab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564bf1fdd80 con 0x5564bf19d8d0 2024-03-23T18:02:39.692 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.688+0000 7f58d223ab80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564bf1fdf40 con 0x5564bf192100 2024-03-23T18:02:39.693 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.688+0000 7f58d223ab80 10 client.0 _ll_register_callbacks cb 0x5564bf08a410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T18:02:39.693 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.688+0000 7f58b7400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564bf192100 0x5564bf1a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:02:39.693 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.688+0000 7f58b7400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564bf192100 0x5564bf1a2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:37916/0 (socket says 172.21.15.86:37916) 2024-03-23T18:02:39.693 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.688+0000 7f58b7400700 1 -- 172.21.15.86:0/2010426545 learned_addr learned my addr 172.21.15.86:0/2010426545 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:02:39.693 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.688+0000 7f58b7e00700 1 --2- 172.21.15.86:0/2010426545 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5564bf19d8d0 0x5564bf1a3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:02:39.693 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.688+0000 7f58b6a00700 1 --2- 172.21.15.86:0/2010426545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564bf192a20 0x5564bf1a2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:02:39.693 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.689+0000 7f58b7400700 1 -- 172.21.15.86:0/2010426545 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5564bf19d8d0 msgr2=0x5564bf1a3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:02:39.693 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.689+0000 7f58b7400700 1 --2- 172.21.15.86:0/2010426545 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5564bf19d8d0 0x5564bf1a3000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:39.693 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.689+0000 7f58b7400700 1 -- 172.21.15.86:0/2010426545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564bf192a20 msgr2=0x5564bf1a2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:02:39.693 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.689+0000 7f58b7400700 1 --2- 172.21.15.86:0/2010426545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564bf192a20 0x5564bf1a2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:39.693 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.689+0000 7f58b7400700 1 -- 172.21.15.86:0/2010426545 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5564bef66350 con 0x5564bf192100 2024-03-23T18:02:39.693 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.689+0000 7f58b7400700 1 --2- 172.21.15.86:0/2010426545 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564bf192100 0x5564bf1a2580 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f58a8097e00 tx=0x7f58a8005ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de14d4be5ae66edb server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:02:39.693 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.689+0000 7f58a1600700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-03-23T18:02:39.693 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.689+0000 7f58a1600700 1 -- 172.21.15.86:0/2010426545 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a80a1480 con 0x5564bf192100 2024-03-23T18:02:39.694 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.689+0000 7f58a1600700 1 -- 172.21.15.86:0/2010426545 <== mon.1 v2:172.21.15.152:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f58a809b390 con 0x5564bf192100 2024-03-23T18:02:39.694 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.689+0000 7f58d223ab80 1 -- 172.21.15.86:0/2010426545 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f58a4004cf0 con 0x5564bf192100 2024-03-23T18:02:39.694 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.689+0000 7f58a1600700 1 -- 172.21.15.86:0/2010426545 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f58a809acf0 con 0x5564bf192100 2024-03-23T18:02:39.694 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.689+0000 7f58d223ab80 10 client.26989 fetch_fsmap learned FSMap version 52 2024-03-23T18:02:39.694 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.689+0000 7f58d223ab80 1 -- 172.21.15.86:0/2010426545 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x5564bef12b00 con 0x5564bf192100 2024-03-23T18:02:39.694 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.690+0000 7f58a1600700 1 -- 172.21.15.86:0/2010426545 <== mon.1 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58a4004cf0 con 0x5564bf192100 2024-03-23T18:02:39.694 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.690+0000 7f58a1600700 1 -- 172.21.15.86:0/2010426545 <== mon.1 v2:172.21.15.152:3300/0 5 ==== fsmap(e 52) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f58a809a2f0 con 0x5564bf192100 2024-03-23T18:02:39.694 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.690+0000 7f58d223ab80 10 client.26989 fetch_fsmap finished waiting for FSMap version 52 2024-03-23T18:02:39.694 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.690+0000 7f58d223ab80 10 client.26989 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T18:02:39.694 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.690+0000 7f58d223ab80 20 client.26989 populate_metadata read hostname 'smithi086' 2024-03-23T18:02:39.694 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.690+0000 7f58d223ab80 1 --2- 172.21.15.86:0/2010426545 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5564bf1ff290 0x5564bf201680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:02:39.694 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.690+0000 7f58d223ab80 4 client.26989 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T18:02:39.694 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.690+0000 7f58d223ab80 1 -- 172.21.15.86:0/2010426545 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5564bef33c90 con 0x5564bf1ff290 2024-03-23T18:02:39.695 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.690+0000 7f58b6a00700 1 --2- 172.21.15.86:0/2010426545 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5564bf1ff290 0x5564bf201680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:02:39.695 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.691+0000 7f58b6a00700 1 --2- 172.21.15.86:0/2010426545 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5564bf1ff290 0x5564bf201680 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9517a34f316947f5 server_cookie=10bc02780b9c7181 in_seq=0 out_seq=0 2024-03-23T18:02:39.695 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.691+0000 7f58a1600700 10 client.26989 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T18:02:39.732 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.732+0000 7f58a1600700 1 -- 172.21.15.86:0/2010426545 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5564bef33c90 con 0x5564bf1ff290 2024-03-23T18:02:39.732 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.732+0000 7f58a1600700 10 client.26989 handle_command_reply: tid=0 2024-03-23T18:02:39.732 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.732+0000 7f58d223ab80 1 client.26989 shutdown 2024-03-23T18:02:39.732 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.732+0000 7f58d223ab80 1 -- 172.21.15.86:0/2010426545 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564bf192100 msgr2=0x5564bf1a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:02:39.733 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.732+0000 7f58d223ab80 1 --2- 172.21.15.86:0/2010426545 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564bf192100 0x5564bf1a2580 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f58a8097e00 tx=0x7f58a8005ea0 comp rx=0 tx=0).stop 2024-03-23T18:02:39.733 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.732+0000 7f58d223ab80 1 -- 172.21.15.86:0/2010426545 shutdown_connections 2024-03-23T18:02:39.733 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.732+0000 7f58d223ab80 1 --2- 172.21.15.86:0/2010426545 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5564bf1ff290 0x5564bf201680 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:39.733 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.732+0000 7f58d223ab80 1 --2- 172.21.15.86:0/2010426545 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5564bf19d8d0 0x5564bf1a3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:39.733 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.732+0000 7f58d223ab80 1 --2- 172.21.15.86:0/2010426545 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564bf192100 0x5564bf1a2580 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:39.733 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.732+0000 7f58d223ab80 1 --2- 172.21.15.86:0/2010426545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564bf192a20 0x5564bf1a2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:39.733 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.732+0000 7f58d223ab80 1 -- 172.21.15.86:0/2010426545 >> 172.21.15.86:0/2010426545 conn(0x5564bf0f2640 msgr2=0x5564bf198eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:02:39.733 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.732+0000 7f58a1600700 0 client.26989 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T18:02:39.734 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.733+0000 7f58d223ab80 1 -- 172.21.15.86:0/2010426545 shutdown_connections 2024-03-23T18:02:39.734 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.733+0000 7f58d223ab80 1 -- 172.21.15.86:0/2010426545 wait complete. 2024-03-23T18:02:39.735 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.733+0000 7f58d223ab80 20 client.26989 trim_cache size 0 max 16384 2024-03-23T18:02:39.735 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-03-23T18:02:39.735 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-03-23T18:02:39.735 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-03-23T18:02:39.735 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-03-23T18:02:39.736 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.734+0000 7f58be800700 1 -- 172.21.15.86:0/2807139437 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f589c0a1290 msgr2=0x7f589c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:02:39.736 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.734+0000 7f58be800700 1 --2- 172.21.15.86:0/2807139437 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f589c0a1290 0x7f589c0a3750 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f58b00093a0 tx=0x7f58b000b040 comp rx=0 tx=0).stop 2024-03-23T18:02:39.736 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.734+0000 7f58be800700 1 -- 172.21.15.86:0/2807139437 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58b80680e0 msgr2=0x7f58b819d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:02:39.736 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.734+0000 7f58be800700 1 --2- 172.21.15.86:0/2807139437 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58b80680e0 0x7f58b819d720 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f58a8000c00 tx=0x7f58a8005740 comp rx=0 tx=0).stop 2024-03-23T18:02:39.736 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.734+0000 7f58be800700 1 -- 172.21.15.86:0/2807139437 shutdown_connections 2024-03-23T18:02:39.736 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.734+0000 7f58be800700 1 --2- 172.21.15.86:0/2807139437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58b8105c40 0x7f58b81a2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:39.738 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.734+0000 7f58be800700 1 --2- 172.21.15.86:0/2807139437 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58b80680e0 0x7f58b819d720 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:39.738 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.734+0000 7f58be800700 1 --2- 172.21.15.86:0/2807139437 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f589c0a1290 0x7f589c0a3750 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:39.738 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.734+0000 7f58be800700 1 --2- 172.21.15.86:0/2807139437 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58b8103c00 0x7f58b819dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:39.738 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.734+0000 7f58be800700 1 -- 172.21.15.86:0/2807139437 >> 172.21.15.86:0/2807139437 conn(0x7f58b8076d50 msgr2=0x7f58b80fd9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:02:39.738 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.734+0000 7f58be800700 1 -- 172.21.15.86:0/2807139437 shutdown_connections 2024-03-23T18:02:39.738 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:39.734+0000 7f58be800700 1 -- 172.21.15.86:0/2807139437 wait complete. 2024-03-23T18:02:39.927 INFO:tasks.workunit.client.0.smithi086.stdout: 1 725875 6.41 MB/sec execute 544 sec latency 20.964 ms 2024-03-23T18:02:40.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:02:39 smithi086 ceph-mon[38221]: pgmap v376: 97 pgs: 97 active+clean; 282 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 838 KiB/s rd, 1.8 MiB/s wr, 172 op/s 2024-03-23T18:02:40.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:02:39 smithi092 ceph-mon[79089]: pgmap v376: 97 pgs: 97 active+clean; 282 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 838 KiB/s rd, 1.8 MiB/s wr, 172 op/s 2024-03-23T18:02:40.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:02:39 smithi152 ceph-mon[47949]: pgmap v376: 97 pgs: 97 active+clean; 282 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 838 KiB/s rd, 1.8 MiB/s wr, 172 op/s 2024-03-23T18:02:40.927 INFO:tasks.workunit.client.0.smithi086.stdout: 1 726099 6.40 MB/sec execute 545 sec latency 828.959 ms 2024-03-23T18:02:41.927 INFO:tasks.workunit.client.0.smithi086.stdout: 1 728150 6.41 MB/sec execute 546 sec latency 17.023 ms 2024-03-23T18:02:42.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:02:41 smithi086 ceph-mon[38221]: pgmap v377: 97 pgs: 97 active+clean; 287 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 255 op/s 2024-03-23T18:02:42.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:02:41 smithi092 ceph-mon[79089]: pgmap v377: 97 pgs: 97 active+clean; 287 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 255 op/s 2024-03-23T18:02:42.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:02:41 smithi152 ceph-mon[47949]: pgmap v377: 97 pgs: 97 active+clean; 287 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 255 op/s 2024-03-23T18:02:42.927 INFO:tasks.workunit.client.0.smithi086.stdout: 1 729178 6.40 MB/sec execute 547 sec latency 18.298 ms 2024-03-23T18:02:43.928 INFO:tasks.workunit.client.0.smithi086.stdout: 1 729638 6.40 MB/sec execute 548 sec latency 725.278 ms 2024-03-23T18:02:44.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:02:43 smithi086 ceph-mon[38221]: pgmap v378: 97 pgs: 97 active+clean; 287 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 1.7 MiB/s wr, 148 op/s 2024-03-23T18:02:44.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:02:43 smithi092 ceph-mon[79089]: pgmap v378: 97 pgs: 97 active+clean; 287 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 1.7 MiB/s wr, 148 op/s 2024-03-23T18:02:44.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:02:43 smithi152 ceph-mon[47949]: pgmap v378: 97 pgs: 97 active+clean; 287 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 1.7 MiB/s wr, 148 op/s 2024-03-23T18:02:44.928 INFO:tasks.workunit.client.0.smithi086.stdout: 1 730336 6.39 MB/sec execute 549 sec latency 1424.292 ms 2024-03-23T18:02:45.928 INFO:tasks.workunit.client.0.smithi086.stdout: 1 731867 6.40 MB/sec execute 550 sec latency 12.509 ms 2024-03-23T18:02:46.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:02:45 smithi086 ceph-mon[38221]: pgmap v379: 97 pgs: 97 active+clean; 283 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 230 op/s 2024-03-23T18:02:46.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:02:45 smithi092 ceph-mon[79089]: pgmap v379: 97 pgs: 97 active+clean; 283 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 230 op/s 2024-03-23T18:02:46.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:02:45 smithi152 ceph-mon[47949]: pgmap v379: 97 pgs: 97 active+clean; 283 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 230 op/s 2024-03-23T18:02:46.928 INFO:tasks.workunit.client.0.smithi086.stdout: 1 733187 6.40 MB/sec execute 551 sec latency 42.106 ms 2024-03-23T18:02:47.928 INFO:tasks.workunit.client.0.smithi086.stdout: 1 733696 6.39 MB/sec execute 552 sec latency 871.500 ms 2024-03-23T18:02:48.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:02:47 smithi086 ceph-mon[38221]: pgmap v380: 97 pgs: 97 active+clean; 283 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 807 KiB/s rd, 1.9 MiB/s wr, 164 op/s 2024-03-23T18:02:48.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:02:47 smithi092 ceph-mon[79089]: pgmap v380: 97 pgs: 97 active+clean; 283 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 807 KiB/s rd, 1.9 MiB/s wr, 164 op/s 2024-03-23T18:02:48.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:02:47 smithi152 ceph-mon[47949]: pgmap v380: 97 pgs: 97 active+clean; 283 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 807 KiB/s rd, 1.9 MiB/s wr, 164 op/s 2024-03-23T18:02:48.928 INFO:tasks.workunit.client.0.smithi086.stdout: 1 735342 6.40 MB/sec execute 553 sec latency 18.213 ms 2024-03-23T18:02:49.928 INFO:tasks.workunit.client.0.smithi086.stdout: 1 736696 6.40 MB/sec execute 554 sec latency 13.976 ms 2024-03-23T18:02:50.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:02:50 smithi086 ceph-mon[38221]: pgmap v381: 97 pgs: 97 active+clean; 283 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 807 KiB/s rd, 1.9 MiB/s wr, 164 op/s 2024-03-23T18:02:50.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:02:50 smithi092 ceph-mon[79089]: pgmap v381: 97 pgs: 97 active+clean; 283 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 807 KiB/s rd, 1.9 MiB/s wr, 164 op/s 2024-03-23T18:02:50.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:02:50 smithi152 ceph-mon[47949]: pgmap v381: 97 pgs: 97 active+clean; 283 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 807 KiB/s rd, 1.9 MiB/s wr, 164 op/s 2024-03-23T18:02:50.928 INFO:tasks.workunit.client.0.smithi086.stdout: 1 736732 6.39 MB/sec execute 555 sec latency 970.605 ms 2024-03-23T18:02:51.928 INFO:tasks.workunit.client.0.smithi086.stdout: 1 738818 6.40 MB/sec execute 556 sec latency 15.827 ms 2024-03-23T18:02:52.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:02:52 smithi086 ceph-mon[38221]: pgmap v382: 97 pgs: 97 active+clean; 289 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 253 op/s 2024-03-23T18:02:52.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:02:52 smithi092 ceph-mon[79089]: pgmap v382: 97 pgs: 97 active+clean; 289 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 253 op/s 2024-03-23T18:02:52.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:02:52 smithi152 ceph-mon[47949]: pgmap v382: 97 pgs: 97 active+clean; 289 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 253 op/s 2024-03-23T18:02:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.895+0000 7f5f99600700 1 -- 172.21.15.86:0/3179894227 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f94106b80 msgr2=0x7f5f94106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:02:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.895+0000 7f5f99600700 1 --2- 172.21.15.86:0/3179894227 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f94106b80 0x7f5f94106f60 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5f84009990 tx=0x7f5f84009ca0 comp rx=0 tx=0).stop 2024-03-23T18:02:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.895+0000 7f5f99600700 1 -- 172.21.15.86:0/3179894227 shutdown_connections 2024-03-23T18:02:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.895+0000 7f5f99600700 1 --2- 172.21.15.86:0/3179894227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f941014a0 0x7f5f9410d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.895+0000 7f5f99600700 1 --2- 172.21.15.86:0/3179894227 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f94106b80 0x7f5f94106f60 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.895+0000 7f5f99600700 1 --2- 172.21.15.86:0/3179894227 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f94100ae0 0x7f5f94100f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.895+0000 7f5f99600700 1 -- 172.21.15.86:0/3179894227 >> 172.21.15.86:0/3179894227 conn(0x7f5f940fc610 msgr2=0x7f5f940fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:02:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.895+0000 7f5f99600700 1 -- 172.21.15.86:0/3179894227 shutdown_connections 2024-03-23T18:02:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.895+0000 7f5f99600700 1 -- 172.21.15.86:0/3179894227 wait complete. 2024-03-23T18:02:52.898 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.896+0000 7f5f99600700 1 Processor -- start 2024-03-23T18:02:52.898 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.896+0000 7f5f99600700 1 -- start start 2024-03-23T18:02:52.898 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.897+0000 7f5f99600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f94100ae0 0x7f5f940730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:02:52.898 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.897+0000 7f5f99600700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f941014a0 0x7f5f940735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:02:52.898 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.897+0000 7f5f99600700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f94106b80 0x7f5f94073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:02:52.898 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.897+0000 7f5f99600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f941a5320 con 0x7f5f94100ae0 2024-03-23T18:02:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.897+0000 7f5f99600700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f941a5490 con 0x7f5f94106b80 2024-03-23T18:02:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.897+0000 7f5f99600700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f941a5650 con 0x7f5f941014a0 2024-03-23T18:02:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.897+0000 7f5f98c00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f94106b80 0x7f5f94073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:02:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.897+0000 7f5f98c00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f94106b80 0x7f5f94073b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:50238/0 (socket says 172.21.15.86:50238) 2024-03-23T18:02:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.897+0000 7f5f98c00700 1 -- 172.21.15.86:0/1442347013 learned_addr learned my addr 172.21.15.86:0/1442347013 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:02:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.897+0000 7f5f93e00700 1 --2- 172.21.15.86:0/1442347013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f94100ae0 0x7f5f940730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:02:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.897+0000 7f5f8be00700 1 --2- 172.21.15.86:0/1442347013 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f941014a0 0x7f5f940735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:02:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.897+0000 7f5f98c00700 1 -- 172.21.15.86:0/1442347013 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f941014a0 msgr2=0x7f5f940735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:02:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.897+0000 7f5f98c00700 1 --2- 172.21.15.86:0/1442347013 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f941014a0 0x7f5f940735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.897+0000 7f5f98c00700 1 -- 172.21.15.86:0/1442347013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f94100ae0 msgr2=0x7f5f940730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:02:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.897+0000 7f5f98c00700 1 --2- 172.21.15.86:0/1442347013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f94100ae0 0x7f5f940730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.897+0000 7f5f98c00700 1 -- 172.21.15.86:0/1442347013 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f840095f0 con 0x7f5f94106b80 2024-03-23T18:02:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.897+0000 7f5f8be00700 1 --2- 172.21.15.86:0/1442347013 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f941014a0 0x7f5f940735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T18:02:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.898+0000 7f5f98c00700 1 --2- 172.21.15.86:0/1442347013 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f94106b80 0x7f5f94073b30 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c007ee0 tx=0x7f5f8c00d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:02:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.898+0000 7f5f91600700 1 -- 172.21.15.86:0/1442347013 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f8c00ddd0 con 0x7f5f94106b80 2024-03-23T18:02:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.898+0000 7f5f99600700 1 -- 172.21.15.86:0/1442347013 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f941a5930 con 0x7f5f94106b80 2024-03-23T18:02:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.898+0000 7f5f91600700 1 -- 172.21.15.86:0/1442347013 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f8c00f040 con 0x7f5f94106b80 2024-03-23T18:02:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.898+0000 7f5f91600700 1 -- 172.21.15.86:0/1442347013 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f8c0048e0 con 0x7f5f94106b80 2024-03-23T18:02:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.898+0000 7f5f99600700 1 -- 172.21.15.86:0/1442347013 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f941a5e40 con 0x7f5f94106b80 2024-03-23T18:02:52.901 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.901+0000 7f5f91600700 1 -- 172.21.15.86:0/1442347013 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f5f8c003a00 con 0x7f5f94106b80 2024-03-23T18:02:52.904 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.902+0000 7f5fae6b7b80 1 Processor -- start 2024-03-23T18:02:52.904 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.902+0000 7f5fae6b7b80 1 -- start start 2024-03-23T18:02:52.904 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.902+0000 7f5f91600700 1 --2- 172.21.15.86:0/1442347013 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f5f780a1290 0x7f5f780a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:02:52.904 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.903+0000 7f5f91600700 1 -- 172.21.15.86:0/1442347013 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f5f8c0cc5a0 con 0x7f5f94106b80 2024-03-23T18:02:52.904 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.903+0000 7f5fae6b7b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fc69492d0 0x561fc69496b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:02:52.904 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.903+0000 7f5fae6b7b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561fc6949bf0 0x561fc695cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:02:52.904 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.903+0000 7f5fae6b7b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561fc695d2c0 0x561fc695f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:02:52.904 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.903+0000 7f5fae6b7b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fc68a2930 con 0x561fc69492d0 2024-03-23T18:02:52.904 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.903+0000 7f5fae6b7b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fc68a02f0 con 0x561fc6949bf0 2024-03-23T18:02:52.904 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.903+0000 7f5fae6b7b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fc689bbe0 con 0x561fc695d2c0 2024-03-23T18:02:52.904 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.903+0000 7f5f8be00700 1 --2- 172.21.15.86:0/1442347013 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f5f780a1290 0x7f5f780a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:02:52.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.903+0000 7f5f93e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fc69492d0 0x561fc69496b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:02:52.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.903+0000 7f5f93e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fc69492d0 0x561fc69496b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35078/0 (socket says 172.21.15.86:35078) 2024-03-23T18:02:52.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.903+0000 7f5f93e00700 1 -- 172.21.15.86:0/2552943323 learned_addr learned my addr 172.21.15.86:0/2552943323 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:02:52.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.903+0000 7f5f98c00700 1 --2- 172.21.15.86:0/2552943323 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561fc695d2c0 0x561fc695f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:02:52.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.903+0000 7f5f8be00700 1 --2- 172.21.15.86:0/2552943323 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561fc6949bf0 0x561fc695cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:02:52.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.904+0000 7f5f93e00700 1 -- 172.21.15.86:0/2552943323 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561fc695d2c0 msgr2=0x561fc695f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:02:52.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.904+0000 7f5f93e00700 1 --2- 172.21.15.86:0/2552943323 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561fc695d2c0 0x561fc695f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:52.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.904+0000 7f5f93e00700 1 -- 172.21.15.86:0/2552943323 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561fc6949bf0 msgr2=0x561fc695cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:02:52.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.904+0000 7f5f93e00700 1 --2- 172.21.15.86:0/2552943323 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561fc6949bf0 0x561fc695cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:52.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.904+0000 7f5f93e00700 1 -- 172.21.15.86:0/2552943323 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561fc671d350 con 0x561fc69492d0 2024-03-23T18:02:52.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.904+0000 7f5f8be00700 1 --2- 172.21.15.86:0/1442347013 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f5f780a1290 0x7f5f780a3750 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c005fd0 tx=0x7f5f7c0158c0 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:02:52.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.904+0000 7f5f93e00700 1 --2- 172.21.15.86:0/2552943323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fc69492d0 0x561fc69496b0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f5f84005f30 tx=0x7f5f84009550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a85d50356b0749db server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:02:52.909 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.904+0000 7f5f89200700 1 -- 172.21.15.86:0/2552943323 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f84005a00 con 0x561fc69492d0 2024-03-23T18:02:52.909 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.904+0000 7f5f89200700 1 -- 172.21.15.86:0/2552943323 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f8401e040 con 0x561fc69492d0 2024-03-23T18:02:52.909 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.904+0000 7f5f89200700 1 -- 172.21.15.86:0/2552943323 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f840039f0 con 0x561fc69492d0 2024-03-23T18:02:52.909 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.905+0000 7f5fae6b7b80 1 -- 172.21.15.86:0/2552943323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fc69492d0 msgr2=0x561fc69496b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:02:52.909 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.905+0000 7f5fae6b7b80 1 --2- 172.21.15.86:0/2552943323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fc69492d0 0x561fc69496b0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f5f84005f30 tx=0x7f5f84009550 comp rx=0 tx=0).stop 2024-03-23T18:02:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.905+0000 7f5fae6b7b80 1 -- 172.21.15.86:0/2552943323 shutdown_connections 2024-03-23T18:02:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.905+0000 7f5fae6b7b80 1 --2- 172.21.15.86:0/2552943323 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561fc695d2c0 0x561fc695f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.905+0000 7f5fae6b7b80 1 --2- 172.21.15.86:0/2552943323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fc69492d0 0x561fc69496b0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.905+0000 7f5fae6b7b80 1 --2- 172.21.15.86:0/2552943323 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561fc6949bf0 0x561fc695cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.905+0000 7f5fae6b7b80 1 -- 172.21.15.86:0/2552943323 >> 172.21.15.86:0/2552943323 conn(0x561fc68a9640 msgr2=0x561fc6945750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:02:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.905+0000 7f5fae6b7b80 1 -- 172.21.15.86:0/2552943323 shutdown_connections 2024-03-23T18:02:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.905+0000 7f5fae6b7b80 1 -- 172.21.15.86:0/2552943323 wait complete. 2024-03-23T18:02:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.906+0000 7f5fae6b7b80 1 -- start start 2024-03-23T18:02:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.906+0000 7f5fae6b7b80 1 Processor -- start 2024-03-23T18:02:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.907+0000 7f5fae6b7b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561fc69492d0 0x561fc695cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:02:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.907+0000 7f5fae6b7b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fc6949bf0 0x561fc695b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:02:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.907+0000 7f5fae6b7b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561fc695d2c0 0x561fc695beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:02:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.907+0000 7f5fae6b7b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fc6837d20 con 0x561fc6949bf0 2024-03-23T18:02:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.907+0000 7f5fae6b7b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fc68379c0 con 0x561fc69492d0 2024-03-23T18:02:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.907+0000 7f5fae6b7b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fc671f640 con 0x561fc695d2c0 2024-03-23T18:02:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.907+0000 7f5fae6b7b80 10 client.0 _ll_register_callbacks cb 0x561fc6841410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T18:02:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.907+0000 7f5f93e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561fc69492d0 0x561fc695cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:02:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.907+0000 7f5f93e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561fc69492d0 0x561fc695cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:50254/0 (socket says 172.21.15.86:50254) 2024-03-23T18:02:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.907+0000 7f5f93e00700 1 -- 172.21.15.86:0/3766215289 learned_addr learned my addr 172.21.15.86:0/3766215289 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:02:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.907+0000 7f5f98c00700 1 --2- 172.21.15.86:0/3766215289 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561fc695d2c0 0x561fc695beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:02:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.907+0000 7f5f8be00700 1 --2- 172.21.15.86:0/3766215289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fc6949bf0 0x561fc695b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:02:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.907+0000 7f5f93e00700 1 -- 172.21.15.86:0/3766215289 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561fc695d2c0 msgr2=0x561fc695beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:02:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.907+0000 7f5f93e00700 1 --2- 172.21.15.86:0/3766215289 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561fc695d2c0 0x561fc695beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.907+0000 7f5f93e00700 1 -- 172.21.15.86:0/3766215289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fc6949bf0 msgr2=0x561fc695b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:02:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.907+0000 7f5f93e00700 1 --2- 172.21.15.86:0/3766215289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fc6949bf0 0x561fc695b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.907+0000 7f5f93e00700 1 -- 172.21.15.86:0/3766215289 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f9410e2b0 con 0x561fc69492d0 2024-03-23T18:02:52.912 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.908+0000 7f5f93e00700 1 --2- 172.21.15.86:0/3766215289 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561fc69492d0 0x561fc695cc90 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f5f84025040 tx=0x7f5f84005780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf6e1197c8d54333 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:02:52.912 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.908+0000 7f5f81600700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T18:02:52.912 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.908+0000 7f5f81600700 1 -- 172.21.15.86:0/3766215289 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f84003790 con 0x561fc69492d0 2024-03-23T18:02:52.912 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.908+0000 7f5f81600700 1 -- 172.21.15.86:0/3766215289 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f5f840038f0 con 0x561fc69492d0 2024-03-23T18:02:52.912 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.908+0000 7f5fae6b7b80 1 -- 172.21.15.86:0/3766215289 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561fc685c320 con 0x561fc69492d0 2024-03-23T18:02:52.912 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.908+0000 7f5f81600700 1 -- 172.21.15.86:0/3766215289 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f84003ab0 con 0x561fc69492d0 2024-03-23T18:02:52.912 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.908+0000 7f5fae6b7b80 10 client.27020 fetch_fsmap learned FSMap version 52 2024-03-23T18:02:52.912 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.908+0000 7f5fae6b7b80 1 -- 172.21.15.86:0/3766215289 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x561fc66c9b00 con 0x561fc69492d0 2024-03-23T18:02:52.912 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.908+0000 7f5f81600700 1 -- 172.21.15.86:0/3766215289 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561fc685c320 con 0x561fc69492d0 2024-03-23T18:02:52.912 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.909+0000 7f5f81600700 1 -- 172.21.15.86:0/3766215289 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 52) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f5f8400fb90 con 0x561fc69492d0 2024-03-23T18:02:52.912 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.909+0000 7f5fae6b7b80 10 client.27020 fetch_fsmap finished waiting for FSMap version 52 2024-03-23T18:02:52.912 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.909+0000 7f5fae6b7b80 10 client.27020 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T18:02:52.912 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.909+0000 7f5fae6b7b80 20 client.27020 populate_metadata read hostname 'smithi086' 2024-03-23T18:02:52.912 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.909+0000 7f5fae6b7b80 1 --2- 172.21.15.86:0/3766215289 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x561fc696cb60 0x561fc696ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:02:52.913 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.909+0000 7f5fae6b7b80 4 client.27020 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T18:02:52.913 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.909+0000 7f5fae6b7b80 1 -- 172.21.15.86:0/3766215289 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561fc68946a0 con 0x561fc696cb60 2024-03-23T18:02:52.913 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.909+0000 7f5f8be00700 1 --2- 172.21.15.86:0/3766215289 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x561fc696cb60 0x561fc696ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:02:52.913 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.909+0000 7f5f81600700 10 client.27020 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T18:02:52.913 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.909+0000 7f5f8be00700 1 --2- 172.21.15.86:0/3766215289 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x561fc696cb60 0x561fc696ef80 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad74da67ef8d192d server_cookie=a93845f2b6120a5d in_seq=0 out_seq=0 2024-03-23T18:02:52.914 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.910+0000 7f5f81600700 1 -- 172.21.15.86:0/3766215289 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x561fc68946a0 con 0x561fc696cb60 2024-03-23T18:02:52.914 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.910+0000 7f5f81600700 10 client.27020 handle_command_reply: tid=0 2024-03-23T18:02:52.914 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.910+0000 7f5fae6b7b80 1 client.27020 shutdown 2024-03-23T18:02:52.914 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.910+0000 7f5fae6b7b80 1 -- 172.21.15.86:0/3766215289 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561fc69492d0 msgr2=0x561fc695cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:02:52.914 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.910+0000 7f5fae6b7b80 1 --2- 172.21.15.86:0/3766215289 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561fc69492d0 0x561fc695cc90 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f5f84025040 tx=0x7f5f84005780 comp rx=0 tx=0).stop 2024-03-23T18:02:52.914 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.910+0000 7f5fae6b7b80 1 -- 172.21.15.86:0/3766215289 shutdown_connections 2024-03-23T18:02:52.914 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.910+0000 7f5fae6b7b80 1 --2- 172.21.15.86:0/3766215289 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x561fc696cb60 0x561fc696ef80 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:52.914 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.910+0000 7f5fae6b7b80 1 --2- 172.21.15.86:0/3766215289 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561fc695d2c0 0x561fc695beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:52.915 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.910+0000 7f5fae6b7b80 1 --2- 172.21.15.86:0/3766215289 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561fc69492d0 0x561fc695cc90 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:52.915 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.910+0000 7f5f81600700 0 client.27020 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T18:02:52.915 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.910+0000 7f5fae6b7b80 1 --2- 172.21.15.86:0/3766215289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fc6949bf0 0x561fc695b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:52.915 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.910+0000 7f5fae6b7b80 1 -- 172.21.15.86:0/3766215289 >> 172.21.15.86:0/3766215289 conn(0x561fc68a9640 msgr2=0x561fc694b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:02:52.915 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.910+0000 7f5fae6b7b80 1 -- 172.21.15.86:0/3766215289 shutdown_connections 2024-03-23T18:02:52.915 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.910+0000 7f5fae6b7b80 1 -- 172.21.15.86:0/3766215289 wait complete. 2024-03-23T18:02:52.915 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.911+0000 7f5fae6b7b80 20 client.27020 trim_cache size 0 max 16384 2024-03-23T18:02:52.930 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.928+0000 7f5fae6b7b80 1 Processor -- start 2024-03-23T18:02:52.930 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.928+0000 7f5fae6b7b80 1 -- start start 2024-03-23T18:02:52.930 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.928+0000 7f5fae6b7b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561fc69492d0 0x561fc6963f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:02:52.930 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.928+0000 7f5fae6b7b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fc6949bf0 0x561fc693f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:02:52.930 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.928+0000 7f5fae6b7b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561fc695d2c0 0x561fc693e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:02:52.930 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.928+0000 7f5fae6b7b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fc693ea80 con 0x561fc6949bf0 2024-03-23T18:02:52.930 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.928+0000 7f5fae6b7b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fc695b2e0 con 0x561fc69492d0 2024-03-23T18:02:52.930 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.928+0000 7f5fae6b7b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fc6973c60 con 0x561fc695d2c0 2024-03-23T18:02:52.930 INFO:tasks.workunit.client.0.smithi086.stdout: 1 740013 6.40 MB/sec execute 557 sec latency 31.341 ms 2024-03-23T18:02:52.934 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.928+0000 7f5f93e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561fc69492d0 0x561fc6963f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:02:52.934 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.928+0000 7f5f93e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561fc69492d0 0x561fc6963f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:50268/0 (socket says 172.21.15.86:50268) 2024-03-23T18:02:52.934 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.928+0000 7f5f93e00700 1 -- 172.21.15.86:0/4056458236 learned_addr learned my addr 172.21.15.86:0/4056458236 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:02:52.935 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.928+0000 7f5f8be00700 1 --2- 172.21.15.86:0/4056458236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fc6949bf0 0x561fc693f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:02:52.935 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.929+0000 7f5f98c00700 1 --2- 172.21.15.86:0/4056458236 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561fc695d2c0 0x561fc693e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:02:52.935 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.929+0000 7f5f93e00700 1 -- 172.21.15.86:0/4056458236 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561fc695d2c0 msgr2=0x561fc693e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:02:52.935 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.929+0000 7f5f93e00700 1 --2- 172.21.15.86:0/4056458236 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561fc695d2c0 0x561fc693e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:52.935 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.929+0000 7f5f93e00700 1 -- 172.21.15.86:0/4056458236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fc6949bf0 msgr2=0x561fc693f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:02:52.935 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.929+0000 7f5f93e00700 1 --2- 172.21.15.86:0/4056458236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fc6949bf0 0x561fc693f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:52.935 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.929+0000 7f5f93e00700 1 -- 172.21.15.86:0/4056458236 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561fc6861e40 con 0x561fc69492d0 2024-03-23T18:02:52.935 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.929+0000 7f5f98c00700 1 --2- 172.21.15.86:0/4056458236 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561fc695d2c0 0x561fc693e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T18:02:52.935 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.929+0000 7f5f8be00700 1 --2- 172.21.15.86:0/4056458236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fc6949bf0 0x561fc693f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T18:02:52.935 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.929+0000 7f5f93e00700 1 --2- 172.21.15.86:0/4056458236 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561fc69492d0 0x561fc6963f10 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f5f840098e0 tx=0x7f5f84009f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c16a4369493a94e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:02:52.935 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.929+0000 7f5f89c00700 1 -- 172.21.15.86:0/4056458236 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f8402b550 con 0x561fc69492d0 2024-03-23T18:02:52.935 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.929+0000 7f5f89c00700 1 -- 172.21.15.86:0/4056458236 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f840333a0 con 0x561fc69492d0 2024-03-23T18:02:52.935 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.929+0000 7f5f89c00700 1 -- 172.21.15.86:0/4056458236 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f840039d0 con 0x561fc69492d0 2024-03-23T18:02:52.936 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.930+0000 7f5fae6b7b80 1 -- 172.21.15.86:0/4056458236 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561fc69492d0 msgr2=0x561fc6963f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:02:52.936 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.930+0000 7f5fae6b7b80 1 --2- 172.21.15.86:0/4056458236 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561fc69492d0 0x561fc6963f10 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f5f840098e0 tx=0x7f5f84009f60 comp rx=0 tx=0).stop 2024-03-23T18:02:52.936 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.930+0000 7f5fae6b7b80 1 -- 172.21.15.86:0/4056458236 shutdown_connections 2024-03-23T18:02:52.936 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.930+0000 7f5fae6b7b80 1 --2- 172.21.15.86:0/4056458236 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561fc695d2c0 0x561fc693e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:52.936 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.930+0000 7f5fae6b7b80 1 --2- 172.21.15.86:0/4056458236 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561fc69492d0 0x561fc6963f10 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:52.936 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.930+0000 7f5fae6b7b80 1 --2- 172.21.15.86:0/4056458236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fc6949bf0 0x561fc693f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:52.936 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.930+0000 7f5fae6b7b80 1 -- 172.21.15.86:0/4056458236 >> 172.21.15.86:0/4056458236 conn(0x561fc68a9640 msgr2=0x561fc69553c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:02:52.936 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.930+0000 7f5fae6b7b80 1 -- 172.21.15.86:0/4056458236 shutdown_connections 2024-03-23T18:02:52.936 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.930+0000 7f5fae6b7b80 1 -- 172.21.15.86:0/4056458236 wait complete. 2024-03-23T18:02:52.936 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.930+0000 7f5fae6b7b80 1 -- start start 2024-03-23T18:02:52.936 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.931+0000 7f5fae6b7b80 1 Processor -- start 2024-03-23T18:02:52.936 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.931+0000 7f5fae6b7b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fc69492d0 0x561fc6961ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:02:52.936 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.931+0000 7f5fae6b7b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561fc6949bf0 0x561fc6962430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:02:52.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.931+0000 7f5fae6b7b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561fc695d2c0 0x561fc69bd640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:02:52.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.931+0000 7f5fae6b7b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fc6962bf0 con 0x561fc69492d0 2024-03-23T18:02:52.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.931+0000 7f5fae6b7b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fc6962d60 con 0x561fc695d2c0 2024-03-23T18:02:52.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.931+0000 7f5fae6b7b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fc69bdbb0 con 0x561fc6949bf0 2024-03-23T18:02:52.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.931+0000 7f5fae6b7b80 10 client.0 _ll_register_callbacks cb 0x561fc6841410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T18:02:52.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.932+0000 7f5f93e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fc69492d0 0x561fc6961ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:02:52.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.932+0000 7f5f8be00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561fc6949bf0 0x561fc6962430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:02:52.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.932+0000 7f5f98c00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561fc695d2c0 0x561fc69bd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:02:52.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.932+0000 7f5f8be00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561fc6949bf0 0x561fc6962430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:42842/0 (socket says 172.21.15.86:42842) 2024-03-23T18:02:52.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.932+0000 7f5f98c00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561fc695d2c0 0x561fc69bd640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:50282/0 (socket says 172.21.15.86:50282) 2024-03-23T18:02:52.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.932+0000 7f5f8be00700 1 -- 172.21.15.86:0/2398009460 learned_addr learned my addr 172.21.15.86:0/2398009460 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:02:52.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.932+0000 7f5f8be00700 1 -- 172.21.15.86:0/2398009460 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561fc695d2c0 msgr2=0x561fc69bd640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:02:52.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.932+0000 7f5f8be00700 1 --2- 172.21.15.86:0/2398009460 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561fc695d2c0 0x561fc69bd640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:52.938 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.932+0000 7f5f8be00700 1 -- 172.21.15.86:0/2398009460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fc69492d0 msgr2=0x561fc6961ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:02:52.938 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.932+0000 7f5f8be00700 1 --2- 172.21.15.86:0/2398009460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fc69492d0 0x561fc6961ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:52.938 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.932+0000 7f5f8be00700 1 -- 172.21.15.86:0/2398009460 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561fc66c9b00 con 0x561fc6949bf0 2024-03-23T18:02:52.938 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.932+0000 7f5f8be00700 1 --2- 172.21.15.86:0/2398009460 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561fc6949bf0 0x561fc6962430 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c015180 tx=0x7f5f7c017040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15fb668200207fc7 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:02:52.938 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.932+0000 7f5f81600700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-03-23T18:02:52.938 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.933+0000 7f5f81600700 1 -- 172.21.15.86:0/2398009460 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c009840 con 0x561fc6949bf0 2024-03-23T18:02:52.938 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.933+0000 7f5f81600700 1 -- 172.21.15.86:0/2398009460 <== mon.1 v2:172.21.15.152:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f5f7c02e480 con 0x561fc6949bf0 2024-03-23T18:02:52.938 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.933+0000 7f5fae6b7b80 1 -- 172.21.15.86:0/2398009460 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5f8401e040 con 0x561fc6949bf0 2024-03-23T18:02:52.938 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.933+0000 7f5f81600700 1 -- 172.21.15.86:0/2398009460 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f7c02e640 con 0x561fc6949bf0 2024-03-23T18:02:52.939 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.933+0000 7f5fae6b7b80 10 client.27001 fetch_fsmap learned FSMap version 52 2024-03-23T18:02:52.939 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.933+0000 7f5fae6b7b80 1 -- 172.21.15.86:0/2398009460 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x561fc6721c70 con 0x561fc6949bf0 2024-03-23T18:02:52.939 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.933+0000 7f5f81600700 1 -- 172.21.15.86:0/2398009460 <== mon.1 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f7c03b020 con 0x561fc6949bf0 2024-03-23T18:02:52.939 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.933+0000 7f5f81600700 1 -- 172.21.15.86:0/2398009460 <== mon.1 v2:172.21.15.152:3300/0 5 ==== fsmap(e 52) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f5f7c022850 con 0x561fc6949bf0 2024-03-23T18:02:52.939 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.933+0000 7f5fae6b7b80 10 client.27001 fetch_fsmap finished waiting for FSMap version 52 2024-03-23T18:02:52.939 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.933+0000 7f5fae6b7b80 10 client.27001 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T18:02:52.939 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.933+0000 7f5fae6b7b80 20 client.27001 populate_metadata read hostname 'smithi086' 2024-03-23T18:02:52.939 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.933+0000 7f5fae6b7b80 1 --2- 172.21.15.86:0/2398009460 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x561fc69bf3e0 0x561fc69c1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:02:52.939 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.933+0000 7f5fae6b7b80 4 client.27001 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T18:02:52.939 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.933+0000 7f5fae6b7b80 1 -- 172.21.15.86:0/2398009460 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561fc66eac90 con 0x561fc69bf3e0 2024-03-23T18:02:52.939 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.934+0000 7f5f93e00700 1 --2- 172.21.15.86:0/2398009460 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x561fc69bf3e0 0x561fc69c1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:02:52.939 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.934+0000 7f5f81600700 10 client.27001 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T18:02:52.940 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.934+0000 7f5f93e00700 1 --2- 172.21.15.86:0/2398009460 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x561fc69bf3e0 0x561fc69c1800 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31a4b5592269acf2 server_cookie=32c773613db2f612 in_seq=0 out_seq=0 2024-03-23T18:02:52.979 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.979+0000 7f5f81600700 1 -- 172.21.15.86:0/2398009460 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561fc66eac90 con 0x561fc69bf3e0 2024-03-23T18:02:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.979+0000 7f5f81600700 10 client.27001 handle_command_reply: tid=0 2024-03-23T18:02:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.979+0000 7f5fae6b7b80 1 client.27001 shutdown 2024-03-23T18:02:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.979+0000 7f5fae6b7b80 1 -- 172.21.15.86:0/2398009460 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561fc6949bf0 msgr2=0x561fc6962430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:02:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.979+0000 7f5fae6b7b80 1 --2- 172.21.15.86:0/2398009460 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561fc6949bf0 0x561fc6962430 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c015180 tx=0x7f5f7c017040 comp rx=0 tx=0).stop 2024-03-23T18:02:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.979+0000 7f5fae6b7b80 1 -- 172.21.15.86:0/2398009460 shutdown_connections 2024-03-23T18:02:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.979+0000 7f5fae6b7b80 1 --2- 172.21.15.86:0/2398009460 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x561fc69bf3e0 0x561fc69c1800 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.979+0000 7f5fae6b7b80 1 --2- 172.21.15.86:0/2398009460 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561fc695d2c0 0x561fc69bd640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.979+0000 7f5fae6b7b80 1 --2- 172.21.15.86:0/2398009460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fc69492d0 0x561fc6961ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.979+0000 7f5f81600700 0 client.27001 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T18:02:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.979+0000 7f5fae6b7b80 1 --2- 172.21.15.86:0/2398009460 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561fc6949bf0 0x561fc6962430 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.979+0000 7f5fae6b7b80 1 -- 172.21.15.86:0/2398009460 >> 172.21.15.86:0/2398009460 conn(0x561fc68a9640 msgr2=0x561fc69588d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:02:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.979+0000 7f5fae6b7b80 1 -- 172.21.15.86:0/2398009460 shutdown_connections 2024-03-23T18:02:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.979+0000 7f5fae6b7b80 1 -- 172.21.15.86:0/2398009460 wait complete. 2024-03-23T18:02:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.979+0000 7f5fae6b7b80 20 client.27001 trim_cache size 0 max 16384 2024-03-23T18:02:52.981 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-03-23T18:02:52.981 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-03-23T18:02:52.981 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-03-23T18:02:52.981 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-03-23T18:02:52.982 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.980+0000 7f5f99600700 1 -- 172.21.15.86:0/1442347013 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f5f780a1290 msgr2=0x7f5f780a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:02:52.982 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.980+0000 7f5f99600700 1 --2- 172.21.15.86:0/1442347013 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f5f780a1290 0x7f5f780a3750 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c005fd0 tx=0x7f5f7c0158c0 comp rx=0 tx=0).stop 2024-03-23T18:02:52.982 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.980+0000 7f5f99600700 1 -- 172.21.15.86:0/1442347013 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f94106b80 msgr2=0x7f5f94073b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:02:52.982 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.980+0000 7f5f99600700 1 --2- 172.21.15.86:0/1442347013 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f94106b80 0x7f5f94073b30 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c007ee0 tx=0x7f5f8c00d3b0 comp rx=0 tx=0).stop 2024-03-23T18:02:52.982 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.980+0000 7f5f99600700 1 -- 172.21.15.86:0/1442347013 shutdown_connections 2024-03-23T18:02:52.982 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.980+0000 7f5f99600700 1 --2- 172.21.15.86:0/1442347013 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f94106b80 0x7f5f94073b30 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:52.982 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.980+0000 7f5f99600700 1 --2- 172.21.15.86:0/1442347013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f94100ae0 0x7f5f940730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:52.982 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.980+0000 7f5f99600700 1 --2- 172.21.15.86:0/1442347013 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f5f780a1290 0x7f5f780a3750 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:52.982 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.980+0000 7f5f99600700 1 --2- 172.21.15.86:0/1442347013 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f941014a0 0x7f5f940735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:02:52.982 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.980+0000 7f5f99600700 1 -- 172.21.15.86:0/1442347013 >> 172.21.15.86:0/1442347013 conn(0x7f5f940fc610 msgr2=0x7f5f940fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:02:52.982 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.980+0000 7f5f99600700 1 -- 172.21.15.86:0/1442347013 shutdown_connections 2024-03-23T18:02:52.983 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:02:52.980+0000 7f5f99600700 1 -- 172.21.15.86:0/1442347013 wait complete. 2024-03-23T18:02:53.928 INFO:tasks.workunit.client.0.smithi086.stdout: 1 740278 6.39 MB/sec execute 558 sec latency 781.943 ms 2024-03-23T18:02:54.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:02:54 smithi086 ceph-mon[38221]: pgmap v383: 97 pgs: 97 active+clean; 289 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 851 KiB/s rd, 2.1 MiB/s wr, 170 op/s 2024-03-23T18:02:54.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:02:54 smithi092 ceph-mon[79089]: pgmap v383: 97 pgs: 97 active+clean; 289 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 851 KiB/s rd, 2.1 MiB/s wr, 170 op/s 2024-03-23T18:02:54.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:02:54 smithi152 ceph-mon[47949]: pgmap v383: 97 pgs: 97 active+clean; 289 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 851 KiB/s rd, 2.1 MiB/s wr, 170 op/s 2024-03-23T18:02:54.928 INFO:tasks.workunit.client.0.smithi086.stdout: 1 740993 6.38 MB/sec execute 559 sec latency 1479.287 ms 2024-03-23T18:02:55.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:02:55 smithi152 ceph-mon[47949]: pgmap v384: 97 pgs: 97 active+clean; 288 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 248 op/s 2024-03-23T18:02:55.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:02:55 smithi086 ceph-mon[38221]: pgmap v384: 97 pgs: 97 active+clean; 288 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 248 op/s 2024-03-23T18:02:55.556 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:02:55 smithi092 ceph-mon[79089]: pgmap v384: 97 pgs: 97 active+clean; 288 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 248 op/s 2024-03-23T18:02:55.928 INFO:tasks.workunit.client.0.smithi086.stdout: 1 742523 6.39 MB/sec execute 560 sec latency 13.932 ms 2024-03-23T18:02:56.929 INFO:tasks.workunit.client.0.smithi086.stdout: 1 743820 6.39 MB/sec execute 561 sec latency 84.948 ms 2024-03-23T18:02:57.929 INFO:tasks.workunit.client.0.smithi086.stdout: 1 744270 6.38 MB/sec execute 562 sec latency 928.044 ms 2024-03-23T18:02:58.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:02:57 smithi086 ceph-mon[38221]: pgmap v385: 97 pgs: 97 active+clean; 288 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 820 KiB/s rd, 1.8 MiB/s wr, 166 op/s 2024-03-23T18:02:58.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:02:57 smithi092 ceph-mon[79089]: pgmap v385: 97 pgs: 97 active+clean; 288 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 820 KiB/s rd, 1.8 MiB/s wr, 166 op/s 2024-03-23T18:02:58.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:02:57 smithi152 ceph-mon[47949]: pgmap v385: 97 pgs: 97 active+clean; 288 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 820 KiB/s rd, 1.8 MiB/s wr, 166 op/s 2024-03-23T18:02:58.929 INFO:tasks.workunit.client.0.smithi086.stdout: 1 745937 6.39 MB/sec execute 563 sec latency 15.640 ms 2024-03-23T18:02:59.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:02:58 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T18:02:59.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:02:58 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T18:02:59.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:02:58 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T18:02:59.929 INFO:tasks.workunit.client.0.smithi086.stdout: 1 747206 6.39 MB/sec execute 564 sec latency 16.012 ms 2024-03-23T18:03:00.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:02:59 smithi086 ceph-mon[38221]: pgmap v386: 97 pgs: 97 active+clean; 288 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 820 KiB/s rd, 1.8 MiB/s wr, 166 op/s 2024-03-23T18:03:00.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:02:59 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T18:03:00.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:02:59 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T18:03:00.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:02:59 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T18:03:00.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:02:59 smithi092 ceph-mon[79089]: pgmap v386: 97 pgs: 97 active+clean; 288 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 820 KiB/s rd, 1.8 MiB/s wr, 166 op/s 2024-03-23T18:03:00.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:02:59 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T18:03:00.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:02:59 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T18:03:00.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:02:59 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T18:03:00.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:02:59 smithi152 ceph-mon[47949]: pgmap v386: 97 pgs: 97 active+clean; 288 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 820 KiB/s rd, 1.8 MiB/s wr, 166 op/s 2024-03-23T18:03:00.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:02:59 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T18:03:00.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:02:59 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T18:03:00.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:02:59 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T18:03:00.929 INFO:tasks.workunit.client.0.smithi086.stdout: 1 748247 6.38 MB/sec execute 565 sec latency 99.831 ms 2024-03-23T18:03:01.929 INFO:tasks.workunit.client.0.smithi086.stdout: 1 749686 6.39 MB/sec execute 566 sec latency 15.067 ms 2024-03-23T18:03:02.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:03:01 smithi086 ceph-mon[38221]: pgmap v387: 97 pgs: 97 active+clean; 294 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 244 op/s 2024-03-23T18:03:02.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:03:01 smithi092 ceph-mon[79089]: pgmap v387: 97 pgs: 97 active+clean; 294 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 244 op/s 2024-03-23T18:03:02.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:03:01 smithi152 ceph-mon[47949]: pgmap v387: 97 pgs: 97 active+clean; 294 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 244 op/s 2024-03-23T18:03:02.929 INFO:tasks.workunit.client.0.smithi086.stdout: 1 750911 6.39 MB/sec execute 567 sec latency 223.442 ms 2024-03-23T18:03:03.929 INFO:tasks.workunit.client.0.smithi086.stdout: 1 750911 6.37 MB/sec execute 568 sec latency 1223.533 ms 2024-03-23T18:03:04.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:03:03 smithi086 ceph-mon[38221]: pgmap v388: 97 pgs: 97 active+clean; 294 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 1.5 MiB/s wr, 155 op/s 2024-03-23T18:03:04.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:03:03 smithi092 ceph-mon[79089]: pgmap v388: 97 pgs: 97 active+clean; 294 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 1.5 MiB/s wr, 155 op/s 2024-03-23T18:03:04.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:03:03 smithi152 ceph-mon[47949]: pgmap v388: 97 pgs: 97 active+clean; 294 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 1.5 MiB/s wr, 155 op/s 2024-03-23T18:03:04.929 INFO:tasks.workunit.client.0.smithi086.stdout: 1 751614 6.37 MB/sec execute 569 sec latency 1923.614 ms 2024-03-23T18:03:05.929 INFO:tasks.workunit.client.0.smithi086.stdout: 1 753042 6.38 MB/sec execute 570 sec latency 32.672 ms 2024-03-23T18:03:06.136 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.135+0000 7f499de00700 1 -- 172.21.15.86:0/290031853 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4998101ee0 msgr2=0x7f49981022c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:03:06.136 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.135+0000 7f499de00700 1 --2- 172.21.15.86:0/290031853 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4998101ee0 0x7f49981022c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f4988009990 tx=0x7f4988009ca0 comp rx=0 tx=0).stop 2024-03-23T18:03:06.136 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.135+0000 7f499de00700 1 -- 172.21.15.86:0/290031853 shutdown_connections 2024-03-23T18:03:06.136 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.135+0000 7f499de00700 1 --2- 172.21.15.86:0/290031853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f499810b2b0 0x7f499810d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:06.136 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.135+0000 7f499de00700 1 --2- 172.21.15.86:0/290031853 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4998101ee0 0x7f49981022c0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:06.136 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.135+0000 7f499de00700 1 --2- 172.21.15.86:0/290031853 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4998102800 0x7f499810ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:06.136 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.135+0000 7f499de00700 1 -- 172.21.15.86:0/290031853 >> 172.21.15.86:0/290031853 conn(0x7f49980fb680 msgr2=0x7f49980fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:03:06.136 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.135+0000 7f499de00700 1 -- 172.21.15.86:0/290031853 shutdown_connections 2024-03-23T18:03:06.136 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.135+0000 7f499de00700 1 -- 172.21.15.86:0/290031853 wait complete. 2024-03-23T18:03:06.137 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.137+0000 7f499de00700 1 Processor -- start 2024-03-23T18:03:06.137 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.137+0000 7f499de00700 1 -- start start 2024-03-23T18:03:06.139 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.138+0000 7f499de00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4998101ee0 0x7f4998199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:06.139 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.138+0000 7f499de00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4998102800 0x7f49981998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:06.139 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.138+0000 7f499de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f499810b2b0 0x7f499819dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:06.139 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.138+0000 7f499de00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f499819e290 con 0x7f499810b2b0 2024-03-23T18:03:06.139 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.138+0000 7f499de00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f499819e400 con 0x7f4998101ee0 2024-03-23T18:03:06.139 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.138+0000 7f499de00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f499819e5c0 con 0x7f4998102800 2024-03-23T18:03:06.139 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.138+0000 7f4996a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4998101ee0 0x7f4998199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:06.139 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.138+0000 7f4996a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4998101ee0 0x7f4998199370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:53646/0 (socket says 172.21.15.86:53646) 2024-03-23T18:03:06.139 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.138+0000 7f4996a00700 1 -- 172.21.15.86:0/258282211 learned_addr learned my addr 172.21.15.86:0/258282211 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:03:06.140 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.138+0000 7f4996000700 1 --2- 172.21.15.86:0/258282211 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4998102800 0x7f49981998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:06.140 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.138+0000 7f4997400700 1 --2- 172.21.15.86:0/258282211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f499810b2b0 0x7f499819dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:06.140 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.138+0000 7f4996a00700 1 -- 172.21.15.86:0/258282211 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4998102800 msgr2=0x7f49981998b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:03:06.140 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.138+0000 7f4996a00700 1 --2- 172.21.15.86:0/258282211 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4998102800 0x7f49981998b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:06.140 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.138+0000 7f4996a00700 1 -- 172.21.15.86:0/258282211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f499810b2b0 msgr2=0x7f499819dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:03:06.140 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.138+0000 7f4996a00700 1 --2- 172.21.15.86:0/258282211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f499810b2b0 0x7f499819dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:06.140 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.138+0000 7f4996a00700 1 -- 172.21.15.86:0/258282211 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49880095f0 con 0x7f4998101ee0 2024-03-23T18:03:06.140 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.138+0000 7f4996000700 1 --2- 172.21.15.86:0/258282211 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4998102800 0x7f49981998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T18:03:06.140 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.138+0000 7f4997400700 1 --2- 172.21.15.86:0/258282211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f499810b2b0 0x7f499819dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T18:03:06.140 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.139+0000 7f4996a00700 1 --2- 172.21.15.86:0/258282211 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4998101ee0 0x7f4998199370 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4988000c00 tx=0x7f4988005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:03:06.140 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.139+0000 7f499ca00700 1 -- 172.21.15.86:0/258282211 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f498801d070 con 0x7f4998101ee0 2024-03-23T18:03:06.140 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.139+0000 7f499ca00700 1 -- 172.21.15.86:0/258282211 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4988022470 con 0x7f4998101ee0 2024-03-23T18:03:06.140 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.139+0000 7f499de00700 1 -- 172.21.15.86:0/258282211 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f499819e840 con 0x7f4998101ee0 2024-03-23T18:03:06.140 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.139+0000 7f499ca00700 1 -- 172.21.15.86:0/258282211 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4988017650 con 0x7f4998101ee0 2024-03-23T18:03:06.141 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.139+0000 7f499de00700 1 -- 172.21.15.86:0/258282211 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49981a60e0 con 0x7f4998101ee0 2024-03-23T18:03:06.142 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.142+0000 7f499ca00700 1 -- 172.21.15.86:0/258282211 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f49880225e0 con 0x7f4998101ee0 2024-03-23T18:03:06.143 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.143+0000 7f49b18b4b80 1 Processor -- start 2024-03-23T18:03:06.143 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.144+0000 7f49b18b4b80 1 -- start start 2024-03-23T18:03:06.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.144+0000 7f499ca00700 1 --2- 172.21.15.86:0/258282211 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f497c0a1290 0x7f497c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:06.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.144+0000 7f49b18b4b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c73871a100 0x55c73871a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:06.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.144+0000 7f49b18b4b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c73871aa20 0x55c738725390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:06.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.144+0000 7f49b18b4b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c7387258d0 0x55c738727cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:06.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.144+0000 7f499ca00700 1 -- 172.21.15.86:0/258282211 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f49880cdf80 con 0x7f4998101ee0 2024-03-23T18:03:06.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.144+0000 7f49b18b4b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c738673930 con 0x55c7387258d0 2024-03-23T18:03:06.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.144+0000 7f49b18b4b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7386712f0 con 0x55c73871aa20 2024-03-23T18:03:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.144+0000 7f49b18b4b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c73866cbe0 con 0x55c73871a100 2024-03-23T18:03:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.144+0000 7f4996000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c73871aa20 0x55c738725390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.144+0000 7f4997400700 1 --2- 172.21.15.86:0/258282211 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f497c0a1290 0x7f497c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.144+0000 7f4996000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c73871aa20 0x55c738725390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:53648/0 (socket says 172.21.15.86:53648) 2024-03-23T18:03:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.144+0000 7f4996000700 1 -- 172.21.15.86:0/1403246322 learned_addr learned my addr 172.21.15.86:0/1403246322 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:03:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.144+0000 7f4996a00700 1 --2- 172.21.15.86:0/1403246322 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c73871a100 0x55c73871a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.145+0000 7f4997400700 1 --2- 172.21.15.86:0/1403246322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c7387258d0 0x55c738727cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.145+0000 7f4996000700 1 -- 172.21.15.86:0/1403246322 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c73871a100 msgr2=0x55c73871a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:03:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.145+0000 7f4996000700 1 --2- 172.21.15.86:0/1403246322 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c73871a100 0x55c73871a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.145+0000 7f4996000700 1 -- 172.21.15.86:0/1403246322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c7387258d0 msgr2=0x55c738727cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:03:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.145+0000 7f4996000700 1 --2- 172.21.15.86:0/1403246322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c7387258d0 0x55c738727cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.145+0000 7f4996000700 1 -- 172.21.15.86:0/1403246322 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c7384ee350 con 0x55c73871aa20 2024-03-23T18:03:06.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.145+0000 7f4996000700 1 --2- 172.21.15.86:0/1403246322 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c73871aa20 0x55c738725390 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f49840115f0 tx=0x7f4984011900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=362eca179af12c2b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:03:06.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.145+0000 7f4997400700 1 --2- 172.21.15.86:0/258282211 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f497c0a1290 0x7f497c0a3750 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x55c73871bb90 tx=0x7f4990009920 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:03:06.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.145+0000 7f4983e00700 1 -- 172.21.15.86:0/1403246322 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4984004230 con 0x55c73871aa20 2024-03-23T18:03:06.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.145+0000 7f4983e00700 1 -- 172.21.15.86:0/1403246322 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4984004390 con 0x55c73871aa20 2024-03-23T18:03:06.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.145+0000 7f4983e00700 1 -- 172.21.15.86:0/1403246322 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4984019830 con 0x55c73871aa20 2024-03-23T18:03:06.150 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.146+0000 7f49b18b4b80 1 -- 172.21.15.86:0/1403246322 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c73871aa20 msgr2=0x55c738725390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:03:06.150 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.146+0000 7f49b18b4b80 1 --2- 172.21.15.86:0/1403246322 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c73871aa20 0x55c738725390 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f49840115f0 tx=0x7f4984011900 comp rx=0 tx=0).stop 2024-03-23T18:03:06.150 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.146+0000 7f49b18b4b80 1 -- 172.21.15.86:0/1403246322 shutdown_connections 2024-03-23T18:03:06.150 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.146+0000 7f49b18b4b80 1 --2- 172.21.15.86:0/1403246322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c7387258d0 0x55c738727cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:06.150 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.146+0000 7f49b18b4b80 1 --2- 172.21.15.86:0/1403246322 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c73871a100 0x55c73871a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:06.150 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.146+0000 7f49b18b4b80 1 --2- 172.21.15.86:0/1403246322 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c73871aa20 0x55c738725390 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:06.150 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.146+0000 7f49b18b4b80 1 -- 172.21.15.86:0/1403246322 >> 172.21.15.86:0/1403246322 conn(0x55c73867a640 msgr2=0x55c738716520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:03:06.151 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.146+0000 7f49b18b4b80 1 -- 172.21.15.86:0/1403246322 shutdown_connections 2024-03-23T18:03:06.151 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.146+0000 7f49b18b4b80 1 -- 172.21.15.86:0/1403246322 wait complete. 2024-03-23T18:03:06.151 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.147+0000 7f49b18b4b80 1 -- start start 2024-03-23T18:03:06.151 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.147+0000 7f49b18b4b80 1 Processor -- start 2024-03-23T18:03:06.151 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.148+0000 7f49b18b4b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c73871a100 0x55c738725220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:06.151 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.148+0000 7f49b18b4b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c73871aa20 0x55c738722510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:06.151 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.148+0000 7f49b18b4b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c7387258d0 0x55c738722a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:06.151 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.148+0000 7f49b18b4b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c738608d20 con 0x55c73871a100 2024-03-23T18:03:06.151 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.148+0000 7f49b18b4b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7386089c0 con 0x55c7387258d0 2024-03-23T18:03:06.151 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.148+0000 7f49b18b4b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7384f0640 con 0x55c73871aa20 2024-03-23T18:03:06.151 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.148+0000 7f49b18b4b80 10 client.0 _ll_register_callbacks cb 0x55c738612410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T18:03:06.151 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.148+0000 7f4996a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c73871a100 0x55c738725220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:06.151 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.148+0000 7f4997400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c7387258d0 0x55c738722a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.148+0000 7f4996000700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c73871aa20 0x55c738722510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.148+0000 7f4996a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c73871a100 0x55c738725220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40898/0 (socket says 172.21.15.86:40898) 2024-03-23T18:03:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.148+0000 7f4996000700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c73871aa20 0x55c738722510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:39204/0 (socket says 172.21.15.86:39204) 2024-03-23T18:03:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.148+0000 7f4996000700 1 -- 172.21.15.86:0/3645894931 learned_addr learned my addr 172.21.15.86:0/3645894931 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:03:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.148+0000 7f4997400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c7387258d0 0x55c738722a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:53658/0 (socket says 172.21.15.86:53658) 2024-03-23T18:03:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.149+0000 7f4996000700 1 -- 172.21.15.86:0/3645894931 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c7387258d0 msgr2=0x55c738722a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:03:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.149+0000 7f4996000700 1 --2- 172.21.15.86:0/3645894931 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c7387258d0 0x55c738722a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.149+0000 7f4996000700 1 -- 172.21.15.86:0/3645894931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c73871a100 msgr2=0x55c738725220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:03:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.149+0000 7f4996000700 1 --2- 172.21.15.86:0/3645894931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c73871a100 0x55c738725220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.149+0000 7f4996000700 1 -- 172.21.15.86:0/3645894931 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4984009d50 con 0x55c73871aa20 2024-03-23T18:03:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.149+0000 7f4997400700 1 --2- 172.21.15.86:0/3645894931 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c7387258d0 0x55c738722a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T18:03:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.149+0000 7f4996000700 1 --2- 172.21.15.86:0/3645894931 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c73871aa20 0x55c738722510 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f49840115c0 tx=0x7f49840052f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2316e3265adac246 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:03:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.149+0000 7f4980c00700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-03-23T18:03:06.153 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.149+0000 7f4980c00700 1 -- 172.21.15.86:0/3645894931 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4984004950 con 0x55c73871aa20 2024-03-23T18:03:06.153 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.149+0000 7f49b18b4b80 1 -- 172.21.15.86:0/3645894931 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c73862d320 con 0x55c73871aa20 2024-03-23T18:03:06.153 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.149+0000 7f4980c00700 1 -- 172.21.15.86:0/3645894931 <== mon.1 v2:172.21.15.152:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f4984004ab0 con 0x55c73871aa20 2024-03-23T18:03:06.153 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.149+0000 7f4980c00700 1 -- 172.21.15.86:0/3645894931 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49840056f0 con 0x55c73871aa20 2024-03-23T18:03:06.153 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.149+0000 7f49b18b4b80 10 client.27013 fetch_fsmap learned FSMap version 52 2024-03-23T18:03:06.153 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.149+0000 7f49b18b4b80 1 -- 172.21.15.86:0/3645894931 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x55c73849ab00 con 0x55c73871aa20 2024-03-23T18:03:06.153 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.150+0000 7f4980c00700 1 -- 172.21.15.86:0/3645894931 <== mon.1 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c73862d320 con 0x55c73871aa20 2024-03-23T18:03:06.153 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.150+0000 7f4980c00700 1 -- 172.21.15.86:0/3645894931 <== mon.1 v2:172.21.15.152:3300/0 5 ==== fsmap(e 52) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f4984021040 con 0x55c73871aa20 2024-03-23T18:03:06.153 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.150+0000 7f49b18b4b80 10 client.27013 fetch_fsmap finished waiting for FSMap version 52 2024-03-23T18:03:06.153 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.150+0000 7f49b18b4b80 10 client.27013 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T18:03:06.153 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.150+0000 7f49b18b4b80 20 client.27013 populate_metadata read hostname 'smithi086' 2024-03-23T18:03:06.153 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.150+0000 7f49b18b4b80 1 --2- 172.21.15.86:0/3645894931 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55c7387354b0 0x55c7387378d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:06.154 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.150+0000 7f49b18b4b80 4 client.27013 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T18:03:06.154 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.150+0000 7f49b18b4b80 1 -- 172.21.15.86:0/3645894931 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c7386656a0 con 0x55c7387354b0 2024-03-23T18:03:06.154 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.150+0000 7f4996a00700 1 --2- 172.21.15.86:0/3645894931 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55c7387354b0 0x55c7387378d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.151+0000 7f4996a00700 1 --2- 172.21.15.86:0/3645894931 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55c7387354b0 0x55c7387378d0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=783f034980e75355 server_cookie=9e5ef0bcdca4bba0 in_seq=0 out_seq=0 2024-03-23T18:03:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.151+0000 7f4980c00700 10 client.27013 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T18:03:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.151+0000 7f4980c00700 1 -- 172.21.15.86:0/3645894931 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55c7386656a0 con 0x55c7387354b0 2024-03-23T18:03:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.151+0000 7f4980c00700 10 client.27013 handle_command_reply: tid=0 2024-03-23T18:03:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.151+0000 7f49b18b4b80 1 client.27013 shutdown 2024-03-23T18:03:06.156 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.151+0000 7f49b18b4b80 1 -- 172.21.15.86:0/3645894931 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c73871aa20 msgr2=0x55c738722510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:03:06.156 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.152+0000 7f49b18b4b80 1 --2- 172.21.15.86:0/3645894931 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c73871aa20 0x55c738722510 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f49840115c0 tx=0x7f49840052f0 comp rx=0 tx=0).stop 2024-03-23T18:03:06.156 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.152+0000 7f49b18b4b80 1 -- 172.21.15.86:0/3645894931 shutdown_connections 2024-03-23T18:03:06.156 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.152+0000 7f49b18b4b80 1 --2- 172.21.15.86:0/3645894931 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55c7387354b0 0x55c7387378d0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:06.156 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.152+0000 7f49b18b4b80 1 --2- 172.21.15.86:0/3645894931 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c7387258d0 0x55c738722a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:06.156 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.152+0000 7f49b18b4b80 1 --2- 172.21.15.86:0/3645894931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c73871a100 0x55c738725220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:06.156 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.152+0000 7f49b18b4b80 1 --2- 172.21.15.86:0/3645894931 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c73871aa20 0x55c738722510 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:06.156 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.152+0000 7f4980c00700 0 client.27013 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T18:03:06.156 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.152+0000 7f49b18b4b80 1 -- 172.21.15.86:0/3645894931 >> 172.21.15.86:0/3645894931 conn(0x55c73867a640 msgr2=0x55c73871c080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:03:06.156 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.152+0000 7f49b18b4b80 1 -- 172.21.15.86:0/3645894931 shutdown_connections 2024-03-23T18:03:06.156 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.152+0000 7f49b18b4b80 1 -- 172.21.15.86:0/3645894931 wait complete. 2024-03-23T18:03:06.156 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.152+0000 7f49b18b4b80 20 client.27013 trim_cache size 0 max 16384 2024-03-23T18:03:06.173 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.167+0000 7f49b18b4b80 1 Processor -- start 2024-03-23T18:03:06.173 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.167+0000 7f49b18b4b80 1 -- start start 2024-03-23T18:03:06.173 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.167+0000 7f49b18b4b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c73871a100 0x55c73873c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:06.173 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.167+0000 7f49b18b4b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c73871aa20 0x55c73872c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:06.173 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.167+0000 7f49b18b4b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c7387258d0 0x55c738715d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:06.174 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.167+0000 7f49b18b4b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c738712c80 con 0x55c73871aa20 2024-03-23T18:03:06.174 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.167+0000 7f49b18b4b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c73871ae70 con 0x55c7387258d0 2024-03-23T18:03:06.174 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.167+0000 7f49b18b4b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c738723b10 con 0x55c73871a100 2024-03-23T18:03:06.174 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.168+0000 7f4996a00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c73871a100 0x55c73873c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:06.174 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.168+0000 7f4996a00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c73871a100 0x55c73873c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:39214/0 (socket says 172.21.15.86:39214) 2024-03-23T18:03:06.174 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.168+0000 7f4996a00700 1 -- 172.21.15.86:0/1489094388 learned_addr learned my addr 172.21.15.86:0/1489094388 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:03:06.174 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.168+0000 7f4997400700 1 --2- 172.21.15.86:0/1489094388 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c7387258d0 0x55c738715d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:06.174 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.168+0000 7f4996000700 1 --2- 172.21.15.86:0/1489094388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c73871aa20 0x55c73872c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:06.174 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.168+0000 7f4996a00700 1 -- 172.21.15.86:0/1489094388 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c7387258d0 msgr2=0x55c738715d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:03:06.174 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.168+0000 7f4996a00700 1 --2- 172.21.15.86:0/1489094388 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c7387258d0 0x55c738715d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:06.174 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.168+0000 7f4996a00700 1 -- 172.21.15.86:0/1489094388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c73871aa20 msgr2=0x55c73872c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:03:06.174 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.168+0000 7f4996a00700 1 --2- 172.21.15.86:0/1489094388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c73871aa20 0x55c73872c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:06.175 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.168+0000 7f4996a00700 1 -- 172.21.15.86:0/1489094388 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c738632e40 con 0x55c73871a100 2024-03-23T18:03:06.175 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.168+0000 7f4996a00700 1 --2- 172.21.15.86:0/1489094388 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c73871a100 0x55c73873c230 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f498800f740 tx=0x7f4988022c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d825848d2db0279b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:03:06.175 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.169+0000 7f498d200700 1 -- 172.21.15.86:0/1489094388 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f498807d750 con 0x55c73871a100 2024-03-23T18:03:06.175 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.169+0000 7f498d200700 1 -- 172.21.15.86:0/1489094388 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f498807f2c0 con 0x55c73871a100 2024-03-23T18:03:06.175 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.169+0000 7f498d200700 1 -- 172.21.15.86:0/1489094388 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f498809f030 con 0x55c73871a100 2024-03-23T18:03:06.175 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.169+0000 7f49b18b4b80 1 -- 172.21.15.86:0/1489094388 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c73871a100 msgr2=0x55c73873c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:03:06.175 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.169+0000 7f49b18b4b80 1 --2- 172.21.15.86:0/1489094388 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c73871a100 0x55c73873c230 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f498800f740 tx=0x7f4988022c90 comp rx=0 tx=0).stop 2024-03-23T18:03:06.175 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.169+0000 7f49b18b4b80 1 -- 172.21.15.86:0/1489094388 shutdown_connections 2024-03-23T18:03:06.175 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.169+0000 7f49b18b4b80 1 --2- 172.21.15.86:0/1489094388 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c7387258d0 0x55c738715d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:06.175 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.169+0000 7f49b18b4b80 1 --2- 172.21.15.86:0/1489094388 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c73871a100 0x55c73873c230 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:06.175 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.169+0000 7f49b18b4b80 1 --2- 172.21.15.86:0/1489094388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c73871aa20 0x55c73872c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:06.175 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.169+0000 7f49b18b4b80 1 -- 172.21.15.86:0/1489094388 >> 172.21.15.86:0/1489094388 conn(0x55c73867a640 msgr2=0x55c738720de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:03:06.175 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.169+0000 7f49b18b4b80 1 -- 172.21.15.86:0/1489094388 shutdown_connections 2024-03-23T18:03:06.176 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.170+0000 7f49b18b4b80 1 -- 172.21.15.86:0/1489094388 wait complete. 2024-03-23T18:03:06.176 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.170+0000 7f49b18b4b80 1 -- start start 2024-03-23T18:03:06.176 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.170+0000 7f49b18b4b80 1 Processor -- start 2024-03-23T18:03:06.176 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.171+0000 7f49b18b4b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c73871a100 0x55c73872c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:06.176 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.171+0000 7f49b18b4b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c73871aa20 0x55c73872a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:06.176 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.171+0000 7f49b18b4b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c7387258d0 0x55c73872ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:06.176 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.171+0000 7f49b18b4b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c73872b3f0 con 0x55c73871a100 2024-03-23T18:03:06.176 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.171+0000 7f49b18b4b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c738785fd0 con 0x55c73871aa20 2024-03-23T18:03:06.176 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.171+0000 7f49b18b4b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c738786110 con 0x55c7387258d0 2024-03-23T18:03:06.176 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.171+0000 7f49b18b4b80 10 client.0 _ll_register_callbacks cb 0x55c738612410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T18:03:06.176 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.171+0000 7f4996a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c73871a100 0x55c73872c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:06.176 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.171+0000 7f4997400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c7387258d0 0x55c73872ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:06.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.171+0000 7f4996000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c73871aa20 0x55c73872a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:06.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.171+0000 7f4997400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c7387258d0 0x55c73872ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:39220/0 (socket says 172.21.15.86:39220) 2024-03-23T18:03:06.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.171+0000 7f4997400700 1 -- 172.21.15.86:0/526308106 learned_addr learned my addr 172.21.15.86:0/526308106 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:03:06.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.171+0000 7f4996000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c73871aa20 0x55c73872a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:53690/0 (socket says 172.21.15.86:53690) 2024-03-23T18:03:06.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.172+0000 7f4996000700 1 -- 172.21.15.86:0/526308106 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c7387258d0 msgr2=0x55c73872ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:03:06.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.172+0000 7f4996000700 1 --2- 172.21.15.86:0/526308106 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c7387258d0 0x55c73872ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:06.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.172+0000 7f4996000700 1 -- 172.21.15.86:0/526308106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c73871a100 msgr2=0x55c73872c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:03:06.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.172+0000 7f4996000700 1 --2- 172.21.15.86:0/526308106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c73871a100 0x55c73872c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:06.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.172+0000 7f4996000700 1 -- 172.21.15.86:0/526308106 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49981a60e0 con 0x55c73871aa20 2024-03-23T18:03:06.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.172+0000 7f4997400700 1 --2- 172.21.15.86:0/526308106 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c7387258d0 0x55c73872ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T18:03:06.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.172+0000 7f4996000700 1 --2- 172.21.15.86:0/526308106 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c73871aa20 0x55c73872a5f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f4984023620 tx=0x7f4984009280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=328ca73a163bbbea server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:03:06.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.172+0000 7f4980c00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T18:03:06.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.172+0000 7f4980c00700 1 -- 172.21.15.86:0/526308106 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4984005360 con 0x55c73871aa20 2024-03-23T18:03:06.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.172+0000 7f4980c00700 1 -- 172.21.15.86:0/526308106 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f4984000e40 con 0x55c73871aa20 2024-03-23T18:03:06.178 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.172+0000 7f49b18b4b80 1 -- 172.21.15.86:0/526308106 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4984004390 con 0x55c73871aa20 2024-03-23T18:03:06.178 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.173+0000 7f4980c00700 1 -- 172.21.15.86:0/526308106 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49840056f0 con 0x55c73871aa20 2024-03-23T18:03:06.178 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.173+0000 7f49b18b4b80 10 client.27044 fetch_fsmap learned FSMap version 52 2024-03-23T18:03:06.178 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.173+0000 7f49b18b4b80 1 -- 172.21.15.86:0/526308106 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x55c7384f2c70 con 0x55c73871aa20 2024-03-23T18:03:06.178 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.173+0000 7f4980c00700 1 -- 172.21.15.86:0/526308106 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4984044020 con 0x55c73871aa20 2024-03-23T18:03:06.178 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.173+0000 7f4980c00700 1 -- 172.21.15.86:0/526308106 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 52) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f4984011970 con 0x55c73871aa20 2024-03-23T18:03:06.178 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.173+0000 7f49b18b4b80 10 client.27044 fetch_fsmap finished waiting for FSMap version 52 2024-03-23T18:03:06.178 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.173+0000 7f49b18b4b80 10 client.27044 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T18:03:06.178 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.173+0000 7f49b18b4b80 20 client.27044 populate_metadata read hostname 'smithi086' 2024-03-23T18:03:06.178 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.173+0000 7f49b18b4b80 1 --2- 172.21.15.86:0/526308106 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55c7387875b0 0x55c7387899d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:06.178 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.173+0000 7f49b18b4b80 4 client.27044 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T18:03:06.179 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.173+0000 7f49b18b4b80 1 -- 172.21.15.86:0/526308106 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c7384bbc90 con 0x55c7387875b0 2024-03-23T18:03:06.179 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.173+0000 7f4996a00700 1 --2- 172.21.15.86:0/526308106 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55c7387875b0 0x55c7387899d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:06.179 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.174+0000 7f4996a00700 1 --2- 172.21.15.86:0/526308106 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55c7387875b0 0x55c7387899d0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60f71ba57a951866 server_cookie=1588508bfa9f4435 in_seq=0 out_seq=0 2024-03-23T18:03:06.179 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.174+0000 7f4980c00700 10 client.27044 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T18:03:06.215 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.214+0000 7f4980c00700 1 -- 172.21.15.86:0/526308106 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c7384bbc90 con 0x55c7387875b0 2024-03-23T18:03:06.215 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.214+0000 7f4980c00700 10 client.27044 handle_command_reply: tid=0 2024-03-23T18:03:06.215 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.214+0000 7f49b18b4b80 1 client.27044 shutdown 2024-03-23T18:03:06.215 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.214+0000 7f49b18b4b80 1 -- 172.21.15.86:0/526308106 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c73871aa20 msgr2=0x55c73872a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:03:06.215 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.214+0000 7f49b18b4b80 1 --2- 172.21.15.86:0/526308106 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c73871aa20 0x55c73872a5f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f4984023620 tx=0x7f4984009280 comp rx=0 tx=0).stop 2024-03-23T18:03:06.215 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.214+0000 7f49b18b4b80 1 -- 172.21.15.86:0/526308106 shutdown_connections 2024-03-23T18:03:06.216 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.214+0000 7f49b18b4b80 1 --2- 172.21.15.86:0/526308106 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55c7387875b0 0x55c7387899d0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:06.216 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.214+0000 7f49b18b4b80 1 --2- 172.21.15.86:0/526308106 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c7387258d0 0x55c73872ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:06.216 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.214+0000 7f49b18b4b80 1 --2- 172.21.15.86:0/526308106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c73871a100 0x55c73872c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:06.216 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.214+0000 7f49b18b4b80 1 --2- 172.21.15.86:0/526308106 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c73871aa20 0x55c73872a5f0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:06.216 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.214+0000 7f49b18b4b80 1 -- 172.21.15.86:0/526308106 >> 172.21.15.86:0/526308106 conn(0x55c73867a640 msgr2=0x55c738715d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:03:06.216 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.214+0000 7f4980c00700 0 client.27044 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T18:03:06.216 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.215+0000 7f49b18b4b80 1 -- 172.21.15.86:0/526308106 shutdown_connections 2024-03-23T18:03:06.216 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.215+0000 7f49b18b4b80 1 -- 172.21.15.86:0/526308106 wait complete. 2024-03-23T18:03:06.216 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.215+0000 7f49b18b4b80 20 client.27044 trim_cache size 0 max 16384 2024-03-23T18:03:06.216 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-03-23T18:03:06.217 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-03-23T18:03:06.217 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-03-23T18:03:06.217 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-03-23T18:03:06.217 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.216+0000 7f499de00700 1 -- 172.21.15.86:0/258282211 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f497c0a1290 msgr2=0x7f497c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:03:06.218 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.216+0000 7f499de00700 1 --2- 172.21.15.86:0/258282211 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f497c0a1290 0x7f497c0a3750 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x55c73871bb90 tx=0x7f4990009920 comp rx=0 tx=0).stop 2024-03-23T18:03:06.218 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.216+0000 7f499de00700 1 -- 172.21.15.86:0/258282211 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4998101ee0 msgr2=0x7f4998199370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:03:06.218 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.216+0000 7f499de00700 1 --2- 172.21.15.86:0/258282211 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4998101ee0 0x7f4998199370 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4988000c00 tx=0x7f4988005740 comp rx=0 tx=0).stop 2024-03-23T18:03:06.218 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.216+0000 7f499de00700 1 -- 172.21.15.86:0/258282211 shutdown_connections 2024-03-23T18:03:06.218 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.216+0000 7f499de00700 1 --2- 172.21.15.86:0/258282211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f499810b2b0 0x7f499819dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:06.218 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.216+0000 7f499de00700 1 --2- 172.21.15.86:0/258282211 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4998101ee0 0x7f4998199370 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:06.218 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.216+0000 7f499de00700 1 --2- 172.21.15.86:0/258282211 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f497c0a1290 0x7f497c0a3750 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:06.218 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.216+0000 7f499de00700 1 --2- 172.21.15.86:0/258282211 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4998102800 0x7f49981998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:06.218 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.216+0000 7f499de00700 1 -- 172.21.15.86:0/258282211 >> 172.21.15.86:0/258282211 conn(0x7f49980fb680 msgr2=0x7f49980fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:03:06.218 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.216+0000 7f499de00700 1 -- 172.21.15.86:0/258282211 shutdown_connections 2024-03-23T18:03:06.218 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:06.216+0000 7f499de00700 1 -- 172.21.15.86:0/258282211 wait complete. 2024-03-23T18:03:06.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:03:05 smithi086 ceph-mon[38221]: pgmap v389: 97 pgs: 97 active+clean; 305 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 255 op/s 2024-03-23T18:03:06.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:03:05 smithi086 ceph-mon[38221]: osdmap e97: 12 total, 12 up, 12 in 2024-03-23T18:03:06.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:03:05 smithi092 ceph-mon[79089]: pgmap v389: 97 pgs: 97 active+clean; 305 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 255 op/s 2024-03-23T18:03:06.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:03:05 smithi092 ceph-mon[79089]: osdmap e97: 12 total, 12 up, 12 in 2024-03-23T18:03:06.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:03:05 smithi152 ceph-mon[47949]: pgmap v389: 97 pgs: 97 active+clean; 305 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 255 op/s 2024-03-23T18:03:06.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:03:05 smithi152 ceph-mon[47949]: osdmap e97: 12 total, 12 up, 12 in 2024-03-23T18:03:06.929 INFO:tasks.workunit.client.0.smithi086.stdout: 1 754287 6.37 MB/sec execute 571 sec latency 33.481 ms 2024-03-23T18:03:07.929 INFO:tasks.workunit.client.0.smithi086.stdout: 1 754969 6.36 MB/sec execute 572 sec latency 712.355 ms 2024-03-23T18:03:08.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:03:07 smithi086 ceph-mon[38221]: pgmap v391: 97 pgs: 97 active+clean; 305 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 977 KiB/s rd, 2.3 MiB/s wr, 213 op/s 2024-03-23T18:03:08.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:03:07 smithi092 ceph-mon[79089]: pgmap v391: 97 pgs: 97 active+clean; 305 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 977 KiB/s rd, 2.3 MiB/s wr, 213 op/s 2024-03-23T18:03:08.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:03:07 smithi152 ceph-mon[47949]: pgmap v391: 97 pgs: 97 active+clean; 305 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 977 KiB/s rd, 2.3 MiB/s wr, 213 op/s 2024-03-23T18:03:08.929 INFO:tasks.workunit.client.0.smithi086.stdout: 1 756628 6.38 MB/sec execute 573 sec latency 22.990 ms 2024-03-23T18:03:09.929 INFO:tasks.workunit.client.0.smithi086.stdout: 1 758009 6.37 MB/sec execute 574 sec latency 15.558 ms 2024-03-23T18:03:10.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:03:09 smithi086 ceph-mon[38221]: pgmap v392: 97 pgs: 97 active+clean; 305 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 977 KiB/s rd, 2.3 MiB/s wr, 213 op/s 2024-03-23T18:03:10.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:03:09 smithi092 ceph-mon[79089]: pgmap v392: 97 pgs: 97 active+clean; 305 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 977 KiB/s rd, 2.3 MiB/s wr, 213 op/s 2024-03-23T18:03:10.338 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:03:09 smithi152 ceph-mon[47949]: pgmap v392: 97 pgs: 97 active+clean; 305 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 977 KiB/s rd, 2.3 MiB/s wr, 213 op/s 2024-03-23T18:03:10.929 INFO:tasks.workunit.client.0.smithi086.stdout: 1 758012 6.36 MB/sec execute 575 sec latency 993.474 ms 2024-03-23T18:03:11.929 INFO:tasks.workunit.client.0.smithi086.stdout: 1 760087 6.38 MB/sec execute 576 sec latency 16.213 ms 2024-03-23T18:03:12.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:03:11 smithi086 ceph-mon[38221]: pgmap v393: 97 pgs: 97 active+clean; 296 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.4 MiB/s wr, 246 op/s 2024-03-23T18:03:12.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:03:11 smithi092 ceph-mon[79089]: pgmap v393: 97 pgs: 97 active+clean; 296 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.4 MiB/s wr, 246 op/s 2024-03-23T18:03:12.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:03:11 smithi152 ceph-mon[47949]: pgmap v393: 97 pgs: 97 active+clean; 296 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.4 MiB/s wr, 246 op/s 2024-03-23T18:03:12.929 INFO:tasks.workunit.client.0.smithi086.stdout: 1 761291 6.37 MB/sec execute 577 sec latency 14.172 ms 2024-03-23T18:03:13.930 INFO:tasks.workunit.client.0.smithi086.stdout: 1 761558 6.36 MB/sec execute 578 sec latency 768.719 ms 2024-03-23T18:03:14.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:03:13 smithi086 ceph-mon[38221]: pgmap v394: 97 pgs: 97 active+clean; 296 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.4 MiB/s wr, 246 op/s 2024-03-23T18:03:14.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:03:13 smithi092 ceph-mon[79089]: pgmap v394: 97 pgs: 97 active+clean; 296 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.4 MiB/s wr, 246 op/s 2024-03-23T18:03:14.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:03:13 smithi152 ceph-mon[47949]: pgmap v394: 97 pgs: 97 active+clean; 296 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.4 MiB/s wr, 246 op/s 2024-03-23T18:03:14.930 INFO:tasks.workunit.client.0.smithi086.stdout: 1 762267 6.36 MB/sec execute 579 sec latency 1479.935 ms 2024-03-23T18:03:15.930 INFO:tasks.workunit.client.0.smithi086.stdout: 1 763791 6.37 MB/sec execute 580 sec latency 16.369 ms 2024-03-23T18:03:16.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:03:15 smithi086 ceph-mon[38221]: pgmap v395: 97 pgs: 97 active+clean; 306 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.2 MiB/s wr, 205 op/s 2024-03-23T18:03:16.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:03:15 smithi086 ceph-mon[38221]: osdmap e98: 12 total, 12 up, 12 in 2024-03-23T18:03:16.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:03:15 smithi092 ceph-mon[79089]: pgmap v395: 97 pgs: 97 active+clean; 306 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.2 MiB/s wr, 205 op/s 2024-03-23T18:03:16.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:03:15 smithi092 ceph-mon[79089]: osdmap e98: 12 total, 12 up, 12 in 2024-03-23T18:03:16.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:03:15 smithi152 ceph-mon[47949]: pgmap v395: 97 pgs: 97 active+clean; 306 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.2 MiB/s wr, 205 op/s 2024-03-23T18:03:16.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:03:15 smithi152 ceph-mon[47949]: osdmap e98: 12 total, 12 up, 12 in 2024-03-23T18:03:16.930 INFO:tasks.workunit.client.0.smithi086.stdout: 1 765101 6.36 MB/sec execute 581 sec latency 94.750 ms 2024-03-23T18:03:17.930 INFO:tasks.workunit.client.0.smithi086.stdout: 1 765594 6.35 MB/sec execute 582 sec latency 917.858 ms 2024-03-23T18:03:18.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:03:17 smithi086 ceph-mon[38221]: pgmap v397: 97 pgs: 97 active+clean; 306 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.2 MiB/s wr, 205 op/s 2024-03-23T18:03:18.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:03:17 smithi092 ceph-mon[79089]: pgmap v397: 97 pgs: 97 active+clean; 306 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.2 MiB/s wr, 205 op/s 2024-03-23T18:03:18.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:03:17 smithi152 ceph-mon[47949]: pgmap v397: 97 pgs: 97 active+clean; 306 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.2 MiB/s wr, 205 op/s 2024-03-23T18:03:18.930 INFO:tasks.workunit.client.0.smithi086.stdout: 1 767265 6.37 MB/sec execute 583 sec latency 19.063 ms 2024-03-23T18:03:19.377 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.377+0000 7f3bfb200700 1 -- 172.21.15.86:0/450764504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bf4100ef0 msgr2=0x7f3bf41012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:03:19.378 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.377+0000 7f3bfb200700 1 --2- 172.21.15.86:0/450764504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bf4100ef0 0x7f3bf41012d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f3be4009990 tx=0x7f3be4009ca0 comp rx=0 tx=0).stop 2024-03-23T18:03:19.378 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.377+0000 7f3bfb200700 1 -- 172.21.15.86:0/450764504 shutdown_connections 2024-03-23T18:03:19.378 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.377+0000 7f3bfb200700 1 --2- 172.21.15.86:0/450764504 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3bf4106010 0x7f3bf410db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:19.378 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.377+0000 7f3bfb200700 1 --2- 172.21.15.86:0/450764504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bf4100ef0 0x7f3bf41012d0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:19.378 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.377+0000 7f3bfb200700 1 --2- 172.21.15.86:0/450764504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bf41018a0 0x7f3bf41058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:19.378 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.377+0000 7f3bfb200700 1 -- 172.21.15.86:0/450764504 >> 172.21.15.86:0/450764504 conn(0x7f3bf40fc750 msgr2=0x7f3bf40feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:03:19.378 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.377+0000 7f3bfb200700 1 -- 172.21.15.86:0/450764504 shutdown_connections 2024-03-23T18:03:19.378 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.377+0000 7f3bfb200700 1 -- 172.21.15.86:0/450764504 wait complete. 2024-03-23T18:03:19.379 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.379+0000 7f3bfb200700 1 Processor -- start 2024-03-23T18:03:19.379 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.379+0000 7f3bfb200700 1 -- start start 2024-03-23T18:03:19.379 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.379+0000 7f3bfb200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bf4100ef0 0x7f3bf4199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:19.380 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.379+0000 7f3bfb200700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3bf4106010 0x7f3bf4199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:19.380 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.379+0000 7f3bfb200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bf419dc10 0x7f3bf419e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:19.380 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.379+0000 7f3bfb200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf419e690 con 0x7f3bf4100ef0 2024-03-23T18:03:19.380 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.379+0000 7f3bfb200700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf419e800 con 0x7f3bf419dc10 2024-03-23T18:03:19.380 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.379+0000 7f3bfb200700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf419e9c0 con 0x7f3bf4106010 2024-03-23T18:03:19.380 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.380+0000 7f3bf3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bf4100ef0 0x7f3bf4199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:19.380 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.380+0000 7f3bf9200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bf419dc10 0x7f3bf419e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:19.380 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.380+0000 7f3bf9200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bf419dc10 0x7f3bf419e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:51112/0 (socket says 172.21.15.86:51112) 2024-03-23T18:03:19.380 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.380+0000 7f3bf9200700 1 -- 172.21.15.86:0/3651860019 learned_addr learned my addr 172.21.15.86:0/3651860019 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:03:19.380 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.380+0000 7f3bf3400700 1 --2- 172.21.15.86:0/3651860019 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3bf4106010 0x7f3bf4199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:19.381 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.380+0000 7f3bf9200700 1 -- 172.21.15.86:0/3651860019 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3bf4106010 msgr2=0x7f3bf4199870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:03:19.381 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.380+0000 7f3bf9200700 1 --2- 172.21.15.86:0/3651860019 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3bf4106010 0x7f3bf4199870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:19.381 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.380+0000 7f3bf9200700 1 -- 172.21.15.86:0/3651860019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bf4100ef0 msgr2=0x7f3bf4199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:03:19.381 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.380+0000 7f3bf9200700 1 --2- 172.21.15.86:0/3651860019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bf4100ef0 0x7f3bf4199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:19.382 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.380+0000 7f3bf9200700 1 -- 172.21.15.86:0/3651860019 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3be40095f0 con 0x7f3bf419dc10 2024-03-23T18:03:19.382 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.380+0000 7f3bf9200700 1 --2- 172.21.15.86:0/3651860019 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bf419dc10 0x7f3bf419e090 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f3bec00ef30 tx=0x7f3bec00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:03:19.382 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.381+0000 7f3bf0c00700 1 -- 172.21.15.86:0/3651860019 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bec018070 con 0x7f3bf419dc10 2024-03-23T18:03:19.382 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.381+0000 7f3bf0c00700 1 -- 172.21.15.86:0/3651860019 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3bec00f040 con 0x7f3bf419dc10 2024-03-23T18:03:19.382 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.381+0000 7f3bfb200700 1 -- 172.21.15.86:0/3651860019 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bf41a5ec0 con 0x7f3bf419dc10 2024-03-23T18:03:19.382 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.381+0000 7f3bf0c00700 1 -- 172.21.15.86:0/3651860019 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bec00cd00 con 0x7f3bf419dc10 2024-03-23T18:03:19.382 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.381+0000 7f3bfb200700 1 -- 172.21.15.86:0/3651860019 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bf41a63f0 con 0x7f3bf419dc10 2024-03-23T18:03:19.384 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.384+0000 7f3bf0c00700 1 -- 172.21.15.86:0/3651860019 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f3bec003a00 con 0x7f3bf419dc10 2024-03-23T18:03:19.387 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.385+0000 7f3bf0c00700 1 --2- 172.21.15.86:0/3651860019 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f3bd80a1310 0x7f3bd80a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:19.387 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.385+0000 7f3c0edc8b80 1 Processor -- start 2024-03-23T18:03:19.387 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.385+0000 7f3c0edc8b80 1 -- start start 2024-03-23T18:03:19.387 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.385+0000 7f3bf0c00700 1 -- 172.21.15.86:0/3651860019 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f3bec0cc750 con 0x7f3bf419dc10 2024-03-23T18:03:19.387 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.385+0000 7f3bf3e00700 1 --2- 172.21.15.86:0/3651860019 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f3bd80a1310 0x7f3bd80a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:19.387 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.386+0000 7f3c0edc8b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e22f39d910 0x55e22f39dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:19.387 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.386+0000 7f3c0edc8b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e22f39e230 0x55e22f3a8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:19.387 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.386+0000 7f3c0edc8b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e22f3a90e0 0x55e22f3ab4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:19.388 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.386+0000 7f3c0edc8b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e22f2ee930 con 0x55e22f39e230 2024-03-23T18:03:19.388 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.386+0000 7f3c0edc8b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e22f2ec2f0 con 0x55e22f3a90e0 2024-03-23T18:03:19.388 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.386+0000 7f3c0edc8b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e22f2e7be0 con 0x55e22f39d910 2024-03-23T18:03:19.388 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.386+0000 7f3bf9200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e22f3a90e0 0x55e22f3ab4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:19.388 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.386+0000 7f3bf3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e22f39e230 0x55e22f3a8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:19.388 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.386+0000 7f3bf3e00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e22f39d910 0x55e22f39dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:19.388 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.386+0000 7f3bf9200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e22f3a90e0 0x55e22f3ab4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:51120/0 (socket says 172.21.15.86:51120) 2024-03-23T18:03:19.388 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.386+0000 7f3bf9200700 1 -- 172.21.15.86:0/2959409215 learned_addr learned my addr 172.21.15.86:0/2959409215 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:03:19.388 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.386+0000 7f3bf3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e22f39e230 0x55e22f3a8ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52742/0 (socket says 172.21.15.86:52742) 2024-03-23T18:03:19.388 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.386+0000 7f3bf9200700 1 -- 172.21.15.86:0/2959409215 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e22f39d910 msgr2=0x55e22f39dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:03:19.388 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.386+0000 7f3bf3e00700 1 --2- 172.21.15.86:0/3651860019 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f3bd80a1310 0x7f3bd80a37d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3be4006010 tx=0x7f3be4005110 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:03:19.388 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.386+0000 7f3bf9200700 1 --2- 172.21.15.86:0/2959409215 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e22f39d910 0x55e22f39dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:19.389 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.386+0000 7f3bf9200700 1 -- 172.21.15.86:0/2959409215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e22f39e230 msgr2=0x55e22f3a8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:03:19.389 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.386+0000 7f3bf9200700 1 --2- 172.21.15.86:0/2959409215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e22f39e230 0x55e22f3a8ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:19.389 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.386+0000 7f3bf9200700 1 -- 172.21.15.86:0/2959409215 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e22f169350 con 0x55e22f3a90e0 2024-03-23T18:03:19.389 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.387+0000 7f3bf9200700 1 --2- 172.21.15.86:0/2959409215 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e22f3a90e0 0x55e22f3ab4d0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f3bec0a1770 tx=0x7f3bec0997b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5aa1ad966e919491 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:03:19.389 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.387+0000 7f3be9600700 1 -- 172.21.15.86:0/2959409215 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bec004ad0 con 0x55e22f3a90e0 2024-03-23T18:03:19.389 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.387+0000 7f3be9600700 1 -- 172.21.15.86:0/2959409215 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3bec01b320 con 0x55e22f3a90e0 2024-03-23T18:03:19.389 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.387+0000 7f3be9600700 1 -- 172.21.15.86:0/2959409215 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bec0a1ca0 con 0x55e22f3a90e0 2024-03-23T18:03:19.394 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.387+0000 7f3c0edc8b80 1 -- 172.21.15.86:0/2959409215 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e22f3a90e0 msgr2=0x55e22f3ab4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:03:19.394 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.387+0000 7f3c0edc8b80 1 --2- 172.21.15.86:0/2959409215 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e22f3a90e0 0x55e22f3ab4d0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f3bec0a1770 tx=0x7f3bec0997b0 comp rx=0 tx=0).stop 2024-03-23T18:03:19.394 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.388+0000 7f3c0edc8b80 1 -- 172.21.15.86:0/2959409215 shutdown_connections 2024-03-23T18:03:19.394 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.388+0000 7f3c0edc8b80 1 --2- 172.21.15.86:0/2959409215 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e22f3a90e0 0x55e22f3ab4d0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:19.394 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.388+0000 7f3c0edc8b80 1 --2- 172.21.15.86:0/2959409215 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e22f39d910 0x55e22f39dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:19.394 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.388+0000 7f3c0edc8b80 1 --2- 172.21.15.86:0/2959409215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e22f39e230 0x55e22f3a8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:19.394 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.388+0000 7f3c0edc8b80 1 -- 172.21.15.86:0/2959409215 >> 172.21.15.86:0/2959409215 conn(0x55e22f2f5640 msgr2=0x55e22f391520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:03:19.395 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.388+0000 7f3c0edc8b80 1 -- 172.21.15.86:0/2959409215 shutdown_connections 2024-03-23T18:03:19.395 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.388+0000 7f3c0edc8b80 1 -- 172.21.15.86:0/2959409215 wait complete. 2024-03-23T18:03:19.395 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.388+0000 7f3c0edc8b80 1 -- start start 2024-03-23T18:03:19.395 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.388+0000 7f3c0edc8b80 1 Processor -- start 2024-03-23T18:03:19.395 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.389+0000 7f3c0edc8b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e22f39d910 0x55e22f3a8b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:19.395 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.389+0000 7f3c0edc8b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e22f39e230 0x55e22f3a5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:19.395 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.389+0000 7f3c0edc8b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e22f3a90e0 0x55e22f3a6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:19.395 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.389+0000 7f3c0edc8b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e22f283d20 con 0x55e22f39d910 2024-03-23T18:03:19.395 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.389+0000 7f3c0edc8b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e22f2839c0 con 0x55e22f39e230 2024-03-23T18:03:19.395 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.389+0000 7f3c0edc8b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e22f16b640 con 0x55e22f3a90e0 2024-03-23T18:03:19.395 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.389+0000 7f3c0edc8b80 10 client.0 _ll_register_callbacks cb 0x55e22f28d410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T18:03:19.395 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.389+0000 7f3bf3400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e22f39e230 0x55e22f3a5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:19.396 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.389+0000 7f3bf3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e22f39d910 0x55e22f3a8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:19.396 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.389+0000 7f3bf3400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e22f39e230 0x55e22f3a5e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:51128/0 (socket says 172.21.15.86:51128) 2024-03-23T18:03:19.396 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.389+0000 7f3bf3400700 1 -- 172.21.15.86:0/535506484 learned_addr learned my addr 172.21.15.86:0/535506484 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:03:19.396 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.389+0000 7f3bf9200700 1 --2- 172.21.15.86:0/535506484 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e22f3a90e0 0x55e22f3a6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:19.396 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.390+0000 7f3bf3400700 1 -- 172.21.15.86:0/535506484 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e22f3a90e0 msgr2=0x55e22f3a6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:03:19.396 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.390+0000 7f3bf3400700 1 --2- 172.21.15.86:0/535506484 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e22f3a90e0 0x55e22f3a6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:19.396 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.390+0000 7f3bf3400700 1 -- 172.21.15.86:0/535506484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e22f39d910 msgr2=0x55e22f3a8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:03:19.396 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.390+0000 7f3bf3400700 1 --2- 172.21.15.86:0/535506484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e22f39d910 0x55e22f3a8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:19.396 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.390+0000 7f3bf3400700 1 -- 172.21.15.86:0/535506484 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bf41a63f0 con 0x55e22f39e230 2024-03-23T18:03:19.397 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.390+0000 7f3bf3400700 1 --2- 172.21.15.86:0/535506484 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e22f39e230 0x55e22f3a5e50 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f3be0012840 tx=0x7f3be0012b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62c2dc934334a90a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:03:19.397 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.390+0000 7f3bde000700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T18:03:19.397 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.390+0000 7f3bde000700 1 -- 172.21.15.86:0/535506484 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be0017070 con 0x55e22f39e230 2024-03-23T18:03:19.397 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.390+0000 7f3bde000700 1 -- 172.21.15.86:0/535506484 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f3be0004830 con 0x55e22f39e230 2024-03-23T18:03:19.397 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.390+0000 7f3c0edc8b80 1 -- 172.21.15.86:0/535506484 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e22f2a8320 con 0x55e22f39e230 2024-03-23T18:03:19.397 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.391+0000 7f3bde000700 1 -- 172.21.15.86:0/535506484 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3be0004a30 con 0x55e22f39e230 2024-03-23T18:03:19.397 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.391+0000 7f3c0edc8b80 10 client.27056 fetch_fsmap learned FSMap version 52 2024-03-23T18:03:19.397 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.391+0000 7f3c0edc8b80 1 -- 172.21.15.86:0/535506484 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x55e22f115b00 con 0x55e22f39e230 2024-03-23T18:03:19.397 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.391+0000 7f3bde000700 1 -- 172.21.15.86:0/535506484 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e22f2a8320 con 0x55e22f39e230 2024-03-23T18:03:19.397 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.391+0000 7f3bde000700 1 -- 172.21.15.86:0/535506484 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 52) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f3be000f4e0 con 0x55e22f39e230 2024-03-23T18:03:19.398 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.391+0000 7f3c0edc8b80 10 client.27056 fetch_fsmap finished waiting for FSMap version 52 2024-03-23T18:03:19.398 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.391+0000 7f3c0edc8b80 10 client.27056 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T18:03:19.398 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.391+0000 7f3c0edc8b80 20 client.27056 populate_metadata read hostname 'smithi086' 2024-03-23T18:03:19.398 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.391+0000 7f3c0edc8b80 1 --2- 172.21.15.86:0/535506484 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55e22f3b8e20 0x55e22f3bb240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:19.398 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.391+0000 7f3c0edc8b80 4 client.27056 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T18:03:19.398 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.391+0000 7f3c0edc8b80 1 -- 172.21.15.86:0/535506484 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e22f2e06a0 con 0x55e22f3b8e20 2024-03-23T18:03:19.398 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.391+0000 7f3bf3e00700 1 --2- 172.21.15.86:0/535506484 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55e22f3b8e20 0x55e22f3bb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:19.399 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.392+0000 7f3bf3e00700 1 --2- 172.21.15.86:0/535506484 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55e22f3b8e20 0x55e22f3bb240 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=348162e263013621 server_cookie=2b5f4f77cf37ced4 in_seq=0 out_seq=0 2024-03-23T18:03:19.399 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.392+0000 7f3bde000700 10 client.27056 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T18:03:19.399 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.392+0000 7f3bde000700 1 -- 172.21.15.86:0/535506484 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55e22f2e06a0 con 0x55e22f3b8e20 2024-03-23T18:03:19.399 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.392+0000 7f3bde000700 10 client.27056 handle_command_reply: tid=0 2024-03-23T18:03:19.399 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.392+0000 7f3c0edc8b80 1 client.27056 shutdown 2024-03-23T18:03:19.399 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.393+0000 7f3c0edc8b80 1 -- 172.21.15.86:0/535506484 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e22f39e230 msgr2=0x55e22f3a5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:03:19.399 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.393+0000 7f3c0edc8b80 1 --2- 172.21.15.86:0/535506484 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e22f39e230 0x55e22f3a5e50 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f3be0012840 tx=0x7f3be0012b50 comp rx=0 tx=0).stop 2024-03-23T18:03:19.399 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.393+0000 7f3c0edc8b80 1 -- 172.21.15.86:0/535506484 shutdown_connections 2024-03-23T18:03:19.399 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.393+0000 7f3c0edc8b80 1 --2- 172.21.15.86:0/535506484 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55e22f3b8e20 0x55e22f3bb240 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:19.399 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.393+0000 7f3c0edc8b80 1 --2- 172.21.15.86:0/535506484 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e22f3a90e0 0x55e22f3a6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:19.399 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.393+0000 7f3c0edc8b80 1 --2- 172.21.15.86:0/535506484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e22f39d910 0x55e22f3a8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:19.400 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.393+0000 7f3bde000700 0 client.27056 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T18:03:19.400 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.393+0000 7f3c0edc8b80 1 --2- 172.21.15.86:0/535506484 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e22f39e230 0x55e22f3a5e50 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:19.400 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.393+0000 7f3c0edc8b80 1 -- 172.21.15.86:0/535506484 >> 172.21.15.86:0/535506484 conn(0x55e22f2f5640 msgr2=0x55e22f39f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:03:19.400 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.393+0000 7f3c0edc8b80 1 -- 172.21.15.86:0/535506484 shutdown_connections 2024-03-23T18:03:19.400 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.393+0000 7f3c0edc8b80 1 -- 172.21.15.86:0/535506484 wait complete. 2024-03-23T18:03:19.400 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.393+0000 7f3c0edc8b80 20 client.27056 trim_cache size 0 max 16384 2024-03-23T18:03:19.412 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.412+0000 7f3c0edc8b80 1 Processor -- start 2024-03-23T18:03:19.412 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.412+0000 7f3c0edc8b80 1 -- start start 2024-03-23T18:03:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.412+0000 7f3c0edc8b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e22f39d910 0x55e22f3a5df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.412+0000 7f3c0edc8b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e22f39e230 0x55e22f3afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.412+0000 7f3c0edc8b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e22f3a90e0 0x55e22f38b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.413+0000 7f3c0edc8b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e22f3bff60 con 0x55e22f39d910 2024-03-23T18:03:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.413+0000 7f3c0edc8b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e22f3b6420 con 0x55e22f39e230 2024-03-23T18:03:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.413+0000 7f3c0edc8b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e22f3a1cb0 con 0x55e22f3a90e0 2024-03-23T18:03:19.415 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.413+0000 7f3bf3400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e22f39e230 0x55e22f3afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:19.415 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.413+0000 7f3bf3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e22f39d910 0x55e22f3a5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:19.415 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.413+0000 7f3bf3400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e22f39e230 0x55e22f3afdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:51142/0 (socket says 172.21.15.86:51142) 2024-03-23T18:03:19.415 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.413+0000 7f3bf3400700 1 -- 172.21.15.86:0/1722013238 learned_addr learned my addr 172.21.15.86:0/1722013238 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:03:19.415 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.413+0000 7f3bf9200700 1 --2- 172.21.15.86:0/1722013238 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e22f3a90e0 0x55e22f38b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:19.415 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.413+0000 7f3bf3400700 1 -- 172.21.15.86:0/1722013238 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e22f3a90e0 msgr2=0x55e22f38b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:03:19.415 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.413+0000 7f3bf3400700 1 --2- 172.21.15.86:0/1722013238 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e22f3a90e0 0x55e22f38b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:19.415 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.413+0000 7f3bf3400700 1 -- 172.21.15.86:0/1722013238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e22f39d910 msgr2=0x55e22f3a5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:03:19.415 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.413+0000 7f3bf3400700 1 --2- 172.21.15.86:0/1722013238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e22f39d910 0x55e22f3a5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:19.415 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.413+0000 7f3bf3400700 1 -- 172.21.15.86:0/1722013238 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e22f2ade40 con 0x55e22f39e230 2024-03-23T18:03:19.415 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.413+0000 7f3bf9200700 1 --2- 172.21.15.86:0/1722013238 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e22f3a90e0 0x55e22f38b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T18:03:19.415 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.413+0000 7f3bf3e00700 1 --2- 172.21.15.86:0/1722013238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e22f39d910 0x55e22f3a5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T18:03:19.416 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.413+0000 7f3bf3400700 1 --2- 172.21.15.86:0/1722013238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e22f39e230 0x55e22f3afdf0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f3be000f870 tx=0x7f3be000fa10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73348dfa31d9b6ae server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:03:19.416 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.414+0000 7f3bea000700 1 -- 172.21.15.86:0/1722013238 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be0017070 con 0x55e22f39e230 2024-03-23T18:03:19.416 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.414+0000 7f3bea000700 1 -- 172.21.15.86:0/1722013238 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3be0018040 con 0x55e22f39e230 2024-03-23T18:03:19.416 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.414+0000 7f3bea000700 1 -- 172.21.15.86:0/1722013238 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be00038e0 con 0x55e22f39e230 2024-03-23T18:03:19.419 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.415+0000 7f3c0edc8b80 1 -- 172.21.15.86:0/1722013238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e22f39e230 msgr2=0x55e22f3afdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:03:19.419 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.415+0000 7f3c0edc8b80 1 --2- 172.21.15.86:0/1722013238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e22f39e230 0x55e22f3afdf0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f3be000f870 tx=0x7f3be000fa10 comp rx=0 tx=0).stop 2024-03-23T18:03:19.419 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.415+0000 7f3c0edc8b80 1 -- 172.21.15.86:0/1722013238 shutdown_connections 2024-03-23T18:03:19.419 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.415+0000 7f3c0edc8b80 1 --2- 172.21.15.86:0/1722013238 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e22f3a90e0 0x55e22f38b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:19.419 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.415+0000 7f3c0edc8b80 1 --2- 172.21.15.86:0/1722013238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e22f39d910 0x55e22f3a5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:19.419 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.415+0000 7f3c0edc8b80 1 --2- 172.21.15.86:0/1722013238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e22f39e230 0x55e22f3afdf0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:19.419 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.415+0000 7f3c0edc8b80 1 -- 172.21.15.86:0/1722013238 >> 172.21.15.86:0/1722013238 conn(0x55e22f2f5640 msgr2=0x55e22f3a56e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:03:19.419 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.415+0000 7f3c0edc8b80 1 -- 172.21.15.86:0/1722013238 shutdown_connections 2024-03-23T18:03:19.419 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.415+0000 7f3c0edc8b80 1 -- 172.21.15.86:0/1722013238 wait complete. 2024-03-23T18:03:19.419 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.415+0000 7f3c0edc8b80 1 -- start start 2024-03-23T18:03:19.419 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.415+0000 7f3c0edc8b80 1 Processor -- start 2024-03-23T18:03:19.419 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.416+0000 7f3c0edc8b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e22f39d910 0x55e22f3afab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:19.419 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.416+0000 7f3c0edc8b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e22f39e230 0x55e22f3adfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:19.420 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.416+0000 7f3c0edc8b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e22f3a90e0 0x55e22f3ae4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:19.420 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.416+0000 7f3c0edc8b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e22f3aea30 con 0x55e22f39d910 2024-03-23T18:03:19.420 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.416+0000 7f3c0edc8b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e22f3aeba0 con 0x55e22f3a90e0 2024-03-23T18:03:19.420 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.416+0000 7f3c0edc8b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e22f3aed60 con 0x55e22f39e230 2024-03-23T18:03:19.420 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.416+0000 7f3c0edc8b80 10 client.0 _ll_register_callbacks cb 0x55e22f28d410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T18:03:19.420 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.416+0000 7f3bf3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e22f39d910 0x55e22f3afab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:19.420 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.416+0000 7f3bf3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e22f39d910 0x55e22f3afab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52764/0 (socket says 172.21.15.86:52764) 2024-03-23T18:03:19.420 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.416+0000 7f3bf3e00700 1 -- 172.21.15.86:0/4176056674 learned_addr learned my addr 172.21.15.86:0/4176056674 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:03:19.420 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.416+0000 7f3bf3400700 1 --2- 172.21.15.86:0/4176056674 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e22f39e230 0x55e22f3adfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:19.420 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.416+0000 7f3bf9200700 1 --2- 172.21.15.86:0/4176056674 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e22f3a90e0 0x55e22f3ae4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:19.420 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.417+0000 7f3bf3400700 1 -- 172.21.15.86:0/4176056674 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e22f3a90e0 msgr2=0x55e22f3ae4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:03:19.420 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.417+0000 7f3bf3400700 1 --2- 172.21.15.86:0/4176056674 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e22f3a90e0 0x55e22f3ae4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:19.420 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.417+0000 7f3bf3400700 1 -- 172.21.15.86:0/4176056674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e22f39d910 msgr2=0x55e22f3afab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:03:19.420 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.417+0000 7f3bf3400700 1 --2- 172.21.15.86:0/4176056674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e22f39d910 0x55e22f3afab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:19.421 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.417+0000 7f3bf3400700 1 -- 172.21.15.86:0/4176056674 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bf410e400 con 0x55e22f39e230 2024-03-23T18:03:19.421 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.417+0000 7f3bf3e00700 1 --2- 172.21.15.86:0/4176056674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e22f39d910 0x55e22f3afab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T18:03:19.421 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.417+0000 7f3bf3400700 1 --2- 172.21.15.86:0/4176056674 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e22f39e230 0x55e22f3adfb0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x55e22f3a73a0 tx=0x7f3be00123a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d1c9cb0ee7533dc server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:03:19.421 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.417+0000 7f3bde000700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-03-23T18:03:19.421 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.417+0000 7f3bde000700 1 -- 172.21.15.86:0/4176056674 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be0017040 con 0x55e22f39e230 2024-03-23T18:03:19.421 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.417+0000 7f3bde000700 1 -- 172.21.15.86:0/4176056674 <== mon.1 v2:172.21.15.152:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f3be0000e40 con 0x55e22f39e230 2024-03-23T18:03:19.421 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.417+0000 7f3c0edc8b80 1 -- 172.21.15.86:0/4176056674 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3bec01b320 con 0x55e22f39e230 2024-03-23T18:03:19.421 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.417+0000 7f3bde000700 1 -- 172.21.15.86:0/4176056674 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3be00052e0 con 0x55e22f39e230 2024-03-23T18:03:19.421 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.417+0000 7f3c0edc8b80 10 client.27037 fetch_fsmap learned FSMap version 52 2024-03-23T18:03:19.421 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.417+0000 7f3c0edc8b80 1 -- 172.21.15.86:0/4176056674 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x55e22f115b00 con 0x55e22f39e230 2024-03-23T18:03:19.422 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.418+0000 7f3bde000700 1 -- 172.21.15.86:0/4176056674 <== mon.1 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3bec01b320 con 0x55e22f39e230 2024-03-23T18:03:19.422 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.418+0000 7f3bde000700 1 -- 172.21.15.86:0/4176056674 <== mon.1 v2:172.21.15.152:3300/0 5 ==== fsmap(e 52) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f3be001c470 con 0x55e22f39e230 2024-03-23T18:03:19.422 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.418+0000 7f3c0edc8b80 10 client.27037 fetch_fsmap finished waiting for FSMap version 52 2024-03-23T18:03:19.422 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.418+0000 7f3c0edc8b80 10 client.27037 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T18:03:19.422 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.418+0000 7f3c0edc8b80 20 client.27037 populate_metadata read hostname 'smithi086' 2024-03-23T18:03:19.422 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.418+0000 7f3c0edc8b80 1 --2- 172.21.15.86:0/4176056674 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55e22f40bbc0 0x55e22f40dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:19.422 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.418+0000 7f3c0edc8b80 4 client.27037 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T18:03:19.422 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.418+0000 7f3c0edc8b80 1 -- 172.21.15.86:0/4176056674 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e22f136c90 con 0x55e22f40bbc0 2024-03-23T18:03:19.422 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.418+0000 7f3bf3e00700 1 --2- 172.21.15.86:0/4176056674 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55e22f40bbc0 0x55e22f40dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:19.422 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.419+0000 7f3bf3e00700 1 --2- 172.21.15.86:0/4176056674 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55e22f40bbc0 0x55e22f40dfe0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eda1d579010865dd server_cookie=ed188b8d18854318 in_seq=0 out_seq=0 2024-03-23T18:03:19.422 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.419+0000 7f3bde000700 10 client.27037 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T18:03:19.460 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.459+0000 7f3bde000700 1 -- 172.21.15.86:0/4176056674 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e22f136c90 con 0x55e22f40bbc0 2024-03-23T18:03:19.460 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.459+0000 7f3bde000700 10 client.27037 handle_command_reply: tid=0 2024-03-23T18:03:19.460 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.459+0000 7f3c0edc8b80 1 client.27037 shutdown 2024-03-23T18:03:19.460 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.459+0000 7f3c0edc8b80 1 -- 172.21.15.86:0/4176056674 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e22f39e230 msgr2=0x55e22f3adfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:03:19.460 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.459+0000 7f3c0edc8b80 1 --2- 172.21.15.86:0/4176056674 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e22f39e230 0x55e22f3adfb0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x55e22f3a73a0 tx=0x7f3be00123a0 comp rx=0 tx=0).stop 2024-03-23T18:03:19.460 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.459+0000 7f3c0edc8b80 1 -- 172.21.15.86:0/4176056674 shutdown_connections 2024-03-23T18:03:19.460 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.459+0000 7f3c0edc8b80 1 --2- 172.21.15.86:0/4176056674 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55e22f40bbc0 0x55e22f40dfe0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:19.461 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.459+0000 7f3c0edc8b80 1 --2- 172.21.15.86:0/4176056674 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e22f3a90e0 0x55e22f3ae4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:19.461 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.459+0000 7f3c0edc8b80 1 --2- 172.21.15.86:0/4176056674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e22f39d910 0x55e22f3afab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:19.461 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.459+0000 7f3c0edc8b80 1 --2- 172.21.15.86:0/4176056674 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e22f39e230 0x55e22f3adfb0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:19.461 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.459+0000 7f3bde000700 0 client.27037 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T18:03:19.461 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.459+0000 7f3c0edc8b80 1 -- 172.21.15.86:0/4176056674 >> 172.21.15.86:0/4176056674 conn(0x55e22f2f5640 msgr2=0x55e22f390d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:03:19.461 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.460+0000 7f3c0edc8b80 1 -- 172.21.15.86:0/4176056674 shutdown_connections 2024-03-23T18:03:19.461 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.460+0000 7f3c0edc8b80 1 -- 172.21.15.86:0/4176056674 wait complete. 2024-03-23T18:03:19.461 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.460+0000 7f3c0edc8b80 20 client.27037 trim_cache size 0 max 16384 2024-03-23T18:03:19.462 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-03-23T18:03:19.462 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-03-23T18:03:19.462 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-03-23T18:03:19.462 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-03-23T18:03:19.462 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.460+0000 7f3bfb200700 1 -- 172.21.15.86:0/3651860019 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f3bd80a1310 msgr2=0x7f3bd80a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:03:19.463 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.460+0000 7f3bfb200700 1 --2- 172.21.15.86:0/3651860019 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f3bd80a1310 0x7f3bd80a37d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3be4006010 tx=0x7f3be4005110 comp rx=0 tx=0).stop 2024-03-23T18:03:19.463 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.461+0000 7f3bfb200700 1 -- 172.21.15.86:0/3651860019 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bf419dc10 msgr2=0x7f3bf419e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:03:19.463 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.461+0000 7f3bfb200700 1 --2- 172.21.15.86:0/3651860019 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bf419dc10 0x7f3bf419e090 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f3bec00ef30 tx=0x7f3bec00c3a0 comp rx=0 tx=0).stop 2024-03-23T18:03:19.463 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.461+0000 7f3bfb200700 1 -- 172.21.15.86:0/3651860019 shutdown_connections 2024-03-23T18:03:19.463 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.461+0000 7f3bfb200700 1 --2- 172.21.15.86:0/3651860019 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bf419dc10 0x7f3bf419e090 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:19.463 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.461+0000 7f3bfb200700 1 --2- 172.21.15.86:0/3651860019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bf4100ef0 0x7f3bf4199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:19.463 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.461+0000 7f3bfb200700 1 --2- 172.21.15.86:0/3651860019 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f3bd80a1310 0x7f3bd80a37d0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:19.463 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.461+0000 7f3bfb200700 1 --2- 172.21.15.86:0/3651860019 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3bf4106010 0x7f3bf4199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:19.463 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.461+0000 7f3bfb200700 1 -- 172.21.15.86:0/3651860019 >> 172.21.15.86:0/3651860019 conn(0x7f3bf40fc750 msgr2=0x7f3bf40fdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:03:19.463 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.461+0000 7f3bfb200700 1 -- 172.21.15.86:0/3651860019 shutdown_connections 2024-03-23T18:03:19.463 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:19.461+0000 7f3bfb200700 1 -- 172.21.15.86:0/3651860019 wait complete. 2024-03-23T18:03:19.930 INFO:tasks.workunit.client.0.smithi086.stdout: 1 768566 6.36 MB/sec execute 584 sec latency 29.161 ms 2024-03-23T18:03:20.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:03:19 smithi086 ceph-mon[38221]: pgmap v398: 97 pgs: 97 active+clean; 306 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.2 MiB/s wr, 205 op/s 2024-03-23T18:03:20.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:03:19 smithi092 ceph-mon[79089]: pgmap v398: 97 pgs: 97 active+clean; 306 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.2 MiB/s wr, 205 op/s 2024-03-23T18:03:20.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:03:19 smithi152 ceph-mon[47949]: pgmap v398: 97 pgs: 97 active+clean; 306 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.2 MiB/s wr, 205 op/s 2024-03-23T18:03:20.930 INFO:tasks.workunit.client.0.smithi086.stdout: 1 768652 6.35 MB/sec execute 585 sec latency 922.439 ms 2024-03-23T18:03:21.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:03:20 smithi086 ceph-mon[38221]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:03:21.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:03:20 smithi092 ceph-mon[79089]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:03:21.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:03:20 smithi152 ceph-mon[47949]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:03:21.930 INFO:tasks.workunit.client.0.smithi086.stdout: 1 770728 6.37 MB/sec execute 586 sec latency 15.706 ms 2024-03-23T18:03:22.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:03:21 smithi086 ceph-mon[38221]: pgmap v399: 97 pgs: 97 active+clean; 300 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.3 MiB/s wr, 202 op/s 2024-03-23T18:03:22.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:03:21 smithi092 ceph-mon[79089]: pgmap v399: 97 pgs: 97 active+clean; 300 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.3 MiB/s wr, 202 op/s 2024-03-23T18:03:22.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:03:21 smithi152 ceph-mon[47949]: pgmap v399: 97 pgs: 97 active+clean; 300 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.3 MiB/s wr, 202 op/s 2024-03-23T18:03:22.930 INFO:tasks.workunit.client.0.smithi086.stdout: 1 771952 6.36 MB/sec execute 587 sec latency 13.493 ms 2024-03-23T18:03:23.930 INFO:tasks.workunit.client.0.smithi086.stdout: 1 772191 6.35 MB/sec execute 588 sec latency 838.289 ms 2024-03-23T18:03:24.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:03:23 smithi086 ceph-mon[38221]: pgmap v400: 97 pgs: 97 active+clean; 300 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.3 MiB/s wr, 202 op/s 2024-03-23T18:03:24.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:03:23 smithi092 ceph-mon[79089]: pgmap v400: 97 pgs: 97 active+clean; 300 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.3 MiB/s wr, 202 op/s 2024-03-23T18:03:24.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:03:23 smithi152 ceph-mon[47949]: pgmap v400: 97 pgs: 97 active+clean; 300 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.3 MiB/s wr, 202 op/s 2024-03-23T18:03:24.930 INFO:tasks.workunit.client.0.smithi086.stdout: 1 772907 6.35 MB/sec execute 589 sec latency 1547.066 ms 2024-03-23T18:03:25.930 INFO:tasks.workunit.client.0.smithi086.stdout: 1 774436 6.35 MB/sec execute 590 sec latency 15.796 ms 2024-03-23T18:03:26.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:03:25 smithi086 ceph-mon[38221]: pgmap v401: 97 pgs: 97 active+clean; 307 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 2.2 MiB/s wr, 198 op/s 2024-03-23T18:03:26.310 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:03:25 smithi092 ceph-mon[79089]: pgmap v401: 97 pgs: 97 active+clean; 307 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 2.2 MiB/s wr, 198 op/s 2024-03-23T18:03:26.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:03:25 smithi152 ceph-mon[47949]: pgmap v401: 97 pgs: 97 active+clean; 307 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 2.2 MiB/s wr, 198 op/s 2024-03-23T18:03:26.931 INFO:tasks.workunit.client.0.smithi086.stdout: 1 775740 6.35 MB/sec execute 591 sec latency 52.755 ms 2024-03-23T18:03:27.931 INFO:tasks.workunit.client.0.smithi086.stdout: 1 776190 6.34 MB/sec execute 592 sec latency 877.018 ms 2024-03-23T18:03:28.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:03:27 smithi086 ceph-mon[38221]: pgmap v402: 97 pgs: 97 active+clean; 307 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 943 KiB/s rd, 2.1 MiB/s wr, 185 op/s 2024-03-23T18:03:28.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:03:27 smithi092 ceph-mon[79089]: pgmap v402: 97 pgs: 97 active+clean; 307 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 943 KiB/s rd, 2.1 MiB/s wr, 185 op/s 2024-03-23T18:03:28.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:03:27 smithi152 ceph-mon[47949]: pgmap v402: 97 pgs: 97 active+clean; 307 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 943 KiB/s rd, 2.1 MiB/s wr, 185 op/s 2024-03-23T18:03:28.931 INFO:tasks.workunit.client.0.smithi086.stdout: 1 777870 6.35 MB/sec execute 593 sec latency 19.981 ms 2024-03-23T18:03:29.931 INFO:tasks.workunit.client.0.smithi086.stdout: 1 779096 6.35 MB/sec execute 594 sec latency 20.665 ms 2024-03-23T18:03:30.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:03:29 smithi086 ceph-mon[38221]: pgmap v403: 97 pgs: 97 active+clean; 307 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 840 KiB/s rd, 1.9 MiB/s wr, 165 op/s 2024-03-23T18:03:30.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:03:29 smithi092 ceph-mon[79089]: pgmap v403: 97 pgs: 97 active+clean; 307 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 840 KiB/s rd, 1.9 MiB/s wr, 165 op/s 2024-03-23T18:03:30.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:03:29 smithi152 ceph-mon[47949]: pgmap v403: 97 pgs: 97 active+clean; 307 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 840 KiB/s rd, 1.9 MiB/s wr, 165 op/s 2024-03-23T18:03:30.931 INFO:tasks.workunit.client.0.smithi086.stdout: 1 779285 6.34 MB/sec execute 595 sec latency 862.897 ms 2024-03-23T18:03:31.931 INFO:tasks.workunit.client.0.smithi086.stdout: 1 781294 6.35 MB/sec execute 596 sec latency 16.575 ms 2024-03-23T18:03:32.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:03:31 smithi086 ceph-mon[38221]: pgmap v404: 97 pgs: 97 active+clean; 310 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 251 op/s 2024-03-23T18:03:32.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:03:31 smithi092 ceph-mon[79089]: pgmap v404: 97 pgs: 97 active+clean; 310 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 251 op/s 2024-03-23T18:03:32.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:03:31 smithi152 ceph-mon[47949]: pgmap v404: 97 pgs: 97 active+clean; 310 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 251 op/s 2024-03-23T18:03:32.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.630+0000 7f99bda00700 1 -- 172.21.15.86:0/715218348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99b8101890 msgr2=0x7f99b81058d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:03:32.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.630+0000 7f99bda00700 1 --2- 172.21.15.86:0/715218348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99b8101890 0x7f99b81058d0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f99a4009990 tx=0x7f99a4009ca0 comp rx=0 tx=0).stop 2024-03-23T18:03:32.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.630+0000 7f99bda00700 1 -- 172.21.15.86:0/715218348 shutdown_connections 2024-03-23T18:03:32.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.630+0000 7f99bda00700 1 --2- 172.21.15.86:0/715218348 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99b8106000 0x7f99b810db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:32.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.630+0000 7f99bda00700 1 --2- 172.21.15.86:0/715218348 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f99b8100ee0 0x7f99b81012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:32.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.630+0000 7f99bda00700 1 --2- 172.21.15.86:0/715218348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99b8101890 0x7f99b81058d0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:32.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.630+0000 7f99bda00700 1 -- 172.21.15.86:0/715218348 >> 172.21.15.86:0/715218348 conn(0x7f99b80fc780 msgr2=0x7f99b80feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:03:32.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.630+0000 7f99bda00700 1 -- 172.21.15.86:0/715218348 shutdown_connections 2024-03-23T18:03:32.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.630+0000 7f99bda00700 1 -- 172.21.15.86:0/715218348 wait complete. 2024-03-23T18:03:32.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.631+0000 7f99bda00700 1 Processor -- start 2024-03-23T18:03:32.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.631+0000 7f99bda00700 1 -- start start 2024-03-23T18:03:32.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.632+0000 7f99bda00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f99b8100ee0 0x7f99b80730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:32.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.632+0000 7f99bda00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99b8106000 0x7f99b80735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:32.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.632+0000 7f99bda00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99b8073b30 0x7f99b81a5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:32.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.632+0000 7f99bda00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99b81a59e0 con 0x7f99b8073b30 2024-03-23T18:03:32.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.632+0000 7f99bda00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99b81a5b50 con 0x7f99b8106000 2024-03-23T18:03:32.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.632+0000 7f99bda00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99b81a5d10 con 0x7f99b8100ee0 2024-03-23T18:03:32.634 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.632+0000 7f99b6000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99b8106000 0x7f99b80735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:32.634 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.632+0000 7f99b6000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99b8106000 0x7f99b80735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:46758/0 (socket says 172.21.15.86:46758) 2024-03-23T18:03:32.634 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.632+0000 7f99b7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99b8073b30 0x7f99b81a5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:32.634 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.632+0000 7f99b6000700 1 -- 172.21.15.86:0/2043790569 learned_addr learned my addr 172.21.15.86:0/2043790569 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:03:32.634 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.632+0000 7f99b6a00700 1 --2- 172.21.15.86:0/2043790569 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f99b8100ee0 0x7f99b80730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:32.634 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.632+0000 7f99b6000700 1 -- 172.21.15.86:0/2043790569 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f99b8100ee0 msgr2=0x7f99b80730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:03:32.634 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.632+0000 7f99b6000700 1 --2- 172.21.15.86:0/2043790569 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f99b8100ee0 0x7f99b80730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:32.634 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.632+0000 7f99b6000700 1 -- 172.21.15.86:0/2043790569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99b8073b30 msgr2=0x7f99b81a5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:03:32.634 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.632+0000 7f99b6000700 1 --2- 172.21.15.86:0/2043790569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99b8073b30 0x7f99b81a5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:32.634 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.632+0000 7f99b6000700 1 -- 172.21.15.86:0/2043790569 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99b0009c40 con 0x7f99b8106000 2024-03-23T18:03:32.634 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.633+0000 7f99b6000700 1 --2- 172.21.15.86:0/2043790569 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99b8106000 0x7f99b80735f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f99a4000c00 tx=0x7f99a4005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:03:32.634 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.633+0000 7f99af400700 1 -- 172.21.15.86:0/2043790569 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a401d070 con 0x7f99b8106000 2024-03-23T18:03:32.634 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.633+0000 7f99af400700 1 -- 172.21.15.86:0/2043790569 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99a4022470 con 0x7f99b8106000 2024-03-23T18:03:32.635 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.633+0000 7f99bda00700 1 -- 172.21.15.86:0/2043790569 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99a40095f0 con 0x7f99b8106000 2024-03-23T18:03:32.635 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.633+0000 7f99af400700 1 -- 172.21.15.86:0/2043790569 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a4017650 con 0x7f99b8106000 2024-03-23T18:03:32.635 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.633+0000 7f99bda00700 1 -- 172.21.15.86:0/2043790569 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99b81a6220 con 0x7f99b8106000 2024-03-23T18:03:32.637 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.637+0000 7f99af400700 1 -- 172.21.15.86:0/2043790569 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f99a400fad0 con 0x7f99b8106000 2024-03-23T18:03:32.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.638+0000 7f99d1531b80 1 Processor -- start 2024-03-23T18:03:32.639 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.639+0000 7f99d1531b80 1 -- start start 2024-03-23T18:03:32.639 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.639+0000 7f99d1531b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fd7d0ca100 0x55fd7d0ca4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:32.639 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.639+0000 7f99d1531b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fd7d0caa20 0x55fd7d0d5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:32.639 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.639+0000 7f99af400700 1 --2- 172.21.15.86:0/2043790569 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f999c0a1310 0x7f999c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:32.639 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.639+0000 7f99d1531b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd7d0d58d0 0x55fd7d0d7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:32.639 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.639+0000 7f99d1531b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd7d023930 con 0x55fd7d0d58d0 2024-03-23T18:03:32.639 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.639+0000 7f99d1531b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd7d0212f0 con 0x55fd7d0ca100 2024-03-23T18:03:32.639 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.639+0000 7f99d1531b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd7d01cbe0 con 0x55fd7d0caa20 2024-03-23T18:03:32.640 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.639+0000 7f99af400700 1 -- 172.21.15.86:0/2043790569 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f99a40cdf10 con 0x7f99b8106000 2024-03-23T18:03:32.641 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.639+0000 7f99b6a00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fd7d0caa20 0x55fd7d0d5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:32.642 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.639+0000 7f99b7400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fd7d0ca100 0x55fd7d0ca4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:32.642 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.639+0000 7f99b6a00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fd7d0caa20 0x55fd7d0d5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:38250/0 (socket says 172.21.15.86:38250) 2024-03-23T18:03:32.642 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.639+0000 7f99b6a00700 1 -- 172.21.15.86:0/4034530513 learned_addr learned my addr 172.21.15.86:0/4034530513 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:03:32.642 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.639+0000 7f99b7400700 1 --2- 172.21.15.86:0/2043790569 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f999c0a1310 0x7f999c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:32.642 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.640+0000 7f99b6a00700 1 -- 172.21.15.86:0/4034530513 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fd7d0ca100 msgr2=0x55fd7d0ca4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:03:32.642 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.640+0000 7f99b6a00700 1 --2- 172.21.15.86:0/4034530513 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fd7d0ca100 0x55fd7d0ca4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:32.642 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.640+0000 7f99b6a00700 1 -- 172.21.15.86:0/4034530513 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd7d0d58d0 msgr2=0x55fd7d0d7cc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-23T18:03:32.642 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.640+0000 7f99b6a00700 1 --2- 172.21.15.86:0/4034530513 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd7d0d58d0 0x55fd7d0d7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:32.642 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.640+0000 7f99b6a00700 1 -- 172.21.15.86:0/4034530513 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd7ce9e350 con 0x55fd7d0caa20 2024-03-23T18:03:32.642 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.640+0000 7f99b6a00700 1 --2- 172.21.15.86:0/4034530513 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fd7d0caa20 0x55fd7d0d5390 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f99a8009d80 tx=0x7f99a8011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce53eeda97803eca server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:03:32.642 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.640+0000 7f99b7400700 1 --2- 172.21.15.86:0/2043790569 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f999c0a1310 0x7f999c0a37d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f99b0009fd0 tx=0x7f99b000b040 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:03:32.642 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.641+0000 7f99a3e00700 1 -- 172.21.15.86:0/4034530513 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a8004230 con 0x55fd7d0caa20 2024-03-23T18:03:32.642 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.641+0000 7f99a3e00700 1 -- 172.21.15.86:0/4034530513 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99a8004390 con 0x55fd7d0caa20 2024-03-23T18:03:32.643 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.641+0000 7f99a3e00700 1 -- 172.21.15.86:0/4034530513 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a8019880 con 0x55fd7d0caa20 2024-03-23T18:03:32.643 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.641+0000 7f99d1531b80 1 -- 172.21.15.86:0/4034530513 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fd7d0caa20 msgr2=0x55fd7d0d5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:03:32.643 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.641+0000 7f99d1531b80 1 --2- 172.21.15.86:0/4034530513 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fd7d0caa20 0x55fd7d0d5390 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f99a8009d80 tx=0x7f99a8011720 comp rx=0 tx=0).stop 2024-03-23T18:03:32.643 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.641+0000 7f99d1531b80 1 -- 172.21.15.86:0/4034530513 shutdown_connections 2024-03-23T18:03:32.643 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.641+0000 7f99d1531b80 1 --2- 172.21.15.86:0/4034530513 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd7d0d58d0 0x55fd7d0d7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:32.643 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.641+0000 7f99d1531b80 1 --2- 172.21.15.86:0/4034530513 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fd7d0ca100 0x55fd7d0ca4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:32.643 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.641+0000 7f99d1531b80 1 --2- 172.21.15.86:0/4034530513 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fd7d0caa20 0x55fd7d0d5390 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:32.643 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.641+0000 7f99d1531b80 1 -- 172.21.15.86:0/4034530513 >> 172.21.15.86:0/4034530513 conn(0x55fd7d02a640 msgr2=0x55fd7d0c6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:03:32.643 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.642+0000 7f99d1531b80 1 -- 172.21.15.86:0/4034530513 shutdown_connections 2024-03-23T18:03:32.643 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.642+0000 7f99d1531b80 1 -- 172.21.15.86:0/4034530513 wait complete. 2024-03-23T18:03:32.646 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.642+0000 7f99d1531b80 1 -- start start 2024-03-23T18:03:32.646 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.642+0000 7f99d1531b80 1 Processor -- start 2024-03-23T18:03:32.646 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.643+0000 7f99d1531b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fd7d0ca100 0x55fd7d0d52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:32.646 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.643+0000 7f99d1531b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fd7d0caa20 0x55fd7d0d3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:32.646 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.643+0000 7f99d1531b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd7d0d58d0 0x55fd7d0d44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:32.646 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.643+0000 7f99d1531b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd7cfb8d20 con 0x55fd7d0d58d0 2024-03-23T18:03:32.647 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.643+0000 7f99d1531b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd7cfb89c0 con 0x55fd7d0ca100 2024-03-23T18:03:32.647 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.643+0000 7f99d1531b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd7cea0640 con 0x55fd7d0caa20 2024-03-23T18:03:32.647 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.643+0000 7f99d1531b80 10 client.0 _ll_register_callbacks cb 0x55fd7cfc2410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T18:03:32.647 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.643+0000 7f99b6a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fd7d0ca100 0x55fd7d0d52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:32.647 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.643+0000 7f99b6a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fd7d0ca100 0x55fd7d0d52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:46772/0 (socket says 172.21.15.86:46772) 2024-03-23T18:03:32.647 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.643+0000 7f99b6a00700 1 -- 172.21.15.86:0/196221030 learned_addr learned my addr 172.21.15.86:0/196221030 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:03:32.647 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.643+0000 7f99b7400700 1 --2- 172.21.15.86:0/196221030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd7d0d58d0 0x55fd7d0d44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:32.647 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.643+0000 7f99b6000700 1 --2- 172.21.15.86:0/196221030 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fd7d0caa20 0x55fd7d0d3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:32.647 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.644+0000 7f99b6a00700 1 -- 172.21.15.86:0/196221030 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fd7d0caa20 msgr2=0x55fd7d0d3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:03:32.647 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.644+0000 7f99b6a00700 1 --2- 172.21.15.86:0/196221030 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fd7d0caa20 0x55fd7d0d3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:32.647 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.644+0000 7f99b6a00700 1 -- 172.21.15.86:0/196221030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd7d0d58d0 msgr2=0x55fd7d0d44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:03:32.647 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.644+0000 7f99b6a00700 1 --2- 172.21.15.86:0/196221030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd7d0d58d0 0x55fd7d0d44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:32.647 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.644+0000 7f99b6a00700 1 -- 172.21.15.86:0/196221030 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99a80099e0 con 0x55fd7d0ca100 2024-03-23T18:03:32.648 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.644+0000 7f99b6000700 1 --2- 172.21.15.86:0/196221030 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fd7d0caa20 0x55fd7d0d3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T18:03:32.648 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.644+0000 7f99b7400700 1 --2- 172.21.15.86:0/196221030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd7d0d58d0 0x55fd7d0d44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T18:03:32.648 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.644+0000 7f99b6a00700 1 --2- 172.21.15.86:0/196221030 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fd7d0ca100 0x55fd7d0d52b0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f99a8005950 tx=0x7f99a80038b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23675a749f2a6aca server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:03:32.648 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.644+0000 7f99a0c00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T18:03:32.648 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.645+0000 7f99a0c00700 1 -- 172.21.15.86:0/196221030 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a8005240 con 0x55fd7d0ca100 2024-03-23T18:03:32.648 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.645+0000 7f99a0c00700 1 -- 172.21.15.86:0/196221030 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f99a8005ce0 con 0x55fd7d0ca100 2024-03-23T18:03:32.648 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.645+0000 7f99d1531b80 1 -- 172.21.15.86:0/196221030 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fd7cfdd320 con 0x55fd7d0ca100 2024-03-23T18:03:32.648 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.645+0000 7f99a0c00700 1 -- 172.21.15.86:0/196221030 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99a8009ed0 con 0x55fd7d0ca100 2024-03-23T18:03:32.648 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.645+0000 7f99d1531b80 10 client.27074 fetch_fsmap learned FSMap version 53 2024-03-23T18:03:32.648 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.645+0000 7f99d1531b80 1 -- 172.21.15.86:0/196221030 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x55fd7ce4ab00 con 0x55fd7d0ca100 2024-03-23T18:03:32.648 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.645+0000 7f99a0c00700 1 -- 172.21.15.86:0/196221030 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fd7cfdd320 con 0x55fd7d0ca100 2024-03-23T18:03:32.648 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.645+0000 7f99a0c00700 1 -- 172.21.15.86:0/196221030 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 53) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f99a8021040 con 0x55fd7d0ca100 2024-03-23T18:03:32.648 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.645+0000 7f99d1531b80 10 client.27074 fetch_fsmap finished waiting for FSMap version 53 2024-03-23T18:03:32.648 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.645+0000 7f99d1531b80 10 client.27074 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T18:03:32.649 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.646+0000 7f99d1531b80 20 client.27074 populate_metadata read hostname 'smithi086' 2024-03-23T18:03:32.649 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.646+0000 7f99d1531b80 1 --2- 172.21.15.86:0/196221030 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55fd7d0e5500 0x55fd7d0e7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:32.649 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.646+0000 7f99d1531b80 4 client.27074 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T18:03:32.649 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.646+0000 7f99d1531b80 1 -- 172.21.15.86:0/196221030 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fd7d0156a0 con 0x55fd7d0e5500 2024-03-23T18:03:32.649 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.646+0000 7f99b6000700 1 --2- 172.21.15.86:0/196221030 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55fd7d0e5500 0x55fd7d0e7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:32.649 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.646+0000 7f99b6000700 1 --2- 172.21.15.86:0/196221030 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55fd7d0e5500 0x55fd7d0e7920 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e309127645157320 server_cookie=51983b587c60cb60 in_seq=0 out_seq=0 2024-03-23T18:03:32.649 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.646+0000 7f99a0c00700 10 client.27074 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T18:03:32.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.647+0000 7f99a0c00700 1 -- 172.21.15.86:0/196221030 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55fd7d0156a0 con 0x55fd7d0e5500 2024-03-23T18:03:32.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.647+0000 7f99a0c00700 10 client.27074 handle_command_reply: tid=0 2024-03-23T18:03:32.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.647+0000 7f99d1531b80 1 client.27074 shutdown 2024-03-23T18:03:32.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.648+0000 7f99d1531b80 1 -- 172.21.15.86:0/196221030 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fd7d0ca100 msgr2=0x55fd7d0d52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:03:32.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.648+0000 7f99d1531b80 1 --2- 172.21.15.86:0/196221030 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fd7d0ca100 0x55fd7d0d52b0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f99a8005950 tx=0x7f99a80038b0 comp rx=0 tx=0).stop 2024-03-23T18:03:32.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.648+0000 7f99d1531b80 1 -- 172.21.15.86:0/196221030 shutdown_connections 2024-03-23T18:03:32.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.648+0000 7f99d1531b80 1 --2- 172.21.15.86:0/196221030 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55fd7d0e5500 0x55fd7d0e7920 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:32.651 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.648+0000 7f99d1531b80 1 --2- 172.21.15.86:0/196221030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd7d0d58d0 0x55fd7d0d44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:32.651 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.648+0000 7f99d1531b80 1 --2- 172.21.15.86:0/196221030 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fd7d0ca100 0x55fd7d0d52b0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:32.651 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.648+0000 7f99d1531b80 1 --2- 172.21.15.86:0/196221030 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fd7d0caa20 0x55fd7d0d3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:32.651 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.648+0000 7f99d1531b80 1 -- 172.21.15.86:0/196221030 >> 172.21.15.86:0/196221030 conn(0x55fd7d02a640 msgr2=0x55fd7d0cc0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:03:32.651 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.648+0000 7f99a0c00700 0 client.27074 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T18:03:32.651 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.648+0000 7f99d1531b80 1 -- 172.21.15.86:0/196221030 shutdown_connections 2024-03-23T18:03:32.651 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.648+0000 7f99d1531b80 1 -- 172.21.15.86:0/196221030 wait complete. 2024-03-23T18:03:32.651 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.648+0000 7f99d1531b80 20 client.27074 trim_cache size 0 max 16384 2024-03-23T18:03:32.667 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.667+0000 7f99d1531b80 1 Processor -- start 2024-03-23T18:03:32.667 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.667+0000 7f99d1531b80 1 -- start start 2024-03-23T18:03:32.667 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.667+0000 7f99d1531b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fd7d0ca100 0x55fd7d0ec240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:32.668 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.667+0000 7f99d1531b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd7d0caa20 0x55fd7d0dc610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:32.668 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.667+0000 7f99d1531b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fd7d0d58d0 0x55fd7d0cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:32.668 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.667+0000 7f99d1531b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd7d0cae40 con 0x55fd7d0caa20 2024-03-23T18:03:32.668 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.667+0000 7f99d1531b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd7d0d0cb0 con 0x55fd7d0ca100 2024-03-23T18:03:32.668 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.667+0000 7f99d1531b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd7d0cbb90 con 0x55fd7d0d58d0 2024-03-23T18:03:32.668 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.667+0000 7f99b6a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fd7d0ca100 0x55fd7d0ec240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:32.668 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.667+0000 7f99b6000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd7d0caa20 0x55fd7d0dc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:32.668 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.667+0000 7f99b7400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fd7d0d58d0 0x55fd7d0cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:32.668 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.667+0000 7f99b6a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fd7d0ca100 0x55fd7d0ec240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:46778/0 (socket says 172.21.15.86:46778) 2024-03-23T18:03:32.668 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.667+0000 7f99b6a00700 1 -- 172.21.15.86:0/3949192276 learned_addr learned my addr 172.21.15.86:0/3949192276 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:03:32.672 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.668+0000 7f99b6a00700 1 -- 172.21.15.86:0/3949192276 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fd7d0d58d0 msgr2=0x55fd7d0cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:03:32.673 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.668+0000 7f99b6a00700 1 --2- 172.21.15.86:0/3949192276 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fd7d0d58d0 0x55fd7d0cceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:32.673 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.668+0000 7f99b6a00700 1 -- 172.21.15.86:0/3949192276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd7d0caa20 msgr2=0x55fd7d0dc610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:03:32.673 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.668+0000 7f99b6a00700 1 --2- 172.21.15.86:0/3949192276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd7d0caa20 0x55fd7d0dc610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:32.673 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.668+0000 7f99b6a00700 1 -- 172.21.15.86:0/3949192276 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd7cfe2e40 con 0x55fd7d0ca100 2024-03-23T18:03:32.673 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.668+0000 7f99b6000700 1 --2- 172.21.15.86:0/3949192276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd7d0caa20 0x55fd7d0dc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T18:03:32.673 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.668+0000 7f99b6a00700 1 --2- 172.21.15.86:0/3949192276 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fd7d0ca100 0x55fd7d0ec240 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f99a800e570 tx=0x7f99a8019560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f65d7f060f0a9894 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:03:32.673 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.668+0000 7f99acc00700 1 -- 172.21.15.86:0/3949192276 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a8009ed0 con 0x55fd7d0ca100 2024-03-23T18:03:32.673 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.668+0000 7f99acc00700 1 -- 172.21.15.86:0/3949192276 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99a8019680 con 0x55fd7d0ca100 2024-03-23T18:03:32.673 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.668+0000 7f99acc00700 1 -- 172.21.15.86:0/3949192276 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a8005350 con 0x55fd7d0ca100 2024-03-23T18:03:32.673 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.669+0000 7f99d1531b80 1 -- 172.21.15.86:0/3949192276 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fd7d0ca100 msgr2=0x55fd7d0ec240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:03:32.673 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.669+0000 7f99d1531b80 1 --2- 172.21.15.86:0/3949192276 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fd7d0ca100 0x55fd7d0ec240 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f99a800e570 tx=0x7f99a8019560 comp rx=0 tx=0).stop 2024-03-23T18:03:32.673 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.669+0000 7f99d1531b80 1 -- 172.21.15.86:0/3949192276 shutdown_connections 2024-03-23T18:03:32.674 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.669+0000 7f99d1531b80 1 --2- 172.21.15.86:0/3949192276 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fd7d0d58d0 0x55fd7d0cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:32.674 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.669+0000 7f99d1531b80 1 --2- 172.21.15.86:0/3949192276 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fd7d0ca100 0x55fd7d0ec240 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:32.674 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.669+0000 7f99d1531b80 1 --2- 172.21.15.86:0/3949192276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd7d0caa20 0x55fd7d0dc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:32.674 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.669+0000 7f99d1531b80 1 -- 172.21.15.86:0/3949192276 >> 172.21.15.86:0/3949192276 conn(0x55fd7d02a640 msgr2=0x55fd7d0d5030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:03:32.674 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.669+0000 7f99d1531b80 1 -- 172.21.15.86:0/3949192276 shutdown_connections 2024-03-23T18:03:32.674 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.669+0000 7f99d1531b80 1 -- 172.21.15.86:0/3949192276 wait complete. 2024-03-23T18:03:32.674 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.669+0000 7f99d1531b80 1 -- start start 2024-03-23T18:03:32.674 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.670+0000 7f99d1531b80 1 Processor -- start 2024-03-23T18:03:32.674 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.670+0000 7f99d1531b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fd7d0ca100 0x55fd7d0dc670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:32.674 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.670+0000 7f99d1531b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fd7d0caa20 0x55fd7d0d9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:32.674 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.670+0000 7f99d1531b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd7d0d58d0 0x55fd7d0da220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:32.674 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.670+0000 7f99d1531b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd7d0da900 con 0x55fd7d0d58d0 2024-03-23T18:03:32.675 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.670+0000 7f99d1531b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd7d135fe0 con 0x55fd7d0ca100 2024-03-23T18:03:32.675 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.670+0000 7f99d1531b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd7d136120 con 0x55fd7d0caa20 2024-03-23T18:03:32.675 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.670+0000 7f99d1531b80 10 client.0 _ll_register_callbacks cb 0x55fd7cfc2410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T18:03:32.675 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.670+0000 7f99b6a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fd7d0ca100 0x55fd7d0dc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:32.675 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.671+0000 7f99b6000700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fd7d0caa20 0x55fd7d0d9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:32.675 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.671+0000 7f99b6a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fd7d0ca100 0x55fd7d0dc670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:46780/0 (socket says 172.21.15.86:46780) 2024-03-23T18:03:32.675 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.671+0000 7f99b6a00700 1 -- 172.21.15.86:0/3973490339 learned_addr learned my addr 172.21.15.86:0/3973490339 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:03:32.675 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.671+0000 7f99b7400700 1 --2- 172.21.15.86:0/3973490339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd7d0d58d0 0x55fd7d0da220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:32.675 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.671+0000 7f99b6000700 1 -- 172.21.15.86:0/3973490339 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fd7d0ca100 msgr2=0x55fd7d0dc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:03:32.675 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.671+0000 7f99b6000700 1 --2- 172.21.15.86:0/3973490339 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fd7d0ca100 0x55fd7d0dc670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:32.675 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.671+0000 7f99b6000700 1 -- 172.21.15.86:0/3973490339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd7d0d58d0 msgr2=0x55fd7d0da220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:03:32.675 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.671+0000 7f99b6000700 1 --2- 172.21.15.86:0/3973490339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd7d0d58d0 0x55fd7d0da220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:32.676 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.671+0000 7f99b6000700 1 -- 172.21.15.86:0/3973490339 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99b81a6220 con 0x55fd7d0caa20 2024-03-23T18:03:32.676 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.671+0000 7f99b7400700 1 --2- 172.21.15.86:0/3973490339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd7d0d58d0 0x55fd7d0da220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T18:03:32.676 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.671+0000 7f99b6a00700 1 --2- 172.21.15.86:0/3973490339 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fd7d0ca100 0x55fd7d0dc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T18:03:32.676 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.671+0000 7f99b6000700 1 --2- 172.21.15.86:0/3973490339 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fd7d0caa20 0x55fd7d0d9ae0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f99a40cb680 tx=0x7f99a4022c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18ada9f0002d232 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:03:32.676 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.671+0000 7f99a0c00700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-03-23T18:03:32.676 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.671+0000 7f99a0c00700 1 -- 172.21.15.86:0/3973490339 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a409ab50 con 0x55fd7d0caa20 2024-03-23T18:03:32.676 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.672+0000 7f99a0c00700 1 -- 172.21.15.86:0/3973490339 <== mon.1 v2:172.21.15.152:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f99a409b8f0 con 0x55fd7d0caa20 2024-03-23T18:03:32.676 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.672+0000 7f99d1531b80 1 -- 172.21.15.86:0/3973490339 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f99a8004390 con 0x55fd7d0caa20 2024-03-23T18:03:32.676 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.672+0000 7f99a0c00700 1 -- 172.21.15.86:0/3973490339 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99a409b1f0 con 0x55fd7d0caa20 2024-03-23T18:03:32.676 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.672+0000 7f99d1531b80 10 client.27055 fetch_fsmap learned FSMap version 53 2024-03-23T18:03:32.676 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.672+0000 7f99d1531b80 1 -- 172.21.15.86:0/3973490339 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x55fd7ce4ab00 con 0x55fd7d0caa20 2024-03-23T18:03:32.676 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.672+0000 7f99a0c00700 1 -- 172.21.15.86:0/3973490339 <== mon.1 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99a8004390 con 0x55fd7d0caa20 2024-03-23T18:03:32.676 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.672+0000 7f99a0c00700 1 -- 172.21.15.86:0/3973490339 <== mon.1 v2:172.21.15.152:3300/0 5 ==== fsmap(e 53) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f99a409b450 con 0x55fd7d0caa20 2024-03-23T18:03:32.677 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.672+0000 7f99d1531b80 10 client.27055 fetch_fsmap finished waiting for FSMap version 53 2024-03-23T18:03:32.677 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.672+0000 7f99d1531b80 10 client.27055 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T18:03:32.677 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.672+0000 7f99d1531b80 20 client.27055 populate_metadata read hostname 'smithi086' 2024-03-23T18:03:32.677 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.672+0000 7f99d1531b80 1 --2- 172.21.15.86:0/3973490339 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55fd7d1374a0 0x55fd7d1398c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:32.677 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.672+0000 7f99d1531b80 4 client.27055 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T18:03:32.677 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.672+0000 7f99d1531b80 1 -- 172.21.15.86:0/3973490339 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fd7ce6bc90 con 0x55fd7d1374a0 2024-03-23T18:03:32.677 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.673+0000 7f99b6a00700 1 --2- 172.21.15.86:0/3973490339 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55fd7d1374a0 0x55fd7d1398c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:32.677 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.673+0000 7f99b6a00700 1 --2- 172.21.15.86:0/3973490339 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55fd7d1374a0 0x55fd7d1398c0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1fc905e50e52ca4 server_cookie=8d43e00a11463d56 in_seq=0 out_seq=0 2024-03-23T18:03:32.677 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.673+0000 7f99a0c00700 10 client.27055 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T18:03:32.725 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.724+0000 7f99a0c00700 1 -- 172.21.15.86:0/3973490339 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fd7ce6bc90 con 0x55fd7d1374a0 2024-03-23T18:03:32.725 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.724+0000 7f99a0c00700 10 client.27055 handle_command_reply: tid=0 2024-03-23T18:03:32.725 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.724+0000 7f99d1531b80 1 client.27055 shutdown 2024-03-23T18:03:32.725 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.724+0000 7f99d1531b80 1 -- 172.21.15.86:0/3973490339 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fd7d0caa20 msgr2=0x55fd7d0d9ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:03:32.725 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.724+0000 7f99d1531b80 1 --2- 172.21.15.86:0/3973490339 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fd7d0caa20 0x55fd7d0d9ae0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f99a40cb680 tx=0x7f99a4022c40 comp rx=0 tx=0).stop 2024-03-23T18:03:32.725 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.724+0000 7f99d1531b80 1 -- 172.21.15.86:0/3973490339 shutdown_connections 2024-03-23T18:03:32.725 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.724+0000 7f99d1531b80 1 --2- 172.21.15.86:0/3973490339 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55fd7d1374a0 0x55fd7d1398c0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:32.725 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.724+0000 7f99d1531b80 1 --2- 172.21.15.86:0/3973490339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fd7d0d58d0 0x55fd7d0da220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:32.726 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.724+0000 7f99d1531b80 1 --2- 172.21.15.86:0/3973490339 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fd7d0ca100 0x55fd7d0dc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:32.726 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.724+0000 7f99d1531b80 1 --2- 172.21.15.86:0/3973490339 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fd7d0caa20 0x55fd7d0d9ae0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:32.726 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.724+0000 7f99d1531b80 1 -- 172.21.15.86:0/3973490339 >> 172.21.15.86:0/3973490339 conn(0x55fd7d02a640 msgr2=0x55fd7d0bf3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:03:32.726 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.725+0000 7f99a0c00700 0 client.27055 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T18:03:32.726 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.725+0000 7f99d1531b80 1 -- 172.21.15.86:0/3973490339 shutdown_connections 2024-03-23T18:03:32.726 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.725+0000 7f99d1531b80 1 -- 172.21.15.86:0/3973490339 wait complete. 2024-03-23T18:03:32.726 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.725+0000 7f99d1531b80 20 client.27055 trim_cache size 0 max 16384 2024-03-23T18:03:32.726 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-03-23T18:03:32.727 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-03-23T18:03:32.727 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-03-23T18:03:32.727 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-03-23T18:03:32.727 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.726+0000 7f99bda00700 1 -- 172.21.15.86:0/2043790569 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f999c0a1310 msgr2=0x7f999c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:03:32.727 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.726+0000 7f99bda00700 1 --2- 172.21.15.86:0/2043790569 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f999c0a1310 0x7f999c0a37d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f99b0009fd0 tx=0x7f99b000b040 comp rx=0 tx=0).stop 2024-03-23T18:03:32.728 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.726+0000 7f99bda00700 1 -- 172.21.15.86:0/2043790569 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99b8106000 msgr2=0x7f99b80735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:03:32.728 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.726+0000 7f99bda00700 1 --2- 172.21.15.86:0/2043790569 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99b8106000 0x7f99b80735f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f99a4000c00 tx=0x7f99a4005740 comp rx=0 tx=0).stop 2024-03-23T18:03:32.728 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.726+0000 7f99bda00700 1 -- 172.21.15.86:0/2043790569 shutdown_connections 2024-03-23T18:03:32.728 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.726+0000 7f99bda00700 1 --2- 172.21.15.86:0/2043790569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99b8073b30 0x7f99b81a5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:32.728 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.726+0000 7f99bda00700 1 --2- 172.21.15.86:0/2043790569 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f999c0a1310 0x7f999c0a37d0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:32.728 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.726+0000 7f99bda00700 1 --2- 172.21.15.86:0/2043790569 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f99b8100ee0 0x7f99b80730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:32.728 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.726+0000 7f99bda00700 1 --2- 172.21.15.86:0/2043790569 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99b8106000 0x7f99b80735f0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:32.728 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.726+0000 7f99bda00700 1 -- 172.21.15.86:0/2043790569 >> 172.21.15.86:0/2043790569 conn(0x7f99b80fc780 msgr2=0x7f99b80fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:03:32.728 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.726+0000 7f99bda00700 1 -- 172.21.15.86:0/2043790569 shutdown_connections 2024-03-23T18:03:32.728 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:32.726+0000 7f99bda00700 1 -- 172.21.15.86:0/2043790569 wait complete. 2024-03-23T18:03:32.931 INFO:tasks.workunit.client.0.smithi086.stdout: 1 782326 6.35 MB/sec execute 597 sec latency 21.691 ms 2024-03-23T18:03:33.931 INFO:tasks.workunit.client.0.smithi086.stdout: 1 783568 6.35 MB/sec execute 598 sec latency 379.813 ms 2024-03-23T18:03:34.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:03:33 smithi086 ceph-mon[38221]: pgmap v405: 97 pgs: 97 active+clean; 310 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 861 KiB/s rd, 1.9 MiB/s wr, 149 op/s 2024-03-23T18:03:34.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:03:33 smithi092 ceph-mon[79089]: pgmap v405: 97 pgs: 97 active+clean; 310 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 861 KiB/s rd, 1.9 MiB/s wr, 149 op/s 2024-03-23T18:03:34.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:03:33 smithi152 ceph-mon[47949]: pgmap v405: 97 pgs: 97 active+clean; 310 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 861 KiB/s rd, 1.9 MiB/s wr, 149 op/s 2024-03-23T18:03:34.931 INFO:tasks.workunit.client.0.smithi086.stdout: 1 785063 6.36 MB/sec execute 599 sec latency 15.014 ms 2024-03-23T18:03:35.931 INFO:tasks.workunit.client.0.smithi086.stdout: 1 cleanup 600 sec 2024-03-23T18:03:36.118 INFO:tasks.workunit.client.0.smithi086.stdout: 0 cleanup 600 sec 2024-03-23T18:03:36.118 INFO:tasks.workunit.client.0.smithi086.stdout: 2024-03-23T18:03:36.118 INFO:tasks.workunit.client.0.smithi086.stdout: Operation Count AvgLat MaxLat 2024-03-23T18:03:36.118 INFO:tasks.workunit.client.0.smithi086.stdout: ---------------------------------------- 2024-03-23T18:03:36.119 INFO:tasks.workunit.client.0.smithi086.stdout: NTCreateX 121709 2.067 4961.434 2024-03-23T18:03:36.119 INFO:tasks.workunit.client.0.smithi086.stdout: Close 89406 0.004 0.493 2024-03-23T18:03:36.119 INFO:tasks.workunit.client.0.smithi086.stdout: Rename 5158 22.696 4655.407 2024-03-23T18:03:36.119 INFO:tasks.workunit.client.0.smithi086.stdout: Unlink 24574 1.922 8008.524 2024-03-23T18:03:36.119 INFO:tasks.workunit.client.0.smithi086.stdout: Deltree 2 94.725 136.647 2024-03-23T18:03:36.119 INFO:tasks.workunit.client.0.smithi086.stdout: Mkdir 1 0.015 0.015 2024-03-23T18:03:36.119 INFO:tasks.workunit.client.0.smithi086.stdout: Qpathinfo 110337 1.180 225.467 2024-03-23T18:03:36.119 INFO:tasks.workunit.client.0.smithi086.stdout: Qfileinfo 19305 0.002 0.036 2024-03-23T18:03:36.119 INFO:tasks.workunit.client.0.smithi086.stdout: Qfsinfo 20226 0.271 201.735 2024-03-23T18:03:36.122 INFO:tasks.workunit.client.0.smithi086.stdout: Sfileinfo 9931 0.033 7.315 2024-03-23T18:03:36.123 INFO:tasks.workunit.client.0.smithi086.stdout: Find 42651 0.351 4675.153 2024-03-23T18:03:36.123 INFO:tasks.workunit.client.0.smithi086.stdout: WriteX 60506 0.087 252.007 2024-03-23T18:03:36.123 INFO:tasks.workunit.client.0.smithi086.stdout: ReadX 190818 0.012 46.045 2024-03-23T18:03:36.123 INFO:tasks.workunit.client.0.smithi086.stdout: LockX 396 0.604 1.688 2024-03-23T18:03:36.123 INFO:tasks.workunit.client.0.smithi086.stdout: UnlockX 396 0.561 2.469 2024-03-23T18:03:36.123 INFO:tasks.workunit.client.0.smithi086.stdout: Flush 8535 2.960 54.714 2024-03-23T18:03:36.123 INFO:tasks.workunit.client.0.smithi086.stdout: 2024-03-23T18:03:36.123 INFO:tasks.workunit.client.0.smithi086.stdout:Throughput 6.35525 MB/sec 1 clients 1 procs max_latency=8008.540 ms 2024-03-23T18:03:36.124 INFO:tasks.workunit.client.0.smithi086.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-03-23T18:03:36.124 INFO:tasks.workunit.client.0.smithi086.stdout: 2024-03-23T18:03:36.124 INFO:tasks.workunit.client.0.smithi086.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-03-23T18:03:36.124 INFO:tasks.workunit.client.0.smithi086.stdout:9 of 10 processes prepared for launch 0 sec 2024-03-23T18:03:36.223 INFO:tasks.workunit.client.0.smithi086.stdout:10 of 10 processes prepared for launch 0 sec 2024-03-23T18:03:36.223 INFO:tasks.workunit.client.0.smithi086.stdout:releasing clients 2024-03-23T18:03:36.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:03:35 smithi086 ceph-mon[38221]: pgmap v406: 97 pgs: 97 active+clean; 311 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 242 op/s 2024-03-23T18:03:36.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:03:35 smithi086 ceph-mon[38221]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:03:36.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:03:35 smithi092 ceph-mon[79089]: pgmap v406: 97 pgs: 97 active+clean; 311 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 242 op/s 2024-03-23T18:03:36.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:03:35 smithi092 ceph-mon[79089]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:03:36.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:03:35 smithi152 ceph-mon[47949]: pgmap v406: 97 pgs: 97 active+clean; 311 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 242 op/s 2024-03-23T18:03:36.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:03:35 smithi152 ceph-mon[47949]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:03:37.223 INFO:tasks.workunit.client.0.smithi086.stdout: 10 412 145.79 MB/sec warmup 1 sec latency 44.748 ms 2024-03-23T18:03:38.223 INFO:tasks.workunit.client.0.smithi086.stdout: 10 873 105.99 MB/sec warmup 2 sec latency 82.733 ms 2024-03-23T18:03:38.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:03:37 smithi086 ceph-mon[38221]: pgmap v407: 97 pgs: 97 active+clean; 311 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 912 KiB/s rd, 2.0 MiB/s wr, 179 op/s 2024-03-23T18:03:38.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:03:37 smithi092 ceph-mon[79089]: pgmap v407: 97 pgs: 97 active+clean; 311 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 912 KiB/s rd, 2.0 MiB/s wr, 179 op/s 2024-03-23T18:03:40.253 INFO:tasks.workunit.client.0.smithi086.stdout: 10 1470 85.51 MB/sec warmup 3 sec latency 82.762 ms 2024-03-23T18:03:40.254 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:03:38 smithi092 ceph-mon[79089]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:03:40.254 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:03:38 smithi086 ceph-mon[38221]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:03:40.254 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:03:37 smithi152 ceph-mon[47949]: pgmap v407: 97 pgs: 97 active+clean; 311 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 912 KiB/s rd, 2.0 MiB/s wr, 179 op/s 2024-03-23T18:03:40.254 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:03:38 smithi152 ceph-mon[47949]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:03:40.254 INFO:tasks.workunit.client.0.smithi086.stdout: 10 2341 69.90 MB/sec warmup 4 sec latency 55.222 ms 2024-03-23T18:03:40.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:03:39 smithi086 ceph-mon[38221]: pgmap v408: 97 pgs: 97 active+clean; 311 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 912 KiB/s rd, 2.0 MiB/s wr, 179 op/s 2024-03-23T18:03:40.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:03:39 smithi086 ceph-mon[38221]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:03:40.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:03:39 smithi092 ceph-mon[79089]: pgmap v408: 97 pgs: 97 active+clean; 311 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 912 KiB/s rd, 2.0 MiB/s wr, 179 op/s 2024-03-23T18:03:40.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:03:39 smithi092 ceph-mon[79089]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:03:40.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:03:39 smithi152 ceph-mon[47949]: pgmap v408: 97 pgs: 97 active+clean; 311 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 912 KiB/s rd, 2.0 MiB/s wr, 179 op/s 2024-03-23T18:03:40.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:03:39 smithi152 ceph-mon[47949]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:03:41.223 INFO:tasks.workunit.client.0.smithi086.stdout: 10 3019 66.67 MB/sec warmup 5 sec latency 132.596 ms 2024-03-23T18:03:42.223 INFO:tasks.workunit.client.0.smithi086.stdout: 10 3551 63.38 MB/sec warmup 6 sec latency 492.992 ms 2024-03-23T18:03:42.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:03:41 smithi086 ceph-mon[38221]: pgmap v409: 97 pgs: 97 active+clean; 321 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.7 MiB/s wr, 320 op/s 2024-03-23T18:03:42.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:03:41 smithi086 ceph-mon[38221]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:03:42.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:03:41 smithi092 ceph-mon[79089]: pgmap v409: 97 pgs: 97 active+clean; 321 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.7 MiB/s wr, 320 op/s 2024-03-23T18:03:42.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:03:41 smithi092 ceph-mon[79089]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:03:42.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:03:41 smithi152 ceph-mon[47949]: pgmap v409: 97 pgs: 97 active+clean; 321 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.7 MiB/s wr, 320 op/s 2024-03-23T18:03:42.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:03:41 smithi152 ceph-mon[47949]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:03:43.223 INFO:tasks.workunit.client.0.smithi086.stdout: 10 3732 55.28 MB/sec warmup 7 sec latency 1493.054 ms 2024-03-23T18:03:44.224 INFO:tasks.workunit.client.0.smithi086.stdout: 10 3914 48.40 MB/sec warmup 8 sec latency 2155.429 ms 2024-03-23T18:03:44.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:03:43 smithi086 ceph-mon[38221]: pgmap v410: 97 pgs: 97 active+clean; 321 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 234 op/s 2024-03-23T18:03:44.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:03:43 smithi092 ceph-mon[79089]: pgmap v410: 97 pgs: 97 active+clean; 321 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 234 op/s 2024-03-23T18:03:44.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:03:43 smithi152 ceph-mon[47949]: pgmap v410: 97 pgs: 97 active+clean; 321 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 234 op/s 2024-03-23T18:03:45.223 INFO:tasks.workunit.client.0.smithi086.stdout: 10 4271 43.93 MB/sec warmup 9 sec latency 170.966 ms 2024-03-23T18:03:45.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:03:44 smithi086 ceph-mon[38221]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:03:45.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:03:44 smithi092 ceph-mon[79089]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:03:45.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:03:44 smithi152 ceph-mon[47949]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:03:45.897 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.896+0000 7fa26c200700 1 -- 172.21.15.86:0/2879845097 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa264107db0 msgr2=0x7fa264108190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:03:45.897 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.896+0000 7fa26c200700 1 --2- 172.21.15.86:0/2879845097 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa264107db0 0x7fa264108190 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa25800b0a0 tx=0x7fa25800b3b0 comp rx=0 tx=0).stop 2024-03-23T18:03:45.897 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.897+0000 7fa26c200700 1 -- 172.21.15.86:0/2879845097 shutdown_connections 2024-03-23T18:03:45.897 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.897+0000 7fa26c200700 1 --2- 172.21.15.86:0/2879845097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa264113f00 0x7fa2641162f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:45.897 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.897+0000 7fa26c200700 1 --2- 172.21.15.86:0/2879845097 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa264107db0 0x7fa264108190 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:45.897 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.897+0000 7fa26c200700 1 --2- 172.21.15.86:0/2879845097 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2641086d0 0x7fa2641139c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:45.897 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.897+0000 7fa26c200700 1 -- 172.21.15.86:0/2879845097 >> 172.21.15.86:0/2879845097 conn(0x7fa26406bd80 msgr2=0x7fa26406c190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:03:45.898 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.897+0000 7fa26c200700 1 -- 172.21.15.86:0/2879845097 shutdown_connections 2024-03-23T18:03:45.898 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.897+0000 7fa26c200700 1 -- 172.21.15.86:0/2879845097 wait complete. 2024-03-23T18:03:45.898 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.898+0000 7fa26c200700 1 Processor -- start 2024-03-23T18:03:45.898 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.898+0000 7fa26c200700 1 -- start start 2024-03-23T18:03:45.899 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.899+0000 7fa26c200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa264107db0 0x7fa26410ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:45.899 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.899+0000 7fa26c200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2641086d0 0x7fa26410bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:45.899 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.899+0000 7fa26c200700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa264113f00 0x7fa26410c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:45.900 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.899+0000 7fa26c200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2641b22a0 con 0x7fa264107db0 2024-03-23T18:03:45.900 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.899+0000 7fa26c200700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2641b23e0 con 0x7fa2641086d0 2024-03-23T18:03:45.900 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.899+0000 7fa26c200700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2641b2550 con 0x7fa264113f00 2024-03-23T18:03:45.900 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.899+0000 7fa26a200700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa264113f00 0x7fa26410c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:45.900 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.899+0000 7fa268e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2641086d0 0x7fa26410bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:45.900 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.899+0000 7fa268e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2641086d0 0x7fa26410bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:33566/0 (socket says 172.21.15.86:33566) 2024-03-23T18:03:45.900 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.899+0000 7fa26a200700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa264113f00 0x7fa26410c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:53730/0 (socket says 172.21.15.86:53730) 2024-03-23T18:03:45.900 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.899+0000 7fa268e00700 1 -- 172.21.15.86:0/694083587 learned_addr learned my addr 172.21.15.86:0/694083587 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:03:45.900 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.899+0000 7fa269800700 1 --2- 172.21.15.86:0/694083587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa264107db0 0x7fa26410ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:45.901 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.900+0000 7fa26a200700 1 -- 172.21.15.86:0/694083587 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2641086d0 msgr2=0x7fa26410bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:03:45.901 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.900+0000 7fa26a200700 1 --2- 172.21.15.86:0/694083587 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2641086d0 0x7fa26410bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:45.901 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.900+0000 7fa26a200700 1 -- 172.21.15.86:0/694083587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa264107db0 msgr2=0x7fa26410ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:03:45.901 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.900+0000 7fa26a200700 1 --2- 172.21.15.86:0/694083587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa264107db0 0x7fa26410ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:45.902 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.900+0000 7fa26a200700 1 -- 172.21.15.86:0/694083587 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa258009d00 con 0x7fa264113f00 2024-03-23T18:03:45.902 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.900+0000 7fa268e00700 1 --2- 172.21.15.86:0/694083587 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2641086d0 0x7fa26410bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T18:03:45.902 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.900+0000 7fa269800700 1 --2- 172.21.15.86:0/694083587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa264107db0 0x7fa26410ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T18:03:45.902 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.900+0000 7fa26a200700 1 --2- 172.21.15.86:0/694083587 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa264113f00 0x7fa26410c4b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa25c00c940 tx=0x7fa25c00cc50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:03:45.902 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.900+0000 7fa262000700 1 -- 172.21.15.86:0/694083587 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa25c00e880 con 0x7fa264113f00 2024-03-23T18:03:45.902 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.901+0000 7fa262000700 1 -- 172.21.15.86:0/694083587 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa25c00f450 con 0x7fa264113f00 2024-03-23T18:03:45.902 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.901+0000 7fa26c200700 1 -- 172.21.15.86:0/694083587 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2641b2830 con 0x7fa264113f00 2024-03-23T18:03:45.902 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.901+0000 7fa262000700 1 -- 172.21.15.86:0/694083587 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa25c00eea0 con 0x7fa264113f00 2024-03-23T18:03:45.902 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.901+0000 7fa26c200700 1 -- 172.21.15.86:0/694083587 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2641b2d90 con 0x7fa264113f00 2024-03-23T18:03:45.904 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.905+0000 7fa262000700 1 -- 172.21.15.86:0/694083587 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fa25c018720 con 0x7fa264113f00 2024-03-23T18:03:45.906 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.906+0000 7fa262000700 1 --2- 172.21.15.86:0/694083587 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fa24c0a1310 0x7fa24c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:45.906 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.906+0000 7fa27fce3b80 1 Processor -- start 2024-03-23T18:03:45.906 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.906+0000 7fa262000700 1 -- 172.21.15.86:0/694083587 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fa25c0cc9c0 con 0x7fa264113f00 2024-03-23T18:03:45.907 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.906+0000 7fa27fce3b80 1 -- start start 2024-03-23T18:03:45.907 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.907+0000 7fa269800700 1 --2- 172.21.15.86:0/694083587 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fa24c0a1310 0x7fa24c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:45.908 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.907+0000 7fa27fce3b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eacb29f100 0x55eacb29f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:45.908 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.907+0000 7fa27fce3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eacb29fa20 0x55eacb2aa390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:45.908 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.907+0000 7fa27fce3b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55eacb2aa8d0 0x55eacb2accc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:45.908 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.907+0000 7fa27fce3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eacb1f8930 con 0x55eacb29fa20 2024-03-23T18:03:45.908 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.907+0000 7fa27fce3b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eacb1f62f0 con 0x55eacb29f100 2024-03-23T18:03:45.908 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.907+0000 7fa27fce3b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eacb1f1be0 con 0x55eacb2aa8d0 2024-03-23T18:03:45.908 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.907+0000 7fa26a200700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55eacb2aa8d0 0x55eacb2accc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:45.908 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.907+0000 7fa269800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eacb29f100 0x55eacb29f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:45.908 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.907+0000 7fa26a200700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55eacb2aa8d0 0x55eacb2accc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:53736/0 (socket says 172.21.15.86:53736) 2024-03-23T18:03:45.908 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.907+0000 7fa26a200700 1 -- 172.21.15.86:0/3861428573 learned_addr learned my addr 172.21.15.86:0/3861428573 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:03:45.908 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.908+0000 7fa268e00700 1 --2- 172.21.15.86:0/3861428573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eacb29fa20 0x55eacb2aa390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:45.908 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.908+0000 7fa26a200700 1 -- 172.21.15.86:0/3861428573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eacb29f100 msgr2=0x55eacb29f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:03:45.909 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.908+0000 7fa26a200700 1 --2- 172.21.15.86:0/3861428573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eacb29f100 0x55eacb29f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:45.909 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.908+0000 7fa26a200700 1 -- 172.21.15.86:0/3861428573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eacb29fa20 msgr2=0x55eacb2aa390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:03:45.909 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.908+0000 7fa26a200700 1 --2- 172.21.15.86:0/3861428573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eacb29fa20 0x55eacb2aa390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:45.909 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.908+0000 7fa269800700 1 --2- 172.21.15.86:0/694083587 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fa24c0a1310 0x7fa24c0a37d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa25800b780 tx=0x7fa258006210 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:03:45.909 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.908+0000 7fa26a200700 1 -- 172.21.15.86:0/3861428573 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eacb073350 con 0x55eacb2aa8d0 2024-03-23T18:03:45.909 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.908+0000 7fa268e00700 1 --2- 172.21.15.86:0/3861428573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eacb29fa20 0x55eacb2aa390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T18:03:45.909 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.908+0000 7fa26a200700 1 --2- 172.21.15.86:0/3861428573 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55eacb2aa8d0 0x55eacb2accc0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa25c0a1a40 tx=0x7fa25c0996b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8261b94fb24a646 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:03:45.910 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.908+0000 7fa256a00700 1 -- 172.21.15.86:0/3861428573 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa25c099af0 con 0x55eacb2aa8d0 2024-03-23T18:03:45.910 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.908+0000 7fa256a00700 1 -- 172.21.15.86:0/3861428573 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa25c09d920 con 0x55eacb2aa8d0 2024-03-23T18:03:45.910 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.908+0000 7fa256a00700 1 -- 172.21.15.86:0/3861428573 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa25c085d40 con 0x55eacb2aa8d0 2024-03-23T18:03:45.910 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.909+0000 7fa27fce3b80 1 -- 172.21.15.86:0/3861428573 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55eacb2aa8d0 msgr2=0x55eacb2accc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:03:45.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.909+0000 7fa27fce3b80 1 --2- 172.21.15.86:0/3861428573 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55eacb2aa8d0 0x55eacb2accc0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa25c0a1a40 tx=0x7fa25c0996b0 comp rx=0 tx=0).stop 2024-03-23T18:03:45.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.909+0000 7fa27fce3b80 1 -- 172.21.15.86:0/3861428573 shutdown_connections 2024-03-23T18:03:45.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.909+0000 7fa27fce3b80 1 --2- 172.21.15.86:0/3861428573 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55eacb2aa8d0 0x55eacb2accc0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:45.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.909+0000 7fa27fce3b80 1 --2- 172.21.15.86:0/3861428573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eacb29f100 0x55eacb29f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:45.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.909+0000 7fa27fce3b80 1 --2- 172.21.15.86:0/3861428573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eacb29fa20 0x55eacb2aa390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:45.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.909+0000 7fa27fce3b80 1 -- 172.21.15.86:0/3861428573 >> 172.21.15.86:0/3861428573 conn(0x55eacb1ff640 msgr2=0x55eacb29b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:03:45.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.909+0000 7fa27fce3b80 1 -- 172.21.15.86:0/3861428573 shutdown_connections 2024-03-23T18:03:45.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.909+0000 7fa27fce3b80 1 -- 172.21.15.86:0/3861428573 wait complete. 2024-03-23T18:03:45.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.910+0000 7fa27fce3b80 1 -- start start 2024-03-23T18:03:45.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.910+0000 7fa27fce3b80 1 Processor -- start 2024-03-23T18:03:45.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.911+0000 7fa27fce3b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eacb29f100 0x55eacb2aa2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:45.913 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.911+0000 7fa27fce3b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55eacb29fa20 0x55eacb2a7690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:45.913 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.911+0000 7fa27fce3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eacb2aa8d0 0x55eacb2a7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:45.914 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.911+0000 7fa27fce3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eacb18dd20 con 0x55eacb2aa8d0 2024-03-23T18:03:45.914 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.911+0000 7fa27fce3b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eacb18d9c0 con 0x55eacb29f100 2024-03-23T18:03:45.914 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.911+0000 7fa27fce3b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eacb075640 con 0x55eacb29fa20 2024-03-23T18:03:45.914 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.911+0000 7fa27fce3b80 10 client.0 _ll_register_callbacks cb 0x55eacb197410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T18:03:45.914 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.911+0000 7fa269800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eacb29f100 0x55eacb2aa2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:45.914 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.911+0000 7fa268e00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55eacb29fa20 0x55eacb2a7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:45.914 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.911+0000 7fa268e00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55eacb29fa20 0x55eacb2a7690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:53738/0 (socket says 172.21.15.86:53738) 2024-03-23T18:03:45.914 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.911+0000 7fa268e00700 1 -- 172.21.15.86:0/2125858073 learned_addr learned my addr 172.21.15.86:0/2125858073 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:03:45.914 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.911+0000 7fa26a200700 1 --2- 172.21.15.86:0/2125858073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eacb2aa8d0 0x55eacb2a7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:45.914 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.911+0000 7fa269800700 1 -- 172.21.15.86:0/2125858073 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55eacb29fa20 msgr2=0x55eacb2a7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:03:45.914 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.911+0000 7fa269800700 1 --2- 172.21.15.86:0/2125858073 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55eacb29fa20 0x55eacb2a7690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:45.915 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.912+0000 7fa269800700 1 -- 172.21.15.86:0/2125858073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eacb2aa8d0 msgr2=0x55eacb2a7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:03:45.915 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.912+0000 7fa269800700 1 --2- 172.21.15.86:0/2125858073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eacb2aa8d0 0x55eacb2a7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:45.915 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.912+0000 7fa269800700 1 -- 172.21.15.86:0/2125858073 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2641b2d90 con 0x55eacb29f100 2024-03-23T18:03:45.915 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.912+0000 7fa26a200700 1 --2- 172.21.15.86:0/2125858073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eacb2aa8d0 0x55eacb2a7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T18:03:45.915 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.913+0000 7fa269800700 1 --2- 172.21.15.86:0/2125858073 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eacb29f100 0x55eacb2aa2e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa2580062a0 tx=0x7fa2580099e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e12f1888312e686e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:03:45.915 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.913+0000 7fa24b400700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T18:03:45.915 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.913+0000 7fa24b400700 1 -- 172.21.15.86:0/2125858073 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa25800e040 con 0x55eacb29f100 2024-03-23T18:03:45.915 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.913+0000 7fa24b400700 1 -- 172.21.15.86:0/2125858073 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fa2580054d0 con 0x55eacb29f100 2024-03-23T18:03:45.915 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.913+0000 7fa27fce3b80 1 -- 172.21.15.86:0/2125858073 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55eacb1b2320 con 0x55eacb29f100 2024-03-23T18:03:45.915 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.913+0000 7fa24b400700 1 -- 172.21.15.86:0/2125858073 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa258005690 con 0x55eacb29f100 2024-03-23T18:03:45.915 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.913+0000 7fa27fce3b80 10 client.27092 fetch_fsmap learned FSMap version 58 2024-03-23T18:03:45.915 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.913+0000 7fa27fce3b80 1 -- 172.21.15.86:0/2125858073 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x55eacb01fb00 con 0x55eacb29f100 2024-03-23T18:03:45.915 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.914+0000 7fa24b400700 1 -- 172.21.15.86:0/2125858073 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55eacb1b2320 con 0x55eacb29f100 2024-03-23T18:03:45.915 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.914+0000 7fa24b400700 1 -- 172.21.15.86:0/2125858073 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 58) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fa258028450 con 0x55eacb29f100 2024-03-23T18:03:45.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.914+0000 7fa27fce3b80 10 client.27092 fetch_fsmap finished waiting for FSMap version 58 2024-03-23T18:03:45.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.914+0000 7fa27fce3b80 10 client.27092 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T18:03:45.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.914+0000 7fa27fce3b80 20 client.27092 populate_metadata read hostname 'smithi086' 2024-03-23T18:03:45.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.914+0000 7fa27fce3b80 1 --2- 172.21.15.86:0/2125858073 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55eacb2ba3e0 0x55eacb2bc800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:45.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.914+0000 7fa27fce3b80 4 client.27092 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T18:03:45.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.914+0000 7fa27fce3b80 1 -- 172.21.15.86:0/2125858073 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eacb1ea6a0 con 0x55eacb2ba3e0 2024-03-23T18:03:45.917 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.915+0000 7fa268e00700 1 --2- 172.21.15.86:0/2125858073 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55eacb2ba3e0 0x55eacb2bc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:45.917 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.916+0000 7fa268e00700 1 --2- 172.21.15.86:0/2125858073 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55eacb2ba3e0 0x55eacb2bc800 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c19db2b72a0ca4fd server_cookie=7d761b45b41a7d2c in_seq=0 out_seq=0 2024-03-23T18:03:45.917 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.916+0000 7fa24b400700 10 client.27092 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T18:03:45.917 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.917+0000 7fa24b400700 1 -- 172.21.15.86:0/2125858073 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55eacb1ea6a0 con 0x55eacb2ba3e0 2024-03-23T18:03:45.917 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.917+0000 7fa24b400700 10 client.27092 handle_command_reply: tid=0 2024-03-23T18:03:45.918 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.917+0000 7fa27fce3b80 1 client.27092 shutdown 2024-03-23T18:03:45.918 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.918+0000 7fa27fce3b80 1 -- 172.21.15.86:0/2125858073 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eacb29f100 msgr2=0x55eacb2aa2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:03:45.918 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.918+0000 7fa27fce3b80 1 --2- 172.21.15.86:0/2125858073 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eacb29f100 0x55eacb2aa2e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa2580062a0 tx=0x7fa2580099e0 comp rx=0 tx=0).stop 2024-03-23T18:03:45.918 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.918+0000 7fa27fce3b80 1 -- 172.21.15.86:0/2125858073 shutdown_connections 2024-03-23T18:03:45.918 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.918+0000 7fa27fce3b80 1 --2- 172.21.15.86:0/2125858073 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55eacb2ba3e0 0x55eacb2bc800 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:45.918 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.918+0000 7fa27fce3b80 1 --2- 172.21.15.86:0/2125858073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eacb2aa8d0 0x55eacb2a7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:45.918 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.918+0000 7fa27fce3b80 1 --2- 172.21.15.86:0/2125858073 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eacb29f100 0x55eacb2aa2e0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:45.918 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.918+0000 7fa27fce3b80 1 --2- 172.21.15.86:0/2125858073 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55eacb29fa20 0x55eacb2a7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:45.918 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.918+0000 7fa24b400700 0 client.27092 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T18:03:45.918 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.918+0000 7fa27fce3b80 1 -- 172.21.15.86:0/2125858073 >> 172.21.15.86:0/2125858073 conn(0x55eacb1ff640 msgr2=0x55eacb2a10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:03:45.919 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.918+0000 7fa27fce3b80 1 -- 172.21.15.86:0/2125858073 shutdown_connections 2024-03-23T18:03:45.919 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.918+0000 7fa27fce3b80 1 -- 172.21.15.86:0/2125858073 wait complete. 2024-03-23T18:03:45.919 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.918+0000 7fa27fce3b80 20 client.27092 trim_cache size 0 max 16384 2024-03-23T18:03:45.935 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.934+0000 7fa27fce3b80 1 Processor -- start 2024-03-23T18:03:45.936 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.934+0000 7fa27fce3b80 1 -- start start 2024-03-23T18:03:45.936 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.935+0000 7fa27fce3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eacb29f100 0x55eacb2b1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:45.936 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.935+0000 7fa27fce3b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55eacb29fa20 0x55eacb2accc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:45.936 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.935+0000 7fa27fce3b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eacb2aa8d0 0x55eacb2a10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:45.936 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.935+0000 7fa27fce3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eacb29fe00 con 0x55eacb29f100 2024-03-23T18:03:45.936 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.935+0000 7fa27fce3b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eacb2a0bc0 con 0x55eacb2aa8d0 2024-03-23T18:03:45.936 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.935+0000 7fa27fce3b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eacb2a3510 con 0x55eacb29fa20 2024-03-23T18:03:45.936 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.935+0000 7fa269800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eacb29f100 0x55eacb2b1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:45.936 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.935+0000 7fa26a200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eacb2aa8d0 0x55eacb2a10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:45.936 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.935+0000 7fa269800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eacb29f100 0x55eacb2b1480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54232/0 (socket says 172.21.15.86:54232) 2024-03-23T18:03:45.936 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.935+0000 7fa269800700 1 -- 172.21.15.86:0/1409706055 learned_addr learned my addr 172.21.15.86:0/1409706055 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:03:45.936 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.935+0000 7fa268e00700 1 --2- 172.21.15.86:0/1409706055 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55eacb29fa20 0x55eacb2accc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:45.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.935+0000 7fa26a200700 1 -- 172.21.15.86:0/1409706055 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55eacb29fa20 msgr2=0x55eacb2accc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:03:45.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.935+0000 7fa26a200700 1 --2- 172.21.15.86:0/1409706055 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55eacb29fa20 0x55eacb2accc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:45.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.935+0000 7fa26a200700 1 -- 172.21.15.86:0/1409706055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eacb29f100 msgr2=0x55eacb2b1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:03:45.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.935+0000 7fa26a200700 1 --2- 172.21.15.86:0/1409706055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eacb29f100 0x55eacb2b1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:45.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.935+0000 7fa26a200700 1 -- 172.21.15.86:0/1409706055 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eacb1b7e40 con 0x55eacb2aa8d0 2024-03-23T18:03:45.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.935+0000 7fa269800700 1 --2- 172.21.15.86:0/1409706055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eacb29f100 0x55eacb2b1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T18:03:45.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.936+0000 7fa26a200700 1 --2- 172.21.15.86:0/1409706055 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eacb2aa8d0 0x55eacb2a10a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fa25c0ca1e0 tx=0x7fa25c095f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be369d906e6d4d92 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:03:45.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.936+0000 7fa257400700 1 -- 172.21.15.86:0/1409706055 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa25c0a1f70 con 0x55eacb2aa8d0 2024-03-23T18:03:45.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.936+0000 7fa257400700 1 -- 172.21.15.86:0/1409706055 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa25c097e90 con 0x55eacb2aa8d0 2024-03-23T18:03:45.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.936+0000 7fa257400700 1 -- 172.21.15.86:0/1409706055 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa25c0175c0 con 0x55eacb2aa8d0 2024-03-23T18:03:45.940 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.937+0000 7fa27fce3b80 1 -- 172.21.15.86:0/1409706055 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eacb2aa8d0 msgr2=0x55eacb2a10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:03:45.940 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.937+0000 7fa27fce3b80 1 --2- 172.21.15.86:0/1409706055 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eacb2aa8d0 0x55eacb2a10a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fa25c0ca1e0 tx=0x7fa25c095f60 comp rx=0 tx=0).stop 2024-03-23T18:03:45.941 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.937+0000 7fa27fce3b80 1 -- 172.21.15.86:0/1409706055 shutdown_connections 2024-03-23T18:03:45.941 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.937+0000 7fa27fce3b80 1 --2- 172.21.15.86:0/1409706055 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eacb2aa8d0 0x55eacb2a10a0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:45.941 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.937+0000 7fa27fce3b80 1 --2- 172.21.15.86:0/1409706055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eacb29f100 0x55eacb2b1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:45.941 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.937+0000 7fa27fce3b80 1 --2- 172.21.15.86:0/1409706055 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55eacb29fa20 0x55eacb2accc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:45.941 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.937+0000 7fa27fce3b80 1 -- 172.21.15.86:0/1409706055 >> 172.21.15.86:0/1409706055 conn(0x55eacb1ff640 msgr2=0x55eacb2a5f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:03:45.941 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.937+0000 7fa27fce3b80 1 -- 172.21.15.86:0/1409706055 shutdown_connections 2024-03-23T18:03:45.941 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.937+0000 7fa27fce3b80 1 -- 172.21.15.86:0/1409706055 wait complete. 2024-03-23T18:03:45.941 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.937+0000 7fa27fce3b80 1 -- start start 2024-03-23T18:03:45.942 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.937+0000 7fa27fce3b80 1 Processor -- start 2024-03-23T18:03:45.942 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.938+0000 7fa27fce3b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55eacb29f100 0x55eacb2af650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:45.942 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.938+0000 7fa27fce3b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eacb29fa20 0x55eacb2afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:45.942 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.938+0000 7fa27fce3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eacb2aa8d0 0x55eacb30aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:45.942 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.938+0000 7fa27fce3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eacb30b400 con 0x55eacb2aa8d0 2024-03-23T18:03:45.942 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.938+0000 7fa27fce3b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eacb30b540 con 0x55eacb29fa20 2024-03-23T18:03:45.942 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.938+0000 7fa27fce3b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eacb30b6d0 con 0x55eacb29f100 2024-03-23T18:03:45.942 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.938+0000 7fa27fce3b80 10 client.0 _ll_register_callbacks cb 0x55eacb197410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T18:03:45.942 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.938+0000 7fa269800700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55eacb29f100 0x55eacb2af650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:45.942 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.938+0000 7fa268e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eacb29fa20 0x55eacb2afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:45.942 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.938+0000 7fa269800700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55eacb29f100 0x55eacb2af650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:53750/0 (socket says 172.21.15.86:53750) 2024-03-23T18:03:45.942 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.938+0000 7fa269800700 1 -- 172.21.15.86:0/1287044890 learned_addr learned my addr 172.21.15.86:0/1287044890 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:03:45.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.938+0000 7fa26a200700 1 --2- 172.21.15.86:0/1287044890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eacb2aa8d0 0x55eacb30aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:45.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.938+0000 7fa269800700 1 -- 172.21.15.86:0/1287044890 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eacb29fa20 msgr2=0x55eacb2afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:03:45.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.938+0000 7fa269800700 1 --2- 172.21.15.86:0/1287044890 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eacb29fa20 0x55eacb2afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:45.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.938+0000 7fa269800700 1 -- 172.21.15.86:0/1287044890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eacb2aa8d0 msgr2=0x55eacb30aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:03:45.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.938+0000 7fa269800700 1 --2- 172.21.15.86:0/1287044890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eacb2aa8d0 0x55eacb30aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:45.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.938+0000 7fa269800700 1 -- 172.21.15.86:0/1287044890 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eacb01fb00 con 0x55eacb29f100 2024-03-23T18:03:45.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.939+0000 7fa269800700 1 --2- 172.21.15.86:0/1287044890 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55eacb29f100 0x55eacb2af650 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa258028c10 tx=0x7fa258028c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c51d5e9001e4604e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:03:45.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.939+0000 7fa24b400700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-03-23T18:03:45.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.939+0000 7fa24b400700 1 -- 172.21.15.86:0/1287044890 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa25800e040 con 0x55eacb29f100 2024-03-23T18:03:45.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.939+0000 7fa24b400700 1 -- 172.21.15.86:0/1287044890 <== mon.1 v2:172.21.15.152:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fa2580054e0 con 0x55eacb29f100 2024-03-23T18:03:45.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.939+0000 7fa27fce3b80 1 -- 172.21.15.86:0/1287044890 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa25c09d920 con 0x55eacb29f100 2024-03-23T18:03:45.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.939+0000 7fa24b400700 1 -- 172.21.15.86:0/1287044890 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa258003ea0 con 0x55eacb29f100 2024-03-23T18:03:45.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.939+0000 7fa27fce3b80 10 client.27073 fetch_fsmap learned FSMap version 58 2024-03-23T18:03:45.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.939+0000 7fa27fce3b80 1 -- 172.21.15.86:0/1287044890 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x55eacb073350 con 0x55eacb29f100 2024-03-23T18:03:45.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.939+0000 7fa24b400700 1 -- 172.21.15.86:0/1287044890 <== mon.1 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa25c09d920 con 0x55eacb29f100 2024-03-23T18:03:45.944 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.939+0000 7fa24b400700 1 -- 172.21.15.86:0/1287044890 <== mon.1 v2:172.21.15.152:3300/0 5 ==== fsmap(e 58) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fa258039c30 con 0x55eacb29f100 2024-03-23T18:03:45.944 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.940+0000 7fa27fce3b80 10 client.27073 fetch_fsmap finished waiting for FSMap version 58 2024-03-23T18:03:45.944 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.940+0000 7fa27fce3b80 10 client.27073 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T18:03:45.944 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.940+0000 7fa27fce3b80 20 client.27073 populate_metadata read hostname 'smithi086' 2024-03-23T18:03:45.944 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.940+0000 7fa27fce3b80 1 --2- 172.21.15.86:0/1287044890 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55eacb30cbe0 0x55eacb30f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:45.944 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.940+0000 7fa27fce3b80 4 client.27073 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T18:03:45.944 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.940+0000 7fa27fce3b80 1 -- 172.21.15.86:0/1287044890 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55eacb040c90 con 0x55eacb30cbe0 2024-03-23T18:03:45.944 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.940+0000 7fa268e00700 1 --2- 172.21.15.86:0/1287044890 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55eacb30cbe0 0x55eacb30f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:45.944 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.940+0000 7fa24b400700 10 client.27073 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T18:03:45.944 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:45.940+0000 7fa268e00700 1 --2- 172.21.15.86:0/1287044890 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55eacb30cbe0 0x55eacb30f000 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a2a4bf6f51a195c server_cookie=311f33192a2e59c in_seq=0 out_seq=0 2024-03-23T18:03:46.224 INFO:tasks.workunit.client.0.smithi086.stdout: 10 4768 43.53 MB/sec warmup 10 sec latency 170.473 ms 2024-03-23T18:03:46.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:03:45 smithi086 ceph-mon[38221]: pgmap v411: 97 pgs: 97 active+clean; 334 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 339 op/s 2024-03-23T18:03:46.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:03:45 smithi092 ceph-mon[79089]: pgmap v411: 97 pgs: 97 active+clean; 334 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 339 op/s 2024-03-23T18:03:46.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:03:45 smithi152 ceph-mon[47949]: pgmap v411: 97 pgs: 97 active+clean; 334 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 339 op/s 2024-03-23T18:03:46.411 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:46.412+0000 7fa24b400700 1 -- 172.21.15.86:0/1287044890 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55eacb040c90 con 0x55eacb30cbe0 2024-03-23T18:03:46.412 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:46.412+0000 7fa24b400700 10 client.27073 handle_command_reply: tid=0 2024-03-23T18:03:46.412 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:46.412+0000 7fa27fce3b80 1 client.27073 shutdown 2024-03-23T18:03:46.413 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:46.412+0000 7fa27fce3b80 1 -- 172.21.15.86:0/1287044890 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55eacb29f100 msgr2=0x55eacb2af650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:03:46.413 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:46.412+0000 7fa27fce3b80 1 --2- 172.21.15.86:0/1287044890 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55eacb29f100 0x55eacb2af650 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa258028c10 tx=0x7fa258028c40 comp rx=0 tx=0).stop 2024-03-23T18:03:46.413 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:46.412+0000 7fa27fce3b80 1 -- 172.21.15.86:0/1287044890 shutdown_connections 2024-03-23T18:03:46.413 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:46.412+0000 7fa27fce3b80 1 --2- 172.21.15.86:0/1287044890 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55eacb30cbe0 0x55eacb30f000 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:46.413 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:46.412+0000 7fa27fce3b80 1 --2- 172.21.15.86:0/1287044890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eacb2aa8d0 0x55eacb30aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:46.413 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:46.412+0000 7fa24b400700 0 client.27073 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T18:03:46.413 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:46.412+0000 7fa27fce3b80 1 --2- 172.21.15.86:0/1287044890 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55eacb29f100 0x55eacb2af650 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:46.413 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:46.412+0000 7fa27fce3b80 1 --2- 172.21.15.86:0/1287044890 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eacb29fa20 0x55eacb2afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:46.413 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:46.412+0000 7fa27fce3b80 1 -- 172.21.15.86:0/1287044890 >> 172.21.15.86:0/1287044890 conn(0x55eacb1ff640 msgr2=0x55eacb2943e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:03:46.413 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:46.412+0000 7fa27fce3b80 1 -- 172.21.15.86:0/1287044890 shutdown_connections 2024-03-23T18:03:46.413 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:46.412+0000 7fa27fce3b80 1 -- 172.21.15.86:0/1287044890 wait complete. 2024-03-23T18:03:46.413 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:46.413+0000 7fa27fce3b80 20 client.27073 trim_cache size 0 max 16384 2024-03-23T18:03:46.414 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-03-23T18:03:46.414 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-03-23T18:03:46.414 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-03-23T18:03:46.414 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-03-23T18:03:46.415 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:46.413+0000 7fa26c200700 1 -- 172.21.15.86:0/694083587 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fa24c0a1310 msgr2=0x7fa24c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:03:46.415 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:46.413+0000 7fa26c200700 1 --2- 172.21.15.86:0/694083587 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fa24c0a1310 0x7fa24c0a37d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa25800b780 tx=0x7fa258006210 comp rx=0 tx=0).stop 2024-03-23T18:03:46.415 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:46.413+0000 7fa26c200700 1 -- 172.21.15.86:0/694083587 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa264113f00 msgr2=0x7fa26410c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:03:46.415 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:46.413+0000 7fa26c200700 1 --2- 172.21.15.86:0/694083587 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa264113f00 0x7fa26410c4b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa25c00c940 tx=0x7fa25c00cc50 comp rx=0 tx=0).stop 2024-03-23T18:03:46.415 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:46.414+0000 7fa26c200700 1 -- 172.21.15.86:0/694083587 shutdown_connections 2024-03-23T18:03:46.415 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:46.414+0000 7fa26c200700 1 --2- 172.21.15.86:0/694083587 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fa24c0a1310 0x7fa24c0a37d0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:46.415 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:46.414+0000 7fa26c200700 1 --2- 172.21.15.86:0/694083587 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa264113f00 0x7fa26410c4b0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:46.415 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:46.414+0000 7fa26c200700 1 --2- 172.21.15.86:0/694083587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa264107db0 0x7fa26410ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:46.416 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:46.414+0000 7fa26c200700 1 --2- 172.21.15.86:0/694083587 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2641086d0 0x7fa26410bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:46.416 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:46.414+0000 7fa26c200700 1 -- 172.21.15.86:0/694083587 >> 172.21.15.86:0/694083587 conn(0x7fa26406bd80 msgr2=0x7fa264109750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:03:46.416 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:46.414+0000 7fa26c200700 1 -- 172.21.15.86:0/694083587 shutdown_connections 2024-03-23T18:03:46.416 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:46.414+0000 7fa26c200700 1 -- 172.21.15.86:0/694083587 wait complete. 2024-03-23T18:03:47.008 INFO:journalctl@ceph.mds.c.smithi152.stdout:Mar 23 18:03:46 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-c[84552]: ignoring --setuser ceph since I am not root 2024-03-23T18:03:47.008 INFO:journalctl@ceph.mds.c.smithi152.stdout:Mar 23 18:03:46 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-c[84552]: ignoring --setgroup ceph since I am not root 2024-03-23T18:03:47.008 INFO:journalctl@ceph.mds.c.smithi152.stdout:Mar 23 18:03:46 smithi152 ceph-mds[84575]: -- 172.21.15.152:0/3069556298 <== mon.1 v2:172.21.15.152:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x563bf1e7c1a0 con 0x563bf1e52800 2024-03-23T18:03:47.008 INFO:journalctl@ceph.mds.c.smithi152.stdout:Mar 23 18:03:46 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-c[84552]: starting mds.c at 2024-03-23T18:03:47.009 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:03:47 smithi152 ceph-mon[47949]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:03:47.009 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:03:47 smithi152 ceph-mon[47949]: daemon mds.c restarted 2024-03-23T18:03:47.009 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:03:47 smithi152 ceph-mon[47949]: osdmap e99: 12 total, 12 up, 12 in 2024-03-23T18:03:47.224 INFO:tasks.workunit.client.0.smithi086.stdout: 10 5445 40.88 MB/sec warmup 11 sec latency 219.850 ms 2024-03-23T18:03:47.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:03:47 smithi086 ceph-mon[38221]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:03:47.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:03:47 smithi086 ceph-mon[38221]: daemon mds.c restarted 2024-03-23T18:03:47.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:03:47 smithi086 ceph-mon[38221]: osdmap e99: 12 total, 12 up, 12 in 2024-03-23T18:03:47.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:03:47 smithi092 ceph-mon[79089]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:03:47.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:03:47 smithi092 ceph-mon[79089]: daemon mds.c restarted 2024-03-23T18:03:47.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:03:47 smithi092 ceph-mon[79089]: osdmap e99: 12 total, 12 up, 12 in 2024-03-23T18:03:48.224 INFO:tasks.workunit.client.0.smithi086.stdout: 10 6009 40.55 MB/sec warmup 12 sec latency 67.090 ms 2024-03-23T18:03:48.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:03:48 smithi086 ceph-mon[38221]: pgmap v412: 97 pgs: 97 active+clean; 334 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 246 op/s 2024-03-23T18:03:48.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:03:48 smithi086 ceph-mon[38221]: mds.? [v2:172.21.15.152:6836/503423616,v1:172.21.15.152:6837/503423616] up:boot 2024-03-23T18:03:48.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:03:48 smithi086 ceph-mon[38221]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-23T18:03:48.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:03:48 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-23T18:03:48.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:03:48 smithi086 ceph-mon[38221]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:03:48.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:03:48 smithi092 ceph-mon[79089]: pgmap v412: 97 pgs: 97 active+clean; 334 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 246 op/s 2024-03-23T18:03:48.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:03:48 smithi092 ceph-mon[79089]: mds.? [v2:172.21.15.152:6836/503423616,v1:172.21.15.152:6837/503423616] up:boot 2024-03-23T18:03:48.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:03:48 smithi092 ceph-mon[79089]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-23T18:03:48.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:03:48 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-23T18:03:48.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:03:48 smithi092 ceph-mon[79089]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:03:48.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:03:48 smithi152 ceph-mon[47949]: pgmap v412: 97 pgs: 97 active+clean; 334 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 246 op/s 2024-03-23T18:03:48.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:03:48 smithi152 ceph-mon[47949]: mds.? [v2:172.21.15.152:6836/503423616,v1:172.21.15.152:6837/503423616] up:boot 2024-03-23T18:03:48.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:03:48 smithi152 ceph-mon[47949]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-23T18:03:48.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:03:48 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-23T18:03:48.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:03:48 smithi152 ceph-mon[47949]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:03:49.224 INFO:tasks.workunit.client.0.smithi086.stdout: 10 6698 41.78 MB/sec warmup 13 sec latency 75.111 ms 2024-03-23T18:03:50.224 INFO:tasks.workunit.client.0.smithi086.stdout: 10 7044 40.73 MB/sec warmup 14 sec latency 65.556 ms 2024-03-23T18:03:50.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:03:50 smithi086 ceph-mon[38221]: pgmap v414: 97 pgs: 97 active+clean; 334 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 295 op/s 2024-03-23T18:03:50.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:03:50 smithi092 ceph-mon[79089]: pgmap v414: 97 pgs: 97 active+clean; 334 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 295 op/s 2024-03-23T18:03:50.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:03:50 smithi152 ceph-mon[47949]: pgmap v414: 97 pgs: 97 active+clean; 334 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 295 op/s 2024-03-23T18:03:51.224 INFO:tasks.workunit.client.0.smithi086.stdout: 10 7211 38.20 MB/sec warmup 15 sec latency 75.020 ms 2024-03-23T18:03:52.224 INFO:tasks.workunit.client.0.smithi086.stdout: 10 7421 35.96 MB/sec warmup 16 sec latency 154.274 ms 2024-03-23T18:03:52.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:03:52 smithi086 ceph-mon[38221]: pgmap v415: 97 pgs: 97 active+clean; 330 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.1 MiB/s wr, 595 op/s 2024-03-23T18:03:52.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:03:52 smithi092 ceph-mon[79089]: pgmap v415: 97 pgs: 97 active+clean; 330 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.1 MiB/s wr, 595 op/s 2024-03-23T18:03:52.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:03:52 smithi152 ceph-mon[47949]: pgmap v415: 97 pgs: 97 active+clean; 330 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.1 MiB/s wr, 595 op/s 2024-03-23T18:03:53.224 INFO:tasks.workunit.client.0.smithi086.stdout: 10 7807 34.65 MB/sec warmup 17 sec latency 235.381 ms 2024-03-23T18:03:53.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:03:53 smithi092 ceph-mon[79089]: pgmap v416: 97 pgs: 97 active+clean; 330 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.1 MiB/s wr, 595 op/s 2024-03-23T18:03:53.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:03:53 smithi152 ceph-mon[47949]: pgmap v416: 97 pgs: 97 active+clean; 330 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.1 MiB/s wr, 595 op/s 2024-03-23T18:03:53.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:03:53 smithi086 ceph-mon[38221]: pgmap v416: 97 pgs: 97 active+clean; 330 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.1 MiB/s wr, 595 op/s 2024-03-23T18:03:54.224 INFO:tasks.workunit.client.0.smithi086.stdout: 10 8335 34.70 MB/sec warmup 18 sec latency 177.392 ms 2024-03-23T18:03:55.224 INFO:tasks.workunit.client.0.smithi086.stdout: 10 9074 33.77 MB/sec warmup 19 sec latency 301.378 ms 2024-03-23T18:03:56.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:03:55 smithi086 ceph-mon[38221]: pgmap v417: 97 pgs: 97 active+clean; 397 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 10 MiB/s wr, 648 op/s 2024-03-23T18:03:56.056 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:03:55 smithi092 ceph-mon[79089]: pgmap v417: 97 pgs: 97 active+clean; 397 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 10 MiB/s wr, 648 op/s 2024-03-23T18:03:56.085 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:03:55 smithi152 ceph-mon[47949]: pgmap v417: 97 pgs: 97 active+clean; 397 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 10 MiB/s wr, 648 op/s 2024-03-23T18:03:56.224 INFO:tasks.workunit.client.0.smithi086.stdout: 10 9649 34.51 MB/sec warmup 20 sec latency 109.327 ms 2024-03-23T18:03:57.224 INFO:tasks.workunit.client.0.smithi086.stdout: 10 10252 35.25 MB/sec warmup 21 sec latency 48.962 ms 2024-03-23T18:03:58.224 INFO:tasks.workunit.client.0.smithi086.stdout: 10 10523 34.35 MB/sec warmup 22 sec latency 83.235 ms 2024-03-23T18:03:58.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:03:57 smithi086 ceph-mon[38221]: pgmap v418: 97 pgs: 97 active+clean; 397 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 10 MiB/s wr, 648 op/s 2024-03-23T18:03:58.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:03:57 smithi092 ceph-mon[79089]: pgmap v418: 97 pgs: 97 active+clean; 397 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 10 MiB/s wr, 648 op/s 2024-03-23T18:03:58.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:03:57 smithi152 ceph-mon[47949]: pgmap v418: 97 pgs: 97 active+clean; 397 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 10 MiB/s wr, 648 op/s 2024-03-23T18:03:59.224 INFO:tasks.workunit.client.0.smithi086.stdout: 10 10723 33.16 MB/sec warmup 23 sec latency 127.926 ms 2024-03-23T18:03:59.689 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.688+0000 7f359a200700 1 -- 172.21.15.86:0/1602899101 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f359410f940 msgr2=0x7f35941082f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:03:59.690 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.688+0000 7f359a200700 1 --2- 172.21.15.86:0/1602899101 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f359410f940 0x7f35941082f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f3584009990 tx=0x7f3584009ca0 comp rx=0 tx=0).stop 2024-03-23T18:03:59.690 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.689+0000 7f359a200700 1 -- 172.21.15.86:0/1602899101 shutdown_connections 2024-03-23T18:03:59.690 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.689+0000 7f359a200700 1 --2- 172.21.15.86:0/1602899101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f359411a590 0x7f359411c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:59.690 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.689+0000 7f359a200700 1 --2- 172.21.15.86:0/1602899101 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f359410f4d0 0x7f3594107db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:59.690 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.689+0000 7f359a200700 1 --2- 172.21.15.86:0/1602899101 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f359410f940 0x7f35941082f0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:59.690 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.689+0000 7f359a200700 1 -- 172.21.15.86:0/1602899101 >> 172.21.15.86:0/1602899101 conn(0x7f359406bd80 msgr2=0x7f359406c190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:03:59.694 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.693+0000 7f359a200700 1 -- 172.21.15.86:0/1602899101 shutdown_connections 2024-03-23T18:03:59.695 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.693+0000 7f359a200700 1 -- 172.21.15.86:0/1602899101 wait complete. 2024-03-23T18:03:59.696 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.695+0000 7f359a200700 1 Processor -- start 2024-03-23T18:03:59.696 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.696+0000 7f359a200700 1 -- start start 2024-03-23T18:03:59.698 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.697+0000 7f359a200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f359410f4d0 0x7f3594118920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:59.698 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.697+0000 7f359a200700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f359410f940 0x7f3594118ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:59.698 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.698+0000 7f359a200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f359411a590 0x7f3594111a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:59.700 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.699+0000 7f358aa00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f359410f940 0x7f3594118ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:59.700 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.699+0000 7f359a200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3594112240 con 0x7f359411a590 2024-03-23T18:03:59.700 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.699+0000 7f359a200700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35941123b0 con 0x7f359410f4d0 2024-03-23T18:03:59.700 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.699+0000 7f359a200700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3594112570 con 0x7f359410f940 2024-03-23T18:03:59.701 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.700+0000 7f3593e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f359411a590 0x7f3594111a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:59.701 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.701+0000 7f3593e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f359411a590 0x7f3594111a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56262/0 (socket says 172.21.15.86:56262) 2024-03-23T18:03:59.702 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.701+0000 7f3593e00700 1 -- 172.21.15.86:0/313583008 learned_addr learned my addr 172.21.15.86:0/313583008 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:03:59.702 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.702+0000 7f3593400700 1 --2- 172.21.15.86:0/313583008 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f359410f4d0 0x7f3594118920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:59.703 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.702+0000 7f3593e00700 1 -- 172.21.15.86:0/313583008 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f359410f940 msgr2=0x7f3594118ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:03:59.703 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.702+0000 7f3593e00700 1 --2- 172.21.15.86:0/313583008 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f359410f940 0x7f3594118ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:59.703 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.702+0000 7f3593e00700 1 -- 172.21.15.86:0/313583008 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f359410f4d0 msgr2=0x7f3594118920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:03:59.704 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.702+0000 7f3593e00700 1 --2- 172.21.15.86:0/313583008 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f359410f4d0 0x7f3594118920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:59.704 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.702+0000 7f3593e00700 1 -- 172.21.15.86:0/313583008 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35840095f0 con 0x7f359411a590 2024-03-23T18:03:59.704 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.703+0000 7f3593e00700 1 --2- 172.21.15.86:0/313583008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f359411a590 0x7f3594111a70 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f358c007d10 tx=0x7f358c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:03:59.706 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.704+0000 7f3590c00700 1 -- 172.21.15.86:0/313583008 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f358c00ad40 con 0x7f359411a590 2024-03-23T18:03:59.706 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.704+0000 7f3590c00700 1 -- 172.21.15.86:0/313583008 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f358c0128e0 con 0x7f359411a590 2024-03-23T18:03:59.706 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.705+0000 7f3590c00700 1 -- 172.21.15.86:0/313583008 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f358c0116f0 con 0x7f359411a590 2024-03-23T18:03:59.707 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.705+0000 7f359a200700 1 -- 172.21.15.86:0/313583008 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3594112850 con 0x7f359411a590 2024-03-23T18:03:59.708 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.705+0000 7f359a200700 1 -- 172.21.15.86:0/313583008 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35941167b0 con 0x7f359411a590 2024-03-23T18:03:59.719 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.718+0000 7f35add6fb80 1 Processor -- start 2024-03-23T18:03:59.722 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.719+0000 7f3590c00700 1 -- 172.21.15.86:0/313583008 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f358c012420 con 0x7f359411a590 2024-03-23T18:03:59.723 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.719+0000 7f35add6fb80 1 -- start start 2024-03-23T18:03:59.723 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.720+0000 7f35add6fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c4da52100 0x562c4da524e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:59.723 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.720+0000 7f35add6fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562c4da52a20 0x562c4da5d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:59.723 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.720+0000 7f35add6fb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562c4da5d8d0 0x562c4da5fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:59.723 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.720+0000 7f35add6fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c4d9ab930 con 0x562c4da52100 2024-03-23T18:03:59.723 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.720+0000 7f35add6fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c4d9a92f0 con 0x562c4da52a20 2024-03-23T18:03:59.723 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.720+0000 7f35add6fb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c4d9a4be0 con 0x562c4da5d8d0 2024-03-23T18:03:59.723 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.721+0000 7f358aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c4da52100 0x562c4da524e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:59.723 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.721+0000 7f358aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c4da52100 0x562c4da524e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56266/0 (socket says 172.21.15.86:56266) 2024-03-23T18:03:59.723 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.721+0000 7f358aa00700 1 -- 172.21.15.86:0/213347634 learned_addr learned my addr 172.21.15.86:0/213347634 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:03:59.723 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.721+0000 7f358aa00700 1 --2- 172.21.15.86:0/213347634 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562c4da5d8d0 0x562c4da5fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:59.725 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.722+0000 7f358aa00700 1 -- 172.21.15.86:0/213347634 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562c4da5d8d0 msgr2=0x562c4da5fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:03:59.725 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.723+0000 7f358aa00700 1 --2- 172.21.15.86:0/213347634 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562c4da5d8d0 0x562c4da5fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:59.725 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.723+0000 7f358aa00700 1 -- 172.21.15.86:0/213347634 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562c4da52a20 msgr2=0x562c4da5d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:03:59.725 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.723+0000 7f3590c00700 1 --2- 172.21.15.86:0/313583008 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f35780a1350 0x7f35780a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:59.725 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.723+0000 7f358aa00700 1 --2- 172.21.15.86:0/213347634 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562c4da52a20 0x562c4da5d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:59.726 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.723+0000 7f358aa00700 1 -- 172.21.15.86:0/213347634 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562c4d826350 con 0x562c4da52100 2024-03-23T18:03:59.726 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.724+0000 7f358aa00700 1 --2- 172.21.15.86:0/313583008 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f35780a1350 0x7f35780a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:59.726 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.723+0000 7f3590c00700 1 -- 172.21.15.86:0/313583008 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f358c0cce50 con 0x7f359411a590 2024-03-23T18:03:59.727 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.726+0000 7f358aa00700 1 --2- 172.21.15.86:0/313583008 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f35780a1350 0x7f35780a3810 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f358400c010 tx=0x7f3584019040 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:03:59.727 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.727+0000 7f358aa00700 1 --2- 172.21.15.86:0/213347634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c4da52100 0x562c4da524e0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f359406e620 tx=0x7f358401a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6270041e467e37e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:03:59.729 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.728+0000 7f3583e00700 1 -- 172.21.15.86:0/213347634 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f358401d070 con 0x562c4da52100 2024-03-23T18:03:59.729 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.728+0000 7f3583e00700 1 -- 172.21.15.86:0/213347634 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f358401e040 con 0x562c4da52100 2024-03-23T18:03:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.729+0000 7f35add6fb80 1 -- 172.21.15.86:0/213347634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c4da52100 msgr2=0x562c4da524e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:03:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.729+0000 7f35add6fb80 1 --2- 172.21.15.86:0/213347634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c4da52100 0x562c4da524e0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f359406e620 tx=0x7f358401a040 comp rx=0 tx=0).stop 2024-03-23T18:03:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.730+0000 7f35add6fb80 1 -- 172.21.15.86:0/213347634 shutdown_connections 2024-03-23T18:03:59.732 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.730+0000 7f35add6fb80 1 --2- 172.21.15.86:0/213347634 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562c4da5d8d0 0x562c4da5fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:59.732 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.730+0000 7f35add6fb80 1 --2- 172.21.15.86:0/213347634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c4da52100 0x562c4da524e0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:59.732 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.730+0000 7f35add6fb80 1 --2- 172.21.15.86:0/213347634 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562c4da52a20 0x562c4da5d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:59.732 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.730+0000 7f35add6fb80 1 -- 172.21.15.86:0/213347634 >> 172.21.15.86:0/213347634 conn(0x562c4d9b2640 msgr2=0x562c4da4e520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:03:59.732 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.731+0000 7f35add6fb80 1 -- 172.21.15.86:0/213347634 shutdown_connections 2024-03-23T18:03:59.733 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.731+0000 7f35add6fb80 1 -- 172.21.15.86:0/213347634 wait complete. 2024-03-23T18:03:59.733 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.731+0000 7f35add6fb80 1 -- start start 2024-03-23T18:03:59.733 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.732+0000 7f35add6fb80 1 Processor -- start 2024-03-23T18:03:59.734 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.733+0000 7f35add6fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562c4da52100 0x562c4da5d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:59.734 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.733+0000 7f35add6fb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562c4da52a20 0x562c4da5a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:59.735 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.735+0000 7f35add6fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c4da5d8d0 0x562c4da5a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.735+0000 7f35add6fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c4d940d20 con 0x562c4da5d8d0 2024-03-23T18:03:59.737 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.735+0000 7f35add6fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c4d9409c0 con 0x562c4da52100 2024-03-23T18:03:59.737 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.735+0000 7f35add6fb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c4d828640 con 0x562c4da52a20 2024-03-23T18:03:59.737 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.735+0000 7f35add6fb80 10 client.0 _ll_register_callbacks cb 0x562c4d94a410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T18:03:59.737 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.736+0000 7f3593e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c4da5d8d0 0x562c4da5a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:59.737 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.736+0000 7f358aa00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562c4da52a20 0x562c4da5a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:59.738 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.737+0000 7f358aa00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562c4da52a20 0x562c4da5a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:60680/0 (socket says 172.21.15.86:60680) 2024-03-23T18:03:59.738 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.737+0000 7f358aa00700 1 -- 172.21.15.86:0/2613288704 learned_addr learned my addr 172.21.15.86:0/2613288704 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:03:59.739 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.738+0000 7f358aa00700 1 -- 172.21.15.86:0/2613288704 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562c4da52100 msgr2=0x562c4da5d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:03:59.739 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.738+0000 7f358aa00700 1 --2- 172.21.15.86:0/2613288704 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562c4da52100 0x562c4da5d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:59.739 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.738+0000 7f358aa00700 1 -- 172.21.15.86:0/2613288704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c4da5d8d0 msgr2=0x562c4da5a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:03:59.739 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.738+0000 7f358aa00700 1 --2- 172.21.15.86:0/2613288704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c4da5d8d0 0x562c4da5a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:59.739 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.738+0000 7f358aa00700 1 -- 172.21.15.86:0/2613288704 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f359411d210 con 0x562c4da52a20 2024-03-23T18:03:59.739 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.738+0000 7f358aa00700 1 --2- 172.21.15.86:0/2613288704 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562c4da52a20 0x562c4da5a470 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3584000c00 tx=0x7f358400ba60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c757acc57cc45e8e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:03:59.739 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.738+0000 7f3580c00700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-03-23T18:03:59.741 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.740+0000 7f3580c00700 1 -- 172.21.15.86:0/2613288704 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f358401d070 con 0x562c4da52a20 2024-03-23T18:03:59.741 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.740+0000 7f3580c00700 1 -- 172.21.15.86:0/2613288704 <== mon.1 v2:172.21.15.152:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f3584031810 con 0x562c4da52a20 2024-03-23T18:03:59.741 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.740+0000 7f35add6fb80 1 -- 172.21.15.86:0/2613288704 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562c4d965320 con 0x562c4da52a20 2024-03-23T18:03:59.742 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.741+0000 7f3580c00700 1 -- 172.21.15.86:0/2613288704 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3584026900 con 0x562c4da52a20 2024-03-23T18:03:59.743 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.742+0000 7f35add6fb80 10 client.27088 fetch_fsmap learned FSMap version 61 2024-03-23T18:03:59.743 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.742+0000 7f35add6fb80 1 -- 172.21.15.86:0/2613288704 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x562c4d7d2b00 con 0x562c4da52a20 2024-03-23T18:03:59.744 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.743+0000 7f3580c00700 1 -- 172.21.15.86:0/2613288704 <== mon.1 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3584046020 con 0x562c4da52a20 2024-03-23T18:03:59.744 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.743+0000 7f3580c00700 1 -- 172.21.15.86:0/2613288704 <== mon.1 v2:172.21.15.152:3300/0 5 ==== fsmap(e 61) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f358402f030 con 0x562c4da52a20 2024-03-23T18:03:59.744 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.743+0000 7f35add6fb80 10 client.27088 fetch_fsmap finished waiting for FSMap version 61 2024-03-23T18:03:59.744 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.743+0000 7f35add6fb80 10 client.27088 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T18:03:59.744 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.743+0000 7f35add6fb80 20 client.27088 populate_metadata read hostname 'smithi086' 2024-03-23T18:03:59.744 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.743+0000 7f35add6fb80 1 --2- 172.21.15.86:0/2613288704 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x562c4da6cfb0 0x562c4da6f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:59.744 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.743+0000 7f35add6fb80 4 client.27088 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T18:03:59.744 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.743+0000 7f35add6fb80 1 -- 172.21.15.86:0/2613288704 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562c4d99d6a0 con 0x562c4da6cfb0 2024-03-23T18:03:59.749 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.749+0000 7f3593400700 1 --2- 172.21.15.86:0/2613288704 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x562c4da6cfb0 0x562c4da6f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:59.759 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.758+0000 7f3593400700 1 --2- 172.21.15.86:0/2613288704 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x562c4da6cfb0 0x562c4da6f3d0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe34a6686c636a98 server_cookie=84ac9b922d8301d6 in_seq=0 out_seq=0 2024-03-23T18:03:59.759 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.758+0000 7f3580c00700 10 client.27088 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T18:03:59.762 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.761+0000 7f3580c00700 1 -- 172.21.15.86:0/2613288704 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x562c4d99d6a0 con 0x562c4da6cfb0 2024-03-23T18:03:59.762 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.761+0000 7f3580c00700 10 client.27088 handle_command_reply: tid=0 2024-03-23T18:03:59.763 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.761+0000 7f35add6fb80 1 client.27088 shutdown 2024-03-23T18:03:59.764 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.763+0000 7f35add6fb80 1 -- 172.21.15.86:0/2613288704 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562c4da52a20 msgr2=0x562c4da5a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:03:59.764 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.763+0000 7f35add6fb80 1 --2- 172.21.15.86:0/2613288704 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562c4da52a20 0x562c4da5a470 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3584000c00 tx=0x7f358400ba60 comp rx=0 tx=0).stop 2024-03-23T18:03:59.764 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.763+0000 7f35add6fb80 1 -- 172.21.15.86:0/2613288704 shutdown_connections 2024-03-23T18:03:59.764 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.763+0000 7f35add6fb80 1 --2- 172.21.15.86:0/2613288704 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x562c4da6cfb0 0x562c4da6f3d0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:59.764 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.763+0000 7f35add6fb80 1 --2- 172.21.15.86:0/2613288704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c4da5d8d0 0x562c4da5a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:59.764 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.763+0000 7f35add6fb80 1 --2- 172.21.15.86:0/2613288704 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562c4da52100 0x562c4da5d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:59.764 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.763+0000 7f35add6fb80 1 --2- 172.21.15.86:0/2613288704 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562c4da52a20 0x562c4da5a470 secure :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3584000c00 tx=0x7f358400ba60 comp rx=0 tx=0).stop 2024-03-23T18:03:59.764 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.763+0000 7f35add6fb80 1 -- 172.21.15.86:0/2613288704 >> 172.21.15.86:0/2613288704 conn(0x562c4d9b2640 msgr2=0x562c4da540a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:03:59.765 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.763+0000 7f3580c00700 0 client.27088 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T18:03:59.765 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.764+0000 7f35add6fb80 1 -- 172.21.15.86:0/2613288704 shutdown_connections 2024-03-23T18:03:59.765 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.764+0000 7f35add6fb80 1 -- 172.21.15.86:0/2613288704 wait complete. 2024-03-23T18:03:59.766 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.765+0000 7f35add6fb80 20 client.27088 trim_cache size 0 max 16384 2024-03-23T18:03:59.807 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.806+0000 7f35add6fb80 1 Processor -- start 2024-03-23T18:03:59.807 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.806+0000 7f35add6fb80 1 -- start start 2024-03-23T18:03:59.808 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.807+0000 7f35add6fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c4d9b2640 0x562c4da644d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:59.808 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.807+0000 7f35add6fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562c4da52100 0x562c4da4e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:59.808 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.807+0000 7f35add6fb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562c4da5d8d0 0x562c4da48500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:59.809 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.807+0000 7f35add6fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c4da4ac80 con 0x562c4d9b2640 2024-03-23T18:03:59.809 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.807+0000 7f35add6fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c4da60120 con 0x562c4da52100 2024-03-23T18:03:59.809 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.807+0000 7f35add6fb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c4da5ad50 con 0x562c4da5d8d0 2024-03-23T18:03:59.809 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.808+0000 7f358aa00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562c4da52100 0x562c4da4e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:59.809 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.808+0000 7f358aa00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562c4da52100 0x562c4da4e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:42292/0 (socket says 172.21.15.86:42292) 2024-03-23T18:03:59.809 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.808+0000 7f358aa00700 1 -- 172.21.15.86:0/4096850874 learned_addr learned my addr 172.21.15.86:0/4096850874 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:03:59.810 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.808+0000 7f3593e00700 1 --2- 172.21.15.86:0/4096850874 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562c4da5d8d0 0x562c4da48500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:59.810 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.809+0000 7f3593400700 1 --2- 172.21.15.86:0/4096850874 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c4d9b2640 0x562c4da644d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:59.811 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.810+0000 7f3593400700 1 -- 172.21.15.86:0/4096850874 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562c4da5d8d0 msgr2=0x562c4da48500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:03:59.811 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.810+0000 7f3593400700 1 --2- 172.21.15.86:0/4096850874 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562c4da5d8d0 0x562c4da48500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:59.811 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.810+0000 7f3593400700 1 -- 172.21.15.86:0/4096850874 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562c4da52100 msgr2=0x562c4da4e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:03:59.811 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.810+0000 7f3593400700 1 --2- 172.21.15.86:0/4096850874 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562c4da52100 0x562c4da4e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:59.811 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.810+0000 7f3593400700 1 -- 172.21.15.86:0/4096850874 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562c4d96ae40 con 0x562c4d9b2640 2024-03-23T18:03:59.812 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.811+0000 7f3593400700 1 --2- 172.21.15.86:0/4096850874 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c4d9b2640 0x562c4da644d0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x562c4da6f9b0 tx=0x7f357c005e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5518d34c4d4e9780 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:03:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.812+0000 7f3589200700 1 -- 172.21.15.86:0/4096850874 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f357c00a390 con 0x562c4d9b2640 2024-03-23T18:03:59.814 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.812+0000 7f3589200700 1 -- 172.21.15.86:0/4096850874 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f357c016040 con 0x562c4d9b2640 2024-03-23T18:03:59.815 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.812+0000 7f3589200700 1 -- 172.21.15.86:0/4096850874 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f357c004b20 con 0x562c4d9b2640 2024-03-23T18:03:59.815 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.814+0000 7f35add6fb80 1 -- 172.21.15.86:0/4096850874 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c4d9b2640 msgr2=0x562c4da644d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:03:59.815 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.814+0000 7f35add6fb80 1 --2- 172.21.15.86:0/4096850874 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c4d9b2640 0x562c4da644d0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x562c4da6f9b0 tx=0x7f357c005e10 comp rx=0 tx=0).stop 2024-03-23T18:03:59.817 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.816+0000 7f35add6fb80 1 -- 172.21.15.86:0/4096850874 shutdown_connections 2024-03-23T18:03:59.817 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.816+0000 7f35add6fb80 1 --2- 172.21.15.86:0/4096850874 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562c4da5d8d0 0x562c4da48500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:59.817 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.816+0000 7f35add6fb80 1 --2- 172.21.15.86:0/4096850874 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c4d9b2640 0x562c4da644d0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:59.817 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.816+0000 7f35add6fb80 1 --2- 172.21.15.86:0/4096850874 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562c4da52100 0x562c4da4e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:59.817 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.816+0000 7f35add6fb80 1 -- 172.21.15.86:0/4096850874 >> 172.21.15.86:0/4096850874 conn(0x562c4da55eb0 msgr2=0x562c4da69120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:03:59.819 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.818+0000 7f35add6fb80 1 -- 172.21.15.86:0/4096850874 shutdown_connections 2024-03-23T18:03:59.819 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.818+0000 7f35add6fb80 1 -- 172.21.15.86:0/4096850874 wait complete. 2024-03-23T18:03:59.819 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.818+0000 7f35add6fb80 1 -- start start 2024-03-23T18:03:59.820 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.819+0000 7f35add6fb80 1 Processor -- start 2024-03-23T18:03:59.823 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.822+0000 7f35add6fb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562c4d9b2640 0x562c4dab7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:59.823 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.822+0000 7f35add6fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562c4da52100 0x562c4dab7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:59.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.823+0000 7f3593400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562c4d9b2640 0x562c4dab7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:59.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.823+0000 7f35add6fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c4da5d8d0 0x562c4dab2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:59.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.823+0000 7f3593400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562c4d9b2640 0x562c4dab7020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:60688/0 (socket says 172.21.15.86:60688) 2024-03-23T18:03:59.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.823+0000 7f358aa00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562c4da52100 0x562c4dab7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:59.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.823+0000 7f35add6fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c4dab2ff0 con 0x562c4da5d8d0 2024-03-23T18:03:59.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.823+0000 7f35add6fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c4dab3130 con 0x562c4da52100 2024-03-23T18:03:59.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.823+0000 7f35add6fb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c4dab32f0 con 0x562c4d9b2640 2024-03-23T18:03:59.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.824+0000 7f35add6fb80 10 client.0 _ll_register_callbacks cb 0x562c4d94a410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T18:03:59.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.824+0000 7f358aa00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562c4da52100 0x562c4dab7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:42300/0 (socket says 172.21.15.86:42300) 2024-03-23T18:03:59.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.824+0000 7f358aa00700 1 -- 172.21.15.86:0/3460352482 learned_addr learned my addr 172.21.15.86:0/3460352482 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:03:59.826 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.824+0000 7f3593400700 1 -- 172.21.15.86:0/3460352482 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562c4da52100 msgr2=0x562c4dab7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:03:59.826 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.824+0000 7f3593400700 1 --2- 172.21.15.86:0/3460352482 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562c4da52100 0x562c4dab7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:59.826 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.824+0000 7f3593400700 1 -- 172.21.15.86:0/3460352482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c4da5d8d0 msgr2=0x562c4dab2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:03:59.826 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.824+0000 7f3593400700 1 --2- 172.21.15.86:0/3460352482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c4da5d8d0 0x562c4dab2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:03:59.827 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.824+0000 7f3593400700 1 -- 172.21.15.86:0/3460352482 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f357c00ab40 con 0x562c4d9b2640 2024-03-23T18:03:59.827 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.825+0000 7f3593400700 1 --2- 172.21.15.86:0/3460352482 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562c4d9b2640 0x562c4dab7020 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f357c005ea0 tx=0x7f357c010ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbceeb22ac26b58c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:03:59.827 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.825+0000 7f3580c00700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-03-23T18:03:59.828 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.827+0000 7f3580c00700 1 -- 172.21.15.86:0/3460352482 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f357c025070 con 0x562c4d9b2640 2024-03-23T18:03:59.829 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.827+0000 7f3580c00700 1 -- 172.21.15.86:0/3460352482 <== mon.1 v2:172.21.15.152:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f357c0049d0 con 0x562c4d9b2640 2024-03-23T18:03:59.829 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.828+0000 7f35add6fb80 1 -- 172.21.15.86:0/3460352482 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f358401e040 con 0x562c4d9b2640 2024-03-23T18:03:59.830 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.830+0000 7f3580c00700 1 -- 172.21.15.86:0/3460352482 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f357c004b90 con 0x562c4d9b2640 2024-03-23T18:03:59.831 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.830+0000 7f35add6fb80 10 client.27094 fetch_fsmap learned FSMap version 61 2024-03-23T18:03:59.831 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.830+0000 7f35add6fb80 1 -- 172.21.15.86:0/3460352482 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x562c4d82ac70 con 0x562c4d9b2640 2024-03-23T18:03:59.831 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.831+0000 7f3580c00700 1 -- 172.21.15.86:0/3460352482 <== mon.1 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f358401e040 con 0x562c4d9b2640 2024-03-23T18:03:59.832 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.831+0000 7f3580c00700 1 -- 172.21.15.86:0/3460352482 <== mon.1 v2:172.21.15.152:3300/0 5 ==== fsmap(e 61) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f357c034680 con 0x562c4d9b2640 2024-03-23T18:03:59.832 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.831+0000 7f35add6fb80 10 client.27094 fetch_fsmap finished waiting for FSMap version 61 2024-03-23T18:03:59.832 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.831+0000 7f35add6fb80 10 client.27094 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T18:03:59.832 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.831+0000 7f35add6fb80 20 client.27094 populate_metadata read hostname 'smithi086' 2024-03-23T18:03:59.833 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.831+0000 7f35add6fb80 1 --2- 172.21.15.86:0/3460352482 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x562c4dabff50 0x562c4dac2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:03:59.833 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.832+0000 7f35add6fb80 4 client.27094 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T18:03:59.833 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.832+0000 7f35add6fb80 1 -- 172.21.15.86:0/3460352482 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562c4d7f3c90 con 0x562c4dabff50 2024-03-23T18:03:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.837+0000 7f358aa00700 1 --2- 172.21.15.86:0/3460352482 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x562c4dabff50 0x562c4dac2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:03:59.848 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.847+0000 7f358aa00700 1 --2- 172.21.15.86:0/3460352482 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x562c4dabff50 0x562c4dac2370 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=324ec96b5bd8f27a server_cookie=3217a42793d66099 in_seq=0 out_seq=0 2024-03-23T18:03:59.848 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:03:59.847+0000 7f3580c00700 10 client.27094 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T18:04:00.225 INFO:tasks.workunit.client.0.smithi086.stdout: 10 10738 31.78 MB/sec warmup 24 sec latency 654.622 ms 2024-03-23T18:04:00.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:03:59 smithi086 ceph-mon[38221]: pgmap v419: 97 pgs: 97 active+clean; 397 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 8.4 MiB/s wr, 544 op/s 2024-03-23T18:04:00.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:03:59 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T18:04:00.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:03:59 smithi092 ceph-mon[79089]: pgmap v419: 97 pgs: 97 active+clean; 397 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 8.4 MiB/s wr, 544 op/s 2024-03-23T18:04:00.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:03:59 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T18:04:00.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:03:59 smithi152 ceph-mon[47949]: pgmap v419: 97 pgs: 97 active+clean; 397 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 8.4 MiB/s wr, 544 op/s 2024-03-23T18:04:00.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:03:59 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T18:04:01.100 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:01.097+0000 7f3580c00700 1 -- 172.21.15.86:0/3460352482 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562c4d7f3c90 con 0x562c4dabff50 2024-03-23T18:04:01.100 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:01.097+0000 7f3580c00700 10 client.27094 handle_command_reply: tid=0 2024-03-23T18:04:01.100 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:01.098+0000 7f35add6fb80 1 client.27094 shutdown 2024-03-23T18:04:01.100 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:01.098+0000 7f35add6fb80 1 -- 172.21.15.86:0/3460352482 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562c4d9b2640 msgr2=0x562c4dab7020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:04:01.100 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:01.098+0000 7f35add6fb80 1 --2- 172.21.15.86:0/3460352482 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562c4d9b2640 0x562c4dab7020 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f357c005ea0 tx=0x7f357c010ca0 comp rx=0 tx=0).stop 2024-03-23T18:04:01.100 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:01.098+0000 7f35add6fb80 1 -- 172.21.15.86:0/3460352482 shutdown_connections 2024-03-23T18:04:01.100 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:01.098+0000 7f35add6fb80 1 --2- 172.21.15.86:0/3460352482 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x562c4dabff50 0x562c4dac2370 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:01.100 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:01.098+0000 7f35add6fb80 1 --2- 172.21.15.86:0/3460352482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c4da5d8d0 0x562c4dab2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:01.100 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:01.098+0000 7f35add6fb80 1 --2- 172.21.15.86:0/3460352482 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562c4d9b2640 0x562c4dab7020 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:01.101 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:01.098+0000 7f35add6fb80 1 --2- 172.21.15.86:0/3460352482 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562c4da52100 0x562c4dab7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:01.101 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:01.098+0000 7f35add6fb80 1 -- 172.21.15.86:0/3460352482 >> 172.21.15.86:0/3460352482 conn(0x562c4da55eb0 msgr2=0x562c4da473e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:04:01.101 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:01.098+0000 7f3580c00700 0 client.27094 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T18:04:01.101 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:01.098+0000 7f35add6fb80 1 -- 172.21.15.86:0/3460352482 shutdown_connections 2024-03-23T18:04:01.101 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:01.098+0000 7f35add6fb80 1 -- 172.21.15.86:0/3460352482 wait complete. 2024-03-23T18:04:01.101 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:01.098+0000 7f35add6fb80 20 client.27094 trim_cache size 0 max 16384 2024-03-23T18:04:01.102 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-03-23T18:04:01.102 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-03-23T18:04:01.102 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-03-23T18:04:01.102 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-03-23T18:04:01.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:01.099+0000 7f359a200700 1 -- 172.21.15.86:0/313583008 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f35780a1350 msgr2=0x7f35780a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:04:01.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:01.099+0000 7f359a200700 1 --2- 172.21.15.86:0/313583008 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f35780a1350 0x7f35780a3810 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f358400c010 tx=0x7f3584019040 comp rx=0 tx=0).stop 2024-03-23T18:04:01.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:01.099+0000 7f359a200700 1 -- 172.21.15.86:0/313583008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f359411a590 msgr2=0x7f3594111a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:04:01.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:01.099+0000 7f359a200700 1 --2- 172.21.15.86:0/313583008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f359411a590 0x7f3594111a70 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f358c007d10 tx=0x7f358c00a340 comp rx=0 tx=0).stop 2024-03-23T18:04:01.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:01.100+0000 7f359a200700 1 -- 172.21.15.86:0/313583008 shutdown_connections 2024-03-23T18:04:01.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:01.100+0000 7f359a200700 1 --2- 172.21.15.86:0/313583008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f359411a590 0x7f3594111a70 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:01.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:01.100+0000 7f359a200700 1 --2- 172.21.15.86:0/313583008 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f359410f4d0 0x7f3594118920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:01.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:01.100+0000 7f359a200700 1 --2- 172.21.15.86:0/313583008 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f35780a1350 0x7f35780a3810 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:01.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:01.100+0000 7f359a200700 1 --2- 172.21.15.86:0/313583008 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f359410f940 0x7f3594118ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:01.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:01.100+0000 7f359a200700 1 -- 172.21.15.86:0/313583008 >> 172.21.15.86:0/313583008 conn(0x7f359406bd80 msgr2=0x7f359406f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:04:01.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:01.100+0000 7f359a200700 1 -- 172.21.15.86:0/313583008 shutdown_connections 2024-03-23T18:04:01.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:01.100+0000 7f359a200700 1 -- 172.21.15.86:0/313583008 wait complete. 2024-03-23T18:04:01.225 INFO:tasks.workunit.client.0.smithi086.stdout: 10 10762 30.55 MB/sec warmup 25 sec latency 1230.137 ms 2024-03-23T18:04:01.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:04:00 smithi086 ceph-mon[38221]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:04:01.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:04:00 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T18:04:01.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:04:00 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T18:04:01.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:04:00 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T18:04:01.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:04:00 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T18:04:01.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:04:00 smithi092 ceph-mon[79089]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:04:01.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:04:00 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T18:04:01.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:04:00 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T18:04:01.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:04:00 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T18:04:01.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:04:00 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T18:04:01.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:04:00 smithi152 ceph-mon[47949]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:04:01.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:04:00 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T18:04:01.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:04:00 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T18:04:01.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:04:00 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T18:04:01.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:04:00 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T18:04:02.225 INFO:tasks.workunit.client.0.smithi086.stdout: 10 11149 29.83 MB/sec warmup 26 sec latency 1038.808 ms 2024-03-23T18:04:02.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:04:01 smithi086 ceph-mon[38221]: pgmap v420: 97 pgs: 97 active+clean; 501 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 18 MiB/s wr, 685 op/s 2024-03-23T18:04:02.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:04:01 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T18:04:02.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:04:01 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T18:04:02.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:04:01 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T18:04:02.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:04:01 smithi092 ceph-mon[79089]: pgmap v420: 97 pgs: 97 active+clean; 501 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 18 MiB/s wr, 685 op/s 2024-03-23T18:04:02.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:04:01 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T18:04:02.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:04:01 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T18:04:02.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:04:01 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T18:04:02.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:04:01 smithi152 ceph-mon[47949]: pgmap v420: 97 pgs: 97 active+clean; 501 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 18 MiB/s wr, 685 op/s 2024-03-23T18:04:02.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:04:01 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T18:04:02.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:04:01 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T18:04:02.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:04:01 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T18:04:03.225 INFO:tasks.workunit.client.0.smithi086.stdout: 10 11954 30.30 MB/sec warmup 27 sec latency 97.869 ms 2024-03-23T18:04:04.225 INFO:tasks.workunit.client.0.smithi086.stdout: 10 12829 30.38 MB/sec warmup 28 sec latency 55.469 ms 2024-03-23T18:04:04.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:04:03 smithi086 ceph-mon[38221]: pgmap v421: 97 pgs: 97 active+clean; 501 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 16 MiB/s wr, 293 op/s 2024-03-23T18:04:04.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:04:03 smithi092 ceph-mon[79089]: pgmap v421: 97 pgs: 97 active+clean; 501 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 16 MiB/s wr, 293 op/s 2024-03-23T18:04:04.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:04:03 smithi152 ceph-mon[47949]: pgmap v421: 97 pgs: 97 active+clean; 501 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 16 MiB/s wr, 293 op/s 2024-03-23T18:04:05.225 INFO:tasks.workunit.client.0.smithi086.stdout: 10 13561 30.87 MB/sec warmup 29 sec latency 300.347 ms 2024-03-23T18:04:06.225 INFO:tasks.workunit.client.0.smithi086.stdout: 10 14316 31.71 MB/sec warmup 30 sec latency 1300.393 ms 2024-03-23T18:04:06.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:04:05 smithi086 ceph-mon[38221]: pgmap v422: 97 pgs: 97 active+clean; 531 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 27 MiB/s wr, 862 op/s 2024-03-23T18:04:06.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:04:05 smithi086 ceph-mon[38221]: osdmap e100: 12 total, 12 up, 12 in 2024-03-23T18:04:06.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:04:05 smithi086 ceph-mon[38221]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:04:06.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:04:05 smithi092 ceph-mon[79089]: pgmap v422: 97 pgs: 97 active+clean; 531 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 27 MiB/s wr, 862 op/s 2024-03-23T18:04:06.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:04:05 smithi092 ceph-mon[79089]: osdmap e100: 12 total, 12 up, 12 in 2024-03-23T18:04:06.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:04:05 smithi092 ceph-mon[79089]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:04:06.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:04:05 smithi152 ceph-mon[47949]: pgmap v422: 97 pgs: 97 active+clean; 531 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 27 MiB/s wr, 862 op/s 2024-03-23T18:04:06.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:04:05 smithi152 ceph-mon[47949]: osdmap e100: 12 total, 12 up, 12 in 2024-03-23T18:04:06.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:04:05 smithi152 ceph-mon[47949]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:04:07.225 INFO:tasks.workunit.client.0.smithi086.stdout: 10 14776 31.16 MB/sec warmup 31 sec latency 2300.452 ms 2024-03-23T18:04:08.225 INFO:tasks.workunit.client.0.smithi086.stdout: 10 15363 30.94 MB/sec warmup 32 sec latency 3300.536 ms 2024-03-23T18:04:08.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:04:07 smithi086 ceph-mon[38221]: pgmap v424: 97 pgs: 97 active+clean; 531 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 24 MiB/s wr, 857 op/s 2024-03-23T18:04:08.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:04:07 smithi092 ceph-mon[79089]: pgmap v424: 97 pgs: 97 active+clean; 531 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 24 MiB/s wr, 857 op/s 2024-03-23T18:04:08.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:04:07 smithi152 ceph-mon[47949]: pgmap v424: 97 pgs: 97 active+clean; 531 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 24 MiB/s wr, 857 op/s 2024-03-23T18:04:09.225 INFO:tasks.workunit.client.0.smithi086.stdout: 10 16221 31.58 MB/sec warmup 33 sec latency 4300.604 ms 2024-03-23T18:04:10.226 INFO:tasks.workunit.client.0.smithi086.stdout: 10 17117 32.16 MB/sec warmup 34 sec latency 5300.645 ms 2024-03-23T18:04:10.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:04:09 smithi086 ceph-mon[38221]: pgmap v425: 97 pgs: 97 active+clean; 531 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 24 MiB/s wr, 857 op/s 2024-03-23T18:04:10.318 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:04:09 smithi092 ceph-mon[79089]: pgmap v425: 97 pgs: 97 active+clean; 531 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 24 MiB/s wr, 857 op/s 2024-03-23T18:04:10.340 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:04:09 smithi152 ceph-mon[47949]: pgmap v425: 97 pgs: 97 active+clean; 531 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 24 MiB/s wr, 857 op/s 2024-03-23T18:04:11.225 INFO:tasks.workunit.client.0.smithi086.stdout: 10 17886 32.70 MB/sec warmup 35 sec latency 6300.686 ms 2024-03-23T18:04:12.226 INFO:tasks.workunit.client.0.smithi086.stdout: 10 18342 32.29 MB/sec warmup 36 sec latency 7300.746 ms 2024-03-23T18:04:12.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:04:11 smithi086 ceph-mon[38221]: pgmap v426: 97 pgs: 97 active+clean; 553 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 15 MiB/s wr, 911 op/s 2024-03-23T18:04:12.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:04:11 smithi092 ceph-mon[79089]: pgmap v426: 97 pgs: 97 active+clean; 553 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 15 MiB/s wr, 911 op/s 2024-03-23T18:04:12.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:04:11 smithi152 ceph-mon[47949]: pgmap v426: 97 pgs: 97 active+clean; 553 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 15 MiB/s wr, 911 op/s 2024-03-23T18:04:13.225 INFO:tasks.workunit.client.0.smithi086.stdout: 10 18915 31.90 MB/sec warmup 37 sec latency 8300.827 ms 2024-03-23T18:04:14.226 INFO:tasks.workunit.client.0.smithi086.stdout: 10 19959 32.48 MB/sec warmup 38 sec latency 9300.950 ms 2024-03-23T18:04:14.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:04:13 smithi086 ceph-mon[38221]: pgmap v427: 97 pgs: 97 active+clean; 553 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 15 MiB/s wr, 911 op/s 2024-03-23T18:04:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.282+0000 7f4e1c000700 1 -- 172.21.15.86:0/2491249878 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4e14100f10 msgr2=0x7f4e141012f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:04:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.282+0000 7f4e1c000700 1 --2- 172.21.15.86:0/2491249878 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4e14100f10 0x7f4e141012f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c00b0a0 tx=0x7f4e0c00b3b0 comp rx=0 tx=0).stop 2024-03-23T18:04:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.282+0000 7f4e1c000700 1 -- 172.21.15.86:0/2491249878 shutdown_connections 2024-03-23T18:04:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.282+0000 7f4e1c000700 1 --2- 172.21.15.86:0/2491249878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e14106030 0x7f4e1410dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.282+0000 7f4e1c000700 1 --2- 172.21.15.86:0/2491249878 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4e14100f10 0x7f4e141012f0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.282+0000 7f4e1c000700 1 --2- 172.21.15.86:0/2491249878 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e141018c0 0x7f4e14105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.282+0000 7f4e1c000700 1 -- 172.21.15.86:0/2491249878 >> 172.21.15.86:0/2491249878 conn(0x7f4e140fc770 msgr2=0x7f4e140feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:04:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.283+0000 7f4e1c000700 1 -- 172.21.15.86:0/2491249878 shutdown_connections 2024-03-23T18:04:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.283+0000 7f4e1c000700 1 -- 172.21.15.86:0/2491249878 wait complete. 2024-03-23T18:04:14.286 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.284+0000 7f4e1c000700 1 Processor -- start 2024-03-23T18:04:14.287 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.284+0000 7f4e1c000700 1 -- start start 2024-03-23T18:04:14.287 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.284+0000 7f4e1c000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e14100f10 0x7f4e14199550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:04:14.287 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.284+0000 7f4e1c000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e14106030 0x7f4e14199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:04:14.287 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.284+0000 7f4e1c000700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4e1419de30 0x7f4e1419e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:04:14.287 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.284+0000 7f4e1c000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e1419e8b0 con 0x7f4e14100f10 2024-03-23T18:04:14.287 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.284+0000 7f4e1c000700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e1419ea20 con 0x7f4e14106030 2024-03-23T18:04:14.287 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.284+0000 7f4e1c000700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e1419ebe0 con 0x7f4e1419de30 2024-03-23T18:04:14.287 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.285+0000 7f4e19600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e14100f10 0x7f4e14199550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:04:14.287 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.285+0000 7f4e18c00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e14106030 0x7f4e14199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:04:14.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.285+0000 7f4e19600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e14100f10 0x7f4e14199550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47300/0 (socket says 172.21.15.86:47300) 2024-03-23T18:04:14.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.285+0000 7f4e19600700 1 -- 172.21.15.86:0/431419355 learned_addr learned my addr 172.21.15.86:0/431419355 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:04:14.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.285+0000 7f4e1a000700 1 --2- 172.21.15.86:0/431419355 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4e1419de30 0x7f4e1419e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:04:14.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.285+0000 7f4e18c00700 1 -- 172.21.15.86:0/431419355 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4e1419de30 msgr2=0x7f4e1419e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:04:14.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.285+0000 7f4e18c00700 1 --2- 172.21.15.86:0/431419355 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4e1419de30 0x7f4e1419e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:14.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.285+0000 7f4e18c00700 1 -- 172.21.15.86:0/431419355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e14100f10 msgr2=0x7f4e14199550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:04:14.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.285+0000 7f4e18c00700 1 --2- 172.21.15.86:0/431419355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e14100f10 0x7f4e14199550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:14.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.285+0000 7f4e18c00700 1 -- 172.21.15.86:0/431419355 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e0c009d00 con 0x7f4e14106030 2024-03-23T18:04:14.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.285+0000 7f4e19600700 1 --2- 172.21.15.86:0/431419355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e14100f10 0x7f4e14199550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T18:04:14.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.285+0000 7f4e1a000700 1 --2- 172.21.15.86:0/431419355 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4e1419de30 0x7f4e1419e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T18:04:14.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.286+0000 7f4e18c00700 1 --2- 172.21.15.86:0/431419355 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e14106030 0x7f4e14199a90 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f4e0000e9c0 tx=0x7f4e0000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:04:14.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.286+0000 7f4e12000700 1 -- 172.21.15.86:0/431419355 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0000c9c0 con 0x7f4e14106030 2024-03-23T18:04:14.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.286+0000 7f4e12000700 1 -- 172.21.15.86:0/431419355 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e00010430 con 0x7f4e14106030 2024-03-23T18:04:14.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.286+0000 7f4e12000700 1 -- 172.21.15.86:0/431419355 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e00018680 con 0x7f4e14106030 2024-03-23T18:04:14.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.286+0000 7f4e1c000700 1 -- 172.21.15.86:0/431419355 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e141a6140 con 0x7f4e14106030 2024-03-23T18:04:14.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.286+0000 7f4e1c000700 1 -- 172.21.15.86:0/431419355 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e141a65d0 con 0x7f4e14106030 2024-03-23T18:04:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.292+0000 7f4e2fc1bb80 1 Processor -- start 2024-03-23T18:04:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.292+0000 7f4e12000700 1 -- 172.21.15.86:0/431419355 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f4e0000cb20 con 0x7f4e14106030 2024-03-23T18:04:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.292+0000 7f4e2fc1bb80 1 -- start start 2024-03-23T18:04:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.292+0000 7f4e2fc1bb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559979b0b100 0x559979b0b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:04:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.292+0000 7f4e2fc1bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559979b0ba20 0x559979b16390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:04:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.292+0000 7f4e2fc1bb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559979b168d0 0x559979b18cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:04:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.292+0000 7f4e2fc1bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x559979a64930 con 0x559979b0ba20 2024-03-23T18:04:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.292+0000 7f4e2fc1bb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x559979a622f0 con 0x559979b0b100 2024-03-23T18:04:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.292+0000 7f4e2fc1bb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x559979a5dbe0 con 0x559979b168d0 2024-03-23T18:04:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.292+0000 7f4e1a000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559979b0b100 0x559979b0b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:04:14.295 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.293+0000 7f4e1a000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559979b0b100 0x559979b0b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:34064/0 (socket says 172.21.15.86:34064) 2024-03-23T18:04:14.295 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.293+0000 7f4e1a000700 1 -- 172.21.15.86:0/3698435889 learned_addr learned my addr 172.21.15.86:0/3698435889 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:04:14.295 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.293+0000 7f4e18c00700 1 --2- 172.21.15.86:0/3698435889 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559979b168d0 0x559979b18cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:04:14.295 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.293+0000 7f4e19600700 1 --2- 172.21.15.86:0/3698435889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559979b0ba20 0x559979b16390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:04:14.295 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.293+0000 7f4e1a000700 1 -- 172.21.15.86:0/3698435889 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559979b168d0 msgr2=0x559979b18cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:04:14.295 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.293+0000 7f4e1a000700 1 --2- 172.21.15.86:0/3698435889 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559979b168d0 0x559979b18cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:14.295 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.293+0000 7f4e1a000700 1 -- 172.21.15.86:0/3698435889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559979b0ba20 msgr2=0x559979b16390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:04:14.295 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.293+0000 7f4e1a000700 1 --2- 172.21.15.86:0/3698435889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559979b0ba20 0x559979b16390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:14.295 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.293+0000 7f4e1a000700 1 -- 172.21.15.86:0/3698435889 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599798df350 con 0x559979b0b100 2024-03-23T18:04:14.295 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.293+0000 7f4e1a000700 1 --2- 172.21.15.86:0/3698435889 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559979b0b100 0x559979b0b4e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f4e08010480 tx=0x7f4e08010790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2627b1335726fc2d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:04:14.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.293+0000 7f4e06600700 1 -- 172.21.15.86:0/3698435889 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0801c480 con 0x559979b0b100 2024-03-23T18:04:14.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.294+0000 7f4e06600700 1 -- 172.21.15.86:0/3698435889 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e08015070 con 0x559979b0b100 2024-03-23T18:04:14.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.294+0000 7f4e06600700 1 -- 172.21.15.86:0/3698435889 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e08011a40 con 0x559979b0b100 2024-03-23T18:04:14.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.294+0000 7f4e12000700 1 --2- 172.21.15.86:0/431419355 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f4dfc0a1310 0x7f4dfc0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:04:14.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.294+0000 7f4e12000700 1 -- 172.21.15.86:0/431419355 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f4e00014070 con 0x7f4e14106030 2024-03-23T18:04:14.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.294+0000 7f4e19600700 1 --2- 172.21.15.86:0/431419355 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f4dfc0a1310 0x7f4dfc0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:04:14.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.294+0000 7f4e2fc1bb80 1 -- 172.21.15.86:0/3698435889 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559979b0b100 msgr2=0x559979b0b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:04:14.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.294+0000 7f4e2fc1bb80 1 --2- 172.21.15.86:0/3698435889 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559979b0b100 0x559979b0b4e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f4e08010480 tx=0x7f4e08010790 comp rx=0 tx=0).stop 2024-03-23T18:04:14.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.294+0000 7f4e2fc1bb80 1 -- 172.21.15.86:0/3698435889 shutdown_connections 2024-03-23T18:04:14.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.294+0000 7f4e2fc1bb80 1 --2- 172.21.15.86:0/3698435889 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559979b168d0 0x559979b18cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:14.298 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.294+0000 7f4e2fc1bb80 1 --2- 172.21.15.86:0/3698435889 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559979b0b100 0x559979b0b4e0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:14.298 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.294+0000 7f4e2fc1bb80 1 --2- 172.21.15.86:0/3698435889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559979b0ba20 0x559979b16390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:14.298 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.295+0000 7f4e2fc1bb80 1 -- 172.21.15.86:0/3698435889 >> 172.21.15.86:0/3698435889 conn(0x559979a6b640 msgr2=0x559979b07520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:04:14.298 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.295+0000 7f4e2fc1bb80 1 -- 172.21.15.86:0/3698435889 shutdown_connections 2024-03-23T18:04:14.298 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.295+0000 7f4e2fc1bb80 1 -- 172.21.15.86:0/3698435889 wait complete. 2024-03-23T18:04:14.298 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.295+0000 7f4e19600700 1 --2- 172.21.15.86:0/431419355 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f4dfc0a1310 0x7f4dfc0a37d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c00b780 tx=0x7f4e0c006210 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:04:14.298 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.295+0000 7f4e2fc1bb80 1 -- start start 2024-03-23T18:04:14.298 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.296+0000 7f4e2fc1bb80 1 Processor -- start 2024-03-23T18:04:14.298 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.297+0000 7f4e2fc1bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559979b0b100 0x559979b13630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:04:14.298 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.297+0000 7f4e2fc1bb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559979b0ba20 0x559979b13b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:04:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.297+0000 7f4e2fc1bb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559979b168d0 0x559979b140b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:04:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.297+0000 7f4e19600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559979b0b100 0x559979b13630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:04:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.297+0000 7f4e1a000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559979b168d0 0x559979b140b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:04:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.297+0000 7f4e19600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559979b0b100 0x559979b13630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47320/0 (socket says 172.21.15.86:47320) 2024-03-23T18:04:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.297+0000 7f4e1a000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559979b168d0 0x559979b140b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:34072/0 (socket says 172.21.15.86:34072) 2024-03-23T18:04:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.297+0000 7f4e19600700 1 -- 172.21.15.86:0/3083430023 learned_addr learned my addr 172.21.15.86:0/3083430023 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:04:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.297+0000 7f4e2fc1bb80 1 -- 172.21.15.86:0/3083430023 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599799f9d20 con 0x559979b0b100 2024-03-23T18:04:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.298+0000 7f4e2fc1bb80 1 -- 172.21.15.86:0/3083430023 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599799f99c0 con 0x559979b168d0 2024-03-23T18:04:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.298+0000 7f4e2fc1bb80 1 -- 172.21.15.86:0/3083430023 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599798e1640 con 0x559979b0ba20 2024-03-23T18:04:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.298+0000 7f4e1a000700 1 -- 172.21.15.86:0/3083430023 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559979b0ba20 msgr2=0x559979b13b70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-23T18:04:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.298+0000 7f4e1a000700 1 --2- 172.21.15.86:0/3083430023 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559979b0ba20 0x559979b13b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:14.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.298+0000 7f4e1a000700 1 -- 172.21.15.86:0/3083430023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559979b0b100 msgr2=0x559979b13630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:04:14.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.298+0000 7f4e1a000700 1 --2- 172.21.15.86:0/3083430023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559979b0b100 0x559979b13630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:14.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.298+0000 7f4e1a000700 1 -- 172.21.15.86:0/3083430023 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e0800bc90 con 0x559979b168d0 2024-03-23T18:04:14.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.299+0000 7f4e2fc1bb80 10 client.0 _ll_register_callbacks cb 0x559979a03410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T18:04:14.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.299+0000 7f4e2fc1bb80 1 -- 172.21.15.86:0/3083430023 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559979a1e320 con 0x559979b168d0 2024-03-23T18:04:14.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.299+0000 7f4e1a000700 1 --2- 172.21.15.86:0/3083430023 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559979b168d0 0x559979b140b0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f4e0800bc30 tx=0x7f4e080042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b7b72735d41f6cb server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:04:14.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.299+0000 7f4dfaa00700 10 client.27128 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T18:04:14.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.299+0000 7f4dfaa00700 1 -- 172.21.15.86:0/3083430023 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e08004590 con 0x559979b168d0 2024-03-23T18:04:14.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.299+0000 7f4dfaa00700 1 -- 172.21.15.86:0/3083430023 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f4e08022030 con 0x559979b168d0 2024-03-23T18:04:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.299+0000 7f4dfaa00700 1 -- 172.21.15.86:0/3083430023 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e08005240 con 0x559979b168d0 2024-03-23T18:04:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.299+0000 7f4e2fc1bb80 10 client.27128 fetch_fsmap learned FSMap version 63 2024-03-23T18:04:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.299+0000 7f4e2fc1bb80 1 -- 172.21.15.86:0/3083430023 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({fsmap=63}) v3 -- 0x55997988bb00 con 0x559979b168d0 2024-03-23T18:04:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.300+0000 7f4dfaa00700 1 -- 172.21.15.86:0/3083430023 <== mon.2 v2:172.21.15.92:3300/0 4 ==== fsmap(e 63) v1 ==== 7510+0+0 (secure 0 0 0) 0x7f4e080053a0 con 0x559979b168d0 2024-03-23T18:04:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.300+0000 7f4e2fc1bb80 10 client.27128 fetch_fsmap finished waiting for FSMap version 63 2024-03-23T18:04:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.300+0000 7f4e2fc1bb80 10 client.27128 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T18:04:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.300+0000 7f4e2fc1bb80 20 client.27128 populate_metadata read hostname 'smithi086' 2024-03-23T18:04:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.300+0000 7f4e2fc1bb80 1 --2- 172.21.15.86:0/3083430023 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x559979b2eb00 0x559979b30f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:04:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.300+0000 7f4e2fc1bb80 4 client.27128 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T18:04:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.300+0000 7f4e2fc1bb80 1 -- 172.21.15.86:0/3083430023 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559979a566a0 con 0x559979b2eb00 2024-03-23T18:04:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.300+0000 7f4e19600700 1 --2- 172.21.15.86:0/3083430023 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x559979b2eb00 0x559979b30f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:04:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.300+0000 7f4dfaa00700 10 client.27128 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T18:04:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.300+0000 7f4e19600700 1 --2- 172.21.15.86:0/3083430023 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x559979b2eb00 0x559979b30f60 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8f28106ae22fdfd server_cookie=72278e433d3e6e0 in_seq=0 out_seq=0 2024-03-23T18:04:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.302+0000 7f4dfaa00700 1 -- 172.21.15.86:0/3083430023 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x559979a566a0 con 0x559979b2eb00 2024-03-23T18:04:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.302+0000 7f4dfaa00700 10 client.27128 handle_command_reply: tid=0 2024-03-23T18:04:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.302+0000 7f4e2fc1bb80 1 client.27128 shutdown 2024-03-23T18:04:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.303+0000 7f4e2fc1bb80 1 -- 172.21.15.86:0/3083430023 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559979b168d0 msgr2=0x559979b140b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:04:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.303+0000 7f4e2fc1bb80 1 --2- 172.21.15.86:0/3083430023 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559979b168d0 0x559979b140b0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f4e0800bc30 tx=0x7f4e080042f0 comp rx=0 tx=0).stop 2024-03-23T18:04:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.303+0000 7f4e2fc1bb80 1 -- 172.21.15.86:0/3083430023 shutdown_connections 2024-03-23T18:04:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.303+0000 7f4e2fc1bb80 1 --2- 172.21.15.86:0/3083430023 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x559979b2eb00 0x559979b30f60 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:14.306 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.303+0000 7f4e2fc1bb80 1 --2- 172.21.15.86:0/3083430023 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559979b168d0 0x559979b140b0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:14.306 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.303+0000 7f4e2fc1bb80 1 --2- 172.21.15.86:0/3083430023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559979b0b100 0x559979b13630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:14.306 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.303+0000 7f4dfaa00700 0 client.27128 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T18:04:14.306 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.303+0000 7f4e2fc1bb80 1 --2- 172.21.15.86:0/3083430023 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559979b0ba20 0x559979b13b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:14.306 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.303+0000 7f4e2fc1bb80 1 -- 172.21.15.86:0/3083430023 >> 172.21.15.86:0/3083430023 conn(0x559979a6b640 msgr2=0x559979b16cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:04:14.306 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.303+0000 7f4e2fc1bb80 1 -- 172.21.15.86:0/3083430023 shutdown_connections 2024-03-23T18:04:14.306 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.303+0000 7f4e2fc1bb80 1 -- 172.21.15.86:0/3083430023 wait complete. 2024-03-23T18:04:14.306 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.303+0000 7f4e2fc1bb80 20 client.27128 trim_cache size 0 max 16384 2024-03-23T18:04:14.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:04:13 smithi092 ceph-mon[79089]: pgmap v427: 97 pgs: 97 active+clean; 553 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 15 MiB/s wr, 911 op/s 2024-03-23T18:04:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.325+0000 7f4e2fc1bb80 1 Processor -- start 2024-03-23T18:04:14.325 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.325+0000 7f4e2fc1bb80 1 -- start start 2024-03-23T18:04:14.326 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.326+0000 7f4e2fc1bb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559979b0b100 0x559979b35900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:04:14.326 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.326+0000 7f4e2fc1bb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559979b0ba20 0x559979b075e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:04:14.326 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.326+0000 7f4e2fc1bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559979b168d0 0x559979b00af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:04:14.326 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.326+0000 7f4e2fc1bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x559979b14ca0 con 0x559979b168d0 2024-03-23T18:04:14.326 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.326+0000 7f4e2fc1bb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x559979b082d0 con 0x559979b0b100 2024-03-23T18:04:14.326 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.326+0000 7f4e2fc1bb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x559979b2acf0 con 0x559979b0ba20 2024-03-23T18:04:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.326+0000 7f4e1a000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559979b168d0 0x559979b00af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:04:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.326+0000 7f4e19600700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559979b0b100 0x559979b35900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:04:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.326+0000 7f4e1a000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559979b168d0 0x559979b00af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47324/0 (socket says 172.21.15.86:47324) 2024-03-23T18:04:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.326+0000 7f4e1a000700 1 -- 172.21.15.86:0/942102144 learned_addr learned my addr 172.21.15.86:0/942102144 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:04:14.328 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.326+0000 7f4e18c00700 1 --2- 172.21.15.86:0/942102144 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559979b0ba20 0x559979b075e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:04:14.328 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.326+0000 7f4e19600700 1 -- 172.21.15.86:0/942102144 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559979b0ba20 msgr2=0x559979b075e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:04:14.328 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.326+0000 7f4e19600700 1 --2- 172.21.15.86:0/942102144 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559979b0ba20 0x559979b075e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:14.328 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.326+0000 7f4e19600700 1 -- 172.21.15.86:0/942102144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559979b168d0 msgr2=0x559979b00af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:04:14.328 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.326+0000 7f4e19600700 1 --2- 172.21.15.86:0/942102144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559979b168d0 0x559979b00af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.327+0000 7f4e19600700 1 -- 172.21.15.86:0/942102144 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559979a23e40 con 0x559979b0b100 2024-03-23T18:04:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.327+0000 7f4e19600700 1 --2- 172.21.15.86:0/942102144 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559979b0b100 0x559979b35900 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x559979b31560 tx=0x7f4e0c01f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=739d6b08794808cb server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:04:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.327+0000 7f4e07000700 1 -- 172.21.15.86:0/942102144 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0c00e040 con 0x559979b0b100 2024-03-23T18:04:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.327+0000 7f4e07000700 1 -- 172.21.15.86:0/942102144 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e0c021070 con 0x559979b0b100 2024-03-23T18:04:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.327+0000 7f4e07000700 1 -- 172.21.15.86:0/942102144 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0c0058a0 con 0x559979b0b100 2024-03-23T18:04:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.328+0000 7f4e2fc1bb80 1 -- 172.21.15.86:0/942102144 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559979b0b100 msgr2=0x559979b35900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:04:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.328+0000 7f4e2fc1bb80 1 --2- 172.21.15.86:0/942102144 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559979b0b100 0x559979b35900 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x559979b31560 tx=0x7f4e0c01f000 comp rx=0 tx=0).stop 2024-03-23T18:04:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.328+0000 7f4e2fc1bb80 1 -- 172.21.15.86:0/942102144 shutdown_connections 2024-03-23T18:04:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.328+0000 7f4e2fc1bb80 1 --2- 172.21.15.86:0/942102144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559979b168d0 0x559979b00af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.328+0000 7f4e2fc1bb80 1 --2- 172.21.15.86:0/942102144 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559979b0b100 0x559979b35900 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:14.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.328+0000 7f4e2fc1bb80 1 --2- 172.21.15.86:0/942102144 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559979b0ba20 0x559979b075e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:14.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.328+0000 7f4e2fc1bb80 1 -- 172.21.15.86:0/942102144 >> 172.21.15.86:0/942102144 conn(0x559979a6b640 msgr2=0x559979b003e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:04:14.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.328+0000 7f4e2fc1bb80 1 -- 172.21.15.86:0/942102144 shutdown_connections 2024-03-23T18:04:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.328+0000 7f4e2fc1bb80 1 -- 172.21.15.86:0/942102144 wait complete. 2024-03-23T18:04:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.329+0000 7f4e2fc1bb80 1 -- start start 2024-03-23T18:04:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.329+0000 7f4e2fc1bb80 1 Processor -- start 2024-03-23T18:04:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.330+0000 7f4e2fc1bb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559979b0b100 0x559979b25900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:04:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.330+0000 7f4e2fc1bb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559979b0ba20 0x559979b25e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:04:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.330+0000 7f4e2fc1bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559979b168d0 0x559979b21f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:04:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.330+0000 7f4e2fc1bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x559979b22620 con 0x559979b168d0 2024-03-23T18:04:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.330+0000 7f4e2fc1bb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x559979b22790 con 0x559979b0ba20 2024-03-23T18:04:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.330+0000 7f4e2fc1bb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x559979b22950 con 0x559979b0b100 2024-03-23T18:04:14.334 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.330+0000 7f4e2fc1bb80 10 client.0 _ll_register_callbacks cb 0x559979a03410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T18:04:14.334 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.330+0000 7f4e19600700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559979b0b100 0x559979b25900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:04:14.334 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.330+0000 7f4e19600700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559979b0b100 0x559979b25900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:54678/0 (socket says 172.21.15.86:54678) 2024-03-23T18:04:14.334 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.330+0000 7f4e19600700 1 -- 172.21.15.86:0/1602222620 learned_addr learned my addr 172.21.15.86:0/1602222620 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:04:14.334 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.330+0000 7f4e18c00700 1 --2- 172.21.15.86:0/1602222620 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559979b0ba20 0x559979b25e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:04:14.334 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.330+0000 7f4e19600700 1 -- 172.21.15.86:0/1602222620 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559979b0ba20 msgr2=0x559979b25e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:04:14.334 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.330+0000 7f4e19600700 1 --2- 172.21.15.86:0/1602222620 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559979b0ba20 0x559979b25e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:14.334 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.330+0000 7f4e19600700 1 -- 172.21.15.86:0/1602222620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559979b168d0 msgr2=0x559979b21f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:04:14.334 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.330+0000 7f4e1a000700 1 --2- 172.21.15.86:0/1602222620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559979b168d0 0x559979b21f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:04:14.334 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.331+0000 7f4e19600700 1 --2- 172.21.15.86:0/1602222620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559979b168d0 0x559979b21f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.331+0000 7f4e19600700 1 -- 172.21.15.86:0/1602222620 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e1410e420 con 0x559979b0b100 2024-03-23T18:04:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.331+0000 7f4e1a000700 1 --2- 172.21.15.86:0/1602222620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559979b168d0 0x559979b21f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T18:04:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.331+0000 7f4e19600700 1 --2- 172.21.15.86:0/1602222620 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559979b0b100 0x559979b25900 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c009850 tx=0x7f4e0c00bd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1821dd66ce160f2e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:04:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.331+0000 7f4dfaa00700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-03-23T18:04:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.331+0000 7f4dfaa00700 1 -- 172.21.15.86:0/1602222620 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0c003a00 con 0x559979b0b100 2024-03-23T18:04:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.331+0000 7f4dfaa00700 1 -- 172.21.15.86:0/1602222620 <== mon.1 v2:172.21.15.152:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f4e0c023070 con 0x559979b0b100 2024-03-23T18:04:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.332+0000 7f4e2fc1bb80 1 -- 172.21.15.86:0/1602222620 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4e08015070 con 0x559979b0b100 2024-03-23T18:04:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.332+0000 7f4dfaa00700 1 -- 172.21.15.86:0/1602222620 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e0c003b60 con 0x559979b0b100 2024-03-23T18:04:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.332+0000 7f4e2fc1bb80 10 client.27109 fetch_fsmap learned FSMap version 63 2024-03-23T18:04:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.332+0000 7f4e2fc1bb80 1 -- 172.21.15.86:0/1602222620 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) v3 -- 0x5599798e3c70 con 0x559979b0b100 2024-03-23T18:04:14.336 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.332+0000 7f4dfaa00700 1 -- 172.21.15.86:0/1602222620 <== mon.1 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e08015070 con 0x559979b0b100 2024-03-23T18:04:14.336 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.332+0000 7f4dfaa00700 1 -- 172.21.15.86:0/1602222620 <== mon.1 v2:172.21.15.152:3300/0 5 ==== fsmap(e 63) v1 ==== 7510+0+0 (secure 0 0 0) 0x7f4e0c01d6c0 con 0x559979b0b100 2024-03-23T18:04:14.336 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.332+0000 7f4e2fc1bb80 10 client.27109 fetch_fsmap finished waiting for FSMap version 63 2024-03-23T18:04:14.336 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.332+0000 7f4e2fc1bb80 10 client.27109 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T18:04:14.336 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.332+0000 7f4e2fc1bb80 20 client.27109 populate_metadata read hostname 'smithi086' 2024-03-23T18:04:14.336 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.332+0000 7f4e2fc1bb80 1 --2- 172.21.15.86:0/1602222620 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x559979b81660 0x559979b83a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:04:14.336 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.332+0000 7f4e2fc1bb80 4 client.27109 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T18:04:14.336 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.332+0000 7f4e2fc1bb80 1 -- 172.21.15.86:0/1602222620 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5599798acc90 con 0x559979b81660 2024-03-23T18:04:14.336 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.333+0000 7f4e18c00700 1 --2- 172.21.15.86:0/1602222620 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x559979b81660 0x559979b83a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:04:14.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:04:13 smithi152 ceph-mon[47949]: pgmap v427: 97 pgs: 97 active+clean; 553 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 15 MiB/s wr, 911 op/s 2024-03-23T18:04:14.337 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.334+0000 7f4e18c00700 1 --2- 172.21.15.86:0/1602222620 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x559979b81660 0x559979b83a80 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9efcdf175b7eed1 server_cookie=962d97423d972650 in_seq=0 out_seq=0 2024-03-23T18:04:14.337 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:14.334+0000 7f4dfaa00700 10 client.27109 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T18:04:15.044 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:15.044+0000 7f4dfaa00700 1 -- 172.21.15.86:0/1602222620 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5599798acc90 con 0x559979b81660 2024-03-23T18:04:15.044 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:15.044+0000 7f4dfaa00700 10 client.27109 handle_command_reply: tid=0 2024-03-23T18:04:15.044 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:15.044+0000 7f4e2fc1bb80 1 client.27109 shutdown 2024-03-23T18:04:15.045 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:15.044+0000 7f4e2fc1bb80 1 -- 172.21.15.86:0/1602222620 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559979b0b100 msgr2=0x559979b25900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:04:15.045 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:15.044+0000 7f4e2fc1bb80 1 --2- 172.21.15.86:0/1602222620 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559979b0b100 0x559979b25900 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c009850 tx=0x7f4e0c00bd80 comp rx=0 tx=0).stop 2024-03-23T18:04:15.045 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:15.044+0000 7f4e2fc1bb80 1 -- 172.21.15.86:0/1602222620 shutdown_connections 2024-03-23T18:04:15.045 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:15.044+0000 7f4e2fc1bb80 1 --2- 172.21.15.86:0/1602222620 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x559979b81660 0x559979b83a80 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:15.045 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:15.044+0000 7f4e2fc1bb80 1 --2- 172.21.15.86:0/1602222620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559979b168d0 0x559979b21f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:15.045 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:15.044+0000 7f4e2fc1bb80 1 --2- 172.21.15.86:0/1602222620 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559979b0b100 0x559979b25900 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:15.046 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:15.044+0000 7f4e2fc1bb80 1 --2- 172.21.15.86:0/1602222620 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559979b0ba20 0x559979b25e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:15.046 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:15.044+0000 7f4e2fc1bb80 1 -- 172.21.15.86:0/1602222620 >> 172.21.15.86:0/1602222620 conn(0x559979a6b640 msgr2=0x559979b07450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:04:15.046 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:15.044+0000 7f4dfaa00700 0 client.27109 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T18:04:15.046 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:15.045+0000 7f4e2fc1bb80 1 -- 172.21.15.86:0/1602222620 shutdown_connections 2024-03-23T18:04:15.046 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:15.045+0000 7f4e2fc1bb80 1 -- 172.21.15.86:0/1602222620 wait complete. 2024-03-23T18:04:15.046 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:15.045+0000 7f4e2fc1bb80 20 client.27109 trim_cache size 0 max 16384 2024-03-23T18:04:15.046 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-03-23T18:04:15.050 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-03-23T18:04:15.050 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-03-23T18:04:15.050 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-03-23T18:04:15.051 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:15.046+0000 7f4e1c000700 1 -- 172.21.15.86:0/431419355 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f4dfc0a1310 msgr2=0x7f4dfc0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:04:15.051 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:15.046+0000 7f4e1c000700 1 --2- 172.21.15.86:0/431419355 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f4dfc0a1310 0x7f4dfc0a37d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c00b780 tx=0x7f4e0c006210 comp rx=0 tx=0).stop 2024-03-23T18:04:15.051 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:15.046+0000 7f4e1c000700 1 -- 172.21.15.86:0/431419355 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e14106030 msgr2=0x7f4e14199a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:04:15.051 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:15.046+0000 7f4e1c000700 1 --2- 172.21.15.86:0/431419355 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e14106030 0x7f4e14199a90 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f4e0000e9c0 tx=0x7f4e0000ecd0 comp rx=0 tx=0).stop 2024-03-23T18:04:15.051 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:15.046+0000 7f4e1c000700 1 -- 172.21.15.86:0/431419355 shutdown_connections 2024-03-23T18:04:15.051 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:15.046+0000 7f4e1c000700 1 --2- 172.21.15.86:0/431419355 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f4dfc0a1310 0x7f4dfc0a37d0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:15.051 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:15.046+0000 7f4e1c000700 1 --2- 172.21.15.86:0/431419355 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4e1419de30 0x7f4e1419e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:15.051 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:15.046+0000 7f4e1c000700 1 --2- 172.21.15.86:0/431419355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e14100f10 0x7f4e14199550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:15.051 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:15.046+0000 7f4e1c000700 1 --2- 172.21.15.86:0/431419355 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e14106030 0x7f4e14199a90 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:15.051 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:15.046+0000 7f4e1c000700 1 -- 172.21.15.86:0/431419355 >> 172.21.15.86:0/431419355 conn(0x7f4e140fc770 msgr2=0x7f4e140fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:04:15.052 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:15.047+0000 7f4e1c000700 1 -- 172.21.15.86:0/431419355 shutdown_connections 2024-03-23T18:04:15.052 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:15.047+0000 7f4e1c000700 1 -- 172.21.15.86:0/431419355 wait complete. 2024-03-23T18:04:15.225 INFO:tasks.workunit.client.0.smithi086.stdout: 10 20349 32.39 MB/sec warmup 39 sec latency 10111.256 ms 2024-03-23T18:04:16.225 INFO:tasks.workunit.client.0.smithi086.stdout: 10 21293 32.96 MB/sec warmup 40 sec latency 1216.602 ms 2024-03-23T18:04:16.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:04:15 smithi086 ceph-mon[38221]: pgmap v428: 97 pgs: 97 active+clean; 577 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.4 MiB/s wr, 535 op/s 2024-03-23T18:04:16.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:04:15 smithi086 ceph-mon[38221]: osdmap e101: 12 total, 12 up, 12 in 2024-03-23T18:04:16.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:04:15 smithi092 ceph-mon[79089]: pgmap v428: 97 pgs: 97 active+clean; 577 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.4 MiB/s wr, 535 op/s 2024-03-23T18:04:16.309 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:04:15 smithi092 ceph-mon[79089]: osdmap e101: 12 total, 12 up, 12 in 2024-03-23T18:04:16.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:04:15 smithi152 ceph-mon[47949]: pgmap v428: 97 pgs: 97 active+clean; 577 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.4 MiB/s wr, 535 op/s 2024-03-23T18:04:16.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:04:15 smithi152 ceph-mon[47949]: osdmap e101: 12 total, 12 up, 12 in 2024-03-23T18:04:17.226 INFO:tasks.workunit.client.0.smithi086.stdout: 10 21977 33.17 MB/sec warmup 41 sec latency 2216.657 ms 2024-03-23T18:04:18.225 INFO:tasks.workunit.client.0.smithi086.stdout: 10 22707 33.20 MB/sec warmup 42 sec latency 3216.702 ms 2024-03-23T18:04:18.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:04:17 smithi086 ceph-mon[38221]: pgmap v430: 97 pgs: 97 active+clean; 579 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.8 MiB/s wr, 612 op/s 2024-03-23T18:04:18.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:04:17 smithi092 ceph-mon[79089]: pgmap v430: 97 pgs: 97 active+clean; 579 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.8 MiB/s wr, 612 op/s 2024-03-23T18:04:18.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:04:17 smithi152 ceph-mon[47949]: pgmap v430: 97 pgs: 97 active+clean; 579 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.8 MiB/s wr, 612 op/s 2024-03-23T18:04:19.226 INFO:tasks.workunit.client.0.smithi086.stdout: 10 23679 33.61 MB/sec warmup 43 sec latency 4216.743 ms 2024-03-23T18:04:19.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:04:18 smithi086 ceph-mon[38221]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:04:19.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:04:18 smithi092 ceph-mon[79089]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:04:19.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:04:18 smithi152 ceph-mon[47949]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:04:20.226 INFO:tasks.workunit.client.0.smithi086.stdout: 10 24451 34.09 MB/sec warmup 44 sec latency 4725.760 ms 2024-03-23T18:04:20.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:04:19 smithi086 ceph-mon[38221]: pgmap v431: 97 pgs: 97 active+clean; 579 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.8 MiB/s wr, 612 op/s 2024-03-23T18:04:20.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:04:19 smithi092 ceph-mon[79089]: pgmap v431: 97 pgs: 97 active+clean; 579 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.8 MiB/s wr, 612 op/s 2024-03-23T18:04:20.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:04:19 smithi152 ceph-mon[47949]: pgmap v431: 97 pgs: 97 active+clean; 579 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.8 MiB/s wr, 612 op/s 2024-03-23T18:04:21.226 INFO:tasks.workunit.client.0.smithi086.stdout: 10 25234 34.17 MB/sec warmup 45 sec latency 60.527 ms 2024-03-23T18:04:22.226 INFO:tasks.workunit.client.0.smithi086.stdout: 10 26022 34.14 MB/sec warmup 46 sec latency 70.758 ms 2024-03-23T18:04:22.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:04:21 smithi086 ceph-mon[38221]: pgmap v432: 97 pgs: 97 active+clean; 602 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 8.3 MiB/s wr, 770 op/s 2024-03-23T18:04:22.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:04:21 smithi092 ceph-mon[79089]: pgmap v432: 97 pgs: 97 active+clean; 602 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 8.3 MiB/s wr, 770 op/s 2024-03-23T18:04:22.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:04:21 smithi152 ceph-mon[47949]: pgmap v432: 97 pgs: 97 active+clean; 602 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 8.3 MiB/s wr, 770 op/s 2024-03-23T18:04:23.226 INFO:tasks.workunit.client.0.smithi086.stdout: 10 27209 34.82 MB/sec warmup 47 sec latency 52.261 ms 2024-03-23T18:04:24.226 INFO:tasks.workunit.client.0.smithi086.stdout: 10 28132 35.21 MB/sec warmup 48 sec latency 46.027 ms 2024-03-23T18:04:24.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:04:23 smithi086 ceph-mon[38221]: pgmap v433: 97 pgs: 97 active+clean; 602 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 8.3 MiB/s wr, 770 op/s 2024-03-23T18:04:24.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:04:23 smithi086 ceph-mon[38221]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:04:24.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:04:23 smithi092 ceph-mon[79089]: pgmap v433: 97 pgs: 97 active+clean; 602 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 8.3 MiB/s wr, 770 op/s 2024-03-23T18:04:24.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:04:23 smithi092 ceph-mon[79089]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:04:24.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:04:23 smithi152 ceph-mon[47949]: pgmap v433: 97 pgs: 97 active+clean; 602 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 8.3 MiB/s wr, 770 op/s 2024-03-23T18:04:24.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:04:23 smithi152 ceph-mon[47949]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:04:25.226 INFO:tasks.workunit.client.0.smithi086.stdout: 10 28985 35.51 MB/sec warmup 49 sec latency 59.003 ms 2024-03-23T18:04:26.227 INFO:tasks.workunit.client.0.smithi086.stdout: 10 29610 35.24 MB/sec warmup 50 sec latency 377.875 ms 2024-03-23T18:04:26.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:04:25 smithi086 ceph-mon[38221]: pgmap v434: 97 pgs: 97 active+clean; 615 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 7.1 MiB/s wr, 598 op/s 2024-03-23T18:04:26.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:04:25 smithi092 ceph-mon[79089]: pgmap v434: 97 pgs: 97 active+clean; 615 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 7.1 MiB/s wr, 598 op/s 2024-03-23T18:04:26.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:04:25 smithi152 ceph-mon[47949]: pgmap v434: 97 pgs: 97 active+clean; 615 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 7.1 MiB/s wr, 598 op/s 2024-03-23T18:04:27.227 INFO:tasks.workunit.client.0.smithi086.stdout: 10 30498 35.77 MB/sec warmup 51 sec latency 99.048 ms 2024-03-23T18:04:28.227 INFO:tasks.workunit.client.0.smithi086.stdout: 10 31342 35.85 MB/sec warmup 52 sec latency 41.219 ms 2024-03-23T18:04:28.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:04:27 smithi086 ceph-mon[38221]: pgmap v435: 97 pgs: 97 active+clean; 634 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 8.4 MiB/s wr, 673 op/s 2024-03-23T18:04:28.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.276+0000 7f0ca5e00700 1 -- 172.21.15.86:0/1134393739 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ca0107530 msgr2=0x7f0ca0107910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:04:28.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.276+0000 7f0ca5e00700 1 --2- 172.21.15.86:0/1134393739 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ca0107530 0x7f0ca0107910 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0c90009990 tx=0x7f0c90009ca0 comp rx=0 tx=0).stop 2024-03-23T18:04:28.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.276+0000 7f0ca5e00700 1 -- 172.21.15.86:0/1134393739 shutdown_connections 2024-03-23T18:04:28.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.276+0000 7f0ca5e00700 1 --2- 172.21.15.86:0/1134393739 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ca010c650 0x7f0ca01141d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:28.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.276+0000 7f0ca5e00700 1 --2- 172.21.15.86:0/1134393739 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ca0107530 0x7f0ca0107910 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:28.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.276+0000 7f0ca5e00700 1 --2- 172.21.15.86:0/1134393739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ca0107ee0 0x7f0ca010bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:28.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.276+0000 7f0ca5e00700 1 -- 172.21.15.86:0/1134393739 >> 172.21.15.86:0/1134393739 conn(0x7f0ca0076ab0 msgr2=0x7f0ca0078ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:04:28.277 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.276+0000 7f0ca5e00700 1 -- 172.21.15.86:0/1134393739 shutdown_connections 2024-03-23T18:04:28.277 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.276+0000 7f0ca5e00700 1 -- 172.21.15.86:0/1134393739 wait complete. 2024-03-23T18:04:28.278 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.278+0000 7f0ca5e00700 1 Processor -- start 2024-03-23T18:04:28.278 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.278+0000 7f0ca5e00700 1 -- start start 2024-03-23T18:04:28.279 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.278+0000 7f0ca5e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ca010c650 0x7f0ca01a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:04:28.279 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.278+0000 7f0ca5e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ca01a3d40 0x7f0ca019d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:04:28.279 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.279+0000 7f0ca5e00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ca019de70 0x7f0ca019e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:04:28.279 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.279+0000 7f0ca5e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca01a4340 con 0x7f0ca010c650 2024-03-23T18:04:28.279 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.279+0000 7f0ca5e00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca01aa610 con 0x7f0ca01a3d40 2024-03-23T18:04:28.279 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.279+0000 7f0ca5e00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca01aa750 con 0x7f0ca019de70 2024-03-23T18:04:28.280 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.279+0000 7f0c9f400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ca019de70 0x7f0ca019e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:04:28.280 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.279+0000 7f0c9f400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ca019de70 0x7f0ca019e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:48552/0 (socket says 172.21.15.86:48552) 2024-03-23T18:04:28.280 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.279+0000 7f0c9f400700 1 -- 172.21.15.86:0/2723198035 learned_addr learned my addr 172.21.15.86:0/2723198035 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:04:28.280 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.279+0000 7f0c9e000700 1 --2- 172.21.15.86:0/2723198035 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ca01a3d40 0x7f0ca019d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:04:28.280 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.279+0000 7f0c9f400700 1 -- 172.21.15.86:0/2723198035 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ca01a3d40 msgr2=0x7f0ca019d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:04:28.280 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.279+0000 7f0c9f400700 1 --2- 172.21.15.86:0/2723198035 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ca01a3d40 0x7f0ca019d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:28.280 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.279+0000 7f0c9f400700 1 -- 172.21.15.86:0/2723198035 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ca010c650 msgr2=0x7f0ca01a3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:04:28.281 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.279+0000 7f0c9f400700 1 --2- 172.21.15.86:0/2723198035 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ca010c650 0x7f0ca01a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:28.281 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.279+0000 7f0c9f400700 1 -- 172.21.15.86:0/2723198035 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c900095f0 con 0x7f0ca019de70 2024-03-23T18:04:28.281 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.280+0000 7f0c9f400700 1 --2- 172.21.15.86:0/2723198035 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ca019de70 0x7f0ca019e300 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0c9800b9e0 tx=0x7f0c9800bcf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:04:28.281 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.280+0000 7f0ca4a00700 1 -- 172.21.15.86:0/2723198035 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c9800d8e0 con 0x7f0ca019de70 2024-03-23T18:04:28.281 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.280+0000 7f0ca4a00700 1 -- 172.21.15.86:0/2723198035 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c98015420 con 0x7f0ca019de70 2024-03-23T18:04:28.281 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.280+0000 7f0ca4a00700 1 -- 172.21.15.86:0/2723198035 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c98014630 con 0x7f0ca019de70 2024-03-23T18:04:28.281 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.280+0000 7f0ca5e00700 1 -- 172.21.15.86:0/2723198035 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ca01aaa30 con 0x7f0ca019de70 2024-03-23T18:04:28.281 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.280+0000 7f0ca5e00700 1 -- 172.21.15.86:0/2723198035 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ca01aaf40 con 0x7f0ca019de70 2024-03-23T18:04:28.285 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.284+0000 7f0ca4a00700 1 -- 172.21.15.86:0/2723198035 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f0c98015590 con 0x7f0ca019de70 2024-03-23T18:04:28.286 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.286+0000 7f0ca4a00700 1 --2- 172.21.15.86:0/2723198035 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f0c840a1310 0x7f0c840a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:04:28.286 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.286+0000 7f0ca4a00700 1 -- 172.21.15.86:0/2723198035 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f0c98011070 con 0x7f0ca019de70 2024-03-23T18:04:28.286 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.286+0000 7f0c9ea00700 1 --2- 172.21.15.86:0/2723198035 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f0c840a1310 0x7f0c840a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:04:28.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.288+0000 7f0c9ea00700 1 --2- 172.21.15.86:0/2723198035 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f0c840a1310 0x7f0c840a37d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0c900098e0 tx=0x7f0c90005fb0 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:04:28.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.288+0000 7f0cb9949b80 1 Processor -- start 2024-03-23T18:04:28.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.288+0000 7f0cb9949b80 1 -- start start 2024-03-23T18:04:28.291 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.290+0000 7f0cb9949b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3c07f4100 0x55f3c07f44e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:04:28.291 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.290+0000 7f0cb9949b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f3c07f4a20 0x55f3c07ff390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:04:28.291 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.290+0000 7f0cb9949b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3c07ff8d0 0x55f3c0801cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:04:28.291 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.290+0000 7f0cb9949b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3c074d930 con 0x55f3c07ff8d0 2024-03-23T18:04:28.291 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.290+0000 7f0cb9949b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3c074b2f0 con 0x55f3c07f4100 2024-03-23T18:04:28.291 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.290+0000 7f0cb9949b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3c0746be0 con 0x55f3c07f4a20 2024-03-23T18:04:28.291 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.291+0000 7f0c9ea00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3c07f4100 0x55f3c07f44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:04:28.291 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.291+0000 7f0c9ea00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3c07f4100 0x55f3c07f44e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:44162/0 (socket says 172.21.15.86:44162) 2024-03-23T18:04:28.291 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.291+0000 7f0c9ea00700 1 -- 172.21.15.86:0/3802137767 learned_addr learned my addr 172.21.15.86:0/3802137767 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:04:28.291 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.291+0000 7f0c9e000700 1 --2- 172.21.15.86:0/3802137767 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f3c07f4a20 0x55f3c07ff390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:04:28.291 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.291+0000 7f0c9f400700 1 --2- 172.21.15.86:0/3802137767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3c07ff8d0 0x55f3c0801cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:04:28.292 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.291+0000 7f0c9ea00700 1 -- 172.21.15.86:0/3802137767 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f3c07f4a20 msgr2=0x55f3c07ff390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:04:28.292 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.291+0000 7f0c9ea00700 1 --2- 172.21.15.86:0/3802137767 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f3c07f4a20 0x55f3c07ff390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:28.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.291+0000 7f0c9ea00700 1 -- 172.21.15.86:0/3802137767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3c07ff8d0 msgr2=0x55f3c0801cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:04:28.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.291+0000 7f0c9ea00700 1 --2- 172.21.15.86:0/3802137767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3c07ff8d0 0x55f3c0801cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:28.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.291+0000 7f0c9ea00700 1 -- 172.21.15.86:0/3802137767 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f3c05c8350 con 0x55f3c07f4100 2024-03-23T18:04:28.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.291+0000 7f0c9e000700 1 --2- 172.21.15.86:0/3802137767 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f3c07f4a20 0x55f3c07ff390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T18:04:28.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.292+0000 7f0c9ea00700 1 --2- 172.21.15.86:0/3802137767 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3c07f4100 0x55f3c07f44e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f0c90009990 tx=0x7f0c9001d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=844133691829ea8e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:04:28.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.292+0000 7f0c8be00700 1 -- 172.21.15.86:0/3802137767 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c9001d070 con 0x55f3c07f4100 2024-03-23T18:04:28.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.292+0000 7f0c8be00700 1 -- 172.21.15.86:0/3802137767 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c9000beb0 con 0x55f3c07f4100 2024-03-23T18:04:28.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.292+0000 7f0c8be00700 1 -- 172.21.15.86:0/3802137767 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c9000f6c0 con 0x55f3c07f4100 2024-03-23T18:04:28.294 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.293+0000 7f0cb9949b80 1 -- 172.21.15.86:0/3802137767 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3c07f4100 msgr2=0x55f3c07f44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:04:28.294 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.293+0000 7f0cb9949b80 1 --2- 172.21.15.86:0/3802137767 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3c07f4100 0x55f3c07f44e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f0c90009990 tx=0x7f0c9001d000 comp rx=0 tx=0).stop 2024-03-23T18:04:28.294 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.293+0000 7f0cb9949b80 1 -- 172.21.15.86:0/3802137767 shutdown_connections 2024-03-23T18:04:28.294 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.293+0000 7f0cb9949b80 1 --2- 172.21.15.86:0/3802137767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3c07ff8d0 0x55f3c0801cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:28.294 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.293+0000 7f0cb9949b80 1 --2- 172.21.15.86:0/3802137767 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3c07f4100 0x55f3c07f44e0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:28.294 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.293+0000 7f0cb9949b80 1 --2- 172.21.15.86:0/3802137767 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f3c07f4a20 0x55f3c07ff390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:28.294 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.293+0000 7f0cb9949b80 1 -- 172.21.15.86:0/3802137767 >> 172.21.15.86:0/3802137767 conn(0x55f3c0754640 msgr2=0x55f3c07f0520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:04:28.294 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.293+0000 7f0cb9949b80 1 -- 172.21.15.86:0/3802137767 shutdown_connections 2024-03-23T18:04:28.295 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.293+0000 7f0cb9949b80 1 -- 172.21.15.86:0/3802137767 wait complete. 2024-03-23T18:04:28.295 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.294+0000 7f0cb9949b80 1 -- start start 2024-03-23T18:04:28.295 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.294+0000 7f0cb9949b80 1 Processor -- start 2024-03-23T18:04:28.296 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.296+0000 7f0cb9949b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f3c07f4100 0x55f3c07ff2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:04:28.296 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.296+0000 7f0cb9949b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3c07f4a20 0x55f3c07fc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:04:28.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.296+0000 7f0cb9949b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3c07ff8d0 0x55f3c07fcb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:04:28.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.296+0000 7f0cb9949b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3c06e2d20 con 0x55f3c07ff8d0 2024-03-23T18:04:28.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.296+0000 7f0cb9949b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3c06e29c0 con 0x55f3c07f4a20 2024-03-23T18:04:28.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.296+0000 7f0cb9949b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3c05ca640 con 0x55f3c07f4100 2024-03-23T18:04:28.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.296+0000 7f0cb9949b80 10 client.0 _ll_register_callbacks cb 0x55f3c06ec410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T18:04:28.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.297+0000 7f0c9ea00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f3c07f4100 0x55f3c07ff2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:04:28.298 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.297+0000 7f0c9e000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3c07f4a20 0x55f3c07fc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:04:28.298 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.297+0000 7f0c9e000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3c07f4a20 0x55f3c07fc560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:44178/0 (socket says 172.21.15.86:44178) 2024-03-23T18:04:28.298 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.297+0000 7f0c9e000700 1 -- 172.21.15.86:0/2426354458 learned_addr learned my addr 172.21.15.86:0/2426354458 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:04:28.298 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.297+0000 7f0c9e000700 1 -- 172.21.15.86:0/2426354458 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f3c07f4100 msgr2=0x55f3c07ff2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:04:28.298 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.297+0000 7f0c9e000700 1 --2- 172.21.15.86:0/2426354458 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f3c07f4100 0x55f3c07ff2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:28.298 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.297+0000 7f0c9e000700 1 -- 172.21.15.86:0/2426354458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3c07ff8d0 msgr2=0x55f3c07fcb90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-23T18:04:28.299 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.298+0000 7f0c9f400700 1 --2- 172.21.15.86:0/2426354458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3c07ff8d0 0x55f3c07fcb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:04:28.299 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.298+0000 7f0c9e000700 1 --2- 172.21.15.86:0/2426354458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3c07ff8d0 0x55f3c07fcb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:28.299 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.298+0000 7f0c9e000700 1 -- 172.21.15.86:0/2426354458 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c8c009650 con 0x55f3c07f4a20 2024-03-23T18:04:28.299 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.299+0000 7f0c9e000700 1 --2- 172.21.15.86:0/2426354458 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3c07f4a20 0x55f3c07fc560 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c0118e0 tx=0x7f0c8c011ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9e309f989beee32 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:04:28.299 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.299+0000 7f0c88c00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T18:04:28.299 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.299+0000 7f0c88c00700 1 -- 172.21.15.86:0/2426354458 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c8c00f730 con 0x55f3c07f4a20 2024-03-23T18:04:28.299 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.299+0000 7f0c88c00700 1 -- 172.21.15.86:0/2426354458 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f0c8c00f890 con 0x55f3c07f4a20 2024-03-23T18:04:28.299 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.299+0000 7f0cb9949b80 1 -- 172.21.15.86:0/2426354458 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f3c0707320 con 0x55f3c07f4a20 2024-03-23T18:04:28.300 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.299+0000 7f0c88c00700 1 -- 172.21.15.86:0/2426354458 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c8c00fa50 con 0x55f3c07f4a20 2024-03-23T18:04:28.300 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.299+0000 7f0cb9949b80 10 client.27143 fetch_fsmap learned FSMap version 65 2024-03-23T18:04:28.300 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.299+0000 7f0cb9949b80 1 -- 172.21.15.86:0/2426354458 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) v3 -- 0x7f0ca0114a40 con 0x55f3c07f4a20 2024-03-23T18:04:28.301 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.301+0000 7f0c88c00700 1 -- 172.21.15.86:0/2426354458 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f3c0707320 con 0x55f3c07f4a20 2024-03-23T18:04:28.301 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.301+0000 7f0c88c00700 1 -- 172.21.15.86:0/2426354458 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 65) v1 ==== 7522+0+0 (secure 0 0 0) 0x7f0c8c01b020 con 0x55f3c07f4a20 2024-03-23T18:04:28.301 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.301+0000 7f0cb9949b80 10 client.27143 fetch_fsmap finished waiting for FSMap version 65 2024-03-23T18:04:28.301 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.301+0000 7f0cb9949b80 10 client.27143 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T18:04:28.301 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.301+0000 7f0cb9949b80 20 client.27143 populate_metadata read hostname 'smithi086' 2024-03-23T18:04:28.301 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.301+0000 7f0cb9949b80 1 --2- 172.21.15.86:0/2426354458 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55f3c080f5f0 0x55f3c0811a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:04:28.301 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.301+0000 7f0cb9949b80 4 client.27143 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T18:04:28.301 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.301+0000 7f0cb9949b80 1 -- 172.21.15.86:0/2426354458 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f3c073f6a0 con 0x55f3c080f5f0 2024-03-23T18:04:28.301 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.301+0000 7f0c9ea00700 1 --2- 172.21.15.86:0/2426354458 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55f3c080f5f0 0x55f3c0811a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:04:28.305 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.305+0000 7f0c9ea00700 1 --2- 172.21.15.86:0/2426354458 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55f3c080f5f0 0x55f3c0811a10 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=804fab55dbb6965f server_cookie=82cbf3c12cb28a7f in_seq=0 out_seq=0 2024-03-23T18:04:28.305 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.305+0000 7f0c88c00700 10 client.27143 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T18:04:28.307 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.306+0000 7f0c88c00700 1 -- 172.21.15.86:0/2426354458 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55f3c073f6a0 con 0x55f3c080f5f0 2024-03-23T18:04:28.307 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.306+0000 7f0c88c00700 10 client.27143 handle_command_reply: tid=0 2024-03-23T18:04:28.307 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.307+0000 7f0cb9949b80 1 client.27143 shutdown 2024-03-23T18:04:28.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:04:27 smithi092 ceph-mon[79089]: pgmap v435: 97 pgs: 97 active+clean; 634 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 8.4 MiB/s wr, 673 op/s 2024-03-23T18:04:28.308 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.307+0000 7f0cb9949b80 1 -- 172.21.15.86:0/2426354458 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3c07f4a20 msgr2=0x55f3c07fc560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:04:28.308 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.307+0000 7f0cb9949b80 1 --2- 172.21.15.86:0/2426354458 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3c07f4a20 0x55f3c07fc560 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c0118e0 tx=0x7f0c8c011ca0 comp rx=0 tx=0).stop 2024-03-23T18:04:28.308 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.307+0000 7f0cb9949b80 1 -- 172.21.15.86:0/2426354458 shutdown_connections 2024-03-23T18:04:28.308 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.307+0000 7f0cb9949b80 1 --2- 172.21.15.86:0/2426354458 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55f3c080f5f0 0x55f3c0811a10 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:28.308 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.307+0000 7f0cb9949b80 1 --2- 172.21.15.86:0/2426354458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3c07ff8d0 0x55f3c07fcb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:28.308 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.307+0000 7f0cb9949b80 1 --2- 172.21.15.86:0/2426354458 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f3c07f4100 0x55f3c07ff2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:28.308 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.307+0000 7f0cb9949b80 1 --2- 172.21.15.86:0/2426354458 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3c07f4a20 0x55f3c07fc560 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:28.308 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.307+0000 7f0c88c00700 0 client.27143 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T18:04:28.308 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.307+0000 7f0cb9949b80 1 -- 172.21.15.86:0/2426354458 >> 172.21.15.86:0/2426354458 conn(0x55f3c0754640 msgr2=0x55f3c07f60a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:04:28.308 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.308+0000 7f0cb9949b80 1 -- 172.21.15.86:0/2426354458 shutdown_connections 2024-03-23T18:04:28.308 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.308+0000 7f0cb9949b80 1 -- 172.21.15.86:0/2426354458 wait complete. 2024-03-23T18:04:28.309 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.308+0000 7f0cb9949b80 20 client.27143 trim_cache size 0 max 16384 2024-03-23T18:04:28.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:04:27 smithi152 ceph-mon[47949]: pgmap v435: 97 pgs: 97 active+clean; 634 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 8.4 MiB/s wr, 673 op/s 2024-03-23T18:04:28.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.338+0000 7f0cb9949b80 1 Processor -- start 2024-03-23T18:04:28.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.338+0000 7f0cb9949b80 1 -- start start 2024-03-23T18:04:28.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.338+0000 7f0cb9949b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f3c07f4100 0x55f3c0806580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:04:28.342 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.338+0000 7f0cb9949b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3c07f4a20 0x55f3c07fc520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:04:28.342 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.339+0000 7f0cb9949b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3c07ff8d0 0x55f3c07ea500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:04:28.342 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.339+0000 7f0cb9949b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3c07f18b0 con 0x55f3c07ff8d0 2024-03-23T18:04:28.342 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.339+0000 7f0cb9949b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3c07fd470 con 0x55f3c07f4a20 2024-03-23T18:04:28.343 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.339+0000 7f0cb9949b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3c07fced0 con 0x55f3c07f4100 2024-03-23T18:04:28.345 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.339+0000 7f0c9ea00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f3c07f4100 0x55f3c0806580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:04:28.345 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.339+0000 7f0c9ea00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f3c07f4100 0x55f3c0806580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:48580/0 (socket says 172.21.15.86:48580) 2024-03-23T18:04:28.345 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.339+0000 7f0c9ea00700 1 -- 172.21.15.86:0/3189516966 learned_addr learned my addr 172.21.15.86:0/3189516966 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:04:28.345 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.339+0000 7f0c9ea00700 1 -- 172.21.15.86:0/3189516966 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3c07f4a20 msgr2=0x55f3c07fc520 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-23T18:04:28.345 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.339+0000 7f0c9ea00700 1 --2- 172.21.15.86:0/3189516966 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3c07f4a20 0x55f3c07fc520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:28.345 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.339+0000 7f0c9ea00700 1 -- 172.21.15.86:0/3189516966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3c07ff8d0 msgr2=0x55f3c07ea500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:04:28.345 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.339+0000 7f0c9ea00700 1 --2- 172.21.15.86:0/3189516966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3c07ff8d0 0x55f3c07ea500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:28.345 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.339+0000 7f0c9ea00700 1 -- 172.21.15.86:0/3189516966 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f3c0574b00 con 0x55f3c07f4100 2024-03-23T18:04:28.345 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.340+0000 7f0c9ea00700 1 --2- 172.21.15.86:0/3189516966 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f3c07f4100 0x55f3c0806580 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0c90000c00 tx=0x7f0c9002fe10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dfca42f430bdee38 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:04:28.345 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.340+0000 7f0c95200700 1 -- 172.21.15.86:0/3189516966 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c90005230 con 0x55f3c07f4100 2024-03-23T18:04:28.346 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.340+0000 7f0c95200700 1 -- 172.21.15.86:0/3189516966 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c900373c0 con 0x55f3c07f4100 2024-03-23T18:04:28.346 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.340+0000 7f0c95200700 1 -- 172.21.15.86:0/3189516966 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c90037a10 con 0x55f3c07f4100 2024-03-23T18:04:28.346 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.341+0000 7f0cb9949b80 1 -- 172.21.15.86:0/3189516966 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f3c07f4100 msgr2=0x55f3c0806580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:04:28.346 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.341+0000 7f0cb9949b80 1 --2- 172.21.15.86:0/3189516966 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f3c07f4100 0x55f3c0806580 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0c90000c00 tx=0x7f0c9002fe10 comp rx=0 tx=0).stop 2024-03-23T18:04:28.346 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.341+0000 7f0cb9949b80 1 -- 172.21.15.86:0/3189516966 shutdown_connections 2024-03-23T18:04:28.346 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.341+0000 7f0cb9949b80 1 --2- 172.21.15.86:0/3189516966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3c07ff8d0 0x55f3c07ea500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:28.346 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.341+0000 7f0cb9949b80 1 --2- 172.21.15.86:0/3189516966 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f3c07f4100 0x55f3c0806580 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:28.346 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.341+0000 7f0cb9949b80 1 --2- 172.21.15.86:0/3189516966 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3c07f4a20 0x55f3c07fc520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:28.346 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.341+0000 7f0cb9949b80 1 -- 172.21.15.86:0/3189516966 >> 172.21.15.86:0/3189516966 conn(0x55f3c0754640 msgr2=0x55f3c07e93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:04:28.346 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.341+0000 7f0cb9949b80 1 -- 172.21.15.86:0/3189516966 shutdown_connections 2024-03-23T18:04:28.346 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.341+0000 7f0cb9949b80 1 -- 172.21.15.86:0/3189516966 wait complete. 2024-03-23T18:04:28.346 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.342+0000 7f0cb9949b80 1 -- start start 2024-03-23T18:04:28.347 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.343+0000 7f0cb9949b80 1 Processor -- start 2024-03-23T18:04:28.347 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.343+0000 7f0cb9949b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3c07f4100 0x55f3c0804750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:04:28.347 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.343+0000 7f0cb9949b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f3c07f4a20 0x55f3c0804c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:04:28.347 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.343+0000 7f0cb9949b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3c07ff8d0 0x55f3c0860110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:04:28.347 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.343+0000 7f0cb9949b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3c0860650 con 0x55f3c07ff8d0 2024-03-23T18:04:28.347 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.343+0000 7f0cb9949b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3c0860790 con 0x55f3c07f4100 2024-03-23T18:04:28.347 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.343+0000 7f0cb9949b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3c08608d0 con 0x55f3c07f4a20 2024-03-23T18:04:28.347 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.344+0000 7f0cb9949b80 10 client.0 _ll_register_callbacks cb 0x55f3c06ec410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T18:04:28.347 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.344+0000 7f0c9e000700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f3c07f4a20 0x55f3c0804c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:04:28.347 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.344+0000 7f0c9e000700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f3c07f4a20 0x55f3c0804c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:48588/0 (socket says 172.21.15.86:48588) 2024-03-23T18:04:28.347 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.344+0000 7f0c9e000700 1 -- 172.21.15.86:0/921906788 learned_addr learned my addr 172.21.15.86:0/921906788 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:04:28.347 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.344+0000 7f0c9f400700 1 --2- 172.21.15.86:0/921906788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3c07ff8d0 0x55f3c0860110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:04:28.347 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.344+0000 7f0c9e000700 1 -- 172.21.15.86:0/921906788 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3c07f4100 msgr2=0x55f3c0804750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:04:28.348 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.344+0000 7f0c9e000700 1 --2- 172.21.15.86:0/921906788 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3c07f4100 0x55f3c0804750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:28.348 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.344+0000 7f0c9e000700 1 -- 172.21.15.86:0/921906788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3c07ff8d0 msgr2=0x55f3c0860110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:04:28.348 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.344+0000 7f0c9e000700 1 --2- 172.21.15.86:0/921906788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3c07ff8d0 0x55f3c0860110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:28.348 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.344+0000 7f0c9e000700 1 -- 172.21.15.86:0/921906788 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f3c05c8350 con 0x55f3c07f4a20 2024-03-23T18:04:28.348 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.344+0000 7f0c9f400700 1 --2- 172.21.15.86:0/921906788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3c07ff8d0 0x55f3c0860110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T18:04:28.348 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.345+0000 7f0c88c00700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-03-23T18:04:28.348 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.345+0000 7f0c9e000700 1 --2- 172.21.15.86:0/921906788 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f3c07f4a20 0x55f3c0804c90 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c014ed0 tx=0x7f0c8c009e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0e01ba3aeefdd5 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:04:28.349 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.346+0000 7f0c88c00700 1 -- 172.21.15.86:0/921906788 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c8c009280 con 0x55f3c07f4a20 2024-03-23T18:04:28.349 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.346+0000 7f0c88c00700 1 -- 172.21.15.86:0/921906788 <== mon.1 v2:172.21.15.152:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f0c8c000e40 con 0x55f3c07f4a20 2024-03-23T18:04:28.349 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.346+0000 7f0cb9949b80 1 -- 172.21.15.86:0/921906788 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0c9000beb0 con 0x55f3c07f4a20 2024-03-23T18:04:28.349 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.347+0000 7f0c88c00700 1 -- 172.21.15.86:0/921906788 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c8c0094f0 con 0x55f3c07f4a20 2024-03-23T18:04:28.349 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.347+0000 7f0cb9949b80 10 client.27124 fetch_fsmap learned FSMap version 65 2024-03-23T18:04:28.349 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.347+0000 7f0cb9949b80 1 -- 172.21.15.86:0/921906788 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) v3 -- 0x55f3c070ce40 con 0x55f3c07f4a20 2024-03-23T18:04:28.349 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.348+0000 7f0c88c00700 1 -- 172.21.15.86:0/921906788 <== mon.1 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c9000beb0 con 0x55f3c07f4a20 2024-03-23T18:04:28.349 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.348+0000 7f0c88c00700 1 -- 172.21.15.86:0/921906788 <== mon.1 v2:172.21.15.152:3300/0 5 ==== fsmap(e 65) v1 ==== 7522+0+0 (secure 0 0 0) 0x7f0c8c036050 con 0x55f3c07f4a20 2024-03-23T18:04:28.350 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.348+0000 7f0cb9949b80 10 client.27124 fetch_fsmap finished waiting for FSMap version 65 2024-03-23T18:04:28.350 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.348+0000 7f0cb9949b80 10 client.27124 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T18:04:28.350 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.348+0000 7f0cb9949b80 20 client.27124 populate_metadata read hostname 'smithi086' 2024-03-23T18:04:28.350 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.348+0000 7f0cb9949b80 1 --2- 172.21.15.86:0/921906788 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55f3c0861ec0 0x55f3c08642e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:04:28.350 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.348+0000 7f0cb9949b80 4 client.27124 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T18:04:28.350 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.348+0000 7f0cb9949b80 1 -- 172.21.15.86:0/921906788 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f3c0595c90 con 0x55f3c0861ec0 2024-03-23T18:04:28.350 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.349+0000 7f0c9ea00700 1 --2- 172.21.15.86:0/921906788 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55f3c0861ec0 0x55f3c08642e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:04:28.350 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.350+0000 7f0c9ea00700 1 --2- 172.21.15.86:0/921906788 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55f3c0861ec0 0x55f3c08642e0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0e5bb11b125d0cb server_cookie=971ee85cffd92870 in_seq=0 out_seq=0 2024-03-23T18:04:28.350 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.350+0000 7f0c88c00700 10 client.27124 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T18:04:28.595 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.595+0000 7f0c88c00700 1 -- 172.21.15.86:0/921906788 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f3c0595c90 con 0x55f3c0861ec0 2024-03-23T18:04:28.596 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.595+0000 7f0c88c00700 10 client.27124 handle_command_reply: tid=0 2024-03-23T18:04:28.596 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.595+0000 7f0cb9949b80 1 client.27124 shutdown 2024-03-23T18:04:28.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.596+0000 7f0cb9949b80 1 -- 172.21.15.86:0/921906788 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f3c07f4a20 msgr2=0x55f3c0804c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:04:28.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.596+0000 7f0cb9949b80 1 --2- 172.21.15.86:0/921906788 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f3c07f4a20 0x55f3c0804c90 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c014ed0 tx=0x7f0c8c009e00 comp rx=0 tx=0).stop 2024-03-23T18:04:28.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.596+0000 7f0cb9949b80 1 -- 172.21.15.86:0/921906788 shutdown_connections 2024-03-23T18:04:28.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.596+0000 7f0cb9949b80 1 --2- 172.21.15.86:0/921906788 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55f3c0861ec0 0x55f3c08642e0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:28.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.596+0000 7f0c88c00700 0 client.27124 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T18:04:28.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.597+0000 7f0cb9949b80 1 --2- 172.21.15.86:0/921906788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3c07ff8d0 0x55f3c0860110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:28.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.597+0000 7f0cb9949b80 1 --2- 172.21.15.86:0/921906788 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3c07f4100 0x55f3c0804750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:28.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.597+0000 7f0cb9949b80 1 --2- 172.21.15.86:0/921906788 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f3c07f4a20 0x55f3c0804c90 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:28.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.597+0000 7f0cb9949b80 1 -- 172.21.15.86:0/921906788 >> 172.21.15.86:0/921906788 conn(0x55f3c0754640 msgr2=0x55f3c07f0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:04:28.599 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.599+0000 7f0cb9949b80 1 -- 172.21.15.86:0/921906788 shutdown_connections 2024-03-23T18:04:28.599 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.599+0000 7f0cb9949b80 1 -- 172.21.15.86:0/921906788 wait complete. 2024-03-23T18:04:28.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.600+0000 7f0cb9949b80 20 client.27124 trim_cache size 0 max 16384 2024-03-23T18:04:28.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-03-23T18:04:28.601 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-03-23T18:04:28.601 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-03-23T18:04:28.601 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-03-23T18:04:28.603 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.603+0000 7f0ca5e00700 1 -- 172.21.15.86:0/2723198035 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f0c840a1310 msgr2=0x7f0c840a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:04:28.603 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.604+0000 7f0ca5e00700 1 --2- 172.21.15.86:0/2723198035 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f0c840a1310 0x7f0c840a37d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0c900098e0 tx=0x7f0c90005fb0 comp rx=0 tx=0).stop 2024-03-23T18:04:28.604 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.604+0000 7f0ca5e00700 1 -- 172.21.15.86:0/2723198035 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ca019de70 msgr2=0x7f0ca019e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:04:28.604 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.604+0000 7f0ca5e00700 1 --2- 172.21.15.86:0/2723198035 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ca019de70 0x7f0ca019e300 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0c9800b9e0 tx=0x7f0c9800bcf0 comp rx=0 tx=0).stop 2024-03-23T18:04:28.604 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.604+0000 7f0ca5e00700 1 -- 172.21.15.86:0/2723198035 shutdown_connections 2024-03-23T18:04:28.605 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.604+0000 7f0ca5e00700 1 --2- 172.21.15.86:0/2723198035 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f0c840a1310 0x7f0c840a37d0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:28.605 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.604+0000 7f0ca5e00700 1 --2- 172.21.15.86:0/2723198035 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ca019de70 0x7f0ca019e300 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:28.605 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.604+0000 7f0ca5e00700 1 --2- 172.21.15.86:0/2723198035 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ca010c650 0x7f0ca01a3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:28.605 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.604+0000 7f0ca5e00700 1 --2- 172.21.15.86:0/2723198035 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ca01a3d40 0x7f0ca019d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:28.605 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.605+0000 7f0ca5e00700 1 -- 172.21.15.86:0/2723198035 >> 172.21.15.86:0/2723198035 conn(0x7f0ca0076ab0 msgr2=0x7f0ca0077f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:04:28.605 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.605+0000 7f0ca5e00700 1 -- 172.21.15.86:0/2723198035 shutdown_connections 2024-03-23T18:04:28.606 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:28.605+0000 7f0ca5e00700 1 -- 172.21.15.86:0/2723198035 wait complete. 2024-03-23T18:04:29.227 INFO:tasks.workunit.client.0.smithi086.stdout: 10 32094 35.58 MB/sec warmup 53 sec latency 207.227 ms 2024-03-23T18:04:29.335 INFO:journalctl@ceph.mds.c.smithi152.stdout:Mar 23 18:04:29 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-c[84552]: ignoring --setuser ceph since I am not root 2024-03-23T18:04:29.336 INFO:journalctl@ceph.mds.c.smithi152.stdout:Mar 23 18:04:29 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-c[84552]: ignoring --setgroup ceph since I am not root 2024-03-23T18:04:29.336 INFO:journalctl@ceph.mds.c.smithi152.stdout:Mar 23 18:04:29 smithi152 ceph-mds[84575]: -- 172.21.15.152:0/957279913 <== mon.1 v2:172.21.15.152:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558e8d3f81a0 con 0x558e8d3ce800 2024-03-23T18:04:29.336 INFO:journalctl@ceph.mds.c.smithi152.stdout:Mar 23 18:04:29 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-c[84552]: starting mds.c at 2024-03-23T18:04:30.227 INFO:tasks.workunit.client.0.smithi086.stdout: 10 33303 36.51 MB/sec warmup 54 sec latency 40.874 ms 2024-03-23T18:04:30.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:04:29 smithi086 ceph-mon[38221]: pgmap v436: 97 pgs: 97 active+clean; 634 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 7.5 MiB/s wr, 599 op/s 2024-03-23T18:04:30.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:04:29 smithi086 ceph-mon[38221]: daemon mds.c restarted 2024-03-23T18:04:30.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:04:29 smithi086 ceph-mon[38221]: osdmap e102: 12 total, 12 up, 12 in 2024-03-23T18:04:30.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:04:29 smithi092 ceph-mon[79089]: pgmap v436: 97 pgs: 97 active+clean; 634 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 7.5 MiB/s wr, 599 op/s 2024-03-23T18:04:30.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:04:29 smithi092 ceph-mon[79089]: daemon mds.c restarted 2024-03-23T18:04:30.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:04:29 smithi092 ceph-mon[79089]: osdmap e102: 12 total, 12 up, 12 in 2024-03-23T18:04:30.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:04:29 smithi152 ceph-mon[47949]: pgmap v436: 97 pgs: 97 active+clean; 634 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 7.5 MiB/s wr, 599 op/s 2024-03-23T18:04:30.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:04:29 smithi152 ceph-mon[47949]: daemon mds.c restarted 2024-03-23T18:04:30.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:04:29 smithi152 ceph-mon[47949]: osdmap e102: 12 total, 12 up, 12 in 2024-03-23T18:04:31.227 INFO:tasks.workunit.client.0.smithi086.stdout: 10 33594 36.13 MB/sec warmup 55 sec latency 447.349 ms 2024-03-23T18:04:31.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:04:31 smithi086 ceph-mon[38221]: mds.? [v2:172.21.15.152:6836/3825827660,v1:172.21.15.152:6837/3825827660] up:boot 2024-03-23T18:04:31.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:04:31 smithi086 ceph-mon[38221]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-23T18:04:31.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:04:31 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-23T18:04:31.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:04:31 smithi086 ceph-mon[38221]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:04:31.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:04:31 smithi092 ceph-mon[79089]: mds.? [v2:172.21.15.152:6836/3825827660,v1:172.21.15.152:6837/3825827660] up:boot 2024-03-23T18:04:31.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:04:31 smithi092 ceph-mon[79089]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-23T18:04:31.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:04:31 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-23T18:04:31.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:04:31 smithi092 ceph-mon[79089]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:04:31.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:04:31 smithi152 ceph-mon[47949]: mds.? [v2:172.21.15.152:6836/3825827660,v1:172.21.15.152:6837/3825827660] up:boot 2024-03-23T18:04:31.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:04:31 smithi152 ceph-mon[47949]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-23T18:04:31.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:04:31 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-23T18:04:31.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:04:31 smithi152 ceph-mon[47949]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:04:32.227 INFO:tasks.workunit.client.0.smithi086.stdout: 10 34000 35.92 MB/sec warmup 56 sec latency 328.972 ms 2024-03-23T18:04:32.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:04:32 smithi086 ceph-mon[38221]: pgmap v438: 97 pgs: 97 active+clean; 649 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 924 op/s 2024-03-23T18:04:32.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:04:32 smithi092 ceph-mon[79089]: pgmap v438: 97 pgs: 97 active+clean; 649 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 924 op/s 2024-03-23T18:04:32.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:04:32 smithi152 ceph-mon[47949]: pgmap v438: 97 pgs: 97 active+clean; 649 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 924 op/s 2024-03-23T18:04:33.227 INFO:tasks.workunit.client.0.smithi086.stdout: 10 34679 35.82 MB/sec warmup 57 sec latency 93.778 ms 2024-03-23T18:04:34.227 INFO:tasks.workunit.client.0.smithi086.stdout: 10 35589 36.04 MB/sec warmup 58 sec latency 61.121 ms 2024-03-23T18:04:34.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:04:34 smithi086 ceph-mon[38221]: pgmap v439: 97 pgs: 97 active+clean; 649 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 924 op/s 2024-03-23T18:04:34.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:04:34 smithi092 ceph-mon[79089]: pgmap v439: 97 pgs: 97 active+clean; 649 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 924 op/s 2024-03-23T18:04:34.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:04:34 smithi152 ceph-mon[47949]: pgmap v439: 97 pgs: 97 active+clean; 649 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 924 op/s 2024-03-23T18:04:35.227 INFO:tasks.workunit.client.0.smithi086.stdout: 10 36448 36.18 MB/sec warmup 59 sec latency 81.641 ms 2024-03-23T18:04:36.227 INFO:tasks.workunit.client.0.smithi086.stdout: 10 37251 36.38 MB/sec warmup 60 sec latency 127.883 ms 2024-03-23T18:04:36.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:04:36 smithi086 ceph-mon[38221]: pgmap v440: 97 pgs: 97 active+clean; 659 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 19 MiB/s wr, 1.20k op/s 2024-03-23T18:04:36.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:04:36 smithi092 ceph-mon[79089]: pgmap v440: 97 pgs: 97 active+clean; 659 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 19 MiB/s wr, 1.20k op/s 2024-03-23T18:04:36.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:04:36 smithi152 ceph-mon[47949]: pgmap v440: 97 pgs: 97 active+clean; 659 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 19 MiB/s wr, 1.20k op/s 2024-03-23T18:04:37.227 INFO:tasks.workunit.client.0.smithi086.stdout: 10 38099 36.49 MB/sec warmup 61 sec latency 65.293 ms 2024-03-23T18:04:37.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:04:37 smithi092 ceph-mon[79089]: pgmap v441: 97 pgs: 97 active+clean; 660 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 22 MiB/s wr, 1.25k op/s 2024-03-23T18:04:37.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:04:37 smithi152 ceph-mon[47949]: pgmap v441: 97 pgs: 97 active+clean; 660 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 22 MiB/s wr, 1.25k op/s 2024-03-23T18:04:37.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:04:37 smithi086 ceph-mon[38221]: pgmap v441: 97 pgs: 97 active+clean; 660 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 22 MiB/s wr, 1.25k op/s 2024-03-23T18:04:38.227 INFO:tasks.workunit.client.0.smithi086.stdout: 10 39104 36.80 MB/sec warmup 62 sec latency 44.909 ms 2024-03-23T18:04:39.227 INFO:tasks.workunit.client.0.smithi086.stdout: 10 40017 37.02 MB/sec warmup 63 sec latency 68.876 ms 2024-03-23T18:04:40.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:04:39 smithi086 ceph-mon[38221]: pgmap v442: 97 pgs: 97 active+clean; 660 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 22 MiB/s wr, 1.25k op/s 2024-03-23T18:04:40.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:04:39 smithi092 ceph-mon[79089]: pgmap v442: 97 pgs: 97 active+clean; 660 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 22 MiB/s wr, 1.25k op/s 2024-03-23T18:04:40.085 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:04:39 smithi152 ceph-mon[47949]: pgmap v442: 97 pgs: 97 active+clean; 660 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 22 MiB/s wr, 1.25k op/s 2024-03-23T18:04:40.228 INFO:tasks.workunit.client.0.smithi086.stdout: 10 40794 37.05 MB/sec warmup 64 sec latency 60.838 ms 2024-03-23T18:04:41.228 INFO:tasks.workunit.client.0.smithi086.stdout: 10 41818 37.32 MB/sec warmup 65 sec latency 53.496 ms 2024-03-23T18:04:41.835 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.835+0000 7f392c600700 1 -- 172.21.15.86:0/2032226885 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3924100f10 msgr2=0x7f39241012f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:04:41.835 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.835+0000 7f392c600700 1 --2- 172.21.15.86:0/2032226885 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3924100f10 0x7f39241012f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f3918009990 tx=0x7f3918009ca0 comp rx=0 tx=0).stop 2024-03-23T18:04:41.836 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.835+0000 7f392c600700 1 -- 172.21.15.86:0/2032226885 shutdown_connections 2024-03-23T18:04:41.836 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.835+0000 7f392c600700 1 --2- 172.21.15.86:0/2032226885 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3924106030 0x7f392410dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:41.836 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.835+0000 7f392c600700 1 --2- 172.21.15.86:0/2032226885 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3924100f10 0x7f39241012f0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:41.836 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.835+0000 7f392c600700 1 --2- 172.21.15.86:0/2032226885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39241018c0 0x7f3924105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:41.836 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.835+0000 7f392c600700 1 -- 172.21.15.86:0/2032226885 >> 172.21.15.86:0/2032226885 conn(0x7f39240fc7b0 msgr2=0x7f39240febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:04:41.836 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.836+0000 7f392c600700 1 -- 172.21.15.86:0/2032226885 shutdown_connections 2024-03-23T18:04:41.836 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.836+0000 7f392c600700 1 -- 172.21.15.86:0/2032226885 wait complete. 2024-03-23T18:04:41.838 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.838+0000 7f392c600700 1 Processor -- start 2024-03-23T18:04:41.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.838+0000 7f392c600700 1 -- start start 2024-03-23T18:04:41.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.838+0000 7f392c600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3924100f10 0x7f39240724d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:04:41.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.838+0000 7f392c600700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3924106030 0x7f3924072a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:04:41.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.838+0000 7f392c600700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f392406c700 0x7f392406cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:04:41.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.838+0000 7f392c600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f392406d2f0 con 0x7f3924100f10 2024-03-23T18:04:41.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.839+0000 7f392c600700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f392406d460 con 0x7f3924106030 2024-03-23T18:04:41.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.839+0000 7f392c600700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39241ac250 con 0x7f392406c700 2024-03-23T18:04:41.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.839+0000 7f3929200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3924106030 0x7f3924072a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:04:41.842 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.839+0000 7f3929c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3924100f10 0x7f39240724d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:04:41.842 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.839+0000 7f3929c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3924100f10 0x7f39240724d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56980/0 (socket says 172.21.15.86:56980) 2024-03-23T18:04:41.842 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.839+0000 7f3929c00700 1 -- 172.21.15.86:0/1364423552 learned_addr learned my addr 172.21.15.86:0/1364423552 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:04:41.842 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.839+0000 7f3929200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3924106030 0x7f3924072a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:39070/0 (socket says 172.21.15.86:39070) 2024-03-23T18:04:41.842 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.839+0000 7f392a600700 1 --2- 172.21.15.86:0/1364423552 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f392406c700 0x7f392406cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:04:41.842 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.840+0000 7f3929c00700 1 -- 172.21.15.86:0/1364423552 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f392406c700 msgr2=0x7f392406cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:04:41.842 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.840+0000 7f3929c00700 1 --2- 172.21.15.86:0/1364423552 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f392406c700 0x7f392406cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:41.843 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.840+0000 7f3929c00700 1 -- 172.21.15.86:0/1364423552 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3924106030 msgr2=0x7f3924072a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:04:41.843 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.840+0000 7f3929c00700 1 --2- 172.21.15.86:0/1364423552 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3924106030 0x7f3924072a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:41.843 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.840+0000 7f3929c00700 1 -- 172.21.15.86:0/1364423552 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3910009520 con 0x7f3924100f10 2024-03-23T18:04:41.843 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.840+0000 7f392a600700 1 --2- 172.21.15.86:0/1364423552 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f392406c700 0x7f392406cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T18:04:41.843 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.840+0000 7f3929c00700 1 --2- 172.21.15.86:0/1364423552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3924100f10 0x7f39240724d0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f3918000c00 tx=0x7f3918005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:04:41.843 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.840+0000 7f3922000700 1 -- 172.21.15.86:0/1364423552 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f391801d070 con 0x7f3924100f10 2024-03-23T18:04:41.843 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.840+0000 7f3922000700 1 -- 172.21.15.86:0/1364423552 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3918021410 con 0x7f3924100f10 2024-03-23T18:04:41.843 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.840+0000 7f3922000700 1 -- 172.21.15.86:0/1364423552 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3918017650 con 0x7f3924100f10 2024-03-23T18:04:41.843 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.840+0000 7f392c600700 1 -- 172.21.15.86:0/1364423552 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39180095f0 con 0x7f3924100f10 2024-03-23T18:04:41.843 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.841+0000 7f392c600700 1 -- 172.21.15.86:0/1364423552 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3924073220 con 0x7f3924100f10 2024-03-23T18:04:41.845 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.845+0000 7f3922000700 1 -- 172.21.15.86:0/1364423552 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f3918021580 con 0x7f3924100f10 2024-03-23T18:04:41.847 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.848+0000 7f3922000700 1 --2- 172.21.15.86:0/1364423552 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f390c0a1350 0x7f390c0a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:04:41.848 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.848+0000 7f3922000700 1 -- 172.21.15.86:0/1364423552 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f39180cdff0 con 0x7f3924100f10 2024-03-23T18:04:41.848 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.848+0000 7f3929200700 1 --2- 172.21.15.86:0/1364423552 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f390c0a1350 0x7f390c0a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:04:41.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.849+0000 7f3940221b80 1 Processor -- start 2024-03-23T18:04:41.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.849+0000 7f3940221b80 1 -- start start 2024-03-23T18:04:41.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.850+0000 7f3940221b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d07e070100 0x55d07e0704e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:04:41.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.850+0000 7f3940221b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d07e070a20 0x55d07e07b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:04:41.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.850+0000 7f3940221b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d07e07b8d0 0x55d07e07dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:04:41.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.850+0000 7f3940221b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d07dfc9930 con 0x55d07e070a20 2024-03-23T18:04:41.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.850+0000 7f3940221b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d07dfc72f0 con 0x55d07e07b8d0 2024-03-23T18:04:41.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.850+0000 7f3940221b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d07dfc2be0 con 0x55d07e070100 2024-03-23T18:04:41.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.850+0000 7f3929c00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d07e070100 0x55d07e0704e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:04:41.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.850+0000 7f3929c00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d07e070100 0x55d07e0704e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:36778/0 (socket says 172.21.15.86:36778) 2024-03-23T18:04:41.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.850+0000 7f3929c00700 1 -- 172.21.15.86:0/2717082883 learned_addr learned my addr 172.21.15.86:0/2717082883 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:04:41.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.850+0000 7f3929200700 1 --2- 172.21.15.86:0/1364423552 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f390c0a1350 0x7f390c0a3810 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f39240fcbb0 tx=0x7f3910009260 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:04:41.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.850+0000 7f3929c00700 1 -- 172.21.15.86:0/2717082883 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d07e07b8d0 msgr2=0x55d07e07dcc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-23T18:04:41.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.850+0000 7f3929c00700 1 --2- 172.21.15.86:0/2717082883 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d07e07b8d0 0x55d07e07dcc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:41.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.850+0000 7f3929c00700 1 -- 172.21.15.86:0/2717082883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d07e070a20 msgr2=0x55d07e07b390 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-23T18:04:41.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.851+0000 7f3929c00700 1 --2- 172.21.15.86:0/2717082883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d07e070a20 0x55d07e07b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:41.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.851+0000 7f3929c00700 1 -- 172.21.15.86:0/2717082883 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d07de44350 con 0x55d07e070100 2024-03-23T18:04:41.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.851+0000 7f3929c00700 1 --2- 172.21.15.86:0/2717082883 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d07e070100 0x55d07e0704e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f39180976e0 tx=0x7f391809b0c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa5b1512da1f8c6d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:04:41.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.852+0000 7f3916a00700 1 -- 172.21.15.86:0/2717082883 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f391809f010 con 0x55d07e070100 2024-03-23T18:04:41.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.852+0000 7f3916a00700 1 -- 172.21.15.86:0/2717082883 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39180a35d0 con 0x55d07e070100 2024-03-23T18:04:41.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.852+0000 7f3916a00700 1 -- 172.21.15.86:0/2717082883 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39180a10c0 con 0x55d07e070100 2024-03-23T18:04:41.856 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.853+0000 7f3940221b80 1 -- 172.21.15.86:0/2717082883 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d07e070100 msgr2=0x55d07e0704e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:04:41.856 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.853+0000 7f3940221b80 1 --2- 172.21.15.86:0/2717082883 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d07e070100 0x55d07e0704e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f39180976e0 tx=0x7f391809b0c0 comp rx=0 tx=0).stop 2024-03-23T18:04:41.856 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.853+0000 7f3940221b80 1 -- 172.21.15.86:0/2717082883 shutdown_connections 2024-03-23T18:04:41.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.853+0000 7f3940221b80 1 --2- 172.21.15.86:0/2717082883 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d07e07b8d0 0x55d07e07dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:41.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.853+0000 7f3940221b80 1 --2- 172.21.15.86:0/2717082883 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d07e070100 0x55d07e0704e0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:41.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.853+0000 7f3940221b80 1 --2- 172.21.15.86:0/2717082883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d07e070a20 0x55d07e07b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:41.858 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.853+0000 7f3940221b80 1 -- 172.21.15.86:0/2717082883 >> 172.21.15.86:0/2717082883 conn(0x55d07dfd0640 msgr2=0x55d07e06c520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:04:41.858 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.853+0000 7f3940221b80 1 -- 172.21.15.86:0/2717082883 shutdown_connections 2024-03-23T18:04:41.859 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.853+0000 7f3940221b80 1 -- 172.21.15.86:0/2717082883 wait complete. 2024-03-23T18:04:41.859 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.855+0000 7f3940221b80 1 -- start start 2024-03-23T18:04:41.859 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.855+0000 7f3940221b80 1 Processor -- start 2024-03-23T18:04:41.861 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.857+0000 7f3940221b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d07e070100 0x55d07e078760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:04:41.862 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.857+0000 7f3940221b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d07e070a20 0x55d07e078ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:04:41.862 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.857+0000 7f3940221b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d07e07b8d0 0x55d07e0791e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:04:41.862 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.857+0000 7f3940221b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d07df5ed20 con 0x55d07e070100 2024-03-23T18:04:41.862 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.857+0000 7f3940221b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d07df5e9c0 con 0x55d07e07b8d0 2024-03-23T18:04:41.862 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.857+0000 7f3940221b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d07de46640 con 0x55d07e070a20 2024-03-23T18:04:41.862 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.857+0000 7f3929200700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d07e070a20 0x55d07e078ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:04:41.862 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.857+0000 7f3929200700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d07e070a20 0x55d07e078ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:36790/0 (socket says 172.21.15.86:36790) 2024-03-23T18:04:41.862 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.857+0000 7f3929200700 1 -- 172.21.15.86:0/1564686928 learned_addr learned my addr 172.21.15.86:0/1564686928 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:04:41.862 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.858+0000 7f3940221b80 10 client.0 _ll_register_callbacks cb 0x55d07df68410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T18:04:41.862 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.858+0000 7f3929c00700 1 --2- 172.21.15.86:0/1564686928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d07e070100 0x55d07e078760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:04:41.862 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.858+0000 7f392a600700 1 --2- 172.21.15.86:0/1564686928 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d07e07b8d0 0x55d07e0791e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:04:41.863 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.858+0000 7f3929200700 1 -- 172.21.15.86:0/1564686928 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d07e07b8d0 msgr2=0x55d07e0791e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:04:41.863 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.858+0000 7f3929200700 1 --2- 172.21.15.86:0/1564686928 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d07e07b8d0 0x55d07e0791e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:41.863 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.858+0000 7f3929200700 1 -- 172.21.15.86:0/1564686928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d07e070100 msgr2=0x55d07e078760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:04:41.863 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.858+0000 7f3929200700 1 --2- 172.21.15.86:0/1564686928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d07e070100 0x55d07e078760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:41.863 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.858+0000 7f3929200700 1 -- 172.21.15.86:0/1564686928 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3924073220 con 0x55d07e070a20 2024-03-23T18:04:41.863 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.858+0000 7f3929c00700 1 --2- 172.21.15.86:0/1564686928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d07e070100 0x55d07e078760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T18:04:41.863 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.859+0000 7f3929200700 1 --2- 172.21.15.86:0/1564686928 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d07e070a20 0x55d07e078ca0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f3910003980 tx=0x7f3910004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be15a97c8800c54e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:04:41.863 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.859+0000 7f390b400700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-03-23T18:04:41.863 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.859+0000 7f390b400700 1 -- 172.21.15.86:0/1564686928 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f391000e8c0 con 0x55d07e070a20 2024-03-23T18:04:41.863 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.859+0000 7f390b400700 1 -- 172.21.15.86:0/1564686928 <== mon.1 v2:172.21.15.152:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f391000ea20 con 0x55d07e070a20 2024-03-23T18:04:41.863 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.859+0000 7f3940221b80 1 -- 172.21.15.86:0/1564686928 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d07df83320 con 0x55d07e070a20 2024-03-23T18:04:41.863 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.860+0000 7f390b400700 1 -- 172.21.15.86:0/1564686928 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f391000ebe0 con 0x55d07e070a20 2024-03-23T18:04:41.864 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.860+0000 7f3940221b80 10 client.27142 fetch_fsmap learned FSMap version 67 2024-03-23T18:04:41.864 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.860+0000 7f3940221b80 1 -- 172.21.15.86:0/1564686928 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) v3 -- 0x55d07ddf0b00 con 0x55d07e070a20 2024-03-23T18:04:41.866 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.860+0000 7f390b400700 1 -- 172.21.15.86:0/1564686928 <== mon.1 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d07df83320 con 0x55d07e070a20 2024-03-23T18:04:41.866 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.861+0000 7f390b400700 1 -- 172.21.15.86:0/1564686928 <== mon.1 v2:172.21.15.152:3300/0 5 ==== fsmap(e 67) v1 ==== 7522+0+0 (secure 0 0 0) 0x7f3910024020 con 0x55d07e070a20 2024-03-23T18:04:41.866 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.861+0000 7f3940221b80 10 client.27142 fetch_fsmap finished waiting for FSMap version 67 2024-03-23T18:04:41.866 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.861+0000 7f3940221b80 10 client.27142 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T18:04:41.866 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.861+0000 7f3940221b80 20 client.27142 populate_metadata read hostname 'smithi086' 2024-03-23T18:04:41.866 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.861+0000 7f3940221b80 1 --2- 172.21.15.86:0/1564686928 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55d07e08b3e0 0x55d07e08d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:04:41.866 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.861+0000 7f3940221b80 4 client.27142 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T18:04:41.866 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.861+0000 7f3940221b80 1 -- 172.21.15.86:0/1564686928 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d07dfbb6a0 con 0x55d07e08b3e0 2024-03-23T18:04:41.866 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.862+0000 7f3929c00700 1 --2- 172.21.15.86:0/1564686928 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55d07e08b3e0 0x55d07e08d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:04:41.866 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.862+0000 7f390b400700 10 client.27142 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T18:04:41.866 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.862+0000 7f3929c00700 1 --2- 172.21.15.86:0/1564686928 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55d07e08b3e0 0x55d07e08d800 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ef51c034032000c server_cookie=9c7f5852e7046d10 in_seq=0 out_seq=0 2024-03-23T18:04:41.867 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.863+0000 7f390b400700 1 -- 172.21.15.86:0/1564686928 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55d07dfbb6a0 con 0x55d07e08b3e0 2024-03-23T18:04:41.867 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.863+0000 7f390b400700 10 client.27142 handle_command_reply: tid=0 2024-03-23T18:04:41.867 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.863+0000 7f3940221b80 1 client.27142 shutdown 2024-03-23T18:04:41.867 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.863+0000 7f3940221b80 1 -- 172.21.15.86:0/1564686928 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d07e070a20 msgr2=0x55d07e078ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:04:41.867 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.863+0000 7f3940221b80 1 --2- 172.21.15.86:0/1564686928 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d07e070a20 0x55d07e078ca0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f3910003980 tx=0x7f3910004750 comp rx=0 tx=0).stop 2024-03-23T18:04:41.867 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.864+0000 7f3940221b80 1 -- 172.21.15.86:0/1564686928 shutdown_connections 2024-03-23T18:04:41.867 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.864+0000 7f3940221b80 1 --2- 172.21.15.86:0/1564686928 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55d07e08b3e0 0x55d07e08d800 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:41.867 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.864+0000 7f3940221b80 1 --2- 172.21.15.86:0/1564686928 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d07e07b8d0 0x55d07e0791e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:41.867 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.864+0000 7f3940221b80 1 --2- 172.21.15.86:0/1564686928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d07e070100 0x55d07e078760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:41.867 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.864+0000 7f3940221b80 1 --2- 172.21.15.86:0/1564686928 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d07e070a20 0x55d07e078ca0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:41.867 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.864+0000 7f390b400700 0 client.27142 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T18:04:41.867 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.864+0000 7f3940221b80 1 -- 172.21.15.86:0/1564686928 >> 172.21.15.86:0/1564686928 conn(0x55d07dfd0640 msgr2=0x55d07e072080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:04:41.867 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.864+0000 7f3940221b80 1 -- 172.21.15.86:0/1564686928 shutdown_connections 2024-03-23T18:04:41.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.864+0000 7f3940221b80 1 -- 172.21.15.86:0/1564686928 wait complete. 2024-03-23T18:04:41.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.864+0000 7f3940221b80 20 client.27142 trim_cache size 0 max 16384 2024-03-23T18:04:41.894 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.894+0000 7f3940221b80 1 Processor -- start 2024-03-23T18:04:41.894 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.894+0000 7f3940221b80 1 -- start start 2024-03-23T18:04:41.895 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.895+0000 7f3940221b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d07e070100 0x55d07e0822c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:04:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.895+0000 7f3940221b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d07e070a20 0x55d07e066500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:04:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.895+0000 7f3940221b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d07e07b8d0 0x55d07e072040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:04:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.895+0000 7f3940221b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d07e087ec0 con 0x55d07e07b8d0 2024-03-23T18:04:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.895+0000 7f3940221b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d07e079dd0 con 0x55d07e070100 2024-03-23T18:04:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.895+0000 7f3940221b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d07e087140 con 0x55d07e070a20 2024-03-23T18:04:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.895+0000 7f3929c00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d07e070100 0x55d07e0822c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:04:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.895+0000 7f3929c00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d07e070100 0x55d07e0822c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:39080/0 (socket says 172.21.15.86:39080) 2024-03-23T18:04:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.895+0000 7f3929c00700 1 -- 172.21.15.86:0/3688993268 learned_addr learned my addr 172.21.15.86:0/3688993268 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:04:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.895+0000 7f392a600700 1 --2- 172.21.15.86:0/3688993268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d07e07b8d0 0x55d07e072040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:04:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.895+0000 7f3929200700 1 --2- 172.21.15.86:0/3688993268 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d07e070a20 0x55d07e066500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:04:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.896+0000 7f3929c00700 1 -- 172.21.15.86:0/3688993268 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d07e070a20 msgr2=0x55d07e066500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:04:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.896+0000 7f3929c00700 1 --2- 172.21.15.86:0/3688993268 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d07e070a20 0x55d07e066500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.896+0000 7f3929c00700 1 -- 172.21.15.86:0/3688993268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d07e07b8d0 msgr2=0x55d07e072040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:04:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.896+0000 7f3929c00700 1 --2- 172.21.15.86:0/3688993268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d07e07b8d0 0x55d07e072040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.896+0000 7f3929c00700 1 -- 172.21.15.86:0/3688993268 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d07df88e40 con 0x55d07e070100 2024-03-23T18:04:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.896+0000 7f3929c00700 1 --2- 172.21.15.86:0/3688993268 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d07e070100 0x55d07e0822c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f391809b3a0 tx=0x7f3918099710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6cfb404812c15b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:04:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.896+0000 7f3917400700 1 -- 172.21.15.86:0/3688993268 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3918099580 con 0x55d07e070100 2024-03-23T18:04:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.896+0000 7f3917400700 1 -- 172.21.15.86:0/3688993268 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f391809f010 con 0x55d07e070100 2024-03-23T18:04:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.896+0000 7f3917400700 1 -- 172.21.15.86:0/3688993268 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f391809a930 con 0x55d07e070100 2024-03-23T18:04:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.897+0000 7f3940221b80 1 -- 172.21.15.86:0/3688993268 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d07e070100 msgr2=0x55d07e0822c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:04:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.897+0000 7f3940221b80 1 --2- 172.21.15.86:0/3688993268 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d07e070100 0x55d07e0822c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f391809b3a0 tx=0x7f3918099710 comp rx=0 tx=0).stop 2024-03-23T18:04:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.897+0000 7f3940221b80 1 -- 172.21.15.86:0/3688993268 shutdown_connections 2024-03-23T18:04:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.897+0000 7f3940221b80 1 --2- 172.21.15.86:0/3688993268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d07e07b8d0 0x55d07e072040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.897+0000 7f3940221b80 1 --2- 172.21.15.86:0/3688993268 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d07e070100 0x55d07e0822c0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:41.899 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.897+0000 7f3940221b80 1 --2- 172.21.15.86:0/3688993268 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d07e070a20 0x55d07e066500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:41.899 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.898+0000 7f3940221b80 1 -- 172.21.15.86:0/3688993268 >> 172.21.15.86:0/3688993268 conn(0x55d07dfd0640 msgr2=0x55d07e077050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:04:41.899 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.898+0000 7f3940221b80 1 -- 172.21.15.86:0/3688993268 shutdown_connections 2024-03-23T18:04:41.899 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.898+0000 7f3940221b80 1 -- 172.21.15.86:0/3688993268 wait complete. 2024-03-23T18:04:41.899 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.898+0000 7f3940221b80 1 -- start start 2024-03-23T18:04:41.899 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.899+0000 7f3940221b80 1 Processor -- start 2024-03-23T18:04:41.900 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.900+0000 7f3940221b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d07e070100 0x55d07e0802a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:04:41.900 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.900+0000 7f3940221b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d07e070a20 0x55d07e0807e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:04:41.900 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.900+0000 7f3940221b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d07e07b8d0 0x55d07e080d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:04:41.901 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.900+0000 7f3929c00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d07e070100 0x55d07e0802a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:04:41.901 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.900+0000 7f3929c00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d07e070100 0x55d07e0802a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:36798/0 (socket says 172.21.15.86:36798) 2024-03-23T18:04:41.901 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.900+0000 7f3929c00700 1 -- 172.21.15.86:0/1344643206 learned_addr learned my addr 172.21.15.86:0/1344643206 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:04:41.902 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.901+0000 7f392a600700 1 --2- 172.21.15.86:0/1344643206 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d07e07b8d0 0x55d07e080d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:04:41.902 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.901+0000 7f3940221b80 1 -- 172.21.15.86:0/1344643206 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d07e0dc030 con 0x55d07e070a20 2024-03-23T18:04:41.902 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.901+0000 7f3940221b80 1 -- 172.21.15.86:0/1344643206 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d07e0dc1a0 con 0x55d07e07b8d0 2024-03-23T18:04:41.902 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.901+0000 7f3940221b80 1 -- 172.21.15.86:0/1344643206 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d07e0dc360 con 0x55d07e070100 2024-03-23T18:04:41.902 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.901+0000 7f3940221b80 10 client.0 _ll_register_callbacks cb 0x55d07df68410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T18:04:41.902 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.901+0000 7f392a600700 1 -- 172.21.15.86:0/1344643206 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d07e070100 msgr2=0x55d07e0802a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:04:41.902 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.901+0000 7f392a600700 1 --2- 172.21.15.86:0/1344643206 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d07e070100 0x55d07e0802a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:41.902 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.901+0000 7f392a600700 1 -- 172.21.15.86:0/1344643206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d07e070a20 msgr2=0x55d07e0807e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:04:41.902 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.901+0000 7f3929200700 1 --2- 172.21.15.86:0/1344643206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d07e070a20 0x55d07e0807e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:04:41.902 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.901+0000 7f392a600700 1 --2- 172.21.15.86:0/1344643206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d07e070a20 0x55d07e0807e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:41.902 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.901+0000 7f392a600700 1 -- 172.21.15.86:0/1344643206 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d07de44350 con 0x55d07e07b8d0 2024-03-23T18:04:41.902 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.901+0000 7f392a600700 1 --2- 172.21.15.86:0/1344643206 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d07e07b8d0 0x55d07e080d20 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x55d07e071b90 tx=0x7f391c007dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a29fb112b9cf476 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:04:41.902 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.901+0000 7f390b400700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T18:04:41.903 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.903+0000 7f390b400700 1 -- 172.21.15.86:0/1344643206 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f391c014040 con 0x55d07e07b8d0 2024-03-23T18:04:41.903 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.903+0000 7f390b400700 1 -- 172.21.15.86:0/1344643206 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f391c004bb0 con 0x55d07e07b8d0 2024-03-23T18:04:41.903 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.903+0000 7f3940221b80 1 -- 172.21.15.86:0/1344643206 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f39180a35d0 con 0x55d07e07b8d0 2024-03-23T18:04:41.904 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.904+0000 7f390b400700 1 -- 172.21.15.86:0/1344643206 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f391c004db0 con 0x55d07e07b8d0 2024-03-23T18:04:41.904 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.904+0000 7f3940221b80 10 client.27164 fetch_fsmap learned FSMap version 67 2024-03-23T18:04:41.904 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.904+0000 7f3940221b80 1 -- 172.21.15.86:0/1344643206 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) v3 -- 0x55d07de48c70 con 0x55d07e07b8d0 2024-03-23T18:04:41.904 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.905+0000 7f390b400700 1 -- 172.21.15.86:0/1344643206 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39180a35d0 con 0x55d07e07b8d0 2024-03-23T18:04:41.904 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.905+0000 7f390b400700 1 -- 172.21.15.86:0/1344643206 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 67) v1 ==== 7522+0+0 (secure 0 0 0) 0x7f391c01a910 con 0x55d07e07b8d0 2024-03-23T18:04:41.906 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.905+0000 7f3940221b80 10 client.27164 fetch_fsmap finished waiting for FSMap version 67 2024-03-23T18:04:41.906 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.905+0000 7f3940221b80 10 client.27164 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T18:04:41.906 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.905+0000 7f3940221b80 20 client.27164 populate_metadata read hostname 'smithi086' 2024-03-23T18:04:41.906 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.905+0000 7f3940221b80 1 --2- 172.21.15.86:0/1344643206 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55d07e0ddc40 0x55d07e0e0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:04:41.906 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.905+0000 7f3940221b80 4 client.27164 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T18:04:41.906 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.905+0000 7f3940221b80 1 -- 172.21.15.86:0/1344643206 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d07de11c90 con 0x55d07e0ddc40 2024-03-23T18:04:41.906 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.905+0000 7f3929c00700 1 --2- 172.21.15.86:0/1344643206 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55d07e0ddc40 0x55d07e0e0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:04:41.907 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.906+0000 7f3929c00700 1 --2- 172.21.15.86:0/1344643206 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55d07e0ddc40 0x55d07e0e0060 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=345af76faeea8f61 server_cookie=36192f8e869ea73 in_seq=0 out_seq=0 2024-03-23T18:04:41.907 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:41.906+0000 7f390b400700 10 client.27164 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T18:04:42.113 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:42.111+0000 7f390b400700 1 -- 172.21.15.86:0/1344643206 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d07de11c90 con 0x55d07e0ddc40 2024-03-23T18:04:42.113 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:42.111+0000 7f390b400700 10 client.27164 handle_command_reply: tid=0 2024-03-23T18:04:42.113 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:42.111+0000 7f3940221b80 1 client.27164 shutdown 2024-03-23T18:04:42.113 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:42.112+0000 7f3940221b80 1 -- 172.21.15.86:0/1344643206 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d07e07b8d0 msgr2=0x55d07e080d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:04:42.113 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:42.112+0000 7f3940221b80 1 --2- 172.21.15.86:0/1344643206 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d07e07b8d0 0x55d07e080d20 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x55d07e071b90 tx=0x7f391c007dd0 comp rx=0 tx=0).stop 2024-03-23T18:04:42.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:42.112+0000 7f3940221b80 1 -- 172.21.15.86:0/1344643206 shutdown_connections 2024-03-23T18:04:42.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:42.112+0000 7f3940221b80 1 --2- 172.21.15.86:0/1344643206 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55d07e0ddc40 0x55d07e0e0060 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:42.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:42.112+0000 7f3940221b80 1 --2- 172.21.15.86:0/1344643206 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d07e07b8d0 0x55d07e080d20 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:42.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:42.112+0000 7f390b400700 0 client.27164 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T18:04:42.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:42.112+0000 7f3940221b80 1 --2- 172.21.15.86:0/1344643206 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d07e070100 0x55d07e0802a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:42.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:42.112+0000 7f3940221b80 1 --2- 172.21.15.86:0/1344643206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d07e070a20 0x55d07e0807e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:42.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:42.112+0000 7f3940221b80 1 -- 172.21.15.86:0/1344643206 >> 172.21.15.86:0/1344643206 conn(0x55d07dfd0640 msgr2=0x55d07e072040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:04:42.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:42.112+0000 7f3940221b80 1 -- 172.21.15.86:0/1344643206 shutdown_connections 2024-03-23T18:04:42.115 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:42.112+0000 7f3940221b80 1 -- 172.21.15.86:0/1344643206 wait complete. 2024-03-23T18:04:42.115 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:42.113+0000 7f3940221b80 20 client.27164 trim_cache size 0 max 16384 2024-03-23T18:04:42.115 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-03-23T18:04:42.115 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-03-23T18:04:42.115 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-03-23T18:04:42.115 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-03-23T18:04:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:42.114+0000 7f392c600700 1 -- 172.21.15.86:0/1364423552 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f390c0a1350 msgr2=0x7f390c0a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:04:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:42.114+0000 7f392c600700 1 --2- 172.21.15.86:0/1364423552 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f390c0a1350 0x7f390c0a3810 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f39240fcbb0 tx=0x7f3910009260 comp rx=0 tx=0).stop 2024-03-23T18:04:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:42.114+0000 7f392c600700 1 -- 172.21.15.86:0/1364423552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3924100f10 msgr2=0x7f39240724d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:04:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:42.114+0000 7f392c600700 1 --2- 172.21.15.86:0/1364423552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3924100f10 0x7f39240724d0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f3918000c00 tx=0x7f3918005740 comp rx=0 tx=0).stop 2024-03-23T18:04:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:42.114+0000 7f392c600700 1 -- 172.21.15.86:0/1364423552 shutdown_connections 2024-03-23T18:04:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:42.114+0000 7f392c600700 1 --2- 172.21.15.86:0/1364423552 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f390c0a1350 0x7f390c0a3810 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:42.114+0000 7f392c600700 1 --2- 172.21.15.86:0/1364423552 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f392406c700 0x7f392406cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:42.114+0000 7f392c600700 1 --2- 172.21.15.86:0/1364423552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3924100f10 0x7f39240724d0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:42.114+0000 7f392c600700 1 --2- 172.21.15.86:0/1364423552 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3924106030 0x7f3924072a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:42.114+0000 7f392c600700 1 -- 172.21.15.86:0/1364423552 >> 172.21.15.86:0/1364423552 conn(0x7f39240fc7b0 msgr2=0x7f39240fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:04:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:42.114+0000 7f392c600700 1 -- 172.21.15.86:0/1364423552 shutdown_connections 2024-03-23T18:04:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:42.115+0000 7f392c600700 1 -- 172.21.15.86:0/1364423552 wait complete. 2024-03-23T18:04:42.228 INFO:tasks.workunit.client.0.smithi086.stdout: 10 42788 37.64 MB/sec warmup 66 sec latency 98.191 ms 2024-03-23T18:04:42.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:04:41 smithi086 ceph-mon[38221]: pgmap v443: 97 pgs: 97 active+clean; 691 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 23 MiB/s wr, 1.39k op/s 2024-03-23T18:04:42.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:04:41 smithi092 ceph-mon[79089]: pgmap v443: 97 pgs: 97 active+clean; 691 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 23 MiB/s wr, 1.39k op/s 2024-03-23T18:04:42.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:04:41 smithi152 ceph-mon[47949]: pgmap v443: 97 pgs: 97 active+clean; 691 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 23 MiB/s wr, 1.39k op/s 2024-03-23T18:04:43.228 INFO:tasks.workunit.client.0.smithi086.stdout: 10 43602 37.64 MB/sec warmup 67 sec latency 75.830 ms 2024-03-23T18:04:43.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:04:42 smithi086 ceph-mon[38221]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:04:43.310 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:04:42 smithi092 ceph-mon[79089]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:04:43.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:04:42 smithi152 ceph-mon[47949]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:04:44.228 INFO:tasks.workunit.client.0.smithi086.stdout: 10 44636 37.94 MB/sec warmup 68 sec latency 45.117 ms 2024-03-23T18:04:44.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:04:43 smithi086 ceph-mon[38221]: pgmap v444: 97 pgs: 97 active+clean; 691 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 16 MiB/s wr, 838 op/s 2024-03-23T18:04:44.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:04:43 smithi092 ceph-mon[79089]: pgmap v444: 97 pgs: 97 active+clean; 691 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 16 MiB/s wr, 838 op/s 2024-03-23T18:04:44.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:04:43 smithi152 ceph-mon[47949]: pgmap v444: 97 pgs: 97 active+clean; 691 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 16 MiB/s wr, 838 op/s 2024-03-23T18:04:45.228 INFO:tasks.workunit.client.0.smithi086.stdout: 10 45776 38.31 MB/sec warmup 69 sec latency 52.751 ms 2024-03-23T18:04:46.229 INFO:tasks.workunit.client.0.smithi086.stdout: 10 46649 38.42 MB/sec warmup 70 sec latency 40.357 ms 2024-03-23T18:04:46.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:04:45 smithi086 ceph-mon[38221]: pgmap v445: 97 pgs: 97 active+clean; 712 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 18 MiB/s wr, 1.13k op/s 2024-03-23T18:04:46.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:04:45 smithi092 ceph-mon[79089]: pgmap v445: 97 pgs: 97 active+clean; 712 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 18 MiB/s wr, 1.13k op/s 2024-03-23T18:04:46.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:04:45 smithi152 ceph-mon[47949]: pgmap v445: 97 pgs: 97 active+clean; 712 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 18 MiB/s wr, 1.13k op/s 2024-03-23T18:04:47.228 INFO:tasks.workunit.client.0.smithi086.stdout: 10 47545 38.55 MB/sec warmup 71 sec latency 72.476 ms 2024-03-23T18:04:48.228 INFO:tasks.workunit.client.0.smithi086.stdout: 10 48613 38.75 MB/sec warmup 72 sec latency 80.053 ms 2024-03-23T18:04:48.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:04:47 smithi086 ceph-mon[38221]: pgmap v446: 97 pgs: 97 active+clean; 715 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 888 op/s 2024-03-23T18:04:48.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:04:47 smithi092 ceph-mon[79089]: pgmap v446: 97 pgs: 97 active+clean; 715 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 888 op/s 2024-03-23T18:04:48.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:04:47 smithi152 ceph-mon[47949]: pgmap v446: 97 pgs: 97 active+clean; 715 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 888 op/s 2024-03-23T18:04:49.229 INFO:tasks.workunit.client.0.smithi086.stdout: 10 49669 39.00 MB/sec warmup 73 sec latency 41.488 ms 2024-03-23T18:04:50.228 INFO:tasks.workunit.client.0.smithi086.stdout: 10 50537 39.15 MB/sec warmup 74 sec latency 70.505 ms 2024-03-23T18:04:50.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:04:49 smithi086 ceph-mon[38221]: pgmap v447: 97 pgs: 97 active+clean; 715 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.1 MiB/s wr, 682 op/s 2024-03-23T18:04:50.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:04:49 smithi092 ceph-mon[79089]: pgmap v447: 97 pgs: 97 active+clean; 715 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.1 MiB/s wr, 682 op/s 2024-03-23T18:04:50.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:04:49 smithi152 ceph-mon[47949]: pgmap v447: 97 pgs: 97 active+clean; 715 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.1 MiB/s wr, 682 op/s 2024-03-23T18:04:51.228 INFO:tasks.workunit.client.0.smithi086.stdout: 10 51498 39.31 MB/sec warmup 75 sec latency 48.582 ms 2024-03-23T18:04:52.228 INFO:tasks.workunit.client.0.smithi086.stdout: 10 52535 39.52 MB/sec warmup 76 sec latency 67.602 ms 2024-03-23T18:04:52.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:04:51 smithi086 ceph-mon[38221]: pgmap v448: 97 pgs: 97 active+clean; 749 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 10 MiB/s wr, 985 op/s 2024-03-23T18:04:52.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:04:51 smithi092 ceph-mon[79089]: pgmap v448: 97 pgs: 97 active+clean; 749 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 10 MiB/s wr, 985 op/s 2024-03-23T18:04:52.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:04:51 smithi152 ceph-mon[47949]: pgmap v448: 97 pgs: 97 active+clean; 749 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 10 MiB/s wr, 985 op/s 2024-03-23T18:04:53.229 INFO:tasks.workunit.client.0.smithi086.stdout: 10 53506 39.69 MB/sec warmup 77 sec latency 75.138 ms 2024-03-23T18:04:54.229 INFO:tasks.workunit.client.0.smithi086.stdout: 10 54517 39.93 MB/sec warmup 78 sec latency 45.249 ms 2024-03-23T18:04:54.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:04:53 smithi086 ceph-mon[38221]: pgmap v449: 97 pgs: 97 active+clean; 749 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.8 MiB/s wr, 694 op/s 2024-03-23T18:04:54.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:04:53 smithi092 ceph-mon[79089]: pgmap v449: 97 pgs: 97 active+clean; 749 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.8 MiB/s wr, 694 op/s 2024-03-23T18:04:54.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:04:53 smithi152 ceph-mon[47949]: pgmap v449: 97 pgs: 97 active+clean; 749 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.8 MiB/s wr, 694 op/s 2024-03-23T18:04:55.229 INFO:tasks.workunit.client.0.smithi086.stdout: 10 55385 39.96 MB/sec warmup 79 sec latency 50.065 ms 2024-03-23T18:04:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.366+0000 7f40b2400700 1 -- 172.21.15.86:0/345141864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40ac073180 msgr2=0x7f40ac073560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:04:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.366+0000 7f40b2400700 1 --2- 172.21.15.86:0/345141864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40ac073180 0x7f40ac073560 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f409c009990 tx=0x7f409c009ca0 comp rx=0 tx=0).stop 2024-03-23T18:04:55.368 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.368+0000 7f40b2400700 1 -- 172.21.15.86:0/345141864 shutdown_connections 2024-03-23T18:04:55.368 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.368+0000 7f40b2400700 1 --2- 172.21.15.86:0/345141864 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f40ac10d540 0x7f40ac10f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:55.368 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.368+0000 7f40b2400700 1 --2- 172.21.15.86:0/345141864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40ac073180 0x7f40ac073560 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:55.368 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.368+0000 7f40b2400700 1 --2- 172.21.15.86:0/345141864 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40ac073aa0 0x7f40ac10d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:55.368 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.368+0000 7f40b2400700 1 -- 172.21.15.86:0/345141864 >> 172.21.15.86:0/345141864 conn(0x7f40ac0fc7b0 msgr2=0x7f40ac0febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:04:55.368 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.368+0000 7f40b2400700 1 -- 172.21.15.86:0/345141864 shutdown_connections 2024-03-23T18:04:55.369 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.368+0000 7f40b2400700 1 -- 172.21.15.86:0/345141864 wait complete. 2024-03-23T18:04:55.370 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.370+0000 7f40b2400700 1 Processor -- start 2024-03-23T18:04:55.370 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.370+0000 7f40b2400700 1 -- start start 2024-03-23T18:04:55.372 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.372+0000 7f40b2400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40ac073180 0x7f40ac199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:04:55.372 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.372+0000 7f40b2400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40ac073aa0 0x7f40ac1999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:04:55.372 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.372+0000 7f40b2400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f40ac10d540 0x7f40ac19dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:04:55.372 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.372+0000 7f40b2400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40ac19e330 con 0x7f40ac073aa0 2024-03-23T18:04:55.372 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.372+0000 7f40b2400700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40ac19e4a0 con 0x7f40ac073180 2024-03-23T18:04:55.372 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.372+0000 7f40b2400700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40ac19e660 con 0x7f40ac10d540 2024-03-23T18:04:55.372 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.372+0000 7f40ab400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40ac073180 0x7f40ac199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:04:55.373 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.372+0000 7f40abe00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f40ac10d540 0x7f40ac19dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:04:55.373 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.372+0000 7f40abe00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f40ac10d540 0x7f40ac19dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:51956/0 (socket says 172.21.15.86:51956) 2024-03-23T18:04:55.373 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.372+0000 7f40ab400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40ac073180 0x7f40ac199460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:44600/0 (socket says 172.21.15.86:44600) 2024-03-23T18:04:55.373 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.372+0000 7f40abe00700 1 -- 172.21.15.86:0/1229837535 learned_addr learned my addr 172.21.15.86:0/1229837535 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:04:55.373 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.372+0000 7f40aaa00700 1 --2- 172.21.15.86:0/1229837535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40ac073aa0 0x7f40ac1999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:04:55.373 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.373+0000 7f40abe00700 1 -- 172.21.15.86:0/1229837535 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40ac073180 msgr2=0x7f40ac199460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:04:55.373 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.373+0000 7f40abe00700 1 --2- 172.21.15.86:0/1229837535 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40ac073180 0x7f40ac199460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:55.373 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.373+0000 7f40abe00700 1 -- 172.21.15.86:0/1229837535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40ac073aa0 msgr2=0x7f40ac1999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:04:55.373 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.373+0000 7f40abe00700 1 --2- 172.21.15.86:0/1229837535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40ac073aa0 0x7f40ac1999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:55.374 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.373+0000 7f40abe00700 1 -- 172.21.15.86:0/1229837535 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f409c0095f0 con 0x7f40ac10d540 2024-03-23T18:04:55.374 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.373+0000 7f40abe00700 1 --2- 172.21.15.86:0/1229837535 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f40ac10d540 0x7f40ac19dcf0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f40a400ef90 tx=0x7f40a400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:04:55.374 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.373+0000 7f40a3e00700 1 -- 172.21.15.86:0/1229837535 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a4018070 con 0x7f40ac10d540 2024-03-23T18:04:55.375 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.374+0000 7f40a3e00700 1 -- 172.21.15.86:0/1229837535 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40a400f040 con 0x7f40ac10d540 2024-03-23T18:04:55.375 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.374+0000 7f40b2400700 1 -- 172.21.15.86:0/1229837535 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40ac19e8e0 con 0x7f40ac10d540 2024-03-23T18:04:55.375 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.374+0000 7f40a3e00700 1 -- 172.21.15.86:0/1229837535 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a4003e00 con 0x7f40ac10d540 2024-03-23T18:04:55.375 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.374+0000 7f40b2400700 1 -- 172.21.15.86:0/1229837535 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40ac1a6180 con 0x7f40ac10d540 2024-03-23T18:04:55.377 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.378+0000 7f40a3e00700 1 -- 172.21.15.86:0/1229837535 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f40a4004000 con 0x7f40ac10d540 2024-03-23T18:04:55.380 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.380+0000 7f40a3e00700 1 --2- 172.21.15.86:0/1229837535 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f40900a1390 0x7f40900a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:04:55.380 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.380+0000 7f40a3e00700 1 -- 172.21.15.86:0/1229837535 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f40a40cc880 con 0x7f40ac10d540 2024-03-23T18:04:55.381 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.381+0000 7f40c5fdab80 1 Processor -- start 2024-03-23T18:04:55.381 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.381+0000 7f40c5fdab80 1 -- start start 2024-03-23T18:04:55.382 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.381+0000 7f40c5fdab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562220017100 0x5622200174e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:04:55.382 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.381+0000 7f40c5fdab80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562220017a20 0x562220022390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:04:55.382 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.381+0000 7f40c5fdab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5622200228d0 0x562220024cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:04:55.382 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.381+0000 7f40c5fdab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56221ff70930 con 0x562220017100 2024-03-23T18:04:55.382 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.381+0000 7f40ab400700 1 --2- 172.21.15.86:0/1229837535 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f40900a1390 0x7f40900a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:04:55.382 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.381+0000 7f40c5fdab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x56221ff6e2f0 con 0x5622200228d0 2024-03-23T18:04:55.382 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.381+0000 7f40c5fdab80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x56221ff69be0 con 0x562220017a20 2024-03-23T18:04:55.382 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.382+0000 7f40abe00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5622200228d0 0x562220024cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:04:55.382 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.382+0000 7f40abe00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5622200228d0 0x562220024cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:44612/0 (socket says 172.21.15.86:44612) 2024-03-23T18:04:55.383 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.382+0000 7f40abe00700 1 -- 172.21.15.86:0/1645121758 learned_addr learned my addr 172.21.15.86:0/1645121758 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:04:55.384 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.382+0000 7f40aaa00700 1 --2- 172.21.15.86:0/1645121758 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562220017a20 0x562220022390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:04:55.384 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.383+0000 7f40abe00700 1 -- 172.21.15.86:0/1645121758 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562220017a20 msgr2=0x562220022390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:04:55.384 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.383+0000 7f40abe00700 1 --2- 172.21.15.86:0/1645121758 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562220017a20 0x562220022390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:55.384 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.383+0000 7f40abe00700 1 -- 172.21.15.86:0/1645121758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562220017100 msgr2=0x5622200174e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:04:55.384 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.383+0000 7f40abe00700 1 --2- 172.21.15.86:0/1645121758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562220017100 0x5622200174e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:55.384 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.383+0000 7f40abe00700 1 -- 172.21.15.86:0/1645121758 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56221fdeb350 con 0x5622200228d0 2024-03-23T18:04:55.384 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.383+0000 7f40aaa00700 1 --2- 172.21.15.86:0/1645121758 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562220017a20 0x562220022390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T18:04:55.384 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.383+0000 7f40abe00700 1 --2- 172.21.15.86:0/1645121758 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5622200228d0 0x562220024cc0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f40a40972d0 tx=0x7f40a4096c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1df7333628dbd18 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:04:55.385 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.383+0000 7f40ab400700 1 --2- 172.21.15.86:0/1229837535 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f40900a1390 0x7f40900a3850 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f409c00b6f0 tx=0x7f409c005c60 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:04:55.385 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.384+0000 7f40a0c00700 1 -- 172.21.15.86:0/1645121758 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a409a1c0 con 0x5622200228d0 2024-03-23T18:04:55.385 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.384+0000 7f40a0c00700 1 -- 172.21.15.86:0/1645121758 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40a409af60 con 0x5622200228d0 2024-03-23T18:04:55.385 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.384+0000 7f40a0c00700 1 -- 172.21.15.86:0/1645121758 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a409ad20 con 0x5622200228d0 2024-03-23T18:04:55.386 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.384+0000 7f40c5fdab80 1 -- 172.21.15.86:0/1645121758 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5622200228d0 msgr2=0x562220024cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:04:55.386 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.384+0000 7f40c5fdab80 1 --2- 172.21.15.86:0/1645121758 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5622200228d0 0x562220024cc0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f40a40972d0 tx=0x7f40a4096c20 comp rx=0 tx=0).stop 2024-03-23T18:04:55.386 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.385+0000 7f40c5fdab80 1 -- 172.21.15.86:0/1645121758 shutdown_connections 2024-03-23T18:04:55.386 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.385+0000 7f40c5fdab80 1 --2- 172.21.15.86:0/1645121758 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5622200228d0 0x562220024cc0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:55.386 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.385+0000 7f40c5fdab80 1 --2- 172.21.15.86:0/1645121758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562220017100 0x5622200174e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:55.386 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.385+0000 7f40c5fdab80 1 --2- 172.21.15.86:0/1645121758 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562220017a20 0x562220022390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:55.386 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.385+0000 7f40c5fdab80 1 -- 172.21.15.86:0/1645121758 >> 172.21.15.86:0/1645121758 conn(0x56221ff77640 msgr2=0x562220013520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:04:55.386 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.385+0000 7f40c5fdab80 1 -- 172.21.15.86:0/1645121758 shutdown_connections 2024-03-23T18:04:55.386 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.385+0000 7f40c5fdab80 1 -- 172.21.15.86:0/1645121758 wait complete. 2024-03-23T18:04:55.387 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.385+0000 7f40c5fdab80 1 -- start start 2024-03-23T18:04:55.387 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.386+0000 7f40c5fdab80 1 Processor -- start 2024-03-23T18:04:55.388 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.386+0000 7f40c5fdab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562220017100 0x5622200222f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:04:55.388 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.386+0000 7f40c5fdab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562220017a20 0x56222001f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:04:55.388 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.386+0000 7f40c5fdab80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5622200228d0 0x56222001fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:04:55.388 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.387+0000 7f40ab400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562220017100 0x5622200222f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:04:55.388 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.387+0000 7f40ab400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562220017100 0x5622200222f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:44620/0 (socket says 172.21.15.86:44620) 2024-03-23T18:04:55.388 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.387+0000 7f40aaa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562220017a20 0x56222001f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:04:55.388 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.387+0000 7f40aaa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562220017a20 0x56222001f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50588/0 (socket says 172.21.15.86:50588) 2024-03-23T18:04:55.388 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.388+0000 7f40ab400700 1 -- 172.21.15.86:0/42322128 learned_addr learned my addr 172.21.15.86:0/42322128 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:04:55.388 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.388+0000 7f40abe00700 1 --2- 172.21.15.86:0/42322128 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5622200228d0 0x56222001fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:04:55.388 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.388+0000 7f40c5fdab80 1 -- 172.21.15.86:0/42322128 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56221ff05d20 con 0x562220017a20 2024-03-23T18:04:55.388 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.388+0000 7f40c5fdab80 1 -- 172.21.15.86:0/42322128 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x56221ff059c0 con 0x562220017100 2024-03-23T18:04:55.388 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.388+0000 7f40c5fdab80 1 -- 172.21.15.86:0/42322128 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x56221fded640 con 0x5622200228d0 2024-03-23T18:04:55.389 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.388+0000 7f40c5fdab80 10 client.0 _ll_register_callbacks cb 0x56221ff0f410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T18:04:55.389 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.388+0000 7f40abe00700 1 -- 172.21.15.86:0/42322128 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562220017100 msgr2=0x5622200222f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:04:55.389 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.389+0000 7f40abe00700 1 --2- 172.21.15.86:0/42322128 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562220017100 0x5622200222f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:55.389 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.389+0000 7f40abe00700 1 -- 172.21.15.86:0/42322128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562220017a20 msgr2=0x56222001f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:04:55.389 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.389+0000 7f40abe00700 1 --2- 172.21.15.86:0/42322128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562220017a20 0x56222001f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:55.389 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.389+0000 7f40abe00700 1 -- 172.21.15.86:0/42322128 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f409800d7e0 con 0x5622200228d0 2024-03-23T18:04:55.389 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.389+0000 7f40ab400700 1 --2- 172.21.15.86:0/42322128 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562220017100 0x5622200222f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T18:04:55.390 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.389+0000 7f40aaa00700 1 --2- 172.21.15.86:0/42322128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562220017a20 0x56222001f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T18:04:55.390 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.390+0000 7f4095600700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-03-23T18:04:55.390 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.390+0000 7f40abe00700 1 --2- 172.21.15.86:0/42322128 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5622200228d0 0x56222001fc60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f40a409aa20 tx=0x7f40a409acf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c72f6cbcd867c46 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:04:55.391 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.391+0000 7f4095600700 1 -- 172.21.15.86:0/42322128 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a40a2c50 con 0x5622200228d0 2024-03-23T18:04:55.391 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.391+0000 7f4095600700 1 -- 172.21.15.86:0/42322128 <== mon.1 v2:172.21.15.152:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f40a409a860 con 0x5622200228d0 2024-03-23T18:04:55.391 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.391+0000 7f40c5fdab80 1 -- 172.21.15.86:0/42322128 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56221ff2a320 con 0x5622200228d0 2024-03-23T18:04:55.391 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.391+0000 7f4095600700 1 -- 172.21.15.86:0/42322128 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40a409a1c0 con 0x5622200228d0 2024-03-23T18:04:55.391 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.391+0000 7f40c5fdab80 10 client.27160 fetch_fsmap learned FSMap version 68 2024-03-23T18:04:55.391 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.391+0000 7f40c5fdab80 1 -- 172.21.15.86:0/42322128 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) v3 -- 0x7f40ac1a6180 con 0x5622200228d0 2024-03-23T18:04:55.392 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.392+0000 7f4095600700 1 -- 172.21.15.86:0/42322128 <== mon.1 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56221ff2a320 con 0x5622200228d0 2024-03-23T18:04:55.392 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.392+0000 7f4095600700 1 -- 172.21.15.86:0/42322128 <== mon.1 v2:172.21.15.152:3300/0 5 ==== fsmap(e 68) v1 ==== 7534+0+0 (secure 0 0 0) 0x7f40a4080070 con 0x5622200228d0 2024-03-23T18:04:55.392 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.392+0000 7f40c5fdab80 10 client.27160 fetch_fsmap finished waiting for FSMap version 68 2024-03-23T18:04:55.392 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.392+0000 7f40c5fdab80 10 client.27160 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T18:04:55.392 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.392+0000 7f40c5fdab80 20 client.27160 populate_metadata read hostname 'smithi086' 2024-03-23T18:04:55.393 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.393+0000 7f40c5fdab80 1 --2- 172.21.15.86:0/42322128 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x56222003aa10 0x56222003ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:04:55.393 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.393+0000 7f40c5fdab80 4 client.27160 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T18:04:55.393 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.393+0000 7f40c5fdab80 1 -- 172.21.15.86:0/42322128 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56221ff626a0 con 0x56222003aa10 2024-03-23T18:04:55.393 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.393+0000 7f40ab400700 1 --2- 172.21.15.86:0/42322128 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x56222003aa10 0x56222003ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:04:55.394 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.394+0000 7f40ab400700 1 --2- 172.21.15.86:0/42322128 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x56222003aa10 0x56222003ce30 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95fed6d3439ab90a server_cookie=eedfd88a1886fd0d in_seq=0 out_seq=0 2024-03-23T18:04:55.394 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.394+0000 7f4095600700 10 client.27160 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T18:04:55.400 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.397+0000 7f4095600700 1 -- 172.21.15.86:0/42322128 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x56221ff626a0 con 0x56222003aa10 2024-03-23T18:04:55.400 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.397+0000 7f4095600700 10 client.27160 handle_command_reply: tid=0 2024-03-23T18:04:55.400 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.397+0000 7f40c5fdab80 1 client.27160 shutdown 2024-03-23T18:04:55.400 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.397+0000 7f40c5fdab80 1 -- 172.21.15.86:0/42322128 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5622200228d0 msgr2=0x56222001fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:04:55.400 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.397+0000 7f40c5fdab80 1 --2- 172.21.15.86:0/42322128 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5622200228d0 0x56222001fc60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f40a409aa20 tx=0x7f40a409acf0 comp rx=0 tx=0).stop 2024-03-23T18:04:55.400 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.398+0000 7f40c5fdab80 1 -- 172.21.15.86:0/42322128 shutdown_connections 2024-03-23T18:04:55.400 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.398+0000 7f40c5fdab80 1 --2- 172.21.15.86:0/42322128 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x56222003aa10 0x56222003ce30 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:55.400 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.398+0000 7f4095600700 0 client.27160 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T18:04:55.400 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.398+0000 7f40c5fdab80 1 --2- 172.21.15.86:0/42322128 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5622200228d0 0x56222001fc60 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:55.401 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.398+0000 7f40c5fdab80 1 --2- 172.21.15.86:0/42322128 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562220017100 0x5622200222f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:55.401 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.398+0000 7f40c5fdab80 1 --2- 172.21.15.86:0/42322128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562220017a20 0x56222001f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:55.401 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.398+0000 7f40c5fdab80 1 -- 172.21.15.86:0/42322128 >> 172.21.15.86:0/42322128 conn(0x56221ff77640 msgr2=0x5622200190a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:04:55.401 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.398+0000 7f40c5fdab80 1 -- 172.21.15.86:0/42322128 shutdown_connections 2024-03-23T18:04:55.401 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.398+0000 7f40c5fdab80 1 -- 172.21.15.86:0/42322128 wait complete. 2024-03-23T18:04:55.401 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.399+0000 7f40c5fdab80 20 client.27160 trim_cache size 0 max 16384 2024-03-23T18:04:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.430+0000 7f40c5fdab80 1 Processor -- start 2024-03-23T18:04:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.431+0000 7f40c5fdab80 1 -- start start 2024-03-23T18:04:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.431+0000 7f40c5fdab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562220017100 0x562220041750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:04:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.431+0000 7f40c5fdab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562220017a20 0x562220031db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:04:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.433+0000 7f40c5fdab80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5622200228d0 0x56222000d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:04:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.433+0000 7f40aaa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562220017a20 0x562220031db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:04:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.433+0000 7f40c5fdab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x562220014aa0 con 0x562220017a20 2024-03-23T18:04:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.433+0000 7f40c5fdab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x56222000f160 con 0x562220017100 2024-03-23T18:04:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.433+0000 7f40c5fdab80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622200140f0 con 0x5622200228d0 2024-03-23T18:04:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.431+0000 7f40ab400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562220017100 0x562220041750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:04:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.433+0000 7f40ab400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562220017100 0x562220041750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:44634/0 (socket says 172.21.15.86:44634) 2024-03-23T18:04:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.433+0000 7f40ab400700 1 -- 172.21.15.86:0/2975108455 learned_addr learned my addr 172.21.15.86:0/2975108455 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:04:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.433+0000 7f40abe00700 1 --2- 172.21.15.86:0/2975108455 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5622200228d0 0x56222000d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:04:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.433+0000 7f40aaa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562220017a20 0x562220031db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50598/0 (socket says 172.21.15.86:50598) 2024-03-23T18:04:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.434+0000 7f40ab400700 1 -- 172.21.15.86:0/2975108455 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5622200228d0 msgr2=0x56222000d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:04:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.434+0000 7f40ab400700 1 --2- 172.21.15.86:0/2975108455 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5622200228d0 0x56222000d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.434+0000 7f40ab400700 1 -- 172.21.15.86:0/2975108455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562220017a20 msgr2=0x562220031db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:04:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.434+0000 7f40ab400700 1 --2- 172.21.15.86:0/2975108455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562220017a20 0x562220031db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.434+0000 7f40ab400700 1 -- 172.21.15.86:0/2975108455 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56221fd97b00 con 0x562220017100 2024-03-23T18:04:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.435+0000 7f40aaa00700 1 --2- 172.21.15.86:0/2975108455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562220017a20 0x562220031db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T18:04:55.437 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.436+0000 7f40ab400700 1 --2- 172.21.15.86:0/2975108455 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562220017100 0x562220041750 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f409c0098e0 tx=0x7f409c025000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9fecb544db312488 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:04:55.437 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.437+0000 7f40a1600700 1 -- 172.21.15.86:0/2975108455 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409c02c040 con 0x562220017100 2024-03-23T18:04:55.437 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.437+0000 7f40a1600700 1 -- 172.21.15.86:0/2975108455 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f409c0313a0 con 0x562220017100 2024-03-23T18:04:55.437 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.437+0000 7f40a1600700 1 -- 172.21.15.86:0/2975108455 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409c00f880 con 0x562220017100 2024-03-23T18:04:55.439 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.438+0000 7f40c5fdab80 1 -- 172.21.15.86:0/2975108455 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562220017100 msgr2=0x562220041750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:04:55.439 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.438+0000 7f40c5fdab80 1 --2- 172.21.15.86:0/2975108455 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562220017100 0x562220041750 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f409c0098e0 tx=0x7f409c025000 comp rx=0 tx=0).stop 2024-03-23T18:04:55.439 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.439+0000 7f40c5fdab80 1 -- 172.21.15.86:0/2975108455 shutdown_connections 2024-03-23T18:04:55.439 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.439+0000 7f40c5fdab80 1 --2- 172.21.15.86:0/2975108455 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5622200228d0 0x56222000d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:55.439 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.439+0000 7f40c5fdab80 1 --2- 172.21.15.86:0/2975108455 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562220017100 0x562220041750 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:55.439 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.439+0000 7f40c5fdab80 1 --2- 172.21.15.86:0/2975108455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562220017a20 0x562220031db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:55.439 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.439+0000 7f40c5fdab80 1 -- 172.21.15.86:0/2975108455 >> 172.21.15.86:0/2975108455 conn(0x56221ff77640 msgr2=0x56222001f5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:04:55.440 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.440+0000 7f40c5fdab80 1 -- 172.21.15.86:0/2975108455 shutdown_connections 2024-03-23T18:04:55.440 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.440+0000 7f40c5fdab80 1 -- 172.21.15.86:0/2975108455 wait complete. 2024-03-23T18:04:55.441 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.441+0000 7f40c5fdab80 1 -- start start 2024-03-23T18:04:55.441 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.442+0000 7f40c5fdab80 1 Processor -- start 2024-03-23T18:04:55.442 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.442+0000 7f40c5fdab80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562220017100 0x562220036570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:04:55.442 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.442+0000 7f40c5fdab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562220017a20 0x562220036ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:04:55.442 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.442+0000 7f40c5fdab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622200228d0 0x56222008b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:04:55.442 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.442+0000 7f40c5fdab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56222008ba30 con 0x5622200228d0 2024-03-23T18:04:55.442 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.442+0000 7f40c5fdab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x56222008bb70 con 0x562220017a20 2024-03-23T18:04:55.443 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.442+0000 7f40c5fdab80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x562220025200 con 0x562220017100 2024-03-23T18:04:55.443 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.442+0000 7f40c5fdab80 10 client.0 _ll_register_callbacks cb 0x56221ff0f410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T18:04:55.443 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.443+0000 7f40abe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622200228d0 0x56222008b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:04:55.443 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.443+0000 7f40ab400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562220017100 0x562220036570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:04:55.443 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.443+0000 7f40ab400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562220017100 0x562220036570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:52004/0 (socket says 172.21.15.86:52004) 2024-03-23T18:04:55.443 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.443+0000 7f40ab400700 1 -- 172.21.15.86:0/3469812515 learned_addr learned my addr 172.21.15.86:0/3469812515 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:04:55.444 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.444+0000 7f40ab400700 1 -- 172.21.15.86:0/3469812515 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562220017a20 msgr2=0x562220036ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:04:55.444 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.444+0000 7f40ab400700 1 --2- 172.21.15.86:0/3469812515 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562220017a20 0x562220036ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:55.444 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.444+0000 7f40ab400700 1 -- 172.21.15.86:0/3469812515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622200228d0 msgr2=0x56222008b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:04:55.444 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.444+0000 7f40ab400700 1 --2- 172.21.15.86:0/3469812515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622200228d0 0x56222008b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:55.444 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.444+0000 7f40ab400700 1 -- 172.21.15.86:0/3469812515 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f409c020040 con 0x562220017100 2024-03-23T18:04:55.445 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.445+0000 7f40ab400700 1 --2- 172.21.15.86:0/3469812515 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562220017100 0x562220036570 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f409c00c010 tx=0x7f409c054900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30a71e3394c3ca97 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:04:55.445 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.445+0000 7f4095600700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-03-23T18:04:55.445 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.445+0000 7f4095600700 1 -- 172.21.15.86:0/3469812515 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409c02c050 con 0x562220017100 2024-03-23T18:04:55.445 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.445+0000 7f4095600700 1 -- 172.21.15.86:0/3469812515 <== mon.1 v2:172.21.15.152:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f409c054e70 con 0x562220017100 2024-03-23T18:04:55.445 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.445+0000 7f40c5fdab80 1 -- 172.21.15.86:0/3469812515 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f40a409af60 con 0x562220017100 2024-03-23T18:04:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.446+0000 7f4095600700 1 -- 172.21.15.86:0/3469812515 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f409c00f500 con 0x562220017100 2024-03-23T18:04:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.446+0000 7f40c5fdab80 10 client.27166 fetch_fsmap learned FSMap version 68 2024-03-23T18:04:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.446+0000 7f40c5fdab80 1 -- 172.21.15.86:0/3469812515 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) v3 -- 0x56221ff2fe40 con 0x562220017100 2024-03-23T18:04:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.446+0000 7f4095600700 1 -- 172.21.15.86:0/3469812515 <== mon.1 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f409c06b020 con 0x562220017100 2024-03-23T18:04:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.446+0000 7f4095600700 1 -- 172.21.15.86:0/3469812515 <== mon.1 v2:172.21.15.152:3300/0 5 ==== fsmap(e 68) v1 ==== 7534+0+0 (secure 0 0 0) 0x7f409c004800 con 0x562220017100 2024-03-23T18:04:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.446+0000 7f40c5fdab80 10 client.27166 fetch_fsmap finished waiting for FSMap version 68 2024-03-23T18:04:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.447+0000 7f40c5fdab80 10 client.27166 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T18:04:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.447+0000 7f40c5fdab80 20 client.27166 populate_metadata read hostname 'smithi086' 2024-03-23T18:04:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.447+0000 7f40c5fdab80 1 --2- 172.21.15.86:0/3469812515 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x56222003aa10 0x562220028630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:04:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.447+0000 7f40c5fdab80 4 client.27166 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T18:04:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.447+0000 7f40c5fdab80 1 -- 172.21.15.86:0/3469812515 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7f40a409af60 con 0x56222003aa10 2024-03-23T18:04:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.447+0000 7f40aaa00700 1 --2- 172.21.15.86:0/3469812515 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x56222003aa10 0x562220028630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:04:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.447+0000 7f4095600700 10 client.27166 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T18:04:55.450 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.450+0000 7f40aaa00700 1 --2- 172.21.15.86:0/3469812515 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x56222003aa10 0x562220028630 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a666e910c3292d5 server_cookie=c4d2c0b23d8a23d5 in_seq=0 out_seq=0 2024-03-23T18:04:55.663 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.663+0000 7f4095600700 1 -- 172.21.15.86:0/3469812515 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7f40a409af60 con 0x56222003aa10 2024-03-23T18:04:55.663 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.663+0000 7f4095600700 10 client.27166 handle_command_reply: tid=0 2024-03-23T18:04:55.663 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.663+0000 7f40c5fdab80 1 client.27166 shutdown 2024-03-23T18:04:55.664 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.664+0000 7f40c5fdab80 1 -- 172.21.15.86:0/3469812515 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562220017100 msgr2=0x562220036570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:04:55.664 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.664+0000 7f40c5fdab80 1 --2- 172.21.15.86:0/3469812515 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562220017100 0x562220036570 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f409c00c010 tx=0x7f409c054900 comp rx=0 tx=0).stop 2024-03-23T18:04:55.664 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.664+0000 7f40c5fdab80 1 -- 172.21.15.86:0/3469812515 shutdown_connections 2024-03-23T18:04:55.665 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.664+0000 7f40c5fdab80 1 --2- 172.21.15.86:0/3469812515 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x56222003aa10 0x562220028630 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:55.665 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.665+0000 7f40c5fdab80 1 --2- 172.21.15.86:0/3469812515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622200228d0 0x56222008b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:55.665 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.665+0000 7f40c5fdab80 1 --2- 172.21.15.86:0/3469812515 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562220017100 0x562220036570 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:55.665 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.665+0000 7f40c5fdab80 1 --2- 172.21.15.86:0/3469812515 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562220017a20 0x562220036ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:55.665 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.665+0000 7f40c5fdab80 1 -- 172.21.15.86:0/3469812515 >> 172.21.15.86:0/3469812515 conn(0x56221ff77640 msgr2=0x56222001de40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:04:55.665 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.665+0000 7f4095600700 0 client.27166 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T18:04:55.666 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.665+0000 7f40c5fdab80 1 -- 172.21.15.86:0/3469812515 shutdown_connections 2024-03-23T18:04:55.666 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.665+0000 7f40c5fdab80 1 -- 172.21.15.86:0/3469812515 wait complete. 2024-03-23T18:04:55.666 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.666+0000 7f40c5fdab80 20 client.27166 trim_cache size 0 max 16384 2024-03-23T18:04:55.666 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-03-23T18:04:55.666 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-03-23T18:04:55.666 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-03-23T18:04:55.666 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-03-23T18:04:55.668 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.668+0000 7f40b2400700 1 -- 172.21.15.86:0/1229837535 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f40900a1390 msgr2=0x7f40900a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:04:55.668 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.668+0000 7f40b2400700 1 --2- 172.21.15.86:0/1229837535 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f40900a1390 0x7f40900a3850 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f409c00b6f0 tx=0x7f409c005c60 comp rx=0 tx=0).stop 2024-03-23T18:04:55.668 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.668+0000 7f40b2400700 1 -- 172.21.15.86:0/1229837535 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f40ac10d540 msgr2=0x7f40ac19dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:04:55.668 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.668+0000 7f40b2400700 1 --2- 172.21.15.86:0/1229837535 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f40ac10d540 0x7f40ac19dcf0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f40a400ef90 tx=0x7f40a400c3a0 comp rx=0 tx=0).stop 2024-03-23T18:04:55.669 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.668+0000 7f40b2400700 1 -- 172.21.15.86:0/1229837535 shutdown_connections 2024-03-23T18:04:55.669 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.669+0000 7f40b2400700 1 --2- 172.21.15.86:0/1229837535 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f40900a1390 0x7f40900a3850 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:55.669 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.669+0000 7f40b2400700 1 --2- 172.21.15.86:0/1229837535 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f40ac10d540 0x7f40ac19dcf0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:55.669 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.669+0000 7f40b2400700 1 --2- 172.21.15.86:0/1229837535 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40ac073180 0x7f40ac199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:55.669 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.669+0000 7f40b2400700 1 --2- 172.21.15.86:0/1229837535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40ac073aa0 0x7f40ac1999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:04:55.669 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.669+0000 7f40b2400700 1 -- 172.21.15.86:0/1229837535 >> 172.21.15.86:0/1229837535 conn(0x7f40ac0fc7b0 msgr2=0x7f40ac0fd600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:04:55.669 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.670+0000 7f40b2400700 1 -- 172.21.15.86:0/1229837535 shutdown_connections 2024-03-23T18:04:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:04:55.670+0000 7f40b2400700 1 -- 172.21.15.86:0/1229837535 wait complete. 2024-03-23T18:04:56.229 INFO:tasks.workunit.client.0.smithi086.stdout: 10 56246 40.05 MB/sec warmup 80 sec latency 153.848 ms 2024-03-23T18:04:56.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:04:55 smithi086 ceph-mon[38221]: pgmap v450: 97 pgs: 97 active+clean; 777 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 9.5 MiB/s wr, 959 op/s 2024-03-23T18:04:56.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:04:55 smithi092 ceph-mon[79089]: pgmap v450: 97 pgs: 97 active+clean; 777 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 9.5 MiB/s wr, 959 op/s 2024-03-23T18:04:56.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:04:55 smithi152 ceph-mon[47949]: pgmap v450: 97 pgs: 97 active+clean; 777 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 9.5 MiB/s wr, 959 op/s 2024-03-23T18:04:57.229 INFO:tasks.workunit.client.0.smithi086.stdout: 10 57219 40.22 MB/sec warmup 81 sec latency 85.337 ms 2024-03-23T18:04:58.229 INFO:tasks.workunit.client.0.smithi086.stdout: 10 58034 40.27 MB/sec warmup 82 sec latency 43.183 ms 2024-03-23T18:04:58.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:04:57 smithi086 ceph-mon[38221]: pgmap v451: 97 pgs: 97 active+clean; 782 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 7.5 MiB/s wr, 712 op/s 2024-03-23T18:04:58.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:04:57 smithi092 ceph-mon[79089]: pgmap v451: 97 pgs: 97 active+clean; 782 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 7.5 MiB/s wr, 712 op/s 2024-03-23T18:04:58.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:04:57 smithi152 ceph-mon[47949]: pgmap v451: 97 pgs: 97 active+clean; 782 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 7.5 MiB/s wr, 712 op/s 2024-03-23T18:04:59.229 INFO:tasks.workunit.client.0.smithi086.stdout: 10 59076 40.42 MB/sec warmup 83 sec latency 37.372 ms 2024-03-23T18:05:00.229 INFO:tasks.workunit.client.0.smithi086.stdout: 10 60055 40.57 MB/sec warmup 84 sec latency 41.191 ms 2024-03-23T18:05:00.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:04:59 smithi086 ceph-mon[38221]: pgmap v452: 97 pgs: 97 active+clean; 782 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.6 MiB/s wr, 611 op/s 2024-03-23T18:05:00.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:04:59 smithi092 ceph-mon[79089]: pgmap v452: 97 pgs: 97 active+clean; 782 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.6 MiB/s wr, 611 op/s 2024-03-23T18:05:00.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:04:59 smithi152 ceph-mon[47949]: pgmap v452: 97 pgs: 97 active+clean; 782 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.6 MiB/s wr, 611 op/s 2024-03-23T18:05:01.229 INFO:tasks.workunit.client.0.smithi086.stdout: 10 60500 40.38 MB/sec warmup 85 sec latency 338.954 ms 2024-03-23T18:05:02.230 INFO:tasks.workunit.client.0.smithi086.stdout: 10 60652 39.99 MB/sec warmup 86 sec latency 652.893 ms 2024-03-23T18:05:02.268 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:05:01 smithi086 ceph-mon[38221]: pgmap v453: 97 pgs: 97 active+clean; 801 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 11 MiB/s wr, 1.01k op/s 2024-03-23T18:05:02.268 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:05:01 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T18:05:02.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:05:01 smithi092 ceph-mon[79089]: pgmap v453: 97 pgs: 97 active+clean; 801 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 11 MiB/s wr, 1.01k op/s 2024-03-23T18:05:02.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:05:01 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T18:05:02.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:05:01 smithi152 ceph-mon[47949]: pgmap v453: 97 pgs: 97 active+clean; 801 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 11 MiB/s wr, 1.01k op/s 2024-03-23T18:05:02.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:05:01 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T18:05:03.229 INFO:tasks.workunit.client.0.smithi086.stdout: 10 60797 39.65 MB/sec warmup 87 sec latency 1221.391 ms 2024-03-23T18:05:03.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:05:03 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T18:05:03.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:05:03 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T18:05:03.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:05:03 smithi152 ceph-mon[47949]: pgmap v454: 97 pgs: 97 active+clean; 801 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.7 MiB/s wr, 712 op/s 2024-03-23T18:05:03.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:05:03 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T18:05:03.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:05:03 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T18:05:03.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:05:03 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T18:05:03.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:05:03 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T18:05:03.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:05:03 smithi086 ceph-mon[38221]: pgmap v454: 97 pgs: 97 active+clean; 801 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.7 MiB/s wr, 712 op/s 2024-03-23T18:05:03.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:05:03 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T18:05:03.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:05:03 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T18:05:03.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:05:03 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T18:05:03.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:05:03 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T18:05:03.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:05:03 smithi092 ceph-mon[79089]: pgmap v454: 97 pgs: 97 active+clean; 801 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.7 MiB/s wr, 712 op/s 2024-03-23T18:05:03.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:05:03 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T18:05:03.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:05:03 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T18:05:04.229 INFO:tasks.workunit.client.0.smithi086.stdout: 10 61335 39.51 MB/sec warmup 88 sec latency 275.747 ms 2024-03-23T18:05:04.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:05:04 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T18:05:04.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:05:04 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T18:05:04.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:05:04 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T18:05:05.230 INFO:tasks.workunit.client.0.smithi086.stdout: 10 62159 39.62 MB/sec warmup 89 sec latency 69.722 ms 2024-03-23T18:05:05.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:05:05 smithi086 ceph-mon[38221]: pgmap v455: 97 pgs: 97 active+clean; 816 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 16 MiB/s wr, 1.12k op/s 2024-03-23T18:05:05.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:05:05 smithi092 ceph-mon[79089]: pgmap v455: 97 pgs: 97 active+clean; 816 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 16 MiB/s wr, 1.12k op/s 2024-03-23T18:05:05.585 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:05:05 smithi152 ceph-mon[47949]: pgmap v455: 97 pgs: 97 active+clean; 816 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 16 MiB/s wr, 1.12k op/s 2024-03-23T18:05:06.230 INFO:tasks.workunit.client.0.smithi086.stdout: 10 62970 39.69 MB/sec warmup 90 sec latency 58.856 ms 2024-03-23T18:05:07.232 INFO:tasks.workunit.client.0.smithi086.stdout: 10 63854 39.64 MB/sec warmup 91 sec latency 54.471 ms 2024-03-23T18:05:08.231 INFO:tasks.workunit.client.0.smithi086.stdout: 10 63952 39.29 MB/sec warmup 92 sec latency 687.209 ms 2024-03-23T18:05:08.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:05:07 smithi086 ceph-mon[38221]: pgmap v456: 97 pgs: 97 active+clean; 810 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 18 MiB/s wr, 1.12k op/s 2024-03-23T18:05:08.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:05:07 smithi092 ceph-mon[79089]: pgmap v456: 97 pgs: 97 active+clean; 810 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 18 MiB/s wr, 1.12k op/s 2024-03-23T18:05:08.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:05:07 smithi152 ceph-mon[47949]: pgmap v456: 97 pgs: 97 active+clean; 810 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 18 MiB/s wr, 1.12k op/s 2024-03-23T18:05:08.901 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.901+0000 7fa424a00700 1 -- 172.21.15.86:0/806998084 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa420106030 msgr2=0x7fa42010dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:05:08.901 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.901+0000 7fa424a00700 1 --2- 172.21.15.86:0/806998084 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa420106030 0x7fa42010dbb0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fa41800b0a0 tx=0x7fa41800b3b0 comp rx=0 tx=0).stop 2024-03-23T18:05:08.901 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.901+0000 7fa424a00700 1 -- 172.21.15.86:0/806998084 shutdown_connections 2024-03-23T18:05:08.901 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.901+0000 7fa424a00700 1 --2- 172.21.15.86:0/806998084 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa420106030 0x7fa42010dbb0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:08.902 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.901+0000 7fa424a00700 1 --2- 172.21.15.86:0/806998084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa420100f10 0x7fa4201012f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:08.902 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.901+0000 7fa424a00700 1 --2- 172.21.15.86:0/806998084 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa4201018c0 0x7fa420105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:08.902 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.901+0000 7fa424a00700 1 -- 172.21.15.86:0/806998084 >> 172.21.15.86:0/806998084 conn(0x7fa4200fc770 msgr2=0x7fa4200feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:05:08.902 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.901+0000 7fa424a00700 1 -- 172.21.15.86:0/806998084 shutdown_connections 2024-03-23T18:05:08.902 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.901+0000 7fa424a00700 1 -- 172.21.15.86:0/806998084 wait complete. 2024-03-23T18:05:08.902 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.902+0000 7fa424a00700 1 Processor -- start 2024-03-23T18:05:08.903 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.903+0000 7fa424a00700 1 -- start start 2024-03-23T18:05:08.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.903+0000 7fa424a00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa420100f10 0x7fa42010ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:05:08.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.903+0000 7fa424a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa420106030 0x7fa42010b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:05:08.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.903+0000 7fa424a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa42010b7b0 0x7fa4201a9870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:05:08.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.903+0000 7fa424a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa42010df10 con 0x7fa420106030 2024-03-23T18:05:08.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.903+0000 7fa424a00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4201a9db0 con 0x7fa42010b7b0 2024-03-23T18:05:08.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.903+0000 7fa424a00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4201a9f20 con 0x7fa420100f10 2024-03-23T18:05:08.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.903+0000 7fa41d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa420106030 0x7fa42010b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:05:08.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.904+0000 7fa41d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa420106030 0x7fa42010b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38896/0 (socket says 172.21.15.86:38896) 2024-03-23T18:05:08.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.904+0000 7fa41d400700 1 -- 172.21.15.86:0/3159263249 learned_addr learned my addr 172.21.15.86:0/3159263249 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:05:08.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.904+0000 7fa41d400700 1 -- 172.21.15.86:0/3159263249 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa420100f10 msgr2=0x7fa42010ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:05:08.906 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.904+0000 7fa41d400700 1 --2- 172.21.15.86:0/3159263249 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa420100f10 0x7fa42010ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:08.906 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.904+0000 7fa41d400700 1 -- 172.21.15.86:0/3159263249 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa42010b7b0 msgr2=0x7fa4201a9870 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-23T18:05:08.906 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.904+0000 7fa41d400700 1 --2- 172.21.15.86:0/3159263249 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa42010b7b0 0x7fa4201a9870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:08.906 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.904+0000 7fa41d400700 1 -- 172.21.15.86:0/3159263249 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa418009d00 con 0x7fa420106030 2024-03-23T18:05:08.906 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.905+0000 7fa41d400700 1 --2- 172.21.15.86:0/3159263249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa420106030 0x7fa42010b270 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fa40c00d950 tx=0x7fa40c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:05:08.906 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.905+0000 7fa416a00700 1 -- 172.21.15.86:0/3159263249 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40c00bb60 con 0x7fa420106030 2024-03-23T18:05:08.906 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.905+0000 7fa416a00700 1 -- 172.21.15.86:0/3159263249 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa40c007620 con 0x7fa420106030 2024-03-23T18:05:08.907 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.907+0000 7fa416a00700 1 -- 172.21.15.86:0/3159263249 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40c005560 con 0x7fa420106030 2024-03-23T18:05:08.908 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.907+0000 7fa424a00700 1 -- 172.21.15.86:0/3159263249 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4201aa200 con 0x7fa420106030 2024-03-23T18:05:08.908 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.908+0000 7fa424a00700 1 -- 172.21.15.86:0/3159263249 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4201aa690 con 0x7fa420106030 2024-03-23T18:05:08.915 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.915+0000 7fa438501b80 1 Processor -- start 2024-03-23T18:05:08.915 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.915+0000 7fa438501b80 1 -- start start 2024-03-23T18:05:08.915 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.915+0000 7fa438501b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf70cf7100 0x55cf70cf74e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:05:08.915 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.915+0000 7fa438501b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf70cf7a20 0x55cf70d02390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:05:08.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.915+0000 7fa438501b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cf70d028d0 0x55cf70d04cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:05:08.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.915+0000 7fa438501b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf70c50930 con 0x55cf70cf7a20 2024-03-23T18:05:08.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.915+0000 7fa438501b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf70c4e2f0 con 0x55cf70cf7100 2024-03-23T18:05:08.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.915+0000 7fa438501b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf70c49be0 con 0x55cf70d028d0 2024-03-23T18:05:08.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.916+0000 7fa41de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf70cf7a20 0x55cf70d02390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:05:08.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.916+0000 7fa41de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf70cf7a20 0x55cf70d02390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38902/0 (socket says 172.21.15.86:38902) 2024-03-23T18:05:08.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.916+0000 7fa41de00700 1 -- 172.21.15.86:0/3552774423 learned_addr learned my addr 172.21.15.86:0/3552774423 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:05:08.917 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.918+0000 7fa41e800700 1 --2- 172.21.15.86:0/3552774423 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf70cf7100 0x55cf70cf74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:05:08.920 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.920+0000 7fa41de00700 1 -- 172.21.15.86:0/3552774423 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cf70d028d0 msgr2=0x55cf70d04cc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-23T18:05:08.921 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.920+0000 7fa41de00700 1 --2- 172.21.15.86:0/3552774423 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cf70d028d0 0x55cf70d04cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:08.921 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.920+0000 7fa41de00700 1 -- 172.21.15.86:0/3552774423 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf70cf7100 msgr2=0x55cf70cf74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:05:08.921 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.920+0000 7fa41de00700 1 --2- 172.21.15.86:0/3552774423 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf70cf7100 0x55cf70cf74e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:08.921 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.920+0000 7fa41de00700 1 -- 172.21.15.86:0/3552774423 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cf70acb350 con 0x55cf70cf7a20 2024-03-23T18:05:08.921 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.921+0000 7fa416a00700 1 -- 172.21.15.86:0/3159263249 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fa40c004750 con 0x7fa420106030 2024-03-23T18:05:08.922 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.922+0000 7fa41de00700 1 --2- 172.21.15.86:0/3552774423 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf70cf7a20 0x55cf70d02390 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fa41000ab20 tx=0x7fa41000ae30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=699d604c8fc6b031 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:05:08.922 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.922+0000 7fa416a00700 1 --2- 172.21.15.86:0/3159263249 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fa4040a1170 0x7fa4040a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:05:08.923 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.923+0000 7fa416a00700 1 -- 172.21.15.86:0/3159263249 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fa40c0cc1e0 con 0x7fa420106030 2024-03-23T18:05:08.925 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.923+0000 7fa40b000700 1 -- 172.21.15.86:0/3552774423 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa41000cc10 con 0x55cf70cf7a20 2024-03-23T18:05:08.925 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.923+0000 7fa40b000700 1 -- 172.21.15.86:0/3552774423 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa41000cd70 con 0x55cf70cf7a20 2024-03-23T18:05:08.925 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.923+0000 7fa40b000700 1 -- 172.21.15.86:0/3552774423 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa410018730 con 0x55cf70cf7a20 2024-03-23T18:05:08.926 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.923+0000 7fa41e800700 1 --2- 172.21.15.86:0/3159263249 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fa4040a1170 0x7fa4040a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:05:08.926 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.926+0000 7fa438501b80 1 -- 172.21.15.86:0/3552774423 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf70cf7a20 msgr2=0x55cf70d02390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:05:08.926 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.926+0000 7fa438501b80 1 --2- 172.21.15.86:0/3552774423 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf70cf7a20 0x55cf70d02390 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fa41000ab20 tx=0x7fa41000ae30 comp rx=0 tx=0).stop 2024-03-23T18:05:08.927 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.927+0000 7fa41e800700 1 --2- 172.21.15.86:0/3159263249 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fa4040a1170 0x7fa4040a3630 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa41800b830 tx=0x7fa418006230 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:05:08.928 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.927+0000 7fa438501b80 1 -- 172.21.15.86:0/3552774423 shutdown_connections 2024-03-23T18:05:08.928 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.927+0000 7fa438501b80 1 --2- 172.21.15.86:0/3552774423 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cf70d028d0 0x55cf70d04cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:08.928 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.927+0000 7fa438501b80 1 --2- 172.21.15.86:0/3552774423 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf70cf7100 0x55cf70cf74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:08.928 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.927+0000 7fa438501b80 1 --2- 172.21.15.86:0/3552774423 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf70cf7a20 0x55cf70d02390 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:08.928 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.927+0000 7fa438501b80 1 -- 172.21.15.86:0/3552774423 >> 172.21.15.86:0/3552774423 conn(0x55cf70c57640 msgr2=0x55cf70cf3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:05:08.928 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.927+0000 7fa438501b80 1 -- 172.21.15.86:0/3552774423 shutdown_connections 2024-03-23T18:05:08.928 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.928+0000 7fa438501b80 1 -- 172.21.15.86:0/3552774423 wait complete. 2024-03-23T18:05:08.929 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.929+0000 7fa438501b80 1 -- start start 2024-03-23T18:05:08.929 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.929+0000 7fa438501b80 1 Processor -- start 2024-03-23T18:05:08.931 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.931+0000 7fa438501b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cf70cf7100 0x55cf70d02350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:05:08.931 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.931+0000 7fa438501b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf70cf7a20 0x55cf70cff690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:05:08.932 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.931+0000 7fa438501b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf70d028d0 0x55cf70cffd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:05:08.932 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.931+0000 7fa438501b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf70be5d20 con 0x55cf70d028d0 2024-03-23T18:05:08.932 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.931+0000 7fa438501b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf70be59c0 con 0x55cf70cf7a20 2024-03-23T18:05:08.932 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.931+0000 7fa438501b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf70acd640 con 0x55cf70cf7100 2024-03-23T18:05:08.932 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.931+0000 7fa41de00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cf70cf7100 0x55cf70d02350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:05:08.932 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.931+0000 7fa41de00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cf70cf7100 0x55cf70d02350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:40854/0 (socket says 172.21.15.86:40854) 2024-03-23T18:05:08.932 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.931+0000 7fa41de00700 1 -- 172.21.15.86:0/284730903 learned_addr learned my addr 172.21.15.86:0/284730903 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:05:08.932 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.932+0000 7fa438501b80 10 client.0 _ll_register_callbacks cb 0x55cf70bef410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T18:05:08.932 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.932+0000 7fa41d400700 1 --2- 172.21.15.86:0/284730903 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf70cf7a20 0x55cf70cff690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:05:08.933 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.932+0000 7fa41de00700 1 -- 172.21.15.86:0/284730903 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf70cf7a20 msgr2=0x55cf70cff690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:05:08.933 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.932+0000 7fa41de00700 1 --2- 172.21.15.86:0/284730903 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf70cf7a20 0x55cf70cff690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:08.933 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.932+0000 7fa41de00700 1 -- 172.21.15.86:0/284730903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf70d028d0 msgr2=0x55cf70cffd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:05:08.933 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.932+0000 7fa41de00700 1 --2- 172.21.15.86:0/284730903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf70d028d0 0x55cf70cffd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:08.933 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.932+0000 7fa41de00700 1 -- 172.21.15.86:0/284730903 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa41000a750 con 0x55cf70cf7100 2024-03-23T18:05:08.934 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.933+0000 7fa41de00700 1 --2- 172.21.15.86:0/284730903 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cf70cf7100 0x55cf70d02350 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa41000cd40 tx=0x7fa41000f710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f098aec1adb3a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:05:08.934 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.933+0000 7fa3ff400700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-03-23T18:05:08.935 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.935+0000 7fa3ff400700 1 -- 172.21.15.86:0/284730903 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa410026070 con 0x55cf70cf7100 2024-03-23T18:05:08.936 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.936+0000 7fa438501b80 1 -- 172.21.15.86:0/284730903 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cf70c0a320 con 0x55cf70cf7100 2024-03-23T18:05:08.936 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.936+0000 7fa3ff400700 1 -- 172.21.15.86:0/284730903 <== mon.1 v2:172.21.15.152:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fa410005ca0 con 0x55cf70cf7100 2024-03-23T18:05:08.936 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.936+0000 7fa3ff400700 1 -- 172.21.15.86:0/284730903 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa41001c040 con 0x55cf70cf7100 2024-03-23T18:05:08.936 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.936+0000 7fa438501b80 10 client.27172 fetch_fsmap learned FSMap version 68 2024-03-23T18:05:08.936 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.936+0000 7fa438501b80 1 -- 172.21.15.86:0/284730903 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) v3 -- 0x55cf70a77b00 con 0x55cf70cf7100 2024-03-23T18:05:08.939 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.939+0000 7fa3ff400700 1 -- 172.21.15.86:0/284730903 <== mon.1 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cf70c0a320 con 0x55cf70cf7100 2024-03-23T18:05:08.940 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.940+0000 7fa3ff400700 1 -- 172.21.15.86:0/284730903 <== mon.1 v2:172.21.15.152:3300/0 5 ==== fsmap(e 68) v1 ==== 7534+0+0 (secure 0 0 0) 0x7fa410029030 con 0x55cf70cf7100 2024-03-23T18:05:08.940 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.940+0000 7fa438501b80 10 client.27172 fetch_fsmap finished waiting for FSMap version 68 2024-03-23T18:05:08.940 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.940+0000 7fa438501b80 10 client.27172 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T18:05:08.940 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.940+0000 7fa438501b80 20 client.27172 populate_metadata read hostname 'smithi086' 2024-03-23T18:05:08.940 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.940+0000 7fa438501b80 1 --2- 172.21.15.86:0/284730903 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55cf70d12520 0x55cf70d14940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:05:08.940 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.940+0000 7fa438501b80 4 client.27172 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T18:05:08.940 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.940+0000 7fa438501b80 1 -- 172.21.15.86:0/284730903 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cf70c426a0 con 0x55cf70d12520 2024-03-23T18:05:08.944 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.941+0000 7fa41d400700 1 --2- 172.21.15.86:0/284730903 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55cf70d12520 0x55cf70d14940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:05:08.947 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.947+0000 7fa3ff400700 10 client.27172 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T18:05:08.947 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.947+0000 7fa41d400700 1 --2- 172.21.15.86:0/284730903 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55cf70d12520 0x55cf70d14940 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36f4824d815c0e92 server_cookie=e7a9cebadbe2f12c in_seq=0 out_seq=0 2024-03-23T18:05:08.950 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.950+0000 7fa3ff400700 1 -- 172.21.15.86:0/284730903 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55cf70c426a0 con 0x55cf70d12520 2024-03-23T18:05:08.950 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.950+0000 7fa3ff400700 10 client.27172 handle_command_reply: tid=0 2024-03-23T18:05:08.950 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.950+0000 7fa438501b80 1 client.27172 shutdown 2024-03-23T18:05:08.951 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.951+0000 7fa438501b80 1 -- 172.21.15.86:0/284730903 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cf70cf7100 msgr2=0x55cf70d02350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:05:08.951 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.951+0000 7fa438501b80 1 --2- 172.21.15.86:0/284730903 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cf70cf7100 0x55cf70d02350 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa41000cd40 tx=0x7fa41000f710 comp rx=0 tx=0).stop 2024-03-23T18:05:08.951 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.951+0000 7fa438501b80 1 -- 172.21.15.86:0/284730903 shutdown_connections 2024-03-23T18:05:08.951 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.951+0000 7fa438501b80 1 --2- 172.21.15.86:0/284730903 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55cf70d12520 0x55cf70d14940 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:08.951 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.951+0000 7fa438501b80 1 --2- 172.21.15.86:0/284730903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf70d028d0 0x55cf70cffd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:08.951 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.951+0000 7fa438501b80 1 --2- 172.21.15.86:0/284730903 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cf70cf7100 0x55cf70d02350 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:08.951 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.951+0000 7fa438501b80 1 --2- 172.21.15.86:0/284730903 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf70cf7a20 0x55cf70cff690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:08.951 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.951+0000 7fa438501b80 1 -- 172.21.15.86:0/284730903 >> 172.21.15.86:0/284730903 conn(0x55cf70c57640 msgr2=0x55cf70cf90a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:05:08.952 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.952+0000 7fa3ff400700 0 client.27172 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T18:05:08.952 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.952+0000 7fa438501b80 1 -- 172.21.15.86:0/284730903 shutdown_connections 2024-03-23T18:05:08.952 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.952+0000 7fa438501b80 1 -- 172.21.15.86:0/284730903 wait complete. 2024-03-23T18:05:08.952 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.952+0000 7fa438501b80 20 client.27172 trim_cache size 0 max 16384 2024-03-23T18:05:08.981 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.977+0000 7fa438501b80 1 Processor -- start 2024-03-23T18:05:08.981 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.977+0000 7fa438501b80 1 -- start start 2024-03-23T18:05:08.981 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.978+0000 7fa438501b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf70cf7100 0x55cf70d095e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:05:08.981 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.978+0000 7fa438501b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf70cf7a20 0x55cf70ced500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:05:08.981 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.978+0000 7fa438501b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cf70d028d0 0x55cf70cfdfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:05:08.981 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.978+0000 7fa438501b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf70cf83b0 con 0x55cf70cf7a20 2024-03-23T18:05:08.981 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.978+0000 7fa438501b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf70d000e0 con 0x55cf70cf7100 2024-03-23T18:05:08.982 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.978+0000 7fa438501b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf70cfa510 con 0x55cf70d028d0 2024-03-23T18:05:08.982 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.979+0000 7fa41d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf70cf7a20 0x55cf70ced500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:05:08.982 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.979+0000 7fa41de00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf70cf7100 0x55cf70d095e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:05:08.982 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.979+0000 7fa41de00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf70cf7100 0x55cf70d095e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:52596/0 (socket says 172.21.15.86:52596) 2024-03-23T18:05:08.982 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.979+0000 7fa41de00700 1 -- 172.21.15.86:0/1141705079 learned_addr learned my addr 172.21.15.86:0/1141705079 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:05:08.982 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.979+0000 7fa41de00700 1 -- 172.21.15.86:0/1141705079 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cf70d028d0 msgr2=0x55cf70cfdfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:05:08.982 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.979+0000 7fa41de00700 1 --2- 172.21.15.86:0/1141705079 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cf70d028d0 0x55cf70cfdfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:08.982 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.979+0000 7fa41de00700 1 -- 172.21.15.86:0/1141705079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf70cf7a20 msgr2=0x55cf70ced500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:05:08.982 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.979+0000 7fa41de00700 1 --2- 172.21.15.86:0/1141705079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf70cf7a20 0x55cf70ced500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:08.982 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.980+0000 7fa41de00700 1 -- 172.21.15.86:0/1141705079 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cf70c0fe40 con 0x55cf70cf7100 2024-03-23T18:05:08.982 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.980+0000 7fa41d400700 1 --2- 172.21.15.86:0/1141705079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf70cf7a20 0x55cf70ced500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T18:05:08.982 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.980+0000 7fa41de00700 1 --2- 172.21.15.86:0/1141705079 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf70cf7100 0x55cf70d095e0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fa4100184d0 tx=0x7fa41000c5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf4d57f1e05f135f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:05:08.982 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.981+0000 7fa40ba00700 1 -- 172.21.15.86:0/1141705079 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa410026030 con 0x55cf70cf7100 2024-03-23T18:05:08.982 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.981+0000 7fa40ba00700 1 -- 172.21.15.86:0/1141705079 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa4100187c0 con 0x55cf70cf7100 2024-03-23T18:05:08.983 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.981+0000 7fa40ba00700 1 -- 172.21.15.86:0/1141705079 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa41001c040 con 0x55cf70cf7100 2024-03-23T18:05:08.984 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.982+0000 7fa438501b80 1 -- 172.21.15.86:0/1141705079 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf70cf7100 msgr2=0x55cf70d095e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:05:08.984 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.982+0000 7fa438501b80 1 --2- 172.21.15.86:0/1141705079 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf70cf7100 0x55cf70d095e0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fa4100184d0 tx=0x7fa41000c5d0 comp rx=0 tx=0).stop 2024-03-23T18:05:08.984 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.982+0000 7fa438501b80 1 -- 172.21.15.86:0/1141705079 shutdown_connections 2024-03-23T18:05:08.984 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.982+0000 7fa438501b80 1 --2- 172.21.15.86:0/1141705079 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cf70d028d0 0x55cf70cfdfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:08.984 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.982+0000 7fa438501b80 1 --2- 172.21.15.86:0/1141705079 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf70cf7100 0x55cf70d095e0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:08.984 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.982+0000 7fa438501b80 1 --2- 172.21.15.86:0/1141705079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf70cf7a20 0x55cf70ced500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:08.984 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.982+0000 7fa438501b80 1 -- 172.21.15.86:0/1141705079 >> 172.21.15.86:0/1141705079 conn(0x55cf70c57640 msgr2=0x55cf70cfa9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:05:08.984 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.982+0000 7fa438501b80 1 -- 172.21.15.86:0/1141705079 shutdown_connections 2024-03-23T18:05:08.984 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.982+0000 7fa438501b80 1 -- 172.21.15.86:0/1141705079 wait complete. 2024-03-23T18:05:08.984 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.983+0000 7fa438501b80 1 -- start start 2024-03-23T18:05:08.984 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.983+0000 7fa438501b80 1 Processor -- start 2024-03-23T18:05:08.985 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.984+0000 7fa438501b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf70cf7100 0x55cf70d075c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:05:08.985 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.984+0000 7fa438501b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cf70cf7a20 0x55cf70d07b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:05:08.985 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.984+0000 7fa438501b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf70d028d0 0x55cf70d08040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:05:08.985 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.985+0000 7fa41d400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cf70cf7a20 0x55cf70d07b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:05:08.985 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.985+0000 7fa41e800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf70d028d0 0x55cf70d08040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:05:08.985 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.985+0000 7fa41de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf70cf7100 0x55cf70d075c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:05:08.985 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.985+0000 7fa41e800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf70d028d0 0x55cf70d08040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:52612/0 (socket says 172.21.15.86:52612) 2024-03-23T18:05:08.986 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.985+0000 7fa41d400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cf70cf7a20 0x55cf70d07b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:40876/0 (socket says 172.21.15.86:40876) 2024-03-23T18:05:08.986 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.985+0000 7fa41de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf70cf7100 0x55cf70d075c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38922/0 (socket says 172.21.15.86:38922) 2024-03-23T18:05:08.986 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.985+0000 7fa41e800700 1 -- 172.21.15.86:0/1054471070 learned_addr learned my addr 172.21.15.86:0/1054471070 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:05:08.986 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.985+0000 7fa438501b80 1 -- 172.21.15.86:0/1054471070 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf70d09990 con 0x55cf70cf7100 2024-03-23T18:05:08.986 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.985+0000 7fa438501b80 1 -- 172.21.15.86:0/1054471070 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf70d63040 con 0x55cf70d028d0 2024-03-23T18:05:08.986 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.985+0000 7fa438501b80 1 -- 172.21.15.86:0/1054471070 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf70d63200 con 0x55cf70cf7a20 2024-03-23T18:05:08.986 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.985+0000 7fa438501b80 10 client.0 _ll_register_callbacks cb 0x7fa41800b420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T18:05:08.987 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.986+0000 7fa41e800700 1 -- 172.21.15.86:0/1054471070 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cf70cf7a20 msgr2=0x55cf70d07b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:05:08.987 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.986+0000 7fa41e800700 1 --2- 172.21.15.86:0/1054471070 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cf70cf7a20 0x55cf70d07b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:08.987 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.986+0000 7fa41e800700 1 -- 172.21.15.86:0/1054471070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf70cf7100 msgr2=0x55cf70d075c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:05:08.987 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.986+0000 7fa41e800700 1 --2- 172.21.15.86:0/1054471070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf70cf7100 0x55cf70d075c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:08.987 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.986+0000 7fa41e800700 1 -- 172.21.15.86:0/1054471070 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cf70a77b00 con 0x55cf70d028d0 2024-03-23T18:05:08.987 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.986+0000 7fa41e800700 1 --2- 172.21.15.86:0/1054471070 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf70d028d0 0x55cf70d08040 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fa41800b0a0 tx=0x7fa418007a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78d9d046e48c4dbb server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:05:08.987 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.987+0000 7fa3ff400700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T18:05:08.987 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.987+0000 7fa3ff400700 1 -- 172.21.15.86:0/1054471070 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa41800e040 con 0x55cf70d028d0 2024-03-23T18:05:08.987 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.987+0000 7fa3ff400700 1 -- 172.21.15.86:0/1054471070 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fa418023050 con 0x55cf70d028d0 2024-03-23T18:05:08.988 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.987+0000 7fa438501b80 1 -- 172.21.15.86:0/1054471070 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa41000cd70 con 0x55cf70d028d0 2024-03-23T18:05:08.988 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.987+0000 7fa3ff400700 1 -- 172.21.15.86:0/1054471070 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa418009180 con 0x55cf70d028d0 2024-03-23T18:05:08.988 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.988+0000 7fa438501b80 10 client.27194 fetch_fsmap learned FSMap version 68 2024-03-23T18:05:08.988 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.988+0000 7fa438501b80 1 -- 172.21.15.86:0/1054471070 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) v3 -- 0x55cf70acfc70 con 0x55cf70d028d0 2024-03-23T18:05:08.988 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.988+0000 7fa3ff400700 1 -- 172.21.15.86:0/1054471070 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa41000cd70 con 0x55cf70d028d0 2024-03-23T18:05:08.989 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.989+0000 7fa3ff400700 1 -- 172.21.15.86:0/1054471070 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 68) v1 ==== 7534+0+0 (secure 0 0 0) 0x7fa4180123f0 con 0x55cf70d028d0 2024-03-23T18:05:08.989 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.989+0000 7fa438501b80 10 client.27194 fetch_fsmap finished waiting for FSMap version 68 2024-03-23T18:05:08.989 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.989+0000 7fa438501b80 10 client.27194 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T18:05:08.989 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.989+0000 7fa438501b80 20 client.27194 populate_metadata read hostname 'smithi086' 2024-03-23T18:05:08.990 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.989+0000 7fa438501b80 1 --2- 172.21.15.86:0/1054471070 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55cf70d64990 0x55cf70d66db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:05:08.990 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.989+0000 7fa438501b80 4 client.27194 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T18:05:08.990 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.989+0000 7fa438501b80 1 -- 172.21.15.86:0/1054471070 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55cf70a98c90 con 0x55cf70d64990 2024-03-23T18:05:08.990 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.990+0000 7fa41de00700 1 --2- 172.21.15.86:0/1054471070 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55cf70d64990 0x55cf70d66db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:05:08.992 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.992+0000 7fa41de00700 1 --2- 172.21.15.86:0/1054471070 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55cf70d64990 0x55cf70d66db0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32fb5e075dc7bc43 server_cookie=3025096a5e86ecf8 in_seq=0 out_seq=0 2024-03-23T18:05:08.992 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:08.992+0000 7fa3ff400700 10 client.27194 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T18:05:09.230 INFO:tasks.workunit.client.0.smithi086.stdout: 10 64453 39.20 MB/sec warmup 93 sec latency 757.857 ms 2024-03-23T18:05:09.249 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:09.248+0000 7fa3ff400700 1 -- 172.21.15.86:0/1054471070 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55cf70a98c90 con 0x55cf70d64990 2024-03-23T18:05:09.249 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:09.248+0000 7fa3ff400700 10 client.27194 handle_command_reply: tid=0 2024-03-23T18:05:09.249 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:09.248+0000 7fa438501b80 1 client.27194 shutdown 2024-03-23T18:05:09.249 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:09.249+0000 7fa438501b80 1 -- 172.21.15.86:0/1054471070 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf70d028d0 msgr2=0x55cf70d08040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:05:09.249 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:09.249+0000 7fa438501b80 1 --2- 172.21.15.86:0/1054471070 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf70d028d0 0x55cf70d08040 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fa41800b0a0 tx=0x7fa418007a00 comp rx=0 tx=0).stop 2024-03-23T18:05:09.250 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:09.250+0000 7fa438501b80 1 -- 172.21.15.86:0/1054471070 shutdown_connections 2024-03-23T18:05:09.250 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:09.250+0000 7fa438501b80 1 --2- 172.21.15.86:0/1054471070 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55cf70d64990 0x55cf70d66db0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:09.250 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:09.250+0000 7fa438501b80 1 --2- 172.21.15.86:0/1054471070 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf70d028d0 0x55cf70d08040 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:09.250 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:09.250+0000 7fa438501b80 1 --2- 172.21.15.86:0/1054471070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf70cf7100 0x55cf70d075c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:09.251 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:09.250+0000 7fa438501b80 1 --2- 172.21.15.86:0/1054471070 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cf70cf7a20 0x55cf70d07b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:09.251 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:09.250+0000 7fa438501b80 1 -- 172.21.15.86:0/1054471070 >> 172.21.15.86:0/1054471070 conn(0x55cf70c57640 msgr2=0x55cf70cf9040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:05:09.251 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:09.250+0000 7fa3ff400700 0 client.27194 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T18:05:09.251 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:09.251+0000 7fa438501b80 1 -- 172.21.15.86:0/1054471070 shutdown_connections 2024-03-23T18:05:09.252 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:09.251+0000 7fa438501b80 1 -- 172.21.15.86:0/1054471070 wait complete. 2024-03-23T18:05:09.252 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:09.251+0000 7fa438501b80 20 client.27194 trim_cache size 0 max 16384 2024-03-23T18:05:09.252 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-03-23T18:05:09.252 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-03-23T18:05:09.252 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-03-23T18:05:09.252 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-03-23T18:05:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:09.254+0000 7fa424a00700 1 -- 172.21.15.86:0/3159263249 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fa4040a1170 msgr2=0x7fa4040a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:05:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:09.254+0000 7fa424a00700 1 --2- 172.21.15.86:0/3159263249 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fa4040a1170 0x7fa4040a3630 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa41800b830 tx=0x7fa418006230 comp rx=0 tx=0).stop 2024-03-23T18:05:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:09.254+0000 7fa424a00700 1 -- 172.21.15.86:0/3159263249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa420106030 msgr2=0x7fa42010b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:05:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:09.254+0000 7fa424a00700 1 --2- 172.21.15.86:0/3159263249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa420106030 0x7fa42010b270 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fa40c00d950 tx=0x7fa40c00dd10 comp rx=0 tx=0).stop 2024-03-23T18:05:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:09.254+0000 7fa424a00700 1 -- 172.21.15.86:0/3159263249 shutdown_connections 2024-03-23T18:05:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:09.254+0000 7fa424a00700 1 --2- 172.21.15.86:0/3159263249 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa42010b7b0 0x7fa4201a9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:09.254+0000 7fa424a00700 1 --2- 172.21.15.86:0/3159263249 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fa4040a1170 0x7fa4040a3630 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:09.254+0000 7fa424a00700 1 --2- 172.21.15.86:0/3159263249 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa420100f10 0x7fa42010ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:09.254+0000 7fa424a00700 1 --2- 172.21.15.86:0/3159263249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa420106030 0x7fa42010b270 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:09.254+0000 7fa424a00700 1 -- 172.21.15.86:0/3159263249 >> 172.21.15.86:0/3159263249 conn(0x7fa4200fc770 msgr2=0x7fa4200fd300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:05:09.255 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:09.254+0000 7fa424a00700 1 -- 172.21.15.86:0/3159263249 shutdown_connections 2024-03-23T18:05:09.255 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:09.254+0000 7fa424a00700 1 -- 172.21.15.86:0/3159263249 wait complete. 2024-03-23T18:05:10.230 INFO:tasks.workunit.client.0.smithi086.stdout: 10 65337 39.32 MB/sec warmup 94 sec latency 202.113 ms 2024-03-23T18:05:10.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:05:09 smithi086 ceph-mon[38221]: pgmap v457: 97 pgs: 97 active+clean; 810 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 17 MiB/s wr, 1.07k op/s 2024-03-23T18:05:10.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:05:09 smithi092 ceph-mon[79089]: pgmap v457: 97 pgs: 97 active+clean; 810 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 17 MiB/s wr, 1.07k op/s 2024-03-23T18:05:10.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:05:09 smithi152 ceph-mon[47949]: pgmap v457: 97 pgs: 97 active+clean; 810 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 17 MiB/s wr, 1.07k op/s 2024-03-23T18:05:11.230 INFO:tasks.workunit.client.0.smithi086.stdout: 10 66163 39.43 MB/sec warmup 95 sec latency 43.923 ms 2024-03-23T18:05:12.230 INFO:tasks.workunit.client.0.smithi086.stdout: 10 66981 39.39 MB/sec warmup 96 sec latency 70.804 ms 2024-03-23T18:05:12.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:05:11 smithi086 ceph-mon[38221]: pgmap v458: 97 pgs: 97 active+clean; 851 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 32 MiB/s wr, 1.70k op/s 2024-03-23T18:05:12.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:05:11 smithi092 ceph-mon[79089]: pgmap v458: 97 pgs: 97 active+clean; 851 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 32 MiB/s wr, 1.70k op/s 2024-03-23T18:05:12.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:05:11 smithi152 ceph-mon[47949]: pgmap v458: 97 pgs: 97 active+clean; 851 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 32 MiB/s wr, 1.70k op/s 2024-03-23T18:05:13.230 INFO:tasks.workunit.client.0.smithi086.stdout: 10 67983 39.55 MB/sec warmup 97 sec latency 72.272 ms 2024-03-23T18:05:14.234 INFO:tasks.workunit.client.0.smithi086.stdout: 10 68910 39.63 MB/sec warmup 98 sec latency 64.271 ms 2024-03-23T18:05:14.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:05:13 smithi086 ceph-mon[38221]: pgmap v459: 97 pgs: 97 active+clean; 851 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 28 MiB/s wr, 1.29k op/s 2024-03-23T18:05:14.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:05:13 smithi092 ceph-mon[79089]: pgmap v459: 97 pgs: 97 active+clean; 851 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 28 MiB/s wr, 1.29k op/s 2024-03-23T18:05:14.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:05:13 smithi152 ceph-mon[47949]: pgmap v459: 97 pgs: 97 active+clean; 851 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 28 MiB/s wr, 1.29k op/s 2024-03-23T18:05:15.230 INFO:tasks.workunit.client.0.smithi086.stdout: 10 69969 39.85 MB/sec warmup 99 sec latency 56.514 ms 2024-03-23T18:05:16.230 INFO:tasks.workunit.client.0.smithi086.stdout: 10 70796 39.94 MB/sec warmup 100 sec latency 304.984 ms 2024-03-23T18:05:16.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:05:15 smithi086 ceph-mon[38221]: pgmap v460: 97 pgs: 97 active+clean; 864 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 30 MiB/s wr, 1.53k op/s 2024-03-23T18:05:16.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:05:15 smithi092 ceph-mon[79089]: pgmap v460: 97 pgs: 97 active+clean; 864 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 30 MiB/s wr, 1.53k op/s 2024-03-23T18:05:16.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:05:15 smithi152 ceph-mon[47949]: pgmap v460: 97 pgs: 97 active+clean; 864 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 30 MiB/s wr, 1.53k op/s 2024-03-23T18:05:17.231 INFO:tasks.workunit.client.0.smithi086.stdout: 10 71761 39.99 MB/sec warmup 101 sec latency 60.456 ms 2024-03-23T18:05:18.231 INFO:tasks.workunit.client.0.smithi086.stdout: 10 72746 40.16 MB/sec warmup 102 sec latency 50.955 ms 2024-03-23T18:05:18.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:05:17 smithi086 ceph-mon[38221]: pgmap v461: 97 pgs: 97 active+clean; 863 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 23 MiB/s wr, 1.21k op/s 2024-03-23T18:05:18.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:05:17 smithi092 ceph-mon[79089]: pgmap v461: 97 pgs: 97 active+clean; 863 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 23 MiB/s wr, 1.21k op/s 2024-03-23T18:05:18.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:05:17 smithi152 ceph-mon[47949]: pgmap v461: 97 pgs: 97 active+clean; 863 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 23 MiB/s wr, 1.21k op/s 2024-03-23T18:05:19.231 INFO:tasks.workunit.client.0.smithi086.stdout: 10 73634 40.17 MB/sec warmup 103 sec latency 51.134 ms 2024-03-23T18:05:20.231 INFO:tasks.workunit.client.0.smithi086.stdout: 10 74652 40.40 MB/sec warmup 104 sec latency 47.446 ms 2024-03-23T18:05:20.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:05:19 smithi086 ceph-mon[38221]: pgmap v462: 97 pgs: 97 active+clean; 863 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 18 MiB/s wr, 946 op/s 2024-03-23T18:05:20.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:05:19 smithi092 ceph-mon[79089]: pgmap v462: 97 pgs: 97 active+clean; 863 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 18 MiB/s wr, 946 op/s 2024-03-23T18:05:20.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:05:19 smithi152 ceph-mon[47949]: pgmap v462: 97 pgs: 97 active+clean; 863 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 18 MiB/s wr, 946 op/s 2024-03-23T18:05:21.231 INFO:tasks.workunit.client.0.smithi086.stdout: 10 75509 40.39 MB/sec warmup 105 sec latency 40.277 ms 2024-03-23T18:05:22.231 INFO:tasks.workunit.client.0.smithi086.stdout: 10 76427 40.48 MB/sec warmup 106 sec latency 57.297 ms 2024-03-23T18:05:22.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:05:21 smithi086 ceph-mon[38221]: pgmap v463: 97 pgs: 97 active+clean; 898 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 22 MiB/s wr, 1.24k op/s 2024-03-23T18:05:22.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:05:21 smithi092 ceph-mon[79089]: pgmap v463: 97 pgs: 97 active+clean; 898 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 22 MiB/s wr, 1.24k op/s 2024-03-23T18:05:22.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:05:21 smithi152 ceph-mon[47949]: pgmap v463: 97 pgs: 97 active+clean; 898 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 22 MiB/s wr, 1.24k op/s 2024-03-23T18:05:22.504 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.503+0000 7f0180400700 1 -- 172.21.15.86:0/664812730 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01780680e0 msgr2=0x7f0178068560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:05:22.504 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.503+0000 7f0180400700 1 --2- 172.21.15.86:0/664812730 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01780680e0 0x7f0178068560 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f0164009990 tx=0x7f0164009ca0 comp rx=0 tx=0).stop 2024-03-23T18:05:22.504 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.504+0000 7f0180400700 1 -- 172.21.15.86:0/664812730 shutdown_connections 2024-03-23T18:05:22.504 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.504+0000 7f0180400700 1 --2- 172.21.15.86:0/664812730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0178068aa0 0x7f017810daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:22.504 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.504+0000 7f0180400700 1 --2- 172.21.15.86:0/664812730 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0178106e00 0x7f01781071e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:22.504 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.504+0000 7f0180400700 1 --2- 172.21.15.86:0/664812730 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01780680e0 0x7f0178068560 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:22.504 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.504+0000 7f0180400700 1 -- 172.21.15.86:0/664812730 >> 172.21.15.86:0/664812730 conn(0x7f0178076d50 msgr2=0x7f0178077160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:05:22.505 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.504+0000 7f0180400700 1 -- 172.21.15.86:0/664812730 shutdown_connections 2024-03-23T18:05:22.505 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.504+0000 7f0180400700 1 -- 172.21.15.86:0/664812730 wait complete. 2024-03-23T18:05:22.506 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.506+0000 7f0180400700 1 Processor -- start 2024-03-23T18:05:22.506 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.507+0000 7f0180400700 1 -- start start 2024-03-23T18:05:22.507 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.507+0000 7f0180400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01780680e0 0x7f017810d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:05:22.507 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.507+0000 7f0180400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0178068aa0 0x7f017810a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:05:22.508 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.507+0000 7f0180400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0178106e00 0x7f017810ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:05:22.508 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.507+0000 7f0180400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f017810b3d0 con 0x7f0178068aa0 2024-03-23T18:05:22.508 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.507+0000 7f0180400700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f017810b540 con 0x7f0178106e00 2024-03-23T18:05:22.508 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.507+0000 7f0180400700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f017810b700 con 0x7f01780680e0 2024-03-23T18:05:22.509 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.508+0000 7f017d000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0178068aa0 0x7f017810a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:05:22.509 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.508+0000 7f017d000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0178068aa0 0x7f017810a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46978/0 (socket says 172.21.15.86:46978) 2024-03-23T18:05:22.509 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.508+0000 7f017d000700 1 -- 172.21.15.86:0/3306693148 learned_addr learned my addr 172.21.15.86:0/3306693148 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:05:22.509 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.508+0000 7f017e400700 1 --2- 172.21.15.86:0/3306693148 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0178106e00 0x7f017810ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:05:22.509 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.508+0000 7f017da00700 1 --2- 172.21.15.86:0/3306693148 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01780680e0 0x7f017810d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:05:22.509 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.509+0000 7f017da00700 1 -- 172.21.15.86:0/3306693148 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0178106e00 msgr2=0x7f017810ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:05:22.509 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.509+0000 7f017da00700 1 --2- 172.21.15.86:0/3306693148 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0178106e00 0x7f017810ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:22.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.509+0000 7f017da00700 1 -- 172.21.15.86:0/3306693148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0178068aa0 msgr2=0x7f017810a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:05:22.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.509+0000 7f017da00700 1 --2- 172.21.15.86:0/3306693148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0178068aa0 0x7f017810a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:22.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.509+0000 7f017da00700 1 -- 172.21.15.86:0/3306693148 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01640095f0 con 0x7f01780680e0 2024-03-23T18:05:22.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.509+0000 7f017da00700 1 --2- 172.21.15.86:0/3306693148 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01780680e0 0x7f017810d760 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0170007ec0 tx=0x7f017000d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:05:22.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.509+0000 7f0176000700 1 -- 172.21.15.86:0/3306693148 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0170004db0 con 0x7f01780680e0 2024-03-23T18:05:22.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.510+0000 7f0176000700 1 -- 172.21.15.86:0/3306693148 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f017000f040 con 0x7f01780680e0 2024-03-23T18:05:22.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.510+0000 7f0176000700 1 -- 172.21.15.86:0/3306693148 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01700058e0 con 0x7f01780680e0 2024-03-23T18:05:22.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.510+0000 7f0180400700 1 -- 172.21.15.86:0/3306693148 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01781a9ec0 con 0x7f01780680e0 2024-03-23T18:05:22.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.510+0000 7f0180400700 1 -- 172.21.15.86:0/3306693148 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01781aa2f0 con 0x7f01780680e0 2024-03-23T18:05:22.515 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.515+0000 7f0176000700 1 -- 172.21.15.86:0/3306693148 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f017000db20 con 0x7f01780680e0 2024-03-23T18:05:22.518 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.518+0000 7f0176000700 1 --2- 172.21.15.86:0/3306693148 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f015c0a1390 0x7f015c0a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:05:22.518 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.518+0000 7f0176000700 1 -- 172.21.15.86:0/3306693148 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f01700cc530 con 0x7f01780680e0 2024-03-23T18:05:22.518 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.519+0000 7f017d000700 1 --2- 172.21.15.86:0/3306693148 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f015c0a1390 0x7f015c0a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:05:22.520 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.519+0000 7f0193fa4b80 1 Processor -- start 2024-03-23T18:05:22.520 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.520+0000 7f017d000700 1 --2- 172.21.15.86:0/3306693148 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f015c0a1390 0x7f015c0a3850 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0164003760 tx=0x7f016400b580 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:05:22.521 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.520+0000 7f0193fa4b80 1 -- start start 2024-03-23T18:05:22.523 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.522+0000 7f0193fa4b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e505b9100 0x556e505b94e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:05:22.523 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.522+0000 7f0193fa4b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556e505b9a20 0x556e505c4390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:05:22.523 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.522+0000 7f0193fa4b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556e505c48d0 0x556e505c6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:05:22.523 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.522+0000 7f0193fa4b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e50512930 con 0x556e505b9100 2024-03-23T18:05:22.523 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.522+0000 7f0193fa4b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e505102f0 con 0x556e505c48d0 2024-03-23T18:05:22.523 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.522+0000 7f0193fa4b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e5050bbe0 con 0x556e505b9a20 2024-03-23T18:05:22.523 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.522+0000 7f017e400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556e505c48d0 0x556e505c6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:05:22.523 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.522+0000 7f017e400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556e505c48d0 0x556e505c6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:41568/0 (socket says 172.21.15.86:41568) 2024-03-23T18:05:22.523 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.522+0000 7f017e400700 1 -- 172.21.15.86:0/793563575 learned_addr learned my addr 172.21.15.86:0/793563575 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:05:22.524 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.523+0000 7f017da00700 1 --2- 172.21.15.86:0/793563575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e505b9100 0x556e505b94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:05:22.524 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.523+0000 7f017d000700 1 --2- 172.21.15.86:0/793563575 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556e505b9a20 0x556e505c4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:05:22.524 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.523+0000 7f017e400700 1 -- 172.21.15.86:0/793563575 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556e505b9a20 msgr2=0x556e505c4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:05:22.524 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.523+0000 7f017e400700 1 --2- 172.21.15.86:0/793563575 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556e505b9a20 0x556e505c4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:22.524 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.523+0000 7f017e400700 1 -- 172.21.15.86:0/793563575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e505b9100 msgr2=0x556e505b94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:05:22.524 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.523+0000 7f017e400700 1 --2- 172.21.15.86:0/793563575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e505b9100 0x556e505b94e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:22.524 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.523+0000 7f017e400700 1 -- 172.21.15.86:0/793563575 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556e5038d350 con 0x556e505c48d0 2024-03-23T18:05:22.525 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.525+0000 7f017e400700 1 --2- 172.21.15.86:0/793563575 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556e505c48d0 0x556e505c6cc0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f016800ac50 tx=0x7f016800af60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=665f56086096a790 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:05:22.525 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.526+0000 7f016ea00700 1 -- 172.21.15.86:0/793563575 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f016800fe60 con 0x556e505c48d0 2024-03-23T18:05:22.526 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.526+0000 7f016ea00700 1 -- 172.21.15.86:0/793563575 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f01680044e0 con 0x556e505c48d0 2024-03-23T18:05:22.527 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.526+0000 7f016ea00700 1 -- 172.21.15.86:0/793563575 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01680186a0 con 0x556e505c48d0 2024-03-23T18:05:22.527 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.527+0000 7f0193fa4b80 1 -- 172.21.15.86:0/793563575 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556e505c48d0 msgr2=0x556e505c6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:05:22.527 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.527+0000 7f0193fa4b80 1 --2- 172.21.15.86:0/793563575 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556e505c48d0 0x556e505c6cc0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f016800ac50 tx=0x7f016800af60 comp rx=0 tx=0).stop 2024-03-23T18:05:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.527+0000 7f0193fa4b80 1 -- 172.21.15.86:0/793563575 shutdown_connections 2024-03-23T18:05:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.527+0000 7f0193fa4b80 1 --2- 172.21.15.86:0/793563575 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556e505c48d0 0x556e505c6cc0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.527+0000 7f0193fa4b80 1 --2- 172.21.15.86:0/793563575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e505b9100 0x556e505b94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.527+0000 7f0193fa4b80 1 --2- 172.21.15.86:0/793563575 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556e505b9a20 0x556e505c4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.527+0000 7f0193fa4b80 1 -- 172.21.15.86:0/793563575 >> 172.21.15.86:0/793563575 conn(0x556e50519640 msgr2=0x556e505b5520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:05:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.527+0000 7f0193fa4b80 1 -- 172.21.15.86:0/793563575 shutdown_connections 2024-03-23T18:05:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.527+0000 7f0193fa4b80 1 -- 172.21.15.86:0/793563575 wait complete. 2024-03-23T18:05:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.528+0000 7f0193fa4b80 1 -- start start 2024-03-23T18:05:22.529 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.530+0000 7f0193fa4b80 1 Processor -- start 2024-03-23T18:05:22.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.530+0000 7f0193fa4b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556e505b9100 0x556e505c42e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:05:22.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.530+0000 7f0193fa4b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556e505b9a20 0x556e505c1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:05:22.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.530+0000 7f0193fa4b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e505c48d0 0x556e505c1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:05:22.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.530+0000 7f0193fa4b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e504a7d20 con 0x556e505c48d0 2024-03-23T18:05:22.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.530+0000 7f0193fa4b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e504a79c0 con 0x556e505b9100 2024-03-23T18:05:22.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.531+0000 7f0193fa4b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e5038f640 con 0x556e505b9a20 2024-03-23T18:05:22.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.531+0000 7f0193fa4b80 10 client.0 _ll_register_callbacks cb 0x556e504b1410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T18:05:22.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.531+0000 7f017da00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556e505b9100 0x556e505c42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:05:22.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.531+0000 7f017da00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556e505b9100 0x556e505c42e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:41584/0 (socket says 172.21.15.86:41584) 2024-03-23T18:05:22.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.531+0000 7f017da00700 1 -- 172.21.15.86:0/2607183008 learned_addr learned my addr 172.21.15.86:0/2607183008 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:05:22.533 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.532+0000 7f017d000700 1 --2- 172.21.15.86:0/2607183008 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556e505b9a20 0x556e505c1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:05:22.533 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.532+0000 7f017da00700 1 -- 172.21.15.86:0/2607183008 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556e505b9a20 msgr2=0x556e505c1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:05:22.533 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.532+0000 7f017da00700 1 --2- 172.21.15.86:0/2607183008 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556e505b9a20 0x556e505c1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:22.533 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.532+0000 7f017da00700 1 -- 172.21.15.86:0/2607183008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e505c48d0 msgr2=0x556e505c1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:05:22.533 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.532+0000 7f017e400700 1 --2- 172.21.15.86:0/2607183008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e505c48d0 0x556e505c1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:05:22.534 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.533+0000 7f017da00700 1 --2- 172.21.15.86:0/2607183008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e505c48d0 0x556e505c1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:22.534 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.533+0000 7f017da00700 1 -- 172.21.15.86:0/2607183008 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f016800a940 con 0x556e505b9100 2024-03-23T18:05:22.534 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.533+0000 7f017e400700 1 --2- 172.21.15.86:0/2607183008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e505c48d0 0x556e505c1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T18:05:22.534 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.533+0000 7f017da00700 1 --2- 172.21.15.86:0/2607183008 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556e505b9100 0x556e505c42e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f017000d700 tx=0x7f0170098f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60cb5f5b8c970e7b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:05:22.534 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.533+0000 7f0163400700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T18:05:22.534 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.533+0000 7f0163400700 1 -- 172.21.15.86:0/2607183008 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0170099630 con 0x556e505b9100 2024-03-23T18:05:22.534 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.533+0000 7f0163400700 1 -- 172.21.15.86:0/2607183008 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f017009d460 con 0x556e505b9100 2024-03-23T18:05:22.534 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.534+0000 7f0193fa4b80 1 -- 172.21.15.86:0/2607183008 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556e504cc320 con 0x556e505b9100 2024-03-23T18:05:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.534+0000 7f0163400700 1 -- 172.21.15.86:0/2607183008 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f01700a1ab0 con 0x556e505b9100 2024-03-23T18:05:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.535+0000 7f0193fa4b80 10 client.27206 fetch_fsmap learned FSMap version 68 2024-03-23T18:05:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.535+0000 7f0193fa4b80 1 -- 172.21.15.86:0/2607183008 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) v3 -- 0x556e50339b00 con 0x556e505b9100 2024-03-23T18:05:22.537 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.537+0000 7f0163400700 1 -- 172.21.15.86:0/2607183008 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f017007c020 con 0x556e505b9100 2024-03-23T18:05:22.537 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.537+0000 7f0163400700 1 -- 172.21.15.86:0/2607183008 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 68) v1 ==== 7534+0+0 (secure 0 0 0) 0x7f0170099890 con 0x556e505b9100 2024-03-23T18:05:22.537 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.538+0000 7f0193fa4b80 10 client.27206 fetch_fsmap finished waiting for FSMap version 68 2024-03-23T18:05:22.538 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.538+0000 7f0193fa4b80 10 client.27206 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T18:05:22.538 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.538+0000 7f0193fa4b80 20 client.27206 populate_metadata read hostname 'smithi086' 2024-03-23T18:05:22.538 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.538+0000 7f0193fa4b80 1 --2- 172.21.15.86:0/2607183008 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x556e505e5690 0x556e505e7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:05:22.538 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.538+0000 7f0193fa4b80 4 client.27206 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T18:05:22.538 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.538+0000 7f0193fa4b80 1 -- 172.21.15.86:0/2607183008 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556e505046a0 con 0x556e505e5690 2024-03-23T18:05:22.539 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.540+0000 7f017d000700 1 --2- 172.21.15.86:0/2607183008 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x556e505e5690 0x556e505e7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:05:22.541 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.542+0000 7f0163400700 10 client.27206 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T18:05:22.541 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.542+0000 7f017d000700 1 --2- 172.21.15.86:0/2607183008 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x556e505e5690 0x556e505e7ab0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=891d1bff87c02e8b server_cookie=61f29324716ba11e in_seq=0 out_seq=0 2024-03-23T18:05:22.544 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.544+0000 7f0163400700 1 -- 172.21.15.86:0/2607183008 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x556e505046a0 con 0x556e505e5690 2024-03-23T18:05:22.544 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.544+0000 7f0163400700 10 client.27206 handle_command_reply: tid=0 2024-03-23T18:05:22.544 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.544+0000 7f0193fa4b80 1 client.27206 shutdown 2024-03-23T18:05:22.544 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.545+0000 7f0193fa4b80 1 -- 172.21.15.86:0/2607183008 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556e505b9100 msgr2=0x556e505c42e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:05:22.545 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.545+0000 7f0193fa4b80 1 --2- 172.21.15.86:0/2607183008 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556e505b9100 0x556e505c42e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f017000d700 tx=0x7f0170098f90 comp rx=0 tx=0).stop 2024-03-23T18:05:22.545 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.545+0000 7f0193fa4b80 1 -- 172.21.15.86:0/2607183008 shutdown_connections 2024-03-23T18:05:22.545 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.545+0000 7f0193fa4b80 1 --2- 172.21.15.86:0/2607183008 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x556e505e5690 0x556e505e7ab0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:22.545 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.545+0000 7f0193fa4b80 1 --2- 172.21.15.86:0/2607183008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e505c48d0 0x556e505c1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:22.546 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.545+0000 7f0163400700 0 client.27206 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T18:05:22.546 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.545+0000 7f0193fa4b80 1 --2- 172.21.15.86:0/2607183008 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556e505b9100 0x556e505c42e0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:22.546 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.545+0000 7f0193fa4b80 1 --2- 172.21.15.86:0/2607183008 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556e505b9a20 0x556e505c1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:22.546 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.545+0000 7f0193fa4b80 1 -- 172.21.15.86:0/2607183008 >> 172.21.15.86:0/2607183008 conn(0x556e50519640 msgr2=0x556e505bb0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:05:22.546 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.545+0000 7f0193fa4b80 1 -- 172.21.15.86:0/2607183008 shutdown_connections 2024-03-23T18:05:22.546 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.546+0000 7f0193fa4b80 1 -- 172.21.15.86:0/2607183008 wait complete. 2024-03-23T18:05:22.547 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.547+0000 7f0193fa4b80 20 client.27206 trim_cache size 0 max 16384 2024-03-23T18:05:22.575 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.574+0000 7f0193fa4b80 1 Processor -- start 2024-03-23T18:05:22.575 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.575+0000 7f0193fa4b80 1 -- start start 2024-03-23T18:05:22.575 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.576+0000 7f0193fa4b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556e505b9100 0x556e505cb610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:05:22.576 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.576+0000 7f0193fa4b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556e505b9a20 0x556e505c15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:05:22.576 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.576+0000 7f0193fa4b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e505c48d0 0x556e505af500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:05:22.576 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.576+0000 7f0193fa4b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e505c2c00 con 0x556e505c48d0 2024-03-23T18:05:22.576 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.576+0000 7f0193fa4b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e505b6af0 con 0x556e505b9100 2024-03-23T18:05:22.576 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.576+0000 7f0193fa4b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e505c2230 con 0x556e505b9a20 2024-03-23T18:05:22.577 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.576+0000 7f017d000700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556e505b9a20 0x556e505c15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:05:22.578 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.576+0000 7f017d000700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556e505b9a20 0x556e505c15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:39826/0 (socket says 172.21.15.86:39826) 2024-03-23T18:05:22.578 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.576+0000 7f017e400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e505c48d0 0x556e505af500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:05:22.578 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.576+0000 7f017d000700 1 -- 172.21.15.86:0/3204085638 learned_addr learned my addr 172.21.15.86:0/3204085638 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:05:22.578 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.576+0000 7f017d000700 1 -- 172.21.15.86:0/3204085638 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556e505b9100 msgr2=0x556e505cb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:05:22.578 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.576+0000 7f017d000700 1 --2- 172.21.15.86:0/3204085638 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556e505b9100 0x556e505cb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:22.578 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.576+0000 7f017d000700 1 -- 172.21.15.86:0/3204085638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e505c48d0 msgr2=0x556e505af500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:05:22.578 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.576+0000 7f017d000700 1 --2- 172.21.15.86:0/3204085638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e505c48d0 0x556e505af500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:22.578 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.576+0000 7f017d000700 1 -- 172.21.15.86:0/3204085638 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556e504d1e40 con 0x556e505b9a20 2024-03-23T18:05:22.578 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.576+0000 7f017e400700 1 --2- 172.21.15.86:0/3204085638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e505c48d0 0x556e505af500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T18:05:22.578 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.577+0000 7f017d000700 1 --2- 172.21.15.86:0/3204085638 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556e505b9a20 0x556e505c15c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f01640092e0 tx=0x7f016401a070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82b0262dea76a33f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:05:22.579 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.578+0000 7f016f400700 1 -- 172.21.15.86:0/3204085638 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0164032190 con 0x556e505b9a20 2024-03-23T18:05:22.579 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.578+0000 7f016f400700 1 -- 172.21.15.86:0/3204085638 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f016401e040 con 0x556e505b9a20 2024-03-23T18:05:22.580 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.578+0000 7f016f400700 1 -- 172.21.15.86:0/3204085638 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0164032190 con 0x556e505b9a20 2024-03-23T18:05:22.580 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.579+0000 7f0193fa4b80 1 -- 172.21.15.86:0/3204085638 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556e505b9a20 msgr2=0x556e505c15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:05:22.580 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.579+0000 7f0193fa4b80 1 --2- 172.21.15.86:0/3204085638 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556e505b9a20 0x556e505c15c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f01640092e0 tx=0x7f016401a070 comp rx=0 tx=0).stop 2024-03-23T18:05:22.580 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.580+0000 7f0193fa4b80 1 -- 172.21.15.86:0/3204085638 shutdown_connections 2024-03-23T18:05:22.581 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.580+0000 7f0193fa4b80 1 --2- 172.21.15.86:0/3204085638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e505c48d0 0x556e505af500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:22.581 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.580+0000 7f0193fa4b80 1 --2- 172.21.15.86:0/3204085638 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556e505b9100 0x556e505cb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:22.581 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.580+0000 7f0193fa4b80 1 --2- 172.21.15.86:0/3204085638 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556e505b9a20 0x556e505c15c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:22.581 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.580+0000 7f0193fa4b80 1 -- 172.21.15.86:0/3204085638 >> 172.21.15.86:0/3204085638 conn(0x556e50519640 msgr2=0x556e505bbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:05:22.581 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.581+0000 7f0193fa4b80 1 -- 172.21.15.86:0/3204085638 shutdown_connections 2024-03-23T18:05:22.581 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.581+0000 7f0193fa4b80 1 -- 172.21.15.86:0/3204085638 wait complete. 2024-03-23T18:05:22.582 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.581+0000 7f0193fa4b80 1 -- start start 2024-03-23T18:05:22.582 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.582+0000 7f0193fa4b80 1 Processor -- start 2024-03-23T18:05:22.583 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.582+0000 7f0193fa4b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556e505b9100 0x556e505cb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:05:22.583 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.582+0000 7f0193fa4b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556e505b9a20 0x556e505c9a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:05:22.583 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.583+0000 7f0193fa4b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e505c48d0 0x556e505c9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:05:22.583 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.583+0000 7f017da00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556e505b9100 0x556e505cb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:05:22.583 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.583+0000 7f017da00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556e505b9100 0x556e505cb3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:39840/0 (socket says 172.21.15.86:39840) 2024-03-23T18:05:22.583 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.583+0000 7f017da00700 1 -- 172.21.15.86:0/1089609090 learned_addr learned my addr 172.21.15.86:0/1089609090 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:05:22.584 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.583+0000 7f0193fa4b80 1 -- 172.21.15.86:0/1089609090 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e505ca580 con 0x556e505c48d0 2024-03-23T18:05:22.584 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.583+0000 7f0193fa4b80 1 -- 172.21.15.86:0/1089609090 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e505ca6f0 con 0x556e505b9a20 2024-03-23T18:05:22.584 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.583+0000 7f0193fa4b80 1 -- 172.21.15.86:0/1089609090 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e505ca8b0 con 0x556e505b9100 2024-03-23T18:05:22.584 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.583+0000 7f0193fa4b80 10 client.0 _ll_register_callbacks cb 0x556e504b1410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T18:05:22.584 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.583+0000 7f017d000700 1 --2- 172.21.15.86:0/1089609090 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556e505b9a20 0x556e505c9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:05:22.584 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.584+0000 7f017e400700 1 --2- 172.21.15.86:0/1089609090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e505c48d0 0x556e505c9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:05:22.585 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.584+0000 7f017da00700 1 -- 172.21.15.86:0/1089609090 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556e505b9a20 msgr2=0x556e505c9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:05:22.585 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.584+0000 7f017da00700 1 --2- 172.21.15.86:0/1089609090 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556e505b9a20 0x556e505c9a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:22.585 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.584+0000 7f017da00700 1 -- 172.21.15.86:0/1089609090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e505c48d0 msgr2=0x556e505c9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:05:22.585 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.584+0000 7f017da00700 1 --2- 172.21.15.86:0/1089609090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e505c48d0 0x556e505c9f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:22.585 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.584+0000 7f017da00700 1 -- 172.21.15.86:0/1089609090 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01780ffd10 con 0x556e505b9100 2024-03-23T18:05:22.585 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.584+0000 7f017e400700 1 --2- 172.21.15.86:0/1089609090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e505c48d0 0x556e505c9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T18:05:22.586 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.585+0000 7f017da00700 1 --2- 172.21.15.86:0/1089609090 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556e505b9100 0x556e505cb3a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f01700960a0 tx=0x7f01700ca1d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54eec4bbbbb27010 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:05:22.586 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.585+0000 7f0163400700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-03-23T18:05:22.587 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.585+0000 7f0163400700 1 -- 172.21.15.86:0/1089609090 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f017009d460 con 0x556e505b9100 2024-03-23T18:05:22.587 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.585+0000 7f0163400700 1 -- 172.21.15.86:0/1089609090 <== mon.1 v2:172.21.15.152:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f017007d080 con 0x556e505b9100 2024-03-23T18:05:22.587 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.586+0000 7f0193fa4b80 1 -- 172.21.15.86:0/1089609090 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f01680044e0 con 0x556e505b9100 2024-03-23T18:05:22.587 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.586+0000 7f0163400700 1 -- 172.21.15.86:0/1089609090 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0170099630 con 0x556e505b9100 2024-03-23T18:05:22.587 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.586+0000 7f0193fa4b80 10 client.27193 fetch_fsmap learned FSMap version 68 2024-03-23T18:05:22.587 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.586+0000 7f0193fa4b80 1 -- 172.21.15.86:0/1089609090 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) v3 -- 0x556e50391c70 con 0x556e505b9100 2024-03-23T18:05:22.587 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.586+0000 7f0163400700 1 -- 172.21.15.86:0/1089609090 <== mon.1 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f017007c030 con 0x556e505b9100 2024-03-23T18:05:22.587 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.587+0000 7f0163400700 1 -- 172.21.15.86:0/1089609090 <== mon.1 v2:172.21.15.152:3300/0 5 ==== fsmap(e 68) v1 ==== 7534+0+0 (secure 0 0 0) 0x7f0170098bb0 con 0x556e505b9100 2024-03-23T18:05:22.587 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.587+0000 7f0193fa4b80 10 client.27193 fetch_fsmap finished waiting for FSMap version 68 2024-03-23T18:05:22.588 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.587+0000 7f0193fa4b80 10 client.27193 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T18:05:22.588 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.587+0000 7f0193fa4b80 20 client.27193 populate_metadata read hostname 'smithi086' 2024-03-23T18:05:22.588 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.587+0000 7f0193fa4b80 1 --2- 172.21.15.86:0/1089609090 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x556e50638620 0x556e5063aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:05:22.588 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.587+0000 7f0193fa4b80 4 client.27193 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T18:05:22.588 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.587+0000 7f0193fa4b80 1 -- 172.21.15.86:0/1089609090 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7f01680044e0 con 0x556e50638620 2024-03-23T18:05:22.588 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.588+0000 7f017d000700 1 --2- 172.21.15.86:0/1089609090 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x556e50638620 0x556e5063aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:05:22.588 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.588+0000 7f017d000700 1 --2- 172.21.15.86:0/1089609090 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x556e50638620 0x556e5063aa40 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94aad09ae22201d8 server_cookie=ac04950d119b7a65 in_seq=0 out_seq=0 2024-03-23T18:05:22.589 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.588+0000 7f0163400700 10 client.27193 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T18:05:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.821+0000 7f0163400700 1 -- 172.21.15.86:0/1089609090 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7f01680044e0 con 0x556e50638620 2024-03-23T18:05:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.821+0000 7f0163400700 10 client.27193 handle_command_reply: tid=0 2024-03-23T18:05:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.821+0000 7f0193fa4b80 1 client.27193 shutdown 2024-03-23T18:05:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.821+0000 7f0193fa4b80 1 -- 172.21.15.86:0/1089609090 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556e505b9100 msgr2=0x556e505cb3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:05:22.822 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.821+0000 7f0193fa4b80 1 --2- 172.21.15.86:0/1089609090 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556e505b9100 0x556e505cb3a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f01700960a0 tx=0x7f01700ca1d0 comp rx=0 tx=0).stop 2024-03-23T18:05:22.822 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.822+0000 7f0193fa4b80 1 -- 172.21.15.86:0/1089609090 shutdown_connections 2024-03-23T18:05:22.822 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.822+0000 7f0193fa4b80 1 --2- 172.21.15.86:0/1089609090 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x556e50638620 0x556e5063aa40 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:22.822 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.822+0000 7f0193fa4b80 1 --2- 172.21.15.86:0/1089609090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e505c48d0 0x556e505c9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:22.822 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.822+0000 7f0193fa4b80 1 --2- 172.21.15.86:0/1089609090 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556e505b9100 0x556e505cb3a0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:22.822 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.822+0000 7f0193fa4b80 1 --2- 172.21.15.86:0/1089609090 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556e505b9a20 0x556e505c9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:22.822 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.822+0000 7f0193fa4b80 1 -- 172.21.15.86:0/1089609090 >> 172.21.15.86:0/1089609090 conn(0x556e50519640 msgr2=0x556e505b4d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:05:22.823 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.822+0000 7f0163400700 0 client.27193 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T18:05:22.823 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.822+0000 7f0193fa4b80 1 -- 172.21.15.86:0/1089609090 shutdown_connections 2024-03-23T18:05:22.823 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.822+0000 7f0193fa4b80 1 -- 172.21.15.86:0/1089609090 wait complete. 2024-03-23T18:05:22.823 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.823+0000 7f0193fa4b80 20 client.27193 trim_cache size 0 max 16384 2024-03-23T18:05:22.823 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-03-23T18:05:22.823 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-03-23T18:05:22.824 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-03-23T18:05:22.824 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-03-23T18:05:22.824 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.824+0000 7f0180400700 1 -- 172.21.15.86:0/3306693148 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f015c0a1390 msgr2=0x7f015c0a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:05:22.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.824+0000 7f0180400700 1 --2- 172.21.15.86:0/3306693148 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f015c0a1390 0x7f015c0a3850 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0164003760 tx=0x7f016400b580 comp rx=0 tx=0).stop 2024-03-23T18:05:22.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.824+0000 7f0180400700 1 -- 172.21.15.86:0/3306693148 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01780680e0 msgr2=0x7f017810d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:05:22.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.824+0000 7f0180400700 1 --2- 172.21.15.86:0/3306693148 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01780680e0 0x7f017810d760 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0170007ec0 tx=0x7f017000d3b0 comp rx=0 tx=0).stop 2024-03-23T18:05:22.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.824+0000 7f0180400700 1 -- 172.21.15.86:0/3306693148 shutdown_connections 2024-03-23T18:05:22.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.824+0000 7f0180400700 1 --2- 172.21.15.86:0/3306693148 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0178106e00 0x7f017810ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:22.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.824+0000 7f0180400700 1 --2- 172.21.15.86:0/3306693148 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f015c0a1390 0x7f015c0a3850 secure :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0164003760 tx=0x7f016400b580 comp rx=0 tx=0).stop 2024-03-23T18:05:22.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.824+0000 7f0180400700 1 --2- 172.21.15.86:0/3306693148 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01780680e0 0x7f017810d760 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:22.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.824+0000 7f0180400700 1 --2- 172.21.15.86:0/3306693148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0178068aa0 0x7f017810a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:22.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.824+0000 7f0180400700 1 -- 172.21.15.86:0/3306693148 >> 172.21.15.86:0/3306693148 conn(0x7f0178076d50 msgr2=0x7f01780fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:05:22.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.825+0000 7f0180400700 1 -- 172.21.15.86:0/3306693148 shutdown_connections 2024-03-23T18:05:22.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:22.825+0000 7f0180400700 1 -- 172.21.15.86:0/3306693148 wait complete. 2024-03-23T18:05:23.231 INFO:tasks.workunit.client.0.smithi086.stdout: 10 77331 40.60 MB/sec warmup 107 sec latency 149.241 ms 2024-03-23T18:05:24.231 INFO:tasks.workunit.client.0.smithi086.stdout: 10 78190 40.67 MB/sec warmup 108 sec latency 47.543 ms 2024-03-23T18:05:24.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:05:23 smithi086 ceph-mon[38221]: pgmap v464: 97 pgs: 97 active+clean; 898 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.8 MiB/s wr, 619 op/s 2024-03-23T18:05:24.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:05:23 smithi092 ceph-mon[79089]: pgmap v464: 97 pgs: 97 active+clean; 898 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.8 MiB/s wr, 619 op/s 2024-03-23T18:05:24.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:05:23 smithi152 ceph-mon[47949]: pgmap v464: 97 pgs: 97 active+clean; 898 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.8 MiB/s wr, 619 op/s 2024-03-23T18:05:25.231 INFO:tasks.workunit.client.0.smithi086.stdout: 10 78953 40.59 MB/sec warmup 109 sec latency 56.767 ms 2024-03-23T18:05:26.231 INFO:tasks.workunit.client.0.smithi086.stdout: 10 80029 40.76 MB/sec warmup 110 sec latency 42.968 ms 2024-03-23T18:05:26.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:05:25 smithi086 ceph-mon[38221]: pgmap v465: 97 pgs: 97 active+clean; 912 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 8.9 MiB/s wr, 859 op/s 2024-03-23T18:05:26.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:05:25 smithi092 ceph-mon[79089]: pgmap v465: 97 pgs: 97 active+clean; 912 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 8.9 MiB/s wr, 859 op/s 2024-03-23T18:05:26.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:05:25 smithi152 ceph-mon[47949]: pgmap v465: 97 pgs: 97 active+clean; 912 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 8.9 MiB/s wr, 859 op/s 2024-03-23T18:05:27.231 INFO:tasks.workunit.client.0.smithi086.stdout: 10 81032 40.90 MB/sec warmup 111 sec latency 51.162 ms 2024-03-23T18:05:28.231 INFO:tasks.workunit.client.0.smithi086.stdout: 10 81976 41.05 MB/sec warmup 112 sec latency 61.312 ms 2024-03-23T18:05:28.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:05:27 smithi086 ceph-mon[38221]: pgmap v466: 97 pgs: 97 active+clean; 920 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 7.6 MiB/s wr, 753 op/s 2024-03-23T18:05:28.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:05:27 smithi092 ceph-mon[79089]: pgmap v466: 97 pgs: 97 active+clean; 920 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 7.6 MiB/s wr, 753 op/s 2024-03-23T18:05:28.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:05:27 smithi152 ceph-mon[47949]: pgmap v466: 97 pgs: 97 active+clean; 920 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 7.6 MiB/s wr, 753 op/s 2024-03-23T18:05:29.231 INFO:tasks.workunit.client.0.smithi086.stdout: 10 82924 41.07 MB/sec warmup 113 sec latency 50.942 ms 2024-03-23T18:05:30.232 INFO:tasks.workunit.client.0.smithi086.stdout: 10 83872 41.22 MB/sec warmup 114 sec latency 51.231 ms 2024-03-23T18:05:30.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:05:29 smithi086 ceph-mon[38221]: pgmap v467: 97 pgs: 97 active+clean; 920 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.7 MiB/s wr, 670 op/s 2024-03-23T18:05:30.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:05:29 smithi086 ceph-mon[38221]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:05:30.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:05:29 smithi092 ceph-mon[79089]: pgmap v467: 97 pgs: 97 active+clean; 920 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.7 MiB/s wr, 670 op/s 2024-03-23T18:05:30.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:05:29 smithi092 ceph-mon[79089]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:05:30.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:05:29 smithi152 ceph-mon[47949]: pgmap v467: 97 pgs: 97 active+clean; 920 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.7 MiB/s wr, 670 op/s 2024-03-23T18:05:30.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:05:29 smithi152 ceph-mon[47949]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:05:31.232 INFO:tasks.workunit.client.0.smithi086.stdout: 10 84736 41.16 MB/sec warmup 115 sec latency 65.509 ms 2024-03-23T18:05:32.232 INFO:tasks.workunit.client.0.smithi086.stdout: 10 85705 41.34 MB/sec warmup 116 sec latency 43.495 ms 2024-03-23T18:05:32.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:05:31 smithi086 ceph-mon[38221]: pgmap v468: 97 pgs: 97 active+clean; 957 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 10 MiB/s wr, 971 op/s 2024-03-23T18:05:32.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:05:31 smithi092 ceph-mon[79089]: pgmap v468: 97 pgs: 97 active+clean; 957 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 10 MiB/s wr, 971 op/s 2024-03-23T18:05:32.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:05:31 smithi152 ceph-mon[47949]: pgmap v468: 97 pgs: 97 active+clean; 957 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 10 MiB/s wr, 971 op/s 2024-03-23T18:05:33.232 INFO:tasks.workunit.client.0.smithi086.stdout: 10 86527 41.34 MB/sec warmup 117 sec latency 39.663 ms 2024-03-23T18:05:34.232 INFO:tasks.workunit.client.0.smithi086.stdout: 10 86744 41.07 MB/sec warmup 118 sec latency 602.078 ms 2024-03-23T18:05:34.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:05:33 smithi086 ceph-mon[38221]: pgmap v469: 97 pgs: 97 active+clean; 957 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.9 MiB/s wr, 673 op/s 2024-03-23T18:05:34.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:05:34 smithi092 ceph-mon[79089]: pgmap v469: 97 pgs: 97 active+clean; 957 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.9 MiB/s wr, 673 op/s 2024-03-23T18:05:34.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:05:33 smithi152 ceph-mon[47949]: pgmap v469: 97 pgs: 97 active+clean; 957 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.9 MiB/s wr, 673 op/s 2024-03-23T18:05:35.232 INFO:tasks.workunit.client.0.smithi086.stdout: 10 87400 41.09 MB/sec warmup 119 sec latency 94.224 ms 2024-03-23T18:05:36.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.065+0000 7f4051400700 1 --2- 172.21.15.86:0/4039024558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40541018c0 0x7f4054105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T18:05:36.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.066+0000 7f4058a00700 1 -- 172.21.15.86:0/4039024558 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4054100f10 msgr2=0x7f40541012f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:05:36.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.066+0000 7f4058a00700 1 --2- 172.21.15.86:0/4039024558 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4054100f10 0x7f40541012f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4044009990 tx=0x7f4044009ca0 comp rx=0 tx=0).stop 2024-03-23T18:05:36.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.066+0000 7f4058a00700 1 -- 172.21.15.86:0/4039024558 shutdown_connections 2024-03-23T18:05:36.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.066+0000 7f4058a00700 1 --2- 172.21.15.86:0/4039024558 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4054106030 0x7f405410dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:36.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.066+0000 7f4058a00700 1 --2- 172.21.15.86:0/4039024558 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4054100f10 0x7f40541012f0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:36.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.066+0000 7f4058a00700 1 --2- 172.21.15.86:0/4039024558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40541018c0 0x7f4054105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:36.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.066+0000 7f4058a00700 1 -- 172.21.15.86:0/4039024558 >> 172.21.15.86:0/4039024558 conn(0x7f40540fc770 msgr2=0x7f40540feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:05:36.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.067+0000 7f4058a00700 1 -- 172.21.15.86:0/4039024558 shutdown_connections 2024-03-23T18:05:36.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.067+0000 7f4058a00700 1 -- 172.21.15.86:0/4039024558 wait complete. 2024-03-23T18:05:36.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.068+0000 7f4058a00700 1 Processor -- start 2024-03-23T18:05:36.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.068+0000 7f4058a00700 1 -- start start 2024-03-23T18:05:36.070 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.069+0000 7f4058a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4054100f10 0x7f40540730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:05:36.070 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.069+0000 7f4058a00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4054106030 0x7f40540735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:05:36.070 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.069+0000 7f4058a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4054073b30 0x7f40541a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:05:36.070 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.069+0000 7f4058a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40541a59b0 con 0x7f4054100f10 2024-03-23T18:05:36.070 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.069+0000 7f4058a00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40541a5b20 con 0x7f4054073b30 2024-03-23T18:05:36.070 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.069+0000 7f4058a00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40541a5ce0 con 0x7f4054106030 2024-03-23T18:05:36.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.069+0000 7f4052800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4054073b30 0x7f40541a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:05:36.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.069+0000 7f4051400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4054106030 0x7f40540735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:05:36.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.069+0000 7f4051400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4054106030 0x7f40540735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:55564/0 (socket says 172.21.15.86:55564) 2024-03-23T18:05:36.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.069+0000 7f4051400700 1 -- 172.21.15.86:0/931960238 learned_addr learned my addr 172.21.15.86:0/931960238 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:05:36.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.069+0000 7f4051e00700 1 --2- 172.21.15.86:0/931960238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4054100f10 0x7f40540730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:05:36.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.070+0000 7f4052800700 1 -- 172.21.15.86:0/931960238 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4054106030 msgr2=0x7f40540735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:05:36.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.070+0000 7f4052800700 1 --2- 172.21.15.86:0/931960238 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4054106030 0x7f40540735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:36.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.070+0000 7f4052800700 1 -- 172.21.15.86:0/931960238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4054100f10 msgr2=0x7f40540730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:05:36.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.070+0000 7f4052800700 1 --2- 172.21.15.86:0/931960238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4054100f10 0x7f40540730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:36.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.070+0000 7f4052800700 1 -- 172.21.15.86:0/931960238 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4040009520 con 0x7f4054073b30 2024-03-23T18:05:36.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.070+0000 7f4051e00700 1 --2- 172.21.15.86:0/931960238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4054100f10 0x7f40540730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T18:05:36.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.070+0000 7f4052800700 1 --2- 172.21.15.86:0/931960238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4054073b30 0x7f40541a53e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f404c00be20 tx=0x7f404c009ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:05:36.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.071+0000 7f404aa00700 1 -- 172.21.15.86:0/931960238 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404c00cb90 con 0x7f4054073b30 2024-03-23T18:05:36.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.071+0000 7f4058a00700 1 -- 172.21.15.86:0/931960238 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40440095f0 con 0x7f4054073b30 2024-03-23T18:05:36.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.071+0000 7f4058a00700 1 -- 172.21.15.86:0/931960238 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40541a6310 con 0x7f4054073b30 2024-03-23T18:05:36.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.071+0000 7f404aa00700 1 -- 172.21.15.86:0/931960238 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f404c00ccf0 con 0x7f4054073b30 2024-03-23T18:05:36.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.071+0000 7f404aa00700 1 -- 172.21.15.86:0/931960238 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404c012670 con 0x7f4054073b30 2024-03-23T18:05:36.078 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.077+0000 7f406c52fb80 1 Processor -- start 2024-03-23T18:05:36.079 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.077+0000 7f404aa00700 1 -- 172.21.15.86:0/931960238 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f404c0128b0 con 0x7f4054073b30 2024-03-23T18:05:36.079 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.078+0000 7f406c52fb80 1 -- start start 2024-03-23T18:05:36.079 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.079+0000 7f406c52fb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564368146100 0x5643681464e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:05:36.080 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.079+0000 7f406c52fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564368146a20 0x564368151390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:05:36.080 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.079+0000 7f406c52fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643681518d0 0x564368153cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:05:36.080 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.079+0000 7f406c52fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56436809f930 con 0x5643681518d0 2024-03-23T18:05:36.080 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.079+0000 7f406c52fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x56436809d2f0 con 0x564368146a20 2024-03-23T18:05:36.080 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.079+0000 7f406c52fb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x564368098be0 con 0x564368146100 2024-03-23T18:05:36.081 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.079+0000 7f4051e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564368146a20 0x564368151390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:05:36.081 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.079+0000 7f4051e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564368146a20 0x564368151390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:34188/0 (socket says 172.21.15.86:34188) 2024-03-23T18:05:36.081 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.079+0000 7f4051e00700 1 -- 172.21.15.86:0/310432568 learned_addr learned my addr 172.21.15.86:0/310432568 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:05:36.081 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.080+0000 7f4051400700 1 --2- 172.21.15.86:0/310432568 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643681518d0 0x564368153cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:05:36.081 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.080+0000 7f4051400700 1 --2- 172.21.15.86:0/310432568 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564368146100 0x5643681464e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:05:36.082 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.081+0000 7f4051400700 1 -- 172.21.15.86:0/310432568 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564368146100 msgr2=0x5643681464e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:05:36.082 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.081+0000 7f4051400700 1 --2- 172.21.15.86:0/310432568 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564368146100 0x5643681464e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:36.082 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.081+0000 7f4051400700 1 -- 172.21.15.86:0/310432568 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564368146a20 msgr2=0x564368151390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:05:36.082 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.081+0000 7f4051400700 1 --2- 172.21.15.86:0/310432568 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564368146a20 0x564368151390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:36.082 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.081+0000 7f4051400700 1 -- 172.21.15.86:0/310432568 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564367f1a350 con 0x5643681518d0 2024-03-23T18:05:36.083 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.082+0000 7f4051400700 1 --2- 172.21.15.86:0/310432568 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643681518d0 0x564368153cc0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f4040009fc0 tx=0x7f4040009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9fdf9b8ba37ad0b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:05:36.083 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.082+0000 7f404aa00700 1 --2- 172.21.15.86:0/931960238 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f40380a1390 0x7f40380a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:05:36.083 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.082+0000 7f404aa00700 1 -- 172.21.15.86:0/931960238 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f404c006e40 con 0x7f4054073b30 2024-03-23T18:05:36.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.082+0000 7f403f000700 1 -- 172.21.15.86:0/310432568 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4040004020 con 0x5643681518d0 2024-03-23T18:05:36.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.083+0000 7f403f000700 1 -- 172.21.15.86:0/310432568 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4040004830 con 0x5643681518d0 2024-03-23T18:05:36.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.083+0000 7f403f000700 1 -- 172.21.15.86:0/310432568 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4040005420 con 0x5643681518d0 2024-03-23T18:05:36.085 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.084+0000 7f4051e00700 1 --2- 172.21.15.86:0/931960238 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f40380a1390 0x7f40380a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:05:36.086 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.084+0000 7f406c52fb80 1 -- 172.21.15.86:0/310432568 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643681518d0 msgr2=0x564368153cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:05:36.086 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.085+0000 7f406c52fb80 1 --2- 172.21.15.86:0/310432568 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643681518d0 0x564368153cc0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f4040009fc0 tx=0x7f4040009b20 comp rx=0 tx=0).stop 2024-03-23T18:05:36.086 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.085+0000 7f406c52fb80 1 -- 172.21.15.86:0/310432568 shutdown_connections 2024-03-23T18:05:36.087 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.085+0000 7f406c52fb80 1 --2- 172.21.15.86:0/310432568 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643681518d0 0x564368153cc0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:36.087 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.085+0000 7f406c52fb80 1 --2- 172.21.15.86:0/310432568 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564368146100 0x5643681464e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:36.087 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.085+0000 7f406c52fb80 1 --2- 172.21.15.86:0/310432568 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564368146a20 0x564368151390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:36.087 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.085+0000 7f406c52fb80 1 -- 172.21.15.86:0/310432568 >> 172.21.15.86:0/310432568 conn(0x5643680a6640 msgr2=0x564368142520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:05:36.087 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.085+0000 7f4051e00700 1 --2- 172.21.15.86:0/931960238 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f40380a1390 0x7f40380a3850 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4044009990 tx=0x7f4044005510 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:05:36.087 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.085+0000 7f406c52fb80 1 -- 172.21.15.86:0/310432568 shutdown_connections 2024-03-23T18:05:36.087 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.085+0000 7f406c52fb80 1 -- 172.21.15.86:0/310432568 wait complete. 2024-03-23T18:05:36.088 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.086+0000 7f406c52fb80 1 -- start start 2024-03-23T18:05:36.088 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.087+0000 7f406c52fb80 1 Processor -- start 2024-03-23T18:05:36.088 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.087+0000 7f406c52fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564368146100 0x5643681512d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:05:36.088 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.087+0000 7f406c52fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564368146a20 0x56436814e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:05:36.088 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.087+0000 7f406c52fb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5643681518d0 0x56436814eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:05:36.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.088+0000 7f4051400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564368146a20 0x56436814e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:05:36.091 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.088+0000 7f4051400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564368146a20 0x56436814e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:34200/0 (socket says 172.21.15.86:34200) 2024-03-23T18:05:36.091 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.088+0000 7f4052800700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5643681518d0 0x56436814eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:05:36.091 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.088+0000 7f4052800700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5643681518d0 0x56436814eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:55582/0 (socket says 172.21.15.86:55582) 2024-03-23T18:05:36.091 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.088+0000 7f4052800700 1 -- 172.21.15.86:0/2115265065 learned_addr learned my addr 172.21.15.86:0/2115265065 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:05:36.091 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.088+0000 7f406c52fb80 1 -- 172.21.15.86:0/2115265065 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x564368034d20 con 0x564368146100 2024-03-23T18:05:36.091 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.088+0000 7f406c52fb80 1 -- 172.21.15.86:0/2115265065 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643680349c0 con 0x564368146a20 2024-03-23T18:05:36.091 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.088+0000 7f406c52fb80 1 -- 172.21.15.86:0/2115265065 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x564367f1c640 con 0x5643681518d0 2024-03-23T18:05:36.091 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.088+0000 7f406c52fb80 10 client.0 _ll_register_callbacks cb 0x56436803e410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T18:05:36.091 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.089+0000 7f4051e00700 1 --2- 172.21.15.86:0/2115265065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564368146100 0x5643681512d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:05:36.091 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.089+0000 7f4052800700 1 -- 172.21.15.86:0/2115265065 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564368146a20 msgr2=0x56436814e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:05:36.091 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.089+0000 7f4052800700 1 --2- 172.21.15.86:0/2115265065 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564368146a20 0x56436814e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:36.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.089+0000 7f4052800700 1 -- 172.21.15.86:0/2115265065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564368146100 msgr2=0x5643681512d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:05:36.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.089+0000 7f4052800700 1 --2- 172.21.15.86:0/2115265065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564368146100 0x5643681512d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:36.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.089+0000 7f4052800700 1 -- 172.21.15.86:0/2115265065 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4040011440 con 0x5643681518d0 2024-03-23T18:05:36.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.089+0000 7f4051e00700 1 --2- 172.21.15.86:0/2115265065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564368146100 0x5643681512d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T18:05:36.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.089+0000 7f4052800700 1 --2- 172.21.15.86:0/2115265065 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5643681518d0 0x56436814eb60 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f404c096010 tx=0x7f404c00ce60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8232560764de5e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:05:36.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.090+0000 7f4033400700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-03-23T18:05:36.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.090+0000 7f4033400700 1 -- 172.21.15.86:0/2115265065 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404c09d3d0 con 0x5643681518d0 2024-03-23T18:05:36.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.090+0000 7f4033400700 1 -- 172.21.15.86:0/2115265065 <== mon.1 v2:172.21.15.152:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f404c0a1a20 con 0x5643681518d0 2024-03-23T18:05:36.092 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.090+0000 7f406c52fb80 1 -- 172.21.15.86:0/2115265065 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564368059320 con 0x5643681518d0 2024-03-23T18:05:36.093 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.091+0000 7f4033400700 1 -- 172.21.15.86:0/2115265065 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f404c099800 con 0x5643681518d0 2024-03-23T18:05:36.093 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.091+0000 7f406c52fb80 10 client.27205 fetch_fsmap learned FSMap version 69 2024-03-23T18:05:36.093 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.091+0000 7f406c52fb80 1 -- 172.21.15.86:0/2115265065 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) v3 -- 0x564367ec6b00 con 0x5643681518d0 2024-03-23T18:05:36.093 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.091+0000 7f4033400700 1 -- 172.21.15.86:0/2115265065 <== mon.1 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564368059320 con 0x5643681518d0 2024-03-23T18:05:36.093 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.091+0000 7f4033400700 1 -- 172.21.15.86:0/2115265065 <== mon.1 v2:172.21.15.152:3300/0 5 ==== fsmap(e 69) v1 ==== 7526+0+0 (secure 0 0 0) 0x7f404c0a1590 con 0x5643681518d0 2024-03-23T18:05:36.093 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.091+0000 7f406c52fb80 10 client.27205 fetch_fsmap finished waiting for FSMap version 69 2024-03-23T18:05:36.093 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.091+0000 7f406c52fb80 10 client.27205 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T18:05:36.093 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.091+0000 7f406c52fb80 20 client.27205 populate_metadata read hostname 'smithi086' 2024-03-23T18:05:36.094 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.091+0000 7f406c52fb80 1 --2- 172.21.15.86:0/2115265065 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x564368169ec0 0x56436816c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:05:36.094 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.091+0000 7f406c52fb80 4 client.27205 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T18:05:36.094 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.091+0000 7f406c52fb80 1 -- 172.21.15.86:0/2115265065 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5643680916a0 con 0x564368169ec0 2024-03-23T18:05:36.094 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.092+0000 7f4051e00700 1 --2- 172.21.15.86:0/2115265065 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x564368169ec0 0x56436816c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:05:36.094 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.092+0000 7f4051e00700 1 --2- 172.21.15.86:0/2115265065 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x564368169ec0 0x56436816c2e0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffb6092611513110 server_cookie=5f9e37999a33567 in_seq=0 out_seq=0 2024-03-23T18:05:36.094 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.092+0000 7f4033400700 10 client.27205 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T18:05:36.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.096+0000 7f4033400700 1 -- 172.21.15.86:0/2115265065 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5643680916a0 con 0x564368169ec0 2024-03-23T18:05:36.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.096+0000 7f4033400700 10 client.27205 handle_command_reply: tid=0 2024-03-23T18:05:36.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.096+0000 7f406c52fb80 1 client.27205 shutdown 2024-03-23T18:05:36.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.096+0000 7f406c52fb80 1 -- 172.21.15.86:0/2115265065 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5643681518d0 msgr2=0x56436814eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:05:36.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.096+0000 7f406c52fb80 1 --2- 172.21.15.86:0/2115265065 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5643681518d0 0x56436814eb60 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f404c096010 tx=0x7f404c00ce60 comp rx=0 tx=0).stop 2024-03-23T18:05:36.097 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.096+0000 7f406c52fb80 1 -- 172.21.15.86:0/2115265065 shutdown_connections 2024-03-23T18:05:36.098 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.096+0000 7f406c52fb80 1 --2- 172.21.15.86:0/2115265065 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x564368169ec0 0x56436816c2e0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:36.098 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.096+0000 7f406c52fb80 1 --2- 172.21.15.86:0/2115265065 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5643681518d0 0x56436814eb60 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:36.098 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.096+0000 7f406c52fb80 1 --2- 172.21.15.86:0/2115265065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564368146100 0x5643681512d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:36.098 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.096+0000 7f406c52fb80 1 --2- 172.21.15.86:0/2115265065 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564368146a20 0x56436814e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:36.098 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.096+0000 7f406c52fb80 1 -- 172.21.15.86:0/2115265065 >> 172.21.15.86:0/2115265065 conn(0x5643680a6640 msgr2=0x5643681480a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:05:36.098 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.096+0000 7f4033400700 0 client.27205 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T18:05:36.098 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.097+0000 7f406c52fb80 1 -- 172.21.15.86:0/2115265065 shutdown_connections 2024-03-23T18:05:36.099 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.097+0000 7f406c52fb80 1 -- 172.21.15.86:0/2115265065 wait complete. 2024-03-23T18:05:36.099 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.097+0000 7f406c52fb80 20 client.27205 trim_cache size 0 max 16384 2024-03-23T18:05:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.118+0000 7f406c52fb80 1 Processor -- start 2024-03-23T18:05:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.118+0000 7f406c52fb80 1 -- start start 2024-03-23T18:05:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.118+0000 7f406c52fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564368146100 0x564368158640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:05:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.118+0000 7f406c52fb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564368146a20 0x56436814ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:05:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.118+0000 7f406c52fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5643681518d0 0x564368150bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:05:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.118+0000 7f406c52fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56436814f440 con 0x564368146100 2024-03-23T18:05:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.118+0000 7f406c52fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x56436814a3d0 con 0x5643681518d0 2024-03-23T18:05:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.118+0000 7f406c52fb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643681431f0 con 0x564368146a20 2024-03-23T18:05:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.119+0000 7f4052800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5643681518d0 0x564368150bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:05:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.119+0000 7f4051e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564368146100 0x564368158640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:05:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.119+0000 7f4052800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5643681518d0 0x564368150bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:34202/0 (socket says 172.21.15.86:34202) 2024-03-23T18:05:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.119+0000 7f4052800700 1 -- 172.21.15.86:0/1885316998 learned_addr learned my addr 172.21.15.86:0/1885316998 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:05:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.119+0000 7f4052800700 1 -- 172.21.15.86:0/1885316998 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564368146a20 msgr2=0x56436814ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:05:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.119+0000 7f4052800700 1 --2- 172.21.15.86:0/1885316998 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564368146a20 0x56436814ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.119+0000 7f4052800700 1 -- 172.21.15.86:0/1885316998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564368146100 msgr2=0x564368158640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:05:36.121 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.119+0000 7f4052800700 1 --2- 172.21.15.86:0/1885316998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564368146100 0x564368158640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:36.121 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.119+0000 7f4052800700 1 -- 172.21.15.86:0/1885316998 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56436805ee40 con 0x5643681518d0 2024-03-23T18:05:36.121 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.120+0000 7f4052800700 1 --2- 172.21.15.86:0/1885316998 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5643681518d0 0x564368150bd0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f404c00bdf0 tx=0x7f404c099de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a7f221c3736dbc2 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:05:36.121 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.120+0000 7f403fa00700 1 -- 172.21.15.86:0/1885316998 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404c0958e0 con 0x5643681518d0 2024-03-23T18:05:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.120+0000 7f403fa00700 1 -- 172.21.15.86:0/1885316998 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f404c0a1a20 con 0x5643681518d0 2024-03-23T18:05:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.120+0000 7f403fa00700 1 -- 172.21.15.86:0/1885316998 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404c098a90 con 0x5643681518d0 2024-03-23T18:05:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.121+0000 7f406c52fb80 1 -- 172.21.15.86:0/1885316998 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5643681518d0 msgr2=0x564368150bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:05:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.121+0000 7f406c52fb80 1 --2- 172.21.15.86:0/1885316998 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5643681518d0 0x564368150bd0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f404c00bdf0 tx=0x7f404c099de0 comp rx=0 tx=0).stop 2024-03-23T18:05:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.122+0000 7f406c52fb80 1 -- 172.21.15.86:0/1885316998 shutdown_connections 2024-03-23T18:05:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.122+0000 7f406c52fb80 1 --2- 172.21.15.86:0/1885316998 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5643681518d0 0x564368150bd0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.122+0000 7f406c52fb80 1 --2- 172.21.15.86:0/1885316998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564368146100 0x564368158640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.122+0000 7f406c52fb80 1 --2- 172.21.15.86:0/1885316998 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564368146a20 0x56436814ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.122+0000 7f406c52fb80 1 -- 172.21.15.86:0/1885316998 >> 172.21.15.86:0/1885316998 conn(0x5643680a6640 msgr2=0x56436814ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:05:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.122+0000 7f406c52fb80 1 -- 172.21.15.86:0/1885316998 shutdown_connections 2024-03-23T18:05:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.122+0000 7f406c52fb80 1 -- 172.21.15.86:0/1885316998 wait complete. 2024-03-23T18:05:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.122+0000 7f406c52fb80 1 -- start start 2024-03-23T18:05:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.123+0000 7f406c52fb80 1 Processor -- start 2024-03-23T18:05:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.123+0000 7f406c52fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564368146100 0x564368158640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:05:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.123+0000 7f406c52fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564368146a20 0x564368156620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:05:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.123+0000 7f406c52fb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5643681518d0 0x564368156d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:05:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.123+0000 7f406c52fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x564368157470 con 0x564368146100 2024-03-23T18:05:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.123+0000 7f406c52fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643681ba9a0 con 0x564368146a20 2024-03-23T18:05:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.124+0000 7f406c52fb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x56436815e6b0 con 0x5643681518d0 2024-03-23T18:05:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.124+0000 7f406c52fb80 10 client.0 _ll_register_callbacks cb 0x56436803e410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T18:05:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.124+0000 7f4051400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564368146a20 0x564368156620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:05:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.124+0000 7f4051400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564368146a20 0x564368156620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:34204/0 (socket says 172.21.15.86:34204) 2024-03-23T18:05:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.124+0000 7f4051400700 1 -- 172.21.15.86:0/814933498 learned_addr learned my addr 172.21.15.86:0/814933498 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:05:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.124+0000 7f4051400700 1 -- 172.21.15.86:0/814933498 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5643681518d0 msgr2=0x564368156d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:05:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.124+0000 7f4051e00700 1 --2- 172.21.15.86:0/814933498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564368146100 0x564368158640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:05:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.124+0000 7f4052800700 1 --2- 172.21.15.86:0/814933498 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5643681518d0 0x564368156d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:05:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.125+0000 7f4051400700 1 --2- 172.21.15.86:0/814933498 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5643681518d0 0x564368156d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.125+0000 7f4051400700 1 -- 172.21.15.86:0/814933498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564368146100 msgr2=0x564368158640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:05:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.125+0000 7f4051400700 1 --2- 172.21.15.86:0/814933498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564368146100 0x564368158640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.125+0000 7f4051400700 1 -- 172.21.15.86:0/814933498 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564367ec6b00 con 0x564368146a20 2024-03-23T18:05:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.125+0000 7f4051e00700 1 --2- 172.21.15.86:0/814933498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564368146100 0x564368158640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T18:05:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.125+0000 7f4052800700 1 --2- 172.21.15.86:0/814933498 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5643681518d0 0x564368156d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T18:05:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.125+0000 7f4051400700 1 --2- 172.21.15.86:0/814933498 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564368146a20 0x564368156620 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x56436814fb90 tx=0x7f404000fc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f76e9ea1215e1d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:05:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.125+0000 7f4033400700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T18:05:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.125+0000 7f4033400700 1 -- 172.21.15.86:0/814933498 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404002b070 con 0x564368146a20 2024-03-23T18:05:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.125+0000 7f4033400700 1 -- 172.21.15.86:0/814933498 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f4040004020 con 0x564368146a20 2024-03-23T18:05:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.125+0000 7f406c52fb80 1 -- 172.21.15.86:0/814933498 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4040004830 con 0x564368146a20 2024-03-23T18:05:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.126+0000 7f4033400700 1 -- 172.21.15.86:0/814933498 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4040023040 con 0x564368146a20 2024-03-23T18:05:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.126+0000 7f406c52fb80 10 client.27224 fetch_fsmap learned FSMap version 69 2024-03-23T18:05:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.126+0000 7f406c52fb80 1 -- 172.21.15.86:0/814933498 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) v3 -- 0x564367f1ec70 con 0x564368146a20 2024-03-23T18:05:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.126+0000 7f4033400700 1 -- 172.21.15.86:0/814933498 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4040004830 con 0x564368146a20 2024-03-23T18:05:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.127+0000 7f4033400700 1 -- 172.21.15.86:0/814933498 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 69) v1 ==== 7526+0+0 (secure 0 0 0) 0x7f404002e030 con 0x564368146a20 2024-03-23T18:05:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.127+0000 7f406c52fb80 10 client.27224 fetch_fsmap finished waiting for FSMap version 69 2024-03-23T18:05:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.127+0000 7f406c52fb80 10 client.27224 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T18:05:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.127+0000 7f406c52fb80 20 client.27224 populate_metadata read hostname 'smithi086' 2024-03-23T18:05:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.127+0000 7f406c52fb80 1 --2- 172.21.15.86:0/814933498 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x56436815fdd0 0x5643681621f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:05:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.127+0000 7f406c52fb80 4 client.27224 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T18:05:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.127+0000 7f406c52fb80 1 -- 172.21.15.86:0/814933498 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564367ee7c90 con 0x56436815fdd0 2024-03-23T18:05:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.127+0000 7f4051e00700 1 --2- 172.21.15.86:0/814933498 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x56436815fdd0 0x5643681621f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:05:36.130 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.128+0000 7f4051e00700 1 --2- 172.21.15.86:0/814933498 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x56436815fdd0 0x5643681621f0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51858976e5f692c server_cookie=1661c85a01ca50a0 in_seq=0 out_seq=0 2024-03-23T18:05:36.130 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.128+0000 7f4033400700 10 client.27224 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T18:05:36.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:05:36 smithi086 ceph-mon[38221]: pgmap v470: 97 pgs: 97 active+clean; 1023 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 14 MiB/s wr, 976 op/s 2024-03-23T18:05:36.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:05:36 smithi092 ceph-mon[79089]: pgmap v470: 97 pgs: 97 active+clean; 1023 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 14 MiB/s wr, 976 op/s 2024-03-23T18:05:36.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:05:36 smithi152 ceph-mon[47949]: pgmap v470: 97 pgs: 97 active+clean; 1023 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 14 MiB/s wr, 976 op/s 2024-03-23T18:05:36.365 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.364+0000 7f4033400700 1 -- 172.21.15.86:0/814933498 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564367ee7c90 con 0x56436815fdd0 2024-03-23T18:05:36.365 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.364+0000 7f4033400700 10 client.27224 handle_command_reply: tid=0 2024-03-23T18:05:36.365 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.364+0000 7f406c52fb80 1 client.27224 shutdown 2024-03-23T18:05:36.366 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.365+0000 7f406c52fb80 1 -- 172.21.15.86:0/814933498 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564368146a20 msgr2=0x564368156620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:05:36.366 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.365+0000 7f406c52fb80 1 --2- 172.21.15.86:0/814933498 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564368146a20 0x564368156620 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x56436814fb90 tx=0x7f404000fc90 comp rx=0 tx=0).stop 2024-03-23T18:05:36.366 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.365+0000 7f406c52fb80 1 -- 172.21.15.86:0/814933498 shutdown_connections 2024-03-23T18:05:36.367 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.365+0000 7f406c52fb80 1 --2- 172.21.15.86:0/814933498 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x56436815fdd0 0x5643681621f0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:36.367 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.365+0000 7f406c52fb80 1 --2- 172.21.15.86:0/814933498 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5643681518d0 0x564368156d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:36.367 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.365+0000 7f406c52fb80 1 --2- 172.21.15.86:0/814933498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564368146100 0x564368158640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:36.367 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.365+0000 7f406c52fb80 1 --2- 172.21.15.86:0/814933498 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564368146a20 0x564368156620 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:36.367 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.365+0000 7f4033400700 0 client.27224 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T18:05:36.367 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.365+0000 7f406c52fb80 1 -- 172.21.15.86:0/814933498 >> 172.21.15.86:0/814933498 conn(0x5643680a6640 msgr2=0x56436813b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:05:36.368 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.367+0000 7f406c52fb80 1 -- 172.21.15.86:0/814933498 shutdown_connections 2024-03-23T18:05:36.368 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.367+0000 7f406c52fb80 1 -- 172.21.15.86:0/814933498 wait complete. 2024-03-23T18:05:36.370 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.369+0000 7f406c52fb80 20 client.27224 trim_cache size 0 max 16384 2024-03-23T18:05:36.370 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-03-23T18:05:36.370 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-03-23T18:05:36.370 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-03-23T18:05:36.370 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-03-23T18:05:36.374 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.369+0000 7f4058a00700 1 -- 172.21.15.86:0/931960238 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f40380a1390 msgr2=0x7f40380a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:05:36.374 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.369+0000 7f4058a00700 1 --2- 172.21.15.86:0/931960238 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f40380a1390 0x7f40380a3850 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4044009990 tx=0x7f4044005510 comp rx=0 tx=0).stop 2024-03-23T18:05:36.374 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.369+0000 7f4058a00700 1 -- 172.21.15.86:0/931960238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4054073b30 msgr2=0x7f40541a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:05:36.374 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.369+0000 7f4058a00700 1 --2- 172.21.15.86:0/931960238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4054073b30 0x7f40541a53e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f404c00be20 tx=0x7f404c009ed0 comp rx=0 tx=0).stop 2024-03-23T18:05:36.374 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.370+0000 7f4058a00700 1 -- 172.21.15.86:0/931960238 shutdown_connections 2024-03-23T18:05:36.374 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.370+0000 7f4058a00700 1 --2- 172.21.15.86:0/931960238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4054073b30 0x7f40541a53e0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:36.374 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.370+0000 7f4058a00700 1 --2- 172.21.15.86:0/931960238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4054100f10 0x7f40540730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:36.374 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.370+0000 7f4058a00700 1 --2- 172.21.15.86:0/931960238 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f40380a1390 0x7f40380a3850 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:36.374 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.370+0000 7f4058a00700 1 --2- 172.21.15.86:0/931960238 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4054106030 0x7f40540735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:36.375 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.370+0000 7f4058a00700 1 -- 172.21.15.86:0/931960238 >> 172.21.15.86:0/931960238 conn(0x7f40540fc770 msgr2=0x7f40540fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:05:36.375 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.370+0000 7f4058a00700 1 -- 172.21.15.86:0/931960238 shutdown_connections 2024-03-23T18:05:36.375 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:36.371+0000 7f4058a00700 1 -- 172.21.15.86:0/931960238 wait complete. 2024-03-23T18:05:37.232 INFO:tasks.workunit.client.0.smithi086.stdout: 10 88960 38.46 MB/sec execute 1 sec latency 138.499 ms 2024-03-23T18:05:38.232 INFO:tasks.workunit.client.0.smithi086.stdout: 10 89900 41.84 MB/sec execute 2 sec latency 79.943 ms 2024-03-23T18:05:38.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:05:38 smithi086 ceph-mon[38221]: pgmap v471: 97 pgs: 97 active+clean; 1.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 15 MiB/s wr, 837 op/s 2024-03-23T18:05:38.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:05:38 smithi092 ceph-mon[79089]: pgmap v471: 97 pgs: 97 active+clean; 1.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 15 MiB/s wr, 837 op/s 2024-03-23T18:05:38.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:05:38 smithi152 ceph-mon[47949]: pgmap v471: 97 pgs: 97 active+clean; 1.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 15 MiB/s wr, 837 op/s 2024-03-23T18:05:39.235 INFO:tasks.workunit.client.0.smithi086.stdout: 10 90729 44.37 MB/sec execute 3 sec latency 215.407 ms 2024-03-23T18:05:40.235 INFO:tasks.workunit.client.0.smithi086.stdout: 10 90913 36.61 MB/sec execute 4 sec latency 978.190 ms 2024-03-23T18:05:40.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:05:40 smithi092 ceph-mon[79089]: pgmap v472: 97 pgs: 97 active+clean; 1.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 704 op/s 2024-03-23T18:05:40.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:05:40 smithi152 ceph-mon[47949]: pgmap v472: 97 pgs: 97 active+clean; 1.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 704 op/s 2024-03-23T18:05:40.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:05:40 smithi086 ceph-mon[38221]: pgmap v472: 97 pgs: 97 active+clean; 1.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 704 op/s 2024-03-23T18:05:41.235 INFO:tasks.workunit.client.0.smithi086.stdout: 10 91367 34.76 MB/sec execute 5 sec latency 360.724 ms 2024-03-23T18:05:42.235 INFO:tasks.workunit.client.0.smithi086.stdout: 10 91922 33.51 MB/sec execute 6 sec latency 101.353 ms 2024-03-23T18:05:42.310 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:05:42 smithi092 ceph-mon[79089]: pgmap v473: 97 pgs: 97 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 25 MiB/s wr, 1.28k op/s 2024-03-23T18:05:42.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:05:42 smithi152 ceph-mon[47949]: pgmap v473: 97 pgs: 97 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 25 MiB/s wr, 1.28k op/s 2024-03-23T18:05:42.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:05:42 smithi086 ceph-mon[38221]: pgmap v473: 97 pgs: 97 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 25 MiB/s wr, 1.28k op/s 2024-03-23T18:05:43.235 INFO:tasks.workunit.client.0.smithi086.stdout: 10 92749 33.77 MB/sec execute 7 sec latency 47.065 ms 2024-03-23T18:05:44.235 INFO:tasks.workunit.client.0.smithi086.stdout: 10 93724 37.19 MB/sec execute 8 sec latency 45.373 ms 2024-03-23T18:05:44.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:05:44 smithi092 ceph-mon[79089]: pgmap v474: 97 pgs: 97 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 22 MiB/s wr, 976 op/s 2024-03-23T18:05:44.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:05:44 smithi152 ceph-mon[47949]: pgmap v474: 97 pgs: 97 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 22 MiB/s wr, 976 op/s 2024-03-23T18:05:44.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:05:44 smithi086 ceph-mon[38221]: pgmap v474: 97 pgs: 97 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 22 MiB/s wr, 976 op/s 2024-03-23T18:05:45.235 INFO:tasks.workunit.client.0.smithi086.stdout: 10 94489 37.37 MB/sec execute 9 sec latency 59.160 ms 2024-03-23T18:05:45.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:05:45 smithi092 ceph-mon[79089]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:05:45.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:05:45 smithi092 ceph-mon[79089]: pgmap v475: 97 pgs: 97 active+clean; 1.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 24 MiB/s wr, 1.19k op/s 2024-03-23T18:05:45.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:05:45 smithi152 ceph-mon[47949]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:05:45.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:05:45 smithi152 ceph-mon[47949]: pgmap v475: 97 pgs: 97 active+clean; 1.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 24 MiB/s wr, 1.19k op/s 2024-03-23T18:05:45.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:05:45 smithi086 ceph-mon[38221]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:05:45.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:05:45 smithi086 ceph-mon[38221]: pgmap v475: 97 pgs: 97 active+clean; 1.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 24 MiB/s wr, 1.19k op/s 2024-03-23T18:05:46.236 INFO:tasks.workunit.client.0.smithi086.stdout: 10 95198 37.27 MB/sec execute 10 sec latency 261.875 ms 2024-03-23T18:05:47.237 INFO:tasks.workunit.client.0.smithi086.stdout: 10 96087 38.22 MB/sec execute 11 sec latency 289.208 ms 2024-03-23T18:05:48.236 INFO:tasks.workunit.client.0.smithi086.stdout: 10 97027 39.80 MB/sec execute 12 sec latency 60.087 ms 2024-03-23T18:05:48.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:05:47 smithi086 ceph-mon[38221]: pgmap v476: 97 pgs: 97 active+clean; 1008 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 20 MiB/s wr, 1.14k op/s 2024-03-23T18:05:48.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:05:47 smithi092 ceph-mon[79089]: pgmap v476: 97 pgs: 97 active+clean; 1008 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 20 MiB/s wr, 1.14k op/s 2024-03-23T18:05:48.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:05:47 smithi152 ceph-mon[47949]: pgmap v476: 97 pgs: 97 active+clean; 1008 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 20 MiB/s wr, 1.14k op/s 2024-03-23T18:05:49.236 INFO:tasks.workunit.client.0.smithi086.stdout: 10 97881 40.21 MB/sec execute 13 sec latency 53.853 ms 2024-03-23T18:05:49.596 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.595+0000 7f3457e00700 1 -- 172.21.15.86:0/2052735010 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3450073aa0 msgr2=0x7f345010d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:05:49.596 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.595+0000 7f3457e00700 1 --2- 172.21.15.86:0/2052735010 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3450073aa0 0x7f345010d000 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f343c0098f0 tx=0x7f343c009c00 comp rx=0 tx=0).stop 2024-03-23T18:05:49.596 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.596+0000 7f3457e00700 1 -- 172.21.15.86:0/2052735010 shutdown_connections 2024-03-23T18:05:49.596 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.596+0000 7f3457e00700 1 --2- 172.21.15.86:0/2052735010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f345010d540 0x7f345010f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:49.596 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.596+0000 7f3457e00700 1 --2- 172.21.15.86:0/2052735010 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3450073180 0x7f3450073560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:49.596 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.596+0000 7f3457e00700 1 --2- 172.21.15.86:0/2052735010 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3450073aa0 0x7f345010d000 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.596+0000 7f3457e00700 1 -- 172.21.15.86:0/2052735010 >> 172.21.15.86:0/2052735010 conn(0x7f34500fc770 msgr2=0x7f34500feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:05:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.596+0000 7f3457e00700 1 -- 172.21.15.86:0/2052735010 shutdown_connections 2024-03-23T18:05:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.596+0000 7f3457e00700 1 -- 172.21.15.86:0/2052735010 wait complete. 2024-03-23T18:05:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.597+0000 7f3457e00700 1 Processor -- start 2024-03-23T18:05:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.598+0000 7f3457e00700 1 -- start start 2024-03-23T18:05:49.598 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.598+0000 7f3457e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3450073180 0x7f3450199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:05:49.598 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.598+0000 7f3457e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3450073aa0 0x7f3450199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:05:49.599 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.598+0000 7f3457e00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f345010d540 0x7f345019dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:05:49.599 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.598+0000 7f3457e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f345019e330 con 0x7f3450073aa0 2024-03-23T18:05:49.599 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.598+0000 7f3457e00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f345019e4a0 con 0x7f3450073180 2024-03-23T18:05:49.599 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.598+0000 7f3457e00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f345019e660 con 0x7f345010d540 2024-03-23T18:05:49.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.599+0000 7f3455400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3450073180 0x7f3450199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:05:49.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.599+0000 7f3455400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3450073180 0x7f3450199410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:46296/0 (socket says 172.21.15.86:46296) 2024-03-23T18:05:49.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.599+0000 7f3455400700 1 -- 172.21.15.86:0/693288259 learned_addr learned my addr 172.21.15.86:0/693288259 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:05:49.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.599+0000 7f3455e00700 1 --2- 172.21.15.86:0/693288259 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f345010d540 0x7f345019dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:05:49.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.599+0000 7f3454a00700 1 --2- 172.21.15.86:0/693288259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3450073aa0 0x7f3450199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:05:49.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.599+0000 7f3455e00700 1 -- 172.21.15.86:0/693288259 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3450073180 msgr2=0x7f3450199410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:05:49.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.599+0000 7f3455e00700 1 --2- 172.21.15.86:0/693288259 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3450073180 0x7f3450199410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:49.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.599+0000 7f3455e00700 1 -- 172.21.15.86:0/693288259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3450073aa0 msgr2=0x7f3450199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:05:49.601 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.599+0000 7f3455e00700 1 --2- 172.21.15.86:0/693288259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3450073aa0 0x7f3450199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:49.601 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.599+0000 7f3455e00700 1 -- 172.21.15.86:0/693288259 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f343c009520 con 0x7f345010d540 2024-03-23T18:05:49.601 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.599+0000 7f3454a00700 1 --2- 172.21.15.86:0/693288259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3450073aa0 0x7f3450199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T18:05:49.601 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.599+0000 7f3455400700 1 --2- 172.21.15.86:0/693288259 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3450073180 0x7f3450199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T18:05:49.601 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.601+0000 7f3455e00700 1 --2- 172.21.15.86:0/693288259 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f345010d540 0x7f345019dcf0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3448007ee0 tx=0x7f344800d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:05:49.602 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.601+0000 7f344e000700 1 -- 172.21.15.86:0/693288259 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344800ddd0 con 0x7f345010d540 2024-03-23T18:05:49.602 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.601+0000 7f3457e00700 1 -- 172.21.15.86:0/693288259 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f345019e940 con 0x7f345010d540 2024-03-23T18:05:49.602 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.601+0000 7f3457e00700 1 -- 172.21.15.86:0/693288259 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3450100d70 con 0x7f345010d540 2024-03-23T18:05:49.602 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.601+0000 7f344e000700 1 -- 172.21.15.86:0/693288259 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f344800f040 con 0x7f345010d540 2024-03-23T18:05:49.602 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.601+0000 7f344e000700 1 -- 172.21.15.86:0/693288259 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344800ddd0 con 0x7f345010d540 2024-03-23T18:05:49.606 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.606+0000 7f344e000700 1 -- 172.21.15.86:0/693288259 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f34480058f0 con 0x7f345010d540 2024-03-23T18:05:49.608 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.609+0000 7f344e000700 1 --2- 172.21.15.86:0/693288259 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f34380a1350 0x7f34380a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:05:49.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.609+0000 7f344e000700 1 -- 172.21.15.86:0/693288259 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f34480cc080 con 0x7f345010d540 2024-03-23T18:05:49.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.609+0000 7f346b931b80 1 Processor -- start 2024-03-23T18:05:49.610 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.610+0000 7f346b931b80 1 -- start start 2024-03-23T18:05:49.610 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.610+0000 7f346b931b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55557701b100 0x55557701b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:05:49.610 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.610+0000 7f346b931b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55557701ba20 0x555577026390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:05:49.611 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.610+0000 7f346b931b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555770268d0 0x555577028cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:05:49.612 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.611+0000 7f346b931b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x555576f74930 con 0x55557701b100 2024-03-23T18:05:49.612 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.611+0000 7f346b931b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x555576f722f0 con 0x55557701ba20 2024-03-23T18:05:49.612 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.611+0000 7f346b931b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x555576f6dbe0 con 0x5555770268d0 2024-03-23T18:05:49.612 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.611+0000 7f3455e00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555770268d0 0x555577028cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:05:49.612 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.611+0000 7f3455e00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555770268d0 0x555577028cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:43204/0 (socket says 172.21.15.86:43204) 2024-03-23T18:05:49.612 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.611+0000 7f3455e00700 1 -- 172.21.15.86:0/2140807827 learned_addr learned my addr 172.21.15.86:0/2140807827 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:05:49.612 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.611+0000 7f3454a00700 1 --2- 172.21.15.86:0/2140807827 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55557701ba20 0x555577026390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:05:49.612 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.612+0000 7f3455e00700 1 -- 172.21.15.86:0/2140807827 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55557701ba20 msgr2=0x555577026390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:05:49.612 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.612+0000 7f3455e00700 1 --2- 172.21.15.86:0/2140807827 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55557701ba20 0x555577026390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:49.612 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.612+0000 7f3455e00700 1 -- 172.21.15.86:0/2140807827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55557701b100 msgr2=0x55557701b4e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-23T18:05:49.613 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.612+0000 7f3455e00700 1 --2- 172.21.15.86:0/2140807827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55557701b100 0x55557701b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:49.613 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.612+0000 7f3455e00700 1 -- 172.21.15.86:0/2140807827 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555576def350 con 0x5555770268d0 2024-03-23T18:05:49.613 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.612+0000 7f3455400700 1 --2- 172.21.15.86:0/693288259 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f34380a1350 0x7f34380a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:05:49.613 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.612+0000 7f3455e00700 1 --2- 172.21.15.86:0/2140807827 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555770268d0 0x555577028cc0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f34480956e0 tx=0x7f34480998e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=efad92b69a43943d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:05:49.614 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.613+0000 7f3442600700 1 -- 172.21.15.86:0/2140807827 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344800ddd0 con 0x5555770268d0 2024-03-23T18:05:49.614 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.613+0000 7f3442600700 1 -- 172.21.15.86:0/2140807827 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f344809d010 con 0x5555770268d0 2024-03-23T18:05:49.614 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.613+0000 7f3442600700 1 -- 172.21.15.86:0/2140807827 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3448099440 con 0x5555770268d0 2024-03-23T18:05:49.614 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.614+0000 7f346b931b80 1 -- 172.21.15.86:0/2140807827 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555770268d0 msgr2=0x555577028cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:05:49.614 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.614+0000 7f3455400700 1 --2- 172.21.15.86:0/693288259 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f34380a1350 0x7f34380a3810 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3444005950 tx=0x7f3444009310 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:05:49.614 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.614+0000 7f346b931b80 1 --2- 172.21.15.86:0/2140807827 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555770268d0 0x555577028cc0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f34480956e0 tx=0x7f34480998e0 comp rx=0 tx=0).stop 2024-03-23T18:05:49.615 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.615+0000 7f346b931b80 1 -- 172.21.15.86:0/2140807827 shutdown_connections 2024-03-23T18:05:49.615 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.615+0000 7f346b931b80 1 --2- 172.21.15.86:0/2140807827 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555770268d0 0x555577028cc0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:49.615 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.615+0000 7f346b931b80 1 --2- 172.21.15.86:0/2140807827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55557701b100 0x55557701b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:49.615 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.615+0000 7f346b931b80 1 --2- 172.21.15.86:0/2140807827 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55557701ba20 0x555577026390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:49.615 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.615+0000 7f346b931b80 1 -- 172.21.15.86:0/2140807827 >> 172.21.15.86:0/2140807827 conn(0x555576f7b640 msgr2=0x555577017520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:05:49.616 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.617+0000 7f346b931b80 1 -- 172.21.15.86:0/2140807827 shutdown_connections 2024-03-23T18:05:49.617 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.617+0000 7f346b931b80 1 -- 172.21.15.86:0/2140807827 wait complete. 2024-03-23T18:05:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.619+0000 7f346b931b80 1 -- start start 2024-03-23T18:05:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.620+0000 7f346b931b80 1 Processor -- start 2024-03-23T18:05:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.620+0000 7f346b931b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55557701b100 0x5555770262f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:05:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.620+0000 7f346b931b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55557701ba20 0x5555770235e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:05:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.620+0000 7f346b931b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555770268d0 0x555577023c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:05:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.620+0000 7f346b931b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x555576f09d20 con 0x55557701ba20 2024-03-23T18:05:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.620+0000 7f346b931b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x555576f099c0 con 0x55557701b100 2024-03-23T18:05:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.620+0000 7f346b931b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x555576df1640 con 0x5555770268d0 2024-03-23T18:05:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.620+0000 7f346b931b80 10 client.0 _ll_register_callbacks cb 0x555576f13410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T18:05:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.621+0000 7f3455400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55557701b100 0x5555770262f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:05:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.621+0000 7f3455400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55557701b100 0x5555770262f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:46304/0 (socket says 172.21.15.86:46304) 2024-03-23T18:05:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.621+0000 7f3455400700 1 -- 172.21.15.86:0/1312473263 learned_addr learned my addr 172.21.15.86:0/1312473263 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:05:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.621+0000 7f3454a00700 1 --2- 172.21.15.86:0/1312473263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55557701ba20 0x5555770235e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:05:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.621+0000 7f3455e00700 1 --2- 172.21.15.86:0/1312473263 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555770268d0 0x555577023c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:05:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.621+0000 7f3455400700 1 -- 172.21.15.86:0/1312473263 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555770268d0 msgr2=0x555577023c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:05:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.621+0000 7f3455400700 1 --2- 172.21.15.86:0/1312473263 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555770268d0 0x555577023c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.621+0000 7f3455400700 1 -- 172.21.15.86:0/1312473263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55557701ba20 msgr2=0x5555770235e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:05:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.621+0000 7f3455400700 1 --2- 172.21.15.86:0/1312473263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55557701ba20 0x5555770235e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.621+0000 7f3455400700 1 -- 172.21.15.86:0/1312473263 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3450100d70 con 0x55557701b100 2024-03-23T18:05:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.621+0000 7f3455e00700 1 --2- 172.21.15.86:0/1312473263 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555770268d0 0x555577023c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T18:05:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.621+0000 7f3454a00700 1 --2- 172.21.15.86:0/1312473263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55557701ba20 0x5555770235e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T18:05:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.622+0000 7f3455400700 1 --2- 172.21.15.86:0/1312473263 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55557701b100 0x5555770262f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3444012870 tx=0x7f34440047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8476caab940eab7e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:05:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.622+0000 7f3436a00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T18:05:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.622+0000 7f3436a00700 1 -- 172.21.15.86:0/1312473263 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344401f830 con 0x55557701b100 2024-03-23T18:05:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.622+0000 7f3436a00700 1 -- 172.21.15.86:0/1312473263 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f3444018070 con 0x55557701b100 2024-03-23T18:05:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.622+0000 7f346b931b80 1 -- 172.21.15.86:0/1312473263 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555576f2e320 con 0x55557701b100 2024-03-23T18:05:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.622+0000 7f3436a00700 1 -- 172.21.15.86:0/1312473263 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f344401f990 con 0x55557701b100 2024-03-23T18:05:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.623+0000 7f346b931b80 10 client.27236 fetch_fsmap learned FSMap version 70 2024-03-23T18:05:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.623+0000 7f346b931b80 1 -- 172.21.15.86:0/1312473263 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x555576d9bb00 con 0x55557701b100 2024-03-23T18:05:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.623+0000 7f3436a00700 1 -- 172.21.15.86:0/1312473263 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555576f2e320 con 0x55557701b100 2024-03-23T18:05:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.624+0000 7f3436a00700 1 -- 172.21.15.86:0/1312473263 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 70) v1 ==== 7530+0+0 (secure 0 0 0) 0x7f3444025020 con 0x55557701b100 2024-03-23T18:05:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.624+0000 7f346b931b80 10 client.27236 fetch_fsmap finished waiting for FSMap version 70 2024-03-23T18:05:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.624+0000 7f346b931b80 10 client.27236 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T18:05:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.624+0000 7f346b931b80 20 client.27236 populate_metadata read hostname 'smithi086' 2024-03-23T18:05:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.624+0000 7f346b931b80 1 --2- 172.21.15.86:0/1312473263 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5555770367b0 0x555577038bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:05:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.624+0000 7f346b931b80 4 client.27236 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T18:05:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.624+0000 7f346b931b80 1 -- 172.21.15.86:0/1312473263 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555576f666a0 con 0x5555770367b0 2024-03-23T18:05:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.625+0000 7f3454a00700 1 --2- 172.21.15.86:0/1312473263 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5555770367b0 0x555577038bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:05:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.626+0000 7f3454a00700 1 --2- 172.21.15.86:0/1312473263 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5555770367b0 0x555577038bd0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=960fd4bafa719486 server_cookie=a51a96fb54f4cc19 in_seq=0 out_seq=0 2024-03-23T18:05:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.626+0000 7f3436a00700 10 client.27236 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T18:05:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.631+0000 7f3436a00700 1 -- 172.21.15.86:0/1312473263 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x555576f666a0 con 0x5555770367b0 2024-03-23T18:05:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.631+0000 7f3436a00700 10 client.27236 handle_command_reply: tid=0 2024-03-23T18:05:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.631+0000 7f346b931b80 1 client.27236 shutdown 2024-03-23T18:05:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.631+0000 7f346b931b80 1 -- 172.21.15.86:0/1312473263 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55557701b100 msgr2=0x5555770262f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:05:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.631+0000 7f346b931b80 1 --2- 172.21.15.86:0/1312473263 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55557701b100 0x5555770262f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3444012870 tx=0x7f34440047c0 comp rx=0 tx=0).stop 2024-03-23T18:05:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.632+0000 7f346b931b80 1 -- 172.21.15.86:0/1312473263 shutdown_connections 2024-03-23T18:05:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.632+0000 7f346b931b80 1 --2- 172.21.15.86:0/1312473263 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5555770367b0 0x555577038bd0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.632+0000 7f346b931b80 1 --2- 172.21.15.86:0/1312473263 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555770268d0 0x555577023c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.632+0000 7f346b931b80 1 --2- 172.21.15.86:0/1312473263 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55557701b100 0x5555770262f0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.632+0000 7f346b931b80 1 --2- 172.21.15.86:0/1312473263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55557701ba20 0x5555770235e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.632+0000 7f346b931b80 1 -- 172.21.15.86:0/1312473263 >> 172.21.15.86:0/1312473263 conn(0x555576f7b640 msgr2=0x55557701d0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:05:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.632+0000 7f3436a00700 0 client.27236 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T18:05:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.632+0000 7f346b931b80 1 -- 172.21.15.86:0/1312473263 shutdown_connections 2024-03-23T18:05:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.632+0000 7f346b931b80 1 -- 172.21.15.86:0/1312473263 wait complete. 2024-03-23T18:05:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.632+0000 7f346b931b80 20 client.27236 trim_cache size 0 max 16384 2024-03-23T18:05:49.654 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.654+0000 7f346b931b80 1 Processor -- start 2024-03-23T18:05:49.655 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.654+0000 7f346b931b80 1 -- start start 2024-03-23T18:05:49.655 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.654+0000 7f346b931b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55557701b100 0x55557702d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:05:49.655 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.654+0000 7f346b931b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55557701ba20 0x5555770235a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:05:49.655 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.655+0000 7f346b931b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5555770268d0 0x555577021eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:05:49.655 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.655+0000 7f346b931b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x555577018ac0 con 0x5555770268d0 2024-03-23T18:05:49.655 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.655+0000 7f346b931b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x555577013400 con 0x55557701b100 2024-03-23T18:05:49.655 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.655+0000 7f346b931b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555770233b0 con 0x55557701ba20 2024-03-23T18:05:49.655 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.655+0000 7f3454a00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55557701ba20 0x5555770235a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:05:49.655 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.655+0000 7f3455400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55557701b100 0x55557702d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:05:49.655 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.655+0000 7f3455400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55557701b100 0x55557702d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:46320/0 (socket says 172.21.15.86:46320) 2024-03-23T18:05:49.656 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.655+0000 7f3454a00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55557701ba20 0x5555770235a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:43220/0 (socket says 172.21.15.86:43220) 2024-03-23T18:05:49.656 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.655+0000 7f3455400700 1 -- 172.21.15.86:0/489303404 learned_addr learned my addr 172.21.15.86:0/489303404 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:05:49.656 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.655+0000 7f3455e00700 1 --2- 172.21.15.86:0/489303404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5555770268d0 0x555577021eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:05:49.656 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.655+0000 7f3454a00700 1 -- 172.21.15.86:0/489303404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55557701b100 msgr2=0x55557702d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:05:49.656 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.655+0000 7f3454a00700 1 --2- 172.21.15.86:0/489303404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55557701b100 0x55557702d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:49.656 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.655+0000 7f3454a00700 1 -- 172.21.15.86:0/489303404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5555770268d0 msgr2=0x555577021eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:05:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.655+0000 7f3454a00700 1 --2- 172.21.15.86:0/489303404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5555770268d0 0x555577021eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.655+0000 7f3454a00700 1 -- 172.21.15.86:0/489303404 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555576f33e40 con 0x55557701ba20 2024-03-23T18:05:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.655+0000 7f3455e00700 1 --2- 172.21.15.86:0/489303404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5555770268d0 0x555577021eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T18:05:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.656+0000 7f3454a00700 1 --2- 172.21.15.86:0/489303404 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55557701ba20 0x5555770235a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f345019a960 tx=0x7f343c005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eadaed91eade253e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:05:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.657+0000 7f3443000700 1 -- 172.21.15.86:0/489303404 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f343c01c040 con 0x55557701ba20 2024-03-23T18:05:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.658+0000 7f3443000700 1 -- 172.21.15.86:0/489303404 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f343c039070 con 0x55557701ba20 2024-03-23T18:05:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.658+0000 7f3443000700 1 -- 172.21.15.86:0/489303404 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f343c01c040 con 0x55557701ba20 2024-03-23T18:05:49.660 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.661+0000 7f346b931b80 1 -- 172.21.15.86:0/489303404 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55557701ba20 msgr2=0x5555770235a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:05:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.661+0000 7f346b931b80 1 --2- 172.21.15.86:0/489303404 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55557701ba20 0x5555770235a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f345019a960 tx=0x7f343c005dc0 comp rx=0 tx=0).stop 2024-03-23T18:05:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.661+0000 7f346b931b80 1 -- 172.21.15.86:0/489303404 shutdown_connections 2024-03-23T18:05:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.661+0000 7f346b931b80 1 --2- 172.21.15.86:0/489303404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5555770268d0 0x555577021eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:49.662 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.661+0000 7f346b931b80 1 --2- 172.21.15.86:0/489303404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55557701b100 0x55557702d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:49.662 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.661+0000 7f346b931b80 1 --2- 172.21.15.86:0/489303404 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55557701ba20 0x5555770235a0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:49.662 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.661+0000 7f346b931b80 1 -- 172.21.15.86:0/489303404 >> 172.21.15.86:0/489303404 conn(0x555576f7b640 msgr2=0x55557701e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:05:49.662 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.661+0000 7f346b931b80 1 -- 172.21.15.86:0/489303404 shutdown_connections 2024-03-23T18:05:49.662 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.662+0000 7f346b931b80 1 -- 172.21.15.86:0/489303404 wait complete. 2024-03-23T18:05:49.662 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.662+0000 7f346b931b80 1 -- start start 2024-03-23T18:05:49.662 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.662+0000 7f346b931b80 1 Processor -- start 2024-03-23T18:05:49.663 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.663+0000 7f346b931b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55557701b100 0x55557702b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:05:49.663 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.664+0000 7f346b931b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55557701ba20 0x55557702bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:05:49.664 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.664+0000 7f346b931b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555770268d0 0x55557702c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:05:49.664 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.664+0000 7f346b931b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555770872f0 con 0x55557701b100 2024-03-23T18:05:49.665 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.664+0000 7f346b931b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x555577087430 con 0x5555770268d0 2024-03-23T18:05:49.665 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.664+0000 7f346b931b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555770875c0 con 0x55557701ba20 2024-03-23T18:05:49.665 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.664+0000 7f346b931b80 10 client.0 _ll_register_callbacks cb 0x555576f13410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T18:05:49.665 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.664+0000 7f3455400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55557701b100 0x55557702b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:05:49.665 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.665+0000 7f3455400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55557701b100 0x55557702b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58916/0 (socket says 172.21.15.86:58916) 2024-03-23T18:05:49.665 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.665+0000 7f3455e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555770268d0 0x55557702c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:05:49.665 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.665+0000 7f3455400700 1 -- 172.21.15.86:0/923629739 learned_addr learned my addr 172.21.15.86:0/923629739 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:05:49.665 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.665+0000 7f3454a00700 1 --2- 172.21.15.86:0/923629739 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55557701ba20 0x55557702bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:05:49.666 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.666+0000 7f3455e00700 1 -- 172.21.15.86:0/923629739 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55557701ba20 msgr2=0x55557702bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:05:49.666 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.666+0000 7f3455e00700 1 --2- 172.21.15.86:0/923629739 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55557701ba20 0x55557702bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:49.666 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.666+0000 7f3455e00700 1 -- 172.21.15.86:0/923629739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55557701b100 msgr2=0x55557702b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:05:49.666 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.666+0000 7f3455e00700 1 --2- 172.21.15.86:0/923629739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55557701b100 0x55557702b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:49.666 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.666+0000 7f3455e00700 1 -- 172.21.15.86:0/923629739 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555576def350 con 0x5555770268d0 2024-03-23T18:05:49.667 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.667+0000 7f3455e00700 1 --2- 172.21.15.86:0/923629739 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555770268d0 0x55557702c070 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x55557701cb90 tx=0x7f34480a1780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34b37b2b13e954c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:05:49.667 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.667+0000 7f3436a00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T18:05:49.667 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.667+0000 7f3436a00700 1 -- 172.21.15.86:0/923629739 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344800ddd0 con 0x5555770268d0 2024-03-23T18:05:49.667 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.667+0000 7f3436a00700 1 -- 172.21.15.86:0/923629739 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8440+0+0 (secure 0 0 0) 0x7f3448062020 con 0x5555770268d0 2024-03-23T18:05:49.667 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.667+0000 7f346b931b80 1 -- 172.21.15.86:0/923629739 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f344809d010 con 0x5555770268d0 2024-03-23T18:05:49.668 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.668+0000 7f3436a00700 1 -- 172.21.15.86:0/923629739 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f34480986d0 con 0x5555770268d0 2024-03-23T18:05:49.668 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.668+0000 7f346b931b80 10 client.27242 fetch_fsmap learned FSMap version 70 2024-03-23T18:05:49.668 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.668+0000 7f346b931b80 1 -- 172.21.15.86:0/923629739 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x7f3450110680 con 0x5555770268d0 2024-03-23T18:05:49.668 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.668+0000 7f3436a00700 1 -- 172.21.15.86:0/923629739 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f344807f020 con 0x5555770268d0 2024-03-23T18:05:49.669 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.669+0000 7f3436a00700 1 -- 172.21.15.86:0/923629739 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 70) v1 ==== 7530+0+0 (secure 0 0 0) 0x7f34480a12e0 con 0x5555770268d0 2024-03-23T18:05:49.669 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.669+0000 7f346b931b80 10 client.27242 fetch_fsmap finished waiting for FSMap version 70 2024-03-23T18:05:49.669 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.669+0000 7f346b931b80 10 client.27242 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T18:05:49.669 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.669+0000 7f346b931b80 20 client.27242 populate_metadata read hostname 'smithi086' 2024-03-23T18:05:49.669 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.669+0000 7f346b931b80 1 --2- 172.21.15.86:0/923629739 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x555577088a40 0x55557708ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:05:49.669 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.669+0000 7f346b931b80 4 client.27242 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T18:05:49.669 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.669+0000 7f346b931b80 1 -- 172.21.15.86:0/923629739 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555576dbcc90 con 0x555577088a40 2024-03-23T18:05:49.669 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.669+0000 7f3455400700 1 --2- 172.21.15.86:0/923629739 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x555577088a40 0x55557708ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:05:49.670 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.669+0000 7f3455400700 1 --2- 172.21.15.86:0/923629739 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x555577088a40 0x55557708ae60 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90dcd709e1e2a707 server_cookie=de072e57dacde1ac in_seq=0 out_seq=0 2024-03-23T18:05:49.670 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.669+0000 7f3436a00700 10 client.27242 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T18:05:49.930 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.928+0000 7f3436a00700 1 -- 172.21.15.86:0/923629739 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555576dbcc90 con 0x555577088a40 2024-03-23T18:05:49.930 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.928+0000 7f3436a00700 10 client.27242 handle_command_reply: tid=0 2024-03-23T18:05:49.930 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.928+0000 7f346b931b80 1 client.27242 shutdown 2024-03-23T18:05:49.931 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.928+0000 7f346b931b80 1 -- 172.21.15.86:0/923629739 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555770268d0 msgr2=0x55557702c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:05:49.931 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.928+0000 7f346b931b80 1 --2- 172.21.15.86:0/923629739 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555770268d0 0x55557702c070 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x55557701cb90 tx=0x7f34480a1780 comp rx=0 tx=0).stop 2024-03-23T18:05:49.931 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.929+0000 7f346b931b80 1 -- 172.21.15.86:0/923629739 shutdown_connections 2024-03-23T18:05:49.931 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.929+0000 7f346b931b80 1 --2- 172.21.15.86:0/923629739 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x555577088a40 0x55557708ae60 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:49.931 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.929+0000 7f346b931b80 1 --2- 172.21.15.86:0/923629739 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555770268d0 0x55557702c070 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:49.932 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.929+0000 7f346b931b80 1 --2- 172.21.15.86:0/923629739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55557701b100 0x55557702b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:49.932 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.929+0000 7f3436a00700 0 client.27242 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T18:05:49.932 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.929+0000 7f346b931b80 1 --2- 172.21.15.86:0/923629739 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55557701ba20 0x55557702bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:49.932 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.929+0000 7f346b931b80 1 -- 172.21.15.86:0/923629739 >> 172.21.15.86:0/923629739 conn(0x555576f7b640 msgr2=0x555577021d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:05:49.932 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.930+0000 7f346b931b80 1 -- 172.21.15.86:0/923629739 shutdown_connections 2024-03-23T18:05:49.932 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.930+0000 7f346b931b80 1 -- 172.21.15.86:0/923629739 wait complete. 2024-03-23T18:05:49.933 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.930+0000 7f346b931b80 20 client.27242 trim_cache size 0 max 16384 2024-03-23T18:05:49.933 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-03-23T18:05:49.933 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-03-23T18:05:49.934 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-03-23T18:05:49.935 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-03-23T18:05:49.936 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.930+0000 7f3457e00700 1 -- 172.21.15.86:0/693288259 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f34380a1350 msgr2=0x7f34380a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:05:49.936 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.930+0000 7f3457e00700 1 --2- 172.21.15.86:0/693288259 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f34380a1350 0x7f34380a3810 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3444005950 tx=0x7f3444009310 comp rx=0 tx=0).stop 2024-03-23T18:05:49.936 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.930+0000 7f3457e00700 1 -- 172.21.15.86:0/693288259 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f345010d540 msgr2=0x7f345019dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:05:49.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.931+0000 7f3457e00700 1 --2- 172.21.15.86:0/693288259 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f345010d540 0x7f345019dcf0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3448007ee0 tx=0x7f344800d3b0 comp rx=0 tx=0).stop 2024-03-23T18:05:49.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.931+0000 7f3457e00700 1 -- 172.21.15.86:0/693288259 shutdown_connections 2024-03-23T18:05:49.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.931+0000 7f3457e00700 1 --2- 172.21.15.86:0/693288259 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f34380a1350 0x7f34380a3810 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:49.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.931+0000 7f3457e00700 1 --2- 172.21.15.86:0/693288259 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f345010d540 0x7f345019dcf0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:49.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.931+0000 7f3457e00700 1 --2- 172.21.15.86:0/693288259 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3450073180 0x7f3450199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:49.938 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.931+0000 7f3457e00700 1 --2- 172.21.15.86:0/693288259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3450073aa0 0x7f3450199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:05:49.938 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.931+0000 7f3457e00700 1 -- 172.21.15.86:0/693288259 >> 172.21.15.86:0/693288259 conn(0x7f34500fc770 msgr2=0x7f34500fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:05:49.938 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.931+0000 7f3457e00700 1 -- 172.21.15.86:0/693288259 shutdown_connections 2024-03-23T18:05:49.938 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:05:49.931+0000 7f3457e00700 1 -- 172.21.15.86:0/693288259 wait complete. 2024-03-23T18:05:50.236 INFO:tasks.workunit.client.0.smithi086.stdout: 10 98689 40.61 MB/sec execute 14 sec latency 149.297 ms 2024-03-23T18:05:50.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:05:49 smithi086 ceph-mon[38221]: pgmap v477: 97 pgs: 97 active+clean; 1008 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 17 MiB/s wr, 1.04k op/s 2024-03-23T18:05:50.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:05:49 smithi092 ceph-mon[79089]: pgmap v477: 97 pgs: 97 active+clean; 1008 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 17 MiB/s wr, 1.04k op/s 2024-03-23T18:05:50.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:05:49 smithi152 ceph-mon[47949]: pgmap v477: 97 pgs: 97 active+clean; 1008 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 17 MiB/s wr, 1.04k op/s 2024-03-23T18:05:51.236 INFO:tasks.workunit.client.0.smithi086.stdout: 10 99489 40.38 MB/sec execute 15 sec latency 54.757 ms 2024-03-23T18:05:52.236 INFO:tasks.workunit.client.0.smithi086.stdout: 10 100398 40.36 MB/sec execute 16 sec latency 66.649 ms 2024-03-23T18:05:52.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:05:51 smithi086 ceph-mon[38221]: pgmap v478: 97 pgs: 97 active+clean; 1.0 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 20 MiB/s wr, 1.32k op/s 2024-03-23T18:05:52.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:05:51 smithi086 ceph-mon[38221]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:05:52.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:05:51 smithi092 ceph-mon[79089]: pgmap v478: 97 pgs: 97 active+clean; 1.0 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 20 MiB/s wr, 1.32k op/s 2024-03-23T18:05:52.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:05:51 smithi092 ceph-mon[79089]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:05:52.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:05:51 smithi152 ceph-mon[47949]: pgmap v478: 97 pgs: 97 active+clean; 1.0 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 20 MiB/s wr, 1.32k op/s 2024-03-23T18:05:52.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:05:51 smithi152 ceph-mon[47949]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:05:53.236 INFO:tasks.workunit.client.0.smithi086.stdout: 10 101225 41.20 MB/sec execute 17 sec latency 183.945 ms 2024-03-23T18:05:54.236 INFO:tasks.workunit.client.0.smithi086.stdout: 10 102122 41.77 MB/sec execute 18 sec latency 33.588 ms 2024-03-23T18:05:54.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:05:53 smithi086 ceph-mon[38221]: pgmap v479: 97 pgs: 97 active+clean; 1.0 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.5 MiB/s wr, 748 op/s 2024-03-23T18:05:54.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:05:53 smithi092 ceph-mon[79089]: pgmap v479: 97 pgs: 97 active+clean; 1.0 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.5 MiB/s wr, 748 op/s 2024-03-23T18:05:54.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:05:53 smithi152 ceph-mon[47949]: pgmap v479: 97 pgs: 97 active+clean; 1.0 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.5 MiB/s wr, 748 op/s 2024-03-23T18:05:55.236 INFO:tasks.workunit.client.0.smithi086.stdout: 10 102892 41.61 MB/sec execute 19 sec latency 57.317 ms 2024-03-23T18:05:56.237 INFO:tasks.workunit.client.0.smithi086.stdout: 10 103871 41.98 MB/sec execute 20 sec latency 45.952 ms 2024-03-23T18:05:56.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:05:55 smithi086 ceph-mon[38221]: pgmap v480: 97 pgs: 97 active+clean; 1.0 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 10 MiB/s wr, 944 op/s 2024-03-23T18:05:56.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:05:55 smithi092 ceph-mon[79089]: pgmap v480: 97 pgs: 97 active+clean; 1.0 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 10 MiB/s wr, 944 op/s 2024-03-23T18:05:56.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:05:55 smithi152 ceph-mon[47949]: pgmap v480: 97 pgs: 97 active+clean; 1.0 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 10 MiB/s wr, 944 op/s 2024-03-23T18:05:57.237 INFO:tasks.workunit.client.0.smithi086.stdout: 10 104815 42.90 MB/sec execute 21 sec latency 55.467 ms 2024-03-23T18:05:57.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:05:56 smithi086 ceph-mon[38221]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:05:57.309 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:05:56 smithi092 ceph-mon[79089]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:05:57.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:05:56 smithi152 ceph-mon[47949]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:05:58.237 INFO:tasks.workunit.client.0.smithi086.stdout: 10 105667 42.30 MB/sec execute 22 sec latency 68.849 ms 2024-03-23T18:05:58.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:05:57 smithi086 ceph-mon[38221]: pgmap v481: 97 pgs: 97 active+clean; 1.0 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 9.7 MiB/s wr, 882 op/s 2024-03-23T18:05:58.309 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:05:57 smithi092 ceph-mon[79089]: pgmap v481: 97 pgs: 97 active+clean; 1.0 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 9.7 MiB/s wr, 882 op/s 2024-03-23T18:05:58.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:05:57 smithi152 ceph-mon[47949]: pgmap v481: 97 pgs: 97 active+clean; 1.0 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 9.7 MiB/s wr, 882 op/s 2024-03-23T18:05:59.237 INFO:tasks.workunit.client.0.smithi086.stdout: 10 106840 43.99 MB/sec execute 23 sec latency 38.071 ms 2024-03-23T18:06:00.237 INFO:tasks.workunit.client.0.smithi086.stdout: 10 107649 43.63 MB/sec execute 24 sec latency 50.405 ms 2024-03-23T18:06:00.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:06:00 smithi086 ceph-mon[38221]: pgmap v482: 97 pgs: 97 active+clean; 1.0 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.4 MiB/s wr, 631 op/s 2024-03-23T18:06:00.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:06:00 smithi092 ceph-mon[79089]: pgmap v482: 97 pgs: 97 active+clean; 1.0 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.4 MiB/s wr, 631 op/s 2024-03-23T18:06:00.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:06:00 smithi152 ceph-mon[47949]: pgmap v482: 97 pgs: 97 active+clean; 1.0 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.4 MiB/s wr, 631 op/s 2024-03-23T18:06:01.237 INFO:tasks.workunit.client.0.smithi086.stdout: 10 107843 42.43 MB/sec execute 25 sec latency 764.352 ms 2024-03-23T18:06:02.237 INFO:tasks.workunit.client.0.smithi086.stdout: 10 107843 40.80 MB/sec execute 26 sec latency 1764.436 ms 2024-03-23T18:06:02.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:06:02 smithi086 ceph-mon[38221]: pgmap v483: 97 pgs: 97 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 9.7 MiB/s wr, 925 op/s 2024-03-23T18:06:02.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:06:02 smithi092 ceph-mon[79089]: pgmap v483: 97 pgs: 97 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 9.7 MiB/s wr, 925 op/s 2024-03-23T18:06:02.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:06:02 smithi152 ceph-mon[47949]: pgmap v483: 97 pgs: 97 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 9.7 MiB/s wr, 925 op/s 2024-03-23T18:06:03.152 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.152+0000 7fcc09600700 1 -- 172.21.15.86:0/839125717 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcc04103120 msgr2=0x7fcc04103500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:06:03.152 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.152+0000 7fcc09600700 1 --2- 172.21.15.86:0/839125717 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcc04103120 0x7fcc04103500 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fcbf4009990 tx=0x7fcbf4009ca0 comp rx=0 tx=0).stop 2024-03-23T18:06:03.153 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.152+0000 7fcc09600700 1 -- 172.21.15.86:0/839125717 shutdown_connections 2024-03-23T18:06:03.153 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.152+0000 7fcc09600700 1 --2- 172.21.15.86:0/839125717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc04108240 0x7fcc0410fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:03.153 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.152+0000 7fcc09600700 1 --2- 172.21.15.86:0/839125717 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcc04103120 0x7fcc04103500 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:03.153 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.152+0000 7fcc09600700 1 --2- 172.21.15.86:0/839125717 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc04103ad0 0x7fcc04107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:03.153 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.152+0000 7fcc09600700 1 -- 172.21.15.86:0/839125717 >> 172.21.15.86:0/839125717 conn(0x7fcc040fe960 msgr2=0x7fcc04100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:06:03.153 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.152+0000 7fcc09600700 1 -- 172.21.15.86:0/839125717 shutdown_connections 2024-03-23T18:06:03.153 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.153+0000 7fcc09600700 1 -- 172.21.15.86:0/839125717 wait complete. 2024-03-23T18:06:03.154 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.154+0000 7fcc09600700 1 Processor -- start 2024-03-23T18:06:03.157 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.155+0000 7fcc09600700 1 -- start start 2024-03-23T18:06:03.157 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.155+0000 7fcc09600700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcc04103120 0x7fcc0410faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:03.157 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.155+0000 7fcc09600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc04108240 0x7fcc0410cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:03.157 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.155+0000 7fcc09600700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc0410d100 0x7fcc0410d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:03.157 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.156+0000 7fcc09600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc041a9d50 con 0x7fcc04108240 2024-03-23T18:06:03.157 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.156+0000 7fcc02a00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcc04103120 0x7fcc0410faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:06:03.157 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.156+0000 7fcc09600700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc041a9ec0 con 0x7fcc0410d100 2024-03-23T18:06:03.157 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.156+0000 7fcc09600700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc041aa080 con 0x7fcc04103120 2024-03-23T18:06:03.157 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.156+0000 7fcc02a00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcc04103120 0x7fcc0410faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:59910/0 (socket says 172.21.15.86:59910) 2024-03-23T18:06:03.157 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.156+0000 7fcc02a00700 1 -- 172.21.15.86:0/477367023 learned_addr learned my addr 172.21.15.86:0/477367023 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:06:03.158 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.157+0000 7fcc02000700 1 --2- 172.21.15.86:0/477367023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc04108240 0x7fcc0410cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:06:03.158 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.157+0000 7fcc03400700 1 --2- 172.21.15.86:0/477367023 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc0410d100 0x7fcc0410d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:06:03.159 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.159+0000 7fcc02a00700 1 -- 172.21.15.86:0/477367023 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc0410d100 msgr2=0x7fcc0410d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:06:03.159 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.159+0000 7fcc02a00700 1 --2- 172.21.15.86:0/477367023 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc0410d100 0x7fcc0410d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:03.159 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.159+0000 7fcc02a00700 1 -- 172.21.15.86:0/477367023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc04108240 msgr2=0x7fcc0410cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:06:03.160 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.159+0000 7fcc02a00700 1 --2- 172.21.15.86:0/477367023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc04108240 0x7fcc0410cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:03.160 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.159+0000 7fcc02a00700 1 -- 172.21.15.86:0/477367023 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbf0009520 con 0x7fcc04103120 2024-03-23T18:06:03.160 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.159+0000 7fcc02000700 1 --2- 172.21.15.86:0/477367023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc04108240 0x7fcc0410cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T18:06:03.160 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.160+0000 7fcc02a00700 1 --2- 172.21.15.86:0/477367023 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcc04103120 0x7fcc0410faf0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fcbf4005910 tx=0x7fcbf4004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:06:03.161 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.160+0000 7fcbfb400700 1 -- 172.21.15.86:0/477367023 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf401d070 con 0x7fcc04103120 2024-03-23T18:06:03.161 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.160+0000 7fcbfb400700 1 -- 172.21.15.86:0/477367023 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcbf4022470 con 0x7fcc04103120 2024-03-23T18:06:03.161 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.160+0000 7fcbfb400700 1 -- 172.21.15.86:0/477367023 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf4017650 con 0x7fcc04103120 2024-03-23T18:06:03.161 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.161+0000 7fcc09600700 1 -- 172.21.15.86:0/477367023 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbf40095f0 con 0x7fcc04103120 2024-03-23T18:06:03.161 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.161+0000 7fcc09600700 1 -- 172.21.15.86:0/477367023 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc041aa540 con 0x7fcc04103120 2024-03-23T18:06:03.165 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.166+0000 7fcbfb400700 1 -- 172.21.15.86:0/477367023 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fcbf40225e0 con 0x7fcc04103120 2024-03-23T18:06:03.167 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.168+0000 7fcbfb400700 1 --2- 172.21.15.86:0/477367023 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fcbe80a1460 0x7fcbe80a3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:03.168 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.168+0000 7fcbfb400700 1 -- 172.21.15.86:0/477367023 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fcbf40cddf0 con 0x7fcc04103120 2024-03-23T18:06:03.168 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.168+0000 7fcc02000700 1 --2- 172.21.15.86:0/477367023 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fcbe80a1460 0x7fcbe80a3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:06:03.169 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.169+0000 7fcc02000700 1 --2- 172.21.15.86:0/477367023 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fcbe80a1460 0x7fcbe80a3920 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fcbf0009fd0 tx=0x7fcbf0009260 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:06:03.170 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.171+0000 7fcc1d0cbb80 1 Processor -- start 2024-03-23T18:06:03.171 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.172+0000 7fcc1d0cbb80 1 -- start start 2024-03-23T18:06:03.172 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.172+0000 7fcc1d0cbb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56483ec54100 0x56483ec544e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:03.173 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.173+0000 7fcc02a00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56483ec54100 0x56483ec544e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:06:03.173 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.173+0000 7fcc1d0cbb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56483ec54a20 0x56483ec5f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:03.173 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.173+0000 7fcc02a00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56483ec54100 0x56483ec544e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:59912/0 (socket says 172.21.15.86:59912) 2024-03-23T18:06:03.173 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.173+0000 7fcc1d0cbb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56483ec5f8d0 0x56483ec61cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:03.173 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.173+0000 7fcc02a00700 1 -- 172.21.15.86:0/3364465388 learned_addr learned my addr 172.21.15.86:0/3364465388 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:06:03.173 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.173+0000 7fcc1d0cbb80 1 -- 172.21.15.86:0/3364465388 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56483ebad930 con 0x56483ec5f8d0 2024-03-23T18:06:03.174 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.173+0000 7fcc1d0cbb80 1 -- 172.21.15.86:0/3364465388 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x56483ebab2f0 con 0x56483ec54a20 2024-03-23T18:06:03.174 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.173+0000 7fcc1d0cbb80 1 -- 172.21.15.86:0/3364465388 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x56483eba6be0 con 0x56483ec54100 2024-03-23T18:06:03.174 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.173+0000 7fcc02000700 1 --2- 172.21.15.86:0/3364465388 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56483ec54a20 0x56483ec5f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:06:03.174 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.174+0000 7fcc02a00700 1 -- 172.21.15.86:0/3364465388 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56483ec54a20 msgr2=0x56483ec5f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:06:03.174 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.174+0000 7fcc02a00700 1 --2- 172.21.15.86:0/3364465388 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56483ec54a20 0x56483ec5f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:03.174 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.174+0000 7fcc02a00700 1 -- 172.21.15.86:0/3364465388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56483ec5f8d0 msgr2=0x56483ec61cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:06:03.175 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.174+0000 7fcc02a00700 1 --2- 172.21.15.86:0/3364465388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56483ec5f8d0 0x56483ec61cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:03.175 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.174+0000 7fcc02a00700 1 -- 172.21.15.86:0/3364465388 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56483ea28350 con 0x56483ec54100 2024-03-23T18:06:03.175 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.174+0000 7fcc02a00700 1 --2- 172.21.15.86:0/3364465388 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56483ec54100 0x56483ec544e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fcbf40a2e40 tx=0x7fcbf4009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f98243aec84defa8 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:06:03.175 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.174+0000 7fcbef400700 1 -- 172.21.15.86:0/3364465388 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf409a8e0 con 0x56483ec54100 2024-03-23T18:06:03.176 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.174+0000 7fcbef400700 1 -- 172.21.15.86:0/3364465388 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcbf409b680 con 0x56483ec54100 2024-03-23T18:06:03.176 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.174+0000 7fcbef400700 1 -- 172.21.15.86:0/3364465388 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf409b440 con 0x56483ec54100 2024-03-23T18:06:03.176 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.175+0000 7fcc1d0cbb80 1 -- 172.21.15.86:0/3364465388 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56483ec54100 msgr2=0x56483ec544e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:06:03.176 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.176+0000 7fcc1d0cbb80 1 --2- 172.21.15.86:0/3364465388 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56483ec54100 0x56483ec544e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fcbf40a2e40 tx=0x7fcbf4009ca0 comp rx=0 tx=0).stop 2024-03-23T18:06:03.176 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.176+0000 7fcc1d0cbb80 1 -- 172.21.15.86:0/3364465388 shutdown_connections 2024-03-23T18:06:03.176 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.176+0000 7fcc1d0cbb80 1 --2- 172.21.15.86:0/3364465388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56483ec5f8d0 0x56483ec61cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:03.176 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.176+0000 7fcc1d0cbb80 1 --2- 172.21.15.86:0/3364465388 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56483ec54100 0x56483ec544e0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:03.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.176+0000 7fcc1d0cbb80 1 --2- 172.21.15.86:0/3364465388 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56483ec54a20 0x56483ec5f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:03.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.176+0000 7fcc1d0cbb80 1 -- 172.21.15.86:0/3364465388 >> 172.21.15.86:0/3364465388 conn(0x56483ebb4640 msgr2=0x56483ec50520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:06:03.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.177+0000 7fcc1d0cbb80 1 -- 172.21.15.86:0/3364465388 shutdown_connections 2024-03-23T18:06:03.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.177+0000 7fcc1d0cbb80 1 -- 172.21.15.86:0/3364465388 wait complete. 2024-03-23T18:06:03.178 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.178+0000 7fcc1d0cbb80 1 -- start start 2024-03-23T18:06:03.178 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.178+0000 7fcc1d0cbb80 1 Processor -- start 2024-03-23T18:06:03.178 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.179+0000 7fcc1d0cbb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56483ec54100 0x56483ec5f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:03.179 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.179+0000 7fcc1d0cbb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56483ec54a20 0x56483ec5c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:03.179 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.179+0000 7fcc1d0cbb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56483ec5f8d0 0x56483ec5cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:03.179 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.179+0000 7fcc1d0cbb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56483eb42d20 con 0x56483ec54a20 2024-03-23T18:06:03.179 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.179+0000 7fcc1d0cbb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x56483eb429c0 con 0x56483ec54100 2024-03-23T18:06:03.179 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.179+0000 7fcc1d0cbb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x56483ea2a640 con 0x56483ec5f8d0 2024-03-23T18:06:03.179 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.179+0000 7fcc1d0cbb80 10 client.0 _ll_register_callbacks cb 0x56483eb4c410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T18:06:03.180 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.179+0000 7fcc03400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56483ec5f8d0 0x56483ec5cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:06:03.180 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.179+0000 7fcc02000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56483ec54a20 0x56483ec5c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:06:03.180 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.179+0000 7fcc03400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56483ec5f8d0 0x56483ec5cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:59928/0 (socket says 172.21.15.86:59928) 2024-03-23T18:06:03.180 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.179+0000 7fcc03400700 1 -- 172.21.15.86:0/1449295299 learned_addr learned my addr 172.21.15.86:0/1449295299 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:06:03.180 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.180+0000 7fcc02a00700 1 --2- 172.21.15.86:0/1449295299 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56483ec54100 0x56483ec5f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:06:03.180 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.180+0000 7fcc02000700 1 -- 172.21.15.86:0/1449295299 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56483ec5f8d0 msgr2=0x56483ec5cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:06:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.180+0000 7fcc02000700 1 --2- 172.21.15.86:0/1449295299 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56483ec5f8d0 0x56483ec5cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.180+0000 7fcc02000700 1 -- 172.21.15.86:0/1449295299 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56483ec54100 msgr2=0x56483ec5f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:06:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.180+0000 7fcc02000700 1 --2- 172.21.15.86:0/1449295299 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56483ec54100 0x56483ec5f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.180+0000 7fcc02000700 1 -- 172.21.15.86:0/1449295299 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc041aa540 con 0x56483ec54a20 2024-03-23T18:06:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.181+0000 7fcc02000700 1 --2- 172.21.15.86:0/1449295299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56483ec54a20 0x56483ec5c660 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fcc0410e2f0 tx=0x7fcbf00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f94f27d01244c3fd server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:06:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.181+0000 7fcbe3e00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-23T18:06:03.182 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.182+0000 7fcbe3e00700 1 -- 172.21.15.86:0/1449295299 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf001e220 con 0x56483ec54a20 2024-03-23T18:06:03.182 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.182+0000 7fcbe3e00700 1 -- 172.21.15.86:0/1449295299 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fcbf000eae0 con 0x56483ec54a20 2024-03-23T18:06:03.182 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.182+0000 7fcc1d0cbb80 1 -- 172.21.15.86:0/1449295299 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56483eb67320 con 0x56483ec54a20 2024-03-23T18:06:03.182 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.182+0000 7fcbe3e00700 1 -- 172.21.15.86:0/1449295299 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcbf000eca0 con 0x56483ec54a20 2024-03-23T18:06:03.182 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.182+0000 7fcc1d0cbb80 10 client.17148 fetch_fsmap learned FSMap version 72 2024-03-23T18:06:03.182 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.182+0000 7fcc1d0cbb80 1 -- 172.21.15.86:0/1449295299 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x56483e9d4b00 con 0x56483ec54a20 2024-03-23T18:06:03.183 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.183+0000 7fcbe3e00700 1 -- 172.21.15.86:0/1449295299 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56483eb67320 con 0x56483ec54a20 2024-03-23T18:06:03.183 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.183+0000 7fcbe3e00700 1 -- 172.21.15.86:0/1449295299 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 72) v1 ==== 7522+0+0 (secure 0 0 0) 0x7fcbf001d3f0 con 0x56483ec54a20 2024-03-23T18:06:03.184 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.183+0000 7fcc1d0cbb80 10 client.17148 fetch_fsmap finished waiting for FSMap version 72 2024-03-23T18:06:03.184 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.183+0000 7fcc1d0cbb80 10 client.17148 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T18:06:03.184 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.183+0000 7fcc1d0cbb80 20 client.17148 populate_metadata read hostname 'smithi086' 2024-03-23T18:06:03.184 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.183+0000 7fcc1d0cbb80 1 --2- 172.21.15.86:0/1449295299 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x56483ec6f3d0 0x56483ec717f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:03.184 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.183+0000 7fcc1d0cbb80 4 client.17148 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T18:06:03.184 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.183+0000 7fcc1d0cbb80 1 -- 172.21.15.86:0/1449295299 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56483eb9f6a0 con 0x56483ec6f3d0 2024-03-23T18:06:03.184 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.185+0000 7fcc02a00700 1 --2- 172.21.15.86:0/1449295299 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x56483ec6f3d0 0x56483ec717f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:06:03.185 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.185+0000 7fcbe3e00700 10 client.17148 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T18:06:03.185 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.185+0000 7fcc02a00700 1 --2- 172.21.15.86:0/1449295299 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x56483ec6f3d0 0x56483ec717f0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d777e5ed23546ca9 server_cookie=737bc5aeb7f1704a in_seq=0 out_seq=0 2024-03-23T18:06:03.192 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.192+0000 7fcbe3e00700 1 -- 172.21.15.86:0/1449295299 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x56483eb9f6a0 con 0x56483ec6f3d0 2024-03-23T18:06:03.192 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.192+0000 7fcbe3e00700 10 client.17148 handle_command_reply: tid=0 2024-03-23T18:06:03.192 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.192+0000 7fcc1d0cbb80 1 client.17148 shutdown 2024-03-23T18:06:03.192 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.192+0000 7fcc1d0cbb80 1 -- 172.21.15.86:0/1449295299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56483ec54a20 msgr2=0x56483ec5c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:06:03.192 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.192+0000 7fcc1d0cbb80 1 --2- 172.21.15.86:0/1449295299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56483ec54a20 0x56483ec5c660 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fcc0410e2f0 tx=0x7fcbf00047c0 comp rx=0 tx=0).stop 2024-03-23T18:06:03.193 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.193+0000 7fcc1d0cbb80 1 -- 172.21.15.86:0/1449295299 shutdown_connections 2024-03-23T18:06:03.193 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.193+0000 7fcc1d0cbb80 1 --2- 172.21.15.86:0/1449295299 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x56483ec6f3d0 0x56483ec717f0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:03.193 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.193+0000 7fcc1d0cbb80 1 --2- 172.21.15.86:0/1449295299 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56483ec5f8d0 0x56483ec5cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:03.193 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.193+0000 7fcc1d0cbb80 1 --2- 172.21.15.86:0/1449295299 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56483ec54100 0x56483ec5f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:03.193 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.193+0000 7fcc1d0cbb80 1 --2- 172.21.15.86:0/1449295299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56483ec54a20 0x56483ec5c660 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:03.193 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.193+0000 7fcc1d0cbb80 1 -- 172.21.15.86:0/1449295299 >> 172.21.15.86:0/1449295299 conn(0x56483ebb4640 msgr2=0x56483ec560a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:06:03.193 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.193+0000 7fcbe3e00700 0 client.17148 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T18:06:03.193 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.193+0000 7fcc1d0cbb80 1 -- 172.21.15.86:0/1449295299 shutdown_connections 2024-03-23T18:06:03.193 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.193+0000 7fcc1d0cbb80 1 -- 172.21.15.86:0/1449295299 wait complete. 2024-03-23T18:06:03.194 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.194+0000 7fcc1d0cbb80 20 client.17148 trim_cache size 0 max 16384 2024-03-23T18:06:03.220 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.220+0000 7fcc1d0cbb80 1 Processor -- start 2024-03-23T18:06:03.220 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.220+0000 7fcc1d0cbb80 1 -- start start 2024-03-23T18:06:03.220 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.220+0000 7fcc1d0cbb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56483ec54100 0x56483ec5cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:03.220 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.220+0000 7fcc1d0cbb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56483ec54a20 0x56483ec662c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:03.220 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.220+0000 7fcc1d0cbb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56483ec5f8d0 0x56483ec5eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:03.220 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.220+0000 7fcc1d0cbb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56483ec5dc90 con 0x56483ec54100 2024-03-23T18:06:03.221 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.220+0000 7fcc1d0cbb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x56483ec51300 con 0x56483ec54a20 2024-03-23T18:06:03.221 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.220+0000 7fcc1d0cbb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x56483ec55cf0 con 0x56483ec5f8d0 2024-03-23T18:06:03.222 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.221+0000 7fcc02000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56483ec54a20 0x56483ec662c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:06:03.222 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.221+0000 7fcc02a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56483ec54100 0x56483ec5cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:06:03.222 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.221+0000 7fcc02000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56483ec54a20 0x56483ec662c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:40726/0 (socket says 172.21.15.86:40726) 2024-03-23T18:06:03.222 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.221+0000 7fcc02000700 1 -- 172.21.15.86:0/1146268430 learned_addr learned my addr 172.21.15.86:0/1146268430 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:06:03.223 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.221+0000 7fcc02a00700 1 -- 172.21.15.86:0/1146268430 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56483ec5f8d0 msgr2=0x56483ec5eea0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-23T18:06:03.223 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.221+0000 7fcc02a00700 1 --2- 172.21.15.86:0/1146268430 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56483ec5f8d0 0x56483ec5eea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:03.223 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.222+0000 7fcc02a00700 1 -- 172.21.15.86:0/1146268430 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56483ec54a20 msgr2=0x56483ec662c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:06:03.223 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.222+0000 7fcc02a00700 1 --2- 172.21.15.86:0/1146268430 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56483ec54a20 0x56483ec662c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:03.223 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.222+0000 7fcc02a00700 1 -- 172.21.15.86:0/1146268430 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56483eb6ce40 con 0x56483ec54100 2024-03-23T18:06:03.223 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.222+0000 7fcc02a00700 1 --2- 172.21.15.86:0/1146268430 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56483ec54100 0x56483ec5cca0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x56483ec54fd0 tx=0x7fcbf4022f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b3e48ecff531b93 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:06:03.223 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.222+0000 7fcbefe00700 1 -- 172.21.15.86:0/1146268430 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf409a8e0 con 0x56483ec54100 2024-03-23T18:06:03.223 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.222+0000 7fcbefe00700 1 -- 172.21.15.86:0/1146268430 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcbf409af80 con 0x56483ec54100 2024-03-23T18:06:03.223 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.222+0000 7fcbefe00700 1 -- 172.21.15.86:0/1146268430 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf409a470 con 0x56483ec54100 2024-03-23T18:06:03.224 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.223+0000 7fcc1d0cbb80 1 -- 172.21.15.86:0/1146268430 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56483ec54100 msgr2=0x56483ec5cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:06:03.224 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.223+0000 7fcc1d0cbb80 1 --2- 172.21.15.86:0/1146268430 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56483ec54100 0x56483ec5cca0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x56483ec54fd0 tx=0x7fcbf4022f90 comp rx=0 tx=0).stop 2024-03-23T18:06:03.224 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.223+0000 7fcc1d0cbb80 1 -- 172.21.15.86:0/1146268430 shutdown_connections 2024-03-23T18:06:03.224 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.223+0000 7fcc1d0cbb80 1 --2- 172.21.15.86:0/1146268430 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56483ec5f8d0 0x56483ec5eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:03.224 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.223+0000 7fcc1d0cbb80 1 --2- 172.21.15.86:0/1146268430 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56483ec54100 0x56483ec5cca0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:03.224 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.223+0000 7fcc1d0cbb80 1 --2- 172.21.15.86:0/1146268430 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56483ec54a20 0x56483ec662c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:03.224 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.223+0000 7fcc1d0cbb80 1 -- 172.21.15.86:0/1146268430 >> 172.21.15.86:0/1146268430 conn(0x56483ebb4640 msgr2=0x56483ec5af50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:06:03.224 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.224+0000 7fcc1d0cbb80 1 -- 172.21.15.86:0/1146268430 shutdown_connections 2024-03-23T18:06:03.224 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.224+0000 7fcc1d0cbb80 1 -- 172.21.15.86:0/1146268430 wait complete. 2024-03-23T18:06:03.225 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.224+0000 7fcc1d0cbb80 1 -- start start 2024-03-23T18:06:03.225 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.225+0000 7fcc1d0cbb80 1 Processor -- start 2024-03-23T18:06:03.226 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.227+0000 7fcc1d0cbb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56483ec54100 0x56483ec5a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:03.227 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.227+0000 7fcc1d0cbb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56483ec54a20 0x56483ec642a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:03.228 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.227+0000 7fcc1d0cbb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56483ec5f8d0 0x56483ec647e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:03.228 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.227+0000 7fcc02a00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56483ec54100 0x56483ec5a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:06:03.228 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.227+0000 7fcc02a00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56483ec54100 0x56483ec5a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:59956/0 (socket says 172.21.15.86:59956) 2024-03-23T18:06:03.228 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.228+0000 7fcc1d0cbb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56483ec64d20 con 0x56483ec54a20 2024-03-23T18:06:03.228 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.228+0000 7fcc1d0cbb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x56483ec64e90 con 0x56483ec5f8d0 2024-03-23T18:06:03.228 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.228+0000 7fcc1d0cbb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x56483ec65000 con 0x56483ec54100 2024-03-23T18:06:03.228 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.228+0000 7fcc1d0cbb80 10 client.0 _ll_register_callbacks cb 0x56483eb4c410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T18:06:03.229 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.228+0000 7fcc02000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56483ec54a20 0x56483ec642a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:06:03.229 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.228+0000 7fcc02000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56483ec54a20 0x56483ec642a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35364/0 (socket says 172.21.15.86:35364) 2024-03-23T18:06:03.229 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.228+0000 7fcc02000700 1 -- 172.21.15.86:0/4271300612 learned_addr learned my addr 172.21.15.86:0/4271300612 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:06:03.229 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.228+0000 7fcc03400700 1 --2- 172.21.15.86:0/4271300612 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56483ec5f8d0 0x56483ec647e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:06:03.230 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.229+0000 7fcc02000700 1 -- 172.21.15.86:0/4271300612 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56483ec54100 msgr2=0x56483ec5a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:06:03.230 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.229+0000 7fcc02000700 1 --2- 172.21.15.86:0/4271300612 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56483ec54100 0x56483ec5a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:03.230 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.229+0000 7fcc02000700 1 -- 172.21.15.86:0/4271300612 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56483ec5f8d0 msgr2=0x56483ec647e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:06:03.230 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.229+0000 7fcc02000700 1 --2- 172.21.15.86:0/4271300612 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56483ec5f8d0 0x56483ec647e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:03.230 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.229+0000 7fcc02000700 1 -- 172.21.15.86:0/4271300612 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56483ea28350 con 0x56483ec54a20 2024-03-23T18:06:03.230 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.229+0000 7fcc03400700 1 --2- 172.21.15.86:0/4271300612 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56483ec5f8d0 0x56483ec647e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T18:06:03.230 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.229+0000 7fcc02000700 1 --2- 172.21.15.86:0/4271300612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56483ec54a20 0x56483ec642a0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fcbf001d750 tx=0x7fcbf001dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa0af670e34ae248 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:06:03.230 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.229+0000 7fcc02a00700 1 --2- 172.21.15.86:0/4271300612 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56483ec54100 0x56483ec5a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-03-23T18:06:03.230 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.229+0000 7fcbe3e00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-23T18:06:03.232 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.230+0000 7fcbe3e00700 1 -- 172.21.15.86:0/4271300612 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf001df00 con 0x56483ec54a20 2024-03-23T18:06:03.232 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.230+0000 7fcbe3e00700 1 -- 172.21.15.86:0/4271300612 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fcbf001e060 con 0x56483ec54a20 2024-03-23T18:06:03.232 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.230+0000 7fcc1d0cbb80 1 -- 172.21.15.86:0/4271300612 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcbf409b680 con 0x56483ec54a20 2024-03-23T18:06:03.232 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.231+0000 7fcbe3e00700 1 -- 172.21.15.86:0/4271300612 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcbf001e220 con 0x56483ec54a20 2024-03-23T18:06:03.232 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.231+0000 7fcc1d0cbb80 10 client.17154 fetch_fsmap learned FSMap version 72 2024-03-23T18:06:03.232 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.231+0000 7fcc1d0cbb80 1 -- 172.21.15.86:0/4271300612 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x56483ea2cc70 con 0x56483ec54a20 2024-03-23T18:06:03.232 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.231+0000 7fcbe3e00700 1 -- 172.21.15.86:0/4271300612 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcbf409b680 con 0x56483ec54a20 2024-03-23T18:06:03.232 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.231+0000 7fcbe3e00700 1 -- 172.21.15.86:0/4271300612 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 72) v1 ==== 7522+0+0 (secure 0 0 0) 0x7fcbf002f410 con 0x56483ec54a20 2024-03-23T18:06:03.232 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.232+0000 7fcc1d0cbb80 10 client.17154 fetch_fsmap finished waiting for FSMap version 72 2024-03-23T18:06:03.232 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.232+0000 7fcc1d0cbb80 10 client.17154 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T18:06:03.232 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.232+0000 7fcc1d0cbb80 20 client.17154 populate_metadata read hostname 'smithi086' 2024-03-23T18:06:03.232 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.232+0000 7fcc1d0cbb80 1 --2- 172.21.15.86:0/4271300612 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x56483ecc1580 0x56483ecc39a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:03.233 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.232+0000 7fcc1d0cbb80 4 client.17154 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T18:06:03.233 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.232+0000 7fcc1d0cbb80 1 -- 172.21.15.86:0/4271300612 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56483e9f5c90 con 0x56483ecc1580 2024-03-23T18:06:03.233 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.232+0000 7fcc02a00700 1 --2- 172.21.15.86:0/4271300612 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x56483ecc1580 0x56483ecc39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:06:03.233 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.233+0000 7fcc02a00700 1 --2- 172.21.15.86:0/4271300612 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x56483ecc1580 0x56483ecc39a0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fb1737c776479a6 server_cookie=c5e2e09bcc87daeb in_seq=0 out_seq=0 2024-03-23T18:06:03.233 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.233+0000 7fcbe3e00700 10 client.17154 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T18:06:03.237 INFO:tasks.workunit.client.0.smithi086.stdout: 10 108090 39.65 MB/sec execute 27 sec latency 2316.214 ms 2024-03-23T18:06:03.449 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.449+0000 7fcbe3e00700 1 -- 172.21.15.86:0/4271300612 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56483e9f5c90 con 0x56483ecc1580 2024-03-23T18:06:03.450 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.449+0000 7fcbe3e00700 10 client.17154 handle_command_reply: tid=0 2024-03-23T18:06:03.450 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.449+0000 7fcc1d0cbb80 1 client.17154 shutdown 2024-03-23T18:06:03.450 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.450+0000 7fcc1d0cbb80 1 -- 172.21.15.86:0/4271300612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56483ec54a20 msgr2=0x56483ec642a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:06:03.450 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.450+0000 7fcc1d0cbb80 1 --2- 172.21.15.86:0/4271300612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56483ec54a20 0x56483ec642a0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fcbf001d750 tx=0x7fcbf001dc30 comp rx=0 tx=0).stop 2024-03-23T18:06:03.451 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.450+0000 7fcc1d0cbb80 1 -- 172.21.15.86:0/4271300612 shutdown_connections 2024-03-23T18:06:03.451 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.450+0000 7fcc1d0cbb80 1 --2- 172.21.15.86:0/4271300612 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x56483ecc1580 0x56483ecc39a0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:03.451 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.450+0000 7fcc1d0cbb80 1 --2- 172.21.15.86:0/4271300612 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56483ec5f8d0 0x56483ec647e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:03.451 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.450+0000 7fcc1d0cbb80 1 --2- 172.21.15.86:0/4271300612 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56483ec54100 0x56483ec5a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:03.451 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.450+0000 7fcc1d0cbb80 1 --2- 172.21.15.86:0/4271300612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56483ec54a20 0x56483ec642a0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:03.451 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.450+0000 7fcc1d0cbb80 1 -- 172.21.15.86:0/4271300612 >> 172.21.15.86:0/4271300612 conn(0x56483ebb4640 msgr2=0x56483ec5be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:06:03.451 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.450+0000 7fcbe3e00700 0 client.17154 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T18:06:03.452 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.451+0000 7fcc1d0cbb80 1 -- 172.21.15.86:0/4271300612 shutdown_connections 2024-03-23T18:06:03.454 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.454+0000 7fcc1d0cbb80 1 -- 172.21.15.86:0/4271300612 wait complete. 2024-03-23T18:06:03.455 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.455+0000 7fcc1d0cbb80 20 client.17154 trim_cache size 0 max 16384 2024-03-23T18:06:03.456 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-03-23T18:06:03.456 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-03-23T18:06:03.456 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-03-23T18:06:03.456 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-03-23T18:06:03.457 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.457+0000 7fcc09600700 1 -- 172.21.15.86:0/477367023 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fcbe80a1460 msgr2=0x7fcbe80a3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:06:03.458 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.457+0000 7fcc09600700 1 --2- 172.21.15.86:0/477367023 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fcbe80a1460 0x7fcbe80a3920 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fcbf0009fd0 tx=0x7fcbf0009260 comp rx=0 tx=0).stop 2024-03-23T18:06:03.458 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.457+0000 7fcc09600700 1 -- 172.21.15.86:0/477367023 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcc04103120 msgr2=0x7fcc0410faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:06:03.458 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.457+0000 7fcc09600700 1 --2- 172.21.15.86:0/477367023 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcc04103120 0x7fcc0410faf0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fcbf4005910 tx=0x7fcbf4004730 comp rx=0 tx=0).stop 2024-03-23T18:06:03.458 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.458+0000 7fcc09600700 1 -- 172.21.15.86:0/477367023 shutdown_connections 2024-03-23T18:06:03.458 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.458+0000 7fcc09600700 1 --2- 172.21.15.86:0/477367023 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc0410d100 0x7fcc0410d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:03.458 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.458+0000 7fcc09600700 1 --2- 172.21.15.86:0/477367023 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fcbe80a1460 0x7fcbe80a3920 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:03.458 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.458+0000 7fcc09600700 1 --2- 172.21.15.86:0/477367023 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcc04103120 0x7fcc0410faf0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:03.458 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.458+0000 7fcc09600700 1 --2- 172.21.15.86:0/477367023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc04108240 0x7fcc0410cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:03.458 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.458+0000 7fcc09600700 1 -- 172.21.15.86:0/477367023 >> 172.21.15.86:0/477367023 conn(0x7fcc040fe960 msgr2=0x7fcc040ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:06:03.459 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.459+0000 7fcc09600700 1 -- 172.21.15.86:0/477367023 shutdown_connections 2024-03-23T18:06:03.459 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:03.459+0000 7fcc09600700 1 -- 172.21.15.86:0/477367023 wait complete. 2024-03-23T18:06:04.237 INFO:tasks.workunit.client.0.smithi086.stdout: 10 108313 38.48 MB/sec execute 28 sec latency 249.426 ms 2024-03-23T18:06:04.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:06:04 smithi086 ceph-mon[38221]: pgmap v484: 97 pgs: 97 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.6 MiB/s wr, 641 op/s 2024-03-23T18:06:04.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:06:04 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T18:06:04.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:06:04 smithi092 ceph-mon[79089]: pgmap v484: 97 pgs: 97 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.6 MiB/s wr, 641 op/s 2024-03-23T18:06:04.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:06:04 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T18:06:04.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:06:04 smithi152 ceph-mon[47949]: pgmap v484: 97 pgs: 97 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.6 MiB/s wr, 641 op/s 2024-03-23T18:06:04.375 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:06:04 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T18:06:05.237 INFO:tasks.workunit.client.0.smithi086.stdout: 10 108609 37.74 MB/sec execute 29 sec latency 255.918 ms 2024-03-23T18:06:05.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:06:05 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T18:06:05.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:06:05 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T18:06:05.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:06:05 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T18:06:05.446 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:06:05 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T18:06:05.446 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:06:05 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T18:06:05.446 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:06:05 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T18:06:05.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:06:05 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T18:06:05.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:06:05 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T18:06:05.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:06:05 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T18:06:06.237 INFO:tasks.workunit.client.0.smithi086.stdout: 10 109016 37.16 MB/sec execute 30 sec latency 285.234 ms 2024-03-23T18:06:06.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:06:06 smithi152 ceph-mon[47949]: pgmap v485: 97 pgs: 97 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 7.7 MiB/s wr, 790 op/s 2024-03-23T18:06:06.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:06:06 smithi152 ceph-mon[47949]: osdmap e103: 12 total, 12 up, 12 in 2024-03-23T18:06:06.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:06:06 smithi086 ceph-mon[38221]: pgmap v485: 97 pgs: 97 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 7.7 MiB/s wr, 790 op/s 2024-03-23T18:06:06.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:06:06 smithi086 ceph-mon[38221]: osdmap e103: 12 total, 12 up, 12 in 2024-03-23T18:06:06.570 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:06:06 smithi092 ceph-mon[79089]: pgmap v485: 97 pgs: 97 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 7.7 MiB/s wr, 790 op/s 2024-03-23T18:06:06.570 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:06:06 smithi092 ceph-mon[79089]: osdmap e103: 12 total, 12 up, 12 in 2024-03-23T18:06:07.237 INFO:tasks.workunit.client.0.smithi086.stdout: 10 109671 37.07 MB/sec execute 31 sec latency 92.013 ms 2024-03-23T18:06:08.238 INFO:tasks.workunit.client.0.smithi086.stdout: 10 110655 37.79 MB/sec execute 32 sec latency 49.960 ms 2024-03-23T18:06:08.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:06:08 smithi092 ceph-mon[79089]: pgmap v487: 97 pgs: 97 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 6.5 MiB/s wr, 792 op/s 2024-03-23T18:06:08.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:06:08 smithi152 ceph-mon[47949]: pgmap v487: 97 pgs: 97 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 6.5 MiB/s wr, 792 op/s 2024-03-23T18:06:08.518 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:06:08 smithi086 ceph-mon[38221]: pgmap v487: 97 pgs: 97 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 6.5 MiB/s wr, 792 op/s 2024-03-23T18:06:09.238 INFO:tasks.workunit.client.0.smithi086.stdout: 10 111342 37.86 MB/sec execute 33 sec latency 250.931 ms 2024-03-23T18:06:09.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:06:09 smithi092 ceph-mon[79089]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:06:09.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:06:09 smithi152 ceph-mon[47949]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:06:09.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:06:09 smithi086 ceph-mon[38221]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:06:10.238 INFO:tasks.workunit.client.0.smithi086.stdout: 10 112256 38.01 MB/sec execute 34 sec latency 49.064 ms 2024-03-23T18:06:10.314 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:06:10 smithi092 ceph-mon[79089]: pgmap v488: 97 pgs: 97 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 6.5 MiB/s wr, 792 op/s 2024-03-23T18:06:10.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:06:10 smithi152 ceph-mon[47949]: pgmap v488: 97 pgs: 97 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 6.5 MiB/s wr, 792 op/s 2024-03-23T18:06:10.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:06:10 smithi086 ceph-mon[38221]: pgmap v488: 97 pgs: 97 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 6.5 MiB/s wr, 792 op/s 2024-03-23T18:06:11.238 INFO:tasks.workunit.client.0.smithi086.stdout: 10 112809 38.07 MB/sec execute 35 sec latency 274.675 ms 2024-03-23T18:06:12.238 INFO:tasks.workunit.client.0.smithi086.stdout: 10 113560 37.82 MB/sec execute 36 sec latency 53.757 ms 2024-03-23T18:06:12.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:06:12 smithi092 ceph-mon[79089]: pgmap v489: 97 pgs: 97 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 6.8 MiB/s wr, 734 op/s 2024-03-23T18:06:12.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:06:12 smithi152 ceph-mon[47949]: pgmap v489: 97 pgs: 97 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 6.8 MiB/s wr, 734 op/s 2024-03-23T18:06:12.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:06:12 smithi086 ceph-mon[38221]: pgmap v489: 97 pgs: 97 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 6.8 MiB/s wr, 734 op/s 2024-03-23T18:06:13.238 INFO:tasks.workunit.client.0.smithi086.stdout: 10 114474 38.33 MB/sec execute 37 sec latency 53.113 ms 2024-03-23T18:06:13.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:06:13 smithi086 ceph-mon[38221]: pgmap v490: 97 pgs: 97 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 6.8 MiB/s wr, 734 op/s 2024-03-23T18:06:13.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:06:13 smithi092 ceph-mon[79089]: pgmap v490: 97 pgs: 97 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 6.8 MiB/s wr, 734 op/s 2024-03-23T18:06:13.585 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:06:13 smithi152 ceph-mon[47949]: pgmap v490: 97 pgs: 97 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 6.8 MiB/s wr, 734 op/s 2024-03-23T18:06:14.238 INFO:tasks.workunit.client.0.smithi086.stdout: 10 115373 38.37 MB/sec execute 38 sec latency 110.366 ms 2024-03-23T18:06:15.238 INFO:tasks.workunit.client.0.smithi086.stdout: 10 116111 38.59 MB/sec execute 39 sec latency 70.071 ms 2024-03-23T18:06:16.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:06:15 smithi086 ceph-mon[38221]: pgmap v491: 97 pgs: 97 active+clean; 1.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 10 MiB/s wr, 838 op/s 2024-03-23T18:06:16.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:06:15 smithi092 ceph-mon[79089]: pgmap v491: 97 pgs: 97 active+clean; 1.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 10 MiB/s wr, 838 op/s 2024-03-23T18:06:16.085 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:06:15 smithi152 ceph-mon[47949]: pgmap v491: 97 pgs: 97 active+clean; 1.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 10 MiB/s wr, 838 op/s 2024-03-23T18:06:16.238 INFO:tasks.workunit.client.0.smithi086.stdout: 10 116706 38.32 MB/sec execute 40 sec latency 185.449 ms 2024-03-23T18:06:16.717 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.716+0000 7ffb2b800700 1 -- 172.21.15.86:0/681633204 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffb24100f10 msgr2=0x7ffb241012f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:06:16.717 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.716+0000 7ffb2b800700 1 --2- 172.21.15.86:0/681633204 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffb24100f10 0x7ffb241012f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c00b0a0 tx=0x7ffb1c00b3b0 comp rx=0 tx=0).stop 2024-03-23T18:06:16.717 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.716+0000 7ffb2b800700 1 -- 172.21.15.86:0/681633204 shutdown_connections 2024-03-23T18:06:16.717 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.716+0000 7ffb2b800700 1 --2- 172.21.15.86:0/681633204 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb24106030 0x7ffb2410dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:16.717 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.716+0000 7ffb2b800700 1 --2- 172.21.15.86:0/681633204 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffb24100f10 0x7ffb241012f0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:16.717 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.716+0000 7ffb2b800700 1 --2- 172.21.15.86:0/681633204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb241018c0 0x7ffb24105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:16.717 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.716+0000 7ffb2b800700 1 -- 172.21.15.86:0/681633204 >> 172.21.15.86:0/681633204 conn(0x7ffb240fc770 msgr2=0x7ffb240feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:06:16.717 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.717+0000 7ffb2b800700 1 -- 172.21.15.86:0/681633204 shutdown_connections 2024-03-23T18:06:16.718 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.717+0000 7ffb2b800700 1 -- 172.21.15.86:0/681633204 wait complete. 2024-03-23T18:06:16.718 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.718+0000 7ffb2b800700 1 Processor -- start 2024-03-23T18:06:16.718 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.718+0000 7ffb2b800700 1 -- start start 2024-03-23T18:06:16.719 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.719+0000 7ffb2b800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb24100f10 0x7ffb241995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:16.719 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.719+0000 7ffb2b800700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffb24106030 0x7ffb24199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:16.719 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.719+0000 7ffb2b800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb2419de40 0x7ffb2419e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:16.719 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.719+0000 7ffb2b800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb2419e8c0 con 0x7ffb2419de40 2024-03-23T18:06:16.720 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.719+0000 7ffb2b800700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb2419ea30 con 0x7ffb24100f10 2024-03-23T18:06:16.720 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.719+0000 7ffb2b800700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb2419ebf0 con 0x7ffb24106030 2024-03-23T18:06:16.720 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.720+0000 7ffb28e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb24100f10 0x7ffb241995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:06:16.720 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.720+0000 7ffb29800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb2419de40 0x7ffb2419e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:06:16.720 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.720+0000 7ffb23e00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffb24106030 0x7ffb24199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:06:16.720 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.720+0000 7ffb29800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb2419de40 0x7ffb2419e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43708/0 (socket says 172.21.15.86:43708) 2024-03-23T18:06:16.721 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.720+0000 7ffb29800700 1 -- 172.21.15.86:0/2086597105 learned_addr learned my addr 172.21.15.86:0/2086597105 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:06:16.721 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.721+0000 7ffb23e00700 1 -- 172.21.15.86:0/2086597105 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb24100f10 msgr2=0x7ffb241995b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:06:16.721 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.721+0000 7ffb23e00700 1 --2- 172.21.15.86:0/2086597105 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb24100f10 0x7ffb241995b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:16.721 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.721+0000 7ffb23e00700 1 -- 172.21.15.86:0/2086597105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb2419de40 msgr2=0x7ffb2419e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:06:16.721 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.721+0000 7ffb23e00700 1 --2- 172.21.15.86:0/2086597105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb2419de40 0x7ffb2419e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:16.722 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.721+0000 7ffb23e00700 1 -- 172.21.15.86:0/2086597105 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb1c009d00 con 0x7ffb24106030 2024-03-23T18:06:16.722 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.721+0000 7ffb29800700 1 --2- 172.21.15.86:0/2086597105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb2419de40 0x7ffb2419e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T18:06:16.722 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.721+0000 7ffb23e00700 1 --2- 172.21.15.86:0/2086597105 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffb24106030 0x7ffb24199af0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ffb24067c80 tx=0x7ffb1000dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:06:16.723 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.723+0000 7ffb21600700 1 -- 172.21.15.86:0/2086597105 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb1000f890 con 0x7ffb24106030 2024-03-23T18:06:16.723 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.723+0000 7ffb21600700 1 -- 172.21.15.86:0/2086597105 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb10010460 con 0x7ffb24106030 2024-03-23T18:06:16.723 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.723+0000 7ffb21600700 1 -- 172.21.15.86:0/2086597105 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb1000e610 con 0x7ffb24106030 2024-03-23T18:06:16.723 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.723+0000 7ffb2b800700 1 -- 172.21.15.86:0/2086597105 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb241a6150 con 0x7ffb24106030 2024-03-23T18:06:16.723 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.723+0000 7ffb2b800700 1 -- 172.21.15.86:0/2086597105 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb241a65e0 con 0x7ffb24106030 2024-03-23T18:06:16.728 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.728+0000 7ffb21600700 1 -- 172.21.15.86:0/2086597105 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7ffb1000e770 con 0x7ffb24106030 2024-03-23T18:06:16.732 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.732+0000 7ffb21600700 1 --2- 172.21.15.86:0/2086597105 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7ffb080a1460 0x7ffb080a3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:16.733 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.733+0000 7ffb21600700 1 -- 172.21.15.86:0/2086597105 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7ffb100cd7e0 con 0x7ffb24106030 2024-03-23T18:06:16.734 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.733+0000 7ffb28e00700 1 --2- 172.21.15.86:0/2086597105 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7ffb080a1460 0x7ffb080a3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:06:16.739 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.736+0000 7ffb28e00700 1 --2- 172.21.15.86:0/2086597105 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7ffb080a1460 0x7ffb080a3920 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c000f80 tx=0x7ffb1c00b7f0 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:06:16.739 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.737+0000 7ffb3f2c6b80 1 Processor -- start 2024-03-23T18:06:16.739 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.737+0000 7ffb3f2c6b80 1 -- start start 2024-03-23T18:06:16.739 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.738+0000 7ffb3f2c6b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5601d0fa0100 0x5601d0fa04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:16.739 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.738+0000 7ffb3f2c6b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601d0fa0a20 0x5601d0fab390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:16.739 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.738+0000 7ffb3f2c6b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601d0fab8d0 0x5601d0fadcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:16.740 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.738+0000 7ffb3f2c6b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601d0ef9930 con 0x5601d0fa0a20 2024-03-23T18:06:16.740 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.738+0000 7ffb3f2c6b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601d0ef72f0 con 0x5601d0fab8d0 2024-03-23T18:06:16.740 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.738+0000 7ffb3f2c6b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601d0ef2be0 con 0x5601d0fa0100 2024-03-23T18:06:16.740 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.739+0000 7ffb23e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601d0fa0a20 0x5601d0fab390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:06:16.740 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.739+0000 7ffb23e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601d0fa0a20 0x5601d0fab390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43712/0 (socket says 172.21.15.86:43712) 2024-03-23T18:06:16.741 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.739+0000 7ffb23e00700 1 -- 172.21.15.86:0/4137577221 learned_addr learned my addr 172.21.15.86:0/4137577221 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:06:16.741 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.739+0000 7ffb28e00700 1 --2- 172.21.15.86:0/4137577221 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5601d0fa0100 0x5601d0fa04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:06:16.741 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.739+0000 7ffb29800700 1 --2- 172.21.15.86:0/4137577221 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601d0fab8d0 0x5601d0fadcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:06:16.741 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.739+0000 7ffb23e00700 1 -- 172.21.15.86:0/4137577221 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5601d0fa0100 msgr2=0x5601d0fa04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:06:16.741 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.739+0000 7ffb23e00700 1 --2- 172.21.15.86:0/4137577221 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5601d0fa0100 0x5601d0fa04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:16.741 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.739+0000 7ffb23e00700 1 -- 172.21.15.86:0/4137577221 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601d0fab8d0 msgr2=0x5601d0fadcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:06:16.741 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.739+0000 7ffb23e00700 1 --2- 172.21.15.86:0/4137577221 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601d0fab8d0 0x5601d0fadcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:16.741 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.739+0000 7ffb23e00700 1 -- 172.21.15.86:0/4137577221 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5601d0d74350 con 0x5601d0fa0a20 2024-03-23T18:06:16.741 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.739+0000 7ffb23e00700 1 --2- 172.21.15.86:0/4137577221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601d0fa0a20 0x5601d0fab390 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7ffb1009a840 tx=0x7ffb1009a520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bcea1ee4698389e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:06:16.741 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.740+0000 7ffb1a000700 1 -- 172.21.15.86:0/4137577221 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb100b8ea0 con 0x5601d0fa0a20 2024-03-23T18:06:16.741 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.740+0000 7ffb1a000700 1 -- 172.21.15.86:0/4137577221 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb1009e6e0 con 0x5601d0fa0a20 2024-03-23T18:06:16.741 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.740+0000 7ffb1a000700 1 -- 172.21.15.86:0/4137577221 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb1000ec80 con 0x5601d0fa0a20 2024-03-23T18:06:16.742 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.741+0000 7ffb3f2c6b80 1 -- 172.21.15.86:0/4137577221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601d0fa0a20 msgr2=0x5601d0fab390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:06:16.742 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.741+0000 7ffb3f2c6b80 1 --2- 172.21.15.86:0/4137577221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601d0fa0a20 0x5601d0fab390 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7ffb1009a840 tx=0x7ffb1009a520 comp rx=0 tx=0).stop 2024-03-23T18:06:16.742 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.741+0000 7ffb3f2c6b80 1 -- 172.21.15.86:0/4137577221 shutdown_connections 2024-03-23T18:06:16.742 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.741+0000 7ffb3f2c6b80 1 --2- 172.21.15.86:0/4137577221 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601d0fab8d0 0x5601d0fadcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:16.743 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.741+0000 7ffb3f2c6b80 1 --2- 172.21.15.86:0/4137577221 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5601d0fa0100 0x5601d0fa04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:16.743 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.741+0000 7ffb3f2c6b80 1 --2- 172.21.15.86:0/4137577221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601d0fa0a20 0x5601d0fab390 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:16.743 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.741+0000 7ffb3f2c6b80 1 -- 172.21.15.86:0/4137577221 >> 172.21.15.86:0/4137577221 conn(0x5601d0f00640 msgr2=0x5601d0f9c520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:06:16.743 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.742+0000 7ffb3f2c6b80 1 -- 172.21.15.86:0/4137577221 shutdown_connections 2024-03-23T18:06:16.743 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.742+0000 7ffb3f2c6b80 1 -- 172.21.15.86:0/4137577221 wait complete. 2024-03-23T18:06:16.743 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.742+0000 7ffb3f2c6b80 1 -- start start 2024-03-23T18:06:16.743 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.743+0000 7ffb3f2c6b80 1 Processor -- start 2024-03-23T18:06:16.744 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.744+0000 7ffb3f2c6b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5601d0fa0100 0x5601d0fab2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:16.745 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.744+0000 7ffb3f2c6b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601d0fa0a20 0x5601d0fa85b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:16.745 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.744+0000 7ffb3f2c6b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601d0fab8d0 0x5601d0fa8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:16.745 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.744+0000 7ffb3f2c6b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601d0e8ed20 con 0x5601d0fab8d0 2024-03-23T18:06:16.745 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.744+0000 7ffb3f2c6b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601d0e8e9c0 con 0x5601d0fa0a20 2024-03-23T18:06:16.745 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.744+0000 7ffb3f2c6b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601d0d76640 con 0x5601d0fa0100 2024-03-23T18:06:16.745 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.744+0000 7ffb3f2c6b80 10 client.0 _ll_register_callbacks cb 0x5601d0e98410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T18:06:16.745 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.745+0000 7ffb29800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601d0fab8d0 0x5601d0fa8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:06:16.745 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.745+0000 7ffb23e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601d0fa0a20 0x5601d0fa85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:06:16.746 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.745+0000 7ffb23e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601d0fa0a20 0x5601d0fa85b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:45018/0 (socket says 172.21.15.86:45018) 2024-03-23T18:06:16.746 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.745+0000 7ffb23e00700 1 -- 172.21.15.86:0/4013382068 learned_addr learned my addr 172.21.15.86:0/4013382068 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:06:16.746 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.745+0000 7ffb28e00700 1 --2- 172.21.15.86:0/4013382068 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5601d0fa0100 0x5601d0fab2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:06:16.746 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.746+0000 7ffb23e00700 1 -- 172.21.15.86:0/4013382068 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5601d0fa0100 msgr2=0x5601d0fab2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:06:16.747 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.746+0000 7ffb23e00700 1 --2- 172.21.15.86:0/4013382068 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5601d0fa0100 0x5601d0fab2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:16.747 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.746+0000 7ffb23e00700 1 -- 172.21.15.86:0/4013382068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601d0fab8d0 msgr2=0x5601d0fa8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:06:16.747 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.746+0000 7ffb23e00700 1 --2- 172.21.15.86:0/4013382068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601d0fab8d0 0x5601d0fa8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:16.747 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.746+0000 7ffb23e00700 1 -- 172.21.15.86:0/4013382068 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb241a65e0 con 0x5601d0fa0a20 2024-03-23T18:06:16.747 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.746+0000 7ffb23e00700 1 --2- 172.21.15.86:0/4013382068 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601d0fa0a20 0x5601d0fa85b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ffb1000df80 tx=0x7ffb1009a7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a504b9561e374b51 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:06:16.747 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.747+0000 7ffb0ea00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T18:06:16.747 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.747+0000 7ffb0ea00700 1 -- 172.21.15.86:0/4013382068 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb1009ab10 con 0x5601d0fa0a20 2024-03-23T18:06:16.747 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.747+0000 7ffb0ea00700 1 -- 172.21.15.86:0/4013382068 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7ffb1009ac70 con 0x5601d0fa0a20 2024-03-23T18:06:16.747 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.747+0000 7ffb3f2c6b80 1 -- 172.21.15.86:0/4013382068 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5601d0eb3320 con 0x5601d0fa0a20 2024-03-23T18:06:16.748 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.747+0000 7ffb0ea00700 1 -- 172.21.15.86:0/4013382068 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb10099da0 con 0x5601d0fa0a20 2024-03-23T18:06:16.748 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.747+0000 7ffb3f2c6b80 10 client.27266 fetch_fsmap learned FSMap version 74 2024-03-23T18:06:16.748 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.747+0000 7ffb3f2c6b80 1 -- 172.21.15.86:0/4013382068 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x5601d0d20b00 con 0x5601d0fa0a20 2024-03-23T18:06:16.748 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.748+0000 7ffb0ea00700 1 -- 172.21.15.86:0/4013382068 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5601d0eb3320 con 0x5601d0fa0a20 2024-03-23T18:06:16.749 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.749+0000 7ffb0ea00700 1 -- 172.21.15.86:0/4013382068 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 74) v1 ==== 7510+0+0 (secure 0 0 0) 0x7ffb100a28a0 con 0x5601d0fa0a20 2024-03-23T18:06:16.749 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.749+0000 7ffb3f2c6b80 10 client.27266 fetch_fsmap finished waiting for FSMap version 74 2024-03-23T18:06:16.749 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.749+0000 7ffb3f2c6b80 10 client.27266 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T18:06:16.749 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.749+0000 7ffb3f2c6b80 20 client.27266 populate_metadata read hostname 'smithi086' 2024-03-23T18:06:16.749 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.749+0000 7ffb3f2c6b80 1 --2- 172.21.15.86:0/4013382068 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5601d0fc3620 0x5601d0fc5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:16.749 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.749+0000 7ffb3f2c6b80 4 client.27266 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T18:06:16.749 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.749+0000 7ffb3f2c6b80 1 -- 172.21.15.86:0/4013382068 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5601d0eeb6a0 con 0x5601d0fc3620 2024-03-23T18:06:16.749 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.749+0000 7ffb28e00700 1 --2- 172.21.15.86:0/4013382068 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5601d0fc3620 0x5601d0fc5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:06:16.750 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.750+0000 7ffb28e00700 1 --2- 172.21.15.86:0/4013382068 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5601d0fc3620 0x5601d0fc5a10 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fce2fe2341c27d8 server_cookie=847ddc173403b718 in_seq=0 out_seq=0 2024-03-23T18:06:16.750 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.750+0000 7ffb0ea00700 10 client.27266 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T18:06:16.752 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.752+0000 7ffb0ea00700 1 -- 172.21.15.86:0/4013382068 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5601d0eeb6a0 con 0x5601d0fc3620 2024-03-23T18:06:16.752 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.752+0000 7ffb0ea00700 10 client.27266 handle_command_reply: tid=0 2024-03-23T18:06:16.752 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.752+0000 7ffb3f2c6b80 1 client.27266 shutdown 2024-03-23T18:06:16.753 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.752+0000 7ffb3f2c6b80 1 -- 172.21.15.86:0/4013382068 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601d0fa0a20 msgr2=0x5601d0fa85b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:06:16.753 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.752+0000 7ffb3f2c6b80 1 --2- 172.21.15.86:0/4013382068 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601d0fa0a20 0x5601d0fa85b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ffb1000df80 tx=0x7ffb1009a7b0 comp rx=0 tx=0).stop 2024-03-23T18:06:16.753 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.753+0000 7ffb3f2c6b80 1 -- 172.21.15.86:0/4013382068 shutdown_connections 2024-03-23T18:06:16.753 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.753+0000 7ffb3f2c6b80 1 --2- 172.21.15.86:0/4013382068 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5601d0fc3620 0x5601d0fc5a10 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:16.753 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.753+0000 7ffb3f2c6b80 1 --2- 172.21.15.86:0/4013382068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601d0fab8d0 0x5601d0fa8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:16.753 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.753+0000 7ffb3f2c6b80 1 --2- 172.21.15.86:0/4013382068 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5601d0fa0100 0x5601d0fab2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:16.753 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.753+0000 7ffb3f2c6b80 1 --2- 172.21.15.86:0/4013382068 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601d0fa0a20 0x5601d0fa85b0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:16.753 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.753+0000 7ffb3f2c6b80 1 -- 172.21.15.86:0/4013382068 >> 172.21.15.86:0/4013382068 conn(0x5601d0f00640 msgr2=0x5601d0fa2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:06:16.753 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.753+0000 7ffb0ea00700 0 client.27266 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T18:06:16.754 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.754+0000 7ffb3f2c6b80 1 -- 172.21.15.86:0/4013382068 shutdown_connections 2024-03-23T18:06:16.754 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.754+0000 7ffb3f2c6b80 1 -- 172.21.15.86:0/4013382068 wait complete. 2024-03-23T18:06:16.754 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.754+0000 7ffb3f2c6b80 20 client.27266 trim_cache size 0 max 16384 2024-03-23T18:06:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.780+0000 7ffb3f2c6b80 1 Processor -- start 2024-03-23T18:06:16.782 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.781+0000 7ffb3f2c6b80 1 -- start start 2024-03-23T18:06:16.782 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.781+0000 7ffb3f2c6b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601d0fa0100 0x5601d0fa3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:16.782 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.781+0000 7ffb3f2c6b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601d0fa0a20 0x5601d0fa8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:16.782 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.781+0000 7ffb3f2c6b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5601d0fab8d0 0x5601d0f9c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:16.783 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.781+0000 7ffb3f2c6b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601d0f98c80 con 0x5601d0fa0a20 2024-03-23T18:06:16.783 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.781+0000 7ffb3f2c6b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601d0fc0810 con 0x5601d0fa0100 2024-03-23T18:06:16.783 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.781+0000 7ffb3f2c6b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601d0fa1bc0 con 0x5601d0fab8d0 2024-03-23T18:06:16.783 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.781+0000 7ffb29800700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5601d0fab8d0 0x5601d0f9c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:06:16.783 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.782+0000 7ffb29800700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5601d0fab8d0 0x5601d0f9c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:44478/0 (socket says 172.21.15.86:44478) 2024-03-23T18:06:16.783 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.782+0000 7ffb29800700 1 -- 172.21.15.86:0/582505400 learned_addr learned my addr 172.21.15.86:0/582505400 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:06:16.783 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.782+0000 7ffb29800700 1 -- 172.21.15.86:0/582505400 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601d0fa0100 msgr2=0x5601d0fa3050 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-23T18:06:16.783 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.782+0000 7ffb29800700 1 --2- 172.21.15.86:0/582505400 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601d0fa0100 0x5601d0fa3050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:16.784 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.782+0000 7ffb29800700 1 -- 172.21.15.86:0/582505400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601d0fa0a20 msgr2=0x5601d0fa8b00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-23T18:06:16.784 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.782+0000 7ffb29800700 1 --2- 172.21.15.86:0/582505400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601d0fa0a20 0x5601d0fa8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:16.784 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.782+0000 7ffb29800700 1 -- 172.21.15.86:0/582505400 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5601d0eb8e40 con 0x5601d0fab8d0 2024-03-23T18:06:16.784 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.782+0000 7ffb29800700 1 --2- 172.21.15.86:0/582505400 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5601d0fab8d0 0x5601d0f9c460 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ffb14009e20 tx=0x7ffb14009210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9e71720dda6549a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:06:16.784 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.783+0000 7ffb1aa00700 1 -- 172.21.15.86:0/582505400 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb14013ae0 con 0x5601d0fab8d0 2024-03-23T18:06:16.784 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.783+0000 7ffb1aa00700 1 -- 172.21.15.86:0/582505400 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb14013c40 con 0x5601d0fab8d0 2024-03-23T18:06:16.784 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.783+0000 7ffb1aa00700 1 -- 172.21.15.86:0/582505400 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb1401e620 con 0x5601d0fab8d0 2024-03-23T18:06:16.785 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.784+0000 7ffb3f2c6b80 1 -- 172.21.15.86:0/582505400 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5601d0fab8d0 msgr2=0x5601d0f9c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:06:16.785 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.784+0000 7ffb3f2c6b80 1 --2- 172.21.15.86:0/582505400 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5601d0fab8d0 0x5601d0f9c460 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ffb14009e20 tx=0x7ffb14009210 comp rx=0 tx=0).stop 2024-03-23T18:06:16.785 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.784+0000 7ffb3f2c6b80 1 -- 172.21.15.86:0/582505400 shutdown_connections 2024-03-23T18:06:16.785 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.784+0000 7ffb3f2c6b80 1 --2- 172.21.15.86:0/582505400 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5601d0fab8d0 0x5601d0f9c460 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:16.785 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.784+0000 7ffb3f2c6b80 1 --2- 172.21.15.86:0/582505400 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601d0fa0100 0x5601d0fa3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:16.785 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.784+0000 7ffb3f2c6b80 1 --2- 172.21.15.86:0/582505400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601d0fa0a20 0x5601d0fa8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:16.785 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.784+0000 7ffb3f2c6b80 1 -- 172.21.15.86:0/582505400 >> 172.21.15.86:0/582505400 conn(0x5601d0f00640 msgr2=0x5601d0f9bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:06:16.785 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.784+0000 7ffb3f2c6b80 1 -- 172.21.15.86:0/582505400 shutdown_connections 2024-03-23T18:06:16.785 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.784+0000 7ffb3f2c6b80 1 -- 172.21.15.86:0/582505400 wait complete. 2024-03-23T18:06:16.786 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.785+0000 7ffb3f2c6b80 1 -- start start 2024-03-23T18:06:16.786 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.785+0000 7ffb3f2c6b80 1 Processor -- start 2024-03-23T18:06:16.787 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.785+0000 7ffb3f2c6b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601d0fa0100 0x5601d0fb8c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:16.787 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.785+0000 7ffb3f2c6b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5601d0fa0a20 0x5601d0fb9140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:16.787 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.785+0000 7ffb3f2c6b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601d0fab8d0 0x5601d101f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:16.787 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.785+0000 7ffb3f2c6b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601d101f630 con 0x5601d0fa0100 2024-03-23T18:06:16.787 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.785+0000 7ffb3f2c6b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601d101f7a0 con 0x5601d0fab8d0 2024-03-23T18:06:16.787 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.785+0000 7ffb3f2c6b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601d101f960 con 0x5601d0fa0a20 2024-03-23T18:06:16.787 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.785+0000 7ffb3f2c6b80 10 client.0 _ll_register_callbacks cb 0x5601d0e98410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T18:06:16.787 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.786+0000 7ffb23e00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5601d0fa0a20 0x5601d0fb9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:06:16.787 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.786+0000 7ffb23e00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5601d0fa0a20 0x5601d0fb9140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:44490/0 (socket says 172.21.15.86:44490) 2024-03-23T18:06:16.787 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.786+0000 7ffb23e00700 1 -- 172.21.15.86:0/1341055268 learned_addr learned my addr 172.21.15.86:0/1341055268 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:06:16.787 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.786+0000 7ffb29800700 1 --2- 172.21.15.86:0/1341055268 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601d0fab8d0 0x5601d101f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:06:16.787 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.786+0000 7ffb28e00700 1 --2- 172.21.15.86:0/1341055268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601d0fa0100 0x5601d0fb8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:06:16.788 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.786+0000 7ffb23e00700 1 -- 172.21.15.86:0/1341055268 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601d0fab8d0 msgr2=0x5601d101f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:06:16.788 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.786+0000 7ffb23e00700 1 --2- 172.21.15.86:0/1341055268 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601d0fab8d0 0x5601d101f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:16.788 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.786+0000 7ffb23e00700 1 -- 172.21.15.86:0/1341055268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601d0fa0100 msgr2=0x5601d0fb8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:06:16.788 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.786+0000 7ffb23e00700 1 --2- 172.21.15.86:0/1341055268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601d0fa0100 0x5601d0fb8c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:16.788 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.786+0000 7ffb23e00700 1 -- 172.21.15.86:0/1341055268 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb14009aa0 con 0x5601d0fa0a20 2024-03-23T18:06:16.789 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.786+0000 7ffb28e00700 1 --2- 172.21.15.86:0/1341055268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601d0fa0100 0x5601d0fb8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T18:06:16.789 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.787+0000 7ffb23e00700 1 --2- 172.21.15.86:0/1341055268 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5601d0fa0a20 0x5601d0fb9140 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ffb10098c00 tx=0x7ffb1009aa30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5c5511c238e2172 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:06:16.789 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.787+0000 7ffb0ea00700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-03-23T18:06:16.789 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.787+0000 7ffb0ea00700 1 -- 172.21.15.86:0/1341055268 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb100a2ab0 con 0x5601d0fa0a20 2024-03-23T18:06:16.789 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.787+0000 7ffb0ea00700 1 -- 172.21.15.86:0/1341055268 <== mon.1 v2:172.21.15.152:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7ffb100a2d30 con 0x5601d0fa0a20 2024-03-23T18:06:16.789 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.787+0000 7ffb3f2c6b80 1 -- 172.21.15.86:0/1341055268 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffb1009e6e0 con 0x5601d0fa0a20 2024-03-23T18:06:16.789 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.787+0000 7ffb0ea00700 1 -- 172.21.15.86:0/1341055268 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb100ba380 con 0x5601d0fa0a20 2024-03-23T18:06:16.789 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.787+0000 7ffb3f2c6b80 10 client.27253 fetch_fsmap learned FSMap version 74 2024-03-23T18:06:16.789 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.788+0000 7ffb3f2c6b80 1 -- 172.21.15.86:0/1341055268 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x5601d0d78c70 con 0x5601d0fa0a20 2024-03-23T18:06:16.789 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.788+0000 7ffb0ea00700 1 -- 172.21.15.86:0/1341055268 <== mon.1 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb1009e6e0 con 0x5601d0fa0a20 2024-03-23T18:06:16.790 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.788+0000 7ffb0ea00700 1 -- 172.21.15.86:0/1341055268 <== mon.1 v2:172.21.15.152:3300/0 5 ==== fsmap(e 74) v1 ==== 7510+0+0 (secure 0 0 0) 0x7ffb10099da0 con 0x5601d0fa0a20 2024-03-23T18:06:16.790 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.788+0000 7ffb3f2c6b80 10 client.27253 fetch_fsmap finished waiting for FSMap version 74 2024-03-23T18:06:16.790 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.788+0000 7ffb3f2c6b80 10 client.27253 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T18:06:16.790 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.788+0000 7ffb3f2c6b80 20 client.27253 populate_metadata read hostname 'smithi086' 2024-03-23T18:06:16.790 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.788+0000 7ffb3f2c6b80 1 --2- 172.21.15.86:0/1341055268 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5601d0faeeb0 0x5601d0fb12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:16.790 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.788+0000 7ffb3f2c6b80 4 client.27253 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T18:06:16.790 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.788+0000 7ffb3f2c6b80 1 -- 172.21.15.86:0/1341055268 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5601d0d41c90 con 0x5601d0faeeb0 2024-03-23T18:06:16.790 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.789+0000 7ffb28e00700 1 --2- 172.21.15.86:0/1341055268 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5601d0faeeb0 0x5601d0fb12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:06:16.791 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.790+0000 7ffb0ea00700 10 client.27253 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T18:06:16.791 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:16.790+0000 7ffb28e00700 1 --2- 172.21.15.86:0/1341055268 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5601d0faeeb0 0x5601d0fb12d0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32afbcc3146510a8 server_cookie=8c08906e5c64a6a0 in_seq=0 out_seq=0 2024-03-23T18:06:17.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:06:16 smithi086 ceph-mon[38221]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:06:17.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:06:16 smithi086 ceph-mon[38221]: osdmap e104: 12 total, 12 up, 12 in 2024-03-23T18:06:17.024 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:17.024+0000 7ffb0ea00700 1 -- 172.21.15.86:0/1341055268 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5601d0d41c90 con 0x5601d0faeeb0 2024-03-23T18:06:17.024 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:17.024+0000 7ffb0ea00700 10 client.27253 handle_command_reply: tid=0 2024-03-23T18:06:17.025 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:17.025+0000 7ffb3f2c6b80 1 client.27253 shutdown 2024-03-23T18:06:17.025 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:17.025+0000 7ffb3f2c6b80 1 -- 172.21.15.86:0/1341055268 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5601d0fa0a20 msgr2=0x5601d0fb9140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:06:17.025 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:17.025+0000 7ffb3f2c6b80 1 --2- 172.21.15.86:0/1341055268 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5601d0fa0a20 0x5601d0fb9140 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ffb10098c00 tx=0x7ffb1009aa30 comp rx=0 tx=0).stop 2024-03-23T18:06:17.025 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:17.025+0000 7ffb3f2c6b80 1 -- 172.21.15.86:0/1341055268 shutdown_connections 2024-03-23T18:06:17.025 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:17.025+0000 7ffb3f2c6b80 1 --2- 172.21.15.86:0/1341055268 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5601d0faeeb0 0x5601d0fb12d0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:17.026 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:17.025+0000 7ffb3f2c6b80 1 --2- 172.21.15.86:0/1341055268 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601d0fab8d0 0x5601d101f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:17.026 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:17.025+0000 7ffb3f2c6b80 1 --2- 172.21.15.86:0/1341055268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601d0fa0100 0x5601d0fb8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:17.026 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:17.025+0000 7ffb0ea00700 0 client.27253 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T18:06:17.026 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:17.025+0000 7ffb3f2c6b80 1 --2- 172.21.15.86:0/1341055268 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5601d0fa0a20 0x5601d0fb9140 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:17.026 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:17.025+0000 7ffb3f2c6b80 1 -- 172.21.15.86:0/1341055268 >> 172.21.15.86:0/1341055268 conn(0x5601d0f00640 msgr2=0x5601d0f9c450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:06:17.026 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:17.025+0000 7ffb3f2c6b80 1 -- 172.21.15.86:0/1341055268 shutdown_connections 2024-03-23T18:06:17.026 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:17.026+0000 7ffb3f2c6b80 1 -- 172.21.15.86:0/1341055268 wait complete. 2024-03-23T18:06:17.026 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:17.026+0000 7ffb3f2c6b80 20 client.27253 trim_cache size 0 max 16384 2024-03-23T18:06:17.027 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-03-23T18:06:17.027 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-03-23T18:06:17.027 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-03-23T18:06:17.027 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-03-23T18:06:17.028 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:17.027+0000 7ffb2b800700 1 -- 172.21.15.86:0/2086597105 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7ffb080a1460 msgr2=0x7ffb080a3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:06:17.028 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:17.027+0000 7ffb2b800700 1 --2- 172.21.15.86:0/2086597105 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7ffb080a1460 0x7ffb080a3920 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c000f80 tx=0x7ffb1c00b7f0 comp rx=0 tx=0).stop 2024-03-23T18:06:17.028 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:17.027+0000 7ffb2b800700 1 -- 172.21.15.86:0/2086597105 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffb24106030 msgr2=0x7ffb24199af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:06:17.028 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:17.027+0000 7ffb2b800700 1 --2- 172.21.15.86:0/2086597105 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffb24106030 0x7ffb24199af0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ffb24067c80 tx=0x7ffb1000dc30 comp rx=0 tx=0).stop 2024-03-23T18:06:17.028 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:17.027+0000 7ffb2b800700 1 -- 172.21.15.86:0/2086597105 shutdown_connections 2024-03-23T18:06:17.028 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:17.027+0000 7ffb2b800700 1 --2- 172.21.15.86:0/2086597105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb2419de40 0x7ffb2419e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:17.028 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:17.027+0000 7ffb2b800700 1 --2- 172.21.15.86:0/2086597105 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb24100f10 0x7ffb241995b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:17.029 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:17.027+0000 7ffb2b800700 1 --2- 172.21.15.86:0/2086597105 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7ffb080a1460 0x7ffb080a3920 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:17.029 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:17.027+0000 7ffb2b800700 1 --2- 172.21.15.86:0/2086597105 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffb24106030 0x7ffb24199af0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:17.029 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:17.027+0000 7ffb2b800700 1 -- 172.21.15.86:0/2086597105 >> 172.21.15.86:0/2086597105 conn(0x7ffb240fc770 msgr2=0x7ffb240feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:06:17.029 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:17.027+0000 7ffb2b800700 1 -- 172.21.15.86:0/2086597105 shutdown_connections 2024-03-23T18:06:17.029 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:17.027+0000 7ffb2b800700 1 -- 172.21.15.86:0/2086597105 wait complete. 2024-03-23T18:06:17.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:06:16 smithi092 ceph-mon[79089]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:06:17.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:06:16 smithi092 ceph-mon[79089]: osdmap e104: 12 total, 12 up, 12 in 2024-03-23T18:06:17.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:06:16 smithi152 ceph-mon[47949]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:06:17.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:06:16 smithi152 ceph-mon[47949]: osdmap e104: 12 total, 12 up, 12 in 2024-03-23T18:06:17.238 INFO:tasks.workunit.client.0.smithi086.stdout: 10 117411 38.40 MB/sec execute 41 sec latency 118.978 ms 2024-03-23T18:06:18.240 INFO:tasks.workunit.client.0.smithi086.stdout: 10 118281 38.64 MB/sec execute 42 sec latency 75.177 ms 2024-03-23T18:06:18.268 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:06:17 smithi086 ceph-mon[38221]: pgmap v493: 97 pgs: 97 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 721 op/s 2024-03-23T18:06:18.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:06:17 smithi092 ceph-mon[79089]: pgmap v493: 97 pgs: 97 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 721 op/s 2024-03-23T18:06:18.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:06:17 smithi152 ceph-mon[47949]: pgmap v493: 97 pgs: 97 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 721 op/s 2024-03-23T18:06:19.240 INFO:tasks.workunit.client.0.smithi086.stdout: 10 119066 38.70 MB/sec execute 43 sec latency 80.009 ms 2024-03-23T18:06:20.240 INFO:tasks.workunit.client.0.smithi086.stdout: 10 119878 38.60 MB/sec execute 44 sec latency 66.320 ms 2024-03-23T18:06:20.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:06:19 smithi086 ceph-mon[38221]: pgmap v494: 97 pgs: 97 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 721 op/s 2024-03-23T18:06:20.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:06:19 smithi092 ceph-mon[79089]: pgmap v494: 97 pgs: 97 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 721 op/s 2024-03-23T18:06:20.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:06:19 smithi152 ceph-mon[47949]: pgmap v494: 97 pgs: 97 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 721 op/s 2024-03-23T18:06:21.240 INFO:tasks.workunit.client.0.smithi086.stdout: 10 120451 38.38 MB/sec execute 45 sec latency 254.357 ms 2024-03-23T18:06:22.240 INFO:tasks.workunit.client.0.smithi086.stdout: 10 121283 38.82 MB/sec execute 46 sec latency 79.430 ms 2024-03-23T18:06:22.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:06:21 smithi086 ceph-mon[38221]: pgmap v495: 97 pgs: 97 active+clean; 1.2 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 16 MiB/s wr, 938 op/s 2024-03-23T18:06:22.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:06:21 smithi092 ceph-mon[79089]: pgmap v495: 97 pgs: 97 active+clean; 1.2 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 16 MiB/s wr, 938 op/s 2024-03-23T18:06:22.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:06:21 smithi152 ceph-mon[47949]: pgmap v495: 97 pgs: 97 active+clean; 1.2 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 16 MiB/s wr, 938 op/s 2024-03-23T18:06:23.240 INFO:tasks.workunit.client.0.smithi086.stdout: 10 122096 38.95 MB/sec execute 47 sec latency 52.120 ms 2024-03-23T18:06:24.240 INFO:tasks.workunit.client.0.smithi086.stdout: 10 122732 38.78 MB/sec execute 48 sec latency 275.878 ms 2024-03-23T18:06:24.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:06:23 smithi086 ceph-mon[38221]: pgmap v496: 97 pgs: 97 active+clean; 1.2 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 16 MiB/s wr, 938 op/s 2024-03-23T18:06:24.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:06:23 smithi092 ceph-mon[79089]: pgmap v496: 97 pgs: 97 active+clean; 1.2 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 16 MiB/s wr, 938 op/s 2024-03-23T18:06:24.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:06:23 smithi152 ceph-mon[47949]: pgmap v496: 97 pgs: 97 active+clean; 1.2 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 16 MiB/s wr, 938 op/s 2024-03-23T18:06:25.240 INFO:tasks.workunit.client.0.smithi086.stdout: 10 123392 38.55 MB/sec execute 49 sec latency 101.573 ms 2024-03-23T18:06:26.240 INFO:tasks.workunit.client.0.smithi086.stdout: 10 123789 38.22 MB/sec execute 50 sec latency 571.698 ms 2024-03-23T18:06:26.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:06:25 smithi086 ceph-mon[38221]: pgmap v497: 97 pgs: 97 active+clean; 1.2 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 15 MiB/s wr, 859 op/s 2024-03-23T18:06:26.309 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:06:25 smithi092 ceph-mon[79089]: pgmap v497: 97 pgs: 97 active+clean; 1.2 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 15 MiB/s wr, 859 op/s 2024-03-23T18:06:26.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:06:25 smithi152 ceph-mon[47949]: pgmap v497: 97 pgs: 97 active+clean; 1.2 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 15 MiB/s wr, 859 op/s 2024-03-23T18:06:27.241 INFO:tasks.workunit.client.0.smithi086.stdout: 10 124398 38.22 MB/sec execute 51 sec latency 203.521 ms 2024-03-23T18:06:28.240 INFO:tasks.workunit.client.0.smithi086.stdout: 10 125350 38.50 MB/sec execute 52 sec latency 84.034 ms 2024-03-23T18:06:28.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:06:27 smithi086 ceph-mon[38221]: pgmap v498: 97 pgs: 97 active+clean; 1.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 15 MiB/s wr, 880 op/s 2024-03-23T18:06:28.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:06:27 smithi092 ceph-mon[79089]: pgmap v498: 97 pgs: 97 active+clean; 1.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 15 MiB/s wr, 880 op/s 2024-03-23T18:06:28.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:06:27 smithi152 ceph-mon[47949]: pgmap v498: 97 pgs: 97 active+clean; 1.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 15 MiB/s wr, 880 op/s 2024-03-23T18:06:29.241 INFO:tasks.workunit.client.0.smithi086.stdout: 10 126145 38.61 MB/sec execute 53 sec latency 45.386 ms 2024-03-23T18:06:29.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:06:29 smithi086 ceph-mon[38221]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:06:29.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:06:29 smithi092 ceph-mon[79089]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:06:29.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:06:29 smithi152 ceph-mon[47949]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:06:30.241 INFO:tasks.workunit.client.0.smithi086.stdout: 10 127003 38.64 MB/sec execute 54 sec latency 47.789 ms 2024-03-23T18:06:30.265 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.265+0000 7f2d46400700 1 -- 172.21.15.86:0/417303037 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d4010b2a0 msgr2=0x7f2d4010d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:06:30.265 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.265+0000 7f2d46400700 1 --2- 172.21.15.86:0/417303037 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d4010b2a0 0x7f2d4010d760 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f2d3800b0a0 tx=0x7f2d3800b3b0 comp rx=0 tx=0).stop 2024-03-23T18:06:30.266 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.266+0000 7f2d46400700 1 -- 172.21.15.86:0/417303037 shutdown_connections 2024-03-23T18:06:30.266 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.266+0000 7f2d46400700 1 --2- 172.21.15.86:0/417303037 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d4010b2a0 0x7f2d4010d760 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:30.266 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.266+0000 7f2d46400700 1 --2- 172.21.15.86:0/417303037 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d40101e80 0x7f2d40102260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:30.266 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.266+0000 7f2d46400700 1 --2- 172.21.15.86:0/417303037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d401027a0 0x7f2d4010ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:30.266 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.266+0000 7f2d46400700 1 -- 172.21.15.86:0/417303037 >> 172.21.15.86:0/417303037 conn(0x7f2d400fb680 msgr2=0x7f2d400fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:06:30.267 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.267+0000 7f2d46400700 1 -- 172.21.15.86:0/417303037 shutdown_connections 2024-03-23T18:06:30.267 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.267+0000 7f2d46400700 1 -- 172.21.15.86:0/417303037 wait complete. 2024-03-23T18:06:30.267 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.268+0000 7f2d46400700 1 Processor -- start 2024-03-23T18:06:30.268 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.268+0000 7f2d46400700 1 -- start start 2024-03-23T18:06:30.270 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.270+0000 7f2d46400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d40101e80 0x7f2d40197170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:30.270 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.270+0000 7f2d46400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d401027a0 0x7f2d401976b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:30.270 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.270+0000 7f2d3f400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d40101e80 0x7f2d40197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:06:30.270 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.270+0000 7f2d3f400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d40101e80 0x7f2d40197170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:57772/0 (socket says 172.21.15.86:57772) 2024-03-23T18:06:30.271 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.270+0000 7f2d46400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d4010b2a0 0x7f2d4019ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:30.271 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.270+0000 7f2d46400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4019c040 con 0x7f2d4010b2a0 2024-03-23T18:06:30.271 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.270+0000 7f2d46400700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4019c1b0 con 0x7f2d40101e80 2024-03-23T18:06:30.271 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.270+0000 7f2d46400700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4019c370 con 0x7f2d401027a0 2024-03-23T18:06:30.271 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.270+0000 7f2d3f400700 1 -- 172.21.15.86:0/363563941 learned_addr learned my addr 172.21.15.86:0/363563941 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:06:30.271 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.271+0000 7f2d3fe00700 1 --2- 172.21.15.86:0/363563941 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d4010b2a0 0x7f2d4019ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:06:30.271 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.271+0000 7f2d3ea00700 1 --2- 172.21.15.86:0/363563941 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d401027a0 0x7f2d401976b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:06:30.272 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.271+0000 7f2d3f400700 1 -- 172.21.15.86:0/363563941 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d401027a0 msgr2=0x7f2d401976b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:06:30.272 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.271+0000 7f2d3f400700 1 --2- 172.21.15.86:0/363563941 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d401027a0 0x7f2d401976b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:30.272 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.271+0000 7f2d3f400700 1 -- 172.21.15.86:0/363563941 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d4010b2a0 msgr2=0x7f2d4019ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:06:30.272 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.271+0000 7f2d3f400700 1 --2- 172.21.15.86:0/363563941 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d4010b2a0 0x7f2d4019ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:30.272 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.271+0000 7f2d3f400700 1 -- 172.21.15.86:0/363563941 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d38009d00 con 0x7f2d40101e80 2024-03-23T18:06:30.272 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.271+0000 7f2d3ea00700 1 --2- 172.21.15.86:0/363563941 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d401027a0 0x7f2d401976b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T18:06:30.272 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.272+0000 7f2d3f400700 1 --2- 172.21.15.86:0/363563941 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d40101e80 0x7f2d40197170 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f2d3000e9a0 tx=0x7f2d3000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:06:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.275+0000 7f2d37e00700 1 -- 172.21.15.86:0/363563941 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d30009790 con 0x7f2d40101e80 2024-03-23T18:06:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.275+0000 7f2d46400700 1 -- 172.21.15.86:0/363563941 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d4019c650 con 0x7f2d40101e80 2024-03-23T18:06:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.275+0000 7f2d37e00700 1 -- 172.21.15.86:0/363563941 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d3000cbd0 con 0x7f2d40101e80 2024-03-23T18:06:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.275+0000 7f2d46400700 1 -- 172.21.15.86:0/363563941 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d400ffbe0 con 0x7f2d40101e80 2024-03-23T18:06:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.275+0000 7f2d37e00700 1 -- 172.21.15.86:0/363563941 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d30009790 con 0x7f2d40101e80 2024-03-23T18:06:30.281 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.282+0000 7f2d37e00700 1 -- 172.21.15.86:0/363563941 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f2d3000cd40 con 0x7f2d40101e80 2024-03-23T18:06:30.284 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.284+0000 7f2d37e00700 1 --2- 172.21.15.86:0/363563941 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f2d240a12f0 0x7f2d240a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:30.284 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.284+0000 7f2d37e00700 1 -- 172.21.15.86:0/363563941 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f2d30014070 con 0x7f2d40101e80 2024-03-23T18:06:30.285 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.285+0000 7f2d59eb9b80 1 Processor -- start 2024-03-23T18:06:30.285 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.285+0000 7f2d3ea00700 1 --2- 172.21.15.86:0/363563941 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f2d240a12f0 0x7f2d240a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:06:30.285 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.285+0000 7f2d59eb9b80 1 -- start start 2024-03-23T18:06:30.285 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.285+0000 7f2d59eb9b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636e1ff8910 0x5636e1ff8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:30.285 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.286+0000 7f2d3ea00700 1 --2- 172.21.15.86:0/363563941 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f2d240a12f0 0x7f2d240a37b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2d40198670 tx=0x7f2d2c005d80 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:06:30.286 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.286+0000 7f2d59eb9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636e1ff9230 0x5636e2003ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:30.287 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.286+0000 7f2d59eb9b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5636e20040e0 0x5636e20064d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:30.287 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.286+0000 7f2d59eb9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636e1f49930 con 0x5636e1ff9230 2024-03-23T18:06:30.287 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.286+0000 7f2d59eb9b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636e1f472f0 con 0x5636e1ff8910 2024-03-23T18:06:30.287 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.286+0000 7f2d59eb9b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636e1f42be0 con 0x5636e20040e0 2024-03-23T18:06:30.287 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.287+0000 7f2d3ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636e1ff9230 0x5636e2003ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:06:30.287 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.287+0000 7f2d3ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636e1ff9230 0x5636e2003ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39826/0 (socket says 172.21.15.86:39826) 2024-03-23T18:06:30.287 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.287+0000 7f2d3ea00700 1 -- 172.21.15.86:0/1086493855 learned_addr learned my addr 172.21.15.86:0/1086493855 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:06:30.287 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.287+0000 7f2d3fe00700 1 --2- 172.21.15.86:0/1086493855 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5636e20040e0 0x5636e20064d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:06:30.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.287+0000 7f2d3f400700 1 --2- 172.21.15.86:0/1086493855 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636e1ff8910 0x5636e1ff8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:06:30.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.288+0000 7f2d3fe00700 1 -- 172.21.15.86:0/1086493855 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636e1ff8910 msgr2=0x5636e1ff8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:06:30.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.288+0000 7f2d3fe00700 1 --2- 172.21.15.86:0/1086493855 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636e1ff8910 0x5636e1ff8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:30.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.288+0000 7f2d3fe00700 1 -- 172.21.15.86:0/1086493855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636e1ff9230 msgr2=0x5636e2003ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:06:30.290 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.290+0000 7f2d3fe00700 1 --2- 172.21.15.86:0/1086493855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636e1ff9230 0x5636e2003ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:30.290 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.290+0000 7f2d3fe00700 1 -- 172.21.15.86:0/1086493855 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5636e1dc4350 con 0x5636e20040e0 2024-03-23T18:06:30.290 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.290+0000 7f2d3ea00700 1 --2- 172.21.15.86:0/1086493855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636e1ff9230 0x5636e2003ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T18:06:30.290 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.290+0000 7f2d3fe00700 1 --2- 172.21.15.86:0/1086493855 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5636e20040e0 0x5636e20064d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2d38009fd0 tx=0x7f2d38009750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=500bf8a643b46c56 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:06:30.292 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.291+0000 7f2d34c00700 1 -- 172.21.15.86:0/1086493855 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d3800e040 con 0x5636e20040e0 2024-03-23T18:06:30.292 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.291+0000 7f2d34c00700 1 -- 172.21.15.86:0/1086493855 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d3800bd30 con 0x5636e20040e0 2024-03-23T18:06:30.292 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.291+0000 7f2d34c00700 1 -- 172.21.15.86:0/1086493855 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d3801c6a0 con 0x5636e20040e0 2024-03-23T18:06:30.292 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.292+0000 7f2d59eb9b80 1 -- 172.21.15.86:0/1086493855 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5636e20040e0 msgr2=0x5636e20064d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:06:30.292 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.292+0000 7f2d59eb9b80 1 --2- 172.21.15.86:0/1086493855 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5636e20040e0 0x5636e20064d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2d38009fd0 tx=0x7f2d38009750 comp rx=0 tx=0).stop 2024-03-23T18:06:30.292 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.292+0000 7f2d59eb9b80 1 -- 172.21.15.86:0/1086493855 shutdown_connections 2024-03-23T18:06:30.292 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.292+0000 7f2d59eb9b80 1 --2- 172.21.15.86:0/1086493855 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5636e20040e0 0x5636e20064d0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:30.292 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.292+0000 7f2d59eb9b80 1 --2- 172.21.15.86:0/1086493855 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636e1ff8910 0x5636e1ff8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:30.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.292+0000 7f2d59eb9b80 1 --2- 172.21.15.86:0/1086493855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636e1ff9230 0x5636e2003ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:30.293 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.292+0000 7f2d59eb9b80 1 -- 172.21.15.86:0/1086493855 >> 172.21.15.86:0/1086493855 conn(0x5636e1f50640 msgr2=0x5636e1fec520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:06:30.294 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.294+0000 7f2d59eb9b80 1 -- 172.21.15.86:0/1086493855 shutdown_connections 2024-03-23T18:06:30.295 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.294+0000 7f2d59eb9b80 1 -- 172.21.15.86:0/1086493855 wait complete. 2024-03-23T18:06:30.295 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.295+0000 7f2d59eb9b80 1 -- start start 2024-03-23T18:06:30.295 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.295+0000 7f2d59eb9b80 1 Processor -- start 2024-03-23T18:06:30.296 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.296+0000 7f2d59eb9b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5636e1ff8910 0x5636e2003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:30.296 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.296+0000 7f2d59eb9b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636e1ff9230 0x5636e2000da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:30.296 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.296+0000 7f2d59eb9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636e20040e0 0x5636e20013d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:30.296 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.296+0000 7f2d59eb9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636e1eded20 con 0x5636e20040e0 2024-03-23T18:06:30.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.296+0000 7f2d59eb9b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636e1ede9c0 con 0x5636e1ff9230 2024-03-23T18:06:30.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.296+0000 7f2d59eb9b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636e1dc6640 con 0x5636e1ff8910 2024-03-23T18:06:30.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.296+0000 7f2d59eb9b80 10 client.0 _ll_register_callbacks cb 0x5636e1ee8410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T18:06:30.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.296+0000 7f2d3f400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5636e1ff8910 0x5636e2003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:06:30.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.296+0000 7f2d3f400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5636e1ff8910 0x5636e2003a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:36342/0 (socket says 172.21.15.86:36342) 2024-03-23T18:06:30.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.296+0000 7f2d3f400700 1 -- 172.21.15.86:0/1844662833 learned_addr learned my addr 172.21.15.86:0/1844662833 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:06:30.298 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.297+0000 7f2d3f400700 1 -- 172.21.15.86:0/1844662833 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636e1ff9230 msgr2=0x5636e2000da0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-23T18:06:30.299 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.297+0000 7f2d3f400700 1 --2- 172.21.15.86:0/1844662833 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636e1ff9230 0x5636e2000da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:30.299 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.297+0000 7f2d3f400700 1 -- 172.21.15.86:0/1844662833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636e20040e0 msgr2=0x5636e20013d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:06:30.299 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.297+0000 7f2d3f400700 1 --2- 172.21.15.86:0/1844662833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636e20040e0 0x5636e20013d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:30.299 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.297+0000 7f2d3f400700 1 -- 172.21.15.86:0/1844662833 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d4010e420 con 0x5636e1ff8910 2024-03-23T18:06:30.299 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.297+0000 7f2d3f400700 1 --2- 172.21.15.86:0/1844662833 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5636e1ff8910 0x5636e2003a60 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2d3000f010 tx=0x7f2d30099cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53d8b23a1fd88f5e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:06:30.299 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.297+0000 7f2d29600700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-03-23T18:06:30.299 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.298+0000 7f2d29600700 1 -- 172.21.15.86:0/1844662833 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d300991b0 con 0x5636e1ff8910 2024-03-23T18:06:30.299 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.298+0000 7f2d29600700 1 -- 172.21.15.86:0/1844662833 <== mon.1 v2:172.21.15.152:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f2d30099310 con 0x5636e1ff8910 2024-03-23T18:06:30.299 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.298+0000 7f2d59eb9b80 1 -- 172.21.15.86:0/1844662833 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5636e1f03320 con 0x5636e1ff8910 2024-03-23T18:06:30.299 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.298+0000 7f2d29600700 1 -- 172.21.15.86:0/1844662833 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d300a1cb0 con 0x5636e1ff8910 2024-03-23T18:06:30.299 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.298+0000 7f2d59eb9b80 10 client.27265 fetch_fsmap learned FSMap version 75 2024-03-23T18:06:30.299 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.298+0000 7f2d59eb9b80 1 -- 172.21.15.86:0/1844662833 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=4+}) v3 -- 0x5636e1d70b00 con 0x5636e1ff8910 2024-03-23T18:06:30.300 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.298+0000 7f2d29600700 1 -- 172.21.15.86:0/1844662833 <== mon.1 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d300b6070 con 0x5636e1ff8910 2024-03-23T18:06:30.300 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.298+0000 7f2d29600700 1 -- 172.21.15.86:0/1844662833 <== mon.1 v2:172.21.15.152:3300/0 5 ==== fsmap(e 75) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f2d30098c20 con 0x5636e1ff8910 2024-03-23T18:06:30.300 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.298+0000 7f2d59eb9b80 10 client.27265 fetch_fsmap finished waiting for FSMap version 75 2024-03-23T18:06:30.300 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.298+0000 7f2d59eb9b80 10 client.27265 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T18:06:30.300 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.298+0000 7f2d59eb9b80 20 client.27265 populate_metadata read hostname 'smithi086' 2024-03-23T18:06:30.300 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.298+0000 7f2d59eb9b80 1 --2- 172.21.15.86:0/1844662833 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5636e2013db0 0x5636e20161d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:30.300 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.298+0000 7f2d59eb9b80 4 client.27265 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T18:06:30.300 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.298+0000 7f2d59eb9b80 1 -- 172.21.15.86:0/1844662833 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5636e1f3b6a0 con 0x5636e2013db0 2024-03-23T18:06:30.300 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.299+0000 7f2d3ea00700 1 --2- 172.21.15.86:0/1844662833 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5636e2013db0 0x5636e20161d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:06:30.300 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.299+0000 7f2d3ea00700 1 --2- 172.21.15.86:0/1844662833 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5636e2013db0 0x5636e20161d0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c18ff3689d2e2cbe server_cookie=7407c8a785ef762e in_seq=0 out_seq=0 2024-03-23T18:06:30.301 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.299+0000 7f2d29600700 10 client.27265 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T18:06:30.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.302+0000 7f2d29600700 1 -- 172.21.15.86:0/1844662833 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5636e1f3b6a0 con 0x5636e2013db0 2024-03-23T18:06:30.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.302+0000 7f2d29600700 10 client.27265 handle_command_reply: tid=0 2024-03-23T18:06:30.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.302+0000 7f2d59eb9b80 1 client.27265 shutdown 2024-03-23T18:06:30.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.303+0000 7f2d59eb9b80 1 -- 172.21.15.86:0/1844662833 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5636e1ff8910 msgr2=0x5636e2003a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:06:30.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.303+0000 7f2d59eb9b80 1 --2- 172.21.15.86:0/1844662833 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5636e1ff8910 0x5636e2003a60 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2d3000f010 tx=0x7f2d30099cc0 comp rx=0 tx=0).stop 2024-03-23T18:06:30.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.303+0000 7f2d59eb9b80 1 -- 172.21.15.86:0/1844662833 shutdown_connections 2024-03-23T18:06:30.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.303+0000 7f2d59eb9b80 1 --2- 172.21.15.86:0/1844662833 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5636e2013db0 0x5636e20161d0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:30.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.303+0000 7f2d59eb9b80 1 --2- 172.21.15.86:0/1844662833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636e20040e0 0x5636e20013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:30.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.303+0000 7f2d59eb9b80 1 --2- 172.21.15.86:0/1844662833 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5636e1ff8910 0x5636e2003a60 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:30.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.303+0000 7f2d59eb9b80 1 --2- 172.21.15.86:0/1844662833 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636e1ff9230 0x5636e2000da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:30.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.303+0000 7f2d59eb9b80 1 -- 172.21.15.86:0/1844662833 >> 172.21.15.86:0/1844662833 conn(0x5636e1f50640 msgr2=0x5636e1ffa8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:06:30.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.303+0000 7f2d29600700 0 client.27265 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T18:06:30.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.303+0000 7f2d59eb9b80 1 -- 172.21.15.86:0/1844662833 shutdown_connections 2024-03-23T18:06:30.304 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.303+0000 7f2d59eb9b80 1 -- 172.21.15.86:0/1844662833 wait complete. 2024-03-23T18:06:30.304 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.303+0000 7f2d59eb9b80 20 client.27265 trim_cache size 0 max 16384 2024-03-23T18:06:30.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:06:30 smithi092 ceph-mon[79089]: pgmap v499: 97 pgs: 97 active+clean; 1.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 783 op/s 2024-03-23T18:06:30.327 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.327+0000 7f2d59eb9b80 1 Processor -- start 2024-03-23T18:06:30.327 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.327+0000 7f2d59eb9b80 1 -- start start 2024-03-23T18:06:30.328 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.328+0000 7f2d59eb9b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636e1ff8910 0x5636e1ffb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:30.328 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.328+0000 7f2d59eb9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636e1ff9230 0x5636e1fffcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:30.328 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.328+0000 7f2d59eb9b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5636e20040e0 0x5636e1fec5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:30.328 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.328+0000 7f2d59eb9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636e1ffa190 con 0x5636e1ff9230 2024-03-23T18:06:30.328 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.328+0000 7f2d59eb9b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636e1fe8400 con 0x5636e1ff8910 2024-03-23T18:06:30.328 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.328+0000 7f2d59eb9b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636e1ff9870 con 0x5636e20040e0 2024-03-23T18:06:30.329 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.328+0000 7f2d3f400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636e1ff8910 0x5636e1ffb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:06:30.329 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.328+0000 7f2d3f400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636e1ff8910 0x5636e1ffb860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:57800/0 (socket says 172.21.15.86:57800) 2024-03-23T18:06:30.329 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.328+0000 7f2d3f400700 1 -- 172.21.15.86:0/7197948 learned_addr learned my addr 172.21.15.86:0/7197948 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:06:30.329 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.328+0000 7f2d3fe00700 1 --2- 172.21.15.86:0/7197948 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5636e20040e0 0x5636e1fec5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:06:30.329 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.328+0000 7f2d3ea00700 1 --2- 172.21.15.86:0/7197948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636e1ff9230 0x5636e1fffcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:06:30.329 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.329+0000 7f2d3fe00700 1 -- 172.21.15.86:0/7197948 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636e1ff8910 msgr2=0x5636e1ffb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:06:30.329 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.329+0000 7f2d3fe00700 1 --2- 172.21.15.86:0/7197948 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636e1ff8910 0x5636e1ffb860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:30.329 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.329+0000 7f2d3fe00700 1 -- 172.21.15.86:0/7197948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636e1ff9230 msgr2=0x5636e1fffcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:06:30.329 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.329+0000 7f2d3fe00700 1 --2- 172.21.15.86:0/7197948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636e1ff9230 0x5636e1fffcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:30.329 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.329+0000 7f2d3fe00700 1 -- 172.21.15.86:0/7197948 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5636e1f08e40 con 0x5636e20040e0 2024-03-23T18:06:30.329 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.329+0000 7f2d3f400700 1 --2- 172.21.15.86:0/7197948 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636e1ff8910 0x5636e1ffb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T18:06:30.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.329+0000 7f2d3fe00700 1 --2- 172.21.15.86:0/7197948 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5636e20040e0 0x5636e1fec5e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x5636e1ffa3a0 tx=0x7f2d38000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17b06a6a9f739c3c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:06:30.331 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.329+0000 7f2d35600700 1 -- 172.21.15.86:0/7197948 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d3800e040 con 0x5636e20040e0 2024-03-23T18:06:30.331 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.329+0000 7f2d35600700 1 -- 172.21.15.86:0/7197948 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d380074a0 con 0x5636e20040e0 2024-03-23T18:06:30.331 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.329+0000 7f2d35600700 1 -- 172.21.15.86:0/7197948 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d38025870 con 0x5636e20040e0 2024-03-23T18:06:30.331 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.330+0000 7f2d59eb9b80 1 -- 172.21.15.86:0/7197948 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5636e20040e0 msgr2=0x5636e1fec5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:06:30.331 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.330+0000 7f2d59eb9b80 1 --2- 172.21.15.86:0/7197948 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5636e20040e0 0x5636e1fec5e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x5636e1ffa3a0 tx=0x7f2d38000f40 comp rx=0 tx=0).stop 2024-03-23T18:06:30.331 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.330+0000 7f2d59eb9b80 1 -- 172.21.15.86:0/7197948 shutdown_connections 2024-03-23T18:06:30.331 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.330+0000 7f2d59eb9b80 1 --2- 172.21.15.86:0/7197948 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5636e20040e0 0x5636e1fec5e0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:30.331 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.330+0000 7f2d59eb9b80 1 --2- 172.21.15.86:0/7197948 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636e1ff8910 0x5636e1ffb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:30.331 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.330+0000 7f2d59eb9b80 1 --2- 172.21.15.86:0/7197948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636e1ff9230 0x5636e1fffcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:30.331 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.330+0000 7f2d59eb9b80 1 -- 172.21.15.86:0/7197948 >> 172.21.15.86:0/7197948 conn(0x5636e1f50640 msgr2=0x5636e1fe53e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:06:30.331 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.330+0000 7f2d59eb9b80 1 -- 172.21.15.86:0/7197948 shutdown_connections 2024-03-23T18:06:30.331 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.331+0000 7f2d59eb9b80 1 -- 172.21.15.86:0/7197948 wait complete. 2024-03-23T18:06:30.331 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.331+0000 7f2d59eb9b80 1 -- start start 2024-03-23T18:06:30.332 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.331+0000 7f2d59eb9b80 1 Processor -- start 2024-03-23T18:06:30.334 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.332+0000 7f2d59eb9b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636e1ff8910 0x5636e200ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:30.334 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.332+0000 7f2d59eb9b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5636e1ff9230 0x5636e2009140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:30.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.332+0000 7f2d59eb9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636e20040e0 0x5636e2009680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:30.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.332+0000 7f2d59eb9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636e2009dc0 con 0x5636e20040e0 2024-03-23T18:06:30.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.332+0000 7f2d59eb9b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636e2009f30 con 0x5636e1ff8910 2024-03-23T18:06:30.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.332+0000 7f2d59eb9b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636e2065920 con 0x5636e1ff9230 2024-03-23T18:06:30.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.332+0000 7f2d59eb9b80 10 client.0 _ll_register_callbacks cb 0x5636e1ee8410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T18:06:30.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.332+0000 7f2d3f400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636e1ff8910 0x5636e200ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:06:30.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.332+0000 7f2d3f400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636e1ff8910 0x5636e200ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:57814/0 (socket says 172.21.15.86:57814) 2024-03-23T18:06:30.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.332+0000 7f2d3f400700 1 -- 172.21.15.86:0/658144589 learned_addr learned my addr 172.21.15.86:0/658144589 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:06:30.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.332+0000 7f2d3ea00700 1 --2- 172.21.15.86:0/658144589 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5636e1ff9230 0x5636e2009140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:06:30.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.332+0000 7f2d3fe00700 1 --2- 172.21.15.86:0/658144589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636e20040e0 0x5636e2009680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:06:30.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.333+0000 7f2d3f400700 1 -- 172.21.15.86:0/658144589 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5636e1ff9230 msgr2=0x5636e2009140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:06:30.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.333+0000 7f2d3f400700 1 --2- 172.21.15.86:0/658144589 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5636e1ff9230 0x5636e2009140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:30.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.333+0000 7f2d3f400700 1 -- 172.21.15.86:0/658144589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636e20040e0 msgr2=0x5636e2009680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:06:30.336 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.333+0000 7f2d3f400700 1 --2- 172.21.15.86:0/658144589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636e20040e0 0x5636e2009680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:30.336 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.333+0000 7f2d3f400700 1 -- 172.21.15.86:0/658144589 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5636e1d70b00 con 0x5636e1ff8910 2024-03-23T18:06:30.336 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.333+0000 7f2d3fe00700 1 --2- 172.21.15.86:0/658144589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636e20040e0 0x5636e2009680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T18:06:30.336 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.334+0000 7f2d3f400700 1 --2- 172.21.15.86:0/658144589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636e1ff8910 0x5636e200ab90 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f2d3000e970 tx=0x7f2d3009daf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74821bfe41f112d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:06:30.336 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.334+0000 7f2d29600700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T18:06:30.336 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.334+0000 7f2d29600700 1 -- 172.21.15.86:0/658144589 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d300a1cb0 con 0x5636e1ff8910 2024-03-23T18:06:30.336 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.334+0000 7f2d29600700 1 -- 172.21.15.86:0/658144589 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f2d300a1e10 con 0x5636e1ff8910 2024-03-23T18:06:30.336 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.334+0000 7f2d59eb9b80 1 -- 172.21.15.86:0/658144589 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2d3800bd30 con 0x5636e1ff8910 2024-03-23T18:06:30.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:06:30 smithi152 ceph-mon[47949]: pgmap v499: 97 pgs: 97 active+clean; 1.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 783 op/s 2024-03-23T18:06:30.337 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.335+0000 7f2d29600700 1 -- 172.21.15.86:0/658144589 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d300991b0 con 0x5636e1ff8910 2024-03-23T18:06:30.337 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.335+0000 7f2d59eb9b80 10 client.27284 fetch_fsmap learned FSMap version 75 2024-03-23T18:06:30.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.335+0000 7f2d59eb9b80 1 -- 172.21.15.86:0/658144589 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=4+}) v3 -- 0x5636e1dc4350 con 0x5636e1ff8910 2024-03-23T18:06:30.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.335+0000 7f2d29600700 1 -- 172.21.15.86:0/658144589 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d30064080 con 0x5636e1ff8910 2024-03-23T18:06:30.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.335+0000 7f2d29600700 1 -- 172.21.15.86:0/658144589 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 75) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f2d300caae0 con 0x5636e1ff8910 2024-03-23T18:06:30.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.335+0000 7f2d59eb9b80 10 client.27284 fetch_fsmap finished waiting for FSMap version 75 2024-03-23T18:06:30.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.335+0000 7f2d59eb9b80 10 client.27284 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T18:06:30.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.335+0000 7f2d59eb9b80 20 client.27284 populate_metadata read hostname 'smithi086' 2024-03-23T18:06:30.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.335+0000 7f2d59eb9b80 1 --2- 172.21.15.86:0/658144589 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5636e2066dc0 0x5636e20691e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:30.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.335+0000 7f2d59eb9b80 4 client.27284 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T18:06:30.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.335+0000 7f2d59eb9b80 1 -- 172.21.15.86:0/658144589 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5636e1d91c90 con 0x5636e2066dc0 2024-03-23T18:06:30.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.336+0000 7f2d3ea00700 1 --2- 172.21.15.86:0/658144589 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5636e2066dc0 0x5636e20691e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:06:30.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.337+0000 7f2d29600700 10 client.27284 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T18:06:30.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.337+0000 7f2d3ea00700 1 --2- 172.21.15.86:0/658144589 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5636e2066dc0 0x5636e20691e0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3cac66fbc2c7895 server_cookie=de9da66305ba781a in_seq=0 out_seq=0 2024-03-23T18:06:30.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:06:30 smithi086 ceph-mon[38221]: pgmap v499: 97 pgs: 97 active+clean; 1.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 783 op/s 2024-03-23T18:06:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.595+0000 7f2d29600700 1 -- 172.21.15.86:0/658144589 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5636e1d91c90 con 0x5636e2066dc0 2024-03-23T18:06:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.595+0000 7f2d29600700 10 client.27284 handle_command_reply: tid=0 2024-03-23T18:06:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.595+0000 7f2d59eb9b80 1 client.27284 shutdown 2024-03-23T18:06:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.596+0000 7f2d59eb9b80 1 -- 172.21.15.86:0/658144589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636e1ff8910 msgr2=0x5636e200ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:06:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.596+0000 7f2d59eb9b80 1 --2- 172.21.15.86:0/658144589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636e1ff8910 0x5636e200ab90 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f2d3000e970 tx=0x7f2d3009daf0 comp rx=0 tx=0).stop 2024-03-23T18:06:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.597+0000 7f2d59eb9b80 1 -- 172.21.15.86:0/658144589 shutdown_connections 2024-03-23T18:06:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.597+0000 7f2d59eb9b80 1 --2- 172.21.15.86:0/658144589 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5636e2066dc0 0x5636e20691e0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.597+0000 7f2d59eb9b80 1 --2- 172.21.15.86:0/658144589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636e20040e0 0x5636e2009680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.597+0000 7f2d59eb9b80 1 --2- 172.21.15.86:0/658144589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636e1ff8910 0x5636e200ab90 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.597+0000 7f2d59eb9b80 1 --2- 172.21.15.86:0/658144589 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5636e1ff9230 0x5636e2009140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.597+0000 7f2d59eb9b80 1 -- 172.21.15.86:0/658144589 >> 172.21.15.86:0/658144589 conn(0x5636e1f50640 msgr2=0x5636e1fec450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:06:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.597+0000 7f2d29600700 0 client.27284 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T18:06:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.597+0000 7f2d59eb9b80 1 -- 172.21.15.86:0/658144589 shutdown_connections 2024-03-23T18:06:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.597+0000 7f2d59eb9b80 1 -- 172.21.15.86:0/658144589 wait complete. 2024-03-23T18:06:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-03-23T18:06:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-03-23T18:06:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-03-23T18:06:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-03-23T18:06:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.597+0000 7f2d59eb9b80 20 client.27284 trim_cache size 0 max 16384 2024-03-23T18:06:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.599+0000 7f2d46400700 1 -- 172.21.15.86:0/363563941 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f2d240a12f0 msgr2=0x7f2d240a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:06:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.599+0000 7f2d46400700 1 --2- 172.21.15.86:0/363563941 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f2d240a12f0 0x7f2d240a37b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2d40198670 tx=0x7f2d2c005d80 comp rx=0 tx=0).stop 2024-03-23T18:06:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.599+0000 7f2d46400700 1 -- 172.21.15.86:0/363563941 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d40101e80 msgr2=0x7f2d40197170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:06:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.599+0000 7f2d46400700 1 --2- 172.21.15.86:0/363563941 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d40101e80 0x7f2d40197170 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f2d3000e9a0 tx=0x7f2d3000ecb0 comp rx=0 tx=0).stop 2024-03-23T18:06:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.599+0000 7f2d46400700 1 -- 172.21.15.86:0/363563941 shutdown_connections 2024-03-23T18:06:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.599+0000 7f2d46400700 1 --2- 172.21.15.86:0/363563941 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d4010b2a0 0x7f2d4019ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.599+0000 7f2d46400700 1 --2- 172.21.15.86:0/363563941 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d40101e80 0x7f2d40197170 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.599+0000 7f2d46400700 1 --2- 172.21.15.86:0/363563941 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f2d240a12f0 0x7f2d240a37b0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.599+0000 7f2d46400700 1 --2- 172.21.15.86:0/363563941 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d401027a0 0x7f2d401976b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.599+0000 7f2d46400700 1 -- 172.21.15.86:0/363563941 >> 172.21.15.86:0/363563941 conn(0x7f2d400fb680 msgr2=0x7f2d400fc390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:06:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.600+0000 7f2d46400700 1 -- 172.21.15.86:0/363563941 shutdown_connections 2024-03-23T18:06:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:30.600+0000 7f2d46400700 1 -- 172.21.15.86:0/363563941 wait complete. 2024-03-23T18:06:31.241 INFO:tasks.workunit.client.0.smithi086.stdout: 10 127841 38.79 MB/sec execute 55 sec latency 145.177 ms 2024-03-23T18:06:32.241 INFO:tasks.workunit.client.0.smithi086.stdout: 10 127901 38.15 MB/sec execute 56 sec latency 955.476 ms 2024-03-23T18:06:32.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:06:32 smithi092 ceph-mon[79089]: pgmap v500: 97 pgs: 97 active+clean; 1.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 22 MiB/s wr, 1.22k op/s 2024-03-23T18:06:32.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:06:32 smithi152 ceph-mon[47949]: pgmap v500: 97 pgs: 97 active+clean; 1.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 22 MiB/s wr, 1.22k op/s 2024-03-23T18:06:32.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:06:32 smithi086 ceph-mon[38221]: pgmap v500: 97 pgs: 97 active+clean; 1.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 22 MiB/s wr, 1.22k op/s 2024-03-23T18:06:33.241 INFO:tasks.workunit.client.0.smithi086.stdout: 10 128331 38.04 MB/sec execute 57 sec latency 398.744 ms 2024-03-23T18:06:34.241 INFO:tasks.workunit.client.0.smithi086.stdout: 10 128773 37.82 MB/sec execute 58 sec latency 120.641 ms 2024-03-23T18:06:34.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:06:34 smithi092 ceph-mon[79089]: pgmap v501: 97 pgs: 97 active+clean; 1.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 16 MiB/s wr, 795 op/s 2024-03-23T18:06:34.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:06:34 smithi152 ceph-mon[47949]: pgmap v501: 97 pgs: 97 active+clean; 1.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 16 MiB/s wr, 795 op/s 2024-03-23T18:06:34.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:06:34 smithi086 ceph-mon[38221]: pgmap v501: 97 pgs: 97 active+clean; 1.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 16 MiB/s wr, 795 op/s 2024-03-23T18:06:35.241 INFO:tasks.workunit.client.0.smithi086.stdout: 10 129576 37.65 MB/sec execute 59 sec latency 57.377 ms 2024-03-23T18:06:36.241 INFO:tasks.workunit.client.0.smithi086.stdout: 10 130529 37.92 MB/sec execute 60 sec latency 60.781 ms 2024-03-23T18:06:36.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:06:36 smithi092 ceph-mon[79089]: pgmap v502: 97 pgs: 97 active+clean; 1.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 21 MiB/s wr, 1.09k op/s 2024-03-23T18:06:36.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:06:36 smithi152 ceph-mon[47949]: pgmap v502: 97 pgs: 97 active+clean; 1.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 21 MiB/s wr, 1.09k op/s 2024-03-23T18:06:36.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:06:36 smithi086 ceph-mon[38221]: pgmap v502: 97 pgs: 97 active+clean; 1.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 21 MiB/s wr, 1.09k op/s 2024-03-23T18:06:37.241 INFO:tasks.workunit.client.0.smithi086.stdout: 10 131474 38.27 MB/sec execute 61 sec latency 46.972 ms 2024-03-23T18:06:37.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:06:37 smithi086 ceph-mon[38221]: pgmap v503: 97 pgs: 97 active+clean; 1.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 26 MiB/s wr, 1.25k op/s 2024-03-23T18:06:37.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:06:37 smithi092 ceph-mon[79089]: pgmap v503: 97 pgs: 97 active+clean; 1.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 26 MiB/s wr, 1.25k op/s 2024-03-23T18:06:37.585 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:06:37 smithi152 ceph-mon[47949]: pgmap v503: 97 pgs: 97 active+clean; 1.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 26 MiB/s wr, 1.25k op/s 2024-03-23T18:06:38.241 INFO:tasks.workunit.client.0.smithi086.stdout: 10 132213 38.27 MB/sec execute 62 sec latency 55.042 ms 2024-03-23T18:06:39.241 INFO:tasks.workunit.client.0.smithi086.stdout: 10 133062 38.46 MB/sec execute 63 sec latency 91.221 ms 2024-03-23T18:06:40.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:06:39 smithi086 ceph-mon[38221]: pgmap v504: 97 pgs: 97 active+clean; 1.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 22 MiB/s wr, 1.07k op/s 2024-03-23T18:06:40.060 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:06:39 smithi092 ceph-mon[79089]: pgmap v504: 97 pgs: 97 active+clean; 1.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 22 MiB/s wr, 1.07k op/s 2024-03-23T18:06:40.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:06:39 smithi152 ceph-mon[47949]: pgmap v504: 97 pgs: 97 active+clean; 1.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 22 MiB/s wr, 1.07k op/s 2024-03-23T18:06:40.241 INFO:tasks.workunit.client.0.smithi086.stdout: 10 133876 38.33 MB/sec execute 64 sec latency 46.937 ms 2024-03-23T18:06:41.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:06:40 smithi086 ceph-mon[38221]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:06:41.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:06:40 smithi092 ceph-mon[79089]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:06:41.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:06:40 smithi152 ceph-mon[47949]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:06:41.241 INFO:tasks.workunit.client.0.smithi086.stdout: 10 134882 38.72 MB/sec execute 65 sec latency 51.198 ms 2024-03-23T18:06:42.241 INFO:tasks.workunit.client.0.smithi086.stdout: 10 135799 38.86 MB/sec execute 66 sec latency 55.868 ms 2024-03-23T18:06:42.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:06:41 smithi086 ceph-mon[38221]: pgmap v505: 97 pgs: 97 active+clean; 1.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 26 MiB/s wr, 1.34k op/s 2024-03-23T18:06:42.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:06:41 smithi086 ceph-mon[38221]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:06:42.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:06:41 smithi092 ceph-mon[79089]: pgmap v505: 97 pgs: 97 active+clean; 1.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 26 MiB/s wr, 1.34k op/s 2024-03-23T18:06:42.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:06:41 smithi092 ceph-mon[79089]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:06:42.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:06:41 smithi152 ceph-mon[47949]: pgmap v505: 97 pgs: 97 active+clean; 1.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 26 MiB/s wr, 1.34k op/s 2024-03-23T18:06:42.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:06:41 smithi152 ceph-mon[47949]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:06:43.241 INFO:tasks.workunit.client.0.smithi086.stdout: 10 136529 38.91 MB/sec execute 67 sec latency 56.109 ms 2024-03-23T18:06:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.812+0000 7f521ea00700 1 -- 172.21.15.86:0/1829907060 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f521806d0c0 msgr2=0x7f5218119f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:06:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.812+0000 7f521ea00700 1 --2- 172.21.15.86:0/1829907060 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f521806d0c0 0x7f5218119f10 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f5204009990 tx=0x7f5204009ca0 comp rx=0 tx=0).stop 2024-03-23T18:06:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.813+0000 7f521ea00700 1 -- 172.21.15.86:0/1829907060 shutdown_connections 2024-03-23T18:06:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.813+0000 7f521ea00700 1 --2- 172.21.15.86:0/1829907060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f521811a450 0x7f521811c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.813+0000 7f521ea00700 1 --2- 172.21.15.86:0/1829907060 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f521806c7a0 0x7f521806cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.813+0000 7f521ea00700 1 --2- 172.21.15.86:0/1829907060 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f521806d0c0 0x7f5218119f10 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.813+0000 7f521ea00700 1 -- 172.21.15.86:0/1829907060 >> 172.21.15.86:0/1829907060 conn(0x7f521806be30 msgr2=0x7f5218109dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:06:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.813+0000 7f521ea00700 1 -- 172.21.15.86:0/1829907060 shutdown_connections 2024-03-23T18:06:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.813+0000 7f521ea00700 1 -- 172.21.15.86:0/1829907060 wait complete. 2024-03-23T18:06:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.814+0000 7f521ea00700 1 Processor -- start 2024-03-23T18:06:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.815+0000 7f521ea00700 1 -- start start 2024-03-23T18:06:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.815+0000 7f521ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f521806c7a0 0x7f52181a5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.815+0000 7f521ea00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f521806d0c0 0x7f52181a6530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.815+0000 7f521ea00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f521811a450 0x7f52181aa8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:43.816 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.815+0000 7f521ea00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52181aaf10 con 0x7f521806c7a0 2024-03-23T18:06:43.816 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.815+0000 7f521ea00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52181ab080 con 0x7f521806d0c0 2024-03-23T18:06:43.816 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.815+0000 7f521ea00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52181ab240 con 0x7f521811a450 2024-03-23T18:06:43.816 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.815+0000 7f5217e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f521806c7a0 0x7f52181a5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:06:43.816 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.815+0000 7f521ca00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f521811a450 0x7f52181aa8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:06:43.816 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.815+0000 7f521ca00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f521811a450 0x7f52181aa8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:33994/0 (socket says 172.21.15.86:33994) 2024-03-23T18:06:43.816 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.815+0000 7f521ca00700 1 -- 172.21.15.86:0/2366983917 learned_addr learned my addr 172.21.15.86:0/2366983917 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:06:43.816 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.815+0000 7f5217e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f521806c7a0 0x7f52181a5ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43752/0 (socket says 172.21.15.86:43752) 2024-03-23T18:06:43.817 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.816+0000 7f521ca00700 1 -- 172.21.15.86:0/2366983917 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f521806d0c0 msgr2=0x7f52181a6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:06:43.817 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.816+0000 7f521ca00700 1 --2- 172.21.15.86:0/2366983917 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f521806d0c0 0x7f52181a6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:43.817 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.816+0000 7f521ca00700 1 -- 172.21.15.86:0/2366983917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f521806c7a0 msgr2=0x7f52181a5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:06:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.816+0000 7f521ca00700 1 --2- 172.21.15.86:0/2366983917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f521806c7a0 0x7f52181a5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.816+0000 7f521ca00700 1 -- 172.21.15.86:0/2366983917 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52040095f0 con 0x7f521811a450 2024-03-23T18:06:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.816+0000 7f521ca00700 1 --2- 172.21.15.86:0/2366983917 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f521811a450 0x7f52181aa8d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f520800d950 tx=0x7f520800dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:06:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.817+0000 7f5214c00700 1 -- 172.21.15.86:0/2366983917 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5208004a30 con 0x7f521811a450 2024-03-23T18:06:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.817+0000 7f521ea00700 1 -- 172.21.15.86:0/2366983917 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52181ab520 con 0x7f521811a450 2024-03-23T18:06:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.817+0000 7f5214c00700 1 -- 172.21.15.86:0/2366983917 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f520800b5c0 con 0x7f521811a450 2024-03-23T18:06:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.817+0000 7f5214c00700 1 -- 172.21.15.86:0/2366983917 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5208009ab0 con 0x7f521811a450 2024-03-23T18:06:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.817+0000 7f521ea00700 1 -- 172.21.15.86:0/2366983917 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52181b2d60 con 0x7f521811a450 2024-03-23T18:06:43.823 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.823+0000 7f5214c00700 1 -- 172.21.15.86:0/2366983917 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f5208009c10 con 0x7f521811a450 2024-03-23T18:06:43.824 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.824+0000 7f5232550b80 1 Processor -- start 2024-03-23T18:06:43.824 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.824+0000 7f5232550b80 1 -- start start 2024-03-23T18:06:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.825+0000 7f5232550b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c2f717100 0x564c2f7174e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.825+0000 7f5232550b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564c2f717a20 0x564c2f722390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.825+0000 7f5232550b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564c2f7228d0 0x564c2f724cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.825+0000 7f5232550b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2f670930 con 0x564c2f717100 2024-03-23T18:06:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.825+0000 7f5232550b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2f66e2f0 con 0x564c2f717a20 2024-03-23T18:06:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.825+0000 7f5232550b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2f669be0 con 0x564c2f7228d0 2024-03-23T18:06:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.825+0000 7f5217400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c2f717100 0x564c2f7174e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:06:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.825+0000 7f5217e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564c2f717a20 0x564c2f722390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:06:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.826+0000 7f5217400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c2f717100 0x564c2f7174e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43756/0 (socket says 172.21.15.86:43756) 2024-03-23T18:06:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.826+0000 7f5217400700 1 -- 172.21.15.86:0/2299061819 learned_addr learned my addr 172.21.15.86:0/2299061819 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:06:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.826+0000 7f5217400700 1 --2- 172.21.15.86:0/2299061819 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564c2f7228d0 0x564c2f724cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:06:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.826+0000 7f5217e00700 1 -- 172.21.15.86:0/2299061819 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564c2f7228d0 msgr2=0x564c2f724cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:06:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.826+0000 7f5217e00700 1 --2- 172.21.15.86:0/2299061819 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564c2f7228d0 0x564c2f724cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.826+0000 7f5217e00700 1 -- 172.21.15.86:0/2299061819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c2f717100 msgr2=0x564c2f7174e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:06:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.826+0000 7f5217e00700 1 --2- 172.21.15.86:0/2299061819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c2f717100 0x564c2f7174e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.826+0000 7f5217e00700 1 -- 172.21.15.86:0/2299061819 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c2f4eb350 con 0x564c2f717a20 2024-03-23T18:06:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.826+0000 7f5217e00700 1 --2- 172.21.15.86:0/2299061819 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564c2f717a20 0x564c2f722390 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f5210011580 tx=0x7f5210009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9157c0d16b0b51b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:06:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.827+0000 7f5214c00700 1 --2- 172.21.15.86:0/2366983917 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f51fc0a13e0 0x7f51fc0a38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:43.828 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.827+0000 7f5214c00700 1 -- 172.21.15.86:0/2366983917 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f52080cc650 con 0x7f521811a450 2024-03-23T18:06:43.828 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.828+0000 7f5217400700 1 --2- 172.21.15.86:0/2366983917 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f51fc0a13e0 0x7f51fc0a38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:06:43.829 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.829+0000 7f5217400700 1 --2- 172.21.15.86:0/2366983917 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f51fc0a13e0 0x7f51fc0a38a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x564c2f718b90 tx=0x7f5204000bc0 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:06:43.829 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.829+0000 7f520d200700 1 -- 172.21.15.86:0/2299061819 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f521000e040 con 0x564c2f717a20 2024-03-23T18:06:43.830 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.830+0000 7f520d200700 1 -- 172.21.15.86:0/2299061819 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f521000f040 con 0x564c2f717a20 2024-03-23T18:06:43.830 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.830+0000 7f520d200700 1 -- 172.21.15.86:0/2299061819 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52100042b0 con 0x564c2f717a20 2024-03-23T18:06:43.830 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.831+0000 7f5232550b80 1 -- 172.21.15.86:0/2299061819 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564c2f717a20 msgr2=0x564c2f722390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:06:43.830 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.831+0000 7f5232550b80 1 --2- 172.21.15.86:0/2299061819 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564c2f717a20 0x564c2f722390 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f5210011580 tx=0x7f5210009920 comp rx=0 tx=0).stop 2024-03-23T18:06:43.831 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.831+0000 7f5232550b80 1 -- 172.21.15.86:0/2299061819 shutdown_connections 2024-03-23T18:06:43.831 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.831+0000 7f5232550b80 1 --2- 172.21.15.86:0/2299061819 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564c2f7228d0 0x564c2f724cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:43.831 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.831+0000 7f5232550b80 1 --2- 172.21.15.86:0/2299061819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c2f717100 0x564c2f7174e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:43.831 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.831+0000 7f5232550b80 1 --2- 172.21.15.86:0/2299061819 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564c2f717a20 0x564c2f722390 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:43.831 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.831+0000 7f5232550b80 1 -- 172.21.15.86:0/2299061819 >> 172.21.15.86:0/2299061819 conn(0x564c2f677640 msgr2=0x564c2f713520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:06:43.831 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.831+0000 7f5232550b80 1 -- 172.21.15.86:0/2299061819 shutdown_connections 2024-03-23T18:06:43.831 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.831+0000 7f5232550b80 1 -- 172.21.15.86:0/2299061819 wait complete. 2024-03-23T18:06:43.832 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.832+0000 7f5232550b80 1 -- start start 2024-03-23T18:06:43.832 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.832+0000 7f5232550b80 1 Processor -- start 2024-03-23T18:06:43.833 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.833+0000 7f5232550b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c2f717100 0x564c2f722210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:43.833 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.833+0000 7f5232550b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564c2f717a20 0x564c2f71f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:43.833 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.833+0000 7f5232550b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564c2f7228d0 0x564c2f71fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:43.834 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.833+0000 7f5232550b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2f605d20 con 0x564c2f717100 2024-03-23T18:06:43.834 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.833+0000 7f5232550b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2f6059c0 con 0x564c2f7228d0 2024-03-23T18:06:43.834 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.833+0000 7f5232550b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2f4ed640 con 0x564c2f717a20 2024-03-23T18:06:43.834 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.833+0000 7f5232550b80 10 client.0 _ll_register_callbacks cb 0x564c2f60f410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T18:06:43.834 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.833+0000 7f5217400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564c2f717a20 0x564c2f71f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:06:43.834 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.833+0000 7f5217400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564c2f717a20 0x564c2f71f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:34014/0 (socket says 172.21.15.86:34014) 2024-03-23T18:06:43.834 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.833+0000 7f5217400700 1 -- 172.21.15.86:0/434987662 learned_addr learned my addr 172.21.15.86:0/434987662 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:06:43.835 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.834+0000 7f521ca00700 1 --2- 172.21.15.86:0/434987662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564c2f7228d0 0x564c2f71fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:06:43.836 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.834+0000 7f5217400700 1 -- 172.21.15.86:0/434987662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564c2f7228d0 msgr2=0x564c2f71fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:06:43.836 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.834+0000 7f5217400700 1 --2- 172.21.15.86:0/434987662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564c2f7228d0 0x564c2f71fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:43.836 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.834+0000 7f5217400700 1 -- 172.21.15.86:0/434987662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c2f717100 msgr2=0x564c2f722210 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-23T18:06:43.836 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.834+0000 7f5217e00700 1 --2- 172.21.15.86:0/434987662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c2f717100 0x564c2f722210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:06:43.836 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.834+0000 7f5217400700 1 --2- 172.21.15.86:0/434987662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c2f717100 0x564c2f722210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:43.836 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.835+0000 7f5217400700 1 -- 172.21.15.86:0/434987662 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5210011230 con 0x564c2f717a20 2024-03-23T18:06:43.836 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.835+0000 7f5217e00700 1 --2- 172.21.15.86:0/434987662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c2f717100 0x564c2f722210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T18:06:43.836 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.835+0000 7f5217400700 1 --2- 172.21.15.86:0/434987662 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564c2f717a20 0x564c2f71f5a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f5204005f50 tx=0x7f5204009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fdcf4013ebb69843 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:06:43.836 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.835+0000 7f5201600700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-03-23T18:06:43.836 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.835+0000 7f5201600700 1 -- 172.21.15.86:0/434987662 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f520402f070 con 0x564c2f717a20 2024-03-23T18:06:43.836 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.835+0000 7f5201600700 1 -- 172.21.15.86:0/434987662 <== mon.1 v2:172.21.15.152:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f52040048f0 con 0x564c2f717a20 2024-03-23T18:06:43.836 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.835+0000 7f5232550b80 1 -- 172.21.15.86:0/434987662 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564c2f62a320 con 0x564c2f717a20 2024-03-23T18:06:43.837 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.836+0000 7f5201600700 1 -- 172.21.15.86:0/434987662 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5204004ab0 con 0x564c2f717a20 2024-03-23T18:06:43.837 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.836+0000 7f5232550b80 10 client.27283 fetch_fsmap learned FSMap version 77 2024-03-23T18:06:43.837 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.836+0000 7f5232550b80 1 -- 172.21.15.86:0/434987662 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=4+}) v3 -- 0x564c2f497b00 con 0x564c2f717a20 2024-03-23T18:06:43.837 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.836+0000 7f5201600700 1 -- 172.21.15.86:0/434987662 <== mon.1 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564c2f62a320 con 0x564c2f717a20 2024-03-23T18:06:43.838 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.836+0000 7f5201600700 1 -- 172.21.15.86:0/434987662 <== mon.1 v2:172.21.15.152:3300/0 5 ==== fsmap(e 77) v1 ==== 7498+0+0 (secure 0 0 0) 0x7f520403ecb0 con 0x564c2f717a20 2024-03-23T18:06:43.838 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.836+0000 7f5232550b80 10 client.27283 fetch_fsmap finished waiting for FSMap version 77 2024-03-23T18:06:43.838 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.836+0000 7f5232550b80 10 client.27283 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T18:06:43.838 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.836+0000 7f5232550b80 20 client.27283 populate_metadata read hostname 'smithi086' 2024-03-23T18:06:43.838 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.836+0000 7f5232550b80 1 --2- 172.21.15.86:0/434987662 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x564c2f732520 0x564c2f734940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:43.838 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.836+0000 7f5232550b80 4 client.27283 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T18:06:43.838 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.836+0000 7f5232550b80 1 -- 172.21.15.86:0/434987662 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564c2f6626a0 con 0x564c2f732520 2024-03-23T18:06:43.838 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.837+0000 7f5217e00700 1 --2- 172.21.15.86:0/434987662 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x564c2f732520 0x564c2f734940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:06:43.838 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.837+0000 7f5217e00700 1 --2- 172.21.15.86:0/434987662 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x564c2f732520 0x564c2f734940 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f06f0e6e54828d7 server_cookie=2c59736417e837b2 in_seq=0 out_seq=0 2024-03-23T18:06:43.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.837+0000 7f5201600700 10 client.27283 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T18:06:43.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.839+0000 7f5201600700 1 -- 172.21.15.86:0/434987662 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x564c2f6626a0 con 0x564c2f732520 2024-03-23T18:06:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.839+0000 7f5201600700 10 client.27283 handle_command_reply: tid=0 2024-03-23T18:06:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.839+0000 7f5232550b80 1 client.27283 shutdown 2024-03-23T18:06:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.839+0000 7f5232550b80 1 -- 172.21.15.86:0/434987662 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564c2f717a20 msgr2=0x564c2f71f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:06:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.839+0000 7f5232550b80 1 --2- 172.21.15.86:0/434987662 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564c2f717a20 0x564c2f71f5a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f5204005f50 tx=0x7f5204009310 comp rx=0 tx=0).stop 2024-03-23T18:06:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.839+0000 7f5232550b80 1 -- 172.21.15.86:0/434987662 shutdown_connections 2024-03-23T18:06:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.839+0000 7f5232550b80 1 --2- 172.21.15.86:0/434987662 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x564c2f732520 0x564c2f734940 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.839+0000 7f5232550b80 1 --2- 172.21.15.86:0/434987662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564c2f7228d0 0x564c2f71fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.839+0000 7f5232550b80 1 --2- 172.21.15.86:0/434987662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c2f717100 0x564c2f722210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.839+0000 7f5232550b80 1 --2- 172.21.15.86:0/434987662 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564c2f717a20 0x564c2f71f5a0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.839+0000 7f5232550b80 1 -- 172.21.15.86:0/434987662 >> 172.21.15.86:0/434987662 conn(0x564c2f677640 msgr2=0x564c2f719040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:06:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.839+0000 7f5201600700 0 client.27283 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T18:06:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.839+0000 7f5232550b80 1 -- 172.21.15.86:0/434987662 shutdown_connections 2024-03-23T18:06:43.841 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.840+0000 7f5232550b80 1 -- 172.21.15.86:0/434987662 wait complete. 2024-03-23T18:06:43.841 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.840+0000 7f5232550b80 20 client.27283 trim_cache size 0 max 16384 2024-03-23T18:06:43.860 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.860+0000 7f5232550b80 1 Processor -- start 2024-03-23T18:06:43.860 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.861+0000 7f5232550b80 1 -- start start 2024-03-23T18:06:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.861+0000 7f5232550b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564c2f717100 0x564c2f729640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.861+0000 7f5232550b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564c2f717a20 0x564c2f70d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.861+0000 7f5232550b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c2f7228d0 0x564c2f70caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.861+0000 7f5232550b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2f720b00 con 0x564c2f7228d0 2024-03-23T18:06:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.861+0000 7f5232550b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2f70d030 con 0x564c2f717100 2024-03-23T18:06:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.861+0000 7f5232550b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2f7183b0 con 0x564c2f717a20 2024-03-23T18:06:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.861+0000 7f5217e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564c2f717100 0x564c2f729640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:06:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.861+0000 7f5217400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564c2f717a20 0x564c2f70d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:06:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.861+0000 7f5217e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564c2f717100 0x564c2f729640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:60956/0 (socket says 172.21.15.86:60956) 2024-03-23T18:06:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.861+0000 7f5217e00700 1 -- 172.21.15.86:0/2289527532 learned_addr learned my addr 172.21.15.86:0/2289527532 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:06:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.861+0000 7f521ca00700 1 --2- 172.21.15.86:0/2289527532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c2f7228d0 0x564c2f70caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:06:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.862+0000 7f5217400700 1 -- 172.21.15.86:0/2289527532 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564c2f717100 msgr2=0x564c2f729640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:06:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.862+0000 7f5217400700 1 --2- 172.21.15.86:0/2289527532 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564c2f717100 0x564c2f729640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.862+0000 7f5217400700 1 -- 172.21.15.86:0/2289527532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c2f7228d0 msgr2=0x564c2f70caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:06:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.862+0000 7f5217400700 1 --2- 172.21.15.86:0/2289527532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c2f7228d0 0x564c2f70caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.862+0000 7f5217400700 1 -- 172.21.15.86:0/2289527532 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c2f62fe40 con 0x564c2f717a20 2024-03-23T18:06:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.862+0000 7f5217400700 1 --2- 172.21.15.86:0/2289527532 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564c2f717a20 0x564c2f70d500 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f52040274a0 tx=0x7f5204009380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30ff6676db4338f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:06:43.864 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.862+0000 7f520dc00700 1 -- 172.21.15.86:0/2289527532 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f520402f040 con 0x564c2f717a20 2024-03-23T18:06:43.864 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.862+0000 7f520dc00700 1 -- 172.21.15.86:0/2289527532 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5204005960 con 0x564c2f717a20 2024-03-23T18:06:43.864 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.862+0000 7f520dc00700 1 -- 172.21.15.86:0/2289527532 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5204035430 con 0x564c2f717a20 2024-03-23T18:06:43.864 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.863+0000 7f5232550b80 1 -- 172.21.15.86:0/2289527532 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564c2f717a20 msgr2=0x564c2f70d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:06:43.864 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.863+0000 7f5232550b80 1 --2- 172.21.15.86:0/2289527532 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564c2f717a20 0x564c2f70d500 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f52040274a0 tx=0x7f5204009380 comp rx=0 tx=0).stop 2024-03-23T18:06:43.864 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.863+0000 7f5232550b80 1 -- 172.21.15.86:0/2289527532 shutdown_connections 2024-03-23T18:06:43.864 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.863+0000 7f5232550b80 1 --2- 172.21.15.86:0/2289527532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c2f7228d0 0x564c2f70caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:43.864 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.863+0000 7f5232550b80 1 --2- 172.21.15.86:0/2289527532 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564c2f717100 0x564c2f729640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:43.864 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.863+0000 7f5232550b80 1 --2- 172.21.15.86:0/2289527532 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564c2f717a20 0x564c2f70d500 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:43.864 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.863+0000 7f5232550b80 1 -- 172.21.15.86:0/2289527532 >> 172.21.15.86:0/2289527532 conn(0x564c2f677640 msgr2=0x564c2f70c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:06:43.864 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.863+0000 7f5232550b80 1 -- 172.21.15.86:0/2289527532 shutdown_connections 2024-03-23T18:06:43.864 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.863+0000 7f5232550b80 1 -- 172.21.15.86:0/2289527532 wait complete. 2024-03-23T18:06:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.864+0000 7f5232550b80 1 -- start start 2024-03-23T18:06:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.864+0000 7f5232550b80 1 Processor -- start 2024-03-23T18:06:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.864+0000 7f5232550b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564c2f717100 0x564c2f726db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.864+0000 7f5232550b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564c2f717a20 0x564c2f7272f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.864+0000 7f5232550b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c2f7228d0 0x564c2f727830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:43.867 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.864+0000 7f5232550b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2f783160 con 0x564c2f7228d0 2024-03-23T18:06:43.867 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.864+0000 7f5232550b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2f7832d0 con 0x564c2f717100 2024-03-23T18:06:43.867 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.864+0000 7f5232550b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2f783490 con 0x564c2f717a20 2024-03-23T18:06:43.867 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.865+0000 7f5232550b80 10 client.0 _ll_register_callbacks cb 0x564c2f60f410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T18:06:43.867 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.865+0000 7f521ca00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c2f7228d0 0x564c2f727830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:06:43.867 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.865+0000 7f5217e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564c2f717100 0x564c2f726db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:06:43.867 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.865+0000 7f521ca00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c2f7228d0 0x564c2f727830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43770/0 (socket says 172.21.15.86:43770) 2024-03-23T18:06:43.867 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.865+0000 7f521ca00700 1 -- 172.21.15.86:0/385689574 learned_addr learned my addr 172.21.15.86:0/385689574 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:06:43.867 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.865+0000 7f5217400700 1 --2- 172.21.15.86:0/385689574 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564c2f717a20 0x564c2f7272f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:06:43.867 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.865+0000 7f5217e00700 1 -- 172.21.15.86:0/385689574 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564c2f717a20 msgr2=0x564c2f7272f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:06:43.867 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.865+0000 7f5217e00700 1 --2- 172.21.15.86:0/385689574 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564c2f717a20 0x564c2f7272f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:43.867 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.865+0000 7f5217e00700 1 -- 172.21.15.86:0/385689574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c2f7228d0 msgr2=0x564c2f727830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:06:43.867 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.865+0000 7f5217e00700 1 --2- 172.21.15.86:0/385689574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c2f7228d0 0x564c2f727830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:43.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.865+0000 7f5217e00700 1 -- 172.21.15.86:0/385689574 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564c2f497b00 con 0x564c2f717100 2024-03-23T18:06:43.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.866+0000 7f5217e00700 1 --2- 172.21.15.86:0/385689574 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564c2f717100 0x564c2f726db0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f5210011b70 tx=0x7f5210009300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9450ccc338259eb1 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:06:43.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.866+0000 7f5201600700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T18:06:43.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.866+0000 7f5201600700 1 -- 172.21.15.86:0/385689574 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f521000e040 con 0x564c2f717100 2024-03-23T18:06:43.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.866+0000 7f5232550b80 1 -- 172.21.15.86:0/385689574 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f521000f040 con 0x564c2f717100 2024-03-23T18:06:43.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.866+0000 7f5201600700 1 -- 172.21.15.86:0/385689574 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f5210003ae0 con 0x564c2f717100 2024-03-23T18:06:43.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.867+0000 7f5201600700 1 -- 172.21.15.86:0/385689574 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5210037450 con 0x564c2f717100 2024-03-23T18:06:43.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.867+0000 7f5232550b80 10 client.27299 fetch_fsmap learned FSMap version 77 2024-03-23T18:06:43.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.867+0000 7f5232550b80 1 -- 172.21.15.86:0/385689574 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=4+}) v3 -- 0x564c2f4efc70 con 0x564c2f717100 2024-03-23T18:06:43.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.868+0000 7f5201600700 1 -- 172.21.15.86:0/385689574 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f521000f040 con 0x564c2f717100 2024-03-23T18:06:43.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.868+0000 7f5201600700 1 -- 172.21.15.86:0/385689574 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 77) v1 ==== 7498+0+0 (secure 0 0 0) 0x7f5210024040 con 0x564c2f717100 2024-03-23T18:06:43.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.868+0000 7f5232550b80 10 client.27299 fetch_fsmap finished waiting for FSMap version 77 2024-03-23T18:06:43.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.868+0000 7f5232550b80 10 client.27299 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T18:06:43.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.868+0000 7f5232550b80 20 client.27299 populate_metadata read hostname 'smithi086' 2024-03-23T18:06:43.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.868+0000 7f5232550b80 1 --2- 172.21.15.86:0/385689574 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x564c2f784c60 0x564c2f787080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:43.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.868+0000 7f5232550b80 4 client.27299 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T18:06:43.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.868+0000 7f5232550b80 1 -- 172.21.15.86:0/385689574 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564c2f4b8c90 con 0x564c2f784c60 2024-03-23T18:06:43.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.871+0000 7f5217400700 1 --2- 172.21.15.86:0/385689574 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x564c2f784c60 0x564c2f787080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:06:43.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.871+0000 7f5217400700 1 --2- 172.21.15.86:0/385689574 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x564c2f784c60 0x564c2f787080 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1654bbe1d0a34de3 server_cookie=701066ebc96a3f6a in_seq=0 out_seq=0 2024-03-23T18:06:43.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:43.871+0000 7f5201600700 10 client.27299 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T18:06:44.151 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:44.151+0000 7f5201600700 1 -- 172.21.15.86:0/385689574 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564c2f4b8c90 con 0x564c2f784c60 2024-03-23T18:06:44.151 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:44.151+0000 7f5201600700 10 client.27299 handle_command_reply: tid=0 2024-03-23T18:06:44.151 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:44.151+0000 7f5232550b80 1 client.27299 shutdown 2024-03-23T18:06:44.152 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:44.152+0000 7f5232550b80 1 -- 172.21.15.86:0/385689574 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564c2f717100 msgr2=0x564c2f726db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:06:44.152 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:44.152+0000 7f5232550b80 1 --2- 172.21.15.86:0/385689574 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564c2f717100 0x564c2f726db0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f5210011b70 tx=0x7f5210009300 comp rx=0 tx=0).stop 2024-03-23T18:06:44.153 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:44.152+0000 7f5232550b80 1 -- 172.21.15.86:0/385689574 shutdown_connections 2024-03-23T18:06:44.153 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:44.152+0000 7f5232550b80 1 --2- 172.21.15.86:0/385689574 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x564c2f784c60 0x564c2f787080 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:44.153 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:44.152+0000 7f5232550b80 1 --2- 172.21.15.86:0/385689574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c2f7228d0 0x564c2f727830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:44.153 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:44.152+0000 7f5232550b80 1 --2- 172.21.15.86:0/385689574 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564c2f717100 0x564c2f726db0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:44.153 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:44.152+0000 7f5201600700 0 client.27299 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T18:06:44.153 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:44.152+0000 7f5232550b80 1 --2- 172.21.15.86:0/385689574 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564c2f717a20 0x564c2f7272f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:44.153 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:44.152+0000 7f5232550b80 1 -- 172.21.15.86:0/385689574 >> 172.21.15.86:0/385689574 conn(0x564c2f677640 msgr2=0x564c2f712d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:06:44.153 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:44.152+0000 7f5232550b80 1 -- 172.21.15.86:0/385689574 shutdown_connections 2024-03-23T18:06:44.153 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:44.152+0000 7f5232550b80 1 -- 172.21.15.86:0/385689574 wait complete. 2024-03-23T18:06:44.153 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:44.153+0000 7f5232550b80 20 client.27299 trim_cache size 0 max 16384 2024-03-23T18:06:44.154 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-03-23T18:06:44.154 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-03-23T18:06:44.154 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-03-23T18:06:44.154 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-03-23T18:06:44.155 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:44.153+0000 7f521ea00700 1 -- 172.21.15.86:0/2366983917 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f51fc0a13e0 msgr2=0x7f51fc0a38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:06:44.155 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:44.153+0000 7f521ea00700 1 --2- 172.21.15.86:0/2366983917 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f51fc0a13e0 0x7f51fc0a38a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x564c2f718b90 tx=0x7f5204000bc0 comp rx=0 tx=0).stop 2024-03-23T18:06:44.155 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:44.153+0000 7f521ea00700 1 -- 172.21.15.86:0/2366983917 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f521811a450 msgr2=0x7f52181aa8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:06:44.155 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:44.153+0000 7f521ea00700 1 --2- 172.21.15.86:0/2366983917 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f521811a450 0x7f52181aa8d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f520800d950 tx=0x7f520800dc60 comp rx=0 tx=0).stop 2024-03-23T18:06:44.155 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:44.154+0000 7f521ea00700 1 -- 172.21.15.86:0/2366983917 shutdown_connections 2024-03-23T18:06:44.155 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:44.154+0000 7f521ea00700 1 --2- 172.21.15.86:0/2366983917 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f51fc0a13e0 0x7f51fc0a38a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:44.155 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:44.154+0000 7f521ea00700 1 --2- 172.21.15.86:0/2366983917 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f521811a450 0x7f52181aa8d0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:44.155 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:44.154+0000 7f521ea00700 1 --2- 172.21.15.86:0/2366983917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f521806c7a0 0x7f52181a5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:44.155 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:44.154+0000 7f521ea00700 1 --2- 172.21.15.86:0/2366983917 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f521806d0c0 0x7f52181a6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:44.155 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:44.154+0000 7f521ea00700 1 -- 172.21.15.86:0/2366983917 >> 172.21.15.86:0/2366983917 conn(0x7f521806be30 msgr2=0x7f5218109dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:06:44.155 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:44.154+0000 7f521ea00700 1 -- 172.21.15.86:0/2366983917 shutdown_connections 2024-03-23T18:06:44.156 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:44.154+0000 7f521ea00700 1 -- 172.21.15.86:0/2366983917 wait complete. 2024-03-23T18:06:44.241 INFO:tasks.workunit.client.0.smithi086.stdout: 10 137226 38.83 MB/sec execute 68 sec latency 163.425 ms 2024-03-23T18:06:44.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:06:43 smithi086 ceph-mon[38221]: pgmap v506: 97 pgs: 97 active+clean; 1.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 17 MiB/s wr, 907 op/s 2024-03-23T18:06:44.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:06:43 smithi092 ceph-mon[79089]: pgmap v506: 97 pgs: 97 active+clean; 1.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 17 MiB/s wr, 907 op/s 2024-03-23T18:06:44.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:06:43 smithi152 ceph-mon[47949]: pgmap v506: 97 pgs: 97 active+clean; 1.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 17 MiB/s wr, 907 op/s 2024-03-23T18:06:44.977 INFO:journalctl@ceph.mds.c.smithi152.stdout:Mar 23 18:06:44 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-c[84552]: ignoring --setuser ceph since I am not root 2024-03-23T18:06:44.977 INFO:journalctl@ceph.mds.c.smithi152.stdout:Mar 23 18:06:44 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-c[84552]: ignoring --setgroup ceph since I am not root 2024-03-23T18:06:44.978 INFO:journalctl@ceph.mds.c.smithi152.stdout:Mar 23 18:06:44 smithi152 ceph-mds[84575]: -- 172.21.15.152:0/3753066660 <== mon.2 v2:172.21.15.92:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c533cc21a0 con 0x55c533c98800 2024-03-23T18:06:44.978 INFO:journalctl@ceph.mds.c.smithi152.stdout:Mar 23 18:06:44 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-c[84552]: starting mds.c at 2024-03-23T18:06:45.242 INFO:tasks.workunit.client.0.smithi086.stdout: 10 138111 38.98 MB/sec execute 69 sec latency 49.944 ms 2024-03-23T18:06:45.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:06:44 smithi086 ceph-mon[38221]: daemon mds.c restarted 2024-03-23T18:06:45.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:06:44 smithi086 ceph-mon[38221]: osdmap e105: 12 total, 12 up, 12 in 2024-03-23T18:06:45.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:06:44 smithi092 ceph-mon[79089]: daemon mds.c restarted 2024-03-23T18:06:45.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:06:44 smithi092 ceph-mon[79089]: osdmap e105: 12 total, 12 up, 12 in 2024-03-23T18:06:45.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:06:44 smithi152 ceph-mon[47949]: daemon mds.c restarted 2024-03-23T18:06:45.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:06:44 smithi152 ceph-mon[47949]: osdmap e105: 12 total, 12 up, 12 in 2024-03-23T18:06:46.241 INFO:tasks.workunit.client.0.smithi086.stdout: 10 138934 38.98 MB/sec execute 70 sec latency 91.957 ms 2024-03-23T18:06:46.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:06:45 smithi086 ceph-mon[38221]: pgmap v507: 97 pgs: 97 active+clean; 1.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 18 MiB/s wr, 1.07k op/s 2024-03-23T18:06:46.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:06:45 smithi086 ceph-mon[38221]: mds.? [v2:172.21.15.152:6836/3300851333,v1:172.21.15.152:6837/3300851333] up:boot 2024-03-23T18:06:46.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:06:45 smithi086 ceph-mon[38221]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-23T18:06:46.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:06:45 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-23T18:06:46.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:06:45 smithi086 ceph-mon[38221]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:06:46.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:06:45 smithi092 ceph-mon[79089]: pgmap v507: 97 pgs: 97 active+clean; 1.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 18 MiB/s wr, 1.07k op/s 2024-03-23T18:06:46.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:06:45 smithi092 ceph-mon[79089]: mds.? [v2:172.21.15.152:6836/3300851333,v1:172.21.15.152:6837/3300851333] up:boot 2024-03-23T18:06:46.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:06:45 smithi092 ceph-mon[79089]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-23T18:06:46.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:06:45 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-23T18:06:46.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:06:45 smithi092 ceph-mon[79089]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:06:46.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:06:45 smithi152 ceph-mon[47949]: pgmap v507: 97 pgs: 97 active+clean; 1.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 18 MiB/s wr, 1.07k op/s 2024-03-23T18:06:46.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:06:45 smithi152 ceph-mon[47949]: mds.? [v2:172.21.15.152:6836/3300851333,v1:172.21.15.152:6837/3300851333] up:boot 2024-03-23T18:06:46.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:06:45 smithi152 ceph-mon[47949]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-23T18:06:46.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:06:45 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-23T18:06:46.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:06:45 smithi152 ceph-mon[47949]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:06:47.241 INFO:tasks.workunit.client.0.smithi086.stdout: 10 139768 39.13 MB/sec execute 71 sec latency 54.534 ms 2024-03-23T18:06:48.241 INFO:tasks.workunit.client.0.smithi086.stdout: 10 140700 39.27 MB/sec execute 72 sec latency 61.891 ms 2024-03-23T18:06:48.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:06:47 smithi086 ceph-mon[38221]: pgmap v509: 97 pgs: 97 active+clean; 1.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 8.1 MiB/s wr, 748 op/s 2024-03-23T18:06:48.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:06:47 smithi092 ceph-mon[79089]: pgmap v509: 97 pgs: 97 active+clean; 1.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 8.1 MiB/s wr, 748 op/s 2024-03-23T18:06:48.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:06:47 smithi152 ceph-mon[47949]: pgmap v509: 97 pgs: 97 active+clean; 1.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 8.1 MiB/s wr, 748 op/s 2024-03-23T18:06:49.242 INFO:tasks.workunit.client.0.smithi086.stdout: 10 141503 39.38 MB/sec execute 73 sec latency 69.348 ms 2024-03-23T18:06:50.242 INFO:tasks.workunit.client.0.smithi086.stdout: 10 142275 39.34 MB/sec execute 74 sec latency 59.992 ms 2024-03-23T18:06:50.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:06:49 smithi086 ceph-mon[38221]: pgmap v510: 97 pgs: 97 active+clean; 1.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 8.1 MiB/s wr, 748 op/s 2024-03-23T18:06:50.309 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:06:50 smithi092 ceph-mon[79089]: pgmap v510: 97 pgs: 97 active+clean; 1.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 8.1 MiB/s wr, 748 op/s 2024-03-23T18:06:50.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:06:49 smithi152 ceph-mon[47949]: pgmap v510: 97 pgs: 97 active+clean; 1.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 8.1 MiB/s wr, 748 op/s 2024-03-23T18:06:51.242 INFO:tasks.workunit.client.0.smithi086.stdout: 10 143303 39.59 MB/sec execute 75 sec latency 56.456 ms 2024-03-23T18:06:52.242 INFO:tasks.workunit.client.0.smithi086.stdout: 10 144101 39.67 MB/sec execute 76 sec latency 49.166 ms 2024-03-23T18:06:52.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:06:52 smithi086 ceph-mon[38221]: pgmap v511: 97 pgs: 97 active+clean; 1.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 8.2 MiB/s wr, 747 op/s 2024-03-23T18:06:52.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:06:52 smithi086 ceph-mon[38221]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:06:52.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:06:52 smithi092 ceph-mon[79089]: pgmap v511: 97 pgs: 97 active+clean; 1.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 8.2 MiB/s wr, 747 op/s 2024-03-23T18:06:52.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:06:52 smithi092 ceph-mon[79089]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:06:52.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:06:52 smithi152 ceph-mon[47949]: pgmap v511: 97 pgs: 97 active+clean; 1.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 8.2 MiB/s wr, 747 op/s 2024-03-23T18:06:52.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:06:52 smithi152 ceph-mon[47949]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:06:53.242 INFO:tasks.workunit.client.0.smithi086.stdout: 10 144807 39.67 MB/sec execute 77 sec latency 51.871 ms 2024-03-23T18:06:54.242 INFO:tasks.workunit.client.0.smithi086.stdout: 10 145419 39.53 MB/sec execute 78 sec latency 234.908 ms 2024-03-23T18:06:54.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:06:54 smithi092 ceph-mon[79089]: pgmap v512: 97 pgs: 97 active+clean; 1.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 8.2 MiB/s wr, 747 op/s 2024-03-23T18:06:54.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:06:54 smithi092 ceph-mon[79089]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:06:54.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:06:54 smithi152 ceph-mon[47949]: pgmap v512: 97 pgs: 97 active+clean; 1.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 8.2 MiB/s wr, 747 op/s 2024-03-23T18:06:54.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:06:54 smithi152 ceph-mon[47949]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:06:54.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:06:54 smithi086 ceph-mon[38221]: pgmap v512: 97 pgs: 97 active+clean; 1.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 8.2 MiB/s wr, 747 op/s 2024-03-23T18:06:54.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:06:54 smithi086 ceph-mon[38221]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:06:55.245 INFO:tasks.workunit.client.0.smithi086.stdout: 10 146304 39.64 MB/sec execute 79 sec latency 53.455 ms 2024-03-23T18:06:56.243 INFO:tasks.workunit.client.0.smithi086.stdout: 10 147258 39.74 MB/sec execute 80 sec latency 35.148 ms 2024-03-23T18:06:56.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:06:56 smithi092 ceph-mon[79089]: pgmap v513: 97 pgs: 97 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 8.4 MiB/s wr, 726 op/s 2024-03-23T18:06:56.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:06:56 smithi092 ceph-mon[79089]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:06:56.338 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:06:56 smithi152 ceph-mon[47949]: pgmap v513: 97 pgs: 97 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 8.4 MiB/s wr, 726 op/s 2024-03-23T18:06:56.338 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:06:56 smithi152 ceph-mon[47949]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:06:56.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:06:56 smithi086 ceph-mon[38221]: pgmap v513: 97 pgs: 97 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 8.4 MiB/s wr, 726 op/s 2024-03-23T18:06:56.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:06:56 smithi086 ceph-mon[38221]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:06:57.242 INFO:tasks.workunit.client.0.smithi086.stdout: 10 148125 39.91 MB/sec execute 81 sec latency 62.556 ms 2024-03-23T18:06:57.369 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.368+0000 7fb7b0800700 1 -- 172.21.15.86:0/1848075245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7a810f4d0 msgr2=0x7fb7a8107db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:06:57.369 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.368+0000 7fb7b0800700 1 --2- 172.21.15.86:0/1848075245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7a810f4d0 0x7fb7a8107db0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fb7a4009fc0 tx=0x7fb7a400b330 comp rx=0 tx=0).stop 2024-03-23T18:06:57.369 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.369+0000 7fb7b0800700 1 -- 172.21.15.86:0/1848075245 shutdown_connections 2024-03-23T18:06:57.369 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.369+0000 7fb7b0800700 1 --2- 172.21.15.86:0/1848075245 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7a811a590 0x7fb7a811c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:57.369 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.369+0000 7fb7b0800700 1 --2- 172.21.15.86:0/1848075245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7a810f4d0 0x7fb7a8107db0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:57.370 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.369+0000 7fb7b0800700 1 --2- 172.21.15.86:0/1848075245 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb7a810f940 0x7fb7a81082f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:57.370 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.369+0000 7fb7b0800700 1 -- 172.21.15.86:0/1848075245 >> 172.21.15.86:0/1848075245 conn(0x7fb7a806bd80 msgr2=0x7fb7a806c190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:06:57.370 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.370+0000 7fb7b0800700 1 -- 172.21.15.86:0/1848075245 shutdown_connections 2024-03-23T18:06:57.370 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.370+0000 7fb7b0800700 1 -- 172.21.15.86:0/1848075245 wait complete. 2024-03-23T18:06:57.373 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.373+0000 7fb7b0800700 1 Processor -- start 2024-03-23T18:06:57.373 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.373+0000 7fb7b0800700 1 -- start start 2024-03-23T18:06:57.373 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.373+0000 7fb7b0800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7a810f4d0 0x7fb7a8118900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:57.373 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.373+0000 7fb7b0800700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb7a810f940 0x7fb7a8118e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:57.373 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.373+0000 7fb7b0800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7a811a590 0x7fb7a8111a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:57.373 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.373+0000 7fb7b0800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a8112200 con 0x7fb7a810f4d0 2024-03-23T18:06:57.373 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.373+0000 7fb7b0800700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a8112370 con 0x7fb7a811a590 2024-03-23T18:06:57.374 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.373+0000 7fb7b0800700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a8112530 con 0x7fb7a810f940 2024-03-23T18:06:57.374 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.374+0000 7fb7ade00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7a810f4d0 0x7fb7a8118900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:06:57.374 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.374+0000 7fb7ade00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7a810f4d0 0x7fb7a8118900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36514/0 (socket says 172.21.15.86:36514) 2024-03-23T18:06:57.375 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.374+0000 7fb7ade00700 1 -- 172.21.15.86:0/2598811890 learned_addr learned my addr 172.21.15.86:0/2598811890 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:06:57.375 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.374+0000 7fb7ae800700 1 --2- 172.21.15.86:0/2598811890 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7a811a590 0x7fb7a8111a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:06:57.375 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.375+0000 7fb7ade00700 1 -- 172.21.15.86:0/2598811890 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb7a810f940 msgr2=0x7fb7a8118e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:06:57.375 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.375+0000 7fb7ade00700 1 --2- 172.21.15.86:0/2598811890 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb7a810f940 0x7fb7a8118e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:57.375 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.375+0000 7fb7ade00700 1 -- 172.21.15.86:0/2598811890 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7a811a590 msgr2=0x7fb7a8111a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:06:57.375 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.375+0000 7fb7ade00700 1 --2- 172.21.15.86:0/2598811890 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7a811a590 0x7fb7a8111a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:57.375 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.375+0000 7fb7ade00700 1 -- 172.21.15.86:0/2598811890 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7a4009c40 con 0x7fb7a810f4d0 2024-03-23T18:06:57.375 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.375+0000 7fb7ade00700 1 --2- 172.21.15.86:0/2598811890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7a810f4d0 0x7fb7a8118900 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fb7a4009040 tx=0x7fb7a4004d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:06:57.377 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.377+0000 7fb7a2a00700 1 -- 172.21.15.86:0/2598811890 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7a400e040 con 0x7fb7a810f4d0 2024-03-23T18:06:57.377 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.377+0000 7fb7a2a00700 1 -- 172.21.15.86:0/2598811890 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb7a400be10 con 0x7fb7a810f4d0 2024-03-23T18:06:57.377 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.377+0000 7fb7a2a00700 1 -- 172.21.15.86:0/2598811890 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7a401b6c0 con 0x7fb7a810f4d0 2024-03-23T18:06:57.377 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.377+0000 7fb7b0800700 1 -- 172.21.15.86:0/2598811890 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7a81127b0 con 0x7fb7a810f4d0 2024-03-23T18:06:57.377 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.378+0000 7fb7b0800700 1 -- 172.21.15.86:0/2598811890 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7a8116670 con 0x7fb7a810f4d0 2024-03-23T18:06:57.394 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.391+0000 7fb7a2a00700 1 -- 172.21.15.86:0/2598811890 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fb7a401d470 con 0x7fb7a810f4d0 2024-03-23T18:06:57.394 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.393+0000 7fb7a2a00700 1 --2- 172.21.15.86:0/2598811890 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fb7900a14a0 0x7fb7900a3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:57.394 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.393+0000 7fb7a2a00700 1 -- 172.21.15.86:0/2598811890 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fb7a40cdfe0 con 0x7fb7a810f4d0 2024-03-23T18:06:57.396 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.396+0000 7fb7c42a9b80 1 Processor -- start 2024-03-23T18:06:57.398 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.398+0000 7fb7ad400700 1 --2- 172.21.15.86:0/2598811890 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fb7900a14a0 0x7fb7900a3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:06:57.401 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.401+0000 7fb7ad400700 1 --2- 172.21.15.86:0/2598811890 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fb7900a14a0 0x7fb7900a3960 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb7a8113100 tx=0x7fb79c006d20 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:06:57.406 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.406+0000 7fb7c42a9b80 1 -- start start 2024-03-23T18:06:57.406 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.406+0000 7fb7c42a9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561449019100 0x5614490194e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:57.406 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.406+0000 7fb7c42a9b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561449019a20 0x561449024390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:57.408 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.406+0000 7fb7c42a9b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5614490248d0 0x561449026cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:57.408 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.406+0000 7fb7c42a9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x561448f72930 con 0x561449019100 2024-03-23T18:06:57.408 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.406+0000 7fb7c42a9b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x561448f702f0 con 0x5614490248d0 2024-03-23T18:06:57.408 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.406+0000 7fb7c42a9b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x561448f6bbe0 con 0x561449019a20 2024-03-23T18:06:57.408 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.408+0000 7fb7ae800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5614490248d0 0x561449026cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:06:57.408 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.408+0000 7fb7ae800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5614490248d0 0x561449026cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:60552/0 (socket says 172.21.15.86:60552) 2024-03-23T18:06:57.408 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.408+0000 7fb7ae800700 1 -- 172.21.15.86:0/604522631 learned_addr learned my addr 172.21.15.86:0/604522631 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:06:57.408 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.408+0000 7fb7ad400700 1 --2- 172.21.15.86:0/604522631 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561449019a20 0x561449024390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:06:57.408 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.408+0000 7fb7ae800700 1 -- 172.21.15.86:0/604522631 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561449019a20 msgr2=0x561449024390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:06:57.409 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.408+0000 7fb7ae800700 1 --2- 172.21.15.86:0/604522631 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561449019a20 0x561449024390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:57.409 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.408+0000 7fb7ae800700 1 -- 172.21.15.86:0/604522631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561449019100 msgr2=0x5614490194e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:06:57.409 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.408+0000 7fb7ae800700 1 --2- 172.21.15.86:0/604522631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561449019100 0x5614490194e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:57.409 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.408+0000 7fb7ae800700 1 -- 172.21.15.86:0/604522631 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561448ded350 con 0x5614490248d0 2024-03-23T18:06:57.411 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.410+0000 7fb7ae800700 1 --2- 172.21.15.86:0/604522631 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5614490248d0 0x561449026cc0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fb79400bc50 tx=0x7fb79400bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f03586b8536d235d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:06:57.414 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.414+0000 7fb79b000700 1 -- 172.21.15.86:0/604522631 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79400fbf0 con 0x5614490248d0 2024-03-23T18:06:57.414 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.414+0000 7fb79b000700 1 -- 172.21.15.86:0/604522631 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb79400fd50 con 0x5614490248d0 2024-03-23T18:06:57.415 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.414+0000 7fb79b000700 1 -- 172.21.15.86:0/604522631 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb794011440 con 0x5614490248d0 2024-03-23T18:06:57.415 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.416+0000 7fb7c42a9b80 1 -- 172.21.15.86:0/604522631 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5614490248d0 msgr2=0x561449026cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:06:57.415 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.416+0000 7fb7c42a9b80 1 --2- 172.21.15.86:0/604522631 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5614490248d0 0x561449026cc0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fb79400bc50 tx=0x7fb79400bf60 comp rx=0 tx=0).stop 2024-03-23T18:06:57.417 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.417+0000 7fb7c42a9b80 1 -- 172.21.15.86:0/604522631 shutdown_connections 2024-03-23T18:06:57.417 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.417+0000 7fb7c42a9b80 1 --2- 172.21.15.86:0/604522631 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5614490248d0 0x561449026cc0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:57.417 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.417+0000 7fb7c42a9b80 1 --2- 172.21.15.86:0/604522631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561449019100 0x5614490194e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:57.418 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.417+0000 7fb7c42a9b80 1 --2- 172.21.15.86:0/604522631 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561449019a20 0x561449024390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:57.418 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.417+0000 7fb7c42a9b80 1 -- 172.21.15.86:0/604522631 >> 172.21.15.86:0/604522631 conn(0x561448f79640 msgr2=0x561449015520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:06:57.418 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.417+0000 7fb7c42a9b80 1 -- 172.21.15.86:0/604522631 shutdown_connections 2024-03-23T18:06:57.418 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.418+0000 7fb7c42a9b80 1 -- 172.21.15.86:0/604522631 wait complete. 2024-03-23T18:06:57.419 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.420+0000 7fb7c42a9b80 1 -- start start 2024-03-23T18:06:57.421 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.421+0000 7fb7c42a9b80 1 Processor -- start 2024-03-23T18:06:57.424 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.424+0000 7fb7c42a9b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561449019100 0x561449024340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:57.425 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.424+0000 7fb7c42a9b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561449019a20 0x5614490216d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:57.425 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.424+0000 7fb7c42a9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614490248d0 0x561449021dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:57.425 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.424+0000 7fb7c42a9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x561448f07d20 con 0x5614490248d0 2024-03-23T18:06:57.425 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.425+0000 7fb7c42a9b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x561448f079c0 con 0x561449019a20 2024-03-23T18:06:57.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.425+0000 7fb7c42a9b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x561448def640 con 0x561449019100 2024-03-23T18:06:57.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.425+0000 7fb7c42a9b80 10 client.0 _ll_register_callbacks cb 0x561448f11410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T18:06:57.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.426+0000 7fb7ae800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614490248d0 0x561449021dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:06:57.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.426+0000 7fb7ae800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614490248d0 0x561449021dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36534/0 (socket says 172.21.15.86:36534) 2024-03-23T18:06:57.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.426+0000 7fb7ae800700 1 -- 172.21.15.86:0/3849299309 learned_addr learned my addr 172.21.15.86:0/3849299309 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:06:57.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.426+0000 7fb7ad400700 1 --2- 172.21.15.86:0/3849299309 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561449019a20 0x5614490216d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:06:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.427+0000 7fb7ad400700 1 -- 172.21.15.86:0/3849299309 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561449019100 msgr2=0x561449024340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:06:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.427+0000 7fb7ad400700 1 --2- 172.21.15.86:0/3849299309 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561449019100 0x561449024340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.427+0000 7fb7ad400700 1 -- 172.21.15.86:0/3849299309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614490248d0 msgr2=0x561449021dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:06:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.427+0000 7fb7ad400700 1 --2- 172.21.15.86:0/3849299309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614490248d0 0x561449021dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.427+0000 7fb7ad400700 1 -- 172.21.15.86:0/3849299309 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb79400b880 con 0x561449019a20 2024-03-23T18:06:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.427+0000 7fb7ae800700 1 --2- 172.21.15.86:0/3849299309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614490248d0 0x561449021dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T18:06:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.427+0000 7fb7ad400700 1 --2- 172.21.15.86:0/3849299309 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561449019a20 0x5614490216d0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fb7a810fd40 tx=0x7fb79c0048d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16fa823b550e70ec server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:06:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.428+0000 7fb78f400700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T18:06:57.430 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.430+0000 7fb78f400700 1 -- 172.21.15.86:0/3849299309 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79c0134d0 con 0x561449019a20 2024-03-23T18:06:57.430 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.430+0000 7fb78f400700 1 -- 172.21.15.86:0/3849299309 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fb79c01e400 con 0x561449019a20 2024-03-23T18:06:57.430 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.430+0000 7fb7c42a9b80 1 -- 172.21.15.86:0/3849299309 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561448f2c320 con 0x561449019a20 2024-03-23T18:06:57.432 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.431+0000 7fb78f400700 1 -- 172.21.15.86:0/3849299309 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb79c01e5c0 con 0x561449019a20 2024-03-23T18:06:57.433 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.432+0000 7fb7c42a9b80 10 client.27317 fetch_fsmap learned FSMap version 82 2024-03-23T18:06:57.433 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.433+0000 7fb7c42a9b80 1 -- 172.21.15.86:0/3849299309 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=4+}) v3 -- 0x561448d99b00 con 0x561449019a20 2024-03-23T18:06:57.433 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.433+0000 7fb78f400700 1 -- 172.21.15.86:0/3849299309 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561448f2c320 con 0x561449019a20 2024-03-23T18:06:57.439 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.439+0000 7fb78f400700 1 -- 172.21.15.86:0/3849299309 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 82) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fb79c026070 con 0x561449019a20 2024-03-23T18:06:57.439 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.439+0000 7fb7c42a9b80 10 client.27317 fetch_fsmap finished waiting for FSMap version 82 2024-03-23T18:06:57.439 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.439+0000 7fb7c42a9b80 10 client.27317 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T18:06:57.439 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.439+0000 7fb7c42a9b80 20 client.27317 populate_metadata read hostname 'smithi086' 2024-03-23T18:06:57.439 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.439+0000 7fb7c42a9b80 1 --2- 172.21.15.86:0/3849299309 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5614490346a0 0x561449036ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:57.439 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.439+0000 7fb7c42a9b80 4 client.27317 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T18:06:57.440 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.439+0000 7fb7c42a9b80 1 -- 172.21.15.86:0/3849299309 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561448f646a0 con 0x5614490346a0 2024-03-23T18:06:57.440 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.439+0000 7fb7ade00700 1 --2- 172.21.15.86:0/3849299309 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5614490346a0 0x561449036ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:06:57.444 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.444+0000 7fb7ade00700 1 --2- 172.21.15.86:0/3849299309 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5614490346a0 0x561449036ac0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c4097cf585cc7c4 server_cookie=eb4380358c22336c in_seq=0 out_seq=0 2024-03-23T18:06:57.444 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.444+0000 7fb78f400700 10 client.27317 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T18:06:57.456 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.455+0000 7fb78f400700 1 -- 172.21.15.86:0/3849299309 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x561448f646a0 con 0x5614490346a0 2024-03-23T18:06:57.456 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.455+0000 7fb78f400700 10 client.27317 handle_command_reply: tid=0 2024-03-23T18:06:57.456 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.456+0000 7fb7c42a9b80 1 client.27317 shutdown 2024-03-23T18:06:57.458 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.457+0000 7fb7c42a9b80 1 -- 172.21.15.86:0/3849299309 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561449019a20 msgr2=0x5614490216d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:06:57.458 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.457+0000 7fb7c42a9b80 1 --2- 172.21.15.86:0/3849299309 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561449019a20 0x5614490216d0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fb7a810fd40 tx=0x7fb79c0048d0 comp rx=0 tx=0).stop 2024-03-23T18:06:57.458 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.458+0000 7fb7c42a9b80 1 -- 172.21.15.86:0/3849299309 shutdown_connections 2024-03-23T18:06:57.458 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.458+0000 7fb7c42a9b80 1 --2- 172.21.15.86:0/3849299309 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5614490346a0 0x561449036ac0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:57.458 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.458+0000 7fb7c42a9b80 1 --2- 172.21.15.86:0/3849299309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614490248d0 0x561449021dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:57.458 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.458+0000 7fb7c42a9b80 1 --2- 172.21.15.86:0/3849299309 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561449019100 0x561449024340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:57.458 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.458+0000 7fb7c42a9b80 1 --2- 172.21.15.86:0/3849299309 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561449019a20 0x5614490216d0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:57.458 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.458+0000 7fb7c42a9b80 1 -- 172.21.15.86:0/3849299309 >> 172.21.15.86:0/3849299309 conn(0x561448f79640 msgr2=0x56144901b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:06:57.458 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.458+0000 7fb78f400700 0 client.27317 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T18:06:57.459 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.458+0000 7fb7c42a9b80 1 -- 172.21.15.86:0/3849299309 shutdown_connections 2024-03-23T18:06:57.459 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.459+0000 7fb7c42a9b80 1 -- 172.21.15.86:0/3849299309 wait complete. 2024-03-23T18:06:57.459 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.459+0000 7fb7c42a9b80 20 client.27317 trim_cache size 0 max 16384 2024-03-23T18:06:57.491 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.491+0000 7fb7c42a9b80 1 Processor -- start 2024-03-23T18:06:57.491 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.491+0000 7fb7c42a9b80 1 -- start start 2024-03-23T18:06:57.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.492+0000 7fb7c42a9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561449019100 0x56144903b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:57.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.492+0000 7fb7c42a9b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561449019a20 0x56144902b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:57.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.492+0000 7fb7c42a9b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5614490248d0 0x56144900f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:57.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.492+0000 7fb7c42a9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x561449019e00 con 0x561449019100 2024-03-23T18:06:57.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.492+0000 7fb7c42a9b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x561449011c80 con 0x561449019a20 2024-03-23T18:06:57.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.492+0000 7fb7c42a9b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x561449016ac0 con 0x5614490248d0 2024-03-23T18:06:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.492+0000 7fb7ad400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561449019a20 0x56144902b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:06:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.493+0000 7fb7ade00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561449019100 0x56144903b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:06:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.493+0000 7fb7ade00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561449019100 0x56144903b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36536/0 (socket says 172.21.15.86:36536) 2024-03-23T18:06:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.493+0000 7fb7ade00700 1 -- 172.21.15.86:0/494351203 learned_addr learned my addr 172.21.15.86:0/494351203 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:06:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.494+0000 7fb7ade00700 1 -- 172.21.15.86:0/494351203 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5614490248d0 msgr2=0x56144900f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:06:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.494+0000 7fb7ade00700 1 --2- 172.21.15.86:0/494351203 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5614490248d0 0x56144900f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.494+0000 7fb7ade00700 1 -- 172.21.15.86:0/494351203 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561449019a20 msgr2=0x56144902b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:06:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.494+0000 7fb7ade00700 1 --2- 172.21.15.86:0/494351203 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561449019a20 0x56144902b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.494+0000 7fb7ade00700 1 -- 172.21.15.86:0/494351203 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561448f31e40 con 0x561449019100 2024-03-23T18:06:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.495+0000 7fb7ade00700 1 --2- 172.21.15.86:0/494351203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561449019100 0x56144903b420 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fb7a409b100 tx=0x7fb7a4007aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51c41e5dfe3c3ba4 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:06:57.498 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.496+0000 7fb79ba00700 1 -- 172.21.15.86:0/494351203 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7a409efa0 con 0x561449019100 2024-03-23T18:06:57.498 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.498+0000 7fb79ba00700 1 -- 172.21.15.86:0/494351203 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb7a40a3560 con 0x561449019100 2024-03-23T18:06:57.498 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.498+0000 7fb79ba00700 1 -- 172.21.15.86:0/494351203 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7a40a30d0 con 0x561449019100 2024-03-23T18:06:57.498 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.499+0000 7fb7c42a9b80 1 -- 172.21.15.86:0/494351203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561449019100 msgr2=0x56144903b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:06:57.499 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.499+0000 7fb7c42a9b80 1 --2- 172.21.15.86:0/494351203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561449019100 0x56144903b420 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fb7a409b100 tx=0x7fb7a4007aa0 comp rx=0 tx=0).stop 2024-03-23T18:06:57.501 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.501+0000 7fb7c42a9b80 1 -- 172.21.15.86:0/494351203 shutdown_connections 2024-03-23T18:06:57.501 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.501+0000 7fb7c42a9b80 1 --2- 172.21.15.86:0/494351203 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5614490248d0 0x56144900f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:57.501 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.501+0000 7fb7c42a9b80 1 --2- 172.21.15.86:0/494351203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561449019100 0x56144903b420 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:57.502 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.501+0000 7fb7c42a9b80 1 --2- 172.21.15.86:0/494351203 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561449019a20 0x56144902b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:57.502 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.501+0000 7fb7c42a9b80 1 -- 172.21.15.86:0/494351203 >> 172.21.15.86:0/494351203 conn(0x561448f79640 msgr2=0x56144901c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:06:57.503 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.502+0000 7fb7c42a9b80 1 -- 172.21.15.86:0/494351203 shutdown_connections 2024-03-23T18:06:57.503 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.502+0000 7fb7c42a9b80 1 -- 172.21.15.86:0/494351203 wait complete. 2024-03-23T18:06:57.503 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.503+0000 7fb7c42a9b80 1 -- start start 2024-03-23T18:06:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.503+0000 7fb7c42a9b80 1 Processor -- start 2024-03-23T18:06:57.506 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.506+0000 7fb7c42a9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561449019100 0x561449028f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.506+0000 7fb7c42a9b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561449019a20 0x561449029450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.506+0000 7fb7c42a9b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5614490248d0 0x5614490851c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.507+0000 7fb7c42a9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x561449029d10 con 0x561449019100 2024-03-23T18:06:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.507+0000 7fb7c42a9b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x561449085700 con 0x561449019a20 2024-03-23T18:06:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.507+0000 7fb7c42a9b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x561449085840 con 0x5614490248d0 2024-03-23T18:06:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.507+0000 7fb7c42a9b80 10 client.0 _ll_register_callbacks cb 0x7fb79c013940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T18:06:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.508+0000 7fb7ad400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561449019a20 0x561449029450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:06:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.508+0000 7fb7ad400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561449019a20 0x561449029450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:60574/0 (socket says 172.21.15.86:60574) 2024-03-23T18:06:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.508+0000 7fb7ad400700 1 -- 172.21.15.86:0/3240781825 learned_addr learned my addr 172.21.15.86:0/3240781825 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:06:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.508+0000 7fb7ade00700 1 --2- 172.21.15.86:0/3240781825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561449019100 0x561449028f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:06:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.509+0000 7fb7ad400700 1 -- 172.21.15.86:0/3240781825 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5614490248d0 msgr2=0x5614490851c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:06:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.509+0000 7fb7ad400700 1 --2- 172.21.15.86:0/3240781825 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5614490248d0 0x5614490851c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.509+0000 7fb7ad400700 1 -- 172.21.15.86:0/3240781825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561449019100 msgr2=0x561449028f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:06:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.509+0000 7fb7ad400700 1 --2- 172.21.15.86:0/3240781825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561449019100 0x561449028f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.509+0000 7fb7ad400700 1 -- 172.21.15.86:0/3240781825 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7a40cbe70 con 0x561449019a20 2024-03-23T18:06:57.516 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.516+0000 7fb78f400700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T18:06:57.516 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.516+0000 7fb7ad400700 1 --2- 172.21.15.86:0/3240781825 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561449019a20 0x561449029450 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fb79c000c00 tx=0x7fb79c006b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e90cde5a738456d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:06:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.517+0000 7fb78f400700 1 -- 172.21.15.86:0/3240781825 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79c01e5c0 con 0x561449019a20 2024-03-23T18:06:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.517+0000 7fb78f400700 1 -- 172.21.15.86:0/3240781825 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 8440+0+0 (secure 0 0 0) 0x7fb79c01e400 con 0x561449019a20 2024-03-23T18:06:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.517+0000 7fb7c42a9b80 1 -- 172.21.15.86:0/3240781825 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb79400fd50 con 0x561449019a20 2024-03-23T18:06:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.520+0000 7fb78f400700 1 -- 172.21.15.86:0/3240781825 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb79c0134d0 con 0x561449019a20 2024-03-23T18:06:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.520+0000 7fb7c42a9b80 10 client.27320 fetch_fsmap learned FSMap version 82 2024-03-23T18:06:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.520+0000 7fb7c42a9b80 1 -- 172.21.15.86:0/3240781825 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=4+}) v3 -- 0x561448df1c70 con 0x561449019a20 2024-03-23T18:06:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.521+0000 7fb78f400700 1 -- 172.21.15.86:0/3240781825 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb79400fd50 con 0x561449019a20 2024-03-23T18:06:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.521+0000 7fb78f400700 1 -- 172.21.15.86:0/3240781825 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 82) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fb79c0123b0 con 0x561449019a20 2024-03-23T18:06:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.521+0000 7fb7c42a9b80 10 client.27320 fetch_fsmap finished waiting for FSMap version 82 2024-03-23T18:06:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.521+0000 7fb7c42a9b80 10 client.27320 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T18:06:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.521+0000 7fb7c42a9b80 20 client.27320 populate_metadata read hostname 'smithi086' 2024-03-23T18:06:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.521+0000 7fb7c42a9b80 1 --2- 172.21.15.86:0/3240781825 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x561449086ed0 0x5614490892f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:06:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.521+0000 7fb7c42a9b80 4 client.27320 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T18:06:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.521+0000 7fb7c42a9b80 1 -- 172.21.15.86:0/3240781825 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561448dbac90 con 0x561449086ed0 2024-03-23T18:06:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.523+0000 7fb7ade00700 1 --2- 172.21.15.86:0/3240781825 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x561449086ed0 0x5614490892f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:06:57.531 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.531+0000 7fb7ade00700 1 --2- 172.21.15.86:0/3240781825 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x561449086ed0 0x5614490892f0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=480eaaf3d5b95565 server_cookie=2508e63642615a4 in_seq=0 out_seq=0 2024-03-23T18:06:57.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.531+0000 7fb78f400700 10 client.27320 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T18:06:57.991 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.989+0000 7fb78f400700 1 -- 172.21.15.86:0/3240781825 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561448dbac90 con 0x561449086ed0 2024-03-23T18:06:57.991 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.989+0000 7fb78f400700 10 client.27320 handle_command_reply: tid=0 2024-03-23T18:06:57.991 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.989+0000 7fb7c42a9b80 1 client.27320 shutdown 2024-03-23T18:06:57.991 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.990+0000 7fb7c42a9b80 1 -- 172.21.15.86:0/3240781825 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561449019a20 msgr2=0x561449029450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:06:57.991 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.990+0000 7fb7c42a9b80 1 --2- 172.21.15.86:0/3240781825 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561449019a20 0x561449029450 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fb79c000c00 tx=0x7fb79c006b30 comp rx=0 tx=0).stop 2024-03-23T18:06:57.991 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.990+0000 7fb7c42a9b80 1 -- 172.21.15.86:0/3240781825 shutdown_connections 2024-03-23T18:06:57.991 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.990+0000 7fb7c42a9b80 1 --2- 172.21.15.86:0/3240781825 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x561449086ed0 0x5614490892f0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:57.991 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.990+0000 7fb7c42a9b80 1 --2- 172.21.15.86:0/3240781825 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5614490248d0 0x5614490851c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:57.991 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.990+0000 7fb7c42a9b80 1 --2- 172.21.15.86:0/3240781825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561449019100 0x561449028f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:57.991 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.990+0000 7fb7c42a9b80 1 --2- 172.21.15.86:0/3240781825 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561449019a20 0x561449029450 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:57.991 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.990+0000 7fb78f400700 0 client.27320 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T18:06:57.992 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.990+0000 7fb7c42a9b80 1 -- 172.21.15.86:0/3240781825 >> 172.21.15.86:0/3240781825 conn(0x561448f79640 msgr2=0x561449015030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:06:57.992 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.990+0000 7fb7c42a9b80 1 -- 172.21.15.86:0/3240781825 shutdown_connections 2024-03-23T18:06:57.992 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.990+0000 7fb7c42a9b80 1 -- 172.21.15.86:0/3240781825 wait complete. 2024-03-23T18:06:57.992 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.990+0000 7fb7c42a9b80 20 client.27320 trim_cache size 0 max 16384 2024-03-23T18:06:57.992 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-03-23T18:06:57.992 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-03-23T18:06:57.992 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-03-23T18:06:57.992 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-03-23T18:06:57.993 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.991+0000 7fb7b0800700 1 -- 172.21.15.86:0/2598811890 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fb7900a14a0 msgr2=0x7fb7900a3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:06:57.993 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.991+0000 7fb7b0800700 1 --2- 172.21.15.86:0/2598811890 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fb7900a14a0 0x7fb7900a3960 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb7a8113100 tx=0x7fb79c006d20 comp rx=0 tx=0).stop 2024-03-23T18:06:57.993 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.991+0000 7fb7b0800700 1 -- 172.21.15.86:0/2598811890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7a810f4d0 msgr2=0x7fb7a8118900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:06:57.993 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.991+0000 7fb7b0800700 1 --2- 172.21.15.86:0/2598811890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7a810f4d0 0x7fb7a8118900 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fb7a4009040 tx=0x7fb7a4004d30 comp rx=0 tx=0).stop 2024-03-23T18:06:57.993 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.991+0000 7fb7b0800700 1 -- 172.21.15.86:0/2598811890 shutdown_connections 2024-03-23T18:06:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.991+0000 7fb7b0800700 1 --2- 172.21.15.86:0/2598811890 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7a811a590 0x7fb7a8111a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.991+0000 7fb7b0800700 1 --2- 172.21.15.86:0/2598811890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7a810f4d0 0x7fb7a8118900 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.991+0000 7fb7b0800700 1 --2- 172.21.15.86:0/2598811890 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fb7900a14a0 0x7fb7900a3960 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.991+0000 7fb7b0800700 1 --2- 172.21.15.86:0/2598811890 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb7a810f940 0x7fb7a8118e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:06:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.991+0000 7fb7b0800700 1 -- 172.21.15.86:0/2598811890 >> 172.21.15.86:0/2598811890 conn(0x7fb7a806bd80 msgr2=0x7fb7a806f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:06:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.992+0000 7fb7b0800700 1 -- 172.21.15.86:0/2598811890 shutdown_connections 2024-03-23T18:06:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:06:57.992+0000 7fb7b0800700 1 -- 172.21.15.86:0/2598811890 wait complete. 2024-03-23T18:06:58.243 INFO:tasks.workunit.client.0.smithi086.stdout: 10 148413 39.63 MB/sec execute 82 sec latency 295.769 ms 2024-03-23T18:06:58.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:06:58 smithi092 ceph-mon[79089]: pgmap v514: 97 pgs: 97 active+clean; 1.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 7.2 MiB/s wr, 605 op/s 2024-03-23T18:06:58.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:06:58 smithi152 ceph-mon[47949]: pgmap v514: 97 pgs: 97 active+clean; 1.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 7.2 MiB/s wr, 605 op/s 2024-03-23T18:06:58.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:06:58 smithi086 ceph-mon[38221]: pgmap v514: 97 pgs: 97 active+clean; 1.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 7.2 MiB/s wr, 605 op/s 2024-03-23T18:06:59.042 INFO:journalctl@ceph.mds.c.smithi152.stdout:Mar 23 18:06:58 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-c[84552]: ignoring --setuser ceph since I am not root 2024-03-23T18:06:59.042 INFO:journalctl@ceph.mds.c.smithi152.stdout:Mar 23 18:06:58 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-c[84552]: ignoring --setgroup ceph since I am not root 2024-03-23T18:06:59.042 INFO:journalctl@ceph.mds.c.smithi152.stdout:Mar 23 18:06:58 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-c[84552]: starting mds.c at 2024-03-23T18:06:59.243 INFO:tasks.workunit.client.0.smithi086.stdout: 10 149000 39.48 MB/sec execute 83 sec latency 61.349 ms 2024-03-23T18:06:59.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:06:59 smithi092 ceph-mon[79089]: daemon mds.c restarted 2024-03-23T18:06:59.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:06:59 smithi092 ceph-mon[79089]: osdmap e106: 12 total, 12 up, 12 in 2024-03-23T18:06:59.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:06:59 smithi152 ceph-mon[47949]: daemon mds.c restarted 2024-03-23T18:06:59.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:06:59 smithi152 ceph-mon[47949]: osdmap e106: 12 total, 12 up, 12 in 2024-03-23T18:06:59.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:06:59 smithi086 ceph-mon[38221]: daemon mds.c restarted 2024-03-23T18:06:59.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:06:59 smithi086 ceph-mon[38221]: osdmap e106: 12 total, 12 up, 12 in 2024-03-23T18:07:00.243 INFO:tasks.workunit.client.0.smithi086.stdout: 10 149783 39.48 MB/sec execute 84 sec latency 39.031 ms 2024-03-23T18:07:00.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:07:00 smithi092 ceph-mon[79089]: pgmap v515: 97 pgs: 97 active+clean; 1.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.6 MiB/s wr, 536 op/s 2024-03-23T18:07:00.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:07:00 smithi092 ceph-mon[79089]: mds.? [v2:172.21.15.152:6836/988988765,v1:172.21.15.152:6837/988988765] up:boot 2024-03-23T18:07:00.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:07:00 smithi092 ceph-mon[79089]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-23T18:07:00.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:07:00 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-23T18:07:00.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:07:00 smithi092 ceph-mon[79089]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:07:00.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:07:00 smithi152 ceph-mon[47949]: pgmap v515: 97 pgs: 97 active+clean; 1.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.6 MiB/s wr, 536 op/s 2024-03-23T18:07:00.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:07:00 smithi152 ceph-mon[47949]: mds.? [v2:172.21.15.152:6836/988988765,v1:172.21.15.152:6837/988988765] up:boot 2024-03-23T18:07:00.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:07:00 smithi152 ceph-mon[47949]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-23T18:07:00.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:07:00 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-23T18:07:00.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:07:00 smithi152 ceph-mon[47949]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:07:00.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:07:00 smithi086 ceph-mon[38221]: pgmap v515: 97 pgs: 97 active+clean; 1.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.6 MiB/s wr, 536 op/s 2024-03-23T18:07:00.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:07:00 smithi086 ceph-mon[38221]: mds.? [v2:172.21.15.152:6836/988988765,v1:172.21.15.152:6837/988988765] up:boot 2024-03-23T18:07:00.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:07:00 smithi086 ceph-mon[38221]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-23T18:07:00.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:07:00 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-23T18:07:00.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:07:00 smithi086 ceph-mon[38221]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:07:01.243 INFO:tasks.workunit.client.0.smithi086.stdout: 10 150537 39.40 MB/sec execute 85 sec latency 249.812 ms 2024-03-23T18:07:01.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:07:01 smithi152 ceph-mon[47949]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:07:01.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:07:01 smithi152 ceph-mon[47949]: pgmap v517: 97 pgs: 97 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 806 op/s 2024-03-23T18:07:01.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:07:01 smithi086 ceph-mon[38221]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:07:01.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:07:01 smithi086 ceph-mon[38221]: pgmap v517: 97 pgs: 97 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 806 op/s 2024-03-23T18:07:01.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:07:01 smithi092 ceph-mon[79089]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:07:01.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:07:01 smithi092 ceph-mon[79089]: pgmap v517: 97 pgs: 97 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 806 op/s 2024-03-23T18:07:02.243 INFO:tasks.workunit.client.0.smithi086.stdout: 10 150537 38.95 MB/sec execute 86 sec latency 1249.878 ms 2024-03-23T18:07:03.244 INFO:tasks.workunit.client.0.smithi086.stdout: 10 150537 38.50 MB/sec execute 87 sec latency 2250.594 ms 2024-03-23T18:07:04.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:07:03 smithi086 ceph-mon[38221]: pgmap v518: 97 pgs: 97 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 806 op/s 2024-03-23T18:07:04.056 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:07:03 smithi092 ceph-mon[79089]: pgmap v518: 97 pgs: 97 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 806 op/s 2024-03-23T18:07:04.085 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:07:03 smithi152 ceph-mon[47949]: pgmap v518: 97 pgs: 97 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 806 op/s 2024-03-23T18:07:04.246 INFO:tasks.workunit.client.0.smithi086.stdout: 10 150537 38.06 MB/sec execute 88 sec latency 3250.690 ms 2024-03-23T18:07:05.244 INFO:tasks.workunit.client.0.smithi086.stdout: 10 150644 37.70 MB/sec execute 89 sec latency 4077.826 ms 2024-03-23T18:07:06.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:07:05 smithi086 ceph-mon[38221]: pgmap v519: 97 pgs: 97 active+clean; 1.4 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 17 MiB/s wr, 806 op/s 2024-03-23T18:07:06.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:07:05 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T18:07:06.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:07:05 smithi092 ceph-mon[79089]: pgmap v519: 97 pgs: 97 active+clean; 1.4 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 17 MiB/s wr, 806 op/s 2024-03-23T18:07:06.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:07:05 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T18:07:06.085 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:07:05 smithi152 ceph-mon[47949]: pgmap v519: 97 pgs: 97 active+clean; 1.4 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 17 MiB/s wr, 806 op/s 2024-03-23T18:07:06.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:07:05 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T18:07:06.244 INFO:tasks.workunit.client.0.smithi086.stdout: 10 150988 37.48 MB/sec execute 90 sec latency 699.001 ms 2024-03-23T18:07:07.244 INFO:tasks.workunit.client.0.smithi086.stdout: 10 151273 37.30 MB/sec execute 91 sec latency 346.579 ms 2024-03-23T18:07:08.244 INFO:tasks.workunit.client.0.smithi086.stdout: 10 151871 37.32 MB/sec execute 92 sec latency 108.971 ms 2024-03-23T18:07:08.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:07:07 smithi086 ceph-mon[38221]: pgmap v520: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 26 MiB/s wr, 905 op/s 2024-03-23T18:07:08.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:07:07 smithi092 ceph-mon[79089]: pgmap v520: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 26 MiB/s wr, 905 op/s 2024-03-23T18:07:08.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:07:07 smithi152 ceph-mon[47949]: pgmap v520: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 26 MiB/s wr, 905 op/s 2024-03-23T18:07:09.244 INFO:tasks.workunit.client.0.smithi086.stdout: 10 152568 37.33 MB/sec execute 93 sec latency 101.813 ms 2024-03-23T18:07:09.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:07:08 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T18:07:09.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:07:08 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T18:07:09.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:07:08 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T18:07:09.309 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:07:08 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T18:07:09.310 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:07:08 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T18:07:09.310 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:07:08 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T18:07:09.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:07:08 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T18:07:09.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:07:08 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T18:07:09.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:07:08 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T18:07:10.245 INFO:tasks.workunit.client.0.smithi086.stdout: 10 153276 37.29 MB/sec execute 94 sec latency 72.015 ms 2024-03-23T18:07:10.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:07:09 smithi086 ceph-mon[38221]: pgmap v521: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 26 MiB/s wr, 905 op/s 2024-03-23T18:07:10.310 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:07:09 smithi092 ceph-mon[79089]: pgmap v521: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 26 MiB/s wr, 905 op/s 2024-03-23T18:07:10.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:07:09 smithi152 ceph-mon[47949]: pgmap v521: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 26 MiB/s wr, 905 op/s 2024-03-23T18:07:11.245 INFO:tasks.workunit.client.0.smithi086.stdout: 10 153735 37.13 MB/sec execute 95 sec latency 73.791 ms 2024-03-23T18:07:11.246 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.245+0000 7ff36e000700 1 -- 172.21.15.86:0/2733398245 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff360013050 con 0x7ff370100f10 2024-03-23T18:07:11.248 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.247+0000 7ff376800700 1 -- 172.21.15.86:0/2733398245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff370100f10 msgr2=0x7ff3701012f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:07:11.248 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.247+0000 7ff376800700 1 --2- 172.21.15.86:0/2733398245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff370100f10 0x7ff3701012f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7ff360013ef0 tx=0x7ff3600097f0 comp rx=0 tx=0).stop 2024-03-23T18:07:11.249 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.248+0000 7ff376800700 1 -- 172.21.15.86:0/2733398245 shutdown_connections 2024-03-23T18:07:11.249 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.248+0000 7ff376800700 1 --2- 172.21.15.86:0/2733398245 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff370106030 0x7ff37010dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:11.249 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.248+0000 7ff376800700 1 --2- 172.21.15.86:0/2733398245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff370100f10 0x7ff3701012f0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:11.249 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.248+0000 7ff376800700 1 --2- 172.21.15.86:0/2733398245 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff3701018c0 0x7ff370105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:11.249 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.248+0000 7ff376800700 1 -- 172.21.15.86:0/2733398245 >> 172.21.15.86:0/2733398245 conn(0x7ff3700fc770 msgr2=0x7ff3700feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:07:11.249 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.248+0000 7ff376800700 1 -- 172.21.15.86:0/2733398245 shutdown_connections 2024-03-23T18:07:11.250 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.248+0000 7ff376800700 1 -- 172.21.15.86:0/2733398245 wait complete. 2024-03-23T18:07:11.250 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.249+0000 7ff376800700 1 Processor -- start 2024-03-23T18:07:11.254 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.250+0000 7ff376800700 1 -- start start 2024-03-23T18:07:11.254 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.250+0000 7ff376800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff370100f10 0x7ff370195060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:07:11.254 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.250+0000 7ff376800700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff370106030 0x7ff3701955a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:07:11.254 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.250+0000 7ff376800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3701998f0 0x7ff370199d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:07:11.254 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.250+0000 7ff376800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff37019a370 con 0x7ff370100f10 2024-03-23T18:07:11.254 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.250+0000 7ff376800700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff37019a4b0 con 0x7ff3701998f0 2024-03-23T18:07:11.254 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.250+0000 7ff376800700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff37019a620 con 0x7ff370106030 2024-03-23T18:07:11.254 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.251+0000 7ff36f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff370100f10 0x7ff370195060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:07:11.254 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.251+0000 7ff36f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff370100f10 0x7ff370195060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54372/0 (socket says 172.21.15.86:54372) 2024-03-23T18:07:11.254 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.251+0000 7ff36f400700 1 -- 172.21.15.86:0/95584818 learned_addr learned my addr 172.21.15.86:0/95584818 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:07:11.254 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.251+0000 7ff36f400700 1 -- 172.21.15.86:0/95584818 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff370106030 msgr2=0x7ff3701955a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:07:11.255 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.252+0000 7ff36f400700 1 --2- 172.21.15.86:0/95584818 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff370106030 0x7ff3701955a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:11.255 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.252+0000 7ff36f400700 1 -- 172.21.15.86:0/95584818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3701998f0 msgr2=0x7ff370199d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:07:11.255 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.252+0000 7ff36f400700 1 --2- 172.21.15.86:0/95584818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3701998f0 0x7ff370199d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:11.255 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.252+0000 7ff36f400700 1 -- 172.21.15.86:0/95584818 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff360013b50 con 0x7ff370100f10 2024-03-23T18:07:11.255 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.252+0000 7ff36f400700 1 --2- 172.21.15.86:0/95584818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff370100f10 0x7ff370195060 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7ff360006010 tx=0x7ff360013630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:07:11.255 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.252+0000 7ff367e00700 1 -- 172.21.15.86:0/95584818 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff360003dc0 con 0x7ff370100f10 2024-03-23T18:07:11.255 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.253+0000 7ff367e00700 1 -- 172.21.15.86:0/95584818 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff36000f450 con 0x7ff370100f10 2024-03-23T18:07:11.255 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.253+0000 7ff367e00700 1 -- 172.21.15.86:0/95584818 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff36001b680 con 0x7ff370100f10 2024-03-23T18:07:11.255 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.253+0000 7ff376800700 1 -- 172.21.15.86:0/95584818 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3701b2bb0 con 0x7ff370100f10 2024-03-23T18:07:11.255 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.253+0000 7ff376800700 1 -- 172.21.15.86:0/95584818 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3701b2f80 con 0x7ff370100f10 2024-03-23T18:07:11.260 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.259+0000 7ff367e00700 1 -- 172.21.15.86:0/95584818 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7ff36000f5c0 con 0x7ff370100f10 2024-03-23T18:07:11.263 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.262+0000 7ff367e00700 1 --2- 172.21.15.86:0/95584818 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7ff3540a13e0 0x7ff3540a38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:07:11.263 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.263+0000 7ff367e00700 1 -- 172.21.15.86:0/95584818 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 8528+0+0 (secure 0 0 0) 0x7ff3600ce120 con 0x7ff370100f10 2024-03-23T18:07:11.265 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.263+0000 7ff36ea00700 1 --2- 172.21.15.86:0/95584818 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7ff3540a13e0 0x7ff3540a38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:07:11.265 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.263+0000 7ff38a231b80 1 Processor -- start 2024-03-23T18:07:11.265 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.263+0000 7ff38a231b80 1 -- start start 2024-03-23T18:07:11.265 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.264+0000 7ff38a231b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df3d5ad100 0x55df3d5ad4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:07:11.266 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.264+0000 7ff38a231b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55df3d5ada20 0x55df3d5b8390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:07:11.266 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.264+0000 7ff38a231b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55df3d5b88d0 0x55df3d5bacc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:07:11.266 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.264+0000 7ff38a231b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df3d506930 con 0x55df3d5ad100 2024-03-23T18:07:11.266 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.264+0000 7ff38a231b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df3d5042f0 con 0x55df3d5ada20 2024-03-23T18:07:11.266 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.264+0000 7ff38a231b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df3d4ffbe0 con 0x55df3d5b88d0 2024-03-23T18:07:11.266 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.264+0000 7ff36fe00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55df3d5b88d0 0x55df3d5bacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:07:11.266 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.264+0000 7ff36fe00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55df3d5b88d0 0x55df3d5bacc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:43990/0 (socket says 172.21.15.86:43990) 2024-03-23T18:07:11.266 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.264+0000 7ff36fe00700 1 -- 172.21.15.86:0/3462437407 learned_addr learned my addr 172.21.15.86:0/3462437407 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:07:11.266 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.265+0000 7ff36f400700 1 --2- 172.21.15.86:0/3462437407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df3d5ad100 0x55df3d5ad4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:07:11.267 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.265+0000 7ff36fe00700 1 -- 172.21.15.86:0/3462437407 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55df3d5ada20 msgr2=0x55df3d5b8390 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-23T18:07:11.267 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.265+0000 7ff36fe00700 1 --2- 172.21.15.86:0/3462437407 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55df3d5ada20 0x55df3d5b8390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:11.267 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.265+0000 7ff36fe00700 1 -- 172.21.15.86:0/3462437407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df3d5ad100 msgr2=0x55df3d5ad4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:07:11.267 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.265+0000 7ff36fe00700 1 --2- 172.21.15.86:0/3462437407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df3d5ad100 0x55df3d5ad4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:11.267 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.265+0000 7ff36fe00700 1 -- 172.21.15.86:0/3462437407 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df3d381350 con 0x55df3d5b88d0 2024-03-23T18:07:11.267 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.266+0000 7ff36fe00700 1 --2- 172.21.15.86:0/3462437407 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55df3d5b88d0 0x55df3d5bacc0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff36800a410 tx=0x7ff36800a720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2e56b08dc19f46b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:07:11.268 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:07:11 smithi086 ceph-mon[38221]: osdmap e107: 12 total, 12 up, 12 in 2024-03-23T18:07:11.268 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.266+0000 7ff36ea00700 1 --2- 172.21.15.86:0/95584818 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7ff3540a13e0 0x7ff3540a38a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff370196560 tx=0x7ff35c006d20 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:07:11.269 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.266+0000 7ff364c00700 1 -- 172.21.15.86:0/3462437407 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff36800f070 con 0x55df3d5b88d0 2024-03-23T18:07:11.269 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.266+0000 7ff364c00700 1 -- 172.21.15.86:0/3462437407 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff36800ae90 con 0x55df3d5b88d0 2024-03-23T18:07:11.269 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.266+0000 7ff364c00700 1 -- 172.21.15.86:0/3462437407 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff368008630 con 0x55df3d5b88d0 2024-03-23T18:07:11.269 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.267+0000 7ff38a231b80 1 -- 172.21.15.86:0/3462437407 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55df3d5b88d0 msgr2=0x55df3d5bacc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:07:11.269 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.267+0000 7ff38a231b80 1 --2- 172.21.15.86:0/3462437407 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55df3d5b88d0 0x55df3d5bacc0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff36800a410 tx=0x7ff36800a720 comp rx=0 tx=0).stop 2024-03-23T18:07:11.269 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.268+0000 7ff38a231b80 1 -- 172.21.15.86:0/3462437407 shutdown_connections 2024-03-23T18:07:11.269 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.268+0000 7ff38a231b80 1 --2- 172.21.15.86:0/3462437407 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55df3d5b88d0 0x55df3d5bacc0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:11.269 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.268+0000 7ff38a231b80 1 --2- 172.21.15.86:0/3462437407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df3d5ad100 0x55df3d5ad4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:11.269 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.268+0000 7ff38a231b80 1 --2- 172.21.15.86:0/3462437407 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55df3d5ada20 0x55df3d5b8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:11.269 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.268+0000 7ff38a231b80 1 -- 172.21.15.86:0/3462437407 >> 172.21.15.86:0/3462437407 conn(0x55df3d50d640 msgr2=0x55df3d5a9520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:07:11.270 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.268+0000 7ff38a231b80 1 -- 172.21.15.86:0/3462437407 shutdown_connections 2024-03-23T18:07:11.270 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.268+0000 7ff38a231b80 1 -- 172.21.15.86:0/3462437407 wait complete. 2024-03-23T18:07:11.270 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.269+0000 7ff38a231b80 1 -- start start 2024-03-23T18:07:11.271 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.270+0000 7ff38a231b80 1 Processor -- start 2024-03-23T18:07:11.274 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.272+0000 7ff38a231b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55df3d5ad100 0x55df3d5b82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:07:11.274 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.272+0000 7ff38a231b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df3d5ada20 0x55df3d5b55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:07:11.274 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.272+0000 7ff38a231b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55df3d5b88d0 0x55df3d5b5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:07:11.274 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.272+0000 7ff38a231b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df3d49bd20 con 0x55df3d5ada20 2024-03-23T18:07:11.274 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.272+0000 7ff38a231b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df3d49b9c0 con 0x55df3d5b88d0 2024-03-23T18:07:11.274 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.272+0000 7ff38a231b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df3d383640 con 0x55df3d5ad100 2024-03-23T18:07:11.274 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.272+0000 7ff38a231b80 10 client.0 _ll_register_callbacks cb 0x55df3d4a5410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T18:07:11.274 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.272+0000 7ff36fe00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55df3d5b88d0 0x55df3d5b5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:07:11.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.272+0000 7ff36fe00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55df3d5b88d0 0x55df3d5b5bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:53574/0 (socket says 172.21.15.86:53574) 2024-03-23T18:07:11.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.272+0000 7ff36f400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55df3d5ad100 0x55df3d5b82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:07:11.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.272+0000 7ff36fe00700 1 -- 172.21.15.86:0/921499264 learned_addr learned my addr 172.21.15.86:0/921499264 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:07:11.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.273+0000 7ff36f400700 1 -- 172.21.15.86:0/921499264 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55df3d5b88d0 msgr2=0x55df3d5b5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:07:11.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.273+0000 7ff36f400700 1 --2- 172.21.15.86:0/921499264 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55df3d5b88d0 0x55df3d5b5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:11.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.273+0000 7ff36f400700 1 -- 172.21.15.86:0/921499264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df3d5ada20 msgr2=0x55df3d5b55a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-23T18:07:11.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.273+0000 7ff36ea00700 1 --2- 172.21.15.86:0/921499264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df3d5ada20 0x55df3d5b55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:07:11.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.274+0000 7ff36f400700 1 --2- 172.21.15.86:0/921499264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df3d5ada20 0x55df3d5b55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:11.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.274+0000 7ff36f400700 1 -- 172.21.15.86:0/921499264 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff368007cb0 con 0x55df3d5ad100 2024-03-23T18:07:11.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.274+0000 7ff36ea00700 1 --2- 172.21.15.86:0/921499264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df3d5ada20 0x55df3d5b55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T18:07:11.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.274+0000 7ff36f400700 1 --2- 172.21.15.86:0/921499264 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55df3d5ad100 0x55df3d5b82d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff3600cb870 tx=0x7ff36000ef80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2f7c95567a965ec server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:07:11.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.275+0000 7ff359600700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-03-23T18:07:11.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.275+0000 7ff359600700 1 -- 172.21.15.86:0/921499264 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff36009f0f0 con 0x55df3d5ad100 2024-03-23T18:07:11.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.275+0000 7ff359600700 1 -- 172.21.15.86:0/921499264 <== mon.1 v2:172.21.15.152:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 8528+0+0 (secure 0 0 0) 0x7ff3600a36b0 con 0x55df3d5ad100 2024-03-23T18:07:11.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.275+0000 7ff38a231b80 1 -- 172.21.15.86:0/921499264 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55df3d4c0320 con 0x55df3d5ad100 2024-03-23T18:07:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.276+0000 7ff359600700 1 -- 172.21.15.86:0/921499264 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff36009a7b0 con 0x55df3d5ad100 2024-03-23T18:07:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.276+0000 7ff38a231b80 10 client.27301 fetch_fsmap learned FSMap version 85 2024-03-23T18:07:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.276+0000 7ff38a231b80 1 -- 172.21.15.86:0/921499264 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=85,monmap=4+}) v3 -- 0x55df3d32db00 con 0x55df3d5ad100 2024-03-23T18:07:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.276+0000 7ff359600700 1 -- 172.21.15.86:0/921499264 <== mon.1 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55df3d4c0320 con 0x55df3d5ad100 2024-03-23T18:07:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.276+0000 7ff359600700 1 -- 172.21.15.86:0/921499264 <== mon.1 v2:172.21.15.152:3300/0 5 ==== fsmap(e 85) v1 ==== 7478+0+0 (secure 0 0 0) 0x7ff3600970b0 con 0x55df3d5ad100 2024-03-23T18:07:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.277+0000 7ff38a231b80 10 client.27301 fetch_fsmap finished waiting for FSMap version 85 2024-03-23T18:07:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.277+0000 7ff38a231b80 10 client.27301 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T18:07:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.277+0000 7ff38a231b80 20 client.27301 populate_metadata read hostname 'smithi086' 2024-03-23T18:07:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.277+0000 7ff38a231b80 1 --2- 172.21.15.86:0/921499264 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55df3d5d9050 0x55df3d5db440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:07:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.277+0000 7ff38a231b80 4 client.27301 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T18:07:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.277+0000 7ff38a231b80 1 -- 172.21.15.86:0/921499264 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55df3d4f86a0 con 0x55df3d5d9050 2024-03-23T18:07:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.278+0000 7ff36ea00700 1 --2- 172.21.15.86:0/921499264 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55df3d5d9050 0x55df3d5db440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:07:11.280 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.279+0000 7ff36ea00700 1 --2- 172.21.15.86:0/921499264 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55df3d5d9050 0x55df3d5db440 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=208a30c81eecada server_cookie=f0d3c9667d4b744b in_seq=0 out_seq=0 2024-03-23T18:07:11.280 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.279+0000 7ff359600700 10 client.27301 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T18:07:11.284 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.283+0000 7ff359600700 1 -- 172.21.15.86:0/921499264 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55df3d4f86a0 con 0x55df3d5d9050 2024-03-23T18:07:11.284 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.283+0000 7ff359600700 10 client.27301 handle_command_reply: tid=0 2024-03-23T18:07:11.284 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.283+0000 7ff38a231b80 1 client.27301 shutdown 2024-03-23T18:07:11.285 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.284+0000 7ff38a231b80 1 -- 172.21.15.86:0/921499264 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55df3d5ad100 msgr2=0x55df3d5b82d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:07:11.285 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.284+0000 7ff38a231b80 1 --2- 172.21.15.86:0/921499264 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55df3d5ad100 0x55df3d5b82d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff3600cb870 tx=0x7ff36000ef80 comp rx=0 tx=0).stop 2024-03-23T18:07:11.285 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.284+0000 7ff38a231b80 1 -- 172.21.15.86:0/921499264 shutdown_connections 2024-03-23T18:07:11.285 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.284+0000 7ff38a231b80 1 --2- 172.21.15.86:0/921499264 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55df3d5d9050 0x55df3d5db440 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:11.285 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.284+0000 7ff38a231b80 1 --2- 172.21.15.86:0/921499264 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55df3d5b88d0 0x55df3d5b5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:11.286 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.284+0000 7ff38a231b80 1 --2- 172.21.15.86:0/921499264 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55df3d5ad100 0x55df3d5b82d0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:11.286 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.284+0000 7ff38a231b80 1 --2- 172.21.15.86:0/921499264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df3d5ada20 0x55df3d5b55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:11.286 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.284+0000 7ff38a231b80 1 -- 172.21.15.86:0/921499264 >> 172.21.15.86:0/921499264 conn(0x55df3d50d640 msgr2=0x55df3d5af0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:07:11.286 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.284+0000 7ff359600700 0 client.27301 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T18:07:11.287 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.285+0000 7ff38a231b80 1 -- 172.21.15.86:0/921499264 shutdown_connections 2024-03-23T18:07:11.287 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.285+0000 7ff38a231b80 1 -- 172.21.15.86:0/921499264 wait complete. 2024-03-23T18:07:11.287 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.285+0000 7ff38a231b80 20 client.27301 trim_cache size 0 max 16384 2024-03-23T18:07:11.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:07:11 smithi092 ceph-mon[79089]: osdmap e107: 12 total, 12 up, 12 in 2024-03-23T18:07:11.314 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.313+0000 7ff38a231b80 1 Processor -- start 2024-03-23T18:07:11.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.314+0000 7ff38a231b80 1 -- start start 2024-03-23T18:07:11.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.314+0000 7ff38a231b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55df3d5ad100 0x55df3d5dfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:07:11.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.314+0000 7ff38a231b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df3d5ada20 0x55df3d5bf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:07:11.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.314+0000 7ff38a231b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55df3d5b88d0 0x55df3d5a3500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:07:11.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.314+0000 7ff38a231b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df3d5a5cb0 con 0x55df3d5ada20 2024-03-23T18:07:11.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.314+0000 7ff38a231b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df3d5b6bf0 con 0x55df3d5b88d0 2024-03-23T18:07:11.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.314+0000 7ff38a231b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df3d5a5400 con 0x55df3d5ad100 2024-03-23T18:07:11.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.314+0000 7ff36fe00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55df3d5b88d0 0x55df3d5a3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:07:11.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.315+0000 7ff36fe00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55df3d5b88d0 0x55df3d5a3500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:53576/0 (socket says 172.21.15.86:53576) 2024-03-23T18:07:11.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.315+0000 7ff36fe00700 1 -- 172.21.15.86:0/3696147421 learned_addr learned my addr 172.21.15.86:0/3696147421 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:07:11.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.315+0000 7ff36fe00700 1 -- 172.21.15.86:0/3696147421 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55df3d5ad100 msgr2=0x55df3d5dfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:07:11.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.315+0000 7ff36fe00700 1 --2- 172.21.15.86:0/3696147421 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55df3d5ad100 0x55df3d5dfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:11.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.315+0000 7ff36fe00700 1 -- 172.21.15.86:0/3696147421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df3d5ada20 msgr2=0x55df3d5bf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:07:11.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.315+0000 7ff36ea00700 1 --2- 172.21.15.86:0/3696147421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df3d5ada20 0x55df3d5bf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:07:11.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.315+0000 7ff36fe00700 1 --2- 172.21.15.86:0/3696147421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df3d5ada20 0x55df3d5bf610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:11.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.315+0000 7ff36fe00700 1 -- 172.21.15.86:0/3696147421 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df3d4c5e40 con 0x55df3d5b88d0 2024-03-23T18:07:11.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.316+0000 7ff36fe00700 1 --2- 172.21.15.86:0/3696147421 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55df3d5b88d0 0x55df3d5a3500 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ff368006130 tx=0x7ff368004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1dcee2057b9e878 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:07:11.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.316+0000 7ff365600700 1 -- 172.21.15.86:0/3696147421 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff368013ea0 con 0x55df3d5b88d0 2024-03-23T18:07:11.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.316+0000 7ff365600700 1 -- 172.21.15.86:0/3696147421 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff368007380 con 0x55df3d5b88d0 2024-03-23T18:07:11.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.316+0000 7ff365600700 1 -- 172.21.15.86:0/3696147421 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff36800f040 con 0x55df3d5b88d0 2024-03-23T18:07:11.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.317+0000 7ff38a231b80 1 -- 172.21.15.86:0/3696147421 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55df3d5b88d0 msgr2=0x55df3d5a3500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:07:11.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.317+0000 7ff38a231b80 1 --2- 172.21.15.86:0/3696147421 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55df3d5b88d0 0x55df3d5a3500 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ff368006130 tx=0x7ff368004ab0 comp rx=0 tx=0).stop 2024-03-23T18:07:11.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.317+0000 7ff38a231b80 1 -- 172.21.15.86:0/3696147421 shutdown_connections 2024-03-23T18:07:11.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.317+0000 7ff38a231b80 1 --2- 172.21.15.86:0/3696147421 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55df3d5b88d0 0x55df3d5a3500 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:11.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.317+0000 7ff38a231b80 1 --2- 172.21.15.86:0/3696147421 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55df3d5ad100 0x55df3d5dfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:11.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.317+0000 7ff38a231b80 1 --2- 172.21.15.86:0/3696147421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df3d5ada20 0x55df3d5bf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:11.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.317+0000 7ff38a231b80 1 -- 172.21.15.86:0/3696147421 >> 172.21.15.86:0/3696147421 conn(0x55df3d50d640 msgr2=0x55df3d5afdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:07:11.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.318+0000 7ff38a231b80 1 -- 172.21.15.86:0/3696147421 shutdown_connections 2024-03-23T18:07:11.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.318+0000 7ff38a231b80 1 -- 172.21.15.86:0/3696147421 wait complete. 2024-03-23T18:07:11.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.318+0000 7ff38a231b80 1 -- start start 2024-03-23T18:07:11.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.319+0000 7ff38a231b80 1 Processor -- start 2024-03-23T18:07:11.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.319+0000 7ff38a231b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55df3d5ad100 0x55df3d623ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:07:11.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.319+0000 7ff38a231b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df3d5ada20 0x55df3d624410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:07:11.321 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.319+0000 7ff38a231b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55df3d5b88d0 0x55df3d5bd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:07:11.321 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.319+0000 7ff38a231b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df3d5bdcd0 con 0x55df3d5ada20 2024-03-23T18:07:11.321 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.319+0000 7ff38a231b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df3d5bde40 con 0x55df3d5b88d0 2024-03-23T18:07:11.321 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.319+0000 7ff38a231b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df3d5be000 con 0x55df3d5ad100 2024-03-23T18:07:11.321 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.319+0000 7ff38a231b80 10 client.0 _ll_register_callbacks cb 0x55df3d4a5410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T18:07:11.322 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.320+0000 7ff36ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df3d5ada20 0x55df3d624410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:07:11.322 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.321+0000 7ff36ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df3d5ada20 0x55df3d624410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54412/0 (socket says 172.21.15.86:54412) 2024-03-23T18:07:11.322 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.321+0000 7ff36ea00700 1 -- 172.21.15.86:0/705603605 learned_addr learned my addr 172.21.15.86:0/705603605 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:07:11.322 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.321+0000 7ff36ea00700 1 -- 172.21.15.86:0/705603605 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55df3d5ad100 msgr2=0x55df3d623ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:07:11.322 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.321+0000 7ff36ea00700 1 --2- 172.21.15.86:0/705603605 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55df3d5ad100 0x55df3d623ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:11.322 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.321+0000 7ff36ea00700 1 -- 172.21.15.86:0/705603605 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55df3d5b88d0 msgr2=0x55df3d5bd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:07:11.322 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.321+0000 7ff36ea00700 1 --2- 172.21.15.86:0/705603605 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55df3d5b88d0 0x55df3d5bd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:11.322 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.321+0000 7ff36ea00700 1 -- 172.21.15.86:0/705603605 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55df3d381350 con 0x55df3d5ada20 2024-03-23T18:07:11.323 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.322+0000 7ff36ea00700 1 --2- 172.21.15.86:0/705603605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df3d5ada20 0x55df3d624410 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7ff35c009170 tx=0x7ff35c00e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fe1a12c2cdd93 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:07:11.323 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.322+0000 7ff359600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-23T18:07:11.323 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.322+0000 7ff359600700 1 -- 172.21.15.86:0/705603605 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff35c021c90 con 0x55df3d5ada20 2024-03-23T18:07:11.324 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.323+0000 7ff359600700 1 -- 172.21.15.86:0/705603605 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 8528+0+0 (secure 0 0 0) 0x7ff35c021df0 con 0x55df3d5ada20 2024-03-23T18:07:11.324 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.323+0000 7ff38a231b80 1 -- 172.21.15.86:0/705603605 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff36800ae90 con 0x55df3d5ada20 2024-03-23T18:07:11.324 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.323+0000 7ff359600700 1 -- 172.21.15.86:0/705603605 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff35c021410 con 0x55df3d5ada20 2024-03-23T18:07:11.324 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.323+0000 7ff38a231b80 10 client.17235 fetch_fsmap learned FSMap version 85 2024-03-23T18:07:11.324 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.323+0000 7ff38a231b80 1 -- 172.21.15.86:0/705603605 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=85,monmap=4+}) v3 -- 0x55df3d385c70 con 0x55df3d5ada20 2024-03-23T18:07:11.325 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.323+0000 7ff359600700 1 -- 172.21.15.86:0/705603605 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff36800ae90 con 0x55df3d5ada20 2024-03-23T18:07:11.325 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.324+0000 7ff359600700 1 -- 172.21.15.86:0/705603605 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 85) v1 ==== 7478+0+0 (secure 0 0 0) 0x7ff35c0324b0 con 0x55df3d5ada20 2024-03-23T18:07:11.325 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.324+0000 7ff38a231b80 10 client.17235 fetch_fsmap finished waiting for FSMap version 85 2024-03-23T18:07:11.325 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.324+0000 7ff38a231b80 10 client.17235 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T18:07:11.325 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.324+0000 7ff38a231b80 20 client.17235 populate_metadata read hostname 'smithi086' 2024-03-23T18:07:11.325 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.324+0000 7ff38a231b80 1 --2- 172.21.15.86:0/705603605 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55df3d62bcb0 0x55df3d62e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:07:11.325 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.324+0000 7ff38a231b80 4 client.17235 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T18:07:11.325 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.324+0000 7ff38a231b80 1 -- 172.21.15.86:0/705603605 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55df3d34ec90 con 0x55df3d62bcb0 2024-03-23T18:07:11.326 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.325+0000 7ff36f400700 1 --2- 172.21.15.86:0/705603605 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55df3d62bcb0 0x55df3d62e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:07:11.327 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.326+0000 7ff36f400700 1 --2- 172.21.15.86:0/705603605 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55df3d62bcb0 0x55df3d62e0d0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=291429c6b1355450 server_cookie=e106b795fbe68c51 in_seq=0 out_seq=0 2024-03-23T18:07:11.327 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.326+0000 7ff359600700 10 client.17235 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T18:07:11.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:07:11 smithi152 ceph-mon[47949]: osdmap e107: 12 total, 12 up, 12 in 2024-03-23T18:07:11.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.624+0000 7ff359600700 1 -- 172.21.15.86:0/705603605 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55df3d34ec90 con 0x55df3d62bcb0 2024-03-23T18:07:11.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.624+0000 7ff359600700 10 client.17235 handle_command_reply: tid=0 2024-03-23T18:07:11.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.624+0000 7ff38a231b80 1 client.17235 shutdown 2024-03-23T18:07:11.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.624+0000 7ff38a231b80 1 -- 172.21.15.86:0/705603605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df3d5ada20 msgr2=0x55df3d624410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:07:11.626 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.624+0000 7ff38a231b80 1 --2- 172.21.15.86:0/705603605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df3d5ada20 0x55df3d624410 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7ff35c009170 tx=0x7ff35c00e000 comp rx=0 tx=0).stop 2024-03-23T18:07:11.626 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.625+0000 7ff38a231b80 1 -- 172.21.15.86:0/705603605 shutdown_connections 2024-03-23T18:07:11.626 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.625+0000 7ff38a231b80 1 --2- 172.21.15.86:0/705603605 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55df3d62bcb0 0x55df3d62e0d0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:11.626 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.625+0000 7ff38a231b80 1 --2- 172.21.15.86:0/705603605 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55df3d5b88d0 0x55df3d5bd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:11.626 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.625+0000 7ff38a231b80 1 --2- 172.21.15.86:0/705603605 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55df3d5ad100 0x55df3d623ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:11.626 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.625+0000 7ff359600700 0 client.17235 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T18:07:11.626 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.625+0000 7ff38a231b80 1 --2- 172.21.15.86:0/705603605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df3d5ada20 0x55df3d624410 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:11.626 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.625+0000 7ff38a231b80 1 -- 172.21.15.86:0/705603605 >> 172.21.15.86:0/705603605 conn(0x55df3d50d640 msgr2=0x55df3d5a8d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:07:11.626 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.625+0000 7ff38a231b80 1 -- 172.21.15.86:0/705603605 shutdown_connections 2024-03-23T18:07:11.627 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.625+0000 7ff38a231b80 1 -- 172.21.15.86:0/705603605 wait complete. 2024-03-23T18:07:11.628 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.628+0000 7ff38a231b80 20 client.17235 trim_cache size 0 max 16384 2024-03-23T18:07:11.629 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-03-23T18:07:11.629 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-03-23T18:07:11.629 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-03-23T18:07:11.629 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-03-23T18:07:11.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.629+0000 7ff376800700 1 -- 172.21.15.86:0/95584818 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7ff3540a13e0 msgr2=0x7ff3540a38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:07:11.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.629+0000 7ff376800700 1 --2- 172.21.15.86:0/95584818 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7ff3540a13e0 0x7ff3540a38a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff370196560 tx=0x7ff35c006d20 comp rx=0 tx=0).stop 2024-03-23T18:07:11.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.629+0000 7ff376800700 1 -- 172.21.15.86:0/95584818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff370100f10 msgr2=0x7ff370195060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:07:11.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.629+0000 7ff376800700 1 --2- 172.21.15.86:0/95584818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff370100f10 0x7ff370195060 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7ff360006010 tx=0x7ff360013630 comp rx=0 tx=0).stop 2024-03-23T18:07:11.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.629+0000 7ff376800700 1 -- 172.21.15.86:0/95584818 shutdown_connections 2024-03-23T18:07:11.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.629+0000 7ff376800700 1 --2- 172.21.15.86:0/95584818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3701998f0 0x7ff370199d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:11.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.629+0000 7ff376800700 1 --2- 172.21.15.86:0/95584818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff370100f10 0x7ff370195060 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:11.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.629+0000 7ff376800700 1 --2- 172.21.15.86:0/95584818 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7ff3540a13e0 0x7ff3540a38a0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:11.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.629+0000 7ff376800700 1 --2- 172.21.15.86:0/95584818 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff370106030 0x7ff3701955a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:11.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.629+0000 7ff376800700 1 -- 172.21.15.86:0/95584818 >> 172.21.15.86:0/95584818 conn(0x7ff3700fc770 msgr2=0x7ff3700fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:07:11.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.629+0000 7ff376800700 1 -- 172.21.15.86:0/95584818 shutdown_connections 2024-03-23T18:07:11.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:11.629+0000 7ff376800700 1 -- 172.21.15.86:0/95584818 wait complete. 2024-03-23T18:07:12.245 INFO:tasks.workunit.client.0.smithi086.stdout: 10 154208 37.01 MB/sec execute 96 sec latency 175.523 ms 2024-03-23T18:07:12.311 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:07:12 smithi092 ceph-mon[79089]: pgmap v523: 97 pgs: 97 active+clean; 1.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 22 MiB/s wr, 1.07k op/s 2024-03-23T18:07:12.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:07:12 smithi152 ceph-mon[47949]: pgmap v523: 97 pgs: 97 active+clean; 1.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 22 MiB/s wr, 1.07k op/s 2024-03-23T18:07:12.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:07:12 smithi086 ceph-mon[38221]: pgmap v523: 97 pgs: 97 active+clean; 1.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 22 MiB/s wr, 1.07k op/s 2024-03-23T18:07:13.245 INFO:tasks.workunit.client.0.smithi086.stdout: 10 154728 37.04 MB/sec execute 97 sec latency 90.865 ms 2024-03-23T18:07:14.245 INFO:tasks.workunit.client.0.smithi086.stdout: 10 155345 36.92 MB/sec execute 98 sec latency 705.564 ms 2024-03-23T18:07:14.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:07:14 smithi092 ceph-mon[79089]: pgmap v524: 97 pgs: 97 active+clean; 1.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 22 MiB/s wr, 1.07k op/s 2024-03-23T18:07:14.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:07:14 smithi152 ceph-mon[47949]: pgmap v524: 97 pgs: 97 active+clean; 1.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 22 MiB/s wr, 1.07k op/s 2024-03-23T18:07:14.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:07:14 smithi086 ceph-mon[38221]: pgmap v524: 97 pgs: 97 active+clean; 1.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 22 MiB/s wr, 1.07k op/s 2024-03-23T18:07:15.245 INFO:tasks.workunit.client.0.smithi086.stdout: 10 155851 36.85 MB/sec execute 99 sec latency 1705.742 ms 2024-03-23T18:07:16.245 INFO:tasks.workunit.client.0.smithi086.stdout: 10 156385 36.73 MB/sec execute 100 sec latency 2705.789 ms 2024-03-23T18:07:16.310 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:07:16 smithi092 ceph-mon[79089]: pgmap v525: 97 pgs: 97 active+clean; 1.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 17 MiB/s wr, 1.06k op/s 2024-03-23T18:07:16.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:07:16 smithi152 ceph-mon[47949]: pgmap v525: 97 pgs: 97 active+clean; 1.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 17 MiB/s wr, 1.06k op/s 2024-03-23T18:07:16.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:07:16 smithi086 ceph-mon[38221]: pgmap v525: 97 pgs: 97 active+clean; 1.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 17 MiB/s wr, 1.06k op/s 2024-03-23T18:07:17.245 INFO:tasks.workunit.client.0.smithi086.stdout: 10 157029 36.76 MB/sec execute 101 sec latency 3705.837 ms 2024-03-23T18:07:17.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:07:17 smithi092 ceph-mon[79089]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:07:17.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:07:17 smithi152 ceph-mon[47949]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:07:17.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:07:17 smithi086 ceph-mon[38221]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:07:18.245 INFO:tasks.workunit.client.0.smithi086.stdout: 10 157516 36.66 MB/sec execute 102 sec latency 4705.882 ms 2024-03-23T18:07:18.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:07:18 smithi092 ceph-mon[79089]: pgmap v526: 97 pgs: 97 active+clean; 1.4 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 6.7 MiB/s wr, 935 op/s 2024-03-23T18:07:18.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:07:18 smithi152 ceph-mon[47949]: pgmap v526: 97 pgs: 97 active+clean; 1.4 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 6.7 MiB/s wr, 935 op/s 2024-03-23T18:07:18.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:07:18 smithi086 ceph-mon[38221]: pgmap v526: 97 pgs: 97 active+clean; 1.4 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 6.7 MiB/s wr, 935 op/s 2024-03-23T18:07:19.245 INFO:tasks.workunit.client.0.smithi086.stdout: 10 158055 36.55 MB/sec execute 103 sec latency 5705.933 ms 2024-03-23T18:07:19.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:07:19 smithi086 ceph-mon[38221]: pgmap v527: 97 pgs: 97 active+clean; 1.4 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 6.7 MiB/s wr, 935 op/s 2024-03-23T18:07:19.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:07:19 smithi092 ceph-mon[79089]: pgmap v527: 97 pgs: 97 active+clean; 1.4 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 6.7 MiB/s wr, 935 op/s 2024-03-23T18:07:19.585 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:07:19 smithi152 ceph-mon[47949]: pgmap v527: 97 pgs: 97 active+clean; 1.4 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 6.7 MiB/s wr, 935 op/s 2024-03-23T18:07:20.245 INFO:tasks.workunit.client.0.smithi086.stdout: 10 158639 36.50 MB/sec execute 104 sec latency 6705.981 ms 2024-03-23T18:07:21.245 INFO:tasks.workunit.client.0.smithi086.stdout: 10 159091 36.39 MB/sec execute 105 sec latency 7706.037 ms 2024-03-23T18:07:22.245 INFO:tasks.workunit.client.0.smithi086.stdout: 10 159477 36.20 MB/sec execute 106 sec latency 8706.094 ms 2024-03-23T18:07:22.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:07:21 smithi086 ceph-mon[38221]: pgmap v528: 97 pgs: 97 active+clean; 1.4 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 5.7 MiB/s wr, 496 op/s 2024-03-23T18:07:22.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:07:21 smithi086 ceph-mon[38221]: osdmap e108: 12 total, 12 up, 12 in 2024-03-23T18:07:22.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:07:21 smithi086 ceph-mon[38221]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:07:22.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:07:21 smithi092 ceph-mon[79089]: pgmap v528: 97 pgs: 97 active+clean; 1.4 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 5.7 MiB/s wr, 496 op/s 2024-03-23T18:07:22.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:07:21 smithi092 ceph-mon[79089]: osdmap e108: 12 total, 12 up, 12 in 2024-03-23T18:07:22.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:07:21 smithi092 ceph-mon[79089]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:07:22.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:07:21 smithi152 ceph-mon[47949]: pgmap v528: 97 pgs: 97 active+clean; 1.4 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 5.7 MiB/s wr, 496 op/s 2024-03-23T18:07:22.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:07:21 smithi152 ceph-mon[47949]: osdmap e108: 12 total, 12 up, 12 in 2024-03-23T18:07:22.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:07:21 smithi152 ceph-mon[47949]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:07:23.245 INFO:tasks.workunit.client.0.smithi086.stdout: 10 160087 36.17 MB/sec execute 107 sec latency 9706.150 ms 2024-03-23T18:07:24.245 INFO:tasks.workunit.client.0.smithi086.stdout: 10 160613 36.18 MB/sec execute 108 sec latency 10706.198 ms 2024-03-23T18:07:24.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:07:23 smithi086 ceph-mon[38221]: pgmap v530: 97 pgs: 97 active+clean; 1.4 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 6.1 MiB/s wr, 528 op/s 2024-03-23T18:07:24.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:07:23 smithi092 ceph-mon[79089]: pgmap v530: 97 pgs: 97 active+clean; 1.4 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 6.1 MiB/s wr, 528 op/s 2024-03-23T18:07:24.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:07:23 smithi152 ceph-mon[47949]: pgmap v530: 97 pgs: 97 active+clean; 1.4 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 6.1 MiB/s wr, 528 op/s 2024-03-23T18:07:24.842 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.842+0000 7fb1c4600700 1 -- 172.21.15.86:0/1960720446 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb1bc100ef0 msgr2=0x7fb1bc1012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:07:24.842 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.842+0000 7fb1c4600700 1 --2- 172.21.15.86:0/1960720446 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb1bc100ef0 0x7fb1bc1012d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb1b0009990 tx=0x7fb1b0009ca0 comp rx=0 tx=0).stop 2024-03-23T18:07:24.842 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.842+0000 7fb1c4600700 1 -- 172.21.15.86:0/1960720446 shutdown_connections 2024-03-23T18:07:24.842 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.842+0000 7fb1c4600700 1 --2- 172.21.15.86:0/1960720446 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1bc106010 0x7fb1bc10db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:24.843 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.842+0000 7fb1c4600700 1 --2- 172.21.15.86:0/1960720446 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb1bc100ef0 0x7fb1bc1012d0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:24.843 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.842+0000 7fb1c4600700 1 --2- 172.21.15.86:0/1960720446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1bc1018a0 0x7fb1bc1058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:24.843 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.842+0000 7fb1c4600700 1 -- 172.21.15.86:0/1960720446 >> 172.21.15.86:0/1960720446 conn(0x7fb1bc0fc750 msgr2=0x7fb1bc0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:07:24.843 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.842+0000 7fb1c4600700 1 -- 172.21.15.86:0/1960720446 shutdown_connections 2024-03-23T18:07:24.843 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.842+0000 7fb1c4600700 1 -- 172.21.15.86:0/1960720446 wait complete. 2024-03-23T18:07:24.843 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.843+0000 7fb1c4600700 1 Processor -- start 2024-03-23T18:07:24.843 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.843+0000 7fb1c4600700 1 -- start start 2024-03-23T18:07:24.844 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.844+0000 7fb1c4600700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb1bc100ef0 0x7fb1bc1a3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:07:24.845 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.844+0000 7fb1c4600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1bc106010 0x7fb1bc1a3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:07:24.845 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.844+0000 7fb1c4600700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1bc10cc80 0x7fb1bc10d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:07:24.845 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.844+0000 7fb1c4600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1bc10d700 con 0x7fb1bc106010 2024-03-23T18:07:24.845 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.844+0000 7fb1c4600700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1bc10d870 con 0x7fb1bc10cc80 2024-03-23T18:07:24.845 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.844+0000 7fb1c4600700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1bc10da30 con 0x7fb1bc100ef0 2024-03-23T18:07:24.845 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.844+0000 7fb1c2600700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1bc10cc80 0x7fb1bc10d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:07:24.845 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.844+0000 7fb1c2600700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1bc10cc80 0x7fb1bc10d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:51408/0 (socket says 172.21.15.86:51408) 2024-03-23T18:07:24.845 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.844+0000 7fb1c2600700 1 -- 172.21.15.86:0/1671098826 learned_addr learned my addr 172.21.15.86:0/1671098826 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:07:24.845 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.844+0000 7fb1c1c00700 1 --2- 172.21.15.86:0/1671098826 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb1bc100ef0 0x7fb1bc1a3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:07:24.845 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.845+0000 7fb1c1200700 1 --2- 172.21.15.86:0/1671098826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1bc106010 0x7fb1bc1a3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:07:24.846 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.845+0000 7fb1c2600700 1 -- 172.21.15.86:0/1671098826 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb1bc100ef0 msgr2=0x7fb1bc1a3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:07:24.846 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.845+0000 7fb1c2600700 1 --2- 172.21.15.86:0/1671098826 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb1bc100ef0 0x7fb1bc1a3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:24.846 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.845+0000 7fb1c2600700 1 -- 172.21.15.86:0/1671098826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1bc106010 msgr2=0x7fb1bc1a3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:07:24.846 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.845+0000 7fb1c2600700 1 --2- 172.21.15.86:0/1671098826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1bc106010 0x7fb1bc1a3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:24.846 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.845+0000 7fb1c2600700 1 -- 172.21.15.86:0/1671098826 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1b00095f0 con 0x7fb1bc10cc80 2024-03-23T18:07:24.847 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.845+0000 7fb1c2600700 1 --2- 172.21.15.86:0/1671098826 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1bc10cc80 0x7fb1bc10d130 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb1b400ef30 tx=0x7fb1b400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:07:24.847 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.845+0000 7fb1ba000700 1 -- 172.21.15.86:0/1671098826 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b4018070 con 0x7fb1bc10cc80 2024-03-23T18:07:24.847 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.845+0000 7fb1ba000700 1 -- 172.21.15.86:0/1671098826 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb1b400f040 con 0x7fb1bc10cc80 2024-03-23T18:07:24.847 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.845+0000 7fb1ba000700 1 -- 172.21.15.86:0/1671098826 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b400cdb0 con 0x7fb1bc10cc80 2024-03-23T18:07:24.847 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.846+0000 7fb1c4600700 1 -- 172.21.15.86:0/1671098826 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1bc1aa3b0 con 0x7fb1bc10cc80 2024-03-23T18:07:24.847 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.846+0000 7fb1c4600700 1 -- 172.21.15.86:0/1671098826 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1bc1aa840 con 0x7fb1bc10cc80 2024-03-23T18:07:24.850 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.850+0000 7fb1ba000700 1 -- 172.21.15.86:0/1671098826 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fb1b4003a00 con 0x7fb1bc10cc80 2024-03-23T18:07:24.851 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.852+0000 7fb1ba000700 1 --2- 172.21.15.86:0/1671098826 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fb1a40a1310 0x7fb1a40a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:07:24.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.852+0000 7fb1ba000700 1 -- 172.21.15.86:0/1671098826 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fb1b40cc790 con 0x7fb1bc10cc80 2024-03-23T18:07:24.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.853+0000 7fb1c1c00700 1 --2- 172.21.15.86:0/1671098826 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fb1a40a1310 0x7fb1a40a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:07:24.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.853+0000 7fb1d805fb80 1 Processor -- start 2024-03-23T18:07:24.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.853+0000 7fb1d805fb80 1 -- start start 2024-03-23T18:07:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.854+0000 7fb1d805fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55704c19e100 0x55704c19e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:07:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.854+0000 7fb1d805fb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55704c19ea20 0x55704c1a9390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:07:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.854+0000 7fb1d805fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55704c1a98d0 0x55704c1abcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:07:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.854+0000 7fb1d805fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55704c0f7930 con 0x55704c1a98d0 2024-03-23T18:07:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.854+0000 7fb1d805fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55704c0f52f0 con 0x55704c19e100 2024-03-23T18:07:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.854+0000 7fb1d805fb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55704c0f0be0 con 0x55704c19ea20 2024-03-23T18:07:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.854+0000 7fb1c2600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55704c1a98d0 0x55704c1abcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:07:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.854+0000 7fb1c2600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55704c1a98d0 0x55704c1abcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51390/0 (socket says 172.21.15.86:51390) 2024-03-23T18:07:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.854+0000 7fb1c2600700 1 -- 172.21.15.86:0/245210796 learned_addr learned my addr 172.21.15.86:0/245210796 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:07:24.856 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.854+0000 7fb1c1200700 1 --2- 172.21.15.86:0/245210796 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55704c19ea20 0x55704c1a9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:07:24.856 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.855+0000 7fb1c1c00700 1 --2- 172.21.15.86:0/245210796 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55704c19e100 0x55704c19e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:07:24.856 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.855+0000 7fb1c2600700 1 -- 172.21.15.86:0/245210796 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55704c19ea20 msgr2=0x55704c1a9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:07:24.856 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.855+0000 7fb1c1c00700 1 --2- 172.21.15.86:0/1671098826 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fb1a40a1310 0x7fb1a40a37d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb1b000b2e0 tx=0x7fb1b0005fd0 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:07:24.856 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.855+0000 7fb1c2600700 1 --2- 172.21.15.86:0/245210796 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55704c19ea20 0x55704c1a9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:24.856 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.855+0000 7fb1c2600700 1 -- 172.21.15.86:0/245210796 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55704c19e100 msgr2=0x55704c19e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:07:24.856 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.855+0000 7fb1c2600700 1 --2- 172.21.15.86:0/245210796 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55704c19e100 0x55704c19e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:24.856 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.855+0000 7fb1c2600700 1 -- 172.21.15.86:0/245210796 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55704bf72350 con 0x55704c1a98d0 2024-03-23T18:07:24.856 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.855+0000 7fb1c1c00700 1 --2- 172.21.15.86:0/245210796 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55704c19e100 0x55704c19e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T18:07:24.856 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.855+0000 7fb1c2600700 1 --2- 172.21.15.86:0/245210796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55704c1a98d0 0x55704c1abcc0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fb1b40a1810 tx=0x7fb1b40997a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=805e316f4e88664d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:07:24.856 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.855+0000 7fb1aea00700 1 -- 172.21.15.86:0/245210796 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b4004ad0 con 0x55704c1a98d0 2024-03-23T18:07:24.856 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.855+0000 7fb1aea00700 1 -- 172.21.15.86:0/245210796 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb1b401b320 con 0x55704c1a98d0 2024-03-23T18:07:24.856 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.855+0000 7fb1aea00700 1 -- 172.21.15.86:0/245210796 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b4099bb0 con 0x55704c1a98d0 2024-03-23T18:07:24.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.856+0000 7fb1d805fb80 1 -- 172.21.15.86:0/245210796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55704c1a98d0 msgr2=0x55704c1abcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:07:24.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.856+0000 7fb1d805fb80 1 --2- 172.21.15.86:0/245210796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55704c1a98d0 0x55704c1abcc0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fb1b40a1810 tx=0x7fb1b40997a0 comp rx=0 tx=0).stop 2024-03-23T18:07:24.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.857+0000 7fb1d805fb80 1 -- 172.21.15.86:0/245210796 shutdown_connections 2024-03-23T18:07:24.858 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.857+0000 7fb1d805fb80 1 --2- 172.21.15.86:0/245210796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55704c1a98d0 0x55704c1abcc0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:24.858 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.857+0000 7fb1d805fb80 1 --2- 172.21.15.86:0/245210796 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55704c19e100 0x55704c19e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:24.858 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.857+0000 7fb1d805fb80 1 --2- 172.21.15.86:0/245210796 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55704c19ea20 0x55704c1a9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:24.858 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.857+0000 7fb1d805fb80 1 -- 172.21.15.86:0/245210796 >> 172.21.15.86:0/245210796 conn(0x55704c0fe640 msgr2=0x55704c19a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:07:24.858 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.857+0000 7fb1d805fb80 1 -- 172.21.15.86:0/245210796 shutdown_connections 2024-03-23T18:07:24.858 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.857+0000 7fb1d805fb80 1 -- 172.21.15.86:0/245210796 wait complete. 2024-03-23T18:07:24.858 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.858+0000 7fb1d805fb80 1 -- start start 2024-03-23T18:07:24.859 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.858+0000 7fb1d805fb80 1 Processor -- start 2024-03-23T18:07:24.859 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.859+0000 7fb1d805fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55704c19e100 0x55704c1a6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:07:24.859 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.859+0000 7fb1d805fb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55704c19ea20 0x55704c1a6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:07:24.859 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.859+0000 7fb1d805fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55704c1a98d0 0x55704c1a70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:07:24.859 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.859+0000 7fb1d805fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55704c08cd20 con 0x55704c1a98d0 2024-03-23T18:07:24.859 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.859+0000 7fb1d805fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55704c08c9c0 con 0x55704c19e100 2024-03-23T18:07:24.859 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.859+0000 7fb1d805fb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55704bf74640 con 0x55704c19ea20 2024-03-23T18:07:24.859 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.859+0000 7fb1d805fb80 10 client.0 _ll_register_callbacks cb 0x55704c096410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T18:07:24.860 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.860+0000 7fb1c1c00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55704c19e100 0x55704c1a6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:07:24.860 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.860+0000 7fb1c1c00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55704c19e100 0x55704c1a6640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:51430/0 (socket says 172.21.15.86:51430) 2024-03-23T18:07:24.860 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.860+0000 7fb1c1c00700 1 -- 172.21.15.86:0/4163554311 learned_addr learned my addr 172.21.15.86:0/4163554311 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:07:24.861 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.860+0000 7fb1c1c00700 1 -- 172.21.15.86:0/4163554311 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55704c19ea20 msgr2=0x55704c1a6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:07:24.861 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.860+0000 7fb1c1c00700 1 --2- 172.21.15.86:0/4163554311 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55704c19ea20 0x55704c1a6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:24.861 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.860+0000 7fb1c1c00700 1 -- 172.21.15.86:0/4163554311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55704c1a98d0 msgr2=0x55704c1a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:07:24.861 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.860+0000 7fb1c1c00700 1 --2- 172.21.15.86:0/4163554311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55704c1a98d0 0x55704c1a70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:24.861 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.860+0000 7fb1c1c00700 1 -- 172.21.15.86:0/4163554311 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1bc1aa840 con 0x55704c19e100 2024-03-23T18:07:24.861 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.860+0000 7fb1c1c00700 1 --2- 172.21.15.86:0/4163554311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55704c19e100 0x55704c1a6640 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fb1b000bea0 tx=0x7fb1b0009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9135b3287437f0f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:07:24.861 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.860+0000 7fb1a3400700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T18:07:24.862 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.861+0000 7fb1a3400700 1 -- 172.21.15.86:0/4163554311 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b000f460 con 0x55704c19e100 2024-03-23T18:07:24.862 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.861+0000 7fb1a3400700 1 -- 172.21.15.86:0/4163554311 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fb1b000f5c0 con 0x55704c19e100 2024-03-23T18:07:24.862 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.861+0000 7fb1d805fb80 1 -- 172.21.15.86:0/4163554311 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55704c0b1320 con 0x55704c19e100 2024-03-23T18:07:24.862 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.861+0000 7fb1a3400700 1 -- 172.21.15.86:0/4163554311 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb1b000f7c0 con 0x55704c19e100 2024-03-23T18:07:24.862 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.861+0000 7fb1d805fb80 10 client.27344 fetch_fsmap learned FSMap version 87 2024-03-23T18:07:24.862 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.861+0000 7fb1d805fb80 1 -- 172.21.15.86:0/4163554311 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=4+}) v3 -- 0x55704bf1eb00 con 0x55704c19e100 2024-03-23T18:07:24.862 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.862+0000 7fb1a3400700 1 -- 172.21.15.86:0/4163554311 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb1b003c020 con 0x55704c19e100 2024-03-23T18:07:24.862 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.862+0000 7fb1a3400700 1 -- 172.21.15.86:0/4163554311 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 87) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb1b002fa30 con 0x55704c19e100 2024-03-23T18:07:24.863 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.863+0000 7fb1d805fb80 10 client.27344 fetch_fsmap finished waiting for FSMap version 87 2024-03-23T18:07:24.863 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.863+0000 7fb1d805fb80 10 client.27344 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T18:07:24.863 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.863+0000 7fb1d805fb80 20 client.27344 populate_metadata read hostname 'smithi086' 2024-03-23T18:07:24.863 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.863+0000 7fb1d805fb80 1 --2- 172.21.15.86:0/4163554311 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55704c1b9640 0x55704c1bba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:07:24.863 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.863+0000 7fb1d805fb80 4 client.27344 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T18:07:24.863 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.863+0000 7fb1d805fb80 1 -- 172.21.15.86:0/4163554311 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55704c0e96a0 con 0x55704c1b9640 2024-03-23T18:07:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.865+0000 7fb1c1200700 1 --2- 172.21.15.86:0/4163554311 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55704c1b9640 0x55704c1bba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:07:24.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.868+0000 7fb1c1200700 1 --2- 172.21.15.86:0/4163554311 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55704c1b9640 0x55704c1bba60 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aed4ba3064c7640b server_cookie=fc3e8df6ecbe4346 in_seq=0 out_seq=0 2024-03-23T18:07:24.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.868+0000 7fb1a3400700 10 client.27344 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T18:07:24.874 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.874+0000 7fb1a3400700 1 -- 172.21.15.86:0/4163554311 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55704c0e96a0 con 0x55704c1b9640 2024-03-23T18:07:24.875 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.874+0000 7fb1a3400700 10 client.27344 handle_command_reply: tid=0 2024-03-23T18:07:24.875 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.874+0000 7fb1d805fb80 1 client.27344 shutdown 2024-03-23T18:07:24.875 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.875+0000 7fb1d805fb80 1 -- 172.21.15.86:0/4163554311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55704c19e100 msgr2=0x55704c1a6640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:07:24.875 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.875+0000 7fb1d805fb80 1 --2- 172.21.15.86:0/4163554311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55704c19e100 0x55704c1a6640 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fb1b000bea0 tx=0x7fb1b0009ca0 comp rx=0 tx=0).stop 2024-03-23T18:07:24.875 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.875+0000 7fb1d805fb80 1 -- 172.21.15.86:0/4163554311 shutdown_connections 2024-03-23T18:07:24.875 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.875+0000 7fb1d805fb80 1 --2- 172.21.15.86:0/4163554311 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55704c1b9640 0x55704c1bba60 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:24.875 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.875+0000 7fb1d805fb80 1 --2- 172.21.15.86:0/4163554311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55704c1a98d0 0x55704c1a70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:24.875 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.875+0000 7fb1d805fb80 1 --2- 172.21.15.86:0/4163554311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55704c19e100 0x55704c1a6640 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:24.875 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.875+0000 7fb1d805fb80 1 --2- 172.21.15.86:0/4163554311 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55704c19ea20 0x55704c1a6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:24.876 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.875+0000 7fb1a3400700 0 client.27344 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T18:07:24.876 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.875+0000 7fb1d805fb80 1 -- 172.21.15.86:0/4163554311 >> 172.21.15.86:0/4163554311 conn(0x55704c0fe640 msgr2=0x55704c1a00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:07:24.876 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.875+0000 7fb1d805fb80 1 -- 172.21.15.86:0/4163554311 shutdown_connections 2024-03-23T18:07:24.876 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.875+0000 7fb1d805fb80 1 -- 172.21.15.86:0/4163554311 wait complete. 2024-03-23T18:07:24.877 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.875+0000 7fb1d805fb80 20 client.27344 trim_cache size 0 max 16384 2024-03-23T18:07:24.902 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.902+0000 7fb1d805fb80 1 Processor -- start 2024-03-23T18:07:24.903 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.902+0000 7fb1d805fb80 1 -- start start 2024-03-23T18:07:24.903 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.902+0000 7fb1d805fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55704c19e100 0x55704c1b05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:07:24.903 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.902+0000 7fb1d805fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55704c19ea20 0x55704c1abcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:07:24.903 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.902+0000 7fb1d805fb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55704c1a98d0 0x55704c1a4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:07:24.903 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.902+0000 7fb1d805fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55704c19b2d0 con 0x55704c19ea20 2024-03-23T18:07:24.903 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.902+0000 7fb1d805fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55704c19bac0 con 0x55704c19e100 2024-03-23T18:07:24.903 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.903+0000 7fb1d805fb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55704c1a7ad0 con 0x55704c1a98d0 2024-03-23T18:07:24.903 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.903+0000 7fb1c2600700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55704c1a98d0 0x55704c1a4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:07:24.903 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.903+0000 7fb1c2600700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55704c1a98d0 0x55704c1a4f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:49586/0 (socket says 172.21.15.86:49586) 2024-03-23T18:07:24.904 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.903+0000 7fb1c2600700 1 -- 172.21.15.86:0/205061241 learned_addr learned my addr 172.21.15.86:0/205061241 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:07:24.904 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.903+0000 7fb1c1200700 1 --2- 172.21.15.86:0/205061241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55704c19ea20 0x55704c1abcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:07:24.904 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.903+0000 7fb1c2600700 1 -- 172.21.15.86:0/205061241 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55704c19e100 msgr2=0x55704c1b05a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-23T18:07:24.904 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.903+0000 7fb1c2600700 1 --2- 172.21.15.86:0/205061241 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55704c19e100 0x55704c1b05a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:24.904 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.903+0000 7fb1c2600700 1 -- 172.21.15.86:0/205061241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55704c19ea20 msgr2=0x55704c1abcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:07:24.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.903+0000 7fb1c2600700 1 --2- 172.21.15.86:0/205061241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55704c19ea20 0x55704c1abcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:24.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.903+0000 7fb1c2600700 1 -- 172.21.15.86:0/205061241 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55704c0b6e40 con 0x55704c1a98d0 2024-03-23T18:07:24.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.904+0000 7fb1c2600700 1 --2- 172.21.15.86:0/205061241 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55704c1a98d0 0x55704c1a4f10 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb1b40ca500 tx=0x7fb1b401b490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3f0b863b1f33eac server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:07:24.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.904+0000 7fb1af400700 1 -- 172.21.15.86:0/205061241 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b40b1070 con 0x55704c1a98d0 2024-03-23T18:07:24.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.904+0000 7fb1af400700 1 -- 172.21.15.86:0/205061241 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb1b40b2270 con 0x55704c1a98d0 2024-03-23T18:07:24.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.904+0000 7fb1af400700 1 -- 172.21.15.86:0/205061241 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b4098e40 con 0x55704c1a98d0 2024-03-23T18:07:24.906 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.905+0000 7fb1d805fb80 1 -- 172.21.15.86:0/205061241 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55704c1a98d0 msgr2=0x55704c1a4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:07:24.906 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.905+0000 7fb1d805fb80 1 --2- 172.21.15.86:0/205061241 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55704c1a98d0 0x55704c1a4f10 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb1b40ca500 tx=0x7fb1b401b490 comp rx=0 tx=0).stop 2024-03-23T18:07:24.906 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.905+0000 7fb1d805fb80 1 -- 172.21.15.86:0/205061241 shutdown_connections 2024-03-23T18:07:24.906 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.905+0000 7fb1d805fb80 1 --2- 172.21.15.86:0/205061241 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55704c1a98d0 0x55704c1a4f10 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:24.906 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.905+0000 7fb1d805fb80 1 --2- 172.21.15.86:0/205061241 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55704c19e100 0x55704c1b05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:24.906 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.905+0000 7fb1d805fb80 1 --2- 172.21.15.86:0/205061241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55704c19ea20 0x55704c1abcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:24.906 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.905+0000 7fb1d805fb80 1 -- 172.21.15.86:0/205061241 >> 172.21.15.86:0/205061241 conn(0x55704c0fe640 msgr2=0x55704c1a1930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:07:24.906 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.906+0000 7fb1d805fb80 1 -- 172.21.15.86:0/205061241 shutdown_connections 2024-03-23T18:07:24.906 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.906+0000 7fb1d805fb80 1 -- 172.21.15.86:0/205061241 wait complete. 2024-03-23T18:07:24.907 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.907+0000 7fb1d805fb80 1 -- start start 2024-03-23T18:07:24.907 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.907+0000 7fb1d805fb80 1 Processor -- start 2024-03-23T18:07:24.908 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.908+0000 7fb1d805fb80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55704c19e100 0x55704c1ae580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:07:24.908 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.908+0000 7fb1d805fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55704c19ea20 0x55704c1aeac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:07:24.908 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.908+0000 7fb1d805fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55704c1a98d0 0x55704c1af000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:07:24.908 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.908+0000 7fb1d805fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55704c2062e0 con 0x55704c19ea20 2024-03-23T18:07:24.908 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.908+0000 7fb1d805fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55704c206450 con 0x55704c1a98d0 2024-03-23T18:07:24.908 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.908+0000 7fb1d805fb80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55704c206610 con 0x55704c19e100 2024-03-23T18:07:24.908 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.908+0000 7fb1d805fb80 10 client.0 _ll_register_callbacks cb 0x55704c096410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T18:07:24.909 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.909+0000 7fb1c1c00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55704c19e100 0x55704c1ae580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:07:24.909 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.909+0000 7fb1c2600700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55704c1a98d0 0x55704c1af000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:07:24.909 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.909+0000 7fb1c2600700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55704c1a98d0 0x55704c1af000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:51452/0 (socket says 172.21.15.86:51452) 2024-03-23T18:07:24.909 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.909+0000 7fb1c2600700 1 -- 172.21.15.86:0/719651441 learned_addr learned my addr 172.21.15.86:0/719651441 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:07:24.909 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.909+0000 7fb1c1200700 1 --2- 172.21.15.86:0/719651441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55704c19ea20 0x55704c1aeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:07:24.910 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.910+0000 7fb1c1200700 1 -- 172.21.15.86:0/719651441 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55704c19e100 msgr2=0x55704c1ae580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:07:24.910 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.910+0000 7fb1c1200700 1 --2- 172.21.15.86:0/719651441 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55704c19e100 0x55704c1ae580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:24.910 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.910+0000 7fb1c1200700 1 -- 172.21.15.86:0/719651441 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55704c1a98d0 msgr2=0x55704c1af000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:07:24.910 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.910+0000 7fb1c1200700 1 --2- 172.21.15.86:0/719651441 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55704c1a98d0 0x55704c1af000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:24.910 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.910+0000 7fb1c1200700 1 -- 172.21.15.86:0/719651441 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55704bf72350 con 0x55704c19ea20 2024-03-23T18:07:24.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.910+0000 7fb1c1200700 1 --2- 172.21.15.86:0/719651441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55704c19ea20 0x55704c1aeac0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x55704c1bc360 tx=0x7fb1a8019500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e45138a58f848b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:07:24.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.910+0000 7fb1a3400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-03-23T18:07:24.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.911+0000 7fb1a3400700 1 -- 172.21.15.86:0/719651441 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a8017040 con 0x55704c19ea20 2024-03-23T18:07:24.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.911+0000 7fb1a3400700 1 -- 172.21.15.86:0/719651441 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fb1a8005050 con 0x55704c19ea20 2024-03-23T18:07:24.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.912+0000 7fb1d805fb80 1 -- 172.21.15.86:0/719651441 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb1b401b320 con 0x55704c19ea20 2024-03-23T18:07:24.914 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.914+0000 7fb1a3400700 1 -- 172.21.15.86:0/719651441 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb1a8005210 con 0x55704c19ea20 2024-03-23T18:07:24.914 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.914+0000 7fb1d805fb80 10 client.17250 fetch_fsmap learned FSMap version 87 2024-03-23T18:07:24.915 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.915+0000 7fb1d805fb80 1 -- 172.21.15.86:0/719651441 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=4+}) v3 -- 0x55704bf76c70 con 0x55704c19ea20 2024-03-23T18:07:24.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.916+0000 7fb1a3400700 1 -- 172.21.15.86:0/719651441 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb1b401b320 con 0x55704c19ea20 2024-03-23T18:07:24.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.916+0000 7fb1a3400700 1 -- 172.21.15.86:0/719651441 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 87) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb1a80114d0 con 0x55704c19ea20 2024-03-23T18:07:24.918 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.917+0000 7fb1d805fb80 10 client.17250 fetch_fsmap finished waiting for FSMap version 87 2024-03-23T18:07:24.918 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.917+0000 7fb1d805fb80 10 client.17250 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T18:07:24.918 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.917+0000 7fb1d805fb80 20 client.17250 populate_metadata read hostname 'smithi086' 2024-03-23T18:07:24.918 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.918+0000 7fb1d805fb80 1 --2- 172.21.15.86:0/719651441 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55704c21d190 0x55704c21f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:07:24.919 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.919+0000 7fb1d805fb80 4 client.17250 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T18:07:24.919 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.919+0000 7fb1d805fb80 1 -- 172.21.15.86:0/719651441 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55704bf3fc90 con 0x55704c21d190 2024-03-23T18:07:24.919 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.920+0000 7fb1c1c00700 1 --2- 172.21.15.86:0/719651441 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55704c21d190 0x55704c21f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:07:24.921 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.920+0000 7fb1c1c00700 1 --2- 172.21.15.86:0/719651441 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55704c21d190 0x55704c21f580 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8aff1c8ab95b79a7 server_cookie=94fb4eb7a9ef0be7 in_seq=0 out_seq=0 2024-03-23T18:07:24.921 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:24.920+0000 7fb1a3400700 10 client.17250 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T18:07:25.179 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:25.179+0000 7fb1a3400700 1 -- 172.21.15.86:0/719651441 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55704bf3fc90 con 0x55704c21d190 2024-03-23T18:07:25.179 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:25.179+0000 7fb1a3400700 10 client.17250 handle_command_reply: tid=0 2024-03-23T18:07:25.179 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:25.179+0000 7fb1d805fb80 1 client.17250 shutdown 2024-03-23T18:07:25.179 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:25.179+0000 7fb1d805fb80 1 -- 172.21.15.86:0/719651441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55704c19ea20 msgr2=0x55704c1aeac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:07:25.179 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:25.179+0000 7fb1d805fb80 1 --2- 172.21.15.86:0/719651441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55704c19ea20 0x55704c1aeac0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x55704c1bc360 tx=0x7fb1a8019500 comp rx=0 tx=0).stop 2024-03-23T18:07:25.180 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:25.180+0000 7fb1d805fb80 1 -- 172.21.15.86:0/719651441 shutdown_connections 2024-03-23T18:07:25.180 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:25.180+0000 7fb1d805fb80 1 --2- 172.21.15.86:0/719651441 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x55704c21d190 0x55704c21f580 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:25.180 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:25.180+0000 7fb1a3400700 0 client.17250 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T18:07:25.180 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:25.180+0000 7fb1d805fb80 1 --2- 172.21.15.86:0/719651441 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55704c1a98d0 0x55704c1af000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:25.180 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:25.180+0000 7fb1d805fb80 1 --2- 172.21.15.86:0/719651441 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55704c19e100 0x55704c1ae580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:25.180 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:25.180+0000 7fb1d805fb80 1 --2- 172.21.15.86:0/719651441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55704c19ea20 0x55704c1aeac0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:25.181 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:25.180+0000 7fb1d805fb80 1 -- 172.21.15.86:0/719651441 >> 172.21.15.86:0/719651441 conn(0x55704c0fe640 msgr2=0x55704c1a4e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:07:25.181 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:25.181+0000 7fb1d805fb80 1 -- 172.21.15.86:0/719651441 shutdown_connections 2024-03-23T18:07:25.181 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:25.181+0000 7fb1d805fb80 1 -- 172.21.15.86:0/719651441 wait complete. 2024-03-23T18:07:25.181 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:25.182+0000 7fb1d805fb80 20 client.17250 trim_cache size 0 max 16384 2024-03-23T18:07:25.182 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-03-23T18:07:25.182 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-03-23T18:07:25.182 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-03-23T18:07:25.182 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-03-23T18:07:25.183 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:25.183+0000 7fb1c4600700 1 -- 172.21.15.86:0/1671098826 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fb1a40a1310 msgr2=0x7fb1a40a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:07:25.183 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:25.183+0000 7fb1c4600700 1 --2- 172.21.15.86:0/1671098826 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fb1a40a1310 0x7fb1a40a37d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb1b000b2e0 tx=0x7fb1b0005fd0 comp rx=0 tx=0).stop 2024-03-23T18:07:25.183 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:25.183+0000 7fb1c4600700 1 -- 172.21.15.86:0/1671098826 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1bc10cc80 msgr2=0x7fb1bc10d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:07:25.183 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:25.183+0000 7fb1c4600700 1 --2- 172.21.15.86:0/1671098826 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1bc10cc80 0x7fb1bc10d130 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb1b400ef30 tx=0x7fb1b400c3a0 comp rx=0 tx=0).stop 2024-03-23T18:07:25.183 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:25.183+0000 7fb1c4600700 1 -- 172.21.15.86:0/1671098826 shutdown_connections 2024-03-23T18:07:25.183 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:25.183+0000 7fb1c4600700 1 --2- 172.21.15.86:0/1671098826 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1bc10cc80 0x7fb1bc10d130 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:25.183 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:25.183+0000 7fb1c4600700 1 --2- 172.21.15.86:0/1671098826 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fb1a40a1310 0x7fb1a40a37d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:25.183 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:25.183+0000 7fb1c4600700 1 --2- 172.21.15.86:0/1671098826 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb1bc100ef0 0x7fb1bc1a3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:25.184 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:25.183+0000 7fb1c4600700 1 --2- 172.21.15.86:0/1671098826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1bc106010 0x7fb1bc1a3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:25.184 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:25.183+0000 7fb1c4600700 1 -- 172.21.15.86:0/1671098826 >> 172.21.15.86:0/1671098826 conn(0x7fb1bc0fc750 msgr2=0x7fb1bc0fda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:07:25.184 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:25.183+0000 7fb1c4600700 1 -- 172.21.15.86:0/1671098826 shutdown_connections 2024-03-23T18:07:25.184 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:25.183+0000 7fb1c4600700 1 -- 172.21.15.86:0/1671098826 wait complete. 2024-03-23T18:07:25.245 INFO:tasks.workunit.client.0.smithi086.stdout: 10 161033 35.94 MB/sec execute 109 sec latency 11317.505 ms 2024-03-23T18:07:26.245 INFO:tasks.workunit.client.0.smithi086.stdout: 10 161896 36.08 MB/sec execute 110 sec latency 10998.999 ms 2024-03-23T18:07:26.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:07:25 smithi086 ceph-mon[38221]: pgmap v531: 97 pgs: 97 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 6.3 MiB/s wr, 504 op/s 2024-03-23T18:07:26.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:07:25 smithi092 ceph-mon[79089]: pgmap v531: 97 pgs: 97 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 6.3 MiB/s wr, 504 op/s 2024-03-23T18:07:26.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:07:25 smithi152 ceph-mon[47949]: pgmap v531: 97 pgs: 97 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 6.3 MiB/s wr, 504 op/s 2024-03-23T18:07:27.245 INFO:tasks.workunit.client.0.smithi086.stdout: 10 162689 36.13 MB/sec execute 111 sec latency 74.641 ms 2024-03-23T18:07:28.245 INFO:tasks.workunit.client.0.smithi086.stdout: 10 163509 36.27 MB/sec execute 112 sec latency 59.510 ms 2024-03-23T18:07:28.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:07:27 smithi086 ceph-mon[38221]: pgmap v532: 97 pgs: 97 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 6.3 MiB/s wr, 470 op/s 2024-03-23T18:07:28.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:07:27 smithi092 ceph-mon[79089]: pgmap v532: 97 pgs: 97 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 6.3 MiB/s wr, 470 op/s 2024-03-23T18:07:28.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:07:27 smithi152 ceph-mon[47949]: pgmap v532: 97 pgs: 97 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 6.3 MiB/s wr, 470 op/s 2024-03-23T18:07:29.245 INFO:tasks.workunit.client.0.smithi086.stdout: 10 164255 36.30 MB/sec execute 113 sec latency 68.738 ms 2024-03-23T18:07:30.246 INFO:tasks.workunit.client.0.smithi086.stdout: 10 164878 36.29 MB/sec execute 114 sec latency 95.882 ms 2024-03-23T18:07:30.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:07:29 smithi086 ceph-mon[38221]: pgmap v533: 97 pgs: 97 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 6.3 MiB/s wr, 470 op/s 2024-03-23T18:07:30.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:07:29 smithi092 ceph-mon[79089]: pgmap v533: 97 pgs: 97 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 6.3 MiB/s wr, 470 op/s 2024-03-23T18:07:30.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:07:29 smithi152 ceph-mon[47949]: pgmap v533: 97 pgs: 97 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 6.3 MiB/s wr, 470 op/s 2024-03-23T18:07:31.246 INFO:tasks.workunit.client.0.smithi086.stdout: 10 165618 36.21 MB/sec execute 115 sec latency 68.292 ms 2024-03-23T18:07:32.246 INFO:tasks.workunit.client.0.smithi086.stdout: 10 166514 36.33 MB/sec execute 116 sec latency 73.560 ms 2024-03-23T18:07:32.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:07:32 smithi086 ceph-mon[38221]: pgmap v534: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 8.4 MiB/s wr, 611 op/s 2024-03-23T18:07:32.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:07:32 smithi092 ceph-mon[79089]: pgmap v534: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 8.4 MiB/s wr, 611 op/s 2024-03-23T18:07:32.338 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:07:32 smithi152 ceph-mon[47949]: pgmap v534: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 8.4 MiB/s wr, 611 op/s 2024-03-23T18:07:33.246 INFO:tasks.workunit.client.0.smithi086.stdout: 10 167462 36.53 MB/sec execute 117 sec latency 78.755 ms 2024-03-23T18:07:34.246 INFO:tasks.workunit.client.0.smithi086.stdout: 10 168117 36.51 MB/sec execute 118 sec latency 62.407 ms 2024-03-23T18:07:34.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:07:34 smithi086 ceph-mon[38221]: pgmap v535: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 7.2 MiB/s wr, 524 op/s 2024-03-23T18:07:34.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:07:34 smithi092 ceph-mon[79089]: pgmap v535: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 7.2 MiB/s wr, 524 op/s 2024-03-23T18:07:34.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:07:34 smithi152 ceph-mon[47949]: pgmap v535: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 7.2 MiB/s wr, 524 op/s 2024-03-23T18:07:35.246 INFO:tasks.workunit.client.0.smithi086.stdout: 10 168707 36.43 MB/sec execute 119 sec latency 123.941 ms 2024-03-23T18:07:36.246 INFO:tasks.workunit.client.0.smithi086.stdout: 10 169469 36.47 MB/sec execute 120 sec latency 75.867 ms 2024-03-23T18:07:36.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:07:36 smithi086 ceph-mon[38221]: pgmap v536: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.1 MiB/s wr, 662 op/s 2024-03-23T18:07:36.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:07:36 smithi086 ceph-mon[38221]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:07:36.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:07:36 smithi092 ceph-mon[79089]: pgmap v536: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.1 MiB/s wr, 662 op/s 2024-03-23T18:07:36.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:07:36 smithi092 ceph-mon[79089]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:07:36.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:07:36 smithi152 ceph-mon[47949]: pgmap v536: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.1 MiB/s wr, 662 op/s 2024-03-23T18:07:36.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:07:36 smithi152 ceph-mon[47949]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:07:37.246 INFO:tasks.workunit.client.0.smithi086.stdout: 10 170232 36.53 MB/sec execute 121 sec latency 70.843 ms 2024-03-23T18:07:38.246 INFO:tasks.workunit.client.0.smithi086.stdout: 10 170917 36.55 MB/sec execute 122 sec latency 103.308 ms 2024-03-23T18:07:38.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:07:38 smithi092 ceph-mon[79089]: pgmap v537: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.9 MiB/s wr, 655 op/s 2024-03-23T18:07:38.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:07:38 smithi152 ceph-mon[47949]: pgmap v537: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.9 MiB/s wr, 655 op/s 2024-03-23T18:07:38.406 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.405+0000 7f9f91c00700 1 -- 172.21.15.86:0/2358922687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f8c0680e0 msgr2=0x7f9f8c068560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:07:38.406 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.405+0000 7f9f91c00700 1 --2- 172.21.15.86:0/2358922687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f8c0680e0 0x7f9f8c068560 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f9f78009990 tx=0x7f9f78009ca0 comp rx=0 tx=0).stop 2024-03-23T18:07:38.406 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.405+0000 7f9f91c00700 1 -- 172.21.15.86:0/2358922687 shutdown_connections 2024-03-23T18:07:38.406 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.405+0000 7f9f91c00700 1 --2- 172.21.15.86:0/2358922687 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f8c068aa0 0x7f9f8c10dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:38.406 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.405+0000 7f9f91c00700 1 --2- 172.21.15.86:0/2358922687 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9f8c106e10 0x7f9f8c1071f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:38.406 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.405+0000 7f9f91c00700 1 --2- 172.21.15.86:0/2358922687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f8c0680e0 0x7f9f8c068560 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:38.406 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.405+0000 7f9f91c00700 1 -- 172.21.15.86:0/2358922687 >> 172.21.15.86:0/2358922687 conn(0x7f9f8c076d50 msgr2=0x7f9f8c077160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:07:38.406 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.406+0000 7f9f91c00700 1 -- 172.21.15.86:0/2358922687 shutdown_connections 2024-03-23T18:07:38.407 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.406+0000 7f9f91c00700 1 -- 172.21.15.86:0/2358922687 wait complete. 2024-03-23T18:07:38.407 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.407+0000 7f9f91c00700 1 Processor -- start 2024-03-23T18:07:38.407 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.407+0000 7f9f91c00700 1 -- start start 2024-03-23T18:07:38.408 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.408+0000 7f9f91c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f8c0680e0 0x7f9f8c194f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:07:38.408 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.408+0000 7f9f91c00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f8c068aa0 0x7f9f8c195480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:07:38.408 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.408+0000 7f9f91c00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9f8c106e10 0x7f9f8c199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:07:38.408 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.408+0000 7f9f91c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8c199e60 con 0x7f9f8c0680e0 2024-03-23T18:07:38.408 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.408+0000 7f9f91c00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8c199fd0 con 0x7f9f8c068aa0 2024-03-23T18:07:38.408 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.408+0000 7f9f91c00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8c19a190 con 0x7f9f8c106e10 2024-03-23T18:07:38.409 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.409+0000 7f9f8b400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9f8c106e10 0x7f9f8c199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:07:38.409 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.409+0000 7f9f8aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f8c0680e0 0x7f9f8c194f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:07:38.409 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.409+0000 7f9f8b400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9f8c106e10 0x7f9f8c199820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:51038/0 (socket says 172.21.15.86:51038) 2024-03-23T18:07:38.409 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.409+0000 7f9f8b400700 1 -- 172.21.15.86:0/3076480510 learned_addr learned my addr 172.21.15.86:0/3076480510 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:07:38.409 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.409+0000 7f9f8a000700 1 --2- 172.21.15.86:0/3076480510 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f8c068aa0 0x7f9f8c195480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:07:38.409 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.409+0000 7f9f8b400700 1 -- 172.21.15.86:0/3076480510 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f8c068aa0 msgr2=0x7f9f8c195480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:07:38.409 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.409+0000 7f9f8b400700 1 --2- 172.21.15.86:0/3076480510 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f8c068aa0 0x7f9f8c195480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:38.409 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.409+0000 7f9f8b400700 1 -- 172.21.15.86:0/3076480510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f8c0680e0 msgr2=0x7f9f8c194f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:07:38.410 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.409+0000 7f9f8b400700 1 --2- 172.21.15.86:0/3076480510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f8c0680e0 0x7f9f8c194f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:38.410 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.409+0000 7f9f8b400700 1 -- 172.21.15.86:0/3076480510 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f7c00d930 con 0x7f9f8c106e10 2024-03-23T18:07:38.410 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.409+0000 7f9f8aa00700 1 --2- 172.21.15.86:0/3076480510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f8c0680e0 0x7f9f8c194f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T18:07:38.410 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.410+0000 7f9f8b400700 1 --2- 172.21.15.86:0/3076480510 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9f8c106e10 0x7f9f8c199820 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f9f84007ee0 tx=0x7f9f8400d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:07:38.410 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.410+0000 7f9f83400700 1 -- 172.21.15.86:0/3076480510 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f84017070 con 0x7f9f8c106e10 2024-03-23T18:07:38.411 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.410+0000 7f9f91c00700 1 -- 172.21.15.86:0/3076480510 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f780095f0 con 0x7f9f8c106e10 2024-03-23T18:07:38.411 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.410+0000 7f9f83400700 1 -- 172.21.15.86:0/3076480510 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f8400f040 con 0x7f9f8c106e10 2024-03-23T18:07:38.411 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.410+0000 7f9f83400700 1 -- 172.21.15.86:0/3076480510 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f84005580 con 0x7f9f8c106e10 2024-03-23T18:07:38.411 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.410+0000 7f9f91c00700 1 -- 172.21.15.86:0/3076480510 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f8c1a1ad0 con 0x7f9f8c106e10 2024-03-23T18:07:38.414 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.414+0000 7f9f83400700 1 -- 172.21.15.86:0/3076480510 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f9f8400db20 con 0x7f9f8c106e10 2024-03-23T18:07:38.418 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.418+0000 7f9f83400700 1 --2- 172.21.15.86:0/3076480510 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f9f700a1390 0x7f9f700a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:07:38.418 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.418+0000 7f9f83400700 1 -- 172.21.15.86:0/3076480510 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f9f840cc420 con 0x7f9f8c106e10 2024-03-23T18:07:38.419 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.418+0000 7f9f8aa00700 1 --2- 172.21.15.86:0/3076480510 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f9f700a1390 0x7f9f700a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:07:38.419 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.419+0000 7f9fa5720b80 1 Processor -- start 2024-03-23T18:07:38.419 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.419+0000 7f9f8aa00700 1 --2- 172.21.15.86:0/3076480510 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f9f700a1390 0x7f9f700a3850 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c012740 tx=0x7f9f7c00f040 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:07:38.420 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.420+0000 7f9fa5720b80 1 -- start start 2024-03-23T18:07:38.420 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.420+0000 7f9fa5720b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559effdf1100 0x559effdf14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:07:38.420 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.420+0000 7f9fa5720b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559effdf1a20 0x559effdfc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:07:38.420 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.420+0000 7f9fa5720b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559effdfc8d0 0x559effdfecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:07:38.421 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.420+0000 7f9fa5720b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x559effd4a930 con 0x559effdf1100 2024-03-23T18:07:38.421 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.420+0000 7f9fa5720b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x559effd482f0 con 0x559effdfc8d0 2024-03-23T18:07:38.421 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.420+0000 7f9fa5720b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x559effd43be0 con 0x559effdf1a20 2024-03-23T18:07:38.422 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.421+0000 7f9f8b400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559effdfc8d0 0x559effdfecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:07:38.422 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.421+0000 7f9f8aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559effdf1100 0x559effdf14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:07:38.422 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.421+0000 7f9f8b400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559effdfc8d0 0x559effdfecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:57922/0 (socket says 172.21.15.86:57922) 2024-03-23T18:07:38.422 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.421+0000 7f9f8a000700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559effdf1a20 0x559effdfc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:07:38.422 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.421+0000 7f9f8b400700 1 -- 172.21.15.86:0/1467042950 learned_addr learned my addr 172.21.15.86:0/1467042950 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:07:38.422 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.421+0000 7f9f8b400700 1 -- 172.21.15.86:0/1467042950 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559effdf1a20 msgr2=0x559effdfc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:07:38.422 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.421+0000 7f9f8b400700 1 --2- 172.21.15.86:0/1467042950 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559effdf1a20 0x559effdfc390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:38.422 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.421+0000 7f9f8b400700 1 -- 172.21.15.86:0/1467042950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559effdf1100 msgr2=0x559effdf14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:07:38.422 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.421+0000 7f9f8b400700 1 --2- 172.21.15.86:0/1467042950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559effdf1100 0x559effdf14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:38.422 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.421+0000 7f9f8b400700 1 -- 172.21.15.86:0/1467042950 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559effbc5350 con 0x559effdfc8d0 2024-03-23T18:07:38.423 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.422+0000 7f9f8b400700 1 --2- 172.21.15.86:0/1467042950 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559effdfc8d0 0x559effdfecc0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f9f84095f60 tx=0x7f9f84095230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af2de2a59e40f538 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:07:38.423 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.422+0000 7f9f77e00700 1 -- 172.21.15.86:0/1467042950 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f8409d320 con 0x559effdfc8d0 2024-03-23T18:07:38.423 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.422+0000 7f9f77e00700 1 -- 172.21.15.86:0/1467042950 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f840a1970 con 0x559effdfc8d0 2024-03-23T18:07:38.423 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.422+0000 7f9f77e00700 1 -- 172.21.15.86:0/1467042950 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f840c9db0 con 0x559effdfc8d0 2024-03-23T18:07:38.424 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.423+0000 7f9fa5720b80 1 -- 172.21.15.86:0/1467042950 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559effdfc8d0 msgr2=0x559effdfecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:07:38.424 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.423+0000 7f9fa5720b80 1 --2- 172.21.15.86:0/1467042950 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559effdfc8d0 0x559effdfecc0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f9f84095f60 tx=0x7f9f84095230 comp rx=0 tx=0).stop 2024-03-23T18:07:38.424 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.424+0000 7f9fa5720b80 1 -- 172.21.15.86:0/1467042950 shutdown_connections 2024-03-23T18:07:38.424 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.424+0000 7f9fa5720b80 1 --2- 172.21.15.86:0/1467042950 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559effdfc8d0 0x559effdfecc0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:38.424 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.424+0000 7f9fa5720b80 1 --2- 172.21.15.86:0/1467042950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559effdf1100 0x559effdf14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:38.424 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.424+0000 7f9fa5720b80 1 --2- 172.21.15.86:0/1467042950 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559effdf1a20 0x559effdfc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:38.424 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.424+0000 7f9fa5720b80 1 -- 172.21.15.86:0/1467042950 >> 172.21.15.86:0/1467042950 conn(0x559effd51640 msgr2=0x559effded520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:07:38.424 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.424+0000 7f9fa5720b80 1 -- 172.21.15.86:0/1467042950 shutdown_connections 2024-03-23T18:07:38.425 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.424+0000 7f9fa5720b80 1 -- 172.21.15.86:0/1467042950 wait complete. 2024-03-23T18:07:38.425 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.425+0000 7f9fa5720b80 1 -- start start 2024-03-23T18:07:38.425 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.425+0000 7f9fa5720b80 1 Processor -- start 2024-03-23T18:07:38.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.426+0000 7f9fa5720b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559effdf1100 0x559effdfc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:07:38.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.426+0000 7f9fa5720b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559effdf1a20 0x559effdf9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:07:38.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.426+0000 7f9fa5720b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559effdfc8d0 0x559effdf9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:07:38.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.426+0000 7f9fa5720b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x559effcdfd20 con 0x559effdf1100 2024-03-23T18:07:38.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.426+0000 7f9fa5720b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x559effcdf9c0 con 0x559effdf1a20 2024-03-23T18:07:38.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.426+0000 7f9fa5720b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x559effbc7640 con 0x559effdfc8d0 2024-03-23T18:07:38.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.426+0000 7f9fa5720b80 10 client.0 _ll_register_callbacks cb 0x559effce9410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T18:07:38.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.426+0000 7f9f8aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559effdf1100 0x559effdfc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:07:38.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.427+0000 7f9f8aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559effdf1100 0x559effdfc310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47376/0 (socket says 172.21.15.86:47376) 2024-03-23T18:07:38.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.427+0000 7f9f8aa00700 1 -- 172.21.15.86:0/2627672262 learned_addr learned my addr 172.21.15.86:0/2627672262 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:07:38.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.427+0000 7f9f8b400700 1 --2- 172.21.15.86:0/2627672262 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559effdfc8d0 0x559effdf9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:07:38.429 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.427+0000 7f9f8b400700 1 -- 172.21.15.86:0/2627672262 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559effdf1a20 msgr2=0x559effdf9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:07:38.429 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.427+0000 7f9f8b400700 1 --2- 172.21.15.86:0/2627672262 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559effdf1a20 0x559effdf9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:38.429 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.427+0000 7f9f8b400700 1 -- 172.21.15.86:0/2627672262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559effdf1100 msgr2=0x559effdfc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:07:38.429 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.427+0000 7f9f8b400700 1 --2- 172.21.15.86:0/2627672262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559effdf1100 0x559effdfc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:38.430 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.427+0000 7f9f8b400700 1 -- 172.21.15.86:0/2627672262 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f8c1a1ad0 con 0x559effdfc8d0 2024-03-23T18:07:38.430 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.427+0000 7f9f8b400700 1 --2- 172.21.15.86:0/2627672262 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559effdfc8d0 0x559effdf9d40 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9f84099480 tx=0x7f9f8409d170 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d60aa3b550dc9f02 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:07:38.430 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.427+0000 7f9f74c00700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-03-23T18:07:38.430 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.428+0000 7f9f74c00700 1 -- 172.21.15.86:0/2627672262 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f840976f0 con 0x559effdfc8d0 2024-03-23T18:07:38.430 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.428+0000 7f9f74c00700 1 -- 172.21.15.86:0/2627672262 <== mon.1 v2:172.21.15.152:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f9f840c9db0 con 0x559effdfc8d0 2024-03-23T18:07:38.430 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.428+0000 7f9fa5720b80 1 -- 172.21.15.86:0/2627672262 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559effd04320 con 0x559effdfc8d0 2024-03-23T18:07:38.430 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.428+0000 7f9f74c00700 1 -- 172.21.15.86:0/2627672262 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f840c9f70 con 0x559effdfc8d0 2024-03-23T18:07:38.430 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.428+0000 7f9fa5720b80 10 client.27325 fetch_fsmap learned FSMap version 88 2024-03-23T18:07:38.430 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.428+0000 7f9fa5720b80 1 -- 172.21.15.86:0/2627672262 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=4+}) v3 -- 0x559effb71b00 con 0x559effdfc8d0 2024-03-23T18:07:38.430 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.428+0000 7f9f74c00700 1 -- 172.21.15.86:0/2627672262 <== mon.1 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559effd04320 con 0x559effdfc8d0 2024-03-23T18:07:38.430 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.428+0000 7f9f74c00700 1 -- 172.21.15.86:0/2627672262 <== mon.1 v2:172.21.15.152:3300/0 5 ==== fsmap(e 88) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f9f840989e0 con 0x559effdfc8d0 2024-03-23T18:07:38.430 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.429+0000 7f9fa5720b80 10 client.27325 fetch_fsmap finished waiting for FSMap version 88 2024-03-23T18:07:38.430 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.429+0000 7f9fa5720b80 10 client.27325 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T18:07:38.430 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.429+0000 7f9fa5720b80 20 client.27325 populate_metadata read hostname 'smithi086' 2024-03-23T18:07:38.431 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.429+0000 7f9fa5720b80 1 --2- 172.21.15.86:0/2627672262 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x559effe1d250 0x559effe1f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:07:38.431 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.429+0000 7f9fa5720b80 4 client.27325 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T18:07:38.431 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.429+0000 7f9fa5720b80 1 -- 172.21.15.86:0/2627672262 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559effd3c6a0 con 0x559effe1d250 2024-03-23T18:07:38.431 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.429+0000 7f9f8aa00700 1 --2- 172.21.15.86:0/2627672262 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x559effe1d250 0x559effe1f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:07:38.431 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.429+0000 7f9f74c00700 10 client.27325 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T18:07:38.431 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.429+0000 7f9f8aa00700 1 --2- 172.21.15.86:0/2627672262 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x559effe1d250 0x559effe1f640 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc94cb1fa6f7ec79 server_cookie=3a2aa4072243ffa2 in_seq=0 out_seq=0 2024-03-23T18:07:38.433 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.432+0000 7f9f74c00700 1 -- 172.21.15.86:0/2627672262 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x559effd3c6a0 con 0x559effe1d250 2024-03-23T18:07:38.433 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.432+0000 7f9f74c00700 10 client.27325 handle_command_reply: tid=0 2024-03-23T18:07:38.433 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.433+0000 7f9fa5720b80 1 client.27325 shutdown 2024-03-23T18:07:38.433 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.433+0000 7f9fa5720b80 1 -- 172.21.15.86:0/2627672262 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559effdfc8d0 msgr2=0x559effdf9d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:07:38.434 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.433+0000 7f9fa5720b80 1 --2- 172.21.15.86:0/2627672262 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559effdfc8d0 0x559effdf9d40 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9f84099480 tx=0x7f9f8409d170 comp rx=0 tx=0).stop 2024-03-23T18:07:38.434 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.433+0000 7f9fa5720b80 1 -- 172.21.15.86:0/2627672262 shutdown_connections 2024-03-23T18:07:38.434 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.433+0000 7f9fa5720b80 1 --2- 172.21.15.86:0/2627672262 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x559effe1d250 0x559effe1f640 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:38.434 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.433+0000 7f9fa5720b80 1 --2- 172.21.15.86:0/2627672262 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559effdfc8d0 0x559effdf9d40 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:38.434 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.433+0000 7f9fa5720b80 1 --2- 172.21.15.86:0/2627672262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559effdf1100 0x559effdfc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:38.434 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.433+0000 7f9fa5720b80 1 --2- 172.21.15.86:0/2627672262 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559effdf1a20 0x559effdf9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:38.434 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.434+0000 7f9f74c00700 0 client.27325 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T18:07:38.434 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.434+0000 7f9fa5720b80 1 -- 172.21.15.86:0/2627672262 >> 172.21.15.86:0/2627672262 conn(0x559effd51640 msgr2=0x559effdf30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:07:38.435 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.434+0000 7f9fa5720b80 1 -- 172.21.15.86:0/2627672262 shutdown_connections 2024-03-23T18:07:38.435 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.434+0000 7f9fa5720b80 1 -- 172.21.15.86:0/2627672262 wait complete. 2024-03-23T18:07:38.435 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.434+0000 7f9fa5720b80 20 client.27325 trim_cache size 0 max 16384 2024-03-23T18:07:38.457 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.458+0000 7f9fa5720b80 1 Processor -- start 2024-03-23T18:07:38.458 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.458+0000 7f9fa5720b80 1 -- start start 2024-03-23T18:07:38.458 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.458+0000 7f9fa5720b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559effdf1100 0x559effe145b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:07:38.458 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.458+0000 7f9fa5720b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559effdf1a20 0x559effdf9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:07:38.458 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.458+0000 7f9fa5720b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559effdfc8d0 0x559effde7500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:07:38.459 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.458+0000 7f9fa5720b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x559effdfac70 con 0x559effdfc8d0 2024-03-23T18:07:38.459 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.458+0000 7f9fa5720b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x559effdf23b0 con 0x559effdf1100 2024-03-23T18:07:38.459 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.458+0000 7f9fa5720b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x559effdfa530 con 0x559effdf1a20 2024-03-23T18:07:38.460 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.458+0000 7f9f8a000700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559effdf1a20 0x559effdf9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:07:38.460 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.459+0000 7f9f8aa00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559effdf1100 0x559effe145b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:07:38.460 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.459+0000 7f9f8a000700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559effdf1a20 0x559effdf9ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:51050/0 (socket says 172.21.15.86:51050) 2024-03-23T18:07:38.460 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.459+0000 7f9f8a000700 1 -- 172.21.15.86:0/1851613045 learned_addr learned my addr 172.21.15.86:0/1851613045 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:07:38.460 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.459+0000 7f9f8b400700 1 --2- 172.21.15.86:0/1851613045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559effdfc8d0 0x559effde7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:07:38.460 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.459+0000 7f9f8b400700 1 -- 172.21.15.86:0/1851613045 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559effdf1a20 msgr2=0x559effdf9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:07:38.460 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.459+0000 7f9f8b400700 1 --2- 172.21.15.86:0/1851613045 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559effdf1a20 0x559effdf9ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:38.460 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.459+0000 7f9f8b400700 1 -- 172.21.15.86:0/1851613045 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559effdf1100 msgr2=0x559effe145b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:07:38.460 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.459+0000 7f9f8b400700 1 --2- 172.21.15.86:0/1851613045 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559effdf1100 0x559effe145b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:38.460 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.459+0000 7f9f8b400700 1 -- 172.21.15.86:0/1851613045 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559effd09e40 con 0x559effdfc8d0 2024-03-23T18:07:38.460 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.459+0000 7f9f8a000700 1 --2- 172.21.15.86:0/1851613045 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559effdf1a20 0x559effdf9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T18:07:38.460 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.459+0000 7f9f8b400700 1 --2- 172.21.15.86:0/1851613045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559effdfc8d0 0x559effde7500 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f9f84022f70 tx=0x7f9f84097aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2f984d7eb1db336 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:07:38.461 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.459+0000 7f9f80c00700 1 -- 172.21.15.86:0/1851613045 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f840a14e0 con 0x559effdfc8d0 2024-03-23T18:07:38.461 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.459+0000 7f9f80c00700 1 -- 172.21.15.86:0/1851613045 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f840b4030 con 0x559effdfc8d0 2024-03-23T18:07:38.461 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.459+0000 7f9f80c00700 1 -- 172.21.15.86:0/1851613045 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f84099920 con 0x559effdfc8d0 2024-03-23T18:07:38.462 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.460+0000 7f9fa5720b80 1 -- 172.21.15.86:0/1851613045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559effdfc8d0 msgr2=0x559effde7500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:07:38.462 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.460+0000 7f9fa5720b80 1 --2- 172.21.15.86:0/1851613045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559effdfc8d0 0x559effde7500 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f9f84022f70 tx=0x7f9f84097aa0 comp rx=0 tx=0).stop 2024-03-23T18:07:38.462 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.461+0000 7f9fa5720b80 1 -- 172.21.15.86:0/1851613045 shutdown_connections 2024-03-23T18:07:38.462 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.461+0000 7f9fa5720b80 1 --2- 172.21.15.86:0/1851613045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559effdfc8d0 0x559effde7500 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:38.462 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.461+0000 7f9fa5720b80 1 --2- 172.21.15.86:0/1851613045 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559effdf1100 0x559effe145b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:38.462 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.461+0000 7f9fa5720b80 1 --2- 172.21.15.86:0/1851613045 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559effdf1a20 0x559effdf9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:38.462 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.461+0000 7f9fa5720b80 1 -- 172.21.15.86:0/1851613045 >> 172.21.15.86:0/1851613045 conn(0x559effd51640 msgr2=0x559effdf48d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:07:38.462 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.461+0000 7f9fa5720b80 1 -- 172.21.15.86:0/1851613045 shutdown_connections 2024-03-23T18:07:38.462 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.461+0000 7f9fa5720b80 1 -- 172.21.15.86:0/1851613045 wait complete. 2024-03-23T18:07:38.462 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.462+0000 7f9fa5720b80 1 -- start start 2024-03-23T18:07:38.462 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.462+0000 7f9fa5720b80 1 Processor -- start 2024-03-23T18:07:38.464 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.463+0000 7f9fa5720b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559effdf1100 0x559effe13d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:07:38.464 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.463+0000 7f9fa5720b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559effdf1a20 0x559effe14280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:07:38.464 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.463+0000 7f9fa5720b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559effdfc8d0 0x559effe123f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:07:38.464 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.463+0000 7f9fa5720b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x559effe12980 con 0x559effdf1a20 2024-03-23T18:07:38.464 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.463+0000 7f9fa5720b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x559effe12af0 con 0x559effdfc8d0 2024-03-23T18:07:38.464 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.463+0000 7f9fa5720b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x559effe12cb0 con 0x559effdf1100 2024-03-23T18:07:38.464 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.463+0000 7f9fa5720b80 10 client.0 _ll_register_callbacks cb 0x559effce9410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T18:07:38.465 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.463+0000 7f9f8aa00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559effdf1100 0x559effe13d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:07:38.465 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.463+0000 7f9f8aa00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559effdf1100 0x559effe13d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:51054/0 (socket says 172.21.15.86:51054) 2024-03-23T18:07:38.465 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.463+0000 7f9f8aa00700 1 -- 172.21.15.86:0/4062391252 learned_addr learned my addr 172.21.15.86:0/4062391252 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:07:38.465 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.463+0000 7f9f8a000700 1 --2- 172.21.15.86:0/4062391252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559effdf1a20 0x559effe14280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:07:38.465 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.463+0000 7f9f8b400700 1 --2- 172.21.15.86:0/4062391252 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559effdfc8d0 0x559effe123f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:07:38.465 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.464+0000 7f9f8aa00700 1 -- 172.21.15.86:0/4062391252 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559effdfc8d0 msgr2=0x559effe123f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:07:38.465 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.464+0000 7f9f8aa00700 1 --2- 172.21.15.86:0/4062391252 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559effdfc8d0 0x559effe123f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:38.465 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.464+0000 7f9f8aa00700 1 -- 172.21.15.86:0/4062391252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559effdf1a20 msgr2=0x559effe14280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:07:38.465 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.464+0000 7f9f8aa00700 1 --2- 172.21.15.86:0/4062391252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559effdf1a20 0x559effe14280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:38.465 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.464+0000 7f9f8aa00700 1 -- 172.21.15.86:0/4062391252 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f8c10e440 con 0x559effdf1100 2024-03-23T18:07:38.465 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.464+0000 7f9f8aa00700 1 --2- 172.21.15.86:0/4062391252 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559effdf1100 0x559effe13d40 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c000c00 tx=0x7f9f7c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=148d6eb7d08a7fec server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:07:38.465 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.464+0000 7f9f74c00700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-03-23T18:07:38.467 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.465+0000 7f9f74c00700 1 -- 172.21.15.86:0/4062391252 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f7c022040 con 0x559effdf1100 2024-03-23T18:07:38.467 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.465+0000 7f9f74c00700 1 -- 172.21.15.86:0/4062391252 <== mon.1 v2:172.21.15.152:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f9f7c004020 con 0x559effdf1100 2024-03-23T18:07:38.467 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.465+0000 7f9fa5720b80 1 -- 172.21.15.86:0/4062391252 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9f840a1970 con 0x559effdf1100 2024-03-23T18:07:38.467 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.465+0000 7f9f74c00700 1 -- 172.21.15.86:0/4062391252 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f7c0041e0 con 0x559effdf1100 2024-03-23T18:07:38.467 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.465+0000 7f9fa5720b80 10 client.27331 fetch_fsmap learned FSMap version 88 2024-03-23T18:07:38.467 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.465+0000 7f9fa5720b80 1 -- 172.21.15.86:0/4062391252 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=4+}) v3 -- 0x559effb71b00 con 0x559effdf1100 2024-03-23T18:07:38.467 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.466+0000 7f9f74c00700 1 -- 172.21.15.86:0/4062391252 <== mon.1 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f840a1970 con 0x559effdf1100 2024-03-23T18:07:38.467 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.466+0000 7f9f74c00700 1 -- 172.21.15.86:0/4062391252 <== mon.1 v2:172.21.15.152:3300/0 5 ==== fsmap(e 88) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f9f7c003680 con 0x559effdf1100 2024-03-23T18:07:38.467 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.466+0000 7f9fa5720b80 10 client.27331 fetch_fsmap finished waiting for FSMap version 88 2024-03-23T18:07:38.468 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.466+0000 7f9fa5720b80 10 client.27331 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T18:07:38.468 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.466+0000 7f9fa5720b80 20 client.27331 populate_metadata read hostname 'smithi086' 2024-03-23T18:07:38.468 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.466+0000 7f9fa5720b80 1 --2- 172.21.15.86:0/4062391252 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x559effe6ff70 0x559effe72390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:07:38.468 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.466+0000 7f9fa5720b80 4 client.27331 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T18:07:38.468 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.466+0000 7f9fa5720b80 1 -- 172.21.15.86:0/4062391252 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559effb92c90 con 0x559effe6ff70 2024-03-23T18:07:38.468 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.466+0000 7f9f8a000700 1 --2- 172.21.15.86:0/4062391252 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x559effe6ff70 0x559effe72390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:07:38.468 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.467+0000 7f9f8a000700 1 --2- 172.21.15.86:0/4062391252 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x559effe6ff70 0x559effe72390 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bc8f8635f341c02 server_cookie=9261b3b8cf38002c in_seq=0 out_seq=0 2024-03-23T18:07:38.468 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.467+0000 7f9f74c00700 10 client.27331 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T18:07:38.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:07:38 smithi086 ceph-mon[38221]: pgmap v537: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.9 MiB/s wr, 655 op/s 2024-03-23T18:07:38.698 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.698+0000 7f9f74c00700 1 -- 172.21.15.86:0/4062391252 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559effb92c90 con 0x559effe6ff70 2024-03-23T18:07:38.698 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.698+0000 7f9f74c00700 10 client.27331 handle_command_reply: tid=0 2024-03-23T18:07:38.698 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.698+0000 7f9fa5720b80 1 client.27331 shutdown 2024-03-23T18:07:38.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.699+0000 7f9fa5720b80 1 -- 172.21.15.86:0/4062391252 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559effdf1100 msgr2=0x559effe13d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:07:38.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.699+0000 7f9fa5720b80 1 --2- 172.21.15.86:0/4062391252 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559effdf1100 0x559effe13d40 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c000c00 tx=0x7f9f7c004750 comp rx=0 tx=0).stop 2024-03-23T18:07:38.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.699+0000 7f9fa5720b80 1 -- 172.21.15.86:0/4062391252 shutdown_connections 2024-03-23T18:07:38.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.699+0000 7f9fa5720b80 1 --2- 172.21.15.86:0/4062391252 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x559effe6ff70 0x559effe72390 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:38.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.699+0000 7f9fa5720b80 1 --2- 172.21.15.86:0/4062391252 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559effdfc8d0 0x559effe123f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:38.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.699+0000 7f9fa5720b80 1 --2- 172.21.15.86:0/4062391252 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559effdf1100 0x559effe13d40 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:38.700 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.699+0000 7f9fa5720b80 1 --2- 172.21.15.86:0/4062391252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559effdf1a20 0x559effe14280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:38.700 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.699+0000 7f9fa5720b80 1 -- 172.21.15.86:0/4062391252 >> 172.21.15.86:0/4062391252 conn(0x559effd51640 msgr2=0x559effdf7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:07:38.700 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.699+0000 7f9f74c00700 0 client.27331 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T18:07:38.700 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.699+0000 7f9fa5720b80 1 -- 172.21.15.86:0/4062391252 shutdown_connections 2024-03-23T18:07:38.700 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.700+0000 7f9fa5720b80 1 -- 172.21.15.86:0/4062391252 wait complete. 2024-03-23T18:07:38.701 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.702+0000 7f9fa5720b80 20 client.27331 trim_cache size 0 max 16384 2024-03-23T18:07:38.702 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-03-23T18:07:38.702 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-03-23T18:07:38.702 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-03-23T18:07:38.702 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-03-23T18:07:38.703 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.703+0000 7f9f91c00700 1 -- 172.21.15.86:0/3076480510 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f9f700a1390 msgr2=0x7f9f700a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:07:38.703 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.703+0000 7f9f91c00700 1 --2- 172.21.15.86:0/3076480510 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f9f700a1390 0x7f9f700a3850 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c012740 tx=0x7f9f7c00f040 comp rx=0 tx=0).stop 2024-03-23T18:07:38.703 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.703+0000 7f9f91c00700 1 -- 172.21.15.86:0/3076480510 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9f8c106e10 msgr2=0x7f9f8c199820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:07:38.703 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.703+0000 7f9f91c00700 1 --2- 172.21.15.86:0/3076480510 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9f8c106e10 0x7f9f8c199820 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f9f84007ee0 tx=0x7f9f8400d3b0 comp rx=0 tx=0).stop 2024-03-23T18:07:38.703 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.703+0000 7f9f91c00700 1 -- 172.21.15.86:0/3076480510 shutdown_connections 2024-03-23T18:07:38.703 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.703+0000 7f9f91c00700 1 --2- 172.21.15.86:0/3076480510 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f9f700a1390 0x7f9f700a3850 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:38.703 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.703+0000 7f9f91c00700 1 --2- 172.21.15.86:0/3076480510 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9f8c106e10 0x7f9f8c199820 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:38.703 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.703+0000 7f9f91c00700 1 --2- 172.21.15.86:0/3076480510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f8c0680e0 0x7f9f8c194f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:38.703 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.703+0000 7f9f91c00700 1 --2- 172.21.15.86:0/3076480510 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f8c068aa0 0x7f9f8c195480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:38.703 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.703+0000 7f9f91c00700 1 -- 172.21.15.86:0/3076480510 >> 172.21.15.86:0/3076480510 conn(0x7f9f8c076d50 msgr2=0x7f9f8c0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:07:38.704 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.704+0000 7f9f91c00700 1 -- 172.21.15.86:0/3076480510 shutdown_connections 2024-03-23T18:07:38.704 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:38.704+0000 7f9f91c00700 1 -- 172.21.15.86:0/3076480510 wait complete. 2024-03-23T18:07:39.247 INFO:tasks.workunit.client.0.smithi086.stdout: 10 171486 36.44 MB/sec execute 123 sec latency 262.221 ms 2024-03-23T18:07:40.246 INFO:tasks.workunit.client.0.smithi086.stdout: 10 172036 36.41 MB/sec execute 124 sec latency 212.508 ms 2024-03-23T18:07:40.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:07:40 smithi092 ceph-mon[79089]: pgmap v538: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 8.2 MiB/s wr, 595 op/s 2024-03-23T18:07:40.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:07:40 smithi152 ceph-mon[47949]: pgmap v538: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 8.2 MiB/s wr, 595 op/s 2024-03-23T18:07:40.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:07:40 smithi086 ceph-mon[38221]: pgmap v538: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 8.2 MiB/s wr, 595 op/s 2024-03-23T18:07:41.246 INFO:tasks.workunit.client.0.smithi086.stdout: 10 172783 36.50 MB/sec execute 125 sec latency 67.604 ms 2024-03-23T18:07:41.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:07:41 smithi092 ceph-mon[79089]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:07:41.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:07:41 smithi152 ceph-mon[47949]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:07:41.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:07:41 smithi086 ceph-mon[38221]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:07:42.246 INFO:tasks.workunit.client.0.smithi086.stdout: 10 173513 36.48 MB/sec execute 126 sec latency 78.390 ms 2024-03-23T18:07:42.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:07:42 smithi152 ceph-mon[47949]: pgmap v539: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 13 MiB/s wr, 933 op/s 2024-03-23T18:07:42.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:07:42 smithi086 ceph-mon[38221]: pgmap v539: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 13 MiB/s wr, 933 op/s 2024-03-23T18:07:42.558 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:07:42 smithi092 ceph-mon[79089]: pgmap v539: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 13 MiB/s wr, 933 op/s 2024-03-23T18:07:43.246 INFO:tasks.workunit.client.0.smithi086.stdout: 10 174236 36.50 MB/sec execute 127 sec latency 60.387 ms 2024-03-23T18:07:43.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:07:43 smithi086 ceph-mon[38221]: pgmap v540: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 8.9 MiB/s wr, 606 op/s 2024-03-23T18:07:43.560 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:07:43 smithi092 ceph-mon[79089]: pgmap v540: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 8.9 MiB/s wr, 606 op/s 2024-03-23T18:07:43.585 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:07:43 smithi152 ceph-mon[47949]: pgmap v540: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 8.9 MiB/s wr, 606 op/s 2024-03-23T18:07:44.247 INFO:tasks.workunit.client.0.smithi086.stdout: 10 174806 36.43 MB/sec execute 128 sec latency 850.765 ms 2024-03-23T18:07:45.247 INFO:tasks.workunit.client.0.smithi086.stdout: 10 174902 36.17 MB/sec execute 129 sec latency 1850.816 ms 2024-03-23T18:07:46.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:07:45 smithi086 ceph-mon[38221]: pgmap v541: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 715 op/s 2024-03-23T18:07:46.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:07:45 smithi086 ceph-mon[38221]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:07:46.057 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:07:45 smithi092 ceph-mon[79089]: pgmap v541: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 715 op/s 2024-03-23T18:07:46.058 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:07:45 smithi092 ceph-mon[79089]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:07:46.089 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:07:45 smithi152 ceph-mon[47949]: pgmap v541: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 715 op/s 2024-03-23T18:07:46.089 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:07:45 smithi152 ceph-mon[47949]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T18:07:46.247 INFO:tasks.workunit.client.0.smithi086.stdout: 10 175360 36.10 MB/sec execute 130 sec latency 2850.867 ms 2024-03-23T18:07:47.247 INFO:tasks.workunit.client.0.smithi086.stdout: 10 175883 36.08 MB/sec execute 131 sec latency 3850.910 ms 2024-03-23T18:07:48.247 INFO:tasks.workunit.client.0.smithi086.stdout: 10 176626 36.13 MB/sec execute 132 sec latency 4850.957 ms 2024-03-23T18:07:48.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:07:47 smithi086 ceph-mon[38221]: pgmap v542: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 754 op/s 2024-03-23T18:07:48.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:07:47 smithi092 ceph-mon[79089]: pgmap v542: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 754 op/s 2024-03-23T18:07:48.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:07:47 smithi152 ceph-mon[47949]: pgmap v542: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 754 op/s 2024-03-23T18:07:49.247 INFO:tasks.workunit.client.0.smithi086.stdout: 10 177218 36.09 MB/sec execute 133 sec latency 5851.006 ms 2024-03-23T18:07:50.247 INFO:tasks.workunit.client.0.smithi086.stdout: 10 177708 35.96 MB/sec execute 134 sec latency 5980.143 ms 2024-03-23T18:07:50.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:07:49 smithi086 ceph-mon[38221]: pgmap v543: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 11 MiB/s wr, 638 op/s 2024-03-23T18:07:50.313 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:07:49 smithi092 ceph-mon[79089]: pgmap v543: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 11 MiB/s wr, 638 op/s 2024-03-23T18:07:50.352 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:07:49 smithi152 ceph-mon[47949]: pgmap v543: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 11 MiB/s wr, 638 op/s 2024-03-23T18:07:51.247 INFO:tasks.workunit.client.0.smithi086.stdout: 10 177878 35.79 MB/sec execute 135 sec latency 793.857 ms 2024-03-23T18:07:51.939 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.939+0000 7f185de00700 1 -- 172.21.15.86:0/308455810 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18581018c0 msgr2=0x7f1858105900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:07:51.939 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.939+0000 7f185de00700 1 --2- 172.21.15.86:0/308455810 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18581018c0 0x7f1858105900 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f1844009990 tx=0x7f1844009ca0 comp rx=0 tx=0).stop 2024-03-23T18:07:51.940 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.939+0000 7f185de00700 1 -- 172.21.15.86:0/308455810 shutdown_connections 2024-03-23T18:07:51.940 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.939+0000 7f185de00700 1 --2- 172.21.15.86:0/308455810 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1858106030 0x7f185810dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:51.940 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.939+0000 7f185de00700 1 --2- 172.21.15.86:0/308455810 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1858100f10 0x7f18581012f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:51.940 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.939+0000 7f185de00700 1 --2- 172.21.15.86:0/308455810 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18581018c0 0x7f1858105900 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:51.940 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.939+0000 7f185de00700 1 -- 172.21.15.86:0/308455810 >> 172.21.15.86:0/308455810 conn(0x7f18580fc770 msgr2=0x7f18580feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:07:51.940 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.940+0000 7f185de00700 1 -- 172.21.15.86:0/308455810 shutdown_connections 2024-03-23T18:07:51.940 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.940+0000 7f185de00700 1 -- 172.21.15.86:0/308455810 wait complete. 2024-03-23T18:07:51.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.943+0000 7f185de00700 1 Processor -- start 2024-03-23T18:07:51.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.943+0000 7f185de00700 1 -- start start 2024-03-23T18:07:51.944 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.944+0000 7f185de00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1858100f10 0x7f185810a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:07:51.944 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.944+0000 7f185de00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1858106030 0x7f185810abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:07:51.944 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.944+0000 7f185de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f185806c7f0 0x7f185806cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:07:51.945 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.944+0000 7f185de00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f185806d270 con 0x7f185806c7f0 2024-03-23T18:07:51.945 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.944+0000 7f185de00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f185806d3e0 con 0x7f1858106030 2024-03-23T18:07:51.945 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.944+0000 7f185de00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f185806d5a0 con 0x7f1858100f10 2024-03-23T18:07:51.945 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.945+0000 7f1857400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f185806c7f0 0x7f185806cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:07:51.946 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.946+0000 7f1857400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f185806c7f0 0x7f185806cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42452/0 (socket says 172.21.15.86:42452) 2024-03-23T18:07:51.947 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.946+0000 7f1857400700 1 -- 172.21.15.86:0/2404119638 learned_addr learned my addr 172.21.15.86:0/2404119638 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:07:51.947 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.946+0000 7f1856000700 1 --2- 172.21.15.86:0/2404119638 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1858106030 0x7f185810abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:07:51.947 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.947+0000 7f1856000700 1 -- 172.21.15.86:0/2404119638 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1858100f10 msgr2=0x7f185810a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:07:51.947 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.947+0000 7f1856000700 1 --2- 172.21.15.86:0/2404119638 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1858100f10 0x7f185810a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:51.947 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.947+0000 7f1856000700 1 -- 172.21.15.86:0/2404119638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f185806c7f0 msgr2=0x7f185806cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:07:51.947 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.947+0000 7f1856000700 1 --2- 172.21.15.86:0/2404119638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f185806c7f0 0x7f185806cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:51.947 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.947+0000 7f1856000700 1 -- 172.21.15.86:0/2404119638 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18440095f0 con 0x7f1858106030 2024-03-23T18:07:51.948 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.948+0000 7f1856000700 1 --2- 172.21.15.86:0/2404119638 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1858106030 0x7f185810abb0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f18440098e0 tx=0x7f1844005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:07:51.949 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.949+0000 7f185ca00700 1 -- 172.21.15.86:0/2404119638 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f184401d070 con 0x7f1858106030 2024-03-23T18:07:51.949 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.949+0000 7f185ca00700 1 -- 172.21.15.86:0/2404119638 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1844022470 con 0x7f1858106030 2024-03-23T18:07:51.949 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.949+0000 7f185ca00700 1 -- 172.21.15.86:0/2404119638 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1844017650 con 0x7f1858106030 2024-03-23T18:07:51.950 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.950+0000 7f185de00700 1 -- 172.21.15.86:0/2404119638 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18580718d0 con 0x7f1858106030 2024-03-23T18:07:51.950 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.950+0000 7f185de00700 1 -- 172.21.15.86:0/2404119638 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1858071d30 con 0x7f1858106030 2024-03-23T18:07:51.957 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.957+0000 7f185ca00700 1 -- 172.21.15.86:0/2404119638 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f184400fad0 con 0x7f1858106030 2024-03-23T18:07:51.961 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.960+0000 7f185ca00700 1 --2- 172.21.15.86:0/2404119638 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f183c0a1460 0x7f183c0a3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:07:51.961 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.960+0000 7f185ca00700 1 -- 172.21.15.86:0/2404119638 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f18440cde80 con 0x7f1858106030 2024-03-23T18:07:51.961 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.961+0000 7f1856a00700 1 --2- 172.21.15.86:0/2404119638 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f183c0a1460 0x7f183c0a3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:07:51.967 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.965+0000 7f1856a00700 1 --2- 172.21.15.86:0/2404119638 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f183c0a1460 0x7f183c0a3920 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f184800d0e0 tx=0x7f18480085a0 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:07:51.969 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.968+0000 7f18718e7b80 1 Processor -- start 2024-03-23T18:07:51.969 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.969+0000 7f18718e7b80 1 -- start start 2024-03-23T18:07:51.970 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.969+0000 7f18718e7b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5603fe594100 0x5603fe5944e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:07:51.972 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.972+0000 7f1856a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5603fe594100 0x5603fe5944e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:07:51.972 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.972+0000 7f1856a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5603fe594100 0x5603fe5944e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:34908/0 (socket says 172.21.15.86:34908) 2024-03-23T18:07:51.972 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.972+0000 7f18718e7b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603fe594a20 0x5603fe59f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:07:51.973 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.972+0000 7f1856a00700 1 -- 172.21.15.86:0/2408707721 learned_addr learned my addr 172.21.15.86:0/2408707721 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:07:51.973 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.972+0000 7f18718e7b80 1 --2- 172.21.15.86:0/2408707721 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5603fe59f8d0 0x5603fe5a1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:07:51.973 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.973+0000 7f18718e7b80 1 -- 172.21.15.86:0/2408707721 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603fe4ed930 con 0x5603fe594a20 2024-03-23T18:07:51.973 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.973+0000 7f18718e7b80 1 -- 172.21.15.86:0/2408707721 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603fe4eb2f0 con 0x5603fe594100 2024-03-23T18:07:51.973 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.973+0000 7f18718e7b80 1 -- 172.21.15.86:0/2408707721 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603fe4e6be0 con 0x5603fe59f8d0 2024-03-23T18:07:51.973 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.973+0000 7f1857400700 1 --2- 172.21.15.86:0/2408707721 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5603fe59f8d0 0x5603fe5a1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:07:51.974 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.974+0000 7f1856a00700 1 -- 172.21.15.86:0/2408707721 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5603fe59f8d0 msgr2=0x5603fe5a1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:07:51.974 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.974+0000 7f1856a00700 1 --2- 172.21.15.86:0/2408707721 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5603fe59f8d0 0x5603fe5a1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:51.974 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.974+0000 7f1856a00700 1 -- 172.21.15.86:0/2408707721 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603fe594a20 msgr2=0x5603fe59f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:07:51.975 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.974+0000 7f1856a00700 1 --2- 172.21.15.86:0/2408707721 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603fe594a20 0x5603fe59f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:51.975 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.974+0000 7f1856a00700 1 -- 172.21.15.86:0/2408707721 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5603fe368350 con 0x5603fe594100 2024-03-23T18:07:51.975 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.975+0000 7f1857400700 1 --2- 172.21.15.86:0/2408707721 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5603fe59f8d0 0x5603fe5a1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T18:07:51.975 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.975+0000 7f1856a00700 1 --2- 172.21.15.86:0/2408707721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5603fe594100 0x5603fe5944e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f184800dfd0 tx=0x7f1848010000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22e7f6674a2495e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:07:51.975 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.976+0000 7f1843e00700 1 -- 172.21.15.86:0/2408707721 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f184801c070 con 0x5603fe594100 2024-03-23T18:07:51.976 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.976+0000 7f1843e00700 1 -- 172.21.15.86:0/2408707721 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1848015420 con 0x5603fe594100 2024-03-23T18:07:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.976+0000 7f1843e00700 1 -- 172.21.15.86:0/2408707721 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f184801c070 con 0x5603fe594100 2024-03-23T18:07:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.977+0000 7f18718e7b80 1 -- 172.21.15.86:0/2408707721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5603fe594100 msgr2=0x5603fe5944e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:07:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.977+0000 7f18718e7b80 1 --2- 172.21.15.86:0/2408707721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5603fe594100 0x5603fe5944e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f184800dfd0 tx=0x7f1848010000 comp rx=0 tx=0).stop 2024-03-23T18:07:51.978 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.977+0000 7f18718e7b80 1 -- 172.21.15.86:0/2408707721 shutdown_connections 2024-03-23T18:07:51.978 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.978+0000 7f18718e7b80 1 --2- 172.21.15.86:0/2408707721 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5603fe59f8d0 0x5603fe5a1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:51.978 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.978+0000 7f18718e7b80 1 --2- 172.21.15.86:0/2408707721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5603fe594100 0x5603fe5944e0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:51.978 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.978+0000 7f18718e7b80 1 --2- 172.21.15.86:0/2408707721 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603fe594a20 0x5603fe59f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:51.978 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.978+0000 7f18718e7b80 1 -- 172.21.15.86:0/2408707721 >> 172.21.15.86:0/2408707721 conn(0x5603fe4f4640 msgr2=0x5603fe590520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:07:51.978 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.978+0000 7f18718e7b80 1 -- 172.21.15.86:0/2408707721 shutdown_connections 2024-03-23T18:07:51.979 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.978+0000 7f18718e7b80 1 -- 172.21.15.86:0/2408707721 wait complete. 2024-03-23T18:07:51.979 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.979+0000 7f18718e7b80 1 -- start start 2024-03-23T18:07:51.979 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.979+0000 7f18718e7b80 1 Processor -- start 2024-03-23T18:07:51.980 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.981+0000 7f18718e7b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603fe594100 0x5603fe59f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:07:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.981+0000 7f18718e7b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5603fe594a20 0x5603fe59c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:07:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.981+0000 7f18718e7b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5603fe59f8d0 0x5603fe59cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:07:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.982+0000 7f1856a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603fe594100 0x5603fe59f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:07:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.982+0000 7f1856a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603fe594100 0x5603fe59f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42470/0 (socket says 172.21.15.86:42470) 2024-03-23T18:07:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.982+0000 7f1856a00700 1 -- 172.21.15.86:0/2337876464 learned_addr learned my addr 172.21.15.86:0/2337876464 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:07:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.982+0000 7f1857400700 1 --2- 172.21.15.86:0/2337876464 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5603fe59f8d0 0x5603fe59cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:07:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.982+0000 7f18718e7b80 1 -- 172.21.15.86:0/2337876464 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603fe482d20 con 0x5603fe594100 2024-03-23T18:07:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.982+0000 7f18718e7b80 1 -- 172.21.15.86:0/2337876464 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603fe4829c0 con 0x5603fe59f8d0 2024-03-23T18:07:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.982+0000 7f18718e7b80 1 -- 172.21.15.86:0/2337876464 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603fe36a640 con 0x5603fe594a20 2024-03-23T18:07:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.983+0000 7f18718e7b80 10 client.0 _ll_register_callbacks cb 0x5603fe48c410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T18:07:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.983+0000 7f1857400700 1 -- 172.21.15.86:0/2337876464 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5603fe594a20 msgr2=0x5603fe59c640 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-23T18:07:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.983+0000 7f1857400700 1 --2- 172.21.15.86:0/2337876464 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5603fe594a20 0x5603fe59c640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.983+0000 7f1857400700 1 -- 172.21.15.86:0/2337876464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603fe594100 msgr2=0x5603fe59f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:07:51.985 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.983+0000 7f1857400700 1 --2- 172.21.15.86:0/2337876464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603fe594100 0x5603fe59f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:51.985 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.983+0000 7f1857400700 1 -- 172.21.15.86:0/2337876464 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1850009930 con 0x5603fe59f8d0 2024-03-23T18:07:51.985 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.984+0000 7f1857400700 1 --2- 172.21.15.86:0/2337876464 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5603fe59f8d0 0x5603fe59cd80 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f18500096d0 tx=0x7f1850011cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d7045e7763b11dc server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:07:51.985 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.985+0000 7f1840c00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T18:07:51.985 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.985+0000 7f1840c00700 1 -- 172.21.15.86:0/2337876464 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1850016070 con 0x5603fe59f8d0 2024-03-23T18:07:51.985 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.985+0000 7f1840c00700 1 -- 172.21.15.86:0/2337876464 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f185000fa80 con 0x5603fe59f8d0 2024-03-23T18:07:51.986 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.986+0000 7f18718e7b80 1 -- 172.21.15.86:0/2337876464 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5603fe4a7320 con 0x5603fe59f8d0 2024-03-23T18:07:51.989 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.989+0000 7f1840c00700 1 -- 172.21.15.86:0/2337876464 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1850016070 con 0x5603fe59f8d0 2024-03-23T18:07:51.989 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.990+0000 7f18718e7b80 10 client.27371 fetch_fsmap learned FSMap version 90 2024-03-23T18:07:51.989 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.990+0000 7f18718e7b80 1 -- 172.21.15.86:0/2337876464 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=4+}) v3 -- 0x7f184800dc80 con 0x5603fe59f8d0 2024-03-23T18:07:51.995 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.995+0000 7f1840c00700 1 -- 172.21.15.86:0/2337876464 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1850004cf0 con 0x5603fe59f8d0 2024-03-23T18:07:51.995 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.995+0000 7f1840c00700 1 -- 172.21.15.86:0/2337876464 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 90) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f18500053a0 con 0x5603fe59f8d0 2024-03-23T18:07:51.995 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.995+0000 7f18718e7b80 10 client.27371 fetch_fsmap finished waiting for FSMap version 90 2024-03-23T18:07:51.998 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.995+0000 7f18718e7b80 10 client.27371 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T18:07:51.998 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.995+0000 7f18718e7b80 20 client.27371 populate_metadata read hostname 'smithi086' 2024-03-23T18:07:51.998 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.995+0000 7f18718e7b80 1 --2- 172.21.15.86:0/2337876464 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5603fe5af7a0 0x5603fe5b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:07:51.998 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.996+0000 7f18718e7b80 4 client.27371 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T18:07:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.996+0000 7f18718e7b80 1 -- 172.21.15.86:0/2337876464 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5603fe4a7320 con 0x5603fe5af7a0 2024-03-23T18:07:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:51.998+0000 7f1856a00700 1 --2- 172.21.15.86:0/2337876464 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5603fe5af7a0 0x5603fe5b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:07:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.000+0000 7f1856a00700 1 --2- 172.21.15.86:0/2337876464 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5603fe5af7a0 0x5603fe5b1bc0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb21ada8539d3ff1 server_cookie=13eed4f07a00f51e in_seq=0 out_seq=0 2024-03-23T18:07:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.000+0000 7f1840c00700 10 client.27371 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T18:07:52.003 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.003+0000 7f1840c00700 1 -- 172.21.15.86:0/2337876464 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5603fe4a7320 con 0x5603fe5af7a0 2024-03-23T18:07:52.003 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.003+0000 7f1840c00700 10 client.27371 handle_command_reply: tid=0 2024-03-23T18:07:52.003 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.003+0000 7f18718e7b80 1 client.27371 shutdown 2024-03-23T18:07:52.003 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.003+0000 7f18718e7b80 1 -- 172.21.15.86:0/2337876464 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5603fe59f8d0 msgr2=0x5603fe59cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:07:52.003 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.003+0000 7f18718e7b80 1 --2- 172.21.15.86:0/2337876464 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5603fe59f8d0 0x5603fe59cd80 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f18500096d0 tx=0x7f1850011cb0 comp rx=0 tx=0).stop 2024-03-23T18:07:52.004 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.004+0000 7f18718e7b80 1 -- 172.21.15.86:0/2337876464 shutdown_connections 2024-03-23T18:07:52.004 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.004+0000 7f18718e7b80 1 --2- 172.21.15.86:0/2337876464 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5603fe5af7a0 0x5603fe5b1bc0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:52.004 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.004+0000 7f18718e7b80 1 --2- 172.21.15.86:0/2337876464 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5603fe59f8d0 0x5603fe59cd80 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:52.004 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.004+0000 7f18718e7b80 1 --2- 172.21.15.86:0/2337876464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603fe594100 0x5603fe59f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:52.004 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.004+0000 7f18718e7b80 1 --2- 172.21.15.86:0/2337876464 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5603fe594a20 0x5603fe59c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:52.004 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.004+0000 7f18718e7b80 1 -- 172.21.15.86:0/2337876464 >> 172.21.15.86:0/2337876464 conn(0x5603fe4f4640 msgr2=0x5603fe5960a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:07:52.004 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.004+0000 7f1840c00700 0 client.27371 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T18:07:52.005 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.004+0000 7f18718e7b80 1 -- 172.21.15.86:0/2337876464 shutdown_connections 2024-03-23T18:07:52.005 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.004+0000 7f18718e7b80 1 -- 172.21.15.86:0/2337876464 wait complete. 2024-03-23T18:07:52.005 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.004+0000 7f18718e7b80 20 client.27371 trim_cache size 0 max 16384 2024-03-23T18:07:52.033 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.033+0000 7f18718e7b80 1 Processor -- start 2024-03-23T18:07:52.034 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.034+0000 7f18718e7b80 1 -- start start 2024-03-23T18:07:52.035 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.034+0000 7f18718e7b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5603fe594100 0x5603fe5b6520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:07:52.035 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.034+0000 7f18718e7b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5603fe594a20 0x5603fe5a66c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:07:52.035 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.034+0000 7f18718e7b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603fe59f8d0 0x5603fe5905c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:07:52.035 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.034+0000 7f18718e7b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603fe594e40 con 0x5603fe59f8d0 2024-03-23T18:07:52.035 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.034+0000 7f18718e7b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603fe58c400 con 0x5603fe594a20 2024-03-23T18:07:52.035 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.034+0000 7f18718e7b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603fe5acd10 con 0x5603fe594100 2024-03-23T18:07:52.035 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.035+0000 7f1856a00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5603fe594100 0x5603fe5b6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:07:52.036 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.035+0000 7f1856a00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5603fe594100 0x5603fe5b6520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:58920/0 (socket says 172.21.15.86:58920) 2024-03-23T18:07:52.036 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.035+0000 7f1856a00700 1 -- 172.21.15.86:0/546385328 learned_addr learned my addr 172.21.15.86:0/546385328 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:07:52.036 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.035+0000 7f1857400700 1 --2- 172.21.15.86:0/546385328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603fe59f8d0 0x5603fe5905c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:07:52.036 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.036+0000 7f1856a00700 1 -- 172.21.15.86:0/546385328 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5603fe594a20 msgr2=0x5603fe5a66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:07:52.036 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.036+0000 7f1856a00700 1 --2- 172.21.15.86:0/546385328 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5603fe594a20 0x5603fe5a66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:52.036 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.036+0000 7f1856a00700 1 -- 172.21.15.86:0/546385328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603fe59f8d0 msgr2=0x5603fe5905c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:07:52.036 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.036+0000 7f1856a00700 1 --2- 172.21.15.86:0/546385328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603fe59f8d0 0x5603fe5905c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:52.037 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.036+0000 7f1856a00700 1 -- 172.21.15.86:0/546385328 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5603fe314b00 con 0x5603fe594100 2024-03-23T18:07:52.037 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.037+0000 7f1856a00700 1 --2- 172.21.15.86:0/546385328 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5603fe594100 0x5603fe5b6520 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f184800d830 tx=0x7f1848015d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55a76f2ef04c8f0b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:07:52.037 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.038+0000 7f184d200700 1 -- 172.21.15.86:0/546385328 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f184802f070 con 0x5603fe594100 2024-03-23T18:07:52.038 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.038+0000 7f184d200700 1 -- 172.21.15.86:0/546385328 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1848004bb0 con 0x5603fe594100 2024-03-23T18:07:52.038 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.038+0000 7f184d200700 1 -- 172.21.15.86:0/546385328 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f184801c040 con 0x5603fe594100 2024-03-23T18:07:52.039 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.039+0000 7f18718e7b80 1 -- 172.21.15.86:0/546385328 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5603fe594100 msgr2=0x5603fe5b6520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:07:52.039 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.039+0000 7f18718e7b80 1 --2- 172.21.15.86:0/546385328 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5603fe594100 0x5603fe5b6520 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f184800d830 tx=0x7f1848015d10 comp rx=0 tx=0).stop 2024-03-23T18:07:52.039 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.039+0000 7f18718e7b80 1 -- 172.21.15.86:0/546385328 shutdown_connections 2024-03-23T18:07:52.039 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.039+0000 7f18718e7b80 1 --2- 172.21.15.86:0/546385328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603fe59f8d0 0x5603fe5905c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:52.039 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.039+0000 7f18718e7b80 1 --2- 172.21.15.86:0/546385328 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5603fe594100 0x5603fe5b6520 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:52.039 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.039+0000 7f18718e7b80 1 --2- 172.21.15.86:0/546385328 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5603fe594a20 0x5603fe5a66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:52.039 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.039+0000 7f18718e7b80 1 -- 172.21.15.86:0/546385328 >> 172.21.15.86:0/546385328 conn(0x5603fe4f4640 msgr2=0x5603fe58feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:07:52.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.041+0000 7f18718e7b80 1 -- 172.21.15.86:0/546385328 shutdown_connections 2024-03-23T18:07:52.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.041+0000 7f18718e7b80 1 -- 172.21.15.86:0/546385328 wait complete. 2024-03-23T18:07:52.042 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.042+0000 7f18718e7b80 1 -- start start 2024-03-23T18:07:52.042 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.042+0000 7f18718e7b80 1 Processor -- start 2024-03-23T18:07:52.043 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.043+0000 7f18718e7b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603fe594100 0x5603fe5a4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:07:52.043 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.043+0000 7f18718e7b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5603fe594a20 0x5603fe5a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:07:52.044 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.043+0000 7f18718e7b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5603fe59f8d0 0x5603fe6002c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:07:52.044 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.043+0000 7f18718e7b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603fe5a54b0 con 0x5603fe594100 2024-03-23T18:07:52.044 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.043+0000 7f18718e7b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603fe600800 con 0x5603fe594a20 2024-03-23T18:07:52.044 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.043+0000 7f18718e7b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603fe600940 con 0x5603fe59f8d0 2024-03-23T18:07:52.044 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.043+0000 7f1857400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5603fe59f8d0 0x5603fe6002c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:07:52.044 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.043+0000 7f18718e7b80 10 client.0 _ll_register_callbacks cb 0x5603fe48c410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T18:07:52.044 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.043+0000 7f1857400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5603fe59f8d0 0x5603fe6002c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:58928/0 (socket says 172.21.15.86:58928) 2024-03-23T18:07:52.044 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.044+0000 7f1857400700 1 -- 172.21.15.86:0/2636260378 learned_addr learned my addr 172.21.15.86:0/2636260378 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:07:52.045 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.044+0000 7f1856a00700 1 --2- 172.21.15.86:0/2636260378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603fe594100 0x5603fe5a4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:07:52.045 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.045+0000 7f1857400700 1 -- 172.21.15.86:0/2636260378 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5603fe594a20 msgr2=0x5603fe5a4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:07:52.045 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.045+0000 7f1857400700 1 --2- 172.21.15.86:0/2636260378 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5603fe594a20 0x5603fe5a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:52.045 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.045+0000 7f1857400700 1 -- 172.21.15.86:0/2636260378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603fe594100 msgr2=0x5603fe5a4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:07:52.045 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.045+0000 7f1857400700 1 --2- 172.21.15.86:0/2636260378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603fe594100 0x5603fe5a4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:52.045 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.045+0000 7f1857400700 1 -- 172.21.15.86:0/2636260378 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5603fe368350 con 0x5603fe59f8d0 2024-03-23T18:07:52.045 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.045+0000 7f1856a00700 1 --2- 172.21.15.86:0/2636260378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603fe594100 0x5603fe5a4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T18:07:52.046 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.045+0000 7f1857400700 1 --2- 172.21.15.86:0/2636260378 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5603fe59f8d0 0x5603fe6002c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f18500115e0 tx=0x7f185000fd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60c911b1de2e9d54 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:07:52.046 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.045+0000 7f1840c00700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-03-23T18:07:52.046 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.046+0000 7f1840c00700 1 -- 172.21.15.86:0/2636260378 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1850016040 con 0x5603fe59f8d0 2024-03-23T18:07:52.046 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.046+0000 7f1840c00700 1 -- 172.21.15.86:0/2636260378 <== mon.1 v2:172.21.15.152:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f1850000e40 con 0x5603fe59f8d0 2024-03-23T18:07:52.046 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.046+0000 7f18718e7b80 1 -- 172.21.15.86:0/2636260378 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1848015420 con 0x5603fe59f8d0 2024-03-23T18:07:52.047 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.046+0000 7f1840c00700 1 -- 172.21.15.86:0/2636260378 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1850023a30 con 0x5603fe59f8d0 2024-03-23T18:07:52.047 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.047+0000 7f18718e7b80 10 client.27343 fetch_fsmap learned FSMap version 90 2024-03-23T18:07:52.047 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.047+0000 7f18718e7b80 1 -- 172.21.15.86:0/2636260378 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=4+}) v3 -- 0x5603fe4ace40 con 0x5603fe59f8d0 2024-03-23T18:07:52.047 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.047+0000 7f1840c00700 1 -- 172.21.15.86:0/2636260378 <== mon.1 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1848015420 con 0x5603fe59f8d0 2024-03-23T18:07:52.048 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.048+0000 7f1840c00700 1 -- 172.21.15.86:0/2636260378 <== mon.1 v2:172.21.15.152:3300/0 5 ==== fsmap(e 90) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f185002c430 con 0x5603fe59f8d0 2024-03-23T18:07:52.048 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.048+0000 7f18718e7b80 10 client.27343 fetch_fsmap finished waiting for FSMap version 90 2024-03-23T18:07:52.048 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.048+0000 7f18718e7b80 10 client.27343 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T18:07:52.048 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.048+0000 7f18718e7b80 20 client.27343 populate_metadata read hostname 'smithi086' 2024-03-23T18:07:52.049 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.048+0000 7f18718e7b80 1 --2- 172.21.15.86:0/2636260378 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5603fe602050 0x5603fe604470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:07:52.049 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.048+0000 7f18718e7b80 4 client.27343 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T18:07:52.049 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.048+0000 7f18718e7b80 1 -- 172.21.15.86:0/2636260378 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5603fe4df6a0 con 0x5603fe602050 2024-03-23T18:07:52.052 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.052+0000 7f1856a00700 1 --2- 172.21.15.86:0/2636260378 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5603fe602050 0x5603fe604470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:07:52.056 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.056+0000 7f1840c00700 10 client.27343 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T18:07:52.056 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.056+0000 7f1856a00700 1 --2- 172.21.15.86:0/2636260378 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5603fe602050 0x5603fe604470 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b661a8f3b1b27acd server_cookie=ea40087bfaddaa43 in_seq=0 out_seq=0 2024-03-23T18:07:52.247 INFO:tasks.workunit.client.0.smithi086.stdout: 10 178343 35.73 MB/sec execute 136 sec latency 173.240 ms 2024-03-23T18:07:52.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:07:52 smithi086 ceph-mon[38221]: pgmap v544: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 19 MiB/s wr, 1.02k op/s 2024-03-23T18:07:52.296 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.295+0000 7f1840c00700 1 -- 172.21.15.86:0/2636260378 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5603fe4df6a0 con 0x5603fe602050 2024-03-23T18:07:52.296 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.295+0000 7f1840c00700 10 client.27343 handle_command_reply: tid=0 2024-03-23T18:07:52.296 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.295+0000 7f18718e7b80 1 client.27343 shutdown 2024-03-23T18:07:52.296 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.295+0000 7f18718e7b80 1 -- 172.21.15.86:0/2636260378 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5603fe59f8d0 msgr2=0x5603fe6002c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:07:52.296 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.295+0000 7f18718e7b80 1 --2- 172.21.15.86:0/2636260378 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5603fe59f8d0 0x5603fe6002c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f18500115e0 tx=0x7f185000fd60 comp rx=0 tx=0).stop 2024-03-23T18:07:52.296 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.295+0000 7f18718e7b80 1 -- 172.21.15.86:0/2636260378 shutdown_connections 2024-03-23T18:07:52.296 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.295+0000 7f18718e7b80 1 --2- 172.21.15.86:0/2636260378 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x5603fe602050 0x5603fe604470 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:52.296 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.295+0000 7f18718e7b80 1 --2- 172.21.15.86:0/2636260378 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5603fe59f8d0 0x5603fe6002c0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:52.296 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.295+0000 7f18718e7b80 1 --2- 172.21.15.86:0/2636260378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603fe594100 0x5603fe5a4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:52.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.295+0000 7f18718e7b80 1 --2- 172.21.15.86:0/2636260378 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5603fe594a20 0x5603fe5a4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:52.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.296+0000 7f18718e7b80 1 -- 172.21.15.86:0/2636260378 >> 172.21.15.86:0/2636260378 conn(0x5603fe4f4640 msgr2=0x5603fe5893e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:07:52.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.296+0000 7f1840c00700 0 client.27343 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T18:07:52.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.296+0000 7f18718e7b80 1 -- 172.21.15.86:0/2636260378 shutdown_connections 2024-03-23T18:07:52.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.296+0000 7f18718e7b80 1 -- 172.21.15.86:0/2636260378 wait complete. 2024-03-23T18:07:52.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.296+0000 7f18718e7b80 20 client.27343 trim_cache size 0 max 16384 2024-03-23T18:07:52.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-03-23T18:07:52.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-03-23T18:07:52.298 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-03-23T18:07:52.298 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-03-23T18:07:52.298 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.297+0000 7f185de00700 1 -- 172.21.15.86:0/2404119638 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f183c0a1460 msgr2=0x7f183c0a3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:07:52.298 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.297+0000 7f185de00700 1 --2- 172.21.15.86:0/2404119638 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f183c0a1460 0x7f183c0a3920 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f184800d0e0 tx=0x7f18480085a0 comp rx=0 tx=0).stop 2024-03-23T18:07:52.298 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.297+0000 7f185de00700 1 -- 172.21.15.86:0/2404119638 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1858106030 msgr2=0x7f185810abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:07:52.299 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.297+0000 7f185de00700 1 --2- 172.21.15.86:0/2404119638 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1858106030 0x7f185810abb0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f18440098e0 tx=0x7f1844005740 comp rx=0 tx=0).stop 2024-03-23T18:07:52.299 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.297+0000 7f185de00700 1 -- 172.21.15.86:0/2404119638 shutdown_connections 2024-03-23T18:07:52.299 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.297+0000 7f185de00700 1 --2- 172.21.15.86:0/2404119638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f185806c7f0 0x7f185806cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:52.299 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.297+0000 7f185de00700 1 --2- 172.21.15.86:0/2404119638 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f183c0a1460 0x7f183c0a3920 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:52.299 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.297+0000 7f185de00700 1 --2- 172.21.15.86:0/2404119638 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1858100f10 0x7f185810a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:52.299 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.297+0000 7f185de00700 1 --2- 172.21.15.86:0/2404119638 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1858106030 0x7f185810abb0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:07:52.299 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.297+0000 7f185de00700 1 -- 172.21.15.86:0/2404119638 >> 172.21.15.86:0/2404119638 conn(0x7f18580fc770 msgr2=0x7f18580fcd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:07:52.299 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.297+0000 7f185de00700 1 -- 172.21.15.86:0/2404119638 shutdown_connections 2024-03-23T18:07:52.299 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:07:52.297+0000 7f185de00700 1 -- 172.21.15.86:0/2404119638 wait complete. 2024-03-23T18:07:52.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:07:52 smithi092 ceph-mon[79089]: pgmap v544: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 19 MiB/s wr, 1.02k op/s 2024-03-23T18:07:52.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:07:52 smithi152 ceph-mon[47949]: pgmap v544: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 19 MiB/s wr, 1.02k op/s 2024-03-23T18:07:53.247 INFO:tasks.workunit.client.0.smithi086.stdout: 10 178948 35.68 MB/sec execute 137 sec latency 178.578 ms 2024-03-23T18:07:54.247 INFO:tasks.workunit.client.0.smithi086.stdout: 10 179690 35.73 MB/sec execute 138 sec latency 60.097 ms 2024-03-23T18:07:54.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:07:54 smithi086 ceph-mon[38221]: pgmap v545: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 15 MiB/s wr, 717 op/s 2024-03-23T18:07:54.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:07:54 smithi092 ceph-mon[79089]: pgmap v545: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 15 MiB/s wr, 717 op/s 2024-03-23T18:07:54.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:07:54 smithi152 ceph-mon[47949]: pgmap v545: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 15 MiB/s wr, 717 op/s 2024-03-23T18:07:55.247 INFO:tasks.workunit.client.0.smithi086.stdout: 10 180493 35.81 MB/sec execute 139 sec latency 68.310 ms 2024-03-23T18:07:56.247 INFO:tasks.workunit.client.0.smithi086.stdout: 10 181229 35.80 MB/sec execute 140 sec latency 78.242 ms 2024-03-23T18:07:56.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:07:56 smithi086 ceph-mon[38221]: pgmap v546: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 20 MiB/s wr, 946 op/s 2024-03-23T18:07:56.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:07:56 smithi092 ceph-mon[79089]: pgmap v546: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 20 MiB/s wr, 946 op/s 2024-03-23T18:07:56.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:07:56 smithi152 ceph-mon[47949]: pgmap v546: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 20 MiB/s wr, 946 op/s 2024-03-23T18:07:57.248 INFO:tasks.workunit.client.0.smithi086.stdout: 10 181850 35.83 MB/sec execute 141 sec latency 77.525 ms 2024-03-23T18:07:58.247 INFO:tasks.workunit.client.0.smithi086.stdout: 10 182495 35.80 MB/sec execute 142 sec latency 135.708 ms 2024-03-23T18:07:58.310 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:07:58 smithi092 ceph-mon[79089]: pgmap v547: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 23 MiB/s wr, 1.08k op/s 2024-03-23T18:07:58.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:07:58 smithi152 ceph-mon[47949]: pgmap v547: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 23 MiB/s wr, 1.08k op/s 2024-03-23T18:07:58.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:07:58 smithi086 ceph-mon[38221]: pgmap v547: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 23 MiB/s wr, 1.08k op/s 2024-03-23T18:07:59.247 INFO:tasks.workunit.client.0.smithi086.stdout: 10 183159 35.72 MB/sec execute 143 sec latency 122.440 ms 2024-03-23T18:08:00.247 INFO:tasks.workunit.client.0.smithi086.stdout: 10 183904 35.81 MB/sec execute 144 sec latency 192.898 ms 2024-03-23T18:08:00.306 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:00 smithi092 ceph-mon[79089]: pgmap v548: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 19 MiB/s wr, 891 op/s 2024-03-23T18:08:00.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:00 smithi152 ceph-mon[47949]: pgmap v548: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 19 MiB/s wr, 891 op/s 2024-03-23T18:08:00.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:00 smithi086 ceph-mon[38221]: pgmap v548: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 19 MiB/s wr, 891 op/s 2024-03-23T18:08:01.248 INFO:tasks.workunit.client.0.smithi086.stdout: 10 184189 35.65 MB/sec execute 145 sec latency 646.870 ms 2024-03-23T18:08:02.248 INFO:tasks.workunit.client.0.smithi086.stdout: 10 184314 35.46 MB/sec execute 146 sec latency 149.610 ms 2024-03-23T18:08:02.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:02 smithi092 ceph-mon[79089]: pgmap v549: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 22 MiB/s wr, 1.13k op/s 2024-03-23T18:08:02.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:02 smithi152 ceph-mon[47949]: pgmap v549: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 22 MiB/s wr, 1.13k op/s 2024-03-23T18:08:02.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:02 smithi086 ceph-mon[38221]: pgmap v549: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 22 MiB/s wr, 1.13k op/s 2024-03-23T18:08:03.248 INFO:tasks.workunit.client.0.smithi086.stdout: 10 184799 35.43 MB/sec execute 147 sec latency 135.429 ms 2024-03-23T18:08:04.248 INFO:tasks.workunit.client.0.smithi086.stdout: 10 185438 35.41 MB/sec execute 148 sec latency 108.944 ms 2024-03-23T18:08:04.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:04 smithi092 ceph-mon[79089]: pgmap v550: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 14 MiB/s wr, 751 op/s 2024-03-23T18:08:04.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:04 smithi152 ceph-mon[47949]: pgmap v550: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 14 MiB/s wr, 751 op/s 2024-03-23T18:08:04.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:04 smithi086 ceph-mon[38221]: pgmap v550: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 14 MiB/s wr, 751 op/s 2024-03-23T18:08:05.248 INFO:tasks.workunit.client.0.smithi086.stdout: 10 185970 35.35 MB/sec execute 149 sec latency 159.833 ms 2024-03-23T18:08:05.550 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.550+0000 7f242a200700 1 -- 172.21.15.86:0/2953759747 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f242410f4d0 msgr2=0x7f2424107db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:05.550 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.550+0000 7f242a200700 1 --2- 172.21.15.86:0/2953759747 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f242410f4d0 0x7f2424107db0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f24140099c0 tx=0x7f2414009cd0 comp rx=0 tx=0).stop 2024-03-23T18:08:05.551 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.550+0000 7f242a200700 1 -- 172.21.15.86:0/2953759747 shutdown_connections 2024-03-23T18:08:05.551 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.550+0000 7f242a200700 1 --2- 172.21.15.86:0/2953759747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f242411a590 0x7f242411c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:05.551 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.550+0000 7f242a200700 1 --2- 172.21.15.86:0/2953759747 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f242410f4d0 0x7f2424107db0 secure :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f24140099c0 tx=0x7f2414009cd0 comp rx=0 tx=0).stop 2024-03-23T18:08:05.551 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.551+0000 7f242a200700 1 --2- 172.21.15.86:0/2953759747 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f242410f940 0x7f24241082f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:05.551 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.551+0000 7f242a200700 1 -- 172.21.15.86:0/2953759747 >> 172.21.15.86:0/2953759747 conn(0x7f242406bd80 msgr2=0x7f242406c190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:08:05.551 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.551+0000 7f242a200700 1 -- 172.21.15.86:0/2953759747 shutdown_connections 2024-03-23T18:08:05.552 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.551+0000 7f242a200700 1 -- 172.21.15.86:0/2953759747 wait complete. 2024-03-23T18:08:05.552 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.552+0000 7f242a200700 1 Processor -- start 2024-03-23T18:08:05.552 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.553+0000 7f242a200700 1 -- start start 2024-03-23T18:08:05.553 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.553+0000 7f242a200700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f242410f4d0 0x7f24241a5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:05.553 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.554+0000 7f242a200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f242410f940 0x7f24241a6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:05.553 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.554+0000 7f242a200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f242411a590 0x7f24241aa8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:05.553 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.554+0000 7f242a200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24241aaf00 con 0x7f242410f940 2024-03-23T18:08:05.553 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.554+0000 7f242a200700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24241ab070 con 0x7f242411a590 2024-03-23T18:08:05.554 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.554+0000 7f242a200700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24241ab230 con 0x7f242410f4d0 2024-03-23T18:08:05.554 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.554+0000 7f2422a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f242410f940 0x7f24241a6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:05.554 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.554+0000 7f2422a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f242410f940 0x7f24241a6520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58564/0 (socket says 172.21.15.86:58564) 2024-03-23T18:08:05.554 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.554+0000 7f2422a00700 1 -- 172.21.15.86:0/2258806761 learned_addr learned my addr 172.21.15.86:0/2258806761 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:08:05.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.557+0000 7f2422a00700 1 -- 172.21.15.86:0/2258806761 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f242410f4d0 msgr2=0x7f24241a5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:05.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.557+0000 7f2422a00700 1 --2- 172.21.15.86:0/2258806761 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f242410f4d0 0x7f24241a5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:05.557 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.557+0000 7f2422a00700 1 -- 172.21.15.86:0/2258806761 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f242411a590 msgr2=0x7f24241aa8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:05.557 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.557+0000 7f2422a00700 1 --2- 172.21.15.86:0/2258806761 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f242411a590 0x7f24241aa8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:05.557 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.557+0000 7f2422a00700 1 -- 172.21.15.86:0/2258806761 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24140095f0 con 0x7f242410f940 2024-03-23T18:08:05.557 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.557+0000 7f2422a00700 1 --2- 172.21.15.86:0/2258806761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f242410f940 0x7f24241a6520 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f241000b550 tx=0x7f241000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:08:05.557 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.558+0000 7f241be00700 1 -- 172.21.15.86:0/2258806761 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2410011830 con 0x7f242410f940 2024-03-23T18:08:05.558 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.558+0000 7f242a200700 1 -- 172.21.15.86:0/2258806761 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24241ab510 con 0x7f242410f940 2024-03-23T18:08:05.558 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.558+0000 7f241be00700 1 -- 172.21.15.86:0/2258806761 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2410011e50 con 0x7f242410f940 2024-03-23T18:08:05.558 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.558+0000 7f241be00700 1 -- 172.21.15.86:0/2258806761 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2410010e40 con 0x7f242410f940 2024-03-23T18:08:05.558 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.558+0000 7f242a200700 1 -- 172.21.15.86:0/2258806761 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24241b2d50 con 0x7f242410f940 2024-03-23T18:08:05.566 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.566+0000 7f241be00700 1 -- 172.21.15.86:0/2258806761 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f241001a400 con 0x7f242410f940 2024-03-23T18:08:05.567 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.566+0000 7f243dc86b80 1 Processor -- start 2024-03-23T18:08:05.567 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.567+0000 7f243dc86b80 1 -- start start 2024-03-23T18:08:05.568 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.568+0000 7f243dc86b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c1d908100 0x557c1d9084e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:05.568 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.568+0000 7f243dc86b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557c1d908a20 0x557c1d913390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:05.568 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.568+0000 7f243dc86b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557c1d9138d0 0x557c1d915cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:05.568 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.568+0000 7f243dc86b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c1d861930 con 0x557c1d908100 2024-03-23T18:08:05.568 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.568+0000 7f243dc86b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c1d85f2f0 con 0x557c1d908a20 2024-03-23T18:08:05.568 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.568+0000 7f243dc86b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c1d85abe0 con 0x557c1d9138d0 2024-03-23T18:08:05.568 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.568+0000 7f2423e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c1d908100 0x557c1d9084e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:05.569 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.569+0000 7f2422a00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557c1d9138d0 0x557c1d915cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:05.569 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.569+0000 7f2423e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c1d908100 0x557c1d9084e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58574/0 (socket says 172.21.15.86:58574) 2024-03-23T18:08:05.569 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.569+0000 7f2423e00700 1 -- 172.21.15.86:0/4141670296 learned_addr learned my addr 172.21.15.86:0/4141670296 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:08:05.570 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.570+0000 7f2423e00700 1 -- 172.21.15.86:0/4141670296 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557c1d9138d0 msgr2=0x557c1d915cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:08:05.570 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.570+0000 7f2423e00700 1 --2- 172.21.15.86:0/4141670296 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557c1d9138d0 0x557c1d915cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:05.571 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.570+0000 7f2423e00700 1 -- 172.21.15.86:0/4141670296 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557c1d908a20 msgr2=0x557c1d913390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:08:05.571 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.570+0000 7f2423e00700 1 --2- 172.21.15.86:0/4141670296 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557c1d908a20 0x557c1d913390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:05.571 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.570+0000 7f2423e00700 1 -- 172.21.15.86:0/4141670296 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c1d6dc350 con 0x557c1d908100 2024-03-23T18:08:05.571 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.570+0000 7f241be00700 1 --2- 172.21.15.86:0/2258806761 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f24080a1310 0x7f24080a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:05.571 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.571+0000 7f241be00700 1 -- 172.21.15.86:0/2258806761 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f24100cc5e0 con 0x7f242410f940 2024-03-23T18:08:05.571 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.571+0000 7f2423400700 1 --2- 172.21.15.86:0/2258806761 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f24080a1310 0x7f24080a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:05.571 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.572+0000 7f2423e00700 1 --2- 172.21.15.86:0/4141670296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c1d908100 0x557c1d9084e0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f241c009c00 tx=0x7f241c009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9ccf064f443645 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:08:05.572 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.573+0000 7f2423400700 1 --2- 172.21.15.86:0/2258806761 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f24080a1310 0x7f24080a37d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f24140098e0 tx=0x7f2414000bc0 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:08:05.575 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.575+0000 7f240fe00700 1 -- 172.21.15.86:0/4141670296 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241c004bb0 con 0x557c1d908100 2024-03-23T18:08:05.575 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.575+0000 7f240fe00700 1 -- 172.21.15.86:0/4141670296 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f241c00f040 con 0x557c1d908100 2024-03-23T18:08:05.576 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.576+0000 7f240fe00700 1 -- 172.21.15.86:0/4141670296 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241c013650 con 0x557c1d908100 2024-03-23T18:08:05.576 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.577+0000 7f243dc86b80 1 -- 172.21.15.86:0/4141670296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c1d908100 msgr2=0x557c1d9084e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:05.577 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.577+0000 7f243dc86b80 1 --2- 172.21.15.86:0/4141670296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c1d908100 0x557c1d9084e0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f241c009c00 tx=0x7f241c009f10 comp rx=0 tx=0).stop 2024-03-23T18:08:05.577 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.577+0000 7f243dc86b80 1 -- 172.21.15.86:0/4141670296 shutdown_connections 2024-03-23T18:08:05.577 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.577+0000 7f243dc86b80 1 --2- 172.21.15.86:0/4141670296 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557c1d9138d0 0x557c1d915cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:05.577 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.577+0000 7f243dc86b80 1 --2- 172.21.15.86:0/4141670296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c1d908100 0x557c1d9084e0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:05.577 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.577+0000 7f243dc86b80 1 --2- 172.21.15.86:0/4141670296 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557c1d908a20 0x557c1d913390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:05.577 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.577+0000 7f243dc86b80 1 -- 172.21.15.86:0/4141670296 >> 172.21.15.86:0/4141670296 conn(0x557c1d868640 msgr2=0x557c1d904520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:08:05.578 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.577+0000 7f243dc86b80 1 -- 172.21.15.86:0/4141670296 shutdown_connections 2024-03-23T18:08:05.578 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.577+0000 7f243dc86b80 1 -- 172.21.15.86:0/4141670296 wait complete. 2024-03-23T18:08:05.579 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.578+0000 7f243dc86b80 1 -- start start 2024-03-23T18:08:05.579 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.578+0000 7f243dc86b80 1 Processor -- start 2024-03-23T18:08:05.580 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.579+0000 7f243dc86b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557c1d908100 0x557c1d913270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:05.580 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.579+0000 7f243dc86b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c1d908a20 0x557c1d9105b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:05.580 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.579+0000 7f243dc86b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557c1d9138d0 0x557c1d910b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:05.580 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.579+0000 7f243dc86b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c1d7f6d20 con 0x557c1d908a20 2024-03-23T18:08:05.580 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.579+0000 7f243dc86b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c1d7f69c0 con 0x557c1d908100 2024-03-23T18:08:05.580 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.579+0000 7f243dc86b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c1d6de640 con 0x557c1d9138d0 2024-03-23T18:08:05.580 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.579+0000 7f243dc86b80 10 client.0 _ll_register_callbacks cb 0x557c1d800410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T18:08:05.580 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.580+0000 7f2423400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557c1d908100 0x557c1d913270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:05.582 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.580+0000 7f2423400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557c1d908100 0x557c1d913270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:38210/0 (socket says 172.21.15.86:38210) 2024-03-23T18:08:05.582 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.580+0000 7f2423400700 1 -- 172.21.15.86:0/312332992 learned_addr learned my addr 172.21.15.86:0/312332992 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:08:05.582 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.580+0000 7f2423e00700 1 --2- 172.21.15.86:0/312332992 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557c1d9138d0 0x557c1d910b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:05.582 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.581+0000 7f2423400700 1 -- 172.21.15.86:0/312332992 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557c1d9138d0 msgr2=0x557c1d910b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:08:05.582 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.581+0000 7f2423400700 1 --2- 172.21.15.86:0/312332992 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557c1d9138d0 0x557c1d910b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:05.583 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.581+0000 7f2423400700 1 -- 172.21.15.86:0/312332992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c1d908a20 msgr2=0x557c1d9105b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:08:05.583 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.581+0000 7f2423400700 1 --2- 172.21.15.86:0/312332992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c1d908a20 0x557c1d9105b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:05.583 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.581+0000 7f2423400700 1 -- 172.21.15.86:0/312332992 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f241c0098c0 con 0x557c1d908100 2024-03-23T18:08:05.587 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.587+0000 7f2423400700 1 --2- 172.21.15.86:0/312332992 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557c1d908100 0x557c1d913270 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f241400ba10 tx=0x7f241400b540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f5129a195c4509b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:08:05.587 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.587+0000 7f240cc00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T18:08:05.587 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.587+0000 7f240cc00700 1 -- 172.21.15.86:0/312332992 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2414029070 con 0x557c1d908100 2024-03-23T18:08:05.587 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.588+0000 7f240cc00700 1 -- 172.21.15.86:0/312332992 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f241400be80 con 0x557c1d908100 2024-03-23T18:08:05.587 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.588+0000 7f243dc86b80 1 -- 172.21.15.86:0/312332992 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557c1d81b320 con 0x557c1d908100 2024-03-23T18:08:05.588 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.589+0000 7f240cc00700 1 -- 172.21.15.86:0/312332992 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f241401d070 con 0x557c1d908100 2024-03-23T18:08:05.588 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.589+0000 7f243dc86b80 10 client.27374 fetch_fsmap learned FSMap version 90 2024-03-23T18:08:05.589 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.589+0000 7f243dc86b80 1 -- 172.21.15.86:0/312332992 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=4+}) v3 -- 0x557c1d688b00 con 0x557c1d908100 2024-03-23T18:08:05.589 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.589+0000 7f240cc00700 1 -- 172.21.15.86:0/312332992 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f241403a080 con 0x557c1d908100 2024-03-23T18:08:05.589 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.589+0000 7f240cc00700 1 -- 172.21.15.86:0/312332992 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 90) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f2414034030 con 0x557c1d908100 2024-03-23T18:08:05.591 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.591+0000 7f243dc86b80 10 client.27374 fetch_fsmap finished waiting for FSMap version 90 2024-03-23T18:08:05.591 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.591+0000 7f243dc86b80 10 client.27374 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T18:08:05.591 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.591+0000 7f243dc86b80 20 client.27374 populate_metadata read hostname 'smithi086' 2024-03-23T18:08:05.592 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.591+0000 7f243dc86b80 1 --2- 172.21.15.86:0/312332992 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x557c1d923570 0x557c1d925990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:05.592 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.591+0000 7f243dc86b80 4 client.27374 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T18:08:05.592 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.591+0000 7f243dc86b80 1 -- 172.21.15.86:0/312332992 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557c1d8536a0 con 0x557c1d923570 2024-03-23T18:08:05.592 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.591+0000 7f2422a00700 1 --2- 172.21.15.86:0/312332992 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x557c1d923570 0x557c1d925990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:05.592 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.592+0000 7f2422a00700 1 --2- 172.21.15.86:0/312332992 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x557c1d923570 0x557c1d925990 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7085b9fef4d6fc60 server_cookie=ab90e2023b8b17c0 in_seq=0 out_seq=0 2024-03-23T18:08:05.592 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.592+0000 7f240cc00700 10 client.27374 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T18:08:05.594 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.595+0000 7f240cc00700 1 -- 172.21.15.86:0/312332992 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x557c1d8536a0 con 0x557c1d923570 2024-03-23T18:08:05.595 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.595+0000 7f240cc00700 10 client.27374 handle_command_reply: tid=0 2024-03-23T18:08:05.595 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.595+0000 7f243dc86b80 1 client.27374 shutdown 2024-03-23T18:08:05.595 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.596+0000 7f243dc86b80 1 -- 172.21.15.86:0/312332992 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557c1d908100 msgr2=0x557c1d913270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:05.595 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.596+0000 7f243dc86b80 1 --2- 172.21.15.86:0/312332992 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557c1d908100 0x557c1d913270 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f241400ba10 tx=0x7f241400b540 comp rx=0 tx=0).stop 2024-03-23T18:08:05.596 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.596+0000 7f243dc86b80 1 -- 172.21.15.86:0/312332992 shutdown_connections 2024-03-23T18:08:05.596 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.596+0000 7f243dc86b80 1 --2- 172.21.15.86:0/312332992 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x557c1d923570 0x557c1d925990 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:05.596 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.596+0000 7f243dc86b80 1 --2- 172.21.15.86:0/312332992 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557c1d9138d0 0x557c1d910b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:05.596 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.596+0000 7f240cc00700 0 client.27374 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T18:08:05.596 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.596+0000 7f243dc86b80 1 --2- 172.21.15.86:0/312332992 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557c1d908100 0x557c1d913270 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:05.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.596+0000 7f243dc86b80 1 --2- 172.21.15.86:0/312332992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c1d908a20 0x557c1d9105b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:05.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.596+0000 7f243dc86b80 1 -- 172.21.15.86:0/312332992 >> 172.21.15.86:0/312332992 conn(0x557c1d868640 msgr2=0x557c1d90a0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:08:05.598 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.598+0000 7f243dc86b80 1 -- 172.21.15.86:0/312332992 shutdown_connections 2024-03-23T18:08:05.599 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.598+0000 7f243dc86b80 1 -- 172.21.15.86:0/312332992 wait complete. 2024-03-23T18:08:05.599 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.599+0000 7f243dc86b80 20 client.27374 trim_cache size 0 max 16384 2024-03-23T18:08:05.628 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.628+0000 7f243dc86b80 1 Processor -- start 2024-03-23T18:08:05.629 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.629+0000 7f243dc86b80 1 -- start start 2024-03-23T18:08:05.629 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.630+0000 7f243dc86b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557c1d908100 0x557c1d910b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:05.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.630+0000 7f243dc86b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557c1d908a20 0x557c1d92a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:05.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.630+0000 7f243dc86b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c1d9138d0 0x557c1d91a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:05.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.630+0000 7f243dc86b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c1d900400 con 0x557c1d9138d0 2024-03-23T18:08:05.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.630+0000 7f243dc86b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c1d909ca0 con 0x557c1d908a20 2024-03-23T18:08:05.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.630+0000 7f243dc86b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c1d910930 con 0x557c1d908100 2024-03-23T18:08:05.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.630+0000 7f2423400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557c1d908100 0x557c1d910b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:05.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.630+0000 7f2423400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557c1d908100 0x557c1d910b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:51434/0 (socket says 172.21.15.86:51434) 2024-03-23T18:08:05.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.630+0000 7f2423400700 1 -- 172.21.15.86:0/2370636680 learned_addr learned my addr 172.21.15.86:0/2370636680 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:08:05.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.631+0000 7f2423e00700 1 --2- 172.21.15.86:0/2370636680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c1d9138d0 0x557c1d91a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:05.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.631+0000 7f2423400700 1 -- 172.21.15.86:0/2370636680 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557c1d908a20 msgr2=0x557c1d92a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:08:05.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.631+0000 7f2423400700 1 --2- 172.21.15.86:0/2370636680 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557c1d908a20 0x557c1d92a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:05.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.631+0000 7f2423400700 1 -- 172.21.15.86:0/2370636680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c1d9138d0 msgr2=0x557c1d91a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:08:05.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.631+0000 7f2423400700 1 --2- 172.21.15.86:0/2370636680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c1d9138d0 0x557c1d91a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:05.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.631+0000 7f2423400700 1 -- 172.21.15.86:0/2370636680 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c1d820e40 con 0x557c1d908100 2024-03-23T18:08:05.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.633+0000 7f2423400700 1 --2- 172.21.15.86:0/2370636680 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557c1d908100 0x557c1d910b30 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f2414009580 tx=0x7f241400bf80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b87565fb65482d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:08:05.634 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.634+0000 7f2419200700 1 -- 172.21.15.86:0/2370636680 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2414029070 con 0x557c1d908100 2024-03-23T18:08:05.634 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.634+0000 7f2419200700 1 -- 172.21.15.86:0/2370636680 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557c1d81b320 con 0x557c1d908100 2024-03-23T18:08:05.634 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.634+0000 7f2419200700 1 -- 172.21.15.86:0/2370636680 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241400f930 con 0x557c1d908100 2024-03-23T18:08:05.635 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.635+0000 7f243dc86b80 1 -- 172.21.15.86:0/2370636680 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557c1d908100 msgr2=0x557c1d910b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:05.635 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.635+0000 7f243dc86b80 1 --2- 172.21.15.86:0/2370636680 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557c1d908100 0x557c1d910b30 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f2414009580 tx=0x7f241400bf80 comp rx=0 tx=0).stop 2024-03-23T18:08:05.636 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.636+0000 7f243dc86b80 1 -- 172.21.15.86:0/2370636680 shutdown_connections 2024-03-23T18:08:05.636 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.636+0000 7f243dc86b80 1 --2- 172.21.15.86:0/2370636680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c1d9138d0 0x557c1d91a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:05.636 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.636+0000 7f243dc86b80 1 --2- 172.21.15.86:0/2370636680 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557c1d908100 0x557c1d910b30 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:05.636 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.636+0000 7f243dc86b80 1 --2- 172.21.15.86:0/2370636680 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557c1d908a20 0x557c1d92a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:05.636 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.636+0000 7f243dc86b80 1 -- 172.21.15.86:0/2370636680 >> 172.21.15.86:0/2370636680 conn(0x557c1d868640 msgr2=0x557c1d903eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:08:05.637 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.636+0000 7f243dc86b80 1 -- 172.21.15.86:0/2370636680 shutdown_connections 2024-03-23T18:08:05.637 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.637+0000 7f243dc86b80 1 -- 172.21.15.86:0/2370636680 wait complete. 2024-03-23T18:08:05.637 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.637+0000 7f243dc86b80 1 -- start start 2024-03-23T18:08:05.637 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.638+0000 7f243dc86b80 1 Processor -- start 2024-03-23T18:08:05.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.638+0000 7f243dc86b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557c1d908100 0x557c1d9187e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:05.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.638+0000 7f243dc86b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557c1d908a20 0x557c1d918d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:05.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.638+0000 7f243dc86b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c1d9138d0 0x557c1d96e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:05.639 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.638+0000 7f243dc86b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c1d96e610 con 0x557c1d9138d0 2024-03-23T18:08:05.639 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.638+0000 7f243dc86b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c1d96e780 con 0x557c1d908a20 2024-03-23T18:08:05.639 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.638+0000 7f243dc86b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c1d96e940 con 0x557c1d908100 2024-03-23T18:08:05.639 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.638+0000 7f243dc86b80 10 client.0 _ll_register_callbacks cb 0x557c1d800410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T18:08:05.639 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.639+0000 7f2422a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557c1d908a20 0x557c1d918d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:05.639 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.639+0000 7f2422a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557c1d908a20 0x557c1d918d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:38240/0 (socket says 172.21.15.86:38240) 2024-03-23T18:08:05.639 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.639+0000 7f2422a00700 1 -- 172.21.15.86:0/2538725054 learned_addr learned my addr 172.21.15.86:0/2538725054 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:08:05.639 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.639+0000 7f2423e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c1d9138d0 0x557c1d96e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:05.640 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.640+0000 7f2422a00700 1 -- 172.21.15.86:0/2538725054 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557c1d908100 msgr2=0x557c1d9187e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:08:05.640 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.640+0000 7f2422a00700 1 --2- 172.21.15.86:0/2538725054 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557c1d908100 0x557c1d9187e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:05.640 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.640+0000 7f2422a00700 1 -- 172.21.15.86:0/2538725054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c1d9138d0 msgr2=0x557c1d96e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T18:08:05.641 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.640+0000 7f2422a00700 1 --2- 172.21.15.86:0/2538725054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c1d9138d0 0x557c1d96e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:05.641 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.641+0000 7f2422a00700 1 -- 172.21.15.86:0/2538725054 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557c1d688b00 con 0x557c1d908a20 2024-03-23T18:08:05.641 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.641+0000 7f2423e00700 1 --2- 172.21.15.86:0/2538725054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c1d9138d0 0x557c1d96e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T18:08:05.641 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.641+0000 7f2422a00700 1 --2- 172.21.15.86:0/2538725054 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557c1d908a20 0x557c1d918d20 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f24100c9ea0 tx=0x7f24100cff50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48f34bfb9636faa0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:08:05.641 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.641+0000 7f240cc00700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-23T18:08:05.641 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.641+0000 7f240cc00700 1 -- 172.21.15.86:0/2538725054 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2410099340 con 0x557c1d908a20 2024-03-23T18:08:05.642 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.641+0000 7f240cc00700 1 -- 172.21.15.86:0/2538725054 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f2410099f00 con 0x557c1d908a20 2024-03-23T18:08:05.642 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.641+0000 7f243dc86b80 1 -- 172.21.15.86:0/2538725054 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f241c00f040 con 0x557c1d908a20 2024-03-23T18:08:05.643 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.643+0000 7f240cc00700 1 -- 172.21.15.86:0/2538725054 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2410099160 con 0x557c1d908a20 2024-03-23T18:08:05.643 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.643+0000 7f243dc86b80 10 client.27377 fetch_fsmap learned FSMap version 90 2024-03-23T18:08:05.643 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.643+0000 7f243dc86b80 1 -- 172.21.15.86:0/2538725054 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=4+}) v3 -- 0x557c1d6e0c70 con 0x557c1d908a20 2024-03-23T18:08:05.644 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.644+0000 7f240cc00700 1 -- 172.21.15.86:0/2538725054 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f241c00f040 con 0x557c1d908a20 2024-03-23T18:08:05.644 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.644+0000 7f240cc00700 1 -- 172.21.15.86:0/2538725054 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 90) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f24100b90b0 con 0x557c1d908a20 2024-03-23T18:08:05.644 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.644+0000 7f243dc86b80 10 client.27377 fetch_fsmap finished waiting for FSMap version 90 2024-03-23T18:08:05.644 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.644+0000 7f243dc86b80 10 client.27377 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T18:08:05.644 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.644+0000 7f243dc86b80 20 client.27377 populate_metadata read hostname 'smithi086' 2024-03-23T18:08:05.645 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.644+0000 7f243dc86b80 1 --2- 172.21.15.86:0/2538725054 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x557c1d923570 0x557c1d9782b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:05.645 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.644+0000 7f243dc86b80 4 client.27377 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T18:08:05.645 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.644+0000 7f243dc86b80 1 -- 172.21.15.86:0/2538725054 --> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557c1d6a9c90 con 0x557c1d923570 2024-03-23T18:08:05.646 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.646+0000 7f2423400700 1 --2- 172.21.15.86:0/2538725054 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x557c1d923570 0x557c1d9782b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:05.648 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.646+0000 7f2423400700 1 --2- 172.21.15.86:0/2538725054 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x557c1d923570 0x557c1d9782b0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a6842a35c3e40af server_cookie=e7b2ac8667f2722e in_seq=0 out_seq=0 2024-03-23T18:08:05.648 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.646+0000 7f240cc00700 10 client.27377 ms_handle_connect on v2:172.21.15.92:6838/1394139887 2024-03-23T18:08:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.975+0000 7f240cc00700 1 -- 172.21.15.86:0/2538725054 <== mds.0 v2:172.21.15.92:6838/1394139887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557c1d6a9c90 con 0x557c1d923570 2024-03-23T18:08:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.975+0000 7f240cc00700 10 client.27377 handle_command_reply: tid=0 2024-03-23T18:08:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.975+0000 7f243dc86b80 1 client.27377 shutdown 2024-03-23T18:08:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.975+0000 7f243dc86b80 1 -- 172.21.15.86:0/2538725054 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557c1d908a20 msgr2=0x557c1d918d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.975+0000 7f243dc86b80 1 --2- 172.21.15.86:0/2538725054 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557c1d908a20 0x557c1d918d20 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f24100c9ea0 tx=0x7f24100cff50 comp rx=0 tx=0).stop 2024-03-23T18:08:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.975+0000 7f243dc86b80 1 -- 172.21.15.86:0/2538725054 shutdown_connections 2024-03-23T18:08:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.975+0000 7f243dc86b80 1 --2- 172.21.15.86:0/2538725054 >> [v2:172.21.15.92:6838/1394139887,v1:172.21.15.92:6839/1394139887] conn(0x557c1d923570 0x557c1d9782b0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.975+0000 7f243dc86b80 1 --2- 172.21.15.86:0/2538725054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c1d9138d0 0x557c1d96e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.975+0000 7f243dc86b80 1 --2- 172.21.15.86:0/2538725054 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557c1d908100 0x557c1d9187e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.975+0000 7f243dc86b80 1 --2- 172.21.15.86:0/2538725054 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557c1d908a20 0x557c1d918d20 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.975+0000 7f240cc00700 0 client.27377 ms_handle_reset on v2:172.21.15.92:6838/1394139887 2024-03-23T18:08:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.975+0000 7f243dc86b80 1 -- 172.21.15.86:0/2538725054 >> 172.21.15.86:0/2538725054 conn(0x557c1d868640 msgr2=0x557c1d903e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:08:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.976+0000 7f243dc86b80 1 -- 172.21.15.86:0/2538725054 shutdown_connections 2024-03-23T18:08:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.976+0000 7f243dc86b80 1 -- 172.21.15.86:0/2538725054 wait complete. 2024-03-23T18:08:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.976+0000 7f243dc86b80 20 client.27377 trim_cache size 0 max 16384 2024-03-23T18:08:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-03-23T18:08:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-03-23T18:08:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-03-23T18:08:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-03-23T18:08:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.977+0000 7f242a200700 1 -- 172.21.15.86:0/2258806761 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f24080a1310 msgr2=0x7f24080a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.977+0000 7f242a200700 1 --2- 172.21.15.86:0/2258806761 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f24080a1310 0x7f24080a37d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f24140098e0 tx=0x7f2414000bc0 comp rx=0 tx=0).stop 2024-03-23T18:08:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.977+0000 7f242a200700 1 -- 172.21.15.86:0/2258806761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f242410f940 msgr2=0x7f24241a6520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.977+0000 7f242a200700 1 --2- 172.21.15.86:0/2258806761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f242410f940 0x7f24241a6520 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f241000b550 tx=0x7f241000b910 comp rx=0 tx=0).stop 2024-03-23T18:08:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.977+0000 7f242a200700 1 -- 172.21.15.86:0/2258806761 shutdown_connections 2024-03-23T18:08:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.977+0000 7f242a200700 1 --2- 172.21.15.86:0/2258806761 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f242411a590 0x7f24241aa8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.977+0000 7f242a200700 1 --2- 172.21.15.86:0/2258806761 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f24080a1310 0x7f24080a37d0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.977+0000 7f242a200700 1 --2- 172.21.15.86:0/2258806761 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f242410f4d0 0x7f24241a5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.977+0000 7f242a200700 1 --2- 172.21.15.86:0/2258806761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f242410f940 0x7f24241a6520 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.977+0000 7f242a200700 1 -- 172.21.15.86:0/2258806761 >> 172.21.15.86:0/2258806761 conn(0x7f242406bd80 msgr2=0x7f2424118580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:08:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.978+0000 7f242a200700 1 -- 172.21.15.86:0/2258806761 shutdown_connections 2024-03-23T18:08:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-03-23T18:08:05.978+0000 7f242a200700 1 -- 172.21.15.86:0/2258806761 wait complete. 2024-03-23T18:08:06.248 INFO:tasks.workunit.client.0.smithi086.stdout: 10 186173 35.22 MB/sec execute 150 sec latency 184.558 ms 2024-03-23T18:08:06.308 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:06 smithi092 ceph-mon[79089]: pgmap v551: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 15 MiB/s wr, 877 op/s 2024-03-23T18:08:06.309 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:06 smithi092 ceph-mon[79089]: osdmap e109: 12 total, 12 up, 12 in 2024-03-23T18:08:06.338 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:06 smithi152 ceph-mon[47949]: pgmap v551: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 15 MiB/s wr, 877 op/s 2024-03-23T18:08:06.339 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:06 smithi152 ceph-mon[47949]: osdmap e109: 12 total, 12 up, 12 in 2024-03-23T18:08:06.518 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:06 smithi086 ceph-mon[38221]: pgmap v551: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 15 MiB/s wr, 877 op/s 2024-03-23T18:08:06.518 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:06 smithi086 ceph-mon[38221]: osdmap e109: 12 total, 12 up, 12 in 2024-03-23T18:08:07.248 INFO:tasks.workunit.client.0.smithi086.stdout: 10 186456 35.07 MB/sec execute 151 sec latency 160.274 ms 2024-03-23T18:08:08.020 INFO:tasks.workunit.client.0.smithi086.stdout:[203711] write failed on handle 12093 (Resource temporarily unavailable) 2024-03-23T18:08:08.020 INFO:tasks.workunit.client.0.smithi086.stdout:Child failed with status 1 2024-03-23T18:08:08.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T18:08:08.055 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2024-03-23T18:08:08.055 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-03-23T18:08:08.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:08 smithi092 ceph-mon[79089]: pgmap v553: 97 pgs: 15 active+clean+snaptrim_wait, 76 active+clean, 6 active+clean+snaptrim; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.4 MiB/s wr, 726 op/s 2024-03-23T18:08:08.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:08 smithi152 ceph-mon[47949]: pgmap v553: 97 pgs: 15 active+clean+snaptrim_wait, 76 active+clean, 6 active+clean+snaptrim; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.4 MiB/s wr, 726 op/s 2024-03-23T18:08:08.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:08 smithi086 ceph-mon[38221]: pgmap v553: 97 pgs: 15 active+clean+snaptrim_wait, 76 active+clean, 6 active+clean+snaptrim; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.4 MiB/s wr, 726 op/s 2024-03-23T18:08:09.242 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/github.com_ceph_ceph-c_82316b04821a6c2c1d458a881920454bbadb8837/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/github.com_ceph_ceph-c_82316b04821a6c2c1d458a881920454bbadb8837/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/github.com_ceph_ceph-c_82316b04821a6c2c1d458a881920454bbadb8837/qa/tasks/workunit.py", line 424, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi086 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=82316b04821a6c2c1d458a881920454bbadb8837 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-03-23T18:08:09.539 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=a3f90f89b38047109c856a74eb2e73d1 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/github.com_ceph_ceph-c_82316b04821a6c2c1d458a881920454bbadb8837/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/github.com_ceph_ceph-c_82316b04821a6c2c1d458a881920454bbadb8837/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/github.com_ceph_ceph-c_82316b04821a6c2c1d458a881920454bbadb8837/qa/tasks/workunit.py", line 424, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi086 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=82316b04821a6c2c1d458a881920454bbadb8837 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-03-23T18:08:09.541 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-03-23T18:08:09.553 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi086.front.sepia.ceph.com 2024-03-23T18:08:09.566 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-03-23T18:08:09.577 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-03-23T18:08:09.577 INFO:teuthology.task.exec:Executing custom commands... 2024-03-23T18:08:09.577 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi086.front.sepia.ceph.com 2024-03-23T18:08:09.577 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-03-23T18:08:09.675 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-03-23T18:08:09.748 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-03-23T18:08:09.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:09.947+0000 7faf7fa00700 1 -- 172.21.15.86:0/1565988923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf78100f10 msgr2=0x7faf781012f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:09.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:09.947+0000 7faf7fa00700 1 --2- 172.21.15.86:0/1565988923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf78100f10 0x7faf781012f0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7faf6c009990 tx=0x7faf6c009ca0 comp rx=0 tx=0).stop 2024-03-23T18:08:09.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:09.947+0000 7faf7fa00700 1 -- 172.21.15.86:0/1565988923 shutdown_connections 2024-03-23T18:08:09.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:09.947+0000 7faf7fa00700 1 --2- 172.21.15.86:0/1565988923 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf78106030 0x7faf7810dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:09.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:09.947+0000 7faf7fa00700 1 --2- 172.21.15.86:0/1565988923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf78100f10 0x7faf781012f0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:09.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:09.947+0000 7faf7fa00700 1 --2- 172.21.15.86:0/1565988923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf781018c0 0x7faf78105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:09.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:09.947+0000 7faf7fa00700 1 -- 172.21.15.86:0/1565988923 >> 172.21.15.86:0/1565988923 conn(0x7faf780fc770 msgr2=0x7faf780feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:08:09.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:09.947+0000 7faf7fa00700 1 -- 172.21.15.86:0/1565988923 shutdown_connections 2024-03-23T18:08:09.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:09.947+0000 7faf7fa00700 1 -- 172.21.15.86:0/1565988923 wait complete. 2024-03-23T18:08:09.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:09.949+0000 7faf7fa00700 1 Processor -- start 2024-03-23T18:08:09.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:09.949+0000 7faf7fa00700 1 -- start start 2024-03-23T18:08:09.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:09.949+0000 7faf7fa00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf78100f10 0x7faf780730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:09.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:09.949+0000 7faf7fa00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf78106030 0x7faf780735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:09.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:09.949+0000 7faf7fa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf78073b30 0x7faf781a5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:09.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:09.950+0000 7faf7fa00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf781a5a30 con 0x7faf78073b30 2024-03-23T18:08:09.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:09.950+0000 7faf7fa00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf781a5ba0 con 0x7faf78106030 2024-03-23T18:08:09.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:09.950+0000 7faf7fa00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf781a5d60 con 0x7faf78100f10 2024-03-23T18:08:09.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:09.950+0000 7faf77e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf78106030 0x7faf780735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:09.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:09.950+0000 7faf7d000700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf78100f10 0x7faf780730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:09.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:09.950+0000 7faf77e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf78106030 0x7faf780735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:38262/0 (socket says 172.21.15.86:38262) 2024-03-23T18:08:09.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:09.950+0000 7faf77e00700 1 -- 172.21.15.86:0/2970374143 learned_addr learned my addr 172.21.15.86:0/2970374143 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:08:09.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:09.950+0000 7faf7da00700 1 --2- 172.21.15.86:0/2970374143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf78073b30 0x7faf781a5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:09.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:09.950+0000 7faf77e00700 1 -- 172.21.15.86:0/2970374143 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf78100f10 msgr2=0x7faf780730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:09.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:09.950+0000 7faf77e00700 1 --2- 172.21.15.86:0/2970374143 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf78100f10 0x7faf780730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:09.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:09.950+0000 7faf77e00700 1 -- 172.21.15.86:0/2970374143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf78073b30 msgr2=0x7faf781a5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:09.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:09.950+0000 7faf77e00700 1 --2- 172.21.15.86:0/2970374143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf78073b30 0x7faf781a5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:09.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:09.950+0000 7faf77e00700 1 -- 172.21.15.86:0/2970374143 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf6c0095f0 con 0x7faf78106030 2024-03-23T18:08:09.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:09.951+0000 7faf77e00700 1 --2- 172.21.15.86:0/2970374143 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf78106030 0x7faf780735f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7faf6400e970 tx=0x7faf6400ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:08:09.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:09.951+0000 7faf75600700 1 -- 172.21.15.86:0/2970374143 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf64009790 con 0x7faf78106030 2024-03-23T18:08:09.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:09.951+0000 7faf75600700 1 -- 172.21.15.86:0/2970374143 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf6400eec0 con 0x7faf78106030 2024-03-23T18:08:09.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:09.951+0000 7faf7fa00700 1 -- 172.21.15.86:0/2970374143 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf781a5fe0 con 0x7faf78106030 2024-03-23T18:08:09.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:09.951+0000 7faf75600700 1 -- 172.21.15.86:0/2970374143 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf64005590 con 0x7faf78106030 2024-03-23T18:08:09.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:09.951+0000 7faf7fa00700 1 -- 172.21.15.86:0/2970374143 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf781a64c0 con 0x7faf78106030 2024-03-23T18:08:09.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:09.952+0000 7faf7fa00700 1 -- 172.21.15.86:0/2970374143 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf781092a0 con 0x7faf78106030 2024-03-23T18:08:09.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:09.955+0000 7faf75600700 1 -- 172.21.15.86:0/2970374143 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7faf6400cbd0 con 0x7faf78106030 2024-03-23T18:08:09.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:09.957+0000 7faf75600700 1 --2- 172.21.15.86:0/2970374143 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7faf600a1290 0x7faf600a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:09.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:09.957+0000 7faf75600700 1 -- 172.21.15.86:0/2970374143 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 8528+0+0 (secure 0 0 0) 0x7faf64014070 con 0x7faf78106030 2024-03-23T18:08:09.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:09.957+0000 7faf7d000700 1 --2- 172.21.15.86:0/2970374143 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7faf600a1290 0x7faf600a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:09.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:09.958+0000 7faf7d000700 1 --2- 172.21.15.86:0/2970374143 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7faf600a1290 0x7faf600a3750 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7faf6c00b2e0 tx=0x7faf6c005fd0 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:08:09.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:09.958+0000 7faf75600700 1 -- 172.21.15.86:0/2970374143 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7faf64096320 con 0x7faf78106030 2024-03-23T18:08:10.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.208+0000 7faf7fa00700 1 -- 172.21.15.86:0/2970374143 --> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7faf780745b0 con 0x7faf600a1290 2024-03-23T18:08:10.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.220+0000 7faf75600700 1 -- 172.21.15.86:0/2970374143 <== mgr.15375 v2:172.21.15.86:6800/4022683855 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+351 (secure 0 0 0) 0x7faf780745b0 con 0x7faf600a1290 2024-03-23T18:08:10.220 INFO:teuthology.orchestra.run.smithi086.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-03-23T00:00:00", "created": "2024-03-23T17:50:39", "first": "2024-03-23T17:51:00", "last": "2024-03-23T18:08:00", "last_pruned": "2024-03-23T18:08:00", "created_count": 18, "pruned_count": 11, "active": true} 2024-03-23T18:08:10.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.223+0000 7faf7fa00700 1 -- 172.21.15.86:0/2970374143 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7faf600a1290 msgr2=0x7faf600a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:10.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.223+0000 7faf7fa00700 1 --2- 172.21.15.86:0/2970374143 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7faf600a1290 0x7faf600a3750 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7faf6c00b2e0 tx=0x7faf6c005fd0 comp rx=0 tx=0).stop 2024-03-23T18:08:10.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.223+0000 7faf7fa00700 1 -- 172.21.15.86:0/2970374143 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf78106030 msgr2=0x7faf780735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:10.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.223+0000 7faf7fa00700 1 --2- 172.21.15.86:0/2970374143 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf78106030 0x7faf780735f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7faf6400e970 tx=0x7faf6400ec80 comp rx=0 tx=0).stop 2024-03-23T18:08:10.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.223+0000 7faf7fa00700 1 -- 172.21.15.86:0/2970374143 shutdown_connections 2024-03-23T18:08:10.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.223+0000 7faf7fa00700 1 --2- 172.21.15.86:0/2970374143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf78073b30 0x7faf781a5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:10.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.223+0000 7faf7fa00700 1 --2- 172.21.15.86:0/2970374143 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7faf600a1290 0x7faf600a3750 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:10.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.223+0000 7faf7fa00700 1 --2- 172.21.15.86:0/2970374143 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf78100f10 0x7faf780730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:10.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.223+0000 7faf7fa00700 1 --2- 172.21.15.86:0/2970374143 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf78106030 0x7faf780735f0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:10.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.223+0000 7faf7fa00700 1 -- 172.21.15.86:0/2970374143 >> 172.21.15.86:0/2970374143 conn(0x7faf780fc770 msgr2=0x7faf780fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:08:10.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.223+0000 7faf7fa00700 1 -- 172.21.15.86:0/2970374143 shutdown_connections 2024-03-23T18:08:10.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.223+0000 7faf7fa00700 1 -- 172.21.15.86:0/2970374143 wait complete. 2024-03-23T18:08:10.241 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-03-23T18:08:10.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:10 smithi152 ceph-mon[47949]: pgmap v554: 97 pgs: 15 active+clean+snaptrim_wait, 76 active+clean, 6 active+clean+snaptrim; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.4 MiB/s wr, 726 op/s 2024-03-23T18:08:10.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:10 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T18:08:10.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.431+0000 7f42f5800700 1 -- 172.21.15.86:0/2256063887 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42f0100ef0 msgr2=0x7f42f01012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:10.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.431+0000 7f42f5800700 1 --2- 172.21.15.86:0/2256063887 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42f0100ef0 0x7f42f01012d0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f42e0009990 tx=0x7f42e0009ca0 comp rx=0 tx=0).stop 2024-03-23T18:08:10.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.431+0000 7f42f5800700 1 -- 172.21.15.86:0/2256063887 shutdown_connections 2024-03-23T18:08:10.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.431+0000 7f42f5800700 1 --2- 172.21.15.86:0/2256063887 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f42f0106010 0x7f42f010db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:10.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.431+0000 7f42f5800700 1 --2- 172.21.15.86:0/2256063887 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42f0100ef0 0x7f42f01012d0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:10.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.431+0000 7f42f5800700 1 --2- 172.21.15.86:0/2256063887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42f01018a0 0x7f42f01058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:10.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.431+0000 7f42f5800700 1 -- 172.21.15.86:0/2256063887 >> 172.21.15.86:0/2256063887 conn(0x7f42f00fc750 msgr2=0x7f42f00feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:08:10.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.432+0000 7f42f5800700 1 -- 172.21.15.86:0/2256063887 shutdown_connections 2024-03-23T18:08:10.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.432+0000 7f42f5800700 1 -- 172.21.15.86:0/2256063887 wait complete. 2024-03-23T18:08:10.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.432+0000 7f42f5800700 1 Processor -- start 2024-03-23T18:08:10.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.433+0000 7f42f5800700 1 -- start start 2024-03-23T18:08:10.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.433+0000 7f42f5800700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42f0100ef0 0x7f42f0199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:10.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.433+0000 7f42f5800700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f42f0106010 0x7f42f0199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:10.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.433+0000 7f42f5800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42f019dc10 0x7f42f019e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:10.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.433+0000 7f42f5800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f019e690 con 0x7f42f019dc10 2024-03-23T18:08:10.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.433+0000 7f42f5800700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f019e800 con 0x7f42f0100ef0 2024-03-23T18:08:10.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.433+0000 7f42f5800700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f019e9c0 con 0x7f42f0106010 2024-03-23T18:08:10.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.433+0000 7f42ef400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42f019dc10 0x7f42f019e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:10.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.433+0000 7f42ee000700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f42f0106010 0x7f42f0199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:10.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.433+0000 7f42ef400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42f019dc10 0x7f42f019e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58630/0 (socket says 172.21.15.86:58630) 2024-03-23T18:08:10.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.433+0000 7f42ef400700 1 -- 172.21.15.86:0/4166068423 learned_addr learned my addr 172.21.15.86:0/4166068423 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:08:10.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.433+0000 7f42eea00700 1 --2- 172.21.15.86:0/4166068423 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42f0100ef0 0x7f42f0199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:10.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.434+0000 7f42ee000700 1 -- 172.21.15.86:0/4166068423 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42f0100ef0 msgr2=0x7f42f0199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:10.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.434+0000 7f42ee000700 1 --2- 172.21.15.86:0/4166068423 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42f0100ef0 0x7f42f0199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:10.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.434+0000 7f42ee000700 1 -- 172.21.15.86:0/4166068423 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42f019dc10 msgr2=0x7f42f019e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:10.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.434+0000 7f42ee000700 1 --2- 172.21.15.86:0/4166068423 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42f019dc10 0x7f42f019e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:10.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.434+0000 7f42ee000700 1 -- 172.21.15.86:0/4166068423 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42dc009520 con 0x7f42f0106010 2024-03-23T18:08:10.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.434+0000 7f42ef400700 1 --2- 172.21.15.86:0/4166068423 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42f019dc10 0x7f42f019e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T18:08:10.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.434+0000 7f42ee000700 1 --2- 172.21.15.86:0/4166068423 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f42f0106010 0x7f42f0199870 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f42dc00e9c0 tx=0x7f42dc00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:08:10.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.434+0000 7f42e7400700 1 -- 172.21.15.86:0/4166068423 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42dc00cb70 con 0x7f42f0106010 2024-03-23T18:08:10.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.434+0000 7f42e7400700 1 -- 172.21.15.86:0/4166068423 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f42dc00ee60 con 0x7f42f0106010 2024-03-23T18:08:10.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.434+0000 7f42f5800700 1 -- 172.21.15.86:0/4166068423 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42e00095f0 con 0x7f42f0106010 2024-03-23T18:08:10.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.434+0000 7f42e7400700 1 -- 172.21.15.86:0/4166068423 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42dc003c60 con 0x7f42f0106010 2024-03-23T18:08:10.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.434+0000 7f42f5800700 1 -- 172.21.15.86:0/4166068423 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42f01a6210 con 0x7f42f0106010 2024-03-23T18:08:10.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.435+0000 7f42f5800700 1 -- 172.21.15.86:0/4166068423 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42f01023c0 con 0x7f42f0106010 2024-03-23T18:08:10.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.437+0000 7f42e7400700 1 -- 172.21.15.86:0/4166068423 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f42dc005030 con 0x7f42f0106010 2024-03-23T18:08:10.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.438+0000 7f42e7400700 1 --2- 172.21.15.86:0/4166068423 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f42d40a1290 0x7f42d40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:10.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.438+0000 7f42e7400700 1 -- 172.21.15.86:0/4166068423 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f42dc014070 con 0x7f42f0106010 2024-03-23T18:08:10.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.439+0000 7f42eea00700 1 --2- 172.21.15.86:0/4166068423 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f42d40a1290 0x7f42d40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:10.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.439+0000 7f42eea00700 1 --2- 172.21.15.86:0/4166068423 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f42d40a1290 0x7f42d40a3750 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f42e000b2e0 tx=0x7f42e0005960 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:08:10.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.441+0000 7f42e7400700 1 -- 172.21.15.86:0/4166068423 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f42dc096710 con 0x7f42f0106010 2024-03-23T18:08:10.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:10 smithi086 ceph-mon[38221]: pgmap v554: 97 pgs: 15 active+clean+snaptrim_wait, 76 active+clean, 6 active+clean+snaptrim; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.4 MiB/s wr, 726 op/s 2024-03-23T18:08:10.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:10 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T18:08:10.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:10 smithi092 ceph-mon[79089]: pgmap v554: 97 pgs: 15 active+clean+snaptrim_wait, 76 active+clean, 6 active+clean+snaptrim; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.4 MiB/s wr, 726 op/s 2024-03-23T18:08:10.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:10 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T18:08:10.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.674+0000 7f42f5800700 1 -- 172.21.15.86:0/4166068423 --> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f42f0101910 con 0x7f42d40a1290 2024-03-23T18:08:10.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.687+0000 7f42e7400700 1 -- 172.21.15.86:0/4166068423 <== mgr.15375 v2:172.21.15.86:6800/4022683855 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f42f0101910 con 0x7f42d40a1290 2024-03-23T18:08:10.687 INFO:teuthology.orchestra.run.smithi086.stdout:/ 1m 6m3h 2024-03-23T18:08:10.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.689+0000 7f42f5800700 1 -- 172.21.15.86:0/4166068423 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f42d40a1290 msgr2=0x7f42d40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:10.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.689+0000 7f42f5800700 1 --2- 172.21.15.86:0/4166068423 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f42d40a1290 0x7f42d40a3750 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f42e000b2e0 tx=0x7f42e0005960 comp rx=0 tx=0).stop 2024-03-23T18:08:10.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.689+0000 7f42f5800700 1 -- 172.21.15.86:0/4166068423 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f42f0106010 msgr2=0x7f42f0199870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:10.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.689+0000 7f42f5800700 1 --2- 172.21.15.86:0/4166068423 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f42f0106010 0x7f42f0199870 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f42dc00e9c0 tx=0x7f42dc00ecd0 comp rx=0 tx=0).stop 2024-03-23T18:08:10.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.689+0000 7f42f5800700 1 -- 172.21.15.86:0/4166068423 shutdown_connections 2024-03-23T18:08:10.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.689+0000 7f42f5800700 1 --2- 172.21.15.86:0/4166068423 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42f019dc10 0x7f42f019e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:10.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.689+0000 7f42f5800700 1 --2- 172.21.15.86:0/4166068423 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42f0100ef0 0x7f42f0199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:10.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.689+0000 7f42f5800700 1 --2- 172.21.15.86:0/4166068423 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f42d40a1290 0x7f42d40a3750 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:10.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.689+0000 7f42f5800700 1 --2- 172.21.15.86:0/4166068423 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f42f0106010 0x7f42f0199870 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:10.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.690+0000 7f42f5800700 1 -- 172.21.15.86:0/4166068423 >> 172.21.15.86:0/4166068423 conn(0x7f42f00fc750 msgr2=0x7f42f00fdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:08:10.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.690+0000 7f42f5800700 1 -- 172.21.15.86:0/4166068423 shutdown_connections 2024-03-23T18:08:10.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.690+0000 7f42f5800700 1 -- 172.21.15.86:0/4166068423 wait complete. 2024-03-23T18:08:10.708 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-03-23T18:08:10.719 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-03-23T18:08:10.719 ERROR:teuthology.run_tasks:Manager failed: fwd_scrub Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_82316b04821a6c2c1d458a881920454bbadb8837/qa/tasks/fwd_scrub.py", line 164, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/github.com_ceph_ceph-c_82316b04821a6c2c1d458a881920454bbadb8837/qa/tasks/fwd_scrub.py", line 99, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2024-03-23T18:08:10.724 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-03-23T18:08:10.734 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T18:08:10.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.936+0000 7fee99e00700 1 -- 172.21.15.86:0/2012104582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee94105f80 msgr2=0x7fee9410db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:10.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.936+0000 7fee99e00700 1 --2- 172.21.15.86:0/2012104582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee94105f80 0x7fee9410db00 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fee8c00b0a0 tx=0x7fee8c00b3b0 comp rx=0 tx=0).stop 2024-03-23T18:08:10.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.936+0000 7fee99e00700 1 -- 172.21.15.86:0/2012104582 shutdown_connections 2024-03-23T18:08:10.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.936+0000 7fee99e00700 1 --2- 172.21.15.86:0/2012104582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee94105f80 0x7fee9410db00 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:10.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.936+0000 7fee99e00700 1 --2- 172.21.15.86:0/2012104582 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee94100e60 0x7fee94101240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:10.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.936+0000 7fee99e00700 1 --2- 172.21.15.86:0/2012104582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee94101810 0x7fee94105850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:10.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.936+0000 7fee99e00700 1 -- 172.21.15.86:0/2012104582 >> 172.21.15.86:0/2012104582 conn(0x7fee940fc700 msgr2=0x7fee940feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:08:10.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.936+0000 7fee99e00700 1 -- 172.21.15.86:0/2012104582 shutdown_connections 2024-03-23T18:08:10.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.936+0000 7fee99e00700 1 -- 172.21.15.86:0/2012104582 wait complete. 2024-03-23T18:08:10.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.937+0000 7fee99e00700 1 Processor -- start 2024-03-23T18:08:10.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.938+0000 7fee99e00700 1 -- start start 2024-03-23T18:08:10.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.938+0000 7fee99e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee94100e60 0x7fee94197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:10.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.938+0000 7fee99e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee94105f80 0x7fee941977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:10.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.938+0000 7fee99e00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee9419bb10 0x7fee9419bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:10.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.938+0000 7fee99e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee9419c590 con 0x7fee94100e60 2024-03-23T18:08:10.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.938+0000 7fee99e00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee9419c700 con 0x7fee94105f80 2024-03-23T18:08:10.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.938+0000 7fee99e00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee9419c8c0 con 0x7fee9419bb10 2024-03-23T18:08:10.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.938+0000 7fee92a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee94100e60 0x7fee94197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:10.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.938+0000 7fee92a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee94100e60 0x7fee94197280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58644/0 (socket says 172.21.15.86:58644) 2024-03-23T18:08:10.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.938+0000 7fee92a00700 1 -- 172.21.15.86:0/3517863448 learned_addr learned my addr 172.21.15.86:0/3517863448 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:08:10.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.938+0000 7fee92000700 1 --2- 172.21.15.86:0/3517863448 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee94105f80 0x7fee941977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:10.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.939+0000 7fee93400700 1 --2- 172.21.15.86:0/3517863448 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee9419bb10 0x7fee9419bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:10.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.939+0000 7fee92a00700 1 -- 172.21.15.86:0/3517863448 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee9419bb10 msgr2=0x7fee9419bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:10.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.939+0000 7fee92a00700 1 --2- 172.21.15.86:0/3517863448 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee9419bb10 0x7fee9419bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:10.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.939+0000 7fee92a00700 1 -- 172.21.15.86:0/3517863448 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee94105f80 msgr2=0x7fee941977c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:10.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.939+0000 7fee92a00700 1 --2- 172.21.15.86:0/3517863448 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee94105f80 0x7fee941977c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:10.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.939+0000 7fee92a00700 1 -- 172.21.15.86:0/3517863448 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee8c009d00 con 0x7fee94100e60 2024-03-23T18:08:10.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.939+0000 7fee92000700 1 --2- 172.21.15.86:0/3517863448 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee94105f80 0x7fee941977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T18:08:10.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.939+0000 7fee92a00700 1 --2- 172.21.15.86:0/3517863448 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee94100e60 0x7fee94197280 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fee8400d980 tx=0x7fee8400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:08:10.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.939+0000 7fee98a00700 1 -- 172.21.15.86:0/3517863448 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee84004220 con 0x7fee94100e60 2024-03-23T18:08:10.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.939+0000 7fee98a00700 1 -- 172.21.15.86:0/3517863448 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee84007640 con 0x7fee94100e60 2024-03-23T18:08:10.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.939+0000 7fee99e00700 1 -- 172.21.15.86:0/3517863448 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee941a3e20 con 0x7fee94100e60 2024-03-23T18:08:10.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.939+0000 7fee98a00700 1 -- 172.21.15.86:0/3517863448 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee84005690 con 0x7fee94100e60 2024-03-23T18:08:10.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.940+0000 7fee99e00700 1 -- 172.21.15.86:0/3517863448 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee941a42a0 con 0x7fee94100e60 2024-03-23T18:08:10.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.941+0000 7fee99e00700 1 -- 172.21.15.86:0/3517863448 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee94109250 con 0x7fee94100e60 2024-03-23T18:08:10.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.942+0000 7fee98a00700 1 -- 172.21.15.86:0/3517863448 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fee84004750 con 0x7fee94100e60 2024-03-23T18:08:10.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.944+0000 7fee98a00700 1 --2- 172.21.15.86:0/3517863448 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fee780a1290 0x7fee780a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:10.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.944+0000 7fee98a00700 1 -- 172.21.15.86:0/3517863448 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 8528+0+0 (secure 0 0 0) 0x7fee840ccae0 con 0x7fee94100e60 2024-03-23T18:08:10.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.944+0000 7fee92000700 1 --2- 172.21.15.86:0/3517863448 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fee780a1290 0x7fee780a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:10.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.945+0000 7fee92000700 1 --2- 172.21.15.86:0/3517863448 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fee780a1290 0x7fee780a3750 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fee94198780 tx=0x7fee80009260 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:08:10.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:10.948+0000 7fee98a00700 1 -- 172.21.15.86:0/3517863448 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fee84095ef0 con 0x7fee94100e60 2024-03-23T18:08:11.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.203+0000 7fee99e00700 1 -- 172.21.15.86:0/3517863448 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fee94198530 con 0x7fee94100e60 2024-03-23T18:08:11.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.205+0000 7fee98a00700 1 -- 172.21.15.86:0/3517863448 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) v1 ==== 74+0+22952 (secure 0 0 0) 0x7fee84016020 con 0x7fee94100e60 2024-03-23T18:08:11.205 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T18:08:11.206 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":109,"fsid":"9d76c9e8-e93c-11ee-95cd-87774f69a715","created":"2024-03-23T17:42:30.097233+0000","modified":"2024-03-23T18:08:05.030338+0000","last_up_change":"2024-03-23T17:48:18.217601+0000","last_in_change":"2024-03-23T17:48:00.250579+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T17:45:26.848397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T17:48:54.331894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T17:48:55.640693+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":30,"snap_epoch":109,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"656655c6-1c7f-4640-92c4-2f42de820143","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6803","nonce":2423505491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6805","nonce":2423505491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6809","nonce":2423505491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6807","nonce":2423505491}]},"public_addr":"172.21.15.86:6803/2423505491","cluster_addr":"172.21.15.86:6805/2423505491","heartbeat_back_addr":"172.21.15.86:6809/2423505491","heartbeat_front_addr":"172.21.15.86:6807/2423505491","state":["exists","up"]},{"osd":1,"uuid":"3e86dc65-20a6-4de8-b9ce-fcf01f68862b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6801","nonce":3776252597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6803","nonce":3776252597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6807","nonce":3776252597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6805","nonce":3776252597}]},"public_addr":"172.21.15.92:6801/3776252597","cluster_addr":"172.21.15.92:6803/3776252597","heartbeat_back_addr":"172.21.15.92:6807/3776252597","heartbeat_front_addr":"172.21.15.92:6805/3776252597","state":["exists","up"]},{"osd":2,"uuid":"09808786-3bd3-4d11-8c7b-1736119dfbb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6801","nonce":1601265891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6803","nonce":1601265891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6807","nonce":1601265891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6805","nonce":1601265891}]},"public_addr":"172.21.15.152:6801/1601265891","cluster_addr":"172.21.15.152:6803/1601265891","heartbeat_back_addr":"172.21.15.152:6807/1601265891","heartbeat_front_addr":"172.21.15.152:6805/1601265891","state":["exists","up"]},{"osd":3,"uuid":"79c61910-4b53-424c-b150-6f5385a2cd68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6811","nonce":694846466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6813","nonce":694846466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6817","nonce":694846466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6815","nonce":694846466}]},"public_addr":"172.21.15.86:6811/694846466","cluster_addr":"172.21.15.86:6813/694846466","heartbeat_back_addr":"172.21.15.86:6817/694846466","heartbeat_front_addr":"172.21.15.86:6815/694846466","state":["exists","up"]},{"osd":4,"uuid":"a2878035-0179-469f-bbd4-8ce2d3f6bddc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6809","nonce":1422821471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6811","nonce":1422821471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6815","nonce":1422821471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6813","nonce":1422821471}]},"public_addr":"172.21.15.92:6809/1422821471","cluster_addr":"172.21.15.92:6811/1422821471","heartbeat_back_addr":"172.21.15.92:6815/1422821471","heartbeat_front_addr":"172.21.15.92:6813/1422821471","state":["exists","up"]},{"osd":5,"uuid":"837366aa-ce25-42ca-b157-5bbb50159d0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6809","nonce":1039375239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6811","nonce":1039375239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6815","nonce":1039375239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6813","nonce":1039375239}]},"public_addr":"172.21.15.152:6809/1039375239","cluster_addr":"172.21.15.152:6811/1039375239","heartbeat_back_addr":"172.21.15.152:6815/1039375239","heartbeat_front_addr":"172.21.15.152:6813/1039375239","state":["exists","up"]},{"osd":6,"uuid":"61024437-7b91-432f-a1af-6443d0ef5213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6819","nonce":4080452707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6821","nonce":4080452707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6825","nonce":4080452707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6823","nonce":4080452707}]},"public_addr":"172.21.15.86:6819/4080452707","cluster_addr":"172.21.15.86:6821/4080452707","heartbeat_back_addr":"172.21.15.86:6825/4080452707","heartbeat_front_addr":"172.21.15.86:6823/4080452707","state":["exists","up"]},{"osd":7,"uuid":"2684bdae-6b30-4afc-af68-4ce41b3a75b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6817","nonce":4187707622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6819","nonce":4187707622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6823","nonce":4187707622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6821","nonce":4187707622}]},"public_addr":"172.21.15.92:6817/4187707622","cluster_addr":"172.21.15.92:6819/4187707622","heartbeat_back_addr":"172.21.15.92:6823/4187707622","heartbeat_front_addr":"172.21.15.92:6821/4187707622","state":["exists","up"]},{"osd":8,"uuid":"f03bbf0d-1281-4bcf-9773-9854388e65ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6817","nonce":3237612748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6819","nonce":3237612748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6823","nonce":3237612748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6821","nonce":3237612748}]},"public_addr":"172.21.15.152:6817/3237612748","cluster_addr":"172.21.15.152:6819/3237612748","heartbeat_back_addr":"172.21.15.152:6823/3237612748","heartbeat_front_addr":"172.21.15.152:6821/3237612748","state":["exists","up"]},{"osd":9,"uuid":"e7e8b775-99ff-48a4-a23a-91e7ab6622bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6827","nonce":2835218294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6829","nonce":2835218294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6833","nonce":2835218294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6831","nonce":2835218294}]},"public_addr":"172.21.15.86:6827/2835218294","cluster_addr":"172.21.15.86:6829/2835218294","heartbeat_back_addr":"172.21.15.86:6833/2835218294","heartbeat_front_addr":"172.21.15.86:6831/2835218294","state":["exists","up"]},{"osd":10,"uuid":"d9d616cd-a229-44de-8768-02a4415df115","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6825","nonce":4259465100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6827","nonce":4259465100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6831","nonce":4259465100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6829","nonce":4259465100}]},"public_addr":"172.21.15.92:6825/4259465100","cluster_addr":"172.21.15.92:6827/4259465100","heartbeat_back_addr":"172.21.15.92:6831/4259465100","heartbeat_front_addr":"172.21.15.92:6829/4259465100","state":["exists","up"]},{"osd":11,"uuid":"e7c0b865-ccf5-4233-8d22-0d9b6dbe5c97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6825","nonce":194662760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6827","nonce":194662760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6831","nonce":194662760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6829","nonce":194662760}]},"public_addr":"172.21.15.152:6825/194662760","cluster_addr":"172.21.15.152:6827/194662760","heartbeat_back_addr":"172.21.15.152:6831/194662760","heartbeat_front_addr":"172.21.15.152:6829/194662760","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:44:47.269167+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:45:05.363655+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:45:23.192677+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:45:44.156228+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:02.532971+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:20.153295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:41.713888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:59.676516+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:47:17.207087+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:47:38.970316+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:47:57.282525+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:48:15.437606+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6801/2734784244":"2024-03-24T17:50:38.377487+0000","172.21.15.86:6800/2734784244":"2024-03-24T17:50:38.377487+0000","172.21.15.86:0/734928365":"2024-03-24T17:50:38.377487+0000","172.21.15.86:0/2354115349":"2024-03-24T17:50:38.377487+0000","172.21.15.86:0/3553500912":"2024-03-24T17:43:28.242492+0000","172.21.15.86:6801/3600278228":"2024-03-24T17:43:28.242492+0000","172.21.15.86:0/887262329":"2024-03-24T17:43:28.242492+0000","172.21.15.86:6800/2313723492":"2024-03-24T17:43:05.043910+0000","172.21.15.86:6800/3600278228":"2024-03-24T17:43:28.242492+0000","172.21.15.86:0/3815705642":"2024-03-24T17:43:05.043910+0000","172.21.15.86:0/883052430":"2024-03-24T17:50:38.377487+0000","172.21.15.86:0/1614943062":"2024-03-24T17:43:28.242492+0000","172.21.15.86:6801/2313723492":"2024-03-24T17:43:05.043910+0000","172.21.15.86:0/2416321716":"2024-03-24T17:43:05.043910+0000","172.21.15.86:0/2938296862":"2024-03-24T17:50:38.377487+0000","172.21.15.86:0/433588188":"2024-03-24T17:43:05.043910+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":18,"length":1},{"begin":30,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":18,"length":1},{"begin":30,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T18:08:11.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.208+0000 7fee99e00700 1 -- 172.21.15.86:0/3517863448 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fee780a1290 msgr2=0x7fee780a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:11.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.208+0000 7fee99e00700 1 --2- 172.21.15.86:0/3517863448 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fee780a1290 0x7fee780a3750 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fee94198780 tx=0x7fee80009260 comp rx=0 tx=0).stop 2024-03-23T18:08:11.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.208+0000 7fee99e00700 1 -- 172.21.15.86:0/3517863448 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee94100e60 msgr2=0x7fee94197280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:11.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.208+0000 7fee99e00700 1 --2- 172.21.15.86:0/3517863448 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee94100e60 0x7fee94197280 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fee8400d980 tx=0x7fee8400dc90 comp rx=0 tx=0).stop 2024-03-23T18:08:11.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.208+0000 7fee99e00700 1 -- 172.21.15.86:0/3517863448 shutdown_connections 2024-03-23T18:08:11.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.208+0000 7fee99e00700 1 --2- 172.21.15.86:0/3517863448 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fee780a1290 0x7fee780a3750 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:11.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.208+0000 7fee99e00700 1 --2- 172.21.15.86:0/3517863448 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee9419bb10 0x7fee9419bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:11.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.208+0000 7fee99e00700 1 --2- 172.21.15.86:0/3517863448 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee94100e60 0x7fee94197280 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:11.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.208+0000 7fee99e00700 1 --2- 172.21.15.86:0/3517863448 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee94105f80 0x7fee941977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:11.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.208+0000 7fee99e00700 1 -- 172.21.15.86:0/3517863448 >> 172.21.15.86:0/3517863448 conn(0x7fee940fc700 msgr2=0x7fee940fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:08:11.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.208+0000 7fee99e00700 1 -- 172.21.15.86:0/3517863448 shutdown_connections 2024-03-23T18:08:11.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.208+0000 7fee99e00700 1 -- 172.21.15.86:0/3517863448 wait complete. 2024-03-23T18:08:11.228 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T17:45:26.848397+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T17:48:54.331894+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T17:48:55.640693+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 30, 'snap_epoch': 109, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T18:08:11.228 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T18:08:11.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.434+0000 7f1ab3400700 1 -- 172.21.15.86:0/1763002684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1aac105ce0 msgr2=0x7f1aac1060c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:11.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.434+0000 7f1ab3400700 1 --2- 172.21.15.86:0/1763002684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1aac105ce0 0x7f1aac1060c0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f1aa0009990 tx=0x7f1aa0009ca0 comp rx=0 tx=0).stop 2024-03-23T18:08:11.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.434+0000 7f1ab3400700 1 -- 172.21.15.86:0/1763002684 shutdown_connections 2024-03-23T18:08:11.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.434+0000 7f1ab3400700 1 --2- 172.21.15.86:0/1763002684 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1aac0687f0 0x7f1aac10db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:11.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.434+0000 7f1ab3400700 1 --2- 172.21.15.86:0/1763002684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1aac105ce0 0x7f1aac1060c0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:11.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.434+0000 7f1ab3400700 1 --2- 172.21.15.86:0/1763002684 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1aac067e30 0x7f1aac0682b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:11.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.434+0000 7f1ab3400700 1 -- 172.21.15.86:0/1763002684 >> 172.21.15.86:0/1763002684 conn(0x7f1aac076d40 msgr2=0x7f1aac077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:08:11.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.434+0000 7f1ab3400700 1 -- 172.21.15.86:0/1763002684 shutdown_connections 2024-03-23T18:08:11.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.434+0000 7f1ab3400700 1 -- 172.21.15.86:0/1763002684 wait complete. 2024-03-23T18:08:11.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.435+0000 7f1ab3400700 1 Processor -- start 2024-03-23T18:08:11.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.435+0000 7f1ab3400700 1 -- start start 2024-03-23T18:08:11.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.436+0000 7f1ab3400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1aac067e30 0x7f1aac199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:11.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.436+0000 7f1ab3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1aac0687f0 0x7f1aac1999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:11.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.436+0000 7f1ab3400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1aac105ce0 0x7f1aac19dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:11.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.436+0000 7f1ab3400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aac19e380 con 0x7f1aac0687f0 2024-03-23T18:08:11.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.436+0000 7f1ab3400700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aac19e4f0 con 0x7f1aac067e30 2024-03-23T18:08:11.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.436+0000 7f1ab3400700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aac19e6b0 con 0x7f1aac105ce0 2024-03-23T18:08:11.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.436+0000 7f1aabe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1aac0687f0 0x7f1aac1999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:11.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.436+0000 7f1ab0a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1aac067e30 0x7f1aac199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:11.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.436+0000 7f1ab0a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1aac067e30 0x7f1aac199460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:38304/0 (socket says 172.21.15.86:38304) 2024-03-23T18:08:11.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.436+0000 7f1ab0a00700 1 -- 172.21.15.86:0/2844506690 learned_addr learned my addr 172.21.15.86:0/2844506690 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:08:11.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.436+0000 7f1ab1400700 1 --2- 172.21.15.86:0/2844506690 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1aac105ce0 0x7f1aac19dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:11.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.436+0000 7f1ab0a00700 1 -- 172.21.15.86:0/2844506690 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1aac105ce0 msgr2=0x7f1aac19dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:11.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.436+0000 7f1ab0a00700 1 --2- 172.21.15.86:0/2844506690 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1aac105ce0 0x7f1aac19dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:11.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.436+0000 7f1ab0a00700 1 -- 172.21.15.86:0/2844506690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1aac0687f0 msgr2=0x7f1aac1999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:11.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.436+0000 7f1ab0a00700 1 --2- 172.21.15.86:0/2844506690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1aac0687f0 0x7f1aac1999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:11.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.436+0000 7f1ab0a00700 1 -- 172.21.15.86:0/2844506690 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a98009520 con 0x7f1aac067e30 2024-03-23T18:08:11.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.437+0000 7f1ab0a00700 1 --2- 172.21.15.86:0/2844506690 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1aac067e30 0x7f1aac199460 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f1aa0005910 tx=0x7f1aa0004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:08:11.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.437+0000 7f1aa9600700 1 -- 172.21.15.86:0/2844506690 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa001d070 con 0x7f1aac067e30 2024-03-23T18:08:11.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.437+0000 7f1aa9600700 1 -- 172.21.15.86:0/2844506690 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1aa0022470 con 0x7f1aac067e30 2024-03-23T18:08:11.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.437+0000 7f1ab3400700 1 -- 172.21.15.86:0/2844506690 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1aa00095f0 con 0x7f1aac067e30 2024-03-23T18:08:11.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.437+0000 7f1aa9600700 1 -- 172.21.15.86:0/2844506690 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa0017650 con 0x7f1aac067e30 2024-03-23T18:08:11.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.437+0000 7f1ab3400700 1 -- 172.21.15.86:0/2844506690 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1aac19ebc0 con 0x7f1aac067e30 2024-03-23T18:08:11.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.438+0000 7f1ab3400700 1 -- 172.21.15.86:0/2844506690 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1aac102430 con 0x7f1aac067e30 2024-03-23T18:08:11.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.440+0000 7f1aa9600700 1 -- 172.21.15.86:0/2844506690 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f1aa00225e0 con 0x7f1aac067e30 2024-03-23T18:08:11.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.441+0000 7f1aa9600700 1 --2- 172.21.15.86:0/2844506690 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f1a940a1290 0x7f1a940a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:11.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.441+0000 7f1aa9600700 1 -- 172.21.15.86:0/2844506690 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f1aa00cdf40 con 0x7f1aac067e30 2024-03-23T18:08:11.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.441+0000 7f1aabe00700 1 --2- 172.21.15.86:0/2844506690 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f1a940a1290 0x7f1a940a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:11.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.442+0000 7f1aabe00700 1 --2- 172.21.15.86:0/2844506690 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f1a940a1290 0x7f1a940a3750 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1aac19a9b0 tx=0x7f1a98009260 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:08:11.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.443+0000 7f1aa9600700 1 -- 172.21.15.86:0/2844506690 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f1aa00973e0 con 0x7f1aac067e30 2024-03-23T18:08:11.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.652+0000 7f1ab3400700 1 -- 172.21.15.86:0/2844506690 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1aac110d60 con 0x7f1aac067e30 2024-03-23T18:08:11.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.653+0000 7f1aa9600700 1 -- 172.21.15.86:0/2844506690 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v109) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1aa009aa00 con 0x7f1aac067e30 2024-03-23T18:08:11.653 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-03-23T18:08:11.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.655+0000 7f1ab3400700 1 -- 172.21.15.86:0/2844506690 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f1a940a1290 msgr2=0x7f1a940a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:11.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.655+0000 7f1ab3400700 1 --2- 172.21.15.86:0/2844506690 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f1a940a1290 0x7f1a940a3750 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1aac19a9b0 tx=0x7f1a98009260 comp rx=0 tx=0).stop 2024-03-23T18:08:11.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.655+0000 7f1ab3400700 1 -- 172.21.15.86:0/2844506690 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1aac067e30 msgr2=0x7f1aac199460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:11.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.655+0000 7f1ab3400700 1 --2- 172.21.15.86:0/2844506690 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1aac067e30 0x7f1aac199460 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f1aa0005910 tx=0x7f1aa0004730 comp rx=0 tx=0).stop 2024-03-23T18:08:11.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.655+0000 7f1ab3400700 1 -- 172.21.15.86:0/2844506690 shutdown_connections 2024-03-23T18:08:11.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.655+0000 7f1ab3400700 1 --2- 172.21.15.86:0/2844506690 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f1a940a1290 0x7f1a940a3750 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:11.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.655+0000 7f1ab3400700 1 --2- 172.21.15.86:0/2844506690 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1aac105ce0 0x7f1aac19dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:11.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.655+0000 7f1ab3400700 1 --2- 172.21.15.86:0/2844506690 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1aac067e30 0x7f1aac199460 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:11.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.655+0000 7f1ab3400700 1 --2- 172.21.15.86:0/2844506690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1aac0687f0 0x7f1aac1999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:11.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.655+0000 7f1ab3400700 1 -- 172.21.15.86:0/2844506690 >> 172.21.15.86:0/2844506690 conn(0x7f1aac076d40 msgr2=0x7f1aac101030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:08:11.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.655+0000 7f1ab3400700 1 -- 172.21.15.86:0/2844506690 shutdown_connections 2024-03-23T18:08:11.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.655+0000 7f1ab3400700 1 -- 172.21.15.86:0/2844506690 wait complete. 2024-03-23T18:08:11.674 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T18:08:11.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.871+0000 7f5fc7e00700 1 -- 172.21.15.86:0/1370343156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5fc00687f0 msgr2=0x7f5fc010db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:11.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.871+0000 7f5fc7e00700 1 --2- 172.21.15.86:0/1370343156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5fc00687f0 0x7f5fc010db50 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f5fb800b0a0 tx=0x7f5fb800b3b0 comp rx=0 tx=0).stop 2024-03-23T18:08:11.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.871+0000 7f5fc7e00700 1 -- 172.21.15.86:0/1370343156 shutdown_connections 2024-03-23T18:08:11.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.871+0000 7f5fc7e00700 1 --2- 172.21.15.86:0/1370343156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5fc00687f0 0x7f5fc010db50 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:11.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.871+0000 7f5fc7e00700 1 --2- 172.21.15.86:0/1370343156 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5fc0105ce0 0x7f5fc01060c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:11.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.871+0000 7f5fc7e00700 1 --2- 172.21.15.86:0/1370343156 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5fc0067e30 0x7f5fc00682b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:11.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.871+0000 7f5fc7e00700 1 -- 172.21.15.86:0/1370343156 >> 172.21.15.86:0/1370343156 conn(0x7f5fc0076d40 msgr2=0x7f5fc0077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:08:11.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.871+0000 7f5fc7e00700 1 -- 172.21.15.86:0/1370343156 shutdown_connections 2024-03-23T18:08:11.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.871+0000 7f5fc7e00700 1 -- 172.21.15.86:0/1370343156 wait complete. 2024-03-23T18:08:11.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.873+0000 7f5fc7e00700 1 Processor -- start 2024-03-23T18:08:11.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.873+0000 7f5fc7e00700 1 -- start start 2024-03-23T18:08:11.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.873+0000 7f5fc7e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5fc0067e30 0x7f5fc00ffc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:11.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.873+0000 7f5fc7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5fc00687f0 0x7f5fc01001b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:11.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.873+0000 7f5fc7e00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5fc0105ce0 0x7f5fc01006f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:11.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.873+0000 7f5fc7e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc01a56d0 con 0x7f5fc00687f0 2024-03-23T18:08:11.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.873+0000 7f5fc7e00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc01a5840 con 0x7f5fc0067e30 2024-03-23T18:08:11.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.873+0000 7f5fc7e00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc01a5a00 con 0x7f5fc0105ce0 2024-03-23T18:08:11.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.874+0000 7f5fc4a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5fc00687f0 0x7f5fc01001b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:11.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.874+0000 7f5fc4a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5fc00687f0 0x7f5fc01001b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58682/0 (socket says 172.21.15.86:58682) 2024-03-23T18:08:11.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.874+0000 7f5fc5400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5fc0067e30 0x7f5fc00ffc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:11.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.874+0000 7f5fc4a00700 1 -- 172.21.15.86:0/2853284591 learned_addr learned my addr 172.21.15.86:0/2853284591 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:08:11.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.874+0000 7f5fc5e00700 1 --2- 172.21.15.86:0/2853284591 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5fc0105ce0 0x7f5fc01006f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:11.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.874+0000 7f5fc4a00700 1 -- 172.21.15.86:0/2853284591 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5fc0105ce0 msgr2=0x7f5fc01006f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:11.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.874+0000 7f5fc4a00700 1 --2- 172.21.15.86:0/2853284591 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5fc0105ce0 0x7f5fc01006f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:11.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.874+0000 7f5fc4a00700 1 -- 172.21.15.86:0/2853284591 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5fc0067e30 msgr2=0x7f5fc00ffc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:11.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.874+0000 7f5fc4a00700 1 --2- 172.21.15.86:0/2853284591 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5fc0067e30 0x7f5fc00ffc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:11.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.874+0000 7f5fc4a00700 1 -- 172.21.15.86:0/2853284591 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fb8009d00 con 0x7f5fc00687f0 2024-03-23T18:08:11.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.874+0000 7f5fc5400700 1 --2- 172.21.15.86:0/2853284591 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5fc0067e30 0x7f5fc00ffc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T18:08:11.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.874+0000 7f5fc4a00700 1 --2- 172.21.15.86:0/2853284591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5fc00687f0 0x7f5fc01001b0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f5fac00e9a0 tx=0x7f5fac00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:08:11.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.875+0000 7f5fbe000700 1 -- 172.21.15.86:0/2853284591 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fac009790 con 0x7f5fc00687f0 2024-03-23T18:08:11.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.875+0000 7f5fbe000700 1 -- 172.21.15.86:0/2853284591 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5fac010430 con 0x7f5fc00687f0 2024-03-23T18:08:11.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.875+0000 7f5fc7e00700 1 -- 172.21.15.86:0/2853284591 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fc01a5ce0 con 0x7f5fc00687f0 2024-03-23T18:08:11.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.875+0000 7f5fbe000700 1 -- 172.21.15.86:0/2853284591 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fac018680 con 0x7f5fc00687f0 2024-03-23T18:08:11.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.875+0000 7f5fc7e00700 1 -- 172.21.15.86:0/2853284591 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fc01a61a0 con 0x7f5fc00687f0 2024-03-23T18:08:11.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.876+0000 7f5fc7e00700 1 -- 172.21.15.86:0/2853284591 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fc0109280 con 0x7f5fc00687f0 2024-03-23T18:08:11.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.877+0000 7f5fbe000700 1 -- 172.21.15.86:0/2853284591 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f5fac0105a0 con 0x7f5fc00687f0 2024-03-23T18:08:11.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.879+0000 7f5fbe000700 1 --2- 172.21.15.86:0/2853284591 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f5fa80a1290 0x7f5fa80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:11.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.879+0000 7f5fbe000700 1 -- 172.21.15.86:0/2853284591 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f5fac014070 con 0x7f5fc00687f0 2024-03-23T18:08:11.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.879+0000 7f5fc5400700 1 --2- 172.21.15.86:0/2853284591 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f5fa80a1290 0x7f5fa80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:11.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.880+0000 7f5fc5400700 1 --2- 172.21.15.86:0/2853284591 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f5fa80a1290 0x7f5fa80a3750 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4007fd0 tx=0x7f5fb40073d0 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:08:11.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:11.883+0000 7f5fbe000700 1 -- 172.21.15.86:0/2853284591 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f5fac0529f0 con 0x7f5fc00687f0 2024-03-23T18:08:12.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.090+0000 7f5fc7e00700 1 -- 172.21.15.86:0/2853284591 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5fc0101100 con 0x7f5fc00687f0 2024-03-23T18:08:12.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.091+0000 7f5fbe000700 1 -- 172.21.15.86:0/2853284591 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v109) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5fac096300 con 0x7f5fc00687f0 2024-03-23T18:08:12.091 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-03-23T18:08:12.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.093+0000 7f5fc7e00700 1 -- 172.21.15.86:0/2853284591 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f5fa80a1290 msgr2=0x7f5fa80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:12.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.093+0000 7f5fc7e00700 1 --2- 172.21.15.86:0/2853284591 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f5fa80a1290 0x7f5fa80a3750 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4007fd0 tx=0x7f5fb40073d0 comp rx=0 tx=0).stop 2024-03-23T18:08:12.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.093+0000 7f5fc7e00700 1 -- 172.21.15.86:0/2853284591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5fc00687f0 msgr2=0x7f5fc01001b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:12.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.093+0000 7f5fc7e00700 1 --2- 172.21.15.86:0/2853284591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5fc00687f0 0x7f5fc01001b0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f5fac00e9a0 tx=0x7f5fac00ecb0 comp rx=0 tx=0).stop 2024-03-23T18:08:12.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.093+0000 7f5fc7e00700 1 -- 172.21.15.86:0/2853284591 shutdown_connections 2024-03-23T18:08:12.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.093+0000 7f5fc7e00700 1 --2- 172.21.15.86:0/2853284591 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f5fa80a1290 0x7f5fa80a3750 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:12.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.093+0000 7f5fc7e00700 1 --2- 172.21.15.86:0/2853284591 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5fc0105ce0 0x7f5fc01006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:12.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.093+0000 7f5fc7e00700 1 --2- 172.21.15.86:0/2853284591 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5fc0067e30 0x7f5fc00ffc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:12.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.093+0000 7f5fc7e00700 1 --2- 172.21.15.86:0/2853284591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5fc00687f0 0x7f5fc01001b0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:12.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.093+0000 7f5fc7e00700 1 -- 172.21.15.86:0/2853284591 >> 172.21.15.86:0/2853284591 conn(0x7f5fc0076d40 msgr2=0x7f5fc00fd860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:08:12.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.094+0000 7f5fc7e00700 1 -- 172.21.15.86:0/2853284591 shutdown_connections 2024-03-23T18:08:12.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.094+0000 7f5fc7e00700 1 -- 172.21.15.86:0/2853284591 wait complete. 2024-03-23T18:08:12.112 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T18:08:12.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.307+0000 7fb297600700 1 -- 172.21.15.86:0/2987376774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb29010d520 msgr2=0x7fb29010f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:12.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.307+0000 7fb297600700 1 --2- 172.21.15.86:0/2987376774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb29010d520 0x7fb29010f990 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fb28800b0a0 tx=0x7fb28800b3b0 comp rx=0 tx=0).stop 2024-03-23T18:08:12.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.307+0000 7fb297600700 1 -- 172.21.15.86:0/2987376774 shutdown_connections 2024-03-23T18:08:12.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.307+0000 7fb297600700 1 --2- 172.21.15.86:0/2987376774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb29010d520 0x7fb29010f990 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:12.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.307+0000 7fb297600700 1 --2- 172.21.15.86:0/2987376774 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2900731e0 0x7fb2900735c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:12.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.307+0000 7fb297600700 1 --2- 172.21.15.86:0/2987376774 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb290073b00 0x7fb29010cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:12.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.307+0000 7fb297600700 1 -- 172.21.15.86:0/2987376774 >> 172.21.15.86:0/2987376774 conn(0x7fb2900fc7f0 msgr2=0x7fb2900fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:08:12.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.307+0000 7fb297600700 1 -- 172.21.15.86:0/2987376774 shutdown_connections 2024-03-23T18:08:12.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.307+0000 7fb297600700 1 -- 172.21.15.86:0/2987376774 wait complete. 2024-03-23T18:08:12.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.308+0000 7fb297600700 1 Processor -- start 2024-03-23T18:08:12.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.308+0000 7fb297600700 1 -- start start 2024-03-23T18:08:12.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.308+0000 7fb297600700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2900731e0 0x7fb290199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:12.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.309+0000 7fb297600700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb290073b00 0x7fb290199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:12.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.309+0000 7fb297600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb29010d520 0x7fb29019ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:12.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.309+0000 7fb297600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb29019e430 con 0x7fb29010d520 2024-03-23T18:08:12.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.309+0000 7fb297600700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb29019e5a0 con 0x7fb2900731e0 2024-03-23T18:08:12.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.309+0000 7fb297600700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb29019e760 con 0x7fb290073b00 2024-03-23T18:08:12.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.309+0000 7fb295600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb29010d520 0x7fb29019ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:12.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.309+0000 7fb294c00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2900731e0 0x7fb290199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:12.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.309+0000 7fb28fe00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb290073b00 0x7fb290199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:12.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.309+0000 7fb28fe00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb290073b00 0x7fb290199a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:51560/0 (socket says 172.21.15.86:51560) 2024-03-23T18:08:12.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.309+0000 7fb28fe00700 1 -- 172.21.15.86:0/1747638679 learned_addr learned my addr 172.21.15.86:0/1747638679 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:08:12.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.309+0000 7fb295600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb29010d520 0x7fb29019ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58696/0 (socket says 172.21.15.86:58696) 2024-03-23T18:08:12.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.309+0000 7fb295600700 1 -- 172.21.15.86:0/1747638679 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb290073b00 msgr2=0x7fb290199a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:12.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.309+0000 7fb295600700 1 --2- 172.21.15.86:0/1747638679 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb290073b00 0x7fb290199a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:12.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.309+0000 7fb295600700 1 -- 172.21.15.86:0/1747638679 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2900731e0 msgr2=0x7fb290199510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:12.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.309+0000 7fb295600700 1 --2- 172.21.15.86:0/1747638679 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2900731e0 0x7fb290199510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:12.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.309+0000 7fb295600700 1 -- 172.21.15.86:0/1747638679 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb288009d00 con 0x7fb29010d520 2024-03-23T18:08:12.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.310+0000 7fb295600700 1 --2- 172.21.15.86:0/1747638679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb29010d520 0x7fb29019ddf0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fb28800b070 tx=0x7fb288007930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:08:12.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.310+0000 7fb28d600700 1 -- 172.21.15.86:0/1747638679 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb28800e040 con 0x7fb29010d520 2024-03-23T18:08:12.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.310+0000 7fb28d600700 1 -- 172.21.15.86:0/1747638679 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb28801d470 con 0x7fb29010d520 2024-03-23T18:08:12.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.310+0000 7fb297600700 1 -- 172.21.15.86:0/1747638679 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb29019e9e0 con 0x7fb29010d520 2024-03-23T18:08:12.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.310+0000 7fb28d600700 1 -- 172.21.15.86:0/1747638679 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2880052a0 con 0x7fb29010d520 2024-03-23T18:08:12.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.310+0000 7fb297600700 1 -- 172.21.15.86:0/1747638679 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2901a6280 con 0x7fb29010d520 2024-03-23T18:08:12.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.311+0000 7fb297600700 1 -- 172.21.15.86:0/1747638679 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb290108730 con 0x7fb29010d520 2024-03-23T18:08:12.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.313+0000 7fb28d600700 1 -- 172.21.15.86:0/1747638679 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fb28801d5e0 con 0x7fb29010d520 2024-03-23T18:08:12.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.314+0000 7fb28d600700 1 --2- 172.21.15.86:0/1747638679 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fb2780a1310 0x7fb2780a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:12.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.314+0000 7fb28d600700 1 -- 172.21.15.86:0/1747638679 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 8528+0+0 (secure 0 0 0) 0x7fb2880cdfe0 con 0x7fb29010d520 2024-03-23T18:08:12.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.314+0000 7fb294c00700 1 --2- 172.21.15.86:0/1747638679 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fb2780a1310 0x7fb2780a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:12.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.315+0000 7fb294c00700 1 --2- 172.21.15.86:0/1747638679 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fb2780a1310 0x7fb2780a37d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb284007be0 tx=0x7fb2840073d0 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:08:12.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.318+0000 7fb28d600700 1 -- 172.21.15.86:0/1747638679 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fb2880974c0 con 0x7fb29010d520 2024-03-23T18:08:12.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:12 smithi152 ceph-mon[47949]: from='client.27383 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T18:08:12.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:12 smithi152 ceph-mon[47949]: pgmap v555: 97 pgs: 11 active+clean+snaptrim_wait, 82 active+clean, 4 active+clean+snaptrim; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 4.8 MiB/s wr, 591 op/s 2024-03-23T18:08:12.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:12 smithi152 ceph-mon[47949]: from='client.27367 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T18:08:12.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:12 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/3517863448' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T18:08:12.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:12 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T18:08:12.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:12 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T18:08:12.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:12 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T18:08:12.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:12 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/2844506690' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T18:08:12.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:12 smithi086 ceph-mon[38221]: from='client.27383 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T18:08:12.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:12 smithi086 ceph-mon[38221]: pgmap v555: 97 pgs: 11 active+clean+snaptrim_wait, 82 active+clean, 4 active+clean+snaptrim; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 4.8 MiB/s wr, 591 op/s 2024-03-23T18:08:12.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:12 smithi086 ceph-mon[38221]: from='client.27367 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T18:08:12.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:12 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/3517863448' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T18:08:12.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:12 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T18:08:12.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:12 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T18:08:12.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:12 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T18:08:12.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:12 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/2844506690' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T18:08:12.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.525+0000 7fb297600700 1 -- 172.21.15.86:0/1747638679 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb29019a810 con 0x7fb29010d520 2024-03-23T18:08:12.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.525+0000 7fb28d600700 1 -- 172.21.15.86:0/1747638679 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v109) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb28809aae0 con 0x7fb29010d520 2024-03-23T18:08:12.525 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-03-23T18:08:12.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.528+0000 7fb297600700 1 -- 172.21.15.86:0/1747638679 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fb2780a1310 msgr2=0x7fb2780a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:12.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.528+0000 7fb297600700 1 --2- 172.21.15.86:0/1747638679 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fb2780a1310 0x7fb2780a37d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb284007be0 tx=0x7fb2840073d0 comp rx=0 tx=0).stop 2024-03-23T18:08:12.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.528+0000 7fb297600700 1 -- 172.21.15.86:0/1747638679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb29010d520 msgr2=0x7fb29019ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:12.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.528+0000 7fb297600700 1 --2- 172.21.15.86:0/1747638679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb29010d520 0x7fb29019ddf0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fb28800b070 tx=0x7fb288007930 comp rx=0 tx=0).stop 2024-03-23T18:08:12.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.528+0000 7fb297600700 1 -- 172.21.15.86:0/1747638679 shutdown_connections 2024-03-23T18:08:12.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.528+0000 7fb297600700 1 --2- 172.21.15.86:0/1747638679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb29010d520 0x7fb29019ddf0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:12.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.528+0000 7fb297600700 1 --2- 172.21.15.86:0/1747638679 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2900731e0 0x7fb290199510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:12.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.528+0000 7fb297600700 1 --2- 172.21.15.86:0/1747638679 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fb2780a1310 0x7fb2780a37d0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:12.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.528+0000 7fb297600700 1 --2- 172.21.15.86:0/1747638679 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb290073b00 0x7fb290199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:12.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.528+0000 7fb297600700 1 -- 172.21.15.86:0/1747638679 >> 172.21.15.86:0/1747638679 conn(0x7fb2900fc7f0 msgr2=0x7fb2900fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:08:12.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.528+0000 7fb297600700 1 -- 172.21.15.86:0/1747638679 shutdown_connections 2024-03-23T18:08:12.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.528+0000 7fb297600700 1 -- 172.21.15.86:0/1747638679 wait complete. 2024-03-23T18:08:12.547 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-03-23T18:08:12.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:12 smithi092 ceph-mon[79089]: from='client.27383 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T18:08:12.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:12 smithi092 ceph-mon[79089]: pgmap v555: 97 pgs: 11 active+clean+snaptrim_wait, 82 active+clean, 4 active+clean+snaptrim; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 4.8 MiB/s wr, 591 op/s 2024-03-23T18:08:12.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:12 smithi092 ceph-mon[79089]: from='client.27367 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T18:08:12.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:12 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/3517863448' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T18:08:12.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:12 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T18:08:12.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:12 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T18:08:12.558 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:12 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' 2024-03-23T18:08:12.558 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:12 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/2844506690' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T18:08:12.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.740+0000 7f071e200700 1 -- 172.21.15.86:0/83607520 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0718100e60 msgr2=0x7f0718101240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:12.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.740+0000 7f071e200700 1 --2- 172.21.15.86:0/83607520 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0718100e60 0x7f0718101240 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f0708009990 tx=0x7f0708009ca0 comp rx=0 tx=0).stop 2024-03-23T18:08:12.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.740+0000 7f071e200700 1 -- 172.21.15.86:0/83607520 shutdown_connections 2024-03-23T18:08:12.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.740+0000 7f071e200700 1 --2- 172.21.15.86:0/83607520 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0718105f80 0x7f071810db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:12.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.740+0000 7f071e200700 1 --2- 172.21.15.86:0/83607520 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0718100e60 0x7f0718101240 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:12.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.740+0000 7f071e200700 1 --2- 172.21.15.86:0/83607520 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0718101810 0x7f0718105850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:12.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.740+0000 7f071e200700 1 -- 172.21.15.86:0/83607520 >> 172.21.15.86:0/83607520 conn(0x7f07180fc700 msgr2=0x7f07180feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:08:12.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.740+0000 7f071e200700 1 -- 172.21.15.86:0/83607520 shutdown_connections 2024-03-23T18:08:12.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.740+0000 7f071e200700 1 -- 172.21.15.86:0/83607520 wait complete. 2024-03-23T18:08:12.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.742+0000 7f071e200700 1 Processor -- start 2024-03-23T18:08:12.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.742+0000 7f071e200700 1 -- start start 2024-03-23T18:08:12.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.742+0000 7f071e200700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0718100e60 0x7f0718199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:12.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.743+0000 7f071e200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0718105f80 0x7f0718199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:12.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.743+0000 7f071e200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f071819dbe0 0x7f071819e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:12.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.743+0000 7f071e200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f071819e660 con 0x7f0718105f80 2024-03-23T18:08:12.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.743+0000 7f071e200700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f071819e7d0 con 0x7f071819dbe0 2024-03-23T18:08:12.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.743+0000 7f071e200700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f071819e990 con 0x7f0718100e60 2024-03-23T18:08:12.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.743+0000 7f0716a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0718105f80 0x7f0718199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:12.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.743+0000 7f0716a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0718105f80 0x7f0718199840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58726/0 (socket says 172.21.15.86:58726) 2024-03-23T18:08:12.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.743+0000 7f0716a00700 1 -- 172.21.15.86:0/2715833445 learned_addr learned my addr 172.21.15.86:0/2715833445 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:08:12.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.743+0000 7f0717400700 1 --2- 172.21.15.86:0/2715833445 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0718100e60 0x7f0718199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:12.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.743+0000 7f0717e00700 1 --2- 172.21.15.86:0/2715833445 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f071819dbe0 0x7f071819e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:12.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.743+0000 7f0716a00700 1 -- 172.21.15.86:0/2715833445 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0718100e60 msgr2=0x7f0718199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:12.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.743+0000 7f0716a00700 1 --2- 172.21.15.86:0/2715833445 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0718100e60 0x7f0718199300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:12.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.743+0000 7f0716a00700 1 -- 172.21.15.86:0/2715833445 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f071819dbe0 msgr2=0x7f071819e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:12.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.743+0000 7f0716a00700 1 --2- 172.21.15.86:0/2715833445 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f071819dbe0 0x7f071819e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:12.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.743+0000 7f0716a00700 1 -- 172.21.15.86:0/2715833445 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07080095f0 con 0x7f0718105f80 2024-03-23T18:08:12.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.743+0000 7f0716a00700 1 --2- 172.21.15.86:0/2715833445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0718105f80 0x7f0718199840 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f070400e970 tx=0x7f070400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:08:12.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.744+0000 7f070fe00700 1 -- 172.21.15.86:0/2715833445 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0704009790 con 0x7f0718105f80 2024-03-23T18:08:12.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.744+0000 7f070fe00700 1 -- 172.21.15.86:0/2715833445 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f070400c7e0 con 0x7f0718105f80 2024-03-23T18:08:12.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.744+0000 7f071e200700 1 -- 172.21.15.86:0/2715833445 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07181a5ef0 con 0x7f0718105f80 2024-03-23T18:08:12.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.744+0000 7f070fe00700 1 -- 172.21.15.86:0/2715833445 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07040054e0 con 0x7f0718105f80 2024-03-23T18:08:12.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.744+0000 7f071e200700 1 -- 172.21.15.86:0/2715833445 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07181a6370 con 0x7f0718105f80 2024-03-23T18:08:12.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.745+0000 7f071e200700 1 -- 172.21.15.86:0/2715833445 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0718102330 con 0x7f0718105f80 2024-03-23T18:08:12.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.747+0000 7f070fe00700 1 -- 172.21.15.86:0/2715833445 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f070400c950 con 0x7f0718105f80 2024-03-23T18:08:12.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.748+0000 7f070fe00700 1 --2- 172.21.15.86:0/2715833445 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f06fc0a1290 0x7f06fc0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:12.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.748+0000 7f070fe00700 1 -- 172.21.15.86:0/2715833445 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f0704014070 con 0x7f0718105f80 2024-03-23T18:08:12.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.748+0000 7f0717400700 1 --2- 172.21.15.86:0/2715833445 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f06fc0a1290 0x7f06fc0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:12.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.749+0000 7f0717400700 1 --2- 172.21.15.86:0/2715833445 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f06fc0a1290 0x7f06fc0a3750 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f070800b2e0 tx=0x7f0708005960 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:08:12.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:12.752+0000 7f070fe00700 1 -- 172.21.15.86:0/2715833445 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f0704096360 con 0x7f0718105f80 2024-03-23T18:08:13.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:13.038+0000 7f071e200700 1 -- 172.21.15.86:0/2715833445 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7f0718067a10 con 0x7f0718105f80 2024-03-23T18:08:13.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:13.049+0000 7f070fe00700 1 -- 172.21.15.86:0/2715833445 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) v1 ==== 81+0+733924 (secure 0 0 0) 0x7f0704099980 con 0x7f0718105f80 2024-03-23T18:08:13.061 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T18:08:13.062 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-23T18:08:13.062 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 26, 2024-03-23T18:08:13.062 INFO:teuthology.orchestra.run.smithi086.stdout: "active_gid": 15375, 2024-03-23T18:08:13.062 INFO:teuthology.orchestra.run.smithi086.stdout: "active_name": "x", 2024-03-23T18:08:13.062 INFO:teuthology.orchestra.run.smithi086.stdout: "active_addrs": { 2024-03-23T18:08:13.062 INFO:teuthology.orchestra.run.smithi086.stdout: "addrvec": [ 2024-03-23T18:08:13.062 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:13.062 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v2", 2024-03-23T18:08:13.063 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "172.21.15.86:6800", 2024-03-23T18:08:13.063 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 4022683855 2024-03-23T18:08:13.063 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.063 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:13.063 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2024-03-23T18:08:13.063 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "172.21.15.86:6801", 2024-03-23T18:08:13.063 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 4022683855 2024-03-23T18:08:13.063 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:13.063 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-03-23T18:08:13.064 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.064 INFO:teuthology.orchestra.run.smithi086.stdout: "active_addr": "172.21.15.86:6801/4022683855", 2024-03-23T18:08:13.064 INFO:teuthology.orchestra.run.smithi086.stdout: "active_change": "2024-03-23T17:50:38.377703+0000", 2024-03-23T18:08:13.064 INFO:teuthology.orchestra.run.smithi086.stdout: "active_mgr_features": 4540138322906710015, 2024-03-23T18:08:13.064 INFO:teuthology.orchestra.run.smithi086.stdout: "available": true, 2024-03-23T18:08:13.064 INFO:teuthology.orchestra.run.smithi086.stdout: "standbys": [ 2024-03-23T18:08:13.064 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:13.064 INFO:teuthology.orchestra.run.smithi086.stdout: "gid": 25409, 2024-03-23T18:08:13.065 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "y", 2024-03-23T18:08:13.065 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr_features": 4540138322906710015, 2024-03-23T18:08:13.065 INFO:teuthology.orchestra.run.smithi086.stdout: "available_modules": [ 2024-03-23T18:08:13.065 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:13.065 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "alerts", 2024-03-23T18:08:13.065 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:13.065 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:13.065 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:13.065 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-03-23T18:08:13.066 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-03-23T18:08:13.066 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-23T18:08:13.066 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.066 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.066 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-03-23T18:08:13.066 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.066 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.066 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.067 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How frequently to reexamine health status", 2024-03-23T18:08:13.067 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.067 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.067 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.067 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.067 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:13.067 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:13.067 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.068 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.068 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.068 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.068 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.068 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.068 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.068 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:13.068 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:13.068 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:13.069 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:13.069 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:13.069 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:13.069 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.069 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.069 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.069 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.069 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.070 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.070 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:13.070 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:13.070 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.070 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.070 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.070 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.070 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.071 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.071 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.071 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.071 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.071 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.071 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.071 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.071 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:13.071 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:13.072 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.072 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.072 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.072 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:13.072 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.072 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.072 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.072 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:13.073 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:13.073 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:13.073 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:13.073 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:13.073 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:13.073 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.073 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.073 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.074 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.074 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.074 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.074 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:13.074 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:13.074 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.074 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.074 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.075 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.075 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.075 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.075 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.075 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.075 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.075 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.075 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.076 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.076 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_destination": { 2024-03-23T18:08:13.076 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_destination", 2024-03-23T18:08:13.076 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.076 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.076 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.076 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.076 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.076 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.077 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.077 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Email address to send alerts to", 2024-03-23T18:08:13.077 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.077 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.077 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.077 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.077 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_from_name": { 2024-03-23T18:08:13.077 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_from_name", 2024-03-23T18:08:13.077 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.078 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.078 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.078 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "Ceph", 2024-03-23T18:08:13.078 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.078 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.078 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.078 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Email From: name", 2024-03-23T18:08:13.078 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.079 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.079 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.079 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.079 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_host": { 2024-03-23T18:08:13.079 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_host", 2024-03-23T18:08:13.079 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.079 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.079 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.080 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.080 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.080 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.080 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.080 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SMTP server", 2024-03-23T18:08:13.080 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.080 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.080 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.081 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.081 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_password": { 2024-03-23T18:08:13.081 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_password", 2024-03-23T18:08:13.081 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.081 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.081 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.081 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.081 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.082 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.082 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.082 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Password to authenticate with", 2024-03-23T18:08:13.082 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.082 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.082 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.082 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.082 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_port": { 2024-03-23T18:08:13.083 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_port", 2024-03-23T18:08:13.083 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:13.083 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.083 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.083 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "465", 2024-03-23T18:08:13.083 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.083 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.083 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.083 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SMTP port", 2024-03-23T18:08:13.084 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.084 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.084 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.084 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.084 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_sender": { 2024-03-23T18:08:13.084 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_sender", 2024-03-23T18:08:13.084 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.084 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.085 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.085 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.085 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.085 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.085 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.085 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SMTP envelope sender", 2024-03-23T18:08:13.085 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.085 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.085 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.086 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.086 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_ssl": { 2024-03-23T18:08:13.086 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_ssl", 2024-03-23T18:08:13.086 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.086 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.086 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.087 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:13.087 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.087 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.087 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.087 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-23T18:08:13.087 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.087 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.087 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.087 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.088 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_user": { 2024-03-23T18:08:13.088 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_user", 2024-03-23T18:08:13.088 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.088 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.088 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.088 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.088 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.088 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.089 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.089 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "User to authenticate as", 2024-03-23T18:08:13.089 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.089 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.089 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.089 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:13.089 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:13.089 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.089 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:13.090 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "balancer", 2024-03-23T18:08:13.090 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:13.090 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:13.090 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:13.090 INFO:teuthology.orchestra.run.smithi086.stdout: "active": { 2024-03-23T18:08:13.090 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "active", 2024-03-23T18:08:13.090 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.091 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.091 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.091 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:13.091 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.091 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.091 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.091 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "automatically balance PGs across cluster", 2024-03-23T18:08:13.091 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.092 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.092 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.092 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.092 INFO:teuthology.orchestra.run.smithi086.stdout: "begin_time": { 2024-03-23T18:08:13.092 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "begin_time", 2024-03-23T18:08:13.092 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.092 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.092 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.093 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0000", 2024-03-23T18:08:13.093 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.093 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.093 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.093 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "beginning time of day to automatically balance", 2024-03-23T18:08:13.093 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-23T18:08:13.093 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.093 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.094 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.094 INFO:teuthology.orchestra.run.smithi086.stdout: "begin_weekday": { 2024-03-23T18:08:13.094 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "begin_weekday", 2024-03-23T18:08:13.094 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-03-23T18:08:13.094 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.094 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.094 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-03-23T18:08:13.094 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "0", 2024-03-23T18:08:13.095 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "6", 2024-03-23T18:08:13.095 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.095 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-23T18:08:13.095 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-23T18:08:13.095 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.095 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.095 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.095 INFO:teuthology.orchestra.run.smithi086.stdout: "crush_compat_max_iterations": { 2024-03-23T18:08:13.096 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crush_compat_max_iterations", 2024-03-23T18:08:13.096 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-03-23T18:08:13.096 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.096 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.096 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "25", 2024-03-23T18:08:13.096 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-03-23T18:08:13.096 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "250", 2024-03-23T18:08:13.096 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.096 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-23T18:08:13.097 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.097 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.097 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.097 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.097 INFO:teuthology.orchestra.run.smithi086.stdout: "crush_compat_metrics": { 2024-03-23T18:08:13.097 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crush_compat_metrics", 2024-03-23T18:08:13.097 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.097 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.098 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.098 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "pgs,objects,bytes", 2024-03-23T18:08:13.098 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.098 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.098 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.098 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-23T18:08:13.098 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-23T18:08:13.098 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.099 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.099 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.099 INFO:teuthology.orchestra.run.smithi086.stdout: "crush_compat_step": { 2024-03-23T18:08:13.099 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crush_compat_step", 2024-03-23T18:08:13.099 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-03-23T18:08:13.099 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.099 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.099 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0.5", 2024-03-23T18:08:13.099 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "0.001", 2024-03-23T18:08:13.100 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "0.999", 2024-03-23T18:08:13.100 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.100 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "aggressiveness of optimization", 2024-03-23T18:08:13.100 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-23T18:08:13.100 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.100 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.100 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.100 INFO:teuthology.orchestra.run.smithi086.stdout: "end_time": { 2024-03-23T18:08:13.100 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "end_time", 2024-03-23T18:08:13.101 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.101 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.101 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.101 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "2359", 2024-03-23T18:08:13.101 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.101 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.101 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.101 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "ending time of day to automatically balance", 2024-03-23T18:08:13.101 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-23T18:08:13.102 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.102 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.102 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.102 INFO:teuthology.orchestra.run.smithi086.stdout: "end_weekday": { 2024-03-23T18:08:13.102 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "end_weekday", 2024-03-23T18:08:13.102 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-03-23T18:08:13.102 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.102 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.103 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-03-23T18:08:13.103 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "0", 2024-03-23T18:08:13.103 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "6", 2024-03-23T18:08:13.103 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.103 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-23T18:08:13.103 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-23T18:08:13.103 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.103 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.103 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.103 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:13.104 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:13.104 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.104 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.104 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.104 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.104 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.104 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.104 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.104 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:13.104 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:13.105 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:13.105 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:13.105 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:13.105 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:13.105 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.105 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.105 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.105 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.105 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.106 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.106 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:13.106 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:13.106 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.106 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.106 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.106 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.106 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.107 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.107 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.107 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.107 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.107 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.107 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.107 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.107 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:13.107 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:13.108 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.108 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.108 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.108 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:13.108 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.108 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.108 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.108 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:13.108 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:13.109 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:13.109 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:13.109 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:13.109 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:13.109 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.109 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.109 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.109 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.109 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.110 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.110 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:13.110 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:13.110 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.110 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.110 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.110 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.110 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.111 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.111 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.111 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.111 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.111 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.111 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.111 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.111 INFO:teuthology.orchestra.run.smithi086.stdout: "min_score": { 2024-03-23T18:08:13.111 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "min_score", 2024-03-23T18:08:13.111 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-03-23T18:08:13.112 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.112 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.112 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-03-23T18:08:13.112 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.112 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.112 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.112 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-23T18:08:13.112 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.112 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.113 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.113 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.113 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": { 2024-03-23T18:08:13.113 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mode", 2024-03-23T18:08:13.113 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.113 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.113 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.113 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "upmap", 2024-03-23T18:08:13.114 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.114 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.114 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.114 INFO:teuthology.orchestra.run.smithi086.stdout: "crush-compat", 2024-03-23T18:08:13.114 INFO:teuthology.orchestra.run.smithi086.stdout: "none", 2024-03-23T18:08:13.114 INFO:teuthology.orchestra.run.smithi086.stdout: "upmap" 2024-03-23T18:08:13.114 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.114 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Balancer mode", 2024-03-23T18:08:13.115 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.115 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.115 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.115 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.115 INFO:teuthology.orchestra.run.smithi086.stdout: "pool_ids": { 2024-03-23T18:08:13.115 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pool_ids", 2024-03-23T18:08:13.115 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.115 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.115 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.116 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.116 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.116 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.116 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.116 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-23T18:08:13.116 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.116 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.117 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.117 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.117 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-03-23T18:08:13.117 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-03-23T18:08:13.117 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-23T18:08:13.117 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.117 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.118 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-03-23T18:08:13.118 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.118 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.118 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.118 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-23T18:08:13.118 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.118 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.118 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.119 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.119 INFO:teuthology.orchestra.run.smithi086.stdout: "upmap_max_deviation": { 2024-03-23T18:08:13.119 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "upmap_max_deviation", 2024-03-23T18:08:13.119 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:13.119 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.119 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.120 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-03-23T18:08:13.120 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-03-23T18:08:13.120 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.120 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.120 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-23T18:08:13.120 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-23T18:08:13.120 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.120 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.120 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.121 INFO:teuthology.orchestra.run.smithi086.stdout: "upmap_max_optimizations": { 2024-03-23T18:08:13.121 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "upmap_max_optimizations", 2024-03-23T18:08:13.121 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-03-23T18:08:13.121 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.121 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.121 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-03-23T18:08:13.122 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.122 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.122 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.122 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-23T18:08:13.122 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.122 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.122 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.122 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:13.123 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:13.123 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.123 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:13.123 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cephadm", 2024-03-23T18:08:13.123 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:13.123 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:13.123 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:13.124 INFO:teuthology.orchestra.run.smithi086.stdout: "agent_down_multiplier": { 2024-03-23T18:08:13.124 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "agent_down_multiplier", 2024-03-23T18:08:13.124 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-03-23T18:08:13.124 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.124 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.124 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "3.0", 2024-03-23T18:08:13.124 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.125 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.125 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.125 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-23T18:08:13.125 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.125 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.125 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.125 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.125 INFO:teuthology.orchestra.run.smithi086.stdout: "agent_refresh_rate": { 2024-03-23T18:08:13.126 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "agent_refresh_rate", 2024-03-23T18:08:13.126 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-23T18:08:13.126 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.126 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.126 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "20", 2024-03-23T18:08:13.126 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.126 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.127 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.127 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-23T18:08:13.127 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.127 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.127 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.127 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.128 INFO:teuthology.orchestra.run.smithi086.stdout: "agent_starting_port": { 2024-03-23T18:08:13.128 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "agent_starting_port", 2024-03-23T18:08:13.128 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:13.128 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.128 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.128 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "4721", 2024-03-23T18:08:13.128 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.129 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.129 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.129 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-23T18:08:13.129 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.129 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.129 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.129 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.129 INFO:teuthology.orchestra.run.smithi086.stdout: "allow_ptrace": { 2024-03-23T18:08:13.130 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "allow_ptrace", 2024-03-23T18:08:13.130 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.130 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.130 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.130 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.130 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.130 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.130 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.131 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-23T18:08:13.131 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-23T18:08:13.131 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.131 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.131 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.131 INFO:teuthology.orchestra.run.smithi086.stdout: "autotune_interval": { 2024-03-23T18:08:13.131 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "autotune_interval", 2024-03-23T18:08:13.131 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-23T18:08:13.131 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.132 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.132 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-03-23T18:08:13.132 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.132 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.132 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.132 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-23T18:08:13.132 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.132 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.133 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.133 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.133 INFO:teuthology.orchestra.run.smithi086.stdout: "autotune_memory_target_ratio": { 2024-03-23T18:08:13.133 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "autotune_memory_target_ratio", 2024-03-23T18:08:13.133 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-03-23T18:08:13.133 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.133 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.133 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0.7", 2024-03-23T18:08:13.134 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.134 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.134 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.134 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-23T18:08:13.134 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.134 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.134 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.134 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.135 INFO:teuthology.orchestra.run.smithi086.stdout: "cgroups_split": { 2024-03-23T18:08:13.135 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cgroups_split", 2024-03-23T18:08:13.135 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.135 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.135 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.135 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:13.135 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.135 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.135 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.136 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-23T18:08:13.136 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.136 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.136 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.136 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.136 INFO:teuthology.orchestra.run.smithi086.stdout: "config_checks_enabled": { 2024-03-23T18:08:13.136 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "config_checks_enabled", 2024-03-23T18:08:13.136 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.137 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.137 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.137 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.137 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.137 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.137 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.137 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-23T18:08:13.137 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.137 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.138 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.138 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.138 INFO:teuthology.orchestra.run.smithi086.stdout: "config_dashboard": { 2024-03-23T18:08:13.138 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "config_dashboard", 2024-03-23T18:08:13.138 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.138 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.138 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.138 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:13.139 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.139 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.139 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.139 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-23T18:08:13.139 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.139 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.139 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.139 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.139 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_alertmanager": { 2024-03-23T18:08:13.140 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_alertmanager", 2024-03-23T18:08:13.140 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.140 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.140 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.140 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-23T18:08:13.140 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.140 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.140 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.140 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-03-23T18:08:13.141 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.141 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.141 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.141 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.141 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_base": { 2024-03-23T18:08:13.141 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_base", 2024-03-23T18:08:13.141 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.141 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.141 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.142 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-23T18:08:13.142 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.142 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.142 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.142 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Container image name, without the tag", 2024-03-23T18:08:13.142 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.142 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.142 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.142 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.143 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_elasticsearch": { 2024-03-23T18:08:13.143 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_elasticsearch", 2024-03-23T18:08:13.143 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.143 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.143 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.143 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-23T18:08:13.143 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.143 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.144 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.144 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "elasticsearch container image", 2024-03-23T18:08:13.144 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.144 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.144 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.144 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.144 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_grafana": { 2024-03-23T18:08:13.144 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_grafana", 2024-03-23T18:08:13.145 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.145 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.145 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.145 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2024-03-23T18:08:13.145 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.145 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.145 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.145 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-03-23T18:08:13.146 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.146 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.146 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.146 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.146 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_haproxy": { 2024-03-23T18:08:13.146 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_haproxy", 2024-03-23T18:08:13.146 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.146 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.147 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.147 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-23T18:08:13.147 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.147 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.147 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.147 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "HAproxy container image", 2024-03-23T18:08:13.147 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.147 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.148 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.148 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.148 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_jaeger_agent": { 2024-03-23T18:08:13.148 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_jaeger_agent", 2024-03-23T18:08:13.148 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.148 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.148 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.148 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-23T18:08:13.149 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.149 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.149 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.149 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Jaeger agent container image", 2024-03-23T18:08:13.149 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.149 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.149 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.149 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.150 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_jaeger_collector": { 2024-03-23T18:08:13.150 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_jaeger_collector", 2024-03-23T18:08:13.150 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.150 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.150 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.150 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-23T18:08:13.150 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.151 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.151 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.151 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Jaeger collector container image", 2024-03-23T18:08:13.151 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.151 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.151 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.151 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.151 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_jaeger_query": { 2024-03-23T18:08:13.152 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_jaeger_query", 2024-03-23T18:08:13.152 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.152 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.152 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.152 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-23T18:08:13.152 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.152 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.152 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.153 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Jaeger query container image", 2024-03-23T18:08:13.153 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.153 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.153 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.153 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.153 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_keepalived": { 2024-03-23T18:08:13.153 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_keepalived", 2024-03-23T18:08:13.153 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.153 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.154 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.154 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-23T18:08:13.154 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.154 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.154 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.154 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Keepalived container image", 2024-03-23T18:08:13.155 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.155 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.155 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.155 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.155 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_loki": { 2024-03-23T18:08:13.155 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_loki", 2024-03-23T18:08:13.155 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.155 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.156 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.156 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-23T18:08:13.156 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.156 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.156 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.156 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Loki container image", 2024-03-23T18:08:13.156 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.156 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.157 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.157 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.157 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_node_exporter": { 2024-03-23T18:08:13.157 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_node_exporter", 2024-03-23T18:08:13.157 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.157 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.157 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.157 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-23T18:08:13.157 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.158 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.158 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.158 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-03-23T18:08:13.158 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.158 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.158 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.158 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.158 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_nvmeof": { 2024-03-23T18:08:13.158 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_nvmeof", 2024-03-23T18:08:13.159 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.159 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.159 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.159 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-23T18:08:13.159 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.159 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.159 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.159 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Nvme-of container image", 2024-03-23T18:08:13.160 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.160 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.160 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.160 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.160 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_prometheus": { 2024-03-23T18:08:13.160 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_prometheus", 2024-03-23T18:08:13.160 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.160 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.161 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.161 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-23T18:08:13.161 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.161 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.161 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.161 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-03-23T18:08:13.162 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.162 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.162 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.162 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.162 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_promtail": { 2024-03-23T18:08:13.163 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_promtail", 2024-03-23T18:08:13.163 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.163 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.163 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.163 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-23T18:08:13.163 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.163 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.163 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.164 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Promtail container image", 2024-03-23T18:08:13.164 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.164 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.164 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.164 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.164 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_snmp_gateway": { 2024-03-23T18:08:13.164 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_snmp_gateway", 2024-03-23T18:08:13.164 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.164 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.165 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.165 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-23T18:08:13.165 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.165 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.165 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.165 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SNMP Gateway container image", 2024-03-23T18:08:13.165 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.165 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.166 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.166 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.166 INFO:teuthology.orchestra.run.smithi086.stdout: "container_init": { 2024-03-23T18:08:13.166 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_init", 2024-03-23T18:08:13.166 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.166 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.166 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.166 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:13.167 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.167 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.167 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.167 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Run podman/docker with `--init`", 2024-03-23T18:08:13.167 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.167 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.167 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.167 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.168 INFO:teuthology.orchestra.run.smithi086.stdout: "daemon_cache_timeout": { 2024-03-23T18:08:13.168 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "daemon_cache_timeout", 2024-03-23T18:08:13.168 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-23T18:08:13.168 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.168 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.168 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-03-23T18:08:13.168 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.169 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.169 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.169 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-23T18:08:13.169 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.169 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.169 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.169 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.169 INFO:teuthology.orchestra.run.smithi086.stdout: "default_cephadm_command_timeout": { 2024-03-23T18:08:13.170 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "default_cephadm_command_timeout", 2024-03-23T18:08:13.170 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:13.170 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.170 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.170 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "900", 2024-03-23T18:08:13.170 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.170 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.170 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.171 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-23T18:08:13.171 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.171 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.171 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.171 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.171 INFO:teuthology.orchestra.run.smithi086.stdout: "default_registry": { 2024-03-23T18:08:13.171 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "default_registry", 2024-03-23T18:08:13.172 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.172 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.172 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.172 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io", 2024-03-23T18:08:13.172 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.172 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.172 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.172 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-23T18:08:13.172 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.173 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.173 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.173 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.173 INFO:teuthology.orchestra.run.smithi086.stdout: "device_cache_timeout": { 2024-03-23T18:08:13.173 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "device_cache_timeout", 2024-03-23T18:08:13.173 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-23T18:08:13.173 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.173 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.174 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "1800", 2024-03-23T18:08:13.174 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.174 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.174 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.174 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "seconds to cache device inventory", 2024-03-23T18:08:13.174 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.174 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.174 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.175 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.175 INFO:teuthology.orchestra.run.smithi086.stdout: "device_enhanced_scan": { 2024-03-23T18:08:13.175 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "device_enhanced_scan", 2024-03-23T18:08:13.175 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.175 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.175 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.175 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.175 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.175 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.176 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.176 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-23T18:08:13.176 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.176 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.176 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.176 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.176 INFO:teuthology.orchestra.run.smithi086.stdout: "facts_cache_timeout": { 2024-03-23T18:08:13.176 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "facts_cache_timeout", 2024-03-23T18:08:13.176 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-23T18:08:13.177 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.177 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.177 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-03-23T18:08:13.177 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.177 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.177 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.177 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "seconds to cache host facts data", 2024-03-23T18:08:13.177 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.178 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.178 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.178 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.178 INFO:teuthology.orchestra.run.smithi086.stdout: "host_check_interval": { 2024-03-23T18:08:13.178 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "host_check_interval", 2024-03-23T18:08:13.178 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-23T18:08:13.178 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.178 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.178 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-03-23T18:08:13.179 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.179 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.179 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.179 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to perform a host check", 2024-03-23T18:08:13.179 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.179 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.179 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.179 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.179 INFO:teuthology.orchestra.run.smithi086.stdout: "hw_monitoring": { 2024-03-23T18:08:13.180 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "hw_monitoring", 2024-03-23T18:08:13.180 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.180 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.180 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.180 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.180 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.180 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.180 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.181 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-23T18:08:13.181 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.181 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.181 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.181 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.181 INFO:teuthology.orchestra.run.smithi086.stdout: "inventory_list_all": { 2024-03-23T18:08:13.181 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "inventory_list_all", 2024-03-23T18:08:13.181 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.182 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.182 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.182 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.182 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.182 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.182 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.182 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-23T18:08:13.183 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.183 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.183 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.183 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.183 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:13.183 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:13.183 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.183 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.184 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.184 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.184 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.184 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.184 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.184 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:13.184 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:13.184 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:13.185 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:13.185 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:13.185 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:13.185 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.185 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.185 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.185 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.185 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.186 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.186 INFO:teuthology.orchestra.run.smithi086.stdout: "log_refresh_metadata": { 2024-03-23T18:08:13.186 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_refresh_metadata", 2024-03-23T18:08:13.186 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.186 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.186 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.186 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.186 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.187 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.187 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.187 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-23T18:08:13.187 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.187 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.187 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.187 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.187 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:13.187 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:13.188 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.188 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.188 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.188 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:13.188 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.188 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.188 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.188 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-23T18:08:13.189 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.189 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.189 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.189 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.189 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:13.189 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:13.189 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.189 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.190 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.190 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:13.190 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.190 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.190 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.190 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:13.190 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:13.190 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:13.190 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:13.191 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:13.191 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:13.191 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.191 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.191 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.191 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.191 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.191 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.192 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:13.192 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:13.192 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.192 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.192 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.192 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.192 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.193 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.193 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.193 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.193 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.193 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.193 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.193 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.193 INFO:teuthology.orchestra.run.smithi086.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-23T18:08:13.193 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-23T18:08:13.194 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.194 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.194 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.194 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.194 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.194 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.194 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.194 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-23T18:08:13.195 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.198 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.198 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.198 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.198 INFO:teuthology.orchestra.run.smithi086.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-23T18:08:13.198 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-23T18:08:13.198 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.199 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.199 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.199 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "*", 2024-03-23T18:08:13.199 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.199 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.199 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.199 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-23T18:08:13.199 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.199 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.200 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.200 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.200 INFO:teuthology.orchestra.run.smithi086.stdout: "max_count_per_host": { 2024-03-23T18:08:13.200 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_count_per_host", 2024-03-23T18:08:13.200 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:13.200 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.200 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.200 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-03-23T18:08:13.200 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.201 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.201 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.201 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "max number of daemons per service per host", 2024-03-23T18:08:13.201 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.201 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.201 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.201 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.201 INFO:teuthology.orchestra.run.smithi086.stdout: "max_osd_draining_count": { 2024-03-23T18:08:13.202 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_osd_draining_count", 2024-03-23T18:08:13.202 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:13.202 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.202 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.202 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-03-23T18:08:13.202 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.202 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.202 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.203 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-23T18:08:13.203 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.203 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.203 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.203 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.203 INFO:teuthology.orchestra.run.smithi086.stdout: "migration_current": { 2024-03-23T18:08:13.203 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "migration_current", 2024-03-23T18:08:13.203 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:13.203 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.204 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.204 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.204 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.204 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.204 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.204 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "internal - do not modify", 2024-03-23T18:08:13.204 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.204 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.204 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.205 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.205 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": { 2024-03-23T18:08:13.205 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mode", 2024-03-23T18:08:13.205 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.205 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.205 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.205 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "root", 2024-03-23T18:08:13.205 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.205 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.206 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.206 INFO:teuthology.orchestra.run.smithi086.stdout: "cephadm-package", 2024-03-23T18:08:13.206 INFO:teuthology.orchestra.run.smithi086.stdout: "root" 2024-03-23T18:08:13.206 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.206 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "mode for remote execution of cephadm", 2024-03-23T18:08:13.206 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.206 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.206 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.207 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.207 INFO:teuthology.orchestra.run.smithi086.stdout: "oob_default_addr": { 2024-03-23T18:08:13.207 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "oob_default_addr", 2024-03-23T18:08:13.207 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.207 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.207 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.207 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "169.254.1.1", 2024-03-23T18:08:13.207 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.208 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.208 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.208 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-23T18:08:13.208 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.208 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.208 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.208 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.208 INFO:teuthology.orchestra.run.smithi086.stdout: "prometheus_alerts_path": { 2024-03-23T18:08:13.208 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "prometheus_alerts_path", 2024-03-23T18:08:13.209 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.209 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.209 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.209 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-23T18:08:13.209 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.209 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.209 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.209 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-23T18:08:13.209 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.210 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.210 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.210 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.210 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_insecure": { 2024-03-23T18:08:13.210 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_insecure", 2024-03-23T18:08:13.210 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.210 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.210 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.211 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.211 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.211 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.211 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.211 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-23T18:08:13.211 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.211 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.211 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.211 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.212 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_password": { 2024-03-23T18:08:13.212 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_password", 2024-03-23T18:08:13.212 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.212 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.212 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.212 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.212 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.212 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.212 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.213 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-23T18:08:13.213 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.213 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.213 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.213 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.213 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_url": { 2024-03-23T18:08:13.213 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_url", 2024-03-23T18:08:13.213 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.214 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.214 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.214 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.214 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.214 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.214 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.214 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-23T18:08:13.214 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.214 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.215 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.215 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.215 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_username": { 2024-03-23T18:08:13.215 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_username", 2024-03-23T18:08:13.215 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.215 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.215 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.215 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.216 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.216 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.216 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.216 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-23T18:08:13.216 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.216 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.216 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.217 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.217 INFO:teuthology.orchestra.run.smithi086.stdout: "secure_monitoring_stack": { 2024-03-23T18:08:13.217 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "secure_monitoring_stack", 2024-03-23T18:08:13.217 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.217 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.217 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.217 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.217 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.218 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.218 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.218 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-23T18:08:13.218 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.218 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.218 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.218 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.218 INFO:teuthology.orchestra.run.smithi086.stdout: "service_discovery_port": { 2024-03-23T18:08:13.218 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "service_discovery_port", 2024-03-23T18:08:13.219 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:13.219 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.219 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.219 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8765", 2024-03-23T18:08:13.219 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.219 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.219 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.219 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "cephadm service discovery port", 2024-03-23T18:08:13.220 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.220 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.220 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.220 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.220 INFO:teuthology.orchestra.run.smithi086.stdout: "ssh_config_file": { 2024-03-23T18:08:13.220 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssh_config_file", 2024-03-23T18:08:13.220 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.220 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.220 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.220 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.221 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.221 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.221 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.221 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-23T18:08:13.221 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.221 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.221 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.221 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.221 INFO:teuthology.orchestra.run.smithi086.stdout: "use_agent": { 2024-03-23T18:08:13.221 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "use_agent", 2024-03-23T18:08:13.222 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.222 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.222 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.222 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.222 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.222 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.222 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.222 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-23T18:08:13.223 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.223 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.223 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.223 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.223 INFO:teuthology.orchestra.run.smithi086.stdout: "use_repo_digest": { 2024-03-23T18:08:13.223 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "use_repo_digest", 2024-03-23T18:08:13.223 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.223 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.223 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.223 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:13.224 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.224 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.224 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.224 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-23T18:08:13.224 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.224 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.224 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.224 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.225 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_on_failed_host_check": { 2024-03-23T18:08:13.225 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_on_failed_host_check", 2024-03-23T18:08:13.225 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.225 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.225 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.225 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:13.225 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.225 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.225 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.226 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise a health warning if the host check fails", 2024-03-23T18:08:13.226 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.226 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.226 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.226 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.226 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_on_stray_daemons": { 2024-03-23T18:08:13.226 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_on_stray_daemons", 2024-03-23T18:08:13.226 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.227 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.227 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.227 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:13.227 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.227 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.227 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.227 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-23T18:08:13.227 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.227 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.228 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.228 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.228 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_on_stray_hosts": { 2024-03-23T18:08:13.228 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_on_stray_hosts", 2024-03-23T18:08:13.228 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.228 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.228 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.228 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:13.228 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.229 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.229 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.229 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-23T18:08:13.229 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.229 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.229 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.229 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:13.229 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:13.230 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.230 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:13.230 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crash", 2024-03-23T18:08:13.230 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:13.230 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:13.230 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:13.230 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:13.230 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:13.230 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.231 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.231 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.231 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.231 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.231 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.231 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.231 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:13.231 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:13.232 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:13.232 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:13.232 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:13.232 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:13.232 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.232 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.232 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.232 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.232 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.233 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.233 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:13.233 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:13.233 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.233 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.233 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.233 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.233 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.233 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.234 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.234 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.234 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.234 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.234 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.234 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.234 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:13.234 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:13.235 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.235 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.235 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.235 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:13.235 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.235 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.235 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.235 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:13.236 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:13.236 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:13.236 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:13.236 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:13.236 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:13.236 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.236 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.236 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.237 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.237 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.237 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.237 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:13.237 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:13.237 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.237 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.237 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.238 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.238 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.238 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.238 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.238 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.238 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.238 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.238 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.238 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.239 INFO:teuthology.orchestra.run.smithi086.stdout: "retain_interval": { 2024-03-23T18:08:13.239 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "retain_interval", 2024-03-23T18:08:13.239 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-23T18:08:13.239 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.239 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.239 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "31536000", 2024-03-23T18:08:13.239 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.239 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.239 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.240 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-23T18:08:13.240 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.240 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.240 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.240 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.240 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_recent_interval": { 2024-03-23T18:08:13.240 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_recent_interval", 2024-03-23T18:08:13.240 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-23T18:08:13.240 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.241 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.241 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "1209600", 2024-03-23T18:08:13.241 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.241 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.241 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.241 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-23T18:08:13.241 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.241 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.242 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.242 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:13.242 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:13.242 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.242 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:13.242 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "dashboard", 2024-03-23T18:08:13.242 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:13.242 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:13.243 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:13.243 INFO:teuthology.orchestra.run.smithi086.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-23T18:08:13.243 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-23T18:08:13.243 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:13.243 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.243 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.243 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-03-23T18:08:13.243 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.244 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.244 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.244 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.244 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.244 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.244 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.244 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.244 INFO:teuthology.orchestra.run.smithi086.stdout: "ALERTMANAGER_API_HOST": { 2024-03-23T18:08:13.244 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-23T18:08:13.245 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.245 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.245 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.245 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.245 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.245 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.245 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.245 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.245 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.246 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.246 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.246 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.246 INFO:teuthology.orchestra.run.smithi086.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-23T18:08:13.246 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-23T18:08:13.246 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.246 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.246 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.247 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:13.247 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.247 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.247 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.247 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.247 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.247 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.247 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.247 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.248 INFO:teuthology.orchestra.run.smithi086.stdout: "AUDIT_API_ENABLED": { 2024-03-23T18:08:13.248 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "AUDIT_API_ENABLED", 2024-03-23T18:08:13.248 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.248 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.248 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.248 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.248 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.248 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.248 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.249 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.249 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.249 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.249 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.249 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.249 INFO:teuthology.orchestra.run.smithi086.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-23T18:08:13.249 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-23T18:08:13.249 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.249 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.250 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.250 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:13.250 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.250 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.250 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.250 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.250 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.250 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.250 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.251 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.251 INFO:teuthology.orchestra.run.smithi086.stdout: "ENABLE_BROWSABLE_API": { 2024-03-23T18:08:13.251 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-23T18:08:13.251 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.251 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.251 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.251 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:13.251 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.251 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.252 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.252 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.252 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.252 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.252 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.252 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.252 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-23T18:08:13.252 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-23T18:08:13.252 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.253 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.253 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.253 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:13.253 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.253 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.253 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.253 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.253 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.253 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.254 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.254 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.254 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-23T18:08:13.254 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-23T18:08:13.254 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.254 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.254 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.254 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:13.254 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.255 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.255 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.255 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.255 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.255 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.255 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.255 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.255 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-23T18:08:13.255 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-23T18:08:13.255 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.256 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.256 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.256 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:13.256 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.256 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.256 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.256 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.256 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.256 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.257 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.257 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.257 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-23T18:08:13.257 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-23T18:08:13.257 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.257 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.257 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.257 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:13.257 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.257 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.258 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.258 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.258 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.258 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.258 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.258 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.258 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-23T18:08:13.258 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-23T18:08:13.259 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.259 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.259 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.259 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:13.259 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.259 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.259 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.259 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.259 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.259 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.260 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.260 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.260 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-23T18:08:13.260 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-23T18:08:13.260 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.260 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.260 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.260 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:13.260 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.261 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.261 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.261 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.261 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.261 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.261 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.261 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.261 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-23T18:08:13.261 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-23T18:08:13.262 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.262 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.262 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.262 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:13.262 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.262 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.262 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.262 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.262 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.263 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.263 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.263 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.263 INFO:teuthology.orchestra.run.smithi086.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-23T18:08:13.263 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-23T18:08:13.263 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.263 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.263 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.263 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.264 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.264 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.264 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.264 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.264 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.264 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.264 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.264 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.264 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_PASSWORD": { 2024-03-23T18:08:13.264 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-23T18:08:13.265 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.265 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.265 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.265 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "admin", 2024-03-23T18:08:13.265 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.265 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.265 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.265 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.266 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.266 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.266 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.266 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.266 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-23T18:08:13.266 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-23T18:08:13.266 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.266 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.267 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.267 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:13.267 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.267 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.267 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.267 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.267 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.267 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.267 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.268 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.268 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_URL": { 2024-03-23T18:08:13.268 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_URL", 2024-03-23T18:08:13.268 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.268 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.268 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.268 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.268 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.269 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.269 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.269 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.269 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.269 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.269 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.269 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.269 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_USERNAME": { 2024-03-23T18:08:13.269 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-23T18:08:13.269 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.270 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.270 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.270 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "admin", 2024-03-23T18:08:13.270 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.270 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.270 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.270 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.270 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.271 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.271 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.271 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.271 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-23T18:08:13.271 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-23T18:08:13.271 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.271 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.271 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.272 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.272 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.272 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.272 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.272 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.272 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.272 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.272 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.272 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.273 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-23T18:08:13.273 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-23T18:08:13.273 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.273 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.273 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.273 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.273 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.273 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.273 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.274 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.274 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.274 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.274 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.274 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.274 INFO:teuthology.orchestra.run.smithi086.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-23T18:08:13.274 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-23T18:08:13.274 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.275 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.275 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.275 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:13.275 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.275 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.275 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.275 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.275 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.276 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.276 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.276 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.276 INFO:teuthology.orchestra.run.smithi086.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-23T18:08:13.276 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-23T18:08:13.276 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.276 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.276 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.276 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.277 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.277 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.277 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.277 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.277 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.277 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.277 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.277 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.277 INFO:teuthology.orchestra.run.smithi086.stdout: "PROMETHEUS_API_HOST": { 2024-03-23T18:08:13.278 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-23T18:08:13.278 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.278 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.278 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.278 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.278 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.278 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.278 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.278 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.279 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.279 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.279 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.279 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.279 INFO:teuthology.orchestra.run.smithi086.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-23T18:08:13.279 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-23T18:08:13.279 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.279 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.280 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.280 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:13.280 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.280 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.280 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.280 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.280 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.280 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.280 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.280 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.281 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-23T18:08:13.281 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-23T18:08:13.281 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.281 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.281 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.281 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.281 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.281 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.281 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.281 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.282 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.282 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.282 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.282 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.282 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-23T18:08:13.282 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-23T18:08:13.282 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.282 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.283 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.283 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.283 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.283 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.283 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.283 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.283 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.283 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.284 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.284 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.284 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-23T18:08:13.284 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-23T18:08:13.284 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.284 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.284 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.284 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:13.285 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.285 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.285 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.285 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.285 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.285 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.285 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.285 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.285 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-23T18:08:13.286 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-23T18:08:13.286 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.286 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.286 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.286 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:13.286 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.286 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.286 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.287 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.287 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.287 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.287 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.287 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.287 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-23T18:08:13.287 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-23T18:08:13.287 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.287 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.288 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.288 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.288 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.288 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.288 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.288 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.288 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.288 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.288 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.289 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.289 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-23T18:08:13.289 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-23T18:08:13.289 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.289 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.289 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.289 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.289 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.290 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.290 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.290 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.290 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.290 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.290 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.290 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.290 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-23T18:08:13.290 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-23T18:08:13.291 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.291 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.291 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.291 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.291 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.291 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.291 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.291 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.291 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.292 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.292 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.292 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.292 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_ENABLED": { 2024-03-23T18:08:13.292 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-23T18:08:13.292 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.292 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.292 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.293 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:13.293 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.293 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.293 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.293 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.293 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.293 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.293 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.293 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.294 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-23T18:08:13.294 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-23T18:08:13.294 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.294 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.294 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.294 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-23T18:08:13.294 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.294 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.295 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.295 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.295 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.295 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.295 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.295 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.295 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-23T18:08:13.295 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-23T18:08:13.295 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:13.296 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.296 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.296 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-03-23T18:08:13.296 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.296 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.296 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.296 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.296 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.296 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.296 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.297 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.297 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-23T18:08:13.297 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-23T18:08:13.297 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:13.297 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.297 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.297 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8", 2024-03-23T18:08:13.297 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.297 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.298 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.298 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.298 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.298 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.298 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.298 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.298 INFO:teuthology.orchestra.run.smithi086.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-23T18:08:13.298 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-23T18:08:13.299 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:13.299 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.299 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.299 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "45", 2024-03-23T18:08:13.299 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.299 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.299 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.300 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.300 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.300 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.300 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.300 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.300 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_ACCESS_KEY": { 2024-03-23T18:08:13.300 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-23T18:08:13.300 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.300 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.301 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.301 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.301 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.301 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.301 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.301 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.301 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.301 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.302 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.302 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.302 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-23T18:08:13.302 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-23T18:08:13.302 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.302 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.302 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.302 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "admin", 2024-03-23T18:08:13.303 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.303 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.303 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.303 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.303 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.303 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.303 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.303 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.303 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_SECRET_KEY": { 2024-03-23T18:08:13.304 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-23T18:08:13.304 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.304 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.304 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.304 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.304 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.304 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.304 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.304 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.305 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.305 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.305 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.305 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.305 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_SSL_VERIFY": { 2024-03-23T18:08:13.305 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-23T18:08:13.305 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.305 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.306 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.306 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:13.306 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.306 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.306 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.306 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.306 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.306 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.306 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.307 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.307 INFO:teuthology.orchestra.run.smithi086.stdout: "UNSAFE_TLS_v1_2": { 2024-03-23T18:08:13.307 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-23T18:08:13.307 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.307 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.307 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.307 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.307 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.308 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.308 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.308 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.308 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.308 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.308 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.308 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.308 INFO:teuthology.orchestra.run.smithi086.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-23T18:08:13.308 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-23T18:08:13.309 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:13.309 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.309 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.309 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-03-23T18:08:13.309 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.309 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.309 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.309 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.309 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.310 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.310 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.310 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.310 INFO:teuthology.orchestra.run.smithi086.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-23T18:08:13.310 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-23T18:08:13.310 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:13.310 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.310 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.310 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-03-23T18:08:13.311 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.311 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.311 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.311 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.311 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.311 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.311 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.311 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.311 INFO:teuthology.orchestra.run.smithi086.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-23T18:08:13.312 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-23T18:08:13.312 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:13.312 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.312 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.312 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-03-23T18:08:13.312 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.312 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.312 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.312 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.313 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.313 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.313 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.313 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.313 INFO:teuthology.orchestra.run.smithi086.stdout: "cross_origin_url": { 2024-03-23T18:08:13.313 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cross_origin_url", 2024-03-23T18:08:13.313 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.313 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.314 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.314 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.314 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.314 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.314 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.314 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.314 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.314 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.314 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.315 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.315 INFO:teuthology.orchestra.run.smithi086.stdout: "crt_file": { 2024-03-23T18:08:13.315 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crt_file", 2024-03-23T18:08:13.315 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.315 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.315 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.315 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.315 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.316 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.316 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.316 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.316 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.316 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.316 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.316 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.316 INFO:teuthology.orchestra.run.smithi086.stdout: "debug": { 2024-03-23T18:08:13.316 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "debug", 2024-03-23T18:08:13.317 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.317 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.317 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.317 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.317 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.317 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.317 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.317 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Enable/disable debug options", 2024-03-23T18:08:13.317 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.318 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.318 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.318 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.318 INFO:teuthology.orchestra.run.smithi086.stdout: "jwt_token_ttl": { 2024-03-23T18:08:13.318 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "jwt_token_ttl", 2024-03-23T18:08:13.318 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:13.318 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.318 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.319 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "28800", 2024-03-23T18:08:13.319 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.319 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.319 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.319 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.319 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.319 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.319 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.320 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.320 INFO:teuthology.orchestra.run.smithi086.stdout: "key_file": { 2024-03-23T18:08:13.320 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "key_file", 2024-03-23T18:08:13.320 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.320 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.320 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.320 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.320 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.320 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.321 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.321 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.321 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.321 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.321 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.321 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.321 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:13.321 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:13.321 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.322 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.322 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.322 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.322 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.322 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.322 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.322 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:13.322 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:13.323 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:13.323 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:13.323 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:13.323 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:13.323 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.323 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.323 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.323 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.324 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.324 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.324 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:13.324 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:13.324 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.324 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.324 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.324 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.324 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.325 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.325 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.325 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.325 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.325 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.325 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.325 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.325 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:13.325 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:13.326 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.326 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.326 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.326 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:13.326 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.326 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.326 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.326 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:13.327 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:13.327 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:13.327 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:13.327 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:13.327 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:13.327 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.327 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.327 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.328 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.328 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.328 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.328 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:13.328 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:13.328 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.328 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.328 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.328 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.329 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.329 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.329 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.329 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.329 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.329 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.329 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.329 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.329 INFO:teuthology.orchestra.run.smithi086.stdout: "motd": { 2024-03-23T18:08:13.330 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "motd", 2024-03-23T18:08:13.330 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.330 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.330 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.330 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.330 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.330 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.330 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.331 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "The message of the day", 2024-03-23T18:08:13.331 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.331 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.331 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.331 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.331 INFO:teuthology.orchestra.run.smithi086.stdout: "redirect_resolve_ip_addr": { 2024-03-23T18:08:13.331 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "redirect_resolve_ip_addr", 2024-03-23T18:08:13.331 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.332 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.332 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.332 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.332 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.332 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.332 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.332 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.332 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.332 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.333 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.333 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.333 INFO:teuthology.orchestra.run.smithi086.stdout: "server_addr": { 2024-03-23T18:08:13.333 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_addr", 2024-03-23T18:08:13.333 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.333 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.333 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.333 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "::", 2024-03-23T18:08:13.333 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.334 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.334 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.334 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.334 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.334 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.334 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.334 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.334 INFO:teuthology.orchestra.run.smithi086.stdout: "server_port": { 2024-03-23T18:08:13.334 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_port", 2024-03-23T18:08:13.335 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:13.335 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.335 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.335 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8080", 2024-03-23T18:08:13.335 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.335 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.335 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.335 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.336 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.336 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.336 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.336 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.336 INFO:teuthology.orchestra.run.smithi086.stdout: "ssl": { 2024-03-23T18:08:13.336 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssl", 2024-03-23T18:08:13.336 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.336 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.336 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.337 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:13.337 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.337 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.337 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.337 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.337 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.337 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.337 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.337 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.338 INFO:teuthology.orchestra.run.smithi086.stdout: "ssl_server_port": { 2024-03-23T18:08:13.338 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssl_server_port", 2024-03-23T18:08:13.338 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:13.338 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.338 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.338 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8443", 2024-03-23T18:08:13.338 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.338 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.338 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.339 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.339 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.339 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.339 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.339 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.339 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_behaviour": { 2024-03-23T18:08:13.339 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_behaviour", 2024-03-23T18:08:13.339 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.340 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.340 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.340 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "redirect", 2024-03-23T18:08:13.340 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.340 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.340 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.340 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:13.340 INFO:teuthology.orchestra.run.smithi086.stdout: "redirect" 2024-03-23T18:08:13.340 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.341 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.341 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.341 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.341 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.341 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.341 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_error_status_code": { 2024-03-23T18:08:13.341 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_error_status_code", 2024-03-23T18:08:13.341 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:13.342 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.342 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.342 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "500", 2024-03-23T18:08:13.342 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "400", 2024-03-23T18:08:13.342 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "599", 2024-03-23T18:08:13.342 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.342 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.342 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.342 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.343 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.343 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.343 INFO:teuthology.orchestra.run.smithi086.stdout: "url_prefix": { 2024-03-23T18:08:13.343 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "url_prefix", 2024-03-23T18:08:13.343 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.343 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.343 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.343 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.344 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.344 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.344 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.344 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.344 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.344 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.344 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.344 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:13.345 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:13.345 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.345 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:13.345 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "devicehealth", 2024-03-23T18:08:13.345 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:13.345 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:13.345 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:13.345 INFO:teuthology.orchestra.run.smithi086.stdout: "enable_monitoring": { 2024-03-23T18:08:13.346 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enable_monitoring", 2024-03-23T18:08:13.346 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.346 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.346 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.346 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:13.346 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.346 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.346 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.346 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "monitor device health metrics", 2024-03-23T18:08:13.347 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.347 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.347 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.347 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.347 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:13.347 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:13.347 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.347 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.348 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.348 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.348 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.348 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.348 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.348 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:13.348 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:13.348 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:13.349 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:13.349 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:13.349 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:13.349 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.349 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.349 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.349 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.349 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.350 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.350 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:13.350 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:13.350 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.350 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.350 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.350 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.350 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.351 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.351 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.351 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.351 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.351 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.351 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.351 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.351 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:13.352 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:13.352 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.352 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.352 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.352 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:13.352 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.352 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.352 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.352 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:13.353 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:13.353 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:13.353 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:13.353 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:13.353 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:13.353 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.353 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.353 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.353 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.354 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.354 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.354 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:13.354 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:13.354 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.354 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.354 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.354 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.355 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.355 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.355 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.355 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.355 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.355 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.355 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.355 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.356 INFO:teuthology.orchestra.run.smithi086.stdout: "mark_out_threshold": { 2024-03-23T18:08:13.356 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mark_out_threshold", 2024-03-23T18:08:13.356 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-23T18:08:13.356 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.356 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.356 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "2419200", 2024-03-23T18:08:13.356 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.356 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.356 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.357 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-23T18:08:13.357 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.357 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.357 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.357 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.357 INFO:teuthology.orchestra.run.smithi086.stdout: "pool_name": { 2024-03-23T18:08:13.357 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pool_name", 2024-03-23T18:08:13.357 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.357 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.358 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.358 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "device_health_metrics", 2024-03-23T18:08:13.358 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.358 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.358 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.358 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-23T18:08:13.358 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.358 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.359 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.359 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.359 INFO:teuthology.orchestra.run.smithi086.stdout: "retention_period": { 2024-03-23T18:08:13.359 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "retention_period", 2024-03-23T18:08:13.359 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-23T18:08:13.359 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.359 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.359 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "15552000", 2024-03-23T18:08:13.360 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.360 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.360 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.360 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how long to retain device health metrics", 2024-03-23T18:08:13.360 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.360 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.360 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.360 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.360 INFO:teuthology.orchestra.run.smithi086.stdout: "scrape_frequency": { 2024-03-23T18:08:13.361 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "scrape_frequency", 2024-03-23T18:08:13.361 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-23T18:08:13.361 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.361 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.361 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "86400", 2024-03-23T18:08:13.361 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.361 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.361 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.361 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-23T18:08:13.362 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.362 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.362 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.362 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.362 INFO:teuthology.orchestra.run.smithi086.stdout: "self_heal": { 2024-03-23T18:08:13.362 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "self_heal", 2024-03-23T18:08:13.362 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.362 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.363 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.363 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:13.363 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.363 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.363 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.363 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-23T18:08:13.363 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.363 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.364 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.364 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.364 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-03-23T18:08:13.364 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-03-23T18:08:13.364 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-23T18:08:13.364 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.364 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.364 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-03-23T18:08:13.364 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.365 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.365 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.365 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to wake up and check device health", 2024-03-23T18:08:13.365 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.365 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.365 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.365 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.365 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_threshold": { 2024-03-23T18:08:13.366 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_threshold", 2024-03-23T18:08:13.366 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-23T18:08:13.366 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.366 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.366 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "7257600", 2024-03-23T18:08:13.366 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.366 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.367 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.367 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-23T18:08:13.367 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.367 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.367 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.367 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:13.367 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:13.367 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.367 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:13.368 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "diskprediction_local", 2024-03-23T18:08:13.368 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:13.368 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:13.368 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:13.368 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:13.368 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:13.368 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.368 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.369 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.369 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.369 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.369 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.369 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.369 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:13.369 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:13.369 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:13.369 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:13.370 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:13.370 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:13.370 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.370 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.370 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.370 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.370 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.370 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.371 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:13.371 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:13.371 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.371 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.371 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.371 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.371 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.371 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.371 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.372 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.372 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.372 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.372 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.372 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.372 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:13.372 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:13.372 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.372 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.373 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.373 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:13.373 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.373 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.373 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.373 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:13.373 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:13.373 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:13.373 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:13.374 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:13.374 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:13.374 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.374 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.374 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.374 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.374 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.374 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.375 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:13.375 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:13.375 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.375 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.375 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.375 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.375 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.375 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.375 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.376 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.376 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.376 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.376 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.376 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.376 INFO:teuthology.orchestra.run.smithi086.stdout: "predict_interval": { 2024-03-23T18:08:13.376 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "predict_interval", 2024-03-23T18:08:13.376 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.376 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.377 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.377 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "86400", 2024-03-23T18:08:13.377 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.377 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.377 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.377 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.377 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.377 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.377 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.378 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.378 INFO:teuthology.orchestra.run.smithi086.stdout: "predictor_model": { 2024-03-23T18:08:13.378 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "predictor_model", 2024-03-23T18:08:13.378 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.378 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.378 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.378 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "prophetstor", 2024-03-23T18:08:13.378 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.378 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.379 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.379 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.379 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.379 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.379 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.379 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.379 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-03-23T18:08:13.379 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-03-23T18:08:13.379 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.380 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.380 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.380 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-03-23T18:08:13.380 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.380 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.380 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.380 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.380 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.380 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.381 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.381 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:13.381 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:13.381 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.381 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:13.381 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "influx", 2024-03-23T18:08:13.381 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": false, 2024-03-23T18:08:13.381 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "influxdb python module not found", 2024-03-23T18:08:13.381 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:13.382 INFO:teuthology.orchestra.run.smithi086.stdout: "batch_size": { 2024-03-23T18:08:13.382 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "batch_size", 2024-03-23T18:08:13.382 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:13.382 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.382 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.382 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5000", 2024-03-23T18:08:13.382 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.382 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.383 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.383 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-23T18:08:13.383 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.383 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.383 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.383 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.383 INFO:teuthology.orchestra.run.smithi086.stdout: "database": { 2024-03-23T18:08:13.383 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "database", 2024-03-23T18:08:13.383 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.384 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.384 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.384 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "ceph", 2024-03-23T18:08:13.384 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.384 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.384 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.384 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-23T18:08:13.384 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.385 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.385 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.385 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.385 INFO:teuthology.orchestra.run.smithi086.stdout: "hostname": { 2024-03-23T18:08:13.385 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "hostname", 2024-03-23T18:08:13.385 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.385 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.385 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.385 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.386 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.386 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.386 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.386 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "InfluxDB server hostname", 2024-03-23T18:08:13.386 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.386 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.386 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.386 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.387 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-03-23T18:08:13.387 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-03-23T18:08:13.387 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-23T18:08:13.387 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.387 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.387 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "30", 2024-03-23T18:08:13.387 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "5", 2024-03-23T18:08:13.387 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.388 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.388 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-23T18:08:13.388 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.388 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.388 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.388 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.388 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:13.388 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:13.389 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.389 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.389 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.389 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.389 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.389 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.389 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.389 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:13.389 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:13.389 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:13.390 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:13.390 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:13.390 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:13.390 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.390 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.390 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.390 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.390 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.390 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.391 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:13.391 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:13.391 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.391 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.391 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.391 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.391 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.391 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.391 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.392 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.392 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.392 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.392 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.392 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.392 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:13.392 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:13.392 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.392 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.393 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.393 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:13.393 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.393 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.393 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.393 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:13.393 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:13.393 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:13.393 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:13.394 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:13.394 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:13.394 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.394 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.394 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.394 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.394 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.394 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.395 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:13.395 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:13.395 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.395 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.395 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.395 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.395 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.395 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.396 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.396 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.396 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.396 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.396 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.396 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.396 INFO:teuthology.orchestra.run.smithi086.stdout: "password": { 2024-03-23T18:08:13.396 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "password", 2024-03-23T18:08:13.396 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.397 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.397 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.397 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.397 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.397 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.397 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.397 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "password of InfluxDB server user", 2024-03-23T18:08:13.397 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.397 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.398 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.398 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.398 INFO:teuthology.orchestra.run.smithi086.stdout: "port": { 2024-03-23T18:08:13.398 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "port", 2024-03-23T18:08:13.398 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:13.398 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.398 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.398 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8086", 2024-03-23T18:08:13.399 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.399 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.399 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.399 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "InfluxDB server port", 2024-03-23T18:08:13.399 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.399 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.399 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.399 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.399 INFO:teuthology.orchestra.run.smithi086.stdout: "ssl": { 2024-03-23T18:08:13.400 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssl", 2024-03-23T18:08:13.400 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.400 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.400 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.400 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "false", 2024-03-23T18:08:13.400 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.400 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.400 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.400 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-23T18:08:13.401 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.401 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.401 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.401 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.401 INFO:teuthology.orchestra.run.smithi086.stdout: "threads": { 2024-03-23T18:08:13.401 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "threads", 2024-03-23T18:08:13.401 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:13.401 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.401 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.402 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-03-23T18:08:13.402 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-03-23T18:08:13.402 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "32", 2024-03-23T18:08:13.402 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.402 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-23T18:08:13.402 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.402 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.402 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.403 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.403 INFO:teuthology.orchestra.run.smithi086.stdout: "username": { 2024-03-23T18:08:13.403 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "username", 2024-03-23T18:08:13.403 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.403 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.403 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.403 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.403 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.403 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.404 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.404 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "username of InfluxDB server user", 2024-03-23T18:08:13.404 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.404 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.404 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.404 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.404 INFO:teuthology.orchestra.run.smithi086.stdout: "verify_ssl": { 2024-03-23T18:08:13.404 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "verify_ssl", 2024-03-23T18:08:13.404 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.405 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.405 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.405 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "true", 2024-03-23T18:08:13.405 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.405 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.405 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.405 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-23T18:08:13.405 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.405 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.406 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.406 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:13.406 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:13.406 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.406 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:13.406 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "insights", 2024-03-23T18:08:13.406 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:13.406 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:13.407 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:13.407 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:13.407 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:13.407 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.407 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.407 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.407 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.407 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.407 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.408 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.408 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:13.408 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:13.408 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:13.408 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:13.408 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:13.408 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:13.408 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.409 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.409 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.409 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.409 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.409 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.409 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:13.409 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:13.409 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.410 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.410 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.410 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.410 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.410 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.410 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.410 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.410 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.411 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.411 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.411 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.411 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:13.411 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:13.411 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.411 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.411 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.411 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:13.412 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.412 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.412 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.412 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:13.412 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:13.412 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:13.412 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:13.412 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:13.412 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:13.413 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.413 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.413 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.413 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.413 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.413 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.413 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:13.413 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:13.414 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.414 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.414 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.414 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.414 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.414 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.414 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.414 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.414 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.415 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.415 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.415 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:13.415 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:13.415 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.415 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:13.415 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "iostat", 2024-03-23T18:08:13.415 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:13.415 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:13.416 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:13.416 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:13.416 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:13.416 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.416 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.416 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.416 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.416 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.416 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.417 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.417 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:13.417 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:13.417 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:13.417 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:13.417 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:13.417 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:13.417 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.417 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.418 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.418 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.418 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.418 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.418 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:13.418 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:13.418 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.418 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.419 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.419 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.419 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.419 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.419 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.419 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.419 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.419 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.419 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.420 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.420 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:13.420 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:13.420 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.420 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.420 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.420 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:13.420 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.420 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.421 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.421 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:13.421 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:13.421 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:13.421 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:13.421 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:13.421 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:13.421 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.421 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.422 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.422 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.422 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.422 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.422 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:13.422 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:13.422 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.422 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.423 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.423 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.423 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.423 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.423 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.423 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.423 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.423 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.423 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.424 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:13.424 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:13.424 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.424 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:13.424 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "k8sevents", 2024-03-23T18:08:13.424 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:13.424 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:13.424 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:13.424 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_event_retention_days": { 2024-03-23T18:08:13.425 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ceph_event_retention_days", 2024-03-23T18:08:13.425 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:13.425 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.425 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.425 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "7", 2024-03-23T18:08:13.425 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.425 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.425 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.426 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-23T18:08:13.426 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.426 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.426 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.426 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.426 INFO:teuthology.orchestra.run.smithi086.stdout: "config_check_secs": { 2024-03-23T18:08:13.426 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "config_check_secs", 2024-03-23T18:08:13.426 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:13.426 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.427 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.427 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-03-23T18:08:13.427 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "10", 2024-03-23T18:08:13.427 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.427 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.427 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-23T18:08:13.427 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.427 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.428 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.428 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.428 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:13.428 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:13.428 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.428 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.428 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.428 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.428 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.429 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.429 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.429 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:13.429 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:13.429 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:13.429 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:13.429 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:13.429 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:13.430 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.430 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.430 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.430 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.430 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.430 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.430 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:13.430 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:13.430 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.431 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.431 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.431 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.431 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.431 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.431 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.431 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.431 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.432 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.432 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.432 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.432 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:13.432 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:13.432 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.432 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.432 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.433 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:13.433 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.433 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.433 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.433 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:13.433 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:13.433 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:13.433 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:13.434 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:13.434 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:13.434 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.434 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.434 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.434 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.434 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.434 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.435 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:13.435 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:13.435 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.435 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.435 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.435 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.435 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.435 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.435 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.436 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.436 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.436 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.436 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.436 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:13.436 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:13.436 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.436 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:13.436 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "localpool", 2024-03-23T18:08:13.437 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:13.437 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:13.437 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:13.437 INFO:teuthology.orchestra.run.smithi086.stdout: "failure_domain": { 2024-03-23T18:08:13.437 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "failure_domain", 2024-03-23T18:08:13.437 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.437 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.437 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.437 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "host", 2024-03-23T18:08:13.438 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.438 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.438 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.438 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "failure domain for any created local pool", 2024-03-23T18:08:13.438 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-23T18:08:13.438 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.438 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.438 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.439 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:13.439 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:13.439 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.439 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.439 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.439 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.439 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.439 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.439 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.440 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:13.440 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:13.440 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:13.440 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:13.440 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:13.440 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:13.440 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.440 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.441 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.441 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.441 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.441 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.441 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:13.441 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:13.441 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.441 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.441 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.442 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.442 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.442 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.442 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.442 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.442 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.442 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.442 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.443 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.443 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:13.443 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:13.443 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.443 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.443 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.443 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:13.443 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.444 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.444 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.444 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:13.444 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:13.444 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:13.444 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:13.444 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:13.444 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:13.445 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.445 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.445 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.445 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.445 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.445 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.445 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:13.445 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:13.445 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.446 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.446 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.446 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.446 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.446 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.446 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.446 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.446 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.447 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.447 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.447 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.447 INFO:teuthology.orchestra.run.smithi086.stdout: "min_size": { 2024-03-23T18:08:13.447 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "min_size", 2024-03-23T18:08:13.447 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:13.447 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.447 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.447 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.448 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.448 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.448 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.448 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "default min_size for any created local pool", 2024-03-23T18:08:13.448 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-23T18:08:13.448 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.448 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.448 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.448 INFO:teuthology.orchestra.run.smithi086.stdout: "num_rep": { 2024-03-23T18:08:13.449 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "num_rep", 2024-03-23T18:08:13.449 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:13.449 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.449 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.449 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "3", 2024-03-23T18:08:13.449 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.449 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.449 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.449 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "default replica count for any created local pool", 2024-03-23T18:08:13.450 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.450 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.450 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.450 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.450 INFO:teuthology.orchestra.run.smithi086.stdout: "pg_num": { 2024-03-23T18:08:13.450 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pg_num", 2024-03-23T18:08:13.450 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:13.450 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.451 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.451 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "128", 2024-03-23T18:08:13.451 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.451 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.451 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.451 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "default pg_num for any created local pool", 2024-03-23T18:08:13.451 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.451 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.451 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.452 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.452 INFO:teuthology.orchestra.run.smithi086.stdout: "prefix": { 2024-03-23T18:08:13.452 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "prefix", 2024-03-23T18:08:13.452 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.452 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.452 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.452 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.452 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.452 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.453 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.453 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "name prefix for any created local pool", 2024-03-23T18:08:13.453 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.453 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.453 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.453 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.453 INFO:teuthology.orchestra.run.smithi086.stdout: "subtree": { 2024-03-23T18:08:13.453 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "subtree", 2024-03-23T18:08:13.453 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.454 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.454 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.454 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "rack", 2024-03-23T18:08:13.454 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.454 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.454 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.454 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-23T18:08:13.455 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-23T18:08:13.455 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.455 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.455 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:13.455 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:13.455 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.455 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:13.455 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mds_autoscaler", 2024-03-23T18:08:13.455 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:13.456 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:13.456 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:13.456 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:13.456 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:13.456 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.456 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.456 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.456 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.456 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.457 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.457 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.457 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:13.457 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:13.457 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:13.457 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:13.457 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:13.457 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:13.457 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.458 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.458 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.458 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.458 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.458 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.458 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:13.458 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:13.458 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.459 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.459 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.459 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.459 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.459 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.459 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.459 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.459 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.460 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.460 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.460 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.460 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:13.460 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:13.460 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.460 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.460 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.460 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:13.461 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.461 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.461 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.461 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:13.461 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:13.461 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:13.461 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:13.461 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:13.461 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:13.462 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.462 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.462 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.462 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.462 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.462 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.462 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:13.462 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:13.463 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.463 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.463 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.463 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.463 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.463 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.463 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.463 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.464 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.464 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.464 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.464 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:13.464 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:13.464 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.464 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:13.464 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mirroring", 2024-03-23T18:08:13.464 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:13.465 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:13.465 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:13.465 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:13.465 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:13.465 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.465 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.465 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.465 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.465 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.466 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.466 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.466 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:13.466 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:13.466 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:13.466 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:13.466 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:13.466 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:13.467 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.467 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.467 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.467 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.467 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.467 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.467 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:13.467 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:13.467 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.468 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.468 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.468 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.468 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.468 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.468 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.468 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.468 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.469 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.469 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.469 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.469 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:13.469 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:13.469 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.469 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.469 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.469 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:13.470 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.470 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.470 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.470 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:13.470 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:13.470 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:13.470 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:13.470 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:13.471 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:13.471 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.471 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.471 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.471 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.471 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.471 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.471 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:13.471 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:13.472 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.472 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.472 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.472 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.472 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.472 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.472 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.472 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.472 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.473 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.473 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.473 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:13.473 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:13.473 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.473 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:13.473 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "nfs", 2024-03-23T18:08:13.473 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:13.473 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:13.474 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:13.474 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:13.474 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:13.474 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.474 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.474 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.474 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.474 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.475 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.475 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.475 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:13.475 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:13.475 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:13.475 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:13.475 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:13.475 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:13.475 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.476 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.476 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.476 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.476 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.476 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.476 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:13.476 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:13.476 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.476 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.476 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.477 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.477 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.477 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.477 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.477 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.477 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.477 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.477 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.478 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.478 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:13.478 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:13.478 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.478 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.478 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.478 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:13.478 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.478 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.479 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.479 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:13.479 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:13.479 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:13.479 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:13.479 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:13.479 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:13.479 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.479 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.480 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.480 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.480 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.480 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.480 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:13.480 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:13.480 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.480 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.481 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.481 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.481 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.481 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.481 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.481 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.481 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.481 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.481 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.482 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:13.482 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:13.482 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.482 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:13.482 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "orchestrator", 2024-03-23T18:08:13.482 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:13.482 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:13.482 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:13.483 INFO:teuthology.orchestra.run.smithi086.stdout: "fail_fs": { 2024-03-23T18:08:13.483 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "fail_fs", 2024-03-23T18:08:13.483 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.483 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.483 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.483 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.483 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.483 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.483 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.484 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-23T18:08:13.484 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.484 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.484 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.484 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.484 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:13.484 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:13.484 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.484 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.485 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.485 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.485 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.485 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.485 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.485 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:13.485 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:13.485 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:13.486 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:13.486 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:13.486 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:13.486 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.486 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.486 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.486 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.486 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.486 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.487 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:13.487 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:13.487 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.487 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.487 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.487 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.487 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.487 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.488 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.488 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.488 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.488 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.488 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.488 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.488 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:13.488 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:13.488 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.489 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.489 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.489 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:13.489 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.489 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.489 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.489 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:13.489 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:13.490 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:13.490 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:13.490 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:13.490 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:13.490 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.490 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.490 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.490 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.490 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.491 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.491 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:13.491 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:13.491 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.491 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.491 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.491 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.491 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.491 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.492 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.492 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.492 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.492 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.492 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.492 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.492 INFO:teuthology.orchestra.run.smithi086.stdout: "orchestrator": { 2024-03-23T18:08:13.492 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "orchestrator", 2024-03-23T18:08:13.492 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.493 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.493 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.493 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.493 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.493 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.493 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.493 INFO:teuthology.orchestra.run.smithi086.stdout: "cephadm", 2024-03-23T18:08:13.493 INFO:teuthology.orchestra.run.smithi086.stdout: "rook", 2024-03-23T18:08:13.493 INFO:teuthology.orchestra.run.smithi086.stdout: "test_orchestrator" 2024-03-23T18:08:13.494 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.494 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Orchestrator backend", 2024-03-23T18:08:13.494 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.494 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.494 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.494 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:13.494 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:13.494 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.495 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:13.495 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "osd_perf_query", 2024-03-23T18:08:13.495 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:13.495 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:13.495 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:13.495 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:13.495 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:13.495 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.495 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.496 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.496 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.496 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.496 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.496 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.496 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:13.496 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:13.496 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:13.496 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:13.497 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:13.497 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:13.497 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.497 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.497 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.497 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.497 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.497 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.497 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:13.498 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:13.498 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.498 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.498 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.498 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.498 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.498 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.498 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.498 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.499 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.499 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.499 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.499 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.499 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:13.499 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:13.499 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.500 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.500 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.500 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:13.500 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.500 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.500 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.500 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:13.500 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:13.500 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:13.501 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:13.501 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:13.501 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:13.501 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.501 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.501 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.501 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.501 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.501 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.502 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:13.502 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:13.502 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.502 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.502 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.502 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.502 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.502 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.503 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.503 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.503 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.503 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.503 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.503 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:13.503 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:13.503 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.503 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:13.504 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "osd_support", 2024-03-23T18:08:13.504 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:13.504 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:13.504 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:13.504 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:13.505 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:13.506 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.506 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.506 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.506 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.506 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.506 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.506 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.506 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:13.507 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:13.507 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:13.507 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:13.507 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:13.507 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:13.507 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.507 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.507 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.507 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.508 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.508 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.508 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:13.508 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:13.508 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.508 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.508 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.508 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.508 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.509 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.509 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.509 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.509 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.509 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.509 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.509 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.509 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:13.509 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:13.510 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.510 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.510 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.510 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:13.510 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.510 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.510 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.510 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:13.511 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:13.511 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:13.511 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:13.511 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:13.511 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:13.511 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.511 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.511 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.511 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.511 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.512 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.512 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:13.512 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:13.512 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.512 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.512 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.512 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.512 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.513 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.513 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.513 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.513 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.513 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.513 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.513 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:13.513 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:13.513 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.514 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:13.514 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pg_autoscaler", 2024-03-23T18:08:13.514 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:13.514 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:13.514 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:13.514 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:13.514 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:13.514 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.514 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.515 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.515 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.515 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.515 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.515 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.515 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:13.515 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:13.515 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:13.515 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:13.516 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:13.516 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:13.516 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.516 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.516 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.516 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.516 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.516 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.517 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:13.517 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:13.517 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.517 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.517 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.517 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.517 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.517 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.517 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.518 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.518 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.518 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.518 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.518 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.518 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:13.518 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:13.518 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.519 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.519 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.519 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:13.519 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.519 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.519 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.519 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:13.519 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:13.519 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:13.520 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:13.520 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:13.520 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:13.520 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.520 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.520 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.520 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.520 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.521 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.521 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:13.521 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:13.521 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.521 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.521 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.521 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.521 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.521 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.522 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.522 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.522 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.522 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.522 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.522 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.522 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-03-23T18:08:13.522 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-03-23T18:08:13.522 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-23T18:08:13.523 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.523 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.523 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-03-23T18:08:13.523 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.523 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.523 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.523 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.523 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.524 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.524 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.524 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.524 INFO:teuthology.orchestra.run.smithi086.stdout: "threshold": { 2024-03-23T18:08:13.524 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "threshold", 2024-03-23T18:08:13.524 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-03-23T18:08:13.524 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.524 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.524 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "3.0", 2024-03-23T18:08:13.525 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1.0", 2024-03-23T18:08:13.525 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.525 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.525 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "scaling threshold", 2024-03-23T18:08:13.525 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-23T18:08:13.525 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.525 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.525 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:13.525 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:13.526 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.526 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:13.526 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "progress", 2024-03-23T18:08:13.526 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:13.526 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:13.526 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:13.526 INFO:teuthology.orchestra.run.smithi086.stdout: "allow_pg_recovery_event": { 2024-03-23T18:08:13.526 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "allow_pg_recovery_event", 2024-03-23T18:08:13.527 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.527 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.527 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.527 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.527 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.527 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.527 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.527 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-23T18:08:13.527 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.528 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.528 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.528 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.528 INFO:teuthology.orchestra.run.smithi086.stdout: "enabled": { 2024-03-23T18:08:13.528 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enabled", 2024-03-23T18:08:13.528 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.528 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.528 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.528 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:13.529 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.529 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.529 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.529 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.529 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.529 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.529 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.529 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.530 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:13.530 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:13.530 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.530 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.530 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.530 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.530 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.530 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.530 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.531 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:13.531 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:13.531 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:13.531 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:13.531 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:13.531 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:13.531 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.531 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.532 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.532 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.532 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.532 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.532 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:13.532 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:13.532 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.532 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.532 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.533 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.533 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.533 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.533 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.533 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.533 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.533 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.533 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.533 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.534 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:13.534 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:13.534 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.534 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.534 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.534 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:13.534 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.534 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.534 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.535 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:13.535 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:13.535 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:13.535 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:13.535 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:13.535 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:13.535 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.535 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.536 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.536 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.536 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.536 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.536 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:13.536 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:13.536 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.536 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.537 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.537 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.537 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.537 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.537 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.537 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.537 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.537 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.538 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.538 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.538 INFO:teuthology.orchestra.run.smithi086.stdout: "max_completed_events": { 2024-03-23T18:08:13.538 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_completed_events", 2024-03-23T18:08:13.538 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:13.538 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.538 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.538 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "50", 2024-03-23T18:08:13.538 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.539 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.539 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.539 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "number of past completed events to remember", 2024-03-23T18:08:13.539 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.539 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.539 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.539 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.539 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-03-23T18:08:13.539 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-03-23T18:08:13.540 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-23T18:08:13.540 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.540 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.540 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-03-23T18:08:13.540 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.540 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.540 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.540 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how long the module is going to sleep", 2024-03-23T18:08:13.540 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.541 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.541 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.541 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:13.541 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:13.541 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.541 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:13.541 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "prometheus", 2024-03-23T18:08:13.541 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:13.541 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:13.542 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:13.542 INFO:teuthology.orchestra.run.smithi086.stdout: "cache": { 2024-03-23T18:08:13.542 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cache", 2024-03-23T18:08:13.542 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.542 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.542 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.542 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:13.542 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.543 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.543 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.543 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.543 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.543 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.543 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.543 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.543 INFO:teuthology.orchestra.run.smithi086.stdout: "exclude_perf_counters": { 2024-03-23T18:08:13.543 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "exclude_perf_counters", 2024-03-23T18:08:13.544 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.544 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.544 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.544 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:13.544 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.544 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.544 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.544 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-23T18:08:13.545 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-23T18:08:13.545 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.545 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.545 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.545 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:13.545 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:13.545 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.545 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.545 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.546 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.546 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.546 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.546 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.546 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:13.546 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:13.546 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:13.546 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:13.546 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:13.547 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:13.547 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.547 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.547 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.547 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.547 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.547 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.547 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:13.547 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:13.548 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.548 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.548 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.548 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.548 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.548 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.548 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.548 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.548 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.549 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.549 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.549 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.549 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:13.549 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:13.549 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.549 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.549 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.549 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:13.549 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.550 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.550 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.550 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:13.550 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:13.550 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:13.550 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:13.550 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:13.550 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:13.550 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.551 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.551 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.551 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.551 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.551 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.551 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:13.551 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:13.551 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.551 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.552 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.552 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.552 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.552 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.552 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.552 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.552 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.552 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.552 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.553 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.553 INFO:teuthology.orchestra.run.smithi086.stdout: "rbd_stats_pools": { 2024-03-23T18:08:13.553 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rbd_stats_pools", 2024-03-23T18:08:13.553 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.553 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.553 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.553 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.553 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.553 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.553 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.554 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.554 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.554 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.554 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.554 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.554 INFO:teuthology.orchestra.run.smithi086.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-23T18:08:13.554 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-23T18:08:13.554 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:13.555 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.555 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.555 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "300", 2024-03-23T18:08:13.555 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.555 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.555 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.555 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.555 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.555 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.555 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.556 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.556 INFO:teuthology.orchestra.run.smithi086.stdout: "scrape_interval": { 2024-03-23T18:08:13.556 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "scrape_interval", 2024-03-23T18:08:13.556 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-03-23T18:08:13.556 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.556 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.556 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "15.0", 2024-03-23T18:08:13.556 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.556 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.557 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.557 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.557 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.557 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.557 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.557 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.557 INFO:teuthology.orchestra.run.smithi086.stdout: "server_addr": { 2024-03-23T18:08:13.557 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_addr", 2024-03-23T18:08:13.558 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.558 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.558 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.558 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "::", 2024-03-23T18:08:13.558 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.558 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.558 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.558 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-23T18:08:13.558 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.559 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.559 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.559 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.559 INFO:teuthology.orchestra.run.smithi086.stdout: "server_port": { 2024-03-23T18:08:13.559 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_port", 2024-03-23T18:08:13.559 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:13.559 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.559 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.560 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "9283", 2024-03-23T18:08:13.560 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.560 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.560 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.560 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-23T18:08:13.560 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.560 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.560 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.560 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.561 INFO:teuthology.orchestra.run.smithi086.stdout: "stale_cache_strategy": { 2024-03-23T18:08:13.561 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "stale_cache_strategy", 2024-03-23T18:08:13.561 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.561 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.561 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.561 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "log", 2024-03-23T18:08:13.561 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.561 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.561 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.562 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.562 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.562 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.562 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.562 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.562 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_behaviour": { 2024-03-23T18:08:13.562 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_behaviour", 2024-03-23T18:08:13.562 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.563 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.563 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.563 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "default", 2024-03-23T18:08:13.563 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.563 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.563 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.563 INFO:teuthology.orchestra.run.smithi086.stdout: "default", 2024-03-23T18:08:13.563 INFO:teuthology.orchestra.run.smithi086.stdout: "error" 2024-03-23T18:08:13.563 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.564 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.564 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.564 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.564 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.564 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.564 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_error_status_code": { 2024-03-23T18:08:13.564 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_error_status_code", 2024-03-23T18:08:13.564 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:13.564 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.565 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.565 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "500", 2024-03-23T18:08:13.565 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "400", 2024-03-23T18:08:13.565 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "599", 2024-03-23T18:08:13.565 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.565 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.565 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.565 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.565 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.566 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:13.566 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:13.566 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.566 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:13.566 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rbd_support", 2024-03-23T18:08:13.566 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:13.566 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:13.566 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:13.566 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:13.567 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:13.567 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.567 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.567 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.567 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.567 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.567 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.567 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.568 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:13.568 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:13.568 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:13.568 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:13.568 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:13.568 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:13.568 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.568 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.568 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.569 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.569 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.569 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.569 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:13.569 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:13.569 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.569 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.569 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.570 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.570 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.570 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.570 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.570 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.570 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.570 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.570 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.570 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.571 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:13.571 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:13.571 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.571 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.571 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.571 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:13.571 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.571 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.572 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.572 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:13.572 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:13.572 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:13.572 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:13.572 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:13.572 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:13.572 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.573 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.573 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.573 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.573 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.573 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.573 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:13.573 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:13.573 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.573 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.574 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.574 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.574 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.574 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.574 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.574 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.574 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.574 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.574 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.575 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.575 INFO:teuthology.orchestra.run.smithi086.stdout: "max_concurrent_snap_create": { 2024-03-23T18:08:13.575 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_concurrent_snap_create", 2024-03-23T18:08:13.575 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:13.575 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.575 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.575 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-03-23T18:08:13.575 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.575 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.576 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.576 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.576 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.576 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.576 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.576 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.576 INFO:teuthology.orchestra.run.smithi086.stdout: "mirror_snapshot_schedule": { 2024-03-23T18:08:13.576 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mirror_snapshot_schedule", 2024-03-23T18:08:13.576 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.576 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.577 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.577 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.577 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.577 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.577 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.577 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.577 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.577 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.577 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.578 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.578 INFO:teuthology.orchestra.run.smithi086.stdout: "trash_purge_schedule": { 2024-03-23T18:08:13.578 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "trash_purge_schedule", 2024-03-23T18:08:13.578 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.578 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.578 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.578 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.578 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.578 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.579 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.579 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.579 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.579 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.579 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.579 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:13.579 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:13.579 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.579 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:13.580 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "restful", 2024-03-23T18:08:13.580 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:13.580 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:13.580 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:13.580 INFO:teuthology.orchestra.run.smithi086.stdout: "enable_auth": { 2024-03-23T18:08:13.580 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enable_auth", 2024-03-23T18:08:13.580 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.580 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.580 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.580 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:13.581 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.581 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.581 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.581 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.581 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.581 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.581 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.581 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.581 INFO:teuthology.orchestra.run.smithi086.stdout: "key_file": { 2024-03-23T18:08:13.581 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "key_file", 2024-03-23T18:08:13.582 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.582 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.582 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.582 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.582 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.582 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.582 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.582 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.583 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.583 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.583 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.583 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.583 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:13.583 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:13.583 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.583 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.583 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.584 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.584 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.584 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.584 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.584 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:13.584 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:13.584 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:13.584 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:13.584 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:13.585 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:13.585 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.585 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.585 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.585 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.585 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.585 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.585 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:13.585 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:13.586 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.586 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.586 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.586 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.586 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.586 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.586 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.586 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.587 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.587 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.587 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.587 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.587 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:13.587 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:13.587 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.588 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.588 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.588 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:13.588 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.588 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.588 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.588 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:13.588 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:13.589 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:13.589 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:13.589 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:13.589 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:13.589 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.589 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.589 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.589 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.590 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.590 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.590 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:13.590 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:13.590 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.590 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.590 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.590 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.590 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.591 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.591 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.591 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.591 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.591 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.591 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.591 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.591 INFO:teuthology.orchestra.run.smithi086.stdout: "server_addr": { 2024-03-23T18:08:13.592 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_addr", 2024-03-23T18:08:13.592 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.592 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.592 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.592 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.592 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.592 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.592 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.593 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.593 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.593 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.593 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.593 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.593 INFO:teuthology.orchestra.run.smithi086.stdout: "server_port": { 2024-03-23T18:08:13.593 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_port", 2024-03-23T18:08:13.593 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.593 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.594 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.594 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.594 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.594 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.594 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.594 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.594 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.594 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.594 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.595 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:13.595 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:13.595 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.595 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:13.595 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rgw", 2024-03-23T18:08:13.595 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:13.595 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:13.595 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:13.596 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:13.596 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:13.596 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.596 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.596 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.596 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.596 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.596 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.596 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.597 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:13.597 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:13.597 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:13.597 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:13.597 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:13.597 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:13.597 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.597 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.597 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.598 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.598 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.598 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.598 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:13.598 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:13.598 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.598 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.598 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.598 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.599 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.599 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.599 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.599 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.599 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.599 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.599 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.600 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.600 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:13.600 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:13.600 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.600 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.600 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.601 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:13.601 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.601 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.601 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.602 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:13.602 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:13.602 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:13.602 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:13.602 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:13.602 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:13.603 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.603 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.603 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.603 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.603 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.603 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.604 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:13.604 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:13.604 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.604 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.604 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.604 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.605 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.605 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.605 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.605 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.605 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.605 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.605 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.606 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:13.606 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:13.606 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.606 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:13.606 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rook", 2024-03-23T18:08:13.606 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:13.607 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:13.607 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:13.607 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:13.607 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:13.607 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.607 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.607 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.608 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.608 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.608 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.608 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.608 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:13.608 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:13.608 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:13.608 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:13.609 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:13.609 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:13.609 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.609 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.609 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.609 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.609 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.609 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.609 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:13.610 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:13.610 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.610 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.610 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.610 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.610 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.610 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.610 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.611 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.611 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.611 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.611 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.611 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.611 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:13.611 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:13.611 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.611 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.612 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.612 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:13.612 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.612 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.612 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.612 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:13.612 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:13.612 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:13.612 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:13.613 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:13.613 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:13.613 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.613 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.613 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.613 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.613 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.613 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.614 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:13.614 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:13.614 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.614 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.614 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.614 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.614 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.614 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.615 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.615 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.615 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.615 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.615 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.615 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.615 INFO:teuthology.orchestra.run.smithi086.stdout: "storage_class": { 2024-03-23T18:08:13.615 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "storage_class", 2024-03-23T18:08:13.615 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.616 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.616 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.616 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "local", 2024-03-23T18:08:13.616 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.616 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.616 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.616 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-23T18:08:13.616 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.616 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.617 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.617 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:13.617 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:13.617 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.617 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:13.617 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "selftest", 2024-03-23T18:08:13.617 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:13.617 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:13.618 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:13.618 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:13.618 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:13.618 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.618 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.618 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.618 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.618 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.619 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.619 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.619 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:13.619 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:13.619 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:13.619 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:13.619 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:13.619 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:13.619 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.620 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.620 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.620 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.620 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.620 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.620 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:13.620 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:13.620 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.621 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.621 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.621 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.621 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.621 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.621 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.621 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.621 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.621 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.622 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.622 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.622 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:13.622 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:13.622 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.622 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.622 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.622 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:13.622 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.623 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.623 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.623 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:13.623 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:13.623 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:13.623 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:13.623 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:13.623 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:13.623 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.624 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.624 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.624 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.624 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.624 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.624 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:13.624 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:13.624 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.625 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.625 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.625 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.625 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.625 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.625 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.625 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.625 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.625 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.626 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.626 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.626 INFO:teuthology.orchestra.run.smithi086.stdout: "roption1": { 2024-03-23T18:08:13.626 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "roption1", 2024-03-23T18:08:13.626 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.626 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.626 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.626 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.627 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.627 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.627 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.627 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.627 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.627 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.627 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.627 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.627 INFO:teuthology.orchestra.run.smithi086.stdout: "roption2": { 2024-03-23T18:08:13.628 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "roption2", 2024-03-23T18:08:13.628 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.628 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.628 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.628 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "xyz", 2024-03-23T18:08:13.628 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.628 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.628 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.629 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.629 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.629 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.629 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.629 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.629 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption1": { 2024-03-23T18:08:13.629 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption1", 2024-03-23T18:08:13.629 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.629 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.630 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.630 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.630 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.630 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.630 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.630 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.630 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.630 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.630 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.631 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.631 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption2": { 2024-03-23T18:08:13.631 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption2", 2024-03-23T18:08:13.631 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:13.631 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.631 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.631 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.631 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.632 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.632 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.632 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.632 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.632 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.632 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.632 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.632 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption3": { 2024-03-23T18:08:13.632 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption3", 2024-03-23T18:08:13.633 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-03-23T18:08:13.633 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.633 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.633 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.633 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.633 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.633 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.633 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.633 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.634 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.634 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.634 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.634 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption4": { 2024-03-23T18:08:13.634 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption4", 2024-03-23T18:08:13.634 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.634 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.634 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.635 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.635 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.635 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.635 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.635 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.635 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.635 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.635 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.635 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.636 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption5": { 2024-03-23T18:08:13.636 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption5", 2024-03-23T18:08:13.636 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.636 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.636 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.636 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.636 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.636 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.637 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.637 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.637 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.637 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.637 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.637 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.637 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption6": { 2024-03-23T18:08:13.637 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption6", 2024-03-23T18:08:13.638 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.638 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.638 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.638 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:13.638 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.638 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.638 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.638 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.638 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.639 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.639 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.639 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.639 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption7": { 2024-03-23T18:08:13.639 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption7", 2024-03-23T18:08:13.639 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:13.639 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.639 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.639 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.640 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-03-23T18:08:13.640 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "42", 2024-03-23T18:08:13.640 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.640 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.640 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.640 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.640 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.640 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.641 INFO:teuthology.orchestra.run.smithi086.stdout: "testkey": { 2024-03-23T18:08:13.641 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "testkey", 2024-03-23T18:08:13.641 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.641 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.641 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.641 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.641 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.641 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.641 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.642 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.642 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.642 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.642 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.642 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.642 INFO:teuthology.orchestra.run.smithi086.stdout: "testlkey": { 2024-03-23T18:08:13.642 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "testlkey", 2024-03-23T18:08:13.642 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.643 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.643 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.643 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.643 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.643 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.643 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.643 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.643 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.644 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.644 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.644 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.644 INFO:teuthology.orchestra.run.smithi086.stdout: "testnewline": { 2024-03-23T18:08:13.644 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "testnewline", 2024-03-23T18:08:13.644 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.644 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.644 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.644 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.645 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.645 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.645 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.645 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.645 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.645 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.645 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.645 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:13.646 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:13.646 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.646 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:13.646 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "snap_schedule", 2024-03-23T18:08:13.646 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:13.646 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:13.646 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:13.646 INFO:teuthology.orchestra.run.smithi086.stdout: "allow_m_granularity": { 2024-03-23T18:08:13.647 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "allow_m_granularity", 2024-03-23T18:08:13.647 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.647 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.647 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.647 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.647 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.647 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.647 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.648 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "allow minute scheduled snapshots", 2024-03-23T18:08:13.648 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.648 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.648 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.648 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.648 INFO:teuthology.orchestra.run.smithi086.stdout: "dump_on_update": { 2024-03-23T18:08:13.648 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "dump_on_update", 2024-03-23T18:08:13.648 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.648 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.649 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.649 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.649 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.649 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.649 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.649 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "dump database to debug log on update", 2024-03-23T18:08:13.649 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.649 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.650 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.650 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.650 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:13.650 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:13.650 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.650 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.650 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.650 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.651 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.651 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.651 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.651 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:13.651 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:13.651 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:13.651 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:13.651 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:13.651 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:13.652 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.652 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.652 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.652 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.652 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.652 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.652 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:13.652 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:13.652 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.653 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.653 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.653 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.653 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.653 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.653 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.653 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.653 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.653 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.654 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.654 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.654 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:13.654 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:13.654 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.654 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.654 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.654 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:13.654 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.655 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.655 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.655 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:13.655 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:13.655 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:13.655 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:13.655 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:13.655 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:13.655 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.656 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.656 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.656 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.656 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.656 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.656 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:13.656 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:13.656 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.656 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.657 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.657 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.657 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.657 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.657 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.657 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.657 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.657 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.657 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.658 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:13.658 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:13.658 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.658 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:13.658 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "stats", 2024-03-23T18:08:13.658 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:13.658 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:13.658 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:13.659 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:13.659 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:13.659 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.659 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.659 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.659 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.659 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.659 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.660 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.660 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:13.660 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:13.660 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:13.660 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:13.660 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:13.660 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:13.660 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.660 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.661 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.661 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.661 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.661 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.661 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:13.661 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:13.661 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.661 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.662 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.662 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.662 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.662 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.662 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.662 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.662 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.662 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.663 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.663 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.663 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:13.663 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:13.663 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.663 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.663 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.663 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:13.663 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.664 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.664 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.664 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:13.664 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:13.664 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:13.664 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:13.664 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:13.664 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:13.664 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.665 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.665 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.665 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.665 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.665 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.665 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:13.665 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:13.665 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.665 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.666 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.666 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.666 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.666 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.666 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.666 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.666 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.666 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.667 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.667 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:13.667 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:13.667 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.667 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:13.667 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "status", 2024-03-23T18:08:13.667 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:13.667 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:13.667 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:13.668 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:13.668 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:13.668 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.668 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.668 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.668 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.668 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.668 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.668 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.669 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:13.669 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:13.669 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:13.669 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:13.669 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:13.669 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:13.669 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.669 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.669 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.670 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.670 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.670 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.670 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:13.670 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:13.670 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.670 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.670 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.670 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.671 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.671 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.671 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.671 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.671 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.671 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.671 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.671 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.672 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:13.672 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:13.672 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.672 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.672 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.672 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:13.672 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.672 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.672 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.673 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:13.673 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:13.673 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:13.673 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:13.673 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:13.673 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:13.673 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.673 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.674 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.674 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.674 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.674 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.674 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:13.674 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:13.674 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.674 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.675 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.675 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.675 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.675 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.675 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.675 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.675 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.675 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.675 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.676 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:13.676 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:13.676 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.676 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:13.676 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "telegraf", 2024-03-23T18:08:13.676 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:13.676 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:13.676 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:13.676 INFO:teuthology.orchestra.run.smithi086.stdout: "address": { 2024-03-23T18:08:13.677 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "address", 2024-03-23T18:08:13.677 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.677 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.677 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.677 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-23T18:08:13.677 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.677 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.677 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.677 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.678 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.678 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.678 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.678 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.678 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-03-23T18:08:13.678 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-03-23T18:08:13.678 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-23T18:08:13.678 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.679 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.679 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "15", 2024-03-23T18:08:13.679 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.679 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.679 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.679 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.679 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.679 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.679 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.680 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.680 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:13.680 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:13.680 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.680 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.680 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.680 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.680 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.680 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.681 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.681 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:13.681 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:13.681 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:13.681 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:13.681 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:13.681 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:13.681 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.682 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.682 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.682 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.682 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.682 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.682 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:13.682 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:13.682 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.683 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.683 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.683 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.683 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.683 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.683 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.683 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.683 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.684 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.684 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.684 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.684 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:13.684 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:13.684 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.684 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.684 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.684 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:13.685 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.685 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.685 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.685 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:13.685 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:13.685 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:13.685 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:13.685 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:13.686 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:13.686 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.686 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.686 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.686 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.686 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.686 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.686 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:13.687 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:13.687 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.687 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.687 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.687 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.687 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.687 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.687 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.688 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.688 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.688 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.688 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.688 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:13.688 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:13.688 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.688 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:13.688 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "telemetry", 2024-03-23T18:08:13.689 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:13.689 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:13.689 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:13.689 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_basic": { 2024-03-23T18:08:13.689 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_basic", 2024-03-23T18:08:13.689 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.689 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.689 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.690 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:13.690 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.690 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.690 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.690 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-23T18:08:13.690 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.690 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.690 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.690 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.691 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_crash": { 2024-03-23T18:08:13.691 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_crash", 2024-03-23T18:08:13.691 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.691 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.691 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.691 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:13.691 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.691 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.691 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.692 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-23T18:08:13.692 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.692 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.692 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.692 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.692 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_device": { 2024-03-23T18:08:13.692 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_device", 2024-03-23T18:08:13.692 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.692 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.693 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.693 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:13.693 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.693 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.693 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.693 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-23T18:08:13.693 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.693 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.694 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.694 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.694 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_ident": { 2024-03-23T18:08:13.694 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_ident", 2024-03-23T18:08:13.694 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.694 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.694 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.694 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.695 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.695 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.695 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.695 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-23T18:08:13.695 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.695 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.695 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.695 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.695 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_perf": { 2024-03-23T18:08:13.696 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_perf", 2024-03-23T18:08:13.696 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.696 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.696 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.696 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.696 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.696 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.696 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.697 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-23T18:08:13.697 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.697 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.697 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.697 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.697 INFO:teuthology.orchestra.run.smithi086.stdout: "contact": { 2024-03-23T18:08:13.697 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "contact", 2024-03-23T18:08:13.697 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.697 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.698 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.698 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.698 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.698 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.698 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.698 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.698 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.698 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.698 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.699 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.699 INFO:teuthology.orchestra.run.smithi086.stdout: "description": { 2024-03-23T18:08:13.699 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "description", 2024-03-23T18:08:13.699 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.699 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.699 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.699 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.699 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.700 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.700 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.700 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.700 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.700 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.700 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.700 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.700 INFO:teuthology.orchestra.run.smithi086.stdout: "device_url": { 2024-03-23T18:08:13.700 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "device_url", 2024-03-23T18:08:13.701 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.701 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.701 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.701 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-23T18:08:13.701 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.701 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.701 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.701 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.701 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.702 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.702 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.702 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.702 INFO:teuthology.orchestra.run.smithi086.stdout: "enabled": { 2024-03-23T18:08:13.702 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enabled", 2024-03-23T18:08:13.702 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.702 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.702 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.703 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.703 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.703 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.703 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.703 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.703 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.703 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.703 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.703 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.704 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-03-23T18:08:13.704 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-03-23T18:08:13.704 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:13.704 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.704 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.704 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "24", 2024-03-23T18:08:13.704 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "8", 2024-03-23T18:08:13.705 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.705 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.705 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.705 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.705 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.705 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.705 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.705 INFO:teuthology.orchestra.run.smithi086.stdout: "last_opt_revision": { 2024-03-23T18:08:13.705 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "last_opt_revision", 2024-03-23T18:08:13.706 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:13.706 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.706 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.706 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "1", 2024-03-23T18:08:13.706 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.706 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.706 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.706 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.707 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.707 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.707 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.707 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.707 INFO:teuthology.orchestra.run.smithi086.stdout: "leaderboard": { 2024-03-23T18:08:13.707 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "leaderboard", 2024-03-23T18:08:13.707 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.707 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.707 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.708 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.708 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.708 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.708 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.708 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.708 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.708 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.708 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.708 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.709 INFO:teuthology.orchestra.run.smithi086.stdout: "leaderboard_description": { 2024-03-23T18:08:13.709 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "leaderboard_description", 2024-03-23T18:08:13.709 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.709 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.709 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.709 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.709 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.709 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.709 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.710 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.710 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.710 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.710 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.710 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.710 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:13.710 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:13.710 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.711 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.711 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.711 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.711 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.711 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.711 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.711 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:13.711 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:13.711 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:13.712 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:13.712 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:13.712 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:13.712 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.712 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.712 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.712 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.712 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.712 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.713 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:13.713 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:13.713 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.713 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.713 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.713 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.713 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.713 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.713 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.714 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.714 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.714 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.714 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.714 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.714 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:13.714 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:13.714 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.715 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.715 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.715 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:13.715 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.715 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.715 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.715 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:13.715 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:13.715 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:13.716 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:13.716 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:13.716 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:13.716 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.716 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.716 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.716 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.716 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.717 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.717 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:13.717 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:13.717 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.717 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.717 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.717 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.717 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.717 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.717 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.718 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.718 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.718 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.718 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.718 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.718 INFO:teuthology.orchestra.run.smithi086.stdout: "organization": { 2024-03-23T18:08:13.718 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "organization", 2024-03-23T18:08:13.718 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.719 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.719 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.719 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.719 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.719 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.719 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.719 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.719 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.719 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.719 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.720 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.720 INFO:teuthology.orchestra.run.smithi086.stdout: "proxy": { 2024-03-23T18:08:13.720 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "proxy", 2024-03-23T18:08:13.720 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.720 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.720 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.720 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.720 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.720 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.720 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.721 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.721 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.721 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.721 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.721 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.721 INFO:teuthology.orchestra.run.smithi086.stdout: "url": { 2024-03-23T18:08:13.721 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "url", 2024-03-23T18:08:13.721 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.722 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.722 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.722 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-23T18:08:13.722 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.722 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.722 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.722 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.722 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.722 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.723 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.723 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:13.723 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:13.723 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.723 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:13.723 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "test_orchestrator", 2024-03-23T18:08:13.723 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:13.723 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:13.723 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:13.724 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:13.724 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:13.724 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.724 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.724 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.724 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.724 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.724 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.724 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.724 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:13.725 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:13.725 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:13.725 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:13.725 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:13.725 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:13.725 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.725 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.725 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.726 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.726 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.726 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.726 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:13.726 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:13.726 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.726 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.727 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.727 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.727 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.727 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.727 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.727 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.727 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.727 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.727 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.728 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.728 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:13.728 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:13.728 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.728 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.728 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.728 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:13.728 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.728 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.729 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.729 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:13.729 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:13.729 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:13.729 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:13.729 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:13.729 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:13.729 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.730 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.730 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.730 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.730 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.730 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.730 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:13.730 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:13.730 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.730 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.731 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.731 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.731 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.731 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.731 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.731 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.731 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.731 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.731 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.732 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:13.732 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:13.732 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.732 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:13.732 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "volumes", 2024-03-23T18:08:13.732 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:13.732 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:13.732 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:13.732 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:13.733 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:13.733 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.733 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.733 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.733 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.733 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.733 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.733 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.734 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:13.734 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:13.734 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:13.734 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:13.734 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:13.734 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:13.734 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.734 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.735 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.735 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.735 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.735 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.735 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:13.735 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:13.735 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.735 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.736 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.736 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.736 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.736 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.736 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.736 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.736 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.736 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.736 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.737 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.737 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:13.737 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:13.737 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.737 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.737 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.737 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:13.737 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.737 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.738 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.738 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:13.738 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:13.738 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:13.738 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:13.738 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:13.738 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:13.738 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.739 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.739 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.739 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.739 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.739 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.739 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:13.739 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:13.739 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.739 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.740 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.740 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.740 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.740 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.740 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.740 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.740 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.740 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.740 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.741 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.741 INFO:teuthology.orchestra.run.smithi086.stdout: "max_concurrent_clones": { 2024-03-23T18:08:13.741 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_concurrent_clones", 2024-03-23T18:08:13.741 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:13.741 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.741 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.741 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "4", 2024-03-23T18:08:13.741 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.742 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.742 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.742 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-23T18:08:13.742 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.742 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.742 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.742 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.742 INFO:teuthology.orchestra.run.smithi086.stdout: "snapshot_clone_delay": { 2024-03-23T18:08:13.742 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "snapshot_clone_delay", 2024-03-23T18:08:13.743 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:13.743 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.743 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.743 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-03-23T18:08:13.743 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.743 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.743 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.743 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-23T18:08:13.743 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.744 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.744 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.744 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.744 INFO:teuthology.orchestra.run.smithi086.stdout: "snapshot_clone_no_wait": { 2024-03-23T18:08:13.744 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "snapshot_clone_no_wait", 2024-03-23T18:08:13.744 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.744 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.744 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.745 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:13.745 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.745 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.745 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.745 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-23T18:08:13.745 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.745 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.745 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.745 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:13.746 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:13.746 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.746 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:13.746 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix", 2024-03-23T18:08:13.746 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:13.746 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:13.746 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:13.746 INFO:teuthology.orchestra.run.smithi086.stdout: "discovery_interval": { 2024-03-23T18:08:13.747 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "discovery_interval", 2024-03-23T18:08:13.747 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-03-23T18:08:13.747 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.747 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.747 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "100", 2024-03-23T18:08:13.747 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.747 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.747 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.747 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.748 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.748 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.748 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.748 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.748 INFO:teuthology.orchestra.run.smithi086.stdout: "identifier": { 2024-03-23T18:08:13.748 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "identifier", 2024-03-23T18:08:13.748 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.748 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.749 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.749 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.749 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.749 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.749 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.749 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.749 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.749 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.749 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.750 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.750 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-03-23T18:08:13.750 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-03-23T18:08:13.750 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-23T18:08:13.750 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.750 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.750 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-03-23T18:08:13.750 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.750 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.751 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.751 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.751 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.751 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.751 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.751 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.751 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:13.751 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:13.751 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.752 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.752 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.752 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.752 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.752 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.752 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.752 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:13.752 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:13.752 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:13.753 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:13.753 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:13.753 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:13.753 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.753 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.753 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.753 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.753 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.753 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.754 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:13.754 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:13.754 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.754 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.754 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.754 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.754 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.754 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.755 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.755 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.755 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.755 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.755 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.755 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.755 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:13.755 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:13.755 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.756 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.756 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.756 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:13.756 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.756 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.756 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.756 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:13.756 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:13.757 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:13.757 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:13.757 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:13.757 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:13.757 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.757 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.757 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.757 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.757 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.758 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.758 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:13.758 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:13.758 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.758 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.758 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.758 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.758 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.758 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.759 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.759 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.759 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.759 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.759 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.759 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.759 INFO:teuthology.orchestra.run.smithi086.stdout: "zabbix_host": { 2024-03-23T18:08:13.759 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix_host", 2024-03-23T18:08:13.759 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.760 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.760 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.760 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.760 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.760 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.760 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.760 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.760 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.761 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.761 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.761 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.761 INFO:teuthology.orchestra.run.smithi086.stdout: "zabbix_port": { 2024-03-23T18:08:13.761 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix_port", 2024-03-23T18:08:13.761 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:13.761 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.761 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.761 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10051", 2024-03-23T18:08:13.762 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.762 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.762 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.762 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.762 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.762 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.762 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.762 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.763 INFO:teuthology.orchestra.run.smithi086.stdout: "zabbix_sender": { 2024-03-23T18:08:13.763 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix_sender", 2024-03-23T18:08:13.763 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.763 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.763 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.763 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-23T18:08:13.763 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.763 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.763 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.764 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.764 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.764 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.764 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.764 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:13.764 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:13.764 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:13.764 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-03-23T18:08:13.764 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.765 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:13.765 INFO:teuthology.orchestra.run.smithi086.stdout: "gid": 25412, 2024-03-23T18:08:13.765 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "z", 2024-03-23T18:08:13.765 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr_features": 4540138322906710015, 2024-03-23T18:08:13.765 INFO:teuthology.orchestra.run.smithi086.stdout: "available_modules": [ 2024-03-23T18:08:13.765 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:13.765 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "alerts", 2024-03-23T18:08:13.765 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:13.766 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:13.766 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:13.766 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-03-23T18:08:13.766 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-03-23T18:08:13.766 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-23T18:08:13.766 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.766 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.766 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-03-23T18:08:13.766 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.767 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.767 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.767 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How frequently to reexamine health status", 2024-03-23T18:08:13.767 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.767 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.767 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.767 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.767 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:13.767 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:13.768 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.768 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.768 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.768 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.768 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.768 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.768 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.768 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:13.769 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:13.769 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:13.769 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:13.769 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:13.769 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:13.769 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.769 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.769 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.770 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.770 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.770 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.770 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:13.770 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:13.771 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.771 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.771 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.771 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.771 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.772 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.772 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.772 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.772 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.772 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.773 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.773 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.773 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:13.773 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:13.773 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.774 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.774 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.774 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:13.774 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.775 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.775 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.775 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:13.775 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:13.775 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:13.776 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:13.776 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:13.776 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:13.776 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.776 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.776 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.777 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.777 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.777 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.777 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:13.777 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:13.777 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.777 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.777 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.777 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.778 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.778 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.778 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.778 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.778 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.778 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.778 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.778 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.778 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_destination": { 2024-03-23T18:08:13.779 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_destination", 2024-03-23T18:08:13.779 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.779 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.779 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.779 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.779 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.779 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.779 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.780 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Email address to send alerts to", 2024-03-23T18:08:13.780 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.780 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.780 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.780 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.780 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_from_name": { 2024-03-23T18:08:13.780 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_from_name", 2024-03-23T18:08:13.780 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.780 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.781 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.781 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "Ceph", 2024-03-23T18:08:13.781 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.781 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.781 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.781 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Email From: name", 2024-03-23T18:08:13.781 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.781 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.782 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.782 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.782 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_host": { 2024-03-23T18:08:13.782 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_host", 2024-03-23T18:08:13.782 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.782 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.782 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.782 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.783 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.783 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.783 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.783 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SMTP server", 2024-03-23T18:08:13.783 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.783 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.783 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.783 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.783 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_password": { 2024-03-23T18:08:13.784 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_password", 2024-03-23T18:08:13.784 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.784 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.784 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.784 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.784 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.784 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.784 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.785 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Password to authenticate with", 2024-03-23T18:08:13.785 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.785 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.785 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.785 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.785 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_port": { 2024-03-23T18:08:13.785 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_port", 2024-03-23T18:08:13.785 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:13.785 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.786 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.786 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "465", 2024-03-23T18:08:13.786 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.786 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.786 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.786 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SMTP port", 2024-03-23T18:08:13.786 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.786 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.787 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.787 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.787 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_sender": { 2024-03-23T18:08:13.787 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_sender", 2024-03-23T18:08:13.787 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.787 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.787 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.787 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.788 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.788 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.788 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.788 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SMTP envelope sender", 2024-03-23T18:08:13.788 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.788 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.788 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.788 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.789 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_ssl": { 2024-03-23T18:08:13.789 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_ssl", 2024-03-23T18:08:13.789 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.789 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.789 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.789 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:13.789 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.789 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.789 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.790 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-23T18:08:13.790 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.790 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.790 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.790 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.790 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_user": { 2024-03-23T18:08:13.790 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_user", 2024-03-23T18:08:13.790 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.791 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.791 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.791 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.791 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.791 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.791 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.791 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "User to authenticate as", 2024-03-23T18:08:13.791 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.791 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.792 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.792 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:13.792 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:13.792 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.792 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:13.792 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "balancer", 2024-03-23T18:08:13.792 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:13.792 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:13.793 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:13.793 INFO:teuthology.orchestra.run.smithi086.stdout: "active": { 2024-03-23T18:08:13.793 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "active", 2024-03-23T18:08:13.793 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.793 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.793 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.793 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:13.793 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.793 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.794 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.794 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "automatically balance PGs across cluster", 2024-03-23T18:08:13.794 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.794 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.794 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.794 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.794 INFO:teuthology.orchestra.run.smithi086.stdout: "begin_time": { 2024-03-23T18:08:13.794 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "begin_time", 2024-03-23T18:08:13.794 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.795 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.795 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.795 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0000", 2024-03-23T18:08:13.795 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.795 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.795 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.795 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "beginning time of day to automatically balance", 2024-03-23T18:08:13.795 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-23T18:08:13.796 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.796 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.796 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.796 INFO:teuthology.orchestra.run.smithi086.stdout: "begin_weekday": { 2024-03-23T18:08:13.796 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "begin_weekday", 2024-03-23T18:08:13.796 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-03-23T18:08:13.796 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.796 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.797 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-03-23T18:08:13.797 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "0", 2024-03-23T18:08:13.797 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "6", 2024-03-23T18:08:13.797 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.797 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-23T18:08:13.797 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-23T18:08:13.797 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.797 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.797 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.798 INFO:teuthology.orchestra.run.smithi086.stdout: "crush_compat_max_iterations": { 2024-03-23T18:08:13.798 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crush_compat_max_iterations", 2024-03-23T18:08:13.798 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-03-23T18:08:13.798 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.798 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.798 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "25", 2024-03-23T18:08:13.798 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-03-23T18:08:13.798 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "250", 2024-03-23T18:08:13.799 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.799 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-23T18:08:13.799 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.799 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.799 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.799 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.799 INFO:teuthology.orchestra.run.smithi086.stdout: "crush_compat_metrics": { 2024-03-23T18:08:13.799 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crush_compat_metrics", 2024-03-23T18:08:13.800 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.800 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.800 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.800 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "pgs,objects,bytes", 2024-03-23T18:08:13.800 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.800 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.800 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.800 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-23T18:08:13.800 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-23T18:08:13.801 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.801 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.801 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.801 INFO:teuthology.orchestra.run.smithi086.stdout: "crush_compat_step": { 2024-03-23T18:08:13.801 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crush_compat_step", 2024-03-23T18:08:13.801 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-03-23T18:08:13.801 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.801 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.801 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0.5", 2024-03-23T18:08:13.802 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "0.001", 2024-03-23T18:08:13.802 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "0.999", 2024-03-23T18:08:13.802 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.802 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "aggressiveness of optimization", 2024-03-23T18:08:13.802 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-23T18:08:13.802 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.802 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.802 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.803 INFO:teuthology.orchestra.run.smithi086.stdout: "end_time": { 2024-03-23T18:08:13.803 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "end_time", 2024-03-23T18:08:13.803 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.803 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.803 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.803 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "2359", 2024-03-23T18:08:13.803 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.803 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.803 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.804 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "ending time of day to automatically balance", 2024-03-23T18:08:13.804 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-23T18:08:13.804 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.804 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.804 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.804 INFO:teuthology.orchestra.run.smithi086.stdout: "end_weekday": { 2024-03-23T18:08:13.804 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "end_weekday", 2024-03-23T18:08:13.804 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-03-23T18:08:13.804 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.805 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.805 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-03-23T18:08:13.805 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "0", 2024-03-23T18:08:13.805 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "6", 2024-03-23T18:08:13.805 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.805 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-23T18:08:13.805 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-23T18:08:13.805 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.805 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.806 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.806 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:13.806 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:13.806 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.806 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.806 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.806 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.806 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.806 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.807 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.807 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:13.807 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:13.807 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:13.807 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:13.807 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:13.807 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:13.807 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.808 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.808 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.808 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.808 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.808 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.808 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:13.808 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:13.808 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.809 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.809 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.809 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.809 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.809 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.809 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.809 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.809 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.810 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.810 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.810 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.810 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:13.810 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:13.810 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.810 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.810 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.810 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:13.811 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.811 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.811 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.811 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:13.811 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:13.811 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:13.811 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:13.811 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:13.811 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:13.812 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.812 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.812 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.812 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.812 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.812 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.812 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:13.812 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:13.812 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.813 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.813 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.813 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.813 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.813 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.813 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.813 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.813 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.813 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.814 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.814 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.814 INFO:teuthology.orchestra.run.smithi086.stdout: "min_score": { 2024-03-23T18:08:13.814 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "min_score", 2024-03-23T18:08:13.814 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-03-23T18:08:13.814 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.814 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.814 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-03-23T18:08:13.815 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.815 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.815 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.815 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-23T18:08:13.815 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.815 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.815 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.815 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.815 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": { 2024-03-23T18:08:13.816 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mode", 2024-03-23T18:08:13.816 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.816 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.816 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.816 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "upmap", 2024-03-23T18:08:13.816 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.816 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.816 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.817 INFO:teuthology.orchestra.run.smithi086.stdout: "crush-compat", 2024-03-23T18:08:13.817 INFO:teuthology.orchestra.run.smithi086.stdout: "none", 2024-03-23T18:08:13.817 INFO:teuthology.orchestra.run.smithi086.stdout: "upmap" 2024-03-23T18:08:13.817 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.817 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Balancer mode", 2024-03-23T18:08:13.817 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.817 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.817 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.817 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.818 INFO:teuthology.orchestra.run.smithi086.stdout: "pool_ids": { 2024-03-23T18:08:13.818 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pool_ids", 2024-03-23T18:08:13.818 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.818 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.818 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.818 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.818 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.818 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.819 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.819 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-23T18:08:13.819 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.819 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.819 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.819 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.819 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-03-23T18:08:13.819 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-03-23T18:08:13.819 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-23T18:08:13.820 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.820 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.820 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-03-23T18:08:13.820 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.820 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.820 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.820 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-23T18:08:13.820 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.820 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.821 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.821 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.821 INFO:teuthology.orchestra.run.smithi086.stdout: "upmap_max_deviation": { 2024-03-23T18:08:13.821 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "upmap_max_deviation", 2024-03-23T18:08:13.821 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:13.821 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.821 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.821 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-03-23T18:08:13.821 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-03-23T18:08:13.822 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.822 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.822 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-23T18:08:13.822 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-23T18:08:13.822 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.822 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.822 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.822 INFO:teuthology.orchestra.run.smithi086.stdout: "upmap_max_optimizations": { 2024-03-23T18:08:13.823 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "upmap_max_optimizations", 2024-03-23T18:08:13.823 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-03-23T18:08:13.823 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.823 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.823 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-03-23T18:08:13.823 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.823 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.823 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.823 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-23T18:08:13.824 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.824 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.824 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.824 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:13.824 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:13.824 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.824 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:13.824 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cephadm", 2024-03-23T18:08:13.824 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:13.825 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:13.825 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:13.825 INFO:teuthology.orchestra.run.smithi086.stdout: "agent_down_multiplier": { 2024-03-23T18:08:13.825 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "agent_down_multiplier", 2024-03-23T18:08:13.825 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-03-23T18:08:13.825 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.825 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.825 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "3.0", 2024-03-23T18:08:13.825 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.826 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.826 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.826 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-23T18:08:13.826 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.826 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.826 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.826 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.826 INFO:teuthology.orchestra.run.smithi086.stdout: "agent_refresh_rate": { 2024-03-23T18:08:13.827 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "agent_refresh_rate", 2024-03-23T18:08:13.827 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-23T18:08:13.827 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.827 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.827 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "20", 2024-03-23T18:08:13.827 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.827 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.827 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.827 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-23T18:08:13.828 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.828 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.828 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.828 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.828 INFO:teuthology.orchestra.run.smithi086.stdout: "agent_starting_port": { 2024-03-23T18:08:13.828 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "agent_starting_port", 2024-03-23T18:08:13.828 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:13.828 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.829 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.829 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "4721", 2024-03-23T18:08:13.829 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.829 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.829 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.829 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-23T18:08:13.829 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.829 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.829 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.830 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.830 INFO:teuthology.orchestra.run.smithi086.stdout: "allow_ptrace": { 2024-03-23T18:08:13.830 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "allow_ptrace", 2024-03-23T18:08:13.830 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.830 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.830 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.830 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.830 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.831 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.831 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.831 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-23T18:08:13.831 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-23T18:08:13.831 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.831 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.831 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.831 INFO:teuthology.orchestra.run.smithi086.stdout: "autotune_interval": { 2024-03-23T18:08:13.832 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "autotune_interval", 2024-03-23T18:08:13.832 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-23T18:08:13.832 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.832 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.832 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-03-23T18:08:13.832 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.832 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.832 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.832 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-23T18:08:13.833 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.833 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.833 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.833 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.833 INFO:teuthology.orchestra.run.smithi086.stdout: "autotune_memory_target_ratio": { 2024-03-23T18:08:13.833 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "autotune_memory_target_ratio", 2024-03-23T18:08:13.833 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-03-23T18:08:13.833 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.833 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.834 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0.7", 2024-03-23T18:08:13.834 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.834 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.834 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.834 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-23T18:08:13.834 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.834 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.834 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.835 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.835 INFO:teuthology.orchestra.run.smithi086.stdout: "cgroups_split": { 2024-03-23T18:08:13.835 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cgroups_split", 2024-03-23T18:08:13.835 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.835 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.835 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.835 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:13.835 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.835 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.836 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.836 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-23T18:08:13.836 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.836 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.836 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.836 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.836 INFO:teuthology.orchestra.run.smithi086.stdout: "config_checks_enabled": { 2024-03-23T18:08:13.836 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "config_checks_enabled", 2024-03-23T18:08:13.837 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.837 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.837 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.837 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.837 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.837 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.837 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.837 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-23T18:08:13.838 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.838 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.838 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.838 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.838 INFO:teuthology.orchestra.run.smithi086.stdout: "config_dashboard": { 2024-03-23T18:08:13.838 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "config_dashboard", 2024-03-23T18:08:13.838 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.838 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.839 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.839 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:13.839 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.839 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.839 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.839 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-23T18:08:13.839 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.839 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.839 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.840 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.840 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_alertmanager": { 2024-03-23T18:08:13.840 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_alertmanager", 2024-03-23T18:08:13.840 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.840 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.840 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.840 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-23T18:08:13.840 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.840 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.841 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.841 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-03-23T18:08:13.841 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.841 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.841 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.841 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.841 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_base": { 2024-03-23T18:08:13.841 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_base", 2024-03-23T18:08:13.842 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.842 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.842 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.842 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-23T18:08:13.842 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.842 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.842 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.842 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Container image name, without the tag", 2024-03-23T18:08:13.843 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.843 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.843 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.843 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.843 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_elasticsearch": { 2024-03-23T18:08:13.843 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_elasticsearch", 2024-03-23T18:08:13.843 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.843 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.843 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.844 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-23T18:08:13.844 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.844 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.844 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.844 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "elasticsearch container image", 2024-03-23T18:08:13.844 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.844 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.844 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.845 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.845 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_grafana": { 2024-03-23T18:08:13.845 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_grafana", 2024-03-23T18:08:13.845 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.845 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.845 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.845 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2024-03-23T18:08:13.845 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.845 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.846 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.846 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-03-23T18:08:13.846 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.846 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.846 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.846 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.846 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_haproxy": { 2024-03-23T18:08:13.846 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_haproxy", 2024-03-23T18:08:13.847 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.847 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.847 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.847 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-23T18:08:13.847 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.847 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.847 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.847 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "HAproxy container image", 2024-03-23T18:08:13.847 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.848 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.848 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.848 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.848 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_jaeger_agent": { 2024-03-23T18:08:13.848 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_jaeger_agent", 2024-03-23T18:08:13.848 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.848 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.848 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.849 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-23T18:08:13.849 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.849 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.849 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.849 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Jaeger agent container image", 2024-03-23T18:08:13.849 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.849 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.849 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.849 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.850 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_jaeger_collector": { 2024-03-23T18:08:13.850 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_jaeger_collector", 2024-03-23T18:08:13.850 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.850 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.850 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.850 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-23T18:08:13.850 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.850 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.850 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.851 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Jaeger collector container image", 2024-03-23T18:08:13.851 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.851 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.851 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.851 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.851 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_jaeger_query": { 2024-03-23T18:08:13.851 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_jaeger_query", 2024-03-23T18:08:13.851 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.852 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.852 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.852 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-23T18:08:13.852 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.852 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.852 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.852 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Jaeger query container image", 2024-03-23T18:08:13.852 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.852 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.853 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.853 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.853 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_keepalived": { 2024-03-23T18:08:13.853 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_keepalived", 2024-03-23T18:08:13.853 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.853 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.853 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.853 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-23T18:08:13.854 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.854 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.854 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.854 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Keepalived container image", 2024-03-23T18:08:13.854 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.854 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.854 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.854 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.855 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_loki": { 2024-03-23T18:08:13.855 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_loki", 2024-03-23T18:08:13.855 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.855 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.855 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.855 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-23T18:08:13.855 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.856 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.856 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.856 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Loki container image", 2024-03-23T18:08:13.856 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.856 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.856 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.856 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.856 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_node_exporter": { 2024-03-23T18:08:13.857 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_node_exporter", 2024-03-23T18:08:13.857 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.857 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.857 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.857 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-23T18:08:13.857 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.857 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.857 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.857 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-03-23T18:08:13.858 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.858 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.858 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.858 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.858 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_nvmeof": { 2024-03-23T18:08:13.858 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_nvmeof", 2024-03-23T18:08:13.858 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.858 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.859 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.859 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-23T18:08:13.859 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.859 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.859 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.859 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Nvme-of container image", 2024-03-23T18:08:13.859 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.859 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.860 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.860 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.860 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_prometheus": { 2024-03-23T18:08:13.860 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_prometheus", 2024-03-23T18:08:13.860 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.860 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.860 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.860 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-23T18:08:13.860 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.861 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.861 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.861 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-03-23T18:08:13.861 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.861 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.861 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.861 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.861 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_promtail": { 2024-03-23T18:08:13.862 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_promtail", 2024-03-23T18:08:13.862 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.862 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.862 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.862 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-23T18:08:13.862 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.862 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.862 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.863 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Promtail container image", 2024-03-23T18:08:13.863 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.863 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.863 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.863 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.863 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_snmp_gateway": { 2024-03-23T18:08:13.863 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_snmp_gateway", 2024-03-23T18:08:13.863 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.864 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.864 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.864 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-23T18:08:13.864 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.864 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.864 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.864 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SNMP Gateway container image", 2024-03-23T18:08:13.864 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.865 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.865 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.865 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.865 INFO:teuthology.orchestra.run.smithi086.stdout: "container_init": { 2024-03-23T18:08:13.865 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_init", 2024-03-23T18:08:13.865 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.865 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.865 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.866 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:13.866 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.866 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.866 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.866 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Run podman/docker with `--init`", 2024-03-23T18:08:13.866 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.866 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.866 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.867 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.867 INFO:teuthology.orchestra.run.smithi086.stdout: "daemon_cache_timeout": { 2024-03-23T18:08:13.867 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "daemon_cache_timeout", 2024-03-23T18:08:13.867 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-23T18:08:13.867 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.867 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.867 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-03-23T18:08:13.867 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.868 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.868 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.868 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-23T18:08:13.868 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.868 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.868 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.868 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.868 INFO:teuthology.orchestra.run.smithi086.stdout: "default_cephadm_command_timeout": { 2024-03-23T18:08:13.868 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "default_cephadm_command_timeout", 2024-03-23T18:08:13.869 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:13.869 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.869 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.869 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "900", 2024-03-23T18:08:13.869 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.869 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.869 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.869 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-23T18:08:13.870 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.870 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.870 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.870 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.870 INFO:teuthology.orchestra.run.smithi086.stdout: "default_registry": { 2024-03-23T18:08:13.870 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "default_registry", 2024-03-23T18:08:13.870 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.870 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.871 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.871 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io", 2024-03-23T18:08:13.871 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.871 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.871 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.871 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-23T18:08:13.871 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.871 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.871 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.872 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.872 INFO:teuthology.orchestra.run.smithi086.stdout: "device_cache_timeout": { 2024-03-23T18:08:13.872 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "device_cache_timeout", 2024-03-23T18:08:13.872 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-23T18:08:13.872 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.872 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.872 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "1800", 2024-03-23T18:08:13.872 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.873 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.873 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.873 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "seconds to cache device inventory", 2024-03-23T18:08:13.873 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.873 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.873 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.873 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.873 INFO:teuthology.orchestra.run.smithi086.stdout: "device_enhanced_scan": { 2024-03-23T18:08:13.873 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "device_enhanced_scan", 2024-03-23T18:08:13.874 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.874 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.874 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.874 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.874 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.874 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.874 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.874 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-23T18:08:13.874 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.875 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.875 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.875 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.875 INFO:teuthology.orchestra.run.smithi086.stdout: "facts_cache_timeout": { 2024-03-23T18:08:13.875 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "facts_cache_timeout", 2024-03-23T18:08:13.875 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-23T18:08:13.875 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.875 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.876 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-03-23T18:08:13.876 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.876 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.876 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.876 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "seconds to cache host facts data", 2024-03-23T18:08:13.876 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.876 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.876 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.876 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.877 INFO:teuthology.orchestra.run.smithi086.stdout: "host_check_interval": { 2024-03-23T18:08:13.877 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "host_check_interval", 2024-03-23T18:08:13.877 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-23T18:08:13.877 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.877 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.877 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-03-23T18:08:13.877 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.878 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.878 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.878 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to perform a host check", 2024-03-23T18:08:13.878 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.878 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.878 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.878 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.878 INFO:teuthology.orchestra.run.smithi086.stdout: "hw_monitoring": { 2024-03-23T18:08:13.878 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "hw_monitoring", 2024-03-23T18:08:13.879 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.879 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.879 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.879 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.879 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.879 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.879 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.879 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-23T18:08:13.880 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.880 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.880 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.880 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.880 INFO:teuthology.orchestra.run.smithi086.stdout: "inventory_list_all": { 2024-03-23T18:08:13.880 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "inventory_list_all", 2024-03-23T18:08:13.880 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.880 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.881 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.881 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.881 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.881 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.881 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.881 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-23T18:08:13.881 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.881 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.881 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.882 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.882 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:13.882 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:13.882 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.882 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.882 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.882 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.882 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.883 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.883 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.883 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:13.883 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:13.883 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:13.883 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:13.883 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:13.883 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:13.883 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.884 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.884 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.884 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.884 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.884 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.884 INFO:teuthology.orchestra.run.smithi086.stdout: "log_refresh_metadata": { 2024-03-23T18:08:13.884 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_refresh_metadata", 2024-03-23T18:08:13.884 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.884 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.885 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.885 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.885 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.885 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.885 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.885 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-23T18:08:13.885 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.885 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.886 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.886 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.886 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:13.886 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:13.886 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.886 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.886 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.886 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:13.886 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.887 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.887 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.887 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-23T18:08:13.887 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.887 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.887 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.887 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.887 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:13.888 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:13.888 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.888 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.888 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.888 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:13.888 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.888 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.888 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.888 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:13.889 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:13.889 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:13.889 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:13.889 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:13.889 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:13.889 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.889 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.889 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.890 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.890 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.890 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.890 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:13.890 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:13.890 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.890 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.890 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.890 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.891 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.891 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.891 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.891 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.891 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.891 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.891 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.891 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.891 INFO:teuthology.orchestra.run.smithi086.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-23T18:08:13.892 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-23T18:08:13.892 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.892 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.892 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.892 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.892 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.892 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.892 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.892 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-23T18:08:13.893 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.893 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.893 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.893 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.893 INFO:teuthology.orchestra.run.smithi086.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-23T18:08:13.893 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-23T18:08:13.893 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.893 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.894 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.894 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "*", 2024-03-23T18:08:13.894 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.894 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.894 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.894 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-23T18:08:13.894 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.894 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.894 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.895 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.895 INFO:teuthology.orchestra.run.smithi086.stdout: "max_count_per_host": { 2024-03-23T18:08:13.895 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_count_per_host", 2024-03-23T18:08:13.895 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:13.895 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.895 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.895 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-03-23T18:08:13.895 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.895 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.896 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.896 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "max number of daemons per service per host", 2024-03-23T18:08:13.896 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.896 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.896 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.896 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.896 INFO:teuthology.orchestra.run.smithi086.stdout: "max_osd_draining_count": { 2024-03-23T18:08:13.896 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_osd_draining_count", 2024-03-23T18:08:13.896 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:13.897 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.897 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.897 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-03-23T18:08:13.897 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.897 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.897 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.897 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-23T18:08:13.897 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.898 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.898 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.898 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.898 INFO:teuthology.orchestra.run.smithi086.stdout: "migration_current": { 2024-03-23T18:08:13.898 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "migration_current", 2024-03-23T18:08:13.898 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:13.898 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.898 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.899 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.899 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.899 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.899 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.899 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "internal - do not modify", 2024-03-23T18:08:13.899 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.899 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.899 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.899 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.899 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": { 2024-03-23T18:08:13.900 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mode", 2024-03-23T18:08:13.900 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.900 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.900 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.900 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "root", 2024-03-23T18:08:13.900 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.900 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.900 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.901 INFO:teuthology.orchestra.run.smithi086.stdout: "cephadm-package", 2024-03-23T18:08:13.901 INFO:teuthology.orchestra.run.smithi086.stdout: "root" 2024-03-23T18:08:13.901 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.901 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "mode for remote execution of cephadm", 2024-03-23T18:08:13.901 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.901 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.901 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.901 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.901 INFO:teuthology.orchestra.run.smithi086.stdout: "oob_default_addr": { 2024-03-23T18:08:13.902 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "oob_default_addr", 2024-03-23T18:08:13.902 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.902 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.902 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.902 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "169.254.1.1", 2024-03-23T18:08:13.902 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.902 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.902 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.903 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-23T18:08:13.903 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.903 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.903 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.903 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.903 INFO:teuthology.orchestra.run.smithi086.stdout: "prometheus_alerts_path": { 2024-03-23T18:08:13.903 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "prometheus_alerts_path", 2024-03-23T18:08:13.903 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.903 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.904 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.904 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-23T18:08:13.904 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.904 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.904 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.904 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-23T18:08:13.904 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.904 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.904 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.905 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.905 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_insecure": { 2024-03-23T18:08:13.905 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_insecure", 2024-03-23T18:08:13.905 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.905 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.905 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.905 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.905 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.905 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.906 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.906 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-23T18:08:13.906 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.906 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.906 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.906 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.906 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_password": { 2024-03-23T18:08:13.906 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_password", 2024-03-23T18:08:13.907 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.907 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.907 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.907 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.907 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.907 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.907 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.907 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-23T18:08:13.907 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.908 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.908 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.908 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.908 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_url": { 2024-03-23T18:08:13.908 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_url", 2024-03-23T18:08:13.908 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.908 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.908 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.908 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.909 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.909 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.909 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.909 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-23T18:08:13.909 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.909 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.909 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.909 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.910 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_username": { 2024-03-23T18:08:13.910 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_username", 2024-03-23T18:08:13.910 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.910 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.910 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.910 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.910 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.910 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.910 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.911 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-23T18:08:13.911 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.911 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.911 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.911 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.911 INFO:teuthology.orchestra.run.smithi086.stdout: "secure_monitoring_stack": { 2024-03-23T18:08:13.911 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "secure_monitoring_stack", 2024-03-23T18:08:13.911 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.912 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.912 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.912 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.912 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.912 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.912 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.912 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-23T18:08:13.912 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.912 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.913 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.913 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.913 INFO:teuthology.orchestra.run.smithi086.stdout: "service_discovery_port": { 2024-03-23T18:08:13.913 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "service_discovery_port", 2024-03-23T18:08:13.913 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:13.913 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.913 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.913 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8765", 2024-03-23T18:08:13.914 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.914 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.914 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.914 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "cephadm service discovery port", 2024-03-23T18:08:13.914 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.914 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.914 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.914 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.915 INFO:teuthology.orchestra.run.smithi086.stdout: "ssh_config_file": { 2024-03-23T18:08:13.915 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssh_config_file", 2024-03-23T18:08:13.915 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.915 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.915 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.915 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.915 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.915 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.915 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.916 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-23T18:08:13.916 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.916 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.916 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.916 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.916 INFO:teuthology.orchestra.run.smithi086.stdout: "use_agent": { 2024-03-23T18:08:13.916 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "use_agent", 2024-03-23T18:08:13.916 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.917 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.917 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.917 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.917 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.917 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.917 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.917 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-23T18:08:13.917 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.918 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.918 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.918 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.918 INFO:teuthology.orchestra.run.smithi086.stdout: "use_repo_digest": { 2024-03-23T18:08:13.918 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "use_repo_digest", 2024-03-23T18:08:13.918 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.918 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.918 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.919 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:13.919 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.919 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.919 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.919 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-23T18:08:13.919 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.919 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.919 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.919 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.920 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_on_failed_host_check": { 2024-03-23T18:08:13.920 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_on_failed_host_check", 2024-03-23T18:08:13.920 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.920 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.920 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.920 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:13.920 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.920 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.921 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.921 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise a health warning if the host check fails", 2024-03-23T18:08:13.921 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.921 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.921 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.921 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.921 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_on_stray_daemons": { 2024-03-23T18:08:13.921 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_on_stray_daemons", 2024-03-23T18:08:13.921 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.922 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.922 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.922 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:13.922 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.922 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.922 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.922 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-23T18:08:13.922 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.923 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.923 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.923 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.923 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_on_stray_hosts": { 2024-03-23T18:08:13.923 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_on_stray_hosts", 2024-03-23T18:08:13.923 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.923 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.923 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.923 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:13.924 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.924 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.924 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.924 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-23T18:08:13.924 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.924 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.924 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.924 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:13.924 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:13.925 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.925 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:13.925 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crash", 2024-03-23T18:08:13.925 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:13.925 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:13.925 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:13.925 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:13.925 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:13.926 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.926 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.926 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.926 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.926 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.926 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.926 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.926 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:13.927 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:13.927 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:13.927 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:13.927 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:13.927 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:13.927 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.927 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.927 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.927 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.928 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.928 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.928 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:13.928 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:13.928 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.928 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.928 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.928 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.929 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.929 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.929 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.929 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.929 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.929 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.929 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.929 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.929 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:13.930 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:13.930 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.930 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.930 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.930 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:13.930 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.930 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.930 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:13.931 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:13.931 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:13.931 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:13.931 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:13.931 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:13.931 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:13.931 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:13.931 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.932 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.932 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.932 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.932 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.932 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:13.932 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:13.932 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.932 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.932 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.933 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.933 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.933 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.933 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.933 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.933 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.933 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.933 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.933 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.934 INFO:teuthology.orchestra.run.smithi086.stdout: "retain_interval": { 2024-03-23T18:08:13.934 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "retain_interval", 2024-03-23T18:08:13.934 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-23T18:08:13.934 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.934 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.934 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "31536000", 2024-03-23T18:08:13.934 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.934 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.935 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.935 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-23T18:08:13.935 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.935 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.935 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.935 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.935 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_recent_interval": { 2024-03-23T18:08:13.935 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_recent_interval", 2024-03-23T18:08:13.936 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-23T18:08:13.936 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.936 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:13.936 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "1209600", 2024-03-23T18:08:13.936 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.936 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.936 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.936 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-23T18:08:13.937 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.937 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.937 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.937 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:13.937 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:13.937 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.937 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:13.937 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "dashboard", 2024-03-23T18:08:13.937 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:13.938 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:13.938 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:13.938 INFO:teuthology.orchestra.run.smithi086.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-23T18:08:13.938 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-23T18:08:13.938 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:13.938 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.938 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.938 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-03-23T18:08:13.939 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.939 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.939 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.939 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.939 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.939 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.939 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.939 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.940 INFO:teuthology.orchestra.run.smithi086.stdout: "ALERTMANAGER_API_HOST": { 2024-03-23T18:08:13.940 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-23T18:08:13.940 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.940 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.940 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.940 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.940 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.940 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.940 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.941 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.941 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.941 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.941 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.941 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.941 INFO:teuthology.orchestra.run.smithi086.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-23T18:08:13.941 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-23T18:08:13.941 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.942 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.942 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.942 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:13.942 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.942 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.942 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.942 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.942 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.942 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.943 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.943 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.943 INFO:teuthology.orchestra.run.smithi086.stdout: "AUDIT_API_ENABLED": { 2024-03-23T18:08:13.943 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "AUDIT_API_ENABLED", 2024-03-23T18:08:13.943 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.943 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.943 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.943 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.943 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.944 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.944 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.944 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.944 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.944 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.944 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.944 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.944 INFO:teuthology.orchestra.run.smithi086.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-23T18:08:13.945 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-23T18:08:13.945 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.945 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.945 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.945 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:13.945 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.945 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.945 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.945 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.946 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.946 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.946 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.946 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.946 INFO:teuthology.orchestra.run.smithi086.stdout: "ENABLE_BROWSABLE_API": { 2024-03-23T18:08:13.946 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-23T18:08:13.946 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.946 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.947 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.947 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:13.947 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.947 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.947 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.947 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.947 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.947 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.947 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.948 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.948 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-23T18:08:13.948 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-23T18:08:13.948 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.948 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.948 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.948 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:13.948 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.948 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.949 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.949 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.949 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.949 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.949 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.949 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.949 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-23T18:08:13.949 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-23T18:08:13.950 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.950 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.950 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.950 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:13.950 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.950 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.950 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.950 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.950 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.951 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.951 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.951 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.951 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-23T18:08:13.951 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-23T18:08:13.951 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.951 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.951 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.952 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:13.952 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.952 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.952 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.952 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.952 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.952 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.952 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.953 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.953 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-23T18:08:13.953 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-23T18:08:13.953 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.953 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.953 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.953 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:13.953 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.953 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.954 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.954 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.954 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.954 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.954 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.954 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.954 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-23T18:08:13.954 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-23T18:08:13.954 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.955 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.955 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.955 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:13.955 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.955 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.955 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.955 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.955 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.956 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.956 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.956 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.956 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-23T18:08:13.956 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-23T18:08:13.956 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.956 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.956 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.956 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:13.957 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.957 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.957 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.957 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.957 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.957 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.957 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.957 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.958 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-23T18:08:13.958 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-23T18:08:13.958 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.958 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.958 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.958 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:13.958 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.958 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.959 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.959 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.959 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.959 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.959 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.959 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.959 INFO:teuthology.orchestra.run.smithi086.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-23T18:08:13.959 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-23T18:08:13.959 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.960 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.960 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.960 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.960 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.960 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.960 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.960 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.960 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.960 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.961 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.961 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.961 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_PASSWORD": { 2024-03-23T18:08:13.961 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-23T18:08:13.961 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.961 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.961 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.961 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "admin", 2024-03-23T18:08:13.962 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.962 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.962 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.962 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.962 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.962 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.962 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.962 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.963 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-23T18:08:13.963 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-23T18:08:13.963 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.963 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.963 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.963 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:13.963 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.963 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.963 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.964 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.964 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.964 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.964 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.964 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.964 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_URL": { 2024-03-23T18:08:13.964 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_URL", 2024-03-23T18:08:13.964 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.964 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.965 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.965 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.965 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.965 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.965 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.965 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.965 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.965 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.965 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.966 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.966 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_USERNAME": { 2024-03-23T18:08:13.966 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-23T18:08:13.966 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.966 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.966 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.966 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "admin", 2024-03-23T18:08:13.966 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.967 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.967 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.967 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.967 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.967 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.967 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.967 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.967 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-23T18:08:13.968 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-23T18:08:13.968 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.968 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.968 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.968 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.968 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.968 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.968 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.968 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.969 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.969 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.969 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.969 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.969 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-23T18:08:13.969 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-23T18:08:13.969 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.969 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.969 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.970 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.970 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.970 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.970 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.970 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.970 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.970 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.970 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.971 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.971 INFO:teuthology.orchestra.run.smithi086.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-23T18:08:13.971 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-23T18:08:13.971 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.971 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.971 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.971 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:13.971 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.971 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.972 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.972 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.972 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.972 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.972 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.972 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.972 INFO:teuthology.orchestra.run.smithi086.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-23T18:08:13.972 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-23T18:08:13.972 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.972 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.973 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.973 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.973 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.973 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.973 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.973 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.973 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.973 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.973 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.974 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.974 INFO:teuthology.orchestra.run.smithi086.stdout: "PROMETHEUS_API_HOST": { 2024-03-23T18:08:13.974 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-23T18:08:13.974 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.974 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.974 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.974 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.974 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.974 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.975 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.975 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.975 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.975 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.975 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.975 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.975 INFO:teuthology.orchestra.run.smithi086.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-23T18:08:13.975 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-23T18:08:13.976 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.976 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.976 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.976 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:13.976 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.976 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.976 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.976 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.977 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.977 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.977 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.977 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.977 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-23T18:08:13.977 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-23T18:08:13.977 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.977 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.977 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.978 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.978 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.978 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.978 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.978 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.978 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.978 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.978 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.979 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.979 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-23T18:08:13.979 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-23T18:08:13.979 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.979 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.979 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.979 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.979 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.979 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.980 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.980 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.980 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.980 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.980 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.980 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.980 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-23T18:08:13.980 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-23T18:08:13.980 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.981 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.981 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.981 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:13.981 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.981 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.981 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.981 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.981 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.981 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.982 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.982 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.982 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-23T18:08:13.982 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-23T18:08:13.982 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.982 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.982 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.982 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:13.983 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.983 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.983 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.983 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.983 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.983 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.983 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.983 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.983 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-23T18:08:13.984 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-23T18:08:13.984 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.984 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.984 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.984 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.984 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.984 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.984 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.984 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.985 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.985 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.985 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.985 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.985 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-23T18:08:13.985 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-23T18:08:13.985 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.985 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.985 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.986 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.986 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.986 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.986 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.986 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.986 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.986 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.986 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.987 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.987 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-23T18:08:13.987 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-23T18:08:13.987 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.987 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.987 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.987 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:13.987 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.988 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.988 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.988 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.988 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.988 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.988 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.988 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.988 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_ENABLED": { 2024-03-23T18:08:13.988 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-23T18:08:13.989 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:13.989 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.989 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.989 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:13.989 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.989 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.989 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.989 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.990 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.990 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.990 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.990 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.990 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-23T18:08:13.990 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-23T18:08:13.990 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.990 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.990 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.991 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-23T18:08:13.991 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.991 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.991 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.991 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.991 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.991 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.991 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.992 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.992 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-23T18:08:13.992 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-23T18:08:13.992 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:13.992 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.992 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.992 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-03-23T18:08:13.992 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.992 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.993 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.993 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.993 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.993 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.993 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.993 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.993 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-23T18:08:13.993 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-23T18:08:13.994 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:13.994 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.994 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.994 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8", 2024-03-23T18:08:13.994 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.994 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.994 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.994 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.994 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.995 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.995 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.995 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.995 INFO:teuthology.orchestra.run.smithi086.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-23T18:08:13.995 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-23T18:08:13.995 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:13.995 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.995 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.995 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "45", 2024-03-23T18:08:13.996 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.996 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.996 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.996 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.996 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.996 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.996 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.996 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.996 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_ACCESS_KEY": { 2024-03-23T18:08:13.997 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-23T18:08:13.997 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.997 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.997 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.997 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:13.997 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.997 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.997 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.997 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.998 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.998 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:13.998 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:13.998 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:13.998 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-23T18:08:13.998 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-23T18:08:13.998 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:13.998 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:13.999 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:13.999 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "admin", 2024-03-23T18:08:13.999 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:13.999 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:13.999 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:13.999 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:13.999 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:13.999 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.000 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.000 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.000 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_SECRET_KEY": { 2024-03-23T18:08:14.000 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-23T18:08:14.000 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.000 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.000 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.000 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.001 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.001 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.001 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.001 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.001 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.001 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.001 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.001 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.001 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_SSL_VERIFY": { 2024-03-23T18:08:14.002 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-23T18:08:14.002 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.002 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.002 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.002 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:14.002 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.002 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.002 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.003 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.003 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.003 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.003 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.003 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.003 INFO:teuthology.orchestra.run.smithi086.stdout: "UNSAFE_TLS_v1_2": { 2024-03-23T18:08:14.003 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-23T18:08:14.003 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.003 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.004 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.004 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.004 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.004 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.004 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.004 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.004 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.004 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.005 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.005 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.005 INFO:teuthology.orchestra.run.smithi086.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-23T18:08:14.005 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-23T18:08:14.005 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:14.005 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.005 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.005 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-03-23T18:08:14.006 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.006 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.006 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.006 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.006 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.006 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.006 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.006 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.007 INFO:teuthology.orchestra.run.smithi086.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-23T18:08:14.007 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-23T18:08:14.007 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:14.007 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.007 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.007 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-03-23T18:08:14.007 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.007 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.008 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.008 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.008 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.008 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.008 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.008 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.008 INFO:teuthology.orchestra.run.smithi086.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-23T18:08:14.008 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-23T18:08:14.009 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:14.009 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.009 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.009 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-03-23T18:08:14.009 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.009 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.009 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.009 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.009 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.010 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.010 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.010 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.010 INFO:teuthology.orchestra.run.smithi086.stdout: "cross_origin_url": { 2024-03-23T18:08:14.010 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cross_origin_url", 2024-03-23T18:08:14.010 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.010 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.011 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.011 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.011 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.011 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.011 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.011 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.011 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.011 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.012 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.012 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.012 INFO:teuthology.orchestra.run.smithi086.stdout: "crt_file": { 2024-03-23T18:08:14.012 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crt_file", 2024-03-23T18:08:14.012 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.012 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.012 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.012 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.012 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.013 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.013 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.013 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.013 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.013 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.013 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.013 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.013 INFO:teuthology.orchestra.run.smithi086.stdout: "debug": { 2024-03-23T18:08:14.013 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "debug", 2024-03-23T18:08:14.014 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.014 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.014 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.014 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.014 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.014 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.014 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.014 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Enable/disable debug options", 2024-03-23T18:08:14.015 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.015 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.015 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.015 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.015 INFO:teuthology.orchestra.run.smithi086.stdout: "jwt_token_ttl": { 2024-03-23T18:08:14.015 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "jwt_token_ttl", 2024-03-23T18:08:14.015 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:14.015 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.015 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.016 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "28800", 2024-03-23T18:08:14.016 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.016 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.016 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.016 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.016 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.016 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.016 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.016 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.016 INFO:teuthology.orchestra.run.smithi086.stdout: "key_file": { 2024-03-23T18:08:14.017 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "key_file", 2024-03-23T18:08:14.017 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.017 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.017 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.017 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.017 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.017 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.017 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.017 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.018 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.018 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.018 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.018 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.018 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:14.018 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:14.018 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.018 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.019 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.019 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.019 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.019 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.019 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.019 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.019 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.019 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.019 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.020 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.020 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.020 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.020 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.020 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.020 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.020 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.020 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.020 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:14.021 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:14.021 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.021 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.021 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.021 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.021 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.021 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.021 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.022 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.022 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.022 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.022 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.022 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.022 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:14.022 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:14.022 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.022 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.023 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.023 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:14.023 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.023 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.023 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.023 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.023 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.023 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.024 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.024 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.024 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.024 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.024 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.024 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.024 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.024 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.024 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.025 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:14.025 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:14.025 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.025 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.025 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.025 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.025 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.025 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.026 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.026 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.026 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.026 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.026 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.026 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.026 INFO:teuthology.orchestra.run.smithi086.stdout: "motd": { 2024-03-23T18:08:14.026 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "motd", 2024-03-23T18:08:14.027 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.027 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.027 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.027 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.027 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.027 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.027 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.027 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "The message of the day", 2024-03-23T18:08:14.028 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.028 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.028 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.028 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.028 INFO:teuthology.orchestra.run.smithi086.stdout: "redirect_resolve_ip_addr": { 2024-03-23T18:08:14.028 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "redirect_resolve_ip_addr", 2024-03-23T18:08:14.028 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.028 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.028 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.029 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.029 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.029 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.029 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.029 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.029 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.029 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.029 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.030 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.030 INFO:teuthology.orchestra.run.smithi086.stdout: "server_addr": { 2024-03-23T18:08:14.030 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_addr", 2024-03-23T18:08:14.030 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.030 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.030 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.030 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "::", 2024-03-23T18:08:14.030 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.030 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.031 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.031 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.031 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.031 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.031 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.031 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.031 INFO:teuthology.orchestra.run.smithi086.stdout: "server_port": { 2024-03-23T18:08:14.031 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_port", 2024-03-23T18:08:14.032 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:14.032 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.032 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.032 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8080", 2024-03-23T18:08:14.032 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.032 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.032 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.032 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.032 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.033 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.033 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.033 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.033 INFO:teuthology.orchestra.run.smithi086.stdout: "ssl": { 2024-03-23T18:08:14.033 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssl", 2024-03-23T18:08:14.033 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.033 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.033 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.034 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:14.034 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.034 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.034 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.034 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.034 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.034 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.034 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.034 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.035 INFO:teuthology.orchestra.run.smithi086.stdout: "ssl_server_port": { 2024-03-23T18:08:14.035 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssl_server_port", 2024-03-23T18:08:14.035 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:14.035 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.035 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.035 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8443", 2024-03-23T18:08:14.035 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.035 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.036 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.036 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.036 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.036 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.036 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.036 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.036 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_behaviour": { 2024-03-23T18:08:14.036 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_behaviour", 2024-03-23T18:08:14.036 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.037 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.037 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.037 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "redirect", 2024-03-23T18:08:14.037 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.037 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.037 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.037 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.037 INFO:teuthology.orchestra.run.smithi086.stdout: "redirect" 2024-03-23T18:08:14.037 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.038 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.038 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.038 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.038 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.038 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.038 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_error_status_code": { 2024-03-23T18:08:14.038 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_error_status_code", 2024-03-23T18:08:14.038 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:14.039 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.039 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.039 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "500", 2024-03-23T18:08:14.039 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "400", 2024-03-23T18:08:14.039 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "599", 2024-03-23T18:08:14.039 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.039 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.039 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.039 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.040 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.040 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.040 INFO:teuthology.orchestra.run.smithi086.stdout: "url_prefix": { 2024-03-23T18:08:14.040 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "url_prefix", 2024-03-23T18:08:14.040 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.040 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.040 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.040 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.040 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.041 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.041 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.041 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.041 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.041 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.041 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.041 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.041 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.041 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.042 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:14.042 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "devicehealth", 2024-03-23T18:08:14.042 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:14.042 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:14.042 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:14.042 INFO:teuthology.orchestra.run.smithi086.stdout: "enable_monitoring": { 2024-03-23T18:08:14.042 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enable_monitoring", 2024-03-23T18:08:14.042 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.043 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.043 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.043 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:14.043 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.043 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.043 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.043 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "monitor device health metrics", 2024-03-23T18:08:14.043 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.043 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.043 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.044 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.044 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:14.044 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:14.044 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.044 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.044 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.044 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.044 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.045 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.045 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.045 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.045 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.045 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.045 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.045 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.045 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.045 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.046 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.046 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.046 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.046 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.046 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.046 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:14.046 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:14.046 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.047 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.047 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.047 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.047 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.047 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.047 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.047 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.047 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.048 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.048 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.048 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.048 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:14.048 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:14.048 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.048 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.048 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.049 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:14.049 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.049 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.049 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.049 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.049 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.049 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.049 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.049 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.050 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.050 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.050 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.050 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.050 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.050 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.050 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.050 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:14.051 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:14.051 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.051 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.051 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.051 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.051 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.051 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.051 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.051 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.052 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.052 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.052 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.052 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.052 INFO:teuthology.orchestra.run.smithi086.stdout: "mark_out_threshold": { 2024-03-23T18:08:14.052 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mark_out_threshold", 2024-03-23T18:08:14.052 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-23T18:08:14.053 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.053 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.053 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "2419200", 2024-03-23T18:08:14.053 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.053 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.053 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.053 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-23T18:08:14.053 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.054 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.054 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.054 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.054 INFO:teuthology.orchestra.run.smithi086.stdout: "pool_name": { 2024-03-23T18:08:14.054 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pool_name", 2024-03-23T18:08:14.054 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.054 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.054 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.054 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "device_health_metrics", 2024-03-23T18:08:14.055 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.055 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.055 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.055 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-23T18:08:14.055 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.055 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.055 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.055 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.055 INFO:teuthology.orchestra.run.smithi086.stdout: "retention_period": { 2024-03-23T18:08:14.056 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "retention_period", 2024-03-23T18:08:14.056 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-23T18:08:14.056 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.056 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.056 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "15552000", 2024-03-23T18:08:14.056 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.056 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.056 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.056 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how long to retain device health metrics", 2024-03-23T18:08:14.057 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.057 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.057 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.057 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.057 INFO:teuthology.orchestra.run.smithi086.stdout: "scrape_frequency": { 2024-03-23T18:08:14.057 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "scrape_frequency", 2024-03-23T18:08:14.057 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-23T18:08:14.057 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.058 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.058 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "86400", 2024-03-23T18:08:14.058 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.058 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.058 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.058 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-23T18:08:14.058 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.058 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.059 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.059 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.059 INFO:teuthology.orchestra.run.smithi086.stdout: "self_heal": { 2024-03-23T18:08:14.059 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "self_heal", 2024-03-23T18:08:14.059 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.059 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.059 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.059 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:14.059 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.060 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.060 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.060 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-23T18:08:14.060 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.060 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.060 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.060 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.060 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-03-23T18:08:14.060 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-03-23T18:08:14.061 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-23T18:08:14.061 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.061 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.061 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-03-23T18:08:14.061 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.061 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.061 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.061 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to wake up and check device health", 2024-03-23T18:08:14.061 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.061 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.062 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.062 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.062 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_threshold": { 2024-03-23T18:08:14.062 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_threshold", 2024-03-23T18:08:14.062 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-23T18:08:14.062 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.062 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.062 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "7257600", 2024-03-23T18:08:14.063 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.063 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.063 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.063 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-23T18:08:14.063 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.063 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.063 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.063 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.063 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.064 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.064 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:14.064 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "diskprediction_local", 2024-03-23T18:08:14.064 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:14.064 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:14.064 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:14.064 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:14.064 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:14.065 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.065 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.065 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.065 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.065 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.065 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.065 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.065 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.065 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.066 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.066 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.066 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.066 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.066 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.066 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.066 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.066 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.067 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.067 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.067 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:14.067 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:14.067 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.067 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.067 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.067 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.068 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.068 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.068 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.068 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.068 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.068 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.068 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.068 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.069 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:14.069 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:14.069 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.069 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.069 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.069 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:14.069 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.070 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.070 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.070 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.070 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.070 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.070 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.070 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.071 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.071 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.071 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.071 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.071 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.071 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.071 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.072 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:14.072 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:14.072 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.072 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.072 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.072 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.072 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.073 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.073 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.073 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.073 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.073 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.073 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.073 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.073 INFO:teuthology.orchestra.run.smithi086.stdout: "predict_interval": { 2024-03-23T18:08:14.074 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "predict_interval", 2024-03-23T18:08:14.074 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.074 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.074 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.074 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "86400", 2024-03-23T18:08:14.074 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.074 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.074 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.075 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.075 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.075 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.075 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.075 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.075 INFO:teuthology.orchestra.run.smithi086.stdout: "predictor_model": { 2024-03-23T18:08:14.075 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "predictor_model", 2024-03-23T18:08:14.075 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.076 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.076 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.076 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "prophetstor", 2024-03-23T18:08:14.076 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.076 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.076 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.076 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.076 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.076 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.077 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.077 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.077 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-03-23T18:08:14.077 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-03-23T18:08:14.077 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.077 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.077 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.077 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-03-23T18:08:14.078 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.078 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.078 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.078 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.078 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.078 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.078 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.078 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.079 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.079 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.079 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:14.079 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "influx", 2024-03-23T18:08:14.079 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": false, 2024-03-23T18:08:14.079 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "influxdb python module not found", 2024-03-23T18:08:14.079 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:14.079 INFO:teuthology.orchestra.run.smithi086.stdout: "batch_size": { 2024-03-23T18:08:14.080 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "batch_size", 2024-03-23T18:08:14.080 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:14.080 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.080 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.080 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5000", 2024-03-23T18:08:14.080 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.080 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.080 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.080 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-23T18:08:14.081 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.081 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.081 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.081 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.081 INFO:teuthology.orchestra.run.smithi086.stdout: "database": { 2024-03-23T18:08:14.081 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "database", 2024-03-23T18:08:14.081 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.081 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.082 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.082 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "ceph", 2024-03-23T18:08:14.082 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.082 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.082 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.082 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-23T18:08:14.082 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.082 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.083 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.083 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.083 INFO:teuthology.orchestra.run.smithi086.stdout: "hostname": { 2024-03-23T18:08:14.083 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "hostname", 2024-03-23T18:08:14.083 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.083 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.083 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.083 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.084 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.084 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.084 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.084 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "InfluxDB server hostname", 2024-03-23T18:08:14.084 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.084 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.084 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.084 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.084 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-03-23T18:08:14.085 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-03-23T18:08:14.085 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-23T18:08:14.085 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.085 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.085 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "30", 2024-03-23T18:08:14.085 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "5", 2024-03-23T18:08:14.085 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.085 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.085 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-23T18:08:14.086 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.086 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.086 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.086 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.086 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:14.086 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:14.086 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.086 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.087 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.087 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.087 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.087 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.087 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.087 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.087 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.087 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.087 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.088 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.088 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.088 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.088 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.088 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.088 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.088 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.088 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.089 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:14.089 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:14.089 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.089 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.089 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.089 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.089 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.090 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.090 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.090 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.090 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.090 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.090 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.090 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.091 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:14.091 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:14.091 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.091 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.091 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.091 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:14.091 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.091 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.092 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.092 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.092 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.092 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.092 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.092 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.092 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.093 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.093 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.093 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.093 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.093 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.093 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.093 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:14.093 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:14.094 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.094 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.094 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.094 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.094 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.094 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.095 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.095 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.095 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.095 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.095 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.095 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.095 INFO:teuthology.orchestra.run.smithi086.stdout: "password": { 2024-03-23T18:08:14.095 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "password", 2024-03-23T18:08:14.096 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.096 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.096 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.096 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.096 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.096 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.096 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.097 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "password of InfluxDB server user", 2024-03-23T18:08:14.097 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.097 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.097 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.097 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.097 INFO:teuthology.orchestra.run.smithi086.stdout: "port": { 2024-03-23T18:08:14.097 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "port", 2024-03-23T18:08:14.098 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:14.098 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.098 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.098 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8086", 2024-03-23T18:08:14.098 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.098 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.098 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.098 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "InfluxDB server port", 2024-03-23T18:08:14.099 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.099 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.099 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.099 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.099 INFO:teuthology.orchestra.run.smithi086.stdout: "ssl": { 2024-03-23T18:08:14.099 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssl", 2024-03-23T18:08:14.099 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.100 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.100 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.100 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "false", 2024-03-23T18:08:14.100 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.100 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.100 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.101 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-23T18:08:14.101 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.101 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.101 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.101 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.101 INFO:teuthology.orchestra.run.smithi086.stdout: "threads": { 2024-03-23T18:08:14.101 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "threads", 2024-03-23T18:08:14.101 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:14.102 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.102 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.102 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-03-23T18:08:14.102 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-03-23T18:08:14.102 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "32", 2024-03-23T18:08:14.102 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.102 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-23T18:08:14.102 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.102 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.103 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.103 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.103 INFO:teuthology.orchestra.run.smithi086.stdout: "username": { 2024-03-23T18:08:14.103 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "username", 2024-03-23T18:08:14.103 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.103 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.103 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.103 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.104 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.104 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.104 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.104 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "username of InfluxDB server user", 2024-03-23T18:08:14.104 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.104 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.104 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.104 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.104 INFO:teuthology.orchestra.run.smithi086.stdout: "verify_ssl": { 2024-03-23T18:08:14.105 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "verify_ssl", 2024-03-23T18:08:14.105 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.105 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.105 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.105 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "true", 2024-03-23T18:08:14.105 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.105 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.105 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.105 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-23T18:08:14.106 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.106 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.106 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.106 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.106 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.106 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.106 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:14.107 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "insights", 2024-03-23T18:08:14.107 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:14.107 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:14.107 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:14.107 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:14.107 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:14.107 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.107 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.108 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.108 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.108 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.108 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.108 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.108 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.108 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.108 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.109 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.109 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.109 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.109 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.109 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.109 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.109 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.109 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.110 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.110 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:14.110 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:14.110 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.110 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.110 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.110 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.110 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.111 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.111 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.111 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.111 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.111 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.111 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.111 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.111 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:14.111 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:14.112 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.112 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.112 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.112 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:14.112 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.112 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.112 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.112 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.112 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.112 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.113 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.113 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.113 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.113 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.113 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.113 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.113 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.113 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.113 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.114 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:14.114 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:14.114 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.114 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.114 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.114 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.114 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.114 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.114 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.115 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.115 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.115 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.115 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.115 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.115 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.115 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.115 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:14.116 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "iostat", 2024-03-23T18:08:14.116 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:14.116 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:14.116 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:14.116 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:14.116 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:14.116 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.116 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.116 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.116 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.117 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.117 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.117 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.117 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.117 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.117 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.117 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.117 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.118 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.118 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.118 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.118 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.118 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.118 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.118 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.118 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:14.119 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:14.119 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.119 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.119 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.119 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.119 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.119 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.119 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.120 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.120 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.120 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.120 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.120 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.120 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:14.120 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:14.120 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.121 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.121 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.121 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:14.121 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.121 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.121 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.121 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.121 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.122 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.122 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.122 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.122 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.122 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.122 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.122 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.123 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.123 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.123 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.123 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:14.123 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:14.123 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.123 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.123 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.124 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.124 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.124 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.124 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.124 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.124 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.124 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.124 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.124 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.125 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.125 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.125 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:14.125 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "k8sevents", 2024-03-23T18:08:14.125 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:14.125 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:14.125 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:14.125 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_event_retention_days": { 2024-03-23T18:08:14.126 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ceph_event_retention_days", 2024-03-23T18:08:14.126 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:14.126 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.126 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.126 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "7", 2024-03-23T18:08:14.126 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.126 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.126 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.127 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-23T18:08:14.127 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.127 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.127 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.127 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.127 INFO:teuthology.orchestra.run.smithi086.stdout: "config_check_secs": { 2024-03-23T18:08:14.128 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "config_check_secs", 2024-03-23T18:08:14.128 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:14.128 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.128 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.128 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-03-23T18:08:14.128 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "10", 2024-03-23T18:08:14.128 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.128 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.128 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-23T18:08:14.129 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.129 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.129 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.129 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.129 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:14.129 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:14.129 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.130 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.130 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.130 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.130 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.130 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.130 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.130 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.130 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.131 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.131 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.131 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.131 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.131 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.131 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.131 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.132 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.132 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.132 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.132 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:14.132 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:14.132 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.132 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.132 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.133 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.133 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.133 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.133 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.133 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.133 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.134 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.134 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.134 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.134 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:14.134 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:14.134 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.134 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.134 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.134 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:14.135 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.135 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.135 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.135 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.135 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.135 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.136 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.136 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.136 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.136 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.136 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.136 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.136 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.137 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.137 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.137 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:14.137 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:14.137 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.137 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.137 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.138 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.138 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.138 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.138 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.138 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.138 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.138 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.139 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.139 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.139 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.139 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.139 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:14.139 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "localpool", 2024-03-23T18:08:14.139 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:14.139 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:14.140 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:14.140 INFO:teuthology.orchestra.run.smithi086.stdout: "failure_domain": { 2024-03-23T18:08:14.140 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "failure_domain", 2024-03-23T18:08:14.140 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.140 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.140 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.140 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "host", 2024-03-23T18:08:14.140 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.141 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.141 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.141 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "failure domain for any created local pool", 2024-03-23T18:08:14.141 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-23T18:08:14.141 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.141 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.141 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.141 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:14.142 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:14.142 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.142 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.142 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.142 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.142 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.142 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.143 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.143 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.143 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.143 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.143 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.143 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.143 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.143 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.144 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.144 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.144 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.144 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.144 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.144 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:14.144 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:14.144 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.145 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.145 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.145 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.145 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.145 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.145 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.145 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.145 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.146 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.146 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.146 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.146 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:14.146 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:14.146 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.146 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.146 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.146 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:14.147 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.147 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.147 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.147 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.147 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.147 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.147 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.147 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.148 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.148 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.148 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.148 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.148 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.148 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.148 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.148 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:14.148 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:14.149 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.149 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.149 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.149 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.149 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.149 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.149 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.149 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.149 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.150 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.150 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.150 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.150 INFO:teuthology.orchestra.run.smithi086.stdout: "min_size": { 2024-03-23T18:08:14.150 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "min_size", 2024-03-23T18:08:14.150 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:14.150 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.150 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.150 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.151 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.151 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.151 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.151 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "default min_size for any created local pool", 2024-03-23T18:08:14.151 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-23T18:08:14.151 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.151 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.151 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.152 INFO:teuthology.orchestra.run.smithi086.stdout: "num_rep": { 2024-03-23T18:08:14.152 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "num_rep", 2024-03-23T18:08:14.152 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:14.152 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.152 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.152 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "3", 2024-03-23T18:08:14.152 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.152 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.153 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.153 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "default replica count for any created local pool", 2024-03-23T18:08:14.153 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.153 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.153 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.153 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.153 INFO:teuthology.orchestra.run.smithi086.stdout: "pg_num": { 2024-03-23T18:08:14.153 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pg_num", 2024-03-23T18:08:14.153 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:14.154 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.154 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.154 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "128", 2024-03-23T18:08:14.154 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.154 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.154 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.154 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "default pg_num for any created local pool", 2024-03-23T18:08:14.154 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.155 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.155 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.155 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.155 INFO:teuthology.orchestra.run.smithi086.stdout: "prefix": { 2024-03-23T18:08:14.155 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "prefix", 2024-03-23T18:08:14.155 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.155 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.155 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.155 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.156 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.156 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.156 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.156 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "name prefix for any created local pool", 2024-03-23T18:08:14.156 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.156 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.156 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.156 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.157 INFO:teuthology.orchestra.run.smithi086.stdout: "subtree": { 2024-03-23T18:08:14.157 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "subtree", 2024-03-23T18:08:14.157 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.157 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.157 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.157 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "rack", 2024-03-23T18:08:14.157 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.157 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.157 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.158 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-23T18:08:14.158 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-23T18:08:14.158 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.158 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.158 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.158 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.158 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.158 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:14.159 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mds_autoscaler", 2024-03-23T18:08:14.159 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:14.159 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:14.159 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:14.159 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:14.159 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:14.159 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.159 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.159 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.160 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.160 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.160 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.160 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.160 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.160 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.160 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.160 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.161 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.161 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.161 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.161 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.161 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.161 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.161 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.162 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.162 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:14.162 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:14.162 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.162 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.162 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.162 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.163 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.163 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.163 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.163 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.163 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.163 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.163 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.163 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.164 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:14.164 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:14.164 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.164 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.164 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.164 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:14.164 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.165 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.165 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.165 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.165 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.165 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.165 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.165 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.165 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.166 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.166 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.166 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.167 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.167 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.167 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.167 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:14.167 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:14.167 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.167 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.168 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.168 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.168 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.168 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.168 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.168 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.168 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.169 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.169 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.169 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.169 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.169 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.169 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:14.170 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mirroring", 2024-03-23T18:08:14.170 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:14.170 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:14.170 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:14.170 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:14.170 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:14.170 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.171 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.171 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.171 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.171 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.171 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.171 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.171 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.172 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.172 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.172 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.172 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.172 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.172 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.173 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.173 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.173 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.173 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.173 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.173 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:14.173 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:14.173 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.174 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.174 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.174 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.174 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.174 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.174 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.174 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.174 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.175 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.175 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.175 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.175 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:14.175 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:14.175 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.175 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.175 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.176 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:14.176 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.176 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.176 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.176 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.176 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.176 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.177 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.177 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.177 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.177 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.177 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.177 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.177 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.177 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.177 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.178 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:14.178 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:14.178 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.178 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.178 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.178 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.178 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.178 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.179 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.179 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.179 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.179 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.179 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.179 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.179 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.179 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.180 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:14.180 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "nfs", 2024-03-23T18:08:14.180 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:14.180 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:14.180 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:14.180 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:14.180 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:14.180 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.180 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.181 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.181 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.181 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.181 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.181 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.181 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.181 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.181 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.182 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.182 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.182 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.182 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.182 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.182 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.182 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.182 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.182 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.183 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:14.183 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:14.183 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.183 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.183 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.183 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.183 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.183 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.184 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.184 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.184 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.184 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.184 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.184 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.184 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:14.184 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:14.184 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.185 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.185 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.185 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:14.185 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.185 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.185 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.185 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.185 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.185 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.186 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.186 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.186 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.186 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.186 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.186 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.186 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.186 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.187 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.187 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:14.187 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:14.187 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.187 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.187 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.187 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.187 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.187 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.188 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.188 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.188 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.188 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.188 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.188 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.188 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.188 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.189 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:14.189 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "orchestrator", 2024-03-23T18:08:14.189 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:14.189 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:14.189 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:14.189 INFO:teuthology.orchestra.run.smithi086.stdout: "fail_fs": { 2024-03-23T18:08:14.189 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "fail_fs", 2024-03-23T18:08:14.189 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.189 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.190 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.190 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.190 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.190 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.190 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.190 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-23T18:08:14.190 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.190 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.191 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.191 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.191 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:14.191 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:14.191 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.191 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.191 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.191 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.192 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.192 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.192 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.192 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.192 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.192 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.192 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.192 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.192 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.193 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.193 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.193 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.193 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.193 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.193 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.193 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:14.193 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:14.193 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.194 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.194 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.194 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.194 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.194 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.194 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.194 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.194 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.195 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.195 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.195 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.195 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:14.195 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:14.195 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.195 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.195 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.195 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:14.196 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.196 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.196 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.196 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.196 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.196 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.196 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.196 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.196 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.197 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.197 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.197 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.197 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.197 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.197 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.197 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:14.197 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:14.197 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.198 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.198 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.198 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.198 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.198 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.198 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.198 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.199 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.199 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.199 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.199 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.199 INFO:teuthology.orchestra.run.smithi086.stdout: "orchestrator": { 2024-03-23T18:08:14.199 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "orchestrator", 2024-03-23T18:08:14.200 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.200 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.200 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.200 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.200 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.200 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.200 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.200 INFO:teuthology.orchestra.run.smithi086.stdout: "cephadm", 2024-03-23T18:08:14.200 INFO:teuthology.orchestra.run.smithi086.stdout: "rook", 2024-03-23T18:08:14.201 INFO:teuthology.orchestra.run.smithi086.stdout: "test_orchestrator" 2024-03-23T18:08:14.201 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.201 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Orchestrator backend", 2024-03-23T18:08:14.201 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.201 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.201 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.201 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.201 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.201 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.202 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:14.202 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "osd_perf_query", 2024-03-23T18:08:14.202 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:14.202 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:14.202 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:14.202 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:14.202 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:14.202 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.202 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.203 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.203 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.203 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.203 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.203 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.203 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.203 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.203 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.204 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.204 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.204 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.204 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.204 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.204 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.204 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.204 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.204 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.205 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:14.205 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:14.205 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.205 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.205 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.205 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.205 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.205 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.205 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.206 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.206 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.206 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.206 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.206 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.206 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:14.206 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:14.206 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.207 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.207 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.207 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:14.207 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.207 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.207 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.207 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.207 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.207 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.208 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.208 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.208 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.208 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.208 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.208 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.208 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.208 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.208 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.209 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:14.209 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:14.209 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.209 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.209 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.209 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.209 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.209 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.209 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.210 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.210 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.210 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.210 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.210 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.210 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.210 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.210 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:14.211 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "osd_support", 2024-03-23T18:08:14.211 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:14.211 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:14.211 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:14.211 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:14.211 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:14.211 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.211 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.211 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.212 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.212 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.212 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.212 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.212 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.212 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.212 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.212 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.212 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.213 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.213 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.213 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.213 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.213 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.213 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.213 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.213 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:14.214 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:14.214 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.214 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.214 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.214 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.214 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.214 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.214 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.215 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.215 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.215 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.215 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.215 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.215 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:14.215 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:14.215 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.215 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.216 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.216 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:14.216 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.216 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.216 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.216 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.216 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.216 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.217 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.217 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.217 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.217 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.217 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.217 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.217 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.217 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.217 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.218 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:14.218 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:14.218 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.218 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.218 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.218 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.218 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.218 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.218 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.219 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.219 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.219 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.219 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.219 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.219 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.219 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.219 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:14.219 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pg_autoscaler", 2024-03-23T18:08:14.220 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:14.220 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:14.220 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:14.221 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:14.222 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:14.222 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.222 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.223 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.223 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.223 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.223 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.223 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.223 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.223 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.223 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.223 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.224 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.224 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.224 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.224 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.224 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.224 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.224 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.224 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.225 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:14.225 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:14.225 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.225 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.225 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.225 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.225 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.225 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.225 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.226 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.226 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.226 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.226 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.226 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.226 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:14.226 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:14.227 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.227 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.227 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.227 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:14.227 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.227 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.227 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.227 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.228 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.228 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.228 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.228 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.228 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.228 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.228 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.228 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.228 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.229 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.229 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.229 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:14.229 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:14.229 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.229 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.229 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.229 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.229 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.230 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.230 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.230 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.230 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.230 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.230 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.230 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.230 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-03-23T18:08:14.230 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-03-23T18:08:14.231 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-23T18:08:14.231 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.231 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.231 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-03-23T18:08:14.231 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.231 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.231 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.231 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.231 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.232 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.232 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.232 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.232 INFO:teuthology.orchestra.run.smithi086.stdout: "threshold": { 2024-03-23T18:08:14.232 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "threshold", 2024-03-23T18:08:14.232 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-03-23T18:08:14.233 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.233 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.233 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "3.0", 2024-03-23T18:08:14.233 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1.0", 2024-03-23T18:08:14.233 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.234 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.234 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "scaling threshold", 2024-03-23T18:08:14.234 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-23T18:08:14.234 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.234 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.235 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.235 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.235 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.235 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:14.236 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "progress", 2024-03-23T18:08:14.236 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:14.236 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:14.236 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:14.237 INFO:teuthology.orchestra.run.smithi086.stdout: "allow_pg_recovery_event": { 2024-03-23T18:08:14.237 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "allow_pg_recovery_event", 2024-03-23T18:08:14.237 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.238 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.238 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.238 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.238 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.238 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.239 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.239 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-23T18:08:14.239 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.239 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.239 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.240 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.240 INFO:teuthology.orchestra.run.smithi086.stdout: "enabled": { 2024-03-23T18:08:14.240 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enabled", 2024-03-23T18:08:14.240 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.240 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.241 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.241 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:14.241 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.241 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.242 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.242 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.242 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.242 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.243 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.243 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.243 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:14.243 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:14.244 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.244 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.244 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.244 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.244 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.245 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.245 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.245 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.245 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.245 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.245 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.245 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.245 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.246 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.246 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.246 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.246 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.246 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.246 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.246 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:14.246 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:14.247 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.247 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.247 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.247 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.247 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.247 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.247 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.248 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.248 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.248 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.248 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.248 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.248 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:14.248 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:14.248 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.248 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.249 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.249 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:14.249 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.249 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.249 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.249 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.249 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.249 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.249 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.250 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.250 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.250 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.250 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.250 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.250 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.250 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.250 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.250 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:14.251 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:14.251 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.251 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.251 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.251 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.251 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.251 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.251 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.251 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.252 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.252 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.252 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.252 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.252 INFO:teuthology.orchestra.run.smithi086.stdout: "max_completed_events": { 2024-03-23T18:08:14.252 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_completed_events", 2024-03-23T18:08:14.252 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:14.252 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.252 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.253 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "50", 2024-03-23T18:08:14.253 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.253 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.253 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.253 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "number of past completed events to remember", 2024-03-23T18:08:14.253 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.253 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.253 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.254 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.254 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-03-23T18:08:14.254 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-03-23T18:08:14.254 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-23T18:08:14.254 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.254 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.254 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-03-23T18:08:14.255 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.255 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.255 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.255 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how long the module is going to sleep", 2024-03-23T18:08:14.255 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.255 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.255 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.256 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.256 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.256 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.256 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:14.256 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "prometheus", 2024-03-23T18:08:14.256 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:14.256 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:14.256 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:14.256 INFO:teuthology.orchestra.run.smithi086.stdout: "cache": { 2024-03-23T18:08:14.257 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cache", 2024-03-23T18:08:14.257 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.257 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.257 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.257 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:14.257 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.258 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.258 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.258 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.258 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.258 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.259 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.259 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.259 INFO:teuthology.orchestra.run.smithi086.stdout: "exclude_perf_counters": { 2024-03-23T18:08:14.259 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "exclude_perf_counters", 2024-03-23T18:08:14.259 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.260 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.260 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.260 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:14.260 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.260 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.261 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.261 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-23T18:08:14.261 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-23T18:08:14.261 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.261 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.261 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.262 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:14.262 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:14.262 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.262 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.262 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.263 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.263 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.263 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.263 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.263 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.264 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.264 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.264 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.264 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.264 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.264 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.264 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.264 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.265 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.265 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.265 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.265 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:14.265 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:14.266 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.266 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.266 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.266 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.266 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.266 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.266 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.266 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.267 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.267 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.267 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.267 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.267 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:14.267 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:14.267 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.267 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.267 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.268 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:14.268 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.268 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.268 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.268 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.268 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.269 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.269 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.269 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.269 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.269 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.269 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.269 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.269 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.270 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.270 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.270 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:14.270 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:14.270 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.270 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.270 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.270 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.271 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.271 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.271 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.271 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.271 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.271 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.271 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.271 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.272 INFO:teuthology.orchestra.run.smithi086.stdout: "rbd_stats_pools": { 2024-03-23T18:08:14.272 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rbd_stats_pools", 2024-03-23T18:08:14.272 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.272 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.272 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.272 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.272 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.272 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.273 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.273 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.273 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.273 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.273 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.273 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.273 INFO:teuthology.orchestra.run.smithi086.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-23T18:08:14.273 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-23T18:08:14.273 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:14.274 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.274 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.274 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "300", 2024-03-23T18:08:14.274 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.274 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.274 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.274 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.274 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.275 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.275 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.275 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.275 INFO:teuthology.orchestra.run.smithi086.stdout: "scrape_interval": { 2024-03-23T18:08:14.275 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "scrape_interval", 2024-03-23T18:08:14.275 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-03-23T18:08:14.275 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.275 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.276 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "15.0", 2024-03-23T18:08:14.276 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.276 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.276 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.276 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.277 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.277 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.277 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.277 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.277 INFO:teuthology.orchestra.run.smithi086.stdout: "server_addr": { 2024-03-23T18:08:14.277 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_addr", 2024-03-23T18:08:14.277 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.277 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.277 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.278 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "::", 2024-03-23T18:08:14.278 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.278 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.278 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.278 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-23T18:08:14.278 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.278 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.279 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.279 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.279 INFO:teuthology.orchestra.run.smithi086.stdout: "server_port": { 2024-03-23T18:08:14.279 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_port", 2024-03-23T18:08:14.279 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:14.279 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.279 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.279 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "9283", 2024-03-23T18:08:14.280 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.280 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.280 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.280 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-23T18:08:14.280 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.280 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.280 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.280 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.281 INFO:teuthology.orchestra.run.smithi086.stdout: "stale_cache_strategy": { 2024-03-23T18:08:14.281 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "stale_cache_strategy", 2024-03-23T18:08:14.281 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.281 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.281 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.281 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "log", 2024-03-23T18:08:14.281 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.281 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.282 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.282 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.282 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.282 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.282 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.282 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.282 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_behaviour": { 2024-03-23T18:08:14.282 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_behaviour", 2024-03-23T18:08:14.283 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.283 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.283 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.283 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "default", 2024-03-23T18:08:14.283 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.283 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.283 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.283 INFO:teuthology.orchestra.run.smithi086.stdout: "default", 2024-03-23T18:08:14.283 INFO:teuthology.orchestra.run.smithi086.stdout: "error" 2024-03-23T18:08:14.284 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.284 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.284 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.284 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.284 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.284 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.284 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_error_status_code": { 2024-03-23T18:08:14.284 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_error_status_code", 2024-03-23T18:08:14.284 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:14.285 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.285 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.285 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "500", 2024-03-23T18:08:14.285 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "400", 2024-03-23T18:08:14.285 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "599", 2024-03-23T18:08:14.285 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.285 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.285 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.285 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.286 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.286 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.286 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.286 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.286 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:14.286 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rbd_support", 2024-03-23T18:08:14.287 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:14.287 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:14.287 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:14.287 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:14.287 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:14.287 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.287 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.288 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.288 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.288 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.288 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.288 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.288 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.288 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.288 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.288 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.289 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.289 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.289 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.289 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.289 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.289 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.290 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.290 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.290 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:14.290 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:14.290 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.290 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.290 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.290 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.290 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.291 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.291 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.291 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.291 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.291 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.291 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.291 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.291 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:14.291 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:14.292 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.292 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.292 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.292 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:14.292 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.292 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.292 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.292 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.292 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.293 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.293 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.293 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.293 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.293 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.293 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.293 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.293 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.293 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.294 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.294 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:14.294 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:14.294 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.294 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.294 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.294 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.294 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.295 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.295 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.295 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.295 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.295 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.295 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.295 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.295 INFO:teuthology.orchestra.run.smithi086.stdout: "max_concurrent_snap_create": { 2024-03-23T18:08:14.295 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_concurrent_snap_create", 2024-03-23T18:08:14.296 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:14.296 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.296 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.296 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-03-23T18:08:14.296 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.296 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.296 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.296 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.297 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.297 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.297 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.297 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.297 INFO:teuthology.orchestra.run.smithi086.stdout: "mirror_snapshot_schedule": { 2024-03-23T18:08:14.297 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mirror_snapshot_schedule", 2024-03-23T18:08:14.298 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.298 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.298 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.298 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.298 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.298 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.298 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.298 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.298 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.299 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.299 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.299 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.299 INFO:teuthology.orchestra.run.smithi086.stdout: "trash_purge_schedule": { 2024-03-23T18:08:14.299 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "trash_purge_schedule", 2024-03-23T18:08:14.299 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.299 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.300 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.300 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.300 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.300 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.300 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.300 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.300 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.301 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.301 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.301 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.301 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.301 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.301 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:14.301 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "restful", 2024-03-23T18:08:14.301 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:14.302 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:14.302 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:14.302 INFO:teuthology.orchestra.run.smithi086.stdout: "enable_auth": { 2024-03-23T18:08:14.302 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enable_auth", 2024-03-23T18:08:14.302 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.302 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.302 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.302 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:14.302 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.303 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.303 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.303 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.303 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.303 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.303 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.303 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.303 INFO:teuthology.orchestra.run.smithi086.stdout: "key_file": { 2024-03-23T18:08:14.304 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "key_file", 2024-03-23T18:08:14.304 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.304 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.304 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.304 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.304 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.304 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.304 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.304 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.305 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.305 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.305 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.305 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.305 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:14.305 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:14.305 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.305 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.305 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.306 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.306 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.306 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.306 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.306 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.306 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.306 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.306 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.306 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.307 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.307 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.307 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.307 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.307 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.307 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.307 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.308 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:14.308 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:14.308 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.308 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.308 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.308 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.308 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.309 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.309 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.309 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.309 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.309 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.309 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.309 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.309 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:14.310 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:14.310 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.310 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.310 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.310 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:14.310 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.311 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.311 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.311 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.311 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.311 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.311 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.311 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.311 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.311 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.312 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.312 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.312 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.312 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.312 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.312 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:14.312 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:14.312 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.312 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.313 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.313 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.313 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.313 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.313 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.313 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.313 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.313 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.314 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.314 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.314 INFO:teuthology.orchestra.run.smithi086.stdout: "server_addr": { 2024-03-23T18:08:14.314 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_addr", 2024-03-23T18:08:14.314 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.314 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.314 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.314 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.314 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.315 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.315 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.315 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.315 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.315 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.315 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.315 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.315 INFO:teuthology.orchestra.run.smithi086.stdout: "server_port": { 2024-03-23T18:08:14.315 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_port", 2024-03-23T18:08:14.316 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.316 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.316 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.316 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.316 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.316 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.316 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.316 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.316 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.317 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.317 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.317 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.317 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.317 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.317 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:14.317 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rgw", 2024-03-23T18:08:14.317 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:14.317 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:14.318 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:14.318 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:14.318 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:14.318 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.318 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.319 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.319 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.319 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.319 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.319 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.319 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.319 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.319 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.320 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.320 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.320 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.320 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.320 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.320 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.320 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.320 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.321 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.321 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:14.321 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:14.321 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.321 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.321 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.321 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.321 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.322 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.322 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.322 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.322 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.322 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.322 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.322 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.322 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:14.322 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:14.323 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.323 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.323 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.323 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:14.323 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.323 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.323 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.323 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.324 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.324 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.324 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.324 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.324 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.324 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.324 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.324 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.324 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.325 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.325 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.325 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:14.325 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:14.325 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.325 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.325 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.325 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.325 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.326 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.326 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.326 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.326 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.326 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.326 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.326 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.326 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.327 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.327 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:14.327 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rook", 2024-03-23T18:08:14.327 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:14.327 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:14.327 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:14.327 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:14.327 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:14.327 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.328 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.328 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.328 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.328 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.328 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.328 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.328 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.328 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.329 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.329 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.329 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.329 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.329 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.329 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.330 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.330 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.330 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.330 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.330 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:14.330 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:14.330 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.330 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.330 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.331 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.331 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.331 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.331 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.331 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.331 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.332 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.332 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.332 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.332 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:14.332 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:14.332 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.332 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.332 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.332 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:14.333 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.333 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.333 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.333 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.333 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.333 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.333 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.333 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.333 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.334 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.334 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.334 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.334 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.334 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.334 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.334 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:14.334 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:14.335 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.335 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.335 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.335 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.335 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.335 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.335 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.335 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.335 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.336 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.336 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.336 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.336 INFO:teuthology.orchestra.run.smithi086.stdout: "storage_class": { 2024-03-23T18:08:14.336 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "storage_class", 2024-03-23T18:08:14.336 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.336 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.337 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.337 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "local", 2024-03-23T18:08:14.337 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.337 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.337 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.337 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-23T18:08:14.337 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.337 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.337 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.338 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.338 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.338 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.338 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:14.338 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "selftest", 2024-03-23T18:08:14.338 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:14.338 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:14.338 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:14.338 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:14.339 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:14.339 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.339 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.339 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.339 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.339 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.340 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.340 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.340 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.340 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.340 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.340 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.340 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.340 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.340 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.341 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.341 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.341 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.341 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.341 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.341 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:14.342 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:14.342 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.342 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.342 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.342 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.342 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.342 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.342 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.342 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.343 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.343 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.343 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.343 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.343 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:14.343 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:14.343 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.343 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.344 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.344 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:14.344 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.344 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.344 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.344 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.344 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.344 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.344 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.345 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.345 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.345 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.345 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.345 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.345 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.345 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.345 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.345 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:14.346 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:14.346 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.346 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.346 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.346 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.346 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.346 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.346 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.346 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.347 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.347 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.347 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.347 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.347 INFO:teuthology.orchestra.run.smithi086.stdout: "roption1": { 2024-03-23T18:08:14.347 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "roption1", 2024-03-23T18:08:14.347 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.348 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.348 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.348 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.348 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.348 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.348 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.348 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.348 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.349 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.349 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.349 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.349 INFO:teuthology.orchestra.run.smithi086.stdout: "roption2": { 2024-03-23T18:08:14.349 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "roption2", 2024-03-23T18:08:14.349 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.349 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.349 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.349 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "xyz", 2024-03-23T18:08:14.350 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.350 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.350 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.350 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.350 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.351 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.351 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.351 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.351 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption1": { 2024-03-23T18:08:14.351 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption1", 2024-03-23T18:08:14.351 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.351 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.351 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.351 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.352 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.352 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.352 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.352 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.352 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.352 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.353 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.353 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.353 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption2": { 2024-03-23T18:08:14.353 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption2", 2024-03-23T18:08:14.353 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:14.353 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.353 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.353 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.353 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.354 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.354 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.354 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.354 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.354 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.354 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.354 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.354 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption3": { 2024-03-23T18:08:14.354 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption3", 2024-03-23T18:08:14.355 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-03-23T18:08:14.355 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.355 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.355 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.355 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.355 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.355 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.355 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.355 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.356 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.356 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.356 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.356 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption4": { 2024-03-23T18:08:14.356 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption4", 2024-03-23T18:08:14.356 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.356 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.356 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.356 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.357 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.357 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.357 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.357 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.357 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.357 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.357 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.358 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.358 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption5": { 2024-03-23T18:08:14.358 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption5", 2024-03-23T18:08:14.358 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.358 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.358 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.358 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.358 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.358 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.359 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.359 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.359 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.359 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.359 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.359 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.359 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption6": { 2024-03-23T18:08:14.359 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption6", 2024-03-23T18:08:14.359 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.360 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.360 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.360 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:14.360 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.360 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.360 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.360 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.361 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.361 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.361 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.361 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.361 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption7": { 2024-03-23T18:08:14.361 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption7", 2024-03-23T18:08:14.362 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:14.362 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.362 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.362 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.362 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-03-23T18:08:14.362 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "42", 2024-03-23T18:08:14.362 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.363 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.363 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.363 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.363 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.363 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.363 INFO:teuthology.orchestra.run.smithi086.stdout: "testkey": { 2024-03-23T18:08:14.363 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "testkey", 2024-03-23T18:08:14.363 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.364 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.364 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.364 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.364 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.364 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.364 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.364 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.364 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.364 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.365 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.365 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.365 INFO:teuthology.orchestra.run.smithi086.stdout: "testlkey": { 2024-03-23T18:08:14.365 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "testlkey", 2024-03-23T18:08:14.365 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.365 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.365 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.365 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.366 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.366 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.366 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.366 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.366 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.366 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.366 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.366 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.366 INFO:teuthology.orchestra.run.smithi086.stdout: "testnewline": { 2024-03-23T18:08:14.367 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "testnewline", 2024-03-23T18:08:14.367 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.367 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.367 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.367 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.367 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.367 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.367 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.368 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.368 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.368 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.368 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.368 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.368 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.368 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.368 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:14.368 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "snap_schedule", 2024-03-23T18:08:14.369 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:14.369 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:14.369 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:14.369 INFO:teuthology.orchestra.run.smithi086.stdout: "allow_m_granularity": { 2024-03-23T18:08:14.369 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "allow_m_granularity", 2024-03-23T18:08:14.369 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.369 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.369 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.370 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.370 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.370 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.370 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.370 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "allow minute scheduled snapshots", 2024-03-23T18:08:14.370 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.370 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.370 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.370 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.371 INFO:teuthology.orchestra.run.smithi086.stdout: "dump_on_update": { 2024-03-23T18:08:14.371 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "dump_on_update", 2024-03-23T18:08:14.371 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.371 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.371 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.371 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.372 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.372 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.372 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.372 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "dump database to debug log on update", 2024-03-23T18:08:14.372 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.373 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.373 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.373 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.373 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:14.374 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:14.374 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.374 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.374 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.375 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.375 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.375 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.375 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.376 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.376 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.376 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.376 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.377 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.377 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.377 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.377 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.378 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.378 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.378 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.378 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.378 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:14.378 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:14.378 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.378 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.378 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.379 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.379 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.379 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.379 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.379 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.379 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.379 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.379 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.379 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.380 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:14.380 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:14.380 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.380 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.380 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.380 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:14.380 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.380 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.380 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.381 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.381 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.381 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.381 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.381 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.381 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.381 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.381 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.382 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.382 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.382 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.382 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.382 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:14.382 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:14.382 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.382 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.382 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.383 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.383 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.383 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.383 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.383 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.383 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.383 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.384 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.384 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.384 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.384 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.384 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:14.384 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "stats", 2024-03-23T18:08:14.384 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:14.384 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:14.385 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:14.385 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:14.385 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:14.385 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.385 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.385 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.385 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.385 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.386 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.386 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.386 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.386 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.386 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.386 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.386 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.386 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.387 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.387 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.387 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.387 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.387 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.387 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.387 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:14.387 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:14.388 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.388 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.388 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.388 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.388 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.389 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.389 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.389 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.389 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.389 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.389 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.389 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.389 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:14.390 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:14.390 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.390 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.390 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.390 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:14.390 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.390 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.390 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.390 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.391 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.391 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.391 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.391 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.391 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.391 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.391 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.391 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.391 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.392 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.392 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.392 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:14.392 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:14.392 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.392 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.392 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.392 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.392 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.393 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.393 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.393 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.393 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.393 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.393 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.393 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.394 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.394 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.394 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:14.394 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "status", 2024-03-23T18:08:14.394 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:14.394 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:14.394 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:14.395 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:14.395 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:14.395 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.395 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.395 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.395 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.395 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.395 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.396 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.396 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.396 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.396 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.396 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.396 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.396 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.396 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.396 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.397 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.397 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.397 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.397 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.397 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:14.397 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:14.397 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.397 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.398 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.398 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.398 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.398 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.398 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.398 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.398 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.399 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.399 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.399 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.399 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:14.399 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:14.399 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.399 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.400 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.400 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:14.400 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.400 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.400 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.400 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.400 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.400 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.400 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.401 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.401 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.401 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.401 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.401 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.401 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.401 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.401 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.402 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:14.402 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:14.402 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.402 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.402 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.402 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.402 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.402 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.403 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.403 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.403 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.403 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.403 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.403 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.403 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.403 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.403 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:14.404 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "telegraf", 2024-03-23T18:08:14.404 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:14.404 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:14.404 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:14.404 INFO:teuthology.orchestra.run.smithi086.stdout: "address": { 2024-03-23T18:08:14.404 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "address", 2024-03-23T18:08:14.404 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.405 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.405 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.405 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-23T18:08:14.405 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.405 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.405 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.405 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.405 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.405 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.406 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.406 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.406 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-03-23T18:08:14.406 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-03-23T18:08:14.406 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-23T18:08:14.406 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.406 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.406 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "15", 2024-03-23T18:08:14.407 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.407 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.407 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.407 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.407 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.407 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.407 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.407 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.407 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:14.408 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:14.408 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.408 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.408 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.408 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.408 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.408 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.408 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.409 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.409 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.409 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.409 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.409 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.409 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.410 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.410 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.410 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.410 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.410 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.410 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.410 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:14.410 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:14.410 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.411 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.411 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.411 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.411 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.411 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.411 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.411 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.411 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.411 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.412 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.412 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.412 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:14.412 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:14.412 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.412 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.412 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.412 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:14.412 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.413 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.413 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.413 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.413 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.413 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.413 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.413 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.413 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.413 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.414 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.414 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.414 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.414 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.414 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.414 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:14.414 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:14.414 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.414 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.415 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.415 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.415 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.415 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.415 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.415 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.415 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.415 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.415 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.416 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.416 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.416 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.416 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:14.416 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "telemetry", 2024-03-23T18:08:14.416 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:14.416 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:14.416 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:14.417 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_basic": { 2024-03-23T18:08:14.417 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_basic", 2024-03-23T18:08:14.417 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.417 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.417 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.417 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:14.417 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.417 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.417 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.418 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-23T18:08:14.418 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.418 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.418 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.418 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.418 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_crash": { 2024-03-23T18:08:14.418 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_crash", 2024-03-23T18:08:14.418 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.419 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.419 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.419 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:14.419 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.419 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.419 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.419 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-23T18:08:14.419 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.420 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.420 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.420 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.420 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_device": { 2024-03-23T18:08:14.420 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_device", 2024-03-23T18:08:14.420 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.420 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.421 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.421 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:14.421 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.421 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.421 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.421 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-23T18:08:14.421 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.421 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.421 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.422 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.422 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_ident": { 2024-03-23T18:08:14.422 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_ident", 2024-03-23T18:08:14.422 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.422 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.422 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.422 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.422 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.423 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.423 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.423 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-23T18:08:14.423 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.423 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.423 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.423 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.423 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_perf": { 2024-03-23T18:08:14.424 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_perf", 2024-03-23T18:08:14.424 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.424 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.424 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.424 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.424 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.424 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.424 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.424 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-23T18:08:14.425 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.425 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.425 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.425 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.425 INFO:teuthology.orchestra.run.smithi086.stdout: "contact": { 2024-03-23T18:08:14.425 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "contact", 2024-03-23T18:08:14.425 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.425 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.425 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.426 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.426 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.426 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.426 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.426 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.426 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.426 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.426 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.427 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.427 INFO:teuthology.orchestra.run.smithi086.stdout: "description": { 2024-03-23T18:08:14.427 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "description", 2024-03-23T18:08:14.427 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.427 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.427 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.427 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.427 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.427 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.428 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.428 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.428 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.428 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.428 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.428 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.428 INFO:teuthology.orchestra.run.smithi086.stdout: "device_url": { 2024-03-23T18:08:14.428 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "device_url", 2024-03-23T18:08:14.429 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.429 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.429 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.429 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-23T18:08:14.429 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.429 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.430 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.430 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.430 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.430 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.430 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.430 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.431 INFO:teuthology.orchestra.run.smithi086.stdout: "enabled": { 2024-03-23T18:08:14.431 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enabled", 2024-03-23T18:08:14.431 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.431 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.431 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.431 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.431 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.431 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.431 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.432 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.432 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.432 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.432 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.432 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.432 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-03-23T18:08:14.433 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-03-23T18:08:14.433 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:14.433 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.433 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.433 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "24", 2024-03-23T18:08:14.433 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "8", 2024-03-23T18:08:14.433 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.433 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.433 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.434 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.434 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.434 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.434 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.434 INFO:teuthology.orchestra.run.smithi086.stdout: "last_opt_revision": { 2024-03-23T18:08:14.434 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "last_opt_revision", 2024-03-23T18:08:14.434 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:14.435 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.435 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.435 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "1", 2024-03-23T18:08:14.435 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.435 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.435 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.435 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.435 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.435 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.436 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.436 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.436 INFO:teuthology.orchestra.run.smithi086.stdout: "leaderboard": { 2024-03-23T18:08:14.436 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "leaderboard", 2024-03-23T18:08:14.436 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.436 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.436 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.436 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.437 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.437 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.437 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.437 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.437 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.437 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.437 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.438 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.438 INFO:teuthology.orchestra.run.smithi086.stdout: "leaderboard_description": { 2024-03-23T18:08:14.438 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "leaderboard_description", 2024-03-23T18:08:14.438 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.438 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.438 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.438 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.438 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.439 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.439 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.439 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.439 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.439 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.439 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.439 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.439 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:14.439 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:14.440 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.440 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.440 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.440 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.440 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.440 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.440 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.440 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.441 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.441 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.441 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.441 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.441 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.441 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.441 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.441 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.442 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.442 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.442 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.442 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:14.442 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:14.442 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.442 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.442 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.443 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.443 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.443 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.443 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.443 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.443 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.443 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.443 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.444 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.444 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:14.444 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:14.444 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.444 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.445 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.445 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:14.445 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.445 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.446 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.446 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.446 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.446 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.447 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.447 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.447 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.447 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.448 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.448 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.448 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.448 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.449 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.449 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:14.449 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:14.449 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.449 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.450 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.450 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.450 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.450 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.451 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.451 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.451 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.451 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.452 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.452 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.452 INFO:teuthology.orchestra.run.smithi086.stdout: "organization": { 2024-03-23T18:08:14.452 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "organization", 2024-03-23T18:08:14.453 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.453 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.453 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.453 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.454 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.454 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.454 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.454 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.454 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.455 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.455 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.455 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.455 INFO:teuthology.orchestra.run.smithi086.stdout: "proxy": { 2024-03-23T18:08:14.456 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "proxy", 2024-03-23T18:08:14.456 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.456 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.456 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.456 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.457 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.457 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.457 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.457 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.458 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.458 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.458 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.458 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.459 INFO:teuthology.orchestra.run.smithi086.stdout: "url": { 2024-03-23T18:08:14.459 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "url", 2024-03-23T18:08:14.459 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.459 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.460 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.460 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-23T18:08:14.460 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.460 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.461 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.461 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.461 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.461 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.462 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.462 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.462 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.462 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.463 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:14.463 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "test_orchestrator", 2024-03-23T18:08:14.463 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:14.464 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:14.464 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:14.464 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:14.464 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:14.464 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.465 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.465 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.465 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.466 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.466 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.466 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.466 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.467 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.467 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.467 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.467 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.468 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.468 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.468 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.468 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.469 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.469 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.469 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.469 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:14.470 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:14.470 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.470 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.470 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.471 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.471 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.471 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.471 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.472 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.472 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.472 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.472 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.473 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.473 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:14.473 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:14.473 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.474 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.474 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.474 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:14.475 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.475 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.475 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.475 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.475 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.476 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.476 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.476 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.476 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.476 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.477 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.477 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.477 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.477 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.477 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.478 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:14.478 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:14.478 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.478 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.478 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.478 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.479 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.479 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.479 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.479 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.479 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.480 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.480 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.480 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.480 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.480 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.481 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:14.481 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "volumes", 2024-03-23T18:08:14.481 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:14.481 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:14.481 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:14.481 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:14.481 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:14.481 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.482 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.482 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.482 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.482 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.482 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.482 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.482 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.482 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.482 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.483 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.483 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.483 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.483 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.483 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.483 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.483 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.483 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.484 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.484 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:14.484 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:14.484 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.484 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.484 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.484 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.484 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.484 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.485 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.485 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.485 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.485 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.485 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.485 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.485 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:14.485 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:14.485 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.486 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.486 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.486 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:14.486 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.486 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.486 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.486 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.487 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.487 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.487 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.487 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.487 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.487 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.487 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.487 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.487 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.488 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.488 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.488 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:14.488 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:14.488 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.488 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.488 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.488 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.489 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.489 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.489 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.489 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.489 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.489 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.489 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.489 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.490 INFO:teuthology.orchestra.run.smithi086.stdout: "max_concurrent_clones": { 2024-03-23T18:08:14.490 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_concurrent_clones", 2024-03-23T18:08:14.490 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:14.490 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.490 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.490 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "4", 2024-03-23T18:08:14.490 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.490 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.490 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.491 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-23T18:08:14.491 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.491 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.491 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.491 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.491 INFO:teuthology.orchestra.run.smithi086.stdout: "snapshot_clone_delay": { 2024-03-23T18:08:14.491 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "snapshot_clone_delay", 2024-03-23T18:08:14.491 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:14.491 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.492 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.492 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-03-23T18:08:14.492 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.492 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.492 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.492 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-23T18:08:14.492 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.492 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.493 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.493 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.493 INFO:teuthology.orchestra.run.smithi086.stdout: "snapshot_clone_no_wait": { 2024-03-23T18:08:14.493 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "snapshot_clone_no_wait", 2024-03-23T18:08:14.493 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.493 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.494 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.494 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:14.494 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.494 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.494 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.495 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-23T18:08:14.495 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.495 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.495 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.496 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.496 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.496 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.496 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:14.496 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix", 2024-03-23T18:08:14.496 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:14.497 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:14.497 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:14.497 INFO:teuthology.orchestra.run.smithi086.stdout: "discovery_interval": { 2024-03-23T18:08:14.497 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "discovery_interval", 2024-03-23T18:08:14.497 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-03-23T18:08:14.497 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.497 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.497 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "100", 2024-03-23T18:08:14.498 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.498 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.498 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.498 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.498 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.498 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.498 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.498 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.498 INFO:teuthology.orchestra.run.smithi086.stdout: "identifier": { 2024-03-23T18:08:14.499 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "identifier", 2024-03-23T18:08:14.499 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.499 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.499 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.499 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.499 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.499 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.499 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.500 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.500 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.500 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.500 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.500 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.500 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-03-23T18:08:14.500 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-03-23T18:08:14.500 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-23T18:08:14.500 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.501 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.501 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-03-23T18:08:14.501 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.501 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.501 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.501 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.501 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.501 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.501 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.502 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.502 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:14.502 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:14.502 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.502 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.502 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.502 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.502 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.502 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.503 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.503 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.503 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.503 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.503 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.503 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.503 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.503 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.503 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.504 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.504 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.504 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.504 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.504 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:14.504 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:14.504 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.504 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.505 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.505 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.505 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.505 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.505 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.505 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.505 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.505 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.505 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.506 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.506 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:14.506 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:14.506 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.506 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.506 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.506 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:14.506 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.507 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.507 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.507 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.507 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.507 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.507 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.507 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.507 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.507 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.508 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.508 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.508 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.508 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.508 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.508 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:14.508 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:14.508 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.508 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.509 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.509 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.509 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.509 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.509 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.509 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.509 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.509 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.510 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.510 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.510 INFO:teuthology.orchestra.run.smithi086.stdout: "zabbix_host": { 2024-03-23T18:08:14.510 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix_host", 2024-03-23T18:08:14.510 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.510 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.510 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.510 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.510 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.511 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.511 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.511 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.511 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.511 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.511 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.511 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.511 INFO:teuthology.orchestra.run.smithi086.stdout: "zabbix_port": { 2024-03-23T18:08:14.511 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix_port", 2024-03-23T18:08:14.512 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:14.512 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.512 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.512 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10051", 2024-03-23T18:08:14.512 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.512 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.512 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.512 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.512 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.513 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.513 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.513 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.513 INFO:teuthology.orchestra.run.smithi086.stdout: "zabbix_sender": { 2024-03-23T18:08:14.513 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix_sender", 2024-03-23T18:08:14.513 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.513 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.513 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.513 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-23T18:08:14.514 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.514 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.514 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.514 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.514 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.514 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.514 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.514 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.515 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.515 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.515 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-03-23T18:08:14.515 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.515 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.515 INFO:teuthology.orchestra.run.smithi086.stdout: "modules": [ 2024-03-23T18:08:14.515 INFO:teuthology.orchestra.run.smithi086.stdout: "cephadm", 2024-03-23T18:08:14.515 INFO:teuthology.orchestra.run.smithi086.stdout: "dashboard", 2024-03-23T18:08:14.515 INFO:teuthology.orchestra.run.smithi086.stdout: "iostat", 2024-03-23T18:08:14.516 INFO:teuthology.orchestra.run.smithi086.stdout: "nfs", 2024-03-23T18:08:14.516 INFO:teuthology.orchestra.run.smithi086.stdout: "restful", 2024-03-23T18:08:14.516 INFO:teuthology.orchestra.run.smithi086.stdout: "snap_schedule" 2024-03-23T18:08:14.516 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.516 INFO:teuthology.orchestra.run.smithi086.stdout: "available_modules": [ 2024-03-23T18:08:14.516 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:14.516 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "alerts", 2024-03-23T18:08:14.516 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:14.516 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:14.517 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:14.517 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-03-23T18:08:14.517 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-03-23T18:08:14.517 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-23T18:08:14.517 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.517 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.517 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-03-23T18:08:14.517 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.517 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.518 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.518 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How frequently to reexamine health status", 2024-03-23T18:08:14.518 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.518 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.518 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.518 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.518 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:14.518 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:14.518 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.519 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.519 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.519 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.519 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.519 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.519 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.519 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.519 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.519 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.520 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.520 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.520 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.520 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.520 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.520 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.520 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.520 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.520 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.521 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:14.521 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:14.521 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.521 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.521 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.521 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.521 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.521 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.521 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.522 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.522 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.522 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.522 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.522 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.522 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:14.522 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:14.522 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.522 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.523 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.523 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:14.523 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.523 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.523 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.523 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.523 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.523 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.523 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.524 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.524 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.524 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.524 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.524 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.524 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.524 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.524 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.524 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:14.525 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:14.525 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.525 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.525 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.525 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.525 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.525 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.525 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.526 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.526 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.526 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.526 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.526 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.526 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_destination": { 2024-03-23T18:08:14.526 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_destination", 2024-03-23T18:08:14.526 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.526 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.527 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.527 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.527 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.527 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.527 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.527 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Email address to send alerts to", 2024-03-23T18:08:14.527 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.527 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.527 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.528 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.528 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_from_name": { 2024-03-23T18:08:14.528 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_from_name", 2024-03-23T18:08:14.528 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.528 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.528 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.528 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "Ceph", 2024-03-23T18:08:14.528 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.528 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.529 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.529 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Email From: name", 2024-03-23T18:08:14.529 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.529 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.529 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.529 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.529 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_host": { 2024-03-23T18:08:14.529 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_host", 2024-03-23T18:08:14.529 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.530 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.530 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.530 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.530 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.530 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.530 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.530 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SMTP server", 2024-03-23T18:08:14.530 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.530 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.531 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.531 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.531 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_password": { 2024-03-23T18:08:14.531 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_password", 2024-03-23T18:08:14.531 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.531 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.531 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.531 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.532 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.532 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.532 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.532 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Password to authenticate with", 2024-03-23T18:08:14.532 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.532 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.532 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.532 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.532 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_port": { 2024-03-23T18:08:14.533 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_port", 2024-03-23T18:08:14.533 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:14.533 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.533 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.533 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "465", 2024-03-23T18:08:14.533 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.533 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.533 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.533 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SMTP port", 2024-03-23T18:08:14.534 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.534 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.534 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.534 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.534 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_sender": { 2024-03-23T18:08:14.534 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_sender", 2024-03-23T18:08:14.534 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.534 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.535 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.535 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.535 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.535 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.535 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.535 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SMTP envelope sender", 2024-03-23T18:08:14.535 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.535 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.536 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.536 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.536 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_ssl": { 2024-03-23T18:08:14.536 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_ssl", 2024-03-23T18:08:14.536 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.536 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.536 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.536 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:14.536 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.537 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.537 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.537 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-23T18:08:14.537 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.537 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.537 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.537 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.537 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_user": { 2024-03-23T18:08:14.537 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_user", 2024-03-23T18:08:14.538 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.538 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.538 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.538 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.538 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.538 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.538 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.538 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "User to authenticate as", 2024-03-23T18:08:14.539 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.539 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.539 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.539 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.539 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.539 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.539 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:14.539 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "balancer", 2024-03-23T18:08:14.539 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:14.540 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:14.540 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:14.540 INFO:teuthology.orchestra.run.smithi086.stdout: "active": { 2024-03-23T18:08:14.540 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "active", 2024-03-23T18:08:14.540 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.540 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.540 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.540 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:14.541 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.541 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.541 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.541 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "automatically balance PGs across cluster", 2024-03-23T18:08:14.541 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.541 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.541 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.541 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.541 INFO:teuthology.orchestra.run.smithi086.stdout: "begin_time": { 2024-03-23T18:08:14.541 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "begin_time", 2024-03-23T18:08:14.542 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.542 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.542 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.542 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0000", 2024-03-23T18:08:14.542 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.542 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.542 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.542 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "beginning time of day to automatically balance", 2024-03-23T18:08:14.543 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-23T18:08:14.543 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.543 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.543 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.543 INFO:teuthology.orchestra.run.smithi086.stdout: "begin_weekday": { 2024-03-23T18:08:14.543 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "begin_weekday", 2024-03-23T18:08:14.543 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-03-23T18:08:14.543 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.543 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.544 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-03-23T18:08:14.544 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "0", 2024-03-23T18:08:14.544 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "6", 2024-03-23T18:08:14.544 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.544 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-23T18:08:14.544 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-23T18:08:14.544 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.544 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.545 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.545 INFO:teuthology.orchestra.run.smithi086.stdout: "crush_compat_max_iterations": { 2024-03-23T18:08:14.545 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crush_compat_max_iterations", 2024-03-23T18:08:14.545 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-03-23T18:08:14.545 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.545 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.545 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "25", 2024-03-23T18:08:14.545 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-03-23T18:08:14.545 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "250", 2024-03-23T18:08:14.546 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.546 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-23T18:08:14.546 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.546 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.546 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.546 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.546 INFO:teuthology.orchestra.run.smithi086.stdout: "crush_compat_metrics": { 2024-03-23T18:08:14.546 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crush_compat_metrics", 2024-03-23T18:08:14.547 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.547 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.547 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.547 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "pgs,objects,bytes", 2024-03-23T18:08:14.547 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.547 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.547 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.547 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-23T18:08:14.547 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-23T18:08:14.548 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.548 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.548 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.548 INFO:teuthology.orchestra.run.smithi086.stdout: "crush_compat_step": { 2024-03-23T18:08:14.548 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crush_compat_step", 2024-03-23T18:08:14.548 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-03-23T18:08:14.548 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.548 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.549 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0.5", 2024-03-23T18:08:14.549 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "0.001", 2024-03-23T18:08:14.549 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "0.999", 2024-03-23T18:08:14.549 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.549 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "aggressiveness of optimization", 2024-03-23T18:08:14.549 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-23T18:08:14.549 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.549 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.549 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.550 INFO:teuthology.orchestra.run.smithi086.stdout: "end_time": { 2024-03-23T18:08:14.550 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "end_time", 2024-03-23T18:08:14.550 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.550 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.550 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.550 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "2359", 2024-03-23T18:08:14.550 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.550 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.551 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.551 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "ending time of day to automatically balance", 2024-03-23T18:08:14.551 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-23T18:08:14.551 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.551 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.551 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.551 INFO:teuthology.orchestra.run.smithi086.stdout: "end_weekday": { 2024-03-23T18:08:14.551 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "end_weekday", 2024-03-23T18:08:14.551 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-03-23T18:08:14.552 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.552 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.552 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-03-23T18:08:14.552 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "0", 2024-03-23T18:08:14.552 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "6", 2024-03-23T18:08:14.552 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.552 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-23T18:08:14.552 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-23T18:08:14.552 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.553 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.553 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.553 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:14.553 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:14.553 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.553 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.553 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.553 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.554 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.554 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.554 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.554 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.554 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.554 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.554 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.554 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.554 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.555 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.555 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.555 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.555 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.555 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.555 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.555 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:14.555 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:14.555 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.556 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.556 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.556 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.556 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.556 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.556 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.556 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.556 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.556 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.557 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.557 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.557 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:14.557 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:14.557 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.557 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.557 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.558 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:14.558 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.558 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.558 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.558 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.558 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.558 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.558 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.558 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.559 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.559 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.559 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.559 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.559 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.559 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.559 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.559 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:14.559 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:14.560 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.560 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.560 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.560 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.560 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.560 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.560 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.560 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.560 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.561 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.561 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.561 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.561 INFO:teuthology.orchestra.run.smithi086.stdout: "min_score": { 2024-03-23T18:08:14.561 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "min_score", 2024-03-23T18:08:14.561 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-03-23T18:08:14.561 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.561 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.562 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-03-23T18:08:14.562 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.562 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.562 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.562 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-23T18:08:14.562 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.562 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.562 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.563 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.563 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": { 2024-03-23T18:08:14.563 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mode", 2024-03-23T18:08:14.563 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.563 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.563 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.563 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "upmap", 2024-03-23T18:08:14.563 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.563 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.564 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.564 INFO:teuthology.orchestra.run.smithi086.stdout: "crush-compat", 2024-03-23T18:08:14.564 INFO:teuthology.orchestra.run.smithi086.stdout: "none", 2024-03-23T18:08:14.564 INFO:teuthology.orchestra.run.smithi086.stdout: "upmap" 2024-03-23T18:08:14.564 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.564 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Balancer mode", 2024-03-23T18:08:14.564 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.564 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.564 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.565 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.565 INFO:teuthology.orchestra.run.smithi086.stdout: "pool_ids": { 2024-03-23T18:08:14.565 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pool_ids", 2024-03-23T18:08:14.565 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.565 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.565 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.565 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.565 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.566 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.566 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.566 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-23T18:08:14.566 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.566 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.566 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.566 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.566 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-03-23T18:08:14.566 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-03-23T18:08:14.567 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-23T18:08:14.567 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.567 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.567 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-03-23T18:08:14.567 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.567 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.567 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.567 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-23T18:08:14.567 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.568 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.568 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.568 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.568 INFO:teuthology.orchestra.run.smithi086.stdout: "upmap_max_deviation": { 2024-03-23T18:08:14.568 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "upmap_max_deviation", 2024-03-23T18:08:14.568 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:14.568 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.568 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.569 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-03-23T18:08:14.569 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-03-23T18:08:14.569 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.569 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.569 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-23T18:08:14.569 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-23T18:08:14.569 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.569 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.569 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.570 INFO:teuthology.orchestra.run.smithi086.stdout: "upmap_max_optimizations": { 2024-03-23T18:08:14.570 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "upmap_max_optimizations", 2024-03-23T18:08:14.570 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-03-23T18:08:14.570 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.570 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.570 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-03-23T18:08:14.570 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.570 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.570 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.571 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-23T18:08:14.571 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.571 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.571 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.571 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.571 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.571 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.571 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:14.572 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cephadm", 2024-03-23T18:08:14.572 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:14.572 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:14.572 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:14.572 INFO:teuthology.orchestra.run.smithi086.stdout: "agent_down_multiplier": { 2024-03-23T18:08:14.572 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "agent_down_multiplier", 2024-03-23T18:08:14.572 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-03-23T18:08:14.572 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.572 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.573 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "3.0", 2024-03-23T18:08:14.573 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.573 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.573 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.573 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-23T18:08:14.573 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.573 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.573 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.573 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.574 INFO:teuthology.orchestra.run.smithi086.stdout: "agent_refresh_rate": { 2024-03-23T18:08:14.574 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "agent_refresh_rate", 2024-03-23T18:08:14.574 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-23T18:08:14.574 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.574 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.574 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "20", 2024-03-23T18:08:14.574 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.574 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.575 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.575 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-23T18:08:14.575 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.575 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.575 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.575 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.575 INFO:teuthology.orchestra.run.smithi086.stdout: "agent_starting_port": { 2024-03-23T18:08:14.575 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "agent_starting_port", 2024-03-23T18:08:14.575 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:14.576 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.576 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.576 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "4721", 2024-03-23T18:08:14.576 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.576 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.576 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.576 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-23T18:08:14.576 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.576 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.577 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.577 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.577 INFO:teuthology.orchestra.run.smithi086.stdout: "allow_ptrace": { 2024-03-23T18:08:14.577 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "allow_ptrace", 2024-03-23T18:08:14.577 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.577 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.577 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.577 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.577 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.578 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.578 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.578 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-23T18:08:14.578 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-23T18:08:14.578 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.578 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.578 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.578 INFO:teuthology.orchestra.run.smithi086.stdout: "autotune_interval": { 2024-03-23T18:08:14.579 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "autotune_interval", 2024-03-23T18:08:14.579 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-23T18:08:14.579 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.579 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.579 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-03-23T18:08:14.579 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.579 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.579 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.579 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-23T18:08:14.580 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.580 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.580 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.580 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.580 INFO:teuthology.orchestra.run.smithi086.stdout: "autotune_memory_target_ratio": { 2024-03-23T18:08:14.580 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "autotune_memory_target_ratio", 2024-03-23T18:08:14.580 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-03-23T18:08:14.580 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.580 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.581 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0.7", 2024-03-23T18:08:14.581 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.581 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.581 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.581 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-23T18:08:14.581 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.581 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.581 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.582 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.582 INFO:teuthology.orchestra.run.smithi086.stdout: "cgroups_split": { 2024-03-23T18:08:14.582 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cgroups_split", 2024-03-23T18:08:14.582 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.582 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.582 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.582 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:14.582 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.582 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.583 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.583 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-23T18:08:14.583 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.583 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.583 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.583 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.583 INFO:teuthology.orchestra.run.smithi086.stdout: "config_checks_enabled": { 2024-03-23T18:08:14.583 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "config_checks_enabled", 2024-03-23T18:08:14.584 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.584 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.584 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.584 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.584 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.584 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.584 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.584 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-23T18:08:14.584 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.585 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.585 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.585 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.585 INFO:teuthology.orchestra.run.smithi086.stdout: "config_dashboard": { 2024-03-23T18:08:14.585 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "config_dashboard", 2024-03-23T18:08:14.585 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.585 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.585 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.585 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:14.586 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.586 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.586 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.586 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-23T18:08:14.586 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.586 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.586 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.586 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.586 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_alertmanager": { 2024-03-23T18:08:14.587 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_alertmanager", 2024-03-23T18:08:14.587 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.587 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.587 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.587 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-23T18:08:14.587 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.587 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.587 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.588 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-03-23T18:08:14.588 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.588 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.588 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.588 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.588 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_base": { 2024-03-23T18:08:14.588 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_base", 2024-03-23T18:08:14.588 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.589 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.589 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.589 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-23T18:08:14.589 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.589 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.589 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.589 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Container image name, without the tag", 2024-03-23T18:08:14.589 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.589 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.590 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.590 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.590 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_elasticsearch": { 2024-03-23T18:08:14.590 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_elasticsearch", 2024-03-23T18:08:14.590 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.590 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.590 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.590 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-23T18:08:14.590 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.591 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.591 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.591 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "elasticsearch container image", 2024-03-23T18:08:14.591 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.591 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.591 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.591 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.591 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_grafana": { 2024-03-23T18:08:14.591 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_grafana", 2024-03-23T18:08:14.592 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.592 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.592 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.592 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2024-03-23T18:08:14.592 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.592 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.592 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.592 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-03-23T18:08:14.592 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.593 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.593 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.593 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.593 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_haproxy": { 2024-03-23T18:08:14.593 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_haproxy", 2024-03-23T18:08:14.593 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.593 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.593 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.593 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-23T18:08:14.594 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.594 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.594 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.594 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "HAproxy container image", 2024-03-23T18:08:14.594 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.594 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.594 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.594 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.595 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_jaeger_agent": { 2024-03-23T18:08:14.595 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_jaeger_agent", 2024-03-23T18:08:14.595 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.595 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.595 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.595 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-23T18:08:14.595 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.595 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.595 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.596 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Jaeger agent container image", 2024-03-23T18:08:14.596 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.596 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.596 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.596 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.596 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_jaeger_collector": { 2024-03-23T18:08:14.596 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_jaeger_collector", 2024-03-23T18:08:14.596 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.597 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.597 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.597 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-23T18:08:14.597 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.597 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.597 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.597 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Jaeger collector container image", 2024-03-23T18:08:14.597 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.598 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.598 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.598 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.598 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_jaeger_query": { 2024-03-23T18:08:14.598 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_jaeger_query", 2024-03-23T18:08:14.598 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.598 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.598 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.599 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-23T18:08:14.599 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.599 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.599 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.599 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Jaeger query container image", 2024-03-23T18:08:14.599 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.599 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.599 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.599 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.600 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_keepalived": { 2024-03-23T18:08:14.600 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_keepalived", 2024-03-23T18:08:14.600 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.600 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.600 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.600 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-23T18:08:14.600 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.600 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.601 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.601 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Keepalived container image", 2024-03-23T18:08:14.601 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.601 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.601 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.601 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.601 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_loki": { 2024-03-23T18:08:14.601 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_loki", 2024-03-23T18:08:14.601 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.602 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.602 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.602 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-23T18:08:14.602 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.602 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.602 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.602 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Loki container image", 2024-03-23T18:08:14.602 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.603 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.603 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.603 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.603 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_node_exporter": { 2024-03-23T18:08:14.603 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_node_exporter", 2024-03-23T18:08:14.603 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.603 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.603 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.603 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-23T18:08:14.603 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.604 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.604 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.604 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-03-23T18:08:14.604 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.604 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.604 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.604 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.604 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_nvmeof": { 2024-03-23T18:08:14.605 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_nvmeof", 2024-03-23T18:08:14.605 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.605 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.605 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.605 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-23T18:08:14.605 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.605 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.605 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.605 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Nvme-of container image", 2024-03-23T18:08:14.606 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.606 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.606 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.606 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.606 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_prometheus": { 2024-03-23T18:08:14.606 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_prometheus", 2024-03-23T18:08:14.606 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.606 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.607 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.607 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-23T18:08:14.607 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.607 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.607 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.607 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-03-23T18:08:14.607 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.607 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.608 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.608 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.608 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_promtail": { 2024-03-23T18:08:14.608 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_promtail", 2024-03-23T18:08:14.608 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.608 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.608 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.608 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-23T18:08:14.608 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.609 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.609 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.609 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Promtail container image", 2024-03-23T18:08:14.609 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.609 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.609 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.609 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.609 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_snmp_gateway": { 2024-03-23T18:08:14.609 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_snmp_gateway", 2024-03-23T18:08:14.610 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.610 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.610 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.610 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-23T18:08:14.610 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.610 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.610 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.610 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SNMP Gateway container image", 2024-03-23T18:08:14.611 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.611 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.611 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.611 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.611 INFO:teuthology.orchestra.run.smithi086.stdout: "container_init": { 2024-03-23T18:08:14.611 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_init", 2024-03-23T18:08:14.611 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.611 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.611 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.612 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:14.612 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.612 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.612 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.612 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Run podman/docker with `--init`", 2024-03-23T18:08:14.612 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.612 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.612 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.612 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.613 INFO:teuthology.orchestra.run.smithi086.stdout: "daemon_cache_timeout": { 2024-03-23T18:08:14.613 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "daemon_cache_timeout", 2024-03-23T18:08:14.613 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-23T18:08:14.613 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.613 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.613 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-03-23T18:08:14.613 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.613 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.613 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.614 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-23T18:08:14.614 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.614 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.614 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.614 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.614 INFO:teuthology.orchestra.run.smithi086.stdout: "default_cephadm_command_timeout": { 2024-03-23T18:08:14.614 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "default_cephadm_command_timeout", 2024-03-23T18:08:14.614 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:14.614 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.615 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.615 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "900", 2024-03-23T18:08:14.615 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.615 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.615 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.615 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-23T18:08:14.615 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.615 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.615 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.616 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.616 INFO:teuthology.orchestra.run.smithi086.stdout: "default_registry": { 2024-03-23T18:08:14.616 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "default_registry", 2024-03-23T18:08:14.616 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.616 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.616 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.616 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io", 2024-03-23T18:08:14.616 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.616 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.617 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.617 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-23T18:08:14.617 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.617 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.617 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.617 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.617 INFO:teuthology.orchestra.run.smithi086.stdout: "device_cache_timeout": { 2024-03-23T18:08:14.617 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "device_cache_timeout", 2024-03-23T18:08:14.617 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-23T18:08:14.618 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.618 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.618 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "1800", 2024-03-23T18:08:14.618 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.618 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.618 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.618 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "seconds to cache device inventory", 2024-03-23T18:08:14.618 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.619 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.619 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.619 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.619 INFO:teuthology.orchestra.run.smithi086.stdout: "device_enhanced_scan": { 2024-03-23T18:08:14.619 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "device_enhanced_scan", 2024-03-23T18:08:14.619 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.619 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.619 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.619 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.620 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.620 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.620 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.620 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-23T18:08:14.620 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.620 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.620 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.620 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.620 INFO:teuthology.orchestra.run.smithi086.stdout: "facts_cache_timeout": { 2024-03-23T18:08:14.621 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "facts_cache_timeout", 2024-03-23T18:08:14.621 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-23T18:08:14.621 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.621 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.621 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-03-23T18:08:14.621 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.621 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.621 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.621 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "seconds to cache host facts data", 2024-03-23T18:08:14.622 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.622 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.622 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.622 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.622 INFO:teuthology.orchestra.run.smithi086.stdout: "host_check_interval": { 2024-03-23T18:08:14.622 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "host_check_interval", 2024-03-23T18:08:14.622 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-23T18:08:14.622 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.622 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.623 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-03-23T18:08:14.623 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.623 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.623 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.623 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to perform a host check", 2024-03-23T18:08:14.623 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.623 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.623 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.624 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.624 INFO:teuthology.orchestra.run.smithi086.stdout: "hw_monitoring": { 2024-03-23T18:08:14.624 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "hw_monitoring", 2024-03-23T18:08:14.624 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.624 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.624 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.624 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.624 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.624 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.625 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.625 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-23T18:08:14.625 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.625 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.625 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.625 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.625 INFO:teuthology.orchestra.run.smithi086.stdout: "inventory_list_all": { 2024-03-23T18:08:14.625 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "inventory_list_all", 2024-03-23T18:08:14.625 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.626 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.626 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.626 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.626 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.626 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.626 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.626 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-23T18:08:14.626 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.627 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.627 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.627 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.627 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:14.627 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:14.627 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.627 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.627 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.627 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.628 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.628 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.628 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.628 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.628 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.628 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.628 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.628 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.628 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.629 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.629 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.629 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.629 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.629 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.629 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.629 INFO:teuthology.orchestra.run.smithi086.stdout: "log_refresh_metadata": { 2024-03-23T18:08:14.629 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_refresh_metadata", 2024-03-23T18:08:14.630 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.630 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.630 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.630 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.630 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.630 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.630 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.630 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-23T18:08:14.630 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.631 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.631 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.631 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.631 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:14.631 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:14.631 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.631 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.631 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.631 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:14.632 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.632 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.632 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.632 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-23T18:08:14.632 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.632 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.632 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.632 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.632 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:14.633 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:14.633 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.633 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.633 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.633 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:14.633 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.633 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.633 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.634 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.634 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.634 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.634 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.634 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.634 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.634 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.634 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.634 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.635 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.635 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.635 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.635 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:14.635 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:14.635 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.635 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.635 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.636 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.636 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.636 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.636 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.636 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.636 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.636 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.636 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.636 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.637 INFO:teuthology.orchestra.run.smithi086.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-23T18:08:14.637 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-23T18:08:14.637 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.637 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.637 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.637 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.637 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.637 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.637 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.638 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-23T18:08:14.638 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.638 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.638 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.638 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.638 INFO:teuthology.orchestra.run.smithi086.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-23T18:08:14.638 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-23T18:08:14.638 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.639 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.639 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.639 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "*", 2024-03-23T18:08:14.639 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.639 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.639 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.639 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-23T18:08:14.639 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.639 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.640 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.640 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.640 INFO:teuthology.orchestra.run.smithi086.stdout: "max_count_per_host": { 2024-03-23T18:08:14.640 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_count_per_host", 2024-03-23T18:08:14.640 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:14.640 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.640 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.640 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-03-23T18:08:14.641 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.641 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.641 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.641 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "max number of daemons per service per host", 2024-03-23T18:08:14.641 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.641 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.641 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.641 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.641 INFO:teuthology.orchestra.run.smithi086.stdout: "max_osd_draining_count": { 2024-03-23T18:08:14.642 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_osd_draining_count", 2024-03-23T18:08:14.642 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:14.642 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.642 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.642 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-03-23T18:08:14.642 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.642 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.642 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.642 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-23T18:08:14.643 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.643 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.643 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.643 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.643 INFO:teuthology.orchestra.run.smithi086.stdout: "migration_current": { 2024-03-23T18:08:14.643 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "migration_current", 2024-03-23T18:08:14.643 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:14.643 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.644 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.644 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.644 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.644 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.644 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.644 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "internal - do not modify", 2024-03-23T18:08:14.644 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.644 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.644 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.645 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.645 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": { 2024-03-23T18:08:14.645 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mode", 2024-03-23T18:08:14.645 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.645 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.645 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.645 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "root", 2024-03-23T18:08:14.645 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.646 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.646 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.646 INFO:teuthology.orchestra.run.smithi086.stdout: "cephadm-package", 2024-03-23T18:08:14.646 INFO:teuthology.orchestra.run.smithi086.stdout: "root" 2024-03-23T18:08:14.646 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.646 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "mode for remote execution of cephadm", 2024-03-23T18:08:14.646 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.646 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.646 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.647 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.647 INFO:teuthology.orchestra.run.smithi086.stdout: "oob_default_addr": { 2024-03-23T18:08:14.647 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "oob_default_addr", 2024-03-23T18:08:14.647 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.647 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.647 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.647 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "169.254.1.1", 2024-03-23T18:08:14.647 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.647 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.648 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.648 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-23T18:08:14.648 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.648 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.648 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.648 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.648 INFO:teuthology.orchestra.run.smithi086.stdout: "prometheus_alerts_path": { 2024-03-23T18:08:14.648 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "prometheus_alerts_path", 2024-03-23T18:08:14.648 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.649 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.649 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.649 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-23T18:08:14.649 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.649 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.649 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.649 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-23T18:08:14.649 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.650 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.650 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.650 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.650 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_insecure": { 2024-03-23T18:08:14.650 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_insecure", 2024-03-23T18:08:14.650 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.650 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.650 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.650 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.651 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.651 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.651 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.651 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-23T18:08:14.651 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.651 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.651 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.651 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.652 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_password": { 2024-03-23T18:08:14.652 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_password", 2024-03-23T18:08:14.652 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.652 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.652 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.652 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.652 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.652 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.652 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.653 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-23T18:08:14.653 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.653 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.653 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.653 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.653 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_url": { 2024-03-23T18:08:14.653 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_url", 2024-03-23T18:08:14.653 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.653 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.654 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.654 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.654 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.654 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.654 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.654 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-23T18:08:14.654 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.654 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.655 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.655 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.655 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_username": { 2024-03-23T18:08:14.655 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_username", 2024-03-23T18:08:14.655 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.655 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.655 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.655 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.655 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.656 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.656 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.656 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-23T18:08:14.656 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.656 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.656 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.656 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.656 INFO:teuthology.orchestra.run.smithi086.stdout: "secure_monitoring_stack": { 2024-03-23T18:08:14.656 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "secure_monitoring_stack", 2024-03-23T18:08:14.657 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.657 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.657 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.657 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.657 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.657 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.657 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.657 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-23T18:08:14.658 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.658 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.658 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.658 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.658 INFO:teuthology.orchestra.run.smithi086.stdout: "service_discovery_port": { 2024-03-23T18:08:14.658 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "service_discovery_port", 2024-03-23T18:08:14.658 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:14.658 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.659 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.659 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8765", 2024-03-23T18:08:14.659 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.659 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.659 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.659 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "cephadm service discovery port", 2024-03-23T18:08:14.659 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.659 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.659 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.660 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.660 INFO:teuthology.orchestra.run.smithi086.stdout: "ssh_config_file": { 2024-03-23T18:08:14.660 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssh_config_file", 2024-03-23T18:08:14.660 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.660 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.660 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.660 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.660 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.661 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.661 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.661 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-23T18:08:14.661 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.661 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.661 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.661 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.661 INFO:teuthology.orchestra.run.smithi086.stdout: "use_agent": { 2024-03-23T18:08:14.661 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "use_agent", 2024-03-23T18:08:14.662 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.662 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.662 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.662 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.662 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.662 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.662 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.662 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-23T18:08:14.663 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.663 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.663 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.663 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.663 INFO:teuthology.orchestra.run.smithi086.stdout: "use_repo_digest": { 2024-03-23T18:08:14.663 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "use_repo_digest", 2024-03-23T18:08:14.663 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.663 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.663 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.663 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:14.664 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.664 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.664 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.664 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-23T18:08:14.664 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.664 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.664 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.664 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.665 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_on_failed_host_check": { 2024-03-23T18:08:14.665 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_on_failed_host_check", 2024-03-23T18:08:14.665 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.665 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.665 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.665 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:14.665 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.665 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.665 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.666 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise a health warning if the host check fails", 2024-03-23T18:08:14.666 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.666 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.666 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.666 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.666 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_on_stray_daemons": { 2024-03-23T18:08:14.666 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_on_stray_daemons", 2024-03-23T18:08:14.666 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.666 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.667 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.667 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:14.667 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.667 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.667 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.667 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-23T18:08:14.667 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.667 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.667 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.668 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.668 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_on_stray_hosts": { 2024-03-23T18:08:14.668 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_on_stray_hosts", 2024-03-23T18:08:14.668 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.668 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.668 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.668 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:14.668 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.669 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.669 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.669 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-23T18:08:14.669 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.669 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.669 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.669 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.669 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.669 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.670 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:14.670 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crash", 2024-03-23T18:08:14.670 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:14.670 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:14.670 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:14.670 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:14.670 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:14.670 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.670 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.671 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.671 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.671 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.671 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.671 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.671 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.671 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.671 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.671 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.672 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.672 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.672 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.672 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.672 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.672 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.672 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.672 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.673 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:14.673 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:14.673 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.673 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.673 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.673 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.673 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.673 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.673 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.674 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.674 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.674 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.674 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.674 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.674 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:14.674 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:14.674 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.675 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.675 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.675 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:14.675 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.675 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.675 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.675 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.675 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.675 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.676 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.676 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.676 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.676 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.676 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.676 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.676 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.676 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.676 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.677 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:14.677 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:14.677 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.677 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.677 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.677 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.677 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.677 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.678 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.678 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.678 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.678 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.678 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.678 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.678 INFO:teuthology.orchestra.run.smithi086.stdout: "retain_interval": { 2024-03-23T18:08:14.678 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "retain_interval", 2024-03-23T18:08:14.679 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-23T18:08:14.679 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.679 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.679 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "31536000", 2024-03-23T18:08:14.679 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.679 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.679 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.679 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-23T18:08:14.679 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.680 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.680 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.680 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.680 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_recent_interval": { 2024-03-23T18:08:14.680 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_recent_interval", 2024-03-23T18:08:14.680 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-23T18:08:14.680 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.680 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.680 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "1209600", 2024-03-23T18:08:14.681 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.681 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.681 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.681 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-23T18:08:14.681 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.681 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.681 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.681 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.681 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.682 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.682 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:14.682 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "dashboard", 2024-03-23T18:08:14.682 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:14.682 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:14.682 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:14.682 INFO:teuthology.orchestra.run.smithi086.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-23T18:08:14.682 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-23T18:08:14.683 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:14.683 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.683 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.683 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-03-23T18:08:14.683 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.683 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.683 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.683 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.683 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.684 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.684 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.684 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.684 INFO:teuthology.orchestra.run.smithi086.stdout: "ALERTMANAGER_API_HOST": { 2024-03-23T18:08:14.684 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-23T18:08:14.684 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.684 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.684 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.684 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.685 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.685 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.685 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.685 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.685 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.685 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.685 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.685 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.685 INFO:teuthology.orchestra.run.smithi086.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-23T18:08:14.686 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-23T18:08:14.686 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.686 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.686 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.686 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:14.686 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.686 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.686 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.687 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.687 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.687 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.687 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.687 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.687 INFO:teuthology.orchestra.run.smithi086.stdout: "AUDIT_API_ENABLED": { 2024-03-23T18:08:14.687 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "AUDIT_API_ENABLED", 2024-03-23T18:08:14.687 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.688 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.688 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.688 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.688 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.688 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.688 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.688 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.688 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.688 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.689 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.689 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.689 INFO:teuthology.orchestra.run.smithi086.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-23T18:08:14.689 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-23T18:08:14.689 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.689 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.689 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.689 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:14.689 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.690 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.690 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.690 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.690 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.690 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.690 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.690 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.690 INFO:teuthology.orchestra.run.smithi086.stdout: "ENABLE_BROWSABLE_API": { 2024-03-23T18:08:14.690 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-23T18:08:14.691 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.691 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.691 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.691 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:14.691 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.691 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.691 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.691 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.691 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.692 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.692 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.692 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.692 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-23T18:08:14.692 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-23T18:08:14.692 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.692 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.692 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.693 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:14.693 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.693 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.693 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.693 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.693 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.693 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.693 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.693 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.694 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-23T18:08:14.694 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-23T18:08:14.694 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.694 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.694 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.694 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:14.694 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.694 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.694 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.695 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.695 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.695 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.695 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.695 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.695 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-23T18:08:14.695 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-23T18:08:14.695 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.695 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.696 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.696 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:14.696 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.696 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.696 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.696 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.696 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.696 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.697 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.697 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.697 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-23T18:08:14.697 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-23T18:08:14.697 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.697 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.697 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.697 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:14.697 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.698 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.698 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.698 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.698 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.698 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.698 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.698 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.698 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-23T18:08:14.698 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-23T18:08:14.699 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.699 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.699 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.699 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:14.699 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.699 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.699 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.699 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.700 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.700 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.700 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.700 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.700 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-23T18:08:14.700 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-23T18:08:14.700 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.700 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.700 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.701 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:14.701 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.701 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.701 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.701 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.701 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.701 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.701 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.701 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.702 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-23T18:08:14.702 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-23T18:08:14.702 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.702 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.702 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.702 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:14.702 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.702 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.703 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.703 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.703 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.703 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.703 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.703 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.703 INFO:teuthology.orchestra.run.smithi086.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-23T18:08:14.703 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-23T18:08:14.703 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.703 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.704 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.704 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.704 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.704 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.704 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.704 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.704 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.704 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.705 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.705 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.705 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_PASSWORD": { 2024-03-23T18:08:14.705 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-23T18:08:14.705 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.705 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.705 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.705 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "admin", 2024-03-23T18:08:14.705 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.706 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.706 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.706 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.706 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.706 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.706 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.706 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.706 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-23T18:08:14.707 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-23T18:08:14.707 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.707 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.707 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.707 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:14.707 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.707 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.707 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.707 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.708 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.708 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.708 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.708 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.708 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_URL": { 2024-03-23T18:08:14.708 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_URL", 2024-03-23T18:08:14.708 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.708 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.708 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.709 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.709 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.709 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.709 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.709 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.709 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.709 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.709 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.710 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.710 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_USERNAME": { 2024-03-23T18:08:14.710 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-23T18:08:14.710 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.710 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.710 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.710 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "admin", 2024-03-23T18:08:14.710 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.710 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.711 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.711 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.711 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.711 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.711 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.711 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.711 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-23T18:08:14.711 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-23T18:08:14.711 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.712 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.712 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.712 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.712 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.712 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.712 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.712 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.712 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.712 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.713 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.713 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.713 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-23T18:08:14.713 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-23T18:08:14.713 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.713 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.713 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.713 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.713 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.714 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.714 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.714 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.714 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.714 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.714 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.714 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.714 INFO:teuthology.orchestra.run.smithi086.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-23T18:08:14.715 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-23T18:08:14.715 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.715 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.715 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.715 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:14.715 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.715 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.715 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.715 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.716 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.716 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.716 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.716 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.716 INFO:teuthology.orchestra.run.smithi086.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-23T18:08:14.716 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-23T18:08:14.716 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.716 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.717 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.717 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.717 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.717 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.717 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.717 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.717 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.717 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.717 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.718 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.718 INFO:teuthology.orchestra.run.smithi086.stdout: "PROMETHEUS_API_HOST": { 2024-03-23T18:08:14.718 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-23T18:08:14.718 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.718 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.718 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.718 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.718 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.718 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.719 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.719 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.719 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.719 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.719 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.719 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.719 INFO:teuthology.orchestra.run.smithi086.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-23T18:08:14.719 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-23T18:08:14.719 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.720 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.720 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.720 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:14.720 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.720 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.720 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.720 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.720 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.721 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.721 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.721 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.721 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-23T18:08:14.721 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-23T18:08:14.721 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.721 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.721 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.721 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.722 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.722 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.722 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.722 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.722 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.722 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.722 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.722 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.723 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-23T18:08:14.723 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-23T18:08:14.723 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.723 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.723 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.723 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.723 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.723 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.723 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.724 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.724 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.724 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.724 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.724 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.724 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-23T18:08:14.724 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-23T18:08:14.724 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.724 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.725 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.725 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:14.725 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.725 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.725 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.725 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.725 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.725 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.726 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.726 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.726 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-23T18:08:14.726 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-23T18:08:14.726 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.726 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.726 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.726 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:14.726 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.727 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.727 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.727 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.727 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.727 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.727 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.727 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.727 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-23T18:08:14.728 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-23T18:08:14.728 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.728 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.728 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.728 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.728 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.728 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.728 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.728 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.729 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.729 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.729 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.729 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.729 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-23T18:08:14.729 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-23T18:08:14.729 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.729 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.730 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.730 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.730 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.730 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.730 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.730 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.730 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.730 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.730 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.731 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.731 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-23T18:08:14.731 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-23T18:08:14.731 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.731 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.731 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.731 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.731 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.731 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.732 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.732 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.732 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.732 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.732 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.732 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.732 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_ENABLED": { 2024-03-23T18:08:14.732 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-23T18:08:14.733 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.733 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.733 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.733 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:14.733 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.733 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.733 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.733 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.733 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.734 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.734 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.734 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.734 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-23T18:08:14.734 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-23T18:08:14.734 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.734 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.734 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.735 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-23T18:08:14.735 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.735 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.735 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.735 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.735 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.735 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.736 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.736 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.736 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-23T18:08:14.736 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-23T18:08:14.736 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:14.736 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.736 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.736 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-03-23T18:08:14.736 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.737 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.737 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.737 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.737 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.738 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.738 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.738 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.738 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-23T18:08:14.738 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-23T18:08:14.739 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:14.739 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.739 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.739 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8", 2024-03-23T18:08:14.739 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.740 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.740 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.740 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.740 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.740 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.740 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.741 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.741 INFO:teuthology.orchestra.run.smithi086.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-23T18:08:14.741 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-23T18:08:14.741 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:14.741 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.741 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.741 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "45", 2024-03-23T18:08:14.741 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.742 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.742 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.742 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.742 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.742 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.742 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.742 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.742 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_ACCESS_KEY": { 2024-03-23T18:08:14.742 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-23T18:08:14.743 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.743 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.743 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.743 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.743 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.743 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.743 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.743 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.743 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.744 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.744 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.744 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.744 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-23T18:08:14.744 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-23T18:08:14.744 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.744 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.744 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.745 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "admin", 2024-03-23T18:08:14.745 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.745 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.745 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.745 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.745 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.745 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.745 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.745 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.746 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_SECRET_KEY": { 2024-03-23T18:08:14.746 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-23T18:08:14.746 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.746 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.746 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.746 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.746 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.746 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.747 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.747 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.747 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.747 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.747 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.747 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.747 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_SSL_VERIFY": { 2024-03-23T18:08:14.747 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-23T18:08:14.748 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.748 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.748 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.748 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:14.748 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.748 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.748 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.748 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.748 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.749 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.749 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.749 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.749 INFO:teuthology.orchestra.run.smithi086.stdout: "UNSAFE_TLS_v1_2": { 2024-03-23T18:08:14.749 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-23T18:08:14.749 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.749 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.749 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.749 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.750 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.750 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.750 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.750 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.750 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.750 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.750 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.751 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.751 INFO:teuthology.orchestra.run.smithi086.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-23T18:08:14.751 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-23T18:08:14.751 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:14.751 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.751 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.751 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-03-23T18:08:14.751 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.751 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.752 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.752 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.752 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.752 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.752 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.752 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.752 INFO:teuthology.orchestra.run.smithi086.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-23T18:08:14.752 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-23T18:08:14.752 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:14.753 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.753 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.753 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-03-23T18:08:14.753 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.753 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.753 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.753 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.753 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.754 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.754 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.754 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.754 INFO:teuthology.orchestra.run.smithi086.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-23T18:08:14.754 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-23T18:08:14.754 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:14.754 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.754 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.754 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-03-23T18:08:14.755 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.755 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.755 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.755 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.755 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.755 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.755 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.755 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.756 INFO:teuthology.orchestra.run.smithi086.stdout: "cross_origin_url": { 2024-03-23T18:08:14.756 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cross_origin_url", 2024-03-23T18:08:14.756 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.756 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.756 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.756 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.756 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.756 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.756 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.756 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.757 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.757 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.757 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.757 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.757 INFO:teuthology.orchestra.run.smithi086.stdout: "crt_file": { 2024-03-23T18:08:14.757 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crt_file", 2024-03-23T18:08:14.757 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.757 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.758 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.758 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.758 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.758 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.758 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.758 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.758 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.758 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.759 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.759 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.759 INFO:teuthology.orchestra.run.smithi086.stdout: "debug": { 2024-03-23T18:08:14.759 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "debug", 2024-03-23T18:08:14.759 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.759 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.759 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.759 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.759 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.760 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.760 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.760 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Enable/disable debug options", 2024-03-23T18:08:14.760 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.760 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.760 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.760 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.760 INFO:teuthology.orchestra.run.smithi086.stdout: "jwt_token_ttl": { 2024-03-23T18:08:14.760 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "jwt_token_ttl", 2024-03-23T18:08:14.761 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:14.761 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.761 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.761 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "28800", 2024-03-23T18:08:14.761 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.761 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.761 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.761 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.762 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.762 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.762 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.762 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.762 INFO:teuthology.orchestra.run.smithi086.stdout: "key_file": { 2024-03-23T18:08:14.762 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "key_file", 2024-03-23T18:08:14.762 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.762 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.762 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.763 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.763 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.763 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.763 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.763 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.763 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.763 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.763 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.764 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.764 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:14.764 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:14.764 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.764 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.764 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.764 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.764 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.764 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.765 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.765 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.765 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.765 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.765 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.765 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.765 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.765 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.765 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.766 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.766 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.766 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.766 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.766 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:14.766 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:14.766 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.766 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.767 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.767 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.767 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.767 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.767 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.767 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.767 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.767 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.768 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.768 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.768 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:14.768 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:14.768 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.768 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.768 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.768 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:14.768 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.769 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.769 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.769 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.769 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.769 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.769 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.769 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.769 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.769 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.770 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.770 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.770 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.770 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.770 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.770 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:14.770 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:14.770 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.771 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.771 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.771 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.771 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.771 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.771 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.771 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.771 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.771 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.771 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.772 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.772 INFO:teuthology.orchestra.run.smithi086.stdout: "motd": { 2024-03-23T18:08:14.772 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "motd", 2024-03-23T18:08:14.772 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.772 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.772 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.772 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.772 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.773 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.773 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.773 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "The message of the day", 2024-03-23T18:08:14.773 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.773 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.773 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.773 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.773 INFO:teuthology.orchestra.run.smithi086.stdout: "redirect_resolve_ip_addr": { 2024-03-23T18:08:14.773 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "redirect_resolve_ip_addr", 2024-03-23T18:08:14.774 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.774 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.774 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.774 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.774 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.774 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.774 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.774 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.775 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.775 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.775 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.775 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.775 INFO:teuthology.orchestra.run.smithi086.stdout: "server_addr": { 2024-03-23T18:08:14.775 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_addr", 2024-03-23T18:08:14.775 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.775 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.775 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.776 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "::", 2024-03-23T18:08:14.776 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.776 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.776 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.776 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.776 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.776 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.776 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.777 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.777 INFO:teuthology.orchestra.run.smithi086.stdout: "server_port": { 2024-03-23T18:08:14.777 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_port", 2024-03-23T18:08:14.777 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:14.777 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.777 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.777 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8080", 2024-03-23T18:08:14.777 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.777 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.778 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.778 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.778 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.778 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.778 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.778 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.778 INFO:teuthology.orchestra.run.smithi086.stdout: "ssl": { 2024-03-23T18:08:14.778 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssl", 2024-03-23T18:08:14.779 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.779 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.779 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.779 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:14.779 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.779 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.779 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.779 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.780 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.780 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.780 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.780 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.780 INFO:teuthology.orchestra.run.smithi086.stdout: "ssl_server_port": { 2024-03-23T18:08:14.780 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssl_server_port", 2024-03-23T18:08:14.780 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:14.780 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.780 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.781 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8443", 2024-03-23T18:08:14.781 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.781 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.781 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.781 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.781 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.781 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.781 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.782 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.782 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_behaviour": { 2024-03-23T18:08:14.782 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_behaviour", 2024-03-23T18:08:14.782 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.782 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.782 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.782 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "redirect", 2024-03-23T18:08:14.782 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.783 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.783 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.783 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.783 INFO:teuthology.orchestra.run.smithi086.stdout: "redirect" 2024-03-23T18:08:14.783 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.783 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.783 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.783 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.783 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.784 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.784 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_error_status_code": { 2024-03-23T18:08:14.784 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_error_status_code", 2024-03-23T18:08:14.784 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:14.784 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.784 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.784 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "500", 2024-03-23T18:08:14.784 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "400", 2024-03-23T18:08:14.784 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "599", 2024-03-23T18:08:14.785 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.785 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.785 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.785 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.785 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.785 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.785 INFO:teuthology.orchestra.run.smithi086.stdout: "url_prefix": { 2024-03-23T18:08:14.785 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "url_prefix", 2024-03-23T18:08:14.786 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.786 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.786 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.786 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.786 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.786 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.786 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.786 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.787 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.787 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.787 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.787 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.787 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.787 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.787 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:14.787 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "devicehealth", 2024-03-23T18:08:14.787 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:14.788 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:14.788 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:14.788 INFO:teuthology.orchestra.run.smithi086.stdout: "enable_monitoring": { 2024-03-23T18:08:14.788 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enable_monitoring", 2024-03-23T18:08:14.788 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.788 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.788 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.788 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:14.788 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.789 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.789 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.789 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "monitor device health metrics", 2024-03-23T18:08:14.789 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.789 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.789 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.789 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.789 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:14.790 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:14.790 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.790 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.790 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.790 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.790 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.790 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.790 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.791 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.791 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.791 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.791 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.791 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.791 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.791 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.791 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.791 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.792 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.792 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.792 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.792 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:14.792 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:14.792 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.792 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.792 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.793 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.793 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.793 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.793 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.793 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.793 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.793 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.793 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.793 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.794 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:14.794 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:14.794 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.794 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.794 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.794 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:14.794 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.794 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.795 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.795 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.795 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.795 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.795 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.795 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.795 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.795 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.795 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.796 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.796 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.796 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.796 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.796 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:14.796 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:14.796 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.796 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.796 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.797 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.797 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.797 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.797 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.797 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.797 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.797 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.797 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.798 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.798 INFO:teuthology.orchestra.run.smithi086.stdout: "mark_out_threshold": { 2024-03-23T18:08:14.798 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mark_out_threshold", 2024-03-23T18:08:14.798 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-23T18:08:14.798 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.798 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.798 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "2419200", 2024-03-23T18:08:14.798 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.798 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.799 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.799 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-23T18:08:14.799 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.799 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.799 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.799 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.799 INFO:teuthology.orchestra.run.smithi086.stdout: "pool_name": { 2024-03-23T18:08:14.799 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pool_name", 2024-03-23T18:08:14.799 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.800 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.800 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.800 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "device_health_metrics", 2024-03-23T18:08:14.800 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.800 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.800 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.800 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-23T18:08:14.800 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.800 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.801 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.801 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.801 INFO:teuthology.orchestra.run.smithi086.stdout: "retention_period": { 2024-03-23T18:08:14.801 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "retention_period", 2024-03-23T18:08:14.801 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-23T18:08:14.801 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.801 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.801 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "15552000", 2024-03-23T18:08:14.802 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.802 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.802 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.802 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how long to retain device health metrics", 2024-03-23T18:08:14.802 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.802 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.802 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.802 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.802 INFO:teuthology.orchestra.run.smithi086.stdout: "scrape_frequency": { 2024-03-23T18:08:14.803 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "scrape_frequency", 2024-03-23T18:08:14.803 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-23T18:08:14.803 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.803 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.803 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "86400", 2024-03-23T18:08:14.803 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.803 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.803 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.803 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-23T18:08:14.804 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.804 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.804 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.804 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.804 INFO:teuthology.orchestra.run.smithi086.stdout: "self_heal": { 2024-03-23T18:08:14.804 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "self_heal", 2024-03-23T18:08:14.804 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.804 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.804 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.805 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:14.805 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.805 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.805 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.805 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-23T18:08:14.805 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.805 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.805 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.805 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.806 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-03-23T18:08:14.806 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-03-23T18:08:14.806 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-23T18:08:14.806 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.806 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.806 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-03-23T18:08:14.806 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.806 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.807 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.807 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to wake up and check device health", 2024-03-23T18:08:14.807 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.807 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.807 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.807 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.807 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_threshold": { 2024-03-23T18:08:14.807 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_threshold", 2024-03-23T18:08:14.807 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-23T18:08:14.807 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.808 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.808 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "7257600", 2024-03-23T18:08:14.808 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.808 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.808 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.808 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-23T18:08:14.808 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.808 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.809 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.809 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.809 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.809 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.809 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:14.809 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "diskprediction_local", 2024-03-23T18:08:14.809 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:14.809 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:14.809 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:14.810 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:14.810 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:14.810 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.810 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.810 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.810 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.810 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.810 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.811 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.811 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.811 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.811 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.811 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.811 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.811 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.811 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.811 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.812 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.812 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.812 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.812 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.812 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:14.812 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:14.812 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.812 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.813 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.813 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.813 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.813 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.813 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.813 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.813 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.813 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.813 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.814 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.814 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:14.814 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:14.814 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.814 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.814 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.814 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:14.814 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.814 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.815 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.815 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.815 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.815 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.815 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.815 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.815 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.815 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.815 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.816 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.816 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.816 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.816 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.816 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:14.816 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:14.816 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.816 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.817 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.817 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.817 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.817 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.817 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.817 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.817 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.817 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.817 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.818 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.818 INFO:teuthology.orchestra.run.smithi086.stdout: "predict_interval": { 2024-03-23T18:08:14.818 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "predict_interval", 2024-03-23T18:08:14.818 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.818 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.818 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.818 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "86400", 2024-03-23T18:08:14.818 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.818 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.819 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.819 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.819 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.819 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.819 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.819 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.819 INFO:teuthology.orchestra.run.smithi086.stdout: "predictor_model": { 2024-03-23T18:08:14.819 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "predictor_model", 2024-03-23T18:08:14.819 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.820 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.820 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.820 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "prophetstor", 2024-03-23T18:08:14.820 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.820 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.820 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.820 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.820 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.820 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.821 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.821 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.821 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-03-23T18:08:14.821 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-03-23T18:08:14.821 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.821 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.821 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.821 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-03-23T18:08:14.822 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.822 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.822 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.822 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.822 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.822 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.822 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.822 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.822 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.823 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.823 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:14.823 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "influx", 2024-03-23T18:08:14.823 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": false, 2024-03-23T18:08:14.823 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "influxdb python module not found", 2024-03-23T18:08:14.823 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:14.823 INFO:teuthology.orchestra.run.smithi086.stdout: "batch_size": { 2024-03-23T18:08:14.823 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "batch_size", 2024-03-23T18:08:14.823 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:14.824 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.824 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.824 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5000", 2024-03-23T18:08:14.824 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.824 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.824 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.824 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-23T18:08:14.824 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.825 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.825 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.825 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.825 INFO:teuthology.orchestra.run.smithi086.stdout: "database": { 2024-03-23T18:08:14.825 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "database", 2024-03-23T18:08:14.825 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.825 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.825 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.825 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "ceph", 2024-03-23T18:08:14.826 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.826 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.826 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.826 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-23T18:08:14.826 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.826 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.826 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.826 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.827 INFO:teuthology.orchestra.run.smithi086.stdout: "hostname": { 2024-03-23T18:08:14.827 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "hostname", 2024-03-23T18:08:14.827 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.827 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.827 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.827 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.827 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.827 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.827 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.828 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "InfluxDB server hostname", 2024-03-23T18:08:14.828 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.828 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.828 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.828 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.828 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-03-23T18:08:14.828 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-03-23T18:08:14.828 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-23T18:08:14.829 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.829 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.829 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "30", 2024-03-23T18:08:14.829 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "5", 2024-03-23T18:08:14.829 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.829 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.829 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-23T18:08:14.829 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.829 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.830 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.830 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.830 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:14.830 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:14.830 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.830 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.830 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.830 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.830 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.831 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.831 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.831 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.831 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.831 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.831 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.831 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.831 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.832 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.832 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.832 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.832 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.832 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.832 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.832 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:14.832 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:14.832 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.833 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.833 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.833 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.833 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.833 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.833 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.833 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.833 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.833 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.834 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.834 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.834 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:14.834 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:14.834 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.834 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.834 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.834 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:14.834 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.835 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.835 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.835 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.835 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.835 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.835 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.835 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.835 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.835 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.836 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.836 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.836 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.836 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.836 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.836 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:14.836 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:14.836 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.836 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.837 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.837 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.837 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.837 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.837 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.837 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.837 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.837 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.837 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.838 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.838 INFO:teuthology.orchestra.run.smithi086.stdout: "password": { 2024-03-23T18:08:14.838 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "password", 2024-03-23T18:08:14.838 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.838 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.838 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.839 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.839 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.839 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.839 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.839 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "password of InfluxDB server user", 2024-03-23T18:08:14.839 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.839 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.839 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.839 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.840 INFO:teuthology.orchestra.run.smithi086.stdout: "port": { 2024-03-23T18:08:14.840 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "port", 2024-03-23T18:08:14.840 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:14.840 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.840 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.840 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8086", 2024-03-23T18:08:14.840 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.840 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.840 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.841 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "InfluxDB server port", 2024-03-23T18:08:14.841 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.841 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.841 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.841 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.841 INFO:teuthology.orchestra.run.smithi086.stdout: "ssl": { 2024-03-23T18:08:14.841 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssl", 2024-03-23T18:08:14.841 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.841 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.842 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.842 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "false", 2024-03-23T18:08:14.842 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.842 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.842 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.842 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-23T18:08:14.842 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.842 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.843 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.843 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.843 INFO:teuthology.orchestra.run.smithi086.stdout: "threads": { 2024-03-23T18:08:14.843 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "threads", 2024-03-23T18:08:14.843 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:14.843 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.843 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.843 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-03-23T18:08:14.843 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-03-23T18:08:14.844 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "32", 2024-03-23T18:08:14.844 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.844 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-23T18:08:14.844 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.844 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.844 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.844 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.844 INFO:teuthology.orchestra.run.smithi086.stdout: "username": { 2024-03-23T18:08:14.844 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "username", 2024-03-23T18:08:14.845 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.845 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.845 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.845 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.845 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.845 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.845 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.845 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "username of InfluxDB server user", 2024-03-23T18:08:14.845 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.846 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.846 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.846 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.846 INFO:teuthology.orchestra.run.smithi086.stdout: "verify_ssl": { 2024-03-23T18:08:14.846 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "verify_ssl", 2024-03-23T18:08:14.846 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.846 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.846 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.847 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "true", 2024-03-23T18:08:14.847 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.847 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.847 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.847 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-23T18:08:14.847 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.847 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.847 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.847 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.848 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.848 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.848 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:14.848 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "insights", 2024-03-23T18:08:14.848 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:14.848 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:14.848 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:14.848 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:14.849 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:14.849 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.849 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.849 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.849 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.849 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.849 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.849 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.849 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.850 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.850 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.850 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.850 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.850 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.850 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.850 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.850 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.850 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.851 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.851 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.851 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:14.851 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:14.851 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.851 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.851 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.851 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.851 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.852 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.852 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.852 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.852 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.852 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.852 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.852 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.852 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:14.853 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:14.853 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.853 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.853 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.853 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:14.853 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.853 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.853 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.853 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.854 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.854 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.854 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.854 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.854 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.854 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.854 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.854 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.854 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.855 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.855 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.855 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:14.855 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:14.855 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.855 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.855 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.855 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.855 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.856 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.856 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.856 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.856 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.856 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.856 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.856 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.856 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.856 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.857 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:14.857 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "iostat", 2024-03-23T18:08:14.857 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:14.857 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:14.857 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:14.857 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:14.857 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:14.857 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.858 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.858 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.858 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.858 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.858 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.858 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.858 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.858 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.859 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.859 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.859 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.859 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.859 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.859 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.859 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.859 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.859 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.860 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.860 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:14.860 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:14.860 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.860 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.860 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.860 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.860 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.860 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.861 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.861 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.861 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.861 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.861 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.861 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.861 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:14.861 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:14.861 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.862 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.862 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.862 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:14.862 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.862 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.862 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.862 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.862 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.862 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.863 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.863 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.863 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.863 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.863 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.863 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.863 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.863 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.863 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.864 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:14.864 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:14.864 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.864 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.864 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.864 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.864 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.864 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.865 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.865 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.865 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.865 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.865 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.865 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.865 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.865 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.865 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:14.866 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "k8sevents", 2024-03-23T18:08:14.866 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:14.866 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:14.866 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:14.866 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_event_retention_days": { 2024-03-23T18:08:14.866 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ceph_event_retention_days", 2024-03-23T18:08:14.866 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:14.866 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.867 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.867 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "7", 2024-03-23T18:08:14.867 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.867 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.867 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.867 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-23T18:08:14.867 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.867 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.867 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.868 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.868 INFO:teuthology.orchestra.run.smithi086.stdout: "config_check_secs": { 2024-03-23T18:08:14.868 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "config_check_secs", 2024-03-23T18:08:14.868 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:14.868 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.868 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.868 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-03-23T18:08:14.868 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "10", 2024-03-23T18:08:14.869 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.869 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.869 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-23T18:08:14.869 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.869 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.869 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.869 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.869 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:14.869 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:14.869 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.870 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.870 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.870 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.870 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.870 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.870 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.870 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.870 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.870 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.871 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.871 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.871 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.871 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.871 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.871 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.871 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.871 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.871 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.872 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:14.872 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:14.872 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.872 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.872 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.872 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.872 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.872 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.873 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.873 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.873 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.873 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.873 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.873 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.873 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:14.873 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:14.873 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.874 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.874 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.874 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:14.874 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.874 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.874 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.874 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.874 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.874 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.875 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.875 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.875 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.875 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.875 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.875 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.875 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.875 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.875 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.876 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:14.876 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:14.876 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.876 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.876 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.876 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.876 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.876 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.876 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.877 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.877 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.877 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.877 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.877 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.877 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.877 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.877 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:14.877 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "localpool", 2024-03-23T18:08:14.878 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:14.878 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:14.878 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:14.878 INFO:teuthology.orchestra.run.smithi086.stdout: "failure_domain": { 2024-03-23T18:08:14.878 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "failure_domain", 2024-03-23T18:08:14.878 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.878 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.878 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.879 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "host", 2024-03-23T18:08:14.879 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.879 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.879 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.879 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "failure domain for any created local pool", 2024-03-23T18:08:14.879 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-23T18:08:14.879 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.879 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.879 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.880 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:14.880 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:14.880 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.880 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.880 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.880 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.880 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.880 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.880 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.881 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.881 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.881 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.881 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.881 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.881 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.881 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.881 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.882 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.882 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.882 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.882 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.882 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:14.882 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:14.882 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.882 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.882 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.883 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.883 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.883 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.883 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.883 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.883 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.883 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.883 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.883 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.884 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:14.884 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:14.884 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.884 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.884 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.884 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:14.884 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.884 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.884 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.885 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.885 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.885 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.885 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.885 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.885 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.885 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.885 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.885 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.886 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.886 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.886 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.886 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:14.886 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:14.886 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.886 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.886 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.887 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.887 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.887 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.887 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.887 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.887 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.887 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.887 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.887 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.888 INFO:teuthology.orchestra.run.smithi086.stdout: "min_size": { 2024-03-23T18:08:14.888 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "min_size", 2024-03-23T18:08:14.888 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:14.888 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.888 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.888 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.888 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.888 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.888 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.889 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "default min_size for any created local pool", 2024-03-23T18:08:14.889 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-23T18:08:14.889 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.889 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.889 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.889 INFO:teuthology.orchestra.run.smithi086.stdout: "num_rep": { 2024-03-23T18:08:14.889 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "num_rep", 2024-03-23T18:08:14.889 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:14.889 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.890 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.890 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "3", 2024-03-23T18:08:14.890 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.890 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.890 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.890 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "default replica count for any created local pool", 2024-03-23T18:08:14.890 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.890 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.891 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.891 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.891 INFO:teuthology.orchestra.run.smithi086.stdout: "pg_num": { 2024-03-23T18:08:14.891 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pg_num", 2024-03-23T18:08:14.891 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:14.891 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.891 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.891 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "128", 2024-03-23T18:08:14.891 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.892 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.892 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.892 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "default pg_num for any created local pool", 2024-03-23T18:08:14.892 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.892 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.892 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.892 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.892 INFO:teuthology.orchestra.run.smithi086.stdout: "prefix": { 2024-03-23T18:08:14.893 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "prefix", 2024-03-23T18:08:14.893 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.893 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.893 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.893 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.893 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.893 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.893 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.893 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "name prefix for any created local pool", 2024-03-23T18:08:14.894 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.894 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.894 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.894 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.894 INFO:teuthology.orchestra.run.smithi086.stdout: "subtree": { 2024-03-23T18:08:14.894 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "subtree", 2024-03-23T18:08:14.894 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.894 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.894 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.895 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "rack", 2024-03-23T18:08:14.895 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.895 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.895 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.895 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-23T18:08:14.895 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-23T18:08:14.895 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.895 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.896 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.896 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.896 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.896 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:14.896 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mds_autoscaler", 2024-03-23T18:08:14.896 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:14.896 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:14.896 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:14.896 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:14.897 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:14.897 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.897 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.897 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.897 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.897 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.897 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.897 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.897 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.898 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.898 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.898 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.898 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.898 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.898 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.898 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.898 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.899 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.899 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.899 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.899 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:14.899 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:14.899 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.899 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.899 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.899 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.900 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.900 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.900 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.900 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.900 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.900 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.900 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.900 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.900 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:14.901 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:14.901 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.901 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.901 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.901 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:14.901 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.901 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.901 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.901 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.902 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.902 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.902 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.902 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.902 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.902 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.902 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.902 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.903 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.903 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.903 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.903 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:14.903 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:14.903 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.903 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.903 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.903 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.904 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.904 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.904 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.904 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.904 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.904 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.904 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.904 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.904 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.905 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.905 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:14.905 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mirroring", 2024-03-23T18:08:14.905 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:14.905 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:14.905 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:14.905 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:14.905 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:14.905 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.906 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.906 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.906 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.906 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.906 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.906 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.906 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.906 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.907 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.907 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.907 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.907 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.907 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.907 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.907 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.907 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.907 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.908 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.908 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:14.908 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:14.908 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.908 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.908 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.908 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.908 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.909 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.909 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.909 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.909 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.909 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.909 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.909 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.909 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:14.909 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:14.910 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.910 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.910 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.910 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:14.910 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.910 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.910 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.910 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.911 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.911 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.911 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.911 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.911 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.911 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.911 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.911 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.911 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.912 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.912 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.912 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:14.912 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:14.912 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.912 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.912 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.912 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.912 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.913 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.913 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.913 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.913 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.913 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.913 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.913 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.913 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.913 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.914 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:14.914 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "nfs", 2024-03-23T18:08:14.914 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:14.914 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:14.914 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:14.914 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:14.914 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:14.914 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.914 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.915 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.915 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.915 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.915 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.915 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.915 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.915 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.915 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.915 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.916 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.916 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.916 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.916 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.916 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.916 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.916 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.916 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.916 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:14.917 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:14.917 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.917 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.917 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.917 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.917 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.917 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.917 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.917 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.918 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.918 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.918 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.918 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.918 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:14.918 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:14.918 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.918 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.918 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.919 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:14.919 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.919 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.919 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.919 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.919 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.919 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.919 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.920 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.920 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.920 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.920 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.920 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.920 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.920 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.920 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.920 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:14.921 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:14.921 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.921 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.921 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.921 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.921 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.921 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.921 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.922 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.922 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.922 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.922 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.922 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.922 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.922 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.922 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:14.922 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "orchestrator", 2024-03-23T18:08:14.923 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:14.923 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:14.923 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:14.923 INFO:teuthology.orchestra.run.smithi086.stdout: "fail_fs": { 2024-03-23T18:08:14.923 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "fail_fs", 2024-03-23T18:08:14.923 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.923 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.923 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.923 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.924 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.924 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.924 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.924 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-23T18:08:14.924 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.924 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.924 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.924 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.925 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:14.925 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:14.925 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.925 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.925 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.925 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.925 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.925 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.925 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.926 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.926 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.926 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.926 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.926 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.926 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.926 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.926 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.927 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.927 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.927 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.927 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.927 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:14.927 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:14.927 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.927 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.928 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.928 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.928 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.928 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.928 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.928 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.928 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.928 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.928 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.929 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.929 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:14.929 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:14.929 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.929 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.929 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.929 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:14.929 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.929 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.930 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.930 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.930 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.930 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.930 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.930 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.930 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.930 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.931 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.931 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.931 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.931 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.931 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.931 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:14.931 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:14.931 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.931 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.932 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.932 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.932 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.932 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.932 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.932 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.932 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.932 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.932 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.933 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.933 INFO:teuthology.orchestra.run.smithi086.stdout: "orchestrator": { 2024-03-23T18:08:14.933 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "orchestrator", 2024-03-23T18:08:14.933 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.933 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.933 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.933 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.933 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.933 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.934 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.934 INFO:teuthology.orchestra.run.smithi086.stdout: "cephadm", 2024-03-23T18:08:14.934 INFO:teuthology.orchestra.run.smithi086.stdout: "rook", 2024-03-23T18:08:14.934 INFO:teuthology.orchestra.run.smithi086.stdout: "test_orchestrator" 2024-03-23T18:08:14.934 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.934 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Orchestrator backend", 2024-03-23T18:08:14.934 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.934 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.935 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.935 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.935 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.935 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.935 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:14.935 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "osd_perf_query", 2024-03-23T18:08:14.935 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:14.935 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:14.935 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:14.936 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:14.936 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:14.936 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.936 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.936 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.936 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.936 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.936 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.936 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.937 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.937 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.937 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.937 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.937 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.937 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.937 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.937 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.937 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.938 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.938 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.938 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.938 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:14.938 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:14.938 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.938 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.938 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.939 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.939 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.939 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.939 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.939 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.939 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.939 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.939 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.939 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.939 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:14.940 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:14.940 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.940 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.940 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.940 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:14.940 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.940 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.940 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.941 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.941 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.941 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.941 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.941 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.941 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.941 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.941 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.941 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.942 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.942 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.942 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.942 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:14.942 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:14.942 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.942 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.942 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.942 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.943 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.943 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.943 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.943 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.943 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.943 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.943 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.943 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.943 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.944 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.944 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:14.944 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "osd_support", 2024-03-23T18:08:14.944 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:14.944 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:14.944 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:14.944 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:14.944 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:14.944 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.945 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.945 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.945 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.945 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.945 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.945 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.945 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.945 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.945 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.946 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.946 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.946 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.946 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.946 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.946 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.946 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.946 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.947 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.947 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:14.947 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:14.947 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.947 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.947 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.947 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.947 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.947 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.948 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.948 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.948 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.948 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.948 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.948 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.948 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:14.948 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:14.948 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.949 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.949 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.949 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:14.949 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.949 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.949 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.949 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.949 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.950 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.950 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.950 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.950 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.950 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.950 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.950 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.950 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.950 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.951 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.951 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:14.951 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:14.951 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.951 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.951 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.951 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.951 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.952 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.952 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.952 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.952 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.952 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.952 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.952 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.952 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.952 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.953 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:14.953 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pg_autoscaler", 2024-03-23T18:08:14.953 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:14.953 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:14.953 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:14.953 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:14.953 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:14.953 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.953 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.954 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.954 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.954 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.954 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.954 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.954 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.954 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.954 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.954 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.955 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.955 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.955 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.955 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.955 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.955 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.955 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.955 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.956 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:14.956 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:14.956 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.956 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.956 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.956 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.956 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.956 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.956 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.957 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.957 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.957 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.957 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.957 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.957 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:14.957 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:14.957 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.957 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.958 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.958 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:14.958 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.958 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.958 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.958 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.958 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.958 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.958 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.959 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.959 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.959 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.959 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.959 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.959 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.959 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.959 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.960 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:14.960 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:14.960 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.960 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.960 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.960 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.960 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.960 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.960 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.961 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.961 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.961 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.961 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.961 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.961 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-03-23T18:08:14.961 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-03-23T18:08:14.961 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-23T18:08:14.961 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.962 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.962 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-03-23T18:08:14.962 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.962 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.962 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.962 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.962 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.962 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.962 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.963 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.963 INFO:teuthology.orchestra.run.smithi086.stdout: "threshold": { 2024-03-23T18:08:14.963 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "threshold", 2024-03-23T18:08:14.963 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-03-23T18:08:14.963 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.963 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.963 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "3.0", 2024-03-23T18:08:14.963 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1.0", 2024-03-23T18:08:14.964 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.964 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.964 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "scaling threshold", 2024-03-23T18:08:14.964 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-23T18:08:14.964 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.964 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.964 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.964 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.964 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.965 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:14.965 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "progress", 2024-03-23T18:08:14.965 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:14.965 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:14.965 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:14.965 INFO:teuthology.orchestra.run.smithi086.stdout: "allow_pg_recovery_event": { 2024-03-23T18:08:14.965 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "allow_pg_recovery_event", 2024-03-23T18:08:14.965 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.965 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.966 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.966 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.966 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.966 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.966 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.966 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-23T18:08:14.966 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.966 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.966 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.967 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.967 INFO:teuthology.orchestra.run.smithi086.stdout: "enabled": { 2024-03-23T18:08:14.967 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enabled", 2024-03-23T18:08:14.967 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.967 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.967 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.967 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:14.967 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.968 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.968 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.968 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.968 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.968 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.968 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.968 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.968 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:14.968 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:14.969 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.969 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.969 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.969 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.969 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.969 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.969 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.969 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.970 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.970 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.970 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.970 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.970 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.970 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.970 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.970 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.970 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.971 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.971 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.971 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:14.971 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:14.971 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.971 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.971 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.971 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.971 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.972 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.972 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.972 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.972 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.972 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.972 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.972 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.972 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:14.972 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:14.973 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.973 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.973 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.973 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:14.973 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.973 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.973 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.973 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.973 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.974 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.974 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.974 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.974 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.974 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.974 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.974 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.974 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.975 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.975 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.975 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:14.975 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:14.975 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.975 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.975 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.975 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.975 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.976 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.976 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.976 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.976 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.976 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.976 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.976 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.976 INFO:teuthology.orchestra.run.smithi086.stdout: "max_completed_events": { 2024-03-23T18:08:14.976 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_completed_events", 2024-03-23T18:08:14.977 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:14.977 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.977 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.977 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "50", 2024-03-23T18:08:14.977 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.977 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.977 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.977 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "number of past completed events to remember", 2024-03-23T18:08:14.977 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.978 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.978 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.978 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.978 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-03-23T18:08:14.978 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-03-23T18:08:14.978 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-23T18:08:14.978 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.978 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.979 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-03-23T18:08:14.979 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.979 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.979 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.979 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how long the module is going to sleep", 2024-03-23T18:08:14.979 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.979 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.979 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.979 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.980 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:14.980 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.980 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:14.980 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "prometheus", 2024-03-23T18:08:14.980 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:14.980 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:14.980 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:14.980 INFO:teuthology.orchestra.run.smithi086.stdout: "cache": { 2024-03-23T18:08:14.980 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cache", 2024-03-23T18:08:14.981 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.981 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.981 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.981 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:14.981 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.981 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.981 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.981 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.981 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.982 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.982 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.982 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.982 INFO:teuthology.orchestra.run.smithi086.stdout: "exclude_perf_counters": { 2024-03-23T18:08:14.982 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "exclude_perf_counters", 2024-03-23T18:08:14.982 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.982 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.982 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.982 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:14.983 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.983 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.983 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.983 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-23T18:08:14.983 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-23T18:08:14.983 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.983 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.983 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.983 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:14.984 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:14.984 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.984 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.984 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.984 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.984 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.984 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.984 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.984 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.985 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.985 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.985 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.985 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.985 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.985 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.985 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.985 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.986 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.986 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.986 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.986 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:14.986 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:14.986 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.986 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.986 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.986 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.987 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.987 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.987 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.987 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.987 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.987 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.987 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.987 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.987 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:14.988 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:14.988 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.988 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.988 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.988 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:14.988 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.988 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.988 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:14.988 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:14.989 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:14.989 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:14.989 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:14.989 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:14.989 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:14.989 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:14.989 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.989 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.989 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.990 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.990 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.990 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:14.990 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:14.990 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:14.990 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.990 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.990 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:14.990 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.991 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.991 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.991 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.991 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.991 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.991 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.991 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.991 INFO:teuthology.orchestra.run.smithi086.stdout: "rbd_stats_pools": { 2024-03-23T18:08:14.991 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rbd_stats_pools", 2024-03-23T18:08:14.992 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.992 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.992 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.992 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:14.992 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.992 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.992 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.992 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.992 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.993 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.993 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.993 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.993 INFO:teuthology.orchestra.run.smithi086.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-23T18:08:14.993 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-23T18:08:14.993 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:14.993 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.993 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.993 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "300", 2024-03-23T18:08:14.994 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.994 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.994 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.994 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.994 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.994 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.994 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.994 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.994 INFO:teuthology.orchestra.run.smithi086.stdout: "scrape_interval": { 2024-03-23T18:08:14.995 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "scrape_interval", 2024-03-23T18:08:14.995 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-03-23T18:08:14.995 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.995 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.995 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "15.0", 2024-03-23T18:08:14.995 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.995 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.995 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.996 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:14.996 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.996 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.996 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.996 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.996 INFO:teuthology.orchestra.run.smithi086.stdout: "server_addr": { 2024-03-23T18:08:14.996 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_addr", 2024-03-23T18:08:14.996 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:14.996 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.997 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:14.997 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "::", 2024-03-23T18:08:14.997 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.997 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.997 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.997 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-23T18:08:14.997 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.997 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.997 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.998 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.998 INFO:teuthology.orchestra.run.smithi086.stdout: "server_port": { 2024-03-23T18:08:14.998 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_port", 2024-03-23T18:08:14.998 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:14.998 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:14.998 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:14.998 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "9283", 2024-03-23T18:08:14.998 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:14.999 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:14.999 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:14.999 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-23T18:08:14.999 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:14.999 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:14.999 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:14.999 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:14.999 INFO:teuthology.orchestra.run.smithi086.stdout: "stale_cache_strategy": { 2024-03-23T18:08:14.999 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "stale_cache_strategy", 2024-03-23T18:08:15.000 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:15.000 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.000 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:15.000 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "log", 2024-03-23T18:08:15.000 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.000 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.000 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:15.000 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.000 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.001 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.001 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.001 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.001 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_behaviour": { 2024-03-23T18:08:15.001 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_behaviour", 2024-03-23T18:08:15.001 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:15.001 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.001 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:15.002 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "default", 2024-03-23T18:08:15.002 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.002 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.002 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:15.002 INFO:teuthology.orchestra.run.smithi086.stdout: "default", 2024-03-23T18:08:15.002 INFO:teuthology.orchestra.run.smithi086.stdout: "error" 2024-03-23T18:08:15.002 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:15.002 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.002 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.003 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.003 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.003 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.003 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_error_status_code": { 2024-03-23T18:08:15.003 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_error_status_code", 2024-03-23T18:08:15.003 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:15.003 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.003 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:15.003 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "500", 2024-03-23T18:08:15.004 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "400", 2024-03-23T18:08:15.004 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "599", 2024-03-23T18:08:15.004 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:15.004 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.004 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.004 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.004 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.004 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:15.005 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:15.005 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.005 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:15.005 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rbd_support", 2024-03-23T18:08:15.005 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:15.005 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:15.005 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:15.005 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:15.005 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:15.006 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:15.006 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.006 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:15.006 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:15.006 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.006 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.006 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:15.006 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:15.006 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:15.007 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:15.007 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:15.007 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:15.007 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:15.007 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:15.007 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.007 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.007 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.008 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.008 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.008 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:15.008 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:15.008 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:15.008 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.008 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:15.008 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:15.009 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.009 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.009 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:15.009 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.009 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.009 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.009 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.009 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.009 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:15.010 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:15.010 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:15.010 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.010 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:15.010 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:15.010 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.010 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.010 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:15.011 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:15.011 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:15.011 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:15.011 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:15.011 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:15.011 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:15.011 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:15.011 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.011 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.011 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.012 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.012 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.012 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:15.012 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:15.012 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:15.012 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.012 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:15.012 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:15.013 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.013 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.013 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:15.013 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.013 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.013 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.013 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.013 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.013 INFO:teuthology.orchestra.run.smithi086.stdout: "max_concurrent_snap_create": { 2024-03-23T18:08:15.014 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_concurrent_snap_create", 2024-03-23T18:08:15.014 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:15.014 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.014 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:15.014 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-03-23T18:08:15.014 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.014 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.014 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:15.015 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.015 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.015 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.015 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.015 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.015 INFO:teuthology.orchestra.run.smithi086.stdout: "mirror_snapshot_schedule": { 2024-03-23T18:08:15.015 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mirror_snapshot_schedule", 2024-03-23T18:08:15.015 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:15.015 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.016 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:15.016 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:15.016 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.016 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.016 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:15.016 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.016 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.016 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.016 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.017 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.017 INFO:teuthology.orchestra.run.smithi086.stdout: "trash_purge_schedule": { 2024-03-23T18:08:15.017 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "trash_purge_schedule", 2024-03-23T18:08:15.017 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:15.017 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.017 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:15.017 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:15.017 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.018 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.018 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:15.018 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.018 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.018 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.018 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.018 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:15.018 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:15.018 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.019 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:15.019 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "restful", 2024-03-23T18:08:15.019 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:15.019 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:15.019 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:15.019 INFO:teuthology.orchestra.run.smithi086.stdout: "enable_auth": { 2024-03-23T18:08:15.019 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enable_auth", 2024-03-23T18:08:15.019 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:15.019 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.020 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:15.020 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:15.020 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.020 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.020 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:15.020 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.020 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.020 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.020 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.021 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.021 INFO:teuthology.orchestra.run.smithi086.stdout: "key_file": { 2024-03-23T18:08:15.021 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "key_file", 2024-03-23T18:08:15.021 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:15.021 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.021 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:15.021 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:15.021 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.021 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.022 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:15.022 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.022 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.022 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.022 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.022 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.022 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:15.022 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:15.023 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:15.023 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.023 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:15.023 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:15.023 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.023 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.023 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:15.023 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:15.023 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:15.024 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:15.024 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:15.024 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:15.024 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:15.024 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:15.024 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.024 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.024 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.024 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.025 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.025 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:15.025 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:15.025 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:15.025 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.025 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:15.025 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:15.025 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.025 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.026 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:15.026 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.026 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.026 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.026 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.026 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.026 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:15.026 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:15.026 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:15.027 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.027 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:15.027 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:15.027 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.027 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.027 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:15.027 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:15.027 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:15.028 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:15.028 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:15.028 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:15.028 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:15.028 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:15.028 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.028 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.028 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.028 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.029 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.029 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:15.029 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:15.029 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:15.029 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.029 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:15.029 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:15.029 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.029 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.030 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:15.030 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.030 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.030 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.030 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.030 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.030 INFO:teuthology.orchestra.run.smithi086.stdout: "server_addr": { 2024-03-23T18:08:15.030 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_addr", 2024-03-23T18:08:15.031 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:15.031 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.031 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:15.031 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:15.031 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.031 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.031 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:15.031 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.031 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.032 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.032 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.032 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.032 INFO:teuthology.orchestra.run.smithi086.stdout: "server_port": { 2024-03-23T18:08:15.032 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_port", 2024-03-23T18:08:15.032 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:15.032 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.032 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:15.032 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:15.033 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.033 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.033 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:15.033 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.033 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.033 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.033 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.033 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:15.034 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:15.034 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.034 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:15.034 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rgw", 2024-03-23T18:08:15.034 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:15.034 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:15.034 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:15.034 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:15.034 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:15.035 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:15.035 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.035 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:15.035 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:15.035 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.035 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.035 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:15.035 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:15.036 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:15.036 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:15.036 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:15.036 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:15.036 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:15.036 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:15.036 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.036 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.036 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.037 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.037 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.037 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:15.037 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:15.037 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:15.037 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.037 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:15.037 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:15.037 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.038 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.038 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:15.038 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.038 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.038 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.038 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.038 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.038 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:15.039 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:15.039 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:15.039 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.039 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:15.039 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:15.039 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.039 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.039 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:15.039 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:15.040 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:15.040 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:15.040 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:15.040 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:15.040 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:15.040 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:15.040 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.040 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.040 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.041 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.041 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.041 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:15.041 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:15.041 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:15.041 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.041 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:15.041 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:15.041 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.042 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.042 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:15.042 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.042 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.042 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.042 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.042 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:15.042 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:15.042 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.043 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:15.043 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rook", 2024-03-23T18:08:15.043 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:15.043 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:15.043 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:15.043 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:15.043 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:15.043 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:15.044 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.044 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:15.044 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:15.044 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.044 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.044 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:15.044 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:15.044 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:15.044 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:15.045 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:15.045 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:15.045 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:15.045 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:15.045 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.045 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.045 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.045 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.045 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.046 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:15.046 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:15.046 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:15.046 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.046 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:15.046 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:15.046 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.046 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.046 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:15.047 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.047 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.047 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.047 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.047 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.047 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:15.047 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:15.047 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:15.047 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.048 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:15.048 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:15.048 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.048 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.048 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:15.048 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:15.048 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:15.048 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:15.048 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:15.049 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:15.049 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:15.049 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:15.049 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.049 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.049 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.049 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.049 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.049 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:15.050 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:15.050 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:15.050 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.050 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:15.050 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:15.050 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.050 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.050 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:15.051 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.051 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.051 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.051 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.051 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.051 INFO:teuthology.orchestra.run.smithi086.stdout: "storage_class": { 2024-03-23T18:08:15.051 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "storage_class", 2024-03-23T18:08:15.051 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:15.051 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.052 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:15.052 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "local", 2024-03-23T18:08:15.052 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.052 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.052 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:15.052 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-23T18:08:15.052 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.052 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.052 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.053 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:15.053 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:15.053 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.053 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:15.053 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "selftest", 2024-03-23T18:08:15.053 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:15.053 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:15.053 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:15.054 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:15.054 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:15.054 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:15.054 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.054 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:15.054 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:15.054 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.054 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.054 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:15.055 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:15.055 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:15.055 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:15.055 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:15.055 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:15.055 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:15.055 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:15.055 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.056 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.056 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.056 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.056 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.056 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:15.056 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:15.056 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:15.056 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.057 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:15.057 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:15.057 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.057 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.057 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:15.057 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.057 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.057 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.057 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.058 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.058 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:15.058 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:15.058 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:15.058 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.058 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:15.058 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:15.058 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.059 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.059 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:15.059 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:15.059 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:15.059 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:15.059 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:15.059 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:15.059 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:15.059 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:15.060 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.060 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.060 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.060 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.060 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.060 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:15.060 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:15.060 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:15.061 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.061 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:15.061 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:15.061 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.061 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.061 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:15.061 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.061 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.061 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.062 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.062 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.062 INFO:teuthology.orchestra.run.smithi086.stdout: "roption1": { 2024-03-23T18:08:15.062 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "roption1", 2024-03-23T18:08:15.062 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:15.062 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.062 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:15.062 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:15.063 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.063 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.063 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:15.063 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.063 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.063 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.063 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.063 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.063 INFO:teuthology.orchestra.run.smithi086.stdout: "roption2": { 2024-03-23T18:08:15.064 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "roption2", 2024-03-23T18:08:15.064 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:15.064 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.064 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:15.064 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "xyz", 2024-03-23T18:08:15.064 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.064 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.064 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:15.064 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.065 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.065 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.065 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.065 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.065 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption1": { 2024-03-23T18:08:15.065 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption1", 2024-03-23T18:08:15.065 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:15.065 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.066 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:15.066 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:15.066 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.066 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.066 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:15.066 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.066 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.066 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.066 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.067 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.067 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption2": { 2024-03-23T18:08:15.067 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption2", 2024-03-23T18:08:15.067 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:15.067 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.067 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:15.067 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:15.067 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.067 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.068 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:15.068 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.068 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.068 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.068 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.068 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.068 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption3": { 2024-03-23T18:08:15.068 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption3", 2024-03-23T18:08:15.069 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-03-23T18:08:15.069 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.069 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:15.069 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:15.069 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.069 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.069 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:15.069 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.069 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.070 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.070 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.070 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.070 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption4": { 2024-03-23T18:08:15.070 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption4", 2024-03-23T18:08:15.070 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:15.070 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.070 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:15.070 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:15.071 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.071 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.071 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:15.071 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.071 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.071 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.071 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.071 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.071 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption5": { 2024-03-23T18:08:15.072 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption5", 2024-03-23T18:08:15.072 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:15.072 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.072 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:15.072 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:15.072 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.072 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.072 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:15.072 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.073 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.073 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.073 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.073 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.073 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption6": { 2024-03-23T18:08:15.073 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption6", 2024-03-23T18:08:15.073 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:15.073 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.073 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:15.074 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:15.074 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.074 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.074 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:15.074 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.074 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.075 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.075 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.075 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.075 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption7": { 2024-03-23T18:08:15.075 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption7", 2024-03-23T18:08:15.075 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:15.075 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.075 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:15.076 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:15.076 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-03-23T18:08:15.076 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "42", 2024-03-23T18:08:15.076 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:15.076 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.076 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.076 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.076 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.076 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.077 INFO:teuthology.orchestra.run.smithi086.stdout: "testkey": { 2024-03-23T18:08:15.077 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "testkey", 2024-03-23T18:08:15.077 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:15.077 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.077 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:15.077 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:15.077 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.077 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.078 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:15.078 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.078 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.078 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.078 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.078 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.078 INFO:teuthology.orchestra.run.smithi086.stdout: "testlkey": { 2024-03-23T18:08:15.078 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "testlkey", 2024-03-23T18:08:15.078 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:15.079 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.079 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:15.079 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:15.079 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.079 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.079 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:15.079 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.079 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.080 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.080 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.080 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.080 INFO:teuthology.orchestra.run.smithi086.stdout: "testnewline": { 2024-03-23T18:08:15.080 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "testnewline", 2024-03-23T18:08:15.080 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:15.080 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.080 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:15.080 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:15.081 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.081 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.081 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:15.081 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.081 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.081 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.081 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.081 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:15.082 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:15.082 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.082 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:15.082 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "snap_schedule", 2024-03-23T18:08:15.082 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:15.082 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:15.082 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:15.082 INFO:teuthology.orchestra.run.smithi086.stdout: "allow_m_granularity": { 2024-03-23T18:08:15.082 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "allow_m_granularity", 2024-03-23T18:08:15.083 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:15.083 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.083 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:15.083 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:15.083 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.083 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.083 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:15.083 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "allow minute scheduled snapshots", 2024-03-23T18:08:15.084 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.084 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.084 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.084 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.084 INFO:teuthology.orchestra.run.smithi086.stdout: "dump_on_update": { 2024-03-23T18:08:15.084 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "dump_on_update", 2024-03-23T18:08:15.084 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:15.084 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.085 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:15.085 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:15.085 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.085 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.085 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:15.085 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "dump database to debug log on update", 2024-03-23T18:08:15.085 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.085 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.085 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.086 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.086 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:15.086 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:15.086 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:15.086 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.086 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:15.086 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:15.086 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.087 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.087 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:15.087 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:15.087 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:15.087 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:15.087 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:15.087 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:15.087 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:15.087 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:15.088 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.088 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.088 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.088 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.088 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.088 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:15.088 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:15.088 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:15.088 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.089 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:15.089 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:15.089 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.089 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.089 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:15.089 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.089 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.089 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.090 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.090 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.090 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:15.090 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:15.090 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:15.090 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.090 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:15.090 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:15.090 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.091 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.091 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:15.091 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:15.091 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:15.091 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:15.091 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:15.091 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:15.091 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:15.091 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:15.091 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.092 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.092 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.092 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.092 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.092 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:15.092 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:15.092 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:15.092 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.092 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:15.093 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:15.093 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.093 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.093 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:15.093 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.093 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.093 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.093 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.093 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:15.094 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:15.094 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.094 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:15.094 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "stats", 2024-03-23T18:08:15.094 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:15.094 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:15.094 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:15.094 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:15.094 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:15.095 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:15.095 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.095 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:15.095 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:15.095 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.095 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.095 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:15.095 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:15.095 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:15.096 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:15.096 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:15.096 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:15.096 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:15.096 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:15.096 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.096 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.096 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.096 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.097 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.097 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:15.097 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:15.097 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:15.097 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.097 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:15.097 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:15.097 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.097 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.098 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:15.098 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.098 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.098 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.098 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.098 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.098 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:15.098 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:15.099 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:15.099 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.099 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:15.099 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:15.099 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.099 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.099 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:15.099 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:15.099 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:15.100 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:15.100 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:15.100 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:15.100 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:15.100 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:15.100 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.100 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.100 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.100 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.101 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.101 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:15.101 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:15.101 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:15.101 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.101 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:15.101 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:15.101 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.101 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.102 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:15.102 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.102 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.102 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.102 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.102 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:15.102 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:15.102 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.102 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:15.103 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "status", 2024-03-23T18:08:15.103 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:15.103 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:15.103 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:15.103 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:15.103 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:15.103 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:15.103 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.104 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:15.104 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:15.104 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.104 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.104 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:15.104 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:15.104 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:15.104 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:15.104 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:15.105 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:15.105 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:15.105 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:15.105 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.105 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.105 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.105 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.105 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.105 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:15.106 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:15.106 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:15.106 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.106 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:15.106 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:15.106 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.106 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.106 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:15.107 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.107 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.107 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.107 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.107 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.107 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:15.107 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:15.107 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:15.107 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.108 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:15.108 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:15.108 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.108 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.108 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:15.108 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:15.108 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:15.108 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:15.108 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:15.109 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:15.109 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:15.109 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:15.109 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.109 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.109 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.109 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.109 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.109 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:15.110 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:15.110 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:15.110 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.110 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:15.110 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:15.110 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.110 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.110 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:15.111 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.111 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.111 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.111 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.111 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:15.111 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:15.111 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.111 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:15.111 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "telegraf", 2024-03-23T18:08:15.112 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:15.112 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:15.112 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:15.112 INFO:teuthology.orchestra.run.smithi086.stdout: "address": { 2024-03-23T18:08:15.112 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "address", 2024-03-23T18:08:15.112 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:15.112 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.112 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:15.113 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-23T18:08:15.113 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.113 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.113 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:15.113 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.113 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.113 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.113 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.113 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.114 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-03-23T18:08:15.114 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-03-23T18:08:15.114 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-23T18:08:15.114 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.114 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:15.114 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "15", 2024-03-23T18:08:15.114 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.114 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.115 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:15.115 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.115 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.115 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.115 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.115 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.115 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:15.115 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:15.115 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:15.116 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.116 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:15.116 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:15.116 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.116 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.116 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:15.116 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:15.116 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:15.117 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:15.117 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:15.117 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:15.117 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:15.117 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:15.117 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.117 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.117 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.117 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.118 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.118 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:15.118 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:15.118 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:15.118 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.118 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:15.118 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:15.118 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.118 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.119 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:15.119 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.119 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.119 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.119 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.119 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.119 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:15.119 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:15.120 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:15.120 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.120 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:15.120 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:15.120 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.120 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.120 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:15.120 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:15.120 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:15.121 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:15.121 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:15.121 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:15.121 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:15.121 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:15.121 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.121 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.121 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.122 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.122 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.122 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:15.122 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:15.122 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:15.122 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.122 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:15.122 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:15.122 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.123 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.123 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:15.123 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.123 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.123 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.123 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.123 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:15.123 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:15.123 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.124 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:15.124 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "telemetry", 2024-03-23T18:08:15.124 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:15.124 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:15.124 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:15.124 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_basic": { 2024-03-23T18:08:15.124 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_basic", 2024-03-23T18:08:15.124 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:15.124 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.125 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:15.125 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:15.125 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.125 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.125 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:15.125 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-23T18:08:15.125 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.125 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.125 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.126 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.126 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_crash": { 2024-03-23T18:08:15.126 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_crash", 2024-03-23T18:08:15.126 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:15.126 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.126 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:15.126 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:15.126 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.127 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.127 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:15.127 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-23T18:08:15.127 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.127 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.127 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.127 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.127 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_device": { 2024-03-23T18:08:15.127 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_device", 2024-03-23T18:08:15.128 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:15.128 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.128 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:15.128 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:15.128 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.128 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.128 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:15.128 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-23T18:08:15.128 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.129 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.129 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.129 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.129 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_ident": { 2024-03-23T18:08:15.129 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_ident", 2024-03-23T18:08:15.129 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:15.129 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.129 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:15.129 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:15.130 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.130 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.130 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:15.130 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-23T18:08:15.130 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.130 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.130 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.130 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.131 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_perf": { 2024-03-23T18:08:15.131 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_perf", 2024-03-23T18:08:15.131 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:15.131 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.131 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:15.131 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:15.131 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.131 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.131 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:15.132 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-23T18:08:15.132 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.132 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.132 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.132 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.132 INFO:teuthology.orchestra.run.smithi086.stdout: "contact": { 2024-03-23T18:08:15.132 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "contact", 2024-03-23T18:08:15.132 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:15.133 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.133 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:15.133 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:15.133 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.133 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.133 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:15.133 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.133 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.133 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.134 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.134 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.134 INFO:teuthology.orchestra.run.smithi086.stdout: "description": { 2024-03-23T18:08:15.134 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "description", 2024-03-23T18:08:15.134 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:15.134 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.134 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:15.134 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:15.134 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.135 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.135 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:15.135 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.135 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.135 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.135 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.135 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.135 INFO:teuthology.orchestra.run.smithi086.stdout: "device_url": { 2024-03-23T18:08:15.136 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "device_url", 2024-03-23T18:08:15.136 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:15.136 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.136 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:15.136 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-23T18:08:15.136 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.136 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.136 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:15.137 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.137 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.137 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.137 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.137 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.137 INFO:teuthology.orchestra.run.smithi086.stdout: "enabled": { 2024-03-23T18:08:15.137 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enabled", 2024-03-23T18:08:15.137 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:15.137 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.138 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:15.138 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:15.138 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.138 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.138 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:15.138 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.138 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.139 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.139 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.139 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.139 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-03-23T18:08:15.139 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-03-23T18:08:15.139 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:15.139 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.139 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:15.139 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "24", 2024-03-23T18:08:15.140 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "8", 2024-03-23T18:08:15.140 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.140 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:15.140 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.140 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.140 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.140 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.140 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.140 INFO:teuthology.orchestra.run.smithi086.stdout: "last_opt_revision": { 2024-03-23T18:08:15.141 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "last_opt_revision", 2024-03-23T18:08:15.141 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:15.141 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.141 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:15.141 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "1", 2024-03-23T18:08:15.141 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.141 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.141 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:15.141 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.142 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.142 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.142 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.142 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.142 INFO:teuthology.orchestra.run.smithi086.stdout: "leaderboard": { 2024-03-23T18:08:15.142 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "leaderboard", 2024-03-23T18:08:15.142 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:15.142 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.143 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:15.143 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:15.143 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.143 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.143 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:15.143 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.143 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.143 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.143 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.144 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.144 INFO:teuthology.orchestra.run.smithi086.stdout: "leaderboard_description": { 2024-03-23T18:08:15.144 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "leaderboard_description", 2024-03-23T18:08:15.144 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:15.144 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.144 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:15.144 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:15.144 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.144 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.145 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:15.145 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.145 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.145 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.145 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.145 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.145 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:15.145 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:15.145 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:15.146 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.146 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:15.146 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:15.146 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.146 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.146 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:15.146 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:15.146 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:15.146 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:15.147 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:15.147 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:15.147 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:15.147 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:15.147 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.147 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.147 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.147 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.147 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.148 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:15.148 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:15.148 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:15.148 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.148 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:15.148 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:15.148 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.148 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.148 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:15.149 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.149 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.149 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.149 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.149 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.149 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:15.149 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:15.149 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:15.150 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.150 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:15.150 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:15.150 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.150 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.150 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:15.150 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:15.150 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:15.150 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:15.151 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:15.151 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:15.151 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:15.151 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:15.151 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.151 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.151 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.151 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.152 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.152 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:15.152 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:15.152 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:15.152 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.152 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:15.152 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:15.152 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.152 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.153 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:15.153 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.153 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.153 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.153 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.153 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.153 INFO:teuthology.orchestra.run.smithi086.stdout: "organization": { 2024-03-23T18:08:15.153 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "organization", 2024-03-23T18:08:15.153 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:15.154 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.154 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:15.154 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:15.154 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.154 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.154 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:15.154 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.154 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.154 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.155 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.155 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.155 INFO:teuthology.orchestra.run.smithi086.stdout: "proxy": { 2024-03-23T18:08:15.155 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "proxy", 2024-03-23T18:08:15.155 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:15.155 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.155 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:15.155 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:15.155 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.156 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.156 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:15.156 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.156 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.156 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.156 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.156 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.156 INFO:teuthology.orchestra.run.smithi086.stdout: "url": { 2024-03-23T18:08:15.157 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "url", 2024-03-23T18:08:15.157 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:15.157 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.157 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:15.157 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-23T18:08:15.157 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.157 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.157 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:15.157 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.158 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.158 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.158 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.158 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:15.158 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:15.158 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.158 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:15.158 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "test_orchestrator", 2024-03-23T18:08:15.158 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:15.159 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:15.159 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:15.159 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:15.159 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:15.159 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:15.159 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.159 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:15.159 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:15.160 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.160 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.160 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:15.160 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:15.160 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:15.160 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:15.160 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:15.160 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:15.160 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:15.161 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:15.161 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.161 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.161 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.161 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.161 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.161 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:15.161 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:15.161 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:15.162 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.162 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:15.162 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:15.162 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.162 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.162 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:15.162 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.162 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.162 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.163 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.163 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.163 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:15.163 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:15.163 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:15.163 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.163 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:15.163 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:15.163 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.164 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.164 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:15.164 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:15.164 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:15.164 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:15.164 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:15.164 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:15.164 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:15.164 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:15.165 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.165 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.165 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.165 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.165 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.165 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:15.165 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:15.165 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:15.166 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.166 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:15.166 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:15.166 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.166 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.166 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:15.166 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.166 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.166 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.167 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.167 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:15.167 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:15.167 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.167 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:15.167 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "volumes", 2024-03-23T18:08:15.167 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:15.167 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:15.167 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:15.168 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:15.168 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:15.168 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:15.168 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.168 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:15.168 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:15.168 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.168 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.168 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:15.169 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:15.169 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:15.169 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:15.169 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:15.169 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:15.169 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:15.169 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:15.169 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.169 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.170 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.170 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.170 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.170 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:15.170 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:15.170 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:15.170 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.170 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:15.171 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:15.171 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.171 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.171 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:15.171 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.171 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.171 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.171 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.171 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.172 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:15.172 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:15.172 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:15.172 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.172 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:15.172 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:15.172 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.172 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.172 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:15.173 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:15.173 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:15.173 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:15.173 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:15.173 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:15.173 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:15.173 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:15.173 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.173 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.174 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.174 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.174 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.174 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:15.174 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:15.174 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:15.174 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.174 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:15.175 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:15.175 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.175 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.175 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:15.175 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.175 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.175 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.175 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.175 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.176 INFO:teuthology.orchestra.run.smithi086.stdout: "max_concurrent_clones": { 2024-03-23T18:08:15.176 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_concurrent_clones", 2024-03-23T18:08:15.176 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:15.176 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.176 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:15.176 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "4", 2024-03-23T18:08:15.176 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.176 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.177 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:15.177 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-23T18:08:15.177 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.177 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.177 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.177 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.177 INFO:teuthology.orchestra.run.smithi086.stdout: "snapshot_clone_delay": { 2024-03-23T18:08:15.177 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "snapshot_clone_delay", 2024-03-23T18:08:15.177 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:15.178 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.178 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:15.178 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-03-23T18:08:15.178 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.178 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.178 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:15.178 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-23T18:08:15.178 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.178 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.179 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.179 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.179 INFO:teuthology.orchestra.run.smithi086.stdout: "snapshot_clone_no_wait": { 2024-03-23T18:08:15.179 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "snapshot_clone_no_wait", 2024-03-23T18:08:15.179 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:15.179 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.179 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:15.179 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-03-23T18:08:15.179 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.180 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.180 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:15.180 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-23T18:08:15.180 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.180 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.180 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.180 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:15.180 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:15.180 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.181 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:15.181 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix", 2024-03-23T18:08:15.181 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-03-23T18:08:15.181 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-03-23T18:08:15.181 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-03-23T18:08:15.181 INFO:teuthology.orchestra.run.smithi086.stdout: "discovery_interval": { 2024-03-23T18:08:15.181 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "discovery_interval", 2024-03-23T18:08:15.181 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-03-23T18:08:15.181 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.182 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:15.182 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "100", 2024-03-23T18:08:15.182 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.182 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.182 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:15.182 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.182 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.182 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.183 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.183 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.183 INFO:teuthology.orchestra.run.smithi086.stdout: "identifier": { 2024-03-23T18:08:15.183 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "identifier", 2024-03-23T18:08:15.183 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:15.183 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.183 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:15.183 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:15.183 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.184 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.184 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:15.184 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.184 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.184 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.184 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.184 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.184 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-03-23T18:08:15.184 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-03-23T18:08:15.185 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-03-23T18:08:15.185 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.185 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:15.185 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-03-23T18:08:15.185 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.185 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.185 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:15.185 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.185 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.186 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.186 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.186 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.186 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-03-23T18:08:15.186 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-03-23T18:08:15.186 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:15.186 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.186 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:15.186 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:15.187 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.187 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.187 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:15.187 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:15.187 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:15.187 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:15.187 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:15.187 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:15.188 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:15.188 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:15.188 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.188 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.188 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.188 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.188 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.188 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-03-23T18:08:15.188 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-03-23T18:08:15.189 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:15.189 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.189 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:15.189 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:15.189 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.189 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.189 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:15.189 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.189 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.190 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.190 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.190 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.190 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-03-23T18:08:15.190 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-03-23T18:08:15.190 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:15.190 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.190 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:15.191 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-03-23T18:08:15.191 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.191 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.191 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-03-23T18:08:15.191 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-03-23T18:08:15.191 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-03-23T18:08:15.191 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-03-23T18:08:15.191 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-03-23T18:08:15.191 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-03-23T18:08:15.192 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-03-23T18:08:15.192 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:15.192 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.192 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.192 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.192 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.192 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.192 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-03-23T18:08:15.192 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-03-23T18:08:15.193 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-03-23T18:08:15.193 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.193 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-03-23T18:08:15.193 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-03-23T18:08:15.193 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.193 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.193 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:15.193 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.194 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.194 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.194 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.194 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.194 INFO:teuthology.orchestra.run.smithi086.stdout: "zabbix_host": { 2024-03-23T18:08:15.194 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix_host", 2024-03-23T18:08:15.194 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:15.194 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.194 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:15.195 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-03-23T18:08:15.195 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.195 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.195 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:15.195 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.195 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.195 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.195 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.195 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.196 INFO:teuthology.orchestra.run.smithi086.stdout: "zabbix_port": { 2024-03-23T18:08:15.196 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix_port", 2024-03-23T18:08:15.196 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-03-23T18:08:15.196 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.196 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:15.196 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10051", 2024-03-23T18:08:15.196 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.196 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.197 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:15.197 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.197 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.197 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.197 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.197 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.197 INFO:teuthology.orchestra.run.smithi086.stdout: "zabbix_sender": { 2024-03-23T18:08:15.197 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix_sender", 2024-03-23T18:08:15.198 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-03-23T18:08:15.198 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-03-23T18:08:15.198 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-03-23T18:08:15.198 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-23T18:08:15.198 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-03-23T18:08:15.198 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-03-23T18:08:15.198 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-03-23T18:08:15.198 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-03-23T18:08:15.198 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-03-23T18:08:15.199 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-03-23T18:08:15.199 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-03-23T18:08:15.199 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:15.199 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:15.199 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:15.199 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:15.199 INFO:teuthology.orchestra.run.smithi086.stdout: "services": { 2024-03-23T18:08:15.199 INFO:teuthology.orchestra.run.smithi086.stdout: "dashboard": "https://172.21.15.86:8443/" 2024-03-23T18:08:15.200 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.200 INFO:teuthology.orchestra.run.smithi086.stdout: "always_on_modules": { 2024-03-23T18:08:15.200 INFO:teuthology.orchestra.run.smithi086.stdout: "octopus": [ 2024-03-23T18:08:15.200 INFO:teuthology.orchestra.run.smithi086.stdout: "balancer", 2024-03-23T18:08:15.200 INFO:teuthology.orchestra.run.smithi086.stdout: "crash", 2024-03-23T18:08:15.200 INFO:teuthology.orchestra.run.smithi086.stdout: "devicehealth", 2024-03-23T18:08:15.200 INFO:teuthology.orchestra.run.smithi086.stdout: "orchestrator", 2024-03-23T18:08:15.200 INFO:teuthology.orchestra.run.smithi086.stdout: "pg_autoscaler", 2024-03-23T18:08:15.200 INFO:teuthology.orchestra.run.smithi086.stdout: "progress", 2024-03-23T18:08:15.201 INFO:teuthology.orchestra.run.smithi086.stdout: "rbd_support", 2024-03-23T18:08:15.201 INFO:teuthology.orchestra.run.smithi086.stdout: "status", 2024-03-23T18:08:15.201 INFO:teuthology.orchestra.run.smithi086.stdout: "telemetry", 2024-03-23T18:08:15.201 INFO:teuthology.orchestra.run.smithi086.stdout: "volumes" 2024-03-23T18:08:15.201 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:15.201 INFO:teuthology.orchestra.run.smithi086.stdout: "pacific": [ 2024-03-23T18:08:15.201 INFO:teuthology.orchestra.run.smithi086.stdout: "balancer", 2024-03-23T18:08:15.201 INFO:teuthology.orchestra.run.smithi086.stdout: "crash", 2024-03-23T18:08:15.201 INFO:teuthology.orchestra.run.smithi086.stdout: "devicehealth", 2024-03-23T18:08:15.202 INFO:teuthology.orchestra.run.smithi086.stdout: "orchestrator", 2024-03-23T18:08:15.202 INFO:teuthology.orchestra.run.smithi086.stdout: "pg_autoscaler", 2024-03-23T18:08:15.202 INFO:teuthology.orchestra.run.smithi086.stdout: "progress", 2024-03-23T18:08:15.202 INFO:teuthology.orchestra.run.smithi086.stdout: "rbd_support", 2024-03-23T18:08:15.202 INFO:teuthology.orchestra.run.smithi086.stdout: "status", 2024-03-23T18:08:15.202 INFO:teuthology.orchestra.run.smithi086.stdout: "telemetry", 2024-03-23T18:08:15.202 INFO:teuthology.orchestra.run.smithi086.stdout: "volumes" 2024-03-23T18:08:15.202 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:15.203 INFO:teuthology.orchestra.run.smithi086.stdout: "quincy": [ 2024-03-23T18:08:15.203 INFO:teuthology.orchestra.run.smithi086.stdout: "balancer", 2024-03-23T18:08:15.203 INFO:teuthology.orchestra.run.smithi086.stdout: "crash", 2024-03-23T18:08:15.203 INFO:teuthology.orchestra.run.smithi086.stdout: "devicehealth", 2024-03-23T18:08:15.203 INFO:teuthology.orchestra.run.smithi086.stdout: "orchestrator", 2024-03-23T18:08:15.203 INFO:teuthology.orchestra.run.smithi086.stdout: "pg_autoscaler", 2024-03-23T18:08:15.203 INFO:teuthology.orchestra.run.smithi086.stdout: "progress", 2024-03-23T18:08:15.203 INFO:teuthology.orchestra.run.smithi086.stdout: "rbd_support", 2024-03-23T18:08:15.204 INFO:teuthology.orchestra.run.smithi086.stdout: "status", 2024-03-23T18:08:15.204 INFO:teuthology.orchestra.run.smithi086.stdout: "telemetry", 2024-03-23T18:08:15.204 INFO:teuthology.orchestra.run.smithi086.stdout: "volumes" 2024-03-23T18:08:15.204 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-03-23T18:08:15.204 INFO:teuthology.orchestra.run.smithi086.stdout: "reef": [ 2024-03-23T18:08:15.204 INFO:teuthology.orchestra.run.smithi086.stdout: "balancer", 2024-03-23T18:08:15.204 INFO:teuthology.orchestra.run.smithi086.stdout: "crash", 2024-03-23T18:08:15.204 INFO:teuthology.orchestra.run.smithi086.stdout: "devicehealth", 2024-03-23T18:08:15.204 INFO:teuthology.orchestra.run.smithi086.stdout: "orchestrator", 2024-03-23T18:08:15.205 INFO:teuthology.orchestra.run.smithi086.stdout: "pg_autoscaler", 2024-03-23T18:08:15.205 INFO:teuthology.orchestra.run.smithi086.stdout: "progress", 2024-03-23T18:08:15.205 INFO:teuthology.orchestra.run.smithi086.stdout: "rbd_support", 2024-03-23T18:08:15.205 INFO:teuthology.orchestra.run.smithi086.stdout: "status", 2024-03-23T18:08:15.205 INFO:teuthology.orchestra.run.smithi086.stdout: "telemetry", 2024-03-23T18:08:15.205 INFO:teuthology.orchestra.run.smithi086.stdout: "volumes" 2024-03-23T18:08:15.205 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-03-23T18:08:15.205 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.205 INFO:teuthology.orchestra.run.smithi086.stdout: "last_failure_osd_epoch": 83, 2024-03-23T18:08:15.206 INFO:teuthology.orchestra.run.smithi086.stdout: "active_clients": [ 2024-03-23T18:08:15.206 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:15.206 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "devicehealth", 2024-03-23T18:08:15.206 INFO:teuthology.orchestra.run.smithi086.stdout: "addrvec": [ 2024-03-23T18:08:15.206 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:15.206 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v2", 2024-03-23T18:08:15.206 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "172.21.15.86:0", 2024-03-23T18:08:15.206 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 2872809287 2024-03-23T18:08:15.207 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:15.207 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-03-23T18:08:15.207 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.207 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:15.207 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "libcephsqlite", 2024-03-23T18:08:15.207 INFO:teuthology.orchestra.run.smithi086.stdout: "addrvec": [ 2024-03-23T18:08:15.207 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:15.207 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v2", 2024-03-23T18:08:15.207 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "172.21.15.86:0", 2024-03-23T18:08:15.208 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 457911104 2024-03-23T18:08:15.208 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:15.208 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-03-23T18:08:15.208 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.208 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:15.208 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rbd_support", 2024-03-23T18:08:15.208 INFO:teuthology.orchestra.run.smithi086.stdout: "addrvec": [ 2024-03-23T18:08:15.208 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:15.208 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v2", 2024-03-23T18:08:15.209 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "172.21.15.86:0", 2024-03-23T18:08:15.209 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 2801433631 2024-03-23T18:08:15.209 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:15.209 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-03-23T18:08:15.209 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.209 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:15.209 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "snap_schedule", 2024-03-23T18:08:15.209 INFO:teuthology.orchestra.run.smithi086.stdout: "addrvec": [ 2024-03-23T18:08:15.210 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:15.210 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v2", 2024-03-23T18:08:15.210 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "172.21.15.86:0", 2024-03-23T18:08:15.210 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 709698460 2024-03-23T18:08:15.210 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:15.210 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-03-23T18:08:15.210 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.210 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:15.210 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "snap_schedule", 2024-03-23T18:08:15.211 INFO:teuthology.orchestra.run.smithi086.stdout: "addrvec": [ 2024-03-23T18:08:15.211 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:15.211 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v2", 2024-03-23T18:08:15.211 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "172.21.15.86:0", 2024-03-23T18:08:15.211 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 1122291704 2024-03-23T18:08:15.211 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:15.211 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-03-23T18:08:15.211 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:15.212 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:15.212 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "volumes", 2024-03-23T18:08:15.212 INFO:teuthology.orchestra.run.smithi086.stdout: "addrvec": [ 2024-03-23T18:08:15.212 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-03-23T18:08:15.212 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v2", 2024-03-23T18:08:15.212 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "172.21.15.86:0", 2024-03-23T18:08:15.212 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 4147537672 2024-03-23T18:08:15.212 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:15.212 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-03-23T18:08:15.213 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:15.213 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-03-23T18:08:15.213 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-23T18:08:15.215 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:13 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/2853284591' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T18:08:15.215 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:13 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/1747638679' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T18:08:15.215 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:13 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/2715833445' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-23T18:08:15.215 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:14 smithi092 ceph-mon[79089]: pgmap v556: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 4.7 MiB/s wr, 598 op/s 2024-03-23T18:08:15.215 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:13 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/2853284591' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T18:08:15.215 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:13 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/1747638679' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T18:08:15.216 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:13 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/2715833445' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-23T18:08:15.216 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:14 smithi152 ceph-mon[47949]: pgmap v556: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 4.7 MiB/s wr, 598 op/s 2024-03-23T18:08:15.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:13.056+0000 7f071e200700 1 -- 172.21.15.86:0/2715833445 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f06fc0a1290 msgr2=0x7f06fc0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:15.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:13.056+0000 7f071e200700 1 --2- 172.21.15.86:0/2715833445 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f06fc0a1290 0x7f06fc0a3750 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f070800b2e0 tx=0x7f0708005960 comp rx=0 tx=0).stop 2024-03-23T18:08:15.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:13.056+0000 7f071e200700 1 -- 172.21.15.86:0/2715833445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0718105f80 msgr2=0x7f0718199840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:15.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:13.056+0000 7f071e200700 1 --2- 172.21.15.86:0/2715833445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0718105f80 0x7f0718199840 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f070400e970 tx=0x7f070400ec80 comp rx=0 tx=0).stop 2024-03-23T18:08:15.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:13.056+0000 7f071e200700 1 -- 172.21.15.86:0/2715833445 shutdown_connections 2024-03-23T18:08:15.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:13.057+0000 7f071e200700 1 --2- 172.21.15.86:0/2715833445 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f071819dbe0 0x7f071819e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:15.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:13.057+0000 7f071e200700 1 --2- 172.21.15.86:0/2715833445 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f06fc0a1290 0x7f06fc0a3750 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:15.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:13.057+0000 7f071e200700 1 --2- 172.21.15.86:0/2715833445 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0718100e60 0x7f0718199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:15.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:13.057+0000 7f071e200700 1 --2- 172.21.15.86:0/2715833445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0718105f80 0x7f0718199840 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:15.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:13.057+0000 7f071e200700 1 -- 172.21.15.86:0/2715833445 >> 172.21.15.86:0/2715833445 conn(0x7f07180fc700 msgr2=0x7f07180fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:08:15.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:13.057+0000 7f071e200700 1 -- 172.21.15.86:0/2715833445 shutdown_connections 2024-03-23T18:08:15.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:13.057+0000 7f071e200700 1 -- 172.21.15.86:0/2715833445 wait complete. 2024-03-23T18:08:15.225 DEBUG:tasks.check_counter:Getting stats from a 2024-03-23T18:08:15.225 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph daemon mds.a perf dump 2024-03-23T18:08:15.226 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:13 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/2853284591' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T18:08:15.226 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:13 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/1747638679' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T18:08:15.226 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:13 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/2715833445' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-23T18:08:15.226 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:14 smithi086 ceph-mon[38221]: pgmap v556: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 4.7 MiB/s wr, 598 op/s 2024-03-23T18:08:15.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:15 smithi152 ceph-mon[47949]: pgmap v557: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.0 MiB/s wr, 417 op/s 2024-03-23T18:08:15.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:15 smithi086 ceph-mon[38221]: pgmap v557: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.0 MiB/s wr, 417 op/s 2024-03-23T18:08:15.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:15 smithi092 ceph-mon[79089]: pgmap v557: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.0 MiB/s wr, 417 op/s 2024-03-23T18:08:15.588 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T18:08:16.774 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-23T18:08:16.774 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-0": { 2024-03-23T18:08:16.774 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 1844, 2024-03-23T18:08:16.774 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 1846, 2024-03-23T18:08:16.774 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 87985936, 2024-03-23T18:08:16.775 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 573485, 2024-03-23T18:08:16.775 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 9, 2024-03-23T18:08:16.775 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 4, 2024-03-23T18:08:16.775 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.366590590, 2024-03-23T18:08:16.775 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.089854614, 2024-03-23T18:08:16.775 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.196756852, 2024-03-23T18:08:16.775 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.057503804, 2024-03-23T18:08:16.775 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T18:08:16.776 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 1844, 2024-03-23T18:08:16.776 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.126060352, 2024-03-23T18:08:16.776 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000068362 2024-03-23T18:08:16.776 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:16.776 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-03-23T18:08:16.776 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-23T18:08:16.776 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-23T18:08:16.777 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-23T18:08:16.777 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:16.777 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-23T18:08:16.777 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 0 2024-03-23T18:08:16.777 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:16.777 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-1": { 2024-03-23T18:08:16.777 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 2371, 2024-03-23T18:08:16.777 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 2297, 2024-03-23T18:08:16.778 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 82187217, 2024-03-23T18:08:16.778 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 791099, 2024-03-23T18:08:16.778 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 6, 2024-03-23T18:08:16.778 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 5, 2024-03-23T18:08:16.778 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.403030089, 2024-03-23T18:08:16.779 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.073787288, 2024-03-23T18:08:16.779 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.268674828, 2024-03-23T18:08:16.779 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.059678161, 2024-03-23T18:08:16.779 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T18:08:16.779 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 2296, 2024-03-23T18:08:16.779 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.108483688, 2024-03-23T18:08:16.779 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000047248 2024-03-23T18:08:16.779 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:16.780 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-03-23T18:08:16.780 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-23T18:08:16.780 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-23T18:08:16.780 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-23T18:08:16.780 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:16.780 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 1999536, 2024-03-23T18:08:16.780 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 184592 2024-03-23T18:08:16.780 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:16.781 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-2": { 2024-03-23T18:08:16.781 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 537, 2024-03-23T18:08:16.781 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 771, 2024-03-23T18:08:16.781 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 88219232, 2024-03-23T18:08:16.781 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 821641, 2024-03-23T18:08:16.781 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 9, 2024-03-23T18:08:16.781 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 6, 2024-03-23T18:08:16.781 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.266394746, 2024-03-23T18:08:16.781 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.034189100, 2024-03-23T18:08:16.782 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.203931932, 2024-03-23T18:08:16.782 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.018365688, 2024-03-23T18:08:16.782 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T18:08:16.782 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 769, 2024-03-23T18:08:16.782 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.080880000, 2024-03-23T18:08:16.782 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000105175 2024-03-23T18:08:16.783 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:16.783 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-03-23T18:08:16.783 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-23T18:08:16.783 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-23T18:08:16.783 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-23T18:08:16.783 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:16.784 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 3424, 2024-03-23T18:08:16.784 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 656800 2024-03-23T18:08:16.784 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:16.784 INFO:teuthology.orchestra.run.smithi086.stdout: "cct": { 2024-03-23T18:08:16.785 INFO:teuthology.orchestra.run.smithi086.stdout: "total_workers": 1, 2024-03-23T18:08:16.785 INFO:teuthology.orchestra.run.smithi086.stdout: "unhealthy_workers": 0 2024-03-23T18:08:16.785 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:16.785 INFO:teuthology.orchestra.run.smithi086.stdout: "finisher-MDSRank": { 2024-03-23T18:08:16.786 INFO:teuthology.orchestra.run.smithi086.stdout: "queue_len": 0, 2024-03-23T18:08:16.786 INFO:teuthology.orchestra.run.smithi086.stdout: "complete_latency": { 2024-03-23T18:08:16.786 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 7238, 2024-03-23T18:08:16.786 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.218158441, 2024-03-23T18:08:16.786 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000030140 2024-03-23T18:08:16.786 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:16.786 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:16.786 INFO:teuthology.orchestra.run.smithi086.stdout: "finisher-PurgeQueue": { 2024-03-23T18:08:16.787 INFO:teuthology.orchestra.run.smithi086.stdout: "queue_len": 0, 2024-03-23T18:08:16.787 INFO:teuthology.orchestra.run.smithi086.stdout: "complete_latency": { 2024-03-23T18:08:16.787 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-23T18:08:16.787 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-23T18:08:16.787 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-23T18:08:16.787 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:16.787 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:16.788 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": { 2024-03-23T18:08:16.788 INFO:teuthology.orchestra.run.smithi086.stdout: "request": 0, 2024-03-23T18:08:16.788 INFO:teuthology.orchestra.run.smithi086.stdout: "reply": 0, 2024-03-23T18:08:16.788 INFO:teuthology.orchestra.run.smithi086.stdout: "reply_latency": { 2024-03-23T18:08:16.788 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-23T18:08:16.788 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-23T18:08:16.788 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-23T18:08:16.789 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:16.789 INFO:teuthology.orchestra.run.smithi086.stdout: "slow_reply": 0, 2024-03-23T18:08:16.789 INFO:teuthology.orchestra.run.smithi086.stdout: "forward": 0, 2024-03-23T18:08:16.789 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_fetch_complete": 0, 2024-03-23T18:08:16.789 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_fetch_keys": 0, 2024-03-23T18:08:16.789 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_commit": 0, 2024-03-23T18:08:16.789 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_split": 0, 2024-03-23T18:08:16.789 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_merge": 0, 2024-03-23T18:08:16.789 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes": 14057, 2024-03-23T18:08:16.790 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_top": 122, 2024-03-23T18:08:16.790 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_bottom": 13935, 2024-03-23T18:08:16.790 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_pin_tail": 0, 2024-03-23T18:08:16.790 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_pinned": 13883, 2024-03-23T18:08:16.790 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_expired": 0, 2024-03-23T18:08:16.790 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_with_caps": 0, 2024-03-23T18:08:16.790 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": 0, 2024-03-23T18:08:16.791 INFO:teuthology.orchestra.run.smithi086.stdout: "subtrees": 14, 2024-03-23T18:08:16.791 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse": 0, 2024-03-23T18:08:16.791 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_hit": 0, 2024-03-23T18:08:16.791 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_forward": 0, 2024-03-23T18:08:16.791 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_discover": 0, 2024-03-23T18:08:16.791 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_dir_fetch": 0, 2024-03-23T18:08:16.791 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_remote_ino": 0, 2024-03-23T18:08:16.791 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_lock": 0, 2024-03-23T18:08:16.791 INFO:teuthology.orchestra.run.smithi086.stdout: "load_cent": 0, 2024-03-23T18:08:16.792 INFO:teuthology.orchestra.run.smithi086.stdout: "q": 0, 2024-03-23T18:08:16.792 INFO:teuthology.orchestra.run.smithi086.stdout: "exported": 0, 2024-03-23T18:08:16.792 INFO:teuthology.orchestra.run.smithi086.stdout: "exported_inodes": 0, 2024-03-23T18:08:16.792 INFO:teuthology.orchestra.run.smithi086.stdout: "imported": 0, 2024-03-23T18:08:16.792 INFO:teuthology.orchestra.run.smithi086.stdout: "imported_inodes": 0, 2024-03-23T18:08:16.792 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_dir_fetch": 0, 2024-03-23T18:08:16.792 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_backtrace_fetch": 0, 2024-03-23T18:08:16.792 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_peer_discover": 0, 2024-03-23T18:08:16.792 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rfiles": 0, 2024-03-23T18:08:16.793 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rbytes": 0, 2024-03-23T18:08:16.793 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rsnaps": 0, 2024-03-23T18:08:16.793 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_backtrace_fetch": 0, 2024-03-23T18:08:16.793 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_set_tag": 0, 2024-03-23T18:08:16.793 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_backtrace_repaired": 0, 2024-03-23T18:08:16.793 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_inotable_repaired": 0, 2024-03-23T18:08:16.793 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dir_inodes": 0, 2024-03-23T18:08:16.793 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dir_base_inodes": 0, 2024-03-23T18:08:16.793 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dirfrag_rstats": 0, 2024-03-23T18:08:16.794 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_file_inodes": 0, 2024-03-23T18:08:16.794 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_inode_file_caps": 0, 2024-03-23T18:08:16.794 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_revoke": 0, 2024-03-23T18:08:16.794 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_grant": 0, 2024-03-23T18:08:16.794 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_trunc": 0, 2024-03-23T18:08:16.794 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-23T18:08:16.794 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-23T18:08:16.794 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_caps": 0, 2024-03-23T18:08:16.795 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_caps_dirty": 0, 2024-03-23T18:08:16.795 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_cap_release": 0, 2024-03-23T18:08:16.795 INFO:teuthology.orchestra.run.smithi086.stdout: "process_request_cap_release": 0 2024-03-23T18:08:16.795 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:16.795 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_cache": { 2024-03-23T18:08:16.795 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_update": 0, 2024-03-23T18:08:16.795 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_update_receipt": 0, 2024-03-23T18:08:16.795 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_try_discover": 0, 2024-03-23T18:08:16.795 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_send_discover": 0, 2024-03-23T18:08:16.796 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_handle_discover": 0, 2024-03-23T18:08:16.796 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays": 0, 2024-03-23T18:08:16.796 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays_delayed": 0, 2024-03-23T18:08:16.796 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays_enqueuing": 0, 2024-03-23T18:08:16.796 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_created": 0, 2024-03-23T18:08:16.796 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_enqueued": 0, 2024-03-23T18:08:16.796 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_reintegrated": 0, 2024-03-23T18:08:16.796 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_migrated": 0, 2024-03-23T18:08:16.797 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_processing": 0, 2024-03-23T18:08:16.797 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_enqueued": 0, 2024-03-23T18:08:16.797 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_prioritized": 0, 2024-03-23T18:08:16.797 INFO:teuthology.orchestra.run.smithi086.stdout: "recovery_started": 0, 2024-03-23T18:08:16.797 INFO:teuthology.orchestra.run.smithi086.stdout: "recovery_completed": 0, 2024-03-23T18:08:16.797 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_enqueue_scrub": 0, 2024-03-23T18:08:16.797 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_exportdir": 0, 2024-03-23T18:08:16.797 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_flush": 0, 2024-03-23T18:08:16.797 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_fragmentdir": 0, 2024-03-23T18:08:16.798 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_fragstats": 0, 2024-03-23T18:08:16.798 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_inodestats": 0 2024-03-23T18:08:16.798 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:16.798 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_log": { 2024-03-23T18:08:16.798 INFO:teuthology.orchestra.run.smithi086.stdout: "evadd": 0, 2024-03-23T18:08:16.798 INFO:teuthology.orchestra.run.smithi086.stdout: "evex": 0, 2024-03-23T18:08:16.798 INFO:teuthology.orchestra.run.smithi086.stdout: "evtrm": 0, 2024-03-23T18:08:16.798 INFO:teuthology.orchestra.run.smithi086.stdout: "ev": 71299, 2024-03-23T18:08:16.798 INFO:teuthology.orchestra.run.smithi086.stdout: "evexg": 0, 2024-03-23T18:08:16.799 INFO:teuthology.orchestra.run.smithi086.stdout: "evexd": 0, 2024-03-23T18:08:16.799 INFO:teuthology.orchestra.run.smithi086.stdout: "segadd": 0, 2024-03-23T18:08:16.799 INFO:teuthology.orchestra.run.smithi086.stdout: "segex": 0, 2024-03-23T18:08:16.799 INFO:teuthology.orchestra.run.smithi086.stdout: "segtrm": 0, 2024-03-23T18:08:16.799 INFO:teuthology.orchestra.run.smithi086.stdout: "seg": 96, 2024-03-23T18:08:16.799 INFO:teuthology.orchestra.run.smithi086.stdout: "segexg": 0, 2024-03-23T18:08:16.799 INFO:teuthology.orchestra.run.smithi086.stdout: "segexd": 0, 2024-03-23T18:08:16.799 INFO:teuthology.orchestra.run.smithi086.stdout: "expos": 4194304, 2024-03-23T18:08:16.799 INFO:teuthology.orchestra.run.smithi086.stdout: "wrpos": 259389480, 2024-03-23T18:08:16.800 INFO:teuthology.orchestra.run.smithi086.stdout: "rdpos": 259378878, 2024-03-23T18:08:16.800 INFO:teuthology.orchestra.run.smithi086.stdout: "jlat": { 2024-03-23T18:08:16.800 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-23T18:08:16.800 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-23T18:08:16.800 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-23T18:08:16.800 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:16.800 INFO:teuthology.orchestra.run.smithi086.stdout: "replayed": 71299 2024-03-23T18:08:16.800 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:16.800 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_mem": { 2024-03-23T18:08:16.801 INFO:teuthology.orchestra.run.smithi086.stdout: "ino": 1169, 2024-03-23T18:08:16.801 INFO:teuthology.orchestra.run.smithi086.stdout: "ino+": 14149, 2024-03-23T18:08:16.801 INFO:teuthology.orchestra.run.smithi086.stdout: "ino-": 12980, 2024-03-23T18:08:16.801 INFO:teuthology.orchestra.run.smithi086.stdout: "dir": 68, 2024-03-23T18:08:16.801 INFO:teuthology.orchestra.run.smithi086.stdout: "dir+": 154, 2024-03-23T18:08:16.801 INFO:teuthology.orchestra.run.smithi086.stdout: "dir-": 86, 2024-03-23T18:08:16.801 INFO:teuthology.orchestra.run.smithi086.stdout: "dn": 14057, 2024-03-23T18:08:16.801 INFO:teuthology.orchestra.run.smithi086.stdout: "dn+": 14374, 2024-03-23T18:08:16.802 INFO:teuthology.orchestra.run.smithi086.stdout: "dn-": 317, 2024-03-23T18:08:16.802 INFO:teuthology.orchestra.run.smithi086.stdout: "cap": 0, 2024-03-23T18:08:16.802 INFO:teuthology.orchestra.run.smithi086.stdout: "cap+": 0, 2024-03-23T18:08:16.802 INFO:teuthology.orchestra.run.smithi086.stdout: "cap-": 0, 2024-03-23T18:08:16.802 INFO:teuthology.orchestra.run.smithi086.stdout: "rss": 67108, 2024-03-23T18:08:16.802 INFO:teuthology.orchestra.run.smithi086.stdout: "heap": 207156 2024-03-23T18:08:16.802 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:16.802 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_server": { 2024-03-23T18:08:16.802 INFO:teuthology.orchestra.run.smithi086.stdout: "dispatch_client_request": 0, 2024-03-23T18:08:16.803 INFO:teuthology.orchestra.run.smithi086.stdout: "dispatch_server_request": 0, 2024-03-23T18:08:16.803 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_request": 0, 2024-03-23T18:08:16.803 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_session": 0, 2024-03-23T18:08:16.803 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_peer_request": 0, 2024-03-23T18:08:16.803 INFO:teuthology.orchestra.run.smithi086.stdout: "req_create_latency": { 2024-03-23T18:08:16.803 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-23T18:08:16.803 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-23T18:08:16.803 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-23T18:08:16.803 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:16.804 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getattr_latency": { 2024-03-23T18:08:16.804 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-23T18:08:16.804 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-23T18:08:16.804 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-23T18:08:16.804 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:16.804 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getfilelock_latency": { 2024-03-23T18:08:16.804 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-23T18:08:16.804 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-23T18:08:16.804 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-23T18:08:16.805 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:16.805 INFO:teuthology.orchestra.run.smithi086.stdout: "req_link_latency": { 2024-03-23T18:08:16.805 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-23T18:08:16.805 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-23T18:08:16.805 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-23T18:08:16.805 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:16.805 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookup_latency": { 2024-03-23T18:08:16.805 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-23T18:08:16.805 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-23T18:08:16.806 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-23T18:08:16.806 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:16.806 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookuphash_latency": { 2024-03-23T18:08:16.806 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-23T18:08:16.806 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-23T18:08:16.806 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-23T18:08:16.806 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:16.806 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupino_latency": { 2024-03-23T18:08:16.806 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-23T18:08:16.807 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-23T18:08:16.807 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-23T18:08:16.807 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:16.807 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupname_latency": { 2024-03-23T18:08:16.807 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-23T18:08:16.807 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-23T18:08:16.807 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-23T18:08:16.807 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:16.808 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupparent_latency": { 2024-03-23T18:08:16.808 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-23T18:08:16.808 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-23T18:08:16.808 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-23T18:08:16.808 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:16.808 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupsnap_latency": { 2024-03-23T18:08:16.809 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-23T18:08:16.809 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-23T18:08:16.809 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-23T18:08:16.809 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:16.809 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lssnap_latency": { 2024-03-23T18:08:16.809 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-23T18:08:16.809 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-23T18:08:16.809 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-23T18:08:16.809 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:16.810 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mkdir_latency": { 2024-03-23T18:08:16.810 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-23T18:08:16.810 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-23T18:08:16.810 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-23T18:08:16.810 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:16.810 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mknod_latency": { 2024-03-23T18:08:16.811 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-23T18:08:16.811 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-23T18:08:16.811 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-23T18:08:16.811 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:16.811 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mksnap_latency": { 2024-03-23T18:08:16.811 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-23T18:08:16.811 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-23T18:08:16.811 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-23T18:08:16.812 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:16.812 INFO:teuthology.orchestra.run.smithi086.stdout: "req_open_latency": { 2024-03-23T18:08:16.812 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-23T18:08:16.812 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-23T18:08:16.812 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-23T18:08:16.812 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:16.812 INFO:teuthology.orchestra.run.smithi086.stdout: "req_readdir_latency": { 2024-03-23T18:08:16.812 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-23T18:08:16.812 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-23T18:08:16.813 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-23T18:08:16.813 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:16.813 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rename_latency": { 2024-03-23T18:08:16.813 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-23T18:08:16.813 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-23T18:08:16.813 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-23T18:08:16.813 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:16.813 INFO:teuthology.orchestra.run.smithi086.stdout: "req_renamesnap_latency": { 2024-03-23T18:08:16.813 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-23T18:08:16.814 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-23T18:08:16.814 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-23T18:08:16.814 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:16.814 INFO:teuthology.orchestra.run.smithi086.stdout: "req_snapdiff_latency": { 2024-03-23T18:08:16.814 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-23T18:08:16.814 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-23T18:08:16.814 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-23T18:08:16.814 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:16.814 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmdir_latency": { 2024-03-23T18:08:16.815 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-23T18:08:16.815 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-23T18:08:16.815 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-23T18:08:16.815 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:16.815 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmsnap_latency": { 2024-03-23T18:08:16.815 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-23T18:08:16.815 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-23T18:08:16.815 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-23T18:08:16.816 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:16.816 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmxattr_latency": { 2024-03-23T18:08:16.816 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-23T18:08:16.816 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-23T18:08:16.816 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-23T18:08:16.816 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:16.816 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setattr_latency": { 2024-03-23T18:08:16.816 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-23T18:08:16.816 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-23T18:08:16.817 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-23T18:08:16.817 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:16.817 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setdirlayout_latency": { 2024-03-23T18:08:16.817 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-23T18:08:16.817 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-23T18:08:16.817 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-23T18:08:16.817 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:16.817 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setfilelock_latency": { 2024-03-23T18:08:16.817 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-23T18:08:16.818 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-23T18:08:16.818 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-23T18:08:16.818 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:16.818 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setlayout_latency": { 2024-03-23T18:08:16.818 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-23T18:08:16.818 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-23T18:08:16.819 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-23T18:08:16.819 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:16.819 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setxattr_latency": { 2024-03-23T18:08:16.819 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-23T18:08:16.819 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-23T18:08:16.819 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-23T18:08:16.820 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:16.820 INFO:teuthology.orchestra.run.smithi086.stdout: "req_symlink_latency": { 2024-03-23T18:08:16.820 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-23T18:08:16.820 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-23T18:08:16.820 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-23T18:08:16.820 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:16.820 INFO:teuthology.orchestra.run.smithi086.stdout: "req_unlink_latency": { 2024-03-23T18:08:16.820 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-23T18:08:16.820 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-23T18:08:16.821 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-23T18:08:16.821 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:16.821 INFO:teuthology.orchestra.run.smithi086.stdout: "cap_revoke_eviction": 0, 2024-03-23T18:08:16.821 INFO:teuthology.orchestra.run.smithi086.stdout: "cap_acquisition_throttle": 0, 2024-03-23T18:08:16.821 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getvxattr_latency": { 2024-03-23T18:08:16.821 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-23T18:08:16.821 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-23T18:08:16.821 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-23T18:08:16.821 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:16.822 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:16.822 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_sessions": { 2024-03-23T18:08:16.822 INFO:teuthology.orchestra.run.smithi086.stdout: "session_count": 1, 2024-03-23T18:08:16.822 INFO:teuthology.orchestra.run.smithi086.stdout: "session_add": 3, 2024-03-23T18:08:16.822 INFO:teuthology.orchestra.run.smithi086.stdout: "session_remove": 2, 2024-03-23T18:08:16.822 INFO:teuthology.orchestra.run.smithi086.stdout: "sessions_open": 3, 2024-03-23T18:08:16.822 INFO:teuthology.orchestra.run.smithi086.stdout: "sessions_stale": 0, 2024-03-23T18:08:16.822 INFO:teuthology.orchestra.run.smithi086.stdout: "total_load": 0, 2024-03-23T18:08:16.823 INFO:teuthology.orchestra.run.smithi086.stdout: "average_load": 0, 2024-03-23T18:08:16.823 INFO:teuthology.orchestra.run.smithi086.stdout: "avg_session_uptime": 5574, 2024-03-23T18:08:16.823 INFO:teuthology.orchestra.run.smithi086.stdout: "mdthresh_evicted": 0 2024-03-23T18:08:16.823 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:16.823 INFO:teuthology.orchestra.run.smithi086.stdout: "mempool": { 2024-03-23T18:08:16.823 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_bytes": 0, 2024-03-23T18:08:16.823 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_items": 0, 2024-03-23T18:08:16.823 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_bytes": 0, 2024-03-23T18:08:16.823 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_items": 0, 2024-03-23T18:08:16.824 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_bytes": 0, 2024-03-23T18:08:16.824 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_items": 0, 2024-03-23T18:08:16.824 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-23T18:08:16.824 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_items": 0, 2024-03-23T18:08:16.824 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-23T18:08:16.824 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_items": 0, 2024-03-23T18:08:16.824 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_bytes": 0, 2024-03-23T18:08:16.824 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_items": 0, 2024-03-23T18:08:16.824 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-23T18:08:16.825 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_items": 0, 2024-03-23T18:08:16.825 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_bytes": 0, 2024-03-23T18:08:16.825 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_items": 0, 2024-03-23T18:08:16.825 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_bytes": 0, 2024-03-23T18:08:16.825 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_items": 0, 2024-03-23T18:08:16.825 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-23T18:08:16.825 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_items": 0, 2024-03-23T18:08:16.825 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-23T18:08:16.825 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_items": 0, 2024-03-23T18:08:16.826 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_bytes": 0, 2024-03-23T18:08:16.826 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_items": 0, 2024-03-23T18:08:16.826 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_bytes": 0, 2024-03-23T18:08:16.826 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_items": 0, 2024-03-23T18:08:16.826 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-23T18:08:16.826 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_items": 0, 2024-03-23T18:08:16.826 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_bytes": 0, 2024-03-23T18:08:16.826 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_items": 0, 2024-03-23T18:08:16.826 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_bytes": 0, 2024-03-23T18:08:16.827 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_items": 0, 2024-03-23T18:08:16.827 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_bytes": 0, 2024-03-23T18:08:16.827 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_items": 0, 2024-03-23T18:08:16.827 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_bytes": 0, 2024-03-23T18:08:16.827 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_items": 0, 2024-03-23T18:08:16.827 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_bytes": 284366, 2024-03-23T18:08:16.827 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_items": 1371, 2024-03-23T18:08:16.827 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_bytes": 0, 2024-03-23T18:08:16.827 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_items": 0, 2024-03-23T18:08:16.828 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_bytes": 0, 2024-03-23T18:08:16.828 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_items": 0, 2024-03-23T18:08:16.828 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_bytes": 0, 2024-03-23T18:08:16.828 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_items": 0, 2024-03-23T18:08:16.828 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_bytes": 0, 2024-03-23T18:08:16.828 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_items": 0, 2024-03-23T18:08:16.828 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_bytes": 8056, 2024-03-23T18:08:16.828 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_items": 168, 2024-03-23T18:08:16.828 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_bytes": 0, 2024-03-23T18:08:16.829 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_items": 0, 2024-03-23T18:08:16.829 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_bytes": 0, 2024-03-23T18:08:16.829 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_items": 0, 2024-03-23T18:08:16.829 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_bytes": 12884556, 2024-03-23T18:08:16.829 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_items": 120214, 2024-03-23T18:08:16.829 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_bytes": 0, 2024-03-23T18:08:16.830 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_items": 0, 2024-03-23T18:08:16.830 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_bytes": 0, 2024-03-23T18:08:16.830 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_items": 0 2024-03-23T18:08:16.830 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:16.830 INFO:teuthology.orchestra.run.smithi086.stdout: "objecter": { 2024-03-23T18:08:16.830 INFO:teuthology.orchestra.run.smithi086.stdout: "op_active": 0, 2024-03-23T18:08:16.830 INFO:teuthology.orchestra.run.smithi086.stdout: "op_laggy": 0, 2024-03-23T18:08:16.830 INFO:teuthology.orchestra.run.smithi086.stdout: "op_send": 4324, 2024-03-23T18:08:16.830 INFO:teuthology.orchestra.run.smithi086.stdout: "op_send_bytes": 21, 2024-03-23T18:08:16.831 INFO:teuthology.orchestra.run.smithi086.stdout: "op_resend": 3, 2024-03-23T18:08:16.831 INFO:teuthology.orchestra.run.smithi086.stdout: "op_reply": 4324, 2024-03-23T18:08:16.831 INFO:teuthology.orchestra.run.smithi086.stdout: "op_latency": { 2024-03-23T18:08:16.831 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 4324, 2024-03-23T18:08:16.831 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 7.779838193, 2024-03-23T18:08:16.831 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.001799222 2024-03-23T18:08:16.831 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:16.831 INFO:teuthology.orchestra.run.smithi086.stdout: "op_inflight": 0, 2024-03-23T18:08:16.831 INFO:teuthology.orchestra.run.smithi086.stdout: "oplen_avg": { 2024-03-23T18:08:16.832 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 4324, 2024-03-23T18:08:16.832 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 4325 2024-03-23T18:08:16.832 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:16.832 INFO:teuthology.orchestra.run.smithi086.stdout: "op": 4324, 2024-03-23T18:08:16.832 INFO:teuthology.orchestra.run.smithi086.stdout: "op_r": 4324, 2024-03-23T18:08:16.832 INFO:teuthology.orchestra.run.smithi086.stdout: "op_w": 0, 2024-03-23T18:08:16.832 INFO:teuthology.orchestra.run.smithi086.stdout: "op_rmw": 0, 2024-03-23T18:08:16.832 INFO:teuthology.orchestra.run.smithi086.stdout: "op_pg": 0, 2024-03-23T18:08:16.833 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_stat": 2392, 2024-03-23T18:08:16.833 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_create": 0, 2024-03-23T18:08:16.833 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_read": 1930, 2024-03-23T18:08:16.833 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_write": 0, 2024-03-23T18:08:16.833 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_writefull": 0, 2024-03-23T18:08:16.833 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_writesame": 0, 2024-03-23T18:08:16.833 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_append": 0, 2024-03-23T18:08:16.833 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_zero": 0, 2024-03-23T18:08:16.834 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_truncate": 0, 2024-03-23T18:08:16.834 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_delete": 0, 2024-03-23T18:08:16.834 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_mapext": 0, 2024-03-23T18:08:16.834 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_sparse_read": 0, 2024-03-23T18:08:16.834 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_clonerange": 0, 2024-03-23T18:08:16.834 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_getxattr": 1, 2024-03-23T18:08:16.834 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_setxattr": 0, 2024-03-23T18:08:16.834 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_cmpxattr": 0, 2024-03-23T18:08:16.835 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_rmxattr": 0, 2024-03-23T18:08:16.835 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_resetxattrs": 0, 2024-03-23T18:08:16.835 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_call": 0, 2024-03-23T18:08:16.835 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_watch": 0, 2024-03-23T18:08:16.835 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_notify": 0, 2024-03-23T18:08:16.835 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_src_cmpxattr": 0, 2024-03-23T18:08:16.835 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_pgls": 0, 2024-03-23T18:08:16.835 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_pgls_filter": 0, 2024-03-23T18:08:16.835 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_other": 0, 2024-03-23T18:08:16.836 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_active": 0, 2024-03-23T18:08:16.836 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_send": 0, 2024-03-23T18:08:16.836 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_resend": 0, 2024-03-23T18:08:16.836 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_ping": 0, 2024-03-23T18:08:16.836 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_active": 0, 2024-03-23T18:08:16.836 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_send": 0, 2024-03-23T18:08:16.836 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_resend": 0, 2024-03-23T18:08:16.836 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_active": 0, 2024-03-23T18:08:16.837 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_send": 0, 2024-03-23T18:08:16.837 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_resend": 0, 2024-03-23T18:08:16.837 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_active": 0, 2024-03-23T18:08:16.837 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_send": 0, 2024-03-23T18:08:16.837 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_resend": 0, 2024-03-23T18:08:16.837 INFO:teuthology.orchestra.run.smithi086.stdout: "command_active": 0, 2024-03-23T18:08:16.837 INFO:teuthology.orchestra.run.smithi086.stdout: "command_send": 0, 2024-03-23T18:08:16.837 INFO:teuthology.orchestra.run.smithi086.stdout: "command_resend": 0, 2024-03-23T18:08:16.838 INFO:teuthology.orchestra.run.smithi086.stdout: "map_epoch": 110, 2024-03-23T18:08:16.838 INFO:teuthology.orchestra.run.smithi086.stdout: "map_full": 0, 2024-03-23T18:08:16.838 INFO:teuthology.orchestra.run.smithi086.stdout: "map_inc": 30, 2024-03-23T18:08:16.838 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_sessions": 11, 2024-03-23T18:08:16.838 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_session_open": 12, 2024-03-23T18:08:16.838 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_session_close": 1, 2024-03-23T18:08:16.838 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_laggy": 0, 2024-03-23T18:08:16.838 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_wr": 0, 2024-03-23T18:08:16.839 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_rd": 2, 2024-03-23T18:08:16.839 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_del": 0 2024-03-23T18:08:16.839 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:16.839 INFO:teuthology.orchestra.run.smithi086.stdout: "oft": { 2024-03-23T18:08:16.839 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_objs": 0, 2024-03-23T18:08:16.839 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_kv_pairs": 0, 2024-03-23T18:08:16.840 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_updates": 0, 2024-03-23T18:08:16.840 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_removes": 0 2024-03-23T18:08:16.840 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:16.840 INFO:teuthology.orchestra.run.smithi086.stdout: "osd-slow-ops": { 2024-03-23T18:08:16.840 INFO:teuthology.orchestra.run.smithi086.stdout: "slow_ops_count": 0 2024-03-23T18:08:16.840 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:16.840 INFO:teuthology.orchestra.run.smithi086.stdout: "purge_queue": { 2024-03-23T18:08:16.840 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_ops": 0, 2024-03-23T18:08:16.840 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_ops_high_water": 0, 2024-03-23T18:08:16.841 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing": 0, 2024-03-23T18:08:16.841 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_high_water": 0, 2024-03-23T18:08:16.841 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executed": 0, 2024-03-23T18:08:16.841 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_item_in_journal": 0 2024-03-23T18:08:16.841 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:16.841 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-23T18:08:16.841 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-03-23T18:08:16.841 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-03-23T18:08:16.842 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-03-23T18:08:16.842 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 4748, 2024-03-23T18:08:16.842 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 257940900, 2024-03-23T18:08:16.842 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-03-23T18:08:16.842 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 4748, 2024-03-23T18:08:16.842 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-03-23T18:08:16.842 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-03-23T18:08:16.842 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 4748, 2024-03-23T18:08:16.842 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 257940900, 2024-03-23T18:08:16.843 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-03-23T18:08:16.843 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-23T18:08:16.843 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-23T18:08:16.843 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-23T18:08:16.843 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:16.843 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:16.843 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-objecter_bytes": { 2024-03-23T18:08:16.843 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-03-23T18:08:16.843 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-03-23T18:08:16.844 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-03-23T18:08:16.844 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-03-23T18:08:16.844 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-03-23T18:08:16.844 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-03-23T18:08:16.844 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-03-23T18:08:16.844 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 4324, 2024-03-23T18:08:16.844 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 255198009, 2024-03-23T18:08:16.844 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 4324, 2024-03-23T18:08:16.844 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 255198009, 2024-03-23T18:08:16.845 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-03-23T18:08:16.845 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-23T18:08:16.845 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-23T18:08:16.845 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-23T18:08:16.845 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:16.845 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:16.845 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-objecter_ops": { 2024-03-23T18:08:16.845 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-03-23T18:08:16.846 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 1024, 2024-03-23T18:08:16.846 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-03-23T18:08:16.846 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-03-23T18:08:16.846 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-03-23T18:08:16.846 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-03-23T18:08:16.846 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-03-23T18:08:16.846 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 4324, 2024-03-23T18:08:16.846 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 4324, 2024-03-23T18:08:16.846 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 4324, 2024-03-23T18:08:16.847 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 4324, 2024-03-23T18:08:16.847 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-03-23T18:08:16.847 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-23T18:08:16.847 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-23T18:08:16.847 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-23T18:08:16.847 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:16.847 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:16.847 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-write_buf_throttle": { 2024-03-23T18:08:16.847 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-03-23T18:08:16.848 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 3758096384, 2024-03-23T18:08:16.848 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-03-23T18:08:16.848 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-03-23T18:08:16.848 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-03-23T18:08:16.848 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-03-23T18:08:16.848 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-03-23T18:08:16.848 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-03-23T18:08:16.848 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-03-23T18:08:16.848 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 0, 2024-03-23T18:08:16.849 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 0, 2024-03-23T18:08:16.849 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-03-23T18:08:16.849 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-23T18:08:16.849 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-23T18:08:16.849 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-23T18:08:16.849 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:16.849 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:16.849 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-write_buf_throttle-0x5633c2748640": { 2024-03-23T18:08:16.849 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-03-23T18:08:16.850 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 3758096384, 2024-03-23T18:08:16.850 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-03-23T18:08:16.850 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-03-23T18:08:16.850 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-03-23T18:08:16.850 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-03-23T18:08:16.850 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-03-23T18:08:16.850 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-03-23T18:08:16.850 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-03-23T18:08:16.850 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 0, 2024-03-23T18:08:16.851 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 0, 2024-03-23T18:08:16.851 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-03-23T18:08:16.851 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-23T18:08:16.851 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-23T18:08:16.851 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-23T18:08:16.851 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:16.851 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:16.851 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-23T18:08:17.141 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:16 smithi086 ceph-mon[38221]: osdmap e110: 12 total, 12 up, 12 in 2024-03-23T18:08:17.166 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-03-23T18:08:17.166 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-03-23T18:08:17.167 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=0 2024-03-23T18:08:17.167 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-03-23T18:08:17.167 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-03-23T18:08:17.167 DEBUG:tasks.check_counter:Getting stats from d 2024-03-23T18:08:17.167 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph daemon mds.d perf dump 2024-03-23T18:08:17.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:16 smithi092 ceph-mon[79089]: osdmap e110: 12 total, 12 up, 12 in 2024-03-23T18:08:17.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:16 smithi152 ceph-mon[47949]: osdmap e110: 12 total, 12 up, 12 in 2024-03-23T18:08:17.527 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T18:08:18.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:18 smithi086 ceph-mon[38221]: pgmap v559: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 190 op/s 2024-03-23T18:08:18.310 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:18 smithi092 ceph-mon[79089]: pgmap v559: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 190 op/s 2024-03-23T18:08:18.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:18 smithi152 ceph-mon[47949]: pgmap v559: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 190 op/s 2024-03-23T18:08:20.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:20 smithi086 ceph-mon[38221]: pgmap v560: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 190 op/s 2024-03-23T18:08:20.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:20 smithi092 ceph-mon[79089]: pgmap v560: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 190 op/s 2024-03-23T18:08:20.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:20 smithi152 ceph-mon[47949]: pgmap v560: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 190 op/s 2024-03-23T18:08:21.583 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-23T18:08:21.584 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-0": { 2024-03-23T18:08:21.584 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 19204, 2024-03-23T18:08:21.584 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 19426, 2024-03-23T18:08:21.584 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 4798064, 2024-03-23T18:08:21.584 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 164442496, 2024-03-23T18:08:21.584 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 25, 2024-03-23T18:08:21.584 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 6, 2024-03-23T18:08:21.584 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 2.625934674, 2024-03-23T18:08:21.585 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 1.012809273, 2024-03-23T18:08:21.585 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.467976593, 2024-03-23T18:08:21.585 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.934072496, 2024-03-23T18:08:21.585 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T18:08:21.585 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 19419, 2024-03-23T18:08:21.585 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 2.066661867, 2024-03-23T18:08:21.585 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000106424 2024-03-23T18:08:21.585 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:21.585 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-03-23T18:08:21.586 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 18, 2024-03-23T18:08:21.586 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000002904, 2024-03-23T18:08:21.586 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000161 2024-03-23T18:08:21.586 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:21.586 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 1072, 2024-03-23T18:08:21.586 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 458160 2024-03-23T18:08:21.586 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:21.586 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-1": { 2024-03-23T18:08:21.587 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 50158, 2024-03-23T18:08:21.587 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 42051, 2024-03-23T18:08:21.587 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 18984878, 2024-03-23T18:08:21.587 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 146209013, 2024-03-23T18:08:21.587 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 8, 2024-03-23T18:08:21.587 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 6, 2024-03-23T18:08:21.587 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 5.063720631, 2024-03-23T18:08:21.587 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 2.093902871, 2024-03-23T18:08:21.588 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 2.480691703, 2024-03-23T18:08:21.588 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.866447677, 2024-03-23T18:08:21.588 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T18:08:21.588 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 42053, 2024-03-23T18:08:21.588 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 4.482497521, 2024-03-23T18:08:21.588 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000106591 2024-03-23T18:08:21.588 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:21.588 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-03-23T18:08:21.588 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 56732, 2024-03-23T18:08:21.589 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.013509489, 2024-03-23T18:08:21.589 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000238 2024-03-23T18:08:21.589 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:21.589 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 2024496, 2024-03-23T18:08:21.589 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 202464 2024-03-23T18:08:21.589 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:21.589 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-2": { 2024-03-23T18:08:21.589 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 334715, 2024-03-23T18:08:21.589 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 392258, 2024-03-23T18:08:21.590 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 111059612, 2024-03-23T18:08:21.590 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 316833223, 2024-03-23T18:08:21.590 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 13, 2024-03-23T18:08:21.590 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 6, 2024-03-23T18:08:21.590 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 49.880647035, 2024-03-23T18:08:21.590 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 31.981780713, 2024-03-23T18:08:21.590 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 22.302789915, 2024-03-23T18:08:21.590 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.754721768, 2024-03-23T18:08:21.590 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T18:08:21.591 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 392258, 2024-03-23T18:08:21.591 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 29.080256448, 2024-03-23T18:08:21.591 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000074135 2024-03-23T18:08:21.591 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:21.591 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-03-23T18:08:21.591 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 639754, 2024-03-23T18:08:21.591 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.178194581, 2024-03-23T18:08:21.591 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000278 2024-03-23T18:08:21.592 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:21.592 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 1072, 2024-03-23T18:08:21.592 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 119040 2024-03-23T18:08:21.592 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:21.592 INFO:teuthology.orchestra.run.smithi086.stdout: "cct": { 2024-03-23T18:08:21.592 INFO:teuthology.orchestra.run.smithi086.stdout: "total_workers": 1, 2024-03-23T18:08:21.592 INFO:teuthology.orchestra.run.smithi086.stdout: "unhealthy_workers": 0 2024-03-23T18:08:21.592 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:21.592 INFO:teuthology.orchestra.run.smithi086.stdout: "finisher-MDSRank": { 2024-03-23T18:08:21.593 INFO:teuthology.orchestra.run.smithi086.stdout: "queue_len": 0, 2024-03-23T18:08:21.593 INFO:teuthology.orchestra.run.smithi086.stdout: "complete_latency": { 2024-03-23T18:08:21.593 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 52662, 2024-03-23T18:08:21.593 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 66.733055107, 2024-03-23T18:08:21.593 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.001267195 2024-03-23T18:08:21.593 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:21.593 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:21.593 INFO:teuthology.orchestra.run.smithi086.stdout: "finisher-PurgeQueue": { 2024-03-23T18:08:21.593 INFO:teuthology.orchestra.run.smithi086.stdout: "queue_len": 0, 2024-03-23T18:08:21.594 INFO:teuthology.orchestra.run.smithi086.stdout: "complete_latency": { 2024-03-23T18:08:21.594 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 32126, 2024-03-23T18:08:21.594 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 5.208178803, 2024-03-23T18:08:21.594 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000162117 2024-03-23T18:08:21.594 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:21.594 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:21.594 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": { 2024-03-23T18:08:21.594 INFO:teuthology.orchestra.run.smithi086.stdout: "request": 117030, 2024-03-23T18:08:21.594 INFO:teuthology.orchestra.run.smithi086.stdout: "reply": 110107, 2024-03-23T18:08:21.595 INFO:teuthology.orchestra.run.smithi086.stdout: "reply_latency": { 2024-03-23T18:08:21.595 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 110107, 2024-03-23T18:08:21.595 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 370.994881031, 2024-03-23T18:08:21.595 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.003369403 2024-03-23T18:08:21.595 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:21.595 INFO:teuthology.orchestra.run.smithi086.stdout: "slow_reply": 0, 2024-03-23T18:08:21.595 INFO:teuthology.orchestra.run.smithi086.stdout: "forward": 6923, 2024-03-23T18:08:21.595 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_fetch_complete": 0, 2024-03-23T18:08:21.595 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_fetch_keys": 0, 2024-03-23T18:08:21.596 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_commit": 86, 2024-03-23T18:08:21.596 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_split": 8, 2024-03-23T18:08:21.596 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_merge": 11, 2024-03-23T18:08:21.596 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes": 1654, 2024-03-23T18:08:21.596 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_top": 590, 2024-03-23T18:08:21.596 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_bottom": 1019, 2024-03-23T18:08:21.596 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_pin_tail": 45, 2024-03-23T18:08:21.596 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_pinned": 811, 2024-03-23T18:08:21.596 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_expired": 402, 2024-03-23T18:08:21.597 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_with_caps": 379, 2024-03-23T18:08:21.597 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": 379, 2024-03-23T18:08:21.597 INFO:teuthology.orchestra.run.smithi086.stdout: "subtrees": 19, 2024-03-23T18:08:21.597 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse": 197264, 2024-03-23T18:08:21.597 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_hit": 115224, 2024-03-23T18:08:21.597 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_forward": 6920, 2024-03-23T18:08:21.597 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_discover": 40, 2024-03-23T18:08:21.597 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_dir_fetch": 0, 2024-03-23T18:08:21.597 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_remote_ino": 0, 2024-03-23T18:08:21.598 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_lock": 3239, 2024-03-23T18:08:21.598 INFO:teuthology.orchestra.run.smithi086.stdout: "load_cent": 172463, 2024-03-23T18:08:21.598 INFO:teuthology.orchestra.run.smithi086.stdout: "q": 0, 2024-03-23T18:08:21.598 INFO:teuthology.orchestra.run.smithi086.stdout: "exported": 8, 2024-03-23T18:08:21.598 INFO:teuthology.orchestra.run.smithi086.stdout: "exported_inodes": 287, 2024-03-23T18:08:21.598 INFO:teuthology.orchestra.run.smithi086.stdout: "imported": 6, 2024-03-23T18:08:21.598 INFO:teuthology.orchestra.run.smithi086.stdout: "imported_inodes": 428, 2024-03-23T18:08:21.598 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_dir_fetch": 0, 2024-03-23T18:08:21.599 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_backtrace_fetch": 5, 2024-03-23T18:08:21.599 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_peer_discover": 0, 2024-03-23T18:08:21.599 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rfiles": 699, 2024-03-23T18:08:21.599 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rbytes": 211642430, 2024-03-23T18:08:21.599 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rsnaps": 7, 2024-03-23T18:08:21.599 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_backtrace_fetch": 5, 2024-03-23T18:08:21.599 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_set_tag": 0, 2024-03-23T18:08:21.600 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_backtrace_repaired": 0, 2024-03-23T18:08:21.600 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_inotable_repaired": 0, 2024-03-23T18:08:21.600 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dir_inodes": 0, 2024-03-23T18:08:21.600 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dir_base_inodes": 0, 2024-03-23T18:08:21.600 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dirfrag_rstats": 0, 2024-03-23T18:08:21.600 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_file_inodes": 5, 2024-03-23T18:08:21.600 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_inode_file_caps": 8, 2024-03-23T18:08:21.600 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_revoke": 20800, 2024-03-23T18:08:21.601 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_grant": 78203, 2024-03-23T18:08:21.601 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_trunc": 0, 2024-03-23T18:08:21.601 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-23T18:08:21.601 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-23T18:08:21.601 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_caps": 47331, 2024-03-23T18:08:21.601 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_caps_dirty": 14718, 2024-03-23T18:08:21.601 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_cap_release": 7465, 2024-03-23T18:08:21.601 INFO:teuthology.orchestra.run.smithi086.stdout: "process_request_cap_release": 83826 2024-03-23T18:08:21.601 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:21.602 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_cache": { 2024-03-23T18:08:21.602 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_update": 24, 2024-03-23T18:08:21.602 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_update_receipt": 126, 2024-03-23T18:08:21.602 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_try_discover": 71, 2024-03-23T18:08:21.602 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_send_discover": 150, 2024-03-23T18:08:21.602 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_handle_discover": 50, 2024-03-23T18:08:21.602 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays": 262, 2024-03-23T18:08:21.602 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays_delayed": 0, 2024-03-23T18:08:21.602 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays_enqueuing": 0, 2024-03-23T18:08:21.603 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_created": 17879, 2024-03-23T18:08:21.603 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_enqueued": 17617, 2024-03-23T18:08:21.603 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_reintegrated": 0, 2024-03-23T18:08:21.603 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_migrated": 0, 2024-03-23T18:08:21.603 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_processing": 0, 2024-03-23T18:08:21.603 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_enqueued": 0, 2024-03-23T18:08:21.603 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_prioritized": 0, 2024-03-23T18:08:21.603 INFO:teuthology.orchestra.run.smithi086.stdout: "recovery_started": 0, 2024-03-23T18:08:21.603 INFO:teuthology.orchestra.run.smithi086.stdout: "recovery_completed": 0, 2024-03-23T18:08:21.604 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_enqueue_scrub": 0, 2024-03-23T18:08:21.604 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_exportdir": 8, 2024-03-23T18:08:21.604 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_flush": 0, 2024-03-23T18:08:21.604 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_fragmentdir": 19, 2024-03-23T18:08:21.604 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_fragstats": 0, 2024-03-23T18:08:21.604 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_inodestats": 0 2024-03-23T18:08:21.604 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:21.604 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_log": { 2024-03-23T18:08:21.604 INFO:teuthology.orchestra.run.smithi086.stdout: "evadd": 97738, 2024-03-23T18:08:21.605 INFO:teuthology.orchestra.run.smithi086.stdout: "evex": 0, 2024-03-23T18:08:21.605 INFO:teuthology.orchestra.run.smithi086.stdout: "evtrm": 0, 2024-03-23T18:08:21.605 INFO:teuthology.orchestra.run.smithi086.stdout: "ev": 97738, 2024-03-23T18:08:21.605 INFO:teuthology.orchestra.run.smithi086.stdout: "evexg": 0, 2024-03-23T18:08:21.605 INFO:teuthology.orchestra.run.smithi086.stdout: "evexd": 0, 2024-03-23T18:08:21.605 INFO:teuthology.orchestra.run.smithi086.stdout: "segadd": 128, 2024-03-23T18:08:21.605 INFO:teuthology.orchestra.run.smithi086.stdout: "segex": 0, 2024-03-23T18:08:21.605 INFO:teuthology.orchestra.run.smithi086.stdout: "segtrm": 0, 2024-03-23T18:08:21.606 INFO:teuthology.orchestra.run.smithi086.stdout: "seg": 128, 2024-03-23T18:08:21.606 INFO:teuthology.orchestra.run.smithi086.stdout: "segexg": 0, 2024-03-23T18:08:21.606 INFO:teuthology.orchestra.run.smithi086.stdout: "segexd": 0, 2024-03-23T18:08:21.606 INFO:teuthology.orchestra.run.smithi086.stdout: "expos": 4194304, 2024-03-23T18:08:21.606 INFO:teuthology.orchestra.run.smithi086.stdout: "wrpos": 378646003, 2024-03-23T18:08:21.606 INFO:teuthology.orchestra.run.smithi086.stdout: "rdpos": 0, 2024-03-23T18:08:21.606 INFO:teuthology.orchestra.run.smithi086.stdout: "jlat": { 2024-03-23T18:08:21.606 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 28918, 2024-03-23T18:08:21.607 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 194.418244574, 2024-03-23T18:08:21.607 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.006723087 2024-03-23T18:08:21.607 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:21.607 INFO:teuthology.orchestra.run.smithi086.stdout: "replayed": 0 2024-03-23T18:08:21.607 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:21.607 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_mem": { 2024-03-23T18:08:21.607 INFO:teuthology.orchestra.run.smithi086.stdout: "ino": 895, 2024-03-23T18:08:21.607 INFO:teuthology.orchestra.run.smithi086.stdout: "ino+": 18737, 2024-03-23T18:08:21.607 INFO:teuthology.orchestra.run.smithi086.stdout: "ino-": 17842, 2024-03-23T18:08:21.608 INFO:teuthology.orchestra.run.smithi086.stdout: "dir": 143, 2024-03-23T18:08:21.608 INFO:teuthology.orchestra.run.smithi086.stdout: "dir+": 287, 2024-03-23T18:08:21.608 INFO:teuthology.orchestra.run.smithi086.stdout: "dir-": 144, 2024-03-23T18:08:21.608 INFO:teuthology.orchestra.run.smithi086.stdout: "dn": 1659, 2024-03-23T18:08:21.608 INFO:teuthology.orchestra.run.smithi086.stdout: "dn+": 34117, 2024-03-23T18:08:21.608 INFO:teuthology.orchestra.run.smithi086.stdout: "dn-": 32458, 2024-03-23T18:08:21.608 INFO:teuthology.orchestra.run.smithi086.stdout: "cap": 334, 2024-03-23T18:08:21.608 INFO:teuthology.orchestra.run.smithi086.stdout: "cap+": 19592, 2024-03-23T18:08:21.608 INFO:teuthology.orchestra.run.smithi086.stdout: "cap-": 19258, 2024-03-23T18:08:21.609 INFO:teuthology.orchestra.run.smithi086.stdout: "rss": 75960, 2024-03-23T18:08:21.609 INFO:teuthology.orchestra.run.smithi086.stdout: "heap": 198964 2024-03-23T18:08:21.609 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:21.609 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_server": { 2024-03-23T18:08:21.609 INFO:teuthology.orchestra.run.smithi086.stdout: "dispatch_client_request": 168435, 2024-03-23T18:08:21.609 INFO:teuthology.orchestra.run.smithi086.stdout: "dispatch_server_request": 872, 2024-03-23T18:08:21.609 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_request": 117030, 2024-03-23T18:08:21.609 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_session": 2598, 2024-03-23T18:08:21.609 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_peer_request": 10358, 2024-03-23T18:08:21.610 INFO:teuthology.orchestra.run.smithi086.stdout: "req_create_latency": { 2024-03-23T18:08:21.610 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 17886, 2024-03-23T18:08:21.610 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 54.982006986, 2024-03-23T18:08:21.610 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.003074024 2024-03-23T18:08:21.610 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:21.610 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getattr_latency": { 2024-03-23T18:08:21.610 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 3181, 2024-03-23T18:08:21.610 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 42.076379636, 2024-03-23T18:08:21.610 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.013227406 2024-03-23T18:08:21.611 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:21.611 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getfilelock_latency": { 2024-03-23T18:08:21.611 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-23T18:08:21.611 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-23T18:08:21.611 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-23T18:08:21.611 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:21.611 INFO:teuthology.orchestra.run.smithi086.stdout: "req_link_latency": { 2024-03-23T18:08:21.611 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-23T18:08:21.611 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-23T18:08:21.612 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-23T18:08:21.612 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:21.612 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookup_latency": { 2024-03-23T18:08:21.612 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 29141, 2024-03-23T18:08:21.612 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 42.923171851, 2024-03-23T18:08:21.612 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.001472947 2024-03-23T18:08:21.612 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:21.612 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookuphash_latency": { 2024-03-23T18:08:21.613 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-23T18:08:21.613 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-23T18:08:21.613 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-23T18:08:21.613 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:21.613 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupino_latency": { 2024-03-23T18:08:21.613 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 1, 2024-03-23T18:08:21.613 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000134810, 2024-03-23T18:08:21.613 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000134810 2024-03-23T18:08:21.613 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:21.614 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupname_latency": { 2024-03-23T18:08:21.614 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-23T18:08:21.614 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-23T18:08:21.614 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-23T18:08:21.614 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:21.614 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupparent_latency": { 2024-03-23T18:08:21.614 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-23T18:08:21.614 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-23T18:08:21.614 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-23T18:08:21.615 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:21.615 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupsnap_latency": { 2024-03-23T18:08:21.615 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-23T18:08:21.615 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-23T18:08:21.615 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-23T18:08:21.615 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:21.615 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lssnap_latency": { 2024-03-23T18:08:21.615 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-23T18:08:21.616 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-23T18:08:21.616 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-23T18:08:21.616 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:21.616 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mkdir_latency": { 2024-03-23T18:08:21.616 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-23T18:08:21.616 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-23T18:08:21.616 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-23T18:08:21.616 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:21.617 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mknod_latency": { 2024-03-23T18:08:21.617 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-23T18:08:21.617 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-23T18:08:21.617 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-23T18:08:21.617 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:21.617 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mksnap_latency": { 2024-03-23T18:08:21.617 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-23T18:08:21.617 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-23T18:08:21.617 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-23T18:08:21.618 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:21.618 INFO:teuthology.orchestra.run.smithi086.stdout: "req_open_latency": { 2024-03-23T18:08:21.618 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-23T18:08:21.618 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-23T18:08:21.618 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-23T18:08:21.618 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:21.618 INFO:teuthology.orchestra.run.smithi086.stdout: "req_readdir_latency": { 2024-03-23T18:08:21.618 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 36371, 2024-03-23T18:08:21.619 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 171.047081369, 2024-03-23T18:08:21.619 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.004702842 2024-03-23T18:08:21.619 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:21.619 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rename_latency": { 2024-03-23T18:08:21.619 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 5230, 2024-03-23T18:08:21.619 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 13.470062594, 2024-03-23T18:08:21.619 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.002575537 2024-03-23T18:08:21.619 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:21.619 INFO:teuthology.orchestra.run.smithi086.stdout: "req_renamesnap_latency": { 2024-03-23T18:08:21.619 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-23T18:08:21.620 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-23T18:08:21.620 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-23T18:08:21.620 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:21.620 INFO:teuthology.orchestra.run.smithi086.stdout: "req_snapdiff_latency": { 2024-03-23T18:08:21.620 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-23T18:08:21.620 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-23T18:08:21.620 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-23T18:08:21.620 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:21.620 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmdir_latency": { 2024-03-23T18:08:21.621 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-23T18:08:21.621 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-23T18:08:21.621 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-23T18:08:21.621 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:21.621 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmsnap_latency": { 2024-03-23T18:08:21.621 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-23T18:08:21.621 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-23T18:08:21.621 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-23T18:08:21.621 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:21.622 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmxattr_latency": { 2024-03-23T18:08:21.622 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-23T18:08:21.622 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-23T18:08:21.622 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-23T18:08:21.622 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:21.622 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setattr_latency": { 2024-03-23T18:08:21.622 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 14, 2024-03-23T18:08:21.622 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.400402534, 2024-03-23T18:08:21.622 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.028600181 2024-03-23T18:08:21.623 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:21.623 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setdirlayout_latency": { 2024-03-23T18:08:21.623 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-23T18:08:21.623 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-23T18:08:21.623 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-23T18:08:21.623 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:21.623 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setfilelock_latency": { 2024-03-23T18:08:21.623 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 404, 2024-03-23T18:08:21.623 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.511358276, 2024-03-23T18:08:21.624 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.001265738 2024-03-23T18:08:21.624 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:21.624 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setlayout_latency": { 2024-03-23T18:08:21.624 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-23T18:08:21.624 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-23T18:08:21.624 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-23T18:08:21.624 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:21.624 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setxattr_latency": { 2024-03-23T18:08:21.624 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-23T18:08:21.625 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-23T18:08:21.625 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-23T18:08:21.625 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:21.625 INFO:teuthology.orchestra.run.smithi086.stdout: "req_symlink_latency": { 2024-03-23T18:08:21.625 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-23T18:08:21.625 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-23T18:08:21.625 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-23T18:08:21.625 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:21.625 INFO:teuthology.orchestra.run.smithi086.stdout: "req_unlink_latency": { 2024-03-23T18:08:21.626 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 17879, 2024-03-23T18:08:21.626 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 45.584282975, 2024-03-23T18:08:21.626 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.002549599 2024-03-23T18:08:21.626 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:21.626 INFO:teuthology.orchestra.run.smithi086.stdout: "cap_revoke_eviction": 0, 2024-03-23T18:08:21.626 INFO:teuthology.orchestra.run.smithi086.stdout: "cap_acquisition_throttle": 0, 2024-03-23T18:08:21.626 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getvxattr_latency": { 2024-03-23T18:08:21.626 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-23T18:08:21.627 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-23T18:08:21.627 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-23T18:08:21.627 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:21.627 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:21.627 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_sessions": { 2024-03-23T18:08:21.627 INFO:teuthology.orchestra.run.smithi086.stdout: "session_count": 1, 2024-03-23T18:08:21.627 INFO:teuthology.orchestra.run.smithi086.stdout: "session_add": 3, 2024-03-23T18:08:21.627 INFO:teuthology.orchestra.run.smithi086.stdout: "session_remove": 2, 2024-03-23T18:08:21.627 INFO:teuthology.orchestra.run.smithi086.stdout: "sessions_open": 1, 2024-03-23T18:08:21.628 INFO:teuthology.orchestra.run.smithi086.stdout: "sessions_stale": 0, 2024-03-23T18:08:21.628 INFO:teuthology.orchestra.run.smithi086.stdout: "total_load": 24353, 2024-03-23T18:08:21.628 INFO:teuthology.orchestra.run.smithi086.stdout: "average_load": 24353, 2024-03-23T18:08:21.628 INFO:teuthology.orchestra.run.smithi086.stdout: "avg_session_uptime": 1057, 2024-03-23T18:08:21.628 INFO:teuthology.orchestra.run.smithi086.stdout: "mdthresh_evicted": 0 2024-03-23T18:08:21.628 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:21.628 INFO:teuthology.orchestra.run.smithi086.stdout: "mempool": { 2024-03-23T18:08:21.628 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_bytes": 0, 2024-03-23T18:08:21.628 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_items": 0, 2024-03-23T18:08:21.629 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_bytes": 0, 2024-03-23T18:08:21.629 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_items": 0, 2024-03-23T18:08:21.629 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_bytes": 0, 2024-03-23T18:08:21.629 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_items": 0, 2024-03-23T18:08:21.629 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-23T18:08:21.629 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_items": 0, 2024-03-23T18:08:21.629 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-23T18:08:21.629 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_items": 0, 2024-03-23T18:08:21.629 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_bytes": 0, 2024-03-23T18:08:21.630 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_items": 0, 2024-03-23T18:08:21.630 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-23T18:08:21.630 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_items": 0, 2024-03-23T18:08:21.630 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_bytes": 0, 2024-03-23T18:08:21.630 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_items": 0, 2024-03-23T18:08:21.630 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_bytes": 0, 2024-03-23T18:08:21.630 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_items": 0, 2024-03-23T18:08:21.630 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-23T18:08:21.630 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_items": 0, 2024-03-23T18:08:21.631 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-23T18:08:21.631 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_items": 0, 2024-03-23T18:08:21.631 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_bytes": 0, 2024-03-23T18:08:21.631 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_items": 0, 2024-03-23T18:08:21.631 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_bytes": 0, 2024-03-23T18:08:21.631 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_items": 0, 2024-03-23T18:08:21.631 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-23T18:08:21.631 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_items": 0, 2024-03-23T18:08:21.632 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_bytes": 0, 2024-03-23T18:08:21.632 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_items": 0, 2024-03-23T18:08:21.632 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_bytes": 0, 2024-03-23T18:08:21.632 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_items": 0, 2024-03-23T18:08:21.632 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_bytes": 0, 2024-03-23T18:08:21.632 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_items": 0, 2024-03-23T18:08:21.632 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_bytes": 0, 2024-03-23T18:08:21.632 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_items": 0, 2024-03-23T18:08:21.632 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_bytes": 915329, 2024-03-23T18:08:21.633 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_items": 872, 2024-03-23T18:08:21.633 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_bytes": 440, 2024-03-23T18:08:21.633 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_items": 5, 2024-03-23T18:08:21.633 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_bytes": 0, 2024-03-23T18:08:21.633 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_items": 0, 2024-03-23T18:08:21.633 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_bytes": 0, 2024-03-23T18:08:21.633 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_items": 0, 2024-03-23T18:08:21.633 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_bytes": 0, 2024-03-23T18:08:21.633 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_items": 0, 2024-03-23T18:08:21.634 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_bytes": 7936, 2024-03-23T18:08:21.634 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_items": 165, 2024-03-23T18:08:21.634 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_bytes": 0, 2024-03-23T18:08:21.634 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_items": 0, 2024-03-23T18:08:21.634 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_bytes": 0, 2024-03-23T18:08:21.634 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_items": 0, 2024-03-23T18:08:21.634 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_bytes": 3963820, 2024-03-23T18:08:21.634 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_items": 69927, 2024-03-23T18:08:21.635 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_bytes": 0, 2024-03-23T18:08:21.635 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_items": 0, 2024-03-23T18:08:21.635 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_bytes": 0, 2024-03-23T18:08:21.635 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_items": 0 2024-03-23T18:08:21.635 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:21.635 INFO:teuthology.orchestra.run.smithi086.stdout: "objecter": { 2024-03-23T18:08:21.635 INFO:teuthology.orchestra.run.smithi086.stdout: "op_active": 0, 2024-03-23T18:08:21.635 INFO:teuthology.orchestra.run.smithi086.stdout: "op_laggy": 0, 2024-03-23T18:08:21.635 INFO:teuthology.orchestra.run.smithi086.stdout: "op_send": 49066, 2024-03-23T18:08:21.636 INFO:teuthology.orchestra.run.smithi086.stdout: "op_send_bytes": 377683191, 2024-03-23T18:08:21.636 INFO:teuthology.orchestra.run.smithi086.stdout: "op_resend": 27, 2024-03-23T18:08:21.636 INFO:teuthology.orchestra.run.smithi086.stdout: "op_reply": 49066, 2024-03-23T18:08:21.636 INFO:teuthology.orchestra.run.smithi086.stdout: "op_latency": { 2024-03-23T18:08:21.636 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 49066, 2024-03-23T18:08:21.636 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 331.578125785, 2024-03-23T18:08:21.636 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.006757798 2024-03-23T18:08:21.636 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:21.637 INFO:teuthology.orchestra.run.smithi086.stdout: "op_inflight": 0, 2024-03-23T18:08:21.637 INFO:teuthology.orchestra.run.smithi086.stdout: "oplen_avg": { 2024-03-23T18:08:21.637 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 49066, 2024-03-23T18:08:21.637 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 49546 2024-03-23T18:08:21.637 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:21.637 INFO:teuthology.orchestra.run.smithi086.stdout: "op": 49066, 2024-03-23T18:08:21.637 INFO:teuthology.orchestra.run.smithi086.stdout: "op_r": 810, 2024-03-23T18:08:21.637 INFO:teuthology.orchestra.run.smithi086.stdout: "op_w": 48256, 2024-03-23T18:08:21.638 INFO:teuthology.orchestra.run.smithi086.stdout: "op_rmw": 0, 2024-03-23T18:08:21.638 INFO:teuthology.orchestra.run.smithi086.stdout: "op_pg": 0, 2024-03-23T18:08:21.638 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_stat": 0, 2024-03-23T18:08:21.638 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_create": 10, 2024-03-23T18:08:21.638 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_read": 805, 2024-03-23T18:08:21.638 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_write": 29723, 2024-03-23T18:08:21.638 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_writefull": 459, 2024-03-23T18:08:21.638 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_writesame": 0, 2024-03-23T18:08:21.639 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_append": 0, 2024-03-23T18:08:21.639 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_zero": 0, 2024-03-23T18:08:21.639 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_truncate": 8, 2024-03-23T18:08:21.639 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_delete": 17771, 2024-03-23T18:08:21.639 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_mapext": 0, 2024-03-23T18:08:21.639 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_sparse_read": 0, 2024-03-23T18:08:21.639 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_clonerange": 0, 2024-03-23T18:08:21.639 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_getxattr": 5, 2024-03-23T18:08:21.639 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_setxattr": 20, 2024-03-23T18:08:21.639 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_cmpxattr": 0, 2024-03-23T18:08:21.640 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_rmxattr": 0, 2024-03-23T18:08:21.640 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_resetxattrs": 0, 2024-03-23T18:08:21.640 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_call": 0, 2024-03-23T18:08:21.640 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_watch": 0, 2024-03-23T18:08:21.640 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_notify": 0, 2024-03-23T18:08:21.640 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_src_cmpxattr": 0, 2024-03-23T18:08:21.640 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_pgls": 0, 2024-03-23T18:08:21.640 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_pgls_filter": 0, 2024-03-23T18:08:21.640 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_other": 0, 2024-03-23T18:08:21.641 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_active": 0, 2024-03-23T18:08:21.641 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_send": 0, 2024-03-23T18:08:21.641 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_resend": 0, 2024-03-23T18:08:21.641 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_ping": 0, 2024-03-23T18:08:21.641 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_active": 0, 2024-03-23T18:08:21.641 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_send": 0, 2024-03-23T18:08:21.641 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_resend": 0, 2024-03-23T18:08:21.641 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_active": 0, 2024-03-23T18:08:21.641 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_send": 0, 2024-03-23T18:08:21.642 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_resend": 0, 2024-03-23T18:08:21.642 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_active": 0, 2024-03-23T18:08:21.642 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_send": 0, 2024-03-23T18:08:21.642 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_resend": 0, 2024-03-23T18:08:21.642 INFO:teuthology.orchestra.run.smithi086.stdout: "command_active": 0, 2024-03-23T18:08:21.642 INFO:teuthology.orchestra.run.smithi086.stdout: "command_send": 0, 2024-03-23T18:08:21.642 INFO:teuthology.orchestra.run.smithi086.stdout: "command_resend": 0, 2024-03-23T18:08:21.642 INFO:teuthology.orchestra.run.smithi086.stdout: "map_epoch": 110, 2024-03-23T18:08:21.642 INFO:teuthology.orchestra.run.smithi086.stdout: "map_full": 0, 2024-03-23T18:08:21.643 INFO:teuthology.orchestra.run.smithi086.stdout: "map_inc": 30, 2024-03-23T18:08:21.643 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_sessions": 12, 2024-03-23T18:08:21.643 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_session_open": 12, 2024-03-23T18:08:21.643 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_session_close": 0, 2024-03-23T18:08:21.643 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_laggy": 0, 2024-03-23T18:08:21.643 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_wr": 550, 2024-03-23T18:08:21.643 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_rd": 0, 2024-03-23T18:08:21.643 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_del": 195 2024-03-23T18:08:21.643 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:21.644 INFO:teuthology.orchestra.run.smithi086.stdout: "oft": { 2024-03-23T18:08:21.644 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_objs": 1, 2024-03-23T18:08:21.644 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_kv_pairs": 0, 2024-03-23T18:08:21.644 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_updates": 0, 2024-03-23T18:08:21.644 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_removes": 0 2024-03-23T18:08:21.644 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:21.644 INFO:teuthology.orchestra.run.smithi086.stdout: "osd-slow-ops": { 2024-03-23T18:08:21.644 INFO:teuthology.orchestra.run.smithi086.stdout: "slow_ops_count": 117909 2024-03-23T18:08:21.644 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:21.645 INFO:teuthology.orchestra.run.smithi086.stdout: "purge_queue": { 2024-03-23T18:08:21.645 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_ops": 0, 2024-03-23T18:08:21.645 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_ops_high_water": 6, 2024-03-23T18:08:21.645 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing": 0, 2024-03-23T18:08:21.645 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_high_water": 6, 2024-03-23T18:08:21.645 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executed": 17617, 2024-03-23T18:08:21.645 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_item_in_journal": 0 2024-03-23T18:08:21.645 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:21.645 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-23T18:08:21.646 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 932, 2024-03-23T18:08:21.646 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-03-23T18:08:21.646 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-03-23T18:08:21.646 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 404075, 2024-03-23T18:08:21.646 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 98450318, 2024-03-23T18:08:21.646 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-03-23T18:08:21.646 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 404075, 2024-03-23T18:08:21.646 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-03-23T18:08:21.647 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-03-23T18:08:21.647 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 404071, 2024-03-23T18:08:21.647 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 98449386, 2024-03-23T18:08:21.647 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-03-23T18:08:21.647 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-23T18:08:21.647 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-23T18:08:21.647 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-23T18:08:21.647 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:21.647 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:21.648 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-objecter_bytes": { 2024-03-23T18:08:21.648 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-03-23T18:08:21.648 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-03-23T18:08:21.648 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-03-23T18:08:21.648 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-03-23T18:08:21.648 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-03-23T18:08:21.648 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-03-23T18:08:21.648 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-03-23T18:08:21.648 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 49066, 2024-03-23T18:08:21.649 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 379464164, 2024-03-23T18:08:21.649 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 49066, 2024-03-23T18:08:21.649 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 379464164, 2024-03-23T18:08:21.649 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-03-23T18:08:21.649 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-23T18:08:21.649 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-23T18:08:21.649 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-23T18:08:21.649 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:21.649 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:21.650 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-objecter_ops": { 2024-03-23T18:08:21.650 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-03-23T18:08:21.650 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 1024, 2024-03-23T18:08:21.650 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-03-23T18:08:21.650 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-03-23T18:08:21.650 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-03-23T18:08:21.650 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-03-23T18:08:21.650 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-03-23T18:08:21.651 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 49066, 2024-03-23T18:08:21.651 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 49066, 2024-03-23T18:08:21.651 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 49066, 2024-03-23T18:08:21.651 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 49066, 2024-03-23T18:08:21.651 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-03-23T18:08:21.651 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-23T18:08:21.651 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-23T18:08:21.651 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-23T18:08:21.651 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:21.652 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:21.652 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-write_buf_throttle": { 2024-03-23T18:08:21.652 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-03-23T18:08:21.652 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 3758096384, 2024-03-23T18:08:21.652 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-03-23T18:08:21.652 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 17617, 2024-03-23T18:08:21.652 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 1780973, 2024-03-23T18:08:21.652 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-03-23T18:08:21.652 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 17617, 2024-03-23T18:08:21.653 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-03-23T18:08:21.653 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-03-23T18:08:21.653 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 805, 2024-03-23T18:08:21.653 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 1780973, 2024-03-23T18:08:21.653 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-03-23T18:08:21.653 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-23T18:08:21.653 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-23T18:08:21.653 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-23T18:08:21.654 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:21.654 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:21.654 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-write_buf_throttle-0x557c463400a0": { 2024-03-23T18:08:21.654 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 426052, 2024-03-23T18:08:21.654 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 3758096384, 2024-03-23T18:08:21.654 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-03-23T18:08:21.654 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 97738, 2024-03-23T18:08:21.654 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 374451699, 2024-03-23T18:08:21.655 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-03-23T18:08:21.655 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 97738, 2024-03-23T18:08:21.655 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-03-23T18:08:21.655 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-03-23T18:08:21.655 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 28918, 2024-03-23T18:08:21.655 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 374025647, 2024-03-23T18:08:21.655 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-03-23T18:08:21.655 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-23T18:08:21.655 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-23T18:08:21.655 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-23T18:08:21.656 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:21.656 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:21.656 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-23T18:08:21.989 INFO:tasks.check_counter:Daemon mds.d mds.exported=8 2024-03-23T18:08:21.990 INFO:tasks.check_counter:Daemon mds.d mds.imported=6 2024-03-23T18:08:21.990 INFO:tasks.check_counter:Daemon mds.d mds.root_rsnaps=7 2024-03-23T18:08:21.990 INFO:tasks.check_counter:Daemon mds.d mds_server.req_mksnap_latency.avgcount=0 2024-03-23T18:08:21.990 INFO:tasks.check_counter:Daemon mds.d mds_server.req_rmsnap_latency.avgcount=0 2024-03-23T18:08:21.990 DEBUG:tasks.check_counter:Getting stats from g 2024-03-23T18:08:21.991 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph daemon mds.g perf dump 2024-03-23T18:08:22.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:22 smithi086 ceph-mon[38221]: pgmap v561: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 1.8 MiB/s wr, 70 op/s 2024-03-23T18:08:22.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:22 smithi092 ceph-mon[79089]: pgmap v561: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 1.8 MiB/s wr, 70 op/s 2024-03-23T18:08:22.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:22 smithi152 ceph-mon[47949]: pgmap v561: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 1.8 MiB/s wr, 70 op/s 2024-03-23T18:08:22.352 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T18:08:23.720 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-23T18:08:23.720 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-0": { 2024-03-23T18:08:23.720 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 0, 2024-03-23T18:08:23.720 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 2, 2024-03-23T18:08:23.720 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 0, 2024-03-23T18:08:23.721 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 0, 2024-03-23T18:08:23.721 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 4, 2024-03-23T18:08:23.721 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 0, 2024-03-23T18:08:23.721 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.003527313, 2024-03-23T18:08:23.721 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.000000000, 2024-03-23T18:08:23.721 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.000759830, 2024-03-23T18:08:23.721 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-23T18:08:23.721 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T18:08:23.722 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-23T18:08:23.722 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-23T18:08:23.722 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-23T18:08:23.722 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:23.722 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-03-23T18:08:23.722 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-23T18:08:23.722 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-23T18:08:23.722 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-23T18:08:23.722 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:23.723 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-23T18:08:23.723 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 0 2024-03-23T18:08:23.723 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:23.723 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-1": { 2024-03-23T18:08:23.723 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 325, 2024-03-23T18:08:23.723 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 314, 2024-03-23T18:08:23.723 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 1588464, 2024-03-23T18:08:23.723 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 179664, 2024-03-23T18:08:23.724 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 2, 2024-03-23T18:08:23.724 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 1, 2024-03-23T18:08:23.724 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.066154212, 2024-03-23T18:08:23.724 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.026613909, 2024-03-23T18:08:23.724 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.041282698, 2024-03-23T18:08:23.724 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.007133421, 2024-03-23T18:08:23.724 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T18:08:23.724 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 313, 2024-03-23T18:08:23.724 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.021125360, 2024-03-23T18:08:23.725 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000067493 2024-03-23T18:08:23.725 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:23.725 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-03-23T18:08:23.725 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-23T18:08:23.725 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-23T18:08:23.725 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-23T18:08:23.726 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:23.726 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 1588464, 2024-03-23T18:08:23.726 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 179664 2024-03-23T18:08:23.726 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:23.726 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-2": { 2024-03-23T18:08:23.726 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 6, 2024-03-23T18:08:23.726 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 241, 2024-03-23T18:08:23.727 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 3264, 2024-03-23T18:08:23.727 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 194592, 2024-03-23T18:08:23.727 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 5, 2024-03-23T18:08:23.727 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 3, 2024-03-23T18:08:23.727 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.029359564, 2024-03-23T18:08:23.727 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.018141095, 2024-03-23T18:08:23.727 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.004314897, 2024-03-23T18:08:23.727 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-23T18:08:23.727 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T18:08:23.728 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 239, 2024-03-23T18:08:23.728 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.030409730, 2024-03-23T18:08:23.728 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000127237 2024-03-23T18:08:23.728 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:23.728 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-03-23T18:08:23.728 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-23T18:08:23.728 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-23T18:08:23.728 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-23T18:08:23.728 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:23.729 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-03-23T18:08:23.729 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 194592 2024-03-23T18:08:23.729 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:23.729 INFO:teuthology.orchestra.run.smithi086.stdout: "cct": { 2024-03-23T18:08:23.729 INFO:teuthology.orchestra.run.smithi086.stdout: "total_workers": 0, 2024-03-23T18:08:23.729 INFO:teuthology.orchestra.run.smithi086.stdout: "unhealthy_workers": 0 2024-03-23T18:08:23.729 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:23.729 INFO:teuthology.orchestra.run.smithi086.stdout: "mempool": { 2024-03-23T18:08:23.729 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_bytes": 0, 2024-03-23T18:08:23.730 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_items": 0, 2024-03-23T18:08:23.730 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_bytes": 0, 2024-03-23T18:08:23.730 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_items": 0, 2024-03-23T18:08:23.730 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_bytes": 0, 2024-03-23T18:08:23.730 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_items": 0, 2024-03-23T18:08:23.730 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-23T18:08:23.730 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_items": 0, 2024-03-23T18:08:23.730 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-23T18:08:23.731 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_items": 0, 2024-03-23T18:08:23.731 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_bytes": 0, 2024-03-23T18:08:23.731 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_items": 0, 2024-03-23T18:08:23.731 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-23T18:08:23.731 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_items": 0, 2024-03-23T18:08:23.731 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_bytes": 0, 2024-03-23T18:08:23.731 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_items": 0, 2024-03-23T18:08:23.731 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_bytes": 0, 2024-03-23T18:08:23.731 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_items": 0, 2024-03-23T18:08:23.732 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-23T18:08:23.732 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_items": 0, 2024-03-23T18:08:23.732 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-23T18:08:23.732 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_items": 0, 2024-03-23T18:08:23.732 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_bytes": 0, 2024-03-23T18:08:23.732 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_items": 0, 2024-03-23T18:08:23.732 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_bytes": 0, 2024-03-23T18:08:23.732 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_items": 0, 2024-03-23T18:08:23.733 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-23T18:08:23.733 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_items": 0, 2024-03-23T18:08:23.733 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_bytes": 0, 2024-03-23T18:08:23.733 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_items": 0, 2024-03-23T18:08:23.733 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_bytes": 0, 2024-03-23T18:08:23.733 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_items": 0, 2024-03-23T18:08:23.733 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_bytes": 0, 2024-03-23T18:08:23.733 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_items": 0, 2024-03-23T18:08:23.733 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_bytes": 0, 2024-03-23T18:08:23.734 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_items": 0, 2024-03-23T18:08:23.734 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_bytes": 148977, 2024-03-23T18:08:23.734 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_items": 30, 2024-03-23T18:08:23.734 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_bytes": 0, 2024-03-23T18:08:23.734 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_items": 0, 2024-03-23T18:08:23.734 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_bytes": 0, 2024-03-23T18:08:23.734 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_items": 0, 2024-03-23T18:08:23.734 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_bytes": 0, 2024-03-23T18:08:23.735 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_items": 0, 2024-03-23T18:08:23.735 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_bytes": 0, 2024-03-23T18:08:23.735 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_items": 0, 2024-03-23T18:08:23.735 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_bytes": 0, 2024-03-23T18:08:23.735 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_items": 0, 2024-03-23T18:08:23.735 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_bytes": 0, 2024-03-23T18:08:23.735 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_items": 0, 2024-03-23T18:08:23.736 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_bytes": 0, 2024-03-23T18:08:23.736 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_items": 0, 2024-03-23T18:08:23.736 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_bytes": 928, 2024-03-23T18:08:23.736 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_items": 2, 2024-03-23T18:08:23.736 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_bytes": 0, 2024-03-23T18:08:23.737 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_items": 0, 2024-03-23T18:08:23.737 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_bytes": 0, 2024-03-23T18:08:23.737 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_items": 0 2024-03-23T18:08:23.737 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:23.737 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-23T18:08:23.737 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-03-23T18:08:23.737 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-03-23T18:08:23.737 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-03-23T18:08:23.737 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 327, 2024-03-23T18:08:23.738 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 1542584, 2024-03-23T18:08:23.738 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-03-23T18:08:23.738 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 327, 2024-03-23T18:08:23.738 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-03-23T18:08:23.738 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-03-23T18:08:23.738 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 327, 2024-03-23T18:08:23.738 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 1542584, 2024-03-23T18:08:23.738 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-03-23T18:08:23.739 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-23T18:08:23.739 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-23T18:08:23.739 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-23T18:08:23.739 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:23.739 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:23.739 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-23T18:08:24.167 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-03-23T18:08:24.167 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-03-23T18:08:24.167 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-03-23T18:08:24.167 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-03-23T18:08:24.167 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-03-23T18:08:24.167 DEBUG:tasks.check_counter:Getting stats from j 2024-03-23T18:08:24.168 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph daemon mds.j perf dump 2024-03-23T18:08:24.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:24 smithi086 ceph-mon[38221]: pgmap v562: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 2.0 MiB/s wr, 63 op/s 2024-03-23T18:08:24.311 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:24 smithi092 ceph-mon[79089]: pgmap v562: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 2.0 MiB/s wr, 63 op/s 2024-03-23T18:08:24.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:24 smithi152 ceph-mon[47949]: pgmap v562: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 2.0 MiB/s wr, 63 op/s 2024-03-23T18:08:24.542 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T18:08:26.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:26 smithi086 ceph-mon[38221]: pgmap v563: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 4.2 MiB/s wr, 110 op/s 2024-03-23T18:08:26.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:26 smithi092 ceph-mon[79089]: pgmap v563: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 4.2 MiB/s wr, 110 op/s 2024-03-23T18:08:26.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:26 smithi152 ceph-mon[47949]: pgmap v563: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 4.2 MiB/s wr, 110 op/s 2024-03-23T18:08:26.388 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-03-23T18:08:26.388 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-0": { 2024-03-23T18:08:26.388 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 0, 2024-03-23T18:08:26.388 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 2, 2024-03-23T18:08:26.389 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 0, 2024-03-23T18:08:26.389 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 0, 2024-03-23T18:08:26.389 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 4, 2024-03-23T18:08:26.389 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 0, 2024-03-23T18:08:26.389 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.004339085, 2024-03-23T18:08:26.389 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.000000000, 2024-03-23T18:08:26.389 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.000851833, 2024-03-23T18:08:26.390 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-23T18:08:26.390 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T18:08:26.390 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-23T18:08:26.390 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-23T18:08:26.390 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-23T18:08:26.390 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:26.391 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-03-23T18:08:26.391 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-23T18:08:26.391 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-23T18:08:26.391 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-23T18:08:26.391 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:26.391 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-23T18:08:26.392 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 0 2024-03-23T18:08:26.392 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:26.392 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-1": { 2024-03-23T18:08:26.392 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 329, 2024-03-23T18:08:26.392 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 318, 2024-03-23T18:08:26.392 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 1589552, 2024-03-23T18:08:26.392 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 182032, 2024-03-23T18:08:26.393 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 2, 2024-03-23T18:08:26.393 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 1, 2024-03-23T18:08:26.393 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.070011283, 2024-03-23T18:08:26.393 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.027375496, 2024-03-23T18:08:26.393 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.045893783, 2024-03-23T18:08:26.394 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.007103350, 2024-03-23T18:08:26.394 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T18:08:26.394 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 317, 2024-03-23T18:08:26.394 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.024384729, 2024-03-23T18:08:26.394 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000076923 2024-03-23T18:08:26.394 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:26.395 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-03-23T18:08:26.395 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-23T18:08:26.395 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-23T18:08:26.395 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-23T18:08:26.395 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:26.395 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 1589552, 2024-03-23T18:08:26.396 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 182032 2024-03-23T18:08:26.396 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:26.396 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-2": { 2024-03-23T18:08:26.396 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 7, 2024-03-23T18:08:26.396 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 246, 2024-03-23T18:08:26.397 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 3424, 2024-03-23T18:08:26.397 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 271776, 2024-03-23T18:08:26.397 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 6, 2024-03-23T18:08:26.397 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 3, 2024-03-23T18:08:26.397 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.029434292, 2024-03-23T18:08:26.397 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.018411791, 2024-03-23T18:08:26.397 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.006869655, 2024-03-23T18:08:26.397 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-23T18:08:26.398 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T18:08:26.398 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 244, 2024-03-23T18:08:26.398 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.067097986, 2024-03-23T18:08:26.398 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000274991 2024-03-23T18:08:26.398 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:26.398 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-03-23T18:08:26.398 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-23T18:08:26.399 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-23T18:08:26.399 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-23T18:08:26.399 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:26.399 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 3424, 2024-03-23T18:08:26.399 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 271776 2024-03-23T18:08:26.399 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:26.399 INFO:teuthology.orchestra.run.smithi086.stdout: "cct": { 2024-03-23T18:08:26.400 INFO:teuthology.orchestra.run.smithi086.stdout: "total_workers": 0, 2024-03-23T18:08:26.400 INFO:teuthology.orchestra.run.smithi086.stdout: "unhealthy_workers": 0 2024-03-23T18:08:26.400 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:26.400 INFO:teuthology.orchestra.run.smithi086.stdout: "mempool": { 2024-03-23T18:08:26.400 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_bytes": 0, 2024-03-23T18:08:26.400 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_items": 0, 2024-03-23T18:08:26.401 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_bytes": 0, 2024-03-23T18:08:26.401 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_items": 0, 2024-03-23T18:08:26.401 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_bytes": 0, 2024-03-23T18:08:26.401 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_items": 0, 2024-03-23T18:08:26.401 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-23T18:08:26.401 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_items": 0, 2024-03-23T18:08:26.402 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-23T18:08:26.402 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_items": 0, 2024-03-23T18:08:26.402 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_bytes": 0, 2024-03-23T18:08:26.402 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_items": 0, 2024-03-23T18:08:26.403 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-23T18:08:26.403 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_items": 0, 2024-03-23T18:08:26.403 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_bytes": 0, 2024-03-23T18:08:26.403 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_items": 0, 2024-03-23T18:08:26.403 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_bytes": 0, 2024-03-23T18:08:26.403 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_items": 0, 2024-03-23T18:08:26.403 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-23T18:08:26.403 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_items": 0, 2024-03-23T18:08:26.404 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-23T18:08:26.404 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_items": 0, 2024-03-23T18:08:26.404 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_bytes": 0, 2024-03-23T18:08:26.404 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_items": 0, 2024-03-23T18:08:26.405 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_bytes": 0, 2024-03-23T18:08:26.405 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_items": 0, 2024-03-23T18:08:26.405 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-23T18:08:26.405 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_items": 0, 2024-03-23T18:08:26.405 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_bytes": 0, 2024-03-23T18:08:26.405 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_items": 0, 2024-03-23T18:08:26.405 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_bytes": 0, 2024-03-23T18:08:26.405 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_items": 0, 2024-03-23T18:08:26.406 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_bytes": 0, 2024-03-23T18:08:26.406 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_items": 0, 2024-03-23T18:08:26.406 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_bytes": 0, 2024-03-23T18:08:26.406 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_items": 0, 2024-03-23T18:08:26.406 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_bytes": 150776, 2024-03-23T18:08:26.407 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_items": 32, 2024-03-23T18:08:26.407 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_bytes": 0, 2024-03-23T18:08:26.407 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_items": 0, 2024-03-23T18:08:26.407 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_bytes": 0, 2024-03-23T18:08:26.407 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_items": 0, 2024-03-23T18:08:26.407 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_bytes": 0, 2024-03-23T18:08:26.408 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_items": 0, 2024-03-23T18:08:26.408 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_bytes": 0, 2024-03-23T18:08:26.408 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_items": 0, 2024-03-23T18:08:26.408 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_bytes": 0, 2024-03-23T18:08:26.408 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_items": 0, 2024-03-23T18:08:26.408 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_bytes": 0, 2024-03-23T18:08:26.409 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_items": 0, 2024-03-23T18:08:26.409 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_bytes": 0, 2024-03-23T18:08:26.409 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_items": 0, 2024-03-23T18:08:26.409 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_bytes": 928, 2024-03-23T18:08:26.409 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_items": 2, 2024-03-23T18:08:26.410 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_bytes": 0, 2024-03-23T18:08:26.410 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_items": 0, 2024-03-23T18:08:26.410 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_bytes": 0, 2024-03-23T18:08:26.410 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_items": 0 2024-03-23T18:08:26.410 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-03-23T18:08:26.410 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-23T18:08:26.410 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-03-23T18:08:26.411 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-03-23T18:08:26.411 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-03-23T18:08:26.411 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 332, 2024-03-23T18:08:26.411 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 1543131, 2024-03-23T18:08:26.411 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-03-23T18:08:26.411 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 332, 2024-03-23T18:08:26.411 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-03-23T18:08:26.412 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-03-23T18:08:26.412 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 332, 2024-03-23T18:08:26.412 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 1543131, 2024-03-23T18:08:26.412 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-03-23T18:08:26.412 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-03-23T18:08:26.412 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-03-23T18:08:26.412 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-03-23T18:08:26.413 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:26.413 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-03-23T18:08:26.413 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-03-23T18:08:26.771 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-03-23T18:08:26.771 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-03-23T18:08:26.771 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-03-23T18:08:26.771 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-03-23T18:08:26.771 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-03-23T18:08:26.771 DEBUG:tasks.check_counter:Getting stats from b 2024-03-23T18:08:26.772 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph daemon mds.b perf dump 2024-03-23T18:08:27.810 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.b/config 2024-03-23T18:08:28.262 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:28 smithi092 ceph-mon[79089]: pgmap v564: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 738 KiB/s rd, 7.2 MiB/s wr, 198 op/s 2024-03-23T18:08:28.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:28 smithi152 ceph-mon[47949]: pgmap v564: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 738 KiB/s rd, 7.2 MiB/s wr, 198 op/s 2024-03-23T18:08:28.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:28 smithi086 ceph-mon[38221]: pgmap v564: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 738 KiB/s rd, 7.2 MiB/s wr, 198 op/s 2024-03-23T18:08:28.964 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-03-23T18:08:28.964 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-0": { 2024-03-23T18:08:28.964 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 431304, 2024-03-23T18:08:28.965 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 543269, 2024-03-23T18:08:28.965 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 131218074, 2024-03-23T18:08:28.965 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 679841079, 2024-03-23T18:08:28.965 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 21, 2024-03-23T18:08:28.965 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 5, 2024-03-23T18:08:28.965 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 23.757983688, 2024-03-23T18:08:28.965 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 9.937619970, 2024-03-23T18:08:28.965 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 15.067569937, 2024-03-23T18:08:28.966 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 1.616859567, 2024-03-23T18:08:28.966 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T18:08:28.966 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 543267, 2024-03-23T18:08:28.966 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 86.767388516, 2024-03-23T18:08:28.966 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000159714 2024-03-23T18:08:28.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:28.966 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-03-23T18:08:28.966 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 749261, 2024-03-23T18:08:28.966 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.130081799, 2024-03-23T18:08:28.967 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000173 2024-03-23T18:08:28.967 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:28.967 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 558992, 2024-03-23T18:08:28.967 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 179648 2024-03-23T18:08:28.967 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:28.967 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-1": { 2024-03-23T18:08:28.967 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 108049, 2024-03-23T18:08:28.967 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 123185, 2024-03-23T18:08:28.968 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 33057895, 2024-03-23T18:08:28.968 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 254733065, 2024-03-23T18:08:28.968 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 197, 2024-03-23T18:08:28.968 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 5, 2024-03-23T18:08:28.968 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 7.390354117, 2024-03-23T18:08:28.968 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 3.052259629, 2024-03-23T18:08:28.968 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 3.673910665, 2024-03-23T18:08:28.968 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 1.094401677, 2024-03-23T18:08:28.969 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T18:08:28.969 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 123185, 2024-03-23T18:08:28.969 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 37.509483569, 2024-03-23T18:08:28.969 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000304497 2024-03-23T18:08:28.969 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:28.969 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-03-23T18:08:28.969 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 156421, 2024-03-23T18:08:28.969 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.027333853, 2024-03-23T18:08:28.969 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000174 2024-03-23T18:08:28.970 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:28.970 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 103312, 2024-03-23T18:08:28.970 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 617248 2024-03-23T18:08:28.970 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:28.970 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-2": { 2024-03-23T18:08:28.970 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 221351, 2024-03-23T18:08:28.970 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 292420, 2024-03-23T18:08:28.970 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 56570970, 2024-03-23T18:08:28.971 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 193025391, 2024-03-23T18:08:28.971 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 9, 2024-03-23T18:08:28.971 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 8, 2024-03-23T18:08:28.971 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 13.567770065, 2024-03-23T18:08:28.971 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 5.747456407, 2024-03-23T18:08:28.971 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 8.592096659, 2024-03-23T18:08:28.971 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.642347025, 2024-03-23T18:08:28.971 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T18:08:28.971 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 292417, 2024-03-23T18:08:28.972 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 34.386434610, 2024-03-23T18:08:28.972 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000117593 2024-03-23T18:08:28.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:28.972 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-03-23T18:08:28.972 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 467772, 2024-03-23T18:08:28.972 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.072241506, 2024-03-23T18:08:28.972 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000154 2024-03-23T18:08:28.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:28.973 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 2043008, 2024-03-23T18:08:28.973 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 203744 2024-03-23T18:08:28.973 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:28.973 INFO:teuthology.orchestra.run.smithi092.stdout: "cct": { 2024-03-23T18:08:28.973 INFO:teuthology.orchestra.run.smithi092.stdout: "total_workers": 1, 2024-03-23T18:08:28.973 INFO:teuthology.orchestra.run.smithi092.stdout: "unhealthy_workers": 0 2024-03-23T18:08:28.973 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:28.973 INFO:teuthology.orchestra.run.smithi092.stdout: "finisher-MDSRank": { 2024-03-23T18:08:28.973 INFO:teuthology.orchestra.run.smithi092.stdout: "queue_len": 0, 2024-03-23T18:08:28.974 INFO:teuthology.orchestra.run.smithi092.stdout: "complete_latency": { 2024-03-23T18:08:28.974 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 119810, 2024-03-23T18:08:28.974 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 99.767220553, 2024-03-23T18:08:28.974 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000832711 2024-03-23T18:08:28.974 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-23T18:08:28.974 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:28.974 INFO:teuthology.orchestra.run.smithi092.stdout: "finisher-PurgeQueue": { 2024-03-23T18:08:28.974 INFO:teuthology.orchestra.run.smithi092.stdout: "queue_len": 0, 2024-03-23T18:08:28.975 INFO:teuthology.orchestra.run.smithi092.stdout: "complete_latency": { 2024-03-23T18:08:28.975 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 35473, 2024-03-23T18:08:28.975 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 7.164167926, 2024-03-23T18:08:28.975 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000201961 2024-03-23T18:08:28.975 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-23T18:08:28.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:28.975 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": { 2024-03-23T18:08:28.975 INFO:teuthology.orchestra.run.smithi092.stdout: "request": 191286, 2024-03-23T18:08:28.976 INFO:teuthology.orchestra.run.smithi092.stdout: "reply": 167833, 2024-03-23T18:08:28.976 INFO:teuthology.orchestra.run.smithi092.stdout: "reply_latency": { 2024-03-23T18:08:28.976 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 167833, 2024-03-23T18:08:28.976 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 1089.515642659, 2024-03-23T18:08:28.976 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.006491665 2024-03-23T18:08:28.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:28.976 INFO:teuthology.orchestra.run.smithi092.stdout: "slow_reply": 0, 2024-03-23T18:08:28.976 INFO:teuthology.orchestra.run.smithi092.stdout: "forward": 23453, 2024-03-23T18:08:28.976 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_fetch_complete": 30, 2024-03-23T18:08:28.977 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_fetch_keys": 2, 2024-03-23T18:08:28.977 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_commit": 2490, 2024-03-23T18:08:28.977 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_split": 12, 2024-03-23T18:08:28.977 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_merge": 11, 2024-03-23T18:08:28.977 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes": 1576, 2024-03-23T18:08:28.977 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_top": 639, 2024-03-23T18:08:28.977 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_bottom": 824, 2024-03-23T18:08:28.977 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_pin_tail": 113, 2024-03-23T18:08:28.977 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_pinned": 664, 2024-03-23T18:08:28.978 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_expired": 2989, 2024-03-23T18:08:28.978 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_with_caps": 276, 2024-03-23T18:08:28.978 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": 276, 2024-03-23T18:08:28.978 INFO:teuthology.orchestra.run.smithi092.stdout: "subtrees": 21, 2024-03-23T18:08:28.978 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse": 281355, 2024-03-23T18:08:28.978 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_hit": 194675, 2024-03-23T18:08:28.978 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_forward": 23434, 2024-03-23T18:08:28.978 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_discover": 118, 2024-03-23T18:08:28.978 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_dir_fetch": 4, 2024-03-23T18:08:28.979 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_remote_ino": 0, 2024-03-23T18:08:28.979 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_lock": 1259, 2024-03-23T18:08:28.979 INFO:teuthology.orchestra.run.smithi092.stdout: "load_cent": 208357, 2024-03-23T18:08:28.979 INFO:teuthology.orchestra.run.smithi092.stdout: "q": 39, 2024-03-23T18:08:28.979 INFO:teuthology.orchestra.run.smithi092.stdout: "exported": 36, 2024-03-23T18:08:28.979 INFO:teuthology.orchestra.run.smithi092.stdout: "exported_inodes": 1960, 2024-03-23T18:08:28.979 INFO:teuthology.orchestra.run.smithi092.stdout: "imported": 19, 2024-03-23T18:08:28.979 INFO:teuthology.orchestra.run.smithi092.stdout: "imported_inodes": 761, 2024-03-23T18:08:28.980 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_dir_fetch": 0, 2024-03-23T18:08:28.980 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_backtrace_fetch": 171, 2024-03-23T18:08:28.980 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_peer_discover": 0, 2024-03-23T18:08:28.980 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rfiles": 719, 2024-03-23T18:08:28.980 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rbytes": 211898458, 2024-03-23T18:08:28.980 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rsnaps": 7, 2024-03-23T18:08:28.980 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_backtrace_fetch": 171, 2024-03-23T18:08:28.980 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_set_tag": 0, 2024-03-23T18:08:28.980 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_backtrace_repaired": 0, 2024-03-23T18:08:28.981 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_inotable_repaired": 0, 2024-03-23T18:08:28.981 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dir_inodes": 57, 2024-03-23T18:08:28.981 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dir_base_inodes": 4, 2024-03-23T18:08:28.981 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dirfrag_rstats": 53, 2024-03-23T18:08:28.981 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_file_inodes": 114, 2024-03-23T18:08:28.981 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_inode_file_caps": 21, 2024-03-23T18:08:28.981 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_revoke": 56765, 2024-03-23T18:08:28.981 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_grant": 79611, 2024-03-23T18:08:28.982 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_trunc": 0, 2024-03-23T18:08:28.982 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-23T18:08:28.982 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-23T18:08:28.982 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_caps": 87769, 2024-03-23T18:08:28.982 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_caps_dirty": 18725, 2024-03-23T18:08:28.982 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_cap_release": 6581, 2024-03-23T18:08:28.982 INFO:teuthology.orchestra.run.smithi092.stdout: "process_request_cap_release": 99593 2024-03-23T18:08:28.982 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:28.982 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_cache": { 2024-03-23T18:08:28.983 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_update": 43, 2024-03-23T18:08:28.983 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_update_receipt": 21, 2024-03-23T18:08:28.983 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_try_discover": 14, 2024-03-23T18:08:28.983 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_send_discover": 125, 2024-03-23T18:08:28.983 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_handle_discover": 363, 2024-03-23T18:08:28.983 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays": 0, 2024-03-23T18:08:28.983 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays_delayed": 0, 2024-03-23T18:08:28.983 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays_enqueuing": 0, 2024-03-23T18:08:28.983 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_created": 19853, 2024-03-23T18:08:28.984 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_enqueued": 19853, 2024-03-23T18:08:28.984 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_reintegrated": 0, 2024-03-23T18:08:28.984 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_migrated": 0, 2024-03-23T18:08:28.984 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_processing": 0, 2024-03-23T18:08:28.984 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_enqueued": 0, 2024-03-23T18:08:28.984 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_prioritized": 0, 2024-03-23T18:08:28.984 INFO:teuthology.orchestra.run.smithi092.stdout: "recovery_started": 0, 2024-03-23T18:08:28.984 INFO:teuthology.orchestra.run.smithi092.stdout: "recovery_completed": 0, 2024-03-23T18:08:28.985 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_enqueue_scrub": 4, 2024-03-23T18:08:28.985 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_exportdir": 37, 2024-03-23T18:08:28.985 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_flush": 0, 2024-03-23T18:08:28.985 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_fragmentdir": 23, 2024-03-23T18:08:28.985 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_fragstats": 0, 2024-03-23T18:08:28.985 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_inodestats": 0 2024-03-23T18:08:28.985 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:28.985 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_log": { 2024-03-23T18:08:28.986 INFO:teuthology.orchestra.run.smithi092.stdout: "evadd": 132507, 2024-03-23T18:08:28.986 INFO:teuthology.orchestra.run.smithi092.stdout: "evex": 131651, 2024-03-23T18:08:28.986 INFO:teuthology.orchestra.run.smithi092.stdout: "evtrm": 131651, 2024-03-23T18:08:28.986 INFO:teuthology.orchestra.run.smithi092.stdout: "ev": 856, 2024-03-23T18:08:28.986 INFO:teuthology.orchestra.run.smithi092.stdout: "evexg": 0, 2024-03-23T18:08:28.986 INFO:teuthology.orchestra.run.smithi092.stdout: "evexd": 2151, 2024-03-23T18:08:28.986 INFO:teuthology.orchestra.run.smithi092.stdout: "segadd": 272, 2024-03-23T18:08:28.986 INFO:teuthology.orchestra.run.smithi092.stdout: "segex": 270, 2024-03-23T18:08:28.987 INFO:teuthology.orchestra.run.smithi092.stdout: "segtrm": 270, 2024-03-23T18:08:28.987 INFO:teuthology.orchestra.run.smithi092.stdout: "seg": 5, 2024-03-23T18:08:28.987 INFO:teuthology.orchestra.run.smithi092.stdout: "segexg": 0, 2024-03-23T18:08:28.987 INFO:teuthology.orchestra.run.smithi092.stdout: "segexd": 3, 2024-03-23T18:08:28.987 INFO:teuthology.orchestra.run.smithi092.stdout: "expos": 624670748, 2024-03-23T18:08:28.987 INFO:teuthology.orchestra.run.smithi092.stdout: "wrpos": 627856550, 2024-03-23T18:08:28.987 INFO:teuthology.orchestra.run.smithi092.stdout: "rdpos": 0, 2024-03-23T18:08:28.987 INFO:teuthology.orchestra.run.smithi092.stdout: "jlat": { 2024-03-23T18:08:28.987 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 63803, 2024-03-23T18:08:28.988 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 340.471268473, 2024-03-23T18:08:28.988 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.005336289 2024-03-23T18:08:28.988 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:28.988 INFO:teuthology.orchestra.run.smithi092.stdout: "replayed": 0 2024-03-23T18:08:28.988 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:28.988 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_mem": { 2024-03-23T18:08:28.988 INFO:teuthology.orchestra.run.smithi092.stdout: "ino": 984, 2024-03-23T18:08:28.989 INFO:teuthology.orchestra.run.smithi092.stdout: "ino+": 22556, 2024-03-23T18:08:28.989 INFO:teuthology.orchestra.run.smithi092.stdout: "ino-": 21572, 2024-03-23T18:08:28.989 INFO:teuthology.orchestra.run.smithi092.stdout: "dir": 153, 2024-03-23T18:08:28.989 INFO:teuthology.orchestra.run.smithi092.stdout: "dir+": 338, 2024-03-23T18:08:28.989 INFO:teuthology.orchestra.run.smithi092.stdout: "dir-": 185, 2024-03-23T18:08:28.989 INFO:teuthology.orchestra.run.smithi092.stdout: "dn": 1581, 2024-03-23T18:08:28.989 INFO:teuthology.orchestra.run.smithi092.stdout: "dn+": 42474, 2024-03-23T18:08:28.989 INFO:teuthology.orchestra.run.smithi092.stdout: "dn-": 40893, 2024-03-23T18:08:28.990 INFO:teuthology.orchestra.run.smithi092.stdout: "cap": 163, 2024-03-23T18:08:28.990 INFO:teuthology.orchestra.run.smithi092.stdout: "cap+": 22036, 2024-03-23T18:08:28.990 INFO:teuthology.orchestra.run.smithi092.stdout: "cap-": 21873, 2024-03-23T18:08:28.990 INFO:teuthology.orchestra.run.smithi092.stdout: "rss": 89784, 2024-03-23T18:08:28.990 INFO:teuthology.orchestra.run.smithi092.stdout: "heap": 207156 2024-03-23T18:08:28.990 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:28.990 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_server": { 2024-03-23T18:08:28.991 INFO:teuthology.orchestra.run.smithi092.stdout: "dispatch_client_request": 271824, 2024-03-23T18:08:28.991 INFO:teuthology.orchestra.run.smithi092.stdout: "dispatch_server_request": 32894, 2024-03-23T18:08:28.991 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_request": 191286, 2024-03-23T18:08:28.991 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_session": 2882, 2024-03-23T18:08:28.991 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_peer_request": 65716, 2024-03-23T18:08:28.991 INFO:teuthology.orchestra.run.smithi092.stdout: "req_create_latency": { 2024-03-23T18:08:28.991 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 20580, 2024-03-23T18:08:28.991 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 107.605654306, 2024-03-23T18:08:28.991 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.005228651 2024-03-23T18:08:28.992 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:28.992 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getattr_latency": { 2024-03-23T18:08:28.992 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 41159, 2024-03-23T18:08:28.992 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 272.805886335, 2024-03-23T18:08:28.992 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.006628098 2024-03-23T18:08:28.992 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:28.992 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getfilelock_latency": { 2024-03-23T18:08:28.992 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:28.993 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:28.993 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:28.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:28.993 INFO:teuthology.orchestra.run.smithi092.stdout: "req_link_latency": { 2024-03-23T18:08:28.993 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:28.993 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:28.993 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:28.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:28.994 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookup_latency": { 2024-03-23T18:08:28.994 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 26749, 2024-03-23T18:08:28.994 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 262.961886892, 2024-03-23T18:08:28.994 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.009830718 2024-03-23T18:08:28.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:28.994 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookuphash_latency": { 2024-03-23T18:08:28.995 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:28.995 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:28.995 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:28.995 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:28.995 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupino_latency": { 2024-03-23T18:08:28.995 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 1, 2024-03-23T18:08:28.995 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000276135, 2024-03-23T18:08:28.995 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000276135 2024-03-23T18:08:28.995 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:28.996 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupname_latency": { 2024-03-23T18:08:28.996 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:28.996 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:28.996 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:28.996 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:28.996 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupparent_latency": { 2024-03-23T18:08:28.996 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:28.996 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:28.997 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:28.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:28.997 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupsnap_latency": { 2024-03-23T18:08:28.997 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:28.997 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:28.997 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:28.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:28.997 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lssnap_latency": { 2024-03-23T18:08:28.997 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 18, 2024-03-23T18:08:28.998 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.010734738, 2024-03-23T18:08:28.998 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000596374 2024-03-23T18:08:28.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:28.998 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mkdir_latency": { 2024-03-23T18:08:28.998 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 119, 2024-03-23T18:08:28.998 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.677812029, 2024-03-23T18:08:28.998 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.005695899 2024-03-23T18:08:28.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:28.999 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mknod_latency": { 2024-03-23T18:08:28.999 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:28.999 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:28.999 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:28.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:28.999 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mksnap_latency": { 2024-03-23T18:08:28.999 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 18, 2024-03-23T18:08:28.999 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 7.452124436, 2024-03-23T18:08:28.999 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.414006913 2024-03-23T18:08:29.000 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:29.000 INFO:teuthology.orchestra.run.smithi092.stdout: "req_open_latency": { 2024-03-23T18:08:29.000 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 9, 2024-03-23T18:08:29.000 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.015088035, 2024-03-23T18:08:29.000 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.001676448 2024-03-23T18:08:29.000 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:29.000 INFO:teuthology.orchestra.run.smithi092.stdout: "req_readdir_latency": { 2024-03-23T18:08:29.000 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 55111, 2024-03-23T18:08:29.000 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 264.831509066, 2024-03-23T18:08:29.001 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.004805420 2024-03-23T18:08:29.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:29.001 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rename_latency": { 2024-03-23T18:08:29.001 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 3611, 2024-03-23T18:08:29.001 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 63.034649717, 2024-03-23T18:08:29.001 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.017456286 2024-03-23T18:08:29.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:29.001 INFO:teuthology.orchestra.run.smithi092.stdout: "req_renamesnap_latency": { 2024-03-23T18:08:29.001 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:29.002 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:29.002 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:29.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:29.002 INFO:teuthology.orchestra.run.smithi092.stdout: "req_snapdiff_latency": { 2024-03-23T18:08:29.002 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:29.002 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:29.002 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:29.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:29.003 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmdir_latency": { 2024-03-23T18:08:29.003 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 1, 2024-03-23T18:08:29.003 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000761442, 2024-03-23T18:08:29.003 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000761442 2024-03-23T18:08:29.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:29.003 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmsnap_latency": { 2024-03-23T18:08:29.003 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 11, 2024-03-23T18:08:29.003 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.427583970, 2024-03-23T18:08:29.004 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.038871270 2024-03-23T18:08:29.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:29.004 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmxattr_latency": { 2024-03-23T18:08:29.004 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:29.004 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:29.004 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:29.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:29.004 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setattr_latency": { 2024-03-23T18:08:29.004 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 26, 2024-03-23T18:08:29.005 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 1.201208837, 2024-03-23T18:08:29.005 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.046200339 2024-03-23T18:08:29.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:29.005 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setdirlayout_latency": { 2024-03-23T18:08:29.005 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:29.005 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:29.005 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:29.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:29.005 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setfilelock_latency": { 2024-03-23T18:08:29.006 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 560, 2024-03-23T18:08:29.006 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 1.777709617, 2024-03-23T18:08:29.006 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.003174481 2024-03-23T18:08:29.006 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:29.006 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setlayout_latency": { 2024-03-23T18:08:29.006 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:29.006 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:29.006 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:29.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:29.007 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setxattr_latency": { 2024-03-23T18:08:29.007 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 7, 2024-03-23T18:08:29.007 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.018878529, 2024-03-23T18:08:29.007 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.002696932 2024-03-23T18:08:29.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:29.007 INFO:teuthology.orchestra.run.smithi092.stdout: "req_symlink_latency": { 2024-03-23T18:08:29.007 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:29.007 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:29.008 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:29.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:29.008 INFO:teuthology.orchestra.run.smithi092.stdout: "req_unlink_latency": { 2024-03-23T18:08:29.008 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 19853, 2024-03-23T18:08:29.008 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 106.693878575, 2024-03-23T18:08:29.008 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.005374194 2024-03-23T18:08:29.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:29.008 INFO:teuthology.orchestra.run.smithi092.stdout: "cap_revoke_eviction": 0, 2024-03-23T18:08:29.009 INFO:teuthology.orchestra.run.smithi092.stdout: "cap_acquisition_throttle": 0, 2024-03-23T18:08:29.009 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getvxattr_latency": { 2024-03-23T18:08:29.009 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:29.009 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:29.009 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:29.009 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-23T18:08:29.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:29.009 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_sessions": { 2024-03-23T18:08:29.009 INFO:teuthology.orchestra.run.smithi092.stdout: "session_count": 2, 2024-03-23T18:08:29.010 INFO:teuthology.orchestra.run.smithi092.stdout: "session_add": 4, 2024-03-23T18:08:29.010 INFO:teuthology.orchestra.run.smithi092.stdout: "session_remove": 2, 2024-03-23T18:08:29.010 INFO:teuthology.orchestra.run.smithi092.stdout: "sessions_open": 2, 2024-03-23T18:08:29.010 INFO:teuthology.orchestra.run.smithi092.stdout: "sessions_stale": 0, 2024-03-23T18:08:29.010 INFO:teuthology.orchestra.run.smithi092.stdout: "total_load": 25513, 2024-03-23T18:08:29.010 INFO:teuthology.orchestra.run.smithi092.stdout: "average_load": 12756, 2024-03-23T18:08:29.010 INFO:teuthology.orchestra.run.smithi092.stdout: "avg_session_uptime": 1065, 2024-03-23T18:08:29.010 INFO:teuthology.orchestra.run.smithi092.stdout: "mdthresh_evicted": 0 2024-03-23T18:08:29.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:29.011 INFO:teuthology.orchestra.run.smithi092.stdout: "mempool": { 2024-03-23T18:08:29.011 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_bytes": 240, 2024-03-23T18:08:29.011 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_items": 240, 2024-03-23T18:08:29.011 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_bytes": 0, 2024-03-23T18:08:29.011 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_items": 0, 2024-03-23T18:08:29.011 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_bytes": 0, 2024-03-23T18:08:29.011 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_items": 0, 2024-03-23T18:08:29.011 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-23T18:08:29.012 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_items": 0, 2024-03-23T18:08:29.012 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-23T18:08:29.012 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_items": 0, 2024-03-23T18:08:29.012 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_bytes": 0, 2024-03-23T18:08:29.012 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_items": 0, 2024-03-23T18:08:29.012 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-23T18:08:29.012 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_items": 0, 2024-03-23T18:08:29.012 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_bytes": 0, 2024-03-23T18:08:29.013 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_items": 0, 2024-03-23T18:08:29.013 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_bytes": 0, 2024-03-23T18:08:29.013 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_items": 0, 2024-03-23T18:08:29.013 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-23T18:08:29.013 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_items": 0, 2024-03-23T18:08:29.013 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-23T18:08:29.013 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_items": 0, 2024-03-23T18:08:29.013 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_bytes": 0, 2024-03-23T18:08:29.013 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_items": 0, 2024-03-23T18:08:29.014 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_bytes": 0, 2024-03-23T18:08:29.014 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_items": 0, 2024-03-23T18:08:29.014 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-23T18:08:29.014 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_items": 0, 2024-03-23T18:08:29.014 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_bytes": 0, 2024-03-23T18:08:29.014 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_items": 0, 2024-03-23T18:08:29.014 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_bytes": 0, 2024-03-23T18:08:29.014 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_items": 0, 2024-03-23T18:08:29.014 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_bytes": 0, 2024-03-23T18:08:29.015 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_items": 0, 2024-03-23T18:08:29.015 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_bytes": 0, 2024-03-23T18:08:29.015 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_items": 0, 2024-03-23T18:08:29.015 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_bytes": 353462, 2024-03-23T18:08:29.015 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_items": 836, 2024-03-23T18:08:29.015 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_bytes": 88, 2024-03-23T18:08:29.015 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_items": 1, 2024-03-23T18:08:29.015 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_bytes": 0, 2024-03-23T18:08:29.015 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_items": 0, 2024-03-23T18:08:29.016 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_bytes": 0, 2024-03-23T18:08:29.016 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_items": 0, 2024-03-23T18:08:29.016 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_bytes": 0, 2024-03-23T18:08:29.016 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_items": 0, 2024-03-23T18:08:29.016 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_bytes": 7936, 2024-03-23T18:08:29.016 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_items": 165, 2024-03-23T18:08:29.016 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_bytes": 0, 2024-03-23T18:08:29.016 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_items": 0, 2024-03-23T18:08:29.017 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_bytes": 0, 2024-03-23T18:08:29.017 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_items": 0, 2024-03-23T18:08:29.017 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_bytes": 3978388, 2024-03-23T18:08:29.017 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_items": 50287, 2024-03-23T18:08:29.017 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_bytes": 0, 2024-03-23T18:08:29.017 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_items": 0, 2024-03-23T18:08:29.017 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_bytes": 0, 2024-03-23T18:08:29.017 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_items": 0 2024-03-23T18:08:29.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:29.018 INFO:teuthology.orchestra.run.smithi092.stdout: "objecter": { 2024-03-23T18:08:29.018 INFO:teuthology.orchestra.run.smithi092.stdout: "op_active": 0, 2024-03-23T18:08:29.018 INFO:teuthology.orchestra.run.smithi092.stdout: "op_laggy": 0, 2024-03-23T18:08:29.018 INFO:teuthology.orchestra.run.smithi092.stdout: "op_send": 101558, 2024-03-23T18:08:29.018 INFO:teuthology.orchestra.run.smithi092.stdout: "op_send_bytes": 647708144, 2024-03-23T18:08:29.018 INFO:teuthology.orchestra.run.smithi092.stdout: "op_resend": 37, 2024-03-23T18:08:29.018 INFO:teuthology.orchestra.run.smithi092.stdout: "op_reply": 101558, 2024-03-23T18:08:29.018 INFO:teuthology.orchestra.run.smithi092.stdout: "op_latency": { 2024-03-23T18:08:29.018 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 101558, 2024-03-23T18:08:29.019 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 1637.603058298, 2024-03-23T18:08:29.019 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.016124806 2024-03-23T18:08:29.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:29.019 INFO:teuthology.orchestra.run.smithi092.stdout: "op_inflight": 0, 2024-03-23T18:08:29.019 INFO:teuthology.orchestra.run.smithi092.stdout: "oplen_avg": { 2024-03-23T18:08:29.019 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 101558, 2024-03-23T18:08:29.019 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 131442 2024-03-23T18:08:29.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:29.020 INFO:teuthology.orchestra.run.smithi092.stdout: "op": 101558, 2024-03-23T18:08:29.020 INFO:teuthology.orchestra.run.smithi092.stdout: "op_r": 895, 2024-03-23T18:08:29.020 INFO:teuthology.orchestra.run.smithi092.stdout: "op_w": 100663, 2024-03-23T18:08:29.020 INFO:teuthology.orchestra.run.smithi092.stdout: "op_rmw": 0, 2024-03-23T18:08:29.020 INFO:teuthology.orchestra.run.smithi092.stdout: "op_pg": 0, 2024-03-23T18:08:29.020 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_stat": 2252, 2024-03-23T18:08:29.020 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_create": 11872, 2024-03-23T18:08:29.020 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_read": 688, 2024-03-23T18:08:29.021 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_write": 64487, 2024-03-23T18:08:29.021 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_writefull": 704, 2024-03-23T18:08:29.021 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_writesame": 0, 2024-03-23T18:08:29.021 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_append": 0, 2024-03-23T18:08:29.021 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_zero": 0, 2024-03-23T18:08:29.021 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_truncate": 12, 2024-03-23T18:08:29.021 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_delete": 20734, 2024-03-23T18:08:29.021 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_mapext": 0, 2024-03-23T18:08:29.021 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_sparse_read": 0, 2024-03-23T18:08:29.022 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_clonerange": 0, 2024-03-23T18:08:29.022 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_getxattr": 193, 2024-03-23T18:08:29.022 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_setxattr": 23744, 2024-03-23T18:08:29.022 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_cmpxattr": 0, 2024-03-23T18:08:29.022 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_rmxattr": 0, 2024-03-23T18:08:29.022 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_resetxattrs": 0, 2024-03-23T18:08:29.022 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_call": 0, 2024-03-23T18:08:29.022 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_watch": 0, 2024-03-23T18:08:29.023 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_notify": 0, 2024-03-23T18:08:29.023 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_src_cmpxattr": 0, 2024-03-23T18:08:29.023 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_pgls": 0, 2024-03-23T18:08:29.023 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_pgls_filter": 0, 2024-03-23T18:08:29.023 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_other": 0, 2024-03-23T18:08:29.023 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_active": 0, 2024-03-23T18:08:29.023 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_send": 0, 2024-03-23T18:08:29.023 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_resend": 0, 2024-03-23T18:08:29.024 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_ping": 0, 2024-03-23T18:08:29.024 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_active": 0, 2024-03-23T18:08:29.024 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_send": 0, 2024-03-23T18:08:29.024 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_resend": 0, 2024-03-23T18:08:29.024 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_active": 0, 2024-03-23T18:08:29.024 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_send": 0, 2024-03-23T18:08:29.025 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_resend": 0, 2024-03-23T18:08:29.025 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_active": 0, 2024-03-23T18:08:29.025 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_send": 0, 2024-03-23T18:08:29.025 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_resend": 0, 2024-03-23T18:08:29.025 INFO:teuthology.orchestra.run.smithi092.stdout: "command_active": 0, 2024-03-23T18:08:29.025 INFO:teuthology.orchestra.run.smithi092.stdout: "command_send": 0, 2024-03-23T18:08:29.025 INFO:teuthology.orchestra.run.smithi092.stdout: "command_resend": 0, 2024-03-23T18:08:29.025 INFO:teuthology.orchestra.run.smithi092.stdout: "map_epoch": 110, 2024-03-23T18:08:29.026 INFO:teuthology.orchestra.run.smithi092.stdout: "map_full": 0, 2024-03-23T18:08:29.026 INFO:teuthology.orchestra.run.smithi092.stdout: "map_inc": 31, 2024-03-23T18:08:29.026 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_sessions": 12, 2024-03-23T18:08:29.026 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_session_open": 12, 2024-03-23T18:08:29.026 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_session_close": 0, 2024-03-23T18:08:29.027 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_laggy": 0, 2024-03-23T18:08:29.027 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_wr": 5263, 2024-03-23T18:08:29.027 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_rd": 64, 2024-03-23T18:08:29.027 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_del": 1429 2024-03-23T18:08:29.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:29.027 INFO:teuthology.orchestra.run.smithi092.stdout: "oft": { 2024-03-23T18:08:29.027 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_objs": 1, 2024-03-23T18:08:29.027 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_kv_pairs": 0, 2024-03-23T18:08:29.028 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_updates": 0, 2024-03-23T18:08:29.028 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_removes": 0 2024-03-23T18:08:29.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:29.028 INFO:teuthology.orchestra.run.smithi092.stdout: "osd-slow-ops": { 2024-03-23T18:08:29.028 INFO:teuthology.orchestra.run.smithi092.stdout: "slow_ops_count": 223628 2024-03-23T18:08:29.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:29.029 INFO:teuthology.orchestra.run.smithi092.stdout: "purge_queue": { 2024-03-23T18:08:29.029 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_ops": 0, 2024-03-23T18:08:29.029 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_ops_high_water": 6, 2024-03-23T18:08:29.029 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing": 0, 2024-03-23T18:08:29.029 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_high_water": 6, 2024-03-23T18:08:29.029 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executed": 19853, 2024-03-23T18:08:29.030 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_item_in_journal": 0 2024-03-23T18:08:29.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:29.030 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-23T18:08:29.030 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-03-23T18:08:29.030 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 104857600, 2024-03-23T18:08:29.030 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-03-23T18:08:29.030 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 760700, 2024-03-23T18:08:29.031 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 152287343, 2024-03-23T18:08:29.031 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-03-23T18:08:29.031 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 760700, 2024-03-23T18:08:29.031 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-03-23T18:08:29.031 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-03-23T18:08:29.031 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 760700, 2024-03-23T18:08:29.032 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 152287343, 2024-03-23T18:08:29.032 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-03-23T18:08:29.032 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:29.032 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:29.032 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:29.032 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-23T18:08:29.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:29.033 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-objecter_bytes": { 2024-03-23T18:08:29.033 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-03-23T18:08:29.033 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 104857600, 2024-03-23T18:08:29.033 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-03-23T18:08:29.033 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-03-23T18:08:29.034 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-03-23T18:08:29.034 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-03-23T18:08:29.034 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-03-23T18:08:29.034 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 101558, 2024-03-23T18:08:29.034 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 649716425, 2024-03-23T18:08:29.034 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 101558, 2024-03-23T18:08:29.034 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 649716425, 2024-03-23T18:08:29.034 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-03-23T18:08:29.035 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:29.035 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:29.035 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:29.035 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-23T18:08:29.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:29.035 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-objecter_ops": { 2024-03-23T18:08:29.035 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-03-23T18:08:29.035 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 1024, 2024-03-23T18:08:29.036 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-03-23T18:08:29.036 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-03-23T18:08:29.036 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-03-23T18:08:29.036 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-03-23T18:08:29.036 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-03-23T18:08:29.036 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 101558, 2024-03-23T18:08:29.037 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 101558, 2024-03-23T18:08:29.037 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 101558, 2024-03-23T18:08:29.037 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 101558, 2024-03-23T18:08:29.037 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-03-23T18:08:29.037 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:29.037 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:29.037 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:29.037 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-23T18:08:29.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:29.038 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-write_buf_throttle": { 2024-03-23T18:08:29.038 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-03-23T18:08:29.038 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 3758096384, 2024-03-23T18:08:29.038 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-03-23T18:08:29.038 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 19853, 2024-03-23T18:08:29.038 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 2008769, 2024-03-23T18:08:29.038 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-03-23T18:08:29.038 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 19853, 2024-03-23T18:08:29.039 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-03-23T18:08:29.039 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-03-23T18:08:29.039 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 684, 2024-03-23T18:08:29.039 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 2008769, 2024-03-23T18:08:29.039 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-03-23T18:08:29.039 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:29.039 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:29.039 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:29.040 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-23T18:08:29.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:29.040 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-write_buf_throttle-0x55c8431440a0": { 2024-03-23T18:08:29.040 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 65890, 2024-03-23T18:08:29.040 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 3758096384, 2024-03-23T18:08:29.040 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-03-23T18:08:29.040 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 132507, 2024-03-23T18:08:29.040 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 623662246, 2024-03-23T18:08:29.041 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-03-23T18:08:29.041 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 132507, 2024-03-23T18:08:29.041 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-03-23T18:08:29.041 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-03-23T18:08:29.041 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 63803, 2024-03-23T18:08:29.041 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 623596356, 2024-03-23T18:08:29.041 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-03-23T18:08:29.041 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:29.042 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:29.042 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:29.042 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-23T18:08:29.042 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-23T18:08:29.042 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-03-23T18:08:29.319 INFO:tasks.check_counter:Daemon mds.b mds.exported=36 2024-03-23T18:08:29.319 INFO:tasks.check_counter:Daemon mds.b mds.imported=19 2024-03-23T18:08:29.320 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=7 2024-03-23T18:08:29.320 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=18 2024-03-23T18:08:29.320 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=11 2024-03-23T18:08:29.320 DEBUG:tasks.check_counter:Getting stats from e 2024-03-23T18:08:29.320 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph daemon mds.e perf dump 2024-03-23T18:08:29.551 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.b/config 2024-03-23T18:08:30.297 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:30 smithi092 ceph-mon[79089]: pgmap v565: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 6.4 MiB/s wr, 176 op/s 2024-03-23T18:08:30.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:30 smithi152 ceph-mon[47949]: pgmap v565: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 6.4 MiB/s wr, 176 op/s 2024-03-23T18:08:30.487 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-03-23T18:08:30.487 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-0": { 2024-03-23T18:08:30.487 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 134700, 2024-03-23T18:08:30.487 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 126265, 2024-03-23T18:08:30.487 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 44517186, 2024-03-23T18:08:30.487 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 118109998, 2024-03-23T18:08:30.488 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 10, 2024-03-23T18:08:30.488 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 4, 2024-03-23T18:08:30.488 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 7.766401112, 2024-03-23T18:08:30.488 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 3.409360215, 2024-03-23T18:08:30.488 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 5.138387041, 2024-03-23T18:08:30.488 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.373895761, 2024-03-23T18:08:30.488 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T18:08:30.489 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 126266, 2024-03-23T18:08:30.489 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 19.018140755, 2024-03-23T18:08:30.489 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000150619 2024-03-23T18:08:30.489 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:30.489 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-03-23T18:08:30.489 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 229839, 2024-03-23T18:08:30.490 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.037115353, 2024-03-23T18:08:30.490 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000161 2024-03-23T18:08:30.490 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:30.490 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-03-23T18:08:30.490 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 448 2024-03-23T18:08:30.490 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:30.490 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-1": { 2024-03-23T18:08:30.491 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 157427, 2024-03-23T18:08:30.491 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 230565, 2024-03-23T18:08:30.491 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 53478316, 2024-03-23T18:08:30.491 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 244630347, 2024-03-23T18:08:30.491 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 20, 2024-03-23T18:08:30.491 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 5, 2024-03-23T18:08:30.491 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 9.408036864, 2024-03-23T18:08:30.492 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 4.008982957, 2024-03-23T18:08:30.492 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 6.445267123, 2024-03-23T18:08:30.492 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.276241230, 2024-03-23T18:08:30.492 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T18:08:30.492 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 230564, 2024-03-23T18:08:30.492 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 56.536753484, 2024-03-23T18:08:30.492 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000245210 2024-03-23T18:08:30.492 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:30.493 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-03-23T18:08:30.493 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 297178, 2024-03-23T18:08:30.493 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.062188749, 2024-03-23T18:08:30.493 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000209 2024-03-23T18:08:30.493 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:30.493 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 2015216, 2024-03-23T18:08:30.493 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 205680 2024-03-23T18:08:30.493 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:30.493 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-2": { 2024-03-23T18:08:30.494 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 15582, 2024-03-23T18:08:30.494 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 15828, 2024-03-23T18:08:30.494 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 3905144, 2024-03-23T18:08:30.494 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 120166037, 2024-03-23T18:08:30.494 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 17, 2024-03-23T18:08:30.494 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 7, 2024-03-23T18:08:30.494 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 1.318522554, 2024-03-23T18:08:30.494 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.469550803, 2024-03-23T18:08:30.495 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.252258690, 2024-03-23T18:08:30.495 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.510795993, 2024-03-23T18:08:30.495 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T18:08:30.495 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 15823, 2024-03-23T18:08:30.495 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 2.460854245, 2024-03-23T18:08:30.495 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000155523 2024-03-23T18:08:30.495 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:30.495 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-03-23T18:08:30.495 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 19, 2024-03-23T18:08:30.496 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000003119, 2024-03-23T18:08:30.496 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000164 2024-03-23T18:08:30.496 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:30.496 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 2304, 2024-03-23T18:08:30.496 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 664368 2024-03-23T18:08:30.496 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:30.496 INFO:teuthology.orchestra.run.smithi092.stdout: "cct": { 2024-03-23T18:08:30.496 INFO:teuthology.orchestra.run.smithi092.stdout: "total_workers": 1, 2024-03-23T18:08:30.496 INFO:teuthology.orchestra.run.smithi092.stdout: "unhealthy_workers": 0 2024-03-23T18:08:30.497 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:30.497 INFO:teuthology.orchestra.run.smithi092.stdout: "finisher-MDSRank": { 2024-03-23T18:08:30.497 INFO:teuthology.orchestra.run.smithi092.stdout: "queue_len": 0, 2024-03-23T18:08:30.497 INFO:teuthology.orchestra.run.smithi092.stdout: "complete_latency": { 2024-03-23T18:08:30.497 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 37076, 2024-03-23T18:08:30.497 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 43.405041606, 2024-03-23T18:08:30.497 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.001170704 2024-03-23T18:08:30.497 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-23T18:08:30.498 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:30.498 INFO:teuthology.orchestra.run.smithi092.stdout: "finisher-PurgeQueue": { 2024-03-23T18:08:30.498 INFO:teuthology.orchestra.run.smithi092.stdout: "queue_len": 0, 2024-03-23T18:08:30.498 INFO:teuthology.orchestra.run.smithi092.stdout: "complete_latency": { 2024-03-23T18:08:30.498 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 23375, 2024-03-23T18:08:30.498 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 3.093754548, 2024-03-23T18:08:30.498 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000132353 2024-03-23T18:08:30.498 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-23T18:08:30.499 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:30.499 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": { 2024-03-23T18:08:30.499 INFO:teuthology.orchestra.run.smithi092.stdout: "request": 86655, 2024-03-23T18:08:30.499 INFO:teuthology.orchestra.run.smithi092.stdout: "reply": 85133, 2024-03-23T18:08:30.499 INFO:teuthology.orchestra.run.smithi092.stdout: "reply_latency": { 2024-03-23T18:08:30.499 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 85133, 2024-03-23T18:08:30.499 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 347.647303399, 2024-03-23T18:08:30.499 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.004083578 2024-03-23T18:08:30.500 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:30.500 INFO:teuthology.orchestra.run.smithi092.stdout: "slow_reply": 0, 2024-03-23T18:08:30.500 INFO:teuthology.orchestra.run.smithi092.stdout: "forward": 1522, 2024-03-23T18:08:30.500 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_fetch_complete": 1, 2024-03-23T18:08:30.500 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_fetch_keys": 0, 2024-03-23T18:08:30.500 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_commit": 82, 2024-03-23T18:08:30.500 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_split": 8, 2024-03-23T18:08:30.500 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_merge": 7, 2024-03-23T18:08:30.501 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes": 2069, 2024-03-23T18:08:30.501 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_top": 703, 2024-03-23T18:08:30.501 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_bottom": 1339, 2024-03-23T18:08:30.501 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_pin_tail": 27, 2024-03-23T18:08:30.501 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_pinned": 1064, 2024-03-23T18:08:30.501 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_expired": 337, 2024-03-23T18:08:30.501 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_with_caps": 260, 2024-03-23T18:08:30.501 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": 260, 2024-03-23T18:08:30.501 INFO:teuthology.orchestra.run.smithi092.stdout: "subtrees": 22, 2024-03-23T18:08:30.502 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse": 126852, 2024-03-23T18:08:30.502 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_hit": 75005, 2024-03-23T18:08:30.502 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_forward": 1512, 2024-03-23T18:08:30.502 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_discover": 62, 2024-03-23T18:08:30.502 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_dir_fetch": 0, 2024-03-23T18:08:30.502 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_remote_ino": 0, 2024-03-23T18:08:30.502 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_lock": 611, 2024-03-23T18:08:30.503 INFO:teuthology.orchestra.run.smithi092.stdout: "load_cent": 30157, 2024-03-23T18:08:30.503 INFO:teuthology.orchestra.run.smithi092.stdout: "q": 0, 2024-03-23T18:08:30.503 INFO:teuthology.orchestra.run.smithi092.stdout: "exported": 9, 2024-03-23T18:08:30.503 INFO:teuthology.orchestra.run.smithi092.stdout: "exported_inodes": 290, 2024-03-23T18:08:30.503 INFO:teuthology.orchestra.run.smithi092.stdout: "imported": 20, 2024-03-23T18:08:30.503 INFO:teuthology.orchestra.run.smithi092.stdout: "imported_inodes": 737, 2024-03-23T18:08:30.503 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_dir_fetch": 0, 2024-03-23T18:08:30.503 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_backtrace_fetch": 7, 2024-03-23T18:08:30.504 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_peer_discover": 0, 2024-03-23T18:08:30.504 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rfiles": 699, 2024-03-23T18:08:30.504 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rbytes": 211642430, 2024-03-23T18:08:30.504 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rsnaps": 7, 2024-03-23T18:08:30.504 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_backtrace_fetch": 7, 2024-03-23T18:08:30.504 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_set_tag": 0, 2024-03-23T18:08:30.504 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_backtrace_repaired": 0, 2024-03-23T18:08:30.504 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_inotable_repaired": 0, 2024-03-23T18:08:30.504 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dir_inodes": 0, 2024-03-23T18:08:30.505 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dir_base_inodes": 0, 2024-03-23T18:08:30.505 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dirfrag_rstats": 0, 2024-03-23T18:08:30.505 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_file_inodes": 7, 2024-03-23T18:08:30.505 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_inode_file_caps": 4, 2024-03-23T18:08:30.505 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_revoke": 20252, 2024-03-23T18:08:30.505 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_grant": 55165, 2024-03-23T18:08:30.505 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_trunc": 0, 2024-03-23T18:08:30.505 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-23T18:08:30.506 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-23T18:08:30.506 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_caps": 41399, 2024-03-23T18:08:30.506 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_caps_dirty": 11711, 2024-03-23T18:08:30.506 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_cap_release": 3701, 2024-03-23T18:08:30.506 INFO:teuthology.orchestra.run.smithi092.stdout: "process_request_cap_release": 59425 2024-03-23T18:08:30.506 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:30.506 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_cache": { 2024-03-23T18:08:30.506 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_update": 28, 2024-03-23T18:08:30.507 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_update_receipt": 128, 2024-03-23T18:08:30.507 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_try_discover": 73, 2024-03-23T18:08:30.507 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_send_discover": 167, 2024-03-23T18:08:30.507 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_handle_discover": 73, 2024-03-23T18:08:30.507 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays": 98, 2024-03-23T18:08:30.507 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays_delayed": 0, 2024-03-23T18:08:30.507 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays_enqueuing": 0, 2024-03-23T18:08:30.508 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_created": 13003, 2024-03-23T18:08:30.508 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_enqueued": 12905, 2024-03-23T18:08:30.508 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_reintegrated": 0, 2024-03-23T18:08:30.508 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_migrated": 0, 2024-03-23T18:08:30.508 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_processing": 0, 2024-03-23T18:08:30.508 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_enqueued": 0, 2024-03-23T18:08:30.508 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_prioritized": 0, 2024-03-23T18:08:30.508 INFO:teuthology.orchestra.run.smithi092.stdout: "recovery_started": 0, 2024-03-23T18:08:30.509 INFO:teuthology.orchestra.run.smithi092.stdout: "recovery_completed": 0, 2024-03-23T18:08:30.509 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_enqueue_scrub": 0, 2024-03-23T18:08:30.509 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_exportdir": 9, 2024-03-23T18:08:30.509 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_flush": 0, 2024-03-23T18:08:30.509 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_fragmentdir": 15, 2024-03-23T18:08:30.509 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_fragstats": 0, 2024-03-23T18:08:30.509 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_inodestats": 0 2024-03-23T18:08:30.509 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:30.510 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_log": { 2024-03-23T18:08:30.510 INFO:teuthology.orchestra.run.smithi092.stdout: "evadd": 71965, 2024-03-23T18:08:30.510 INFO:teuthology.orchestra.run.smithi092.stdout: "evex": 0, 2024-03-23T18:08:30.510 INFO:teuthology.orchestra.run.smithi092.stdout: "evtrm": 0, 2024-03-23T18:08:30.510 INFO:teuthology.orchestra.run.smithi092.stdout: "ev": 71965, 2024-03-23T18:08:30.510 INFO:teuthology.orchestra.run.smithi092.stdout: "evexg": 0, 2024-03-23T18:08:30.510 INFO:teuthology.orchestra.run.smithi092.stdout: "evexd": 0, 2024-03-23T18:08:30.510 INFO:teuthology.orchestra.run.smithi092.stdout: "segadd": 97, 2024-03-23T18:08:30.511 INFO:teuthology.orchestra.run.smithi092.stdout: "segex": 0, 2024-03-23T18:08:30.511 INFO:teuthology.orchestra.run.smithi092.stdout: "segtrm": 0, 2024-03-23T18:08:30.511 INFO:teuthology.orchestra.run.smithi092.stdout: "seg": 97, 2024-03-23T18:08:30.511 INFO:teuthology.orchestra.run.smithi092.stdout: "segexg": 0, 2024-03-23T18:08:30.511 INFO:teuthology.orchestra.run.smithi092.stdout: "segexd": 0, 2024-03-23T18:08:30.511 INFO:teuthology.orchestra.run.smithi092.stdout: "expos": 4194304, 2024-03-23T18:08:30.511 INFO:teuthology.orchestra.run.smithi092.stdout: "wrpos": 262319695, 2024-03-23T18:08:30.511 INFO:teuthology.orchestra.run.smithi092.stdout: "rdpos": 0, 2024-03-23T18:08:30.512 INFO:teuthology.orchestra.run.smithi092.stdout: "jlat": { 2024-03-23T18:08:30.512 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 20445, 2024-03-23T18:08:30.512 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 157.687513981, 2024-03-23T18:08:30.512 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.007712766 2024-03-23T18:08:30.512 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:30.512 INFO:teuthology.orchestra.run.smithi092.stdout: "replayed": 0 2024-03-23T18:08:30.512 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:30.512 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_mem": { 2024-03-23T18:08:30.512 INFO:teuthology.orchestra.run.smithi092.stdout: "ino": 1036, 2024-03-23T18:08:30.513 INFO:teuthology.orchestra.run.smithi092.stdout: "ino+": 14210, 2024-03-23T18:08:30.513 INFO:teuthology.orchestra.run.smithi092.stdout: "ino-": 13174, 2024-03-23T18:08:30.513 INFO:teuthology.orchestra.run.smithi092.stdout: "dir": 144, 2024-03-23T18:08:30.513 INFO:teuthology.orchestra.run.smithi092.stdout: "dir+": 287, 2024-03-23T18:08:30.513 INFO:teuthology.orchestra.run.smithi092.stdout: "dir-": 143, 2024-03-23T18:08:30.514 INFO:teuthology.orchestra.run.smithi092.stdout: "dn": 1909, 2024-03-23T18:08:30.514 INFO:teuthology.orchestra.run.smithi092.stdout: "dn+": 27711, 2024-03-23T18:08:30.514 INFO:teuthology.orchestra.run.smithi092.stdout: "dn-": 25802, 2024-03-23T18:08:30.514 INFO:teuthology.orchestra.run.smithi092.stdout: "cap": 160, 2024-03-23T18:08:30.515 INFO:teuthology.orchestra.run.smithi092.stdout: "cap+": 14650, 2024-03-23T18:08:30.515 INFO:teuthology.orchestra.run.smithi092.stdout: "cap-": 14490, 2024-03-23T18:08:30.515 INFO:teuthology.orchestra.run.smithi092.stdout: "rss": 69472, 2024-03-23T18:08:30.515 INFO:teuthology.orchestra.run.smithi092.stdout: "heap": 198964 2024-03-23T18:08:30.516 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:30.516 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_server": { 2024-03-23T18:08:30.516 INFO:teuthology.orchestra.run.smithi092.stdout: "dispatch_client_request": 120214, 2024-03-23T18:08:30.516 INFO:teuthology.orchestra.run.smithi092.stdout: "dispatch_server_request": 985, 2024-03-23T18:08:30.517 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_request": 86655, 2024-03-23T18:08:30.517 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_session": 1670, 2024-03-23T18:08:30.517 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_peer_request": 5000, 2024-03-23T18:08:30.517 INFO:teuthology.orchestra.run.smithi092.stdout: "req_create_latency": { 2024-03-23T18:08:30.518 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 12959, 2024-03-23T18:08:30.518 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 34.787195934, 2024-03-23T18:08:30.518 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.002684404 2024-03-23T18:08:30.518 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:30.519 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getattr_latency": { 2024-03-23T18:08:30.519 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 1009, 2024-03-23T18:08:30.519 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 17.539624835, 2024-03-23T18:08:30.519 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.017383176 2024-03-23T18:08:30.520 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:30.520 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getfilelock_latency": { 2024-03-23T18:08:30.520 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:30.521 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:30.521 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:30.521 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:30.521 INFO:teuthology.orchestra.run.smithi092.stdout: "req_link_latency": { 2024-03-23T18:08:30.522 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:30.522 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:30.522 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:30.522 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:30.523 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookup_latency": { 2024-03-23T18:08:30.523 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 22236, 2024-03-23T18:08:30.523 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 48.759982916, 2024-03-23T18:08:30.523 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.002192839 2024-03-23T18:08:30.524 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:30.524 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookuphash_latency": { 2024-03-23T18:08:30.524 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:30.524 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:30.525 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:30.525 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:30.525 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupino_latency": { 2024-03-23T18:08:30.525 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:30.525 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:30.526 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:30.526 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:30.526 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupname_latency": { 2024-03-23T18:08:30.526 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:30.527 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:30.527 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:30.527 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:30.527 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupparent_latency": { 2024-03-23T18:08:30.527 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:30.527 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:30.528 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:30.528 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:30.528 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupsnap_latency": { 2024-03-23T18:08:30.528 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:30.528 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:30.528 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:30.529 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:30.529 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lssnap_latency": { 2024-03-23T18:08:30.529 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:30.529 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:30.530 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:30.530 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:30.530 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mkdir_latency": { 2024-03-23T18:08:30.530 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:30.530 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:30.530 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:30.531 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:30.531 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mknod_latency": { 2024-03-23T18:08:30.531 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:30.531 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:30.531 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:30.531 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:30.531 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mksnap_latency": { 2024-03-23T18:08:30.531 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:30.532 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:30.532 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:30.532 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:30.532 INFO:teuthology.orchestra.run.smithi092.stdout: "req_open_latency": { 2024-03-23T18:08:30.532 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 1, 2024-03-23T18:08:30.532 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000471953, 2024-03-23T18:08:30.532 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000471953 2024-03-23T18:08:30.532 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:30.532 INFO:teuthology.orchestra.run.smithi092.stdout: "req_readdir_latency": { 2024-03-23T18:08:30.533 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 33279, 2024-03-23T18:08:30.533 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 164.330144903, 2024-03-23T18:08:30.533 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.004937953 2024-03-23T18:08:30.533 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:30.533 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rename_latency": { 2024-03-23T18:08:30.533 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 2248, 2024-03-23T18:08:30.533 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 45.840370876, 2024-03-23T18:08:30.533 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.020391624 2024-03-23T18:08:30.534 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:30.534 INFO:teuthology.orchestra.run.smithi092.stdout: "req_renamesnap_latency": { 2024-03-23T18:08:30.534 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:30.534 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:30.534 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:30.534 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:30.534 INFO:teuthology.orchestra.run.smithi092.stdout: "req_snapdiff_latency": { 2024-03-23T18:08:30.534 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:30.535 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:30.535 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:30.535 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:30.535 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmdir_latency": { 2024-03-23T18:08:30.535 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:30.535 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:30.535 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:30.535 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:30.535 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmsnap_latency": { 2024-03-23T18:08:30.536 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:30.536 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:30.536 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:30.536 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:30.536 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmxattr_latency": { 2024-03-23T18:08:30.536 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:30.536 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:30.536 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:30.536 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:30.537 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setattr_latency": { 2024-03-23T18:08:30.537 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 10, 2024-03-23T18:08:30.537 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.145932493, 2024-03-23T18:08:30.537 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.014593249 2024-03-23T18:08:30.537 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:30.537 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setdirlayout_latency": { 2024-03-23T18:08:30.537 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:30.537 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:30.537 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:30.538 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:30.538 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setfilelock_latency": { 2024-03-23T18:08:30.538 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 388, 2024-03-23T18:08:30.538 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.411280316, 2024-03-23T18:08:30.538 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.001060000 2024-03-23T18:08:30.538 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:30.538 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setlayout_latency": { 2024-03-23T18:08:30.538 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:30.539 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:30.539 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:30.539 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:30.539 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setxattr_latency": { 2024-03-23T18:08:30.539 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:30.539 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:30.539 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:30.539 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:30.539 INFO:teuthology.orchestra.run.smithi092.stdout: "req_symlink_latency": { 2024-03-23T18:08:30.540 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:30.540 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:30.540 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:30.540 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:30.540 INFO:teuthology.orchestra.run.smithi092.stdout: "req_unlink_latency": { 2024-03-23T18:08:30.540 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 13003, 2024-03-23T18:08:30.540 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 35.832299173, 2024-03-23T18:08:30.541 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.002755694 2024-03-23T18:08:30.541 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:30.541 INFO:teuthology.orchestra.run.smithi092.stdout: "cap_revoke_eviction": 0, 2024-03-23T18:08:30.541 INFO:teuthology.orchestra.run.smithi092.stdout: "cap_acquisition_throttle": 0, 2024-03-23T18:08:30.541 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getvxattr_latency": { 2024-03-23T18:08:30.541 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:30.541 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:30.541 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:30.541 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-23T18:08:30.542 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:30.542 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_sessions": { 2024-03-23T18:08:30.542 INFO:teuthology.orchestra.run.smithi092.stdout: "session_count": 1, 2024-03-23T18:08:30.542 INFO:teuthology.orchestra.run.smithi092.stdout: "session_add": 3, 2024-03-23T18:08:30.542 INFO:teuthology.orchestra.run.smithi092.stdout: "session_remove": 2, 2024-03-23T18:08:30.542 INFO:teuthology.orchestra.run.smithi092.stdout: "sessions_open": 1, 2024-03-23T18:08:30.543 INFO:teuthology.orchestra.run.smithi092.stdout: "sessions_stale": 0, 2024-03-23T18:08:30.543 INFO:teuthology.orchestra.run.smithi092.stdout: "total_load": 21330, 2024-03-23T18:08:30.543 INFO:teuthology.orchestra.run.smithi092.stdout: "average_load": 21330, 2024-03-23T18:08:30.543 INFO:teuthology.orchestra.run.smithi092.stdout: "avg_session_uptime": 1086, 2024-03-23T18:08:30.543 INFO:teuthology.orchestra.run.smithi092.stdout: "mdthresh_evicted": 0 2024-03-23T18:08:30.543 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:30.543 INFO:teuthology.orchestra.run.smithi092.stdout: "mempool": { 2024-03-23T18:08:30.544 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_bytes": 0, 2024-03-23T18:08:30.544 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_items": 0, 2024-03-23T18:08:30.544 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_bytes": 0, 2024-03-23T18:08:30.544 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_items": 0, 2024-03-23T18:08:30.544 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_bytes": 0, 2024-03-23T18:08:30.544 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_items": 0, 2024-03-23T18:08:30.545 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-23T18:08:30.545 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_items": 0, 2024-03-23T18:08:30.545 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-23T18:08:30.545 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_items": 0, 2024-03-23T18:08:30.546 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_bytes": 0, 2024-03-23T18:08:30.546 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_items": 0, 2024-03-23T18:08:30.546 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-23T18:08:30.546 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_items": 0, 2024-03-23T18:08:30.547 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_bytes": 0, 2024-03-23T18:08:30.547 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_items": 0, 2024-03-23T18:08:30.547 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_bytes": 0, 2024-03-23T18:08:30.547 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_items": 0, 2024-03-23T18:08:30.548 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-23T18:08:30.548 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_items": 0, 2024-03-23T18:08:30.548 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-23T18:08:30.548 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_items": 0, 2024-03-23T18:08:30.549 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_bytes": 0, 2024-03-23T18:08:30.549 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_items": 0, 2024-03-23T18:08:30.549 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_bytes": 0, 2024-03-23T18:08:30.549 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_items": 0, 2024-03-23T18:08:30.550 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-23T18:08:30.550 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_items": 0, 2024-03-23T18:08:30.550 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_bytes": 0, 2024-03-23T18:08:30.550 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_items": 0, 2024-03-23T18:08:30.551 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_bytes": 0, 2024-03-23T18:08:30.551 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_items": 0, 2024-03-23T18:08:30.551 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_bytes": 0, 2024-03-23T18:08:30.551 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_items": 0, 2024-03-23T18:08:30.552 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_bytes": 0, 2024-03-23T18:08:30.552 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_items": 0, 2024-03-23T18:08:30.552 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_bytes": 1702230, 2024-03-23T18:08:30.552 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_items": 1286, 2024-03-23T18:08:30.553 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_bytes": 352, 2024-03-23T18:08:30.553 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_items": 4, 2024-03-23T18:08:30.553 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_bytes": 0, 2024-03-23T18:08:30.553 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_items": 0, 2024-03-23T18:08:30.554 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_bytes": 0, 2024-03-23T18:08:30.554 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_items": 0, 2024-03-23T18:08:30.554 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_bytes": 0, 2024-03-23T18:08:30.554 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_items": 0, 2024-03-23T18:08:30.555 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_bytes": 7936, 2024-03-23T18:08:30.555 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_items": 165, 2024-03-23T18:08:30.555 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_bytes": 0, 2024-03-23T18:08:30.555 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_items": 0, 2024-03-23T18:08:30.556 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_bytes": 0, 2024-03-23T18:08:30.556 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_items": 0, 2024-03-23T18:08:30.556 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_bytes": 4777694, 2024-03-23T18:08:30.556 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_items": 74114, 2024-03-23T18:08:30.557 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_bytes": 0, 2024-03-23T18:08:30.557 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_items": 0, 2024-03-23T18:08:30.557 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_bytes": 0, 2024-03-23T18:08:30.557 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_items": 0 2024-03-23T18:08:30.558 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:30.558 INFO:teuthology.orchestra.run.smithi092.stdout: "objecter": { 2024-03-23T18:08:30.558 INFO:teuthology.orchestra.run.smithi092.stdout: "op_active": 0, 2024-03-23T18:08:30.558 INFO:teuthology.orchestra.run.smithi092.stdout: "op_laggy": 0, 2024-03-23T18:08:30.559 INFO:teuthology.orchestra.run.smithi092.stdout: "op_send": 35525, 2024-03-23T18:08:30.559 INFO:teuthology.orchestra.run.smithi092.stdout: "op_send_bytes": 260164413, 2024-03-23T18:08:30.559 INFO:teuthology.orchestra.run.smithi092.stdout: "op_resend": 33, 2024-03-23T18:08:30.560 INFO:teuthology.orchestra.run.smithi092.stdout: "op_reply": 35525, 2024-03-23T18:08:30.560 INFO:teuthology.orchestra.run.smithi092.stdout: "op_latency": { 2024-03-23T18:08:30.560 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 35525, 2024-03-23T18:08:30.560 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 295.612529253, 2024-03-23T18:08:30.561 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.008321253 2024-03-23T18:08:30.561 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:30.561 INFO:teuthology.orchestra.run.smithi092.stdout: "op_inflight": 0, 2024-03-23T18:08:30.561 INFO:teuthology.orchestra.run.smithi092.stdout: "oplen_avg": { 2024-03-23T18:08:30.562 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 35525, 2024-03-23T18:08:30.562 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 35986 2024-03-23T18:08:30.562 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:30.563 INFO:teuthology.orchestra.run.smithi092.stdout: "op": 35525, 2024-03-23T18:08:30.563 INFO:teuthology.orchestra.run.smithi092.stdout: "op_r": 677, 2024-03-23T18:08:30.563 INFO:teuthology.orchestra.run.smithi092.stdout: "op_w": 34848, 2024-03-23T18:08:30.563 INFO:teuthology.orchestra.run.smithi092.stdout: "op_rmw": 0, 2024-03-23T18:08:30.564 INFO:teuthology.orchestra.run.smithi092.stdout: "op_pg": 0, 2024-03-23T18:08:30.564 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_stat": 0, 2024-03-23T18:08:30.564 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_create": 10, 2024-03-23T18:08:30.564 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_read": 669, 2024-03-23T18:08:30.565 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_write": 21114, 2024-03-23T18:08:30.565 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_writefull": 408, 2024-03-23T18:08:30.565 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_writesame": 0, 2024-03-23T18:08:30.565 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_append": 0, 2024-03-23T18:08:30.566 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_zero": 0, 2024-03-23T18:08:30.566 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_truncate": 8, 2024-03-23T18:08:30.566 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_delete": 13027, 2024-03-23T18:08:30.566 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_mapext": 0, 2024-03-23T18:08:30.567 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_sparse_read": 0, 2024-03-23T18:08:30.567 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_clonerange": 0, 2024-03-23T18:08:30.567 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_getxattr": 7, 2024-03-23T18:08:30.568 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_setxattr": 20, 2024-03-23T18:08:30.568 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_cmpxattr": 0, 2024-03-23T18:08:30.568 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_rmxattr": 0, 2024-03-23T18:08:30.568 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_resetxattrs": 0, 2024-03-23T18:08:30.569 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_call": 0, 2024-03-23T18:08:30.569 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_watch": 0, 2024-03-23T18:08:30.569 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_notify": 0, 2024-03-23T18:08:30.570 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_src_cmpxattr": 0, 2024-03-23T18:08:30.570 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_pgls": 0, 2024-03-23T18:08:30.570 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_pgls_filter": 0, 2024-03-23T18:08:30.570 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_other": 0, 2024-03-23T18:08:30.571 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_active": 0, 2024-03-23T18:08:30.571 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_send": 0, 2024-03-23T18:08:30.571 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_resend": 0, 2024-03-23T18:08:30.571 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_ping": 0, 2024-03-23T18:08:30.572 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_active": 0, 2024-03-23T18:08:30.572 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_send": 0, 2024-03-23T18:08:30.572 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_resend": 0, 2024-03-23T18:08:30.572 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_active": 0, 2024-03-23T18:08:30.572 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_send": 0, 2024-03-23T18:08:30.572 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_resend": 0, 2024-03-23T18:08:30.572 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_active": 0, 2024-03-23T18:08:30.572 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_send": 0, 2024-03-23T18:08:30.573 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_resend": 0, 2024-03-23T18:08:30.573 INFO:teuthology.orchestra.run.smithi092.stdout: "command_active": 0, 2024-03-23T18:08:30.573 INFO:teuthology.orchestra.run.smithi092.stdout: "command_send": 0, 2024-03-23T18:08:30.573 INFO:teuthology.orchestra.run.smithi092.stdout: "command_resend": 0, 2024-03-23T18:08:30.573 INFO:teuthology.orchestra.run.smithi092.stdout: "map_epoch": 110, 2024-03-23T18:08:30.573 INFO:teuthology.orchestra.run.smithi092.stdout: "map_full": 0, 2024-03-23T18:08:30.573 INFO:teuthology.orchestra.run.smithi092.stdout: "map_inc": 30, 2024-03-23T18:08:30.573 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_sessions": 12, 2024-03-23T18:08:30.574 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_session_open": 12, 2024-03-23T18:08:30.574 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_session_close": 0, 2024-03-23T18:08:30.574 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_laggy": 0, 2024-03-23T18:08:30.575 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_wr": 533, 2024-03-23T18:08:30.575 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_rd": 2, 2024-03-23T18:08:30.575 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_del": 188 2024-03-23T18:08:30.575 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:30.575 INFO:teuthology.orchestra.run.smithi092.stdout: "oft": { 2024-03-23T18:08:30.576 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_objs": 1, 2024-03-23T18:08:30.576 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_kv_pairs": 0, 2024-03-23T18:08:30.576 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_updates": 0, 2024-03-23T18:08:30.576 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_removes": 0 2024-03-23T18:08:30.576 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:30.577 INFO:teuthology.orchestra.run.smithi092.stdout: "osd-slow-ops": { 2024-03-23T18:08:30.577 INFO:teuthology.orchestra.run.smithi092.stdout: "slow_ops_count": 87663 2024-03-23T18:08:30.577 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:30.577 INFO:teuthology.orchestra.run.smithi092.stdout: "purge_queue": { 2024-03-23T18:08:30.577 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_ops": 0, 2024-03-23T18:08:30.578 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_ops_high_water": 6, 2024-03-23T18:08:30.578 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing": 0, 2024-03-23T18:08:30.578 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_high_water": 6, 2024-03-23T18:08:30.578 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executed": 12905, 2024-03-23T18:08:30.578 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_item_in_journal": 0 2024-03-23T18:08:30.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:30.579 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-23T18:08:30.579 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-03-23T18:08:30.579 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 104857600, 2024-03-23T18:08:30.580 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-03-23T18:08:30.580 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 307705, 2024-03-23T18:08:30.580 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 74180817, 2024-03-23T18:08:30.580 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-03-23T18:08:30.580 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 307705, 2024-03-23T18:08:30.580 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-03-23T18:08:30.581 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-03-23T18:08:30.581 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 307705, 2024-03-23T18:08:30.581 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 74180817, 2024-03-23T18:08:30.581 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-03-23T18:08:30.581 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:30.582 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:30.582 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:30.582 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-23T18:08:30.582 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:30.582 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-objecter_bytes": { 2024-03-23T18:08:30.583 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-03-23T18:08:30.583 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 104857600, 2024-03-23T18:08:30.583 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-03-23T18:08:30.583 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-03-23T18:08:30.583 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-03-23T18:08:30.584 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-03-23T18:08:30.584 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-03-23T18:08:30.584 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 35525, 2024-03-23T18:08:30.584 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 261470002, 2024-03-23T18:08:30.585 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 35525, 2024-03-23T18:08:30.585 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 261470002, 2024-03-23T18:08:30.585 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-03-23T18:08:30.585 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:30.585 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:30.585 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:30.586 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-23T18:08:30.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:30.586 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-objecter_ops": { 2024-03-23T18:08:30.586 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-03-23T18:08:30.586 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 1024, 2024-03-23T18:08:30.587 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-03-23T18:08:30.587 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-03-23T18:08:30.587 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-03-23T18:08:30.587 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-03-23T18:08:30.587 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-03-23T18:08:30.587 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 35525, 2024-03-23T18:08:30.587 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 35525, 2024-03-23T18:08:30.588 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 35525, 2024-03-23T18:08:30.588 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 35525, 2024-03-23T18:08:30.588 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-03-23T18:08:30.589 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:30.589 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:30.590 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:30.590 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-23T18:08:30.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:30.590 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-write_buf_throttle": { 2024-03-23T18:08:30.590 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-03-23T18:08:30.590 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 3758096384, 2024-03-23T18:08:30.590 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-03-23T18:08:30.591 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 12905, 2024-03-23T18:08:30.591 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 1305605, 2024-03-23T18:08:30.591 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-03-23T18:08:30.591 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 12905, 2024-03-23T18:08:30.591 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-03-23T18:08:30.591 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-03-23T18:08:30.591 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 669, 2024-03-23T18:08:30.591 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 1305605, 2024-03-23T18:08:30.591 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-03-23T18:08:30.592 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:30.592 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:30.592 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:30.592 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-23T18:08:30.592 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:30.592 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-write_buf_throttle-0x55dee70d20a0": { 2024-03-23T18:08:30.592 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 780887, 2024-03-23T18:08:30.592 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 3758096384, 2024-03-23T18:08:30.593 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-03-23T18:08:30.593 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 71965, 2024-03-23T18:08:30.593 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 258125391, 2024-03-23T18:08:30.593 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-03-23T18:08:30.593 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 71965, 2024-03-23T18:08:30.593 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-03-23T18:08:30.593 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-03-23T18:08:30.593 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 20445, 2024-03-23T18:08:30.593 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 257344504, 2024-03-23T18:08:30.594 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-03-23T18:08:30.594 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:30.594 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:30.594 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:30.594 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-23T18:08:30.594 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-23T18:08:30.594 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-03-23T18:08:30.595 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:30 smithi086 ceph-mon[38221]: pgmap v565: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 6.4 MiB/s wr, 176 op/s 2024-03-23T18:08:30.851 INFO:tasks.check_counter:Daemon mds.e mds.exported=9 2024-03-23T18:08:30.852 INFO:tasks.check_counter:Daemon mds.e mds.imported=20 2024-03-23T18:08:30.852 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=7 2024-03-23T18:08:30.852 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-03-23T18:08:30.852 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-03-23T18:08:30.852 DEBUG:tasks.check_counter:Getting stats from h 2024-03-23T18:08:30.852 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph daemon mds.h perf dump 2024-03-23T18:08:31.098 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.b/config 2024-03-23T18:08:32.044 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-03-23T18:08:32.045 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-0": { 2024-03-23T18:08:32.045 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 376154, 2024-03-23T18:08:32.045 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 447437, 2024-03-23T18:08:32.045 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 123051510, 2024-03-23T18:08:32.045 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 385715883, 2024-03-23T18:08:32.045 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 16, 2024-03-23T18:08:32.046 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 4, 2024-03-23T18:08:32.046 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 20.242350444, 2024-03-23T18:08:32.046 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 8.855693959, 2024-03-23T18:08:32.046 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 14.371497831, 2024-03-23T18:08:32.046 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.521886400, 2024-03-23T18:08:32.046 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T18:08:32.046 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 447439, 2024-03-23T18:08:32.046 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 69.417280690, 2024-03-23T18:08:32.046 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000155143 2024-03-23T18:08:32.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:32.047 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-03-23T18:08:32.047 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 686895, 2024-03-23T18:08:32.047 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.120689605, 2024-03-23T18:08:32.047 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000175 2024-03-23T18:08:32.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:32.047 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-23T18:08:32.047 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 0 2024-03-23T18:08:32.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:32.048 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-1": { 2024-03-23T18:08:32.048 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 47335, 2024-03-23T18:08:32.048 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 56519, 2024-03-23T18:08:32.048 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 11618324, 2024-03-23T18:08:32.048 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 155541410, 2024-03-23T18:08:32.048 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 16, 2024-03-23T18:08:32.048 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 7, 2024-03-23T18:08:32.048 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 3.401866844, 2024-03-23T18:08:32.049 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 1.203897580, 2024-03-23T18:08:32.049 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 1.449540281, 2024-03-23T18:08:32.049 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.807722091, 2024-03-23T18:08:32.049 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T18:08:32.049 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 56516, 2024-03-23T18:08:32.049 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 11.026838110, 2024-03-23T18:08:32.049 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000195110 2024-03-23T18:08:32.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:32.050 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-03-23T18:08:32.050 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 57039, 2024-03-23T18:08:32.050 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.011232390, 2024-03-23T18:08:32.050 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000196 2024-03-23T18:08:32.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:32.050 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 1392, 2024-03-23T18:08:32.050 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 664592 2024-03-23T18:08:32.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:32.050 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-2": { 2024-03-23T18:08:32.051 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 31471, 2024-03-23T18:08:32.051 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 35183, 2024-03-23T18:08:32.051 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 12019837, 2024-03-23T18:08:32.051 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 142577484, 2024-03-23T18:08:32.051 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 16, 2024-03-23T18:08:32.051 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 9, 2024-03-23T18:08:32.051 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 2.496442653, 2024-03-23T18:08:32.051 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.875057063, 2024-03-23T18:08:32.051 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.872143336, 2024-03-23T18:08:32.052 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.749168117, 2024-03-23T18:08:32.052 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T18:08:32.052 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 35179, 2024-03-23T18:08:32.052 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 8.373401489, 2024-03-23T18:08:32.052 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000238022 2024-03-23T18:08:32.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:32.052 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-03-23T18:08:32.052 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 26380, 2024-03-23T18:08:32.052 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.005333240, 2024-03-23T18:08:32.053 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000202 2024-03-23T18:08:32.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:32.053 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 2035520, 2024-03-23T18:08:32.053 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 225696 2024-03-23T18:08:32.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:32.053 INFO:teuthology.orchestra.run.smithi092.stdout: "cct": { 2024-03-23T18:08:32.053 INFO:teuthology.orchestra.run.smithi092.stdout: "total_workers": 1, 2024-03-23T18:08:32.053 INFO:teuthology.orchestra.run.smithi092.stdout: "unhealthy_workers": 0 2024-03-23T18:08:32.054 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:32.054 INFO:teuthology.orchestra.run.smithi092.stdout: "finisher-MDSRank": { 2024-03-23T18:08:32.054 INFO:teuthology.orchestra.run.smithi092.stdout: "queue_len": 0, 2024-03-23T18:08:32.054 INFO:teuthology.orchestra.run.smithi092.stdout: "complete_latency": { 2024-03-23T18:08:32.054 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 62346, 2024-03-23T18:08:32.054 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 68.669173002, 2024-03-23T18:08:32.054 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.001101420 2024-03-23T18:08:32.054 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-23T18:08:32.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:32.055 INFO:teuthology.orchestra.run.smithi092.stdout: "finisher-PurgeQueue": { 2024-03-23T18:08:32.055 INFO:teuthology.orchestra.run.smithi092.stdout: "queue_len": 0, 2024-03-23T18:08:32.055 INFO:teuthology.orchestra.run.smithi092.stdout: "complete_latency": { 2024-03-23T18:08:32.055 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 34104, 2024-03-23T18:08:32.055 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 4.904637585, 2024-03-23T18:08:32.055 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000143814 2024-03-23T18:08:32.055 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-23T18:08:32.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:32.056 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": { 2024-03-23T18:08:32.056 INFO:teuthology.orchestra.run.smithi092.stdout: "request": 124341, 2024-03-23T18:08:32.056 INFO:teuthology.orchestra.run.smithi092.stdout: "reply": 116719, 2024-03-23T18:08:32.056 INFO:teuthology.orchestra.run.smithi092.stdout: "reply_latency": { 2024-03-23T18:08:32.056 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 116719, 2024-03-23T18:08:32.056 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 504.833422811, 2024-03-23T18:08:32.056 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.004325203 2024-03-23T18:08:32.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:32.056 INFO:teuthology.orchestra.run.smithi092.stdout: "slow_reply": 0, 2024-03-23T18:08:32.057 INFO:teuthology.orchestra.run.smithi092.stdout: "forward": 7621, 2024-03-23T18:08:32.057 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_fetch_complete": 2, 2024-03-23T18:08:32.057 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_fetch_keys": 0, 2024-03-23T18:08:32.057 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_commit": 63, 2024-03-23T18:08:32.057 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_split": 6, 2024-03-23T18:08:32.057 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_merge": 4, 2024-03-23T18:08:32.057 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes": 1681, 2024-03-23T18:08:32.057 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_top": 459, 2024-03-23T18:08:32.058 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_bottom": 1001, 2024-03-23T18:08:32.058 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_pin_tail": 221, 2024-03-23T18:08:32.058 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_pinned": 1018, 2024-03-23T18:08:32.058 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_expired": 702, 2024-03-23T18:08:32.058 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_with_caps": 186, 2024-03-23T18:08:32.058 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": 186, 2024-03-23T18:08:32.058 INFO:teuthology.orchestra.run.smithi092.stdout: "subtrees": 24, 2024-03-23T18:08:32.058 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse": 211519, 2024-03-23T18:08:32.058 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_hit": 123777, 2024-03-23T18:08:32.059 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_forward": 7606, 2024-03-23T18:08:32.059 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_discover": 76, 2024-03-23T18:08:32.059 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_dir_fetch": 0, 2024-03-23T18:08:32.059 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_remote_ino": 0, 2024-03-23T18:08:32.059 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_lock": 3088, 2024-03-23T18:08:32.059 INFO:teuthology.orchestra.run.smithi092.stdout: "load_cent": 35203, 2024-03-23T18:08:32.059 INFO:teuthology.orchestra.run.smithi092.stdout: "q": 0, 2024-03-23T18:08:32.059 INFO:teuthology.orchestra.run.smithi092.stdout: "exported": 10, 2024-03-23T18:08:32.059 INFO:teuthology.orchestra.run.smithi092.stdout: "exported_inodes": 446, 2024-03-23T18:08:32.060 INFO:teuthology.orchestra.run.smithi092.stdout: "imported": 10, 2024-03-23T18:08:32.060 INFO:teuthology.orchestra.run.smithi092.stdout: "imported_inodes": 756, 2024-03-23T18:08:32.060 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_dir_fetch": 0, 2024-03-23T18:08:32.060 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_backtrace_fetch": 9, 2024-03-23T18:08:32.060 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_peer_discover": 0, 2024-03-23T18:08:32.060 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rfiles": 699, 2024-03-23T18:08:32.060 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rbytes": 211642430, 2024-03-23T18:08:32.060 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rsnaps": 7, 2024-03-23T18:08:32.060 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_backtrace_fetch": 9, 2024-03-23T18:08:32.061 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_set_tag": 0, 2024-03-23T18:08:32.061 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_backtrace_repaired": 0, 2024-03-23T18:08:32.061 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_inotable_repaired": 0, 2024-03-23T18:08:32.061 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dir_inodes": 0, 2024-03-23T18:08:32.061 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dir_base_inodes": 0, 2024-03-23T18:08:32.061 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dirfrag_rstats": 0, 2024-03-23T18:08:32.061 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_file_inodes": 9, 2024-03-23T18:08:32.061 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_inode_file_caps": 14, 2024-03-23T18:08:32.062 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_revoke": 26537, 2024-03-23T18:08:32.062 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_grant": 81557, 2024-03-23T18:08:32.062 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_trunc": 0, 2024-03-23T18:08:32.062 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-23T18:08:32.062 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-23T18:08:32.062 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_caps": 58795, 2024-03-23T18:08:32.063 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_caps_dirty": 20364, 2024-03-23T18:08:32.063 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_cap_release": 4660, 2024-03-23T18:08:32.063 INFO:teuthology.orchestra.run.smithi092.stdout: "process_request_cap_release": 89466 2024-03-23T18:08:32.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:32.063 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_cache": { 2024-03-23T18:08:32.063 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_update": 6, 2024-03-23T18:08:32.063 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_update_receipt": 168, 2024-03-23T18:08:32.063 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_try_discover": 104, 2024-03-23T18:08:32.063 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_send_discover": 197, 2024-03-23T18:08:32.064 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_handle_discover": 112, 2024-03-23T18:08:32.064 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays": 0, 2024-03-23T18:08:32.064 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays_delayed": 0, 2024-03-23T18:08:32.064 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays_enqueuing": 0, 2024-03-23T18:08:32.064 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_created": 19052, 2024-03-23T18:08:32.064 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_enqueued": 19052, 2024-03-23T18:08:32.064 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_reintegrated": 0, 2024-03-23T18:08:32.064 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_migrated": 0, 2024-03-23T18:08:32.064 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_processing": 0, 2024-03-23T18:08:32.065 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_enqueued": 0, 2024-03-23T18:08:32.065 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_prioritized": 0, 2024-03-23T18:08:32.065 INFO:teuthology.orchestra.run.smithi092.stdout: "recovery_started": 0, 2024-03-23T18:08:32.065 INFO:teuthology.orchestra.run.smithi092.stdout: "recovery_completed": 0, 2024-03-23T18:08:32.065 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_enqueue_scrub": 0, 2024-03-23T18:08:32.065 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_exportdir": 13, 2024-03-23T18:08:32.065 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_flush": 0, 2024-03-23T18:08:32.065 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_fragmentdir": 10, 2024-03-23T18:08:32.065 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_fragstats": 0, 2024-03-23T18:08:32.066 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_inodestats": 0 2024-03-23T18:08:32.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:32.066 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_log": { 2024-03-23T18:08:32.066 INFO:teuthology.orchestra.run.smithi092.stdout: "evadd": 107843, 2024-03-23T18:08:32.066 INFO:teuthology.orchestra.run.smithi092.stdout: "evex": 11015, 2024-03-23T18:08:32.066 INFO:teuthology.orchestra.run.smithi092.stdout: "evtrm": 11015, 2024-03-23T18:08:32.066 INFO:teuthology.orchestra.run.smithi092.stdout: "ev": 96828, 2024-03-23T18:08:32.066 INFO:teuthology.orchestra.run.smithi092.stdout: "evexg": 0, 2024-03-23T18:08:32.067 INFO:teuthology.orchestra.run.smithi092.stdout: "evexd": 1024, 2024-03-23T18:08:32.067 INFO:teuthology.orchestra.run.smithi092.stdout: "segadd": 139, 2024-03-23T18:08:32.067 INFO:teuthology.orchestra.run.smithi092.stdout: "segex": 11, 2024-03-23T18:08:32.067 INFO:teuthology.orchestra.run.smithi092.stdout: "segtrm": 11, 2024-03-23T18:08:32.067 INFO:teuthology.orchestra.run.smithi092.stdout: "seg": 129, 2024-03-23T18:08:32.067 INFO:teuthology.orchestra.run.smithi092.stdout: "segexg": 0, 2024-03-23T18:08:32.067 INFO:teuthology.orchestra.run.smithi092.stdout: "segexd": 1, 2024-03-23T18:08:32.067 INFO:teuthology.orchestra.run.smithi092.stdout: "expos": 19761768, 2024-03-23T18:08:32.067 INFO:teuthology.orchestra.run.smithi092.stdout: "wrpos": 402833659, 2024-03-23T18:08:32.068 INFO:teuthology.orchestra.run.smithi092.stdout: "rdpos": 0, 2024-03-23T18:08:32.068 INFO:teuthology.orchestra.run.smithi092.stdout: "jlat": { 2024-03-23T18:08:32.068 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 34792, 2024-03-23T18:08:32.068 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 240.425627888, 2024-03-23T18:08:32.068 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.006910371 2024-03-23T18:08:32.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:32.068 INFO:teuthology.orchestra.run.smithi092.stdout: "replayed": 0 2024-03-23T18:08:32.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:32.068 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_mem": { 2024-03-23T18:08:32.069 INFO:teuthology.orchestra.run.smithi092.stdout: "ino": 894, 2024-03-23T18:08:32.069 INFO:teuthology.orchestra.run.smithi092.stdout: "ino+": 20411, 2024-03-23T18:08:32.069 INFO:teuthology.orchestra.run.smithi092.stdout: "ino-": 19517, 2024-03-23T18:08:32.069 INFO:teuthology.orchestra.run.smithi092.stdout: "dir": 152, 2024-03-23T18:08:32.069 INFO:teuthology.orchestra.run.smithi092.stdout: "dir+": 303, 2024-03-23T18:08:32.069 INFO:teuthology.orchestra.run.smithi092.stdout: "dir-": 151, 2024-03-23T18:08:32.069 INFO:teuthology.orchestra.run.smithi092.stdout: "dn": 1568, 2024-03-23T18:08:32.069 INFO:teuthology.orchestra.run.smithi092.stdout: "dn+": 32743, 2024-03-23T18:08:32.069 INFO:teuthology.orchestra.run.smithi092.stdout: "dn-": 31175, 2024-03-23T18:08:32.070 INFO:teuthology.orchestra.run.smithi092.stdout: "cap": 186, 2024-03-23T18:08:32.070 INFO:teuthology.orchestra.run.smithi092.stdout: "cap+": 20748, 2024-03-23T18:08:32.070 INFO:teuthology.orchestra.run.smithi092.stdout: "cap-": 20562, 2024-03-23T18:08:32.070 INFO:teuthology.orchestra.run.smithi092.stdout: "rss": 76368, 2024-03-23T18:08:32.070 INFO:teuthology.orchestra.run.smithi092.stdout: "heap": 198964 2024-03-23T18:08:32.070 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:32.070 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_server": { 2024-03-23T18:08:32.070 INFO:teuthology.orchestra.run.smithi092.stdout: "dispatch_client_request": 187187, 2024-03-23T18:08:32.071 INFO:teuthology.orchestra.run.smithi092.stdout: "dispatch_server_request": 2826, 2024-03-23T18:08:32.071 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_request": 124341, 2024-03-23T18:08:32.071 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_session": 4377, 2024-03-23T18:08:32.071 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_peer_request": 14340, 2024-03-23T18:08:32.071 INFO:teuthology.orchestra.run.smithi092.stdout: "req_create_latency": { 2024-03-23T18:08:32.071 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 19072, 2024-03-23T18:08:32.071 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 65.364885079, 2024-03-23T18:08:32.071 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.003427269 2024-03-23T18:08:32.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:32.072 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getattr_latency": { 2024-03-23T18:08:32.072 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 5392, 2024-03-23T18:08:32.072 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 93.964931375, 2024-03-23T18:08:32.072 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.017426730 2024-03-23T18:08:32.072 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:32.072 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getfilelock_latency": { 2024-03-23T18:08:32.072 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:32.072 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:32.073 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:32.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:32.073 INFO:teuthology.orchestra.run.smithi092.stdout: "req_link_latency": { 2024-03-23T18:08:32.073 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:32.073 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:32.073 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:32.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:32.073 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookup_latency": { 2024-03-23T18:08:32.073 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 30324, 2024-03-23T18:08:32.074 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 75.761002835, 2024-03-23T18:08:32.074 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.002498384 2024-03-23T18:08:32.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:32.074 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookuphash_latency": { 2024-03-23T18:08:32.074 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:32.074 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:32.074 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:32.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:32.075 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupino_latency": { 2024-03-23T18:08:32.075 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:32.075 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:32.075 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:32.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:32.075 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupname_latency": { 2024-03-23T18:08:32.075 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:32.075 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:32.075 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:32.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:32.076 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupparent_latency": { 2024-03-23T18:08:32.076 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:32.076 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:32.076 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:32.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:32.076 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupsnap_latency": { 2024-03-23T18:08:32.076 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:32.076 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:32.077 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:32.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:32.077 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lssnap_latency": { 2024-03-23T18:08:32.077 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:32.077 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:32.077 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:32.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:32.078 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mkdir_latency": { 2024-03-23T18:08:32.078 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:32.078 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:32.078 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:32.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:32.078 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mknod_latency": { 2024-03-23T18:08:32.078 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:32.078 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:32.078 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:32.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:32.079 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mksnap_latency": { 2024-03-23T18:08:32.079 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:32.079 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:32.079 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:32.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:32.079 INFO:teuthology.orchestra.run.smithi092.stdout: "req_open_latency": { 2024-03-23T18:08:32.079 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 1, 2024-03-23T18:08:32.079 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000472914, 2024-03-23T18:08:32.080 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000472914 2024-03-23T18:08:32.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:32.080 INFO:teuthology.orchestra.run.smithi092.stdout: "req_readdir_latency": { 2024-03-23T18:08:32.080 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 36668, 2024-03-23T18:08:32.080 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 198.441433564, 2024-03-23T18:08:32.080 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.005411842 2024-03-23T18:08:32.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:32.080 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rename_latency": { 2024-03-23T18:08:32.080 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 4851, 2024-03-23T18:08:32.081 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 16.706987635, 2024-03-23T18:08:32.081 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.003444029 2024-03-23T18:08:32.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:32.081 INFO:teuthology.orchestra.run.smithi092.stdout: "req_renamesnap_latency": { 2024-03-23T18:08:32.081 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:32.081 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:32.081 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:32.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:32.082 INFO:teuthology.orchestra.run.smithi092.stdout: "req_snapdiff_latency": { 2024-03-23T18:08:32.082 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:32.082 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:32.082 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:32.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:32.082 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmdir_latency": { 2024-03-23T18:08:32.082 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:32.082 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:32.082 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:32.083 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:32.083 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmsnap_latency": { 2024-03-23T18:08:32.083 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:32.083 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:32.083 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:32.083 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:32.083 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmxattr_latency": { 2024-03-23T18:08:32.083 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:32.083 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:32.084 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:32.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:32.084 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setattr_latency": { 2024-03-23T18:08:32.084 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 39, 2024-03-23T18:08:32.084 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.633892067, 2024-03-23T18:08:32.084 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.016253642 2024-03-23T18:08:32.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:32.084 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setdirlayout_latency": { 2024-03-23T18:08:32.084 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:32.085 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:32.085 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:32.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:32.085 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setfilelock_latency": { 2024-03-23T18:08:32.085 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 1320, 2024-03-23T18:08:32.085 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 1.033726100, 2024-03-23T18:08:32.085 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000783125 2024-03-23T18:08:32.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:32.085 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setlayout_latency": { 2024-03-23T18:08:32.086 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:32.086 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:32.086 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:32.086 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:32.086 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setxattr_latency": { 2024-03-23T18:08:32.086 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:32.086 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:32.086 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:32.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:32.087 INFO:teuthology.orchestra.run.smithi092.stdout: "req_symlink_latency": { 2024-03-23T18:08:32.087 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:32.087 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:32.087 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:32.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:32.087 INFO:teuthology.orchestra.run.smithi092.stdout: "req_unlink_latency": { 2024-03-23T18:08:32.087 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 19052, 2024-03-23T18:08:32.087 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 52.926091242, 2024-03-23T18:08:32.088 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.002777980 2024-03-23T18:08:32.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:32.088 INFO:teuthology.orchestra.run.smithi092.stdout: "cap_revoke_eviction": 0, 2024-03-23T18:08:32.088 INFO:teuthology.orchestra.run.smithi092.stdout: "cap_acquisition_throttle": 0, 2024-03-23T18:08:32.088 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getvxattr_latency": { 2024-03-23T18:08:32.088 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:32.088 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:32.088 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:32.088 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-23T18:08:32.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:32.089 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_sessions": { 2024-03-23T18:08:32.089 INFO:teuthology.orchestra.run.smithi092.stdout: "session_count": 2, 2024-03-23T18:08:32.089 INFO:teuthology.orchestra.run.smithi092.stdout: "session_add": 4, 2024-03-23T18:08:32.089 INFO:teuthology.orchestra.run.smithi092.stdout: "session_remove": 2, 2024-03-23T18:08:32.089 INFO:teuthology.orchestra.run.smithi092.stdout: "sessions_open": 2, 2024-03-23T18:08:32.089 INFO:teuthology.orchestra.run.smithi092.stdout: "sessions_stale": 0, 2024-03-23T18:08:32.089 INFO:teuthology.orchestra.run.smithi092.stdout: "total_load": 24486, 2024-03-23T18:08:32.089 INFO:teuthology.orchestra.run.smithi092.stdout: "average_load": 12243, 2024-03-23T18:08:32.090 INFO:teuthology.orchestra.run.smithi092.stdout: "avg_session_uptime": 1030, 2024-03-23T18:08:32.090 INFO:teuthology.orchestra.run.smithi092.stdout: "mdthresh_evicted": 0 2024-03-23T18:08:32.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:32.090 INFO:teuthology.orchestra.run.smithi092.stdout: "mempool": { 2024-03-23T18:08:32.090 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_bytes": 240, 2024-03-23T18:08:32.090 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_items": 240, 2024-03-23T18:08:32.091 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_bytes": 0, 2024-03-23T18:08:32.091 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_items": 0, 2024-03-23T18:08:32.091 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_bytes": 0, 2024-03-23T18:08:32.091 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_items": 0, 2024-03-23T18:08:32.091 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-23T18:08:32.091 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_items": 0, 2024-03-23T18:08:32.091 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-23T18:08:32.091 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_items": 0, 2024-03-23T18:08:32.091 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_bytes": 0, 2024-03-23T18:08:32.092 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_items": 0, 2024-03-23T18:08:32.092 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-23T18:08:32.092 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_items": 0, 2024-03-23T18:08:32.092 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_bytes": 0, 2024-03-23T18:08:32.092 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_items": 0, 2024-03-23T18:08:32.092 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_bytes": 0, 2024-03-23T18:08:32.092 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_items": 0, 2024-03-23T18:08:32.092 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-23T18:08:32.092 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_items": 0, 2024-03-23T18:08:32.093 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-23T18:08:32.093 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_items": 0, 2024-03-23T18:08:32.093 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_bytes": 0, 2024-03-23T18:08:32.093 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_items": 0, 2024-03-23T18:08:32.093 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_bytes": 0, 2024-03-23T18:08:32.093 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_items": 0, 2024-03-23T18:08:32.093 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-23T18:08:32.093 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_items": 0, 2024-03-23T18:08:32.093 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_bytes": 0, 2024-03-23T18:08:32.094 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_items": 0, 2024-03-23T18:08:32.094 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_bytes": 0, 2024-03-23T18:08:32.094 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_items": 0, 2024-03-23T18:08:32.094 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_bytes": 0, 2024-03-23T18:08:32.094 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_items": 0, 2024-03-23T18:08:32.094 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_bytes": 0, 2024-03-23T18:08:32.094 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_items": 0, 2024-03-23T18:08:32.094 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_bytes": 1492200, 2024-03-23T18:08:32.095 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_items": 1027, 2024-03-23T18:08:32.095 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_bytes": 88, 2024-03-23T18:08:32.095 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_items": 1, 2024-03-23T18:08:32.095 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_bytes": 0, 2024-03-23T18:08:32.095 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_items": 0, 2024-03-23T18:08:32.095 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_bytes": 0, 2024-03-23T18:08:32.095 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_items": 0, 2024-03-23T18:08:32.095 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_bytes": 0, 2024-03-23T18:08:32.095 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_items": 0, 2024-03-23T18:08:32.096 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_bytes": 7936, 2024-03-23T18:08:32.096 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_items": 165, 2024-03-23T18:08:32.096 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_bytes": 0, 2024-03-23T18:08:32.096 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_items": 0, 2024-03-23T18:08:32.096 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_bytes": 0, 2024-03-23T18:08:32.096 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_items": 0, 2024-03-23T18:08:32.096 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_bytes": 4126555, 2024-03-23T18:08:32.096 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_items": 50428, 2024-03-23T18:08:32.096 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_bytes": 0, 2024-03-23T18:08:32.097 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_items": 0, 2024-03-23T18:08:32.097 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_bytes": 0, 2024-03-23T18:08:32.097 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_items": 0 2024-03-23T18:08:32.097 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:32.097 INFO:teuthology.orchestra.run.smithi092.stdout: "objecter": { 2024-03-23T18:08:32.097 INFO:teuthology.orchestra.run.smithi092.stdout: "op_active": 0, 2024-03-23T18:08:32.097 INFO:teuthology.orchestra.run.smithi092.stdout: "op_laggy": 0, 2024-03-23T18:08:32.097 INFO:teuthology.orchestra.run.smithi092.stdout: "op_send": 56519, 2024-03-23T18:08:32.097 INFO:teuthology.orchestra.run.smithi092.stdout: "op_send_bytes": 402277322, 2024-03-23T18:08:32.098 INFO:teuthology.orchestra.run.smithi092.stdout: "op_resend": 23, 2024-03-23T18:08:32.098 INFO:teuthology.orchestra.run.smithi092.stdout: "op_reply": 56519, 2024-03-23T18:08:32.098 INFO:teuthology.orchestra.run.smithi092.stdout: "op_latency": { 2024-03-23T18:08:32.098 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 56519, 2024-03-23T18:08:32.098 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 396.301642245, 2024-03-23T18:08:32.098 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.007011830 2024-03-23T18:08:32.098 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:32.098 INFO:teuthology.orchestra.run.smithi092.stdout: "op_inflight": 0, 2024-03-23T18:08:32.098 INFO:teuthology.orchestra.run.smithi092.stdout: "oplen_avg": { 2024-03-23T18:08:32.099 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 56519, 2024-03-23T18:08:32.099 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 57016 2024-03-23T18:08:32.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:32.099 INFO:teuthology.orchestra.run.smithi092.stdout: "op": 56519, 2024-03-23T18:08:32.099 INFO:teuthology.orchestra.run.smithi092.stdout: "op_r": 870, 2024-03-23T18:08:32.099 INFO:teuthology.orchestra.run.smithi092.stdout: "op_w": 55649, 2024-03-23T18:08:32.099 INFO:teuthology.orchestra.run.smithi092.stdout: "op_rmw": 0, 2024-03-23T18:08:32.099 INFO:teuthology.orchestra.run.smithi092.stdout: "op_pg": 0, 2024-03-23T18:08:32.100 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_stat": 0, 2024-03-23T18:08:32.100 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_create": 21, 2024-03-23T18:08:32.100 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_read": 859, 2024-03-23T18:08:32.100 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_write": 35651, 2024-03-23T18:08:32.100 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_writefull": 511, 2024-03-23T18:08:32.100 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_writesame": 0, 2024-03-23T18:08:32.100 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_append": 0, 2024-03-23T18:08:32.100 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_zero": 0, 2024-03-23T18:08:32.100 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_truncate": 6, 2024-03-23T18:08:32.101 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_delete": 19194, 2024-03-23T18:08:32.101 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_mapext": 0, 2024-03-23T18:08:32.101 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_sparse_read": 0, 2024-03-23T18:08:32.101 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_clonerange": 0, 2024-03-23T18:08:32.101 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_getxattr": 9, 2024-03-23T18:08:32.101 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_setxattr": 42, 2024-03-23T18:08:32.101 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_cmpxattr": 0, 2024-03-23T18:08:32.101 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_rmxattr": 0, 2024-03-23T18:08:32.101 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_resetxattrs": 0, 2024-03-23T18:08:32.102 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_call": 0, 2024-03-23T18:08:32.102 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_watch": 0, 2024-03-23T18:08:32.102 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_notify": 0, 2024-03-23T18:08:32.102 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_src_cmpxattr": 0, 2024-03-23T18:08:32.102 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_pgls": 0, 2024-03-23T18:08:32.102 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_pgls_filter": 0, 2024-03-23T18:08:32.102 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_other": 0, 2024-03-23T18:08:32.102 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_active": 0, 2024-03-23T18:08:32.102 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_send": 0, 2024-03-23T18:08:32.103 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_resend": 0, 2024-03-23T18:08:32.103 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_ping": 0, 2024-03-23T18:08:32.103 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_active": 0, 2024-03-23T18:08:32.103 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_send": 0, 2024-03-23T18:08:32.103 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_resend": 0, 2024-03-23T18:08:32.103 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_active": 0, 2024-03-23T18:08:32.103 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_send": 0, 2024-03-23T18:08:32.103 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_resend": 0, 2024-03-23T18:08:32.103 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_active": 0, 2024-03-23T18:08:32.104 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_send": 0, 2024-03-23T18:08:32.104 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_resend": 0, 2024-03-23T18:08:32.104 INFO:teuthology.orchestra.run.smithi092.stdout: "command_active": 0, 2024-03-23T18:08:32.104 INFO:teuthology.orchestra.run.smithi092.stdout: "command_send": 0, 2024-03-23T18:08:32.104 INFO:teuthology.orchestra.run.smithi092.stdout: "command_resend": 0, 2024-03-23T18:08:32.104 INFO:teuthology.orchestra.run.smithi092.stdout: "map_epoch": 110, 2024-03-23T18:08:32.104 INFO:teuthology.orchestra.run.smithi092.stdout: "map_full": 0, 2024-03-23T18:08:32.104 INFO:teuthology.orchestra.run.smithi092.stdout: "map_inc": 30, 2024-03-23T18:08:32.104 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_sessions": 12, 2024-03-23T18:08:32.105 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_session_open": 12, 2024-03-23T18:08:32.105 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_session_close": 0, 2024-03-23T18:08:32.105 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_laggy": 0, 2024-03-23T18:08:32.105 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_wr": 514, 2024-03-23T18:08:32.105 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_rd": 4, 2024-03-23T18:08:32.105 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_del": 205 2024-03-23T18:08:32.105 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:32.105 INFO:teuthology.orchestra.run.smithi092.stdout: "oft": { 2024-03-23T18:08:32.106 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_objs": 1, 2024-03-23T18:08:32.106 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_kv_pairs": 0, 2024-03-23T18:08:32.106 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_updates": 0, 2024-03-23T18:08:32.106 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_removes": 0 2024-03-23T18:08:32.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:32.106 INFO:teuthology.orchestra.run.smithi092.stdout: "osd-slow-ops": { 2024-03-23T18:08:32.106 INFO:teuthology.orchestra.run.smithi092.stdout: "slow_ops_count": 126916 2024-03-23T18:08:32.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:32.106 INFO:teuthology.orchestra.run.smithi092.stdout: "purge_queue": { 2024-03-23T18:08:32.107 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_ops": 0, 2024-03-23T18:08:32.107 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_ops_high_water": 6, 2024-03-23T18:08:32.107 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing": 0, 2024-03-23T18:08:32.107 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_high_water": 6, 2024-03-23T18:08:32.107 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executed": 19052, 2024-03-23T18:08:32.107 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_item_in_journal": 0 2024-03-23T18:08:32.107 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:32.107 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-23T18:08:32.107 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-03-23T18:08:32.108 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 104857600, 2024-03-23T18:08:32.108 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-03-23T18:08:32.108 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 454956, 2024-03-23T18:08:32.108 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 105716845, 2024-03-23T18:08:32.108 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-03-23T18:08:32.108 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 454956, 2024-03-23T18:08:32.108 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-03-23T18:08:32.108 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-03-23T18:08:32.108 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 454956, 2024-03-23T18:08:32.109 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 105716845, 2024-03-23T18:08:32.109 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-03-23T18:08:32.109 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:32.109 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:32.109 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:32.109 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-23T18:08:32.109 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:32.109 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-objecter_bytes": { 2024-03-23T18:08:32.109 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-03-23T18:08:32.110 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 104857600, 2024-03-23T18:08:32.110 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-03-23T18:08:32.110 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-03-23T18:08:32.110 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-03-23T18:08:32.110 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-03-23T18:08:32.110 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-03-23T18:08:32.110 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 56519, 2024-03-23T18:08:32.110 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 404203726, 2024-03-23T18:08:32.111 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 56519, 2024-03-23T18:08:32.111 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 404203726, 2024-03-23T18:08:32.111 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-03-23T18:08:32.111 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:32.111 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:32.111 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:32.112 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-23T18:08:32.112 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:32.112 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-objecter_ops": { 2024-03-23T18:08:32.112 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-03-23T18:08:32.112 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 1024, 2024-03-23T18:08:32.112 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-03-23T18:08:32.112 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-03-23T18:08:32.112 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-03-23T18:08:32.112 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-03-23T18:08:32.113 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-03-23T18:08:32.113 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 56519, 2024-03-23T18:08:32.113 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 56519, 2024-03-23T18:08:32.113 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 56519, 2024-03-23T18:08:32.113 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 56519, 2024-03-23T18:08:32.113 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-03-23T18:08:32.113 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:32.113 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:32.113 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:32.114 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-23T18:08:32.114 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:32.114 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-write_buf_throttle": { 2024-03-23T18:08:32.114 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-03-23T18:08:32.114 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 3758096384, 2024-03-23T18:08:32.114 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-03-23T18:08:32.114 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 19052, 2024-03-23T18:08:32.114 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 1926436, 2024-03-23T18:08:32.115 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-03-23T18:08:32.115 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 19052, 2024-03-23T18:08:32.115 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-03-23T18:08:32.115 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-03-23T18:08:32.115 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 859, 2024-03-23T18:08:32.115 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 1926436, 2024-03-23T18:08:32.115 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-03-23T18:08:32.115 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:32.115 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:32.116 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:32.116 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-23T18:08:32.116 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:32.116 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-write_buf_throttle-0x5605644d20a0": { 2024-03-23T18:08:32.116 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 5397, 2024-03-23T18:08:32.116 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 3758096384, 2024-03-23T18:08:32.116 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-03-23T18:08:32.116 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 107843, 2024-03-23T18:08:32.116 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 398639355, 2024-03-23T18:08:32.117 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-03-23T18:08:32.117 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 107843, 2024-03-23T18:08:32.117 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-03-23T18:08:32.117 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-03-23T18:08:32.117 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 34792, 2024-03-23T18:08:32.117 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 398633958, 2024-03-23T18:08:32.117 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-03-23T18:08:32.117 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:32.117 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:32.118 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:32.118 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-23T18:08:32.118 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-23T18:08:32.118 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-03-23T18:08:32.205 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:32 smithi092 ceph-mon[79089]: pgmap v566: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 8.7 MiB/s wr, 277 op/s 2024-03-23T18:08:32.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:32 smithi152 ceph-mon[47949]: pgmap v566: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 8.7 MiB/s wr, 277 op/s 2024-03-23T18:08:32.402 INFO:tasks.check_counter:Daemon mds.h mds.exported=10 2024-03-23T18:08:32.402 INFO:tasks.check_counter:Daemon mds.h mds.imported=10 2024-03-23T18:08:32.402 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=7 2024-03-23T18:08:32.402 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-03-23T18:08:32.402 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-03-23T18:08:32.403 DEBUG:tasks.check_counter:Getting stats from k 2024-03-23T18:08:32.403 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph daemon mds.k perf dump 2024-03-23T18:08:32.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:32 smithi086 ceph-mon[38221]: pgmap v566: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 8.7 MiB/s wr, 277 op/s 2024-03-23T18:08:32.623 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.b/config 2024-03-23T18:08:33.553 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-03-23T18:08:33.553 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-0": { 2024-03-23T18:08:33.553 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 2291, 2024-03-23T18:08:33.553 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 2293, 2024-03-23T18:08:33.554 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 134896363, 2024-03-23T18:08:33.554 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 712502, 2024-03-23T18:08:33.554 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 8, 2024-03-23T18:08:33.554 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 4, 2024-03-23T18:08:33.554 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.241827496, 2024-03-23T18:08:33.554 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.055312793, 2024-03-23T18:08:33.554 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.112884449, 2024-03-23T18:08:33.554 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.056910877, 2024-03-23T18:08:33.555 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T18:08:33.555 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 2291, 2024-03-23T18:08:33.555 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.134541108, 2024-03-23T18:08:33.555 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000058725 2024-03-23T18:08:33.555 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:33.555 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-03-23T18:08:33.555 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:33.555 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:33.555 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:33.556 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:33.556 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-23T18:08:33.556 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 0 2024-03-23T18:08:33.556 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:33.556 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-1": { 2024-03-23T18:08:33.556 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 1680, 2024-03-23T18:08:33.556 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 1921, 2024-03-23T18:08:33.556 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 150911569, 2024-03-23T18:08:33.556 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 1099562, 2024-03-23T18:08:33.557 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 9, 2024-03-23T18:08:33.557 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 7, 2024-03-23T18:08:33.557 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.209052160, 2024-03-23T18:08:33.557 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.035130997, 2024-03-23T18:08:33.557 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.121670818, 2024-03-23T18:08:33.557 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.040245304, 2024-03-23T18:08:33.557 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T18:08:33.557 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 1919, 2024-03-23T18:08:33.558 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.084495334, 2024-03-23T18:08:33.558 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000044030 2024-03-23T18:08:33.558 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:33.558 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-03-23T18:08:33.558 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:33.558 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:33.558 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:33.558 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:33.558 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-03-23T18:08:33.559 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 578880 2024-03-23T18:08:33.559 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:33.559 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-2": { 2024-03-23T18:08:33.559 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 1044, 2024-03-23T18:08:33.559 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 970, 2024-03-23T18:08:33.559 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 94432831, 2024-03-23T18:08:33.559 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 380858, 2024-03-23T18:08:33.559 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 5, 2024-03-23T18:08:33.559 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 4, 2024-03-23T18:08:33.560 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.145189834, 2024-03-23T18:08:33.560 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.028028030, 2024-03-23T18:08:33.560 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.102477524, 2024-03-23T18:08:33.560 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.022321477, 2024-03-23T18:08:33.560 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T18:08:33.560 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 969, 2024-03-23T18:08:33.560 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.058625580, 2024-03-23T18:08:33.560 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000060501 2024-03-23T18:08:33.561 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:33.561 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-03-23T18:08:33.561 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:33.561 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:33.561 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:33.561 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:33.561 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 2001984, 2024-03-23T18:08:33.561 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 189904 2024-03-23T18:08:33.561 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:33.562 INFO:teuthology.orchestra.run.smithi092.stdout: "cct": { 2024-03-23T18:08:33.562 INFO:teuthology.orchestra.run.smithi092.stdout: "total_workers": 1, 2024-03-23T18:08:33.562 INFO:teuthology.orchestra.run.smithi092.stdout: "unhealthy_workers": 0 2024-03-23T18:08:33.562 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:33.562 INFO:teuthology.orchestra.run.smithi092.stdout: "finisher-MDSRank": { 2024-03-23T18:08:33.562 INFO:teuthology.orchestra.run.smithi092.stdout: "queue_len": 0, 2024-03-23T18:08:33.562 INFO:teuthology.orchestra.run.smithi092.stdout: "complete_latency": { 2024-03-23T18:08:33.562 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 7584, 2024-03-23T18:08:33.562 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.159064588, 2024-03-23T18:08:33.563 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000020973 2024-03-23T18:08:33.563 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-23T18:08:33.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:33.563 INFO:teuthology.orchestra.run.smithi092.stdout: "finisher-PurgeQueue": { 2024-03-23T18:08:33.563 INFO:teuthology.orchestra.run.smithi092.stdout: "queue_len": 0, 2024-03-23T18:08:33.563 INFO:teuthology.orchestra.run.smithi092.stdout: "complete_latency": { 2024-03-23T18:08:33.563 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:33.563 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:33.563 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:33.564 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-23T18:08:33.564 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:33.564 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": { 2024-03-23T18:08:33.564 INFO:teuthology.orchestra.run.smithi092.stdout: "request": 0, 2024-03-23T18:08:33.564 INFO:teuthology.orchestra.run.smithi092.stdout: "reply": 0, 2024-03-23T18:08:33.564 INFO:teuthology.orchestra.run.smithi092.stdout: "reply_latency": { 2024-03-23T18:08:33.564 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:33.564 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:33.564 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:33.565 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:33.565 INFO:teuthology.orchestra.run.smithi092.stdout: "slow_reply": 0, 2024-03-23T18:08:33.565 INFO:teuthology.orchestra.run.smithi092.stdout: "forward": 0, 2024-03-23T18:08:33.565 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_fetch_complete": 0, 2024-03-23T18:08:33.565 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_fetch_keys": 0, 2024-03-23T18:08:33.565 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_commit": 0, 2024-03-23T18:08:33.565 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_split": 0, 2024-03-23T18:08:33.565 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_merge": 0, 2024-03-23T18:08:33.565 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes": 18573, 2024-03-23T18:08:33.566 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_top": 32, 2024-03-23T18:08:33.566 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_bottom": 18541, 2024-03-23T18:08:33.566 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_pin_tail": 0, 2024-03-23T18:08:33.566 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_pinned": 18527, 2024-03-23T18:08:33.566 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_expired": 0, 2024-03-23T18:08:33.566 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_with_caps": 0, 2024-03-23T18:08:33.566 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": 0, 2024-03-23T18:08:33.566 INFO:teuthology.orchestra.run.smithi092.stdout: "subtrees": 8, 2024-03-23T18:08:33.567 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse": 0, 2024-03-23T18:08:33.567 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_hit": 0, 2024-03-23T18:08:33.567 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_forward": 0, 2024-03-23T18:08:33.567 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_discover": 0, 2024-03-23T18:08:33.567 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_dir_fetch": 0, 2024-03-23T18:08:33.567 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_remote_ino": 0, 2024-03-23T18:08:33.567 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_lock": 0, 2024-03-23T18:08:33.567 INFO:teuthology.orchestra.run.smithi092.stdout: "load_cent": 0, 2024-03-23T18:08:33.567 INFO:teuthology.orchestra.run.smithi092.stdout: "q": 0, 2024-03-23T18:08:33.568 INFO:teuthology.orchestra.run.smithi092.stdout: "exported": 0, 2024-03-23T18:08:33.568 INFO:teuthology.orchestra.run.smithi092.stdout: "exported_inodes": 0, 2024-03-23T18:08:33.568 INFO:teuthology.orchestra.run.smithi092.stdout: "imported": 0, 2024-03-23T18:08:33.568 INFO:teuthology.orchestra.run.smithi092.stdout: "imported_inodes": 0, 2024-03-23T18:08:33.568 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_dir_fetch": 0, 2024-03-23T18:08:33.568 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_backtrace_fetch": 0, 2024-03-23T18:08:33.568 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_peer_discover": 0, 2024-03-23T18:08:33.568 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rfiles": 0, 2024-03-23T18:08:33.568 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rbytes": 0, 2024-03-23T18:08:33.569 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rsnaps": 0, 2024-03-23T18:08:33.569 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_backtrace_fetch": 0, 2024-03-23T18:08:33.569 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_set_tag": 0, 2024-03-23T18:08:33.569 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_backtrace_repaired": 0, 2024-03-23T18:08:33.569 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_inotable_repaired": 0, 2024-03-23T18:08:33.569 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dir_inodes": 0, 2024-03-23T18:08:33.569 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dir_base_inodes": 0, 2024-03-23T18:08:33.569 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dirfrag_rstats": 0, 2024-03-23T18:08:33.569 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_file_inodes": 0, 2024-03-23T18:08:33.570 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_inode_file_caps": 0, 2024-03-23T18:08:33.570 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_revoke": 0, 2024-03-23T18:08:33.570 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_grant": 0, 2024-03-23T18:08:33.570 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_trunc": 0, 2024-03-23T18:08:33.570 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-23T18:08:33.570 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-23T18:08:33.570 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_caps": 0, 2024-03-23T18:08:33.570 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_caps_dirty": 0, 2024-03-23T18:08:33.570 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_cap_release": 0, 2024-03-23T18:08:33.571 INFO:teuthology.orchestra.run.smithi092.stdout: "process_request_cap_release": 0 2024-03-23T18:08:33.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:33.571 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_cache": { 2024-03-23T18:08:33.571 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_update": 0, 2024-03-23T18:08:33.571 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_update_receipt": 0, 2024-03-23T18:08:33.571 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_try_discover": 0, 2024-03-23T18:08:33.571 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_send_discover": 0, 2024-03-23T18:08:33.571 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_handle_discover": 0, 2024-03-23T18:08:33.571 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays": 0, 2024-03-23T18:08:33.572 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays_delayed": 0, 2024-03-23T18:08:33.572 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays_enqueuing": 0, 2024-03-23T18:08:33.572 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_created": 0, 2024-03-23T18:08:33.572 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_enqueued": 0, 2024-03-23T18:08:33.572 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_reintegrated": 0, 2024-03-23T18:08:33.572 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_migrated": 0, 2024-03-23T18:08:33.572 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_processing": 0, 2024-03-23T18:08:33.572 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_enqueued": 0, 2024-03-23T18:08:33.572 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_prioritized": 0, 2024-03-23T18:08:33.573 INFO:teuthology.orchestra.run.smithi092.stdout: "recovery_started": 0, 2024-03-23T18:08:33.573 INFO:teuthology.orchestra.run.smithi092.stdout: "recovery_completed": 0, 2024-03-23T18:08:33.573 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_enqueue_scrub": 0, 2024-03-23T18:08:33.573 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_exportdir": 0, 2024-03-23T18:08:33.573 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_flush": 0, 2024-03-23T18:08:33.573 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_fragmentdir": 0, 2024-03-23T18:08:33.573 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_fragstats": 0, 2024-03-23T18:08:33.573 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_inodestats": 0 2024-03-23T18:08:33.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:33.574 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_log": { 2024-03-23T18:08:33.574 INFO:teuthology.orchestra.run.smithi092.stdout: "evadd": 0, 2024-03-23T18:08:33.574 INFO:teuthology.orchestra.run.smithi092.stdout: "evex": 0, 2024-03-23T18:08:33.574 INFO:teuthology.orchestra.run.smithi092.stdout: "evtrm": 0, 2024-03-23T18:08:33.574 INFO:teuthology.orchestra.run.smithi092.stdout: "ev": 98289, 2024-03-23T18:08:33.574 INFO:teuthology.orchestra.run.smithi092.stdout: "evexg": 0, 2024-03-23T18:08:33.574 INFO:teuthology.orchestra.run.smithi092.stdout: "evexd": 0, 2024-03-23T18:08:33.574 INFO:teuthology.orchestra.run.smithi092.stdout: "segadd": 0, 2024-03-23T18:08:33.575 INFO:teuthology.orchestra.run.smithi092.stdout: "segex": 0, 2024-03-23T18:08:33.575 INFO:teuthology.orchestra.run.smithi092.stdout: "segtrm": 0, 2024-03-23T18:08:33.575 INFO:teuthology.orchestra.run.smithi092.stdout: "seg": 128, 2024-03-23T18:08:33.575 INFO:teuthology.orchestra.run.smithi092.stdout: "segexg": 0, 2024-03-23T18:08:33.575 INFO:teuthology.orchestra.run.smithi092.stdout: "segexd": 0, 2024-03-23T18:08:33.575 INFO:teuthology.orchestra.run.smithi092.stdout: "expos": 4194304, 2024-03-23T18:08:33.575 INFO:teuthology.orchestra.run.smithi092.stdout: "wrpos": 381165473, 2024-03-23T18:08:33.575 INFO:teuthology.orchestra.run.smithi092.stdout: "rdpos": 381164874, 2024-03-23T18:08:33.575 INFO:teuthology.orchestra.run.smithi092.stdout: "jlat": { 2024-03-23T18:08:33.576 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:33.576 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:33.576 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:33.576 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:33.576 INFO:teuthology.orchestra.run.smithi092.stdout: "replayed": 98289 2024-03-23T18:08:33.576 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:33.576 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_mem": { 2024-03-23T18:08:33.576 INFO:teuthology.orchestra.run.smithi092.stdout: "ino": 593, 2024-03-23T18:08:33.576 INFO:teuthology.orchestra.run.smithi092.stdout: "ino+": 18679, 2024-03-23T18:08:33.577 INFO:teuthology.orchestra.run.smithi092.stdout: "ino-": 18086, 2024-03-23T18:08:33.577 INFO:teuthology.orchestra.run.smithi092.stdout: "dir": 55, 2024-03-23T18:08:33.577 INFO:teuthology.orchestra.run.smithi092.stdout: "dir+": 123, 2024-03-23T18:08:33.577 INFO:teuthology.orchestra.run.smithi092.stdout: "dir-": 68, 2024-03-23T18:08:33.577 INFO:teuthology.orchestra.run.smithi092.stdout: "dn": 18573, 2024-03-23T18:08:33.577 INFO:teuthology.orchestra.run.smithi092.stdout: "dn+": 18854, 2024-03-23T18:08:33.577 INFO:teuthology.orchestra.run.smithi092.stdout: "dn-": 281, 2024-03-23T18:08:33.577 INFO:teuthology.orchestra.run.smithi092.stdout: "cap": 0, 2024-03-23T18:08:33.577 INFO:teuthology.orchestra.run.smithi092.stdout: "cap+": 0, 2024-03-23T18:08:33.578 INFO:teuthology.orchestra.run.smithi092.stdout: "cap-": 0, 2024-03-23T18:08:33.578 INFO:teuthology.orchestra.run.smithi092.stdout: "rss": 71420, 2024-03-23T18:08:33.578 INFO:teuthology.orchestra.run.smithi092.stdout: "heap": 207156 2024-03-23T18:08:33.578 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:33.578 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_server": { 2024-03-23T18:08:33.578 INFO:teuthology.orchestra.run.smithi092.stdout: "dispatch_client_request": 0, 2024-03-23T18:08:33.578 INFO:teuthology.orchestra.run.smithi092.stdout: "dispatch_server_request": 0, 2024-03-23T18:08:33.578 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_request": 0, 2024-03-23T18:08:33.578 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_session": 0, 2024-03-23T18:08:33.579 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_peer_request": 0, 2024-03-23T18:08:33.579 INFO:teuthology.orchestra.run.smithi092.stdout: "req_create_latency": { 2024-03-23T18:08:33.579 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:33.579 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:33.579 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:33.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:33.579 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getattr_latency": { 2024-03-23T18:08:33.579 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:33.579 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:33.580 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:33.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:33.580 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getfilelock_latency": { 2024-03-23T18:08:33.580 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:33.580 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:33.580 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:33.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:33.580 INFO:teuthology.orchestra.run.smithi092.stdout: "req_link_latency": { 2024-03-23T18:08:33.581 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:33.581 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:33.581 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:33.581 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:33.581 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookup_latency": { 2024-03-23T18:08:33.581 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:33.581 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:33.581 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:33.581 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:33.582 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookuphash_latency": { 2024-03-23T18:08:33.582 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:33.582 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:33.582 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:33.582 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:33.582 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupino_latency": { 2024-03-23T18:08:33.582 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:33.582 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:33.582 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:33.583 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:33.583 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupname_latency": { 2024-03-23T18:08:33.583 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:33.583 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:33.583 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:33.583 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:33.583 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupparent_latency": { 2024-03-23T18:08:33.583 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:33.583 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:33.584 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:33.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:33.584 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupsnap_latency": { 2024-03-23T18:08:33.584 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:33.584 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:33.584 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:33.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:33.584 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lssnap_latency": { 2024-03-23T18:08:33.585 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:33.585 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:33.585 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:33.585 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:33.585 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mkdir_latency": { 2024-03-23T18:08:33.585 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:33.585 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:33.585 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:33.585 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:33.586 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mknod_latency": { 2024-03-23T18:08:33.586 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:33.586 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:33.586 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:33.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:33.586 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mksnap_latency": { 2024-03-23T18:08:33.586 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:33.586 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:33.586 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:33.587 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:33.587 INFO:teuthology.orchestra.run.smithi092.stdout: "req_open_latency": { 2024-03-23T18:08:33.587 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:33.587 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:33.587 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:33.587 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:33.587 INFO:teuthology.orchestra.run.smithi092.stdout: "req_readdir_latency": { 2024-03-23T18:08:33.587 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:33.588 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:33.588 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:33.588 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:33.588 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rename_latency": { 2024-03-23T18:08:33.588 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:33.588 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:33.588 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:33.588 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:33.588 INFO:teuthology.orchestra.run.smithi092.stdout: "req_renamesnap_latency": { 2024-03-23T18:08:33.589 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:33.589 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:33.589 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:33.589 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:33.589 INFO:teuthology.orchestra.run.smithi092.stdout: "req_snapdiff_latency": { 2024-03-23T18:08:33.589 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:33.589 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:33.589 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:33.589 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:33.590 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmdir_latency": { 2024-03-23T18:08:33.590 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:33.590 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:33.590 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:33.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:33.590 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmsnap_latency": { 2024-03-23T18:08:33.590 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:33.590 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:33.591 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:33.591 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:33.591 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmxattr_latency": { 2024-03-23T18:08:33.591 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:33.591 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:33.591 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:33.591 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:33.591 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setattr_latency": { 2024-03-23T18:08:33.591 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:33.592 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:33.592 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:33.592 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:33.592 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setdirlayout_latency": { 2024-03-23T18:08:33.592 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:33.592 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:33.592 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:33.592 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:33.592 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setfilelock_latency": { 2024-03-23T18:08:33.593 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:33.593 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:33.593 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:33.593 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:33.593 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setlayout_latency": { 2024-03-23T18:08:33.593 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:33.593 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:33.593 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:33.593 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:33.594 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setxattr_latency": { 2024-03-23T18:08:33.594 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:33.594 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:33.594 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:33.594 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:33.594 INFO:teuthology.orchestra.run.smithi092.stdout: "req_symlink_latency": { 2024-03-23T18:08:33.594 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:33.594 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:33.594 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:33.595 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:33.595 INFO:teuthology.orchestra.run.smithi092.stdout: "req_unlink_latency": { 2024-03-23T18:08:33.595 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:33.595 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:33.595 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:33.595 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:33.595 INFO:teuthology.orchestra.run.smithi092.stdout: "cap_revoke_eviction": 0, 2024-03-23T18:08:33.595 INFO:teuthology.orchestra.run.smithi092.stdout: "cap_acquisition_throttle": 0, 2024-03-23T18:08:33.595 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getvxattr_latency": { 2024-03-23T18:08:33.596 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:33.596 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:33.596 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:33.596 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-23T18:08:33.596 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:33.596 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_sessions": { 2024-03-23T18:08:33.596 INFO:teuthology.orchestra.run.smithi092.stdout: "session_count": 1, 2024-03-23T18:08:33.596 INFO:teuthology.orchestra.run.smithi092.stdout: "session_add": 3, 2024-03-23T18:08:33.596 INFO:teuthology.orchestra.run.smithi092.stdout: "session_remove": 2, 2024-03-23T18:08:33.597 INFO:teuthology.orchestra.run.smithi092.stdout: "sessions_open": 3, 2024-03-23T18:08:33.597 INFO:teuthology.orchestra.run.smithi092.stdout: "sessions_stale": 0, 2024-03-23T18:08:33.597 INFO:teuthology.orchestra.run.smithi092.stdout: "total_load": 0, 2024-03-23T18:08:33.597 INFO:teuthology.orchestra.run.smithi092.stdout: "average_load": 0, 2024-03-23T18:08:33.597 INFO:teuthology.orchestra.run.smithi092.stdout: "avg_session_uptime": 6729, 2024-03-23T18:08:33.597 INFO:teuthology.orchestra.run.smithi092.stdout: "mdthresh_evicted": 0 2024-03-23T18:08:33.597 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:33.597 INFO:teuthology.orchestra.run.smithi092.stdout: "mempool": { 2024-03-23T18:08:33.598 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_bytes": 0, 2024-03-23T18:08:33.598 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_items": 0, 2024-03-23T18:08:33.598 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_bytes": 0, 2024-03-23T18:08:33.598 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_items": 0, 2024-03-23T18:08:33.598 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_bytes": 0, 2024-03-23T18:08:33.598 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_items": 0, 2024-03-23T18:08:33.598 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-23T18:08:33.598 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_items": 0, 2024-03-23T18:08:33.598 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-23T18:08:33.599 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_items": 0, 2024-03-23T18:08:33.599 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_bytes": 0, 2024-03-23T18:08:33.599 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_items": 0, 2024-03-23T18:08:33.599 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-23T18:08:33.599 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_items": 0, 2024-03-23T18:08:33.599 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_bytes": 0, 2024-03-23T18:08:33.599 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_items": 0, 2024-03-23T18:08:33.599 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_bytes": 0, 2024-03-23T18:08:33.599 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_items": 0, 2024-03-23T18:08:33.600 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-23T18:08:33.600 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_items": 0, 2024-03-23T18:08:33.600 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-23T18:08:33.600 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_items": 0, 2024-03-23T18:08:33.600 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_bytes": 0, 2024-03-23T18:08:33.600 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_items": 0, 2024-03-23T18:08:33.600 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_bytes": 0, 2024-03-23T18:08:33.600 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_items": 0, 2024-03-23T18:08:33.600 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-23T18:08:33.601 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_items": 0, 2024-03-23T18:08:33.601 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_bytes": 0, 2024-03-23T18:08:33.601 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_items": 0, 2024-03-23T18:08:33.601 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_bytes": 0, 2024-03-23T18:08:33.601 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_items": 0, 2024-03-23T18:08:33.601 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_bytes": 0, 2024-03-23T18:08:33.601 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_items": 0, 2024-03-23T18:08:33.601 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_bytes": 0, 2024-03-23T18:08:33.601 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_items": 0, 2024-03-23T18:08:33.602 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_bytes": 268636, 2024-03-23T18:08:33.602 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_items": 875, 2024-03-23T18:08:33.602 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_bytes": 0, 2024-03-23T18:08:33.602 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_items": 0, 2024-03-23T18:08:33.602 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_bytes": 0, 2024-03-23T18:08:33.602 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_items": 0, 2024-03-23T18:08:33.602 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_bytes": 0, 2024-03-23T18:08:33.602 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_items": 0, 2024-03-23T18:08:33.602 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_bytes": 0, 2024-03-23T18:08:33.603 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_items": 0, 2024-03-23T18:08:33.603 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_bytes": 7936, 2024-03-23T18:08:33.603 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_items": 165, 2024-03-23T18:08:33.603 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_bytes": 0, 2024-03-23T18:08:33.603 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_items": 0, 2024-03-23T18:08:33.603 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_bytes": 0, 2024-03-23T18:08:33.603 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_items": 0, 2024-03-23T18:08:33.603 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_bytes": 14920933, 2024-03-23T18:08:33.603 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_items": 102250, 2024-03-23T18:08:33.604 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_bytes": 0, 2024-03-23T18:08:33.604 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_items": 0, 2024-03-23T18:08:33.604 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_bytes": 0, 2024-03-23T18:08:33.604 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_items": 0 2024-03-23T18:08:33.604 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:33.604 INFO:teuthology.orchestra.run.smithi092.stdout: "objecter": { 2024-03-23T18:08:33.604 INFO:teuthology.orchestra.run.smithi092.stdout: "op_active": 0, 2024-03-23T18:08:33.604 INFO:teuthology.orchestra.run.smithi092.stdout: "op_laggy": 0, 2024-03-23T18:08:33.605 INFO:teuthology.orchestra.run.smithi092.stdout: "op_send": 4579, 2024-03-23T18:08:33.605 INFO:teuthology.orchestra.run.smithi092.stdout: "op_send_bytes": 21, 2024-03-23T18:08:33.605 INFO:teuthology.orchestra.run.smithi092.stdout: "op_resend": 3, 2024-03-23T18:08:33.605 INFO:teuthology.orchestra.run.smithi092.stdout: "op_reply": 4579, 2024-03-23T18:08:33.605 INFO:teuthology.orchestra.run.smithi092.stdout: "op_latency": { 2024-03-23T18:08:33.605 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 4579, 2024-03-23T18:08:33.605 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 7.667780063, 2024-03-23T18:08:33.605 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.001674553 2024-03-23T18:08:33.605 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:33.606 INFO:teuthology.orchestra.run.smithi092.stdout: "op_inflight": 0, 2024-03-23T18:08:33.606 INFO:teuthology.orchestra.run.smithi092.stdout: "oplen_avg": { 2024-03-23T18:08:33.606 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 4579, 2024-03-23T18:08:33.606 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 4580 2024-03-23T18:08:33.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:33.606 INFO:teuthology.orchestra.run.smithi092.stdout: "op": 4579, 2024-03-23T18:08:33.606 INFO:teuthology.orchestra.run.smithi092.stdout: "op_r": 4579, 2024-03-23T18:08:33.606 INFO:teuthology.orchestra.run.smithi092.stdout: "op_w": 0, 2024-03-23T18:08:33.607 INFO:teuthology.orchestra.run.smithi092.stdout: "op_rmw": 0, 2024-03-23T18:08:33.607 INFO:teuthology.orchestra.run.smithi092.stdout: "op_pg": 0, 2024-03-23T18:08:33.607 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_stat": 2527, 2024-03-23T18:08:33.607 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_create": 0, 2024-03-23T18:08:33.607 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_read": 2050, 2024-03-23T18:08:33.607 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_write": 0, 2024-03-23T18:08:33.607 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_writefull": 0, 2024-03-23T18:08:33.607 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_writesame": 0, 2024-03-23T18:08:33.607 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_append": 0, 2024-03-23T18:08:33.608 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_zero": 0, 2024-03-23T18:08:33.608 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_truncate": 0, 2024-03-23T18:08:33.608 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_delete": 0, 2024-03-23T18:08:33.608 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_mapext": 0, 2024-03-23T18:08:33.608 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_sparse_read": 0, 2024-03-23T18:08:33.608 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_clonerange": 0, 2024-03-23T18:08:33.608 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_getxattr": 1, 2024-03-23T18:08:33.608 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_setxattr": 0, 2024-03-23T18:08:33.608 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_cmpxattr": 0, 2024-03-23T18:08:33.609 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_rmxattr": 0, 2024-03-23T18:08:33.609 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_resetxattrs": 0, 2024-03-23T18:08:33.609 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_call": 0, 2024-03-23T18:08:33.609 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_watch": 0, 2024-03-23T18:08:33.609 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_notify": 0, 2024-03-23T18:08:33.609 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_src_cmpxattr": 0, 2024-03-23T18:08:33.609 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_pgls": 0, 2024-03-23T18:08:33.609 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_pgls_filter": 0, 2024-03-23T18:08:33.609 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_other": 0, 2024-03-23T18:08:33.610 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_active": 0, 2024-03-23T18:08:33.610 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_send": 0, 2024-03-23T18:08:33.610 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_resend": 0, 2024-03-23T18:08:33.610 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_ping": 0, 2024-03-23T18:08:33.610 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_active": 0, 2024-03-23T18:08:33.610 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_send": 0, 2024-03-23T18:08:33.610 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_resend": 0, 2024-03-23T18:08:33.610 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_active": 0, 2024-03-23T18:08:33.610 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_send": 0, 2024-03-23T18:08:33.611 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_resend": 0, 2024-03-23T18:08:33.611 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_active": 0, 2024-03-23T18:08:33.611 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_send": 0, 2024-03-23T18:08:33.611 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_resend": 0, 2024-03-23T18:08:33.611 INFO:teuthology.orchestra.run.smithi092.stdout: "command_active": 0, 2024-03-23T18:08:33.611 INFO:teuthology.orchestra.run.smithi092.stdout: "command_send": 0, 2024-03-23T18:08:33.611 INFO:teuthology.orchestra.run.smithi092.stdout: "command_resend": 0, 2024-03-23T18:08:33.611 INFO:teuthology.orchestra.run.smithi092.stdout: "map_epoch": 110, 2024-03-23T18:08:33.611 INFO:teuthology.orchestra.run.smithi092.stdout: "map_full": 0, 2024-03-23T18:08:33.612 INFO:teuthology.orchestra.run.smithi092.stdout: "map_inc": 30, 2024-03-23T18:08:33.612 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_sessions": 11, 2024-03-23T18:08:33.612 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_session_open": 11, 2024-03-23T18:08:33.612 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_session_close": 0, 2024-03-23T18:08:33.612 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_laggy": 0, 2024-03-23T18:08:33.612 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_wr": 0, 2024-03-23T18:08:33.612 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_rd": 2, 2024-03-23T18:08:33.612 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_del": 0 2024-03-23T18:08:33.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:33.613 INFO:teuthology.orchestra.run.smithi092.stdout: "oft": { 2024-03-23T18:08:33.613 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_objs": 0, 2024-03-23T18:08:33.613 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_kv_pairs": 0, 2024-03-23T18:08:33.613 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_updates": 0, 2024-03-23T18:08:33.613 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_removes": 0 2024-03-23T18:08:33.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:33.613 INFO:teuthology.orchestra.run.smithi092.stdout: "osd-slow-ops": { 2024-03-23T18:08:33.613 INFO:teuthology.orchestra.run.smithi092.stdout: "slow_ops_count": 0 2024-03-23T18:08:33.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:33.614 INFO:teuthology.orchestra.run.smithi092.stdout: "purge_queue": { 2024-03-23T18:08:33.614 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_ops": 0, 2024-03-23T18:08:33.614 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_ops_high_water": 0, 2024-03-23T18:08:33.614 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing": 0, 2024-03-23T18:08:33.614 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_high_water": 0, 2024-03-23T18:08:33.614 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executed": 0, 2024-03-23T18:08:33.614 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_item_in_journal": 0 2024-03-23T18:08:33.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:33.615 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-23T18:08:33.615 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-03-23T18:08:33.615 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 104857600, 2024-03-23T18:08:33.615 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-03-23T18:08:33.615 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 5011, 2024-03-23T18:08:33.615 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 379765183, 2024-03-23T18:08:33.615 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-03-23T18:08:33.615 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 5011, 2024-03-23T18:08:33.616 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-03-23T18:08:33.616 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-03-23T18:08:33.616 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 5011, 2024-03-23T18:08:33.616 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 379765183, 2024-03-23T18:08:33.616 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-03-23T18:08:33.616 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:33.616 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:33.616 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:33.617 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-23T18:08:33.617 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:33.617 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-objecter_bytes": { 2024-03-23T18:08:33.617 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-03-23T18:08:33.617 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 104857600, 2024-03-23T18:08:33.617 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-03-23T18:08:33.617 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-03-23T18:08:33.617 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-03-23T18:08:33.617 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-03-23T18:08:33.618 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-03-23T18:08:33.618 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 4579, 2024-03-23T18:08:33.618 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 376976387, 2024-03-23T18:08:33.618 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 4579, 2024-03-23T18:08:33.618 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 376976387, 2024-03-23T18:08:33.618 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-03-23T18:08:33.618 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:33.618 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:33.619 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:33.619 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-23T18:08:33.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:33.619 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-objecter_ops": { 2024-03-23T18:08:33.619 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-03-23T18:08:33.619 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 1024, 2024-03-23T18:08:33.619 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-03-23T18:08:33.619 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-03-23T18:08:33.620 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-03-23T18:08:33.620 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-03-23T18:08:33.620 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-03-23T18:08:33.620 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 4579, 2024-03-23T18:08:33.620 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 4579, 2024-03-23T18:08:33.620 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 4579, 2024-03-23T18:08:33.620 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 4579, 2024-03-23T18:08:33.620 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-03-23T18:08:33.620 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:33.621 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:33.621 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:33.621 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-23T18:08:33.621 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:33.621 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-write_buf_throttle": { 2024-03-23T18:08:33.621 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-03-23T18:08:33.621 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 3758096384, 2024-03-23T18:08:33.621 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-03-23T18:08:33.621 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-03-23T18:08:33.622 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-03-23T18:08:33.622 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-03-23T18:08:33.622 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-03-23T18:08:33.622 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-03-23T18:08:33.622 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-03-23T18:08:33.622 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 0, 2024-03-23T18:08:33.622 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 0, 2024-03-23T18:08:33.622 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-03-23T18:08:33.623 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:33.623 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:33.623 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:33.623 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-23T18:08:33.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-23T18:08:33.623 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-write_buf_throttle-0x5640e412c500": { 2024-03-23T18:08:33.623 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-03-23T18:08:33.623 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 3758096384, 2024-03-23T18:08:33.623 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-03-23T18:08:33.624 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-03-23T18:08:33.624 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-03-23T18:08:33.624 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-03-23T18:08:33.624 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-03-23T18:08:33.624 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-03-23T18:08:33.624 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-03-23T18:08:33.624 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 0, 2024-03-23T18:08:33.624 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 0, 2024-03-23T18:08:33.625 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-03-23T18:08:33.625 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-23T18:08:33.625 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-23T18:08:33.625 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-23T18:08:33.626 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-23T18:08:33.626 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-23T18:08:33.626 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-03-23T18:08:33.910 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-03-23T18:08:33.910 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-03-23T18:08:33.910 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=0 2024-03-23T18:08:33.910 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-03-23T18:08:33.910 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-03-23T18:08:33.911 DEBUG:tasks.check_counter:Getting stats from c 2024-03-23T18:08:33.911 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph daemon mds.c perf dump 2024-03-23T18:08:34.159 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.c/config 2024-03-23T18:08:34.274 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:34 smithi152 ceph-mon[47949]: pgmap v567: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 8.2 MiB/s wr, 274 op/s 2024-03-23T18:08:34.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:34 smithi092 ceph-mon[79089]: pgmap v567: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 8.2 MiB/s wr, 274 op/s 2024-03-23T18:08:34.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:34 smithi086 ceph-mon[38221]: pgmap v567: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 8.2 MiB/s wr, 274 op/s 2024-03-23T18:08:35.147 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-03-23T18:08:35.147 INFO:teuthology.orchestra.run.smithi152.stdout: "AsyncMessenger::Worker-0": { 2024-03-23T18:08:35.147 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_messages": 258, 2024-03-23T18:08:35.147 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages": 260, 2024-03-23T18:08:35.148 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_bytes": 28235721, 2024-03-23T18:08:35.148 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_bytes": 80248, 2024-03-23T18:08:35.148 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_created_connections": 7, 2024-03-23T18:08:35.148 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_active_connections": 3, 2024-03-23T18:08:35.148 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_total_time": 0.039156995, 2024-03-23T18:08:35.148 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_send_time": 0.005373009, 2024-03-23T18:08:35.148 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_recv_time": 0.023522862, 2024-03-23T18:08:35.148 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_fast_dispatch_time": 0.007344014, 2024-03-23T18:08:35.148 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T18:08:35.149 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 258, 2024-03-23T18:08:35.149 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.018540421, 2024-03-23T18:08:35.149 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000071862 2024-03-23T18:08:35.149 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:35.149 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_handle_ack_lat": { 2024-03-23T18:08:35.149 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:35.149 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:35.149 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:35.150 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:35.150 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-23T18:08:35.150 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_encrypted_bytes": 0 2024-03-23T18:08:35.150 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:35.150 INFO:teuthology.orchestra.run.smithi152.stdout: "AsyncMessenger::Worker-1": { 2024-03-23T18:08:35.150 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_messages": 133, 2024-03-23T18:08:35.150 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages": 126, 2024-03-23T18:08:35.150 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_bytes": 21184164, 2024-03-23T18:08:35.151 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_bytes": 44572, 2024-03-23T18:08:35.151 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_created_connections": 5, 2024-03-23T18:08:35.151 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_active_connections": 4, 2024-03-23T18:08:35.151 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_total_time": 0.023503936, 2024-03-23T18:08:35.151 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_send_time": 0.003137111, 2024-03-23T18:08:35.151 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_recv_time": 0.018359889, 2024-03-23T18:08:35.151 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_fast_dispatch_time": 0.002902808, 2024-03-23T18:08:35.151 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T18:08:35.151 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 125, 2024-03-23T18:08:35.152 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.015044124, 2024-03-23T18:08:35.152 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000120352 2024-03-23T18:08:35.152 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:35.152 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_handle_ack_lat": { 2024-03-23T18:08:35.152 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:35.152 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:35.152 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:35.152 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:35.153 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_encrypted_bytes": 201248, 2024-03-23T18:08:35.153 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_encrypted_bytes": 18080 2024-03-23T18:08:35.153 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:35.153 INFO:teuthology.orchestra.run.smithi152.stdout: "AsyncMessenger::Worker-2": { 2024-03-23T18:08:35.153 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_messages": 74, 2024-03-23T18:08:35.153 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages": 95, 2024-03-23T18:08:35.153 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_bytes": 16798958, 2024-03-23T18:08:35.153 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_bytes": 140964, 2024-03-23T18:08:35.153 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_created_connections": 6, 2024-03-23T18:08:35.154 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_active_connections": 5, 2024-03-23T18:08:35.154 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_total_time": 0.014396696, 2024-03-23T18:08:35.154 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_send_time": 0.002333300, 2024-03-23T18:08:35.154 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_recv_time": 0.009483346, 2024-03-23T18:08:35.154 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_fast_dispatch_time": 0.001965859, 2024-03-23T18:08:35.154 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T18:08:35.154 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 94, 2024-03-23T18:08:35.155 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.015447844, 2024-03-23T18:08:35.155 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000164338 2024-03-23T18:08:35.155 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:35.155 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_handle_ack_lat": { 2024-03-23T18:08:35.155 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:35.155 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:35.155 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:35.155 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:35.156 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_encrypted_bytes": 2768, 2024-03-23T18:08:35.156 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_encrypted_bytes": 119184 2024-03-23T18:08:35.156 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:35.156 INFO:teuthology.orchestra.run.smithi152.stdout: "cct": { 2024-03-23T18:08:35.156 INFO:teuthology.orchestra.run.smithi152.stdout: "total_workers": 1, 2024-03-23T18:08:35.156 INFO:teuthology.orchestra.run.smithi152.stdout: "unhealthy_workers": 0 2024-03-23T18:08:35.156 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:35.156 INFO:teuthology.orchestra.run.smithi152.stdout: "finisher-MDSRank": { 2024-03-23T18:08:35.156 INFO:teuthology.orchestra.run.smithi152.stdout: "queue_len": 0, 2024-03-23T18:08:35.157 INFO:teuthology.orchestra.run.smithi152.stdout: "complete_latency": { 2024-03-23T18:08:35.157 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 669, 2024-03-23T18:08:35.157 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.039833601, 2024-03-23T18:08:35.157 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000059542 2024-03-23T18:08:35.157 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-03-23T18:08:35.157 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:35.157 INFO:teuthology.orchestra.run.smithi152.stdout: "finisher-PurgeQueue": { 2024-03-23T18:08:35.157 INFO:teuthology.orchestra.run.smithi152.stdout: "queue_len": 0, 2024-03-23T18:08:35.158 INFO:teuthology.orchestra.run.smithi152.stdout: "complete_latency": { 2024-03-23T18:08:35.158 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:35.158 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:35.158 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:35.158 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-03-23T18:08:35.158 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:35.158 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": { 2024-03-23T18:08:35.158 INFO:teuthology.orchestra.run.smithi152.stdout: "request": 0, 2024-03-23T18:08:35.159 INFO:teuthology.orchestra.run.smithi152.stdout: "reply": 0, 2024-03-23T18:08:35.159 INFO:teuthology.orchestra.run.smithi152.stdout: "reply_latency": { 2024-03-23T18:08:35.159 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:35.159 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:35.159 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:35.159 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:35.159 INFO:teuthology.orchestra.run.smithi152.stdout: "slow_reply": 0, 2024-03-23T18:08:35.159 INFO:teuthology.orchestra.run.smithi152.stdout: "forward": 0, 2024-03-23T18:08:35.159 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_fetch_complete": 0, 2024-03-23T18:08:35.160 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_fetch_keys": 0, 2024-03-23T18:08:35.160 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_commit": 0, 2024-03-23T18:08:35.160 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_split": 0, 2024-03-23T18:08:35.160 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_merge": 0, 2024-03-23T18:08:35.160 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes": 4017, 2024-03-23T18:08:35.160 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_top": 2490, 2024-03-23T18:08:35.160 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_bottom": 1527, 2024-03-23T18:08:35.160 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_pin_tail": 0, 2024-03-23T18:08:35.160 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_pinned": 468, 2024-03-23T18:08:35.161 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_expired": 0, 2024-03-23T18:08:35.161 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_with_caps": 0, 2024-03-23T18:08:35.161 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": 0, 2024-03-23T18:08:35.161 INFO:teuthology.orchestra.run.smithi152.stdout: "subtrees": 11, 2024-03-23T18:08:35.161 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse": 0, 2024-03-23T18:08:35.161 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_hit": 0, 2024-03-23T18:08:35.161 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_forward": 0, 2024-03-23T18:08:35.161 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_discover": 0, 2024-03-23T18:08:35.162 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_dir_fetch": 0, 2024-03-23T18:08:35.162 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_remote_ino": 0, 2024-03-23T18:08:35.162 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_lock": 0, 2024-03-23T18:08:35.162 INFO:teuthology.orchestra.run.smithi152.stdout: "load_cent": 0, 2024-03-23T18:08:35.162 INFO:teuthology.orchestra.run.smithi152.stdout: "q": 0, 2024-03-23T18:08:35.163 INFO:teuthology.orchestra.run.smithi152.stdout: "exported": 0, 2024-03-23T18:08:35.163 INFO:teuthology.orchestra.run.smithi152.stdout: "exported_inodes": 0, 2024-03-23T18:08:35.163 INFO:teuthology.orchestra.run.smithi152.stdout: "imported": 0, 2024-03-23T18:08:35.163 INFO:teuthology.orchestra.run.smithi152.stdout: "imported_inodes": 0, 2024-03-23T18:08:35.163 INFO:teuthology.orchestra.run.smithi152.stdout: "openino_dir_fetch": 0, 2024-03-23T18:08:35.163 INFO:teuthology.orchestra.run.smithi152.stdout: "openino_backtrace_fetch": 0, 2024-03-23T18:08:35.163 INFO:teuthology.orchestra.run.smithi152.stdout: "openino_peer_discover": 0, 2024-03-23T18:08:35.163 INFO:teuthology.orchestra.run.smithi152.stdout: "root_rfiles": 719, 2024-03-23T18:08:35.163 INFO:teuthology.orchestra.run.smithi152.stdout: "root_rbytes": 214645921, 2024-03-23T18:08:35.164 INFO:teuthology.orchestra.run.smithi152.stdout: "root_rsnaps": 7, 2024-03-23T18:08:35.164 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_backtrace_fetch": 0, 2024-03-23T18:08:35.164 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_set_tag": 0, 2024-03-23T18:08:35.164 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_backtrace_repaired": 0, 2024-03-23T18:08:35.164 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_inotable_repaired": 0, 2024-03-23T18:08:35.164 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_dir_inodes": 0, 2024-03-23T18:08:35.164 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_dir_base_inodes": 0, 2024-03-23T18:08:35.164 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_dirfrag_rstats": 0, 2024-03-23T18:08:35.165 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_file_inodes": 0, 2024-03-23T18:08:35.165 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_inode_file_caps": 0, 2024-03-23T18:08:35.165 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_revoke": 0, 2024-03-23T18:08:35.165 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_grant": 0, 2024-03-23T18:08:35.165 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_trunc": 0, 2024-03-23T18:08:35.165 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-23T18:08:35.165 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-23T18:08:35.165 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_caps": 0, 2024-03-23T18:08:35.166 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_caps_dirty": 0, 2024-03-23T18:08:35.166 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_cap_release": 0, 2024-03-23T18:08:35.166 INFO:teuthology.orchestra.run.smithi152.stdout: "process_request_cap_release": 0 2024-03-23T18:08:35.166 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:35.166 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_cache": { 2024-03-23T18:08:35.166 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_update": 0, 2024-03-23T18:08:35.166 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_update_receipt": 0, 2024-03-23T18:08:35.166 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_try_discover": 0, 2024-03-23T18:08:35.167 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_send_discover": 0, 2024-03-23T18:08:35.167 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_handle_discover": 0, 2024-03-23T18:08:35.167 INFO:teuthology.orchestra.run.smithi152.stdout: "num_strays": 0, 2024-03-23T18:08:35.167 INFO:teuthology.orchestra.run.smithi152.stdout: "num_strays_delayed": 0, 2024-03-23T18:08:35.167 INFO:teuthology.orchestra.run.smithi152.stdout: "num_strays_enqueuing": 0, 2024-03-23T18:08:35.167 INFO:teuthology.orchestra.run.smithi152.stdout: "strays_created": 0, 2024-03-23T18:08:35.167 INFO:teuthology.orchestra.run.smithi152.stdout: "strays_enqueued": 0, 2024-03-23T18:08:35.167 INFO:teuthology.orchestra.run.smithi152.stdout: "strays_reintegrated": 0, 2024-03-23T18:08:35.167 INFO:teuthology.orchestra.run.smithi152.stdout: "strays_migrated": 0, 2024-03-23T18:08:35.168 INFO:teuthology.orchestra.run.smithi152.stdout: "num_recovering_processing": 0, 2024-03-23T18:08:35.168 INFO:teuthology.orchestra.run.smithi152.stdout: "num_recovering_enqueued": 0, 2024-03-23T18:08:35.168 INFO:teuthology.orchestra.run.smithi152.stdout: "num_recovering_prioritized": 0, 2024-03-23T18:08:35.168 INFO:teuthology.orchestra.run.smithi152.stdout: "recovery_started": 0, 2024-03-23T18:08:35.168 INFO:teuthology.orchestra.run.smithi152.stdout: "recovery_completed": 0, 2024-03-23T18:08:35.168 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_enqueue_scrub": 0, 2024-03-23T18:08:35.168 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_exportdir": 0, 2024-03-23T18:08:35.168 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_flush": 0, 2024-03-23T18:08:35.169 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_fragmentdir": 0, 2024-03-23T18:08:35.169 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_fragstats": 0, 2024-03-23T18:08:35.169 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_inodestats": 0 2024-03-23T18:08:35.169 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:35.169 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_log": { 2024-03-23T18:08:35.169 INFO:teuthology.orchestra.run.smithi152.stdout: "evadd": 0, 2024-03-23T18:08:35.169 INFO:teuthology.orchestra.run.smithi152.stdout: "evex": 0, 2024-03-23T18:08:35.169 INFO:teuthology.orchestra.run.smithi152.stdout: "evtrm": 0, 2024-03-23T18:08:35.169 INFO:teuthology.orchestra.run.smithi152.stdout: "ev": 15894, 2024-03-23T18:08:35.170 INFO:teuthology.orchestra.run.smithi152.stdout: "evexg": 0, 2024-03-23T18:08:35.170 INFO:teuthology.orchestra.run.smithi152.stdout: "evexd": 0, 2024-03-23T18:08:35.170 INFO:teuthology.orchestra.run.smithi152.stdout: "segadd": 0, 2024-03-23T18:08:35.170 INFO:teuthology.orchestra.run.smithi152.stdout: "segex": 0, 2024-03-23T18:08:35.170 INFO:teuthology.orchestra.run.smithi152.stdout: "segtrm": 0, 2024-03-23T18:08:35.170 INFO:teuthology.orchestra.run.smithi152.stdout: "seg": 3, 2024-03-23T18:08:35.170 INFO:teuthology.orchestra.run.smithi152.stdout: "segexg": 0, 2024-03-23T18:08:35.170 INFO:teuthology.orchestra.run.smithi152.stdout: "segexd": 0, 2024-03-23T18:08:35.171 INFO:teuthology.orchestra.run.smithi152.stdout: "expos": 624670748, 2024-03-23T18:08:35.171 INFO:teuthology.orchestra.run.smithi152.stdout: "wrpos": 627943254, 2024-03-23T18:08:35.171 INFO:teuthology.orchestra.run.smithi152.stdout: "rdpos": 627904374, 2024-03-23T18:08:35.171 INFO:teuthology.orchestra.run.smithi152.stdout: "jlat": { 2024-03-23T18:08:35.171 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:35.171 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:35.171 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:35.171 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:35.172 INFO:teuthology.orchestra.run.smithi152.stdout: "replayed": 15894 2024-03-23T18:08:35.172 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:35.172 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_mem": { 2024-03-23T18:08:35.172 INFO:teuthology.orchestra.run.smithi152.stdout: "ino": 296, 2024-03-23T18:08:35.172 INFO:teuthology.orchestra.run.smithi152.stdout: "ino+": 3804, 2024-03-23T18:08:35.172 INFO:teuthology.orchestra.run.smithi152.stdout: "ino-": 3508, 2024-03-23T18:08:35.173 INFO:teuthology.orchestra.run.smithi152.stdout: "dir": 62, 2024-03-23T18:08:35.173 INFO:teuthology.orchestra.run.smithi152.stdout: "dir+": 91, 2024-03-23T18:08:35.173 INFO:teuthology.orchestra.run.smithi152.stdout: "dir-": 29, 2024-03-23T18:08:35.173 INFO:teuthology.orchestra.run.smithi152.stdout: "dn": 4017, 2024-03-23T18:08:35.173 INFO:teuthology.orchestra.run.smithi152.stdout: "dn+": 4076, 2024-03-23T18:08:35.173 INFO:teuthology.orchestra.run.smithi152.stdout: "dn-": 59, 2024-03-23T18:08:35.173 INFO:teuthology.orchestra.run.smithi152.stdout: "cap": 0, 2024-03-23T18:08:35.173 INFO:teuthology.orchestra.run.smithi152.stdout: "cap+": 0, 2024-03-23T18:08:35.174 INFO:teuthology.orchestra.run.smithi152.stdout: "cap-": 0, 2024-03-23T18:08:35.174 INFO:teuthology.orchestra.run.smithi152.stdout: "rss": 54916, 2024-03-23T18:08:35.174 INFO:teuthology.orchestra.run.smithi152.stdout: "heap": 207132 2024-03-23T18:08:35.174 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:35.174 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_server": { 2024-03-23T18:08:35.174 INFO:teuthology.orchestra.run.smithi152.stdout: "dispatch_client_request": 0, 2024-03-23T18:08:35.174 INFO:teuthology.orchestra.run.smithi152.stdout: "dispatch_server_request": 0, 2024-03-23T18:08:35.175 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_request": 0, 2024-03-23T18:08:35.175 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_session": 0, 2024-03-23T18:08:35.175 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_peer_request": 0, 2024-03-23T18:08:35.175 INFO:teuthology.orchestra.run.smithi152.stdout: "req_create_latency": { 2024-03-23T18:08:35.175 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:35.175 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:35.175 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:35.175 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:35.176 INFO:teuthology.orchestra.run.smithi152.stdout: "req_getattr_latency": { 2024-03-23T18:08:35.176 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:35.176 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:35.176 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:35.176 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:35.176 INFO:teuthology.orchestra.run.smithi152.stdout: "req_getfilelock_latency": { 2024-03-23T18:08:35.176 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:35.176 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:35.177 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:35.177 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:35.177 INFO:teuthology.orchestra.run.smithi152.stdout: "req_link_latency": { 2024-03-23T18:08:35.177 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:35.177 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:35.177 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:35.177 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:35.178 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookup_latency": { 2024-03-23T18:08:35.178 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:35.178 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:35.178 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:35.178 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:35.178 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookuphash_latency": { 2024-03-23T18:08:35.178 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:35.178 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:35.179 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:35.179 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:35.179 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookupino_latency": { 2024-03-23T18:08:35.179 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:35.179 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:35.179 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:35.179 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:35.180 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookupname_latency": { 2024-03-23T18:08:35.180 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:35.180 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:35.180 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:35.180 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:35.180 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookupparent_latency": { 2024-03-23T18:08:35.180 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:35.181 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:35.181 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:35.181 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:35.181 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookupsnap_latency": { 2024-03-23T18:08:35.181 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:35.181 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:35.181 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:35.181 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:35.182 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lssnap_latency": { 2024-03-23T18:08:35.182 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:35.182 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:35.182 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:35.182 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:35.182 INFO:teuthology.orchestra.run.smithi152.stdout: "req_mkdir_latency": { 2024-03-23T18:08:35.182 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:35.183 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:35.183 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:35.183 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:35.183 INFO:teuthology.orchestra.run.smithi152.stdout: "req_mknod_latency": { 2024-03-23T18:08:35.183 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:35.183 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:35.183 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:35.183 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:35.184 INFO:teuthology.orchestra.run.smithi152.stdout: "req_mksnap_latency": { 2024-03-23T18:08:35.184 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:35.184 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:35.184 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:35.184 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:35.184 INFO:teuthology.orchestra.run.smithi152.stdout: "req_open_latency": { 2024-03-23T18:08:35.184 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:35.184 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:35.184 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:35.185 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:35.185 INFO:teuthology.orchestra.run.smithi152.stdout: "req_readdir_latency": { 2024-03-23T18:08:35.185 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:35.185 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:35.185 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:35.185 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:35.185 INFO:teuthology.orchestra.run.smithi152.stdout: "req_rename_latency": { 2024-03-23T18:08:35.185 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:35.186 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:35.186 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:35.186 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:35.186 INFO:teuthology.orchestra.run.smithi152.stdout: "req_renamesnap_latency": { 2024-03-23T18:08:35.186 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:35.186 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:35.186 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:35.186 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:35.187 INFO:teuthology.orchestra.run.smithi152.stdout: "req_snapdiff_latency": { 2024-03-23T18:08:35.187 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:35.187 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:35.187 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:35.187 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:35.187 INFO:teuthology.orchestra.run.smithi152.stdout: "req_rmdir_latency": { 2024-03-23T18:08:35.187 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:35.187 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:35.187 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:35.188 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:35.188 INFO:teuthology.orchestra.run.smithi152.stdout: "req_rmsnap_latency": { 2024-03-23T18:08:35.188 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:35.188 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:35.188 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:35.188 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:35.188 INFO:teuthology.orchestra.run.smithi152.stdout: "req_rmxattr_latency": { 2024-03-23T18:08:35.188 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:35.189 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:35.189 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:35.189 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:35.189 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setattr_latency": { 2024-03-23T18:08:35.189 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:35.189 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:35.189 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:35.189 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:35.190 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setdirlayout_latency": { 2024-03-23T18:08:35.190 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:35.190 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:35.190 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:35.190 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:35.190 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setfilelock_latency": { 2024-03-23T18:08:35.190 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:35.190 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:35.190 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:35.191 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:35.191 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setlayout_latency": { 2024-03-23T18:08:35.191 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:35.191 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:35.191 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:35.191 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:35.191 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setxattr_latency": { 2024-03-23T18:08:35.191 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:35.192 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:35.192 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:35.192 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:35.192 INFO:teuthology.orchestra.run.smithi152.stdout: "req_symlink_latency": { 2024-03-23T18:08:35.192 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:35.192 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:35.192 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:35.192 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:35.192 INFO:teuthology.orchestra.run.smithi152.stdout: "req_unlink_latency": { 2024-03-23T18:08:35.193 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:35.193 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:35.193 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:35.193 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:35.193 INFO:teuthology.orchestra.run.smithi152.stdout: "cap_revoke_eviction": 0, 2024-03-23T18:08:35.193 INFO:teuthology.orchestra.run.smithi152.stdout: "cap_acquisition_throttle": 0, 2024-03-23T18:08:35.193 INFO:teuthology.orchestra.run.smithi152.stdout: "req_getvxattr_latency": { 2024-03-23T18:08:35.193 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:35.194 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:35.194 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:35.194 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-03-23T18:08:35.194 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:35.194 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_sessions": { 2024-03-23T18:08:35.194 INFO:teuthology.orchestra.run.smithi152.stdout: "session_count": 2, 2024-03-23T18:08:35.194 INFO:teuthology.orchestra.run.smithi152.stdout: "session_add": 2, 2024-03-23T18:08:35.194 INFO:teuthology.orchestra.run.smithi152.stdout: "session_remove": 0, 2024-03-23T18:08:35.194 INFO:teuthology.orchestra.run.smithi152.stdout: "sessions_open": 0, 2024-03-23T18:08:35.195 INFO:teuthology.orchestra.run.smithi152.stdout: "sessions_stale": 0, 2024-03-23T18:08:35.195 INFO:teuthology.orchestra.run.smithi152.stdout: "total_load": 0, 2024-03-23T18:08:35.195 INFO:teuthology.orchestra.run.smithi152.stdout: "average_load": 0, 2024-03-23T18:08:35.195 INFO:teuthology.orchestra.run.smithi152.stdout: "avg_session_uptime": 2943, 2024-03-23T18:08:35.195 INFO:teuthology.orchestra.run.smithi152.stdout: "mdthresh_evicted": 0 2024-03-23T18:08:35.195 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:35.195 INFO:teuthology.orchestra.run.smithi152.stdout: "mempool": { 2024-03-23T18:08:35.195 INFO:teuthology.orchestra.run.smithi152.stdout: "bloom_filter_bytes": 0, 2024-03-23T18:08:35.196 INFO:teuthology.orchestra.run.smithi152.stdout: "bloom_filter_items": 0, 2024-03-23T18:08:35.196 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_alloc_bytes": 0, 2024-03-23T18:08:35.196 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_alloc_items": 0, 2024-03-23T18:08:35.196 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_data_bytes": 0, 2024-03-23T18:08:35.196 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_data_items": 0, 2024-03-23T18:08:35.196 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-23T18:08:35.196 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_onode_items": 0, 2024-03-23T18:08:35.196 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-23T18:08:35.197 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_meta_items": 0, 2024-03-23T18:08:35.197 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_other_bytes": 0, 2024-03-23T18:08:35.197 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_other_items": 0, 2024-03-23T18:08:35.197 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-23T18:08:35.197 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_buffer_items": 0, 2024-03-23T18:08:35.197 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_extent_bytes": 0, 2024-03-23T18:08:35.197 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_extent_items": 0, 2024-03-23T18:08:35.197 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_blob_bytes": 0, 2024-03-23T18:08:35.197 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_blob_items": 0, 2024-03-23T18:08:35.198 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-23T18:08:35.198 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_shared_blob_items": 0, 2024-03-23T18:08:35.198 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-23T18:08:35.198 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_inline_bl_items": 0, 2024-03-23T18:08:35.198 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_fsck_bytes": 0, 2024-03-23T18:08:35.198 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_fsck_items": 0, 2024-03-23T18:08:35.198 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_txc_bytes": 0, 2024-03-23T18:08:35.198 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_txc_items": 0, 2024-03-23T18:08:35.199 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-23T18:08:35.199 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_deferred_items": 0, 2024-03-23T18:08:35.199 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_bytes": 0, 2024-03-23T18:08:35.199 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_items": 0, 2024-03-23T18:08:35.199 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_bytes": 0, 2024-03-23T18:08:35.199 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_items": 0, 2024-03-23T18:08:35.199 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_reader_bytes": 0, 2024-03-23T18:08:35.199 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_reader_items": 0, 2024-03-23T18:08:35.199 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_writer_bytes": 0, 2024-03-23T18:08:35.200 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_writer_items": 0, 2024-03-23T18:08:35.200 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_anon_bytes": 238971, 2024-03-23T18:08:35.200 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_anon_items": 538, 2024-03-23T18:08:35.200 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_meta_bytes": 0, 2024-03-23T18:08:35.200 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_meta_items": 0, 2024-03-23T18:08:35.200 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_bytes": 0, 2024-03-23T18:08:35.200 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_items": 0, 2024-03-23T18:08:35.200 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_mapbl_bytes": 0, 2024-03-23T18:08:35.201 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_mapbl_items": 0, 2024-03-23T18:08:35.201 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_pglog_bytes": 0, 2024-03-23T18:08:35.201 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_pglog_items": 0, 2024-03-23T18:08:35.201 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_bytes": 7936, 2024-03-23T18:08:35.201 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_items": 165, 2024-03-23T18:08:35.201 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_mapping_bytes": 0, 2024-03-23T18:08:35.201 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_mapping_items": 0, 2024-03-23T18:08:35.201 INFO:teuthology.orchestra.run.smithi152.stdout: "pgmap_bytes": 0, 2024-03-23T18:08:35.201 INFO:teuthology.orchestra.run.smithi152.stdout: "pgmap_items": 0, 2024-03-23T18:08:35.202 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_co_bytes": 3781976, 2024-03-23T18:08:35.202 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_co_items": 26563, 2024-03-23T18:08:35.202 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_1_bytes": 0, 2024-03-23T18:08:35.202 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_1_items": 0, 2024-03-23T18:08:35.202 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_2_bytes": 0, 2024-03-23T18:08:35.202 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_2_items": 0 2024-03-23T18:08:35.202 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:35.202 INFO:teuthology.orchestra.run.smithi152.stdout: "objecter": { 2024-03-23T18:08:35.203 INFO:teuthology.orchestra.run.smithi152.stdout: "op_active": 0, 2024-03-23T18:08:35.203 INFO:teuthology.orchestra.run.smithi152.stdout: "op_laggy": 0, 2024-03-23T18:08:35.203 INFO:teuthology.orchestra.run.smithi152.stdout: "op_send": 413, 2024-03-23T18:08:35.203 INFO:teuthology.orchestra.run.smithi152.stdout: "op_send_bytes": 26, 2024-03-23T18:08:35.203 INFO:teuthology.orchestra.run.smithi152.stdout: "op_resend": 4, 2024-03-23T18:08:35.203 INFO:teuthology.orchestra.run.smithi152.stdout: "op_reply": 413, 2024-03-23T18:08:35.203 INFO:teuthology.orchestra.run.smithi152.stdout: "op_latency": { 2024-03-23T18:08:35.203 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 413, 2024-03-23T18:08:35.204 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 1.370985514, 2024-03-23T18:08:35.204 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.003319577 2024-03-23T18:08:35.204 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:35.204 INFO:teuthology.orchestra.run.smithi152.stdout: "op_inflight": 0, 2024-03-23T18:08:35.204 INFO:teuthology.orchestra.run.smithi152.stdout: "oplen_avg": { 2024-03-23T18:08:35.204 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 413, 2024-03-23T18:08:35.204 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 414 2024-03-23T18:08:35.204 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:35.204 INFO:teuthology.orchestra.run.smithi152.stdout: "op": 413, 2024-03-23T18:08:35.205 INFO:teuthology.orchestra.run.smithi152.stdout: "op_r": 413, 2024-03-23T18:08:35.205 INFO:teuthology.orchestra.run.smithi152.stdout: "op_w": 0, 2024-03-23T18:08:35.205 INFO:teuthology.orchestra.run.smithi152.stdout: "op_rmw": 0, 2024-03-23T18:08:35.205 INFO:teuthology.orchestra.run.smithi152.stdout: "op_pg": 0, 2024-03-23T18:08:35.205 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_stat": 216, 2024-03-23T18:08:35.205 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_create": 0, 2024-03-23T18:08:35.205 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_read": 194, 2024-03-23T18:08:35.205 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_write": 0, 2024-03-23T18:08:35.206 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_writefull": 0, 2024-03-23T18:08:35.206 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_writesame": 0, 2024-03-23T18:08:35.206 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_append": 0, 2024-03-23T18:08:35.206 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_zero": 0, 2024-03-23T18:08:35.206 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_truncate": 0, 2024-03-23T18:08:35.206 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_delete": 0, 2024-03-23T18:08:35.206 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_mapext": 0, 2024-03-23T18:08:35.206 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_sparse_read": 0, 2024-03-23T18:08:35.206 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_clonerange": 0, 2024-03-23T18:08:35.207 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_getxattr": 2, 2024-03-23T18:08:35.207 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_setxattr": 0, 2024-03-23T18:08:35.207 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_cmpxattr": 0, 2024-03-23T18:08:35.207 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_rmxattr": 0, 2024-03-23T18:08:35.207 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_resetxattrs": 0, 2024-03-23T18:08:35.207 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_call": 0, 2024-03-23T18:08:35.207 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_watch": 0, 2024-03-23T18:08:35.207 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_notify": 0, 2024-03-23T18:08:35.208 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_src_cmpxattr": 0, 2024-03-23T18:08:35.208 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_pgls": 0, 2024-03-23T18:08:35.208 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_pgls_filter": 0, 2024-03-23T18:08:35.208 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_other": 0, 2024-03-23T18:08:35.208 INFO:teuthology.orchestra.run.smithi152.stdout: "linger_active": 0, 2024-03-23T18:08:35.208 INFO:teuthology.orchestra.run.smithi152.stdout: "linger_send": 0, 2024-03-23T18:08:35.208 INFO:teuthology.orchestra.run.smithi152.stdout: "linger_resend": 0, 2024-03-23T18:08:35.208 INFO:teuthology.orchestra.run.smithi152.stdout: "linger_ping": 0, 2024-03-23T18:08:35.208 INFO:teuthology.orchestra.run.smithi152.stdout: "poolop_active": 0, 2024-03-23T18:08:35.209 INFO:teuthology.orchestra.run.smithi152.stdout: "poolop_send": 0, 2024-03-23T18:08:35.209 INFO:teuthology.orchestra.run.smithi152.stdout: "poolop_resend": 0, 2024-03-23T18:08:35.209 INFO:teuthology.orchestra.run.smithi152.stdout: "poolstat_active": 0, 2024-03-23T18:08:35.209 INFO:teuthology.orchestra.run.smithi152.stdout: "poolstat_send": 0, 2024-03-23T18:08:35.209 INFO:teuthology.orchestra.run.smithi152.stdout: "poolstat_resend": 0, 2024-03-23T18:08:35.209 INFO:teuthology.orchestra.run.smithi152.stdout: "statfs_active": 0, 2024-03-23T18:08:35.209 INFO:teuthology.orchestra.run.smithi152.stdout: "statfs_send": 0, 2024-03-23T18:08:35.209 INFO:teuthology.orchestra.run.smithi152.stdout: "statfs_resend": 0, 2024-03-23T18:08:35.210 INFO:teuthology.orchestra.run.smithi152.stdout: "command_active": 0, 2024-03-23T18:08:35.210 INFO:teuthology.orchestra.run.smithi152.stdout: "command_send": 0, 2024-03-23T18:08:35.210 INFO:teuthology.orchestra.run.smithi152.stdout: "command_resend": 0, 2024-03-23T18:08:35.210 INFO:teuthology.orchestra.run.smithi152.stdout: "map_epoch": 110, 2024-03-23T18:08:35.210 INFO:teuthology.orchestra.run.smithi152.stdout: "map_full": 0, 2024-03-23T18:08:35.210 INFO:teuthology.orchestra.run.smithi152.stdout: "map_inc": 4, 2024-03-23T18:08:35.210 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_sessions": 9, 2024-03-23T18:08:35.211 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_session_open": 9, 2024-03-23T18:08:35.211 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_session_close": 0, 2024-03-23T18:08:35.211 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_laggy": 0, 2024-03-23T18:08:35.211 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_wr": 0, 2024-03-23T18:08:35.211 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_rd": 2, 2024-03-23T18:08:35.211 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_del": 0 2024-03-23T18:08:35.211 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:35.211 INFO:teuthology.orchestra.run.smithi152.stdout: "oft": { 2024-03-23T18:08:35.212 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_total_objs": 0, 2024-03-23T18:08:35.212 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_total_kv_pairs": 0, 2024-03-23T18:08:35.212 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_total_updates": 0, 2024-03-23T18:08:35.212 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_total_removes": 0 2024-03-23T18:08:35.212 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:35.212 INFO:teuthology.orchestra.run.smithi152.stdout: "osd-slow-ops": { 2024-03-23T18:08:35.212 INFO:teuthology.orchestra.run.smithi152.stdout: "slow_ops_count": 0 2024-03-23T18:08:35.212 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:35.213 INFO:teuthology.orchestra.run.smithi152.stdout: "purge_queue": { 2024-03-23T18:08:35.213 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executing_ops": 0, 2024-03-23T18:08:35.213 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executing_ops_high_water": 0, 2024-03-23T18:08:35.213 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executing": 0, 2024-03-23T18:08:35.213 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executing_high_water": 0, 2024-03-23T18:08:35.213 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executed": 0, 2024-03-23T18:08:35.213 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_item_in_journal": 0 2024-03-23T18:08:35.213 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:35.214 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-23T18:08:35.214 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-03-23T18:08:35.214 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 104857600, 2024-03-23T18:08:35.214 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-03-23T18:08:35.214 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 462, 2024-03-23T18:08:35.214 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 66172252, 2024-03-23T18:08:35.214 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-03-23T18:08:35.214 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 462, 2024-03-23T18:08:35.215 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 0, 2024-03-23T18:08:35.215 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 0, 2024-03-23T18:08:35.215 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 462, 2024-03-23T18:08:35.215 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 66172252, 2024-03-23T18:08:35.215 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-03-23T18:08:35.215 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:35.216 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:35.216 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:35.216 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-03-23T18:08:35.216 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:35.216 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-objecter_bytes": { 2024-03-23T18:08:35.216 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-03-23T18:08:35.216 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 104857600, 2024-03-23T18:08:35.216 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-03-23T18:08:35.216 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 0, 2024-03-23T18:08:35.217 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 0, 2024-03-23T18:08:35.217 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-03-23T18:08:35.217 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 0, 2024-03-23T18:08:35.217 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 413, 2024-03-23T18:08:35.217 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 65895362, 2024-03-23T18:08:35.217 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 413, 2024-03-23T18:08:35.217 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 65895362, 2024-03-23T18:08:35.217 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-03-23T18:08:35.218 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:35.218 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:35.218 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:35.218 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-03-23T18:08:35.218 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:35.218 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-objecter_ops": { 2024-03-23T18:08:35.218 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-03-23T18:08:35.218 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 1024, 2024-03-23T18:08:35.219 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-03-23T18:08:35.219 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 0, 2024-03-23T18:08:35.219 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 0, 2024-03-23T18:08:35.219 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-03-23T18:08:35.219 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 0, 2024-03-23T18:08:35.219 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 413, 2024-03-23T18:08:35.219 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 413, 2024-03-23T18:08:35.219 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 413, 2024-03-23T18:08:35.220 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 413, 2024-03-23T18:08:35.220 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-03-23T18:08:35.220 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:35.220 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:35.220 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:35.220 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-03-23T18:08:35.220 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:35.220 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-write_buf_throttle": { 2024-03-23T18:08:35.220 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-03-23T18:08:35.221 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 3758096384, 2024-03-23T18:08:35.221 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-03-23T18:08:35.221 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 0, 2024-03-23T18:08:35.221 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 0, 2024-03-23T18:08:35.221 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-03-23T18:08:35.221 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 0, 2024-03-23T18:08:35.221 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 0, 2024-03-23T18:08:35.221 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 0, 2024-03-23T18:08:35.222 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 0, 2024-03-23T18:08:35.222 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 0, 2024-03-23T18:08:35.222 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-03-23T18:08:35.222 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:35.222 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:35.223 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:35.223 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-03-23T18:08:35.223 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:35.223 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-write_buf_throttle-0x5586815885a0": { 2024-03-23T18:08:35.223 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-03-23T18:08:35.223 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 3758096384, 2024-03-23T18:08:35.223 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-03-23T18:08:35.223 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 0, 2024-03-23T18:08:35.223 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 0, 2024-03-23T18:08:35.224 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-03-23T18:08:35.224 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 0, 2024-03-23T18:08:35.224 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 0, 2024-03-23T18:08:35.224 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 0, 2024-03-23T18:08:35.224 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 0, 2024-03-23T18:08:35.224 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 0, 2024-03-23T18:08:35.224 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-03-23T18:08:35.224 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:35.224 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:35.225 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:35.225 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-03-23T18:08:35.225 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-03-23T18:08:35.225 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-03-23T18:08:35.510 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-03-23T18:08:35.511 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-03-23T18:08:35.511 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=7 2024-03-23T18:08:35.511 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-03-23T18:08:35.511 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-03-23T18:08:35.511 DEBUG:tasks.check_counter:Getting stats from f 2024-03-23T18:08:35.511 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph daemon mds.f perf dump 2024-03-23T18:08:35.734 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.c/config 2024-03-23T18:08:36.233 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:36 smithi152 ceph-mon[47949]: pgmap v568: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 8.0 MiB/s wr, 277 op/s 2024-03-23T18:08:36.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:36 smithi092 ceph-mon[79089]: pgmap v568: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 8.0 MiB/s wr, 277 op/s 2024-03-23T18:08:36.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:36 smithi086 ceph-mon[38221]: pgmap v568: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 8.0 MiB/s wr, 277 op/s 2024-03-23T18:08:36.739 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-03-23T18:08:36.739 INFO:teuthology.orchestra.run.smithi152.stdout: "AsyncMessenger::Worker-0": { 2024-03-23T18:08:36.740 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_messages": 2740, 2024-03-23T18:08:36.740 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages": 2742, 2024-03-23T18:08:36.740 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_bytes": 125712719, 2024-03-23T18:08:36.740 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_bytes": 852147, 2024-03-23T18:08:36.740 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_created_connections": 8, 2024-03-23T18:08:36.740 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_active_connections": 4, 2024-03-23T18:08:36.741 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_total_time": 0.271375361, 2024-03-23T18:08:36.741 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_send_time": 0.063192458, 2024-03-23T18:08:36.741 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_recv_time": 0.116122424, 2024-03-23T18:08:36.741 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_fast_dispatch_time": 0.072289358, 2024-03-23T18:08:36.741 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T18:08:36.741 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 2740, 2024-03-23T18:08:36.742 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.111192180, 2024-03-23T18:08:36.742 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000040581 2024-03-23T18:08:36.742 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:36.742 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_handle_ack_lat": { 2024-03-23T18:08:36.742 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:36.743 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:36.743 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:36.743 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:36.743 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-23T18:08:36.743 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_encrypted_bytes": 0 2024-03-23T18:08:36.744 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:36.744 INFO:teuthology.orchestra.run.smithi152.stdout: "AsyncMessenger::Worker-1": { 2024-03-23T18:08:36.744 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_messages": 1218, 2024-03-23T18:08:36.744 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages": 1135, 2024-03-23T18:08:36.744 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_bytes": 94503586, 2024-03-23T18:08:36.744 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_bytes": 431499, 2024-03-23T18:08:36.745 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_created_connections": 6, 2024-03-23T18:08:36.745 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_active_connections": 5, 2024-03-23T18:08:36.745 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_total_time": 0.165054387, 2024-03-23T18:08:36.745 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_send_time": 0.031057792, 2024-03-23T18:08:36.745 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_recv_time": 0.113368995, 2024-03-23T18:08:36.745 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_fast_dispatch_time": 0.025903232, 2024-03-23T18:08:36.746 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T18:08:36.746 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 1135, 2024-03-23T18:08:36.746 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.082138423, 2024-03-23T18:08:36.746 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000072368 2024-03-23T18:08:36.746 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:36.746 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_handle_ack_lat": { 2024-03-23T18:08:36.747 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:36.747 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:36.747 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:36.747 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:36.747 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_encrypted_bytes": 2030688, 2024-03-23T18:08:36.747 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_encrypted_bytes": 189168 2024-03-23T18:08:36.748 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:36.748 INFO:teuthology.orchestra.run.smithi152.stdout: "AsyncMessenger::Worker-2": { 2024-03-23T18:08:36.748 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_messages": 873, 2024-03-23T18:08:36.748 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages": 1115, 2024-03-23T18:08:36.748 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_bytes": 130247692, 2024-03-23T18:08:36.748 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_bytes": 850705, 2024-03-23T18:08:36.749 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_created_connections": 9, 2024-03-23T18:08:36.749 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_active_connections": 6, 2024-03-23T18:08:36.749 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_total_time": 0.144402382, 2024-03-23T18:08:36.749 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_send_time": 0.023926942, 2024-03-23T18:08:36.749 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_recv_time": 0.090992998, 2024-03-23T18:08:36.749 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_fast_dispatch_time": 0.021312752, 2024-03-23T18:08:36.750 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T18:08:36.750 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 1112, 2024-03-23T18:08:36.750 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.081091569, 2024-03-23T18:08:36.750 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000072924 2024-03-23T18:08:36.750 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:36.751 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_handle_ack_lat": { 2024-03-23T18:08:36.751 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:36.751 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:36.751 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:36.751 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:36.751 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-23T18:08:36.752 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_encrypted_bytes": 579824 2024-03-23T18:08:36.752 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:36.752 INFO:teuthology.orchestra.run.smithi152.stdout: "cct": { 2024-03-23T18:08:36.752 INFO:teuthology.orchestra.run.smithi152.stdout: "total_workers": 1, 2024-03-23T18:08:36.752 INFO:teuthology.orchestra.run.smithi152.stdout: "unhealthy_workers": 0 2024-03-23T18:08:36.753 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:36.753 INFO:teuthology.orchestra.run.smithi152.stdout: "finisher-MDSRank": { 2024-03-23T18:08:36.753 INFO:teuthology.orchestra.run.smithi152.stdout: "queue_len": 0, 2024-03-23T18:08:36.753 INFO:teuthology.orchestra.run.smithi152.stdout: "complete_latency": { 2024-03-23T18:08:36.753 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 7349, 2024-03-23T18:08:36.753 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.166996044, 2024-03-23T18:08:36.754 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000022723 2024-03-23T18:08:36.754 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-03-23T18:08:36.754 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:36.754 INFO:teuthology.orchestra.run.smithi152.stdout: "finisher-PurgeQueue": { 2024-03-23T18:08:36.754 INFO:teuthology.orchestra.run.smithi152.stdout: "queue_len": 0, 2024-03-23T18:08:36.755 INFO:teuthology.orchestra.run.smithi152.stdout: "complete_latency": { 2024-03-23T18:08:36.755 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:36.755 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:36.755 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:36.755 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-03-23T18:08:36.756 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:36.756 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": { 2024-03-23T18:08:36.756 INFO:teuthology.orchestra.run.smithi152.stdout: "request": 0, 2024-03-23T18:08:36.756 INFO:teuthology.orchestra.run.smithi152.stdout: "reply": 0, 2024-03-23T18:08:36.756 INFO:teuthology.orchestra.run.smithi152.stdout: "reply_latency": { 2024-03-23T18:08:36.757 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:36.757 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:36.757 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:36.757 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:36.757 INFO:teuthology.orchestra.run.smithi152.stdout: "slow_reply": 0, 2024-03-23T18:08:36.757 INFO:teuthology.orchestra.run.smithi152.stdout: "forward": 0, 2024-03-23T18:08:36.758 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_fetch_complete": 0, 2024-03-23T18:08:36.758 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_fetch_keys": 0, 2024-03-23T18:08:36.758 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_commit": 0, 2024-03-23T18:08:36.758 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_split": 0, 2024-03-23T18:08:36.758 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_merge": 0, 2024-03-23T18:08:36.759 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes": 23324, 2024-03-23T18:08:36.759 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_top": 340, 2024-03-23T18:08:36.759 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_bottom": 22984, 2024-03-23T18:08:36.759 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_pin_tail": 0, 2024-03-23T18:08:36.759 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_pinned": 22839, 2024-03-23T18:08:36.759 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_expired": 0, 2024-03-23T18:08:36.760 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_with_caps": 0, 2024-03-23T18:08:36.760 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": 0, 2024-03-23T18:08:36.760 INFO:teuthology.orchestra.run.smithi152.stdout: "subtrees": 12, 2024-03-23T18:08:36.760 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse": 0, 2024-03-23T18:08:36.760 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_hit": 0, 2024-03-23T18:08:36.760 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_forward": 0, 2024-03-23T18:08:36.761 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_discover": 0, 2024-03-23T18:08:36.761 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_dir_fetch": 0, 2024-03-23T18:08:36.761 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_remote_ino": 0, 2024-03-23T18:08:36.761 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_lock": 0, 2024-03-23T18:08:36.761 INFO:teuthology.orchestra.run.smithi152.stdout: "load_cent": 0, 2024-03-23T18:08:36.762 INFO:teuthology.orchestra.run.smithi152.stdout: "q": 0, 2024-03-23T18:08:36.762 INFO:teuthology.orchestra.run.smithi152.stdout: "exported": 0, 2024-03-23T18:08:36.762 INFO:teuthology.orchestra.run.smithi152.stdout: "exported_inodes": 0, 2024-03-23T18:08:36.762 INFO:teuthology.orchestra.run.smithi152.stdout: "imported": 0, 2024-03-23T18:08:36.762 INFO:teuthology.orchestra.run.smithi152.stdout: "imported_inodes": 0, 2024-03-23T18:08:36.762 INFO:teuthology.orchestra.run.smithi152.stdout: "openino_dir_fetch": 0, 2024-03-23T18:08:36.763 INFO:teuthology.orchestra.run.smithi152.stdout: "openino_backtrace_fetch": 0, 2024-03-23T18:08:36.763 INFO:teuthology.orchestra.run.smithi152.stdout: "openino_peer_discover": 0, 2024-03-23T18:08:36.763 INFO:teuthology.orchestra.run.smithi152.stdout: "root_rfiles": 0, 2024-03-23T18:08:36.763 INFO:teuthology.orchestra.run.smithi152.stdout: "root_rbytes": 0, 2024-03-23T18:08:36.763 INFO:teuthology.orchestra.run.smithi152.stdout: "root_rsnaps": 0, 2024-03-23T18:08:36.763 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_backtrace_fetch": 0, 2024-03-23T18:08:36.764 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_set_tag": 0, 2024-03-23T18:08:36.764 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_backtrace_repaired": 0, 2024-03-23T18:08:36.764 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_inotable_repaired": 0, 2024-03-23T18:08:36.764 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_dir_inodes": 0, 2024-03-23T18:08:36.764 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_dir_base_inodes": 0, 2024-03-23T18:08:36.764 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_dirfrag_rstats": 0, 2024-03-23T18:08:36.765 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_file_inodes": 0, 2024-03-23T18:08:36.765 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_inode_file_caps": 0, 2024-03-23T18:08:36.765 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_revoke": 0, 2024-03-23T18:08:36.765 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_grant": 0, 2024-03-23T18:08:36.765 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_trunc": 0, 2024-03-23T18:08:36.765 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-23T18:08:36.766 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-23T18:08:36.766 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_caps": 0, 2024-03-23T18:08:36.766 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_caps_dirty": 0, 2024-03-23T18:08:36.766 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_cap_release": 0, 2024-03-23T18:08:36.766 INFO:teuthology.orchestra.run.smithi152.stdout: "process_request_cap_release": 0 2024-03-23T18:08:36.767 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:36.767 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_cache": { 2024-03-23T18:08:36.767 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_update": 0, 2024-03-23T18:08:36.767 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_update_receipt": 0, 2024-03-23T18:08:36.767 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_try_discover": 0, 2024-03-23T18:08:36.768 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_send_discover": 0, 2024-03-23T18:08:36.768 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_handle_discover": 0, 2024-03-23T18:08:36.768 INFO:teuthology.orchestra.run.smithi152.stdout: "num_strays": 0, 2024-03-23T18:08:36.768 INFO:teuthology.orchestra.run.smithi152.stdout: "num_strays_delayed": 0, 2024-03-23T18:08:36.768 INFO:teuthology.orchestra.run.smithi152.stdout: "num_strays_enqueuing": 0, 2024-03-23T18:08:36.768 INFO:teuthology.orchestra.run.smithi152.stdout: "strays_created": 0, 2024-03-23T18:08:36.769 INFO:teuthology.orchestra.run.smithi152.stdout: "strays_enqueued": 0, 2024-03-23T18:08:36.769 INFO:teuthology.orchestra.run.smithi152.stdout: "strays_reintegrated": 0, 2024-03-23T18:08:36.769 INFO:teuthology.orchestra.run.smithi152.stdout: "strays_migrated": 0, 2024-03-23T18:08:36.769 INFO:teuthology.orchestra.run.smithi152.stdout: "num_recovering_processing": 0, 2024-03-23T18:08:36.769 INFO:teuthology.orchestra.run.smithi152.stdout: "num_recovering_enqueued": 0, 2024-03-23T18:08:36.769 INFO:teuthology.orchestra.run.smithi152.stdout: "num_recovering_prioritized": 0, 2024-03-23T18:08:36.770 INFO:teuthology.orchestra.run.smithi152.stdout: "recovery_started": 0, 2024-03-23T18:08:36.770 INFO:teuthology.orchestra.run.smithi152.stdout: "recovery_completed": 0, 2024-03-23T18:08:36.770 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_enqueue_scrub": 0, 2024-03-23T18:08:36.770 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_exportdir": 0, 2024-03-23T18:08:36.770 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_flush": 0, 2024-03-23T18:08:36.771 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_fragmentdir": 0, 2024-03-23T18:08:36.771 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_fragstats": 0, 2024-03-23T18:08:36.771 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_inodestats": 0 2024-03-23T18:08:36.771 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:36.771 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_log": { 2024-03-23T18:08:36.771 INFO:teuthology.orchestra.run.smithi152.stdout: "evadd": 0, 2024-03-23T18:08:36.771 INFO:teuthology.orchestra.run.smithi152.stdout: "evex": 0, 2024-03-23T18:08:36.771 INFO:teuthology.orchestra.run.smithi152.stdout: "evtrm": 0, 2024-03-23T18:08:36.771 INFO:teuthology.orchestra.run.smithi152.stdout: "ev": 105090, 2024-03-23T18:08:36.772 INFO:teuthology.orchestra.run.smithi152.stdout: "evexg": 0, 2024-03-23T18:08:36.772 INFO:teuthology.orchestra.run.smithi152.stdout: "evexd": 0, 2024-03-23T18:08:36.772 INFO:teuthology.orchestra.run.smithi152.stdout: "segadd": 0, 2024-03-23T18:08:36.772 INFO:teuthology.orchestra.run.smithi152.stdout: "segex": 0, 2024-03-23T18:08:36.772 INFO:teuthology.orchestra.run.smithi152.stdout: "segtrm": 0, 2024-03-23T18:08:36.772 INFO:teuthology.orchestra.run.smithi152.stdout: "seg": 129, 2024-03-23T18:08:36.772 INFO:teuthology.orchestra.run.smithi152.stdout: "segexg": 0, 2024-03-23T18:08:36.772 INFO:teuthology.orchestra.run.smithi152.stdout: "segexd": 0, 2024-03-23T18:08:36.773 INFO:teuthology.orchestra.run.smithi152.stdout: "expos": 7151064, 2024-03-23T18:08:36.773 INFO:teuthology.orchestra.run.smithi152.stdout: "wrpos": 351401720, 2024-03-23T18:08:36.773 INFO:teuthology.orchestra.run.smithi152.stdout: "rdpos": 351401121, 2024-03-23T18:08:36.773 INFO:teuthology.orchestra.run.smithi152.stdout: "jlat": { 2024-03-23T18:08:36.773 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:36.773 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:36.773 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:36.773 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:36.773 INFO:teuthology.orchestra.run.smithi152.stdout: "replayed": 105090 2024-03-23T18:08:36.774 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:36.774 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_mem": { 2024-03-23T18:08:36.774 INFO:teuthology.orchestra.run.smithi152.stdout: "ino": 628, 2024-03-23T18:08:36.774 INFO:teuthology.orchestra.run.smithi152.stdout: "ino+": 23377, 2024-03-23T18:08:36.774 INFO:teuthology.orchestra.run.smithi152.stdout: "ino-": 22749, 2024-03-23T18:08:36.774 INFO:teuthology.orchestra.run.smithi152.stdout: "dir": 56, 2024-03-23T18:08:36.774 INFO:teuthology.orchestra.run.smithi152.stdout: "dir+": 97, 2024-03-23T18:08:36.774 INFO:teuthology.orchestra.run.smithi152.stdout: "dir-": 41, 2024-03-23T18:08:36.775 INFO:teuthology.orchestra.run.smithi152.stdout: "dn": 23324, 2024-03-23T18:08:36.775 INFO:teuthology.orchestra.run.smithi152.stdout: "dn+": 23667, 2024-03-23T18:08:36.775 INFO:teuthology.orchestra.run.smithi152.stdout: "dn-": 343, 2024-03-23T18:08:36.775 INFO:teuthology.orchestra.run.smithi152.stdout: "cap": 0, 2024-03-23T18:08:36.775 INFO:teuthology.orchestra.run.smithi152.stdout: "cap+": 0, 2024-03-23T18:08:36.775 INFO:teuthology.orchestra.run.smithi152.stdout: "cap-": 0, 2024-03-23T18:08:36.775 INFO:teuthology.orchestra.run.smithi152.stdout: "rss": 72832, 2024-03-23T18:08:36.775 INFO:teuthology.orchestra.run.smithi152.stdout: "heap": 207156 2024-03-23T18:08:36.775 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:36.776 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_server": { 2024-03-23T18:08:36.776 INFO:teuthology.orchestra.run.smithi152.stdout: "dispatch_client_request": 0, 2024-03-23T18:08:36.776 INFO:teuthology.orchestra.run.smithi152.stdout: "dispatch_server_request": 0, 2024-03-23T18:08:36.776 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_request": 0, 2024-03-23T18:08:36.776 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_session": 0, 2024-03-23T18:08:36.776 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_peer_request": 0, 2024-03-23T18:08:36.776 INFO:teuthology.orchestra.run.smithi152.stdout: "req_create_latency": { 2024-03-23T18:08:36.776 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:36.776 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:36.777 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:36.777 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:36.777 INFO:teuthology.orchestra.run.smithi152.stdout: "req_getattr_latency": { 2024-03-23T18:08:36.777 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:36.777 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:36.777 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:36.777 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:36.777 INFO:teuthology.orchestra.run.smithi152.stdout: "req_getfilelock_latency": { 2024-03-23T18:08:36.778 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:36.778 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:36.778 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:36.778 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:36.778 INFO:teuthology.orchestra.run.smithi152.stdout: "req_link_latency": { 2024-03-23T18:08:36.778 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:36.778 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:36.778 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:36.778 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:36.779 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookup_latency": { 2024-03-23T18:08:36.779 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:36.779 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:36.779 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:36.779 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:36.779 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookuphash_latency": { 2024-03-23T18:08:36.779 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:36.779 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:36.780 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:36.780 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:36.780 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookupino_latency": { 2024-03-23T18:08:36.780 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:36.780 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:36.780 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:36.780 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:36.780 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookupname_latency": { 2024-03-23T18:08:36.780 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:36.781 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:36.781 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:36.781 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:36.781 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookupparent_latency": { 2024-03-23T18:08:36.781 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:36.781 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:36.781 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:36.781 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:36.782 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookupsnap_latency": { 2024-03-23T18:08:36.782 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:36.782 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:36.782 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:36.782 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:36.782 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lssnap_latency": { 2024-03-23T18:08:36.782 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:36.782 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:36.783 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:36.783 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:36.783 INFO:teuthology.orchestra.run.smithi152.stdout: "req_mkdir_latency": { 2024-03-23T18:08:36.783 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:36.783 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:36.783 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:36.783 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:36.783 INFO:teuthology.orchestra.run.smithi152.stdout: "req_mknod_latency": { 2024-03-23T18:08:36.784 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:36.784 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:36.784 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:36.784 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:36.784 INFO:teuthology.orchestra.run.smithi152.stdout: "req_mksnap_latency": { 2024-03-23T18:08:36.784 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:36.784 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:36.784 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:36.784 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:36.785 INFO:teuthology.orchestra.run.smithi152.stdout: "req_open_latency": { 2024-03-23T18:08:36.785 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:36.785 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:36.785 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:36.785 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:36.785 INFO:teuthology.orchestra.run.smithi152.stdout: "req_readdir_latency": { 2024-03-23T18:08:36.785 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:36.785 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:36.785 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:36.786 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:36.786 INFO:teuthology.orchestra.run.smithi152.stdout: "req_rename_latency": { 2024-03-23T18:08:36.786 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:36.786 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:36.786 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:36.786 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:36.786 INFO:teuthology.orchestra.run.smithi152.stdout: "req_renamesnap_latency": { 2024-03-23T18:08:36.786 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:36.787 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:36.787 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:36.787 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:36.787 INFO:teuthology.orchestra.run.smithi152.stdout: "req_snapdiff_latency": { 2024-03-23T18:08:36.787 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:36.787 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:36.787 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:36.787 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:36.788 INFO:teuthology.orchestra.run.smithi152.stdout: "req_rmdir_latency": { 2024-03-23T18:08:36.788 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:36.788 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:36.788 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:36.788 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:36.788 INFO:teuthology.orchestra.run.smithi152.stdout: "req_rmsnap_latency": { 2024-03-23T18:08:36.788 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:36.788 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:36.788 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:36.789 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:36.789 INFO:teuthology.orchestra.run.smithi152.stdout: "req_rmxattr_latency": { 2024-03-23T18:08:36.789 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:36.789 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:36.789 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:36.789 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:36.789 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setattr_latency": { 2024-03-23T18:08:36.789 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:36.790 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:36.790 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:36.790 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:36.790 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setdirlayout_latency": { 2024-03-23T18:08:36.790 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:36.790 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:36.790 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:36.790 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:36.790 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setfilelock_latency": { 2024-03-23T18:08:36.791 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:36.791 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:36.791 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:36.791 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:36.791 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setlayout_latency": { 2024-03-23T18:08:36.791 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:36.791 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:36.791 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:36.792 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:36.792 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setxattr_latency": { 2024-03-23T18:08:36.792 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:36.792 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:36.792 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:36.792 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:36.792 INFO:teuthology.orchestra.run.smithi152.stdout: "req_symlink_latency": { 2024-03-23T18:08:36.792 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:36.792 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:36.793 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:36.793 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:36.793 INFO:teuthology.orchestra.run.smithi152.stdout: "req_unlink_latency": { 2024-03-23T18:08:36.793 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:36.793 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:36.793 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:36.793 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:36.793 INFO:teuthology.orchestra.run.smithi152.stdout: "cap_revoke_eviction": 0, 2024-03-23T18:08:36.794 INFO:teuthology.orchestra.run.smithi152.stdout: "cap_acquisition_throttle": 0, 2024-03-23T18:08:36.794 INFO:teuthology.orchestra.run.smithi152.stdout: "req_getvxattr_latency": { 2024-03-23T18:08:36.794 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:36.794 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:36.794 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:36.794 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-03-23T18:08:36.794 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:36.794 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_sessions": { 2024-03-23T18:08:36.795 INFO:teuthology.orchestra.run.smithi152.stdout: "session_count": 1, 2024-03-23T18:08:36.795 INFO:teuthology.orchestra.run.smithi152.stdout: "session_add": 3, 2024-03-23T18:08:36.795 INFO:teuthology.orchestra.run.smithi152.stdout: "session_remove": 2, 2024-03-23T18:08:36.795 INFO:teuthology.orchestra.run.smithi152.stdout: "sessions_open": 3, 2024-03-23T18:08:36.795 INFO:teuthology.orchestra.run.smithi152.stdout: "sessions_stale": 0, 2024-03-23T18:08:36.795 INFO:teuthology.orchestra.run.smithi152.stdout: "total_load": 0, 2024-03-23T18:08:36.795 INFO:teuthology.orchestra.run.smithi152.stdout: "average_load": 0, 2024-03-23T18:08:36.795 INFO:teuthology.orchestra.run.smithi152.stdout: "avg_session_uptime": 6678, 2024-03-23T18:08:36.796 INFO:teuthology.orchestra.run.smithi152.stdout: "mdthresh_evicted": 0 2024-03-23T18:08:36.796 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:36.796 INFO:teuthology.orchestra.run.smithi152.stdout: "mempool": { 2024-03-23T18:08:36.796 INFO:teuthology.orchestra.run.smithi152.stdout: "bloom_filter_bytes": 0, 2024-03-23T18:08:36.796 INFO:teuthology.orchestra.run.smithi152.stdout: "bloom_filter_items": 0, 2024-03-23T18:08:36.796 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_alloc_bytes": 0, 2024-03-23T18:08:36.796 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_alloc_items": 0, 2024-03-23T18:08:36.796 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_data_bytes": 0, 2024-03-23T18:08:36.796 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_data_items": 0, 2024-03-23T18:08:36.797 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-23T18:08:36.797 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_onode_items": 0, 2024-03-23T18:08:36.797 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-23T18:08:36.797 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_meta_items": 0, 2024-03-23T18:08:36.797 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_other_bytes": 0, 2024-03-23T18:08:36.797 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_other_items": 0, 2024-03-23T18:08:36.797 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-23T18:08:36.797 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_buffer_items": 0, 2024-03-23T18:08:36.797 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_extent_bytes": 0, 2024-03-23T18:08:36.798 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_extent_items": 0, 2024-03-23T18:08:36.798 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_blob_bytes": 0, 2024-03-23T18:08:36.798 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_blob_items": 0, 2024-03-23T18:08:36.798 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-23T18:08:36.798 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_shared_blob_items": 0, 2024-03-23T18:08:36.798 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-23T18:08:36.798 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_inline_bl_items": 0, 2024-03-23T18:08:36.798 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_fsck_bytes": 0, 2024-03-23T18:08:36.799 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_fsck_items": 0, 2024-03-23T18:08:36.799 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_txc_bytes": 0, 2024-03-23T18:08:36.799 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_txc_items": 0, 2024-03-23T18:08:36.799 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-23T18:08:36.799 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_deferred_items": 0, 2024-03-23T18:08:36.799 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_bytes": 0, 2024-03-23T18:08:36.799 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_items": 0, 2024-03-23T18:08:36.799 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_bytes": 0, 2024-03-23T18:08:36.800 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_items": 0, 2024-03-23T18:08:36.800 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_reader_bytes": 0, 2024-03-23T18:08:36.800 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_reader_items": 0, 2024-03-23T18:08:36.800 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_writer_bytes": 0, 2024-03-23T18:08:36.800 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_writer_items": 0, 2024-03-23T18:08:36.800 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_anon_bytes": 267859, 2024-03-23T18:08:36.800 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_anon_items": 874, 2024-03-23T18:08:36.800 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_meta_bytes": 0, 2024-03-23T18:08:36.801 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_meta_items": 0, 2024-03-23T18:08:36.801 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_bytes": 0, 2024-03-23T18:08:36.801 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_items": 0, 2024-03-23T18:08:36.801 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_mapbl_bytes": 0, 2024-03-23T18:08:36.801 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_mapbl_items": 0, 2024-03-23T18:08:36.801 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_pglog_bytes": 0, 2024-03-23T18:08:36.801 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_pglog_items": 0, 2024-03-23T18:08:36.801 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_bytes": 7936, 2024-03-23T18:08:36.802 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_items": 165, 2024-03-23T18:08:36.802 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_mapping_bytes": 0, 2024-03-23T18:08:36.802 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_mapping_items": 0, 2024-03-23T18:08:36.802 INFO:teuthology.orchestra.run.smithi152.stdout: "pgmap_bytes": 0, 2024-03-23T18:08:36.802 INFO:teuthology.orchestra.run.smithi152.stdout: "pgmap_items": 0, 2024-03-23T18:08:36.802 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_co_bytes": 18314951, 2024-03-23T18:08:36.802 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_co_items": 120524, 2024-03-23T18:08:36.802 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_1_bytes": 0, 2024-03-23T18:08:36.803 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_1_items": 0, 2024-03-23T18:08:36.803 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_2_bytes": 0, 2024-03-23T18:08:36.803 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_2_items": 0 2024-03-23T18:08:36.803 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:36.803 INFO:teuthology.orchestra.run.smithi152.stdout: "objecter": { 2024-03-23T18:08:36.803 INFO:teuthology.orchestra.run.smithi152.stdout: "op_active": 0, 2024-03-23T18:08:36.803 INFO:teuthology.orchestra.run.smithi152.stdout: "op_laggy": 0, 2024-03-23T18:08:36.803 INFO:teuthology.orchestra.run.smithi152.stdout: "op_send": 4390, 2024-03-23T18:08:36.804 INFO:teuthology.orchestra.run.smithi152.stdout: "op_send_bytes": 21, 2024-03-23T18:08:36.804 INFO:teuthology.orchestra.run.smithi152.stdout: "op_resend": 3, 2024-03-23T18:08:36.804 INFO:teuthology.orchestra.run.smithi152.stdout: "op_reply": 4390, 2024-03-23T18:08:36.804 INFO:teuthology.orchestra.run.smithi152.stdout: "op_latency": { 2024-03-23T18:08:36.804 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 4390, 2024-03-23T18:08:36.804 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 6.247930414, 2024-03-23T18:08:36.804 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.001423218 2024-03-23T18:08:36.804 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:36.804 INFO:teuthology.orchestra.run.smithi152.stdout: "op_inflight": 0, 2024-03-23T18:08:36.805 INFO:teuthology.orchestra.run.smithi152.stdout: "oplen_avg": { 2024-03-23T18:08:36.805 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 4390, 2024-03-23T18:08:36.805 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 4391 2024-03-23T18:08:36.805 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:36.805 INFO:teuthology.orchestra.run.smithi152.stdout: "op": 4390, 2024-03-23T18:08:36.805 INFO:teuthology.orchestra.run.smithi152.stdout: "op_r": 4390, 2024-03-23T18:08:36.805 INFO:teuthology.orchestra.run.smithi152.stdout: "op_w": 0, 2024-03-23T18:08:36.805 INFO:teuthology.orchestra.run.smithi152.stdout: "op_rmw": 0, 2024-03-23T18:08:36.805 INFO:teuthology.orchestra.run.smithi152.stdout: "op_pg": 0, 2024-03-23T18:08:36.806 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_stat": 2456, 2024-03-23T18:08:36.806 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_create": 0, 2024-03-23T18:08:36.806 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_read": 1932, 2024-03-23T18:08:36.806 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_write": 0, 2024-03-23T18:08:36.806 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_writefull": 0, 2024-03-23T18:08:36.806 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_writesame": 0, 2024-03-23T18:08:36.806 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_append": 0, 2024-03-23T18:08:36.806 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_zero": 0, 2024-03-23T18:08:36.807 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_truncate": 0, 2024-03-23T18:08:36.807 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_delete": 0, 2024-03-23T18:08:36.807 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_mapext": 0, 2024-03-23T18:08:36.807 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_sparse_read": 0, 2024-03-23T18:08:36.807 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_clonerange": 0, 2024-03-23T18:08:36.807 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_getxattr": 1, 2024-03-23T18:08:36.807 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_setxattr": 0, 2024-03-23T18:08:36.807 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_cmpxattr": 0, 2024-03-23T18:08:36.807 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_rmxattr": 0, 2024-03-23T18:08:36.808 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_resetxattrs": 0, 2024-03-23T18:08:36.808 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_call": 0, 2024-03-23T18:08:36.808 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_watch": 0, 2024-03-23T18:08:36.808 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_notify": 0, 2024-03-23T18:08:36.808 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_src_cmpxattr": 0, 2024-03-23T18:08:36.808 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_pgls": 0, 2024-03-23T18:08:36.808 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_pgls_filter": 0, 2024-03-23T18:08:36.808 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_other": 0, 2024-03-23T18:08:36.809 INFO:teuthology.orchestra.run.smithi152.stdout: "linger_active": 0, 2024-03-23T18:08:36.809 INFO:teuthology.orchestra.run.smithi152.stdout: "linger_send": 0, 2024-03-23T18:08:36.809 INFO:teuthology.orchestra.run.smithi152.stdout: "linger_resend": 0, 2024-03-23T18:08:36.809 INFO:teuthology.orchestra.run.smithi152.stdout: "linger_ping": 0, 2024-03-23T18:08:36.809 INFO:teuthology.orchestra.run.smithi152.stdout: "poolop_active": 0, 2024-03-23T18:08:36.809 INFO:teuthology.orchestra.run.smithi152.stdout: "poolop_send": 0, 2024-03-23T18:08:36.809 INFO:teuthology.orchestra.run.smithi152.stdout: "poolop_resend": 0, 2024-03-23T18:08:36.810 INFO:teuthology.orchestra.run.smithi152.stdout: "poolstat_active": 0, 2024-03-23T18:08:36.810 INFO:teuthology.orchestra.run.smithi152.stdout: "poolstat_send": 0, 2024-03-23T18:08:36.810 INFO:teuthology.orchestra.run.smithi152.stdout: "poolstat_resend": 0, 2024-03-23T18:08:36.810 INFO:teuthology.orchestra.run.smithi152.stdout: "statfs_active": 0, 2024-03-23T18:08:36.810 INFO:teuthology.orchestra.run.smithi152.stdout: "statfs_send": 0, 2024-03-23T18:08:36.810 INFO:teuthology.orchestra.run.smithi152.stdout: "statfs_resend": 0, 2024-03-23T18:08:36.810 INFO:teuthology.orchestra.run.smithi152.stdout: "command_active": 0, 2024-03-23T18:08:36.810 INFO:teuthology.orchestra.run.smithi152.stdout: "command_send": 0, 2024-03-23T18:08:36.811 INFO:teuthology.orchestra.run.smithi152.stdout: "command_resend": 0, 2024-03-23T18:08:36.811 INFO:teuthology.orchestra.run.smithi152.stdout: "map_epoch": 110, 2024-03-23T18:08:36.811 INFO:teuthology.orchestra.run.smithi152.stdout: "map_full": 0, 2024-03-23T18:08:36.811 INFO:teuthology.orchestra.run.smithi152.stdout: "map_inc": 30, 2024-03-23T18:08:36.811 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_sessions": 11, 2024-03-23T18:08:36.811 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_session_open": 12, 2024-03-23T18:08:36.811 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_session_close": 1, 2024-03-23T18:08:36.811 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_laggy": 0, 2024-03-23T18:08:36.811 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_wr": 0, 2024-03-23T18:08:36.812 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_rd": 2, 2024-03-23T18:08:36.812 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_del": 0 2024-03-23T18:08:36.812 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:36.812 INFO:teuthology.orchestra.run.smithi152.stdout: "oft": { 2024-03-23T18:08:36.812 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_total_objs": 0, 2024-03-23T18:08:36.812 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_total_kv_pairs": 0, 2024-03-23T18:08:36.812 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_total_updates": 0, 2024-03-23T18:08:36.812 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_total_removes": 0 2024-03-23T18:08:36.813 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:36.813 INFO:teuthology.orchestra.run.smithi152.stdout: "osd-slow-ops": { 2024-03-23T18:08:36.813 INFO:teuthology.orchestra.run.smithi152.stdout: "slow_ops_count": 0 2024-03-23T18:08:36.813 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:36.813 INFO:teuthology.orchestra.run.smithi152.stdout: "purge_queue": { 2024-03-23T18:08:36.813 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executing_ops": 0, 2024-03-23T18:08:36.813 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executing_ops_high_water": 0, 2024-03-23T18:08:36.813 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executing": 0, 2024-03-23T18:08:36.813 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executing_high_water": 0, 2024-03-23T18:08:36.814 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executed": 0, 2024-03-23T18:08:36.814 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_item_in_journal": 0 2024-03-23T18:08:36.814 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:36.814 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-23T18:08:36.814 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-03-23T18:08:36.814 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 104857600, 2024-03-23T18:08:36.814 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-03-23T18:08:36.814 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 4827, 2024-03-23T18:08:36.815 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 350004756, 2024-03-23T18:08:36.815 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-03-23T18:08:36.815 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 4827, 2024-03-23T18:08:36.815 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 0, 2024-03-23T18:08:36.815 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 0, 2024-03-23T18:08:36.815 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 4827, 2024-03-23T18:08:36.815 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 350004756, 2024-03-23T18:08:36.815 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-03-23T18:08:36.815 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:36.816 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:36.816 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:36.816 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-03-23T18:08:36.816 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:36.816 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-objecter_bytes": { 2024-03-23T18:08:36.816 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-03-23T18:08:36.816 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 104857600, 2024-03-23T18:08:36.816 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-03-23T18:08:36.816 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 0, 2024-03-23T18:08:36.817 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 0, 2024-03-23T18:08:36.817 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-03-23T18:08:36.817 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 0, 2024-03-23T18:08:36.817 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 4390, 2024-03-23T18:08:36.817 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 347219795, 2024-03-23T18:08:36.817 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 4390, 2024-03-23T18:08:36.817 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 347219795, 2024-03-23T18:08:36.817 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-03-23T18:08:36.817 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:36.818 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:36.818 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:36.818 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-03-23T18:08:36.818 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:36.818 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-objecter_ops": { 2024-03-23T18:08:36.818 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-03-23T18:08:36.818 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 1024, 2024-03-23T18:08:36.818 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-03-23T18:08:36.819 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 0, 2024-03-23T18:08:36.819 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 0, 2024-03-23T18:08:36.819 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-03-23T18:08:36.819 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 0, 2024-03-23T18:08:36.819 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 4390, 2024-03-23T18:08:36.819 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 4390, 2024-03-23T18:08:36.819 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 4390, 2024-03-23T18:08:36.819 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 4390, 2024-03-23T18:08:36.819 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-03-23T18:08:36.820 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:36.820 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:36.820 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:36.820 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-03-23T18:08:36.820 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:36.820 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-write_buf_throttle": { 2024-03-23T18:08:36.820 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-03-23T18:08:36.820 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 3758096384, 2024-03-23T18:08:36.820 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-03-23T18:08:36.821 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 0, 2024-03-23T18:08:36.821 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 0, 2024-03-23T18:08:36.821 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-03-23T18:08:36.821 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 0, 2024-03-23T18:08:36.821 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 0, 2024-03-23T18:08:36.821 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 0, 2024-03-23T18:08:36.821 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 0, 2024-03-23T18:08:36.821 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 0, 2024-03-23T18:08:36.822 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-03-23T18:08:36.822 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:36.822 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:36.822 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:36.822 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-03-23T18:08:36.822 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:36.822 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-write_buf_throttle-0x560d40256500": { 2024-03-23T18:08:36.822 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-03-23T18:08:36.823 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 3758096384, 2024-03-23T18:08:36.823 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-03-23T18:08:36.823 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 0, 2024-03-23T18:08:36.823 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 0, 2024-03-23T18:08:36.823 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-03-23T18:08:36.823 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 0, 2024-03-23T18:08:36.823 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 0, 2024-03-23T18:08:36.823 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 0, 2024-03-23T18:08:36.823 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 0, 2024-03-23T18:08:36.824 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 0, 2024-03-23T18:08:36.824 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-03-23T18:08:36.824 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:36.824 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:36.824 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:36.824 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-03-23T18:08:36.824 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-03-23T18:08:36.824 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-03-23T18:08:37.094 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-03-23T18:08:37.094 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-03-23T18:08:37.094 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-03-23T18:08:37.094 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-03-23T18:08:37.094 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-03-23T18:08:37.094 DEBUG:tasks.check_counter:Getting stats from i 2024-03-23T18:08:37.095 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph daemon mds.i perf dump 2024-03-23T18:08:37.316 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.c/config 2024-03-23T18:08:38.222 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-03-23T18:08:38.222 INFO:teuthology.orchestra.run.smithi152.stdout: "AsyncMessenger::Worker-0": { 2024-03-23T18:08:38.222 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_messages": 245258, 2024-03-23T18:08:38.222 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages": 348860, 2024-03-23T18:08:38.222 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_bytes": 83356390, 2024-03-23T18:08:38.223 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_bytes": 368481600, 2024-03-23T18:08:38.223 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_created_connections": 20, 2024-03-23T18:08:38.223 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_active_connections": 4, 2024-03-23T18:08:38.223 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_total_time": 13.730354542, 2024-03-23T18:08:38.223 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_send_time": 5.874342538, 2024-03-23T18:08:38.223 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_recv_time": 8.786027453, 2024-03-23T18:08:38.223 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_fast_dispatch_time": 0.601553024, 2024-03-23T18:08:38.223 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T18:08:38.223 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 348859, 2024-03-23T18:08:38.224 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 21.521972321, 2024-03-23T18:08:38.224 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000061692 2024-03-23T18:08:38.224 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:38.224 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_handle_ack_lat": { 2024-03-23T18:08:38.224 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 477736, 2024-03-23T18:08:38.224 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.078513735, 2024-03-23T18:08:38.224 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000164 2024-03-23T18:08:38.224 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:38.225 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_encrypted_bytes": 2034640, 2024-03-23T18:08:38.225 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_encrypted_bytes": 196048 2024-03-23T18:08:38.225 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:38.225 INFO:teuthology.orchestra.run.smithi152.stdout: "AsyncMessenger::Worker-1": { 2024-03-23T18:08:38.225 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_messages": 116817, 2024-03-23T18:08:38.225 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages": 106108, 2024-03-23T18:08:38.225 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_bytes": 37403167, 2024-03-23T18:08:38.225 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_bytes": 110871033, 2024-03-23T18:08:38.225 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_created_connections": 17, 2024-03-23T18:08:38.226 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_active_connections": 6, 2024-03-23T18:08:38.226 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_total_time": 5.972716489, 2024-03-23T18:08:38.226 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_send_time": 2.366071604, 2024-03-23T18:08:38.226 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_recv_time": 3.876770128, 2024-03-23T18:08:38.226 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_fast_dispatch_time": 0.437257622, 2024-03-23T18:08:38.226 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T18:08:38.226 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 106106, 2024-03-23T18:08:38.226 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 3.058885692, 2024-03-23T18:08:38.227 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000028828 2024-03-23T18:08:38.227 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:38.227 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_handle_ack_lat": { 2024-03-23T18:08:38.227 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 188616, 2024-03-23T18:08:38.227 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.024379040, 2024-03-23T18:08:38.227 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000129 2024-03-23T18:08:38.227 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:38.227 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_encrypted_bytes": 7232, 2024-03-23T18:08:38.227 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_encrypted_bytes": 272944 2024-03-23T18:08:38.228 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:38.228 INFO:teuthology.orchestra.run.smithi152.stdout: "AsyncMessenger::Worker-2": { 2024-03-23T18:08:38.228 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_messages": 33314, 2024-03-23T18:08:38.228 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages": 33508, 2024-03-23T18:08:38.228 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_bytes": 11377490, 2024-03-23T18:08:38.228 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_bytes": 142386579, 2024-03-23T18:08:38.228 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_created_connections": 10, 2024-03-23T18:08:38.228 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_active_connections": 5, 2024-03-23T18:08:38.229 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_total_time": 2.284628066, 2024-03-23T18:08:38.229 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_send_time": 0.817885714, 2024-03-23T18:08:38.229 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_recv_time": 0.818405490, 2024-03-23T18:08:38.229 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_fast_dispatch_time": 0.627264429, 2024-03-23T18:08:38.229 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T18:08:38.229 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 33506, 2024-03-23T18:08:38.229 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 1.992514093, 2024-03-23T18:08:38.229 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000059467 2024-03-23T18:08:38.229 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:38.230 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_handle_ack_lat": { 2024-03-23T18:08:38.230 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 23740, 2024-03-23T18:08:38.230 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.003848737, 2024-03-23T18:08:38.230 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000162 2024-03-23T18:08:38.230 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:38.230 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-23T18:08:38.230 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_encrypted_bytes": 399440 2024-03-23T18:08:38.230 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:38.230 INFO:teuthology.orchestra.run.smithi152.stdout: "cct": { 2024-03-23T18:08:38.231 INFO:teuthology.orchestra.run.smithi152.stdout: "total_workers": 1, 2024-03-23T18:08:38.231 INFO:teuthology.orchestra.run.smithi152.stdout: "unhealthy_workers": 0 2024-03-23T18:08:38.231 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:38.231 INFO:teuthology.orchestra.run.smithi152.stdout: "finisher-MDSRank": { 2024-03-23T18:08:38.231 INFO:teuthology.orchestra.run.smithi152.stdout: "queue_len": 0, 2024-03-23T18:08:38.231 INFO:teuthology.orchestra.run.smithi152.stdout: "complete_latency": { 2024-03-23T18:08:38.231 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 56791, 2024-03-23T18:08:38.231 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 45.743104283, 2024-03-23T18:08:38.231 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000805463 2024-03-23T18:08:38.232 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-03-23T18:08:38.232 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:38.232 INFO:teuthology.orchestra.run.smithi152.stdout: "finisher-PurgeQueue": { 2024-03-23T18:08:38.232 INFO:teuthology.orchestra.run.smithi152.stdout: "queue_len": 0, 2024-03-23T18:08:38.232 INFO:teuthology.orchestra.run.smithi152.stdout: "complete_latency": { 2024-03-23T18:08:38.232 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 42620, 2024-03-23T18:08:38.232 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 2.767750757, 2024-03-23T18:08:38.232 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000064940 2024-03-23T18:08:38.232 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-03-23T18:08:38.233 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:38.233 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": { 2024-03-23T18:08:38.233 INFO:teuthology.orchestra.run.smithi152.stdout: "request": 125728, 2024-03-23T18:08:38.233 INFO:teuthology.orchestra.run.smithi152.stdout: "reply": 118732, 2024-03-23T18:08:38.233 INFO:teuthology.orchestra.run.smithi152.stdout: "reply_latency": { 2024-03-23T18:08:38.233 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 118732, 2024-03-23T18:08:38.233 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 278.997203926, 2024-03-23T18:08:38.233 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.002349806 2024-03-23T18:08:38.234 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:38.234 INFO:teuthology.orchestra.run.smithi152.stdout: "slow_reply": 0, 2024-03-23T18:08:38.234 INFO:teuthology.orchestra.run.smithi152.stdout: "forward": 6996, 2024-03-23T18:08:38.234 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_fetch_complete": 0, 2024-03-23T18:08:38.234 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_fetch_keys": 0, 2024-03-23T18:08:38.234 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_commit": 46, 2024-03-23T18:08:38.234 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_split": 4, 2024-03-23T18:08:38.234 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_merge": 3, 2024-03-23T18:08:38.234 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes": 1214, 2024-03-23T18:08:38.235 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_top": 378, 2024-03-23T18:08:38.235 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_bottom": 756, 2024-03-23T18:08:38.235 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_pin_tail": 80, 2024-03-23T18:08:38.235 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_pinned": 671, 2024-03-23T18:08:38.235 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_expired": 391, 2024-03-23T18:08:38.235 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_with_caps": 148, 2024-03-23T18:08:38.235 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": 148, 2024-03-23T18:08:38.235 INFO:teuthology.orchestra.run.smithi152.stdout: "subtrees": 27, 2024-03-23T18:08:38.236 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse": 225243, 2024-03-23T18:08:38.236 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_hit": 111784, 2024-03-23T18:08:38.236 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_forward": 6994, 2024-03-23T18:08:38.236 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_discover": 40, 2024-03-23T18:08:38.236 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_dir_fetch": 0, 2024-03-23T18:08:38.236 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_remote_ino": 0, 2024-03-23T18:08:38.236 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_lock": 1794, 2024-03-23T18:08:38.236 INFO:teuthology.orchestra.run.smithi152.stdout: "load_cent": 32021, 2024-03-23T18:08:38.236 INFO:teuthology.orchestra.run.smithi152.stdout: "q": 0, 2024-03-23T18:08:38.237 INFO:teuthology.orchestra.run.smithi152.stdout: "exported": 3, 2024-03-23T18:08:38.237 INFO:teuthology.orchestra.run.smithi152.stdout: "exported_inodes": 322, 2024-03-23T18:08:38.237 INFO:teuthology.orchestra.run.smithi152.stdout: "imported": 11, 2024-03-23T18:08:38.237 INFO:teuthology.orchestra.run.smithi152.stdout: "imported_inodes": 623, 2024-03-23T18:08:38.237 INFO:teuthology.orchestra.run.smithi152.stdout: "openino_dir_fetch": 0, 2024-03-23T18:08:38.237 INFO:teuthology.orchestra.run.smithi152.stdout: "openino_backtrace_fetch": 19, 2024-03-23T18:08:38.237 INFO:teuthology.orchestra.run.smithi152.stdout: "openino_peer_discover": 0, 2024-03-23T18:08:38.237 INFO:teuthology.orchestra.run.smithi152.stdout: "root_rfiles": 719, 2024-03-23T18:08:38.238 INFO:teuthology.orchestra.run.smithi152.stdout: "root_rbytes": 215584975, 2024-03-23T18:08:38.238 INFO:teuthology.orchestra.run.smithi152.stdout: "root_rsnaps": 7, 2024-03-23T18:08:38.238 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_backtrace_fetch": 19, 2024-03-23T18:08:38.238 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_set_tag": 0, 2024-03-23T18:08:38.238 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_backtrace_repaired": 0, 2024-03-23T18:08:38.238 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_inotable_repaired": 0, 2024-03-23T18:08:38.238 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_dir_inodes": 0, 2024-03-23T18:08:38.238 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_dir_base_inodes": 0, 2024-03-23T18:08:38.239 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_dirfrag_rstats": 0, 2024-03-23T18:08:38.239 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_file_inodes": 19, 2024-03-23T18:08:38.239 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_inode_file_caps": 4, 2024-03-23T18:08:38.239 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_revoke": 13109, 2024-03-23T18:08:38.239 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_grant": 82391, 2024-03-23T18:08:38.239 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_trunc": 0, 2024-03-23T18:08:38.239 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-23T18:08:38.239 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-23T18:08:38.239 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_caps": 31415, 2024-03-23T18:08:38.240 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_caps_dirty": 9852, 2024-03-23T18:08:38.240 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_cap_release": 4294, 2024-03-23T18:08:38.240 INFO:teuthology.orchestra.run.smithi152.stdout: "process_request_cap_release": 101832 2024-03-23T18:08:38.240 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:38.240 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_cache": { 2024-03-23T18:08:38.240 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_update": 120, 2024-03-23T18:08:38.240 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_update_receipt": 106, 2024-03-23T18:08:38.240 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_try_discover": 66, 2024-03-23T18:08:38.241 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_send_discover": 113, 2024-03-23T18:08:38.241 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_handle_discover": 154, 2024-03-23T18:08:38.241 INFO:teuthology.orchestra.run.smithi152.stdout: "num_strays": 0, 2024-03-23T18:08:38.241 INFO:teuthology.orchestra.run.smithi152.stdout: "num_strays_delayed": 0, 2024-03-23T18:08:38.241 INFO:teuthology.orchestra.run.smithi152.stdout: "num_strays_enqueuing": 0, 2024-03-23T18:08:38.241 INFO:teuthology.orchestra.run.smithi152.stdout: "strays_created": 22555, 2024-03-23T18:08:38.241 INFO:teuthology.orchestra.run.smithi152.stdout: "strays_enqueued": 22555, 2024-03-23T18:08:38.241 INFO:teuthology.orchestra.run.smithi152.stdout: "strays_reintegrated": 0, 2024-03-23T18:08:38.242 INFO:teuthology.orchestra.run.smithi152.stdout: "strays_migrated": 0, 2024-03-23T18:08:38.242 INFO:teuthology.orchestra.run.smithi152.stdout: "num_recovering_processing": 0, 2024-03-23T18:08:38.242 INFO:teuthology.orchestra.run.smithi152.stdout: "num_recovering_enqueued": 0, 2024-03-23T18:08:38.242 INFO:teuthology.orchestra.run.smithi152.stdout: "num_recovering_prioritized": 0, 2024-03-23T18:08:38.242 INFO:teuthology.orchestra.run.smithi152.stdout: "recovery_started": 0, 2024-03-23T18:08:38.242 INFO:teuthology.orchestra.run.smithi152.stdout: "recovery_completed": 0, 2024-03-23T18:08:38.242 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_enqueue_scrub": 0, 2024-03-23T18:08:38.242 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_exportdir": 7, 2024-03-23T18:08:38.243 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_flush": 0, 2024-03-23T18:08:38.243 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_fragmentdir": 7, 2024-03-23T18:08:38.243 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_fragstats": 0, 2024-03-23T18:08:38.243 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_inodestats": 0 2024-03-23T18:08:38.243 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:38.243 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_log": { 2024-03-23T18:08:38.243 INFO:teuthology.orchestra.run.smithi152.stdout: "evadd": 105093, 2024-03-23T18:08:38.243 INFO:teuthology.orchestra.run.smithi152.stdout: "evex": 2048, 2024-03-23T18:08:38.243 INFO:teuthology.orchestra.run.smithi152.stdout: "evtrm": 2048, 2024-03-23T18:08:38.244 INFO:teuthology.orchestra.run.smithi152.stdout: "ev": 103045, 2024-03-23T18:08:38.244 INFO:teuthology.orchestra.run.smithi152.stdout: "evexg": 0, 2024-03-23T18:08:38.244 INFO:teuthology.orchestra.run.smithi152.stdout: "evexd": 1024, 2024-03-23T18:08:38.244 INFO:teuthology.orchestra.run.smithi152.stdout: "segadd": 130, 2024-03-23T18:08:38.244 INFO:teuthology.orchestra.run.smithi152.stdout: "segex": 2, 2024-03-23T18:08:38.244 INFO:teuthology.orchestra.run.smithi152.stdout: "segtrm": 2, 2024-03-23T18:08:38.244 INFO:teuthology.orchestra.run.smithi152.stdout: "seg": 129, 2024-03-23T18:08:38.244 INFO:teuthology.orchestra.run.smithi152.stdout: "segexg": 0, 2024-03-23T18:08:38.244 INFO:teuthology.orchestra.run.smithi152.stdout: "segexd": 1, 2024-03-23T18:08:38.245 INFO:teuthology.orchestra.run.smithi152.stdout: "expos": 7151064, 2024-03-23T18:08:38.245 INFO:teuthology.orchestra.run.smithi152.stdout: "wrpos": 351406651, 2024-03-23T18:08:38.245 INFO:teuthology.orchestra.run.smithi152.stdout: "rdpos": 0, 2024-03-23T18:08:38.245 INFO:teuthology.orchestra.run.smithi152.stdout: "jlat": { 2024-03-23T18:08:38.245 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 29555, 2024-03-23T18:08:38.245 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 157.168635914, 2024-03-23T18:08:38.245 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.005317835 2024-03-23T18:08:38.245 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:38.246 INFO:teuthology.orchestra.run.smithi152.stdout: "replayed": 0 2024-03-23T18:08:38.246 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:38.246 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_mem": { 2024-03-23T18:08:38.246 INFO:teuthology.orchestra.run.smithi152.stdout: "ino": 669, 2024-03-23T18:08:38.246 INFO:teuthology.orchestra.run.smithi152.stdout: "ino+": 23441, 2024-03-23T18:08:38.246 INFO:teuthology.orchestra.run.smithi152.stdout: "ino-": 22772, 2024-03-23T18:08:38.246 INFO:teuthology.orchestra.run.smithi152.stdout: "dir": 138, 2024-03-23T18:08:38.246 INFO:teuthology.orchestra.run.smithi152.stdout: "dir+": 232, 2024-03-23T18:08:38.246 INFO:teuthology.orchestra.run.smithi152.stdout: "dir-": 94, 2024-03-23T18:08:38.247 INFO:teuthology.orchestra.run.smithi152.stdout: "dn": 1214, 2024-03-23T18:08:38.247 INFO:teuthology.orchestra.run.smithi152.stdout: "dn+": 36789, 2024-03-23T18:08:38.247 INFO:teuthology.orchestra.run.smithi152.stdout: "dn-": 35575, 2024-03-23T18:08:38.247 INFO:teuthology.orchestra.run.smithi152.stdout: "cap": 148, 2024-03-23T18:08:38.247 INFO:teuthology.orchestra.run.smithi152.stdout: "cap+": 23838, 2024-03-23T18:08:38.247 INFO:teuthology.orchestra.run.smithi152.stdout: "cap-": 23690, 2024-03-23T18:08:38.247 INFO:teuthology.orchestra.run.smithi152.stdout: "rss": 70152, 2024-03-23T18:08:38.247 INFO:teuthology.orchestra.run.smithi152.stdout: "heap": 198964 2024-03-23T18:08:38.248 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:38.248 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_server": { 2024-03-23T18:08:38.248 INFO:teuthology.orchestra.run.smithi152.stdout: "dispatch_client_request": 201188, 2024-03-23T18:08:38.248 INFO:teuthology.orchestra.run.smithi152.stdout: "dispatch_server_request": 1, 2024-03-23T18:08:38.248 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_request": 125728, 2024-03-23T18:08:38.248 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_session": 1461, 2024-03-23T18:08:38.248 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_peer_request": 15510, 2024-03-23T18:08:38.248 INFO:teuthology.orchestra.run.smithi152.stdout: "req_create_latency": { 2024-03-23T18:08:38.248 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 22564, 2024-03-23T18:08:38.249 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 70.607001720, 2024-03-23T18:08:38.249 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.003129188 2024-03-23T18:08:38.249 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:38.249 INFO:teuthology.orchestra.run.smithi152.stdout: "req_getattr_latency": { 2024-03-23T18:08:38.249 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 3423, 2024-03-23T18:08:38.249 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 32.669513676, 2024-03-23T18:08:38.249 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.009544117 2024-03-23T18:08:38.249 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:38.250 INFO:teuthology.orchestra.run.smithi152.stdout: "req_getfilelock_latency": { 2024-03-23T18:08:38.250 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:38.250 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:38.250 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:38.250 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:38.250 INFO:teuthology.orchestra.run.smithi152.stdout: "req_link_latency": { 2024-03-23T18:08:38.250 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:38.250 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:38.251 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:38.251 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:38.251 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookup_latency": { 2024-03-23T18:08:38.251 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 27272, 2024-03-23T18:08:38.251 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 23.647125044, 2024-03-23T18:08:38.251 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000867084 2024-03-23T18:08:38.251 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:38.251 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookuphash_latency": { 2024-03-23T18:08:38.251 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:38.252 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:38.252 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:38.252 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:38.252 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookupino_latency": { 2024-03-23T18:08:38.252 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:38.252 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:38.252 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:38.252 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:38.253 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookupname_latency": { 2024-03-23T18:08:38.253 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:38.253 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:38.253 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:38.253 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:38.253 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookupparent_latency": { 2024-03-23T18:08:38.253 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:38.253 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:38.253 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:38.254 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:38.254 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookupsnap_latency": { 2024-03-23T18:08:38.254 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:38.254 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:38.254 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:38.254 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:38.254 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lssnap_latency": { 2024-03-23T18:08:38.254 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:38.255 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:38.255 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:38.255 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:38.255 INFO:teuthology.orchestra.run.smithi152.stdout: "req_mkdir_latency": { 2024-03-23T18:08:38.255 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:38.255 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:38.255 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:38.255 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:38.255 INFO:teuthology.orchestra.run.smithi152.stdout: "req_mknod_latency": { 2024-03-23T18:08:38.256 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:38.256 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:38.256 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:38.256 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:38.256 INFO:teuthology.orchestra.run.smithi152.stdout: "req_mksnap_latency": { 2024-03-23T18:08:38.256 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:38.256 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:38.256 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:38.256 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:38.257 INFO:teuthology.orchestra.run.smithi152.stdout: "req_open_latency": { 2024-03-23T18:08:38.257 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:38.257 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:38.257 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:38.257 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:38.257 INFO:teuthology.orchestra.run.smithi152.stdout: "req_readdir_latency": { 2024-03-23T18:08:38.257 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 39121, 2024-03-23T18:08:38.257 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 86.460639334, 2024-03-23T18:08:38.258 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.002210082 2024-03-23T18:08:38.258 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:38.258 INFO:teuthology.orchestra.run.smithi152.stdout: "req_rename_latency": { 2024-03-23T18:08:38.258 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 3482, 2024-03-23T18:08:38.258 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 12.986160487, 2024-03-23T18:08:38.258 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.003729511 2024-03-23T18:08:38.258 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:38.258 INFO:teuthology.orchestra.run.smithi152.stdout: "req_renamesnap_latency": { 2024-03-23T18:08:38.258 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:38.259 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:38.259 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:38.259 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:38.259 INFO:teuthology.orchestra.run.smithi152.stdout: "req_snapdiff_latency": { 2024-03-23T18:08:38.259 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:38.259 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:38.259 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:38.259 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:38.259 INFO:teuthology.orchestra.run.smithi152.stdout: "req_rmdir_latency": { 2024-03-23T18:08:38.260 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:38.260 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:38.260 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:38.260 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:38.260 INFO:teuthology.orchestra.run.smithi152.stdout: "req_rmsnap_latency": { 2024-03-23T18:08:38.260 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:38.260 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:38.260 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:38.260 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:38.261 INFO:teuthology.orchestra.run.smithi152.stdout: "req_rmxattr_latency": { 2024-03-23T18:08:38.261 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:38.261 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:38.261 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:38.261 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:38.261 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setattr_latency": { 2024-03-23T18:08:38.261 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 11, 2024-03-23T18:08:38.261 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.179992869, 2024-03-23T18:08:38.261 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.016362988 2024-03-23T18:08:38.262 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:38.262 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setdirlayout_latency": { 2024-03-23T18:08:38.262 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:38.262 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:38.262 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:38.262 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:38.262 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setfilelock_latency": { 2024-03-23T18:08:38.262 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 304, 2024-03-23T18:08:38.263 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.168430107, 2024-03-23T18:08:38.263 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000554046 2024-03-23T18:08:38.263 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:38.263 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setlayout_latency": { 2024-03-23T18:08:38.263 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:38.263 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:38.263 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:38.263 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:38.263 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setxattr_latency": { 2024-03-23T18:08:38.264 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:38.264 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:38.264 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:38.264 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:38.264 INFO:teuthology.orchestra.run.smithi152.stdout: "req_symlink_latency": { 2024-03-23T18:08:38.264 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:38.264 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:38.264 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:38.264 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:38.265 INFO:teuthology.orchestra.run.smithi152.stdout: "req_unlink_latency": { 2024-03-23T18:08:38.265 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 22555, 2024-03-23T18:08:38.265 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 52.278340689, 2024-03-23T18:08:38.265 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.002317816 2024-03-23T18:08:38.265 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:38.265 INFO:teuthology.orchestra.run.smithi152.stdout: "cap_revoke_eviction": 0, 2024-03-23T18:08:38.265 INFO:teuthology.orchestra.run.smithi152.stdout: "cap_acquisition_throttle": 0, 2024-03-23T18:08:38.265 INFO:teuthology.orchestra.run.smithi152.stdout: "req_getvxattr_latency": { 2024-03-23T18:08:38.265 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:38.266 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:38.266 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:38.266 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-03-23T18:08:38.266 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:38.266 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_sessions": { 2024-03-23T18:08:38.266 INFO:teuthology.orchestra.run.smithi152.stdout: "session_count": 1, 2024-03-23T18:08:38.266 INFO:teuthology.orchestra.run.smithi152.stdout: "session_add": 3, 2024-03-23T18:08:38.266 INFO:teuthology.orchestra.run.smithi152.stdout: "session_remove": 2, 2024-03-23T18:08:38.267 INFO:teuthology.orchestra.run.smithi152.stdout: "sessions_open": 1, 2024-03-23T18:08:38.267 INFO:teuthology.orchestra.run.smithi152.stdout: "sessions_stale": 0, 2024-03-23T18:08:38.267 INFO:teuthology.orchestra.run.smithi152.stdout: "total_load": 20833, 2024-03-23T18:08:38.267 INFO:teuthology.orchestra.run.smithi152.stdout: "average_load": 20833, 2024-03-23T18:08:38.267 INFO:teuthology.orchestra.run.smithi152.stdout: "avg_session_uptime": 1017, 2024-03-23T18:08:38.267 INFO:teuthology.orchestra.run.smithi152.stdout: "mdthresh_evicted": 0 2024-03-23T18:08:38.267 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:38.267 INFO:teuthology.orchestra.run.smithi152.stdout: "mempool": { 2024-03-23T18:08:38.267 INFO:teuthology.orchestra.run.smithi152.stdout: "bloom_filter_bytes": 0, 2024-03-23T18:08:38.268 INFO:teuthology.orchestra.run.smithi152.stdout: "bloom_filter_items": 0, 2024-03-23T18:08:38.268 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_alloc_bytes": 0, 2024-03-23T18:08:38.268 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_alloc_items": 0, 2024-03-23T18:08:38.268 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_data_bytes": 0, 2024-03-23T18:08:38.268 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_data_items": 0, 2024-03-23T18:08:38.268 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-23T18:08:38.268 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_onode_items": 0, 2024-03-23T18:08:38.268 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-23T18:08:38.268 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_meta_items": 0, 2024-03-23T18:08:38.269 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_other_bytes": 0, 2024-03-23T18:08:38.269 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_other_items": 0, 2024-03-23T18:08:38.269 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-23T18:08:38.269 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_buffer_items": 0, 2024-03-23T18:08:38.269 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_extent_bytes": 0, 2024-03-23T18:08:38.269 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_extent_items": 0, 2024-03-23T18:08:38.269 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_blob_bytes": 0, 2024-03-23T18:08:38.269 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_blob_items": 0, 2024-03-23T18:08:38.269 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-23T18:08:38.270 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_shared_blob_items": 0, 2024-03-23T18:08:38.270 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-23T18:08:38.270 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_inline_bl_items": 0, 2024-03-23T18:08:38.270 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_fsck_bytes": 0, 2024-03-23T18:08:38.270 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_fsck_items": 0, 2024-03-23T18:08:38.270 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_txc_bytes": 0, 2024-03-23T18:08:38.270 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_txc_items": 0, 2024-03-23T18:08:38.270 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-23T18:08:38.271 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_deferred_items": 0, 2024-03-23T18:08:38.271 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_bytes": 0, 2024-03-23T18:08:38.271 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_items": 0, 2024-03-23T18:08:38.271 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_bytes": 0, 2024-03-23T18:08:38.271 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_items": 0, 2024-03-23T18:08:38.271 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_reader_bytes": 0, 2024-03-23T18:08:38.271 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_reader_items": 0, 2024-03-23T18:08:38.271 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_writer_bytes": 0, 2024-03-23T18:08:38.271 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_writer_items": 0, 2024-03-23T18:08:38.271 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_anon_bytes": 332944, 2024-03-23T18:08:38.272 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_anon_items": 598, 2024-03-23T18:08:38.272 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_meta_bytes": 264, 2024-03-23T18:08:38.272 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_meta_items": 3, 2024-03-23T18:08:38.272 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_bytes": 0, 2024-03-23T18:08:38.272 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_items": 0, 2024-03-23T18:08:38.272 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_mapbl_bytes": 0, 2024-03-23T18:08:38.272 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_mapbl_items": 0, 2024-03-23T18:08:38.272 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_pglog_bytes": 0, 2024-03-23T18:08:38.273 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_pglog_items": 0, 2024-03-23T18:08:38.273 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_bytes": 7936, 2024-03-23T18:08:38.273 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_items": 165, 2024-03-23T18:08:38.273 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_mapping_bytes": 0, 2024-03-23T18:08:38.273 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_mapping_items": 0, 2024-03-23T18:08:38.273 INFO:teuthology.orchestra.run.smithi152.stdout: "pgmap_bytes": 0, 2024-03-23T18:08:38.273 INFO:teuthology.orchestra.run.smithi152.stdout: "pgmap_items": 0, 2024-03-23T18:08:38.273 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_co_bytes": 2880285, 2024-03-23T18:08:38.273 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_co_items": 26419, 2024-03-23T18:08:38.274 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_1_bytes": 0, 2024-03-23T18:08:38.274 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_1_items": 0, 2024-03-23T18:08:38.274 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_2_bytes": 0, 2024-03-23T18:08:38.274 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_2_items": 0 2024-03-23T18:08:38.274 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:38.274 INFO:teuthology.orchestra.run.smithi152.stdout: "objecter": { 2024-03-23T18:08:38.274 INFO:teuthology.orchestra.run.smithi152.stdout: "op_active": 0, 2024-03-23T18:08:38.274 INFO:teuthology.orchestra.run.smithi152.stdout: "op_laggy": 0, 2024-03-23T18:08:38.274 INFO:teuthology.orchestra.run.smithi152.stdout: "op_send": 54727, 2024-03-23T18:08:38.275 INFO:teuthology.orchestra.run.smithi152.stdout: "op_send_bytes": 351153885, 2024-03-23T18:08:38.275 INFO:teuthology.orchestra.run.smithi152.stdout: "op_resend": 33, 2024-03-23T18:08:38.275 INFO:teuthology.orchestra.run.smithi152.stdout: "op_reply": 54727, 2024-03-23T18:08:38.275 INFO:teuthology.orchestra.run.smithi152.stdout: "op_latency": { 2024-03-23T18:08:38.275 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 54727, 2024-03-23T18:08:38.275 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 315.252493431, 2024-03-23T18:08:38.275 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.005760456 2024-03-23T18:08:38.275 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:38.275 INFO:teuthology.orchestra.run.smithi152.stdout: "op_inflight": 0, 2024-03-23T18:08:38.276 INFO:teuthology.orchestra.run.smithi152.stdout: "oplen_avg": { 2024-03-23T18:08:38.276 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 54727, 2024-03-23T18:08:38.276 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 55185 2024-03-23T18:08:38.276 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:38.276 INFO:teuthology.orchestra.run.smithi152.stdout: "op": 54727, 2024-03-23T18:08:38.276 INFO:teuthology.orchestra.run.smithi152.stdout: "op_r": 882, 2024-03-23T18:08:38.276 INFO:teuthology.orchestra.run.smithi152.stdout: "op_w": 53845, 2024-03-23T18:08:38.276 INFO:teuthology.orchestra.run.smithi152.stdout: "op_rmw": 0, 2024-03-23T18:08:38.276 INFO:teuthology.orchestra.run.smithi152.stdout: "op_pg": 0, 2024-03-23T18:08:38.277 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_stat": 0, 2024-03-23T18:08:38.277 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_create": 13, 2024-03-23T18:08:38.277 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_read": 863, 2024-03-23T18:08:38.277 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_write": 30418, 2024-03-23T18:08:38.277 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_writefull": 486, 2024-03-23T18:08:38.277 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_writesame": 0, 2024-03-23T18:08:38.277 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_append": 0, 2024-03-23T18:08:38.277 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_zero": 0, 2024-03-23T18:08:38.277 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_truncate": 4, 2024-03-23T18:08:38.278 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_delete": 22673, 2024-03-23T18:08:38.278 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_mapext": 0, 2024-03-23T18:08:38.278 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_sparse_read": 0, 2024-03-23T18:08:38.278 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_clonerange": 0, 2024-03-23T18:08:38.278 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_getxattr": 19, 2024-03-23T18:08:38.278 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_setxattr": 26, 2024-03-23T18:08:38.278 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_cmpxattr": 0, 2024-03-23T18:08:38.278 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_rmxattr": 0, 2024-03-23T18:08:38.279 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_resetxattrs": 0, 2024-03-23T18:08:38.279 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_call": 0, 2024-03-23T18:08:38.279 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_watch": 0, 2024-03-23T18:08:38.279 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_notify": 0, 2024-03-23T18:08:38.279 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_src_cmpxattr": 0, 2024-03-23T18:08:38.279 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_pgls": 0, 2024-03-23T18:08:38.279 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_pgls_filter": 0, 2024-03-23T18:08:38.279 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_other": 0, 2024-03-23T18:08:38.279 INFO:teuthology.orchestra.run.smithi152.stdout: "linger_active": 0, 2024-03-23T18:08:38.280 INFO:teuthology.orchestra.run.smithi152.stdout: "linger_send": 0, 2024-03-23T18:08:38.280 INFO:teuthology.orchestra.run.smithi152.stdout: "linger_resend": 0, 2024-03-23T18:08:38.280 INFO:teuthology.orchestra.run.smithi152.stdout: "linger_ping": 0, 2024-03-23T18:08:38.280 INFO:teuthology.orchestra.run.smithi152.stdout: "poolop_active": 0, 2024-03-23T18:08:38.280 INFO:teuthology.orchestra.run.smithi152.stdout: "poolop_send": 0, 2024-03-23T18:08:38.280 INFO:teuthology.orchestra.run.smithi152.stdout: "poolop_resend": 0, 2024-03-23T18:08:38.280 INFO:teuthology.orchestra.run.smithi152.stdout: "poolstat_active": 0, 2024-03-23T18:08:38.280 INFO:teuthology.orchestra.run.smithi152.stdout: "poolstat_send": 0, 2024-03-23T18:08:38.280 INFO:teuthology.orchestra.run.smithi152.stdout: "poolstat_resend": 0, 2024-03-23T18:08:38.281 INFO:teuthology.orchestra.run.smithi152.stdout: "statfs_active": 0, 2024-03-23T18:08:38.281 INFO:teuthology.orchestra.run.smithi152.stdout: "statfs_send": 0, 2024-03-23T18:08:38.281 INFO:teuthology.orchestra.run.smithi152.stdout: "statfs_resend": 0, 2024-03-23T18:08:38.281 INFO:teuthology.orchestra.run.smithi152.stdout: "command_active": 0, 2024-03-23T18:08:38.281 INFO:teuthology.orchestra.run.smithi152.stdout: "command_send": 0, 2024-03-23T18:08:38.281 INFO:teuthology.orchestra.run.smithi152.stdout: "command_resend": 0, 2024-03-23T18:08:38.281 INFO:teuthology.orchestra.run.smithi152.stdout: "map_epoch": 110, 2024-03-23T18:08:38.281 INFO:teuthology.orchestra.run.smithi152.stdout: "map_full": 0, 2024-03-23T18:08:38.281 INFO:teuthology.orchestra.run.smithi152.stdout: "map_inc": 30, 2024-03-23T18:08:38.282 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_sessions": 12, 2024-03-23T18:08:38.282 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_session_open": 12, 2024-03-23T18:08:38.282 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_session_close": 0, 2024-03-23T18:08:38.282 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_laggy": 0, 2024-03-23T18:08:38.282 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_wr": 485, 2024-03-23T18:08:38.282 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_rd": 0, 2024-03-23T18:08:38.282 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_del": 198 2024-03-23T18:08:38.282 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:38.282 INFO:teuthology.orchestra.run.smithi152.stdout: "oft": { 2024-03-23T18:08:38.283 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_total_objs": 1, 2024-03-23T18:08:38.283 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_total_kv_pairs": 0, 2024-03-23T18:08:38.283 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_total_updates": 0, 2024-03-23T18:08:38.283 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_total_removes": 0 2024-03-23T18:08:38.283 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:38.283 INFO:teuthology.orchestra.run.smithi152.stdout: "osd-slow-ops": { 2024-03-23T18:08:38.283 INFO:teuthology.orchestra.run.smithi152.stdout: "slow_ops_count": 125743 2024-03-23T18:08:38.283 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:38.283 INFO:teuthology.orchestra.run.smithi152.stdout: "purge_queue": { 2024-03-23T18:08:38.284 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executing_ops": 0, 2024-03-23T18:08:38.284 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executing_ops_high_water": 6, 2024-03-23T18:08:38.284 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executing": 0, 2024-03-23T18:08:38.284 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executing_high_water": 6, 2024-03-23T18:08:38.284 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executed": 22555, 2024-03-23T18:08:38.284 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_item_in_journal": 0 2024-03-23T18:08:38.284 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:38.284 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-23T18:08:38.284 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-03-23T18:08:38.285 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 104857600, 2024-03-23T18:08:38.285 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-03-23T18:08:38.285 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 395385, 2024-03-23T18:08:38.285 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 96525401, 2024-03-23T18:08:38.285 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-03-23T18:08:38.285 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 395385, 2024-03-23T18:08:38.285 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 0, 2024-03-23T18:08:38.285 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 0, 2024-03-23T18:08:38.285 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 395385, 2024-03-23T18:08:38.286 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 96525401, 2024-03-23T18:08:38.286 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-03-23T18:08:38.286 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:38.286 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:38.286 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:38.286 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-03-23T18:08:38.286 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:38.286 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-objecter_bytes": { 2024-03-23T18:08:38.286 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-03-23T18:08:38.287 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 104857600, 2024-03-23T18:08:38.287 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-03-23T18:08:38.287 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 0, 2024-03-23T18:08:38.287 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 0, 2024-03-23T18:08:38.287 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-03-23T18:08:38.287 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 0, 2024-03-23T18:08:38.287 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 54727, 2024-03-23T18:08:38.287 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 353434108, 2024-03-23T18:08:38.287 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 54727, 2024-03-23T18:08:38.288 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 353434108, 2024-03-23T18:08:38.288 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-03-23T18:08:38.288 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:38.288 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:38.288 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:38.288 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-03-23T18:08:38.288 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:38.288 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-objecter_ops": { 2024-03-23T18:08:38.288 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-03-23T18:08:38.289 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 1024, 2024-03-23T18:08:38.289 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-03-23T18:08:38.289 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 0, 2024-03-23T18:08:38.289 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 0, 2024-03-23T18:08:38.289 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-03-23T18:08:38.289 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 0, 2024-03-23T18:08:38.289 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 54727, 2024-03-23T18:08:38.289 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 54727, 2024-03-23T18:08:38.289 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 54727, 2024-03-23T18:08:38.290 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 54727, 2024-03-23T18:08:38.290 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-03-23T18:08:38.290 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:38.290 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:38.290 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:38.290 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-03-23T18:08:38.290 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:38.290 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-write_buf_throttle": { 2024-03-23T18:08:38.291 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-03-23T18:08:38.291 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 3758096384, 2024-03-23T18:08:38.291 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-03-23T18:08:38.291 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 22555, 2024-03-23T18:08:38.291 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 2280223, 2024-03-23T18:08:38.291 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-03-23T18:08:38.291 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 22555, 2024-03-23T18:08:38.291 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 0, 2024-03-23T18:08:38.291 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 0, 2024-03-23T18:08:38.292 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 863, 2024-03-23T18:08:38.292 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 2280223, 2024-03-23T18:08:38.292 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-03-23T18:08:38.292 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:38.292 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:38.292 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:38.292 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-03-23T18:08:38.292 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:38.293 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-write_buf_throttle-0x55f1200ce0a0": { 2024-03-23T18:08:38.293 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-03-23T18:08:38.293 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 3758096384, 2024-03-23T18:08:38.293 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-03-23T18:08:38.293 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 105093, 2024-03-23T18:08:38.293 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 347212347, 2024-03-23T18:08:38.293 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-03-23T18:08:38.293 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 105093, 2024-03-23T18:08:38.293 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 0, 2024-03-23T18:08:38.294 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 0, 2024-03-23T18:08:38.294 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 29555, 2024-03-23T18:08:38.294 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 347212347, 2024-03-23T18:08:38.294 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-03-23T18:08:38.294 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:38.294 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:38.294 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:38.294 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-03-23T18:08:38.294 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-03-23T18:08:38.295 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-03-23T18:08:38.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:38 smithi092 ceph-mon[79089]: pgmap v569: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 6.2 MiB/s wr, 237 op/s 2024-03-23T18:08:38.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:38 smithi152 ceph-mon[47949]: pgmap v569: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 6.2 MiB/s wr, 237 op/s 2024-03-23T18:08:38.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:38 smithi086 ceph-mon[38221]: pgmap v569: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 6.2 MiB/s wr, 237 op/s 2024-03-23T18:08:38.587 INFO:tasks.check_counter:Daemon mds.i mds.exported=3 2024-03-23T18:08:38.587 INFO:tasks.check_counter:Daemon mds.i mds.imported=11 2024-03-23T18:08:38.587 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=7 2024-03-23T18:08:38.587 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-03-23T18:08:38.588 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-03-23T18:08:38.588 DEBUG:tasks.check_counter:Getting stats from l 2024-03-23T18:08:38.588 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 shell --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 -- ceph daemon mds.l perf dump 2024-03-23T18:08:38.810 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/mon.c/config 2024-03-23T18:08:39.798 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-03-23T18:08:39.798 INFO:teuthology.orchestra.run.smithi152.stdout: "AsyncMessenger::Worker-0": { 2024-03-23T18:08:39.799 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_messages": 1266, 2024-03-23T18:08:39.799 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages": 1268, 2024-03-23T18:08:39.799 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_bytes": 134751529, 2024-03-23T18:08:39.799 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_bytes": 393790, 2024-03-23T18:08:39.799 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_created_connections": 8, 2024-03-23T18:08:39.799 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_active_connections": 4, 2024-03-23T18:08:39.799 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_total_time": 0.167829361, 2024-03-23T18:08:39.799 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_send_time": 0.026781216, 2024-03-23T18:08:39.800 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_recv_time": 0.099129620, 2024-03-23T18:08:39.800 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_fast_dispatch_time": 0.031633223, 2024-03-23T18:08:39.800 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T18:08:39.800 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 1266, 2024-03-23T18:08:39.800 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.064919557, 2024-03-23T18:08:39.800 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000051279 2024-03-23T18:08:39.800 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:39.800 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_handle_ack_lat": { 2024-03-23T18:08:39.801 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:39.801 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:39.801 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:39.801 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:39.801 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-23T18:08:39.801 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_encrypted_bytes": 0 2024-03-23T18:08:39.801 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:39.801 INFO:teuthology.orchestra.run.smithi152.stdout: "AsyncMessenger::Worker-1": { 2024-03-23T18:08:39.801 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_messages": 1389, 2024-03-23T18:08:39.802 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages": 1532, 2024-03-23T18:08:39.802 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_bytes": 119685752, 2024-03-23T18:08:39.802 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_bytes": 1037394, 2024-03-23T18:08:39.802 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_created_connections": 7, 2024-03-23T18:08:39.802 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_active_connections": 5, 2024-03-23T18:08:39.802 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_total_time": 0.173279903, 2024-03-23T18:08:39.802 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_send_time": 0.044503712, 2024-03-23T18:08:39.802 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_recv_time": 0.105927313, 2024-03-23T18:08:39.802 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_fast_dispatch_time": 0.027325995, 2024-03-23T18:08:39.803 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T18:08:39.803 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 1531, 2024-03-23T18:08:39.803 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.118250691, 2024-03-23T18:08:39.803 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000077237 2024-03-23T18:08:39.803 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:39.803 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_handle_ack_lat": { 2024-03-23T18:08:39.803 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:39.803 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:39.803 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:39.804 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:39.804 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_encrypted_bytes": 2002848, 2024-03-23T18:08:39.804 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_encrypted_bytes": 740384 2024-03-23T18:08:39.804 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:39.804 INFO:teuthology.orchestra.run.smithi152.stdout: "AsyncMessenger::Worker-2": { 2024-03-23T18:08:39.804 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_messages": 2363, 2024-03-23T18:08:39.804 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages": 2389, 2024-03-23T18:08:39.804 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_bytes": 147507958, 2024-03-23T18:08:39.804 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_bytes": 850266, 2024-03-23T18:08:39.805 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_created_connections": 8, 2024-03-23T18:08:39.805 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_active_connections": 6, 2024-03-23T18:08:39.805 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_total_time": 0.284952637, 2024-03-23T18:08:39.805 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_send_time": 0.053260523, 2024-03-23T18:08:39.805 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_recv_time": 0.144798146, 2024-03-23T18:08:39.806 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_fast_dispatch_time": 0.067367691, 2024-03-23T18:08:39.806 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T18:08:39.806 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 2387, 2024-03-23T18:08:39.806 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.109369184, 2024-03-23T18:08:39.806 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000045818 2024-03-23T18:08:39.806 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:39.807 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_handle_ack_lat": { 2024-03-23T18:08:39.807 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:39.807 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:39.807 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:39.807 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:39.807 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-03-23T18:08:39.807 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_encrypted_bytes": 116928 2024-03-23T18:08:39.808 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:39.808 INFO:teuthology.orchestra.run.smithi152.stdout: "cct": { 2024-03-23T18:08:39.808 INFO:teuthology.orchestra.run.smithi152.stdout: "total_workers": 1, 2024-03-23T18:08:39.808 INFO:teuthology.orchestra.run.smithi152.stdout: "unhealthy_workers": 0 2024-03-23T18:08:39.808 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:39.808 INFO:teuthology.orchestra.run.smithi152.stdout: "finisher-MDSRank": { 2024-03-23T18:08:39.808 INFO:teuthology.orchestra.run.smithi152.stdout: "queue_len": 0, 2024-03-23T18:08:39.808 INFO:teuthology.orchestra.run.smithi152.stdout: "complete_latency": { 2024-03-23T18:08:39.809 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 7604, 2024-03-23T18:08:39.809 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.172529485, 2024-03-23T18:08:39.809 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000022689 2024-03-23T18:08:39.809 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-03-23T18:08:39.810 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:39.810 INFO:teuthology.orchestra.run.smithi152.stdout: "finisher-PurgeQueue": { 2024-03-23T18:08:39.810 INFO:teuthology.orchestra.run.smithi152.stdout: "queue_len": 0, 2024-03-23T18:08:39.810 INFO:teuthology.orchestra.run.smithi152.stdout: "complete_latency": { 2024-03-23T18:08:39.810 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:39.810 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:39.810 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:39.811 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-03-23T18:08:39.811 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:39.811 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": { 2024-03-23T18:08:39.811 INFO:teuthology.orchestra.run.smithi152.stdout: "request": 0, 2024-03-23T18:08:39.811 INFO:teuthology.orchestra.run.smithi152.stdout: "reply": 0, 2024-03-23T18:08:39.811 INFO:teuthology.orchestra.run.smithi152.stdout: "reply_latency": { 2024-03-23T18:08:39.811 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:39.811 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:39.811 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:39.812 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:39.812 INFO:teuthology.orchestra.run.smithi152.stdout: "slow_reply": 0, 2024-03-23T18:08:39.812 INFO:teuthology.orchestra.run.smithi152.stdout: "forward": 0, 2024-03-23T18:08:39.812 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_fetch_complete": 0, 2024-03-23T18:08:39.812 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_fetch_keys": 0, 2024-03-23T18:08:39.812 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_commit": 0, 2024-03-23T18:08:39.812 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_split": 0, 2024-03-23T18:08:39.812 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_merge": 0, 2024-03-23T18:08:39.813 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes": 20116, 2024-03-23T18:08:39.813 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_top": 1020, 2024-03-23T18:08:39.813 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_bottom": 19096, 2024-03-23T18:08:39.813 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_pin_tail": 0, 2024-03-23T18:08:39.813 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_pinned": 18503, 2024-03-23T18:08:39.813 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_expired": 0, 2024-03-23T18:08:39.813 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_with_caps": 0, 2024-03-23T18:08:39.813 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": 0, 2024-03-23T18:08:39.813 INFO:teuthology.orchestra.run.smithi152.stdout: "subtrees": 15, 2024-03-23T18:08:39.814 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse": 0, 2024-03-23T18:08:39.814 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_hit": 0, 2024-03-23T18:08:39.814 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_forward": 0, 2024-03-23T18:08:39.814 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_discover": 0, 2024-03-23T18:08:39.814 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_dir_fetch": 0, 2024-03-23T18:08:39.814 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_remote_ino": 0, 2024-03-23T18:08:39.814 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_lock": 0, 2024-03-23T18:08:39.814 INFO:teuthology.orchestra.run.smithi152.stdout: "load_cent": 0, 2024-03-23T18:08:39.815 INFO:teuthology.orchestra.run.smithi152.stdout: "q": 0, 2024-03-23T18:08:39.815 INFO:teuthology.orchestra.run.smithi152.stdout: "exported": 0, 2024-03-23T18:08:39.815 INFO:teuthology.orchestra.run.smithi152.stdout: "exported_inodes": 0, 2024-03-23T18:08:39.815 INFO:teuthology.orchestra.run.smithi152.stdout: "imported": 0, 2024-03-23T18:08:39.815 INFO:teuthology.orchestra.run.smithi152.stdout: "imported_inodes": 0, 2024-03-23T18:08:39.815 INFO:teuthology.orchestra.run.smithi152.stdout: "openino_dir_fetch": 0, 2024-03-23T18:08:39.815 INFO:teuthology.orchestra.run.smithi152.stdout: "openino_backtrace_fetch": 0, 2024-03-23T18:08:39.815 INFO:teuthology.orchestra.run.smithi152.stdout: "openino_peer_discover": 0, 2024-03-23T18:08:39.816 INFO:teuthology.orchestra.run.smithi152.stdout: "root_rfiles": 0, 2024-03-23T18:08:39.816 INFO:teuthology.orchestra.run.smithi152.stdout: "root_rbytes": 0, 2024-03-23T18:08:39.816 INFO:teuthology.orchestra.run.smithi152.stdout: "root_rsnaps": 0, 2024-03-23T18:08:39.816 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_backtrace_fetch": 0, 2024-03-23T18:08:39.816 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_set_tag": 0, 2024-03-23T18:08:39.816 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_backtrace_repaired": 0, 2024-03-23T18:08:39.816 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_inotable_repaired": 0, 2024-03-23T18:08:39.817 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_dir_inodes": 0, 2024-03-23T18:08:39.817 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_dir_base_inodes": 0, 2024-03-23T18:08:39.817 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_dirfrag_rstats": 0, 2024-03-23T18:08:39.817 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_file_inodes": 0, 2024-03-23T18:08:39.817 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_inode_file_caps": 0, 2024-03-23T18:08:39.817 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_revoke": 0, 2024-03-23T18:08:39.817 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_grant": 0, 2024-03-23T18:08:39.818 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_trunc": 0, 2024-03-23T18:08:39.818 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-23T18:08:39.818 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-23T18:08:39.818 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_caps": 0, 2024-03-23T18:08:39.818 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_caps_dirty": 0, 2024-03-23T18:08:39.818 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_cap_release": 0, 2024-03-23T18:08:39.818 INFO:teuthology.orchestra.run.smithi152.stdout: "process_request_cap_release": 0 2024-03-23T18:08:39.819 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:39.819 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_cache": { 2024-03-23T18:08:39.819 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_update": 0, 2024-03-23T18:08:39.819 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_update_receipt": 0, 2024-03-23T18:08:39.819 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_try_discover": 0, 2024-03-23T18:08:39.819 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_send_discover": 0, 2024-03-23T18:08:39.819 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_handle_discover": 0, 2024-03-23T18:08:39.819 INFO:teuthology.orchestra.run.smithi152.stdout: "num_strays": 0, 2024-03-23T18:08:39.819 INFO:teuthology.orchestra.run.smithi152.stdout: "num_strays_delayed": 0, 2024-03-23T18:08:39.820 INFO:teuthology.orchestra.run.smithi152.stdout: "num_strays_enqueuing": 0, 2024-03-23T18:08:39.820 INFO:teuthology.orchestra.run.smithi152.stdout: "strays_created": 0, 2024-03-23T18:08:39.820 INFO:teuthology.orchestra.run.smithi152.stdout: "strays_enqueued": 0, 2024-03-23T18:08:39.820 INFO:teuthology.orchestra.run.smithi152.stdout: "strays_reintegrated": 0, 2024-03-23T18:08:39.820 INFO:teuthology.orchestra.run.smithi152.stdout: "strays_migrated": 0, 2024-03-23T18:08:39.820 INFO:teuthology.orchestra.run.smithi152.stdout: "num_recovering_processing": 0, 2024-03-23T18:08:39.820 INFO:teuthology.orchestra.run.smithi152.stdout: "num_recovering_enqueued": 0, 2024-03-23T18:08:39.820 INFO:teuthology.orchestra.run.smithi152.stdout: "num_recovering_prioritized": 0, 2024-03-23T18:08:39.821 INFO:teuthology.orchestra.run.smithi152.stdout: "recovery_started": 0, 2024-03-23T18:08:39.821 INFO:teuthology.orchestra.run.smithi152.stdout: "recovery_completed": 0, 2024-03-23T18:08:39.821 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_enqueue_scrub": 0, 2024-03-23T18:08:39.821 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_exportdir": 0, 2024-03-23T18:08:39.821 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_flush": 0, 2024-03-23T18:08:39.821 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_fragmentdir": 0, 2024-03-23T18:08:39.821 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_fragstats": 0, 2024-03-23T18:08:39.821 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_inodestats": 0 2024-03-23T18:08:39.821 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:39.822 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_log": { 2024-03-23T18:08:39.822 INFO:teuthology.orchestra.run.smithi152.stdout: "evadd": 0, 2024-03-23T18:08:39.822 INFO:teuthology.orchestra.run.smithi152.stdout: "evex": 0, 2024-03-23T18:08:39.822 INFO:teuthology.orchestra.run.smithi152.stdout: "evtrm": 0, 2024-03-23T18:08:39.822 INFO:teuthology.orchestra.run.smithi152.stdout: "ev": 107847, 2024-03-23T18:08:39.822 INFO:teuthology.orchestra.run.smithi152.stdout: "evexg": 0, 2024-03-23T18:08:39.822 INFO:teuthology.orchestra.run.smithi152.stdout: "evexd": 0, 2024-03-23T18:08:39.822 INFO:teuthology.orchestra.run.smithi152.stdout: "segadd": 0, 2024-03-23T18:08:39.822 INFO:teuthology.orchestra.run.smithi152.stdout: "segex": 0, 2024-03-23T18:08:39.823 INFO:teuthology.orchestra.run.smithi152.stdout: "segtrm": 0, 2024-03-23T18:08:39.823 INFO:teuthology.orchestra.run.smithi152.stdout: "seg": 128, 2024-03-23T18:08:39.823 INFO:teuthology.orchestra.run.smithi152.stdout: "segexg": 0, 2024-03-23T18:08:39.823 INFO:teuthology.orchestra.run.smithi152.stdout: "segexd": 0, 2024-03-23T18:08:39.823 INFO:teuthology.orchestra.run.smithi152.stdout: "expos": 19761768, 2024-03-23T18:08:39.823 INFO:teuthology.orchestra.run.smithi152.stdout: "wrpos": 402859668, 2024-03-23T18:08:39.823 INFO:teuthology.orchestra.run.smithi152.stdout: "rdpos": 402853425, 2024-03-23T18:08:39.823 INFO:teuthology.orchestra.run.smithi152.stdout: "jlat": { 2024-03-23T18:08:39.824 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:39.824 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:39.824 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:39.824 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:39.824 INFO:teuthology.orchestra.run.smithi152.stdout: "replayed": 107847 2024-03-23T18:08:39.824 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:39.824 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_mem": { 2024-03-23T18:08:39.824 INFO:teuthology.orchestra.run.smithi152.stdout: "ino": 862, 2024-03-23T18:08:39.824 INFO:teuthology.orchestra.run.smithi152.stdout: "ino+": 20249, 2024-03-23T18:08:39.825 INFO:teuthology.orchestra.run.smithi152.stdout: "ino-": 19387, 2024-03-23T18:08:39.825 INFO:teuthology.orchestra.run.smithi152.stdout: "dir": 73, 2024-03-23T18:08:39.825 INFO:teuthology.orchestra.run.smithi152.stdout: "dir+": 123, 2024-03-23T18:08:39.825 INFO:teuthology.orchestra.run.smithi152.stdout: "dir-": 50, 2024-03-23T18:08:39.825 INFO:teuthology.orchestra.run.smithi152.stdout: "dn": 20116, 2024-03-23T18:08:39.825 INFO:teuthology.orchestra.run.smithi152.stdout: "dn+": 20560, 2024-03-23T18:08:39.825 INFO:teuthology.orchestra.run.smithi152.stdout: "dn-": 444, 2024-03-23T18:08:39.825 INFO:teuthology.orchestra.run.smithi152.stdout: "cap": 0, 2024-03-23T18:08:39.825 INFO:teuthology.orchestra.run.smithi152.stdout: "cap+": 0, 2024-03-23T18:08:39.826 INFO:teuthology.orchestra.run.smithi152.stdout: "cap-": 0, 2024-03-23T18:08:39.826 INFO:teuthology.orchestra.run.smithi152.stdout: "rss": 73052, 2024-03-23T18:08:39.826 INFO:teuthology.orchestra.run.smithi152.stdout: "heap": 207156 2024-03-23T18:08:39.826 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:39.826 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_server": { 2024-03-23T18:08:39.826 INFO:teuthology.orchestra.run.smithi152.stdout: "dispatch_client_request": 0, 2024-03-23T18:08:39.826 INFO:teuthology.orchestra.run.smithi152.stdout: "dispatch_server_request": 0, 2024-03-23T18:08:39.826 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_request": 0, 2024-03-23T18:08:39.827 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_session": 0, 2024-03-23T18:08:39.827 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_peer_request": 0, 2024-03-23T18:08:39.827 INFO:teuthology.orchestra.run.smithi152.stdout: "req_create_latency": { 2024-03-23T18:08:39.827 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:39.827 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:39.827 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:39.827 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:39.827 INFO:teuthology.orchestra.run.smithi152.stdout: "req_getattr_latency": { 2024-03-23T18:08:39.828 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:39.828 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:39.828 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:39.828 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:39.828 INFO:teuthology.orchestra.run.smithi152.stdout: "req_getfilelock_latency": { 2024-03-23T18:08:39.828 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:39.828 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:39.828 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:39.829 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:39.829 INFO:teuthology.orchestra.run.smithi152.stdout: "req_link_latency": { 2024-03-23T18:08:39.829 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:39.829 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:39.829 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:39.829 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:39.829 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookup_latency": { 2024-03-23T18:08:39.829 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:39.829 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:39.830 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:39.830 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:39.830 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookuphash_latency": { 2024-03-23T18:08:39.830 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:39.830 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:39.830 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:39.830 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:39.830 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookupino_latency": { 2024-03-23T18:08:39.830 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:39.831 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:39.831 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:39.831 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:39.831 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookupname_latency": { 2024-03-23T18:08:39.831 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:39.831 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:39.831 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:39.831 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:39.832 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookupparent_latency": { 2024-03-23T18:08:39.832 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:39.832 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:39.832 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:39.832 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:39.832 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookupsnap_latency": { 2024-03-23T18:08:39.832 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:39.832 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:39.832 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:39.833 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:39.833 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lssnap_latency": { 2024-03-23T18:08:39.833 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:39.833 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:39.833 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:39.833 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:39.833 INFO:teuthology.orchestra.run.smithi152.stdout: "req_mkdir_latency": { 2024-03-23T18:08:39.833 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:39.833 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:39.834 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:39.834 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:39.834 INFO:teuthology.orchestra.run.smithi152.stdout: "req_mknod_latency": { 2024-03-23T18:08:39.834 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:39.834 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:39.834 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:39.834 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:39.835 INFO:teuthology.orchestra.run.smithi152.stdout: "req_mksnap_latency": { 2024-03-23T18:08:39.835 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:39.835 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:39.835 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:39.835 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:39.835 INFO:teuthology.orchestra.run.smithi152.stdout: "req_open_latency": { 2024-03-23T18:08:39.835 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:39.835 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:39.835 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:39.836 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:39.836 INFO:teuthology.orchestra.run.smithi152.stdout: "req_readdir_latency": { 2024-03-23T18:08:39.836 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:39.836 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:39.836 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:39.836 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:39.836 INFO:teuthology.orchestra.run.smithi152.stdout: "req_rename_latency": { 2024-03-23T18:08:39.837 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:39.837 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:39.837 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:39.837 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:39.837 INFO:teuthology.orchestra.run.smithi152.stdout: "req_renamesnap_latency": { 2024-03-23T18:08:39.837 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:39.837 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:39.837 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:39.837 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:39.838 INFO:teuthology.orchestra.run.smithi152.stdout: "req_snapdiff_latency": { 2024-03-23T18:08:39.838 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:39.838 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:39.838 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:39.838 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:39.838 INFO:teuthology.orchestra.run.smithi152.stdout: "req_rmdir_latency": { 2024-03-23T18:08:39.838 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:39.838 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:39.839 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:39.839 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:39.839 INFO:teuthology.orchestra.run.smithi152.stdout: "req_rmsnap_latency": { 2024-03-23T18:08:39.839 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:39.839 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:39.839 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:39.839 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:39.839 INFO:teuthology.orchestra.run.smithi152.stdout: "req_rmxattr_latency": { 2024-03-23T18:08:39.839 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:39.840 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:39.840 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:39.840 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:39.840 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setattr_latency": { 2024-03-23T18:08:39.840 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:39.840 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:39.840 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:39.840 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:39.840 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setdirlayout_latency": { 2024-03-23T18:08:39.841 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:39.841 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:39.841 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:39.841 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:39.841 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setfilelock_latency": { 2024-03-23T18:08:39.841 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:39.841 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:39.841 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:39.842 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:39.842 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setlayout_latency": { 2024-03-23T18:08:39.842 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:39.842 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:39.842 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:39.842 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:39.842 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setxattr_latency": { 2024-03-23T18:08:39.842 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:39.843 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:39.843 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:39.843 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:39.843 INFO:teuthology.orchestra.run.smithi152.stdout: "req_symlink_latency": { 2024-03-23T18:08:39.843 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:39.843 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:39.843 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:39.843 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:39.843 INFO:teuthology.orchestra.run.smithi152.stdout: "req_unlink_latency": { 2024-03-23T18:08:39.844 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:39.844 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:39.844 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:39.844 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:39.844 INFO:teuthology.orchestra.run.smithi152.stdout: "cap_revoke_eviction": 0, 2024-03-23T18:08:39.844 INFO:teuthology.orchestra.run.smithi152.stdout: "cap_acquisition_throttle": 0, 2024-03-23T18:08:39.844 INFO:teuthology.orchestra.run.smithi152.stdout: "req_getvxattr_latency": { 2024-03-23T18:08:39.844 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:39.844 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:39.845 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:39.845 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-03-23T18:08:39.845 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:39.845 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_sessions": { 2024-03-23T18:08:39.845 INFO:teuthology.orchestra.run.smithi152.stdout: "session_count": 2, 2024-03-23T18:08:39.845 INFO:teuthology.orchestra.run.smithi152.stdout: "session_add": 4, 2024-03-23T18:08:39.845 INFO:teuthology.orchestra.run.smithi152.stdout: "session_remove": 2, 2024-03-23T18:08:39.845 INFO:teuthology.orchestra.run.smithi152.stdout: "sessions_open": 4, 2024-03-23T18:08:39.845 INFO:teuthology.orchestra.run.smithi152.stdout: "sessions_stale": 0, 2024-03-23T18:08:39.846 INFO:teuthology.orchestra.run.smithi152.stdout: "total_load": 0, 2024-03-23T18:08:39.846 INFO:teuthology.orchestra.run.smithi152.stdout: "average_load": 0, 2024-03-23T18:08:39.846 INFO:teuthology.orchestra.run.smithi152.stdout: "avg_session_uptime": 4814, 2024-03-23T18:08:39.846 INFO:teuthology.orchestra.run.smithi152.stdout: "mdthresh_evicted": 0 2024-03-23T18:08:39.846 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:39.846 INFO:teuthology.orchestra.run.smithi152.stdout: "mempool": { 2024-03-23T18:08:39.846 INFO:teuthology.orchestra.run.smithi152.stdout: "bloom_filter_bytes": 0, 2024-03-23T18:08:39.846 INFO:teuthology.orchestra.run.smithi152.stdout: "bloom_filter_items": 0, 2024-03-23T18:08:39.847 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_alloc_bytes": 0, 2024-03-23T18:08:39.847 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_alloc_items": 0, 2024-03-23T18:08:39.847 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_data_bytes": 0, 2024-03-23T18:08:39.847 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_data_items": 0, 2024-03-23T18:08:39.847 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-23T18:08:39.847 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_onode_items": 0, 2024-03-23T18:08:39.847 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-23T18:08:39.847 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_meta_items": 0, 2024-03-23T18:08:39.848 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_other_bytes": 0, 2024-03-23T18:08:39.848 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_other_items": 0, 2024-03-23T18:08:39.848 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-23T18:08:39.848 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_buffer_items": 0, 2024-03-23T18:08:39.848 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_extent_bytes": 0, 2024-03-23T18:08:39.848 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_extent_items": 0, 2024-03-23T18:08:39.848 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_blob_bytes": 0, 2024-03-23T18:08:39.848 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_blob_items": 0, 2024-03-23T18:08:39.849 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-23T18:08:39.849 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_shared_blob_items": 0, 2024-03-23T18:08:39.849 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-23T18:08:39.849 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_inline_bl_items": 0, 2024-03-23T18:08:39.849 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_fsck_bytes": 0, 2024-03-23T18:08:39.849 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_fsck_items": 0, 2024-03-23T18:08:39.849 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_txc_bytes": 0, 2024-03-23T18:08:39.849 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_txc_items": 0, 2024-03-23T18:08:39.849 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-23T18:08:39.850 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_deferred_items": 0, 2024-03-23T18:08:39.850 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_bytes": 0, 2024-03-23T18:08:39.850 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_items": 0, 2024-03-23T18:08:39.850 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_bytes": 0, 2024-03-23T18:08:39.850 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_items": 0, 2024-03-23T18:08:39.850 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_reader_bytes": 0, 2024-03-23T18:08:39.850 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_reader_items": 0, 2024-03-23T18:08:39.850 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_writer_bytes": 0, 2024-03-23T18:08:39.851 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_writer_items": 0, 2024-03-23T18:08:39.851 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_anon_bytes": 275935, 2024-03-23T18:08:39.851 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_anon_items": 1110, 2024-03-23T18:08:39.851 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_meta_bytes": 0, 2024-03-23T18:08:39.851 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_meta_items": 0, 2024-03-23T18:08:39.851 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_bytes": 0, 2024-03-23T18:08:39.851 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_items": 0, 2024-03-23T18:08:39.851 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_mapbl_bytes": 0, 2024-03-23T18:08:39.852 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_mapbl_items": 0, 2024-03-23T18:08:39.852 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_pglog_bytes": 0, 2024-03-23T18:08:39.852 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_pglog_items": 0, 2024-03-23T18:08:39.852 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_bytes": 7936, 2024-03-23T18:08:39.852 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_items": 165, 2024-03-23T18:08:39.852 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_mapping_bytes": 0, 2024-03-23T18:08:39.852 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_mapping_items": 0, 2024-03-23T18:08:39.852 INFO:teuthology.orchestra.run.smithi152.stdout: "pgmap_bytes": 0, 2024-03-23T18:08:39.853 INFO:teuthology.orchestra.run.smithi152.stdout: "pgmap_items": 0, 2024-03-23T18:08:39.853 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_co_bytes": 16482768, 2024-03-23T18:08:39.853 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_co_items": 103756, 2024-03-23T18:08:39.853 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_1_bytes": 0, 2024-03-23T18:08:39.853 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_1_items": 0, 2024-03-23T18:08:39.853 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_2_bytes": 0, 2024-03-23T18:08:39.853 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_2_items": 0 2024-03-23T18:08:39.853 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:39.853 INFO:teuthology.orchestra.run.smithi152.stdout: "objecter": { 2024-03-23T18:08:39.854 INFO:teuthology.orchestra.run.smithi152.stdout: "op_active": 0, 2024-03-23T18:08:39.854 INFO:teuthology.orchestra.run.smithi152.stdout: "op_laggy": 0, 2024-03-23T18:08:39.854 INFO:teuthology.orchestra.run.smithi152.stdout: "op_send": 4579, 2024-03-23T18:08:39.854 INFO:teuthology.orchestra.run.smithi152.stdout: "op_send_bytes": 21, 2024-03-23T18:08:39.854 INFO:teuthology.orchestra.run.smithi152.stdout: "op_resend": 3, 2024-03-23T18:08:39.854 INFO:teuthology.orchestra.run.smithi152.stdout: "op_reply": 4579, 2024-03-23T18:08:39.854 INFO:teuthology.orchestra.run.smithi152.stdout: "op_latency": { 2024-03-23T18:08:39.854 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 4579, 2024-03-23T18:08:39.855 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 7.615915505, 2024-03-23T18:08:39.855 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.001663226 2024-03-23T18:08:39.855 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:39.855 INFO:teuthology.orchestra.run.smithi152.stdout: "op_inflight": 0, 2024-03-23T18:08:39.855 INFO:teuthology.orchestra.run.smithi152.stdout: "oplen_avg": { 2024-03-23T18:08:39.855 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 4579, 2024-03-23T18:08:39.855 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 4580 2024-03-23T18:08:39.855 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:39.855 INFO:teuthology.orchestra.run.smithi152.stdout: "op": 4579, 2024-03-23T18:08:39.856 INFO:teuthology.orchestra.run.smithi152.stdout: "op_r": 4579, 2024-03-23T18:08:39.856 INFO:teuthology.orchestra.run.smithi152.stdout: "op_w": 0, 2024-03-23T18:08:39.856 INFO:teuthology.orchestra.run.smithi152.stdout: "op_rmw": 0, 2024-03-23T18:08:39.856 INFO:teuthology.orchestra.run.smithi152.stdout: "op_pg": 0, 2024-03-23T18:08:39.856 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_stat": 2530, 2024-03-23T18:08:39.856 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_create": 0, 2024-03-23T18:08:39.856 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_read": 2047, 2024-03-23T18:08:39.856 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_write": 0, 2024-03-23T18:08:39.857 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_writefull": 0, 2024-03-23T18:08:39.857 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_writesame": 0, 2024-03-23T18:08:39.857 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_append": 0, 2024-03-23T18:08:39.857 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_zero": 0, 2024-03-23T18:08:39.857 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_truncate": 0, 2024-03-23T18:08:39.857 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_delete": 0, 2024-03-23T18:08:39.857 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_mapext": 0, 2024-03-23T18:08:39.857 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_sparse_read": 0, 2024-03-23T18:08:39.857 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_clonerange": 0, 2024-03-23T18:08:39.858 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_getxattr": 1, 2024-03-23T18:08:39.858 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_setxattr": 0, 2024-03-23T18:08:39.858 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_cmpxattr": 0, 2024-03-23T18:08:39.858 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_rmxattr": 0, 2024-03-23T18:08:39.858 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_resetxattrs": 0, 2024-03-23T18:08:39.858 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_call": 0, 2024-03-23T18:08:39.858 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_watch": 0, 2024-03-23T18:08:39.858 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_notify": 0, 2024-03-23T18:08:39.859 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_src_cmpxattr": 0, 2024-03-23T18:08:39.859 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_pgls": 0, 2024-03-23T18:08:39.859 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_pgls_filter": 0, 2024-03-23T18:08:39.859 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_other": 0, 2024-03-23T18:08:39.859 INFO:teuthology.orchestra.run.smithi152.stdout: "linger_active": 0, 2024-03-23T18:08:39.859 INFO:teuthology.orchestra.run.smithi152.stdout: "linger_send": 0, 2024-03-23T18:08:39.859 INFO:teuthology.orchestra.run.smithi152.stdout: "linger_resend": 0, 2024-03-23T18:08:39.859 INFO:teuthology.orchestra.run.smithi152.stdout: "linger_ping": 0, 2024-03-23T18:08:39.859 INFO:teuthology.orchestra.run.smithi152.stdout: "poolop_active": 0, 2024-03-23T18:08:39.860 INFO:teuthology.orchestra.run.smithi152.stdout: "poolop_send": 0, 2024-03-23T18:08:39.860 INFO:teuthology.orchestra.run.smithi152.stdout: "poolop_resend": 0, 2024-03-23T18:08:39.860 INFO:teuthology.orchestra.run.smithi152.stdout: "poolstat_active": 0, 2024-03-23T18:08:39.860 INFO:teuthology.orchestra.run.smithi152.stdout: "poolstat_send": 0, 2024-03-23T18:08:39.860 INFO:teuthology.orchestra.run.smithi152.stdout: "poolstat_resend": 0, 2024-03-23T18:08:39.860 INFO:teuthology.orchestra.run.smithi152.stdout: "statfs_active": 0, 2024-03-23T18:08:39.860 INFO:teuthology.orchestra.run.smithi152.stdout: "statfs_send": 0, 2024-03-23T18:08:39.860 INFO:teuthology.orchestra.run.smithi152.stdout: "statfs_resend": 0, 2024-03-23T18:08:39.860 INFO:teuthology.orchestra.run.smithi152.stdout: "command_active": 0, 2024-03-23T18:08:39.861 INFO:teuthology.orchestra.run.smithi152.stdout: "command_send": 0, 2024-03-23T18:08:39.861 INFO:teuthology.orchestra.run.smithi152.stdout: "command_resend": 0, 2024-03-23T18:08:39.861 INFO:teuthology.orchestra.run.smithi152.stdout: "map_epoch": 110, 2024-03-23T18:08:39.861 INFO:teuthology.orchestra.run.smithi152.stdout: "map_full": 0, 2024-03-23T18:08:39.861 INFO:teuthology.orchestra.run.smithi152.stdout: "map_inc": 30, 2024-03-23T18:08:39.861 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_sessions": 11, 2024-03-23T18:08:39.861 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_session_open": 11, 2024-03-23T18:08:39.861 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_session_close": 0, 2024-03-23T18:08:39.861 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_laggy": 0, 2024-03-23T18:08:39.862 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_wr": 0, 2024-03-23T18:08:39.862 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_rd": 2, 2024-03-23T18:08:39.862 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_del": 0 2024-03-23T18:08:39.862 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:39.862 INFO:teuthology.orchestra.run.smithi152.stdout: "oft": { 2024-03-23T18:08:39.862 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_total_objs": 0, 2024-03-23T18:08:39.862 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_total_kv_pairs": 0, 2024-03-23T18:08:39.862 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_total_updates": 0, 2024-03-23T18:08:39.863 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_total_removes": 0 2024-03-23T18:08:39.863 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:39.863 INFO:teuthology.orchestra.run.smithi152.stdout: "osd-slow-ops": { 2024-03-23T18:08:39.863 INFO:teuthology.orchestra.run.smithi152.stdout: "slow_ops_count": 0 2024-03-23T18:08:39.863 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:39.863 INFO:teuthology.orchestra.run.smithi152.stdout: "purge_queue": { 2024-03-23T18:08:39.863 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executing_ops": 0, 2024-03-23T18:08:39.863 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executing_ops_high_water": 0, 2024-03-23T18:08:39.863 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executing": 0, 2024-03-23T18:08:39.864 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executing_high_water": 0, 2024-03-23T18:08:39.864 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executed": 0, 2024-03-23T18:08:39.864 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_item_in_journal": 0 2024-03-23T18:08:39.864 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:39.864 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-23T18:08:39.864 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-03-23T18:08:39.864 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 104857600, 2024-03-23T18:08:39.864 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-03-23T18:08:39.865 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 5014, 2024-03-23T18:08:39.865 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 401469242, 2024-03-23T18:08:39.865 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-03-23T18:08:39.865 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 5014, 2024-03-23T18:08:39.865 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 0, 2024-03-23T18:08:39.865 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 0, 2024-03-23T18:08:39.865 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 5014, 2024-03-23T18:08:39.865 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 401469242, 2024-03-23T18:08:39.866 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-03-23T18:08:39.866 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:39.866 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:39.866 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:39.866 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-03-23T18:08:39.866 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:39.866 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-objecter_bytes": { 2024-03-23T18:08:39.866 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-03-23T18:08:39.867 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 104857600, 2024-03-23T18:08:39.867 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-03-23T18:08:39.867 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 0, 2024-03-23T18:08:39.867 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 0, 2024-03-23T18:08:39.867 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-03-23T18:08:39.867 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 0, 2024-03-23T18:08:39.867 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 4579, 2024-03-23T18:08:39.867 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 398679465, 2024-03-23T18:08:39.867 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 4579, 2024-03-23T18:08:39.868 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 398679465, 2024-03-23T18:08:39.868 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-03-23T18:08:39.868 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:39.868 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:39.868 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:39.868 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-03-23T18:08:39.868 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:39.868 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-objecter_ops": { 2024-03-23T18:08:39.869 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-03-23T18:08:39.869 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 1024, 2024-03-23T18:08:39.869 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-03-23T18:08:39.869 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 0, 2024-03-23T18:08:39.869 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 0, 2024-03-23T18:08:39.869 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-03-23T18:08:39.869 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 0, 2024-03-23T18:08:39.869 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 4579, 2024-03-23T18:08:39.869 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 4579, 2024-03-23T18:08:39.870 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 4579, 2024-03-23T18:08:39.870 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 4579, 2024-03-23T18:08:39.870 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-03-23T18:08:39.870 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:39.870 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:39.870 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:39.870 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-03-23T18:08:39.870 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:39.871 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-write_buf_throttle": { 2024-03-23T18:08:39.871 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-03-23T18:08:39.871 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 3758096384, 2024-03-23T18:08:39.871 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-03-23T18:08:39.871 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 0, 2024-03-23T18:08:39.871 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 0, 2024-03-23T18:08:39.871 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-03-23T18:08:39.871 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 0, 2024-03-23T18:08:39.871 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 0, 2024-03-23T18:08:39.872 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 0, 2024-03-23T18:08:39.872 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 0, 2024-03-23T18:08:39.872 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 0, 2024-03-23T18:08:39.872 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-03-23T18:08:39.872 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:39.872 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:39.872 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:39.872 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-03-23T18:08:39.872 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-03-23T18:08:39.873 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-write_buf_throttle-0x56163cc24500": { 2024-03-23T18:08:39.873 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-03-23T18:08:39.873 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 3758096384, 2024-03-23T18:08:39.873 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-03-23T18:08:39.873 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 0, 2024-03-23T18:08:39.873 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 0, 2024-03-23T18:08:39.873 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-03-23T18:08:39.873 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 0, 2024-03-23T18:08:39.873 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 0, 2024-03-23T18:08:39.874 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 0, 2024-03-23T18:08:39.874 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 0, 2024-03-23T18:08:39.874 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 0, 2024-03-23T18:08:39.874 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-03-23T18:08:39.874 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-03-23T18:08:39.874 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-03-23T18:08:39.874 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-03-23T18:08:39.874 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-03-23T18:08:39.875 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-03-23T18:08:39.875 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-03-23T18:08:40.161 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-03-23T18:08:40.161 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-03-23T18:08:40.161 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2024-03-23T18:08:40.161 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-03-23T18:08:40.161 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-03-23T18:08:40.162 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-03-23T18:08:40.173 INFO:tasks.kclient:Unmounting kernel clients... 2024-03-23T18:08:40.173 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-23T18:08:40.173 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T18:08:40.224 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-23T18:08:40.224 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T18:08:40.270 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T18:08:40.307 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:40 smithi092 ceph-mon[79089]: pgmap v570: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 2.9 MiB/s wr, 140 op/s 2024-03-23T18:08:40.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:40 smithi152 ceph-mon[47949]: pgmap v570: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 2.9 MiB/s wr, 140 op/s 2024-03-23T18:08:40.371 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:40 smithi086 ceph-mon[38221]: pgmap v570: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 2.9 MiB/s wr, 140 op/s 2024-03-23T18:08:40.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.510+0000 7f5be0600700 1 -- 172.21.15.86:0/712560223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bd8105f80 msgr2=0x7f5bd810db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:40.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.510+0000 7f5be0600700 1 --2- 172.21.15.86:0/712560223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bd8105f80 0x7f5bd810db00 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f5bd0009990 tx=0x7f5bd0009ca0 comp rx=0 tx=0).stop 2024-03-23T18:08:40.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.510+0000 7f5be0600700 1 -- 172.21.15.86:0/712560223 shutdown_connections 2024-03-23T18:08:40.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.510+0000 7f5be0600700 1 --2- 172.21.15.86:0/712560223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bd8105f80 0x7f5bd810db00 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:40.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.510+0000 7f5be0600700 1 --2- 172.21.15.86:0/712560223 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5bd8100e60 0x7f5bd8101240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:40.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.510+0000 7f5be0600700 1 --2- 172.21.15.86:0/712560223 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5bd8101810 0x7f5bd8105850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:40.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.510+0000 7f5be0600700 1 -- 172.21.15.86:0/712560223 >> 172.21.15.86:0/712560223 conn(0x7f5bd80fc700 msgr2=0x7f5bd80feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:08:40.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.510+0000 7f5be0600700 1 -- 172.21.15.86:0/712560223 shutdown_connections 2024-03-23T18:08:40.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.510+0000 7f5be0600700 1 -- 172.21.15.86:0/712560223 wait complete. 2024-03-23T18:08:40.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.511+0000 7f5be0600700 1 Processor -- start 2024-03-23T18:08:40.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.512+0000 7f5be0600700 1 -- start start 2024-03-23T18:08:40.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.512+0000 7f5be0600700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5bd8100e60 0x7f5bd80730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:40.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.512+0000 7f5be0600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bd8105f80 0x7f5bd80735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:40.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.512+0000 7f5be0600700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5bd8074d20 0x7f5bd8073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:40.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.512+0000 7f5be0600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd81a9d00 con 0x7f5bd8105f80 2024-03-23T18:08:40.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.512+0000 7f5be0600700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd81a9e70 con 0x7f5bd8074d20 2024-03-23T18:08:40.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.512+0000 7f5be0600700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd81aa030 con 0x7f5bd8100e60 2024-03-23T18:08:40.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.512+0000 7f5bddc00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5bd8100e60 0x7f5bd80730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:40.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.512+0000 7f5bddc00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5bd8100e60 0x7f5bd80730b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:42346/0 (socket says 172.21.15.86:42346) 2024-03-23T18:08:40.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.512+0000 7f5bddc00700 1 -- 172.21.15.86:0/2325363650 learned_addr learned my addr 172.21.15.86:0/2325363650 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:08:40.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.512+0000 7f5bdd200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bd8105f80 0x7f5bd80735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:40.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.512+0000 7f5bde600700 1 --2- 172.21.15.86:0/2325363650 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5bd8074d20 0x7f5bd8073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:40.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.513+0000 7f5bddc00700 1 -- 172.21.15.86:0/2325363650 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5bd8074d20 msgr2=0x7f5bd8073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:40.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.513+0000 7f5bddc00700 1 --2- 172.21.15.86:0/2325363650 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5bd8074d20 0x7f5bd8073b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:40.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.513+0000 7f5bddc00700 1 -- 172.21.15.86:0/2325363650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bd8105f80 msgr2=0x7f5bd80735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:40.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.513+0000 7f5bddc00700 1 --2- 172.21.15.86:0/2325363650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bd8105f80 0x7f5bd80735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:40.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.513+0000 7f5bddc00700 1 -- 172.21.15.86:0/2325363650 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bd00095f0 con 0x7f5bd8100e60 2024-03-23T18:08:40.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.513+0000 7f5bddc00700 1 --2- 172.21.15.86:0/2325363650 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5bd8100e60 0x7f5bd80730b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc0060b0 tx=0x7f5bcc00d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:08:40.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.513+0000 7f5bd6000700 1 -- 172.21.15.86:0/2325363650 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc00ddd0 con 0x7f5bd8100e60 2024-03-23T18:08:40.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.513+0000 7f5be0600700 1 -- 172.21.15.86:0/2325363650 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bd81aa310 con 0x7f5bd8100e60 2024-03-23T18:08:40.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.513+0000 7f5bd6000700 1 -- 172.21.15.86:0/2325363650 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5bcc00f040 con 0x7f5bd8100e60 2024-03-23T18:08:40.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.513+0000 7f5bd6000700 1 -- 172.21.15.86:0/2325363650 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc014690 con 0x7f5bd8100e60 2024-03-23T18:08:40.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.513+0000 7f5be0600700 1 -- 172.21.15.86:0/2325363650 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bd81aa870 con 0x7f5bd8100e60 2024-03-23T18:08:40.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.514+0000 7f5be0600700 1 -- 172.21.15.86:0/2325363650 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bd8109250 con 0x7f5bd8100e60 2024-03-23T18:08:40.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.516+0000 7f5bd6000700 1 -- 172.21.15.86:0/2325363650 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f5bcc015400 con 0x7f5bd8100e60 2024-03-23T18:08:40.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.517+0000 7f5bd6000700 1 --2- 172.21.15.86:0/2325363650 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f5bc00a1290 0x7f5bc00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:40.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.518+0000 7f5bd6000700 1 -- 172.21.15.86:0/2325363650 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f5bcc0cc720 con 0x7f5bd8100e60 2024-03-23T18:08:40.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.518+0000 7f5bdd200700 1 --2- 172.21.15.86:0/2325363650 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f5bc00a1290 0x7f5bc00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:40.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.518+0000 7f5bdd200700 1 --2- 172.21.15.86:0/2325363650 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f5bc00a1290 0x7f5bc00a3750 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8074790 tx=0x7f5bc40073d0 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:08:40.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.521+0000 7f5bd6000700 1 -- 172.21.15.86:0/2325363650 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f5bcc095b30 con 0x7f5bd8100e60 2024-03-23T18:08:40.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.767+0000 7f5be0600700 1 -- 172.21.15.86:0/2325363650 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5bd8102330 con 0x7f5bd8100e60 2024-03-23T18:08:40.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.769+0000 7f5bd6000700 1 -- 172.21.15.86:0/2325363650 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+22884 (secure 0 0 0) 0x7f5bcc099150 con 0x7f5bd8100e60 2024-03-23T18:08:40.769 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T18:08:40.769 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":110,"fsid":"9d76c9e8-e93c-11ee-95cd-87774f69a715","created":"2024-03-23T17:42:30.097233+0000","modified":"2024-03-23T18:08:15.989252+0000","last_up_change":"2024-03-23T17:48:18.217601+0000","last_in_change":"2024-03-23T17:48:00.250579+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T17:45:26.848397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T17:48:54.331894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T17:48:55.640693+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":30,"snap_epoch":109,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"656655c6-1c7f-4640-92c4-2f42de820143","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6803","nonce":2423505491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6805","nonce":2423505491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6809","nonce":2423505491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6807","nonce":2423505491}]},"public_addr":"172.21.15.86:6803/2423505491","cluster_addr":"172.21.15.86:6805/2423505491","heartbeat_back_addr":"172.21.15.86:6809/2423505491","heartbeat_front_addr":"172.21.15.86:6807/2423505491","state":["exists","up"]},{"osd":1,"uuid":"3e86dc65-20a6-4de8-b9ce-fcf01f68862b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6801","nonce":3776252597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6803","nonce":3776252597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6807","nonce":3776252597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6805","nonce":3776252597}]},"public_addr":"172.21.15.92:6801/3776252597","cluster_addr":"172.21.15.92:6803/3776252597","heartbeat_back_addr":"172.21.15.92:6807/3776252597","heartbeat_front_addr":"172.21.15.92:6805/3776252597","state":["exists","up"]},{"osd":2,"uuid":"09808786-3bd3-4d11-8c7b-1736119dfbb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6801","nonce":1601265891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6803","nonce":1601265891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6807","nonce":1601265891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6805","nonce":1601265891}]},"public_addr":"172.21.15.152:6801/1601265891","cluster_addr":"172.21.15.152:6803/1601265891","heartbeat_back_addr":"172.21.15.152:6807/1601265891","heartbeat_front_addr":"172.21.15.152:6805/1601265891","state":["exists","up"]},{"osd":3,"uuid":"79c61910-4b53-424c-b150-6f5385a2cd68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6811","nonce":694846466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6813","nonce":694846466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6817","nonce":694846466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6815","nonce":694846466}]},"public_addr":"172.21.15.86:6811/694846466","cluster_addr":"172.21.15.86:6813/694846466","heartbeat_back_addr":"172.21.15.86:6817/694846466","heartbeat_front_addr":"172.21.15.86:6815/694846466","state":["exists","up"]},{"osd":4,"uuid":"a2878035-0179-469f-bbd4-8ce2d3f6bddc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6809","nonce":1422821471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6811","nonce":1422821471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6815","nonce":1422821471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6813","nonce":1422821471}]},"public_addr":"172.21.15.92:6809/1422821471","cluster_addr":"172.21.15.92:6811/1422821471","heartbeat_back_addr":"172.21.15.92:6815/1422821471","heartbeat_front_addr":"172.21.15.92:6813/1422821471","state":["exists","up"]},{"osd":5,"uuid":"837366aa-ce25-42ca-b157-5bbb50159d0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6809","nonce":1039375239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6811","nonce":1039375239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6815","nonce":1039375239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6813","nonce":1039375239}]},"public_addr":"172.21.15.152:6809/1039375239","cluster_addr":"172.21.15.152:6811/1039375239","heartbeat_back_addr":"172.21.15.152:6815/1039375239","heartbeat_front_addr":"172.21.15.152:6813/1039375239","state":["exists","up"]},{"osd":6,"uuid":"61024437-7b91-432f-a1af-6443d0ef5213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6819","nonce":4080452707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6821","nonce":4080452707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6825","nonce":4080452707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6823","nonce":4080452707}]},"public_addr":"172.21.15.86:6819/4080452707","cluster_addr":"172.21.15.86:6821/4080452707","heartbeat_back_addr":"172.21.15.86:6825/4080452707","heartbeat_front_addr":"172.21.15.86:6823/4080452707","state":["exists","up"]},{"osd":7,"uuid":"2684bdae-6b30-4afc-af68-4ce41b3a75b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6817","nonce":4187707622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6819","nonce":4187707622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6823","nonce":4187707622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6821","nonce":4187707622}]},"public_addr":"172.21.15.92:6817/4187707622","cluster_addr":"172.21.15.92:6819/4187707622","heartbeat_back_addr":"172.21.15.92:6823/4187707622","heartbeat_front_addr":"172.21.15.92:6821/4187707622","state":["exists","up"]},{"osd":8,"uuid":"f03bbf0d-1281-4bcf-9773-9854388e65ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6817","nonce":3237612748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6819","nonce":3237612748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6823","nonce":3237612748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6821","nonce":3237612748}]},"public_addr":"172.21.15.152:6817/3237612748","cluster_addr":"172.21.15.152:6819/3237612748","heartbeat_back_addr":"172.21.15.152:6823/3237612748","heartbeat_front_addr":"172.21.15.152:6821/3237612748","state":["exists","up"]},{"osd":9,"uuid":"e7e8b775-99ff-48a4-a23a-91e7ab6622bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6827","nonce":2835218294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6829","nonce":2835218294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6833","nonce":2835218294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6831","nonce":2835218294}]},"public_addr":"172.21.15.86:6827/2835218294","cluster_addr":"172.21.15.86:6829/2835218294","heartbeat_back_addr":"172.21.15.86:6833/2835218294","heartbeat_front_addr":"172.21.15.86:6831/2835218294","state":["exists","up"]},{"osd":10,"uuid":"d9d616cd-a229-44de-8768-02a4415df115","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6825","nonce":4259465100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6827","nonce":4259465100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6831","nonce":4259465100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6829","nonce":4259465100}]},"public_addr":"172.21.15.92:6825/4259465100","cluster_addr":"172.21.15.92:6827/4259465100","heartbeat_back_addr":"172.21.15.92:6831/4259465100","heartbeat_front_addr":"172.21.15.92:6829/4259465100","state":["exists","up"]},{"osd":11,"uuid":"e7c0b865-ccf5-4233-8d22-0d9b6dbe5c97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6825","nonce":194662760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6827","nonce":194662760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6831","nonce":194662760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6829","nonce":194662760}]},"public_addr":"172.21.15.152:6825/194662760","cluster_addr":"172.21.15.152:6827/194662760","heartbeat_back_addr":"172.21.15.152:6831/194662760","heartbeat_front_addr":"172.21.15.152:6829/194662760","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:44:47.269167+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:45:05.363655+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:45:23.192677+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:45:44.156228+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:02.532971+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:20.153295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:41.713888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:59.676516+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:47:17.207087+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:47:38.970316+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:47:57.282525+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:48:15.437606+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6801/2734784244":"2024-03-24T17:50:38.377487+0000","172.21.15.86:6800/2734784244":"2024-03-24T17:50:38.377487+0000","172.21.15.86:0/734928365":"2024-03-24T17:50:38.377487+0000","172.21.15.86:0/2354115349":"2024-03-24T17:50:38.377487+0000","172.21.15.86:0/3553500912":"2024-03-24T17:43:28.242492+0000","172.21.15.86:6801/3600278228":"2024-03-24T17:43:28.242492+0000","172.21.15.86:0/887262329":"2024-03-24T17:43:28.242492+0000","172.21.15.86:6800/2313723492":"2024-03-24T17:43:05.043910+0000","172.21.15.86:6800/3600278228":"2024-03-24T17:43:28.242492+0000","172.21.15.86:0/3815705642":"2024-03-24T17:43:05.043910+0000","172.21.15.86:0/883052430":"2024-03-24T17:50:38.377487+0000","172.21.15.86:0/1614943062":"2024-03-24T17:43:28.242492+0000","172.21.15.86:6801/2313723492":"2024-03-24T17:43:05.043910+0000","172.21.15.86:0/2416321716":"2024-03-24T17:43:05.043910+0000","172.21.15.86:0/2938296862":"2024-03-24T17:50:38.377487+0000","172.21.15.86:0/433588188":"2024-03-24T17:43:05.043910+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":18,"length":1},{"begin":30,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T18:08:40.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.771+0000 7f5be0600700 1 -- 172.21.15.86:0/2325363650 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f5bc00a1290 msgr2=0x7f5bc00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:40.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.771+0000 7f5be0600700 1 --2- 172.21.15.86:0/2325363650 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f5bc00a1290 0x7f5bc00a3750 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8074790 tx=0x7f5bc40073d0 comp rx=0 tx=0).stop 2024-03-23T18:08:40.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.771+0000 7f5be0600700 1 -- 172.21.15.86:0/2325363650 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5bd8100e60 msgr2=0x7f5bd80730b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:40.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.771+0000 7f5be0600700 1 --2- 172.21.15.86:0/2325363650 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5bd8100e60 0x7f5bd80730b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc0060b0 tx=0x7f5bcc00d3b0 comp rx=0 tx=0).stop 2024-03-23T18:08:40.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.771+0000 7f5be0600700 1 -- 172.21.15.86:0/2325363650 shutdown_connections 2024-03-23T18:08:40.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.771+0000 7f5be0600700 1 --2- 172.21.15.86:0/2325363650 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5bd8074d20 0x7f5bd8073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:40.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.771+0000 7f5be0600700 1 --2- 172.21.15.86:0/2325363650 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f5bc00a1290 0x7f5bc00a3750 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:40.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.771+0000 7f5be0600700 1 --2- 172.21.15.86:0/2325363650 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5bd8100e60 0x7f5bd80730b0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:40.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.771+0000 7f5be0600700 1 --2- 172.21.15.86:0/2325363650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bd8105f80 0x7f5bd80735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:40.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.771+0000 7f5be0600700 1 -- 172.21.15.86:0/2325363650 >> 172.21.15.86:0/2325363650 conn(0x7f5bd80fc700 msgr2=0x7f5bd80feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:08:40.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.771+0000 7f5be0600700 1 -- 172.21.15.86:0/2325363650 shutdown_connections 2024-03-23T18:08:40.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.772+0000 7f5be0600700 1 -- 172.21.15.86:0/2325363650 wait complete. 2024-03-23T18:08:40.791 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T17:45:26.848397+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T17:48:54.331894+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T17:48:55.640693+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 30, 'snap_epoch': 109, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T18:08:40.792 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T18:08:40.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.990+0000 7f965c600700 1 -- 172.21.15.86:0/2887702617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9654100f60 msgr2=0x7f9654101340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:40.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.990+0000 7f965c600700 1 --2- 172.21.15.86:0/2887702617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9654100f60 0x7f9654101340 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f9648009990 tx=0x7f9648009ca0 comp rx=0 tx=0).stop 2024-03-23T18:08:40.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.990+0000 7f965c600700 1 -- 172.21.15.86:0/2887702617 shutdown_connections 2024-03-23T18:08:40.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.990+0000 7f965c600700 1 --2- 172.21.15.86:0/2887702617 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9654106080 0x7f965410dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:40.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.990+0000 7f965c600700 1 --2- 172.21.15.86:0/2887702617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9654100f60 0x7f9654101340 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:40.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.990+0000 7f965c600700 1 --2- 172.21.15.86:0/2887702617 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9654101910 0x7f9654105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:40.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.990+0000 7f965c600700 1 -- 172.21.15.86:0/2887702617 >> 172.21.15.86:0/2887702617 conn(0x7f96540fc7c0 msgr2=0x7f96540febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:08:40.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.990+0000 7f965c600700 1 -- 172.21.15.86:0/2887702617 shutdown_connections 2024-03-23T18:08:40.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.990+0000 7f965c600700 1 -- 172.21.15.86:0/2887702617 wait complete. 2024-03-23T18:08:40.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.991+0000 7f965c600700 1 Processor -- start 2024-03-23T18:08:40.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.991+0000 7f965c600700 1 -- start start 2024-03-23T18:08:40.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.992+0000 7f965c600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9654100f60 0x7f9654199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:40.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.992+0000 7f965c600700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9654106080 0x7f9654199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:40.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.992+0000 7f965c600700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f965419dd00 0x7f965419e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:40.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.992+0000 7f965c600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f965419e780 con 0x7f9654100f60 2024-03-23T18:08:40.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.992+0000 7f965c600700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f965419e8f0 con 0x7f965419dd00 2024-03-23T18:08:40.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.992+0000 7f965c600700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f965419eab0 con 0x7f9654106080 2024-03-23T18:08:40.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.992+0000 7f9659200700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9654106080 0x7f9654199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:40.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.992+0000 7f9659c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9654100f60 0x7f9654199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:40.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.992+0000 7f9659200700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9654106080 0x7f9654199960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.86:42354/0 (socket says 172.21.15.86:42354) 2024-03-23T18:08:40.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.992+0000 7f9659200700 1 -- 172.21.15.86:0/2142352687 learned_addr learned my addr 172.21.15.86:0/2142352687 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:08:40.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.992+0000 7f965a600700 1 --2- 172.21.15.86:0/2142352687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f965419dd00 0x7f965419e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:40.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.992+0000 7f9659200700 1 -- 172.21.15.86:0/2142352687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f965419dd00 msgr2=0x7f965419e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:40.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.992+0000 7f9659200700 1 --2- 172.21.15.86:0/2142352687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f965419dd00 0x7f965419e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:40.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.992+0000 7f9659200700 1 -- 172.21.15.86:0/2142352687 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9654100f60 msgr2=0x7f9654199420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:40.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.992+0000 7f9659200700 1 --2- 172.21.15.86:0/2142352687 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9654100f60 0x7f9654199420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:40.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.993+0000 7f9659200700 1 -- 172.21.15.86:0/2142352687 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96480095f0 con 0x7f9654106080 2024-03-23T18:08:40.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.993+0000 7f9659c00700 1 --2- 172.21.15.86:0/2142352687 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9654100f60 0x7f9654199420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T18:08:40.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.993+0000 7f965a600700 1 --2- 172.21.15.86:0/2142352687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f965419dd00 0x7f965419e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T18:08:40.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.993+0000 7f9659200700 1 --2- 172.21.15.86:0/2142352687 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9654106080 0x7f9654199960 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f964000e970 tx=0x7f964000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:08:40.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.993+0000 7f9652000700 1 -- 172.21.15.86:0/2142352687 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9640009790 con 0x7f9654106080 2024-03-23T18:08:40.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.993+0000 7f9652000700 1 -- 172.21.15.86:0/2142352687 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f964000c7e0 con 0x7f9654106080 2024-03-23T18:08:40.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.993+0000 7f965c600700 1 -- 172.21.15.86:0/2142352687 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96541a5fb0 con 0x7f9654106080 2024-03-23T18:08:40.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.993+0000 7f9652000700 1 -- 172.21.15.86:0/2142352687 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96400054e0 con 0x7f9654106080 2024-03-23T18:08:40.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.993+0000 7f965c600700 1 -- 172.21.15.86:0/2142352687 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96541a6490 con 0x7f9654106080 2024-03-23T18:08:40.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.994+0000 7f965c600700 1 -- 172.21.15.86:0/2142352687 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9654102430 con 0x7f9654106080 2024-03-23T18:08:40.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.996+0000 7f9652000700 1 -- 172.21.15.86:0/2142352687 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f964000c950 con 0x7f9654106080 2024-03-23T18:08:40.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.998+0000 7f9652000700 1 --2- 172.21.15.86:0/2142352687 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f963c0a1290 0x7f963c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:40.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.998+0000 7f9652000700 1 -- 172.21.15.86:0/2142352687 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f9640014070 con 0x7f9654106080 2024-03-23T18:08:40.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.998+0000 7f9659c00700 1 --2- 172.21.15.86:0/2142352687 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f963c0a1290 0x7f963c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:40.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:40.998+0000 7f9659c00700 1 --2- 172.21.15.86:0/2142352687 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f963c0a1290 0x7f963c0a3750 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9648006010 tx=0x7f964800b260 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:08:40.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.000+0000 7f9652000700 1 -- 172.21.15.86:0/2142352687 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f9640096510 con 0x7f9654106080 2024-03-23T18:08:41.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.207+0000 7f965c600700 1 -- 172.21.15.86:0/2142352687 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9654101910 con 0x7f9654106080 2024-03-23T18:08:41.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.208+0000 7f9652000700 1 -- 172.21.15.86:0/2142352687 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v110) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9640099b30 con 0x7f9654106080 2024-03-23T18:08:41.208 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-03-23T18:08:41.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.210+0000 7f965c600700 1 -- 172.21.15.86:0/2142352687 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f963c0a1290 msgr2=0x7f963c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:41.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.210+0000 7f965c600700 1 --2- 172.21.15.86:0/2142352687 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f963c0a1290 0x7f963c0a3750 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9648006010 tx=0x7f964800b260 comp rx=0 tx=0).stop 2024-03-23T18:08:41.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.210+0000 7f965c600700 1 -- 172.21.15.86:0/2142352687 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9654106080 msgr2=0x7f9654199960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:41.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.210+0000 7f965c600700 1 --2- 172.21.15.86:0/2142352687 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9654106080 0x7f9654199960 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f964000e970 tx=0x7f964000ec80 comp rx=0 tx=0).stop 2024-03-23T18:08:41.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.210+0000 7f965c600700 1 -- 172.21.15.86:0/2142352687 shutdown_connections 2024-03-23T18:08:41.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.210+0000 7f965c600700 1 --2- 172.21.15.86:0/2142352687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f965419dd00 0x7f965419e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:41.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.210+0000 7f965c600700 1 --2- 172.21.15.86:0/2142352687 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9654100f60 0x7f9654199420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:41.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.210+0000 7f965c600700 1 --2- 172.21.15.86:0/2142352687 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f963c0a1290 0x7f963c0a3750 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:41.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.210+0000 7f965c600700 1 --2- 172.21.15.86:0/2142352687 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9654106080 0x7f9654199960 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:41.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.210+0000 7f965c600700 1 -- 172.21.15.86:0/2142352687 >> 172.21.15.86:0/2142352687 conn(0x7f96540fc7c0 msgr2=0x7f96540fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:08:41.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.210+0000 7f965c600700 1 -- 172.21.15.86:0/2142352687 shutdown_connections 2024-03-23T18:08:41.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.210+0000 7f965c600700 1 -- 172.21.15.86:0/2142352687 wait complete. 2024-03-23T18:08:41.229 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T18:08:41.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:41 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/2325363650' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T18:08:41.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.424+0000 7f3e21a00700 1 -- 172.21.15.86:0/3935813917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e1c100f60 msgr2=0x7f3e1c101340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:41.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.424+0000 7f3e21a00700 1 --2- 172.21.15.86:0/3935813917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e1c100f60 0x7f3e1c101340 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c009990 tx=0x7f3e0c009ca0 comp rx=0 tx=0).stop 2024-03-23T18:08:41.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.425+0000 7f3e21a00700 1 -- 172.21.15.86:0/3935813917 shutdown_connections 2024-03-23T18:08:41.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.425+0000 7f3e21a00700 1 --2- 172.21.15.86:0/3935813917 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e1c106080 0x7f3e1c10dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:41.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.425+0000 7f3e21a00700 1 --2- 172.21.15.86:0/3935813917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e1c100f60 0x7f3e1c101340 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:41.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.425+0000 7f3e21a00700 1 --2- 172.21.15.86:0/3935813917 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3e1c101910 0x7f3e1c105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:41.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.425+0000 7f3e21a00700 1 -- 172.21.15.86:0/3935813917 >> 172.21.15.86:0/3935813917 conn(0x7f3e1c0fc7c0 msgr2=0x7f3e1c0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:08:41.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.425+0000 7f3e21a00700 1 -- 172.21.15.86:0/3935813917 shutdown_connections 2024-03-23T18:08:41.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.425+0000 7f3e21a00700 1 -- 172.21.15.86:0/3935813917 wait complete. 2024-03-23T18:08:41.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.426+0000 7f3e21a00700 1 Processor -- start 2024-03-23T18:08:41.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.426+0000 7f3e21a00700 1 -- start start 2024-03-23T18:08:41.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.426+0000 7f3e21a00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3e1c100f60 0x7f3e1c199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:41.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.426+0000 7f3e21a00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e1c106080 0x7f3e1c199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:41.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.426+0000 7f3e21a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e1c19dce0 0x7f3e1c19e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:41.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.426+0000 7f3e21a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e1c19e760 con 0x7f3e1c19dce0 2024-03-23T18:08:41.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.426+0000 7f3e21a00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e1c19e8d0 con 0x7f3e1c106080 2024-03-23T18:08:41.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.426+0000 7f3e21a00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e1c19ea90 con 0x7f3e1c100f60 2024-03-23T18:08:41.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.426+0000 7f3e1b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e1c19dce0 0x7f3e1c19e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:41.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.426+0000 7f3e1aa00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3e1c100f60 0x7f3e1c199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:41.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.426+0000 7f3e1b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e1c19dce0 0x7f3e1c19e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46756/0 (socket says 172.21.15.86:46756) 2024-03-23T18:08:41.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.426+0000 7f3e1b400700 1 -- 172.21.15.86:0/1690535429 learned_addr learned my addr 172.21.15.86:0/1690535429 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:08:41.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.426+0000 7f3e1a000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e1c106080 0x7f3e1c199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:41.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.427+0000 7f3e1b400700 1 -- 172.21.15.86:0/1690535429 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3e1c100f60 msgr2=0x7f3e1c199400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:41.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.427+0000 7f3e1b400700 1 --2- 172.21.15.86:0/1690535429 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3e1c100f60 0x7f3e1c199400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:41.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.427+0000 7f3e1b400700 1 -- 172.21.15.86:0/1690535429 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e1c106080 msgr2=0x7f3e1c199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:41.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.427+0000 7f3e1b400700 1 --2- 172.21.15.86:0/1690535429 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e1c106080 0x7f3e1c199940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:41.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.427+0000 7f3e1b400700 1 -- 172.21.15.86:0/1690535429 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e0c0095f0 con 0x7f3e1c19dce0 2024-03-23T18:08:41.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.427+0000 7f3e1b400700 1 --2- 172.21.15.86:0/1690535429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e1c19dce0 0x7f3e1c19e160 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f3e14007d70 tx=0x7f3e1400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:08:41.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.427+0000 7f3e13400700 1 -- 172.21.15.86:0/1690535429 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e1400ac90 con 0x7f3e1c19dce0 2024-03-23T18:08:41.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.427+0000 7f3e13400700 1 -- 172.21.15.86:0/1690535429 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e1400adf0 con 0x7f3e1c19dce0 2024-03-23T18:08:41.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.427+0000 7f3e21a00700 1 -- 172.21.15.86:0/1690535429 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e1c1a5ff0 con 0x7f3e1c19dce0 2024-03-23T18:08:41.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.427+0000 7f3e13400700 1 -- 172.21.15.86:0/1690535429 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e140115e0 con 0x7f3e1c19dce0 2024-03-23T18:08:41.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.427+0000 7f3e21a00700 1 -- 172.21.15.86:0/1690535429 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e1c1a6520 con 0x7f3e1c19dce0 2024-03-23T18:08:41.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.428+0000 7f3e21a00700 1 -- 172.21.15.86:0/1690535429 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e1c102430 con 0x7f3e1c19dce0 2024-03-23T18:08:41.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.430+0000 7f3e13400700 1 -- 172.21.15.86:0/1690535429 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f3e14004ad0 con 0x7f3e1c19dce0 2024-03-23T18:08:41.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.432+0000 7f3e13400700 1 --2- 172.21.15.86:0/1690535429 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f3e000a1310 0x7f3e000a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:41.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.432+0000 7f3e13400700 1 -- 172.21.15.86:0/1690535429 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f3e140cd0b0 con 0x7f3e1c19dce0 2024-03-23T18:08:41.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.432+0000 7f3e1aa00700 1 --2- 172.21.15.86:0/1690535429 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f3e000a1310 0x7f3e000a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:41.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.432+0000 7f3e1aa00700 1 --2- 172.21.15.86:0/1690535429 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f3e000a1310 0x7f3e000a37d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c00b2e0 tx=0x7f3e0c005fd0 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:08:41.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.434+0000 7f3e13400700 1 -- 172.21.15.86:0/1690535429 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f3e140965d0 con 0x7f3e1c19dce0 2024-03-23T18:08:41.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:41 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/2325363650' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T18:08:41.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:41 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/2325363650' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T18:08:41.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.639+0000 7f3e21a00700 1 -- 172.21.15.86:0/1690535429 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3e1c101910 con 0x7f3e1c19dce0 2024-03-23T18:08:41.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.639+0000 7f3e13400700 1 -- 172.21.15.86:0/1690535429 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v110) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3e140160a0 con 0x7f3e1c19dce0 2024-03-23T18:08:41.639 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-03-23T18:08:41.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.641+0000 7f3e21a00700 1 -- 172.21.15.86:0/1690535429 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f3e000a1310 msgr2=0x7f3e000a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:41.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.641+0000 7f3e21a00700 1 --2- 172.21.15.86:0/1690535429 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f3e000a1310 0x7f3e000a37d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c00b2e0 tx=0x7f3e0c005fd0 comp rx=0 tx=0).stop 2024-03-23T18:08:41.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.641+0000 7f3e21a00700 1 -- 172.21.15.86:0/1690535429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e1c19dce0 msgr2=0x7f3e1c19e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:41.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.641+0000 7f3e21a00700 1 --2- 172.21.15.86:0/1690535429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e1c19dce0 0x7f3e1c19e160 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f3e14007d70 tx=0x7f3e1400a340 comp rx=0 tx=0).stop 2024-03-23T18:08:41.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.641+0000 7f3e21a00700 1 -- 172.21.15.86:0/1690535429 shutdown_connections 2024-03-23T18:08:41.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.641+0000 7f3e21a00700 1 --2- 172.21.15.86:0/1690535429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e1c19dce0 0x7f3e1c19e160 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:41.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.641+0000 7f3e21a00700 1 --2- 172.21.15.86:0/1690535429 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f3e000a1310 0x7f3e000a37d0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:41.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.641+0000 7f3e21a00700 1 --2- 172.21.15.86:0/1690535429 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3e1c100f60 0x7f3e1c199400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:41.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.641+0000 7f3e21a00700 1 --2- 172.21.15.86:0/1690535429 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e1c106080 0x7f3e1c199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:41.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.641+0000 7f3e21a00700 1 -- 172.21.15.86:0/1690535429 >> 172.21.15.86:0/1690535429 conn(0x7f3e1c0fc7c0 msgr2=0x7f3e1c0feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:08:41.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.642+0000 7f3e21a00700 1 -- 172.21.15.86:0/1690535429 shutdown_connections 2024-03-23T18:08:41.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.642+0000 7f3e21a00700 1 -- 172.21.15.86:0/1690535429 wait complete. 2024-03-23T18:08:41.660 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T18:08:41.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.855+0000 7f8182600700 1 -- 172.21.15.86:0/2698502669 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f817c101920 msgr2=0x7f817c105960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:41.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.855+0000 7f8182600700 1 --2- 172.21.15.86:0/2698502669 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f817c101920 0x7f817c105960 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f81680098f0 tx=0x7f8168009c00 comp rx=0 tx=0).stop 2024-03-23T18:08:41.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.855+0000 7f8182600700 1 -- 172.21.15.86:0/2698502669 shutdown_connections 2024-03-23T18:08:41.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.855+0000 7f8182600700 1 --2- 172.21.15.86:0/2698502669 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f817c106090 0x7f817c10dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:41.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.855+0000 7f8182600700 1 --2- 172.21.15.86:0/2698502669 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f817c100f70 0x7f817c101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:41.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.855+0000 7f8182600700 1 --2- 172.21.15.86:0/2698502669 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f817c101920 0x7f817c105960 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:41.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.855+0000 7f8182600700 1 -- 172.21.15.86:0/2698502669 >> 172.21.15.86:0/2698502669 conn(0x7f817c0fc7f0 msgr2=0x7f817c0fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:08:41.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.855+0000 7f8182600700 1 -- 172.21.15.86:0/2698502669 shutdown_connections 2024-03-23T18:08:41.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.855+0000 7f8182600700 1 -- 172.21.15.86:0/2698502669 wait complete. 2024-03-23T18:08:41.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.856+0000 7f8182600700 1 Processor -- start 2024-03-23T18:08:41.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.856+0000 7f8182600700 1 -- start start 2024-03-23T18:08:41.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.856+0000 7f8182600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f817c100f70 0x7f817c199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:41.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.856+0000 7f8182600700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f817c106090 0x7f817c1999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:41.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.857+0000 7f8182600700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f817c19dd40 0x7f817c19e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:41.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.857+0000 7f8182600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f817c19e7c0 con 0x7f817c100f70 2024-03-23T18:08:41.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.857+0000 7f8182600700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f817c19e930 con 0x7f817c19dd40 2024-03-23T18:08:41.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.857+0000 7f8182600700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f817c19eaf0 con 0x7f817c106090 2024-03-23T18:08:41.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.857+0000 7f817b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f817c100f70 0x7f817c199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:41.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.857+0000 7f817b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f817c100f70 0x7f817c199460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46774/0 (socket says 172.21.15.86:46774) 2024-03-23T18:08:41.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.857+0000 7f817b400700 1 -- 172.21.15.86:0/161979529 learned_addr learned my addr 172.21.15.86:0/161979529 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:08:41.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.857+0000 7f817be00700 1 --2- 172.21.15.86:0/161979529 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f817c19dd40 0x7f817c19e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:41.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.857+0000 7f817aa00700 1 --2- 172.21.15.86:0/161979529 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f817c106090 0x7f817c1999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:41.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.857+0000 7f817b400700 1 -- 172.21.15.86:0/161979529 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f817c106090 msgr2=0x7f817c1999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:41.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.857+0000 7f817b400700 1 --2- 172.21.15.86:0/161979529 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f817c106090 0x7f817c1999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:41.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.857+0000 7f817b400700 1 -- 172.21.15.86:0/161979529 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f817c19dd40 msgr2=0x7f817c19e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:41.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.857+0000 7f817b400700 1 --2- 172.21.15.86:0/161979529 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f817c19dd40 0x7f817c19e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:41.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.857+0000 7f817b400700 1 -- 172.21.15.86:0/161979529 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8168009520 con 0x7f817c100f70 2024-03-23T18:08:41.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.857+0000 7f817b400700 1 --2- 172.21.15.86:0/161979529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f817c100f70 0x7f817c199460 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f816c00e9c0 tx=0x7f816c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:08:41.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.858+0000 7f8173e00700 1 -- 172.21.15.86:0/161979529 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f816c004a30 con 0x7f817c100f70 2024-03-23T18:08:41.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.858+0000 7f8173e00700 1 -- 172.21.15.86:0/161979529 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f816c00cc80 con 0x7f817c100f70 2024-03-23T18:08:41.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.858+0000 7f8182600700 1 -- 172.21.15.86:0/161979529 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f817c1a5ff0 con 0x7f817c100f70 2024-03-23T18:08:41.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.858+0000 7f8173e00700 1 -- 172.21.15.86:0/161979529 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f816c0054e0 con 0x7f817c100f70 2024-03-23T18:08:41.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.858+0000 7f8182600700 1 -- 172.21.15.86:0/161979529 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f817c1a64c0 con 0x7f817c100f70 2024-03-23T18:08:41.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.859+0000 7f8182600700 1 -- 172.21.15.86:0/161979529 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f817c102440 con 0x7f817c100f70 2024-03-23T18:08:41.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.861+0000 7f8173e00700 1 -- 172.21.15.86:0/161979529 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f816c004b90 con 0x7f817c100f70 2024-03-23T18:08:41.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.863+0000 7f8173e00700 1 --2- 172.21.15.86:0/161979529 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f81600a1310 0x7f81600a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:41.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.863+0000 7f8173e00700 1 -- 172.21.15.86:0/161979529 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f816c014070 con 0x7f817c100f70 2024-03-23T18:08:41.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.863+0000 7f817aa00700 1 --2- 172.21.15.86:0/161979529 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f81600a1310 0x7f81600a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:41.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.863+0000 7f817aa00700 1 --2- 172.21.15.86:0/161979529 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f81600a1310 0x7f81600a37d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8168005e90 tx=0x7f8168019040 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:08:41.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:41.867+0000 7f8173e00700 1 -- 172.21.15.86:0/161979529 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f816c096160 con 0x7f817c100f70 2024-03-23T18:08:42.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:42.076+0000 7f8182600700 1 -- 172.21.15.86:0/161979529 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f817c101920 con 0x7f817c100f70 2024-03-23T18:08:42.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:42.077+0000 7f8173e00700 1 -- 172.21.15.86:0/161979529 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v110) v1 ==== 100+0+11 (secure 0 0 0) 0x7f816c099780 con 0x7f817c100f70 2024-03-23T18:08:42.077 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-03-23T18:08:42.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:42.079+0000 7f8182600700 1 -- 172.21.15.86:0/161979529 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f81600a1310 msgr2=0x7f81600a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:42.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:42.079+0000 7f8182600700 1 --2- 172.21.15.86:0/161979529 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f81600a1310 0x7f81600a37d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8168005e90 tx=0x7f8168019040 comp rx=0 tx=0).stop 2024-03-23T18:08:42.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:42.079+0000 7f8182600700 1 -- 172.21.15.86:0/161979529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f817c100f70 msgr2=0x7f817c199460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:42.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:42.079+0000 7f8182600700 1 --2- 172.21.15.86:0/161979529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f817c100f70 0x7f817c199460 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f816c00e9c0 tx=0x7f816c00ecd0 comp rx=0 tx=0).stop 2024-03-23T18:08:42.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:42.079+0000 7f8182600700 1 -- 172.21.15.86:0/161979529 shutdown_connections 2024-03-23T18:08:42.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:42.079+0000 7f8182600700 1 --2- 172.21.15.86:0/161979529 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f817c19dd40 0x7f817c19e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:42.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:42.079+0000 7f8182600700 1 --2- 172.21.15.86:0/161979529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f817c100f70 0x7f817c199460 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:42.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:42.079+0000 7f8182600700 1 --2- 172.21.15.86:0/161979529 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f81600a1310 0x7f81600a37d0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:42.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:42.079+0000 7f8182600700 1 --2- 172.21.15.86:0/161979529 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f817c106090 0x7f817c1999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:42.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:42.079+0000 7f8182600700 1 -- 172.21.15.86:0/161979529 >> 172.21.15.86:0/161979529 conn(0x7f817c0fc7f0 msgr2=0x7f817c0feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:08:42.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:42.080+0000 7f8182600700 1 -- 172.21.15.86:0/161979529 shutdown_connections 2024-03-23T18:08:42.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:42.080+0000 7f8182600700 1 -- 172.21.15.86:0/161979529 wait complete. 2024-03-23T18:08:42.099 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-23T18:08:42.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:42.292+0000 7fe831e00700 1 -- 172.21.15.86:0/1991850815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe82c101910 msgr2=0x7fe82c105950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:42.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:42.292+0000 7fe831e00700 1 --2- 172.21.15.86:0/1991850815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe82c101910 0x7fe82c105950 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fe814009990 tx=0x7fe814009ca0 comp rx=0 tx=0).stop 2024-03-23T18:08:42.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:42.292+0000 7fe831e00700 1 -- 172.21.15.86:0/1991850815 shutdown_connections 2024-03-23T18:08:42.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:42.292+0000 7fe831e00700 1 --2- 172.21.15.86:0/1991850815 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe82c106080 0x7fe82c10dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:42.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:42.292+0000 7fe831e00700 1 --2- 172.21.15.86:0/1991850815 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe82c100f60 0x7fe82c101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:42.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:42.292+0000 7fe831e00700 1 --2- 172.21.15.86:0/1991850815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe82c101910 0x7fe82c105950 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:42.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:42.292+0000 7fe831e00700 1 -- 172.21.15.86:0/1991850815 >> 172.21.15.86:0/1991850815 conn(0x7fe82c0fc7c0 msgr2=0x7fe82c0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:08:42.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:42.292+0000 7fe831e00700 1 -- 172.21.15.86:0/1991850815 shutdown_connections 2024-03-23T18:08:42.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:42.292+0000 7fe831e00700 1 -- 172.21.15.86:0/1991850815 wait complete. 2024-03-23T18:08:42.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:42.293+0000 7fe831e00700 1 Processor -- start 2024-03-23T18:08:42.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:42.293+0000 7fe831e00700 1 -- start start 2024-03-23T18:08:42.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:42.293+0000 7fe831e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe82c100f60 0x7fe82c0730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:42.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:42.293+0000 7fe831e00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe82c106080 0x7fe82c0735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:42.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:42.294+0000 7fe831e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe82c073b30 0x7fe82c1a53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:42.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:42.294+0000 7fe831e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe82c1a59e0 con 0x7fe82c100f60 2024-03-23T18:08:42.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:42.294+0000 7fe831e00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe82c1a5b50 con 0x7fe82c073b30 2024-03-23T18:08:42.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:42.294+0000 7fe831e00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe82c1a5d10 con 0x7fe82c106080 2024-03-23T18:08:42.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:42.294+0000 7fe82aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe82c100f60 0x7fe82c0730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:42.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:42.294+0000 7fe82aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe82c100f60 0x7fe82c0730b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46792/0 (socket says 172.21.15.86:46792) 2024-03-23T18:08:42.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:42.294+0000 7fe82aa00700 1 -- 172.21.15.86:0/2693030424 learned_addr learned my addr 172.21.15.86:0/2693030424 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:08:42.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:42.294+0000 7fe823e00700 1 --2- 172.21.15.86:0/2693030424 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe82c106080 0x7fe82c0735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:42.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:42.294+0000 7fe82b400700 1 --2- 172.21.15.86:0/2693030424 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe82c073b30 0x7fe82c1a53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:42.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:42.294+0000 7fe82aa00700 1 -- 172.21.15.86:0/2693030424 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe82c106080 msgr2=0x7fe82c0735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:42.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:42.294+0000 7fe82aa00700 1 --2- 172.21.15.86:0/2693030424 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe82c106080 0x7fe82c0735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:42.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:42.294+0000 7fe82aa00700 1 -- 172.21.15.86:0/2693030424 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe82c073b30 msgr2=0x7fe82c1a53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:42.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:42.294+0000 7fe82aa00700 1 --2- 172.21.15.86:0/2693030424 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe82c073b30 0x7fe82c1a53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:42.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:42.294+0000 7fe82aa00700 1 -- 172.21.15.86:0/2693030424 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8140095f0 con 0x7fe82c100f60 2024-03-23T18:08:42.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:42.294+0000 7fe82aa00700 1 --2- 172.21.15.86:0/2693030424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe82c100f60 0x7fe82c0730b0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fe81c00e9a0 tx=0x7fe81c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:08:42.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:42.295+0000 7fe830a00700 1 -- 172.21.15.86:0/2693030424 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe81c009790 con 0x7fe82c100f60 2024-03-23T18:08:42.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:42.295+0000 7fe830a00700 1 -- 172.21.15.86:0/2693030424 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe81c010430 con 0x7fe82c100f60 2024-03-23T18:08:42.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:42.295+0000 7fe831e00700 1 -- 172.21.15.86:0/2693030424 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe82c1a5ff0 con 0x7fe82c100f60 2024-03-23T18:08:42.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:42.295+0000 7fe830a00700 1 -- 172.21.15.86:0/2693030424 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe81c018680 con 0x7fe82c100f60 2024-03-23T18:08:42.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:42.295+0000 7fe831e00700 1 -- 172.21.15.86:0/2693030424 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe82c1a63c0 con 0x7fe82c100f60 2024-03-23T18:08:42.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:42.296+0000 7fe831e00700 1 -- 172.21.15.86:0/2693030424 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe82c109350 con 0x7fe82c100f60 2024-03-23T18:08:42.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:42.298+0000 7fe830a00700 1 -- 172.21.15.86:0/2693030424 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fe81c0105a0 con 0x7fe82c100f60 2024-03-23T18:08:42.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:42.299+0000 7fe830a00700 1 --2- 172.21.15.86:0/2693030424 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fe8100a1290 0x7fe8100a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:42.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:42.299+0000 7fe830a00700 1 -- 172.21.15.86:0/2693030424 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fe81c014070 con 0x7fe82c100f60 2024-03-23T18:08:42.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:42.299+0000 7fe823e00700 1 --2- 172.21.15.86:0/2693030424 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fe8100a1290 0x7fe8100a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:42.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:42.300+0000 7fe823e00700 1 --2- 172.21.15.86:0/2693030424 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fe8100a1290 0x7fe8100a3750 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe81400b2e0 tx=0x7fe814005960 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:08:42.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:42.301+0000 7fe830a00700 1 -- 172.21.15.86:0/2693030424 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fe81c096490 con 0x7fe82c100f60 2024-03-23T18:08:42.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:42 smithi152 ceph-mon[47949]: pgmap v571: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 2.9 MiB/s wr, 146 op/s 2024-03-23T18:08:42.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:42 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/2142352687' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T18:08:42.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:42 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/1690535429' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T18:08:42.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:42 smithi086 ceph-mon[38221]: pgmap v571: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 2.9 MiB/s wr, 146 op/s 2024-03-23T18:08:42.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:42 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/2142352687' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T18:08:42.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:42 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/1690535429' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T18:08:42.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:42 smithi092 ceph-mon[79089]: pgmap v571: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 2.9 MiB/s wr, 146 op/s 2024-03-23T18:08:42.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:42 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/2142352687' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T18:08:42.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:42 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/1690535429' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T18:08:42.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:42.559+0000 7fe831e00700 1 -- 172.21.15.86:0/2693030424 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe82c102430 con 0x7fe82c100f60 2024-03-23T18:08:42.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:42.560+0000 7fe830a00700 1 -- 172.21.15.86:0/2693030424 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v110) v1 ==== 81+0+942 (secure 0 0 0) 0x7fe81c026080 con 0x7fe82c100f60 2024-03-23T18:08:42.560 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6801/2734784244 2024-03-24T17:50:38.377487+0000 2024-03-23T18:08:42.560 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6800/2734784244 2024-03-24T17:50:38.377487+0000 2024-03-23T18:08:42.560 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/734928365 2024-03-24T17:50:38.377487+0000 2024-03-23T18:08:42.560 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/2354115349 2024-03-24T17:50:38.377487+0000 2024-03-23T18:08:42.560 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/3553500912 2024-03-24T17:43:28.242492+0000 2024-03-23T18:08:42.560 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6801/3600278228 2024-03-24T17:43:28.242492+0000 2024-03-23T18:08:42.561 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/887262329 2024-03-24T17:43:28.242492+0000 2024-03-23T18:08:42.561 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6800/2313723492 2024-03-24T17:43:05.043910+0000 2024-03-23T18:08:42.561 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6800/3600278228 2024-03-24T17:43:28.242492+0000 2024-03-23T18:08:42.561 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/3815705642 2024-03-24T17:43:05.043910+0000 2024-03-23T18:08:42.561 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/883052430 2024-03-24T17:50:38.377487+0000 2024-03-23T18:08:42.561 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/1614943062 2024-03-24T17:43:28.242492+0000 2024-03-23T18:08:42.561 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6801/2313723492 2024-03-24T17:43:05.043910+0000 2024-03-23T18:08:42.561 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/2416321716 2024-03-24T17:43:05.043910+0000 2024-03-23T18:08:42.561 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/2938296862 2024-03-24T17:50:38.377487+0000 2024-03-23T18:08:42.562 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/433588188 2024-03-24T17:43:05.043910+0000 2024-03-23T18:08:42.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:42.562+0000 7fe831e00700 1 -- 172.21.15.86:0/2693030424 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fe8100a1290 msgr2=0x7fe8100a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:42.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:42.562+0000 7fe831e00700 1 --2- 172.21.15.86:0/2693030424 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fe8100a1290 0x7fe8100a3750 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe81400b2e0 tx=0x7fe814005960 comp rx=0 tx=0).stop 2024-03-23T18:08:42.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:42.562+0000 7fe831e00700 1 -- 172.21.15.86:0/2693030424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe82c100f60 msgr2=0x7fe82c0730b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:42.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:42.562+0000 7fe831e00700 1 --2- 172.21.15.86:0/2693030424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe82c100f60 0x7fe82c0730b0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fe81c00e9a0 tx=0x7fe81c00ecb0 comp rx=0 tx=0).stop 2024-03-23T18:08:42.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:42.562+0000 7fe831e00700 1 -- 172.21.15.86:0/2693030424 shutdown_connections 2024-03-23T18:08:42.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:42.562+0000 7fe831e00700 1 --2- 172.21.15.86:0/2693030424 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe82c073b30 0x7fe82c1a53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:42.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:42.562+0000 7fe831e00700 1 --2- 172.21.15.86:0/2693030424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe82c100f60 0x7fe82c0730b0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:42.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:42.562+0000 7fe831e00700 1 --2- 172.21.15.86:0/2693030424 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fe8100a1290 0x7fe8100a3750 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:42.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:42.562+0000 7fe831e00700 1 --2- 172.21.15.86:0/2693030424 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe82c106080 0x7fe82c0735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:42.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:42.562+0000 7fe831e00700 1 -- 172.21.15.86:0/2693030424 >> 172.21.15.86:0/2693030424 conn(0x7fe82c0fc7c0 msgr2=0x7fe82c0fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:08:42.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:42.563+0000 7fe831e00700 1 -- 172.21.15.86:0/2693030424 shutdown_connections 2024-03-23T18:08:42.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:42.563+0000 7fe831e00700 1 -- 172.21.15.86:0/2693030424 wait complete. 2024-03-23T18:08:42.565 INFO:teuthology.orchestra.run.smithi086.stderr:listed 16 entries 2024-03-23T18:08:42.581 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-23T18:08:42.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T18:08:42.581 DEBUG:teuthology.orchestra.run.smithi086:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-23T18:08:43.235 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-03-23T18:08:43.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T18:08:43.236 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-23T18:08:43.349 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T18:08:43.349 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-03-23T18:08:43.441 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-23T18:08:43.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T18:08:43.443 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-23T18:08:43.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:43 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/161979529' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T18:08:43.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:43 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/2693030424' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-23T18:08:43.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T18:08:43.554 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link delete ceph-brx 2024-03-23T18:08:43.556 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:43 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/161979529' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T18:08:43.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:43 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/2693030424' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-23T18:08:43.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:43 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/161979529' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T18:08:43.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:43 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/2693030424' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-23T18:08:43.669 INFO:tasks.kclient:Unmounting kernel clients... 2024-03-23T18:08:43.669 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-03-23T18:08:43.670 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T18:08:43.714 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345668k,nr_inodes=4086417,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=38,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275388k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/bd981bc6b7424497e0c6d9ead3e4afef5584acb41ad666b4fd2cc6f1a713dcae/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/7KU2XEJAPJ7RSHFHEL6EMSALIM:/var/lib/containers/storage/overlay/l/RZ6KBVOVOS3276J47X7XFBIXLO,upperdir=/var/lib/containers/storage/overlay/bd981bc6b7424497e0c6d9ead3e4afef5584acb41ad666b4fd2cc6f1a713dcae/diff,workdir=/var/lib/containers/storage/overlay/bd981bc6b7424497e0c6d9ead3e4afef5584acb41ad666b4fd2cc6f1a713dcae/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/0516646552d847334955c1027d1ba98cbe4a492a50afd209b33ee6744dc72c9a/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/7KU2XEJAPJ7RSHFHEL6EMSALIM:/var/lib/containers/storage/overlay/l/RZ6KBVOVOS3276J47X7XFBIXLO,upperdir=/var/lib/containers/storage/overlay/0516646552d847334955c1027d1ba98cbe4a492a50afd209b33ee6744dc72c9a/diff,workdir=/var/lib/containers/storage/overlay/0516646552d847334955c1027d1ba98cbe4a492a50afd209b33ee6744dc72c9a/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275388k,mode=700 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,relatime 0 0 overlay /var/lib/containers/storage/overlay/9cc5a2b3a5f521ccb6ea3532cedea9106f630be87e55ba35edfce23404c72e89/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/7KU2XEJAPJ7RSHFHEL6EMSALIM:/var/lib/containers/storage/overlay/l/RZ6KBVOVOS3276J47X7XFBIXLO,upperdir=/var/lib/containers/storage/overlay/9cc5a2b3a5f521ccb6ea3532cedea9106f630be87e55ba35edfce23404c72e89/diff,workdir=/var/lib/containers/storage/overlay/9cc5a2b3a5f521ccb6ea3532cedea9106f630be87e55ba35edfce23404c72e89/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/1aecb0fe868ced24c36875e96789ec9a9207e23fd2d3b0ba9f17b99a9de5fa5b/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/7KU2XEJAPJ7RSHFHEL6EMSALIM:/var/lib/containers/storage/overlay/l/RZ6KBVOVOS3276J47X7XFBIXLO,upperdir=/var/lib/containers/storage/overlay/1aecb0fe868ced24c36875e96789ec9a9207e23fd2d3b0ba9f17b99a9de5fa5b/diff,workdir=/var/lib/containers/storage/overlay/1aecb0fe868ced24c36875e96789ec9a9207e23fd2d3b0ba9f17b99a9de5fa5b/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/b9b73e9320125a1e11ad271e163931c61a24834f8245a4084511cada9d352908/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/7KU2XEJAPJ7RSHFHEL6EMSALIM:/var/lib/containers/storage/overlay/l/RZ6KBVOVOS3276J47X7XFBIXLO,upperdir=/var/lib/containers/storage/overlay/b9b73e9320125a1e11ad271e163931c61a24834f8245a4084511cada9d352908/diff,workdir=/var/lib/containers/storage/overlay/b9b73e9320125a1e11ad271e163931c61a24834f8245a4084511cada9d352908/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/22d586185045f71ff72a589c553b6acad04b88eb2d137404e31a54eb58a2b87b/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/7KU2XEJAPJ7RSHFHEL6EMSALIM:/var/lib/containers/storage/overlay/l/RZ6KBVOVOS3276J47X7XFBIXLO,upperdir=/var/lib/containers/storage/overlay/22d586185045f71ff72a589c553b6acad04b88eb2d137404e31a54eb58a2b87b/diff,workdir=/var/lib/containers/storage/overlay/22d586185045f71ff72a589c553b6acad04b88eb2d137404e31a54eb58a2b87b/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/65a340eb9c2f0290183971eec184726a07502bf760a7b2c53e4507534d2d69fb/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/7KU2XEJAPJ7RSHFHEL6EMSALIM:/var/lib/containers/storage/overlay/l/RZ6KBVOVOS3276J47X7XFBIXLO,upperdir=/var/lib/containers/storage/overlay/65a340eb9c2f0290183971eec184726a07502bf760a7b2c53e4507534d2d69fb/diff,workdir=/var/lib/containers/storage/overlay/65a340eb9c2f0290183971eec184726a07502bf760a7b2c53e4507534d2d69fb/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/2ed49fc1767f7cf74d288e7445ba5950ef6feac8828ecaeda8cba7814330ae3a/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/7KU2XEJAPJ7RSHFHEL6EMSALIM:/var/lib/containers/storage/overlay/l/RZ6KBVOVOS3276J47X7XFBIXLO,upperdir=/var/lib/containers/storage/overlay/2ed49fc1767f7cf74d288e7445ba5950ef6feac8828ecaeda8cba7814330ae3a/diff,workdir=/var/lib/containers/storage/overlay/2ed49fc1767f7cf74d288e7445ba5950ef6feac8828ecaeda8cba7814330ae3a/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/9d00837e7b6652d69467c10ad263a76979614d67a9559a8a660c9829d67f8ea8/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/7KU2XEJAPJ7RSHFHEL6EMSALIM:/var/lib/containers/storage/overlay/l/RZ6KBVOVOS3276J47X7XFBIXLO,upperdir=/var/lib/containers/storage/overlay/9d00837e7b6652d69467c10ad263a76979614d67a9559a8a660c9829d67f8ea8/diff,workdir=/var/lib/containers/storage/overlay/9d00837e7b6652d69467c10ad263a76979614d67a9559a8a660c9829d67f8ea8/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/24579110df3545c390a5638e21ca7c747165e3fc0c11c3a75c4b00a80ba50e00/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/7KU2XEJAPJ7RSHFHEL6EMSALIM:/var/lib/containers/storage/overlay/l/RZ6KBVOVOS3276J47X7XFBIXLO,upperdir=/var/lib/containers/storage/overlay/24579110df3545c390a5638e21ca7c747165e3fc0c11c3a75c4b00a80ba50e00/diff,workdir=/var/lib/containers/storage/overlay/24579110df3545c390a5638e21ca7c747165e3fc0c11c3a75c4b00a80ba50e00/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 2024-03-23T18:08:43.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T18:08:43.714 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-03-23T18:08:43.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T18:08:43.797 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link delete ceph-brx 2024-03-23T18:08:43.908 INFO:teuthology.orchestra.run.smithi086.stderr:Cannot find device "ceph-brx" 2024-03-23T18:08:43.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T18:08:43.911 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-03-23T18:08:43.922 INFO:tasks.cephadm:Teardown begin 2024-03-23T18:08:43.923 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-23T18:08:43.923 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T18:08:44.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.116+0000 7fa960400700 1 -- 172.21.15.86:0/1560366658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa958101810 msgr2=0x7fa958105850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:44.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.116+0000 7fa960400700 1 --2- 172.21.15.86:0/1560366658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa958101810 0x7fa958105850 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fa944009990 tx=0x7fa944009ca0 comp rx=0 tx=0).stop 2024-03-23T18:08:44.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.116+0000 7fa960400700 1 -- 172.21.15.86:0/1560366658 shutdown_connections 2024-03-23T18:08:44.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.116+0000 7fa960400700 1 --2- 172.21.15.86:0/1560366658 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa958105f80 0x7fa95810db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:44.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.116+0000 7fa960400700 1 --2- 172.21.15.86:0/1560366658 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa958100e60 0x7fa958101240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:44.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.116+0000 7fa960400700 1 --2- 172.21.15.86:0/1560366658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa958101810 0x7fa958105850 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:44.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.116+0000 7fa960400700 1 -- 172.21.15.86:0/1560366658 >> 172.21.15.86:0/1560366658 conn(0x7fa9580fc760 msgr2=0x7fa9580feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:08:44.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.116+0000 7fa960400700 1 -- 172.21.15.86:0/1560366658 shutdown_connections 2024-03-23T18:08:44.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.116+0000 7fa960400700 1 -- 172.21.15.86:0/1560366658 wait complete. 2024-03-23T18:08:44.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.117+0000 7fa960400700 1 Processor -- start 2024-03-23T18:08:44.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.117+0000 7fa960400700 1 -- start start 2024-03-23T18:08:44.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.117+0000 7fa960400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa958100e60 0x7fa958199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:44.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.117+0000 7fa960400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa958105f80 0x7fa9581998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:44.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.117+0000 7fa960400700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa95819dc50 0x7fa95819e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:44.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.117+0000 7fa960400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa95819e6d0 con 0x7fa958105f80 2024-03-23T18:08:44.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.117+0000 7fa960400700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa95819e840 con 0x7fa958100e60 2024-03-23T18:08:44.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.117+0000 7fa960400700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa95819ea00 con 0x7fa95819dc50 2024-03-23T18:08:44.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.118+0000 7fa95d000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa958105f80 0x7fa9581998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:44.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.118+0000 7fa95da00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa958100e60 0x7fa958199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:44.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.118+0000 7fa95da00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa958100e60 0x7fa958199370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:49852/0 (socket says 172.21.15.86:49852) 2024-03-23T18:08:44.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.118+0000 7fa95d000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa958105f80 0x7fa9581998b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46814/0 (socket says 172.21.15.86:46814) 2024-03-23T18:08:44.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.118+0000 7fa95da00700 1 -- 172.21.15.86:0/2204171608 learned_addr learned my addr 172.21.15.86:0/2204171608 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:08:44.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.118+0000 7fa95e400700 1 --2- 172.21.15.86:0/2204171608 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa95819dc50 0x7fa95819e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:44.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.118+0000 7fa95d000700 1 -- 172.21.15.86:0/2204171608 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa95819dc50 msgr2=0x7fa95819e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:44.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.118+0000 7fa95d000700 1 --2- 172.21.15.86:0/2204171608 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa95819dc50 0x7fa95819e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:44.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.118+0000 7fa95d000700 1 -- 172.21.15.86:0/2204171608 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa958100e60 msgr2=0x7fa958199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:44.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.118+0000 7fa95d000700 1 --2- 172.21.15.86:0/2204171608 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa958100e60 0x7fa958199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:44.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.118+0000 7fa95d000700 1 -- 172.21.15.86:0/2204171608 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9440095f0 con 0x7fa958105f80 2024-03-23T18:08:44.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.118+0000 7fa95da00700 1 --2- 172.21.15.86:0/2204171608 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa958100e60 0x7fa958199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T18:08:44.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.118+0000 7fa95d000700 1 --2- 172.21.15.86:0/2204171608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa958105f80 0x7fa9581998b0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fa9440098e0 tx=0x7fa944005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:08:44.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.119+0000 7fa956000700 1 -- 172.21.15.86:0/2204171608 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa94401d070 con 0x7fa958105f80 2024-03-23T18:08:44.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.119+0000 7fa956000700 1 -- 172.21.15.86:0/2204171608 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa944022470 con 0x7fa958105f80 2024-03-23T18:08:44.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.119+0000 7fa960400700 1 -- 172.21.15.86:0/2204171608 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9581a5f00 con 0x7fa958105f80 2024-03-23T18:08:44.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.119+0000 7fa956000700 1 -- 172.21.15.86:0/2204171608 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa944017650 con 0x7fa958105f80 2024-03-23T18:08:44.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.119+0000 7fa960400700 1 -- 172.21.15.86:0/2204171608 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9581a62c0 con 0x7fa958105f80 2024-03-23T18:08:44.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.120+0000 7fa960400700 1 -- 172.21.15.86:0/2204171608 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa958102330 con 0x7fa958105f80 2024-03-23T18:08:44.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.122+0000 7fa956000700 1 -- 172.21.15.86:0/2204171608 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fa944022850 con 0x7fa958105f80 2024-03-23T18:08:44.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.124+0000 7fa956000700 1 --2- 172.21.15.86:0/2204171608 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fa9400a1290 0x7fa9400a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:44.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.124+0000 7fa956000700 1 -- 172.21.15.86:0/2204171608 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 8484+0+0 (secure 0 0 0) 0x7fa9440cdf10 con 0x7fa958105f80 2024-03-23T18:08:44.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.124+0000 7fa95da00700 1 --2- 172.21.15.86:0/2204171608 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fa9400a1290 0x7fa9400a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:44.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.124+0000 7fa95da00700 1 --2- 172.21.15.86:0/2204171608 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fa9400a1290 0x7fa9400a3750 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa94c011550 tx=0x7fa94c009260 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:08:44.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.128+0000 7fa956000700 1 -- 172.21.15.86:0/2204171608 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa9440975e0 con 0x7fa958105f80 2024-03-23T18:08:44.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:44 smithi086 ceph-mon[38221]: pgmap v572: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 657 KiB/s wr, 45 op/s 2024-03-23T18:08:44.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:44 smithi152 ceph-mon[47949]: pgmap v572: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 657 KiB/s wr, 45 op/s 2024-03-23T18:08:44.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.383+0000 7fa960400700 1 -- 172.21.15.86:0/2204171608 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa958101810 con 0x7fa958105f80 2024-03-23T18:08:44.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.383+0000 7fa956000700 1 -- 172.21.15.86:0/2204171608 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+10921 (secure 0 0 0) 0x7fa944027070 con 0x7fa958105f80 2024-03-23T18:08:44.383 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T18:08:44.384 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":90,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14445,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1433140219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1433140219},{"type":"v1","addr":"172.21.15.86:6835","nonce":1433140219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6839/1879457730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1879457730},{"type":"v1","addr":"172.21.15.86:6839","nonce":1879457730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":90,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-23T17:48:56.202323+0000","modified":"2024-03-23T18:07:44.740222+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24553,"mds_2":24518,"mds_3":24505,"mds_4":24500},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14514":{"gid":14514,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/3026943866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3026943866},{"type":"v1","addr":"172.21.15.86:6841","nonce":3026943866}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24464":{"gid":24464,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6833/1467091186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1467091186},{"type":"v1","addr":"172.21.15.92:6833","nonce":1467091186}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6833/3075554623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3075554623},{"type":"v1","addr":"172.21.15.152:6833","nonce":3075554623}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":55,"addr":"172.21.15.92:6835/908900027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":908900027},{"type":"v1","addr":"172.21.15.92:6835","nonce":908900027}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"d","rank":3,"incarnation":23,"state":"up:active","state_seq":57,"addr":"172.21.15.86:6837/2933881328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2933881328},{"type":"v1","addr":"172.21.15.86:6837","nonce":2933881328}]},"join_fscid":-1,"export_targets":[1,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":54,"addr":"172.21.15.92:6837/3967837948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":3967837948},{"type":"v1","addr":"172.21.15.92:6837","nonce":3967837948}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6835/3137085021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3137085021},{"type":"v1","addr":"172.21.15.152:6835","nonce":3137085021}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.152:6839/1100723992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":1100723992},{"type":"v1","addr":"172.21.15.152:6839","nonce":1100723992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6839/1394139887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1394139887},{"type":"v1","addr":"172.21.15.92:6839","nonce":1394139887}]},"join_fscid":-1,"export_targets":[1,2,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_27326":{"gid":27326,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6837/988988765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":988988765},{"type":"v1","addr":"172.21.15.152:6837","nonce":988988765}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-23T18:08:44.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.386+0000 7fa960400700 1 -- 172.21.15.86:0/2204171608 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fa9400a1290 msgr2=0x7fa9400a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:44.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.386+0000 7fa960400700 1 --2- 172.21.15.86:0/2204171608 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fa9400a1290 0x7fa9400a3750 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa94c011550 tx=0x7fa94c009260 comp rx=0 tx=0).stop 2024-03-23T18:08:44.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.386+0000 7fa960400700 1 -- 172.21.15.86:0/2204171608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa958105f80 msgr2=0x7fa9581998b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:44.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.386+0000 7fa960400700 1 --2- 172.21.15.86:0/2204171608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa958105f80 0x7fa9581998b0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fa9440098e0 tx=0x7fa944005740 comp rx=0 tx=0).stop 2024-03-23T18:08:44.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.386+0000 7fa960400700 1 -- 172.21.15.86:0/2204171608 shutdown_connections 2024-03-23T18:08:44.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.386+0000 7fa960400700 1 --2- 172.21.15.86:0/2204171608 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fa9400a1290 0x7fa9400a3750 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:44.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.386+0000 7fa960400700 1 --2- 172.21.15.86:0/2204171608 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa95819dc50 0x7fa95819e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:44.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.386+0000 7fa960400700 1 --2- 172.21.15.86:0/2204171608 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa958100e60 0x7fa958199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:44.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.386+0000 7fa960400700 1 --2- 172.21.15.86:0/2204171608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa958105f80 0x7fa9581998b0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:44.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.386+0000 7fa960400700 1 -- 172.21.15.86:0/2204171608 >> 172.21.15.86:0/2204171608 conn(0x7fa9580fc760 msgr2=0x7fa9580fdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:08:44.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.386+0000 7fa960400700 1 -- 172.21.15.86:0/2204171608 shutdown_connections 2024-03-23T18:08:44.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.386+0000 7fa960400700 1 -- 172.21.15.86:0/2204171608 wait complete. 2024-03-23T18:08:44.388 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 90 2024-03-23T18:08:44.406 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T18:08:44.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:44 smithi092 ceph-mon[79089]: pgmap v572: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 657 KiB/s wr, 45 op/s 2024-03-23T18:08:44.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.597+0000 7ff2bb200700 1 -- 172.21.15.86:0/3888603925 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2bc102750 msgr2=0x7ff2bc10ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:44.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.597+0000 7ff2bb200700 1 --2- 172.21.15.86:0/3888603925 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2bc102750 0x7ff2bc10ac30 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ff2a8009990 tx=0x7ff2a8009ca0 comp rx=0 tx=0).stop 2024-03-23T18:08:44.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.598+0000 7ff2bb200700 1 -- 172.21.15.86:0/3888603925 shutdown_connections 2024-03-23T18:08:44.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.598+0000 7ff2bb200700 1 --2- 172.21.15.86:0/3888603925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2bc10b200 0x7ff2bc10d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:44.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.598+0000 7ff2bb200700 1 --2- 172.21.15.86:0/3888603925 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff2bc101e30 0x7ff2bc102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:44.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.598+0000 7ff2bb200700 1 --2- 172.21.15.86:0/3888603925 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2bc102750 0x7ff2bc10ac30 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:44.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.598+0000 7ff2bb200700 1 -- 172.21.15.86:0/3888603925 >> 172.21.15.86:0/3888603925 conn(0x7ff2bc0fb5d0 msgr2=0x7ff2bc0fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:08:44.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.598+0000 7ff2bb200700 1 -- 172.21.15.86:0/3888603925 shutdown_connections 2024-03-23T18:08:44.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.598+0000 7ff2bb200700 1 -- 172.21.15.86:0/3888603925 wait complete. 2024-03-23T18:08:44.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.599+0000 7ff2bb200700 1 Processor -- start 2024-03-23T18:08:44.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.599+0000 7ff2bb200700 1 -- start start 2024-03-23T18:08:44.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.599+0000 7ff2bb200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2bc101e30 0x7ff2bc199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:44.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.599+0000 7ff2bb200700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff2bc102750 0x7ff2bc1997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:44.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.599+0000 7ff2bb200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2bc10b200 0x7ff2bc19db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:44.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.599+0000 7ff2bb200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2bc19e1b0 con 0x7ff2bc10b200 2024-03-23T18:08:44.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.599+0000 7ff2bb200700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2bc19e320 con 0x7ff2bc101e30 2024-03-23T18:08:44.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.599+0000 7ff2bb200700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2bc19e4e0 con 0x7ff2bc102750 2024-03-23T18:08:44.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.599+0000 7ff2ba800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2bc10b200 0x7ff2bc19db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:44.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.600+0000 7ff2b9e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2bc101e30 0x7ff2bc199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:44.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.600+0000 7ff2b9e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2bc101e30 0x7ff2bc199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:49868/0 (socket says 172.21.15.86:49868) 2024-03-23T18:08:44.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.600+0000 7ff2b9e00700 1 -- 172.21.15.86:0/1400005718 learned_addr learned my addr 172.21.15.86:0/1400005718 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:08:44.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.600+0000 7ff2b9400700 1 --2- 172.21.15.86:0/1400005718 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff2bc102750 0x7ff2bc1997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:44.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.600+0000 7ff2b9e00700 1 -- 172.21.15.86:0/1400005718 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff2bc102750 msgr2=0x7ff2bc1997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:44.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.600+0000 7ff2b9e00700 1 --2- 172.21.15.86:0/1400005718 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff2bc102750 0x7ff2bc1997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:44.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.600+0000 7ff2b9e00700 1 -- 172.21.15.86:0/1400005718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2bc10b200 msgr2=0x7ff2bc19db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:44.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.600+0000 7ff2b9e00700 1 --2- 172.21.15.86:0/1400005718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2bc10b200 0x7ff2bc19db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:44.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.600+0000 7ff2b9e00700 1 -- 172.21.15.86:0/1400005718 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2a80095f0 con 0x7ff2bc101e30 2024-03-23T18:08:44.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.600+0000 7ff2ba800700 1 --2- 172.21.15.86:0/1400005718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2bc10b200 0x7ff2bc19db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T18:08:44.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.600+0000 7ff2b9400700 1 --2- 172.21.15.86:0/1400005718 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff2bc102750 0x7ff2bc1997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T18:08:44.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.600+0000 7ff2b9e00700 1 --2- 172.21.15.86:0/1400005718 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2bc101e30 0x7ff2bc199290 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac00e9a0 tx=0x7ff2ac00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:08:44.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.600+0000 7ff2b2a00700 1 -- 172.21.15.86:0/1400005718 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ac009790 con 0x7ff2bc101e30 2024-03-23T18:08:44.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.601+0000 7ff2b2a00700 1 -- 172.21.15.86:0/1400005718 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff2ac010430 con 0x7ff2bc101e30 2024-03-23T18:08:44.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.601+0000 7ff2bb200700 1 -- 172.21.15.86:0/1400005718 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2bc19e7c0 con 0x7ff2bc101e30 2024-03-23T18:08:44.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.601+0000 7ff2b2a00700 1 -- 172.21.15.86:0/1400005718 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ac018680 con 0x7ff2bc101e30 2024-03-23T18:08:44.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.601+0000 7ff2bb200700 1 -- 172.21.15.86:0/1400005718 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2bc1a6000 con 0x7ff2bc101e30 2024-03-23T18:08:44.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.602+0000 7ff2bb200700 1 -- 172.21.15.86:0/1400005718 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2bc0695c0 con 0x7ff2bc101e30 2024-03-23T18:08:44.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.604+0000 7ff2b2a00700 1 -- 172.21.15.86:0/1400005718 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7ff2ac0105a0 con 0x7ff2bc101e30 2024-03-23T18:08:44.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.605+0000 7ff2b2a00700 1 --2- 172.21.15.86:0/1400005718 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7ff2a00a1290 0x7ff2a00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:44.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.605+0000 7ff2b2a00700 1 -- 172.21.15.86:0/1400005718 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 8484+0+0 (secure 0 0 0) 0x7ff2ac014070 con 0x7ff2bc101e30 2024-03-23T18:08:44.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.605+0000 7ff2b9400700 1 --2- 172.21.15.86:0/1400005718 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7ff2a00a1290 0x7ff2a00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:44.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.606+0000 7ff2b9400700 1 --2- 172.21.15.86:0/1400005718 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7ff2a00a1290 0x7ff2a00a3750 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff2a80098e0 tx=0x7ff2a8005960 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:08:44.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.607+0000 7ff2b2a00700 1 -- 172.21.15.86:0/1400005718 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ff2ac096520 con 0x7ff2bc101e30 2024-03-23T18:08:44.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.868+0000 7ff2bb200700 1 -- 172.21.15.86:0/1400005718 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff2bc19a590 con 0x7ff2bc101e30 2024-03-23T18:08:44.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.869+0000 7ff2b2a00700 1 -- 172.21.15.86:0/1400005718 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+10921 (secure 0 0 0) 0x7ff2ac026090 con 0x7ff2bc101e30 2024-03-23T18:08:44.870 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T18:08:44.871 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":90,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14445,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1433140219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1433140219},{"type":"v1","addr":"172.21.15.86:6835","nonce":1433140219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6839/1879457730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":1879457730},{"type":"v1","addr":"172.21.15.86:6839","nonce":1879457730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":90,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-23T17:48:56.202323+0000","modified":"2024-03-23T18:07:44.740222+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24553,"mds_2":24518,"mds_3":24505,"mds_4":24500},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14514":{"gid":14514,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/3026943866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3026943866},{"type":"v1","addr":"172.21.15.86:6841","nonce":3026943866}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24464":{"gid":24464,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6833/1467091186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1467091186},{"type":"v1","addr":"172.21.15.92:6833","nonce":1467091186}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6833/3075554623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3075554623},{"type":"v1","addr":"172.21.15.152:6833","nonce":3075554623}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":55,"addr":"172.21.15.92:6835/908900027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":908900027},{"type":"v1","addr":"172.21.15.92:6835","nonce":908900027}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"d","rank":3,"incarnation":23,"state":"up:active","state_seq":57,"addr":"172.21.15.86:6837/2933881328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2933881328},{"type":"v1","addr":"172.21.15.86:6837","nonce":2933881328}]},"join_fscid":-1,"export_targets":[1,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":54,"addr":"172.21.15.92:6837/3967837948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":3967837948},{"type":"v1","addr":"172.21.15.92:6837","nonce":3967837948}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6835/3137085021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3137085021},{"type":"v1","addr":"172.21.15.152:6835","nonce":3137085021}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.152:6839/1100723992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":1100723992},{"type":"v1","addr":"172.21.15.152:6839","nonce":1100723992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6839/1394139887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1394139887},{"type":"v1","addr":"172.21.15.92:6839","nonce":1394139887}]},"join_fscid":-1,"export_targets":[1,2,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_27326":{"gid":27326,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6837/988988765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":988988765},{"type":"v1","addr":"172.21.15.152:6837","nonce":988988765}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-23T18:08:44.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.871+0000 7ff2bb200700 1 -- 172.21.15.86:0/1400005718 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7ff2a00a1290 msgr2=0x7ff2a00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:44.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.871+0000 7ff2bb200700 1 --2- 172.21.15.86:0/1400005718 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7ff2a00a1290 0x7ff2a00a3750 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff2a80098e0 tx=0x7ff2a8005960 comp rx=0 tx=0).stop 2024-03-23T18:08:44.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.871+0000 7ff2bb200700 1 -- 172.21.15.86:0/1400005718 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2bc101e30 msgr2=0x7ff2bc199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:44.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.872+0000 7ff2bb200700 1 --2- 172.21.15.86:0/1400005718 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2bc101e30 0x7ff2bc199290 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac00e9a0 tx=0x7ff2ac00ecb0 comp rx=0 tx=0).stop 2024-03-23T18:08:44.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.872+0000 7ff2bb200700 1 -- 172.21.15.86:0/1400005718 shutdown_connections 2024-03-23T18:08:44.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.872+0000 7ff2bb200700 1 --2- 172.21.15.86:0/1400005718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2bc10b200 0x7ff2bc19db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:44.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.872+0000 7ff2bb200700 1 --2- 172.21.15.86:0/1400005718 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2bc101e30 0x7ff2bc199290 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:44.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.872+0000 7ff2bb200700 1 --2- 172.21.15.86:0/1400005718 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7ff2a00a1290 0x7ff2a00a3750 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:44.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.872+0000 7ff2bb200700 1 --2- 172.21.15.86:0/1400005718 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff2bc102750 0x7ff2bc1997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:44.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.872+0000 7ff2bb200700 1 -- 172.21.15.86:0/1400005718 >> 172.21.15.86:0/1400005718 conn(0x7ff2bc0fb5d0 msgr2=0x7ff2bc100ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:08:44.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.872+0000 7ff2bb200700 1 -- 172.21.15.86:0/1400005718 shutdown_connections 2024-03-23T18:08:44.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:44.872+0000 7ff2bb200700 1 -- 172.21.15.86:0/1400005718 wait complete. 2024-03-23T18:08:44.875 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 90 2024-03-23T18:08:44.892 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T18:08:45.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.083+0000 7f4551000700 1 -- 172.21.15.86:0/168982698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f454c10d490 msgr2=0x7f454c10f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:45.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.083+0000 7f4551000700 1 --2- 172.21.15.86:0/168982698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f454c10d490 0x7f454c10f8c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f454400b0a0 tx=0x7f454400b3b0 comp rx=0 tx=0).stop 2024-03-23T18:08:45.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.083+0000 7f4551000700 1 -- 172.21.15.86:0/168982698 shutdown_connections 2024-03-23T18:08:45.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.083+0000 7f4551000700 1 --2- 172.21.15.86:0/168982698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f454c10d490 0x7f454c10f8c0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:45.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.083+0000 7f4551000700 1 --2- 172.21.15.86:0/168982698 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f454c073180 0x7f454c073560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:45.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.083+0000 7f4551000700 1 --2- 172.21.15.86:0/168982698 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f454c073aa0 0x7f454c10cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:45.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.083+0000 7f4551000700 1 -- 172.21.15.86:0/168982698 >> 172.21.15.86:0/168982698 conn(0x7f454c0fc760 msgr2=0x7f454c0feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:08:45.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.083+0000 7f4551000700 1 -- 172.21.15.86:0/168982698 shutdown_connections 2024-03-23T18:08:45.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.083+0000 7f4551000700 1 -- 172.21.15.86:0/168982698 wait complete. 2024-03-23T18:08:45.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.084+0000 7f4551000700 1 Processor -- start 2024-03-23T18:08:45.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.084+0000 7f4551000700 1 -- start start 2024-03-23T18:08:45.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.085+0000 7f4551000700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f454c073180 0x7f454c199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:45.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.085+0000 7f4551000700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f454c073aa0 0x7f454c199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:45.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.085+0000 7f4551000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f454c10d490 0x7f454c19dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:45.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.085+0000 7f4551000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f454c19e360 con 0x7f454c10d490 2024-03-23T18:08:45.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.085+0000 7f4551000700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f454c19e4d0 con 0x7f454c073aa0 2024-03-23T18:08:45.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.085+0000 7f4551000700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f454c19e690 con 0x7f454c073180 2024-03-23T18:08:45.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.085+0000 7f4549400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f454c073aa0 0x7f454c199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:45.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.085+0000 7f454a800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f454c10d490 0x7f454c19dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:45.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.085+0000 7f4549e00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f454c073180 0x7f454c199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:45.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.085+0000 7f4549400700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f454c073aa0 0x7f454c199980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:37500/0 (socket says 172.21.15.86:37500) 2024-03-23T18:08:45.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.085+0000 7f4549400700 1 -- 172.21.15.86:0/932102536 learned_addr learned my addr 172.21.15.86:0/932102536 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:08:45.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.086+0000 7f4549400700 1 -- 172.21.15.86:0/932102536 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f454c073180 msgr2=0x7f454c199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:45.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.086+0000 7f4549400700 1 --2- 172.21.15.86:0/932102536 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f454c073180 0x7f454c199440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:45.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.086+0000 7f4549400700 1 -- 172.21.15.86:0/932102536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f454c10d490 msgr2=0x7f454c19dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:45.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.086+0000 7f4549400700 1 --2- 172.21.15.86:0/932102536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f454c10d490 0x7f454c19dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:45.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.086+0000 7f4549400700 1 -- 172.21.15.86:0/932102536 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4544009d00 con 0x7f454c073aa0 2024-03-23T18:08:45.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.086+0000 7f454a800700 1 --2- 172.21.15.86:0/932102536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f454c10d490 0x7f454c19dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T18:08:45.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.086+0000 7f4549e00700 1 --2- 172.21.15.86:0/932102536 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f454c073180 0x7f454c199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T18:08:45.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.086+0000 7f4549400700 1 --2- 172.21.15.86:0/932102536 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f454c073aa0 0x7f454c199980 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f453800e9a0 tx=0x7f453800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:08:45.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.087+0000 7f4542a00700 1 -- 172.21.15.86:0/932102536 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4538009790 con 0x7f454c073aa0 2024-03-23T18:08:45.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.087+0000 7f4542a00700 1 -- 172.21.15.86:0/932102536 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4538010430 con 0x7f454c073aa0 2024-03-23T18:08:45.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.087+0000 7f4551000700 1 -- 172.21.15.86:0/932102536 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f454c19e970 con 0x7f454c073aa0 2024-03-23T18:08:45.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.087+0000 7f4542a00700 1 -- 172.21.15.86:0/932102536 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4538018680 con 0x7f454c073aa0 2024-03-23T18:08:45.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.087+0000 7f4551000700 1 -- 172.21.15.86:0/932102536 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f454c1a61b0 con 0x7f454c073aa0 2024-03-23T18:08:45.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.088+0000 7f4551000700 1 -- 172.21.15.86:0/932102536 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f454c1086a0 con 0x7f454c073aa0 2024-03-23T18:08:45.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.090+0000 7f4542a00700 1 -- 172.21.15.86:0/932102536 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f45380105a0 con 0x7f454c073aa0 2024-03-23T18:08:45.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.092+0000 7f4542a00700 1 --2- 172.21.15.86:0/932102536 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f45300a1310 0x7f45300a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:45.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.092+0000 7f4542a00700 1 -- 172.21.15.86:0/932102536 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f4538014070 con 0x7f454c073aa0 2024-03-23T18:08:45.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.092+0000 7f4549e00700 1 --2- 172.21.15.86:0/932102536 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f45300a1310 0x7f45300a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:45.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.092+0000 7f4549e00700 1 --2- 172.21.15.86:0/932102536 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f45300a1310 0x7f45300a37d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f453c011490 tx=0x7f453c009260 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:08:45.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.098+0000 7f4542a00700 1 -- 172.21.15.86:0/932102536 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f45380529f0 con 0x7f454c073aa0 2024-03-23T18:08:45.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:45 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/2204171608' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T18:08:45.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:45 smithi086 ceph-mon[38221]: pgmap v573: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 189 KiB/s wr, 27 op/s 2024-03-23T18:08:45.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:45 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/1400005718' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T18:08:45.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.336+0000 7f4551000700 1 -- 172.21.15.86:0/932102536 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f454c112da0 con 0x7f454c073aa0 2024-03-23T18:08:45.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.338+0000 7f4542a00700 1 -- 172.21.15.86:0/932102536 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+22884 (secure 0 0 0) 0x7f45380961f0 con 0x7f454c073aa0 2024-03-23T18:08:45.339 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-03-23T18:08:45.340 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":110,"fsid":"9d76c9e8-e93c-11ee-95cd-87774f69a715","created":"2024-03-23T17:42:30.097233+0000","modified":"2024-03-23T18:08:15.989252+0000","last_up_change":"2024-03-23T17:48:18.217601+0000","last_in_change":"2024-03-23T17:48:00.250579+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T17:45:26.848397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T17:48:54.331894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T17:48:55.640693+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":30,"snap_epoch":109,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"656655c6-1c7f-4640-92c4-2f42de820143","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6803","nonce":2423505491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6805","nonce":2423505491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6809","nonce":2423505491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2423505491},{"type":"v1","addr":"172.21.15.86:6807","nonce":2423505491}]},"public_addr":"172.21.15.86:6803/2423505491","cluster_addr":"172.21.15.86:6805/2423505491","heartbeat_back_addr":"172.21.15.86:6809/2423505491","heartbeat_front_addr":"172.21.15.86:6807/2423505491","state":["exists","up"]},{"osd":1,"uuid":"3e86dc65-20a6-4de8-b9ce-fcf01f68862b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6801","nonce":3776252597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6803","nonce":3776252597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6807","nonce":3776252597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3776252597},{"type":"v1","addr":"172.21.15.92:6805","nonce":3776252597}]},"public_addr":"172.21.15.92:6801/3776252597","cluster_addr":"172.21.15.92:6803/3776252597","heartbeat_back_addr":"172.21.15.92:6807/3776252597","heartbeat_front_addr":"172.21.15.92:6805/3776252597","state":["exists","up"]},{"osd":2,"uuid":"09808786-3bd3-4d11-8c7b-1736119dfbb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6801","nonce":1601265891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6803","nonce":1601265891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6807","nonce":1601265891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1601265891},{"type":"v1","addr":"172.21.15.152:6805","nonce":1601265891}]},"public_addr":"172.21.15.152:6801/1601265891","cluster_addr":"172.21.15.152:6803/1601265891","heartbeat_back_addr":"172.21.15.152:6807/1601265891","heartbeat_front_addr":"172.21.15.152:6805/1601265891","state":["exists","up"]},{"osd":3,"uuid":"79c61910-4b53-424c-b150-6f5385a2cd68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6811","nonce":694846466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6813","nonce":694846466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6817","nonce":694846466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":694846466},{"type":"v1","addr":"172.21.15.86:6815","nonce":694846466}]},"public_addr":"172.21.15.86:6811/694846466","cluster_addr":"172.21.15.86:6813/694846466","heartbeat_back_addr":"172.21.15.86:6817/694846466","heartbeat_front_addr":"172.21.15.86:6815/694846466","state":["exists","up"]},{"osd":4,"uuid":"a2878035-0179-469f-bbd4-8ce2d3f6bddc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6809","nonce":1422821471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6811","nonce":1422821471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6815","nonce":1422821471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1422821471},{"type":"v1","addr":"172.21.15.92:6813","nonce":1422821471}]},"public_addr":"172.21.15.92:6809/1422821471","cluster_addr":"172.21.15.92:6811/1422821471","heartbeat_back_addr":"172.21.15.92:6815/1422821471","heartbeat_front_addr":"172.21.15.92:6813/1422821471","state":["exists","up"]},{"osd":5,"uuid":"837366aa-ce25-42ca-b157-5bbb50159d0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6809","nonce":1039375239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6811","nonce":1039375239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6815","nonce":1039375239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1039375239},{"type":"v1","addr":"172.21.15.152:6813","nonce":1039375239}]},"public_addr":"172.21.15.152:6809/1039375239","cluster_addr":"172.21.15.152:6811/1039375239","heartbeat_back_addr":"172.21.15.152:6815/1039375239","heartbeat_front_addr":"172.21.15.152:6813/1039375239","state":["exists","up"]},{"osd":6,"uuid":"61024437-7b91-432f-a1af-6443d0ef5213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6819","nonce":4080452707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6821","nonce":4080452707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6825","nonce":4080452707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":4080452707},{"type":"v1","addr":"172.21.15.86:6823","nonce":4080452707}]},"public_addr":"172.21.15.86:6819/4080452707","cluster_addr":"172.21.15.86:6821/4080452707","heartbeat_back_addr":"172.21.15.86:6825/4080452707","heartbeat_front_addr":"172.21.15.86:6823/4080452707","state":["exists","up"]},{"osd":7,"uuid":"2684bdae-6b30-4afc-af68-4ce41b3a75b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6817","nonce":4187707622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6819","nonce":4187707622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6823","nonce":4187707622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":4187707622},{"type":"v1","addr":"172.21.15.92:6821","nonce":4187707622}]},"public_addr":"172.21.15.92:6817/4187707622","cluster_addr":"172.21.15.92:6819/4187707622","heartbeat_back_addr":"172.21.15.92:6823/4187707622","heartbeat_front_addr":"172.21.15.92:6821/4187707622","state":["exists","up"]},{"osd":8,"uuid":"f03bbf0d-1281-4bcf-9773-9854388e65ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6817","nonce":3237612748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6819","nonce":3237612748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6823","nonce":3237612748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":3237612748},{"type":"v1","addr":"172.21.15.152:6821","nonce":3237612748}]},"public_addr":"172.21.15.152:6817/3237612748","cluster_addr":"172.21.15.152:6819/3237612748","heartbeat_back_addr":"172.21.15.152:6823/3237612748","heartbeat_front_addr":"172.21.15.152:6821/3237612748","state":["exists","up"]},{"osd":9,"uuid":"e7e8b775-99ff-48a4-a23a-91e7ab6622bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6827","nonce":2835218294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6829","nonce":2835218294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6833","nonce":2835218294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":2835218294},{"type":"v1","addr":"172.21.15.86:6831","nonce":2835218294}]},"public_addr":"172.21.15.86:6827/2835218294","cluster_addr":"172.21.15.86:6829/2835218294","heartbeat_back_addr":"172.21.15.86:6833/2835218294","heartbeat_front_addr":"172.21.15.86:6831/2835218294","state":["exists","up"]},{"osd":10,"uuid":"d9d616cd-a229-44de-8768-02a4415df115","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6825","nonce":4259465100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6827","nonce":4259465100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6831","nonce":4259465100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":4259465100},{"type":"v1","addr":"172.21.15.92:6829","nonce":4259465100}]},"public_addr":"172.21.15.92:6825/4259465100","cluster_addr":"172.21.15.92:6827/4259465100","heartbeat_back_addr":"172.21.15.92:6831/4259465100","heartbeat_front_addr":"172.21.15.92:6829/4259465100","state":["exists","up"]},{"osd":11,"uuid":"e7c0b865-ccf5-4233-8d22-0d9b6dbe5c97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6825","nonce":194662760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6827","nonce":194662760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6831","nonce":194662760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":194662760},{"type":"v1","addr":"172.21.15.152:6829","nonce":194662760}]},"public_addr":"172.21.15.152:6825/194662760","cluster_addr":"172.21.15.152:6827/194662760","heartbeat_back_addr":"172.21.15.152:6831/194662760","heartbeat_front_addr":"172.21.15.152:6829/194662760","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:44:47.269167+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:45:05.363655+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:45:23.192677+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:45:44.156228+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:02.532971+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:20.153295+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:41.713888+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:46:59.676516+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:47:17.207087+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:47:38.970316+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:47:57.282525+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T17:48:15.437606+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6801/2734784244":"2024-03-24T17:50:38.377487+0000","172.21.15.86:6800/2734784244":"2024-03-24T17:50:38.377487+0000","172.21.15.86:0/734928365":"2024-03-24T17:50:38.377487+0000","172.21.15.86:0/2354115349":"2024-03-24T17:50:38.377487+0000","172.21.15.86:0/3553500912":"2024-03-24T17:43:28.242492+0000","172.21.15.86:6801/3600278228":"2024-03-24T17:43:28.242492+0000","172.21.15.86:0/887262329":"2024-03-24T17:43:28.242492+0000","172.21.15.86:6800/2313723492":"2024-03-24T17:43:05.043910+0000","172.21.15.86:6800/3600278228":"2024-03-24T17:43:28.242492+0000","172.21.15.86:0/3815705642":"2024-03-24T17:43:05.043910+0000","172.21.15.86:0/883052430":"2024-03-24T17:50:38.377487+0000","172.21.15.86:0/1614943062":"2024-03-24T17:43:28.242492+0000","172.21.15.86:6801/2313723492":"2024-03-24T17:43:05.043910+0000","172.21.15.86:0/2416321716":"2024-03-24T17:43:05.043910+0000","172.21.15.86:0/2938296862":"2024-03-24T17:50:38.377487+0000","172.21.15.86:0/433588188":"2024-03-24T17:43:05.043910+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":18,"length":1},{"begin":30,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T18:08:45.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.340+0000 7f4551000700 1 -- 172.21.15.86:0/932102536 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f45300a1310 msgr2=0x7f45300a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:45.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.340+0000 7f4551000700 1 --2- 172.21.15.86:0/932102536 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f45300a1310 0x7f45300a37d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f453c011490 tx=0x7f453c009260 comp rx=0 tx=0).stop 2024-03-23T18:08:45.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.340+0000 7f4551000700 1 -- 172.21.15.86:0/932102536 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f454c073aa0 msgr2=0x7f454c199980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:45.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.340+0000 7f4551000700 1 --2- 172.21.15.86:0/932102536 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f454c073aa0 0x7f454c199980 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f453800e9a0 tx=0x7f453800ecb0 comp rx=0 tx=0).stop 2024-03-23T18:08:45.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.340+0000 7f4551000700 1 -- 172.21.15.86:0/932102536 shutdown_connections 2024-03-23T18:08:45.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.340+0000 7f4551000700 1 --2- 172.21.15.86:0/932102536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f454c10d490 0x7f454c19dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:45.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.340+0000 7f4551000700 1 --2- 172.21.15.86:0/932102536 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f45300a1310 0x7f45300a37d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:45.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.340+0000 7f4551000700 1 --2- 172.21.15.86:0/932102536 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f454c073180 0x7f454c199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:45.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.340+0000 7f4551000700 1 --2- 172.21.15.86:0/932102536 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f454c073aa0 0x7f454c199980 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:45.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.340+0000 7f4551000700 1 -- 172.21.15.86:0/932102536 >> 172.21.15.86:0/932102536 conn(0x7f454c0fc760 msgr2=0x7f454c0fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:08:45.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.340+0000 7f4551000700 1 -- 172.21.15.86:0/932102536 shutdown_connections 2024-03-23T18:08:45.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.340+0000 7f4551000700 1 -- 172.21.15.86:0/932102536 wait complete. 2024-03-23T18:08:45.361 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-23T18:08:45.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.554+0000 7f8f6f200700 1 -- 172.21.15.86:0/1138980129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f68101830 msgr2=0x7f8f68105870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:45.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.554+0000 7f8f6f200700 1 --2- 172.21.15.86:0/1138980129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f68101830 0x7f8f68105870 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f8f54009990 tx=0x7f8f54009ca0 comp rx=0 tx=0).stop 2024-03-23T18:08:45.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.554+0000 7f8f6f200700 1 -- 172.21.15.86:0/1138980129 shutdown_connections 2024-03-23T18:08:45.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.554+0000 7f8f6f200700 1 --2- 172.21.15.86:0/1138980129 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f68105fa0 0x7f8f6810db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:45.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.554+0000 7f8f6f200700 1 --2- 172.21.15.86:0/1138980129 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f68100e80 0x7f8f68101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:45.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.554+0000 7f8f6f200700 1 --2- 172.21.15.86:0/1138980129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f68101830 0x7f8f68105870 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:45.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.554+0000 7f8f6f200700 1 -- 172.21.15.86:0/1138980129 >> 172.21.15.86:0/1138980129 conn(0x7f8f680fc720 msgr2=0x7f8f680feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:08:45.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.554+0000 7f8f6f200700 1 -- 172.21.15.86:0/1138980129 shutdown_connections 2024-03-23T18:08:45.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.554+0000 7f8f6f200700 1 -- 172.21.15.86:0/1138980129 wait complete. 2024-03-23T18:08:45.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:45 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/2204171608' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T18:08:45.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:45 smithi092 ceph-mon[79089]: pgmap v573: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 189 KiB/s wr, 27 op/s 2024-03-23T18:08:45.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:45 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/1400005718' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T18:08:45.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.555+0000 7f8f6f200700 1 Processor -- start 2024-03-23T18:08:45.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.555+0000 7f8f6f200700 1 -- start start 2024-03-23T18:08:45.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.555+0000 7f8f6f200700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f68100e80 0x7f8f680730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:45.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.555+0000 7f8f6f200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f68105fa0 0x7f8f680735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:45.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.555+0000 7f8f6f200700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f68073b30 0x7f8f681a5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:45.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.555+0000 7f8f6f200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f681a5910 con 0x7f8f68105fa0 2024-03-23T18:08:45.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.555+0000 7f8f6f200700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f681a5a80 con 0x7f8f68073b30 2024-03-23T18:08:45.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.555+0000 7f8f6f200700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f681a5c40 con 0x7f8f68100e80 2024-03-23T18:08:45.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.555+0000 7f8f67400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f68105fa0 0x7f8f680735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:45.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.556+0000 7f8f67400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f68105fa0 0x7f8f680735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55804/0 (socket says 172.21.15.86:55804) 2024-03-23T18:08:45.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.556+0000 7f8f67400700 1 -- 172.21.15.86:0/1746234015 learned_addr learned my addr 172.21.15.86:0/1746234015 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:08:45.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.556+0000 7f8f67e00700 1 --2- 172.21.15.86:0/1746234015 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f68100e80 0x7f8f680730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:45.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.556+0000 7f8f6d200700 1 --2- 172.21.15.86:0/1746234015 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f68073b30 0x7f8f681a5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:45.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.556+0000 7f8f67400700 1 -- 172.21.15.86:0/1746234015 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f68100e80 msgr2=0x7f8f680730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:45.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.556+0000 7f8f67400700 1 --2- 172.21.15.86:0/1746234015 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f68100e80 0x7f8f680730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:45.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.556+0000 7f8f67400700 1 -- 172.21.15.86:0/1746234015 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f68073b30 msgr2=0x7f8f681a5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:45.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.556+0000 7f8f67400700 1 --2- 172.21.15.86:0/1746234015 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f68073b30 0x7f8f681a5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:45.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.556+0000 7f8f67400700 1 -- 172.21.15.86:0/1746234015 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f540095f0 con 0x7f8f68105fa0 2024-03-23T18:08:45.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.556+0000 7f8f67400700 1 --2- 172.21.15.86:0/1746234015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f68105fa0 0x7f8f680735f0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f8f54000c00 tx=0x7f8f54004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:08:45.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.556+0000 7f8f64c00700 1 -- 172.21.15.86:0/1746234015 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f5401d070 con 0x7f8f68105fa0 2024-03-23T18:08:45.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.556+0000 7f8f64c00700 1 -- 172.21.15.86:0/1746234015 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f54021410 con 0x7f8f68105fa0 2024-03-23T18:08:45.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.556+0000 7f8f6f200700 1 -- 172.21.15.86:0/1746234015 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f681a5ec0 con 0x7f8f68105fa0 2024-03-23T18:08:45.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.556+0000 7f8f64c00700 1 -- 172.21.15.86:0/1746234015 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f54017650 con 0x7f8f68105fa0 2024-03-23T18:08:45.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.557+0000 7f8f6f200700 1 -- 172.21.15.86:0/1746234015 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f681a6280 con 0x7f8f68105fa0 2024-03-23T18:08:45.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.557+0000 7f8f6f200700 1 -- 172.21.15.86:0/1746234015 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f68109270 con 0x7f8f68105fa0 2024-03-23T18:08:45.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.559+0000 7f8f64c00700 1 -- 172.21.15.86:0/1746234015 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f8f54021580 con 0x7f8f68105fa0 2024-03-23T18:08:45.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.561+0000 7f8f64c00700 1 --2- 172.21.15.86:0/1746234015 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f8f4c0a1280 0x7f8f4c0a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:45.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.561+0000 7f8f64c00700 1 -- 172.21.15.86:0/1746234015 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 8484+0+0 (secure 0 0 0) 0x7f8f540cde90 con 0x7f8f68105fa0 2024-03-23T18:08:45.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.561+0000 7f8f67e00700 1 --2- 172.21.15.86:0/1746234015 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f8f4c0a1280 0x7f8f4c0a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:45.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.561+0000 7f8f67e00700 1 --2- 172.21.15.86:0/1746234015 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f8f4c0a1280 0x7f8f4c0a3740 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8f58009c60 tx=0x7f8f58009260 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:08:45.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.564+0000 7f8f64c00700 1 -- 172.21.15.86:0/1746234015 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f8f54097330 con 0x7f8f68105fa0 2024-03-23T18:08:45.585 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:45 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/2204171608' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T18:08:45.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:45 smithi152 ceph-mon[47949]: pgmap v573: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 189 KiB/s wr, 27 op/s 2024-03-23T18:08:45.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:45 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/1400005718' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T18:08:45.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.820+0000 7f8f6f200700 1 -- 172.21.15.86:0/1746234015 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f8f68074590 con 0x7f8f68105fa0 2024-03-23T18:08:45.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.831+0000 7f8f64c00700 1 -- 172.21.15.86:0/1746234015 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v91) v1 ==== 161+0+0 (secure 0 0 0) 0x7f8f5409a950 con 0x7f8f68105fa0 2024-03-23T18:08:45.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.834+0000 7f8f6f200700 1 -- 172.21.15.86:0/1746234015 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f8f4c0a1280 msgr2=0x7f8f4c0a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:45.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.834+0000 7f8f6f200700 1 --2- 172.21.15.86:0/1746234015 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f8f4c0a1280 0x7f8f4c0a3740 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8f58009c60 tx=0x7f8f58009260 comp rx=0 tx=0).stop 2024-03-23T18:08:45.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.834+0000 7f8f6f200700 1 -- 172.21.15.86:0/1746234015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f68105fa0 msgr2=0x7f8f680735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:45.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.834+0000 7f8f6f200700 1 --2- 172.21.15.86:0/1746234015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f68105fa0 0x7f8f680735f0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f8f54000c00 tx=0x7f8f54004730 comp rx=0 tx=0).stop 2024-03-23T18:08:45.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.834+0000 7f8f6f200700 1 -- 172.21.15.86:0/1746234015 shutdown_connections 2024-03-23T18:08:45.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.834+0000 7f8f6f200700 1 --2- 172.21.15.86:0/1746234015 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f68073b30 0x7f8f681a5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:45.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.834+0000 7f8f6f200700 1 --2- 172.21.15.86:0/1746234015 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f8f4c0a1280 0x7f8f4c0a3740 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:45.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.834+0000 7f8f6f200700 1 --2- 172.21.15.86:0/1746234015 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f68100e80 0x7f8f680730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:45.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.834+0000 7f8f6f200700 1 --2- 172.21.15.86:0/1746234015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f68105fa0 0x7f8f680735f0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:45.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.834+0000 7f8f6f200700 1 -- 172.21.15.86:0/1746234015 >> 172.21.15.86:0/1746234015 conn(0x7f8f680fc720 msgr2=0x7f8f680fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:08:45.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.834+0000 7f8f6f200700 1 -- 172.21.15.86:0/1746234015 shutdown_connections 2024-03-23T18:08:45.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:45.834+0000 7f8f6f200700 1 -- 172.21.15.86:0/1746234015 wait complete. 2024-03-23T18:08:45.837 INFO:teuthology.orchestra.run.smithi086.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-23T18:08:45.857 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-23T18:08:46.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:46.061+0000 7f9024e00700 1 -- 172.21.15.86:0/3068553044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9020073b00 msgr2=0x7f902010cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:46.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:46.061+0000 7f9024e00700 1 --2- 172.21.15.86:0/3068553044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9020073b00 0x7f902010cff0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f9008009990 tx=0x7f9008009ca0 comp rx=0 tx=0).stop 2024-03-23T18:08:46.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:46.061+0000 7f9024e00700 1 -- 172.21.15.86:0/3068553044 shutdown_connections 2024-03-23T18:08:46.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:46.061+0000 7f9024e00700 1 --2- 172.21.15.86:0/3068553044 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f902010d530 0x7f902010f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:46.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:46.061+0000 7f9024e00700 1 --2- 172.21.15.86:0/3068553044 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f90200731e0 0x7f90200735c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:46.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:46.061+0000 7f9024e00700 1 --2- 172.21.15.86:0/3068553044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9020073b00 0x7f902010cff0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:46.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:46.061+0000 7f9024e00700 1 -- 172.21.15.86:0/3068553044 >> 172.21.15.86:0/3068553044 conn(0x7f90200fc7c0 msgr2=0x7f90200febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:08:46.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:46.061+0000 7f9024e00700 1 -- 172.21.15.86:0/3068553044 shutdown_connections 2024-03-23T18:08:46.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:46.061+0000 7f9024e00700 1 -- 172.21.15.86:0/3068553044 wait complete. 2024-03-23T18:08:46.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:46.062+0000 7f9024e00700 1 Processor -- start 2024-03-23T18:08:46.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:46.062+0000 7f9024e00700 1 -- start start 2024-03-23T18:08:46.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:46.062+0000 7f9024e00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f90200731e0 0x7f9020199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:46.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:46.062+0000 7f9024e00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9020073b00 0x7f9020199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:46.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:46.062+0000 7f9024e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f902010d530 0x7f902019dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:46.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:46.062+0000 7f9024e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f902019e360 con 0x7f902010d530 2024-03-23T18:08:46.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:46.062+0000 7f9024e00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f902019e4d0 con 0x7f9020073b00 2024-03-23T18:08:46.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:46.062+0000 7f9024e00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f902019e690 con 0x7f90200731e0 2024-03-23T18:08:46.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:46.063+0000 7f901e800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f902010d530 0x7f902019dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:46.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:46.063+0000 7f901e800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f902010d530 0x7f902019dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55866/0 (socket says 172.21.15.86:55866) 2024-03-23T18:08:46.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:46.063+0000 7f901e800700 1 -- 172.21.15.86:0/2469799771 learned_addr learned my addr 172.21.15.86:0/2469799771 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:08:46.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:46.063+0000 7f9017e00700 1 --2- 172.21.15.86:0/2469799771 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9020073b00 0x7f9020199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:46.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:46.063+0000 7f901de00700 1 --2- 172.21.15.86:0/2469799771 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f90200731e0 0x7f9020199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:46.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:46.063+0000 7f901e800700 1 -- 172.21.15.86:0/2469799771 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f90200731e0 msgr2=0x7f9020199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:46.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:46.063+0000 7f901e800700 1 --2- 172.21.15.86:0/2469799771 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f90200731e0 0x7f9020199440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:46.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:46.063+0000 7f901e800700 1 -- 172.21.15.86:0/2469799771 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9020073b00 msgr2=0x7f9020199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:46.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:46.063+0000 7f901e800700 1 --2- 172.21.15.86:0/2469799771 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9020073b00 0x7f9020199980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:46.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:46.063+0000 7f901e800700 1 -- 172.21.15.86:0/2469799771 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9010009520 con 0x7f902010d530 2024-03-23T18:08:46.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:46.063+0000 7f901e800700 1 --2- 172.21.15.86:0/2469799771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f902010d530 0x7f902019dd20 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f901800ef30 tx=0x7f901800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:08:46.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:46.063+0000 7f9016a00700 1 -- 172.21.15.86:0/2469799771 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9018018070 con 0x7f902010d530 2024-03-23T18:08:46.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:46.064+0000 7f9016a00700 1 -- 172.21.15.86:0/2469799771 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f901800f040 con 0x7f902010d530 2024-03-23T18:08:46.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:46.064+0000 7f9024e00700 1 -- 172.21.15.86:0/2469799771 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90080095f0 con 0x7f902010d530 2024-03-23T18:08:46.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:46.064+0000 7f9016a00700 1 -- 172.21.15.86:0/2469799771 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f901800cd00 con 0x7f902010d530 2024-03-23T18:08:46.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:46.064+0000 7f9024e00700 1 -- 172.21.15.86:0/2469799771 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90201a5fd0 con 0x7f902010d530 2024-03-23T18:08:46.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:46.065+0000 7f9024e00700 1 -- 172.21.15.86:0/2469799771 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9020108740 con 0x7f902010d530 2024-03-23T18:08:46.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:46.066+0000 7f9016a00700 1 -- 172.21.15.86:0/2469799771 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f9018003a00 con 0x7f902010d530 2024-03-23T18:08:46.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:46.068+0000 7f9016a00700 1 --2- 172.21.15.86:0/2469799771 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f90040a1290 0x7f90040a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:46.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:46.068+0000 7f9016a00700 1 -- 172.21.15.86:0/2469799771 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 8870+0+0 (secure 0 0 0) 0x7f90180cc950 con 0x7f902010d530 2024-03-23T18:08:46.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:46.068+0000 7f901de00700 1 --2- 172.21.15.86:0/2469799771 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f90040a1290 0x7f90040a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:46.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:46.068+0000 7f901de00700 1 --2- 172.21.15.86:0/2469799771 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f90040a1290 0x7f90040a3750 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9010009df0 tx=0x7f9010009260 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:08:46.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:46.072+0000 7f9016a00700 1 -- 172.21.15.86:0/2469799771 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f9018095c70 con 0x7f902010d530 2024-03-23T18:08:46.092 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:45 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mon-a[38198]: 2024-03-23T18:08:45.821+0000 7f70a3000700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-23T18:08:46.092 INFO:journalctl@ceph.mds.a.smithi086.stdout:Mar 23 18:08:45 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-a[90474]: ignoring --setuser ceph since I am not root 2024-03-23T18:08:46.093 INFO:journalctl@ceph.mds.a.smithi086.stdout:Mar 23 18:08:45 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-a[90474]: ignoring --setgroup ceph since I am not root 2024-03-23T18:08:46.093 INFO:journalctl@ceph.mds.a.smithi086.stdout:Mar 23 18:08:45 smithi086 ceph-mds[90497]: -- 172.21.15.86:0/1203336434 <== mon.2 v2:172.21.15.92:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5626026cc1a0 con 0x5626026a4c00 2024-03-23T18:08:46.093 INFO:journalctl@ceph.mds.a.smithi086.stdout:Mar 23 18:08:45 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-a[90474]: starting mds.a at 2024-03-23T18:08:46.093 INFO:journalctl@ceph.mds.d.smithi086.stdout:Mar 23 18:08:45 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-d[89510]: ignoring --setuser ceph since I am not root 2024-03-23T18:08:46.093 INFO:journalctl@ceph.mds.d.smithi086.stdout:Mar 23 18:08:45 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-d[89510]: ignoring --setgroup ceph since I am not root 2024-03-23T18:08:46.094 INFO:journalctl@ceph.mds.d.smithi086.stdout:Mar 23 18:08:45 smithi086 ceph-mds[89533]: -- 172.21.15.86:0/921893012 <== mon.2 v2:172.21.15.92:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556548b801a0 con 0x556548b58800 2024-03-23T18:08:46.094 INFO:journalctl@ceph.mds.d.smithi086.stdout:Mar 23 18:08:45 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-d[89510]: starting mds.d at 2024-03-23T18:08:46.307 INFO:journalctl@ceph.mds.h.smithi092.stdout:Mar 23 18:08:45 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-h[114780]: ignoring --setuser ceph since I am not root 2024-03-23T18:08:46.307 INFO:journalctl@ceph.mds.h.smithi092.stdout:Mar 23 18:08:45 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-h[114780]: ignoring --setgroup ceph since I am not root 2024-03-23T18:08:46.307 INFO:journalctl@ceph.mds.h.smithi092.stdout:Mar 23 18:08:45 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-h[114780]: starting mds.h at 2024-03-23T18:08:46.308 INFO:journalctl@ceph.mds.b.smithi092.stdout:Mar 23 18:08:45 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-b[115122]: ignoring --setuser ceph since I am not root 2024-03-23T18:08:46.308 INFO:journalctl@ceph.mds.b.smithi092.stdout:Mar 23 18:08:45 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-b[115122]: ignoring --setgroup ceph since I am not root 2024-03-23T18:08:46.308 INFO:journalctl@ceph.mds.b.smithi092.stdout:Mar 23 18:08:45 smithi092 ceph-mds[115145]: -- 172.21.15.92:0/3715527452 <== mon.1 v2:172.21.15.152:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ab59c381a0 con 0x55ab59c0e800 2024-03-23T18:08:46.308 INFO:journalctl@ceph.mds.b.smithi092.stdout:Mar 23 18:08:45 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-b[115122]: starting mds.b at 2024-03-23T18:08:46.309 INFO:journalctl@ceph.mds.e.smithi092.stdout:Mar 23 18:08:45 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-e[114438]: ignoring --setuser ceph since I am not root 2024-03-23T18:08:46.309 INFO:journalctl@ceph.mds.e.smithi092.stdout:Mar 23 18:08:45 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-e[114438]: ignoring --setgroup ceph since I am not root 2024-03-23T18:08:46.309 INFO:journalctl@ceph.mds.e.smithi092.stdout:Mar 23 18:08:45 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-e[114438]: starting mds.e at 2024-03-23T18:08:46.310 INFO:journalctl@ceph.mds.k.smithi092.stdout:Mar 23 18:08:45 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-k[114091]: ignoring --setuser ceph since I am not root 2024-03-23T18:08:46.310 INFO:journalctl@ceph.mds.k.smithi092.stdout:Mar 23 18:08:45 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-k[114091]: ignoring --setgroup ceph since I am not root 2024-03-23T18:08:46.310 INFO:journalctl@ceph.mds.k.smithi092.stdout:Mar 23 18:08:45 smithi092 ceph-mds[114114]: -- 172.21.15.92:0/2405566526 <== mon.2 v2:172.21.15.92:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f7be3841a0 con 0x55f7be35b000 2024-03-23T18:08:46.310 INFO:journalctl@ceph.mds.k.smithi092.stdout:Mar 23 18:08:45 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-k[114091]: starting mds.k at 2024-03-23T18:08:46.311 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:46 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/932102536' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T18:08:46.311 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:46 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/1746234015' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-23T18:08:46.311 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:46 smithi092 ceph-mon[79089]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-23T18:08:46.311 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:46 smithi092 ceph-mon[79089]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-23T18:08:46.311 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:46 smithi092 ceph-mon[79089]: Health check cleared: MDS_DAMAGE (was: 3 MDSs report damaged metadata) 2024-03-23T18:08:46.311 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:46 smithi092 ceph-mon[79089]: osdmap e111: 12 total, 12 up, 12 in 2024-03-23T18:08:46.311 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:46 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/1746234015' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-23T18:08:46.311 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:46 smithi092 ceph-mon[79089]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-03-23T18:08:46.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:46.327+0000 7f9024e00700 1 -- 172.21.15.86:0/2469799771 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f902019a740 con 0x7f902010d530 2024-03-23T18:08:46.336 INFO:journalctl@ceph.mds.c.smithi152.stdout:Mar 23 18:08:45 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-c[84552]: ignoring --setuser ceph since I am not root 2024-03-23T18:08:46.336 INFO:journalctl@ceph.mds.c.smithi152.stdout:Mar 23 18:08:45 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-c[84552]: ignoring --setgroup ceph since I am not root 2024-03-23T18:08:46.336 INFO:journalctl@ceph.mds.c.smithi152.stdout:Mar 23 18:08:45 smithi152 ceph-mds[84575]: -- 172.21.15.152:0/1796555454 <== mon.2 v2:172.21.15.92:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a7db6f81a0 con 0x55a7db6cf000 2024-03-23T18:08:46.336 INFO:journalctl@ceph.mds.c.smithi152.stdout:Mar 23 18:08:45 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-c[84552]: starting mds.c at 2024-03-23T18:08:46.337 INFO:journalctl@ceph.mds.i.smithi152.stdout:Mar 23 18:08:45 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-i[84894]: ignoring --setuser ceph since I am not root 2024-03-23T18:08:46.337 INFO:journalctl@ceph.mds.i.smithi152.stdout:Mar 23 18:08:45 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-i[84894]: ignoring --setgroup ceph since I am not root 2024-03-23T18:08:46.337 INFO:journalctl@ceph.mds.i.smithi152.stdout:Mar 23 18:08:45 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-i[84894]: starting mds.i at 2024-03-23T18:08:46.338 INFO:journalctl@ceph.mds.f.smithi152.stdout:Mar 23 18:08:45 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-f[84209]: ignoring --setuser ceph since I am not root 2024-03-23T18:08:46.338 INFO:journalctl@ceph.mds.f.smithi152.stdout:Mar 23 18:08:45 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-f[84209]: ignoring --setgroup ceph since I am not root 2024-03-23T18:08:46.338 INFO:journalctl@ceph.mds.f.smithi152.stdout:Mar 23 18:08:45 smithi152 ceph-mds[84232]: -- 172.21.15.152:0/2452919276 <== mon.2 v2:172.21.15.92:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561edd35e1a0 con 0x561edd334c00 2024-03-23T18:08:46.338 INFO:journalctl@ceph.mds.f.smithi152.stdout:Mar 23 18:08:45 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-f[84209]: starting mds.f at 2024-03-23T18:08:46.339 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:46 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/932102536' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T18:08:46.339 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:46 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/1746234015' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-23T18:08:46.339 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:46 smithi152 ceph-mon[47949]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-23T18:08:46.339 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:46 smithi152 ceph-mon[47949]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-23T18:08:46.339 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:46 smithi152 ceph-mon[47949]: Health check cleared: MDS_DAMAGE (was: 3 MDSs report damaged metadata) 2024-03-23T18:08:46.339 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:46 smithi152 ceph-mon[47949]: osdmap e111: 12 total, 12 up, 12 in 2024-03-23T18:08:46.339 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:46 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/1746234015' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-23T18:08:46.340 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:46 smithi152 ceph-mon[47949]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-03-23T18:08:46.340 INFO:journalctl@ceph.mds.l.smithi152.stdout:Mar 23 18:08:45 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-l[83861]: ignoring --setuser ceph since I am not root 2024-03-23T18:08:46.340 INFO:journalctl@ceph.mds.l.smithi152.stdout:Mar 23 18:08:45 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-l[83861]: ignoring --setgroup ceph since I am not root 2024-03-23T18:08:46.340 INFO:journalctl@ceph.mds.l.smithi152.stdout:Mar 23 18:08:45 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-l[83861]: starting mds.l at 2024-03-23T18:08:46.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:46 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/932102536' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T18:08:46.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:46 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/1746234015' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-23T18:08:46.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:46 smithi086 ceph-mon[38221]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-23T18:08:46.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:46 smithi086 ceph-mon[38221]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-23T18:08:46.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:46 smithi086 ceph-mon[38221]: Health check cleared: MDS_DAMAGE (was: 3 MDSs report damaged metadata) 2024-03-23T18:08:46.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:46 smithi086 ceph-mon[38221]: osdmap e111: 12 total, 12 up, 12 in 2024-03-23T18:08:46.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:46 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/1746234015' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-23T18:08:46.517 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:46 smithi086 ceph-mon[38221]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-03-23T18:08:46.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:46.833+0000 7f9016a00700 1 -- 172.21.15.86:0/2469799771 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v92) v1 ==== 104+0+0 (secure 0 0 0) 0x7f9018099290 con 0x7f902010d530 2024-03-23T18:08:46.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:46.837+0000 7f9024e00700 1 -- 172.21.15.86:0/2469799771 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f90040a1290 msgr2=0x7f90040a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:46.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:46.837+0000 7f9024e00700 1 --2- 172.21.15.86:0/2469799771 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f90040a1290 0x7f90040a3750 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9010009df0 tx=0x7f9010009260 comp rx=0 tx=0).stop 2024-03-23T18:08:46.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:46.837+0000 7f9024e00700 1 -- 172.21.15.86:0/2469799771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f902010d530 msgr2=0x7f902019dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:46.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:46.837+0000 7f9024e00700 1 --2- 172.21.15.86:0/2469799771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f902010d530 0x7f902019dd20 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f901800ef30 tx=0x7f901800c3a0 comp rx=0 tx=0).stop 2024-03-23T18:08:46.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:46.837+0000 7f9024e00700 1 -- 172.21.15.86:0/2469799771 shutdown_connections 2024-03-23T18:08:46.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:46.837+0000 7f9024e00700 1 --2- 172.21.15.86:0/2469799771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f902010d530 0x7f902019dd20 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:46.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:46.837+0000 7f9024e00700 1 --2- 172.21.15.86:0/2469799771 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7f90040a1290 0x7f90040a3750 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:46.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:46.837+0000 7f9024e00700 1 --2- 172.21.15.86:0/2469799771 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f90200731e0 0x7f9020199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:46.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:46.837+0000 7f9024e00700 1 --2- 172.21.15.86:0/2469799771 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9020073b00 0x7f9020199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:46.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:46.837+0000 7f9024e00700 1 -- 172.21.15.86:0/2469799771 >> 172.21.15.86:0/2469799771 conn(0x7f90200fc7c0 msgr2=0x7f90200fe0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:08:46.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:46.837+0000 7f9024e00700 1 -- 172.21.15.86:0/2469799771 shutdown_connections 2024-03-23T18:08:46.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:46.838+0000 7f9024e00700 1 -- 172.21.15.86:0/2469799771 wait complete. 2024-03-23T18:08:46.859 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-23T18:08:47.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:47.055+0000 7fb7da600700 1 -- 172.21.15.86:0/3109229825 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb7d4106080 msgr2=0x7fb7d410dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:47.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:47.055+0000 7fb7da600700 1 --2- 172.21.15.86:0/3109229825 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb7d4106080 0x7fb7d410dc00 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc00b0a0 tx=0x7fb7cc00b3b0 comp rx=0 tx=0).stop 2024-03-23T18:08:47.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:47.055+0000 7fb7da600700 1 -- 172.21.15.86:0/3109229825 shutdown_connections 2024-03-23T18:08:47.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:47.055+0000 7fb7da600700 1 --2- 172.21.15.86:0/3109229825 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb7d4106080 0x7fb7d410dc00 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:47.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:47.055+0000 7fb7da600700 1 --2- 172.21.15.86:0/3109229825 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7d4100f60 0x7fb7d4101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:47.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:47.055+0000 7fb7da600700 1 --2- 172.21.15.86:0/3109229825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7d4101910 0x7fb7d4105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:47.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:47.055+0000 7fb7da600700 1 -- 172.21.15.86:0/3109229825 >> 172.21.15.86:0/3109229825 conn(0x7fb7d40fc7c0 msgr2=0x7fb7d40febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:08:47.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:47.055+0000 7fb7da600700 1 -- 172.21.15.86:0/3109229825 shutdown_connections 2024-03-23T18:08:47.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:47.056+0000 7fb7da600700 1 -- 172.21.15.86:0/3109229825 wait complete. 2024-03-23T18:08:47.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:47.057+0000 7fb7da600700 1 Processor -- start 2024-03-23T18:08:47.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:47.057+0000 7fb7da600700 1 -- start start 2024-03-23T18:08:47.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:47.057+0000 7fb7da600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7d4100f60 0x7fb7d4197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:47.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:47.057+0000 7fb7da600700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7d4106080 0x7fb7d4197890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:47.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:47.057+0000 7fb7da600700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb7d419bc30 0x7fb7d419c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:47.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:47.057+0000 7fb7da600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d419c6b0 con 0x7fb7d4100f60 2024-03-23T18:08:47.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:47.057+0000 7fb7da600700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d419c820 con 0x7fb7d4106080 2024-03-23T18:08:47.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:47.057+0000 7fb7da600700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d419c9e0 con 0x7fb7d419bc30 2024-03-23T18:08:47.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:47.057+0000 7fb7d3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7d4100f60 0x7fb7d4197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:47.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:47.057+0000 7fb7cbe00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7d4106080 0x7fb7d4197890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:47.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:47.057+0000 7fb7d3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7d4100f60 0x7fb7d4197350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55886/0 (socket says 172.21.15.86:55886) 2024-03-23T18:08:47.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:47.057+0000 7fb7d3400700 1 -- 172.21.15.86:0/780507119 learned_addr learned my addr 172.21.15.86:0/780507119 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:08:47.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:47.058+0000 7fb7d3e00700 1 --2- 172.21.15.86:0/780507119 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb7d419bc30 0x7fb7d419c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:47.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:47.058+0000 7fb7d3400700 1 -- 172.21.15.86:0/780507119 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb7d419bc30 msgr2=0x7fb7d419c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:47.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:47.058+0000 7fb7d3400700 1 --2- 172.21.15.86:0/780507119 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb7d419bc30 0x7fb7d419c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:47.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:47.058+0000 7fb7d3400700 1 -- 172.21.15.86:0/780507119 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7d4106080 msgr2=0x7fb7d4197890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:47.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:47.058+0000 7fb7d3400700 1 --2- 172.21.15.86:0/780507119 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7d4106080 0x7fb7d4197890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:47.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:47.058+0000 7fb7d3400700 1 -- 172.21.15.86:0/780507119 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7cc009d00 con 0x7fb7d4100f60 2024-03-23T18:08:47.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:47.058+0000 7fb7cbe00700 1 --2- 172.21.15.86:0/780507119 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7d4106080 0x7fb7d4197890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T18:08:47.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:47.058+0000 7fb7d3400700 1 --2- 172.21.15.86:0/780507119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7d4100f60 0x7fb7d4197350 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fb7c400e9a0 tx=0x7fb7c400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:08:47.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:47.058+0000 7fb7d0c00700 1 -- 172.21.15.86:0/780507119 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c4009790 con 0x7fb7d4100f60 2024-03-23T18:08:47.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:47.058+0000 7fb7d0c00700 1 -- 172.21.15.86:0/780507119 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb7c4010430 con 0x7fb7d4100f60 2024-03-23T18:08:47.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:47.058+0000 7fb7da600700 1 -- 172.21.15.86:0/780507119 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7d41a3f40 con 0x7fb7d4100f60 2024-03-23T18:08:47.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:47.058+0000 7fb7d0c00700 1 -- 172.21.15.86:0/780507119 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c4018680 con 0x7fb7d4100f60 2024-03-23T18:08:47.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:47.058+0000 7fb7da600700 1 -- 172.21.15.86:0/780507119 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7d41a4410 con 0x7fb7d4100f60 2024-03-23T18:08:47.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:47.060+0000 7fb7da600700 1 -- 172.21.15.86:0/780507119 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7d4109350 con 0x7fb7d4100f60 2024-03-23T18:08:47.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:47.061+0000 7fb7d0c00700 1 -- 172.21.15.86:0/780507119 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fb7c40105a0 con 0x7fb7d4100f60 2024-03-23T18:08:47.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:47.063+0000 7fb7d0c00700 1 --2- 172.21.15.86:0/780507119 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fb7b80a1310 0x7fb7b80a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:47.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:47.063+0000 7fb7d0c00700 1 -- 172.21.15.86:0/780507119 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 8870+0+0 (secure 0 0 0) 0x7fb7c4014070 con 0x7fb7d4100f60 2024-03-23T18:08:47.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:47.063+0000 7fb7cbe00700 1 --2- 172.21.15.86:0/780507119 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fb7b80a1310 0x7fb7b80a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:47.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:47.063+0000 7fb7cbe00700 1 --2- 172.21.15.86:0/780507119 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fb7b80a1310 0x7fb7b80a37d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb7d41988a0 tx=0x7fb7bc009260 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:08:47.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:47.067+0000 7fb7d0c00700 1 -- 172.21.15.86:0/780507119 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fb7c40529f0 con 0x7fb7d4100f60 2024-03-23T18:08:47.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:47 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/2469799771' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-23T18:08:47.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:47 smithi086 ceph-mon[38221]: pgmap v575: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 192 KiB/s wr, 18 op/s 2024-03-23T18:08:47.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:47 smithi086 ceph-mon[38221]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-23T18:08:47.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:47 smithi086 ceph-mon[38221]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-23T18:08:47.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:47 smithi086 ceph-mon[38221]: Cluster is now healthy 2024-03-23T18:08:47.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:47 smithi086 ceph-mon[38221]: mds.? [v2:172.21.15.152:6834/3374019754,v1:172.21.15.152:6835/3374019754] up:boot 2024-03-23T18:08:47.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:47 smithi086 ceph-mon[38221]: mds.? [v2:172.21.15.92:6833/2633104629,v1:172.21.15.92:6835/2633104629] up:boot 2024-03-23T18:08:47.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:47 smithi086 ceph-mon[38221]: mds.? [v2:172.21.15.152:6837/1350317937,v1:172.21.15.152:6839/1350317937] up:boot 2024-03-23T18:08:47.268 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:47 smithi086 ceph-mon[38221]: mds.? [v2:172.21.15.152:6836/2909473996,v1:172.21.15.152:6838/2909473996] up:boot 2024-03-23T18:08:47.268 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:47 smithi086 ceph-mon[38221]: mds.? [v2:172.21.15.152:6832/4056042649,v1:172.21.15.152:6833/4056042649] up:boot 2024-03-23T18:08:47.268 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:47 smithi086 ceph-mon[38221]: mds.? [v2:172.21.15.92:6836/301800424,v1:172.21.15.92:6837/301800424] up:boot 2024-03-23T18:08:47.268 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:47 smithi086 ceph-mon[38221]: mds.? [v2:172.21.15.92:6838/3622725597,v1:172.21.15.92:6839/3622725597] up:boot 2024-03-23T18:08:47.268 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:47 smithi086 ceph-mon[38221]: mds.? [v2:172.21.15.92:6832/3039977494,v1:172.21.15.92:6834/3039977494] up:boot 2024-03-23T18:08:47.268 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:47 smithi086 ceph-mon[38221]: mds.? [v2:172.21.15.86:6836/1156936309,v1:172.21.15.86:6840/1156936309] up:boot 2024-03-23T18:08:47.268 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:47 smithi086 ceph-mon[38221]: mds.? [v2:172.21.15.86:6837/833138623,v1:172.21.15.86:6841/833138623] up:boot 2024-03-23T18:08:47.269 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:47 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/2469799771' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-23T18:08:47.269 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:47 smithi086 ceph-mon[38221]: fsmap 12 up:standby 2024-03-23T18:08:47.269 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:47 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-23T18:08:47.269 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:47 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-23T18:08:47.269 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:47 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-23T18:08:47.269 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:47 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-23T18:08:47.269 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:47 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-23T18:08:47.269 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:47 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-23T18:08:47.269 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:47 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-23T18:08:47.270 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:47 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-23T18:08:47.270 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:47 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-23T18:08:47.270 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:47 smithi086 ceph-mon[38221]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-23T18:08:47.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:47.277+0000 7fb7da600700 1 -- 172.21.15.86:0/780507119 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb7d4102430 con 0x7fb7d4100f60 2024-03-23T18:08:47.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:47 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/2469799771' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-23T18:08:47.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:47 smithi092 ceph-mon[79089]: pgmap v575: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 192 KiB/s wr, 18 op/s 2024-03-23T18:08:47.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:47 smithi092 ceph-mon[79089]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-23T18:08:47.558 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:47 smithi092 ceph-mon[79089]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-23T18:08:47.558 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:47 smithi092 ceph-mon[79089]: Cluster is now healthy 2024-03-23T18:08:47.558 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:47 smithi092 ceph-mon[79089]: mds.? [v2:172.21.15.152:6834/3374019754,v1:172.21.15.152:6835/3374019754] up:boot 2024-03-23T18:08:47.558 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:47 smithi092 ceph-mon[79089]: mds.? [v2:172.21.15.92:6833/2633104629,v1:172.21.15.92:6835/2633104629] up:boot 2024-03-23T18:08:47.558 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:47 smithi092 ceph-mon[79089]: mds.? [v2:172.21.15.152:6837/1350317937,v1:172.21.15.152:6839/1350317937] up:boot 2024-03-23T18:08:47.558 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:47 smithi092 ceph-mon[79089]: mds.? [v2:172.21.15.152:6836/2909473996,v1:172.21.15.152:6838/2909473996] up:boot 2024-03-23T18:08:47.558 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:47 smithi092 ceph-mon[79089]: mds.? [v2:172.21.15.152:6832/4056042649,v1:172.21.15.152:6833/4056042649] up:boot 2024-03-23T18:08:47.559 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:47 smithi092 ceph-mon[79089]: mds.? [v2:172.21.15.92:6836/301800424,v1:172.21.15.92:6837/301800424] up:boot 2024-03-23T18:08:47.559 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:47 smithi092 ceph-mon[79089]: mds.? [v2:172.21.15.92:6838/3622725597,v1:172.21.15.92:6839/3622725597] up:boot 2024-03-23T18:08:47.559 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:47 smithi092 ceph-mon[79089]: mds.? [v2:172.21.15.92:6832/3039977494,v1:172.21.15.92:6834/3039977494] up:boot 2024-03-23T18:08:47.559 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:47 smithi092 ceph-mon[79089]: mds.? [v2:172.21.15.86:6836/1156936309,v1:172.21.15.86:6840/1156936309] up:boot 2024-03-23T18:08:47.559 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:47 smithi092 ceph-mon[79089]: mds.? [v2:172.21.15.86:6837/833138623,v1:172.21.15.86:6841/833138623] up:boot 2024-03-23T18:08:47.559 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:47 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/2469799771' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-23T18:08:47.559 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:47 smithi092 ceph-mon[79089]: fsmap 12 up:standby 2024-03-23T18:08:47.559 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:47 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-23T18:08:47.559 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:47 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-23T18:08:47.560 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:47 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-23T18:08:47.560 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:47 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-23T18:08:47.560 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:47 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-23T18:08:47.560 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:47 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-23T18:08:47.560 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:47 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-23T18:08:47.560 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:47 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-23T18:08:47.560 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:47 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-23T18:08:47.560 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:47 smithi092 ceph-mon[79089]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-23T18:08:47.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:47 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/2469799771' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-23T18:08:47.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:47 smithi152 ceph-mon[47949]: pgmap v575: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 192 KiB/s wr, 18 op/s 2024-03-23T18:08:47.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:47 smithi152 ceph-mon[47949]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-23T18:08:47.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:47 smithi152 ceph-mon[47949]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-23T18:08:47.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:47 smithi152 ceph-mon[47949]: Cluster is now healthy 2024-03-23T18:08:47.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:47 smithi152 ceph-mon[47949]: mds.? [v2:172.21.15.152:6834/3374019754,v1:172.21.15.152:6835/3374019754] up:boot 2024-03-23T18:08:47.587 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:47 smithi152 ceph-mon[47949]: mds.? [v2:172.21.15.92:6833/2633104629,v1:172.21.15.92:6835/2633104629] up:boot 2024-03-23T18:08:47.587 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:47 smithi152 ceph-mon[47949]: mds.? [v2:172.21.15.152:6837/1350317937,v1:172.21.15.152:6839/1350317937] up:boot 2024-03-23T18:08:47.587 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:47 smithi152 ceph-mon[47949]: mds.? [v2:172.21.15.152:6836/2909473996,v1:172.21.15.152:6838/2909473996] up:boot 2024-03-23T18:08:47.587 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:47 smithi152 ceph-mon[47949]: mds.? [v2:172.21.15.152:6832/4056042649,v1:172.21.15.152:6833/4056042649] up:boot 2024-03-23T18:08:47.588 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:47 smithi152 ceph-mon[47949]: mds.? [v2:172.21.15.92:6836/301800424,v1:172.21.15.92:6837/301800424] up:boot 2024-03-23T18:08:47.588 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:47 smithi152 ceph-mon[47949]: mds.? [v2:172.21.15.92:6838/3622725597,v1:172.21.15.92:6839/3622725597] up:boot 2024-03-23T18:08:47.588 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:47 smithi152 ceph-mon[47949]: mds.? [v2:172.21.15.92:6832/3039977494,v1:172.21.15.92:6834/3039977494] up:boot 2024-03-23T18:08:47.588 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:47 smithi152 ceph-mon[47949]: mds.? [v2:172.21.15.86:6836/1156936309,v1:172.21.15.86:6840/1156936309] up:boot 2024-03-23T18:08:47.588 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:47 smithi152 ceph-mon[47949]: mds.? [v2:172.21.15.86:6837/833138623,v1:172.21.15.86:6841/833138623] up:boot 2024-03-23T18:08:47.588 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:47 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/2469799771' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-23T18:08:47.588 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:47 smithi152 ceph-mon[47949]: fsmap 12 up:standby 2024-03-23T18:08:47.588 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:47 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-23T18:08:47.588 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:47 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-23T18:08:47.589 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:47 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-23T18:08:47.589 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:47 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-23T18:08:47.589 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:47 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-23T18:08:47.589 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:47 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-23T18:08:47.589 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:47 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-23T18:08:47.589 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:47 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-23T18:08:47.589 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:47 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-23T18:08:47.589 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:47 smithi152 ceph-mon[47949]: from='mgr.15375 172.21.15.86:0/398423972' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-23T18:08:48.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:48.114+0000 7fb7d0c00700 1 -- 172.21.15.86:0/780507119 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v112) v1 ==== 181+0+0 (secure 0 0 0) 0x7fb7c4099d00 con 0x7fb7d4100f60 2024-03-23T18:08:48.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:48.119+0000 7fb7da600700 1 -- 172.21.15.86:0/780507119 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fb7b80a1310 msgr2=0x7fb7b80a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:48.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:48.119+0000 7fb7da600700 1 --2- 172.21.15.86:0/780507119 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fb7b80a1310 0x7fb7b80a37d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb7d41988a0 tx=0x7fb7bc009260 comp rx=0 tx=0).stop 2024-03-23T18:08:48.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:48.119+0000 7fb7da600700 1 -- 172.21.15.86:0/780507119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7d4100f60 msgr2=0x7fb7d4197350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:48.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:48.119+0000 7fb7da600700 1 --2- 172.21.15.86:0/780507119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7d4100f60 0x7fb7d4197350 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fb7c400e9a0 tx=0x7fb7c400ecb0 comp rx=0 tx=0).stop 2024-03-23T18:08:48.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:48.120+0000 7fb7da600700 1 -- 172.21.15.86:0/780507119 shutdown_connections 2024-03-23T18:08:48.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:48.120+0000 7fb7da600700 1 --2- 172.21.15.86:0/780507119 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7fb7b80a1310 0x7fb7b80a37d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:48.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:48.120+0000 7fb7da600700 1 --2- 172.21.15.86:0/780507119 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb7d419bc30 0x7fb7d419c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:48.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:48.120+0000 7fb7da600700 1 --2- 172.21.15.86:0/780507119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7d4100f60 0x7fb7d4197350 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:48.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:48.120+0000 7fb7da600700 1 --2- 172.21.15.86:0/780507119 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7d4106080 0x7fb7d4197890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:48.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:48.120+0000 7fb7da600700 1 -- 172.21.15.86:0/780507119 >> 172.21.15.86:0/780507119 conn(0x7fb7d40fc7c0 msgr2=0x7fb7d40feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:08:48.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:48.120+0000 7fb7da600700 1 -- 172.21.15.86:0/780507119 shutdown_connections 2024-03-23T18:08:48.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:48.120+0000 7fb7da600700 1 -- 172.21.15.86:0/780507119 wait complete. 2024-03-23T18:08:48.124 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_metadata' removed 2024-03-23T18:08:48.155 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-23T18:08:48.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:48.413+0000 7ff72ec00700 1 -- 172.21.15.86:0/60972541 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff728105ce0 msgr2=0x7ff7281060c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:48.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:48.413+0000 7ff72ec00700 1 --2- 172.21.15.86:0/60972541 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff728105ce0 0x7ff7281060c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ff7180099c0 tx=0x7ff718009cd0 comp rx=0 tx=0).stop 2024-03-23T18:08:48.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:48.414+0000 7ff72ec00700 1 -- 172.21.15.86:0/60972541 shutdown_connections 2024-03-23T18:08:48.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:48.414+0000 7ff72ec00700 1 --2- 172.21.15.86:0/60972541 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff7280687f0 0x7ff72810db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:48.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:48.414+0000 7ff72ec00700 1 --2- 172.21.15.86:0/60972541 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff728105ce0 0x7ff7281060c0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:48.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:48.414+0000 7ff72ec00700 1 --2- 172.21.15.86:0/60972541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff728067e30 0x7ff7280682b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:48.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:48.414+0000 7ff72ec00700 1 -- 172.21.15.86:0/60972541 >> 172.21.15.86:0/60972541 conn(0x7ff728076d40 msgr2=0x7ff728077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:08:48.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:48.414+0000 7ff72ec00700 1 -- 172.21.15.86:0/60972541 shutdown_connections 2024-03-23T18:08:48.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:48.414+0000 7ff72ec00700 1 -- 172.21.15.86:0/60972541 wait complete. 2024-03-23T18:08:48.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:48.415+0000 7ff72ec00700 1 Processor -- start 2024-03-23T18:08:48.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:48.415+0000 7ff72ec00700 1 -- start start 2024-03-23T18:08:48.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:48.415+0000 7ff72ec00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff728067e30 0x7ff7280ffc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:48.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:48.415+0000 7ff72ec00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff7280687f0 0x7ff7281001b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:48.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:48.415+0000 7ff72ec00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff728105ce0 0x7ff7281006f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:48.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:48.415+0000 7ff72ec00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7281a55e0 con 0x7ff728067e30 2024-03-23T18:08:48.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:48.415+0000 7ff72ec00700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7281a5750 con 0x7ff728105ce0 2024-03-23T18:08:48.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:48.415+0000 7ff72ec00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7281a5910 con 0x7ff7280687f0 2024-03-23T18:08:48.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:48.415+0000 7ff72cc00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff728105ce0 0x7ff7281006f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:48.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:48.415+0000 7ff727e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff728067e30 0x7ff7280ffc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:48.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:48.415+0000 7ff72cc00700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff728105ce0 0x7ff7281006f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.86:37608/0 (socket says 172.21.15.86:37608) 2024-03-23T18:08:48.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:48.415+0000 7ff72cc00700 1 -- 172.21.15.86:0/4099289374 learned_addr learned my addr 172.21.15.86:0/4099289374 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-03-23T18:08:48.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:48.416+0000 7ff727400700 1 --2- 172.21.15.86:0/4099289374 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff7280687f0 0x7ff7281001b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:48.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:48.416+0000 7ff72cc00700 1 -- 172.21.15.86:0/4099289374 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff7280687f0 msgr2=0x7ff7281001b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:48.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:48.416+0000 7ff72cc00700 1 --2- 172.21.15.86:0/4099289374 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff7280687f0 0x7ff7281001b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:48.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:48.416+0000 7ff72cc00700 1 -- 172.21.15.86:0/4099289374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff728067e30 msgr2=0x7ff7280ffc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:48.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:48.416+0000 7ff72cc00700 1 --2- 172.21.15.86:0/4099289374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff728067e30 0x7ff7280ffc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:48.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:48.416+0000 7ff72cc00700 1 -- 172.21.15.86:0/4099289374 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7180095f0 con 0x7ff728105ce0 2024-03-23T18:08:48.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:48.416+0000 7ff727e00700 1 --2- 172.21.15.86:0/4099289374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff728067e30 0x7ff7280ffc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T18:08:48.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:48.416+0000 7ff72cc00700 1 --2- 172.21.15.86:0/4099289374 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff728105ce0 0x7ff7281006f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ff72000ef30 tx=0x7ff72000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:08:48.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:48.416+0000 7ff724c00700 1 -- 172.21.15.86:0/4099289374 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff720018070 con 0x7ff728105ce0 2024-03-23T18:08:48.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:48.416+0000 7ff724c00700 1 -- 172.21.15.86:0/4099289374 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff72000f040 con 0x7ff728105ce0 2024-03-23T18:08:48.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:48.416+0000 7ff72ec00700 1 -- 172.21.15.86:0/4099289374 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7281a5b90 con 0x7ff728105ce0 2024-03-23T18:08:48.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:48.416+0000 7ff724c00700 1 -- 172.21.15.86:0/4099289374 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff72000cd00 con 0x7ff728105ce0 2024-03-23T18:08:48.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:48.417+0000 7ff72ec00700 1 -- 172.21.15.86:0/4099289374 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7281a60f0 con 0x7ff728105ce0 2024-03-23T18:08:48.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:48.417+0000 7ff72ec00700 1 -- 172.21.15.86:0/4099289374 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff728109230 con 0x7ff728105ce0 2024-03-23T18:08:48.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:48.420+0000 7ff724c00700 1 -- 172.21.15.86:0/4099289374 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 26) v1 ==== 135996+0+0 (secure 0 0 0) 0x7ff720003a00 con 0x7ff728105ce0 2024-03-23T18:08:48.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:48.421+0000 7ff724c00700 1 --2- 172.21.15.86:0/4099289374 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7ff70c0a1310 0x7ff70c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T18:08:48.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:48.421+0000 7ff724c00700 1 -- 172.21.15.86:0/4099289374 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8345+0+0 (secure 0 0 0) 0x7ff7200cd0b0 con 0x7ff728105ce0 2024-03-23T18:08:48.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:48.421+0000 7ff727e00700 1 --2- 172.21.15.86:0/4099289374 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7ff70c0a1310 0x7ff70c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T18:08:48.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:48.422+0000 7ff727e00700 1 --2- 172.21.15.86:0/4099289374 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7ff70c0a1310 0x7ff70c0a37d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff718009990 tx=0x7ff718005fb0 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T18:08:48.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:48.427+0000 7ff724c00700 1 -- 172.21.15.86:0/4099289374 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ff7200965e0 con 0x7ff728105ce0 2024-03-23T18:08:48.516 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:48 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/780507119' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-23T18:08:48.559 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:48 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/780507119' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-23T18:08:48.585 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:48 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/780507119' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-23T18:08:48.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:48.729+0000 7ff72ec00700 1 -- 172.21.15.86:0/4099289374 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff728102430 con 0x7ff728105ce0 2024-03-23T18:08:49.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:49.129+0000 7ff724c00700 1 -- 172.21.15.86:0/4099289374 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v113) v1 ==== 169+0+0 (secure 0 0 0) 0x7ff720025380 con 0x7ff728105ce0 2024-03-23T18:08:49.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:49.133+0000 7ff72ec00700 1 -- 172.21.15.86:0/4099289374 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7ff70c0a1310 msgr2=0x7ff70c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:49.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:49.133+0000 7ff72ec00700 1 --2- 172.21.15.86:0/4099289374 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7ff70c0a1310 0x7ff70c0a37d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff718009990 tx=0x7ff718005fb0 comp rx=0 tx=0).stop 2024-03-23T18:08:49.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:49.133+0000 7ff72ec00700 1 -- 172.21.15.86:0/4099289374 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff728105ce0 msgr2=0x7ff7281006f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T18:08:49.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:49.133+0000 7ff72ec00700 1 --2- 172.21.15.86:0/4099289374 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff728105ce0 0x7ff7281006f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ff72000ef30 tx=0x7ff72000c3a0 comp rx=0 tx=0).stop 2024-03-23T18:08:49.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:49.133+0000 7ff72ec00700 1 -- 172.21.15.86:0/4099289374 shutdown_connections 2024-03-23T18:08:49.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:49.133+0000 7ff72ec00700 1 --2- 172.21.15.86:0/4099289374 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff728105ce0 0x7ff7281006f0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:49.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:49.133+0000 7ff72ec00700 1 --2- 172.21.15.86:0/4099289374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff728067e30 0x7ff7280ffc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:49.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:49.133+0000 7ff72ec00700 1 --2- 172.21.15.86:0/4099289374 >> [v2:172.21.15.86:6800/4022683855,v1:172.21.15.86:6801/4022683855] conn(0x7ff70c0a1310 0x7ff70c0a37d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:49.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:49.133+0000 7ff72ec00700 1 --2- 172.21.15.86:0/4099289374 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff7280687f0 0x7ff7281001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T18:08:49.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:49.133+0000 7ff72ec00700 1 -- 172.21.15.86:0/4099289374 >> 172.21.15.86:0/4099289374 conn(0x7ff728076d40 msgr2=0x7ff7280fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T18:08:49.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:49.133+0000 7ff72ec00700 1 -- 172.21.15.86:0/4099289374 shutdown_connections 2024-03-23T18:08:49.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-03-23T18:08:49.133+0000 7ff72ec00700 1 -- 172.21.15.86:0/4099289374 wait complete. 2024-03-23T18:08:49.136 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_data' removed 2024-03-23T18:08:49.162 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-23T18:08:49.254 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-23T18:08:49.294 DEBUG:teuthology.orchestra.run.smithi152:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-23T18:08:49.326 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-03-23T18:08:49.326 DEBUG:teuthology.orchestra.run.smithi086:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-23T18:08:49.372 DEBUG:teuthology.orchestra.run.smithi092:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-23T18:08:49.388 DEBUG:teuthology.orchestra.run.smithi152:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-23T18:08:49.405 INFO:tasks.cephadm:Stopping all daemons... 2024-03-23T18:08:49.405 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-03-23T18:08:49.405 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@mon.a 2024-03-23T18:08:49.470 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:49 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/780507119' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-23T18:08:49.470 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:49 smithi086 ceph-mon[38221]: osdmap e112: 12 total, 12 up, 12 in 2024-03-23T18:08:49.471 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:49 smithi086 ceph-mon[38221]: pgmap v577: 65 pgs: 65 active+clean; 373 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T18:08:49.471 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:49 smithi086 ceph-mon[38221]: from='client.? 172.21.15.86:0/4099289374' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-23T18:08:49.471 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:49 smithi086 ceph-mon[38221]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-23T18:08:49.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:49 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/780507119' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-23T18:08:49.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:49 smithi092 ceph-mon[79089]: osdmap e112: 12 total, 12 up, 12 in 2024-03-23T18:08:49.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:49 smithi092 ceph-mon[79089]: pgmap v577: 65 pgs: 65 active+clean; 373 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T18:08:49.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:49 smithi092 ceph-mon[79089]: from='client.? 172.21.15.86:0/4099289374' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-23T18:08:49.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:49 smithi092 ceph-mon[79089]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-23T18:08:49.585 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:49 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/780507119' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-23T18:08:49.585 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:49 smithi152 ceph-mon[47949]: osdmap e112: 12 total, 12 up, 12 in 2024-03-23T18:08:49.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:49 smithi152 ceph-mon[47949]: pgmap v577: 65 pgs: 65 active+clean; 373 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T18:08:49.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:49 smithi152 ceph-mon[47949]: from='client.? 172.21.15.86:0/4099289374' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-23T18:08:49.586 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:49 smithi152 ceph-mon[47949]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-23T18:08:49.768 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:49 smithi086 systemd[1]: Stopping Ceph mon.a for 9d76c9e8-e93c-11ee-95cd-87774f69a715... 2024-03-23T18:08:50.137 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:49 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mon-a[38198]: 2024-03-23T18:08:49.884+0000 7f70a9e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T18:08:50.138 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:49 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mon-a[38198]: 2024-03-23T18:08:49.884+0000 7f70a9e00700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-03-23T18:08:50.138 INFO:journalctl@ceph.mon.a.smithi086.stdout:Mar 23 18:08:50 smithi086 bash[118452]: ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mon-a 2024-03-23T18:08:50.559 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@mon.a.service' 2024-03-23T18:08:50.653 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T18:08:50.653 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-03-23T18:08:50.654 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-03-23T18:08:50.654 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@mon.b 2024-03-23T18:08:50.941 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:50 smithi092 systemd[1]: Stopping Ceph mon.b for 9d76c9e8-e93c-11ee-95cd-87774f69a715... 2024-03-23T18:08:50.941 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:50 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mon-b[79066]: 2024-03-23T18:08:50.939+0000 7f362d7f7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T18:08:50.941 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:50 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mon-b[79066]: 2024-03-23T18:08:50.939+0000 7f362d7f7700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-03-23T18:08:51.557 INFO:journalctl@ceph.mon.b.smithi092.stdout:Mar 23 18:08:51 smithi092 bash[122596]: ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mon-b 2024-03-23T18:08:51.631 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@mon.b.service' 2024-03-23T18:08:51.683 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T18:08:51.685 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-03-23T18:08:51.685 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-03-23T18:08:51.685 DEBUG:teuthology.orchestra.run.smithi152:> sudo systemctl stop ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@mon.c 2024-03-23T18:08:51.992 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:51 smithi152 systemd[1]: Stopping Ceph mon.c for 9d76c9e8-e93c-11ee-95cd-87774f69a715... 2024-03-23T18:08:52.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:51 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mon-c[47926]: 2024-03-23T18:08:51.990+0000 7fdf0cd4f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T18:08:52.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:51 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mon-c[47926]: 2024-03-23T18:08:51.990+0000 7fdf0cd4f700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-03-23T18:08:52.699 INFO:journalctl@ceph.mon.c.smithi152.stdout:Mar 23 18:08:52 smithi152 bash[94118]: ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mon-c 2024-03-23T18:08:52.709 DEBUG:teuthology.orchestra.run.smithi152:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@mon.c.service' 2024-03-23T18:08:52.780 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T18:08:52.780 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-03-23T18:08:52.780 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-03-23T18:08:52.780 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@mgr.x 2024-03-23T18:08:53.153 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 18:08:52 smithi086 systemd[1]: Stopping Ceph mgr.x for 9d76c9e8-e93c-11ee-95cd-87774f69a715... 2024-03-23T18:08:53.404 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 18:08:53 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T18:08:53.151+0000 7feadf800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T18:08:53.404 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 18:08:53 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x[38472]: 2024-03-23T18:08:53.151+0000 7feadf800700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-03-23T18:08:53.730 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Mar 23 18:08:53 smithi086 bash[118588]: ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-x 2024-03-23T18:08:53.814 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@mgr.x.service' 2024-03-23T18:08:53.898 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T18:08:53.898 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-03-23T18:08:53.898 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-03-23T18:08:53.898 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@mgr.y 2024-03-23T18:08:54.273 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Mar 23 18:08:53 smithi092 systemd[1]: Stopping Ceph mgr.y for 9d76c9e8-e93c-11ee-95cd-87774f69a715... 2024-03-23T18:08:54.557 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Mar 23 18:08:54 smithi092 bash[122734]: ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-y 2024-03-23T18:08:54.557 INFO:journalctl@ceph.mgr.y.smithi092.stdout:Mar 23 18:08:54 smithi092 systemd[1]: ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-03-23T18:08:54.641 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@mgr.y.service' 2024-03-23T18:08:54.678 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T18:08:54.678 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-03-23T18:08:54.678 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-03-23T18:08:54.679 DEBUG:teuthology.orchestra.run.smithi152:> sudo systemctl stop ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@mgr.z 2024-03-23T18:08:55.062 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Mar 23 18:08:54 smithi152 systemd[1]: Stopping Ceph mgr.z for 9d76c9e8-e93c-11ee-95cd-87774f69a715... 2024-03-23T18:08:55.336 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Mar 23 18:08:55 smithi152 bash[94260]: ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mgr-z 2024-03-23T18:08:55.336 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Mar 23 18:08:55 smithi152 systemd[1]: ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-03-23T18:08:55.422 DEBUG:teuthology.orchestra.run.smithi152:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@mgr.z.service' 2024-03-23T18:08:55.465 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T18:08:55.465 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-03-23T18:08:55.465 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-03-23T18:08:55.465 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@osd.0 2024-03-23T18:08:55.835 INFO:journalctl@ceph.osd.0.smithi086.stdout:Mar 23 18:08:55 smithi086 systemd[1]: Stopping Ceph osd.0 for 9d76c9e8-e93c-11ee-95cd-87774f69a715... 2024-03-23T18:08:56.266 INFO:journalctl@ceph.osd.0.smithi086.stdout:Mar 23 18:08:55 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-0[56695]: 2024-03-23T18:08:55.834+0000 7fbaae000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T18:08:56.266 INFO:journalctl@ceph.osd.0.smithi086.stdout:Mar 23 18:08:55 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-0[56695]: 2024-03-23T18:08:55.834+0000 7fbaae000700 -1 osd.0 113 *** Got signal Terminated *** 2024-03-23T18:08:56.266 INFO:journalctl@ceph.osd.0.smithi086.stdout:Mar 23 18:08:55 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-0[56695]: 2024-03-23T18:08:55.834+0000 7fbaae000700 -1 osd.0 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-23T18:09:02.016 INFO:journalctl@ceph.osd.0.smithi086.stdout:Mar 23 18:09:01 smithi086 bash[118727]: ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-0 2024-03-23T18:09:03.229 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@osd.0.service' 2024-03-23T18:09:03.294 INFO:journalctl@ceph.osd.0.smithi086.stdout:Mar 23 18:09:03 smithi086 systemd[1]: ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@osd.0.service: Succeeded. 2024-03-23T18:09:03.294 INFO:journalctl@ceph.osd.0.smithi086.stdout:Mar 23 18:09:03 smithi086 systemd[1]: Stopped Ceph osd.0 for 9d76c9e8-e93c-11ee-95cd-87774f69a715. 2024-03-23T18:09:03.320 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T18:09:03.321 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-03-23T18:09:03.321 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-03-23T18:09:03.321 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@osd.1 2024-03-23T18:09:03.569 INFO:journalctl@ceph.osd.1.smithi092.stdout:Mar 23 18:09:03 smithi092 systemd[1]: Stopping Ceph osd.1 for 9d76c9e8-e93c-11ee-95cd-87774f69a715... 2024-03-23T18:09:04.057 INFO:journalctl@ceph.osd.1.smithi092.stdout:Mar 23 18:09:03 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-1[84104]: 2024-03-23T18:09:03.568+0000 7fb8311e9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T18:09:04.057 INFO:journalctl@ceph.osd.1.smithi092.stdout:Mar 23 18:09:03 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-1[84104]: 2024-03-23T18:09:03.568+0000 7fb8311e9700 -1 osd.1 113 *** Got signal Terminated *** 2024-03-23T18:09:04.057 INFO:journalctl@ceph.osd.1.smithi092.stdout:Mar 23 18:09:03 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-1[84104]: 2024-03-23T18:09:03.568+0000 7fb8311e9700 -1 osd.1 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-23T18:09:09.557 INFO:journalctl@ceph.osd.1.smithi092.stdout:Mar 23 18:09:09 smithi092 bash[122877]: ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-1 2024-03-23T18:09:10.439 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@osd.1.service' 2024-03-23T18:09:10.540 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T18:09:10.540 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-03-23T18:09:10.540 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-03-23T18:09:10.541 DEBUG:teuthology.orchestra.run.smithi152:> sudo systemctl stop ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@osd.2 2024-03-23T18:09:10.835 INFO:journalctl@ceph.osd.2.smithi152.stdout:Mar 23 18:09:10 smithi152 systemd[1]: Stopping Ceph osd.2 for 9d76c9e8-e93c-11ee-95cd-87774f69a715... 2024-03-23T18:09:11.336 INFO:journalctl@ceph.osd.2.smithi152.stdout:Mar 23 18:09:10 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-2[53792]: 2024-03-23T18:09:10.862+0000 7f5ec8af0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T18:09:11.336 INFO:journalctl@ceph.osd.2.smithi152.stdout:Mar 23 18:09:10 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-2[53792]: 2024-03-23T18:09:10.862+0000 7f5ec8af0700 -1 osd.2 113 *** Got signal Terminated *** 2024-03-23T18:09:11.336 INFO:journalctl@ceph.osd.2.smithi152.stdout:Mar 23 18:09:10 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-2[53792]: 2024-03-23T18:09:10.862+0000 7f5ec8af0700 -1 osd.2 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-23T18:09:17.086 INFO:journalctl@ceph.osd.2.smithi152.stdout:Mar 23 18:09:16 smithi152 bash[94405]: ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-2 2024-03-23T18:09:17.849 DEBUG:teuthology.orchestra.run.smithi152:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@osd.2.service' 2024-03-23T18:09:17.890 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T18:09:17.891 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-03-23T18:09:17.891 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-03-23T18:09:17.891 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@osd.3 2024-03-23T18:09:18.266 INFO:journalctl@ceph.osd.3.smithi086.stdout:Mar 23 18:09:17 smithi086 systemd[1]: Stopping Ceph osd.3 for 9d76c9e8-e93c-11ee-95cd-87774f69a715... 2024-03-23T18:09:18.766 INFO:journalctl@ceph.osd.3.smithi086.stdout:Mar 23 18:09:18 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-3[65335]: 2024-03-23T18:09:18.289+0000 7f7b01200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T18:09:18.767 INFO:journalctl@ceph.osd.3.smithi086.stdout:Mar 23 18:09:18 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-3[65335]: 2024-03-23T18:09:18.289+0000 7f7b01200700 -1 osd.3 113 *** Got signal Terminated *** 2024-03-23T18:09:18.767 INFO:journalctl@ceph.osd.3.smithi086.stdout:Mar 23 18:09:18 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-3[65335]: 2024-03-23T18:09:18.289+0000 7f7b01200700 -1 osd.3 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-23T18:09:22.266 INFO:journalctl@ceph.osd.3.smithi086.stdout:Mar 23 18:09:21 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-3[65335]: 2024-03-23T18:09:21.942+0000 7f7af8400700 -1 osd.3 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:59.708275+0000 front 2024-03-23T18:08:59.708299+0000 (oldest deadline 2024-03-23T18:09:21.408449+0000) 2024-03-23T18:09:22.835 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 23 18:09:22 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-5[61654]: 2024-03-23T18:09:22.571+0000 7ff40b2f9700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:59.491692+0000 front 2024-03-23T18:08:59.491634+0000 (oldest deadline 2024-03-23T18:09:22.391775+0000) 2024-03-23T18:09:22.946 INFO:journalctl@ceph.osd.6.smithi086.stdout:Mar 23 18:09:22 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-6[74047]: 2024-03-23T18:09:22.589+0000 7f801c200700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:58.518283+0000 front 2024-03-23T18:08:58.518258+0000 (oldest deadline 2024-03-23T18:09:22.018402+0000) 2024-03-23T18:09:23.266 INFO:journalctl@ceph.osd.3.smithi086.stdout:Mar 23 18:09:22 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-3[65335]: 2024-03-23T18:09:22.946+0000 7f7af8400700 -1 osd.3 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:59.708275+0000 front 2024-03-23T18:08:59.708299+0000 (oldest deadline 2024-03-23T18:09:21.408449+0000) 2024-03-23T18:09:23.335 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:23 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:23.007+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.653471+0000 front 2024-03-23T18:09:00.653556+0000 (oldest deadline 2024-03-23T18:09:22.953547+0000) 2024-03-23T18:09:23.835 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 23 18:09:23 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-5[61654]: 2024-03-23T18:09:23.526+0000 7ff40b2f9700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:59.491692+0000 front 2024-03-23T18:08:59.491634+0000 (oldest deadline 2024-03-23T18:09:22.391775+0000) 2024-03-23T18:09:24.016 INFO:journalctl@ceph.osd.6.smithi086.stdout:Mar 23 18:09:23 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-6[74047]: 2024-03-23T18:09:23.558+0000 7f801c200700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:58.518283+0000 front 2024-03-23T18:08:58.518258+0000 (oldest deadline 2024-03-23T18:09:22.018402+0000) 2024-03-23T18:09:24.335 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:23 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:23.972+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.653471+0000 front 2024-03-23T18:09:00.653556+0000 (oldest deadline 2024-03-23T18:09:22.953547+0000) 2024-03-23T18:09:24.516 INFO:journalctl@ceph.osd.3.smithi086.stdout:Mar 23 18:09:24 smithi086 bash[119008]: ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-3 2024-03-23T18:09:24.807 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:24 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:24.338+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:59.796052+0000 front 2024-03-23T18:08:59.796002+0000 (oldest deadline 2024-03-23T18:09:23.895742+0000) 2024-03-23T18:09:24.835 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 23 18:09:24 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-5[61654]: 2024-03-23T18:09:24.542+0000 7ff40b2f9700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:59.491692+0000 front 2024-03-23T18:08:59.491634+0000 (oldest deadline 2024-03-23T18:09:22.391775+0000) 2024-03-23T18:09:25.016 INFO:journalctl@ceph.osd.6.smithi086.stdout:Mar 23 18:09:24 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-6[74047]: 2024-03-23T18:09:24.574+0000 7f801c200700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:58.518283+0000 front 2024-03-23T18:08:58.518258+0000 (oldest deadline 2024-03-23T18:09:22.018402+0000) 2024-03-23T18:09:25.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:24 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:24.576+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.122328+0000 front 2024-03-23T18:09:00.122374+0000 (oldest deadline 2024-03-23T18:09:24.122945+0000) 2024-03-23T18:09:25.335 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:24 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:24.935+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.653471+0000 front 2024-03-23T18:09:00.653556+0000 (oldest deadline 2024-03-23T18:09:22.953547+0000) 2024-03-23T18:09:25.672 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@osd.3.service' 2024-03-23T18:09:25.759 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T18:09:25.759 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-03-23T18:09:25.760 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-03-23T18:09:25.760 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@osd.4 2024-03-23T18:09:25.790 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:25 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:25.315+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:59.796052+0000 front 2024-03-23T18:08:59.796002+0000 (oldest deadline 2024-03-23T18:09:23.895742+0000) 2024-03-23T18:09:25.835 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 23 18:09:25 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-5[61654]: 2024-03-23T18:09:25.584+0000 7ff40b2f9700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:59.491692+0000 front 2024-03-23T18:08:59.491634+0000 (oldest deadline 2024-03-23T18:09:22.391775+0000) 2024-03-23T18:09:26.016 INFO:journalctl@ceph.osd.6.smithi086.stdout:Mar 23 18:09:25 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-6[74047]: 2024-03-23T18:09:25.544+0000 7f801c200700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:58.518283+0000 front 2024-03-23T18:08:58.518258+0000 (oldest deadline 2024-03-23T18:09:22.018402+0000) 2024-03-23T18:09:26.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:25 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:25.585+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.122328+0000 front 2024-03-23T18:09:00.122374+0000 (oldest deadline 2024-03-23T18:09:24.122945+0000) 2024-03-23T18:09:26.056 INFO:journalctl@ceph.osd.4.smithi092.stdout:Mar 23 18:09:25 smithi092 systemd[1]: Stopping Ceph osd.4 for 9d76c9e8-e93c-11ee-95cd-87774f69a715... 2024-03-23T18:09:26.057 INFO:journalctl@ceph.osd.4.smithi092.stdout:Mar 23 18:09:25 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-4[92840]: 2024-03-23T18:09:25.988+0000 7f65a7a91700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T18:09:26.057 INFO:journalctl@ceph.osd.4.smithi092.stdout:Mar 23 18:09:25 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-4[92840]: 2024-03-23T18:09:25.988+0000 7f65a7a91700 -1 osd.4 113 *** Got signal Terminated *** 2024-03-23T18:09:26.057 INFO:journalctl@ceph.osd.4.smithi092.stdout:Mar 23 18:09:25 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-4[92840]: 2024-03-23T18:09:25.988+0000 7f65a7a91700 -1 osd.4 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-23T18:09:26.335 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:25 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:25.920+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.653471+0000 front 2024-03-23T18:09:00.653556+0000 (oldest deadline 2024-03-23T18:09:22.953547+0000) 2024-03-23T18:09:26.807 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:26 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:26.309+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:59.796052+0000 front 2024-03-23T18:08:59.796002+0000 (oldest deadline 2024-03-23T18:09:23.895742+0000) 2024-03-23T18:09:26.835 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 23 18:09:26 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-5[61654]: 2024-03-23T18:09:26.573+0000 7ff40b2f9700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:59.491692+0000 front 2024-03-23T18:08:59.491634+0000 (oldest deadline 2024-03-23T18:09:22.391775+0000) 2024-03-23T18:09:27.016 INFO:journalctl@ceph.osd.6.smithi086.stdout:Mar 23 18:09:26 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-6[74047]: 2024-03-23T18:09:26.518+0000 7f801c200700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:58.518283+0000 front 2024-03-23T18:08:58.518258+0000 (oldest deadline 2024-03-23T18:09:22.018402+0000) 2024-03-23T18:09:27.017 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:26 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:26.632+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.122328+0000 front 2024-03-23T18:09:00.122374+0000 (oldest deadline 2024-03-23T18:09:24.122945+0000) 2024-03-23T18:09:27.335 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:26 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:26.964+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.653471+0000 front 2024-03-23T18:09:00.653556+0000 (oldest deadline 2024-03-23T18:09:22.953547+0000) 2024-03-23T18:09:27.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:27 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:27.283+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:59.796052+0000 front 2024-03-23T18:08:59.796002+0000 (oldest deadline 2024-03-23T18:09:23.895742+0000) 2024-03-23T18:09:27.766 INFO:journalctl@ceph.osd.6.smithi086.stdout:Mar 23 18:09:27 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-6[74047]: 2024-03-23T18:09:27.477+0000 7f801c200700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:58.518283+0000 front 2024-03-23T18:08:58.518258+0000 (oldest deadline 2024-03-23T18:09:22.018402+0000) 2024-03-23T18:09:27.767 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:27 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:27.603+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.122328+0000 front 2024-03-23T18:09:00.122374+0000 (oldest deadline 2024-03-23T18:09:24.122945+0000) 2024-03-23T18:09:27.835 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 23 18:09:27 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-5[61654]: 2024-03-23T18:09:27.526+0000 7ff40b2f9700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:59.491692+0000 front 2024-03-23T18:08:59.491634+0000 (oldest deadline 2024-03-23T18:09:22.391775+0000) 2024-03-23T18:09:28.335 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:27 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:27.950+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.653471+0000 front 2024-03-23T18:09:00.653556+0000 (oldest deadline 2024-03-23T18:09:22.953547+0000) 2024-03-23T18:09:28.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:28 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:28.250+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:59.796052+0000 front 2024-03-23T18:08:59.796002+0000 (oldest deadline 2024-03-23T18:09:23.895742+0000) 2024-03-23T18:09:28.835 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 23 18:09:28 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-5[61654]: 2024-03-23T18:09:28.534+0000 7ff40b2f9700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:59.491692+0000 front 2024-03-23T18:08:59.491634+0000 (oldest deadline 2024-03-23T18:09:22.391775+0000) 2024-03-23T18:09:29.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:28 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:28.572+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.122328+0000 front 2024-03-23T18:09:00.122374+0000 (oldest deadline 2024-03-23T18:09:24.122945+0000) 2024-03-23T18:09:29.017 INFO:journalctl@ceph.osd.6.smithi086.stdout:Mar 23 18:09:28 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-6[74047]: 2024-03-23T18:09:28.526+0000 7f801c200700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:58.518283+0000 front 2024-03-23T18:08:58.518258+0000 (oldest deadline 2024-03-23T18:09:22.018402+0000) 2024-03-23T18:09:29.335 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:28 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:28.957+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.653471+0000 front 2024-03-23T18:09:00.653556+0000 (oldest deadline 2024-03-23T18:09:22.953547+0000) 2024-03-23T18:09:29.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:29 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:29.247+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:59.796052+0000 front 2024-03-23T18:08:59.796002+0000 (oldest deadline 2024-03-23T18:09:23.895742+0000) 2024-03-23T18:09:29.835 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 23 18:09:29 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-5[61654]: 2024-03-23T18:09:29.584+0000 7ff40b2f9700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:59.491692+0000 front 2024-03-23T18:08:59.491634+0000 (oldest deadline 2024-03-23T18:09:22.391775+0000) 2024-03-23T18:09:29.835 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 23 18:09:29 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-5[61654]: 2024-03-23T18:09:29.584+0000 7ff40b2f9700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.293129+0000 front 2024-03-23T18:09:08.293074+0000 (oldest deadline 2024-03-23T18:09:29.393230+0000) 2024-03-23T18:09:30.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:29 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:29.579+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.122328+0000 front 2024-03-23T18:09:00.122374+0000 (oldest deadline 2024-03-23T18:09:24.122945+0000) 2024-03-23T18:09:30.017 INFO:journalctl@ceph.osd.6.smithi086.stdout:Mar 23 18:09:29 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-6[74047]: 2024-03-23T18:09:29.543+0000 7f801c200700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:58.518283+0000 front 2024-03-23T18:08:58.518258+0000 (oldest deadline 2024-03-23T18:09:22.018402+0000) 2024-03-23T18:09:30.017 INFO:journalctl@ceph.osd.6.smithi086.stdout:Mar 23 18:09:29 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-6[74047]: 2024-03-23T18:09:29.543+0000 7f801c200700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.720334+0000 front 2024-03-23T18:09:06.720273+0000 (oldest deadline 2024-03-23T18:09:29.020358+0000) 2024-03-23T18:09:30.335 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:29 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:29.908+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.653471+0000 front 2024-03-23T18:09:00.653556+0000 (oldest deadline 2024-03-23T18:09:22.953547+0000) 2024-03-23T18:09:30.557 INFO:journalctl@ceph.osd.4.smithi092.stdout:Mar 23 18:09:30 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-4[92840]: 2024-03-23T18:09:30.077+0000 7f65a4888700 -1 osd.4 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.887437+0000 front 2024-03-23T18:09:06.887452+0000 (oldest deadline 2024-03-23T18:09:29.187676+0000) 2024-03-23T18:09:30.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:30 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:30.297+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:59.796052+0000 front 2024-03-23T18:08:59.796002+0000 (oldest deadline 2024-03-23T18:09:23.895742+0000) 2024-03-23T18:09:30.941 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 23 18:09:30 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-5[61654]: 2024-03-23T18:09:30.587+0000 7ff40b2f9700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:59.491692+0000 front 2024-03-23T18:08:59.491634+0000 (oldest deadline 2024-03-23T18:09:22.391775+0000) 2024-03-23T18:09:30.941 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 23 18:09:30 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-5[61654]: 2024-03-23T18:09:30.587+0000 7ff40b2f9700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.293129+0000 front 2024-03-23T18:09:08.293074+0000 (oldest deadline 2024-03-23T18:09:29.393230+0000) 2024-03-23T18:09:31.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:30 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:30.593+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.122328+0000 front 2024-03-23T18:09:00.122374+0000 (oldest deadline 2024-03-23T18:09:24.122945+0000) 2024-03-23T18:09:31.016 INFO:journalctl@ceph.osd.6.smithi086.stdout:Mar 23 18:09:30 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-6[74047]: 2024-03-23T18:09:30.529+0000 7f801c200700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:58.518283+0000 front 2024-03-23T18:08:58.518258+0000 (oldest deadline 2024-03-23T18:09:22.018402+0000) 2024-03-23T18:09:31.016 INFO:journalctl@ceph.osd.6.smithi086.stdout:Mar 23 18:09:30 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-6[74047]: 2024-03-23T18:09:30.529+0000 7f801c200700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.720334+0000 front 2024-03-23T18:09:06.720273+0000 (oldest deadline 2024-03-23T18:09:29.020358+0000) 2024-03-23T18:09:31.335 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:30 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:30.941+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.653471+0000 front 2024-03-23T18:09:00.653556+0000 (oldest deadline 2024-03-23T18:09:22.953547+0000) 2024-03-23T18:09:31.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:31 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:31.260+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:59.796052+0000 front 2024-03-23T18:08:59.796002+0000 (oldest deadline 2024-03-23T18:09:23.895742+0000) 2024-03-23T18:09:31.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:31 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:31.260+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.796583+0000 front 2024-03-23T18:09:06.796752+0000 (oldest deadline 2024-03-23T18:09:30.896916+0000) 2024-03-23T18:09:31.835 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 23 18:09:31 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-5[61654]: 2024-03-23T18:09:31.546+0000 7ff40b2f9700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:59.491692+0000 front 2024-03-23T18:08:59.491634+0000 (oldest deadline 2024-03-23T18:09:22.391775+0000) 2024-03-23T18:09:31.836 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 23 18:09:31 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-5[61654]: 2024-03-23T18:09:31.546+0000 7ff40b2f9700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.293129+0000 front 2024-03-23T18:09:08.293074+0000 (oldest deadline 2024-03-23T18:09:29.393230+0000) 2024-03-23T18:09:32.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:31 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:31.632+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.122328+0000 front 2024-03-23T18:09:00.122374+0000 (oldest deadline 2024-03-23T18:09:24.122945+0000) 2024-03-23T18:09:32.017 INFO:journalctl@ceph.osd.6.smithi086.stdout:Mar 23 18:09:31 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-6[74047]: 2024-03-23T18:09:31.544+0000 7f801c200700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:58.518283+0000 front 2024-03-23T18:08:58.518258+0000 (oldest deadline 2024-03-23T18:09:22.018402+0000) 2024-03-23T18:09:32.017 INFO:journalctl@ceph.osd.6.smithi086.stdout:Mar 23 18:09:31 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-6[74047]: 2024-03-23T18:09:31.544+0000 7f801c200700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.720334+0000 front 2024-03-23T18:09:06.720273+0000 (oldest deadline 2024-03-23T18:09:29.020358+0000) 2024-03-23T18:09:32.057 INFO:journalctl@ceph.osd.4.smithi092.stdout:Mar 23 18:09:31 smithi092 bash[123131]: ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-4 2024-03-23T18:09:32.057 INFO:journalctl@ceph.osd.7.smithi092.stdout:Mar 23 18:09:31 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-7[100796]: 2024-03-23T18:09:31.636+0000 7fb53fa71700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.172261+0000 front 2024-03-23T18:09:06.172297+0000 (oldest deadline 2024-03-23T18:09:31.472456+0000) 2024-03-23T18:09:32.335 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:31 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:31.915+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.653471+0000 front 2024-03-23T18:09:00.653556+0000 (oldest deadline 2024-03-23T18:09:22.953547+0000) 2024-03-23T18:09:32.549 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:32 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:32.257+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:59.796052+0000 front 2024-03-23T18:08:59.796002+0000 (oldest deadline 2024-03-23T18:09:23.895742+0000) 2024-03-23T18:09:32.549 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:32 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:32.257+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.796583+0000 front 2024-03-23T18:09:06.796752+0000 (oldest deadline 2024-03-23T18:09:30.896916+0000) 2024-03-23T18:09:32.807 INFO:journalctl@ceph.osd.7.smithi092.stdout:Mar 23 18:09:32 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-7[100796]: 2024-03-23T18:09:32.628+0000 7fb53fa71700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.172261+0000 front 2024-03-23T18:09:06.172297+0000 (oldest deadline 2024-03-23T18:09:31.472456+0000) 2024-03-23T18:09:32.835 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 23 18:09:32 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-5[61654]: 2024-03-23T18:09:32.575+0000 7ff40b2f9700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:59.491692+0000 front 2024-03-23T18:08:59.491634+0000 (oldest deadline 2024-03-23T18:09:22.391775+0000) 2024-03-23T18:09:32.835 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 23 18:09:32 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-5[61654]: 2024-03-23T18:09:32.575+0000 7ff40b2f9700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.293129+0000 front 2024-03-23T18:09:08.293074+0000 (oldest deadline 2024-03-23T18:09:29.393230+0000) 2024-03-23T18:09:32.867 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@osd.4.service' 2024-03-23T18:09:32.948 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T18:09:32.949 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-03-23T18:09:32.949 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-03-23T18:09:32.949 DEBUG:teuthology.orchestra.run.smithi152:> sudo systemctl stop ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@osd.5 2024-03-23T18:09:33.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:32 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:32.631+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.122328+0000 front 2024-03-23T18:09:00.122374+0000 (oldest deadline 2024-03-23T18:09:24.122945+0000) 2024-03-23T18:09:33.016 INFO:journalctl@ceph.osd.6.smithi086.stdout:Mar 23 18:09:32 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-6[74047]: 2024-03-23T18:09:32.550+0000 7f801c200700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:58.518283+0000 front 2024-03-23T18:08:58.518258+0000 (oldest deadline 2024-03-23T18:09:22.018402+0000) 2024-03-23T18:09:33.016 INFO:journalctl@ceph.osd.6.smithi086.stdout:Mar 23 18:09:32 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-6[74047]: 2024-03-23T18:09:32.550+0000 7f801c200700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.720334+0000 front 2024-03-23T18:09:06.720273+0000 (oldest deadline 2024-03-23T18:09:29.020358+0000) 2024-03-23T18:09:33.185 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 23 18:09:32 smithi152 systemd[1]: Stopping Ceph osd.5 for 9d76c9e8-e93c-11ee-95cd-87774f69a715... 2024-03-23T18:09:33.185 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 23 18:09:33 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-5[61654]: 2024-03-23T18:09:33.184+0000 7ff41250a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T18:09:33.186 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:32 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:32.917+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.653471+0000 front 2024-03-23T18:09:00.653556+0000 (oldest deadline 2024-03-23T18:09:22.953547+0000) 2024-03-23T18:09:33.186 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:32 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:32.917+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.155613+0000 front 2024-03-23T18:09:08.155550+0000 (oldest deadline 2024-03-23T18:09:32.855633+0000) 2024-03-23T18:09:33.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:33 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:33.221+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:59.796052+0000 front 2024-03-23T18:08:59.796002+0000 (oldest deadline 2024-03-23T18:09:23.895742+0000) 2024-03-23T18:09:33.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:33 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:33.221+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.796583+0000 front 2024-03-23T18:09:06.796752+0000 (oldest deadline 2024-03-23T18:09:30.896916+0000) 2024-03-23T18:09:33.585 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 23 18:09:33 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-5[61654]: 2024-03-23T18:09:33.184+0000 7ff41250a700 -1 osd.5 113 *** Got signal Terminated *** 2024-03-23T18:09:33.585 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 23 18:09:33 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-5[61654]: 2024-03-23T18:09:33.184+0000 7ff41250a700 -1 osd.5 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-23T18:09:33.961 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 23 18:09:33 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-5[61654]: 2024-03-23T18:09:33.585+0000 7ff40b2f9700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:59.491692+0000 front 2024-03-23T18:08:59.491634+0000 (oldest deadline 2024-03-23T18:09:22.391775+0000) 2024-03-23T18:09:33.961 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 23 18:09:33 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-5[61654]: 2024-03-23T18:09:33.585+0000 7ff40b2f9700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.293129+0000 front 2024-03-23T18:09:08.293074+0000 (oldest deadline 2024-03-23T18:09:29.393230+0000) 2024-03-23T18:09:34.016 INFO:journalctl@ceph.osd.6.smithi086.stdout:Mar 23 18:09:33 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-6[74047]: 2024-03-23T18:09:33.523+0000 7f801c200700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:58.518283+0000 front 2024-03-23T18:08:58.518258+0000 (oldest deadline 2024-03-23T18:09:22.018402+0000) 2024-03-23T18:09:34.016 INFO:journalctl@ceph.osd.6.smithi086.stdout:Mar 23 18:09:33 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-6[74047]: 2024-03-23T18:09:33.523+0000 7f801c200700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.720334+0000 front 2024-03-23T18:09:06.720273+0000 (oldest deadline 2024-03-23T18:09:29.020358+0000) 2024-03-23T18:09:34.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:33 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:33.618+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.122328+0000 front 2024-03-23T18:09:00.122374+0000 (oldest deadline 2024-03-23T18:09:24.122945+0000) 2024-03-23T18:09:34.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:33 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:33.618+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:05.125484+0000 front 2024-03-23T18:09:05.125462+0000 (oldest deadline 2024-03-23T18:09:33.326776+0000) 2024-03-23T18:09:34.057 INFO:journalctl@ceph.osd.7.smithi092.stdout:Mar 23 18:09:33 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-7[100796]: 2024-03-23T18:09:33.612+0000 7fb53fa71700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.172261+0000 front 2024-03-23T18:09:06.172297+0000 (oldest deadline 2024-03-23T18:09:31.472456+0000) 2024-03-23T18:09:34.335 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:09:34 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T18:09:34.123+0000 7f722d382700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.458202+0000 front 2024-03-23T18:09:08.458278+0000 (oldest deadline 2024-03-23T18:09:33.158318+0000) 2024-03-23T18:09:34.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:33 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:33.961+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.653471+0000 front 2024-03-23T18:09:00.653556+0000 (oldest deadline 2024-03-23T18:09:22.953547+0000) 2024-03-23T18:09:34.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:33 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:33.961+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.155613+0000 front 2024-03-23T18:09:08.155550+0000 (oldest deadline 2024-03-23T18:09:32.855633+0000) 2024-03-23T18:09:34.556 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:34 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:34.239+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:59.796052+0000 front 2024-03-23T18:08:59.796002+0000 (oldest deadline 2024-03-23T18:09:23.895742+0000) 2024-03-23T18:09:34.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:34 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:34.239+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.796583+0000 front 2024-03-23T18:09:06.796752+0000 (oldest deadline 2024-03-23T18:09:30.896916+0000) 2024-03-23T18:09:34.835 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 23 18:09:34 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-5[61654]: 2024-03-23T18:09:34.550+0000 7ff40b2f9700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:59.491692+0000 front 2024-03-23T18:08:59.491634+0000 (oldest deadline 2024-03-23T18:09:22.391775+0000) 2024-03-23T18:09:34.835 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 23 18:09:34 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-5[61654]: 2024-03-23T18:09:34.550+0000 7ff40b2f9700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.293129+0000 front 2024-03-23T18:09:08.293074+0000 (oldest deadline 2024-03-23T18:09:29.393230+0000) 2024-03-23T18:09:35.016 INFO:journalctl@ceph.osd.6.smithi086.stdout:Mar 23 18:09:34 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-6[74047]: 2024-03-23T18:09:34.523+0000 7f801c200700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:58.518283+0000 front 2024-03-23T18:08:58.518258+0000 (oldest deadline 2024-03-23T18:09:22.018402+0000) 2024-03-23T18:09:35.016 INFO:journalctl@ceph.osd.6.smithi086.stdout:Mar 23 18:09:34 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-6[74047]: 2024-03-23T18:09:34.523+0000 7f801c200700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.720334+0000 front 2024-03-23T18:09:06.720273+0000 (oldest deadline 2024-03-23T18:09:29.020358+0000) 2024-03-23T18:09:35.017 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:34 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:34.636+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.122328+0000 front 2024-03-23T18:09:00.122374+0000 (oldest deadline 2024-03-23T18:09:24.122945+0000) 2024-03-23T18:09:35.017 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:34 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:34.636+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:05.125484+0000 front 2024-03-23T18:09:05.125462+0000 (oldest deadline 2024-03-23T18:09:33.326776+0000) 2024-03-23T18:09:35.056 INFO:journalctl@ceph.osd.7.smithi092.stdout:Mar 23 18:09:34 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-7[100796]: 2024-03-23T18:09:34.659+0000 7fb53fa71700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.172261+0000 front 2024-03-23T18:09:06.172297+0000 (oldest deadline 2024-03-23T18:09:31.472456+0000) 2024-03-23T18:09:35.335 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:09:35 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T18:09:35.073+0000 7f722d382700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.458202+0000 front 2024-03-23T18:09:08.458278+0000 (oldest deadline 2024-03-23T18:09:33.158318+0000) 2024-03-23T18:09:35.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:35 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:35.006+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.653471+0000 front 2024-03-23T18:09:00.653556+0000 (oldest deadline 2024-03-23T18:09:22.953547+0000) 2024-03-23T18:09:35.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:35 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:35.006+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.155613+0000 front 2024-03-23T18:09:08.155550+0000 (oldest deadline 2024-03-23T18:09:32.855633+0000) 2024-03-23T18:09:35.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:35 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:35.245+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:59.796052+0000 front 2024-03-23T18:08:59.796002+0000 (oldest deadline 2024-03-23T18:09:23.895742+0000) 2024-03-23T18:09:35.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:35 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:35.245+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.796583+0000 front 2024-03-23T18:09:06.796752+0000 (oldest deadline 2024-03-23T18:09:30.896916+0000) 2024-03-23T18:09:35.835 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 23 18:09:35 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-5[61654]: 2024-03-23T18:09:35.501+0000 7ff40b2f9700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:59.491692+0000 front 2024-03-23T18:08:59.491634+0000 (oldest deadline 2024-03-23T18:09:22.391775+0000) 2024-03-23T18:09:35.835 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 23 18:09:35 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-5[61654]: 2024-03-23T18:09:35.501+0000 7ff40b2f9700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.293129+0000 front 2024-03-23T18:09:08.293074+0000 (oldest deadline 2024-03-23T18:09:29.393230+0000) 2024-03-23T18:09:36.016 INFO:journalctl@ceph.osd.6.smithi086.stdout:Mar 23 18:09:35 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-6[74047]: 2024-03-23T18:09:35.535+0000 7f801c200700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:58.518283+0000 front 2024-03-23T18:08:58.518258+0000 (oldest deadline 2024-03-23T18:09:22.018402+0000) 2024-03-23T18:09:36.017 INFO:journalctl@ceph.osd.6.smithi086.stdout:Mar 23 18:09:35 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-6[74047]: 2024-03-23T18:09:35.535+0000 7f801c200700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.720334+0000 front 2024-03-23T18:09:06.720273+0000 (oldest deadline 2024-03-23T18:09:29.020358+0000) 2024-03-23T18:09:36.017 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:35 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:35.654+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.122328+0000 front 2024-03-23T18:09:00.122374+0000 (oldest deadline 2024-03-23T18:09:24.122945+0000) 2024-03-23T18:09:36.017 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:35 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:35.654+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:05.125484+0000 front 2024-03-23T18:09:05.125462+0000 (oldest deadline 2024-03-23T18:09:33.326776+0000) 2024-03-23T18:09:36.056 INFO:journalctl@ceph.osd.7.smithi092.stdout:Mar 23 18:09:35 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-7[100796]: 2024-03-23T18:09:35.698+0000 7fb53fa71700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.172261+0000 front 2024-03-23T18:09:06.172297+0000 (oldest deadline 2024-03-23T18:09:31.472456+0000) 2024-03-23T18:09:36.335 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:36 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:36.040+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.653471+0000 front 2024-03-23T18:09:00.653556+0000 (oldest deadline 2024-03-23T18:09:22.953547+0000) 2024-03-23T18:09:36.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:36 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:36.040+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.155613+0000 front 2024-03-23T18:09:08.155550+0000 (oldest deadline 2024-03-23T18:09:32.855633+0000) 2024-03-23T18:09:36.336 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:09:36 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T18:09:36.051+0000 7f722d382700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.458202+0000 front 2024-03-23T18:09:08.458278+0000 (oldest deadline 2024-03-23T18:09:33.158318+0000) 2024-03-23T18:09:36.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:36 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:36.214+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:59.796052+0000 front 2024-03-23T18:08:59.796002+0000 (oldest deadline 2024-03-23T18:09:23.895742+0000) 2024-03-23T18:09:36.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:36 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:36.214+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.796583+0000 front 2024-03-23T18:09:06.796752+0000 (oldest deadline 2024-03-23T18:09:30.896916+0000) 2024-03-23T18:09:36.835 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 23 18:09:36 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-5[61654]: 2024-03-23T18:09:36.474+0000 7ff40b2f9700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:59.491692+0000 front 2024-03-23T18:08:59.491634+0000 (oldest deadline 2024-03-23T18:09:22.391775+0000) 2024-03-23T18:09:36.836 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 23 18:09:36 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-5[61654]: 2024-03-23T18:09:36.474+0000 7ff40b2f9700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.293129+0000 front 2024-03-23T18:09:08.293074+0000 (oldest deadline 2024-03-23T18:09:29.393230+0000) 2024-03-23T18:09:37.016 INFO:journalctl@ceph.osd.6.smithi086.stdout:Mar 23 18:09:36 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-6[74047]: 2024-03-23T18:09:36.556+0000 7f801c200700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:58.518283+0000 front 2024-03-23T18:08:58.518258+0000 (oldest deadline 2024-03-23T18:09:22.018402+0000) 2024-03-23T18:09:37.016 INFO:journalctl@ceph.osd.6.smithi086.stdout:Mar 23 18:09:36 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-6[74047]: 2024-03-23T18:09:36.556+0000 7f801c200700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.720334+0000 front 2024-03-23T18:09:06.720273+0000 (oldest deadline 2024-03-23T18:09:29.020358+0000) 2024-03-23T18:09:37.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:36 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:36.675+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.122328+0000 front 2024-03-23T18:09:00.122374+0000 (oldest deadline 2024-03-23T18:09:24.122945+0000) 2024-03-23T18:09:37.017 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:36 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:36.675+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:05.125484+0000 front 2024-03-23T18:09:05.125462+0000 (oldest deadline 2024-03-23T18:09:33.326776+0000) 2024-03-23T18:09:37.056 INFO:journalctl@ceph.osd.7.smithi092.stdout:Mar 23 18:09:36 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-7[100796]: 2024-03-23T18:09:36.679+0000 7fb53fa71700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.172261+0000 front 2024-03-23T18:09:06.172297+0000 (oldest deadline 2024-03-23T18:09:31.472456+0000) 2024-03-23T18:09:37.335 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:09:37 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T18:09:37.011+0000 7f722d382700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.458202+0000 front 2024-03-23T18:09:08.458278+0000 (oldest deadline 2024-03-23T18:09:33.158318+0000) 2024-03-23T18:09:37.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:37 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:37.090+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.653471+0000 front 2024-03-23T18:09:00.653556+0000 (oldest deadline 2024-03-23T18:09:22.953547+0000) 2024-03-23T18:09:37.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:37 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:37.090+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.155613+0000 front 2024-03-23T18:09:08.155550+0000 (oldest deadline 2024-03-23T18:09:32.855633+0000) 2024-03-23T18:09:37.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:37 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:37.180+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:59.796052+0000 front 2024-03-23T18:08:59.796002+0000 (oldest deadline 2024-03-23T18:09:23.895742+0000) 2024-03-23T18:09:37.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:37 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:37.180+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.796583+0000 front 2024-03-23T18:09:06.796752+0000 (oldest deadline 2024-03-23T18:09:30.896916+0000) 2024-03-23T18:09:37.835 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 23 18:09:37 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-5[61654]: 2024-03-23T18:09:37.435+0000 7ff40b2f9700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:59.491692+0000 front 2024-03-23T18:08:59.491634+0000 (oldest deadline 2024-03-23T18:09:22.391775+0000) 2024-03-23T18:09:37.835 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 23 18:09:37 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-5[61654]: 2024-03-23T18:09:37.435+0000 7ff40b2f9700 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.293129+0000 front 2024-03-23T18:09:08.293074+0000 (oldest deadline 2024-03-23T18:09:29.393230+0000) 2024-03-23T18:09:38.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:37 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:37.696+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.122328+0000 front 2024-03-23T18:09:00.122374+0000 (oldest deadline 2024-03-23T18:09:24.122945+0000) 2024-03-23T18:09:38.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:37 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:37.696+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:05.125484+0000 front 2024-03-23T18:09:05.125462+0000 (oldest deadline 2024-03-23T18:09:33.326776+0000) 2024-03-23T18:09:38.017 INFO:journalctl@ceph.osd.6.smithi086.stdout:Mar 23 18:09:37 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-6[74047]: 2024-03-23T18:09:37.556+0000 7f801c200700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:58.518283+0000 front 2024-03-23T18:08:58.518258+0000 (oldest deadline 2024-03-23T18:09:22.018402+0000) 2024-03-23T18:09:38.017 INFO:journalctl@ceph.osd.6.smithi086.stdout:Mar 23 18:09:37 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-6[74047]: 2024-03-23T18:09:37.556+0000 7f801c200700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.720334+0000 front 2024-03-23T18:09:06.720273+0000 (oldest deadline 2024-03-23T18:09:29.020358+0000) 2024-03-23T18:09:38.056 INFO:journalctl@ceph.osd.7.smithi092.stdout:Mar 23 18:09:37 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-7[100796]: 2024-03-23T18:09:37.692+0000 7fb53fa71700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.172261+0000 front 2024-03-23T18:09:06.172297+0000 (oldest deadline 2024-03-23T18:09:31.472456+0000) 2024-03-23T18:09:38.336 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:09:38 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T18:09:38.057+0000 7f722d382700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.458202+0000 front 2024-03-23T18:09:08.458278+0000 (oldest deadline 2024-03-23T18:09:33.158318+0000) 2024-03-23T18:09:38.336 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:09:38 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T18:09:38.057+0000 7f722d382700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:13.158888+0000 front 2024-03-23T18:09:13.158856+0000 (oldest deadline 2024-03-23T18:09:37.858881+0000) 2024-03-23T18:09:38.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:38 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:38.051+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.653471+0000 front 2024-03-23T18:09:00.653556+0000 (oldest deadline 2024-03-23T18:09:22.953547+0000) 2024-03-23T18:09:38.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:38 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:38.051+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.155613+0000 front 2024-03-23T18:09:08.155550+0000 (oldest deadline 2024-03-23T18:09:32.855633+0000) 2024-03-23T18:09:38.337 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:38 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:38.051+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.057340+0000 front 2024-03-23T18:09:15.057318+0000 (oldest deadline 2024-03-23T18:09:37.357408+0000) 2024-03-23T18:09:38.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:38 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:38.179+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:59.796052+0000 front 2024-03-23T18:08:59.796002+0000 (oldest deadline 2024-03-23T18:09:23.895742+0000) 2024-03-23T18:09:38.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:38 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:38.179+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.796583+0000 front 2024-03-23T18:09:06.796752+0000 (oldest deadline 2024-03-23T18:09:30.896916+0000) 2024-03-23T18:09:38.766 INFO:journalctl@ceph.osd.6.smithi086.stdout:Mar 23 18:09:38 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-6[74047]: 2024-03-23T18:09:38.507+0000 7f801c200700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:58.518283+0000 front 2024-03-23T18:08:58.518258+0000 (oldest deadline 2024-03-23T18:09:22.018402+0000) 2024-03-23T18:09:38.767 INFO:journalctl@ceph.osd.6.smithi086.stdout:Mar 23 18:09:38 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-6[74047]: 2024-03-23T18:09:38.507+0000 7f801c200700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.720334+0000 front 2024-03-23T18:09:06.720273+0000 (oldest deadline 2024-03-23T18:09:29.020358+0000) 2024-03-23T18:09:38.767 INFO:journalctl@ceph.osd.6.smithi086.stdout:Mar 23 18:09:38 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-6[74047]: 2024-03-23T18:09:38.507+0000 7f801c200700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:14.922410+0000 front 2024-03-23T18:09:14.922473+0000 (oldest deadline 2024-03-23T18:09:38.422603+0000) 2024-03-23T18:09:38.767 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:38 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:38.696+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.122328+0000 front 2024-03-23T18:09:00.122374+0000 (oldest deadline 2024-03-23T18:09:24.122945+0000) 2024-03-23T18:09:38.767 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:38 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:38.696+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:05.125484+0000 front 2024-03-23T18:09:05.125462+0000 (oldest deadline 2024-03-23T18:09:33.326776+0000) 2024-03-23T18:09:39.056 INFO:journalctl@ceph.osd.7.smithi092.stdout:Mar 23 18:09:38 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-7[100796]: 2024-03-23T18:09:38.643+0000 7fb53fa71700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.172261+0000 front 2024-03-23T18:09:06.172297+0000 (oldest deadline 2024-03-23T18:09:31.472456+0000) 2024-03-23T18:09:39.057 INFO:journalctl@ceph.osd.7.smithi092.stdout:Mar 23 18:09:38 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-7[100796]: 2024-03-23T18:09:38.643+0000 7fb53fa71700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.573435+0000 front 2024-03-23T18:09:15.573470+0000 (oldest deadline 2024-03-23T18:09:38.473559+0000) 2024-03-23T18:09:39.335 INFO:journalctl@ceph.osd.5.smithi152.stdout:Mar 23 18:09:38 smithi152 bash[94668]: ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-5 2024-03-23T18:09:39.336 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:09:39 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T18:09:39.017+0000 7f722d382700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.458202+0000 front 2024-03-23T18:09:08.458278+0000 (oldest deadline 2024-03-23T18:09:33.158318+0000) 2024-03-23T18:09:39.336 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:09:39 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T18:09:39.017+0000 7f722d382700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:13.158888+0000 front 2024-03-23T18:09:13.158856+0000 (oldest deadline 2024-03-23T18:09:37.858881+0000) 2024-03-23T18:09:39.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:39 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:39.055+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.653471+0000 front 2024-03-23T18:09:00.653556+0000 (oldest deadline 2024-03-23T18:09:22.953547+0000) 2024-03-23T18:09:39.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:39 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:39.055+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.155613+0000 front 2024-03-23T18:09:08.155550+0000 (oldest deadline 2024-03-23T18:09:32.855633+0000) 2024-03-23T18:09:39.337 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:39 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:39.055+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.057340+0000 front 2024-03-23T18:09:15.057318+0000 (oldest deadline 2024-03-23T18:09:37.357408+0000) 2024-03-23T18:09:39.556 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:39 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:39.192+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:59.796052+0000 front 2024-03-23T18:08:59.796002+0000 (oldest deadline 2024-03-23T18:09:23.895742+0000) 2024-03-23T18:09:39.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:39 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:39.192+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.796583+0000 front 2024-03-23T18:09:06.796752+0000 (oldest deadline 2024-03-23T18:09:30.896916+0000) 2024-03-23T18:09:40.016 INFO:journalctl@ceph.osd.6.smithi086.stdout:Mar 23 18:09:39 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-6[74047]: 2024-03-23T18:09:39.540+0000 7f801c200700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:58.518283+0000 front 2024-03-23T18:08:58.518258+0000 (oldest deadline 2024-03-23T18:09:22.018402+0000) 2024-03-23T18:09:40.016 INFO:journalctl@ceph.osd.6.smithi086.stdout:Mar 23 18:09:39 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-6[74047]: 2024-03-23T18:09:39.540+0000 7f801c200700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.720334+0000 front 2024-03-23T18:09:06.720273+0000 (oldest deadline 2024-03-23T18:09:29.020358+0000) 2024-03-23T18:09:40.016 INFO:journalctl@ceph.osd.6.smithi086.stdout:Mar 23 18:09:39 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-6[74047]: 2024-03-23T18:09:39.540+0000 7f801c200700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:14.922410+0000 front 2024-03-23T18:09:14.922473+0000 (oldest deadline 2024-03-23T18:09:38.422603+0000) 2024-03-23T18:09:40.017 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:39 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:39.676+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.122328+0000 front 2024-03-23T18:09:00.122374+0000 (oldest deadline 2024-03-23T18:09:24.122945+0000) 2024-03-23T18:09:40.017 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:39 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:39.676+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:05.125484+0000 front 2024-03-23T18:09:05.125462+0000 (oldest deadline 2024-03-23T18:09:33.326776+0000) 2024-03-23T18:09:40.017 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:39 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:39.676+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:13.327558+0000 front 2024-03-23T18:09:13.327314+0000 (oldest deadline 2024-03-23T18:09:39.227552+0000) 2024-03-23T18:09:40.056 INFO:journalctl@ceph.osd.7.smithi092.stdout:Mar 23 18:09:39 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-7[100796]: 2024-03-23T18:09:39.681+0000 7fb53fa71700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.172261+0000 front 2024-03-23T18:09:06.172297+0000 (oldest deadline 2024-03-23T18:09:31.472456+0000) 2024-03-23T18:09:40.057 INFO:journalctl@ceph.osd.7.smithi092.stdout:Mar 23 18:09:39 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-7[100796]: 2024-03-23T18:09:39.681+0000 7fb53fa71700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.573435+0000 front 2024-03-23T18:09:15.573470+0000 (oldest deadline 2024-03-23T18:09:38.473559+0000) 2024-03-23T18:09:40.180 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:40 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:40.084+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.653471+0000 front 2024-03-23T18:09:00.653556+0000 (oldest deadline 2024-03-23T18:09:22.953547+0000) 2024-03-23T18:09:40.181 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:40 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:40.084+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.155613+0000 front 2024-03-23T18:09:08.155550+0000 (oldest deadline 2024-03-23T18:09:32.855633+0000) 2024-03-23T18:09:40.181 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:40 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:40.084+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.057340+0000 front 2024-03-23T18:09:15.057318+0000 (oldest deadline 2024-03-23T18:09:37.357408+0000) 2024-03-23T18:09:40.181 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:09:40 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T18:09:40.003+0000 7f722d382700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.458202+0000 front 2024-03-23T18:09:08.458278+0000 (oldest deadline 2024-03-23T18:09:33.158318+0000) 2024-03-23T18:09:40.181 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:09:40 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T18:09:40.003+0000 7f722d382700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:13.158888+0000 front 2024-03-23T18:09:13.158856+0000 (oldest deadline 2024-03-23T18:09:37.858881+0000) 2024-03-23T18:09:40.187 DEBUG:teuthology.orchestra.run.smithi152:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@osd.5.service' 2024-03-23T18:09:40.228 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T18:09:40.229 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-03-23T18:09:40.229 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-03-23T18:09:40.229 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@osd.6 2024-03-23T18:09:40.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:40 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:40.154+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:59.796052+0000 front 2024-03-23T18:08:59.796002+0000 (oldest deadline 2024-03-23T18:09:23.895742+0000) 2024-03-23T18:09:40.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:40 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:40.154+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.796583+0000 front 2024-03-23T18:09:06.796752+0000 (oldest deadline 2024-03-23T18:09:30.896916+0000) 2024-03-23T18:09:40.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:40 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:40.154+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.498831+0000 front 2024-03-23T18:09:15.498800+0000 (oldest deadline 2024-03-23T18:09:39.598988+0000) 2024-03-23T18:09:40.569 INFO:journalctl@ceph.osd.6.smithi086.stdout:Mar 23 18:09:40 smithi086 systemd[1]: Stopping Ceph osd.6 for 9d76c9e8-e93c-11ee-95cd-87774f69a715... 2024-03-23T18:09:41.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:40 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:40.644+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.122328+0000 front 2024-03-23T18:09:00.122374+0000 (oldest deadline 2024-03-23T18:09:24.122945+0000) 2024-03-23T18:09:41.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:40 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:40.644+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:05.125484+0000 front 2024-03-23T18:09:05.125462+0000 (oldest deadline 2024-03-23T18:09:33.326776+0000) 2024-03-23T18:09:41.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:40 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:40.644+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:13.327558+0000 front 2024-03-23T18:09:13.327314+0000 (oldest deadline 2024-03-23T18:09:39.227552+0000) 2024-03-23T18:09:41.017 INFO:journalctl@ceph.osd.6.smithi086.stdout:Mar 23 18:09:40 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-6[74047]: 2024-03-23T18:09:40.567+0000 7f801c200700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:58.518283+0000 front 2024-03-23T18:08:58.518258+0000 (oldest deadline 2024-03-23T18:09:22.018402+0000) 2024-03-23T18:09:41.017 INFO:journalctl@ceph.osd.6.smithi086.stdout:Mar 23 18:09:40 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-6[74047]: 2024-03-23T18:09:40.567+0000 7f801c200700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.720334+0000 front 2024-03-23T18:09:06.720273+0000 (oldest deadline 2024-03-23T18:09:29.020358+0000) 2024-03-23T18:09:41.017 INFO:journalctl@ceph.osd.6.smithi086.stdout:Mar 23 18:09:40 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-6[74047]: 2024-03-23T18:09:40.567+0000 7f801c200700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:14.922410+0000 front 2024-03-23T18:09:14.922473+0000 (oldest deadline 2024-03-23T18:09:38.422603+0000) 2024-03-23T18:09:41.017 INFO:journalctl@ceph.osd.6.smithi086.stdout:Mar 23 18:09:40 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-6[74047]: 2024-03-23T18:09:40.615+0000 7f8025000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T18:09:41.017 INFO:journalctl@ceph.osd.6.smithi086.stdout:Mar 23 18:09:40 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-6[74047]: 2024-03-23T18:09:40.615+0000 7f8025000700 -1 osd.6 113 *** Got signal Terminated *** 2024-03-23T18:09:41.017 INFO:journalctl@ceph.osd.6.smithi086.stdout:Mar 23 18:09:40 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-6[74047]: 2024-03-23T18:09:40.615+0000 7f8025000700 -1 osd.6 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-23T18:09:41.057 INFO:journalctl@ceph.osd.7.smithi092.stdout:Mar 23 18:09:40 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-7[100796]: 2024-03-23T18:09:40.634+0000 7fb53fa71700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.172261+0000 front 2024-03-23T18:09:06.172297+0000 (oldest deadline 2024-03-23T18:09:31.472456+0000) 2024-03-23T18:09:41.057 INFO:journalctl@ceph.osd.7.smithi092.stdout:Mar 23 18:09:40 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-7[100796]: 2024-03-23T18:09:40.634+0000 7fb53fa71700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.573435+0000 front 2024-03-23T18:09:15.573470+0000 (oldest deadline 2024-03-23T18:09:38.473559+0000) 2024-03-23T18:09:41.335 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:09:41 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T18:09:41.001+0000 7f722d382700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.458202+0000 front 2024-03-23T18:09:08.458278+0000 (oldest deadline 2024-03-23T18:09:33.158318+0000) 2024-03-23T18:09:41.336 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:09:41 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T18:09:41.001+0000 7f722d382700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:13.158888+0000 front 2024-03-23T18:09:13.158856+0000 (oldest deadline 2024-03-23T18:09:37.858881+0000) 2024-03-23T18:09:41.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:41 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:41.050+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.653471+0000 front 2024-03-23T18:09:00.653556+0000 (oldest deadline 2024-03-23T18:09:22.953547+0000) 2024-03-23T18:09:41.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:41 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:41.050+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.155613+0000 front 2024-03-23T18:09:08.155550+0000 (oldest deadline 2024-03-23T18:09:32.855633+0000) 2024-03-23T18:09:41.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:41 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:41.050+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.057340+0000 front 2024-03-23T18:09:15.057318+0000 (oldest deadline 2024-03-23T18:09:37.357408+0000) 2024-03-23T18:09:41.556 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:41 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:41.189+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:59.796052+0000 front 2024-03-23T18:08:59.796002+0000 (oldest deadline 2024-03-23T18:09:23.895742+0000) 2024-03-23T18:09:41.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:41 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:41.189+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.796583+0000 front 2024-03-23T18:09:06.796752+0000 (oldest deadline 2024-03-23T18:09:30.896916+0000) 2024-03-23T18:09:41.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:41 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:41.189+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.498831+0000 front 2024-03-23T18:09:15.498800+0000 (oldest deadline 2024-03-23T18:09:39.598988+0000) 2024-03-23T18:09:42.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:41 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:41.640+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.122328+0000 front 2024-03-23T18:09:00.122374+0000 (oldest deadline 2024-03-23T18:09:24.122945+0000) 2024-03-23T18:09:42.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:41 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:41.640+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:05.125484+0000 front 2024-03-23T18:09:05.125462+0000 (oldest deadline 2024-03-23T18:09:33.326776+0000) 2024-03-23T18:09:42.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:41 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:41.640+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:13.327558+0000 front 2024-03-23T18:09:13.327314+0000 (oldest deadline 2024-03-23T18:09:39.227552+0000) 2024-03-23T18:09:42.017 INFO:journalctl@ceph.osd.6.smithi086.stdout:Mar 23 18:09:41 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-6[74047]: 2024-03-23T18:09:41.558+0000 7f801c200700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:58.518283+0000 front 2024-03-23T18:08:58.518258+0000 (oldest deadline 2024-03-23T18:09:22.018402+0000) 2024-03-23T18:09:42.017 INFO:journalctl@ceph.osd.6.smithi086.stdout:Mar 23 18:09:41 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-6[74047]: 2024-03-23T18:09:41.558+0000 7f801c200700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.720334+0000 front 2024-03-23T18:09:06.720273+0000 (oldest deadline 2024-03-23T18:09:29.020358+0000) 2024-03-23T18:09:42.017 INFO:journalctl@ceph.osd.6.smithi086.stdout:Mar 23 18:09:41 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-6[74047]: 2024-03-23T18:09:41.558+0000 7f801c200700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:14.922410+0000 front 2024-03-23T18:09:14.922473+0000 (oldest deadline 2024-03-23T18:09:38.422603+0000) 2024-03-23T18:09:42.057 INFO:journalctl@ceph.osd.7.smithi092.stdout:Mar 23 18:09:41 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-7[100796]: 2024-03-23T18:09:41.624+0000 7fb53fa71700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.172261+0000 front 2024-03-23T18:09:06.172297+0000 (oldest deadline 2024-03-23T18:09:31.472456+0000) 2024-03-23T18:09:42.057 INFO:journalctl@ceph.osd.7.smithi092.stdout:Mar 23 18:09:41 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-7[100796]: 2024-03-23T18:09:41.624+0000 7fb53fa71700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.573435+0000 front 2024-03-23T18:09:15.573470+0000 (oldest deadline 2024-03-23T18:09:38.473559+0000) 2024-03-23T18:09:42.335 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:09:41 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T18:09:41.986+0000 7f722d382700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.458202+0000 front 2024-03-23T18:09:08.458278+0000 (oldest deadline 2024-03-23T18:09:33.158318+0000) 2024-03-23T18:09:42.336 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:09:41 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T18:09:41.986+0000 7f722d382700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:13.158888+0000 front 2024-03-23T18:09:13.158856+0000 (oldest deadline 2024-03-23T18:09:37.858881+0000) 2024-03-23T18:09:42.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:42 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:42.057+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.653471+0000 front 2024-03-23T18:09:00.653556+0000 (oldest deadline 2024-03-23T18:09:22.953547+0000) 2024-03-23T18:09:42.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:42 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:42.057+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.155613+0000 front 2024-03-23T18:09:08.155550+0000 (oldest deadline 2024-03-23T18:09:32.855633+0000) 2024-03-23T18:09:42.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:42 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:42.057+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.057340+0000 front 2024-03-23T18:09:15.057318+0000 (oldest deadline 2024-03-23T18:09:37.357408+0000) 2024-03-23T18:09:42.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:42 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:42.212+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:59.796052+0000 front 2024-03-23T18:08:59.796002+0000 (oldest deadline 2024-03-23T18:09:23.895742+0000) 2024-03-23T18:09:42.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:42 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:42.212+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.796583+0000 front 2024-03-23T18:09:06.796752+0000 (oldest deadline 2024-03-23T18:09:30.896916+0000) 2024-03-23T18:09:42.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:42 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:42.212+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.498831+0000 front 2024-03-23T18:09:15.498800+0000 (oldest deadline 2024-03-23T18:09:39.598988+0000) 2024-03-23T18:09:43.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:42 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:42.627+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.122328+0000 front 2024-03-23T18:09:00.122374+0000 (oldest deadline 2024-03-23T18:09:24.122945+0000) 2024-03-23T18:09:43.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:42 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:42.627+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:05.125484+0000 front 2024-03-23T18:09:05.125462+0000 (oldest deadline 2024-03-23T18:09:33.326776+0000) 2024-03-23T18:09:43.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:42 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:42.627+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:13.327558+0000 front 2024-03-23T18:09:13.327314+0000 (oldest deadline 2024-03-23T18:09:39.227552+0000) 2024-03-23T18:09:43.017 INFO:journalctl@ceph.osd.6.smithi086.stdout:Mar 23 18:09:42 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-6[74047]: 2024-03-23T18:09:42.582+0000 7f801c200700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:58.518283+0000 front 2024-03-23T18:08:58.518258+0000 (oldest deadline 2024-03-23T18:09:22.018402+0000) 2024-03-23T18:09:43.017 INFO:journalctl@ceph.osd.6.smithi086.stdout:Mar 23 18:09:42 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-6[74047]: 2024-03-23T18:09:42.582+0000 7f801c200700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.720334+0000 front 2024-03-23T18:09:06.720273+0000 (oldest deadline 2024-03-23T18:09:29.020358+0000) 2024-03-23T18:09:43.017 INFO:journalctl@ceph.osd.6.smithi086.stdout:Mar 23 18:09:42 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-6[74047]: 2024-03-23T18:09:42.582+0000 7f801c200700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:14.922410+0000 front 2024-03-23T18:09:14.922473+0000 (oldest deadline 2024-03-23T18:09:38.422603+0000) 2024-03-23T18:09:43.056 INFO:journalctl@ceph.osd.7.smithi092.stdout:Mar 23 18:09:42 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-7[100796]: 2024-03-23T18:09:42.664+0000 7fb53fa71700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.172261+0000 front 2024-03-23T18:09:06.172297+0000 (oldest deadline 2024-03-23T18:09:31.472456+0000) 2024-03-23T18:09:43.057 INFO:journalctl@ceph.osd.7.smithi092.stdout:Mar 23 18:09:42 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-7[100796]: 2024-03-23T18:09:42.664+0000 7fb53fa71700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.573435+0000 front 2024-03-23T18:09:15.573470+0000 (oldest deadline 2024-03-23T18:09:38.473559+0000) 2024-03-23T18:09:43.335 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:09:42 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T18:09:42.961+0000 7f722d382700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.458202+0000 front 2024-03-23T18:09:08.458278+0000 (oldest deadline 2024-03-23T18:09:33.158318+0000) 2024-03-23T18:09:43.336 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:09:42 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T18:09:42.961+0000 7f722d382700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:13.158888+0000 front 2024-03-23T18:09:13.158856+0000 (oldest deadline 2024-03-23T18:09:37.858881+0000) 2024-03-23T18:09:43.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:43 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:43.091+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.653471+0000 front 2024-03-23T18:09:00.653556+0000 (oldest deadline 2024-03-23T18:09:22.953547+0000) 2024-03-23T18:09:43.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:43 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:43.091+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.155613+0000 front 2024-03-23T18:09:08.155550+0000 (oldest deadline 2024-03-23T18:09:32.855633+0000) 2024-03-23T18:09:43.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:43 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:43.091+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.057340+0000 front 2024-03-23T18:09:15.057318+0000 (oldest deadline 2024-03-23T18:09:37.357408+0000) 2024-03-23T18:09:43.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:43 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:43.217+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:59.796052+0000 front 2024-03-23T18:08:59.796002+0000 (oldest deadline 2024-03-23T18:09:23.895742+0000) 2024-03-23T18:09:43.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:43 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:43.217+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.796583+0000 front 2024-03-23T18:09:06.796752+0000 (oldest deadline 2024-03-23T18:09:30.896916+0000) 2024-03-23T18:09:43.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:43 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:43.217+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.498831+0000 front 2024-03-23T18:09:15.498800+0000 (oldest deadline 2024-03-23T18:09:39.598988+0000) 2024-03-23T18:09:44.016 INFO:journalctl@ceph.osd.6.smithi086.stdout:Mar 23 18:09:43 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-6[74047]: 2024-03-23T18:09:43.551+0000 7f801c200700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:58.518283+0000 front 2024-03-23T18:08:58.518258+0000 (oldest deadline 2024-03-23T18:09:22.018402+0000) 2024-03-23T18:09:44.016 INFO:journalctl@ceph.osd.6.smithi086.stdout:Mar 23 18:09:43 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-6[74047]: 2024-03-23T18:09:43.551+0000 7f801c200700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.720334+0000 front 2024-03-23T18:09:06.720273+0000 (oldest deadline 2024-03-23T18:09:29.020358+0000) 2024-03-23T18:09:44.016 INFO:journalctl@ceph.osd.6.smithi086.stdout:Mar 23 18:09:43 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-6[74047]: 2024-03-23T18:09:43.551+0000 7f801c200700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:14.922410+0000 front 2024-03-23T18:09:14.922473+0000 (oldest deadline 2024-03-23T18:09:38.422603+0000) 2024-03-23T18:09:44.017 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:43 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:43.636+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.122328+0000 front 2024-03-23T18:09:00.122374+0000 (oldest deadline 2024-03-23T18:09:24.122945+0000) 2024-03-23T18:09:44.017 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:43 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:43.636+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:05.125484+0000 front 2024-03-23T18:09:05.125462+0000 (oldest deadline 2024-03-23T18:09:33.326776+0000) 2024-03-23T18:09:44.017 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:43 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:43.636+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:13.327558+0000 front 2024-03-23T18:09:13.327314+0000 (oldest deadline 2024-03-23T18:09:39.227552+0000) 2024-03-23T18:09:44.056 INFO:journalctl@ceph.osd.7.smithi092.stdout:Mar 23 18:09:43 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-7[100796]: 2024-03-23T18:09:43.616+0000 7fb53fa71700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.172261+0000 front 2024-03-23T18:09:06.172297+0000 (oldest deadline 2024-03-23T18:09:31.472456+0000) 2024-03-23T18:09:44.057 INFO:journalctl@ceph.osd.7.smithi092.stdout:Mar 23 18:09:43 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-7[100796]: 2024-03-23T18:09:43.616+0000 7fb53fa71700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.573435+0000 front 2024-03-23T18:09:15.573470+0000 (oldest deadline 2024-03-23T18:09:38.473559+0000) 2024-03-23T18:09:44.335 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:09:43 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T18:09:43.927+0000 7f722d382700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.458202+0000 front 2024-03-23T18:09:08.458278+0000 (oldest deadline 2024-03-23T18:09:33.158318+0000) 2024-03-23T18:09:44.336 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:09:43 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T18:09:43.927+0000 7f722d382700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:13.158888+0000 front 2024-03-23T18:09:13.158856+0000 (oldest deadline 2024-03-23T18:09:37.858881+0000) 2024-03-23T18:09:44.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:44 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:44.141+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.653471+0000 front 2024-03-23T18:09:00.653556+0000 (oldest deadline 2024-03-23T18:09:22.953547+0000) 2024-03-23T18:09:44.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:44 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:44.141+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.155613+0000 front 2024-03-23T18:09:08.155550+0000 (oldest deadline 2024-03-23T18:09:32.855633+0000) 2024-03-23T18:09:44.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:44 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:44.141+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.057340+0000 front 2024-03-23T18:09:15.057318+0000 (oldest deadline 2024-03-23T18:09:37.357408+0000) 2024-03-23T18:09:44.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:44 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:44.256+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:59.796052+0000 front 2024-03-23T18:08:59.796002+0000 (oldest deadline 2024-03-23T18:09:23.895742+0000) 2024-03-23T18:09:44.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:44 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:44.256+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.796583+0000 front 2024-03-23T18:09:06.796752+0000 (oldest deadline 2024-03-23T18:09:30.896916+0000) 2024-03-23T18:09:44.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:44 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:44.256+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.498831+0000 front 2024-03-23T18:09:15.498800+0000 (oldest deadline 2024-03-23T18:09:39.598988+0000) 2024-03-23T18:09:45.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:44 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:44.604+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.122328+0000 front 2024-03-23T18:09:00.122374+0000 (oldest deadline 2024-03-23T18:09:24.122945+0000) 2024-03-23T18:09:45.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:44 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:44.604+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:05.125484+0000 front 2024-03-23T18:09:05.125462+0000 (oldest deadline 2024-03-23T18:09:33.326776+0000) 2024-03-23T18:09:45.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:44 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:44.604+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:13.327558+0000 front 2024-03-23T18:09:13.327314+0000 (oldest deadline 2024-03-23T18:09:39.227552+0000) 2024-03-23T18:09:45.017 INFO:journalctl@ceph.osd.6.smithi086.stdout:Mar 23 18:09:44 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-6[74047]: 2024-03-23T18:09:44.539+0000 7f801c200700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:58.518283+0000 front 2024-03-23T18:08:58.518258+0000 (oldest deadline 2024-03-23T18:09:22.018402+0000) 2024-03-23T18:09:45.017 INFO:journalctl@ceph.osd.6.smithi086.stdout:Mar 23 18:09:44 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-6[74047]: 2024-03-23T18:09:44.539+0000 7f801c200700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.720334+0000 front 2024-03-23T18:09:06.720273+0000 (oldest deadline 2024-03-23T18:09:29.020358+0000) 2024-03-23T18:09:45.017 INFO:journalctl@ceph.osd.6.smithi086.stdout:Mar 23 18:09:44 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-6[74047]: 2024-03-23T18:09:44.539+0000 7f801c200700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:14.922410+0000 front 2024-03-23T18:09:14.922473+0000 (oldest deadline 2024-03-23T18:09:38.422603+0000) 2024-03-23T18:09:45.057 INFO:journalctl@ceph.osd.7.smithi092.stdout:Mar 23 18:09:44 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-7[100796]: 2024-03-23T18:09:44.590+0000 7fb53fa71700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.172261+0000 front 2024-03-23T18:09:06.172297+0000 (oldest deadline 2024-03-23T18:09:31.472456+0000) 2024-03-23T18:09:45.057 INFO:journalctl@ceph.osd.7.smithi092.stdout:Mar 23 18:09:44 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-7[100796]: 2024-03-23T18:09:44.590+0000 7fb53fa71700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.573435+0000 front 2024-03-23T18:09:15.573470+0000 (oldest deadline 2024-03-23T18:09:38.473559+0000) 2024-03-23T18:09:45.335 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:09:44 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T18:09:44.942+0000 7f722d382700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.458202+0000 front 2024-03-23T18:09:08.458278+0000 (oldest deadline 2024-03-23T18:09:33.158318+0000) 2024-03-23T18:09:45.336 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:09:44 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T18:09:44.942+0000 7f722d382700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:13.158888+0000 front 2024-03-23T18:09:13.158856+0000 (oldest deadline 2024-03-23T18:09:37.858881+0000) 2024-03-23T18:09:45.336 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:09:44 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T18:09:44.942+0000 7f722d382700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:21.359871+0000 front 2024-03-23T18:09:21.359947+0000 (oldest deadline 2024-03-23T18:09:44.259789+0000) 2024-03-23T18:09:45.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:45 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:45.116+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.653471+0000 front 2024-03-23T18:09:00.653556+0000 (oldest deadline 2024-03-23T18:09:22.953547+0000) 2024-03-23T18:09:45.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:45 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:45.116+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.155613+0000 front 2024-03-23T18:09:08.155550+0000 (oldest deadline 2024-03-23T18:09:32.855633+0000) 2024-03-23T18:09:45.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:45 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:45.116+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.057340+0000 front 2024-03-23T18:09:15.057318+0000 (oldest deadline 2024-03-23T18:09:37.357408+0000) 2024-03-23T18:09:45.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:45 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:45.255+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:59.796052+0000 front 2024-03-23T18:08:59.796002+0000 (oldest deadline 2024-03-23T18:09:23.895742+0000) 2024-03-23T18:09:45.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:45 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:45.255+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.796583+0000 front 2024-03-23T18:09:06.796752+0000 (oldest deadline 2024-03-23T18:09:30.896916+0000) 2024-03-23T18:09:45.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:45 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:45.255+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.498831+0000 front 2024-03-23T18:09:15.498800+0000 (oldest deadline 2024-03-23T18:09:39.598988+0000) 2024-03-23T18:09:46.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:45 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:45.573+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.122328+0000 front 2024-03-23T18:09:00.122374+0000 (oldest deadline 2024-03-23T18:09:24.122945+0000) 2024-03-23T18:09:46.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:45 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:45.573+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:05.125484+0000 front 2024-03-23T18:09:05.125462+0000 (oldest deadline 2024-03-23T18:09:33.326776+0000) 2024-03-23T18:09:46.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:45 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:45.573+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:13.327558+0000 front 2024-03-23T18:09:13.327314+0000 (oldest deadline 2024-03-23T18:09:39.227552+0000) 2024-03-23T18:09:46.017 INFO:journalctl@ceph.osd.6.smithi086.stdout:Mar 23 18:09:45 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-6[74047]: 2024-03-23T18:09:45.536+0000 7f801c200700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:58.518283+0000 front 2024-03-23T18:08:58.518258+0000 (oldest deadline 2024-03-23T18:09:22.018402+0000) 2024-03-23T18:09:46.017 INFO:journalctl@ceph.osd.6.smithi086.stdout:Mar 23 18:09:45 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-6[74047]: 2024-03-23T18:09:45.536+0000 7f801c200700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.720334+0000 front 2024-03-23T18:09:06.720273+0000 (oldest deadline 2024-03-23T18:09:29.020358+0000) 2024-03-23T18:09:46.017 INFO:journalctl@ceph.osd.6.smithi086.stdout:Mar 23 18:09:45 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-6[74047]: 2024-03-23T18:09:45.536+0000 7f801c200700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:14.922410+0000 front 2024-03-23T18:09:14.922473+0000 (oldest deadline 2024-03-23T18:09:38.422603+0000) 2024-03-23T18:09:46.017 INFO:journalctl@ceph.osd.6.smithi086.stdout:Mar 23 18:09:45 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-6[74047]: 2024-03-23T18:09:45.536+0000 7f801c200700 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:22.926280+0000 front 2024-03-23T18:09:22.926253+0000 (oldest deadline 2024-03-23T18:09:45.127325+0000) 2024-03-23T18:09:46.056 INFO:journalctl@ceph.osd.7.smithi092.stdout:Mar 23 18:09:45 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-7[100796]: 2024-03-23T18:09:45.608+0000 7fb53fa71700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.172261+0000 front 2024-03-23T18:09:06.172297+0000 (oldest deadline 2024-03-23T18:09:31.472456+0000) 2024-03-23T18:09:46.057 INFO:journalctl@ceph.osd.7.smithi092.stdout:Mar 23 18:09:45 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-7[100796]: 2024-03-23T18:09:45.608+0000 7fb53fa71700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.573435+0000 front 2024-03-23T18:09:15.573470+0000 (oldest deadline 2024-03-23T18:09:38.473559+0000) 2024-03-23T18:09:46.335 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:09:45 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T18:09:45.970+0000 7f722d382700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.458202+0000 front 2024-03-23T18:09:08.458278+0000 (oldest deadline 2024-03-23T18:09:33.158318+0000) 2024-03-23T18:09:46.336 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:09:45 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T18:09:45.970+0000 7f722d382700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:13.158888+0000 front 2024-03-23T18:09:13.158856+0000 (oldest deadline 2024-03-23T18:09:37.858881+0000) 2024-03-23T18:09:46.336 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:09:45 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T18:09:45.970+0000 7f722d382700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:21.359871+0000 front 2024-03-23T18:09:21.359947+0000 (oldest deadline 2024-03-23T18:09:44.259789+0000) 2024-03-23T18:09:46.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:46 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:46.107+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.653471+0000 front 2024-03-23T18:09:00.653556+0000 (oldest deadline 2024-03-23T18:09:22.953547+0000) 2024-03-23T18:09:46.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:46 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:46.108+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.155613+0000 front 2024-03-23T18:09:08.155550+0000 (oldest deadline 2024-03-23T18:09:32.855633+0000) 2024-03-23T18:09:46.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:46 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:46.108+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.057340+0000 front 2024-03-23T18:09:15.057318+0000 (oldest deadline 2024-03-23T18:09:37.357408+0000) 2024-03-23T18:09:46.516 INFO:journalctl@ceph.osd.6.smithi086.stdout:Mar 23 18:09:46 smithi086 bash[119273]: ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-6 2024-03-23T18:09:46.556 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:46 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:46.235+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:59.796052+0000 front 2024-03-23T18:08:59.796002+0000 (oldest deadline 2024-03-23T18:09:23.895742+0000) 2024-03-23T18:09:46.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:46 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:46.235+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.796583+0000 front 2024-03-23T18:09:06.796752+0000 (oldest deadline 2024-03-23T18:09:30.896916+0000) 2024-03-23T18:09:46.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:46 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:46.235+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.498831+0000 front 2024-03-23T18:09:15.498800+0000 (oldest deadline 2024-03-23T18:09:39.598988+0000) 2024-03-23T18:09:46.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:46 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:46.235+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:21.900315+0000 front 2024-03-23T18:09:21.900091+0000 (oldest deadline 2024-03-23T18:09:45.400070+0000) 2024-03-23T18:09:46.953 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:46 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:46.561+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.122328+0000 front 2024-03-23T18:09:00.122374+0000 (oldest deadline 2024-03-23T18:09:24.122945+0000) 2024-03-23T18:09:46.953 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:46 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:46.561+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:05.125484+0000 front 2024-03-23T18:09:05.125462+0000 (oldest deadline 2024-03-23T18:09:33.326776+0000) 2024-03-23T18:09:46.954 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:46 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:46.561+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:13.327558+0000 front 2024-03-23T18:09:13.327314+0000 (oldest deadline 2024-03-23T18:09:39.227552+0000) 2024-03-23T18:09:47.056 INFO:journalctl@ceph.osd.7.smithi092.stdout:Mar 23 18:09:46 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-7[100796]: 2024-03-23T18:09:46.566+0000 7fb53fa71700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.172261+0000 front 2024-03-23T18:09:06.172297+0000 (oldest deadline 2024-03-23T18:09:31.472456+0000) 2024-03-23T18:09:47.057 INFO:journalctl@ceph.osd.7.smithi092.stdout:Mar 23 18:09:46 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-7[100796]: 2024-03-23T18:09:46.566+0000 7fb53fa71700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.573435+0000 front 2024-03-23T18:09:15.573470+0000 (oldest deadline 2024-03-23T18:09:38.473559+0000) 2024-03-23T18:09:47.335 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:09:46 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T18:09:46.930+0000 7f722d382700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.458202+0000 front 2024-03-23T18:09:08.458278+0000 (oldest deadline 2024-03-23T18:09:33.158318+0000) 2024-03-23T18:09:47.336 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:09:46 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T18:09:46.930+0000 7f722d382700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:13.158888+0000 front 2024-03-23T18:09:13.158856+0000 (oldest deadline 2024-03-23T18:09:37.858881+0000) 2024-03-23T18:09:47.336 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:09:46 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T18:09:46.930+0000 7f722d382700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:21.359871+0000 front 2024-03-23T18:09:21.359947+0000 (oldest deadline 2024-03-23T18:09:44.259789+0000) 2024-03-23T18:09:47.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:47 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:47.140+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.653471+0000 front 2024-03-23T18:09:00.653556+0000 (oldest deadline 2024-03-23T18:09:22.953547+0000) 2024-03-23T18:09:47.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:47 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:47.140+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.155613+0000 front 2024-03-23T18:09:08.155550+0000 (oldest deadline 2024-03-23T18:09:32.855633+0000) 2024-03-23T18:09:47.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:47 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:47.140+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.057340+0000 front 2024-03-23T18:09:15.057318+0000 (oldest deadline 2024-03-23T18:09:37.357408+0000) 2024-03-23T18:09:47.337 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:47 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:47.140+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:20.858463+0000 front 2024-03-23T18:09:20.858478+0000 (oldest deadline 2024-03-23T18:09:46.758574+0000) 2024-03-23T18:09:47.537 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:47 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:47.241+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:59.796052+0000 front 2024-03-23T18:08:59.796002+0000 (oldest deadline 2024-03-23T18:09:23.895742+0000) 2024-03-23T18:09:47.538 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:47 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:47.241+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.796583+0000 front 2024-03-23T18:09:06.796752+0000 (oldest deadline 2024-03-23T18:09:30.896916+0000) 2024-03-23T18:09:47.538 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:47 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:47.241+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.498831+0000 front 2024-03-23T18:09:15.498800+0000 (oldest deadline 2024-03-23T18:09:39.598988+0000) 2024-03-23T18:09:47.538 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:47 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:47.241+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:21.900315+0000 front 2024-03-23T18:09:21.900091+0000 (oldest deadline 2024-03-23T18:09:45.400070+0000) 2024-03-23T18:09:47.766 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:47 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:47.563+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.122328+0000 front 2024-03-23T18:09:00.122374+0000 (oldest deadline 2024-03-23T18:09:24.122945+0000) 2024-03-23T18:09:47.766 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:47 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:47.563+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:05.125484+0000 front 2024-03-23T18:09:05.125462+0000 (oldest deadline 2024-03-23T18:09:33.326776+0000) 2024-03-23T18:09:47.766 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:47 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:47.563+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:13.327558+0000 front 2024-03-23T18:09:13.327314+0000 (oldest deadline 2024-03-23T18:09:39.227552+0000) 2024-03-23T18:09:47.806 INFO:journalctl@ceph.osd.7.smithi092.stdout:Mar 23 18:09:47 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-7[100796]: 2024-03-23T18:09:47.537+0000 7fb53fa71700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.172261+0000 front 2024-03-23T18:09:06.172297+0000 (oldest deadline 2024-03-23T18:09:31.472456+0000) 2024-03-23T18:09:47.806 INFO:journalctl@ceph.osd.7.smithi092.stdout:Mar 23 18:09:47 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-7[100796]: 2024-03-23T18:09:47.537+0000 7fb53fa71700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.573435+0000 front 2024-03-23T18:09:15.573470+0000 (oldest deadline 2024-03-23T18:09:38.473559+0000) 2024-03-23T18:09:47.807 INFO:journalctl@ceph.osd.7.smithi092.stdout:Mar 23 18:09:47 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-7[100796]: 2024-03-23T18:09:47.537+0000 7fb53fa71700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:20.675053+0000 front 2024-03-23T18:09:20.675172+0000 (oldest deadline 2024-03-23T18:09:46.575221+0000) 2024-03-23T18:09:47.885 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@osd.6.service' 2024-03-23T18:09:47.971 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T18:09:47.971 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-03-23T18:09:47.972 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-03-23T18:09:47.972 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@osd.7 2024-03-23T18:09:48.307 INFO:journalctl@ceph.osd.7.smithi092.stdout:Mar 23 18:09:48 smithi092 systemd[1]: Stopping Ceph osd.7 for 9d76c9e8-e93c-11ee-95cd-87774f69a715... 2024-03-23T18:09:48.307 INFO:journalctl@ceph.osd.7.smithi092.stdout:Mar 23 18:09:48 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-7[100796]: 2024-03-23T18:09:48.200+0000 7fb542c7a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T18:09:48.307 INFO:journalctl@ceph.osd.7.smithi092.stdout:Mar 23 18:09:48 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-7[100796]: 2024-03-23T18:09:48.200+0000 7fb542c7a700 -1 osd.7 113 *** Got signal Terminated *** 2024-03-23T18:09:48.307 INFO:journalctl@ceph.osd.7.smithi092.stdout:Mar 23 18:09:48 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-7[100796]: 2024-03-23T18:09:48.200+0000 7fb542c7a700 -1 osd.7 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-23T18:09:48.308 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:48 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:48.251+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:59.796052+0000 front 2024-03-23T18:08:59.796002+0000 (oldest deadline 2024-03-23T18:09:23.895742+0000) 2024-03-23T18:09:48.308 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:48 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:48.251+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.796583+0000 front 2024-03-23T18:09:06.796752+0000 (oldest deadline 2024-03-23T18:09:30.896916+0000) 2024-03-23T18:09:48.308 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:48 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:48.252+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.498831+0000 front 2024-03-23T18:09:15.498800+0000 (oldest deadline 2024-03-23T18:09:39.598988+0000) 2024-03-23T18:09:48.308 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:48 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:48.252+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:21.900315+0000 front 2024-03-23T18:09:21.900091+0000 (oldest deadline 2024-03-23T18:09:45.400070+0000) 2024-03-23T18:09:48.335 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:09:47 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T18:09:47.936+0000 7f722d382700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.458202+0000 front 2024-03-23T18:09:08.458278+0000 (oldest deadline 2024-03-23T18:09:33.158318+0000) 2024-03-23T18:09:48.335 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:09:47 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T18:09:47.936+0000 7f722d382700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:13.158888+0000 front 2024-03-23T18:09:13.158856+0000 (oldest deadline 2024-03-23T18:09:37.858881+0000) 2024-03-23T18:09:48.336 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:09:47 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T18:09:47.936+0000 7f722d382700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:21.359871+0000 front 2024-03-23T18:09:21.359947+0000 (oldest deadline 2024-03-23T18:09:44.259789+0000) 2024-03-23T18:09:48.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:48 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:48.173+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.653471+0000 front 2024-03-23T18:09:00.653556+0000 (oldest deadline 2024-03-23T18:09:22.953547+0000) 2024-03-23T18:09:48.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:48 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:48.173+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.155613+0000 front 2024-03-23T18:09:08.155550+0000 (oldest deadline 2024-03-23T18:09:32.855633+0000) 2024-03-23T18:09:48.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:48 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:48.173+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.057340+0000 front 2024-03-23T18:09:15.057318+0000 (oldest deadline 2024-03-23T18:09:37.357408+0000) 2024-03-23T18:09:48.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:48 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:48.173+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:20.858463+0000 front 2024-03-23T18:09:20.858478+0000 (oldest deadline 2024-03-23T18:09:46.758574+0000) 2024-03-23T18:09:48.807 INFO:journalctl@ceph.osd.7.smithi092.stdout:Mar 23 18:09:48 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-7[100796]: 2024-03-23T18:09:48.548+0000 7fb53fa71700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.172261+0000 front 2024-03-23T18:09:06.172297+0000 (oldest deadline 2024-03-23T18:09:31.472456+0000) 2024-03-23T18:09:48.807 INFO:journalctl@ceph.osd.7.smithi092.stdout:Mar 23 18:09:48 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-7[100796]: 2024-03-23T18:09:48.548+0000 7fb53fa71700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.573435+0000 front 2024-03-23T18:09:15.573470+0000 (oldest deadline 2024-03-23T18:09:38.473559+0000) 2024-03-23T18:09:48.807 INFO:journalctl@ceph.osd.7.smithi092.stdout:Mar 23 18:09:48 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-7[100796]: 2024-03-23T18:09:48.548+0000 7fb53fa71700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:20.675053+0000 front 2024-03-23T18:09:20.675172+0000 (oldest deadline 2024-03-23T18:09:46.575221+0000) 2024-03-23T18:09:49.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:48 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:48.613+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.122328+0000 front 2024-03-23T18:09:00.122374+0000 (oldest deadline 2024-03-23T18:09:24.122945+0000) 2024-03-23T18:09:49.017 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:48 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:48.613+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:05.125484+0000 front 2024-03-23T18:09:05.125462+0000 (oldest deadline 2024-03-23T18:09:33.326776+0000) 2024-03-23T18:09:49.017 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:48 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:48.613+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:13.327558+0000 front 2024-03-23T18:09:13.327314+0000 (oldest deadline 2024-03-23T18:09:39.227552+0000) 2024-03-23T18:09:49.335 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:09:48 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T18:09:48.927+0000 7f722d382700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.458202+0000 front 2024-03-23T18:09:08.458278+0000 (oldest deadline 2024-03-23T18:09:33.158318+0000) 2024-03-23T18:09:49.336 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:09:48 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T18:09:48.927+0000 7f722d382700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:13.158888+0000 front 2024-03-23T18:09:13.158856+0000 (oldest deadline 2024-03-23T18:09:37.858881+0000) 2024-03-23T18:09:49.336 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:09:48 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T18:09:48.927+0000 7f722d382700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:21.359871+0000 front 2024-03-23T18:09:21.359947+0000 (oldest deadline 2024-03-23T18:09:44.259789+0000) 2024-03-23T18:09:49.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:49 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:49.160+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.653471+0000 front 2024-03-23T18:09:00.653556+0000 (oldest deadline 2024-03-23T18:09:22.953547+0000) 2024-03-23T18:09:49.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:49 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:49.160+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.155613+0000 front 2024-03-23T18:09:08.155550+0000 (oldest deadline 2024-03-23T18:09:32.855633+0000) 2024-03-23T18:09:49.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:49 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:49.160+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.057340+0000 front 2024-03-23T18:09:15.057318+0000 (oldest deadline 2024-03-23T18:09:37.357408+0000) 2024-03-23T18:09:49.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:49 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:49.160+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:20.858463+0000 front 2024-03-23T18:09:20.858478+0000 (oldest deadline 2024-03-23T18:09:46.758574+0000) 2024-03-23T18:09:49.513 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:49 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:49.233+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:59.796052+0000 front 2024-03-23T18:08:59.796002+0000 (oldest deadline 2024-03-23T18:09:23.895742+0000) 2024-03-23T18:09:49.514 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:49 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:49.233+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.796583+0000 front 2024-03-23T18:09:06.796752+0000 (oldest deadline 2024-03-23T18:09:30.896916+0000) 2024-03-23T18:09:49.514 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:49 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:49.233+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.498831+0000 front 2024-03-23T18:09:15.498800+0000 (oldest deadline 2024-03-23T18:09:39.598988+0000) 2024-03-23T18:09:49.514 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:49 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:49.233+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:21.900315+0000 front 2024-03-23T18:09:21.900091+0000 (oldest deadline 2024-03-23T18:09:45.400070+0000) 2024-03-23T18:09:49.807 INFO:journalctl@ceph.osd.7.smithi092.stdout:Mar 23 18:09:49 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-7[100796]: 2024-03-23T18:09:49.514+0000 7fb53fa71700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.172261+0000 front 2024-03-23T18:09:06.172297+0000 (oldest deadline 2024-03-23T18:09:31.472456+0000) 2024-03-23T18:09:49.807 INFO:journalctl@ceph.osd.7.smithi092.stdout:Mar 23 18:09:49 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-7[100796]: 2024-03-23T18:09:49.514+0000 7fb53fa71700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.573435+0000 front 2024-03-23T18:09:15.573470+0000 (oldest deadline 2024-03-23T18:09:38.473559+0000) 2024-03-23T18:09:49.807 INFO:journalctl@ceph.osd.7.smithi092.stdout:Mar 23 18:09:49 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-7[100796]: 2024-03-23T18:09:49.514+0000 7fb53fa71700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:20.675053+0000 front 2024-03-23T18:09:20.675172+0000 (oldest deadline 2024-03-23T18:09:46.575221+0000) 2024-03-23T18:09:50.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:49 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:49.643+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.122328+0000 front 2024-03-23T18:09:00.122374+0000 (oldest deadline 2024-03-23T18:09:24.122945+0000) 2024-03-23T18:09:50.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:49 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:49.643+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:05.125484+0000 front 2024-03-23T18:09:05.125462+0000 (oldest deadline 2024-03-23T18:09:33.326776+0000) 2024-03-23T18:09:50.017 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:49 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:49.643+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:13.327558+0000 front 2024-03-23T18:09:13.327314+0000 (oldest deadline 2024-03-23T18:09:39.227552+0000) 2024-03-23T18:09:50.017 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:49 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:49.643+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:23.129785+0000 front 2024-03-23T18:09:23.129607+0000 (oldest deadline 2024-03-23T18:09:49.029801+0000) 2024-03-23T18:09:50.202 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:09:49 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T18:09:49.921+0000 7f722d382700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.458202+0000 front 2024-03-23T18:09:08.458278+0000 (oldest deadline 2024-03-23T18:09:33.158318+0000) 2024-03-23T18:09:50.202 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:09:49 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T18:09:49.921+0000 7f722d382700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:13.158888+0000 front 2024-03-23T18:09:13.158856+0000 (oldest deadline 2024-03-23T18:09:37.858881+0000) 2024-03-23T18:09:50.202 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:09:49 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T18:09:49.921+0000 7f722d382700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:21.359871+0000 front 2024-03-23T18:09:21.359947+0000 (oldest deadline 2024-03-23T18:09:44.259789+0000) 2024-03-23T18:09:50.554 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:50 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:50.202+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:59.796052+0000 front 2024-03-23T18:08:59.796002+0000 (oldest deadline 2024-03-23T18:09:23.895742+0000) 2024-03-23T18:09:50.555 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:50 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:50.202+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.796583+0000 front 2024-03-23T18:09:06.796752+0000 (oldest deadline 2024-03-23T18:09:30.896916+0000) 2024-03-23T18:09:50.555 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:50 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:50.202+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.498831+0000 front 2024-03-23T18:09:15.498800+0000 (oldest deadline 2024-03-23T18:09:39.598988+0000) 2024-03-23T18:09:50.555 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:50 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:50.202+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:21.900315+0000 front 2024-03-23T18:09:21.900091+0000 (oldest deadline 2024-03-23T18:09:45.400070+0000) 2024-03-23T18:09:50.585 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:50 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:50.203+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.653471+0000 front 2024-03-23T18:09:00.653556+0000 (oldest deadline 2024-03-23T18:09:22.953547+0000) 2024-03-23T18:09:50.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:50 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:50.203+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.155613+0000 front 2024-03-23T18:09:08.155550+0000 (oldest deadline 2024-03-23T18:09:32.855633+0000) 2024-03-23T18:09:50.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:50 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:50.203+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.057340+0000 front 2024-03-23T18:09:15.057318+0000 (oldest deadline 2024-03-23T18:09:37.357408+0000) 2024-03-23T18:09:50.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:50 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:50.203+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:20.858463+0000 front 2024-03-23T18:09:20.858478+0000 (oldest deadline 2024-03-23T18:09:46.758574+0000) 2024-03-23T18:09:50.806 INFO:journalctl@ceph.osd.7.smithi092.stdout:Mar 23 18:09:50 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-7[100796]: 2024-03-23T18:09:50.554+0000 7fb53fa71700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.172261+0000 front 2024-03-23T18:09:06.172297+0000 (oldest deadline 2024-03-23T18:09:31.472456+0000) 2024-03-23T18:09:50.807 INFO:journalctl@ceph.osd.7.smithi092.stdout:Mar 23 18:09:50 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-7[100796]: 2024-03-23T18:09:50.554+0000 7fb53fa71700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.573435+0000 front 2024-03-23T18:09:15.573470+0000 (oldest deadline 2024-03-23T18:09:38.473559+0000) 2024-03-23T18:09:50.807 INFO:journalctl@ceph.osd.7.smithi092.stdout:Mar 23 18:09:50 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-7[100796]: 2024-03-23T18:09:50.554+0000 7fb53fa71700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:20.675053+0000 front 2024-03-23T18:09:20.675172+0000 (oldest deadline 2024-03-23T18:09:46.575221+0000) 2024-03-23T18:09:51.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:50 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:50.669+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.122328+0000 front 2024-03-23T18:09:00.122374+0000 (oldest deadline 2024-03-23T18:09:24.122945+0000) 2024-03-23T18:09:51.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:50 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:50.669+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:05.125484+0000 front 2024-03-23T18:09:05.125462+0000 (oldest deadline 2024-03-23T18:09:33.326776+0000) 2024-03-23T18:09:51.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:50 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:50.669+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:13.327558+0000 front 2024-03-23T18:09:13.327314+0000 (oldest deadline 2024-03-23T18:09:39.227552+0000) 2024-03-23T18:09:51.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:50 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:50.669+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:23.129785+0000 front 2024-03-23T18:09:23.129607+0000 (oldest deadline 2024-03-23T18:09:49.029801+0000) 2024-03-23T18:09:51.185 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:09:50 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T18:09:50.909+0000 7f722d382700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.458202+0000 front 2024-03-23T18:09:08.458278+0000 (oldest deadline 2024-03-23T18:09:33.158318+0000) 2024-03-23T18:09:51.186 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:09:50 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T18:09:50.909+0000 7f722d382700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:13.158888+0000 front 2024-03-23T18:09:13.158856+0000 (oldest deadline 2024-03-23T18:09:37.858881+0000) 2024-03-23T18:09:51.186 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:09:50 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T18:09:50.909+0000 7f722d382700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:21.359871+0000 front 2024-03-23T18:09:21.359947+0000 (oldest deadline 2024-03-23T18:09:44.259789+0000) 2024-03-23T18:09:51.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:51 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:51.198+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:59.796052+0000 front 2024-03-23T18:08:59.796002+0000 (oldest deadline 2024-03-23T18:09:23.895742+0000) 2024-03-23T18:09:51.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:51 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:51.198+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.796583+0000 front 2024-03-23T18:09:06.796752+0000 (oldest deadline 2024-03-23T18:09:30.896916+0000) 2024-03-23T18:09:51.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:51 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:51.198+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.498831+0000 front 2024-03-23T18:09:15.498800+0000 (oldest deadline 2024-03-23T18:09:39.598988+0000) 2024-03-23T18:09:51.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:51 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:51.198+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:21.900315+0000 front 2024-03-23T18:09:21.900091+0000 (oldest deadline 2024-03-23T18:09:45.400070+0000) 2024-03-23T18:09:51.585 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:51 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:51.186+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.653471+0000 front 2024-03-23T18:09:00.653556+0000 (oldest deadline 2024-03-23T18:09:22.953547+0000) 2024-03-23T18:09:51.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:51 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:51.186+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.155613+0000 front 2024-03-23T18:09:08.155550+0000 (oldest deadline 2024-03-23T18:09:32.855633+0000) 2024-03-23T18:09:51.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:51 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:51.186+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.057340+0000 front 2024-03-23T18:09:15.057318+0000 (oldest deadline 2024-03-23T18:09:37.357408+0000) 2024-03-23T18:09:51.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:51 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:51.186+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:20.858463+0000 front 2024-03-23T18:09:20.858478+0000 (oldest deadline 2024-03-23T18:09:46.758574+0000) 2024-03-23T18:09:52.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:51 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:51.681+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.122328+0000 front 2024-03-23T18:09:00.122374+0000 (oldest deadline 2024-03-23T18:09:24.122945+0000) 2024-03-23T18:09:52.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:51 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:51.681+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:05.125484+0000 front 2024-03-23T18:09:05.125462+0000 (oldest deadline 2024-03-23T18:09:33.326776+0000) 2024-03-23T18:09:52.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:51 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:51.681+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:13.327558+0000 front 2024-03-23T18:09:13.327314+0000 (oldest deadline 2024-03-23T18:09:39.227552+0000) 2024-03-23T18:09:52.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:51 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:51.681+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:23.129785+0000 front 2024-03-23T18:09:23.129607+0000 (oldest deadline 2024-03-23T18:09:49.029801+0000) 2024-03-23T18:09:52.057 INFO:journalctl@ceph.osd.7.smithi092.stdout:Mar 23 18:09:51 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-7[100796]: 2024-03-23T18:09:51.559+0000 7fb53fa71700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.172261+0000 front 2024-03-23T18:09:06.172297+0000 (oldest deadline 2024-03-23T18:09:31.472456+0000) 2024-03-23T18:09:52.057 INFO:journalctl@ceph.osd.7.smithi092.stdout:Mar 23 18:09:51 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-7[100796]: 2024-03-23T18:09:51.559+0000 7fb53fa71700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.573435+0000 front 2024-03-23T18:09:15.573470+0000 (oldest deadline 2024-03-23T18:09:38.473559+0000) 2024-03-23T18:09:52.057 INFO:journalctl@ceph.osd.7.smithi092.stdout:Mar 23 18:09:51 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-7[100796]: 2024-03-23T18:09:51.559+0000 7fb53fa71700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:20.675053+0000 front 2024-03-23T18:09:20.675172+0000 (oldest deadline 2024-03-23T18:09:46.575221+0000) 2024-03-23T18:09:52.151 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:09:51 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T18:09:51.881+0000 7f722d382700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.458202+0000 front 2024-03-23T18:09:08.458278+0000 (oldest deadline 2024-03-23T18:09:33.158318+0000) 2024-03-23T18:09:52.151 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:09:51 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T18:09:51.881+0000 7f722d382700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:13.158888+0000 front 2024-03-23T18:09:13.158856+0000 (oldest deadline 2024-03-23T18:09:37.858881+0000) 2024-03-23T18:09:52.151 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:09:51 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T18:09:51.881+0000 7f722d382700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:21.359871+0000 front 2024-03-23T18:09:21.359947+0000 (oldest deadline 2024-03-23T18:09:44.259789+0000) 2024-03-23T18:09:52.517 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:52 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:52.186+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:59.796052+0000 front 2024-03-23T18:08:59.796002+0000 (oldest deadline 2024-03-23T18:09:23.895742+0000) 2024-03-23T18:09:52.517 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:52 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:52.186+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.796583+0000 front 2024-03-23T18:09:06.796752+0000 (oldest deadline 2024-03-23T18:09:30.896916+0000) 2024-03-23T18:09:52.517 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:52 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:52.186+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.498831+0000 front 2024-03-23T18:09:15.498800+0000 (oldest deadline 2024-03-23T18:09:39.598988+0000) 2024-03-23T18:09:52.518 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:52 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:52.186+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:21.900315+0000 front 2024-03-23T18:09:21.900091+0000 (oldest deadline 2024-03-23T18:09:45.400070+0000) 2024-03-23T18:09:52.585 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:52 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:52.151+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.653471+0000 front 2024-03-23T18:09:00.653556+0000 (oldest deadline 2024-03-23T18:09:22.953547+0000) 2024-03-23T18:09:52.585 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:52 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:52.151+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.155613+0000 front 2024-03-23T18:09:08.155550+0000 (oldest deadline 2024-03-23T18:09:32.855633+0000) 2024-03-23T18:09:52.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:52 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:52.151+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.057340+0000 front 2024-03-23T18:09:15.057318+0000 (oldest deadline 2024-03-23T18:09:37.357408+0000) 2024-03-23T18:09:52.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:52 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:52.151+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:20.858463+0000 front 2024-03-23T18:09:20.858478+0000 (oldest deadline 2024-03-23T18:09:46.758574+0000) 2024-03-23T18:09:52.807 INFO:journalctl@ceph.osd.7.smithi092.stdout:Mar 23 18:09:52 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-7[100796]: 2024-03-23T18:09:52.518+0000 7fb53fa71700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.172261+0000 front 2024-03-23T18:09:06.172297+0000 (oldest deadline 2024-03-23T18:09:31.472456+0000) 2024-03-23T18:09:52.807 INFO:journalctl@ceph.osd.7.smithi092.stdout:Mar 23 18:09:52 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-7[100796]: 2024-03-23T18:09:52.518+0000 7fb53fa71700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.573435+0000 front 2024-03-23T18:09:15.573470+0000 (oldest deadline 2024-03-23T18:09:38.473559+0000) 2024-03-23T18:09:52.807 INFO:journalctl@ceph.osd.7.smithi092.stdout:Mar 23 18:09:52 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-7[100796]: 2024-03-23T18:09:52.518+0000 7fb53fa71700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:20.675053+0000 front 2024-03-23T18:09:20.675172+0000 (oldest deadline 2024-03-23T18:09:46.575221+0000) 2024-03-23T18:09:52.807 INFO:journalctl@ceph.osd.7.smithi092.stdout:Mar 23 18:09:52 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-7[100796]: 2024-03-23T18:09:52.518+0000 7fb53fa71700 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-03-23T18:09:26.575851+0000 front 2024-03-23T18:09:26.575865+0000 (oldest deadline 2024-03-23T18:09:52.475985+0000) 2024-03-23T18:09:53.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:52 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:52.643+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.122328+0000 front 2024-03-23T18:09:00.122374+0000 (oldest deadline 2024-03-23T18:09:24.122945+0000) 2024-03-23T18:09:53.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:52 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:52.643+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:05.125484+0000 front 2024-03-23T18:09:05.125462+0000 (oldest deadline 2024-03-23T18:09:33.326776+0000) 2024-03-23T18:09:53.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:52 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:52.643+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:13.327558+0000 front 2024-03-23T18:09:13.327314+0000 (oldest deadline 2024-03-23T18:09:39.227552+0000) 2024-03-23T18:09:53.017 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:52 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:52.643+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:23.129785+0000 front 2024-03-23T18:09:23.129607+0000 (oldest deadline 2024-03-23T18:09:49.029801+0000) 2024-03-23T18:09:53.335 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:09:52 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T18:09:52.921+0000 7f722d382700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.458202+0000 front 2024-03-23T18:09:08.458278+0000 (oldest deadline 2024-03-23T18:09:33.158318+0000) 2024-03-23T18:09:53.336 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:09:52 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T18:09:52.921+0000 7f722d382700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:13.158888+0000 front 2024-03-23T18:09:13.158856+0000 (oldest deadline 2024-03-23T18:09:37.858881+0000) 2024-03-23T18:09:53.336 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:09:52 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T18:09:52.921+0000 7f722d382700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:21.359871+0000 front 2024-03-23T18:09:21.359947+0000 (oldest deadline 2024-03-23T18:09:44.259789+0000) 2024-03-23T18:09:53.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:53 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:53.170+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.653471+0000 front 2024-03-23T18:09:00.653556+0000 (oldest deadline 2024-03-23T18:09:22.953547+0000) 2024-03-23T18:09:53.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:53 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:53.170+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.155613+0000 front 2024-03-23T18:09:08.155550+0000 (oldest deadline 2024-03-23T18:09:32.855633+0000) 2024-03-23T18:09:53.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:53 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:53.170+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.057340+0000 front 2024-03-23T18:09:15.057318+0000 (oldest deadline 2024-03-23T18:09:37.357408+0000) 2024-03-23T18:09:53.337 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:53 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:53.170+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:20.858463+0000 front 2024-03-23T18:09:20.858478+0000 (oldest deadline 2024-03-23T18:09:46.758574+0000) 2024-03-23T18:09:53.337 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:53 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:53.170+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-03-23T18:09:26.759276+0000 front 2024-03-23T18:09:26.759351+0000 (oldest deadline 2024-03-23T18:09:52.659216+0000) 2024-03-23T18:09:53.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:53 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:53.164+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:59.796052+0000 front 2024-03-23T18:08:59.796002+0000 (oldest deadline 2024-03-23T18:09:23.895742+0000) 2024-03-23T18:09:53.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:53 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:53.164+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.796583+0000 front 2024-03-23T18:09:06.796752+0000 (oldest deadline 2024-03-23T18:09:30.896916+0000) 2024-03-23T18:09:53.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:53 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:53.164+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.498831+0000 front 2024-03-23T18:09:15.498800+0000 (oldest deadline 2024-03-23T18:09:39.598988+0000) 2024-03-23T18:09:53.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:53 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:53.164+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:21.900315+0000 front 2024-03-23T18:09:21.900091+0000 (oldest deadline 2024-03-23T18:09:45.400070+0000) 2024-03-23T18:09:53.558 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:53 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:53.164+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-03-23T18:09:29.501333+0000 front 2024-03-23T18:09:29.501303+0000 (oldest deadline 2024-03-23T18:09:53.001492+0000) 2024-03-23T18:09:54.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:53 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:53.594+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.122328+0000 front 2024-03-23T18:09:00.122374+0000 (oldest deadline 2024-03-23T18:09:24.122945+0000) 2024-03-23T18:09:54.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:53 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:53.594+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:05.125484+0000 front 2024-03-23T18:09:05.125462+0000 (oldest deadline 2024-03-23T18:09:33.326776+0000) 2024-03-23T18:09:54.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:53 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:53.594+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:13.327558+0000 front 2024-03-23T18:09:13.327314+0000 (oldest deadline 2024-03-23T18:09:39.227552+0000) 2024-03-23T18:09:54.017 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:53 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:53.594+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:23.129785+0000 front 2024-03-23T18:09:23.129607+0000 (oldest deadline 2024-03-23T18:09:49.029801+0000) 2024-03-23T18:09:54.131 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:09:53 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T18:09:53.877+0000 7f722d382700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.458202+0000 front 2024-03-23T18:09:08.458278+0000 (oldest deadline 2024-03-23T18:09:33.158318+0000) 2024-03-23T18:09:54.131 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:09:53 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T18:09:53.877+0000 7f722d382700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:13.158888+0000 front 2024-03-23T18:09:13.158856+0000 (oldest deadline 2024-03-23T18:09:37.858881+0000) 2024-03-23T18:09:54.131 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:09:53 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T18:09:53.877+0000 7f722d382700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:21.359871+0000 front 2024-03-23T18:09:21.359947+0000 (oldest deadline 2024-03-23T18:09:44.259789+0000) 2024-03-23T18:09:54.131 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:09:53 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T18:09:53.877+0000 7f722d382700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-03-23T18:09:30.061831+0000 front 2024-03-23T18:09:30.061844+0000 (oldest deadline 2024-03-23T18:09:53.561913+0000) 2024-03-23T18:09:54.200 INFO:journalctl@ceph.osd.7.smithi092.stdout:Mar 23 18:09:53 smithi092 bash[123391]: ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-7 2024-03-23T18:09:54.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:54 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:54.200+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:59.796052+0000 front 2024-03-23T18:08:59.796002+0000 (oldest deadline 2024-03-23T18:09:23.895742+0000) 2024-03-23T18:09:54.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:54 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:54.200+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.796583+0000 front 2024-03-23T18:09:06.796752+0000 (oldest deadline 2024-03-23T18:09:30.896916+0000) 2024-03-23T18:09:54.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:54 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:54.200+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.498831+0000 front 2024-03-23T18:09:15.498800+0000 (oldest deadline 2024-03-23T18:09:39.598988+0000) 2024-03-23T18:09:54.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:54 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:54.200+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:21.900315+0000 front 2024-03-23T18:09:21.900091+0000 (oldest deadline 2024-03-23T18:09:45.400070+0000) 2024-03-23T18:09:54.558 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:54 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:54.200+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-03-23T18:09:29.501333+0000 front 2024-03-23T18:09:29.501303+0000 (oldest deadline 2024-03-23T18:09:53.001492+0000) 2024-03-23T18:09:54.585 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:54 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:54.132+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.653471+0000 front 2024-03-23T18:09:00.653556+0000 (oldest deadline 2024-03-23T18:09:22.953547+0000) 2024-03-23T18:09:54.585 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:54 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:54.132+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.155613+0000 front 2024-03-23T18:09:08.155550+0000 (oldest deadline 2024-03-23T18:09:32.855633+0000) 2024-03-23T18:09:54.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:54 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:54.132+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.057340+0000 front 2024-03-23T18:09:15.057318+0000 (oldest deadline 2024-03-23T18:09:37.357408+0000) 2024-03-23T18:09:54.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:54 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:54.132+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:20.858463+0000 front 2024-03-23T18:09:20.858478+0000 (oldest deadline 2024-03-23T18:09:46.758574+0000) 2024-03-23T18:09:54.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:54 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:54.132+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-03-23T18:09:26.759276+0000 front 2024-03-23T18:09:26.759351+0000 (oldest deadline 2024-03-23T18:09:52.659216+0000) 2024-03-23T18:09:55.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:54 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:54.633+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.122328+0000 front 2024-03-23T18:09:00.122374+0000 (oldest deadline 2024-03-23T18:09:24.122945+0000) 2024-03-23T18:09:55.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:54 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:54.633+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:05.125484+0000 front 2024-03-23T18:09:05.125462+0000 (oldest deadline 2024-03-23T18:09:33.326776+0000) 2024-03-23T18:09:55.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:54 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:54.633+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:13.327558+0000 front 2024-03-23T18:09:13.327314+0000 (oldest deadline 2024-03-23T18:09:39.227552+0000) 2024-03-23T18:09:55.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:54 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:54.633+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:23.129785+0000 front 2024-03-23T18:09:23.129607+0000 (oldest deadline 2024-03-23T18:09:49.029801+0000) 2024-03-23T18:09:55.085 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:09:54 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T18:09:54.832+0000 7f722d382700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.458202+0000 front 2024-03-23T18:09:08.458278+0000 (oldest deadline 2024-03-23T18:09:33.158318+0000) 2024-03-23T18:09:55.086 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:09:54 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T18:09:54.832+0000 7f722d382700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:13.158888+0000 front 2024-03-23T18:09:13.158856+0000 (oldest deadline 2024-03-23T18:09:37.858881+0000) 2024-03-23T18:09:55.086 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:09:54 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T18:09:54.832+0000 7f722d382700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:21.359871+0000 front 2024-03-23T18:09:21.359947+0000 (oldest deadline 2024-03-23T18:09:44.259789+0000) 2024-03-23T18:09:55.086 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:09:54 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T18:09:54.832+0000 7f722d382700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-03-23T18:09:30.061831+0000 front 2024-03-23T18:09:30.061844+0000 (oldest deadline 2024-03-23T18:09:53.561913+0000) 2024-03-23T18:09:55.112 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@osd.7.service' 2024-03-23T18:09:55.151 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T18:09:55.152 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-03-23T18:09:55.152 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-03-23T18:09:55.152 DEBUG:teuthology.orchestra.run.smithi152:> sudo systemctl stop ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@osd.8 2024-03-23T18:09:55.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:55 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:55.172+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:59.796052+0000 front 2024-03-23T18:08:59.796002+0000 (oldest deadline 2024-03-23T18:09:23.895742+0000) 2024-03-23T18:09:55.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:55 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:55.172+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.796583+0000 front 2024-03-23T18:09:06.796752+0000 (oldest deadline 2024-03-23T18:09:30.896916+0000) 2024-03-23T18:09:55.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:55 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:55.172+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.498831+0000 front 2024-03-23T18:09:15.498800+0000 (oldest deadline 2024-03-23T18:09:39.598988+0000) 2024-03-23T18:09:55.558 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:55 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:55.172+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:21.900315+0000 front 2024-03-23T18:09:21.900091+0000 (oldest deadline 2024-03-23T18:09:45.400070+0000) 2024-03-23T18:09:55.558 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:55 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:55.172+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-03-23T18:09:29.501333+0000 front 2024-03-23T18:09:29.501303+0000 (oldest deadline 2024-03-23T18:09:53.001492+0000) 2024-03-23T18:09:55.586 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:09:55 smithi152 systemd[1]: Stopping Ceph osd.8 for 9d76c9e8-e93c-11ee-95cd-87774f69a715... 2024-03-23T18:09:55.586 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:09:55 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T18:09:55.397+0000 7f723058b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T18:09:55.586 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:09:55 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T18:09:55.397+0000 7f723058b700 -1 osd.8 113 *** Got signal Terminated *** 2024-03-23T18:09:55.587 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:09:55 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T18:09:55.397+0000 7f723058b700 -1 osd.8 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-23T18:09:55.587 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:55 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:55.148+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.653471+0000 front 2024-03-23T18:09:00.653556+0000 (oldest deadline 2024-03-23T18:09:22.953547+0000) 2024-03-23T18:09:55.587 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:55 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:55.148+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.155613+0000 front 2024-03-23T18:09:08.155550+0000 (oldest deadline 2024-03-23T18:09:32.855633+0000) 2024-03-23T18:09:55.587 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:55 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:55.148+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.057340+0000 front 2024-03-23T18:09:15.057318+0000 (oldest deadline 2024-03-23T18:09:37.357408+0000) 2024-03-23T18:09:55.588 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:55 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:55.148+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:20.858463+0000 front 2024-03-23T18:09:20.858478+0000 (oldest deadline 2024-03-23T18:09:46.758574+0000) 2024-03-23T18:09:55.588 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:55 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:55.148+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-03-23T18:09:26.759276+0000 front 2024-03-23T18:09:26.759351+0000 (oldest deadline 2024-03-23T18:09:52.659216+0000) 2024-03-23T18:09:56.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:55 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:55.619+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.122328+0000 front 2024-03-23T18:09:00.122374+0000 (oldest deadline 2024-03-23T18:09:24.122945+0000) 2024-03-23T18:09:56.017 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:55 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:55.619+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:05.125484+0000 front 2024-03-23T18:09:05.125462+0000 (oldest deadline 2024-03-23T18:09:33.326776+0000) 2024-03-23T18:09:56.017 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:55 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:55.619+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:13.327558+0000 front 2024-03-23T18:09:13.327314+0000 (oldest deadline 2024-03-23T18:09:39.227552+0000) 2024-03-23T18:09:56.017 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:55 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:55.619+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:23.129785+0000 front 2024-03-23T18:09:23.129607+0000 (oldest deadline 2024-03-23T18:09:49.029801+0000) 2024-03-23T18:09:56.085 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:09:55 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T18:09:55.802+0000 7f722d382700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.458202+0000 front 2024-03-23T18:09:08.458278+0000 (oldest deadline 2024-03-23T18:09:33.158318+0000) 2024-03-23T18:09:56.085 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:09:55 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T18:09:55.802+0000 7f722d382700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:13.158888+0000 front 2024-03-23T18:09:13.158856+0000 (oldest deadline 2024-03-23T18:09:37.858881+0000) 2024-03-23T18:09:56.086 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:09:55 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T18:09:55.802+0000 7f722d382700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:21.359871+0000 front 2024-03-23T18:09:21.359947+0000 (oldest deadline 2024-03-23T18:09:44.259789+0000) 2024-03-23T18:09:56.086 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:09:55 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T18:09:55.802+0000 7f722d382700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-03-23T18:09:30.061831+0000 front 2024-03-23T18:09:30.061844+0000 (oldest deadline 2024-03-23T18:09:53.561913+0000) 2024-03-23T18:09:56.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:56 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:56.133+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:59.796052+0000 front 2024-03-23T18:08:59.796002+0000 (oldest deadline 2024-03-23T18:09:23.895742+0000) 2024-03-23T18:09:56.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:56 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:56.133+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.796583+0000 front 2024-03-23T18:09:06.796752+0000 (oldest deadline 2024-03-23T18:09:30.896916+0000) 2024-03-23T18:09:56.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:56 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:56.133+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.498831+0000 front 2024-03-23T18:09:15.498800+0000 (oldest deadline 2024-03-23T18:09:39.598988+0000) 2024-03-23T18:09:56.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:56 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:56.133+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:21.900315+0000 front 2024-03-23T18:09:21.900091+0000 (oldest deadline 2024-03-23T18:09:45.400070+0000) 2024-03-23T18:09:56.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:56 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:56.133+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-03-23T18:09:29.501333+0000 front 2024-03-23T18:09:29.501303+0000 (oldest deadline 2024-03-23T18:09:53.001492+0000) 2024-03-23T18:09:56.585 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:56 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:56.119+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.653471+0000 front 2024-03-23T18:09:00.653556+0000 (oldest deadline 2024-03-23T18:09:22.953547+0000) 2024-03-23T18:09:56.585 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:56 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:56.119+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.155613+0000 front 2024-03-23T18:09:08.155550+0000 (oldest deadline 2024-03-23T18:09:32.855633+0000) 2024-03-23T18:09:56.585 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:56 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:56.119+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.057340+0000 front 2024-03-23T18:09:15.057318+0000 (oldest deadline 2024-03-23T18:09:37.357408+0000) 2024-03-23T18:09:56.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:56 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:56.119+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:20.858463+0000 front 2024-03-23T18:09:20.858478+0000 (oldest deadline 2024-03-23T18:09:46.758574+0000) 2024-03-23T18:09:56.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:56 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:56.119+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-03-23T18:09:26.759276+0000 front 2024-03-23T18:09:26.759351+0000 (oldest deadline 2024-03-23T18:09:52.659216+0000) 2024-03-23T18:09:57.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:56 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:56.620+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.122328+0000 front 2024-03-23T18:09:00.122374+0000 (oldest deadline 2024-03-23T18:09:24.122945+0000) 2024-03-23T18:09:57.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:56 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:56.620+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:05.125484+0000 front 2024-03-23T18:09:05.125462+0000 (oldest deadline 2024-03-23T18:09:33.326776+0000) 2024-03-23T18:09:57.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:56 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:56.620+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:13.327558+0000 front 2024-03-23T18:09:13.327314+0000 (oldest deadline 2024-03-23T18:09:39.227552+0000) 2024-03-23T18:09:57.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:56 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:56.620+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:23.129785+0000 front 2024-03-23T18:09:23.129607+0000 (oldest deadline 2024-03-23T18:09:49.029801+0000) 2024-03-23T18:09:57.085 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:09:56 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T18:09:56.766+0000 7f722d382700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.458202+0000 front 2024-03-23T18:09:08.458278+0000 (oldest deadline 2024-03-23T18:09:33.158318+0000) 2024-03-23T18:09:57.085 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:09:56 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T18:09:56.766+0000 7f722d382700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:13.158888+0000 front 2024-03-23T18:09:13.158856+0000 (oldest deadline 2024-03-23T18:09:37.858881+0000) 2024-03-23T18:09:57.086 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:09:56 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T18:09:56.766+0000 7f722d382700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:21.359871+0000 front 2024-03-23T18:09:21.359947+0000 (oldest deadline 2024-03-23T18:09:44.259789+0000) 2024-03-23T18:09:57.086 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:09:56 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T18:09:56.766+0000 7f722d382700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-03-23T18:09:30.061831+0000 front 2024-03-23T18:09:30.061844+0000 (oldest deadline 2024-03-23T18:09:53.561913+0000) 2024-03-23T18:09:57.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:57 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:57.118+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:59.796052+0000 front 2024-03-23T18:08:59.796002+0000 (oldest deadline 2024-03-23T18:09:23.895742+0000) 2024-03-23T18:09:57.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:57 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:57.118+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.796583+0000 front 2024-03-23T18:09:06.796752+0000 (oldest deadline 2024-03-23T18:09:30.896916+0000) 2024-03-23T18:09:57.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:57 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:57.118+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.498831+0000 front 2024-03-23T18:09:15.498800+0000 (oldest deadline 2024-03-23T18:09:39.598988+0000) 2024-03-23T18:09:57.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:57 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:57.118+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:21.900315+0000 front 2024-03-23T18:09:21.900091+0000 (oldest deadline 2024-03-23T18:09:45.400070+0000) 2024-03-23T18:09:57.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:57 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:57.118+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-03-23T18:09:29.501333+0000 front 2024-03-23T18:09:29.501303+0000 (oldest deadline 2024-03-23T18:09:53.001492+0000) 2024-03-23T18:09:57.585 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:57 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:57.158+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.653471+0000 front 2024-03-23T18:09:00.653556+0000 (oldest deadline 2024-03-23T18:09:22.953547+0000) 2024-03-23T18:09:57.585 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:57 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:57.158+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.155613+0000 front 2024-03-23T18:09:08.155550+0000 (oldest deadline 2024-03-23T18:09:32.855633+0000) 2024-03-23T18:09:57.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:57 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:57.158+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.057340+0000 front 2024-03-23T18:09:15.057318+0000 (oldest deadline 2024-03-23T18:09:37.357408+0000) 2024-03-23T18:09:57.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:57 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:57.158+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:20.858463+0000 front 2024-03-23T18:09:20.858478+0000 (oldest deadline 2024-03-23T18:09:46.758574+0000) 2024-03-23T18:09:57.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:57 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:57.158+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-03-23T18:09:26.759276+0000 front 2024-03-23T18:09:26.759351+0000 (oldest deadline 2024-03-23T18:09:52.659216+0000) 2024-03-23T18:09:58.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:57 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:57.584+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.122328+0000 front 2024-03-23T18:09:00.122374+0000 (oldest deadline 2024-03-23T18:09:24.122945+0000) 2024-03-23T18:09:58.017 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:57 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:57.584+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:05.125484+0000 front 2024-03-23T18:09:05.125462+0000 (oldest deadline 2024-03-23T18:09:33.326776+0000) 2024-03-23T18:09:58.017 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:57 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:57.584+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:13.327558+0000 front 2024-03-23T18:09:13.327314+0000 (oldest deadline 2024-03-23T18:09:39.227552+0000) 2024-03-23T18:09:58.017 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:57 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:57.584+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:23.129785+0000 front 2024-03-23T18:09:23.129607+0000 (oldest deadline 2024-03-23T18:09:49.029801+0000) 2024-03-23T18:09:58.085 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:09:57 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T18:09:57.732+0000 7f722d382700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.458202+0000 front 2024-03-23T18:09:08.458278+0000 (oldest deadline 2024-03-23T18:09:33.158318+0000) 2024-03-23T18:09:58.086 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:09:57 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T18:09:57.732+0000 7f722d382700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:13.158888+0000 front 2024-03-23T18:09:13.158856+0000 (oldest deadline 2024-03-23T18:09:37.858881+0000) 2024-03-23T18:09:58.086 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:09:57 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T18:09:57.732+0000 7f722d382700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:21.359871+0000 front 2024-03-23T18:09:21.359947+0000 (oldest deadline 2024-03-23T18:09:44.259789+0000) 2024-03-23T18:09:58.086 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:09:57 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T18:09:57.732+0000 7f722d382700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-03-23T18:09:30.061831+0000 front 2024-03-23T18:09:30.061844+0000 (oldest deadline 2024-03-23T18:09:53.561913+0000) 2024-03-23T18:09:58.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:58 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:58.076+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:59.796052+0000 front 2024-03-23T18:08:59.796002+0000 (oldest deadline 2024-03-23T18:09:23.895742+0000) 2024-03-23T18:09:58.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:58 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:58.076+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.796583+0000 front 2024-03-23T18:09:06.796752+0000 (oldest deadline 2024-03-23T18:09:30.896916+0000) 2024-03-23T18:09:58.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:58 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:58.076+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.498831+0000 front 2024-03-23T18:09:15.498800+0000 (oldest deadline 2024-03-23T18:09:39.598988+0000) 2024-03-23T18:09:58.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:58 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:58.076+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:21.900315+0000 front 2024-03-23T18:09:21.900091+0000 (oldest deadline 2024-03-23T18:09:45.400070+0000) 2024-03-23T18:09:58.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:58 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:58.076+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-03-23T18:09:29.501333+0000 front 2024-03-23T18:09:29.501303+0000 (oldest deadline 2024-03-23T18:09:53.001492+0000) 2024-03-23T18:09:58.585 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:58 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:58.206+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.653471+0000 front 2024-03-23T18:09:00.653556+0000 (oldest deadline 2024-03-23T18:09:22.953547+0000) 2024-03-23T18:09:58.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:58 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:58.206+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.155613+0000 front 2024-03-23T18:09:08.155550+0000 (oldest deadline 2024-03-23T18:09:32.855633+0000) 2024-03-23T18:09:58.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:58 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:58.206+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.057340+0000 front 2024-03-23T18:09:15.057318+0000 (oldest deadline 2024-03-23T18:09:37.357408+0000) 2024-03-23T18:09:58.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:58 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:58.206+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:20.858463+0000 front 2024-03-23T18:09:20.858478+0000 (oldest deadline 2024-03-23T18:09:46.758574+0000) 2024-03-23T18:09:58.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:58 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:58.206+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-03-23T18:09:26.759276+0000 front 2024-03-23T18:09:26.759351+0000 (oldest deadline 2024-03-23T18:09:52.659216+0000) 2024-03-23T18:09:59.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:58 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:58.586+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.122328+0000 front 2024-03-23T18:09:00.122374+0000 (oldest deadline 2024-03-23T18:09:24.122945+0000) 2024-03-23T18:09:59.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:58 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:58.586+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:05.125484+0000 front 2024-03-23T18:09:05.125462+0000 (oldest deadline 2024-03-23T18:09:33.326776+0000) 2024-03-23T18:09:59.017 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:58 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:58.586+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:13.327558+0000 front 2024-03-23T18:09:13.327314+0000 (oldest deadline 2024-03-23T18:09:39.227552+0000) 2024-03-23T18:09:59.017 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:58 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:58.586+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:23.129785+0000 front 2024-03-23T18:09:23.129607+0000 (oldest deadline 2024-03-23T18:09:49.029801+0000) 2024-03-23T18:09:59.086 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:09:58 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T18:09:58.723+0000 7f722d382700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.458202+0000 front 2024-03-23T18:09:08.458278+0000 (oldest deadline 2024-03-23T18:09:33.158318+0000) 2024-03-23T18:09:59.086 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:09:58 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T18:09:58.723+0000 7f722d382700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:13.158888+0000 front 2024-03-23T18:09:13.158856+0000 (oldest deadline 2024-03-23T18:09:37.858881+0000) 2024-03-23T18:09:59.086 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:09:58 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T18:09:58.723+0000 7f722d382700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:21.359871+0000 front 2024-03-23T18:09:21.359947+0000 (oldest deadline 2024-03-23T18:09:44.259789+0000) 2024-03-23T18:09:59.086 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:09:58 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T18:09:58.723+0000 7f722d382700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-03-23T18:09:30.061831+0000 front 2024-03-23T18:09:30.061844+0000 (oldest deadline 2024-03-23T18:09:53.561913+0000) 2024-03-23T18:09:59.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:59 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:59.112+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:59.796052+0000 front 2024-03-23T18:08:59.796002+0000 (oldest deadline 2024-03-23T18:09:23.895742+0000) 2024-03-23T18:09:59.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:59 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:59.112+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.796583+0000 front 2024-03-23T18:09:06.796752+0000 (oldest deadline 2024-03-23T18:09:30.896916+0000) 2024-03-23T18:09:59.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:59 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:59.112+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.498831+0000 front 2024-03-23T18:09:15.498800+0000 (oldest deadline 2024-03-23T18:09:39.598988+0000) 2024-03-23T18:09:59.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:59 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:59.112+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:21.900315+0000 front 2024-03-23T18:09:21.900091+0000 (oldest deadline 2024-03-23T18:09:45.400070+0000) 2024-03-23T18:09:59.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:09:59 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:09:59.112+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-03-23T18:09:29.501333+0000 front 2024-03-23T18:09:29.501303+0000 (oldest deadline 2024-03-23T18:09:53.001492+0000) 2024-03-23T18:09:59.585 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:59 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:59.200+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.653471+0000 front 2024-03-23T18:09:00.653556+0000 (oldest deadline 2024-03-23T18:09:22.953547+0000) 2024-03-23T18:09:59.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:59 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:59.200+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.155613+0000 front 2024-03-23T18:09:08.155550+0000 (oldest deadline 2024-03-23T18:09:32.855633+0000) 2024-03-23T18:09:59.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:59 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:59.200+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.057340+0000 front 2024-03-23T18:09:15.057318+0000 (oldest deadline 2024-03-23T18:09:37.357408+0000) 2024-03-23T18:09:59.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:59 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:59.200+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:20.858463+0000 front 2024-03-23T18:09:20.858478+0000 (oldest deadline 2024-03-23T18:09:46.758574+0000) 2024-03-23T18:09:59.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:09:59 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:09:59.200+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-03-23T18:09:26.759276+0000 front 2024-03-23T18:09:26.759351+0000 (oldest deadline 2024-03-23T18:09:52.659216+0000) 2024-03-23T18:10:00.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:59 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:59.619+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.122328+0000 front 2024-03-23T18:09:00.122374+0000 (oldest deadline 2024-03-23T18:09:24.122945+0000) 2024-03-23T18:10:00.018 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:59 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:59.619+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:05.125484+0000 front 2024-03-23T18:09:05.125462+0000 (oldest deadline 2024-03-23T18:09:33.326776+0000) 2024-03-23T18:10:00.018 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:59 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:59.619+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:13.327558+0000 front 2024-03-23T18:09:13.327314+0000 (oldest deadline 2024-03-23T18:09:39.227552+0000) 2024-03-23T18:10:00.018 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:09:59 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:09:59.619+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:23.129785+0000 front 2024-03-23T18:09:23.129607+0000 (oldest deadline 2024-03-23T18:09:49.029801+0000) 2024-03-23T18:10:00.086 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:09:59 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T18:09:59.690+0000 7f722d382700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.458202+0000 front 2024-03-23T18:09:08.458278+0000 (oldest deadline 2024-03-23T18:09:33.158318+0000) 2024-03-23T18:10:00.086 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:09:59 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T18:09:59.690+0000 7f722d382700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:13.158888+0000 front 2024-03-23T18:09:13.158856+0000 (oldest deadline 2024-03-23T18:09:37.858881+0000) 2024-03-23T18:10:00.086 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:09:59 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T18:09:59.690+0000 7f722d382700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:21.359871+0000 front 2024-03-23T18:09:21.359947+0000 (oldest deadline 2024-03-23T18:09:44.259789+0000) 2024-03-23T18:10:00.086 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:09:59 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T18:09:59.690+0000 7f722d382700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-03-23T18:09:30.061831+0000 front 2024-03-23T18:09:30.061844+0000 (oldest deadline 2024-03-23T18:09:53.561913+0000) 2024-03-23T18:10:00.086 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:09:59 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8[70248]: 2024-03-23T18:09:59.690+0000 7f722d382700 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-03-23T18:09:34.062660+0000 front 2024-03-23T18:09:34.062636+0000 (oldest deadline 2024-03-23T18:09:59.362792+0000) 2024-03-23T18:10:00.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:00 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:00.121+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:59.796052+0000 front 2024-03-23T18:08:59.796002+0000 (oldest deadline 2024-03-23T18:09:23.895742+0000) 2024-03-23T18:10:00.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:00 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:00.121+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.796583+0000 front 2024-03-23T18:09:06.796752+0000 (oldest deadline 2024-03-23T18:09:30.896916+0000) 2024-03-23T18:10:00.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:00 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:00.121+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.498831+0000 front 2024-03-23T18:09:15.498800+0000 (oldest deadline 2024-03-23T18:09:39.598988+0000) 2024-03-23T18:10:00.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:00 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:00.121+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:21.900315+0000 front 2024-03-23T18:09:21.900091+0000 (oldest deadline 2024-03-23T18:09:45.400070+0000) 2024-03-23T18:10:00.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:00 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:00.121+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-03-23T18:09:29.501333+0000 front 2024-03-23T18:09:29.501303+0000 (oldest deadline 2024-03-23T18:09:53.001492+0000) 2024-03-23T18:10:00.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:00 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:00.121+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-03-23T18:09:37.102661+0000 front 2024-03-23T18:09:37.102759+0000 (oldest deadline 2024-03-23T18:10:00.002501+0000) 2024-03-23T18:10:00.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:00 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:00.245+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.653471+0000 front 2024-03-23T18:09:00.653556+0000 (oldest deadline 2024-03-23T18:09:22.953547+0000) 2024-03-23T18:10:00.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:00 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:00.245+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.155613+0000 front 2024-03-23T18:09:08.155550+0000 (oldest deadline 2024-03-23T18:09:32.855633+0000) 2024-03-23T18:10:00.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:00 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:00.245+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.057340+0000 front 2024-03-23T18:09:15.057318+0000 (oldest deadline 2024-03-23T18:09:37.357408+0000) 2024-03-23T18:10:00.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:00 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:00.245+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:20.858463+0000 front 2024-03-23T18:09:20.858478+0000 (oldest deadline 2024-03-23T18:09:46.758574+0000) 2024-03-23T18:10:00.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:00 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:00.245+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-03-23T18:09:26.759276+0000 front 2024-03-23T18:09:26.759351+0000 (oldest deadline 2024-03-23T18:09:52.659216+0000) 2024-03-23T18:10:01.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:10:00 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:10:00.592+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.122328+0000 front 2024-03-23T18:09:00.122374+0000 (oldest deadline 2024-03-23T18:09:24.122945+0000) 2024-03-23T18:10:01.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:10:00 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:10:00.592+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:05.125484+0000 front 2024-03-23T18:09:05.125462+0000 (oldest deadline 2024-03-23T18:09:33.326776+0000) 2024-03-23T18:10:01.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:10:00 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:10:00.592+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:13.327558+0000 front 2024-03-23T18:09:13.327314+0000 (oldest deadline 2024-03-23T18:09:39.227552+0000) 2024-03-23T18:10:01.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:10:00 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:10:00.592+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:23.129785+0000 front 2024-03-23T18:09:23.129607+0000 (oldest deadline 2024-03-23T18:09:49.029801+0000) 2024-03-23T18:10:01.335 INFO:journalctl@ceph.osd.8.smithi152.stdout:Mar 23 18:10:01 smithi152 bash[94924]: ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-8 2024-03-23T18:10:01.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:01 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:01.224+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.653471+0000 front 2024-03-23T18:09:00.653556+0000 (oldest deadline 2024-03-23T18:09:22.953547+0000) 2024-03-23T18:10:01.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:01 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:01.224+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.155613+0000 front 2024-03-23T18:09:08.155550+0000 (oldest deadline 2024-03-23T18:09:32.855633+0000) 2024-03-23T18:10:01.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:01 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:01.224+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.057340+0000 front 2024-03-23T18:09:15.057318+0000 (oldest deadline 2024-03-23T18:09:37.357408+0000) 2024-03-23T18:10:01.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:01 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:01.224+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:20.858463+0000 front 2024-03-23T18:09:20.858478+0000 (oldest deadline 2024-03-23T18:09:46.758574+0000) 2024-03-23T18:10:01.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:01 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:01.224+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-03-23T18:09:26.759276+0000 front 2024-03-23T18:09:26.759351+0000 (oldest deadline 2024-03-23T18:09:52.659216+0000) 2024-03-23T18:10:01.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:01 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:01.224+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-03-23T18:09:36.160352+0000 front 2024-03-23T18:09:36.160374+0000 (oldest deadline 2024-03-23T18:10:00.761076+0000) 2024-03-23T18:10:01.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:01 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:01.163+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:59.796052+0000 front 2024-03-23T18:08:59.796002+0000 (oldest deadline 2024-03-23T18:09:23.895742+0000) 2024-03-23T18:10:01.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:01 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:01.163+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.796583+0000 front 2024-03-23T18:09:06.796752+0000 (oldest deadline 2024-03-23T18:09:30.896916+0000) 2024-03-23T18:10:01.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:01 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:01.163+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.498831+0000 front 2024-03-23T18:09:15.498800+0000 (oldest deadline 2024-03-23T18:09:39.598988+0000) 2024-03-23T18:10:01.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:01 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:01.163+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:21.900315+0000 front 2024-03-23T18:09:21.900091+0000 (oldest deadline 2024-03-23T18:09:45.400070+0000) 2024-03-23T18:10:01.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:01 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:01.163+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-03-23T18:09:29.501333+0000 front 2024-03-23T18:09:29.501303+0000 (oldest deadline 2024-03-23T18:09:53.001492+0000) 2024-03-23T18:10:01.558 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:01 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:01.163+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-03-23T18:09:37.102661+0000 front 2024-03-23T18:09:37.102759+0000 (oldest deadline 2024-03-23T18:10:00.002501+0000) 2024-03-23T18:10:02.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:10:01 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:10:01.635+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.122328+0000 front 2024-03-23T18:09:00.122374+0000 (oldest deadline 2024-03-23T18:09:24.122945+0000) 2024-03-23T18:10:02.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:10:01 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:10:01.635+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:05.125484+0000 front 2024-03-23T18:09:05.125462+0000 (oldest deadline 2024-03-23T18:09:33.326776+0000) 2024-03-23T18:10:02.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:10:01 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:10:01.635+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:13.327558+0000 front 2024-03-23T18:09:13.327314+0000 (oldest deadline 2024-03-23T18:09:39.227552+0000) 2024-03-23T18:10:02.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:10:01 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:10:01.635+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:23.129785+0000 front 2024-03-23T18:09:23.129607+0000 (oldest deadline 2024-03-23T18:09:49.029801+0000) 2024-03-23T18:10:02.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:10:01 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:10:01.635+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-03-23T18:09:37.231501+0000 front 2024-03-23T18:09:37.231420+0000 (oldest deadline 2024-03-23T18:10:01.331392+0000) 2024-03-23T18:10:02.284 DEBUG:teuthology.orchestra.run.smithi152:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@osd.8.service' 2024-03-23T18:10:02.324 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T18:10:02.325 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-03-23T18:10:02.325 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-03-23T18:10:02.325 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@osd.9 2024-03-23T18:10:02.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:02 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:02.189+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:59.796052+0000 front 2024-03-23T18:08:59.796002+0000 (oldest deadline 2024-03-23T18:09:23.895742+0000) 2024-03-23T18:10:02.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:02 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:02.189+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.796583+0000 front 2024-03-23T18:09:06.796752+0000 (oldest deadline 2024-03-23T18:09:30.896916+0000) 2024-03-23T18:10:02.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:02 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:02.189+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.498831+0000 front 2024-03-23T18:09:15.498800+0000 (oldest deadline 2024-03-23T18:09:39.598988+0000) 2024-03-23T18:10:02.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:02 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:02.189+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:21.900315+0000 front 2024-03-23T18:09:21.900091+0000 (oldest deadline 2024-03-23T18:09:45.400070+0000) 2024-03-23T18:10:02.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:02 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:02.189+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-03-23T18:09:29.501333+0000 front 2024-03-23T18:09:29.501303+0000 (oldest deadline 2024-03-23T18:09:53.001492+0000) 2024-03-23T18:10:02.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:02 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:02.189+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-03-23T18:09:37.102661+0000 front 2024-03-23T18:09:37.102759+0000 (oldest deadline 2024-03-23T18:10:00.002501+0000) 2024-03-23T18:10:02.585 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:02 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:02.206+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.653471+0000 front 2024-03-23T18:09:00.653556+0000 (oldest deadline 2024-03-23T18:09:22.953547+0000) 2024-03-23T18:10:02.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:02 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:02.206+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.155613+0000 front 2024-03-23T18:09:08.155550+0000 (oldest deadline 2024-03-23T18:09:32.855633+0000) 2024-03-23T18:10:02.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:02 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:02.206+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.057340+0000 front 2024-03-23T18:09:15.057318+0000 (oldest deadline 2024-03-23T18:09:37.357408+0000) 2024-03-23T18:10:02.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:02 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:02.206+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:20.858463+0000 front 2024-03-23T18:09:20.858478+0000 (oldest deadline 2024-03-23T18:09:46.758574+0000) 2024-03-23T18:10:02.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:02 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:02.206+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-03-23T18:09:26.759276+0000 front 2024-03-23T18:09:26.759351+0000 (oldest deadline 2024-03-23T18:09:52.659216+0000) 2024-03-23T18:10:02.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:02 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:02.206+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-03-23T18:09:36.160352+0000 front 2024-03-23T18:09:36.160374+0000 (oldest deadline 2024-03-23T18:10:00.761076+0000) 2024-03-23T18:10:02.679 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:10:02 smithi086 systemd[1]: Stopping Ceph osd.9 for 9d76c9e8-e93c-11ee-95cd-87774f69a715... 2024-03-23T18:10:03.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:10:02 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:10:02.679+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.122328+0000 front 2024-03-23T18:09:00.122374+0000 (oldest deadline 2024-03-23T18:09:24.122945+0000) 2024-03-23T18:10:03.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:10:02 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:10:02.679+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:05.125484+0000 front 2024-03-23T18:09:05.125462+0000 (oldest deadline 2024-03-23T18:09:33.326776+0000) 2024-03-23T18:10:03.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:10:02 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:10:02.679+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:13.327558+0000 front 2024-03-23T18:09:13.327314+0000 (oldest deadline 2024-03-23T18:09:39.227552+0000) 2024-03-23T18:10:03.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:10:02 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:10:02.679+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:23.129785+0000 front 2024-03-23T18:09:23.129607+0000 (oldest deadline 2024-03-23T18:09:49.029801+0000) 2024-03-23T18:10:03.017 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:10:02 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:10:02.679+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-03-23T18:09:37.231501+0000 front 2024-03-23T18:09:37.231420+0000 (oldest deadline 2024-03-23T18:10:01.331392+0000) 2024-03-23T18:10:03.017 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:10:02 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:10:02.744+0000 7fa31f600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T18:10:03.017 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:10:02 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:10:02.744+0000 7fa31f600700 -1 osd.9 113 *** Got signal Terminated *** 2024-03-23T18:10:03.017 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:10:02 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:10:02.744+0000 7fa31f600700 -1 osd.9 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-23T18:10:03.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:03 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:03.143+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:59.796052+0000 front 2024-03-23T18:08:59.796002+0000 (oldest deadline 2024-03-23T18:09:23.895742+0000) 2024-03-23T18:10:03.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:03 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:03.143+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.796583+0000 front 2024-03-23T18:09:06.796752+0000 (oldest deadline 2024-03-23T18:09:30.896916+0000) 2024-03-23T18:10:03.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:03 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:03.143+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.498831+0000 front 2024-03-23T18:09:15.498800+0000 (oldest deadline 2024-03-23T18:09:39.598988+0000) 2024-03-23T18:10:03.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:03 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:03.143+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:21.900315+0000 front 2024-03-23T18:09:21.900091+0000 (oldest deadline 2024-03-23T18:09:45.400070+0000) 2024-03-23T18:10:03.558 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:03 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:03.143+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-03-23T18:09:29.501333+0000 front 2024-03-23T18:09:29.501303+0000 (oldest deadline 2024-03-23T18:09:53.001492+0000) 2024-03-23T18:10:03.558 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:03 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:03.143+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-03-23T18:09:37.102661+0000 front 2024-03-23T18:09:37.102759+0000 (oldest deadline 2024-03-23T18:10:00.002501+0000) 2024-03-23T18:10:03.585 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:03 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:03.182+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.653471+0000 front 2024-03-23T18:09:00.653556+0000 (oldest deadline 2024-03-23T18:09:22.953547+0000) 2024-03-23T18:10:03.585 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:03 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:03.182+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.155613+0000 front 2024-03-23T18:09:08.155550+0000 (oldest deadline 2024-03-23T18:09:32.855633+0000) 2024-03-23T18:10:03.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:03 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:03.182+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.057340+0000 front 2024-03-23T18:09:15.057318+0000 (oldest deadline 2024-03-23T18:09:37.357408+0000) 2024-03-23T18:10:03.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:03 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:03.182+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:20.858463+0000 front 2024-03-23T18:09:20.858478+0000 (oldest deadline 2024-03-23T18:09:46.758574+0000) 2024-03-23T18:10:03.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:03 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:03.182+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-03-23T18:09:26.759276+0000 front 2024-03-23T18:09:26.759351+0000 (oldest deadline 2024-03-23T18:09:52.659216+0000) 2024-03-23T18:10:03.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:03 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:03.182+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-03-23T18:09:36.160352+0000 front 2024-03-23T18:09:36.160374+0000 (oldest deadline 2024-03-23T18:10:00.761076+0000) 2024-03-23T18:10:04.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:10:03 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:10:03.705+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.122328+0000 front 2024-03-23T18:09:00.122374+0000 (oldest deadline 2024-03-23T18:09:24.122945+0000) 2024-03-23T18:10:04.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:10:03 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:10:03.705+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:05.125484+0000 front 2024-03-23T18:09:05.125462+0000 (oldest deadline 2024-03-23T18:09:33.326776+0000) 2024-03-23T18:10:04.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:10:03 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:10:03.705+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:13.327558+0000 front 2024-03-23T18:09:13.327314+0000 (oldest deadline 2024-03-23T18:09:39.227552+0000) 2024-03-23T18:10:04.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:10:03 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:10:03.705+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:23.129785+0000 front 2024-03-23T18:09:23.129607+0000 (oldest deadline 2024-03-23T18:09:49.029801+0000) 2024-03-23T18:10:04.017 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:10:03 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:10:03.705+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-03-23T18:09:37.231501+0000 front 2024-03-23T18:09:37.231420+0000 (oldest deadline 2024-03-23T18:10:01.331392+0000) 2024-03-23T18:10:04.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:04 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:04.189+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:59.796052+0000 front 2024-03-23T18:08:59.796002+0000 (oldest deadline 2024-03-23T18:09:23.895742+0000) 2024-03-23T18:10:04.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:04 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:04.189+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.796583+0000 front 2024-03-23T18:09:06.796752+0000 (oldest deadline 2024-03-23T18:09:30.896916+0000) 2024-03-23T18:10:04.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:04 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:04.189+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.498831+0000 front 2024-03-23T18:09:15.498800+0000 (oldest deadline 2024-03-23T18:09:39.598988+0000) 2024-03-23T18:10:04.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:04 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:04.189+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:21.900315+0000 front 2024-03-23T18:09:21.900091+0000 (oldest deadline 2024-03-23T18:09:45.400070+0000) 2024-03-23T18:10:04.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:04 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:04.189+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-03-23T18:09:29.501333+0000 front 2024-03-23T18:09:29.501303+0000 (oldest deadline 2024-03-23T18:09:53.001492+0000) 2024-03-23T18:10:04.558 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:04 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:04.189+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-03-23T18:09:37.102661+0000 front 2024-03-23T18:09:37.102759+0000 (oldest deadline 2024-03-23T18:10:00.002501+0000) 2024-03-23T18:10:04.585 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:04 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:04.188+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.653471+0000 front 2024-03-23T18:09:00.653556+0000 (oldest deadline 2024-03-23T18:09:22.953547+0000) 2024-03-23T18:10:04.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:04 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:04.188+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.155613+0000 front 2024-03-23T18:09:08.155550+0000 (oldest deadline 2024-03-23T18:09:32.855633+0000) 2024-03-23T18:10:04.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:04 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:04.188+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.057340+0000 front 2024-03-23T18:09:15.057318+0000 (oldest deadline 2024-03-23T18:09:37.357408+0000) 2024-03-23T18:10:04.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:04 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:04.188+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:20.858463+0000 front 2024-03-23T18:09:20.858478+0000 (oldest deadline 2024-03-23T18:09:46.758574+0000) 2024-03-23T18:10:04.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:04 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:04.188+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-03-23T18:09:26.759276+0000 front 2024-03-23T18:09:26.759351+0000 (oldest deadline 2024-03-23T18:09:52.659216+0000) 2024-03-23T18:10:04.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:04 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:04.188+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-03-23T18:09:36.160352+0000 front 2024-03-23T18:09:36.160374+0000 (oldest deadline 2024-03-23T18:10:00.761076+0000) 2024-03-23T18:10:05.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:10:04 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:10:04.669+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.122328+0000 front 2024-03-23T18:09:00.122374+0000 (oldest deadline 2024-03-23T18:09:24.122945+0000) 2024-03-23T18:10:05.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:10:04 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:10:04.669+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:05.125484+0000 front 2024-03-23T18:09:05.125462+0000 (oldest deadline 2024-03-23T18:09:33.326776+0000) 2024-03-23T18:10:05.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:10:04 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:10:04.669+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:13.327558+0000 front 2024-03-23T18:09:13.327314+0000 (oldest deadline 2024-03-23T18:09:39.227552+0000) 2024-03-23T18:10:05.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:10:04 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:10:04.669+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:23.129785+0000 front 2024-03-23T18:09:23.129607+0000 (oldest deadline 2024-03-23T18:09:49.029801+0000) 2024-03-23T18:10:05.017 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:10:04 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:10:04.669+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-03-23T18:09:37.231501+0000 front 2024-03-23T18:09:37.231420+0000 (oldest deadline 2024-03-23T18:10:01.331392+0000) 2024-03-23T18:10:05.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:05 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:05.196+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:59.796052+0000 front 2024-03-23T18:08:59.796002+0000 (oldest deadline 2024-03-23T18:09:23.895742+0000) 2024-03-23T18:10:05.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:05 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:05.196+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.796583+0000 front 2024-03-23T18:09:06.796752+0000 (oldest deadline 2024-03-23T18:09:30.896916+0000) 2024-03-23T18:10:05.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:05 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:05.196+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.498831+0000 front 2024-03-23T18:09:15.498800+0000 (oldest deadline 2024-03-23T18:09:39.598988+0000) 2024-03-23T18:10:05.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:05 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:05.196+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:21.900315+0000 front 2024-03-23T18:09:21.900091+0000 (oldest deadline 2024-03-23T18:09:45.400070+0000) 2024-03-23T18:10:05.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:05 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:05.196+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-03-23T18:09:29.501333+0000 front 2024-03-23T18:09:29.501303+0000 (oldest deadline 2024-03-23T18:09:53.001492+0000) 2024-03-23T18:10:05.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:05 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:05.196+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-03-23T18:09:37.102661+0000 front 2024-03-23T18:09:37.102759+0000 (oldest deadline 2024-03-23T18:10:00.002501+0000) 2024-03-23T18:10:05.585 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:05 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:05.175+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.653471+0000 front 2024-03-23T18:09:00.653556+0000 (oldest deadline 2024-03-23T18:09:22.953547+0000) 2024-03-23T18:10:05.585 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:05 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:05.175+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.155613+0000 front 2024-03-23T18:09:08.155550+0000 (oldest deadline 2024-03-23T18:09:32.855633+0000) 2024-03-23T18:10:05.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:05 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:05.175+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.057340+0000 front 2024-03-23T18:09:15.057318+0000 (oldest deadline 2024-03-23T18:09:37.357408+0000) 2024-03-23T18:10:05.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:05 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:05.175+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:20.858463+0000 front 2024-03-23T18:09:20.858478+0000 (oldest deadline 2024-03-23T18:09:46.758574+0000) 2024-03-23T18:10:05.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:05 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:05.175+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-03-23T18:09:26.759276+0000 front 2024-03-23T18:09:26.759351+0000 (oldest deadline 2024-03-23T18:09:52.659216+0000) 2024-03-23T18:10:05.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:05 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:05.175+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-03-23T18:09:36.160352+0000 front 2024-03-23T18:09:36.160374+0000 (oldest deadline 2024-03-23T18:10:00.761076+0000) 2024-03-23T18:10:06.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:10:05 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:10:05.704+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.122328+0000 front 2024-03-23T18:09:00.122374+0000 (oldest deadline 2024-03-23T18:09:24.122945+0000) 2024-03-23T18:10:06.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:10:05 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:10:05.704+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:05.125484+0000 front 2024-03-23T18:09:05.125462+0000 (oldest deadline 2024-03-23T18:09:33.326776+0000) 2024-03-23T18:10:06.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:10:05 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:10:05.704+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:13.327558+0000 front 2024-03-23T18:09:13.327314+0000 (oldest deadline 2024-03-23T18:09:39.227552+0000) 2024-03-23T18:10:06.017 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:10:05 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:10:05.704+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:23.129785+0000 front 2024-03-23T18:09:23.129607+0000 (oldest deadline 2024-03-23T18:09:49.029801+0000) 2024-03-23T18:10:06.017 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:10:05 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:10:05.704+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-03-23T18:09:37.231501+0000 front 2024-03-23T18:09:37.231420+0000 (oldest deadline 2024-03-23T18:10:01.331392+0000) 2024-03-23T18:10:06.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:06 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:06.234+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:59.796052+0000 front 2024-03-23T18:08:59.796002+0000 (oldest deadline 2024-03-23T18:09:23.895742+0000) 2024-03-23T18:10:06.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:06 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:06.234+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.796583+0000 front 2024-03-23T18:09:06.796752+0000 (oldest deadline 2024-03-23T18:09:30.896916+0000) 2024-03-23T18:10:06.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:06 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:06.234+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.498831+0000 front 2024-03-23T18:09:15.498800+0000 (oldest deadline 2024-03-23T18:09:39.598988+0000) 2024-03-23T18:10:06.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:06 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:06.234+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:21.900315+0000 front 2024-03-23T18:09:21.900091+0000 (oldest deadline 2024-03-23T18:09:45.400070+0000) 2024-03-23T18:10:06.558 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:06 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:06.234+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-03-23T18:09:29.501333+0000 front 2024-03-23T18:09:29.501303+0000 (oldest deadline 2024-03-23T18:09:53.001492+0000) 2024-03-23T18:10:06.558 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:06 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:06.234+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-03-23T18:09:37.102661+0000 front 2024-03-23T18:09:37.102759+0000 (oldest deadline 2024-03-23T18:10:00.002501+0000) 2024-03-23T18:10:06.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:06 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:06.216+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.653471+0000 front 2024-03-23T18:09:00.653556+0000 (oldest deadline 2024-03-23T18:09:22.953547+0000) 2024-03-23T18:10:06.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:06 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:06.216+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.155613+0000 front 2024-03-23T18:09:08.155550+0000 (oldest deadline 2024-03-23T18:09:32.855633+0000) 2024-03-23T18:10:06.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:06 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:06.216+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.057340+0000 front 2024-03-23T18:09:15.057318+0000 (oldest deadline 2024-03-23T18:09:37.357408+0000) 2024-03-23T18:10:06.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:06 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:06.216+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:20.858463+0000 front 2024-03-23T18:09:20.858478+0000 (oldest deadline 2024-03-23T18:09:46.758574+0000) 2024-03-23T18:10:06.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:06 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:06.216+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-03-23T18:09:26.759276+0000 front 2024-03-23T18:09:26.759351+0000 (oldest deadline 2024-03-23T18:09:52.659216+0000) 2024-03-23T18:10:06.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:06 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:06.216+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-03-23T18:09:36.160352+0000 front 2024-03-23T18:09:36.160374+0000 (oldest deadline 2024-03-23T18:10:00.761076+0000) 2024-03-23T18:10:07.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:10:06 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:10:06.679+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.122328+0000 front 2024-03-23T18:09:00.122374+0000 (oldest deadline 2024-03-23T18:09:24.122945+0000) 2024-03-23T18:10:07.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:10:06 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:10:06.679+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:05.125484+0000 front 2024-03-23T18:09:05.125462+0000 (oldest deadline 2024-03-23T18:09:33.326776+0000) 2024-03-23T18:10:07.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:10:06 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:10:06.679+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:13.327558+0000 front 2024-03-23T18:09:13.327314+0000 (oldest deadline 2024-03-23T18:09:39.227552+0000) 2024-03-23T18:10:07.017 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:10:06 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:10:06.679+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:23.129785+0000 front 2024-03-23T18:09:23.129607+0000 (oldest deadline 2024-03-23T18:09:49.029801+0000) 2024-03-23T18:10:07.017 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:10:06 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:10:06.679+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-03-23T18:09:37.231501+0000 front 2024-03-23T18:09:37.231420+0000 (oldest deadline 2024-03-23T18:10:01.331392+0000) 2024-03-23T18:10:07.017 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:10:06 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:10:06.679+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-03-23T18:09:41.832731+0000 front 2024-03-23T18:09:41.832710+0000 (oldest deadline 2024-03-23T18:10:06.532732+0000) 2024-03-23T18:10:07.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:07 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:07.274+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:59.796052+0000 front 2024-03-23T18:08:59.796002+0000 (oldest deadline 2024-03-23T18:09:23.895742+0000) 2024-03-23T18:10:07.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:07 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:07.274+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.796583+0000 front 2024-03-23T18:09:06.796752+0000 (oldest deadline 2024-03-23T18:09:30.896916+0000) 2024-03-23T18:10:07.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:07 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:07.274+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.498831+0000 front 2024-03-23T18:09:15.498800+0000 (oldest deadline 2024-03-23T18:09:39.598988+0000) 2024-03-23T18:10:07.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:07 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:07.274+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:21.900315+0000 front 2024-03-23T18:09:21.900091+0000 (oldest deadline 2024-03-23T18:09:45.400070+0000) 2024-03-23T18:10:07.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:07 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:07.274+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-03-23T18:09:29.501333+0000 front 2024-03-23T18:09:29.501303+0000 (oldest deadline 2024-03-23T18:09:53.001492+0000) 2024-03-23T18:10:07.558 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:07 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:07.274+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-03-23T18:09:37.102661+0000 front 2024-03-23T18:09:37.102759+0000 (oldest deadline 2024-03-23T18:10:00.002501+0000) 2024-03-23T18:10:07.558 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:07 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:07.274+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-03-23T18:09:45.303491+0000 front 2024-03-23T18:09:45.303537+0000 (oldest deadline 2024-03-23T18:10:06.403399+0000) 2024-03-23T18:10:07.585 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:07 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:07.209+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.653471+0000 front 2024-03-23T18:09:00.653556+0000 (oldest deadline 2024-03-23T18:09:22.953547+0000) 2024-03-23T18:10:07.585 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:07 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:07.209+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.155613+0000 front 2024-03-23T18:09:08.155550+0000 (oldest deadline 2024-03-23T18:09:32.855633+0000) 2024-03-23T18:10:07.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:07 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:07.209+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.057340+0000 front 2024-03-23T18:09:15.057318+0000 (oldest deadline 2024-03-23T18:09:37.357408+0000) 2024-03-23T18:10:07.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:07 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:07.209+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:20.858463+0000 front 2024-03-23T18:09:20.858478+0000 (oldest deadline 2024-03-23T18:09:46.758574+0000) 2024-03-23T18:10:07.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:07 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:07.209+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-03-23T18:09:26.759276+0000 front 2024-03-23T18:09:26.759351+0000 (oldest deadline 2024-03-23T18:09:52.659216+0000) 2024-03-23T18:10:07.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:07 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:07.209+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-03-23T18:09:36.160352+0000 front 2024-03-23T18:09:36.160374+0000 (oldest deadline 2024-03-23T18:10:00.761076+0000) 2024-03-23T18:10:08.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:10:07 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:10:07.644+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.122328+0000 front 2024-03-23T18:09:00.122374+0000 (oldest deadline 2024-03-23T18:09:24.122945+0000) 2024-03-23T18:10:08.017 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:10:07 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:10:07.644+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:05.125484+0000 front 2024-03-23T18:09:05.125462+0000 (oldest deadline 2024-03-23T18:09:33.326776+0000) 2024-03-23T18:10:08.017 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:10:07 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:10:07.644+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:13.327558+0000 front 2024-03-23T18:09:13.327314+0000 (oldest deadline 2024-03-23T18:09:39.227552+0000) 2024-03-23T18:10:08.017 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:10:07 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:10:07.644+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:23.129785+0000 front 2024-03-23T18:09:23.129607+0000 (oldest deadline 2024-03-23T18:09:49.029801+0000) 2024-03-23T18:10:08.017 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:10:07 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:10:07.644+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-03-23T18:09:37.231501+0000 front 2024-03-23T18:09:37.231420+0000 (oldest deadline 2024-03-23T18:10:01.331392+0000) 2024-03-23T18:10:08.017 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:10:07 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9[81974]: 2024-03-23T18:10:07.644+0000 7fa316800700 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-03-23T18:09:41.832731+0000 front 2024-03-23T18:09:41.832710+0000 (oldest deadline 2024-03-23T18:10:06.532732+0000) 2024-03-23T18:10:08.585 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:08 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:08.184+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.653471+0000 front 2024-03-23T18:09:00.653556+0000 (oldest deadline 2024-03-23T18:09:22.953547+0000) 2024-03-23T18:10:08.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:08 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:08.184+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.155613+0000 front 2024-03-23T18:09:08.155550+0000 (oldest deadline 2024-03-23T18:09:32.855633+0000) 2024-03-23T18:10:08.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:08 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:08.184+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.057340+0000 front 2024-03-23T18:09:15.057318+0000 (oldest deadline 2024-03-23T18:09:37.357408+0000) 2024-03-23T18:10:08.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:08 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:08.184+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:20.858463+0000 front 2024-03-23T18:09:20.858478+0000 (oldest deadline 2024-03-23T18:09:46.758574+0000) 2024-03-23T18:10:08.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:08 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:08.184+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-03-23T18:09:26.759276+0000 front 2024-03-23T18:09:26.759351+0000 (oldest deadline 2024-03-23T18:09:52.659216+0000) 2024-03-23T18:10:08.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:08 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:08.184+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-03-23T18:09:36.160352+0000 front 2024-03-23T18:09:36.160374+0000 (oldest deadline 2024-03-23T18:10:00.761076+0000) 2024-03-23T18:10:08.807 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:08 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:08.313+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:59.796052+0000 front 2024-03-23T18:08:59.796002+0000 (oldest deadline 2024-03-23T18:09:23.895742+0000) 2024-03-23T18:10:08.807 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:08 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:08.313+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.796583+0000 front 2024-03-23T18:09:06.796752+0000 (oldest deadline 2024-03-23T18:09:30.896916+0000) 2024-03-23T18:10:08.807 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:08 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:08.313+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.498831+0000 front 2024-03-23T18:09:15.498800+0000 (oldest deadline 2024-03-23T18:09:39.598988+0000) 2024-03-23T18:10:08.807 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:08 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:08.313+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:21.900315+0000 front 2024-03-23T18:09:21.900091+0000 (oldest deadline 2024-03-23T18:09:45.400070+0000) 2024-03-23T18:10:08.807 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:08 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:08.313+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-03-23T18:09:29.501333+0000 front 2024-03-23T18:09:29.501303+0000 (oldest deadline 2024-03-23T18:09:53.001492+0000) 2024-03-23T18:10:08.807 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:08 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:08.313+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-03-23T18:09:37.102661+0000 front 2024-03-23T18:09:37.102759+0000 (oldest deadline 2024-03-23T18:10:00.002501+0000) 2024-03-23T18:10:08.807 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:08 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:08.313+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-03-23T18:09:45.303491+0000 front 2024-03-23T18:09:45.303537+0000 (oldest deadline 2024-03-23T18:10:06.403399+0000) 2024-03-23T18:10:09.016 INFO:journalctl@ceph.osd.9.smithi086.stdout:Mar 23 18:10:08 smithi086 bash[119536]: ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-9 2024-03-23T18:10:09.585 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:09 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:09.148+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.653471+0000 front 2024-03-23T18:09:00.653556+0000 (oldest deadline 2024-03-23T18:09:22.953547+0000) 2024-03-23T18:10:09.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:09 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:09.148+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.155613+0000 front 2024-03-23T18:09:08.155550+0000 (oldest deadline 2024-03-23T18:09:32.855633+0000) 2024-03-23T18:10:09.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:09 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:09.148+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.057340+0000 front 2024-03-23T18:09:15.057318+0000 (oldest deadline 2024-03-23T18:09:37.357408+0000) 2024-03-23T18:10:09.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:09 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:09.148+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:20.858463+0000 front 2024-03-23T18:09:20.858478+0000 (oldest deadline 2024-03-23T18:09:46.758574+0000) 2024-03-23T18:10:09.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:09 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:09.148+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-03-23T18:09:26.759276+0000 front 2024-03-23T18:09:26.759351+0000 (oldest deadline 2024-03-23T18:09:52.659216+0000) 2024-03-23T18:10:09.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:09 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:09.148+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-03-23T18:09:36.160352+0000 front 2024-03-23T18:09:36.160374+0000 (oldest deadline 2024-03-23T18:10:00.761076+0000) 2024-03-23T18:10:09.807 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:09 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:09.348+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:59.796052+0000 front 2024-03-23T18:08:59.796002+0000 (oldest deadline 2024-03-23T18:09:23.895742+0000) 2024-03-23T18:10:09.807 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:09 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:09.348+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.796583+0000 front 2024-03-23T18:09:06.796752+0000 (oldest deadline 2024-03-23T18:09:30.896916+0000) 2024-03-23T18:10:09.807 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:09 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:09.348+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.498831+0000 front 2024-03-23T18:09:15.498800+0000 (oldest deadline 2024-03-23T18:09:39.598988+0000) 2024-03-23T18:10:09.807 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:09 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:09.348+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:21.900315+0000 front 2024-03-23T18:09:21.900091+0000 (oldest deadline 2024-03-23T18:09:45.400070+0000) 2024-03-23T18:10:09.807 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:09 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:09.348+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-03-23T18:09:29.501333+0000 front 2024-03-23T18:09:29.501303+0000 (oldest deadline 2024-03-23T18:09:53.001492+0000) 2024-03-23T18:10:09.807 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:09 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:09.348+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-03-23T18:09:37.102661+0000 front 2024-03-23T18:09:37.102759+0000 (oldest deadline 2024-03-23T18:10:00.002501+0000) 2024-03-23T18:10:09.807 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:09 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:09.348+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-03-23T18:09:45.303491+0000 front 2024-03-23T18:09:45.303537+0000 (oldest deadline 2024-03-23T18:10:06.403399+0000) 2024-03-23T18:10:10.171 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@osd.9.service' 2024-03-23T18:10:10.255 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T18:10:10.256 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-03-23T18:10:10.256 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-03-23T18:10:10.256 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@osd.10 2024-03-23T18:10:10.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:10 smithi092 systemd[1]: Stopping Ceph osd.10 for 9d76c9e8-e93c-11ee-95cd-87774f69a715... 2024-03-23T18:10:10.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:10 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:10.355+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:59.796052+0000 front 2024-03-23T18:08:59.796002+0000 (oldest deadline 2024-03-23T18:09:23.895742+0000) 2024-03-23T18:10:10.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:10 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:10.355+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.796583+0000 front 2024-03-23T18:09:06.796752+0000 (oldest deadline 2024-03-23T18:09:30.896916+0000) 2024-03-23T18:10:10.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:10 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:10.355+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.498831+0000 front 2024-03-23T18:09:15.498800+0000 (oldest deadline 2024-03-23T18:09:39.598988+0000) 2024-03-23T18:10:10.558 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:10 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:10.355+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:21.900315+0000 front 2024-03-23T18:09:21.900091+0000 (oldest deadline 2024-03-23T18:09:45.400070+0000) 2024-03-23T18:10:10.558 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:10 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:10.355+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-03-23T18:09:29.501333+0000 front 2024-03-23T18:09:29.501303+0000 (oldest deadline 2024-03-23T18:09:53.001492+0000) 2024-03-23T18:10:10.558 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:10 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:10.355+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-03-23T18:09:37.102661+0000 front 2024-03-23T18:09:37.102759+0000 (oldest deadline 2024-03-23T18:10:00.002501+0000) 2024-03-23T18:10:10.558 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:10 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:10.355+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-03-23T18:09:45.303491+0000 front 2024-03-23T18:09:45.303537+0000 (oldest deadline 2024-03-23T18:10:06.403399+0000) 2024-03-23T18:10:10.558 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:10 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:10.504+0000 7f7ab1a97700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T18:10:10.558 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:10 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:10.504+0000 7f7ab1a97700 -1 osd.10 113 *** Got signal Terminated *** 2024-03-23T18:10:10.558 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:10 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:10.504+0000 7f7ab1a97700 -1 osd.10 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-23T18:10:10.585 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:10 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:10.156+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.653471+0000 front 2024-03-23T18:09:00.653556+0000 (oldest deadline 2024-03-23T18:09:22.953547+0000) 2024-03-23T18:10:10.585 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:10 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:10.156+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.155613+0000 front 2024-03-23T18:09:08.155550+0000 (oldest deadline 2024-03-23T18:09:32.855633+0000) 2024-03-23T18:10:10.585 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:10 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:10.156+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.057340+0000 front 2024-03-23T18:09:15.057318+0000 (oldest deadline 2024-03-23T18:09:37.357408+0000) 2024-03-23T18:10:10.585 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:10 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:10.156+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:20.858463+0000 front 2024-03-23T18:09:20.858478+0000 (oldest deadline 2024-03-23T18:09:46.758574+0000) 2024-03-23T18:10:10.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:10 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:10.156+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-03-23T18:09:26.759276+0000 front 2024-03-23T18:09:26.759351+0000 (oldest deadline 2024-03-23T18:09:52.659216+0000) 2024-03-23T18:10:10.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:10 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:10.156+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-03-23T18:09:36.160352+0000 front 2024-03-23T18:09:36.160374+0000 (oldest deadline 2024-03-23T18:10:00.761076+0000) 2024-03-23T18:10:11.585 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:11 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:11.126+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.653471+0000 front 2024-03-23T18:09:00.653556+0000 (oldest deadline 2024-03-23T18:09:22.953547+0000) 2024-03-23T18:10:11.585 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:11 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:11.126+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.155613+0000 front 2024-03-23T18:09:08.155550+0000 (oldest deadline 2024-03-23T18:09:32.855633+0000) 2024-03-23T18:10:11.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:11 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:11.126+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.057340+0000 front 2024-03-23T18:09:15.057318+0000 (oldest deadline 2024-03-23T18:09:37.357408+0000) 2024-03-23T18:10:11.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:11 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:11.126+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:20.858463+0000 front 2024-03-23T18:09:20.858478+0000 (oldest deadline 2024-03-23T18:09:46.758574+0000) 2024-03-23T18:10:11.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:11 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:11.126+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-03-23T18:09:26.759276+0000 front 2024-03-23T18:09:26.759351+0000 (oldest deadline 2024-03-23T18:09:52.659216+0000) 2024-03-23T18:10:11.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:11 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:11.126+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-03-23T18:09:36.160352+0000 front 2024-03-23T18:09:36.160374+0000 (oldest deadline 2024-03-23T18:10:00.761076+0000) 2024-03-23T18:10:11.807 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:11 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:11.379+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:59.796052+0000 front 2024-03-23T18:08:59.796002+0000 (oldest deadline 2024-03-23T18:09:23.895742+0000) 2024-03-23T18:10:11.807 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:11 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:11.379+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.796583+0000 front 2024-03-23T18:09:06.796752+0000 (oldest deadline 2024-03-23T18:09:30.896916+0000) 2024-03-23T18:10:11.807 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:11 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:11.379+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.498831+0000 front 2024-03-23T18:09:15.498800+0000 (oldest deadline 2024-03-23T18:09:39.598988+0000) 2024-03-23T18:10:11.807 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:11 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:11.379+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:21.900315+0000 front 2024-03-23T18:09:21.900091+0000 (oldest deadline 2024-03-23T18:09:45.400070+0000) 2024-03-23T18:10:11.807 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:11 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:11.379+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-03-23T18:09:29.501333+0000 front 2024-03-23T18:09:29.501303+0000 (oldest deadline 2024-03-23T18:09:53.001492+0000) 2024-03-23T18:10:11.807 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:11 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:11.379+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-03-23T18:09:37.102661+0000 front 2024-03-23T18:09:37.102759+0000 (oldest deadline 2024-03-23T18:10:00.002501+0000) 2024-03-23T18:10:11.808 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:11 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:11.379+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-03-23T18:09:45.303491+0000 front 2024-03-23T18:09:45.303537+0000 (oldest deadline 2024-03-23T18:10:06.403399+0000) 2024-03-23T18:10:12.585 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:12 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:12.089+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.653471+0000 front 2024-03-23T18:09:00.653556+0000 (oldest deadline 2024-03-23T18:09:22.953547+0000) 2024-03-23T18:10:12.585 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:12 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:12.089+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.155613+0000 front 2024-03-23T18:09:08.155550+0000 (oldest deadline 2024-03-23T18:09:32.855633+0000) 2024-03-23T18:10:12.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:12 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:12.089+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.057340+0000 front 2024-03-23T18:09:15.057318+0000 (oldest deadline 2024-03-23T18:09:37.357408+0000) 2024-03-23T18:10:12.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:12 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:12.089+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:20.858463+0000 front 2024-03-23T18:09:20.858478+0000 (oldest deadline 2024-03-23T18:09:46.758574+0000) 2024-03-23T18:10:12.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:12 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:12.089+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-03-23T18:09:26.759276+0000 front 2024-03-23T18:09:26.759351+0000 (oldest deadline 2024-03-23T18:09:52.659216+0000) 2024-03-23T18:10:12.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:12 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:12.089+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-03-23T18:09:36.160352+0000 front 2024-03-23T18:09:36.160374+0000 (oldest deadline 2024-03-23T18:10:00.761076+0000) 2024-03-23T18:10:12.807 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:12 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:12.335+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:59.796052+0000 front 2024-03-23T18:08:59.796002+0000 (oldest deadline 2024-03-23T18:09:23.895742+0000) 2024-03-23T18:10:12.807 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:12 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:12.335+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.796583+0000 front 2024-03-23T18:09:06.796752+0000 (oldest deadline 2024-03-23T18:09:30.896916+0000) 2024-03-23T18:10:12.807 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:12 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:12.335+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.498831+0000 front 2024-03-23T18:09:15.498800+0000 (oldest deadline 2024-03-23T18:09:39.598988+0000) 2024-03-23T18:10:12.807 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:12 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:12.335+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:21.900315+0000 front 2024-03-23T18:09:21.900091+0000 (oldest deadline 2024-03-23T18:09:45.400070+0000) 2024-03-23T18:10:12.807 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:12 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:12.335+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-03-23T18:09:29.501333+0000 front 2024-03-23T18:09:29.501303+0000 (oldest deadline 2024-03-23T18:09:53.001492+0000) 2024-03-23T18:10:12.808 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:12 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:12.335+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-03-23T18:09:37.102661+0000 front 2024-03-23T18:09:37.102759+0000 (oldest deadline 2024-03-23T18:10:00.002501+0000) 2024-03-23T18:10:12.808 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:12 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:12.335+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-03-23T18:09:45.303491+0000 front 2024-03-23T18:09:45.303537+0000 (oldest deadline 2024-03-23T18:10:06.403399+0000) 2024-03-23T18:10:13.338 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:13 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:13.053+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.653471+0000 front 2024-03-23T18:09:00.653556+0000 (oldest deadline 2024-03-23T18:09:22.953547+0000) 2024-03-23T18:10:13.339 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:13 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:13.053+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.155613+0000 front 2024-03-23T18:09:08.155550+0000 (oldest deadline 2024-03-23T18:09:32.855633+0000) 2024-03-23T18:10:13.339 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:13 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:13.053+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.057340+0000 front 2024-03-23T18:09:15.057318+0000 (oldest deadline 2024-03-23T18:09:37.357408+0000) 2024-03-23T18:10:13.339 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:13 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:13.053+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:20.858463+0000 front 2024-03-23T18:09:20.858478+0000 (oldest deadline 2024-03-23T18:09:46.758574+0000) 2024-03-23T18:10:13.339 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:13 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:13.053+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-03-23T18:09:26.759276+0000 front 2024-03-23T18:09:26.759351+0000 (oldest deadline 2024-03-23T18:09:52.659216+0000) 2024-03-23T18:10:13.339 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:13 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:13.053+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-03-23T18:09:36.160352+0000 front 2024-03-23T18:09:36.160374+0000 (oldest deadline 2024-03-23T18:10:00.761076+0000) 2024-03-23T18:10:13.807 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:13 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:13.343+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:59.796052+0000 front 2024-03-23T18:08:59.796002+0000 (oldest deadline 2024-03-23T18:09:23.895742+0000) 2024-03-23T18:10:13.807 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:13 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:13.343+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.796583+0000 front 2024-03-23T18:09:06.796752+0000 (oldest deadline 2024-03-23T18:09:30.896916+0000) 2024-03-23T18:10:13.807 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:13 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:13.343+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.498831+0000 front 2024-03-23T18:09:15.498800+0000 (oldest deadline 2024-03-23T18:09:39.598988+0000) 2024-03-23T18:10:13.807 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:13 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:13.343+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:21.900315+0000 front 2024-03-23T18:09:21.900091+0000 (oldest deadline 2024-03-23T18:09:45.400070+0000) 2024-03-23T18:10:13.807 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:13 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:13.343+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-03-23T18:09:29.501333+0000 front 2024-03-23T18:09:29.501303+0000 (oldest deadline 2024-03-23T18:09:53.001492+0000) 2024-03-23T18:10:13.807 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:13 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:13.343+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-03-23T18:09:37.102661+0000 front 2024-03-23T18:09:37.102759+0000 (oldest deadline 2024-03-23T18:10:00.002501+0000) 2024-03-23T18:10:13.808 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:13 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:13.343+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-03-23T18:09:45.303491+0000 front 2024-03-23T18:09:45.303537+0000 (oldest deadline 2024-03-23T18:10:06.403399+0000) 2024-03-23T18:10:14.335 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:14 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:14.047+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.653471+0000 front 2024-03-23T18:09:00.653556+0000 (oldest deadline 2024-03-23T18:09:22.953547+0000) 2024-03-23T18:10:14.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:14 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:14.047+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.155613+0000 front 2024-03-23T18:09:08.155550+0000 (oldest deadline 2024-03-23T18:09:32.855633+0000) 2024-03-23T18:10:14.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:14 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:14.047+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.057340+0000 front 2024-03-23T18:09:15.057318+0000 (oldest deadline 2024-03-23T18:09:37.357408+0000) 2024-03-23T18:10:14.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:14 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:14.047+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:20.858463+0000 front 2024-03-23T18:09:20.858478+0000 (oldest deadline 2024-03-23T18:09:46.758574+0000) 2024-03-23T18:10:14.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:14 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:14.047+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-03-23T18:09:26.759276+0000 front 2024-03-23T18:09:26.759351+0000 (oldest deadline 2024-03-23T18:09:52.659216+0000) 2024-03-23T18:10:14.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:14 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:14.047+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-03-23T18:09:36.160352+0000 front 2024-03-23T18:09:36.160374+0000 (oldest deadline 2024-03-23T18:10:00.761076+0000) 2024-03-23T18:10:14.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:14 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:14.300+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:59.796052+0000 front 2024-03-23T18:08:59.796002+0000 (oldest deadline 2024-03-23T18:09:23.895742+0000) 2024-03-23T18:10:14.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:14 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:14.300+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.796583+0000 front 2024-03-23T18:09:06.796752+0000 (oldest deadline 2024-03-23T18:09:30.896916+0000) 2024-03-23T18:10:14.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:14 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:14.300+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.498831+0000 front 2024-03-23T18:09:15.498800+0000 (oldest deadline 2024-03-23T18:09:39.598988+0000) 2024-03-23T18:10:14.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:14 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:14.300+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:21.900315+0000 front 2024-03-23T18:09:21.900091+0000 (oldest deadline 2024-03-23T18:09:45.400070+0000) 2024-03-23T18:10:14.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:14 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:14.300+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-03-23T18:09:29.501333+0000 front 2024-03-23T18:09:29.501303+0000 (oldest deadline 2024-03-23T18:09:53.001492+0000) 2024-03-23T18:10:14.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:14 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:14.300+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-03-23T18:09:37.102661+0000 front 2024-03-23T18:09:37.102759+0000 (oldest deadline 2024-03-23T18:10:00.002501+0000) 2024-03-23T18:10:14.558 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:14 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:14.300+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-03-23T18:09:45.303491+0000 front 2024-03-23T18:09:45.303537+0000 (oldest deadline 2024-03-23T18:10:06.403399+0000) 2024-03-23T18:10:15.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:15 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:15.292+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:08:59.796052+0000 front 2024-03-23T18:08:59.796002+0000 (oldest deadline 2024-03-23T18:09:23.895742+0000) 2024-03-23T18:10:15.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:15 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:15.292+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:06.796583+0000 front 2024-03-23T18:09:06.796752+0000 (oldest deadline 2024-03-23T18:09:30.896916+0000) 2024-03-23T18:10:15.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:15 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:15.292+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.498831+0000 front 2024-03-23T18:09:15.498800+0000 (oldest deadline 2024-03-23T18:09:39.598988+0000) 2024-03-23T18:10:15.557 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:15 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:15.292+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:21.900315+0000 front 2024-03-23T18:09:21.900091+0000 (oldest deadline 2024-03-23T18:09:45.400070+0000) 2024-03-23T18:10:15.558 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:15 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:15.292+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-03-23T18:09:29.501333+0000 front 2024-03-23T18:09:29.501303+0000 (oldest deadline 2024-03-23T18:09:53.001492+0000) 2024-03-23T18:10:15.558 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:15 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:15.292+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-03-23T18:09:37.102661+0000 front 2024-03-23T18:09:37.102759+0000 (oldest deadline 2024-03-23T18:10:00.002501+0000) 2024-03-23T18:10:15.558 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:15 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10[108847]: 2024-03-23T18:10:15.292+0000 7f7aae88e700 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-03-23T18:09:45.303491+0000 front 2024-03-23T18:09:45.303537+0000 (oldest deadline 2024-03-23T18:10:06.403399+0000) 2024-03-23T18:10:15.585 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:15 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:15.091+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.653471+0000 front 2024-03-23T18:09:00.653556+0000 (oldest deadline 2024-03-23T18:09:22.953547+0000) 2024-03-23T18:10:15.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:15 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:15.091+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.155613+0000 front 2024-03-23T18:09:08.155550+0000 (oldest deadline 2024-03-23T18:09:32.855633+0000) 2024-03-23T18:10:15.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:15 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:15.091+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.057340+0000 front 2024-03-23T18:09:15.057318+0000 (oldest deadline 2024-03-23T18:09:37.357408+0000) 2024-03-23T18:10:15.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:15 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:15.091+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:20.858463+0000 front 2024-03-23T18:09:20.858478+0000 (oldest deadline 2024-03-23T18:09:46.758574+0000) 2024-03-23T18:10:15.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:15 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:15.091+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-03-23T18:09:26.759276+0000 front 2024-03-23T18:09:26.759351+0000 (oldest deadline 2024-03-23T18:09:52.659216+0000) 2024-03-23T18:10:15.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:15 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:15.091+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-03-23T18:09:36.160352+0000 front 2024-03-23T18:09:36.160374+0000 (oldest deadline 2024-03-23T18:10:00.761076+0000) 2024-03-23T18:10:16.335 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:16 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:16.081+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.653471+0000 front 2024-03-23T18:09:00.653556+0000 (oldest deadline 2024-03-23T18:09:22.953547+0000) 2024-03-23T18:10:16.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:16 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:16.081+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.155613+0000 front 2024-03-23T18:09:08.155550+0000 (oldest deadline 2024-03-23T18:09:32.855633+0000) 2024-03-23T18:10:16.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:16 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:16.081+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.057340+0000 front 2024-03-23T18:09:15.057318+0000 (oldest deadline 2024-03-23T18:09:37.357408+0000) 2024-03-23T18:10:16.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:16 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:16.081+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:20.858463+0000 front 2024-03-23T18:09:20.858478+0000 (oldest deadline 2024-03-23T18:09:46.758574+0000) 2024-03-23T18:10:16.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:16 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:16.081+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-03-23T18:09:26.759276+0000 front 2024-03-23T18:09:26.759351+0000 (oldest deadline 2024-03-23T18:09:52.659216+0000) 2024-03-23T18:10:16.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:16 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:16.081+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-03-23T18:09:36.160352+0000 front 2024-03-23T18:09:36.160374+0000 (oldest deadline 2024-03-23T18:10:00.761076+0000) 2024-03-23T18:10:16.556 INFO:journalctl@ceph.osd.10.smithi092.stdout:Mar 23 18:10:16 smithi092 bash[123648]: ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-10 2024-03-23T18:10:17.325 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@osd.10.service' 2024-03-23T18:10:17.335 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:17 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:17.039+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.653471+0000 front 2024-03-23T18:09:00.653556+0000 (oldest deadline 2024-03-23T18:09:22.953547+0000) 2024-03-23T18:10:17.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:17 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:17.039+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.155613+0000 front 2024-03-23T18:09:08.155550+0000 (oldest deadline 2024-03-23T18:09:32.855633+0000) 2024-03-23T18:10:17.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:17 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:17.039+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.057340+0000 front 2024-03-23T18:09:15.057318+0000 (oldest deadline 2024-03-23T18:09:37.357408+0000) 2024-03-23T18:10:17.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:17 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:17.039+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:20.858463+0000 front 2024-03-23T18:09:20.858478+0000 (oldest deadline 2024-03-23T18:09:46.758574+0000) 2024-03-23T18:10:17.337 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:17 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:17.039+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-03-23T18:09:26.759276+0000 front 2024-03-23T18:09:26.759351+0000 (oldest deadline 2024-03-23T18:09:52.659216+0000) 2024-03-23T18:10:17.337 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:17 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:17.039+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-03-23T18:09:36.160352+0000 front 2024-03-23T18:09:36.160374+0000 (oldest deadline 2024-03-23T18:10:00.761076+0000) 2024-03-23T18:10:17.337 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:17 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:17.039+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6820 osd.7 since back 2024-03-23T18:09:48.362606+0000 front 2024-03-23T18:09:48.362623+0000 (oldest deadline 2024-03-23T18:10:16.563182+0000) 2024-03-23T18:10:17.366 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T18:10:17.367 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-03-23T18:10:17.367 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-03-23T18:10:17.368 DEBUG:teuthology.orchestra.run.smithi152:> sudo systemctl stop ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@osd.11 2024-03-23T18:10:17.835 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:17 smithi152 systemd[1]: Stopping Ceph osd.11 for 9d76c9e8-e93c-11ee-95cd-87774f69a715... 2024-03-23T18:10:17.836 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:17 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:17.606+0000 7fcac777e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T18:10:17.836 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:17 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:17.607+0000 7fcac777e700 -1 osd.11 113 *** Got signal Terminated *** 2024-03-23T18:10:17.836 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:17 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:17.607+0000 7fcac777e700 -1 osd.11 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-23T18:10:18.335 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:18 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:18.039+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.653471+0000 front 2024-03-23T18:09:00.653556+0000 (oldest deadline 2024-03-23T18:09:22.953547+0000) 2024-03-23T18:10:18.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:18 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:18.039+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.155613+0000 front 2024-03-23T18:09:08.155550+0000 (oldest deadline 2024-03-23T18:09:32.855633+0000) 2024-03-23T18:10:18.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:18 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:18.039+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.057340+0000 front 2024-03-23T18:09:15.057318+0000 (oldest deadline 2024-03-23T18:09:37.357408+0000) 2024-03-23T18:10:18.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:18 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:18.039+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:20.858463+0000 front 2024-03-23T18:09:20.858478+0000 (oldest deadline 2024-03-23T18:09:46.758574+0000) 2024-03-23T18:10:18.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:18 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:18.039+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-03-23T18:09:26.759276+0000 front 2024-03-23T18:09:26.759351+0000 (oldest deadline 2024-03-23T18:09:52.659216+0000) 2024-03-23T18:10:18.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:18 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:18.039+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-03-23T18:09:36.160352+0000 front 2024-03-23T18:09:36.160374+0000 (oldest deadline 2024-03-23T18:10:00.761076+0000) 2024-03-23T18:10:18.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:18 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:18.039+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6820 osd.7 since back 2024-03-23T18:09:48.362606+0000 front 2024-03-23T18:09:48.362623+0000 (oldest deadline 2024-03-23T18:10:16.563182+0000) 2024-03-23T18:10:19.335 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:19 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:19.039+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.653471+0000 front 2024-03-23T18:09:00.653556+0000 (oldest deadline 2024-03-23T18:09:22.953547+0000) 2024-03-23T18:10:19.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:19 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:19.039+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.155613+0000 front 2024-03-23T18:09:08.155550+0000 (oldest deadline 2024-03-23T18:09:32.855633+0000) 2024-03-23T18:10:19.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:19 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:19.039+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.057340+0000 front 2024-03-23T18:09:15.057318+0000 (oldest deadline 2024-03-23T18:09:37.357408+0000) 2024-03-23T18:10:19.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:19 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:19.039+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:20.858463+0000 front 2024-03-23T18:09:20.858478+0000 (oldest deadline 2024-03-23T18:09:46.758574+0000) 2024-03-23T18:10:19.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:19 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:19.039+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-03-23T18:09:26.759276+0000 front 2024-03-23T18:09:26.759351+0000 (oldest deadline 2024-03-23T18:09:52.659216+0000) 2024-03-23T18:10:19.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:19 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:19.039+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-03-23T18:09:36.160352+0000 front 2024-03-23T18:09:36.160374+0000 (oldest deadline 2024-03-23T18:10:00.761076+0000) 2024-03-23T18:10:19.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:19 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:19.039+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6820 osd.7 since back 2024-03-23T18:09:48.362606+0000 front 2024-03-23T18:09:48.362623+0000 (oldest deadline 2024-03-23T18:10:16.563182+0000) 2024-03-23T18:10:20.335 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:20 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:20.075+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.653471+0000 front 2024-03-23T18:09:00.653556+0000 (oldest deadline 2024-03-23T18:09:22.953547+0000) 2024-03-23T18:10:20.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:20 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:20.075+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.155613+0000 front 2024-03-23T18:09:08.155550+0000 (oldest deadline 2024-03-23T18:09:32.855633+0000) 2024-03-23T18:10:20.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:20 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:20.075+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.057340+0000 front 2024-03-23T18:09:15.057318+0000 (oldest deadline 2024-03-23T18:09:37.357408+0000) 2024-03-23T18:10:20.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:20 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:20.075+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:20.858463+0000 front 2024-03-23T18:09:20.858478+0000 (oldest deadline 2024-03-23T18:09:46.758574+0000) 2024-03-23T18:10:20.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:20 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:20.075+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-03-23T18:09:26.759276+0000 front 2024-03-23T18:09:26.759351+0000 (oldest deadline 2024-03-23T18:09:52.659216+0000) 2024-03-23T18:10:20.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:20 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:20.075+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-03-23T18:09:36.160352+0000 front 2024-03-23T18:09:36.160374+0000 (oldest deadline 2024-03-23T18:10:00.761076+0000) 2024-03-23T18:10:20.336 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:20 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:20.075+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6820 osd.7 since back 2024-03-23T18:09:48.362606+0000 front 2024-03-23T18:09:48.362623+0000 (oldest deadline 2024-03-23T18:10:16.563182+0000) 2024-03-23T18:10:21.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:21 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:21.102+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.653471+0000 front 2024-03-23T18:09:00.653556+0000 (oldest deadline 2024-03-23T18:09:22.953547+0000) 2024-03-23T18:10:21.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:21 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:21.102+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.155613+0000 front 2024-03-23T18:09:08.155550+0000 (oldest deadline 2024-03-23T18:09:32.855633+0000) 2024-03-23T18:10:21.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:21 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:21.102+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.057340+0000 front 2024-03-23T18:09:15.057318+0000 (oldest deadline 2024-03-23T18:09:37.357408+0000) 2024-03-23T18:10:21.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:21 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:21.102+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:20.858463+0000 front 2024-03-23T18:09:20.858478+0000 (oldest deadline 2024-03-23T18:09:46.758574+0000) 2024-03-23T18:10:21.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:21 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:21.102+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-03-23T18:09:26.759276+0000 front 2024-03-23T18:09:26.759351+0000 (oldest deadline 2024-03-23T18:09:52.659216+0000) 2024-03-23T18:10:21.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:21 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:21.102+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-03-23T18:09:36.160352+0000 front 2024-03-23T18:09:36.160374+0000 (oldest deadline 2024-03-23T18:10:00.761076+0000) 2024-03-23T18:10:21.587 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:21 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:21.102+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6820 osd.7 since back 2024-03-23T18:09:48.362606+0000 front 2024-03-23T18:09:48.362623+0000 (oldest deadline 2024-03-23T18:10:16.563182+0000) 2024-03-23T18:10:22.585 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:22 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:22.112+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-03-23T18:09:00.653471+0000 front 2024-03-23T18:09:00.653556+0000 (oldest deadline 2024-03-23T18:09:22.953547+0000) 2024-03-23T18:10:22.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:22 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:22.112+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-03-23T18:09:08.155613+0000 front 2024-03-23T18:09:08.155550+0000 (oldest deadline 2024-03-23T18:09:32.855633+0000) 2024-03-23T18:10:22.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:22 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:22.112+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-03-23T18:09:15.057340+0000 front 2024-03-23T18:09:15.057318+0000 (oldest deadline 2024-03-23T18:09:37.357408+0000) 2024-03-23T18:10:22.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:22 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:22.112+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-03-23T18:09:20.858463+0000 front 2024-03-23T18:09:20.858478+0000 (oldest deadline 2024-03-23T18:09:46.758574+0000) 2024-03-23T18:10:22.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:22 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:22.112+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-03-23T18:09:26.759276+0000 front 2024-03-23T18:09:26.759351+0000 (oldest deadline 2024-03-23T18:09:52.659216+0000) 2024-03-23T18:10:22.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:22 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:22.112+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-03-23T18:09:36.160352+0000 front 2024-03-23T18:09:36.160374+0000 (oldest deadline 2024-03-23T18:10:00.761076+0000) 2024-03-23T18:10:22.586 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:22 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11[78460]: 2024-03-23T18:10:22.112+0000 7fcac056d700 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.92:6820 osd.7 since back 2024-03-23T18:09:48.362606+0000 front 2024-03-23T18:09:48.362623+0000 (oldest deadline 2024-03-23T18:10:16.563182+0000) 2024-03-23T18:10:23.574 INFO:journalctl@ceph.osd.11.smithi152.stdout:Mar 23 18:10:23 smithi152 bash[95180]: ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-osd-11 2024-03-23T18:10:24.273 DEBUG:teuthology.orchestra.run.smithi152:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@osd.11.service' 2024-03-23T18:10:24.314 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T18:10:24.315 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-03-23T18:10:24.315 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-03-23T18:10:24.315 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@mds.a 2024-03-23T18:10:24.699 INFO:journalctl@ceph.mds.a.smithi086.stdout:Mar 23 18:10:24 smithi086 systemd[1]: Stopping Ceph mds.a for 9d76c9e8-e93c-11ee-95cd-87774f69a715... 2024-03-23T18:10:25.016 INFO:journalctl@ceph.mds.a.smithi086.stdout:Mar 23 18:10:24 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-a[90474]: 2024-03-23T18:10:24.697+0000 7f261ac00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T18:10:25.016 INFO:journalctl@ceph.mds.a.smithi086.stdout:Mar 23 18:10:24 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-a[90474]: 2024-03-23T18:10:24.697+0000 7f261ac00700 -1 mds.a *** got signal Terminated *** 2024-03-23T18:10:26.516 INFO:journalctl@ceph.mds.a.smithi086.stdout:Mar 23 18:10:26 smithi086 bash[119796]: ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-a 2024-03-23T18:10:26.580 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@mds.a.service' 2024-03-23T18:10:26.661 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T18:10:26.662 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-03-23T18:10:26.662 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-03-23T18:10:26.662 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@mds.d 2024-03-23T18:10:27.016 INFO:journalctl@ceph.mds.d.smithi086.stdout:Mar 23 18:10:26 smithi086 systemd[1]: Stopping Ceph mds.d for 9d76c9e8-e93c-11ee-95cd-87774f69a715... 2024-03-23T18:10:27.516 INFO:journalctl@ceph.mds.d.smithi086.stdout:Mar 23 18:10:27 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-d[89510]: 2024-03-23T18:10:27.066+0000 7fa196800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T18:10:27.516 INFO:journalctl@ceph.mds.d.smithi086.stdout:Mar 23 18:10:27 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-d[89510]: 2024-03-23T18:10:27.066+0000 7fa196800700 -1 mds.d *** got signal Terminated *** 2024-03-23T18:10:30.490 INFO:journalctl@ceph.mds.d.smithi086.stdout:Mar 23 18:10:30 smithi086 bash[119937]: ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-d 2024-03-23T18:10:30.515 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@mds.d.service' 2024-03-23T18:10:30.593 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T18:10:30.594 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-03-23T18:10:30.595 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-03-23T18:10:30.595 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@mds.g 2024-03-23T18:10:30.766 INFO:journalctl@ceph.mds.g.smithi086.stdout:Mar 23 18:10:30 smithi086 systemd[1]: Stopping Ceph mds.g for 9d76c9e8-e93c-11ee-95cd-87774f69a715... 2024-03-23T18:10:31.266 INFO:journalctl@ceph.mds.g.smithi086.stdout:Mar 23 18:10:30 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-g[90085]: 2024-03-23T18:10:30.947+0000 7f8cfe800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T18:10:31.266 INFO:journalctl@ceph.mds.g.smithi086.stdout:Mar 23 18:10:30 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-g[90085]: 2024-03-23T18:10:30.947+0000 7f8cfe800700 -1 mds.g *** got signal Terminated *** 2024-03-23T18:10:33.766 INFO:journalctl@ceph.mds.g.smithi086.stdout:Mar 23 18:10:33 smithi086 bash[120076]: ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-g 2024-03-23T18:10:33.961 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@mds.g.service' 2024-03-23T18:10:34.039 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T18:10:34.040 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-03-23T18:10:34.040 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-03-23T18:10:34.040 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@mds.j 2024-03-23T18:10:34.266 INFO:journalctl@ceph.mds.j.smithi086.stdout:Mar 23 18:10:34 smithi086 systemd[1]: Stopping Ceph mds.j for 9d76c9e8-e93c-11ee-95cd-87774f69a715... 2024-03-23T18:10:34.766 INFO:journalctl@ceph.mds.j.smithi086.stdout:Mar 23 18:10:34 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-j[89124]: 2024-03-23T18:10:34.413+0000 7f5bcc800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T18:10:34.766 INFO:journalctl@ceph.mds.j.smithi086.stdout:Mar 23 18:10:34 smithi086 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-j[89124]: 2024-03-23T18:10:34.413+0000 7f5bcc800700 -1 mds.j *** got signal Terminated *** 2024-03-23T18:10:36.016 INFO:journalctl@ceph.mds.j.smithi086.stdout:Mar 23 18:10:35 smithi086 bash[120215]: ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-j 2024-03-23T18:10:36.073 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@mds.j.service' 2024-03-23T18:10:36.153 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T18:10:36.154 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-03-23T18:10:36.154 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-03-23T18:10:36.154 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@mds.b 2024-03-23T18:10:36.557 INFO:journalctl@ceph.mds.b.smithi092.stdout:Mar 23 18:10:36 smithi092 systemd[1]: Stopping Ceph mds.b for 9d76c9e8-e93c-11ee-95cd-87774f69a715... 2024-03-23T18:10:36.557 INFO:journalctl@ceph.mds.b.smithi092.stdout:Mar 23 18:10:36 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-b[115122]: 2024-03-23T18:10:36.381+0000 7f91bde8c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T18:10:36.557 INFO:journalctl@ceph.mds.b.smithi092.stdout:Mar 23 18:10:36 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-b[115122]: 2024-03-23T18:10:36.381+0000 7f91bde8c700 -1 mds.b *** got signal Terminated *** 2024-03-23T18:10:38.480 INFO:journalctl@ceph.mds.b.smithi092.stdout:Mar 23 18:10:38 smithi092 bash[123911]: ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-b 2024-03-23T18:10:38.487 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@mds.b.service' 2024-03-23T18:10:38.524 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T18:10:38.525 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-03-23T18:10:38.525 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-03-23T18:10:38.525 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@mds.e 2024-03-23T18:10:38.781 INFO:journalctl@ceph.mds.e.smithi092.stdout:Mar 23 18:10:38 smithi092 systemd[1]: Stopping Ceph mds.e for 9d76c9e8-e93c-11ee-95cd-87774f69a715... 2024-03-23T18:10:39.056 INFO:journalctl@ceph.mds.e.smithi092.stdout:Mar 23 18:10:38 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-e[114438]: 2024-03-23T18:10:38.780+0000 7f23ea543700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T18:10:39.057 INFO:journalctl@ceph.mds.e.smithi092.stdout:Mar 23 18:10:38 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-e[114438]: 2024-03-23T18:10:38.780+0000 7f23ea543700 -1 mds.e *** got signal Terminated *** 2024-03-23T18:10:42.418 INFO:journalctl@ceph.mds.e.smithi092.stdout:Mar 23 18:10:42 smithi092 bash[124049]: ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-e 2024-03-23T18:10:42.424 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@mds.e.service' 2024-03-23T18:10:42.459 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T18:10:42.459 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-03-23T18:10:42.459 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-03-23T18:10:42.459 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@mds.h 2024-03-23T18:10:42.721 INFO:journalctl@ceph.mds.h.smithi092.stdout:Mar 23 18:10:42 smithi092 systemd[1]: Stopping Ceph mds.h for 9d76c9e8-e93c-11ee-95cd-87774f69a715... 2024-03-23T18:10:43.056 INFO:journalctl@ceph.mds.h.smithi092.stdout:Mar 23 18:10:42 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-h[114780]: 2024-03-23T18:10:42.721+0000 7f76ca66c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T18:10:43.057 INFO:journalctl@ceph.mds.h.smithi092.stdout:Mar 23 18:10:42 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-h[114780]: 2024-03-23T18:10:42.721+0000 7f76ca66c700 -1 mds.h *** got signal Terminated *** 2024-03-23T18:10:46.445 INFO:journalctl@ceph.mds.h.smithi092.stdout:Mar 23 18:10:46 smithi092 bash[124187]: ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-h 2024-03-23T18:10:46.450 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@mds.h.service' 2024-03-23T18:10:46.484 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T18:10:46.484 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-03-23T18:10:46.484 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-03-23T18:10:46.485 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@mds.k 2024-03-23T18:10:46.744 INFO:journalctl@ceph.mds.k.smithi092.stdout:Mar 23 18:10:46 smithi092 systemd[1]: Stopping Ceph mds.k for 9d76c9e8-e93c-11ee-95cd-87774f69a715... 2024-03-23T18:10:47.056 INFO:journalctl@ceph.mds.k.smithi092.stdout:Mar 23 18:10:46 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-k[114091]: 2024-03-23T18:10:46.745+0000 7f67d84e0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T18:10:47.057 INFO:journalctl@ceph.mds.k.smithi092.stdout:Mar 23 18:10:46 smithi092 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-k[114091]: 2024-03-23T18:10:46.745+0000 7f67d84e0700 -1 mds.k *** got signal Terminated *** 2024-03-23T18:10:50.466 INFO:journalctl@ceph.mds.k.smithi092.stdout:Mar 23 18:10:50 smithi092 bash[124327]: ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-k 2024-03-23T18:10:50.472 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@mds.k.service' 2024-03-23T18:10:50.507 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T18:10:50.508 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-03-23T18:10:50.508 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-03-23T18:10:50.508 DEBUG:teuthology.orchestra.run.smithi152:> sudo systemctl stop ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@mds.c 2024-03-23T18:10:50.835 INFO:journalctl@ceph.mds.c.smithi152.stdout:Mar 23 18:10:50 smithi152 systemd[1]: Stopping Ceph mds.c for 9d76c9e8-e93c-11ee-95cd-87774f69a715... 2024-03-23T18:10:50.835 INFO:journalctl@ceph.mds.c.smithi152.stdout:Mar 23 18:10:50 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-c[84552]: 2024-03-23T18:10:50.755+0000 7f04c2aa9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T18:10:50.836 INFO:journalctl@ceph.mds.c.smithi152.stdout:Mar 23 18:10:50 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-c[84552]: 2024-03-23T18:10:50.755+0000 7f04c2aa9700 -1 mds.c *** got signal Terminated *** 2024-03-23T18:10:54.310 INFO:journalctl@ceph.mds.c.smithi152.stdout:Mar 23 18:10:54 smithi152 bash[95444]: ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-c 2024-03-23T18:10:54.318 DEBUG:teuthology.orchestra.run.smithi152:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@mds.c.service' 2024-03-23T18:10:54.356 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T18:10:54.357 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-03-23T18:10:54.357 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-03-23T18:10:54.357 DEBUG:teuthology.orchestra.run.smithi152:> sudo systemctl stop ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@mds.f 2024-03-23T18:10:54.586 INFO:journalctl@ceph.mds.f.smithi152.stdout:Mar 23 18:10:54 smithi152 systemd[1]: Stopping Ceph mds.f for 9d76c9e8-e93c-11ee-95cd-87774f69a715... 2024-03-23T18:10:55.085 INFO:journalctl@ceph.mds.f.smithi152.stdout:Mar 23 18:10:54 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-f[84209]: 2024-03-23T18:10:54.629+0000 7f8fd00c1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T18:10:55.085 INFO:journalctl@ceph.mds.f.smithi152.stdout:Mar 23 18:10:54 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-f[84209]: 2024-03-23T18:10:54.629+0000 7f8fd00c1700 -1 mds.f *** got signal Terminated *** 2024-03-23T18:10:58.335 INFO:journalctl@ceph.mds.f.smithi152.stdout:Mar 23 18:10:58 smithi152 bash[95583]: ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-f 2024-03-23T18:10:58.375 DEBUG:teuthology.orchestra.run.smithi152:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@mds.f.service' 2024-03-23T18:10:58.409 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T18:10:58.409 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-03-23T18:10:58.409 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-03-23T18:10:58.410 DEBUG:teuthology.orchestra.run.smithi152:> sudo systemctl stop ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@mds.i 2024-03-23T18:10:58.670 INFO:journalctl@ceph.mds.i.smithi152.stdout:Mar 23 18:10:58 smithi152 systemd[1]: Stopping Ceph mds.i for 9d76c9e8-e93c-11ee-95cd-87774f69a715... 2024-03-23T18:10:59.085 INFO:journalctl@ceph.mds.i.smithi152.stdout:Mar 23 18:10:58 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-i[84894]: 2024-03-23T18:10:58.670+0000 7f5c7098b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T18:10:59.085 INFO:journalctl@ceph.mds.i.smithi152.stdout:Mar 23 18:10:58 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-i[84894]: 2024-03-23T18:10:58.670+0000 7f5c7098b700 -1 mds.i *** got signal Terminated *** 2024-03-23T18:11:02.335 INFO:journalctl@ceph.mds.i.smithi152.stdout:Mar 23 18:11:02 smithi152 bash[95721]: ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-i 2024-03-23T18:11:02.441 DEBUG:teuthology.orchestra.run.smithi152:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@mds.i.service' 2024-03-23T18:11:02.474 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T18:11:02.475 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-03-23T18:11:02.475 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-03-23T18:11:02.475 DEBUG:teuthology.orchestra.run.smithi152:> sudo systemctl stop ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@mds.l 2024-03-23T18:11:02.744 INFO:journalctl@ceph.mds.l.smithi152.stdout:Mar 23 18:11:02 smithi152 systemd[1]: Stopping Ceph mds.l for 9d76c9e8-e93c-11ee-95cd-87774f69a715... 2024-03-23T18:11:03.085 INFO:journalctl@ceph.mds.l.smithi152.stdout:Mar 23 18:11:02 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-l[83861]: 2024-03-23T18:11:02.744+0000 7f96dc6e2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T18:11:03.085 INFO:journalctl@ceph.mds.l.smithi152.stdout:Mar 23 18:11:02 smithi152 ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-l[83861]: 2024-03-23T18:11:02.744+0000 7f96dc6e2700 -1 mds.l *** got signal Terminated *** 2024-03-23T18:11:06.335 INFO:journalctl@ceph.mds.l.smithi152.stdout:Mar 23 18:11:06 smithi152 bash[95860]: ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715-mds-l 2024-03-23T18:11:06.434 DEBUG:teuthology.orchestra.run.smithi152:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d76c9e8-e93c-11ee-95cd-87774f69a715@mds.l.service' 2024-03-23T18:11:06.468 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T18:11:06.468 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-03-23T18:11:06.468 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 --force --keep-logs 2024-03-23T18:11:06.723 INFO:teuthology.orchestra.run.smithi086.stdout:Deleting cluster with fsid: 9d76c9e8-e93c-11ee-95cd-87774f69a715 2024-03-23T18:11:09.572 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 --force --keep-logs 2024-03-23T18:11:09.826 INFO:teuthology.orchestra.run.smithi092.stdout:Deleting cluster with fsid: 9d76c9e8-e93c-11ee-95cd-87774f69a715 2024-03-23T18:11:11.294 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 --force --keep-logs 2024-03-23T18:11:11.481 INFO:teuthology.orchestra.run.smithi152.stdout:Deleting cluster with fsid: 9d76c9e8-e93c-11ee-95cd-87774f69a715 2024-03-23T18:11:12.859 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-23T18:11:12.935 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-23T18:11:12.971 DEBUG:teuthology.orchestra.run.smithi152:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-23T18:11:13.002 INFO:tasks.cephadm:Archiving crash dumps... 2024-03-23T18:11:13.004 DEBUG:teuthology.misc:Transferring archived files from smithi086:/var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/crash to /home/teuthworker/archive/yuriw-2024-03-22_13:15:24-fs-wip-yuri11-testing-2024-03-21-0851-reef-distro-default-smithi/7617485/remote/smithi086/crash 2024-03-23T18:11:13.005 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar c -f - -C /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/crash -- . 2024-03-23T18:11:13.088 INFO:teuthology.orchestra.run.smithi086.stderr:tar: /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/crash: Cannot open: No such file or directory 2024-03-23T18:11:13.089 INFO:teuthology.orchestra.run.smithi086.stderr:tar: Error is not recoverable: exiting now 2024-03-23T18:11:13.092 DEBUG:teuthology.misc:Transferring archived files from smithi092:/var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/crash to /home/teuthworker/archive/yuriw-2024-03-22_13:15:24-fs-wip-yuri11-testing-2024-03-21-0851-reef-distro-default-smithi/7617485/remote/smithi092/crash 2024-03-23T18:11:13.093 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/crash -- . 2024-03-23T18:11:13.142 INFO:teuthology.orchestra.run.smithi092.stderr:tar: /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/crash: Cannot open: No such file or directory 2024-03-23T18:11:13.142 INFO:teuthology.orchestra.run.smithi092.stderr:tar: Error is not recoverable: exiting now 2024-03-23T18:11:13.144 DEBUG:teuthology.misc:Transferring archived files from smithi152:/var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/crash to /home/teuthworker/archive/yuriw-2024-03-22_13:15:24-fs-wip-yuri11-testing-2024-03-21-0851-reef-distro-default-smithi/7617485/remote/smithi152/crash 2024-03-23T18:11:13.145 DEBUG:teuthology.orchestra.run.smithi152:> sudo tar c -f - -C /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/crash -- . 2024-03-23T18:11:13.182 INFO:teuthology.orchestra.run.smithi152.stderr:tar: /var/lib/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/crash: Cannot open: No such file or directory 2024-03-23T18:11:13.182 INFO:teuthology.orchestra.run.smithi152.stderr:tar: Error is not recoverable: exiting now 2024-03-23T18:11:13.183 INFO:tasks.cephadm:Checking cluster log for badness... 2024-03-23T18:11:13.183 DEBUG:teuthology.orchestra.run.smithi086:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-03-23T18:11:13.259 INFO:teuthology.orchestra.run.smithi086.stderr:grep: /var/log/ceph/9d76c9e8-e93c-11ee-95cd-87774f69a715/ceph.log: No such file or directory 2024-03-23T18:11:13.264 INFO:tasks.cephadm:Compressing logs... 2024-03-23T18:11:13.265 DEBUG:teuthology.orchestra.run.smithi086:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-23T18:11:13.269 DEBUG:teuthology.orchestra.run.smithi092:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-23T18:11:13.271 DEBUG:teuthology.orchestra.run.smithi152:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-23T18:11:13.298 INFO:teuthology.orchestra.run.smithi152.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-03-23T18:11:13.315 INFO:teuthology.orchestra.run.smithi092.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-03-23T18:11:13.328 INFO:teuthology.orchestra.run.smithi086.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-03-23T18:18:18.417 INFO:tasks.cephadm:Archiving logs... 2024-03-23T18:18:18.418 DEBUG:teuthology.misc:Transferring archived files from smithi086:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-03-22_13:15:24-fs-wip-yuri11-testing-2024-03-21-0851-reef-distro-default-smithi/7617485/remote/smithi086/log 2024-03-23T18:18:18.419 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-23T18:18:37.971 DEBUG:teuthology.misc:Transferring archived files from smithi092:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-03-22_13:15:24-fs-wip-yuri11-testing-2024-03-21-0851-reef-distro-default-smithi/7617485/remote/smithi092/log 2024-03-23T18:18:37.972 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-23T18:19:30.012 DEBUG:teuthology.misc:Transferring archived files from smithi152:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-03-22_13:15:24-fs-wip-yuri11-testing-2024-03-21-0851-reef-distro-default-smithi/7617485/remote/smithi152/log 2024-03-23T18:19:30.013 DEBUG:teuthology.orchestra.run.smithi152:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-23T18:19:52.008 INFO:tasks.cephadm:Removing cluster... 2024-03-23T18:19:52.008 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 --force 2024-03-23T18:19:52.263 INFO:teuthology.orchestra.run.smithi086.stdout:Deleting cluster with fsid: 9d76c9e8-e93c-11ee-95cd-87774f69a715 2024-03-23T18:19:53.131 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 --force 2024-03-23T18:19:53.507 INFO:teuthology.orchestra.run.smithi092.stdout:Deleting cluster with fsid: 9d76c9e8-e93c-11ee-95cd-87774f69a715 2024-03-23T18:19:54.148 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9d76c9e8-e93c-11ee-95cd-87774f69a715 --force 2024-03-23T18:19:54.342 INFO:teuthology.orchestra.run.smithi152.stdout:Deleting cluster with fsid: 9d76c9e8-e93c-11ee-95cd-87774f69a715 2024-03-23T18:19:54.762 INFO:tasks.cephadm:Removing cephadm ... 2024-03-23T18:19:54.763 DEBUG:teuthology.orchestra.run.smithi086:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-23T18:19:54.810 DEBUG:teuthology.orchestra.run.smithi092:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-23T18:19:54.839 DEBUG:teuthology.orchestra.run.smithi152:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-23T18:19:54.856 INFO:tasks.cephadm:Teardown complete 2024-03-23T18:19:54.857 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-23T18:19:54.868 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-23T18:19:54.869 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-23T18:19:54.873 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-23T18:19:54.882 DEBUG:teuthology.orchestra.run.smithi152:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-23T18:19:55.014 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-23T18:19:55.025 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-23T18:19:55.025 DEBUG:teuthology.orchestra.run.smithi086:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-23T18:19:55.030 DEBUG:teuthology.orchestra.run.smithi092:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-23T18:19:55.032 DEBUG:teuthology.orchestra.run.smithi152:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-23T18:19:55.047 INFO:teuthology.orchestra.run.smithi092.stderr:bash: ntpq: command not found 2024-03-23T18:19:55.049 INFO:teuthology.orchestra.run.smithi152.stderr:bash: ntpq: command not found 2024-03-23T18:19:55.053 INFO:teuthology.orchestra.run.smithi152.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-23T18:19:55.053 INFO:teuthology.orchestra.run.smithi152.stdout:=============================================================================== 2024-03-23T18:19:55.053 INFO:teuthology.orchestra.run.smithi152.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 32 -947us[ -946us] +/- 125ms 2024-03-23T18:19:55.053 INFO:teuthology.orchestra.run.smithi152.stdout:^* hv02.front.sepia.ceph.com 3 6 377 29 -291us[ -290us] +/- 74ms 2024-03-23T18:19:55.054 INFO:teuthology.orchestra.run.smithi152.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 29 +970us[ +971us] +/- 81ms 2024-03-23T18:19:55.054 INFO:teuthology.orchestra.run.smithi152.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T18:19:55.067 INFO:teuthology.orchestra.run.smithi086.stderr:bash: ntpq: command not found 2024-03-23T18:19:55.074 INFO:teuthology.orchestra.run.smithi086.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-23T18:19:55.074 INFO:teuthology.orchestra.run.smithi086.stdout:=============================================================================== 2024-03-23T18:19:55.075 INFO:teuthology.orchestra.run.smithi086.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 26 -933us[ -933us] +/- 125ms 2024-03-23T18:19:55.075 INFO:teuthology.orchestra.run.smithi086.stdout:^* hv02.front.sepia.ceph.com 3 6 377 28 -289us[ -283us] +/- 74ms 2024-03-23T18:19:55.075 INFO:teuthology.orchestra.run.smithi086.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 27 +972us[ +972us] +/- 81ms 2024-03-23T18:19:55.075 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T18:19:55.079 INFO:teuthology.orchestra.run.smithi092.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-23T18:19:55.079 INFO:teuthology.orchestra.run.smithi092.stdout:=============================================================================== 2024-03-23T18:19:55.080 INFO:teuthology.orchestra.run.smithi092.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 28 -988us[ -960us] +/- 125ms 2024-03-23T18:19:55.080 INFO:teuthology.orchestra.run.smithi092.stdout:^* hv02.front.sepia.ceph.com 3 6 377 27 -321us[ -292us] +/- 74ms 2024-03-23T18:19:55.080 INFO:teuthology.orchestra.run.smithi092.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 28 +935us[ +964us] +/- 81ms 2024-03-23T18:19:55.080 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T18:19:55.081 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-23T18:19:55.091 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-23T18:19:55.092 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-23T18:19:55.102 DEBUG:teuthology.orchestra.run.smithi086:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-23T18:19:55.252 DEBUG:teuthology.orchestra.run.smithi092:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-23T18:19:55.341 DEBUG:teuthology.orchestra.run.smithi152:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-23T18:19:55.412 DEBUG:teuthology.orchestra.run.smithi086:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-23T18:19:55.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T18:19:55.482 DEBUG:teuthology.orchestra.run.smithi092:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-23T18:19:55.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T18:19:55.513 DEBUG:teuthology.orchestra.run.smithi152:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-23T18:19:55.543 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T18:19:55.543 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-23T18:19:55.554 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-23T18:19:55.566 INFO:teuthology.task.internal:Duration was 3226.341774 seconds 2024-03-23T18:19:55.566 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-23T18:19:55.578 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-23T18:19:55.578 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-23T18:19:55.582 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-23T18:19:55.585 DEBUG:teuthology.orchestra.run.smithi152:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-23T18:19:55.632 INFO:teuthology.orchestra.run.smithi152.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-23T18:19:55.660 INFO:teuthology.orchestra.run.smithi092.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-23T18:19:55.690 INFO:teuthology.orchestra.run.smithi086.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-23T18:19:56.099 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-23T18:19:56.100 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi086.front.sepia.ceph.com 2024-03-23T18:19:56.100 DEBUG:teuthology.orchestra.run.smithi086:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-23T18:19:56.157 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi092.front.sepia.ceph.com 2024-03-23T18:19:56.157 DEBUG:teuthology.orchestra.run.smithi092:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-23T18:19:56.180 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi152.front.sepia.ceph.com 2024-03-23T18:19:56.181 DEBUG:teuthology.orchestra.run.smithi152:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-23T18:19:56.203 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-23T18:19:56.203 DEBUG:teuthology.orchestra.run.smithi086:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-23T18:19:56.208 DEBUG:teuthology.orchestra.run.smithi092:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-23T18:19:56.223 DEBUG:teuthology.orchestra.run.smithi152:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-23T18:19:56.302 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-23T18:19:56.303 DEBUG:teuthology.orchestra.run.smithi086:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-23T18:19:56.307 DEBUG:teuthology.orchestra.run.smithi092:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-23T18:19:56.309 DEBUG:teuthology.orchestra.run.smithi152:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-23T18:19:56.464 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-23T18:19:56.476 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-23T18:19:56.476 DEBUG:teuthology.orchestra.run.smithi086:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-23T18:19:56.542 DEBUG:teuthology.orchestra.run.smithi092:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-23T18:19:56.574 DEBUG:teuthology.orchestra.run.smithi152:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-23T18:19:56.602 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-23T18:19:56.617 DEBUG:teuthology.orchestra.run.smithi086:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-23T18:19:56.622 DEBUG:teuthology.orchestra.run.smithi092:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-23T18:19:56.625 DEBUG:teuthology.orchestra.run.smithi152:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-23T18:19:56.656 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern = core 2024-03-23T18:19:56.668 INFO:teuthology.orchestra.run.smithi152.stdout:kernel.core_pattern = core 2024-03-23T18:19:56.681 INFO:teuthology.orchestra.run.smithi086.stdout:kernel.core_pattern = core 2024-03-23T18:19:56.715 DEBUG:teuthology.orchestra.run.smithi086:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-23T18:19:56.755 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T18:19:56.755 DEBUG:teuthology.orchestra.run.smithi092:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-23T18:19:56.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T18:19:56.772 DEBUG:teuthology.orchestra.run.smithi152:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-23T18:19:56.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T18:19:56.788 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-23T18:19:56.804 INFO:teuthology.task.internal:Transferring archived files... 2024-03-23T18:19:56.806 DEBUG:teuthology.misc:Transferring archived files from smithi086:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-03-22_13:15:24-fs-wip-yuri11-testing-2024-03-21-0851-reef-distro-default-smithi/7617485/remote/smithi086 2024-03-23T18:19:56.806 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-23T18:19:56.887 DEBUG:teuthology.misc:Transferring archived files from smithi092:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-03-22_13:15:24-fs-wip-yuri11-testing-2024-03-21-0851-reef-distro-default-smithi/7617485/remote/smithi092 2024-03-23T18:19:56.888 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-23T18:19:56.929 DEBUG:teuthology.misc:Transferring archived files from smithi152:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-03-22_13:15:24-fs-wip-yuri11-testing-2024-03-21-0851-reef-distro-default-smithi/7617485/remote/smithi152 2024-03-23T18:19:56.929 DEBUG:teuthology.orchestra.run.smithi152:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-23T18:19:56.971 INFO:teuthology.task.internal:Removing archive directory... 2024-03-23T18:19:56.971 DEBUG:teuthology.orchestra.run.smithi086:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-23T18:19:56.976 DEBUG:teuthology.orchestra.run.smithi092:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-23T18:19:56.978 DEBUG:teuthology.orchestra.run.smithi152:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-23T18:19:57.021 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-23T18:19:57.036 INFO:teuthology.task.internal:Not uploading archives. 2024-03-23T18:19:57.036 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-23T18:19:57.049 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-23T18:19:57.049 DEBUG:teuthology.orchestra.run.smithi086:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-23T18:19:57.058 DEBUG:teuthology.orchestra.run.smithi092:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-23T18:19:57.060 DEBUG:teuthology.orchestra.run.smithi152:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-23T18:19:57.074 INFO:teuthology.orchestra.run.smithi092.stdout: 262157 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 23 18:19 /home/ubuntu/cephtest 2024-03-23T18:19:57.077 INFO:teuthology.orchestra.run.smithi152.stdout: 262157 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 23 18:19 /home/ubuntu/cephtest 2024-03-23T18:19:57.095 INFO:teuthology.orchestra.run.smithi086.stdout: 524301 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 23 18:19 /home/ubuntu/cephtest 2024-03-23T18:19:57.098 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-23T18:19:57.112 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-23T18:19:57.139 INFO:teuthology.nuke:Checking targets against current locks 2024-03-23T18:19:57.166 DEBUG:teuthology.nuke:shortname: smithi086 2024-03-23T18:19:57.166 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-23T18:19:57.193 DEBUG:teuthology.nuke:shortname: smithi092 2024-03-23T18:19:57.193 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-23T18:19:57.198 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi086.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-03-22_13:15:24-fs-wip-yuri11-testing-2024-03-21-0851-reef-distro-default-smithi/7617485', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-23 17:15:05.012809', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJRQyru1W+ghUxJWxdqkZUO5kVJdVfWONkkLLciLrJ9z+5QoL7EDbRzS8VPdBSbWKQArG55IemPWMBMHzILdm4w='} 2024-03-23T18:19:57.216 DEBUG:teuthology.nuke:shortname: smithi152 2024-03-23T18:19:57.217 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-23T18:19:57.225 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi092.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-03-22_13:15:24-fs-wip-yuri11-testing-2024-03-21-0851-reef-distro-default-smithi/7617485', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-23 17:15:05.011460', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIEuY40VvuSMzhcnfJdlnURI+2xwRebarDR06M8+49B80T++wnnyoBDtnKIPpV4d10Njp3bOEwZA099mEhzidMU='} 2024-03-23T18:19:57.241 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi152.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-03-22_13:15:24-fs-wip-yuri11-testing-2024-03-21-0851-reef-distro-default-smithi/7617485', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-23 17:15:05.014011', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHPklPFNoqQV2QJdy1E8+l4+OUuE8WnKB+jELm1CuTAauM1CiWsTsxcdIpYvOYaYJv8unQ5ChARjrAVR0O63nNs='} 2024-03-23T18:19:57.250 INFO:teuthology.orchestra.console.smithi086:Power off 2024-03-23T18:19:57.250 DEBUG:teuthology.orchestra.console.smithi086:pexpect command: ipmitool -H smithi086.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-23T18:19:57.274 DEBUG:teuthology.orchestra.console.smithi086:power off output: Chassis Power Control: Down/Off 2024-03-23T18:19:57.274 DEBUG:teuthology.orchestra.console.smithi086:pexpect command: ipmitool -H smithi086.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-23T18:19:57.294 INFO:teuthology.orchestra.console.smithi092:Power off 2024-03-23T18:19:57.294 DEBUG:teuthology.orchestra.console.smithi092:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-23T18:19:57.305 DEBUG:teuthology.orchestra.console.smithi086:check power output: Chassis Power is on 2024-03-23T18:19:57.317 DEBUG:teuthology.orchestra.console.smithi092:power off output: Chassis Power Control: Down/Off 2024-03-23T18:19:57.317 DEBUG:teuthology.orchestra.console.smithi092:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-23T18:19:57.332 INFO:teuthology.orchestra.console.smithi152:Power off 2024-03-23T18:19:57.332 DEBUG:teuthology.orchestra.console.smithi152:pexpect command: ipmitool -H smithi152.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-23T18:19:57.341 DEBUG:teuthology.orchestra.console.smithi092:check power output: Chassis Power is on 2024-03-23T18:19:57.354 DEBUG:teuthology.orchestra.console.smithi152:power off output: Chassis Power Control: Down/Off 2024-03-23T18:19:57.354 DEBUG:teuthology.orchestra.console.smithi152:pexpect command: ipmitool -H smithi152.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-23T18:19:57.377 DEBUG:teuthology.orchestra.console.smithi152:check power output: Chassis Power is on 2024-03-23T18:20:01.306 DEBUG:teuthology.orchestra.console.smithi086:pexpect command: ipmitool -H smithi086.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-23T18:20:01.342 DEBUG:teuthology.orchestra.console.smithi092:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-23T18:20:01.379 DEBUG:teuthology.orchestra.console.smithi152:pexpect command: ipmitool -H smithi152.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-23T18:20:01.417 DEBUG:teuthology.orchestra.console.smithi086:check power output: Chassis Power is on 2024-03-23T18:20:01.452 DEBUG:teuthology.orchestra.console.smithi092:check power output: Chassis Power is on 2024-03-23T18:20:01.498 DEBUG:teuthology.orchestra.console.smithi152:check power output: Chassis Power is on 2024-03-23T18:20:05.422 DEBUG:teuthology.orchestra.console.smithi086:pexpect command: ipmitool -H smithi086.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-23T18:20:05.498 DEBUG:teuthology.orchestra.console.smithi092:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-23T18:20:05.506 DEBUG:teuthology.orchestra.console.smithi152:pexpect command: ipmitool -H smithi152.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-23T18:20:05.533 DEBUG:teuthology.orchestra.console.smithi086:check power output: Chassis Power is off 2024-03-23T18:20:05.609 DEBUG:teuthology.orchestra.console.smithi092:check power output: Chassis Power is off 2024-03-23T18:20:05.617 DEBUG:teuthology.orchestra.console.smithi152:check power output: Chassis Power is on 2024-03-23T18:20:05.635 INFO:teuthology.orchestra.console.smithi086:Power off completed 2024-03-23T18:20:05.710 INFO:teuthology.orchestra.console.smithi092:Power off completed 2024-03-23T18:20:09.621 DEBUG:teuthology.orchestra.console.smithi152:pexpect command: ipmitool -H smithi152.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-23T18:20:09.730 DEBUG:teuthology.orchestra.console.smithi152:check power output: Chassis Power is off 2024-03-23T18:20:09.831 INFO:teuthology.orchestra.console.smithi152:Power off completed 2024-03-23T18:20:09.937 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/dbench}} duration: 3226.341774225235 failure_reason: 'Command failed (workunit test suites/dbench.sh) on smithi086 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=82316b04821a6c2c1d458a881920454bbadb8837 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh''' flavor: default owner: scheduled_yuriw@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=a3f90f89b38047109c856a74eb2e73d1 status: fail success: false 2024-03-23T18:20:09.938 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-23T18:20:10.027 INFO:teuthology.run:FAIL